How Adversarial Rem-Dreams May Facilitate Creativity, and Why We Become Aware of Them

1. A Computational Model for Creative Dreams

Even though dreaming is a universal phenomenon, characterizing its role is still, up to this day, a challenging task. For instance, it is difficult to assess whether the improvement of skills after a night of sleep is due to the occurrence of a certain dream, or to other physiological features of sleep such as hippocampal replay during NREM sleep [27]. In other words, the potential effects of sleeping and dreaming are entangled and thus confounded, since they are naturally co-occurring. Nonetheless, there have been some attempts to investigate the effect of dreaming, notably through the use of pharmacological interventions to suppress REM sleep in participants [see, e.g., [28]. Despite these efforts, disentangling the effects of dreams within sleep remains complex and largely unclear.

Here, computational models can help to quantify physiological features such as dreams and replay through simulations, and thereby decipher their contributions within a defined task. This was the challenge of our study that, through the construction of a “perturbed and adversarial dreaming” (PAD) model, suggests a role for dreams in learning to classify objects that come in a variety of versions in different visual scenes [25]. The PAD model is composed of a cortical feedforward (FF) pathway with a discriminator network (D), and a feedback (FB) pathway that generates sensory activity from a hippocampal memory. The system can either be in a wake state, where external stimuli are directly driving the FF pathway, or in a dream state, where sensory activity is generated by a hippocampal memory replay fed into the FB pathway.

During wakefulness (fig:elifea), the cortex is exposed to diverse natural images that are processed along the cortical hierarchy through the FF pathway, until incorporated in high-level neuronal representations, for example in the Inferior-Temporal (IT) cortex [29]. Snapshots of the activity in these representations are temporarily stored in the hippocampus as episodic memory, that, for instance during sleep, can be replayed into the cortical representation. During the forward processing of the sensory input, the discriminator network D learns to recognize that the sensory activity evoked by an image presentation is real (fig:elifea, red `D’).

During the REM sleep phase (fig:elifeb), the representation stored from the previous day is replayed from the hippocampus (“dog” memory) along with past, sometimes unrelated memories (“car” memory) and some additional cortical background activity (which is modeled as noise). The resulting activity in higher cortical areas is representing the dream content. It is processed along the cortical FB pathway down to low-level sensory areas where details are added to the dream. Due to the combination of diverse memories, the dream might contain various unrelated elements, such as a car having the texture or shape of a dog. Yet, the mere superposition of the abstract representations in the higher cortical area does not entail instructions of how the FB pathway can succeed in generating a realistic sensory activity out of the novel combination. This is where the principle of GANs, i.e., adversarial learning, comes into play. The dream is processed through the FF pathway and the discriminator D. During the REM sleep, the discriminator learns that the internally generated sensory activity is dreamed. The FB pathway, however, learns to improve the realism of its generated activity, and with that makes the task for the discriminator to correctly classify the activity as dream more difficult. This FB pathway represents the adversarial learning, leading to the adversarial dreams that try to evoke a misclassification of the discriminator.

The described PAD model of sleep is trained by repeating many wake-sleep cycles, whereby a set of natural images is repeatedly processed by the FF and FB pathways and the discriminator D. Once training is completed, we can examine the quality of the learned high-level representations (fig:elifec-d) within the learned latent space of the model (the highest cortical representation). Note that these representations are not learned in a supervised manner with an explicit teaching signal that would indicate the ground truth category of the observed object. Instead, the representations are formed through unsupervised learning, leading to a clustered higher-level neuronal representation of visual objects. The structure of the latent space can be illustrated by Principal Component Analysis [PCA, [30] applied to the activity in the last layer of the FF network. Projecting the activity vectors to the first two principal components, this procedure allows visualizing how the representations of various learned objects cluster in the latent space. The visualization is not part of the model, although PCA could itself be modeled by a neuronal network [31].

Comparing PCA projections after different training regimes allows for investigating the effect of wakefulness versus REM dreams: If only the wake phase is present, the obtained PCA projection shows that representations from different object categories are entangled, indicating that wakefulness is not sufficient to construct structured semantic representations (fig:elifec). When both Wake and REM sleep phases are simulated (fig:elifed), the PCA projection shows relatively distinct clusters of latent representations according to the semantic category (“class identity”) of the corresponding images. The model thus tends to organize latent representations such that high-level, semantic clusters are distinguishable, potentially helping humans and other animals to discern different object categories from their sensorium. This is in particular important for animals, that do not receive explicit teaching signals in the way humans and their children do throughout development.

These results can be quantified by evaluating the performance of a linear decoder (classifier) trained on high-level cortical representations obtained throughout the model simulation. If the REM phase is included in the training, the accuracy of the classifier tends to be much higher than if only the Wake phase is simulated (fig:elifee-f). The results show that the generation of dreams during REM sleep is essential to organize high-level representations according to the semantics of the sensorium, suggesting that dreaming is an essential component of learning.

2. Semantization Requires More than Memory Replay

Previous cognitive theories of sleep, such as “semantization” theories [32,33], suggest that the commonalities between multiple experienced episodes are extracted during NREM sleep to form a cortical semantic representation. A cognitive model [34] proposed that semantic formation is based on the invariant overlapping and statistical regularities between single replayed episodic memories. Areas of overlap are strengthened via Hebbian learning, allowing for an abstraction of shared elements among these memories, the so-called semantic “gist". For example, the reactivation of various memories of “cat experiences" facilitates the extraction and consolidation of the concept “cat" from repeating features with episodic memories (four legs, pointed ears, tail, etc.) in cortical representations.

The PAD model offers a further perspective on the semantization process during sleep. It suggests that the semantization of cortical representations is more likely induced by REM dreams, featuring combinations of episodic memories, than by the replay of single episodic memories during NREM sleep. We incorporated NREM sleep into our model by a phase where individual hippocampal memories are replayed without combination with other memories. Instead, the activity in the sensory area after processing through the FB pathway is perturbed by randomized patches (see fig:fullmodel and section on NREM sleep below). We find that NREM sleep in our model has little or no impact on the acquisition of semantic representations—even when adversarial learning based on individual memory replays is enabled in the FF/D and FB pathway (see Deperrois et al. [25] for details). The simulations indicate that, to extract semantic concepts from sensory data, the brain must go beyond merely replaying previous experiences as classically accounted for in NREM sleep [35]. It appears that novel but realistic contents must be internally created from simultaneous multiple memories that explore the boundaries between object categories, and help to form a representation of these objects based on contents and content differences. The suggested role of creative REM dreams may help to refine cognitive theories about sleep function and to delineate the role of NREM and REM sleep in memory semantization.

The described creative memory replay can be seen in the light of the overfitted brain hypothesis, which proposes that “nightly dreams evolved to combat the brain’s overfitting during its daily learning” [24]. More specifically, the overfitted brain hypothesis posits that the creation of corrupted sensory activity from stochastic memory replay during sleep increases the generalizability of the object representations learned during the day. The stochastic memory replay aligns with the NREM phase in our model. In our NREM phase, hippocampal memories are fed back into the cortex and randomly perturbed, in order to train the cortical representations in the FF pathway to become noise resistant. Yet, as we showed, the random perturbations in single memories alone are not as efficient in separating objects in the cortical representation as when memories are paired in a creative replay during REM [25], fig:elifed]. The PAD model therefore suggests that sleep increases generalizability with both, stochastic replay during NREM and – going beyond the overfitted brain hypothesis [24] – also with creative replay during REM sleep.

3. Adversarial dreams on the edge between fantasy and reality

Aside from proposing a role for dream function, the adversarial dreaming principle of the PAD model suggests a mechanism for how internal sensory activity can be generated in the brain, and how an equilibrium between fantastic and realistic aspects can be maintained, potentially beyond the stage of dreaming. Following the PAD framework, we assume that different memories, for example a memory of an eagle and a human (the dreamer), are concurrently replayed from the hippocampus and combined in high-level cortical areas (fig:cliff). The combined activity is sent through the FB pathway, leading to the generation of a dream representing a flying human, which is, while being novel and creative, not very convincing in terms of realism. This lack of realism allows the discriminator to easily detect that the sensory activity is dreamed and not real.

The adversarial learning of the FB pathway will restore the realism in this scene. The key point of adversarial dreaming is that the FB pathway attempts to fool the discriminator into believing that the dream actually is a sensory experience. There are two ways to implement this fooling: either by reverting the synaptic plasticity in the FB pathway and keep the correct target for the discriminator (as described and simulated in Deperrois et al. [25]), or keep the plasticity in the FB pathway untouched, while reverting the target for D. It is this second version that we suggest here as a biological implementation. The reason is that the target imposed on the discriminator may provide an awareness signal for the content, as we elaborate later (see fig:awereness). The REM phase still needs a sign-switch in the synaptic plasticity, and this is now on the synapses of the discriminator network during REM sleep (fig:cliff). In fact, while D learns during wakefulness to correctly classify externally generated sensory activity as real, it should learn to correctly classify internally generated sensory activity as dreamed during the REM phase. A sign-switch of synaptic plasticity (necessary in the D network when the target remains real during the REM phase) was experimentally observed by the action of acetylcholine [38] that is also involved in the regulation of REM sleep [21].

In detail, the “adversarial game” during the REM sleep starts by imposing the wrong target real! on the discriminator output. This typically leads to an error between the target and what the input from D would like to produce. To still improve the discriminator, plasticity in D is inverted, as just explained. But plasticity in the FB pathway remains the same, so that the FB network now tries to produce a sensory activity that becomes more real, as requested by the target real! for D. The FB pathway is also told how to improve because the error from D, that carries the instruction to improve the realism in the sensory activity, is propagating down through D and the FF pathway to the FB network (for a biological form error backpropagation see Sacramento et al. [39]). In other words, the FB pathway adapts such that the internally generated activity in the sensory area will more likely fool the discriminator into believing that the activity is real and produced from the outside. As a consequence, after thousands of wake-sleep cycles, the dream is thus better blended into the reality of the outside world. The same computational principles may also apply to improve creative imagination during wakefulness (see Discussion).

To come back to our example, we consider the experience of dreaming about flying through the clouds, resulting from combining the hippocampal memory of a bird and a human (fig:cliff). The dream could be easily unmasked as “fake” by the discriminator D. Yet, sometimes humans engage in sports such as cliff jumping, whereby it appears as if the jumper is flying. Accordingly, the realism of the dream could be increased by adding visual details such as some cliffs and water. On the other hand, the omission of these details signifies errors that must be corrected to enhance the dream’s plausibility. In this case, the synaptic connections of the FB pathway are modified to generate a more realistic, plausible dream. Consequently, in a subsequent REM phase, we might find ourselves leaping off a cliff to fly over the sea. The novel dreams created during the REM phase in our model will also change the early cortical activity produced during wakefulness when `mind-wandering’ through the latent representation. This in turn may influence our future actions. For instance, we may go cliff jumping the next day after the REM dream has generated the corresponding scene.

Adversarial learning might thus explain how dreams, originating in a creative combination of memories, can be constrained to become more realistic and make them compatible with our waking experiences and actions. This is in line with Hobson’s pioneer activation-synthesis theory that claims that REM dreams result from the brain “making the best of a bad job in producing even partially coherent dream imagery from the relatively noisy signals sent up to it from the brain stem” [40], p. 1347]. In the spirit of this citation, the modeled generative process during REM sleep starts with a noise signal added to random combinations of memories and tries to produce a coherent and realistic cortical activity through adversarial learning. We will next see how the resulting balance between fantastic and creative dream elements can be useful to trigger creative insights.

4. Adversarial Dreaming at the Heart of Creativity?

Adversarial dreaming may also influence creativity during wakefulness. As a consequence of adversarial dreaming, the FB projection can lead to the generation of sensory activity patterns that have unlikely been evoked by previously experienced stimuli (novel combination of episodic memories), but that nevertheless may be possible in reality (increased realism through the adversarial game). This kind of constrained simulation may have functional benefits in terms of creative thinking and gaining insights in general.

Novel cognitive insights were argued to result from a period of “incubation”, where non-obvious, remote associations from memory (or knowledge) elements are brought into our thinking [41,42]. These associations can sometimes be compatible with reality, in which case they can provide a solution to a complex problem through a creative insight [43], “Aha moments”] that deliberate reasoning alone may not provide. Dreaming appears to be an ideal stage to promote novel associations and eventually enhance creative insights, as previously suggested [9,44,45,46]. More recent studies show that subjects with narcolepsy, characterized by falling asleep directly into REM sleep and having a higher percentage of REM sleep [47], show higher creative potential [36].

Creative problem-solving is also promoted by the twilight stage before sleep, as allegedly exploited by the great inventor Thomas Edison [37]. While napping down, he held two balls that would drop and awaken him as he entered deep sleep, enabling him to capture thoughts about his inventions that would otherwise be lost. Experimental examination confirmed this anecdotal evidence and revealed that creative problems are easier solved on the verge of falling asleep and being awakened from the first stage of NREM sleep (N1), as opposed to being awakened from the second stage of NREM sleep (N2) or staying awake throughout [37]. Creative problem-solving is further fostered by a 60-minute nap as opposed to a 60-minute rest [48]. In a similar line, Ritter et al. [49] found that creative performance is already boosted by a simple eye-closure as opposed to keeping eyes opened. To relate these findings to our model of creative dreaming and imagination, we postulate that during the described creative stages our adversarial processes are triggered. The specific form of memory replay and synaptic plasticity arising in the adversarial learning may be induced through differential activation of neuromodulators, as it is also observed during the transition across sleep stages [50], and in shaping the memory replay in general [51].

Beyond Edison’s nap technique, anecdotal evidence exists from August Kekulé who discovered the Benzene structure through a dream in which he combined two non-related concepts, a snake biting its tail, and the carbohydrate molecular chain [1]. Here, the adversarial dreaming framework could explain the occurrence of such insights (fig:benzenestructure). The memories of a depiction of the ouroboros symbol (a snake eating its tail) and the hexane molecule could be randomly replayed from the hippocampus and combined in high-level areas during REM sleep. Propagating this activity down through the FB pathway, the adversarial learning mechanism could allow the generation of a dreamed molecule that contains aspects of the ouroboros, such as its cyclic shape. By forcing this dream to be realistic against the discriminator judgement, the dreamed molecule could be harmonized with known properties of chemistry, so that it might exist in reality. Naturally, not all creative combinations experienced during dreaming are useful, and their usefulness is ultimately determined by how compatible they are with the actual external world.

This suggests that additional steps may be involved, such as experimentally verifying the occurred insight—reflected in Kekulé ’s words: “Let us learn to dream, gentlemen, and then perhaps we shall learn the truth... but let us beware of publishing our dreams before they have been put to the proof by the waking understanding.” (account of his famous dream of the benzene structure, as quoted in Olah [52], p. 54)

5. NREM: Improving Memory Robustness by Perturbed Replay

The suggested adversarial REM dreams represent part of the full PAD model that also includes NREM sleep. For the past decades, NREM sleep has been associated with memory consolidation [35,53,54] and memory semantization [32,34]. The main mechanism hypothesized to drive these consolidation processes is the reactivation of hippocampal representations observed during slow-wave sleep, the deepest stage of NREM sleep[55]. Accordingly, replaying hippocampal memories allows a transfer to cortical networks for long-term retention via Hebbian learning [54], possibly through an abstraction of semantic concepts by discarding spatiotemporal details and keeping the commonalities between replayed memories [32,33,34].

The PAD model also incorporates hippocampal replay during NREM sleep and suggests a complementary role of REM sleep in memory consolidation and semantization. While memory replay has been extensively associated with memory consolidation, as such, it is not obvious how the reactivation of previous memories alone could improve cortical representations. Here we argue that non-creative dreams might be beneficial if some perturbations are applied, making the recognition of dreamed objects more challenging. During NREM phases in the PAD model, a single episodic memory is retrieved from the hippocampus (instead of multiple memories in the case of REM) (fig:fullmodel, middle), resulting in dreaming of a sensory input previously experienced. Additionally, this dream is perturbed by some randomized occlusions in the sensory area, and the FF pathway is trained to map the perturbed dream to the initially replayed latent activity in the higher cortical area. Such a learning paradigm is reminiscent of the sleep phase in the Wake-Sleep algorithm [56]. In this algorithm, the FF recognition pathway is trained during the Sleep phase so that it inverts the FB generative pathway. The generative FB pathway is itself trained during the Wake phase so that it learns to reproduce the sensory activity from the internal representation generated by the FF pathway.

By replaying and perturbing previously experienced inputs, the PAD model shows that learned cortical representations are more robust to perturbations that might occur in the visual field when an object is partially hidden by obstacles (see Figure 5c&d in Deperrois et al. [25]). While REM dreams tend to semanticize cortical representations through their creative process, NREM dreams make them more robust to environmental noise. Together, NREM and REM dreams act complementary to construct semantic, robust representations.

In the PAD model the REM phase is the main driver of semantization. Future modeling work could identify other elements so that the NREM phase, while still replaying a single memory at a time, could contribute more to memory semantization, not only to making memories robust against noise. As recently suggested [57], NREM sleep could, for instance, host a contrastive learning process to push object representation farther apart and hence improve semantization. Contrastive learning [58,59,60] is in fact a training procedure used in recent artificial intelligence models as a way to efficiently learn semantic representations. The main idea is to construct a latent space such that similar sensory inputs (positive examples) are represented close together, while dissimilar sensory inputs (negative examples) are pushed apart within the latent space. During the NREM phase, positive examples can be obtained by systematically perturbing the activity generated in the sensory area out of the single memory replay, for instance by mimicking the view point of the object, and with this creating two different instances from the same hippocampal memory. These two instances of the low-level object representation are then learned to be mapped close to each other in the high-level representation. Negative examples in a NREM memory replay can be obtained by pushing the projection of the FF pathway to the higher cortical area away from previous hippocampal memories replays in this higher area [57].

6. Adversarial Processes Explaining Dream Awareness?

Our model for unsupervised learning of object representations out of naturalistic sensory inputs – what we call semantization – implicitly assumes various meta-structures that gate the learning process. A first meta-structure pops up when considering the classical Wake-Sleep algorithm [56] described above as a model for NREM sleep (that we extended by random perturbations in the replay). For instance, when replaying a memory from the hippocampus into the FB pathway at the level of the higher cortical area, this activity serves as a target for the activity that is re-created through the FB→FF loop back to the same higher cortical area (red `?’ in fig:fullmodel). One way to biologically implement the hippocampal replay as a target or “teaching signal” for the FF neurons is to “nudge” these neurons in the soma, while the synapses projecting from the FF pathway to their dendrites learn to reproduce the somatic nudging [61]. When the somatic nudging is strong enough, the match between the dendritic input and somatic activity may elicit a dendritic calcium spike, and with this also the perception [62]. It was argued that during sleep, when such dendritic calcium spikes arise, the dreamer is becoming aware of the content of the dream [63], although not necessarily of the fact that this is a dream (fig:awerenessb). Becoming aware of the dreaming content is what dreams differentiate from the more general state of sleep without dreaming.

During the Wake, the FB neurons in the sensory area are nudged by the FF sensory neurons (that are themselves driven by the sensory input). A possible match between the teacher signal an the top-down signal through the FB pathway may elicit a dendritic calcium spike in these FB neurons, and again signal the awareness of the sensory content (Takahashi et al. [62], fig:awerenessa). Crucially, and different to NREM sleep, the output of the discriminator network D receives an additional nudging input from the conductor C, signaling the target real. This conductor is itself a neuronal population representing the meta-information whether the sensory activity should be perceived as externally or internally generated. We postulate that the teaching signal in the FB neurons from the low-level FF neurons makes us aware of seeing an object (content-awareness), while the teaching signal in the D neurons from the conductor makes us aware that the object is real (state-awareness, upper red in part in fig:awerenessa). The same conductor was suggested to be also involved in forming consciousness Benitez et al. [64].

During REM sleep, the adversarial learning also requires activity in the discriminator network. As we suggest here, the conductor adversarially sets the target real for the output of the discriminator D (while it reverts the plasticity in D so that this still can learn that the sensory activity is dreamed). With the real-target being backpropagated, the FB pathway is told how to improve the realism of the sensory activity it generates. As in Wake, the conductor signal in the discriminator neurons tells the subject that the sensory activity is (incorrectly) generated from outside and is considered as real (state-awareness). In addition, subjects may become aware of the dream content because the FF neurons in the higher cortical areas are nudged by the FB neurons, initiated by the memory replay (content-awareness, fig:awerenessc). It is tempting to postulate that the double nudging of the FF neurons and the specific real-discriminator neurons during REM sleep is the reason that subjects perceive the sensory activity as real during sleep, and remember more often REM dreams, as compared to NREM activity where the D neurons do not receive a teaching signal. According to the PAD model we become aware of a REM dream because it is adversarial (fig:awerenessc).

Lucid dreaming in this model is explained by nudging the higher cortical FF neurons with the target activity set by the FB pathway (as in NREM and REM), but the discriminator neurons are now nudged to encode dreamed, corresponding to the true (non-adversarial) state of the sensory activity (fig:awerenessd). The teaching signal dreamed for the discriminator neurons may give the subject the awareness that its current state is the dream, as in fact happening in lucid dreaming [65]. To keep a functional benefit of lucid dreaming, the plasticity in the FB pathway has to be turned of, as in NREM sleep (fig:fullmodel), so that the FB pathway is not unlearned by reverting the discriminator target. Correspondingly, plasticity in the discriminator network should not be reverted (and instead can be as in the Wake state).

If during lucid dreaming the described plasticity modulations are not correctly implemented, while the target in D is still switched from real to dream, negative consequences of the dream experience are expected. With a discriminator target being dream and plasticity in the FB pathway remaining turned on, random combinations of memory replays would be pushed further away from any realism, easy detectable by D as dream. In fact, failed induction of lucid dreaming, potentially explained by an unilateral switch of the discriminator target alone without stopping FB plasticity, may lead to harrowing dysphoric dreams [66]. However, if the switch in the D-label is correctly synchronised with the required plasticity changes, the awareness of the dream and the dream content during lucid dreaming may help to consolidate specifically selected memories, as also exploited in therapy therapeutic applications [66].

Overall, the meta-structure of a discriminator and a conductor coming with adversarial dreaming opens a door to differentiate between (i) sensory activity that we become aware of, or not, during wakefulness, ($ii$) the memory replay during sleep without dreaming, likely happening in NREM sleep, ($iii$) the awareness of the dream content while sleeping, but not being aware of the dream state itself, as in REM sleep, and ($iv$) the awareness of the dream content and the dream state, as in lucid dreaming. These awareness modulations, and in particular the state-awareness of the dream, is potentially modulated by the same ratio of acetylcholine over noradrenaline and serotonin that is also shown to tune NREM sleep and other metacognitive processes [21,67].