Studying the Reduction of Entropy in the Brain During Sleep Through Spiking Neural Networks

1. Introduction

Sleep is a critical yet confusing biological phenomenon, essential for maintaining cognitive and physiological well-being. One proposed function of sleep is the reduction of neural entropy, aligning with the brain’s need to stabilize neural processes disrupted during wakefulness. This study explores this hypothesis by employing spiking neural networks (SNNs) to model entropic measurement, drawing parallels between biological mechanisms like synaptic homeostasis and artificial processes such as spike-timing-dependent plasticity (STDP). By incorporating these mechanisms, SNNs can simulate key functions of sleep, including memory consolidation and neural stability.

The research introduces a prototype framework using statistical mechanics to quantify entropy within the network, employing SNN architectures optimized for temporal dynamics. Additionally, mechanisms such as noise injection replicate stochastic dreaming processes, and synaptic downscaling mimics the brain’s method of preserving crucial synaptic connections while reducing overall complexity. Through these models, this work aims to provide computational insights into the entropic role of sleep, offering a foundation for future advancements in understanding neural processes and improving artificial intelligence systems.

This paper contributes to the interdisciplinary dialogue between neuroscience and machine learning, demonstrating how artificial networks can be used to simulate and study sleep.

2. Design

2.1. Why Neural Networks?

Neural networks were originally designed on the basis of certain biological principles, including those inspired by the human brain itself. As such, they are an adequate way to simulate certain neural functions in the human body. Structural similarities between neural networks and the brain (evolution of output depending on the corresponding layer, where each layer corresponds to a certain cortex [1]; learning in both systems involves adjustment of the strength of neural connections- adjustment of synaptic strength via synaptic plasticity in the brain, and up-scaling / downscaling of synaptics in NNs [2].

There is also an inherent similarity between how information is processed in the brain and in neural networks. Like the brain, neural networks also process information in parallel, where each neuron in a layer of a neural network processes information simultaneously with other neurons in the same layer. Previous studies have also shown that internal representations of neural networks are similar to those of the brain, specifically in visual tasks [1,3].

2.2. Why Spiking Neural Networks?

We prefer Spiking Neural Networks due to their ability to mimic the temporal dynamics of biological neurons. Unlike other ANNs, SNNs communicate through discrete, timed ’spikes’ or ’events’, similar to how the brain transmits information. This allows them to represent both timing and intensity of neural activity, which is essential to simulate essential cognitive functions like decision making and memory formation, the latter of which is imperative for our study.

SNNs utilize Spike-Timing-Dependent Plasticity (STDP), a type of temporal synaptic plasticity which has the crucial role of synaptic weight adjustment based on the timing of pre and postsynaptic neuron spikes. This stems from the brain engaging in neural hippocampal replay- a process where prior neural activity patterns are “replayed” in a controlled manner. This is deemed necessary for functions such as systems memory consolidation, recall and spatial working memory, navigational planning, and reinforcement learning [4]. This replay mechanism is essential to synaptic weight optimisation via STDP, this selective synaptic tuning reinforces meaningful connections and prunes the redundant ones, leaving less neural trees to be read through, reducing neural entropy.

SNNs are also able to “handle” noisy and incomplete data much better than other types of ANNs due to properties such as pattern completion, perceptual rivalry, distortion resistance, and prototype extraction [5].

3. Defining the Function of Sleep in Context of Entropic Reduction

Wakefulness is defined as the period of human activity directly involved with the increase of overall entropy in the brain from a statistical mechanics, information entropy, and a biological standpoint. Continuous sensory input, cognitive processing, and metabolic activity, all of which contribute to both information and thermodynamic entropy. Entropy is defined, roughly, as a measure of randomness or disorder in a system; an increase in such entropy directly correlates with the number of accessible microstates. As such, an increase in entropy brought about by wakefulness can vary the number of accessible CNS microstates, or disrupt their dynamics, such as their frequency, duration, or transitions. This increased disorder contributes to a non-ideal form of neural functioning; therefore, a process which brings about reduction in this very entropy is needed.

According to [6], one of sleep’s primary functions is to disrupt the continuous increase of entropy, which is brought about by wakefulness. This is done through processes such as homeostatic synaptic downscaling which reduces the overall strength of synaptic connections and the firing rate of neurons [7], which effectively reduces "chaotic order," and hence the entropy as well. Sleep also facilitates in memory consolidation [8] by strengthening connections relevant to the individual’s experiences while weakening or eliminating those which are not. This selective consolidation reduces the complexity of the neural network and in turn contributes to reduction of the entropy.

How We Plan on Implementing Sleep in the SNN and Biological Equivalence of the Implementations:

3.1. Spike-Timing Dependent Plasticity

STDP finds its way into our paper through feeding of the 60,000 data points of the training set into subdivisions of 10, fed on repeat to the network. Spike-timing-dependent-plasticity is relatively analogous to the biological processes observed in the hippocampus and neocortex, where synaptic plasticity underlies learning and memory formation (Antoinietti et al., 2019) .

STDP in the brain is closely related to certain synaptic mechanisms that regulate learning through activity dependent plasticity. For example, long-term-potentiation in the hippocampal CA1 region happens as a result of the activation of N-methyl-D-aspartate (NMDA) receptors, which are sensitive to both spike timing and synaptic activity. Long-term-depression mechanisms in the brain contribute to motor learning through synaptic adjustment that are based on the timing of spikes between granule cells and Purkinje cells [9].

STDP is also part of synaptic modifications in the neocortex, where it supports sensory processing and perceptual learning. Particularly through visual and auditory cortices, where the neurons undergo real-time synaptic adjustment based on the timing of sensory inputs. [10]. STDP is heavily involved in synaptic modifications in the neocortex, where it supports sensory processing and perceptual learning. Neurons in the visual and auditory cortices undergo synaptic adjustments based on the precise timing of sensory inputs.

Relative to this paper, STDP is used as a method of memory consolidation via reactivation of neural pathways after learning. This is analogous to replay mechanisms of the brain during offline periods (sleep or rest), which are essential for solidifying memories. This mimics the brain’s replay mechanisms, specifically hippocampal replay, where neural pathways are reactivated to “replay” certain events to reinforce learning and improve memory retention, during offline periods, such as sleep or rest, which are crucial for solidifying memories [11].

This can also be seen as a method of mitigation of catastrophic forgetting in neural networks, where NNs tend to abruptly lose track and forget all previously learned information upon learning something new.

Memory replay mechanisms like these enable the network to learn from interleaved and unconnected experiences over shorter timespans, which help mitigate forgetting [12]. SNNs may also use memory replay to predict future events by recognising temporal patterns in information recalling learned sequences in an order which enhances their ability to make context-specific predictions. Similar to this, the brain replays sequences of past events to inform itself to anticipate future actions or decisions. In the brain, replay phenomena have been linked to the optimisation of reward-based learning and decision-making, with evidence showing that hippocampal reactivation during sharp-wave ripples can influence downstream circuits involved in reward processing, such as the ventral striatum and prefrontal cortex [13].

In the brain, replay phenomena such as those being implemented via STDp have been linked to optimisation of decision making and reward/result based learning. Evidence suggests that hippocampal reactivation during SWR influences downstream circuits that are involved in reward processing, such as the prefrontal cortex [13]. These interactions integrate episodic memory with goal-oriented behaviours, which thus enhance adaptive behaviour through evaluation of past experiences in light of future goals. STDP in the brain is often modulated by neurotransmitters and/or hormones like dopamine which selectively enhance replay of events linked with rewarding outcomes. This means that replay mechanisms in biological and artificial systems can be optimised to prioritise more relevant experiences, which would enhance learning efficiency [14].

3.2. Synaptic Homeostasis

Sleep serves the role of resetting synaptic weights, which prevents the saturation of synaptic strength that occurs during wakefulness.. This is because wakefulness, characterised by constant sensory input and informational processing, strengthens synaptic connections across the brain. This continual reinforcement increases metabolic burden, and directly contributes to increased neural entropy.

During slow-wave sleep, these strengthened synaptic connections are globally scaled down without a loss of information, bringing the brain’s energy and entropy levels down. This is known as synaptic downscaling, a process which ensures the brain’s neuroplasticity is retained by avoiding synaptic saturation, which would otherwise impede learning and information storage. [7,15]. Downscaling not only restores synaptic balance but also brings down the complexity and disorder of neural circuits, thereby reducing neural entropy and making sure the brain is ready for the next learning phase [8].

3.3. Homeostatic Plasticity Mechanisms

Biological neural systems balance synaptic strengths through homeostatic plasticity to prevent runaway activity or dormancy, a balance that’s necessary to maintain for learning and memory [16]. Similarly, SNNs incorporate homeostatic plasticity rules for regulation of firing rates to maintain stability in networks. These mechanisms may enable the simulation of sleep-like states, where spiking activity stabilizes, enhancing the network’s memory retention and learning efficiency by mimicking biological processes such as spike-timing-dependent plasticity paired with homeostatic scaling [17,18]. Research has shown that integrating homeostatic plasticity with reinforcement learning frameworks in SNNs supports memory and energy efficiency during rest phases. This is analogous with the function of non-rapid eye movement (NREM) sleep in biological systems, which reduces overall synaptic weight while preserving critical memories. Models like Plasticity-Driven Learning Frameworks (PDLF) showcase how networks encode input stimuli into synaptic weights, reflecting biological equivalence to neuronal "resting" states during sleep cycles, critical for memory consolidation [17]. Hybrid SNN architectures like these also replicate sleep’s role in maintaining network robustness and adaptability, similar to observed brain dynamics [19].

3.4. Noise Injection

Noise injection in spiking neural networks (SNNs) can simulate "dreaming" by introducing stochastic perturbations that mimic the random neuronal firing observed during REM sleep. This noise facilitates the exploration of stored memory patterns and the generation of novel associations, resembling the consolidation and reorganization processes of biological dreaming. By triggering spontaneous activity, noise helps reinforce memory traces and promotes neural plasticity, somewhat similar to the random activation of hippocampal and cortical circuits during sleep. These approaches also align with findings that stochasticity in the brain supports robust learning and memory retention.

4. Calculation of Entropy

Implementing STDP here, we feed the same "type" of data to a similar set of neural networks, over and over again.

Assume a group of layered networks with fixed architectures and variable weights described by its configuration space W. All values within the configuration space correspond to a list of values of all weights needed to select a network design within the chosen architecture. [20]

Using density ${\rho }_{o}W$ We limit the effective volume of the configuration space to

$$Z_o=\int {\rho }_o\left(\mathbf{W}\right)d\mathbf{W}$$

(1)

Assuming the network utilises the input output function $y=f_W\left(x\right)$, regions corresponding to the implementation of this function can be identified by a masking function such that all final implementations (refer to GitHub) that have similar architectures count as 1.

$${\mathsf{\Theta }}_f\left(\mathbf{W}\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{f}_{\mathbf{W}}=f\hfill \\ 0\hfill & \mathrm{if}{f}_{\mathbf{W}}\ne f\hfill \end{array}\right.$$

Where $f_{\mathbf{W}}$ = 1 indicates that there exists an implementation of a certain, known mapping.

As such $Z\left(f\right)$ should be a function that numerically increases as more implementations with the same architecture prop up, essentially corresponding to the number of independent implementations within the configuration space and occupy a volume:

$$Z\left(f\right)=\int {\theta }_f\left(\mathbf{W}\right){\rho }_0\left(\mathbf{W}\right)d\mathbf{W}$$

(2)

The probability of on such a space of functions is thus defined as

$$P_0\left(f\right)={\displaystyle \frac{Z\left(f\right)}{Z_0}}$$

(3)

Stemming from the original hypothesis, testing on larger datasets should cause a steady decrease in the value of $Z\left(f\right)$, causing a fall in the value of $P_0\left(f\right)$, such that $P_0\left(f\right)$ is the probability that randomly chosen network in the configuration space will correspond to a particular implementation $f_n$. The class of functions implementable by a given architecture is given as:

$$\mathbf{F}=\left\{f|P_0\left(f\right)\ne 0\right\}$$

(4)

Class F therefore contains only configurations that have been realised.

The entropy of the distribution is given as

$$S_0=-\sum _f{P}_0{\left(f\right)}^{2}ln{P}_0\left(f\right)$$

(5)

5. Methodology

For a relatively accurate mirroring of the process of sleep through neural networks, we have used a Spiking Neural Network programmed in Python that utilises the Neuromorphic MNIST dataset.

5.1. Data Preprocessing

Temporal data is preprocessed via denoising and conversion into frame representations using a fixed time window of 1000 $\mu s$. The training dataset is then split into ten disjoint subsets each representing 10% of the data ( 6,000 samples for each set) to train multiple neural networks.

5.2. Network Architecture

Each of the 10 SNNs comprises of:

• Convolutional and pooling layers for extracting spatial features from input frames.
• Spiking neuron layers that implement leaky integrate-and-fire (LIF) dynamics parameterized by a decay constant $\left(\beta \right)$ of 0.5 and a surrogate gradient function for backpropagation.
• A final spiking output layer.

5.3. Training Protocol

Each network is trained on individual subsets with an event-based mean square error loss, using a correct-to-incorrect spike rate ratio of $0.8:0.2$.

5.4. Testing Protocol

Each networks is tested on an external testing dataset with shuffled label mappings. This external dataset is developed using a custom program that randomizes the label-to-class correspondence of the original NMNIST testing set of 10,000 samples.

5.5. Grouping of Functions

After testing, the networks’ mappings are compared pairwise and grouped if at least the specified number (threshold) of their label-to-class associations agree. For this paper, the threshold stands at 5 meaning that two networks are grouped together if 5 or more shuffled labels map to the same original class in both networks.

5.6. Sample Output and Entropy Calculation

A sample iteration of the program with the threshold set to 5, and 10 total neural networks in training returns the following output:

• Networks with similar mapping f1= 1
• Networks with similar mapping f2= 3
• Networks with similar mapping f3= 5
• Networks with similar mapping f4= 1

The $P_0$ for f3 for example, comes out to be ${\displaystyle \frac{5}{10}}$. Calculating the $P_0$ for each "f" and plugging it into equation 5 gives a sum total $S_0$ (for this iteration) of $0.32769$.

6. Future Work

We will be coming out with more prototypes in the coming future which implement more functions of sleep as discussed earlier. Testing and training will be done on a larger number of neural networks as well.