Automated Sleep Spindle Analysis in Epilepsy EEG Using Deep Learning

1. Introduction

Sleep spindles are transient oscillatory events in the electroencephalogram (EEG), typically observed during non-rapid eye movement (NREM) stage 2 sleep (N2), where they occur together with K-complexes as distinctive hallmarks of neuronal activity. They appear as sinusoidal cycles in the 9–16 Hz range, lasting on average 0.5–3 seconds, with their spindle-shaped envelope inspiring their name [1,2,3]. Generated primarily through thalamocortical interactions, spindles are considered cortical correlates of thalamic function and are thought to reflect the strength and plasticity of thalamocortical networks [2]. They play an essential role in sleep continuity, memory consolidation, and synaptic plasticity, while suppressing external sensory input and facilitating neural reorganization [4,5,6]. Spindle dynamics also develop characteristic profiles across the lifespan, paralleling cortical maturation [2]. Numerous studies have demonstrated strong associations between spindle activity and cognitive performance, including memory, information processing speed, and IQ [7,8], with fast spindles (12–16 Hz) linked to efficient memory consolidation and information processing [5,9]. Abnormal spindle activity has been reported in neurological and psychiatric disorders, with epilepsy being a notable case where altered spindle generation may reflect pathological thalamocortical reorganization [8,10,11].

Altered spindle mechanisms have been observed in epilepsy, where activity and quantitative parameters can be disrupted [12]. Although mechanisms are not fully understood, several consistent findings have emerged. Spindle density may decrease in certain epilepsies, often coinciding with spike–wave discharges (SWD) [13], which are characteristic 3–4 Hz bilateral events in idiopathic generalized epilepsies [14]. In childhood epilepsy with centrotemporal spikes, reduced spindle activity anticorrelates with spike frequency [15]. Spindle frequency can also be modulated by antiepileptic drugs [16], while increased spindle duration has been observed immediately before seizures in nocturnal frontal lobe epilepsy [11]. Together, these findings suggest that spindle activity changes may serve as biomarkers of epileptic foci. Beyond simple measures such as frequency or count, comprehensive characterization, including duration, amplitude, spatial distribution, and localization, may provide deeper insights into neural dynamics in both health and disease [17].

Despite this importance, spindle research faces persistent challenges. As noted in [18], limited dataset sizes, inconsistent reporting, and poor agreement on detection methodologies complicate studies. EEG analysis is inherently time-intensive, often resulting in small datasets, and highlighting the need for automated spindle detection to increase diagnostic accessibility while reducing expert burden.

Deep neural networks (DNNs) now represent the state of the art in spindle detection, surpassing wavelet-based methods [20,32,33,34,35] and classical machine learning [36,37,38,39]. Traditional approaches rely on hand-crafted features [19,22,23], which are labor-intensive and prone to bias. By contrast, DNNs learn hierarchical features directly from raw data, minimizing manual design and improving scalability [24,25,26]. Nevertheless, current models face limitations, including high false positive rates, poor generalizability across datasets due to protocol differences, and computational demands that hinder clinical adoption. These challenges underscore the need for robust, efficient, and clinically viable deep learning models.

The aim of this work is to analyze sleep spindles in EEG recordings from healthy subjects and patients with epilepsy using deep learning. We evaluate different network architectures for automated spindle detection under two strategies: (i) a segmentation-based pipeline, where spindles are first detected and then characterized, and (ii) a segmentation-free pipeline, where spindle characteristics are predicted directly from raw EEG. Performance is assessed using precision, recall, and F1-score to determine the most suitable framework for clinical use. Using the optimal approach, we then compare spindle characteristics between epilepsy patients and controls to investigate alterations in spindle dynamics.

The remainder of this paper is organized as follows. Section 2 described the materials and datasets. Section 3 details the methodology. Section 4 presents experimental results, including architecture comparison, evaluation of segmentation-based versus segmentation-free approaches, and spindle alterations in epilepsy. Section 5 discusses the implications of our findings, and Section 6 summarizes conclusions and outlines directions for future work.

2. Materials and Data Recording Details

In this study, we perform a retrospective analysis of video-EEG recordings obtained from the archive of the Video-EEG Laboratory Genomed (Moscow, Russia). Recordings were acquired during both wakefulness and sleep, with functional activation tests included when clinically indicated. The monitoring duration for each subject ranged from 3 to 10 hours. EEG was recorded using the Neuroscope system (Russia), with electrodes positioned according to the international 10–20 system.

2.1. Ethics Statement

All study procedures were reviewed and approved by the Bioethics Committee of the Institute of Biology and Biomedicine, National Research Lobachevsky State University of Nizhny Novgorod. The study was conducted in accordance with the Declaration of Helsinki and relevant institutional guidelines.

2.2. Participants

A total of 24 individuals were included in the study: 12 patients with a confirmed diagnosis of epilepsy and 12 neurologically healthy controls.

Both groups were further stratified into the following age cohorts:

• Early childhood (1–3 years): 2 patients (ages 2 and 3),
• Preschool age (4–6 years): 3 patients (ages 4, 5, and 6),
• Primary school age (7–11 years): 6 patients (ages 7, 8, 8, 8, 8, and 10),
• Adolescence (12–18 years): 1 patient (age 12).

The median age in both groups was 7.5 years (range: 2–12 years).

2.3. Data Acquisition and Automatic Preprocessing by Neuroscope Software

EEG recordings were reviewed using both bipolar and monopolar (referential) montages to enable complementary assessment of spatial and temporal activity patterns. The acquisition and display parameters were set as follows:

• Sweep speed: 30 mm/s
• Sensitivity: 7–15 $\mu$V/mm
• Low-frequency cutoff: 0.5 Hz
• High-frequency cutoff: 70 Hz
• Notch filter: 50 Hz (to suppress power-line interference)

Automatic preprocessing was performed using the Neuroscope software, which applied standardized filtering and artifact reduction procedures to ensure signal quality and consistency across recordings.

2.4. Sleep Spindle Annotation

Ten-minute segments of stage N2 non-REM sleep, characterized by rhythmic spindle activity (9–15 Hz) in fronto-central and vertex regions, were selected for manual annotation. Spindle markings were independently reviewed by two EEG experts with more than five years of experience. In cases of uncertainty, the final boundaries were determined through consensus.

Sleep spindles were classified into three categories according to amplitude, duration, and morphology, with each category providing unique benefits for optimizing DL model training:

• Definite spindles (SP): amplitude $\ge 20\mu$V, duration $\ge 1$ s, located in fronto-central and vertex regions. SP serve as high-confidence “anchor” examples, offering clean prototypes of frequency, temporal profile, and waveform shape. They reduce noise in the training set and ensure stable learning during early stages.
• Probable spindles (PS): amplitude $<20\mu$V, duration $\ge 0.5$ s, located in vertex and partially fronto-temporal regions. PS capture borderline cases between genuine spindles and other oscillations (e.g., partial arousals, artifacts). Including these examples improves the model’s ability to generalize and reduces overfitting to only idealized patterns.
• Dubious spindles (DS): amplitude 10–15 $\mu$V or duration $<0.5$ s, isolated in frontal or vertex regions, with unclear morphology. DS act as challenging negative or weakly positive examples. They train the network to reject artifacts and ambiguous oscillations, reflecting the complexity of real-world EEG recordings, particularly in epilepsy. Balanced sampling of these categories across training batches prevents overrepresentation of any one class and improves gradient stability.

The inclusion of “definite,” “probable,” and “dubious” categories encourages the model to learn generalized spindle features rather than overfitting to specific patients or recording systems. This strategy increases robustness to variability in EEG equipment, recording conditions, and age-related signal differences.

2.5. EEG Data Sets Containing Sleep Spindles

Despite EEG’s central role in epilepsy diagnosis, consciousness disorders, and neurological research, resources explicitly dedicated to large-scale EEG remain underrepresented. Namely, databases associated with the DREAMS project [27] are very restricted in size. The Sleep Heart Health Study also proposes a PSG database derived from a prospective cohort study [28]. However, this large collection failed to become accepted as a standard resource by third parties. Potential causes for this limited use may be associated with the fact that the investigators proposing the database were not aiming to provide an open-access and general purpose archive. Indeed, it is oriented specifically towards the investigation of relationships between sleep-disordered breathing and heart diseases, and its recordings are available only upon special request and approval. Also, the EEG montage (C3-A2 and C4-A1 channels, sampled at 125 Hz) is relatively limited for general-purpose investigations in sleep research.

Another source of databases is Physionet. For example, here can be found the CAP sleep database, which include 108 nights with three (or more) EEG channels, from different populations (16 healthy and 92 pathological), targeting the study of cyclic alternating patterns. One can also mention the archive from St Vincent’s University Hospital, which consists of 25 overnight PSGs with two EEG channels from a population of subjects with sleep apnea. Also on Physionet the database from [29] is available. It consists of 61 PSGs with two EEG channels, from subjects with mild sleep onset difficulties, recorded at 100 Hz. However, these archives are limited in different aspects (number of channels, sampling frequency, number of records). Also, in our opinion, none of these databases has been widely accepted for benchmarking automated systems.

Note that recently a large-scale, deidentified, and standardized EEG database supporting artificial intelligence-driven and reproducible research in epilepsy and broader clinical neuroscience was released, Harvard Electroencephalography Database (HEEDB) [31].

It is also worth to mention here crowd-sourcing annotation projects. The most famous of them, Massive Online Data Annotation (MODA) is a web-based open source online scoring platform, which was used in order to produce a large open-source dataset of human-scored sleep spindles (5342 spindles, from 180 subjects). Polysomnographic data for annotation from 180 subjects was sourced from the MASS database. The dataset was split into two ’phases’, where phase 1 consisted of 100 younger subjects (mean age of $24.1$ years old) and phase 2 consisted of 80 older subjects (mean age of $62.0$ years old). A subset of N2 stage sleep from the C3 channel was sampled from each subject (see methods for details). After that 25 sec epochs of this single channel EEG were presented to expert PSG technologists, researchers, and non-expert scorers. As a result, almost 100000 candidate spindles were identified by all scorers combined.

Nevertheless, a custom databases created for specific needs of a certain study are still required. Therefore, in our study we collected custom database of clinical EEG records from patients with epilepsy and controls during stage N2 non-REM sleep, characterized by rhythmic spindle activity in fronto-central and vertex regions, and manually annotate them.

2.6. The Montreal Archive of Sleep Studies (MASS)

One of the most widely used database in sleep studies is the Montreal Archive of Sleep Studies (MASS) [30]. It is an open-access collaborative database of laboratory-based polysomnography (PSG) recordings which includes whole-night recordings from 200 participants, 97 males (aged $42.9\pm 19.8$ years) and 103 females (aged $38.3\pm 18.9$ years). These recordings were pooled from eight different research protocols performed in three different hospital-based sleep laboratories. All recordings feature a sampling frequency of 256 Hz and EEG montage of 4–20 channels plus standard electro-oculography (EOG), electromyography (EMG), electrocardiography (ECG) and respiratory signals. Nevertheless, this database include only healthy controls and therefore is of limited applicability in epilepsy studies.

In this study, we used the SS2 subset, which includes expert annotations of sleep spindles and K-complexes. The SS2 subset contains recordings from 19 healthy subjects (8 males and 11 females). Sleep spindles and K-complexes were annotated by two independent experts: the first expert labeled all 19 recordings, while the second expert labeled sleep spindles in 15 recordings.

EEG data were acquired using the standard 10–20 electrode placement system at a sampling rate of 256 Hz. The primary montage consisted of 16 channels (C3, C4, Cz, F3, F4, F7, F8, O1, O2, P3, P4, Pz, T3, T4, T5, T6), with additional electrodes (Fp1, Fp2, Fpz, Pz, Cz) included depending on the subset and reference CLE channel. The dataset also provides complementary physiological signals, including EOG (4 channels: left, right, up, down), EMG (1 bipolar channel), ECG (1 channel), and respiratory thermistance.

For this work, we considered only the 15 recordings annotated by both experts. Reference labels were obtained by merging their annotations. Obvious labeling artifacts were identified and removed prior to model training.

We employed MASS as a benchmark dataset for the spindle segmentation task. Details of this evaluation are provided in Section 4.1.

3. Methodology

The methodology of this study includes of the following main steps: (i) data preprocessing, (ii) characterization of sleep spindles, (iii) construction and evaluation of deep neural network (DNN) architectures, (iv) sleep event detection, and (v) training and evaluation using quantitative metrics.

3.1. Data Preprocessing for Sleep Spindle Analysis

To select the optimal frequency range for band-pass filtering, we first calculated spectrograms for the EEG recordings:

$$X_k=\left|\sum _{n=0}^{N-1}\left(x_n{e}^{-i2\pi {\displaystyle \frac{k}{N}}n}\right)\right|,$$

(1)

where $x_n$ denotes the EEG sample from a given channel, $X_k$ is the spectral power at frequency k, and N is the length of the fast Fourier transform (FFT), set to 256. The step size between consecutive FFT windows was 128.

Labels provided in the dataset were mapped onto the spectrograms, resulting in two groups: (i) spectrograms corresponding to sleep spindle segments, and (ii) spectrograms corresponding to non-spindle segments. We then calculated the average spectra for both groups and analyzed their differences for each EEG channel.

As expected, the most prominent differences appeared in the spindle frequency range. Based on these findings, a band-pass filter of $0.1$–35 Hz was applied to the EEG recordings. Following filtering, each channel was normalized to zero mean and unit variance, and artifacts were attenuated by clipping amplitudes at $\pm 10$. However, subsequent experiments demonstrated that normalization was not essential for DNN training, as model performance remained nearly identical with or without normalization.

3.2. Sleep Spindle Characteristics

Sleep spindles are defined by their spectral (frequency content and intra-frequency structure) and temporal (shape and duration) properties [53]. In this study, we quantified four main characteristics: (i) average duration, (ii) average maximal amplitude, (iii) average frequency, and (iv) number of spindles, as illustrated in Figure 1.

Let x denote a segment of an EEG recording, and $SS=\{s{s}_0,s{s}_1,\dots ,s{s}_n\}$ be the set of sleep spindles detected in this segment.

The average duration was calculated as:

$$D={\displaystyle \frac{1}{\left|SS\right|}}\sum _{ss\in SS}\left(t_{\mathrm{finish}}\left(ss\right)-t_{\mathrm{start}}\left(ss\right)\right),$$

(2)

where $t_{\mathrm{start}}\left(ss\right)$ and $t_{\mathrm{finish}}\left(ss\right)$ are the start and end times of spindle $ss$, respectively.

To compute amplitude and frequency, the EEG signal from each channel was centered:

$$\widehat{x_t}=x_t-E\left(x\right),$$

(3)

where $E\left(x\right)$ is the mean of x over the analysis window.

The average maximal amplitude was then obtained as:

$$A={\displaystyle \frac{1}{\left|SS\right|}}\sum _{ss\in SS}\left(\underset{t=t_{\mathrm{start}}\left(ss\right)}{\overset{t_{\mathrm{finish}}\left(ss\right)}{max}}\widehat{x_t}\right).$$

(4)

Amplitude values were averaged across all EEG channels.

The average frequency was estimated using zero-crossing counts:

$$F={\displaystyle \frac{1}{\left|SS\right|}}\sum _{ss\in SS}\left({\displaystyle \frac{ZC\left(\widehat{x}\left(ss\right)\right)}{t_{\mathrm{finish}}\left(ss\right)-t_{\mathrm{start}}\left(ss\right)}}\right),$$

(5)

where $ZC\left(x\right)$ denotes the number of zero-crossings in the signal x, and $\widehat{x}\left(ss\right)$ represents the signal segment corresponding to spindle $ss$. Frequencies were averaged across all channels.

The number of spindles is defined as the cardinality of the set $SS$:

$$N=\left|SS\right|.$$

(6)

Finally, the spindle density was computed as the proportion of recording time occupied by spindles [54,55]:

$$\mathrm{Density}={\displaystyle \frac{1}{T}}\sum _{ss\in SS}\left(t_{\mathrm{finish}}\left(ss\right)-t_{\mathrm{start}}\left(ss\right)\right),$$

(7)

where T is the duration of the analyzed EEG segment x.

3.3. Performance of DNN Architectures on Pathological Spindles

Automated sleep spindle detection is crucial for advancing sleep research and clinical diagnostics, offering scalability and objectivity beyond manual scoring. While traditional methods, including wavelet analysis [20,32,33,34,35] and classical machine learning (e.g., SVM [36], KNN [37], decision trees [38], bagging classifiers [39]), have laid the groundwork, deep neural networks (DNNs) now represent the state of the art due to their superior ability to learn complex features directly from raw data.

Approaches based on DNN architectures show superior performance with respect to the F1-score. Three main types of DNN architectures can be used in this task: (i) convolutional neural networks (CNN), capable of automatic feature extraction and learning, (ii) recurrent neural networks (RNN), particularly LSTMs, designed for time series processing, and (iii) mixed architectures allowing the benefits of both approaches to be used to improve the quality of sleep spindles detection. Mixed architectures often can be additionally equipped with attention heads or specific data preprocessing or feature selection pipelines, see, e.g., [40,41].

In this study, we investigated two families of DNN architectures: (1) CNNs based on ResNet, and (2) hybrid models combining CNN and recurrent layers.

3.3.1. SlumberNet Architecture

SlumberNet is a convolutional deep learning model based on the residual network (ResNet) architecture, originally developed by Jha et al. [50] to classify sleep stages in mice using EEG and EMG signals. The authors also demonstrated how the model could be adapted for human polysomnographic data.

In our work, we modified SlumberNet for spindle characteristic prediction in a segmentation-free framework. Specifically, the original classification task was reformulated as a regression problem, and an encoder module was added to obtain compressed latent representations of EEG segments. A schematic of the adapted architecture is shown in Figure 2.

To reduce the original time resolution before residual blocks, we introduced two initial convolutional layers. Each residual block contains convolution, batch normalization, and dropout layers. The network is composed of multiple such blocks (eight in our implementation), followed by a global average pooling layer and a fully connected layer. The compression ratio of the Conv1d layers and the number of residual blocks were optimized via grid search.

3.3.2. U-Net Architecture

The U-Net model is one of the most widely used CNN architectures for segmentation tasks. The 1D U-Net is an adaptation of the original 2D image segmentation model to one-dimensional sequential data, making it well suited for biomedical signal analysis, including EEG. The schematic representation of the 1D U-Net architecture is presented in Figure 3.

Like its 2D counterpart, the 1D U-Net follows an encoder–decoder structure with skip connections. The encoder progressively reduces the temporal resolution of the input through convolution and downsampling layers, while the decoder reconstructs the signal using upsampling layers. Skip connections bridge encoder and decoder stages, ensuring that fine-grained temporal features are preserved.

The network receives an EEG segment of size $W\times F$ as input. The encoder comprises L downsampling blocks, each consisting of convolutions (to increase the number of channels) and D convolutional layers with residual connections. The decoder contains L upsampling blocks, each concatenating the corresponding encoder output and applying convolutions with decreasing channel dimensionality. A final one-dimensional convolution with kernel size 1 produces the output segmentation mask of size $W\times 2$.

One of the main advantages of the 1D U-Net is its ability to capture both short- and long-range dependencies in sequential data with relatively low computational cost. Its symmetric encoder–decoder design ensures that information is preserved across different temporal scales, which is essential for accurate spindle segmentation. Input window size, number of encoder/decoder blocks, and convolutional depth D were tuned as hyperparameters via grid search.

In this study, we compared the 1D U-Net and SEED models for the task of sleep spindle segmentation (see Section 4.1).

3.4. Sleep EEG Event Detector

The Sleep EEG Event Detector (SEED) is a state-of-the-art deep learning model for sleep spindle segmentation [51]. It combines CNNs for local feature extraction with bidirectional long short-term memory (BiLSTM) layers for contextual modeling, thereby capturing both fine-grained temporal structure and long-range dependencies in EEG signals.

The SEED workflow consists of three stages: local encoding, contextualization, and sample-wise classification. Input segments contain 5040 time samples, of which the central 4000 samples constitute the prediction window. To mitigate boundary artifacts, 520 samples are appended on each side. The model outputs a dense probability sequence of 500 samples, corresponding to one prediction per 8 input samples.

In the local encoding stage, a convolutional block extracts low-level temporal features and downsamples the signal by a factor of eight, yielding a compressed multivariate time series of length 500. The contextualization stage then applies BiLSTMs to integrate information across distant samples, enabling the network to capture dependencies beyond the receptive field of convolutional filters. Finally, in the classification stage, a one-dimensional convolution followed by a softmax layer produces sample-wise probabilities of belonging to the positive (spindle) or negative (background) class.

The architecture of SEED is illustrated in Figure 4. It begins with two one-dimensional convolutions (kernel size = 3, output channels $F=64$), followed by max pooling and convolutional multi-dilated blocks (MDBs), which capture patterns across multiple temporal scales. Boundary shortening is applied to align feature maps, and two dropout layers ($q_1=0.2$, $q_2=0.5$) provide regularization. Contextual dependencies are modeled by BiLSTMs with $N_1=128$ hidden units. The final classification stage uses two one-dimensional convolutions (kernel size = 1): the first maps to $N_2=256$ features, while the second produces two output channels corresponding to event vs. background classes.

To extend SEED beyond its original design, we introduced several modifications. For segmentation tasks, upsampling layers were incorporated to preserve temporal resolution. For regression tasks, the upsampling layers were replaced with linear layers, and the degree of compression in the early convolutional blocks was increased to obtain more compact latent representations.

Model hyperparameters were tuned via grid search. For segmentation, the prediction window size and input expansion were optimized based on the average F1-score across test or cross-validation folds. For regression, the hidden size of the final linear layer was treated as a hyperparameter and optimized according to the mean squared error (MSE) loss.

In this study, we employed SEED in both classification and regression settings (see Section 4.1, Section 4.2 for details).

3.5. Training Process and Evaluation Metrics

For clinical EEG recordings, model training was carried out using a cross-validation scheme. Given the limited number of subjects, in each iteration all but two recordings were used for training, while the remaining two served as the evaluation set. This strategy was chosen instead of a single train–test split (used for the MASS dataset) to obtain a more reliable estimate of model generalization across different patients. Performance metrics were averaged across folds, reducing bias from random data partitioning and providing a more robust evaluation in real-world scenarios.

3.5.1. Segmentation-Based vs. Segmentation-Free Approaches

Two strategies for predicting sleep spindle characteristics were compared:

• Segmentation-based approach: Sleep spindles were first detected using DNN models in a sequence labeling framework, where each EEG time point was classified as spindle or non-spindle. Spindle characteristics were then computed from the resulting segmentation.
• Segmentation-free approach: Spindle characteristics were predicted directly from raw EEG segments using regression models, without an intermediate segmentation step.

Within the segmentation-free approach, we examined two subsettings: (i) training a network to predict each characteristic independently, and (ii) training a single network to predict all four characteristics simultaneously using normalized outputs and weighted loss terms. Numerical experiments demonstrated that the multi-output setting consistently underperformed the single-output models in terms of MSE.

3.5.2. Loss Functions

For segmentation tasks, the binary cross-entropy (BCE) loss was used:

$$BCE=-{\displaystyle \frac{1}{n}}\sum _{i=1}^n\left(y_{i}log\left(\widehat{y_i}\right)+(1-y_i)log(1-\widehat{y_i})\right),$$

(8)

where n is the number of training samples, $y_i$ is the ground-truth label, and $\widehat{y_i}$ is the predicted probability.

For regression tasks (direct prediction of spindle characteristics), the MSE loss was optimized:

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-\widehat{y_i})}^2.$$

(9)

3.5.3. Evaluation Metrics

For segmentation, model performance was evaluated at the level of complete spindle events, not individual samples, using precision, recall, and F1-score:

$$Precision={\displaystyle \frac{TP}{TP+FP}},Recall={\displaystyle \frac{TP}{TP+FN}},F1=2·{\displaystyle \frac{Precision·Recall}{Precision+Recall}},$$

(10)

where $TP$, $FP$, and $FN$ denote the numbers of true positives, false positives, and false negatives, respectively.

3.5.4. Hyperparameter Optimization

Hyperparameters were optimized via grid search. For segmentation, the optimal settings were chosen based on the averaged F1-score across folds; for regression, the mean MSE loss was used as the selection criterion. In segmentation, the input window size was set to 4000 samples for the 1D U-Net, and to $4000+520$ samples (with boundary padding) for SEED. The predicted label segment length was 4000 samples for both networks. In regression mode, SEED used a hidden layer size of 256 in the final linear block.

4. Results

In this section, we present the principal findings of our study, supported by empirical evidence and quantitative analysis. For each set of experiments, we provide a focused discussion that interprets the results, situates them within existing literature, and highlights their implications for understanding epilepsy-related sleep spindle dynamics in EEG recordings.

4.1. Comparison of DL Architectures for Sleep Spindle Segmentation

We first evaluated the quality of sleep spindle segmentation across different DNN architectures to identify the most suitable model. The experiments were conducted on a 10-minute EEG fragment for clinical recordings and on standard-length recordings for the MASS dataset. Model performance was assessed using precision, recall, and F1-score, with automatic annotations compared against expert labels on a spindle-by-spindle (event-level) basis.

As shown in Figure 5, both architectures produced accurate spindle segmentations, closely aligned with expert annotations. The quantitative results are summarized in Table 1 (MASS dataset) and Table 2 (clinical dataset).

The results show that both U-Net and SEED achieve comparable F1-scores on the MASS dataset and clinical data. However, the trade-off between precision and recall differs: SEED exhibits higher precision (fewer false positives), whereas U-Net achieves higher recall (fewer missed spindles). This difference reflects their architectural design: recurrent layers in SEED favor specificity, while U-Net’s convolutional structure prioritizes sensitivity. Depending on the clinical objective, one architecture may be preferable over the other, for example, U-Net when minimizing missed detections, and SEED when reducing false alarms.

4.2. Comparison of Segmentation-based and Segmentation-Free Approaches

Next, we compared two strategies for predicting spindle characteristics. The first approach (segmentation-based) derives spindle properties using formulas from Section 3.2 applied to automatically segmented events. The second approach (segmentation-free) directly predicts characteristics using regression networks, bypassing explicit segmentation.

4.2.1. Distribution Analysis

We first compared distributions of spindle characteristics obtained via the two approaches against expert-derived ground truth. This analysis was carried out using CNN-based models (1D U-Net and SlumberNet) and hybrid architectures with recurrent layers (SEED). The distributions are shown in Figure 6. For both model families, the segmentation-based approach produces distributions that more closely match expert annotations. By contrast, direct regression tends to generate overly smoothed (averaged) outputs. An exception is observed with SEED, where amplitude and frequency distributions remain relatively consistent across both approaches.

4.2.2. Prediction Accuracy

Scatter plots in Figure 7 illustrate the relationship between predicted and expert-calculated characteristics. Segmentation-based predictions cluster more closely around the diagonal (ideal match), while segmentation-free predictions show larger deviations, consistent with their averaging tendency.

4.2.3. Error Analysis

We further quantified errors using the absolute relative error (ARE), defined as

$$ARE={\displaystyle \frac{|x-\widehat{x}|}{\left|x\right|}}\times 100\%,$$

(11)

where x is the expert-derived value and $\widehat{x}$ is the model-predicted value. Boxplots in Figure 8 show that segmentation-based approaches consistently achieve lower AREs. Statistical testing (paired t-test, $\rho <0.01$) confirmed the significance of these differences.

To better characterize error distributions, we also computed empirical cumulative distribution functions (ECDFs) of ARE (Figure 9). Segmentation-based methods consistently achieve higher fractions of predictions with low error, underscoring their robustness.

Overall, the segmentation-based approach outperforms segmentation-free regression. The explicit localization of events during segmentation enables more precise estimation of spindle properties. By contrast, direct regression without segmentation must implicitly infer event structure, often leading to information loss and characteristic averaging. These findings suggest that segmentation remains a critical step for accurate and clinically reliable characterization of sleep spindles.

4.3. Alterations in Sleep Spindle Properties in Established Epilepsy

We compared sleep spindle characteristics between patients with epilepsy and control subjects using expert annotations and the outputs of our 1D U-Net segmentation model, which demonstrated superior performance (highest F1-score). Representative examples of sleep spindles from both groups are shown in Figure 10, illustrating clear morphological alterations in the epileptic cohort.

Quantitative analysis revealed significant alterations in key spindle parameters in epilepsy. Figure 11 presents the distributions of spindle duration, amplitude, oscillatory frequency, count, and density. The most pronounced and statistically significant group differences were observed in spindle duration and density, a finding consistent across both expert annotations and model-derived segmentations (Mann-Whitney U test, *p* < 0.01 for both comparisons).

To evaluate the separability of the two groups in a multivariate feature space, we employed a k-nearest neighbors (KNN) classifier. Spindle density was selected as a key integrative feature, as it encapsulates information on both spindle occurrence and duration. Figure 12 visualizes the decision boundaries generated by the KNN model in two feature subspaces. The analysis demonstrated high group separability, with classification accuracies of 82% (amplitude vs. density) and 85% (frequency vs. density), confirming that the identified alterations in spindle properties provide a robust electrophysiological signature of epilepsy.

In summary, our results demonstrate that epilepsy is associated with significant and quantifiable alterations in sleep spindle morphology and incidence. The automated analysis of these features provides a robust biomarker that can reveal a detailed picture of thalamocortical dysfunction in epilepsy. This approach has strong potential to facilitate rapid diagnosis and guide targeted therapeutic strategies.

5. Discussion

This study provides the first rigorous evaluation of modern DNN architectures for detecting and characterizing sleep spindles in EEG recordings affected by epilepsy. We show that while both 1D U-Net and SEED achieve competitive overall performance (F1-scores), their precision–recall trade-offs differ markedly in the clinical setting. This highlights a key insight: model selection cannot rely solely on F1-scores but must be tailored to the clinical objective and tolerance for error type.

5.1. Clinical Model Selection Framework

Our results support a principled strategy for selecting models in clinical practice:

• High-Precision Option (e.g., SEED): Prioritizes minimizing false positives. This is advantageous in diagnostic contexts, where misclassifying epileptiform discharges or noise as spindles can corrupt biomarker quantification and lead to misleading conclusions. In this setting, SEED provides high confidence in detected spindles.
• High-Recall Option (e.g., 1D U-Net): Prioritizes capturing all true spindles, tolerating more false positives. This is valuable for screening or longitudinal monitoring, where the cost of missing altered spindle activity outweighs the burden of reviewing additional candidate events.

This framework empowers clinicians and researchers to select architectures not only based on accuracy but also in alignment with the clinical trade-off between false positives and false negatives.

5.2. Synthesis of Challenges and Contributions

Our work addresses two central challenges in automated sleep EEG analysis:

• Limited annotated clinical data. We mitigate this barrier by introducing a curated dataset of sleep spindles in pediatric epilepsy, showing that DNNs remain effective despite pathological spindle morphology. This resource provides a foundation for future model development and benchmarking in clinical populations.
• Building reliable clinical pipelines. We demonstrate and validate robust spindle segmentation as a key pipeline component. Automating this step reduces expert workload, improves reproducibility, and enables integration of spindle analysis into diagnostic and monitoring workflows.

The divergent performance of the models can be explained by their architectural inductive biases. CNNs such as the 1D U-Net specialize in capturing local patterns, which enhances sensitivity but risks false positives from events with similar local structure. In contrast, hybrid models like SEED leverage bidirectional recurrent layers to integrate long-range temporal dependencies, allowing better discrimination of true spindles from epileptiform discharges or other artifacts, thereby increasing precision.

5.3. The Critical Data Gap in Epilepsy Sleep Research

A persistent obstacle in this field is the lack of large-scale, annotated EEG datasets reflecting pathological sleep. Existing public archives, including DREAMS [27], CAP, St Vincent’s, and Kemp et al. [29] on PhysioNet, suffer from small size, limited electrode montages, low sampling rates, or a primary focus on conditions such as sleep apnea. The Montreal Archive of Sleep Studies (MASS) [30] remains a high-quality benchmark, but as it contains only healthy subjects, it is inadequate for studying spindle alterations in epilepsy. Even recent large-scale initiatives such as the Harvard EEG Database (HEEDB) [31] are not designed for spindle-specific research.

The gap extends to spindle annotations. Efforts such as MODA [46] have successfully generated large-scale labels, but these are again based on MASS recordings. Consequently, state-of-the-art models (e.g., DOSED [42], SpindleNet [25], RED-CWT [43], SpindleU-Net [40], and SUMO [45]) have been validated only on normal spindles. Their robustness to pathological morphologies, particularly those altered by epileptiform discharges, remained untested prior to this study. This limitation has slowed the clinical translation of powerful CNN, RNN, and hybrid architectures such as SEED [51] into epilepsy care.

To bridge this gap, we developed a unique database of annotated EEG from pediatric patients with epilepsy and matched controls during N2 sleep. This dataset enables systematic investigation of spindle alterations associated with epilepsy and serves as a benchmark for evaluating advanced DNNs under pathological conditions. Beyond advancing methodology, it provides a translational step toward clinically viable, automated spindle analysis in epilepsy.

6. Conclusions and Future Directions

This study demonstrates the efficacy of deep learning for identifying and analyzing alterations in sleep spindles within EEG recordings from patients with epilepsy. Through a structured investigation, we have established several key findings with significant implications for both neuroscience research and clinical diagnostics.

Our primary contribution is the creation of a novel, expert-annotated dataset of sleep spindles in a pediatric cohort with established epilepsy, addressing a critical gap in publicly available resources that predominantly feature data from healthy subjects. Utilizing this dataset, we rigorously benchmarked state-of-the-art DNN architectures, namely 1D U-Net and SEED, for spindle segmentation. We confirmed that while both models achieve competitive performance, they exhibit a crucial clinical trade-off: the 1D U-Net architecture favors high recall, maximizing the detection of true spindles, whereas the SEED model favors high precision, minimizing false positives. This distinction provides a principled framework for model selection based on specific clinical scenarios, moving beyond a sole reliance on the F1-score.

Furthermore, our analysis definitively established that an automated segmentation step is indispensable for the accurate quantification of spindle characteristics. By applying this optimized pipeline, we identified statistically significant alterations in key spindle parameters, including duration, amplitude, and density, in patients with epilepsy compared to matched controls. These quantifiable differences manifest as well-separable clusters in the feature space, confirming that DNN models can effectively capture the pathological signature of epilepsy embedded within sleep microarchitecture.

In summary, this work provides a robust foundation for using automated tools to extract reliable biomarkers from sleep EEG in epilepsy. The precision-recall trade-off we identified offers clinicians a flexible choice between a sensitive screening tool (high-recall model) and a specific diagnostic confirmatory tool (high-precision model).

Future Directions will build directly upon these findings:

(i)

Dataset Expansion: Curating a larger, multi-center dataset encompassing diverse epilepsy syndromes and age groups to enhance model generalizability and robustness.

(ii)

Clinical Translation: Developing real-time detection algorithms and integrating spindle analysis with other EEG biomarkers (e.g., slow waves, epileptiform discharges) into a unified clinical dashboard to aid diagnosis and monitor therapy response.

(iii)

Validation: The essential next step towards clinical implementation is a rigorous external validation of our models and proposed pipeline on a completely independent patient cohort to confirm their efficacy and reliability.

By elucidating the strengths and limitations of different DNN architectures, this study provides valuable guidance for researchers and clinicians, paving the way for advanced, automated tools that can reduce diagnostic latency and improve patient care in epilepsy.