Deep Learning-Based Assessment of Brainstem Volume Changes in Spinocerebellar Ataxia Type 2 (SCA2): A Study on Patients and Preclinical Subjects

1. Introduction

Spinocerebellar Ataxia type 2 (SCA2) is a rare neurodegenerative disorder characterized by progressive degeneration of the brainstem and cerebellum. As one of the most prevalent spinocerebellar ataxias globally [1,2,3,4], it exhibits a notably high incidence in Holguin, Cuba [1,5]. Clinical manifestations include a cerebellar syndrome, slowing of the saccadic ocular movements, cognitive disorders, sensory neuropathy, etc.

Three patterns of macroscopic atrophy reflecting damage of different neuronal system are recognized in spinocerebellar ataxias, named spinal atrophy (SA), olivopontocerebellar atrophy (OPCA) and cortico-cerebellar atrophy (CCA) [6]. Neuroimaging has played a pivotal role in diagnosing these atrophy patterns since 1995, when Kumas et al. [7] first described OPCA features in pediatric populations. Magnetic resonance imaging (MRI) remains the gold standard for structural segmentation and volumetric analysis due to its superior resolution [8], outperforming other modalities such as single-photon emission computed tomography (SPECT) and positron emission tomography (PET) in visualizing SA, OPCA, and CCA [9]. As emphasized by Klaes et al. [10], MRI is the most extensively validated biomarker candidate for spinocerebellar ataxias.

Brainstem atrophy has been documented across both symptomatic and prodromal stages of SCA2 [1,2,3,11,12,13,14,15,16,17,18,19]. However, most studies rely on manual segmentation, a method constrained by time-intensive workflows, inter-rater variability, and scalability limitations in large cohorts. To address these challenges, this work introduces an automated deep learning framework for quantifying volumetric changes in SCA2 patients, preclinical carriers, and healthy controls.

Convolutional neural networks (CNNs or convnets) [20,21] have demonstrated outstanding performances at tasks such as hand-written digit classification, face and contour detection [22], automatic video processing [23], and neurological behavior analysis and prediction [24,25,26]. In neuroimaging, CNNs have been applied to brain lesion segmentation using 3D architectures [27], U-Net models [28,29,30,31,32] and adversarial training [33], leveraging hierarchical feature extraction. While 2D CNNs reduce computational costs [34,35,36,37], they often sacrifice accuracy compared to hybrid 2D-3D approaches that optimize feature integration [38,39]. One of the most interesting advances includes cerebellum parcellation via cascaded CNNs [40,41], highlighting their potential for fine-grained neuroanatomical analysis.

Building on prior work [42], this study applies CNNs to brainstem segmentation in MRI, with a focus on mesencephalon, pons, and medulla volumetric changes. To date, no studies have employed deep learning to compare brainstem atrophy between symptomatic SCA2 patients, preclinical carriers, and controls in Cuba. This approach aims to establish a scalable, objective tool for identifying early biomarkers of SCA2 progression.

2. Materials and Methods

The proposed model architecture builds upon the method described in [42], utilizing a 3D U-Net-like framework to perform volumetric segmentations. This design processes the input images in their native 3D spatial context, which allows to preserve anatomical relationships. The main architecture contains four downsampling and upsampling operations. To enhance feature extraction, each convolutional layer was replaced with an inception module [37], which optimizes computational efficiency by processing multi-scale features in parallel. Additionally, skip connections were refined using three consecutive Convolutional Block Attention Modules (CBAM) [43], enabling the model to focus on spatially and channel-wise relevant features in these branches. The overall model and inception architectures are illustrated in Figure 1, with detailed schematics of the CBAM blocks provided in Figure 2.

The study employed a cohort of 42 MRI scans obtained from the Cuban Neurosciences Center. These scans correspond to 25 individuals, comprising five healthy controls, seven preclinical subjects, and 13 SCA2 patients. Given hardware limitations, the model was designed to balance computational efficiency with performance, ensuring feasibility on available infrastructure while maintaining robust segmentation accuracy.

2.1. Image Preparation

The full preparation process for one single image can be depicted in Figure 3. All MRI scans underwent preprocessing to ensure consistency and improve segmentation accuracy. First, N4 bias field correction [44] was applied to address intensity inhomogeneities, enhancing image quality for subsequent analysis. Following this, each scan was registered to the ICBM 2009c nonlinear symmetric template [45] using Advanced Normalization Tools (ANTS) [46]. The registration process employed a three-stage approach: rigid, affine, and symmetric normalization (SyN), ensuring optimal alignment of individual scans to the standardized template space. The Figure 3a shows the original image, and 3b displays the result of N4+MNIregistration.

To optimize computational efficiency, MRI scans were automatically cropped to focus exclusively on the brainstem region. Using training set segmentations as reference, a standardized region of interest (ROI) measuring 80×80×96 voxels was extracted for each scan. This approach reduced the computational load by 93%, decreasing processed volumes from approximately 8.5 million voxels per full scan to ~614,000 voxels. The cropped ROIs enabled efficient model training and inference while preserving all relevant anatomical data for brainstem analysis. The Figure 3d shows the result of the cropping operation. Following the crop, intensity normalization was applied for every image (Figure 3e). The reduced volumes were used as inputs to the 3D U-Net. The Figure 3f shows one fully preprocessed image overlapped with its manual segmentations.

2.2. Analysis Description

The study was implemented in Python 3.9, utilizing TensorFlow [47] and Keras [48] for model development and training. The model was trained over 250 epochs using the Adam optimizer [49] with default parameters. To mitigate overfitting, a dropout rate of 0.2 was applied before the final layer.

For training, the label maps are converted into C binary channels, and each channel represents a label. At the same time, each voxel can only belong to one label. The loss function used was one minus the average Dice score (DSC) across all channels, and the DSC is computed using equation (1), where N is the number of voxels in the image domain, $x_{ci}$ is the $i^{th}$voxel in the $c^{th}$ channel of the prediction, and ε prevents division by zero [41].

$$DSC={\displaystyle \frac{1}{C}}{\sum }_{c=1}^C{\displaystyle \frac{\epsilon +{\sum }_{i=1}^N{2x}_{ci}{y}_{ci}}{\epsilon +{\sum }_{i=1}^N\left(x_{ci}+y_{ci}\right)}}$$

(1)

The experiment was conducted on an NVIDIA RTX 3060 6GB GPU. The dataset was partitioned into 17 images for training, 3 for validation, and 22 for testing. To enhance model generalization, data augmentation techniques were applied during training, including rotations, translations, and flipping. No augmentation was applied to testing images.

Volumetric comparisons were performed using the percentage of the region of interest with respect to the total intracranial volume (% TICV).

3. Results

The evaluation results, summarized in Table 1, demonstrate high segmentation accuracy across all regions of interest. Mean Dice similarity coefficients (DSC) exceeded 0.95 for all structures, with the highest score (0.97) achieved for the whole brainstem. The mesencephalon exhibited the lowest mean DSC (0.93), indicating consistent yet slightly reduced performance in this region. These results highlight the model’s robustness and reliability in segmenting brainstem subregions.

Figure 4 presents a qualitative comparison of segmentation results, showing a representative test image alongside its predicted segmentation. The proposed method was evaluated against three established approaches: (1) an upscaled version of the model architecture from [42], (2) the cerebellar parcellation network by Han et al. [41], and (3) the brainstem parcellation model by Magnusson et al. [50]. To ensure a fair comparison, all models were re-trained under identical conditions, maintaining consistent training protocols (loss functions, optimizer parameters, and regularization strategies). Quantitative results of this comparative analysis are presented in Table 1.

The quantitative results in Table 1 demonstrate that the proposed method achieves consistently superior segmentation performance across all brainstem substructures compared to existing approaches. The method attains higher DSC for the medulla (0.96 ± 0.022 vs. ≤0.93), pons (0.96 ± 0.015 vs. ≤0.94), mesencephalon (0.95 ± 0.021 vs. ≤0.92), and full brainstem (0.96 ± 0.008 vs. ≤0.95), alongside reduced standard deviations, indicating enhanced robustness. These improvements can be attributed to the two key architectural changes made to the U-Net: (1) the integration of attention mechanisms within skip connections to refine feature aggregation, and (2) the replacement of conventional convolutional layers with inception modules to capture multi-scale contextual information more effectively. By addressing limitations related to structural heterogeneity and boundary ambiguity—common challenges in brain MRI segmentation—the method offers improved accuracy and consistency. Its performance suggests strong potential for clinical applications requiring precise anatomical delineation, such as degenerative disease monitoring.

Quantitative evaluation of computational efficiency revealed segmentation times of <1 second per image when utilizing GPU acceleration (NVIDIA RTX 3060 MOBILE, 6GB GDDR6), while CPU-based processing (Intel Core i5-10500H, 16GB DDR4 RAM) required 210±15 seconds per case. This represents a 60-120× speed improvement compared to manual segmentation protocols while maintaining diagnostic-grade accuracy.

Using the segmentation results for all the images of the initial cohort, volumetric changes were calculated for SCA2 patients, preclinical subjects, and healthy controls. Volumes were normalized as a percentage of the total intracranial volume (% TICV), with TICV computed using ROBEX [51]. The results, illustrated in Table 2, align with findings reported by Reetz et al. [52]. In all cases, mean volumes for patients were lower than those for preclinical subjects, which in turn were lower than controls.

The most pronounced differences were observed in the pons, with mean volumes of 0.47% TICV for patients, 0.76% TICV for preclinical subjects, and 0.82% TICV for controls. Notably, the median volume for controls was nearly double that of patients. Differences between preclinical subjects and controls were less pronounced. In the mesencephalon, mean volumes were 0.40% TICV for patients, 0.44% TICV for preclinical subjects, and 0.48% TICV for controls. The medulla exhibited the smallest volumetric differences, with values of 0.26%, 0.29%, and 0.31% TICV for patients, preclinical subjects, and controls, respectively. At the whole brainstem level, mean volumes were 1.12%, 1.49%, and 1.62% TICV for patients, preclinical subjects, and controls, respectively, further highlighting the progressive nature of brainstem atrophy in SCA2.

4. Discussion

This study presented a deep learning-based framework for analyzing MRI scans to quantify volumetric changes in the brainstem of SCA2 patients and preclinical subjects compared to healthy controls. To the best of our knowledge, this represents the first such study conducted in Cuba, addressing a critical need for accessible and efficient tools to study neurodegenerative diseases in resource-constrained settings.

The success of our approach stems from the inherent advantages of the 3D U-Net for medical image segmentation. Unlike classical techniques (e.g., atlas-based or graph-cut methods) that rely on handcrafted features—which often fail to capture complex anatomical variability [29]—CNNs automatically learn discriminative hierarchical features, enabling precise parcellation of challenging structures like the brainstem substructures [53]. Our modified 3D U-Net architecture incorporates two key changes—attention mechanisms in skip connections and inception modules replacing standard convolutions. This advanced and complex architecture achieves expert-level segmentation accuracy (Dice >0.95 for all brainstem substructures). The attention modules enable precise localization of anatomical boundaries by selectively emphasizing relevant spatial features, while the inception modules capture multi-scale contextual information critical for distinguishing between adjacent brainstem subregions.

A notable limitation of the proposed approach is its reliance on the registration phase. While registration was successful for all MRI scans in this study, its performance may vary with different datasets, potentially leading to segmentation errors if registration fails. Future work will focus on developing registration-free pipelines to enhance robustness and generalizability.

Despite this limitation, the results demonstrate that deep learning techniques can effectively characterize brainstem atrophy, enabling rapid differentiation between patients, preclinical subjects, and controls. The computational efficiency of the proposed method represents a significant advancement, with GPU-based segmentation completing in under one second per image—a dramatic improvement over manual segmentation, which can require over an hour per case. Even without GPU acceleration, segmentation times of up to 3.5 minutes on a standard CPU still offer a substantial reduction in processing time compared to manual methods.

These findings suggest that the proposed framework can be integrated into larger neuroimaging pipelines to assess volumetric changes in SCA2 patients and preclinical subjects. The development of user-friendly software based on this approach could provide clinicians with a powerful tool for rapid diagnosis and monitoring of disease progression. By highlighting key atrophic changes and enabling longitudinal evaluation, such tools could improve patient care and support early intervention strategies.

In the broader context of neurodegenerative disease research, this work underscores the potential of deep learning to bridge gaps in neuroimaging analysis, particularly in regions with limited access to advanced computational resources. Future research directions include expanding the dataset to improve model generalizability, exploring multi-modal imaging approaches, and investigating the applicability to other neurodegenerative conditions.

5. Conclusions

This study introduced a deep learning-based framework to quantify brainstem atrophy in SCA2 patients, preclinical subjects, and healthy controls, representing a pioneering effort in Cuba. By achieving mean Dice scores above 0.96 for the whole brainstem and 0.95 for its subregions, the approach demonstrates high accuracy in detecting significant volumetric differences, particularly in the pons. These findings highlight the potential of deep learning to address critical gaps in neuroimaging analysis. The method enables rapid, scalable assessments, reducing reliance on time-intensive manual segmentation and supporting earlier diagnosis and monitoring of SCA2.