Deep Learning-Based Assessment of Brainstem Volume Changes in Spinocerebellar Ataxia Type 2 (SCA2): A Study on Patients and Preclinical Subjects

1. Introduction

Spinocerebellar Ataxia type 2 (SCA2) is a rare neurodegenerative disorder characterized by progressive degeneration of the brainstem and cerebellum. As one of the most prevalent spinocerebellar ataxias globally [1,2,3,4], it exhibits a notably high incidence in Holguin, Cuba [1,5]. Clinical manifestations include a cerebellar syndrome, slowing of the saccadic ocular movements, cognitive disorders, sensory neuropathy, etc [6].

Three patterns of macroscopic atrophy reflecting damage of different neuronal system are recognized in spinocerebellar ataxias, named spinal atrophy (SA), olivopontocerebellar atrophy (OPCA) and cortico-cerebellar atrophy (CCA) [7]. Neuroimaging has played a pivotal role in diagnosing these atrophy patterns since 1995, and magnetic resonance imaging (MRI) remains the gold standard for structural segmentation and volumetric analysis due to its superior resolution [8], outperforming other modalities such as single-photon emission computed tomography (SPECT) and positron emission tomography (PET) in visualizing SA, OPCA, and CCA [9]. According to recent literature [10,11], MRI is one of the most common biomarker candidate for spinocerebellar ataxias.

Brainstem atrophy has been documented across both symptomatic and prodromal stages of SCA2 [1,2,3,12,13,14,15,16,17,18]. However, most studies rely on manual segmentation, a method constrained by time-intensive workflows, inter-rater variability, and scalability limitations in large cohorts. To address these challenges, this work introduces an automated deep learning framework for quantifying volumetric changes in SCA2 patients, preclinical carriers, and healthy controls.

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance across diverse domains, including handwritten digit classification [19], face and contour detection [20], automatic video processing [21,22] and neurological behavior analysis and prediction[23,24,25]. In neuroimaging, CNNs have become indispensable for brain lesion segmentation and structural parcellation [26], with architectures like U-Net [27,28,29,30], ResNet [31], and VGG-Net [32], leveraging adversarial training [33] and hierarchical feature extraction to enhancing learning robustness. Among the most interesting advances in brain structure segmentation, we can find cerebellum parcellation via cascaded CNNs used by Han et al. [34], highlighting their potential for fine-grained neuroanatomical analysis and the 2D approach proposed by Faber et al. [29]. Although 2D convolutional models exhibit greater computational efficiency and lower resource demands by processing individual image slices independently, 3D convolutional models ultimately achieve superior segmentation performance by leveraging volumetric spatial context, which is critical for accurately analyzing anatomical continuity and pathological structures across adjacent slices in neuroimaging data.

Recent architectural innovations have significantly advanced CNN-based segmentation performance in neuroimaging through sophisticated feature refinement and multi-scale processing. Convolutional Block Attention Modules (CBAM) [35,36,37] enhance segmentation precision by sequentially applying channel and spatial attention mechanisms, enabling the model to focus on diagnostically relevant features while suppressing noise - particularly valuable for heterogeneous tumor regions and subtle subcortical boundaries. Parallel developments in inception modules [38,39,40,41] address scale variance through parallel convolutional pathways with differing receptive fields, capturing both local texture details and global anatomical context essential for brain structures.

Building on prior work [40], this study applies CNNs to brainstem segmentation in MRI, with a focus on mesencephalon, pons, and medulla volumetric changes. The main advancement over the previous architecture is the incorporation of CBAM modules, which greatly improves the model’s accuracy, allowing to accurately segment brainstem structures while maintaining a small number of parameters. To date, no studies have employed deep learning to compare brainstem atrophy between symptomatic SCA2 patients, preclinical carriers, and controls in Cuba. This approach aims to establish a scalable, objective tool for identifying early biomarkers of SCA2 progression.

2. Materials and Methods

The proposed model architecture builds upon the method described in [40], utilizing a 3D U-Net-like framework to perform volumetric segmentations. This design processes the input images in their native 3D spatial context, which allows to preserve anatomical relationships. The main architecture contains four downsampling and upsampling operations. To enhance feature extraction, each convolutional layer was replaced with an inception module [42], which optimizes computational efficiency by processing multi-scale features in parallel. Additionally, skip connections were refined using three consecutive Convolutional Block Attention Modules (CBAM) [43], enabling the model to focus on spatially and channel-wise relevant features in these branches.

The inclusion of inception modules and CBAM in the proposed U-Net variant was motivated by the need to address two key challenges in brainstem segmentation: (1) the multi-scale nature of anatomical features (e.g., the compact mesencephalon vs. the pons and medulla), and (2) the subtle intensity changes between adjacent substructures. Inception modules enable efficient multi-scale feature extraction, while CBAM refines skip connections to prioritize anatomically relevant regions. The overall model and inception architectures are illustrated in Figure 1, with detailed schematics of the CBAM blocks provided in Figure 2.

The study employed a cohort of 42 MRI scans obtained from the Cuban Neurosciences Center. These scans correspond to 25 individuals, comprising five healthy controls, seven preclinical subjects, and 13 SCA2 patients. The participants belong to different age, SARA scores, and years of SCA2 evolution (more information can be found in Table 1). This study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee of the Cuban Center for Neuroscience on November, 2020. Written informed consent was obtained from all subjects involved in the study.

Given hardware limitations, the model was designed to balance computational efficiency with performance, ensuring feasibility on available infrastructure while maintaining robust segmentation accuracy.

2.1. Image Preparation

The full preparation process for one single image can be depicted in Figure 3. All MRI scans underwent preprocessing to ensure consistency and improve segmentation accuracy. First, N4 bias field correction [44] was applied to address intensity inhomogeneities, enhancing image quality for subsequent analysis. Following this, each scan was registered to the ICBM 2009c nonlinear symmetric template [45] using Advanced Normalization Tools (ANTS) [46]. The registration process employed a three-stage approach: rigid, affine, and symmetric normalization (SyN), ensuring optimal alignment of individual scans to the standardized template space. The ICBM 2009c template was selected for its validated ability to represent adult brain anatomy and compatibility with neuroimaging studies [47,48,49]. The hierarchical registration pipeline minimizes anatomical mismatch by progressively aligning scans to the template’s symmetric space, ensuring consistent preprocessing critical for mod generalizability. The Figure 3a shows the original image, and 3b displays the result of N4+MNIregistration.

To optimize computational efficiency, MRI scans were automatically cropped to focus exclusively on the brainstem region. Using training set segmentations as reference, a standardized region of interest (ROI) measuring 80×80×96 voxels was extracted for each scan. This approach reduced the computational load by 93%, decreasing processed volumes from approximately 8.5 million voxels per full scan to ~614,000 voxels. The cropped ROIs enabled efficient model training and inference while preserving all relevant anatomical data for brainstem analysis. The Figure 3d shows the result of the cropping operation. Following the crop, intensity normalization was applied for every image (Figure 3e). The reduced volumes were used as inputs to the 3D U-Net. The Figure 3f shows one fully preprocessed image overlapped with its manual segmentations.

2.2. Analysis Description

The study was implemented in Python 3.9, utilizing TensorFlow [50] and Keras [51] for model development and training. The model was trained over 250 epochs using the Adam optimizer [52] with default parameters, and a constant learning rate of 10^-4. To mitigate overfitting, a dropout rate of 0.2 was applied before the final convolutional layer.

For training, the label maps are converted into C binary channels, and each channel represents a label. At the same time, each voxel can only belong to one label. The loss function used was one minus the average Dice score (DSC) across all channels, and the DSC is computed as mentioned by Han et al. [34].

The experiment was conducted on a computer provided with an Intel Core i5-10500H microprocessor, 16GB RAM, and a NVIDIA RTX 3060 6GB GPU. The dataset was partitioned into 17 images for training, 3 for validation, and 22 for testing. To enhance model generalization, and given the relatively small size of the available cohort, data augmentation techniques were applied during training, including random rotations (-15° to 15°), translations (±15 pixels on each direction), and flipping (probabilistic). No augmentation was applied to the images of the test set.

The proposed method was evaluated against four UNet-based approaches: (1) an upscaled version of the model architecture from [40], (2) the cerebellar parcellation network by Han et al. [34], (3) the brainstem parcellation model by Magnusson et al. [53], and the whole-brain parcellation model used by Nishimaki et al. [54]. The model by Han et al. [34] was constructed using the exact computer code provided by the authors in their paper, while the models proposed by Magnusson et al. [53] and Nishimaki et al. [54] were implemented by the authors of the current research.

To ensure a fair comparison, all models were trained under identical conditions, maintaining consistent training protocols (loss functions, optimizer parameters, regularization strategies, number of epochs, learning rate). Due to computational constraints, the model proposed by Nishimaki et al. [54] was impossible to train with the original filter sizes for each UNet stage. For this approach, the number of filters were changed from [64, 128, 256, 512, 1024] to [32, 64, 128, 256, 512], ensuring a proper fit in our graphics card.

The segmentations generated by the top-performing model were subsequently utilized for volumetric analyses, with regional volumes normalized as a percentage of total intracranial volume (%TICV) to account for interindividual variability in head size. Additionally, we investigated potential clinical and developmental correlations by examining the relationship between brainstem subregion volumes and both age and Scale for the Assessment and Rating of Ataxia (SARA) scores. These analyses aimed to elucidate whether volumetric variations in the medulla, pons, and mesencephalon were associated with neurological function or age-related morphological changes.

3. Results

The evaluation results demonstrate high segmentation accuracy across all regions of interest. Mean Dice similarity coefficients (DSC) exceeded 0.95 for all structures, with the highest score (0.97) achieved for the whole brainstem. The mesencephalon exhibited the lowest mean DSC (0.93), indicating consistent yet slightly reduced performance in this region. These results highlight the model’s robustness and reliability in segmenting brainstem subregions.

Table 2 shows the number of parameters for each model used in the comparison.

Table 3 shows the mean Dice Score for each model evaluated in the test set. The evaluations are present for each brainstem region (medulla, pons and mesencephalon) and full brainstem. The Table 4 shows the results of the Intersection over Union (IoU), Haussdorf Distance (HD95), Specificity, Sensitivity and Precision, calculated for the full brainstem.

Figure 4 presents a qualitative comparison of segmentation results, showing a representative test image alongside its predicted segmentation. The visual assessment of the segmentations generated by the five models reveals a consistent pattern in the spatial distribution of errors. While all models achieve high performance metrics—including Dice Similarity Coefficient (DSC), 95th percentile Hausdorff Distance (HD95), sensitivity, and specificity—qualitative inspection of the multiplanar error maps (Figure 5) indicates that discrepancies predominantly occur along the outer boundaries of the brainstem rather than at the internal interfaces between substructures (i.e., the medulla-pons and pons-mesencephalon borders). This observation suggests that the models excel in delineating the internal architecture of the brainstem, accurately capturing the transitions between adjacent subregions, but exhibit minor inaccuracies in defining the precise outer margins of the brainstem itself.

The concentration of errors along the periphery may reflect inherent challenges in boundary definition due to partial volume effects at the brainstem’s interface with surrounding cerebrospinal fluid or adjacent soft tissues. Additionally, slight variations in image contrast or resolution near the edges could contribute to this phenomenon. Importantly, the robustness of internal parcellation underscores the models’ ability to learn and reproduce the complex anatomical relationships between substructures, which is critical for clinical and research applications. Future work could explore postprocessing refinements or targeted training strategies to further improve boundary precision without compromising the already high accuracy of internal segmentation.

Based on the information provided in Tables 2 to 4, and Figure 4 and Figure 5, the proposed method achieves superior performance while maintaining the lowest computational footprint. This model consistently outperforms or matches competing methods across all evaluated metrics, including DSC, HD95, and IoU. Specifically, it attains the highest DSC for all brainstem substructures (0.96 for the mesencephalon and pons, 0.95 for the medulla, and 0.96 for the full brainstem), demonstrating robust segmentation accuracy. Additionally, its HD95 of 2.71mm indicates precise boundary delineation, comparable to Nishimaki et al.’s marginally better 2.65mm [54], but using a substantially smaller model.

Beyond accuracy, the proposed model stands out for its efficiency. With only 5.25 million parameters, it is significantly leaner than competing models, which range from 8.79 million to 22.6 million parameters. This efficiency does not come at the cost of performance, as the model matches or exceeds the scores in secondary metrics such as sensitivity (0.95) and precision (0.96). In contrast, larger models like Han et al. [34] (21.64M parameters) and Nishimaki et al. [54] (22.6M parameters) offer negligible improvements while demanding far greater computational resources. This makes the proposed approach particularly suitable for real-world clinical settings, where hardware limitations and inference speed are practical concerns.

The implications of these findings are substantial for both researchers and practitioners. For clinicians, the model’s high accuracy ensures reliable segmentation for diagnostic purposes. For researchers, its parameter efficiency translates to faster inference times and lower hardware costs, facilitating the deployment in resource-constrained environments. From a research perspective, this work establishes a new benchmark for balancing performance and efficiency in medical imaging segmentation. The improvements can be attributed to the two key architectural changes made to the U-Net: (1) the integration of attention mechanisms within skip connections to refine feature aggregation, and (2) the replacement of conventional convolutional layers with inception modules to capture multi-scale contextual information more effectively. By addressing limitations related to structural heterogeneity and boundary ambiguity—common challenges in brain MRI segmentation—the method offers improved accuracy and consistency, allowing to produce high-quality segmentation with smaller models. While future studies could explore further optimizations, the current model demonstrates that state-of-the-art accuracy does not come at the expense of larger models.

Quantitative evaluation of computational efficiency revealed segmentation times of <1 second per image when utilizing GPU acceleration (NVIDIA RTX 3060 MOBILE, 6GB GDDR6), while CPU-based processing (Intel Core i5-10500H, 16GB DDR4 RAM) required 210±15 seconds per case. This represents a 60-120× speed improvement compared to manual segmentation protocols while maintaining diagnostic-grade accuracy.

Using the segmentation results for all the images of the initial cohort, volumetric changes were calculated for SCA2 patients, preclinical subjects, and healthy controls. Volumes were normalized as a percentage of the total intracranial volume (% TICV), with TICV computed using ROBEX [55]. Consistent with prior findings by Reetz et al. [56] , our results (Table 5) revealed a progressive volumetric reduction: SCA2 patients exhibited significantly smaller brainstem subregion volumes compared to preclinical subjects, which in turn were reduced relative to healthy controls. These findings validate the model’s ability to detect subtle neuroanatomical changes, reinforcing its utility in both clinical assessment and longitudinal studies of neurodegenerative disorders.

The most pronounced differences were observed in the pons, with mean volumes of 0.47% TICV for patients, 0.76% TICV for preclinical subjects, and 0.82% TICV for controls. Notably, the median volume for controls was nearly double that of patients. Differences between preclinical subjects and controls were less pronounced. In the mesencephalon, mean volumes were 0.40% TICV for patients, 0.44% TICV for preclinical subjects, and 0.48% TICV for controls. The medulla exhibited the smallest volumetric differences, with values of 0.26%, 0.29%, and 0.31% TICV for patients, preclinical subjects, and controls, respectively. At the whole brainstem level, mean volumes were 1.12%, 1.49%, and 1.62% TICV for patients, preclinical subjects, and controls, respectively, further highlighting the progressive nature of brainstem atrophy in SCA2.

Leveraging the automated segmentations generated by our proposed model, we assessed the relationship between brainstem subregion volumes and clinical measures, including the Scale for the Assessment and Rating of Ataxia (SARA) scores and disease duration (Table 6). The analysis included 11 SCA2 patients and 8 preclinical carriers (one patient was excluded due to missing data). For disease duration, correlations were restricted to the patient cohort.

The correlation analysis between brainstem subregion volumes (%TICV) and clinical parameters yielded several key findings. All brainstem subdivisions showed significant negative correlations with SARA scores (pons: r=−0.69, p<0.01; whole brainstem: r=−0.71, p<0.01), indicating that smaller volumes are associated with worse ataxia severity. Notably, the pons demonstrated the strongest association, aligning with its known role in motor coordination. Disease duration correlated most prominently with pons volume (r=−0.55, p<0.05), suggesting this region may serve as biomarker of progression. In contrast, medulla and mesencephalon volumes were not significantly associated with disease duration (p>0.05), highlighting regional selectivity in neurodegeneration. CAG repeat length exhibited consistent negative correlations across all structures (pons: r=−0.56; whole brainstem: r=−0.55; mesencephalon: r=-0.49; both p<0.05), supporting the genetic contribution to volumetric decline. The medulla showed a marginally nonsignificant trend (p=0.06), possibly reflecting smaller effect sizes or sample size limitations.

4. Discussion

This study presented a deep learning-based framework for analyzing MRI scans to quantify volumetric changes in the brainstem of SCA2 patients and preclinical subjects compared to healthy controls. To the best of our knowledge, this represents the first such study conducted in Cuba, addressing a critical need for accessible and efficient tools to study neurodegenerative diseases in resource-constrained settings.

The success of our approach stems from the inherent advantages of the 3D U-Net for medical image segmentation. Unlike classical techniques (e.g., atlas-based or graph-cut methods) that rely on handcrafted features—which often fail to capture complex anatomical variability [57]—CNNs automatically learn discriminative hierarchical features, enabling precise parcellation of challenging structures like the brainstem substructures [58]. Our modified 3D U-Net architecture incorporates two key changes - attention mechanisms in skip connections and inception modules replacing standard convolutions. This advanced and complex architecture achieves expert-level segmentation accuracy (Dice >0.95 for all brainstem substructures). The attention modules enable precise localization of anatomical boundaries by selectively emphasizing relevant spatial features, while the inception modules capture multi-scale contextual information critical for distinguishing between adjacent brainstem subregions.

The results demonstrate that deep learning techniques can effectively characterize brainstem atrophy, enabling rapid differentiation between patients, preclinical subjects, and controls. Our findings demonstrate a robust inverse relationship between pons volume and SARA scores (r=−0.69, p<0.01), underscoring the pons’ pivotal role in SCA2-related motor dysfunction. This correlation suggests that automated brainstem volumetry serve as a biomarker for clinical trials, enabling early intervention in preclinical carriers. Future longitudinal studies will validate these associations and explore multimodal imaging to refine prognostic models. These findings suggest that the proposed framework can be integrated into larger neuroimaging pipelines to assess volumetric changes in SCA2 patients and preclinical subjects. The development of user-friendly software based on this approach could provide clinicians with a powerful tool for rapid diagnosis and monitoring of disease progression. By highlighting key atrophic changes and enabling longitudinal evaluation, such tools could improve patient care and support early intervention strategies.

The computational efficiency of the proposed method offers significant advantages for clinical implementation. While GPU-based segmentation completes in under one second per image - representing an important time reduction compared to manual segmentation – we observed important trade-offs in CPU-based processing (3.5 minutes per case) that warrant discussion. These processing times reflect the brainstem-specific segmentation only, with the full pipeline (including registration) requiring additional computation. The choice between GPU and CPU implementation presents practical considerations: CPU acceleration enables real-time processing for clinical workflows, while CPU processing remains viable for resource-constrained environments at the cost of increased processing time. Importantly, both approaches maintain equivalent segmentation accuracy, with the computational differences arising solely from hardware parallelization capabilities. For large-scale deployments, GPU implementations are recommended when available, as they provide a most balanced combination of speed and precision. The method’s memory requirements make it deployable on most modern medical imaging workstations without specialized hardware.

4.1. Limitations and Future Work

The main limitations of the current work can be resumed in:

a)

Registration dependency: While the hierarchical registration pipeline ensures robust alignment to the ICBM 2009c template, this preprocessing step might introduce critical limitations, such as failure cases (a failure in registration step inevitably conduces to a wrong segmentation) and template bias (the ICBM 2009c may not generalize to other populations, potentially biasing volumetric estimates).

b)

Regional bias due to dataset homogeneity: All the collected data belongs to Cuban individuals, which may limit the generalizability to global SCA2 populations with differing genetic/environmental profiles.

c)

Small cohort size: While the proposed model demonstrates strong performance in the used cohort, deep learning models typically benefit from larger and more diverse datasets to ensure robustness across populations and imaging protocols.

d)

Cross-sectional design: Volumetric differences are reported in a single timepoint, precluding causal inferences about atrophy progression.

To assess these limitations, future research will be mainly oriented to the development of registration-free pipelines, allowing to enhance robustness and generalizability. The use of vision transformers could be a possible path for exploration, but their innate need for computational resources might negatively impact the processing time. Another way to avoid registration could be using a cascaded approach, similar to proposed by Han et al. [34], which uses two consecutive U-Nets for cerebellar localization and latter segmentation. Incrementing the size of the dataset will positively influence the model’s generalizability too. For this purpose, new SCA2 patients and preclinical carriers should be added to the cohort, but also other neurodegenerative diseases can be added (i.e. other types of SCA, Parkinson’s Disease). The use of multimodal imaging should be explored too, incorporating information of multiple modalities (e.g., DTI/fMRI to map microstructural degeneration). Finally, the use of longitudinal MRI data will be explored too, aiming to a more intrinsic correlate between volumetric trajectories and clinical decline.

5. Conclusions

This study introduced a deep learning-based framework to quantify brainstem atrophy in SCA2 patients, preclinical subjects, and healthy controls, representing a pioneering effort in Cuba. By achieving mean Dice scores above 0.96 for the whole brainstem and 0.95 for its subregions, the approach demonstrates high accuracy in detecting significant volumetric differences, particularly in the pons. Also, the experiment showed highly negative correlations between all brainstem structures and SARA scores. These findings highlight the potential of deep learning to address critical gaps in neuroimaging analysis. The method enables rapid, scalable assessments, reducing reliance on time-intensive manual segmentation and supporting earlier diagnosis and monitoring of SCA2. While the framework demonstrates high accuracy, its reliance on registration and homogeneous data limits immediate clinical translation. Future work will prioritize registration-free architectures, multi-center validation, and longitudinal designs to establish causal links between atrophy and symptom progression. By addressing these limitations, we aim to deploy this tool as a scalable solution for neurodegenerative disease monitoring.