Predicting Multiple Sclerosis Disease Activity from Longitudinal MRI and EDSS Data Using Multimodal Deep Learning: A Pilot Study

1. Introduction

Multiple sclerosis (MS) is a chronic autoimmune inflammatory disease of the central nervous system (CNS) that affects both the brain and spinal cord, leading to progressive demyelination and neurodegeneration. The condition exhibits substantial heterogeneity in its clinical manifestations, including visual impairment, sensory disturbances, motor dysfunction, pelvic organ disorders, and cognitive decline [1]. The growing prevalence of MS and its considerable impact on patients’ quality of life, particularly among young and economically active individuals, underscore its clinical and social significance.

According to international epidemiological studies, the global prevalence of multiple sclerosis (MS) is approximately 35.9 cases per 100,000 individuals; however, these rates vary considerably depending on geographic region, ethnic background, and climatic conditions [2]. The highest prevalence has been reported in countries with temperate and northern climates, including Canada, Northern European nations, and Russia [3]. In Europe, the average prevalence of MS exceeds 100 cases per 100,000 population, while in various regions of Russia it ranges from 30 to 80 cases per 100,000 individuals [4].

The onset of multiple sclerosis (MS) typically occurs between the ages of 20 and 40 years, with women being affected approximately twice as often as men [5]. Recent evidence indicates a rising incidence of MS over the past decades, which may be partially attributed to improvements in diagnostic capabilities, as well as to environmental influences, lifestyle changes, and epigenetic mechanisms [6]. MS remains one of the leading causes of neurological disability among young adults, posing a substantial burden on both healthcare systems and society as a whole [7].

These characteristics make multiple sclerosis (MS) not only a medical but also a socio-economic challenge. In the absence of a radical cure, the focus shifts toward early detection, prognosis, and personalized treatment approaches, which necessitate the integration of advanced technologies, including those based on artificial intelligence methods.

Given the heterogeneity of MS clinical progression and the complexity of its pathogenesis, the task of predicting disease development remains of critical importance. The ability to forecast the transition to an active phase or disease progression enables timely adjustment of therapeutic strategies, reduction of disability risks, and improvement of patients’ quality of life.

Currently, several approaches are employed to predict the course of multiple sclerosis (MS):

• Clinical prognostic models based on initial disease manifestations, age-related characteristics, intervals between relapses, and the degree of recovery after each episode. Although these parameters are routinely considered in clinical practice, their predictive accuracy remains limited [8].
• Disability scales, such as the Expanded Disability Status Scale (EDSS), are used to assess a patient’s current functional state. While EDSS dynamics over time can serve as a reference for evaluating disease progression, the scale itself is not designed to predict future relapses or the rate of deterioration [9].
• MRI markers, including the number and volume of new lesions, the presence of active contrast enhancement, and the degree of brain atrophy, offer greater sensitivity to pathomorphological changes and are widely applied for assessing disease activity. However, even with advanced MRI protocols, predictive value remains constrained due to substantial interindividual variability [10].
• Blood and cerebrospinal fluid biomarkers, such as neurofilament light chain (NfL), glymphatic markers, and cytokines, demonstrate correlations with inflammatory activity and neurodegeneration. Nonetheless, their clinical implementation is still limited, primarily due to the lack of validated threshold values [11].

Machine learning and artificial intelligence (AI) methods enable the development of multimodal models that integrate clinical, MRI, and laboratory data. Such models have the potential to uncover nonlinear relationships and predict outcomes with higher accuracy, particularly when large datasets are available [12].

The implementation of prognostic models in clinical practice is essential for advancing personalized therapy. A predicted high risk of relapse may serve as grounds for switching to second line disease-modifying therapies (DMTs), such as natalizumab or alemtuzumab, whereas a favorable prognosis may justify continued observation or a less intensive treatment regimen [13].

Furthermore, prognostication plays a pivotal role in planning non-pharmacological interventions, including neurorehabilitation, cognitive support, and lifestyle adjustments. The earlier information about potential deterioration is obtained, the more effectively preventive strategies can be applied.

Among all available diagnostic modalities, magnetic resonance imaging (MRI) demonstrates the highest sensitivity to structural changes associated with multiple sclerosis. It provides non-invasive visualization of both old and new demyelinating lesions, contrast-enhancing, inflammation, axonal loss, and brain atrophy, thus covering both early and late pathophysiological stages of the disease [14].

In the context of predicting the course of multiple sclerosis (MS), MRI provides important quantitative and qualitative biomarkers, including the number and volume of T2-hyperintense lesions, the appearance of new T1-hypointense “black holes”, the presence of gadolinium-enhancing (Gd+) lesions, and the rate of progressive gray and white matter atrophy. These parameters show robust correlations with clinical disease activity, the rate of progression, and the degree of disability.

Moreover, in the monitoring of DMT efficacy, MRI remains the gold standard for assessing subclinical disease activity, particularly in cases where overt clinical relapses are absent. Many contemporary clinical trials and patient management protocols employ the concept of “no evidence of disease activity” (NEDA), which encompasses not only the absence of clinical relapses and disability progression but also the absence of new or enlarging MRI lesions.

Thus, MRI combines high diagnostic value, prognostic relevance, and versatility as a marker of treatment effectiveness, making it an indispensable component not only of diagnosis but also of longitudinal monitoring in patients with MS. At the same time, the application of artificial intelligence methods to MRI data opens new perspectives for identifying patterns that may escape visual assessment and for developing more accurate individualized prognostic models.

Given the high variability of MS progression and the limited accuracy of traditional prognostic approaches, there is a growing need to introduce novel methods based on the analysis of multimodal data, particularly MRI and clinical disability scales. Modern artificial intelligence techniques, especially deep learning, make it possible to process such data with a high degree of granularity and to detect patterns that are not accessible to conventional visual analysis.

The aim of this study was to develop and validate a deep learning-based model for predicting the probability of transition of multiple sclerosis to an active disease phase within a one-year horizon time horizon, using longitudinal MRI data and temporal dynamics of EDSS scores.

The main results of this study are:

A multimodal deep learning pipeline combining 3D MRI and EDSS subscales is proposed;

5.

Three architectures (ResNet3D, PretrainedResNet2D, and ResNeXt) are compared on a rare clinical dataset;

6.

Transfer learning is demonstrated to compensate for data limitations in predicting multiple sclerosis.

2. Materials and Methods

2.1. Dataset

The study included data from 28 patients with a clinically confirmed diagnosis of multiple sclerosis (according to the McDonald criteria), who had been under observation for a minimum of two years. For each patient, the following data were available: a series of MRI examinations (covering one to four years of follow-up) and Expanded Disability Status Scale (EDSS) scores. The target variable, reflecting the presence of an active disease phase in the subsequent year, was defined as disease activity. Accordingly, the study cohort was stratified by a binary label: activity and remission.

Each observation comprised 3D MRI images (T1-weighted sequence, DICOM format) and tabular data collected in the year of the corresponding examination.

The target variable (class label) for each examination was assigned based on the following screening procedure. The target variable took one of two values: 1 if the patient’s condition deteriorated at the subsequent examination, and 0 if the condition improved or remained stable.

The Expanded Disability Status Scale (EDSS) is a widely used instrument for assessing the severity of disease progression in patients with multiple sclerosis. Patient status is evaluated across seven functional system subscales, each scored either on a range from 0 to 5 or from 0 to 6, along with an eighth subscale scored from 0 to 12. The EDSS encompasses the following functional domains: visual function, brainstem function, pyramidal function, cerebellar function, sensory function, bladder and bowel function, cerebral (cognitive) function, and ambulation. Based on these eight subscales, a composite EDSS score is derived: 0 indicates a neurologically intact individual, while scores ranging from 1.0 to 10.0 in increments of 0.5 reflect varying degrees of neurological impairment.

2.2. Metrics

The following metrics were employed for model evaluation.

Confusion matrix. The confusion matrix provides a structured summary of the model’s predictive performance, characterizing the number of correct and incorrect predictions across all classes (Table 1). All subsequent evaluation metrics are derived from the confusion matrix.

Accuracy. Accuracy is a standard metric for classification problems, computed as the ratio of the number of correct predictions to the total number of predictions. However, in the present study its interpretability is limited due to class imbalance: for instance, a trivial model that predicts only the negative class (constant zero) would still achieve an accuracy of approximately 0.9:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+FN+TN+FP}}$$

Precision. Precision is defined as the proportion of instances predicted as positive that are in fact positive:

$$Precision={\displaystyle \frac{TP}{TP+FP}}.$$

Recall. Recall is defined as the proportion of truly positive instances that are correctly identified by the classifier as positive:

$$Recall={\displaystyle \frac{TP}{TP+FN}}.$$

F₁-score. The F1-score is the harmonic mean of precision and recall and thus provides a single summary metric that balances both components:

$$F_1=2{\displaystyle \frac{Recall\ast Precision}{Recall+Precision}}.$$

F_β-score. The F_β -score is a weighted harmonic mean of precision and recall. It quantifies classifier performance when recall is considered β times more important than precision. It reduces to the F1-score for β = 1.

$$F_{\beta }={\displaystyle \frac{\left({\beta }^2+1\right)\left(Recall\mathrm{*}Precision\right)}{Recall+{\beta }^{2}Precision}}.$$

2.3. Data Preprocessing

The MRI data exhibited several characteristics that necessitated careful preprocessing:

Color encoding. Unlike the standard RGB color model, in which pixel intensity values are bounded within the range of 0 to 255, MRI data may contain substantially higher pixel intensity values, with the maximum varying across different scans (exceeding 3000 in some cases). Consequently, pixel intensity values were normalized to the range of 0 to 255 for each MRI scan individually.

T1- and T2-weighted images. T1- and T2-weighted images represent the two primary acquisition types in magnetic resonance imaging, differing in the mechanism of contrast generation due to variations in the physical properties of tissues. As a result, different tissue types may appear with distinct signal characteristics (see Table 2). Each MRI examination contains multiple images of the same brain acquired at the same time point but under different pulse sequences: some correspond to T1-weighted variants, while others correspond to T2-weighted variants. Given that T1-weighted images are more numerous across the dataset and that T2-weighted sequences are not consistently available for all scans — whereas T1-weighted sequences are present in all cases — T1-weighted images were selected as the primary input modality. Accordingly, only the T1-weighted slices from each scan were passed to the model.

Image dimensions. Each MRI examination can be conceptualized as a stack of sequentially arranged two-dimensional axial slices. Across the dataset, the number of slices per examination varied, with the total range spanning from 30 to 60 layers. This variability necessitated standardization to a uniform depth. A fixed depth of 50 axial slices was adopted, as this value minimized the number of examinations affected by information loss due to slice reduction. For MRI volumes containing fewer than 50 slices, zero-padded images were appended symmetrically at both the superior and inferior ends to reach the target depth. In addition, each individual slice was spatially resampled to a standardized in-plane resolution of 128 × 128 pixels.

As a result, the data for each observation of each patient consisted of a sequence of MRI examinations, EDSS scores, and the corresponding disease status (Table 3). Data preprocessing was performed for both MRI and EDSS data.

For the MRI data, 50 axial slices were extracted from each T1-weighted sequence per patient, uniformly distributed along the anatomical axis. When the number of available slices was insufficient, zero-filled (empty) slices were symmetrically added; when the number exceeded 50, slices were subsampled at regular intervals. All images were resampled to a spatial resolution of 128 × 128 pixels, and voxel intensities were normalized to the range 0–255. In this way, 3D tensors of size [50 × 128 × 128] were constructed and then aggregated across follow-up years into 4D sequences.

EDSS data were preprocessed as follows. Instead of using the composite EDSS score, we employed its components, namely the seven functional system subscores, each of which reflects the severity of a specific functional domain (e.g., visual function, sensory deficits, bladder and bowel function). This approach helped to avoid excessive collinearity between features and to capture more differentiated patterns of disability.

The subscores were normalized and represented as a temporal sequence aligned with each year of follow-up.

The target variable for each observation was defined as the Status (Active/Not active) at the subsequent visit, as the model was designed to predict future disease activity. The resulting dataset was imbalanced: cases corresponding to an active disease phase accounted for approximately 10% of all observations. To mitigate this issue, we applied filtering of observations with an excessive number of negative labels, performed data augmentation by adding Gaussian noise to 20% of MRI volumes corresponding to active MS, and excluded incomplete or corrupted images. As a result, a total of 67 training samples (annual observations) were obtained, each comprising a 3D MRI volume, the EDSS subscore vector, and the activity label.

3. Model Architectures

A series of multimodal deep neural network architectures were trained and validated, from which three models with the best performance metrics were selected: ResNet3D, PretrainedResNet2D, and ResNeXt. Each of these models employs a distinct strategy for processing imaging data and tabular features. The primary criterion for model selection was their ability to predict the onset of an active phase of MS within a one-year horizon based on MRI examinations and EDSS data.

3.1. ResNet3D Architecture

The ResNet3D model is based on the standard Residual Network (ResNet) architecture introduced by He et al. in 2015 [15]. The central idea of the original architecture is the use of skip connections between layers, which facilitate the optimization of deep networks and improve training stability.

The modification of ResNet3D proposed in this study is tailored to the following assumptions. First, the model must operate directly on three-dimensional image data. Second, it should apply convolutions along the slice axis without aggressively reducing its resolution, thereby preserving the spatial context across slices.

The proposed 3D-ResNet architecture is depicted in Figure 2. Each MRI examination is first fed into a 3D convolutional layer with a kernel size of 7 × 7 × 7, followed by normalization, a ReLU activation function, and a max-pooling operation. The resulting feature maps are then passed through a stack of 3D residual blocks (ResidualBlock3D), followed by average pooling. The pooled features are flattened into a one-dimensional vector, concatenated with the EDSS subscores, and subsequently passed to a recurrent neural network, which produces the final prediction outputs.

The architecture is composed of ResidualBlock3D units with explicit skip connections (Figure 3). When the convolution stride is equal to 1 and the number of channels remains unchanged, the data are processed using the second configuration (Way 2); otherwise, the first configuration (Way 1) is applied. In Way 1, the main branch first passes through a 3D convolution with a kernel size of 3 × 3 × 3 and stride (1, s, s), which maps m input channels to n output channels (where s, m, and n are block hyperparameters), followed by normalization, a ReLU activation, and a second 3D convolution with a 3 × 3 × 3 kernel, unit stride (1, 1, 1), and unchanged channel dimensionality, again followed by normalization. In the skip branch, the input undergoes a 1 × 1 × 1 convolution with stride (1, s, s) to project from m to n channels, followed by normalization. The outputs of the main and skip branches are then added element-wise and passed through a final ReLU activation. In Way 2, the skip branch remains an identity mapping, and the input is forwarded unchanged along the residual connection.

After the 3D-ResNet component, which processes only the MRI volumes, the resulting feature representation is concatenated with the EDSS scores and fed into a recurrent neural network (Figure 2). It should be noted that, in all models considered in this work, a standard recurrent neural network (RNN) architecture is employed. This choice is motivated by the fact that more complex recurrent architectures, such as LSTMs or GRUs, are primarily advantageous for modeling long sequences, whereas in the present study all temporal sequences are limited to a maximum length of five time points.

3.2. PretrainedResNet2D Architecture

The second model in this study leverages transfer learning techniques. To enable the use of pretrained two-dimensional architectures, we employ an embedding procedure that maps three-dimensional MRI volumes into a two-dimensional representation. In this approach, a small convolutional network (Figure 4) first reshapes the 3D data into a 2D layout and then applies a 2D convolution with a kernel size of 1 × 1 and stride (1, 1), transforming 60 input channels into 3 output channels.

The core of the model is a pretrained ResNet18 network, chosen as a backbone due to its favorable trade-off between efficiency and model complexity. The model processes each MRI examination through the embedding module and then feeds the resulting 2D representation into ResNet18. The extracted features are subsequently concatenated with the EDSS scores and passed to a recurrent neural network, which produces the final predictions (Figure 5).

3.3. ResNeXt Architecture

To address the prediction task, we employed a ResNeXt-based architecture [17], illustrated in Figure 6. In this architecture, the input to the convolutional block, consisting of $C_{in}$ channels, is processed by $b$ parallel convolutional branches with 1 × 1 kernels, each mapping the input to an intermediate representation with $C_{mid}$ channels. Each of these intermediate feature maps is then passed through an additional convolution that preserves the channel dimensionality. The resulting $b$ feature maps are concatenated along the channel dimension to form a single tensor with $b\cdot C_{mid}$ channels, which is subsequently transformed back to the original channel dimensionality by a 1 × 1 convolution and added to the block input via a residual connection.

The standard ResNeXt implementation was further modified by incorporating ReZero-based skip connections [18], applying an attention mechanism to the output of the convolutional component [19], and implementing the model in accordance with SNN design principles [20]:

• The inputs to the model are normalized to have zero mean and unit variance.
• The scaled exponential linear unit (SELU) activation function is employed.
• Batch normalization is not used.
• Network weights are initialized from a normal distribution with zero mean and variance equal to 1/N, where N denotes the number of incoming connections.
• Instead of standard dropout, alpha-dropout is applied; rather than setting units to zero at random, it randomly sets them to the limiting value of SELU(x) as x→−∞ which equals −λα.

4. Model Training and Evaluation Metrics

All three proposed models were trained using the following computational resources: an Intel Core i9 CPU and an NVIDIA GeForce RTX 4060 Ti GPU with 8 GB of memory. The target value for the F1-score metric was set to 0.7; once this threshold was exceeded, the model was considered to perform sufficiently well in terms of prediction quality

For training each of the models, the CrossEntropyLoss function was employed in combination with the Softmax activation function. The latter is used to transform the vector of raw prediction logits into a probability distribution over the possible classes, whereas the former quantifies the error of these probabilistic predictions.

The definition of the Softmax activation function is:

$${Softmax\left(y\right)}_i={\displaystyle \frac{e^{y_i}}{\sum _j{e}^{y_j}}}$$

The CrossEntropyLoss function:

$$Loss=-\sum _i{y}_i\mathrm{*}\log {\widehat{y}}_i$$

The 3D-ResNet model was trained for 120 epochs using the Adam optimizer with a learning rate of 1∙10^-4 . The loss function was defined as a weighted CrossEntropyLoss with class weights of 1:3.5 to compensate for the dataset imbalance. This choice of class weights was empirically determined to yield the most effective model performance during experimentation.

The model’s precision exceeded 0.70; however, the F1-score remained below the predefined target threshold of 0.7 (Table 4). Error analysis indicated a tendency toward overfitting, particularly in the presence of non-informative or augmented sequences. This behavior can be attributed to the limited size of the training set and the model’s sensitivity to class balance.

Figure 7. Confusion matrix for the ResNet3D model on the test set.

Figure 7. Confusion matrix for the ResNet3D model on the test set.

The PretrainedResNet2D model was trained for 120 epochs using the same hyperparameters as the 3D-ResNet model and demonstrated stable convergence and strong generalization performance. The achieved F1-score exceeded the target threshold by 0.1 (Table 5), and the balance between precision and recall indicates high model reliability. This approach effectively leverages the advantages of transfer learning, reducing the need for large training datasets in this setting.

Figure 8. Confusion matrix for the PretrainedResNet2D model on the test set.

Figure 8. Confusion matrix for the PretrainedResNet2D model on the test set.

The ResNeXt model was trained for 23 epochs using the Nadam optimizer, which extends the standard Adam algorithm by incorporating Nesterov momentum, with the following hyperparameters: a learning rate of 1⋅10⁻³ and a weight decay of 0.05. As in the previous experiments, the loss function was defined as a weighted CrossEntropyLoss with class weights of 1:3.5.

The ResNeXt model also achieved an F1-score exceeding the predefined target value (Table 6). Although its performance metrics were slightly inferior to those of the PretrainedResNet2D model, it should be noted that no transfer learning was applied in this architecture, indicating that its potential for further performance improvement has not yet been fully exploited.

Figure 9. Confusion matrix for the ResNeXt model on the test set.

Figure 9. Confusion matrix for the ResNeXt model on the test set.

5. Discussion

The results of this study confirm the promise of deep learning methods for predicting multiple sclerosis (MS) activity over a one-year horizon using multimodal data. Within the proposed framework, we evaluated three neural network architectures, including 3D convolutional models and architectures combining embeddings with pretrained backbones. The performance metrics of these models are summarized in Table 7.

The best performance was achieved by the PretrainedResNet2D model, which is based on a pretrained 2D ResNet receiving 2D embeddings derived from 3D MRI data. This finding supports the hypothesis that transfer learning and pretrained convolutional layers can effectively compensate for limited training data in medical imaging tasks. The obtained F1-score of 0.80 exceeds the predefined target threshold and indicates both high sensitivity and high precision of the model.

The 3D-ResNet model demonstrated acceptable, though less compelling, performance. This is likely due to the fact that a fully trainable 3D network typically requires a substantially larger amount of data than was available in the present study. In turn, the ResNetX -based model with self-normalizing layers did not achieve adequate sensitivity, which makes it of limited utility in a clinical context where failing to detect an active disease phase is particularly undesirable.

It is important to note that this study has several limitations.

Small sample size. The training cohort comprised 28 patients, which constrains the generalizability of the models.

Restricted data modalities. Only T1-weighted MRI sequences were used, without additional modalities such as FLAIR, T2-weighted, or contrast-enhanced images.

Target definition. The binary annual activity label may not fully capture the complexity and heterogeneity of the clinical course of MS.

Lack of external validation. The models were not evaluated on independent external cohorts, which limits the robustness and real-world applicability of the findings.

6. Conclusion

This pilot study demonstrates the feasibility of multimodal deep learning for predicting MS disease activity. Among the three evaluated architectures, PretrainedResNet2D achieved the highest performance (F1 = 0.80), outperforming fully trainable ResNet3D (F1 = 0.62), confirming that transfer learning effectively compensates for limited data availability in rare clinical settings.

The development of algorithms for predicting MS activity has important practical implications. A model capable of estimating the probability of transition to an active disease phase in advance can:

• Enable timely revision of therapy, including escalation to more aggressive DMTs.
• Improve the efficiency of scheduling MRI follow-up and other diagnostic assessments.
• Serve as a tool for patient stratification in clinical trials.
• Function as a component of clinical decision support systems in neurological practice.

Thus, the proposed approach may be valuable not only for personalized medicine, but also as an integral element of a digital clinic infrastructure. The study demonstrates that neural network models trained on MRI data and EDSS scores can be successfully used to predict multiple sclerosis activity with high accuracy. The proposed methodology shows strong potential for clinical integration and provides a foundation for the development of digital assistants to support the management of patients with MS.

Building on the findings of this study, future work should prioritize expanding the patient cohort and validating the proposed models on independent external datasets to enhance robustness and generalizability. Another key direction is the integration of additional MRI modalities (such as FLAIR, T2-weighted, and contrast-enhanced sequences) and complementary biomarkers, which may allow a more comprehensive characterization of MS pathophysiology and disease activity. From a methodological perspective, further improvements could be achieved by investigating advanced multimodal fusion strategies and uncertainty-aware prediction frameworks to increase the reliability of model outputs in clinically critical scenarios. Finally, prospective clinical studies are required to assess the real-world utility of the proposed models when embedded into clinical decision support systems and digital patient pathways for MS management.