2. Materials and Methods
This section delineates the methodological framework established to systematically evaluate multi-sequence shoulder MRI and clinical metadata fusion for rotator cuff tear classification. It describes the retrospective cohort definition, ethical considerations, and parameters governing multiplanar MRI acquisitions. Furthermore, it details the problem formulation, rigorous image preprocessing, slice standardization techniques, data augmentation strategies, and patient-level cross-validation protocols. Finally, it presents the specific deep learning architectures (ResNet50 and Vision Transformers), structured clinical metadata encoding pipelines, multimodal fusion strategies, optimization schedules, and visual explainability methodologies in accordance with transparent medical AI reporting standards.
The global experimental architecture and the five-stage procedural flow of this study are illustrated in
Figure 1. To preserve a clear structural connection between the subheadings and the operational flow depicted in the schematic, detailed descriptions of these stages are organized and given as follows:
Stage 1 (Cohort Assembly): Detailed patient baseline and ethical parameters are provided in
Section 2.1.
Stage 2 (Multimodal Inputs): Technical MRI acquisition profiles and structured metadata encoding pipelines are described in
Section 2.2 and
Section 2.3.
Stage 4 (Empirical Best Model Topology): Multimodal fusion strategies, regularization schedules, and optimization details are evaluated in
Section 2.9 and
Section 2.10.
Stage 5 (Patient-Level Evaluation): Cross-validation protocols, statistical output metrics, explainability mapping, and reporting standards are presented in
Section 2.6,
Section 2.12 and
Section 2.13, and 2.14.
2.1. Study Design, Ethical Context, and Cohort Definition
This retrospective, single-center study included 199 patients who underwent shoulder MRI and had binary labels for rotator cuff tear classification. The positive class was defined as the presence of a rotator cuff tear, and the negative class was defined as the absence of a tear. The cohort comprised 100 tear cases and 99 no-tear cases. The mean age of the cohort was
years, and the mean body mass index (BMI) was
. The cohort included 118 female and 81 male patients. Laterality was right-sided in 74 patients, left-sided in 56 patients, and unspecified in 69 patients. MRI examinations were performed in either 2024 (
) or 2025 (
). Detailed demographic and clinical characteristics are provided in
Table 1.
The study was designed as a patient-level diagnostic modeling study. To prevent data leakage arising from slice-level correlation, all slices belonging to a single patient were strictly confined to the same cross-validation fold. This partitioning strategy is critical because shoulder MRI examinations contain multiple adjacent slices from the same anatomical volume; if slices from the same patient appear across both training and test partitions, model performance metrics may be artificially inflated.
The study was conducted in accordance with the Declaration of Helsinki. The research protocol, entitled “Multimodal Diagnosis and Virtual Assistant Use in Magnetic Resonance Imaging for Rotator Cuff Tear Diagnosis,” was reviewed and approved by the Non-Interventional Clinical Research Ethics Committee of Bilecik Şeyh Edebali University (Approval Date: 30 October 2025; Meeting Number: 2025/10; Decision Number: 9).
2.2. MRI Acquisition and Sequence-Level Inputs
For each patient, four distinct MRI sequences were evaluated: T1-weighted turbo spin-echo (TSE) coronal, T2-weighted TSE fat-suppressed sagittal, proton density (PD)-weighted TSE fat-suppressed coronal, and PD-weighted TSE fat-suppressed axial/transverse imaging. Acquisition metadata indicated a
image matrix across all four sequence groups. The mean repetition time (TR) and echo time (TE) were approximately 669.2/20.1 ms for T1 TSE coronal, 4258.2/71.0 ms for T2 TSE fat-suppressed sagittal, 3115.7/34.1 ms for PD TSE fat-suppressed coronal, and 3137.3/26.0 ms for PD TSE fat-suppressed axial/transverse imaging. The mean slice thickness was approximately 4.3 mm across all sequences. These acquisition parameters are summarized in
Table 2.
For concise reporting, these sequences are abbreviated throughout the manuscript as T1, T2, PDc, and PDt, respectively. The distinction between sequence identity (contrast weighting) and anatomical plane was rigorously preserved throughout the analysis, as deep learning models can exploit both contrast-specific and plane-specific features. Consequently, the sequence-combination analysis treats each sequence-plane acquisition as a distinct, independent input channel at the examination level, rather than assuming that all MRI series are interchangeable.
2.3. Structured Clinical Metadata
Structured clinical metadata included age, sex, weight, height, BMI, laterality, and acquisition study year. The metadata pipeline ingested nine numeric inputs: age, sex, weight (in kilograms), height (in meters), BMI, right laterality, left laterality, unspecified laterality, and study year. Continuous variables were scaled using min-max normalization, where parameters were fitted exclusively on the training set within each fold and subsequently applied to the corresponding validation and test sets. Laterality was one-hot encoded into three dimensions (right, left, or unspecified). This fold-specific preprocessing strategy was strictly enforced to prevent validation or test information from leaking into the training pipeline.
Metadata was treated as a distinct, separate modality rather than a surrogate for imaging. A metadata-only model baseline was established to quantify the baseline diagnostic signal that could be extracted from structured variables alone. In multimodal configurations, the clinical metadata vector was encoded by a small multilayer perceptron (MLP) and either concatenated with image-derived features or converted into an independent decision component, depending on the fusion architecture. This design permitted an empirical evaluation of whether metadata improves specific sequence combinations, rather than merely reporting a single global image-plus-metadata result.
2.4. Problem Formulation
The diagnostic task was formulated as a patient-level binary classification problem leveraging a set of sequence-specific MRI stacks and a structured metadata vector. For patient
, the multi-sequence MRI input is represented as:
where
denotes an individual MRI sequence from the sequence set
, and
for
denotes the chronologically ordered stack of slices for that specific sequence
. Each individual slice
is a two-dimensional grayscale MRI image resized and formatted as a three-channel tensor to ensure compatibility with ImageNet-pretrained backbones. The patient-level ground-truth label is defined as
, where 1 denotes a rotator cuff tear and 0 denotes no tear. The structured clinical features are represented as a metadata vector
.
The optimization objective is to learn a mapping , where is the predicted probability of a rotator cuff tear.
In Study 0, contains only a single sequence, and the model estimates from a single image stack or from the metadata vector alone.
In Study 1, encompasses all four MRI sequences alongside metadata to determine the optimal architectural fusion strategy.
In Study 2, comprises a systematically selected subset of two, three, or four MRI sequences evaluated across matched metadata-free and metadata-enhanced variants. This formulation treats the sequence selection configuration itself as an explicit experimental variable.
2.5. Image Loading, Normalization, and Slice Standardization
MRI slices were parsed using a slice-level metadata index that mapped patient identifiers, binary labels, sequence definitions, instance numbers, and image file paths. For each patient and sequence, slices were sorted by instance number to preserve true anatomical ordering. Images were normalized and resized to pixels using bilinear interpolation. To satisfy fixed-length batch requirements, each sequence stack was standardized to exactly 20 slices via center-padding or center-trimming. Center-trimming was executed if the original stack exceeded 20 slices, while symmetric zero-padding was applied if the stack contained fewer than 20 slices. A binary slice validity mask was retained to differentiate real anatomical images from padded positions.
This slice mask is central to the pipeline's engineering implementation. Let
be the feature vector extracted from slice
of sequence
, and let
be the corresponding binary validity indicator (
for real slices,
for padded frames). The aggregated sequence-level representation
was computed via masked averaging:
This masking formulation prevents zero-padded slices from corrupting the sequence-level representation and allows a uniform architecture to process variable slice counts within a batch.
During training, data augmentation was applied exclusively to valid anatomical slices. The augmentation pipeline included horizontal flipping (probability = 0.5), random rotation (), spatial scaling (0.9 to 1.1), shear transforms (up to ), intensity shifting (up to 0.1), and the injection of Gaussian noise (). Augmentations were restricted entirely to the training sets and were never applied to validation or test folds.
2.6. Patient-Level Cross-Validation and Data Loading
All experiments were evaluated using a stratified three-fold outer cross-validation scheme initialized with random seed 42. Within each outer split, a stratified inner validation partition comprising 15% of the training pool was utilized for early stopping and hyperparameter optimization. The held-out test folds contained 67, 66, and 66 patients, respectively, ensuring that all 199 patients were evaluated exactly once across the three outer folds. The target class was the presence of a rotator cuff tear. Identical fold boundaries were maintained across all model families and sequence combinations to ensure paired, statistically rigorous performance comparisons.
Class imbalances were addressed at both the data-loading and loss-function levels. Image models integrated a weighted random sampler wherein individual training samples were weighted inversely to their respective class frequencies within that specific training fold. Furthermore, the positive-class weight term for the binary cross-entropy loss function was dynamically calculated per training fold as the exact ratio of negative to positive cases. These dual mitigation strategies were implemented because small, fold-level class variations can destabilize decision thresholds, particularly when optimizing high-dimensional imaging models on relatively small patient cohorts.
2.7. Backbone Encoders and Patient-Level Representations
The primary convolutional neural network backbone evaluated was an ImageNet-pretrained ResNet50 architecture [
41]. For each slice, the final fully connected classification layer was removed, enabling the convolutional feature extractor to yield a 2048-dimensional feature vector. For a given MRI sequence, slice-level features were aggregated using the previously defined masked mean pooling to derive a singular sequence-level vector. Single-sequence ResNet variants utilized this vector directly (or concatenated it with metadata representations when applicable). Multi-sequence ResNet architectures dedicated an independent sequence encoder to each respective MRI series, subsequently merging the sequence-level vectors according to the specified fusion paradigm.
The Vision Transformer comparator leveraged a pretrained vit_base_patch16_224 backbone [
42]. Each
pixel slice was tokenized into
patches, projected into a 768-dimensional embedding space, and processed through the transformer blocks. To construct a patient-level representation, individual slice embeddings were passed through a temporal sequence-encoding module and aggregated using a class-token ([CLS]) representation. Sequence-specific learnable embeddings were integrated to preserve MRI sequence identity when multiple sequences were processed simultaneously. The implementation adheres to established deep learning guidelines in medical imaging AI, including transfer learning, strict patient-level validation tracking, and reproducible reporting of preprocessing configurations [
43,
44]. The Vision Transformer models serve exclusively as a comparator framework in Study 0 and Study 1 rather than being presented as a novel architectural contribution.
2.8. Metadata Encoder and Classification Heads
The metadata encoder consisted of a two-layer MLP. The raw nine-dimensional metadata vector was projected to a 64-dimensional hidden layer, followed by layer normalization and a Gaussian Error Linear Unit (GELU) activation function, and then projected to a final 128-dimensional metadata representation with an additional layer normalization and GELU activation. The standalone metadata baseline model utilized a separate MLP architecture configured with hidden dimensions of 64 and 32, layer normalization, GELU activations, a dropout rate of 0.3, and a final scalar logit output.
Image-based classification heads were constructed using fully connected layers combined with layer normalization, GELU activations, and dropout regularizers. The default classification head projected fused representations through hidden dimensions of 256 and 64 prior to computing the final binary logit. For models executing explicit feature concatenation across multiple parallel sequences, an expanded classification head with hidden dimensions of 512 and 128 was utilized to accommodate the substantially increased input feature dimensionality. This architectural adaptation is necessary because feature concatenation preserves high-dimensional joint spaces, whereas late decision fusion merges low-dimensional scalar logits, thereby requiring fewer trainable parameters at the final fusion layer.
2.9. Multimodal Fusion Strategies
Study 1 systematically compared several fusion methodologies leveraging all four MRI sequences alongside clinical metadata.
Late decision-level fusion first generated independent logits for each individual MRI sequence and, where applicable, a separate logit from the clinical metadata vector. The unified final logit was computed as a learned, softmax-weighted combination:
where the attention vector
is derived via
, and
indexes the respective sequence and metadata pathways. This configuration isolates modality-specific decision pathways and delays interaction until the logit stage.
where
denotes the encoded metadata vector. This strategy allows multi-sequence feature interactions within the classification head but scales up dimensionality, increasing vulnerability to overfitting on constrained datasets.
Attention-weighted fusion dynamically learned sequence importances from projected feature states using a tanh transformation coupled with a scalar attention scoring mechanism followed by softmax normalization across sequences.
Compact bilinear pooling iteratively fused sequence vectors through randomized count-sketch projections and elementwise multiplication in the Fourier domain, finalized by signed square-root and normalizations.
Early fusion was restricted to structurally compatible paired sequences and concatenated raw image channels prior to ResNet encoding, utilizing a modified six-channel first convolutional layer adapted from the pretrained three-channel weights.
The purpose of these evaluations was to select a highly stable and performant fusion archetype for the subsequent sequence-combination experiments. Late fusion was selected as the core paradigm for Study 2 because it yielded the highest Area Under the Receiver Operating Characteristic curve (ROC-AUC) in Study 1 and exhibited excellent cross-metric stability. The competitive F1-score performance of the feature concatenation architecture was retained during subsequent interpretation to ensure multi-metric validation.
2.10. Optimization, Regularization, and Reproducibility
All deep learning models were trained for a maximum of 30 epochs with an early stopping patience of 7 epochs. To reduce the risk of selecting an epoch with an inflated AUC but poor threshold-dependent behavior, early stopping was governed by a composite validation score:
Label smoothing () was applied during training. The objective function was minimized using binary cross-entropy with logits, integrating the fold-specific positive-class weights.
Optimization was driven by the AdamW optimizer paired with a weight decay of and a cosine annealing warm-restart learning rate scheduler. ResNet models utilized a backbone learning rate of and a classification head learning rate of . Vision Transformer variants utilized a backbone learning rate of and a head learning rate of . ResNet models were trained with a batch size of 8 and a gradient accumulation step of 2, whereas Vision Transformer models used a batch size of 4 and a gradient accumulation step of 4. Model backbones were frozen for the first 2 epochs for ResNet and the first 3 epochs for Vision Transformers. Gradient clipping with a maximum norm of 1.0 was enforced. All cross-validation experiments were explicitly seeded at execution initialization, and deterministic cuDNN execution flags were set to ensure pipeline reproducibility.
2.11. Hierarchical Experimental Design
The research pipeline was partitioned into three sequential, hierarchical experimental blocks, summarized in
Table 3.
Study 0 evaluated each individual MRI sequence independently using both ResNet50 and Vision Transformer backbones alongside the standalone metadata MLP baseline. This block established which singular imaging sequence retained the highest intrinsic diagnostic signal and verified whether the transformer architecture provided an advantage over the convolutional baseline on this cohort size.
Study 1 comprehensively evaluated multimodal fusion strategies using all four MRI sequences combined with clinical metadata. Competing methods included ResNet late decision-level fusion, feature concatenation, attention-weighted fusion, compact bilinear pooling, early channel fusion, and parallel Vision Transformer fusion variants. This block determined the optimal fusion architecture to carry forward into the sequence subset analysis.
Study 2 executed a systematic evaluation of partial MRI sequence combinations with and without clinical metadata. Because single-sequence models were thoroughly charted in Study 0, Study 2 focused specifically on all possible pairwise, three-sequence, and full four-sequence permutations. The incremental value of clinical data was strictly isolated and quantified as the performance delta (, , , and ) between precisely matched metadata-enhanced and metadata-free configurations.
2.12. Performance Metrics, Confidence Intervals, and Statistical Outputs
Model classification performance was evaluated across the three independent patient-level test folds. Computed evaluation metrics included ROC-AUC, average precision (AP), accuracy, sensitivity, specificity, precision, F1-score, the Matthews correlation coefficient (MCC), negative predictive value (NPV), and cross-entropy loss. Threshold-dependent metrics were computed using a fixed decision threshold of 0.5. Specificity was calculated as , sensitivity as , precision as , and NPV as . The MCC was tracked to provide a balanced assessment of binary classification quality that remains robust against fold-level variations in class proportions.
Confidence intervals for the ROC-AUC metrics were generated via a bootstrapping procedure leveraging 200 stratified resamples. Paired ROC-AUC statistical significances were computed using DeLong's test [
45]. Model selection criteria avoided reliance on any singular performance metric; instead, the final selection integrated ROC-AUC, F1-score, sensitivity/specificity balance, precision, MCC stability, and the variance of confidence intervals across the outer validation folds.
2.13. Explainability and Visual Quality Control
Experimental outputs generated by the evaluation pipeline included ROC curves, confusion matrices, fold-level prediction spreads, and explainability galleries. For ResNet-based models, Gradient-weighted Class Activation Mapping (Grad-CAM) visualizations were computed from the final feature maps of the terminal convolutional layer [
46]. For Vision Transformer comparators, attention-based saliency maps were extracted from the self-attention weights. To prevent heatmaps from misallocating importance to uninformative zero-padded background matrices or air artifacts outside the anatomical boundaries, a binary foreground mask was derived directly from the source image and applied to the computed saliency maps. These explainability outputs were interpreted strictly as qualitative architectural checks rather than clinical proof of human-like radiological reasoning.
The primary visual evidence compiled for manuscript reporting consists of the hierarchical experimental workflow diagram, the Study 0 sequence baseline matrix, the Study 1 fusion-strategy comparison chart, the Study 2 sequence-combination performance heatmap, the metadata contribution delta plot, and the comprehensive diagnostic performance panel. While explainability maps offer qualitative support for anatomical plausibility, the core scientific findings of this study rely strictly on patient-level quantitative test fold data.
2.14. Reporting Standards
The structure and reporting of this manuscript align strictly with established medical imaging AI guidelines, including transparent cohort assembly criteria, explicit patient-level split tracking, precise task definitions, fold-specific metric reporting, and a clear disclosure of external validation constraints. The CLAIM (Clinical AI Modeling), TRIPOD+AI, and STARD-AI reporting frameworks were utilized as foundational checklists for diagnostic accuracy and medical AI study execution [
47,
48,
49]. Accordingly, this study should be interpreted as a rigorous model-development and internal-validation effort, rather than a clinical trial of a deployed clinical decision-support system.