Self-Supervised Pretraining Reduces Labeled Data Requirements for Whole-Brain Parcellation of Magnetic Resonance Images

1. Introduction

Whole-brain parcellation can be simply defined as the process in which every voxel of a structural MRI volume is assigned to an anatomical structure, such as the hippocampus, cortex, and so on. This step is an important process in quantitative neuroimaging and in the diagnosis of brain diseases. Volumetric morphometry, atrophy tracking, and most region-of-interest analyses always begin by segmenting the brain into labeled compartments, and almost every downstream measurement depends on this initial labeling being accurate. If any error is introduced at this stage, it propagates into every analysis done on the basis of the initial analysis. Many tools are used for this task, but the most prominent are atlas-based and Bayesian methods, which work by comparing a new scan against a reference atlas of known anatomy and using probabilistic reasoning to assign the most likely label to each voxel [1]. The FreeSurfer aseg pipeline is one of the established tools based on these atlas-based and Bayesian methods and is found to be widely used in brain parcellation [1,2]. Although these tools are accurate and well validated, they are time-consuming, as they take hours of computation per subject’s MRI scan [3].

This computational cost has motivated the search for faster alternatives that are capable of preserving the accuracy at a fraction of the runtime. One of the most widely used alternatives is learning-based models, such as convolutional parcellation networks, which have successfully reduced computation time from hours to about a minute [3]. However, these models require large annotated cohorts. This annotation, or labeling, is done either manually or generated automatically, and in both cases the process is slow, time-consuming, and expensive [1,3]. Even when labeling is automatic, it must still be verified manually. So, although the models themselves are readily available, labeled data remains limited.

A widely adopted alternative to address the scarcity of labeled data in medical imaging is self-supervised learning. In this type of learning, the model learns useful representations from unlabeled data before being trained on a smaller labeled dataset [4,5]. However, it is found that this approach typically relies on large amounts of external unlabeled data collected from different sites [6]. In cases of limited labeled data, unlabeled data, and funding, the performance of this method needs to be determined, as it assumes pretraining on a large amount of data.

The purpose of this study is to address a real-world problem in which a single laboratory can typically assemble only a small number of images, and annotating even a few of them costs a substantial amount of money. As mentioned earlier, a common remedy for this problem is self-supervised pretraining [7], but it is usually evaluated and demonstrated on large external corpora [6]. This paper therefore evaluates masked-reconstruction pretraining [5] under deliberately unfavorable conditions, which we regard as the worst realistic case for this method. The major research question is whether the gains from pretraining justify its additional computational cost and effort.

In this paper, the study is framed as a controlled characterization rather than a method proposal. Every comparison is made on the same subjects and the same folds and differs from its counterpart in exactly one variable, so the question is when and where pretraining helps, not whether a new architecture outperforms existing ones. The main contributions are fourfold. First, a fold-matched label-efficiency study across four annotation budgets in which pretraining is performed per fold on training subjects only, so that the cross-validation estimate remains uncontaminated. Second, evidence that the benefit is concentrated in the low-annotation regime, and that it survives a statistical correction for the dependence between folds only there. Third, a structure-wise analysis showing that the gain is not explained by structure size. Fourth, a subject-level analysis of 148 participants spanning the range from cognitively normal to moderate dementia, showing no detectable dependence of segmentation accuracy on disease severity once scanner is accounted for, a substantially larger effect of acquisition hardware, and no detectable variation of the pretraining benefit with clinical severity.

2. Related Work

2.1. Atlas-Based and Learned Parcellation

The reference approach to whole-brain segmentation is atlas-based. The FreeSurfer aseg pipeline [1] registers each subject to a probabilistic atlas built from manually labeled brains and resolves labels under a Markov random field prior. It remains the de facto standard and, as in the present study, frequently supplies the labels on which learned methods are trained. SAMSEG later removed the dependence on a fixed acquisition protocol [8]. Both are well validated, but the full FreeSurfer pipeline requires hours per subject [3], and the reliability of both varies by structure [2].

Learned replacements descend from the encoder–decoder network of Ronneberger et al. [9] and its volumetric successors [10,11], more recently extended with transformer backbones [12]. nnU-Net [13] showed that a self-configuring pipeline, which adapts preprocessing, network topology, patch size and batch size to each dataset, outperformed most specialized solutions across a wide range of segmentation tasks. QuickNAT [14] trains on labels generated by an existing automated pipeline before fine-tuning on a smaller manually annotated set, and FastSurfer [3] extends this into a complete pipeline. SynthSeg [15] instead trains on images synthesized from label maps and generalizes across acquisition protocols without retraining, while SLANT [16], AssemblyNet [17] and DeepNAT [18] rely on spatially localized networks, large ensembles and hierarchical segmentation, respectively. We do not propose a new architecture: a deliberately standard 3D U-Net is used so that the variable under study is initialization rather than model design.

2.2. Self-Supervised Pretraining for Volumetric Medical Imaging

Models Genesis [4] established image restoration as an effective pretext task for 3D medical volumes. Masked autoencoding [5] showed in natural images that reconstructing a large hidden fraction of the input forces a model to learn semantics rather than local interpolation, and SimMIM [19] showed that a plain pixel-regression head suffices. Tang et al. [6] combined masked inpainting with contrastive and rotation objectives to pretrain the Swin transformer encoder of Swin UNETR [20] on several thousand CT volumes. Contrastive and view-alignment methods [21,22], 3D adaptations of pretext tasks such as rotation prediction and jigsaw puzzles [23], and context restoration [24] form alternative families, and in one small-scale comparison on medical image classification, masked-image pretraining was more robust than contrastive pretraining to reductions in training-set size [7]. Closest to the present work, Zhou et al. [25] pretrained masked autoencoders on the training set of the target task itself and reported gains for medical image classification and segmentation.

The major point that distinguishes the present study from most of this work is scale and provenance rather than method. Most of the works above pretrain on external corpora that are typically much larger than ours, roughly fifty times larger in the case of Tang et al. [6]. Relative to self pre-training [25], we target whole-brain parcellation with a convolutional network, pretrain on approximately 104 volumes per fold drawn from the same cohort that supplies the supervision, and report what that provides at each of four annotation budgets. Because published figures derive from different cohorts, label sets, averaging conventions and reference standards, we draw no direct numerical comparison with them; all comparisons in this paper are internal and fold-matched.

3. Materials and Methods

Figure 1 summarizes the study design. Further implementation detail is given in the Supporting Information (Section S1).

3.1. Dataset and Label Set

We used $N=148$ T1-weighted structural MRI volumes from the third release of the Open Access Series of Imaging Studies (OASIS-3) [26], a longitudinal study of normal aging and Alzheimer disease that extends the earlier cross-sectional OASIS release [27]. The data were obtained from the NITRC Image Repository (NITRC-IR; https://www.nitrc.org, RRID:SCR_003430). We retained a single imaging session per participant, so the 148 volumes correspond to 148 distinct individuals and every cross-validation split is subject-disjoint by construction.

Cohort characteristics are summarized in Table 1. Participants ranged from cognitively normal to moderate dementia: by global Clinical Dementia Rating (CDR) [28], 36 participants ($24.3\%$) carried measurable impairment, so the cohort contains substantial atrophy and ventricular enlargement as well as healthy anatomy. All images were acquired on Siemens scanners, 147 of them at 3 T. The OASIS-3 reference [26] lists the scanner models and sequence types, and per-scan acquisition parameters for all scanners are distributed with the data.

Each volume was accompanied by a whole-brain parcellation produced with the FreeSurfer aseg pipeline [1], which served as the reference standard throughout. This reference is itself the output of an automated method rather than manual expert annotation (Section 6). Images and label maps were supplied in FreeSurfer conformed space, $256\times 256\times 256$ voxels at 1 mm isotropic resolution. We retained 28 classes (27 anatomical structures plus background), chosen so that every lateralized structure appears on both sides: cerebral white matter and cortex, cerebellar white matter and cortex, the lateral, third and fourth ventricles, thalamus, caudate, putamen, pallidum, hippocampus, ventral diencephalon, choroid plexus, and brainstem. Sparse FreeSurfer indices were mapped to contiguous class indices from 0 to 27 through a fixed lookup table, and any voxel carrying another index was assigned to background. The background class was therefore heterogeneous: it contained extracerebral voxels together with all excluded anatomical structures, including the amygdala, accumbens area and inferior lateral ventricle.

3.2. Preprocessing and Network Architecture

Volumes were reoriented to the canonical RAS orientation, resampled to 1 mm isotropic resolution and z-score normalized over non-zero voxels. Foreground cropping to the brain bounding box was applied during training only; evaluation operated on the uncropped volume. During training, ${128}^3$ patches were sampled with an equal probability of centering on a labeled or a background voxel, two patches per volume at a batch size of two volumes. Augmentation comprised random intensity scaling ($\pm 10\%$) and shift ($\pm 0.1$). Random left–right flipping was deliberately not applied: because the label set distinguishes left from right homologues, mirroring a volume without swapping the lateral label pairs supplies anatomically inconsistent supervision. Ninety-degree rotations were likewise excluded.

All experiments used a single residual variant [30] of the 3D U-Net [9,10] implemented in MONAI [31], with five resolution levels (channel widths 32, 64, 128, 256, 320), residual units of two convolutions at each level, PReLU activation and dropout $p=0.1$. Instance normalization [32] was preferred to batch normalization because the small batches imposed by ${128}^3$ patches make batch statistics unreliable [33]. The segmentation network has 12.93 M parameters. The same architecture was used in both stages of the study; only the output layer differs (28 channels for segmentation, one for reconstruction).

3.3. Self-Supervised Pretraining

Pretraining used masked-volume reconstruction, a form of self-supervised learning (SSL). As no external unlabeled corpus was available, pretraining was performed in domain, on the same images later used for fine-tuning but without access to their labels. For each input patch, a binary mask m was formed by drawing 40 axis-aligned cuboids with per-axis extents sampled uniformly from 15–40 % of the patch dimension, at uniformly random positions; their union covers on average 45.8 % of the patch. The network received the masked volume $x\odot (1-m)$ and reconstructed the original intensities, with the loss computed over masked voxels only:

$${\mathcal{L}}_{\mathrm{SSL}}={\displaystyle \frac{{\sum }_v{m}_v|{\widehat{x}}_v-x_v|}{{\sum }_v{m}_v+\epsilon }}.$$

(1)

Block-structured occlusions were preferred to per-voxel dropout because a voxel hidden in isolation is recoverable by local interpolation, whereas the interior of a cuboid about 35 mm across can only be recovered from anatomical context at the scale of the structures to be segmented (Figure 2).

Pretraining ran 150 epochs with AdamW (learning rate $1\times {10}^{-4}$, weight decay $1\times {10}^{-5}$) under a cosine-annealed schedule, drawing two patches per volume at random positions. It was repeated independently for each cross-validation fold, using only that fold’s training split, so that validation and test subjects were never seen; pretraining once on the full cohort would leak test images into the model. At fine-tuning, all parameter tensors whose shapes matched between the two networks were transferred (59 of 63 tensors, or 99.5 % of weights). The final transposed convolution and the output convolution were randomly reinitialized, and no layers were frozen.

3.4. Supervised Fine-Tuning and Label-Efficiency Protocol

Fine-tuning minimized an equally weighted sum of a Dice loss [11], excluding the background channel, and a cross-entropy loss over all 28 channels. Optimization used AdamW [34] (learning rate $2\times {10}^{-4}$, weight decay $1\times {10}^{-5}$) with cosine annealing [35] over 200 epochs and automatic mixed precision [36]. Validation ran every 5 epochs, and the checkpoint with the highest validation Dice was retained for testing. Settings were identical for both initializations, so that initialization is the only variable distinguishing the two arms.

Subjects were partitioned into five folds by a single seeded permutation. For each fold, the held-out fold formed the test set, and 12 % of the remaining training pool was reserved for validation, giving 104–105 training, 14 validation and 29–30 test subjects per fold. Fold assignments were reused unchanged across every condition, so all comparisons are paired at the fold level, and test subjects were unseen at every stage, including pretraining.

Each fold was fine-tuned at four labeled-data budgets, $n\in \{10,25,50,104\}$. The largest budget was set to 104 rather than to each fold’s full training split, so that it is equal in every fold. Subsets were drawn as prefixes of the fold’s randomly permuted training list, so that smaller budgets were strict subsets of larger ones. Every budget was run once from random initialization and once from that fold’s pretrained weights, giving $4\times 2\times 5=40$ fine-tuning runs.

3.5. Evaluation and Statistical Analysis

Predictions were obtained by sliding-window inference (${128}^3$ window, 50 % overlap) followed by a voxelwise argmax over the 28 output channels, with no post-processing, test-time augmentation or ensembling. The Dice similarity coefficient [37] against the FreeSurfer reference was computed per structure and macro-averaged over the 27 foreground classes, so that every structure contributes equally regardless of its volume. Surface-distance metrics proved unstable and are not reported (Supporting Information, Section S3).

Because conditions are paired by fold, comparisons used paired t-tests with 95 % confidence intervals, complemented by Wilcoxon signed-rank tests and paired Cohen’s d. Cross-validation folds share training subjects, which makes the ordinary paired t-test anti-conservative. We therefore also report the corrected resampled t-test of Nadeau and Bengio [38], in the form applied to cross-validation by Bouckaert and Frank [39], which inflates the variance of the mean difference by a factor $\rho =1/k+n_{\mathrm{test}}/n_{\mathrm{train}}$. With $k=5$, $n_{\mathrm{test}}=30$ and $n_{\mathrm{train}}=104$ (the pretraining stage used all training images of the fold at every budget), $\rho =0.489$ against the uncorrected $0.200$. All conclusions are drawn on the corrected basis.

3.6. Subject-Level Analysis

Because every subject appears in exactly one test fold, each of the 148 participants receives one out-of-sample Dice score, produced by the randomly initialized supervised cross-validation run (Section 4.1). We related these scores to CDR, the Mini-Mental State Examination (MMSE) [29], age, sex and scanner model by rank correlation, Mann–Whitney and Kruskal–Wallis tests, and ordinary least squares with all of these factors and test fold as covariates. The single 1.5 T acquisition was excluded from scanner comparisons. Scanner and clinical status are confounded in this cohort: 31.4 % of TrioTim participants were impaired against 10.8 % of Biograph mMR participants (${\chi }^2$ test with Yates correction, $p=0.026$). We therefore separated them by comparing scanners among cognitively normal participants only and disease within TrioTim acquisitions only. To ask whether the benefit of pretraining depends on the participant, we also formed for each subject the paired difference between the pretrained and randomly initialized models at the full budget. These subject-level analyses describe how effects distribute across participants; inferential claims about pretraining rest on the fold-level tests.

3.7. Implementation

All experiments were implemented in PyTorch 2.11 with MONAI 1.7 (development build) and executed on a single NVIDIA A100 (80 GB) GPU, with all randomness derived from one seed (42). Full-budget fine-tuning took roughly 40 minutes per fold and pretraining about 30 minutes. Code, fold definitions and analysis notebooks are available at https://github.com/anupbagale/ssl-brain-parcellation.

4. Results

4.1. Supervised Baseline

Trained from random initialization on each fold’s full training split (104 or 105 subjects), the network achieved a cross-validated mean Dice of $0.7983$ (SD $0.0146$ across folds; fold means 0.7797–0.8163), with no catastrophic failures. This run supplies the subject-level scores of Section 4.4. The randomly initialized $n=104$ arm of the label-efficiency protocol, which fixes the full budget at the same size in every fold, gave $0.7997\pm 0.0194$ (Table 2) and is the reference for all paired comparisons.

4.2. Effect of Self-Supervised Pretraining

Table 2 and Figure 3 report test Dice for both initializations at each budget, and Table 3 gives the paired comparisons. Pretraining conferred its largest advantage when annotation was scarce. With 25 labeled subjects the pretrained model exceeded its randomly initialized counterpart by 0.0610 Dice (95 % CI $[+0.042,+0.080]$, $d=3.91$), and with 10 subjects by 0.0480 ($[+0.020,+0.076]$, $d=2.11$). A pretrained model fine-tuned on 25 subjects thus achieved 95.7 % of the Dice of a randomly initialized model trained on 104. Figure 4 shows both models at this budget on the test subject with the median gain in one fold: most changes were corrections, in particular in a frontal region that the randomly initialized model left partly unlabeled, but pretraining also introduced errors, mainly along the medial cortex. The advantage narrowed as annotation increased, to $+0.0143$ at the full budget ($[+0.003,+0.025]$), about one quarter of the effect at $n=25$.

Under the corrected resampled t-test, the two low-budget results remained significant ($p=0.0390$ at $n=10$ and $p=0.0050$ at $n=25$), whereas the full-budget result did not ($p=0.0802$), and the $n=50$ comparison ($+0.0100$) was null under either test. With a Bonferroni adjustment across the four budgets, only the $n=25$ effect remained significant. We therefore treat the $n=25$ effect as the secure finding of this study, the $n=10$ effect as supported, and the $n=104$ effect as suggestive but not established. At $n=10$, 25 and 104 the Wilcoxon test returned $p=0.0625$, the smallest value attainable with five pairs, which implies that all five folds favored pretraining. Further statistical detail, including the $n=50$ result and run-to-run stability, is given in the Supporting Information (Section S2).

4.3. Structure-Wise Analysis

At $n=25$, the budget with the largest aggregate difference, pretraining improved all 27 structures and degraded none (Table 4, Figure 5). Gains ranged from $+0.1544$ (left lateral ventricle) to $+0.0104$ (brainstem). Structure size did not predict the gain (Spearman $\rho =0.269$, $p=0.175$): both small structures (left caudate, 0.37 % of labeled voxels, $+0.1535$) and large ones (left cerebral cortex, 20.2 %, $+0.0842$) were among the strongest beneficiaries, while the smallest gains occurred in structures of very different size (brainstem, 1.96 %; fourth ventricle, 0.17 %, $+0.0153$). Choroid plexus had the lowest Dice under both schemes (0.47 and 0.55 with pretraining, left and right).

Across hemispheres, eleven of the twelve lateralized structure pairs were segmented more accurately on the right than on the left in the supervised baseline run (mean advantage $+0.029$ Dice; sign test $p=0.006$), and the pattern replicated in both arms at $n=25$. Because mirroring was not used, it cannot originate in that augmentation. The per-pair values and possible origins are given in the Supporting Information (Section S4).

4.4. Performance Across Clinical Severity and Acquisition

Table 5 summarizes the subject-level analysis. Taken at face value, accuracy appeared to fall with disease severity: cognitively normal participants averaged $0.8027$ Dice against $0.7856$ for impaired participants, and Dice correlated weakly but significantly with CDR (Spearman $\rho =-0.169$, $p=0.040$). Accuracy also differed sharply by scanner, from $0.8261$ on Biograph mMR to $0.7868$ on TrioTim (Kruskal–Wallis $p<0.0001$).

The stratified comparisons resolved which of these effects is real. Among cognitively normal participants alone, the scanner difference was nearly unchanged: $0.8273$ on Biograph mMR ($n=33$) against $0.7892$ on TrioTim ($n=70$), a difference of $0.0381$ (Mann–Whitney $p<{10}^{-6}$, Cohen’s $d=0.99$). Within TrioTim acquisitions alone, the disease effect disappeared: $0.7892$ for cognitively normal against $0.7817$ for impaired participants ($p=0.87$). In the multivariable model, Biograph mMR exceeded TrioTim by $0.0313$ Dice ($p<0.001$), whereas CDR ($p=0.068$), MMSE ($p=0.78$), age ($p=0.75$) and sex ($p=0.31$) were not significant ($R^2=0.258$, against $0.128$ without scanner). The apparent effect of disease was therefore the effect of scanner: impaired participants were over-represented on the scanner on which the model performed worse.

The benefit of pretraining did not vary detectably across participant groups. At the full budget the per-subject gain averaged $+0.0142$ (SD $0.0141$) and was positive in 129 of 148 subjects ($87.2\%$). It did not vary with CDR (Spearman $\rho =+0.044$, $p=0.59$), MMSE ($p=0.32$) or age ($p=0.88$), and was nearly identical for cognitively normal ($+0.0139$) and impaired ($+0.0151$) participants ($p=0.70$). Across scanners it was largest on TrioTim ($+0.0159$) and smallest on Biograph mMR ($+0.0102$), a trend that did not reach significance (Kruskal–Wallis $p=0.078$).

5. Discussion

This study asked a deliberately narrow question: on a cohort of the size a single group can realistically annotate, does self-supervised pretraining on those same images justify the effort of running it? The answer is a qualified yes. Masked-volume pretraining gave higher mean Dice at every annotation budget we tested, but the size of the improvement was governed almost entirely by how much annotation was already available: $+0.0610$ Dice at 25 labeled subjects against $+0.0143$ at 104.

5.1. Practical Meaning of the Low-Budget Result

The most important number in Table 2 is not a difference between two models but an equivalence between two annotation regimes: a pretrained network fine-tuned on 25 labeled volumes achieved $95.7\%$ of the Dice of a randomly initialized network trained on 104. For a laboratory deciding how much of a new cohort to send for manual delineation, most of the achievable parcellation quality was recoverable from roughly a quarter of the labeled volumes. Two caveats apply. Every condition also consumed a fully labeled 14-subject validation set, so the true annotation cost was 39 volumes against 118, a saving closer to threefold than fourfold. In addition, the residual $4.3\%$ deficit is not uniform: compared with the supervised baseline run, it falls mainly on choroid plexus ($-0.092$ left, $-0.067$ right), the third ventricle ($-0.088$), the left caudate ($-0.070$), cerebral white matter and the cerebral cortex. A study whose endpoint depends on these structures should check them before treating the two regimes as interchangeable.

5.2. In-Domain Pretraining at This Scale

Self-supervised pretraining is usually motivated by, and demonstrated with, an external unlabeled corpus, often a large one [4,6]. Ours was neither large nor external: approximately 104 volumes per fold, drawn from the same images later used for supervision. That this was sufficient to produce a measurable effect at low budgets is, we think, the result most likely to be useful to other groups, because it removes the precondition that makes self-supervision impractical for many of them. At $n=104$ the pretraining and fine-tuning sets are the same images, so the pretrained model sees no datum the baseline does not. The $+0.0143$ difference there, consistent in direction across all five folds but not significant under the corrected test, can therefore reflect only the point in parameter space from which supervised optimization begins, a representation-quality rather than a data-quantity effect. At the smaller budgets the two effects are superimposed, so the $n=10$ and $n=25$ comparisons are properly read as “n labeled volumes” against “n labeled plus $\approx 104$ unlabeled volumes”.

5.3. What Reconstruction Pretraining Learns

We had expected pretraining to act chiefly as a remedy for structures too small to be learned from labels alone, but the gain was unrelated to structure size (Section 4.3). A more consistent pattern emerges if the structures are ordered by how far their delineation depends on surrounding context. The largest gains accrued to the lateral ventricles, caudate and thalamus, which have stereotyped position, strong intensity contrast against their immediate neighbors, and a boundary largely determined by what surrounds them. Most of these large gains were on the left side, where the randomly initialized model was weakest, so part of the pattern may also reflect the room available for improvement. The smallest gains accrued to the brainstem and fourth ventricle, which a randomly initialized network already segments well. This is what one would expect of an objective that requires the network to infer hidden intensities from visible context, and the reconstructions in Figure 2 fit this reading: inside large hidden regions the pretrained network restores the ventricles and the overall gray–white layout, but not the fine cortical folding. Choroid plexus, which is thin and ambiguously bounded even in the reference, remained far below every other structure. This suggests that reconstruction-based initialization makes better use of well-defined supervision but does not compensate for ambiguity in the reference standard.

5.4. Disease Severity and Acquisition

The subject-level analysis began by suggesting that the model struggles with diseased brains and ended by finding no evidence that it does: the weak association with dementia severity vanished once scanner was held fixed. Only 13 participants had a CDR of 1 or more, so small disease-dependent differences cannot be excluded, but the analysis illustrates why the confound had to be addressed. The scanner effect is the larger finding. The difference between Biograph mMR and TrioTim acquisitions, $0.038$ Dice among cognitively normal participants, is more than twice the benefit of pretraining at the full budget. TrioTim supplied $69\%$ of the cohort and therefore of every training split, so the deficit is not a matter of under-representation. Either its images are intrinsically harder to segment, or the FreeSurfer reference is less reliable on them; because we measure agreement with aseg, both produce the same pattern. Scanner affects both arms of every paired comparison identically, so it does not contaminate the pretraining result, but it does calibrate it: at 25 labeled subjects the benefit of pretraining is about $1.6$ times the scanner difference.

5.5. When the Method Is Worth Applying

Pretraining is not free: it added about 30 minutes per fold to roughly 40 minutes of full-budget fine-tuning, a $\sim 75\%$ increase in training time that at the full budget returned only $+0.0143$ Dice. The trade is different when the pretrained checkpoint is amortized: a single checkpoint per fold served all four label budgets, and the regime in which pretraining pays best, $n=25$, is also the one in which fine-tuning is cheapest. If a group already has labels for its whole cohort, pretraining offers a small and easily outweighed improvement. If it has a cohort of images and is deciding how many to annotate, pretraining on the unlabeled cohort first and annotating roughly a quarter of it (a third if the validation set is counted) is the better allocation of the same total effort.

6. Limitations

Every figure in this paper measures agreement with FreeSurfer aseg output rather than with expert manual delineation, so Dice here quantifies imitation of that pipeline, including its systematic errors, rather than anatomical accuracy. Structures for which the reference is least reliable, choroid plexus above all, are penalized twice, and we cannot determine whether lower agreement on TrioTim images reflects a harder segmentation problem or a less reliable reference. The label set merges structures of direct clinical interest, in particular the amygdala and accumbens area, into background, and our results say nothing about them.

Three features of the protocol affect the label-efficiency curve but not the paired comparisons, because each applies identically to both arms. A fixed 200 epochs means that the number of optimizer updates scales with the budget (roughly 1,000 at $n=10$ against 10,400 at $n=104$); labeled subsets were drawn once per fold; and the 14-subject validation set adds to every nominal budget. Cross-validation folds are not independent, which inflates the type I error of the ordinary paired test [40], and the corrected test is itself approximate, since no universal unbiased estimator of the variance of K-fold cross-validation exists [41]. If the labeled budget rather than 104 is entered as $n_{\mathrm{train}}$, the $n=25$ effect remains significant ($p=0.030$) but the $n=10$ effect does not ($p=0.30$). Each configuration was run once, and one incidental repeat of an identical full-budget configuration differed by $0.028$ Dice, more than the $n=104$ effect, so multi-seed replication is needed to settle it. Finally, 95th-percentile Hausdorff distances were not interpretable without post-processing, so boundary quality, to which Dice is insensitive [42], is not quantified. These points are detailed in the Supporting Information (Sections S2 and S3).

The study uses a single cohort, 148 subjects and one modality, and 147 of the 148 volumes were acquired at 3 T on Siemens hardware, so generalization to other vendors, sites and protocols is untested; in-domain pretraining may be particularly sensitive to such shift. The pretraining corpus was small, and whether the benefit grows or saturates with more unlabeled data is not answerable from these data. No independently published method was evaluated on this cohort and label set, because the study is a controlled comparison of two initializations rather than a method proposal, and we used no test-time augmentation, ensembling or post-processing. The absolute figures are therefore a floor rather than a ceiling.

7. Conclusions

We evaluated masked-volume self-supervised pretraining for whole-brain parcellation under the constraint that most groups actually face: a modest cohort, no external unlabeled corpus, and a labeling budget that is the binding limitation. Across a fold-matched five-fold protocol on 148 T1-weighted volumes, pretraining on each fold’s own training images gave higher mean Dice at every annotation budget tested. The improvement was concentrated where it matters most, with $+0.0610$ Dice at 25 labeled subjects (corrected $p=0.005$) against a non-significant $+0.0143$ at 104, and a pretrained model fine-tuned on 25 labeled volumes recovered $95.7\%$ of the performance of a randomly initialized model trained on 104. The gain was not explained by structure size, which suggests that the pretext task supplies a spatial and contextual prior rather than additional statistical power for rare classes. Segmentation accuracy showed no detectable dependence on dementia severity once acquisition was controlled, while scanner model had a large effect, and the benefit of pretraining did not vary detectably with clinical severity. Future work should include a manually delineated subset, multi-seed replication, folds stratified by scanner and clinical status, and an external cohort from other vendors and sites to test whether in-domain pretraining survives domain shift.