Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability

1. Introduction

Cervical cancer remains a significant global health challenge, in which over 600,000 new cases and 340,000 deaths are reported annually worldwide [1]. The primary strategy for reducing mortality is early detection through Pap smear screening, which enables identification of precancerous lesions before they progress to invasive cancer. However, manual analysis of Pap smear slides is inherently time-consuming, labor-intensive, and subject to substantial interobserver variability, in which diagnostic accuracy varies significantly among cytotechnologists [2,3]. These limitations highlight the critical need for automated, reliable, and interpretable diagnostic solutions that can augment human expertise while maintaining the accuracy essential for clinical decision-making.

Deep learning approaches, particularly Convolutional Neural Networks (CNNs), have revolutionized medical image analysis by providing automated feature extraction and achieving remarkable diagnostic accuracy across various medical imaging modalities [4,5]. CNNs have demonstrated substantial success in cervical cancer detection, which surpass traditional machine learning methods through their ability to learn hierarchical feature representations directly from image data [6,7]. However, CNN architecture faces inherent limitations in medical imaging applications. Their reliance on local receptive fields and hierarchical feature extraction can miss important long-range spatial relationships that may be crucial for accurate diagnosis of cellular abnormalities [8,9]. Additionally, the limited interpretability of CNN decision-making processes poses significant barriers to clinical adoption, where understanding the rationale behind diagnostic predictions is essential for building trust and enabling clinical validation [10,11].

Related automation-oriented work in healthcare has also explored hyperautomation-based leukemia detection and classification [12], supporting the broader relevance of AI-driven clinical decision-support workflows beyond a single disease domain. In a closely related study applying rigorous evaluation protocols to structured clinical data, [13] evaluated hybrid deep learning–gradient boosting ensembles for breast cancer survival prediction using SEER and METABRIC cohorts, finding that under nested cross-validation with Nadeau–Bengio corrected statistical testing, hybrid ensembles did not outperform a class-weighted logistic regression baseline, and that cross-cohort degradation was concentrated in calibration rather than discrimination — underscoring that methodological rigor in evaluation is as consequential as architectural choice across AI-driven clinical applications.

Vision Transformers (ViTs) have emerged as a promising alternative architecture that addresses many limitations of traditional CNNs. By adapting the transformer architecture from natural language processing to computer vision, ViTs [14] leverage self-attention mechanisms to capture global contextual relationships across entire images, which enable more comprehensive analysis of spatial patterns [15]. This global perspective is particularly valuable for medical image analysis, where diagnostic features may be distributed across different regions of an image and require integration of multiple cellular characteristics [16]. Furthermore, the attention mechanisms inherent in ViTs provide natural interpretability through attention maps, which offer insights into which image regions influence diagnostic decisions—a crucial requirement for clinical acceptance and regulatory approval [17]. Recent studies have further validated the effectiveness of Vision Transformers in medical imaging applications, demonstrating improved performance in modeling both global and local context, enhanced computational efficiency, and robustness to limited labeled data [18,19].

Despite the promising potential of ViTs in medical imaging, their application to cervical cancer screening remains underexplored. Existing studies have not systematically examined optimization requirements for ViTs in this domain, nor have they provided the rigorous statistical validation necessary for clinical adoption. Critical gaps remain in the evaluation of data augmentation strategies, class imbalance handling techniques, and hyperparameter optimization tailored for cervical cell classification. Furthermore, the interpretability advantages of ViTs have not been fully investigated in the context of cytopathological analysis, where alignment with established diagnostic criteria is essential.

This study addresses these gaps by presenting a comprehensive evaluation of ViT architectures for automated cervical cancer screening using Pap smear images. The contributions of this study are fourfold: (1) systematic optimization of augmentation strategies and class weighting approaches to address class imbalance while preserving biological validity; (2) rigorous statistical validation through repeated experiments and pairwise comparisons to identify statistically equivalent high-performing configurations; (3) enhanced Gradient-weighted Class Activation Mapping (Grad-CAM) interpretability aligned with cytopathological diagnostic criteria; and (4) demonstration that ViTs can achieve clinically relevant performance while providing the transparency and reliability essential for medical AI applications.

The remainder of this paper is organized as follows: Section 2 reviews prior work on traditional machine learning, deep learning, and emerging transformer-based methods for cervical cancer detection. Section 3 provides a detailed description of the methodology, which includes dataset preparation, model architecture, optimization strategies, and interpretability frameworks. Section 4 presents experimental results and discussion, which include statistical analyses and interpretability findings. Section 5 concludes with a summary of contributions and directions for future research.

2. Related Literature

The application of artificial intelligence techniques in cervical cancer detection and classification has advanced significantly over the past decade, which led to the development of increasingly accurate and automated diagnostic methods. As illustrated in Figure 1, these techniques can be broadly categorized into machine learning and deep learning approaches, in which each category encompasses a range of methodologies tailored to improve diagnostic performance.

2.1. Machine Learning Approaches

Traditional machine learning approaches for cervical cancer detection rely heavily on handcrafted feature extraction followed by classical classification algorithms. This subsection reviews studies that employ various feature-engineering and machine-learning techniques. Support Vector Machine (SVM) and statistical learning approaches have been extensively utilized. Several studies [20,21] developed hybrid linear iterative clustering with Bayes classification, k-Nearest Neighbors (k-NN) with fuzzy logic, and machine learning-assisted detection frameworks, respectively. Mariarputham and Stephen [22] used handcrafted texture features with SVM and neural networks on the Herlev dataset, which found SVM performed best. Optimization-based methods [23] explored quantum hybrid Particle Swarm Optimization (PSO) with fuzzy k-NN for feature selection and cell classification. Cross-validation and prediction-based machine-learning techniques [24,25] investigated stratified k-fold cross-validation, early detection frameworks, and prediction using various machine learning methods. Additional ML approaches addressed Pap smear image classification and early prediction [26,27]. Nandanwar and Dhonde [28] proposed a hybrid stacked ensemble model that combines feature extraction with segmentation, followed by classifier-level fusion using Histogram and Hu Moments techniques.

Traditional machine learning approaches in cervical cancer diagnosis often lack integrated data augmentation strategies, which limits their ability to generalize across diverse image variations. Furthermore, these systems are prone to high false negative rates, which can result in missed diagnoses and pose significant risks in clinical settings. These limitations highlight the need for more robust, automated, and generalizable solutions in medical diagnostics.

2.2. Deep Learning Approaches

Deep learning approaches have demonstrated remarkable success in medical image analysis, particularly for cervical cancer detection and classification. The following review encompasses studies that employ various deep neural network architectures for cervical cancer classification.

CNNs are widely adopted deep learning models known for their ability to extract spatial and hierarchical features from input data, which make them suitable for a variety of pattern recognition tasks. CNN architecture has been extensively employed for cervical cancer detection across multiple imaging modalities. Several studies [29,30] employed various CNN variants that included Mask Region-based Convolutional Neural Networks (R-CNN) with Visual Geometry Group (VGG) and Residual Network (ResNet) components, Capsule Networks (CapsNet) preprocessing with VGG-like networks, specialized CNN architectures using deformable Faster Region-based Convolutional Neural Network with Feature Pyramid Networks (R-CNN-FPN) with pretrained backbones for cervical image analysis, and improved Inception-ResNet-V2. Transfer learning approaches [31] implemented and optimized Squeeze-and-Excitation Residual Network-152 (SE-ResNet152) models for multi-class cervical cancer detection. Recent CNN-based work [32,33] explored privacy-preserved detection, lightweight CNN architectures, ResNet-based automated screening, and Attention-Fused Squeeze-and-Excitation Network (AF-SENet) using pre-trained models for feature fusion. Additional CNN approaches [34,35] investigated various CNN variants with different backbone networks for enhanced performance.

In a closely related cervical cytology study, Albzour and Lam investigated deep-learning-based segmentation and classification of Pap smear images for cervical cancer detection [36], which provides a relevant foundation for the present transformer-based extension.

U-Net is a deep learning architecture originally developed for biomedical applications, which is recognized for its encoder-decoder structure and ability to capture both high-level context and fine-grained details. U-Net variants and advanced architectures have shown exceptional performance in cervical cancer applications. Enhanced U-Net architectures [37,38] incorporated residual SE blocks, feature fusion from multiple pretrained models, and encoder-weighted designs for improved analysis.

Ensemble learning refers to the technique of combining predictions from multiple models to achieve better performance and generalization than any single model alone. Ensemble and hybrid deep learning approaches have emerged to improve robustness and diagnostic accuracy in cervical cancer classification. Several studies have proposed hybrid frameworks that integrate deep feature fusion, pretrained networks, and classifier-level combinations to enhance performance [39,40] . In addition, radiomics-guided deep learning approaches have combined handcrafted feature descriptors with neural network representations to further strengthen feature expressiveness and diagnostic robustness [41].

Following advancements in CNNs, U-Nets, and ensemble learning, transformer-based architectures have recently attracted significant attention due to their capability to capture global contextual relationships [14,42]. Recent studies have further advanced Vision Transformer frameworks through systematic optimization strategies and attention-enhanced architectures to improve classification accuracy, computational efficiency, and contextual feature learning in medical image analysis [43,44]. Vision Transformers have demonstrated strong potential in learning long-range dependencies in medical images, though their performance often benefits from pretraining and large datasets. In a companion benchmarking study, the present authors compared a compact Vision Transformer (ViT-Tiny) against four widely used CNN baselines on the Herlev dataset under a uniform training recipe, finding that the transformer matched the strongest CNNs in accuracy while offering substantial parameter and memory efficiency advantages [45].

Despite their successes, deep learning approaches face several limitations that impact their clinical applicability. Common challenges include class imbalance between normal and abnormal cells, and lack of interpretability tools such as saliency maps or attention visualizations. These issues can reduce model robustness and make predictions less transparent and more difficult to validate for clinical use. This study specifically addresses two critical limitations: class imbalance through systematic optimization of class weighting approaches, and lack of interpretability through comprehensive Grad-CAM visualization that aligns model attention with established cytopathological criteria. Addressing these limitations is essential to improve trust, generalizability, and the practical deployment of AI models in cervical cancer diagnosis.

3. Methodology

Figure 2 provides a comprehensive overview of the entire methodology employed in this study. Each stage plays a crucial role that ensures the effectiveness and robustness of the classification framework, which underscores the systematic approach adopted to tackle the challenges in cervical cancer diagnosis.

3.1. Dataset Preparation

The Herlev Pap Smear dataset, which comprises 917 images, was used and regrouped from seven cytological classes into a binary screening task (normal: 242; abnormal: 675), consistent with clinical triage. This publicly available dataset, introduced by Jantzen et al. [46], is a standard benchmark for cervical cytology classification. The binary reformulation reflects the primary objective of population-level screening, in which the clinically decisive step is separating cases that require cytopathologist review (any abnormal grade) from those that do not (normal), rather than assigning a precise dysplasia grade; the seven-class grading task is therefore treated as a complementary objective and is discussed as a direction for future work in Section 5. Images were inspected for readability and converted to a unified input size compatible with ViT (RGB, standardized resolution). Pixel intensities were normalized using ImageNet statistics to leverage pretrained foundations. Because the Herlev dataset does not provide patient-level subject identifiers, stratified k-fold cross-validation was performed at the image level, with class proportions preserved across folds. The absence of subject-level separation is acknowledged as a limitation in Section 5. All preprocessing and augmentations were applied on the fly within the training pipeline to avoid data leakage.

3.2. Data Augmentation

Augmentations were designed to increase data diversity while preserving cytological validity (cell/nuclear morphology, chromatin patterns). The augmentation policy included the following transformations:

• Geometric: horizontal flip, small rotations, translations, and mild scaling to simulate slide handling variability.
• Photometric: limited brightness/contrast jitter and color perturbations to reflect staining variability.
• Regularization: light Gaussian blur/noise where appropriate. Each transformation used clinically conservative ranges and per-operation probabilities to ensure that augmented images remained biologically plausible. All augmentations were applied on the fly during training (virtual augmentation): each image was transformed stochastically at load time rather than expanded into a fixed enlarged dataset on disk, so the nominal dataset size of 917 images was unchanged while the effective diversity seen across epochs increased. Augmentations were applied only to training data.

3.3. Class Weighting

Given the class imbalance (normal < abnormal), class-weighted cross-entropy was employed to mitigate bias toward the majority class. For class $c$ with $N_c$ samples and a total of $N$ samples across $C$ classes, the class weight $W_c$ was defined as:

$$W_c={\displaystyle \frac{N}{C\times N_c}}$$

where $N$ is the total number of samples, $C$ is the total number of classes, and $N_c$ is the number of samples in class $c$. For the binary case ($C=2$), this formulation assigns a larger weight to the minority (normal) class, thereby penalizing its misclassification more heavily.

3.4. ViT Model Training

The primary backbone was ViT-Tiny (vit_tiny_patch16_224; 12 transformer blocks, embedding dimension 192, patch size 16, approximately 5.5 million parameters), a compact Vision Transformer adapted for binary classification by replacing the classification head with a two-logit linear layer. Figure 3 illustrates the overall architecture, showing the patch-embedding pipeline and the internal structure of a transformer encoder block. The encoder was initialized with ImageNet pretraining to enhance data efficiency. Training used a modern optimization recipe: AdamW with weight decay, label smoothing, and a learning-rate schedule (warm-up followed by cosine/step decay). Early stopping and checkpointing were triggered by validation loss to prevent overfitting. A full grid search was performed over the three hyperparameters that most affect convergence on this dataset: batch size {16, 32, 64}, learning rate {1×10⁻⁴, 5×10⁻⁴, 1×10⁻³}, and number of epochs {5, 10, 15}, yielding 27 configurations (Section 4.4). The optimizer was AdamW, the input resolution was 224×224 pixels (RGB), and the encoder was initialized from ImageNet pretrained weights. The primary metric used to select the optimal configuration was the mean cross-validation F1-score, with accuracy used as a secondary criterion (Section 3.6). The same training recipe underpins the controlled cross-architecture benchmark reported in our companion study [45].

3.5. Grad-CAM Application

To enhance model interpretability, Grad-CAM was applied to ViT predictions using a transformer-compatible implementation that computes gradients with respect to the final attention or feature maps. Because Grad-CAM was originally formulated for the spatial feature maps of convolutional networks, it was adapted to the transformer as follows. The sequence of patch tokens output by the final encoder block (excluding the [CLS] token) was reshaped back into its 14×14 spatial grid, which serves as the target activation map in place of a convolutional feature map. Gradients of the target-class logit were computed with respect to these reshaped token activations, channel-wise averaged to obtain importance weights, combined with the activations, and passed through a ReLU to yield the class-discriminative localization map, which was then upsampled to the input resolution. This reshape-and-attribute procedure is the standard means of applying gradient-based localization to ViTs. Heatmaps were generated for validation images to visualize regions that contribute to the classification decision. Visualization parameters (smoothing, normalization, overlay opacity) were maintained consistently across all runs. This procedure enables qualitative assessment of whether the model’s attention corresponds to clinically relevant morphological regions such as nuclei, irregular boundaries, or chromatin distribution. The resulting observations are analyzed in Section 4.

3.6. Evaluation Protocol

For completeness, performance was assessed with accuracy, precision, recall, and F1-score on the held-out folds. Because false negatives and false positives carry different clinical consequences in screening, sensitivity (recall on the abnormal class) and specificity (recall on the normal class) were reported separately, and discrimination was additionally summarized using the area under the receiver operating characteristic curve (ROC-AUC). These metrics, together with the cross-architecture comparison in Section 4.4, follow the evaluation protocol of the companion benchmark study [45]. The use of multiple complementary metrics together with feature-based model assessment is consistent with prior health-analytics work on clinical outcome prediction, where careful metric selection improved both performance and interpretability [47]. The metrics are defined as follows:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

$$Recall\left(Sensitivity\right)={\displaystyle \frac{TP}{TP+FN}}$$

$$F1-score={\displaystyle \frac{2·(Precision·Recall)}{Precision+Recall}}$$

where $TP$, $TN$, $FP$, and $FN$ represent true positives, true negatives, false positives, and false negatives, respectively. To enable consistent comparison across configurations, fold-wise means and 95% confidence intervals (CI) were computed as follows:

$$CI=\bar{x}\pm z.{\displaystyle \frac{s}{\sqrt{n}}}$$

where $\overline{x}$ = mean of fold-wise results, $s$ = standard deviation, $n$ = number of folds, $z=1.96$ (for 95% confidence interval). Interpretability outputs were summarized with representative heatmaps; quantitative alignment with expert-defined regions can be added when annotations are available.

4. Results and Discussion

This section reports experimental outcomes following the same five-stage workflow described in Section 3 and Figure 2: (1) Dataset Preparation, (2) Data Augmentation, (3) Class Weighting, (4) ViT Model Training, and (5) Grad-CAM Interpretability. Organizing results in parallel with methodology enables direct tracing from design choices to performance.

4.1. Dataset Preparation Results

The Herlev dataset was successfully preprocessed into 917 images with binary classification (242 normal, 675 abnormal). All images were standardized to RGB format with consistent resolution. ImageNet normalization statistics were applied (mean= [0.485, 0.456, 0.406], std= [0.229, 0.224, 0.225]). Stratified 5-fold cross-validation maintained the 27.8% normal, 72.2% abnormal distribution across all folds.

4.2. Data Augmentation Results

Table 1 presents the performance comparison of seven augmentation strategies (three single and four combined augmentation strategies) evaluated using 5-fold cross-validation with optimal class weights.

Single augmentations showed varying effectiveness. Color Jitter performed worst with an accuracy of 89.87% and a precision of 80.40%, which suggests color variations obscured key diagnostic cues such as chromatin texture. Random Affine resulted in an accuracy of 91.39% but with a reduced recall (83.40%), likely due to geometric distortions that affected nuclear shape. In contrast, Horizontal Flip achieved strong overall performance 94.77% accuracy and 91.30% recall, which indicates that left–right invariance is a beneficial augmentation for cervical cytology images without distorting diagnostically relevant structures.

Combined augmentations showed limited synergy. Color Jitter + Horizontal Flip (94.33%) was slightly below horizontal flip alone, while Color Jitter + Random Affine (93.23%) achieved the highest recall (95.00%) but lower precision (84.10%), which increased false positives. Horizontal Flip + Random Affine (92.26%) and All Three Combined (94.22%) offered no significant gains over simpler (single) augmentations.

Overall, augmentation effectiveness depends on biological plausibility rather than variety. Transformations that preserve diagnostic features (horizontal flip) outperformed those introducing artificial distortions, which confirms that simpler (single) augmentations as well as clinically valid augmentations yield the most reliable results.

4.3. Class Weighting Results

Table 2 summarizes the systematic evaluation of five weight multiplier configurations to address class imbalance.

Varying the abnormal-to-normal weight ratio produced distinct performance trade-offs. Case 4 (0.7 × 1.3 multipliers) achieved a balanced trade-off between recall and precision, which resulted in the highest F1-score (91.90%). Case 4 achieved the best overall performance, with 95.64% accuracy and 93.40% recall, which effectively prioritizes abnormal-cell detection without excessive misclassification of normal samples. These results confirm that moderate weighting enhances screening reliability, which is critical for medical applications where missing abnormal cases carry higher clinical risk.

4.4. ViT Model Training Results

A comprehensive hyperparameter optimization study evaluated 27 combinations of batch sizes (16, 32, 64), learning rates (0.0001, 0.0005, 0.001), and numbers of epochs (5, 10, 15). All hyperparameter configurations were evaluated using stratified 5-fold cross-validation, and Table 3 reports the average performance across folds for all 27 configurations, with the top-performing configurations highlighted in bold.

A learning rate of 0.0001 consistently outperformed higher values, which ensured stable convergence and prevented accuracy loss caused by overshooting the optimal loss minimum during optimization. Batch size affected training efficiency; Batch 16 required more epochs due to noisy gradients, while Batch 32 achieved the best balance with the highest accuracy (96.51%). Batch 64 showed slight degradation, likely from reduced gradient diversity. Training duration analysis indicated that 15 epochs yielded optimal convergence, whereas shorter runs led to underfitting.

To assess result stability and reduce the impact of stochastic training effects, each selected configuration was replicated 10 times using different random initializations. Replication mitigates variability introduced by random weight initialization and data shuffling, which provides a more reliable estimate of model performance. Table 4 reports the mean and standard deviation across these replications for both cross-validation and application-level evaluation. In Table 4, “CV” denotes cross-validation performance computed on the held-out fold in each split, and “App” (application-level) denotes resubstitution on the full dataset used to fit the final model.

As shown in Table 4, B32_E15 (batch size of 32 and 15 epochs) achieved the highest cross-validation accuracy (95.15%) with minimal variation, which indicates stable convergence and strong generalization. The application-level columns in Table 4 are reported only for completeness: they reflect resubstitution on the full dataset used to fit the final model and therefore overestimate generalization. Accordingly, all comparative claims in this study rely exclusively on the cross-validation estimates, and the application-level figures should not be interpreted as held-out performance. Their large standard deviations (for example, an application F1 standard deviation of 9.02 for B32_E15) further underscore that these values are not a reliable basis for model selection.

Statistical analysis that used pairwise t-tests (Table 5 and Table 6) verified if the performance differences in both accuracy and F1-score were significant. The statistical analysis in Table 5 and Table 6 revealed that only the comparison between B32_E10 and B32_E15 showed statistically significant differences (p = 0.035 for accuracy, p = 0.045 for F1-score). The remaining configurations (B16_E15, B32_E15, and B64_E15) exhibited no significant differences, which indicate comparable performance levels. These results confirm that model variations were minor beyond the optimal configuration, as visualized in Figure 4. Pairwise testing was reported for accuracy and F1-score because, in this binary screening setting, F1-score already integrates precision and recall into a single balanced measure of the minority (abnormal) class. Precision and recall were not tested individually here; given the clinical importance of false negatives versus false positives, the cross-architecture comparison in Section 4.4 and the companion benchmark study [45] additionally examine sensitivity and specificity directly, and formal pairwise testing of precision and recall across configurations is a useful extension for future work.

4.5. Grad-CAM Interpretability Results

Gradient-weighted Class Activation Mapping was applied to the best-performing configuration (B32_E15) to visualize model decision-making patterns and provide transparency in the classification process.

Figure 5 presents Grad-CAM visualizations for correctly classified abnormal cells and false negative errors. Here, the reported “focus scores” denote the normalized fraction of total Grad-CAM activation that falls within manually delineated nuclear/abnormal versus normal regions for each example, scaled to the range [0,1]; the scores are illustrative of individual representative cases rather than a dataset-wide quantitative metric. For correctly classified abnormal cells, the model demonstrated strong focus on nuclear regions (focus score: 1.000) with minimal attention to normal features (score: 0.000). In contrast, false negative cases shown in Figure 5 exhibited high normal focus scores (0.994-0.998) with insufficient attention to abnormal features (0.002-0.006), which indicates the model incorrectly focused on normal-appearing regions within abnormal samples.

Figure 6 illustrates the attention patterns for false positive errors and correctly classified normal cells. False positive cases in Figure 6 showed moderate abnormal focus scores (0.712-0.725) on benign features that superficially resembled abnormal patterns, which often result from staining artifacts or cellular overlapping. Correctly classified normal cells demonstrated appropriate normal focus (0.997-1.000) with minimal abnormal attention (0.000-0.003).

The attention patterns revealed in Figure 5 and Figure 6 consistently aligned with established cytopathological criteria, which focus on nuclear morphology, chromatin distribution, and cellular boundaries—the same features utilized by trained cytopathologists for diagnosis. This alignment demonstrates that the Vision Transformer model learns clinically relevant features for cervical cancer detection.

4.6. Comparison with CNN Baselines and Computational Efficiency

To contextualize the performance of the ViT-Tiny model against established convolutional architectures under identical conditions, a controlled cross-architecture benchmark was conducted in our companion study [45], in which ViT-Tiny and four widely used CNN baselines (ResNet50, EfficientNet-B0, VGG16, and DenseNet121) were trained on the same Herlev binary task using a single uniform training recipe with matched data splits and replication. Under that controlled comparison, ViT-Tiny matched the strongest CNN baselines (VGG16 and DenseNet121) on classification performance, with paired statistical testing finding no significant difference between them, while significantly outperforming ResNet50 and EfficientNet-B0 [45]. This indicates that, among modern competitive architectures, the choice of backbone is not the dominant determinant of accuracy on this dataset, which shifts the practical selection criterion toward efficiency. Full per-architecture results are reported in the companion study [45]; the present paper’s own ViT-Tiny results (for example, the 95.15% cross-validation accuracy of the optimal configuration in Section 4.4) were obtained under a different optimization protocol and remain this study’s primary results.

Beyond classification accuracy, the same study quantified computational efficiency on an NVIDIA A100 GPU, which is the decisive factor once accuracy saturates. ViT-Tiny (5.52 M parameters) uses approximately 24× fewer parameters than VGG16 (134.27 M) and about 15× less peak GPU memory, while delivering roughly 2.3× higher batch-level throughput; relative to the more compact DenseNet121 it still uses fewer parameters and markedly less memory at higher throughput. These reductions translate directly into deployment advantages for the resource-constrained, often GPU-free laboratory settings where cervical cancer screening is most needed, and they provide the practical justification for selecting a compact Vision Transformer over heavier CNN backbones. Full per-architecture latency, throughput, and memory measurements are reported in [45].

4.7. Comparison with Prior Work

To position the present results against prior work on the same dataset, Table 7 compares binary-classification studies on the Herlev Pap smear dataset in terms of method, reported performance, validation technique, split type, and methodological rigor. Early approaches relied on classical machine learning, such as the hybrid genetic-algorithm and nearest-neighbor scheme of Marinakis et al. [48], whereas more recent work has shifted toward deep learning and, most recently, transformer-based models. The proposed ViT-Tiny model is the only study in this comparison that simultaneously employs cross-validation, statistical significance testing, and clinical interpretability. While some earlier studies report higher headline accuracy, those results were obtained on single train/test splits without cross-validation or significance testing, which tend to yield optimistic, higher-variance estimates that are not directly comparable to cross-validated performance.

As Table 7 indicates, the proposed ViT-Tiny model achieves competitive cross-validated accuracy (95.15%) while being the only approach that combines stratified cross-validation, pairwise statistical testing, and Grad-CAM interpretability aligned with cytopathological criteria. Among the cross-validated studies, Pirovano et al. (2019) is the closest comparable baseline, reporting 94.0% binary accuracy on Herlev using 4-fold CV and ResNet-101 with Integrated Gradients interpretability — our approach exceeds this by approximately 1.15 percentage points under a stricter protocol (stratified 5-fold CV with 10 replications and pairwise statistical testing). Ghoneim et al. (2020) also used 5-fold cross-validation and achieved a higher accuracy of 99.5%; however, their work lacks both statistical significance testing and clinical interpretability, which are essential for trustworthy medical AI. The remaining studies — including CerviFormer (94.57%, single train/test split 90/10), Kaur et al. (95.0%, single train/val/test split 60/20/20), and Yilmaz and Kantar (85.0–93.0%, single train/test split 85/15) — relied on single fixed splits without cross-validation, statistical testing, or interpretability analysis. Single-split estimates tend to yield optimistic, higher-variance results that are not directly comparable to cross-validated performance. The present work therefore represents the most methodologically complete approach in this comparison, combining competitive cross-validated accuracy with the statistical rigor and Grad-CAM interpretability essential for transparent and reliable deployment in cervical cancer screening.

5. Conclusions

This study demonstrates the potential of Vision Transformers as accurate and interpretable tools for automated cervical cancer screening. By restructuring Pap smear classification into a binary screening task, the experiments were aligned with clinical practice and demonstrated that optimized ViT models can achieve ~95% cross-validation accuracy, with pairwise t-tests indicating that the top configurations were statistically comparable. Interpretability analysis suggested that, for the representative cases examined, model attention aligned with clinically meaningful morphological features, supporting their potential as decision-support tools pending broader validation.

Several limitations should be acknowledged in this study. First, the Herlev dataset contains only 917 images from a single institution, which may not fully represent the diversity of cervical cell morphology across different populations, staining protocols, and imaging equipment. Second, the binary classification approach, while clinically relevant for initial screening, simplifies the nuanced multi-class grading system used in cytopathology practice. Third, the study relied on pre-existing annotations without inter-rater reliability assessment, which potentially introduce label noise. Future work should mitigate this through expert consensus labeling by multiple cytopathologists with reporting of an agreement statistic such as Cohen’s or Fleiss’ kappa, and through uncertainty-aware learning techniques that down-weight or flag ambiguous samples. Fourth, the computational requirements of Vision Transformers may limit deployment in resource-constrained settings where cervical cancer screening is most needed; the companion benchmark study [45] partly addresses this by quantifying that the compact ViT-Tiny backbone used here requires substantially fewer parameters and less memory than common CNN baselines while remaining competitive in accuracy. Finally, while Grad-CAM provides valuable insights into model attention, it may not fully capture all aspects of the transformer’s decision-making process, particularly the complex interactions between attention heads. A further important limitation is that evaluation was confined to the single-source Herlev dataset; external validation on independent, multi-center datasets such as SIPaKMeD, and cross-dataset transfer experiments, are necessary to establish generalizability across populations, staining protocols, and imaging equipment, and are a priority for subsequent work.

While these results are promising, further validation on larger and multi-center datasets is essential to ensure robustness across diverse populations and imaging protocols. Additionally, prospective clinical trials that evaluate AI-assisted screening in real-world workflows with cytotechnologists and pathologists are crucial for clinical translation, which assess not only diagnostic accuracy but also workflow integration, efficiency, and user acceptance in routine medical practice. Ultimately, this research contributes an important step toward bridging cutting-edge AI with clinical practice, which paves the way for AI-assisted cytopathology systems that can enhance early cervical cancer screening, reduce diagnostic errors, and expand global access to preventive care.