A Unified Multi-Task Vision Transformer for Interpretable Ovarian Tumor Analysis

1. Introduction

Ovarian cancer (OC) is the cancer of the ovaries and is a prevalent gynaecological malignancy causing many deaths annually. According to the World Cancer Research Fund [1], 206,965 deaths were recorded out of the 324,603 cases recorded in 2022. In spite of all ongoing research efforts, early detection of OC remains challenging, with only 25% of cases being diagnosed at the early stages. However, patients that were diagnosed early (Stage 1) have a 95% survival rate of five years while those diagnosed late (Stage 4) have a 15% survival rate of five years. The observed difference in survival rates suggests that accurate early detection can play a significant role in improving patient outcomes.

There are two commonly used screening approaches employed by clinicians to diagnose OC. The first approach involves blood testing to measure CA125 protein levels, a biomarker frequently elevated in OC patients. However, this screening approach lacks specificity since various other benign conditions can also elevate CA125 concentrations. The second approach uses ultrasound imaging which is non-invasive and provides immediate real-time diagnostic imaging capabilities [2,3]. However, interpreting ovarian ultrasound images can be challenging, as it often depends on the operator’s skill and experience. This variability in interpretation can lead to possible misdiagnoses, which highlights the need for a diagnostic tool that is more objective, reliable and interpretable to the clinician [4]. This diagnostic challenge demands standardised, reproducible, and interpretable diagnostic approaches.

The advent of deep learning has advanced medical image analysis by automating and improving various diagnostic tasks such as classification of the images and segmentation of the region of interest within the images [5,6,7,8]. Convolution Neural Networks (CNN) have shown strong performance in categorizing and delineating anomalies within the ovarian medical images [9,10,11,12,13]. More recently, Vision transformers (ViTs) have gained attention as promising alternatives especially in their ability to capture long range dependencies and global contextual information which are crucial for comprehensive ovarian medical image understanding [14,15,16,17]. Our preliminary single-task study comparing ViT and CNN backbones on ovarian ultrasound informs the backbone selection adopted in this work [18].

Although deep learning models have achieved strong predictive performance, their `black-box’ nature limit their acceptance in clinical settings, especially in situations where decisions carry high risk. Clinicians do not only need accurate predictions but also they need to clearly understand the reason behind those predictions so that they can align that with the medical knowledge and practice. This has led to growing interest in Explainable Artificial Intelligence (XAI) [19,20]. One promising approach is the use of Concept Bottleneck Models (CBMs) [21,22], which introduce human-interpretable concepts as an intermediate step in the decision-making process. Incorporating interpretability directly into the model is important for building trust and supporting clinical use, rather than relying on post-hoc saliency-based methods such as Grad-CAM [23] and attention visualization that fail to provide clinical relevant explanations that aligns with medical knowledge [11,24].

Despite the advancements in deep learning for ovarian ultrasound [11,13,17], multi modal learning  [12], and Explainable Artificial Intelligence (XAI) [11], a notable research gap still exists. Most studies focus on single-task approaches such as classification or segmentation or employ XAI as post-hoc process. No prior research has attempted to integrate concept-based interpretability directly into multi-task framework using vision transformers for a comprehensive ovarian ultrasound image analysis.

To address this gap we proposed a Vision Transformer-based multi-task deep learning pipeline for comprehensive ovarian ultrasound analysis. Unlike previous approaches that treat classification, segmentation and Interpretability as separate post-hoc processes, our framework simultaneously performs: 8-class ovarian tumor classification, 3-class malignancy detection (Normal, Benign, Malignant), precise ovarian tumor segmentation, and medical concept interpretability guided by established clinical criteria.

Our architectural design reflects how radiologists actually work when they analyze ovarian ultrasound images. Each computational task maps to a specific clinical step: classification provides the initial tumor categorization, malignancy detection enables risk stratification, segmentation supports surgical planning, and concept prediction gives diagnostic justification. By integrating these tasks into a single model rather than handling them separately, the system ensures that risk assessment and morphological analysis both come from the same underlying feature representations. This helps maintain consistency across predictions, which is similar to how radiologists consider multiple characteristics together rather than in isolation.

Contributions

UM-TOTA makes three distinct technical contributions to the field of ultrasound image analysis of ovarian cancer:

• We introduce UM-TOTA, the first ViT-based architecture for ovarian ultrasound that integrates concept-based interpretability directly into the multi-task learning framework, rather than applying it post-hoc. All predictions pass through a clinical concept bottleneck whose targets are informed by the International Ovarian Tumor Analysis(IOTA) and Ovarian-Adnexal Reporting and Data System(O-RADS) criteria, ensuring that model explanations reflect the actual decision process rather than approximating it after the fact.
• We develop a knowledge-guided concept supervision framework that maps ground-truth tumor labels to IOTA- and O-RADS-defined clinical concept targets, going beyond generic concept bottleneck implementations to embed domain-specific diagnostic knowledge directly into model training. This enables clinicians to verify model reasoning using criteria that they already apply in practice.
• We demonstrate that a unified four-task architecture performing classification, malignancy detection, segmentation, and concept-based interpretability in a single forward pass achieves a 66.7% parameter reduction and 65.5% inference latency reduction over sequential single-task deployment, with concrete hardware measurements provided, while maintaining comparable diagnostic accuracy to specialised single-task models.

The remainder of this paper is organised as follows. Section 2 reviews related work. Section 3 details our proposed UM-TOTA methodology. Section 4 presents our detailed results and discussion. Section 5 concludes with a summary of contributions and clinical impact.

2. Related Works

2.1. Deep Learning Architectures and Multi-Task Strategies in Ovarian Cancer

Deep learning has shown strong capabilities in medical imaging tasks, with CNN emerging as the dominant approach for automated diagnosis [25]. Early deep learning approaches for OC diagnosis by Wu et al. [26] explored classification methods for ultrasound images, demonstrating improved accuracy compared to traditional manual methods. Also, Wang et al. [27] employed transfer learning techniques to distinguish between benign, borderline, and malignant ovarian masses. Furthermore, [28] demonstrated the effectiveness of fine-tuned VGG-16 networks for ovarian cyst detection, establishing that transfer learning approaches are viable for this medical domain.

Recent developments have seen the emergence of comprehensive diagnostic systems that extend beyond simple classification. This approach aligns with multi-task attention frameworks observed in adjacent ultrasound tasks [29], which combine localized segmentation maps and pathological descriptors within a unified learning architecture. This approach is similar to the ensemble methods demonstrated by [11], which achieved accuracy improvements through model integration. Another study [30] used datasets from multiple health centres to prove that deep learning approaches can be clinically viable across diverse settings. The integration of multi-modal approaches in studies such as [13,31,32], has opened new avenues for comprehensive analysis combining ultrasound imaging with clinical parameters.

Vision Transformers (ViT) have achieved competitive performance when compared with CNNs by providing superior capability for modelling global relationships within images [33]. This advancement builds upon the foundational Transformer architecture introduced by [34]. Hybrid CNN-Transformer architectures have shown consistent advantages in medical applications. TransUNet [35] combines Transformers with U-Net for medical image segmentation, while Swin-Unet [36] utilizes hierarchical ViT to capture multi-scale features effectively. Furthermore, [37] demonstrated pure transformer architectures for 3D medical image segmentation with UNETR. PMFFNet [38] addressed CNN limitations by combining feature pyramid networks with ViT components for ovarian tumor segmentation. Multiple ViT variants, including ViT-Large-P32-384, have been reported to achieve strong performance in ovarian cancer classification tasks [15].

Multi-task learning has been successfully applied to other medical domains, such as the end-to-end multi-task architecture for brain tumor analysis in MRI proposed by [39] and the breast cancer framework by [29], which integrates BI-RADS descriptors with SHAP-based post-hoc explainability. However, it is observed that ViT use in multi-task OC diagnosis remains under explored, particularly for simultaneous classification, segmentation, and concept based interpretability in multi-task medical settings. Most existing systems address tasks in isolation or use post-hoc interpretability [11,29]. The choice of ViT in this work is motivated by its ability to capture long-range spatial dependencies through self-attention mechanisms, which is essential for sonographic interpretation, where understanding relationships across the entire image matters for accurate diagnosis.

2.2. From Post-Hoc Explanation to Concept-Based Interpretability in Ovarian Tumor Diagnosis

The increasing adoption of artificial intelligence in medical practice has created a growing demand for interpretable and explainable medical AI systems. Explainable AI methods in medical imaging are important for clinical decision-making transparency [20]. Traditional post-hoc explanation methods such as Grad-CAM [23], LIME [40], and SHAP [41] have been applied to several medical Interpretability tasks, enabling clinicians to understand which image regions influence model decisions.

Concept bottleneck models represent a significant advance in interpretable machine learning for medical diagnosis. While foundational post-hoc approaches like Testing with Concept Activation Vectors (TCAV) [42] measure a model’s sensitivity to high-level ideas after training, the framework presented by [43] utilizes a concept bottleneck paradigm, enabling models to explicitly learn human-understandable concepts as intermediate representations for interpretable medical diagnosis. The work by [44] demonstrated concept-based explanations for dermatology. Another study [45] emphasized the importance of inherently interpretable models rather than post-hoc explanations, which aligns with medical requirements for transparent decision-making. The work by [15] which integrates ViT with LIME for OC classification, has demonstrated the potential for combining advanced architectures with interpretability methods.

Current interpretability approaches primarily rely on generic explanation techniques rather than incorporating domain-specific medical knowledge. For instance, [29] demonstrated post-hoc explainability of BI-RADS descriptors for breast cancer detection. Post-hoc methods approximate the model’s reasoning and may not be faithful to the actual decision process [45]. Unlike these approaches, our framework incorporates interpretability directly into the model through the concept bottleneck, ensuring that decisions are causally linked to the detected clinical concepts. The challenge in medical interpretability extends beyond generic explanation methods to require integration of established clinical frameworks. In OC diagnosis, clinical assessment relies on well-established criteria such as the IOTA (International Ovarian Tumor Analysis) simple rules [46] and O-RADS (Ovarian-Adnexal Reporting and Data System) guidelines [47]. These provide standards for characterizing ovarian masses based on some morphological features that can be encoded into AI decision-making processes. Current interpretability approaches fail to incorporate domain-specific features established in medical knowledge frameworks like IOTA/O-RADS criteria into AI decision-making processes.

2.3. Evaluation Methodologies in Medical AI

Cross-validation methods specific to medical imaging address unique challenges in healthcare data. According to [48], foundational cross-validation principles remain relevant for medical AI applications, while [49] provided theoretical foundations for model selection. The study by [50] provided some good guidelines for cross-validation in medical image analysis, highlighting the importance of patient-level splits to avoid optimistic bias.

The importance of ablation studies for understanding model component contributions through systematic evaluation of architectural elements was demonstrated by PMFFNet [38]. Comprehensive empirical evaluation and benchmarking practices in medical image analysis are exemplified by nnU-Net [51] and the survey of Litjens et al. [52].

Statistical significance testing in medical AI has gained increased attention as systems approach clinical deployment. Statistical comparisons of classifiers over multiple datasets require careful consideration [53], while [54] addressed challenges of inference for generalization error. The recent work by [55] highlighted the importance of accounting for randomness in deep learning experiments, which is particularly relevant for medical AI where reproducibility is crucial for clinical acceptance.

Current evaluation methodologies frequently lack systematic analysis of individual task contributions in multi-task learning scenarios. While single-task evaluations are well-established, comprehensive ablation studies that examine how different tasks interact and contribute to overall system performance remain uncommon. This limitation is particularly problematic for complex medical AI systems where understanding the relative importance and interdependencies of different clinical objectives is crucial for system optimization and clinical acceptance.

3. Methodology

3.1. Dataset and Data Preparation

3.1.1. Dataset

We use the Multi-Modality Ovarian Tumor Ultrasound (MMOTU) dataset [13] that has 1,469 2D ultrasound images from 294 patients captured from different viewing angle during transvaginal examination. These images were obtained using a Mindray Resona8 ultrasonic diagnostic instrument at Beijing Shijitan Hospital, Capital Medical University. The dataset has expert annotations from 27 specialists in Obstetrics and Gynecology. Each image undergoes a two-stage annotation process where one expert provides initial annotations, followed by verification from another expert, ensuring high-quality ground truth labels essential for medical AI applications. The dataset has eight distinct ovarian tumor categories and we label each tumor class with a number as thus: 0-chocolate cyst, 1-serous cystadenoma, 2-teratoma, 3-theca cell tumor, 4-simple cyst, 5-normal ovary, 6-mucinous cystadenoma, and 7-high-grade serous carcinoma. This research maps the eight-class categorization to a three-class malignancy classification scheme following established clinical practice as follows:

$$M\left(c\right)=\left\{\begin{array}{cc}0,\hfill & c\in \{0,1,2,3,4,6\}\left(\mathrm{benign}\right)\hfill \\ 1,\hfill & c\in \left\{5\right\}\left(\mathrm{normal}\right)\hfill \\ 2,\hfill & c\in \left\{7\right\}\left(\mathrm{malignant}\right)\hfill \end{array}\right.$$

(1)

Where c represents the 8-class tumor category and $M\left(c\right)$ represents the corresponding malignancy classification.

Although the MMOTU dataset is publicly available and the original authors obtained ethics approval for data collection and release, we secured additional departmental ethics approval from our institution (Ethics Approval Number: 2457) to ensure full compliance with research standards.

3.1.2. Data Preparation and Augmentation

The images in the dataset has various resolutions but we resized them to 224×224 pixels to match the ViT backbone requirements. We then applied a normalisation strategy using ImageNet channel statistics [56] to align the input distribution with that of the pre-trained backbone and to support effective domain adaptation. We then employed an augmentation strategy with probabilistic transformations to prevent overfitting:

$$T_{\mathrm{aug}}\left(I\right)=T_{\mathrm{rot}}\left(T_{\mathrm{vflip}}(T_{\mathrm{hflip}}(I,p_h),p_v),\theta \right)$$

(2)

Where $T_{\mathrm{hflip}}$ and $T_{\mathrm{vflip}}$ represent horizontal and vertical flipping with probabilities $p_h=p_v=0.5$, and $T_{\mathrm{rot}}$ applies random rotation with angle $\theta \sim U(0^{\circ },{90}^{\circ })$. According to general image augmentation frameworks [57], these geometric transformations mitigate overfitting by artificially increasing dataset diversity without altering the underlying semantic labels.

In addition to geometric transformations, intensity augmentations were applied including RandomBrightnessContrast ($p=0.4$) and CLAHE ($p=0.4$). Noise augmentations included GaussNoise ($p=0.3$) and CoarseDropout ($p=0.15$). Images were resized to $224\times 224$ pixels and normalized using ImageNet statistics.

3.2. Unified Multi-Task Ovarian Tumor Architecture (UM-TOTA)

We propose UM-TOTA, a unified multi-task architecture for ovarian tumor analysis integrating Vision Transformer (ViT)-based representation learning with concept-driven clinical interpretability. Unlike traditional single-task models, UM-TOTA concurrently performs:

1.

Eight-class tumor classification

2.

Three-class malignancy detection

3.

ROI-based tumor segmentation

4.

Prediction of clinical semantic concepts for transparent diagnosis

The system consists of five interconnected modules:

• Transformer Backbone: A ViT encoder which extracts global contextual features from ultrasound images.
• Task Heads: Parallel classification, malignancy detection and segmentation heads generate respective predictions.
• Concept Bottleneck Module: Predicts interpretable clinical indicators such as boundary clarity, shape regularity, vascularization, and solid-component presence.
• Clinical Reasoning Unit: Integrates concept activations into attention-weighted decision pathways.
• Multi-Task Loss Coordinator: Joint optimization with dynamic weighting ensures stable convergence during training.

Figure 1 illustrates the high-level structure of UM-TOTA. The design enforces semantic alignment between clinical concepts and final decisions, producing interpretable predictions suitable for radiology-assisted assessment.

The architectural design is motivated by the clinical workflow where radiologists simultaneously assess multiple characteristics when analyzing ovarian ultrasound images. Rather than treating these tasks independently, UM-TOTA leverages shared feature representations to improve performance across all objectives while maintaining clinical interpretability through explicit concept modeling. This approach represents a significant shift from conventional CNN-based medical imaging systems, offering both performance and clinical transparency.

3.2.1. ViT Backbone Architecture

The choice of ViT over conventional CNN architectures was motivated by factors specific to our multi-task interpretability framework; firstly, the self-attention mechanism supports our clinical reasoning module by capturing global relationships between anatomical structures across the entire ultrasound image. This entire view is essential for learning IOTA/O-RADS concepts which require assessment of features distributed across the image rather than localized regions. In contrast, the local receptive fields of CNNs would require explicit mechanisms to aggregate such global context. Secondly, the way ViT processes images in patches is very similar to how radiologists examine ovarian ultrasounds. They look at features like the edges, solid areas, and cystic regions separately, but also consider how these parts relate to each other. This representation naturally handles the varying scales of ovarian tumors without requiring explicit multi-scale feature extraction modules.

To provide direct empirical justification for this choice, we conducted a systematic single-task comparison of ViT-Base/16 against ResNet50, DenseNet-121, EfficientNet-B0, and EfficientNetV2-m on the same MMOTU dataset under identical 5-fold stratified cross-validation conditions [18]. ViT achieved 82.98% ± 1.89% 8-class classification accuracy and 91.15% ± 1.48% 3-class malignancy accuracy, outperforming the best CNN baseline (EfficientNet-B0 at 80.66%) by 2.32 percentage points. This empirical superiority in the single-task setting, combined with the global attention properties described above, motivates the ViT backbone selection for UM-TOTA.

The ViT backbone processes input images of size $224\times 224$ pixels by dividing them into $16\times 16$ patches, resulting in a sequence of 196 patch embeddings. Each patch embedding is projected to a 768-dimensional feature space through the learned linear transformation.

The transformer encoder processes patch embeddings through 12 layers of multi-head self-attention [34], producing a 768-dimensional feature representation via global average pooling that serves as the shared feature space for all downstream tasks (Figure 2).

To address the data efficiency challenges inherent in Vision Transformers when trained on medical datasets of moderate size ($N=1,469$), we employed transfer learning strategy. The ViT backbone was initialised with ImageNet-1K pre-trained ViT-Base/16 weights to draw on learned textural and shape features before fine-tuning on the MMOTU dataset. This stabilizes the training and prevents overfitting which occurs when training Transformers from scratch on small datasets.

3.2.2. Task-Specific Head Architecture Design

The shared 768-dimensional feature representation from the ViT backbone feeds into four specialized prediction heads, each architecturally designed for specific clinical objectives as shown in Figure 2. This design ensures that while features are shared at the backbone level, each task receives specialized processing appropriate to its clinical requirements.

As illustrated in Figure 2, the shared 768-dimensional ViT feature embeddings feed into four task-specific heads for classification, malignancy detection, segmentation and clinical concept reasoning.

The segmentation head implements a decoder architecture that progressively upsamples the feature representation through transposed convolutions. Unlike traditional U-Net architectures that require encoder–decoder symmetry, this approach leverages the rich feature representations from the transformer backbone.

3.2.3. Concept Bottleneck Model Integration

The distinguishing feature of the UM-TOTA architecture is the integration of a concept bottleneck model. Ten clinical concepts informed by the IOTA and O-RADS guidelines were modelled into the architecture, building on concept bottleneck models [22] and their extension to clinically guided supervision [21]. The concept learning utilizes knowledge-guided supervision, where we mapped ground-truth tumor labels to concept targets according to IOTA and O-RADS guidelines. For example, a lesion classified as Serous Cystadenoma was linked to the concepts high boundary clarity and Avascular. This design ensures that the bottleneck layer reflects clinical criteria familiar to practicing clinicians. For presentation in Figure 3, the ten concepts are grouped into three categories: Boundary/Structure (boundary clarity, shape regularity, acoustic shadowing, posterior enhancement), Tissue/Component (homogeneous texture, cystic components, solid components, papillary projections), and Clinical Signs (vascularization, ascites presence).

The concept bottleneck accepts the 768-dimensional ViT features $F\in {\mathbb{R}}^{768}$ and processes them through individual concept predictors to generate interpretable clinical representations. As shown in Figure 3, it utilizes ten individual concept predictors rather than a single joint layer. Each predictor is formulated as:

$$C_i=h_{{\mathrm{concept}}_i}\left(F\right)$$

(3)

where $h_{{\mathrm{concept}}_i}$ represents the i-th concept predictor defined as:

$$h_{{\mathrm{concept}}_i}\left(F\right)={\mathrm{Linear}}^1\left(\mathrm{ReLU}\left({\mathrm{Linear}}^{64}\left(\mathrm{Dropout}(F,0.1)\right)\right)\right)$$

(4)

where the numerical superscripts ($64,1$) represent the exact terminal output dimensionality of the respective linear transformation layers, ensuring each individual task head reduces the shared feature map down to a single raw scalar logit $C_i\in \mathbb{R}$. The ten clinical concepts modelled are: boundary clarity, shape regularity, acoustic shadowing, posterior enhancement, texture homogeneity, cystic components, solid components, papillary projections, vascularization patterns, and ascites presence.

The individual concept logits are stacked into a joint vector $C={[C_1,C_2,\dots ,C_{10}]}^T\in {\mathbb{R}}^{10}$ and converted to non-mutually exclusive probability scores using an element-wise sigmoid activation function:

$$C_{\mathrm{scores}}=\sigma \left(C\right)={[\sigma \left(C_1\right),\sigma \left(C_2\right),\dots ,\sigma \left(C_{10}\right)]}^T$$

(5)

where $C_{\mathrm{scores}}\in {\mathbb{R}}^{10}$ represents the final probability scores for each clinical concept, providing interpretable intermediate representations bounded tightly between $[0,1]$ that align with radiological assessment criteria established by [46].

3.2.4. Clinical Reasoning Module Architecture

From the concept bottleneck module shown in Figure 3, the clinical reasoning module processes the extracted concept activations through a vectorized attention mechanism that weights the relative importance of different clinical attributes for the final diagnostic decision. This module bridges the soft checklists and malignancy decision.

Rather than relying on post-hoc local approximations or disjoint scalar updates, the global attention distribution weights are derived directly from the concept probability vector $C_{\mathrm{scores}}\in {\mathbb{R}}^{10}$ through a joint linear projection layer wrapped in a spatial softmax normalization function:

$${\mathrm{attention}}_{\mathrm{weights}}=\mathrm{Softmax}\left(W_a·C_{\mathrm{scores}}+b_a\right)$$

(6)

where $W_a\in {\mathbb{R}}^{10\times 10}$ represents the learnable attention transformation weight matrix, $b_a\in {\mathbb{R}}^{10}$ is the structural layer bias vector, and ${\mathrm{attention}}_{\mathrm{weights}}\in {\mathbb{R}}^{10}$ dictates the relative clinical priority assigned across the predicted checklist.

The contextual vector containing the scaled, human-interpretable features is subsequently derived by performing a strict Hadamard product over the concept probability and normalized attention tensors:

$${\mathrm{attended}}_{\mathrm{concepts}}=C_{\mathrm{scores}}\odot {\mathrm{attention}}_{\mathrm{weights}}$$

(7)

where ⊙ denotes element-wise multiplication, yielding an isolated structural representation vector ${\mathrm{attended}}_{\mathrm{concepts}}\in {\mathbb{R}}^{10}$. This attention-weighted framework enables clinicians to directly track and audit which morphological features contribute most significantly to the model’s diagnostic conclusions [47].

The clinical reasoning network processes these integrated concept structures through a multi-layer perceptron (MLP) configuration ($h_{\mathrm{reasoning}}$) to isolate the final categorical tumor malignancy estimation. For parameter efficiency, the hidden layer transformations, activation thresholds, and dropout are defined sequentially:

$$\left\{\begin{array}{c}{z}_1={\mathrm{Dropout}}_{0.2}\left(\mathrm{ReLU}\left({\mathrm{Linear}}^{64}\left(x\right)\right)\right)\hfill \\ z_2={\mathrm{Dropout}}_{0.1}\left(\mathrm{ReLU}\left({\mathrm{Linear}}^{32}\left(z_1\right)\right)\right)\hfill \end{array}\right.$$

(8)

The terminal forward execution path then passes the contextually regularized representations through the output network head to isolate the absolute risk index:

$${\mathrm{reasoning}}_{\mathrm{output}}=h_{\mathrm{reasoning}}\left({\mathrm{attended}}_{\mathrm{concepts}}\right)=\sigma \left({\mathrm{Linear}}^1\left(z_2\right)\right)$$

(9)

where $z_1\in {\mathbb{R}}^{64}$ and $z_2\in {\mathbb{R}}^{32}$ represent the hidden latent spaces initialized by the clinical features, and $\sigma$ represents the terminal sigmoid activation block matching the operational execution configuration of the pipeline.

3.2.5. Multi-Task Loss Coordination System

UM-TOTA architecture implements advanced loss functions specifically designed to address the challenges of multi-task learning in medical image analysis. We adopted the state-of-the-art adaptive t-vMF Dice loss (AdaptiveTvMFDiceLoss) [58] for segmentation tasks, incorporating adaptive weighting mechanisms to handle class imbalance and boundary sensitivity issues common in medical segmentation. This loss function provides superior performance compared to traditional Dice loss formulations by dynamically adjusting the loss contribution based on prediction difficulty.

We used boundary enhanced Dice loss (BoundaryEnhancedDiceLoss) to specifically address the challenge of accurate boundary delineation in medical segmentation tasks. This loss function combines traditional Dice loss with boundary-aware components that penalize incorrect boundary predictions more heavily than interior regions. We implement label smoothing for classification tasks following the methodology established in [59] which provides benefits of regularization and improved calibration for medical classification problems. The loss weighting and balancing strategies employ dynamic weighting schemes to ensure balanced training across multiple tasks, preventing any single task from dominating the learning process as recommended by multi-task learning literature [60].

The multi-task loss coordination system implements a dynamic normalization strategy to stabilize concurrent parameter optimization across diverse clinical objectives. Let T represent the active subset of diagnostic objectives determined by the specific structural ablation configuration, where $T\subseteq \{\mathrm{classification},\mathrm{malignancy},\mathrm{segmentation},\mathrm{boundary},\mathrm{concepts}\}$. To protect the shared backbone from gradient imbalance and prevent any single objective from dominating the backpropagation trajectory, individual raw task losses ($L_t$) are coordinated using dynamically scaled weighting ratios:

$${\overline{w}}_t={\displaystyle \frac{w_t}{{\sum }_{i\in T}{w}_i}}$$

(10)

$$L_{\mathrm{total}}=\sum _{t\in T}{\overline{w}}_t\times L_t$$

(11)

where $w_t$ corresponds to the predefined baseline static weight assigned to task component t upon loss module initialization, and ${\overline{w}}_t$ represents the dynamically normalized relative task coefficient computed on the fly to enforce a strict unit sum constraint where ${\sum }_{t\in T}{\overline{w}}_t=1.0$ across all training folds [60].

3.3. Ablation Study Design and Task Configuration

This research implements a systematic ablation study framework to evaluate task contributions through controlled component removal. Let $\mathcal{M}$ represent the full model and $M_{-T_i}$ represent the model with task $T_i$ removed. The task contribution is quantified as:

$${\Delta }_{T_i}=\mathrm{Performance}\left(\mathcal{M}\right)-\mathrm{Performance}\left(M_{-T_i}\right)$$

(12)

The ablation variants are defined as:

$$\begin{array}{cc}\hfill \mathrm{Full}\mathrm{model}\text{:}& M_{\mathrm{full}}=\{T_{\mathrm{cls}},T_{\mathrm{mal}},T_{\mathrm{seg}},T_{\mathrm{concept}}\}\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill \mathrm{No}\mathrm{concepts}\text{:}& M_{-\mathrm{concept}}=\{T_{\mathrm{cls}},T_{\mathrm{mal}},T_{\mathrm{seg}}\}\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill \mathrm{Classification}\mathrm{only}\text{:}& M_{\mathrm{cls}}=\{T_{\mathrm{cls}},T_{\mathrm{mal}}\}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \mathrm{No}\mathrm{Classification}\text{:}& M_{-\mathrm{cls}}=\{T_{\mathrm{seg}},T_{\mathrm{concept}}\}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill \mathrm{Segmentation}\mathrm{only}\text{:}& M_{\mathrm{seg}}=\left\{T_{\mathrm{seg}}\right\}\hfill \end{array}$$

(17)

Where cls, mal, seg, concept means classification, malignancy, segmentation and concept. This systematic approach provides insight into task interactions and individual contributions. A fixed random seed ($s=42$) was applied across all variants to ensure reproducible initialization [55].

3.4. Training and Validation Protocol

We employed 5-fold stratified cross-validation [48] to preserve class distribution across splits, reporting mean and standard deviation across folds for all metrics. Statistical significance between UM-TOTA and baseline variants was assessed using paired t-tests ($p<0.05$) [53].

We used the AdamW optimizer for the model training with initial learning rate of $5\times {10}^{-5}$ and maximum learning rate of $1\times {10}^{-4}$. The learning rate was scheduled using OneCycle schedule which starts from the initial value, increases to the maximum, and then smoothly decreases. We ran the training for 250 epochs with a batch size of 16, implemented in pyTorch 2.5.1

3.5. Comprehensive Evaluation Framework

The evaluation framework uses a multi-tiered quantitative paradigm. For the multi-class categorization and malignancy detection tasks, performance criteria are measured using Accuracy, Precision, Sensitivity (Recall), F1-Score (the harmonic mean of precision and recall), and the Area Under the Receiver Operating Characteristic curve (AUC-ROC). For the pixel-wise segmentation task, accuracy is quantified using the Dice Similarity Coefficient (DSC) and the Intersection over Union (IoU). Following established medical image analysis evaluation standards [61], these indicators provide a balanced validation that remains informative under severe dataset class imbalance.

To systematically evaluate the fidelity of the intermediate explainable layer, the internal parameters of the continuous concept bottleneck are benchmarked using an absolute concept alignment metric. The Mean Absolute Error (MAE) alignment score for the j-th clinical attribute across an assessment cohort of N validation images is formulated as:

$${\mathrm{Alignment}}_{{\mathrm{concept}}_j}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left|C_{\mathrm{pred}}(i,j)-C_{\mathrm{target}}(i,j)\right|$$

(18)

where $j\in \{1,2,\dots ,10\}$ represents the specific clinical descriptor index, and N represents the total number of validation samples evaluated ($N=1,469$). The scalar value $C_{\mathrm{pred}}(i,j)\in [0,1]$ represents the sigmoid-activated continuous probability score output by the model for image i regarding concept j, while $C_{\mathrm{target}}(i,j)\in [0,1]$ represents the respective deterministic clinical target profile derived from expert consensus International Ovarian Tumor Analysis (IOTA) simple rules [62]. Values near zero indicate close alignment between the model’s concept activations and the clinical ground truth [42].

4. Results and Discussion

4.1. Multi-Task Learning Performance and Clinical Significance

We implemented the UM-TOTA framework on an ASUS TUF GAMING A15 system with AMD Ryzen 9 processor with RTX4070 GPU 12GB VRAM and 32GB RAM running Ubuntu 24.04 LTS with PyTorch 2.5.1+cu121. The performance of our UM-TOTA model, when handling all four tasks simultaneously, was promising. This outcome supports our central hypothesis that an integrated learning model can provide a form of unified diagnostic support that is valuable in a clinical setting (Section 2.3). Table 1 provides a full summary of the 5-fold stratified cross-validation results.

4.1.1. Task-Specific Performance Analysis and Comparison with related studies

For the 8-Class Tumor Classification, the system showed consistent performance across the different tumor types. The ROC analysis in Figure 4 demonstrates robust discriminative capability. Figure 4 shows respective fold wise AUC for 8-class classification as well as the mean AUC = $0.950\pm 0.009$. This level of performance alongside the specificity of $97.06\%\pm 0.15\%$ means that our approach is capable of differentiating between tumor types that look similar, like serous and mucinous cystadenomas, a known challenge in clinical practice. The F1-score of $80.24\%\pm 1.00\%$ from Table 1 also indicates a balanced performance across the tumor classes.

For the 3-Class Malignancy Detection, we observed that it has the strongest performance with an accuracy of $90.88\%\pm 1.14\%$ and specificity of $90.41\%\pm 1.60\%$. The lower specificity relative to the 8-class task is largely a consequence of the class structure: under one-vs-rest macro averaging, the 8-class setting has seven negative classes per comparison, which inflates the true-negative count, whereas the 3-class malignancy setting has fewer negative classes per comparison. The difference is therefore primarily a statistical artefact of the averaging scheme rather than a direct measure of clinical difficulty. The ROC analysis plots for the individual fold is presented in Figure 4. It can be observed that the model achieved a high discriminative power with mean AUC = $0.946\pm 0.047$, and reliably separate benign, malignant, and normal tissues. This result is comparable to clinical validation studies, for example where IOTA simple rules have shown sensitivity of $91.66\%$ and specificity of $84.84\%$ [63].

In the Tumor Segmentation task, the model produced Dice scores of $77.29\%\pm 1.29\%$, which we consider acceptable for clinical use, as shown in Figure 5. There was also a strong correlation found between the Dice and IoU scores $r=0.980$. This suggests that the model’s ability to delineate boundaries is consistent, which is important for applications like planning for surgery or monitoring treatment.

Comparing our classification accuracy of $80.26\%\pm 1.10\%$ to those reported in the existing literature, the performance can be considered competitive. For example, recent meta-analyses have reported overall sensitivities of $81\%$ (95% CI, 0.80–0.82) and specificities of $92\%$ (95% CI, 0.92–0.93) for diagnosing ovarian cancer from ultrasound imaging [64]. In addition, the study that introduced the dataset used in this work achieved an accuracy of $80.6\%$ [13].

The simultaneous training approach adopted in UM-TOTA has shown advantages when compared to separate post-hoc processing pipelines. This is reflected in the stable performance observed across all tasks and the consistent convergence patterns shown in the training curves presented in Figure 6. The Vision Transformer backbone effectively learns shared feature representations that facilitate knowledge transfer between tasks. For instance, features learned for classification contribute to improved segmentation precision, while segmentation-related features help the classification task become more boundary-aware.

To contextualize the contributions of UM-TOTA, Table 2 presents a benchmark for our framework against existing methods. Since no prior study simultaneously integrates classification, malignancy detection, segmentation, and concept-based interpretability for ovarian ultrasound, we compare our results against state-of-the-art single-task models, clinical rules, and recent multi-task frameworks from related medical imaging domains. Although, the breast ultrasound [29] and brain MRI [39] entries are reported on different datasets but they are included for cross-domain context rather than as direct comparisons on MMOTU.

By separating our results into specific tasks, the advantages become clear. Our UM-TOTA 8-class classification achieves a specificity of 97.06%, outperforming all compared methods shown in Table 2. High specificity is particularly important in medical diagnosis, as it reduces false positive rates and helps to avoid unnecessary surgical interventions for patients.

For malignancy detection, UM-TOTA achieves an accuracy of 90.88%, outperforming the 80.60% and 86.66% reported by [13] and the IOTA simple rules [63], respectively. In addition, UM-TOTA attains a sensitivity of 90.88%, which is competitive with established clinical rules while offering the added benefit of inherent interpretability.

When compared with recent multi-task learning approaches in other medical imaging domains, the unique contribution of UM-TOTA becomes evident. [29] proposed MT-BI-RADS for breast ultrasound, achieving strong classification and sensitivity performance through a multi-task framework; however, their interpretability relies on post-hoc SHAP explanations rather than inherent concept-based reasoning. Similarly, [39] developed an end-to-end multi-task architecture for brain tumor analysis combining classification and segmentation, but without any explicit interpretability mechanism.

In contrast, UM-TOTA integrates concept-based reasoning directly into the diagnostic pipeline, guided by established IOTA and O-RADS clinical guidelines. This inherent interpretability enables clinicians to understand, verify, and trust model predictions using familiar diagnostic criteria, addressing a critical requirement for clinical adoption that remains unmet by existing multi-task approaches.

4.1.2. Comparison with CNN-Based Multi-Task Baseline

To validate the ViT backbone selection with direct experimental evidence, we trained a ResNet50-based multi-task baseline (CNN-MTL) under identical experimental conditions: the same four tasks, loss functions, 5-fold stratified cross-validation protocol, and MMOTU dataset. The only difference is the backbone. The CNN used was ResNet50 with 2048-dimensional features versus ViT-Base/16 with 768-dimensional features, with task heads adapted accordingly.

Table 3 presents the results. CNN-MTL achieved 79.31% ± 2.96% 8-class classification accuracy, 90.13% ± 0.88% malignancy detection accuracy, and 77.71% ± 0.52% Dice score. Paired t-tests across the five folds confirmed no statistically significant difference between UM-TOTA and CNN-MTL on any task (classification: $p=0.178$, malignancy: $p=0.396$, segmentation: $p=0.463$). Notably, UM-TOTA demonstrated greater cross-fold stability on the classification task (±1.10% vs ±2.96%), suggesting more consistent generalisation across data splits.

Although the CNN-MTL framework with a ResNet-50 backbone demonstrated improved classification performance relative to the single-task baseline reported in our earlier study [18], this outcome is consistent with established multi-task learning theory. In particular, multi-task optimisation can improve the performance of moderately strong baseline models by encouraging shared feature representations across related auxiliary tasks. The improvements observed here are therefore in line with prior findings that multi-task learning acts as a form of implicit regularisation, leading to better generalisation in limited data settings [60,65,66]. This regularisation benefit and the gradient-interference cost discussed below are not contradictory: their net effect depends on backbone capacity, and our ablation in Section 4.3 shows that the higher-capacity ViT, already strong in the single-task setting, gains less from multi-task regularisation and is more exposed to interference between objectives.

However, the reduction in ViT accuracy from $82.98\%$ in our previous single-task benchmarks [18] to $80.26\%$ in this multi-task configuration reflects a common trend in joint optimisation settings where competing objectives introduce gradient interference. In this case, balancing a global semantic task such as tumour classification alongside localised pixel-wise segmentation introduces interference between optimisation objectives. This phenomenon can hinder the convergence path and limit the extent to which higher-capacity backbones realise their full isolated representational advantage [67].

Consequently, while joint optimisation narrows the performance gap between the ResNet-50 and the stronger ViT backbone, the ViT-based UM-TOTA model offers the strongest overall trade-off between accuracy and interpretability among the multi-task configurations evaluated. This is supported by its superior performance in the single-task setting, its ability to model globally distributed features relevant to IOTA-based reasoning, and its favourable trade-off between performance and deployment efficiency, as discussed in Section 4.1.3.

4.1.3. Deployment Efficiency Analysis

Table 4 presents concrete efficiency measurements comparing UM-TOTA as a unified model against three sequential single-task ViT models which is the realistic clinical alternative for pipelines handling classification, segmentation, and reporting independently. The comparison uses three sequential models rather than four because concept prediction in UM-TOTA is a lightweight head intrinsically coupled to the classification and malignancy heads; it does not require a separately deployed backbone, so the realistic sequential baseline comprises three deployable models. All experiments were performed on an NVIDIA RTX 4070 GPU (12 GB VRAM) using PyTorch 2.5.1, with 10 GPU warmup runs followed by 100 timed inference passes per image.

UM-TOTA processes a single ultrasound image in $8.98\pm 1.30$ ms, compared to 26.01 ms for three sequential models which is a 65.5% latency reduction. GPU memory consumption is 7.4 MB versus 20.8 MB (64.2% reduction), and throughput improves from 48.9 to 147.9 images per second (202.7% increase). The throughput measured on the hardware with batch size of 16. Execution timing are captured directly on GPU using asynchronous CUDA events (torch.cuda.Event()). These efficiency gains are directly relevant to clinical deployment, where real-time processing during ultrasound examination and GPU memory constraints at point-of-care settings are practical requirements.

4.2. Clinical Interpretability Results and Trust Building

Our UM-TOTA Clinical Reasoning module was able to learn clinical concepts that align with established IOTA and O-RADS guidelines. Figure 7 presents the real medical concept interpretability analysis. It shows correlations between learned concepts and clinical tumor characteristics that are statistically significant.

4.2.1. IOTA/O-RADS Guideline Alignment

Examining the concept activation patterns in Figure 7, the alignment with clinical criteria is notable. We distinguish here between two related but separate analyses: intra-class concept activations (the mean activation of a concept within a single tumor class) and between-group statistical comparisons (benign versus malignant), which are reported with p-values in Table 5. As an example of the intra-class activations, serous cystadenomas had high activation scores for `boundary_clarity’ (0.86), `cystic_components’ (0.84), and `homogeneous_texture’ (0.83). This is a good match for the IOTA B-features, which describe unilocular cysts with smooth walls. In contrast, high-grade serous carcinomas showed higher activations for `vascularization’ (0.53), `solid_components’ (0.58), and `papillary_projections’ (0.36), which are known M-features indicating malignancy.

Quantitative Concept Activation Analysis: To move beyond descriptive validation, we computed weighted mean concept activation values separately for benign and malignant tumours across all 1,469 samples. Statistical significance was assessed for each concept. All ten IOTA-grounded concepts showed statistically significant differences between benign and malignant cases (all $p<0.01$, eight concepts $p<{10}^{-6}$), as presented in Table 5.

The three strongest malignancy indicators were vascularization (benign: 0.167 vs malignant: 0.399, importance: $+0.375$, $p=5.48\times {10}^{-96}$), papillary projections (0.072 vs 0.204, importance: $+0.270$, $p=3.41\times {10}^{-90}$), and solid components (0.313 vs 0.644, importance: $+0.262$, $p=3.40\times {10}^{-8}$). Conversely, benign tumours showed significantly higher activations for homogeneous texture (0.687 vs 0.493, $p=2.79\times {10}^{-19}$), cystic components (0.606 vs 0.356, $p=1.93\times {10}^{-8}$), and boundary clarity (0.669 vs 0.626, $p=1.63\times {10}^{-7}$). These activation patterns are directly consistent with established IOTA simple rules and O-RADS scoring criteria [46,47], providing quantitative evidence that the model has learned clinically meaningful concept representations rather than arbitrary internal features.

Statistical Validation of Clinical Concepts: Our analysis of feature importance for malignancy confirmed that the model identified clinically relevant features as shown in Figure 7 which is called Key Concept for malignancy (*$p<0.05$, **$p<0.01$, ***$p<0.001$). `Vascularization’ was the strongest predictor of malignancy (importance: $+0.375$, $p<{10}^{-90}$). This was followed by `papillary projections’ ($+0.270$, $p<{10}^{-80}$) and `solid components’ ($+0.262$, $p<{10}^{-8}$). All of these are considered high-risk features in the O-RADS system [47]. Features associated with benign tumors showed strong negative correlations, such as `boundary_clarity’ ($-0.355$, $p<{10}^{-7}$), `cystic_components’ ($-0.338$, $p<{10}^{-8}$), and `homogeneous_texture’ ($-0.329$, $p<{10}^{-19}$).

4.2.2. Clinical Reasoning Transparency and Trust Building

Our UM-TOTA Clinical Reasoning module provides decision pathways that are transparent. This allows radiologists to validate what the system is doing, which we believe helps build trust. Figure 8 shows some example cases where the concept activations link directly to what a radiologist would see, creating an understandable connection between the UM-TOTA’s prediction and known diagnostic rules.

Attention-Based Clinical Reasoning: The model’s attention mechanism seems to weigh the importance of concepts based on their clinical relevance. It can be seen from the clinical attention weight by class heat map in Figure 9 that attention patterns also changed in an appropriate way for different tumor types. For malignant cases, the system gave more weight to vascularization and solid components. For benign cases, it focused more on boundary clarity and cystic features.

Case-by-Case Interpretability: Looking at individual predictions from Figure 10, we can see consistent pathways from concept to diagnosis, even with different kinds of tumors. The system gives an explanation for each prediction with a confidence weight. This could allow a clinician to check the AI’s reasoning against their own judgement.

4.2.3. Interpretable Multi-Task Integration

The concept bottleneck design makes it possible to have a unified clinical reasoning for all four tasks at once. The correlation analysis shown in the concept patterns graph in Figure 7 suggests that the concepts the model learns for classification are also used to inform malignancy detection and segmentation. This helps ensure the system processes cases in an integrated manner, analogous to how radiologists assess cases.

4.3. Ablation Study Insights and Technical Innovations

To better understand the contribution of each component, we conducted a series of ablation studies on five different versions of the architecture which are full_model, no_concepts, class_mal_only, segmentation_only, and no_class_no_mal. These revealed some complex patterns of how the tasks interact. The ablation variants are defined as follows:

• full_model: {classification: True, malignancy: True, segmentation: True, concepts: True}
• no_concepts: {classification: True, malignancy: True, segmentation: True, concepts: False}
• no_class_no_mal: {classification: False, malignancy: False, segmentation: True, concepts: True}
• segmentation_only: {classification: False, malignancy: False, segmentation: True, concepts: False}
• class_mal_only: {classification: True, malignancy: True, segmentation: False, concepts: False}

4.3.1. Task Combination Impact Analysis

The interactions between tasks were found to be complex, going beyond simple synergy as shown in Figure 11, for:

• Classification Figure: no_concepts (82.44% ± 1.94%) > class_mal_only (80.67% ± 2.25%) > full_model (80.26% ± 1.10%)
• Malignancy Figure: no_concepts (92.38% ± 0.85%) > class_mal_only (91.22% ± 1.22%) > full_model (90.88% ± 1.14%)
• Segmentation Figure: segmentation_only (80.88% ± 1.04%) > full_model (77.29% ± 1.29%) > no_concepts (75.50% ± 0.94%)

The result indicated that the models trained for a single task do better in that specific task. In medical imaging, multi-task learning often has difficulty balancing local spatial features needed for segmentation with global semantic features needed for classification [66,68]. Our segmentation-only model reached a Dice score of 80.88% (Figure 11), which is 3.6 percentage points higher than the full model. We observed that, within the multi-task setting, the segmentation ranking shows full_model (77.29%) above no_concepts (75.50%); that is, adding the concept bottleneck improves segmentation by 1.79 percentage points relative to the no-concepts multi-task variant. The segmentation penalty therefore arises from joint optimisation with the classification and malignancy tasks, not from the concept bottleneck.

The analysis of training curves for the multi-task model is illustrated in the training dynamics visualizations. The segmentation_only model has the smoothest convergence curves as shown in Figure 12a. In contrast, the full_model training curves Figure 6 and no_concepts training curves Figure 12b show more complex training patterns since the optimizer tries to balance gradients pulling in different directions thereby resulting in noisy oscillating curves. The class_mal_only model shown in Figure 12c also shows stable and smooth convergence without any degradation. With these we can deduce that segmentation introduces optimization complexity thereby causing gradient pulling in different directions.

We validated statistical significance using paired Student’s t-tests on the 5-fold cross-validation results, as shown in Table 6. For classification, the UM-TOTA model achieved an accuracy of $80.26\%\pm 1.10\%$, which showed no statistically significant difference from the task-matched class_mal_only variant ($80.67\%\pm 2.25\%$; $t=-0.43$; $p=0.692$). Similarly, malignancy detection exhibited no significant variance ($90.88\%\pm 1.14\%$ vs $91.22\%\pm 1.37\%$; $t=-0.85$; $p=0.445$). To isolate the explicit architectural impact of the concept bottleneck module, a direct baseline comparison was conducted between the full multi-task network and the concept-free variant model (no_concepts). Removing the concept layer yielded a marginal absolute increase of 2.18 percentage points in classification accuracy ($82.44\%\pm 2.17\%$) and 1.50 percentage points in malignancy detection ($92.38\%\pm 0.95\%$). Formal paired t-tests confirmed that these absolute variations are not statistically significant (classification: $t=1.85,p=0.139$; malignancy: $t=1.61,p=0.182$), indicating that the integration of the 10-concept interpretability framework provides vital clinical explainability without compromising downstream multi-class diagnostic accuracy.

By contrast, segmentation performance demonstrated a highly significant difference ($t=-9.88$; $p<0.001$). The UM-TOTA model achieved a Dice score of $77.29\%\pm 1.29\%$ compared with the specialized segmentation_only variant’s $80.88\%\pm 1.04\%$. This trade-off is visibly apparent in the cross-validation training dynamics illustrated in Figure 12, where the unified architecture balances local, pixel-level spatial dimensions against the global semantic features required for multi-class optimization. This reduction in topological accuracy is a known trade-off attributable to multi-task learning optimization conflicts. Despite this localized degradation, the unified UM-TOTA model delivers transparent, fully auditable decision-making pipelines structured directly around clinical International Ovarian Tumor Analysis (IOTA) guidelines while preserving competitive performance boundaries. We explicitly note that, with a 5-fold cross-validation design (degrees of freedom $=4$), these paired tests possess inherently conservative statistical power; non-significant results are interpreted as a validation of baseline performance stability rather than definitive absolute equivalence.

4.3.2. Technical Component Validation

Interpretability-Performance Trade-off Analysis: When we compared full_model with no_concepts variants, we observed that there was a trade-off between the 10-concept interpretability module and predictive performance on the classification and malignancy tasks. Removing the concept module increased classification accuracy by 2.18 percentage points (80.26% → 82.44%) and malignancy accuracy by 1.50 percentage points (90.88% → 92.38%); for segmentation, however, removing the concept module reduced the Dice score by 1.79 percentage points (77.29% → 75.50%). This indicates a tension between interpretability and peak per-task accuracy for the global semantic tasks, while the concept bottleneck is, if anything, beneficial for segmentation within the multi-task setting. Figure 11 shows the corresponding performance levels.

4.3.2.1. Quantified Interpretability Effect:

• Classification: 2.18 percentage points cost for clinical interpretability
• Malignancy detection: 1.50 percentage points cost for transparent reasoning
• Segmentation: 1.79 percentage points gain from concept-guided features

Advanced Loss Function Contributions: Our specific loss function design, which used an adaptive t-vMF Dice loss, a boundary-enhanced loss, and progressive task weighting, appears to have contributed to a more stable training process as seen in Figure 12. The boundary-enhanced Dice loss in particular helps to address a common problem with standard Dice loss by putting more emphasis on getting the boundaries correct.

Progressive Training Dynamics: Looking at the learning rate schedules from the plots in Figure 12 for the different models, our progressive training approach seems to be effective. The OneCycle learning rate scheduler showed consistent patterns in all variants, with smooth acceleration and controlled annealing phases.

4.3.3. Optimal Clinical Configuration

The recommendations below are practical guidance for deployment scenarios and do not retract the central argument of this paper, which is that the unified, interpretable model is the primary contribution. Based on our findings, we suggest different configurations for different operational situations:

• High-volume screening: The no_concepts variant (82.44% ± 1.94% classification, malignancy).
• Surgical planning: The segmentation_only variant (80.88% ± 1.04% Dice), as it is focused on boundary precision.
• Research/teaching: The full_model variant, which gives balanced performance and interpretable reasoning.
• Real-time decision support: The class_mal_only variant for its good performance and efficiency.

Even though the single-task models performed better on their specific tasks, the full_model still offers a critical advantage for a complete clinical workflow. Its ability to provide 8-class classification (80.26% ± 1.10%), malignancy detection (90.88% ± 1.14%), and segmentation (77.29% ± 1.29% Dice) all at once, with explainable reasoning, presents a unified framework. This eliminates the complexity and inconsistencies you get with sequential post-hoc approaches.

4.4. Comparative Analysis and Clinical Deployment

Our UM-TOTA architecture was designed to address some of the known limitations of traditional approaches where tasks are done sequentially. These limitations relate to performance consistency, workflow efficiency, and whether predictions are coherent.

As mentioned, our results are competitive with established benchmarks. Published meta-analyses show sensitivities of 81% and specificities of 92% [64]. Our result is up to par when compared with the result that created the MMOTU dataset [13] we used for this experiment. Our model’s performance for 8-class classification (80.26% ± 1.10%) and 3-class malignancy detection (90.88% ± 1.14%) is in this range, but with the added capabilities of segmentation and clinical concept interpretability, which are missing in existing single-task models.

With post-hoc approaches, there is a risk of getting conflicting results for the same patient. Our simultaneous approach ensures that all predictions originate from shared feature representation which provides the radiologists with diagnostic information that is internally consistent.

Using a sequence of models increases computational steps, memory usage, and pipeline complexity. We analyzed the parameter load of our UM-TOTA architecture compared to such a sequential pipeline. In a traditional single-task framework, performing classification, malignancy detection, and segmentation requires three separate models. If ${\theta }_{\mathrm{enc}}$ represents the backbone parameters (ViT encoder, which serves as feature extractor), this results in a total load of $3\times {\theta }_{\mathrm{enc}}$ for the three tasks. In contrast, UM-TOTA employs a shared encoder where feature extraction is performed only once ($1\times {\theta }_{\mathrm{enc}}$) per image. The reduction in computational load is calculated as:

$$\mathrm{Saved}\mathrm{Load}=\mathrm{Total}\mathrm{Work}\left(\mathrm{Sequential}\right)-\mathrm{Total}\mathrm{Work}(\mathrm{UM}-\mathrm{TOTA})$$

(19)

$$\mathrm{Saved}\mathrm{Load}=3{\theta }_{\mathrm{enc}}-1{\theta }_{\mathrm{enc}}=2{\theta }_{\mathrm{enc}}$$

(20)

$$\mathrm{Efficiency}\mathrm{Gain}\approx {\displaystyle \frac{\mathrm{Saved}\mathrm{Load}}{\mathrm{Sequential}\mathrm{Load}}}={\displaystyle \frac{2{\theta }_{\mathrm{enc}}}{3{\theta }_{\mathrm{enc}}}}\approx 66.7\%$$

(21)

This shared encoder strategy reduces the backbone computational load by approximately two-thirds (66.7%). This efficiency advantage, combined with the preservation of diagnostic accuracy of the classification and malignancy, justifies the segmentation performance trade-off observed in Section 4, as similarly demonstrated in other medical imaging applications [65]. Table 4 summarises all the efficiency improvements.

4.4.1. State-of-the-Art Positioning and Clinical Adoption Advantages

To our knowledge, our approach is the first to use a clinically-guided concept learning framework for ovarian ultrasound analysis, by building IOTA and O-RADS knowledge directly into the model. The 10 medical concepts we used map directly to clinical assessment criteria [47,62]. This helps radiologists validate the model’s predictions using a framework they are already familiar with.

The concept activation patterns show relationships that are clinically meaningful. For instance, malignant tumors are associated with increased vascularization ($+0.375$ importance), papillary projections ($+0.270$), and solid components ($+0.262$). Benign lesions, in contrast, are associated with higher boundary clarity ($-0.355$ importance, i.e., higher boundary clarity indicates benignity) and cystic characteristics ($-0.338$). This kind of clinical consistency is important for building radiologist confidence and for addressing a major barrier to AI adoption, namely trust in how the algorithm makes its decisions.

4.5. Limitations and Future Directions

Despite the promising results, this study has certain limitations. First, our evaluation was based on data from a single centre (the MMOTU dataset), which is small for training a Vision Transformer from scratch. Although we used a transfer learning strategy with ImageNet weights to stabilise training and prevent overfitting, validation with data from multiple institutions is needed to confirm that our model can generalise, since clinics differ in their ultrasound machines and patient populations.

Second, our current analysis is based on 2D images. This omits volumetric information present in 3D ultrasound, which could provide additional context.

Third, the clinical concept annotations, which we based on the IOTA and O-RADS guidelines, may carry some subjectivity, as they rely on expert interpretation. Relatedly, the 8-to-3 class malignancy mapping in Equation (1) is a clinical simplification; categories assigned here to the benign group can in practice be borderline or malignant (for example, immature teratomas, sex-cord-stromal theca cell tumors, and borderline mucinous cystadenomas). This mapping was agreed with our clinical co-author (A.A.H.) as a tractable target for malignancy detection.

Lastly, our ablation study focused on task contribution analysis rather than backbone architecture comparison.

Future investigation should examine the integration of multiple data types, such as combining 2D ultrasound with Doppler flow or 3D data, which presents a viable avenue for enriching the diagnostic context. We also suggest the exploration of weakly supervised learning to automatically discover new clinical concepts. The UM-TOTA architecture could also be extended to other gynaecological imaging problems beyond ovarian tumors.

5. Conclusion

In this work, we addressed the limitation of current AI systems that handle OC classification, segmentation, and interpretability as separate post-hoc processes, which is inefficient. We proposed the UM-TOTA architecture, which performs these tasks simultaneously. This unified multi-task approach contributes a coherent diagnostic pipeline in which predictions are derived from shared feature representations, unlike fragmented baselines.

Our model achieved competitive performance for 8-class classification (80.26% ± 1.10% accuracy, 97.06% one-vs-rest macro specificity), 3-class malignancy detection (90.88% ± 1.14% accuracy, 90.41% specificity), and segmentation (77.29% ± 1.29% Dice). We note that the 97.06% value is a multiclass macro-averaged specificity and is not directly comparable with the binary clinical specificities reported elsewhere (for example, 84.84% for the IOTA rules [63] and 92% in meta-analysis [64]). Most importantly, our architecture achieves an approximately 66.7% reduction in computational parameter load compared with deploying equivalent sequential models, demonstrating significant efficiency for clinical environments. The model also produced interpretability that aligns with clinical guidelines such as IOTA and O-RADS, correctly identifying key malignancy predictors such as vascularization, solid components, and papillary projections. These results were achieved using the loss functions we developed and a progressive training strategy that supported stable multi-task learning.

The genuine contribution of UM-TOTA is not state-of-the-art per-task accuracy, since single-task models outperform it on each individual task in our ablation, but rather a unified, interpretable, and efficient framework. Its central novelty is concept-based interpretability embedded directly in the decision process: each prediction is accompanied by clinically grounded concept activations and attention weights that map to IOTA and O-RADS criteria, allowing clinicians to audit the model’s reasoning using familiar diagnostic descriptors. The primary clinical significance therefore extends beyond model performance, to improving radiologist workflow and trust through the provision of integrated and interpretable predictions within a single system.

Further research is needed to fully realise this potential. We suggest that integrating multi-modal data, such as Doppler flow or 3D ultrasound, represents a viable avenue for enriching the diagnostic context. Furthermore, external validation on multi-center clinical dataset is necessary to confirm that the model performs consistently across different patient populations and real world diagnostic environments. We hope this work serves as a model for developing medical AI that effectively supports clinical expertise for better OC patient outcomes.

Abbreviations

The following abbreviations are used in this manuscript:

AI	Artificial Intelligence
AUC	Area Under the Curve
CA125	Cancer Antigen 125
CBM	Concept Bottleneck Model
CI	Confidence Interval
CNN	Convolutional Neural Network
CNN-MTL	CNN-based Multi-Task Learning baseline
DSC	Dice Similarity Coefficient
IoU	Intersection over Union
IOTA	International Ovarian Tumor Analysis
LIME	Local Interpretable Model-agnostic Explanations
MMOTU	Multi-Modality Ovarian Tumor Ultrasound
MTL	Multi-Task Learning
OC	Ovarian Cancer
O-RADS	Ovarian-Adnexal Reporting and Data System
ROC	Receiver Operating Characteristic
SHAP	SHapley Additive exPlanations
TCAV	Testing with Concept Activation Vectors
UM-TOTA	Unified Multi-Task Ovarian Tumor Architecture
ViT	Vision Transformer
VRAM	Video Random Access Memory
XAI	Explainable Artificial Intelligence