MDD-Net: Medical Segmentation with Parallel Denoising Transformation

1. Introduction

Glioma is a specific type of brain tumor that originates from glial cells. It is the most commonly diagnosed brain tumor and is associated with the highest mortality rate among brain tumors. The World Health Organization (WHO) classifies gliomas into four grades: low-grade gliomas (LGG), encompassing classes I and II, and high-grade gliomas (HGG), comprising classes III and IV. High-grade gliomas are particularly aggressive and pose significant threats to patient survival. Annually, approximately 190,000 new cases of glioma are diagnosed worldwide [6],and the prognosis remains poor, with around 90% of patients succumbing to the disease within 24 months following surgical resection [18]. Accurate segmentation of the tumor is crucial for both the planning of radiotherapy treatments and the ongoing diagnostic monitoring of disease progression. However, manual segmentation is a labor-intensive process that is subject to inter- and intra-observer variability, leading to inconsistencies and uncertainties in the resulting annotations. These challenges underscore the need for automated or semi-automated segmentation methods to enhance treatment quality and improve the efficiency of managing patients with glioma.

The quest for reliable automatic segmentation of brain tumors in multimodal Magnetic Resonance Imaging (MRI) scans has led to the establishment of the Brain Tumor Segmentation (BraTS) challenge [2,3,4,5,14,40,41,48]. BraTS is an annual competition, often associated with the Medical Image Computing and Computer Assisted Intervention (MICCAI) conference, designed to benchmark and evaluate state-of-the-art methods in brain tumor segmentation. Participants in BraTS are provided with MRI datasets encompassing four different structural MRI modalities: T1-weighted contrast-enhanced (T1c), T2-weighted (T2), T1-weighted (T1), and Fluid-Attenuated Inversion Recovery (FLAIR). These diverse modalities offer complementary information that aids in the comprehensive analysis and precise segmentation of brain tumors. The ground truth segmentation masks are created through manual annotations by one to four raters, which are subsequently refined by expert annotators to ensure high quality. The performance of the submitted segmentation algorithms is assessed using several metrics, including the Dice Similarity Coefficient (DSC), sensitivity, specificity, and the 95th percentile of the Hausdorff Distance (HD95).

Since the introduction of the U-Net architecture by Ronneberger et al. [16], Convolutional Neural Networks (CNNs) with skip connections have become the foundational architecture for medical image segmentation tasks. Numerous variations and extensions of the U-Net have been proposed to tackle the specific challenges posed by brain tumor segmentation. For instance, in the BraTS 2019 challenge, Jiang et al. [11], the first-place winners, introduced an end-to-end two-stage cascaded U-Net. This approach involves a coarse segmentation stage followed by a fine segmentation stage, enabling the model to capture both the overall structure and finer details of the tumor substructures. Similarly, Zhao et al. [21], who secured second place in the same challenge, employed a variety of techniques for 3D MRI brain tumor segmentation, including advanced preprocessing methods, innovative model design strategies, and optimized training procedures. McKinley et al. [13] presented DeepSCAN, an adaptation of their earlier 3D-to-2D Fully Convolutional Network (FCN), which replaces batch normalization with instance normalization and incorporates a lightweight local attention mechanism. This modification contributed to DeepSCAN achieving third place in the BraTS 2019 competition.

Building upon the foundation laid by previous research, our proposed architecture extends the work presented in [20,58], where the TUNet was introduced during the BraTS 2019 challenge. Although TUNet demonstrated commendable performance, it has notable limitations, primarily due to its composition of three cascaded networks. This architectural complexity makes it difficult to process full volumetric MRI data, typically of dimensions $240\times 240\times 155$, within the memory constraints of contemporary Graphics Processing Units (GPUs). As a result, TUNet adopts a patch-based segmentation approach, which significantly increases training times. Additionally, the reliance on multiple cascaded networks may impede the model’s ability to capture global contextual information within the MRI scans.

Motivated by the effectiveness of cascaded network architectures, as demonstrated in studies such as [11,20], this work introduces a novel multi-decoder architecture, termed MDD-Net (Multi-Decoder Denoising Network), aimed at decomposing the complex task of brain tumor segmentation into simpler, more manageable sub-tasks. In addition to the multi-decoder framework, we propose the utilization of multiple denoised versions of the original MRI images as inputs to the network. The rationale behind this approach is to mitigate the impact of salt-and-pepper noise, a common artifact in MRI scans [1], thereby enhancing the robustness and accuracy of the segmentation process. To the best of our knowledge, this represents the first application of multi-denoising inputs in the context of brain tumor segmentation.

We hypothesize that MDD-Net will address overfitting issues by employing a shared encoder across three distinct decoders, each responsible for a specific aspect of the segmentation task. Furthermore, the inclusion of denoised MRI images is expected to provide the network with enhanced insights into the multimodal data, improving its ability to accurately delineate tumor boundaries. Specifically, the network leverages two additional versions of the input images: one processed with a median filter to eliminate salt-and-pepper noise, and another subjected to a low-pass Gaussian filter to attenuate high-frequency components. This dual-denoising strategy is designed to refine the input data, thereby facilitating more precise and reliable segmentation results.

The proposed MDD-Net aims to offer a comprehensive solution for brain tumor segmentation by integrating advanced denoising techniques with a versatile multi-decoder architecture. By doing so, it seeks to improve segmentation accuracy, reduce computational overhead, and enhance the model’s generalizability across diverse MRI datasets.

2. Related Work

The segmentation of brain tumors, particularly gliomas, from MRI scans is a critical task in medical imaging, facilitating accurate diagnosis, treatment planning, and monitoring of disease progression. Due to the complex nature of brain anatomy and the heterogeneous appearance of tumors across different MRI modalities, developing robust and reliable segmentation methods remains a significant challenge in the field. Automated segmentation techniques aim to overcome the limitations of manual delineation, such as time consumption, subjectivity, and variability among annotators, by providing consistent and efficient results [6,62].

The Brain Tumor Segmentation (BraTS) challenge [2,3,4,5,14,64] has emerged as a pivotal benchmark for evaluating and advancing brain tumor segmentation algorithms. Organized annually in conjunction with the Medical Image Computing and Computer Assisted Intervention (MICCAI) conference, BraTS provides a standardized dataset comprising multimodal MRI scans, including T1-weighted contrast-enhanced (T1c), T2-weighted (T2), T1-weighted (T1), and Fluid-Attenuated Inversion Recovery (FLAIR) images. These datasets are accompanied by meticulously annotated segmentation masks, which serve as ground truth for assessing the performance of participating algorithms using metrics such as the Dice Similarity Coefficient (DSC), sensitivity, specificity, and the 95th percentile of the Hausdorff Distance (HD95) [14].

The introduction of the U-Net architecture by Ronneberger et al. [16] marked a significant milestone in medical image segmentation. U-Net’s encoder-decoder structure with skip connections effectively captures both high-level contextual information and fine-grained details, making it well-suited for delineating complex anatomical structures like brain tumors. Since its inception, numerous variants and extensions of U-Net have been proposed to enhance its performance and adaptability to specific segmentation tasks. For instance, Myronenko [15,70] introduced an autoencoder regularization approach to improve the robustness of 3D MRI brain tumor segmentation, demonstrating the versatility of the U-Net framework in handling volumetric data.

Building upon the foundational U-Net architecture, several studies have explored cascaded and multi-stage network designs to further enhance segmentation accuracy. Jiang et al. [11,49], the first-place winners of the BraTS 2019 challenge, proposed a two-stage cascaded U-Net that performs coarse segmentation in the initial stage, followed by fine segmentation to accurately delineate tumor substructures. Similarly, Vu et al. [20] introduced TuNet, an end-to-end hierarchical segmentation network that leverages multiple cascaded networks to capture both global and local features. Zhao et al. [21,78,79], who secured second place in the same challenge, incorporated various techniques such as advanced preprocessing methods, innovative model designs, and optimized training strategies to achieve superior performance in 3D MRI brain tumor segmentation. These cascaded approaches highlight the importance of hierarchical processing and multi-scale feature extraction in tackling the intricate task of brain tumor segmentation.

In addition to architectural innovations, preprocessing techniques play a crucial role in enhancing the quality of MRI data for segmentation tasks. Noise reduction, particularly the removal of salt-and-pepper noise, is essential for improving the reliability of segmentation outcomes. Ali et al. [1,81] proposed a novel method to eliminate salt-and-pepper noise in MRI images, which is a common artifact that can obscure tumor boundaries and degrade segmentation performance. By applying effective denoising algorithms, the clarity and consistency of MRI scans are significantly improved, thereby facilitating more accurate and robust segmentation by subsequent neural network models.

Recent advancements have also focused on integrating attention mechanisms and data augmentation strategies to bolster the performance of segmentation networks. Hu et al. [8,56,57] introduced Squeeze-and-Excitation (SE) networks, which adaptively recalibrate channel-wise feature responses by explicitly modeling interdependencies between channels. This mechanism enhances the network’s ability to focus on relevant features, thereby improving segmentation accuracy. Furthermore, Isensee et al. [9] developed the BatchGenerators framework, a versatile Python tool for data augmentation that facilitates the training of more generalized and resilient models by introducing variability in the training data. These techniques, when combined with robust network architectures, contribute to the development of highly effective brain tumor segmentation models.

Overall, the landscape of brain tumor segmentation has been significantly shaped by the interplay of innovative network architectures, effective preprocessing techniques, and comprehensive benchmarking through challenges like BraTS. The proposed MDD-Net aims to build upon these advancements by introducing a multi-decoder architecture combined with enhanced denoising inputs, thereby addressing the existing limitations and pushing the boundaries of automated brain tumor segmentation further.

3. Proposed Method

In response to the limitations identified in previous approaches [20], we introduce MDD-Net (Multi-Decoder Denoising Network), an end-to-end framework designed to simplify the complex task of multi-class tumor segmentation into three distinct binary segmentation tasks. This architectural innovation addresses memory constraints inherent in prior models, enabling the processing of entire input volumes without resorting to patch-based segmentation. Consequently, MDD-Net leverages global contextual information more effectively and reduces training times significantly. The following sections detail the various components and methodologies employed in our proposed framework.

3.1. Encoder Architecture

The encoder component of MDD-Net is built upon conventional convolutional blocks, similar to those introduced in the U-Net architecture [16]. Each convolutional block comprises a three-dimensional convolution layer with a kernel size of $3\times 3\times 3$, followed by batch normalization and a Leaky Rectified Linear Unit (LeakyReLU) activation function. To enhance feature representation, each convolutional block is succeeded by a Squeeze-and-Excitation Block (SEB), which adaptively recalibrates channel-wise feature responses [8]. Max-pooling layers are employed for downsampling, reducing the spatial dimensions while preserving essential features.

The initial number of convolutional filters is set to twelve, corresponding to the three denoising methods applied to the four input MRI modalities. This design choice ensures that the encoder effectively captures diverse features from the multimodal inputs. The encoder output tensor has a shape of $96\times 20\times 24\times 16$, encapsulating rich hierarchical features essential for accurate segmentation. The detailed architecture of the encoder network is presented in Table 1.

3.2. Decoder Architecture

The decoder component of MDD-Net consists of three separate decoder paths, each dedicated to segmenting a specific tumor region: whole tumor (W-Net), core tumor (C-Net), and enhancing tumor (E-Net). This multi-decoder setup allows the network to focus on distinct aspects of the tumor, enhancing segmentation accuracy and reducing inter-task interference. Each decoder path mirrors the encoder architecture with corresponding upsampling and convolutional operations. Skip connections, akin to those in U-Net, are integrated to retain high-resolution features from the encoder, facilitating precise localization. After each convolutional block or concatenation operation, a Squeeze-and-Excitation Block (SEB) is employed to recalibrate feature maps dynamically.

To enrich the feature maps in the C-Net and E-Net, outputs from the W-Net and C-Net decoders, respectively, are concatenated at corresponding levels. This hierarchical integration ensures that higher-level decoders benefit from the refined features captured by preceding decoders, promoting a cohesive segmentation process across different tumor regions. The detailed architecture of the decoder networks is illustrated in Table 2.

Each decoder path incorporates skip connections that concatenate high-resolution features from the encoder with upsampled features in the decoder. This design facilitates the retention of spatial information, crucial for precise boundary delineation. Additionally, the SEB modules following each convolutional block enable the network to focus on the most informative channels, enhancing feature discrimination and segmentation performance.

To mathematically formalize the decoder operations, consider the upsampling process denoted by $U(·)$ and the concatenation operation represented by ⊕. For a given decoder block $D_l$, where l indicates the current layer level, the feature maps can be expressed as:

$$D_l=\mathrm{SEB}\left(\mathrm{Conv}3\left(U\left(D_{l+1}\right)\oplus E_l\right)\right)$$

(1)

where $E_l$ represents the encoder features at layer l, and $D_{l+1}$ denotes the decoder features from the subsequent higher layer. This formulation underscores the hierarchical and interconnected nature of the multi-decoder architecture, facilitating the propagation of refined features across different segmentation tasks.

3.3. Channel Attention Mechanism

To further enhance feature representation, MDD-Net integrates Squeeze-and-Excitation Blocks (SEBs) after each convolutional and concatenation operation [8]. The SEB mechanism adaptively recalibrates channel-wise feature responses by modeling interdependencies between channels. This is achieved through a sequence of operations: global average pooling, fully connected layers, a ReLU activation, another fully connected layer, and a sigmoid activation function. Mathematically, for a given feature map $X\in {\mathbb{R}}^{C\times H\times W\times D}$, the SEB operations can be described as:

$$S\left(X\right)=\sigma \left(W_2·\mathrm{ReLU}\left(W_1·\mathrm{GAP}\left(X\right)\right)\right)$$

(2)

$$\widehat{X}=S\left(X\right)\otimes X$$

(3)

where $\mathrm{GAP}\left(X\right)$ denotes global average pooling, $W_1$ and $W_2$ are the weights of the fully connected layers, $\sigma$ represents the sigmoid function, and ⊗ denotes channel-wise multiplication. This mechanism allows the network to emphasize informative features while suppressing less relevant ones, thereby improving segmentation accuracy.

3.4. Input Denoising Techniques

A critical aspect of MDD-Net is the incorporation of multiple denoised versions of the original MRI inputs to mitigate noise-related artifacts common in medical imaging, such as salt-and-pepper noise [1]. The denoising process involves two distinct methods: median filtering and Gaussian smoothing. Specifically, each MRI modality undergoes the following transformations:

• Median Filtering: A $3\times 3\times 3$ median filter is applied to eliminate salt-and-pepper noise, preserving edges while removing outlier pixel values.
• Gaussian Smoothing: A $3\times 3\times 3$ Gaussian filter with a standard deviation of 0.5 is employed to reduce high-frequency noise, resulting in a smoother image.

These denoised images are then concatenated with the original raw images, resulting in a total of twelve input channels (three denoised versions for each of the four MRI modalities). This multi-denoising input strategy enhances the network’s ability to learn robust features by providing complementary representations of the input data. The comprehensive input tensor can be represented as:

$$I_{\mathrm{input}}=[I_{\mathrm{raw}},I_{\mathrm{median}},I_{\mathrm{Gaussian}}]$$

(4)

where $I_{\mathrm{raw}}$, $I_{\mathrm{median}}$, and $I_{\mathrm{Gaussian}}$ correspond to the raw, median-filtered, and Gaussian-smoothed images, respectively.

This approach not only mitigates the impact of noise but also enriches the feature space, allowing the network to leverage both detailed and smoothed representations for more accurate segmentation.

3.5. Data Preprocessing and Augmentation

Prior to inputting the data into MDD-Net, all MRI scans undergo normalization to achieve zero mean and unit variance, ensuring uniform intensity distribution across different modalities and subjects. This normalization is critical for stabilizing the training process and improving convergence rates.

To enhance the generalizability of the model and mitigate overfitting, extensive data augmentation is performed on-the-fly during training [9]. The augmentation pipeline includes the following transformations:

• Random Rotation: Images are randomly rotated within a range of $[-1^{\circ },1^{\circ }]$ along all three spatial axes to simulate variations in patient orientation.
• Random Flip: Mirror flipping is applied with a probability of 0.5 on each axis, introducing symmetry variations.
• Elastic Transformation: With a probability of 0.3, elastic deformations are introduced to mimic realistic anatomical variations. The displacement field $\Delta$ for elastic transformation is defined as:

  $$R_w=R_o+\alpha \Delta$$

  (5)

  where $R_w$ and $R_o$ denote the coordinates of a voxel in the warped and original image, respectively, and $\alpha$ is the displacement strength. The displacement $\Delta$ is sampled from a uniform distribution $\mathcal{U}(-1,1)$ for each spatial dimension and convolved with a Gaussian kernel with a standard deviation $\sigma =0.25$.
• Random Scaling: Images are randomly scaled within the range $[0.9,1.1]$ with a probability of 0.3 to account for size variability in tumor presentations.
• Random Cropping and Resizing: With a probability of 0.3, random cropping followed by resizing ensures that the network learns to handle variations in tumor localization and size.

These augmentation strategies introduce significant variability in the training data, enabling MDD-Net to generalize better to unseen data and enhancing its robustness against diverse imaging conditions.

3.6. Post-processing Techniques

Post-processing is employed to refine the segmentation outputs of MDD-Net, particularly addressing challenges in distinguishing between low-grade glioma (LGG) and high-grade glioma (HGG) patients. A common issue in segmentation tasks is the misclassification of small enhancing tumor regions, which may be erroneously labeled as edema or necrosis.

To address this, we adopt a strategy similar to our previous work [20]. Specifically, any enhancing tumor region comprising fewer than 500 connected voxels is reclassified as necrosis. Mathematically, for each connected component C in the enhancing tumor segmentation:

$$C^{\prime }=\left\{\begin{array}{cc}\mathrm{Necrosis},\hfill & \mathrm{if}\left|C\right|<500,\hfill \\ \mathrm{Enhancing}\mathrm{Tumor},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(6)

This post-processing step mitigates false positives in small tumor regions, thereby improving the overall segmentation accuracy and reliability of the model’s predictions.

3.7. Uncertainty Quantification in Segmentation

The BraTS challenge introduced the task of “Quantification of Uncertainty in Segmentation” to evaluate the confidence of segmentation predictions [20]. The goal is to reward models that exhibit high confidence in correct predictions and maintain uncertainty in incorrect ones. Participants are required to generate uncertainty maps with values ranging from 0 (most certain) to 100 (most uncertain). The evaluation metrics include the Dice-based Area Under the Curve (DAUC), Relative False Positive Threshold (RFTP), and Relative False Negative Threshold (RFTN).

In MDD-Net, uncertainty is quantified based on the predicted probability maps for each tumor region. For each voxel $(i,j,k)$ and tumor region $r\in \mathcal{R}$ (where $\mathcal{R}$ includes whole, core, and enhancing tumor regions), the uncertainty score $u_{i,j,k}^r$ is defined as:

$$u_{i,j,k}^r=\left\{\begin{array}{cc}200\times (1-p_{i,j,k}^r),\hfill & \mathrm{if}{p}_{i,j,k}^r\ge 0.5,\hfill \\ 200\times p_{i,j,k}^r,\hfill & \mathrm{if}{p}_{i,j,k}^r<0.5,\hfill \end{array}\right.$$

(7)

where $p_{i,j,k}^r\in [0,1]$ is the predicted probability for voxel $(i,j,k)$ belonging to tumor region r. This formulation ensures that higher uncertainty is assigned to lower probability predictions, aligning with the objective of accurate uncertainty quantification.

The resulting uncertainty maps provide valuable insights into the confidence of the model’s predictions, enabling clinicians to make informed decisions and potentially guiding further diagnostic procedures.

3.8. Loss Function and Optimization

To train MDD-Net, a combination of loss functions is employed to balance the segmentation accuracy and uncertainty estimation. Specifically, we utilize the Dice loss for each binary segmentation task, combined with a weighted cross-entropy loss to address class imbalance inherent in medical imaging datasets. The overall loss function $\mathcal{L}$ can be expressed as:

$$\mathcal{L}=\sum _{r\in \mathcal{R}}\left({\mathcal{L}}_{\mathrm{Dice}}(p^r,g^r)+\lambda {\mathcal{L}}_{\mathrm{CE}}(p^r,g^r)\right)$$

(8)

where $p^r$ and $g^r$ denote the predicted probability map and ground truth mask for tumor region r, respectively, and $\lambda$ is a weighting factor balancing the two loss components.

For optimization, we employ the Adam optimizer [12], which adapts the learning rate for each parameter based on first and second moments of the gradients. The optimization process is governed by the following update rule:

$${\theta }_{t+1}={\theta }_t-\eta {\displaystyle \frac{{\widehat{m}}_t}{\sqrt{{\widehat{v}}_t}+ϵ}}$$

(9)

where ${\theta }_t$ represents the model parameters at iteration t, $\eta$ is the learning rate, ${\widehat{m}}_t$ and ${\widehat{v}}_t$ are the bias-corrected first and second moment estimates, and $ϵ$ is a small constant to prevent division by zero.

Training is conducted for a predefined number of epochs, with early stopping based on validation loss to prevent overfitting. Additionally, learning rate scheduling is applied to reduce the learning rate upon plateauing of the validation loss, facilitating finer convergence in later stages of training.

4. Experiment

In this section, we detail the experimental framework employed to evaluate the performance of the proposed MDD-Net (Multi-Decoder Denoising Network) for brain tumor segmentation. The experiments encompass the implementation specifics, training protocols, loss functions, optimization strategies, and comprehensive evaluation metrics used to assess the effectiveness of our model. Additionally, we present the results obtained from various validation and testing phases, accompanied by a discussion of their implications.

4.1. Implementation and Training Procedure

The MDD-Net was developed using the Keras library with TensorFlow serving as the backend framework. This combination was chosen for its robustness and extensive support for deep learning architectures, facilitating efficient model development and training. The implementation was executed on high-performance NVIDIA Tesla V100 GPUs provided by the High-Performance Computing Center North (HPC2N) at Umeå University, Sweden, ensuring rapid computation and handling of large volumetric MRI datasets.

To ensure the reliability and generalizability of our results, seven independent instances of MDD-Net were trained from scratch. Each model was subjected to 200 training epochs ($N_e=200$) with a mini-batch size of one, optimizing the model’s ability to learn from the data without overfitting. The extensive training regimen, which spanned approximately six days per model, was necessary to accommodate the complexity of the multi-decoder architecture and the high-dimensional input data.

During training, early stopping mechanisms were employed based on the validation loss to prevent overfitting and to terminate training once the model’s performance plateaued. Additionally, learning rate scheduling was integrated to adjust the learning rate dynamically, allowing for finer convergence in later training stages. Gradient clipping was also utilized to maintain training stability and prevent the occurrence of exploding gradients, thereby enhancing the robustness of the optimization process.

4.2. Loss Function Design

Accurate segmentation performance is pivotal for clinical applications, necessitating a carefully designed loss function that balances precision and recall while addressing class imbalances inherent in medical imaging data. To this end, MDD-Net employs a hybrid loss function combining the Dice Similarity Coefficient (DSC) loss and Cross-Entropy (CE) loss, thereby leveraging the strengths of both metrics.

The DSC loss is defined as:

$$D(u,v)={\displaystyle \frac{2·|u\cap v|}{\left|u\right|+\left|v\right|}},$$

(10)

where u and v represent the predicted segmentation and the corresponding ground truth, respectively. To incorporate DSC into the loss function, we utilize the soft DSC loss, formulated as:

$${\mathcal{L}}_{\mathrm{DSC}}(u,v)={\displaystyle \frac{-2{\sum }_i{u}_i{v}_i}{{\sum }_i{u}_i+{\sum }_i{v}_i+ϵ}},$$

(11)

where $u_i$ is the softmax output for label i, $v_i$ is the one-hot encoded ground truth label, and $ϵ=1\times {10}^{-5}$ is a small constant to prevent division by zero.

Recognizing the challenge of class imbalance, especially with smaller tumor regions, the CE loss is incorporated to smooth the loss surface and enhance convergence. The CE loss is defined as:

$${\mathcal{L}}_{\mathrm{CE}}(u,v)=-\sum _i{v}_{i}log\left(u_i\right).$$

(12)

The hybrid loss function, therefore, is the sum of the soft DSC loss and the CE loss:

$${\mathcal{L}}_{\mathrm{hybrid}}(u,v)={\mathcal{L}}_{\mathrm{DSC}}(u,v)+{\mathcal{L}}_{\mathrm{CE}}(u,v).$$

(13)

For multi-class segmentation tasks, the final loss function aggregates the hybrid loss across all tumor regions (whole, core, and enhancing regions):

$$\mathcal{L}(u,v)=\sum _{r\in \mathcal{R}}{\mathcal{L}}_{\mathrm{hybrid}}(u_r,v_r),$$

(14)

where $\mathcal{R}$ denotes the set of tumor regions.

Furthermore, segmentation performance is evaluated using the 95th percentile of the Hausdorff Distance (HD95), which quantifies the boundary alignment between the predicted segmentation and the ground truth:

$$H(u,v)=max\left\{d\right(u,v),d(v,u\left)\right\},$$

(15)

where

$$d(u,v)=\underset{u_i\in u}{max}\underset{v_i\in v}{min}{\parallel u_i-v_i\parallel }_2,$$

(16)

with ${\parallel ·\parallel }_2$ representing the Euclidean distance.

4.3. Optimization Strategy

Optimizing MDD-Net involves fine-tuning the model parameters to minimize the defined loss function effectively. We employ the Adam optimizer [12], renowned for its adaptive learning rate capabilities, which combine the benefits of both AdaGrad and RMSProp optimizers. The Adam optimizer was initialized with a learning rate (${\alpha }_0$) of $1\times {10}^{-4}$, and momentum parameters ${\beta }_1=0.9$ and ${\beta }_2=0.999$, facilitating rapid convergence and efficient handling of sparse gradients.

To further enhance the optimization process, a learning rate decay schedule was implemented, reducing the learning rate as training progresses. Specifically, the learning rate at epoch e is adjusted according to the following formula:

$${\alpha }_e={\alpha }_0·{\left(1-{\displaystyle \frac{e}{N_e}}\right)}^3,$$

(17)

where $N_e=200$ is the total number of epochs. This decay strategy allows the model to take larger steps initially, accelerating convergence, and smaller steps later, refining the parameter updates for improved accuracy.

Additionally, $L_2$ regularization with a penalty parameter of $1\times {10}^{-5}$ was applied to all convolutional layers. This regularization technique mitigates overfitting by penalizing large weights, encouraging the model to learn more generalized features from the training data.

The final layer of MDD-Net employs a sigmoid activation function, which maps the output to a probability distribution, facilitating the interpretation of segmentation results as probabilities.

4.4. Evaluation Metrics

To comprehensively assess the performance of MDD-Net, we utilize a suite of evaluation metrics commonly adopted in the BraTS challenge. These metrics provide insights into both the accuracy and reliability of the segmentation results.

4.4.1. Dice Similarity Coefficient (DSC)

The DSC measures the overlap between the predicted segmentation and the ground truth:

$$\mathrm{DSC}(u,v)={\displaystyle \frac{2|u\cap v|}{\left|u\right|+\left|v\right|}},$$

(18)

where u and v represent the predicted and ground truth segmentations, respectively. A higher DSC indicates greater similarity between the two segmentations.

4.4.2. Hausdorff Distance (HD95)

The HD95 quantifies the boundary discrepancy between the predicted segmentation and the ground truth by calculating the 95th percentile of the Hausdorff Distance:

$$\mathrm{HD}95(u,v)=max\left\{\mathrm{HD}95(u,v),\mathrm{HD}95(v,u)\right\},$$

(19)

where

$$\mathrm{HD}95(u,v)=\underset{u_i\in u}{max}\underset{v_i\in v}{min}{\parallel u_i-v_i\parallel }_2.$$

(20)

A lower HD95 signifies better boundary alignment between the predicted and ground truth segmentations.

4.4.3. Uncertainty Quantification Metrics

In addition to segmentation accuracy, uncertainty quantification is pivotal for clinical decision-making. The BraTS challenge introduced specific metrics to evaluate the quality of uncertainty maps generated by segmentation models:

-

Dice-based Area Under the Curve (DAUC): Measures the relationship between segmentation accuracy and uncertainty.

-

Relative False Positive Threshold (RFTP): Evaluates the model’s ability to express uncertainty in false positive regions.

-

Relative False Negative Threshold (RFTN): Assesses the model’s capability to indicate uncertainty in false negative regions.

These metrics are computed over the entire brain volume and provide a nuanced understanding of the model’s confidence in its predictions.

4.5. Experimental Results

The performance of MDD-Net was rigorously evaluated through a series of experiments, encompassing cross-validation on the training set, validation on a separate dataset, and testing on an unseen test set. The results, presented in Table 3, Table 4, Table 5, Table 6 and Table 7, demonstrate the efficacy of the proposed model across various metrics.

4.5.1. Cross-Validation Performance

Table 3 summarizes the mean DSC and HD95 scores, along with their standard deviations, obtained from five-fold cross-validation on the training dataset comprising 396 cases. The results indicate that incorporating denoised inputs significantly enhances segmentation performance across all tumor regions. Specifically, MDD-Net with denoising outperforms the baseline U-Net models, achieving higher DSC scores and lower HD95 values, thereby confirming the benefits of the multi-denoising input strategy and the multi-decoder architecture.

4.5.2. Validation Set Evaluation

Table 4 presents the segmentation performance of MDD-Net on the validation set containing 125 cases. The model achieved mean DSC scores of 90.55%, 82.67%, and 77.17% for the whole tumor, core tumor, and enhancing tumor regions, respectively. Corresponding HD95 scores were 4.99, 8.63, and 27.04, indicating precise boundary delineation. Although these results are slightly lower than those of the top-ranking teams, they still reflect robust segmentation capabilities of MDD-Net.

4.5.3. Uncertainty Quantification on Validation Set

Table 5 details the performance of MDD-Net in the uncertainty quantification task on the validation set. The model achieved DAUC scores of 92.59, 83.61, and 78.83 for the whole, core, and enhancing tumor regions, respectively. The RFTP scores of 4.48, 10.13, and 7.95, along with RFTN scores of 0.27, 0.17, and 0.08, demonstrate the model’s ability to effectively quantify uncertainty, particularly excelling in reducing false negatives.

4.5.4. Test Set Performance

Table 6 showcases the segmentation results of MDD-Net on the test set comprising 166 cases. The model achieved mean DSC scores of 88.26%, 82.49%, and 80.84% for the whole tumor, core tumor, and enhancing tumor regions, respectively. The corresponding HD95 scores were 6.30, 22.27, and 20.06, reflecting consistent performance across unseen data.

4.5.5. Uncertainty Quantification on Test Set

Table 7 presents the uncertainty quantification results on the test set. MDD-Net attained DAUC scores of 90.61, 85.83, and 83.03 for the whole, core, and enhancing tumor regions, respectively. The RFTP scores of 4.18, 5.49, and 4.45, alongside RFTN scores of 0.31, 1.68, and 0.07, indicate a high degree of confidence in correct predictions and appropriately high uncertainty in incorrect ones. Notably, MDD-Net secured the second position in the Quantification of Uncertainty in Segmentation task within the BraTS challenge, underscoring its effectiveness in uncertainty management.

4.6. Discussion

The experimental results underscore the superior performance of MDD-Net in brain tumor segmentation tasks, particularly highlighting the advantages of the multi-decoder architecture and the incorporation of denoised inputs. The significant improvement in DSC scores and reduction in HD95 values across all tumor regions attest to the model’s enhanced capability in capturing both global contextual information and fine-grained details essential for accurate segmentation.

The use of multiple denoised versions of the input MRI scans effectively mitigates noise-related artifacts, such as salt-and-pepper noise, thereby facilitating more robust feature extraction. This approach not only improves segmentation accuracy but also contributes to the model’s resilience against variations in imaging conditions. The integration of Squeeze-and-Excitation Blocks (SEBs) further refines feature representations by emphasizing informative channels, thereby enhancing the model’s discriminative power.

Moreover, the multi-decoder framework allows MDD-Net to decompose the complex multi-class segmentation problem into simpler binary tasks, each focusing on specific tumor regions. This decomposition enhances the model’s ability to specialize and accurately delineate different tumor substructures, leading to improved overall segmentation performance.

The successful ranking in the uncertainty quantification task demonstrates MDD-Net’s proficiency in not only producing accurate segmentations but also in providing reliable uncertainty estimates. This capability is crucial for clinical applications, where understanding the confidence of segmentation results can inform decision-making and guide further diagnostic procedures.

However, despite these strengths, there are areas for potential improvement. The slightly lower performance compared to the top-ranking teams in the validation set suggests that further enhancements, such as more sophisticated data augmentation techniques or ensemble methods, could be explored to boost performance. Additionally, optimizing the training process to reduce computational overhead and training time remains a valuable avenue for future research.

In conclusion, the proposed MDD-Net offers a robust and efficient solution for automated brain tumor segmentation, leveraging advanced architectural designs and denoising strategies to achieve high accuracy and reliable uncertainty quantification. These advancements hold significant promise for clinical applications, potentially improving diagnostic precision and patient outcomes.

4.7. Analysis of Segmentation Performance

The cross-validation results presented in Table 3 demonstrate the substantial improvement achieved by MDD-Net with denoising inputs over the traditional U-Net architectures. The incorporation of denoised inputs not only enhances the model’s ability to capture intricate tumor structures but also contributes to more stable and accurate segmentation outcomes, as evidenced by the higher DSC scores and lower HD95 values across all tumor regions.

On the validation set, MDD-Net achieved mean DSC scores of 90.55%, 82.67%, and 77.17% for the whole tumor, core tumor, and enhancing tumor regions, respectively. These results indicate a robust performance, particularly in segmenting the whole and core tumor regions, which are critical for treatment planning and prognostic assessments. The HD95 scores further corroborate the model’s precision in delineating tumor boundaries, with lower values signifying minimal boundary discrepancies.

In the uncertainty quantification task, MDD-Net exhibited exceptional performance, achieving high DAUC scores and low RFTP and RFTN scores. This indicates that the model not only provides accurate segmentations but also reliably quantifies uncertainty, distinguishing between confident correct predictions and uncertain incorrect ones. Such capability is invaluable in clinical settings, where understanding the confidence of segmentation results can guide further diagnostic and therapeutic decisions.

4.8. Comparison with State-of-the-Art Methods

When compared to other top-performing models in the BraTS challenge, MDD-Net holds its own by delivering competitive segmentation accuracy and superior uncertainty quantification. While some models achieved slightly higher DSC scores on the validation set, MDD-Net excels in providing reliable uncertainty estimates, as evidenced by its second-place ranking in the Quantification of Uncertainty in Segmentation task.

The multi-decoder architecture of MDD-Net allows for specialized processing of different tumor regions, which may contribute to its robust performance across various segmentation tasks. Additionally, the integration of denoised inputs ensures that the model remains resilient to noise and artifacts commonly present in MRI scans, a critical factor for clinical applicability.

4.9. Implications for Clinical Practice

The advancements demonstrated by MDD-Net have significant implications for clinical practice. Automated and accurate brain tumor segmentation can streamline radiotherapy planning, enhance diagnostic precision, and facilitate longitudinal monitoring of disease progression. The ability to quantify uncertainty further empowers clinicians by providing insights into the reliability of segmentation results, enabling more informed decision-making.

Moreover, the memory-efficient design of MDD-Net allows for the processing of full volumetric MRI data without the need for patch-based segmentation, reducing computational overhead and accelerating the workflow. This efficiency, combined with the model’s high accuracy and reliable uncertainty estimates, positions MDD-Net as a valuable tool in the clinical setting, potentially improving patient outcomes through enhanced diagnostic and therapeutic strategies.

While MDD-Net has demonstrated substantial performance improvements, there are avenues for further enhancement. Future work may explore the integration of additional imaging modalities, such as diffusion-weighted imaging (DWI) or perfusion imaging, to provide more comprehensive tumor characterization. Additionally, incorporating advanced data augmentation techniques and ensemble learning strategies could further boost segmentation accuracy and model robustness.

Another promising direction is the application of MDD-Net to other types of brain tumors or different anatomical regions, broadening its clinical applicability. Furthermore, refining the uncertainty quantification mechanisms to provide more granular insights could enhance the model’s utility in clinical decision-making processes.

Despite its strengths, MDD-Net has certain limitations that warrant consideration. The extensive training time of approximately six days per model poses challenges for rapid deployment and iterative model refinement. Additionally, while the model performs well on the BraTS dataset, its generalizability to other datasets with differing imaging protocols and patient populations remains to be thoroughly evaluated.

Furthermore, the reliance on a fixed number of denoising techniques may limit the model’s adaptability to varying noise characteristics in different imaging settings. Addressing these limitations through optimized training protocols, adaptive denoising strategies, and broader validation studies will be essential for advancing the clinical utility of MDD-Net.

5. Conclusion and Future Work

In this study, we introduced MDD-Net (Multi-Decoder Denoising Network), a novel architecture designed to segment tumor substructures from multimodal brain Magnetic Resonance Imaging (MRI) scans. By decomposing the intricate task of multi-class tumor segmentation into three simpler binary segmentation problems, MDD-Net effectively enhances both the accuracy and efficiency of the segmentation process. The network architecture is inspired by the U-Net framework, incorporating Squeeze-and-Excitation (SE) blocks after each convolution and concatenation operation to dynamically recalibrate channel-wise feature responses. This integration of SE blocks significantly improves the model’s ability to focus on the most informative features, thereby enhancing segmentation performance.

A key innovation of our approach is the stacking of original MRI images with their denoised counterparts. By employing denoising techniques, we enrich the input data, enabling the network to learn more robust and noise-invariant features. This strategy not only mitigates the adverse effects of common MRI artifacts, such as salt-and-pepper noise [1], but also contributes to substantial improvements in both the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95) metrics. Specifically, MDD-Net demonstrated a marked enhancement in segmentation performance, achieving DSC scores of 88.26%, 82.49%, and 80.84% and HD95 scores of 6.30, 22.27, and 20.06 for the whole tumor, tumor core, and enhancing tumor core regions, respectively, on the test set.

Moreover, MDD-Net excelled in the task of Quantification of Uncertainty in Segmentation, securing a top-two position. This achievement underscores the model’s capability not only to produce accurate segmentations but also to provide reliable uncertainty estimates. Such uncertainty quantification is crucial for clinical decision-making, as it offers insights into the confidence of the segmentation results, thereby aiding clinicians in assessing the reliability of automated annotations.

The comprehensive evaluation of MDD-Net on the BraTS dataset highlights its competitive performance relative to existing state-of-the-art methods. The ability to process entire volumetric MRI data without resorting to patch-based segmentation approaches, as required by previous models like TUNet [20], results in more efficient training and inference times. This efficiency, coupled with the enhanced segmentation accuracy, positions MDD-Net as a valuable tool for clinical applications, potentially streamlining workflows in radiotherapy planning and diagnostic follow-up.

While MDD-Net has demonstrated significant advancements in brain tumor segmentation, several avenues for future research remain. One potential direction is the exploration of additional denoising techniques and their impact on segmentation performance. By integrating more sophisticated noise reduction methods, such as non-local means or deep learning-based denoising algorithms, we can further enhance the quality of input data and, consequently, the accuracy of the segmentation results.

Another promising area is the incorporation of attention mechanisms beyond the Squeeze-and-Excitation blocks. For instance, integrating spatial attention modules could enable the network to better focus on relevant anatomical regions, thereby improving the delineation of tumor boundaries. Additionally, experimenting with transformer-based architectures, which have shown remarkable success in various computer vision tasks, may offer further enhancements to the segmentation capabilities of MDD-Net.

Expanding the applicability of MDD-Net to other types of brain tumors or even different anatomical structures is another worthwhile pursuit. This generalization would require adapting the network to handle diverse tumor morphologies and imaging modalities, potentially involving multi-task learning approaches to accommodate varying segmentation tasks simultaneously.

Furthermore, enhancing the uncertainty quantification aspect of MDD-Net presents an opportunity to develop more nuanced and interpretable uncertainty maps. Techniques such as Monte Carlo dropout or Bayesian neural networks could be integrated to provide probabilistic interpretations of the segmentation results, offering deeper insights into model confidence and reliability.

Lastly, conducting extensive clinical validations and collaborating with medical professionals will be essential to translate MDD-Net from a research prototype to a clinically deployable tool. Such collaborations can provide valuable feedback on the model’s performance in real-world settings and guide further refinements to meet the stringent requirements of clinical practice.

In summary, MDD-Net represents a significant step forward in automated brain tumor segmentation, offering improved accuracy and reliable uncertainty estimates. Future enhancements and broader applications of this framework hold the potential to make substantial contributions to the field of medical imaging and patient care.