Submitted:
28 December 2025
Posted:
30 December 2025
You are already at the latest version
Abstract
Tumor boundary delineation is particularly challenging due to the presence of infiltrative margins, low-contrast regions, and ambiguous transitions between healthy and diseased tissue. DBMT-Net addresses these issues with a hybrid architecture that combines the global contextual modeling capability of Mamba with the fine-grained boundary refinement strength of a Transformer decoder. A dynamic routing mechanism selectively emphasizes global or local cues depending on boundary uncertainty, enabling adaptive focus near ambiguous margins. On BraTS2023 (1,350 training and 350 validation scans), DBMT-Net yields a Boundary-F1 of 0.812, improving upon TransBTS (0.723, +12.3%), while overall Dice increases from 0.887 to 0.917 (+3.0%). HD95 decreases from 18.4 mm to 11.7 mm (−36.4%), and boundary uncertainty is reduced by 18.6%.
Keywords:
tumor margin detection
; boundary refinement
; Mamba-Transformer hybrid
; fuzzy boundary modeling
; BraTS2023
; MRI tumor segmentation
1. Introduction
Accurate delineation of tumor margins is essential for neurosurgical resection, radiotherapy planning, and longitudinal treatment assessment. In glioma imaging, however, the boundary between tumor tissue and normal brain parenchyma is often ambiguous. Infiltrative growth, peritumoral edema, and weak contrast on magnetic resonance imaging (MRI) frequently produce gradual transitions rather than sharp edges. As a result, manual annotations show substantial variability, with frequent disagreement occurring near tumor borders rather than in the tumor core [1,2]. Recent reviews consistently report that the majority of segmentation errors are concentrated along these uncertain margins, underscoring the clinical and methodological importance of boundary-aware modeling [3]. Deep learning has substantially improved global tumor segmentation performance on public benchmarks such as BraTS. U-Net–based architectures enhanced with multi-scale features, residual connections, or channel–spatial attention mechanisms often achieve high Dice scores across tumor subregions [4]. Nevertheless, these models continue to struggle with thin, irregular, and low-contrast margins. Their reliance on predominantly local receptive fields can lead to overly smooth contours and loss of fine boundary details. Transformer-based approaches address some of these limitations by capturing long-range spatial dependencies and have demonstrated improved separation of tumor subregions [5]. More recent work further indicates that explicitly emphasizing lesion centers and learning structured representations can stabilize segmentation behavior and improve robustness in regions with ambiguous appearance [6]. Despite these advances, most existing models still optimize region-level objectives and do not adapt their processing strategy according to local boundary uncertainty. A number of studies have introduced explicit boundary-aware components to mitigate contour errors. Auxiliary edge prediction branches, boundary-sensitive loss functions, and contour refinement modules have been shown to sharpen segmentation outputs and improve boundary alignment [7,8]. While effective to some extent, these designs typically apply fixed corrections across the entire image and do not change their behavior in response to spatially varying uncertainty. Other approaches incorporate distance maps, confidence maps, or uncertainty estimates to highlight unreliable regions [9]. However, such cues are usually treated as auxiliary outputs rather than being used to dynamically regulate how global context and local detail are combined during inference. State space models, including Mamba, have recently emerged as efficient alternatives to Transformers for long-range dependency modeling. These architectures provide large receptive fields with lower computational cost and have shown competitive performance in organ and tumor segmentation tasks [10,11]. Their ability to model global structure is particularly useful for capturing overall tumor shape, orientation, and spatial extent. At the same time, Transformer-based decoders remain effective for enhancing fine-grained features and refining local details, albeit with increased sensitivity to noise in low-contrast regions [12]. Recent hybrid architectures that combine convolutional, Transformer, and Mamba components demonstrate strong region-level performance on BraTS-style datasets [13,14]. However, few of these models are explicitly designed to address fuzzy tumor margins or to coordinate global and local cues based on boundary clarity.
Existing literature therefore reveals several persistent limitations. Boundary quality is often underemphasized relative to global overlap metrics such as Dice score, even though metrics that directly reflect contour accuracy, including Boundary-F1 and contour distance, are more sensitive to clinically relevant errors. Boundary-aware and uncertainty-aware modules generally apply uniform strategies across the image and lack mechanisms to adjust their influence when boundary ambiguity varies spatially [15]. In addition, current Mamba-based and hybrid models do not provide explicit coordination between global structure modeling and local boundary refinement in response to uncertainty.
This study proposes DBMT-Net, a Dynamic Boundary-aware Mamba–Transformer network designed specifically for glioma segmentation with fuzzy margins. The proposed framework leverages Mamba blocks to capture global tumor structure and spatial coherence, while a Transformer-based decoder focuses on refining local boundary details. A dynamic routing module is introduced to adaptively balance global and local features according to a learned boundary-uncertainty map. This mechanism enables the model to rely more on global context in well-defined regions and to emphasize local detail where tumor boundaries are ambiguous. We evaluate DBMT-Net on the BraTS2023 dataset using not only Dice score and HD95, but also boundary-sensitive metrics such as Boundary-F1 and uncertainty-based measures. The results demonstrate that dynamically coordinating global and local information leads to more accurate and anatomically consistent tumor margin delineation, highlighting the potential of uncertainty-aware hybrid modeling for clinically meaningful glioma segmentation.
2. Materials and Methods
2.1. Sample Collection and Study Area
This study used 1,350 training and 350 validation subjects from the BraTS2023 dataset. Each subject had T1, T1-contrast, T2, and FLAIR scans collected under routine clinical protocols. All images were aligned, resampled to a fixed spacing, and checked for complete brain coverage. The dataset included tumors with both clear and infiltrative margins, allowing analysis across a wide range of boundary appearances. Scans with missing modalities or severe motion were excluded.
2.2. Experimental Design and Control Groups
The scans were divided into training and validation groups with no overlap. The training group was used to learn all parts of DBMT-Net, including the Mamba encoder, Transformer decoder, and the routing module. The validation group was used to measure performance. Two control models were trained: a simple CNN model without boundary modules, and a Transformer-only model without Mamba blocks or routing. These comparisons allowed us to examine how global modeling, boundary refinement, and adaptive routing each affected performance. All models were trained with the same data augmentation settings.
2.3. Measurement Procedures and Quality Control
All MRI sequences were checked for correct alignment and stable intensity ranges. Expert labels for whole tumor, tumor core, and enhancing regions served as the reference. Before training, each scan was normalized with percentile-based scaling to reduce scanner differences. A second check confirmed that all four modalities remained aligned after preprocessing. During training, small rotations, flips, and mild contrast changes were applied to reduce overfitting while keeping tumor margins visually realistic. Scans that failed alignment or showed strong artifacts were removed.
2.4. Data Processing and Model Formulas
The four MRI sequences were stacked as a multi-channel input. Let be the input and the predicted mask. The model learned a mapping
Where is the set of learnable parameters. Segmentation accuracy was measured by the Dice score:
where is the reference mask. A boundary loss was used to emphasize edge regions. Let be the reference boundary map and the predicted boundary map. The loss was
Small isolated components outside the main tumor area were removed in post-processing to avoid false positives near weak boundaries.
2.5. Statistical Analysis
All results were reported as mean and standard deviation. DBMT-Net and the control models were compared using paired statistical tests. Boundary-F1, Dice, and HD95 were analyzed separately to capture differences in both region-level and boundary-level accuracy. Confidence intervals were estimated with bootstrap sampling. A value of p < 0.05 was considered statistically significant.
3. Results and Discussion
3.1. Overall Segmentation and Boundary Accuracy
DBMT-Net shows clear gains over the comparison models on the BraTS2023 validation set. Boundary-F1 increases from 0.723 to 0.812, and Dice rises from 0.887 to 0.917. HD95 decreases from 18.4 mm to 11.7 mm, which indicates fewer large boundary errors. These improvements appear across subjects with different tumor sizes and shapes.
Figure 1.
Boundary-F1, Dice, and HD95 of DBMT-Net compared with the other models on the BraTS2023 dataset.
Figure 1.
Boundary-F1, Dice, and HD95 of DBMT-Net compared with the other models on the BraTS2023 dataset.

3.2. Performance Near Fuzzy or Low-Contrast Tumor Margins
The largest gains appear in regions with unclear or infiltrative margins. Boundary uncertainty decreases by 18.6%, and visual review shows fewer missing edges and fewer small gaps along the tumor border. In many low-contrast cases, TransBTS produces smooth but slightly shifted contours. DBMT-Net follows the visible boundary more closely and preserves thin extensions that are often lost by the baseline model [16].
Figure 2.
Example tumor boundaries from DBMT-Net and TransBTS with the corresponding distance profiles.
Figure 2.
Example tumor boundaries from DBMT-Net and TransBTS with the corresponding distance profiles.

3.3. Contribution of Mamba Blocks, Transformer Decoding, and Dynamic Routing
Ablation results show that each part of DBMT-Net plays a different role. When the Mamba encoder is replaced by a standard CNN encoder, both Dice and Boundary-F1 drop, and HD95 increases. This indicates that long-range information helps maintain coherent tumor shape, especially when the tumor has complex geometry. When the Transformer decoder is removed, Dice remains close, but boundary scores fall, showing that local refinement is still needed for accurate edge placement. Several recent studies report similar findings when refinement modules are removed from 3D networks [17]. The largest loss occurs when dynamic routing is replaced with fixed fusion weights. Under this setting, Boundary-F1 approaches the TransBTS level. This suggests that ambiguous margins require not only strong global and local features, but also the ability to adjust their relative weight based on boundary clarity—an aspect that most previous hybrid models do not include [18].
3.4. Generalization, Limitations, and Directions for Future Work
Although DBMT-Net shows good performance on BraTS2023, some limitations remain. BraTS provides preprocessed, aligned, and skull-stripped scans, which are not always representative of routine clinical data. Previous multi-center studies have shown that models trained only on BraTS may lose accuracy when applied directly to scans with different noise levels or missing sequences [19,20]. DBMT-Net also relies on consensus labels and does not reflect inter-rater variation, which tends to be highest near infiltrative margins. In addition, this work focuses on segmentation and does not yet test how improved boundary detection affects prognosis or treatment planning. Future work may include external validation on independent cohorts, training under domain shift, adding uncertainty estimates to flag unclear boundary regions, and linking margin-related features with outcome models. These extensions may help assess the clinical value of more accurate tumor boundary detection.
4. Conclusion
This study introduces DBMT-Net, a model designed to improve tumor boundary detection in multi-modal MRI. The network combines long-range signals from Mamba blocks with local edge cues from a Transformer decoder and adjusts their influence through a simple routing step. On the BraTS2023 dataset, DBMT-Net shows higher Boundary-F1 and Dice and a clear drop in HD95 compared with the reference models. These gains suggest that adaptive use of global and local information helps when tumor margins are weak or infiltrative. The method may support tasks that depend on accurate boundary placement, including surgical planning and treatment monitoring. The work has several limits: the dataset is preprocessed, label variation between experts is not modeled, and external clinical testing remains to be done. Future work may include evaluation on routine hospital scans, the use of uncertainty maps to mark unclear regions, and the study of how boundary features relate to patient outcomes.
References
- Joskowicz, L.; Cohen, D.; Caplan, N.; Sosna, J. Inter-observer variability of manual contour delineation of structures in CT. European radiology 2019, 29, 1391–1399. [Google Scholar] [CrossRef] [PubMed]
- Rozenblatt-Rosen, O.; Regev, A.; Oberdoerffer, P.; Nawy, T.; Hupalowska, A.; Rood, J. E.; Ashenberg, O.; Cerami, E.; Coffey, R.J.; Demir, E.; et al. The human tumor atlas network: charting tumor transitions across space and time at single-cell resolution. Cell 2020, 181, 236–249. [Google Scholar] [CrossRef] [PubMed]
- Raposo, H. (2024). Intelligent imaging: A systematic review of artificial intelligence techniques in disease detection, segmentation, and classification. Segmentation, and Classification (May 13, 2024).
- Zha, D.; Gamez, J.; Ebrahimi, S. M.; Wang, Y.; Verma, N.; Poe, A. J.; White, S.; Shah, R.; Kramerov, A.A.; Sawant, O.B.; et al. Oxidative stress-regulatory role of miR-10b-5p in the diabetic human cornea revealed through integrated multi-omics analysis. Diabetologia 2025, 1–16. [Google Scholar] [CrossRef] [PubMed]
- Shah, O. I.; Rizvi, D. R.; Mir, A. N. Transformer-Based Innovations in Medical Image Segmentation: A Mini Review. SN Computer Science 2025, 6, 375. [Google Scholar] [CrossRef]
- Tian, Y.; Yang, Z.; Liu, C.; Su, Y.; Hong, Z.; Gong, Z.; Xu, J. CenterMamba-SAM: Center-Prioritized Scanning and Temporal Prototypes for Brain Lesion Segmentation. arXiv 2025, arXiv:2511.01243. [Google Scholar]
- Wang, Y. Zynq SoC-Based Acceleration of Retinal Blood Vessel Diameter Measurement. Archives of Advanced Engineering Science 2025, 1–9. [Google Scholar] [CrossRef]
- Abdel-Nabi, H.; Ali, M.; Awajan, A.; Daoud, M.; Alazrai, R.; Suganthan, P. N.; Ali, T. A comprehensive review of the deep learning-based tumor analysis approaches in histopathological images: segmentation, classification and multi-learning tasks. Cluster Computing 2023, 26, 3145–3185. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, L.; Wen, Y.; Wu, X.; Cai, H. Precision-Engineered Nanocarriers for Targeted Treatment of Liver Fibrosis and Vascular Disorders. 2025. [Google Scholar] [CrossRef] [PubMed]
- Nzakimuena, C. B. (2020). Automated analysis of retinal and choroidal OCT and OCTA images in AMD. Ecole Polytechnique, Montreal (Canada).
- Chen, D.; Liu, S.; Chen, D.; Liu, J.; Wu, J.; Wang, H.; Su, Y.; Kwak, G.; Zuo, X.; Rao, D.; et al. A two-pronged pulmonary gene delivery strategy: a surface-modified fullerene nanoparticle and a hypotonic vehicle. Angewandte Chemie International Edition 2021, 60, 15225–15229. [Google Scholar] [CrossRef] [PubMed]
- Jameel, U.; Belcari, N. High-Fidelity CT Image Denoising with De-TransGAN: A Transformer-Augmented GAN Framework with Attention Mechanisms. Bioengineering 2025, 12, 1350. [Google Scholar] [CrossRef] [PubMed]
- Gui, H.; Fu, Y.; Wang, B.; Lu, Y. Optimized Design of Medical Welded Structures for Life Enhancement. 2025. [Google Scholar] [CrossRef]
- Jeon, Y. S., Yang, H., Fu, H., & Feng, M. (2025). Teaching ai the anatomy behind the scan: Addressing anatomical flaws in medical image segmentation with learnable prior. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 24024-24033).
- Xie, Y., Liao, H., Zhang, D., & Chen, F. (2022, September). Uncertainty-aware cascade network for ultrasound image segmentation with ambiguous boundary. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 268-278). Cham: Springer Nature Switzerland.
- Wang, Y.; Wen, Y.; Wu, X.; Cai, H. Comprehensive Evaluation of GLP1 Receptor Agonists in Modulating Inflammatory Pathways and Gut Microbiota. 2024. [Google Scholar] [CrossRef] [PubMed]
- Deng, T., Huang, M., Xu, K., Lu, Y., Xu, Y., Chen, S., ... & Sun, X. (2024). LEGEND: Identifying Co-expressed Genes in Multimodal Transcriptomic Sequencing Data. bioRxiv, 2024-10.
- Nalisnick, E.; Matsukawa, A.; Teh, Y. W.; Gorur, D.; Lakshminarayanan, B. Hybrid models with deep and invertible features. In International Conference on Machine Learning; PMLR, 2019; pp. 4723–4732. [Google Scholar]
- Dong, C. Genetic and environmental influences on human brain changes in ageing. Doctoral dissertation, University of New South Wales (Australia), 2024. [Google Scholar]
- Zha, D.; Mahmood, N.; Kellar, R. S.; Gluck, J. M.; King, M. W. Fabrication of PCL Blended Highly Aligned Nanofiber Yarn from Dual-Nozzle Electrospinning System and Evaluation of the Influence on Introducing Collagen and Tropoelastin. ACS Biomaterials Science & Engineering 2025. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.