GuideNet: Leveraging UNet as Guidance for Anatomical Landmark-Based Breast Segmentation in Thermograms

1. Introduction

Technological developments in the use of artificial intelligence for automating the analysis of clinical data have revolutionized its application in the medical field. These advancements enable more efficient and accurate decision-making processes, thereby improving patient care and minimizing intra- and inter-operator variability. Among the various medical fields benefiting from AI, oncology is one where these technologies hold significant promise, particularly in improving early detection and diagnosis of complex diseases. One such disease is breast cancer, which remains a major global health challenge. It is the most common cancer in women and the leading cause of cancer-related deaths worldwide, with an estimated 2.3 million new cases and approximately 685,000 deaths reported in 2020, according to the World Health Organization (WHO) [1]. In Mexico, it is the most frequent malignancy and the leading cause of cancer-related deaths among women, with an upward trend over the last decade. In 2020, breast cancer accounted for 16.5% of all cancer cases diagnosed in women and resulted in over 7,800 deaths, according to the Mexican National Institute of Statistics and Geography (INEGI) [2]. These figures underscore the urgent need for effective early detection methods and advanced diagnostic techniques to address this public health problem.

Medical imaging plays a pivotal role in breast cancer analysis, enabling the non-invasive extraction of detailed tissue and metabolic activity information across various spatial scales. This versatility is achieved through multiple modalities: Microscopy, which measures optical density through reflection, scattering, and light absorption at the cellular level [3]; Magnetic Resonance Imaging (MRI), regarded as the gold standard for soft tissue visualization due to its ability to assess molecular density related to hydrogen proportions [4]; X-ray and Computed Tomography (CT), which analyze radiological attenuation to evaluate structures such as bones, air interfaces, and contrast media [5]; Ultrasound, offering insights into acoustic impedance at soft tissue boundaries [6]; PET, SPECT and Scintigraphy, which detect metabolic activity by tracing radiomarker uptake in targeted cells [7]; and Thermography, which reflects surface thermal patterns influenced by hybrid metabolic-anatomical processes [8].

These modalities, depending on their physical principles, allow the extraction of relevant data with different levels of specificity and sensitivity.

In particular, thermography has gained relevance due to its practicality, the absence of ionizing radiation, and its cost-benefit ratio, making it a valuable initial screening pre-diagnostic tool [9]. To improve the use of this technique as a complementary screening tool, it is necessary to automate the thermogram analysis process. This will depend on an accurate delineating of breast contours from thermal images.

Early approaches to this challenge relied on traditional computer vision techniques, such as thresholding and edge detection, to extract these contours [10,11]. However, these methods present significant limitations, particularly under high noise condition, low resolution images [12]. In addition to the anatomical differences between patients.

In the literature, several machine learning methods, such as region-based clustering and active contour models [13,14] has been explored for breast thermography segmentation. Techniques like K-means clustering and Gaussian Mixture Models (GMM) [15] have been used to identify and segment breast regions by grouping pixels based on their thermal properties and spatial features. Additionally, machine learning models, such as decision trees combined with feature extraction techniques [16], have also been proposed. However, these methods often require intensive manual feature engineering and may not generalize well to varying imaging conditions or patient anatomies, limiting their robustness.

Deep learning has opened new possibilities for breast segmentation in thermography. Convolutional Neural Network (CNN) architectures such as U-Net [17], SegNet [18], and customized variants have proven effective in extracting hierarchical features that enable precise segmentation of complex structures, such as breast contours. However, the adaptation of these architectures to the context of thermographic images remains limited, with only a few studies attempting to customize these models to improve accuracy in detecting and segmenting thermal patterns specific to breast cancer [19].

In this work, we propose a hybrid approach that combines traditional computer vision algorithms with deep neural networks, using anatomical references extracted directly from the same image to optimize segmentation. This method seeks to overcome the limitations of previous approaches by increasing the precision and robustness of thermographic image analysis. Our contribution focuses on demonstrating how this hybridization significantly improves segmentation compared to exclusively traditional or deep learning-based methods.

The remainder of this paper is structured as follows: Section 2 describes the proposed methodology, including data preprocessing, neural network architecture, and training procedures. Section 3 presents the experimental results and comparative analysis with existing methods. Finally, Section 4 concludes the paper with a discussion of findings, limitations, and future directions for research.

2. Materials and Methods

In this section, we describe the key features of thermal maps (according to anatomic breast) that are essential to identify the breast area limits, looking to develop a method with a high reproducibility, which is based on richness of the breast structures. Then, we describe out our hybridized approach, which integrates neural networks with traditional computer vision methods, to achieve a more robust segmentation by effectively combining their strengths.

2.1. Anatomic Breast Structure

In a previous work, we proposed a method based exclusively on traditional computer vision to segment breasts in thermal images [20]. This approach enables the evaluation of intra- and inter-breast temperatures, where variations in heterogeneity are often related to pathological angiogenesis (e.g., tumors) or other abnormal thermal asymmetries (e.g., mastitis). In thermal maps, temperature changes induce a contrast between anatomical regions, which are analized based on three classic thermal patterns to define breast boundaries: armpits are characterized by a high concentration of sweat glands, limited ventilation, and proximity to major blood vessels like the axillary artery; body is defined by constant metabolic activity and heat distribution through blood circulation, contributing to thermal regulation; and inframamary region which exhibits a V-shaped thermal contrast—referred to here as the ’inverse V region’—is characterized by skin-to-skin contact, lack of ventilation, sweat glands, and thermal insulation provided by adipose tissue, see Figure 1.

The thermal patterns of the body edges and armpits exhibit high, reproducible contrast under controlled conditions, such as the lower background temperature and the exposure of warm armpits when the arms are raised. While the inframammary region displays significant variability due to its V-shaped thermal contrast, which changes based on breast size, these patterns remain detectable. In contrast, the supramammary border is either virtually nonexistent or highly ambiguous, making it significantly more challenging to define.

Building upon this, by leveraging the identified patterns, we can partially segment the breast masks using state-of-the-art neural network models for segmentation. Furthermore, these results enable the refinement of the upper breast boundary through traditional computer vision techniques, enhancing the overall accuracy and robustness of the segmentation process.

2.2. Breast Segmentation

The proposed method consists of two main stages: (1) Multiclass segmentation of the expanded inframammary-lateral mask, including axillary markers, and (2) definition of the supramammary boundaries. These stages are detailed as follows.

The first stage involves supervised segmentation using a 2D dense multiscale U-Net [21]. This architecture is a variant of the traditional U-Net architecture, designed to enhance feature extraction and segmentation accuracy by incorporating two key innovations: dense connectivity and multi-scale convolutional blocks. The main components of the model are described below:

• Dense Double Convolutional Blocks: Each downsampling and upsampling operation includes a dense connection between intermediate layers, ensuring efficient reuse of features and promoting richer feature extraction.
• Multi-Scale Convolutional Bottleneck: At the bottleneck, the model employs a multi-scale convolutional block that processes input features with kernels of varying sizes (3x3, 5x5, and 7x7). This enables the model to capture features at different spatial scales, improving its ability to detect patterns of varying sizes in medical images.
• Encoder-Decoder Structure: The model retains the encoder-decoder architecture of the original U-Net, with symmetrical downsampling and upsampling paths. The skip connections ensure spatial details are preserved during the reconstruction process.
• Deep Supervision: By integrating dense connectivity and multi-scale processing, the model strengthens its ability to capture both local and global context, which is critical for medical image segmentation tasks.
• Efficient Regularization: Dropout layers are added in the downsampling path to reduce overfitting and improve generalization.

The output of this model is a multiclass segmentation mask, denoted ${\alpha }_{mask}(bg,br,ar)$ that distinguishes between the background ($bg$), the breasts ($br$) extending from their lower boundary to the upper limit of the body with no constraints except for the lateral edge), and the armpits ($ar$), see Figure 2.

In the second stage, with the extracted contour from ${\alpha }_{mask}\left(br\right)$, where boundary pixels are identified based on their neighborhood connectivity, the inter-inframammary inflection point is detected through further analysis.

Pixels with incomplete connectivity within a $3\times 3$ neighborhood are marked as boundary points and organized into an ordered list of coordinates $(x_i,y_i)$. This contour serves as the basis for further analysis, including curvature estimation and inflection point detection.

A localized approach is employed, analyzing curvature variations within a range of $\pm \Delta d$ around each point on the contour. Each curvature point is evaluated to detect significant changes, but specifically, the inter-inframammary inflection point, denoted $p_1$, is defined as the location where the curvature reaches a local maximum, marking a significant directional change along the contour [22].

Finally, the axillary regions (${\alpha }_{mask}\left(ar\right)$) are treated as distinct objects (i.e. a connected component). The lowest pixel attribute within these regions is projected horizontally along the x-axis (image columns) to the nearest lateral edge of ${\alpha }_{mask}\left(br\right)$. Points $p_1$ and $p_2$ are then utilized to draw a straight line, which serves to define the supramammary boundary, resulting in a complete segmentation of the breasts, see Figure 3.

Validation Metrics

To evaluate the performance of the segmentation models, the Dice Similarity Coefficient (DSC) and the Intersection over Union (IoU) metrics were utilized. These metrics are particularly suitable for binary segmentation tasks, as they focus on the overlap between the predicted and ground truth regions, which in this study were carefully annotated by medical specialists. This ensures a reliable and clinically accurate benchmark for evaluating the models.

Multiclass validation was specifically employed during the training and validation stages of our multiclass masks to ensure the proper segmentation of distinct regions, such as the background, breast areas, and axillary regions. This approach provided a robust framework to evaluate the model’s ability to handle multiple classes simultaneously, thereby capturing the complexity of the segmentation task. Furthermore, it guided the training process by influencing backpropagation and weight adaptation, ensuring the model effectively learned the unique characteristics of each class.

The use of accuracy as a validation metric was avoided, as it tends to overestimate performance in segmentation tasks where the background dominates the image. Including the true negatives of the background in accuracy calculations could overshadow errors in the segmentation of the region of interest, leading to an overly optimistic evaluation of the model’s effectiveness.

2.3. Experimentation

The dataset consists of 180 thermographic images, evenly sourced from two Mexican institutions: 90 images were collected from the Jalisco Cancer Institute (IJC) and 90 from the Institute for Social Security and Services for Mexican State Workers (ISSSTE). Further information on the dataset can be found in [23]. All images were acquired under standardized conditions as outlined by the Glamorgan Protocol [24], a protocol specifically designed to regulate thermal imaging procedures for consistency and reliability. According to this protocol, subjects were first acclimated to room temperature for a period of 15 minutes, allowing for stabilization of skin temperature to counteract frictional heating caused by the removal of clothing covering the breast area. During the thermographic capture, participants were instructed to lift their arms and place their hands behind their neck with elbows fully extended. This posture was maintained throughout the imaging process to ensure unobstructed exposure of critical regions such as the armptis and inframammary areas, essential for accurate thermal analysis.

2.3.1. Data Partitioning and Justification

The dataset, comprising 180 images, was divided into three subsets: training (70%, 126 images), validation (15%, 27 images), and testing (15%, 27 images). This partitioning strategy ensures an adequate amount of data for training while reserving sufficient samples for validation and testing, which are crucial for evaluating the model’s performance and preventing overfitting.

The training set, comprising the majority of the data, allows the model to learn the underlying patterns and generalize effectively. The validation set is employed during training to fine-tune hyperparameters, monitor for overfitting, and ensure that the model performs well on unseen data by providing an early indication of potential overfitting. The testing set remains isolated throughout the training process and is used exclusively for the final evaluation, offering an unbiased assessment of the model’s performance on new, unseen data.

This 70%-15%-15% split is widely accepted in machine learning tasks with moderate dataset sizes, striking a balance between learning, optimization, and evaluation. Additionally, images were randomly shuffled before splitting to avoid any potential biases in the subsets, further supporting the generalization of the model.

2.3.2. Data Augmentation

Data augmentation was applied as a prepropocessing stage during training and validation to enhance the diversity of the dataset while preserving anatomical realism, especially given the coronal nature of breast images. Transformations were applied to account for normal variations in human acquisitions, including width and height shifts ($\pm 20\%$), horizontal flips, rotations within $\pm {20}^{\circ }$, and zooming around the breast area (see Figure 4). These augmentations ensured that no abrupt or biologically implausible changes were introduced. Starting from the 126 training images, random variations were applied in each iteration, effectively simulating a larger dataset. Similarly, the 27 validation images were augmented to improve the robustness of hyperparameter tuning. The number of effective images processed by the network is proportional to the number of epochs ($126\times n_{epochs}$ for training and $27\times n_{epochs}$ for validation), significantly improving generalization while preventing overfitting and maintaining clinical relevance. Starting with 126 base training images, augmentation expanded the dataset to simulate a total of 12,600 images over 100 epochs. Similarly, the validation dataset, initially comprising 27 images, generated 2,700 effective images over the same epochs. The dataset was processed in small batches of 8 images each, enabling efficient backpropagation during the training stages. This batch-wise approach allowed the optimizer, Adam, to iteratively adjust the model weights based on the gradient updates, ensuring a balance between computational efficiency and learning accuracy. By exposing the network to diverse augmented samples in each batch, the setup effectively minimized overfitting risks and enhanced generalization, ultimately optimizing the network’s capacity to learn meaningful patterns from the data.

3. Results and Discussion

Table 1 compares the performance of traditional computer vision methods, a Dense Multiscale U-Net, and our proposed hybrid approach for breast segmentation, evaluated using the DSC and IoU. While neural networks demonstrate high reproducibility and achieve metrics comparable to traditional computer vision methods, their primary challenge lies in accurately defining the superior breast boundary, which in some cases appears eroded or poorly delineated, see Figure 5. This limitation is particularly evident in the Dense Multiscale U-Net results, where DSC and IoU scores reached 0.972 and 0.922, respectively, trailing slightly behind traditional methods. In contrast, our hybrid approach achieved the highest performance across both metrics, with a DSC of 0.987 and an IoU of 0.963, surpassing the computer vision method (DSC: 0.983, IoU: 0.934). This improvement underscores the effectiveness of combining traditional anatomical priors with the adaptability of neural networks, allowing for better boundary delineation and superior overall segmentation accuracy.

The hybrid method successfully mitigates the limitations of standalone approaches: traditional methods, while precise, rely heavily on handcrafted features and lack generalization; neural networks, though robust, often struggle with anatomical nuances such as the superior breast boundary. By leveraging the strengths of both, our approach not only enhances segmentation performance but also ensures clinical relevance and scalability for diverse datasets.

These results validate the hybrid strategy as a powerful solution for breast segmentation, addressing critical challenges in anatomical complexity and advancing the field of medical image analysis.

4. Conclusions

We demostrated that the proposed hybrid method has a high performance in breast segmentation, by combining deep learning with anatomical boundary definitions to address the limitations of standalone approaches. By leveraging features such as the lowest axillary point and the inframammary inflection point, this approach outperformed both traditional computer vision methods and the 2D Dense Multiscale U-Net across all metrics.

The hybrid approach combines the adaptability and automation of neural networks with the precision of domain-specific anatomical priors, such as the lowest axillary point and the inframammary inflection point. This integration allowed for more accurate and reproducible delineation of the breast boundaries, even in cases where neural networks or traditional methods alone faced limitations. Notably, the hybrid method addressed the inherent variability in anatomical features, such as the axilla-breast boundary, where automated delineation showed reproducibility in many cases but required enhanced robustness in others.

The results emphasize the importance of leveraging domain knowledge in medical imaging to address the limitations of standalone methods. Traditional computer vision methods demonstrated high precision but struggled with generalization, while deep learning models provided robust automation yet faced challenges with subtle anatomical details. By bridging these gaps, the hybrid approach offers a clinically relevant and scalable solution, particularly for datasets with diverse anatomical variability.

Beyond breast segmentation, this study underscores the broader potential of hybrid strategies in medical image analysis. Integrating machine learning with anatomical priors can enhance performance in other complex segmentation tasks, paving the way for new advancements in radiology and related fields. Future work will focus on validating the scalability of this method across larger datasets and exploring its applicability to other anatomical regions and imaging modalities.