Comparative Evaluation of Machine Learning-Based Radiomics and Deep Learning for Breast Lesion Classification in Mammography

1. Introduction

Breast cancer is the most common malignancy among women worldwide, second only to lung cancer, affecting approximately 10–12% of the female population and causing around 500,000 deaths annually [1]. The highest incidence occurs in women aged 40–49 years (41%), with a lower prevalence in those over 71 (21%). Risk factors include genetic predisposition (e.g., BRCA1/BRCA2 mutations), lifestyle, prolonged hormone therapy, therapeutic radiation, and benign proliferative breast diseases [2]. Early stages are often asymptomatic, but initial signs may include hard lesions, breast shape alterations, nipple changes, or unexplained weight loss. Advancements in therapeutic strategies and screening programs have progressively reduced mortality rates. The World Health Organization (WHO) revised its 2003 classification to improve lesion characterization, distinguishing categories such as intraductal proliferative lesions, papillary lesions, mesenchymal and fibroepithelial tumours, and rarer malignancies like lymphomas and metastases [3,4]. Additionally, abnormalities such as masses and microcalcifications, while not strictly histological, are key mammographic indicators of breast cancer risk [5]. Accurate diagnosis begins with medical history, clinical examination, and imaging techniques. Mammography remains the gold standard for early detection, particularly in asymptomatic cases, but tissue overlap can hinder interpretation, leading to false positives or negatives [6,7,8]. To address these limitations, tomosynthesis provides a near 3D breast reconstruction, reducing tissue superimposition and improving lesion detection [9,10]. Ultrasound complements mammography by distinguishing solid from cystic formations, particularly in dense breast tissue, where mammography is less effective [11,12]. MRI further enhances sensitivity (up to 92%) but has lower specificity (~60%) due to challenges in lesion characterization [13,14]. The Breast Imaging Reporting and Data System (BI-RADS) ensures standardized reporting, aiding risk assessment and clinical decision-making [15]. Computer-aided diagnosis (CAD) systems refine mass classification, improving diagnostic precision [16]. Radiomics is gaining prominence for its ability to extract high-dimensional quantitative features (e.g., shape, texture, intensity) from medical images, correlating them with clinical outcomes [17,18,19,20]. The radiomics workflow includes image acquisition, preprocessing, tumour segmentation, feature extraction, and machine learning (ML)-based model development. Deep learning (DL) advancements have expanded radiomics applications, enabling predictive modelling for cancer prognosis and treatment response [21,22,23,24]. Convolutional neural networks (CNNs) have demonstrated high accuracy in cancer risk assessment and prognosis prediction [25,26].

This study evaluates a radiomics-based predictive workflow implemented using matRadiomics [27,28] integrating preprocessing techniques to enhance mammographic image quality. ML classifiers were tested for their ability to classify breast masses and microcalcifications, with a comparative analysis against the DL approach. A private dataset was used for external validation, emphasizing the importance of model generalization in breast lesion classification. By juxtaposing ML-based radiomics with DL, this research highlights the potential of advanced methodologies to improve the accuracy and reliability of diagnosis. By addressing key challenges such as methodological standardization and clinical applicability [29], this work contributes to the evolving role of image processing in automated breast cancer diagnosis.

2. Materials and Methods

This section outlines the tools and methodologies used for radiomics and DL analysis. The CBIS-DDSM database (Curated Breast Imaging Subset of Digital Database for Screening Mammography) [30] containing mammographic images and lesion segmentation masks, was used. The analysis used matRadiomics [31] with the integration of pre-processing techniques implemented specifically for this type of images. Furthermore, the EfficientNetB6 convolutional neural network was used to explore the potential of DL and compare it with traditional ML methods for lesion classification.

2.1. Datasets

2.1.1. Dataset for Internal Validation

The CBIS-DDSM database comprises mammographic images, available for public download from the Cancer Imaging Archive (TCIA), a repository of medical images of cancer. This dataset divided the images into microcalcifications (1,872 images) and masses (1,696 images), representing 753 and 891 patients, respectively. Among these 3,568 images, 2,111 cases are benign, while 1,457 are malignant. Each patient has multiple IDs corresponding to scans from different projections (e.g., craniocaudal -CC- or mediolateral oblique -MLO) and specific anomalies, identified by unique codes. For each patient, there are at least two series of images: a full mammogram and additional images highlighting the lesion, including a segmented region of interest (ROI) and a cropped image of the affected area. Some patients exhibit both types of anomalies. All images are in DICOM (Digital Imaging and Communications in Medicine) format with digital breast tomosynthesis extensions. The database includes comprehensive metadata in CSV files, detailing breast density (based on the BI-RADS system), breast side (left or right), image view (CC or MLO), anomaly identifiers, anomaly type (mass or calcification), and specific characteristics such as mass shape, calcification type, margins, distribution, BI-RADS assessment, pathology (benign or malignant), and anomaly subtlety (graded from 1 to 5). These details offer a rich dataset for exploring breast cancer imaging [30].

2.1.2. Dataset for External Validation

The external validation of the model was conducted using a database provided by the Fondazione Istituto “G. Giglio” of Cefalù. This dataset consists of 222 images from 180 patients, including 149 cases with calcifications and 73 with masses. It was utilized to assess the performance of the proposed radiomics workflow and validate the model in an independent setting, ensuring its generalizability beyond the training data. This retrospective study was approved by the local ethical review board.

2.2. Toolbox for Radiomics Analysis: matRadiomics

matRadiomics is a user-friendly, Image Biomarker Standardization Initiative (IBSI)-compliant framework designed to optimize and simplify the radiomics workflow in both clinical and preclinical contexts [27,28]. Built on MATLAB and Python, it integrates several powerful tools for medical image analysis, including the PyRadiomics library for feature extraction [32], ComBat for data harmonization [33], and various ML algorithms for predictive modeling. The platform supports the processing of medical images from different modalities, such as Positron Emission Tomography (PET) [34,35,36], CT (Computed Tomography) [37], and MRI (Magnetic Resonance Imaging) [38], and facilitates lesion classification and differentiation, including benign vs. malignant lesions. Key functionalities of matRadiomics include the ability to create and import radiomics studies, perform target segmentation, and extract radiomics features. To enhance image quality before feature extraction, we incorporated image preprocessing algorithms like Contrast Limited Adaptive Histogram Equalization (CLAHE) and Median Filter.

2.3. Preprocessing

Two preprocessing techniques were applied to improve mammographic image quality and optimize lesion classification: CLAHE and Median Filtering. These methods enhance image quality, leading to more accurate feature extraction and predictive modeling [39,40]. CLAHE effectively increases contrast in low-contrast mammographic images while minimizing noise amplification, whereas the Median Filter excels at noise reduction while preserving critical image structures, outperforming traditional mean filters [41].

Specifically, CLAHE is an advanced histogram equalization technique designed to enhance local contrast while preventing excessive noise amplification and artifacts in high-intensity regions. Unlike traditional histogram equalization, which operates on the entire image, CLAHE divides the image into small, non-overlapping tiles and computes a local histogram for each tile. The histogram is then clipped at a predefined limit to avoid over-amplification of certain intensity levels. After clipping, the modified histogram is normalized and used to remap pixel intensities, enhancing local contrast. To ensure smooth transitions between adjacent tiles and avoid visible artifacts at the edges, bilinear interpolation is applied, blending intensity values between neighboring regions. Additionally, a non-linear median filtering technique can be used to further refine contrast enhancement, particularly in the presence of impulsive noise, such as salt-and-pepper noise, while preserving important edge details [42].

The mathematical formulation of CLAHE is presented through the following steps::

• Histogram calculation for each tile:

$$h_k=\sum _I\delta \left(I-I_k\right), \forall k$$

(1)

where $h_k$ is the histogram for a tile $k$, $I_k$ is the pixel intensity for that region and $I$ is the specific intensity that is counted in that region by the dirac delta function.

2.

Clip limit application:

$$h_k\left(I\right)=min(h_k\left(I\right),C)$$

(2)

where the sum of the pixel intensity values does not exceed the C threshold, limiting the number of occurrences.

3.

Local equalization:

$$T_k\left(I\right)={\displaystyle \frac{H_k^{clip}\left(I\right)}{\sum _I{H}_k^{clip}\left(I\right)}}$$

(3)

where $T_k\left(I\right)$ is the intensity $\left(I\right)$ transformation in the tile $k$ and $H_k^{clip}$ is the clipped histogram. The total sum of $H_k^{clip}\left(I\right)$ values is normalized so that the intensity is evenly distributed, improving contrast [43]. To avoid visible artifacts along the edges of adjacent regions, linear interpolation is applied between tiles, ensuring smooth transitions. In addition, taking into account the characteristics of the images and the noise that often degrading visual quality and obscuring critical details, among denoising techniques emerges the median filter, particularly effective for removing impulsive noise, while preserving edges and fine details. Unlike linear filters, which average pixel values and may blur edges, the median filter operates non-linearly, replacing each pixel with the median value of its surrounding neighborhood. To formalize this process, consider a window of size NxN (e.g. 3x3) that slides across the image and $X=\left\{x_1,x_2\dots x_n\right\}$ the pixel values within that window. The median filter, or median(X) replaces the value of the pixel $f^{\text{'}}\left(x_i\right)$ with the median of its neighbors in the window:

$$f^{\text{'}}\left(x_i\right)=median\left(X\right)$$

(4)

By selecting the median rather than the mean, this method effectively removes outlier values caused by noise, preventing excessive blurring and preserving the structural integrity of the image. Together, these preprocessing techniques enhance image contrast and reduce noise, highlighting areas of interest while maintaining the integrity of the image's structural details [44].

2.4. Radiomics Features

Radiomics features are quantitative attributes extracted from medical images, providing critical insights into tissue properties and lesion morphology. These features, derived through advanced algorithms, form the implementation of predictive and diagnostic models, making their selection essential for improving accuracy and reliability in radiomics, [45,46]. Radiomics features are typically classified into three categories:

• First-order features describe the intensity distribution within the region of interest (ROI), including metrics such as mean, standard deviation, skewness, and kurtosis.
• Second-order features analyze texture by assessing the spatial relationships between voxel intensities, capturing patterns that reflect tissue heterogeneity.
• Shape features define the geometry and morphology of the ROI, including volume, surface area, and sphericity. These are particularly useful for distinguishing lesions, as benign ones tend to be more regular, while malignant ones often exhibit irregular shapes.

2.4.1. Feature Extraction

In this study, the workflow was carefully designed to maximize feature diversity and discriminative power. Three image transformations were applied—Original, Wavelet, and LoG (Laplacian of Gaussian)—to capture a broader range of structural details. While original images preserve raw data, wavelet transformations decompose images into low- and high-frequency components, enhancing fine textures. LoG filtering improves edge detection and highlights local intensity variations, aiding in the identification of subtle patterns. To balance texture preservation and noise reduction, discretization was set to 64 bins, a value supported by previous studies as an effective trade-off [47].

2.4.2. Feature Selection

Feature selection is a crucial step in the radiomics pipeline, reducing redundancy and non-informative features to enhance model performance, lower computational costs, and improve generalization [48,49,50]. Common approaches include filter, wrapper, and embedded methods [51,52]. In this study, we used the mixed descriptive-inferential sequential approach as described in [53]. For each feature, the point biserial correlation (PBC) index between features and the dichotomic outcome was calculated, sorting the features in PBC descending order. The PBC ranks features based on their correlation with the binary target variable (e.g., benign vs. malignant tumors). It serves as a pre-filtering step, ordering features by correlation strength before stepwise logistic regression selects the most relevant ones. This approach is particularly effective in high-dimensional datasets with non-standard distributions [54,55,56].

2.5. Machine Learning Predictive Models

The ML methods used in this work belong to the supervised learning category, in which there is a set of labeled training data (benign tumor labeled 1 and malignant tumor labeled 0) used to estimate or map the input data to the desired output. Based on matRadiomics, after the features selection process, Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) were used through k cross-validation.

2.5.1. Linear Discriminant Analysis

LDA is a statistical ML technique primarily used for supervised classification. The goal of LDA is to project high-dimensional data onto a lower-dimensional space while maximizing the separation between classes. Specifically, LDA identifies a projection direction that maximizes the separation between class means while minimizing the variance within each class.

LDA relies on two key scatter matrices:

• Within-class scatter matrix (S_w): measures the variance within each class and should be minimized for optimal classification.
• Between-class scatter matrix (S_b): measures the variance between class means and should be maximized.

The optimal projection direction will be the one that maximizes the ratio between the S_b and the S_w. It will be given by the following formula:

$$w^{\ast }=S_w^{-1}({\mu }_1-{\mu }_2)$$

(5)

where ${\mu }_1$ and ${\mu }_2$ are the centroids of the various classes.

The S_w is found as:

$$S_w=\sum _{i=1}^C\sum _{x\in classe i}(x-{\mu }_i){(x-{\mu }_i)}^T$$

(6)

The method is straightforward to implement and computationally efficient, making it a popular choice in many practical applications. By reducing the dimensionality of a dataset, LDA simplifies complex problems and helps in visualizing data in a two- or three-dimensional space. Additionally, the focus on maximizing the separation between classes ensures that LDA often provides robust performance for problems where the classes are well separated, and the underlying assumptions of normality and equal variance are satisfied. LDA is particularly effective in cases where the number of features is high compared to the number of samples, as it reduces the feature space to the most discriminative dimensions. Its mathematical foundation also provides clear interpretability, allowing practitioners to understand how the model achieves its classification decisions [57,58].

2.5.2. Support Vector Machine

SVM is a supervised learning algorithm primarily used for binary classification. Its objective is to identify a hyperplane that best separates the two classes by maximizing the margin, i.e. the minimum distance between the data points of each class and the hyperplane itself. This ensures that the classification is robust and less sensitive to small variations in the data [59].

A hyperplane can be described by the equation:

$$w\ast x+b=0$$

(7)

where w is the normal vector to the hyperplane, b is the bias, and x represents the data points. To find the optimal hyperplane, the aim is to solve the problem:

$$\underset{w,b}{\min }{\displaystyle \frac{1}{2}}{\left|\left|w\right|\right|}^2$$

(8)

With the constraint:

$$y_i\left(w·x_i+b\right)\ge 1 \forall i$$

(9)

where $w$ is plane normal vector, $b$ is the bias and $y_i$are the labels of the data (+1 or -1).

After finding the hyperplane, a new point x is classified with:

$$f\left(x\right)=sign(w\ast x+b)$$

(10)

2.6. Performance Metrics

The final step in radiomics analysis is assessing the trained model's performance in classifying mammographic lesions. This assessment relies on performance metrics that evaluate the model’s predictive accuracy and reliability. In this study, two critical metrics are examined: the Area Under the Curve (AUC) of Receiver Operator Characteristic (ROC) curve and Accuracy. These metrics are widely used in medical applications as they provide complementary insights into the classifier's ability to distinguish between classes and make reliable predictions. To understand these metrics, the starting point is the confusion matrix, which summarizes the classifier’s predictions against the true labels. The components of the confusion matrix are defined as follows:

• True Positives (TP): Positive examples correctly classified as positive.
• True Negatives (TN): Negative examples correctly classified as negative.
• False Positives (FP): Negative examples incorrectly classified as positive.
• False Negatives (FN): Positive examples incorrectly classified as negative.

From the confusion matrix, it is possible to calculate key rates such as the True Positive Rate (TPR) and False Positive Rate (FPR), which serve as the axes of the ROC curve. These two rates are described as follows:

$$TPR={\displaystyle \frac{TP}{P}}={\displaystyle \frac{TP}{(TP+FN)}}$$

(11)

Sensitivity, i.e. the TPR, that is the ratio between cases correctly classified as positive and all true positives and measures the ability of a model to correctly identify positives.

$$FPR={\displaystyle \frac{FP}{N}}={\displaystyle \frac{FP}{(FP+TN)}}$$

(12)

The False Positive Rate represents the x-axis of the ROC space and is the ratio between the negative classes classified as positive and the true negatives.

These rates define the ROC curve, where TPR is plotted against FPR at various classification thresholds. An ideal classifier would achieve TPR = 1 (perfect sensitivity) and FPR = 0 (no false positives), placing it in the top-left corner of the ROC space. To compare the performance of multiple classifiers and to summarize the performance of a classifier across all possible thresholds, the AUC is calculated, a numerical parameter that represents the classifier’s performance. AUC quantifies the overall ability of the classifier, its value ranges is [0; 1]:

• An AUC of 1 indicates perfect classification, where the model distinguishes all positive from negative instances.
• An AUC of 0 suggests a completely inverted classifier.
• An AUC of 0.5 indicates random guessing, with no predictive power.

Thus, the AUC provides a useful numerical summary of classifier performance, allowing for direct comparison across different models and thresholds. In addition, from the confusion matrix, it is possible to derive Total Actual Positives (P) and Total Actual Negatives (N):

$$P=TP+FN$$

(13)

$$N=FP+TN$$

(14)

These two metrics allow accuracy to be derived, as follows:

$$ACC={\displaystyle \frac{T}{\Sigma }}={\displaystyle \frac{TP+TN}{P+N}}$$

(15)

Accuracy represents the proportion of correctly classified instances among all observations and provides a measure of overall correctness, complementing the ROC curve and AUC. The choice of evaluation metrics depends on the clinical application. In this study, the primary goal is to determine whether a breast lesion is benign or malignant from a clinical decision support perspective. High sensitivity is crucial for the accurate identification of malignant cases, thereby minimizing false negatives. On the other hand, precision is especially important in situations with imbalanced datasets. Combining multiple metrics provides a more comprehensive evaluation of radiomics features and model effectiveness [60,61,62].

2.7. Deep Radiomics

In addition to the conventional radiomics workflow, a Deep Radiomics approach was implemented using the EfficientNetB6 convolutional neural network (CNN). The selection of EfficientNetB6 was motivated by its unique compound scaling capability, which optimizes the network’s depth, width, and resolution simultaneously. This joint scaling strategy enables EfficientNet to achieve an optimal trade-off between performance and resource efficiency, reducing the number of parameters and computational operations (measured as FLOPS) compared to traditional architectures such as ResNet. The foundational architecture of EfficientNet, known as EfficientNetB0, incorporates Mobile Inverted Bottleneck Convolutions (MBConv), a design optimized for computational efficiency. Additionally, it integrates Squeeze-and-Excitation (SE) blocks, which enhance the model's ability to capture relevant features by recalibrating channel-wise feature responses. These architectural innovations collectively improve both the accuracy and efficiency of the network. In this study, the EfficientNetB6 variant was chosen due to its balanced combination of computational complexity and predictive performance. Compared to other networks, EfficientNetB6 offers superior accuracy with a comparable number of parameters, making it particularly suitable for tasks that demand high performance without excessive computational cost. This balance is especially critical in medical imaging applications, where both precision and efficiency are paramount [63].

2.7.1. Implementation of EfficientnetB6

The EfficientNetB6 network was pre-trained on the ImageNet dataset to leverage optimized weights, enhancing generalization on smaller datasets. The adaptation process included two phases: transfer learning and fine-tuning. The original classification head of the pre-trained model was replaced with a custom head that included fully connected layers. During this phase, only the new head was trained, while the weights of the convolutional layers were kept frozen to retain the knowledge acquired during pre-training. In the fine-tuning phase, the entire model was refined. All weights, including those in the convolutional layers, were updated, allowing the model to adapt to the specific task while preserving the valuable features learned during the initial pre-training. In case of masses classification, the head was replaced with a Global Average Pooling operation, followed by a Dense Layer of 1024 neurons and RELU activation function, a Dropout layer (percentage = 0.3), and a final Dense Layer of 1 neuron and Sigmoid activation function. While, for calcifications classification, the head was replaced with a Global Average Pooling operation, followed by a Dense Layer of 256 neurons and RELU activation function, a Dropout layer (percentage = 0.3), and a final Dense Layer of 1 neuron and Sigmoid activation function.

2.7.2. Dataset Preparation for the Neural Network

The dataset used for training was derived from the CBIS-DDSM database, focusing exclusively on images containing masses. The following steps were applied: i) Images were cropped to 512x512 pixels using a bounding box to isolate the lesion of interest. Padding was applied using image values rather than zeros to retain critical information; ii) the same pre-processing techniques (CLAHE and Median Filtering) as in the classical radiomics workflow were used; iii) To mitigate overfitting due to limited sample size, augmentation techniques were applied with random probabilities:

• Horizontal and vertical flips.
• Random translations and rotations (±20°).
• Brightness and contrast adjustments.
• Gaussian filtering and elastic deformations.

2.7.3. Model Training Configuration

For masses classification the initial learning rate for the transfer phase was set to 1 × 10^-4 and decreased to 1 × 10^-7 by Polynomial Decay with power 0.9. Training was scheduled for a total of 50 epochs. During fine-tuning, initial learning rate was set to 5 × 10^-6 and decreased to 1 × 10^-8 by Polynomial Decay with power equal to 1. and training was scheduled for 20 epochs. For calcifications classification same settings were used except for the scheduled number of epochs during transfer and fine tuning, which were set both to 50.

Loss Function measures the extent to which a model's predictions differ from the desired output, and this deviation needs to be minimized during the training process. In this context, the loss function consists of two components [64,65].

Binary Cross-Entropy (BCE) represents a measure of error in binary classification problems and is based on a heavy penalty for incorrect forecasts. Its formulation is as follows:

$$BCE=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N[y_i\log \left(pi\right)+(1-y_i)\log (1-p_i)]$$

(16)

where $p_i$ is the probability predicted by the model for class 1, $y_i$ is the label (0 or 1) and $N$ is the number of examples [66]. During Transfer learning and fine-tuning phases, Area Under Curve values were tracked and models were saved only when AUC improved to prevent overfitting. In this case the final models aro those based on maximum validation AUC. Models were trained leveraging Google Colab Pro+ resources, Python, Tensorflow and Keras libraries.

3. Results

This section presents the results of the breast lesion classification model based on matRadiomics. The evaluation was performed using stratified k-fold cross validation with k=10, and the percentage of data reserved for internal testing was set at 30% (Test set). The external validation was performed using the database of the Fondazione G. Giglio Institute in Cefalù. Finally, the results obtained through DL are also presented.

3.1. CBIS-DDSM Database

In this study the database used is the CBIS-DDSM, that contains mammographic images representative of the two types of lesions (mass and calcifications) accompanied by appropriate ROI masks, identifying the lesions. In particular, as regards calcifications, those masks that did not represent the lesion, for example that covered the entire breast, were excluded from the study. In Figure 1, an example of a mammographic image containing a microcalcification accompanied by its ROI mask and its cropped image is shown, while in Figure 2 there is a similar example, but with an image containing a mass.

3.2. Preprocessing

The results discussed in this section are based on the cascade application of the CLAHE method and the Median Filter. An example of implementation of these techniques on a mammographic image is shown in Figure 3.

3.3. Machine Learning-Based Radiomics

The selected features are fundamental in evaluating the effectiveness of the model implemented in matRadiomics. Analyzing these features provides insight into how the model operates during classification, identifying the most discriminative aspects of the images. As an initial step, the distinction between masses and calcifications was examined using the entire dataset, which includes 1,219 images (638 masses and 581 microcalcifications). Through PBC analysis, the shape feature original_shape2D_sphericity was identified as a key discriminative factor. This finding is particularly relevant, as it confirms that the model distinguishes lesions based on their shape—masses tend to be more rounded and spherical, whereas calcifications often appear elongated or thread-like.

The results presented in Table 1 highlight the strong differentiation capability of the LDA model, while Figure 4 illustrates the corresponding ROC curve (a) and accuracy metrics (b), further validating the model’s performance.

In the context of lesion classification, the model is designed to differentiate between benign and malignant cases. Compared to the previous task, the selected features evolve, reflecting the need for different types of information.

For masses, the key selected feature is wavelet_LL_firstorder_Energy, which is no longer related to shape but to texture. This shift indicates that the model requires more in-depth information beyond morphology, as benign and malignant masses often exhibit similar shapes. Texture analysis enables the detection of subtle variations in intensity distribution, thereby enhancing classification accuracy.

Conversely, for calcifications, the selected feature is original_shape2D_perimeter, emphasizing the role of shape in classification. This suggests that benign and malignant calcifications exhibit more pronounced morphological differences than masses, making perimeter a critical distinguishing factor.

Regarding masses, the LDA model achieved:

-

A mean validation ROC AUC of 61.53%,

-

A test ROC AUC of 67.14%,

-

A mean accuracy of 57.43%,

-

A test accuracy of 62.67% (see Figure 5).

For calcifications, the results were notably higher:

-

Mean validation ROC AUC: 68.28%,

-

Test ROC AUC: 75.81%,

-

Mean accuracy: 65.50%,

-

Test accuracy: 68% (see Figure 6).

For masses, the SVM model achieved:

-

Mean validation ROC AUC: 60.22%

-

Test ROC AUC: 66.7%

-

Mean accuracy: 57.14%

-

Test accuracy: 62% (see Figure 7).

For calcifications, the model performed better, with:

-

Mean validation ROC AUC: 63.26%

-

Test ROC AUC: 74.31%

-

Mean accuracy: 65.99%

-

Test accuracy: 68% (see Figure 8).

These results indicate that SVM struggles with mass classification, likely due to the similarity in morphological patterns between benign and malignant lesions. However, calcifications appear to be more effectively distinguished, reinforcing the idea that shape-related features play a crucial role in their classification.

Based on the LDA, the performance of the external validation based on the private database of the Fondazione G. Giglio Institute of Cefalù is shown in Table 2 and Figure 9.

3.4. Deep Learning

Through deep radiomics, performance metrics superior to those obtained with the traditional radiomics flow were obtained, confirming the power of CNN networks that work directly on images. As shown in Table 3 and Figure 10, DL masse classification resulted in an AUC of 81.52%, accuracy of 78%, recall of 66.70%, precision of 74.24% and F1-Score of 70.25%, while DL calcification classification resulted in an AUC of 76.24%, accuracy of 71.10%, recall of 85.78%, precision of 81.96%, F1-Score of 70.25%.

4. Discussion

This study aimed to compare a radiomics workflow based on ML with a DL approach for the differentiation and classification of breast lesions. In both cases, preprocessing techniques were applied to enhance lesion detection and classification accuracy [67] by reducing noise and artifacts. The adoption of radiomics aligns with recent advancements in automated diagnostic tools for medical imaging [68]. The results obtained from ML-based breast lesion classification showed suboptimal performance. The LDA model performed better than SVM with an average validation AUC of 68.28% for microcalcifications and 61.53% for masses. We employed a private dataset, offering an external validation framework to investigate the generalization to independent cohorts. In this case, an AUC of 66.9% and 61.5% were obtained, respectively. However, our results were notably lower compared to a study based on a different public dataset, in which the authors in [69] reported an AUC of 88% using LDA, with SVM providing slightly worse performance, as in our case. However, a direct comparison is not feasible due to the differences in datasets. Given the limitations observed in our study with traditional ML techniques, a second phase of development was undertaken to explore the potential of DL models. This phase aimed to determine whether advanced neural networks could enhance classification performance in distinguishing between benign and malignant lesions. In this context, the EfficientNetB6 model outperformed the radiomics approach, demonstrating its potential for improved diagnostic accuracy, with an AUC of 81.52% and 76.24%, respectively.

In the literature, Gerbasi et al [70] achieved remarkable results in breast lesion classification using the same dataset (CBIS-DDSM), obtaining an AUC of 89% through a fully automated pipeline. Specifically, their approach, DeepMiCa, consisted of: (1) preprocessing of raw scans, (2) automatic patch-based semantic segmentation using a U-Net-based network, and (3) classification of detected lesions through a deep transfer-learning approach. The most significant difference compared to our work lies in the segmentation strategy. Unlike our study, DeepMiCa performed a new segmentation of the target lesions. Since CBIS-DDSM is a public dataset, many segmentations may contain substantial errors, negatively impacting classification performance. Future research efforts will be aimed at overcoming this limitation by automating the segmentation process. The study [71] proposes an advanced technique for breast cancer classification by combining a CNN with an enhanced metaheuristic optimization algorithm. To overcome existing limitations, the authors modified the Ant Colony Optimization (ACO) algorithm by integrating it with opposition-based learning (OBL), creating the Enhanced Ant Colony Optimization (EACO). This method was used to optimize hyperparameters in the CNN architecture, specifically combining ResNet101 with the EACO algorithm, resulting in the EACO–ResNet101 model. Experimental analyses conducted on the CBIS-DDSM dataset demonstrated that the proposed model achieved an accuracy of 98.63%, outperforming traditional methods. Finally, a hybrid system combining ResNet50 with SVM achieved an AUC of 98.46%, on a small subset of 330 cases of the CBIS-DDSM dataset demonstrating the potential of integrating deep learning with traditional machine learning approaches [72]. These findings underscore the potential of DL methodologies in enhancing diagnostic accuracy for breast cancer detection.

Given these considerations, future developments will focus on enhancing the radiomics pipeline by integrating DL models capable of leveraging raw imaging data directly. The use of EfficientNetB6 in forthcoming studies aims to explore the synergy between radiomics and neural networks, potentially improving classification accuracy while preserving the interpretability of handcrafted features. Moreover, incorporating image rotation, and multi-view image fusion could further optimize lesion characterization. Beyond preprocessing improvements, the adoption of ensemble learning techniques and more sophisticated feature selection strategies could significantly enhance classification robustness and reliability. The growing trend of hybrid models, which combine traditional ML with DL architectures, represents a promising direction for achieving higher diagnostic accuracy and clinical integration. Ultimately, while radiomics remains a powerful tool for lesion characterization, its full potential can be un-locked by bridging the gap between feature engineering and data-driven DL approaches. By refining the proposed workflow and incorporating state-of-the-art AI techniques, this research lays the groundwork for more accurate, reliable, and clinically applicable tools in breast cancer detection.

5. Conclusions

The proposed study highlights the significant potential of artificial intelligence models for breast lesion classification. Traditional models such as LDA and SVM have shown promising results. The use of a proprietary dataset has strengthened the model's generalizability, ensuring its clinical applicability. However, DL approaches promise to further improve classification accuracy and robustness, providing an exciting direction for future research with the potential to significantly enhance diagnostic tools in clinical settings.