A Hybrid Fuzzy–Ensemble Machine Learning Framework for Non-Invasive Prediction of HER2 Status in Breast Cancer

1. Introduction

Breast cancer is one of the most common malignancies affecting both women and, less frequently, men, and it is primarily caused by the uncontrolled growth of breast cells [1,2]. The human epidermal growth factor receptor 2 (HER2) is a transmembrane receptor protein that plays a critical role in regulating cell growth and division [3]. In approximately 15–20% of breast cancer cases, HER2 is overexpressed, leading to aggressive tumor behavior [4]. Accurate identification of HER2 status is therefore crucial for guiding targeted therapies, such as trastuzumab, which have significantly improved clinical outcomes [5]. In spite of its clinical importance, determining HER2 status typically relies on conventional diagnostic techniques, which face several limitations.

Despite its clinical importance, HER2 assessment is predominantly performed using conventional diagnostic techniques, namely immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH) [1]. While these methods are widely accepted as clinical standards, they suffer from several limitations. Both IHC and FISH require specialized laboratory infrastructure, trained personnel, and high-quality reagents, and they are often time-consuming and costly [1,6]. In addition, pre-analytical factors such as tissue fixation, processing, and staining variability may adversely affect diagnostic accuracy, potentially leading to equivocal or false results [6]. These challenges motivate the exploration of alternative, data-driven approaches that can support or complement existing diagnostic workflows.

Machine learning (ML) techniques have recently emerged as promising alternatives for predicting HER2 status using clinical, histopathological, or genomic data [1,7]. Several studies have shown that ML models, including support vector machines, random forests, and deep learning techniques, can achieve competitive accuracy compared to conventional diagnostic methods [8]. However, most of these studies suffer from important limitations [9,10]. Many rely on relatively small datasets, which restrict their generalizability [11]. Others apply individual algorithms without leveraging ensemble strategies, thereby failing to capture the complementary strengths of different classifiers. Furthermore, relatively little attention has been devoted to feature engineering or uncertainty-aware modeling, both of which are critical for ensuring the reliability of ML-based systems in clinical decision-making.

To address these limitations, this study proposes a hybrid ensemble machine learning framework for HER2 status prediction based on routinely available clinical and immunohistochemical biomarkers. The proposed approach is evaluated on a dataset comprising 624 breast cancer patients collected from Mahdieh Clinic in Kermanshah, Iran. Our framework integrates multiple tree-based classifiers, namely Random Forest, XGBoost, and LightGBM, within ensemble strategies and incorporates fuzzy feature engineering and decision threshold optimization to enhance robustness in borderline and uncertain cases.

2. Related Work

Recent advances in machine learning (ML) and data-driven modeling have demonstrated significant potential in improving diagnostic and prognostic decision-making in Breast Cancer (BC). In particular, the integration of ML with radiomics, multi-omics, and clinical data has enabled the extraction of complex patterns that are difficult to identify using conventional statistical approaches. For instance, Chen et al. conducted a bicentric retrospective study employing ultrasound radiomics and ML techniques to predict pathological prognostic stages in a cohort of 578 BC patients [12]. While promising, the reliance on imaging-derived features and limited external validation may constrain the applicability of such models in routine clinical settings.

Complementary efforts have explored the use of multi-omics data for prognosis estimation. Song et al. proposed a prognostic framework for elderly BC patients by integrating mRNA, miRNA, lncRNA, copy number variations (CNVs), and single nucleotide variants (SNVs), highlighting the prognostic relevance of hypoxia-related pathways and immune microenvironment heterogeneity [13]. Although multi-omics models provide rich biological insights, their clinical deployment is often challenged by high costs, data complexity, and limited accessibility. Interpretability and robust feature selection have also gained attention in clinically oriented ML pipelines. For example, Ahmadian et al. proposed an explainable feature selection approach combining particle swarm optimisation with adaptive LASSO for MRI radiogenomics, demonstrating transferable signatures and improved generalisability in a two-center setting [14].

Beyond imaging and omics-based approaches, ML models have also been applied to predict treatment-related outcomes. Lin et al. developed an XGBoost-based model for predicting radiation dermatitis severity in breast cancer patients, incorporating clinical factors, patient-reported outcomes, and cytokine biomarkers [15]. Similarly, Miglietta et al. utilized ML techniques to predict HER2-low phenotype conversion in recurrent breast cancer, emphasizing the role of artificial intelligence in optimizing patient stratification and treatment accessibility [16]. Despite encouraging results, many of these studies are limited by relatively small sample sizes and the use of single-model architectures, which may restrict robustness and generalizability.

The application of ML in oncology extends beyond breast cancer and further underscores its prognostic utility. In bladder cancer, Zhang et al. developed a machine learning-based prognostic signature utilizing proteomics data to predict patient outcomes and treatment response [17]. In prostate cancer, Gao et al. constructed a programmed cell death-related gene signature using a random forest model, demonstrating that higher risk scores were associated with poorer survival outcomes and diminished immunotherapy benefits [18]. Similarly, Maimaitiyiming et al. proposed a mast cell gene signature that stratified prostate cancer patients into distinct immune-risk groups [19]. In gastric cancer, Liu et al. introduced a deep learning-based pathomics model that achieved high predictive performance for survival outcomes [20].

More recently, ML-based prognostic models have also been investigated in specific breast cancer subpopulations. Wu et al. identified senescence-related molecular subtypes in geriatric breast cancer with distinct prognostic significance [21]. In addition, Emily et al. compared Cox proportional hazards and survival random forest models for breast cancer survival prediction, reporting superior performance of the Cox model in their cohort [22].

Despite these notable advances, several limitations persist across the existing literature. Many studies rely on single-modality data sources, lack ensemble or uncertainty-aware decision mechanisms, or are evaluated on relatively small and population-specific datasets. Moreover, the integration of heterogeneous clinical and immunohistochemical features within robust ensemble frameworks remains underexplored, particularly for HER2 status prediction. Addressing these gaps, the present study proposes a hybrid fuzzy-enhanced ensemble approach that combines multiple tree-based classifiers with fuzzy feature engineering and decision calibration, aiming to improve predictive robustness while maintaining clinical feasibility.

3. Methods and Materials

To ensure reproducibility and methodological transparency, all experiments were implemented in Python using widely adopted machine learning libraries, including Scikit-learn, XGBoost, and LightGBM. Model development and evaluation were conducted in a Jupyter-based environment with fixed random seeds to guarantee consistent results. The proposed framework follows a hybrid pipeline that integrates clinical and immunohistochemical features with fuzzy modeling and ensemble learning. The workflow begins with data preprocessing and feature engineering, followed by fuzzy feature construction to capture uncertainty in key biomarkers. Tree-based classifiers—Random Forest, XGBoost, and LightGBM—are then trained and evaluated both as standalone models and within ensemble strategies. Finally, the model performance is evaluated using standard classification metrics to assess discriminative power and clinical reliability.

3.1. Data Description

This study utilized a retrospective clinical dataset consisting of 624 confirmed breast cancer cases collected from Mahdieh Clinic, Kermanshah, Iran. The dataset includes a combination of clinical and pathological variables routinely used in breast cancer assessment. Specifically, the available features comprise estrogen receptor (ER), progesterone receptor (PR), p53 status, perineural invasion, vascular invasion, metastasis, lymph node involvement, number of involved lymph nodes, tumor size, and the Ki67 proliferation index.

The HER2 status (positive or negative) was defined as the target variable, framing the problem as a binary classification task. All cases were extracted from clinical records and validated by domain experts to ensure data consistency and reliability prior to model development.

3.2. Preprocessing Steps

Several preprocessing steps were applied to enhance data quality and ensure robust model performance. First, missing values in numerical features were handled using mean imputation, which was selected due to the relatively low proportion of missing entries and the approximately symmetric distribution of the variables.

Subsequently, all continuous features were normalized using Min–Max scaling to ensure that variables with different measurement ranges contributed equally to the learning process. The normalization was performed according to:

$$X_{\mathrm{scaled}}={\displaystyle \frac{X-X_{min}}{X_{max}-X_{min}}}$$

(1)

To address the inherent class imbalance in HER2 status, a Random Over-Sampling strategy was applied to the training set. This approach increases the representation of the minority class by randomly duplicating its samples, thereby mitigating model bias toward the majority class. The class balance can be expressed as:

$$\mathrm{Ratio}={\displaystyle \frac{N_{\mathrm{minority}}}{N_{\mathrm{majority}}}}$$

(2)

By applying oversampling exclusively to the training data, data leakage was avoided, and the integrity of the test set was preserved for unbiased performance evaluation.

3.3. Fuzzy Feature Engineering

Clinical biomarkers in breast cancer often exhibit uncertainty, imprecision, and gradual transitions rather than crisp boundaries. Conventional machine learning models typically rely on sharp thresholds, which may fail to adequately represent the inherent vagueness of biological processes. To address this limitation, fuzzy feature engineering was employed to transform selected clinical variables into interpretable fuzzy representations.

In this study, fuzzy membership functions were constructed for clinically significant variables, including the Ki67 proliferation index, tumor size, and estrogen receptor (ER) status. These variables were selected based on their established relevance to tumor aggressiveness and HER2 expression. Each variable was mapped into multiple fuzzy sets using triangular membership functions, enabling the model to capture nonlinear relationships and implicit interactions among biomarkers.

For the Ki67 index, three fuzzy sets—low, medium, and high—were defined to represent varying levels of cellular proliferation. Similarly, tumor size was categorized into small, medium, and large fuzzy sets, reflecting clinically meaningful tumor growth stages. ER status was represented using binary fuzzy memberships corresponding to positive and negative expression. The triangular membership function used for fuzzification is defined as:

$$\mu \left(x\right)=\left\{\begin{array}{cc}0,\hfill & x\le a\hfill \\ {\displaystyle \frac{x-a}{b-a}},\hfill & a<x\le b\hfill \\ {\displaystyle \frac{c-x}{c-b}},\hfill & b<x<c\hfill \\ 0,\hfill & x\ge c\hfill \end{array}\right.$$

(3)

where a, b, and c denote the lower bound, peak point, and upper bound of the fuzzy set, respectively. The parameters of the membership functions were defined based on clinical knowledge and empirical data distribution.

The resulting fuzzy membership degrees were incorporated as additional input features and concatenated with the original numerical and categorical variables. This approach allowed the model to encode uncertainty-aware representations while preserving the original feature space.

3.4. Proposed Hybrid Ensemble Model Architecture

This study proposes a hybrid ensemble learning framework that integrates tree-based machine learning models with fuzzy modeling to predict HER2 status in breast cancer patients. The proposed architecture is designed to capture nonlinear relationships among biomarkers, exploit model diversity, and manage uncertainty inherent in clinical decision-making. An overview of the proposed model is illustrated in Figure 1.

3.5. Model Training, Validation, and Evaluation Metrics

The proposed framework was trained and evaluated using a structured and reproducible experimental protocol. The dataset was randomly partitioned into training and testing subsets using an 80/20 split with stratified sampling to preserve the original class distribution. All preprocessing steps, including feature scaling, oversampling, and fuzzy feature generation, were applied exclusively to the training set to prevent data leakage.

To enhance decision robustness, threshold optimization was applied to the ensemble probability scores by maximizing the F1-score on the validation data.

Model performance was assessed using accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (AUC):

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(4)

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(5)

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

(6)

$$\mathrm{F}1-\mathrm{score}={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(7)

The AUC was computed as:

$$\mathrm{AUC}={\int }_0^1\mathrm{TPR}\left(\mathrm{FPR}\right)d\left(\mathrm{FPR}\right)$$

(8)

3.6. Experimental Setup

3.6.1. Dataset Overview

All experiments were conducted using Python-based machine learning libraries, including Scikit-learn, XGBoost, and LightGBM, within a Kaggle computational environment. The dataset was partitioned into training (80%) and testing (20%) subsets using stratified sampling to preserve the original class distribution. To prevent information leakage, all preprocessing steps including feature scaling, fuzzy feature construction, and class balancing were applied exclusively to the training data and subsequently transferred to the test set. Model performance was evaluated on the unseen test set using multiple complementary metrics, namely accuracy, F1-score, and the area under the receiver operating characteristic curve (AUC), providing a balanced assessment of both classification correctness and discriminative capability.

3.7. Role of Base Learners in the Ensemble Framework

In the first stage of this study, each base learner was independently employed to predict HER2 status using the same training and testing splits, as well as identical preprocessing procedures. This experiment was conducted to establish baseline predictive performance and to enable a fair comparison with the proposed ensemble framework. Random Forest, XGBoost, and LightGBM models were trained separately on the processed dataset, and their predictive performance was evaluated on the held-out test set. Model performance was assessed using accuracy, F1-score, and area under the ROC curve (AUC), which are commonly adopted metrics in binary medical classification tasks.

Table 1 summarizes the quantitative results obtained by the individual base learners. As shown, all three models achieved competitive performance, with Random Forest yielding the highest AUC, while XGBoost and LightGBM demonstrated comparable accuracy and F1-score values.

These results indicate that each base learner is capable of capturing relevant patterns in the clinical and pathological data, while exhibiting different strengths in terms of discrimination and classification balance. These complementary behaviors motivate their integration within an ensemble framework, which is investigated in the subsequent subsection.

3.8. Performance Evaluation of the Proposed Model

The predictive performance of the proposed hybrid fuzzy stacking ensemble model was evaluated on the test set using accuracy, F1-score, and area under the ROC curve (AUC). The model achieved an accuracy of 0.816, an F1-score of 0.814, and an AUC of 0.862, indicating strong discriminative capability in distinguishing HER2-positive from HER2-negative cases.

The ROC curves of the evaluated models are shown in Figure 2. The curves demonstrate the discriminative performance of the base learners and ensemble strategies across thresholds. Overall, ensemble-based approaches achieve competitive and more stable performance relative to individual classifiers.

To further investigate the effectiveness of ensemble learning, the predictive performance of individual base learners was compared with various ensemble strategies, including hard voting, soft voting, stacking, and an intelligent weighted ensemble. Figure 3 summarizes the performance metrics obtained for all evaluated models. Among individual classifiers, Random Forest achieved the highest discriminative performance (AUC = 0.873). Ensemble-based approaches exhibited competitive performance, with the intelligent weighted ensemble achieving accuracy and F1-score comparable to the best-performing individual model while maintaining robust AUC values. These findings suggest that ensemble strategies can improve stability and balance by integrating complementary decision patterns from multiple classifiers.

3.9. Effect of Threshold Optimization

To further improve classification reliability, threshold optimization was applied to the ensemble probability outputs. Instead of adopting a fixed threshold of 0.5, an optimal threshold was selected by maximizing the F1-score on the validation data. The impact of this optimization on classification accuracy is visualized in Figure 4. As illustrated, threshold tuning leads to a noticeable improvement in balanced performance, particularly by reducing false negative predictions, which is crucial in clinical decision-making contexts.

The confusion matrix of the proposed model at the optimized threshold is presented in Figure 5. The matrix demonstrates a balanced distribution of true positive and true negative predictions, confirming the effectiveness of the fuzzy stacking framework in minimizing misclassification errors. Notably, the model achieves a high true positive rate for HER2-positive cases, which is clinically significant, as misclassifying HER2-positive patients may delay access to targeted therapies.

Overall, the experimental results demonstrate that the proposed hybrid fuzzy stacking ensemble model achieves strong and balanced predictive performance. The integration of fuzzy feature engineering and ensemble learning contributes to improved discrimination stability and decision reliability, supporting the potential of the proposed framework as a decision-support tool for HER2 status prediction.

4. Discussion

Accurate and reliable prediction of HER2 status is a cornerstone of personalized breast cancer management, as it directly influences treatment selection and clinical outcomes. In this study, we investigated a hybrid learning framework that integrates fuzzy feature engineering with multiple ensemble strategies to address key challenges in clinical data, including heterogeneity, uncertainty, and moderate class imbalance.

The experimental results reveal several important insights. First, among individual classifiers, Random Forest demonstrated the strongest overall discriminative performance, achieving an accuracy of 0.816 and an AUC of 0.873, outperforming both XGBoost and LightGBM. This finding suggests that tree-based bagging methods may be particularly well-suited for modeling nonlinear interactions among clinical and pathological biomarkers in HER2 prediction tasks. Second, ensemble strategies did not uniformly outperform the best-performing base learner across all evaluation metrics. While the intelligent weighted ensemble and hard voting approaches achieved accuracy and F1-scores comparable to Random Forest, their AUC values were slightly lower. This observation highlights an important methodological point: ensemble integration does not inherently guarantee superior discrimination, particularly when base learners exhibit correlated decision boundaries or similar error patterns. In such cases, ensemble models may primarily improve prediction stability rather than maximizing separability between classes.

Notably, the stacking ensemble achieved a competitive AUC (0.866), indicating improved ranking capability compared to individual boosting models, even though its overall accuracy remained comparable. This suggests that the stacking mechanism effectively aggregates complementary probabilistic information from base learners, enhancing robustness across decision thresholds rather than optimizing a single operating point.

The role of fuzzy feature engineering is particularly relevant in this context. By transforming continuous biomarkers such as tumor size and Ki67 into fuzzy representations, the model captures gradual transitions between clinical risk states, thereby reducing information loss associated with rigid thresholds. This fuzzy modeling contributes to more stable probability estimates, which is reflected in the relatively consistent AUC values observed across ensemble variants. Importantly, this effect becomes more apparent when threshold optimization is applied, underscoring the interaction between fuzzy representations and decision calibration.

Threshold optimization further enhanced clinical relevance by improving the balance between sensitivity and specificity. Given that false negative predictions in HER2-positive patients may delay access to targeted therapies, prioritizing recall without severely sacrificing precision is critical. The optimized threshold reduced false negatives, as confirmed by the confusion matrix analysis, demonstrating that performance improvements are not solely numerical but clinically meaningful.

From a clinical decision-support perspective, these findings emphasize that model selection should be guided by the intended use case. While Random Forest achieved the highest AUC, ensemble approaches offered comparable accuracy with improved robustness and interpretability through aggregation. Therefore, the proposed fuzzy-enhanced ensemble framework should be viewed not as a replacement for strong base learners, but as a complementary strategy that enhances reliability under varying clinical conditions.

Despite these promising results, several limitations should be acknowledged. The dataset was obtained from a single clinical center, which may limit generalizability. Additionally, the absence of external validation restricts conclusions regarding real-world deployment. Future work should focus on multi-center validation, incorporation of additional data modalities such as imaging or genomic profiles, and integration of explainability techniques (e.g., SHAP or rule-based fuzzy inference) to strengthen clinician trust.

Table 2. Comparative performance analysis of individual classifiers and ensemble strategies, highlighting their effectiveness and suitability for clinical decision-support in HER2 status prediction.

Model	Accuracy	F1-score	AUC	Key Strength	Clinical Implication
Random Forest	0.816	0.812	0.873	Strong discrimination	Reliable baseline model
XGBoost	0.800	0.800	0.856	Nonlinear boosting	Sensitive to hyperparameters
LightGBM	0.784	0.783	0.835	Computational efficiency	Moderate discrimination
Hard Voting	0.816	0.814	–	Stability	Threshold-independent
Soft Voting	0.800	0.799	0.862	Probabilistic fusion	Improved calibration
Stacking	0.800	0.800	0.866	Robust ranking	Better threshold robustness
Intelligent Weighted Ensemble	0.816	0.814	0.862	Balanced aggregation	Clinically robust decisions

Table 2. Comparative performance analysis of individual classifiers and ensemble strategies, highlighting their effectiveness and suitability for clinical decision-support in HER2 status prediction.

Model	Accuracy	F1-score	AUC	Key Strength	Clinical Implication
Random Forest	0.816	0.812	0.873	Strong discrimination	Reliable baseline model
XGBoost	0.800	0.800	0.856	Nonlinear boosting	Sensitive to hyperparameters
LightGBM	0.784	0.783	0.835	Computational efficiency	Moderate discrimination
Hard Voting	0.816	0.814	–	Stability	Threshold-independent
Soft Voting	0.800	0.799	0.862	Probabilistic fusion	Improved calibration
Stacking	0.800	0.800	0.866	Robust ranking	Better threshold robustness
Intelligent Weighted Ensemble	0.816	0.814	0.862	Balanced aggregation	Clinically robust decisions

Overall, this study demonstrates that fuzzy-enhanced ensemble learning provides a flexible and clinically meaningful framework for HER2 status prediction. Rather than maximizing a single performance metric, the proposed approach emphasizes robustness, decision reliability, and stable discrimination—key factors for practical adoption in precision oncology.

5. Conclusions

In this study, a hybrid learning framework combining fuzzy feature engineering with ensemble-based classification was proposed for predicting HER2 status in breast cancer patients using routinely available clinical and immunohistochemical data. The primary objective was to develop a robust and clinically meaningful decision-support model capable of handling uncertainty, nonlinear feature interactions, and moderate class imbalance inherent in real-world medical datasets.

Comprehensive experimental evaluations demonstrated that tree-based learning models, particularly Random Forest, achieved strong discriminative performance, with an AUC of 0.873 and an accuracy of 81.6%. Ensemble strategies, including stacking and weighted voting, yielded comparable accuracy and F1-scores, while exhibiting slightly lower but stable AUC values. These findings indicate that, rather than maximizing a single performance metric, the proposed fuzzy-enhanced ensemble framework improves prediction robustness and decision consistency across varying thresholds.

The incorporation of fuzzy feature engineering played a key role in stabilizing probabilistic outputs by modeling gradual transitions in continuous biomarkers such as tumor size and Ki67. This representation reduced information loss caused by hard discretization and contributed to improved calibration and balanced precision–recall behavior. Furthermore, threshold optimization significantly enhanced clinical relevance by reducing false negative predictions for HER2-positive cases, which is critical for timely access to targeted therapies.

Unlike conventional HER2 assessment methods that rely on invasive procedures or costly molecular assays, the proposed framework offers a cost-effective and non-invasive alternative based solely on routinely collected clinical and pathological features. As such, it has the potential to function as a complementary decision-support tool, assisting clinicians in risk stratification and treatment planning rather than replacing standard diagnostic protocols. Nevertheless, several limitations must be acknowledged. The dataset used in this study was derived from a single clinical center, which may limit generalizability. Additionally, the absence of external validation restricts direct clinical deployment.

Future work will focus on validating the proposed framework on multi-center cohorts, incorporating additional data modalities such as imaging radiomics or genomic profiles, and enhancing interpretability through explainable artificial intelligence techniques, including SHAP or fuzzy rule-based explanations. Future work will focus on validating the proposed framework on multi-center cohorts, extending it to additional data modalities (e.g., radiomics and whole-slide pathology) while preserving clinical feasibility, and enhancing interpretability through explainable AI techniques (e.g., SHAP and rule-based fuzzy explanations) [23].

In summary, this study demonstrates that fuzzy-enhanced ensemble learning provides a flexible, reliable, and clinically relevant approach for HER2 status prediction. By prioritizing robustness and decision reliability over isolated performance gains, the proposed framework represents a meaningful step toward intelligent decision-support systems in personalized breast cancer management.