Development of a Machine Learning Model for Predicting Treatment-Related Amenorrhea in Young Women with Breast Cancer

1. Introduction

Breast cancer is one of the most frequently diagnosed cancers among women worldwide, accounting for 1 in 4 cancer cases and totalling 2.3 million instances in 2022 [1]. Globally, more than 100,000 women under the age of 40 are diagnosed with breast cancer each year [2]. While premenopausal patients with breast cancer receiving modern curative-intent treatment regimens, often including chemotherapy, have excellent long-term outcomes [3,4]. However, chemotherapy poses a risk of a loss of ovarian function and may result in early menopause or premature ovarian insufficiency (POI) [5-8]. This commonly presents as hypergonadotropic hypogonadism, resulting in amenorrhea (the cessation of menstrual cycles) [9]. The risk of treatment-related amenorrhea (termed “amenorrhea” hereafter) varies widely depending on factors such as age, chemotherapy regimen, and pretreatment ovarian reserve [10]. For example, most women aged 35–39 are at intermediate risk (31–58%) of treatment-related amenorrhea, with the risk increasing sharply with age—rising to approximately 77% in women over 40—depending on the chemotherapy regimen [11]. Many modern chemotherapy regimens are less damaging to ovarian function than older treatments, but they can still result in short- and long-term ovarian dysfunction [12,13]. Women who remain amenorrhoeic for 12 months following cytotoxic treatment are likely to experience permanent cessation of menstruation and a loss of reproductive capacity (i.e. infertility) [14,15]. The use of gonadotropin-releasing hormone agonists (GnRHa) during chemotherapy has been shown to reduce the risk of POI by temporarily suppressing ovarian function and thereby protecting ovarian reserve [16,17]. However, the evidence is not consistent [18,19].

The impact of ovarian function loss and infertility can be profound. Nearly 70% of young women diagnosed with early breast cancer express a desire to have children in the future [20,21]. Infertility often leads to substantial psychological distress, with levels of depression double that of the general population and diminished quality of life observed in areas such as emotional well-being, sexuality, and relationships [22]. The prospect of infertility can be a distressing outcome of such treatments [23]. Even for individuals without immediate plans for children, the threat of infertility can evoke feelings of anger and a sense of loss [22,24]. Concerns about infertility and the inability to conceive in the future may influence treatment decisions, leading some patients to opt for less effective cancer treatments or not adhering to recommended endocrine therapy to preserve their fertility [25-27]. Providing accurate, personalised risk predictions can help patients better manage expectations, and lead to greater satisfaction with their treatment-related decisions and reduced regret [28]. This may contribute to improved long-term psychosocial outcomes and an enhanced quality of life, even for women who remain childless [29-32]. Consequently, predicting the risk of developing amenorrhea after breast cancer treatment, prior to initiating gonadotoxic treatment has emerged as an important area in current breast cancer research.

With the rapid proliferation of medical data and the advancement of modern statistical techniques and information technology, the utilisation of data science in constructing risk prediction models is a prominent area of research. Traditional risk prediction models, typically based on prior hypothesised knowledge, often consider the relationships between dependent variables. In contrast, machine learning (ML) methods have the potential to learn data models spontaneously, without requiring any implicit assumptions, and are capable of handling interdependence and complicated relationships between variables [33]. ML techniques excel in addressing the vast number of complex higher-order interactions present in medical data. Therefore, risk prediction models developed using ML methods have a high potential for application in clinical practice. An increasing number of clinical studies have leveraged ML methods to develop prediction models, which have found applications in diagnosis [34-36], disease risk prediction [37], and disease recurrence forecasting [38]. Despite increasing interest in predicting the risk of amenorrhea in young women diagnosed with breast cancer [39], existing studies often face significant limitations. This includes small datasets [40-48], exclusion of cases with missing data [40-51], reliance on a limited number of selected features [40-51], a focus on associations rather than predictions [52-55], incomplete reporting [40-51], and the absence of external validation [40-48,50]. To address these gaps, this study establishes a framework for applying machine learning methods in oncofertility research for amenorrhoea risk prediction. Specifically, we developed a predictive model combining multiple datasets to estimate the risk of amenorrhea 12 months after starting the gonadotoxic treatment in women with breast cancer, achieving improved discrimination and calibration.

2. Materials & Methods

Cohort Design, Outcome Definition, Features, and Variables

The FoRECAsT (Infertility after Cancer Predictor) databank [56] comprises multiple datasets sourced from various national and international studies, including those from Australia, the UK, USA, Hong Kong, France, Denmark, Italy, Belgium, and International Trial Groups. In this research, menstrual status was collected at the beginning of chemotherapy. The outcome of amenorrhea was defined as the absence of menses for at least three consecutive months or more after starting the treatment, irrespective of any subsequent resumption of menses. Patients were considered premenopausal if they had their last menstrual period within the previous 12 months of treatment initiation/diagnosis.

From the FoRECAsT databank, seven datasets were selected and combined for analysis because they included the outcome variable amenorrhoea at 12 months from the initiation of treatment. Amenorrhoea at 12 months was a binary indicator, where 1 denoted the presence of amenorrhea and 0 indicated its absence. Among the seven selected datasets, six (‘A’, ‘D’, ‘E’, ‘F’, ‘G’, and ‘N’) were combined for model building and internal validation. The remaining dataset (‘M’), which was collected separately in a different research setting, was designated for external validation, as this configuration—using ‘M’ as the external validation set—resulted in the best model performance compared to other possible dataset divisions.

In this study, a feature referred to a non-outcome column in a dataset. Notably, a categorical feature could generate multiple variables, whereas a numerical or binary feature yielded only one variable. For numerical or binary features, the two terms ‘feature’ and ‘variable’ could be used interchangeably. In the selected datasets, not all features were utilized since some of them were not relevant. Only relevant features, selected based on domain knowledge, were incorporated in the data analysis.

Data Cleaning, Missing Values, and Cross Imputation

In the combined dataset, some features contained invalid values, necessitating data cleaning. Through careful examination and manual verification, all invalid values were either corrected to valid values or replaced as missing values to ensure data integrity for subsequent analysis.

Additionally, missing values in the combined dataset arose from missing data at the time of collection, and variations in research settings across different datasets, leading to misalignment of certain features when combined and significant missingness. This misalignment meant that certain features existing in one dataset may be entirely missing in another.

To address these missing values, we employed cross imputation [57], a technique whereby missing values in one dataset are imputed using values from other datasets. This was accomplished using the k-nearest neighbours (KNN) algorithm and set the value of k to 10, meaning missing values were estimated based on the 10 closest neighbours in a multi-dimensional space. This approach is found to be effective in handling large missingness [58]. Remarkably, even in cases of very high missingness (> 80%), it could still yield predictive models comparable to those trained with complete datasets [57,59]. The imputation process was facilitated using ‘recipes’ package in R [60], ensuring that as many records as possible were utilised to develop the predictive model.

Model building, Internal Validation, and Calibration

Lasso regression, utilizing a binomial family, was pivotal in constructing the model. It maximized the log-likelihood while minimizing the sum of the absolute values of regression coefficients and automatically removing unnecessary variables [61]. The R package glmnet [62] was utilized for model construction.

Model building and internal validation involved two phases: i) feature selection and ii) final model building. In the feature selection phase, the combined dataset was randomly split into training data and test data in 1:1 ratio. A predictive model was built from the imputed training data and was applied to the imputed test data to get the internal validation results (Figure 1A). To mitigate randomness, this process was repeated 100 times, with unnecessary variables automatically pruned in each iteration. Features were selected if at least one of their corresponding variables demonstrated significant relevance (P-value < 0.05) and appeared at least 50 times across the 100-round repetition.

In the final model building step, following feature selection, the process was further repeated 100 times. The final model’s coefficients were calculated from the mean values obtained across the 100 repetitions in this step. All internal validation results from the 100 repetitions were combined to yield the final internal validation results. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC). For calculating sensitivity and precision, the cutoff was derived by maximising the F score with β = 1.5.

Calibration of predicted risks was performed as follows. The predicted risks of internal validation were divided into 20 percentage ranges (5% each) based on their values from smallest to largest. The calibrated predicted risk for each range was calculated as the percentage of cases falling within that range.

External Validation and Model Comparison

The external validation framework is illustrated in Figure 1B. Initially, cross imputation was carried out on the combined dataset. Subsequently, features unique to dataset 'M' that were absent from the combined dataset were excluded, as these features were not utilized during cross imputation or validation. Following this, cross imputation was applied to the dataset ‘M’ using the imputed combined dataset. Finally, our model was validated using the imputed dataset ‘M’.

To further assess our model’s performance, we compared it to five existing studies [40,46,49,52,55] that provided sufficient model details—specifically, the coefficients of associated variables—enabling direct application to our data. For each study, we applied the reported coefficients of associated variables to the imputed dataset ‘M’ and compared the results with our model’s external validation. However, since each study was conducted using its own dataset, the performance may be compromised when their coefficients are applied to the imputed dataset ‘M’. Additionally, we conducted a comparison between our model and a model constructed using the imputed combined dataset, utilizing the same associated variables and methods employed in each study. Some other studies [53,54] were excluded from such comparisons because their use of different associated variables made direct evaluation infeasible, as these variables were not all present in the imputed dataset ‘M’.

3. Results

Study Participants and Missing Values

The seven selected datasets (‘A’, ‘D’, ‘E’, ‘F’, ‘G’, ‘M’, and ‘N’) from the FoRECAsT databank comprised 3,795 individual patient records. Of these, 962 records (25.3%) were excluded due to missing outcome data. The remaining 2,833 individual patient records were used for model development and validation. Following data cleaning and preparation, these records were subjected to feature selection and modelling to construct the amenorrhoea risk prediction model. Detailed summary statistics for the selected datasets can be found in Table 1.

The model construction and validation dataset comprised of 6 datasets (‘A’, ‘D’, ‘E’, ‘F’, ‘G’, and ‘N’) containing 2,679 individual patient records with 53 features, reflecting an amenorrhea prevalence rate of 48.3%. The combined dataset exhibited 61.4% missingness, with 91,464 missing cells out of 149,024 total cells (Table 1). The missing rate for each feature is detailed in Supplementary File 1, while the explanations of these features can be found in Supplementary File 2.

Model Development and Associated Variables

Sixty-two variables from 53 features are summarized in Supplementary File 3. During feature selection, twenty-two variables from 20 features had significant relevance (P-value < 0.05) and appeared at least 50 times in the 100 rounds of repetition. Consequently, these 20 features were selected for the final model building phase, which underwent another 100 rounds of repetition. In this phase, two features, namely “Contraception” and “Radiotherapy treatment”, were further excluded due to their corresponding variables appearing less than 50 times. In the final model, 20 variables from 18 features, along with the intercept value, were utilized to predict amenorrhea at 12 months after chemotherapy initiation (Table 2). The average coefficients from the 100-round repeats were used for the final model. Utilizing the coefficient values of the final model, we can predict a patient’s risk of amenorrhea and calculate the odds using the following formulas:

$$probability={\displaystyle \frac{1}{1+e^{-\left({\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\dots +{\beta }_{20}{x}_{20}\right)}}}$$

$$odds={\displaystyle \frac{probabilityofevent}{1-probabilityofevent}}$$

Our model identified 20 variables associated with amenorrhea (Table 2). According to our findings, the BRCA pathogenic variants emerged as the leading predictor of amenorrhea. BRCA2 carriers exhibited 356% higher odds compared to BRCA non-carriers, while BRCA1 carriers showed 74% higher odds compared to BRCA non-carriers. Another significant variable was the number of treatment cycles involving a combination of adriamycin, cyclophosphamide, and methotrexate, fluorouracil (AC+CMF cycles), with each additional cycle associated with 68% higher odds. Furthermore, each additional year of age was associated with a 6% increase in odds. For a comprehensive overview of our findings, please refer to Table 2.

Model Evaluation

During internal validation, the AUC was 0.820 (95% CI: 0.817-0.823) (Figure 2A). Given the objective of prioritizing sensitivity over precision in predicting amenorrhea, the F score with β = 1.5 was maximized to derive a cutoff value of 0.20. As a result, the sensitivity reached 91.3% while precision stood at 61.7%. Figure 2B provides a clear visualization, indicating that most patients experiencing amenorrhea at 12 months after treatment initiation fell to the right of the cutoff threshold. Details of the calibrated predicted risks can be found in Table 3.

For external validation, the independent dataset ‘M’ was employed, comprising 154 patients and 13 features, with the prevalence of amenorrhea being 54.5% and 27.1% missingness (Table 1). All features present in the dataset ‘M’ were also included in the combined dataset (Supplementary File 4). Notably, age was the only feature available in dataset ‘M’ that overlapped with the 18 features required by the final model (Table 2 and Supplementary File 4). However, the remaining 17 required features were cross imputed using the existing 13 features available in ‘M’. Following the external validation procedure (Figure 1B), we obtained an AUC of 0.743 and calculated a 95% CI of 0.666-0.818 using bootstrapping. Employing a cutoff of 0.20, the sensitivity and precision were determined as 92.9% and 60.0%, respectively. A lower cutoff of 0.20 was selected to achieve higher sensitivity, as false negatives—patients incorrectly classified as likely to regain menses— potentially have more profound impact than false positives due to missed opportunity for timely fertility preservation for those who wish to pursue it.

Our model’s performance underwent further assessment through comparison with the five existing studies [40,46,49,52,55]. Applying these studies to the imputed dataset ‘M’ yielded AUC values of 0.459, 0.506, 0.54, 0.6 and 0.61, respectively. Additionally, comparisons were made with models built using the imputed combined dataset, utilising the same associated variables and methods employed in the studies. These comparisons resulted in AUC values of 0.459, 0.506, 0.582, 0.588 and 0.612, respectively (Figure 3). Notably, both sets of comparisons underscored the superior performance of our model.

4. Discussion

We developed and externally validated a machine learning model to accurately predict treatment-related amenorrhea in young women with breast cancer 12 months after chemotherapy initiation. The model achieved strong discrimination in both internal and external validation and was optimised for high sensitivity to minimise missed opportunities for fertility preservation, a priority in clinical decision-making for this population. This is the largest and most diverse dataset used for this outcome to date, with the integration of six international datasets from the FoRECAsT databank combined with robust methods to handle substantial missingness and select relevant features, produced a model that outperformed five existing prediction models when applied to an independent dataset.

A key strength of this work is the integration of multiple datasets from diverse clinical settings, encompassing wide variation in patient characteristics, treatment regimens, and follow-up patterns. This diversity created a more representative and comprehensive feature space than in previous studies. Previous studies focused on identifying a limited number of variables associated with amenorrhea or building predictive models with a handful of features. However, constructing a precise amenorrhea risk prediction model demands careful consideration of all potentially relevant features. Our study was the first to screen many potentially relevant features to build a predictive model. In contrast to conventional logistic regression approaches, our study leveraged Lasso regression to construct the model. This choice allowed for the automatic selection of pertinent predictors, and effective mitigation of overfitting while preserving the model’s interpretability. Importantly, we deliberately avoided black-box approaches, as model interpretability is essential in healthcare settings to ensure transparency, trust, and reduced bias [63]. The robustness of our approach is evident from the superior performance of our model during external validation compared to prior five existing studies [40,46,49,52,55]. This study is a part of the broader FoRECAsT research project, with the goal of developing a web-based clinical predictive tool for amenorrhea risk, similar to predicting the survival rates after surgery with different treatment combinations for young women with breast cancer [64].

The findings from this study should be interpreted with caution, as the 12-month amenorrhoea outcome from the chemotherapy initiation does not necessarily reflect long-term implications. Some women may regain menses later if sufficient primordial follicles remain in the resting pool. In contrast, others may experience delayed amenorrhoea beyond this point as ovarian reserve continues to decline over time. Our aim in this study was to demonstrate how different clinical datasets can be compiled and utilised—with imputation—within this modelling framework for future risk prediction. Our subsequent publications will provide further insight by reporting long-term amenorrhoea outcomes over a 12–60-month period.

In modelling, different studies may employ various ML algorithms and feature selections, making direct comparisons of coefficient values for shared variables challenging. Instead, the focus should be on evaluating the relevance of features and their impact on risk. Liem et al. (2015) [55] identified only older age as a risk factor for amenorrhea, with BMI, smoking, chemotherapy, and trastuzumab showing no significant relevance. Zhang et al. (2018) [49] found that older age, lower E2, and higher FSH were associated with increased risk, while BMI was not relevant. Ruddy et al. (2021) [52] suggested that older age, lower BMI, and lower AMH increased risk, with chemotherapy and tamoxifen showing no relevance. Poorvu et al. (2021) [54] indicated that older age, lower BMI, tamoxifen, and chemotherapy were linked to increased risk, with smoking showing no significant relevance. Kabirian et al. (2023) [53] identified older age, hot flashes at diagnosis, endocrine therapy, and trastuzumab as factors increasing risk, with BMI and smoking showing no significant relevance. These findings indicate that the associated variable sets found in previous studies do not always match. In our study, we examined 53 potential features and identified factors such as BRCA mutations, a higher number of AC+CMF cycles, older age, lower E2, lower AMH, and higher FSH as contributors to increased amenorrhea risk (Table 2). In contrast, factors such as lower BMI, smoking, trastuzumab, and tamoxifen were considered during feature selection but were excluded from the final model due to their limited importance (Supplementary File 3). Overall, our study’s findings regarding features associated with amenorrhea align with most of those reported in previous studies, providing consistency and further insight into the predictors of amenorrhea.

Cross imputation played a pivotal role in the development and application of our model. Previous studies have taken varied approaches to handling missing values. Some studies did not explicitly address missing values [55], while others treated them as a separate category [49] or removed records with missing values entirely [52]. Another study predicting primary ovarian insufficiency after chemotherapy used multiple imputations by chained equations (MICE) to handle missing values, assuming data were missing at random [65]. However, given the nature of our combined dataset, treating missing values in the same manner as these studies was not feasible. Instead, we employed cross imputation, which allowed us to generate a fully imputed dataset comprising 2,679 samples with 53 features for constructing our model. These 53 features include 18 modelling features utilized by our model and 35 non-modelling features. When applying our model to an external dataset, it is plausible that the external dataset may lack some modelling features while containing some non-modelling features. Although the non-modelling features do not contribute to prediction directly, they are utilized for imputing modelling features in the external dataset during cross imputation (Figure 1B). This approach enables our model to be applied to external datasets collected in diverse research settings. The external validation conducted with the dataset ‘M’ underscored the robustness of our approach and demonstrated the strength of our study. Furthermore, this cross-imputation approach holds promise for adaptation by other medical researchers seeking to build predictive models in similar contexts.

Limitations

This study has several limitations. First, despite being the largest dataset assembled to date for this outcome, some contributing datasets were small (e.g. 96 to 154 records) and showed marked variation in amenorrhea prevalence (10.8 to 78.1%), which may introduce bias. Secondly, the model was externally validated on a single independent dataset; broader validation across multiple independent and prospective cohorts where there are primary data on the selected features is needed to confirm generalisability. Thirdly, two chemotherapy-related variables (CMF dose and CMF treatment) showed an inverse association with amenorrhea, contrary to clinical expectations, likely reflecting multicollinearity with other chemotherapy measures or dataset heterogeneity; these should be interpreted within the context of the full model. Fourthly, while the model was reduced to 18 features to balance accuracy and feasibility, some settings may still require fewer inputs, future work could explore a panel of models with varying complexity to suit different contexts. Finally, our outcome—amenorrhea at 12 months after chemotherapy initiation, does not fully capture long-term ovarian function; some women may recover menses later, while others may develop delayed amenorrhea. Extending the prediction horizon to 12–60 months is a priority for future research.

5. Conclusions

We presented a robust, interpretable, and externally validated machine learning model for predicting treatment-related amenorrhea in young women with breast cancer. By integrating diverse datasets and applying rigorous imputation and feature selection methods, our model incorporating 20 established and novel variables achieved superior predictive performance compared with five existing models. The framework is adaptable to other clinical contexts where heterogeneous datasets with substantial missingness must be combined to develop reliable prediction models, offering a scalable approach to precision oncofertility and broader precision oncology applications.