Machine Learning Classifiers for Predictive Biomarkers Combining Clinical and Radiomic Data in Testicular Cancer

1. Introduction

Testicular germ cell tumours (TGCTs) are the most common form of cancer in males between 15 and 40 years (1,2). This disease is distinguished from other types of cancer by its unique patient population and high treatment success rates, representing an outstanding achievement in cancer treatment (3,4).

Apart from cure, minimising treatment's immediate and long-term side effects is the main objective. This is of particular importance because of the young age of the patients and their longevity after cure (5–10).

TGCTs are diagnosed by physical examination, ultrasound scans and specific markers such as alpha-fetoprotein (AFP), beta-hCG and lactate dehydrogenase (LDH) (11,12).There are two main histological types: seminoma and non-seminoma, with the latter including both pure non-seminoma and mixed germ cell tumours.

Considering that 95% of metastases from testicular germ cell tumours involve the ipsilateral retroperitoneal lymph nodes, German guidelines recommend adjuvant therapy (one to two cycles of carboplatin or 20 Gy paraaortic radiotherapy) for early-stage seminoma if the tumour is larger than 4 cm in diameter (13).

Although the common tumour markers AFP, beta-HCG and LDH are widely used, they lack specificity and are present in approximately 60% of cases of testicular cancer (12,14). Furthermore, conditions such as liver disease or genetic factors can cause these markers to become falsely elevated (15). Without reliable biomarkers for predicting recurrence, TGCT patients are usually followed with active surveillance (16).

CT scans are effective at detecting enlarged lymph nodes. However, they have difficulty distinguishing between benign and infiltrated nodes, especially smaller ones (17). Lymph nodes are usually considered malignant if their short-axis diameter is between 7-8 mm, but it's important to note that micrometastatic lymph nodes can be present in 30% of cases (18). To date, imaging, tumour markers and clinical nomograms are unreliable in predicting lymph node metastasis (19,20).

Significant advances have been made in medical imaging by integrating high-resolution imaging, advanced computing technologies and artificial intelligence (AI). This integration has paved the way for the emerging field of radiomics (21,22).

Radiomics enables the identification and extraction of specific diagnostic image patterns, which are then transformed into quantifiable and analysable 'big data' through data characterisation algorithms (23,24). In precision medicine, AI image analysis can help identify important image details that human radiologists may miss, offering repeatable and accessible ways to assess challenging lesions in the body to improve the detection, classification and monitoring of both the primary tumour and its associated metastases in various cancers, such as gastric, rectal and bladder cancers (25–28).

Our presented study combines machine learning-based radiomics with clinical predictors to improve the accuracy of predicting lymph node metastasis in early-stage TGCT patients. This advancement has the potential to significantly improve the accuracy of imaging in the clinical oncology setting.

2. Materials and Methods

2.1. Patients

This retrospective study includes retroperitoneal lymph nodes of early-stage testicular cancer patients from January 2006 to December 2016.

A comprehensive review of electronic medical records and the radiology information system was used to collect patient demographic, laboratory and clinical data. Incomplete clinical or imaging records as well as missing histological confirmation following surgery were used as exclusion criteria.

The study's primary objective was to investigate retroperitoneal LN metastases from TGCT using clinical and imaging studies retrieved from electronic medical records. From an initial screening of 167 patients, only 91 patients met the selection criteria and were part of the final cohort.

Figure 1. Recruitment pathway.

Figure 1. Recruitment pathway.

2.2. Image acquisition and segmentation

All patients underwent contrast-enhanced CT according to standard clinical scanning protocols (detailed scanning parameters are provided in supplemental table S1) The images were acquired as part of the routine staging procedure prior to orchiectomy to determine the status of the disease (+/- 2 weeks, mean time 3 ± 11 days, range 2 to 24 days).

For image segmentation and analysis, all reconstructed images were retrieved from the hospital's picture archiving and communication system (PACS).

2.3. Segmentation and Radiomic Feature Extraction

The evaluation of image features, such as histogram features or features from the co-occurrence matrix, was first introduced by Haralick et al. in 1973 (29) and has shown considerable potential in different types of cancer and for different types of questions (22,30).

Two experienced radiologists (>10 years' experience in interpreting CT scans and with a strong background in texture analysis) contoured three retroperitoneal infrarenal lymph nodes per patient using Mint Lesion software (v3.8.4). Regions of interest (ROIs) were drawn on the CT images along the lesion contour on each successive slice within the boundaries of the retroperitoneal lymph node, excluding adjacent vessels, fat and normal tissue. The flowchart illustrating the ROI segmentation and feature extraction used to develop the model is shown in Figure 2.

Texture feature descriptors were used according to the guidelines of the Image Biomarker Standardisation Initiative (IBSI) (24). A total of 85 image features were extracted from each of the 273 ROIs, covering the size and shape of the lymph node in three dimensions. In addition, the distribution of voxel intensities within the ROI was described using first-order statistics. Texture-based features were calculated from the grey level co-occurrence matrix (GLCM) to capture voxel intensity patterns. A list of all features and parameters is provided in supplemental table S1.

2.4. Development of the Predictive Machine Learning Models

We used four classic machine learning algorithms to identify the best radiomics model for predicting lymph node metastases in testicular cancer: Random Forest (RF), Light Gradient Boosting Machine (LGBM), Support Vector Machine Classifier (SVC), and K-nearest neighbors (KNN) classifiers.

To improve the model's performance and maximise the area under the receiver operating characteristic curve (AUC-ROC), the optimal hyperparameters of the model were determined using a grid search procedure (31). (Details are provided in supplemental table S2).

We constructed a radiomics-only, clinical-only, and combined radiomics-clinical prediction model for each classifier.

Clinical factors included in our analysis were age, pre-orchiectomy serum tumour markers AFP and B-HCG, histotype (seminoma vs. non-seminoma) and body mass index (BMI). These factors were included because they all have shown predictive value in TGCT (32–35).

The predictive performance of each classifier for LN metastases was evaluated using receiver operating characteristic (ROC) curve analysis. The clinical utility of the predictive models was assessed using decision curve analysis (DCA) which evaluates the net benefit of the predictive models at different cut-offs in the training population and compares the performance of the models.

The clinical radiomics prediction models were constructed using custom-developed software implemented using the Python Scikit-learn package (36,37). (Details are provided in supplemental table S2).

3. Results

3.1. Clinicopathological Characteristics

Age, AFP levels, HCG levels, histotype and body mass index (BMI) were not statistically significantly different between LNM-positive and LNM-negative patients.

The baseline clinicopathological characteristics of the patients are listed in supplemental table S3).

3.2. Dataset Characteristics and Preprocessing

Three infrarenal retroperitoneal lymph nodes were segmented per patient, yielding 273 sample instances. We used a group shuffle split to divide the data into training (70%) and test (30%) groups on a patient-by-patient basis. This patient-by-patient split ensured that the lymph nodes of a given patient would remain together in either the training set or the test set. A tenfold cross-validation checked the robustness of the procedure.

There were 33 instances in the category “relapse of disease in terms of lymph node metastases” (minority class) and 240 instances in the category “without relapse of disease” (majority class).

Due to class imbalance in the dataset, we used the SMOTE oversampling technique to balance the data (38). The balanced data were used as input variables for the machine learning modelling.

3.3. Performance Evaluation of the Prediction Models

85 CT-derived radiomic features were fed into the machine-learning models using RF, LGBM, SVC and KNN (a list of all features and parameters is provided in supplemental table S1).

Based on the random forest algorithm, the combined clinical-radiomics model showed the best prediction accuracy with 87% (AUC 0.95±0.03; 95% CI), indicating that adding radiomics features could improve the predictive performance. The second best performer was LGBM with an AUC of 0.93 (±0.05; 95% CI), accuracy 83%, precision 87%, recall 80% and F1 score 82%. Details of the performance of the radiomics-only, clinical-only and combined clinical-radiomics models of all classifiers are shown in supplementary Table S4. See Figure 3 with the merged confusion matrix of the Random Forest-based models using radiomics-only and clinical-only values for classification.

Figure 4 shows the receiver operating characteristic (ROC) curves for the clinical, the radiomics and the combined clinical-radiomics models based on random forest algorithms.

Figure 5 shows the receiver operating characteristic (ROC) curves for the clinical, the radiomics and the combined clinical-radiomics models for the RF, LGBM, SVC and KNN.

To assess the clinical utility of the clinical-radiomics model combination, a decision curve analysis was performed. Figure 6 shows the net utility versus threshold probability trade-offs between true positives and false positives.

4. Discussion

Medical imaging has made great advances in cancer diagnosis and treatment planning with the emergence of "radiomics" - a field involving high-throughput medical image data mining. Radiomics has significant potential to improve clinical decision support in cancer care by providing a non-invasive and cost-effective approach (23,39). As radiomics deals with large amounts of medical image data ("big data"), efficient methods are needed to extract relevant information from these large radiomic datasets (21).

In this study, we analysed the performance of different machine learning methods, namely random forest (RF), light gradient boosting machine (LGBM), support vector machine classifier (SVC) and k-nearest neighbour classifier (KNN), in predicting lymph node metastases in patients with early-stage testicular germ cell tumours (TGCTs).

We constructed radiomics-only, clinical-only and combined predictive models for each classifier, integrating clinical and radiomic features to identify patients who require adjuvant therapy and those who do not. We compared the results with what we found in the previous study, where we used logistic regression analysis to predict lymph node metastases.

Our main findings can be summarised by the following:

The combined radiomics–clinical model based on the random forest algorithm showed the best predictive accuracy of 87% (AUC 0.95 ± 0.03; 95% CI), indicating that the addition of clinical features improved the predictive performance (accuracy radiomics-only model 85% vs. clinical only model 79 %).

The LGBM classifier performed second best with a prediction accuracy of 83% (AUC 0.93 ±0.05; 95% CI). In contrast to the RF, the addition of clinical features to the radiomics prediction model led to a worsening of the predictive performance (accuracy radiomics-only model 85% vs. clinical-only model 73%).

In our analysis, SVC- and KNN-based prediction models performed significantly worse than RF and LGBM. However, it is worth noting that in both cases the combined radiomics-clinical model outperformed the radiomics-only prediction models.

This is in line with the results of our previous study (40) in which we demonstrated that logistic regression analysis is useful for the prediction of lymph node metastasis in TGCT patients, with the best predictive performance being the combined clinical-radiomics model with a predictive accuracy of 87% (AUC 0.95 ± 0.03; 95% CI)

The serum biomarkers AFP, β-HCG and LDH play an important role in the diagnosis and prognosis of TGCTs and are included in the prognostic index of the International Germ Cell Cancer Consensus Group(41). However, their sensitivity is limited, as around 40% of men have 'normal' levels at recurrence (42). Studies have suggested additional prognostic factors such as age and BMI, but their role is unclear and continues to be debated (32–35).

So far, there is no evidence that imaging, preoperative serum tumour markers or clinical nomograms can reliably predict nodal involvement (19,20). Inadequate management of TGCT patients places them at risk of adverse outcomes, as both overtreatment and undertreatment carry equal risks.

There are few studies on discriminating between benign and malignant LNs in testicular cancer. In patients with metastatic non-seminomatous testicular germ cell tumours, Baessler et al. (43) found that a CT radiomics-based machine learning classifier could predict lymph node histopathology after dissection following chemotherapy. They used a support vector machine learning classifier in their single-centre retrospective study of 80 patients and 204 lesions. The model distinguished between benign and malignant histopathologies with an accuracy of 81%.

But in contrast to our study, they didn't include clinical factors in their radiomics approach to improve diagnostic accuracy. They also split their moderate-sized dataset into three subgroups: 63 patients were assigned to train and only 19 to test. The splitting of data for validation purposes is common practice. But doing so reduces statistical power because the sample sizes in both groups are smaller than in the initial sample.

To overcome this problem, we used a cross-validation technique that makes use of multiple splits of the data to avoid overfitting while still providing accurate estimates of the coefficients of the model (44).

We are confident that our combined prediction model will generalise better to novel data due to our tenfold cross-validation approach, the a priori inhomogeneity of our dataset and the integration of clinical risk factors. Therefore, future prospective studies should be conducted to further validate our trained model.

To date, several clinical models have been developed to predict the dignity of LN metastases. But, these models have shown inconsistent results and have not yet been adopted for clinical decision-making (19,45,46).

In summary, the identification and implementation of novel biomarkers may be helpful for early diagnosis and disease recurrence monitoring.

The present study, however, has some limitations that we acknowledge. First, the study's retrospective nature and the small cohort sizes might have led to unavoidable selection bias.

Secondly, two different scanners were used to acquire the CT images. Thirdly, the results of this study were obtained from a single centre. Due to the high cure rate of stage I TGCTs, it is a challenge for power studies to investigate prognostic and predictive factors adequately. Therefore, prospective and multicentre validation is warranted to provide higher-level evidence in the following studies.

Fourth, due to small sample size and relapse events, we could not include the classical prognostic pathohistological factors (primary tumour size and rete testis invasion for seminoma and lymphovascular invasion and presence of embryonal carcinoma for non-seminoma) in our analysis. Their inclusion in the combined radiomics-clinical model may have further improved accuracy and is a promising addition for future validation studies.

Finally, in addition to protein-based tumour markers, non-coding RNAs, especially stem cell-associated microRNAs such as miR-371a-3p and miR-302/367 clusters, show superior sensitivity compared to traditional markers in the detection of newly diagnosed TGCT patients, demonstrating their potential as biomarkers (47,48).

5. Conclusion

In summary, our combined random forest-based radiomics-clinical model represents an exciting tool for better prediction of lymph node involvement in early-stage TGCTs, with the potential to reduce over- and undertreatment in this young patient population. Further validation in larger prospective clinical trials should combine this approach with novel clinical biomarkers.