Detection of Local Prostate Cancer Recurrence from PET/CT Scans Using Deep Learning

1. Introduction

Prostate cancer remains one of the leading causes of cancer-related deaths in men, with over 1.2 million new cases diagnosed and more than 350,000 deaths annually [1,2]. Early detection and accurate staging are critical for successful treatment, and monitoring therapy effectiveness is typically done by tracking serum prostate-specific antigen (PSA) levels. When PSA levels rise, indicating biochemical recurrence, patients are often referred for re-staging, a process that is crucial for determining the most appropriate treatment strategy. A key focus during re-staging is identifying whether there is a local recurrence of cancer in the prostate region. Several diagnostic methods are used for staging and detecting prostate cancer, including digital rectal examination, transrectal ultrasound (TRUS), magnetic resonance imaging (MRI), and positron emission tomography (PET) combined with computed tomography (CT). Among these, PET/CT has emerged as a reliable technique. In ¹⁸F-PSMA-PET, the radioactive tracer ¹⁸ F-PSMA-1007 binds selectively to prostate-specific membrane antigen (PSMA), a protein highly expressed on prostate cancer cells and is performed in combination with the CT scan allowing for precise anatomical imaging. Typically, a whole-body PET/CT scan is conducted to detect potential metastases. Specifically, for prostate cancer diagnosis and staging during remission, each PET/CT examination generates high-dimensional data, comprising hundreds of image slices with thousands of pixels per slice for both the PET and CT parts. Interpreting this complex data to answer the critical binary question—whether there is a local recurrence of prostate cancer in the prostate bed or not—requires trained experts.

Recently, artificial neural networks have shown great potential in assisting with similar tasks, such as tumor segmentation and outcome prediction in head and neck cancers [3], as well as the identification of pathological mediastinal lymph nodes in lung cancer [4]. These advancements highlight the promise of AI in improving diagnostic accuracy and efficiency in prostate cancer staging and treatment [5].

Therefore, the aim of this work was to develop a neural network capable of detecting local recurrence in the prostate or prostate bed with high accuracy of at least 90% using ¹⁸F-PSMA-1007 PET/CT examinations.

2. Materials and Methods

2.1. Study Population

Details concerning the study group are shown in Table 1. Briefly, 1074 patients with 1459 ¹⁸F-PSMA-1007 PET/CT scans performed at the Department for Nuclear Medicine at the University Hospital Würzburg between 2019 and 2023 were included in this study. The training and validation set consists of 1259 PET/CT scans, while 200 scans were used for the test set.

We used hybrid PET/CT scanners with an extended field-of-view for the PET and a 64- or 128-slice spiral CT (Biograph64 or 128, Siemens Healthineers; Erlangen, Germany). All cases were re-evaluated explicitly for this project by a trained nuclear medicine physician (H.E.) to label each examination with 0 (no local recurrence in the prostate region, n=658), 1 (local recurrence in the prostate region, n=737), and 2 (uncertain whether there is local recurrence in the prostate region). Instances with label 2 were excluded from training. In summary, the following metadata were available for each examination besides the PET/CT scans: patient pseudo-ID, age, sex, staging, prostatectomy status, PSA level, and label.

Even though the question of local recurrence makes sense only at restaging we decided to include primary staging patients in the training as they are examples of scans with cancerous tissue in the prostate region. We assigned all primary staging scans to the training set. The remaining examinations were assigned randomly into training and validation sets, ensuring that all scans from the same patient ended up in the same set.

The test set comprises an additional 200 scans from 198 patients on the same scanners. This set contains re-staging examinations only. However, 83 patients (84 scans) already had scans in the training/validation set, while 115 patients (116 scans) were novel. Even for known patients, these are new examinations, and both metadata and the label might have changed. The parameters for measurement were identical to those from the training/validation set; the same metadata were supplied, and the same labeling procedure was employed.

2.2. Data Processing

Images were exported from the clinic PACS in DICOM format and converted to NIfTI via dcm2niix [7]. In 59 cases, the conversion failed, so these cases were excluded. Data loaders for deep learning in Python (version 3.12.3) were implemented using nibabel (version 5.2.1, [8]), PyTorch (version 2.3.0, [9]) and Monai (version 1.3.1, [10]). CT and PET images for each patient are measured at different resolutions but cover the same volume (Figure 1). Therefore, after reading both files individually, they are re-scaled to a standard resolution of 150x150x150. Additional transformations like cropping or augmentation transformations are applied depending on the model. Metadata and labels are provided as a simple csv file and loaded via Pandas (version 2.2.2, [11,12]).

2.3. Models

As part of this project, many models have been trained to explore the effects of including different kinds of data, pre-processing, and deep learning techniques. New models build on previous successful models. Besides initial experimentation [13], more than twenty models and variants were trained systematically. Here, we only focus on four models that show incremental improvements. All other models and their variants are described in Appendix A. Source code, and weights are shared in the GitHub repository and on Zenodo. All models use the DenseNet architecture [14] with 121 layers, as implemented in Monai [10]. All models used cross-entropy as the loss function, the Adam optimizer, and a learning rate of 10^-5 for 15 epochs. The batch size was set to 16, except for Model A, where this exceeded the GPU memory; in this case, a batch size of 8 was used. After each epoch, the validation accuracy was logged, and the weights were saved. We report the maximum accuracy and provide the model weights from the epoch that reached this maximum. Training was performed on an NVIDIA GeForce RTX 4090 GPU with 24 GB of RAM.

2.3.1. Model A (1) – Base Model

The initial model is supposed to provide the baseline performance, with the most naïve approach, just passing the resized data to a DenseNet121 including moderate augmentation (random rotations by up to 0.2 radians).

2.3.2. Model B (2) – Cropped Around the Prostate

The data includes the whole body, while the specific question (local recurrence) can be answered by focusing on the area around the prostate (region). To automatically crop image volumes around the prostate, the location of the prostate is determined by TotalSegmentator (version 2.1.0, [15]). This worked in most cases, even for patients after radical prostatectomy. In the remaining cases, the position of the urinary bladder was used as a proxy (Figure 2). Images were cropped to a 10cm x 10cm x 10cm cube around the centroid of the prostate (or urinary bladder) in patient coordinates and re-scaled to 70x70x70 voxels. Variants with stronger cropping have been explored (Appendix A).

2.3.3. Model C (6c) – Adding Prostatectomy Status and PSA Level

The status of prostatectomy (px, 0 = no prostatectomy, 1= radical prostatectomy) and the PSA levels are potentially informative. Therefore, this information was added to the model as separate image layers with repeated values (0 or 1 for px and a floating-point number with the normalized PSA level). Training separate models for px = 0 and px = 1 had high accuracy but low balanced accuracy as it overfits the respective majority class (Appendix A).

2.3.4. Model D (7d) – More Extensive Augmentation and Hyperparameter Optimization

Training usually converged rather quickly. So, to increase data heterogeneity (without generating new training sets), we employed more extensive augmentation. With a growing number of augmentation transformations, the number of hyperparameters to tune also increased. Therefore, we applied the hyperparameter optimization toolkit Optuna [16] to find optimal hyperparameter combinations for spatial cropping around the center (CenterSpatialCropd), random flipping (RandFlipd), and random zooming (RandZoomd).

2.4. Evaluation

The classification performance of all models was evaluated using the accuracy and the balanced accuracy of the validation set. Accuracy is the number of correct predictions divided by the number of total predictions. This metric is susceptible to class imbalance. [17]. Therefore, this metric is complemented by the balanced accuracy, which is equivalent to the arithmetic mean of sensitivity and specificity in the case of binary classification.

Only the best-performing model was evaluated on the hold-out test set. Missing values were imputed before prediction in the few cases with missing metadata (two cases missing px, eleven cases missing PSA). For px, the majority class (px=1) and for PSA the mean value (30.1) were used. In addition to accuracy and balanced accuracy, the confusion matrix is reported. Furthermore, accuracy is separately determined for known patients and novel patients.

3. Results

The initial model A, which used the entire volume with moderate augmentation, reached an accuracy of 61.3% with a balanced accuracy of 48.7% (Table 2). Restricting the input data to a 70x70x70 volume around the prostate gland, model B reached an accuracy of 70.7% and a balanced accuracy of 66.9%. Including prostatectomy status and PSA levels as additional data layers (model C) further increased the accuracy to 75.9% with a balanced accuracy of 72.3%. Finally, using Optuna to optimize the hyperparameters and augmentation settings returned the following settings: CenterSpatialCropd with dimensions 65x46x69, RandFlipd along the spatial axis 1 with a probability of 1 (100%), and RandZoomd with a minimum and maximum zoom of 0.5 and a probability of 1 (100%). The model D trained with these settings had the highest accuracy (77.1%) with slightly reduced balanced accuracy (70.6%). Neither of these models comes close to our target accuracy of 90%. As the model with the highest validation accuracy (Table 2), we selected model 7d for evaluation on the hold-out test set.

Model D reached an accuracy of 68.5 % and a balanced accuracy of 70.3 % on the test set (Table 3). While the model had high specificity (86.5%), the sensitivity was poor (54.1%). In 116 cases, the patients were not previously known. Of these cases, 73.3% were correctly identified, while 61.9% of the 84 cases with previously known patients were correctly identified.

4. Discussion

This work aimed to develop a highly accurate neural network to predict local recurrence in prostate cancer patients from PSMA-PET/CT images. We iteratively refined our approach as the initial naïve model had insufficient accuracy. Specifically, we cropped the images around the prostate gland to focus on this region, included important metadata as separate image layers and performed a hyperparameter optimization of certain augmentations to increase the data heterogeneity (Table 1). Furthermore, we modified the dimensions of the region of interest as well as tried masking the PET channel in certain organs or everywhere except the prostate to let the model focus on the prostate alone (Appendix A). While these refinements generally increased the accuracy, the final test set accuracy of 68.5% is still far from acceptable in a clinical context. In particular, the poor sensitivity is concerning in this context as it corresponds to patients with local recurrence not being identified. Sensitivity can generally be increased using a weighted loss function or adapting the threshold for the predicted probability. However, this usually trades specificity for sensitivity. As the overall performance of our models was far from ideal, we did not attempt fine-tuning this trade-off. While we cannot postulate that we reached a theoretical accuracy limit for this task (given our data set), we employed many advanced techniques to improve our models iteratively and reached a comparable accuracy to the 71.4% previously reported for the prediction of prostate cancer recurrence based on ¹⁸F-FACBC PET [18]. By documenting this evolution and publishing all code and models, we provide a blueprint for approaching DL to answer a medical research question. While it is undoubtedly possible to marginally increase the accuracy of artificial neural networks by extending the hyperparameter space (e.g., by exploring different network architectures), we postulate that to reach accuracies above 90%, a completely different technique, sophisticated domain-specific adaptations (e.g. attenuation correction[19]) or much more data is required. Transfer learning is an interesting technique that lowers the necessary training data. [20]. A pre-trained model on a related task is required to perform transfer learning. Alternatively, a large dataset for a related task can be used to pre-train a model yourself. We could not find any suitable model or dataset, but now we provide our models as starting points for transfer learning by others. As datasets can usually not be shared because of data protection requirements, we hope our models will be used for TL and that the resulting models will be shared again. We plan to evaluate the performance of an iteratively re-trained model on the initial dataset.

5. Conclusions

For the presented task, 1259 examinations were insufficient to reach an accuracy of over 90% even when employing data augmentation, including additional metadata and performing automated hyperparameter optimization. We facilitate future research and the development of better models by openly sharing our source code and all pre-trained models for transfer learning.