Artificial Intelligence Assisted Detection and Localization of Spinal Metastases

1. Introduction

1.1. Machine Learning in Healthcare

Radiomics presents itself as a novel discipline of artificial intelligence. It refers to the analysis of medical images extracting malignancy-associated information – pathophysiology significant data integrated into mathematical parameters. [1] There are clinical portfolios for the integration of radiomics. First, the initial step relates to a clearly defined target population and the use of radiomics, which leads to improvements compared with standard-of-care examination diagnostics. In a second aspect, the technical approach of statistical elements must be considered. Throughout image stratification, minimizing variability is suggested. Machine learning aspects such as possible bias and other computational parameters could be specified, for example, within Bayesian updating. In a third aspect, the experiment is expected to have reproducible test performance. [2] Another important aspect reflects the European Union’s General Data Protection Regulation to enhance transparency in clinical applications. [1]

A deep residual convolutional neural network is used for the detection of metastatic bone lesions with automatically segmented regions. Deep-learning segmentation can be adapted into Computed Tomography scan (CT scan) and Magnetic Resonance Imaging (MRI). [3] Furthermore, a CT scan can reach up to 74% sensitivity and 56% specificity. CT scan provides whole skeletal system assessment and systemic staging with the benefit minimizing patient`s radioactivity performance. Moreover, MRI shows the finest resolution for bone and soft tissue. The proposed sensitivity is 95% and specificity is 90%. [1]

Bone metastases can be diagnosed using a variety of imaging modalities, each offering specific advantages and limitations. Plain radiographs (X-rays) are typically the first imaging method for patients presenting with bone pain. However, X-rays have limited utility in asymptomatic patients and for evaluating bones with a high cortex-to-marrow ratio, such as ribs, due to their low sensitivity in detecting subtle or early-stage metastases. [4] Computed tomography (CT) is preferred for these cases, offering superior resolution of cortical and trabecular bone structures, and allowing for adjustments in window width and level. CT scans provide a detailed view across multiple planes, improving diagnostic sensitivity and specificity. According to recent studies, CT has a pooled sensitivity of 72.9% and a specificity of 94.8% in detecting bone metastases, particularly in areas like the ribs. [5]

In oncology, CT is commonly used for staging and follow-up in cancers affecting the thorax and abdomen, as it covers large portions of the axial skeleton. This enables clinicians to not only assess the spread of metastatic disease but also distinguish between metastatic and degenerative changes. CT also assists in evaluating structural abnormalities that may have been detected with other modalities, such as magnetic resonance imaging (MRI) and scintigraphy. [6]

Although other imaging modalities, such as MRI, positron emission tomography/computed tomography (PET/CT), single-photon emission computed tomography (SPECT), and bone scintigraphy, offer higher sensitivity (91% for MRI, 90% for PET/CT, and 86% for scintigraphy), these methods are often not used for routine screening or follow-up due to their higher cost and limited availability, particularly in countries with constrained healthcare resources. [7] For instance, MRI is typically reserved for cases requiring detailed soft-tissue contrast or assessing bone marrow involvement, while PET/CT and scintigraphy are often used in specific cases but are less frequently employed for general screening due to expense and logistical challenges. [8]

Selecting the appropriate imaging modality based on clinical needs is crucial for optimizing diagnostic accuracy. CT remains an accessible and reliable choice for detecting bone metastases, especially in complex areas like the ribs, where other imaging techniques may offer limited utility.

Artificial intelligence (AI) is a rapidly advancing technology that has demonstrated significant potential across various domains, including medicine. AI is poised to transform numerous aspects of the medical field, such as patient care, administrative processes, diagnostics, treatment planning, and scientific research. In radiology, AI is often used in conjunction with radiomics, a technique that extracts quantitative features from medical images to uncover patterns not visible to the human eye. These radiomic features, when combined with AI methodologies such as machine learning and deep learning, can enhance diagnostic accuracy, prognostic predictions, and personalized treatment strategies. AI technologies, including machine learning, natural language processing, and robotics, can be applied independently or synergistically to analyse clinical data, generate reports, assist in diagnosing conditions, and predict treatment outcomes based on patient-specific variables. The integration of radiomics and AI has the potential to refine medical imaging analysis, offering deeper insights into disease characterization and treatment efficacy. [9]

Artificial intelligence makes sense of massive amounts of clinical, genomic, and imaging data. It can improve physician efficiency, increase diagnostic accuracy, and personalize treatment. Perioperative management, rehabilitation assistance, drug production, and new specialist education are only a few of the fields where artificial intelligence is showing its endless possibilities.

Higher diagnostic accuracy prevents unnecessary tests that take time and finances and can be a psychological burden to patients, as well as potentially dangerous due to use of ionising radiation and toxic contrast media. Based on research, artificial intelligence is showing promise with high accuracy in diagnostics of the following specializations:

• Radiology: recognition of tuberculosis in chest X-ray images, benign and malignant nodules in lungs based on CT data, breast cancer lesions in mammography and detection and classification of other tumours.
• Pathology: differentiation of melanocytic lesions, gastric cancer types, prediction of gene mutations associated with cancer, determination of kidney function from biopsy results.
• Ophthalmology: diagnostic of retinal diseases, glaucoma, keratoconus, grading cataracts.
• Cardiology: improvement in cardiovascular risk and pulmonary hypertension patient’s outcome prediction accuracy.
• Gastroenterology: endoscopic detection of colorectal polyps, gastric and esophageal cancer, Barret’s oesophagus, squamous carcinoma cell, and other lesions.

Artificial intelligence can be also used in the therapy processes:

• Three-dimensional printing: it is possible to print individual models from patient’s data that help doctors get more visualization, make more detailed preoperative plans, and practice simulated surgery in advance. This technology can be also used with bioactive materials to make body implants.
• Virtual reality: an opportunity for surgeons to practice, improve skills, get assistance during surgeries.
• Da Vinci surgical artificial intelligence system: it has proven to be minimally invasive, provide a clearer image, make operation more accurate and convenient, provide the possibility to do the operation remotely, lower complication rate, and be beneficial in terms of postoperative recovery. [10]

The need for health care services and the advancement of artificial intelligence have resulted in the creation of conversational agents – chatbots, speech recognition screening systems – that can assist with various health-related tasks such as change of behaviour, treatment support, health monitoring, training, triage, and screening. Most studies have shown that these conversational agents are generally effective and satisfactory. [11]

Jobs in healthcare specialties involving digital information, like radiology and pathology, are more likely to get automated than those requiring direct doctor-patient contact. However, artificial intelligence is not expected to replace healthcare specialists but to support their skills and help them put in more effort in patient care and jobs that require unique human skills like empathy, persuasion, and big-picture integration. Artificial intelligence is expected to have a significant impact on the future of medicine, but its integration in healthcare also presents ethical, legal, and practical challenges that need to be carefully addressed and limited. However, further research is needed to fully understand the long-term effects and ensure the safe and effective integration of artificial intelligence-based technologies into healthcare. [12]

1.2. Vertebral Metastases

Vertebral metastases represent the secondary involvement of the vertebral spine by hematogenous disseminated metastatic cells. [13] It is the 3rd most common site of metastasis, after lungs and liver and is a major cause for morbidity, characterized by severe pain, impaired mobility, pathologic fractures etc. [14] In 90% of cases they are asymptomatic and are present in 60-70% of patients with systemic cancer.

80% of primary tumours give off bone metastases. [14] They are classified as osteolytic, osteoblastic or mixed, according to the primary mechanism of interference with normal bone remodelling. Osteolytic characterized by destruction of normal bone and sclerotic by deposition of new bone. [15] Primary tumours with predominantly osteolytic metastases are breast cancer (65-75%), thyroid cancer (65-75%), urothelial cancer (20-25%), renal cell carcinoma (20-25%), melanoma (14-45%), non-Hodgkin lymphoma and multiple myeloma. Primary tumours with predominantly sclerotic metastases are prostate (60-80%), small cell lung cancer (30-40%), Hodgkin lymphoma. [13,14,15] Mixed type of lesions is present in breast cancer (15-20%), gastrointestinal cancers and squamous cancers. [15]

Nowadays incidence of spinal metastatic disease increases due to improved patient survival and advanced diagnostic techniques. [16] The median-survival from diagnosis of bone metastasis is: 6 months in melanoma; 6-7 months in lung; 6-9 months in bladder; 12 months in renal cells carcinoma; 12-53 months in prostate; 19-25 months in BC and 48 months in thyroid. [15]

Unfortunately, no treatment has been proven to increase the life expectancy of patients with spinal metastases. The goals of therapy are pain control and functional preservation. [17] Thus it is important not only to diagnose spinal metastases, but also to follow progression of disease, evaluate stability of the vertebral column and identify patients who will benefit from surgical consultation or intervention.

There are multiple scoring systems available for evaluation of different aspects of well-being of a patient with metastatic spine disease. One of them is The Spinal Instability Neoplastic Score (SINS), which is used to evaluate spinal instability. It also acts as a prognostic tool for surgical decision making. [18]

The SINS is a scoring system based on one clinical factor (pain) and five radiographic parameters (location, bone lesion quality, spinal alignment, vertebral body collapse and involvement of posterolateral spinal elements). Each component is assigned a score reflecting its contribution to overall instability of the spinal segment. The six individual scores of each component are added for a cumulative score ranging from 0 to 18. The higher the total score signals more severe instability. [19]

In the evaluation of 131 surgically stabilized spine metastasis patients the SINS [18] demonstrated near-perfect inter- and intra-observer reliability in determining three clinically relevant categories of stability. With SINS ≥7 surgical stabilization significantly improved patients’ quality of life. [16]

The presented research serves as a proof of concept for the upcoming project in terms of which we plan to create a representative cohort group with even age, sex and oncology stage distribution among the dataset data.

Our main objective is to locate metastases in patient computer tomography if there are such. On daily bases small, vaguely visible occurrences of metastases in computer tomography could be easily missed by healthcare professionals. A well-tuned system which could indicate the region of possible disease could be crucial for a patient’s life. If there is any deformation of the bone morphology structure, the artificial intelligence can record it. What has been done manually by professionals could be done faster and with better quality by AI. After deformation region indication healthcare professionals can decide is it a metastasis or not.

U-net segmentation architecture initially was built for medical imaging data analyses. [20] Architecture as an output provide segmentation masks with the same size as input, which is ideal indication of possible metastases. For this project, a 3D version of the U-net architecture is utilized to work with the 3-dimensional nature of computer tomography data along with the 2D version. [20,21]

One study showed that a deep-learning algorithm (DLA) could help radiologists detect possible spinal cancers on CT scans. The system, which uses a U-Net-like architecture, had a sensitivity of 75% in identifying possibly malignant spinal bone lesions, considerably boosting radiologists' ability to detect incidental lesions that would otherwise go unnoticed due to scan focus or diagnostic bias. In this situation, AI serves as a second reader, significantly increasing detection sensitivity without leading. [22]

Another important component of AI in spinal metastatic imaging is its involvement in early detection and therapy, which is key for avoiding problems and enhancing patient quality of life. Recent research has looked into the use of AI approaches in image processing, diagnosis, decision support, and therapy aid, summarizing the current evidence of AI applications in spinal metastasis care. These technologies have demonstrated encouraging outcomes in boosting work productivity and lowering unpleasant events, but further study is needed to evaluate clinical performance and allow adoption into ordinary practice. [23]

There is a similar study that introduced a deep learning (DL) algorithm designed for diagnosing lumbar spondylolisthesis using lateral radiographs. This research aimed at improving the accuracy of medical diagnostics by assisting doctors in reducing errors in disease detection and treatment. The study was retrospective, involving multiple institutions, and focused on patients with lumbar spondylolisthesis. The DL model utilized included Faster R-CNN and RetinaNet for lumbar spondylolisthesis detection, showing the potential of AI to significantly enhance diagnostic accuracy in spinal condition. [24]

2. Materials and Methods

Our research is structured into two distinct stages, employing a previously validated methodology for the preprocessing of radiological data. [25] The initial stage focuses on the localization of the patient's spine, a critical step for highlighting regions potentially affected by spinal metastases. In this stage, each vertebra is isolated and segmented starting from the cervical region down to the lower spine, including the combination of sacrum and coccyx. The second stage is dedicated to the identification of metastases through the application of segmentation masks, identifying two types of metastases in the spinal region: lytic and sclerotic. Our methodology employs two U-Net based neural networks: one network is trained to locate the spine, while the second network is trained for the instance segmentation of metastases, including type prediction. This structured approach ensures precise localization and identification of spinal metastases, facilitating targeted clinical interventions.

Before training, the data were converted into a "nearly raw raster data" format to optimize input/output operations during the training process. The first dataset for vertebra segmentation comprises 115 patients diagnosed with polytrauma but with relatively undamaged spines, all of whom underwent full-body CT scanning at the local hospital RAKUS (Rīgas Austrumu klīniskā universitātes slimnīca).

The second dataset, used for metastasis detection, consists of patients diagnosed with spinal metastases and is detailed in Table 1. [26]

We have utilized the built-in validation mechanism of the nnUnet library in the form of 5 cross-validation. In its turn, all the segmentation masks for the training data were created by medical professionals from Rigas Stradiņš University using the 3D Slicer software platform for medical image informatics, image processing, and three-dimensional visualization. [27] Following data preparation, four U-Net architecture subtypes were trained: 2D images in the form of single slices from CT scans, 3D low resolution with downsampled input image data, 3D full resolution utilizing the original resolution of the CT scans, and 3D cascade full resolution which uses downsampled images to understand the overall structure on a large scale before learning details from the full resolution image data as is stated in the utilized library. [28] The best performing subtype was selected for testing purposes.

3. Results

The process begins with identifying the Region of Interest (ROI), with the spine being the ROI for metastases in CT scans. In the initial stage, the spine is segmented to facilitate metastases analysis. The U-Net architecture, trained using cross-validation, employs a total loss function combining Dice loss and cross-entropy loss. The model is trained on patches extracted from the original image, with the Dice metric calculated based on these patches. Post-training, the sliding window method is used for inference, which may result in patches different from those used during training, potentially decreasing the Dice score. Validation patches are sampled similarly to training patches, and the Dice coefficient is computed over all sampled patches collectively. Figure 1 and Figure 2 depict the pseudo-Dice metric, which is used to monitor training progress and detect overfitting. This metric is updated continuously and is distinct from the actual Dice coefficient.

The actual Dice similarity coefficient, computed at the end of the training, is not calculated on patches but on the whole images using a sliding window approach. This provides the actual performance metrics, whereas the pseudo-Dice metrics are used solely to sanity-check the training process. All metrics are calculated based on validation, utilizing the network architecture that demonstrated the best performance during the training process. For vertebra segmentation, these metrics are depicted in Table 2, while for metastasis instance segmentation, they are shown in Table 3.

4. Discussion

Mathematical metric properties are often neglected, for example, when using the Dice similarity coefficient (DSC) in the presence of particularly small structures, making it a poor metric for metastasis segmentation. Recent research suggests that more preferable metrics for evaluating model performance in instance segmentation tasks are the F-beta score and panoptic quality. [29] However, the panoptic quality metric (presented in Table 3) has been severely criticized for use in cases rich in small segmentations with variable shapes and where the background is treated as a separate class, as is the case in metastasis segmentation. [30] Consequently, the F-beta score has been adjusted to suit the needs of oncological diagnostics, where an F-beta score with greater emphasis on recall is preferable, such as when minimizing false negatives is critical but false positives remain significant. This adjustment resulted in a beta value of 2.

The highest overall performance for the vertebra segmentation model was achieved using the 3D full-resolution architecture. The predicted mask utilizing this model is demonstrated in Figure 3.

The highest overall performance for the metastasis segmentation model was achieved using the 3D full-resolution architecture. The predicted mask utilizing this model is demonstrated in Figure 4. It worth to mention that the model was able to correctly detect and segment metastasis not only in spine, but also in others bone structures, which can be seen in aforementioned figure (sternum).

Recent studies on spinal metastasis segmentation using deep learning have shown notable advancements, particularly in MRI applications. Our research aligns with these trends, offering valuable metrics that demonstrate the model's performance across lytic and sclerotic metastases.

For lytic metastases, our model achieved a Dice Similarity Coefficient (DSC) of 0.71, an F-beta score of 0.68, and a Panoptic Quality of 0.45. These results are comparable to Kim et al. (2024), who reported a mean per-lesion sensitivity of 0.746 and a positive predictive value of 0.701 using a U-Net-based model. [31] Similarly, Liu et al. (2021) achieved a precision of 0.76 and a recall of 0.67 for pelvic bone metastasis segmentation. [10] Our F-beta score and DSC values indicate a strong ability to detect and segment lytic metastases, consistent with these findings.

However, the performance for sclerotic metastases was lower, with a DSC of 0.61, an F-beta score of 0.57, and a Panoptic Quality of 0.30. Ong et al. (2022) reported a DSC of up to 0.78 for spinal metastasis segmentation with sensitivity rates of 78.9%, suggesting that sclerotic metastases pose greater challenges due to their subtle imaging characteristics. [32] Our results reflect this difficulty, with lower scores across all metrics for sclerotic lesions.

It is worth highlighting the VerSe: Large Scale Vertebrae Segmentation Challenge, where state-of-the-art performance achieved a mean vertebrae identification rate of 96.6% and a Dice coefficient of 91.7%. The challenge made a significant contribution to the field by providing a dataset of 374 multi-detector CT scans, which has been instrumental in advancing vertebrae segmentation research. [33] In comparison, our dataset is considerably smaller, and our training process was shorter (250 epochs). Despite these limitations in dataset size and training duration, our model demonstrated competitive performance, particularly for lytic metastasis segmentation, though there remains room for improvement in handling more complex lesion types like sclerotic metastases. This comparison underscores the potential of our approach even with more constrained computational and data resources.

Despite the promising results of our study, several limitations need to be acknowledged: The dataset used for training the model primarily consisted of patients from a single medical centre, which may limit the generalizability of the results. A more diverse and larger dataset would be beneficial to improve the robustness of the model across different populations and imaging conditions. Additionally, the quality of the segmentation masks created by medical professionals could introduce variability, as inter-observer variability in manual segmentations can affect the training quality and performance evaluation of the model. Furthermore, while the model shows relatively high accuracy in detecting spinal metastases in the provided dataset, it has not yet been validated in a real-world clinical setting. Further clinical trials and validations are required to assess its practical utility and effectiveness in improving patient outcomes.

Another critical aspect to consider is the technical limitations related to the scanning protocols, which can vary significantly between cases. For optimal performance, inference should be conducted on radiological data that closely match the technical properties of the data used to train the model. The resolution of the radiological image data is particularly crucial, as variations in resolution can significantly impact the model's accuracy and effectiveness. The relatively limited dataset for metastasis detection negatively impacted the overall performance of the model, highlighting an area for enhancement. Expanding the dataset to include a broader range of cases with diverse patient demographics and imaging conditions would likely improve the model's robustness and generalizability. A larger dataset would also allow for better training and validation of the model, enabling it to accurately detect and segment metastases across various clinical scenarios. This enhancement is crucial for increasing the model's diagnostic accuracy and effectiveness in real-world applications​. [34,35]

5. Conclusions

In this study, we have developed and trained two models for the detection and instance segmentation of vertebrae and spinal metastases using advanced AI techniques. The results demonstrate that our models are capable of accurately identifying and segmenting these structures, providing a valuable tool for clinical applications. Specifically, the F-beta score for vertebra segmentation ranged from 0.88 to 0.96 across all vertebra classes, indicating high performance in this task. For spinal metastases, the model achieved an F-beta score of 0.68 for lytic type and 0.57 for the sclerotic type, showcasing the potential of our approach in identifying different types of metastases.

Furthermore, our model exhibited the ability to detect isolated metastatic nodes in other bones, extending its utility beyond the primary objective of spinal metastasis detection. This capability underscores the model's robustness and versatility in different clinical scenarios, providing comprehensive diagnostic support.