Is Personalized Mechanical Thrombectomy Based on Clot Characteristics Feasible? A Radiomics Model Using NCECT to Predict FPE in AIS Patients Undergoing Thromboaspiration

1. Introduction

Recanalization of the occluded vessel in acute ischemic stroke (AIS) patients after a mechanical thrombectomy (MT) y is measured with the TICI (Thrombolysis In Cerebral Infarction) scale [1]. Classically, successful repermeabilization was defined as a thrombectomy resulting in a TICI ≥2B [2]. However, it has been shown that patients who achieve a recanalization TICI 2C or 3 have better clinical outcomes 3 months after thrombectomy than patients who achieve TICI 2B [3]. In patients undergoing mechanical thrombectomy, the first pass effect (FPE) is defined as the complete recanalization (TICI≥2C) of an occluded vessel following a single pass thrombectomy. It has been demonstrated that FPE is associated with improved clinical outcomes and lower mortality rates in patients with AIS [4,5]. Initially, the FPE was described for stent retriever devices. Subsequent studies evaluated the rate of FPE in patients undergoing stent retriever therapy compared to those undergoing thromboaspiration. The results were found to be similar in both groups in randomized trials [6,7]. Furthermore, an association has been observed between the molecular composition of the thrombus and the probability of achieving an FPE with MT [8,9]. Moreover, research has been conducted that demonstrates a correlation between the molecular composition of thrombi and their radiological appearance, as well as the correlation between the radiological appearance and the probability of FPE [10,11,12,13].

In the domain of diagnostic imaging, radiomics has witnessed a surge in interest in recent years. This emerging field involves the use of quantitative analysis techniques to process radiological image data which escapes human visual detection [14]. In its early days, radiomics focused primarily on oncological pathology, but more recently there have been more publications on radiomic modelling in non-oncological pathology, including in patients with AIS [15]. It has been revealed that radiomic data may vary depending on the molecular composition of the thrombi causing AIS [16]. However, there is a paucity of studies on the prediction of successful repermeabilization in patients undergoing MT using radiomic data from radiological images [17,18]. No studies have been identified in which the objective is to predict FPE with thromboaspiration based on NCECT-based radiomics.

We propose a retrospective study to identify which patients are most likely to achieve an FPE with thromboaspiration, using a radiomic model with features obtained from NCECT. Given the relatively rapid nature of thromboaspiration and its cost-effectiveness compared with stent retrievers in cases where an FPE is achieved [19], we believe that such information may be very important in prethrombectomy planning in patients with AIS.

2. Materials and Methods

2.1. Study Design and Ethical Aproval

This retrospective observational study was conducted at University Hospital of Santiago de Compostela, a public and tertiary-level hospital that serves as a referral center for the treatment of AIS patients, covering a population of 600,000. The study was performed in accordance with the principles of the Declaration of Helsinki of the World Medical Association and approved by the Santiago-Lugo Ethics Committee (code 2023/299).

2.2. Patient Selection

A total of 334 mechanical thrombectomies were performed at this hospital between the years 2021 and 2023. All these patients were admitted to the stroke unit of the same hospital and treated according to national society protocols by trained neurologists and neuroradiologists experienced in cerebrovascular disease. Patients presenting to the emergency department with suspected AIS underwent brain NCECT, CT perfusion and CT angiography of the supra-aortic arteries. For recanalization treatment, interventional neuroradiologists employed mechanical thrombectomy by thromboaspiration as a primary technique. The stent-retriever and combined (aspiration and stent retriever) techniques were employed as second-intention techniques. The procedures were conducted by neuroradiologists with expertise in both diagnostic and interventional neuroradiology.

Data were collected retrospectively from patients who underwent mechanical thrombectomy at University Hospital of Santiago de Compostela between 1 January 2020 and 31 May 2023. The inclusion criteria were: (1) brain NCECT with a slice thickness of 0.625 mm; (2) a visible thrombus on NCECT located in the distal internal carotid artery (ICA) or in the middle cerebral artery (MCA) (M1 or proximal M2 segments); and (3) treatment with at least the first thrombectomy attempt performed with thromboaspiration. The exclusion criteria were: (1) thrombus in another intracranial artery or patients with tandem occlusion; and (2) patients for whom imaging studies were conducted in an external hospital. Intravenous tissue plasminogen activator (tPA) was administered to patients following brain NCECT, in accordance with the established clinical guidelines.

2.3. Image Adquisition

All patients enrolled in the study underwent a NCECT at our public hospital using two different CT scanners (16 rows of detectors, 120 kV) of the same make and model (Phillips Ingenuity; Amsterdam, Netherlands) during the diagnosis process of AIS. Patients were randomly assigned to each scanner. The images obtained had a slice thickness of 0.625 mm. Although reconstructions with a thickness of 1 mm were available, they were not used for analysis. The window width and center were set at 80 and 40 Hounsfield units (HU), respectively (Figure 1).

2.4. Segmentation and Feature Extraction

The Digital Imaging and Communications in Medicine (DICOM) files of the brain NCECT were imported into 3D-Slicer version 5.2.2 (free software available at www.slicer.org) [21]. The region of interest (ROI) was defined as the visible thrombus in the ICA and/or proximal MCA (M1 and M2 segments). A 3D-semi-automatic segmentation was performed using the segment editor of 3D-Slicer [22]. The selected segmentation tool is "Level Tracing", whereby the outline is defined by the movement of the mouse, resulting in pixels that share the same background density value as the mouse's current background pixel. The subsequent application of the outline to the region of interest is a decision that is made by the radiologist. Segmentation was performed by a radiology resident who had received specialized training. The segmentation was validated by three neuroradiologists with expertise in neurointervention and neuroimaging. (Figure 1). The RFs were also extracted in 3D-Slicer using Radiomics module [23]. For all ROIs, image voxel resampling to a dimension of 1 mm × 1 mm × 1 mm was performed. Smoothing with a Gaussian filter and a fixed grey bin width value of 25 has also been performed to normalize the images. Wavelet-based features and a kernel size of 3, 5 and 7 were also performed.

Figure 2. Same patient as in Figure 1. Example of semi-automated segmentation using 3D Slicer software: (a) Sagittal view segmented; (b) 3D reconstruction of thrombus segmentation. The selected segmentation tool is "Level Tracing”.

Figure 2. Same patient as in Figure 1. Example of semi-automated segmentation using 3D Slicer software: (a) Sagittal view segmented; (b) 3D reconstruction of thrombus segmentation. The selected segmentation tool is "Level Tracing”.

The RFs extracted were: (1) first-order features (evaluate pixel/voxel distribution); texture features such: (2) gray-level cooccurrence matrix (GLCM), (3) gray-level dependence matrix (GLDM), (4) gray-level run length matrix (GLRLM), (5) gray-level size zone matrix (GLSZM), (6) neighboring gray-tone difference matrix (NGTDM); and (7) shape features (geometric evaluation of segmented area). A total of 106,197 radiomic features were extracted from the 91 patients included in the study, with 1,167 features per patient (Figure 3).

Figure 3. Radiomics workflow: RFs are obtained from visible thrombus in NCECT. Segmentation is semi-automatic. A total of 1,167 radiomic features were obtained for each patient, with the most significant ones selected using LASSO regression. Finally, a logistic regression identified six statistically significant RFs for the prediction model. Accuracy, sensitivity, specificity, the ROC curve and the Cohen's kappa index were then calculated.

Figure 3. Radiomics workflow: RFs are obtained from visible thrombus in NCECT. Segmentation is semi-automatic. A total of 1,167 radiomic features were obtained for each patient, with the most significant ones selected using LASSO regression. Finally, a logistic regression identified six statistically significant RFs for the prediction model. Accuracy, sensitivity, specificity, the ROC curve and the Cohen's kappa index were then calculated.

The segmentation, extraction of RFs, and analysis of the results were performed using a system with an Intel CORE i7 processor (Santa Clara, California, USA), 16 GB RAM, 1 TB hard disk, and Microsoft Windows 11 operating system (Redmond, Washington, USA).

2.6. Clinical Data

The study also explored the relationship between various clinical variables and the achievement of FPE. The clinical variables analyzed included cardiovascular risk factors such as arterial hypertension, diabetes mellitus, dyslipidemia, and smoking history, along with sex, age, and thrombus laterality. In addition, the HU of each thrombus were collected and subjected to analysis.

2.7. Feature Selection and Statistical Analysis

The feature selection and the construction of the predictive model were carried out using Orange: Data Mining Toolbox in Phyton (Ljubljana, Slovenia) and Statistical Package for the Social Sciences (SPSS) version 22.0 (SPSS Inc., Chicago, IL, USA) [24,25]. For the purpose of feature normalization, a Z-score normalization (with a range between 0-1) was applied using the tool 'Continuize', which is available at Orange. A LASSO (Least Absolute Shrinkage and Selection Operator) regression was used to select the significant RFs, with a nonzero coefficient. The RFs selected and the clinical variables were analyzed using with a Logistic Regression, with Backward LR as selection method. A confusion matrix, odds ratios and 95% confidence intervals were obtained. With regard to the confusion matrix, the true positive (TP) is defined as the number of FPE thrombectomies correctly identified as such. The false positive (FP) is defined as the number of non-FPE thrombectomies incorrectly identified as FPE thrombectomies. The true negative (TN) is defined as the number of non-FPE thrombectomies correctly identified as such. Finally, the false negative (FN) is defined as the number of FPE thrombectomies incorrectly identified as non-FPE thrombectomies (Table 1). The accuracy, sensitivity (Se) and specificity (Sp) of the model were calculated, where Se is the model's ability to correctly classify patients with FPE thrombectomy and Sp is the model's ability to correctly classify patients with non-FPE thrombectomy.

Table 1. Representation of a confusion matrix used to visualize the performance of the prediction of FPE. Columns represent the predicted class. The rows represent the true class.

		Predicted
		FPE	Non-FPE
Observed	FPE	TP	FP
	Non-FPE	FN	TN

Table 1. Representation of a confusion matrix used to visualize the performance of the prediction of FPE. Columns represent the predicted class. The rows represent the true class.

		Predicted
		FPE	Non-FPE
Observed	FPE	TP	FP
	Non-FPE	FN	TN

The performance of the model was evaluated using the receiver operating characteristic (ROC) curve. The Hosmer-Lemeshow goodness-of-fit test was also calculated. The Cohen's Kappa coefficient (%) has also been calculated, which is defined for classification problems with two categories, in our case FPE and non-FPE thrombectomies, as

$$K=100(Pa-Pe)/(1-Pe)$$

were $Pa=\left(TP+TN\right)/N$ and $\mathrm{P}\mathrm{e}=\left(TP+FN\right)\left(TP+FP\right)/N^2+\left(FP+TN\right)\left(FN+TN\right)/N^2$. The Kappa index is a measure that adjusts for the effect of chance and is considered a more robust index than accuracy or AUC [26].

Finally, the The Radiomic Quality Score (RQS) was calculated to measure the methodology comparing with other radiomics studies. RQS consists of 36 checkpoints that assess radiomic studies to promote best scientific practice [20]. The score in our study was 17 (47.22%) (Appendix A.1). The present study was also elaborated according with the CLEAR guidelines (CheckList for EvaluAtion of Radiomics research) (Supplemental material: Table S1) [27].

3. Results

3.1. Patient Information

A total of 279 patients with AIS underwent mechanical thrombectomy between January 2021 and May 2023. In 119 patients the occlusion was in an artery other than the ICA/MCA, or a tandem occlusion. In 37 patients, the initial attempt at mechanical thrombectomy was conducted using the stent retriever or combined technique. In 32 cases, the precise location of the thrombus could not be determined using NCECT. Finally, 91 patients were included in the study (Figure 4).

Figure 4. Decision tree for inclusion of patients in the study. From a total of 279 patients, 91 patients were selected after applying the inclusion and exclusion criteria.

Figure 4. Decision tree for inclusion of patients in the study. From a total of 279 patients, 91 patients were selected after applying the inclusion and exclusion criteria.

Of the patients included, 51 achieved FPE, while 40 did not. The cohort consisted of 31 male and 60 female patients. In the FPE group, 36 (70.6%) were female patients, 41 patients (80.4%) had arterial hypertension, 16 (31.4%) had diabetes, 34 (66.7%) had dyslipidaemias, five (9.8%) had a history of smoking, and 27 clots (53%) were located on the left side. In the non-FPE group, 24 (60%) were female patients, 30 patients (75%) had arterial hypertension, 9 (22.5%) had diabetes, 22 (55%) had dyslipidaemia, six (15%) had a history of smoking, and 22 clots (55%) were located on the left side. The mean age was 77.74 years (standard deviation [SD] 12.13) in the FPE group and 77.70 years (SD 8.66) in the non-FPE group. The mean HU for the FPE group was 58.77 (SD 10.17), compared to 63.38 (SD 26.01) for the non-FPE group (Table 2).

Table 2. Baseline characteristics of patients included in the study.

Patient data	FPE	Non-FPE	p-value
Sex	Female: 36 (70.6%1)	Female: 24 (60%)	0.290
Age (mean)	77.74 (SD1 12.13)	77.70 (SD 8.66)	0.566
Hounsfield Units (mean)	58.77 (SD 10.17)	63.38 (SD 26.01)	0.246
Arterial Hypertension	41 (80.4%)	30 (75%)	0.538
Diabetes	16 (31.4%)	9 (22.5%%)	0.347
Dyslipidemia	34 (66.7%)	22 (55%)	0.347
Smoke	5 (9.8%)	6 (15%)	0.370
Laterality	Left: 27 (53%)	Left: 22 (55%)	0.845

¹ Percentage represents proportion of patients with condition in group (FPE or non-FPE). ² SD: Standard Deviation.

Table 2. Baseline characteristics of patients included in the study.

Patient data	FPE	Non-FPE	p-value
Sex	Female: 36 (70.6%1)	Female: 24 (60%)	0.290
Age (mean)	77.74 (SD1 12.13)	77.70 (SD 8.66)	0.566
Hounsfield Units (mean)	58.77 (SD 10.17)	63.38 (SD 26.01)	0.246
Arterial Hypertension	41 (80.4%)	30 (75%)	0.538
Diabetes	16 (31.4%)	9 (22.5%%)	0.347
Dyslipidemia	34 (66.7%)	22 (55%)	0.347
Smoke	5 (9.8%)	6 (15%)	0.370
Laterality	Left: 27 (53%)	Left: 22 (55%)	0.845

¹ Percentage represents proportion of patients with condition in group (FPE or non-FPE). ² SD: Standard Deviation.

3.2. Features Analysis

A total of 1167 RFs were obtained from each patient. The Least Absolute Shrinkage and Selection Operator (LASSO) method selected 9 RFs with nonzero coefficients. The RFs selected were: 1 shape feature (Maximum 2D diameter for rows), 6 texture features (GLRLM-Gray level non-uniformity, GLRLM-Gray level non-uniformity normalized, GLRLM-Run entropy, GLSZM-Zone entropy, GLSZM-Size zone non-uniformity normalized, GLDM-large dependence low gray level emphasis) and 2 first order features (90percentile, kurtosis). However, of the non-radiomics variables collected (gender, age, arterial hypertension, diabetes, dyslipidemia, smoking, laterality of clot and HU of clot) included in the logistic regression analysis, none were found to be statistically significantly associated with FPE (p-value > 0.05) (Table 2). The Backward LR selection method of the Logistic Regression selected 6 RFs that were statistically associated with FPE (p-value < 0.05) for the final model prediction (Table 3).

Table 3. The LASSO regression identified 9 RFs with non-zero coefficients. Logistic regression showed that 6 of these 9 RFs were statistically associated with the likelihood of achieving FPE with thromboaspiration.

RFs	Type of RF	OR	p-value
90 Percentile	Firstorder	0.809	0.045
Kurtosis	Firstorder	0.536	0.020
Maximum 2D Diameter Row	Shape	0.646	0.004
Size Zone Non-Uniformity Normalized	GLSZM	1.848 E-4	0.010
Zone Entropy	GLSZM	0.205	0.006
Gray Level Non-Uniformity Normalized	GLRLM	0.06	0.005
Run Entropy	GLRLM	0.268	0.605
Gray Level Non-Uniformity	GLRLM	1.383	0.240
Large Dependence Low Gray Level Emphasis	GLDM	1.003	0.317

Table 3. The LASSO regression identified 9 RFs with non-zero coefficients. Logistic regression showed that 6 of these 9 RFs were statistically associated with the likelihood of achieving FPE with thromboaspiration.

RFs	Type of RF	OR	p-value
90 Percentile	Firstorder	0.809	0.045
Kurtosis	Firstorder	0.536	0.020
Maximum 2D Diameter Row	Shape	0.646	0.004
Size Zone Non-Uniformity Normalized	GLSZM	1.848 E-4	0.010
Zone Entropy	GLSZM	0.205	0.006
Gray Level Non-Uniformity Normalized	GLRLM	0.06	0.005
Run Entropy	GLRLM	0.268	0.605
Gray Level Non-Uniformity	GLRLM	1.383	0.240
Large Dependence Low Gray Level Emphasis	GLDM	1.003	0.317

3.3. Prediction of FPE

The 9 RF selected via LASSO regression, in conjunction with the clinical variables collected, were utilised to execute the logistic regression and thereby categorise the patients into FPE and non-FPE groups. The backward selection model of the Logistic Regresion eliminates the clinical variables and 3 RFs (GLRM-Run Entropy, GLRLM-Gray Level Non-Uniformity and GLDM-Large Dependence Low Gray Level Emphasis) (p-value>0.05). The final prediction model is exclusively predicated on the following radiomic features: 90 Percentile, Kurtosis, Maximum 2D Diameter of Rows, GLSZM-Size Zone Non-Uniformity Normalized, GLSZM-Zone Entropy, GLRLM-Gray Level Non-Uniformity Normalized (p-value<0.005). With these 6 selected RFs, the confusion matrix and ROC curve were obtained, giving an accuracy of 0.813, a Se of 0.815, a Sp of 0.811 and an AUC of 0.890 (p<0.05) (Table 4) (Figure 5).

Table 4. Confusion matrix obtained from logistic regression analysis using the 6 RFs associated with the likelihood of achieving FPE with thromboaspiration.

		Predicted		∑
		FPE	Non-FPE
Observed	FPE	44	7	51
	Non-FPE	10	30	40
∑		54	37	91

Table 4. Confusion matrix obtained from logistic regression analysis using the 6 RFs associated with the likelihood of achieving FPE with thromboaspiration.

		Predicted		∑
		FPE	Non-FPE
Observed	FPE	44	7	51
	Non-FPE	10	30	40
∑		54	37	91

Figure 5. The ROC curve was obtained with SPSS using the 6 RFs selected in the logistic regression.

Figure 5. The ROC curve was obtained with SPSS using the 6 RFs selected in the logistic regression.

The Kappa coefficient result was 74.90%, indicating a substantial agreement between the prediction model and the actual observations (Table 5) [28]. For the Hosmer-Lemeshow goodness-of-fit test, the Chi-square test result was 12.002 with a p-value of 0.151. This means that there is no reason to believe that the predicted results are different from the observed results and the model can be considered acceptable (Table 6).

4. Discussion

The objective of this study was to predict the FPE in patients with MCA/ICA stroke who were undergoing MT by thromboaspiration. This was achieved by utilising RFs obtained from semi-automated segmentation of the hyperdense thrombus on the NCECT. The results demonstrate that radiomic data can help to predict FPE in these patients. Six RFs were independent predictors for FPE prediction (p-value < 0.05). The accuracy of FPE prediction is 0.813 when these six RFs are employed. The clinical data obtained and the clot density were not demonstrated to be independent predictors for FPE (p-value>0.05). The findings of the study suggest that thrombi visible on NCECT contain information associated with the efficiency of thrombus removal by thromboaspiration.

Forecasting the likelihood of achieving FPE through thromboaspiration could be crucial in selecting the most cost-effective method for arterial reperfusion. The effectiveness of different thrombectomy techniques remains a topic of debate. Both thromboaspiration and stent retriever show similar outcomes in large vessel occlusions [29]. Many hospitals use thromboaspiration as the primary technique, reserving stent retrievers for second-line use when thromboaspiration fails or as a first-line option for distal occlusions in smaller vessels. Few studies have directly compared the cost-effectiveness of these techniques. Although a procedure that begins with thromboaspiration and subsequently requires stent retrieval is generally less cost-effective than starting with a stent retriever, higher cost-effectiveness has been observed when FPE is achieved with a single pass of thromboaspiration compared to using a stent retriever [30].

The relationship between radiomic features and successful permeabilization in AIS patients has been previously investigated. Hofmeister et al. developed a radiomic model to predict TICI>2b with a single pass of thromboaspiration in a cohort of 47 patients [17]. The model demonstrated an accuracy of 0.851, a sensitivity of 0.50, a specificity of 0.971, a positive predictive value of 0.857, and a negative predictive value of 0.850. A Support Vector Machine (SVM) classifier was employed, and the model was based on 1,485 radiomic features, with nine selected for the final prediction model. Additionally, the study identified a correlation between radiomic features and the number of thrombectomy attempts. Despite the smaller size of their cohort, their findings suggest a correlation between radiomic data and the success of mechanical thrombectomy. In contrast to our study, which aims to predict FPE (TICI ≥2c), they defined successful reperfusion as TICI ≥2b. Hofmeister et al. employed semi-automated segmentation and SVM automatic classifier, whereas our study achieved optimal results with the logistic regression analysis. The study previously mentioned reported a radiomic quality score of 17 out of 36 (47.22%), which is equivalent to the present study's score. Sarioglu et al. conducted another similar study. They developed a prediction model for FPE in 52 patients treated primarily with stent retrievers. Their model achieved an AUC of 0.83 in predicting FPE and demonstrated that clot-based radiomics can aid in estimating the success of mechanical thrombectomy in AIS patients. They performed manual segmentation of thrombi on non-contrast CT and validated the thrombus location using CT angiography, extracting 88 radiomic features. They identified two RFs as independent predictors of FPE and found that incorporating these features into a model based on ASPECTS and patient sex improved prediction accuracy. The findings of their study also emphasize the significance of RFs in the planning of endovascular procedures. Nevertheless, they attempted to pre-predict FPE with a stent-retriever, their sample size was smaller, and the number of features they employed was also smaller than that of the present article. Furthermore, they utilized manual segmentation, which resulted in limitations in terms of reproducibility of ROI segmentation [18].

Our study has several limitations. First, only patients with visible thrombi on NCECT were included in this study. Although the relationship between thrombus density and the likelihood of achieving FPE has been investigated, no statistically significant association was found (31). Similarly, analysis of thrombus density in the present article revealed no significant correlation between mean HU and FPE. For practical integration into clinical settings, it's essential that the impact of these tools on image analysis time be minimized. Segmentation of visible thrombi in NCECT is quick and straightforward, whereas including non-visible thrombi on NCECT can significantly delay analysis when localization is based on angio-CT. Future research should explore the inclusion of non-visible thrombi at NCECT by integrating additional imaging techniques for the segmentation. Second, it is a retrospective single centre study. However, this study includes patients whose images were obtained using two different CT scanners of the same make and model. In this context, we suggest that multicenter and prospective studies be conducted to broaden the current body of literature. Third, the lack of standardization in the procedures for obtaining and analyzing radiomic data remains a limitation in applying findings to other research groups. Therefore, it is crucial to provide a detailed description of the methodology used and adhere to established standards for radiomic studies, such as the Radiomics Quality Score (RQS) and CLEAR checklist.

Currently, the decision to perform mechanical thrombectomy using either aspiration or a stent retriever is guided by the neuroradiologist's personal preference, the patient's vascular anatomy, and the location of the thrombus, but does not usually consider the thrombus composition. Radiomics provides additional insights into the radiological appearance of the thrombus, which is closely related to its composition (32,33). This study has shown that radiomic data can provide valuable information for predicting FPE with thromboaspiration. By integrating this information, neuroradiologists could better predict the likelihood of success of mechanical thrombectomy and select the most appropriate technique for each case, if targeted studies support these findings. This approach would transform mechanical thrombectomy into a more personalized technique.

5. Conclusions

Our study demonstrates that radiomic features can help predict the achievement of FPE with thromboaspiration, potentially improving procedural planning and allowing a more tailored approach to mechanical thrombectomy.

Appendix A.1: Radiomics Quality Score (RQS).

Table A1. Please find below the RQS questionnaire and total score. For full details of the questionnaire, please refer to https://www.radiomics.world/rqs.

Questions	Answers
Image protocol quality	Protocols well documented and public
Multiple segmentations	Yes
Phantom study	No
Imaging at multiple time points	No
Feature reduction	Either measure is implemented
Multivariable analysis with non-RFs	Yes
Detect and discuss biological correlates	Yes
Cut-off analyses	No
Discrimination statistics	Discrimination statistic and its significance
Calibration statistics	Calibration statistic and its significance
Prospective study	No
Validation	No validation
Comparison to “gold standard”	Yes
Potential clinical utility	Yes
Cost-effectiveness analysis	No
Open science and data	The code is open sourced
Total Score	17 (47.22%)

Table A1. Please find below the RQS questionnaire and total score. For full details of the questionnaire, please refer to https://www.radiomics.world/rqs.

Questions	Answers
Image protocol quality	Protocols well documented and public
Multiple segmentations	Yes
Phantom study	No
Imaging at multiple time points	No
Feature reduction	Either measure is implemented
Multivariable analysis with non-RFs	Yes
Detect and discuss biological correlates	Yes
Cut-off analyses	No
Discrimination statistics	Discrimination statistic and its significance
Calibration statistics	Calibration statistic and its significance
Prospective study	No
Validation	No validation
Comparison to “gold standard”	Yes
Potential clinical utility	Yes
Cost-effectiveness analysis	No
Open science and data	The code is open sourced
Total Score	17 (47.22%)