Impact of Preprocessing Parameters in Multimodality Imaging-Based Radiomic Studies: A Review

1. Introduction

In recent years, there has been a notable increase in research examining the potential of quantitative imaging features to reveal additional information from medical images beyond what is perceptible to the human eye [1,2,3]. Handcrafted radiomics systematically extracts quantitative imaging features from medical images to interpret biological information [4,5]. The term "radiomics" yields over 11,100 studies on PubMed. This approach takes manually delineated regions of interest, such as tumors, on medical images and extracts a high number of quantitative features with pre-determined mathematical formulas [6]. One of the biggest advantages of radiomics is that those values can be extracted from medical images which can be normally acquired from the clinical pathway of the patient. In radiomics studies, the hypothesis is that radiomic features (RFs) can act as clinical biomarkers, individually or in combination, even with clinical parameters [4].

Numerous investigations have explored and documented RFs' ability to predict clinical outcomes, including both a prognostic aim (overall survival, treatment response) as well as diagnostic aim (e.g., differential diagnosis) [7,8,9,10,11,12,13,14,15,16]. These studies highlight the potential clinical application of radiomics, offering non-invasive, reliable, accessible, and cost-effective alternatives to invasive procedures like tissue biopsies. Moreover, with proper implementation, radiomics could provide useful predictions to help the clinician and reduce their workload [17,18].

As the field of radiomics advanced, there has been increasing attention to its limitations [5,19]. The primary identified limitation is the sensitivity of RFs to variations in image acquisition and reconstruction parameters [20,21,22,23,24,25]. To be effective clinical biomarkers, RFs must demonstrate reproducibility across different imaging parameters for generalization [25]. However, numerous studies have documented differences in imaging acquisition and reconstruction parameters [25,26,27,28,29,30,31,32,33,34,35,36,37]. Moreover, the impact of variation in a single acquisition or reconstruction parameter on RF reproducibility depends on the individual feature [31,38]. Two works also highlight that reconstruction kernels notably affect RF reproducibility [21,37]. There are several works exploring various methods to address RF reproducibility across differently acquired scans, as well as how to standardize them.

Image preprocessing occurs between the image segmentation and feature extraction stages. Preprocessing steps involve pixel spacing, grey-level intensities, bins of the grey-level histogram, and similar parameters, ensuring a lighter computational burden when extracting radiomic features. Preliminary findings indicate that the repeatability of extracted radiomic features is significantly influenced by the image preprocessing settings employed [39,40,41,42,43,44]. Usually, the acquisition and reconstructed voxels are not isotropic, especially if no voxel re-segmentation is applied (e.g., CT scans are usually not isotropic at acquisition). However, it is possible to achieve isotropic ones using an interpolation typically available in the software used for radiomic feature calculation.

Normalization or range re-segmentation is used to remove voxels in the segmented volume of interest (VOI) that fall outside a specific grey-level range. The normalization technique is commonly used to ensure data consistency, especially for imaging modalities with calibrated units. In fact, normalization is typically necessary for CT and PET to exclude voxels representing air or bone within a tumor VOI. However, this method does not apply to MRI data, which employs arbitrary intensity units instead. In this case, intensity outlier filtering is applied by calculating the mean (µ) and the standard deviation (σ) of grey levels within the VOI and excluding grey levels outside the range of µ ± n σ, where n is usually chosen to 3 [44,45,46,47]. Furthermore, other relative normalizations have been proposed to determine MRI radiomic feature robustness [48,49]. Discretization involves grouping the original gray-level values into specific range intervals (bins), similar to creating a histogram. This step makes the calculation of features manageable [44].

This work focused on gathering information about voxel resampling, normalization, and discretization preprocessing parameters from our included studies. We also investigated the study aim, anatomic district, and comparison metrics to better understand the up-to-date overview of the radiomic studies. This review aims to provide an up-to-date and comprehensive overview of the most commonly used preprocessing techniques.

2. Materials and Methods

This systematic review followed the recommendations of the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) 2020 [50]. The inclusion criteria for the records in the review process were the presence of specific preprocessing parameters evaluated in CBCT, CT, MRI, and PET/CT radiomic studies.

We performed a comprehensive literature search using four electronic databases: PubMed, Cochrane Library, Embase, and Scopus. Since radiomic studies are relatively recent, we conducted the literature search from January 2008 until December 2023. Mesh terms and free text were modeled in search strategies for databases using the one designed for PubMed: (((("Positron-Emission Tomography" [Mesh]) OR "Tomography, X-Ray Computed" [Mesh]) OR "Magnetic Resonance Imaging" [Mesh] OR PET OR Positron emission tomography OR computed tomography OR CT OR MRI OR magnetic resonance) AND (radiomics)) AND (preprocessing OR preprocessing). The study was exclusively confined to English-language research without publication status restrictions. The authors did not contact other institutions or researchers to uncover additional studies.

Two reviewers independently screened records by reviewing their titles and abstracts and extracting general study characteristics (such as study title, first author's name, publication year, journal, abstract, corresponding reviewer, and research keywords) for records meeting the inclusion criteria, using a customized data extraction form. In this first step, the authors independently used the ASReview tool [40] to speed up the article selection. The authors compared their decisions, resolving disagreements until a consensus was reached. Records lacking information on CT parameters were excluded, and any eligible studies with full-text versions were included in the review process, while conference proceedings and monographs were disregarded. Additional articles not meeting the inclusion criteria upon full-text reading were identified and excluded at this stage. Reviewers independently selected full-text articles, resolving any uncertainties by reaching a consensus.

From each selected full-text article, the following parameters were identified and extracted:

• acquisition modality (CBCT, CT, MRI, PET/CT);
• number of patients or phantom;
• disease/s name (if appropriate);
• equipment vendor and model;
• presence of acquisition parameters;
• the total number of features;
• type of features subsampled in FO (first order), SM (shape metric), and TA textural features;
• type of software used in the radiomic feature extraction;
• image filtering used (Y/N, if Y the type was reported);
• voxel resampling;
• normalization process;
• discretization technique;
• retrospective study (Y/N or NA);
• statistical analysis: intraclass correlation coefficient (ICC), concordance correlation coefficient (CCC), area under the receiver operation curve (AUC), Mean, Average percentage difference, Relative difference, Spearman correlation, Kolmogorov Smirnov, double sample test, and two-way ANOVA;
• type of study (reproducibility/repeatability/both or best performance);
• main findings.

No contact with the authors of the records for complementary information was necessary.

Quality scoring for study selection was not utilized. The reviewers reviewed articles in duplicate, resolving disagreements by achieving an agreement. Data from the studies were standardized to mitigate potential biases. Data extracted from studies that did not meet the criteria for standardization were excluded.

This review aimed to describe the most used preprocessing pipeline in the radiomic workflow of the CBCT, CT, MRI, and PET/CT studies. It also sought to find possible standard radiomic preprocessing setups and describe a potential evolution over time, aiming to uncover any increased awareness of the technique's standardization.

Regarding voxel resampling, we counted the number of works that used: single isotropic, multiple isotropic, and non-isotropic voxel interpolation. When the original voxel size was used, we put "N.A.", and when the information was not indicated, "None".

Concerning the normalization strategies, we counted the number of papers that chose absolute, relative, or combination strategies. When this information was not applied, we set "None". A normalization strategy enhances the robustness and reliability of the radiomic analysis. The absolute normalization strategy involves standardizing the intensity values of imaging by transforming them to a common scale based on absolute reference points. Conversely, relative normalization concerns adjusting the intensity values of images relative to a certain reference within each individual image or dataset; this technique often includes scaling the pixel values based on the mean (µ), median, or a specific percentile of the intensity values in the image. This approach accounts for differences in overall intensity levels between images by normalizing the intensities relative to their own distribution.

In terms of discretization strategies, we identified the number of studies that utilized: a bin number (BN), a bin width (BW), or both the bin number and width (BN + BW) approach. When this information was not applied or not specified, we set "None". The discretization strategy involves adjusting values to standardize the feature extraction process across different images or imaging modalities. The BN discretization ensures that the radiomic features are comparable and consistent by transforming the intensity values into a fixed number of discrete bins. The BW method assures that each bin represents a fixed range of intensity values. The BN + BW strategy is a dual approach that guarantees the simultaneous adjustment of the number of bins and the width of each bin used to discretize image intensity values.

In addition, we investigated the study aims, the anatomical districts considered, and the metric used within the works. Concerning the study aims, we counted how many studies used best performance, repeatability, or both strategies. The anatomical districts were categorized into "abdomen", "brain", "thorax", and "pelvis" for simplicity's sake. When no anatomical part was specified, e.g., for phantom studies, we set "N.A.". Moreover, the metric indexes were grouped into ICC, CCC, AUC, or "Other" when different statistical metrics were used.

We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to investigate the risk of bias [51]. The tool evaluates four key domains: patient selection, index test, reference standard, and flow and timing. Each domain is evaluated for the risk of bias and applicability. We used this tool only for the first two domains because reference standards and flow and timing are not defined in this review's field. Regarding patient selection's risk of bias, we checked the method in doing it (lower marks were given in case of inappropriate exclusions or not consecutive/random selection), while the applicability was evaluated if the included patients matched the typical clinical scenario.

We considered the index test the metric used in the result comparison. In this context, the risk of bias consisted of consistently evaluating the calculation/use of the aforementioned metric, and the applicability was referred to the methodology of its use in practice.

3. Results

3.1. Literature Search

The computer-assisted search across PubMed, Cochrane Library, Embase, and Scopus yielded 546 records (as shown in Table 1). Following the elimination of duplicate entries, 459 unique records remained. Subsequently, a screening process based on the coherence of the title and abstract reduced the number of articles to 72 (refer to Figure 1).

Reviews and conference proceedings were excluded from further consideration. Consequently, 53 full-text articles underwent evaluation by the reviewers to determine their eligibility. Ten articles were excluded from the database for the following reasons: six lacked preprocessing results, one focused on a modality outside the scope of this review, two utilized only a digital phantom, and one was unfound.

3.2. Data Collection and Elaboration

The selected papers were grouped by considering the acquisition modality: one about CBCT, 12 about CT, 26 about MRI, and five about PET/CT.

CT studies were the most represented from 2013 to 2023. MRI works denoted an increment of publications over the years, starting from 1 in 2017 to 9 in 2023. PET/CT studies were published in 2019 (2) and 2021 (3).

In 2017, CT phantom studies were published to study the stability and reproducibility of radiomic features using phantoms made of different materials compared to those traditionally used in quality control (QC) [39,52,53].

The number of patients has increased, although not monotonically, from 256 in 2013 to 3,000 in 2023. The modality that saw the greatest increase was MRI, demonstrating the growing interest of the scientific community in this type of imaging.

3.2.1. Acquisition Parameters Presence and Voxel Resampling

Table 2 shows the studies reporting the acquisition parameters and the voxel resampling information. Only in PET and CBCT did we find that most papers used isotropic voxel resampling, while in CT and MRI, only 41.7% and 41.3% used it, respectively.

For CT and CBCT, the most studied voxel resampling interpolation was 1 x 1 x 1 mm³. Furthermore, a significant number of studies used 1 x 1 x1 mm³ in MRI, but more voxel sizes were investigated in the range of 0.9 – 4.8 mm³. This result is expected because in MRI the image characteristics are strongly influenced by the acquisition protocols.

For PET/CT, voxel resampling dimensions were in the range of 1 – 4 mm³. This imaging modality, which employs a small resampling dimension, might introduce biases due to its intrinsic resolution (around non-isotropic 3 - 5 mm³).

3.2.2. Normalization Strategies

Table 3 reports the normalization strategies adopted by the authors, considering the different modalities. In CT and PET/CT, most works did not use a normalization strategy, while in MRI, relative normalization was the most used, as expected. In CBCT, a relative strategy was used because the image grey levels are not in Hounsfield but relative units.

3.2.3. Discretization Strategies

Most studies opted for a discretization strategy: 75% in CT, 100% in CBCT, 92.3% in MRI, and 100% in PET/CT. The combination of BN + BW was the most represented in this latter modality (Table 4).

In CT, the most investigated BNs were 32, 64, and 128, while BW ranged between 5 and 50 Hounsfield units (HU). In CBCT, the only work selected used BN equal to 64, 128, and 256.

In MRI, the BN = 32 was the most used bin number, and eleven works studied it (42.3%), followed by BN = 64 and 128 (38.5% of the papers).

In PET/CT, the most studied BN were 32 and 64, while BW typically ranged from 0.01 to 0.5 SUV.

3.2.4. Study Aims

Table 5 shows the count of study aims for each imaging modality.

Most works employing CT and MRI aimed to study reproducibility and its association with repeatability. It is worth noting that, since 2021, best performance has also been well investigated. Moreover, the CBCT work studied the aim of repeatability and reproducibility. PET/CT works consisted of 3 works about repeatability published in 2019 and 2021 and two works about best performance issued in 2021.

3.2.5. Anatomic District

Table 6 reports the anatomic districts categorized in the main four body regions. The CT studies focused on the trunk's anatomical regions (chest, abdomen, and pelvis), while MRI studies were polarized between the brain and pelvis. In addition, CBCT and PET/CT investigations were centered on the pelvis or chest.

3.2.5. Comparison Metrics

Table 7 illustrates the most common metrics used in the result comparison. In absolute terms, the most used metric was ICC, which is used predominantly in MRI, with 15 out of 26 works. Some works used more than one metric, so the number of papers reported in Tab. 7 is more than the number of works included.

3.3. Risk of Bias Analysis

The risk of bias assessment revealed that about 35% of the studies had a low risk of bias across the patient selection domain, while the remainder did not specify or were not clearly specified. In addition, 60% had a low risk of bias across the index test. The patient selection and index test applicability was higher than 72% (Figure 2).

4. Discussion

The volume of published studies on predictive modeling using radiomic features has been proliferating. However, no global consensus exists on which features are consistently repeatable and reproducible. This lack of agreement could potentially hinder future discussions on clinical applicability and the feasibility of prospective multi-institutional external validation trials.

This review's main objective was to identify preprocessing strategies for radiomic features in the four primary 3D imaging modalities: CBCT, CT, MRI, and PET. It also aimed to identify common preprocessing parameter settings to evaluate the current state of the art in this field and raise awareness, thereby promoting more stable and reproducible results. In fact, the preprocessing technique has the greatest impact on feature reproducibility [57].

In 2017, CT phantom studies were published to study the stability and reproducibility of radiomic features using phantoms made of materials different from those traditionally used in QC [39,52,53]. Moreover, Palani et al. [62] published a phantom study in 2023, highlighting the scientific interest in this topic.

This review showed how the choice of 1 x 1 x 1 mm³ could be a feasible starting point for CT and MRI examinations if the original voxel size parameters are not too different from these values. On the other hand, for CT, the dimension related to slice thickness (the third dimension) is notably larger compared to the other two, which in turn depend only on the FOV and reconstruction matrix size (typically 512 x 512). Also, for this reason, Larue et al. [52] and Shafiq-ul Hassan [39] studied 1 x 1 x 3 and 1 x 1 x 2 mm³, respectively. In MRI, however, the discussion is broader. From a radiomic analysis perspective, a volumetric acquisition might be the best choice to introduce minimal bias in voxel resampling. However, this choice is not necessarily optimal for all anatomical regions/lesions, and in any case, it does not apply to retrospective studies. For PET/CT, it seems clear that an isotropic voxel of approximately 3 x 3 x 3 or 4 x 4 x 4 mm3 is a widely accepted choice and a good compromise for obtaining comparable results. The challenges in isotropic reconstruction are similar to those previously described for CT, even though they relate to different physical processes.

A limitation of this review is that it did not delve into the issue of the resampling function used (e.g., linear, bicubic, nearest neighbor interpolation, etc.) because the studies were too varied to obtain a meaningful result. Larue et al. [52] demonstrated that linear interpolation resulted in the narrowest range of feature values for nearly half of the features, and cubic interpolation did so for a smaller portion. In contrast, nearest neighbor interpolation produced the broadest range for most CT study features.

Different techniques have been considered for normalization strategies. Starting from CT, Larue et al. [52] studied the influence of BW from 5 to 50 HU spaced by 5 HU. Fave et al. [60] preprocessed the CT images with 8-bit depth resampling using a 16 HU bin width. Kolossváry et al. [59] compared two equally sized bins and equally probable bins where each bin contains a proportion of the data, finding that all GLRLM features were significantly affected by binning type; besides, BNs significantly affected the values for all GLCM and GLRLM. Gray level resampling affects only second and higher-order radiomics features, while voxel size variation can influence first-, second-, and higher-order features [39]. The optimal discretization methods for feature inter- and intra-sample reproducibility depend on the imaging modality. The choice of discretization significantly affects intensity distributions, feature values, and reproducibility [45]. Defining only bin number or width is insufficient, as these two quantities are related to the maximum number of gray levels used. The IBSI group mentions preprocessing recommendations in their work [45]. In modalities that use calibrated imaging intensity units (e.g., CT and PET/CT), when a re-segmentation range is defined, either fixed bin size or fixed bin width can be defined. If no re-segmentation range is used in calibrated imaging modalities, then fixed bin number is recommended. When non-calibrated imaging modality (e.g., MRI) is employed, they recommend using fixed bin number method without an intensity re-segmentation. The issue when using BN is that it introduces a normalization effect, directly comparing feature values across patients, whereas BW preserves the relationship between PET units and their physical meaning. Xu et al. [90] indicate that the BN discretization scheme achieves relatively high stability compared to BW in line with IBSI recommendation [45], contrasting with Pfaehler et al. [28].

Most CT and MRI studies have focused on reproducibility and its relationship with repeatability. Notably, since 2021, there has been significant investigation into achieving optimal performance. Additionally, CBCT studies have examined both repeatability and reproducibility. PET/CT research includes three studies on repeatability published in 2019 and 2021, and two studies on optimal performance published in 2021. The selection of the best performance as the criterion for evaluating the quality of a model (including the choice of preprocessing parameters) should be limited, as this choice could introduce biases.

About the anatomical part investigated, CT studies focused on the torso's anatomical regions (chest, abdomen, and pelvis), while MRI studies regarded the brain and pelvis regions. Additionally, CBCT and PET/CT investigations were centered on the pelvis or chest. The distribution of the anatomical regions per imaging modality aligns with what is expected in a therapeutic pathway (e.g., using MRI for further investigation of brain or prostate/cervix due to its capability to distinguish between soft tissues compared to CT).

Concerning comparison metrics, the most commonly used metric was ICC, predominantly applied in MRI studies, with 15 out of 26 works. ICC is extensively studied in radiomics research due to its role in quantitative assessment, reliability, wide applicability, and standard practice. It serves as a quantitative measure to gauge agreement or consistency between measurements or observations, crucial for evaluating the reproducibility of radiomic features. ICC evaluates measurement reliability by accounting for systematic and random variations, making it effective for assessing feature stability across different scans or observers. Its versatility extends to various data types, including continuous and categorical variables, enabling the evaluation of different radiomic features across diverse imaging modalities and patient groups. Overall, ICC is favored in radiomics for its comprehensive and standardized approach to assessing feature reproducibility and stability, thereby supporting the development and validation of radiomics as a quantitative imaging biomarker. CCC is another essential statistical metric used in radiomics research to evaluate measurement agreement and reliability. It quantifies the agreement between two sets of continuous measurements, assessing precision (how closely measurements cluster around the line of perfect agreement) and accuracy (how close measurements are to the line of identity), offering a comprehensive measure of agreement that is easy to interpret. Like ICC, CCC accounts for systematic bias (shift) and random variation (spread) between measurements. This makes it suitable for evaluating the reproducibility and stability of radiomic features across different scans or observers. AUC quantifies the overall discriminatory ability of a diagnostic test or biomarker. In radiomics, this metric assesses the ability of radiomic features or models to distinguish between different clinical conditions or outcomes based on imaging data. Higher AUC values indicate the better discriminatory performance of the radiomic feature or model.

From our investigated studies, it was not possible to identify a recommended set of preprocessing parameters or a subset of stable features due to the variability in the experimental setups. However, higher-order class features (e.g., GLRM, GLCM-based) are more affected by preprocessing parameters as well as acquisition and reconstruction parameters, so they should be used with caution when dealing with retrospective multicentric studies. When building a predictive model, first-order class features should be preferred for inclusion should they show a correlation with the chosen outcome, especially if a preprocessing strategy is not employed.

Regarding the risk of bias analysis, it was not possible to assess it for the majority of the studies for patient selection. Since most of the radiomic studies are retrospective, it is not possible to understand how the patient selection process might affect the results. Many works showed low concern for risk of bias in the index test, due to the widely accepted choice of well-known statistical indexes, already broadly used in literature to test repeatability even in other scientific fields. Regarding applicability, most studies raised low concern for both patient selection and index test.

From our analysis, we also found out that a significant number of our included studies pursued best model performance in order to find the optimal preprocessing parameters. While this is still a feasible approach given the early stage of literature and research in this field, it would be advised to strive toward a standardization that is based on the feature stability rather than individual model performances as these are influenced by other variables (such as training/validation datasets).

From our analysis, we also found out that a significant number of our included studies pursued best model performance in order to find the optimal preprocessing parameters. While this is still a feasible approach given the early stage of literature and research in this field, it would be advised to strive toward a standardization that is based on the feature stability rather than individual model performances as these are influenced by other variables (such as training/validation datasets).

The problem of generalizability remains open, but using a preprocessing strategy improves the robustness of the analysis, especially when multiple vendors and machine models are included.

5. Conclusions

To date, it does not seem possible to establish a standardized recipe for the unequivocal selection of preprocessing parameter combinations for radiomic analysis. However, it is noteworthy that certain combinations of voxel resampling, normalization, and digitalization are more commonly used than others.

For reproducibility, a study must report the sequence of preprocessing parameters used so that the methodology employed is evident and transparent.

From our work, we highlighted the need to harmonize the use of preprocessing parameters and their values, especially considering future studies of prospective studies, which are still lacking in the current literature.