Lung Cancer Prediction Using Machine Learning on Structured Clinical Data: A Systematic Review of Diagnosis, Risk, and Survival Models

1. Introduction

Lung cancer remains a leading cause of global cancer mortality, underscoring the critical need for advanced methods to predict its onset, identify its presence, and forecast patient outcomes Siegel et al. (2023). The accurate prediction of lung cancer, encompassing diagnosis, risk assessment, and survival analysis, is a cornerstone for improving survival rates through timely screening and personalized intervention strategies. Clinical practice has long relied on traditional statistical models, such as the PLCOM2012 Tammemägi et al. (2013), Bach Bach et al. (2003), and LCRAT (Lung Cancer Risk Assessment Tool) Katki et al. (2016), which utilize risk factors like age and smoking history to stratify populations and guide screening protocols. While these models provide a solid foundation, the emergence of machine learning (ML) offers a paradigm shift, enabling the capture of complex, non-linear patterns within high-dimensional data that traditional methods may overlook.

The proliferation of Electronic Health Records (EHRs) and large-scale clinical registries has created vast repositories of structured, tabular data, including demographics, clinical parameters, and treatment histories. The application of ML to these structured datasets presents a significant opportunity to develop more accurate and personalized predictive models for the entire lung cancer care continuum Dritsas and Trigka (2022). However, the existing body of literature is fragmented. Previous review papers, such as those by Mamun et al. (2022) Mamun et al. (2022), Altuhaifa et al. (2023) Altuhaifa et al. (2023), and Juang et al. (2025) Juang et al. (2025), have provided valuable but narrow insights, often focusing on a single prediction task or including studies that are not exclusively based on ML or structured data. This fragmentation leaves a clear gap for a comprehensive synthesis that systematically analyzes ML applications across all major prediction tasks using structured data.

To address this gap, this systematic literature review (SLR) was conducted to provide a holistic overview of machine learning techniques for lung cancer prediction using structured clinical data. The specific contributions of this review are delineated as follows:

• Comprehensive Task Coverage: Unlike previous reviews, this SLR simultaneously investigates three critical prediction tasks—diagnosis, risk assessment, and survival analysis—providing a unified perspective on the ML landscape.
• Exclusive Focus on Structured Data: A strict focus is maintained on models built primarily on structured, tabular data, offering a dedicated resource for this data modality and distinguishing this work from reviews centered on medical imaging.
• Rigorous and Transparent Methodology: The review adheres to PRISMA 2020 guidelines Page et al. (2021), employing a systematic search across five major databases with tailored search strings, resulting in a robust synthesis of 42 high-quality, peer-reviewed studies.
• Task-Oriented Synthesis: A novel, task-categorized analysis of the literature is presented, detailing the specific datasets, feature sets, and ML models that are most effective for diagnosis, risk, and survival prediction, respectively.
• Critical Analysis and Future Directions: Beyond summarization, a critical appraisal of current methodologies is provided, highlighting performance trends, identifying common pitfalls such as the lack of external validation, and outlining clear pathways for future research.

Scope of this review: This review examines peer-reviewed studies that develop or evaluate machine learning models for lung cancer prediction using structured (tabular) data. The included studies are focused on three distinct tasks: (1) Diagnosis prediction – identifying the presence of lung cancer; (2) Risk prediction – estimating an individual’s future likelihood of developing lung cancer; and (3) Survival prediction – forecasting patient outcomes such as survival time or mortality. Studies that incorporate imaging-derived or genomic features are included only if these features are integrated into a structured, tabular format alongside clinical data.

The remainder of this paper is organized as follows. Section 2 reviews related works and prior reviews. Section 3 details the SLR methodology. Section 4 presents the results structured by the research questions. Section 5 provides a synthesized performance analysis. Section 6 discusses the overarching findings, limitations, and implications. Section 7 outlines the limitations of this review and proposes future work. Finally, Section 8 concludes the paper.

2. Related Works

The application of machine learning to structured clinical data for lung cancer prediction has gained substantial momentum in recent years. Primary studies have demonstrated promising results across various prediction tasks. For instance, Chandran et al. (2023) developed ML models using electronic health records to identify high-risk patients, achieving an AUC of 0.76 with LASSO logistic regression Chandran et al. (2023). Similarly, Wu et al. (2024) compared six ML models for predicting lymph node metastasis in NSCLC, finding that Generalized Linear Models performed best (AUC=0.810) Wu et al. (2024). Maurya et al. (2024) conducted a comparative analysis of twelve algorithms, reporting that K-Nearest Neighbors achieved 92.86% accuracy for lung cancer prediction Maurya et al. (2024). These studies collectively highlight the potential of ML approaches while revealing substantial heterogeneity in methodologies, datasets, and evaluation practices.

Several systematic reviews have attempted to synthesize this growing body of literature, though each exhibits specific limitations in scope or methodology. Mamun et al. (2022) focused on early-stage lung cancer prediction but included only ten studies, providing insufficient coverage of the field. Their review also lacked transparency regarding dataset availability and search methodology Mamun et al. (2022). Altuhaifa et al. (2023) exclusively addressed survival prediction using clinical data, identifying Random Forest as the predominant model across 30 studies. However, their narrow focus on survival outcomes and use of only three databases limited the review’s comprehensiveness Altuhaifa et al. (2023). Juang et al. (2025) conducted a broad examination of risk prediction models across Western and Asian populations but included numerous non-ML approaches and traditional statistical models, making it difficult to assess the specific contributions of machine learning techniques Juang et al. (2025).

Additional reviews have addressed adjacent topics while leaving the core focus unexplored. For example, Zhuan et al. (2023) systematically reviewed machine learning for radiation pneumonitis prediction in lung cancer patients, demonstrating excellent diagnostic performance (AUC=0.93) but focusing exclusively on this treatment-related complication rather than core prediction tasks Zhuan et al. (2023). Other reviews have emphasized imaging-based approaches or combined multiple data modalities without specifically addressing the unique challenges and opportunities presented by structured data analysis.

This systematic literature review addresses several critical gaps in the existing review landscape. Unlike previous works, it provides comprehensive coverage across all three major prediction tasks—diagnosis, risk assessment, and survival analysis—while maintaining an exclusive focus on structured data approaches. The review employs a rigorous methodology following PRISMA guidelines, with systematic searches across five major databases and transparent reporting of inclusion criteria. By synthesizing 42 high-quality studies, it offers robust insights into optimal datasets, feature sets, model architectures, and validation practices specific to structured data applications. Furthermore, it provides critical analysis of methodological limitations and future directions, aspects often underemphasized in previous reviews. This comprehensive approach enables a holistic understanding of the current state and future potential of ML for lung cancer prediction using structured clinical data.

3. Methodology

This systematic literature review was conducted following the PRISMA 2020 guidelines Page et al. (2021) to ensure methodological rigor and transparency. The review process encompassed research question formulation, search strategy development, study selection, quality assessment, data extraction, and synthesis.

3.1. Research Questions

Nine research questions (RQs) were formulated to comprehensively analyze the machine learning pipeline for lung cancer prediction from structured data. Table 1 presents the RQs alongside their rationale, providing a framework for the subsequent analysis.

3.2. Eligibility Criteria

Clear inclusion and exclusion criteria were established to ensure study relevance and quality. As shown in Table 2, inclusion criteria IC1, IC2, and IC3 were mandatory, focusing on lung cancer prediction relevance, structured data usage, and machine learning application.

3.3. Search Strategy

Search strategy is one of the most crucial aspects of finding relevant research papers. To make the SLR searching strategy reproducible by other researchers, the development of an appropriate search strategy using the most cited papers in the relevant field was done. First, identification of the primary papers for Lung Cancer was done. Using that as a base, keywords to be used as search strings were developed. Some of the primary keywords that were identified are: “Lung Cancer” or “Non-Small cell Lung Cancer” and “Detection” or “Prediction”. Synonyms of those words were also explored to build a suitable search string. After that, 5 publicly available databases: Pubmed, Scopus, IEEE, ACM, and Science Direct were explored. As the databases used different search methodologies, search strings were compiled for each of the databases separately. The complete process for selecting the studies is given below (PRISMA Flow Diagram), ensured transparency and reproducibility:

• The search strings were used for the retrieval of the relevant papers. A total of 772 papers were found using the initial search string.
• Inclusion/ Exclusion Criteria were applied for each of the papers and the total number of studies decreased to a total of 138 papers.
• After going through the abstract, methodology, and conclusion, several papers were removed due to being irrelevant, and ended up with 38 Papers.
• Snowballing was done on the selected papers, and the final number was 61.
• After eliminating the duplicates the total number of papers came up to 56.
• Finally, all the 56 papers were read thoroughly. Quality Assessments were applied. Resulting in the final 42 papers to be used for data extraction and synthesis process.

Table 3. Database-wise Search Strings.

Database Name	Search String	#Initial Papers	#Studies after IC/EC	After Screening Title/Abstract	Papers from Snowballing + QA	Duplicates Removed	Final Selection (Full-Text)
PubMed	("Lung Cancer"[tiab] OR "Non-Small Cell Lung Cancer"[tiab] OR "lung neoplasm"[tiab] OR "pulmonary cancer"[tiab]) AND ("Detection"[tiab] OR "Prediction"[tiab] OR "Diagnosis"[tiab] OR "prognosis"[tiab] OR "early detection"[tiab] OR "risk assessment"[tiab]) AND ("ML"[tiab] OR "Machine Learning"[tiab] OR "AI"[tiab] OR "Artificial Intelligence"[tiab] OR "data mining"[tiab] OR "ensemble learning"[tiab] OR "Ensemble Stacking"[tiab] OR "Stacking"[tiab] OR "Bagging"[tiab] OR "Boosting"[tiab]) AND ("tabular data"[tiab] OR "structured data"[tiab] OR "clinical data"[tiab] OR "patient records"[tiab] OR "health records"[tiab] OR "EHR"[tiab] OR "electronic health records"[tiab]) NOT ("Image"[ti] OR "Imaging"[ti] OR "Medical Imaging"[ti] OR "histopathology"[ti] OR "segmentation"[ti] OR "MRI"[ti] OR "X-ray"[ti] OR "ultrasound"[ti] OR "CT"[ti] OR "tomography"[ti] OR "blood"[ti] OR "vitamin"[ti] OR "mechanism"[ti] OR "transmission"[ti] OR "prevention"[ti] OR "forecasting"[ti] OR clinical studies[ptype] OR Editorial[ptyp] OR Comment[ptyp] OR Case Reports[ptyp] OR "survey"[tiab] OR "case study"[ti] OR "review"[tiab] OR "preprints"[tiab]) AND English[lang]	80	19	19	9	5	42
IEEE	((Lung Cancer OR Non-Small Cell Lung Cancer OR lung neoplasm OR pulmonary cancer) AND (Detection OR Prediction OR Diagnosis OR Prognosis OR Early Detection OR Risk Assessment) AND (Machine Learning OR ML OR Deep Learning OR AI OR Artificial Intelligence OR Data Mining OR Ensemble Learning OR Supervised Learning OR Decision Trees OR Random Forest OR SVM OR Logistic Regression OR Neural Networks OR Gradient Boosting OR XGBoost OR Ensemble Stacking OR Stacking OR Bagging OR Boosting) AND (Tabular Data OR Structured Data OR Clinical Data OR Electronic Health Records OR EHR OR Health Records OR Patient Records OR Patient Demographics) NOT (Image OR Imaging OR Medical Imaging OR Radiology OR Histopathology OR Segmentation OR MRI OR X-ray OR Ultrasound OR CT OR Tomography OR PET Scan OR Radiomics OR Blood OR Vitamin OR Mechanism OR Transmission OR Prevention OR Forecasting OR Trends OR Epidemiology OR Case Study OR Survey OR Review))	103	4	4	–	–	–
Scopus	(TITLE-ABS-KEY("Lung Cancer" OR "Non-Small Cell Lung Cancer" OR "lung neoplasm" OR "pulmonary cancer")) AND (TITLE-ABS-KEY("Detection" OR "Prediction" OR "Diagnosis" OR "Prognosis" OR "Early Detection" OR "Risk Assessment")) AND (TITLE-ABS-KEY("Machine Learning" OR "ML" OR "AI" OR "Artificial Intelligence" OR "Data Mining" OR "Ensemble Learning" OR "Supervised Learning" OR "Ensemble Stacking" OR "Stacking" OR "Bagging" OR "Boosting")) AND (TITLE-ABS-KEY("Tabular Data" OR "Structured Data" OR "Clinical Data" OR "EHR" OR "Health Records" OR "Patient Records")) AND (NOT TITLE-ABS-KEY("Image" OR "Imaging" OR "Medical Imaging" OR "Radiology" OR "Histopathology" OR "Segmentation" OR "MRI" OR "X-ray" OR "Ultrasound" OR "CT" OR "Tomography" OR "PET Scan" OR "Radiomics" OR "Blood" OR "Vitamin" OR "Mechanism" OR "Transmission" OR "Prevention" OR "Forecasting" OR "Trends" OR "Epidemiology" OR "Case Study" OR "Survey" OR "Review" OR "preprints")) AND (LIMIT-TO(LANGUAGE, "English"))	104	84	5	–	–	–
ACM	("Lung Cancer" OR "Non-Small Cell Lung Cancer" OR "lung neoplasm" OR "pulmonary cancer") AND ("Detection" OR "Prediction" OR "Diagnosis" OR "prognosis" OR "early detection" OR "risk assessment") AND ("ML" OR "Machine Learning" OR "AI" OR "Artificial Intelligence" OR "data mining" OR "ensemble learning" OR "Ensemble Stacking" OR "Stacking" OR "Bagging" OR "Boosting") AND ("tabular data" OR "structured data" OR "clinical data" OR "patient records" OR "health records" OR "EHR" OR "electronic health records") AND NOT ("Image" OR "Imaging" OR "Medical Imaging" OR "histopathology" OR "segmentation" OR "MRI" OR "X-ray" OR "ultrasound" OR "CT" OR "tomography" OR "blood" OR "vitamin" OR "mechanism" OR "transmission" OR "prevention" OR "forecasting" OR "clinical studies" OR "editorial" OR "comment" OR "case reports" OR "survey" OR "case study" OR "review" OR "preprints")	6	0	0	–	–	–
Science Direct	"Lung Cancer" AND ("Prediction" OR "Detection") AND ("Ensemble Learning" OR "Machine Learning") AND ("Tabular Data" OR "Structured Data" OR "Clinical Data")	479	15	10	–	–	–
Total	–	772	138	38	9	5	42

Table 3. Database-wise Search Strings.

Database Name	Search String	#Initial Papers	#Studies after IC/EC	After Screening Title/Abstract	Papers from Snowballing + QA	Duplicates Removed	Final Selection (Full-Text)
PubMed	("Lung Cancer"[tiab] OR "Non-Small Cell Lung Cancer"[tiab] OR "lung neoplasm"[tiab] OR "pulmonary cancer"[tiab]) AND ("Detection"[tiab] OR "Prediction"[tiab] OR "Diagnosis"[tiab] OR "prognosis"[tiab] OR "early detection"[tiab] OR "risk assessment"[tiab]) AND ("ML"[tiab] OR "Machine Learning"[tiab] OR "AI"[tiab] OR "Artificial Intelligence"[tiab] OR "data mining"[tiab] OR "ensemble learning"[tiab] OR "Ensemble Stacking"[tiab] OR "Stacking"[tiab] OR "Bagging"[tiab] OR "Boosting"[tiab]) AND ("tabular data"[tiab] OR "structured data"[tiab] OR "clinical data"[tiab] OR "patient records"[tiab] OR "health records"[tiab] OR "EHR"[tiab] OR "electronic health records"[tiab]) NOT ("Image"[ti] OR "Imaging"[ti] OR "Medical Imaging"[ti] OR "histopathology"[ti] OR "segmentation"[ti] OR "MRI"[ti] OR "X-ray"[ti] OR "ultrasound"[ti] OR "CT"[ti] OR "tomography"[ti] OR "blood"[ti] OR "vitamin"[ti] OR "mechanism"[ti] OR "transmission"[ti] OR "prevention"[ti] OR "forecasting"[ti] OR clinical studies[ptype] OR Editorial[ptyp] OR Comment[ptyp] OR Case Reports[ptyp] OR "survey"[tiab] OR "case study"[ti] OR "review"[tiab] OR "preprints"[tiab]) AND English[lang]	80	19	19	9	5	42
IEEE	((Lung Cancer OR Non-Small Cell Lung Cancer OR lung neoplasm OR pulmonary cancer) AND (Detection OR Prediction OR Diagnosis OR Prognosis OR Early Detection OR Risk Assessment) AND (Machine Learning OR ML OR Deep Learning OR AI OR Artificial Intelligence OR Data Mining OR Ensemble Learning OR Supervised Learning OR Decision Trees OR Random Forest OR SVM OR Logistic Regression OR Neural Networks OR Gradient Boosting OR XGBoost OR Ensemble Stacking OR Stacking OR Bagging OR Boosting) AND (Tabular Data OR Structured Data OR Clinical Data OR Electronic Health Records OR EHR OR Health Records OR Patient Records OR Patient Demographics) NOT (Image OR Imaging OR Medical Imaging OR Radiology OR Histopathology OR Segmentation OR MRI OR X-ray OR Ultrasound OR CT OR Tomography OR PET Scan OR Radiomics OR Blood OR Vitamin OR Mechanism OR Transmission OR Prevention OR Forecasting OR Trends OR Epidemiology OR Case Study OR Survey OR Review))	103	4	4	–	–	–
Scopus	(TITLE-ABS-KEY("Lung Cancer" OR "Non-Small Cell Lung Cancer" OR "lung neoplasm" OR "pulmonary cancer")) AND (TITLE-ABS-KEY("Detection" OR "Prediction" OR "Diagnosis" OR "Prognosis" OR "Early Detection" OR "Risk Assessment")) AND (TITLE-ABS-KEY("Machine Learning" OR "ML" OR "AI" OR "Artificial Intelligence" OR "Data Mining" OR "Ensemble Learning" OR "Supervised Learning" OR "Ensemble Stacking" OR "Stacking" OR "Bagging" OR "Boosting")) AND (TITLE-ABS-KEY("Tabular Data" OR "Structured Data" OR "Clinical Data" OR "EHR" OR "Health Records" OR "Patient Records")) AND (NOT TITLE-ABS-KEY("Image" OR "Imaging" OR "Medical Imaging" OR "Radiology" OR "Histopathology" OR "Segmentation" OR "MRI" OR "X-ray" OR "Ultrasound" OR "CT" OR "Tomography" OR "PET Scan" OR "Radiomics" OR "Blood" OR "Vitamin" OR "Mechanism" OR "Transmission" OR "Prevention" OR "Forecasting" OR "Trends" OR "Epidemiology" OR "Case Study" OR "Survey" OR "Review" OR "preprints")) AND (LIMIT-TO(LANGUAGE, "English"))	104	84	5	–	–	–
ACM	("Lung Cancer" OR "Non-Small Cell Lung Cancer" OR "lung neoplasm" OR "pulmonary cancer") AND ("Detection" OR "Prediction" OR "Diagnosis" OR "prognosis" OR "early detection" OR "risk assessment") AND ("ML" OR "Machine Learning" OR "AI" OR "Artificial Intelligence" OR "data mining" OR "ensemble learning" OR "Ensemble Stacking" OR "Stacking" OR "Bagging" OR "Boosting") AND ("tabular data" OR "structured data" OR "clinical data" OR "patient records" OR "health records" OR "EHR" OR "electronic health records") AND NOT ("Image" OR "Imaging" OR "Medical Imaging" OR "histopathology" OR "segmentation" OR "MRI" OR "X-ray" OR "ultrasound" OR "CT" OR "tomography" OR "blood" OR "vitamin" OR "mechanism" OR "transmission" OR "prevention" OR "forecasting" OR "clinical studies" OR "editorial" OR "comment" OR "case reports" OR "survey" OR "case study" OR "review" OR "preprints")	6	0	0	–	–	–
Science Direct	"Lung Cancer" AND ("Prediction" OR "Detection") AND ("Ensemble Learning" OR "Machine Learning") AND ("Tabular Data" OR "Structured Data" OR "Clinical Data")	479	15	10	–	–	–
Total	–	772	138	38	9	5	42

Figure 1. Search Process.

Figure 1. Search Process.

3.4. Selection Process

Selection process is a crucial step in ensuring that only the most relevant and high-quality studies are included in this review. By establishing clear evaluation criteria, we can systematically evaluate the rigor, reliability, and contribution of each paper. A two-phase selection process was employed. First, titles and abstracts were screened against eligibility criteria by two independent reviewers. Second, full-text assessment was conducted for potentially relevant studies. A quality assessment checklist (Table 4) was applied, with studies requiring a minimum score of 60% for inclusion. All 42 included studies met or exceeded this threshold.

• Research Objective Clarity
  • Does the paper have any relevance to lung cancer prediction?
• Dataset Structure and Relevance
  • Does the paper contain a clear description of the dataset used and is it structured?
• Descriptive Methodology & Use of Machine Learning
  • Does the paper clearly describe the methodology used?
  • Does it include machine learning models?
• Performance Metrics Mentioned
  • Does the paper denote the performance metrics of their prescribed methodology?
• Preprocessing Methods Described
  • Does the paper contain details on feature selection and preprocessing methods used?
• Dataset Availability and Details
  • Does the paper describe the availability of the dataset (e.g., public, private)?
• Comparisons Made
  • Does the paper make comparisons with other methods/models?
• Explainability and Interpretability Methods
  • Does the paper use any Explainable AI to describe the model’s output?
• Challenges & Limitations
  • Does the paper discuss the limitations of their methodologies?
• Recent Publication
  • Did the paper publish within the last 5–7 years?

After the assessment of the papers using the Selection Process Criteria, no papers were found that scored below the benchmark that was used. The Excel sheet used for the Selection Process is uploaded online for reference in Appendix A.1.

3.5. Data Collection

A data extraction form was created with excel to complete the data collection process. Any conflicts regarding a paper were discussed and resolved. The data collection is divided into 4 sections: 1. Metadata 2. Dataset 3. Methods 4. Evaluation. Table 5 describes the details of the form used for the data collection process and their relevance to the research questions.

3.6. Data Synthesis

Both Qualitative such as Feature Selection Methods, Dimensionality Reduction Methods, Preprocessing Techniques, Interpretability Methods, Best Performing Model and Quantitative data such as Sample Size (Numerical: Count), Performance Metrics (Numerical: Accuracy, F1-score, AUC, etc.) were extracted for this SLR. The data was gathered using the tabulation method recommended by Page et al. (2021). An excel file was created for the data collection process and relevant graphs and charts were created to answer the research questions.

4. Results

RQ1: What is the status of machine learning models for the prediction of lung cancer from tabular datasets over the years?

From Figure 2, the distribution of papers over the last 5-7 years published which are relevant to lung cancer prediction is shown. It can be seen that in 2018 - 2019 (2 Papers per year). However, an upward trend can be seen in the following year, with the paper counts rising to almost double (4 Papers) in 2020, 6 in 2021, 8 in 2022, and 14 in 2024. This signifies an interest and advancement in using Machine Learning models with tabular datasets for lung cancer prediction.

The projected increase to 16 or more papers further underscores the growing emphasis on this field, suggesting continued innovation and prioritization of lung cancer prediction in academic and clinical research.

RQ2: What are the key features (attributes) used in lung cancer prediction models?

From Table 6, we can see that the key attributes used in lung cancer prediction models span diverse categories, reflecting a multifaceted approach to lung cancer diagnosis. It is observed that Demographic features and Clinical parameters are the most frequently used features, appearing in 22 and 20 studies. Lifestyle factors and symptoms are moderately presented, with each having 8 studies. Meanwhile, treatment modalities and survival/outcomes are included in 10 and 9 studies. Molecular/genetic biomarkers and imaging/diagnostics are less common but emerging. Social/Economic Factors are least represented, highlighting a potential gap in socioeconomic data integration.

RQ3: Which tabular datasets are frequently used in lung cancer prediction?

3.1. Which datasets are used for lung cancer diagnosis prediction?

RQ3 reveals a diverse array of tabular datasets used in Lung Cancer Diagnosis prediction which can be visualized from Table 7. Population-based/Registry datasets such as SEER, Danish Registers EPIC, JPHC, and large oncologic DB provide extensive datasets for lung cancer diagnosis prediction, while clinical and EHR datasets like Flatiron Health EHR, Maine HIE, WSIC, Taiwan NHIRD, CLARO, and CLARIFY provide detailed clinical patient records. In addition, Imaging and Radiology datasets such as NLST & LIDC, ACRIN NSCLC-DICOM-PET and datasets from LUNA-16 and Kaggle contribute to lung cancer diagnosis prediction significantly. The inclusion of genomic and multi-omics datasets like TCGA, GTEx, and SLCG provide molecular and datasets from specialized sources or sensors such as Synthea, eNose, Pesticide & Lung cancer, Blood nanosensor, Multi-cancer, and Multi-modal provide an innovative approach to lung cancer diagnosis prediction.

3.2. Which datasets are used for lung cancer risk prediction?

The following Table 8 categorizes the datasets used for Lung Cancer Risk Prediction. Population-based/Registry datasets, such as the Danish National Registers, EPIC (European Prospective Investigation into Cancer and Nutrition), and Japan Public Health Center Study Cohort (JPHC), are widely used due to their large-scale, long-term follow-up nature. These datasets often include extensive demographic, lifestyle, and medical history data, making them valuable for studying lung cancer risk factors at a population level.

Similarly, Clinical/EHR (Electronic Health Record) data sources, such as the Taiwan National Health Insurance Research Database (NHIRD) and the Lung Cancer Prediction Dataset, provide real-world medical records, including diagnostic codes, prescriptions, and treatment histories. Such datasets enable machine learning models to predict lung cancer risk based on patient-specific clinical features.

In contrast, Synthetic/Sensor-Based datasets, such as Synthea Synthetic Patient Data, are less commonly used in lung cancer risk prediction. This suggests a research gap where more work is needed to explore how sensor-based data (e.g., wearable devices, environmental exposure tracking) and synthetic patient simulations can contribute to early detection and risk modeling.

3.3. Which datasets are used for lung cancer survival prediction?

Table 9 describes the datasets used for Lung Cancer Survival Prediction, categorized by data type and source. The inclusion of diverse dataset types reflects the review’s scope of including studies where data is ultimately used in a structured, tabular format for machine learning analysis.

Population-Based/Registry datasets, such as SEER (Surveillance, Epidemiology, and End Results) and Large Oncologic Databases, are widely used due to their extensive collection of long-term survival outcomes, patient demographics, and treatment histories. These datasets represent the core of structured data approaches, enabling large-scale epidemiological studies on lung cancer survival rates through their comprehensive tabular format.

Clinical/EHR datasets, including CLARIFY, NHIRD, etrospective cohort from CUCH, Flatiron Health EHR, CLARO, and the Wisconsin Prognostic Lung Cancer Subdirectory, provide real-world patient records with treatment details, laboratory results, and disease progression. These datasets are crucial for developing personalized survival prediction models and align perfectly with the structured data focus of this review.

Imaging and Radiology datasets, such as ACRIN, NSCLC-DICOM-PET, warrant specific justification for inclusion. While these originate from medical imaging scans (e.g., CT, PET scans), they are included because the studies that use them typically extract quantitative, structured features (radiomics) such as texture, shape, and intensity metrics. These extracted features are then organized in tabular format alongside clinical variables, making them compatible with traditional machine learning algorithms designed for structured data.

Genomic and Multi-Omics datasets, such as TCGA (The Cancer Genome Atlas) and SLCG, provide molecular-level insights including gene expression, mutations, and epigenetic alterations. These datasets are included because they represent structured biological data that can be integrated with clinical variables in tabular format. This facilitates precision oncology approaches by linking genetic variations to lung cancer survival through machine learning models that operate on structured input.

Synthetic/Sensor-Based/Specialized datasets, such as the Pesticide & Lung Cancer dataset, are less common but provide valuable insights into environmental and occupational exposure-related risk factors. These datasets are included when the sensor data or synthetic features are represented in a structured format suitable for machine learning analysis, highlighting the need for further research on non-genetic influences on survival.

RQ4: What are the preprocessing techniques used to feed them into the models?

Table 10 categorizes the key preprocessing techniques applied in the selected studies, illustrating a comprehensive approach to preparing data for modeling. The table identifies several essential methods: encoding and transformation, normalization/standardization, missing data handling, data cleaning and filtering, data splitting and validation, and domain-specific preprocessing.

In many studies, encoding and transformation techniques are employed to convert categorical data into numerical representations, which is essential for enabling machine learning algorithms to process diverse datasets effectively. Normalization and standardization techniques are similarly crucial; they scale the data so that each feature contributes equally during model training, improving stability and convergence. These steps are particularly important when dealing with variables that differ significantly in scale.

Handling missing data is another critical preprocessing step. The studies referenced in the table apply various strategies—such as imputation or exclusion—to mitigate the effects of incomplete data, thereby reducing bias and enhancing model reliability. Equally important are data cleaning and filtering processes that remove noise, errors, and outliers, ensuring that the input data is as accurate and consistent as possible.

The table also highlights the use of data splitting and validation, where datasets are partitioned into training, validation, and testing sets. This practice is fundamental for assessing model performance and ensuring that models generalize well to new, unseen data. Lastly, domain-specific preprocessing steps have been introduced in several studies to tailor the data preparation process to the unique characteristics of the specific dataset or application domain. These customized approaches utilize expert knowledge to extract nuanced patterns that might be overlooked by more generic preprocessing techniques.

In summary, Table 10 illustrates a multifaceted preprocessing pipeline that enhances model performance by addressing different aspects of data quality and consistency. The careful combination of these techniques—from standard transformations and scaling to specialized, domain-driven adjustments—ensures that the data is robust, well-structured, and ready for effective model training. This comprehensive approach not only improves predictive accuracy but also sets the stage for future advancements through the standardization and automation of preprocessing pipelines.

RQ5: What are the most commonly used machine learning algorithms for lung cancer prediction on tabular datasets?

From Figure 3, we can observe that Gradient Boosting, Artificial Neural Network, Logistic Regression, Naive Bayes are the most used ML models, where as Transformer Based Model, Voting Classifier, Generalized Linear Model are the least used ML models for lung cancer prediction. Table 11 provides a detailed overview of the machine learning models employed across selected studies for lung cancer prediction using tabular data. The table shows a wide range of algorithms, from classical statistical methods to advanced deep learning architectures, highlighting the diversity of approaches in the field. Many studies have favored probabilistic and ensemble-based approaches, with Naïve Bayes, Decision Trees, Random Forests, and Logistic Regression being among the most frequently applied methods. Naïve Bayes and Decision Trees are commonly used for their simplicity and interpretability, making them suitable for preliminary analysis and feature importance assessment. Random Forests and Gradient Boosting methods, on the other hand, are extensively adopted due to their robustness in handling high-dimensional data and their ability to mitigate overfitting through ensemble techniques. Support Vector Machines, k-Nearest Neighbors, and Artificial Neural Networks are also popular choices, often chosen for their flexibility in modeling complex, non-linear relationships. Logistic Regression remains a staple in the predictive modeling domain due to its efficiency and the straightforward interpretability of its coefficients. Additionally, specialized models such as the Kaplan-Meier estimator, Survival Models, and Fuzzy Inference Systems indicate that some studies have focused on time-to-event data or uncertainty handling, further emphasizing the domain-specific challenges of lung cancer prediction. Emerging techniques, including deep learning approaches like CNNs and Transformer-Based models, along with Graph Machine Learning and Variational Autoencoders (VAEs), signal a growing interest in leveraging advanced algorithms to capture intricate patterns within the data. The inclusion of models such as Linear Regression, K-Means Clustering, and LDA also points to exploratory analyses and dimensionality reduction efforts that precede more complex modeling. Overall, Table 11 underscores the comprehensive utilization of a broad spectrum of machine learning techniques in lung cancer prediction. The selection of these models reflects a balance between traditional methods, which offer interpretability and simplicity, and modern, sophisticated algorithms, which promise enhanced predictive performance and the ability to model complex relationships in the data. This diversity not only caters to the varying nature of tabular data but also paves the way for future research aimed at integrating multiple approaches to improve model robustness and accuracy.

5.1. Which models are commonly used for diagnosis prediction?

Table 12 provides an overview of the machine learning models that have been commonly employed for diagnosis prediction across various studies. The table reflects a diverse selection of models ranging from traditional statistical approaches to more advanced ensemble and deep learning methods.

Logistic Regression is frequently used, as evidenced by its presence in several studies (PS20, PS26, PS32, PS35). Its popularity can be attributed to its efficiency and the clear interpretability of its coefficients, which are particularly valuable in a clinical context. Support Vector Machines (SVM) also appear prominently (PS20, PS24, PS26, PS39), offering robust classification through margin-based separation that helps in distinguishing between diagnostic categories effectively.

Ensemble methods such as Random Forest (PS24, PS26, PS31, PS36, PS37) and XGBoost (PS4, PS20, PS24, PS26, PS36) are widely applied. These models are favored for their ability to handle complex, non-linear relationships and reduce overfitting through the aggregation of multiple decision trees. Decision Trees (PS24, PS39, PS35) provide intuitive insights into the decision-making process, making them a useful tool for understanding diagnostic criteria.

Other models like k-Nearest Neighbors (PS20, PS26) and Naïve Bayes (PS20, PS24, PS26, PS36, PS35) are also in use, offering simplicity and speed for classification tasks based on proximity or probabilistic assumptions, respectively. Neural Networks, including various architectures such as ANN, MLP, and CNN (PS24, PS20, PS26, PS31, PS32, PS34, PS36, PS38, PS39, PS35, PS40), have gained traction for their ability to capture complex, high-dimensional patterns in diagnostic data.

Additionally, models like Linear Discriminant Analysis (PS1, PS39) serve a dual role in classification and dimensionality reduction. Specialized ensemble techniques, such as Extra Tree (PS26) and AdaBoost (PS26), have been explored to further improve predictive performance. Emerging methods such as CATBoost (PS36), Bayesian Networks (PS35), and Variational Autoencoders (PS38) illustrate the field’s ongoing efforts to integrate more advanced and nuanced algorithms into diagnostic prediction frameworks.

Overall, the distribution of models presented in Table 12 underscores a broad methodological spectrum. This diversity reflects the balance between the need for interpretable, reliable models and the pursuit of higher predictive accuracy through advanced machine learning techniques. The varying approaches also suggest that the choice of model may be influenced by specific dataset characteristics and the unique requirements of each diagnostic task.

5.2. Which models are commonly used for risk prediction?

Table 13 outlines the range of models used for risk prediction in lung cancer, revealing a preference for both traditional and advanced machine learning techniques. Logistic Regression is a staple in these studies, appearing in multiple works (PS6, PS10, PS14, PS22, PS23, PS28, PS33, PS13) due to its straightforward interpretability and efficiency in estimating risk probabilities. Ensemble methods also play a significant role, with Random Forest models frequently adopted (PS3, PS6, PS22, PS23, PS28, PS33, PS13) for their ability to manage non-linear relationships and reduce overfitting through aggregation of decision trees.

XGBoost, another ensemble method known for its gradient boosting framework, is similarly prevalent (PS22, PS23, PS28, PS33, PS13), offering high accuracy and robustness in handling complex data structures. Support Vector Machines are also utilized (PS6, PS22, PS28, PS33) to delineate risk boundaries in high-dimensional spaces, while Decision Trees provide an intuitive means for risk stratification (PS3, PS22, PS23, PS33).

Advanced boosting techniques such as Gradient Boosting Machines (PS6, PS23, PS22) further refine risk predictions by sequentially correcting errors made by previous models. Neural network-based approaches, represented by Multi-Layer Perceptrons (PS6, PS33), add another layer of sophistication by capturing intricate, non-linear patterns in the data. In addition, models like LightGBM (PS13) have been explored for their efficiency and scalability.

The table also indicates the use of specialized methods tailored to specific aspects of risk prediction, such as Parametric Survival Models (PS9, PS17), which incorporate time-to-event data, and Bayesian Networks (PS22) that model probabilistic relationships. Other notable approaches include LASSO (PS28) for feature selection, Fuzzy Inference Systems (PS3) to handle uncertainty, and optimization methods like Stochastic Gradient Descent (PS22). Finally, combinations of ensemble techniques like J48, AdaBoost, and Rotation Forest (PS22) underscore the experimental nature of research in this area.

Overall, the models employed for risk prediction reflect a balance between classic statistical methods and modern machine learning algorithms. This diverse methodological landscape not only accommodates the varied characteristics of clinical and tabular datasets but also supports the development of robust risk prediction frameworks that can be fine-tuned to specific clinical applications.

5.3. Which models are commonly used for survival prediction?

Table 14 provides an overview of the machine learning models that have been commonly employed for survival prediction across various studies. The table illustrates a diverse selection of models, ranging from traditional statistical methods to advanced deep learning techniques, each addressing the unique challenges posed by survival data.

Random Survival Forest appears in studies PS2 and PS16, showcasing its strength in handling censored data and capturing non-linear relationships through ensemble learning. The Cox Proportional Hazards model, featured in PS10, PS15, and PS16, remains a staple in survival analysis due to its interpretability and established framework for estimating hazard ratios.

Deep learning techniques have also made inroads in survival prediction. DeepSurv (PS15, PS21) and Neural Networks (Deep Learning) (PS5, PS25, PS29, PS41, PS10) illustrate the field’s move toward models that can uncover complex, high-dimensional patterns in survival outcomes. More traditional classification methods are still in use, with Logistic Regression (PS8, PS12, PS29, PS13, PS10) and Support Vector Machines (SVM) (PS5, PS8, PS12, PS27, PS41) being applied for their robustness and efficiency.

Additionally, ensemble methods such as Random Forest (PS8, PS12, PS13) and XGBoost (PS8, PS13) are favored for their ability to mitigate overfitting and manage intricate interactions within the data. Decision Trees (PS5, PS7, PS8, PS12, PS27, PS13) offer a clear, interpretable structure for decision-making, while Gradient Boosting Machine (PS12, PS27, PS41) and Linear Regression (PS27, PS41) provide further avenues for predictive modeling with varying degrees of complexity. Complementary models like K-Nearest Neighbors (PS7, PS8, PS12), Naive Bayes (PS7, PS12), and REPTree (PS7) round out the table, each contributing unique advantages in terms of speed, simplicity, or alternative tree-based strategies.

Overall, the distribution of models in Table 14 reflects a balance between leveraging traditional, interpretable approaches and embracing modern, computationally intensive methods, ensuring that survival prediction frameworks can be tailored to the specific characteristics and requirements of different datasets.

5. Performance Analysis

The analysis of performance metrics across the 42 studies, detailed in Table 21, reveals a wide spectrum of results, with many models reporting high accuracy and AUC values. However, a critical synthesis is essential to contextualize these findings beyond the raw metrics. Performance was highly contingent on the specific prediction task. In diagnosis prediction, models frequently reported the highest performance, with several studies achieving accuracies and AUCs above 95%. For instance, Multi-class Neural Networks reached 100% accuracy (PS25), XGBoost achieved 97.5% (PS30), and an MLP with feature selection delivered 98% accuracy (PS12). Ensemble methods like XGBoost and Random Forest, along with deep learning hybrids such as the ConvXGB model (97.43% accuracy, PS36), were consistently among the top performers. In contrast, performance for risk prediction was generally more modest but clinically realistic, with AUCs typically ranging from 0.75 to 0.88, as seen in XGBoost (0.881 AUC, PS28) and other ensemble methods, reflecting the inherent challenge of forecasting future events in diverse populations. For survival analysis, the Concordance Index (C-index) was the primary metric, with values generally between 0.70 and 0.80. Machine learning models like DeepSurv (C-Index of 0.7181 in PS21 and 0.74 in PS15) demonstrated competitive performance against traditional benchmarks.

While these results, including exceptional reports such as 99.866% accuracy from Decision Trees with k-means clustering (PS18) and 99% accuracy from XGBoost with SMOTE+ENN+PCA (PS33), are impressive, they must be interpreted with caution. Such figures can be inflated by several methodological factors prevalent in the reviewed literature. The high performance in diagnostic tasks often originates from relatively small, curated datasets or employs sophisticated techniques to address class imbalance, which may not generalize to broader, more heterogeneous clinical populations. Furthermore, the scarcity of external validation means the robustness of these models on independent, unseen datasets from different institutions is largely unknown, raising significant concerns about overfitting. Potential data leakage from inadequate separation of training and test sets can also lead to optimistically biased performance estimates. Therefore, while Table 21 provides a comprehensive overview of reported capabilities, the most reliable finding of this review is not the absolute performance numbers, but the consistent trend identifying ensemble methods, particularly XGBoost and Random Forest, as the most effective and robust model types for structured data across all prediction tasks.

6. Discussion

This systematic review provides a structured synthesis of machine learning applications for lung cancer prediction using structured data, explicitly covering three distinct tasks—diagnosis, risk assessment, and survival analysis. Our findings reveal a field where the optimal ML approach is highly task-dependent, yet dominated by a consistent set of powerful techniques. Contrary to the broader AI trend, it is evident that while Deep Learning has gained popularity, traditional Machine Learning models, particularly ensemble methods, are preferred for prediction tasks on structured clinical data. Among these, Gradient Boosting (especially XGBoost) and Random Forest emerged as the most prevalent and high-performing models across the board, valued for their robustness with tabular data. Their application, however, is nuanced: for diagnosis, they are often part of complex pipelines with heavy feature engineering; for risk assessment, they provide a powerful upgrade to traditional statistical models; and for survival analysis, specialized ensembles are gaining traction against established benchmarks like Cox regression.

A key finding is the critical importance of task-specific data sources and predictive features. The SEER registry was a cornerstone for diagnosis and survival studies, while population registries (e.g., Danish, EPIC) were pivotal for risk prediction. The most influential predictive features were consistently demographics—with attributes such as Age, Gender, Race, and Smoking History being almost universal—and core clinical parameters like TNM stage and comorbidities. The widespread adoption of explainable AI (XAI) techniques, particularly SHAP and LIME, signifies a maturation in the field, enabling practitioners to understand the reasoning behind model predictions and fostering necessary trust for potential clinical integration.

However, this review also uncovers pressing methodological issues that threaten the translation of these models into clinical practice. The overwhelming reliance on holdout validation and the rarity of external validation cast doubt on the real-world robustness and generalizability of the reported high performances. Furthermore, while XAI tools are widely adopted, their use is often superficial, indicating a need for more standardized reporting and a stronger link between model interpretations and established clinical knowledge. Finally, the reliance on private datasets and frequently incomplete reporting of preprocessing and feature selection steps severely hampers reproducibility and independent verification, representing a significant barrier to scientific progress. By synthesizing evidence across the entire prediction continuum, this review not only maps the current landscape but also provides a foundational roadmap, highlighting that future work must prioritize robust validation, deepened interpretability, and enhanced reproducibility to develop truly clinically actionable ML tools.

7. Limitations and Future Works

This systematic review was designed with specific methodological boundaries to ensure a focused and in-depth analysis, and these deliberate choices both define its unique contribution and highlight pathways for future research. The strict focus on structured, tabular data was central to providing a dedicated synthesis for this data modality, enabling a precise analysis of its specific challenges and opportunities; however, this necessarily excluded innovative approaches utilizing non-tabular data like free-text clinical notes or raw medical images. Furthermore, the decision to adopt a comprehensive scope encompassing diagnosis, risk, and survival prediction provides a holistic, clinically-relevant view of the patient journey. We contend that this inclusive approach is a strength that offers a unified perspective absent from more narrowly focused reviews, even if it broadens the scope beyond a strict definition of "early prediction." Finally, while the search strategy was rigorous, the rapid evolution of the field and the focus on peer-reviewed literature mean some recent or preliminary work may not have been captured.

The synthesis conducted within these boundaries has clearly delineated several critical priorities for the field. The most pressing need is a paradigm shift towards robustness and reproducibility, necessitating the standard adoption of external validation and the creation of public benchmark datasets to overcome the current overreliance on private data and internal validation, which severely limits clinical credibility. Subsequently, research must deepen its commitment to clinical utility by moving beyond the routine application of explainable AI tools towards developing standardized, clinically-grounded interpretation frameworks and conducting real-world feasibility studies. From a technical standpoint, there is significant, untapped potential for architectural innovation beyond the dominant ensemble models, particularly in exploring advanced strategies like stacking and blending or pioneering hybrid architectures that fuse different learning paradigms. Ultimately, by addressing these priorities and building upon this foundational review through the integration of multi-modal data and cross-disciplinary applications, the field can bridge the gap between technical performance and genuine clinical impact in precision oncology.

8. Conclusion

This systematic review has synthesized the landscape of machine learning applications for lung cancer prediction using structured clinical data, comprehensively addressing diagnosis, risk assessment, and survival analysis. Through a rigorous analysis of 42 studies, our task-oriented synthesis reveals that ensemble methods, particularly XGBoost and Random Forest, are the most effective and prevalent algorithms across all prediction tasks. Their performance, however, is profoundly shaped by the specific clinical objective and the corresponding dataset, with key predictors consistently being demographics like age and smoking history, and core clinical parameters such as tumor stage. Despite the field reporting impressive performance metrics, this review uncovers a critical impediment to clinical translation: a pervasive lack of external validation and standardized reporting, which casts doubt on the real-world robustness of these models. Furthermore, while explainability tools like SHAP and LIME are widely adopted, their application requires deeper clinical integration. By mapping the state-of-the-art and clearly delineating these methodological gaps, this review provides a foundational roadmap for developing transparent, reproducible, and clinically actionable ML tools. Addressing these challenges is paramount for the research community to truly advance precision oncology and improve patient outcomes in lung cancer.

Appendix A.1

Access the supplementary data here:

• Google Sheets: Lung Cancer Study Selection Process