Preprints.org logo
Preprint
Review

This version is not peer-reviewed.

Multimodal Artificial Intelligence for Pneumonia Prognosis and Clinical Deterioration: A Systematic Review

Submitted:

18 August 2026

Posted:

19 August 2026

You are already at the latest version

Abstract
Objective: To review multimodal artificial intelligence (AI) models for predicting adverse outcomes in pneumonia, with emphasis on prognosis, validation, benchmarking, explainability, and clinical implementation readiness. Materials and Methods: We conducted a PROSPERO-registered systematic review. PubMed, Scopus, IEEE Xplore, and supplementary register/citation-search sources were searched between 01 January 2000 and 27 February 2026. Eligible studies integrated at least two heterogeneous clinical data modalities within one predictive framework for pneumonia-related deterioration, mortality, escalation of care, ventilation, or related adverse outcomes. We extracted modality composition, fusion architecture, outcome alignment, validation, performance metrics, clinical benchmarking, explainability, predictive uncertainty, workflow integration, governance reporting, and risk of bias. Results: Of 10,460 identified records, 49 studies were included. The evidence base was dominated by COVID-era studies. Structured electronic health record data, laboratories, and vital signs were more common than imaging or clinical notes. Early fusion was the predominant strategy, while attention-based and graph-based fusion were rare. External validation was reported in 19/49 studies, calibration in 9/49, temporal validation in 5/49, decision curve analysis in 3/49, and prospective evaluation in 3/49. Incremental AUROC over the best unimodal baseline was computable in 19 studies and was usually modest. Discussion: The literature emphasised retrospective discrimination more than clinical utility, with limited reporting of added value, workflow, predictive uncertainty, and governance. Conclusion: Multimodal pneumonia prognosis research is active but not yet implementation-ready. Future studies should prioritise externally validated, calibrated, explainable, clinician-aligned, and workflow-aware models.
Keywords: 
pneumonia
;  
artificial intelligence
;  
machine learning
;  
prognosis
;  
clinical deterioration
Subject: 
Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Background and Significance

Pneumonia and lower respiratory infections remain major causes of hospital admission, respiratory deterioration, intensive care unit (ICU) escalation, mechanical ventilation, and death worldwide. 1–4 In practice, the key informatics problem is not simply classifying pneumonia on a single image, but predicting whether a patient will deteriorate, require escalation, or die within a clinically meaningful time window. 2–4 Such models must fit real pathways in which observations change over time, laboratory tests and imaging may be delayed or absent, and decisions are made by multidisciplinary teams. 4–6
AI and machine learning are increasingly used to combine heterogeneous hospital data for risk prediction. 5,7 In pneumonia, multimodal modelling may integrate EHR variables, laboratories, vital signs, CXR, CT, radiology-derived features, notes, microbiology, or omics, as seen across the included studies. 8–56 The rationale is plausible: observations capture instability, laboratories reflect inflammation and organ dysfunction, imaging shows anatomical disease burden, and notes may contain contextual information unavailable in structured fields. 2–4,8,20,27,56
However, “multimodal” is used inconsistently: studies may concatenate variables, combine outputs, learn shared representations, apply attention, or construct patient-similarity graphs. 7–9,13,20,38,56
For adoption, high AUROC is insufficient. A useful deterioration model should show added value over simpler baselines, calibration, validation beyond the development setting, robustness to missing or poor-quality modalities, comparison with clinical scores, and a plausible workflow route. 5,6,57,58 It should also communicate predictive uncertainty and explanations in ways clinicians can review and challenge. 5,6 This review therefore treats multimodal pneumonia prediction as a clinical adoption problem, especially because much evidence was generated during COVID-era pathways that may not generalise to routine non-COVID pneumonia care.

2. Objective

This systematic review aimed to characterise multimodal fusion architectures for adverse outcome prediction in pneumonia and to assess their validation maturity, incremental benefit, benchmarking, explainability, and clinical implementation readiness. The review asked: which modalities are used; which fusion strategies dominate; whether multimodal models improve over unimodal baselines; how well studies report calibration, external or temporal validation, and prospective evaluation; whether models are compared with clinical benchmarks; and whether the literature addresses predictive uncertainty, human oversight, workflow integration, governance, and clinical deployment.

3. Materials and Methods

3.1. Protocol and Registration

The review was registered in PROSPERO as: Harib W, Fouad S. Multimodal Artificial Intelligence Models for Predicting Clinical Deterioration in Patients with Pneumonia: A Systematic Review. PROSPERO 2026 CRD420261328187. 59 Reporting was guided by the PRISMA 2020 statement and flow-diagram structure for new systematic reviews. 60 The manuscript title was refined to emphasise pneumonia prognosis, clinical deterioration, fusion architecture, validation maturity, benchmarking, and clinical deployment gaps; this did not alter the registered scope, which concerned multimodal AI for pneumonia-related deterioration or adverse prognosis.

3.2. Search Strategy and Eligibility

The final search was completed on 27 February 2026. Searches covered English-language publications from 2000 to 2026 in PubMed, Scopus, and IEEE Xplore, using terms for pneumonia, prognosis or clinical deterioration, and AI or machine learning. Supplementary register/citation-search sources were used to capture potentially missed eligible records, including protocol/register checking and citation-based snowballing during final eligibility reconciliation. These records were screened using the same eligibility criteria as database records. All records were exported and deduplicated in Zotero. Full search strategies are provided in Supplementary Appendix 1. Studies were eligible if they developed, validated, or evaluated an AI or machine-learning model for adverse outcome prediction in patients with pneumonia and used at least two heterogeneous clinical data modalities integrated within one predictive framework. Eligible outcomes included mortality, deterioration, ICU admission, mechanical ventilation, escalation of care, respiratory failure, and composite adverse outcomes. We excluded unimodal models, laboratory-only models, vital-sign-only models, CXR-only models, CT-only models, structured-EHR-only models, notes-only models, non-integrated multi-input studies, reviews, unavailable full texts, and studies with surrogate or unclear outcomes. Ensemble or late-stage combination approaches were retained only when modality-specific information was brought together for the target outcome and were coded separately to avoid overstating architectural fusion.

3.3. Study Selection, Extraction, and Coding

Title/abstract and full-text screening were led by one reviewer against the final eligibility definitions. To reduce eligibility drift, the final included and excluded full-text set was re-reviewed twice against the multimodal fusion criteria before the dataset was frozen. The final included set and risk-assessment file were then reviewed by academic and clinical co-authors for eligibility consistency, pneumonia relevance, outcome alignment, and clinical interpretability.
Data extraction captured bibliographic information, country, setting, sample size, pneumonia type, COVID-related status, modality composition, fusion type, fusion level, outcome category, prediction horizon, index time, AUROC, area under the precision-recall curve (AUPRC), best unimodal AUROC, calibration, decision curve analysis, clinical benchmark comparison, external validation, temporal validation, multicentre status, prospective evaluation, missing-modality handling, predictive uncertainty quantification, workflow integration, real-time modelling, regulatory or governance discussion, explainability, and leakage concerns from each included report. 8–56 Outcome categories were not mutually exclusive: a single study could contribute to more than one outcome category where multiple endpoints were modelled.
Fusion strategies were coded as early fusion, late fusion, hybrid fusion, attention fusion, graph-based fusion, or ensemble/no true fusion, using multimodal-learning taxonomy as a conceptual guide and study-level architecture descriptions as the source of classification. 7 Early fusion denoted input-level or feature-level integration before prediction; late fusion denoted output-level or model-level combination; hybrid fusion denoted mixed or staged fusion; attention fusion denoted explicit learned cross-modal weighting; graph-based fusion denoted patient, feature, modality, or similarity graph modelling; and ensemble/no true fusion denoted a combined predictive framework without deeper cross-modal representation learning. The studies assigned to each fusion category are cited in the Results and fully itemised in Supplementary Table S2.
Implementation-readiness variables were coded conservatively, following clinical AI evaluation and reporting guidance. 5,6,58 Workflow integration required an explicit route into clinical work, not a generic usefulness statement. Real-time modelling required live or near-real-time prediction rather than retrospective application. Human oversight and explainable risk communication were coded separately from generic feature importance or saliency, because technical explanations do not show that clinicians reviewed, trusted, or acted on the output. 5,6 Full study-level mappings for modality, fusion, outcome, validation, implementation readiness, incremental benefit, explainability, governance, and risk of bias are provided in the Supplementary Material.

3.4. Validation Maturity, Incremental Benefit, and Risk Assessment

Validation maturity was scored pragmatically using one point for each reported component: multicentre data, external validation, temporal validation, calibration, decision curve analysis, prospective evaluation, and explicit missing-modality handling. These components reflect issues emphasised in prediction-model and clinical-AI reporting guidance, including validation, calibration, clinical utility, deployment context, and missing data. 5,6,57 Scores could range from 0 to 7. Incremental benefit was calculated as Δ AUROC = AUROC(multimodal model) minus AUROC(best unimodal baseline), and was considered computable only where both values were clearly reported or extractable. For the primary summary distribution, we retained one comparable study-level contrast per study; additional endpoint-specific, site-specific, horizon-specific, index-augmentation, or otherwise non-comparable values were tabulated descriptively in the Supplementary Material. Risk of bias and applicability were assessed using domains aligned with PROBAST: participants, predictors, outcome, and analysis. 57 No meta-analysis was undertaken because populations, modalities, outcomes, horizons, architectures, and validation designs were heterogeneous.

3.5. Data Synthesis and Evidence Mapping

Data synthesis was descriptive and evidence-mapping based. Counts used the 49 included studies as the denominator unless otherwise stated. Outcome assignments were non-mutually exclusive because some studies modelled more than one endpoint. Row-level mappings for study characteristics, modality and fusion categories, outcome assignments, validation components, implementation readiness variables, explainability and governance signals, risk-of-bias judgements, and primary and descriptive Δ AUROC values are provided in the Supplementary Material.

4. Results

4.1. Study Selection

The searches identified 10,460 records: 10,455 from databases and 5 from registers/other sources. After removal of 1,761 duplicates, 8,699 records were screened and 769 reports were assessed for eligibility. Of these, 724 were excluded at full-text stage, most commonly because they were unimodal, non-integrated multi-input studies, or had surrogate or unclear outcomes. Forty-five studies were included from database/search records, and four further eligible studies were added through snowballing, giving 49 included studies (Figure 1; Table 1).
Table 1. Included studies and key characteristics.
Table 1. Included studies and key characteristics.
Study n Modalities Fusion Outcome(s) Validation
Mugisha et al. 8 850 EHR; vitals; notes Hybrid Death/discharge Internal
Xing et al. 9 412 CT omics; clinical; labs; microbial Graph Survival Internal/same-centre
Hong et al. 10 1979 CT; proteomics; clinical/labs Hybrid Hospital mortality Internal
Lizzi et al. 11 1589 CXR; clinical; labs; vitals Early Severity External
Keicher et al. 12 132 CT; radiomics; clinical; labs; vitals Graph ICU admission External
Zhou et al. 13 2362 CT/CXR; clinical; labs; text fields Attention ICU; ventilation; death External
Zhou et al. 14 103 CT; clinical; labs; vitals Ensemble ARDS Internal
Butler et al. 15 3571 CXR; EHR; labs; vitals Early ARDS; ICU; death Internal holdout
Chassagnon et al. 16 693 CT; radiomics; clinical; biological Early Severity; intubation; death Independent multicentre
Chao et al. 17 295 CT; radiomics; demographics; vitals; labs Early ICU admission Cross-site/internal
Ning et al. 18 1521 CT; clinical; labs Late Morbidity; mortality Cross-validation
Li et al. 19 516 CXR; clinical; blood tests; vaccination Early Respiratory sequelae Internal
Shamout et al. 20 3661 CXR; vitals; demographics; labs Late Deterioration Internal; prospective signal
Lassau et al. 21 931 CT; clinical; biological Late Oxygen need; ventilation; death External
Zheng et al. 22 198 CT features; clinical; labs Early Severe/critical progression Internal
Purkayastha et al. 23 981 CT radiomics; clinical; symptoms; labs Hybrid Critical illness Internal
Fang et al. 24 1040 Sequential CT; clinical; history; labs Early Malignant progression Multicentre/external
Jakob et al. 25 3487 Demographics; symptoms; comorbidities; labs; vitals Early Advanced COVID-19 Temporal
Hiremath et al. 26 869 CT; imaging score; clinical/labs Late Ventilation; ECMO; death External site
Dayan et al. 27 16148 CXR; age; vitals; labs; oxygen device Early Oxygen requirement/death Federated/external
Pournazari et al. 28 724 Clinical; vitals; labs; echocardiography Early Mortality; ECMO Internal
Mu et al. 29 6278 CXR; EHR scores; vitals/time series Late Clinical deterioration Internal
Cheng et al. 30 654 Longitudinal CXR; clinical; labs; comorbidities Ensemble In-hospital mortality External
Girardi et al. 31 110996 Demographics; symptoms; comorbidities; vitals; labs Hybrid Hospitalisation Internal
Whitfield et al. 32 2797 EHR; labs; clinical; text-derived Early Prognosis subtype Cross-validation
Bendavid et al. 33 11816; 1061 Demographics; vitals; labs; treatment/time series Early Invasive ventilation Transfer/internal
Wu et al. 34 1628; 201; 315 EHR; comorbidities; labs; vitals; CXR Late 30-day mortality Internal/external
Lee et al. 35 2282 Clinical; labs; CXR AI outputs Early Oxygen; ventilation; ECMO; ICU; death Centre-based external
Walston et al. 36 1356 CXR; demographics; history; vitals; labs Early Mortality; ICU; ventilation Internal holdout
Di Napoli et al. 37 1051 3D CT; demographics; symptoms; comorbidities; labs Hybrid Mortality; intubation; ICU External hospital
Guarrasi and Soda 38 820; 283 CXR; clinical; labs; vitals Hybrid Mild vs severe CV; leave-one-centre-out; external
Dimitsaki et al. 39 500 Proteomics; age; comorbidity Early Serious COVID-19 Cross-dataset/external
Min et al. 40 3025 Demographics; symptoms; CT signs; comorbidities Early Deterioration/recovery Internal
Charkoftaki et al. 41 111 Clinical; comorbidities; labs; respiratory; metabolomics Early Survival; LOS; ventilation/intubation Internal
Tehrani et al. 42 321 CT; demographics; symptoms; vitals; labs Early High-risk death/recovery Internal CV
Honchar et al. 43 166 Demographics; labs; oxygen; functional tests Early Post-COVID syndrome Internal
He et al. 44 922 Immune phenotyping; labs; cytokines Early Mild vs critical Additional cohorts
Zhan et al. 45 2113 CT radiomics; demographics; symptoms; labs; EHR Early ICU admission Repeated internal
Maghsoudi et al. 46 1230 Clinical; CT; lung ultrasound Early In-hospital mortality Internal
García-Blanco et al. 47 10000 EHR; ICD-10; vaccination; NLP/report features Early Acute respiratory failure Internal
Del Tedesco Júnior et al. 48 528 Clinical; CT; biomarkers; genetic variants Early Critical disease/death Internal
da Silva and Pazin-Filho 49 844 Structured labs; monitoring; notes/NLP Early In-hospital mortality Internal
Mirus et al. 50 375 CT radiomics; clinical; labs; ventilator variables Early Need for vv-ECMO Temporal internal
Olar et al. 51 1103 CXR; metadata; blood variables; Brixia score Early Ventilation/death Internal
Dipaola et al. 52 1296 ED tabular; clinical text; reports; labs Late 30-day mortality; ICU Internal
Gong et al. 53 146 CT consolidation; demographics; SpO2; labs Early ICU admission/death Internal CV
Kwon et al. 54 499 CXR; ED clinical; labs; vitals Early 30-day intubation/death Temporal internal
Kim et al. 55 2309 CXR; clinical; labs; demographics; comorbidities Late Critical event/time to event Internal/external
Jiao et al. 56 2309 CXR; clinical; labs; demographics; comorbidities Late Critical disease/time to event Internal/external

4.2. Study Characteristics and Evidence-Base Context

The 49 included studies evaluated multimodal models for pneumonia-related adverse outcomes including death, deterioration, ICU admission, mechanical ventilation, respiratory failure, escalation, and composite outcomes. 8–56 The individual studies and their key characteristics are summarised in Table 1; full row-level extraction is provided in the Supplementary Material. The evidence base was highly COVID-era shaped: 47/49 studies were COVID-related, 10–56 whereas only two were non-COVID pneumonia studies. 8,9 Outcome alignment was direct in 31 studies 11–17,20–24,26,27,29,32,33,35–38,40,45,47,50–56 and partial in 18. 8–10,18,19,25,28,30,31,34,39,41–44,46,48,49 External validation was reported in 19/49 studies 11–13,16,17,21,24,26,27,30,33–35,37–39,44,55,56 and prospective evaluation in 3/49. 9,10,20 Overall risk of bias was high in 47/49, unclear in 2/49, 21,27 and low in none. Study-level labels are shown in Table 1; detailed extraction and coding are provided in the Supplementary Material.

4.3. Modalities and Fusion Architectures

Modality use showed that multimodal pneumonia prognosis was not primarily an image-plus-EHR literature (Figure 2). Structured EHR variables appeared in 48/49 studies, 8–43,45–56 laboratory data in 43/49, 9–28,30–38,41–45,48–56 and vital signs in 36/49. 8–12,14,15,17–22,24,25,27–31,33,34,36,38,41–43,46,48–51,53–56 Imaging was less common, with CT used in 21/49 studies 9,10,12–14,16–18,21–24,26,37,40,42,45,46,48,50,53 and CXR in 16/49; 11,13,15,19,20,27,29,30,34–36,38,51,54–56 clinical notes appeared in 6/49 studies. 8,13,32,47,49,52 The dominant pattern was therefore structured clinical multimodality, with imaging and text used in a minority of models. Study-level modality mappings are visible in Table 1 and fully itemised in the Supplementary Material. Fusion strategies were unevenly distributed. Early fusion was the primary strategy in 29/49 studies, 11,15–17,19,22,24,25,27,28,32,33,35,36,39–51,53,54 followed by late fusion in 9/49 18,20,21,26,29,34,52,55,56 and hybrid fusion in 6/49. 8,10,23,31,37,38 Graph-based fusion was reported in 2/49 studies, 9,12 ensemble/no true fusion in 2/49, 14,30 and attention fusion in 1/49. 13 This suggests that many studies treated multimodality mainly as feature aggregation, while explicit cross-modal architectures were uncommon. The full study-by-study fusion coding is shown in Table 1 and the Supplementary Material. These findings do not show that more complex fusion is superior; rather, fusion design remains under-evaluated. Future studies should compare architectures under missing, delayed, and low-quality data.

4.4. Outcome Taxonomy and Prognosis Alignment

The outcome taxonomy contained 88 outcome-category assignments because studies could report more than one endpoint (Figure 3). Mortality/death was the most frequent category, with both directly aligned mortality endpoints 13,15,16,20,21,26,27,29,35–38,52–54 and broader or more indirectly aligned mortality outcomes. 9,10,18,28,30,34,41,42,46,48,49 Severity progression or composite deterioration was also common, 11,16,18–20,22–25,29,38–40,42,44,48,51,55,56 followed by ICU admission or escalation 11–15,17,20,29,35–38,42,45,52,53 and mechanical ventilation, intubation, or ECMO. 13,20,21,26,28,29,33,35–38,41,50,51,54 Respiratory failure, oxygen requirement, sequelae, and post-discharge outcomes were less frequent. 14,19,21,27,43,47 Directly aligned outcomes were most evident for ICU/escalation, ventilation, and acute respiratory failure endpoints, whereas partial alignment was more common for broad mortality categories and longer-term outcomes. This distinction is important because a model predicting broad in-hospital death is not equivalent to a model designed to support early escalation decisions on a ward. Full outcome-category source mappings are provided in the Supplementary Material.

4.5. Validation Maturity and Implementation Readiness Signals

Validation maturity was low to moderate (Figure 4). Multicentre data were reported in 30/49 studies 10,11,13,16–18,21–27,30–32,34,35,37–40,44,45,51–56 and external validation in 19/49. 11–13,16,17,21,24,26,27,30,33–35,37–39,44,55,56 Missing-modality handling was described in 24/49 studies, 10,11,13,17,18,20,21,23–25,27,28,31–34,36,38,40,43,49,51,52,54 but often as technical imputation rather than as a deployment scenario involving delayed, unavailable, or poor-quality modalities. Calibration was reported in 9/49 studies, 14,15,19–21,23,27,49,52 temporal validation in 5/49, 18,20,25,50,54 decision curve analysis in 3/49, 19,26,46 and prospective evaluation in 3/49. 9,10,20 Validation maturity scores ranged from 0 to 4 out of 7, were concentrated at 0 to 3 points, and no study scored above 4. Full study-level mappings are provided in the Supplementary Material.
Implementation-readiness reporting was even less frequent. Clinical benchmark comparison was reported in 1/49 studies; 21 AUPRC in 4/49; 12,13,20,49 predictive uncertainty quantification in 3/49; 41,45,51 workflow integration in 8/49 studies; 10,13–15,20,54–56 real-time modelling in 3/49; 20,41,55 and regulatory discussion in 2/49. 27,35 These gaps matter for clinical adoption because a deterioration model must be calibrated, interpretable, available at the right time, connected to an escalation pathway, and safe when required information is missing or uncertain. Full implementation readiness mappings are provided in the Supplementary Material. External validation alone is therefore insufficient evidence for adoption if calibration, thresholds, comparator testing, or response pathways are unclear. We treated validation maturity as transportability evidence and implementation readiness as evidence that a hospital team could understand, monitor, and use the model.

4.6. Incremental Benefit of Multimodality

A primary Δ AUROC summary value was computable in 19/49 studies with one comparable multimodal-versus-best-unimodal contrast per study (Figure 5). The median Δ AUROC was 0.027 (IQR 0.017–0.081), with a range from -0.085 to 0.233. Three studies reported negative incremental AUROC values. 42,49,51 Because matched multimodal and best-unimodal AUROCs were available in only a minority of studies, this analysis reflects both performance and reporting completeness. The 19 values used in the primary summary distribution are labelled in Figure 5; additional endpoint-specific, site-specific, horizon-specific, index-augmentation, or otherwise descriptive Δ AUROC values are reported separately in the Supplementary Material.

4.7. Explainability, Human Oversight, and Governance Reporting

Explainability was reported in 32/49 studies, but usually as technical feature importance, saliency, or visual explanation rather than as clinically tested risk communication (Figure 6). Study-level explainability mappings are provided in the Supplementary Material. No study clinically validated the impact of explanations, and only 1/49 reported human or radiologist review of explanations. 19 Predictive uncertainty was quantified in 3/49 studies 41,45,51 and was not used for clinical flagging in any study. Leakage risk was noted in 10/49, 8,9,21–23,27,28,31,32,50 while regulatory or governance discussion was reported in 2/49. 27,35 This pattern shows a difference between producing explanations and designing explainable, clinician-reviewable systems. These findings support framing inclusivity as clinician inclusion and co-design, not only demographic fairness. The evidence base rarely described how doctors, radiologists, nurses, or governance teams would review model outputs, challenge unsafe predictions, or decide when a prediction should be withheld.

4.8. Risk of Bias and Applicability Concerns

Overall risk of bias was high in 47/49 studies and unclear in 2/49; 21,27 no study was low risk overall (Table 2). Low indicates adequate reporting with no major identified concern, unclear indicates insufficient or ambiguous reporting, and high indicates design, analysis, timing, validation, or applicability concerns likely to affect interpretation. The analysis domain was the main driver of high risk, with frequent concerns about internal-only validation, unclear predictor timing, possible leakage, incomplete missing-data reporting, limited calibration, and sparse clinical utility assessment. 8,9,21–23,27,28,31,32,50 Applicability concerns were also common: 33/49 studies had high overall applicability concern, 8–10,15,19,25–52 often because of COVID-heavy cohorts, partial outcome alignment, unclear pneumonia definitions, or limited reporting of intended clinical use. Study-level risk-of-bias and applicability judgements are provided in the Supplementary Material. These assessments should not suggest that the reviewed models lack value; instead, many studies provided insufficient detail to assess safety, applicability, and clinical relevance. In deterioration prediction, vague index times, predictor windows, repeated measures, or future data may inflate performance estimates. Pandemic-era models may also transfer poorly to routine wards, non-COVID pneumonia, or hospitals with different escalation practices.
Table 2. Risk of bias and applicability summary.
Table 2. Risk of bias and applicability summary.
Judgement domain Low Unclear High
Participants risk of bias 3 41 5
Predictors risk of bias 3 43 3
Outcome risk of bias 3 38 8
Analysis risk of bias 0 2 47
Overall risk of bias 0 2 47
Overall applicability concern 2 14 33

5. Discussion

5.1. Principal Findings

This review shows that multimodal AI for pneumonia adverse outcome prediction is active but translationally immature. The field is dominated by structured EHR, laboratory, and vital-sign data rather than image-plus-EHR models, and early fusion remains the main architecture while attention-based and graph-based designs are rare (Figure 2; Table 1). The evidence base also lacks several features needed for clinical adoption: complete unimodal baselines, calibration, AUPRC, clinical benchmarking, predictive uncertainty handling, workflow evaluation, governance, and clinician review (Figure 4–6). The central message is that multimodality should be treated as a health-system intervention, not only as a modelling choice. Adding modalities changes data engineering, alerting, responsibility, interpretation, and monitoring. Adoption therefore depends on whether clinical and safety teams can understand when the prediction is made, what evidence supports it, how predictive uncertainty is handled, and what action should follow.

5.2. Implications for Clinical Adoption

Clinical adoption requires moving from “high AUROC” to safe decision support in a real pneumonia pathway. Models should predict at defined decision points, use data available at those time points, remain calibrated when case mix changes, and produce outputs that clinicians can interpret and contest. This is especially important in deterioration prediction, where false reassurance, unnecessary escalation, and alert fatigue can harm care. Operational feasibility is equally important. A pneumonia prognosis system may depend on laboratory interfaces, imaging archives, EHR observations, and alerting tools. These dependencies determine whether missing modalities block prediction, who receives the output, and whether the model complements existing escalation scores. The COVID-heavy evidence base also limits applicability to broader community-acquired, hospital-acquired, and mixed-aetiology pneumonia pathways.

5.3. Explainability, Responsible AI, and Clinical Teams

Explainability should support risk communication rather than simply produce saliency maps or feature plots. Clinicians need to know why a patient was flagged, which data streams were influential, whether any modality was missing or low quality, and how uncertain the prediction was. In this review, technical explainability was common, but clinical validation of explanations, human review, and predictive-uncertainty flagging were rare (Figure 6). Clinician involvement should therefore start before modelling. 6,58 Doctors, radiologists, nurses, and operational leads should help define the prediction horizon, escalation action, acceptable alert burden, and minimum evidence required before a model can influence care. Future studies should also evaluate fairness across sites, demographic groups, disease subtypes, imaging quality, and data-availability patterns. 5,6 Our related work illustrates the type of uncertainty-aware and auditable logic that may be needed, using selective abstention and audit logs to avoid forcing unreliable predictions into clinical decisions. 61

5.4. Benchmarking and Added Value

The Δ AUROC findings should be interpreted as an added-value signal, not as proof that multimodality is automatically superior. Although several multimodal models improved over the best unimodal comparator, the median gain was small and three studies reported negative incremental AUROC values. 42,49,51 These negative values are important because they show that added modalities can introduce noise, instability, or redundant information as well as complementary signal. More importantly, most studies did not report the matched unimodal baselines needed to judge whether each added modality justified its engineering, governance, and workflow cost. Clinical benchmarking was also rare, yet multimodal AI will be compared with existing scores, local escalation rules, and clinician judgement. NEWS2, 4 CURB-65, 3 the Pneumonia Severity Index, 2 SOFA/qSOFA, 62,63 APACHE II, 64 and SAPS II 65 may not fit every setting, but they provide useful comparators. AUPRC should also be reported more consistently, especially for imbalanced deterioration outcomes; only four studies reported it. 12,13,20,49 A model should therefore be judged not only by AUROC, but by calibration, net benefit, threshold behaviour, availability of required modalities, and whether the prediction can trigger a safe clinical action. The negative Δ AUROC findings also caution against assuming that an imaging branch will automatically improve a multimodal prognosis model. Variable image quality, acquisition artefacts, and preprocessing choices are plausible sources of instability; related work showed that degraded radiographs and preprocessing choices can affect model robustness. 66 This reinforces the need to report image-quality handling, predictive uncertainty, and fallback logic when imaging is added to pneumonia deterioration models.

5.5. Recommendations for Future Studies

Future multimodal pneumonia prognosis studies should be designed as clinically testable deterioration pathways rather than as retrospective model demonstrations. The starting point should be a clearly specified decision context: emergency triage, ward escalation, ICU transfer, ventilation risk, discharge planning, or post-discharge monitoring. Each study should state the index time, prediction horizon, outcome window, intended user, intended action, and exact data available at that moment. This is where the next phase of work should build reproducible pneumonia-specific cohorts that separate COVID and non-COVID disease, preserve temporal ordering, define pneumonia consistently, and distinguish direct deterioration endpoints from broader or partially aligned prognostic outcomes. Future datasets should allow structured-only, image-only, text-only, and fused models to be compared on the same patients. 67
The next generation of studies should first establish strong clinical and unimodal baselines before claiming multimodal benefit. Vitals, laboratory trajectories, demographics, comorbidity, and routinely available EHR variables should form transparent structured baselines. These should be compared not only with the best unimodal AI model, but also with established or locally used clinical comparators such as NEWS2, 4 CURB-65, 3 the Pneumonia Severity Index, 2 SOFA/qSOFA, 62,63 APACHE II, 64 SAPS II, 65 local escalation criteria, and clinician judgement where feasible. Added modalities should then be required to earn their place by improving calibration, AUPRC, threshold behaviour, decision-curve net benefit, or operational usefulness, not only AUROC. CXR, CT, and clinical text should be treated as conditional signals rather than automatically beneficial inputs. For imaging, studies should report view position, timing, acquisition quality, preprocessing, segmentation or enhancement logic, missing images, and whether image-quality failure triggers abstention, fallback, or human review. For clinical notes or radiology reports, studies should define the text source, timestamp, leakage controls, and whether the text would genuinely be available at the intended prediction time. 68 Fusion should be evaluated modularly by comparing early, late, hybrid, graph-based, and attention-based approaches under complete-data, delayed-data, and missing-modality scenarios. This would show whether fusion is learning complementary clinical information or simply benefiting from larger feature sets. Future studies should also move beyond internal retrospective discrimination. External and temporal validation, calibration assessment, AUPRC, decision curve analysis, subgroup performance, and prospective silent evaluation should become routine. 67,69 Silent evaluation is particularly important because it can measure alert burden, workflow fit, fairness, calibration drift, override patterns, and safety events before a model affects care. 69 Explainability should be tested as clinician-facing risk communication, not only as feature importance or saliency. Predictive uncertainty should support abstention, escalation for review, or fallback to simpler models. Governance reporting should include leakage checks, audit logs, monitoring plans, model-updating rules, accountability, and post-deployment surveillance. 70,71 This agenda aligns with TRIPOD+AI, 5 DECIDE-AI, 6 and CONSORT-AI where trial reporting is relevant, 58 and provides a practical route from this review to implementation-ready multimodal pneumonia AI. These priorities are summarised in Table 3.
Table 3. Reporting priorities for future studies.
Table 3. Reporting priorities for future studies.
Priority Minimum reporting expectation
Clinical task definition Index time, prediction horizon, outcome window, intended user, and intended action.
Added-value testing Best unimodal baseline, clinical benchmark, AUROC, AUPRC, calibration, and decision curves.
Generalisation External and temporal validation, cohort subtype, site mix, and COVID/non-COVID status.
Explainable risk communication Explanation method, clinician interpretation, predictive uncertainty display, and explanation validation.
Human oversight and co-design Clinician involvement, review workflow, escalation pathway, and override or abstention rules.
Governance and monitoring Leakage checks, missing-modality logic, prospective silent evaluation, safety monitoring, and regulatory discussion.

5.6. Strengths and Limitations

This review focuses on multimodal adverse outcome prediction in pneumonia, distinguishes prognosis and deterioration from image classification, separates modality composition from fusion architecture, and maps validation, added-value, implementation, explainability, and governance gaps. It is supported by detailed supplementary extraction and academic and clinical co-author review of eligibility and interpretation. Limitations include single-reviewer-led screening and extraction, although the final set was re-reviewed and checked by co-authors. Eligibility required judgement because multimodality and fusion terminology were inconsistent. The synthesis was descriptive rather than meta-analytic because cohorts, outcomes, modalities, horizons, and validation designs were heterogeneous. Δ AUROC was computable only where suitable unimodal baselines were reported, and the COVID-heavy evidence base limits generalisability.

6. Conclusions

Multimodal AI for pneumonia adverse outcome prediction is promising, but the current evidence base remains insufficiently mature for confident clinical adoption. Most studies focused on retrospective model development, while fewer reported the elements hospitals need before implementation: calibration, AUPRC, clinical benchmark comparison, temporal validation, prospective evaluation, explainable risk communication, predictive uncertainty, clinician oversight, workflow integration, and governance. Future studies should define the clinical task clearly, including index time, prediction horizon, intended user, and intended action; compare multimodal models against both unimodal and clinical baselines; validate across sites and time; and describe how predictions, explanations, predictive uncertainty, and missing-modality logic would operate within pneumonia deterioration pathways. Multimodal modelling may add value, but future work needs to show that this value is reliable, interpretable, and actionable in real clinical settings.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org. Supplementary material includes the full search strategy summary, row-level study extraction, outcome taxonomy, incremental benefit values, validation maturity coding, implementation readiness signals, explainability and governance reporting, risk-of-bias assessment, exclusion summary, figure alt text, and supplementary data tables.

Author Contributions

W.H. contributed conceptualisation, methodology, investigation, data curation, formal analysis, visualisation, writing of the original draft, and writing, review, and editing. S.F. contributed conceptualisation, methodology, supervision, validation, and writing, review, and editing. A.E. contributed methodology, supervision, validation, and writing, review, and editing. B.B. contributed clinical validation, clinical interpretation, and writing, review, and editing. A.R. contributed clinical validation, clinical interpretation, and writing, review, and editing. All authors reviewed and approved the final manuscript.

Funding

This systematic review forms part of the PhD+ programme, funded by Aston University and Sandwell and West Birmingham Hospitals NHS Trust.

Data Availability Statement

The data underlying this article are provided in the Supplementary Material. These supplementary data include the extracted study-level variables, risk-of-bias and applicability judgements, exclusion reasons, outcome taxonomy, incremental benefit calculations, validation maturity coding, implementation readiness coding, explainability and governance reporting variables, figure alt text, and summary tables. No patient-level data were generated or analysed in this review.

Conflicts of Interest

The authors declare no competing interests.

References

  1. Sirota, S.B.; Bender, R.G.; Villanueva Dominguez, R.-M.; et al. Global burden of lower respiratory infections and aetiologies, 1990–2023: a systematic analysis for the Global Burden of Disease Study 2023. Lancet Infect. Dis. 2026, 26(4), 343–361. [Google Scholar] [CrossRef]
  2. Fine, M.J.; Auble, T.E.; Yealy, D.M.; et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl. J. Med. 1997, 336(4), 243–250. [Google Scholar] [CrossRef] [PubMed]
  3. Lim, W.S.; van der Eerden, M.M.; Laing, R.; et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax 2003, 58(5), 377–382. [Google Scholar] [CrossRef] [PubMed]
  4. Royal College of Physicians. National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS; Royal College of Physicians: London, 2017. [Google Scholar]
  5. Collins, G.S.; Dhiman, P.; Ma, J.; et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024, 385, e078378. [Google Scholar] [CrossRef] [PubMed]
  6. Vasey, B.; Nagendran, M.; Campbell, B.; et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 2022, 377, e070904. [Google Scholar] [CrossRef] [PubMed]
  7. Baltrušaitis, T.; Ahuja, C.; Morency, L.P. Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 41(2), 423–443. [Google Scholar] [CrossRef] [PubMed]
  8. Mugisha, C.; Paik, I. Pneumonia Outcome Prediction Using Structured And Unstructured Data From EHR. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); IEEE; Volume 2020, pp. 2640–2646. [CrossRef]
  9. Xing, W.; Li, M.; Liu, Y.; et al. Multi-Omics Graph Knowledge Representation for Pneumonia Prognostic Prediction. IEEE J. BioMed Health Inf. 2025, 29(4), 3021–3034. [Google Scholar] [CrossRef] [PubMed]
  10. Hong, Y.; Chen, L.; Yu, Y.; et al. Deep learning integration of chest computed tomography and plasma proteomics to identify novel aspects of severe COVID-19 pneumonia. J. Intensive Med. 2025, 5(3), 252–261. [Google Scholar] [CrossRef] [PubMed]
  11. Lizzi, F.; Brero, F.; Fantacci, M.E.; et al. A Multi-input Deep Learning Model to Classify COVID-19 Pneumonia Severity from Imaging and Clinical Data. In Lecture Notes in Computer Science; Springer, 2024; Volume 14849, pp. 237–248. [Google Scholar] [CrossRef]
  12. Keicher, M.; Burwinkel, H.; Bani-Harouni, D.; et al. Multimodal graph attention network for COVID-19 outcome prediction. Sci. Rep. 2023, 13(1), 19539. [Google Scholar] [CrossRef] [PubMed]
  13. Zhou, H.Y.; Yu, Y.; Wang, C.; et al. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nat. BioMed Eng. 2023, 7(6), 743–755. [Google Scholar] [CrossRef] [PubMed]
  14. Zhou, Y.; Feng, J.; Mei, S.; et al. A deep learning model for predicting COVID-19 ARDS in critically ill patients. Front Med. 2023, 10, 1221711. [Google Scholar] [CrossRef] [PubMed]
  15. Butler, L.; Karabayir, I.; Tootooni, M.S.; et al. Image and structured data analysis for prognostication of health outcomes in patients presenting to the ED during the COVID-19 pandemic. Int. J. Med. Inf. 2022, 158, 104662. [Google Scholar] [CrossRef] [PubMed]
  16. Chassagnon, G.; Vakalopoulou, M.; Battistella, E.; et al. AI-driven quantification, staging and outcome prediction of COVID-19 pneumonia. Med. Image Anal. 2021, 67, 101860. [Google Scholar] [CrossRef] [PubMed]
  17. Chao, H.; Fang, X.; Zhang, J.; et al. Integrative analysis for COVID-19 patient outcome prediction. Med. Image Anal. 2021, 67, 101844. [Google Scholar] [CrossRef] [PubMed]
  18. Ning, W.; Lei, S.; Yang, J.; et al. Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning. Nat. BioMed Eng. 2020, 4(12), 1197–1207. [Google Scholar] [CrossRef] [PubMed]
  19. Li, Y.; Liao, L.; Hu, X.; et al. AI-driven prediction of severe respiratory sequelae in COVID-19 patients. Ann. Med. 2025, 57(1), 2598914. [Google Scholar] [CrossRef] [PubMed]
  20. Shamout, F.E.; Shen, Y.; Wu, N.; et al. An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department. npj Digit Med. 2021, 4(1), 80. [Google Scholar] [CrossRef] [PubMed]
  21. Lassau, N.; Ammari, S.; Chouzenoux, E.; et al. Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients. Nat. Commun. 2021, 12(1), 634. [Google Scholar] [CrossRef] [PubMed]
  22. Zheng, B.; Cai, Y.; Zeng, F.; et al. An Interpretable Model-Based Prediction of Severity and Crucial Factors in Patients with COVID-19. BioMed Res. Int. 2021, 2021, 8840835. [Google Scholar] [CrossRef] [PubMed]
  23. Purkayastha, S.; Xiao, Y.; Jiao, Z.; et al. Machine Learning-Based Prediction of COVID-19 Severity and Progression to Critical Illness Using CT Imaging and Clinical Data. Korean J. Radiol. 2021, 22(7), 1213–1224. [Google Scholar] [CrossRef] [PubMed]
  24. Fang, C.; Bai, S.; Chen, Q.; et al. Deep learning for predicting COVID-19 malignant progression. Med. Image Anal. 2021, 72, 102096. [Google Scholar] [CrossRef] [PubMed]
  25. Jakob, C.E.M.; Mahajan, U.M.; Oswald, M.; et al. Prediction of COVID-19 deterioration in high-risk patients at diagnosis: an early warning score for advanced COVID-19 developed by machine learning. Infection 2022, 50(2), 359–370. [Google Scholar] [CrossRef] [PubMed]
  26. Hiremath, A.; Bera, K.; Yuan, L.; et al. Integrated Clinical and CT Based Artificial Intelligence Nomogram for Predicting Severity and Need for Ventilator Support in COVID-19 Patients: A Multi-Site Study. IEEE J. BioMed Health Inf. 2021, 25(11), 4110–4118. [Google Scholar] [CrossRef] [PubMed]
  27. Dayan, I.; Roth, H.R.; Zhong, A.; et al. Federated learning for predicting clinical outcomes in patients with COVID-19. Nat. Med. 2021, 27(10), 1735–1743. [Google Scholar] [CrossRef] [PubMed]
  28. Pournazari, P.; Spangler, A.L.; Ameer, F.; et al. Cardiac involvement in hospitalized patients with COVID-19 and its incremental value in outcomes prediction. Sci. Rep. 2021, 11(1), 19450. [Google Scholar] [CrossRef] [PubMed]
  29. Mu, E.; Jabbour, S.; Dalca, A.V.; et al. Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration. PLoS ONE 2022, 17(2), e0263922. [Google Scholar] [CrossRef] [PubMed]
  30. Cheng, J.; Sollee, J.; Hsieh, C.; et al. COVID-19 mortality prediction in the intensive care unit with deep learning based on longitudinal chest X-rays and clinical data. Eur. Radiol. 2022, 32(7), 4446–4456. [Google Scholar] [CrossRef] [PubMed]
  31. Girardi, I.; Vagenas, P.; Arcos-Díaz, D.; et al. On the explainability of hospitalization prediction on a large COVID-19 patient dataset. AMIA Annu Symp. Proc. 2021, 2021, 526–535. [Google Scholar] [PubMed]
  32. Whitfield, E.; Coffey, C.; Zhang, H.; et al. Axes of Prognosis: Identifying Subtypes of COVID-19 Outcomes. AMIA Annu Symp. Proc. 2021, 2021, 1198–1207. [Google Scholar] [PubMed]
  33. Bendavid, I.; Statlender, L.; Shvartser, L.; et al. A novel machine learning model to predict respiratory failure and invasive mechanical ventilation in critically ill patients suffering from COVID-19. Sci. Rep. 2022, 12(1), 10573. [Google Scholar] [CrossRef] [PubMed]
  34. Wu, J.T.Y.; de la Hoz, M.A.A.; Kuo, P.C.; et al. Developing and Validating Multi-Modal Models for Mortality Prediction in COVID-19 Patients: a Multi-center Retrospective Study. J. Digit Imaging 2022, 35(6), 1514–1529. [Google Scholar] [CrossRef] [PubMed]
  35. Lee, J.H.; Ahn, J.S.; Chung, M.J.; et al. Development and Validation of a Multimodal-Based Prognosis and Intervention Prediction Model for COVID-19 Patients in a Multicenter Cohort. Sensors 2022, 22(13), 5007. [Google Scholar] [CrossRef] [PubMed]
  36. Walston, S.L.; Matsumoto, T.; Miki, Y.; et al. Artificial intelligence-based model for COVID-19 prognosis incorporating chest radiographs and clinical data; a retrospective model development and validation study. Br. J. Radiol. 2022, 95(1140), 20220058. [Google Scholar] [CrossRef] [PubMed]
  37. Di Napoli, A.; Tagliente, E.; Pasquini, L.; et al. 3D CT-Inclusive Deep-Learning Model to Predict Mortality, ICU Admittance, and Intubation in COVID-19 Patients. J. Digit Imaging 2023, 36(2), 603–616. [Google Scholar] [CrossRef] [PubMed]
  38. Guarrasi, V.; Soda, P. Multi-objective optimisation determines when, which and how to fuse deep networks: An application to predict COVID-19 outcomes. Comput Biol. Med. 2023, 154, 106625. [Google Scholar] [CrossRef] [PubMed]
  39. Dimitsaki, S.; Gavriilidis, G.I.; Dimitriadis, V.K.; et al. Benchmarking of Machine Learning classifiers on plasma proteomic for COVID-19 severity prediction through interpretable artificial intelligence. Artif. Intell. Med. 2023, 137, 102490. [Google Scholar] [CrossRef] [PubMed]
  40. Min, K.; Cheng, Z.; Liu, J.; et al. Early-stage predictors of deterioration among 3145 nonsevere SARS-CoV-2-infected people community-isolated in Wuhan, China: A combination of machine learning algorithms and competing risk survival analyses. J. Evid. Based Med. 2023, 16(2), 166–177. [Google Scholar] [CrossRef] [PubMed]
  41. Charkoftaki, G.; Aalizadeh, R.; Santos-Neto, A.; et al. An AI-powered patient triage platform for future viral outbreaks using COVID-19 as a disease model. Hum. Genom. 2023, 17(1), 80. [Google Scholar] [CrossRef] [PubMed]
  42. Tehrani, S.S.M.; Zarvani, M.; Amiri, P.; et al. Visual transformer and deep CNN prediction of high-risk COVID-19 infected patients using fusion of CT images and clinical data. BMC Med. Inf. Decis. Mak. 2023, 23(1), 265. [Google Scholar] [CrossRef] [PubMed]
  43. Honchar, O.; Ashcheulova, T.; Chumachenko, T.; et al. A prognostic model and pre-discharge predictors of post-COVID-19 syndrome after hospitalization for SARS-CoV-2 infection. Front Public Health 2023, 11, 1276211. [Google Scholar] [CrossRef] [PubMed]
  44. He, X.; Cui, X.; Zhao, Z.; et al. A generalizable and easy-to-use COVID-19 stratification model for the next pandemic via immune-phenotyping and machine learning. Front Immunol. 2024, 15, 1372539. [Google Scholar] [CrossRef] [PubMed]
  45. Zhan, X.; Xu, Q.; Zheng, Y.; et al. Reliability-enhanced data cleaning in biomedical machine learning using inductive conformal prediction. PLoS Comput Biol. 2025, 21(2), e1012803. [Google Scholar] [CrossRef] [PubMed]
  46. Maghsoudi, M.R.; Alirezaei, A.; Soltanzadi, A.; et al. Prognostication and integration of bedside lung ultrasound and computed tomography imaging findings with clinical features to predict COVID-19 in-hospital mortality and ICU admission. Emerg. Radiol. 2025, 32(2), 255–266. [Google Scholar] [CrossRef] [PubMed]
  47. García-Blanco, A.; Mirabella, A.G.; Román-Villarán, E.; et al. FLANDERS: Fast Learning COVID-19 Care System. Stud. Health Technol. Inf. 2025, 327, 599–600. [Google Scholar] [CrossRef] [PubMed]
  48. Del Tedesco Júnior, W.L.; Danelli, T.; Tano, Z.N.; et al. Precision medicine and machine learning to predict critical disease and death due to Coronavirus disease 2019 (COVID-19). Hum. Immunol. 2025, 86(4), 111337. [Google Scholar] [CrossRef] [PubMed]
  49. da Silva, R.P.; Pazin-Filho, A. The incremental value of unstructured data via natural language processing in machine learning-based COVID-19 mortality prediction: a comparative study. BMC Med. Inf. Decis. Mak. 2025, 25(1), 333. [Google Scholar] [CrossRef] [PubMed]
  50. Mirus, M.; Leitert, E.; Bockholt, R.; et al. Radiomics-enhanced modelling approach for predicting the need for ECMO in ARDS patients: a retrospective cohort study. Sci. Rep. 2025, 15(1), 34120. [Google Scholar] [CrossRef] [PubMed]
  51. Olar, A.; Biricz, A.; Bedőházi, Z.; et al. Automated prediction of COVID-19 severity upon admission by chest X-ray images and clinical metadata aiming at accuracy and explainability. Sci. Rep. 2023, 13(1), 4226. [Google Scholar] [CrossRef] [PubMed]
  52. Dipaola, F.; Gatti, M.; Giaj Levra, A.; et al. Multimodal deep learning for COVID-19 prognosis prediction in the emergency department: a bi-centric study. Sci. Rep. 2023, 13(1), 10868. [Google Scholar] [CrossRef] [PubMed]
  53. Gong, K.; Wu, D.; Arru, C.D.; et al. A multi-center study of COVID-19 patient prognosis using deep learning-based CT image analysis and electronic health records. Eur. J. Radiol. 2021, 139, 109583. [Google Scholar] [CrossRef] [PubMed]
  54. Kwon, Y.J.; Toussie, D.; Finkelstein, M.; et al. Combining Initial Radiographs and Clinical Variables Improves Deep Learning Prognostication in Patients with COVID-19 from the Emergency Department. Radiol. Artif. Intell. 2021, 3(2), e200098. [Google Scholar] [CrossRef] [PubMed]
  55. Kim, C.K.; Choi, J.W.; Jiao, Z.; et al. An automated COVID-19 triage pipeline using artificial intelligence based on chest radiographs and clinical data. npj Digit Med. 2022, 5(1), 5. [Google Scholar] [CrossRef] [PubMed]
  56. Jiao, Z.; Choi, J.W.; Halsey, K.; et al. Prognostication of patients with COVID-19 using artificial intelligence based on chest x-rays and clinical data: a retrospective study. Lancet Digit Health 2021, 3(5), e286–e294. [Google Scholar] [CrossRef] [PubMed]
  57. Wolff, R.F.; Moons, K.G.M.; Riley, R.D.; et al. PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Ann. Intern Med. 2019, 170(1), 51–58. [Google Scholar] [CrossRef] [PubMed]
  58. Liu, X.; Rivera, S.C.; Moher, D.; et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. BMJ 2020, 370, m3164. [Google Scholar] [CrossRef] [PubMed]
  59. Harib, W.; Fouad, S. PROSPERO CRD420261328187; Multimodal Artificial Intelligence Models for Predicting Clinical Deterioration in Patients with Pneumonia: A Systematic Review. 2026.
  60. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef] [PubMed]
  61. Harib, W.; Fouad, S.; Mahmoud, T.F. A Dirichlet Distribution-Based Trust-Adaptive Ensemble Approach for Pneumonia Classification from Chest X-Ray Images. In 2026 IEEE 23rd International Symposium on Biomedical Imaging (ISBI); IEEE: London, United Kingdom, 2026; pp. 1–4. [Google Scholar] [CrossRef]
  62. Vincent, J.L.; Moreno, R.; Takala, J.; et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996, 22(7), 707–710. [Google Scholar] [CrossRef] [PubMed]
  63. Seymour, C.W.; Liu, V.X.; Iwashyna, T.J.; et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016, 315(8), 762–774. [Google Scholar] [CrossRef] [PubMed]
  64. Knaus, W.A.; Draper, E.A.; Wagner, D.P.; et al. APACHE II: a severity of disease classification system. Crit. Care Med. 1985, 13(10), 818–829. [Google Scholar] [CrossRef]
  65. Le Gall, J.R.; Lemeshow, S.; Saulnier, F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993, 270(24), 2957–2963. [Google Scholar] [CrossRef]
  66. Harib, W.; Fouad, S.; Ekart, A.; et al. Evaluating Pneumonia Classification Robustness Across Architectures Under Variable-Quality Chest Radiographs. In Artificial Intelligence in Healthcare. AIiH 2026. Lecture Notes in Computer Science; Ni, H., Cafolla, D., Eds.; Springer: Cham; vol 16876 2027, pp. 17–31. [CrossRef]
  67. Yang, C.; Kors, J.A.; Ioannou, S.; et al. Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review. J. Am. Med. Inf. Assoc. 2022, 29(5), 983–989. [Google Scholar] [CrossRef] [PubMed]
  68. Seinen, T.M.; Fridgeirsson, E.A.; Ioannou, S.; et al. Use of unstructured text in prognostic clinical prediction models: a systematic review. J. Am. Med. Inf. Assoc. 2022, 29(7), 1292–1302. [Google Scholar] [CrossRef] [PubMed]
  69. van der Vegt, A.H.; Scott, I.A.; Dermawan, K.; et al. Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. J. Am. Med. Inf. Assoc. 2023, 30(7), 1349–1361. [Google Scholar] [CrossRef] [PubMed]
  70. Labkoff, S.; Oladimeji, B.; Kannry, J.; et al. Toward a responsible future: recommendations for AI-enabled clinical decision support. J. Am. Med. Inf. Assoc. 2024, 31(11), 2730–2739. [Google Scholar] [CrossRef] [PubMed]
  71. Davis, S.E.; Embi, P.J.; Matheny, M.E. Sustainable deployment of clinical prediction tools—a 360° approach to model maintenance. J. Am. Med. Inf. Assoc. 2024, 31(5), 1195–1202. [Google Scholar] [CrossRef] [PubMed]
Preprints 228990 g001
Figure 1. PRISMA flow diagram.
Figure 1. PRISMA flow diagram.
Preprints 228990 g001
Preprints 228990 g002
Figure 2. Modality and fusion map.
Figure 2. Modality and fusion map.
Preprints 228990 g002
Preprints 228990 g003
Figure 3. Outcome taxonomy.
Figure 3. Outcome taxonomy.
Preprints 228990 g003
Preprints 228990 g004
Figure 4. Validation and implementation readiness.
Figure 4. Validation and implementation readiness.
Preprints 228990 g004
Preprints 228990 g005
Figure 5. Incremental AUROC.
Figure 5. Incremental AUROC.
Preprints 228990 g005
Preprints 228990 g006
Figure 6. Explainability and governance reporting.
Figure 6. Explainability and governance reporting.
Preprints 228990 g006
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.