Rethinking the Evaluation of Generative AI in Clinical Diagnosis and Prognosis

1. Introduction

Generative artificial intelligence (AI) extends beyond traditional medical AI systems. Previous research has predominantly used AI algorithms for discriminative tasks, including medical image classification and patient risk stratification. In contrast, generative AI enables both discriminative tasks and generative clinical applications, producing new clinical data or content instead of only classifying existing data. Unlike conventional machine learning (ML) models constrained by narrow pattern recognition, generative AI systems can synthesize diverse data sources, such as structured patient histories and unstructured clinical notes, to perform clinical reasoning [1]. It generates differential diagnoses, formulates evidence-based treatment recommendations, adapts to individual clinical scenarios, and offers conversational interfaces that integrate into clinical workflows [2]. This progress has been driven primarily by large language models (LLMs), which have demonstrated strong capabilities across medical examinations, patient and clinician education, disease diagnosis [3], and clinical documentation [4]. However, as these systems move toward clinical deployment, their reliability in the real world remains uncertain, mainly due to limitations in current evaluation approaches [5]. Model performance is frequently summarized using accuracy-based metrics, which can obscure errors, including high-confidence incorrect predictions and non-faithful or logically inconsistent reasoning [6]. As a result, models may appear to perform well on benchmark datasets while remaining unsuitable for clinical deployment due to issues such as lack of robustness, which are not captured by standard benchmark metrics [7]. Moreover, current approaches lack standardization and rely heavily on subjective human judgments, making it difficult to assess performance and safety [8]. This fragmented evaluation landscape exacerbates the risk of premature clinical adoption, as researchers lack unified frameworks for deployment. Without rigorous evaluation methodologies, the gap between experimental promise and real clinical reliability will continue to widen.

In this paper, we review existing evaluation metrics for generative AI in clinical settings, with a focus on LLMs, vision language models (VLMs), and multimodal large language models (MLLMs). We focus on diagnosis and prognosis tasks, which involve the synthesis of multiple sources of evidence and clinical reasoning. We exclude administrative documentation and patient education, as these focus on efficiency and communication rather than the synthesis of incomplete evidence and reasoning under the uncertainty that diagnosis and prognosis require. Although other reviews examine the evaluation of generative AI in healthcare, this work addresses gaps through four contributions. First, we restrict our scope to clinical diagnosis, inference of current conditions and prognosis, prediction of future outcomes, providing a targeted framework for these specific applications rather than a broad assessment of AI in healthcare [7,9,10,11,12]. Second, existing reviews predominantly exclude multimodal AI systems [7,13,14]. Third, while existing reviews provide primarily descriptive overviews, we provide a technical introduction to evaluation metrics for the readers, examining their appropriate applications, limitations, and clinical applicability. Finally, we synthesize a comprehensive evaluation framework that covers multiple evaluation dimensions, including anthropomorphism, the attribution of characteristics similar to humans to non-human entities [15]. In clinical AI, behaviors of the anthropomorphic system shape clinician perceptions of accuracy, reliability, and trust [16]. We define clinical anthropomorphism as the extent to which AI systems exhibit human-like communication styles, affective cues, and empathetic responses during clinical interactions. Evaluating this dimension is critical because excessive anthropomorphism may lead to over-reliance or misplaced trust, while insufficient human-like interaction may hinder clinical adoption.

This review is structured as follows: We first describe the literature search strategy and inclusion criteria used to identify recent studies evaluating generative AI systems for clinical diagnosis and prognosis. We then introduce a task-based taxonomy of clinical applications and analyze how task formulation shapes evaluation design and metric choice. Next, we examine evaluation methodologies, distinguishing between automated metrics and human-centered assessment. We classify evaluation approaches in the literature across five core dimensions: Factual and Contextual Correctness, Safety, Alignment, and Robustness, Reasoning Quality, Calibration and Uncertainty Assessment, and Clinical Anthropomorphism. We also review existing medical benchmarks and discuss their limitations in reflecting real-world clinical reasoning, safety, and uncertainty. Finally, we identify key gaps and open challenges and offer recommendations for developing reliable and clinically meaningful evaluation practices for generative AI systems.

2. Search Methodology

We searched Web of Science and PubMed for publications that use generative AI to generate textual outputs for clinical diagnosis and prognosis. The primary search covered studies published between 2022 and 2025, with additional relevant papers identified during manuscript preparation. Appendix A details our search queries. From 2,267 papers, we selected 99 studies with explicit evaluation methodologies (Figure 1) published in peer-reviewed venues. Our inclusion criteria focused on primary research on diagnosis, risk assessment, and prognosis prediction that applied generative AI models (LLMs, VLMs, MLLMs) to generate free-text clinical output. We excluded applications such as administrative documentation and patient education, as these tasks differ from diagnostic decision-making, which involves reasoning under uncertainty and carries a higher risk of patient harm. We also excluded review papers to avoid using secondary analysis. When multiple studies used identical evaluation setups and metrics, representative examples were retained to avoid over-representation of a single evaluation strategy.

3. Models

This review examines three categories of generative AI architectures in clinical applications: LLMs, VLMs, and MLLMs. All three generate natural language outputs for clinical decision support, but use different input modalities. Throughout this review, we refer to these three architectures collectively as generative AI models.

LLMs process exclusively textual input; studies typically linearize structured clinical data into natural language prompts along with task-specific instructions [18]. VLMs process both image and text inputs [19]. MLLMs process multiple modalities, including text, audio, and images simultaneously [20]. (Figure 2) presents an abstracted schematic of these three families of models and their respective input-output characteristics. The applications of VLMs and MLLMs reflect the nature of clinical decision-making, which is often inherently multimodal [21,22]. Practitioners routinely combine radiological images, laboratory results, clinical notes, and patient histories to make decisions. An underexplored challenge in multimodal AI evaluation is assessing whether systems truly integrate multiple modalities or rely on a single dominant modality. Specifically, robust evaluation frameworks should assess whether these architectures integrate cross-modal information or continue to learn only unimodal signals.

4. Taxonomy of Clinical Tasks

Drawing from established frameworks [9,13,25,26] and tasks observed across the reviewed studies, we developed a taxonomy categorizing clinical applications of generative AI into six groups (Table 1). These task groups range from knowledge-retrieval applications, such as Open-ended Clinical Question and Answer (OQA) and Multiple Choice Questions (MCQ), commonly used as proxies for clinical competency in evaluations, to applied clinical workflow tasks requiring evidence synthesis, including Diagnostic Decision Support (DDS), Risk Prediction (RP), Triage and Referral (TR), and Report Generation (RG). Some tasks, such as MCQ, can assess both factual recall and clinical reasoning depending on the design of the question. Each task demands a different evaluation approach based on its clinical complexity. For example, MCQ tasks typically emphasize classification metrics and factual correctness measures. The DDS, RP, TR, and OQA studies also evaluate clinical reasoning quality, uncertainty quantification, adherence to guidelines, and safety considerations. The RG research focuses on text quality and coherence. The TR studies prioritize the model’s ability to rank patient severity. Tasks influence whether evaluation can be fully automated or needs human judgment. MCQs support automatic grading through answer-matching, whereas OQA, DDS, RP, TR, and RG require human-centric evaluation to assess reasoning quality and contextual correctness.

These tasks can be implemented using any of the model architectures described in Section 3. There is no strict one-to-one mapping between the task type and the model family; the modality of the input data primarily drives the architecture choice. Most reviewed studies use LLMs, reflecting the predominance of text-based clinical datasets and evaluation benchmarks. The clinical literature shows substantial overlap between VLMs and MLLMs, with many studies using MLLMs even for tasks that involve only text and image inputs. This direction likely reflects the growing availability of state-of-the-art MLLMs, which are often applied even when fewer modalities are required.

RG relies most often on VLMs and MLLMs because it typically requires interpreting images, such as radiological images, alongside textual clinical information. In contrast, TR, RP, MCQ, and OQA usually use LLMs and, less frequently, require VLMs or MLLMs. DDS covers a broad range of problem types, including text-only, image-based, and multimodal settings, and appears in studies using LLMs, VLMs, and MLLMs.

5. Evaluation

Standard evaluation metrics for discriminative models fail to capture the complexity of generative AI in clinical settings. Whereas discriminative models can often be evaluated using well-defined metrics such as accuracy, generative models produce open-ended textual outputs whose evaluation spans multiple dimensions, including coherence, relevance, and safety. This challenge is particularly pronounced in clinical contexts, where correctness alone is insufficient, and outputs must also be clinically actionable, relevant, and safe [27]. Thus, researchers are increasingly relying on human feedback and expert evaluation alongside automated metrics to assess the quality of system outputs [28]. In this paper, we organize the evaluation approaches into human and automatic methods.

5.1. Automated Evaluation

We categorize automated evaluation approaches for generative AI into four broad classes: (i) classification metrics, (ii) readability metrics, (iii) text similarity and semantic overlap measures, and (iv) LLM-as-a-judge methods, which employ large language models to assess generation quality along higher-level criteria.

5.1.1. Classification Metrics

Clinical tasks using discriminative approaches often rely on automated metrics [30,31,32]. As illustrated in (Table 2), we categorize these metrics into accuracy-, calibration-, and uncertainty-based measures. Classification metrics are particularly prevalent in MCQ [33] and RP [34] tasks, where models select from a predefined set of discrete outcomes, including binary (e.g., risk vs. no risk) or multiclass labels. In practice, researchers often prompt the generative models to provide a classification or use regular expression (Regex) parsing to map the model’s free-text generation onto a specific class label for quantitative evaluation [35,36].

Although Accuracy is widely used, it is unsuitable for datasets with significant imbalance, which are common in medical contexts. The F1-score and Matthews Correlation Coefficient (MCC) [30,37,38] are often used instead as more robust alternatives. F1-score, the harmonic mean of precision and recall, can be reported using micro or macro averaging. Micro-F1 aggregates classification outcomes across all classes prior to computing precision and recall, and therefore reflects prevalence-weighted, sample-level performance. Macro-F1 averages F1 scores across all classes with equal contribution from each class [39,40]. Unlike accuracy, MCC accounts for all four quadrants of the confusion matrix, ensuring that high performance is not the result of a model over-predicting the majority class [41]. Additionally, Likelihood Ratios offer more clinical utility than standard sensitivity by quantifying how much a positive or negative test result shifts the probability of a disease [29].

Top-k Accuracy is the proportion of examples in which the reference label, or any of multiple valid reference labels, appears within the model’s top-k highest-confidence predictions. This evaluation is prevalent in differential diagnosis generation, and TR, where clinical utility derives from presenting a ranked shortlist of plausible diagnoses rather than requiring a single correct prediction [42]. Top-k accuracy reflects clinical workflows where multiple valid diagnostic hypotheses may coexist, and comprehensive differential coverage may be more valuable than precise rank ordering. The Area Under the Receiver Operating Characteristic curve (AUROC) [43] and the Area Under Precision–Recall Curve (AUCPR) [44] are widely used discrimination metrics in healthcare classification tasks. AUCPR is often preferred for imbalanced clinical datasets, such as rare disease diagnosis, because AUROC gives overly optimistic estimates when performance in the minority class is poor [45]. However, recent theoretical analysis shows that the primary difference between AUROC and AUCPR is how they weigh model mistakes [46]. AUROC evaluates model performance uniformly across samples, treating improvements equally regardless of prediction confidence. AUCPR is sensitive to performance improvements in samples assigned high-confidence scores, those that the model predicts with greatest certainty as positive, while demonstrating relative insensitivity to changes in the ranking of lower-confidence predictions. This property has important clinical implications because AUCPR may prioritize model improvement in high-prevalence subpopulations at the expense of lower-prevalence groups, increasing algorithmic unfairness in diverse healthcare settings. Therefore, metric selection should be guided by clinical priorities and population characteristics. AUROC is appropriate for prevalence-invariant model comparisons and for safety-critical applications where false negative errors have severe consequences, such as cancer screening. AUCPR is more suitable in situations where false positives incur substantial costs and distributional fairness is a secondary concern, such as prioritizing high-confidence candidates in early-stage drug discovery pipelines rather than disease diagnosis and prognosis [46]. In multiclass classification, both metrics are typically calculated using aggregation strategies that reduce the problem to multiple binary evaluations [47].

The second category of metrics focuses on Uncertainty Quantification and Calibration. Calibration refers to the alignment between the predicted probabilities of a model and the true frequency of the outcomes and includes metrics such as Brier Score (BS) [38,48,49] and Expected Calibration Error (ECE) [32,50]. Although uncertainty quantification evaluates the model’s ability to express confidence levels in its predictions and includes metrics such as Monte Carlo (MC) Dropout, which obtains uncertainty estimates by performing multiple stochastic forward passes through the network at inference time with dropout mechanisms maintained active and calculating the sample variance across passes [51]. In high-stakes healthcare settings, the ability of a model to be aware of its uncertainty can be as important as its predictive capacity [52,53]. Overconfident but incorrect predictions can undermine clinician trust and lead to inappropriate interventions, while under-confident predictions may result in diagnostic delays. This assessment is vital for safe integration into clinical workflows, where the low confidence expression of an AI can trigger a manual review by a human specialist [54].

5.1.2. Readability Metrics

The readability and linguistic complexity metrics assess how easily diverse audiences can understand AI-generated medical content. To evaluate this, the researchers employ a variety of automated formulas as summarized in (Table 3). The FRE formula estimates the educational level required to comprehend a text [55]. The FKGL [56] is a simplified adaptation of the FRE that calculates the ease score using a grade scale rather than a numerical value from 0 to 100. GFI and Linsear Write Formula (LWF), also estimate grade level but factors in the number of complex words, those with three or more syllables [57,58]. Higher grade levels indicate greater text complexity. The SMOG [59] index calculates how many years of education a reader needs to comprehend a text by focusing on the number of polysyllabic words, defined as words with three or more syllables. ARI [60] and CLI measure complexity in terms of the number of letters per word and words per sentence. The Dale-Chall score [61] and Spache Readability score [62] use sentence length and word familiarity lists. These lists constitute a set of words that are considered difficult because they are not commonly used. These metrics help ensure that text remains accessible to patients and non-specialist clinicians by avoiding excessive medical or technical terminology [58,63].

5.1.3. Text Similarity and Semantic Overlap Measures

Traditional evaluations that use discrete labels for comparison cannot robustly assess open-ended output from tasks such as RG, OQA, and DDS. (Table 4) shows our categorization of text generation metrics into two main types: lexical overlap and embedding-based similarity. These metrics generally compare model output with gold-standard or expert-generated reference texts.

In the first group, lexical overlap, metrics rely on text matching between the model’s output and ground truth texts. Although computationally efficient and widely adopted due to their simplicity, they often do not capture semantic coherence, context, or clinical reasoning quality. They are based on n-gram matching, where n represents the number of consecutive words evaluated as a unit. The Bilingual Evaluation Understudy (BLEU) calculates a weighted geometric mean of n-gram precision scores [64]. Despite its popularity across generation tasks, BLEU often correlates poorly with human judgments in open-ended scenarios, likely because developers originally designed it for machine translation [65]. Exact Match (EM) quantifies the proportion of model predictions that match ground truth labels, providing a strict correctness measure. The F-score can also be used to quantify the overlap between generated and reference outputs by computing precision and recall over text units [66]. Recall-Oriented Understudy for Gisting Evaluation (ROUGE) measures n-gram recall and commonly assesses summarization tasks. ROUGE includes several variants based on the length of n-gram, most commonly measured as unigrams (ROUGE-1), bigrams (ROUGE-2), and the longest common subsequence (ROUGE-L) [67]. METEOR goes beyond word matching by considering synonym matching and morphological stemming, often better correlated with human rankings in contexts such as image captioning [65,68]. Embedding-based similarity measures move beyond surface-level lexical overlap and compare word vector representations of text. A commonly used metric is cosine similarity, which quantifies the semantic similarity between two embedding vectors [69]. BERTScore, illustrated in (Figure 3), computes cosine similarities between contextualized embeddings derived from a BERT model. By leveraging contextual embeddings, BERTScore is more aligned with semantic similarity than n-gram-based metrics, especially in summaries in RG or OQA settings [70].

5.1.4. LLM-as-a-Judge

The LLM-as-a-judge evaluation paradigm [71,72] prompts an LLM to assess the quality of the output of another generative AI model (Figure 4). This method combines the scalability and efficiency of automated evaluation with the potential to capture nuanced, multidimensional expert-like judgments. Building on the framework from [71], LLM-as-a-judge includes three variations: (i) Pairwise comparison: The LLM judge receives a question and two responses, then determines which answer is superior or whether they are equivalent in quality; (ii) Single-answer grading: In this method, the LLM rates a response with a score; (iii) Reference-guided grading: The LLM judge is provided with a gold-standard reference answer and uses it to evaluate the quality and correctness of the model-generated output.

Reviewed studies show a clear shift from simple, binary "Is this accurate?" queries to sophisticated, grounded evaluation frameworks. Instead of relying on unstructured model judgments, researchers often provide evaluator models with explicit rubrics and predefined criteria for assessment [73,74,75]. The choice of prompting strategy varies depending on the evaluation setting. Reference-guided grading is most commonly used when ground-truth answers are available, such as in DDS [74,76], whereas pairwise comparison [73] and single-answer grading are less common than reference-guided grading. Overall, the literature shows a trend toward increasingly structured prompt design that incorporates domain knowledge, explicit criteria, or reference-based grounding to obtain more reliable clinical evaluations.

LLM-as-a-judge can still work without ground-truth references, especially for criteria like conciseness and human-likeness. This advantage, lacking in traditional automated metrics, improves scalability. However, using one LLM to judge another raises concerns about the validity of the judgment and amplifies weaknesses such as prompt dependence, overconfident assessments, and bias of the evaluator’s model. Recent frameworks address these issues with techniques such as multi-source evaluation, in which LLM judgments are aggregated across multiple independent evaluation configurations to improve robustness and reduce stochastic variation. In practice, this often means running repeated evaluations on identical content using different prompts [77], as shown in the multi-round evaluation (Figure 4). To reduce single-model biases, some researchers use multi-model evaluation, in which different LLMs independently evaluate model output, as shown in (Figure 4). For example, some frameworks use voting mechanisms across multiple judge models to determine final evaluation outcomes, while others use peer-review architectures [78] wherein LLMs evaluate each other’s outputs [79].

Automated evaluation methods are often preferred due to their scalability, reproducibility, and suitability for large-scale benchmarking. However, widely used text-similarity measures, such as embedding-based metrics, have been shown to correlate poorly with expert clinical judgments [27]. In comparison, LLM-as-a-judge frameworks have a stronger alignment with human evaluations by assessing higher-level qualitative attributes such as clinical reasoning, coherence, and correctness [72]. This positions LLM-as-a-judge as a promising intermediary evaluation paradigm that partially bridges the gap between scalable automated metrics and expert human assessment.

Despite LLM-as-a-judge’s promise, human evaluation remains essential. Human experts detect subtle medical errors, context-dependent risks, and clinical nuances, such as whether recommendations balance individual risk and benefit. Robust evaluation frameworks increasingly emphasize hybrid strategies that combine automated metrics with human assessment. Some approaches implement human-in-the-loop validation, where expert evaluators review a subset of LLM-as-a-judge outputs, reducing workload while preserving expert oversight and reliability [74].

5.2. Human Evaluation

Despite advances in automated evaluation methods, expert human evaluation remains the gold standard for assessing generative AI in healthcare. Most studies use structured questionnaires or rating scales, in which clinicians assess predefined criteria such as correctness, guideline adherence, safety, empathy, and communicative quality [80,81]. Based on our review, the majority of studies use small panels of domain experts to evaluate the model’s outputs, typically involving one [82] to six clinicians [80,83], who evaluate multiple dimensions beyond correctness, including relevance, completeness, helpfulness, quality of explanations, and safety or guideline adherence. Larger evaluator panels have been reported in a limited number of studies (e.g., 11 or 28 evaluators [84,85]), although this remains rare due to resource constraints. Following the framework established by [65], we organize human evaluation approaches into intrinsic and extrinsic evaluation methods. We also examine commonly used questionnaires in healthcare settings and discuss inter-evaluator agreement metrics. We present a taxonomy of the human-based evaluation criteria most commonly used to evaluate generative AI systems in clinical settings in (Table 5).

5.2.1. Intrinsic Evaluation

Intrinsic evaluation focuses on assessing the quality of the outputs generated independently of any performance of the downstream task [65]. In clinical AI, this typically involves expert clinicians rating model responses across multiple criteria, including explanation quality, using structured rating instruments. The intrinsic evaluation criteria are presented in (Table 5). The Likert scale is the most commonly used metric here, which allows evaluators to assign scores within a specified range [83,86], where higher values indicate a better overall response quality. In our review, 5-point scales emerged as the predominant choice, although some studies employ 7-point scales to capture more nuanced distinctions in quality [87,88]. An alternative approach uses categorical grading schemes with descriptive labels (e.g., "excellent," "good," "acceptable," "poor," "very poor") [83].

5.2.2. Extrinsic Evaluation

Extrinsic human-evaluation assess the impact of a model on downstream clinical tasks. For example, [89] evaluated the performance of ChatGPT in managing postoperative complications after facial esthetic surgery. The study tested the model’s ability to perform clinical tasks, including generating differential diagnoses, gathering relevant patient histories, formulating diagnoses, and recommending appropriate dispositions, ranging from urgent emergency referrals to routine surgical follow-ups. The Extrinsic evaluation criteria are presented in (Table 5). Extrinsic evaluation directly measures the downstream clinical impact and provides a more meaningful assessment of the behavior of the model.

5.2.3. Questionnaires

A wide range of structured questionnaires are used for human evaluation in healthcare generative AI studies, such as DISCERN [90] and modified DISCERN (mDISCERN), Global Quality Score (GQS), JAMA benchmark criteria, and custom clinical relevance or safety rubrics tailored to specific specialties and tasks [91,92,93,94]. DISCERN is a standardized questionnaire originally developed to assess the quality and reliability of written health information with respect to treatment choices. It consists of a structured set of items that evaluate key dimensions, such as the clarity of aims, the relevance and balance of information, the transparency of sources, the discussion of benefits and risks, and the acknowledgement of uncertainty. Each item is typically rated on a Likert scale. In healthcare generative AI evaluation, DISCERN and its adapted variants (e.g., modified DISCERN, [95], which represents a subset of the original DISCERN tool [96]) are widely used in human-based evaluations to assess the reliability, clinical soundness, and overall quality of generated responses [97,98,99,97]. GQS [100] is a shorter instrument than DISCERN that evaluates the overall quality and usefulness of health information. It uses a five-point Likert scale, with scores from 1 (poor) to 5 (excellent). Originally designed for patient-facing health websites, researchers now use GQS to assess generative AI output because of its simplicity, clinical intuitiveness, and ability to summarize usefulness in one score. JAMA benchmarks criteria [101] are based on the evaluation of four key dimensions: (i) transparency of authorship, requiring clear identification of contributors; (ii) attribution, ensuring that all claims are supported by explicit references; (iii) disclosure, mandating full visibility of website ownership and potential conflicts of interest; and (iv) currency, which evaluates whether publication and update dates are clearly reported to reflect the timeliness of the content. This allows researchers to verify whether generative AI models can reference guidelines correctly [97].

5.3. Inter-Rater Agreement

Inter-rater agreement measures the reliability and consistency of expert assessments. Clinical interpretation and evaluation are subjective and can vary with experience, training, and institutional practices, so studies recruit multiple evaluators and report agreement metrics to address this subjectivity. Beyond measuring reliability, inter-rater agreement serves as a valuable indicator of multiple factors such as evaluator expertise, ambiguity or difficulty of the clinical task being evaluated, and the clarity of evaluation criteria and rating scales. Low agreement may signal poorly defined evaluation rubrics, clinical ambiguity requiring a larger number of experts, or insufficient evaluator training. High agreement may suggest well-operationalized assessment dimensions and consistent interpretation of quality standards [65]. The reviewed literature frequently uses Cohen’s kappa, Fleiss’s kappa, and the Intraclass Correlation Coefficient.

5.3.0.1. Cohen’s Kappa ($\kappa$).

It measures the agreement between two evaluators assigning categorical or ordinal labels while correcting for the agreement expected by chance [102]. It is widely used for categorical judgments such as correctness, safety, presence of hallucinations, or adherence to guidelines. The formula for Cohen’s Kappa is shown in Eq. (1), where $P_o$ denotes the observed proportion of agreement between evaluators, and $P_c$ represents the expected agreement by chance. The values of $\kappa$ range from $-1$ to 1, where 1 is perfect agreement between the evaluators, and 0 indicates chance-level agreement.

$$\kappa ={\displaystyle \frac{P_o-P_c}{1-P_c}}$$

(1)

5.3.0.2. Fleiss’s Kappa ($\kappa$).

It extends Cohen’s kappa to scenarios involving more than two evaluators assigning categorical ratings [103]. It is commonly applied in studies that rely on expert panels to assess dimensions such as quality, safety, or clinical appropriateness. The formula for Fleiss’s Kappa is shown in Eq. (2), where $\overline{P}$ denotes the mean observed agreement across all evaluated items, while ${\overline{P}}_e$ represents the mean agreement expected by chance.

$$\kappa ={\displaystyle \frac{\overline{P}-{\overline{P}}_e}{1-{\overline{P}}_e}}$$

(2)

5.3.0.3. Intraclass Correlation Coefficient (ICC).

It is used to assess agreement for continuous or ordinal ratings. ICC is variance-based and quantifies the proportion of total variance attributable to differences between the evaluated items. There are multiple ICC formulations, depending on the experimental design and the definition of agreement; more details can be found in [104].

The above metrics are important when using multiple human raters to ensure alignment with experts. Consequently, transparent reporting of the evaluator scoring procedures and agreement metrics is essential to establish the credibility and reproducibility of human-based evaluation in the generative AI research of healthcare [65]. Although human evaluation is often considered the gold standard, it presents substantial methodological challenges. Evaluators can use crowd-sourcing for general-domain tasks, but clinical evaluation requires domain specialists, which is expensive to obtain. In addition, recruiting evaluators with comparable expertise and speciality training across studies is difficult, and many publications provide insufficient detail about their evaluation protocols, making replication difficult.

Human evaluation can introduce bias into expert assessments of AI-generated output. Human evaluators may show two contrasting forms of bias. First, evaluators can show automation bias by over-relying on or uncritically accepting AI recommendations, overestimating algorithmic capabilities [16,105,106]. Evaluators may also show algorithm aversion, applying overly strict standards to AI outputs due to skepticism about machine reliability or expectations of systematic errors [107]. These biases can compromise the validity of the evaluation, as quality ratings may reflect the prior beliefs of the evaluators about AI rather than objective assessments of clinical suitability and output quality. To mitigate these biases, some studies use blinded assessment, so human evaluators do not know whether outputs come from clinicians or AI systems [108,109]. This approach ensures that evaluators judge quality on the merit of the content rather than the source, isolating the clinical quality assessment from confounding beliefs about technology versus human competence.

Automated metrics, human evaluation, and LLM-as-a-judge approaches each offer distinct advantages, but none alone can evaluate generative AI systems for clinical diagnosis and prognosis. Automated metrics provide scalability and reproducibility, but do not capture clinically critical properties such as reasoning validity, safety, or contextual appropriateness. Human evaluation remains the gold standard for assessing these dimensions, but is limited by subjectivity, restricted reproducibility, and significant resource demands. LLM-as-a-judge frameworks partially address these limitations by enabling scalable assessment of higher-level qualities, but they introduce additional sources of bias and sensitivity to prompts. Evaluators should select and combine strategies based on the clinical risk level, the intended use case, and the maturity of the deployment. Low-risk exploratory tasks can be based on automated metrics, while high-risk diagnostic and prognostic applications require hybrid frameworks that combine expert human oversight with structured automated assessment. In clinical settings, relying on a single evaluation method risks misleading performance estimates and unwarranted confidence in the readiness of the system.

6. A Multi-Dimensional Evaluation Framework

Clinical diagnosis and prognosis require more than an accurate prediction; they require sound reasoning, clear expression of uncertainty, and responsible communication. Current evaluation practices for generative AI remain fragmented, and evaluations primarily focus on accuracy while overlooking essential properties for clinical reliability. To address these gaps, we unify existing methodologies into a five-dimensional evaluation framework and extend them to include under-evaluated aspects, such as reasoning quality. The framework includes: (i) Factual and Contextual Correctness, (ii) Safety, Alignment, and Robustness, (iii) Reasoning Quality, (iv) Calibration and Uncertainty, and (v) Clinical Anthropomorphism (see Figure 5 for a comprehensive taxonomy). Structuring evaluation along these dimensions provides a systematic basis for identifying weaknesses, comparing systems, and assessing readiness for real-world clinical integration.

6.1. Dimension I: Factual and Contextual Correctness

Because model outputs directly influence patient safety and clinical decision-making, factual and contextual correctness constitute a fundamental requirement. The literature (Table 6, Table 7, Table 8) primarily evaluates this dimension using either classification metrics or embedding similarity and semantic overlap measures. For classification tasks such as RP, researchers prompt models to generate discrete binary risk labels and rely on traditional classification metrics, including Accuracy, Recall, Specificity, Precision, and F1-score. For generative tasks such as DDS, RG, or OQA, correctness is evaluated using automated metrics, human evaluation, or a combination of both. Automated methods range from lexical overlap metrics, such as BLEU, METEOR, and ROUGE, to embedding-based metrics such as BERTScore and cosine similarity. In clinical settings, some studies also use medical language embeddings to evaluate whether the generated output accurately reflects domain-specific clinical concepts and terminology [87]. Beyond automated metrics, human evaluation remains a key component of factual correctness assessment, especially for validating clinical correctness, completeness, relevance, and detection of hallucinations. Researchers often systematize these qualitative assessments using a questionnaire or custom Likert scales.

Our review shows that most studies incorporate evaluation criteria from this dimension, either alone or with others. This is expected since factual correctness is the primary prerequisite for the validity and reliability of the system. However, focusing solely on correctness metrics is insufficient for clinical adoption. To facilitate acceptance among clinicians, evaluation frameworks should address additional dimensions, such as safety, transparency of reasoning, uncertainty quantification, and dynamics of human-system interaction. Without evaluation across additional dimensions, systems assessed solely on correctness metrics may demonstrate strong benchmark performance, yet remain unsuitable for clinical deployment.

6.2. Dimension II: Safety, Alignment, and Robustness

This dimension covers clinical safety, expert alignment, and output consistency. (Table 9) shows that the evaluation of these properties now plays a central role in the evaluation of AI generative systems in clinical settings. We organize this dimension into safety and clinical appropriateness, inter-rater concordance, output stability, and biases [75,123].

The safety evaluation focuses on identifying unsafe recommendations, possible patient harm, and clinical appropriateness. These assessments typically require expert human judgment or LLM-as-a-judge approaches, since they cannot be reliably quantified using standard automated metrics [76,80,81]. Another concern is over-recommendation, which involves unnecessary interventions, and under-recommendation, which involves failing to escalate appropriate care. Each has significant clinical and economic consequences.

Agreement metrics constitute a critical foundation for evaluating model alignment and trustworthiness within clinical workflows. These metrics capture both external and internal forms of agreement. External agreement metrics assess concordance between model output, experts, and established clinical standards, and consistency across outputs from different generative models [155]. The consistency across different generative models highlights whether independent systems converge on similar clinical conclusions when presented with identical cases. High agreement may indicate robust performance, while divergence warrants investigation of the sources of discordance.

Internal agreement metrics measure the consistency of a model’s own outputs across repeated evaluations. The stochastic nature of generative AI introduces variability across repeated inferences. Unlike deterministic systems that produce identical outputs for identical inputs, generative models may produce different responses when queried multiple times with the same clinical scenario due to decoding strategies. Researchers are increasingly evaluating output consistency by conducting multiple inference runs and quantifying response stability [125,133,156].

The bias assessment examines performance across linguistic and demographic variation. Some studies observe a stronger performance for English prompts than for other languages [157], whereas others report a more consistent multilingual performance [158]. These findings suggest that performance may differ across languages, possibly because some languages, such as English, are more prevalent in the model’s training data. When deploying clinical AI systems in underrepresented languages, researchers should perform language-specific evaluation to ensure reliable performance.

A limited but growing body of work evaluates whether diagnostic or treatment recommendations vary according to the characteristics of the protected patient [159]. Such biases pose substantial safety and equity concerns, particularly in large healthcare systems serving demographically diverse patient populations. [160] evaluated model performance across identical clinical scenarios with and without explicit demographic racial information and found that including racial identifiers led to inferior treatment recommendations while diagnostic accuracy largely stayed the same. These findings highlight the need to evaluate bias across demographic variables such as race, ethnicity, sex, age, and socioeconomic status, to identify and mitigate algorithmic inequity before deployment. This assessment ensures that generative AI systems provide equitable clinical decision support for all patients.

6.3. Dimension III: Reasoning Quality

This dimension examines whether the model reasons logically and justifies its clinical decisions appropriately. Our analysis included studies that explicitly evaluated model reasoning (Table 10). Researchers typically use chain-of-thought (CoT) prompting [161] to extract model reasoning, instructing models to think step by step and explain their reasoning. Reasoning evaluation metrics in healthcare AI research remain poorly standardized and vary considerably across studies. To improve reproducibility, we define four core metrics: (i) logical consistency, (ii) clinical reasoning, (iii) reasoning error, and (iv) reasoning clarity. Logical consistency refers to the alignment between the explanation of a model and its final diagnosis. For example, if a model diagnoses influenza, the symptoms described in its reasoning should match those of influenza rather than those of a generic respiratory infection. Clinical reasoning evaluates whether a model follows the clinical decision-making processes used by healthcare practitioners. Specifically, the model should (i) synthesize patient symptoms and medical histories, (ii) identify relevant clinical information, (iii) generate and assess diagnostic hypotheses, and (iv) provide a justified final diagnosis [162]. Reasoning error examines logical flaws within the reasoning process, such as incorrect intermediate steps, reasoning that violates clinical guidelines, or unsupported conclusions. Reasoning clarity evaluates how transparent and understandable the model’s reasoning steps are, ensuring that clinical experts can follow the logical progression from symptoms to diagnosis. These metrics are primarily evaluated using either human evaluators or LLM-as-a-judge approaches. Most studies rely on expert clinician evaluation rather than automated methods. This reflects both the complexity of reasoning evaluation and the current limitations in automated metrics to capture clinical logic. Future work should follow standardized reasoning evaluation with public datasets.

6.4. Dimension IV: Calibration and Uncertainty Assessment

Assessing model uncertainty is essential for the safe deployment of generative AI systems. This is true more so in healthcare environments, where a model’s capacity to recognize the limits of its knowledge is as important as its predictive accuracy [6,52]. Overconfident erroneous predictions may precipitate inappropriate clinical interventions or diagnostic errors, whereas under-confident predictions can result in diagnostic delays, unnecessary additional testing, or delayed escalation.

Multiple approaches have been developed to extract uncertainty estimates from generative AI models (Table 11), broadly categorized into white-box methods that require model access and black-box methods applicable to proprietary systems. Monte Carlo (MC) Dropout represents a white-box approach, and a key advantage is its practical accessibility, where uncertainty estimates can be extracted from existing dropout-trained networks without architectural modifications or retraining [51,110]. However, MC Dropout requires direct model access to estimate uncertainty, presenting significant challenges when evaluating closed-source generative AI, which provides limited access to internal model parameters. Closed-source models methodologies (Figure 6) include: (i) Confidence elicitation (CE) [163] which directly prompts the model to verbalize its uncertainty on a numerical scale, typically 0-100; (ii) Token-level probabilities (TLP) derive uncertainty metrics directly from the conditional probability distributions $p\left(y_l\right|y_{<l},x)$ calculated at each generation step (as defined in Figure 2). By accessing the probabilities of the tokens, researchers can calculate uncertainty scores such as the average, maximum, minimum, or total entropy across the generated sequence. (iii) Sample consistency (SC) leverages the stochastic nature of generative AI by submitting identical prompts multiple times and measuring inter-response agreement. The underlying assumption is that confident models produce consistent responses, whereas uncertain models produce divergent responses [49,164].

Calibration is an important requirement for clinical deployment, because predicted probabilities should reflect true outcome frequencies. For example, when a diagnostic model expresses low confidence in its output, clinicians may choose to request additional tests or override automated recommendations. When employing metrics such as ECE BS, generative models are prompted to output confidence scores, which are then used to compute these metrics [32,38]. Well-calibrated models ensure that predictions reported with 90% confidence are correct approximately 90% of the time [50]. Current evidence indicates that calibration performance varies considerably across LLM uncertainty estimation methods. SC approaches demonstrate a stronger discriminative ability than CE and TLP methods to distinguish correct from incorrect predictions. Both CE and TLP frequently exhibit poor calibration. In particular, CE methods overestimate model certainty, with calibration plots revealing that models report high confidence even when actual accuracy is lower [49,136]. Overall, these findings suggest that reliable calibration and uncertainty estimation remain an unresolved challenge, underscoring the need for improved techniques and standardized protocols for evaluation before deploying generative AI systems in clinical settings.

6.5. Dimension V: Clinical Anthropomorphism

Human-like communication by generative AI can influence clinician trust and reliance beyond diagnostic accuracy; clinical anthropomorphism now plays a larger role in evaluation. Most studies (see Table 12) use human judgment to assess qualities such as empathy, helpfulness, user-friendliness, and human-likeness. Early studies measured this indirectly through readability indices and verbosity metrics. However, recent research has begun to explicitly examine how anthropomorphism affects trust and trust behaviors of clinicians [106]. Empirical evidence shows that the behaviors of the anthropomorphic system alter clinical decision processes. Specifically, recent work demonstrates that human-like linguistic competences and conversational framing can enhance perceived usability while increasing the risk of inappropriate reliance on erroneous recommendations [16]. This is especially salient in high-stakes clinical contexts, where persuasive, empathetic, or fluent responses may be interpreted as indicators of competence rather than as stylistic attributes [106]. The reviewed literature reveals two patterns. The first, which we term perceptual alignment, focuses on subjective impressions of system behavior, including empathy, human-likeness, helpfulness, and overall user friendliness. These metrics are typically assessed through human ratings or LLM-as-a-judge. The second, cognitive accessibility, measures how easily users process model output, typically using conciseness scores and established readability indices such as Flesch Reading Ease and Flesch–Kincaid Grade Level. These measures allow for automatic computation. Although conciseness is often valued to avoid "verbosity" issues, recent studies suggest that elaboration of recommendations, providing detailed justifications rather than brief answers, is a behavior that increases clinician trust [106]. This suggests that "helpfulness" in a clinical context is tied to the transparency of reasoning rather than the brevity of the response. [16] shows that anthropomorphism emerges during multi-turn interactions. As a result, static, single-turn evaluation protocols, which dominate the literature, likely underestimate the anthropomorphic impact of generative AI in real-world clinical tasks. These behavioral traits matter more in use cases that require sustained interaction (e.g., history-taking, follow-up consultations, clinical supervision) than in single-turn tasks like diagnosis or prognosis generation based on OQA [80,88] and case descriptions [81,123].

Dimension V remains less developed than traditional evaluation dimensions, such as correctness. Anthropomorphic behaviors influence clinician trust and perceived reliability, mediating the impact of other evaluation dimensions such as factual correctness, reasoning quality, and uncertainty communication. Developing anthropomorphism-aware evaluation frameworks can enhance clinician awareness of automation bias and support effective, usable human-AI interaction. As generative AI becomes increasingly integrated into clinical workflows, further methodological development to evaluate clinical anthropomorphism may be necessary to better understand its influence on clinician trust, responsible adoption, and potential automation bias.

7. The Evolving Landscape of Clinical Benchmarks

Although previous sections detail evaluation metrics and protocols, medical benchmarks provide the clearest empirical platform for testing generative AI in healthcare. These benchmarks combine diverse clinical tasks, from knowledge retrieval to reasoning, into standardized datasets for comparison. Typical evaluation benchmarks rely on question-answering and factual recall metrics, overlooking the longitudinal, context-sensitive demands of diagnosis and prognosis. Models that excel in standardized datasets often fail to generalize to real clinical workflows [165]. Recent studies show that LLMs often struggle in complex or rare scenarios [166], with errors in reasoning [167], temporal context [168], and uncertainty quantification [169]. The research community has increasingly recognized the limitations of generic evaluation frameworks, catalyzing a surge in specialized clinical benchmark datasets. To bridge the gap between theoretical evaluation and practical assessment, we curated a selection of benchmarks (Table 13) explicitly mapped to our five proposed dimensions. Although English-language datasets and MCQ formats remain prevalent due to their ease of automated scoring, we prioritize benchmarks that move beyond these constraints, incorporating multimodal inputs, multilingual evaluation, and structured assessment of clinical reasoning.

Benchmark contamination through data leakage in datasets such as MedMCQA [180] and MedQA [181] artificially inflates performance and obscures the true generalizability [182,183]. For example, a study [6] showed that when the benchmark answer choices were intentionally altered to include incorrect options and a ’None of the above’ choice, the model accuracy dropped, indicating a lack of understanding. This highlights the need for Dimension II and IV, which can be addressed through simple modifications of the dataset, such as restructuring multiple-choice questions to include scenarios without a correct answer [6]. Most benchmarks support automatic evaluation, but new frameworks increasingly stress the need for human evaluation of generative AI, especially open-ended responses [177]. Human evaluation, however, creates scalability challenges, making the development and refinement of LLM-as-a-judge methods even more crucial for large-scale assessment.

8. Five-Dimensional Evaluation Framework Tradeoffs and Staging

Although the proposed five-dimensional framework aims to provide comprehensive coverage of the evaluation of generative AI in clinical diagnosis and prognosis, not all dimensions are equally important in all stages of deployment. Evaluation priorities must therefore be context-dependent and staged, reflecting system maturity. We conceptualize the framework as a progressive, dependency-aware evaluation pipeline in which later dimensions presuppose satisfactory performance from earlier ones (Figure 7).

8.0.0.4. Stage I: Factual and Safety Preconditions.

In this stage, we focus on Dimensions I (Factual and Contextual Correctness) and II (Safety, Alignment, and Robustness), which constitute mandatory prerequisites for all clinical applications. The evaluation at this stage determines whether a system produces clinically valid information and avoids unsafe, harmful, or misaligned recommendations. Models that fail here cannot be meaningfully evaluated along higher-level dimensions, since incorrect or unsafe output undermines downstream reasoning, uncertainty assessment, and interaction quality.

8.0.0.5. Stage II: Internal Validity and Knowledge Reliability.

Once the factual correctness and safety are established, the evaluation moves toward assessing the internal validity of the behavior of the model. Dimension III (Reasoning Quality) examines whether outputs are supported by coherent, clinically plausible reasoning rather than superficial pattern matching or memorization. Dimension IV (Calibration and Uncertainty Assessment) evaluates whether the model appropriately represents confidence, acknowledges uncertainty, and exhibits awareness of its own limitations. Together, these dimensions probe whether the correct answers arise from reliable internal processes.

8.0.0.6. Stage III: Interaction and Deployment Readiness.

Dimension V (Clinical Anthropomorphism) is primarily relevant in patient-facing systems or applications that involve extended human–AI interaction, such as longitudinal monitoring or conversational decision support. At this stage, the evaluation focuses on communicative qualities, including empathy, tone, clarity, and perceived helpfulness, which influence trust, adherence, and usability.

Moving from benchmarks to real-world deployment introduces constraints that traditional research often overlooks. For example, a recent pilot study [184] at Marburg University Hospital evaluated a generative AI platform for triage, diagnosis, and report generation for non-urgent emergency department patients. The study, involving 20 patients, found strong usability of the system and high levels of trust in the system among participants. However, the average interaction time was 10.6 minutes, compared to 4 minutes for clinician-led triage. The interaction time refers to the patient assessment led by the system, which involves dialogue with the AI interface and the collection of clinical data. This discrepancy highlights the "clinical applicability" gap, which our framework assesses in its second evaluation dimension. Even a safe and accurate model may fail in practice if its interaction demands exceed the operational realities of an emergency department.

The staged approach highlights dependencies and tradeoffs among evaluation dimensions. By structuring the process linearly, researchers can diagnose specific points of failure and prevent premature emphasis on interaction quality before establishing clinical validity. Overall, the framework is not intended as a universal scoring rubric but as a structured evaluation logic that enforces clinical safety and correctness first, true model understanding second, and interaction quality last, aligning evaluation priorities with real-world clinical deployment. As underdeveloped regulatory frameworks [185] continue to evolve, these standardized criteria will provide the necessary rigor for systematic validation and safe clinical integration.

9. Open Challenges and Future Directions

Despite rapid progress in generative AI for clinical diagnosis and prognosis, the field still lacks a unified, rigorous evaluation approach that ensures scientific reproducibility and clinical safety. Most studies focus on isolated performance metrics and overlook critical aspects such as reasoning transparency, calibration, and multimodal integration. To address this gap, we propose a five-dimensional evaluation framework designed to provide a structured, reproducible methodology to validate generative AI systems in clinical settings.

This framework complements recently established reporting guidelines, most notably TRIPOD-LLM [25]. Although TRIPOD-LLM provides an essential consensus-based checklist that outlines what researchers must report when developing and validating LLM for clinical use, our framework provides a practical stage-based methodology for how to execute these evaluations, and which metrics evaluate which criteria. For example, TRIPOD-LLM mandates the transparent reporting of model uncertainty and safety protocols; our framework operationalizes these requirements through Dimensions II (Safety, Alignment, and Robustness) and IV (Calibration and Uncertainty) by detailing specific automated and human-in-the-loop metrics to extract these data. By adopting this multi-dimensional evaluation pipeline, researchers can ensure that the empirical data used to satisfy TRIPOD-LLM reporting standards are robust, clinically meaningful, and accurately reflect the model’s readiness for real-world deployment.

9.1. Evaluation of the Transparency of Clinical Reasoning

Researchers must evaluate the internal consistency of clinical reasoning. The current evaluation neglects whether the models demonstrate faithful clinical reasoning. Future frameworks should go beyond factual verification to examine the consistency and transparency of model reasoning. Achieving this at scale requires hybrid strategies that combine automated metrics with human-in-the-loop assessment. For example, researchers can use automatic metrics, such as BERTScore from Dimension I (Factual and Contextual Correctness), to verify answer correctness against a golden reference, and then use human-in-the-loop metrics, such as clinical reasoning from Dimension III (Reasoning Quality), to verify reasoning faithfulness. Emerging LLM-as-judge approaches may help scalability if evaluators account for their limitations.

9.2. Standardizing Uncertainty and Calibration Metrics

Most current evaluations lack uncertainty estimation and calibration. Future studies should report calibration and uncertainty as standard reporting requirements rather than optional analysis, since generative AI models tend to be overconfident [6]. Uncertainty estimation methods such as semantic entropy have been increasingly adopted and show promise in the assessment of model uncertainty [186,187]. Integrating such measures into clinical evaluation is essential for safe deployment in high-stakes environments.

9.3. Advancing Multimodal Integration Assessment

The evaluation of MLLMs and VLMs requires greater attention, particularly with respect to their ability to integrate information across modalities. Although a limited number of studies have begun to examine this aspect [137,188,189,190,191], the depth of multimodal evaluation remains insufficient. Evaluators must determine how much each modality contributes to performance. If a model relies mainly on one modality, multimodal integration might be unnecessary. Strong performance from a single modality where multimodal input is clinically required should prompt skepticism and deeper investigation.

This review has several limitations. First, the literature search covered only English-language publications in Web of Science and PubMed, likely excluding relevant studies in other languages or alternative databases. Second, the review primarily relies on peer-reviewed publications. Since generative AI research evolves rapidly, many recent findings appear first in preprints and may not be fully represented here. Third, publication bias likely affects the field, as studies with positive results are published more often than those with negative or inconclusive outcomes, which can create an overly optimistic view of the current model capabilities. Finally, as evaluation practices evolve, some metrics and frameworks discussed here may change with new consensus guidelines. Despite these limitations, this work provides a structured synthesis of current evaluation approaches and proposes a framework that can incorporate new metrics to support more rigorous and reproducible assessment of generative AI in clinical diagnosis and prognosis.

10. Conclusions

Traditional ML systems require task-specific labeled datasets, while generative AI models use a few-shot and zero-shot adaptation to handle diverse clinical tasks. This flexibility lowers barriers to rapid deployment, but evaluation practices have not kept pace. Current assessments remain fragmented and focus largely on correctness metrics. As generative systems assume broader clinical roles, evaluation must extend beyond accuracy to include safety, reasoning quality, uncertainty expression, and interaction effects.

This survey consolidates fragmented evaluation practices into a five-dimensional framework that reveals gaps in the current assessment of generative AI systems for clinical use. Instead of calling for additional metrics, it underscores the necessity of standardized, multidimensional evaluation protocols that collectively capture correctness, robustness, reasoning behavior, uncertainty expression, and interaction effects. Future research must translate these dimensions into standardized reporting practices and reproducible evaluation protocols that can be adopted across institutions.

Appendix A.1. Pubmed Query

( ( "Large Language Model"[Title/Abstract] OR "Large Language Models"[Title/Abstract] OR LLM[Title/Abstract] OR LLMs[Title/Abstract] OR "Multimodal Large Language Model"[Title/Abstract] OR "Multimodal Large Language Models"[Title/Abstract] OR MLLM[Title/Abstract] OR MLLMs[Title/Abstract] OR "Vision Language Model"[Title/Abstract] OR "Vision Language Models"[Title/Abstract] OR VLM[Title/Abstract] OR VLMs[Title/Abstract] OR ChatGPT[Title/Abstract] OR Llama[Title/Abstract] OR LLaMA[Title/Abstract] OR Gemini[Title/Abstract] OR "Generative AI"[Title/Abstract] )

AND ( Health[Title/Abstract] OR Medical[Title/Abstract] OR Clinical[Title/Abstract] OR Medicine[Title/Abstract] )

AND ( Test[Title/Abstract] OR Evaluate[Title/Abstract] OR Performance[Title/Abstract] OR Assess[Title/Abstract] )

AND ( Diagnos*[Title/Abstract] OR Prognos*[Title/Abstract] OR "Risk prediction"[Title/Abstract] ) )

AND ("2022/01/01"[Date - Publication] : "2025/12/31"[Date - Publication])

Appendix A.1.1. Web of Science Query

TS=( ( "Large Language Model*" OR LLM* OR "Multimodal Large Language Model*" OR MLLM* OR "Vision Language Model*" OR VLM* OR ChatGPT OR Llama OR LLaMA OR Gemini OR "Generative AI" )

AND ( Health OR Medical OR Clinical OR Medicine )

AND ( Test OR Evaluate OR Performance OR Assess )

AND ( Diagnos* OR Prognos* OR "Risk prediction" ) )

AND PY=(2022-2025)