Between Rigor and Relevance: Why the EU Hta Guidelines on Indirect Comparisons Miss the Mark

1. Introduction

The implementation of Joint Clinical Assessments (JCAs) under Regulation (European Union [EU]) 2021/2282 marks a significant transformation in the landscape of health technology assessment (HTA) within the EU [1]. Central to this framework is the need to evaluate the certainty of the relative clinical effectiveness of new health technologies across a diversity of national healthcare contexts [1,2]. A distinctive feature of JCAs is the multiplicity of PICO (Population, Intervention, Comparator, Outcome) framework questions, reflecting the varied clinical practices, treatment comparators, outcomes of interest, populations and subgroups relevant across Member States (MS) [3,4,5,6]. This complexity presents considerable methodological challenges for comparative effectiveness research [7]. A recent exercise by the JCA subgroup identified 13 PICOs for two oncology indications [8].

In this setting, indirect treatment comparisons (ITCs) are an indispensable analytical tool. In many instances, head-to-head randomized controlled trials (RCTs) comparing the intervention with all relevant national comparators are not available at the time of assessment. As a result, robust and transparent methodologies for indirect evidence synthesis are essential to fill the evidence gaps and support HTA assessment and reimbursement decision-making across jurisdictions.

The methodological importance of ITCs has been acknowledged in previous HTA frameworks, which refer to rigorous guidance for performing network meta-analyses (NMAs) and Population-Adjusted Indirect Comparisons (PAICs) in the absence of direct evidence. The most well-known and influential guidelines on this topic are probably those by the National Institute for Health and Care Excellence (NICE), Decision Support Unit (DSU) and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force, [9,10,11]. The recent guidelines issued by the HTA Coordination Group (HTACG) extend this tradition by specifying harmonized methodological standards for the conduct and assessment of quantitative evidence synthesis within JCAs [11,12].

However, the HTACG guidelines also raise significant concerns related to the validity of ITCs. These guidelines explicitly require the verification of key assumptions such as similarity, homogeneity, and consistency to justify the validity of synthesis of evidence from multiple trials [11,12]. These assumptions are often difficult to meet in practice, particularly in the case of disconnected networks or when effect modifiers are unevenly distributed across studies. The increasing use of advanced methods, such as Matching-Adjusted Indirect Comparisons (MAIC), Simulated Treatment Comparisons (STC), and Multilevel Network Meta-Regression (ML-NMR), has further complicated the evidentiary landscape, calling for careful scrutiny of methodological rigor, transparency, and robustness [13]. To date, there is no clear definition of methods that can be universally and reliably applied to assess the value of products.

This paper provides a critical review of the HTACG’s methodological and practical guidelines on direct and indirect comparisons, with particular emphasis on indirect evidence synthesis. It highlights areas of ambiguity or methodological contention and evaluates their applicability in the JCA context. Special attention is given to the implications of these guidelines for health technology developers (HTDs).

2. Summary of the Guidelines

Overall, the HTACG promotes a conservative, hypothesis-driven approach to evidence synthesis. Direct comparisons using robust RCT data are preferred, and anchored ITCs are acceptable if all methodological requirements are met. When PAICs or unanchored comparisons are necessary, their use should be carefully justified, transparently reported, and subjected to extensive sensitivity analysis. The guidelines emphasize the importance of pre-specified analytical plans, expert statistical input, and critical evaluation of residual uncertainties throughout the synthesis process [11,12].

Key assumptions

All forms of evidence synthesis, whether direct or indirect, are based on the fundamental assumption of exchangeability, which is achieved through three essential criteria [11,13,14,15,16]:

i.

Similarity of studies with respect to effect modifiers.

ii.

Homogeneity of relative treatment effects across trials comparing the same interventions; and

iii.

Consistency between direct and indirect comparisons within an evidence network.

Violation of any of these conditions undermines the validity of the analysis [17]. Similarity, in particular, is highlighted as difficult to verify in practice, given the potential presence of unobserved or even unknown effect modifiers [11].

Direct Comparisons

Direct comparisons are typically synthesized using standard pairwise meta-analysis methods [18]. Fixed-effect models (e.g., inverse variance or Mantel-Haenszel methods) are acceptable when the assumption of a common effect is tenable, although this requires strong justification [11]. In most cases, random effects models are preferred because of the expected heterogeneity between studies. The Knapp-Hartung method with the Paule-Mandel estimator is recommended when at least five studies are available; for fewer studies, the use of variance correction, the DerSimonian-Laird (DSL) method, qualitative summaries, or the alternative use of Bayesian methods with weakly informative priors are recommended [11].

Indirect treatment comparisons

In the absence of head-to-head trials, or when multiple interventions need to be evaluated simultaneously, indirect comparisons are necessary. The guidelines categorize ITC methods into three primary types [11]:

1.

Anchored indirect comparisons

These preserve randomization by estimating relative treatment effects using a common comparator. The Bucher method is appropriate for comparisons with a common comparator and can be applied to simple star networks [19]. The NMA is required for more complex evidence networks [16]. Both frequentist and Bayesian implementations of NMA are acceptable, with the latter offering advantages in handling sparse data and incorporating prior knowledge [11]. Validity depends on the assumptions of similarity, homogeneity, and consistency, all of which must be rigorously evaluated. Tools such as node splitting [20] and inconsistency models [21] can be used to formally evaluate inconsistency within closed loops.

2.

Population-Adjusted Indirect Comparisons

When the similarity assumption is not met, due to observed imbalances in effect modifiers, PAICs may be considered [11]. These require access to individual patient-level data (IPD) for at least one study and include methods such as MAIC, STC, and ML-NMR [13]. These approaches rely on the assumption of conditional constancy of relative effects, i.e., that all relevant effect modifiers have been correctly identified and included. Because of their complexity and sensitivity to modeling choices, guidelines recommend that PAICs be prespecified and accompanied by sensitivity analyses, including formal tests against shifted null hypotheses to account for residual uncertainty [11,12].

3.

Unanchored comparisons and non-randomized evidence

In disconnected networks without a common comparator, analyses are equivalent to comparisons based on non-randomized evidence. Such comparisons require full access to IPD and rigorous adjustment for confounding [22]. The guideline briefly acknowledges advanced IPD-based methods for confounding adjustment, including multiple regression, instrumental variables, and g-computation [11], but does not elaborate on their implementation or endorse them as standard practice. Instead, it emphasizes propensity score-based methods, such as matching or inverse probability weighting. These require that all known relevant confounders be measured and that assumptions of positivity, overlap, and covariate balance be satisfied [13]. Even when these conditions are met, results from non-randomized data are considered to have a higher uncertainty and risk of bias, and should be interpreted with caution [11]. Unanchored MAICs and STCs that use aggregate comparator data rather than IPDs are considered insufficient.

3. Critical Review

The HTACG guidelines on direct and indirect comparisons set an ambitious standard for methodological rigor in the JCA framework, going beyond those previously set by the NICE DSU, the ISPOR Task Force on ITC, or the Cochrane Handbook [9,10,23,24].

However, a critical analysis reveals several problematic areas where the guidelines are either overly conservative, inconsistently reasoned, or insufficiently aligned with accepted best practice in evidence synthesis.

1.

Ambiguity in the role of indirect evidence alongside direct comparisons

The guidelines are unclear about whether indirect evidence can complement direct comparisons. While established methods such as NMA allow the integration of both evidence types, subject to a consistency check, the HTACG does not explicitly support this approach. The methodological guideline states for example: “when treatments have not been directly compared in RCTs, indirect comparisons can be used”[11]. This formulation is ambiguous and might be interpreted as “when treatments have been directly compared, indirect comparisons cannot be used.” The latter interpretation would be aligned with another statement from the HTACG: “the inclusion of additional comparators and studies beyond those required to connect the network should generally be avoided.” The omission of clear support for combining direct and indirect data creates uncertainty, which contradicts the international consensus favoring mixed treatment comparisons when properly justified [25,26].

Discouraging the use of larger networks stands in opposition to the principle that broader networks, incorporating (more) indirect evidence, yield more precise estimates and support more generalizable inferences, when used appropriately [27].

2.

Overly restrictive interpretation of assumption violations

The HTACG’s strict stance on assumption violations is a central concern. The methodological guideline asserts that “if any of these properties [similarity, homogeneity, consistency] do not hold, the results of an anchored indirect comparison are unlikely to provide a meaningful estimate” [11]. This fails to recognize the role of methods such as random-effects models, meta-regression, and sensitivity analyses in mitigating these limitations rather than discarding analyses altogether. NICE DSU guidance and other international sources promote such techniques as a standard part of evidence synthesis [26,28].

Furthermore, the guidelines suggest that comparisons should not proceed if data on a relevant effect modifier are missing in one or more studies: “If data on a relevant effect modifier is unavailable from one or more studies, then such a comparison cannot be made, and this should be clearly reported as a limitation” [12]. The guidelines then allow for the possible use of proxies, but it appears that the preferred approach of HTACG is to do nothing or perhaps to exclude the studies with missing data for the effect modifier. Excluding studies for a single missing covariate risks unnecessary loss of information. Best practice, as recommended in NICE DSU Technical Support Documents (TSDs) 18 and ISPOR guidelines, favors using proxy variables, modeling missingness, or applying bias-adjustment techniques over outright exclusion [24,29]. Evidence-based medicine (EBM) best practices emphasize the importance of incorporating all available evidence, even studies with limitations, to ensure comprehensive consideration in decision making. A grading of the strength of evidence is then provided to support the recommendation [30,31].

3.

Population-Adjusted ITCs: overcautious appraisal

The guidelines acknowledge the relevance of methods such as MAIC and STC in the context of anchored comparisons, yet caution that they are “often more suitable as an exploratory analysis rather than as the primary analysis”[12]. The HTACG’s concern that “the number of methods and potential covariate combinations available to the modeler raises the possibility of selecting the method that produces the most favorable results for the intervention under assessment” is understandable [12]. However, adjusted analyses generally provide the most plausible estimate of treatment effect, and a more appropriate recommendation would be to report the adjusted analysis as the primary analysis and the unadjusted one as secondary.

Moreover, the guideline uniquely recommends applying shifted null hypothesis testing to PAICs to account for possible unmeasured effect modifiers, without requiring the same for standard NMA or Bucher methods [11]. All indirect comparisons, adjusted or unadjusted, are subject to unverifiable assumptions about effect modifiers [24]. If this approach of shifted null hypothesis testing is considered informative, it is unclear why it would apply only in the case of PAICs. Furthermore, it should be noted that this approach is not standard and therefore does not align with the stated objective of describing the “most commonly used methods” for ITC.

In a sense, a shifted null is a way to make it harder to claim an effect is statistically significant. But we already account for uncertainty via confidence intervals (CI), credible intervals, or prediction intervals. If bias is suspected, one would typically interpret the results qualitatively with caution or adjust the level of certainty (e.g., GRADE “very low certainty” if there is a high risk of bias). Imposing a formal test threshold Δ could be seen as an arbitrary second hurdle. For instance, if a hazard ratio is 0.85 with 95% CI 0.72–0.99 (p < 0.05 for H0:1), a shifted null approach might say “we will only consider it significant if p < 0.05 for H0:0.90” – effectively requiring the CI to exclude 0.90. Why 0.90? The guideline offers no specific literature, leaving it to the HTD to justify. This could cause debates between companies and assessors over the choice of threshold rather than focusing on the actual data [11]. Furthermore, the guideline explicitly defers threshold setting to MS, which somewhat contradicts the idea of a unified assessment. France’s Haute Autorité de Santé (HAS) and Germany’s Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) remain highly resistant to ITC evidence [32,33,34,35], while Sweden, Netherlands, Norway, and Belgium show greater openness to such data [33,36]. Allowing MS to set their own thresholds enables them to accept or reject ITC results based on their preferences. This does not align with JCA’s goal to standardize clinical assessment between countries. The approach of E-values could be considered as an alternative [37].

4.

Meta-regression vs. subgroup analyses: misplaced prioritization

Another controversial recommendation is the endorsement of subgroup analyses over meta-regression. The methodological guideline states that “in the context of JCA, subgroup analyses are often more useful than meta-regression”[11]. This is a debatable position. Subgroup analyses, especially those based on aggregate trial-level data, are underpowered, prone to spurious findings, often require dichotomization of continuous variables, and sometimes violate randomization. In contrast, meta-regression, when applied appropriately, can exploit continuous covariates across trials and detect trends that would be missed by binary subgroup splits [38].

This contradicts the view presented in the NICE DSU TSD 3, which emphasizes the use of meta-regression where feasible and only warns against ecological bias, without suggesting that one method is inherently superior [26]. A more balanced approach would advocate for a case-by-case evaluation, with sensitivity analyses using both methods where appropriate.

5.

Underappreciation of Bayesian methods

Bayesian methods receive only cursory mention in the guidelines, despite being especially advantageous in the small-sample and rare-event scenarios explicitly acknowledged as problematic for frequentist methods [39]. For example, the practical guideline admits that Knapp-Hartung methods may yield “misleadingly narrow” CIs in homogeneous settings and “non-informative results” when data are sparse [12]. Nonetheless, Bayesian approaches are described merely as “alternative” and are conditioned on the availability of priors [12]. By contrast, widely accepted literature endorses Bayesian methods as preferred in sparse or uncertain settings because of their ability to incorporate prior knowledge and produce interpretable posterior distributions [40,41,42].

Furthermore, the HTACG guidelines may result in assessment dossiers employing a mix of frequentist and Bayesian methods across different comparators or outcomes, depending on data availability and the specific context. While acknowledging data limitations, this approach can introduce complexity. Specifically, combining frequentist and Bayesian methods within a single assessment can pose challenges to maintaining a consistent inferential framework. This could lead to difficulties in the interpretation of results, conflicting with the Neyman-Pearson principles that underlie much of frequentist hypothesis testing. Prioritizing a single, unified statistical framework within an assessment could enhance consistency in interpreting results and reduce potential ambiguity in the overall synthesis. Moreover, given the complexities associated with applying frequentist multiplicity controls in the diverse landscape of HTA, a consistent application of a Bayesian framework may offer a more coherent and interpretable approach to conducting ITCs.

6.

Unanchored comparisons and non-randomized evidence: an overly restrictive stance

The HTACG takes a strong negative stance on unanchored indirect comparisons, stating that “only anchored indirect comparisons are appropriate, as these respect within-study randomization” [11]. While preserving randomization is methodologically ideal, this categorical rejection of unanchored methods is both impractical and inconsistent with modern HTA practice [43]. Single-arm trials (SATs) are increasingly used, especially in oncology and rare diseases. Between 2002 and 2021, approximately 31% of United States Food and Drug Administration (FDA) oncology approvals were based solely on SATs [44]. The European Medicines Agency (EMA) also recognizes that SATs may serve as pivotal evidence in specific regulatory contexts [45].

The guidelines risk eliminating informative analyses in situations where no other comparative evidence is available. This is particularly problematic when high-quality real-world data or external control arms (ECAs) can be used in conjunction with advanced methodologies such as propensity score adjustment, doubly robust estimators, or target trial emulation frameworks [46,47]. Such approaches are endorsed in both regulatory and academic literature as credible strategies when randomized data are unavailable [43,48]. Despite their limitations, unanchored MAIC and STC also have a role to play in informing decision-making [43]. The rigid exclusion of unanchored comparisons thus contrasts with a growing international consensus that, while such methods carry higher uncertainty, they may still support decision-making when carefully designed and transparently reported. This stance also conflicts with EBM principles, which advocate using all available evidence.

Although the HTACG considers unanchored comparisons to be generally “inappropriate”, the guidelines do provide a succinct description of relevant methods [11,12]. When the pivotal evidence for the assessed product is based on an SAT, the guidelines stipulate that “adjustment methods require access to the full IPD”, for both the assessed product and the comparator. This effectively positions MAICs or STCs, which utilize aggregate comparator data, as less desirable than ECAs. This is a contentious point. In practice, IPD are often unavailable for comparator trials, necessitating the use of ECAs derived from retrospective real-world data [49]. Consequently, the choice between an ECA based on potentially lower-quality real-world data and a MAIC or STC using higher-quality aggregate data is often complex, and the guidelines’ preference for ECAs in this context is debatable.

7.

Overlooked considerations in living evidence synthesis

While the guideline references the need for evidence synthesis to be current, it omits any discussion of living NMA or updating mechanisms [11]. This is a missed opportunity, particularly in fast-evolving fields like oncology, where evidence landscapes shift rapidly. Cochrane and other methodological leaders now emphasize living NMAs as essential for maintaining relevance and responsiveness to emerging data [50].

8.

Support for advanced time-to-event (TTE) methods

The guideline’s handling of TTE data is more balanced. It rightly calls for testing the proportional hazards (PH) assumption and recommends alternatives such as restricted mean survival time (RMST) and flexible survival models when PH does not hold. This is consistent with emerging best practices, particularly in oncology where PH violations are common [51].

4. Implications for JCA Submissions

The HTACG’s guidelines on direct and indirect comparisons impose significant methodological and procedural expectations that have substantial implications for HTDs preparing JCA submissions. The reporting requirements for ITCs go well beyond traditional HTA standards

First, the guidelines emphasize the need for prespecified strategies and transparent criteria to identify covariates that may impact relative treatment effects. More specifically, HTDs must now conduct not only a systematic literature review (SLR) to identify clinical evidence on treatment effects, but also a second comprehensive review to identify potential effect modifiers, as well consultations with health care professionals (HCPs) with experience in the disease area [11,12]. The implication is that reliance on subgroup analyses identified in the clinical study report and SLR and consultations with clinicians employed by the HTD will no longer suffice. The need to formally involve HCPs with disease-specific expertise in identifying effect modifiers adds further operational complexity, as this consultation must presumably be documented and justified within the submission.

Second, the guidelines mandate that JCA dossiers include both population-level and comparator-level evidence networks when conducting NMAs [12]. While this may enhance transparency for national decision-makers, it introduces an analytical burden that could be significant for submissions with multiple comparators. HTDs will need to construct and validate several network structures, increasing the risk of type I error and complicating consistency evaluations. No empirical justification is provided for this dual approach, which may lead to duplication of effort without corresponding benefit.

Third, the guideline’s preference for shifted null hypothesis testing in adjusted ITCs (e.g. MAIC/STC) imposes an additional evidentiary hurdle. HTDs are expected to defend the choice of a clinically meaningful threshold (e.g., a relative risk reduction of 10%) without a standard benchmark, leaving room for arbitrary or inconsistent assessments across MS. This creates uncertainty about how population-adjusted estimates will be received and whether they will be discounted based on subjective criteria.

Fourth, the guidelines recommend that prediction intervals, not just confidence or credible intervals, be routinely reported in random-effects models. This requirement is methodologically sound, but rarely fulfilled in current reports of ITCs [52]. HTDs must now plan to generate and interpret prediction intervals, particularly for sparse data scenarios where between-study heterogeneity may be high.

Fifth, while the use of unanchored comparisons is generally discouraged, there is a clear preference for ECAs over PAIC using aggregate data for the comparator, such as MAICs and STCs. This stance diverges from recent HTA trends: many relative effectiveness assessments based on SATs, especially in oncology, have relied on MAICs and STCs, while the use of formal ECAs remains relatively rare in European regulatory and HTA contexts [49,53]. Recent empirical reviews have found that MAIC and STC methods are commonly used and accepted in health technology submissions when SATs serve as pivotal evidence, largely due to the IPD requirement being limited to sponsor-held data [46]. In contrast, ECAs, requiring either high-quality observational data or access to relevant patient registries, pose significant feasibility challenges in terms of data access, endpoint alignment, and methodological transparency [54]. Nonetheless, as real-world data infrastructures mature and the methodological landscape evolves, HTDs should increasingly consider the feasibility of developing ECAs in such contexts. Approaches such as target trial emulation, leveraging observational datasets with robust confounding adjustment (e.g., propensity scores or doubly robust estimators), may offer a viable pathway toward more acceptable unanchored evidence [48].

Given the multiplicity of PICO questions [3,4,5,8] inherent in the JCA process and the need for sensitivity analyses and subgroup analyses, these expectations cumulatively lead to a marked increase in analytical and submission workload. In sum, HTDs will need to allocate additional resources—statistical, clinical, and procedural—to ensure compliance with the HTACG’s expectations. These guidelines not only raise the methodological bar, but also introduce substantial practical complexity, which, if not addressed proactively, could jeopardize the acceptability of the indirect evidence submitted in support of new interventions.

5. Conclusions

The HTACG’s methodological and practical guidelines on ITCs represent an important step towards harmonizing evidence synthesis practices across Europe within the JCA framework. The guidelines promote transparency, detailed reporting, and rigorous assessment of key assumptions - principles that are essential for robust comparative effectiveness assessments in an increasingly complex healthcare landscape [11,12].

However, this review has identified several important limitations that warrant careful consideration. First, the guidelines take an overly conservative stance toward assumption violations in ITCs, with a tendency to encourage rejection of analyses rather than consideration of available adjustment methods. Techniques such as meta-regression, population-adjusted indirect comparisons, and Bayesian modeling are acknowledged, but often relegated to secondary or exploratory roles, despite their widespread acceptance in international HTA and regulatory contexts [32,33].

Second, the guidelines do not adequately address the integration of direct and indirect evidence, nor do they clearly support the use of mixed treatment comparisons when both types of evidence are available. In areas where assumptions may not fully hold, such as the presence of unknown effect modifiers or missing covariates, the recommendations favor study exclusion over analytical adjustment, potentially leading to loss of evidence and biased conclusions.

Third, while the guidelines express methodological caution about unanchored comparisons, they do not adequately recognize the practical realities that necessitate such approaches, particularly in rare diseases and oncology, where SATs remain prevalent [55]. The lack of detailed guidance on acceptable methods for ECAs and on strategies such as target trial emulation represents a missed opportunity to promote more sophisticated use of real-world evidence in HTA [47].

Fourth, the guidance introduces several operational requirements for HTDs, including dual systematic reviews, comparator-level network analyses, and shifted hypothesis testing, that significantly raise the complexity of JCA submissions. While these requirements may enhance methodological thoroughness, they risk reducing the feasibility and efficiency of submissions, considering the short timelines [56], particularly for smaller companies or in therapeutic areas with fragmented evidence bases.

Finally, it is worth noting that the developers of the HTACG guidelines do not appear to have adhered to the same evidence-based standards that they recommend for submissions. The guidelines lack transparent documentation of the evidence base underpinning their methodological choices. Specifically, there is no mention of results from empirical evaluations or simulation studies that assess the performance, validity, or limitations of different evidence synthesis and ITC approaches. This omission raises concerns about the robustness and transparency of the recommendations. Best practice in guideline development, such as that advocated by GRADE [57], emphasizes the importance of grounding recommendations in a systematic review of methodological evidence.

Despite the EU HTA guidelines’ aim to create a unified methodological framework, it is doubtful that they will have a practical impact on harmonizing the perception and utilization of ITCs across Europe. Historically, HTA agencies have exhibited markedly divergent attitudes toward indirect evidence, with agencies like IQWiG in Germany rejecting 94% of ITCs submitted in benefit assessments between 2011 and 2017, largely due to concerns about study suitability, similarity, and statistical methods [58]. Similarly, the French Haute Autorité de Santé (HAS) has maintained a cautious stance on accepting adjusted indirect evidence. The current EU HTA guidelines risk reinforcing these conservative positions by providing formal arguments for rejecting analyses without offering mechanisms to promote alignment or mutual recognition of evidence standards across Member States. The development of a framework to grade the certainty of indirect evidence could be part of such harmonization mechanisms, whereas the current guidelines tend to dismiss evidence in a binary manner.

Guidelines alone are unlikely to address the concern feeding the resistance of some HTA agencies against indirect evidence. Randomized controlled trials (RCTs) remain the gold standard for clinical evidence, in part because all analytical methods are prespecified in study protocols and statistical analysis plans, limiting the scope for post hoc methodological choices. In contrast, ITCs—especially those requiring population or outcome adjustments—are often conducted after trial completion, making full pre-specification difficult, as the methodology needs to be adapted according to available data. Also, the suspicion surrounding ITCs may stem less from flaws in the methodologies themselves than from the flexibility they afford health technology developers (HTDs) in selecting analytic strategies that may yield more favorable outcomes. Nonetheless, ITCs are indispensable in the health technology assessment (HTA) process: it is neither feasible nor ethical to conduct head-to-head RCTs for every comparator in every population of interest. If the EU HTA regulation is to facilitate access to innovative therapies across MS, it must allow for PICO questions to be answered through rigorously conducted and transparently reported ITCs, and HTA agencies must accept adjusted analyses. If concerns about selective reporting persist, greater involvement of independent agencies—either through independent conduct of ITCs or supervisory roles during their design and analysis—may be necessary to ensure both credibility and acceptability of adjusted indirect evidence. As those agencies may not be able to access IPD and therefore may not be able to conduct ITCs directly, their role could be exercised through active supervision of the planning and implementation of analyses.

In conclusion, while the HTACG guidance documents establish a rigorous foundation for evidence synthesis in the EU, their current formulation may inadvertently constrain methodological flexibility and limit the practical use of valid ITCs. To ensure both scientific credibility and operational feasibility, future updates should aim to:

• Provide clearer support for the integration of direct and indirect evidence (i.e. “mixed” treatment comparisons);
• Allow for appropriately adjusted analyses in the presence of assumption violations;
• Guide, rather than restrict, the cautious use of unanchored comparisons when no alternatives exist, including more guidance on the use of real-world evidence and ECAs;
• Enhance support for Bayesian methods: the HTACG may have given more attention to frequentists methods assuming that assessors are more familiar with them, although these guidelines provide in fact an opportunity to help assessors understand the value of Bayesian methods;
• Incorporate operational guidance that reflects the complexity and diversity of real-world submissions; and
• Propose a framework for grading the strength of indirect evidence.

Addressing these issues could enhance the guidelines’ utility, support more consistent and equitable HTA outcomes across MS, and ultimately improve timely patient access to effective therapies in the EU.

• Text box 1: Key limitations identified in the EU HTA Guidelines on quantitative evidence synthesis

∙

Overly conservative stance on assumption violations, risking blanket rejection of indirect treatment comparisons (ITCs)

∙

Ambiguous guidance on combining direct and indirect evidence, with no clear support for mixed treatment comparisons

∙

Restrictive preference for subgroup analyses over meta-regression, despite lower statistical power and greater risk of false positives

∙

Limited endorsement of population-adjusted methods (e.g., MAIC, STC), despite their relevance when effect modifiers are imbalanced

∙

Minimal consideration of Bayesian approaches, despite their strengths, particularly in sparse or rare-event settings

∙

Rigid dismissal of unanchored comparisons, overlooking recent advances in causal inference frameworks and evolving methods to quantify biases

∙

No indication that recommendations are grounded in empirical validation or simulation studies

• Text box 2: Suggested improvements for the EU HTA guidelines on indirect treatment comparisons

∙

Reconsider the guideline’s advice against using indirect evidence when direct comparisons exist, as this restriction lacks empirical or methodological justification

∙

Consider the wider use of Bayesian methods, which are well suited to quantitative evidence synthesis; provide readers with the knowledge required to engage with Bayesian approaches; reassess the appropriateness of combining Bayesian and frequentist analyses within the same dossier (currently allowed by the guidelines)

∙

Offer balanced recommendations on population-adjusted methods (e.g., MAIC, STC), acknowledging their relevance as primary analyses in many situations

∙

Explicitly acknowledge that all evidence has limitations, and propose a framework for grading the certainty of indirect evidence (e.g., in the manner of GRADE) to support decisions under imperfect evidence, potentially incorporating quantitative bias analysis

∙

Expand guidance on leveraging real-world evidence and external control arms, including the adoption of emerging methods such as target trial emulation

∙

Consider the practical feasibility of the guidelines, recognizing the limited time and resources available to both health technology developers and national assessors

∙

Ground methodological recommendations in a systematic review of empirical validation studies and simulation research, ensuring that guidance reflects best available evidence