Acceptability of external control-arm use in nononcology health technology assessment submissions

Abstract

Aim:

This study assessed the acceptability of external control-arm (ECA) use in nononcology health technology assessment (HTA) submissions in Europe.

Materials & methods:

We conducted a sequential mixed method study to investigate the study objective. First, we summarized published documentation from three HTA agencies in Europe – the National Institute for Health and Care Excellence (NICE) in England, the French National Authority for Health (HAS) and the German Institute for Quality and Efficiency in Healthcare (IQWiG) – to assess the availability of guidance on ECA methodology and implementation. We then reviewed independent nononcology HTA appraisals common across England, France, Germany and Italy to understand variations in agencies’ perceptions of ECA use. Finally, we conducted six double-blinded interviews with HTA experts from England, France, Germany and Italy to validate the findings and obtain illustrative insights on drivers of acceptability.

Results:

While NICE and HAS provide some level of ECA-related guidance on topics such as data suitability, methods and reporting, guidance from IQWiG remains limited. Overall, ECA use is mainly restricted to oncology, particularly given that only two nononcology appraisals were common across HTA agencies. However, NICE appears more open to accepting ECA use in supplementing clinical trial data, whereas IQWiG has a strong preference for traditional controlled clinical trials. Experts indicate that ECA use is most acceptable when accompanied by valid justification, suitable data sources and a rigorous methodology to minimize the risk of bias. Situations that experts perceive as appropriate for ECA use include missing comparators (i.e., single-arm trials), limited comparator data availability, or rapidly changing standards of care.

Conclusion:

There is a need to focus awareness on the value of ECA use as a supplement to randomized controlled trials, and to engage with HTA agencies early in clinical development.

Plain language summary

What is this article about?

This study evaluated European health technology assessment (HTA) guidelines to understand the acceptability of external control-arm (ECA) use in nononcology HTA submissions.

What methodology is described?

The authors reviewed published guidance from three European HTA agencies (NICE in England, HAS in France and IQWiG in Germany) to understand the availability of ECA-related guidance; analyzed nononcology HTA appraisals in England, France, Germany and Italy to understand variations in agencies’ perceptions of ECA use; and conducted six double-blinded interviews with HTA experts from England, France, Germany and Italy to obtain illustrative insights on drivers of acceptability.

What were the results?

While NICE and HAS provided some guidance on ECA use, IQWiG provided limited guidance. Overall, ECAs use was mainly confined to oncology, with NICE being more open to using ECAs beyond oncology to supplement clinical trial data. IQWiG, however, preferred traditional controlled clinical trials. Experts believed that ECA use was acceptable with valid data sources and rigorous methodology to minimize bias. Appropriate situations for ECA use included the lack of head-to-head comparisons (i.e., situations where only single-arm trials or placebo-controlled trials are available), limited comparator data, ethical or feasibility concerns, or in disease states with rapidly changing standards of care.

What do the results mean?

These findings suggest that, when conducted according to established best practices, ECAs may be used to strengthen evidence packages in nononcology settings, particularly when supported by collaboration with HTA agencies to address remaining uncertainties.

Randomized controlled trials (RCT) are considered the gold standard for clinical evidence supporting health technology assessment (HTA). However, complementary real-world evidence (RWE) generated through real-world data (RWD) is increasingly used to facilitate, inform and enrich HTA submissions [1,2]. Examples of submissions incorporating supplemental RWD include disease prevalence and unmet-need studies; observational studies supporting safety, label expansion or reimbursement decisions [3] and augmented or supplemented trials, where an external control arm (ECA) serves as a comparator [4].

ECAs derive from RWD and/or historical RCT data and simulate a control group for the treatment arm [3,4]. To build an ECA, researchers identify data suitable for comparison with the internal treatment arm [4]. Key practical and methodological considerations, particularly those stemming from European Union regulation 2021/2282 on HTA, include timing of data collection compared with the internal RCT arm (i.e., historical vs concurrent data), appropriateness of the comparator data source, data comparability and analytic measures controlling for potential confounding factors [3,5,6].

Studies leveraging the use of ECAs can be broadly categorized as supplemented single-arm trials (SATs), augmented RCTs and RCT+ designs [4]. In supplemented SATs, the ECA serves as the sole comparator group, enabling researchers to assess efficacy and safety outcomes [4]. In augmented RCTs, ECAs are used to increase statistical power by adding similar patient-level data points to small, internal control arms [4]. In RCT+ designs, ECAs supplement fully powered RCTs by using RWD as an additional comparator (e.g., a third arm) [4]. RCT+ ECA supplementation can be used to generalize efficacy and safety outcomes to broader real-world patient populations, avoid the issue of losing patients to follow-up, evaluate longer-term outcomes after RCT completion and validate reference values for proxy variables in RWE studies [3,7,8].

Supplemental ECA use is common in rare diseases, where the use of a control may be difficult owing to small sample sizes and/or lack of available treatments, and in oncology, where the rapid evolution of the standard of care may introduce ethical or feasibility challenges [9,10]. Furthermore, ECAs can also add value to studies beyond oncology and rare diseases [9,11], potentially enhancing clinical research by reducing clinical development time and facilitating early access to novel therapies [12]. However, external controls are not without limitations. Most limitations stem from the lack of randomization and blinding, which poses challenges to causal inference and can lead to observer and analyst biases [13]. One notable issue is the unreliability of statistical significance testing in studies using ECAs [13]. Consequently, ECA-related outcomes are considered persuasive only when the relative treatment-effect size is sufficiently large [13]. Therefore, methodological considerations that can help address ECA limitations are pivotal to ensure the validity of the generated insights [3,5,6].

Given the potential value of appropriate ECA use in clinical research, we aimed to understand how ECA use can effectively complement the nononcology HTA evidence package. To this aim, we reviewed published guidance from three European HTA agencies regarding ECA implementation and methodological appropriateness. In addition, we conducted a review of independent nononcology HTA appraisals common across England, France, Germany and Italy to understand variations in agencies’ perceptions of ECA outside their commonly accepted use. Finally, we report key insights from interviews with European HTA experts regarding both the acceptability of ECA use in HTA evaluations and drivers of ECA acceptability.

Materials & methods

We conducted a targeted review of published ECA guidance from three European HTA agencies: the National Institute for Health and Care Excellence (NICE) in England, the National Authority for Health in France (HAS) and the Institute for Quality and Efficiency in HealthCare in Germany (IQWiG). We selected these agencies since each maintains its own published RWE guidelines and influences overall European HTA guidance. We assessed the documents for implementation and methodological guidance, summarizing key ECA-related messages. Guideline documents included the NICE Real-World Evidence Framework (published 23 June 2022), the HAS Methodological Guide: Real-World Studies for the Assessment of Medicinal Products and Medical Devices (published 10 June 2021), and the IQWiG General Methods V6.1 (published 24 January 2022).

We then conducted a review of independent HTA appraisals common across agencies in England, France, Germany and Italy. Italy was included as representative of countries without formal ECA related guidance to gauge whether such countries would appraise ECA use similarly to those with more formal guidance. The review focused on nononcology product appraisals; oncology and other high-mortality indications were not considered since ECA use is common in these areas and has been the subject of previous research [14]. The current review utilized IQVIA Market Access Insights, a manually curated database of HTA reports from over 40 countries that links reimbursement, pricing, clinical trial and regulatory information across global markets. To identify published appraisals, we applied the following search criteria: ‘technology appraisals’ (NICE), ‘opinions’ (HAS), ‘early benefit assessments’ (Germany’s Gemeinsamer Bundesausschuss; G-BA) and ‘technical reports’ (Italy’s Agenzia Italiana del Farmaco; AIFA). We also searched agency websites for committee papers and submission documents. All published appraisals up to 27 August 2024 were included in the review. For each included appraisal, data for over 400 extracted parameters available in IQVIA Market Access Insights was downloaded for qualitative review and synthesis.

Lastly, from 23 January to 20 February 2023, we conducted one-to-one, 60 min interviews with European HTA experts (regarded throughout as ‘experts’) to understand their perceptions regarding acceptability of ECA in HTA submissions, specific situations in which agencies would consider ECA use acceptable and drivers of ECA acceptability. These categories were selected based on our review and synthesis of published guidelines and appraisals and further refined through iterative feedback among authors. To ensure clarity and relevance, the interview guide was iteratively refined through internal discussions (i.e., employees of IQVIA) with individuals with longstanding experience in HTA and payer research. Their feedback informed improvements to question wording, flow and interpretability. The final guide was administered in a semi-structured format by experienced qualitative researchers, allowing for probing and clarification as needed. We applied the following criteria to identify candidates: experts must (a) have previously served as members or advisers to national HTA agencies (England, France and Germany) or served as regional payers (applicable to Italy, given its strong history of national registry use and data collection); and (b) be familiar with the use of RWE and ECA in HTA. The interviews were conducted in a double-blinded manner (i.e., the research sponsor did not know the identity of the expert, and vice versa). Due to the limited number of participants (n = 6 in total), expert interviews were exploratory in nature, intended to complement the broader analysis and obtain individual perspectives rather than consensus views.

Results

Published ECA guidance

Figure 1 summarizes RWE guidance alignment between NICE, HAS and IQWiG. Detailed results are provided in the Supplementary Material, available online. Based on published RWE documents, we identified various levels of detailed guidance regarding ECA use across agencies.

Figure 1. . Level of real-world evidence guidance alignment among health technology assessment agencies.

HTA: Health technology assessment; RWE: Real-world evidence.

In England, NICE recognizes that ECAs can be used to supplement internal or concurrent controls in RCTs but carry a potential risk of bias [15]. NICE also acknowledges several methods available to combine internal and external controls, which place different weight on external data [15]. To optimize supplemental ECA use, NICE provides guidance on methods, reporting checklists for observational studies, supporting documents and methodological papers and criteria for using multiple data sources, minimizing bias and performing statistical analyses [15].

In France, HAS provides a list of methodological points that they consider when assessing an external comparison between an uncontrolled trial and an ECA [16,17]. HAS also gives details regarding statistical tools used for assessing RWE, focuses on quality of life as assessed through patient-reported outcome measures, and supplies detailed information regarding the use of the French Health Data Hub and the Système National des Données de Santé (SNDS) dataset [16]. HAS supports and aligns closely with European Union regulatory initiatives, methodological tools and documents, including the latest European Medicines Agency (EMA) guidelines [16].

In Germany, IQWiG considers study types other than RCTs as usually unsuitable for causal inference purposes and requires robust RCT data to verify the appropriateness of RWD analyses [18,19]. IQWiG accepts nonrandomized intervention analyses and observational studies in justified exceptional cases only (e.g., nonfeasibility of an RCT) and disapproves of the use of nonadjusted indirect comparisons (i.e., the naive use of single-study arms) [18]. However, given that evidence available at market access is often insufficient, IQWiG allows the use of routine practice data in early drug-benefit assessments based on the following conditions: only data collected specifically for the study, or data collected from a patient registry, are considered acceptable, and for comparative studies without randomization, conclusions are considered meaningful only if results exceed a certain effect size [20].

Review of past ECA appraisals

Based on our search, atidarsagene autotemcel (Libmeldy^®) and onasemnogene abeparvovec (Zolgensma^®) were found to be the only nononcology products with ECA-relevant HTA appraisals in England, France, Germany and Italy [21–30]. Both products are one-time gene therapies indicated for patients with rare infantile or juvenile-onset genetic abnormalities, i.e., metachromatic leukodystrophy and spinal muscular atrophy, respectively [21,22]. Atidarsagene autotemcel and onasemnogene abeparvovec received conditional EMA marketing authorization on 17 December 2020 and 18 May 2020, respectively, on the basis of nonrandomized trials [31,32]. The registrational study for atidarsagene autotemcel included a natural history study using prospectively collected RWD as a control arm; the main efficacy data for onasemnogene abeparvovec were derived from a phase III SAT, which was complemented by descriptive comparisons with two natural history studies that used retrospectively collected RWD [21,22].

England

NICE’s highly specialized technologies guidance for atidarsagene autotemcel expressed concern about gaps in baseline clinical data and the lack of baseline data from the natural history cohorts. However, the evaluation committee accepted that the product showed a clear advantage in slowing disease progression and likely significantly improved quality of life and mortality. It therefore recommended atidarsagene autotemcel for reimbursement in routine practice through a patient-access scheme (PAS) discount [33].

In the absence of direct comparisons for the appraisal of onasemnogene abeparvovec, the natural history studies played a key role in providing economic modeling assumptions for the best supportive-care arm, with NICE preferring data from the natural history studies rather than assumptions from clinicians. Reimbursement for the drug was recommended through a PAS discount [27].

France

The HAS Transparency Committee provided no specific critiques on ECA use for the atidarsagene autotemcel study, presumably because it was prespecified in the main study protocol and data were collected from the same clinic. However, for the population that was assessed, HAS designated a moderate clinical added-value rating (ASMR III) [25].

In the assessment of onasemnogene abeparvovec, HAS accepted the co-primary and key secondary end points from the phase III SAT, particularly given the presence of formal thresholds for statistical significance based on prespecified analyses from a prospective natural history study. The Transparency Committee considered the other secondary end points as exploratory and accepted the nonpopulation-adjusted indirect treatment comparison versus nusinersen based on clinical trial data only for showing similar efficacy. Results could not be sufficiently adjusted due to the small sample sizes and poor overlap in study populations, limiting the possibility of mitigating confounding issues. As a result, HAS designated onasemnogene abeparvovec, like nusinersen, with a moderate clinical added-value rating (ASMR III) versus best supportive care [29].

Germany

In the benefit assessment of atidarsagene autotemcel, the G-BA scrutinized the differences in participant populations between the registration trial and the ECA. However, the G-BA later published two addenda to the benefit assessment, accepting the ECA and awarding a major benefit rating since a sibling comparison was possible. This was the first time the G-BA awarded a quantifiable added benefit on the basis of an ECA comparison [27].

In the onasemnogene abeparvovec assessment, the G-BA evaluated only the indirect comparison to nusinersen, as the submitting company did not conduct an indirect comparison with best supportive care using the natural history studies. IQWiG and the G-BA analyzed the populations of the onasemnogene abeparvovec and nusinersen studies patient by patient and were not able to conclude additional benefit, since potential advantages (i.e., low hazard ratios potentially indicating a ‘dramatic effect’, which is an effect size so large that it more than compensates for the risk of bias) depended on uncertain assumptions of comparability between individual patients. As a result, onasemnogene abeparvovec is now the first product to undergo mandatory routine data collection in Germany, which is expected to provide the necessary RWD for comparison with relevant clinical practice data [28].

Italy

The AIFA technical report on atidarsagene autotemcel did not critique ECA use as presented in the European Public Assessment Report [21,26]. The lack of critique likely stemmed from the fact that, in sensitivity analyses, the incremental cost-effectiveness ratio (ICER) was minimally impacted (less than 2 percent) when inputs for control-arm effectiveness were varied according to published data from three other natural history studies. Importantly, however, the most impactful assumptions in the economic model were apparently elicited from clinical experts [26].

In the AIFA technical report on onasemnogene abeparvovec, two natural history studies were accepted as sources of effectiveness data for the control arm, but neither study was discussed in detail nor included in the sensitivity analyses. The comparison with nusinersen seemed to have had more impact on the ICER and was subject to additional scrutiny. Many of the ICER test scenarios were sourced from HTA reports from other agencies, such as NICE and the US Institute for Clinical and Economic Review [30].

Overall, AIFA seemed open to various approaches for deriving economic modeling assumptions from RWE, but no single option was preferred. AIFA acknowledged therapeutic innovativeness for atidarsagene autotemcel and onasemnogene abeparvovec despite the low quality of evidence, with downgrades in both cases due to lack of generalizability (i.e., the ECAs differed from the target studies in eligibility criteria) and imprecision (i.e., study outcomes featured wide confidence intervals due to small sample sizes) [26,30].

Expert insights

Based on our criteria, we identified six experts: two in England, two in France and one each in Germany and Italy.

Perceived acceptability of ECA use in HTA submissions

Figure 2 summarizes expert ratings evaluating the acceptability of ECA use in HTA submissions. Experts provided responses to five key questions based on a 5-point scale, with five representing ‘acceptable’ and 1 ‘not acceptable’. In addition to the quantitative ratings, we also analyzed the expert verbal feedback, which is summarized below.

Figure 2. . Expert ratings assessing the acceptability of ECAs in HTA evaluations, including their inclusion in evidence submission packages (in general) and their use in specific situations (i.e., to supplement RCTs as opposed to SATs, in therapeutic areas outside oncology and rare diseases, when clear justification for ECA use is provided, and relative to other indirect-comparison methods).

5 = Acceptable; 4 = Somewhat acceptable; 3 = Neutral; 2 = Somewhat not acceptable; 1 = Not acceptable.

ECA: External control arm; HTA: Health technology assessment; RCT: Randomized controlled trial; SAT: Single-arm trial.

Overall, experts agree that ECAs may add value to HTA submissions but face high hurdles for acceptability. The use of ECAs to supplement fully powered RCTs is typically less accepted than in SATs. ECA use is also typically less accepted in indications beyond oncology or rare diseases (e.g., chronic diseases), where lack of relevant comparators and ethical concerns are not as common. According to experts' feedback, agencies typically prefer RCT evidence (even indirect treatment comparisons using published RCT evidence) given their greater familiarity with (and the robust nature of) RCT methodology.

There are differences between countries, however. According to one expert in Italy, HTA authorities are typically more accepting of ECA use with historical RCT data than with RWD because of preferences for RCT evidence. In England, HTA experts view ECA use with historical RCT or RWD with a similar level of acceptability. In Germany, one expert emphasized that the acceptance of ECAs does not depend on the therapeutic area but on the presence of a dramatic effect. Note that IQWiG defines ‘dramatic effect’ as a statistically significant observation at p ≤ 0.01 that exceeds a tenfold increase or decrease in relative risk [34]. All experts recommended that submissions include language justifying ECA use, even though justification may not influence acceptability in France or Germany.

Situations in which agencies would consider ECA use acceptable

Figure 3 summarizes experts' ratings evaluating situations where ECAs best contribute to the HTA submission package. Experts provided responses to seven specific situations based on three ratings: ‘ECA use considered’, ‘ECA use not accepted’ and ‘N/E’ (ECA use not evaluated).

Figure 3. . Expert ratings assessing situations where external control arms best contribute to the health technology assessment submission package.

ECA: External control arm; N/E: Not evaluated.

According to experts, use of ECAs is typically acceptable only in specific situations, which include the impossibility of identifying a relevant comparator or lack of data for comparison, situations where ethical concerns prohibit use of direct comparisons, in subgroup analyses or small sample sizes, and in economic comparisons where RWD can inform cost-effectiveness analyses. According to experts, HTA authorities in France, Germany and Italy typically regard ECA use as unacceptable in RCTs with inappropriate comparators; in such situations, researchers would need to provide a strong justification supporting ECA use, such as rapidly changing standard of care. In Germany, one HTA expert highlighted that ECA use is unacceptable in most situations, except for the presence of a dramatic effect or the lack of a comparator in rare disease studies. The G-BA only accepts very closely matched ECA studies and only after considering the risk of confounding to be minimal for each individual patient included in the analysis. Any unexplained missingness or dependence on assumptions that go beyond the data appear to cause the G-BA to reject ECA studies.

Considerations for designing an ECA (i.e., drivers of ECA acceptability)

Figure 4 summarizes expert ratings assessing the importance of ECA design elements. Experts provided responses to nine key ECA design elements based on a 5-point scale, with 5 representing ‘Important’, 4 ‘Somewhat important’, 3 ‘Neutral’, 2 ‘Somewhat not important’, and 1 ‘Not important’.

Figure 4. . Drivers of external control arm acceptability.

5 = Important; 4 = Somewhat important; 3 = Neutral, 2 = Somewhat not important, 1 = Not important.

According to experts, researchers must justify their choice of source data when designing an ECA, ensuring that justification excludes any commercial motives. For HAS submissions, researchers must use an appropriate French database or, if applicable, justify its omission. In their justification, researchers should describe how RWD emulate patient characteristics, disease stage, treatment pathways and comorbidities with RCT data. They should also explain the reasons for the performance gap (i.e., expected differences in efficacy between RCT and RWD).

According to experts, when designing an ECA, researchers should ensure homogeneity with the target RCT in patient characteristics and treatment pathways. In general, researchers should avoid mixing data from various global regions but can do so as long as data homogeneity is maintained and an adequate rationale is provided. Researchers must also explain how a lack of specific end points or patient characteristics can affect the analysis, describing the reasons for missing data and ensuring the randomness of missing data.

Researchers must include efficacy end points when designing an ECA. In Germany, end points must be patient-relevant (e.g., mortality, morbidity, side effects and health-related quality of life) [35], and in England, researchers should aim to include quality-of-life end points, since NICE focuses on cost-effectiveness.

Lastly, researchers must minimize bias as much as possible, prioritize data accuracy as an important design element, and ensure transparency and reproducibility in reporting when performing an ECA study.

Discussion

This study assessed HTA perceptions, prior submissions and published guidance to advance a common understanding of ECA use and acceptability. Overall, the current acceptability of ECA use in HTA submissions remains limited (Figure 2). The exception is England (Figures 2–4), where alternative evidence sources beyond RCTs are considered acceptable with valid justification (Figure 2). Based on our guidelines review, HTA agencies can accept ECA use in the absence of a comparator in clinical trials (Figure 3) but require certain drivers of acceptability, including source justification, pertinent data/end point availability, bias minimization and trial emulation when evaluating submission packages (Figure 4). Unlike other agencies, IQWiG considers ECA use acceptable only in limited contexts, requiring the presence of a dramatic effect as a key determinant (Figure 3). Our finding of limited acceptability aligns with the results of a recent analysis of seven ECA case studies in oncology by Jaska et al. [14].

Based on consolidated expert ratings by country (Figures 2–4), the level of detailed guidance provided by national HTA agencies (Figure 1), and evidence from common appraisals, our findings suggest that agencies accept ECA use in certain submissions. ECA use remains limited outside oncology, as our search identified only two independent nononcology product appraisals common across European agencies.

Among agencies, however, NICE in England is most accepting of RWE use (Figures 2 & 3). A recent systematic review reported similar findings [36]. The RWE framework expresses NICE’s aspiration to use RWD to address knowledge gaps and promote patient access to therapies [15], which may explain the comparatively greater RWE acceptance in England as compared with France and Germany.

The importance of ECA-use justification as an acceptability catalyst in HTA submissions represents a key finding in our study (rating of 3.5 out of 5; Figure 2). As Griffiths and colleagues [37] point out, ECA acceptability increases if researchers provide an adequate justification for using a RWD ECA and demonstrate a convincing (or ‘dramatic’) treatment effect. While the importance of justification applies irrespective of therapeutic area, it is especially critical for indications beyond oncology and rare disease – areas in which, according to our interview findings, ECA use is still not accepted (Figure 2).

According to unanimous expert ratings (Figure 3), ECAs best contribute to HTA evidence packages when used to supplement SATs. This is consistent with practical and methodological guidance from the Member State Coordination Group for Health Technology Assessment (HTA CG), which details considerations for using ECAs in nonrandomized comparisons [5,6].

Our findings show that bias minimization represents another key driver of ECA-use acceptability across agencies (Figure 4). This finding reflects the results of other published research, particularly by Jaska et al. [14], which shows that, for the studies that elicited agency feedback, recommendations were mainly centered on the importance of mitigating selection bias and unmeasured confounding. Gray et al. [4] identify biases commonly affecting ECA use and outline a stepwise approach for addressing bias, including assessing data similarity, potential confounding variables, the use of quantitative bias analysis, or the use of Bayesian dynamic borrowing. NICE and HAS both provide methodological guidance to address bias [15,16]. To account for bias minimization, guidance from the HTA CG recommends rigorous adjustment for all known confounders in a prespecified and transparent manner, and shifting the null hypothesis to a threshold large enough to account for any plausible unknown or missing confounder [5,6].

Experts also identify trial emulation as another key driver of ECA acceptability (Figure 4). Trial emulation can help minimize any differences in patient characteristics, disease stage, treatment history, current treatment and comorbidities between the ECA and target study [38]. Relevant guidance for implementation has been published by Vanier and colleagues [17] on early, systematic source-data identification to help ensure the best fit for the target study, while Thorland and colleagues [9] provide details for optimizing fit, which include ensuring similarities in patient demographic characteristics, measured outcomes, conducting sensitivity analyses to estimate the effect of ECA limitations on outcomes and evaluating potential biases that may be apparent in the dataset.

While ECA-related HTA decisions primarily occur on a case-by-case basis, a collaborative approach to RWD use early in clinical development [17] will further drive use of RWE in HTA [39]. NICE, HAS and IQWiG each provide early scientific advice services, which researchers can use to receive feedback on optimizing RWD use in evidence generation [15]. HTA agencies can consider leveraging knowledge gained from scientific advice experiences to publish additional guidance on ECA use. As the uptake of ECA for HTA increases and the development and adoption of analytical methods for ECA evolves [40], HTA agencies’ guidance on ECA use is also expected to evolve over time [41].

Conclusion

Our findings provide insights for researchers seeking to maximize the likelihood of ECA use and acceptability in HTA submissions beyond oncology and rare disease studies. To drive acceptability, researchers should provide detailed and transparent ECA-use justification to HTA agencies; engage with HTA agencies to discuss trial emulation in terms of the target study; follow appropriate ECA design and implementation of methodologies to minimize bias; adhere to RWE reporting requirements using appropriate checklists; and demonstrate a dramatic effect if submitting in Germany. While RCTs remain the gold standard in HTA, ECA use can complement (rather than replace) robust study designs by providing additional evidence to better inform decision-making.

Summary points

• This study was conducted to assess the acceptability of external control arm (ECA) use in health technology assessment (HTA) submissions beyond its application in oncology and rare diseases.

• Researchers reviewed published guidance from three HTA agencies in Europe (NICE in England, HAS in France and IQWiG in Germany) to understand the availability of ECA related guidance.

• Researchers also analyzed two independent, nononcology HTA appraisals common across England, France, Germany and Italy, and conducted six double-blinded interviews with HTA experts from these countries.

• This study found that while NICE and HAS provide some guidance on ECA use, IQWiG provides limited guidance.

• ECA use is mainly confined to oncology, with NICE being more open to using ECAs beyond oncology to supplement clinical trial data.

• Experts believe that ECA use is acceptable when justified with valid data sources and rigorous methodology to minimize bias.

• There is a need to focus awareness on the value of ECA use as a supplement to fully powered RCTs – particularly in generalizing efficacy and safety outcomes to broader real-world patient populations, avoiding the issue of losing patients to follow-up, evaluating longer-term outcomes after RCT completion, and validating reference values for proxy variables in real-world evidence studies.

• Engaging HTA agencies early in clinical development is important for the successful implementation of ECAs.