External Control Arms in Japan: A Systematic Case Analysis of Documented Regulatory Implementation and Uncertainty Management

Abstract

External control arms (ECAs) have emerged as important tools to provide benchmark evidence on comparator effectiveness and thereby support regulatory decision making in situations where randomized controlled trials are difficult to conduct. However, how key methodological elements for ECAs are documented and implemented in practice and how residual uncertainty is managed at the time of approval remain unclear. Regulatory approvals of new medical products in Japan between 2019 and 2024 were systematically screened to identify cases employing ECAs. Among 674 eligible approvals screened from 835 approvals identified, 23 cases using ECAs were included and evaluated using a structured framework of 20 key attributes derived from regulatory guidance and methodological recommendations across planning, selection, design, analysis, and reporting domains. The post‐marketing requirements associated with each approval were also analyzed to assess how residual uncertainty was addressed following approval. ECAs were predominantly used in settings characterized by rare diseases and small patient populations. Although attributes related to study planning and data‐source selection were relatively well described in public documentation, there was substantial heterogeneity in the documentation of design and analysis elements. In particular, explicit specifications of estimands and approaches to handling intercurrent events and missing data were inconsistently described. Residual uncertainty was frequently addressed through post‐marketing requirements, often involving studies of efficacy considerations and long‐term surveillance, supporting a lifecycle‐based approach to uncertainty management. These findings highlight opportunities to enhance transparency and consistency in the documentation of ECA‐based evidence and to further develop structured approaches to uncertainty management in regulatory practice.

Study Highlights.

• WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

External control arms (ECAs), often derived from real‐world data, patient registries, or historical clinical trials, have emerged as potential tools to aid the analysis of comparator drug effectiveness and thereby support regulatory decision making in situations where randomized controlled trials are difficult to conduct. However, how key methodological elements for ECAs are documented and implemented in practice and how residual uncertainty is managed at the time of approval remain incompletely understood.

• WHAT QUESTION DID THIS STUDY ADDRESS?

We conducted a systematic evaluation of regulatory approvals of new medical products in Japan between 2019 and 2024, based on systematic screening and identification of cases in which ECAs were used to support regulatory decision making. Using a structured framework, we evaluated how ECAs were documented across the planning, selection, design, analysis, and results‐reporting domains. In addition, we examined associated post‐marketing requirements to understand how residual uncertainty was addressed following approval.

• WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?

ECAs were predominantly used in settings characterized by rare diseases and small patient populations. Although attributes related to study planning and data‐source selection were relatively well described, there was substantial heterogeneity in the design and analysis domains. In particular, explicit specifications of estimands and approaches to handling intercurrent events and missing data were inconsistently described. Residual uncertainty was frequently addressed through post‐marketing requirements.

• HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?

The findings highlight opportunities to enhance transparency across the planning, data‐source selection, design, analysis, and reporting of ECA‐based evidence and to further develop structured approaches to uncertainty management in regulatory practice.

Therapies targeting rare diseases and small patient populations present an increasing challenge to conventional paradigms of evidence generation. ¹ Randomized controlled trials (RCTs) may be infeasible or unethical in these settings, prompting a growing need for alternative sources of comparative evidence. External control arms (ECAs), representing data derived from sources external to the investigational trial, such as real‐world data (RWD), patient registries, or historical clinical trials, have emerged as potential tools to aid the analysis of comparator drug effectiveness and to support regulatory decision making. ²

Regulators have increasingly acknowledged the potential role of RWD and ECAs in supporting drug development and evaluations. ³ , ⁴ , ⁵ , ⁶ , ⁷ Guidance documents and methodological recommendations from the US Food and Drug Administration (FDA), ⁸ the Medicines and Healthcare products Regulatory Agency (MHRA) of the United Kingdom, ⁹ the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH), ¹⁰ and the Pharmaceuticals and Medical Devices Agency (PMDA) in Japan ¹¹ have outlined key considerations for the appropriate use of ECAs, including issues related to the selection of data source(s), comparability of patient populations, validity of clinical outcomes, controlling bias and confounding, management of missing data, and transparency in reporting based on prior consultation with the regulatory authorities. Various efforts have been made to examine planning and methodological frameworks for ECA studies, ¹² share lessons learned from practical applications, ¹³ , ¹⁴ and examine the feasibility of their implementation. ¹⁵ These frameworks and initiatives reflect a growing recognition that evidence generation may need to adapt to practical and ethical constraints in certain contexts, while maintaining scientific rigor.

Despite these developments, empirical evidence on the documentation and implementation of regulatory expectations for ECAs in practice remains limited. Liu et al. reported suboptimal practices in the design, conduct, and analysis of ECA studies, identifying several critical methodological issues, including lack of justification for the use of ECAs, failure to prespecify ECAs in the study protocols, insufficient confounding adjustment, and limited use of sensitivity analyses in a cross‐sectional evaluation of 180 externally controlled trials. ¹⁶ However, although these findings provide important insights into methodological challenges, they do not assess whether or how such practices have evolved over time or in response to emerging regulatory guidance.

Likewise, the use of ECAs in regulatory submissions has been described across several jurisdictions, ² , ⁷ , ¹⁴ but less is known about how key methodological elements are documented and implemented in practice and how residual uncertainty is managed at the time of approval. Notably, the extent to which decision‐critical attributes, such as the rationale for using ECAs, selection of data source(s), comparability of patient populations, validity of outcomes, and statistical approaches to control bias and confounding, are explicitly articulated in regulatory documentation remains unclear. Furthermore, how such uncertainties are subsequently addressed through post‐marketing requirements has not been systematically examined.

To address these gaps, we conducted a systematic evaluation of regulatory approvals of new medical products in Japan, based on screening and identifying cases in which ECAs were used to support regulatory decision making. Using a structured framework of key attributes derived from regulatory guidance and methodological recommendations, we evaluated how ECAs were documented across planning, selection, design, analysis, and results‐reporting domains. We also examined the post‐marketing requirements of the identified cases to understand how residual uncertainty needed to be addressed following approval.

MATERIALS AND METHODS

Study design and data sources

This study was a systematic case analysis of ECAs used to support new medical products approvals. All types of regulatory approvals of new medical products in Japan, including new drugs and regenerative medical products, approved by the Ministry of Health, Labour and Welfare between January 1, 2019, and December 31, 2024, were included. The study period was selected to reflect contemporary regulatory practice following the increasing formalization of guidance on the use of RWD and ECAs, while ensuring sufficient availability and completeness of publicly accessible regulatory documentation.

Among the identified approvals, we excluded approvals for public knowledge‐based applications and applications for biosimilars and similar ethical combination drugs, considering that these applications were based on already established efficacy and safety profiles. We also excluded approvals for applications related to severe acute respiratory syndrome coronavirus 2 virus infection and vaccines, because many of these were subject to special review processes during the coronavirus pandemic. Information sources were based on publicly accessible documents, including summary technical documentation and review reports identified in the medical drug ¹⁷ and regenerative medical information search systems ¹⁸ provided by the PMDA.

In addition to Japanese regulatory documentation, supplementary and publicly available materials from other regulatory authorities, including the European Medicines Agency (EMA) and the US FDA, were reviewed to provide contextual understanding of ECA use, and to support the interpretation of publicly available case information, rather than replace the primary regulatory documentation. The assessment framework was applied consistently across cases, focusing on the presence and clarity of methodological elements described in publicly available sources. The current evaluation thus reflects the extent to which key attributes were documented in publicly accessible regulatory sources, including summary technical documentation submitted by the sponsor and review reports provided by the PMDA, rather than directly assessing the underlying methodological adequacy. Therefore, the analysis should be interpreted as an assessment of publicly documented implementation of ECA‐related methodological attributes. Attributes that were considered during development or regulatory review but not described in publicly available sources may not have been captured.

Post‐marketing requirements associated with each approval were also extracted from regulatory documentation, mainly review reports and risk‐management plans for individual products, to examine how residual uncertainty at the time of approval needed to be addressed. These included obligations related to efficacy evaluations, long‐term surveillance requiring > 5 years of follow‐up during reexamination periods or longer, all‐case surveillance, numbers of studies planned, and study type, where applicable.

Identification of ECA cases

Approvals were reviewed to identify cases in which ECAs were used to support clinical evidence in regulatory submissions. ECAs were defined as comparator cohorts derived from sources external to the clinical trials (mostly single‐arm trials) of the tested treatment, including historical clinical trials, patient registries, or RWD sources (e.g., routinely collected information extracted from electronic health records or healthcare claims). Although claims data may be used in certain contexts, their application in ECAs may be limited by a lack of detailed clinical outcomes and key prognostic variables required for robust confounding adjustment, and their suitability thus depends on the clinical context and study objectives. Among the identified ECA cases, we selected cases that were explicitly described as part of the clinical‐data package within the applications. Selected cases were further divided into those submitted in relation to the evaluation materials and those submitted as reference materials, categorized as “main” and “supportive” evidence, respectively.

Development of evaluation framework

We assessed the implementation of ECAs in regulatory practice by developing a structured framework of key attributes based on regulatory guidance and methodological recommendations related to the use of ECAs. We identified the draft guidance of design and conduct of externally controlled trials by the FDA ⁸ and the MHRA of the United Kingdom ⁹ up to December 12, 2025 (data cutoff point). The ICH E10 guidance regarding the choice of control group and related issues in clinical trials ¹⁰ and the early consideration notice concerning externally controlled trials issued by the PMDA ¹¹ were also included. The EMA also published a draft plan for a reflection paper on ECAs. ¹⁹ Regulatory activity within the EMA has further progressed since the data cutoff, including a multi‐stakeholder workshop held in November 2025 to inform the development of this reflection paper; however, at the time of framework development, the document mainly outlined the purpose and timeline and was therefore not included in the framework. Because some guidance documents and methodological recommendations were published after part of the approval period, the framework was used to assess the extent to which ECA‐related methodological attributes were documented in relation to current and emerging expectations, rather than to judge compliance with requirements in force at the time of each approval. Table 1 shows the developed evaluation framework. The framework consisted of 20 attributes organized into five domains: planning, selection, design, analysis, and results reporting. The planning domain included attributes related to alignment with regulatory context, pre‐specification of study protocol, and an attribute related to estimand specification. The selection domain focused on data‐source identification and evaluation. The design domain addressed elements including cohort definition, comparability of the study population, treatment data collection, time‐zero alignment and observability, outcome evaluation, and feasibility study. The analysis domain included attributes related to statistical analysis plan, matching methods, sensitivity and supplementary analysis, and management of missing data. The results‐reporting domain assessed other considerations related to results reporting, including the interpretation of effect size and reporting results using multiple ECAs.

Table 1.

Evaluation framework for cases using external control arms and correspondence with each guidance and methodological recommendation

Domain	Component	Descriptions of attributes	FDA (2023) 8	PMDA (2025) 11	MHRA (2025) 9	ICH (2000) 10
Plan–1 (P–1)	Consultation with regulators	Sponsor consulted with regulators early in drug development program for plans to conduct externally controlled trial instead of RCT	Included	Included	Included
Plan–2 (P–2)	Rationale for ECA approach	Sponsor provided detailed descriptions regarding use of ECA approach: Reasons why ECA approach was appropriate Proposed data sources for ECA Planned statistical analysis methods RCT should be conducted whenever possible as clinical trials that provide primary evidence of efficacy and safety	Included	Included	Included	Included
Plan–3 (P–3)	Prespecified study protocol	Study protocol finalized before externally controlled trial started: Selection of ECA and analytic approach All key design elements including data sources, eligibility criteria, exposure definitions and windows, study endpoints, analytic plans, and approaches to minimize missing data and sources of bias Adequate justification needed if ECA identified after trial finished	Included	Included	Included	Included
Plan–4 (P–4)	Estimands specification	Estimand framework considered when treatment effect of interest in externally controlled trial specified	Included	Included	Included
Selection–1 (S–1)	Justification of data sources prior to the study	Selection of data source justified before deciding if externally controlled trial was suitable design to answer research question: Consideration of data quality, reliability, and suitability of data for intended comparison Confirmation of availability of important prognostic characteristics	Included		Included
Selection–2Ca (S–2C)	Evaluation of data sources from other clinical trials	Consideration points when using data from other clinical trials: Comparability between two trial arms regarding participant eligibility criteria, treatment administration, patterns of care (e.g., sites locations), recording of concomitant medications, and assessments of adverse events and outcomes Extent and reasons for any missing data Acceptability of using clinical trial data conducted in different time periods based on changes in disease assessment and management Justification of selection of specific clinical trial for ECA without arbitrariness	Included	Included
Selection–2Ra (S‐2R)	Evaluation of data sources from RWD	Consideration points when using RWD such as registries: Comparability of patient characteristics, timing and frequency of data collection, and patterns of care Missing information related to important confounding factors and prognostic characteristics Sufficient number of patients with disease of interest receiving relevant treatment General considerations and guidelines for registry use (e.g., personal information protection, data quality, reliability, and acceptability to individual patient data)	Included	Included	Included
Selection–3 (S–3)	Consideration of time periods	Differences in timing of data collection between test treatment arm and ECA considered: Changes in assessment and management of disease (including supportive care) such as methods of diagnosis, staging, treatment, etc. Concurrent control in prospectively collected data rather than historical external controls generally preferable Consideration of if and how different time frames in each arm impact interpretability of results	Included	Included	Included	Included
Design–1 (D–1)	Comparability of the study population	Comparability of participants/patients between test treatment arm and ECA carefully considered: Evaluation of relevant confounding factors and appropriateness of data‐collection method Application of eligibility criteria of externally controlled trial in a manner that allows for selection of similar patients in test treatment arm and ECA Specified plans to select ECA as similar as possible to test treatment arm without arbitrariness including matching method if used	Included	Included	Included	Included
Design–2 (D–2)	Geographic, ethnic, and other factors	Consideration of differences in geographic, ethnic, and other factors related to healthcare system and environment given in externally controlled trial to reduce impact of confounding based on such differences and explain applicability of results in local healthcare settings	Included	Included
Design–3 (D–3)	Treatment data collection	Potentially important imbalances with respect to treatment between test treatment arm and ECA identified and adequately accounted for: Comprehensive details describing administration of treatment such as dosage regimen, treatment duration, treatment adherence Information on concomitant or supportive therapies Differences in medical care received in clinical trials and clinical practice Management of treatment‐ or disease‐related adverse events Additional factors influencing treatment and delivery of care	Included	Included	Included
Design–4 (D–4)	Index date and observation period	Determination of index date consistent, and durations of follow‐up period comparable between test treatment and ECA: Careful determination of index date to avoid including immortal time in both arms Consideration of clinical context related to assigning index date, eligibility criteria, and data availability across compared arms Confirmation that durations of follow‐up periods comparable across compared arms	Included	Included	Included	Included
Design–5 (D–5)	Outcomes evaluations	Bias due to unblinded treatment assignment and differences in evaluation environment between test treatment arm and ECA carefully considered: Using objective, clearly defined, clinically meaningful outcomes that address study objectives Consistent assessment of outcomes across compared arm considering differences in sources of data or criteria used to establish outcomes Consideration of potential impact due to the lack of standardization and training in definitions and use of certain clinical outcomes assessments in routine clinical settings Consideration of outcome evaluations blinded to treatment status such as independent central review whenever possible and for suitable endpoints	Included	Included	Included	Included
Design–6 (D–6)	Intercurrent events	Differential capture of intercurrent events with potential to bias estimated effect of treatment adequately accounted for: Evaluation of extent of information available regarding occurrence of relevant intercurrent events and impact on outcome assessment due to different capture of intercurrent events across compared arms Using consistent definition and handling of intercurrent events across compared arms	Included	Included	Included
Design–7 (D–7)	Feasibility study	Feasibility study may be conducted to determine expected sample size and availability of important data‐collection items: Assessment of extent of missing data and sample size in ECA Conducting detailed investigations into endpoints and related items not recommended Recoding timing and content of feasibility study	Included	Included
Analysis–1 (A–1)	Prespecified analysis plan	Pre‐specified analysis plan developed including necessary assumptions and details of analysis method: Submission of statistical analysis plan with protocol to regulators before initiation of clinical trial Measures to avoid arbitrary selection of statistical analysis methods If the analysis plan was changed for unavoidable reasons, any revisions date‐stamped, and corresponding rationale and content clearly provided and discussed with regulators Adequate details of analysis plan included plans to select study population, assess similarity across compared arms, and conduct intergroup comparisons with adjustment for confounding factors	Included	Included	Included
Analysis–2 (A–2)	Matching methods	Details of matching methods prespecified: Covariates and estimation methods used to compute propensity score Criteria for determining imbalance across compared arms and methods to recalculate score when adequate balance would not be achieved	Included	Included		Included
Analysis–3 (A–3)	Sensitivity and supplementary analyses	Potential sources of bias and important confounding factors carefully considered and examined in prespecified analysis plan: Sensitivity and supplementary analyses to evaluate extent of confounding and bias Explicit descriptions regarding assumptions involved in statistical methodologies and sensitivity analyses and model diagnostics to examine such assumptions	Included	Included	Included
Analysis–4 (A–4)	Consideration of missing data	Potential impact of missing data carefully considered and examined: Evaluation of lack of information on events that cause missing data (e.g., intercurrent events, type and frequency of assessments, disenrollment, or other reasons) Strategy for dealing with missing data such as methods for missing data imputation Consideration of sensitivity and supplementary analyses to evaluate potential impact of deviations from statistical assumptions including assumption about missing data mechanisms	Included	Included	Included
Results reporting–1 (R–1)	Large effect size on well‐characterized outcomes	Externally controlled trials more likely to provide convincing results when effect size of well‐characterized outcome on treatment is large: Consideration of anticipated effect size based on available scientific data Prespecified plans for assessing impact of confounding factors and bias	Included		Included	Included
Results reporting–2 (R–2)	Replicating results using multiple ECAs	Replicating results using multiple ECAs may be used where no obvious single optimal external control exists: Concordance of estimated effect of treatment from different data sources may help address concerns around unmeasured confounding factors Analyses should be prespecified, standardized, and should control type I error			Included	Included

ECA, external control arm; FDA, Food and Drug Administration; ICH, The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use; MHRA, Medicines and Healthcare products Regulatory Agency; PMDA, Pharmaceuticals and Medical Devices Agency; RCT, randomized controlled trial; RWD, real‐world data.

^a Applied to each case depending on data source used. S–2C applied to cases using other clinical trials for ECAs; S–2R applied to cases using RWD.

Assessment of documented attribute implementation

Each case was evaluated against the predefined framework to determine if individual attributes were addressed in regulatory documents or associated publications. Attribute implementation was categorized as “Documented as addressed,” “Documented as partially addressed (with some items not documented or unaddressed),” “Documented limitations,” or “Not described in public sources” based on the level of detail provided in publicly available sources. “Documented limitation” was assigned when the absence or incomplete handling of an attribute was explicitly identified as a study limitation by the sponsor or in the review report. The assessment focused on the presence and clarity of information relevant to each attribute rather than on the methodological appropriateness of specific approaches. The assessment was mainly conducted by the authors, with the use of a predefined framework, standardized classification criteria, and iterative cross‐checking against source materials to minimize subjectivity and enhance consistency in interpretation.

Data analysis

The characteristics of the included cases and extent to which key attributes were implemented across domains were summarized using descriptive analyses. Numbers and percentages of cases were summarized based on the types of applications, information on pediatric use, orphan drug designation status, Anatomical Therapeutic Chemical (ATC) classification, data sources, and the purpose of ECA use in all identified cases. The frequency and distribution of attribute implementation were assessed across cases, and patterns of variability were represented visually using heatmaps. Cases were stratified as main or supportive evidence to evaluate the documented attribute implementation considering differences in the context of use and positioning within the applications.

RESULTS

A total of 835 approvals for new drugs and regenerative medical products between January 1, 2019, and December 31, 2024, were identified, of which 674 approvals that met the eligibility criteria were screened for the use of ECAs. Twenty‐six approvals involved studies with ECAs, of which three that did not use ECAs in their clinical packages were excluded. Twenty‐three cases were therefore included in the analysis (Figure S1 ). The characteristics of the included cases are summarized in Table 2 . Detailed information on the identified ECA cases, including drug name, target disease/indication, study population, comparators, and evaluated outcomes is listed in Table S1 . The use of ECAs was concentrated in settings involving rare diseases and small patient populations. The identified cases employed various types of outcomes, ranging from objective outcomes such as overall survival to clinical outcome assessments requiring standardization measures, including assessments of physical function, developmental testing, and symptom measurements.

Table 2.

Characteristics of cases using external control arms included in the study

	All cases (n = 23)
Approval year, n (%)
2019	3 (13.0)
2020	5 (21.7)
2021	2 (8.7)
2022	8 (34.8)
2023	2 (8.7)
2024	3 (13.0)
Application type, n (%)
New drugs or new regenerative medical products	16 (69.6)
Extension of indication	6 (26.1)
New route of administration	1 (4.3)
Orphan drug designation, n (%)	14 (60.9)
Oncology medical products, n (%)	5 (21.7)
Information for pediatric use, n (%)
Adult use	12 (52.2)
Pediatric and adult use	6 (26.1)
Pediatric use	5 (21.7)
Study category, n (%)
Main	15 (65.2)
Supportive	8 (34.8)
Data sourcea, n (%)
Primary data collection based on prospective cohort study	4 (17.4)
Primary data collection based on retrospective data	7 (30.4)
Registry studies	9 (39.1)
Secondary data sources based on electronic health records or claims data	3 (13.0)
Other clinical trials	5 (21.7)
Concurrent (or partially concurrent) data collection for ECA, n (%)	6 (26.1)
Purpose of using ECA, n (%)
Demonstration of efficacy	20 (87.0)
Demonstration of safety	0 (0)
Efficacy and safety	3 (13.0)
Type of external controla, n (%)
Drug	5 (21.7)
Placebo	4 (17.4)
Standard of care	11 (47.8)
No treatment	4 (17.4)
Phase of clinical trials for the test treatment arma, n (%)
Phase 1	2 (8.7)
Phase 1/2	3 (13.0)
Phase 2	9 (39.1)
Phase 3	12 (52.2)
Sample size of the test treatment arm, n (%)
< 30	9 (39.1)
30–100	9 (39.1)
101–300	5 (21.7)
> 300	0 (0)
ATC classification of product, n (%)
Alimentary tract and metabolism	4 (17.4)
Blood and blood forming organs	2 (8.7)
Anti‐infectives for systemic use	3 (13.0)
Antineoplastic and immunomodulating agents	8 (34.8)
Musculoskeletal system	3 (13.0)
Nervous system	3 (13.0)

ATC classification, Anatomical Therapeutic Chemical classification; ECA, external control arm.

^a Not mutually exclusive because multiple data sources were used in the same case.

Documented implementation of key attributes across individual cases

The documented implementation of key attributes across included ECA cases is shown in Figure 1 . For cases categorized in the main evidence, attributes related to planning and data‐source selection were described more consistently, whereas there was substantial variability in the design and analysis domains. Notably, explicit specification of estimands (P–4) was not identified in publicly available documentation across most cases in the main evidence, and approaches to handling intercurrent events (D–6) and missing data (A–4) were described inconsistently. This may partly reflect variability in reporting practices rather than an absence of methodological consideration; however, the limited visibility of these elements in regulatory documentation may affect the interpretability and reproducibility of ECA‐based analyses. The collection of treatment‐related information (D–3) was partially described in most cases; however, details related to concomitant or supportive therapies and treatment‐related adverse events were often limited. There were no documented instances of quantitative bias analysis among the identified cases. Matching methods (A–2) were commonly described when applied, but information on the criteria used to assess balance or approaches to address residual imbalance was limited.

Figure 1.

Documented implementation of key attributes across included cases. Results for ECAs categorized in (a) main evidence and (b) supportive evidence. Detailed information on identified ECA cases, including drug name, target disease/indication, study population, comparators, and evaluated outcomes is shown in Table S1 . ECA, external control arm.

Compared with the main evidence, cases in the supportive evidence tended to have lower documented attribute implementation in each domain; however, recent cases also showed a tendency for the implementation of a relatively high number of attributes. Case 22 employed the target trial emulation approach to compare data from a single‐arm trial with a historical cohort collected through a non‐interventional study and addressed attributes that were not adequately covered in other cases, such as explicit specification of estimands and handling of intercurrent events. ²⁰ Several cases explicitly identified limitations in attributes that were not implemented, including a case in which the expected sample size was not confirmed in advance, resulting in a substantially smaller sample size than required for ECA (Case 1), a case in which the historical cohort was also included in the treatment arm without statistical adjustment (Case 16), and a case with limited information on intercurrent events relevant to outcome measurement and substantial missing values for important prognostic factors (Case 17).

Across cases, several practical measures were taken to address uncertainties (Table 3 ). For instance, several cases used similar eligibility criteria, endpoints, data‐collection intervals, and trial sites or geographies between the investigational trial and the historical clinical trials intended to be used as ECAs, thereby increasing comparability in study population, treatment data collection, and endpoint measurements. Measures, such as training for assessors and selection of trial sites and data sources considering the capability of implementing clinical outcomes assessments, were taken to address the impact caused by the lack of standardization and training for the use of clinical outcome assessments. The central analysis and external review were blinded to comparison arms to address concerns regarding outcome consistency across compared arms and bias due to the unblinded treatment assignment. In some cases, historical data were collected at the same institutions that participated in the clinical trial of the test treatment, or the clinical trials of the test treatment were conducted at sites affiliated with the registry network. These approaches were used to mitigate uncertainty related to differences in patterns of care across institutions, healthcare systems, and environments.

Table 3.

Summary of practical measures used to address uncertainties in individual cases

Category	Uncertainties	Practical measures employed
Comparability	Comparability between two arms regarding participant eligibility criteria, treatment administration, concomitant treatment, management of adverse events, and patterns of care	Assessing eligibility of individual cases included in ECA by independent medical review committee
		Application of eligibility criteria with respect to concomitant treatment and important prognostic factors to select similar population to test treatment arm in ECA
		Designing externally controlled trial using similar eligibility criteria, endpoints, data collection intervals, and trial sites/geographies of historical clinical trials intended to be used for ECA
	Differences in medical care received in clinical trials and clinical practice	Designing clinical trial without imposing strict inclusion and exclusion criteria, thereby reducing potential bias of healthier population in the treatment cohort
		Contacting as many eligible patients as possible to offer opportunities to participate in clinical trial to include wide range of patients in the trial
Outcome validity	Objectivity and clinical significance of outcomes set in accordance with study objective	Employing established outcomes in regulatory guidance and clinical practice guidelines
	Impact due to lack of standardization and training for use of clinical outcome assessments	Training all assessors on clinical outcome assessments and performed assessment by the central review committee
		Ensuring that only trained assessors able to conduct outcomes assessments
		Selected data sources with standardized outcome assessment methods for ECA (e.g., registry studies with prospective data collection)
		Collecting historical data for ECA at institutions capable of performing outcomes assessments at same level as clinical trial
	Consistency of outcome assessment across compared arms considering differences in source of data or criteria used to establish outcomes	Conducting central analyses for assessment of outcomes (e.g., for outcome assessment based on imaging data)
	Bias due to unblinded treatment assignment and differences in evaluation environment	Organizing external data‐review committee conducting blinded comparison and evaluation of validity of outcomes
Bias control	Bias related to factors related to patterns of care at institutions, healthcare system, and environment	Collecting historical data for ECA at same institutions participating in clinical trial
		Conducting clinical trials at institutions affiliated with network of registry study and using registry data for ECA
	Differences in timing of data collection between test treatment arm and ECA	Collecting concurrent data for ECA and conducting analysis in line with schedule of clinical trial
	Possibility that other matching methods could achieve favorable alignment between arms compared with primary matching method	Conducting sensitivity analysis using several variations of primary matching method and presenting key summary statistics and treatment effects
	Potential bias and confounding factors not considered in analysis	Conducting sensitivity and supplementary analyses using different matching conditions or in different patient subsets, such as method intentionally designed to increase bias toward test treatment arm or ECA, and establishing multiple independent matching datasets for robustness
	Potential impact of missing data for results interpretation	Evaluating extent of missing data across compared arms and imputing missing data using external sources or a predefined method
		Reporting number and percentage of patients requiring imputation for missing data and evaluating effect of imputation on study results
	Differential capture of intercurrent events with potential to bias estimated effect	Specifying main and secondary estimands of study using ECA and explicitly reporting in submission documents
		Using consistent definition and handling of intercurrent events based on specified estimands
Data reliability and transparency	Data quality, reliability, and suitability of data for regulatory purposes	Using registry data developed with scope of utilization for regulatory purposes
		Consulting with regulatory guidance and authorities to develop standard procedures and documentation to fulfill requirement related to data quality
	Arbitrary selection of statistical analysis and matching methods with planner knowing a part of results	Submission of statistical analysis plan with protocol before initiation of clinical trial
		Conducting matching procedure between clinical trial and registry data independently from sponsor by physician blinded to clinical trial results

ECA, external control arm.

Post‐marketing requirements

Post‐marketing requirements following the approval of identified ECA cases are summarized in Table 4 . Of the identified cases, 22 (95.7%) required post‐marketing surveillance to be planned with planned sample sizes ranging from approximately several tens to a thousand patients depending on the anticipated treated population size. All‐case surveillance was planned in 17 (73.9%) cases. In addition, 11 (47.8%) cases required studies to further evaluate efficacy. Long‐term surveillance, defined as studies requiring more than 5 years of follow‐up during the reexamination period or beyond, was planned in nine (39.1%) cases, with one case requiring follow‐up extending to 15 years. The scope of post‐marketing requirements often included both efficacy and safety evaluations. Compared with cases that used ECAs for supportive studies, cases using ECAs as part of the main evidence package more frequently required post‐marketing studies for efficacy considerations and long‐term surveillance.

Table 4.

Post‐marketing requirements following approval of identified cases

	All cases (n = 23)	Main (n = 15)	Supportive (n = 8)
Number of post‐marketing surveillance types planned, n (%)
0	1 (4.3)	1 (6.7)	0 (0)
1	19 (82.6)	12 (80)	7 (87.5)
≥ 2	3 (13.0)	2 (13.3)	1 (12.5)
Required studies for efficacy considerations, n (%)	11 (47.8)	8 (53.3)	3 (37.5)
Required long‐term surveillance during the reexamination period or longer, n (%)	9 (39.1)	7 (46.7)	2 (25)
Types of post‐marketing surveillance planneda, n (%)
All‐case drug utilization survey	17 (73.9)	10 (66.7)	7 (87.5)
Drug utilization survey	5 (21.7)	4 (26.7)	1 (12.5)
Database study	2 (8.7)	1 (6.7)	1 (12.5)
Post‐approval clinical trial	1 (4.3)	1 (6.7)	0 (0)

^a Non‐mutually exclusive because multiple studies are planned for some cases.

DISCUSSION

In this study, we systematically evaluated ECA use cases that supported regulatory approvals of new medical products in Japan using a structured framework of key methodological attributes. Although analyses confined to a single jurisdiction may not capture all international variations, the ability to systematically evaluate all approvals within a defined period allowed a detailed examination of documented implementation and uncertainty‐management practices. ECAs were predominantly used in settings characterized by rare diseases and small patient populations. Although attributes related to study planning and data‐source selection were relatively well described for cases used in the main evidence, there was substantial heterogeneity in the design and analysis domains. Notably, estimands were only explicitly specified in one case (4.3%), and approaches to handling intercurrent events and missing data were inconsistently described. Detailed explanations of treatment‐data collection and matching methods were partially described, but some elements (e.g., concomitant or supportive therapies, treatment‐related adverse events, and criteria to evaluate imbalance across compared arms) were inconsistently explained. These findings suggest that key elements influencing the interpretability of treatment effects remain variably implemented in current practice. This variability likely reflects a combination of factors, including the evolving nature of methodological guidance for ECAs, differences in sponsor experience and analytical capabilities, and variable levels of detail in the regulatory documentation, rather than a single underlying cause.

Regulatory guidance and emerging practice indicate that ECAs are generally considered acceptable in specific contexts where RCTs are infeasible or unethical, most commonly in rare diseases or settings with high unmet medical needs. A consistent set of principles appears across guidance from regulatory authorities including the FDA, PMDA, and evolving perspectives from the EMA. These include early consultation with regulators, clear justification for the use of an external control instead of randomization, and the use of data sources that are demonstrably fit‐for‐purpose. Regulatory guidance consistently recommends early consultation with authorities regarding plans to use ECAs in drug development programs. ⁸ , ⁹ , ¹¹ In practice, such consultation is most beneficial when initiated at the earliest feasible stage, ideally sufficiently prior to the finalization of the pivotal clinical trial design, to allow for regulatory input on the appropriateness of the ECA approach, the selection of data sources, and the pre‐specification of analysis plans. In addition, alignment between the treatment arm and external control in terms of eligibility criteria, index date, follow‐up period, and outcome definitions is critical. Robust approaches to address confounding and bias, including appropriate matching or adjustment methods, as well as prespecified strategies for handling missing data and intercurrent events, are also essential. Sensitivity analyses play a key role in evaluating the robustness of the findings. Importantly, regulatory acceptability is typically determined on a case‐by‐case basis, taking account of the clinical context, expected treatment effect, and quality and completeness of the available data.

Variability among cases was most pronounced in domains directly related to the characterization and interpretation of treatment effects. Similar findings were reported previously, ¹⁶ including infrequent use of feasibility studies to evaluate the adequacy of data sources, limited specification of methods for handling missing data, inadequate descriptions related to confounding adjustment and sensitivity analyses, and almost complete absence of quantitative bias analyses. The benefit of using estimands that precisely describe the treatment effect, reflecting clinical questions posed by trial objectives, for causal inference in ECAs has been well recognized. ¹² , ²¹ Likewise, several studies have evaluated the utility of quantitative bias analysis for exploring sensitivity to unmeasured confounding in non‐randomized trials using ECAs. ²² , ²³ These concepts are being increasingly implemented in clinical studies, ²⁴ and their coherence in regulatory guidance and methodological recommendations ⁸ , ⁹ , ¹⁰ , ¹¹ means that these elements are expected to become increasingly recognized as important factors affecting the validity of comparative analyses using ECAs. The degree to which such considerations were explicitly articulated varied, suggesting that, despite underlying analytical considerations, their visibility in regulatory documentation remains limited. Recent efforts have also focused on developing structured frameworks to prospectively assess the viability and appropriateness of ECAs prior to their implementation, ²⁵ complementing the retrospective evaluation of documented practices in the current study. It is noteworthy that the majority of cases identified in this study predated the publication of several key regulatory guidance documents. As such, the variability observed in documented attribute implementation may partly reflect a period during which regulatory expectations for ECAs were still being formalized. As these guidelines become established, a more consistent approach to the design, conduct, analysis, and reporting of externally controlled trials is anticipated. Indeed, our analysis of recent cases suggests a trend toward more comprehensive documentation of key attributes.

The identified cases used various practical measures to manage residual uncertainties. Notably, several cases involving rare diseases utilized specialized clinical outcome assessments, such as symptom‐rating scales, physical function, and developmental testing, and the assessment of progression for rare tumor types. These cases required additional measures, such as the recruitment of sites/investigators capable of specialized assessments and central reviews of outcomes to ensure consistency in the assessment. Data reliability also required case‐specific consideration. Several efforts were also required to ensure data reliability, including the development of procedures and data‐collection systems through consultation with regulatory authorities and related guidance documents, ²⁶ suggesting the importance of the transparent and comprehensive evaluation of data reliability in terms of accuracy, completeness, and traceability, ²⁷ in addition to assessment of data relevance through feasibility evaluations. Time‐related sources of bias are particularly important when ECAs are used to support regulatory decision making. ²⁸ Although some cases explicitly documented these elements, descriptions of start dates and follow‐up durations across compared arms varied in this study. Incomplete reporting of methodological details has also been identified as a barrier to reproducibility in studies using clinical practice data. ²⁹ Enhancing the explicit specification of key methodological elements may thus improve the interpretability, robustness, and reproducibility of ECA‐based analyses used to support regulatory decision making.

Importantly, this study found that a substantial proportion of identified ECA cases were subject to post‐marketing requirements, including long‐term and all‐case investigations. These requirements often included evaluations of effectiveness and generalizability in real‐world clinical settings, suggesting that residual uncertainty at the time of approval was not fully resolved, but was rather managed through continuous evidence generation. This pattern reflects a broader lifecycle‐based perspective, in which ECAs may support initial regulatory decision making under conditions of uncertainty, while additional post‐approval evidence is generated to further characterize the benefit–risk profiles. This approach is particularly relevant in settings involving rare diseases, advanced therapeutic products, and high unmet medical needs, where regulatory decisions are frequently made in the presence of limited or evolving evidence. ³⁰ , ³¹ In this context, ECAs derived from RWD or historical clinical trials may complement traditional evidence‐generation strategies and contribute to a more integrated framework linking preapproval and post‐marketing evidence generation. ³² Based on the observed implementation patterns and post‐marketing uncertainty management practices, we synthesized a conceptual lifecycle framework for the design, evaluation, and regulatory management of ECAs (Figure 2 ).

Figure 2.

Conceptual lifecycle framework for design, evaluation, and regulatory management of external control arms.

Although ECAs provide an important alternative to RCT in specific settings, their use is associated with inherent methodological limitations. The absence of randomization introduces risks of residual confounding and selection bias, despite advanced statistical adjustment methods. ¹⁶ , ²² Temporal differences between data sources may introduce bias related to evolving standards of care, diagnostic practices, or supportive treatments, which may affect the comparability of treatment effects. ²⁸ Differences in data capture, outcome definitions, and follow‐up procedures may further impact consistency across study arms. In addition, incomplete or inconsistent reporting of methodological details in publicly available documentation may limit reproducibility and external evaluation. These considerations highlight the importance of careful study design, transparent reporting, and the appropriate interpretation of ECA‐based evidence.

Based on these methodological considerations, these findings also suggest opportunities for further development of regulatory and methodological guidance. Future work may benefit from more explicit expectations regarding the use of estimands in ECA studies, structured approaches to assessing and reporting residual bias, and minimum requirements for data quality and completeness, particularly for registry‐based data sources. In addition, the development of standardized reporting frameworks for ECA‐based submissions and regulatory review documentation may enhance transparency, interpretability, and reproducibility. A lifecycle‐oriented framework linking preapproval evidence generation with post‐marketing commitments may further support the appropriate integration of ECAs into regulatory decision making.

This study had several strengths. First, it involved a comprehensive screening of regulatory approvals within a defined period, enabling systematic identification and assessment of ECA implementation across all eligible cases. Second, the use of a structured framework derived from regulatory guidance and methodological recommendations allowed for the consistent evaluation of key methodological attributes across cases. Third, by integrating analysis of post‐marketing requirements, this study provides a lifecycle‐based perspective on the management of uncertainty in regulatory decision making, extending beyond prior studies focused mainly on pre‐approval evidence.

Several limitations should also be considered. First, the analysis was limited to regulatory approvals in Japan and may not fully capture variations in regulatory practices across different jurisdictions; however, the comprehensive coverage of approvals within a single regulatory framework enabled the systematic evaluation of implementation practices. The key methodological attributes evaluated in this study are derived from guidance documents issued by multiple regulatory authorities, suggesting that the findings may have relevance beyond the Japanese regulatory context. Future studies examining ECA use across multiple jurisdictions would enable comparison of documentation practices and may identify opportunities for international harmonization of expectations for ECA‐based submissions. Second, the analysis relied on publicly accessible sources of information, including summary technical documentation and review reports. Methodological considerations that were addressed during regulatory review but not explicitly documented—such as detailed discussions on data quality, analytic choices, sensitivity analyses, or conditions discussed during the review process—may not be fully captured. The documented attribute implementation assessed in this study therefore represents a lower bound of the actual methodological rigor applied in these cases. Third, the number of approvals supported by ECAs was limited (23 cases), reflecting the relatively infrequent use of ECAs in regulatory approvals. This is consistent with the expectation that ECAs remain a complement to, rather than a replacement for, RCTs and are typically reserved for specific circumstances where randomization is infeasible or unethical. As regulatory frameworks and methodological tools continue to evolve, it is possible that the scope and frequency of ECA use may increase, which may enable a more robust analysis with a large sample size. Finally, the assessment of documented attribute implementation involved the interpretation of regulatory documentation, and some degree of subjectivity therefore cannot be entirely excluded; however, the use of a predefined framework helped to ensure consistency and transparency of the analysis.

CONCLUSIONS

This study involved the systematic evaluation of cases that used ECAs to support regulatory approvals of new medical products in Japan. ECAs played an important role in supporting regulatory decision making in settings characterized by limited evidence; however, key methodological attributes were variably documented and implemented across cases, particularly in the design and analysis domains. Residual uncertainty was frequently addressed through post‐marketing requirements, highlighting a lifecycle‐based approach to evidence generation and uncertainty management for these cases. These findings highlight opportunities to enhance transparency across the planning, data‐source selection, design, analysis, and reporting of ECA‐based evidence, as well as the importance of more structured and consistent frameworks to support regulatory decision making for the benefit of regulators, sponsors, researchers, and other stakeholders involved in evaluating ECA‐based evidence.

FUNDING

No specific funding was received for this work.

CONFLICT OF INTEREST

S.O. is an employee of Bayer Pharmaceuticals. A.S. is an employee of Astellas Pharma Europe Ltd.

AUTHOR CONTRIBUTIONS

S.O. and A.S. wrote the manuscript; S.O. designed the research; S.O. and A.S. performed the research; and S.O. analyzed the data.

Supporting information

Figure S1.

DATA AVAILABILITY STATEMENT

All necessary data required to interpret and conclude the findings of this study are included in the main text and supplementary materials.