Leveraging External Data in the Design and Analysis of Clinical Trials in Neuro-Oncology

Abstract

The integration of external control data, with patient-level information, in clinical trials has the potential to accelerate the development of new treatments in neuro-oncology, by contextualizing single arm studies and improving decision making (e.g. early stopping decisions) in randomized trials. Based upon a series of presentations at the 2020 Clinical Trials Think Tank hosted by the Society of Neuro-Oncology, we provide an overview on the use of external control data, representative of the standard of care, in the design and analysis of clinical trials. High quality patient-level records, rigorous methods and validation analyses are necessary to effectively leverage external data. We review study designs, risks and potential distortions in leveraging external data, data sources, evaluations of designs and methods based on data collections from completed trials and real world data, data sharing models, and ongoing work and applications in glioblastoma.

Introduction

Drug development is associated with inefficiency, high failure rate and long timelines with poor success rates in oncology where less than 10% of drug candidates are ultimately approved by the Food and Drug Administration (FDA).^1,2 As new, unproven therapies emerge at an accelerated pace across oncology, there has been an increasing interest in novel approaches to clinical trial design that improve efficiency.^3,4

Within neuro-oncology, the use of trial designs with potential for increased efficiency are of interest, particularly in the study of glioblastoma (GBM), a disease setting with a critical need for better therapies as it continues to be associated with a dismal prognosis.⁵ There are several distinctive challenges in drug development for GBM including the inability to completely resect tumors, the blood-brain barrier, tumor heterogeneity, challenges with imaging to monitor disease course, and the unique immune environment.^6,7 With few treatment advances over the last two decades, the clinical trial landscape in GBM has been characterized by long development times, low patient participation, problematic surrogate outcomes, and poor go/no-go decision making.^8,9 Poor early phase decision making has been repeatedly highlighted as a major problem in the development of therapeutics¹⁰ and continues to stimulate interest in novel clinical trial designs.

Randomized controlled trials (RCT) are the gold standard for clinical experimentation and evaluation of therapies. RCTs control for systematic bias from known and unknown confounders by randomizing patients to receive either an experimental therapy or standard of care, which allows for the evaluation of treatment effects. RCTs, however can be difficult to conduct in some neuro-oncology settings. A relatively small percentage of patients participate in clinical trials,¹¹ and RCTs can suffer from slow accrual due to patient reluctance to enroll on studies with a control arm, which is a pronounced problem in settings with ineffective standard of care treatments (e.g. recurrent GBM).^12–14 Precision medicine further complicates this picture by focusing trials on biomarker-defined subgroups of patients who may benefit from targeted therapies.¹⁵ These subgroups are often comprised of a small proportion of patients with a given tumor type, resulting in significant challenges to conducting RCTs with adequate sample sizes to detect treatment effects.^15,16

Recently, the design and implementation of clinical trials that leverage external datasets, with patient level information on pre-treatment clinical profiles and outcomes to support testing of experimental therapies and study decision making, has attracted interest in neuro-oncology.^17,18 A recent phase 2b recurrent GBM trial used a pre-specified eligibility-matched external control arm (ECA, a dataset which includes individual pre-treatment profiles and outcomes), developed with data from GBM patients from major neurosurgery centers, as a comparator arm to evaluate an experimental therapy (MDNA55). After implementation of this trial design, investigators reported evidence of improved survival in patients receiving MDNA55 relative to the matched ECA cohort.¹⁹ Several neuro-oncology trials under development are actively exploring similar approaches to leverage external data in the design and analysis of clinical studies.

The Society of Neuro-Oncology hosted the 2020 Clinical Trials Think Tank on November 6, 2020 with a virtual session dedicated to trial designs leveraging external data. Experts in the field of neuro-oncology were paired with experts in data science and biostatistics, and representatives from industry, patient advocacy, and the FDA. The interdisciplinary session focused on challenges in drug development, challenges to data sharing and access, regulatory considerations on novel trial designs, and emerging methodological approaches to leveraging external data. While there was broad participation, most participants were from the United States and provided a US-centric perspective on the topic. The discussion from the Think Tank serves as a framework for this review, which focuses on the use of external data to design, conduct, and analyze clinical trials, with an emphasis on possible applications in neuro-oncology. We review trial designs, methodologies, approaches for the evaluation of designs and external datasets, regulatory considerations, and current barriers to data sharing and access.

Search Strategy and Selection Criteria

We searched the literature using PubMed with the search terms “external control arms”, “synthetic control arms”, “neuro-oncology trial design”, “glioblastoma trial design”, from January 2000 until May 2021. Articles were also identified through searches of the authors’ own files. Only papers published in English were reviewed. The final reference list was generated based upon relevance to the scope of this review.

Early Phase Trial Designs

Early-phase trials are typically designed to obtain preliminary estimates of treatment efficacy and toxicity that will inform the decision to pursue a definitive phase 3 trial or stop drug development. Often in neuro-oncology, these early phase studies are single arm trials (SATs) that test the superiority of the experimental therapeutic compared to an established benchmark parameter for the current standard of care (e.g. median OS or other point estimates).²⁰ Importantly, there can be significant differences between populations or standards to assess outcomes across trials,²¹ which can lead to inappropriate comparisons and inadequate evaluations of the experimental therapy. An additional major challenge with SATs is the choice of the primary efficacy endpoint. Response rate is difficult to interpret in GBM,²² and single arm studies are suboptimal for reliable inference on improvements of time-to-event endpoints such as survival. Based on these known limitations, single arm designs have been posited as a possible reason for poor go/no-go decision making and recent failed phase 3 trials in GBM.^23,24

The risk of biased conclusions of SATs has been examined extensively and frameworks have been developed to help guide the choice between RCT vs. single-arm designs for GBM.^10,25 Despite well documented limitations, SATs remain the most common trial design in early phase trials in GBM.²⁵ Alternative trial designs have been proposed to overcome limitations of SATs and to improve the evaluation of therapeutic candidates in the early phase of a drug’s development, including the incorporation of randomization, seamless phase 2/3 study designs²⁶ and Bayesian outcome-adaptive trials.^27–29

Overview of Trial Designs that Leverage External Data

Trial designs that leverage external data can generate valuable inferences in settings where SATs are suboptimal and RCTs are infeasible.³⁰ External data can play a role in supporting key decisions of the drug development process, including regulatory approvals and go/no-go decision making in early-phase trials. The use of external patient-level datasets has the potential to improve the accuracy of trial findings and inform decision making (e.g., determining the sample size of a subsequent confirmatory phase 3 trial, or selecting the phase 3 patient population). External data can also be incorporated into RCTs,³¹ for example within interim analyses,¹⁸ though these designs remain largely unexplored.

Externally augmented clinical trial (EACT) designs refer to the broad class of designs that leverage external data for decision making during a study or in the final analysis. EACTs rely on access to well-curated patient-level data for the standard of care treatment, from one or more relevant data sources, to allow for adjustments of differences of pre-treatment covariates between the enrolled patients and the external data, and to derive treatment effect estimates. Given the need for statistical adjustments, the external dataset ideally includes a comprehensive set of potential confounders.³² In considering such designs for gliomas, pre-treatment covariates have been thoroughly studied for adult primary brain tumors.^33,34

An example of an EACT design consists of a single arm study combined with an ECA (i.e., an external dataset with patient-level outcomes and pre-treatment profiles), which is used as a comparator to evaluate the experimental treatment. This design, indicated as ECA-SAT (Figure 1A), is a type of EACT that infers the treatment effect by using adjustment methods, to account for differences in pre-treatment patient profiles between the external control group and the experimental arm.³⁵ In this design the ECA is used to contextualize the outcome data from a single arm study. In contrast to the use of a benchmark estimates (e.g. median survival) of the standard of care efficacy in SATs, data analyses and treatment effect estimates are based on patient-level data from an external dataset.

Figure 1:

Schematic representation of clinical trial designs. (A) A clinical study with patients enrollment to a single experimental arm and an external control arm (ECA-SAT). Adjustment methods are used to compare the experimental arm and the external control arm. (B) An example of a two-staged hybrid randomized trial design. (C) An example of a randomized trial design that utilizes external data for interim futility analyses. The external dataset is used to support the decision to continue or discontinue the clinical study. If the trial is not discontinued for futility, the final analysis does not utilize external data.

Hybrid randomized trial designs constitute another type of EACT. These designs, with adequate external data and statistical plan³⁶, have the potential of reducing the overall sample size while maintaining the benefits of randomization. We describe an example of a two-stage hybrid design (Figure 1B). The study has an initial 1:1 randomization to the experimental arm and the internal control arm. If the interim analysis does not identify differences between the adjusted primary outcome distributions in the internal (randomized) control group and the external control group, then different randomization ratios (e.g., 2:5) can be used in the second stage of the study. In contrast, if there is evidence of inconsistencies between the external and internal control groups (e.g., unmeasured confounders or different measurement standards of outcomes and prognostic variables), the trial can continue with 1:1 randomization.

In the outlined example, the potential increase of the randomization probability for the experimental arm can be attractive and may accelerate trial accrual. Indeed, brain tumor patients, with an inadequate standard of care, may be more likely to enroll onto a trial if the probability of receiving the experimental therapy is higher.²⁵

EACT Designs

Along with high-quality and complete data, a statistically rigorous study design is the most important element of an EACT. As with any clinical trial, the design, including the sample size, a detailed plan for interim decisions, and statistical methods for data analyses, should be prespecified. Additionally, a plan for how missing data in the trial and external data sources will be handled is important. Potential distortion mechanisms that can bias the treatment effect estimates and undermine the scientific validity of EACT findings have been carefully examined and include unmeasured or misclassified confounders and data quality issues such as the use of different standards to capture or measure outcomes.^37–39

The risks of introducing bias (Table 1) and of compromising the control of false positive and false negative results by leveraging external patient-level data can differ substantially across candidate EACT designs, which span from single-arm studies (ECA-SAT design, Figure 1A) to hybrid randomized studies (Figure 1B). Quantitative analyses of these and other risks (e.g., exposure of patients to inferior treatments) are necessary prior to trial initiation. The decision to leverage external data should account for several factors in addition to the study population and the available patient-level datasets, including:

1. the stage of the drug development process (e.g., early phase 2 vs. confirmatory trials);

2. the specific decisions (e.g., early stopping of a phase 2 study for futility⁴⁰ or sample size re-estimation during the study⁴¹) that will be supported by external data;

3. resources (including maximum sample size); and

4. potential trial designs and statistical methodologies for data analyses.

Table 1:

Potential causes of bias in clinical trials with an external control group

	Description	Example	Methods to avoid or reduce the bias
Measured confounders	The distributions of pre-treatment patient characteristics that correlates with the outcomes in the trial population and in the external control group are different.	The external control group has on average a higher Karnofsky performance status or age than the trial population.	Matching. Inverse probability weighting. Marginal structural models.
Unmeasured confounders	The distributions of unmeasured pre-treatment patient characteristics that correlates with the outcomes in the trial population and in the external control group are different.	Supportive care (not captured in the datasets) differs between patients in the clinical trial and in the external control group.	Validation analyses can indicate the risk of bias before the onset of the trial.
Differences in defining prognostic variables / outcomes	The definition of clinical measurements may vary between datasets leading to differences in the definitions of outcomes or prognostic variables between the clinical trial and the external control group.	Measurement of tumor response with different response criteria or at different intervals in external control arms	Data dictionaries and validation analyses can reveal these discrepancies before the onset of the trial.
Immortal time bias	In the external dataset the time-to-event outcome cannot occur during a time window, because of the study design or other causes.	In GBM, different real world datasets capture patient survival from diagnosis or from a different time points	Explicit and detailed definitions of the time-to-event outcomes for the trial and the external dataset can reveal the risk of bias.

Candidate EACT designs and statistical methodologies for data analysis can present markedly different trade-offs between potential efficiencies (e.g., discontinuing early randomized studies of ineffective treatments by leveraging external data), and risks of poor operating characteristics (e.g., bias, poor control of false positive results). In other words, the value of integrating external data is context-specific, and it is strictly dependent on the specific EACT design and methodology selected for data analyses and decision making.

We describe three examples of EACTs with markedly different risks of poor operating characteristics. The purpose of these examples is to illustrate how external information can be leveraged for making different decisions during or at completion of a trial.

1). Single arm trial with an external control group (ECA-SAT).

We consider either binary primary outcomes (e.g., tumor response) or time to event outcomes with censoring (e.g., overall survival). The ECA-SAT design uses procedures developed for observational studies,⁴² such as matching, propensity score methods,^43,44 or inverse probability weighting,⁴⁵ which are applicable to the comparison of (i) data from a SAT (experimental treatment) and (ii) external patient-level data, representative of the standard of care therapy (external control, Figure 1A). These procedures have been developed to estimate treatment effects in non-randomized studies and have generated an extensive number of contributions in the statistical literature.⁴⁶ They compare outcome data Y under the experimental and control treatment with adjustments that account for confounders X.

(A). Evaluation of the ECA-SAT design using a collection of datasets.

The literature on adjustment methods applicable to EACT designs (e.g., matching)^43,44 is largely anchored to assumptions that are difficult or impossible to demonstrate,⁴² including the absence of unmeasured confounders.⁴⁷ In the context of ECA-SAT designs, where these assumptions might be violated, the investigator can attempt to evaluate the risk of bias and other statistical properties of treatment effects estimates (e.g., the coverage of confidence intervals) computed using adjustment methods. Patient-level data from a library of recently completed RCTs in a specific clinical setting, e.g. newly diagnosed GBM patients, facilitate the comparison of ECA-SAT, RCT, and SAT designs. For example, a treatment effect estimate computed using only data from a previously completed RCT can be compared to a second treatment effect estimate, computed using only the experimental arm of the same RCT and external data (Figure 2).⁴⁸ The comparison can be repeated considering different RCTs, adjustment methods, and external datasets. These comparisons allow one to describe the consistency between the RCT results and hypothetical results obtained from a smaller ECA-SAT (i.e., the experimental arm of the RCT, or part of it) leveraging external data; similar evaluation frameworks have been discussed recently.^49,50

Figure 2:

Schematic representation of a validation schema. A treatment effect estimate computed using only data from a previously completed RCT is compared to a second treatment effect estimate, computed using only the experimental arm of the same RCT and external control data.

(B). Leave-one-out algorithm.

An alternative evaluation approach that requires a collection of recently completed RCTs with the same control treatment has been proposed recently.¹⁷ The algorithm has been used to compare the application of candidate causal inference methods in ECA-SATs. This approach has been applied to a collection of newly diagnosed GBM studies, and it requires only pre-treatment profiles and outcomes from patients treated with the standard of care therapy, radiation and temozolomide (RT/TMZ). This is relevant because data access barriers can be substantially different for the control and the experimental arms. The algorithm¹⁷ iterates the following three operations for each RCT in the data collection:

Experimental Treatment

• iit randomly selects n (the sample size of a hypothetical ECA-SAT trial) patients (without replacement) from the RT/TMZ arm (control) of the trial, and uses patient pre-treatment profiles X and outcomes Y of these patients to define a fictitious single-arm study (i.e., the treatment group);

Control

• iithe data on patients treated with RT/TMZ in the remaining studies are used as external data (these datasets are combined into a single data matrix, the control group); and

Analysis

• iiia treatment effect estimate is computed by comparing the (fictitious) single-arm ECT study (step i) and the external data (step ii), using a candidate adjustment method, which is also used to test the null hypothesis H0 that the treatment does not improve the primary outcome.

For each study in the data collection, these steps (i-iii), which are similar to cross-validation, can be repeated to evaluate bias, variability of the treatment effect estimate, and the risk of false positive results. By construction, the treatment effect in this fictitious comparison (steps i-iii) is null, as patients receiving RT/TMZ are being compared to other patients receiving RT/TMZ from a different study (RT/TMZ vs. RT/TMZ). This facilitates interpretability and produces bias summaries for the ECA-SAT statistical plan. A recent analysis using this leave-one-out algorithm approach in newly diagnosed GBM illustrated high false positive error rates of standard SAT designs (above the alpha-level),¹⁰ which can be considerably reduced (up to 30% reduction) by using external control data from previously completed clinical trials in ECA-SAT designs.¹⁷

The first approach (A) attempts to replicate the results of a completed RCT, while the leave-one-out algorithm approach (B) is based on subsampling a control arm. Both approaches are valuable strategies that can detect potential distortion mechanisms (e.g., unmeasured confounders or inconsistent definitions of primary outcomes), which undermine the scientific validity of ECA-SAT designs. These approaches require patient-level data from several RCTs, with adequate sample sizes, to produce reliable analyses of the risk of bias and false positive results in future ECA-SATs. It is also important to not overinterpret positive findings from retrospective analyses using either approach, as relevant changes of the available treatments, technologies, or other factors can rapidly make the entire data collection obsolete and inadequate.⁵¹

2). Hybrid randomized trial designs with internal and external control groups.

Hybrid randomized trial designs combine external and randomized control data to estimate potential treatment effects.⁵² Figure 1B represents a two-stage hybrid design. In the first stage, n(E,1) and n(C,1) patients are randomized to the experimental arm and the (internal) control arm, respectively. The interim analysis is used for futility early stopping and to determine sample sizes n(E,2) and n(C,2), for the experimental and control arm in the second stage. These decisions are based on (i) a similarity measure comparing estimates of the conditional outcome distributions Pr(Y|X) of the external and internal control groups, and (ii) preliminary treatment effect estimates. The proportion of patients randomized to the internal control arm during the second phase can be reduced or increased based on the pre-specified interim analysis, which involve summaries in support or against the integration of external data to estimate the effects of the experimental treatment.

Based on recent results from a phase 2 study of an experimental therapeutic MDNA55,¹⁹ investigators are currently planning an open-label phase 3 registration study with implementation of a hybrid randomized design in recurrent IDH-wildtype GBM patients. The study team is considering a 3:1 randomization ratio for allocation to the experimental and control arms, with a final comparison of overall survival between patients receiving the experimental agent (MDNA55) and the control groups (external and internal control arms).⁵³

3). Randomized controlled trials that incorporate external data to support futility analyses.

External data can be incorporated into RCTs for other purposes³¹ such as leveraging external data for interim decisions.¹⁸ In the design illustrated in Figure 1C, interim analyses utilize predictions based on early data from the RCT combined with external data. These predictions express the probability that the trial will generate significant evidence of positive treatment effects. The trial is discontinued by design if the predictive probability becomes smaller than a fixed threshold. The final analysis, after completing the enrollment and follow-up phases, does not utilize external data. Indeed, the primary result of the trial is positive, indicating evidence of improved outcomes with the experimental treatment, if a standard p-value, computed using only the RCT data (excluding external information) has a value below the targeted control of false positive results α.

In ideal settings, without unmeasured confounders and other distortion mechanisms, leveraging external data for interim futility analyses can (i) reduce the expected sample size of the RCT when the experimental treatment is ineffective, and (ii) reduce the early stopping probability when the experimental therapy is superior, thus increasing the power.¹⁸ Additionally, the outlined design maintains a rigorous control of the RCT type I error probability, even in presence of unmeasured confounders, because the external data are excluded from the final data analyses. The efficiency gains and risks associated with the described integration of external data into interim decisions have been quantified for newly diagnosed GBM trials, with evaluation analyses that built upon a collection of datasets from completed RCTs, the leave-one-out algorithm outlined above and other similar procedures.¹⁸

As EACTs require a number of context-specific considerations, from relevant aspects of the external datasets to the feasibility of alternative designs, a discussion with regulatory agencies in early stages of trial planning is strongly recommended.

External Data Sources

The use of external controls to evaluate new treatments is dependent on the availability of high-quality external data. Selecting appropriate datasets is critical and checklists have been developed to provide guidance on data quality.⁵⁴ Data considerations for external controls include appropriate capture of patient-level data,⁵⁵ consistent definition of covariates and endpoints, and adequate temporality of the data, as small temporal lags can significantly affect the trial analysis.⁴⁹ Investigators should consider potential biases that occur if the endpoints definitions are inconsistent across studies. For example, survival can be measured from the date of diagnosis, the date of randomization, or the initial date of adjuvant treatment. In other words, the definition of the “time zero” should be explicit and consistent, during the trial and in the external datasets.⁵⁶ Missing data is another important consideration in analyses with external data.^57,58 While there are methods to address missing data (e.g., multiple imputation and likelihood-based methods), their use within EACT designs has not been well studied.

Statistical methods should be employed to adjust for differences, but in general, the population of the external control and the trial population should be similar to reduce the risk of bias. Potential unmeasured confounders, inconsistencies in definitions, and differential measurement standards of covariates and outcomes across datasets need to be scrutinized using data dictionaries and study protocols. Contemporaneous controls are ideal, but historical controls with patient-level data may be helpful in appropriate contexts. For example, disease settings without a recent change in the standard of care (e.g. GBM) or long-track record of time-stable outcomes may have more flexibility in the temporality of data, but this should be weighed against possibility of unmeasured aspects of care such as advances in imaging, radiation therapy, surgical techniques, and supportive care that may change over time.⁵⁹ Additionally, in order to support a marketing application, the data should be traceable (i.e. an audit trail should be available to document the data management processes).⁵⁴

The two most relevant sources of external data are previously completed clinical trials and non-trial real world data (RWD) derived from clinical practice, each of which have strengths and weaknesses. The use of data from previously completed clinical trials can be advantageous, given that the data are typically collected in a rigorous trial environment with vetting procedures. Clinical trials are often conducted in specialized institutions and enroll clearly defined segments of their patient populations. Patients previously enrolled on RCTs and treated with the standard of care may be more likely than RWD cohorts to contain pre-treatment profiles similar to patients that will be enrolled in future trials. The use of detailed data collection forms, intensive monitoring, and specialized personnel facilitates adherence to clear protocols that produce standardized data.¹⁵ In GBM in recent years, there have been several negative phase 3 randomized trials with hundreds of patients receiving current standard of care therapy (RT/TMZ) in studies conducted by cooperative groups and industry.^60–63 While external data from previously completed clinical trials are more likely to be complete and accurate, data access can be challenging due to impediments to data sharing⁶⁴ and contemporary trial data may not be made available by trial sponsors.

RWD represents a distinct data source derived from registries, claims and billing data, personal devices or applications, or electronic health records (EHR). As RWD is generally not collected for research purposes, there can be concerns about data organization, data quality, confounding, selection mechanisms, and ultimately bias.^65–67 Advances in the quality of EHR data have created opportunities, however, with newer datasets that can be well curated and linked with molecular or radiologic data with high fidelity. Efforts to harmonize RWD from disparate data sources and novel methods to incorporate such data into clinical studies provide an avenue to inform trial designs⁶⁸ and regulatory decision making,⁶⁹ but further work is required to validate these approaches.

Although differences between RWD and data from clinical studies have been reported,⁷⁰ methodological work on the use of joint models and analyses, to compensate for the scarcity of trial data and the potential distortions of RWD (e.g. measurement errors or unknown selection mechanisms), is currently in its early stages.

Existing methodological work has primarily focused on overall survival,^17,48 which is more likely to be adequately captured in external datasets relative to other outcomes. Radiologic endpoints such as progression-free survival require caution because of the risk of inconsistent assessments across datasets. In RWD data sources, radiologic outcomes may not be determined by formal response assessment criteria, and central radiologic review would likely not be routinely implemented. Although recently completed brain tumor clinical trials often use consensus guidelines produced by the Response Assessment in Neuro-Oncology working group for response assessment,^71,72 these criteria include subjective components⁷³ and datasets of previously completed trials can still include misclassification errors. Quality of life, neurologic function, and neurocognitive outcomes are increasingly incorporated into clinical trials.^72,74 These non-survival outcomes can provide meaningful measures of the clinical benefit of a therapy, and they can serve as valuable endpoints in neuro-oncology trials.⁷⁵ Nonetheless, missing measurements of these outcomes are common across datasets from completed trials and RWD. Exploration of the use of external data, to analyze radiographic outcomes, patient centered outcomes, and safety outcomes in neuro-oncology trials remains limited.

Examples of External Control Groups Beyond Neuro-Oncology

Carrigan et al. leveraged a curated RWD dataset of 48,856 patients (EHR from the Flatiron database) to re-analyze 11 completed trials in advanced non-small cell lung cancer.⁴⁸ In this study, the external control arms were defined with matching methods. The external control arms were able to recapitulate the treatment effect estimates (hazard ratios) for 10 of the 11 RCTs. The study suggests the potential utility of RWD as external controls. This is likely due, at least in part, to the large number of patients in the external data, with large subsets of patient records that met the RCTs inclusion and exclusion criteria. Of note, the EHR-derived external control arms did not re-capitulate the results for one of the 11 RCTs. On inspection, the authors felt this discordance was due to a biomarker-subgroup population not sufficiently represented in the EHR dataset. These findings underscore the need to account for biomarkers and well represented subpopulations in the external control groups.

Another recent example supports the utility of external data to contextualize SATs.⁷⁶ In a FDA-led retrospective analysis, the outcomes (invasive disease-free survival) from a single arm study⁷⁷ of adjuvant paclitaxel and trastuzumab in HER2-positive early breast cancer patients were analyzed using an external control group derived from clinical trials (control therapy: anthracycline/cyclophosphamide/taxane/trastuzumab or taxane/carboplatin/trastuzumab).⁷⁸ The de-escalated regimen that combines adjuvant paclitaxel and trastuzumab had been adopted in clinical practice based upon initial SAT results.⁷⁷ This retrospective analysis used propensity score matching to adjust for differences in pre-treatment patient profiles in the SAT and in the external control dataset. The analysis estimated comparable outcome distributions for the adjuvant paclitaxel and trastuzumab regimen and the control regimen, which supported the use of the de-escalated regimen, particularly in light of higher toxicity with the control regimen.

Considerations and Implications for Regulatory Decision Making

In the United States, the 21^st Century Cures Act directed the FDA to develop guidance for the evaluation and use of RWD, and to consider potential roles for RWD in drug development and regulatory decision making. For example, RWD could be used to support approvals for new indications or be integrated into existing monitoring requirements after approval.⁷⁹ Accordingly, the FDA launched a RWD program to lay the foundation for rigorous use of such data in regulatory decisions.⁸⁰ Several ongoing initiatives are providing guidance on data quality, data standards, and study designs that incorporate RWD.⁸¹ Also, other regulatory institutions such as the European Medicines Agency and Health Canada have demonstrated an openness towards better understanding and potentially leveraging RWD for drug development.^82,83

The use of external control data, within a regulatory scope, can support expedited approval, extend the label for a therapy to a new indication or subgroup, and more generally support regulatory decision making.⁸⁴ For example, in a rare disease setting, the FDA approved blinatumomab for adult relapsed/refractory acute lymphoblastic leukemia in a study that used data from a previously conducted clinical trial as a comparator.⁸⁵

In a regulatory context, there is an understandably high burden of proof for investigators to demonstrate the scientific rigor of study designs and analyses that leverage external control data, with an appropriate risk level. Comparative analyses with standard RCT designs are fundamental to evaluate robustness and efficiencies of EACT designs. The external data, study design, and analytics should be tailored to each specific clinical context and intended regulatory use. Each of these elements should carefully considered and scrutinized in evaluating the risks of biased treatment effect estimates and inadequate control of false positive findings.

Data Sharing Models

Despite the appeal of patient-level data from prior clinical trials, data access is a barrier to studying and implementing EACT designs, in both early-phase and late-phase trials. Data sharing efforts from industry-funded RCTs are increasing, but there remains substantial room for improvement.^86,87 Beyond implications for EACTs, clinical trial data sharing allows investigators to carry out analyses that can generate new knowledge, analyses which have been deemed to be “essential for expedited translation of research results into knowledge, products and procedures to improve human health” by the National Institutes of Health.⁸⁸ Significant challenges and appropriate concerns about data sharing remain,⁸⁹ including the need to ensure patient privacy and academic credit; the use of adequate standards for combining data from different sources; and the allotment of resources required to deidentify patient records and to provide infrastructures for data sharing. The patient perspective serves as an important counterpoint; assuming that privacy is protected, studies indicate that patients are in favor of having data shared for purposes that can help advance clinical outcomes.⁹⁰

Advances towards simpler data access could transform the ability to perform secondary analyses⁹¹ and to leverage external data in future clinical studies. For many data-sharing platforms, a gatekeeper model is utilized, often with long approval processes, restrictive criteria for data access, and limitations on data use. These requirements act as a mechanism of passive resistance and delay access to data from completed trials. An increasing number of data sharing platforms such as Vivli,⁹² YODA,⁹³ and Project Data Sphere,⁹⁴ are aligned with more open-sharing models for clinical trial datasets.⁹⁵ Nonetheless, data from previously completed neuro-oncology trials remain largely difficult to access.

New policies may be necessary for data sharing and to accelerate the study of new therapeutics. An important consideration is the modification of incentives for data sharing.⁹⁶ A systematic effort from cooperative groups, industry, academics, and other stakeholders could help achieve this goal. Regulatory requirements that ensure timely data sharing and patient advocacy groups could play key roles in hastening this process. In addition, initiatives and agreements to prospectively share patient-level data from the control arms of multiple cooperative RCTs could be beneficial to the participating studies and create opportunities to extend data sharing.

Future directions and conclusion

At the conclusion of our think tank session, there was a strong interest and desire to continue to collaborate and to critically investigate, validate and implement EACT designs in GBM and on a broader level, in neuro-oncology. Efforts to form industry-cooperative group partnerships, and selection of datasets and statistical methods were set as goals to continue towards an advancement of our understanding of the role of EACTs for drug development in neuro-oncology.

The use of external data to design and analyze clinical studies has the potential to accelerate drug development and can contribute to rigorous evaluation of new treatments. RCTs will remain the indisputable gold standard for the evaluation of treatments, but external datasets can supplement information gleaned from RCTs and single arm studies. Further methodological work can help identify the appropriate clinical contexts, data, and statistical designs for EACTs that generate inference on treatment effects of experimental therapies, with well controlled risks on their accuracy and scientific validity.

There is a continuum of approaches for leveraging external data, and the use of EACTs should be tailored to the disease context. An emphasis on high quality patient-level data, rigorous methods and biostatistical expertise are critical in the successful implementation of EACTs. Data access to previously completed clinical trials and RWD is improving, but new policies and initiatives for data sharing could further unlock the value of external data. Continued collaborations between stakeholders including industry, academics, biostatisticians, clinicians, regulatory agencies, and patient advocates are crucial to understand the appropriate use of EACT designs in neuro-oncology.