Critical concepts in adaptive clinical trials

Abstract

Adaptive clinical trials are an innovative trial design aimed at reducing resources, decreasing time to completion and number of patients exposed to inferior interventions, and improving the likelihood of detecting treatment effects. The last decade has seen an increasing use of adaptive designs, particularly in drug development. They frequently differ importantly from conventional clinical trials as they allow modifications to key trial design components during the trial, as data is being collected, using preplanned decision rules. Adaptive designs have increased likelihood of complexity and also potential bias, so it is important to understand the common types of adaptive designs. Many clinicians and investigators may be unfamiliar with the design considerations for adaptive designs. Given their complexities, adaptive trials require an understanding of design features and sources of bias. Herein, we introduce some common adaptive design elements and biases and specifically address response adaptive randomization, sample size reassessment, Bayesian methods for adaptive trials, seamless trials, and adaptive enrichment using real examples.

Introduction

Clinical trials are needed to determine the effectiveness (and occasionally harms) of therapeutic agents. A new approach to clinical trials, termed adaptive clinical trials, allows for modifying trial design features during the trial. The decision to make a modification is based on the data collected throughout the trial. This is considerably different from traditional trial designs as modifications to the trial design are not generally allowed during the course of enrollment and follow-up of patients, and where the learning about efficacy and safety occurs after the trial is completed. Adaptive designs aim to reduce resources, decrease time to completion, decrease the number of participants required, and/or improve the likelihood of detecting treatment effects in a clinical trial. A recent review of trials registered on ClinicalTrials.gov showed a large increase in the number of adaptive design trials completed since 2006: there were only 10 registered adaptive trials before 2006, but 133 registered from 2006 to 2013 with most trials undertaken in the field of oncology.¹ The use of adaptive trials in oncology is primarily for pharmaceutical drug development.² Reviews of adaptive trials suggest that their use has increased drastically in the past decade, particularly in pharmaceutical research.^1,2

The increasing use of adaptive designs in clinical trials should be noted as adaptive designs and traditional trial designs have important differences. Just as there are differences in the planning and conduct of these trials, so too are the reasons and interpretation of bias. Regulatory agencies such as the United States Food and Drug Administration³ and European Medicines Agency⁴ have recognized the validity of adaptive trial designs and provided guidance on how investigators should consider their regulatory considerations during the presubmission (i.e., planning) and trial stages. However, the understanding and appreciation of adaptive clinical trial features among clinicians and researchers are typically limited.⁵ To help improve understanding of adaptive designs, herein, we review common adaptive features. We review methods for response adaptive randomization (RAR), sample size reassessment (SSR), Bayesian adaptive methods, seamless design, and adaptive enrichment. We provide real-life examples of clinical trials that have been designed with these adaptive designs.

Common adaptive designs

In this section, we discuss common adaptive designs. To support the comprehension of these common designs, we provide illustrative figures here. Figure 1 demonstrates an RAR design; Figure 2 demonstrates an SSR design and Figure 3 shows a seamless trial design. Figure 4 illustrates an adaptive enrichment trial design with an SSR component. Table 1 summarizes common adaptive designs outlined in the next sections.

Figure 1.

Response adaptive randomization.

Notes: The first interim analysis shows serious toxicity for the high-dose arm and promising results for the medium dose. The RAR design allows the allocation ratio to be changed to zero for the high-dose arm after the first interim analysis, so that patients will no longer be enrolled to this treatment. The allocation ratio for the medium dose, on the other hand, can be increased allowing more patients to be enrolled to this arm. Then, the trial stops after the medium dose demonstrates superiority over the low-dose arm. This example shows how an RAR design can potentially allow for a larger number of patients to be allocated to the superior treatment.

Abbreviation: RAR, response adaptive randomization.

Figure 2.

Sample size reassessment.

Notes: If the first interim analysis shows worse results than expected, an SSR can be performed using the interim results. SSR is not permitted in a traditional nonadaptive trial, so even when the original planned sample size is reached, the trial may be underpowered (Option 1). If SSR is permitted, the enrolment target could be increased to ensure that the trial is adequately powered (Option 2).

Abbreviation: SSR, sample size reassessment.

Figure 3.

Seamless trials.

Notes: After first interim analysis, the high-dose arm showing serious toxicity could be discontinued from the trial. Thereafter, the trial transitions seamlessly from the feasibility into the pivotal phase with standard therapy arm being introduced into the trial.

Figure 4.

Trial with adaptive enrichment design with SSR.

Notes: In this example, the first interim analysis shows the experimental intervention has more promising results on one subgroup of patients (illustrated in gray), but it is not shown to be effective for other patients. The study eligibility criteria could then be modified to investigate the efficacy of the intervention in the gray subgroup (i.e., an adaptive enrichment design) with an SSR ensuring a sufficient sample size for this subgroup.

Abbreviation: SSR, sample size reassessment.

Table 1.

Summary of common adaptive design features

Adaptive designs	Details	Advantages	Disadvantages	Case study trials
Response adaptive randomization	Changes in treatment allocation ratio based on interim analyses over the course of the trial	Allow larger proportion of new subjects to be allocated to more promising study arm(s)	Prone to error/chance especially if done in an interim analysis performed at an early stage of the trial with only few data points	ASTIN trial9
Sample size reassessment	Increase or decrease in sample size target based on interim analysis	Smaller number of subjects may be needed and recruited at the end of the trial or can ensure that adequate sample size is recruited at the end	Adaptations can be misguided if interim analysis done with small sample size Increasing sample size may logistically (e.g., cost and time) be challenging	Casali et al 201512
Seamless designs	Allows for immediate continuation from one phase to the subsequent phase, if there is an overlap of essential components to the trial design	Can be more efficient, as a single trial can be used to achieve multiple objectives that are normally evaluated in two or more separated trials	Can logistically (e.g., cost and time) and methodologically be more challenging	Joura et al 201515
Adaptive enrichment	Modification of the trial eligibility criteria	May help identify subgroup of patients that respond more favorably to the investigated intervention	Limited generalizability to the study population and only to the subgroup of population May be prone to error/chance if adaptations made early with interim analysis with few data points	I-SPY 2,19,20 BATTLE,21 and Rosenblum and Hanley 201722

Abbreviation: ASTIN, acute stroke therapy by inhibition of neutrophils

Response adaptive randomization

Adaptive designs (Figure 1) with RAR allow the treatment allocation ratio to be adapted based on interim analyses over the course of the trial. In RAR, the allocation ratio adapts to favor the treatment arm with more favorable interim results. RAR designs can reduce the overall number of deleterious clinical outcomes observed in a trial and may reduce the overall sample size, without meaningful loss of statistical precision. Probably, the most well-known case of RAR is the “play-the-winner” design.⁶ In this simple RAR design, one starts with two urns, representing each treatment, containing one ball each to represent an initial 1:1 allocation ratio. Every time a success is observed with one treatment a ball is added to the corresponding urn changing the allocation ratio for the next patient to be enrolled. This RAR design has a high risk of extreme allocation to a treatment with more promising initial results that may occur due to chance. RAR trials generally use more sophisticated rules and apply computational algorithms (e.g., Bayesian predictive probability) to ensure that the allocation ratio does not become extreme and converges at a more stable rate.⁷ In adaptive trials with RAR design, allocation to treatment arm(s) is often reduced based on poor effectiveness or safety at early looks at the data. This affords both ethical benefits as well as additional resources to explore effective intervention arms. Rigorous planning is required to ensure that an arm is not erroneously dropped early. Statistical methods for constraining such error risks to acceptable levels have been developed for adaptive and conventional clinical trials; these methods are discussed in more detail by Bauer et al.⁸

An example of RAR trial is the ASTIN (Acute Stroke Therapy by Inhibition of Neutrophils) trial.⁹ In this dose–response placebo-controlled trial, RAR was used to determine the optimal dose of recombinant neutrophil inhibitor factor (rNIF) from 15 doses ranging from 10 to 120 mg for acute ischemic stroke patients.⁹ The primary outcome was the change in Scandinavian Stroke Scale (DSSS), a score representing recovery in acute stroke patients, from baseline to day 90.⁹ Continual reassessment method was used as the dose–response model and to update the response allocation. Burn-in for response allocation was 200 patients with fixed 15% allocated to placebo and equal allocation to all of rNIF doses. Bayesian posterior estimates of the dose–response curve were used to obtain the effective dose 95% (ED95) and its variance, and after burn-in period, treatment allocation was determined to the dose that minimized the variance of ED95. In order to allow the response allocation to be updated before a patient’s day 90 scores became available, longitudinal modeling was used to estimate the day 90 SSS score based on the patient’s interim scores.⁹ The protocol prespecified the maximum sample size target of 1,300 patients with minimum 15% of patients being allocated to placebo.⁹ This trial enrolled a total of 966 patients (26% in the placebo); >40% (the majority) of patients receiving rNIF were allocated to the top three doses (96, 108, and 120 mg) of rNIF.¹⁰ Although rNIF was shown to be well tolerated in these patients, this trial was stopped early for futility, as rNIF was shown not to improve recovery from acute ischemic stroke.⁹

Sample size reassessment

Although the use of adaptive designs has become more popularized recently, the development of confirmatory adaptive design methodology has been popularized over the last three decades.⁸ SSR (i.e., change in the originally estimated sample size [Figure 2]) has been around for several decades and was a major focus in the early waves of adaptive design methodology development.⁸ The motivating factor behind SSR development is that a clinical trial may prove to be underpowered, even when the originally planned sample size is reached, due to common incorrect sample size estimation in the planning phase.⁸ When a trial fails to meet its prespecified endpoints under such a circumstance, simply enrolling more patients is not a valid option since the risk of a false-positive finding will no longer be under control, but SSR can be preplanned to control for the risk of a false positive.¹¹ SSR is not intended to salvage a failed trial, but rather, it minimizes the risk that a trial is under- or overpowered in the first place. SSR may also be used to determine whether to continue or terminate the trial; for instance, if an SSR shows the new sample size required to be too large, the investigators may choose to end the trial early, or switch the objective of the trial from superiority to noninferiority. The required sample size may be adjusted in one or more interim looks at the data depending on the prespecified plan.¹¹ Scenario analyses using analytical approaches or simulations are recommended to inform the preplanning for intended and allowed adjustments.

A motivating example of a recently conducted trial with SSR design includes the trial by Casali et al.¹² This trial is a two-arm design that assessed the efficacy and safety of imatinib as an adjuvant therapy for localized gastrointestinal stromal tumors and had originally intended to recruit 400 patients over the 5-year period starting December 2004.¹² The decision to recruit more patients (908 patients) was made 3 years later after an interim analysis using a significance level of 0.015 showed a survival rate that was higher than expected in the control group.¹² With the SSR, this trial showed an improvement of adjuvant imatinib therapy for relapse-free survival that the originally planned sample size was not adequately powered enough to do.¹²

Bayesian adaptive approaches

Conventional statistics used in clinical trials, such as p-values and confidence intervals, come from the frequentist paradigm of statistics, which is rooted in the philosophy that statistical inferences should be drawn from the probability of observing similar data in a series of subsequent hypothetical examples of the study being conducted. This paradigm is confusing not only to most clinicians, but also to many statisticians. Bayesian statistics is another paradigm of statistics rooted in the philosophy that our future beliefs (e.g., about a treatment effect) are a product of our current belief and newly gathered evidence. In adaptive trials where no clinical trial is ever a replication of itself and adaptations to the design are dependent specifically on our prior beliefs and all recent evidence collected, the Bayesian framework is ideal.

In Bayesian statistics, the prior belief (typically based on limited prior data) is translated into a probability distribution (the “prior distribution”), which is continually updated as data are accumulated (the “posterior distribution”). In the context of adaptive trials, this affords the flexibility to inspect interim results at any time and assess whether adaptations should be made.¹³ In particular, the updated probability distribution (i.e., the “posterior distribution”) is used to gather probabilities of different outcomes of interest occurring if the trial is continued with the same design. For example, a very low (Bayesian) probability of a treatment being superior can result in a treatment arm being dropped, or at least, inform a change in the allocation ratio to minimize the number of patients exposed to an inferior treatment. Bayesian statistics also have the advantage that the prior information can be varied to reflect anything on the spectrum from optimism to pessimism and from high to low uncertainty. Thus, when clinical trial investigators’ opinions about the likelihood of trial success differ during the trial planning stage, Bayesian statistics provides a way of assessing the degree to which the spectrum of postulated success probabilities impact the actual probability of success upon trial completion. A range of such scenario analyses varying, for example, the target sample size (i.e., Bayesian SSR) can then provide a balanced overview of where the optimal (additional) sample size is likely to be reached. Likewise, scenario analyses’ varying beliefs about the treatment effects can yield a balanced overview of where response adaptive allocation adaptations or treatment terminations for superiority, inferiority, or equivalence should be applied. Such a sensitivity analysis is in accordance with good Bayesian practice, and thus, the quality of any adaptive clinical trials incorporating Bayesian methodology should be appraised by the adequacy of its “prior” sensitivity analyses.

Seamless trials

Traditionally, an intervention tested in humans is evaluated in three conceptual phases: Phase I tests tolerability of the intervention in healthy individuals, Phase II primarily tests tolerable dose ranges in individuals with the health condition of interest, and Phase III primarily tests the efficacy and safety of the intervention in individuals with the health condition of interest. In addition, some investigations may continue on to Phase IV trials where patients are tested in a setting corresponding more to a real-world clinical setting compared with the often much controlled setting of the Phase III trial. With conventional randomized clinical trials (RCTs), the knowledge from each phase can be used only once the phase is finished, and the evaluation of an intervention pauses between the phases of the clinical trials. Seamless studies are a type of design that allows for immediate continuation from one phase to the subsequent phase. Because Phase II and III are similar in their clinical settings, seamless designs are most commonly used to seamlessly combine Phase II and III trials.¹⁴ A seamless Phase II/III RCT can be feasible even with significant changes to the design across phases, but does require overlap in some essential components. For example, if a Phase II trial examines the tolerability of five doses and three of these are terminated at the end of the trial, a Phase III trial can seamlessly continue with enrollment of the two remaining doses (treatment arms) and likely also add additional arms such as a placebo group. Seamless trials require that early data can be rapidly analyzed and interpreted to inform decisions about seamless transition into the subsequent phase.¹⁴ Seamless designs, therefore, have the capacity to significantly reduce time from first Phase II randomized patients to reaching a definitive conclusion about efficacy and safety by the completion of the Phase III trial.¹⁴

For a motivating example of an adaptive seamless design trial, we refer to the seamless II/III trial on 9-valent human papillomavirus (HPV) vaccine by Joura et al.¹⁵ The overall goal of this trial was to compare the efficacy and safety of 9-valent HPV vaccine against quadrivalent vaccine, the gold standard for HPV vaccine.¹⁵ This seamless design started as a Phase II trial with the goal of identifying an optimal formulation of 9-valent HPV vaccine between three different formulations with varying doses (low, medium, and high) based on their immunogenicity (the ability of the vaccine to provoke an immune response and produce HPV-neutralizing antibiotics).¹⁵ The interim analysis of 1,240 patients based on immunological endpoints resulted in selecting the medium dose vaccine, which was thereafter seamlessly tested in Phase III in this adaptive trial. Data from 310 patients who received the medium dose in the first phase were concatenated with the data from the subsequent 13,578 patients enrolled in the next phase (1:1 ratio 9-valent HPV vaccine vs quadrivalent vaccine).¹⁵ Collectively, these were used to test the noninferiority of 9-valent HPV vaccine vs quadrivalent vaccine for both efficacy (e.g., prevention of infections and disease) and safety (e.g., adverse events to vaccine).¹⁵

Continuing seamlessly from the feasibility to the confirmatory phase saved time and resources to achieve the overall goal of testing for efficacy and safety of 9-valent HPV vaccine to the quadrivalent vaccine. In this study, patients who received the selected vaccine dose formulation of 9-valent HPV vaccine (i.e., medium dose) in the early phase could continue into the subsequent phase and followed up for efficacy and safety, reducing the overall sample size required to conduct these trials separately.¹⁶ Time and resources were saved, as the trial infrastructure for patient recruitment and follow-up could be tested and established in the feasibility phase.

Adaptive enrichment

Adaptive enrichment refers to a modification of the trial eligibility criteria.¹⁷ If an interim analysis shows that prespecified patient subgroups have different responses, the eligibility criteria can be modified to include only the favorable group moving forward.¹⁷ Here, an SSR could also be performed separately to modify the sample size requirement for each subgroup. Vice versa, if new external evidence surfaces during the course of the trial that a broader population than that defined by the trial’s eligibility criteria might benefit, the trial may also be enriched by broadening the eligibility criteria.

There have been different proposals of adaptive enrichment design trials.¹⁸ With the emergence of the precision medicine era and its emphasis on biomarkers, there have been increasing calls for examination of biomarker subgroup effects. The I-SPY2 and BATTLE adaptive trials are two examples where enrichment was an integral part of the design. I-SPY 2 is a recently completed (large) Phase II trial that evaluated 12 neoadjuvant therapies’ efficacies against 10 biomarker signatures.^19,20 The treatment(s) with a high predictive probability of being effective in patients with their corresponding biomarker signature continued onto the subsequent Phase III trial. The BATTLE trial is another biomarker-driven enrichment adaptive trial.²¹ At the first stage, BATTLE had a goal of identifying predictive biomarkers that could be used to randomize advanced non-small cell lung cancer (NSCLC) patients into one of four possible treatment arms. On the basis of the first stage, BATTLE-2 then ensued with an emphasis of evaluating targeting therapies for NSCLC with KRAS mutations.

A review by Rosenblum and Hanley²² provides a good motivating example where an adaptive enrichment design can be an important consideration for design of Phase III trial using results from a completed Phase II clinical trial for stroke interventions (i.e., MISTIE trial).^23,24 In the MISTIE trial, there were two subgroups of interest: small and large intraventricular hemorrhage (IVH) patients (small IVH patients were defined with IVH level ≤10 mL, and large IVH patients were defined with IVH level >10 mL at baseline).²⁴ The results of the MISTIE trial showed stronger evidence that small IVH patients benefit from treatment, whereas the treatment effect was not clear for large IVH patients. As one of the motivating factors for an adaptive enrichment design is to enroll subgroups of patients with greater potential of benefits, Rosenblum and Hanley²² performed clinical trial simulations using the results from the MISTIE trial and compared the sample size required to achieve the desired power and type I error for a Phase III trial that is designed with a standard design versus with an adaptive enrichment design. In the standard design, interim analyses were allowed where the entire trial (i.e., for both subgroups) could be stopped early, and in the investigated adaptive enrichment design, interim analyses could be used only to enroll subgroup of patients with promising results (e.g., small IVH patients).²² The estimated number of patients for the subgroup with favorable Phase II results was larger in the adaptive enrichment design when compared with the standard design (598 versus 491), but the overall population required was shown to be less for the adaptive enrichment design (1,148 versus 1,473) with fewerpatients being recruited for the subgroup with less promising treatment effects (550 versus 982).²²

Advantages and limitations of adaptive designs

Adaptive designs can be more cost and time efficient and can enhance patient protection. Multiple interim analyses are allowed in adaptive designs, so if a trial demonstrates unanticipated results, it can be stopped early. Interim analyses can also be used to adjust the allocation probability such that fewer patients are exposed to an experiment treatment(s) with inferior efficacy or safety. If an inferior efficacy or smaller treatment effect is observed, terminating the entire trial will save resources, or dropping the inferior treatment arm(s) will act to enhance patient protection as well. Adaptive designs can improve the chance of conducting a successful trial because SSRs, as they are preplanned, can be performed without a penalty. If an SSR shows a smaller treatment effect than the magnitude that was anticipated, without a modification, the trial can be underpowered under the originally planned sample size. In such a case, if the new required sample size is too large, the trial can be terminated. If the new required sample size is reasonable, the trial can be continued with the larger recruitment target.

Adaptive designs are not without limitations and biases. An adaptive trial requires thorough planning and transparent reporting. Preplanning of modifications and statistical analyses requires extensive efforts that may be complicated and difficult to plan or execute. The overall results can be more difficult to interpret. Incorrectly planned and/or executed adaptive designs can introduce operational bias and statistical bias that may be difficult for a reader to determine. A trial modification can be confounding and make it more difficult to interpret the trial results. Particularly, in trials with SSR and seamless designs, the sample size and the trial duration could also be greater due to the modification(s). For adaptive enrichment trials, generalizability is a potential issue; in trials that end up modifying the eligibility criteria to a particular subgroup, there is generalizability only to the subgroup population, not to the larger disease population. As well, it can be difficult to recruit the subgroup with promising results, so these adaptive designs may be more logistically complicated to conduct. If there is a long delay in time before outcomes of interest can be measured, it is important to consider that there could be insufficient information available at an interim analysis to make an appropriate modification to the eligibility criteria, patient recruitment, and allocation ratio.

Adaptive designs can also introduce investigator-driven bias that is difficult to detect as changes may be post-hoc decided based on results, rather than preplanned based on arising circumstances. The systematic review by Mistry et al² on adaptive design methods in oncology trials found that 91% (49/54 papers) stated their adaptive methods to be predetermined; however, it is probable that this review missed a large proportion of existing adaptive designs in their trials, as only 4% (2/54 papers) explicitly stated their trials as being “adaptive”. It is possible that there may be a higher number of existing adaptive trials that may have made post-hoc adaptations to their study design than what was reported in the paper by Mistry et al.² Therefore, for an adaptive trial, it is critical to have an independent data monitoring committee that can adhere to the protocol, make per-protocol adaptations during the trial, and avoid post-hoc adaptations that may compromise the scientific validity of the trial. Moreover, while there are potential ethical advantages of adaptive trial designs (e.g., RAR that may expose fewer participants to ineffective treatments), it is important to acknowledge the potential ethical disadvantages such as potential for less transparency and potential concerns about unbinding if without appropriate firewalls.²⁵

Conclusion

Adaptive designs are an increasingly common approach to evaluate interventions and offer many advantages over better-known traditional designs. Their utility is limited, however, if clinicians or other readers are unfamiliar with underlying concepts. Herein, we have reviewed common critical concepts at an introductory level to assist clinicians and readers.