Pragmatic, adaptive clinical trials: Is 2020 the dawning of a new age?

Abstract

Given the high case fatality rate of SARS-CoV-2, for which there is no cure and no vaccine, clinicians are forced to make decisions about how best to manage patients with limited high-quality evidence to guide treatment. Traditional randomized controlled trials provide strong experimental evidence, however, tend to be slow, inflexible, and have limited generalizability. Adaptive and pragmatic designs are an attractive alternative, which meet our ethical obligation during the SARS-CoV-2 pandemic to balance speed, agility, and generalizability with both prospective study and scientific rigor.

As of April 2020, the SARS-CoV-2 infected more than 2 million people and caused more than 130,000 deaths world-wide, with no end in sight [1]. Given the high case fatality rate of SARS-CoV-2, for which there is no cure and no vaccine, the imperative to save lives in the present compels clinicians to embrace any potentially effective treatment, including those that are empirical or unproven. On the time-honored pyramid of medical evidence—increasing in validity from expert opinion, preliminary data derived from basic pre-clinical laboratory studies, case reports, and observational studies to the pinnacle of rigorous scientific investigation, the randomized controlled trial— clinicians on the frontlines find themselves stuck at the bottom.

Since the mid-20th century, the gold standard of quality scientific evidence is derived from double-blind, placebo-controlled, multi-center, randomized controlled trials (RCTs), however, generating data from these studies is as cumbersome as their name suggests. Designing, executing, and analyzing outcomes from such RCTs takes years and consumes vast sums of funds and manpower. The conduct of traditional RCTs is challenging, they require large numbers of patients with restrictive eligibility criteria, and are often limited in terms of their generalizability to “real world” patients. Such RCTs—particularly those conducted in the U.S.—often proceed at a glacial speed and are associated with recruitment delays and cost overruns. By their nature, traditional RCTs are inflexible and besieged by long gaps between study completion, publication, dissemination, and implementation. Clearly, none of these attributes are desirable during a global pandemic that requires answers that can be achieved and actualized quickly. SARS-CoV-2 is a modern apocalypse that mandates a different, more nimble and agile, investigational approach.

While traditional RCTs address the question, “Exactly how effective is a given treatment in a pre-specified and well-defined population?”, during a pandemic, there is an urgent need to answer a different question, “Is this treatment effective at all?” We are witnessing, in 2020, a crisis within the clinical research community and the need to pivot away from our usual comfort zone—rigid, conventional clinical trials targeted to a limited population-- toward more agile study designs, which can generate sound experimental evidence in a timeframe that matches the virus's swift spread. There is an ethical imperative to obtain scientifically sound data quickly and with as few subjects and resources as possible. Clinicians and investigators share the goal of delivering the most effective and safest treatments, and the speed and magnitude of the SARS-CoV-2 pandemic highlight the need to pursue this goal as rapidly as possible.

Conventional RCTs have for decades been the “top” of the evidence pyramid. However, beyond traditional designs, such as conventional RCTs and observational evidence, exists a worthy modern alternative: adaptive and pragmatic clinical trials. These designs strike a delicate balance between the conflicting needs of clinicians-- who need to make bedside decisions and to save lives now-- and the needs of the broader medical and scientific community, which needs to collect evidence of sufficient quality and scientific rigor to ensure that progressively better clinical decisions are made as the SARS-CoV-2 pandemic evolves. Pragmatic and adaptive trials produce true “experimental evidence—" unlike case reports and observational studies—and thus results generated from these designs provide a sound basis for medical and policy decision making. Despite many attractive aspects of pragmatic and adaptive trials, their adoption has been limited for various reasons, including difficulty interpreting messy “real-world” data and challenges with implementing complex randomization schemes. However, data interpretation can be improved by using validated quality checklists, such as the CONSORT extension for adaptive trials,[2,3] and methodologic advancements over the past decade created an opening for more widespread adoption and acceptance of the validity of these designs.

Adaptive and pragmatic designs, as the name implies, have two essential features: pragmatism and adaptation. Pragmatism means that healthcare providers continue to do what they think is right for their patients and are not limited by restrictive investigational protocols, which is critical for recruitment and buy-in in a pandemic setting. This approach encourages rapid recruitment of a broad population and is appropriate if standard of care is not precisely defined at study onset or is likely to change during the trial.[4,5] Pragmatism also allows for different providers to interpret rapidly evolving data differently, which is critical for conducting research during a global pandemic. For example, data regarding the utility of hydroxychloroquine (HCQ) for the management of COVID-19 changed on a nearly hourly basis, with initial reports suggesting astonishing efficacy [6], subsequent reports that were ultimately retracted suggesting substantial harm [7,8], before a randomized controlled clinical trial suggested neither a benefit nor a harm [9]. Critical to conducting research in the setting of rapid diffusion of limited data, a pragmatic design allows for testing of a second therapy (for example, IL-6 inhibition) [10] while allowing clinicians to prescribe other medications and interventions according to their own interpretation of the best-available data at the moment.

Adaptation implies flexibility, and the ability to pivot how the trial is conducted as more data becomes available. Adaptations can come in many forms, including removing arms that appear to be ineffective or harmful, changing interventions as new data evolves, or weighting future randomization schemes as new data emerges [11]. An example of the latter is the “play-the-winner” design used in 1984 to study the impact of extracorporeal membrane oxygenation (ECMO) on survival in neonates with respiratory distress [12]. Adaptive randomization approaches use outcomes data collected from patients enrolled earlier in the trial to weight future randomizations to the arm that appears to be the most effective, and has been used successfully in some phase II and phase III clinical studies.[13,14] Thus, play-the-winner designs and other adaptive randomization approaches result in the largest number of participants receiving the intervention that appears to be the most effective and may minimize the number of participants required to achieve an answer. For example, in a study testing IL-6 inhibition for treating COVID-19, if there is evidence from early participants that suggests efficacy, then future participants will be more likely to be randomized to the treatment arm. This design satisfies the ethical need for equipoise and allows investigators to balance collecting strong evidence to help future patients while optimizing outcomes for the individual patients in the study, who have dedicated their health, well-being, and time knowing that their efforts may only benefit others. Play-the-winner designs maintain randomization, which is the key element of clinical trials that reduces the chance that confounding factors will explain the relationship between the exposure and the outcome.

In 1984, the highly efficient ECMO study required only 12 neonates to reach the conclusion that ECMO was effective for reducing mortality: 11 in the treatment arm (10 survived) and 1 in the control arm, who died. However, in 1984, this study was highly controversial, suggesting a magnitude of benefit too good to be true, and was not accepted as high-quality evidence. A subsequent ECMO study with a different adaptive design randomized neonates 1:1 until there were 4 deaths in one group, with the prespecified plan to treat additional patients with the more effective approach until 4 deaths occurred in that group or statistical significance was achieved [15]. The first 4 deaths all occurred quickly and all in the standard-of-care group, and the final results of the study, published in 1989, were 6/10 surviving with standard-of-care and 28/29 with ECMO. This adaptive design was also rejected as insufficiently robust, and a conventional RCT was undertaken from 1993 to 1995 [16]. Results of this traditional trial proved that ECMO was, indeed, effective, allowing a precise estimate that 3–4 neonates would need to be treated to prevent 1 death [16]. Based on these estimates, 30 neonates who would have survived, if offered treatment demonstrated to be effective in the imprecise but convincing adaptive trial published 10 years earlier, died in pursuit of the highest quality evidence.

In 2020, with SARS-CoV-2 threatening millions of lives, can we afford the luxury of conventional RCTs where the tradeoff of time spent in rigorous trial execution is incremental loss of life? Can we as a clinical research community adapt to meet the challenges of our time? We live in the information age, and effective evidence-based interventions must be allowed to spread as quickly as SARS-CoV-2 itself. Adaptive and pragmatic designs meet our ethical obligation during the SARS-CoV-2 pandemic to balance the rapidly changing standards of care with speed, agility, and scientific rigor as we seek the best treatments for our patients. The age of the pragmatic, adaptive clinical trial has come.

Funding

WBE is supported by NHLBI 1K12HL138049-01.

Declaration of competing interest

All authors have no conflicts of interest to report.