Multigroup, Adaptively Randomized Trials Are Advantageous for Comparing Coronavirus Disease 2019 (COVID-19) Interventions

Abstract

In this commentary, the authors explain and call for a broader use of outcome-adaptive randomization when designing clinical trials to test multiple COVID-19 interventions. This design potentially reduces the number of deaths or other adverse outcomes incurred during a trial.

We propose platform trials with outcome-adaptive randomization to efficiently select the most effective coronavirus disease 2019 (COVID-19) treatments. The global spread of severe acute respiratory syndrome coronavirus 2 infection is alarming in its geographic scope and in the number of associated deaths. There are currently no treatments proven to decrease mortality from COVID-19 further than what can be achieved through supportive care. Thus far, the choice of therapeutics has been limited to existing, repurposed medications. Given that some of the medications are perceived to have low toxicity, many have been embraced without evidence. Although remdesivir was recently found to shorten time to symptom resolution, evidence for survival benefit is inconclusive (1).

Well-designed, placebo-controlled trials have begun. On 1 June 2020, we searched ClinicalTrials.gov and found 308 phase 2 and 3 intervention studies aimed at COVID-19 that were open to enrollment; an additional 287 trials were posted but not yet recruiting (2). Those already recruiting are largely treatment trials testing known antiviral medications (remdesivir, favipiravir, and oseltamivir), antimalarials (hydroxychloroquine), immunosuppressive drugs known to be effective in the treatment of inflammatory or autoimmune disorders (sarilumab, tocilizumab, baricitinib, and others), or antiretrovirals for treatment of HIV infection (lopinavir, ritonavir, darunavir, and cobicistat).

Several large trials are already engaged in simultaneous testing of multiple treatment strategies in separate groups, with plans to discontinue any group that is definitively inferior at planned interim analyses, a format known as a platform trial. For example, the DisCoVeRy (Trial of Treatments for COVID-19 in Hospitalized Adults) trial in France is testing standard of care; remdesivir; lopinavir and ritonavir; and lopinavir, ritonavir, and interferon β-1a. The World Health Organization's Solidarity trial includes hydroxychloroquine in addition to those interventions listed above. The United Kingdom's RECOVERY (Randomised Evaluation of COVID-19 Therapy) trial is simultaneously testing 5 treatments as well, and REMAP-CAP (Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia) has amended its international treatment trial for community-acquired pneumonia (3–6). Other multigroup trials are taking place in Belgium, Norway, Spain, and the United States. In a short time, many trials have been initiated to test potentially beneficial interventions, some with adaptive design incorporated. Although cost, availability, and regulatory issues may limit the appropriateness or feasibility of testing certain treatments in some settings, these large trials demonstrate the feasibility of testing several interventions simultaneously to facilitate direct comparison.

A further innovation in the design of the clinical trials would be a broader use of outcome-adaptive randomization, a specific adaptive design feature that potentially reduces the number of deaths or other adverse outcomes incurred during a trial. To favor groups with more advantageous outcomes, outcome-adaptive randomization updates the allocation proportions on the basis of observed outcomes from cumulatively enrolled persons to date. For example, consider a study with a favorable or desirable binary outcome. After early group-specific event rates are observed during an interim analysis, outcome-adaptive randomization permits reallocation of twice as many subsequent participants to a group with a high event rate than to a group with half that event rate. These later-enrolled participants stand to benefit from the experience gained earlier in the trial. This approach is suitable for COVID-19 disease outcomes, which are known rapidly: Patients either recover or die within a few weeks. As such, the follow-up period for antiviral studies using remdesivir ranges from 14 to 29 days (1, 3). Although platform trials can drop ineffective groups at interim analyses, outcome-adaptive randomization further concentrates allocation among the current, better-performing groups. A simulation study done in 2016 showed that, relative to platform trials allowing early dropping of ineffective groups, a platform trial additionally incorporating outcome-adaptive randomization (or response-adaptive randomization) can lead to an 18% decrease in the number of poor outcomes (7). The advantages of updating randomization allocation are thus also measured in adverse outcomes averted.

A few issues related to adaptive randomization require consideration. The first is drift, which occurs when participants randomly assigned at later stages of the trial have a different pretreatment outcome risk relative to those enrolled earlier. Drift could occur in this setting if the stage of illness at the time of presentation for medical care changes as diagnostics become more available, or if the virulence of the infection changes through viral mutation or repeated person-to-person transmissions. For example, if widespread diagnostic testing is made available, persons enrolling later may present at an earlier stage of infection. In this scenario, genuine treatment benefit leading to higher allocation in some groups may not be distinguishable from benefit due to earlier presentation. However, this problem can be mitigated through stratification on the stages of allocation or through the use of previously developed rerandomization tests (8).

The second issue to consider when updating randomization proportions during a trial is that early study results can be highly variable. When few participants are enrolled, some groups can appear to do better than others on the basis of chance alone. The problems of bias and inflated significance induced by decisions made early in a sequentially designed trial can be addressed by validated post hoc corrections that account for potentially exaggerated treatment benefit estimates (9).

A third concern is the need for clinicians providing care to patients to remain agnostic about the relative potential benefit of the treatments they provide, even as randomization allocations change. Should care providers' equipoise falter before the end of the study, they may be strongly tempted to ignore subsequent treatment assignments. Approaches that mitigate this concern include masking investigators, when possible, and separating the roles of clinicians who are providing treatment from those assessing outcomes (10).

In summary, an adaptively designed, multigroup trial with outcome-adaptive randomization is especially appropriate for this large-scale severe acute respiratory syndrome coronavirus 2 outbreak. If interventions are tested separately over the next few months, additional time will be required to conduct direct comparison of the most effective treatments. A collaborative effort will help us to widely implement the most effective treatments as quickly as possible, and with potentially more persons receiving the most effective treatments. These recommendations complement those of the World Health Organization's R&D Blueprint group, which recently encouraged core protocols that are maintained until a definitive answer about efficacy is reached, perhaps spanning multiple infectious disease outbreaks (11).

Biography

Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M20-2933.

Corresponding Author: Amalia S. Magaret, PhD, University of Washington, Box 359300, Seattle, WA 98195-9300; e-mail, amag@uw.edu.

Current author addresses and author contributions are available at Annals.org.

Current Author Addresses: Dr. Magaret: University of Washington, Box 359300, Seattle, WA 98195-9300.

Dr. Jacob: Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, United Kingdom.

Drs. Halloran and Guthrie and Mr. Magaret: Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, PO Box 19024, Seattle, WA 98109-1024.

Drs. Johnston and Wald: Harborview Medical Center, University of Washington, 325 9th Avenue, Box 359928, Seattle, WA 98104.

Dr. Simon: University of Washington, Box 357232, Seattle, WA 98195-7232.

Author Contributions: Conception and design: A.S. Magaret, M.E. Halloran, A. Wald.

Drafting of the article: A.S. Magaret, C.A. Magaret, A. Wald.

Critical revision of the article for important intellectual content: A.S. Magaret, S.T. Jacob, M.E. Halloran, K.A. Guthrie, A. Wald.

Final approval of the article: A.S. Magaret, S.T. Jacob, M.E. Halloran, K.A. Guthrie, C.A. Magaret, C. Johnston, N.R. Simon, A. Wald.

Statistical expertise: A.S. Magaret, M.E. Halloran, K.A. Guthrie, C.A. Magaret, N.R. Simon.

Administrative, technical, or logistic support: C.A. Magaret, A. Wald.