Clinical trial design during and beyond the pandemic: the I-SPY COVID trial

To the Editor —

It is difficult to forget the chaos, anxiety and heightened urgency of early 2020, when COVID-19 became a global pandemic. Infections around the world were skyrocketing, while clinicians faced tremendous uncertainty over how to treat this devastating new infection. Hundreds of different therapeutic approaches were proposed, many of which had a relatively weak link to the pathophysiology of COVID-19, our understanding of which was rapidly evolving. Without data to guide clinicians, tens of thousands of patients received a wide range of untested therapies.

As chaotic as the early pandemic was, it also spurred tremendous innovation in clinical trial design. The RECOVERY adaptive platform trial in the United Kingdom demonstrated the substantial value of pragmatic phase 3 trials that could test a variety of well-established therapies in COVID-19, as has REMAP-CAP^1–5. Despite the enormous contributions of these studies, there remained an unmet need for a phase 2 mechanism for rapidly screening and triaging potential treatments for severe COVID-19 in a systematic and expedient fashion.

To address this need, in March 2020 we began planning a phase 2 adaptive platform trial, I-SPY COVID (Fig. 1). Given the large number of potential therapeutic approaches being proposed, the study was designed to rapidly evaluate and prioritize promising agents for further phase 3 testing. A pre-existing collaboration with the I-SPY clinical trials group to investigate the use of advanced study designs and precision medicine approaches in the critical care arena allowed us to leverage the experience and infrastructure gained in the highly successful and archetypal I-SPY2 trial in breast cancer^6–8. Several features of the I-SPY COVID trial may provide lessons that could be useful beyond the current pandemic.

Fig. 1 |. Study schematic for I-SPY COVID.

The I-SPy COVID adaptive platform trial is a trial for patients with severe COVID-19 in which up to four agents are evaluated in parallel on a backbone of standard of care. Participants who do not wish to participate in the randomized cohort or who meet exclusion criteria are enrolled in an observational arm (per a waiver of informed consent from the Institutional review Board) in which clinical and outcomes data are collected through medical records.

As a platform trial, I-SPY COVID evaluates up to four novel therapeutic agents in parallel, each on a therapeutic backbone (currently remdesivir and steroids) appropriate for severe COVID-19, defined as requiring ≥6 liter min⁻¹ nasal cannula oxygen. The therapeutic backbone also serves as a separate contemporaneous control arm. A master protocol permits agents to enter and leave the study through a simple protocol amendment. The primary study outcome is time to durable recovery (at least two days at WHO COVID level 4 or below, e.g., <6 liter min⁻¹ nasal cannula oxygen), with a co-primary endpoint of time to mortality.

Using a Bayesian analytic framework, between 40 and 125 patients are enrolled for each therapeutic arm, with pre-specified criteria for graduation (that is, declaring a therapy to be likely efficacious) or futility. Although initially drawing from the existing I-SPY2 site network, the trial has expanded to more than 30 sites across the United States and has enrolled over 2,100 patients. The trial’s agents committee, made up primarily of study investigators, has considered over 70 agents for evaluation; 10 of these agents entered the study, with 6 meeting the pre-determined futility threshold, 1 being halted due to logistical difficulties in drug administration and 3 actively being tested at the time of writing.

What are the lessons learned from this trial experience that may have important implications for future pandemics and/or clinical trials in a similar treatment space? First, there is a unique niche for phase 2 clinical trials that can rapidly evaluate repurposed or novel agents for which preliminary safety data exist, but fewer data are available in support of efficacy than would be advisable in a standard phase 3 study. Features such as the open-label design and the comparative effectiveness approach that forgoes placebo permit flexibility, particularly with variable routes of administration (intravenous, subcutaneous, inhaled). Moreover, we determined at the outset to seek strong signals of efficacy, while accepting the risk of missing more modest benefits in exchange for the goal of rapidly cycling and testing several agents at a time. These design decisions have enabled fast progress, with the caveat that I-SPY COVID is signal-finding rather than definitive and so subsequent phase 3 studies will be required for agents that graduate from the trial. Similar approaches may be useful in future pandemic settings when disease mechanisms are poorly defined, multiple agents need to be rapidly triaged, and/or there is potential for major therapeutic wins.

Second, cooperation across a wide variety of stakeholders was essential for both spurring innovation and speeding implementation. In addition to academic research hospitals, we intentionally recruited community-based sites that do not traditionally participate in clinical trials to enhance enrollment of a broad population. Engagement with trialists and statisticians experienced in adaptive Bayesian trial design, patient advocates, regulatory agencies familiar with the complexities of platform trials, and companies willing to provide their repurposed and novel agents (via the COVID R&D Consortium) was fundamental. For example, to allay the perceived risk to companies proposing candidate therapies for study, they required reassurance that a lack of efficacy in COVID-19 would not be taken to necessarily reflect upon their potential benefit in classical acute respiratory distress syndrome or sepsis.

Third, studying a variety of agents across many months in a global pandemic has also highlighted the enormous advantages of platform trials that employ a concurrent control arm able to evolve with changes in the standard of care — a relatively unique feature of I-SPY COVID. The standard of care for severe COVID has shifted dramatically over the course of the pandemic, beginning with remdesivir in late spring 2020 and the addition of dexamethasone shortly thereafter. Different viral variants have also emerged during the pandemic, which may also influence outcomes, as have effective vaccines. For these reasons, concurrent controls are critical as a means to reduce temporal bias, though this feature is not common to all platform trials during the COVID-19 pandemic. In our view, this lesson should be an enduring one that lingers even after the pandemic has drawn to an end.

Finally, our experience in I-SPY COVID has taught us that it is possible to balance pragmatism, safety and discovery in the context of a phase 2 trial, even during a pandemic. I-SPY COVID takes a moderately pragmatic approach to streamlined data collection, with a standardized method to ascertain adverse events and outcomes across all arms. An observational cohort of patients who meet trial criteria but are not randomized provides a real-world comparator arm. The trial is working to automate data collection from electronic medical records in order to ease the burden of conducting time-sensitive research when resources may be overstretched. I-SPY COVID is also unique in its biomarker development initiative, which incorporates the collection and study of biospecimens to investigate the biologic heterogeneity of severe COVID-19 that may influence outcomes and/or treatment effects⁹.

In conclusion, we hope that lessons learned from the I-SPY COVID trial will have important implications for the treatment of patients with severe COVID-19 and also for future trials in critically ill patients more generally. Our network is continuing to learn from our experience so far and to strive for ongoing improvements in our trial design; we hope to continue beyond the pandemic to identify effective pharmacotherapies for other critically ill patients, incorporating biologic phenotypes or treatable traits that may accelerate therapeutic discovery by identifying treatment-responsive subgroups.