Establishing COVID-19 trials at scale and pace: Experience from the RECOVERY trial

Abstract

The Randomised Evaluation of COVID-19 Therapy (RECOVERY) Trial was set up in March 2020 to evaluate treatments for people hospitalised with COVID-19. To maximise recruitment it was designed to fit into routine clinical care throughout the UK, and as a result it has enrolled more patients than any other COVID-19 treatment trial. RECOVERY has shown four drugs to be life-saving – dexamethasone, tocilizumab, baricitinib and casirivimab-imdevimab – and a further six have been shown to be of little or no benefit. In each case, results from RECOVERY were clear enough to rapidly influence global practice. Some of the reasons for this success relate to its particular setting in the UK during the SARS-CoV-2 pandemic, but many are generalisable to other contexts. In particular, its focus on recruiting large numbers of patients to identify or rule out moderate but worthwhile benefits of treatment, and the design decisions that followed from this. Similar large streamlined trials could produce similarly clear answers about the treatment of many other common diseases.

1. Introduction

In January 2020, at the start of the SARS-CoV-2 pandemic, we knew nothing specific about COVID-19 treatment. In 2022, after two years of unprecedented attention on a single disease by researchers worldwide, we know much more. Several antivirals have been identified that can alter the course of disease, largely eliminating the risk of complications if started within the first few days of infection (Gottlieb et al., 2021, 2022; Gupta et al., 2021; Hammond et al., 2022; Weinreich et al., 2021). For those who have developed significant lung disease, several drugs are now known to reduce the risk of death by interfering with the immune-mediated damage that causes respiratory failure (RECOVERY Collaborative Group, 2021a; RECOVERY Collaborative Group et al., 2022; The WHO Rapid Evidence Appraisal, 2020). The pandemic has led to many innovations in the identification of candidate therapies and their pre-clinical development, but each advance in COVID-19 therapy has also required robust results from randomised clinical trials. At least 3000 COVID-19 treatment trials, evaluating hundreds of different therapies, have been planned since the start of the pandemic, and over 500 have reported results, but nearly all have been too small to provide answers clear enough to guide clinical practice or direct future research. (The COVID-NMA initiative) As a result, we have been dependent on relatively few large trials to produce most of the reliable data on COVID-19 treatment. The RECOVERY Trial (Randomised Evaluation of COVID-19 Therapy) is the largest of these, and has now produced clear answers on ten different therapies. Here we describe the rationale for large simple trials like RECOVERY, and the consequent design decisions that contributed to its success.

2. Principles

To produce definitive answers that inform clinical practice, trials are needed that can reliably identify or rule out moderate but worthwhile benefits of treatment on important outcomes, for example a reduction of ‘only’ 15–20% in the risk of death in patients hospitalised with COVID-19 (Yusuf et al., 1984). Effects of this size can justify widespread treatment use, and could avoid many deaths for a common disease. Moderate benefits may also add up, leading to large benefits when several effective treatments are given together. Alternatively, demonstrating the absence of even a moderate benefit of treatment can justify abandoning its use, and direct future research towards other areas. In order to identify or rule out moderate treatment effects in a clinical trial two things are needed:

• i)Randomisation, to eliminate biases that may be introduced if patient characteristics influence the treatment they receive
• ii)Adequate size, to ensure the play of chance cannot obscure a worthwhile treatment effect

The increasing availability of datasets containing medical records from tens or hundreds of thousands of patients makes it appealing to use these to identify effective treatments. Random errors may be negligible in observational studies of this size, but it remains almost impossible to ensure that moderate systematic errors (biases) are eliminated during analysis (Collins et al., 2020). There is also a greater risk of selective reporting in observational studies, as the exact population analysed and statistical model used are rarely prespecified, leading to another potential source of bias. Throughout the pandemic, the reliability of non-randomised studies has often been exaggerated, contributing to the inappropriate treatment of millions of people (a fraction of whom would have produced reliable evidence if taking part in a randomised trial) (Million et al., 2021; Casadevall et al., 2021; Garibaldi et al., 2021). In randomised trials the opposite problem predominates, as although biases can easily be eliminated, many difficulties conspire against the conduct of trials that are sufficiently large to minimise the impact of random error and provide the precision needed to change practice. These difficulties include the challenges of creating a collaboration that spans many different healthcare organisations, the time required to identify and recruit each patient, the need to intervene in clinical care conducted by busy clinicians, and extensive regulatory requirements (some of which do not contribute to patient safety or the validity of the trial results). Producing clear answers requires a focus on recruiting adequate numbers of patients whilst maintaining the few critical features needed to produce robust results, in particular randomisation with allocation concealment, good adherence to allocated treatment, minimal loss to follow up, and intention-to-treat analysis. (The Good Clinical Trials Collaborative).

3. Design

Planning for RECOVERY started in early March 2020, weeks before the arrival of the first wave of COVID-19 in the UK. Several treatments had already been proposed, but none with good evidence to support their use (Lenzer, 2020; Mahase, 2020). Around a quarter of patients admitted to hospital with COVID-19 were dying from the disease, so the priority was to rapidly identify treatments that could reduce the risk of death, even if only moderately. As a platform trial, RECOVERY was designed to simultaneously evaluate several different treatments for hospitalised patients with COVID-19, with the possibility of adding new drugs that appeared promising, and removing old ones when a worthwhile benefit had been identified or ruled out (Pessoa-Amorim et al., 2021). It was clear that this would require the recruitment of tens or hundreds of times more patients than are typically involved in clinical trials, and that it would need to be set up within weeks, before hospitals became overwhelmed with the first wave of infection. These two considerations have led to the major design features of the trial:

3.1. Streamlined

An overriding consideration is the need to avoid additional burden on busy clinicians, so trial procedures are streamlined as far as possible. The randomisation form is simple, collecting a few major baseline characteristics and ensuring eligibility, and a single follow up form is completed at the earliest of discharge, death or at 28 days. Extraneous procedures are minimised, although some comparisons have included biological sampling, with specimens mailed to a central laboratory. The trial is open label, as use of placebo was not practical when setting up the trial at speed or with the frequent changes in the therapies. Monitoring is proportionate, accepting that occasional discrepancies between data sources, which include routine health records, do not affect the validity of the trial results.

3.2. Web-based

RECOVERY is coordinated by a team of around 20 people at the Nuffield Department of Population Health in Oxford, and in-person contact with sites has been largely impossible because of pandemic restrictions. The collaboration has been helped greatly by a comprehensive trial website that includes study documents, succinct training modules, trial updates, and results. Randomisation and follow-up use web-based forms, and the use of off-the-shelf IT systems meant these could be set-up quickly and updated as frequently as needed.

3.3. Efficient randomisation

RECOVERY initially compared multiple treatment arms with a shared control group, and more recently has used factorial randomisation, in which patients are randomised to active treatment or usual care independently for each of the suitable study treatments. These features have meant that each patient has contributed to an average of two treatment comparisons, doubling the effective size of the trial.

3.4. Inclusive

Eligibility criteria are simple, with few exclusions, and aspects of care other than the receipt of allocated treatments are left to the patient's clinical team. This facilitates prompt recruitment of a wide range of patients, and provides widely applicable results.

3.5. Linked to national datasets

Using national healthcare records for the measurement of trial outcomes reduces the need for data collection by local teams and improves completeness of follow-up, improving reliability of study outcomes. This also allows low-cost long-term follow-up.

3.6. Relevant outcomes

The main trial outcomes are objective and important: death, progression to ventilation, and time to discharge. This means that clear results are more likely to change practice, and avoids biases that could affect subjective outcomes given the lack of placebo.

3.7. Appropriate oversight

The Data Monitoring Committee have met frequently – fortnightly during periods of rapid recruitment – to review unblinded data and ensure treatment comparisons do not continue after producing a definitive answer.

3.8. Rapid communication

Results are typically announced within weeks of a trial arm closing, with simultaneous availability of preprint manuscripts in most cases, avoiding publication delays.

These design features have been necessary, but not sufficient for the success of RECOVERY, and just as important has been the collaboration underpinning the trial. The initial protocol and over twenty revisions have been given expedited approval by the Medicines and Healthcare Regulatory Agency and by a single research ethics committee, with no need for local duplication. The trial had early endorsement by National Health Service (NHS) leadership and the UK Chief Medical Officers, who established it as one of two national COVID-19 treatment trials in hospitalised patients, and encouraged randomisation rather than off-label prescribing of untested COVID-19 treatments. The NHS is a single, centrally organised health system that provides all acute medical care in the UK, which made it feasible to rapidly set up RECOVERY in around 180 acute hospital organisations covering the whole country, using a standard non-negotiated contract with each of the hospital organisations. This involvement of RECOVERY with the national response to COVID-19 also encouraged the active involvement of clinicians and patients. The trial was supported by the National Institute for Health Research Clinical Research Network, who at each site identified a principal investigator and provided support from research nurses. Finally, the national COVID-19 Therapeutics Advisory Panel systematically considered candidate therapies, and recommended the most promising for evaluation in RECOVERY. (Outcomes of the UK COVID) Few other trials will have had the benefits of a similarly coordinated healthcare and research infrastructure.

4. Impact

The first patient was recruited to RECOVERY on March 19, 2020, less than three weeks after planning started, and two months later over 10,000 had been randomised. The total is now more than 47,000, including over 1000 from non-UK sites in Nepal, Indonesia, Vietnam, South Africa, Ghana and India. In the UK, most of the hospitals that recruited most successfully were non-academic centres that had previously had little involvement in clinical trials. This scale has provided clear answers for ten different COVID-19 therapies, with five more currently under evaluation (Fig. 1 ). Four effective drugs have been identified so far, each reducing the risk of death by 10–20%. Three of these are immunomodulators – dexamethasone, tocilizumab, baricitinib – and one is an antiviral monoclonal antibody combination – casirivimab-imdevimab (RECOVERY Collaborative Group, 2021a; RECOVERY Collaborative Group et al., 2022; RECOVERY Collaborative Group et al., 2021; RECOVERY Collaborative Group, 2022). Dexamethasone, a cheap and widely available drug, was the first life-saving treatment identified for COVID-19, with results reported three months after recruitment of the first patient. Each result has been clear enough to be rapidly incorporated into treatment guidelines, and the effects of these treatments are additive, meaning that combination therapy could reduce the risk of death by around 40–50% in hospitalised patients (RECOVERY Collaborative Group, 2021a; RECOVERY Collaborative Group et al., 2022; RECOVERY Collaborative Group, 2022).

Fig. 1.

Timeline of recruitment and the treatments tested in RECOVERY.

Six other drugs have been found to be of no material benefit in the treatment of hospitalised patients: hydroxychloroquine (which may actually be harmful), lopinavir, azithromycin, convalescent plasma, aspirin and colchicine (Axfors et al., 2021; RECOVERY Collaborative Group et al., 2020; RECOVERY Collaborative Group, 2020; RECOVERY Collaborative Group, 2021b; RECOVERY Collaborative Group, 2021c; RECOVERY Collaborative Group, 2021d; RECOVERY Collaborative Group, 2021e). Many were widely used prior to the RECOVERY results, so these clear results have also changed global practice.

5. Conclusions

RECOVERY provides an example of the power of large-scale randomised evidence to produce reliable answers that change clinical practice and improve lives. Some things that led to its success may be difficult to replicate outside of its particular setting – in a relatively unified national health and research infrastructure during a pandemic. This should not obscure the features that were also necessary, and which are transferrable to trials in other settings. In particular, a pragmatic, streamlined design with a clear focus on randomising many thousands of patients to definitively answer each question addressed. A lack of adequately sized trials is common in most areas of medicine, and could be improved by better clinical trials regulation, wider recognition of the need for randomised evidence, and a clearer focus on the few things really needed to produce reliable results. (The Good Clinical Trials Collaborative) Greater focus on these could lead to substantial progress in the treatment of many other common diseases, long after the current pandemic is over.

Funding

The RECOVERY trial was funded by UK Research and Innovation (Medical Research Council) and the National Institute of Health and Care Research.

CRediT authorship contribution statement

Leon Peto: Writing – original draft, Writing – review & editing. Peter Horby: Writing – review & editing, Supervision, Funding acquisition. Martin Landray: Writing – review & editing, Supervision, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

No data was used for the research described in the article.