Making trials part of good clinical care: lessons from the RECOVERY trial

ABSTRACT

When COVID-19 hit the UK in early 2020, there were no known treatments for a condition that results in the death of around one in four patients hospitalised with this disease. Around the world, possible treatments were administered to huge numbers of patients, without any reliable assessments of safety and efficacy. The rapid generation of high-quality evidence was vital. RECOVERY is a streamlined, pragmatic, randomised controlled trial, which was set up in response to this challenge. As of April 2021, over 39,000 patients have been enrolled from 178 hospital sites in the UK. Within 100 days of its initiation, RECOVERY demonstrated that dexamethasone improves survival for patients with severe disease; a result that was rapidly implemented in the UK and internationally saving hundreds of thousands of lives. Importantly, it also showed that other widely used treatments (such as hydroxychloroquine and azithromycin) have no meaningful benefit for hospitalised patients. This was only possible through randomisation of large numbers of patients and the adoption of streamlined and pragmatic procedures focused on quality, together with widespread collaboration focused on a single goal. RECOVERY illustrates how clinical trials and healthcare can be integrated, even in a pandemic. This approach provides new opportunities to generate the evidence needed for high-quality healthcare not only for a pandemic but for the many other conditions that place a burden on patients and the healthcare system.

Introduction

Over a year on from the emergence of the COVID-19 pandemic, communities and healthcare systems remain severely affected.¹ As of April 2021, global deaths have surpassed 3 million, with the UK among the worst-affected countries in terms of deaths per capita.² The burden on healthcare services has been profound: in January 2021, the daily numbers of new total and symptomatic cases in the UK, as well as hospitalisations and patients in intensive care beds, all hit record levels.^3–6

At the beginning of the pandemic, many potential treatments were proposed based on limited scientific evidence.^7–9 Unfortunately, the health crisis led to widespread adoption of these untested (and potentially harmful) therapies.^10–14 A robust response demanded the launch of well-designed randomised controlled trials (RCTs), capable of generating the evidence so vitally needed. Faced with a new virus and associated disease, there was substantial uncertainty about which treatments may cause benefit or harm. Observational analyses began to emerge and to be acted upon, despite them being unable to address unmeasured or unknown confounders.^15–18 Hence, RCTs are necessary to resolve such uncertainty.^19,20

Moderate treatment effects (ie proportional reductions in mortality of 20%–25%) were plausible, and still hugely relevant.^15,21 Discriminating such modest treatment effects from the absence of any meaningful benefit at all requires both randomisation (to avoid bias) and large sample sizes (to minimise the play of chance).

Simultaneously, the scientific response needed to be fast and coordinated and minimise additional burden for overstretched healthcare professionals but without compromising the quality of the evidence produced.^22–24 Elsewhere, failure to adhere to these principles has resulted in duplication and wasted efforts during the pandemic.²⁵

The RECOVERY (Randomised Evaluation of COVID-19 Therapy) trial was set up in response to this emergency. The trial aims to integrate randomised assessment of interventions for COVID-19 into routine NHS care by providing an adaptive trial platform. Recruitment is simple, fast and inclusive, with all patients hospitalised with confirmed or suspected COVID-19 in the UK being potentially eligible. Data collection is streamlined and complemented with linkage to national healthcare datasets, facilitating rapid enrolment and complete follow-up of acutely ill patients. This approach has allowed the generation of reliable information about which treatments have meaningful clinical benefits (and which do not). The results have been rapidly translated into clinical practice, benefiting both patients and healthcare services.

Here, we describe the design and setup of the RECOVERY trial as an example of how a streamlined platform trial can quickly provide reliable scientific evidence in a time of global uncertainty. We discuss the principles underlying the trial design, and how this experience may help shape a new vision for an integrated future of healthcare and medical research.

The RECOVERY trial

Quality-by-design as a framework for simple and effective trials

RECOVERY is a randomised, controlled, open-label, platform clinical trial assessing potential therapies for hospitalised COVID-19 patients (Fig 1). The study uses a pragmatic approach based on quality-by-design principles; the aim is to inform clinical care by focusing on the vital aspects necessary to produce high-quality evidence, while removing any extraneous requirements (Table 1).^26–29

Fig 1.

RECOVERY trial design. a) Original trial design. b) Latest RECOVERY design (April 2021). R = randomisation.

Table 1.

Key design features of the RECOVERY trial, exemplifying the application of quality-by-design principles to clinical trials and the production of rapid, reliable results

Design feature	Rationale
Randomised	Avoidance of systematic error (bias).
Large	Avoidance of play of chance. Provide adequate statistical power to detect moderate, but important, treatment effects across a broad range of circumstances.
Simple	Focus on critical components of quality that preserve safety and reliability. Do not include extraneous procedures. Streamlined study procedures at set-up, training, recruitment, randomisation, treatment delivery and follow-up facilitate timely results for patients and protect busy healthcare staff.
Inclusive	Facilitates prompt recruitment. Provides translatable results for all patients (both in and outside the UK).
Objective clinical outcomes	Clinically relevant results. Resistant to bias. Allow data linkage for complete follow-up.
Linkage to national healthcare datasets	Reduced data collection by local research teams. Complete follow-up improving study reliability. Rapid collection of data to inform data monitoring committee analyses. Low-cost long-term follow-up.
Collaborative	Buy-in from stakeholders at every level into study quality. Transparency facilitates trust in study procedures, safety and results. Resources are not wasted.
Robust web-based systems	Accessible to site staff. Reliable randomisation. Built-in options on online forms encourage consistency and completeness of data.
Data quality monitoring	Study procedures are subject to ongoing and continuous quality evaluation during the trial.
Trial oversight	Investigators at local hospital best placed to oversee high-quality study conduct locally. Experienced data monitoring committee meeting regularly provide reassurances on efficacy, accountability and safety.

A quality-by-design approach focuses on identifying the critical aspects that could threaten trial quality; that is, the safety of patients in the trial and the reliability of the results (which are then used to inform the treatment of future patients), with careful planning, review and, if necessary, modification.³⁰ By prioritising what is truly important, this risk-based model allowed RECOVERY to be greatly streamlined, facilitating rapid and extensive recruitment by busy staff and the generation of randomised evidence as part of standard care, while preserving (and potentially improving) trial quality.^31,32 As put by the UK chief medical officers: ‘use of treatments outside of a trial, where participation was possible, is a wasted opportunity to create information that will benefit others.’³³

Study design and oversight

Eligibility is based on simple and clinically-relevant criteria: patients need to be hospitalised with proven or suspected COVID-19 and considered suitable for inclusion, as determined by their clinical team. There are no complex inclusion or exclusion criteria, and children and pregnant women can be recruited. The trial includes a diverse, representative population (almost 10% of all hospitalised COVID-19 patients in the UK have been included), facilitating translation of results into clinical practice.³

The platform design allows the trial to adapt: a master protocol provides a framework in which treatment arms can be added and removed as evidence emerges.^34,35 The focus on a disease rather than a particular drug, along with the broad eligibility criteria, makes the study flexible and resilient; in other words, the trial can evolve along with our knowledge of the pandemic.

The initial design included a multi-arm randomisation to usual standard-of-care alone vs usual standard-of-care plus either dexamethasone, hydroxychloroquine, lopinavir-ritonavir or azithromycin in a 2:1:1:1:1 ratio, respectively. This ratio aimed to avoid spurious findings arising as a result of unexpectedly good (or bad) outcomes by chance in the shared control group. Randomisation was driven by the uncertainty principle; if a treatment was definitely indicated or contra-indicated for a particular patient (or if it was unavailable in that hospital at the time), that treatment could be excluded from randomisation. However, the patient could still be included if at least one treatment arm was suitable and available.³⁶ The ability to easily exclude patients from randomisation to an intervention that is not available at a particular site provides great flexibility to roll out interventions in a staged or selective way. Since exclusions are only possible before randomisation, and controls are only compared with active treatment arms to which they could have been allocated (ie the intervention was both available and suitable for that patient), this does not introduce any bias in the randomised comparisons. An initial modification to the study design introduced a second randomisation to tocilizumab for sicker patients: those who were hypoxic and inflamed (Fig 1).

The trial subsequently introduced a factorial design, allowing eligible patients to be simultaneously included in more than one randomised comparison (an approach pioneered successfully in trials of treatments for cardiovascular prevention and acute myocardial infarction in the 1980s):^37,38

• part A (early immunomodulation 1): comprised the initial treatment arms; these were gradually closed as results became available and replaced by colchicine and, later, dimethyl fumarate versus usual standard of care (SOC) in a 1:1 ratio

• part B (antibody-based therapy): comparing convalescent plasma or a synthetic monoclonal antibody cocktail versus SOC in a 1:1:1 ratio

• part C (anti-thromboembolic therapy): comparing aspirin with SOC in a 1:1 ratio

• part D (early immunomodulation 2): comparing baricitinib with SOC in a 1:1 ratio.

The choice of drugs to be studied in the trial was initially based on advice from the New and Emerging Respiratory Virus Threats Advisory Group (NERVTAG), and subsequently by the COVID-19 Therapeutics Advisory Panel (UK-CTAP) established specifically for this purpose.^39,40 UK-CTAP considers potential treatments based on the emerging literature and suggestions made via a public website. Key factors that determine the prioritisation are the existing information on efficacy, safety and dosing, and the overlap with ongoing evaluations in RECOVERY and in other trials around the world. This independent, national committee helps to ensure that the trial is aligned with national priorities and makes its recommendations to the RECOVERY chief investigators and the chief medical officer for England.

RECOVERY is sponsored by the University of Oxford and run by a central coordinating office composed of staff from the two clinical trials units within the Nuffield Department of Population Health, in close collaboration with the NIHR Clinical Research Networks and NHS trusts. Decisions on the conduct of the study are made by a steering committee, blind to the study results. An independent data monitoring committee (DMC) oversees the study by regularly reviewing unblinded data and advises on continuing, stopping or modifying recruitment to the treatment arms.

Streamlined setup

Trial set-up was expedited, with a standard contract for hospital sites and no specific recruitment targets; the emphasis being on recruiting as many eligible patients as quickly as possible. The study is conducted by local staff with appropriate education, training and experience for their role. Specific training is delivered as a series of short online tutorials focused on those aspects of the trial that differ from the way usual care is provided in NHS hospitals.

The trial took just 9 days from protocol finalisation to inclusion of the first patient (a process that would usually take many months), with the 1,000th patient recruited just 15 days later (Fig 2).⁴¹

Fig 2.

The first 100 days of RECOVERY trial.

Study procedures

Informed consent is obtained by local clinical teams using a simple information leaflet and consent form. If a patient lacks capacity, consent may be obtained from a legal representative (either a relative or, if a suitable relative is not available after reasonable efforts to locate one, an independent doctor). Consent is then sought with the patient if they recover capacity.

Study treatments, determined by the web-based randomisation system, are prescribed and administered in line with local policies. The trial is open-label, meaning that patients, clinicians and the study team are aware of individual treatment allocations, and no placebos are used; this simplifies study procedures, as development of and provision of adequate placebos would have delayed initiation. Potential biases arising from the open-label nature were minimised by focusing on objective clinical endpoints (such as mortality) and complementing follow-up through central record linkage (which is unlikely to be influenced by knowledge of treatment allocation). Importantly, aggregated results based on treatment allocation are not available to the research teams, patients or the trial steering committee, who are blind to the study outcomes unless otherwise advised by the DMC.

Outcomes and data collection

The primary outcome is all-cause mortality within 28 days of randomisation. Secondary outcomes are duration of hospitalisation and, among participants not on ventilation at baseline, a composite of death or need for invasive mechanical ventilation. These outcomes represent meaningful, objective clinical events which are easily ascertained.

Data collection is simple and performed using web-based systems; inclusion requires only the completion of a short form (taking on average 10 minutes); a brief follow-up form is then completed at the time of either discharge, death or 28 days after randomisation (whichever occurs first). Finally, the evaluation of safety is largely driven by the results of unblinded analyses conducted by the independent DMC which compare clinical event rates between those randomly allocated with an active treatment arm vs usual care alone. Given that the study treatments are largely well-known, reporting of individual serious adverse events focuses on those that are considered with reasonable probability to be related to the study treatments. For particular therapies (eg antibody-based therapies), simple direct questions are used to seek information on specific adverse events of interest (eg transfusion reactions for convalescent plasma, thrombosis and haemorrhage for aspirin, and cardiac arrhythmia for hydroxychloroquine).

Use of routine healthcare data

Data recorded by local staff is complemented by linkage to centrally-held datasets generated as part of routine NHS care and national registries: these cover hospital admissions and intensive care, death registrations, SARS-CoV-2 testing, renal replacement therapy as well as primary care records and medications dispensed in the community. In total, RECOVERY is harnessing over 25 different datasets. The extensive data linkage allows the randomisation and follow-up forms to be shortened, minimising burden for unwell patients and busy healthcare staff, while ensuring extremely high follow-up completeness (>99% for the primary outcome).^35,42–46 Furthermore, since many patients are transferred between hospitals for provision of intensive care, the routine healthcare datasets allow the ascertainment of outcomes which could not be recorded by the local research team. Follow-up will continue longer term through ongoing linkage, potentially including additional parameters. For each of the key analysis variables (eg death, hospital discharge and type of respiratory support), algorithms were developed to combine information from the different sources available; for example, information on fact, date and cause of death is acquired from nine different data sources.

Statistical analysis framework

As the trial was being planned, not enough information was available to adequately estimate appropriate sample sizes: the scale of the pandemic, the within-trial clinical event rate and plausible size of any benefits of treatment were all unknown. The trial, therefore, has no fixed effect size for any treatment comparison, with the goal being to recruit as many eligible patients as possible. The DMC regularly reviews unblinded data to assess whether sufficiently strong evidence has emerged on mortality (with a range of uncertainty around the study results that is narrow enough) to inform national and global policy.^47,48 Alternatively, the trial steering committee may also choose to stop a particular treatment arm when it is believed that, based on blinded assessments of recruitment and event rates, the trial has sufficient power to detect a clinically important result for the primary outcome; for example, the steering committee (blinded to treatment allocations) chose to close the dexamethasone arm once 2,000 patients had been randomly allocated to the active arm and could be compared with 4,000 patients allocated to usual care alone since there was over 90% power at two-sided p=0.01 to detect a 20% proportional reduction in the primary outcome of 28-day mortality from 20% to 16%.³⁵

Comparisons of study outcomes include all participants randomised to the treatment arms being considered, regardless of whether patients received (or not) their allocated treatment, following the ‘intention-to-treat’ principle.

Impact on patient care and public health

As of April 2021, RECOVERY has recruited over 39,000 patients from 178 sites representing all acute UK hospital trusts (Fig 3). Within 100 days of opening, the trial showed that dexamethasone (an inexpensive and widely available drug) can reduce 28-day mortality by up to one-third in hospitalised patients requiring oxygen or ventilatory support.³⁵ These results have subsequently been validated by other trials.⁴⁹ It is estimated that dexamethasone may have already saved approximately 1 million lives globally.^50,51 More recently, RECOVERY demonstrated that tocilizumab can provide an important additional mortality benefit in severely ill patients already receiving steroids, such that the combination of both treatments reduces mortality by one-third for those on oxygen and nearly one-half for those on invasive ventilation.⁴⁵

Fig 3.

a) Map of RECOVERY trial sites across the UK. Each green circle represents the location of an active study site with the size corresponding to the number of participants recruited. b) RECOVERY recruitment timeline with daily COVID-19 hospitalisations in Great Britain (data from NHS).

Conversely (and just as importantly), RECOVERY has shown an absence of meaningful benefit from broadly used and promoted drugs such as hydroxychloroquine, lopinavir-ritonavir and azithromycin, as well as convalescent plasma (a therapy offered to hundreds of thousands of patients globally), sometimes refuting previous small or biased studies.^42–44,46 These negative results have permitted the dismissal of ineffective, potentially harmful and costly therapies, saving resources (in terms of both treatment supply and healthcare staff time) and protecting patients from unfounded and unproven promises of benefit.

RECOVERY has shown that a simple (but robust) randomised trial design is able to provide the answers needed to differentiate evidence from hypothesis, fact from belief and truth from hype. This clarity has brought benefits to individual patients and public health, while easing the pressure on the health service and contributing to the broader societal response to the COVID-19 crisis.

A new vision for the joint future of clinical research and healthcare

Randomisation as part of routine care

COVID-19 has reasserted the importance of high-quality clinical trials, and provided an opportunity to rethink how trials and routine care interact. Albeit commonly seen as worlds apart, both share the same goal: the provision of effective and safe care that improves outcomes for patients. The pragmatic approach employed by RECOVERY has shown that the staff, procedures and data involved in routine patient care are capable of producing reliable and important scientific evidence, and can be a powerful driver of progress. RCTs should be seen as an everyday part of clinical care, with patients being randomised whenever there is substantial uncertainty regarding an intervention (‘when in doubt, randomise’). With an appropriate question, randomisation is no more ethically challenging than treatment selected on the basis of no evidence. Similar collaborative platform trials could be launched for important disease areas to resolve both current and future clinical questions.⁵² A small number of robust, coordinated, large RCTs do not need to be cost-prohibitive; in fact, they are likely to be more cost-effective than the current model of sequential single-issue clinical trials.^29,32,53,54 The safety and efficacy of the proposed interventions can then be reliably assessed using data continuously generated as part of clinical care, improving outcomes and efficiency; for example, all patients with acute myocardial infarction, stroke, specific cancers, community acquired pneumonia or antibiotic-resistant infections could be offered participation in perpetual randomised trials as part of routine NHS care. The importance of such programmes becomes particularly evident when considering that cardiovascular disease and cancer alone were responsible for 167,000 and 166,000 deaths in the UK in 2018, respectively, while antibiotic-resistant infections accounted for 12,000 deaths in 2016.^55–57

Achieving success through an open, collaborative and coordinated response

The success of RECOVERY has only been possible through unprecedented collaboration: from the staff at every site across the UK to the extraordinary contribution of all patients involved, and the support and involvement of key stakeholders at various levels, including ethics committees, chief medical officers, the Department of Health and Social Care, the Medicines and Healthcare products Regulatory Agency and the National Institute for Health Research Clinical Research Network. A vital aspect of this outstanding buy-in has been transparency; throughout the trial, trial enrolment numbers and all study materials (including the study protocol and statistical analysis plan) have been openly available on the trial website (www.recoverytrial.net), helping to build the trust of clinicians, patients and the general public, thus facilitating rapid translation. Simultaneously, RECOVERY illustrates the immense opportunity offered by a coordinated NHS at a time other less integrated healthcare services have generally failed to generate reliable scientific evidence.^58–60 This experience can cement the UK as the global leader in the robust evaluation of health interventions, improving care for current and future patients in the NHS and across the world.

For this to become a reality, high-quality RCTs need to be fully embedded at the heart of good quality clinical care. Long-term funding is needed to support and expand the established clinical trials capacity. In parallel, RCTs should become part of every clinician's work, and not just the preserve of academics and teaching hospitals. Doctors and other healthcare professionals should have more time assigned to clinical trials, which ought to be seen as an inherent part of training and professional development, potentially through initiatives such as the NIHR Associate Principal Investigator Scheme.⁶¹ Data collected within the NHS needs to be made promptly and securely accessible to bona fide researchers. Research, information governance and trials regulations need to be streamlined, coordinated and proportionate in order to remove burdensome requirements that do not improve quality.^32,62 Finally, public health bodies, and professional and academic societies should advocate strongly for RCTs to resolve areas of therapeutic uncertainty, rather than issuing recommendations based on low-quality evidence (Table 2).

Table 2.

Actions required to fulfil a new vision for the joint future of clinical research and healthcare

Action	Rationale
Incorporate randomisation as part of routine care	Generation of reliable, conclusive results that benefit future patients through the delivery of standard NHS care. Engagement of clinicians and patients in continuously improving care.
Set-up small number of carefully selected perpetual large, simple randomised trials	Drive forward improvements in patient care for the most relevant healthcare challenges without prohibitive expense.
Collaboration at every level	Stakeholder buy-in at every level facilitating active engagement at every hospital. Coordination of effort and resources.
Prioritisation of clinical research	Reflect importance of clinical research in political, clinical, regulatory and academic leadership. Support initiative with adequate resources including funding, training and time for all healthcare staff.
Removal of barriers in research	Promote inclusivity and collaboration. Facilitate the generation of timely results to benefit patients. Make data transparent for use in bona fide research that benefits patients. Refocus regulation on quality.
Become a learning healthcare system	Position the NHS as the global leader in integrated clinical research that benefits patients worldwide.

Conclusion

The COVID-19 pandemic has required a refocus on the essential aspects of clinical trials needed to provide rapid and reliable answers to key therapeutic questions. Arbitrary use of unproven treatments must be avoided; instead, large, robust, simple RCTs should become a regular component of routine care. RECOVERY has demonstrated what can be delivered when this is realised. Lifesaving treatments have been identified and ineffective treatments have been abandoned, delivering benefits to patients all around the world. The lessons learned need to be consolidated and expanded to ensure future trials remain focused on reliably answering the right questions for patients at the right time. The delivery of high-quality evidence-based medicine requires a health system that is honed to generate robust evidence of the impact of healthcare interventions through randomised clinical trials.