Trials in pandemics: here we go again?

Editor—Coronavirus disease 2019 (COVID-19) demanded the wholesale re-organisation of health services and supportive care in the UK. And yet, heading into winter 2020 and facing a likely second surge, we remain unsure of how best to re-deploy resources to meet the needs of COVID-19 and non-COVID-19 patients alike. Simple and as yet unanswered clinical questions continue to impact the whole of health services around the world including the NHS in the UK.

We face a stark choice in our approach to the management of COVID-19. It is not the choice between drug A and drug B, but whether or not we are prepared to continue to derive policy without robust scientific evidence. There are pockets of excellence such as RECOVERY (arguably the most successful drug trial in COVID-19) and ISARIC (already reporting crucial observational evidence), and pockets of hope such as RECOVERY-RS (a rapidly deployed trial of ventilation strategy for COVID-19).1, 2, 3 But while both necessary and impressive, these trials are not sufficient.

Take just one example: the limited question of ventilatory support for severe COVID-19 pneumonia. At University College Hospital London, our first COVID-19 patient arrived on March 6, 2020. Reports from China and Italy indicated that our ICU would be overrun. We contacted colleagues from those countries and designed a new care pathway overnight. We triaged patients after an ‘oxygen challenge’ to either CPAP for those who responded, or invasive mechanical ventilation (IMV) for those who did not. In a neighbouring London teaching hospital, a different team made the same phone calls but came to very different conclusions. There, CPAP was largely excluded in favour of early IMV. This latter approach became the organising principle for the Nightingale Hospital in London. Nationally, the impact went further as we repurposed production lines in the airline and motoring industries to build new ventilators from scratch.^{4 , 5}

Neither the hypothesis that early IMV protected the lungs⁶ nor our assertion that CPAP was safe have yet been tested. Both contributed to the shutdown of non-COVID-19 care as critical care beds were filled. And nearly 6 months later, having treated many thousands of critically ill COVID-19 patients, we still do not know for which patients CPAP is sufficient, when to switch from CPAP to IMV, or for whom early IMV is the right choice. The tragedy is that from the outset we had both the tools (RCTs) and the opportunity (the patient numbers) to answer this and many other questions.

The success of the RECOVERY drug trial shows that it is possible to learn at pace and at scale. Just 6 days after Simon Steven's ‘call to arms’,¹ UK investigators submitted a protocol for an RCT studying antiviral, steroid, and antibiotic treatments. By June 2020, more than 11 000 patients from 176 hospitals had been recruited making it the ‘largest [RCT] … of potential COVID-19 treatments in the world.’ Early results have already debunked hydroxychloroquine and shown that dexamethasone saved lives.^{7 , 8} What is RECOVERY doing to make this work? It evaluates five treatment arms simultaneously. Those sites can participate as long as they can deliver two or more. Recruitment is excellent because clinicians are not obliged to randomise to interventions they consider unsuitable for a specific patient. A simple case definition suffices for enrolment and, because hospital mortality is high, the outcome is easy to capture. And finally, randomisation is done centrally, and once the treatment has been allocated there is no further work for the bedside clinical team.

RCTs require many patients in their numbers but we do not need to rehearse the scale of the pandemic. Even a simple simulation solely within the domain of critical care (Fig. 1 ) shows how quickly we could have answers with even a fraction of the resource that was spent on the Nightingale Hospitals, or the Ventilator Challenge.^{4 , 5} Ventilation strategy was just one speciality-specific example. The response to COVID-19 is much broader. It has forced us to modify and reduce many urgent, clinical services, from cancer to cardiac surgery. And we are now seeing worrying decreases in healthcare utilisation for non-COVID emergencies, and early signs of decreasing cancer survival.^{9 , 10}

Fig 1.

A simulation to illustrate how we might have learnt, had all patients admitted to ICU for coronavirus disease 2019 (COVID-19) been recruited into trials for non-pharmacological interventions. The simulation runs trials in groups of 446 patients, which provides 80% power to detect an absolute risk reduction of 10% from a baseline mortality of 50%, at an alpha threshold of 0.2. This is an intentionally relaxed set of thresholds to investigate a large number of candidate therapies, with the specific goal of identifying those with a maximum signal for harm or benefit. We use the daily admission numbers for COVID-19 to ICU as reported by the Intensive Care National Audit and Research Centre (ICNARC). The current best estimate of mortality is ∼50%, hence a reduction from 50% to 40% mortality (10% actual risk reduction, 20% relative risk reduction) would be a ‘big signal’. Assuming we run one trial sequentially after another, we could expect 17 trials to complete during the first surge. This is without implementing an adaptive Bayesian framework, which would not only be more efficient, but would allow for additive learning to improve the grade of evidence to a confirmatory level should a signal appear. We made the following technical assumptions. (1) The true underlying treatment effect is drawn from a zero mean normal distribution with a standard deviation of 0.2. This means most interventions have a relatively small signal for harm or benefit, while a few will have a much larger effect size that is observable even with small samples. (2) Patients are recruited in accordance with the observed number of admissions to ICUs within the ICNARC network.

We urgently need to adopt the paradigm of the drug trial to the broader questions of health services, and see specific non-drug interventions robustly evaluated. The rapid generation of scientific evidence to inform policy and treatment needs to be part of pandemic preparedness in the same way that we should stockpile personal protective equipment, and build depth to our testing infrastructure. We could and should go further and deploy ‘pandemic data officers’ to the frontline to assist with data collection, minimise logistical burden, and accelerate learning. We could cluster randomise hospitals to strategies that preserve surgical pathways for cancer to a greater or lesser extent.

These are pared down solutions for extraordinary times, but could still be implemented for this winter with only modest resource. The pandemic is not over, and so the cost of unanswered questions escalates. The sooner we optimise care pathways for COVID-19 patients, the sooner we can redeploy resources to essential non-COVID-19 care. To restore the health service for all, we must immediately learn from the successes of clinical drug trials. We must not be content with retrospective policy review. We must not continue to create policy on the basis of phone calls to friends.

Authors' contributions

Jointly contributed to the idea and the writing of the commentary: all authors

Declarations of interest

The authors declare that they have no conflicts of interest.