Desperate times call for evidence-based measures: Prioritizing science during the COVID-19 pandemic

The COVID-19 pandemic represents one of the largest acute global health threats in a century, and scientific and public interest in the disease is substantial. Clinicians, infection control practitioners, epidemiologists, policy-makers, and concerned citizens worldwide are looking to medical journals, preprint servers, and social media for updates on the prevention and treatment of this disease.

As scientists and clinicians scramble to understand this new infection, there has been a deluge of scientific publications about the epidemiology, pathophysiology, diagnosis, and treatment of COVID-19. There have been some remarkable milestones in phase 3 clinical trials going through design, ethics approval, enrolment, analysis, and publication within the past 6 months. The first is the randomized controlled trial by Cao et al on the use of lopinavir-ritonavir for severe COVID-19 (1). The trial began enrolment on January 18, 2020, only weeks after the discovery of SARS-CoV-2, and was published only 2 months after enrolment. The same group successfully completed a 2:1 randomized controlled trial on remdesivir versus placebo, and although recruitment was hindered by the end of the local outbreak, it still contributed useful findings (2). The first robust randomized controlled trial to be published on COVID-19 involved the recruitment of over 1,000 individuals from 10 countries to receive remdesivir or placebo, a remarkable achievement in the context of a pandemic with a short time frame (3). These trials have been paramount in informing practice and generating policy while awaiting larger definitive trials. Trials such as RECOVERY in the United Kingdom, have begun to release results, including the finding of significant mortality benefit with dexamethasone among inpatients requiring oxygen or mechanical ventilation (4).

Despite the high-quality evidence being published to date, there has been a proliferation and publication of studies that have been scientifically inadequate. These studies have had outsized effects by leading to mass confusion and uneven policy development. Shortly after a French group published an uncontrolled study that suffered from major methodologic flaws (5) on the effectiveness of hydroxychloroquine and azithromycin, President Donald Trump touted hydroxychloroquine as a potential “game-changer,” the Food and Drug Administration authorized emergency use, and widespread off-label use (6) caused global supply chain shortages, thus exposing individuals to risk and simultaneously threatening the health of people who take these medications for proven indications such as systemic lupus erythematosus. Without evidence of efficacy, the Indian Council of Medical Research recommended pre-exposure prophylactic hydroxychloroquine to the scores of health care workers in that country who may provide care for someone with COVID-19 (7).

Even high impact medical journals have included studies that do not meet the most basic standards of scientific publishing. The Lancet recently published a large observational study of over 10,000 individuals taking hydroxychloroquine or chloroquine that showed no significant benefit, with an increase in mortality seen in this group compared to over 80,000 patients who did not receive these drugs (8). The downstream effects of this study included a hold on the hydroxychloroquine arm of the World Health Organization Solidarity Trial, as well as repeals on the use of the drug in France. However, as scientists took a closer look at this publication, it became evident that there were significant concerns about the validity and veracity of the data (9). In fact, close attention was also turned toward a study using the same registry that had been published in the New England Journal of Medicine three weeks earlier. It soon became evident that the data could not be verified, and both articles were retracted (10,11). Annals of Internal Medicine published an experiment in which four COVID-19 patients coughed into a petri dish with and without cotton and surgical masks; the study reported that masks did not effectively reduce SARS-CoV-2 emission (12). However, the authors failed to appreciate that the quantities in all cases were below the assay’s limit of detection, and thus the results were uninterpretable. The study has since been retracted (13). These articles, despite their low quality of evidence and lack of context to the findings, lead to significant questions surrounding the transmission dynamics, pathophysiology, and management of COVID-19.

Rewinding to a century ago, syphilis was a significant cause of morbidity and mortality across the Old and New worlds. The emergence of treatment strategies in syphilis, which were uncontrolled and extremely toxic, holds a unique position in medical history. A study published in JAMA in 1903 noted with regard to mercury-based therapy that

This knowledge, though purely empirical, has been so clearly and conclusively established, by centuries of observation and study, that it has become one of the most evident and acceptable of medical facts [ … ] (14 p1626)

Further research on arsenic-based therapy and therapeutic hyperthermia—achieved by infecting patients with malaria—also became medical standards and even worthy of the Nobel Prize. These therapies were offered to patients of all ages and degrees of infection based on a collection of anecdotes and uncontrolled studies.

Today, we look back on these studies with a sense of incredulity, as the advent and maturation of evidenced-based medicine have reframed the type and quality of studies that should be accepted for changing clinical practice. Yet, over the course of this pandemic, the evidence base upon which recommendations for unproven treatments are predicated is reminiscent of the standards of a century ago. Why are we repeating the mistakes of a century ago? Dealing with a threat with high stakes and no proven treatment is akin to being thrust back into the pre-antibiotic era, where desperation reigns. Long after our medical predecessors resorted to heavy metals or iatrogenic malaria for treating syphilis, we are now disregarding the hard-won principles of evidence-based medicine—at our peril. It is imperative that clinical decisions and public health policy remain grounded in the fundamental hierarchy of scientific evidence with the prioritization of well-designed studies, including appropriate controls.

What is the way forward? With an emerging disease, there may be a rush to treat with unproven therapies for the sake of offering patients something rather than just providing supportive care. In some settings, where a treatment is very obviously needed to change morbidity and mortality (such as the use of antimicrobials for bacterial sepsis), it would be unethical to complete a placebo randomized controlled trial. In the case of COVID-19, there is clearly clinical equipoise in a number of treatment modalities. The mandate of research institutions and academic centres should be to encourage the creation and/or synthesis of the best possible evidence. In the context of COVID-19, this should mean prioritization of generating high-quality randomized controlled evidence wherever possible. Clinicians should provide excellent supportive care rather than prescribing experimental therapies (with unknown benefits and potential harms) outside of clinical trials.

Funding:

No funding was received for this work.

Disclosures:

The authors have nothing to disclose.