COVID-19 outbreak: the gold rush and the responsibilities of the scientific community

The coronavirus disease 2019 (COVID-19) pandemic is one of the most severe that the world has had to face in this century following the severe acute respiratory syndrome in 2000–2004, the flu pandemic in 2009, the Middle-East respiratory syndrome coronavirus in 2012 and the Ebola virus disease in 2013–2016 (although this last was primarily concentrated in Guinea, Liberia and Sierra Leone). With 5 304 772 confirmed cases [1], major efforts from national governments and scientific society aim to set in place actions to contain and limit further spread, and to improve global knowledge on the disease pathogenesis and valid therapeutic options.

We are witnessing a massive rise in the published literature on COVID-19. If we perform a simple search on PubMed using the terms ‘2019 novel coronavirus disease OR 2019 novel coronavirus infection OR 2019-nCoV disease OR 2019-nCoV infection OR coronavirus disease 2019 OR coronavirus disease-19 OR COVID’, filtered for humans, we will retrieve 3838 articles. Considering the timeline of the diffusion of the infection and the date of the first official report [2], this number represents a median of 767 articles per month. Clearly an unmanageable number of papers.

At this point, an obvious question comes to mind: how much can we rely on this literature? Is it really the product of evidence-based medicine? The first impression is that this pandemic has led to a sort of ‘gold rush’ where the gold is the publication at the expense of the quality of the paper contents. The final result of this hectic publication rhythm is to increase uncertainties rather than decreasing them as on any given day there may be ‘evidence’ for a treatment that is likely to be contradicted the day after.

For instance, in the maze of flourishing modelling studies, case-fatality risk estimates show significant variabilities and rapid changes [3]. This is not surprising considering that case-fatality risk is based on numbers that are constantly evolving (e.g. number of reported cases, number of deaths, the risk period) and on variables that are not always available or measurable (e.g. number of undetected cases, patient demographics and characteristics, access to health-care system, implementation of infection control strategies, testing policies and selection bias) [4]. An example is the report on case-fatality risk in Italy, which highlights how rates depend on the prevalence of co-morbidities and the average age of the population (which independently contribute to the risk of death); variability in testing strategies (which affects the denominator); and definition of COVID-19-related deaths [5]. However, these projections are used indiscriminately by the media to inform the public without appropriate filters, so fostering a feeling of unreliability, and to guide policy-makers for designing public health interventions in response to the emergency, with the risk of an inappropriate use of resources.

Under the pressure of this emergency, the list of clinical trials has exploded, with 1717 studies registered on ClinicalTrials.gov. Most of the interventional trials are seeking to evaluate the efficacy of various treatment regimens/combinations, from antiviral agents (e.g. remdesivir, favipiravir, lopinavir/ritonavir, darunavir) to different immunomodulatory drugs (e.g. hydroxychloroquine, tocilizumab, sarilumab, baricitinib, ruxolitinib, anakinra, siltuzimab) or hyperimmune plasma. Nevertheless, we should question whether such a number of studies is really essential. Having too many studies with partially overlapping options needs to be carefully weighted as it not risks only undermining trust in the coherence of trials development and registration but also makes clinical decisions more difficult, lastly penalizing patient care.

The Infectious Disease Society of America has published guidelines on the treatment and management of patients with COVID-19 based on the need to perform a first critical appraisal in support of the proposed management strategies [6]. As a result, the seven identified recommendations (about hydroxychloroquine/chloroquine; hydroxychloroquine/chloroquine plus azithromycin; lopinavir/ritonavir; tocilizumab; corticosteroids in COVID-19 pneumonia and acute respiratory distress syndrome, respectively; and COVID-19 convalescent plasma) rely on studies at high risk of bias, with very low certainty and the use of the above mentioned treatments should be limited to the context of clinical trials. Similar conclusions are drawn in another recent review on pharmacological treatments for COVID-19 [7]. All of this is understandable given the emergency of the current situation and the urgent need to share data that are published without following the usual peer-review track. Ultimately, the role of the guidelines is to provide a summary of the best available evidence to support clinicians and make them aware of the use of drugs for which the harm to benefit ratio is doubtful. However, by maintaining this trend, the questionable quality of the studies will jeopardize the accuracy of future results and here we would go back to the original question—how can we trust this literature? Paraphrasing Richard Feynman, if we allow ourselves to live with answers that might be wrong, as we progress, we remain unsure and we will leave opportunities for alternatives.

We need accurate evidence before exposing, unethically, individuals to unproven therapies where benefits might exceed potential harms. Evidence-based medicine relies on a combination of well-designed randomized controlled trials, observational studies and clinical expertise without ever forgetting the real research end point: what does the best for the patient? It has already been underlined how randomized controlled trials represent the study design that allows the enrolment of a representative sample of the population, provides more reliable information on drug efficacy and, importantly, ensures prompt identification of adverse events [8]. Assurance of the integrity of trials development and drugs approval is the prerogative of regulatory agencies, which should be the first to grant that tested treatments are effectively demanded to fill a real knowledge gap of clinical and public health importance. In that respect, it might be advisable, especially in time of crisis, to raise the bar for registration of clinical studies by clearly specifying a minimum of methodological prerequisites to be accomplished (e.g. consideration of the added value of the information that will be produced to previous or contemporary evidence, definition of relevant outcome measures and effect-size, transparent presentation of the results) and the setting up of rigorous criteria to assess the appropriateness of the interventions [9]. It might be argued that this would disadvantage investigators and restrict the possibility of conducting trials but, conversely, it would implement research quality, multicentre collaboration and sound competition.

The speed of trials development is undoubtedly a positive response in this time of urgency, but the volume of studies is no longer enough. This time might be an opportunity to promote research as a means towards better rather than quantitative knowledge, which is essential to preserve the integrity of the scientific community to the public and the trust in national and global health responses [10].

Transparency declaration

The author declares no conflict of interests. No external funding was received.

Editor: L. Leibovici