Poor quality research and clinical practice during COVID-19

The COVID-19 pandemic has turned our lives upside down. Health services have adapted to the challenges posed by the pandemic at eye-watering speed. Telemedicine has seen a rapid uptake in order for patient–physician encounters to comply with social distancing regulations. Elective surgeries have been put on hold to make room in hospitals for patients with COVID-19 and save valuable personal protective equipment. Many pre-pandemic research projects have been put on hold, and legions of medical researchers are now dedicated to researching COVID-19.

Short abstract

Breathe Chief Editor @ClaudiaCDobler on how #COVID19 amplifies flaws in clinical research and practice https://bit.ly/3cX0jpO

The COVID-19 pandemic has turned our lives upside down. Health services have adapted to the challenges posed by the pandemic at eye-watering speed. Telemedicine has seen a rapid uptake in order for patient–physician encounters to comply with social distancing regulations. Elective surgeries have been put on hold to make room in hospitals for patients with COVID-19 and save valuable personal protective equipment. Many pre-pandemic research projects have been put on hold, and legions of medical researchers are now dedicated to researching COVID-19.

The number of publications on COVID-19, many published on pre-print servers that allow sharing of a research publicly before it has been peer reviewed, is skyrocketing. Many journals haveseen a drastic increase in manuscript submissions. The demand for quick distribution of information on a disease that we still know very little about is understandable and justified, and rapid dissemination of knowledge that can be accessed freely without a paywall is desirable. COVID-19, however, also highlights and amplifies some of the challenges clinical research and practice are facing in general.

Studies published on pre-print servers before peer review allow unrestricted and quick access to research results. There is, however, a significant danger that flawed results are picked up and disseminated by the media [1]. Studies with positive results, indicating effectiveness of an intervention, are likely to receive more attention than studies with negative results, even if the latter are of higher quality.

There are numerous trials on different drug interventions competing for the same pool of COVID-19 patients, many with poor study design (e.g. small sample size, no comparator group, not randomised, single centre and no study protocol publicly available) [1]. Trying to draw conclusions about the effectiveness of interventions based on (biased) nonrandomised studies can be grossly misleading. In the rush to make the latest science available, editorial scrutiny may fall short of usual benchmarks, as evidenced by the publication of large numbers of poorly designed studies, but also poor editing including increases in clunky, awkward or otherwise poor phrasing, grammatical errors, and mislabelling of tables and figures.

Some COVID-19 investigators have never conducted research in patients with respiratory infections before and are not embedded in research networks with the required expertise to deliver high-quality trials. This will likely lead to wasteful, poor-quality research. National research funding bodies in most countries have missed the opportunity to streamline research efforts and focus on a small number of large, high-quality trials supported by extensive clinical research networks rather than a large number of small (and likely underpowered) studies. During any pandemic, evidence is rapidly developing and treatments might no longer be of interest by the time a trial starts recruiting patients. Bayesian adaptive trial designs are therefore particularly suited to these circumstances. They will allow us to assess the most promising treatments based on the state of knowledge at the time and will increase the probability of patients being randomised to the best-performing treatments [2].

There is a substantial imbalance in trial topics with only a small proportion of COVID-19 trials registered on ClinicalTrials.gov focusing on non-drug interventions [1]. This pertains to important preventive non-drug interventions, such as hand hygiene and wearing of masks, but also non-drug interventions to treat patients with COVID-19, such as noninvasive positive pressure ventilation (NIPPV). The use of NIPPV is discouraged in some settings in favour of early intubation to minimise viral aerosolisation (and therefore the risk of viral transmission to healthcare workers). It is, however, unknown whether the use of NIPPV is associated with increased viral aerosolisation and transmission compared with oxygen therapy delivered via nasal prongs, Hudson mask or high-flow nasal cannula. Patients who overcome COVID-19 with NIPPV as opposed to intubation will likely make a quicker recovery, but it is unknown whether delayed intubation in patients with COVID-19 in whom NIPPV fails is associated with a higher risk of complications. On 17 May 2020, 1528 studies were registered on ClinicalTrials.gov using the term “COVID”, of which only three were randomised trials to assess NIPPV, despite the urgent need for evidence to inform clinical practice in this area.

Physicians are frequently using experimental therapies in critically ill patients with COVID-19 in the absence of proven effective treatments. They may, however, significantly underestimate potential harms of these drugs, for example, the potential cardiotoxicity of hydroxychloroquine, ritonavir, lopinavir, interferon-α2β, azithromycin and methylprednisolone [3]. Clinicians must resist the impulse “to do something” and only use unproven treatments within clinical (ideally adaptive) trials [3]. The tendency to prefer action over inaction is also known as commission bias and is motivated by avoiding regret about a missed opportunity when a treatment is not given, even if its efficacy is unclear [4]. Commission bias is a potential barrier to the implementation of evidence-based practice, and drives overtreatment and “low-value care” (care that provides little or no benefit and may cause harm to patients) [4]. As clinicians, we must reflect on the biases that may influence our clinical decision-making, critically appraise the evidence of COVID-19 treatments, and act with caution and reason [5].

There might be exceptions to the rule, especially for non-drug interventions, where the use of the “precautionary principle” is sensible. This principle is a strategy to “adopt precautionary measures when scientific evidence about an environmental or human health hazard is uncertain and the stakes are high” [6]. It has been proposed that this principle should be applied to the wearing of face masks in public, because COVID-19 is a serious threat and the potential benefits of wearing a mask likely outweigh potential downsides, including a false sense of security and reduced compliance with other infection control measures [7]. In a similar way, the precautionary principle should be applied to hand hygiene to reduce the spread of coronavirus.

It is likely that COVID-19 will pose a challenge to our healthcare systems for the foreseeable future, and we will have to find ways to treat patients with COVID-19 while not neglecting patients with other diseases. With this in mind, I hope that the June issue of Breathe on the topic of “Rare and orphan lung diseases” will be informative and inspirational.