COVID-19 research in critical care: the good, the bad, and the ugly

The extraordinary pace of research on coronavirus disease 2019 (COVID-19) has been one of the major success stories of the pandemic. Therapeutic trials involving thousands of patients, which usually take years to complete, have been reported in a matter of months. National and international registries and networks have reported on tens of thousands of patients in near real time. However, there have also been many challenges: hundreds of trials have been underpowered, duplicated, or of poor quality; excessive bureaucracy has complicated study initiation; and only a small percentage of eligible patients worldwide have been enrolled in studies, while many others have been treated with off-label, unproven therapies. All of this has been complicated by an “infodemic” of low-quality medical information, accelerated by social media. The goal of the present article is to discuss the challenges, achievements, and future directions of critical care research during the pandemic (Table 1).

Table 1.

Demographics and baseline characteristics (open-label safety analysis set)

What went wrong?	Solutions
Small observational studies with highly variable results	Creation of national or international registries with harmonized data collection for emerging disease outbreaks that are operational both during and outside pandemic periods
Absence of observational data from outside traditional academic or research centres	Creation of national or international registries that collect data from all hospitals within a given jurisdiction
Few observational studies from low- and middle-income countries	Implementation of national electronic medical record systems in low and middle-income countries to allow for easy (or automated) data collection. National audits or registries for critical illness
Too many small clinical trials with inconclusive results	Avoidance of single centre and single region therapeutic trials Creation of national or regional ethics review boards Coordinated data collection between clinical trial networks Prioritization of large scale multicentre studies as well as collaborations between investigators with overlapping interests Medical societies to encourage clinical trial participation in place of off-label therapy use
Too many overlapping/competing clinical trials	Collaboration across clinical trial networks and funding agencies to ensure fewer, large-scale studies are funded or that small studies work together to generate larger datasets Prioritization of adaptive platform trials that enable parallel testing of multiple therapeutic options
Bureaucratic delays in setting up clinical trials	Fast-track approvals for pandemic-related clinical trials Creation of national or regional ethics review boards
Few therapeutic trials in low- and middle-income countries	Inclusion of low and middle-income countries in international multi-centre clinical trials Creation of clinical trial networks that include both high- and low-income countries Simplification of data collection forms Simplification of trial protocols Simplification of regulatory requirements for trials using already approved medications

Early studies of COVID-19 patients, whether observational or interventional, involved small numbers of subjects and incomplete outcome data. This lack of scale, along with abbreviated follow-up, produced poor quality, inconsistent data. Early intensive care unit (ICU) mortality estimates, for example, ranged from 0 to 85% [1].

Large-scale studies of critically ill COVID-19 patients, based on national or international registries, have produced better quality data, allowing greater understanding of the influence of patient-level and system-based factors on the incidence and outcome of severe COVID-19 [2, 3]. Unfortunately, many countries lack ICU registries and many existing registries do not include key data points that are relevant to emerging infectious diseases.

The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) was created in 2012 to overcome barriers to the study of emerging infectious diseases through the creation of protocols and case report forms (CRFs) that could be rapidly adapted to new outbreaks [4]. To date, ISARIC has reported data on over 200,000 hospitalized COVID-19 patients in more than 40 countries [5]. Differences in datasets across studies remain a challenge, however, and integrating a minimal dataset into national registries would facilitate demographic studies and between-country comparisons. The use of registry-embedded clinical trials is a strategy that facilitates trial enrollment and management, thus increasing study participation.

Agility is a key element for successful COVID-19 therapeutic studies. The use of platform trials has facilitated testing of multiple treatments in parallel and increasing the efficiency of the search for effective interventions [6], while adaptive designs have allowed for the introduction of new therapies into ongoing studies along with the removal of unsuccessful therapies after meeting pre-specified stopping rules. These strategies have enabled trials such as the Randomised Evaluation of COVID-19 Therapy (RECOVERY) [7], the Randomized, Embedded, Multifactorial Adaptive Platform for Community-acquired Pneumonia (REMAP-CAP) [8], and the World Health Organization SOLIDARITY trial [9] to generate high-quality data on multiple therapeutic options within the first year of the pandemic. These studies are operational in several countries including LMICs. Platform trials are complex and require careful planning and statistical oversight. However, the benefits in generating high-quality data in a low-cost and efficient way during a pandemic are clear [10].

COVID-19 studies have embraced pragmatic designs that reduce the burden of participation. These include open-label medications, shortened CRFs and/or automated data collection, and a focus on therapies with pre-existing regulatory approval, reducing the level of regulatory oversight. This pragmatic approach has enabled hospitals without extensive research experience to participate, thereby maximizing patient recruitment.

In spite of these efforts, COVID-19 study involvement, particularly in low- and middle-income countries, remains poor. The COALIZÃO trials in Brazil [11, 12] have circumvented some of these challenges using a pre-existing research network and pragmatic study design. However, there remains a significant discrepancy between trial participation in high-income and low-income countries. This is not only a question of research equity and generalizability of results, but also of safety, as a lack of clinical trial access may increase the likelihood of off-label medication use [13, 14].

Leadership that prioritizes and supports high-quality clinical research has been a key ingredient in successful pandemic research, and it has a role in encouraging the population to participate in trials. The RECOVERY trial in the UK, supported by the National Health Service (NHS), enrolled over 10,000 patients at 176 hospitals in 3 months [7]. Similarly, the WHO has promoted participation in clinical trials, such as its SOLIDARITY Trial, which is now recruiting in over 500 hospitals worldwide [15].

Leadership is also required in setting research priorities. Many overlapping (and often underpowered) trials have taken place worldwide. To avoid wasted time and effort, funding agencies and scientific societies should prioritize funding of large-scale trials that are likely to produce definitive results and encourage collaboration between investigators with similar interests.

Agile randomised controlled trials (RCTs) are only possible if a fast, albeit rigorous, process of ethics approval is in place. National or regional Research Ethics Boards (REBs) can greatly simplify this process by allowing a single centralized approval for multiple sites. In the UK, for example, a single REB approval is valid for the entire country. Regulatory issues are particularly challenging for international studies.

In the current pandemic, we have been fighting not only a disease but an infodemic, which has been greatly amplified by social media [16]. Additionally, data have been rushed into the public domain through the publication of preprints and press releases [16]. In the absence of complete and verified data, medical professionals are left uncertain as to how to care for their patients. In July 2020, for example, Gilead issued a press release reporting a 62% mortality benefit for remdesivir based on an unpublished retrospective comparison, an astonishing result which was not confirmed by subsequent RCTs [9].

Although peer-review slows the pace of publishing, it is a fundamental step to the production of high-quality data. Peer-review does not eliminate poor-quality research, as shown by the publication of uncontrolled studies of hydroxychloroquine and azithromycin, amongst others. Nor does it eliminate potential fraud, as shown by the publication (and retraction) of the Surgisphere dataset. However, medical journals have a vested interest in publishing accurate data and, when an article is called into question, investigating and retracting if necessary. The scientific and medical communities must lead by example and wait for the publication of high-quality peer-reviewed data prior to making decisions about potential therapies [17].

During COVID-19 pandemic, agility, pre-existing infrastructure, platform trials, adaptive design, pragmatic design, and centralized REB approvals contributed to successful research outcomes. Worldwide, however, only a small percentage of hospitals and patients are participating in COVID-19 research and there remains enormous untapped potential, particularly in low- and middle-income countries. Clinicians and health systems must adopt the mentality that efficient and systematic research participation is an integral part of pandemic response, and that every patient not given the opportunity to participate in a study represents a valuable opportunity lost [18].

Compliance with ethical standard

Conflicts of interest

The authors have no competing interests.