Use of pragmatic and explanatory trial designs in acute care research: lessons from COVID-19

Jonathan D Casey; Laura M Beskow; Jeremy Brown; Samuel M Brown; Étienne Gayat; Michelle Ng Gong; Michael O Harhay; Samir Jaber; Jacob C Jentzer; Pierre-François Laterre; John C Marshall; Michael A Matthay; Todd W Rice; Yves Rosenberg; Alison E Turnbull; Lorraine B Ware; Wesley H Self; Alexandre Mebazaa; Sean P Collins

doi:10.1016/S2213-2600(22)00044-3

. 2022 Jun 13;10(7):700–714. doi: 10.1016/S2213-2600(22)00044-3

Use of pragmatic and explanatory trial designs in acute care research: lessons from COVID-19

Jonathan D Casey ^a,^*, Laura M Beskow ^b, Jeremy Brown ^e, Samuel M Brown ^f, Étienne Gayat ^g,^h, Michelle Ng Gong ⁱ, Michael O Harhay ^j, Samir Jaber ^k, Jacob C Jentzer ^l, Pierre-François Laterre ^m, John C Marshall ⁿ, Michael A Matthay ^o, Todd W Rice ^a, Yves Rosenberg ^p, Alison E Turnbull ^q, Lorraine B Ware ^a, Wesley H Self ^c,^d,^†, Alexandre Mebazaa ^g,^h,^†, Sean P Collins ^c,^r,^†

^aDivision of Allergy, Pulmonary, and Critical Care Medicine, Vanderbilt University Medical Center, Nashville, TN, USA

^bVanderbilt Center for Biomedical Ethics and Society, Vanderbilt University Medical Center, Nashville, TN, USA

^cDepartment of Emergency Medicine, Vanderbilt University Medical Center, Nashville, TN, USA

^dVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN, USA

^eOffice of Emergency Care Research, National Institute of Neurological Disorders and Stroke, Division of Clinical Research, National Institutes of Health, Bethesda, MD, USA

^fDivision of Pulmonary and Critical Care Medicine, Intermountain Medical Center and University of Utah, Salt Lake City, UT, USA

^gDepartment of Anesthesia, Burn and Critical Care, University Hospitals Saint-Louis-Lariboisière, AP-HP, Paris, France

^hINSERM UMR-S 942, MASCOT, Université Paris Cité, Paris, France

ⁱDivision of Critical Care Medicine and Division of Pulmonary Medicine, Department of Medicine, Montefiore Medical Center, Albert Einstein College of Medicine, New York, NY, USA

^jPalliative and Advanced Illness Research (PAIR) Center Clinical Trials Methods and Outcomes Lab, and Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA

^kSaint Eloi Intensive Care Unit, Montpellier University Hospital, and PhyMedExp, INSERM, CNRS, Université de Montpellier, Montpellier, France

^lDepartment of Cardiovascular Medicine, Mayo Clinic, Rochester, MN, USA

^mDepartment of Intensive Care, Cliniques St-Luc, Université catholique de Louvain, Brussels, Belgium

ⁿLi Ka Shing Knowledge Institute, St Michael's Hospital, Unity Health, Toronto, ON, Canada

^oCardiovascular Research Institute, University of California, San Francisco, CA, USA

^pNational Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, MD, USA

^qDivision of Pulmonary and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, USA

^rGeriatric Research, Education,and Clinical Center, Tennessee Valley Healthcare System, Nashville, TN, USA

Correspondence to: Dr Jonathan Casey, Department of Allergy, Pulmonary, and Critical Care, Vanderbilt University Medical Center, Nashville, TN 37232, USA

^†

Contributed equally

PMCID: PMC9191864 PMID: 35709825

Abstract

Unique challenges arise when conducting trials to evaluate therapies already in common clinical use, including difficulty enrolling patients owing to widespread open-label use of trial therapies and the need for large sample sizes to detect small but clinically meaningful treatment effects. Despite numerous successes in trials evaluating novel interventions such as vaccines, traditional explanatory trials have struggled to provide definitive answers to time-sensitive questions for acutely ill patients with COVID-19. Pragmatic trials, which can increase efficiency by allowing some or all trial procedures to be embedded into clinical care, are increasingly proposed as a means to evaluate therapies that are in common clinical use. In this Personal View, we use two concurrently conducted COVID-19 trials of hydroxychloroquine (the US ORCHID trial and the UK RECOVERY trial) to contrast the effects of explanatory and pragmatic trial designs on trial conduct, trial results, and the care of patients managed outside of clinical trials. In view of the potential advantages and disadvantages of explanatory and pragmatic trial designs, we make recommendations for their optimal use in the evaluation of therapies in the acute care setting.

Introduction

The COVID-19 pandemic has affected nearly every aspect of medicine and society. Globally, it has caused considerable personal and economic hardship, but recent progress with vaccines to prevent infections and severe disease1, 2, 3 and effective treatments for early4, 5, 6, 7 and late-stage8, 9, 10 COVID-19 offer hope for a post-pandemic life. Moreover, there is optimism that the unprecedented investment in clinical research during the pandemic has created new opportunities in other areas of medicine—eg, newly validated technologies such as mRNA vaccines are already being evaluated as treatments for HIV,¹¹ chikungunya,¹² and various malignancies.¹³

A similar but less recognised evolution has occurred during the pandemic in the design, execution, and analysis of clinical trials comparing therapies already in common clinical use, leading to stark contrasts between the approaches used by different countries to evaluate and use repurposed therapies (corticosteroids, hydroxychloroquine, convalescent plasma, anticoagulation strategies, and lopinavir–ritonavir) as treatments for patients admitted to hospital for COVID-19. In many countries, including the USA, these therapies were used clinically before meaningful results were available, and small explanatory trials that focused on intermediate patient-centred outcomes were conducted in the setting of ubiquitous clinical use of these therapies. In the UK, off-label use of medications for COVID-19 was discouraged in favour of large pragmatic trials that focused on mortality. The UK studies embedded trial procedures into clinical care and allowed modifications of traditional consent procedures to facilitate rapid enrolment. Although much attention has been given to the way in which trials, such as those conducted in the UK, used a single infrastructure to answer multiple questions (adaptive platform design) and leveraged modern methods of statistical analysis to maximise the likelihood of providing definitive evidence for each intervention (Bayesian sequential analysis),¹⁴ little attention has been given to how the approaches to enrolment, consent, and intervention delivery for these pragmatic trials differed from methods used in traditional, explanatory trials.

This Personal View represents a distillation of views expressed at the 2020 Critical Care Clinical Trialists (3CT) Workshop (Washington, DC, USA; Feb 27–29, 2020) and during subsequent discussions between the authors in view of the research response to the COVID-19 pandemic. The annual 3CT Workshop brings together a diverse group of international clinical trialists, epidemiologists, bioethicists, regulators, personnel from federal funding agencies, industry representatives, clinicians, and patient advocates to identify problems with and potential solutions to the design, conduct, and interpretation of clinical trials in critically ill adults. In this paper, we review the barriers to conducting acute care clinical trials during a pandemic and—using the US ORCHID (Outcomes Related to COVID-19 Treated With Hydroxychloroquine Among Inpatients With Symptomatic Disease) and UK RECOVERY (Randomised Evaluation of COVID-19 Therapy) trials as examples—we discuss the impact of explanatory and pragmatic trial methods on trial conduct, trial results, and the care of patients managed outside of clinical trials. We consider unresolved controversies surrounding pragmatic trials, including blinding, informed consent, and adverse event monitoring, and discuss the types of questions for acute care research that are well suited to pragmatic trials and those that are not. Finally, we propose changes to facilitate pragmatic acute care trials and to match the intensity of oversight and regulation to the risks of the research being conducted.

Key messages.

•
Traditional explanatory trials are ideally suited to drug discovery research and can be used to evaluate novel therapies on a timescale that is relevant during a pandemic, but are less well suited to the evaluation of therapies already in common clinical use
•
Pragmatic trial methods allow rapid and efficient enrolment of a representative trial population, and are ideally suited to studies evaluating processes of care and therapies already in common clinical use; although pragmatic trials have potential to improve patient outcomes during a pandemic, they also raise ethical and regulatory questions regarding approaches to informed consent and safety monitoring
•
The approaches to use and evaluation of hydroxychloroquine in the USA (through the ORCHID trial) and the UK (through the RECOVERY trial) provide a unique opportunity to compare explanatory and pragmatic trial methods in terms of efficiency, risks to participants, and effects on patients receiving treatment in usual care during a pandemic
•
Facilitating broader adoption of pragmatic trial methods for comparisons of commonly used interventions would require: (1) regulators, funders, and health-care system leadership to discourage potentially harmful, arbitrary variation in practice in usual care in favour of structured variation through pragmatic comparative effectiveness trials; (2) regulatory guidance to evaluate the risk of a research study relative to the risk of routine clinical care; (3) development of novel methods to demonstrate respect for patients and families (eg, community consultation, pre-enrolment public disclosure, and post-enrolment notification) that could be used to allow pragmatic trials for low-risk interventions; (4) advances in information technology to facilitate screening, randomisation, intervention delivery, and outcome assessment in electronic health records; and (5) improved incentives for hospitals to enhance clinical outcomes through participation in pragmatic trials evaluating therapies already in common clinical use

Challenges for acute care research

Trials among critically ill patients pose distinct challenges.15, 16 Many interventions for critically ill patients are most effective when used during narrow time windows, limiting the ability of research teams to identify suitable candidates, obtain approval from clinical teams, obtain consent, and deliver the trial intervention. Furthermore, critically ill patients are a vulnerable population, frequently lacking the capacity to provide consent owing to their acute illness, and surrogate decision-makers might be unavailable or too distraught to discuss trial enrolment within the narrow time window of the studied intervention. Finally, an incomplete understanding of the biology of common syndromes in critical care could lead to trials including heterogeneous patient populations and disease states with differing responses to treatment, leading to overall null study results and, given the challenges of conducting acute care research, inadequate statistical power for subgroup analysis.17, 18

The 2020 3CT Workshop included discussions on the topics of trial design, selection of meaningful trial endpoints, the interface between septic shock and acute respiratory distress syndrome, and the challenge of balancing human research protections with meaningful trial design. The first death in the USA from COVID-19 occurred on the second day of the workshop, and critical care physicians based in France joined the workshop by video to discuss their experiences in caring for some of the earliest cases of COVID-19 in Europe. On the basis of these discussions, attendees sought to describe the barriers to conducting acute care clinical trials during a pandemic and to propose changes to improve human research protections to allow the rapid conduct of trials of available therapies.

Explanatory trials

Assessment of novel therapies

Modern human research protections were developed around the paradigm of explanatory trials, typified by drug development trials, which aim to determine whether a therapy demonstrates its hypothesised mechanism of action under idealised conditions. These trials are designed to maximise the likelihood of finding a clinical benefit (if there is one) while minimising risks (biological efficacy).

Explanatory trials determine whether a proposed therapy can work by using strict eligibility criteria to assemble a homogeneous population of patients who are expected to benefit from the therapy on the basis of the proposed mechanism of action. Such trials exclude patients who are receiving co-interventions that might interfere with the studied intervention, patients who are likely to have a clinical outcome for a reason other than the condition being evaluated (eg, a life-limiting comorbidity), and patients who are at increased risk of adverse events. Explanatory trials also commonly prioritise the collection of a broad and deep range of variables and biospecimens to explore the mechanisms underlying the intervention and interactions with co-interventions, evaluate unexpected adverse events, and build databases and biorepositories for secondary analyses and future hypothesis-generating research. Obtaining regulatory approval is often the overall goal of explanatory trial study design. To accomplish all of these aims, explanatory trials use highly trained research teams to identify and approach trial candidates, obtain informed consent, deliver trial interventions, monitor for adherence to trial procedures and adverse events, and record patient details in an extensive research database that is separate from the clinical data system. The need to collect such extensive data typically requires a substantial investment of effort, time, and capital.

Although the COVID-19 pandemic created a sense of urgency in finding ways to rapidly complete trials evaluating novel therapies and address challenges regarding patient communication and documentation of informed consent (as non-clinical staff were frequently barred from entering patients’ rooms or hospitals), the pandemic did not lessen the importance of human research protections. Furthermore, studies such as those evaluating remdesivir,⁷ anti-granulocyte macrophage colony-stimulating factor antibodies,¹⁹ and monoclonal antibodies²⁰ have shown that trials of novel therapies can be rapidly completed during a pandemic using traditional explanatory techniques by leveraging electronic²¹ and other methods of remote consent.

Assessment of clinically available therapies

The majority of decisions faced by clinicians treating acutely ill patients are not supported by high-quality evidence.22, 23, 24, 25 The expense and time required to conduct traditional, explanatory trials limit the number of trials that can be conducted, so decades-old questions in critical care, such as the best treatment for alcohol withdrawal, the best ventilator mode for acute respiratory distress syndrome, and the best vasopressor for septic shock, have remained unanswered despite the fact that millions of patients receive these therapies each year. Clinicians must, therefore, often choose between available treatments for a given condition in an individual patient in the absence of high-quality evidence, leading to considerable variability in treatments provided between clinicians, hospitals, specialists, or geographical regions. The lack of evidence systematically exposes patients to suboptimal, ineffective, or harmful therapies and represents a “profoundly serious moral problem” for current clinical practice.²⁶ Despite an unprecedented investment of resources in research focused on COVID-19, the inability to answer basic clinical questions has persisted during the pandemic. Many of the earliest clinical controversies regarding the management of critically ill patients with COVID-19, such as the optimal approach to non-invasive respiratory support and the timing of tracheal intubation, remain open to debate.

Relative to the challenge of designing a study to evaluate a novel drug with unknown safety and efficacy, comparing the effectiveness of clinically available therapies might seem straightforward. Traditional explanatory randomised trials, however, are poorly suited to the evaluation of available therapies for two reasons. First, the goals and design of explanatory trials do not match the goal of evaluating the effectiveness of available therapies—particularly when rapid evaluation is needed during a pandemic. Second, the regulatory intensity of patient protections in explanatory trials might be overly rigorous when applied to the evaluation of clinically available therapies, for which the risk to participants is nearly identical to that encountered as part of routine clinical care.

Explanatory trials typically use research personnel and data systems that are entirely separate from the personnel and data systems of clinical care. This approach is expensive and time-consuming, and most explanatory trials in the critical care setting are often unable to enrol more than a few patients per month. Although ideal for tightly controlled drug discovery trials, this inefficiency is problematic during a pandemic when clinicians have urgent questions about the safety and efficacy of therapies that are already being given to patients as part of routine clinical care. Furthermore, enrolment in explanatory trials of carefully selected populations of patients to determine whether a therapy can work under idealised conditions might result in a highly selected trial population that is not representative of the full range of patients treated in usual care. Clinicians might question whether trial results apply to patients who they treat in routine clinical care, particularly those intentionally excluded from trials. Finally, some studies of available therapies compare two active therapies (comparative effectiveness research)—eg, trials that compare different anticoagulation strategies among patients admitted to hospital with COVID-19.²⁷ In these trials, the differences between two active therapies might be smaller than the difference between a novel drug and placebo, but these small differences might be clinically important, especially when clinicians must choose one of the available options and the population that could receive the therapy is large. Detecting these small differences might require trials many times larger than can be feasibly completed on a timescale that is relevant for a pandemic using traditional explanatory trial methods.

Explanatory research methods are designed around the principles of protecting patients from a therapy with unknown safety and efficacy profiles. For drug discovery trials, in which the trial involves administering a therapy that the patient could not receive in routine care, trial participation presents a unique set of risks and benefits to patients. In this setting, conducting a thorough consent discussion accompanied by a witnessed signing and attestation of a written informed consent document is appropriate.²⁸ The risks to patients are usually lower in studies of approved medications or interventions with known benefits and risks. Furthermore, when the treatments are the same as those that the patient would receive in usual care, the risks of research participation might not be substantially different from the risks of routine clinical care. In this setting, completing a lengthy consent process seems inconsistent with the risk presented by a therapy that might be provided to patients in adjacent rooms with no discussion of risks or benefits.

Pragmatic trials

Increased generalisability and efficiency

Pragmatic trials were first envisioned 50 years ago to obtain real-world evidence to complement the evidence on efficacy produced by explanatory trials.²⁹ Whereas explanatory trials evaluate whether a therapy can improve outcomes, pragmatic trials evaluate whether a therapy does improve outcomes in actual practice (effectiveness). Every aspect of clinical trial conduct (eg, eligibility criteria, delivery of the trial intervention, or follow-up) exists on a spectrum from explanatory to pragmatic (table 1 ),³⁰ with the primary effects of pragmatic trial design features being increased generalisability and efficiency (figure 1 ).8, 31, 32 Some trials adopt a mix of pragmatic features (eg, broad eligibility criteria) and explanatory features (eg, frequent blood sampling for biomarker measurement).

Table 1.

Design and conduct of pragmatic versus explanatory trials for acute care research

	Explanatory trials	Pragmatic trials
Patient population	Inclusion criteria are tight, often with multiple exclusion criteria, intended to minimise the number of patients needed to detect a treatment effect, including any of the following approaches: • Creating a homogeneous trial population (to minimise statistical noise) • Enrolling patients who are likely to respond to the treatment (predictive enrichment) • Enrolling patients who are likely to experience the primary outcome (prognostic enrichment) • Excluding patients who are expected to have poor adherence to treatment or follow-up • Excluding patients with comorbidities that might lead to poor outcomes through mechanisms other than those targeted by the trial intervention (competing risks) • Excluding patients at high risk of adverse events on the basis of their age or comorbidities	Inclusion criteria are broad, with few exclusions; trial population tends to be larger and more similar to those who receive treatment as part of usual care
Recruitment and enrolment	Screening and enrolment are conducted by a research team, separate from treating clinicians	Screening (identification of trial candidates) and enrolment are conducted by treating clinicians (embedded in routine care)
Delivery of intervention	Intervention is delivered by the research team in a way that differs from delivery in usual care, including the following approaches: • Providing additional study-specific resources (eg, a study physician dedicated solely to ventilator titration for enrolled participants) • Providing substantial levels of trial-specific training on the delivery of the intervention beyond what would be available during future clinical use • Using experts to deliver the intervention (eg, using plastic surgeons in a trial that compares available techniques for repairing lacerations even though this is normally done by emergency medicine physicians in routine care) • Incorporating additional measures to improve adherence to treatment • Incorporating co-interventions into the protocol to minimise variability	Intervention is delivered in the way that it would be delivered as part of usual care outside of a trial (eg, by treating clinicians without any additional trial-specific resources or training)
Follow-up	Follow-up is more intense than would occur in usual care, and can include: • Additional research visits • Follow-up phone calls to increase treatment adherence or measure outcomes • Clinical visits triggered by study outcomes or adverse events in a way that might mitigate their severity	Treatment and follow-up are performed as they would be in usual care with minimal (if any) trial-specific follow-up
Primary outcome	Outcomes might not be relevant to patients (eg, surrogate outcomes, biomarkers, laboratory or radiographic outcomes) or might require testing that would not occur in usual care (adjudication of the primary outcome(s) by a blinded panel of experts, study-specific imaging, biopsies)	Patient-centred outcomes are routinely available from data collected as part of usual care (eg, mortality, intubation, hospital admission)

Open in a new tab

The considerations presented here are adapted from the PRagmatic Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool,³⁰ which can be used to aid design decisions consistent with the intended purpose of a trial. This validated tool has nine domains—eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis—scored from 1 (very explanatory) to 5 (very pragmatic) to facilitate design decisions.

Enrolment and intervention delivery in explanatory and pragmatic trials

This figure serves to highlight the differences between explanatory and pragmatic trials (green arrows) and how they impact routine clinical care (black arrows). (A) Explanatory trials. Many hospitals do not have research teams capable of conducting explanatory clinical trials (community hospital, top). When there is uncertainty regarding the relative effectiveness of two therapies, arbitrary variation develops as part of usual care in the community hospital setting. Some treating clinicians (purple) choose to use one therapy (therapy A) as part of usual care and others choose to use a different therapy (therapy B). Because this practice, which applies to all patients (grey), is not incorporated into a clinical trial, it does not generate generalisable knowledge. Conversely, in hospitals capable of conducting explanatory trials (research hospital, bottom), members of a dedicated research team (red) perform screening, enrolment, randomisation, intervention delivery, and outcome assessment among a carefully selected, homogeneous group of patients (blue) while excluding most of the patients receiving those therapies as part of usual care (green and orange). Explanatory trials are typically underpowered to detect differences in patient-centred outcomes such as mortality, and therefore focus on biomarkers or other surrogate outcomes. (B) Pragmatic trials. In contrast to the explanatory trial, pragmatic trials embed screening, enrolment, intervention delivery, and outcome assessment into routine clinical care. Using these efficient methods, pragmatic trials can enrol enough patients to detect differences in patient-centred outcomes (eg, mortality), and by enrolling all patients receiving the therapy as part of clinical care (blue, green, and orange patients at community and research hospitals), pragmatic trials provide generalisable results. While some pragmatic trials such as the RECOVERY trial also leverage a platform design (to cycle in and out proposed interventions using a shared trial infrastructure) or Bayesian sequential analysis (to ensure that enrolment is continued until trial evidence is definitive for effectiveness or futility), these aspects of trial design can be combined with both pragmatic and explanatory trials. Furthermore, the separate research teams employed in explanatory trials might be the ideal method of evaluating trials of novel therapies for which the safety profile is unknown.

The generalisability of pragmatic trial results stems from the enrolment of the range of patients who are likely to receive an intervention in routine clinical practice and the delivery of trial interventions in a manner that is similar to delivery in routine care. Pragmatic trials obtain representative patient populations by using broad inclusion criteria and minimising exclusions. For some interventions, the method of delivery might have an equally important effect on generalisability. For instance, if the trial intervention requires intensive study-specific resources (eg, research staff performing ventilator titration every 10 min) or expertise that is unlikely to be available during clinical care (eg, using plastic surgeons in a trial that compares available techniques for repairing lacerations even though emergency medicine clinicians most commonly repair lacerations), an explanatory trial might overestimate the benefits or underestimate the risks of the intervention when used in routine care. Pragmatic trials provide more generalisable treatment effect estimates because the trial intervention is delivered by the same clinicians who would administer the therapy in routine care. For interventions that do not require substantial resources or expertise (eg, administration of a pill or a one-time intravenous injection), the means of delivery (by research staff or treating clinicians) might be less important for generalisability.

Some features of pragmatic trial design increase both generalisability and trial efficiency. Embedding intervention delivery into routine clinical care might improve generalisability while also enabling research staff to focus on other aspects of trial conduct. However, other domains of pragmatic trial design (table 1) are focused almost entirely on efficiency. Streamlined data collection is a frequently referenced aspect of pragmatic trials. For a pragmatic trial, determining the effectiveness of trial therapies on a clinical outcome might be its sole goal. This is particularly true for pragmatic trials evaluating available therapies for which mechanisms of action and side-effect profiles are well known. Therefore, pragmatic trials might prioritise collection of the smallest number of variables needed to definitively answer the primary study question, and commonly include simple, patient-oriented outcomes such as mortality, which are readily available from clinical data. Using routinely collected clinical variables as trial outcomes creates the opportunity to conduct automated abstraction of data from electronic health records,33, 34 rather than relying on the resource-intense process of primary data collection solely for research. Large health systems and countries with unified electronic health records have unique opportunities to conduct trials using automated extraction of trial data, particularly when paired with comprehensive vital status registries.³⁵ Most research networks are limited by a lack of interoperability between electronic health record systems at various sites,³⁶ but some recent pragmatic trials have used research staff to deliver study interventions and have then collected data from pre-existing, prospective, observational patient registries that are already tracking long-term outcomes among the trial population.37, 38, 39 By prioritising efficiency, however, pragmatic trials might miss opportunities to understand the heterogeneity of treatment effect or mechanism of the intervention, particularly when collection of outcomes is limited to data from electronic health records or outcome registries.⁴⁰ Furthermore, collection of data outside of usual care, such as patient-reported outcomes and social determinants of health, might be difficult in a pragmatic trial.

Large pragmatic trials were uncommon in the acute care setting until recently (table 2 ).33, 38, 41, 42, 43, 44 Pre-pandemic examples of pragmatic trials in acute care research include trials using treating clinicians to enrol, randomise, and deliver study interventions for the investigation of emergency tracheal intubation,45, 46, 47, 48, 49 cluster-randomised trials using electronic health records to enrol patients and control the choice of balanced crystalloids or saline,33, 34, 50 a registry-based cluster-randomised trial comparing the method of stress ulcer prophylaxis among mechanically ventilated patients,³⁷ and a trial using prognostic models in electronic health records to identify patients with impending acute renal failure and to randomly notify half of clinicians.⁵¹

Table 2.

Examples of pre-pandemic explanatory and pragmatic acute care trials

	Study population and enrolment	Interventions and study type	Sample size	Duration	Outcomes	Findings
Explanatory acute care trials
Trial of 12 mL/kg vs 6 mL/kg tidal volume positive-pressure ventilation for treatment of acute lung injury and ARDS (ARMA)⁴¹	Highly selected population of patients (≥18 years) with ARDS recruited by a dedicated research team at multiple sites in the USA	Traditional ventilator management (an initial tidal volume of 12 mL/kg ideal bodyweight) versus ventilation with a lower tidal volume (6 mL/kg ideal bodyweight); protocol specified tight control of all ventilator management and co-interventions such as ventilator weaning; multicentre, randomised controlled trial	861 patients randomly assigned (1:1) to traditional ventilator management (n=429) or ventilation with a lower tidal volume (n=432)	3 years (287 patients per year)	Primary outcomes: death before discharge home and number of ventilator-free days from day 1 to day 28; additional outcomes included extensive physiological data and biomarkers	Ventilation with lower tidal volumes reduced mortality
Efficacy and safety of drotrecogin alfa (activated) in adult patients with septic shock (PROWESS-SHOCK) trial⁴²	Highly selected population of patients (≥18 years) with sepsis, shock, and clinical evidence of hypoperfusion recruited by a dedicated research team at multiple sites in several countries	Human activated protein C drotrecogin alfa (activated; 24 μg/kg per h for 96 h) versus placebo; multicentre, randomised, double-blind, placebo-controlled trial	1696 patients randomly assigned (1:1) to drotrecogin alfa (n=851) or placebo (n=845)	3·5 years (565 patients per year)	Primary outcome: mortality at 28 days; plasma protein C levels and SOFA score obtained daily for 7 days	Drotrecogin alfa (activated) did not significantly reduce mortality
Fluids And Catheters Treatment Trial (FACTT)⁴³	Highly selected population of patients (≥13 years) with ARDS recruited by a dedicated research team at multiple sites in North America	Fluid-management with lower (conservative use of fluids) versus higher (liberal use of fluids) intravascular pressure guided by a pulmonary artery catheter or a central venous catheter; protocol specified tight control of fluid management in both groups; multicentre, randomised trial with a two-by-two factorial design	1000 patients randomly assigned (1:1) to conservative fluid management (n=503) or liberal fluid management (n=497)	5·5 years (182 patients per year)	Primary outcome: death before discharge home within the first 60 days	Conservative use of fluids did not reduce mortality but was associated with more ventilator-free days; pulmonary artery catheter-guided management did not improve survival and was associated with more complications
Pragmatic acute care trials
Thrombus Aspiration in ST-Elevation myocardial infarction in Scandinavia (TASTE) trial³⁸	Patients (≥18 years) with STEMI at 31 centres enrolled within the existing Swedish Coronary Angiography and Angioplasty Registry with broad eligibility criteria; enrolment embedded into routine clinical care	Manual thrombus aspiration followed by PCI versus PCI only; intervention delivery embedded into routine clinical care; multicentre, open-label, randomised controlled trial	7244 patients randomly assigned (1:1) to manual thrombus aspiration and PCI (n=3621) or PCI only (n=3623)	3 years (2414 patients per year)	Primary outcome: all-cause 30-day mortality; all outcomes from a pre-existing registry	Thrombus aspiration before PCI reduced mortality among patients with STEMI
Corticosteroid Randomisation After Significant Head injury (CRASH) trial⁴⁴	Patients (judged to be ≥16 years) with head injury and coma enrolled within 8 hours of injury at 239 hospitals from 49 countries with broad eligibility criteria; enrolment embedded into routine clinical care	48-h infusion of methylprednisolone or placebo; treatment embedded into routine clinical care; multicentre, randomised, double-blind, placebo-controlled trial	10 008 patients randomly assigned (1:1) to high-dose corticosteroids (n=5007) or placebo (n=5001)	5 years (2002 patients per year)	Primary outcomes: death at 2 weeks and death or disability at 6 months	Corticosteroid use after head injury increased mortality
Isotonic Solutions and Major Adverse Renal events Trial (SMART)³³	All patients (≥18 years) admitted to one of five ICUs at an academic medical centre during the study period; enrolment, intervention delivery, and outcome assessment using electronic health records	Physiologically balanced isotonic crystalloids (lactated Ringer's solution or Plasma-Lyte A, according to treating clinician's preference) versus 0·9% saline; intervention delivery embedded into routine clinical care; open-label, cluster-randomised, multiple-crossover trial	15 802 patients randomly assigned (according to randomisation unit) to balanced crystalloids (n=7942) or saline (n=7860)	2 years (7901 patients per year)	Primary outcome: major adverse kidney event within 30 days (composite of death from any cause, new renal-replacement therapy, or persistent renal dysfunction)	Use of balanced crystalloids reduced the rate of death from any cause, new renal-replacement therapy, or persistent renal dysfunction

Open in a new tab

ARDS=acute respiratory distress syndrome. ICU=intensive care unit. PCI=percutaneous coronary intervention. SOFA=Sequential Organ Failure Assessment. STEMI=ST-segment elevation myocardial infarction.

Implications for human research protections

Although some aspects of pragmatic trial design could be used with traditional written informed consent, many recent pragmatic trials have used alterations or waiver of informed consent, prompting questions about the adequacy of human research protections in such trials. Although acute care research presents unique challenges to the protection of human participants, the principles for the ethical conduct of research are the same as in other types of human research and include:⁵² (1) informed consent; (2) respect for potential and enrolled participants (eg, confidentiality, welfare, and withdrawal); (3) social and scientific value; (4) scientific validity (including equipoise); (5) favourable risk–benefit ratio; (6) independent review by an institutional review board; and (7) fair participant selection. Informed consent receives the most attention and consumes the most time and resources of study staff.53, 54, 55 Informed consent requires patients or their proxies to understand the purpose, procedures, risks, benefits, and alternatives to the proposed research, and its bearing on their clinical situation before making a voluntary and uncoerced decision about whether to participate.⁵² Although the documentation of informed consent is often a point of focus, the presence of a signed consent form does not guarantee that an adequate informed consent process was completed.

Furthermore, informed consent alone is not sufficient to protect human participants, and is not required in all cases.33, 34, 37, 47, 50, 51, 52, 56 Consent practices vary by country, but most have mechanisms to waive or alter informed consent for research that involves no more than minimal risk to patients and research focusing on life-saving interventions for which consent is not feasible and available treatments are unsatisfactory (eg, if the patient is unconscious and the time window for intervention is too short to contact surrogates).⁵⁷ The process of waiving consent for emergency interventions might involve other methods to demonstrate respect for patient autonomy and welfare, such as community consultation (obtaining involvement from community stakeholders during trial design and conduct), public disclosure of a planned trial (general notification to community members that they might be enrolled in a trial if they become critically ill and consent is not feasible), or contacting the patient or surrogate at the earliest possible opportunity for delayed consent or notification of enrolment.58, 59, 60, 61

The ethics of alteration or waiver of consent for pragmatic trials have received considerable attention.26, 62, 63, 64, 65 Much of the debate focuses on how the risk of research is assessed in clinical trials. When trials evaluate novel therapies (eg, a new surgical technique) or therapies with substantial prior evidence of superiority, trial participation includes clear potential risks and benefits.⁶⁶ However, for two existing interventions already in common clinical use, when both are considered usual care and neither is known to be superior, some researchers and bioethicists have proposed that there is no difference between the risk of being randomly assigned to one of the two interventions and the risk of receiving one in usual care, in which the choice between the two therapies is made arbitrarily on the basis of preferences of the treating clinicians.25, 63, 67, 68, 69 Others have proposed that being randomly assigned in a study, by definition, involves more risk than routine care because some patients will receive an intervention that is different from the intervention(s) that they would otherwise have received outside of the study.68, 69 Commentators following this line of reasoning have pointed out that outcomes for many comparative effectiveness trials are major morbidity outcomes or mortality, highlighting the researchers’ hypothesis that there might be substantial risk differences between the two therapies. An additional controversy stems from the definition of usual care.⁷⁰ For many comparative effectiveness questions, there might be considerable variability in provider practice at the population level, with more uniformity at local levels with specific clinicians, units, or hospitals having strong historical preferences for a particular therapy. Some researchers and bioethicists suggest that a lack of evidence of superiority and variability in practice at the population level is a sufficient demonstration of equipoise, whereas others would insist that equipoise requires variability in provider practice at every level of practice.⁷⁰

Lessons from trials of hydroxychloroquine for COVID-19

Early in the pandemic, hydroxychloroquine was identified in the USA and the UK as a potential therapy for patients admitted to hospital with COVID-19. The way in which the two countries used and evaluated the safety and effectiveness of this available medication, however, serves to demonstrate many of the key differences between explanatory and pragmatic clinical trial methods (table 3 ).31, 71, 72, 73, 74

Table 3.

Comparison of ORCHID and RECOVERY trials in patients with COVID-19

	ORCHID trial	RECOVERY trial
Design	Explanatory	Pragmatic
Patient population	Patients admitted to hospital with COVID-19	Patients admitted to hospital with COVID-19
Study setting	34 academic medical centres	176 hospitals (range of urban or rural, academic or community hospitals)
Intervention(s)	Hydroxychloroquine	Hydroxychloroquine, corticosteroids, lopinavir–ritonavir, azithromycin, tocilizumab, convalescent plasma
Control	Placebo (blinded)	Usual care (unblinded)
Screening and enrolment	Research team	Treating clinicians
Consent	Research team using seven-page informed consent document	Treating clinicians using one-page consent document
Drug delivery	Investigational pharmacy (placebo-controlled)	Clinical pharmacy
Safety monitoring	Daily assessments by research staff during the intervention period; electrocardiography according to protocol; systematic collection of safety outcomes	Patients monitored as they would be in usual care; no study-specific adverse event monitoring; systematic collection of safety outcomes⁷¹
Data collection	Manual collection with follow-up phone calls	Limited to in-hospital outcomes available in electronic health records
Patients enrolled from March 2020 to June 202031, 72 (proportion of reported cases)73, 74	479 patients (1 in 5000 of the 2·2 million cases reported in the USA)	11 874 patients^* (1 in 25 of the 300 000 cases reported in the UK)
Results	Hydroxychloroquine is highly unlikely to improve clinical status at 14 days after initiation of treatment	Hydroxychloroquine, lopinavir–ritonavir, azithromycin, and convalescent plasma are highly unlikely to improve mortality; dexamethasone and tocilizumab improve mortality

Open in a new tab

ORCHID=Outcomes Related to COVID-19 Treated With Hydroxychloroquine Among Inpatients With Symptomatic Disease. RECOVERY=Randomised Evaluation of COVID-19 Therapy.

Enrolment continues in the RECOVERY trial, which had enrolled more than 45 000 patients as of May 31, 2022.

In the USA, knowledge gaps regarding hydroxychloroquine and other available therapies led to two parallel and sometimes conflicting responses: rapid deployment of explanatory randomised trials and widespread variation in treatments provided to patients off label. Within weeks of the first death in the USA attributed to COVID, the ORCHID trial was designed, federally funded, and approved by the US Food and Drug Administration (FDA) and a central institutional review board with unprecedented speed.⁷⁵ Dedicated research staff screened, obtained consent, and randomly assigned patients, performed daily assessments for adherence to trial therapy and adverse events, and manually collected study outcomes by reviewing electronic health records and making follow-up phone calls. Delivery of the study intervention (hydroxychloroquine vs placebo) was blinded to maximise the chances that differences observed between groups were due to hydroxychloroquine. Although enrolment was extremely rapid for an explanatory trial, the effort required by research teams to screen, approach, enrol, and monitor patients resulted in a mean enrolment of only one to two patients per week at each of the 34 participating hospitals, despite high volumes of patients with COVID-19. The trial was stopped on June 19, 2020, after enrolment of 479 patients when the data and safety monitoring board determined that the trial was unlikely to show efficacy for the chosen outcome of clinical status at 14 days. This decision was also informed by new evidence from other studies suggesting that hydroxychloroquine was not effective.⁷² Although the results from these 479 patients contributed to the strong evidence base that hydroxychloroquine is not an effective therapy for COVID-19, a much greater number of patients, estimated to be in the hundreds of thousands, were treated with hydroxychloroquine as part of clinical care while the trial was being conducted,⁷⁶ with the potential risks (and benefits) of the drug, and with minimal, if any, contribution to knowledge for the care of future patients.

During the same period in the UK, the National Health Service (NHS) designed the RECOVERY trial.³¹ Like ORCHID, this trial aimed to compare the effectiveness of available therapies for COVID-19. The RECOVERY investigators chose to use a radically different, pragmatic approach to the study of hydroxychloroquine. At the direction of the NHS, hospitals discouraged the use of hydroxychloroquine outside of trials, instead encouraging clinicians to enrol patients in the RECOVERY trial to facilitate rapid generation of evidence. Health system leaders set a target of enrolling 60% of all patients admitted to hospital with COVID-19 into the RECOVERY trial.⁷⁷ To facilitate rapid enrolment, the trial was approved with an alteration of informed consent, allowing screening, enrolment, randomisation, and intervention delivery to be embedded within routine care by the very large pool of treating clinicians (as opposed to the small pool of research staff available only at some NHS hospitals). Consent for participation in the trial was obtained by clinical personnel using a one-page document. The primary outcome, 28-day mortality, was routinely collected as part of usual care, and the small number of variables collected for the trial were extracted from electronic health records. Using these methods, RECOVERY enrolled approximately 11 000 patients between March 25 and June 5, 2020.⁷⁸ This was about 23 times as many patients as the ORCHID trial enrolled during the same period,8, 72, 78 despite case counts being approximately a tenth of those of the USA.⁷⁹ Given the large number of patients enrolled, RECOVERY was able to provide definitive evidence that hydroxychloroquine did not improve survival for patients with COVID-19 (figure 2 ).8, 31, 32, 72, 78, 80, 81, 82 The NHS rapidly put these results into practice, and by June 16, 2020, the UK Medicines and Healthcare products Regulatory Agency had instructed “clinical trialists using hydroxychloroquine to treat or prevent coronavirus (COVID-19) to suspend recruitment of further participants”.⁸³ Finally, whereas the ORCHID trial was designed to answer only one question, the RECOVERY trial has been able to rapidly enrol a sufficient number of patients to provide evidence on a range of proposed therapies, including corticosteroids, azithromycin, and antivirals.8, 31, 32 Progress in understanding the effects of these therapies followed a similar pattern to that for hydroxychloroquine, with RECOVERY providing robust estimates of mortality, whereas explanatory trials evaluating the same questions enrolled far fewer patients and were unable to provide results on the timescale needed during the pandemic (figure 3 ).8, 31, 32, 84, 85, 86, 87, 88

Explanatory and pragmatic trials of hydroxychloroquine in COVID-19

Five trials (three explanatory and two pragmatic) that randomly assigned at least 100 patients admitted to hospital with COVID-19 to hydroxychloroquine or a control (placebo or usual care) are shown. Total enrolment by trial and treatment group are shown on the left; survival outcomes are included in the stacked bars (orange, patients who died; blue, patients who survived). The ORs and 95% CIs for mortality for each of the five trials are shown on the right. The black vertical line represents an OR of 1·0 (no effect) and the two adjacent grey dashed lines denote ORs of 0·7 (benefit from hydroxychloroquine) and 1·3 (harm from hydroxychloroquine). Each of the pragmatic trials (RECOVERY and SOLIDARITY) enrolled more patients than the three explanatory trials (TEACH, COALITION, and ORCHID) combined. COALITION=Safety and Efficacy of Hydroxychloroquine Associated With Azithromycin in SARS-CoV-2 Virus.⁸⁰ OR=odds ratio. ORCHID=Outcomes Related to COVID-19 Treated With Hydroxychloroquine Among Inpatients With Symptomatic Disease.⁷² RECOVERY=Randomised Evaluation of COVID-19 Therapy.8, 31, 32 SOLIDARITY=Trial of Treatments for COVID-19 in Hospitalized Adults.⁸¹ TEACH=Treating COVID-19 With Hydroxychloroquine.⁸²

Explanatory and pragmatic trials of corticosteroids in COVID-19

Seven trials (five explanatory and two pragmatic) that randomly assigned patients admitted to hospital with COVID-19 to corticosteroids or control (placebo or usual care) are shown. Total enrolment by trial and treatment group are shown on the left; survival outcomes are included in the stacked bars (orange, patients who died; blue, patients who survived). The ORs and 95% CIs for mortality for each of the seven trials are shown on the right. The black vertical line represents an OR of 1·0 (no effect). RECOVERY enrolled more than twelve times as many patients as the five explanatory trials combined. Furthermore, none of the explanatory trials was sufficiently powered to demonstrate statistically significant results for mortality, and a meta-analysis combining all of the explanatory trials would still have failed to conclusively demonstrate that corticosteroids improve mortality in COVID-19. CAPE COVID=Community-Acquired Pneumonia: Evaluation of Corticosteroids.⁸⁴ CoDEX=COVID-19-associated ARDS Treated With Dexamethasone: Alliance Covid-19 Brasil III.⁸⁵ COVID STEROID=Hydrocortisone for COVID-19 and Severe Hypoxia.⁸⁶ Steroids-SARI=Glucocorticoid Therapy for COVID-19 Critically Ill Patients With Severe Acute Respiratory Failure.⁸⁶ DEXA-COVID19=Efficacy of Dexamethasone Treatment for Patients With ARDS Caused by COVID-19.⁸⁷ OR=odds ratio. RECOVERY=Randomised Evaluation of COVID-19 Therapy.8, 31, 32 REMAP-CAP=Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia.⁸⁸

The widespread use of hydroxychloroquine outside of clinical trials in the USA might have been amplified by the substantial attention that it received in the press and an emergency use authorisation from the FDA.⁸⁹ However, lopinavir–ritonavir, azithromycin, and tocilizumab were similarly administered to hundreds of thousands of patients in the USA before any clinical trial data were available to establish safety or efficacy as treatments for COVID-19.⁹⁰ If the USA had, instead, invested in large-scale pragmatic trials, many of these patients could have been included in trials while others were spared exposure to ineffective or potentially harmful therapies.

Unresolved questions for pragmatic trials

The RECOVERY trial provided rapid evidence on the effectiveness of corticosteroids and tocilizumab and the ineffectiveness of hydroxychloroquine, lopinavir–ritonavir, azithromycin, and convalescent plasma to clinicians across the world as they learned how to treat a new disease, probably saving thousands of lives.10, 31, 32, 78, 91 However, there is no perfect trial design, and the RECOVERY trial also highlighted potential challenges that will need to be addressed if pragmatic trials are to be used more broadly after the COVID-19 pandemic.

Embedding screening, enrolment, and randomisation into routine clinical care in pragmatic trials in acute care settings will often require alteration or waiver of consent. As noted above, many researchers and bioethicists would endorse an alteration or waiver of consent for comparative effectiveness research evaluating two treatments that are already in common clinical use with no evidence of either being superior.25, 63, 67, 68, 69 However, COVID-19 presented a situation involving a novel infectious threat with no standards of care, making the risks and benefits of randomisation to drugs approved for other uses hard to assess. Questions remain about appropriate alterations to consent forms and processes, as well as post-enrolment notification practices, in these circumstances. Although traditional explanatory approaches to informed consent have been criticised for the length and readability of informed consent documents and inadequate comprehension by enrolled participants,92, 93 the use of treating clinicians to obtain consent for pragmatic trials could raise concerns regarding the lack of training of these clinicians in protections for human participants and the potential risk for therapeutic misconception (in which patients and families do not appreciate or understand that trial participation advances ends other than care).⁹⁴

Although there is considerable interest in the conduct of pragmatic trials that embed aspects of trial procedures into electronic health records, this approach depends on the completeness and validity of electronic health record data. The validity of electronic health record data might be high for some conditions such as stroke and acute coronary syndrome and low for others such as acute respiratory distress syndrome.95, 96 Furthermore, depending on the informatic tool used, data might not be available in real time, raising concerns about safety monitoring and adverse event reporting.

Finally, most pragmatic trials are not blinded owing to the complexity of masking treatment assignment for an intervention that is delivered as part of routine clinical care. It is generally argued that blinding is less important for objective outcomes such as mortality, the outcome of the RECOVERY trial.⁹⁷ However, the lack of blinding can introduce operational biases, such as differential receipt of co-interventions, which might cause differences in outcomes unrelated to the assigned intervention.⁹⁸ Although treatment adherence and crossover were not major issues in RECOVERY, embedding trial interventions into usual care might also result in trials with poor adherence and high rates of crossover that could interfere with interpretation of trial results.37, 98

Recommendations for acute care research

The RECOVERY trial has demonstrated the power and efficiency of pragmatic trials for acute care research during a pandemic, while simultaneously raising questions about unblinded trials and the balance between respecting patient autonomy and facilitating beneficial research. Below, and in Table 4, Table 5 , we summarise our recommendations regarding the clinical questions that are (and are not) well suited to pragmatic trial designs, and the barriers that must be addressed to facilitate more widespread adoption of pragmatic trials in post-pandemic acute care research.

Table 4.

Recommended use of pragmatic and explanatory trial designs for particular clinical scenarios

	Suggested trial design
Comparative effectiveness research comparing two standard-of-care therapies (eg, anticoagulation strategies)	Pragmatic trial suggested; trial represents minimal additional risk beyond those of routine clinical care
Evaluation of a medication that is already in common clinical use but not approved for the studied indication (off-label), compared with a control (usual care or a placebo)	Pragmatic trial preferred; trial represents minimal additional risk beyond those of routine clinical care
Evaluation of a medication that is approved for another indication but is not approved or already in common clinical use for the studied indication	Explanatory trial preferred; trial represents substantial risks to participants beyond those of routine clinical care and requires traditional informed consent and adverse event monitoring; however, efficiency could be improved by transitioning to a more pragmatic design as safety data become available from early participants, which could be accomplished using pre-planned adaptive trial designs (similar to the seamless phase 2/3 designs used in drug development trials)
Evaluation of a novel, unapproved treatment	Explanatory trial suggested; benefits and risks of treatment are unknown; trial represents substantial risks to participants beyond those of routine clinical care and requires traditional informed consent and adverse event monitoring
Evaluation of a complex or time-intensive intervention that is not in clinical use (eg, a novel mode of ventilation that requires frequent assessments and titration)	Explanatory trial suggested; benefits and risks of treatment are unknown; trial represents substantial risks to participants beyond those of routine clinical care and requires traditional informed consent; intervention fidelity and patient safety require additional resources that are unlikely to be present in routine clinical care

Open in a new tab

Table 5.

Recommendations to facilitate conduct of pragmatic trials in acute care

	Rationale
Assess the risk of research relative to the risk of routine clinical care	Critically ill patients are at risk of serious outcomes, including death, as a consequence of their acute illness and comorbidities; human research protections should focus on additional risk created by study interventions or other study procedures; inappropriately attributing risks arising from a patient's acute clinical scenario to interventions provided in research studies prevents potentially beneficial research and could harm patients
Allow alteration or waiver of consent for trials comparing therapies that patients would receive as part of routine clinical care	If all trial interventions are commonly used in usual care and patients are likely to be exposed to a therapy regardless of trial participation, randomisation represents a minimal incremental risk to participants
Define usual care on the basis of provider practice for similar patients or settings at a population level	If there is no evidence of superiority between two given interventions and there is provider practice variability at the population level for similar patients in similar settings, this would represent sufficient equipoise to allow enrolment at any site, not just those already using a mix of the studied therapies
Develop new methods of demonstrating respect for patients and families in pragmatic trials with greater than minimal risk where consent is impractical	Protecting patients might require novel methods to show respect for people when consent cannot be obtained without impeding potentially beneficial research; these methods could include community consultation, public disclosure, and patient or family notification⁵⁸
Invest in bioinformatics tools for electronic health system interoperability and automated information technology tools within electronic health records to facilitate screening, enrolment, and data abstraction	Improving the efficiency of pragmatic trials will facilitate the generation of evidence to improve the care of future patients
Create incentives for health systems and researchers to improve patient outcomes through the conduct of embedded pragmatic trials evaluating interventions commonly used in clinical care	Investments will be needed to engage all stakeholders to show that arbitrary variation in the absence of evidence hurts patients, whereas structured variation through pragmatic trials holds the potential to help patients

Open in a new tab

We believe that highly efficient, pragmatic trials, such as RECOVERY, are ideal for comparative effectiveness research evaluating therapies that are already in common clinical use (eg, different anticoagulation strategies among patients admitted to hospital with COVID-19).²⁷ Although research regulations in many countries already allow alteration or waiver of consent for various types of minimal risk research, debate continues in some countries over how to define and operationalise the concept of minimal risk for comparative effectiveness research. For the evaluation of readily available therapies with a low risk of adverse events, such as hydroxychloroquine, azithromycin, and convalescent plasma, we argue that the conduct of pragmatic trials with alteration of consent (as in RECOVERY) is preferable to widespread clinical use outside of clinical trials. A much smaller proportion of patients were exposed to ineffective therapies such as hydroxychloroquine in UK hospitals (inside or outside of the RECOVERY trial) than were exposed to those therapies in routine care during the same period in the USA. We believe that the most ethical system of research and clinical care is one that minimises the number of patients exposed to ineffective therapies, efficiently identifies effective therapies, and ensures that effective therapies are administered to future patients. It is notable, therefore, that research regulations in many countries, including the USA, would not have allowed the approaches to enrolment and monitoring that were used in RECOVERY. In our view, it would have been advantageous for countries such as the USA to consider using alterations of consent to facilitate the conduct of pragmatic trials and, when possible, to avoid the widespread use of unproven therapies in routine clinical care.

Conversely, the pragmatic approach to consent and adverse event monitoring, and the lack of blinding used in RECOVERY, might not be rigorous enough to evaluate novel, unapproved medications with less well described safety profiles such as those studied in the recently completed monoclonal antibody arm of RECOVERY.⁹⁹ We believe that exposing patients to therapies with unknown risks and benefits that they could not have received as part of usual care requires the intense patient protections afforded by explanatory trials to ensure the safety of prospective participants. Furthermore, trials that could be used as evidence for the approval and marketing of a novel therapy should include rigorous efforts to prevent bias (eg, blinding) and evaluate for unexpected side-effects (intensive monitoring for adverse events) to ensure definitive trial results and to protect future patients who might receive the studied medication as part of clinical care after approval. Future trials of novel therapies could benefit, however, from combining aspects of pragmatic trials such as RECOVERY (eg, screening, intervention delivery, and outcome assessment) with stronger protections from traditional, explanatory trials (eg, blinding and intensive safety monitoring).

Additional ways to overcome barriers to the adoption of pragmatic trial methods include improved assessment of risks in clinical trials and a standardised approach to establishing usual care. For many countries, conducting any pragmatic trial would require regulatory changes to the way in which risk of trial participation (vs the risk of routine clinical care) is assessed. We believe that an acute care trial evaluating treatments that patients are likely to receive as part of usual care adds little, if any, risk to the participant, and an alteration or waiver of consent should be considered when consent is impracticable. For trials using a waiver of consent, researchers and regulatory agencies should actively engage with patients and families to explore alternative methods of showing respect for participants that also facilitate beneficial research (eg, community consultation, pre-enrolment public disclosure, and post-enrolment notification). For comparative effectiveness research, we also suggest that usual care should be defined on the basis of provider practice variability within a geographical region (eg, country or large regional health-care system) for similar patients in similar settings. If, for example, some hospitals use a mix of therapy A and therapy B, while other hospitals give every patient therapy A, and others give every patient therapy B, we suggest that this represents equipoise that would be sufficient to allow the conduct of a trial comparing therapy A and therapy B at any of these hospitals, and not just those already using a mix of the two therapies.

Embedding pragmatic trials into clinical care will also require improved incentives for health-care systems to enhance clinical outcomes through participation in research evaluating therapies already in common clinical use.¹⁰⁰ The conduct of pragmatic trials would be further facilitated by addressing challenges with electronic health system interoperability and developing information technology tools for trial screening, enrolment, and data extraction, including consideration of patient-reported outcomes. Finally, embedding pragmatic trials into clinical care will require engagement from many relevant stakeholders, including patients, families, health system leaders, clinicians, researchers, experts in human participant protection, and funding agencies, to develop a shared understanding. Arbitrary variation in provider practices in usual care could harm patients, whereas structured variation through pragmatic trials is unlikely to increase risks to the patients, holds the potential to improve outcomes by generating new evidence, and facilitates the rapid dissemination and adoption of new findings.

Conclusions

Although explanatory trials are ideal for the evaluation of novel therapies, pragmatic trial methods such as those used for the RECOVERY trial have the potential to increase considerably the efficiency and generalisability of acute care clinical trials and provide definitive evidence for many commonly used treatments during and after the COVID-19 pandemic. Broader adoption of pragmatic trial methods to definitively assess commonly used treatments will require alignment of incentives for health-care systems, engagement from many relevant stakeholder groups, and improved regulations for human research protection, matching the intensity of the protections to the risk attributable to enrolment in the trial.

Search strategy and selection criteria

We searched PubMed for articles published from Jan 1, 2020, to Dec 13, 2021, using combinations of the search terms “pragmatic”, “explanatory”, “comparative effectiveness”, “waiver of consent”, “alteration of consent”, “minimal risk”, “hydroxychloroquine”, “recovery”, “ethics AND acute care research”, “ethics AND comparative effectiveness”, “pragmatic AND (COVID-19 OR SARS-CoV-2)”, and “explanatory AND (COVID-19 OR SARS-CoV-2)”. Only articles published in English were included. Although this Personal View represents the opinions of the authors and is not intended as a systematic review, the PubMed search was used to generate the final reference list and focused on articles with relevance to the topics covered in this paper: the impact of provider practice variation on clinical outcomes, the ethics of acute care research, and the use of pragmatic and explanatory trial designs for comparative effectiveness research. Preprints relevant to the aims of the paper were considered; publication details were updated as the paper was revised and prepared for publication.

This online publication has been corrected. The corrected version first appeared at thelancet.com/respiratory on November 2, 2022

Declaration of interests

SMB reports personal fees from Hamilton for chairing a data safety and monitoring board (DSMB). EG reports grants and research contracts from Philips and Radiometer, consulting fees from Baxter, and speaker's fees from Edwards. MNG reports fees from Regeneron for DSMB participation; she has received support to attend board meetings for the American Thoracic Society. MOH reports fees from Elsevier for statistical peer review, from Berkeley Research Group, Pura Vida Investments, and Guidepoint Advisers for statistical consulting, and from the American Thoracic Society for contributions as a statistical editor of the Annals of the American Thoracic Society. SJ reports fees from Drager, Fisher-Paykel, Baxter, Medtronic, and Fresenius-Xenios for lectures and training courses for physicians and nurses. JCM reports consulting fees from Gilead, and travel support from the Bill and Melinda Gates Foundation. MAM reports research funding from Roche-Genentec; he has received fees from Novartis, Johnson & Johnson, and Citius for DSMB participation. TWR reports personal fees from Sanofi for DSMB participation, and consulting fees and stock options from Cumberland Pharmaceuticals for contributions on a consultancy basis as Director of Medical Affairs. LBW has received research contracts from Boehringer Ingelheim, Genentech, and CSL Behring; she reports consulting fees from Boehringer Ingelheim, Merck, Quark, Citius, and Foresee; she has served on a data monitoring committee for the SIGNET trial and on DSMBs for the INVENT COVID and CounterCovid trials. WHS has received research funding from Endpoint Health and Merck; he reports personal consulting fees from Aerpio Pharmaceuticals, Merck, and Baxter; he has received personal fees for DSMB participation from BioAegis. AM reports grants to his institution from Roche Diagnostics, Abbott Laboratories, 4TEEN4 Pharmaceuticals, and Windtree Therapeutics; honoraria for presentations from Roche Diagnostics, Bayer, and Merck Sharp & Dohme; and fees for DSMB participation from Corteria Pharmaceuticals, S-Form Pharma, FIRE-1 Foundry, 4TEEN4 Pharmaceuticals, Implicity, and Adrenomed. AM is coinventor on a patent for combination therapy for patients with acute or persistent dyspnoea (S-Form Pharma). SPC reports consulting fees from Vir Biotechnology. The other authors declare no competing interests.

Acknowledgments

The 2020 Critical Care Clinical Trialists (3CT) Workshop received unrestricted funding from Abbott Diagnostics and AM-Pharma to partially cover travel and lodging costs, when necessary, but no further payments were made to participants. These organisations had no role in the scientific programme for the conference; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication. JDC was supported by the US National Institutes of Health (NIH)/National Heart, Lung, and Blood Institute (NHLBI) grant K23HL153584. LMB was supported by UL1-TR-002243 from the US National Center for Advancing Translational Sciences. SMB receives research funding from NIH, the Centers for Disease Control and Prevention (CDC), and the Department of Defense (DoD). MNG receives grants from NIH (OT2 HL156812), NHLBI (U01 HL122998), the Agency for Healthcare Research and Quality (ARHQ; R18 HS026188), and CDC (200-2016-91801). MOH was the recipient of grants from NIH (R00 HL141678) and the Patient-Centered Outcomes Research Institute (PCORI; ME-2020C1-19220). WHS receives research funding from NIH (42-312-0217571-66406L; 1OT2HL156812), CDC (75-D-30122-C-12914), DoD (JW190515), and PCORI (1409-24099). TWR was supported by NIH/NHLBI grant U01 HL123033 (K01HL141637). AET was supported by NIH/NHLBI grant K01HL141637. LBW was supported by NIH grants HL103836 and HL135849SPC. SPC receives research funding from NHLBI. The content of this Personal View is solely the responsibility of the authors and does not necessarily represent the official views of ARHQ, CDC, DoD, NHLBI, NIH, the US Department of Health and Human Services, PCORI, or PCORI's Board of Governors or Methodology Committee.

Contributors

The authors contributed to discussions at the 2020 Critical Care Clinical Trialists (3CT) Workshop, the selection of content for the manuscript, and the writing of this Personal View. The first draft of the manuscript was written by JDC, LMB, and SPC, and all authors reviewed, edited, and commented on several versions of the manuscript. All authors read and approved the final manuscript.

References

1.Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403–416. doi: 10.1056/NEJMoa2035389. [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021;384:1412–1423. doi: 10.1056/NEJMoa2101765. [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Weinreich DM, Sivapalasingam S, Norton T, et al. REGN-COV2, a neutralizing antibody cocktail, in outpatients with Covid-19. N Engl J Med. 2021;384:238–251. doi: 10.1056/NEJMoa2035002. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Chen P, Nirula A, Heller B, et al. SARS-CoV-2 neutralizing antibody LY-CoV555 in outpatients with Covid-19. N Engl J Med. 2021;384:229–237. doi: 10.1056/NEJMoa2029849. [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Cohen MS. Monoclonal antibodies to disrupt progression of early Covid-19 infection. N Engl J Med. 2021;384:289–291. doi: 10.1056/NEJMe2034495. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of Covid-19 — final report. N Engl J Med. 2020;383:1813–1826. doi: 10.1056/NEJMoa2007764. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Horby P, Lim WS, Emberson JR, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384:693–704. doi: 10.1056/NEJMoa2021436. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.REMAP-CAP Investigators Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N Engl J Med. 2021;384:1491–1502. doi: 10.1056/NEJMoa2100433. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Abani O, Abbas A, Abbas F, et al. Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397:1637–1645. doi: 10.1016/S0140-6736(21)00676-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Mu Z, Haynes BF, Cain DW. HIV mRNA vaccines-progress and future paths. Vaccines (Basel) 2021;9:134. doi: 10.3390/vaccines9020134. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Kose N, Fox JM, Sapparapu G, et al. A lipid-encapsulated mRNA encoding a potently neutralizing human monoclonal antibody protects against chikungunya infection. Sci Immunol. 2019;4 doi: 10.1126/sciimmunol.aaw6647. [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Miao L, Zhang Y, Huang L. mRNA vaccine for cancer immunotherapy. Mol Cancer. 2021;20:41. doi: 10.1186/s12943-021-01335-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Angus DC, Gordon AC, Bauchner H. Emerging lessons from COVID-19 for the US clinical research enterprise. JAMA. 2021;325:1159–1161. doi: 10.1001/jama.2021.3284. [DOI] [PubMed] [Google Scholar]
15.Harhay MO, Casey JD, Clement M, et al. Contemporary strategies to improve clinical trial design for critical care research: insights from the First Critical Care Clinical Trialists Workshop. Intensive Care Med. 2020;46:930–942. doi: 10.1007/s00134-020-05934-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Beskow LM, Lindsell CJ, Rice TW. Consent for acute care research and the regulatory “gray zone”. Am J Bioeth. 2020;20:26–28. doi: 10.1080/15265161.2020.1745950. [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of heterogeneity of treatment effect for reporting and analysis of randomized trials in critical care. Am J Respir Crit Care Med. 2015;192:1045–1051. doi: 10.1164/rccm.201411-2125CP. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Reddy K, Sinha P, O'Kane CM, Gordon AC, Calfee CS, McAuley DF. Subphenotypes in critical care: translation into clinical practice. Lancet Respir Med. 2020;8:631–643. doi: 10.1016/S2213-2600(20)30124-7. [DOI] [PubMed] [Google Scholar]
19.Patel J, Beishuizen A, Ruiz XB, et al. A randomized trial of otilimab in severe COVID-19 pneumonia (OSCAR) medRxiv. 2021 doi: 10.1101/2021.04.14.21255475. published online Apr 17. (preprint). [DOI] [PMC free article] [PubMed] [Google Scholar]
20.ACTIV-3/TICO LY-CoV555 Study Group A Neutralizing monoclonal antibody for hospitalized patients with Covid-19. N Engl J Med. 2021;384:905–914. doi: 10.1056/NEJMoa2033130. [DOI] [PMC free article] [PubMed] [Google Scholar]
21.US Food and Drug Administration Use of electronic informed consent in clinical investigations – questions and answers. December 2016. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-electronic-informed-consent-clinical-investigations-questions-and-answers
22.Fanaroff AC, Califf RM, Windecker S, Smith SC, Jr, Lopes RD. Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008–2018. JAMA. 2019;321:1069–1080. doi: 10.1001/jama.2019.1122. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Sims CR, Warner MA, Stelfox HT, Hyder JA. Above the GRADE: evaluation of guidelines in critical care medicine. Crit Care Med. 2019;47:109–113. doi: 10.1097/CCM.0000000000003467. [DOI] [PubMed] [Google Scholar]
24.Zhang Z, Hong Y, Liu N. Scientific evidence underlying the recommendations of critical care clinical practice guidelines: a lack of high level evidence. Intensive Care Med. 2018;44:1189–1191. doi: 10.1007/s00134-018-5142-8. [DOI] [PubMed] [Google Scholar]
25.Simon GE, Platt R, Hernandez AF. Evidence from pragmatic trials during routine care — slouching toward a learning health system. N Engl J Med. 2020;382:1488–1491. doi: 10.1056/NEJMp1915448. [DOI] [PubMed] [Google Scholar]
26.Faden RR, Kass NE, Goodman SN, Pronovost P, Tunis S, Beauchamp TL. An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep. 2013;43:S16–S27. doi: 10.1002/hast.134. [DOI] [PubMed] [Google Scholar]
27.REMAP-CAP, ACTIV-4a, and ATTACC Investigators Therapeutic anticoagulation with heparin in critically ill patients with Covid-19. N Engl J Med. 2021;385:777–789. doi: 10.1056/NEJMoa2103417. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Koyfman SA, Reddy CA, Hizlan S, Leek AC, Kodish AE. Informed consent conversations and documents: A quantitative comparison. Cancer. 2016;122:464–469. doi: 10.1002/cncr.29759. [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637–648. doi: 10.1016/0021-9681(67)90041-0. [DOI] [PubMed] [Google Scholar]
30.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350 doi: 10.1136/bmj.h2147. [DOI] [PubMed] [Google Scholar]
31.RECOVERY Collaborative Group Lopinavir–ritonavir in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2020;396:1345–1352. doi: 10.1016/S0140-6736(20)32013-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Abaleke E, Abbas M, Abbasi S, et al. Azithromycin in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397:605–612. doi: 10.1016/S0140-6736(21)00149-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Semler MW, Self WH, Wanderer JP, et al. Balanced crystalloids versus saline in critically ill adults. N Engl J Med. 2018;378:829–839. doi: 10.1056/NEJMoa1711584. [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Self WH, Semler MW, Wanderer JP, et al. Balanced crystalloids versus saline in noncritically ill adults. N Engl J Med. 2018;378:819–828. doi: 10.1056/NEJMoa1711586. [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Shah ASV, Anand A, Strachan FE, et al. High-sensitivity troponin in the evaluation of patients with suspected acute coronary syndrome: a stepped-wedge, cluster-randomised controlled trial. Lancet. 2018;392:919–928. doi: 10.1016/S0140-6736(18)31923-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Adler-Milstein J. Moving past the EHR interoperability blame game. NEJM Catalyst. 2017 https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0448 published online July 18. [Google Scholar]
37.Young PJ, Bagshaw SM, Forbes AB, et al. Effect of stress ulcer prophylaxis with proton pump inhibitors vs histamine-2 receptor blockers on in-hospital mortality among ICU patients receiving invasive mechanical ventilation: the PEPTIC randomized clinical trial. JAMA. 2020;323:616–626. doi: 10.1001/jama.2019.22190. [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Fröbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med. 2013;369:1587–1597. doi: 10.1056/NEJMoa1308789. [DOI] [PubMed] [Google Scholar]
39.Marquis-Gravel G, Roe MT, Robertson HR, et al. Rationale and design of the aspirin dosing—a patient-centric trial assessing benefits and long-term effectiveness (ADAPTABLE) trial. JAMA Cardiol. 2020;5:598–607. doi: 10.1001/jamacardio.2020.0116. [DOI] [PubMed] [Google Scholar]
40.Matthay MA, Luetkemeyer AF. IL-6 receptor antagonist therapy for patients hospitalized for COVID-19: who, when, and how? JAMA. 2021;326:483–485. doi: 10.1001/jama.2021.11121. [DOI] [PubMed] [Google Scholar]
41.Acute Respiratory Distress Syndrome Network Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med. 2000;342:1301–1308. doi: 10.1056/NEJM200005043421801. [DOI] [PubMed] [Google Scholar]
42.Ranieri VM, Thompson BT, Barie PS, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med. 2012;366:2055–2064. doi: 10.1056/NEJMoa1202290. [DOI] [PubMed] [Google Scholar]
43.National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network Comparison of two fluid-management strategies in acute lung injury. N Engl J Med. 2006;354:2564–2575. doi: 10.1056/NEJMoa062200. [DOI] [PubMed] [Google Scholar]
44.Roberts I, Yates D, Sandercock P, et al. Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet. 2004;364:1321–1328. doi: 10.1016/S0140-6736(04)17188-2. [DOI] [PubMed] [Google Scholar]
45.Janz DR, Semler MW, Lentz RJ, et al. Randomized trial of video laryngoscopy for endotracheal intubation of critically ill adults. Crit Care Med. 2016;44:1980–1987. doi: 10.1097/CCM.0000000000001841. [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Semler MW, Janz DR, Russell DW, et al. A multicenter, randomized trial of ramped position vs sniffing position during endotracheal intubation of critically ill adults. Chest. 2017;152:712–722. [Google Scholar]
47.Casey JD, Janz DR, Russell DW, et al. Bag-mask ventilation during tracheal intubation of critically ill adults. N Engl J Med. 2019;380:811–821. doi: 10.1056/NEJMoa1812405. [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Driver BE, Prekker ME, Klein LR, et al. Effect of use of a bougie vs endotracheal tube and stylet on first-attempt intubation success among patients with difficult airways undergoing emergency intubation: a randomized clinical trial. JAMA. 2018;319:2179–2189. doi: 10.1001/jama.2018.6496. [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Caputo N, Azan B, Domingues R, et al. Emergency department use of apneic oxygenation versus usual care during rapid sequence intubation: a randomized controlled trial (the ENDAO trial) Acad Emerg Med. 2017;24:1387–1394. doi: 10.1111/acem.13274. [DOI] [PubMed] [Google Scholar]
50.Semler MW, Wanderer JP, Ehrenfeld JM, et al. Balanced crystalloids versus saline in the intensive care unit. The SALT randomized trial. Am J Respir Crit Care Med. 2017;195:1362–1372. doi: 10.1164/rccm.201607-1345OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Wilson FP, Martin M, Yamamoto Y, et al. Electronic health record alerts for acute kidney injury: multicenter, randomized clinical trial. BMJ. 2021;372 doi: 10.1136/bmj.m4786. [DOI] [PMC free article] [PubMed] [Google Scholar]
52.Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283:2701–2711. doi: 10.1001/jama.283.20.2701. [DOI] [PubMed] [Google Scholar]
53.Nuremberg Military Tribunal The Nuremberg Code. JAMA. 1996;276 [PubMed] [Google Scholar]
54.World Medical Association declaration of Helsinki Recommendations guiding physicians in biomedical research involving human subjects. JAMA. 1997;277:925–926. [PubMed] [Google Scholar]
55.US Department of Health and Human Services Office for Human Research Protections The Belmont report. Ethical principles and guidelines for the protection of human subjects of research. April 18, 1979. https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html
56.Hillier TA, Pedula KL, Ogasawara KK, et al. A pragmatic, randomized clinical trial of gestational diabetes screening. N Engl J Med. 2021;384:895–904. doi: 10.1056/NEJMoa2026028. [DOI] [PMC free article] [PubMed] [Google Scholar]
57.US Department of Health and Human Services Office for Human Research Protections 45 CFR 46. Feb 16, 2016. https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html
58.US Food and Drug Administration Exception from informed consent requirements for emergency research. April 2013. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/exception-informed-consent-requirements-emergency-research
59.Feldman WB, Hey SP, Kesselheim AS. A systematic review of the Food And Drug Administration's ‘exception from informed consent’ pathway. Health Aff (Millwood) 2018;37:1605–1614. doi: 10.1377/hlthaff.2018.0501. [DOI] [PubMed] [Google Scholar]
60.Klein L, Moore J, Biros M. A 20-year review: the use of exception from informed consent and waiver of informed consent in emergency research. Acad Emerg Med. 2018;25:1169–1177. doi: 10.1111/acem.13438. [DOI] [PubMed] [Google Scholar]
61.Feldman WB, Hey SP, Franklin JM, Kesselheim AS. Public approval of exception from informed consent in emergency clinical trials: a systematic review of community consultation surveys. JAMA Netw Open. 2019;2 doi: 10.1001/jamanetworkopen.2019.7591. [DOI] [PMC free article] [PubMed] [Google Scholar]
62.Faden RR, Beauchamp TL, Kass NE. Informed consent, comparative effectiveness, and learning health care. N Engl J Med. 2014;370:766–768. doi: 10.1056/NEJMhle1313674. [DOI] [PubMed] [Google Scholar]
63.Kass N, Faden R, Tunis S. Addressing low-risk comparative effectiveness research in proposed changes to US federal regulations governing research. JAMA. 2012;307:1589–1590. doi: 10.1001/jama.2012.491. [DOI] [PubMed] [Google Scholar]
64.Faden RR, Beauchamp TL, Kass NE. Informed consent for comparative effectiveness trials. N Engl J Med. 2014;370:1959–1960. doi: 10.1056/NEJMc1403310. [DOI] [PubMed] [Google Scholar]
65.Morain SR, Tambor E, Moloney R, et al. Stakeholder perspectives regarding alternate approaches to informed consent for comparative effectiveness research. Learn Health Syst. 2017;2 doi: 10.1002/lrh2.10047. [DOI] [PMC free article] [PubMed] [Google Scholar]
66.Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–926. doi: 10.1136/bmj.39489.470347.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
67.Platt R, Kass NE, McGraw D. Ethics, regulation, and comparative effectiveness research: time for a change. JAMA. 2014;311:1497–1498. doi: 10.1001/jama.2014.2144. [DOI] [PubMed] [Google Scholar]
68.Lantos JD, Wendler D, Septimus E, Wahba S, Madigan R, Bliss G. Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials. Clin Trials. 2015;12:485–493. doi: 10.1177/1740774515597687. [DOI] [PMC free article] [PubMed] [Google Scholar]
69.Goldstein CE, Weijer C, Brehaut JC, et al. Ethical issues in pragmatic randomized controlled trials: a review of the recent literature identifies gaps in ethical argumentation. BMC Med Ethics. 2018;19:14. doi: 10.1186/s12910-018-0253-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
70.Thompson BT, Schoenfeld D. Usual care as the control group in clinical trials of nonpharmacologic interventions. Proc Am Thorac Soc. 2007;4:577–582. doi: 10.1513/pats.200706-072JK. [DOI] [PMC free article] [PubMed] [Google Scholar]
71.Morain S, O'Rourke P, for the Ethics and Regulatory Core. NIH Health Care Systems Research Collaboratory Data monitoring in pragmatic clinical trials: points to consider. June 28, 2021. https://dcricollab.dcri.duke.edu/sites/NIHKR/KR/PCT%20Data%20Monitoring%20Committees-Points%20to%20Consider.pdf
72.Self WH, Semler MW, Leither LM, et al. Effect of hydroxychloroquine on clinical status at 14 days in hospitalized patients with COVID-19: a randomized clinical trial. JAMA. 2020;324:2165–2176. doi: 10.1001/jama.2020.22240. [DOI] [PMC free article] [PubMed] [Google Scholar]
73.Statista Cumulative number of coronavirus (COVID-19) cases in the United Kingdom (UK) since. January 2020. https://www.statista.com/statistics/1101958/cumulative-coronavirus-cases-in-the-uk/
74.Centers for Disease Control and Prevention COVID data tracker. March 28, 2020. https://covid.cdc.gov/covid-data-tracker
75.Casey JD, Johnson NJ, Semler MW, et al. Rationale and design of ORCHID: a randomized placebo-controlled clinical trial of hydroxychloroquine for adults hospitalized with COVID-19. Ann Am Thorac Soc. 2020;17:1144–1153. doi: 10.1513/AnnalsATS.202005-478SD. [DOI] [PMC free article] [PubMed] [Google Scholar]
76.Bull-Otterson L, Gray EB, Budnitz DS, et al. Hydroxychloroquine and chloroquine prescribing patterns by provider specialty following initial reports of potential benefit for COVID-19 treatment — United States, January–June 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1210–1215. doi: 10.15585/mmwr.mm6935a4. [DOI] [PMC free article] [PubMed] [Google Scholar]
77.National Institute for Health Research UK's Chief Medical Officers urge hospitals to recruit 60% of eligible COVID-19 patients into RECOVERY trial. Aug 10, 2020. https://www.nihr.ac.uk/news/uks-chief-medical-officers-urge-hospitals-to-recruit-60-of-eligible-covid-19-patients-into-recovery-trial/25515
78.Horby P, Mafham M, Linsell L, et al. Effect of hydroxychloroquine in hospitalized patients with Covid-19. N Engl J Med. 2020;383:2030–2040. doi: 10.1056/NEJMoa2022926. [DOI] [PMC free article] [PubMed] [Google Scholar]
79.Financial Times Coronavirus tracked: see how your country compares. https://ig.ft.com/coronavirus-chart
80.Cavalcanti AB, Zampieri FG, Rosa RG, et al. Hydroxychloroquine with or without azithromycin in mild-to-moderate Covid-19. N Engl J Med. 2020;383:2041–2052. doi: 10.1056/NEJMoa2019014. [DOI] [PMC free article] [PubMed] [Google Scholar]
81.WHO Solidarity Trial Consortium Repurposed antiviral drugs for Covid-19 — interim WHO Solidarity trial results. N Engl J Med. 2021;384:497–511. doi: 10.1056/NEJMoa2023184. [DOI] [PMC free article] [PubMed] [Google Scholar]
82.Ulrich RJ, Troxel AB, Carmody E, et al. Treating COVID-19 With Hydroxychloroquine (TEACH): a multicenter, double-blind randomized controlled trial in hospitalized patients. Open Forum Infect Dis. 2020;7 doi: 10.1093/ofid/ofaa446. [DOI] [PMC free article] [PubMed] [Google Scholar]
83.Medicines and Healthcare products Regulatory Agency MHRA suspends recruitment to COVID-19 hydroxychloroquine trials. June 16, 2020. https://www.gov.uk/government/news/mhra-suspends-recruitment-to-covid-19-hydroxychloroquine-trials
84.Dequin P-F, Heming N, Meziani F, et al. Effect of hydrocortisone on 21-day mortality or respiratory support among critically ill patients with COVID-19: a randomized clinical trial. JAMA. 2020;324:1298–1306. doi: 10.1001/jama.2020.16761. [DOI] [PMC free article] [PubMed] [Google Scholar]
85.Tomazini BM, Maia IS, Cavalcanti AB, et al. Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX randomized clinical trial. JAMA. 2020;324:1307–1316. doi: 10.1001/jama.2020.17021. [DOI] [PMC free article] [PubMed] [Google Scholar]
86.The WHO Rapid Evidence Appraisal for COVID-19 Therapies REACT Working Group. Sterne JAC, Murthy S, et al. Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis. JAMA. 2020;324:1330–1341. doi: 10.1001/jama.2020.17023. [DOI] [PMC free article] [PubMed] [Google Scholar]
87.WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis. JAMA. 2020;324:1330–1341. doi: 10.1001/jama.2020.17023. [DOI] [PMC free article] [PubMed] [Google Scholar]
88.Angus DC, Derde L, Al-Beidh F, et al. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: the REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA. 2020;324:1317–1329. doi: 10.1001/jama.2020.17022. [DOI] [PMC free article] [PubMed] [Google Scholar]
89.US Food and Drug Administration Emergency use authorization for use of chloroquine phosphate or hydroxychloroquine sulfate supplied from the strategic national stockpile for treatment of 2019 coronavirus disease. March 28, 2020. https://www.fda.gov/media/136534/download
90.Gupta S, Hayek SS, Wang W, et al. Factors associated with death in critically ill patients with coronavirus disease 2019 in the US. JAMA Intern Med. 2020;180:1436–1447. doi: 10.1001/jamainternmed.2020.3596. [DOI] [PMC free article] [PubMed] [Google Scholar]
91.Abani O, Abbas A, Abbas F, et al. Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised controlled, open-label, platform trial. Lancet. 2021;397:2049–2059. doi: 10.1016/S0140-6736(21)00897-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
92.Emanuel EJ, Boyle CW. Assessment of length and readability of informed consent documents for COVID-19 vaccine trials. JAMA Netw Open. 2021;4 doi: 10.1001/jamanetworkopen.2021.10843. [DOI] [PMC free article] [PubMed] [Google Scholar]
93.Flory J, Emanuel E. Interventions to improve research participants' understanding in informed consent for research: a systematic review. JAMA. 2004;292:1593–1601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
94.Kimmelman J. The therapeutic misconception at 25: treatment, research, and confusion. Hastings Cent Rep. 2007;37:36–42. doi: 10.1353/hcr.2007.0092. [DOI] [PubMed] [Google Scholar]
95.Davidson J, Banerjee A, Muzambi R, Smeeth L, Warren-Gash C. Validity of acute cardiovascular outcome diagnoses recorded in European electronic health records: a systematic review. Clin Epidemiol. 2020;12:1095–1111. doi: 10.2147/CLEP.S265619. [DOI] [PMC free article] [PubMed] [Google Scholar]
96.McKown AC, Brown RM, Ware LB, Wanderer JP. External validity of electronic sniffers for automated recognition of acute respiratory distress syndrome. J Intensive Care Med. 2019;34:946–954. doi: 10.1177/0885066617720159. [DOI] [PMC free article] [PubMed] [Google Scholar]
97.Day SJ, Altman DG. Statistics notes: blinding in clinical trials and other studies. BMJ. 2000;321:504. doi: 10.1136/bmj.321.7259.504. [DOI] [PMC free article] [PubMed] [Google Scholar]
98.Rice TW, Kripalani S, Lindsell CJ. Proton pump inhibitors vs histamine-2 receptor blockers for stress ulcer prophylaxis in critically ill patients: issues of interpretability in pragmatic trials. JAMA. 2020;323:611–613. doi: 10.1001/jama.2019.22436. [DOI] [PubMed] [Google Scholar]
99.RECOVERY Collaborative Group Casirivimab and imdevimab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2022;399:665–676. doi: 10.1016/S0140-6736(22)00163-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
100.Platt R, Simon GE, Hernandez AF. Is learning worth the trouble? — Improving health care system participation in embedded research. N Engl J Med. 2021;385:5–7. doi: 10.1056/NEJMp2101700. [DOI] [PubMed] [Google Scholar]

[bib1] 1.Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403–416. doi: 10.1056/NEJMoa2035389. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib2] 2.Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib3] 3.Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021;384:1412–1423. doi: 10.1056/NEJMoa2101765. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib4] 4.Weinreich DM, Sivapalasingam S, Norton T, et al. REGN-COV2, a neutralizing antibody cocktail, in outpatients with Covid-19. N Engl J Med. 2021;384:238–251. doi: 10.1056/NEJMoa2035002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib5] 5.Chen P, Nirula A, Heller B, et al. SARS-CoV-2 neutralizing antibody LY-CoV555 in outpatients with Covid-19. N Engl J Med. 2021;384:229–237. doi: 10.1056/NEJMoa2029849. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib6] 6.Cohen MS. Monoclonal antibodies to disrupt progression of early Covid-19 infection. N Engl J Med. 2021;384:289–291. doi: 10.1056/NEJMe2034495. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib7] 7.Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of Covid-19 — final report. N Engl J Med. 2020;383:1813–1826. doi: 10.1056/NEJMoa2007764. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib8] 8.Horby P, Lim WS, Emberson JR, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384:693–704. doi: 10.1056/NEJMoa2021436. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib9] 9.REMAP-CAP Investigators Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N Engl J Med. 2021;384:1491–1502. doi: 10.1056/NEJMoa2100433. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib10] 10.Abani O, Abbas A, Abbas F, et al. Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397:1637–1645. doi: 10.1016/S0140-6736(21)00676-0. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib11] 11.Mu Z, Haynes BF, Cain DW. HIV mRNA vaccines-progress and future paths. Vaccines (Basel) 2021;9:134. doi: 10.3390/vaccines9020134. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib12] 12.Kose N, Fox JM, Sapparapu G, et al. A lipid-encapsulated mRNA encoding a potently neutralizing human monoclonal antibody protects against chikungunya infection. Sci Immunol. 2019;4 doi: 10.1126/sciimmunol.aaw6647. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib13] 13.Miao L, Zhang Y, Huang L. mRNA vaccine for cancer immunotherapy. Mol Cancer. 2021;20:41. doi: 10.1186/s12943-021-01335-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib14] 14.Angus DC, Gordon AC, Bauchner H. Emerging lessons from COVID-19 for the US clinical research enterprise. JAMA. 2021;325:1159–1161. doi: 10.1001/jama.2021.3284. [DOI] [PubMed] [Google Scholar]

[bib15] 15.Harhay MO, Casey JD, Clement M, et al. Contemporary strategies to improve clinical trial design for critical care research: insights from the First Critical Care Clinical Trialists Workshop. Intensive Care Med. 2020;46:930–942. doi: 10.1007/s00134-020-05934-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib16] 16.Beskow LM, Lindsell CJ, Rice TW. Consent for acute care research and the regulatory “gray zone”. Am J Bioeth. 2020;20:26–28. doi: 10.1080/15265161.2020.1745950. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib17] 17.Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of heterogeneity of treatment effect for reporting and analysis of randomized trials in critical care. Am J Respir Crit Care Med. 2015;192:1045–1051. doi: 10.1164/rccm.201411-2125CP. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib18] 18.Reddy K, Sinha P, O'Kane CM, Gordon AC, Calfee CS, McAuley DF. Subphenotypes in critical care: translation into clinical practice. Lancet Respir Med. 2020;8:631–643. doi: 10.1016/S2213-2600(20)30124-7. [DOI] [PubMed] [Google Scholar]

[bib19] 19.Patel J, Beishuizen A, Ruiz XB, et al. A randomized trial of otilimab in severe COVID-19 pneumonia (OSCAR) medRxiv. 2021 doi: 10.1101/2021.04.14.21255475. published online Apr 17. (preprint). [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib20] 20.ACTIV-3/TICO LY-CoV555 Study Group A Neutralizing monoclonal antibody for hospitalized patients with Covid-19. N Engl J Med. 2021;384:905–914. doi: 10.1056/NEJMoa2033130. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib21] 21.US Food and Drug Administration Use of electronic informed consent in clinical investigations – questions and answers. December 2016. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-electronic-informed-consent-clinical-investigations-questions-and-answers

[bib22] 22.Fanaroff AC, Califf RM, Windecker S, Smith SC, Jr, Lopes RD. Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008–2018. JAMA. 2019;321:1069–1080. doi: 10.1001/jama.2019.1122. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib23] 23.Sims CR, Warner MA, Stelfox HT, Hyder JA. Above the GRADE: evaluation of guidelines in critical care medicine. Crit Care Med. 2019;47:109–113. doi: 10.1097/CCM.0000000000003467. [DOI] [PubMed] [Google Scholar]

[bib24] 24.Zhang Z, Hong Y, Liu N. Scientific evidence underlying the recommendations of critical care clinical practice guidelines: a lack of high level evidence. Intensive Care Med. 2018;44:1189–1191. doi: 10.1007/s00134-018-5142-8. [DOI] [PubMed] [Google Scholar]

[bib25] 25.Simon GE, Platt R, Hernandez AF. Evidence from pragmatic trials during routine care — slouching toward a learning health system. N Engl J Med. 2020;382:1488–1491. doi: 10.1056/NEJMp1915448. [DOI] [PubMed] [Google Scholar]

[bib26] 26.Faden RR, Kass NE, Goodman SN, Pronovost P, Tunis S, Beauchamp TL. An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics. Hastings Cent Rep. 2013;43:S16–S27. doi: 10.1002/hast.134. [DOI] [PubMed] [Google Scholar]

[bib27] 27.REMAP-CAP, ACTIV-4a, and ATTACC Investigators Therapeutic anticoagulation with heparin in critically ill patients with Covid-19. N Engl J Med. 2021;385:777–789. doi: 10.1056/NEJMoa2103417. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib28] 28.Koyfman SA, Reddy CA, Hizlan S, Leek AC, Kodish AE. Informed consent conversations and documents: A quantitative comparison. Cancer. 2016;122:464–469. doi: 10.1002/cncr.29759. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib29] 29.Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637–648. doi: 10.1016/0021-9681(67)90041-0. [DOI] [PubMed] [Google Scholar]

[bib30] 30.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350 doi: 10.1136/bmj.h2147. [DOI] [PubMed] [Google Scholar]

[bib31] 31.RECOVERY Collaborative Group Lopinavir–ritonavir in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2020;396:1345–1352. doi: 10.1016/S0140-6736(20)32013-4. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib32] 32.Abaleke E, Abbas M, Abbasi S, et al. Azithromycin in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397:605–612. doi: 10.1016/S0140-6736(21)00149-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib33] 33.Semler MW, Self WH, Wanderer JP, et al. Balanced crystalloids versus saline in critically ill adults. N Engl J Med. 2018;378:829–839. doi: 10.1056/NEJMoa1711584. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib34] 34.Self WH, Semler MW, Wanderer JP, et al. Balanced crystalloids versus saline in noncritically ill adults. N Engl J Med. 2018;378:819–828. doi: 10.1056/NEJMoa1711586. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib35] 35.Shah ASV, Anand A, Strachan FE, et al. High-sensitivity troponin in the evaluation of patients with suspected acute coronary syndrome: a stepped-wedge, cluster-randomised controlled trial. Lancet. 2018;392:919–928. doi: 10.1016/S0140-6736(18)31923-8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib36] 36.Adler-Milstein J. Moving past the EHR interoperability blame game. NEJM Catalyst. 2017 https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0448 published online July 18. [Google Scholar]

[bib37] 37.Young PJ, Bagshaw SM, Forbes AB, et al. Effect of stress ulcer prophylaxis with proton pump inhibitors vs histamine-2 receptor blockers on in-hospital mortality among ICU patients receiving invasive mechanical ventilation: the PEPTIC randomized clinical trial. JAMA. 2020;323:616–626. doi: 10.1001/jama.2019.22190. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib38] 38.Fröbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med. 2013;369:1587–1597. doi: 10.1056/NEJMoa1308789. [DOI] [PubMed] [Google Scholar]

[bib39] 39.Marquis-Gravel G, Roe MT, Robertson HR, et al. Rationale and design of the aspirin dosing—a patient-centric trial assessing benefits and long-term effectiveness (ADAPTABLE) trial. JAMA Cardiol. 2020;5:598–607. doi: 10.1001/jamacardio.2020.0116. [DOI] [PubMed] [Google Scholar]

[bib40] 40.Matthay MA, Luetkemeyer AF. IL-6 receptor antagonist therapy for patients hospitalized for COVID-19: who, when, and how? JAMA. 2021;326:483–485. doi: 10.1001/jama.2021.11121. [DOI] [PubMed] [Google Scholar]

[bib41] 41.Acute Respiratory Distress Syndrome Network Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med. 2000;342:1301–1308. doi: 10.1056/NEJM200005043421801. [DOI] [PubMed] [Google Scholar]

[bib42] 42.Ranieri VM, Thompson BT, Barie PS, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med. 2012;366:2055–2064. doi: 10.1056/NEJMoa1202290. [DOI] [PubMed] [Google Scholar]

[bib43] 43.National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network Comparison of two fluid-management strategies in acute lung injury. N Engl J Med. 2006;354:2564–2575. doi: 10.1056/NEJMoa062200. [DOI] [PubMed] [Google Scholar]

[bib44] 44.Roberts I, Yates D, Sandercock P, et al. Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet. 2004;364:1321–1328. doi: 10.1016/S0140-6736(04)17188-2. [DOI] [PubMed] [Google Scholar]

[bib45] 45.Janz DR, Semler MW, Lentz RJ, et al. Randomized trial of video laryngoscopy for endotracheal intubation of critically ill adults. Crit Care Med. 2016;44:1980–1987. doi: 10.1097/CCM.0000000000001841. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib46] 46.Semler MW, Janz DR, Russell DW, et al. A multicenter, randomized trial of ramped position vs sniffing position during endotracheal intubation of critically ill adults. Chest. 2017;152:712–722. [Google Scholar]

[bib47] 47.Casey JD, Janz DR, Russell DW, et al. Bag-mask ventilation during tracheal intubation of critically ill adults. N Engl J Med. 2019;380:811–821. doi: 10.1056/NEJMoa1812405. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib48] 48.Driver BE, Prekker ME, Klein LR, et al. Effect of use of a bougie vs endotracheal tube and stylet on first-attempt intubation success among patients with difficult airways undergoing emergency intubation: a randomized clinical trial. JAMA. 2018;319:2179–2189. doi: 10.1001/jama.2018.6496. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib49] 49.Caputo N, Azan B, Domingues R, et al. Emergency department use of apneic oxygenation versus usual care during rapid sequence intubation: a randomized controlled trial (the ENDAO trial) Acad Emerg Med. 2017;24:1387–1394. doi: 10.1111/acem.13274. [DOI] [PubMed] [Google Scholar]

[bib50] 50.Semler MW, Wanderer JP, Ehrenfeld JM, et al. Balanced crystalloids versus saline in the intensive care unit. The SALT randomized trial. Am J Respir Crit Care Med. 2017;195:1362–1372. doi: 10.1164/rccm.201607-1345OC. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib51] 51.Wilson FP, Martin M, Yamamoto Y, et al. Electronic health record alerts for acute kidney injury: multicenter, randomized clinical trial. BMJ. 2021;372 doi: 10.1136/bmj.m4786. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib52] 52.Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283:2701–2711. doi: 10.1001/jama.283.20.2701. [DOI] [PubMed] [Google Scholar]

[bib53] 53.Nuremberg Military Tribunal The Nuremberg Code. JAMA. 1996;276 [PubMed] [Google Scholar]

[bib54] 54.World Medical Association declaration of Helsinki Recommendations guiding physicians in biomedical research involving human subjects. JAMA. 1997;277:925–926. [PubMed] [Google Scholar]

[bib55] 55.US Department of Health and Human Services Office for Human Research Protections The Belmont report. Ethical principles and guidelines for the protection of human subjects of research. April 18, 1979. https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html

[bib56] 56.Hillier TA, Pedula KL, Ogasawara KK, et al. A pragmatic, randomized clinical trial of gestational diabetes screening. N Engl J Med. 2021;384:895–904. doi: 10.1056/NEJMoa2026028. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib57] 57.US Department of Health and Human Services Office for Human Research Protections 45 CFR 46. Feb 16, 2016. https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html

[bib58] 58.US Food and Drug Administration Exception from informed consent requirements for emergency research. April 2013. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/exception-informed-consent-requirements-emergency-research

[bib59] 59.Feldman WB, Hey SP, Kesselheim AS. A systematic review of the Food And Drug Administration's ‘exception from informed consent’ pathway. Health Aff (Millwood) 2018;37:1605–1614. doi: 10.1377/hlthaff.2018.0501. [DOI] [PubMed] [Google Scholar]

[bib60] 60.Klein L, Moore J, Biros M. A 20-year review: the use of exception from informed consent and waiver of informed consent in emergency research. Acad Emerg Med. 2018;25:1169–1177. doi: 10.1111/acem.13438. [DOI] [PubMed] [Google Scholar]

[bib61] 61.Feldman WB, Hey SP, Franklin JM, Kesselheim AS. Public approval of exception from informed consent in emergency clinical trials: a systematic review of community consultation surveys. JAMA Netw Open. 2019;2 doi: 10.1001/jamanetworkopen.2019.7591. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib62] 62.Faden RR, Beauchamp TL, Kass NE. Informed consent, comparative effectiveness, and learning health care. N Engl J Med. 2014;370:766–768. doi: 10.1056/NEJMhle1313674. [DOI] [PubMed] [Google Scholar]

[bib63] 63.Kass N, Faden R, Tunis S. Addressing low-risk comparative effectiveness research in proposed changes to US federal regulations governing research. JAMA. 2012;307:1589–1590. doi: 10.1001/jama.2012.491. [DOI] [PubMed] [Google Scholar]

[bib64] 64.Faden RR, Beauchamp TL, Kass NE. Informed consent for comparative effectiveness trials. N Engl J Med. 2014;370:1959–1960. doi: 10.1056/NEJMc1403310. [DOI] [PubMed] [Google Scholar]

[bib65] 65.Morain SR, Tambor E, Moloney R, et al. Stakeholder perspectives regarding alternate approaches to informed consent for comparative effectiveness research. Learn Health Syst. 2017;2 doi: 10.1002/lrh2.10047. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib66] 66.Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–926. doi: 10.1136/bmj.39489.470347.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib67] 67.Platt R, Kass NE, McGraw D. Ethics, regulation, and comparative effectiveness research: time for a change. JAMA. 2014;311:1497–1498. doi: 10.1001/jama.2014.2144. [DOI] [PubMed] [Google Scholar]

[bib68] 68.Lantos JD, Wendler D, Septimus E, Wahba S, Madigan R, Bliss G. Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials. Clin Trials. 2015;12:485–493. doi: 10.1177/1740774515597687. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib69] 69.Goldstein CE, Weijer C, Brehaut JC, et al. Ethical issues in pragmatic randomized controlled trials: a review of the recent literature identifies gaps in ethical argumentation. BMC Med Ethics. 2018;19:14. doi: 10.1186/s12910-018-0253-x. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib70] 70.Thompson BT, Schoenfeld D. Usual care as the control group in clinical trials of nonpharmacologic interventions. Proc Am Thorac Soc. 2007;4:577–582. doi: 10.1513/pats.200706-072JK. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib71] 71.Morain S, O'Rourke P, for the Ethics and Regulatory Core. NIH Health Care Systems Research Collaboratory Data monitoring in pragmatic clinical trials: points to consider. June 28, 2021. https://dcricollab.dcri.duke.edu/sites/NIHKR/KR/PCT%20Data%20Monitoring%20Committees-Points%20to%20Consider.pdf

[bib72] 72.Self WH, Semler MW, Leither LM, et al. Effect of hydroxychloroquine on clinical status at 14 days in hospitalized patients with COVID-19: a randomized clinical trial. JAMA. 2020;324:2165–2176. doi: 10.1001/jama.2020.22240. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib73] 73.Statista Cumulative number of coronavirus (COVID-19) cases in the United Kingdom (UK) since. January 2020. https://www.statista.com/statistics/1101958/cumulative-coronavirus-cases-in-the-uk/

[bib74] 74.Centers for Disease Control and Prevention COVID data tracker. March 28, 2020. https://covid.cdc.gov/covid-data-tracker

[bib75] 75.Casey JD, Johnson NJ, Semler MW, et al. Rationale and design of ORCHID: a randomized placebo-controlled clinical trial of hydroxychloroquine for adults hospitalized with COVID-19. Ann Am Thorac Soc. 2020;17:1144–1153. doi: 10.1513/AnnalsATS.202005-478SD. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib76] 76.Bull-Otterson L, Gray EB, Budnitz DS, et al. Hydroxychloroquine and chloroquine prescribing patterns by provider specialty following initial reports of potential benefit for COVID-19 treatment — United States, January–June 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1210–1215. doi: 10.15585/mmwr.mm6935a4. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib77] 77.National Institute for Health Research UK's Chief Medical Officers urge hospitals to recruit 60% of eligible COVID-19 patients into RECOVERY trial. Aug 10, 2020. https://www.nihr.ac.uk/news/uks-chief-medical-officers-urge-hospitals-to-recruit-60-of-eligible-covid-19-patients-into-recovery-trial/25515

[bib78] 78.Horby P, Mafham M, Linsell L, et al. Effect of hydroxychloroquine in hospitalized patients with Covid-19. N Engl J Med. 2020;383:2030–2040. doi: 10.1056/NEJMoa2022926. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib79] 79.Financial Times Coronavirus tracked: see how your country compares. https://ig.ft.com/coronavirus-chart

[bib80] 80.Cavalcanti AB, Zampieri FG, Rosa RG, et al. Hydroxychloroquine with or without azithromycin in mild-to-moderate Covid-19. N Engl J Med. 2020;383:2041–2052. doi: 10.1056/NEJMoa2019014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib81] 81.WHO Solidarity Trial Consortium Repurposed antiviral drugs for Covid-19 — interim WHO Solidarity trial results. N Engl J Med. 2021;384:497–511. doi: 10.1056/NEJMoa2023184. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib82] 82.Ulrich RJ, Troxel AB, Carmody E, et al. Treating COVID-19 With Hydroxychloroquine (TEACH): a multicenter, double-blind randomized controlled trial in hospitalized patients. Open Forum Infect Dis. 2020;7 doi: 10.1093/ofid/ofaa446. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib83] 83.Medicines and Healthcare products Regulatory Agency MHRA suspends recruitment to COVID-19 hydroxychloroquine trials. June 16, 2020. https://www.gov.uk/government/news/mhra-suspends-recruitment-to-covid-19-hydroxychloroquine-trials

[bib84] 84.Dequin P-F, Heming N, Meziani F, et al. Effect of hydrocortisone on 21-day mortality or respiratory support among critically ill patients with COVID-19: a randomized clinical trial. JAMA. 2020;324:1298–1306. doi: 10.1001/jama.2020.16761. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib85] 85.Tomazini BM, Maia IS, Cavalcanti AB, et al. Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX randomized clinical trial. JAMA. 2020;324:1307–1316. doi: 10.1001/jama.2020.17021. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib86] 86.The WHO Rapid Evidence Appraisal for COVID-19 Therapies REACT Working Group. Sterne JAC, Murthy S, et al. Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis. JAMA. 2020;324:1330–1341. doi: 10.1001/jama.2020.17023. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib87] 87.WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis. JAMA. 2020;324:1330–1341. doi: 10.1001/jama.2020.17023. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib88] 88.Angus DC, Derde L, Al-Beidh F, et al. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: the REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA. 2020;324:1317–1329. doi: 10.1001/jama.2020.17022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib89] 89.US Food and Drug Administration Emergency use authorization for use of chloroquine phosphate or hydroxychloroquine sulfate supplied from the strategic national stockpile for treatment of 2019 coronavirus disease. March 28, 2020. https://www.fda.gov/media/136534/download

[bib90] 90.Gupta S, Hayek SS, Wang W, et al. Factors associated with death in critically ill patients with coronavirus disease 2019 in the US. JAMA Intern Med. 2020;180:1436–1447. doi: 10.1001/jamainternmed.2020.3596. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib91] 91.Abani O, Abbas A, Abbas F, et al. Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised controlled, open-label, platform trial. Lancet. 2021;397:2049–2059. doi: 10.1016/S0140-6736(21)00897-7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib92] 92.Emanuel EJ, Boyle CW. Assessment of length and readability of informed consent documents for COVID-19 vaccine trials. JAMA Netw Open. 2021;4 doi: 10.1001/jamanetworkopen.2021.10843. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib93] 93.Flory J, Emanuel E. Interventions to improve research participants' understanding in informed consent for research: a systematic review. JAMA. 2004;292:1593–1601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]

[bib94] 94.Kimmelman J. The therapeutic misconception at 25: treatment, research, and confusion. Hastings Cent Rep. 2007;37:36–42. doi: 10.1353/hcr.2007.0092. [DOI] [PubMed] [Google Scholar]

[bib95] 95.Davidson J, Banerjee A, Muzambi R, Smeeth L, Warren-Gash C. Validity of acute cardiovascular outcome diagnoses recorded in European electronic health records: a systematic review. Clin Epidemiol. 2020;12:1095–1111. doi: 10.2147/CLEP.S265619. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib96] 96.McKown AC, Brown RM, Ware LB, Wanderer JP. External validity of electronic sniffers for automated recognition of acute respiratory distress syndrome. J Intensive Care Med. 2019;34:946–954. doi: 10.1177/0885066617720159. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib97] 97.Day SJ, Altman DG. Statistics notes: blinding in clinical trials and other studies. BMJ. 2000;321:504. doi: 10.1136/bmj.321.7259.504. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib98] 98.Rice TW, Kripalani S, Lindsell CJ. Proton pump inhibitors vs histamine-2 receptor blockers for stress ulcer prophylaxis in critically ill patients: issues of interpretability in pragmatic trials. JAMA. 2020;323:611–613. doi: 10.1001/jama.2019.22436. [DOI] [PubMed] [Google Scholar]

[bib99] 99.RECOVERY Collaborative Group Casirivimab and imdevimab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2022;399:665–676. doi: 10.1016/S0140-6736(22)00163-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib100] 100.Platt R, Simon GE, Hernandez AF. Is learning worth the trouble? — Improving health care system participation in embedded research. N Engl J Med. 2021;385:5–7. doi: 10.1056/NEJMp2101700. [DOI] [PubMed] [Google Scholar]

PERMALINK

Use of pragmatic and explanatory trial designs in acute care research: lessons from COVID-19

Jonathan D Casey, MD

Laura M Beskow, Prof, PhD

Jeremy Brown, MD

Samuel M Brown, Prof, MD

Étienne Gayat, Prof, MD

Michelle Ng Gong, Prof, MD

Michael O Harhay, PhD

Samir Jaber, Prof, MD

Jacob C Jentzer, MD

Pierre-François Laterre, Prof, MD

John C Marshall, Prof, MD

Michael A Matthay, Prof, MD

Todd W Rice, MD

Yves Rosenberg, MD

Alison E Turnbull, PhD

Lorraine B Ware, Prof, MD

Wesley H Self, MD

Alexandre Mebazaa, Prof, MD

Sean P Collins, Prof, MD

Abstract

Introduction

Key messages.

Challenges for acute care research

Explanatory trials

Assessment of novel therapies

Assessment of clinically available therapies

Pragmatic trials

Increased generalisability and efficiency

Table 1.

Figure 1.

Table 2.

Implications for human research protections

Lessons from trials of hydroxychloroquine for COVID-19

Table 3.

Figure 2.

Figure 3.

Unresolved questions for pragmatic trials

Recommendations for acute care research

Table 4.

Table 5.

Conclusions

Search strategy and selection criteria

Declaration of interests

Acknowledgments

Acknowledgments

Contributors

References

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases