Randomized Trials Versus Common Sense and Clinical Observation

Abstract

Concerns about the external validity of traditional randomized clinical trials (RCTs), together with the widespread availability of real-world data and advanced data analytic tools, have led to claims that common sense and clinical observation, rather than RCTs, should be the preferred method to generate evidence to support clinical decision-making. However, over the past 4 decades, results from well-done RCTs have repeatedly contradicted practices supported by common sense and clinical observation. Common sense and clinical observation fail for several reasons: incomplete understanding of pathophysiology, biases and unmeasured confounding in observational research, and failure to understand risks and benefits of treatments within complex systems. Concerns about traditional RCT models are legitimate, but randomization remains a critical tool to understand the causal relationship between treatments and outcomes. Instead, development and promulgation of tools to apply randomization to real-world data are needed to build the best evidence base in cardiovascular medicine.

Central Illustration

The digitization of health care has led to the proliferation of “real-world data”—data collected for nonresearch purposes, like patient care or billing—and observational studies based on these data (1). Expansion of statistical techniques for the analysis of observational data, including the application of machine learning to health care, has accelerated the growth of observational real-world data analyses (2). At the same time, the growing complexity and cost of traditional randomized clinical trials (RCTs) and a focus by many academic and community health care systems on maximizing clinical volume at the expense of research participation has led to RCTs that enroll selected patients at selected centers (3,4). These trends have converged in concerns that RCTs have become too complex and selective to be the preferred method to generate evidence to support clinical decision-making (5, 6, 7).

Despite these arguments, RCTs remain the best current method to understand the causal relationship between an intervention and subsequent population-level outcomes for most common chronic illnesses (8). In the absence of data from RCTs, patients, their families, clinicians, payers, and health systems are forced to rely on “common sense” and observation, defined in this review as the application of anatomy, physiology, pathology, and pharmacology to a complex clinical problem, supplemented by personal clinical experience or aggregated clinical experience in the form of observational studies. However, most cardiovascular diseases develop from the interplay of genetics, cellular and molecular biology, physiology, behavioral health, environmental exposures, social determinants of health, and health care delivery systems. A patient’s individual response to any intervention will be similarly dependent on a multitude of factors. With such complexity, estimating the balance of benefits and risks on overall clinical outcome is a profound challenge. This incomplete understanding, combined with inability to measure all determinants of outcome, emerges in observational studies as bias and confounding (9). For common, chronic diseases, this complexity is further complicated by the moderate effect sizes of individual treatments, which can be overwhelmed by unmeasured factors in observational studies (10). Even the most sophisticated adjustment techniques, including artificial intelligence and machine learning, cannot adjust for unmeasured factors (11). The problem of unmeasured confounding in observational research is not resolved by larger data sources, which only increase the precision of biased (and incorrect) estimates of treatment effect. The infrequency with which promising molecular entities demonstrate clear clinical benefit and reach the market shows the limitations of common sense: Only 7% of cardiovascular drugs entering clinical testing between 1999 and 2004 and 42% of those that reached Phase 3 trials ultimately reached the marketplace (12,13).

Robust regulatory oversight often prevents novel pharmaceutical agents from reaching the market without well-conducted RCTs. However, regulatory oversight does not protect patients from common sense–based off-label use of pharmaceutical agents, and patients have even less protection from other classes of intervention. Device regulation is less stringent than drug regulation, and behavioral interventions, health system interventions, and novel payment reforms aimed at health care delivery have little to no regulatory oversight to drive high-quality evidence generation. There are many examples of RCTs overturning decades of accumulated medical wisdom built on common sense and observation (Table 1 ). Examined critically, the results of these RCTs highlight several reasons that “common sense” and observation often fail in their assessment of human therapeutics (Central Illustration ).

Table 1.

Common Sense and Observational Findings Versus Clinical Trials

	Common Sense or Observational Findings	Clinical Trial and Results	Reason for Common Sense Failure
Suppressing PVCs after MI with Class 1 antiarrhythmic agents	↓ mortality	CAST: ↑ mortality	Marker of risk, not target; incomplete understanding of pharmacological agent in complex system
Opening occluded arteries late after MI presentation	↓ mortality	OAT: ↔ CV events or mortality	Marker of risk, not target
Increasing HDL-C pharmacologically	↓ CV events	ACCORD, ACCELERATE, ILLUMINATE, dal-OUTCOMES, HPS2-THRIVE: ↑ or ↔ in CV events	Marker of risk, not target
Revascularizing ischemic myocardium	↓ death/MI	COURAGE, BARI 2D, ISCHEMIA: ↔ death/MI	Marker of risk, not target
Ventricular reconstruction in ischemic cardiomyopathy	↓ death/hospitalization	STICH: ↔ death/hospitalization, ↔ quality of life	Marker of risk, not target
Mitral valve surgery in ischemic mitral regurgitation	↓ death/hospitalization	CTSN: ↔ ventricular size, ↔ death, ↔ hospitalization	Marker of risk, not target
Avoidance of CABG in ischemic cardiomyopathy without myocardial viability	Myocardial viability mediates response to myocardial revascularization	STICH: No interaction between myocardial viability and coronary artery bypass graft outcomes	Marker of risk, not target
Erythropoietin analogues for anemia in systolic heart failure	↓ death/hospitalization	RED-HF: ↔ in death/hospitalization	Marker of risk, not target
Erythropoietin analogues for anemia in type 2 diabetes with chronic kidney disease	↓ death, CV events, and renal events	TREAT: ↔ in CV or renal events; ↑ stroke	Marker of risk, not target
Strict rate control in atrial fibrillation	↓ CV and bleeding events	RACE II: ↔ in CV or bleeding events	Marker of risk, not target
Intensive blood pressure control in type 2 diabetes mellitus	↓ CV events	ACCORD: ↔ in CV events	Failure to understand balance of risks and benefits in complex disease process
Intensive glycemic control type 2 diabetes mellitus	↓ CV events	ACCORD: ↓ MI, ↑ mortality	Failure to understand balance of risks and benefits in complex disease process
Complete revascularization in STEMI and cardiogenic shock	↓ death	CULPRIT-SHOCK: ↑ death, ↑ renal failure	Failure to understand balance of risks and benefits in complex disease process
Intra-aortic balloon pump in cardiogenic shock	↓ death	IABP-SHOCK II: ↔ death	Surrogate measures do not translate to clinical outcomes
Percutaneous axial flow pump in high-risk PCI	↓ CV events	PROTECT II: ↔ CV events	Surrogate measures do not translate to clinical outcomes
Milrinone in severe symptomatic heart failure	↓ death and heart failure hospitalizations	PROMISE: ↑ death, ↑ heart failure hospitalizations	Surrogate measures do not translate to clinical outcomes
Rhythm control in atrial fibrillation	↓ mortality	AFFIRM: ↔ mortality; ↑ hospitalization	Surrogate measures do not translate to clinical outcomes
Rhythm control in atrial fibrillation and congestive heart failure	↓ mortality, ↑ quality of life	AF-CHF: ↔ mortality, ↔ quality of life	Surrogate measures do not translate to clinical outcomes
Routine thrombus aspiration in STEMI	↓ CV events	TASTE, TOTAL: ↔ CV events	Surrogate measures do not translate to clinical outcomes
Anticoagulation after TAVR	↓ leaflet thrombosis and CV events	GALLILEO: ↓ leaflet thrombosis; ↔ CV events	Surrogate measures do not translate to clinical outcomes
Vitamin C supplementation	↓ CV events	PHS II: ↔ CV events	Healthy user bias
Vitamin E supplementation	↓ CV events	HOPE, PHS II: ↔ CV events	Healthy user bias
Vitamin D supplementation	↓ CV events	VITAL: ↔ CV events	Healthy user bias
Folate supplementation	↓ CV events	HOPE 2, NORVIT: ↔ CV events	Healthy user bias
Vitamin B6 supplementation	↓ CV events	HOPE 2, NORVIT: ↔ CV events	Healthy user bias
Vitamin B12 supplementation	↓ CV events	HOPE 2, NORVIT: ↔ CV events	Healthy user bias
Multivitamin supplementation	↓ CV events	PHS II: ↔ CV events	Healthy user bias
Hormonal therapy in perimenopausal women	↓ CV events	WHI: ↑ CV events	Healthy user bias
Multivessel revascularization in STEMI patients	↑ mortality	COMPLETE: ↓ death/MI	Confounding by indication
Stroke prevention in atrial fibrillation	↓ ischemic stroke, ↑ hemorrhagic stroke	RE-LY: ↓ ischemic stroke, ↓ hemorrhagic stroke ARISTOTLE, ROCKET AF, ENGAGE-AF: ↔ ischemic stroke, ↓ hemorrhagic stroke	Incomplete understanding of therapeutic mechanism
Glucose lowering therapy in patients with heart failure and no diabetes	↔ CV events	DAPA-HF: ↓ death/worsening heart failure	Incomplete understanding of therapeutic mechanism

↑ = increased; ↓ = decreased; ↔ = unchanged; ACCELERATE = Assessment of Clinical Effects of Cholesteryl Ester Transfer Protein Inhibition with Evacetrapib in Patients at a High Risk for Vascular Outcomes; ACCORD = Action to Control Cardiovascular Risk in Diabetes; AF-CHF = Atrial Fibrillation and Congestive Heart Failure; AFFIRM = Atrial Fibrillation Follow-up Investigation of Rhythm Management; ARISTOTLE = Apixaban for Reduction in Stroke and Other Thromboembolic Events in Atrial Fibrillation; BARI 2D = Bypass Angioplasty Revascularization Investigation 2 Diabetes; CABG = coronary artery bypass graft; CAST = Cardiac Arrhythmia Suppression Trial; COMPLETE = Complete versus Culprit-Only Revascularization Strategies to Treat Multivessel Disease after Early PCI for STEMI; COURAGE = Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation; CTSN = Cardiothoracic Surgical Trials Network; CUPRIT-SHOCK = Culprit Lesion Only PCI versus Multivessel PCI in Cardiogenic Shock; CV = cardiovascular; dal-OUTCOMES = A Randomized, Double-blind, Placebo-controlled Study Assessing the Effect of RO4607381 on Cardiovascular Mortality and Morbidity in Clinically Stable Patients With a Recent Acute Coronary Syndrome; DAPA-HF = Dapagliflozin and Prevention of Adverse Outcomes in Heart Failure; ENGAGE AF = Effective Anticoagulation with Factor Xa Next Generation in Atrial Fibrillation; GALLILEO = Global Study Comparing a Rivaroxaban-based Antithrombotic Strategy to an Antiplatelet-based Strategy after Transcatheter Aortic Valve Replacement to Optimize Clinical Outcomes; HDL-C = high-density lipoprotein cholesterol; HOPE = Heart Outcomes Prevention Evaluation; HPS2-THRIVE = Heart Protection Study 2 -Treatment of HDL to Reduce the Incidence of Vascular Events; IABP-SHOCK II = Intraaortic Balloon Pump in Cardiogenic Shock II; ILLUMINATE = Investigation of Lipid Level Management to Understand its Impact in Atherosclerotic Events; ISCHEMIA = International Study of Comparative Health Effectiveness with Medical and Invasive Approaches; MI = myocardial infarction; NORVIT = Norwegian Vitamin; OAT = Occluded Artery Trial; PCI = percutaneous coronary intervention; PHS II = Physicians’ Health Study II; PROMISE = Prospective Randomized Milrinone Survival Evaluation; PROTECT II = A Prospective, Multi-center, Randomized Controlled Trial of the IMPELLA RECOVER LP 2.5 System Versus Intra Aortic Balloon Pump in Patients Undergoing Non Emergent High Risk PCI; PVC = premature ventricular contraction; RACE II = Rate Control Efficacy in Permanent Atrial Fibrillation: a Comparison between Lenient versus Strict Rate Control II; RED-HF = Reduction of Events by Darbepoetin Alfa in Heart Failure; RE-LY = Randomized Evaluation of Long-Term Anticoagulation Therapy; ROCKET AF = Rivaroxaban Once Daily Oral Direct Factor Xa Inhibition Compared with Vitamin K Antagonism for Prevention of Stroke and Embolism Trial in Atrial Fibrillation; STEMI = ST-segment elevation myocardial infarction; STICH = Surgical Treatment for Ischemic Heart Failure; TASTE = Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia; TAVR = transcatheter aortic valve replacement; TOTAL = Trial of Routine Aspiration Thrombectomy with PCI versus PCI Alone in Patients with STEMI; TREAT = Trial to Reduce Cardiovascular Events with Aranesp Therapy; VITAL = Vitamin D and Omega-3 Trial; WHI = Women's Health Initiative.

Central Illustration.

Randomization Is Critical for Understanding Treatment Effect

Common sense and clinical observation may fail to accurately describe the effect of a treatment on outcomes for multiple reasons. Randomization bypasses these failure mechanisms, creating a controlled experiment to understand the true effect of a treatment on outcomes. When common sense and clinical observation are the basis for treatment decisions, there is great uncertainty about benefit and risk, such that treatments may be ineffective and/or put the patient at risk. In contrast, treatment decisions based on high-quality randomized controlled trials have well-defined benefit and risk profiles allowing for effective decision-making.

This review might be particularly relevant in the context of the current global coronavirus disease-2019 (COVID-19) pandemic. The urgent need for an effective treatment for COVID-19 has led to a flurry of proposed therapies with mechanistically plausible (“common sense”) potential benefits, with some arguing that RCTs will result in unnecessary and harmful delays to delivering these treatments in clinical practice. Our review underscores the flaws in such reasoning and highlights the limitations of nonrandomized studies for identifying treatment effects. Now, more than ever, we need timely, high-quality evidence from adequately powered RCTs with clinically relevant endpoints to determine how to best to treat patients with COVID-19.

Mechanisms of Common Sense and Observation Failure

Incomplete understanding of pathophysiology

Common sense may fail because of an incomplete understanding of pathophysiology. One subcategory of this type of failure occurs when a clinical observation merely represents a marker of risk, and not a modifiable target. In the 1970s, investigators identified an association between premature ventricular contractions and mortality following myocardial infarction (MI) (14), and cardiologists routinely used antiarrhythmic drugs to treat patients with nonsustained ventricular tachycardia (15). When the hypothesis that treating ventricular ectopy with Class I antiarrhythmic drugs would improve outcomes was tested in the randomized controlled CAST (Cardiac Arrhythmia Suppression Trial), patients assigned to antiarrhythmic drugs had higher mortality than those assigned to placebo (16). The CAST experience was a dramatic example of this type of failure of common sense, but it is far from the only one.

In the 1990s, observational studies suggested that patients with a patent infarct-related artery after MI had better survival than those who did not (17), but the OAT (Occluded Artery Trial) RCT showed that percutaneous revascularization of an occluded infarct-related artery 3 to 28 days after MI did not reduce cardiovascular events over long-term follow-up (18). Similarly, epidemiological studies demonstrated an association between low levels of high-density lipoprotein cholesterol (HDL-C) and cardiovascular mortality (19), but in RCTs, multiple HDL-C–raising drugs did not reduce cardiovascular events (20, 21, 22, 23, 24). Another example is myocardial ischemia, which has been associated with increased risk of mortality and MI in observational studies (25), but multiple RCTs have shown that relieving ischemia by revascularization does not reduce the risk of mortality or MI (26, 27, 28). Similarly, anemia has been associated with poor outcomes in patients with heart failure and type 2 diabetes with chronic kidney disease (29,30), but randomized controlled trials of erythropoietin analogues in these populations have failed to improve clinical outcomes (31,32). Higher heart rates in patients with atrial fibrillation are associated with worse outcomes (33), but tighter heart rate control did not translate into improved outcomes in an RCT (34). In cardiothoracic surgery, high-quality RCTs have challenged common-sense therapies that relied on modifying disease markers ultimately shown not to be modifiable targets, including ventricular reconstruction to reduce ventricular size in ischemic cardiomyopathy (35), mitral valve surgery to reduce ischemic mitral regurgitation (36), and avoidance of coronary artery bypass grafting in ischemic cardiomyopathy patients without myocardial viability (37).

Another subcategory of incomplete understanding of pathophysiology occurs when a modifiable target that is put forward as a putative surrogate outcome, such as acute improvements in hemodynamics or imaging parameters, fails to translate into clinical benefits. In the 1970s, the intra-aortic balloon pump (IABP) was shown in observational studies to (modestly) augment cardiac output in patients with cardiogenic shock (38); however, when patients with cardiogenic shock were randomized to IABP or usual care, the IABP did not reduce mortality nor improve quality of life in survivors (39). A percutaneously inserted axial flow pump supports cardiac output more than the IABP (40), but did not improve outcomes in a small RCT of patients undergoing high-risk percutaneous coronary intervention, although it did raise vascular and bleeding complications (41). In patients with heart failure with reduced ejection fraction, oral milrinone increased ejection fraction while also increasing mortality in the PROMISE (Prospective Randomized Milrinone Survival Evaluation) RCT, similar to the results of smaller studies with other intravenous and oral inotropes (42). In the AFFIRM (Atrial Fibrillation Follow-up Investigation of Rhythm Management) RCT, a rhythm-control strategy for atrial fibrillation using cardioversion and/or antiarrhythmic drugs reduced the time patients spent in atrial fibrillation, but did not affect mortality, increased hospitalization, and did not improve quality of life (43,44), a pattern that repeated in an RCT enrolling patients with atrial fibrillation and heart failure (45). Rivaroxaban, compared with placebo, reduced subclinical leaflet thrombosis in patients undergoing transcatheter aortic valve replacement (46), but this imaging surrogate did not translate into improved clinical outcomes (47). Coronary thrombus aspiration in patients with ST-segment elevation MI improves myocardial perfusion and ST-segment elevation (48), but these surrogate benefits did not translate into improvement in outcomes in 2 high-quality RCTs (49,50).

Although biomarkers and surrogates are essential in the rational development of therapies, the complexity of biology, behavior, environment, and social interaction necessarily limits the ability of single or even multiple measures to reliably predict the holistic effect of an intervention on health outcomes. Because of the multidimensional nature of human biology, any individual marker can only predict a portion of expected outcomes related to the pathway in which it plays a role, leaving other causal pathways and off-target effects unmeasured (51,52). In diseases with poor outcomes and ineffective treatments, or for certain conditions with highly validated surrogates (such as blood pressure control), accelerated pathways have been developed to enable early access to market, but post-market RCTs with clinical endpoints are needed to provide robust evidence.

Biases and unmeasured confounding

Common sense may also fail because of biases and unmeasured confounding inherent to observational research. One important bias is healthy user bias. Observational studies showed associations among folate; vitamins B6, B12, C, D, and E; and multivitamin supplementation with lower cardiovascular mortality (53, 54, 55, 56, 57, 58), but in large, well-conducted RCTs, supplements failed to improve cardiovascular outcomes (59, 60, 61, 62, 63, 64). Similarly, hormone replacement therapy with estrogen and progesterone in perimenopausal women was associated with a lower incidence of cardiovascular events in observational studies (65), but increased risk in large RCTs (66). Among the reasons why RCTs failed to replicate the results of the observational studies might be that patients who took vitamins or other supplements tended to be healthier than those who did not, including in ways that are not measured in most databases (67).

Another important bias is confounding by indication. Revascularization of nonculprit arteries in patients presenting with ST-segment elevation MI was a Class III (“should not do”) recommendation in consensus guidelines on the basis of observational studies showing an increase in mortality with this strategy (68), but a large, well-conducted RCT showed a large reduction in cardiovascular events with complete revascularization versus culprit-only revascularization (69). With hindsight, it is clear that observational studies conducted in this area were confounded by the selective performance of multivessel revascularization in higher-risk patients than culprit-lesion revascularization, with high-risk features incompletely characterized and adjusted for.

Incomplete understanding of balance of benefits and risks in complex systems

In complex disease processes involving multiple organ systems, an incomplete understanding of the balance of risks and benefits may cause common sense to fail. In patients with diabetes mellitus, observational studies, consistent with common sense, showed associations between higher blood pressure and worse glycemic control and worse cardiovascular outcomes (70,71). However, when patients with diabetes mellitus were randomized to intensive (<120 mm Hg systolic) or standard (<140 mm Hg systolic) blood pressure control and to intensive (hemoglobin A1c <6%) or standard (hemoglobin A1c 7% to 7.9%) glycemic control, intensive blood pressure control did not reduce the risk of cardiovascular events, and intensive glycemic control lowered the risk of MI but increased the risk of mortality (72,73). Both intensive treatment strategies caused a higher likelihood of adverse events compared with standard therapies.

Common sense also may fail because of the inability of common sense to fully comprehend the interaction between a pharmacological agent and a disease process within the context of a complex biopsychosocial system. CAST not only demonstrated the falsity of the premature ventricular contraction suppression hypothesis, but also uncovered previously unrecognized off-target harms from Class I antiarrhythmic agents (16). Similarly, the cholesterylester transfer protein inhibitor torcetrapib increased cardiovascular events in RCTs despite a large increase in HDL-C, potentially a result of an off-target effect that increased blood pressure (24). In the SP-AF (Stroke Prevention in Atrial Fibrillation) I and II trials, antithrombotic therapy, compared with placebo, reduced ischemic strokes while increasing hemorrhagic stroke, and warfarin, compared with aspirin, did the same (74,75). A common-sense interpretation of this data would suggest that the relationship between ischemic and hemorrhagic strokes was related to the potency of antithrombotic therapy, with increased hemorrhagic stroke an inevitable consequence of efforts to decrease ischemic stroke. However, this common-sense interpretation represented an incomplete understanding of how warfarin contributed to hemorrhagic stroke. Subsequent trials comparing the nonvitamin K antagonist oral anticoagulants with warfarin showed substantial reductions in hemorrhagic stroke with nonvitamin K antagonist oral anticoagulants, with no difference in the rate of ischemic stroke (76).

Sodium-glucose cotransporter 2 (SGLT2) inhibitors were initially approved for use based on RCTs showing that they were modestly effective glucose-lowering agents in patients with type 2 diabetes mellitus. However, the manufacturers of these agents were required by regulatory agencies to perform large RCTs assessing the effect of these agents on cardiovascular outcomes. In the course of these RCTs, there was evidence that SGLT2 inhibitors reduced heart failure hospitalizations (77), and an RCT in patients with heart failure, with or without diabetes mellitus, showed that these agents reduced all-cause death and worsening heart failure (78). There was no compelling common-sense reason to anticipate this benefit, and without pivotal RCTs, the therapeutic potential of SGLT2 inhibitors in heart failure may never have been realized.

The Path Forward: Bringing Randomization to the Real World

The critical need for randomization should not be equated with traditional, regulated RCTs with their bureaucracy and expensive, unnecessary activities (79). Federal legislation, including the Cures Act and User Fee Agreements, specify the use of randomization in the context of streamlined, “real-world” studies as an essential direction, embraced by the U.S. Food and Drug Administration (80). Recent examples have led to regulatory approvals and widespread adoption (81,82), as have “large, simple trials” for decades in cardiovascular drug and device development (83). Other recent examples have highlighted the importance of randomization in health system intervention (84, 85, 86, 87).

Importantly, randomization is not a cure-all. Underpowered RCTs, or those with methodological or operational flaws, might also fail to generate evidence that can reliably be used to guide patient care. For example, both the PRAGUE-18 (Comparison of Prasugrel and Ticagrelor in the Treatment of Acute Myocardial Infarction) and ISAR-REACT 5 (Intracoronary Stenting and Antithrombotic Regimen: Rapid Early Action for Coronary Treatment) trials randomized patients with acute MI to prasugrel or ticagrelor; PRAGUE-18 found no difference between the 2 medications, whereas ISAR-REACT 5 found that treatment with prasugrel reduced the incidence of death, recurrent MI, or stroke by >25% (88,89). Between the 2 trials, there were a total of 353 deaths, recurrent MIs, or strokes, compared with >1,400 such events in each of the pivotal trials that showed the superiority of ticagrelor and prasugrel over clopidogrel. When interpreting such small trials, it can be difficult to know how much the play of chance affects results. Interpretation of trials with operational flaws can similarly be difficult. A meta-analysis of RCTs of paclitaxel-coated balloons for the treatment of peripheral artery disease showed increased mortality, but incomplete follow-up in the component trials and informative censoring has led some to question the results (90,91). The ATLAS 2-ACS (Anti Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects with Acute Coronary Syndromes) RCT, which showed that adding low-dose rivaroxaban to standard therapy after acute coronary syndrome reduced the incidence of ischemic events, did not lead to regulatory approval of low-dose rivaroxaban in the United States, partly due to questions about missing data and unknown vital status (92,93). These examples highlight the importance of conducting not just RCTs but high-quality RCTs. Other design features specific to individual RCTs—including inadequate duration of treatment, incorrect choice of study population or outcome—may result in a failure of RCTs to detect treatment effects that do exist. Furthermore, in some circumstances, including identifying larger effects on very rare outcomes or effects that only emerge with prolonged treatment, observational studies might be preferred over RCTs.

Despite these limitations, randomization remains essential for identifying the effect of a treatment on outcomes in nearly all circumstances. However, the clinical trials ecosystem as currently constituted is incapable of generating sufficient evidence from RCTs to support clinical decision-making (94). Fewer than 15% of European and American cardiology guideline recommendations are supported by evidence from RCTs, a proportion that has not changed over the past decade (95). The development of real-world data sources into research-ready platforms, supported by governmental agencies in multiple countries, has highlighted the value of real-world data in identifying rare adverse events related to drugs and devices (96), but has further underscored the limitations of observational study designs for understanding treatment effects.

The critical next step is to apply randomization to real-world data on a broad scale, harnessing the power of randomization to understand treatment effects, and the power of real-world data to generate large, representative, infinitely reusable study populations (97). To do so will require health systems to reinvest in electronic health records to build systems with increased interoperability, and the ability to identify, randomize, and enroll patients in RCTs and then capture research-quality baseline and follow-up data. Such systems would be strengthened by international collaboration to define data standards to enable multinational trials. Health systems may also need to realign priorities to reduce the emphasis on clinical volume in favor of greater emphasis on research and patient care. Regulatory authorities will need to work with researchers to modify regulations that create red tape and stifle creative RCT design, moving toward centralized institutional review boards and uniform language for contracts in multisite RCTs. Last, regulatory authorities should provide clear guidance to the pharmaceutical and device industries on when real-world data is acceptable for regulatory purposes. Facilitating trials within the real world will require a reimagining of the clinical research enterprise, but the alternative is capitulating and basing treatment decisions on common sense and clinical observation. As the experience of the past 40 years shows, there is no substitution for randomization.