Do we need to reconsider how we design and conduct randomized controlled trials?

This editorial refers to ‘Real-world management and outcomes of 7 million patients with acute coronary syndrome according to clinical research trial enrolment status: A propensity matched analysis’, by Matetic et al. published in this issue of the European Heart Journal—Quality of Care and Clinical Outcomes.

Cardiovascular disease (CVD) is the leading cause of morbidity and mortality worldwide, with increasing incidence due to an ageing population. During the last decades, the outcome for patients with CVD has dramatically improved due to the implementation of novel pharmaceutical agents, interventions, and medical devices shown effective in prospective randomized controlled trials (RCTs).¹ High-quality RCTs play a major role in evidence development and are considered the gold standard in medical research when comparing two treatment strategies.² The beauty of the RCT resides in the randomization process, which allows fair comparison of the relative efficacy and safety of different treatment strategies and minimizes bias and confounding. Based on this background, international guideline committees require two or more adequate well-controlled RCTs to recommend a treatment with the highest level of evidence.³ Unfortunately, only a minority of all pharmaceutical agents, interventions, and medical devices within the field of cardiology can be assigned the highest level of scientific evidence in clinical practice guidelines.⁴

While the methodology of RCTs has its strengths, it also has serious limitations that are sometimes overlooked. Many large-scale RCTs are complex, expensive, time consuming, and in many cases underpowered, with too small patient populations or with surrogate or composite endpoints that might not reflect real-world outcomes. Another major weakness of RCTs is the inclusion of highly selected patients who often do not represent the broad real-world patients whom a therapy might eventually target. For patients with CVD, data from real-world registries continue to show substantial differences in the baseline characteristics and multimorbidity profile between patients enrolled in RCTs and their non-enrolled counterparts.⁵ Despite this, it is generally assumed that findings from RCTs can be extrapolated to the whole spectrum of real-life patients among whom an important subgroup might not have been enrolled although fulfilling the main eligibility criteria. However, if concealed trial selection criteria systematically exclude a subgroup of patients, important information about the efficacy and safety in these patients will be unknown. As a result, pharmaceutical agents, interventions, and medical devices might paradoxically be used in clinical practice outside their evidence-based indication.

In this issue of the European Heart Journal—Quality of Care and Clinical Outcomes, Matetic et al. used a national database to compare care quality and in-hospital outcomes according to evaluation for RCT participation in patients with acute coronary syndrome (ACS). The analysis comprised data for over 7 million hospital admissions with a diagnosis of ACS between 2004 and 2015. Based on the documentation of the International Classification of Diseases, ninth revision (ICD-9), code V70.7 at discharge, 19 684 (0.3%) patients were recorded to have been enrolled into a clinical trial during their hospital stay. The clinical characteristics, treatments, and in-hospital outcomes were compared with one cohort of 3 485 514 (49.2%) admissions fulfilling the eligibility criteria and one cohort of 3 585 980 (50.6%) admissions not fulfilling the eligibility criteria from four landmark ACS trials. Matetic et al. found that the trial cohort was younger, had less comorbidities, and more often underwent invasive interventions such as coronary angiography and percutaneous coronary intervention (PCI). In addition, the trial cohort had lower in-hospital mortality and in-hospital major bleeding events compared with the cohort not fulfilling the eligibility criteria. When compared with the trial cohort, eligible patients not enrolled were less likely to receive coronary angiography and PCI but had similar mortality rates.

Despite the inevitable limitations of a retrospective analysis based on admissions rather than individual patients, the uncertain categorization of the cohorts, and the lack of long-term outcomes, the authors should be commended for highlighting the problems that RCTs continue to encounter in the current healthcare environment, i.e. that only a minority of patients are enrolled into trials and that trial participants might not reflect the broad spectrum of patients seen in clinical practice. These observations are important since the findings remain unchanged despite similar reports in previous studies. In the pre-PCI era, patients with ST-elevation myocardial infarction treated with fibrinolytic agents enrolled into RCTs had lower baseline risk and better long-term prognosis compared with trial-ineligible patients treated with fibrinolytic agents.⁶ Similar findings have since then been reported for trial participants vs. eligible non-participants and trial-ineligible patients with ACS.^7,8

The reason for the improved risk-adjusted outcomes among trial participants vs. eligible non-participants and trial-ineligible patients is unknown. It has been postulated that participation in RCTs is associated with closer follow-up as part of the trial and thereby better access to medical care.⁷ Possibly, residual confounders may also differentiate trial participants from their counterparts. However, these findings do not reduce the value of RCTs to establish the relative efficacy and safety of novel treatment strategies for the total patient population, despite the absolute risk–benefits being different for the non-included patients.⁹ Rather the observations provide a good example of the remaining problems with RCTs and emphasise the need to improve the design and performance of clinical trials. Also, these observations highlight the importance of continuous monitoring and the need for standardized processes of care for all patients regardless of enrolment in clinical trials.

One strategy to overcome the problem with the generalizability of RCTs is to embed trials into routine care by basing recruitment and follow-up on the continuously collected data in electronic health records, clinical registries, and administrative databases. Several countries are currently running nationwide all-comer registries that continuously collect structured real-world data as the basis for meaningful, generalizable, and cost-effective observational research. Such a strategy has proven efficient and effective in generating evidence and changing the patterns of care delivery.¹⁰ However, data from registries often suffer from selection bias and confounding factors (despite advanced statistical modelling), which limits the conclusions that can be drawn when performing comparative observational studies.

The emerging concept of registry-based RCTs (R-RCTs) in recent years has led to a growing interest in clinical registries as a potential platform for conducting pragmatic trials.¹¹ Unlike traditional clinical trials, R-RCTs integrate the randomization process into a clinical registry facilitating the collection of baseline characteristics, patient consent, follow-up details, and the study endpoints using the registry platform.¹² By integrating clinical research into routine data collection endeavours, a balance between generalizability, validity, and cost effectiveness may be achieved, creating a unified system for continuous quality improvement and clinical research (Figure 1). If designed properly, registries may even facilitate the identification of eligible participants for a given trial and allow the capture of relevant information at different time points in accordance with the trial aims and specifications.¹² However, the R-RCT concept is not always feasible and for new drugs and devices where there is a need for comprehensive safety monitoring, traditional RCTs with blinding, dedicated follow-up and formal adjudication might still remain the preferred alternative.¹³

Figure 1.

The concept of registry-based randomized controlled trials.

The findings by Matetic et al. emphasise the need for an infrastructure across different clinical and geographical boarders to improve the representation in clinical trials. The European Unified Registries On Heart Care Evaluation and Randomised Trials (EuroHeart) initiative of the European Society of Cardiology is currently developing standards for continuous registration of pre-defined structured data in patients with common CVDs. The aim is to establish European all-inclusive clinical registries for continuous data collection, which will facilitate collaboration on international pragmatic R-RCTs and enable evidence generation in broad real-world patient populations.¹⁴ This integration of real-world practice with clinical trials is crucial to overcome some of the shortcomings highlighted in the paper by Matetic et al.

Notes

The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal—Quality of Care and Clinical Outcomes or of the European Society of Cardiology.

Conflict of interest

G.B. reports honoraria for lectures and scientific advice from AstraZeneca, Boehringer Ingelheim, Novo Nordisk, and Sanofi.

L.W. reports institutional research grants from AstraZeneca, Boehringer Ingelheim, Bristol Myers Squibb/Pfizer, GlaxoSmithKline, Merck & Co, and Roche Diagnostics.