It’s Time to Get Practical – Consequences of Slow Progress Towards Pragmatism

The randomized clinical trial (RCT) is the gold standard for deriving evidence-based practices in medicine. For common, chronic diseases with multifactorial etiologies, the modest treatment effects that can be expected from new therapies usually require randomization to draw a causal inference that observed differences are due to the treatment. Despite this, most multi-society cardiovascular disease guidelines rely on inconclusive data from small trials, observational studies and expert consensus rather than definitive RCT results; in fact, a recent study found that less than 10 percent of guideline recommendations are supported by the highest quality evidence (Level of Evidence A), which is typically derived from an RCT.¹ In essence, for the leading cause of death we most often depend on consensus opinion rather than firm evidence for decisions that affect life and death and have a huge effect on quality of life.

It is impractical to expect that we can answer all of our day-to-day clinical questions with traditional RCTs; over time RCTs have accrued methodological baggage that is costly, inefficient, and does not clearly accomplish the mission of informing decisions. Their current designs make recruitment and retention of participants challenging. Furthermore, it is difficult to enroll diverse patients, which then limits the generalizability of study findings to real-world populations, while much of the process of clinical trials makes limited contribution to answering the questions that will provide needed evidence. While most other industries have moved to a much more efficient digital infrastructure, traditional RCTs are mired in traditions that distract from the primary simple concepts that underlie appropriate causal inference: proper randomized treatment assignment, measurement of adherence, and complete ascertainment of critical outcomes.

Clinical trialists have developed a construct known as the pragmatic-explanatory continuum indicator summary (PRECIS) which quantifies the degree to which a trial incorporates pragmatic principles.² The updated version of this tool scores 9 elements of trial design: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.³ Each element is scored from 1 (very explanatory) to 5 (very pragmatic). In this issue of JAMA Cardiology, Sepehrvand and colleagues use PRECIS to assess how the degree of clinical trial pragmatism has changed in the past 20 years.⁴ The authors critically examined cardiovascular-related RCTs published in the top 3 medical and top 3 cardiovascular journals in 2000, 2005, 2010, and 2015. Over time, the degree of pragmatism increased only modestly in the 616 RCTs included; the increase was most notable in the domains of eligibility, setting, intervention delivery, and primary endpoints.

This slow pace of transformation of the clinical trials enterprise has consequences for people with chronic diseases. The United States has experienced four consecutive years of a decline in life expectancy, and an uptick is occurring in cardiovascular and stroke mortality after decades of steady reductions.⁵ It is ironic that at a time when we know so much about disturbing epidemiological trends and the enormous evidence gap is becoming clear, we are so slow to make the necessary changes to accelerate the production of high quality evidence to guide efforts to stem the reduction in life expectancy.

Today’s rapid pace of discovery and these concerning trends necessitate more creative ways to conduct RCTs. The good news is that the digital transformation of healthcare and other parts of society is giving us the tools to move quickly to change the field. Although we have access to increasingly large observational datasets (i.e., big data) that provide context and contribute to comparative effectiveness, randomization is still the best choice for inferring treatment effects between or among diagnostic or therapeutic choices.⁶ Accordingly, we need to focus on using randomization efficiently. This was anticipated in the 21^st Century Cures legislation, which mandates a transition to more use of “real world data” and “real world evidence,” carefully defined to include randomization in the context of the real world.⁷

In contrast to explanatory and mechanistic trials, pragmatic clinical trials are designed to answer questions that inform decision-makers about health and healthcare.⁸ Unfortunately, the nomenclature has become muddied by two different definitions, one focused on streamlining procedures and the other focused on deriving knowledge in a manner that mimics clinical practice. Both types of pragmatic trials are useful for understanding how different interventions compare across heterogeneous study populations. Frequently-encountered clinical questions about therapeutic effects and care delivery approaches lend themselves well to such trials given the large numbers of patients affected by these diseases and modest treatment effects. For example, the Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) clinical trial seeks to answer an important yet simple and unanswered clinical question – what is the appropriate dose of aspirin for patients with established cardiovascular disease?⁹ The study, conducted by the Patient-Centered Outcomes Research Network (PCORnet), has randomized approximately 15,000 participants using electronic health records as the primary data source across a few dozen US healthcare systems, thereby streamlining enrollment and data extraction. Furthermore, simple tools such as smartphone applications have the potential to enable rapid enrollment, consent, and interactive participation of tens of thousands of patients at a fraction of the time and cost it would take a traditional clinical trial.^10,11 Such use of technology combined with reform of the human activities can also enable diversification of the participants and improve their engagement by regularly capturing activity levels and patient-reported outcomes.

Sepehrvand and colleagues found that neutral trials and those further along in the study process (phase III/IV) were more likely to be pragmatic. This finding deserves careful thought. The goal of PRECIS is not to strive for the “highest score.” Rather, the criteria should be used as a tool to evaluate the best design for the question addressed by the trial, an approach known as “quality by design.”¹² For example, for new therapies, there is good reason to improve adherence to assigned therapy as part of the study design, but collection of minor adverse events beyond the first 1500 patients is an expensive distraction from what matters in trials. On the other hand, most regulated clinical trials for evaluation of products for marketing use inclusion and exclusion criteria and extreme focus on adherence to optimize the likelihood of a beneficial treatment effect in a way that cannot be replicated in practice. The key is to use tools like PRECIS to come to a rational judgment on trial design.

Interestingly, the authors found that there were no differences in PRECIS-2 scores by funding source, industry versus government. This finding raises the question of whether the NIH, as the dominant government funder of clinical trials, is investing enough in stimulating the transformation of this field. The NIH Collaboratory and similar efforts by individual institutes at NIH provide important efforts to learn about the best methods. However, specifically in cardiovascular medicine, the NHLBI appears to be shrinking its proportional commitment to clinical trials, even in the face of the documented mortality increase and massive evidence gaps.

We commend the authors for investigating an important and timely topic in clinical trial design. Their straightforward descriptive study carefully outlines the characteristics of pragmatic RCTs over the past two decades and points out that pragmatism and exploration exist along a continuum. Clinicians, health systems, and policy makers continue to struggle to use the limited high-quality evidence in daily decisions around therapeutic and diagnostic choices. Clearly, new methods are needed to accelerate the efficient generation of evidence using the RCT design. The concept of pragmatic clinical trials is hardly new,¹³ yet as Sepehrvant and colleagues show more work is needed to break from traditional trial design. As the authors point out, it would be advisable for clinical trialists to self-rate the degree of pragmatism of their proposed trial using a tool like PRECIS-2 and, when appropriate for the study question, they should make every effort to increase the degree of pragmatism. For example, recruiting patients through their usual sites of care and using e-based, patient-centric consent forms can greatly improve a trial’s applicability to real-world settings.

Overall, this study reports only a very modest improvement in published cardiovascular RCT pragmatism over the past 20 years. Now is the time to think critically about how to leverage digital tools and large data platforms to effectively, efficiently, and inclusively generate the practical cardiovascular evidence needed for clinical decision-making and healthcare policy-making. The deterioration in health outcomes in the US calls upon the biomedical community to accelerate pragmatic evidence development to inform decisions.