Real-World Evidence for Medication Safety: Causal, Confounded, or Capable?

Randomized clinical trials (RCTs) are the gold standard for determining treatment effects for medical interventions. Nonetheless, contemporary clinical trials in heart failure (HF) and other areas of cardiology are becoming increasingly more difficult to execute due to slow patient enrollment, long study durations, and high costs. Given these difficulties and the seemingly persistent inability of the evidence “supply” to keep up with the “demand,” it is easy to understand the interest in using real-world data (RWD) for clinical research. Use of existing data can reduce cost, offer rapid insights, and produce findings relevant to a broader spectrum of patients treated in everyday clinical settings. This enthusiasm is further amplified by the ongoing “fourth industrial revolution,” with more patient data being collected than ever before and continued technological advances in the way we gather and analyze information.(1)

However, while the promise of RWD is clear, challenges remain. Although RCTs with varying degrees of pragmatism have been successfully performed, the consistent feasibility of the “randomized real-world trial” remains to be seen. As a result, existing use of RWD primarily rests upon observational study design, a common approach to post-market safety monitoring of therapeutics. Most notably, the United States Food and Drug Administration (FDA) has historically supported the use of large-scale RWD for safety surveillance and accordingly launched the Sentinel Initiative, a collaborative network of 18 data partners used for post-market monitoring for over a decade. Building on this precedent with its recent publication of the strategic framework for the FDA Real World Evidence Program, the Agency has galvanized its commitment to explore expanded use of RWD in regulatory decision-making.

In this issue of JACC: Heart Failure, Miró and colleagues present the results of the CORT-AHF (Corticosteroids in Acute Heart Failure) study, an observational study aimed to evaluate the safety of corticosteroid therapy among patients hospitalized for acute HF (AHF).(2) The study included 11,356 AHF patients enrolled in the Epidemiology of Acute Heart Failure in Emergency Departments registry across 41 hospitals in Spain. Patients were stratified by presence or absence of chronic obstructive pulmonary disease (COPD), and then categorized by whether they were treated with systemic corticosteroids during the index AHF hospitalization. Study endpoints included a variety of outcomes, including in-hospital mortality, hospital length of stay, 90-day mortality, and the composite of 90-day mortality or HF rehospitalization. Irrespective of COPD status, patients treated with corticosteroids tended to have more risk factors and greater disease severity at baseline. As a result, unadjusted analyses found that corticosteroid use was associated with higher risk of all study endpoints among patients without comorbid COPD, and higher risk of in-hospital mortality and prolonged length of stay among patients with COPD. After adjustment for differing patient characteristics, all associations were attenuated and no longer significant. This lack of independent association between corticosteroid use and any study outcome remained consistent across sensitivity analyses using varying statistical methods and endpoint follow-up intervals.

The authors are to be congratulated for examining a common clinical scenario and providing preliminary evidence that use of corticosteroids is not associated with obvious harm among patients subsequently diagnosed with AHF. However, on a broader scale, this analysis provides an opportunity to carefully examine the practical use and interpretation of real-world evidence (RWE) for evaluating medication safety. Although the true real-world nature of CORT-AHF can be debated since patients were required to provide informed consent, the study highlights the potential advantages of using clinical registries to evaluate patient outcomes. Registries can be a particularly valuable RWE resource because they often contain clinical information not available in administrative data, such as vital signs and laboratories. This value is demonstrated by the current study’s use of standardized clinical criteria, lab measurements, and echocardiography to confirm AHF diagnosis, a strategy likely to enhance specificity compared with relying on diagnosis codes alone. In contrast, the current approach for identifying COPD was based on past medical report and/or report of the patient or relative without verification through respiratory testing. Given the limitations of patient-reported COPD diagnosis in the general population, sub-study specific results in this analysis should be interpreted with the caveat that some patients were likely misclassified in both the COPD and non-COPD subpopulations. An additional notable study design feature was the use of sensitivity analyses, including analyses that retained patients receiving corticosteroid therapy prior to admission. While prevalent user bias can be introduced by including individuals previously treated with corticosteroids, real patient populations comprise a mix of people who have and have not used the therapy in the past. As a result, including prevalent users examines safety in a patient population more reflective of real-world practice, and therefore complements the primary analysis restricted to incident corticosteroid users. This methodology is also consistent with the FDA’s pharmacoepidemiologic guidance, which encourages use of multiple comparator groups where appropriate to enhance validity.(3)

Despite the strengths and novelty, the authors correctly describe their work as exploratory and hypothesis-generating, highlighting the inability of this study to definitively determine cause and effect. Although observational RWE represents a practical and frequently utilized approach for evaluation of medication safety, the possibility of residual confounding persists despite sophisticated statistical methods. Thus, while observational evaluations of medication safety can offer a degree of reassurance or cause for concern, they must be interpreted in the context of the potential public health consequences of the findings and the totality of available evidence. As an illustrative example, a large, observational, real-world analysis from the SWEDEHEART (Swedish Web-System for Enhancement and Development of Evidence-Based Care in Heart Disease Evaluated According to Recommended Therapies) registry examined the safety of warfarin therapy among patients with chronic kidney disease, atrial fibrillation, and recent myocardial infarction.(4) After risk adjustment, the authors determined that use of warfarin at hospital discharge was not associated with increased risk of bleeding, as compared with no warfarin.(4) This surprising conclusion opposes biologic plausibility (i.e., would expect higher bleeding risk with anticoagulant compared with no anticoagulant) and other data, and suggests selective use of warfarin in patients with low overall bleeding risk or presence of other confounding factors. Thus, unless carefully scrutinized, studies such as this may provide false reassurance of medication safety. As another example, the DIG (Digitalis Investigation Group) trial randomized 6,800 patients with HF with reduced ejection fraction to digoxin or placebo.(5) Despite the trial protocol targeting serum digoxin concentrations of 0.5–2.0 ng/mL (i.e., well above the current guideline recommendation of 0.5–0.9 ng/mL), the study concluded that digoxin therapy did not increase all-cause mortality.(5) Subsequently, a number of observational analyses from real-world and other data sources have raised concerns that digoxin increases mortality among HF patients. While such observational data certainly warrant consideration, the possibility of residual confounding remains and examples exist where alternative statistical methods applied to the same database yield opposing conclusions regarding digoxin safety. Thus, these observational data must be interpreted in the context of the gold standard RCT evidence, and digoxin remains in current HF treatment guidelines.

In summary, RWD and RWE represent welcome additions to the clinical research armamentarium. RWE should not be viewed as oppositional to traditional explanatory RCTs, but as complementary. Although traditional explanatory RCTs remain the “purest” means to determine treatment effects, the high degree of confidence in trial results must be balanced against potential concerns regarding generalizability, feasibility, and cost. Large cardiovascular outcome trials designed to ensure safety are currently performed in select circumstances (e.g., glucose-lowering therapies among patients with type 2 diabetes), but the sheer number of therapies and clinical scenarios precludes every permutation from being tested within a traditional RCT. Unmeasured bias remains an inherent limitation of all non-randomized studies, but as illustrated in CORT-AHF, confounding can be reasonably addressed with robust study design in line with current guidance.(3) Thus, a single RWE study is best considered as neither definitively causal nor hopelessly confounded, but as research that must be interpreted in the context of plausibility, public health implications, and the totality of evidence. In this appropriate context, RWE serves as an efficient, informative, and evolving approach to the evaluation of medication safety.