Key Points
Question
Can health care databases be used to confirm a supplemental indication that has been demonstrated in a randomized clinical trial for an approved medication?
Findings
This cohort study replicated the results of a randomized clinical trial that established the supplemental indication for telmisartan by using data from a US health care database (insurance claims data) available at the time that the supplemental indication was approved. Similar to the randomized clinical trial, our study revealed a decreased risk of angioedema with telmisartan compared with ramipril.
Meaning
In certain clinical scenarios, database studies may support supplemental effectiveness applications for already approved medications.
Abstract
Importance
Manufacturers of US Food and Drug Administration–approved prescription drugs often apply for additional indications based on randomized clinical trials. Real-world database analyses on a medication’s use and outcomes in routine settings of care might help to inform decision making regarding such supplemental indications.
Objective
To examine whether longitudinal data from a health care database can support the results of a randomized clinical trial that led to a supplemental indication for telmisartan.
Design, Setting, and Participants
This cohort study of patients newly prescribed telmisartan or ramipril used insurance claims data from a nationwide health care database from January 1, 2003, through September 30, 2009, to compare patient outcomes. This study replicated the inclusion and exclusion criteria used in the Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial (ONTARGET) and used propensity score matching to balance 74 patient characteristics. Data analysis was performed from February 15, 2017, to May 24, 2017.
Exposures
Telmisartan use vs ramipril use.
Main Outcomes and Measures
The primary outcome was a composite of myocardial infarction, stroke, or hospitalization for congestive heart failure.
Results
Of the 640 951 patients included in the study, 48 053 were newly prescribed ramipril (mean [SD] age, 68.29 [9.52] years; 31 940 male [66.5%]) and 4665 were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 male [51.7%]). After propensity score matching, a total of 4665 patients were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 [51.7%]), and 4665 patients were newly prescribed ramipril (mean [SD] age, 69.36 [9.67] years; 2343 male [50.2%]). As seen in ONTARGET, the composite risk of stroke, myocardial infarction, and hospitalization for congestive heart failure was similar for the 2 medications (hazard ratio, 1.0; 95% CI, 0.9-1.1). In addition, the study found that telmisartan was associated with a substantially decreased risk of angioedema (hazard ratio, 0.1; 95% CI, 0.03-0.56) compared with ramipril.
Conclusions and Relevance
Real-world data analyses of patients receiving routine care provided findings similar to those found in the randomized clinical trial that established telmisartan’s supplemental indication. In certain situations, database studies may support supplemental applications for effectiveness for already approved medications.
This study examines whether longitudinal data from a health care database can support the results of a randomized clinical trial that led to a supplemental indication of telmisartan.
Introduction
In December 2016, the 21st Century Cures Act was signed into law in the United States. It contained a provision intended to promote real-world data studies of medication use and outcomes in routine clinical settings in US Food and Drug Administration (FDA) authorization of additional indications for already approved prescription drugs. Such data, with or without randomization, are drawn from health care use data, insurance claims, registry studies, and/or electronic health record systems in typical clinical settings of care. Although the FDA has long used such data to clarify the safety of medications, the data can seldom establish a drug’s effectiveness. Well-designed randomized clinical trials are the criterion standard for assessing whether a drug is efficacious because random treatment assignment and a controlled research environment can more readily support causal inferences.
In recent years, new methodologic approaches have improved the validity and reproducibility of nonrandomized data, including new-user designs, active comparators, propensity score (PS) matching, and controlling for disease risk scores. Other important aspects include assessing covariates before cohort entry (to avoid adjusting for intermediate variables) and defining cohort entry as the time when the patient first receives the exposure of interest (to decrease the possibility of immortal time bias).
Can such analytic techniques confirm supplemental indications for already approved drugs? Approximately half of all drugs approved in the United States are later approved for supplemental indications, modifications to the initial indication, or expanded populations. Supplemental indications are typically identified on the basis of prospective clinical trials. To determine whether real-world data analyses can confirm a supplemental indication, we identified a supplemental approval amenable to study and applied the same inclusion and exclusion criteria and outcomes measurements that were used in the pivotal randomized clinical trial.
Methods
Our cohort study was conducted in commercially insured patients using the MarketScan health care database provided by Truven (January 1, 2003, through September 30, 2009). This nationwide database captures anonymized longitudinal, individual-level data on health care use, patient demographics, inpatient and outpatient diagnostic and procedural codes, and pharmacy dispensing of prescription drugs for more than 60 million commercially insured people in the United States. The study was approved by the institutional review board at Brigham and Women’s Hospital, including a waiver for informed consent, and a valid data licensing agreement was in place. All data were anonymized and deidentified.
Data Sources
To identify an experimental setting, we reviewed all supplemental applications to the FDA from 2005 to 2014 and their accompanying clinical trials. The supplemental indications were classified into 3 mutually exclusive categories: new indication (n = 138), modification (n = 86), and expansion (n = 66) (eAppendix in the Supplement). Of the 138 new indications, 108 (78.3%) of the pivotal clinical trials had a primary outcome that was not identifiable in US longitudinal health care databases (eg, pathology results, change in clinical scores, and radiologic tumor response), 12 (8.7%) did not have an active comparator, 4 (2.9%) were based on in-hospital medication administration (eg, postoperative nausea medication, anesthetic medications), and 14 (10.1%) were potentially replicable with the claims data available to us. Of the 14, we selected telmisartan a priori and did not analyze data for the other 13 (eAppendix in the Supplement).
The angiotensin receptor blocker (ARB) telmisartan (Micardis) was approved as an antihypertensive in 1998. In October 2009, it was approved supplementarily for cardiovascular risk reduction in patients 55 years or older who are unable to take angiotensin-converting enzyme inhibitors (ACE-Is) and have a high risk of major cardiovascular events. Telmisartan was an optimal case study for 3 reasons. First, the primary outcome in the pivotal supplemental indication trial could be accurately identified in health care use data. Second, the randomized clinical trial used an active comparator, the ACE-I ramipril (Altace), which would minimize confounding in cohort studies. Third, the inclusion criteria, exclusion criteria, and baseline patient characteristics were identifiable in claims data.
The trial that identified the supplemental indication for telmisartan for cardiovascular risk reduction, Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial (ONTARGET), was published in April 2008. ONTARGET’s primary objectives were to determine whether telmisartan was at least as effective as ramipril at reducing cardiovascular risk and to assess whether the combination of telmisartan and ramipril was more effective than ramipril alone. The trial was conducted across 733 centers in 40 countries between 2001 and 2008.
Study Cohort
Potentially eligible patients must have had at least 6 months of continuous enrollment in a participating health plan before the date of cohort entry. Our inclusion and exclusion criteria mirrored those of ONTARGET. We included patients 55 years or older who filled a new prescription for telmisartan or ramipril (no fills for either drug or any other ACE-I or ARB during the prior 180 days). Cohort entry date was the first day of a prescription fill. As in ONTARGET, we included patients with a diagnosis of coronary artery disease, peripheral artery disease, cerebrovascular disease, or diabetes mellitus during the 180 days before cohort entry.
As in ONTARGET, we excluded patients with a limited life expectancy (ie, living in a hospice, palliative care facility, or a nursing home and those with cancer), liver disease, syncope or a recent myocardial infarction (within 2 days of cohort entry), transient ischemic attack (within 7 days of cohort entry), percutaneous transluminal coronary angiography (within 30 days of cohort entry), or hospitalization for congestive heart failure during the 180 days before cohort entry. Other exclusion criteria used in ONTARGET were not applied because they were not readily identifiable (known allergy to study medication, unable to tolerate study medication, hemodynamically significant primary valvular or outflow tract obstruction, uncorrected volume or sodium depletion, planned cardiac procedure, blood pressure >160/100 mm Hg despite treatment, significant renal artery stenosis, and angina in the absence of multivessel coronary artery disease) or rare (hereditary fructose intolerance, complex congenital heart disease, primary hyperaldosteronism, and heart transplant). We also excluded patients who previously received any ACE-I or ARB.
Study End Point
Our primary outcome was a composite of myocardial infarction, stroke, or hospitalization for congestive heart failure using the primary discharge diagnosis code for an inpatient visit (see eTable 1 in the Supplement for International Classification of Diseases, Ninth Revision codes). These definitions have satisfactory measurement characteristics; the positive predictive value for myocardial infarction was 93% or higher; stroke, 81% or higher; and congestive heart failure, 87% or higher. Cardiovascular deaths were included in the composite outcome if they occurred during a hospitalization for myocardial infarction, stroke, or heart failure but not outside the hospital.
Statistical Analysis
Our primary analysis compared the rates of the composite end point among patients initiating treatment with telmisartan vs ramipril. Data were censored for patients when they discontinued use of their initial medication, switched to the comparator medication, experienced a study outcome, disenrolled from their health plan, or died, or on September 30, 2009. To address confounding, we adjusted for 74 patient characteristics, including demographics, comorbid conditions, concurrent medications, and health care use measures, using PS methods (Table 1 and eTable 2 in the Supplement). To balance patient characteristics, we used 1:1 PS matching with a caliper of 0.05 and did not perform further variable selection. We compared standardized differences to evaluate the level of balance achieved in patient characteristics after PS matching and used unstratified Cox proportional hazards regression to compute hazard ratios (HRs) and 95% CIs. We then performed a predefined secondary analysis that carried forward the exposure to the first-used medication for 365 days.
Table 1. Baseline Patient Characteristics Before Receiving Telmisartan or Ramiprila,b.
| Characteristic | Unmatched Population | PS-Matched Population | ||||
|---|---|---|---|---|---|---|
| Ramipril (n = 48 053) |
Telmisartan (n = 4665) |
Standardized Difference | Ramipril (n = 4665) |
Telmisartan (n = 4665) |
Standardized Difference | |
| Age, mean (SD), y | 68.29 (9.52) | 69.43 (9.60) | 0.119 | 69.36 (9.67) | 69.43 (9.60) | 0.007 |
| Age category, y | ||||||
| 55-59 | 9747 (20.3) | 802 (17.2) | 0.149 | 839 (18.0) | 802 (17.2) | 0.031 |
| 60-64 | 11 539 (24.0) | 985 (21.1) | 947 (20.3) | 985 (21.1) | ||
| 65-69 | 6262 (13.0) | 626 (13.4) | 655 (14.0) | 620 (13.4) | ||
| 70-74 | 6468 (13.5) | 681 (14.6) | 666 (14.3) | 681 (14.6) | ||
| ≥75 | 14 037 (29.2) | 1571 (33.7) | 1558 (33.4) | 1571 (33.7) | ||
| Male | 31 940 (66.5) | 2413 (51.7) | 0.303 | 2343 (50.2) | 2413 (51.7) | 0.030 |
| Date of cohort entry | ||||||
| First quarter | 13 667 (28.4) | 1198 (25.7) | 0.046 | 1149 (24.6) | 1198 (25.7) | 0.053 |
| Second quarter | 10 080 (21.0) | 1038 (22.3) | 1005 (21.5) | 1038 (22.3) | ||
| Third quarter | 12 730 (26.5) | 1310 (28.1) | 1395 (29.9) | 1310 (28.1) | ||
| Fourth quarter | 11 576 (24.1) | 1119 (24.0) | 1116 (23.9) | 1119 (24.0) | ||
| Comorbid conditions | ||||||
| Hypertension | 21 361 (44.5) | 2835 (60.8) | 0.331 | 2832 (60.7) | 2835 (60.8) | 0.001 |
| Coronary artery disease | 37 591 (78.2) | 3105 (66.6) | 0.263 | 3053 (65.4) | 3105 (66.6) | 0.024 |
| Diabetes mellitus | 14 375 (29.9) | 1524 (32.7) | 0.059 | 1514 (32.5) | 1524 (32.7) | 0.005 |
| PAD | 2651 (5.5) | 362 (7.8) | 0.090 | 355 (7.6) | 362 (7.8) | 0.006 |
| Stroke or TIA | 5727 (11.9) | 730 (15.6) | 0.108 | 783 (16.8) | 730 (15.6) | 0.031 |
| Angina | 11 272 (23.5) | 815 (17.5) | 0.149 | 817 (17.5) | 815 (17.5) | 0.001 |
| Heart failure | 7205 (15.0) | 510 (10.9) | 0.121 | 526 (11.3) | 510 (10.9) | 0.011 |
| Renal disease | 3549 (7.4) | 545 (11.7) | 0.147 | 515 (11.0) | 545 (11.7) | 0.020 |
| Smoking | 1734 (3.6) | 115 (2.5) | 0.067 | 128 (2.7) | 115 (2.5) | 0.017 |
| Previous CABG or PCI | 5454 (11.3) | 124 (2.7) | 0.346 | 111 (2.4) | 124 (2.7) | 0.018 |
| Medications | ||||||
| Statin | 22 441 (46.7) | 2104 (45.1) | 0.032 | 2073 (44.4) | 2104 (45.1) | 0.013 |
| β-Blocker | 20 957 (43.6) | 1926 (41.3) | 0.047 | 1913 (41.0) | 1926 (41.3) | 0.006 |
| Antiplatelet agent | 11 031 (23.0) | 1127 (24.2) | 0.028 | 1148 (24.6) | 1127 (24.2) | 0.010 |
| Calcium channel blocker | 5386 (11.2) | 833 (17.9) | 0.189 | 825 (17.7) | 833 (17.9) | 0.004 |
| Diuretic | 11 396 (23.7) | 1342 (28.8) | 0.115 | 1325 (28.4) | 1342 (28.8) | 0.008 |
| ACE-I or ARB | 0 | 0 | 0 | 0 | 0 | 0 |
| Health care use | ||||||
| Cardiology visit | 29 928 (62.3) | 2526 (54.1) | 0.165 | 2585 (55.4) | 2526 (54.1) | 0.025 |
| General practitioner visit | 35 314 (73.5) | 3571 (76.5) | 0.071 | 3573 (76.6) | 3571 (76.5) | 0.001 |
| Emergency department visit | 9946 (20.7) | 907 (19.4) | 0.031 | 911 (19.5) | 907 (19.4) | 0.002 |
| Influenza vaccination | 4141 (8.6) | 401 (8.6) | 0.001 | 392 (8.4) | 401 (8.6) | 0.007 |
| Transthoracic echocardiogram | 19 496 (40.6) | 1589 (34.1) | 0.135 | 1638 (35.1) | 1589 (34.1) | 0.022 |
Abbreviations: ACE-I, angiotensin-converting enzyme inhibitor; ARB, angiotensin receptor blocker; CABG, coronary artery bypass graft; PAD, peripheral arterial disease; PCI, percutaneous coronary intervention; PS, propensity score; TIA, transient ischemic attack.
Data are presented as number (percentage) of patients unless otherwise indicated.
In the preceding 180 days.
To assess the robustness of our results, we also sought to confirm the well-established increased risk of angioedema for ramipril, expecting that rates of angioedema would be lower for telmisartan, as also demonstrated in ONTARGET. To further assess robustness, we replicated all study end points using a larger cohort derived from less stringent exclusion criteria by creating a cohort that allowed for past ACE-I or ARB use other than telmisartan or ramipril in the preceding 180 days. All analyses were conducted using the Aetion platform and R, version 3.1.2.5 (R Foundation for Statistical Computing), which has been previously validated for a range of studies and for predicting clinical trial findings.
Results
We identified 640 951 patients who filled a prescription for ramipril or telmisartan from January 1, 2003, through September 30, 2009, and had a sufficient baseline enrollment period of at least 180 days. After applying study inclusion and exclusion criteria, 52 739 patients were included (Figure), of whom 48 053 were newly prescribed ramipril (mean [SD] age, 68.29 [9.52] years; 31 940 male [66.5%]) and 4665 were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 male [51.7%]) (a total of 21 patients did not begin follow-up). Patients prescribed ramipril were more likely to be male and have cardiac disease, whereas patients prescribed telmisartan were more likely to have hypertension, kidney disease, and previous transient ischemic attack or stroke and be prescribed a calcium channel blocker (Table 1). After PS matching 4665 telmisartan users (mean [SD] age, 69.43 [9.60] years; 2413 [51.7%]) to 4665 ramipril users (mean [SD] age, 69.36 [9.67] years; 2343 male [50.2%]), these differences were well balanced with standardized differences less than 0.1 (Table 1). Most frequencies of baseline characteristics were consistent with ONTARGET (eg, similar age, rates of hypertension, coronary artery disease, diabetes, and stroke), whereas some were not (ie, lower rates of angina, lower rates of smoking, less documented antiplatelet use, and more women included in our study) (eTable 2 in the Supplement). In the unmatched cohort, mean follow-up time was 232 days (interquartile range, 113-454 days) for the ramipril group and 188 days (interquartile range, 108-427 days) for the telmisartan group. The most common reason for censoring was treatment discontinuation, in 32 135 ramipril users (66.9%) and 3483 telmisartan users (74.7%).
Figure. Flowchart of Patients in the Study .
Cohort criteria included receiving a prescription for ramipril or telmisartan between January 2003 (start of available data) and September 2009 (prior to the Food and Drug Administration approval of the supplemental indication). ACE-I indicates angiotensin converting enzyme inhibitor; ARB, angiotensin receptor blocker; CAD, coronary artery disease; CHF, congestive heart failure; CVD, cerebrovascular disease; MI, myocardial infarction; PAD, peripheral arterial disease; PCI, percutaneous coronary intervention; T2DM, type 2 diabetes mellitus; and TIA, transient ischemic attack.
In ONTARGET, the relative risk of the composite outcome of death from cardiovascular causes, myocardial infarction, stroke, or hospitalization for congestive heart failure was 1.01 (95% CI, 0.94-1.09), indicating no significant difference between telmisartan and ramipril. In our study, the PS-matched relative risk of the composite of myocardial infarction, stroke, or hospitalization for congestive heart failure was almost identical (HR, 0.99; 95% CI, 0.85-1.14) (Table 2).
Table 2. Incidence of the Composite End Point, Its Components, and the Risk of Angioedema.
| Variable | Observational Cohort Study | ONTARGET Clinical Trial | ||
|---|---|---|---|---|
| Ramipril (n = 4665) |
Telmisartan (n = 4665) |
Ramipril (n = 8576) |
Telmisartan (n = 8542) |
|
| Composite End Point | ||||
| No. of person-years | 5579 | 4570 | NA | NA |
| No. of events | 403 | 343 | 1412 | 1423 |
| Incidence rate per 1000 person-years | 72.23 | 75.05 | NA | NA |
| Rate difference per 1000 person-years | 1 [Reference] | 2.82 (−7.80 to 13.44) | NA | NA |
| Unadjusted relative risk | 1 [Reference] | 0.99 (0.89 to 1.11) | NA | NA |
| Relative risk | 1 [Reference] | 0.99 (0.85 to 1.14)a | 1 [Reference] | 1.01 (0.94 to 1.09) |
| Stroke | ||||
| No. of person-years | 5808 | 4718 | NA | NA |
| No. of events | 107 | 86 | 405 | 369 |
| Incidence rate per 1000 person-years | 18.42 | 18.23 | NA | NA |
| Rate difference per 1000 person-years | 1 [Reference] | −0.20 (−5.40 to 5.00) | NA | NA |
| Unadjusted relative risk | 1 [Reference] | 1.08 (0.87 to 1.35) | NA | NA |
| Relative risk | 1 [Reference] | 0.95 (0.71 to 1.26)a | 1 [Reference] | 0.91 (0.70 to 1.05) |
| Myocardial Infarction | ||||
| No. of person-years | 5824 | 4726 | NA | NA |
| No. of events | 84 | 68 | 413 | 440 |
| Incidence rate per 1000 person-years | 14.42 | 14.39 | NA | NA |
| Rate difference per 1000 person-years | 1 [Reference] | −0.03 (−4.64 to 4.57) | NA | NA |
| Unadjusted relative risk | 1 [Reference] | 0.97 (0.76 to 1.24) | NA | NA |
| Relative risk | 1 [Reference] | 0.92 (0.67 to 1.27)a | 1 [Reference] | 1.07 (0.94 to 1.22) |
| Hospitalization for Heart Failure | ||||
| No. of person-years | 5684 | 4656 | NA | NA |
| No. of events | 284 | 231 | 354 | 394 |
| Incidence rate per 1000 person-years | 49.97 | 49.61 | NA | NA |
| Rate difference per 1000 person-years | 1 [Reference] | −0.35 (−9.00 to 8.29) | NA | NA |
| Unadjusted relative risk | 1 [Reference] | 0.94 (0.82 to 1.08) | NA | NA |
| Relative risk | 1 [Reference] | 0.95 (0.79 to 1.13)a | 1 [Reference] | 1.12 (0.97 to 1.29) |
| Angioedema | ||||
| No. of person-years | 5885 | 4772 | NA | NA |
| No. of events | 18 | 2 | 25 | 10 |
| Incidence rate per 1000 person-years | 3.06 | 0.42 | NA | NA |
| Rate difference per 1000 person-years | 1 [Reference] | −2.64 (−4.17 to −1.11) | NA | NA |
| Unadjusted relative risk | 1 [Reference] | 0.18 (0.04 to 0.70) | NA | NA |
| Relative risk | 1 [Reference] | 0.13 (0.03 to 0.56)a | 1 [Reference] | 0.4 (P = .01)b |
Abbreviations: NA, not applicable; ONTARGET, Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial.
Relative risk using 1:1 propensity score matching.
The CIs are not provided in the ONTARGET article.
Sensitivity Analysis
A sensitivity analysis using the last exposure to the first-used medication for 365 days without considering treatment discontinuation found that the primary end point occurred in 402 ramipril users (86 events per 1000 patients) and 363 telmisartan users (78 events per 1000 patients). This resulted in no significant difference in risk after PS matching (HR, 0.90; 95% CI, 0.77-1.04).
Validation Against a Known Causal Association
Among PS-matched individuals, there were 18 angioedema events in new users of ramipril (3.1 events per 1000 person-years) and 2 events in new users of telmisartan (0.4 events per 1000 person-years). A decreased risk (HR, 0.13; 95% CI, 0.03-0.56) of angioedema with telmisartan was also observed in ONTARGET (HR, 0.40; P = .01).
Robustness of Findings
In the cohort with less stringent exclusion criteria to allow for past ACE-I or ARB use apart from ramipril or telmisartan, we identified 8656 PS-matched new users of telmisartan and 8656 PS-matched new users of ramipril. In this cohort, there was a similar PS-matched relative risk of the composite of myocardial infarction, stroke, or hospitalization for congestive heart failure (HR, 0.97; 95% CI, 0.88-1.08) (Table 3). A decreased PS-matched risk of angioedema with telmisartan compared with ramipril (HR, 0.35; 95% CI, 0.17-0.71) was also revealed.
Table 3. Results From Secondary Analyses That Expanded the Population by Including Patients Who Had Used Angiotensin-Converting Enzyme Inhibitors or Angiotensin Receptor Blockers in the Past 180 Days.
| Variable | Ramipril (n = 8656) |
Telmisartan (n = 8656) |
|---|---|---|
| Composite End Point | ||
| No. of person-years | 10 227 | 8749 |
| No. of events | 799 | 695 |
| Incidence rate per 1000 person-years | 78.13 | 79.44 |
| Unadjusted | 1 [Reference] | 0.96 (0.89-1.04) |
| PS matched | 1 [Reference] | 0.97 (0.88-1.08) |
| Stroke | ||
| No. of person-years | 10 760 | 9098 |
| No. of events | 184 | 173 |
| Incidence rate per 1000 person-years | 17.1 | 19.01 |
| Unadjusted | 1 [Reference] | 1.08 (0.92-1.27) |
| PS matched | 1 [Reference] | 1.07 (0.87-1.32) |
| Myocardial Infarction | ||
| No. of person-years | 10 755 | 9132 |
| No. of events | 156 | 126 |
| Incidence rate per 1000 person-years | 14.5 | 13.8 |
| Unadjusted | 1 [Reference] | 0.87 (0.72-1.04) |
| PS matched | 1 [Reference] | 0.91 (0.72-1.15) |
| Hospitalization for Heart Failure | ||
| No. of person-years | 10 433 | 8928 |
| No. of events | 589 | 481 |
| Incidence rate per 1000 person-years | 56.46 | 53.88 |
| Unadjusted | 1 [Reference] | 0.93 (0.84-1.02) |
| PS matched | 1 [Reference] | 0.91 (0.81-1.03) |
| Angioedema | ||
| No. of person-years | 10 885 | 9220 |
| No. of events | 32 | 10 |
| Incidence rate per 1000 person-years | 2.94 | 1.08 |
| Unadjusted | 1 [Reference] | 0.43 (0.23-0.82) |
| PS matched | 1 [Reference] | 0.35 (0.17-0.71) |
Abbreviation: PS, propensity score.
Discussion
Among patients newly prescribed telmisartan and ramipril before the FDA’s decision to approve a supplemental indication for telmisartan, we found results that were almost identical to those of the randomized clinical trial that led to telmisartan’s supplemental indication. We further identified and quantified the known causal association between ramipril and angioedema. This finding suggests that our data and analysis plan were sufficiently valid to detect known causal associations first identified in a prospective trial.
This study is one of the largest to analyze real-world data to mirror a large randomized clinical trial that had established the clinical basis for a supplemental indication for a medication. In contrast to ONTARGET, which took approximately 7 years to complete and cost tens of millions of dollars, our study took approximately 12 weeks to implement for less than a hundredth of the cost. The fact that our case study bolstered the conclusions of a trial designed to identify a supplemental indication for a marketed medication and was done relatively efficiently using available data sets, rigorous epidemiologic methods, and modern software platforms supports the concept of conducting similar database analyses as part of routine practice for manufacturers submitting applications for supplemental indications to the FDA.
Results concordant with the pivotal clinical trial can provide regulators with greater confidence in approving the indication, whereas discordant results could warrant deeper reexamination of the clinical trial or nonrandomized data. When results are discordant with the pivotal trial, an in-depth analysis of the trial and the nonrandomized study will be necessary to identify reasons for this discordance. These reasons can include issues related to study design, statistical analysis, and patient population. Additional research will be necessary to help navigate this scenario. Eventually, the FDA can develop empirically based guidance on when database analyses are useful in this context and when they are less reliable as a confirmatory source.
Validity of Nonrandomized Real-world Data Analyses
There have been examples of real-world data providing results before the randomized clinical trial was completed and nonrandomized real-world studies that changed prescribing practices for which there will likely never be randomized clinical trial findings. A common signal of quality among these studies and our current study was the use of a new-user, active-comparator design. This approach compares 2 groups of patients who newly start taking a medication and avoids comparing 2 groups with intrinsically discrepant risk profiles as would be found using a nonuser comparator or comparing new users with ongoing users. The new-user design with an active comparator allows a more homogeneous baseline population and was one of the main reasons why the observed baseline characteristics for our patients were similar even before matching. By design, approximately 80% of the 74 baseline characteristics were well balanced before PS matching, suggesting that unmeasured factors may be equally balanced. Similar results were observed in the recent new-user, active-comparator study by Graham et al that compared the safety and effectiveness of rivaroxaban with those of dabigatran.
By contrast, some nonrandomized real-world studies found results that differed from those in subsequent randomized clinical trials. This difference can occur for many reasons, including incorrect study design implementation, reverse causation, immortal time bias, depletion of susceptibles, failure to identify important unmeasured confounding factors, or the inclusion of a different study population than was used in the clinical trial. In particular, comparators that use patients defined as those who did not fill a prescription (nonusers) may introduce treatment selection bias that may not be controllable with any statistical method.
Studies such as ours require that inclusion and exclusion criteria and end points be adequately defined in a randomized clinical trial report and subsequently identifiable in the health care data set being studied. Many trials include study end points that are not recorded in claims data or electronic health care records (eg, rating scales used in trials of psychiatric medications) or not easily identifiable (eg, progression-free survival used in oncology trials) without requiring challenging natural language processing of free-text information. It would also be difficult to replicate results from randomized clinical trials that include different treatment modalities with substantially different risk-benefit profiles (eg, implantable cardioverter defibrillators compared with medical therapy) because of fundamental differences in risk profiles between the 2 populations.
Pharmacoepidemiology analysis of data from nonrandomized, real-world health care databases can be used to support supplemental indications established in prospective randomized clinical trials of marketed medications. This is powerful because they represent outcomes in settings of typical care, rather than the highly controlled research environments of RCTs, and can be accomplished quickly and inexpensively. The analyses can also include subgroups of patients who are underrepresented in clinical trials, including elderly individuals, patients with many comorbidities, pregnant women, and other at-risk groups. In our study, for example, 50% of patients were women compared with approximately 26% in ONTARGET. Finally, such studies can evaluate a larger population of patients and can assess end points that trials are often underpowered to detect, such as rare adverse events.
Limitations
Our observed null finding might reflect limitations within our data set (eg, lack of out-of-hospital death data), duration of follow-up, or study design rather than a true observation. It is well established that noninferiority can appear to be present because of inadequate rigor or scale in any study, whether a randomized clinical trial or an observational analysis. However, this does not explain the increased risk of angioedema that we observed with ramipril but not telmisartan. Some authors have questioned the value of PS matching over traditional risk-adjusted regression analysis, neither of which guarantee full account for unmeasured confounding. However, our unadjusted primary, secondary, and sensitivity analyses did not change meaningfully after PS matching. Another limitation of our study was an inability to assess medication adherence beyond prescription filling, although this is generally seen as a valid measure of actual use.
Conclusions
The FDA is currently considering how it will use nonrandomized, real-world data as part of supplemental indication applications. In the absence of large-scale empirical comparative analyses that identify the reasons for failure and success to replicate randomized controlled findings with real-world data analyses, we performed a case study that highlights some important considerations. Many context-specific questions about study design, confounding control, data quality, and outcome validity will need to be considered. Preregistering study designs and analysis plans and providing a publicly available summary of the results when available, similar to the current practice of randomized clinical trials, promotes ethical conduct of these studies.
Even well-designed analyses sometimes result in incorrect conclusions, and some randomized clinical trials may be inaccurate. Retrospective reviews of the literature provide single summarizations of the differences between these 2 approaches but provide few insights on the validity of individual real-world data analyses. To establish a meaningful baseline, the FDA will need many sets of randomized clinical trials with prospectively designed, nonrandomized analyses to match the populations included in randomized clinical trials across a range of clinical questions, each investigated with a set of designs and methods following rigorous epidemiologic principles.
Regulators have a difficult task in providing specific rules for decision making in this maturing yet still developing and highly context-specific field. However, if done selectively and with principled methods, it might be feasible to use nonrandomized, real-world data to provide supportive evidence in establishing supplemental drug indications.
eAppendix. Methodology for Selecting Telmisartan
eTable 1. Covariate Definitions
eTable 2. Additional Baseline Characteristics
References
- 1.Kesselheim AS, Avorn J. New “21st century cures” legislation. JAMA. 2017;317(6):581-582. [DOI] [PubMed] [Google Scholar]
- 2.Sherman RE, Anderson SA, Dal Pan GJ, et al. . Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293-2297. [DOI] [PubMed] [Google Scholar]
- 3.Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin Pharmacol Ther. 2007;82(2):143-156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Fralick M, Kesselheim AS. Periodic benefit-risk evaluation reports have substantial promise to guide patient care and should be made publicly available. Pharmacoepidemiol Drug Saf. 2017;26(5):597-599. [DOI] [PubMed] [Google Scholar]
- 5.Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003;158(9):915-920. [DOI] [PubMed] [Google Scholar]
- 6.Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmacoepidemiol Drug Saf. 2010;19(8):858-868. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Goodman SN, Schneeweiss S, Baiocchi M. Using design thinking to differentiate useful from misleading evidence in observational research. JAMA. 2017;317(7):705-707. [DOI] [PubMed] [Google Scholar]
- 8.Suissa S. Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492-499. [DOI] [PubMed] [Google Scholar]
- 9.Wang B, Kesselheim AS. Characteristics of efficacy evidence supporting approval of supplemental indications for prescription drugs in United States, 2005-14: systematic review. BMJ. 2015;351:h4679. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Berndt ER, Cockburn IM, Grépin KA. The impact of incremental innovation in biopharmaceuticals: drug utilisation in original and supplemental indications. Pharmacoeconomics. 2006;24(suppl 2):69-86. [DOI] [PubMed] [Google Scholar]
- 11.Yusuf S, Teo KK, Pogue J, et al. ; ONTARGET Investigators . Telmisartan, ramipril, or both in patients at high risk for vascular events. N Engl J Med. 2008;358(15):1547-1559. [DOI] [PubMed] [Google Scholar]
- 12.McCormick N, Lacaille D, Bhole V, Avina-Zubieta JA. Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS One. 2014;9(3):e92286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.McCormick N, Bhole V, Lacaille D, Avina-Zubieta JA. Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS One. 2015;10(8):e0135834. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.McCormick N, Lacaille D, Bhole V, Avina-Zubieta JA. Validity of heart failure diagnoses in administrative databases: a systematic review and meta-analysis. PLoS One. 2014;9(8):e104519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58(4):323-337. [DOI] [PubMed] [Google Scholar]
- 16.Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083-3107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Wang SV, Verpillat P, Rassen JA, Patrick A, Garry EM, Bartels DB. Transparency and reproducibility of observational cohort studies using large healthcare databases. Clin Pharmacol Ther. 2016;99(3):325-332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Fralick M, Schneeweiss S, Patorno E. Risk of diabetic ketoacidosis after initiation of an sglt2 inhibitor. N Engl J Med. 2017;376(23):2300-2302. [DOI] [PubMed] [Google Scholar]
- 19.Kim SC, Solomon DH, Rogers JR, et al. . Cardiovascular safety of tocilizumab versus tumor necrosis factor inhibitors in patients with rheumatoid arthritis: a multi-database cohort study. Arthritis Rheumatol. 2017;69(6):1154-1164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments, part 1: ethical and scientific issues. Ann Intern Med. 2000;133(6):455-463. [DOI] [PubMed] [Google Scholar]
- 21.Institute of Medicine of the National Academies Ethical and scientific issues in studying the safety of approved drugs. The national academy of press website. May 2012. https://www.nap.edu/catalog/13219/ethical-and-scientific-issues-in-studying-the-safety-of-approved-drugs. Accessed June 7, 2017. [PubMed]
- 22.Franklin JM, Schneeweiss S. When and how can real world data analyses substitute for randomized controlled trials? Clin Pharmacol Ther. 2017;102(6):924-933. doi: 10.1002/cpt.857 [DOI] [PubMed] [Google Scholar]
- 23.Schneeweiss S, Seeger JD, Landon J, Walker AM. Aprotinin during coronary-artery bypass grafting and risk of death. N Engl J Med. 2008;358(8):771-783. [DOI] [PubMed] [Google Scholar]
- 24.Choi N-K, Solomon DH, Tsacogianis TN, Landon JE, Song HJ, Kim SC. Comparative safety and effectiveness of denosumab versus zoledronic acid in patients with osteoporosis: a cohort study. J Bone Miner Res. 2017;32(3):611-617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Patorno E, Goldfine A, Schneeweiss S, Glynn R, Liu J, Kim S Safety of canagliflozin vs. other antidiabetic agents in routine care. Paper presented at: 77th Scientific Sessions of the American Diabetes Association; June 9, 2017; San Diego, California. [Google Scholar]
- 26.Graham DJ, Reichman ME, Wernecke M, et al. . Stroke, bleeding, and mortality risks in elderly medicare beneficiaries treated with dabigatran or rivaroxaban for nonvalvular atrial fibrillation. JAMA Intern Med. 2016;176(11):1662-1671. [DOI] [PubMed] [Google Scholar]
- 27.Gagne JJ, Choudhry NK, Kesselheim AS, et al. . Comparative effectiveness of generic and brand-name statins on patient outcomes: a cohort study. Ann Intern Med. 2014;161(6):400-407. [DOI] [PubMed] [Google Scholar]
- 28.Wang PS, Schneeweiss S, Avorn J, et al. . Risk of death in elderly users of conventional vs. atypical antipsychotic medications. N Engl J Med. 2005;353(22):2335-2341. [DOI] [PubMed] [Google Scholar]
- 29.Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Atypical antipsychotic drugs and the risk of sudden cardiac death. N Engl J Med. 2009;360(3):225-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Finucane FF, Madans JH, Bush TL, Wolf PH, Kleinman JC. Decreased risk of stroke among postmenopausal hormone users: results from a national cohort. Arch Intern Med. 1993;153(1):73-79. [PubMed] [Google Scholar]
- 31.Blais L, Desgagné A, LeLorier J. 3-Hydroxy-3-methylglutaryl coenzyme A reductase inhibitors and the risk of cancer: a nested case-control study. Arch Intern Med. 2000;160(15):2363-2368. [DOI] [PubMed] [Google Scholar]
- 32.Wassertheil-Smoller S, Hendrix SL, Limacher M, et al. ; WHI Investigators . Effect of estrogen plus progestin on stroke in postmenopausal women: the Women’s Health Initiative: a randomized trial. JAMA. 2003;289(20):2673-2684. [DOI] [PubMed] [Google Scholar]
- 33.Dale KM, Coleman CI, Henyan NN, Kluger J, White CM. Statins and cancer risk: a meta-analysis. JAMA. 2006;295(1):74-80. [DOI] [PubMed] [Google Scholar]
- 34.Ioannidis JP, Haidich AB, Pappa M, et al. . Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286(7):821-830. [DOI] [PubMed] [Google Scholar]
- 35.Gruber S, Chakravarty A, Heckbert SR, et al. . Design and analysis choices for safety surveillance evaluations need to be tuned to the specifics of the hypothesized drug-outcome association. Pharmacoepidemiol Drug Saf. 2016;25(9):973-981. [DOI] [PubMed] [Google Scholar]
- 36.Moride Y, Abenhaim L. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47(7):731-737. [DOI] [PubMed] [Google Scholar]
- 37.Schneeweiss S, Patrick AR, Stürmer T, et al. . Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results. Med Care. 2007;45(10)(suppl 2):S131-S142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Setoguchi S, Warner Stevenson L, Stewart GC, et al. . Influence of healthy candidate bias in assessing clinical effectiveness for implantable cardioverter-defibrillators: cohort study of older patients with heart failure. BMJ. 2014;348:g2866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.D’Agostino RB Sr, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003;22(2):169-186. [DOI] [PubMed] [Google Scholar]
- 40.Snapinn SM. Noninferiority trials. Curr Control Trials Cardiovasc Med. 2000;1(1):19-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Agoritsas T, Merglen A, Shah ND, O’Donnell M, Guyatt GH, et al. . Adjusted analyses in studies addressing therapy and harm users’ guides to the medical literature clinical scenario. JAMA. 2017;317(7):748-759. [DOI] [PubMed] [Google Scholar]
- 42.Fonarow GC. Randomization—there is no substitute. JAMA Cardiol. 2016;1(6):633-635. [DOI] [PubMed] [Google Scholar]
- 43.Freemantle N, Marston L, Walters K, Wood J, Reynolds MR, Petersen I. Making inferences on treatment effects from real world data: propensity scores, confounding by indication, and other perils for the unwary in observational research. BMJ. 2013;347:f6409. [DOI] [PubMed] [Google Scholar]
- 44.Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Anglemyer A, Horvath HT, Bero L Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. In: Bero L, ed. Cochrane Database of Systematic Reviews. Chichester, England: John Wiley & Sons; 2014:MR000034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887-1892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. 2016;352:i493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Dahabreh IJ, Sheldrick RC, Paulus JK, et al. . Do observational studies using propensity score methods agree with randomized trials? a systematic comparison of studies on acute coronary syndromes. Eur Heart J. 2012;33(15):1893-1901. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix. Methodology for Selecting Telmisartan
eTable 1. Covariate Definitions
eTable 2. Additional Baseline Characteristics