STANDFIRST
A recent network meta-analysis confirmed an increased risk of cardiovascular events with use of all non-steroidal anti-inflammatory drugs, both non-selective and selective. In this article, we discuss strengths and weaknesses of comparative safety research using data from randomized clinical trials.
Patients, policy-makers and providers are demanding better comparative effectiveness and comparative safety information. With a wide variety of approved medications for many clinical indications, the relevant question becomes which is the most effective and safest treatment, not whether a treatment is better than placebo. Randomized controlled trials (RCTs) typically allow for the most valid estimate of a treatments efficacy, but they have many well-recognized limitations. 1, 2 First, they rarely compare all (or some) of the relevant treatments, instead including only active treatment and a placebo or a limited range of comparators. Second, follow-up is typically too short and sample sizes are too small to reliably estimate the true range of treatment toxicities. Finally, subjects often do not reflect typical patients, limiting the generalizability of the treatment’s benefits and toxicities outside of the narrowly defined study cohort.
Meta-analyses of RCTs play an essential role in evidence-based medicine as these can strengthen the evidence by combining findings on treatment efficacy and/or safety across individual RCTs.3 Most “traditional” meta-analyses focus on the direct comparisons of treatments (e.g., treatment versus placebo) assigned in the source trials. A network meta-analysis enables one to assess the relative efficacy and/or safety of treatments, by pooling direct and indirect evidence from clinical trials.4–7 Even in the absence of RCTs comparing treatment A with treatment B, a network meta-analysis provides indirect comparison of these treatments by using a common comparator for treatments A and B.
A recent paper 8 by Trelle et al. compared the cardiovascular safety of several non-steroidal anti-inflammatory drugs (NSAIDs) using a sophisticated network meta-analysis of 31 RCTs, each with at least 100 patient-years of follow-up. Trials of the three most commonly used non-selective NSAIDs (ibuprofen, naproxen, and diclofenac) and four different selective cyclooxygenase-2 inhibitors (celecoxib, rofecoxib, etoricoxib, and lumiracoxib) were included in the analyses. The authors presented the risk of individual NSAIDs by specific cardiovascular outcome, such as myocardial infarction (MI), stroke, cardiovascular death, death from any cause, and the Antiplatelet Trialists’ Collaboration composite outcome of non-fatal MI, non-fatal stroke, or cardiovascular death (Table 1). Naproxen was noted to be the safest option with regard to cardiovascular risks. The authors largely confirmed prior findings that all NSAIDs, both non-selective and selective, were associated with a greater risk of cardiovascular adverse events and supported previous regulatory actions. While their main findings were without respect to dosage, they performed a sensitivity analysis (results available in the Web appendix) including only the high-dose trials which showed similar results.
Table 1.
Estimates of rate ratios and 95% credibility intervals for non-steroidal anti-inflammatory drugs compared with placebo in the network meta-analysis by Trelle et al. *
| Myocardial infarction | Stroke | Cardiovascular death | Death from any cause | APTC composite | |
|---|---|---|---|---|---|
| Direct comparison | |||||
|
| |||||
| Naproxen | 0.82 (0.37–1.67) | 1.76 (0.91–3.33) | 0.98 (0.41–2.37) | 1.23 (0.71–2.12) | 1.22 (0.78–1.93) |
| Celecoxib | 1.35 (0.71–2.72) | 1.12 (0.60–2.06) | 2.07 (0.98–4.55) | 1.50 (0.96–2.54) | 1.43 (0.94–2.16) |
| Rofecoxib | 2.12 (1.26–3.56) | 1.07 (0.60–1.82) | 1.58 (0.88–2.84) | 1.56 (1.04–2.23) | 1.44 (1.00–1.99) |
| Lumiracoxib | 2.00 (0.71–6.21) | 2.81 (1.05–7.48) | 1.89 (0.64–7.09) | 1.75 (0.78–4.17) | 2.04 (1.13–4.24) |
|
| |||||
| Indirect comparison | |||||
|
| |||||
| Ibuprofen | 1.61 (0.50–5.77) | 3.36 (1.00–11.60) | 2.39 (0.69–8.64) | 1.77 (0.73–4.30) | 2.26 (1.11–4.89) |
| Diclofenac | 0.82 (0.29–2.20) | 2.86 (1.09–8.36) | 3.98 (1.48–12.70) | 2.31 (1.00–4.95) | 1.60 (0.85–2.99) |
| Etoricoxib | 0.75 (0.23–2.39) | 2.67 (0.82–8.72) | 4.07 (1.23–15.70) | 2.29 (0.94–5.71) | 1.53 (0.74–3.17) |
APTC= Antiplatelet Trialists’ Collaboration; numbers in bold are elevated rate ratios that are statistically significant.
The network meta-analysis of Trelle and colleagues is a creative and rigorous method for comparative effectiveness, limited by the data presented in the original RCTs. Although the study included 116,429 patients with more than 115,000 patient-years of follow-up, it is noteworthy that their estimates were imprecise owing to the relatively few cardiovascular events for individual drugs in the included RCTs. Further sensitivity analyses suggested that their main findings were robust, but most of the 95% credibility intervals reported span the null, limiting meaningful interpretation of the risks associated with each drug (Table 1). One could also question how valid results from the indirect comparisons of a network meta-analysis are. While this type of work is a useful tool for comparative effectiveness research when direct comparisons are not available, it is based on many assumptions (e.g., homogeneity of treatment effects, consistency of the network), some of which may not hold in all cases. It also appears that patient-level data was not used; thereby potentially introducing bias if follow-up differed across arms, which is often the case when an active drug is compared with placebo.
While cardiovascular risk looms large as a potential risk of non-selective and selective NSAIDs, gastrointestinal toxicity accounts for substantial morbidity and mortality. When patients and providers discuss benefits and risks of treatments, risks are usually considered in the aggregate, not focusing on one organ system. As the authors of this meta-analysis suggest, risks and benefits need to be weighed when making treatment decisions. To achieve these goals, better composite measures of risks and benefits need to be developed, so that researchers can answer to the relevant questions that patients and clinicians have. It is also more clinically meaningful when there are at least two active comparators, not drug A versus placebo, for the outcomes as placebos are not a treatment option in clinical practice. A recent comparative safety study of analgesics using large health care utilization data showed that use of both selective cyclooxygenase-2 inhibitors and opioids was associated with an increased risk of cardiovascular events compared to non-selective cyclooxygenase-2 inhibitors. 9 It would have been extremely difficult to compare safety of these different analgesics even with indirect comparison methods if one could only use clinical trial data. Large health care utilization databases have been increasingly used as sources of comparative effectiveness or comparative safety research. While studies using these databases are often limited by the lack of certain clinical information, they exhibit important strengths; these include their size, detailed exposure information, typical patient populations, capacity to study multiple outcomes of various active treatments for a given condition, and availability. Meta-analyses of RCTs and large observational studies each contribute important information and should be considered complementary sources of comparative effectiveness or safety data.
Acknowledgments
Dr. Kim is supported by the National Institutes of Health grant (K23 AR059677). Dr. Kim has received research support from Takeda Pharmaceuticals North America, Inc.
Dr. Solomon is supported by the National Institutes of Health grants (K24 AR055989, P60 AR047782, R21 DE018750, and R01 AR056215). Dr. Solomon has received research support from Abbott Immunology and Amgen, and an educational grant from Bristol-Myers Squibb. He serves an unpaid member of an Executive Committee and a Data Safety Monitoring Board for two analgesic trials sponsored by Pfizer.
References
- 1.Csizmadi I, Collet J-P. Bias and confounding in pharmacoepidemiology. In: Strom B, Kimmel S, editors. Textbook of Pharmacoepidemiology. John Wiley & Sons, Ltd; 2006. pp. 167–171. [Google Scholar]
- 2.Schneeweiss S, Patrick A, Stürmer T, et al. Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results. Med Care. 2007;45(10 Supl 2):S131–142. doi: 10.1097/MLR.0b013e318070c08e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clin Trials. 1986;7:177–88. doi: 10.1016/0197-2456(86)90046-2. [DOI] [PubMed] [Google Scholar]
- 4.Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002;21(16):2313–2324. doi: 10.1002/sim.1201. [DOI] [PubMed] [Google Scholar]
- 5.Bucher H, Guyatt G, Griffith L, Walter S. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997;50:68391. doi: 10.1016/s0895-4356(97)00049-8. [DOI] [PubMed] [Google Scholar]
- 6.Caldwell D, Ades A, Higgins J. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331(7521):897–900. doi: 10.1136/bmj.331.7521.897. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Glenny A, Altman D, Song F, et al. Indirect comparisons of competing interventions. Health Technol Assess. 2005;9(26):1148. doi: 10.3310/hta9260. [DOI] [PubMed] [Google Scholar]
- 8.Trelle S, Reichenbach S, Wandel S, et al. Cardiovascular safety of non-steroidal anti-inflammatory drugs: network meta-analysis. BMJ. 2011;342:c7086. doi: 10.1136/bmj.c7086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Solomon D, Rassen J, Glynn R, et al. The comparative safety of analgesics in older adults with arthritis. Arch Intern Med. 2010;170(22):1968–1976. doi: 10.1001/archinternmed.2010.391. [DOI] [PubMed] [Google Scholar]
