Abstract
Background
Roflumilast, a phosphodiesterase 4 inhibitor, was approved for the prevention of COPD exacerbations. It is unclear in which patients roflumilast will have a favorable benefit-harm balance. Our aim was to quantitatively assess the benefits and harms of roflumilast (500 mcg per day) compared to placebo.
Methods and Findings
We used trial data released by the US Food and Drug Administration to estimate the treatment effects of roflumilast. We used data from observational studies when available to estimate the baseline risks for COPD exacerbations and gastrointestinal, neurological and psychiatric harms associated with roflumilast. Using simulation, we calculated the probability that roflumilast provides net benefit. We examined the impacts of different baseline risks for exacerbations and the severity of exacerbations. We varied weights (i.e., relative importance) for outcomes and treated death as a competing risk in the analyses. The probability that roflumilast provides net benefit approximates 0% across different age categories of men and women with varying baseline risks for exacerbations. Using different weights for outcomes did not change the probability that roflumilast provides net benefit. Only in the sensitivity analysis restricted to the prevention of severe exacerbations there was a probability of >50% that roflumilast provides net benefit if the baseline risk of having at least one severe exacerbation per year exceeds 22%.
Conclusions
Our results suggest roflumilast only provides net benefit to patients at a high risk of severe exacerbations. Guideline developers should consider different recommendations for COPD patients at different baseline risks for exacerbations.
Keywords: Pulmonary Disease, Chronic Obstructive; Phosphodiesterase 4 Inhibitors; Risk Assessment
Introduction
Chronic obstructive pulmonary disease (COPD) poses a great burden for patients and health care systems because it is a leading cause for mortality and morbidity worldwide.[1,2] While inhaled drug treatments, pulmonary rehabilitation, long-term oxygen and surgery provide some benefits to COPD patients,[3] finding additional drug treatments that can effectively target important goals of the management of COPD such as symptom relief and reduction of COPD exacerbations is currently a major focus of research and drug development.[4–7]
Phosphodiesterase 4 (PDE4) inhibitors are among the drugs that have raised hope for more effective COPD treatment. Many phase II and III randomized controlled trials (RCTs) have been initiated to explore the efficacy and safety of the PDE4 inhibitors roflumilast and cilomilast in patients with COPD. As a result, roflumilast was recently approved by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) for reducing the risk of exacerbations in patients with severe COPD and a history of exacerbations.[8,9] Cilomilast was not approved by FDA or EMA in large part because its benefits did not appear to exceed its harms. A Cochrane systematic review on PDE4 inhibitors in COPD concluded that while roflumilast reduced the (relative) risk of exacerbations by around 20%, it was indeed associated with harms such as diarrhea, nausea, depression, and weight loss.[10]
Currently, it is still difficult to judge whether or not the benefits of roflumilast outweigh the harms. Clinicians are left with uncertainty about whether to recommend roflumilast to patients or not, and whether the balance of benefits and harms varies among subpopulations within the approved population. Judging the benefit-harm balance of roflumilast is challenging because roflumilast has differential effects on distinct outcomes that are of varying importance to patients and that occur with different frequency. Trial reports and systematic reviews typically provide data on benefit and harm outcomes separately with no intention to integrate the two types of outcomes. Therefore, our aim was to estimate the benefits and harms of roflumilast compared to placebo in COPD patients considering multiple outcomes, baseline risks, and patient preferences, and using a systematic approach for identifying and synthesizing different relevant sources of evidence.
Methods
Definition of the target population, the intervention and the outcomes
The population of interest included patients with moderate to severe COPD with a history of exacerbations, for whom roflumilast (1 × 500 mcg tablet per day) was approved.[9] The outcomes, evaluated over one year, included moderate or severe exacerbations prevented and harms. We defined a moderate exacerbation, following the definition of the manufacturer, as a COPD event requiring outpatient treatment and a severe exacerbation as a COPD event resulting in hospitalization or death.[11] Harm outcomes included gastrointestinal (acute pancreatitis, diarrhea, nausea and weight loss), psychiatric (insomnia, anxiety, depression and suicide), and neurological (headache and dizziness) symptoms or disorders.
Selection of data sources
-
Treatment effect estimates
We identified a Cochrane review[10] on PDE4 inhibitors and the FDA documents publicly available online (medical reviews of roflumilast)[11] as the best available data source for estimating treatment effects of roflumilast. The FDA documents included all data presented in the review but provided substantial additional information, for example on incidence rate ratios and on harm outcomes.
-
Baseline risks without roflumilast for a time period of one year
Baseline risk, the risk for outcomes at treatment initiation, has a large impact on benefit-harm assessment. We relied on observational studies for estimating baseline risk for outcomes whenever possible because the control group risk in RCTs might not reflect the risk of COPD patients seen in practice. For some outcomes where we could not find appropriate data from observational studies we used the risk in the control group of RCTs. Although a time horizon of longer than one year would be relevant for patients and physicians we decided not to do any analyses beyond one year because of the absence of trial data on the long term effects of roflumilast.
-
Weights (relative importance) for outcomes
We considered that the benefit and harm outcomes are, on average, of different importance to patients. We could not find a study (e.g., preference-eliciting survey using conjoint analysis) that provides weights for all outcomes for our decision-making context. But we used various approaches to assigning weights to outcomes in the analysis to explore how they affect the benefit-harm balance.
Statistical analysis
We conducted a benefit-harm assessment of roflumilast following the Gail approach that was developed in a large scale effort by the National Cancer Institute to estimate the benefit-harm balance of tamoxifen for the prevention of breast cancer.[12] This approach combines data on treatment effects, baseline risks, and relative importance of outcomes to provide a net benefit-harm index for decision-making. For a COPD patient at certain age, sex and with a certain baseline risk of exacerbations, the net benefit-harm index indicates whether roflumilast increases or decreases the occurrence of patient-centered outcomes overall (weighted by relative importance of outcomes) as compared to placebo over one year. A positive index indicates that roflumilast provides more benefit than harm. We used simulation to calculate the probability of roflumilast being beneficial as the probability that the index is positive. In sensitivity analyses, we examined the impacts of the severity of exacerbations on the index. The Appendix (Web-only) provides details on the Gail approach.
Results
Data inputs
Table 1 lists the data selected for the benefit-harm assessment of roflumilast. Treatment effect estimates of roflumilast on exacerbations are based on data from the “pivotal studies pool” of the FDA documents. The pivotal studies pool consisted of two pivotal RCTs that focused on a population with severe COPD and a history of exacerbations, which reflected the indication for roflumilast and the target population of this analysis.[13] In all other RCTs the inclusion criteria did not require participants to have a history of exacerbations and exacerbations were not the primary outcome. Therefore, we did not consider these data for calculating the effects of roflumilast on exacerbations. Treatment effect estimates on harms are based on data from the “COPD safety pool” of the FDA documents that consisted of 14 RCTs. The harms associated with roflumilast are likely to be independent of the history of exacerbations so we did not restrict the safety data to the pivotal RCTs but included all 14 RCTs.
Table 1.
Data for the benefit-harm assessment of roflumilast for patients with COPD
| Numbers of patients with ≥1 event, person-years and relative risks of roflumilast for patients with COPD [11]* | |||||
|---|---|---|---|---|---|
| Number of patients with ≥1 event | Person-year | Relative risk (95% CI) | |||
| Type of outcome | Roflumilast 500 mcg | Placebo | Roflumilast 500 mcg | Placebo | |
| Moderate or severe exacerbation | 717 | 821 | 1186 | 1240 | 0.91 (0.82–1.01) |
| Severe exacerbation (for sensitivity analysis) | 157 | 198 | 1186 | 1240 | 0.83 (0.67–1.03) |
| Acute pancreatitis | 5 | 1 | 3261 | 3405 | 5.22 (0.58–246.93) |
| Insomnia | 178 | 61 | 3261 | 3405 | 3.05 (2.27–4.15) |
| Anxiety | 82 | 44 | 3261 | 3405 | 1.95 (1.33–2.87) |
| Depression | 80 | 49 | 3261 | 3405 | 1.70 (1.18–2.48) |
| Suicide (completed)† | 2 | 0 | 3261 | 3405 | 6.00 |
| Diarrhea | 585 | 143 | 3261 | 3405 | 4.27 (3.55–5.17) |
| Nausea | 297 | 79 | 3261 | 3405 | 3.93 (3.05–5.10) |
| Weight loss | 394 | 101 | 3261 | 3405 | 4.07 (3.27–5.12) |
| Headache | 266 | 110 | 3261 | 3405 | 2.52 (2.01–3.18) |
| Dizziness | 139 | 65 | 3261 | 3405 | 2.23 (1.65–3.05) |
| Incidence rates and mortality rates (per 1000 person-years) without roflumilast for patients with COPD | |||||
| Type of outcome | Incidence rate | Type of outcome | Incidence rate | ||
| Acute pancreatitis [16] | 0.077 | Weight loss [15] | 13.4 | ||
| Insomnia [11] | 17.9 | Headache [11] | 32.3 | ||
| Anxiety [11] | 12.9 | Dizziness [11] | 19.1 | ||
| Depression [14] | 16.2 | Mortality [18] | Men | Age <65 | 87.8 |
| Suicide (completed) [17] | 0.124 | Age ≥65 | 106.4 | ||
| Diarrhea [11] | 42.0 | Women | Age <65 | 72.1 | |
| Nausea [11] | 23.2 | Age ≥65 | 85.2 | ||
| Weights (relative importance); Numerically (e.g., 0.5 as used in analysis) and visualized from lowest (•) to highest importance (••••••••••) | ||||
| Type of outcome | Analysis I | Analysis II | Analysis III (main analysis) | Sensitivity analysis |
| Equal weights | Weights based on importance of outcomes | Weights based on importance and co-occurrence of harm outcomes | Analysis focusing on severe exacerbations | |
| Exacerbation | 1.0 •••••••••• |
0.5 ••••• |
0.5 ••••• |
1.0 •••••••••• |
| Acute pancreatitis | 1.0 •••••••••• |
0.5 ••••• |
0.5 ••••• |
0.5 ••••• |
| Insomnia | 1.0 •••••••••• |
0.5 ••••• |
0.25 ••• |
0.25 ••• |
| Anxiety | 1.0 •••••••••• |
0.5 ••••• |
0.25 ••• |
0.25 ••• |
| Depression | 1.0 •••••••••• |
0.5 ••••• |
0.25 ••• |
0.25 ••• |
| Suicide (completed) | 1.0 •••••••••• |
1.0 •••••••••• |
1.0 •••••••••• |
1.0 •••••••••• |
| Diarrhea | 1.0 •••••••••• |
0.1 • |
0.05 • |
0.05 • |
| Nausea | 1.0 •••••••••• |
0.1 • |
0.05 • |
0.05 • |
| Weight loss | 1.0 •••••••••• |
0.1 • |
0.05 • |
0.05 • |
| Headache | 1.0 •••••••••• |
0.1 • |
0.05 • |
0.05 • |
| Dizziness | 1.0 •••••••••• |
0.1 • |
0.05 • |
0.05 • |
Abbreviations: CI = confidence interval; COPD = chronic obstructive pulmonary disease.
We estimated the treatment effects for exacerbations based on the “pivotal studies pool” (including trials M2–124 and M2–125). We estimated the treatment effects for other harm outcomes based on the “COPD safety pool” (including trials FK1–101, FK1–103, M2–107, M2–110, M2–111, M2–112, M2–118, M2–119, M2–121, M2–124, M2–125, M2–127, M2–128, and IN–108).
The relative risk for suicide (completed) was not estimable. We assumed the relative risk was 6.00.
Estimates for the incidence of depression[14] and weight loss[15] are based on observational studies in COPD patients. Estimates for the incidence of acute pancreatitis[16] and suicide[17] are from observational studies or surveillance of general populations because these are more rare events for which COPD-specific data are not available or very imprecise. Baseline risks for other harm outcomes where we could not find appropriate data from observational studies are based on the placebo group in the FDA’s “COPD safety pool.”[11] Estimates of mortality stratified by age (<65 years or ≥65 years) and sex are from ten COPD cohorts with 13,914 patients (1,350 deaths).[18] As the Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE) study suggested,[19] around 70% of patients with ≥ 1 exacerbations in year one (thus those who would potentially qualify for roflumilast) had at least 1 exacerbation in year two of the study. To cover a realistic spectrum of COPD patients with different baseline risks we performed the benefit-harm analysis for patients at 30%, 60% or 90% risk for at least one moderate or severe exacerbation per year, respectively. In sensitivity analysis restricted to the prevention of severe exacerbations, we considered three different levels of patients’ baseline risk of severe exacerbations (10%, 20% or 30% risk for at least one severe exacerbation per year) because that reflects a range around the risk of severe exacerbations observed in the pivotal trials (16%).
The different approaches to assigning weight (relative importance of outcomes) are shown in Table 1. We considered first equal weights (1.0) for all outcomes, and then weights of 1.0 for life threatening outcomes (suicide), 0.5 for serious outcomes (exacerbation, acute pancreatitis, insomnia, anxiety, and depression), and 0.1 for mild outcomes (diarrhea, nausea, weight loss, headache, and dizziness), a similar approach to what was proposed by Gail.[12] We considered smaller weights (0.25 or 0.05) for those harm outcomes that tend to occur together (e.g., patients may have both depression and anxiety) in order not to over-estimate the harms. For sensitivity analysis focusing on severe exacerbations, we considered the same weights for all outcomes as in our main analysis but changed the weight for exacerbations from 0.5 to 1.0. We also used the treatment effect specifically for severe exacerbations.
Benefit-harm assessment
Table 2 shows the expected number of cases for the 11 outcomes in 10,000 men under age 65 treated with and without roflumilast over one year, stratified by patients’ baseline risk of moderate to severe exacerbations. For example, in men under age 65 with a baseline risk where 60% of patients have at least one moderate or severe exacerbation over the year, 4,055 patients are expected to have at least one exacerbation in the treatment group and 4,338 patients in the placebo group. Hence, moderate to severe exacerbations are prevented in 284 patients if 10,000 patients are treated with roflumilast. However, at the same time four cases of acute pancreatitis are expected in the treatment group while one case is expected in the placebo group. Thus, there are three excess cases of acute pancreatitis. As the patients’ baseline risk of moderate to severe exacerbations increases, more cases of exacerbation would be prevented (positive numbers) whereas the harms would remain the same (negative numbers) because in our model the baseline risks of harm outcomes do not depend on the baseline risk of exacerbations.
Table 2.
Expected numbers of cases (patients with ≥1 event) over 1 year for every 10,000 patients (men, age <65) treated with and without roflumilast
| Patients' projected 1-year risk of having ≥1 moderate or severe COPD exacerbation | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 30% | 60% | 90% | |||||||
| Type of outcome | Roflumilast | Placebo | Number of cases (patients with ≥1 event) prevented or caused* |
Roflumilast | Placebo | Number of cases (patients with ≥1 event) prevented or caused* |
Roflumilast | Placebo | Number of cases (patients with ≥1 event) prevented or caused* |
| Moderate or severe exacerbation | 2299 | 2487 | 188 | 4055 | 4338 | 284 | 5397 | 5718 | 321 |
| Acute pancreatitis | 4 | 1 | −3 | 4 | 1 | −3 | 4 | 1 | −3 |
| Insomnia | 509 | 170 | −339 | 509 | 170 | −339 | 509 | 170 | −339 |
| Anxiety | 237 | 123 | −115 | 237 | 123 | −115 | 237 | 123 | −115 |
| Depression | 260 | 154 | −106 | 260 | 154 | −106 | 260 | 154 | −106 |
| Suicide (completed) | 7 | 1 | −6 | 7 | 1 | −6 | 7 | 1 | −6 |
| Diarrhea | 1574 | 394 | −1180 | 1574 | 394 | −1180 | 1574 | 394 | −1180 |
| Nausea | 834 | 220 | −615 | 834 | 220 | −615 | 834 | 220 | −615 |
| Weight loss | 508 | 127 | −381 | 508 | 127 | −381 | 508 | 127 | −381 |
| Headache | 751 | 304 | −446 | 751 | 304 | −446 | 751 | 304 | −446 |
| Dizziness | 399 | 181 | −218 | 399 | 181 | −218 | 399 | 181 | −218 |
Abbreviations: COPD = chronic obstructive pulmonary disease.
Positive number indicates cases prevented and negative number indicates cases caused by roflumilast.
In Table 3, the benefits and harms are combined (i.e., net benefit-harm index) for male and female patients at different age groups (<65 years or ≥65 years) with varying baseline risk of moderate to severe exacerbations. We conducted three types of analyses using different weights as shown in Table 1, where we weighted the outcomes equally (Analysis I), based on the importance of outcomes (Analysis II), or based on both the importance and co-occurrence of harm outcomes (Analysis III, main analysis). For example, in men under age 65 where 60% of patients have at least one moderate or severe exacerbation per year, the net benefit-harm index is −148 with a 0.0% probability that the index is positive and roflumilast beneficial, respectively. We found that roflumilast has no net benefit in every scenario in Table 3 (patients at different baseline risks of moderate or severe exacerbations), and the probabilities that the index is positive all equal 0.0%. We found that sex and age had little effect on the estimates of the net benefit-harm index.
Table 3.
Net benefit-harm index for treatment of COPD with roflumilast
| Net benefit-harm index* per 10,000 patients treated over 1 year by patient profiles | ||||||||||||
| Type of analysis | Patients' projected 1-year risk of having ≥1 moderate or severe COPD exacerbation | |||||||||||
| 30% | 60% | 90% | ||||||||||
| Men | Women | Men | Women | Men | Women | |||||||
|
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
|
|
Analysis I: Equal weights |
−3221 (0.0%)† |
−3192 (0.0%) |
−3246 (0.0%) |
−3225 (0.0%) |
−3125 (0.0%) |
−3097 (0.0%) |
−3150 (0.0%) |
−3129 (0.0%) |
−3088 (0.0%) |
−3059 (0.0%) |
−3112 (0.0%) |
−3092 (0.0%) |
|
Analysis II: Weights based on importance of outcomes |
−477 (0.0%) |
−473 (0.0%) |
−481 (0.0%) |
−478 (0.0%) |
−430 (0.0%) |
−426 (0.0%) |
−433 (0.0%) |
−430 (0.0%) |
−411 (0.0%) |
−407 (0.0%) |
−414 (0.0%) |
−411 (0.0%) |
|
Analysis III (main analysis): Weights based on importance and co-occurrence of harm outcomes |
−195 (0.0%) |
−194 (0.0%) |
−197 (0.0%) |
−196 (0.0%) |
−148 (0.0%) |
−146 (0.0%) |
−149 (0.0%) |
−148 (0.0%) |
−129 (0.0%) |
−127 (0.0%) |
−130 (0.0%) |
−129 (0.0%) |
| Patients' projected 1-year risk of having ≥1 severe COPD exacerbation | ||||||||||||
| 10% | 20% | 30% | ||||||||||
| Men | Women | Men | Women | Men | Women | |||||||
|
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
Age <65 |
Age ≥65 |
|
|
Sensitivity analysis focusing on severe exacerbations |
−140 (0.0%) |
−139 (0.0%) |
−141 (0.0%) |
−140 (0.0%) |
−16 (37.6%) |
−16 (37.5%) |
−16 (37.0%) |
−16 (37.1%) |
85 (86.9%) |
85 (87.0%) |
86 (86.9%) |
85 (87.0%) |
Abbreviations: COPD = chronic obstructive pulmonary disease
Negative values of the index = roflumilast is harmful (harms outweigh benefits); positive values of the index = roflumilast is beneficial (benefits outweigh harms).
The numbers in parentheses is the probability that the index is positive (the probability that roflumilast is beneficial).
We also conducted a sensitivity analysis focusing on severe COPD exacerbations (Table 3). We found that roflumilast is beneficial, as the indexes suggest, in some patients with elevated baseline risk of severe exacerbations. The Figure shows that the probability that the index is positive is greater than 50% if the patients’ baseline risk of severe exacerbations is greater than 22%.
Figure.
The Figure shows the probability that the net benefit-harm index is positive (benefits outweigh harms) when treating patients with varying projected 1-year risk of severe COPD exacerbation.
Discussion
Our analyses showed that roflumilast has no net benefit for younger and older men and women with COPD and a history of moderate to severe exacerbations irrespective of how the benefit and harm outcomes were weighted and irrespective of the baseline risk for moderate to severe exacerbations. However, when we restricted the analysis to severe exacerbations, we found that roflumilast provides more benefit than harm above a baseline risk of 22% for a severe exacerbation over the course of a year.
Our results reflect the complicated regulatory history for PDE4 inhibitors where concerns about gastrointestinal, neurological, and psychiatric harms delayed approval of roflumilast.[20] The FDA provides a quantitative benefit-harm analysis in their publicly available statistical review of roflumilast[21] but the analysis only considered gastrointestinal harms, did not consider different baseline risks or competing risks, and made no attempt to provide an estimate of net benefit (i.e., a summary of benefits and harms). The statistical review explicitly stated that the benefit-harm assessment is up to clinical judgment but the medical review and the transcript of the advisory committee did not entail a quantitative analysis nor a discussion of the benefit-harm balance.[11,22]
In our analysis, evidence for each key element (treatment effects, baseline risks, importance of outcomes and competing risks) is explicitly laid out and considered simultaneously, which allows to consider different scenarios where roflumilast may provide more benefit than harm, and when it may not. Such a systematic and transparent benefit-harm assessment is likely to provide a solid basis for making treatment recommendations and health care decisions.
In order to assess if our analyses were robust to the selection of data sources, we conducted a series of sensitivity analyses. We varied the relative weighting of outcomes, arguably the most controversial part of any benefit-harm assessment, from treating all outcomes the same (i.e., assigning the same weights) to an approach where we considered some harms of less importance and where we accounted for the joint occurrence of harm outcomes in order not to over-estimate harms. Exacerbations received a substantial weight in all analyses. Of note, a weight of 0.5 for exacerbations in the main analysis did not minimize their importance because all of the harm outcomes except for suicide received the same or less weight. Although suicide is rare and had a minimal impact on the benefit-harm balance, we considered it most consequential and therefore, assigned the largest weight. We did not consider different treatment effects in the sensitivity analyses because we believed that the pivotal and safety pool of the FDA RCTs provided the most valid evidence. Also, we did not consider different baseline risks for harm outcomes since COPD-specific estimates for some gastrointestinal, neurological and psychiatric outcomes are relatively scarce. The observational studies and the placebo groups we identified are likely to provide the best available evidence. However, it still may be reasonable to challenge our decisions about data sources, which highlights the importance of using a comprehensive and transparent approach for quantitative benefit-harm assessment.
We believe that the results of this study will aid guideline developers to make evidence-based recommendations. The available scientific evidence and the benefit-harm balance are among the key elements for developing practice recommendations as outlined in the frequently used frameworks such as the Grading of Recommendations Assessment, Development and Evaluation.[23,24] Based on our analysis, the benefit-harm balance is not favorable when looking at the entire group of COPD patients with a history of prior exacerbations, with a probability of roflumilast to provide more net benefit than harm approximating 0%. A guideline panel may consider issuing a strong or at least a weak recommendation against the use of roflumilast in COPD patients with a history of moderate exacerbations. Determining an explicit risk for severe exacerbations requiring hospital admission is difficult without widely validated risk assessment tools. One can assume safely that patients with repeated hospital admissions are likely to have a one-year risk for severe exacerbations that exceeds 20%. For these patients at high risk of a severe exacerbation, a guideline panel may come up with a weak or even strong recommendation for using roflumilast depending on cost and local circumstances. Our considerations of possible recommendations described here are not meant to be directive but they illustrate the usefulness of having separate quantitative estimates for the benefit-harm balance according to the risk and severity of exacerbations.[25]
Strengths of our study include the careful identification of the best available evidence. By using FDA data and data from large observational studies, we went considerably beyond the published RCTs and the Cochrane review, respectively, and provided the best available evidence for treatment effects of roflumilast and risks of outcomes in patients with COPD. By using trial data released by the FDA, we believe that we are less prone to publication bias and because these trials were conducted by the same manufacturer, the heterogeneity among trails is likely to be smaller. Another strength is the use of a transparent approach for quantitative benefit-harm assessment that allows for sensitivity analyses as presented here and additional sensitivity analyses in the future. Also, we considered the statistical uncertainty of treatment effects and risks for outcomes in our analyses. Our approach assessed a wide variety of scenarios for different patient groups and sources of evidence to facilitate identification of the subgroup of patients who may benefit from an intervention.
A weakness of this analysis is the incomplete adjustment for the joint distribution of outcomes. We accounted for death as a competing risk and accounted for the co-occurrence of harm outcomes. But ideally, the observed correlations of all outcomes involved could inform the analyses, which would require availability of and access to individual patient data.[26] We based our analyses on RCTs that compared roflumilast to placebo and did not consider recent or ongoing RCTs that investigate roflumilast as adds-on treatment to inhaled agents. In these RCTs, the treatments effects are likely to be smaller with roflumilast compared to the evidence we considered here. We selected evidence for harms from a larger pool of trials that is more comprehensive, but the harm outcomes may not be uniformly captured across these trials. We modeled the benefit-harm balance in one year for our analysis but the time horizon would not be sufficient to include all potential harms or benefits caused by roflumilast that might occur later.
Finally, some may argue that we should have included lung function or health-related quality of life in our analyses. We did not consider lung function in our benefit-harm assessment because it is not a patient-centered outcome, but rather a surrogate for patient important outcomes we already included in the analyses. We did not consider health-related quality of life because it combines the consequences of exacerbation avoidance and harms whereas we were interested in specific benefit and harm outcomes and their individual contribution to the benefit-harm balance.
In conclusion, our systematic and transparent benefit-harm assessment of roflumilast for COPD patients with a history of exacerbations suggests that roflumilast has no net benefit for most patients. However, if patients are at a high one-year risk of severe exacerbations (>22%), roflumilast is likely to provide more benefit than harm. Guideline developers should consider issuing different recommendations for patients at different risks for moderate and severe exacerbations.
Supplementary Material
What is the key question?
What is the benefit-harm balance of roflumilast (500 mcg per day) in patients with moderate to severe COPD and a history of exacerbations compared to placebo.
What is the bottom line?
The benefit-harm balance of roflumilast is only favorable for a small subgroup of COPD patients at high risk for severe exacerbations. Guideline developers should consider different recommendations for and against roflumilast for patients at different baseline risks for moderate or severe exacerbations, respectively.
Why read on?
This is one of the first benefit-harm assessments of roflumilast that used a quantitative, comprehensive, and transparent approach.
Acknowledgments
Support:
Supported by Grant Number UL1 RR 025005 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research.
Footnotes
Author’s contributions:
TY contributed to the study protocol, obtained the data, contributed to the statistical analysis, and drafted the first version of the report. KF contributed to the study protocol, obtained the data and additional information from the US Food and Drug Administration, contributed to the statistical analysis, and revised the report. CMB, SS, CW and TL contributed to the study protocol and the statistical analysis, and revised the report. RV contributed to the study protocol and the statistical analysis. MAP conceived the study idea, contributed to the study protocol, obtained the data, contributed to the statistical analysis, and drafted the first version of the report.
Conflict of interest:
The authors have no conflicts of interest to disclose.
References
- 1.Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the global burden of disease study 2010. Lancet. 2012;380(9859):2095–2128. doi: 10.1016/S0140-6736(12)61728-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Vos T, Flaxman AD, Naghavi M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: A systematic analysis for the global burden of disease study 2010. Lancet. 2012;380(9859):2163–2196. doi: 10.1016/S0140-6736(12)61729-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Qaseem A, Wilt TJ, Weinberger SE, et al. Diagnosis and management of stable chronic obstructive pulmonary disease: A clinical practice guideline update from the American college of physicians, American college of chest physicians, American thoracic society, and european respiratory society. Ann Intern Med. 2011;155(3):179–191. doi: 10.7326/0003-4819-155-3-201108020-00008. [DOI] [PubMed] [Google Scholar]
- 4.Anzueto A, Sethi S, Martinez FJ. Exacerbations of chronic obstructive pulmonary disease. Proc Am Thorac Soc. 2007;4(7):554–564. doi: 10.1513/pats.200701-003FM. [DOI] [PubMed] [Google Scholar]
- 5.Decramer M, Janssens W, Miravitlles M. Chronic obstructive pulmonary disease. Lancet. 2012;379(9823):1341–1351. doi: 10.1016/S0140-6736(11)60968-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Hansel TT, Barnes PJ. New drugs for exacerbations of chronic obstructive pulmonary disease. Lancet. 2009;374(9691):744–755. doi: 10.1016/S0140-6736(09)61342-8. [DOI] [PubMed] [Google Scholar]
- 7.Loukides S, Bartziokas K, Vestbo J, et al. Novel anti-inflammatory agents in COPD: Targeting lung and systemic inflammation. Curr Drug Targets. 2013;14(2):235–245. doi: 10.2174/1389450111314020008. [DOI] [PubMed] [Google Scholar]
- 8.European Medicines Agency. European public assessment report (EPAR) for Daliresp. [accessed March 29, 2013];2011 http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/002398/human_med_001415.jsp&mid=WC0b01ac058001d124. [Google Scholar]
- 9.US Department of Health and Human Services. Food and Drug Administration. [accessed March 29, 2013];2011 Approval Package for: Application number: 022522Orig1s000. http://www.accessdata.fda.gov/drugsatfda_docs/nda/2011/022522Orig1s000Approv.pdf.
- 10.Chong J, Poole P, Leung B, et al. Phosphodiesterase 4 inhibitors for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2011;(5):CD002309. doi: 10.1002/14651858.CD002309.pub3. [DOI] [PubMed] [Google Scholar]
- 11.US Department of Health and Human Services. Food and Drug Administration. Center for Drug Evaluation and Research. [accessed March 29, 2013];2010 Medical reviews. Application number: 022522Orig1s000. http://www.accessdata.fda.gov/drugsatfda_docs/nda/2011/022522Orig1s000MedR.pdf.
- 12.Gail MH, Costantino JP, Bryant J, et al. Weighing the risks and benefits of tamoxifen treatment for preventing breast cancer. J Natl Cancer Inst. 1999;91(21):1829–1846. doi: 10.1093/jnci/91.21.1829. [DOI] [PubMed] [Google Scholar]
- 13.Calverley PM, Rabe KF, Goehring UM, et al. Roflumilast in symptomatic chronic obstructive pulmonary disease: two randomised clinical trials. Lancet. 2009;374(9691):685–694. doi: 10.1016/S0140-6736(09)61255-1. [DOI] [PubMed] [Google Scholar]
- 14.Schneider C, Jick SS, Bothner U, et al. COPD and the risk of depression. Chest. 2010;137(2):341–347. doi: 10.1378/chest.09-0614. [DOI] [PubMed] [Google Scholar]
- 15.Schneider C, Jick SS, Bothner U, et al. Reflux disease, gastrointestinal ulcer or weight loss in patients with COPD. COPD. 2010;7(3):172–178. doi: 10.3109/15412555.2010.481698. [DOI] [PubMed] [Google Scholar]
- 16.Levy P, Barthet M, Mollard BR, et al. Estimation of the prevalence and incidence of chronic pancreatitis and its complications. Gastroenterol Clin Biol. 2006;30(6–7):838–844. doi: 10.1016/s0399-8320(06)73330-9. [DOI] [PubMed] [Google Scholar]
- 17.Centers for Disease Control and Prevention. National Center for Injury Prevention and Control. [accessed March, 29, 2013];National Suicide Statistics at a Glance. http://www.cdc.gov/ViolencePrevention/suicide/statistics/index.html.
- 18.Puhan MA, Hansel NN, Sobradillo P, et al. Large-scale international validation of the ADO index in subjects with COPD: An individual subject data analysis of 10 cohorts. BMJ Open. 2012;2(6) doi: 10.1136/bmjopen-2012-002152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hurst JR, Vestbo J, Anzueto A, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363(12):1128–1138. doi: 10.1056/NEJMoa0909883. [DOI] [PubMed] [Google Scholar]
- 20.Puhan M. Phosphodiesterase 4 inhibitors for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2011;(8):ED000028. doi: 10.1002/14651858.ED000028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.US Department of Health and Human Services. Food and Drug Administration. [accessed March 29, 2013];Center for Drug Evaluation and Research. Statistical reviews. 2010 Application number: 022522Orig1s000. http://www.accessdata.fda.gov/drugsatfda_docs/nda/2011/022522Orig1s000StatR.pdf.
- 22.US Department of Health and Human Services. Food and Drug Administration. Transcript for the April 7, 2010 Meeting of the Pulmonary-Allergy Drugs Advisory Committee. [accessed March 29, 2013];2010 http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/Pulmonary-AllergyDrugsAdvisoryCommittee/UCM212606.pdf.
- 23.Andrews J, Guyatt G, Oxman AD, et al. GRADE guidelines: 15. going from evidence to recommendations: The significance and presentation of recommendations. J Clin Epidemiol. 2013;66(7):726–735. doi: 10.1016/j.jclinepi.2012.03.013. [DOI] [PubMed] [Google Scholar]
- 24.Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–394. doi: 10.1016/j.jclinepi.2010.04.026. [DOI] [PubMed] [Google Scholar]
- 25.Yu T, Vollenweider D, Varadhan R, et al. Support of personalized medicine through risk-stratified treatment recommendations - an environmental scan of clinical practice guidelines. BMC Medicine. 2013;11(7) doi: 10.1186/1741-7015-11-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Puhan MA, Singh S, Weiss CO, et al. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol. 2012;12:173. doi: 10.1186/1471-2288-12-173. 2288-12-173. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.