How should we assess the value of innovative drugs in oncology? Lessons from cost-effectiveness analyses

To the editor:

Introduction

Although the high price of cancer drugs and their marginal benefits (often only weeks to months) have been increasingly criticized, cost-effectiveness analyses (CEAs) often reach favorable conclusions regarding these agents. Recently, Saret and colleagues, using data from the Tufts CEA Registry (www.cearegistry.org),¹ systematically reviewed published CEAs of hematologic cancer drugs and found that most CEA ratios fell below $50,000 (73%) or $100,000 (86%), 2 traditional thresholds of effectiveness. Although there are surely cases of transformative drugs (imatinib, rituximab), the authors’ general conclusion that “innovative treatments for hematologic malignancies may provide reasonable value” is surprising. Many novel drugs confer very modest survival benefits at tremendous costs; how then can the majority of CEAs be favorable? The recent analysis serves as an opportunity to highlight some broad lessons regarding how best to interpret the cancer drug cost-effectiveness literature and recurring challenges.

In the recent review, as their title “Value of Innovation in Hematologic Malignancies” suggests, Saret and colleagues aim to provide a general synopsis of cancer drug value. Yet, by systematically appraising the published literature only, the authors are not examining all drugs for hematologic malignancies, or even a standardized set of such drugs; instead, they preferentially examine certain drugs, used for certain indications, and are unfortunately subject to the hazards of selective reporting and publication bias. For example, although their inclusion criteria permit all drugs for hematologic malignancies between 1997 and 2012, they ultimately examine only 9 different agents for which published studies were available.

Second, the majority of published CEAs in this study, and many in the literature, are sponsored by the drug industry.¹ Twenty-two of 29 studies (76%) Saret et al examined were industry funded. Prior research has shown that industry-sponsored trials are 2.1 to 3.3 times more likely to report favorable CEA estimates than those with nonindustry sponsors.² CEA estimates from industry-sponsored analyses often err on the side of favorability.

Third, the need for CEA analysis is greater in nations outside of the United States, as cost-effectiveness often enters into coverage decisions abroad. For that reason, many CEA studies are conducted in Europe or Canada. In fact, in the Saret article, just 31% of included analyses were done in the United States. When CEA studies are conducted abroad, assumptions regarding the cost of drugs, the cost of alternate treatments, and the cost of treating complications (induced or averted) are all based on local standards. Although the authors ultimately back-calculate all values into US dollars, it is unlikely that the resulting value accurately reflects what the cost would be in the United States. In addition to different drug costs, other nations likely perform health care services at costs that differ from those in the United States. As such, readers of CEA studies must ask themselves whether the costs used in analyses reflect their own local standards.

Fourth, CEAs continued to be performed in settings where efficacy is assumed, and postulated, rather than demonstrated. Consider one³ of the 29 CEAs included by Saret et al, which examined whether giving maintenance rituximab (after R-CHOP [rituximab with cyclophosphamide, doxorubicin, vincristine, and prednisolone]) in second remission of follicular lymphoma is cost-effective. The CEA study drew upon a randomized trial, which reported a large benefit of the intervention. In 2006, with a median follow-up of 33.3 months, maintenance rituximab was found to increase the median progression-free survival (PFS) to 51.8 months from 23.0 months (hazard ratio, 0.54; P = .004).⁴ For their 2008 CEA, the authors assumed that the 5-year PFS for maintenance and observation would be 47% and 22% (a difference of 25 percentage points) and 5-year overall survival would be 73% and 61% (a difference of 12 percentage points), respectively. Based on these figures, the authors estimate the incremental cost-effectiveness ratio to be $19,522 per quality-adjusted life year gained, a very favorable value. Yet, the actual results reported 2 years later did not live up to expectations.⁵ The 5-year PFS for maintenance vs observation was 50% and 38% (a difference of 12 percentage points), and there was no significant overall survival benefit (P = .4). The true CEA thus exceeds the upper-bound estimate of the original sensitivity analysis, which was $181,105 (assuming the difference in survival was 7 percentage points), and was beyond any reasonable threshold for cost-effectiveness.

A recent CEA of cetuximab vs bevacizumab in combination with chemotherapy in the first-line treatment of colorectal cancer is also based on problematic efficacy assumptions. The authors assume that cetuximab added to the chemotherapy FOLFIRI (folinic acid, fluorouracil, irinotecan) compared with bevacizumab added to FOLFOX (folinic acid, fluorouracil, oxaliplatin) would improve median survival from 30.4 to 37.7 months, and the cost-effectiveness would thus be €15,020 per year of life gained.⁶ Yet, when these drugs were tested directly in a randomized trial, both resulted in equivalent survival, but cetuximab led to greater costs and toxicity; thus, the drug is not cost-effective at any threshold.⁷

In short, CEA analyses often make fundamental efficacy assumptions, which may not hold true, upon which the model greatly depends. Readers should closely examine CEA studies to assess whether the purported benefits have been shown, and whether they are plausible.

Fifth, a truly systematic look at the cost-effectiveness of cancer and hematologic drugs is inconsistent with the results of Saret et al. Howard and colleagues⁸ examined 58 anticancer drugs approved between 1996 and 2014, and calculated a cost per life year gained for each agent. The authors found that just 20 of 58 drugs (34%) met the cost-effectiveness threshold of $100,000 per life year. Looking more closely at the 14 distinct hematologic malignancy drugs, Howard and colleagues found that 4 (29%) had CEA ratios <$50,000, and just 6 (43%) <$100,000, whereas more than half (8 of 14, 57%) boasted ratios >$150,000.⁸ Moreover, the authors’ analysis shows a slow and steady rise in the cost per life year over the last 2 decades, with the cost per life year gain rising from just over $50,000 in 1995 to over $200,000 in 2014. This trend has occurred as drugs are priced higher, whereas average efficacy remains unchanged.⁸ Collectively, these data provide a far more representative and measured view than Saret and colleagues.

Conclusion

Reliance on published (but not unpublished) CEAs, those primarily sponsored by the industry, for some drugs and indications but not others does not adequately capture the general value of innovative treatments. Moreover, CEAs done abroad and those that postulate the efficacy of cancer treatments may not capture the true cost-effectiveness of drugs. Finally, empirical evidence suggests that the majority of cancer drugs fail to meet conventional cost-effectiveness thresholds. In short, CEA may often, as in this case, be overly optimistic. Despite the fact that some drugs have been transformative, the majority of novel hematologic malignancy drugs do not provide sufficient value for the money. Such a conclusion should provide impetus for change, and a reconsideration of current pricing models.

Authorship

Acknowledgments: The views and opinions expressed in this article are those of the authors and do not necessarily reflect those of the National Cancer Institute or the National Institutes of Health.

Contribution: V.P. and S.M. conceived and designed the project, performed the data collection and review of the literature, wrote the first draft and revised the manuscript critically, are both guarantors, and reviewed and approved the manuscript prior to submission.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Sham Mailankody, Myeloma Service, Division of Hematologic Malignancies, Memorial Sloan Kettering Cancer Center, SR-505N3, Scholars Building, 5th Floor, 1233 York Ave, New York, NY 10065; e-mail: mailanks@mskcc.org.