The Association Between Clinical Value and Financial Cost of Cancer Treatments: A Cross-Sectional Analysis

Abstract

Purpose:

The cost of cancer treatment has risen significantly in recent decades, but it is unclear if these costs have been associated with commensurate improvement in clinical value. We aimed to assess the association between the cost of cancer treatment and the National Comprehensive Cancer Network (NCCN) Evidence Blocks’ four measures of clinical value: Efficacy, Safety, Quality of Evidence, and Consistency of Evidence.

Methods:

Cross-sectional, observational study. We obtained NCCN Evidence Blocks ratings for all recommended, first-line and/or maintenance treatments for the thirty most prevalent cancers in the US, and calculated direct pharmacologic treatment costs (drug acquisition, administration fees, guideline-concordant supportive care medications) using Medicare reimbursement rates in January 2019. We used generalized estimating equations to estimate the association between Evidence Blocks measures and treatment cost with clustering at the level of the treatment indication.

Results:

1,386 treatments were included. Among time-unlimited treatments (those administered on an ongoing basis without predetermined stopping point), monthly cost was positively associated with Efficacy ($3,036, 95%CI: $1,782, $4,289) and Quality of Evidence ($1,509, 95%CI: $171, $2,847) but negatively associated with Safety (−$1,470, 95%CI: −$2,790, −$151) and Consistency of Evidence (−$2,003, 95%CI −$3,420, −$586). Among time-limited treatments (those administered for a predetermined interval or number of cycles), no Evidence Blocks measure was significantly associated with treatment cost.

Conclusion:

An association between NCCN Evidence Blocks measures and treatment cost was inconsistent, and the magnitude of the association small in comparison to the degree of cost variation among treatments with the same Evidence Blocks scores. The clinical value of cancer treatments does not appear to be a primary determinant of treatment cost.

Introduction

Drug prices have become a significant problem in oncology. Initial list prices have increased at an exponential rate,¹ and most new cancer drugs experience additional post-marketing price increases.^2,3 These trends have resulted in sufficient financial toxicity.⁴

However, it is not clear that newer, more-expensive drugs have produce clinical benefit commensurate with their financial burden. As prices have increased, the financial cost necessary to achieve the same survival improvement with newer drugs has increased as well.⁵ Additionally, a number of studies evaluating the prices of newly-approved cancer drugs with respect to their clinical trial data have failed to find a positive relationship between the magnitude of improvements in patient outcomes and drug prices.^6–13

Despite these observations, pharmaceutical industry representatives have long asserted that drug prices reflect underlying clinical value, and are therefore justified.^14–16 Whether rising cancer drug prices reflect significant improvements in effectiveness for cancer patients is therefore germane to ongoing legislative attempts at drug price reforms.

Whether treatment price may reflect clinical benefit across all cancer treatment settings, not only newly-approved treatments for on-label indications as previously studied, has not been evaluated. Additionally, the influence of recently-approved immunotherapy drugs on the price-benefit association has not been evaluated. Therefore, we aimed to assess the correlation between the clinical value of cancer treatments and the cost of treatment, inclusive of all treatment regimens across all recommended uses, both approved and off-label.

Methods

The National Comprehensive Cancer Network (NCCN) publishes the Evidence Blocks (EB), which assesses cancer treatments on the following five measures, on a simple 1–5 scoring system: Efficacy (the extent to which treatment improves survival and/or symptoms), Safety (the likelihood of side effects, with fewer side effects receiving higher scores), Quality of Evidence (the number and rigor of the supporting clinical trials), Consistency of Evidence (the degree to which clinical trials agree on the degree of benefit), and Affordability (the overall cost of treatment, with less expensive treatments receiving higher scores).¹⁷ EB scores reflect a synthesis of clinical evidence and expert opinion. Then NCCN EB is the most comprehensive of the “value frameworks” developed in oncology, including all recommended treatments regardless of approval date or level of clinical evidence, and is the most widely-recognized framework by oncologists.¹⁸

We extracted EB scores for all treatments used in the adjuvant, neoadjuvant, definitive treatment, first line, or maintenance settings for the 30 highest-incidence cancers in the US, current as of December 31st, 2018. For each treatment, we calculated Medicare treatment costs, using ASP file and Medicare Plan Finder prices, inclusive of administration and supportive care costs, as previously described (for full methodologic details, see Appendix).¹⁹ Ancillary and non-medical costs were not included. To avoid making misleading cost comparisons, we categorized each treatment as either “time-limited” (often, adjuvant or neo-adjuvant treatments) or “time-unlimited” (often, treatments for advanced/metastatic disease). For time-limited treatments, we calculated costs across the full course of therapy; for time-unlimited treatments, we calculated the monthly cost of treatment.

To enable comparisons between treatments that were similar (e.g., those that might be weighed against each other in a clinical treatment decision), we categorized all treatments into treatment groups defined by 1) treatment indication, as defined by NCCN EB, 2) time-limited vs. time-unlimited, and 3) inclusion of radiation therapy (RT) and/or hematopoietic stem cell transfer (SCT) as part of the treatment.

We used descriptive statistics to assess the distribution of treatment costs across NCCN EB scores, separately for time-limited and time-unlimited treatments. We assessed the distribution for each measure separately, and for the overall sum of Efficacy, Safety, Quality, and Consistency; we omitted Affordability because it is a measure of treatment cost, which we measured separately.

We used generalized estimating equations, clustered at the level of the treatment group, to assess the independent association between treatment costs and each EB measure. Models were multivariable, with treatment cost as the dependent variable and the four included EB measures as independent variables. We conducted planned sensitivity analyses which included: 1) general linear models without clustering, 2) omission of treatment groups containing N=1 treatment, 3) no supportive care costs, 4) more aggressive supportive care utilization, 5) modeling logged treatment costs, 6) omission of treatment groups using RT or SCT, 7) rank-order testing (Kruskal-Wallis test), and 8) using treatment group-level fixed effects.

Results

The analysis included 1,386 treatments, 541 of which were time-unlimited and 845 were time-limited. For each NCCN EB measure, there was wide variation in treatment cost across each possible score. For example, for time-limited regimens that received an Efficacy of 5 (highest efficacy), the mean cost was $35,796, with a range of $2,292–217,998 (Table 1).

Table 1: Cost Mean and Range at Each Level of the NCCN Evidence Blocks Measures.

NCCN Evidence Blocks measures are scaled such that higher scores are always preferable. Time-unlimited treatments (N = 541) include treatments intended to be given for an undefined time period, and time-limited treatments (N = 845) include those intended to be given for a predetermined time period and/or number of cycles. For time-unlimited, treatment cost represents cost per month of therapy; for time-limited, treatment cost represents cost for full course of treatment. USD.

	NCCN EB Measure	EB Score:	1	2	3	4	5
Time-unlimited (N = 541)	Efficacy	Mean	N/A	4,591	5,898	10,228	11,584
		Range	N/A	(4–43,078)	(4–31,190)	(6–64,630)	(1,387–17,093)
		N	0	121	241	164	15
	Safety	Mean	8,050	8,591	7,264	5,135	N/A
		Range	(8,050–8,050)	(242–64,630)	(16–43,079)	(4–21,537)	N/A
		N	2	105	320	114	0
	Quality	Mean	27,260	5,087	5,422	10,086	7,368
		Range	N/A	(6–31,190)	(4–43,079)	(262–64,630)	(6–19,119)
		N	1	76	251	178	35
	Consistency	Mean	N/A	7,389	5,244	9,370	6,926
		Range	N/A	(4–31,190)	(4–43,079)	(207–64,630)	(6–19,119)
		N	0	43	260	204	34
Time-limited (N = 845)	Efficacy	Mean	N/A	29,899	24,673	36,494	35,796
		Range	N/A	(498–294,412)	(0–775,559)	(0–500,982)	(2,292–217,998)
		N	0	75	347	359	64
	Safety	Mean	33,607	32,846	27,475	44,140	N/A
		Range	(1,637–149,110)	(769–775,559)	(0–487,859)	(527–294,412)	N/A
		N	12	265	479	89	0
	Quality	Mean	3,718	24,020	31,062	33,454	26,186
		Range	(2,584–4,853)	(678–487,859)	(0–500,982)	(527–775,559)	(1,637–116,677)
		N	2	94	416	315	18
	Consistency	Mean	3,718	28,890	27,625	35,605	25,428
		Range	(2,584–4,853)	(678–192,615)	(0–487,859)	(527–775,559)	(1,637–134,794)
		N	2	28	424	366	25

Wide variation was also seen when comparing the cost of each treatment to its summary NCCN EB score (the sum of Efficacy, Safety, Quality, and Consistency) (Figure 1).

Figure 1: Distribution of Treatment Costs and NCCN Evidence Blocks Scores.

X-axis represents the numeric sum of the Efficacy, Safety, Quality of Evidence, and Consistency of Evidence scores for each treatment. Because each measure ranges from the 1–5, the x-axis range is from 4 (lowest clinical value treatments) to 20 (highest clinical value treatments); NCCN Evidence Blocks measures are scaled such that higher scores are always preferable. Time-unlimited treatments include treatments intended to be given for an undefined time period, and time-limited treatments include those intended to be given for a predetermined time period and/or number of cycles. For time-unlimited, treatment cost represents cost per month of therapy; for time-limited, treatment cost represents cost for full course of treatment. N = 541 for time-unlimited treatments, N = 845 for time-limited treatments. For the two treatments with drug costs of $0, we assigned a cost of $1 to allow for inclusion on a logarithmic scale.

Assessing the independent association between each NCCN EB measure and cost for time-unlimited treatments, Efficacy was positively associated with treatment cost (estimated change in cost associated with one-unit increase on Efficacy measure; $3,036, 95%CI: $1,782, $4,289), as was Quality of Evidence ($1,509, 95%CI: $171, $2,847]) while Safety (−$1,470, 95%CI: −$2,790, −$151) and Consistency of Evidence (−$2,003, 95%CI: −$3,420, −$586) were negatively associated with cost (Table 2). Among time-limited treatments, no NCCN EB measure was significantly associated with cost.

Table 2: Association of NCCN Evidence Blocks Measures with Treatment Costs.

Time-unlimited treatments include treatments intended to be given for an undefined time period, and time-limited treatments include those intended to be given for a predetermined time period and/or number of cycles. For time-unlimited, treatment cost represents cost per month of therapy; for time-limited, treatment cost represents cost for full course of treatment. Estimates represent the change in treatment cost associated with a one-unit increase in the score on the indicated Evidence Blocks measure. NCCN Evidence Blocks measures are scaled such that higher scores are always preferable. N = 541 for time-unlimited treatments, N = 845 for time-limited treatments. USD.

	Time-unlimited					Time-limited
Evidence Blocks Measure	Estimate	Lower 95% CI	Upper 95% CI	P Value		Estimate	Lower 95% CI	Upper 95% CI	P Value
Efficacy	3,036	1,782	4,289	<0.001		5,248	−5,798	16,293	0.35
Safety	−1,470	−2,790	−151	0.03		1,964	−6,026	9,953	0.63
Quality	1,509	171	2,847	0.03		−2,066	−15,531	11,399	0.76
Consistency	−2,003	−3,420	−586	0.01		3,274	−7,954	14,502	0.57

Model estimates were largely consistent across sensitivity analyses (Supplementary Tables 1–8). Among time-unlimited treatments, Efficacy and Quality of Evidence were positively associated with cost, while Safety and Consistency of Evidence were negatively associated with cost; among time-limited treatments, Evidence Blocks measures were not associated with cost. A notable exception was fixed-effect modeling (sensitivity analysis 8), which suggested a positive association between Efficacy and cost and a negative association between Quality of Evidence and cost among time-limited treatments.

Discussion

This analysis suggests there may be an association between the NCCN Evidence Blocks measures and treatment cost. However, in our primary analysis this association was present among only one group of treatments (those intended to be administered until disease progression or unacceptable toxicity, designated herein as “time-unlimited”), and explained little of the significant variation in treatment cost. Monthly treatment costs varied by tens of thousands of dollars within each level of the NCCN Efficacy score; the estimated $3,036 greater cost per month associated with a one-unit increase in Efficacy suggests that this measure accounts for little of the difference in cost between treatments. Implicitly, treatment costs are determined primarily by factors other than their clinical “value,” as assessed by the NCCN Evidence Blocks measures.

The finding that some NCCN Evidence Blocks measures were – among time-unlimited treatments – inversely associated with cost is unexpected and intriguing. Because Evidence Blocks measures are scaled so that higher scores are always “better,” this finding would imply that treatments tend to have higher prices when performing worse on these measures. As these associations were present on multivariable analysis, they would not be explained by correlation between measures (e.g., treatments with greater Efficacy tended to have lower Safety). Rather, all other things being equal, more-toxic treatments (for example) appear to be more costly. As the inverse association between Safety and cost was unaffected in sensitivity analysis excluding supportive care costs, a greater need for supportive medications for more-toxic treatments also fails to explain this observation. The inverse associations between treatment cost and both Safety and Consistency of Evidence may support the hypothesis that treatment costs poorly reflect clinical value.

Sensitivity analyses were largely consistent with the primary analysis. One notable exception was the fixed-effect model, which found a positive association between Efficacy and cost among time-limited treatments as well as time-unlimited. This suggests that such an association may be present, after accounting for the substantial inter-treatment group variation in cost. However, the same model still also failed to find an association with Safety or Consistency of Evidence, and Quality of Evidence was negatively associated with cost.

This study has limitations related to the NCCN Evidence Blocks data source. As Evidence Blocks are determined through expert consensus, there is some degree of subjectivity, which makes misclassification possible. It should also be noted that in many cases where we did not identify a statistically significant association between Evidence Blocks measures and costs, the wide confidence intervals of our estimates indicate that our results would be consistent with potentially large associations.

To the extent that the associations identified in this study were inconsistent (evident in only one of the two treatment categories) and the correlation between cost and clinical value was minimal (explaining little of the variation in cost), these findings suggest that the clinical value of treatments is not an important determinant of costs. However, our findings did suggest statistically significant associations between cost and some measures of clinical value; unexpectedly, some of these were “negative” associations, suggesting that less-valuable drugs were priced higher. Therefore, to the extent that these associations do reflect a role of clinical value in determining treatment costs, our functional definition of “value” may prioritize some components of value (effectiveness of treatments, or Efficacy) while failing to account for others (toxicity of treatment, or Safety).