Challenges in the Clinical Application of the American Society of Clinical Oncology Value Framework: A Medicare Cost-Benefit Analysis in Chronic Lymphocytic Leukemia

Purpose:

The ASCO Value Framework calculates the value of cancer therapies. Given costly novel therapeutics for chronic lymphocytic leukemia, we used the framework to compare net health benefit (NHB) and cost within Medicare of all regimens listed in the National Comprehensive Cancer Network (NCCN) guidelines.

Methods:

The current NCCN guidelines for chronic lymphocytic leukemia were reviewed. All referenced studies were screened, and only randomized controlled prospective trials were included. The revised ASCO Value Framework was used to calculate NHB. Medicare drug pricing was used to calculate the cost of therapies.

Results:

Forty-nine studies were screened. The following observations were made: only 10 studies (20%) could be evaluated; when comparing regimens studied against the same control arm, ranking NHB scores were comparable to their preference in guidelines; NHB scores varied depending on which variables were used, and there were no clinically validated thresholds for low or high values; treatment-related deaths were not weighted in the toxicity scores; and six of the 10 studies used less potent control arms, ranked as the least-preferred NCCN-recommended regimens.

Conclusion:

The ASCO Value Framework is an important initial step to quantify value of therapies. Essential limitations include the lack of clinically relevant validated thresholds for NHB scores and lack of incorporation of grade 5 toxicities/treatment-related mortality into its methodology. To optimize its application for clinical practice, we urge investigators/sponsors to incorporate and report the required variables to calculate the NHB of regimens and encourage trials with stronger comparator arms to properly quantify the relative value of therapies.

INTRODUCTION

The costs of cancer care of chronic lymphocytic leukemia (CLL) are accumulating as a result of increased use of expensive novel therapeutics. Although new therapies for CLL are promising, the cost of these therapies will cause a substantial economic burden for both patients and payers.^1,2 Given the concern over unsustainable costs in health care, the National Academy of Sciences set a health quality initiative defining six core aims: safety, efficacy, patient centeredness, timeliness, efficiency, and equitability.³ These six core components define the value of a therapy. Therapeutic decision making should consider estimates of the comparative value of therapies, including evaluation of their relative costs.

Major oncology groups, including the American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), and National Comprehensive Cancer Network (NCCN), have proposed tools to help make clear the clinical benefit of a therapy to patients. The ASCO Value Framework was initially proposed in 2015.⁴ This framework incorporates three of the six aforementioned concepts of quality care, with a focus on efficacy (clinical benefit), safety (toxicity), and efficiency (cost).

Advances in the treatment of CLL parallel the advances made in chronic myelogenous leukemia, when the standard of care transitioned from chemotherapy to tyrosine kinase inhibitors. The cost sharing for patients is usually less with chemoimmunotherapy compared with oral therapeutics such as ibrutinib, because infusional therapies are covered by insurers. Chemoimmunotherapy remains an important preferred first-line therapy, particularly in young fit patients. However, ibrutinib is preferred in patients who harbor deletion 17p and in elderly patients who are not ideal candidates for chemoimmunotherapy.⁵ Differences in clinical benefit and cost are important factors to consider when making therapeutic decisions in CLL. We used the revised ASCO Value Framework⁶ to compare net health benefit (NHB) and cost among regimens in CLL.

METHODS

Using the advanced disease model equations from the revised ASCO Value Framework, we calculated the NHBs of all the regimens listed for CLL in the recent NCCN guidelines (versions 3.2016 and 1.2017).^5,7 Retrospective phase I single-arm trials, trials comparing different dosing/scheduling of the same regimen, and trials containing a control arm not listed in NCCN guidelines were excluded. Stem-cell transplantation was not evaluated, because it is used infrequently in CLL, and its complex cost analysis is beyond the scope of this study. If studies had data for certain subsets of patients (eg, separated cohorts defined as age < 65 and age ≥ 65 years), the NHB scores of the arms were calculated separately.

Methodology of calculating clinical benefit scores, toxicity scores, bonus points, and NHBs was followed as described in the framework.⁶ A brief summary of methodology is as follows: The resultant NHB score reflects the benefit of an experimental arm compared with a control arm. This NHB score is derived from a clinical benefit score, toxicity score, and bonus points if available. Clinical benefit scores were determined using hazard ratios (HRs) for overall (OS) or progression-free survival (PFS) or overall response rate (ORR; in order of preference and weighted accordingly). Toxicity scores were determined by an equation that placed more weight on the percentage of grade 3 to 4 versus grade 1 to 2 toxicities. The revised methodology from 2016⁶ does not describe how to incorporate deaths or grade 5 toxicities. The original methodology described in 2015⁴ incorporated only grade 3 to 5 toxicities. Bonus points were defined as the tail of the curve (ie, if time point on survival curve at 2× median OS or PFS had ≥ 50% improvement in proportion of patients alive) and/or statistically significant improvement in cancer-related symptoms, quality of life, and/or treatment-free interval.

The following additional modifications were made: For the clinical benefit scores, reported HRs that demonstrated significant differences between the two treatment arms (P < .05) were preferred for calculation. For the toxicity score, neutropenia, thrombocytopenia, and anemia were included because they are clinically relevant in CLL; however, other laboratory values were excluded. Bonus points were included if they were reported in the same clinical study. If there were grade 5 toxicities described, these were included with grade 3 to 4 toxicities (similar to the original methodology).

To compare drug acquisition cost (DAC) of infusional therapies, average sales price from the January 2017 Medicare Part B data⁸ was used to calculate six cycles or a complete course of infusional therapy, with doses according to referenced studies, on the basis of an average 81.5-kg person with a body surface area of 1.96 m². DAC (cost without any prescription coverage) and estimated cost sharing (out-of-pocket patient expenses including monthly premiums, annual deductible, copayments/coinsurance, and drug costs not covered by drug plans and not including subsidies) of oral drugs were calculated using the Medicare plan finder tool on the basis of a 12-month supply quoted as a range of 64 Medicare drug plans from February 7, 2017, using prices for Wayne County in Michigan.⁹ The cost metrics used for the ASCO Value Framework are limited to the direct costs of the drugs and do not include the cost of clinic visits or inpatient hospitalizations or indirect costs (eg, travel, childcare, missed work). We also compared ranking of regimens calculated by the framework with the most recent NCCN guidelines (version 1.2017),⁷ which are listed in order of preference.

RESULTS

Forty-nine referenced studies were available for screening (Data Supplement). Of these studies, only 10 (20%) could be evaluated by the framework; these included 17 different comparisons among 14 different regimens.^10-19 Six of the 10 studies evaluating newer therapies used less potent control arms, ranked as least-preferred NCCN guideline regimens (chlorambucil or rituximab monotherapy). Table 1 lists the available comparison regimens and studies evaluated.

Table 1.

Clinical Benefit Scores, Toxicity Scores, and NHBs of CLL Regimens

Clinical Benefit Scores

The clinical benefit score reflects the benefit of an experimental regimen against the control (the higher the score, the better the efficacy compared with the control). Clinical benefit scores were determined using OS, PFS, or ORR, with one of these variables used for calculation in this order of preference.

We preferred using end points that were statistically significantly different at the .05 level between the two regimens, although this is not specified in the framework methodology. If we were to use, for example, an HR for OS that was not statistically significant, and hence a value equal to 1, instead of the HR for PFS in the same study that showed a significant difference, the framework would not distinguish a clinical benefit difference despite a known difference in PFS. The disadvantage of this method is the inconsistency of the clinical variables used or reported.

Of our studies, four comparisons had an HR available for OS, ten comparisons had an HR available for PFS, median PFS was used for two comparisons, and one had only a response rate available to calculate the clinical benefit score. Table 1 lists the clinical variables used and clinical benefit scores.

Currently, there is no threshold number to determine clinically relevant high versus low values. Therefore, we evaluated whether the ranking of these scores reflected what we knew in terms of the efficacy of these regimens. For this evaluation, we performed a cross-study comparison of scores among first-line trials that used chlorambucil monotherapy as the control arm. There are limitations to comparing these trials, because chlorambucil dosing and eligibility criteria were different. However, when comparing regimens studied against chlorambucil monotherapy, the calculated clinical benefit scores ranked ibrutinib with the highest clinical benefit and alemtuzumab with the lowest clinical benefit, which is a reasonable comparison of efficacy (Table 2).

Table 2.

Ranking of CLL Regimens Compared With Chlorambucil Monotherapy Based on Clinical Benefit, Toxicity, and NHB

Calculated clinical benefit differed depending on the clinical variable reported. In the BR (bendamustine and rituximab) versus FCR (fludarabine, cyclophosphamide, and rituximab) study, using HR for progression resulted in a negative calculated benefit score for BR compared with FCR for all patients. However, when analyzing different age cohorts (age ≤ 65 or > 65 years), different clinical variables were significant. Among younger patients, HRs for OS or PFS were not reported, and median PFS was used in the calculation of clinical benefit. In the older cohort, HRs for OS or PFS were not reported, and median PFS was not reached; therefore, ORR was used to calculate the clinical benefit score.

Using ORR, especially if a regimen has > 80% response, can exaggerate the clinical benefit score (Table 1). Additionally, the current methodology does not specify whether to calculate the difference in ORR between the two regimens; therefore, the ORR used was only for BR. In this study, clinical benefit scores varied widely depending on the clinical variables used in our calculations (Data Supplement).

Toxicity Scores

Toxicity score reflects the adverse effects of an experimental arm compared with the control. The higher the score, the more toxic an experimental regimen is compared with the standard. The methodology places more weight on grade 3 to 4 toxicities than grade 1 to 2 toxicities, as well as on greater percentage of absolute number of toxicities. Some studies reported grade 5 toxicities in their toxicity tables, and these were included with grade 3 to 4 toxicities. Calculated toxicity scores are listed in Table 1.

There is no validated threshold for determining clinically acceptable high versus low toxicity. If the resultant score is negative, the experimental arm is less toxic than the control arm. If the score is close to 1, there is a small difference between the two regimens. Because a dichotomous threshold is not available, we performed a similar cross-study comparison of toxicity scores among trials that used chlorambucil monotherapy as the control arm. When ranking calculated toxicity scores for these regimens, this resulted in alemtuzumab with the highest toxicity and ofatumumab plus chlorambucil with the lowest toxicity, which is a reasonable judgment of relative toxicity (Table 2). Interestingly, the BR versus FCR calculations did not find a substantial toxicity difference between the two regimens and computed BR to be slightly more toxic than FCR.

NHB Scores

NHB score was calculated by the clinical benefit score minus the toxicity score plus bonus points (including treatment-free interval, quality of life, and palliation of cancer-related symptoms). Only two studies had treatment-free interval scores that could be incorporated into the calculated NHB scores. One study had information on cancer-related symptoms and quality of life, which was not significantly different between the treatments.

When ranking the regimens studied against chlorambucil monotherapy, ibrutinib and obinutuzumab plus chlorambucil were ranked highest, and alemtuzumab was ranked lowest, similar to the ranking of these regimens in the NCCN guidelines. NHB scores and rankings in NCCN guidelines are listed in Table 3.

Table 3.

Comparison of NHBs, Costs, and NCCN Guideline Preference of CLL Regimens

Calculating NHB score is helpful if the control arm contains a placebo or observation group, because the score reflects the value of the experimental drug. For instance, we can derive NHB scores for ibrutinib and idelalisib using the trials comparing these drugs in combination with BR versus placebo with BR. In both trials, HRs for PFS were statistically significant in favor of the experimental groups. The clinical benefit scores for both drugs were comparable; ibrutinib had a score of 60, and the score for idelalisib was 55 (excluding patients with deletion 17p/TP53 disease). In the ofatumumab maintenance trial versus observation, the resultant NHB score of 25 was lower.

Cost Comparison

The ASCO Value Framework considers both the DAC and the patient copayment. Total direct out-of-pocket cost of a drug would include the patient’s cost sharing. The DAC and estimated cost sharing for a six-cycle course or 12-month supply for all regimens are listed in Table 3.

The newer oral therapies, ibrutinib and idelalisib, are administered chronically and sometimes indefinitely and are the most expensive for the patient. In contrast, chemoimmunotherapy is less costly for the patient, despite having the highest DAC, because infusional therapies are usually covered under Medicare Part B. However, these costs can be formidable depending on the patient’s deductible threshold. Some regimens demonstrated lower NHB scores with high DACs, such as alemtuzumab and ofatumumab maintenance therapy.

DISCUSSION

The ASCO Value Framework displays the benefit and cost of regimens to help physicians and patients have a meaningful discussion about the value of their treatment options. Its conception was an important initial step with some advantages but also vital limitations with potential for improvement.

In our evaluation in one disease, only 20% of studies referenced in the current NCCN guidelines could be used by the framework because of the limitation to prospective randomized controlled trials. Six of the 10 studies evaluating newer therapies used less potent control arms. Such acceptable but perhaps less ideal treatment options are often used in clinical trials to more easily establish efficacy for initial licensing of new therapies. However, these comparisons can exaggerate the clinical benefit scores.

When comparing regimens using the same control arm, the clinical benefit score, toxicity score, and resultant NHB score ranked similarly to their preferences in NCCN guidelines. However, validated thresholds to define clinically meaningful high versus low values are still needed.

When calculating the clinical benefit score, the framework is able to rank regimens that are compared with the same standard. There is a degree of variability in these scores depending on what clinical variable (OS, PFS, or ORR) is used, as demonstrated with our calculations for BR versus FCR (Data Supplement). Additionally, bonus points are hard to incorporate, because ideally, we would use variables that were reported within the same study. Often variables such as quality of life are gathered but not reported. We would strongly encourage investigators and sponsors to take a more uniform approach in reporting the clinical variables needed to optimize comparisons made using the framework (Table 4).

Table 4.

Clinical Variables Needed to Calculate NHB Using ASCO Value Framework

The framework is able to grossly rank regimens compared with a similar control arm with regard to their overall toxicity. However, we did note crucial limitations in the methodology for calculating toxicity scores. The toxicity score methodology failed to appreciate a difference between BR and FCR, suggesting that incorporation of treatment-related mortality and placing of higher weights on certain toxicities are needed to decipher clinically appreciated differences. Historically, FCR has produced long-lasting cytopenias, post-treatment infections,²⁰ and an increased risk of secondary malignancies,²¹ making it less preferable in frail patients. In this study, FCR resulted in more cytopenias, severe infections, and treatment-related deaths compared with BR.¹⁴ Because grade 5 toxicities were reported, we incorporated these into the grade 3 to 4 toxicities in our calculations but still did not demonstrate a significant difference. Parenthetically, it is unclear why treatment-related death (grade 5 toxicity) is not incorporated into the current framework. Another consideration would be to place more weight on certain toxicities, such as severe infections and secondary malignancies.

When comparing regimens by NHB score, the results were similar to their preferred listings in NCCN guidelines, which is reassuring. However, NHB scores are currently not validated with a defined scale, and it is difficult to know what a particular numeric difference actually means clinically. By simply ranking all NHB scores numerically in one disease, we are essentially performing cross comparisons among studies using different control arms and eligibility criteria. Performing more calculations in different diseases and comparing them with how they rank in national guidelines might help to decipher the clinical relevance of high- versus low-value scores.

Although the chronic oral therapeutics for CLL are relatively new, physicians have recognized the unsustainable cost of high-value drugs, using chronic myelogenous leukemia tyrosine kinase inhibitors as an example, and encouraged possible solutions. Discussions of drug cost and value represent one of many proposed interventions.^22,23 Although DAC and cost sharing for Medicare Part B patients are publicly obtainable information, estimated costs using other insurers are not readily available or transparent. This can make the framework difficult to use in some circumstances, although such information may be available through Web-based applications in the future. The goal would be to have cost information easily available for clinicians and counselors during clinic visits. Another important focus of future studies would be to evaluate how patients might alter their decision making on the basis of this information.

We agree that prospective randomized controlled trials are the most appropriate studies to compare two regimens; however, the comparisons are not always between two representative choices in clinical practice. For instance, we are often not deciding between ibrutinib and chlorambucil, because there are other preferred standard regimens to consider. It would be more useful to compare the value of tyrosine kinase inhibitors versus chemoimmunotherapy; completed cooperative group clinical trials investigating ibrutinib versus BR and ibrutinib versus FCR will provide this information in the future.

There is an evolving movement to quantify value for pricing purposes, value-based insurance design, and value-based physician reimbursement models. We would suggest that to optimize the usefulness of these value frameworks, direct comparator trials using two regimens with strong efficacy be highly encouraged.

In conclusion, quantifying the value of a drug treatment is a challenging task, particularly in oncology. Using traditional value metrics such as incremental cost-effectiveness ratio is difficult because many effective oncology drugs can be cost prohibitive, making their incremental cost-effectiveness ratios too high to be deemed cost effective.²⁴ It is recognized that prices do not necessarily correlate to efficacy of cancer drugs.²⁵ Therefore, making a distinction in value is not only helpful for physician-patient communication but also for value-based interventions designed to mitigate cost for patients, including the use of clinical pathways²⁶ and value-based insurance designs.²⁷ Having a validated, quantitative method of assessing value can also help to rank the most preferred treatments, which could decrease potential bias when generating guidelines.²⁸

The two most important limitations we identified were the inconsistencies in the clinical variables reported and the limitation to prospective randomized controlled trials. Clinical benefit scores and bonus points could vary depending on what variables were available. These findings emphasize the need for a uniform approach in publishing the necessary clinical variables, so NHB scores can be more easily compared. Additionally, calculating value against control arms with lower efficacy inflates the value of the experimental regimen, and therefore, clinical trials with stronger comparator arms are required. To properly assess value in this era when models for pricing, reimbursement, and cost discussions need to quantify clinical benefit, the development of tools such as the ASCO Value Framework is an important initial step, albeit with some shortcomings, as we have discussed. The ultimate goal will be to translate these value frameworks into practice, where their applicability will be directly dependent on how we incorporate variables when designing and reporting clinical trials.

Supplementary Material

jop-13-e1002-s001.png ^{(86.2KB, png)}

AUTHOR CONTRIBUTIONS

Conception and design: All authors

Collection and assembly of data: Erlene K. Seymour, Jonas A. de Souza

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Challenges in the Clinical Application of the American Society of Clinical Oncology Value Framework: A Medicare Cost-Benefit Analysis in Chronic Lymphocytic Leukemia

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO’s conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jop/site/ifc/journal-policies.html.

Erlene K. Seymour

No relationship to disclose

Charles A. Schiffer

Consulting or Advisory Role: Celgene, TEVA Pharmaceuticals Industries, Pfizer, Takeda Pharmaceuticals, Ambit BioSciences, Pharmacyclics, Juno Therapeutics, Astellas Pharma, Curis

Research Funding: Bristol-Myers Squibb (Inst), Celgene (Inst), ARIAD Pharmaceuticals (Inst), Novartis (Inst), Micromedic (Inst)

Jonas A. de Souza

Employment: Humana

Stock or Other Ownership: Humana