Value Frameworks in Oncology: Comparative Analysis and Implications to the Pharmaceutical Industry

Abstract

Background

As the cost of oncology care continues to rise, composite value models that variably capture the diverse concerns of patients, physicians, payers, policymakers, and the pharmaceutical industry have begun to take shape.

Objectives

To review the capabilities and limitations of 5 of the most notable value frameworks in oncology that have emerged in recent years and to compare their relative value and application among the intended stakeholders.

Methods

We compared the methodology of the American Society of Clinical Oncology (ASCO) Value Framework (version 2.0), the National Comprehensive Cancer Network Evidence Blocks, Memorial Sloan Kettering Cancer Center DrugAbacus, the Institute for Clinical and Economic Review Value Assessment Framework, and the European Society for Medical Oncology Magnitude of Clinical Benefit Scale, using a side-by-side comparative approach in terms of the input, scoring methodology, and output of each framework. In addition, we gleaned stakeholder insights about these frameworks and their potential real-world applications through dialogues with physicians and payers, as well as through secondary research and an aggregate analysis of previously published survey results.

Results

The analysis identified several framework-specific themes in their respective focus on clinical trial elements, breadth of evidence, evidence weighting, scoring methodology, and value to stakeholders. Our dialogues with physicians and our aggregate analysis of previous surveys revealed a varying level of awareness of, and use of, each of the value frameworks in clinical practice. For example, although the ASCO Value Framework appears nascent in clinical practice, physicians believe that the frameworks will be more useful in practice in the future as they become more established and as their outputs are more widely accepted.

Conclusions

Along with patients and payers, who bear the burden of treatment costs, physicians and policymakers have waded into the discussion of defining value in oncology care, as well as pharmaceutical companies that seek to understand the impact of these value frameworks on each stakeholder, as they model the value and financial threshold of innovative, high-cost drugs.

The value of drugs, whether individually or comparatively, and the definition of value itself, have emerged as acute concerns in oncology, where the cost of cancer care has evoked issues of financial toxicity.1–3 In the United States alone, the costs associated with cancer treatment have been forecast to increase 27% from their 2010 levels, to approximately $157.8 billion by 2020.2 Along with patients and payers, who bear the burden of these costs, physicians and policymakers have waded into the discussion of defining value in oncology care. Although providers and payers require new frameworks to assess the value of therapy options based on their individual perspectives and needs, payers require tools to better guide patient care.4 In addition, the pharmaceutical industry seeks to refine its scenario modeling of the value and financial pain points (ie, willingness of payers and ability of patients to cover the treatment costs) of increasingly complex drugs to achieve greater alignment among payers and their treatment-eligible populations.

Into this complex marketplace, 5 major value frameworks have emerged in the past 2 years, representing years of efforts by diverse organizations and institutions to quantify and evaluate the benefits, harms, and (in some cases) costs, and arrive at a composite value metric or metrics.5–10 But how robust and reliable are these very different frameworks? How can the healthcare industry best apply them to make better decisions, and what do drug manufacturers need to know about these frameworks and their potential for application in the care paradigm?

KEY POINTS

• ▸Several value frameworks have recently emerged to capture the diverse needs of healthcare stakeholders.
• ▸This article examines these 5 recent value frameworks, stakeholder awareness of them, and their use in clinical practice.
• ▸Findings from surveys and interviews with payers and providers suggest that payers are aware of the value frameworks and see their potential, but are still unclear if they will use them.
• ▸Physicians are also considering the use of these frameworks, but they are not yet fully clear about their applications and limitations in clinical decision-making.
• ▸Frameworks that rely on consensus or combined analysis of multiple clinical trials and end points may provide an enhanced indication of the value of a drug in the larger population.
• ▸The relevance of the 5 frameworks to direct (ie, providers, payers) and indirect (ie, drug manufacturers) stakeholders will depend on the adoption of each framework.
• ▸Which value framework becomes widely accepted, and by what stakeholder, will influence how the pharmaceutical industry will shape its oncology drugs development programs.

We compared the methodology of these frameworks using a side-by-side comparative approach in terms of the input, scoring methodology, and output of each framework. In addition, we gleaned stakeholder insights regarding these frameworks and their potential real-world application through dialogues with physicians and payers, as well as through secondary research and an aggregate analysis of previously published surveys. As detailed below, US physician and payer awareness and utilization of the frameworks indicate a rift between their current use and future application that should be noted by those stakeholders seeking to model the impact of these frameworks on their decision-making.

Frameworks Share Some Commonality, but Reflect Very Different Origins and Purposes

The American Society of Clinical Oncology (ASCO), a professional organization representing more than 40,000 physicians of all oncology subspecialties, launched its Value Framework in June 2015,11 targeting physicians and patients (Table 1).6–10 This framework is intended to be a physician-guided tool to facilitate shared decision-making between physicians and patients.6,11

Table 1.

The 5 Value Frameworks Differ in Emphasis

	Value Frameworks6–10
Emphasis	ASCO	NCCN	MSKCC	ICER	ESMO
Application
Target stakeholder	Patient Physician	Patient Physician	Physician Policymaker	Payer Policymaker	Payer Policymaker
Conditions addressed	Oncology: solid, blood	Oncology: solid, blood, radiology, surgery	Oncology: solid, blood	All conditions, focus on new drugs of high impact	Oncology: solid, blood, radiology, surgery
Combination therapy evaluation	Yes	Yes	No	Yes	Yes
Clinical trial data
Breadth of evidence	1 trial, RCT	Published data, panel members' clinical experience, case reports	1 trial, registration trial of first indication (FDA label)	RCT meta-analysis and manufacturer-provided data	1 trial, RCT, comparative outcomes study, meta-analysis
Trial sample size accounted	No	Yes	Yes	Yes	Indirectly, through lower bound of 95% CI
Allows for single-arm trials	Partially	Likely	Yes	Yes	No
Acknowledges trial contamination	No	Likely	No	Yes	Yes
Accounts for patient preference	No	Yes	Yes	No	No
Readout
Outcomes	Net health benefit score	Evidence Blocks scores	DrugAbacus price	Cost-effectiveness; budget impact	ESMO MCBS
Cost/price	Price (WAC or ASP+) per month or course of therapy	Affordability scale	Abacus price per month or course of therapy	Cost per year	Not specified, left to payers to evaluate

ASCO indicates American Society of Clinical Oncology; ASP, average sales price; CI, confidence interval; ESMO, European Society for Medical Oncology; FDA, US Food and Drug Administration; ICER, Institute for Clinical and Economic Review; MCBS, Magnitude of Clinical Benefit Scale; MSKCC, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network; RCT, randomized controlled trial; WAC, wholesale acquisition cost.

Developed by the ASCO Value in Cancer Care Task Force, this approach provides a net health benefit score derived from efficacy, safety, and bonus points for secondary end points.6,11 It also includes a comparison to direct treatment costs and versions in the advanced (noncurative) disease and curative disease settings. In 2016, ASCO updated its Value Framework with changes to the scoring methodology, providing additional secondary end points, such as improvement in quality of life and significant survival improvement in the tail of the curve for which bonus points could be earned.6

The National Comprehensive Cancer Network (NCCN), a nonprofit alliance of 26 cancer centers throughout the United States, launched its evidence block framework in October 2015, to give “the healthcare provider and the patient [similar to ASCO's] information to make informed choices when selecting systemic therapies based upon measures related to treatment, supporting data and cost.”7 Guided by staff from the NCCN, in consultation with the group's members, this approach uses a standardized scale to provide consensus-based scoring of the efficacy, safety, and affordability of a drug or a regimen and the quality and consistency of the evidence associated with that drug or regimen. Each of the 5 measures in the NCCN's approach is displayed as a solid block using a scale from 1 to 5, where 1 is considered least favorable and 5 is most favorable.7

The Memorial Sloan Kettering Cancer Center (MSKCC) DrugAbacus tool, conceived by Peter B. Bach, MD, MAPP, Director of the Center for Health Policy and Outcomes at MSKCC and launched in June 2015, targets physicians and policymakers (not patients as done by ASCO and NCCN) with an “interactive tool [that] takes more than 50 cancer drugs and lets you compare the company's price to one based on value.”8 This system delivers a value-based price for a drug that graphically represents the user's weighted preferences and estimated monthly costs relative to 52 cancer drugs.12

The Institute for Clinical and Economic Review (ICER), an independent nonprofit organization founded in 2005 by Harvard physician-researcher Steven D. Pearson, MD, MSc, launched its assessment program in July 2015, with guidance from an advisory committee of payers, patient organizations, physician organizations, and the biopharmaceutical industry. Targeting payers and policymakers (and not physicians as with the ASCO and NCCN frameworks and the DrugAbacus), ICER delivers a value-based price benchmark anchored in the real benefits that a specific drug brings to patients.9

The European Society for Medical Oncology (ESMO), a nonprofit professional medical oncology society that provides evidence-based recommendations for basic standards of cancer care, launched the Magnitude of Clinical Benefit Scale in May 2015 to assess new anticancer drugs approved by the European Medicines Agency. Similarly geared to payers and policymakers, this framework was designed to assist oncologists in evaluating the most effective anticancer medicines for their patients, providing a relative ranking for each drug on a magnitude of clinical benefit scale for curative and noncurative settings.10

Dissection and Cross-Comparison of the 5 Frameworks

Compiling a comparative cross-section of these frameworks from their source bodies,6–10 the methodology of each framework was further parsed into emphasis (Table 1), input (Table 2),5–10 scoring (Table 3),6–10 and output (Table 4).6–10

Table 2.

The 5 Value Frameworks: Inputs

	Value Frameworks6–10
Input	ASCO 2.0	NCCN	MSKCC	ICER	ESMO
Primary end points
Efficacy	Advanced disease: HR (death), OS, PFS, response rate Adjuvant therapy: HR (death), OS, DFS	Vary, dependent on indication	Improvement in OS or surrogate end point	Vary, dependent on indication	Advanced disease: OS, PFS, palliation of symptoms, response rate
Safety/toxicity	Based on side-effect frequency, grade	Effect on daily life	Grade 3/4; probability of discontinuing	Severe side effects	Grade 3/4; severe side effects
Secondary end points
Treatment-free interval	Yes	No	No	No	No
Tail of the curve	Yes	No	No	No	No
Quality of life/palliation	Yes	No	No	Yes	Yes
Patient preferences	No	No	No	No	No
Epidemiologic factors
Disease burden/incidence	No	No	Yes	Yes	No
Unmet need	No	No	Yes	No	No
R&D factors
Novelty	No	No	Yes	No	No
Research cost	No	No	Yes	No	No
Cost
Drug costs	Advanced disease: drug acquisition cost per month Adjuvant therapy: drug acquisition cost/entire treatment regimen	Total treatment cost	ASP/AWP	Total cost per person, total cost to payers	Not specified, left to payers to evaluate
Cost to healthcare system	No	Yes	Yes	Yes	No

ASCO indicates American Society of Clinical Oncology; ASP, average sales price; AWP, average wholesale price; DFS, disease-free survival; ESMO, European Society for Medical Oncology; HR, hazard ratio; ICER, Institute for Clinical and Economic Review; MSKCC, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network; OS, overall survival; PFS, progression-free survival; R&D, research and development.

Table 3.

The 5 Value Frameworks Scoring Algorithms

	Value Frameworks6–10
Scoring	ASCO	NCCN	MSKCC	ICER	ESMO
Formulaic/expert judgment	Formulaic	Expert-based	Formulaic	Formulaic and expert-based	Formulaic

ASCO indicates American Society of Clinical Oncology; ESMO, European Society for Medical Oncology; ICER, Institute for Clinical and Economic Review; MSKCC, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network.

Table 4.

The 5 Value Frameworks: Outputs

	Value Frameworks6–10
Output	ASCO	NCCN	MSKCC	ICER	ESMO
Health benefit	Net health benefit	Score (1–5) for each of 5 key measures displayed as Evidence Blocks	No	Assessment of care value (high/intermediate/low)	A relative ranking of the magnitude of clinically meaningful benefit
Cost readout	Directly reported as regimen cost (WAC or ASP) Advanced disease: drug acquisition cost per month Adjuvant therapy: drug acquisition cost for entire treatment	Reported as relative affordability, considers overall cost of intervention (eg, cost of drug, infusions, supportive care, management)	DrugAbacus value-based price per month or course of therapy; a user-generated value assessment directly compared with reported Medicare payment limit, 106% ASP	Cost per year; cost-effectiveness of drug, with recommendations on what drug price should be to be cost-effective	Not specified; left to payers to evaluate
Drug cost, relative or absolute value	Yes	Yes	Yes	Yes	No
Cost to patient	Yes	No	No	No	No
Cost to healthcare system	No	Total drug and medical costs	Rarity per budget impact	Incremental cost-effectiveness ratio and budget impact	No

ASCO indicates American Society of Clinical Oncology; ASP, average sales price; ESMO, European Society for Medical Oncology; ICER, Institute for Clinical and Economic Review; MSKCC, Memorial Sloan Kettering Cancer Center; NCCN, National Comprehensive Cancer Network; WAC, wholesale acquisition cost.

Together, this matrix has allowed us to draw comparisons between the frameworks' scope, capabilities, and limitations. Given the diversity in composite output(s) of each framework, direct cross-framework comparisons should be discouraged. Certain frameworks (eg, ASCO's) also discourage comparison within the framework, given the nature of the clinical trial input for each drug (ie, single-arm vs head-to-head clinical trial). Considering the systems collectively, several key recurring themes, which are outlined below, warrant further investigation, given their impact on the intended stakeholders' (ie, patients, physicians, and/or payers) adoption of the frameworks.

Inputs

Beginning with inputs (Table 2), randomized controlled trials, by the nature of their design to minimize selection bias, are often not representative of the demographic distribution of the actual patient population nor the practical choices that patients encounter.13 Thus, frameworks (eg, NCCN's and ICER's frameworks) that rely on consensus or combined analysis of multiple clinical trials and a variety of clinical end points may, in some circumstances, provide a better indication of the therapeutic value of a drug in the larger population.

A recent survey of 50 oncologists and 55 payers revealed a substantial lack of confidence in the ASCO Value Framework, which quantifies a single randomized controlled trial.14 Rather, survey responders' preferences leaned toward comparison across clinical trials and a measure of cost within the core net health benefit score.14 Notably, these factors are provided through the NCCN Evidence Blocks,7 considering that value is derived from the consistency of quality evidence across multiple sources filtered through the prism of accumulated clinical experience.

Lack of Real-World Evidence

Across the oncology value frameworks, there remains a lack of real-world evidence and ready access to subpopulation analyses across patient types. Indeed, conclusions from a workshop of patients, patient advocates, and pharmaceutical and medical device manufacturers indicated a need to better account for the heterogeneity of the patient population that exists outside of a randomized clinical trial, specifically the gap between the randomized sample set utilized for evidence generation and the subsequent variability of response among patient subgroups.15 An anonymous attendee at that workshop observed an absence of clinical trial design models providing either inputs or outputs for the consideration of patient heterogeneity, or even subpopulations,15 which is echoed by the survey of 50 oncologists and 55 payers discussed earlier.14

Outputs

Understanding the methodologies of the scoring algorithms (Table 3) is crucial to evaluating the value and limitations of the frameworks' outputs. In ASCO's framework, for example, the interchangeability of various primary end points, such as survival hazard ratio or overall survival with progression-free survival or recurrence rate, belies their subtle differences. Given the interplay between shared decision-making and the usability of information, Schwartzberg and colleagues have asserted that the information should be accessible to patients, as well as understandable and usable.16

Suboptimally, the framework and its outputs should at least be in a form that providers can easily relay to patients. However, from our interviews with payer and secondary research exploring usability, we note a dearth of investigation into patient and provider usability. Indeed, our informal discussions with physicians and payers regarding the analysis of value confirm a hesitancy in applying these frameworks in practice until they better understand how to apply and extract value from the frameworks' inputs. This sentiment was particularly expressed in regard to the ASCO and the NCCN frameworks.

The Evidence

Evidence supporting the specific fixed weights of certain variables, as well as their binary inclusion, is not available. Similarly, the logic supporting the potential ranges of variable weights (of MSKCC DrugAbacus and a future version of ASCO's framework) remain vague. For example, a side-by-side comparison of the ASCO and ESMO frameworks revealed disparity in criteria stringency.17 Similarly, the correlation between ASCO's net health benefit score and ICER's comparative clinical and incremental cost-effectiveness outcomes is low.18

Consequently, ESMO's or ICER's rating for a specific treatment is not predictive of the score it will receive under ASCO's framework, or vice versa. It has been suggested that clinical benefits and toxicities be evaluated for context by clinicians who have extensive and active expertise in the disease area being examined.16 However, it is unclear how this methodology is standardized, if at all.

Scientific Standards

Schwartzberg and colleagues have suggested that organizations authoring value frameworks should be held to scientific standards via peer review.16 They posit that the publication of their methodologies in relevant peer-reviewed journals would invoke authorship standards for scientific documents, resulting in greater clarity about the characterization process, terminology utilized, and the quantitative relation of outcomes to calculated results.16

This would allow the assumptions and methodology to be more transparently documented, explained, referenced, and evaluated for potential bias. For example, Wilson and colleagues revealed significant scoring reliability issues when they asked 8 clinicians to complete the ASCO Value Framework for 11 anticancer medications.18

Patient Perspective

How these frameworks will be patient-tailored remains unclear, because individual patient disease characteristics are not considered by the frameworks developed by ASCO, the NCCN, or ESMO.6,7,10 Stakeholders at the workshop mentioned earlier noted that although value to the individual patient remains the most important consideration in any patient-centric treatment assessment tool, none of the current value frameworks considers short- or long-term value from the individual patient's perspective.15

Total Cost of Care

Beyond limited individual drug cost reporting or comparative assessment of treatment options, the final framework outputs (Table 4) are often of little help in defining the total cost of care, given that pharmaceuticals represent a mere 5% to 20% of the total cost of cancer treatment.19 Neither ASCO nor MSKCC considers other medical costs, such as reducing the need for surgery or hospitalization.6,8 Conversely, the NCCN defines its affordability measure as the overall cost of an intervention, including the drug, infusions, supportive care, toxicity monitoring and management, and the probability of care being delivered in the hospital.7 To an ever-greater degree, ICER considers the total cost per patient along with the aggregate cost.9

However, in none of these frameworks, not even the patient-oriented frameworks, is affordability considered at the patient-tailored level.6–10 As such, how can providers assess what is affordable, especially because an individual patient's budget impact and tradeoffs, length of treatment, and considerations related to insurance coverage maximums, copays, and dynamic formulary inclusion and exclusion criteria may vary between patients, as well as within the treatment period for that patient?15

Implications for Payers and Physicians

Healthcare stakeholders have adopted a wait-and-see attitude as the different frameworks iterate and develop followings. Our structured interviews with payers engaged in evaluating oncology drugs over the past 2 years and spanning more than 100 hours of discussion indicate a split in payer expectations of these frameworks to influence oncology drug assessments in the near future. This split was confirmed through an aggregate analysis of 3 separate studies between 2015 and 2016 comprising a total of 101 payers, where a mere 53% expect that these frameworks will influence their assessment of the value of oncology drugs in the near future.20–22 In addition, our dialogues with payers spanning more than 100 hours of discussion further confirm the findings of a recent study, demonstrating payer familiarity with at least ASCO's and NCCN's value frameworks.22

Our structured interviews with US physicians engaged in evaluating oncology drugs, over the past 2 years and spanning more than 100 hours of discussion, similarly confirm the results of previous research comprising 93 medical or hematologic oncologists, which showed that the greatest awareness among physicians remains with the ASCO and the NCCN frameworks.23

Although the use of value frameworks in oncology is increasing in clinical practice,5 especially regarding the ASCO framework, physicians have noted in discussion with us that the frameworks will be more useful in clinical practice in the future, as they become more established and their outputs are more widely accepted. Indeed, these physicians indicated to us that they were beginning to consider how to utilize value frameworks in the near future, emphasizing the importance of providing a comparative assessment of various treatment options available and their relative financial implications to patients.

This was confirmed through our combined analysis of 243 physicians from surveys collected in 2016, in which 51% of providers indicated they were considering utilizing value frameworks in their practice compared with 11% who were not considering utilizing them, and 38% who were unsure.20,23

Implications for Drug Manufacturers

Although a dominant value framework to guide the pharmaceutical industry has yet to emerge, the input of patients, providers, and payers, coupled with the current approaches for assessing value through the frameworks, can provide the drug manufacturers with additional considerations in modeling value in the analysis of their clinical and commercial strategy in oncology. However, the pharmaceutical industry would be wise to remain current with these value frameworks and their potential updates, as well as contribute to discussions with framework bodies (ie, ASCO, NCCN, ESMO) and seek resources to understand payer and prescriber perspectives about these models.

Integrating comparative analysis methods into strategy, risk management, and value modeling may allow pharmaceutical manufacturers to better position their oncology assets at all stages of development. Integrating value framework considerations into clinical trial design (ie, preliminary or interim clinical data that hint at value outputs) can provide an additional tool for modeling their oncology drugs' value and the value of the associated clinical trials or subsequent forecast clinical trials, even before the trial is complete. Standardizing appropriate clinical trial comparators would also reduce the incomplete inputs that are crucial for internal value framework analysis and external analysis by various stakeholders, particularly payers.

Incorporating value framework considerations may further assist drug makers in their internal price modeling of an oncology drug when developing drug launch price strategies. Insight into the attitudes of payers toward individual value frameworks, as well as toward specific aspects of a particular framework, may be vital to developing optimal drug pricing and reimbursement strategies (eg, premium pricing).

By understanding the meaning of the output(s) generated by each framework, as well as the value of each framework to the various stakeholders, pharmaceutical companies have an opportunity to selectively utilize these frameworks to shape their value modeling in the analysis of their clinical and commercial strategies.

In addition, value framework assessments may prove valuable in identifying innovative access strategies to a specific therapy for access to favorable situations. Even with these potential benefits, stakeholder education through outreach strategies will still be crucial when communicating with various healthcare stakeholders, including payers, physicians, and patients; such context will be essential for understanding why one or several value frameworks are favored over others when reaching specific conclusions.

Conclusions

Our findings suggest that payers are aware of the frameworks and see the potential in them, but they are split on whether they will utilize the frameworks in the near future. Likewise, physicians are considering the use of these frameworks, yet they do not fully appreciate their applications and limitations in decision-making. Consequently, although incorporating elements of these algorithms may assist pharmaceutical companies in internal pricing and reimbursement strategies, our dialogues with pharmaceutical stakeholders confirm our primary and secondary research findings among physicians and payers, regarding the uncertainty as to which framework will gain the greatest traction among stakeholders, whether it is the patient, provider, or payer.

Although each of the 5 value frameworks has selectively targeted patients, physicians, and/or payers using a unique interpretation of value, the relevance of the 5 frameworks to direct (ie, providers, payers) and indirect (ie, drug manufacturers) stakeholders will depend on adoption of the framework. Factors that may drive value framework adoption include appreciation for real-world evidence, the relation of cost to affordability, transparency regarding the quantification process and outputs, and, where stakeholder-relevant, the framework's applicability to individual decision-making. Which value framework becomes more established and widely accepted, and by what stakeholder, will influence how the pharmaceutical industry will shape the clinical and commercial development of its oncology drugs.