Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations

Abstract

OBJECTIVES:

This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures which are often underutilized. We also present a set of “core outcome measures” that may be considered in future oncology alternative payment models (APM).

STUDY DESIGN:

Our research consists of a focused literature review, content analysis and a quality measure synthesis and categorization.

METHODS:

We present a focused literature review to generate key evidence on quality measures in oncology. We study 7 oncology quality assessment frameworks, encompassing 142 quality metrics, and synthesize our recommendations using the Centers for Medicare and Medicaid Services APM toolkit.

RESULTS:

We present 34 outcome-based oncology quality measures for consideration, classified into five domains: clinical care (e.g., hospital and emergency department visits, treatment effectiveness, and mortality), safety (e.g., infections and hospital adverse events), care coordination (for hospital and hospice care), patient and caregiver experience, and population health and prevention. Both general and indication-specific outcome measures should be considered in oncology APMs as appropriate. Utilizing outcome-based measures will require addressing multiple challenges, ranging from risk adjustment to data quality assurance.

CONCLUSIONS:

Oncology care will benefit from a more rigorous approach to quality assessment. The success of oncology APMs will require a robust set of quality measures that are relevant to patients, providers and payers.

Introduction

Several alternative payment models (APMs) are being piloted to address affordability, equity and quality-of-care challenges in oncology care. Of these arrangements, the Oncology Care Model (OCM), developed by the Centers for Medicare and Medicaid Innovation (CMMI), is among the most extensive, covering about 200,000 chemotherapy episodes annually. The OCM relies on multiple quality measures to determine the level of payment for each provider, with the goal of incentivizing higher-quality care in a cost-effective manner. The OCM’s payment design is described in Box 1.

Box 1: Oncology Care Model.

The Centers for Medicare and Medicaid Innovation launched the Oncology Care Model (OCM) on July 1, 2016 with an objective to improve: 1) care coordination; 2) appropriateness of care; and 3) access for beneficiaries undergoing chemotherapy. It consists of a flat per-beneficiary-per-month payment, and a performance-based payment, whose level is set based on a practice’s performance in the specific quality measures relative to a matched comparison group. OCM is set to run from July 2016 to June 2021. It applies to both Medicare Fee-for-Service (OCM-FFS) beneficiaries and patients covered by other payers (OCM-OP).

The flat monthly payment aims to cover “care management services for Medicare beneficiaries in a 6-month OCM Episode of Care triggered by the administration of chemotherapy,” totaling $160 per beneficiary per month. In contrast, the performance-based payment is an episode-based, risk-adjusted payment calculated based on the participant’s achievement on a range of quality measures, such as a reduction in all-cause hospital admissions or improved adherence to clinical guidelines in some cancer types. OCM draws on both process and outcome-based quality measures, with the former relatively more represented. Outcome-based measures used in OCM range from all-cause hospital admissions to emergency department visits, mortality after more than 3 days in hospice, and patient-reported experience of care.

Despite innovations in the payment landscape, limited consensus exists about what constitutes indispensable quality measures in oncology. The absence of such consensus may not only limit the development of better payment models, which increasingly link payment to quality of care, but may also result in a lack of agreement on how value should be defined (and demonstrated) in an era of innovative, ever more expensive cancer therapies. In 2017, a national expert roundtable recommended that policy makers “prioritize and develop effective cross-cutting measures that assess clinical and patient-reported outcomes, including shared decision making, care planning, and symptom control” and highlighted an “overreliance on condition-specific process measures.”¹ Disease-specific quality measures have been developed, but there is a lack of consensus on what quality measures ought to be utilized across multiple cancers, especially when measuring clinical outcomes.² As oncology APMs evolve, there are practical considerations in the design and implementation of outcome-based measures.

The development of new quality measures for oncology has been underway for many years. For example, the Centers for Medicare and Medicaid Services (CMS) with the America’s Health Insurance Plans (AHIP) developed Core Quality Measures in eight therapeutic areas, including medical oncology, to assess provider performance.³ This initiative includes quality indicators focusing on breast cancer, colorectal cancer, prostate cancer and, more generally, end-of-life care.⁴ It also identified areas for future measure development in oncology, including pain control, and hospital admission and five-year cure rates,⁴ and highlighted challenges related to data access and measurement as standard of cancer care progresses, requiring frequent reassessments.⁴ In parallel, the American Society for Radiation Oncology (ASTRO) has been working with the American Society of Clinical Oncology (ASCO) to develop measures “for utilization by both organizations in various quality programs and reporting environments”.⁵ However, no comprehensive set of core outcome-based quality measures in oncology has been published.

Research on health care quality measures typically differentiates between two key categories: process-based measures, which focus on proper reporting and procedure execution, and outcome-based measures, which involve clinical outcomes and patient-reported experience of care. This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures which are underutilized given their perceived benefits. We also present a set of “core outcome measures” that may be considered in future oncology APMs. Our study does not aim to provide a definitive list but, rather, to present a diverse set of outcome measures most commonly included in quality initiatives and payment models in oncology.

Methods and data

Our research consists of a focused literature review, content analysis and measure categorization, similar to Macefield et al. (2014).⁶ First, our focused literature review summarizes key evidence related to quality measures in oncology, with an emphasis on classification, their unique advantages and disadvantages, and the challenges related to implementation in clinical practice. Second, our descriptive analysis of the most commonly used quality measures in oncology draws on a convenience sample of existing payment models and other quality assessment frameworks. Our sample includes 7 oncology quality assessment programs, frameworks and payment models, which encompass 142 quality measures:

• The Oncology Care Model (OCM) by the Centers for Medicare and Medicaid Innovation (CMMI)

• Quality Oncology Practice Initiative (QOPI) by American Society of Clinical Oncology (ASCO)

• Prospective Payment System by CMS (Exempt Cancer Hospital Quality Reporting (PCHQR) Program)

• Core Quality Measures Collaborative Core Sets (CMS/America’s Health Insurance Plans)

• Oncology Medical Home (OMH) program (Community Oncology Alliance)

• National Osteoporosis Foundation (NOF) and National Bone Health Alliance (NBHA) Osteoporosis Quality Improvement Registry (QIR)

• Qualified Clinical Data Registry by the Oncology Nursing Society (ONS).

We categorize these measures into process- vs. outcome-based and analyze their frequency. Given our primary focus on outcome-based quality measures, we expand our literature review to include the evidence base for each of the key outcome measure categories identified. In addition, we review and summarize published reports on the impact of emerging oncology APMs on clinical outcomes and spending. We conclude with a synthesis of existing evidence on key outcome-based measures and their appropriateness in future oncology APMs. Finally, we discuss directions for customization and further validation of oncology core outcome measures.

Results

Advantages of outcome- and process-based measures

Previous scholarship finds that both process- and outcome-based quality measures have advantages and disadvantages (Table 1). For example, it is generally easier to generate actionable feedback based on process-based measures and there is mostly no or limited need for risk adjustment (unlike the case for quality measures such as mortality, where complex case-mix, indication and disease stage adjustments are often required).⁷ In addition, data collection for process measures is generally faster, can draw on smaller sample sizes and does not require advanced statistical analysis to yield practical results.⁷

Table 1:

Key advantages of process- and outcome-based quality measures

	Process measures	Outcome measures
Actionable feedback for quality improvement	✓
No or limited risk adjustment	✓
Rapid data collection	✓
Evidence available to support measure development		✓
Low cost of testing measure validity		✓
Value to patients and non-physicians		✓
Ease of defining comprehensive measures		✓
Improvement in measure leads is linked to better care		✓

Source: Rubin et al. (2001)⁷

In turn, outcome measures are generally based on clinical endpoints with proven significance in the quality of care. They are better understandable by patients and non-clinicians and are easier to define comprehensively (e.g. hospice admissions for at least 3 days prior to death).⁷ Relatedly, an improvement in process measures may be a useful step in care coordination, but may not always have an observable effect on the improvement in clinical outcomes, especially when included for billing purposes only.⁸ Given these realities, the Agency for Health-care Research and Quality (AHRQ) regards outcome-based measures as the “gold standard” in quality measurement.⁹ Expert groups such as the Health-care Association of New York State suggest that “regulators and payers […] focus on overall performance (outcome measures), and defer the operations and use of process measures for internal quality improvement by health-care providers.”¹⁰

Process-based measures dominate the OCM and other oncology quality assessment frameworks, yet outcome-based measures have an important role to play. Outcome-based measures are directly connected to real-world outcomes, ranging from hospital admissions to mortality and patient-reported outcomes, reflecting what patients and providers care about most.

Outcome measures in existing oncology quality frameworks

Of the 142 quality measures from 7 oncology APMs we reviewed, 80.3% were process-based measures and 19.7% were outcome-based measures. An earlier analysis of the National Quality Measures Clearinghouse (NQMC) found an even lower proportion of outcome-based measures (7.1%), based on 1,958 quality indicators from a wide range of therapeutic areas.⁸Of the nearly 2,000 indicators, only 1.6% are patient-reported outcome measures.⁸

We condensed the 28 outcome-based measures into 23 “unique” outcome measures by merging identical or near-identical measures and grouping them into five categories: 1) admissions and hospital visits (including emergency department visits), 2) hospice care, 3) mortality, 4) patient-reported outcomes, and 5) adverse events (Table 2)

Table 2:

Outcome measures identified in oncology APMs

Category	Unique Outcome Measures
ADMISSIONS AND HOSPITAL VISITS	Admission to ICU in last 30 days of life
	Admissions and ED visits
	All-cause ED visits
	All-cause hospital admissions
	ED visits in last 30 days of life
	Hospitalization in last 30 days of life
	Unscheduled readmissions within 30 days
HOSPICE CARE	Hospice admission for over 3 days
	Hospice admission for under 3 days
	Hospice enrollment
	Hospice enrollment or palliative services
	Length of hospice care
MORTALITY	Death outside of a hospice
	Deaths in acute care setting
	Died after 3+ days in hospice
	Died in extended care facility (with hospice/palliative care)
PATIENT-REPORTED OUTCOMES	Fatigue
	Patient-reported experience
ADVERSE EVENTS	Inpatient hospital-onset clostridium difficile infections
	Inpatient hospital-onset methicillin-resistant staphylococcus aureus (MRSA) bacteremia
	Healthcare-associated, catheter-associated urinary tract infections
	Surgical site infection
	Central line-associated bloodstream infections

Source: Authors’ analysis of a convenience sample of quality assessment frameworks in oncology.

Admissions and hospital visits

Admissions and hospital visits, after risk-adjustment, are important indicators of the appropriateness and timeliness of care, as up to 50% of emergency department visits are related to complications from chemotherapy, potentially indicating sub-optimal management of the disease and care coordination (ranging from information sharing among providers to education about end-of-life care).¹¹

Significant variation observed in admission rates and hospital/emergency department visits between different providers, even when controlling for other factors, has spurred research related to avoidable hospitalizations and appropriateness of care, especially in late-stage cancer care.¹²

In 2016, for example, CMS announced the inclusion of inpatient admissions and emergency department visits for patients receiving outpatient chemotherapy in its Hospital Outpatient Quality Reporting (HOQR) Program.¹³ Including hospital visits in payment models aims to “encourage reporting facilities to take steps to prevent and improve management of side effects and complications from treatment.”¹⁴

Hospice care

Though it offers patients, caregivers, and the health care system advantages relative to hospital settings, hospice care is generally underutilized. Quality measures related to hospice care may improve the quality of life of late-stage cancer patients, reduce spending, and reduce burden among providers and caretakers.

Allowing cancer patients to receive palliative care in a hospice setting is traditionally associated with improvements in quality of life as well as system efficiencies. Yet up to 66% of cancer patients are not enrolled in hospice in the last 30 days of life, and less than 29% are enrolled for at least 2 months (considered appropriate care), based on an international review of 78 studies published between 1998 and 2011).¹⁵ One factor explaining the sub-optimal transition to palliative care may be doctors’ tendency to overestimate survival prospects of a patient.¹⁶ Hospice-related measures aim to reduce wasteful spending on care that is unlikely to improve clinical outcomes and to provide patients with a higher quality of life. Specific provisions may be needed for palliative chemotherapy and other treatments that can extend the length of life while in palliative care.

While outcome-based measures in this domain tend to focus on hospice settings alone, recent discussions suggest that palliative care may improve patient quality of life if initiated earlier in the cancer treatment course.^17,18 In 2012, the National Quality Forum endorsed 14 measures related to end-of-life care, of which several are outcome-based, including ‘comfortable dying’ and bereaved family survey measures.¹⁹ Finally, patient- and caregiver-reported outcomes may have a more prominent role to play as end-of-life care should reflect patient and caregiver preferences.²⁰

Mortality

Mortality is a common outcome indicator used in both clinical practice and clinical trials, and may be reported in different ways (patient mortality over a specific period, overall survival, progression-free survival, as well as by the setting of a patient’s death). Additionally, the setting of death (in a hospital vs. at home or in a hospice) may play an important role in the patients’ quality of life,^15,21 and is sometimes used as part of mortality-related quality measures.

Patient-reported outcomes

Patient-reported outcomes (PROs), which range from pain to social function evaluation, are increasingly used to evaluate appropriateness of care given their ability to reflect patient needs and preferences, which may vary significantly.

The FDA issued a PRO-specific guidance in 2009,²² defining PROs as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”, and states that in general, “findings measured by a well-defined and reliable PRO instrument in appropriately designed investigations can be used to support a claim in medical product labeling if the claim is consistent with the instrument’s documented measurement capability.”²²

Patient-reported outcomes are relevant in both early and late disease stages, complementing other measures that address outcomes in a hospital setting. In 2016, a multi-stakeholder roundtable on “Improving Oncology Measurement” recommended that PROs be collected “before, during, and after treatment.”²³ While more evidence is needed, patient-reported outcome measures (PROMs) have been studied for use during an initial consultation and during shared decision-making regarding patient care, aside from tracking treatment progress and patient satisfaction.²⁴ A 2013 report on PROs by the National Quality Forum indicates that several guiding principles for selecting PROMs should be followed: psychometric soundness, person-centricity, meaningfulness, amenability to change, and implementability.²⁵ Recent progress in drawing on “electronic patient-reported outcomes” (ePROs) may catalyze the ability of plans to systematically and reliably collect patient- and caregiver-reported outcomes, especially if they impose minimal burden on staff and patients.²⁶

Nonetheless, the use of PROs has been associated with multiple challenges, including representativeness, inclusion of patient-reported outcomes in medication labels, necessity for both standardized and customizable PROs, as well as operational and organizational barriers to collecting and analyzing them.²⁷ In addition, PRO collection is often resource-intensive, validity of disease-specific PROs may be limited, and many PROs lack predictive value.²⁷ Despite these challenges, PROs can be useful tools to obtain insight into patient needs and preferences in order to make better patient-level as well as policy decisions, and to support further research and development.

Adverse events

Quality measures based on reporting of adverse events aim to lower the number of avoidable incidents, potentially shortening hospitalization length, reducing costs as well as mortality.

Initial guidance on the reporting of adverse events in oncology trials was published by the Consolidated Standards of Reporting Trials (CONSORT) in 2003 and adverse events related to oncology care are now understood better.²⁸ However, adverse events in clinical settings are thought to be significantly underreported, partly driven by voluntary reporting and the use of instruments that may be prone to lower sensitivity.²⁹ Improvements in documentation and electronic reporting are expected to improve the reliability of data about adverse events observed in clinical practice (most hospitals do not use electronic health records to “directly measure [or record] patient harm”).³⁰

If implemented broadly, the magnitude of potential improvements may be significant: studies have shown that adverse events can extend the length of hospitalizations, increase costs of care, and increase mortality up to two-fold.³¹

Evidence from early quality initiatives in oncology

Although no formal evaluation of OCM has taken place, limited evidence suggests that previous quality initiatives with outcome-based components have improved care while reducing costs. For instance, during a two-year pilot in Texas involving 221 oncology patients (Innovent Oncology program by McKesson Specialty Health, Texas Oncology (TXO) and Aetna), savings of over $0.5 million were achieved.³² The Program has been shown to improve adherence with clinical pathways and clinical outcomes: pathway adherence has increased from 63% to 76%, reductions in emergency department visits, hospital admissions, and hospital days of 48%, 34%, and 44%, respectively, were observed, and average in-patient days decreased from 2.1 to 1.2 days.^32,33 Innovent Oncology based its value-based reimbursement on three pillars: 1) Level I Pathways Program (aiming to increase the use of evidence-based treatment guidelines), 2) clinical benchmarking (based on a number of quality indicators), and 3) contract negotiation services.³⁴ Among the quality measures included have been gastrointestinal toxicities, infection, thromboembolic events, pain, and depression.²⁰

Similarly, an oncology pilot by United Health-care which drew on episode payments for over 800 breast, colon- and lung cancer patients in 5 oncology practices achieved net savings of more than $33 million (a 34% reduction of the predicted total medical cost).³⁵ Some of the key quality measures used by this pilot included emergency room and hospitalization rates, admissions for cancer symptoms, febrile neutropenia occurrence rate, admissions for treatment-related symptoms, days from last chemotherapy to death, and hospice days for patients who died.³⁵

However, a lack of a counterfactual (via a matched control group, for example) undermines a direct causal link between quality measurement and observed outcomes in these pilots, and more comprehensive evaluations are still needed.

Recommendations for outcome-based measures in oncology

We present a synthesis and recommendations for future core outcome sets in oncology in Table 3. We include measures that are generally seen as closely tied to the quality of care received by oncology patients. We classify these measures into five quality domains identified by a Center for Medicare and Medicaid Innovation APM toolkit³⁶: clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention. Where possible, this set of outcome measures should be tailored to unique patient populations, diseases, providers or other factors in individual payment models. In addition, some measures such as hospice care, albeit appropriate for patients with more advanced disease, may not be relevant for patients with curable, early stage cancer. Future oncology APMs should implement outcome measures relevant to the disease type and stage(s). For a detailed justification and discussion of individual categories and measures, please see the Appendix.

Table 3.

Proposed Outcome-based Quality Measures in Oncology

Clinical Care	Hospital and ED Visits	Treatment Effectiveness	Mortality
	• All-cause ED visits • All-cause hospital admissions • Unscheduled readmissions within 30 days • Hospice enrollment or palliative services	• Response rate • Progression-free survival • Overall survival	• Deaths in acute care setting • Death outside of a hospice • Died after 3+ days in hospice • Died in extended care facility (with hospice/palliative care)
Safety	Infections	Hospital Adverse Events
	• Inpatient hospital-onset clostridium difficile infections • Inpatient hospital-onset methicillin-resistant staphylococcus aureus (MRSA) bacteremia • Healthcare-associated, catheter-associated urinary tract infections • Surgical site infection • Central line-associated bloodstream infections	• Patient Safety Indicators  ○ Pressure ulcer rate  ○ Retained surgical item or unretrieved device fragment count  ○ Iatrogenic pneumothorax rate  ○ In-hospital fall with hip fracture rate  ○ Perioperative hemorrhage or hematoma rate  ○ Postoperative acute kidney injury requiring dialysis  ○ Postoperative respiratory failure rate  ○ Perioperative pulmonary embolism or deep vein thrombosis rate  ○ Postoperative sepsis rate  ○ Postoperative wound dehiscence rate  ○ Unrecognized abdominopelvic accidental puncture/laceration rate  ○ Transfusion reaction count
Care Coordination	Hospital Care	Hospice Care
	• ED visits in last 30 days of life • Hospitalization in last 30 days of life • Admission to ICU in last 30 days of life	• Hospice admission for over 3 days • Hospice admission for under 3 days • Mortality after more than 3 days in hospice • Percentage of deaths at home or in hospice, versus in hospital • Length of hospice care
Patient and Caregiver Experience	Patient-reported outcomes	Caregiver-reported outcomes
	• Symptomatic adverse events • Physical function • Disease-related symptoms	Caregiver burden • Caregiver need • Quality of life
Population Health and Prevention	• Stage of cancer diagnosis • Median time to diagnosis resolution • Time from diagnosis to the initiation of treatment

Collecting outcomes data in all five domains of cancer care is fraught with challenges that have been documented in multiple studies. For example, to measure and track outcomes properly, programs often require “big data” that involve multiple sources such as EHRs, health insurance claims, and patient/caregiver surveys, but whether data are complete and accessible and can be translated into clinical practice remains a challenging issue.Adibuzzaman, DeLaurentis ³⁷ Many outcome-based measures rely on administrative claims data, which tend to have a long report lag. Some outcomes data, such as hospice care, may be challenging to access, especially when the patient is transferred from one payer to another. Chung and Basch (2015)³⁸ discuss specific challenges related to collecting and using patient-generated health data (including PROs), ranging from “provider concerns, workflow issues, standardization of patient-generated health data and interoperability of devices/sensors, security and privacy issues” to a “lack of the necessary EHR functionalities and software innovations.” Additionally, statistical challenges related to missing values, highly dimensional datasets and confounding (bias) require robust statistical approaches which are not yet available in broad clinical practice.³⁹ Nonetheless, new approaches are being tested as outcome measures gain support from clinicians, patients and payers, including a collaborative pilot on “Establishing a Framework to Evaluate Real-World Endpoints” in advanced non-small cell lung cancer led by the Friends of Cancer Research and supported by both public and private stakeholders.⁴⁰

Conclusions

As highlighted in this paper, both OCM and other quality initiatives in oncology rely on process or outcome-based quality measures to determine the quality of care and – in some cases – determine level of payment. Given evidence from the literature and an analysis of 7 oncology APMs, we present a set of outcome-based measures that should be considered in future payment models in oncology. While some measures may be omitted in specific cases, we believe the inclusion of measures related to all five domains – clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention – is highly desirable in future oncology APMs. Selective measurement of one outcome domain may create perverse incentives for providers to improve performance by underutilizing appropriate care and jeopardize optimal patient outcomes. Where appropriate, indication-specific quality measures should be included to account for quality-of-care complexities associated with individual cancer types and disease stages.

Overcoming hurdles to broader utilization of outcome-based measures in oncology will require a consensus between both payers and providers. These efforts should highlight the benefits of implementing outcome-based measures in oncology APMs (especially relative to the cost of implementation) and solutions to data and evaluation challenges (including risk-adjustment and bias control). Future research is also need to develop best practices for the inclusion and implementation of outcome measures in oncology clinical pathways.⁴¹ Additional considerations include developing strategies for quality control, dispute resolution and administrative burden on providers and payers.

Given the steadily increasing costs of oncology care and, in some cases, the availability of multiple high-cost treatment options for individual cancer patients, oncology care is in need of a more rigorous approach to quality assessment. The success of emerging oncology APMs will depend on a robust set of quality indicators that are relevant to patients, providers and payers alike.