Using Patient-Reported Outcome Measures as Quality Indicators in Routine Cancer Care

Patient-reported outcome (PRO) measures are increasingly being used in routine cancer care and may soon be used as indicators of the quality of care received. When a clinician reviews PRO measures with a patient during a visit, the patient has an opportunity to elaborate on symptoms that may need to be managed.¹ Reviewing PRO measures with cancer patients increases communication between clinicians and patients, detection of problematic symptoms, and satisfaction with care.^1,2 PROs are a critical tool for increasing patient centeredness in oncology clinical trials, comparative effectiveness research, and cost-effectiveness research. There also may be a future role for PROs in quality monitoring and improvement.

Historically, PRO measures assessing symptoms and physical functioning have not been included in quality-of-care assessments.^3,4 More typically, quality of care is measured with administrative data, such as infection or hospital readmission rates.⁵ Yet symptoms and physical function decrements are far more common than hospital complications among patients with cancer,^6,7 and they often go undetected by clinicians.^8,9 PRO measures are the preferred approach for assessing symptoms¹⁰ and have the potential to provide quality assessments that are meaningful and useful to patients and other stakeholders making health care decisions contingent on differential quality at the provider, practice, and system levels.

In the current issue of Cancer, Smith et al¹¹ describe the rationale for national cancer PRO surveillance and an innovative methodology from the Patient Reported Outcomes Symptoms and Side Effects Study (PROSSES). PROSSES is a landmark achievement because it is the first PRO study to use the Commission on Cancer’s Rapid Quality Reporting System to identify patients and manage data flow from 17 programs participating in the National Cancer Institute’s Community Cancer Centers Program. Their novel methodology was successful, with greater than 90% eligibility and greater than 60% response rates across most clinical and demographic characteristics. Smith et al were also able to overcome privacy, feasibility, and logistical issues that plagued prior studies. This study underscores the importance of symptom management in quality assessment and provides valuable data and methodology toward future work.

Although this is a significant step forward, there is more work to be done. The next steps for future research are to develop standardized methods to overcome remaining challenges:

1. Establish what outcomes are appropriate for quality measurement on the basis of meaningfulness to patients and clinical actionability (ie, whether an efficacious treatment is available).

2. Establish whether existing PRO measures can adequately be used as indicators of care quality.

3. Develop standards informing how PRO quality measurement at the practice level should be adjusted for differing characteristics of patients and hospitals to promote fair comparisons (case mix and risk adjustment).

4. Develop optimal methods for collecting and reporting PRO quality information directly from patients in oncology practices (eg, infrastructure and storage).

First, formative work is needed to identify areas of health care delivery that are important to patients and clinicians with respect to quality of care and are amenable to assessment with PRO measures. For instance, would it be more meaningful to assess quality with outcome measures (the proportion of patients reporting a symptom), process measures (the proportion of patients whose self-reported symptoms were collected and reviewed during routine visits), or both? The American Society of Clinical Oncology (ASCO), which develops and administers quality measures broadly in the United States, established a dedicated PRO workgroup to answer this question.

The ASCO workgroup determined that it was important to evaluate both process and outcome measures and developed a standardized approach to creating and testing patient-reported quality of care measures, which are called patient-reported outcome–based performance measures (PRO-PMs).^3,4 ASCO developed 3 initial illustrative measure specifications that are meaningful to patients and clinicians and have clinical actionability^3,4:

1. Outcome: the proportion of patients receiving moderately or highly emetogeneic chemotherapy who experience moderate to high levels of nausea within 1 week of chemotherapy administration.

2. Outcome: the proportion of patients with metastatic cancer who report moderate or greater pain.

3. Process: the proportion of patients receiving chemotherapy or biologic therapy who are asked to self-report symptoms at or within 2 weeks of their last visit.

Research is needed to examine the sensitivity, validity, and reliability of these PRO-PMs to detect quality differences across providers and practices.

Second, it is not known whether existing PRO measures can be valid and reliable indicators of quality. Smith et al¹¹ assessed pain, fatigue, and emotional distress with Patient Reported Outcomes Measurement Information System (PROMIS) measures, which have undergone extensive testing and are valid for assessing symptoms across a number of health conditions.¹² However, PROMIS measures have not been evaluated as quality indicators. Similarly, the recall period of PROMIS items was adapted to “past month” to better coincide with treatment (instead of “last 7 days”). It is impressive that cognitive testing was performed for the adapted items with 9 diverse survivors to ensure that the questions were comprehensible. However, interpretation of the results may be unclear because patients’ accuracy of recall for symptoms decreases over time.^13,14 Recall tasks suggest that patients have increasing difficulty remembering symptom levels beyond 1 to 2 weeks.¹⁴ Future research will need to determine which PRO measures are valid and reliable indicators of quality and what time frames are useful for making quality comparisons between providers. Future quality assessment will likely follow the lead of clinical trials and comparative effectiveness research in recommending standardized lists of symptoms and PRO measures that have been validated in cancer populations.^15,16

It may also be important to establish metrics in the future to evaluate whether patients perceive that their symptoms are well managed or appropriate to their particular clinical contexts as a companion to the symptom-oriented PRO measures. This is relevant because a patient’s symptoms may not always be well controlled, but he or she may perceive that the care received and symptoms levels are acceptable under the circumstances. For example, a patient may tolerate slightly more pain to take less of a narcotic drug and thereby be more lucid during the day or experience less constipation.

Third, there are no standards informing how PRO quality measurement should be adjusted for the differing characteristics of patients and hospitals to promote fair comparisons (case mix and risk adjustment). It will be critical for practices to collect patient-level characteristics (demographic and clinical) and to submit hospital characteristics (eg, number of beds and rurality) because PROs will need to be adjusted to make quality comparisons across providers. For instance, the impact of control variables will need to be estimated, standardized scores will need to be created to compare each provider’s actual and estimated outcomes with a common comparator, and scores will need to be aggregated to the practice and hospital-system levels.¹⁷

Finally, optimal methods for collecting and reporting PRO-PM information directly from patients in oncology practices are not established. For instance, research is warranted to examine whether PRO-PMs should be collected by individual clinics or from a centralized (nonaffiliated) organization. Optimal infrastructure, storage, and implementation questions will also need to be answered. Data collection approaches for maximizing response rates are also necessary, especially for underserved populations that are at risk for poor outcomes.¹⁸ Smith et al¹¹ attained a 60% response rate, but non-Hispanic African American patients were less likely to respond. More personalized contact may be desirable initially, particularly among older African American participants, to increase survey participation.¹⁹

A state-level approach exists that could help inform the next stages of development of PROSSES. Minnesota Community Measurement (MNCM) is a nonprofit quality reporting and measurement system that has been publicly reporting health care quality results since 2003.²⁰ MNCM uses a consensus-based, multistakeholder process to develop quality standards and select measures (including PROs), many of which are endorsed by the National Quality Forum. In 2008, Minnesota’s Health Reform Law directed the Commissioner of Health to establish a system of quality incentive payments based on performance targets and care improvement over time. All health care practices (any type) in Minnesota are required to submit deidentified medical record data to the state through a secure online portal. Most of MNCM’s work to date has not been in cancer, but its work in primary care, psychiatry, and orthopedic practices is informative. For example, MNCM has publicly reported depression care quality since 2010.²⁰ Currently, they report on 7 measures related to major depression, including the PRO measure called Patient Health Questionnaire 9 (PHQ-9). High-quality depression care is defined as a PHQ-9 score <5 (on a 1–27 metric) for adult patients at 6 and 12 months who initially scored 9 or higher and had an International Classification of Diseases, Ninth Revision code for major depression (bipolar disorder is excluded).²⁰ MNCM created a process to combine data from each health plan, adjust for the characteristics of patients and clinics, and publicly report the results by medical group. Health plans in Minnesota are now using this information as a quality indicator in their pay-for-performance programs.²⁰ MNCM is currently working with the ASCO toward the development and testing of cancer-specific PRO quality measures.

To use PROSSES for similar PRO quality-improvement initiatives, eligibility criteria would need to be restricted to active cancer treatment times such as 2 to 8 months after diagnosis. Smith et al¹¹ note that PROSSES may be less suited for studies of cancer patients undergoing active treatment because the median time from diagnosis to sampling was 9.5 months after diagnosis for the first cohort (range, 4.3–13.4 months) and 7.5 months for subsequent samples (range, 4.2–13.8 months).

In summary, PROSSES and the Commission on Cancer’s Rapid Quality Reporting System have taken a considerable step toward developing a patient-centered approach to assess quality of care. Before national implementation, methodological advancements for using PRO measures as metrics for quality of care, such as strategies for adjusting patient- and hospital-level characteristics to promote fair quality comparisons, are necessary. There is momentum to advance the science of patient-reported data in quality assessment for oncology. Anticipated uptake of PRO-PMs is expected to be high because national organizations are interested in using PRO measures as metrics for quality of care.³