Patient-Reported Outcome–Based Performance Measures for Older Adults with Multiple Chronic Conditions

Short abstract

RAND analysts tested performance measures of health-related quality of life to better understand the practical use of patient-reported outcome-based performance measures among elderly primary care patients with multiple chronic conditions.

Abstract

As measures of health care quality have become more sophisticated, the goals of patient care have expanded into helping patients optimize their functional status and well-being. Patient-reported outcome (PRO) based performance measures (PMs) can measure how well these aspects of care are being delivered and compare the performance of health care systems and different provider groups. Most PMs focus on technical quality of care or such outcomes as survival. For older adults, especially those over age 80 with multiple chronic conditions (MCC), it might be equally important or even more important to have a good quality of life. Therefore, policymakers and researchers have been particularly interested in designing PMs that reflect these patients' goals. To date, no PRO-based PMs have been formally developed or validated specifically for use in older adults with MCC.

RAND analysts tested PMs that were based on two prominent instruments for assessing health-related quality of life: the Veterans RAND 36 Item Health Survey (VR-36) and the Patient-Reported Outcomes Measurement Information System 29-item (PROMIS-29) profile instrument. The PROMIS-29 is in widespread use but has undergone limited validation in a geriatric population with MCC. The study had two main aims: first, to validate the PROMIS-29 in this population, and second, to develop a better understanding of the practical use of PRO-based PMs in a geriatric population. To this end, the analysts assessed PM performance based on serial administration of the VR-36 or PROMIS-29, specifically in the MCC population studied.

As efforts to measure health care quality have become more sophisticated, the goals of patient care have extended beyond ensuring survival and curing disease to helping patients optimize their functional status and well-being. Assessments that use patient-reported outcome (PRO) performance measures (PMs) can measure how well these aspects of care are being delivered and compare the performance of health care systems and different provider groups. Most PMs focus on technical quality of care or such outcomes as survival. For older adults, especially those over the age of 80 with multiple chronic conditions (MCC), it might be equally important or even more important to have a good quality of life. Therefore, policymakers and researchers have been particularly interested in designing PMs that reflect the goals of these patients. To date, no PRO-based PMs have been formally developed or validated specifically for use in older adults with MCC.

We tested PMs that were based on two prominent instruments for assessing health-related quality of life (HRQoL): the Veterans RAND 36 Item Health Survey (VR-36) (which, despite its name, is also widely used in nonveteran populations) and the Patient-Reported Outcomes Measurement Information System 29-item (PROMIS-29) profile instrument. The PROMIS-29 is in widespread use but has undergone limited validation in a geriatric population with MCC. There were two main aims of the study: first, to validate the PROMIS-29 in this population, and second, to develop a better understanding of the practical use of PRO-based PMs in a geriatric population. To this end, we assessed the performance of PMs based on serial administration of the VR-36 or PROMIS-29, specifically in the MCC population that we studied.

We surveyed members of Kaiser Permanente Colorado (KPCO), a not-for-profit integrated delivery system. All participants were age 65 or older, and we oversampled those age 80 and older. Each participant had at least two of 13 specific chronic conditions: arthritis, cancer, chronic lung disease, congestive heart failure, depression, diabetes, hypertension, inflammatory bowel disease, ischemic heart disease, osteoporosis, other heart problems, sciatica, or stroke. We identified participants meeting eligibility criteria using KPCO clinical data, and we randomized participants to respond to the survey via the web, telephone, or mail mode. We randomly selected a subset of those who had not responded to the web mode after three weeks to complete the baseline survey by a different mode (mail or telephone). Six months later, we randomly selected 400 baseline web respondents for a second web survey.

PROMIS-29 scores were distributed in expected ways based on such characteristics as age, sex, and number of chronic conditions. After an exhaustive search, we found no evidence of differential item functioning in the sample. Case mix–adjusted PROMIS-29 scores demonstrated convergent construct validity with utilization of primary care (a marker for illness burden) and with scores from the VR-36.

We constructed eight PRO-based PMs, based on four HRQoL scores: the two summary scores of the VR-36 (physical component score and mental component score) and two summary scales of the PROMIS-29 (Physical Health and Mental Health) (see HealthMeasures, undated). Each of these four scores was developed into two different PMs: one for the proportion of patients alive after six months with a stable or improved score, and the other for the mean change in the score at each site of care being profiled. These eight PMs generally performed poorly in terms of distinguishing among the 13 KPCO ambulatory clinics that we profiled. These clinics averaged 26 respondents per clinic. It is generally agreed that PMs must have a reliability of at least 0.7 to be acceptable for use; none of the PMs that we examined achieved a reliability above 0.13. According to the data that we collected, most PMs did not generate measurable reliability under these conditions. A few PMs would have required several thousand observations per entity, which is theoretically possible but still infeasible.

We conclude that scores on the PROMIS-29 are valid for use in an older population with MCC. Using data collected at baseline and a six-month follow-up, PMs based on the VR-36 or PROMIS-29 would have required infeasible or impossible sample sizes to achieve acceptable reliability. Efforts to develop PRO-based PMs for general use should continue because this is such an important goal. However, our findings do not support the general use of these PMs without further refinement and testing. Our results suggest that a major limitation of the approach taken here was the small amount of change in HRQoL that can be expected over a six-month interval. Future efforts to develop PRO-based PMs should focus on changes in HRQoL over longer follow-up intervals.

Three main policy-relevant implications emerge from this work:

1. It is feasible to collect HRQoL data among older adults with MCC in an integrated health system.

2. PROMIS-29 is valid for use in older adults with MCC.

3. Future efforts to develop PRO-based PMs require longer follow-up intervals and larger samples than we used.