Measuring Patient-Centeredness of Care for Seriously Ill Individuals: Challenges and Opportunities for Accountability Initiatives

Abstract

Patient- and family-centeredness of care is particularly important for individuals with serious illness. In this article, we describe methodological challenges of using measures of patient- and family-centeredness in accountability initiatives such as public reporting and pay for performance. We begin with background on measuring patient- and family-centered care using standardized surveys, describe evidence of the use of these measures for quality improvement, and highlight methodological challenges in the development and implementation of these measures for use in accountability. To ensure that patient- and family-centeredness is the cornerstone of public and private accountability initiatives designed to promote high-quality care to seriously ill patients, we recommend development of (1) a nationally endorsed survey instrument that assesses patient and family experiences of serious illness care across the full range of patient trajectories and care settings in which this care is provided; (2) administrative data infrastructure that allows for identification and outreach to the most knowledgeable respondents for the survey, regardless of the patient's setting of care; and (3) a broad toolkit of quality improvement approaches to ensure that as the emphasis on accountability grows, providers across settings have access to tools that can help them improve patient- and family-centeredness of care for the seriously ill.

Introduction

Increasingly, national, state, and provider-level accountability initiatives, including public reporting and pay-for-performance, are being implemented to promote transparency and quality improvement across a range of health care settings. Evidence suggests that when these programs use a simple design with sufficiently large incentives, apply quality measures with a strong tie to evidence, and target a focused set of areas in greatest need for improvement, they can promote improvements in care.^1–4

Patient-centeredness is a core element of high-quality health care,⁵ and accordingly, measures of patient-centeredness are increasingly included in national public reporting and pay-for-performance programs as complements to clinical process and outcomes measures. For seriously ill individuals, care is provided to both the patient and his or her family as a unit, and therefore, the goal is patient- and family-centered care, “…an approach to the planning, delivery, and evaluation of health care that is grounded in mutually beneficial partnerships among health care providers, patients, and families…” in which “patients and families…determine how they will participate in care and decision-making.”⁶

Patient- and family-centered care—which can be directly measured through surveys of patients and their family caregivers—is associated with higher levels of adherence to recommended prevention and treatment processes, better clinical outcomes, better patient safety within hospitals, and less health care utilization.^2,7

Patient- and family-centeredness is particularly important for seriously ill patients, given great variability across patients with regard to both preferences for care intensity and tradeoffs between quality and length of life. In this article, we describe methodological challenges of using measures of patient- and family-centeredness in accountability initiatives such as public reporting and pay-for-performance. We begin with some background on measuring patient-centered care using standardized surveys, describe evidence of the use of these measures for quality improvement, and highlight methodological challenges in the development and implementation of these measures for use in accountability.

Measuring Patient-Centered Care with Standardized Surveys

The Agency for Healthcare Research and Quality (AHRQ)'s Consumer Assessment of Healthcare Providers and Systems (CAHPS) family of surveys includes standardized tools for assessing the experience of consumers receiving different types of health care,^8–12 including ambulatory care delivered by health plans,^10,13,14 physicians and physician groups,^15,16 accountable care organizations,¹⁷ and facility-based care in hospitals¹⁸ and nursing homes,^19,20 among several other settings and patient populations.

Most recently, the Centers for Medicare and Medicaid Services commissioned the development of the CAHPS Hospice Survey, which covers the domains of communication, timeliness of care, emotional and religious support, symptom management, respect, provision of hospice care training to family members, as well as overall ratings of care and willingness to recommend the hospice.²¹

Numerous other surveys have been developed and implemented to assess patient and family experiences with care at the end of life in particular (Table 1).²² Experiences in developing and implementing these surveys are instructive for future measure development that aims to assess care experiences of seriously ill adults, children, and infants across a range of disease trajectories and care settings.

Table 1.

Frequently-Cited, Validated Survey Instruments for Assessing Experiences with Care for the Seriously Ill

The Family Evaluation of Hospice Care survey, an after-death survey designed to report hospice quality from the perspective of the patient's family or other informal caregiver. The most recent version of the survey, in use by hospices nationwide until 2015, included 10 sections with 55 questions (including 54 closed-ended questions and one open-ended) addressing care coordination, attention to family needs, symptom management, and communication. Survey results have been used by the National Hospice and Palliative Care Organization (NHPCO) to feed back information regarding hospice care quality to their provider members.39,45–51

The After-Death Bereaved Family Interview, a 74-item tool, includes questions related to information and care planning, symptom management, spiritual and religious support, and psychosocial and personal care. It can be used to assess care across settings.52–59

The View of Informal Carers—Evaluation of Services (VOICES) questionnaire asks caregivers about the problems they or the patient faced during the last three months of life, and about the care they received in different settings (i.e., home, nursing home, hospital, and hospice). In four of the five sections, respondents answer questions about the type and frequency of pain the care recipient experienced as well as therapies, communication with staff, quality and the type of care that the care recipient received, and overall satisfaction in these care settings. The last section consists of items not specific to the care setting related to patient/caregiver–provider communication, socioeconomic issues, and stress.60–63

The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments used an eight-item questionnaire to measure satisfaction with (1) patient comfort and (2) patient–provider communication and decision making. Family members answer questions on if more could have been done to make the patient comfortable during the last weeks of life, how well medical staff was able to explain the patient's condition, if staff provided moral support to family members, and if staff carried out their decisions about patient care.54

The Family Assessment of Treatment at the End of Life is an after-death survey that assesses experiences of end-of-life care for Veterans. The 32-item questionnaire includes questions regarding communication, care around time of death, emotional and spiritual support, symptom management, and respect for treatment preferences.64,65 The FATE was shortened to the FATE-S, later referred to as the Bereaved Family Survey (BFS).66,67 The BFS contains only 14 questions to limit the time of interview, but still captures the majority of pertinent information collected for the FATE. The BFS is now completed for all inpatient Veteran Administration deaths in an effort to measure and improve quality of end-of-life care in this population.

The Quality of Dying and Death survey was developed at University of Washington, Seattle after literature review and qualitative study identifying the following six domains of care: symptoms and personal care, preparation for death, moment of death, family, treatment preferences, and whole person concerns. The entire survey is 31 questions and is intended for after-death interviews with family members.68–73

The Family Satisfaction with Advanced Cancer Care is a 16-item instrument designed to measure experiences with cancer care.74–82

Value of Patient-Centered Measures in Quality Improvement

Across a range of care settings, health care providers have shown that they can use survey measure results to identify and target areas for improvement of patient and family experiences of care. For example, in California, patient experiences with physicians significantly improved following the introduction of statewide measurement, reporting, and performance-based financial incentives tied to scores on the CAHPS Clinician & Group Survey.²³ Hospitals succeeded in improving Hospital CAHPS scores in response to required survey administration and public reporting,^24,25 although additional improvements may not be attributable to introduction of value-based purchasing.²⁶ Although there has been limited empirical study of the effects of quality improvement initiatives targeting patient-centeredness, case studies are instructive.²⁷ Table 2 presents highlights of case studies describing successful initiatives targeted at improving patient- and family-centeredness.

Table 2.

Case Study Examples of Successful Initiatives to Improve Patient and Family Care Experiences

Case Study 1. An initiative to improve patient access to specialty care found that clarifying staff roles and providing customer service training resulted in a two to six percentage point increase in patients' reported willingness to recommend their doctor, with smaller improvements in the overall Consumer Assessment of Healthcare Providers and Systems composite score for patient access.83

Case Study 2. An acute care academic hospital that aimed to improve the emotional support provided to patients launched an initiative that strengthened guidelines on patient–nurse interactions and added reminders in the electronic health record for staff to interact with patients, among other efforts. The initiative resulted in a statistically significant increase of four points in the proportion of patients reporting that nurses discussed patients' anxieties and fears with them.84

Case Study 3. A multispecialty group implemented an electronic health record system that allowed providers to give patients an after-visit summary that described the visit, test results, and medication, and provided patients with a list of actionable next steps to care for themselves. Among patients of internal medicine providers, the proportion of patients reporting that they received clear instructions rose from 84% to 95%, while the proportion rating their provider a 9 or 10 out of 10 rose from 70% to 82%.85

Methodological Challenges in Development and Implementation of Measures of Patient-Centeredness for Use in Accountability

For use in accountability, quality measures must have several important features. They must (1) be “measures that matter” to the consumers who will use them to select among providers and to the providers being evaluated; (2) be collected and calculated in such a way as to be representative and valid assessments of care within a given provider; (3) have sufficient ability to distinguish between the performance of providers (i.e., adequate interprovider reliability); (4) leave room for improvement (i.e., not be “topped out”); and (5) allow for fair comparisons across providers. In this section, we highlight challenges to each of these criteria for development and implementation of measures of patient- and family-centeredness for the seriously ill.

Measures that matter

Seriously ill individuals, families, and providers must be integrally involved in development of survey measures to ensure their relevance. Seriously ill individuals vary greatly with regard to primary diagnosis, disease trajectory, and acuity of illness; these factors also affect patients' likelihood of receiving care at home, in an acute care hospital, or a skilled nursing facility. Measures developed for use across these diverse patient subgroups and settings must therefore address goals of care that are common to all patient populations (regardless of the setting in which they receive care), such as symptom management, emotional support, and provider communication.

Representative and valid

Declining response rates to patient surveys have led to concern regarding representativeness of those who do respond, and in particular, whether nonrespondents differ from respondents in ways that are associated with how they respond to a given survey, resulting in nonresponse bias. Achieving the highest possible response rates is critical to reduce the perception and possibility of nonresponse bias, and to maximize the number of providers for whom scores can be stably estimated. Evidence to date suggests that mixed-mode administration (mail with telephone follow-up) produces the highest response rates on care experience surveys,²⁸ although mixed-mode data collection is expensive relative to other modes.

Alternative, potentially less-costly modes, such as Internet-based platforms and point-of-service surveys, should be investigated carefully. Internet surveys may actually result in worse response rates than traditional modes,²⁹ and it may not be possible to survey a representative sample, which is needed for accountability. Internet surveys yield their highest response rates when survey links are delivered by email; however, prior studies have shown great variability in comprehensive availability of valid email addresses across health care providers.³⁰

Point of service survey administration (i.e., surveying patients or family members at the location of care) faces a similarly important barrier in the context of accountability: ensuring that all (or a random sample of) eligible patients are invited to participate. This has proved challenging to achieve even in experimental settings,^30,31 introducing potentially systematic bias to the survey sample. In particular, when staff members from the health care provider are responsible for recruiting respondents, they may (intentionally or unintentionally) bias who is sampled and how they respond. An experiment comparing point-of-service administration to other modes found that patients invited to participate in a survey by clinic staff at the point of service gave more favorable responses than patients responding via other modes.³⁰

Lower response rates to high-quality surveys based on probability samples are not consistently related to nonresponse bias³²; to address the possibility of biased comparisons, however, results can be adjusted using available data. For example, case-mix adjustment models (described in more detail below) that include factors such as age and health status that are related to nonresponse help to compensate for bias associated with these factors when comparing health care providers; some patient survey analyses further adjust for possible nonresponse bias resulting from differential response rates across hospitals.³³

An additional challenge to representativeness of survey results for the seriously ill is that direct reports from such patients may not be feasible because of illness acuity and speed of clinical decline. Patients who are able and willing to participate in surveys therefore are not broadly representative of all seriously ill patients. For patients with dementia and other diagnoses and disease progressions that impact cognitive function, family caregivers are critical informants for understanding care experiences. Regardless of diagnosis, the role of family caregivers tends to increase over the trajectory of the illness. To ensure that surveys capture the care experiences of seriously ill patients regardless of their ability to participate, family caregivers must be allowed to act as proxy respondents for surveys administered during the course of patient care. In addition, family caregivers are the only available respondents for after-death surveys that assess care in the final days of life.

As members of the unit of care, family caregivers can provide direct reports of their own experiences with the care team and provide proxy reports of their family members' experiences. When acting as proxies, family caregivers' responses have moderate-to-high agreement with patient responses regarding observable symptoms, such as fatigue or shortness of breath, and quality of care, such as being kept informed by health care providers;^34,35 however, family caregivers' responses to questions regarding less observable symptoms, such as pain, anxiety, and depression, exhibit lower agreement with patient responses. This suggests that survey questions should be designed with the respondents' role as an “expert observer” in mind—that is, question content should be framed to capture reports of observed experiences rather than inferences regarding patient experiences.

Although proxy respondents may sometimes answer differently than patients would have, there is no reason to believe that these differences reduce the validity of comparisons of care experiences by proxies across provider organizations. Survey results can be statistically adjusted to address the effects of proxy responses; such adjustments address both the systematic differences in response tendencies resulting from proxy response, as well as the underlying differences in health that result in proxies responding rather than patients.

Distinguishing between providers

Many health care providers are either too small, or care for too few patients at the end of their lives, to generate a sufficient number of completed surveys to promote adequate measure reliability. Reliability is the ability of a measure to distinguish between the performances of health care providers; reliability generally improves when sample sizes increase. When reliability is poor, measure scores are of limited value for comparing and rewarding the performance of providers or informing consumer choice.

The most direct approach to addressing the small numbers problem is to pool measure data in various ways. Pooling survey measure data over longer time periods, for example, improves the overall number of surveys used to compute the measure scores. The downside of this approach is that it includes data from a considerable lag, forgoing some responsiveness and immediacy; however, that tradeoff is usually worthwhile, since change is often slower to take place than might be anticipated.³⁶ Statistical smoothing techniques allow for data to be pooled over time, but still to be used to predict current performance without masking improvements or declines occurring over the whole period.^37,38

Alternatively, survey measure data can be pooled across small facilities or facilities with common characteristics. The appropriateness of this pooling can be assessed empirically. For example, if a parent provider organization has many sites providing care to the seriously ill, is the organization-level driving performance or perhaps are setting-specific factors more dominant such that performance measurement should be aggregated within setting, then within the organization?

Topped-out measures

When a measure has low reliability (i.e., it does not have strong ability to distinguish the performance of one hospice from another), and its mean is near the top of the scale (i.e., near 100 for a top-box score), it has little to no room for quality improvement and may be deemed “topped-out.” Patients report very favorable care experiences at the end of life, perhaps reflecting better access to care; nonetheless, capturing variation in care experiences for seriously ill patients is critical for making comparisons across providers and identifying areas in need of further quality improvement.

Topped-out measures are often retired and replaced in accountability programs, although continued incentives may be important to maintain high performance on these measures. As an alternative to retiring topped-out measures, high-performing measures can be used to identify problems rather than to compare performance for all entities.³⁹ Although it may be difficult to distinguish exceptional hospices or capture improvement in the context of topped-out measures, there may be better power to identify providers that are problematic. From the perspective of payers, policy makers, and patients and families, being able to flag problematic providers actually may be of higher priority than differentiating between excellent and very good providers.

Care experiences vary substantially by setting; Teno et al. reported that family members of those who received care from hospice at home more likely to report favorable experiences than those whose family members received care in nursing homes or hospitals, particularly with regard to emotional support, for example.⁴⁰ Consequently, measures may be topped-out in some settings of care, but not others. Such measures are still valuable in accountability programs. Variability in measure scores across settings can help providers and payers to steer patients toward settings most likely to meet their needs. In addition, measures with lower scores in a given setting represent opportunities for improvement within that setting.

Fair comparisons between providers

As noted above, relevant topical domains, patient populations, and caregiver familiarity with care may vary considerably across settings of care. To ensure that survey scores can be fairly compared across providers, despite differences in the types of patients they care for and the settings in which they provide care, survey scores must be adjusted for factors that influence how patients and family caregivers respond to survey questions, but are (1) not within the providers' control, and (2) not associated with the quality of care provided.⁴⁰

Case-mix adjustment uses statistical models to predict what each provider's score would have been for a standard patient population, thereby removing from comparisons the predictable effects of differences in patient characteristics that vary across providers. A case-mix adjusted measure score for a provider, therefore, can be interpreted as what we would expect that provider's score to be if it had a patient population with characteristics that were the same as the standard patient population (e.g., the national average).

Importantly, adjustments should not be made for factors that might be a consequence of the care provided or are within the control of the provider. For example, as noted above, care experiences vary by care setting; thus, survey scores should not be adjusted for care setting, since doing so would mask true differences in care quality across settings within a provider. Similarly, since family involvement in care may be somewhat in the control of health care providers, scores should not be adjusted for level of caregiver involvement, lest these adjustments mask true differences in care quality between providers.

While some measures assess content that is broadly applicable across settings, other survey content might be applied only for a subset of settings for which they are relevant. Regardless of whether setting-specific measures are included in accountability measure sets, these measures can serve as important guides to quality improvement within given settings of care. Setting-specific measures can be operationalized by either (1) asking questions only of those respondents who had experience with the given setting or (2) asking questions of all respondents, regardless of setting, but calculating measure scores only for those with experience in the relevant setting(s). In addition, linear mixed-effect models allow for assessment of how a provider may perform differently on a given measure across the settings, in which it provides care.

Priorities for Research and Practice

Attention to several priorities is needed to ensure that patient-centeredness is the cornerstone of public and private accountability initiatives designed to promote high-quality care to seriously ill patients.

First, there must be a nationally endorsed survey instrument that assesses patient and family experiences of serious illness care across the full range of patient trajectories and care settings in which this care is provided. Such an instrument could be built off the extensive effort that has gone into development of existing surveys, such as those described in Table 1, with refinements based on new qualitative inquiry with patients, families, and providers in each relevant setting to ensure that the instrument reflects the highest-priority care domains and those most amenable to quality improvement at each stage of care, from enrollment through ongoing care, care at times of crisis, and the actively dying phase.

Development work conducted to inform the development of existing surveys suggests that domains likely to be of greatest importance are communication and shared decision making, physical comfort, dignity and respect, emotional support, and educating family caregivers to care for their family member at home, among others.^22,42,43 In addition, particular attention will need to be paid to development of measures in the domains of consistency of care with patient preferences and shared decision making.

Second, administration of surveys to family caregivers requires consistent and comprehensive methods and data systems to identify the most knowledgeable respondent for the survey, regardless of the patient's setting of care. The CAHPS Hospice Survey relies on hospice administrative records to identify the primary caregiver most knowledgeable about the care received by the decedent; to apply a similar approach to surveys of care in other settings and programs that provide care to the seriously ill, providers would need to maintain current and accurate records for family caregivers, specifically identifying caregivers who are most involved in the care of the patient, as well as up-to-date contact information, including email addresses and mobile telephone numbers, to open the possibility of alternative modes of survey administration.

Finally, the literature on quality improvement initiatives aimed at experiences of care among seriously ill patients and their families is limited.⁴⁴ In tandem with development of new tools to measure care experiences of these groups, development of a broad toolkit of quality improvement approaches to address key domains of patient- and family-centeredness is a key priority to ensure that as the emphasis on accountability grows, providers across settings have access to tools that can help them improve.

Acknowledgment

We gratefully acknowledge support from the Gordon and Betty Moore Foundation for the preparation of this article.

Author Disclosure Statement

No competing financial interests exist.