Quality of Care and Quality of Dying in Nursing Homes: Two Measurement Models

Abstract

Background

There is consistent evidence of significant variation in the quality of end-of-life care among nursing homes, with many facilities ill-prepared to provide optimal physical and psychological care that is culturally sensitive and respectful of the needs and preferences of residents and their family members. There is continued evidence that what is impeding efforts to improve care is that most measurement tools are hampered by a lack of distinction between quality of care and quality of dying as well as a lack of complete psychometric evaluation. Further, health services researchers cite the need to include “system-level” factors, variables that reflect leadership, culture, or informal practices, all of which influence end-of-life care and can be used to differentiate one setting from another. The purpose of this article is to report advancement in conceptualizing quality end-of-life care in nursing homes and to offer a refined approach to measurement.

Methods

Two latent constructs are tested: quality of care (composed of system-level factors) and quality of dying (comprised of resident/family outcomes). Data obtained from 85 Midwestern nursing homes and 1282 interviews with bereaved family members were used to evaluate both constructs.

Results

Confirmatory factor analyses were conducted and evidence of validity and reliability were obtained for both.

Conclusion

For health services researchers, expanded models that include system-level factors as well as more comprehensive and psychometrically sound models of resident outcomes stand to inform efforts to improve care in this very important area.

Introduction

Over the past 25 years there has been a steady shift in the location of death in the United States from hospital to community. At present, ∼25% of all deaths occur in nursing homes. Of those >85 years of age, a fast growing segment of the population, 35% presently die in nursing homes. It is estimated that by 2020, 40% of those >65 years of age will receive their end-of-life care in a nursing home.¹

These demographic trends suggest a growing challenge for nursing homes to provide the specialized palliative care necessary to ensure high quality care in life's final days. Palliative care, although often used synonymously with end-of-life care, actually should be available to and reflected in care provided to all permanently placed residents.^2,3 Unfortunately, there is consistent evidence of significant variation in the quality of end-of-life care among nursing homes,^2,4 with many facilities ill prepared to provide optimal physical and psychological care that is culturally sensitive and respectful of the needs and preferences of residents and their family members. A facility's expertise with palliative and end-of-life care requires a fundamental shift away from default care typically prompted and reinforced by the regulatory and reimbursement environment that encourages, for example, feeding tubes for weight loss and unnecessary hospitalizations for conditions that could be managed by the nursing home staff.^2,3,5–8

Because of the inconsistency of practices and the wide variation in the quality of care (QOC), nursing homes are an important setting for research to guide improvements. Robust, inclusive, and validated conceptual models and measures are necessary to make progress. The purpose of this article is to report advancement in conceptualizing quality end-of-life care in nursing homes and to offer a refined approach to measurement.

A conceptual structure–process–outcome model is often employed to identify variables important to QOC.⁹ For example, Stewart et al.¹⁰ adapted Donabedian's oft- acknowledged work in evaluating structure, process, and outcomes of care at the end of life. Stewart defined three sets of variables that affect outcomes of care: patient factors, such as clinical status; structure of care factors, such as the physical environment; and process of care factors, such as communication and decision making. Outcomes were broadly defined as satisfaction with care and quality of life. Although reasonable and often cited, this model falls short of drawing upon the influence of multidimensional factors that can be viewed as “system-level factors,” which are known to influence the process of care, such as the nursing home's own administrative leadership, culture, and informal or semi-formal practices, all of which play a role in shaping care delivery. For example, a factor reflecting a nursing home's system or culture of care would be the extent to which permanently placed residents, who experience normal, functional decline, are transferred to the hospital or placed on feeding tubes.^5,8 As another example, nursing homes that use a simple, standardized advance directive form, discussed with the resident and family upon admission, have been shown to reduce unnecessary hospitalizations and increase comfort care in the final days of residents' lives.^11,12 Unruh and Wan¹³ urged capturing such system factors in order to expand models for better evaluating care quality in nursing homes.

With regard to measuring variables at play in end-of-life care, a number of tools have been reported. In a review of tools and their psychometric properties, Mularski et al.¹⁴ identified the most commonly measured variables as quality of life, satisfaction with care, and symptom experience. They noted gaps in measurement in such key areas as advance care planning, continuity of care, and caregiver well-being. Further, most measures lacked thorough evaluation and sound psychometric properties.^14,15

This point was made again by van Soest-Poortvliet et al.,¹⁶ in their evaluation of 11 tools that measure aspects of QOC and quality of dying (QOD) in nursing homes. They concluded that measures are hampered by lack of distinction between care processes (QOC) and care outcomes (QOD). Of 11 tools, only 4 measured one distinct construct; the remaining 7 drew from aspects of both the QOC and the QOD, thereby confounding independent and dependent variables. Therefore, in order for research in nursing homes to yield evidence for quality improvement, conceptual refinement of system-level factors and better measures that draw from distinct constructs are needed.

The purpose of this study is to test the measurement models of the QOC (composed of system-level factors) and QOD (composed of resident/family outcomes). We postulated that nursing homes that have a stronger palliative care focus would be characterized by the following system-level factors: a greater proportion of residents with advance directives and/or who received hospice care, and fewer residents transferred to hospital or placed on feeding tubes. Advance directives can be a marker of discussions regarding end-of-life preferences, and homes in which a higher proportion of residents have advance directives typically have engaged residents and family members in discussions that are reflective of a palliative approach and necessary to promote better end-of-life outcomes.¹⁷ With respect to hospice, homes that utilize the hospice model of care reflect administrative leadership and policies more facilitative of higher quality care.^18,19 On the reverse side, homes that have higher hospitalization rates or who initiate feeding tubes to sustain nutrition are indicative of systems less inclined toward a palliative culture that recognizes the needs and preferences of residents. Therefore, a nursing home's system-level factors (QOC) were indicated by four resident variables: advance directives, hospice use, hospitalization for a medical event, and the use of feeding tubes.

The quality of one's dying is subjective, and is composed of physical, psychosocial, and spiritual aspects. With respect to the measurement of QOD, our approach to capturing this construct consisted of selecting existing tools with conceptual clarity and reported psychometric validity and reliability. Further, tools were selected to represent the multidimensional nature of dying; therefore, QOD was composed of resident symptom distress, family caregiver strain, resident preferences honored, resident- and family-centered care, and family satisfaction with care.

Methods

The study reported here was part of a larger study that examined the relationship of organizational attributes such as staff communication and teamwork, and system-level factors such as use of advance directives and feeding tubes, to resident and family outcomes. A prospective, correlational design guided the collection of data from family members of dying residents of 85 nursing homes in two Midwestern states. Facility level data for the QOC variables were obtained from the minimum dataset (MDS) administrative data base; QOD variables were resident and family-level data that were obtained from family member telephone surveys.

Setting and sample

A random sample of 102 nursing homes agreed to participate; 85 (83%) completed the study protocol. Nursing homes that completed the study had significantly fewer beds and deficiencies, as well as lower rates of administrator and site coordinator turnover. Only nursing homes with ≥60 beds were recruited; although some facilities reduced bed size during the course of the study. The decision to recruit facilities of this size rested on a need to have an adequate number of deaths per facility. The average bed size for the 85 nursing homes was 90 (range=39 to 254; SD=33); 62.4% were rural; and 52.9% were for-profit.

Data were collected from family members after the death of residents who met the following eligibility criteria: 1) having been permanently placed, i.e., excluding residents who were actively receiving rehabilitation under Medicare Part A; 2) having resided in the home for a minimum of 14 days; and 3) having a family member who was involved in care. Residents who were transferred to the hospital and died within 1 week remained eligible. The family sample included one family respondent per deceased resident (∼12 family respondents per nursing home). Family respondent eligibility criteria included: being ≥19 years of age and having been somewhat involved or very involved in the decedent's care during the last month of life using a 5-point scale (“0” not at all involved to “4” very involved), and being the main person who made decisions. Although concerns have been raised about the validity of proxy respondents;²⁰ proxies can reliably describe observable symptoms and information regarding treatments and services.²¹

A total of 1282 family members completed the study. The mean age for the family member was 63.6 years (SD=10.9) with 13% spouses, 3% friends, and 84% other relatives such as son, daughter, or sibling. Of the residents, 64% were ≥85 years old at the time of their death. The majority of the deceased residents (68%) and their family members (71%) were female and Protestant (75% and 74%, respectively). Fifty-five percent of the family members were employed at the time of the survey.

Procedures

Following institutional review board (IRB) approval, facility administrators were sent a letter describing the study purpose and inviting participation with a follow-up phone call to answer questions and seek consent. Once verbal intent to participate was obtained from the administrator, a facility site coordinator was identified to assist with the identification of deceased residents and to notify family members of the facility participation and the purpose of the study, and to invite the family member's participation. Resident and family data were collected through measures included in the telephone survey with family members. Data collection occurred ∼6 weeks following the resident's death and phone interviews lasted ∼1 hour. Research assistants received extensive training in therapeutic communication with bereaved family members, skill development in active listening, techniques for assessment of clinical depression, as well as supervised practice with the telephone protocol to achieve inter-rater consistency across data collectors. Inter-rater reliability was evaluated on 10% of all interviews; percent agreement ranged from 97% to 100%. Data from the MDS version 2.5 administrative database were obtained from the Center for Medicare and Medicaid Services (CMS).

Measures

QOC

Four system-level factors of QOC were collected from the MDS regarding all residents in each participating facility, the 1) proportion of residents with advance directives (defined as living wills, do not resuscitate orders, do not hospitalize orders, and feeding restrictions); 2) proportion of residents in hospice; 3) proportion of residents hospitalized or sent to an emergency room; and 4) proportion of residents on feeding tubes. The length of facility enrollment varied because of differing rates of resident deaths. Therefore, each of these variables was summed across all submitted MDS reports (e.g., quarterly, significant change) during the quarterly reporting periods the facility was enrolled in the study. Summed variables were divided by the total number of residents reported across each quarter to create a proportion.

QOD

QOD included five resident/family outcomes as shown in Table 1. Symptom distress was measured by a version of the Memorial Symptom Assessment Scale Global Distress Index, adapted by Hickman et al.²² for retrospective administration to family respondents regarding symptoms experienced by decedents during the last week of life. These items assess four psychological symptoms (sadness, worry, irritability, nervousness) and six physical symptoms (lack of appetite, lack of energy, feeling drowsy, constipation, dry mouth, and pain). An additional item, shortness of breath, was added by Hickman et al.²², making a total of seven physical symptoms. Resident preferences, resident-and-family-centered care, and family satisfaction with care are each subscales contained in the After-Death Bereaved Family Member Interview.^23,24 Caregiver strain was measured by the Caregiver Strain Index.²⁵ We used the version reported by Tilden et al.²⁶ with a 5-point response option for level of distress (“not at all” to “a great deal”) to enhance variability. Family members were asked to respond to strain they had experienced in the last month of the resident's life. All measures have reported reliability and validity with references noted in Table 1.

Table 1.

Quality of Dying: Five Measures Defined and Operationalized

Measure	Definition	Items	Reliability validity
Symptom Distress Memorial Symptom Assessment Scale Global Distress Index	Global physical and psychological distress	11 physical and psychological items 5-point response option (not-at-all to a great-deal) Higher summed scores reflect greater distress	Portenoy et al.33 Hickman et al.22
Caregiver strain Caregiver Strain Index	Subjective and objective elements of caregiver strain	13 items 5-point response option (not-at-all to a great-deal) Higher summed scores reflect greater strain	Robinson25 Tilden et al. 26
aPreferences honored	Adherence to advance care planning preferences	Three dichotomous items Higher summed scores reflect fewer problems or more advance care planning	Teno et al.23 Teno et al.24 Casarett et al.34
aResident/family- centered care	Emotional support for resident and family, shared decision making, coordination of care, focus on individual	26 items: 18 dichotomous items 1 item 3-point response option (less-than-was-needed to more-than-was-needed) 7 items 4-point response option (never to always) Higher summed scores reflect fewer problems or more resident/family- centered care	Teno et al.23 Teno et al.24 Casarett et al.34
aFamily satisfaction with care	Family perceptions about end-of-life care	Six 10-point items (worst care to best care possible) Higher summed scores reflect better satisfaction with care	Teno et al.23 Teno et al.24 Casarett et al.34

^aEach measure is a subscale from the After-Death Bereaved Family Member Interview.

Data analysis

Confirmatory factor analysis was used to derive two latent constructs: QOC and QOD. Latent variables are not observed directly, but are used to represent hypothetical constructs. Latent constructs are also used to combine measured variables from different sources.²⁷

Confirmatory factor analysis²⁸ with Mplus 5.21 was used to test two measurement models, latent variable representations of QOC and QOD. The standardized estimates (β), representing the relationship between an observed variable and latent variable, were tested using Z-statistics at p<0.05 level. The validity of the measurement model was assessed using root mean square error of approximation (RMSEA) and comparative fit index (CFI).²⁸ An RMSEA <0.08 and a CFI >0.90 were considered an adequate fit. We calculated reliability using a unified approach from the confirmatory factor analysis. Shrout's²⁹ guidelines were used for the purposes of reporting reliability: 0.00–0.10, virtually none; 0.11–0.40, slight; 0.41–0.60, fair; 0.61–0.80, moderate; and 0.81–1.0, substantial. The adequacy of 85 nursing homes for confirmatory factor analysis was supported with a Monte Carlo simulation.²⁸

Results

Descriptive statistics for QOD and QOC variables are presented in Table 2. Each is aggregated to the facility level.

Table 2.

Descriptive Statistics for Quality of Care and Quality of Dying Variables Measured at the Facility Level

Variable	Minimum	Maximum	Mean	SD
Resident symptom distress	3.43	11.10	6.77	1.73
Family caregiver strain	6.22	18.82	12.55	2.77
Residents preferences honored	1.47	2.33	1.81	0.20
Resident/Family-centered care	14.07	19.64	17.33	1.15
Family satisfaction with care	42.91	58.51	52.60	3.36
Proportion of residents with advance directives	0.34	1.00	0.78	0.15
Proportion of residents in hospice	0.00	0.23	0.08	0.05
Proportion of residents hospitalized	0.00	0.75	0.44	0.18
Proportion of residents with feeding tubes	0.00	0.15	0.03	0.02

Our hypothesized model for QOC had good model fit as indicated by its fit indices of CFI=0.976 and RMSEA=0.085 (see Table 3). The standardized estimates representing the correlation of variable to construct are β=0.241–0.892, all of which are significant (p<0.05) (See Figure 1). As shown in Table 3, the “entire reliability” is 0.83 and all test re-test reliability measures are in the “substantial” category (>0.8, Shrout²⁹) except for hospice, which has “fair” reliability.

Table 3.

Summary of Models and with Respective Fit Indices

Model (M)	CFI (>0.90)	RMSEA (<0.08)	Entire reliability/test re-test reliability
Quality of care	0.976	0.085	0.83
Advance directives			0.89
Hospitalized			0.80
Hospice			0.56
Feeding tubes			0.89
Quality of dying	1.000	0.000	0.98

CFI, comparative fit index ; RMSEA, root mean square error of approximation.

FIG. 1.

Quality of care measurement model.

Our hypothesized model for QOD had excellent model fit as indicated by its fit indices of CFI=1.000 and RMSEA=0.000 (see Table 3). The standardized estimates representing the correlation of variable to construct were β=0.324–0.991, all of which were significant (p<0.05). The largest estimate was the variable “resident/family centered care” (See Figure 2). Also shown in Table 3, the “entire reliability” is 0.98 (>0.80).

FIG. 2.

Quality of dying measurement model.

Using confirmatory factor analysis as well as entire reliability, we demonstrated that variables that are used to measure the latent constructs QOC and QOD have substantial validity and reliability. All measured facility-level factors, except hospice, have substantial reliability.

Discussion

Policy makers, clinicians, researchers, and nursing home residents stand to benefit from sound evidence-based guidelines to enhance the delivery of palliative and end-of-life care in nursing homes. Guidelines must be developed from rigorous research; however, as recently as 2011,^15,16 the majority of measurement tools lacked thorough psychometric evaluation. This study advances the conceptualization and measurement of end-of-life care by validating two distinct latent constructs to be used in nursing home research: QOC, which captures system-level factors reflective of a palliative approach to care, and QOD, which captures resident/family outcomes of care.

Improving care delivery within a complex and dynamic health care system requires having measurement models that draw from the influence of the care setting as an essential component of evaluation. As postulated previously, facility system-level factors such as the proportion of residents who receive advance care planning or hospice services are reflective of a palliative approach to the care for all permanently placed residents. A validated construct that reflects a palliative approach to care processes, expands systems-focused research in end-of-life care. A systems-level representation of palliative care would assist in the differentiation of care practices from one home to another, and could be used as information for evaluation by regulators, researchers, and family members. One caveat is worth noting here. For the sake of clarity, we viewed the four system-level factors as a latent representation of a nursing home's level of engagement in palliative care. We acknowledge that there are many aspects to the provision of palliative care, and that future research efforts should not be limited by these four system-level factors, but should test and validate others as well.

Previously, researchers have demonstrated that nursing homes that regularly refer patients to hospice provide a greater palliative care approach for all residents, resulting in lower hospitalization and feeding tube rates and better pain management.^30,31 Therefore, access to hospice services is an important element of both palliative and end-of-life care; however, in this study, hospice had the weakest relationship to the construct QOC. In 2006, approximately one-third of all nursing home residents received hospice services;³² however, 30% of these residents received hospice services for ≤7 days. Although the use of hospice by nursing homes is increasing,³² the weak relationship of hospice to the construct QOC reflects measurement error inherent in the use of the MDS to capture this variable. A more accurate measure of hospice is needed for evaluation and requires a change in how and when this referral is documented in the MDS data. An alternative, albeit a more complicated, data management design is to measure hospice services using Medicare and Medicaid data.

The construct QOD provides a multidimensional evaluation of dying that captures physical and psychological symptoms, preferences honored, caregiver strain, and other important aspects unique to end-of-life experiences. Death is a unique experience; individual interpretations of dying are influenced by culture and life experiences and are not universal. The latent construct QOD fills the need for a multidimensional measure; one that more effectively captures a spectrum of experiences. Further, the construct is comprised of measures noted by Mularski et al¹⁴ as lacking in many studies such as preferences honored and caregiver strain. The validation of this construct promotes a more comprehensive evaluation of resident outcomes at the end of life.

In summary, two latent constructs have been validated for use in nursing home research. For health services researchers, expanded models that include system-level factors as well as more comprehensive and psychometrically sound models of resident outcomes stand to inform efforts to improve care in this very important area.

Acknowledgment

This study was funded by National Institute of Nursing Research: NR009547.

Author Disclosure Statement

No competing financial interests exist.