End-of-Life Quality-of-Care Measures for Nursing Homes: Place of Death and Hospice

Abstract

Objective

The Centers for Medicare and Medicaid Services (CMS) publishes a web-based quality report card for nursing homes. The quality measures (QMs) do not assess quality of end-of-life (EOL) care, which affects a large proportion of residents. This study developed prototype EOL QMs that can be calculated from data sources available for all nursing homes nationally.

Methods

The study included approximately 1.5 million decedents residing in 16,000 nursing homes during 2003–2007, nationally. Minimum Data Set (MDS) data were linked to Medicare enrollment files, hospital claims, and hospice claims. Random effect logistic models were estimated to develop risk-adjustment models predicting two outcome measures (place of death [POD] and hospice enrollment), which were then used to construct two EOL QMs. The distributional properties of the QMs were investigated.

Results

The QMs exhibited moderate stability over time. They were more stable in identifying quality outliers among the larger nursing homes and in identifying poor-quality outliers than high-quality outliers.

Conclusions

This study offers two QMs specialized to EOL care in nursing homes that can be calculated from data that are readily available and could be incorporated in the Nursing Home Compare (NHC) report card. Further work to validate the QMs is required.

Introduction

The Centers for Medicare and Medicaid Services (CMS) publish a web-based quality report card, Nursing Home Compare (NHC), with 19 quality measures (QMs). Despite its extensive coverage of many aspects of the care provided to residents, none of the measures included in it is specifically designed to capture end-of-life (EOL) care. The majority of long-term-care (LTC) residents will die in this setting. During the period preceding death, their care needs may differ from those of other residents. For them, the concept of “quality care” may have a different meaning and more specialized QMs, capturing the domains of care of greatest value at EOL, are needed. Several such domains have been proposed, including appropriate management of pain and other symptoms, aggressiveness of treatment, advance care planning, spirituality, and grief and bereavement support for the patient and family.^1–4

Our objective was to investigate the possibility of using nationally available, administrative nursing home data, the Minimum Data Set (MDS), in conjunction with other Medicare claims data, to construct QMs that would be relevant for EOL care and could be applied to all nursing homes in the country, in a fashion similar to the QMs currently included in NHC. We present two measures. They were chosen based on the following criteria: 1) domains that are important to nursing home residents; 2) QMs with evidence of reliability and validity; 3) QMs that can be measured from administrative data; and 4) QMs based on outcomes that are amenable to improvement by nursing homes.

The first QM is based on place of death (POD) and reflects the notion and empirical evidence that, when appropriate, many nursing home residents prefer to avoid death in the hospital.^5,6 The second is based on hospice enrollment prior to death and is motivated by studies documenting better EOL care for hospice nursing home patients. For example, nursing home hospice patients were shown to be twice as likely to receive daily pain treatment,⁷ are less likely to be physically restrained or to have feeding tubes,⁸ and their family's satisfaction is higher.⁹

The QMs presented here require further refinement and validation before they could be widely applied. Furthermore, other QMs, capturing other aspects of EOL care should also be developed.

Methods

Data and sample

The sample included all LTC residents who died in U.S. Medicare- and Medicaid-certified nursing homes, or within 8 days of a nursing home stay, during 2003–2007. We obtained the MDS, an individual level data set with information collected at regular intervals. It includes data about the person's sociodemographics, physical and mental health status, and treatments. These data are submitted to CMS, which uses them to calculate Medicare payment rates and the QMs for NHC.¹⁰ Many of the data elements used in this analysis were shown to be of high reliability.^11–13

Because the MDS does not include information about hospice enrollment and the information about death and hospitalization may not be reliable, it was linked, using the individual beneficiary identifier, to hospice claims to verify hospice enrollment, to the Medicare enrollment file for date of death, and to hospital claims to verify hospital admission.¹⁴

LTC residents were defined as those who either stayed in nursing homes for more than 90 days or whose stay was not Medicare reimbursable at the time they died. We limited the sample to LTC residents because postacute residents are typically admitted for short stays of several weeks with the expectation that they will be discharged to the community. For them death is not an expected outcome, and although it occurs, it is more likely to be viewed as an adverse outcome and a failure of care. We also excluded LTC decedents who were in a coma (1.2%), because for them the outcomes are not relevant, and those enrolled in an HMO (11.3%), because their Medicare inpatient data are not accurate. We identified 1,467,789 decedents in more than 16,000 nursing homes.

Variables

Outcome variables

We defined two outcome variables. The first was POD, defined as either death in the hospital (=1) or death in the nursing home (=0). Six percent of observations could not be assigned to either the hospital or the nursing home. Because we do not have any information on the POD for these persons (e.g., they could have died on the way to the hospital) we excluded them from both the numerator and the denominator. The second outcome was defined as either hospice enrollment while in the nursing home (=1), or not (=0).

Risk factors

Because the proportion of patients who may be appropriately hospitalized will vary based on their underlying characteristics, and because not all nursing home residents would be equally likely candidates for hospice care, rates of these outcomes should be adjusted for the differences across facilities in resident case mix. Adjusting for characteristics that influence the outcome rate but are outside the control of the facility levels the playing field and makes the comparison across nursing home more meaningful and fair.^15,16

We started by choosing the initial set of risk factors for POD and hospice enrollment by performing an extensive and iterative Medline search, using key words such as nursing home, quality, EOL, hospice, and hospitalization. This was followed by examining and reviewing the information available in the MDS with a team of geriatricians experienced in EOL and nursing home care. Criteria for including risk factors were: 1) an individual resident characteristic likely to influence the outcome; and 2) a characteristic not likely to be influenced by treatment choices made by caregivers in the facility.^15,17 The final models presented include those risk factors that were significant at the 0.2 level or higher. All excluded risk factors were subjected to a joint F test to verify that they were not jointly significant at the 0.2 level or higher.

Time window for risk factors selection

An important consideration in developing EOL quality measures is to determine when “end of life” begins. Ideally, we would have liked to be able to identify the point in time at which the patient and the caregivers determine it as such. This, however, is not feasible.

We, therefore, chose to rely on the last health assessment available for the decedent in the MDS. Given the data collection structure of the MDS, which mandates an assessment every 90 days, and the fact that death is not correlated with the time of data collection (80% of last assessments are regularly scheduled and not due to change of status), the last assessment prior to death provides patients' risk factors randomly distributed during the EOL period. An analysis of the data shows that the median time between the last assessment and death was 18 days and the 25th and 75th percentiles were 7 and 44 days, respectively.

Assessment types, information content, and imputation

The MDS includes several assessment types: admission, annual, change of status, and quarterly. The first three include the maximum amount of information. We refer to them as full-information. The quarterly assessments include only a subset of the information available in the full-information assessments. For those individuals whose last assessment was a quarterly, we imputed the missing data by locating their prior full assessment and obtaining the missing information from that prior full assessment. However, not all risk factors were deemed appropriate for imputation. For example, once a chronic disease such as diabetes was recorded for the individual we assumed it persists even if the last assessment was a quarterly and the information had to be imputed from a previous assessment. On the other hand, pneumonia, which is transient, was not assumed to persist and was not imputed. This difference in information structure by assessment type necessitated estimation of a separate risk-adjustment model depending on the last assessment type available for each individual.

Risk factor definitions

All risk variables are based on the MDS version 2.0. We refer the reader to the MDS manual¹⁸ for exact definitions and discuss only those variables that require additional detail. The length-of-stay variable was calculated as the time the individual resided in the nursing home and was motivated by the hypothesis that longer stays offer more opportunities for staff to discuss with the resident/family the option of hospice, thus potentially impacting hospice choice. The activities of daily living (ADL) variable was based on the self-performance assessment rather than assistance received. Dementia was defined as a diagnosis of Alzheimer's disease or the more general diagnosis of dementia.

Analyses

The data were randomly split into a development and test data sets. The models were estimated first on the development data and then validated on the test data. The final estimates are based on the full data. We estimated separate risk-adjustment models for each outcome and assessment type with the initial set of risk factors chosen by the clinicians. We estimated logistic models at the individual resident level with random facility intercept effects to account for patient clustering at the facility level. Continuous variables were entered into the model as squared and cubed terms to test for nonlinear relationships with the outcome. We eliminated risk factors with p values greater than 0.2 and verified that the p value for the hypothesis of their joint exclusion from the model exceeded 0.2. The goodness of fit of the models was assessed by the C statistic.¹⁹ These models were used to predict for each individual the probability of the outcome, conditional on the type of assessment available for that individual and his or her risk factors.

The POD QM for each facility was defined as the difference between the observed facility outcome rate for the year and the expected, risk-adjusted outcome rate. The latter was calculated as the average of the predicted probabilities for all individuals residing in the facility during the year. QM values exceeding zero indicate rates that exceed the national average, and because the outcome is undesirable, can be interpreted as potentially indicative of poor quality. Because we consider risk-adjusted hospice enrollment to be a desirable outcome, the hospice QM was defined as the difference between the expected and the observed rate, such that the interpretation of the QM remains consistent with the POD QM, with larger values indicating lower quality.

To investigate the properties of these measures we categorized nursing homes into deciles based on both POD and hospice QMs and examined their stability over time, that is, the percent of facilities that remain in the same decile in 2 consecutive years. We performed a similar analysis comparing the concordance of the POD and the hospice QMs in classifying nursing homes. Because the accuracy of these QMs declines as the number of decedents in the facility decreases, we repeated these analyses, following the current practice of CMS in NHC, limiting the analysis to those facilities that had more than the mean number of decedents in a year.

Results

Table 1 shows descriptive statistics. There were about 1.5 million individuals in the POD sample of whom 20.3% died in the hospital. The average number of decedents in a facility was 20. There were about 1.1 million individuals in the hospice sample with 32.7% enrolled in hospice and a mean of 17 decedents in each facility. The hospice sample included fewer observations because change in ADLs that required a prior assessment was not available for all decedents. This variable was not deemed an important risk factor for the POD QM because the outcome was not death, but rather the location of death, for which change in function was not considered an important predictor.

Table 1.

Descriptive Statistics^a

POD sample (n=1,447,926)
	Percent	Mean	Standard deviation
Variable:
Dependent Variable: POD is Hospital	20.3%		—
Demographics:
Female	69.0%		—
Age in Years	86.3		7.83
Diseases:
Diabetes	27.3%		—
Number of Cardiovascular Diseases		1.47	1.10
Number of Musculoskeletal Diseases		0.08	0.28
Number of Neurological Diseases		0.32	0.55
Asthma, COPD, or Both		0.24	0.46
Pressure or Stasis Ulcer, Stage 2 or Higher	22.2%		—
Renal Failure	11.4%		—
Pneumonia	10.9%		—
Septicemia	1.7%		—
Internal Bleeding	1.7%		—
Hip Fracture in Last 180 Days	3.0%		—
Other Fracture in Last 180 Days	3.4%		—
Tuberculosis	0.04%		—
Treatments:
Do Not Resuscitate	73.7%		—
Do Not Hospitalize	8.3%		—
Feeding Tube	9.1%		—
Dialysis	1.7%		—
Radiation	0.4%		—
Tracheostomy Care	0.8%		—
Ventilator/Respirator	0.6%

Hospice sample (n=1,104,511)
Variable:
Dependent Variable: Resident utilized hospice	32.7%		—
Demographics:
Female	64.9%		—
Age in Years		85.5	7.82
Length of Stay in Years		1.13	1.65
African American	7.2%		—
Asian or Native American	1.5%		—
Hispanic	2.5%		—
No Schooling	1.1%		—
8th Grade or Less	20.8%		—
Bachelor Degree or Higher	18.1%		—
Not Married	73.0%		—
Diseases:
Sum of ADLs (Range 0–64)		27.9	8.43
Number of Diagnoses (Range 0–43)		1.46	1.30
Congestive Heart Failure	31.0%		—
Dementia	49.9%		—
Emphysema or COPD	22.2%		—
Cancer	17.3%		—
Hip Fracture in Last 180 Days	4.0%		—
End-Stage Disease	14.8%		—
Swallowing Problem	30.1%		—
Weight Loss	24.6%		—
Weight Gain	6.5%		—
Pressure Ulcer: At Least Stage 3	8.6%		—
HIV	0.01%
Treatments:
Chemotherapy	0.8%		—
Dialysis	2.1%		—
Oxygen Therapy	35.3%		—
Radiation	0.6%		—
Suctioning	2.5%		—
Tracheostomy Care	1.0%		—
Ventilator/Respirator	0.7%		—

^aThe table provides descriptive statistics for all variables included in the final models.

ADL, activities of daily living; COPD, chronic obstructive pulmonary disease; POD, place of death.

Tables 2 and 3 present the risk adjustment models. Most variables were highly significant (at the 0.001 level). This is to be expected given the large samples. The models for the quarterly assessments, which involved imputation, indicate which variables were imputed, which were not included because they were not statistically significant at the 0.2 level, and which were not included because they were judged inappropriate for imputation because they indicate transient health conditions. The risk factors that were significant in the development models were also significant in the validation models and the goodness of fit in the validation models were only slightly worse (lower C statistics) then in the development models. C statistics ranged from 0.61 to 0.67, values that are typical for risk-adjustment models developed from MDS data.¹⁵

Table 2.

Place of Death Risk Adjustment Logistic Models: Dependent Variable: Place of Death Is Hospital

Risk factors	Model estimated on full-information assessmentsa,b	Model estimated on quarterly assessments with imputed informationb
Demographics:
Female	0.057	0.097
Age in Years	−0.336	−0.435
Age Squared	0.443×10−2	−0.557×10−2
Age Cubed	0.199×10−4	−0.259×10−4
Treatments:
Do Not Resuscitate	−0.695	−0.656c
Do Not Hospitalize	−0.628	−0.828c
Feeding Tube	0.157	0.100
Dialysis	0.424	0.619
Radiation	−0.178	NS
Tracheostomy Care	−0.308	−0.527
Ventilator/Respirator	NS	0.147
Diseases:
Diabetes	0.115	0.192c
Number of Cardiovascular Diseases	0.065	0.089c
Number of Musculoskeletal Diseases	0.063	NA
Number of Neurological Diseases	−0.045	NA
Asthma or COPD, or Both	0.116	0.126c
Pressure or Stasis Ulcer, Stage 2 or higher	−0.198	−0.273
Renal Failure	−0.034	NA
Pneumonia	−0.116	NA
Septicemia	−0.101	NA
Internal Bleeding	−0.103	NA
Hip Fracture in Last 180 Days	−0.073	NS
Other Fracture in Last 180 Days	0.05	0.145
Tuberculosis	NS	0.428
Constant	7.640	10.240
N	748,611	699,315
C statistic	0.61	0.64

Note: Variables included in the initial analysis but excluded from the final model (jointly not significantly different from zero at the 0.2 level): cancer, viral hepatitis, syncope (fainting), burns, surgical wounds, infection of the foot, and chemotherapy.

^aFull-information assessments include admission, annual and change of status assessments

^bAll risks factors were significant at the 0.001 level except for ventilator which was significant at the 0.05 level.

^cIndicates that these variables were imputed from a prior full information observation.

NA: In the quarterly assessments, these variables were not recorded and not imputed.

NS: Not significant at the 0.05 level.

COPD, chronic obstructive pulmonary disease.

Table 3.

Hospice Risk-Adjustment Logistic Models: Dependent Variable: Decedent Enrolled in Hospice

Variables	Model estimated on admission assessmenta	Model estimated on annual assessmenta	Model estimated on other full-information assessmenta,b	Model estimated on quarterly with imputed information assessmenta
Demographics:
Female	0.148	0.188	0.147	0.159
Age in Years	0.076***	0.055***	−0.016***	−0.057***
Age Squared	−0.096×10−2	−0.033×10−2	0.033×10−2***	0.077×10−2***
Age Cubed	0.041×10−4***	0.001×10−4***	0.021×10−4***	−0.035×10−4***
Length of Stay in Years	0.080**	−0.019***	0.121	0.037
Length of Stay Squared	−0.017	−0.283×10−2***	−0.018	−0.008
African American	−0.077	NS	NS	−0.184c
Asian or Native American	−0.425	−0.440	−0.470	−0.522c
Hispanic	NS	NS	0.258	NSc
No Schooling	−0.146	−0.292**	−0.197	−0.109c
8th Grade or Less	−0.063	−0.164	−0.172	−0.066c
Bachelor Degree or Higher	0.017	NS	NS	NSc
Not Married	NS	−0.132	−0.015	−0.111c
Diseases:
Sum of ADLs (Range 0–64)	0.667×10−2	0.030	0.012	0.022
Number of Diagnoses (Range 0–43)	−0.068	−0.024*	−0.083	NA
Congestive Heart Failure	−0.104	−0.114	−0.060	−0.111c
Dementia	0.099	0.188	0.118	0.088c
Emphysema or COPD	−0.047	NS	NS	−0.063c
Cancer	0.700	0.385	0.321	0.444c
Hip Fracture in Last 180 Days	−0.162	NS	NS	−0.058*
End-Stage Disease	1.547	2.585	1.462	2.246
Swallowing Problem	−0.098	0.178	−0.029**	NSc
Weight Loss	0.115	0.417	0.075	0.391
Weight Gain	−0.099	NS	−0.063	−0.059
Pressure Ulcer: at Least Stage 3	−0.040**	0.484	NS	0.309
HIV	NS	NS	NS	0.654*
Treatments:
Chemotherapy	−0.225	NS	NS	−0.568
Dialysis	−0.433	−0.400	−0.521	NS
Oxygen Therapy	0.02	0.211	NS	0.172
Radiation	−0.113**	NS	−0.169*	NS
Suctioning	−0.340	NS	−0.341	−0.243
Tracheostomy Care	−0.365	−0.683	−0.182*	−0.241
Ventilator/Respirator	−0.594	−1.060	−0.635	−1.113
Constant	−3.783	−4.081	−0.994	−0.300
N	403,405	77,217	262,788	357,846
C statistic	0.67	0.66	0.66	0.66

Note: Variables included in the initial analysis but excluded from the final model (jointly not significantly different from zero at the 0.2 level): renal failure, hip fracture interacted with dementia.

^aAll variables are significant at the 0.001 level unless otherwise noted as follows:

^*Significant at the 0.05 level.

^**Significant at the 0.01 level.

^***Jointly significant at the 0.10 level.

^bOther assessments include significant change of status and significant correction to prior full assessments.

^cVariable was imputed from a prior full-information observation.

NA: In the quarterly assessments, these variables were not recorded and not inputed.

NS: Not significant at the 0.05 level.

Table 4 presents means, standard deviations, and ranges for both QMs for all years. The average for the POD QM declined from 0.030 in 2003 to 0.016 in 2007, and the mean of the hospice QM increased from 0.010 to 0.129 during the same period. Both had large standard deviations, indicating high variability across nursing homes, thus offering the ability to differentiate between nursing homes using these outcomes. Figures 1 and 2 present the distribution of the two QMs in 2006.

Table 4.

Quality Measures: Descriptive Statistics

Place of death
	Number of nursing homes	Mean	Standard deviation	Minimum	Maximum
2003	15,203	0.030	0.184	−0.408	0.903
2004	15,167	0.029	0.180	−0.386	0.920
2005	15,036	0.026	0.180	−0.390	0.915
2006	14,987	0.021	0.180	−0.405	0.926
2007	14,987	0.016	0.191	−0.423	0.900

Hospice
2003	15,256	0.010	0.252	−0.838	1.000
2004	15,365	0.049	0.252	−0.713	1.000
2005	15,265	0.075	0.254	−0.773	1.000
2006	15,265	0.108	0.260	−0.726	1.000
2007	15,212	0.129	0.273	−0.771	1.000

FIG. 1.

Distribution of place-of-death quality measures, 2006 (14,987 nursing homes).

FIG. 2.

Distribution of hospice quality measures, 2006 (15,212 nursing homes).

We focus the remainder of the presentation on the QMs for 2005 and 2006, as the results for other years are similar. Figure 3 presents information about the stability of each QM over time and the relation between the two QMs. Figures 3A, 3C, and 3E include all nursing homes. Figures 3B, 3D, and 3F include only facilities with more than 17 decedents. The findings, however, are similar. Figures 3A and 3B show how many nursing homes changed deciles based on the POD QM between 2005 and 2006. Among all nursing homes (Fig. 3A), 20% remained in the same decile in both years, 14% improved by one decile, and 13% experienced deterioration of one decile. In total, 47% of all nursing homes experienced at most a one-decile change in their POD QM classification in 2 consecutive years.

FIG. 3.

Stability of place-of-death quality measures over time, 2005 versus 2006.

Figures 3C and 3D present similar information for the hospice QM: Fig. 3C shows that 28% of all nursing homes experienced no change, 17% improved, and 18% deteriorated for a total 63% of experiencing at most a one-decile change.

Figures 3E and 3F compare the classification of the same nursing home by the two QMs, in 2006. Seven percent of nursing homes were classified into the same decile by both QMs, wheras 7% each were classified into either one lower or higher decile by the two QMs.

The analysis in Table 5 also focuses on the issue of stability, but for outlier facilities. It answers the questions: 1) what percent of nursing homes that were classified as high (low) quality in 2005 were also classified as high (low) quality in 2006?; and 2) what percent of facilities classified as high (low) quality based on the POD QM were also classified as high (low) quality based on the hospice QM? Outlier facilities were defined in two ways: 1) nursing homes above the 90th or below the 10th percentile; and 2) nursing homes above the 75th or below the 25th percentile. Results are shown for all facilities and for those with more than the average number of decedents.

Table 5.

Stability in Identifying Outliers

Part I: Stability in identifying high-quality outliers	All nursing homesa	Nursing homes with more than average decedentsb
I. POD QM: 2005 versus 2006c
a) Highest-quality decile in 2005	25.6%	32.5%
b) Highest-quality quartile in 2005	43.8%	49.5%
II. Hospice QM: 2005 versus 2006
a) Highest-quality decile in 2005	40.1%	49.3%
b) Highest-quality quartile in 2005	61.4%	67.7%
III. POD QM versus Hospice QM for 2006d
a) Highest-quality POD decile	22.9%	23.9%
b) Highest-quality POD quartile	41.0%	42.7%

Part II: Stability in Identifying Low-Quality Outliers
I. POD QM: 2005 versus 2006
a) Lowest-quality decile in 2005	47.7%	63.9%
b) Lowest-quality quartile in 2005	56.2%	66.7%
II. Hospice QM: 2005 versus 2006
a) Lowest-quality decile in 2005	48.3%	63.1%
b) Lowest-quality quartile in 2005	71.2%	76.2%
III. POD QM versus Hospice QM for 2006
a) Lowest-quality POD decile	37.3%	43.3%
b) Lowest-quality POD quartile	47.4%	52.8%

^aThe sample size for the POD QMs included 15,036 nursing homes and for the hospice QM 15,265 nursing homes.

^bThe sample size for the POD QMs included 6892 nursing homes and for the hospice QM 6375 nursing homes.

^cExample: Of the 1504 nursing homes in 2005 in the top decile of quality, 25.6% remained in the top decile in 2006.

^dExample: In 2006 1523 nursing homes were at the top decile based on the POD QM; 349 of those (or 22.9%) were also classified into the top decile by the hospice QM.

POD, place of death; QM, quality measure.

Part I of Table 5 presents information for the highest-quality outliers. In general, there is higher agreement when the analyses are limited to nursing homes with more than average number of decedents (last column) as their QMs can be expected to be more stable over time. We also observe higher agreement when outliers are defined based on the 25th top percentile rather than the 10th top percentile of the quality distribution. The hospice QM shows substantially more stability between 2005 and 2006 compared with the POD QM. The percent of nursing homes that were classified as high quality by the POD QM as well as by the hospice QM was relatively high, similar to the percent classified consistently over time as high by the POD QM.

Part II of Table 5 presents similar information for the poor-quality outliers. We observe the same general trends as for the high-quality outliers except that all percents are higher, indicating more stability over time and more agreement between the POD and hospice measures.

We also observe that for both high and low outliers, agreement is higher when the comparison is limited to facilities with more than the average number of decedents. This is expected, as the larger sample size minimizes the random noise in the QMs and hence would lead to more stability in ranking nursing homes.

Discussion

This paper presents two new prototype EOL QMs for nursing homes. They were designed to be measured based on administrative data, the MDS, and Medicare claims, which are available for all Medicare and Medicaid certified nursing homes in the country, and could be integrated into a national quality reporting system. They are intended to fill a gap in the scope of quality domains that are currently included in NHC. Other EOL QMs may be needed as well.

The usefulness of these measures depends on their ability to accurately identify nursing homes providing low and high EOL quality of care. Ideally, we would have liked to validate the QMs against a gold standard. However, in the absence of a gold standard we typically rely on assessment of their face, content, and construct validity.^16,20 Face validity addresses the question of whether the QMs measure the domain of interest. As the two QMs are based on outcomes that are considered important to LTC residents at EOL^5–9 they do have face validity. Content validity relates to whether the measures include all the important risk factors.¹⁶ It is derived in this case from the very large number of risk factors that are available in the MDS and that were considered in the risk adjustment. Construct validity is typically assessed by comparing the behavior of the measures to other measures of the same domain. We do not have other “external” measures to compare the two proposed QMs with, but we have compared them with each other and have found that they show moderate to high agreement in identifying outliers, ranging from 20% to 50% (see Table 5). Clearly, additional work is needed to both refine these measures and to further validate them.

We developed these QMs on the MDS version 2.0 data. As of October 2010, nursing homes have been collecting MDS 3.0 in which some of the variables have different definitions. CMS has published a crosswalk between the two versions,²¹ and we have consulted it in developing the QMs. We anticipate that most of the difference between the QMs we present and QMs based on MDS 3.0 will relate to calibration of the risk models (i.e., the magnitude of the coefficients of the risk factors), rather than choice of the risk factors. However, future work to refine and validate these measures should be based on MDS 3.0.

Our finding that both QMs are more stable over time and have higher degree of agreement when identifying low-quality outliers than when identifying high-quality outliers may be because the QMs are better at identifying low-quality outliers. It may also reflect real differences in practices, with poor EOL practices being more persistent than high-quality practices. Answering this question is beyond the scope of this study.

Several limitations should be noted. The QMs we chose do not capture all the relevant domains for EOL care. Aspects of care, such as pain management, dyspnea care, bereavement counseling, and others are important to patients and their families and should be monitored and reported as part of a comprehensive EOL report card. In fact, many of these outcomes are the expected benefits of hospice enrollment. And although hospice enrollment might be viewed as a proxy for these services, enrollment in hospice does not guarantee that patients indeed receive them. Furthermore, although we have been able to adjust for many patient-level risk factors, we were unable to account for individual and family preferences. Another possible limitation might arise due to the arbitrary determination of the EOL period based on the last assessment. However, we do not expect this last limitation to introduce a systematic bias.

Finally, we reiterate that our objective in this work was to demonstrate that administrative data sets such as the MDS, when merged with Medicare claims, can be used to develop EOL QMs. More work is required to further refine and validate them. They should be subject to scrutiny by stakeholders and studied in relation to processes of care and other quality measures. Additional EOL QMs, capturing other aspects of care, should also be developed.

Ultimately, the development of reliable and valid EOL QMs, such as the ones proposed here and others, would enable public reporting of this domain of nursing home care. Prior studies have shown that such information can influence consumers' choice of providers,^22,23 make health care markets more efficient, and most importantly, by changing the incentives providers face^24,25 improve quality of EOL care. Such measures can also be incorporated in pay-for-performance (P4P) initiatives, in which payers create direct financial incentives to improve quality. The current CMS P4P pilot program for nursing homes^26,27 uses several of the QMs included in NHC, as well as hospitalizations. It could also include measures for EOL QMs, when they become available for all nursing homes.

Acknowledgments

We gratefully acknowledge funding from the National Institute of Nursing Research, Grant NR010727.

Author Disclosure Statement

No competing financial interests exist.