Minimum Data Set Changes in Health, End-Stage Disease and Symptoms and Signs Scale: A Revised Measure to Predict Mortality in Nursing Home Residents

Abstract

OBJECTIVES

To revise the Minimum Data Set (MDS) Changes in Health, End-stage disease and Symptoms and Signs (CHESS) scale, an MDS 2.0-based measure widely used to predict mortality in institutional settings, in response to the release of MDS 3.0.

DESIGN

Development of a predictive scale using observational data from the MDS and Medicare Master Beneficiary Summary File.

SETTING

All Centers for Medicare and Medicaid Services (CMS)-certified nursing homes in the United States.

PARTICIPANTS

Development cohort of 1.3 million Medicare beneficiaries newly admitted to a CMS-certified nursing home during 2012. Primary validation cohort of 1.2 million Medicare recipients who were newly admitted to a CMS-certified nursing home during 2013.

MEASUREMENTS

Items from the MDS 3.0 assessments identified as likely to predict mortality. Death information was obtained from the Medicare Master Beneficiary Summary File.

RESULTS

MDS-CHESS 3.0 scores ranges from 0 (most stable) to 5 (least stable). Ninety-two percent of the primary validation sample with a CHESS scale score of 5 and 15% with a CHESS scale of 0 died within 1 year. The risk of dying was 1.63 times as great (95% CI=1.628–1.638) for each unit increase in CHESS scale score. The MDS-CHESS 3.0 is also strongly related to hospitalization within 30 days and successful discharge to the community. The scale predicted death in long-stay residents at 30 days (C=0.759, 95% confidence interval (CI)=0.756–0.761), 60 days (C=0.716, 95% CI=0.714–0.718) and 1 year (C=0.655, 95% CI=0.654–0.657).

CONCLUSION

The MDS-CHESS 3.0 predicts mortality in newly admitted and long-stay nursing home populations. The additional relationship to hospitalizations and successful discharges to community increases the utility of this scale as a potential risk adjustment tool.

The Changes in Health, End-stage disease and Symptoms and Signs (CHESS) scale is a composite measure specifically designed to identify health instability in long-term care populations using routinely collected data.¹ Since its development in 2003, it has been used, primarily as a risk adjustment tool, to identify people with high health instability who are likely to experience adverse health outcomes, including death.^2–10

The assessment instrument from which the CHESS scale was derived, the Minimum Data Set (MDS) version 2.0, underwent extensive revision, and version 3.0 was released for use in October 2010.¹¹ MDS 3.0 does not contain several items from the original MDS-CHESS scale, including change in cognitive status, change in ability to perform activities of daily living (ADLs) self, edema, and leaving 25% of food uneaten. The purpose of this study was to revise the CHESS scale based on the elements available in the MDS 3.0 assessment using the U.S. national population of nursing home residents. The availability of a revised scale might allow researchers to continue to identify vulnerable nursing home residents at risk of hospitalization and death.

METHODS

Data

Data were taken from the MDS 3.0 and the Medicare Master Beneficiary Summary File (MBSF). The MDS 3.0 is a standardized resident assessment required for every resident receiving care in Medicare- and Medicaid-certified nursing facilities. Residents are evaluated at regular intervals and at significant events, such as admission, discharge, and change in health status. The MDS 3.0 contains measures and information relevant to the care of residents, including active diagnoses and cognitive and physical functioning. The MBSF contains demographic information, including sex, race, age, and date of death.

Study Samples

Development Cohort

The MDS-CHESS 3.0 scale was developed using Medicare beneficiaries aged 65 and older newly admitted to a nursing home in 2012 (N=1,297,117). New admissions were defined as individuals with no MDS assessments in the prior year whose first MDS assessment in 2012 was an admission record, a 5-day prospective payment system assessment, or an entry tracking record immediately followed by a 5-day prospective payment system or admission assessment.

Primary Validation Cohort

The MDS-CHESS 3.0 scale was validated using Medicare beneficiaries aged 65 and older newly admitted to a nursing home in 2013 (N=1,217,008) defined using criteria identical to those used for the development cohort.

Secondary Validation Cohort

We also tested the validity of the revised CHESS scale for long-stay nursing home residents who had been in the nursing home for at least 90 days, defined using the first annual or quarterly assessment in 2013 (N=1,070,672). Long-stay residents tend to be older and more functionally and cognitively impaired and are often considered separately in analyses.¹²

Measures

Candidate Items

Members of the research team, including a practicing geriatrician and a palliative care physician, identified items from the MDS 3.0 that might be related to the likelihood of death. Supplementary Table S1 shows a complete list of the candidate measures selected. Candidate measures are factors identified in the literature as consistently predicting death in nursing home residents, including clinical signs and symptoms, comorbid conditions, ADL dependency, and cognitive decline and related behaviors.^13–16

Cognitive functioning was assessed according to an item indicating acute mental status change and the Cognitive Function Scale (CFS).¹⁷ The CFS combines staff perceptions of resident cognition with the results of a 3-task cognitive screener used to classify residents as cognitively intact (1), mildly impaired (2), moderately impaired (3), or severely impaired (4). Cognitive impairment sequelae were also measured using the Aggressive Behavior Scale (ABS) and a measure of impairment in daily cognitive function. The ABS measures the presence and frequency of physical behavioral symptoms directed at others (e.g., hitting or kicking), verbal behavioral symptoms directed at others (e.g. threatening or cursing), behavioral symptoms not directed at others (e.g., scratching oneself), and rejection of care.¹⁸ Response categories for all 4 items range from behavior was not exhibited in the last week (0) to behavior occurred daily (3).

Physical functioning is measured according to a dependency summary score for ADLs.¹⁹ For each of seven ADLs, residents are rated as independent (0), requiring supervision (1), requiring limited assistance (2), requiring extensive assistance (3), or being totally dependent (4). Severe functional impairment is defined as having a summary score of 21 or higher, meaning that the resident requires at least extensive assistance on all 7 named tasks.

Outcomes

Date of death was obtained from the MBSF. All cohorts were followed for 2 years to determine whether death occurred within that time. Follow-up time was calculated as days from the qualifying assessment to date of death or end of follow-up. Cut-points were created for death at 30, 60, and 365 days after assessment. Cases were not censored because the MBSF is considered to be a complete record of death, regardless of location.

For the primary validation cohort, we also examined hospitalization within 30 days of the qualifying assessment and successful discharge. Successful discharge was defined as discharge to the community within 90 days of the qualifying assessment and remaining alive and out of a nursing home for at least 30 days after discharge. ²⁰

Analysis

Bivariate cox proportional hazard models were used to determine which candidate measures predicted death over the 2-year observation window in the development sample. Consistent with the development of the original scale, measures with a hazard ratio (HR) less than 1.5 were excluded from the list of potential scale inputs.¹ A series of 48 item arrangements were tested by calculating a score and examining results from a Cox proportional hazards model. The final configuration was decided by considering the hazard ratios of the score arrangement, the bivariate HRs of the components, and the parsimoniousness of the score.

The resulting score was calculated in the primary validation cohort and evaluated using the HR from a Cox proportional hazards model. Survival curves were produced according to CHESS value. Binary outcomes, including death at 30, 60, 365, and 730 days; hospitalization; and community discharge, were examined at each scale value, and significant differences between levels of the score were assessed using logistic regression and chi-square tests. In the secondary validation cohort, we assessed binary outcomes of death at 30, 60, 365, and 730 days according to score. Analyses were conducted using SAS version 9.4 (SAS Institute, Ind., Cary NC) and Stata version 14 (Stata Corp., College Station, TX).

RESULTS

Development Cohort

The strongest predictors of death over 2 years in the development cohort were clinician determination of life expectancy of less than 6 months (HR=3.64), severe cognitive impairment (HR=2.72), severe functional impairment (HR=2.57), moderately or severely impaired daily decision-making (HR=2.41), worst pressure ulcer is slough or necrotic (HR=2.17), acute mental status change (HR=2.15), and dehydration (HR=2.13). Additional items in the final scale include Stage 3 or greater pressure ulcers (HR=2.03), swallowing disorder (HR=1.93), respiratory failure (HR=1.89), shortness of breath (HR=1.86), heart failure (HR=1.86), and an aggressive behavior score of 3 or greater (HR=1.67). Items that were considered but did not improve the fit of the scale were cancer, cirrhosis, pneumonia, internal bleeding, total urinary or bowel incontinence, and renal failure.

The indicators were grouped in a manner similar to that of the original CHESS scale, resulting in a scale ranging from 0 (most stable) to 5 (least stable). Individual indicators contribute 1 point to the overall scale as follows: clinician determination of life expectancy of less than 6 months (end-stage disease); ADL score of 21 or greater; or CFS of 4, acute mental status change, ABS of 3 or greater, or at least moderate impairment in daily decision-making. The remaining health conditions (respiratory failure, heart failure, dehydration, swallowing disorder, shortness of breath, pressure ulcers) contribute a maximum of 2 points, with 0 points for no conditions present, 1 point for one condition present, and 2 points for 2 or more conditions present. Pressure ulcers are defined as either presence of Stage 3 or greater, or worst pressure ulcer is slough or necrotic.

Primary Validation Cohort

The development (2012) and validation (2013) cohorts had similar proportions of residents with the risk factors included in the revised CHESS scale and similar mortality (Table 1). At admission, 48.5% of the primary validation cohort had a CHESS score of 0 (590,390), 30.5% a score of 1 (n=370,766), 15.3% a score of 2 (n=186,399), 4.0% a score of 3 (n=48,266), 0.85% a score of 4 (n=10,358), and 0.06% a score of 5 (n=693).

Table 1.

Mortality and Minimum Data Set Changes in Health, End-Stage Disease and Symptoms and Signs 3.0 (MDS-CHESS 3.0) Predictors in the Development and Validation Cohorts of New Nursing Home Residents

Cohort Characteristics	Development Cohort (2012)	Validation Cohort (2013)
Observations, n	1,297,117	1,217,008
30-day mortality (%)	6.0	3.3
60-day mortality (%)	10.2	6.4
1-year mortality (%)	26.5	23.8
2-year mortality (%)	38.1	35.2
Age, mean ± SD	81.6 ± 8.2	81.7 ± 8.2
Female (%)	64.3	64.7
Dual Medicare and Medicaid coverage (%)	22.2	24.7
Length of stay, days, mean ± SD	149.7 ± 360.2	150.4 ± 319.4
Measures included in MDS-CHESS 3.0
Life expectancy <6 months	1.0	1.1
Severe cognitive impairment (Cognitive Function Scale Score 4)	3.6	3.3
Acute mental status change	2.2	1.7
Aggressive behavior score ≥ 3	2.5	2.4
Impaired daily decision-making	7.5	6.7
Severe physical impairment (Activity of Daily Living Scale score ≥ 21)	18.4	17.6
Dehydration	0.4	0.4
Pressure ulcersa	4.7	3.4
Swallowing disorderb	5.6	5.1
Respiratory failure	2.2	2.5
Shortness of breath when lying, sitting, or with exertion	19.5	18.9
Heart failure	18.7	19.5

^aPresence of Stage 3 or greater or worst pressure ulcer is slough or necrotic.

^bChoking, holding food, lost liquid, or painful swallowing.

SD = standard deviation.

Death

Ninety-five percent of the primary validation cohort with a CHESS scale score of 5 and 23.8% of those with a CHESS scale score of 0 died within 2 years (Table 2). The same trends were observed for death at 30 days (C=0.759, 95% confidence interval (CI)=0.756–0.761), 60 days (C=0.716, 95% CI=0.714–0.718), and 365 days (C=0.655, 95% CI=0.654–0.657). Survival curves for each level of the CHESS scale score are displayed in Figure 1. In the primary validation cohort, higher CHESS scores at admission were associated with greater risk of death over the next 2 years. The risk of dying was 1.63 times as great (95% CI=1.628–1.638) for each unit increase in CHESS scale score, meaning that the least stable individuals (CHESS scale score 5) were 11.5 times more likely to die in 2 years as the most stable individuals (CHESS scale score 0).

Table 2.

Selected Outcomes for the Validation Cohorts of New Nursing Home Admissions and Long-Stay Nursing Home Residents According to Minimum Data Set Changes in Health, End-Stage Disease and Symptoms and Signs 3.0 (MDS-CHESS 3.0) Scale Score

Outcome	MDS-CHESS 3.0 Score
	0	1	2	3	4	5
Admissions, n	590,390	370,766	186,399	48,266	10,358	693
Total sample, %	48.5	30.5	15.3	4.0	0.85	0.06
Death in 30 days, %	0.9	2.9	6.5	15.0	29.3	73.7
Death in 60 days, %	2.5	6.3	12.3	23.0	38.2	78.8
Death in 1 year, %	14.5	26.3	38.3	51.1	64.9	91.8
Death in 2 years, %	23.8	39.3	52.6	64.2	75.9	95.2
Hospitalized in 30 days, %	12.6	19.1	24.8	30.0	31.7	7.8
Successful discharge from nursing home, %	75.6	65.6	56.1	37.6	24.9	20.1
Long-stay residents, n	424,200	348,522	216,288	66,283	14,342	1,037
Total sample, %	39.6	32.6	20.2	6.2	1.3	0.10
Death in 30 days, %	0.0	0.1	0.3	0.7	1.7	2.4
Death in 60 days, %	0.2	0.4	0.9	1.8	3.4	3.5
Death in 1 year, %	6.3	10.4	14.1	20.3	28.9	36.4
Death in 2 years, %	17.5	25.2	30.6	39.1	49.5	58.7

Chi-Square tests for all outcomes were statistically significant (p<.001).

Higher scores indicate greater health instability.

Figure 1.

Two-year survival according to Minimum Data Set Changes in Health, End-Stage Disease and Symptoms and Signs 3.0 (MDS-CHESS 3.0). MDS-CHESS 3.0 scores range from 0 (low-instability in health) to 5 (high instability in health)

Hospitalization

In the primary validation cohort, we observed a curvilinear relationship between CHESS score and 30-day hospitalization. Having a CHESS score of 3 or 4 was associated with greater risk of 30-day hospitalization (30%) than a score of 0 (12.6%) or a score of 5 (7.8%) (C=0.597, 95% CI=0.596–0.598).

Successful Discharge to the Community

Higher CHESS scores are associated with lower likelihood of successful discharge to the community (C=0.614, 95% CI=0.613–0.615). In the primary validation cohort, 75.6% of residents with a CHESS scale score of 0 were discharged to community within 90 days of qualifying assessment and remained out of the nursing home for at least 30 days, compared with 20.1% of the least stable residents.

Secondary (Long-Stay) Validation Cohort

Although developed and validated in admission cohorts, the CHESS scale also predicts death in the long-stay population of nursing home residents. Overall, 39.6% of the long-stay cohort had a CHESS score of 0, 32.6% a score of 1, 20.2% a score of 2, 6.2% a score of 3, 1.3% a score of 4, and 0.1% a score of 5. Two-year mortality was 18% in the most stable long-stay residents and 59% in the least stable (Table 2). This upward trend was also evident in death within 30, 60, and 365 days.

DISCUSSION

Using measures currently available for all nursing home residents in the United States, we provide a revision to the CHESS scale. For the outcome of death within the 2-year observation window, our scale performs similarly to the original; a 1-point increase in CHESS score was associated with a risk of death over the 2-year observation window that was 1.63 times as greater (95% CI=1.63–1.64). Of individuals admitted to a U.S. nursing home with a CHESS score of 5, 95.2% died within 2 years, compared with 23.8% of those admitted with a baseline CHESS score of 0. Our finding that CHESS acuity is also related to hospitalizations adds to the breadth of possible uses of the revised scale.

In addition to updating the score to include items available in the current version of the MDS, our methods addressed 2 of the limitations previously identified.¹ First, the original scale was developed using a cohort of fewer than 30,000 people being treated in complex continuing care (CCCs) hospitals in Ontario, Canada. We developed our score using a population-based cohort of new admissions from all CMS-certified nursing homes in the United States, approximately 1.3 million beneficiaries in 15,923 nursing homes across the country. Second, using the MBSF, we were able to capture all deaths and dates, regardless of location. In the development of the original scale, censoring occurred when people were discharged from the CCCs, so deaths were captured only if they occurred in the CCC setting. Having complete death information meant that people were not censored, which improved the ability to identify the items most strongly related to death in 2 years.

We have also demonstrated the relationship between the score and outcomes of interest other than mortality. For the admission cohort, the revised scale was related to 30-day hospitalization and successful discharge from the nursing facility. Hospitalization is a central measure for the CMS Skilled Nursing Facility Value-Based Purchasing Program.²¹ Being able to identify persons at risk of hospitalization is important to skilled nursing facilities interested in evaluating risk associated with participation in such programs. Furthermore, having a measure of who is likely to be discharged from and stay out of nursing homes will help facilities involved in rebalancing initiatives²² to identify nursing home residents who are likely to thrive in community settings, if given the appropriate supports.

The revised CHESS score performs similarly to other composite measures designed to predict death (c-statistics between 0.60 and 0.70),²³ but a significant amount of variation remains explained. A limitation of our analyses was that we were limited to 2 years of follow-up time to have a training and validation sample. Internal validation cohorts are critical to the development of predictive models^24,25 and improve their acceptability and value in research. In addition, the quality of the data that the MDS captures, which has been shown to vary according to facility and state, limited our analyses.^26,27 There may be some unmeasured facility characteristics affecting the MDS assessments and measurements used to build and evaluate the score.

Finally, although the current CHESS score is primarily used for risk adjustment, we believe the revised score has implications for advance care planning and identifying residents for hospice referral. Clinician determination of life expectancy of less than 6 months was the strongest predictor of death over follow-up, but this indicator is rarely used. Clinician uncertainty or discomfort with prognosis may contribute to the unnecessary and burdensome care that many nursing home residents receive at the end of life.^28,29 The literature supports a median end-of-life period of 3.25 years.³⁰ With a median end-of-life period of 3.25 years, advance care planning conversations would certainly benefit those with a CHESS score of 2 or greater (53% of whom are likely to die within 2 years).

In summary, the revised CHESS scale can be easily constructed using data that U.S. nursing homes routinely collect using the current version of the standardized resident assessment. We expect this revised CHESS scale to be a valuable tool for researchers, nursing home administrators, and policy-makers who want to direct existing resources and new policies toward nursing home residents at greatest risk of adverse outcomes.

Supplementary Material

1

Table S1: Additional information on the complete set of items included in the developmental bivariate models, including frequencies, hazard ratios, and p-values.