The Minimum Data Set 3.0 Cognitive Function Scale

Abstract

Background

The Minimum Data Set (MDS) 3.0 introduced the Brief Interview for Mental Status (BIMS), a short performance-based cognitive screener for nursing home (NH) residents. Not all residents are able to complete the BIMS and are consequently assessed by staff. We designed a Cognitive Function Scale (CFS) integrating self-report and staff-report data and present evidence of the scale’s construct validity.

Design

Retrospective cohort study.

Subjects

Consisted of three cohorts: 1) long-stay NH residents (N=941,077) and 2) new admissions (N=2,066,580) during 2011–2012, and 3) residents with the older MDS 2.0 assessment in 2010 and the newer MDS 3.0 assessment (n=688,511).

Measures

MDS 3.0 items were used to create a single, integrated four-category hierarchical CFS that was compared to residents’ prior MDS 2.0 Cognitive Performance Scale scores and other concurrent MDS 3.0 measures of construct validity.

Results

The new CFS suggests that 28% of the long-stay cohort in 2011–2012 were cognitively intact, 22% were mildly impaired, 33% were moderately impaired, and 17% were severely impaired. For the admission cohort, the CFS noted 56% as cognitively intact, 23% as mildly impaired, 17% as moderately impaired, and 4% as severely impaired. The CFS corresponded closely with residents’ prior MDS 2.0 Cognitive Performance Scale scores and with performance of Activities of Daily Living, and nurses’ judgments of function and behavior in both the admission and long-stay cohorts.

Conclusion

The new CFS is valuable to researchers as it provides a single, integrated measure of NH residents’ cognitive function, regardless of the mode of assessment.

INTRODUCTION

The new Minimum Data Set (MDS) 3.0 was designed to improve the federally mandated assessment process of nursing home residents through a number of mechanisms. One is that it incorporates the patient’s voice by requiring interviews with residents with scripted questions to assess subjective states such as cognitive functioning.¹ The designers of the MDS 3.0 posited that having residents involved in the assessment process would promote resident-centered care and improve the accuracy of reporting subjective states such as pain, mood, and cognition.^2,3

The MDS 3.0 added the Brief Interview for Mental Status (BIMS), a short performance-based cognitive screener expressly designed to facilitate cognitive screening.⁴ For residents who are deemed unable to participate in this standardized cognitive screener, or unable to complete the interview questions, their cognitive functioning is assessed by nursing staff using standardized instructions. Staff do not complete the cognition assessment items for residents able to complete the BIMS. Findings from the national evaluation of the MDS 3.0 suggested that 90% of residents were able to complete the BIMS.⁴ Other research examining the rate of completion among long-stay residents who were rated as being able to understand and able to be understood (therefore appropriate for participation), found that almost 17% of these residents did not complete the BIMS⁵ and were therefore assessed by staff. Given that the items that comprise the BIMS and the staff’s assessment differ and cannot be combined, the ability of researchers, policy makers, and clinicians to stratify the entire nursing home population by one universal cognition measure becomes challenging.

In the previous version of the resident assessment instrument, MDS 2.0, a Cognitive Performance Scale (CPS) was developed using assessment items to assign residents into easily understood cognitive performance categories.⁶ In the new MDS 3.0 and introduction of the BIMS, two items that comprised this well-known and validated scale are not available for individuals who complete the BIMS cognitive screener: staff’s ratings of a resident’s cognitive skills in daily decision making and a resident’s short term memory. Therefore, there is not a comprehensive measure available for all residents in the MDS 3.0 to capture their cognitive functioning. As such, we describe the development and initial construct validity testing of the Cognitive Function Scale (CFS), a single, integrated measure of cognition based on variables readily available in the MDS 3.0. The CFS provides a detailed description of the resident’s cognitive status without additional assessment burden since it emerges directly from the required admission, quarterly, significant change, annual, and discharge assessments, and includes applications for research, care planning, quality measurement, outcome measurement, and resource utilization.

METHODS

Data

Data for the development of the CFS and for the construct validity testing of the new scale come from the Minimum Data Set (MDS) versions 3.0 and 2.0, standardized, primary screening and assessment tools of a nursing home resident’s health status. MDS assessments are the foundation of the comprehensive assessment for all residents in a Medicare and/or Medicaid-certified nursing home. The MDS contains items that measure physical, psychological, and psychosocial functioning and is completed on admission, quarterly, annually, discharge and whenever there are significant changes in residents’ health statuses. While its primary purpose as an assessment tool is for resident care planning, data collected from MDS assessments are also used for reimbursement systems and monitoring the quality of care provided to nursing home residents.

Sample

To validate the new CFS, we assembled three samples: The first sample consisted of all long-stay nursing home residents identified by having an annual MDS assessment completed between July 1, 2011 and June 30, 2012 (N=941,077). The second sample included all new admissions to the nursing home as identified by having a full admission assessment (not an admission tracking or re-entry assessment) during July 1, 2011-June 30, 2012 (N=2,066,580). This cohort included residents who were newly admitted to the NH to receive post-acute Skilled Nursing Facility (SNF) care. The long-stay and new admission cohorts were used to support the construct validity of the CFS with concurrently measured indicators of cognition. The third group was a sample of nursing home residents who had full assessments completed using the MDS 2.0 in July or August of 2010 and then again with the new MDS 3.0 within 100 days of the previous MDS 2.0 assessment (n= 688,511). This sample was designed to compare residents’ previous MDS 2.0 CPS score to their new MDS 3.0 CFS score.

Variables

The BIMS performance-based screener includes five items in total: three questions measure temporal orientation and two questions assess recall. The response options allow for differential scoring for answers to temporal orientation questions that are “close” to correct and partial credit when a resident can recall an item after being prompted or cued. The scores from each item are summed to create a total BIMS score ranging from 15 (all items correct) to 0 (no items correct). The BIMS has displayed high levels of sensitivity (94.4%) and specificity (78.6%) in identifying cognitive impairment⁷ and has been categorized into 3 levels: intact/borderline cognition (13–15) moderate cognitive impairment (8–12), severe cognitive impairment (0–7).^4,7,8

The CPS score is calculated using a complex algorithm which assigns residents a score between 0–6 based on five MDS items (i.e., daily decision making, eating self-performance, ability to make self understood, short term memory, and whether or not the resident is comatose). All residents in the MDS 3.0 have information on their ability to be understood, their eating performance, and an indicator if they are comatose. However, for residents who complete the BIMS screener in the MDS 3.0 assessors do not complete the two items Short Term Memory and Cognitive Skills for Daily Decision Making. Staff complete these items for residents who are deemed inappropriate or unable to complete the BIMS screener but do not complete them if residents are able to complete the BIMS assessment, regardless of their BIMS score. Therefore, without these two measures, a CPS score cannot be derived for residents who complete the BIMS. However, for residents who do not complete the BIMS, we are able to calculate the CPS score using the algorithm described elsewhere.⁶

Creating the Cognitive Function Scale

We created a hierarchical, four-level Cognitive Function Scale, a priori, using either the BIMS or CPS score, depending on which method of assessment was completed. The highest level of impairment, “severely impaired,” includes individuals who did not complete the BIMS and have a CPS score of 5 or 6. Residents were assigned a value of “moderately impaired” if they scored between a 0–7 on the BIMS or a 3 or 4 on the CPS. Residents with a BIMS score of 8–12 or a CPS score of 0–2 were considered to be “mildly impaired.” Residents were considered “cognitively intact” if they were able to complete the BIMS and scored between 13 and 15.

To test the construct validity of the CFS, we examined it in relation to a number of concurrently measured behavioral observations associated with poor cognitive performance captured on the MDS 3.0. These included two measures of communication skills (i.e. ability to understand and ability to make self understood), wandering behaviors (i.e. resident wanders daily), and measures of ADL functional performance. For the ADL items, we included the individual functions that remain intact for the shortest and longest time during the course of a progressive dementing disorder — dressing and eating, respectively.^9,10 We also included the ADL scale score,¹¹ which ranges from 0–28 (higher scores indicate more impairment) and includes items for bed mobility, transfer, locomotion on unit, dressing, eating, toilet use, and personal hygiene. Finally, we included whether residents exhibited any signs or symptoms of delirium using the items from the Confusion Assessment Method (CAM).¹²

Analyses

We first described the distribution of CFS, BIMS, and CPS scores for the samples of long-stay residents and new admissions. We then tested the CFS measure’s cross-sectional construct validity separately for long-stay residents and new admissions using ANOVA and chi-square analyses and MDS 3.0 data on functioning that was measured concurrently. To test the CFS’ longitudinal validity for the sample for the sample of residents with both MDS 2.0 and MDS 3.0 assessments, we examining the correlation between individuals’ CPS scores on the MDS 2.0 and their CFS scores on the MDS 3.0. We also examined the MDS 2.0 CPS scores in relation to the MDS 3.0 BIMS scores for those who were able to complete the BIMS (n=501,350)) and the relation of the MDS 2.0 CPS to the MDS 3.0 CPS for individuals who did not complete the BIMS (n=187,161). We tested the strength of the correlation between the MDS 2.0 CPS score and the ADL scale score as well as the MDS 3.0 CFS score and the ADL scale score.

RESULTS

Long-Stay Cohort

The new CFS reveals that 28% of the long-stay cohort was cognitively intact, 22% were mildly impaired, 33% were moderately impaired, and 17% were severely impaired on their annual MDS 3.0 assessments (see Table 1). Over the study period, our examination found that the BIMS was attempted with 79% of long-stay residents on their annual assessment. Of those who started the interview, 7% were unable to complete the cognitive test. As a result, a total of 27% of long-stay residents required staff reporting of cognitive status. The average BIMS score of those in the long-stay cohort who completed the BIMS was 9.6 (SD= 4.8) indicating that the average resident in our sample that completed the BIMS had moderate cognitive impairment.

Table 1.

Distribution of Selected Measures of Cognition-Related Clinical Items, Behaviors, and Demographic Characteristics by Cognitive Function Scale Score for the Long-Stay (n=941,077) and Admission Cohorts (n=2,066,580)

		Long-Stay Cohort					Admission Cohort
		Intact	Mild Impairment	Moderate Impairment	Severe Impairment	Total	Intact	Mild Impairment	Moderate Impairment	Severe Impairment	Total
	Distribution of CFS Impairment Levels	28%	22%	33%	17%	100%	56%	23%	17%	4%	100%
Communication Patterns
	Never Makes Self Understood	0.0%	0.1%	4%	60%	11%	0.0%	0.1%	3%	50%	3%
	Never Able to Understand	0.0%	0.1%	2%	50%	9%	0.0%	0.1%	2%	40%	2%
Functional Impairments
	Totally Dependent in Dressing	8%	11%	19%	59%	21%	3%	6%	13%	48%	8%
	Totally Dependent in Eating	3%	4%	9%	50%	13%	2%	3%	9%	45%	5%
	Average ADL Score (28 Point Scale)	15.9	17	19	24.2	18.9	16.4	17.6	19.3	23.6	17.5
Wandering Behaviors
	Wandering	0%	1%	4%	5%	3%	0%	1%	3%	4%	1%
Signs and Symptoms of Delirium
	CAM Measure: Inattention (Not Present)	96%	89%	68%	45%	76%	98%	91%	69%	43%	89%
	CAM Measure: Disorganized Thinking (Not Present)	96%	89%	71%	58%	79%	98%	91%	73%	60%	90%
	CAM Measure: Altered Level of Consciousness (Not Present)	99%	98%	95%	81%	95%	100%	98%	93%	73%	97%
	CAM Measure: Psychomotor Retardation (Not Present)	99%	98%	95%	81%	94%	100%	98%	94%	76%	97%
Demographics
	Female	64%	66%	72%	75%	69%	64%	59%	62%	60%	62%
	From Community	10%	12%	14%	14%	13%	4%	7%	12%	11%	7%
	Married	26%	20%	15%	18%	20%	14%	12%	11%	17%	13%
	Average Age	72.8	78	81.5	77.1	78	75	80	82	77	77.5
	White	78%	77%	76%	73%	76%	82%	80%	78%	69%	81%

Note: All row differences, within cohort, tested with Chi-Squares or ANOVAs and statistically significant at the p<.001 level.

ADL Score on a scale of 0–28 where 0= Completely Independent and 28= Complete Dependence

ADL=Activities of Daily Living; CAM= Confusion Assessment Method

Of the 51,442 residents in the long-stay sample who were not able to complete the BIMS and the 166,424 who did not receive the BIMS, 62% were severely impaired with an MDS 3.0 CPS score of 5 or 6. 33% of the long-stay cohort had MDS 3.0 CPS scores of 3 or 4 and only 4% had scores of 0–2. This suggests that the majority of long-stay residents who did not attempt or did not complete the BIMS were cognitively impaired.

Admission Cohort

For the admission cohort, the CFS found 56% of residents to be cognitively intact, 23% as mildly impaired, 17% as moderately impaired, and 4% as severely impaired (see Table 1). Among the admission cohort, 94% of residents (N= 1,939,966) attempted the BIMS. Of new admissions that started the cognition screener, 3% were unable to complete it. Thus, only 9% of newly admitted patients required staff assessment of their cognition status. The average BIMS score of residents admitted to the nursing home who were able to complete the BIMS was 12 (SD= 3.9) confirming that the admission cohort is less cognitively impaired than the long-stay population.

Of the residents in the admission cohort who either started and did not complete the BIMS (n= 55,470) or who were not administered the BIMS (N=126,624), 44% had an MDS 3.0 CPS score of 5–6 suggesting severe impairment. Furthermore, 39% of the admission cohort had MDS 3.0 CPS scores of 3 or 4 and 17% had scores of 0–2. This suggests that individuals newly admitted to nursing homes who are unable to complete the BIMS are cognitively impaired.

When comparing the CFS scores for the admission and long-stay cohorts to a number of cognition-related conditions and diagnoses measured concurrently in the MDS 3.0 (See Table 1), we see that higher CFS impairment levels are consistently accompanied by statistically significant increases in the percentages of the two samples that are unable to understand or make themselves understood, total dependence in dressing and eating, and wandering. In addition, we see that the average ADL scale score increases with cognition severity in both the admission and long-stay cohort. Finally, lower levels of cognitive function are associated with higher proportions of residents who never exhibit signs and symptoms of delirium.

Cohort with MDS 2.0 and MDS 3.0 Assessments

For the sample of residents with both MDS 2.0 and MDS 3.0 assessments, their prior MDS 2.0 CPS scores are presented with their MDS 3.0 BIMS or CPS scores (Table 3). The majority of residents who were very impaired on their MDS 2.0 CPS (78%) did not complete the BIMS on a subsequent MDS 3.0 assessment. This suggests that the greatest level of cognitive impairment is associated with an inability to complete the BIMS. Of those severely impaired with an MDS 2.0 CPS score of 5 or 6, 19% had an MDS 3.0 BIMS score between 0–7, with the majority having BIMS scores of zeros. Ninety three percent of residents rated as being cognitively intact or mildly impaired on the MDS 2.0 CPS (scores of 0–2) completed the BIMS on their first MDS 3.0 assessment. Half of those residents with MDS 2.0 CPS scores of 0–2 had the highest BIMS scores (13–15) on their next assessment and 27% had BIMS scores between 8–12. Among residents who were rated as having moderate cognitive impairment (scores of 3 or 4) on their prior MDS 2.0 CPS, the largest proportion (43%) scored between 0–7 on the BIMS. This suggests that the lowest BIMS scores correspond with moderate impairment levels of the CPS.

Table 3.

Correspondences between Residents’ Minimum Data Set 2.0 Cognitive Performance Scale Scores and their Minimum Data Set 3.0 Brief Interview for Mental Status and Cognitive Performance Scale Scores (n= 688,511).

	MDS 3.0 CPS Categories (n=187,161)				MDS 3.0 BIMS Categories (n=501,350)
MDS 2.0 CPS Categories		Scores 0–2 Intact/Mild Impairment	Scores 3 & 4 Moderate Impairment	Scores 5 & 6 Severe Impairment	Scores 15–13 Intact/ Borderline	Scores 2–8 Moderate Impairment	Scores 7–0 Severe Impairment
Scores 0–2 Intact/Mild Impairment	N	14630	4115	2847	143282	78430	42372
	Row %	4%	2%	1%	50%	27%	15%
Scores 3 & 4 Moderate Impairment	N	14830	40964	28965	30016	57535	127391
	Row %	1%	18%	10%	10%	19%	43%
Scores 5 & 6 Severe Impairment	N	1076	5681	74053	728	2468	19128
	Row %	0%	6%	72%	1%	2%	19%

Notes. MDS= Minimum Data Set; CPS= Cognitive Performance Scale; BIMS= Brief Interview of Mental Status

For the sample of residents who had MDS 2.0 and MDS 3.0 assessments, we found that CPS scores on the MDS 2.0 were highly correlated with individuals’ MDS 3.0 CFS scores (r=0.72, P <.001; Table 2). Of those with scores suggesting intact/mild impairment (0–2) on the MDS 2.0 CPS, 50% and 33% had ratings of “Intact” and “Mild Impairment,” respectively, on their MDS 3.0 CFS. The majority of residents (56%) with MDS 2.0 CPS scores signally moderate impairment (a score of 3 or 4) had an MDS 3.0 CFS rating of “Moderate Impairment.” Finally, 72% of individuals with an MDS 2.0 CPS score of 5–6, signaling severe impairment, had an MDS 3.0 CFS rating of “Severe Impairment.” This suggests that the new CFS corresponds well with previous ratings of cognitive status and that the degree to which the scores correspond with one another does not differ by cognitive status.

Table 2.

Correspondences between Residents’ Minimum Data Set 2.0 Cognitive Performance Scale Scores and their Minimum Data Set 3.0 Cognitive Function Scale Scores (n= 688,511).

MDS 2.0		MDS 3.0 CFS Score
		Intact	Mild Impairment	Moderate Impairment	Severe Impairment
CPS Score 0–2 Intact/Mild Impairment	N	143282	93060	46487	2847
	Row %	50	33	16	1
	Col %	82	55	19	3
CPS Score 3–4 Moderate Impairment	N	30016	72365	168355	28965
	Row %	10	24	56	10
	Col %	17	43	70	27
CPS Score 5–6 Severe Impairment	N	728	3544	24809	74053
	Row %	<1	3	24	72
	Col %	<1	2	10	70

Notes. MDS= Minimum Data Set; CFS= Cognitive Function Scale; CPS= Cognitive Performance Scale; Col= Column;

Results from correlation analyses suggest that MDS 2.0 CPS scores are significantly correlated with residents’ ADL scale scores (r=0.455, p<.001). The new MDS 3.0 CFS score is similarly correlated with residents’ ADL scale scores (r=0.457, p<.001). This suggests that the strength of the relationship between cognition measures and ADL performance remains the same with the new CFS.

DISCUSSION

Our previous research⁵ suggests that 17% of long-stay nursing home residents who are eligible to complete the BIMS on the MDS 3.0 either fail to complete or are not administered the cognitive screener. All residents, whether they are able to complete the BIMS interview or not, should have an easily identifiable way to be categorized viz. their cognitive functioning. Therefore, we developed the CFS as a methodological tool to define all residents’ cognitive functional status in the nursing home setting, regardless of the mode of assessment on the MDS 3.0. We believe that the CFS has many applications and could prove useful to researchers, clinicians, and care planners to uniformly determine a resident’s cognitive status, regardless of how their cognitive function was assessed. CFS levels are strongly related to other validated measures revealing underlying cognitive function therefore indicating good face validity.

The addition of the new resident self-report items, particularly the BIMS, is an improvement to the nursing home resident assessment process over what existed with the MDS 2.0. However, our data suggests that 26% of long-stay residents and 11% of new admissions either were not asked or were not able to complete the BIMS screener on their annual/admission assessment leaving staff to assess these residents’ cognitive function. Without a scale that utilizes both types of assessment, there is no way to consistently assign a value of cognitive function to the entire population of nursing home residents with the MDS 3.0. The CFS represents an approach to this objective for the cognition domain.

Without a tool like the CFS, it would be impossible to track changes within facilities as well as make comparisons across facilities; particularly for those facilities without high rates of resident self-report. This measure is also important for case-mix adjustment and comparisons across the types of patients being served by a NH since all patients can be classified by their level of cognitive function, regardless of their ability to respond to the BIMS. Additionally, this tool will provide consistency for measurement of residents over time—particularly if they lose the ability to participate in BIMS testing due to progression of disease or new insult.

There are some limitations to this study worth noting. Our findings rest on the assumption that MDS data are collected and scored in accordance with federally recommended protocols. It can be expected that some MDS users may not follow the recommended assessment protocols, and our findings do not indicate how sensitive the CFS might be to such variation. Because in the sample comparing MDS 2.0 and MDS 3.0 data, the follow-up took place between October 9, 2010 and December 9, 2010 (100 days after an MDS 2.0 assessment in July or August 2010), just after the introduction of MDS 3.0 when staff would have less familiarity with new procedures. This might have increased the number of BIMS scores that could not be completed as well as their accuracy. In addition, our study looks at a year cross-section of data. It would be worthwhile to compare the change in CFS scores for residents over time as another form of construct validation testing. Future independent testing of the CFS against gold standard clinical measures of cognition such as the Folstein Mini-Mental Status Examination (MMSE),¹³ the Montreal Cognitive Assessment (MoCA),¹⁴ or the St Louis Mental Status Examination (SLUMS)¹⁵ is also indicated.

CONCLUSIONS

The new CFS has good face validity and can become a valuable tool for classifying all residents’ cognitive function, regardless of their mode of assessment. When combined with other MDS measures, the CFS provides information to researchers, program planners, and policy-makers and promotes a better understanding of nursing home residents with cognitive impairment.