A Prospective Comparison of Informant- and Performance-Based Dementia Screening Tools to Predict In-hospital Delirium

Abstract

Dementia is an important risk factor for delirium, but the optimal strategy for incorporating cognitive impairment into delirium risk assessment at the time of hospital admission is unknown. We compared two informant-based screening tools for dementia and mild cognitive impairment (AD8 and D=(MC)²) to the Mini Mental State Exam (MMSE) and Mini-cog in predicting hospital-acquired delirium. This prospective cohort study at an academic medical center consisted of 162 medical inpatients over age 50 without delirium upon admission. Each participant was evaluated using the MMSE, Mini-cog, AD8, and D=(MC)² upon admission and was assessed daily for delirium. An MMSE ≤ 24 carried a 5.5 (95% CI 2.7 – 11.1) relative risk for delirium, whereas cognitive impairment detected by the Mini-cog, D=(MC)² or AD8 carried a 2-fold risk. Adding the D=(MC)² to the MMSE increased the sensitivity for predicting delirium from 52% (32 – 73) for the MMSE alone to 65% (46 – 85) if either test was positive. If both were positive, specificity was maximized at 97% (94 – 100) but sensitivity was 17% (2 – 33). The MMSE and Mini-cog identify a large proportion of patients at risk for hospital-acquired delirium, but the combination of performance- and an informant-based screens may maximize specificity and sensitivity.

INTRODUCTION

Delirium is characterized by an acute and fluctuating cognitive disturbance characterized chiefly by impaired attention.^{1, 2} Delirium occurs in 10–60% of hospitalized elderly patients.^{3, 4} Delirium is associated with protracted hospital stays and higher in-hospital and post-discharge morbidity and mortality.^5–9 Failure to identify and prevent delirium carries an estimated annual cost of $38–152 billion dollars.¹⁰ However, evidence-based interventions to prevent delirium exist, providing opportunities to improve patient outcomes and highlighting the need for effective risk-assessment tools.^{11, 12}

A key risk factor for delirium in hospitalized patients is baseline cognitive impairment: two-thirds of delirium cases occur in patients with dementia.^{8, 13} However, baseline cognitive function is rarely known at the time of hospital admission, and patients with early stage dementia or mild cognitive impairment rarely carry that diagnosis in their medical record.^14–16 Therefore, an efficient method of identifying patients with cognitive dysfunction at admission is a critical part of delirium risk-assessment.

Cognitive function can be assessed using performance-based measures such as the Mini Mental State Examination (MMSE) or informant-based measures.^{17, 18} The MMSE is a widely used method for grading cognitive function, and has been used in the development of existing delirium prediction tools.^{7, 19} However, administration of the MMSE can be time consuming, is insensitive for detecting early dementia, and results in the acute setting may not accurately reflect baseline cognitive function because of factors such as fatigue and inattention.²⁰ Some authors advocate using the brief Mini-cog, as an alternative performance-based screen for cognitive dysfunction.^{21, 22}

Informant-based screening tools may be more sensitive for detecting early dementia, are less likely to be influenced by the acute condition of the patient, and may complement performance-based tools when used in combination.²³ The AD8 and Dementia=(MC)² are two very brief informant-based screening tools that detect early dementia and mild cognitive impairment (Table 1).^{24, 25} We prospectively administered these screening tools at the time of admission to a cohort of non-delirious adult medical inpatients who were subsequently followed for the development of delirium during their hospitalization. We hypothesized that the combination of informant-based and performance-based screening tools would be more sensitive and specific than either tool alone.

Table 1.

Patient characteristics and outcome

	All No. (%)	No delirium No. (%)	Delirium No. (%)	p
Mean age ± SD	69 ± 12	68 ± 11	75 ± 13	0.007
Male sex	89(55)	76(55)	13(57)	0.9
Race				0.7
Asian/PI	15(9)	14(10)	1(4)
Black	24(15)	20(14)	4 (14)
Caucasian	123(76)	105(76)	18(78)
Visual impairment	42(26)	35(25)	7(30)	0.6
Hearing loss	39 (24)	30 (22)	9 (39)	0.1
Alcohol abuse	7 (4)	7 (5)	0 (0)	0.6
Depression	12 (7)	11 (8)	1 (4)	1.0
Known dementia	6 (4)	3 (2)	3 (13)	0.04
Moderate or severe illness	114 (76)	93 (72)	21 (95)	0.02
Dehydration	71 (44)	60 (43)	11 (48)	0.8
Dementia screening methods
MMSE ≤24	27 (17)	15 (11)	12 (52)	<0.001
Mini-cog ≤2	46 (29)	34 (25)	12 (52)	0.01
D=(MC)2 ≥2	24 (15)	17(12)	7(30)	0.02
AD8 ≥2	34(21)	25(18)	9 (39)	0.02

SD = standard deviation; PI = Pacific Islander; MMSE = mini-mental state exam

METHODS

Design and Setting

This was a nested prospective cohort study including all subjects with available informants enrolled in a larger study of delirium prediction at an academic medical center.

Participants and Measurements

Subjects were identified by reviewing daily admission logs for the University of California, San Francisco (UCSF) Moffitt-Long and Mount Zion hospitals. All non-intensive care unit patients aged 50 or older admitted through the Emergency Department (ED) to the medicine, cardiology, or neurology services from May 2010 to November 2010 were screened for eligibility through chart review or in person within 24 hours of admission by a trained research assistant. In-person screening included an assessment for delirium using the long version of the Confusion Assessment Method (CAM).²⁶ Subjects who were delirious upon initial evaluation, did not have an available informant, were non-English speaking or severely aphasic, or were admitted for alcohol withdrawal or comfort/hospice care were excluded. All patients or their surrogates provided written informed consent, and the study was approved by the Institutional Review Board at UCSF.

Of 1241 patients screened, 439 were eligible for enrollment. Of these, 180 declined to participate, 50 were discharged prior to the first follow-up visit and 47 did not have an available informant, leaving 162 included subjects.

Upon enrollment, cognitive performance was assessed with the MMSE and the mini-cog.^{17, 27} Permission for administration of the MMSE was granted by Psychological Assessment Resources, Inc., and each administration was paid for. The AD8 and Dementia=(MC)² questionnaires were administered to informants.^{24, 25} Baseline data regarding known delirium risk factors (age, visual and hearing impairment, alcohol use, depression, prior history of dementia in the medical record, illness severity, and dehydration) were collected from subjects and charts for use in the multivariable regression model. Details about the measurement of these variables have been described previously.¹⁹

Assessment of Outcomes

All subjects were assessed for delirium for 6 consecutive days after enrollment or until discharge, whichever came first. The short CAM, an internationally recognized and validated tool, was used to assess delirium.²⁶ The short CAM consists of a scripted interview, an MMSE and forward digit span, and a conversation with the subject’s nurse for any change or fluctuations in mental status.

Follow up assessments were performed by research assistants who were not blinded to the initial D=(MC)² or AD8 score. Some weekend follow-ups were performed by PGY-2, 3 or 4 neurology residents blinded to the initial assessment. Research assistants were trained by a board-certified neurologist (VCD) in the administration and interpretation of the CAM using published methods prior to enrollment of any subjects.²⁸ Training included the performance of independent long-version CAMs by the trainer and the trainee on a series of delirious and non-delirious patients until there was consistent agreement for each item on the CAM in 5 consecutive patients. In addition, a board-certified neurologist supervised the first 5 administrations of the CAM performed by each research assistant. All potential outcomes were validated in person by a board-certified neurologist (VCD or SAJ) blinded to the initial assessment.

Data Analysis

All variables were dichotomized with the exception of age. On the MMSE, a score of 24 or lower was considered evidence of dementia. The Mini-cog was considered positive if the score was 0, 1, or 2. A score of 2 or higher qualifies as a positive screen on the D=(MC)² and likewise for the AD8. To test for selection bias, we compared baseline characteristics between subjects included in the study and those excluded due to the lack of an informant. We then compared baseline characteristics between patients who developed delirium and those who did not using Fisher’s exact test or Chi Square for categorical variables and the two-tailed t-test for continuous variables. Clinical test characteristics for each of the four cognitive screening tools used alone or in combination were calculated with 95% confidence intervals (CI) using published methods.²⁹

To determine whether cognitive screening tools predict delirium independent of other known risk factors, we created multivariable logistic regression models incorporating variables known to be risk factors for delirium from previous studies: known dementia, age, vision impairment, hearing impairment, high BUN/Cr ratio, depression, and severe illness. Alcohol dependence was excluded from the multivariable analysis since none of the seven patients who screened positive developed delirium. A separate multivariable model was constructed for each cognitive screening tool. All statistical analyses were performed using STATA software.³⁰

RESULTS

The cohort consisted of 162 elderly patients (mean age 69 ± 12 years) and included more men than women (Table 1). The subjects were predominantly Caucasian, with the remainder split evenly between blacks and Asian/pacific islander. 75% of the informants were family members, 16% friends, and 4% caregivers or case managers. The mean screening MMSE score was 27 (range 7 – 30). A similar proportion of patients screened positive for cognitive impairment with the AD8 and the D=(MC)² (21 vs. 15%; p=0.15).

Delirium developed in 23 (14%) patients within the first 6 days of their hospitalization (Table 1). Patients who developed delirium were older, more likely carry a diagnosis of dementia or have moderate or severe illness. Patients with delirium were also more likely to have a low MMSE or Mini-cog score or screen positive on the AD8 or D=(MC)² (Table 1).

We calculated the relative risk, sensitivity, specificity, and likelihood ratios (LR+ and LR−) for the ability of the MMSE, Mini-cog, AD8, D=(MC)², and the MMSE in combination with AD8 or D=(MC)² to predict development of delirium (Table 2). The MMSE and Mini-cog each demonstrated the highest sensitivity at 52% (95% CI 32 – 73) for both tests, while the Mini-cog was the least specific at 75% (68 – 82). When considered in combination, sensitivity for delirium prediction rose to 65 – 70% if either informant-based measure or the MMSE was positive. Specificity was highest when both the D=(MC)² and the MMSE were positive (97% (94 – 100); Table 2). The results for the combination of informant-based measures with the Mini-cog were similar except for a loss of specificity.

Table 2.

Clinical performance of cognitive screening tools for predicting delirium

Tool	RR (95% CI)	Sn (95% CI)	Sp (95% CI)	LR+ (95% CI)	LR− (95% CI)
D=(MC)2 ≥2	2.5 (1.2, 5.5)	0.30 (0.12, 0.49)	0.88 (0.82, 0.93)	2.5 (1.2, 5.3)	0.79 (0.60, 0.94)
AD8 ≥2	2.4 (1.1, 5.1)	0.39 (0.19, 0.59)	0.82 (0.76, 0.88)	2.2 (1.2, 4.1)	0.74 (0.53, 0.90)
MMSE ≤24	5.5 (2.7, 11.1)	0.52 (0.32, 0.73)	0.89 (0.84, 0.94)	4.8 (2.6, 9.0)	0.54 (0.35, 0.76)
Mini-cog ≤2	2.7 (1.3, 5.7)	0.52 (0.32, 0.73)	0.75 (0.68, 0.82)	2.1 (1.3, 3.4)	0.64 (0.41, 0.82)
MMSE ≤24 AND D=(MC)2 ≥2	4.1 (1.8, 9.1)	0.17 (0.02, 0.33)	0.97 (0.94, 1.00)	6.0 (1.6, 22.5)	0.85 (0.70, 0.97)
MMSE ≤24 AND AD8 ≥2	2.7 (1.2, 6.3)	0.22 (0.05, 0.39)	0.93 (0.89, 0.97)	3.0 (1.1, 8.0)	0.84 (0.68, 0.96)
MMSE ≤24 OR D=(MC)2 ≥2	5.2 (2.4, 11.4)	0.65 (0.46, 0.85)	0.80 (0.73, 0.87)	3.2 (2.1, 5.1)	0.44 (0.25, 0.62)
MMSE ≤24 OR AD8 ≥2	5.8 (2.5, 13.1)	0.70 (0.51, 0.88)	0.78 (0.72, 0.85)	3.2 (2.1, 4.9)	0.39 (0.21, 0.55)

Sn = Sensitivity; Sp = Specificity; LR+ = positive likelihood ratio; LR− = negative likelihood ratio; RR = relative risk; CI = Confidence Interval

Of the four cognitive screening tools, only a low MMSE score reached statistical significance as an independent predictor in multivariable logistic regression incorporating known delirium risk factors (Table 3). Other independent predictors of delirium in the model included age (OR per year 1.07 (95% CI 1.01 – 1.13); p=0.02) and moderate or severe illness (OR 14.7 (1.6 – 136.1); p=0.02).

Table 3.

Multivariable analyses adjusting for known delirium risk factors

Tool	OR (95% CI)	p
D=(MC)2 ≥2	3.0 (0.7, 12.3)	0.12
AD8 ≥2	1.9 (0.6, 6.3)	0.28
MMSE ≤24	9.1 (2.4, 34.4)	0.001
Mini-cog ≤2	2.5 (0.8, 7.8)	0.12

Multivariable logistic regression adjusting for known dementia, age, vision impairment, hearing impairment, high Blood Urea Nitrogen to Creatinine ratio, depression, and severe illness. OR = Odds Ratio; CI = Confidence Interval

In clinical practice, one might choose to omit cognitive screening when trying to predict delirium in patients who are already diagnosed with dementia. We therefore repeated our analysis after omitting those patients with a known dementia diagnosis. Only 6 patients in the cohort carried a diagnosis of dementia in their medical record; 3 of these became delirious and all 3 screened positive with the four cognitive screening tools. With these 6 patients removed from the analysis, the RR for delirium among those who screened positive with the D=(MC)² or AD8 no longer reached statistical significance (1.8 (0.7 – 4.8); p=0.3 and 2.0 (0.8 – 4.7); p=0.1 respectively) while it remained significant both the MMSE and Mini-cog (5.0 (2.3 – 10.6); p<0.001 and 2.3 (1.0 – 5.2); p=0.04 respectively).

To look for possible bias introduced by excluding those patients without informants, we compared delirium risk factors and outcomes in those with and without informants. Patients without informants were younger (65 vs. 69 years; p=0.03), were more likely to have poor vision (45% vs. 26%; p=0.01), and were less likely to have a BUN/Cr ratio > 18 (19% vs. 44%; p=0.002). There was no difference in the mean MMSE score (27 in both groups; p=0.88) or the proportion of patients with low MMSE (17% in both groups; p=0.95) or Mini-cog scores (28 vs. 29%; p=0.91). Two (4%) excluded patients and 23 (14%) included patients became delirious (p=0.06).

DISCUSSION

The MMSE, Mini-cog, AD8, and D=(MC) are tools that have been validated for the detection of dementia and cognitive impairment, which is a leading risk factor for developing delirium in medically hospitalized patients. Low MMSE or Mini-cog scores are proven risk factors for delirium but informant-based measures of cognitive function have not been as extensively studied as tools for delirium prediction. In this study we sought to determine whether two simple informant-based cognitive screening tools could either replace or augment the MMSE or Mini-cog in the risk stratification of hospital-acquired delirium.

We found that the AD8 and the D=(MC)² can each identify patients at risk for delirium. However both performance-based measures captured a greater percentage of patients at risk for delirium without sacrificing specificity. If either a performance-based measure or an informant-based measure was positive, 65 – 70% of patients who would eventually develop delirium could be identified, and if both measures were positive delirium could be predicted with high specificity.

Likelihood ratios help clinicians determine the post-test probability of an outcome for individual patients. The incidence of delirium in our cohort was 14%, comparable with prior studies.¹³ As an illustrative example, if a patient were to screen positive with either the MMSE or the D=(MC)², based on the likelihood ratios presented here the risk of that patient developing delirium would be 34%. A negative screen on both tests would result in a delirium risk of 7%. If a patient screened positive on both the MMSE and the D=(MC)², the risk of developing delirium would be 49%.

Although dementia is an important risk factor for delirium, it alone does not account for all cases. Age independent of dementia, severe illness or major surgery, hip fracture, and sensory impairment are also known risk factors in addition to iatrogenic precipitants such as medications and sleep deprivation. Thus there is an inevitable ceiling effect in the sensitivity of any dementia test used to assess delirium risk. This could explain the observation that even when the screening tools studied here were used in combination to maximize sensitivity, only 70% of delirium cases could be predicted. Therefore any comprehensive delirium risk-assessment model must take into consideration factors beyond cognitive impairment.

Our multivariable analysis suggests that cognitive impairment as measured by the MMSE is an independent risk factor for delirium whereas the Mini-cog, D=(MC)² and AD8 are not. This may be a Type II error due to the effect of our small sample size, which is a limitation of this study. A further limitation is the predictive ability of the D=(MC)² and AD8 was only apparent when the 6 patients who already carried a diagnosis of dementia in their medical record were incorporated into the analysis, suggesting that in this cohort the informant-based screening tools were not as sensitive as the MMSE in detecting incident cognitive impairment as opposed to prevalent dementia. However, in many cases a patient’s full medical record may not be available at the time of hospital admission and an informant-based screening tool may be a sensitive proxy.

Additional limitations of this study include the lack of informants for every patient in the full cohort and the lack of ethnic diversity. It is possible that an unmeasured bias was introduced by excluding patients without informants from this study. The included patients were older than the excluded patients, which could explain the trend toward more delirium in the nested cohort. It is unlikely there would have been a major difference between D=(MC)² and AD8 results between included and excluded patients because there was no difference in MMSE and Mini-cog scores. The difference in BUN/Cr ratios would not be expected to introduce bias since it was not associated with delirium in either the nested or the entire cohort. An additional limitation is the fact that delirium was only assessed by the CAM once daily; because delirium fluctuates it is possible some cases were missed. We tried to mitigate this by including an interview with each subject’s caregiver or bedside nurse asking about overnight events and fluctuations in cognition over the preceding 24 hours. The sample make up of predominantly English-speaking Caucasians precludes generalizing our results to other ethnicities or non-English speakers. In addition, the population only consisted of medical inpatients so we are unable to generalize any findings to surgical patients.

In conclusion, this study provides evidence for the relative effectiveness of four different methods of screening for cognitive dysfunction in order to predict hospital-acquired delirium, two of which have not been studied in this context previously. In addition to reinforcing the understanding that both the MMSE and Mini-cog predict delirium, it gives health care providers two additional, informant-based options for delirium prediction.