Four Sensitive Screening Tools to Detect Cognitive Dysfunction in Geriatric Emergency Department Patients: Brief Alzheimer’s Screen, Short Blessed Test, Ottawa 3DY, and the Caregiver-completed AD8

Abstract

Background

Cognitive dysfunction, including dementia and delirium, is prevalent in geriatric emergency department (ED) patients, but often remains undetected. One barrier to reliable identification of acutely or chronically impaired cognitive function is the lack of an acceptable screening tool. While multiple brief screening instruments have been derived, ED validation trials have not previously demonstrated tools that are appropriately sensitive for clinical use.

Objectives

The primary objective was to evaluate and compare the Ottawa 3DY (O3DY), Brief Alzheimer’s Screen (BAS), Short Blessed Test (SBT), and caregiver-completed AD8 (cAD8) diagnostic test performance for cognitive dysfunction in geriatric ED patients using the Mini Mental Status Exam (MMSE) as the criterion standard. A secondary objective was to assess the diagnostic accuracy for the cAD8 (which is an informant-based instrument) when used in combination with the other performance-based screening tools.

Methods

In an observational cross-sectional cohort study at one urban academic university-affiliated medical center, trained research assistants collected patients’ responses on the Confusion Assessment Method for the Intensive Care Unit, BAS, and SBT. When available, reliable caregivers completed the cAD8. The MMSE was then obtained. The O3DY was reconstructed from elements of the MMSE and the BAS. Consenting subjects were non-critically ill, English-speaking adults over age 65 years, who had not received potentially sedating medications prior to or during cognitive testing. Using an MMSE score ≤ 23 as the criterion standard for cognitive dysfunction, the sensitivity, specificity, likelihood ratios, and receiver operating characteristic area under the curve were computed. Venn diagrams were constructed to quantitatively compare the degree of overlap among positive test results between the performance-based instruments.

Results

The prevalence of cognitive dysfunction for the 163 patients enrolled with complete data collection was 37%, including 5.5% with delirium. Dementia was self-reported in 3%. Caregivers were available to complete the cAD8 for 56% of patients. The SBT, BAS, and O3DY each demonstrated 95% sensitivity, compared with 83% sensitivity for the cAD8. The SBT had a superior specificity of 65%. No combination of instruments with the cAD8 significantly improved diagnostic accuracy. The SBT provided the optimal overlap with the MMSE.

Conclusions

The SBT, BAS, and O3DY are three brief performance-based screening instruments to identify geriatric patients with cognitive dysfunction more rapidly than the MMSE. Among these three instruments, the SBT provides the best diagnostic test characteristics and overlap with MMSE results. The addition of the cAD8 to the other instruments does not enhance diagnostic accuracy.

INTRODUCTION

The term cognitive dysfunction includes mild cognitive impairment, delirium, and various stages of dementia. Dementia describes a neurodegenerative process characterized by problems with memory, judgment, orientation, and executive functioning. These deficits must be severe enough to impair social or occupational capabilities, and they must represent a decline from previous baseline functioning.¹ Alzheimer’s disease (AD), which will afflict 1 in 85 persons by 2050,² is one well-recognized etiology of dementia, but other causes include strokes, Parkinson’s disease, and head injury.³ Where dementia is characterized by a constant and continual decline in higher cognitive functioning, delirium is a temporary disorder of mental capabilities that is a symptom of an acute medical illness. The Diagnostic and Statistical Manual, 4^th revision criteria to diagnose delirium includes a fluctuating disturbance in consciousness and change in cognition that develops over time, caused by an acute physiological stressor.⁴ Mild cognitive impairment is early AD, manifesting with problems in memory, language, or other mental functions that can be detected by certain screening tests, but that do not otherwise interfere with daily living.^1,5 In this manuscript, we use the term “cognitive dysfunction” to describe dementia with or without delirium, but we cannot extrapolate our findings to include mild cognitive impairment.

Cognitive dysfunction in geriatric adults is often unrecognized in emergency department (ED) encounters.^6–9 These are potentially vulnerable subsets of society since many live alone at home without a reliable safety net.¹⁰ Aging baby-boomers will increase the proportion of emergency patients with occult cognitive dysfunction for the next 30 years.^11,12 If at that time current interventions can delay cognitive decline by 1 year, over 9 million fewer persons would need more expensive higher levels of care.² Today, delirium-related care costs $152 billion, and dementia expenses exceed $150 billion annually in the United States.^13,14 Most of the latter expenses are borne by caregivers.¹⁵ Worldwide, the cost of dementia in 2009 was estimated at $422 billion, which represented a 34% increase between 2005 and 2009.¹⁶

Outpatient physicians often fail to diagnose cognitive dysfunction, so emergency-based case-finding is an opportunity to identify and intervene for this condition.^17–19 Delirium in ED patients is associated with increased mortality and inpatient length of stay.^20–22 Dementia has been identified as an independent predictor of short-term post-ED discharge recidivism, functional decline, and institutionalization.^22–25 Accordingly, several expert panels have recently incorporated the assessment of cognitive dysfunction as a minimal core competency for emergency medicine residents, and a quality indicator for all ED providers.^26,27 Unfortunately, emergency providers miss up to 70% of patients with an Mini Mental Status Exam (MMSE) ≤ 23 during routine care.^6,7,28 One barrier to routine screening of older adults for cognitive dysfunction is the lack of a suitably brief, ED-validated, sufficiently sensitive and reasonably specific instrument by which to identify high-risk patients during busy acute care encounters.^29,30 Although a variety of simple screening instruments have been tested in ED settings, all have previously demonstrated suboptimal diagnostic accuracy.^9,29,31–33 Based upon the authors’ recent synopsis of a systematic review of dementia diagnostic instruments,²⁹ in addition to personal communication with the original investigators who had derived the tools, four appropriately brief (< 1 minute) and relatively straightforward screening instruments were selected to evaluate further in geriatric ED patients. In selecting the optimal instruments, we considered the cognitive domains tested, as well as the face validity, content validity, and internal consistency of each tool. Each of the instruments we selected to evaluate was originally derived to evaluate dementia, and each tests a patient’s orientation, registration-recall, verbal fluency, and/or attention.³⁴

The Short Blessed Test (SBT, Data Supplement 1), sometimes called the Orientation-Memory-Concentration Test, is a weighted six-item instrument originally designed to identify dementia.³⁵ The SBT evaluates orientation, registration, and attention. It was originally validated on patients in a skilled nursing facility and active community-dwelling senior citizens. Results were correlated with senile plaque burden on autopsy, but not against biomarkers, amyloid tracer uptake, or performance-based parameters of dementia that have recently been recommended for the diagnosis of Alzheimer’s disease.^1,5 The SBT has demonstrated excellent reliability with scores generally within four errors of their original score within three weeks of testing.³⁶

The Brief Alzheimer’s Screen (BAS, Data Supplement 2) is a four-item instrument tied to an algebraic equation that was recently designed to distinguish individuals with mild dementia from normal elders.³⁷ It was retrospectively derived with patients from the Consortium to Establish a Registry for Alzheimer’s Disease database that consists of AD patients and normal controls from research centers around the United States.³⁸ The BAS was then retrospectively validated on a cohort of mild AD patients and non-demented controls from the University of Kentucky Alzheimer’s Disease Research Center using the National Institute of Neurological and Communicative Disorders, and Stroke-Alzheimer’s Disease and Related Disorders Association criteria as the criterion standard for AD.³⁹ Previous research had demonstrated that animal naming was especially useful to discriminate early AD patients and non-demented subsets.³⁸ Reliability and other measures of internal consistency were not evaluated in the derivation trial. No prospective validation trials have previously been reported.

The Ottawa 3 Day-Year (O3DY, Data Supplement 3) is also a four-item screening instrument designed to quickly identify subsets of patients at higher risk for cognitive dysfunction.⁴⁰ The elements of the O3DY are shared by the MMSE (day and year orientation) and the BAS (date orientation, spelling “world” backwards), testing orientation and verbal fluency, respectively. The O3DY was retrospectively derived from the Canadian Study of Health and Aging (CSHA-1), a randomly selected sampling of Canadian adults over age 65 years beginning in 1991.⁴¹ In deriving the O3DY, investigators excluded institutionalized or severely demented subjects, non-English speakers, and vision- or hearing-impaired subjects. The criterion standard for dementia was the consensus of a neuropsychologist, nurse, and physician using a variety of bedside screening instruments, historical information, physical exam, and normative data for their population. Candidate variables for the O3DY were abstracted from the Modified MMSE.⁴² The instrument was validated on individuals from CSHA-1, who were contacted again in 1996. The original investigators did not prospectively assess the O3DY, nor did they evaluate the reliability or internal consistency of this instrument.

The SBT, BAS, and O3DY require a cooperative patient. Furthermore, these instruments do not assess the impact of cognitive dysfunction on social or occupational activities of daily living (i.e. performance-based measures), which have recently been promoted as an important criterion to distinguish demented and non-demented populations.^1,43 On the other hand, the Alzheimer’s Disease-8 (AD8) was designed to administer to informant-caregivers or, if caregivers were not available, to patients in order to distinguish mild dementia from those without dementia through eight performance and memory-based questions (Data Supplement 4).^44–46 The AD8 was derived to detect dementia in research participants, and was validated on consecutive patients referred to one memory clinic. The derivation and validation samples included non-demented subjects (Clinical Dementia Rating [CDR] scale⁴⁷ = 0) to severe dementia patients (CDR = 3). The original rationale in constructing the rule was the observation that informant-based assessment of intraindividual change was generally more sensitive than brief performance-based instruments in capturing the earliest signs of dementia. The criterion standard for dementia was an expert clinician’s assessment with at least one other cognitive domain dysfunction, plus interference with daily activities using the CDR. The psychometric properties of the AD8, including internal consistency, construct and criterion validity, inter-rater, intra-rater, and inter-modal reliability all support the AD8 as a valid and reliable dementia screening measure compared with the criterion standard CDR and neuropsychological assessments.^44,45,48 The AD8 was most strongly correlated with tests of episodic memory, psychomotor function, and executive ability, but not with semantic memory or verbal fluency, providing a rationale for combining the AD8 with instruments like the BAS that incorporate these domains. More recently, the AD8 has been validated against AD biomarkers with a strong relationship to cerebrospinal fluid amyloid and tau measurements, and positron emission tomography scans using the amyloid ligand Pittsburgh compound B.⁴⁸ Thus, the AD8 is an appropriate screening tool for dementia, but may not be sensitive to other more acute causes of cognitive dysfunction.

With the exception of the AD8, which had previously been administered to patients or caregivers in ED settings, none of these tools has previously been validated against an acceptable criterion standard in emergency settings.^9,29 Furthermore, no prior ED trials have evaluated these instruments while distinguishing delirium from other forms of cognitive dysfunction in aging adults. The objective of our research was to evaluate and compare the diagnostic accuracy for cognitive dysfunction of four brief screening instruments in geriatric ED patients.

METHODS

Study Design

This was a prospective, cross-sectional, convenience sampling in the ED of one urban academic medical center. The study was approved by the Barnes Jewish Hospital Institutional Review Board with written informed consent required.

Study Setting and Population

Barnes-Jewish Hospital is a Level I trauma center academic teaching hospital in St. Louis, Missouri with more than 90,000 total ED visits annually, 20% of which are aged 65 years or older. According to the availability of three research assistants (RAs) from June 10, 2009 to March 9, 2010, all ED patients over age 65 years were approached for enrollment in a convenience sampling.

Study Protocol

Enrollment occurred in the ED over equally distributed day, evening, and overnight shifts on weekdays and weekends. RAs were medical or pre-medical students who received standard training to administer the Richmond Agitation and Sedation Scale (RASS),⁴⁹ Confusion Assessment Method for the Intensive Care Unit (CAM-ICU),⁵⁰ the MMSE,⁵¹ SBT, BAS, and the caregiver-completed AD8 (cAD8). The three-item recall in the MMSE and BAS used different terms (MMSE used pineapple-desk-quarter while BAS used apple-table-penny) in order to avoid the bias of a learning phenomenon by re-testing the same three items. Standard training consisted of one-on-one review of the original derivation trials for each instrument, followed by practice administration of each instrument in sequence to other RAs. These practice sessions were directly observed by one investigator (CRC) and any variations in test administration were eliminated before data collection ensued. In addition, the first five subjects enrolled by each RA were under the supervision of the same investigator.

During enrollment hours, RAs monitored the electronic medical record board for potentially eligible patients. When such patients were identified, the emergency physician was approached for permission to describe the study to potential subjects. Exclusion criteria included patients receiving mental-status altering medications (anti-emetics, benzodiazepines, or narcotics) prior to or during the testing period, emergency physician judgment of critical illness precluding informed consent or safe data collection, subject inability to consent or comply with data acquisition, non-English speaking, or refusal to complete the questioning. If obviously cognitively impaired patients were recognized by the RA before testing, caregiver informed consent with subject assent was employed.⁵² RAs noted psychoactive medication administration via computer physician-order entry time stamps.

For eligible and consenting or assenting patients, the research assistant administered the RASS followed by the CAM-ICU, followed by the SBT and BAS. The MMSE was evaluated last. All of these instruments were administered by the RA using a standardized data collection form. When present, the cAD8 was completed by a reliable caregiver, as defined by a co-residing family member, or family or friend with daily exposure to the patient. The MMSE, SBT, BAS, and cAD8 were not scored until the completion of data collection for each individual patient. Because of the potential for subject test-retest bias,^53,54 and test fatigue for acutely ill older adults with the administration of multiple instruments in the time-constrained, stressful ED environment, the O3DY was not administered in its original form. Instead, the elements of the O3DY were constructed post-hoc from shared components of the MMSE and BAS. As part of a larger project, the Deficit Accumulation Index was administered to patients and caregivers as one measurement of frailty.⁵⁵ One component of the Deficit Accumulation Index inquires about past history of dementia.

Data Analysis

Analysis was conducted according to the Standards for Reporting of Diagnostic Accuracy criteria⁵⁶ using SPSS (version 16.0; SPSS Inc, Chicago IL) and MEDCALC (version 11.3.6; MedCalc Software, Mariakerke, Belgium). Basic demographic features between non-enrolled and enrolled patients were assessed for normally distributed data using t-tests, or chi-square for non-parametric data where appropriate. The criterion standard for cognitive dysfunction was an MMSE score ≤ 23. Standard operating characteristics of diagnostic tests were computed, including sensitivity, specificity, likelihood ratios, and receiver operating characteristic (ROC) curves with area under the curve (AUC), and 95% confidence intervals (CI).⁵⁷ The AUCs were also compared for statistically significant differences.⁵⁸ Our a priori Type I error rate was a p-value less than 0.05, and no adjustments for multiple comparisons were made to define statistical significance. In addition to computing the sensitivity and specificity at each score for the SBT, BAS, and cAD8, the ROC curve was visually inspected to determine the optimal cutoff point for each instrument that would simultaneously maximize sensitivity and specificity. To assess convergent validity, Venn diagrams were constructed to evaluate proportional overlap between abnormal cognitive test results for the SBT, BAS, O3DY, and MMSE.

Sample Size

The previous validation trials for the BAS, O3DY, and cAD8 to identify dementia reported sensitivities of 99%, 80%, and 92% respectively.^37,40,45 Verifying a sensitivity of 85% with 5% range of error (in other words if the point estimate was 90% then the true value would be somewhere between 85% and 95%, or standard deviation of 5%) and a baseline prevalence of cognitive dysfunction of 35%⁹ would require 146 subjects to be enrolled with complete data collection.⁵⁹ Assuming 15% of subjects would have incomplete data collection based upon our previous trials, we planned a priori to enroll 170 subjects.⁹

RESULTS

Between June 2009 and March 2010 we approached 630 patients, excluded 461, and enrolled 169 subjects (Figure 1). Enrolled subjects did not differ significantly from non-enrolled subjects by age (non-enrolled mean age 77 years), or sex (non-enrolled 64% female). Six patients did not complete the MMSE because they were discharged or admitted before the completion of data collection, or because they simply refused to continue, so 163 subjects were included in this analysis.

Figure 1.

Flow Diagram for Patient Enrollment

Enrolled subjects had a mean age of 78 years, and 61% were female. African Americans represented 49% of the patients; the remaining 51% were white. As defined by the MMSE, cognitive dysfunction was identified in 60 out of 163 (37%; 95% CI = 29% to 45%), and delirium was identified in nine (5.5%; 95% CI = 2.7% to 10.5%). Caregivers were present and willing to complete the cAD8 for 91 patients (56%). Dementia or Alzheimer’s was reported on the Deficit Accumulation Index in 3.5% (95% CI = 2.9% to 4.4%). Compared with the MMSE, which identified 37% with a score ≤ 23, each of the screening instruments would have identified substantially more patients with cognitive dysfunction: BAS, 65%; cAD8, 55%; O3DY, 66%; and SBT, 43%. As demonstrated in Figure 2, the largest area of overlap with a positive MMSE was with the SBT. The cAD8 was not assessed in the Venn diagram because 44% did not have the test available.

Figure 2. Venn Diagram for Abnormal Cognitive Screening Tests.

All subjects in this Venn diagram had abnormal cognitive screening test results. The overlapping regions of the rectangles represent agreement between the various instruments and are proportional to the actual results. The following key depicts the regions of agreement. The caregiver-taken Alzheimer’s Disease-8 (cAD8) was excluded from this analysis because not every subject had a cAD8 completed.

A = Mini Mental Status Exam (MMSE) only abnormal

B = Short Blessed Test (SBT) only abnormal

C = Brief Alzheimer’s Screen (BAS) only abnormal

D = Ottawa 3-Day-Year (O3DY) only abnormal

E = MMSE and SBT abnormal

F = MMSE and BAS abnormal

G = SBT and BAS abnormal

H = MMSE and O3DY abnormal

I = SBT and O3DY abnormal

J = O3DY and BAS abnormal

K = MMSE, SBT, and BAS abnormal

L = MMSE, O3DY, and SBT abnormal

M = MMSE, BAS, and O3DY abnormal

N = SBT, O3DY, and BAS abnormal

O = MMSE, SBT, BAS, and O3DY abnormal

As demonstrated in Table 1, the sensitivity of the SBT, BAS, and O3DY for an MMSE ≤ 23 were all 95%, all significantly better than the cAD8 and prior descriptions of the Six Item Screener.^9,31,32 In addition, the ROC AUCs of the SBT (0.930) and BAS (0.934) are statistically superior to that of the cAD8 (0.816, p = 0.01 compared with the BAS, and p = 0.008 compared with the SBT). Review of the ROC curve (Table 1 and Figure 3) did not identify a superior cutoff point by which to define abnormal cognitive function compared to previous non-ED based validation trials (SBT > 4, BAS < 26, cAD8 ≥ 2). The O3DY ROC curve was not assessed because it is a dichotomous test, unlike the other screening instruments. The specificities of the SBT and cAD8 were significantly better than the BAS or O3DY. The negative likelihood ratios for the SBT, BAS, and O3DY are less than 0.1, and would each significantly reduce the post-test probability of cognitive dysfunction as defined by an MMSE score ≤ 23.⁶⁰ However, the positive likelihood ratio for each test in isolation is insufficient to significantly increase the post-test probability of cognitive dysfunction.

Table 1.

Diagnostic Test Characteristics of SBT, BAS, O3DY, and cAD8

	Sensitivity, % (95% CI††)	Specificity, % (95% CI)	Positive Likelihood Ratio (95% CI)	Negative Likelihood Ratio (95% CI)	AUC^ (95% CI)
SBT	95 (88–98)	65 (61–67)	2.7 (2.2–3.0)	0.08 (0.03–0.2)	0.930 (0.862–0.971)
BAS	95 (88–98)	52 (48–54)	2.0 (1.7–2.2)	0.10 (0.03–0.3)	0.934 (0.867–0.974)
O3DY	95 (85–99)	51 (46–53)	2.0 (1.6–2.1)	0.10 (0.03–0.3)	N/A
cAD8	83 (71–91)	63 (55–68)	2.2 (1.6–2.8)	0.27 (0.1–0.5)	0.816 (0.727–0.886)
SBT+cAD8*	91 (81–97)	27 (20–30)	1.2 (1.0–1.4)	0.32 (0.1–0.9)	N/A
BAS+cAD8†	97 (90–99)	11 (6–12)	1.1 (0.9–1.1)	0.27 (0.04–1.6)	N/A
O3DY+cAD8‡	100	0	N/A	N/A	N/A

^*Abnormal SBT or abnormal cAD8.

^†Abnormal BAS or abnormal cAD8.

^‡Abnormal O3DY or abnormal cAD8.

^{^}n = 102 subjects who had data for all four instruments collected.

^††Confidence intervals for the estimated parameters were computed based upon constant chi-square boundaries as described in Fleiss JL. Statistical Methods for Rates and Proportions. New York, NY: Wiley, 1981.

SBT = Short Blessed Test; BAS = Brief Alzheimer’s Screen; O3DY = Ottawa 3-Day-Year; AUC = area under the receiver operating characteristic curve; cAD8 = caregiver-completed Alzheimer’s Disease-8

Figure 3.

ROC Curves for SBT, BAS, and cAD8*

	AUC (95% CI)
BAS	0.934 (0.867–0.974)
SBT	0.930 (0.862–0.971)
cAD8	0.816 (0.727–0.886)†

*N= 102 subjects who completed the MMSE, SBT, and BAS and had a caregiver present to complete the cAD8.

†cAD8 ROC AUC differs significantly from BAS (p = 0.01) and SBT (p = 0.008).

ROC = receiver operating characteristic; SBT = Short Blessed Test; BAS = Basic Alzheimer’s Screen; cAD8 = caregiver-completed Alzheimer’s Disease -8; AUC = area under the ROC curve

The CAM-ICU identified nine patients with delirium. The SBT, BAS, and O3DY demonstrated cognitive dysfunction in all nine of these patients. The diagnostic test characteristics for each instrument did not change when the nine delirium patients were excluded from the analysis.

When assessing the diagnostic accuracy of various combinations of the patient-administered tests in conjunction with the cAD8 for the 102 subjects who had all four instruments available, sensitivity was only improved with the combination of either an abnormal BAS or O3DY with an abnormal cAD8, but at the expense of significantly worse specificities (Table 1).

DISCUSSION

The National Institutes of Health State of the Science Conference Statement on the prevention of Alzheimer’s disease and cognitive decline recently emphasized the importance of diagnosing neurodegenerative processes, despite the paucity of evidence to support specific therapeutic interventions at this time.⁶¹ Dementia and delirium are prevalent in geriatric ED populations, representing up to 40% of such patients.^6–9,31,32 Previously validated, paperless, and brief cognitive dysfunction screening instruments did not replicate with sufficient sensitivity for ED application,^9,32 although they improved on clinician gestalt, which was missing 70% of cases. The ideal ED screening instrument would be quick, with 100% sensitivity and specificity, high inter-rater reliability, and requiring no specific equipment or unwieldy operator memorization. Since such a test does not exist for ED cognitive dysfunction, clinicians need to assess acceptable levels of imperfection in new instruments against the backdrop of standard practice and the risk-to-benefit tradeoffs of false-negative and false-positive results.⁶² For example, screening tests like the O3DY were derived to be extremely brief, reasonably sensitive instruments at the expense of specificity, which likely explains the lower specificity we observed.

The SBT, BAS, and O3DY each offer excellent sensitivity to identify geriatric cognitive dysfunction as defined by an MMSE score ≤ 23. Among the three, the SBT offers the best associated specificity for the cutoff value of > 4 to define abnormal. Unfortunately, both the SBT and the BAS are somewhat onerous to remember, and each requires computations that limit their usefulness in the busy ED.³⁰ One solution to offset the laborious calculations and need to memorize the individual components of the screening tests would be to incorporate them into handheld devices or electronic medical records.⁶³ Future trials will need to explore the cost-effectiveness of different implementation strategies once the optimal screening tools have been validated and accepted.

The criteria to define and distinguish mild cognitive impairment and early Alzheimer’s are being refined.^1,5 Because many experts advocate for a metric of cognitive decline among the criteria for dementia,⁵ the cAD8 may remain beneficial. The cAD8 is the only instrument to assess the effect of cognitive decline on activities of daily living, and in reference to the patient’s baseline cognitive capabilities. The cAD8 is also the only instrument that does not rely upon the patient’s responsiveness. The validation of the cAD8 identified a sensitivity of 92%, approximating the 83% sensitivity identified in our trials.^9,45 However, depending on caregivers during hectic ED encounters may prove unreliable.⁶⁴ This limitation would likely extend to other informant-based assessments such as the Informant Questionnaire on Cognitive Decline in the Elderly, or the General Practitioner Assessment of Cognition, both of which are longer and more difficult to score than the cAD8.^65,66 In addition, no caregiver was willing or able to provide cAD8 responses for 44% of our subjects. Because the cAD8 was designed to detect cognitive dysfunction due to dementia, it may be inappropriate in the acute setting where delirium or other confusional states may not be accurately captured by the domains of the screening instrument.⁶⁷ In the ED setting, performance-based tests, like the other instruments being evaluated, may be more appropriate to define the presence of impairment and perhaps to monitor symptom resolution via serial score improvement, although we did not test this hypothesis in the current diagnostic accuracy study. On the other hand, the cAD8 may be helpful in detecting the presence of an underlying dementia following resolution of the acute event.

Opponents of geriatric syndrome screening in particular, or ED-based preventive medicine in general, will argue that case-finding is not part of the emergency medicine mission. However, emergency care of aging adults can be a sentinel event, providing the opportunity to address the context of an illness prior to discharge or admission in order to improve or stabilize functional capacity and quality of life, all while containing health care costs by decreasing repeat visits and preventable admissions.⁶⁸ Despite the unfortunate reality that primary care is often difficult for frail older adults to access, and often fails to detect cognitive dysfunction anyway,^18,19 emergency medicine stands to directly benefit from a point-of-care recognition of impaired mental status. Older adults consume more time and personnel resources than younger patients, yet often depart the ED confused and dissatisfied with the service rendered.^69–72 Currently available pharmacologic and caregiver support programs can cost-effectively delay dementia symptom progression to maintain non-institutionalized residence.^73,74 Similarly, inpatient care models have cost-effectively reduced the severity and duration of delirium in some patient subsets.^75,76 Prompt recognition of cognitive dysfunction can permit busy emergency physicians to ensure appropriate admission planning or discharge comprehension by patients and caregivers.⁷⁷ In addition, multiple community resources exist to which emergency physicians could establish patient and family awareness.^78,79 Some interventional models have suggested that educating patients about community-based alternatives to the ED can reduce return visits by 30%.^80–84

Cognitive dysfunction in geriatric ED patients is also a risk factor for accelerated functional decline, standing level falls,⁸⁵ driver safety, diminished patient satisfaction scores,⁸⁶ and lower caregiver quality of life,⁸⁷ so its recognition could launch a multitude of ED-based quality improvement initiatives. Geriatric syndrome screening opponents will recognize the broad-based support for cognitive assessment documentation and resulting management alterations being promulgated for resident education and clinician quality indicators.^26,27 The results of this study will provide validated instruments to empower resident educators and policy makers to fulfill these recommendations.

LIMITATIONS

First and foremost, the MMSE is a suboptimal criterion standard for the identification of mild cognitive impairment.⁸⁸ Some have estimated the sensitivity of the MMSE for mild cognitive impairment as low as 18%.⁸⁹ Although one recent systematic review identified the MMSE median positive likelihood ratio to be 9.5 and the median negative likelihood ratio to be 0.18 for identifying moderate to severe dementia, the MMSE has not been recommended for the diagnosis of mild cognitive impairment.³⁴ In addition, the MMSE may demonstrate increased false-positive rates for poorly educated and lower socio-economic groups,^88,90,91 and increased false-negative rates for highly educated populations.⁸⁸ Our results indicate that the SBT, BAS, or O3DY provide 95% sensitivity for MMSE scores ≤ 23, although this criterion standard itself may misidentify substantial numbers of patients with cognitive dysfunction, particularly those with mild cognitive impairment. Furthermore, the MMSE does not distinguish delirium from dementia, but our study did differentiate the two forms of cognitive dysfunction without demonstrating any effect on diagnostic test performance for any of the instruments when the delirium subjects were excluded. However, the CAM-ICU is not validated for ED use, although it has been widely adopted as a delirium criterion standard.^8,92 Similarly, the MMSE has longstanding acceptance as a criterion standard for clinical emergency medicine,^29,93 and has been employed in multiple prior research settings.^9,31,32 The BAS, SBT, or O3DY can be administered 10–12 minutes faster than the MMSE, and do not require the patient to read or write. Future ED trials of these instruments will need to include validated metrics of socio-economic status,⁹⁴ health literacy,⁹⁵ mild cognitive impairment,⁸⁹ and a larger sampling of delirium patients in order to fully evaluate their diagnostic properties when stratified by these confounding variables.

The second limitation was the single-center academic hospital setting in which we excluded a substantial proportion of subjects who had received potentially sedating medications before enrollment, in addition to critically ill, non-English speaking, and non-consentable subjects. However, non-enrolled subjects did not differ significantly from enrolled subjects by age or sex. We also had a non-consecutive sampling of patients. Our exclusion of potentially sedating medications is the most likely reason why our incidence of delirium differs from previous ED research reports, which approach 10%.^8,28 Our results may not accurately reflect the diagnostic performance of the BAS, SBT, O3DY, or cAD8 in different populations.

Third, our methods did not randomize the administration of the criterion standard MMSE and the new diagnostic tests. In other words, we did not administer the MMSE before the BAS, SBT, and cAD8 in half the subjects and after the BAS, SBT, and cAD8 in the other half. This may be important because patient’s cognitive test performance could differ depending upon when they encounter the criterion standard relative to the new test. We cannot ascertain any learning phenomenon or test-retest effect that might increase false-positive or false-negative results.^53,54 Alternatively, administration of multiple instruments consecutively to already ill and emotionally stressed geriatric adults could theoretically decrease test performance on the latter tests secondary to patient fatigue, clinical distracters, or the generally disruptive ED environment. However, the specific questions composing each screening tool are unique other than person/place/time orientation, and we feel that the possibility of a learning phenomenon is likely of inconsequential effect. Nonetheless, future trials should re-assess diagnostic test performance via a random administration of the criterion standard and new tests.⁵⁶

Fourth, we did not assess either reliability of any instruments or de novo administration of the O3DY. By reconstructing the O3DY from elements of the criterion standard MMSE, our results may suffer incorporation bias, which can falsely increase diagnostic test performance.^96,97 Prior research has demonstrated that ED-based cognitive function screening is reproducible.⁹⁸ In addition, we are conducting short-term home follow-up assessments of community-dwelling geriatric adults to measure test-retest performance of cognitive screening instruments in our population.

Fifth, multiple other screening tools have been described that might merit further testing, including the clock-drawing test,⁹⁹ St. Louis University Mental Status Examination,¹⁰⁰ the Memory Impairment Screen,¹⁰¹ Hopkins Verbal Learning Test,¹⁰² and the General Practitioner Assessment of Cognition.⁶⁵ Future trials could assess the performance of these tests as either brief screening tools or as alternative criterion standards, although we believe that we have selected the instruments most appropriate and available for ED application and compared their performances with the most accepted criterion standard.

Finally, our research did not assess any patient-important outcomes. Although increased ED clinician and caregiver recognition of potential cognitive dysfunction offers the opportunity to initiate early interventions to prolong functional independence and quality of life, we did not assess for any such effect.³³ For clinicians, awareness of cognitive dysfunction might facilitate disposition decisions,¹⁰³ ensure clarity of discharge instructions,⁷⁷ or expedite referral to outpatient resources for more definitive cognitive testing and appropriate interventions.¹⁰⁴ Future trials of ED-based cognitive dysfunction case-finding will need to assess the effect of such screening on patient-oriented outcomes like preventable recidivism, functional decline, and quality of life for patients and caregivers.

CONCLUSIONS

Cognitive dysfunction remains prevalent in geriatric ED patients. Brief, sufficiently sensitive screening instruments to rapidly identify patients at lower risk for cognitive dysfunction have now been described. Among those instruments, the Short Blessed Test offers the best negative likelihood ratio.