Diagnosing Delirium in Older Emergency Department Patients: Validity and Reliability of the Delirium Triage Screen and the Brief Confusion Assessment Method

Abstract

Background

Delirium is a common form of acute brain dysfunction with prognostic significance. Health care professionals caring for older emergency department (ED) patients miss delirium approximately 75% of cases. This error results from a lack of available measures that can be performed rapidly enough to be incorporated into clinical practice. Therefore, we developed and evaluated a novel two-step approach to delirium surveillance for the ED.

Methods

This prospective observational study was conducted at an academic ED in patients ≥ 65 years old. A research assistant (RA) and physician performed the Delirium Triage Screen (DTS), designed to be a highly sensitive rule-out test, and the Brief Confusion Assessment Method (bCAM), designed to be a highly specific rule-in test for delirium. The reference standard for delirium was a comprehensive psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria. All assessments were independently conducted within 3 hours of each other. Sensitivities, specificities, and likelihood ratios with their 95% confidence intervals (95%CI) were calculated.

Results

Of 406 enrolled patients, 50 (12.3%) had delirium diagnosed by the psychiatrist reference standard. The DTS was 98.0% (95%CI: 89.5% – 99.5%) sensitive with an expected specificity of approximately 55% for both raters. The DTS’ negative likelihood ratio was 0.04 (95%CI: 0.01 – 0.25) in both raters. As the complement, the bCAM had a specificity of 95.8% (95%CI: 93.2% – 97.4%) and 96.9% (95%CI: 94.6% – 98.3%) and a sensitivity of 84.0% (95%CI: 71.5% – 91.7%) and 78.0% (95%CI: 64.8% – 87.2%) when performed by the physician and RA, respectively. The positive likelihood ratios for the bCAM were 19.9 (95%CI: 12.0 – 33.2) and 25.2 (95%CI: 13.9 – 46.0), respectively. If the RA DTS was followed by the physician bCAM, the sensitivity of this combination was 84.0% (95%CI: 71.5% – 91.7%) and the specificity was 95.8% (95%CI: 93.2% – 97.4%). If the RA performed both the DTS and bCAM, this combination was 78.0% (95%CI: 64.8% – 87.2%) sensitive and 97.2% (95%CI: 94.9% – 98.5%) specific. If the physician performed both the DTS and bCAM, this combination was 82.0% (95%CI: 69.2% – 90.2%) sensitive and 95.8% (95CI: 93.2% – 97.4%) specific.

Conclusions

In older ED patients, this two-step approach (highly sensitive DTS followed by highly specific bCAM) may enable healthcare professionals, regardless of clinical background, to efficiently screen for delirium. Larger, multi-centered trials are needed to confirm these findings and to determine the impact of these assessments on delirium recognition in the ED.

Introduction

Background

Delirium is an acute disturbance in attention accompanied by a rapid change in cognition that affects 1.5 million older emergency department (ED) patients in the US annually.^1–3 This form of acute brain dysfunction often leads to devastating consequences such as death and accelerated cognitive decline.^4–6 Unfortunately, delirium is frequently missed by healthcare professionals in all clinical settings,^{2, 7} and this is especially the case in the ED where it is missed 57% to 83% of the time.^{2, 8–13}

Importance

The ED is the nexus of the healthcare system.¹⁴ Missing delirium in this setting has the potential to compromise patient safety and may have downstream implications for clinical care and patient health.¹⁵ Because delirious patients are less likely to provide an accurate reason of why they are in the ED, missing delirium may lead to inadequate diagnostic workups, inappropriate dispositions, and delays in the diagnosis of their underlying medical illness.^{16, 17} Up to 25% of delirious ED patients are discharged home^{2, 12, 18} and are less likely to comprehend their discharge instructions which may lead to decreased compliance.^{16, 19} Lastly, if admitted, delirium that is missed in the ED will also be missed in the inpatient setting in over 90% of cases.¹³

Delirium is often clinically silent and will remain unrecognized without a formal delirium assessment. Most healthcare professionals do not screen for delirium in their clinical practice,^{13, 20} because there is a dearth of brief and easy to use assessments. Many existing assessments take more than 5 minutes to complete and may not be feasible to perform in busy clinical environments.²¹ Because physicians often evaluate a large number of patients in a short period of time, non-physician hospital personnel (nurses, patient care technicians, etc.,) who have more exposure to patients may be better suited for delirium screening. Unfortunately, most delirium assessments are subjective and their diagnostic accuracy may be reduced in non-physicians.^{22, 23}

One method to improve delirium detection would be to use a two-step approach: a sequential testing strategy that utilizes brief, valid, and reliable delirium assessments that can be performed by healthcare professionals of all clinical backgrounds. The first step would be to perform a very brief (<20 seconds), highly sensitive delirium screen to rule-out delirium. This rapid rule-out screen can be incorporated into the ED triage assessment or can be part of the initial nursing assessment after the patient has been placed in an ED bed. A negative screen would rule out delirium, reduce the number of formal delirium assessments needed, and enhance screening efficiency. A positive screen would trigger a formal delirium assessment performed by another health care provider (i.e., physician) at the patient’s bedside that would be highly specific to rule-in delirium. Ideally, this rule-in assessment should be brief (<1 minute) to maximize feasibility. We developed Delirium Triage Screen (DTS) and the Brief Confusion Assessment Method (bCAM) to serve as the rule-out and rule-in tests for the two-step approach to delirium surveillance (Figure 1), respectively.

Figure 1. The 2-step approach to delirium surveillance for the emergency department: the Delirium Triage Screen (DTS) on the top and Brief Confusion Assessment Method (bCAM) on bottom.

Step 1 (DTS) can be integrated into the nurses’ triage assessment. If the DTS result is negative, then delirium is ruled out and no additional testing is needed. If the DTS result is positive, then a confirmatory delirium assessment such as the bCAM should be performed. Both assessments use the Richmond Agitation Sedation Scale (RASS), which assesses for arousal and ranges from −5 (coma) to +4 (combative);²⁵ a score of 0 indicates normal level of consciousness. The DTS is courtesy of Vanderbilt University, Nashville, TN. Copyright 2012. The bCAM is adapted from: Inouye SK, et al. Ann Intern Med. 1990; 113:941-948. Confusion Assessment Method. Copyright 2003, Hospital Elder Life Program, LLC.^28,41 Not to be reproduced without permission.

Goals of this Investigation

We sought to determine the diagnostic performances of these novel assessments in older ED patients using the psychiatrist’s assessment as the reference standard.

Methods

Study Design and Setting

This was a prospective observational study conducted at a tertiary care, academic ED. The local institutional review board (IRB) reviewed and approved this study. Informed consent from the patient or an authorized surrogate was obtained whenever possible. Because this was an observational study and posed minimal risk to the patient, the IRB granted a waiver of consent for patients who were both unable provide consent and without an authorized surrogate available in the ED or by phone.

Selection of Participants

A convenience sample was enrolled. The enrollment window was based upon the psychiatrist’s availability and occurred from July 2009 to February 2012, Monday through Friday between 8AM and 4PM. Because of the extensiveness of the psychiatric evaluations, enrollment was limited to one patient per day. Patients were included if they were 65 years or older, in the ED for less than 12 hours at the time of enrollment, and not in a hallway bed. The 12-hour cut-off was set to include patients who presented in the evening and early morning hours, while minimizing extraneous factors that might precipitate new-onset delirium. Patients were excluded if they were non-English speaking, previously enrolled, deaf or blind, comatose, non-verbal or unable to follow simple commands prior to their current illness, or left the ED prior to completing all the study assessments. Patients who were unable to follow simple commands prior to their acute illness (determined by surrogate interview or medical record review) were considered to have end-stage dementia and were excluded; diagnosing delirium in this patient population can be challenging, even for a psychiatrist. During the enrollment window, research assistants (RA) initially screened patients for the inclusion criteria using the ED Electronic Whiteboard which contained the current ED census, including the patient’s age, location, and length of stay.²⁴ The RA approached those who met inclusion criteria and determined if any exclusion criteria were present. If none of the exclusion criteria were met, then the patient was considered eligible for enrollment.

Methods of Measurement

The DTS (Figure 1) was designed to be very brief (<20 seconds) so that it could be easily integrated into the clinical workflow. The DTS consists consisted of two components: 1) Level of consciousness - measuring the Richmond Agitation Sedation Scale (RASS) and 2) Attention - spelling the word “LUNCH” backwards. The RASS is an arousal scale commonly used in the intensive care unit to assess level of sedation. It ranges from −5 (unarousable) to +4 (combative), and 0 indicates a normal level of alertness.²⁵ Spelling a word backwards is commonly used to assess for inattention.²⁶ If a patient had a RASS other than 0, or made >1 error on the “LUNCH” backwards spelling test, then the DTS was considered positive.

The bCAM (Figure 1) was a modification of the CAM-ICU,²⁷ both of which were progeny of the Confusion Assessment Method (CAM) developed by Inouye et al.²⁸ The CAM-ICU takes a median of 55 seconds to complete.²⁹ It is over 90% sensitive and specific in critically ill patients but it has not been validated in older ED patients.^{27, 30} Based upon our previous experience,¹³ the bCAM was developed to improve the CAM-ICU’s sensitivity and enhance its brevity. Similar to the CAM-ICU, the bCAM had four features: 1) altered mental status or fluctuating course, 2) inattention, 3) altered level of consciousness, and 4) disorganized thinking. A patient was considered to be delirious if both features 1 (altered mental status or fluctuating course) and feature 2 (inattention) were present, and either feature 3 (altered level of consciousness) or feature 4 (disorganized thinking) were present.

Feature 1 was the same as the CAM-ICU and was positive if the patient had altered mental status or a fluctuating course. This feature was primarily determined by surrogate interview in the ED or by phone. If the patient was from a nursing home, then the nursing home nurse was contacted. If the patient lived alone at home and no collateral history was available, then the medical record was reviewed to help determine the patient’s baseline mental status. If no information about the patient’s baseline mental status was available and the patient was feature 2 (inattention) positive and either feature 3 (altered level of consciousness) or 4 (disorganized thinking) positive, then it was assumed that the patient was feature 1 positive. Instead of the Vigilance A random letter and picture recognition tests used by the CAM-ICU,²⁷ the bCAM simply asked the patient to recite months backwards from December to July to assess for inattention (feature 2).^31–33 If the patient made >1 error, or was unable or refused to perform the task, then he/she was considered to be positive for inattention. We limited the recitation to 6 months so that brevity, reliability, and feasibility could be maximized. Because some patients would occasionally perseverate on specific months, the assessment was stopped after a 15 second break (i.e. pause or repetition of a month) in order to minimize the time spent on this task. Similar to the CAM-ICU, altered level of consciousness (feature 3) was determined by the RASS and disorganized thinking (feature 4) was determined by asking the patient four yes/no questions and to follow a simple command. If the RASS was other than 0 (alert, normal level of consciousness) then the patient was considered to have altered level of consciousness. If the patient made any errors (as opposed to > 1 error in the CAM-ICU) in the disorganized thinking assessment, then he/she was considered to be positive for feature 4.

A physician and a RA performed the DTS and bCAM concurrently. The physician was board-certified in emergency medicine and had experience in performing the CAM-ICU, but no other formal instruction on performing mental status assessments on elders. RAs had diverse backgrounds consisting of college graduates, basic emergency medical technicians, and paramedics. Prior to study initiation, all RAs participated in a 6 to 8 hour training session which included didactic lectures, and simulated and live patient encounters.

For the study, one rater performed the DTS and bCAM while the other observed, and both were blinded to each other’s assessments. We chose this method of reliability testing to avoid inherent problems with test-retest reliability comparisons such as learning on the part of the patient and a change in mental status between ratings.³⁴ The decision of who performed the study assessment and who observed was randomly selected using a random number generator (www.random.org) after informed consent was obtained.

The reference standard for delirium was a comprehensive consultation-liaison psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria.³⁵ These psychiatrists had an average of 11 years of clinical experience and diagnosed delirium as part of their routine clinical practice. To meet DSM-IV-TR criteria for delirium, a patient had to have all of the following: 1) a disturbance of consciousness (i.e., reduced clarity of awareness of the environment) with reduced ability to focus, sustain or shift attention, 2) a change in cognition or the development of a perceptual disturbance that is not better accounted for by a preexisting, established or evolving dementia, and 3) the disturbance develops over a short period of time (usually hours to days) and tends to fluctuate during the course of the day.

To arrive at the diagnosis of delirium, they interviewed those who best understood the patient’s mental status (e.g., the patient’s surrogates, physician, and nursing staff) and reviewed the patient’s medical record, laboratory and radiology results, and nursing assessments. They routinely performed comprehensive bedside cognitive testing that included (but was not limited to) Mini-Mental State Examination, Clock Drawing Test, Luria hand sequencing task, and tests for verbal fluency. A focused neurological examination (i.e., screening for paraphasic errors, tremors, tone, asterixis, frontal release signs etc.,) and evaluation for affective lability, hallucinations, and level of alertness were also conducted routinely. Confrontational naming, proverb interpretation or similarities, and assessments for apraxias were performed at the discretion of the reference psychiatrists, especially if the diagnosis of delirium was inconclusive. The psychiatrist’s assessment took approximately 30 minute to complete.

The RAs and physician typically performed the DTS and bCAM first. The psychiatrist’s DSM-IV-TR reference assessment was then performed within three hours of the DTS and bCAM. All assessors were blinded to each other’s assessments and were recorded onto paper-based case report forms, which were then transferred into a secure Research Electronic Data Capture (REDCap) database by the RA. All data entry was double checked for accuracy by the principal investigator.

Additional Variables Collected

Medical record review was performed to measure the Charlson Comorbidity Index which is a weighted index that takes into account the number and seriousness of 19 preexisting comorbid conditions; scores range from 0 (no comorbidity) to 37 (high comorbidity).^{36, 37} To quantify severity of illness, we obtained the Acute Physiology Score (APS) and the Emergency Severity Index (ESI). The APS is part of the Acute Physiology and Chronic Health Evaluation II, and is based upon the initial values of 12 routine physiologic measurements; higher scores represent higher severities of illness.³⁸ The ESI is 5-level triage algorithm that stratifies patients from level 1 (most urgent) to level 5 (least urgent) on the basis of acuity and resource needs.³⁹ To ascertain the patient’s premorbid cognition, the medical record was reviewed, looking for any documentation of dementia in the patient’s clinical problem list or physician history and physical examination from the outpatient and inpatient settings. A RA initially performed the medical record review and entered the data directly into a REDCap database. Validation rules were implemented to minimize data entry errors. The principal investigator then reviewed the medical record and double checked the database for accuracy.

Data Analyses

Measures of central tendency and dispersion for continuous variables were reported as medians and interquartile ranges. Categorical variables were reported as proportions. Sensitivities, specificities, and likelihood ratios with their 95% confidence intervals (95%CI) were calculated for both the physician and RA using the psychiatrist’s assessment as the reference standard.⁴⁰ Inter-observer reliability between the physician and RA was assessed by calculating the raw agreement and kappa statistic with their 95%CI. The combined diagnostic performance of the DTS and bCAM used sequentially was also calculated. We assumed that the DTS was performed by a research assistant. If the research assistant DTS was negative, we assumed that delirium was ruled out and no additional assessment was needed. We also assumed that a positive DTS would trigger a bCAM assessment performed by a physician. The combined diagnostic performances of the DTS and bCAM when both performed by the research assistant or both by the physician were also calculated.

To determine the usefulness of the DTS and bCAM in non-critically ill hospitalized patients, the aforementioned analyses were repeated in a subgroup of patients that were admitted to non-ICUs. This subgroup was taken from the existing cohort enrolled for the primary analysis; no additional patients were recruited. All statistical analyses were performed with open source R statistical software, version 2.15.1 (http://www.r-project.org/).

Results

A total of 953 patients were screened, 406 patients met enrollment criteria (Figure 2), and of those enrolled, 50 (12.3%) were diagnosed with delirium by the psychiatrist. Baseline characteristics can be seen in Table 1; 24 (5.9%) were from assisted living facilities, 11 (2.7%) were from nursing homes, and none were mechanically ventilated. During the study period, 22,168 potentially eligible ED patients who were 65 years or older presented to the ED. Both enrolled and potentially eligible patients were similar in age and gender (Table 1). However, enrolled patients were more likely to have an ESI of 2, be admitted to the hospital, and have chest pain.

Figure 2.

Enrollment flow diagram.

Table 1.

Patient characteristics and demographics

Patient Characteristics	Enrolled patients (n = 406)	All potentially eligible patients (N=22,168)
Median Age (IQR)	73.5 (69, 80)	74 (69, 81)
Female gender	202 (49.8%)	11,969 (54·0%)
Non-white race	57 (14·0%)	-
Education
Elementary or below	9 (2.2%)
Middle School	48 (11.8%)
High School	163 (40.2%)	-
College	118 (29.1%)
Graduate School	67 (16.5%)
Missing	1 (0.3%)
Dementia in Medical Record	24 (5.9%)	-
Median Charlson (IQR)	2 (1, 4)	-
Emergency Severity Index
1	1 (0.3%)	494 (2.2%)
2	264 (65.0%)	12,890 (58.2%)
3	135 (33.3%)	8,220 (37.1%)
4	5 (1.2%)	430 (1.9%)
5	0 (0.0%)	20 (0.1%)
Unknown	1 (0.3%)	114 (0.5%)
Median APS (IQR)	2 (1, 4)	-
ED Chief Complaint
Abdominal pain	17 (4.2%)	1,222 (5.5%)
Altered mental status	23 (6.2%)	1002 (4.5%)
Chest pain	67 (16.5%)	2,297 (10.4%)
Generalized weakness	40 (9.9%)	1,546 (7.0%)
Shortness of breath	46 (11.3%)	2,035 (9.2%)
Syncope	23 (5.7%)	608 (2.7%)
Admitted to the hospital	294 (72.4%)	13,533 (62.1%)
Admitted to Non-ICU	278 (94.6%)	-
Admitted to ICU	16 (5.4%)	-

Abbreviations: IQR, Interquartile range; APS, Acute Physiology Score; ED, Emergency Department; ICU, Intensive Care Unit.

The DTS’ diagnostic performance can be seen in the Table 2. The DTS had excellent sensitivity and moderate specificity when performed by the physician and RA. Both raters had a negative likelihood ratio of 0.04 (95%CI: 0.01 – 0.25), indicating that a negative DTS essentially ruled out delirium. The kappa was 0.79 (95%CI: 0.73 – 0.85) between the two raters indicating very good inter-observer reliability. The bCAM’s diagnostic performance in the physician and RAs with their inter-observer reliabilities can be seen in Table 2. The bCAM had very good sensitivity and excellent specificity for both the physician and RA. The bCAM’s positive likelihood ratios were 19.9 (95%CI: 12.0 – 33.2) and 25.2 (95%CI: 13.9 – 46.0) when performed by the physician and RAs, respectively. This indicated that a positive bCAM strongly increased the likelihood of delirium. The kappa was 0.88 (95%CI: 0.81 – 0.95) indicating excellent inter-observer reliability between the physician and RA rater.

Table 2.

Validity and reliability data of DTS and bCAM in older emergency department patients.

		TP	FN	TN	FP	Sensitivity (95%CI)	Specificity (95%CI)	PLR (95%CI)	NLR (95%CI)
DTS	Physician	49	1	195	161	98.0% (89.5% – 99.5%)	54.8% (49.6% – 59.9%)	2.17 (1.92 – 2.45)	0.04 (0.01 – 0.25)
	RA	49	1	200	156	98.0% (89.5% – 99.5%)	56.2% (51.0% – 61.3%)	2.24 (1.98 – 2.53)	0.04 (0.01 – 0.25)
	% Concordance between Physician and RA = 89.4% (95%CI: 86.0% – 92.0%) Kappa = 0.79 (95%CI: 0.73 – 0.85)
bCAM	Physician	42	8	341	15	84.0% (71.5% – 91.7%)	95.8% (93.2% – 97.4%)	19.94 (11.97 – 33.19)	0.17 (0.09 – 0.32)
	RA	39	11	345	11	78.0% (64.8% – 87.2%)	96.9% (94.6% – 98.3%)	25.24 (13.85 – 46.00)	0.23 (0.14 – 0.38)
	% Concordance between Physician and RA = 97.3% (95%CI: 95.2% – 98.5%) Kappa = 0.88 (95%CI: 0.81 – 0.95)
Combined	RA DTS + Physician bCAM	42	8	341	15	84.0% (71.5% – 91.7%)	95.8% (93.2% – 97.4%)	19.94 (11.97 – 33.19)	0.17 (0.09 – 0.32)
	RA DTS + RA bCAM	39	11	346	10	78.0% (64.8% – 87.2%)	97.2% (94.9% – 98.5%)	27.77 (14.81 – 52.06)	0.23 (0.13 – 0.38)
	Physician DTS + Physician bCAM	41	9	341	15	82.0% (69.2% – 90.2%)	95.8% (93.2% – 97.4%)	19.46 (11.66 – 32.48)	0.19 (0.10 – 0.34)

The Delirium Triage Screen (DTS) and Brief Confusion Assessment Method (bCAM) were performed by the physician and research assistants (RA) in older emergency department patients. Sensitivities, specificities, positive likelihood ratios (PLR) and negative likelihood ratios (NLR) are reported with their 95% confidence intervals (95%CI) for the DTS, bCAM and the combined approach. The reference standard for delirium was a psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision criteria.

Abbreviations: TP, true positive; FN, false negative; TN, true negative; FP, false positive.

The overall diagnostic performances of the two-step approach sequentially performed by various raters are also reported in Table 2. If the RA DTS was followed by the physician bCAM, the sensitivity of this combination was 84.0% (95%CI: 71.5% – 91.7%) and specificity was 95.8% (95%CI: 93.2% – 97.4%). If the RA performed both the DTS and bCAM, this combination was 78.0% (95%CI: 64.8% – 87.2%) sensitive and 97.2% (95%CI: 94.9% – 98.5%) specific. If the physician performed both the DTS and bCAM, this combination was 82.0% (95%CI: 69.2% – 90.2%) sensitive and 95.8% (95CI: 93.2% – 97.4%) specific.

With regard to the eight false negative physician bCAMs, seven could recite the months backwards from December to July perfectly. Three had very subtle symptoms and required a more comprehensive psychiatrist evaluation than would have been typically needed to detect their inattention. For one false negative, the surrogate stated the patient was at her mental status baseline and there was no evidence of a fluctuating course to the physician and RAs; this patient may have developed delirium during the ED course. Another false negative evaluation received morphine during the ED course, possibly confounding the assessments.

To determine how generalizable our findings were to non-critically ill hospitalized older patients, we analyzed a subgroup of 278 older ED patients who were admitted to non-ICUs; of these, 38 (13.7%) were delirious. The DTS and bCAM’s diagnostic performances and inter-observer reliabilities were similar as the main analysis (Table 3). The overall diagnostic performances of the two-step approach performed sequentially by various raters were also similar in this subgroup.

Table 3.

Validity and reliability data of DTS and bCAM in a subgroup of hospitalized patients.

		TP	FN	TN	FP	Sensitivity (95%CI)	Specificity (95%CI)	PLR (95%CI)	NLR (95%CI)
DTS	Physician	37	1	136	194	97.4% (86.5% – 99.5%)	56.7% (50.3% – 62.8%)	2.25 (1.93 – 2.62)	0·05 (0.01 – 0.32)
	RA	37	1	138	102	97.4% (86.5% – 99.5%)	57.5% (51.2% – 63.6%)	2.29 (1.96 – 2.68)	0·05 (0.01 – 0.32)
	% Concordance between Physician and RA = 88.5% (95%CI: 84.2% – 91.7%) Kappa = 0.77 (95%CI: 0.69 – 0.85)
bCAM	Physician	31	7	228	12	81.6% (66.6% – 90.8%)	95.0% (91.5% – 97.1%)	16.32 (9.21 – 28.90)	0·19 (0.10 – 0.38)
	RA	29	9	231	9	76.3% (60.8% – 87.0%)	96.3% (93.0% – 98·0%)	20.35 (10.47 – 39.57)	0·25 (0.14 – 0.44)
	% Concordance between Physician and RA = 96.8% (95%CI: 94.0% – 98.3%) Kappa = 0.87 (95%CI: 0. 79 – 0.95)
Combined	RA DTS + Physician bCAM	31	7	228	12	81.6% (66.6% – 90.8%)	95.0% (91.5% – 97.1%)	16.32 (9.21 – 28.90)	0.19 (0.10 – 0.38)
	RA DTS + RA bCAM	29	9	232	8	76.3% (60.8% – 87.0%)	96.7% (93.6% – 98.3%)	22.90 (11.32 – 46.29)	0.25 (0.14 – 0.43)
	Physician DTS + Physician bCAM	30	8	228	12	78.9% (63.7% – 88.9%)	95.0% (91.5% – 97.1%)	15.79 (8.88 – 28.07)	0.22 (0.12 – 0.41)

This was a subgroup analysis of 278 enrolled patients who were admitted to non-ICUs. The Delirium Triage Screen (DTS) and Brief Confusion Assessment Method (bCAM) were performed by the physician and research assistants (RA). Sensitivities, specificities, positive likelihood ratios (PLR) and negative likelihood ratios (NLR) are reported with their 95% confidence intervals (95%CI) for the DTS, bCAM and the combined approach. The reference standard for delirium was a psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision criteria.

Abbreviations: TP, true positive; FN, false negative; TN, true negative; FP, false positive.

Limitations

This study has several limitations. Because screening over 22,000 older ED patients over a 2.5 year period was not feasible, especially with the psychiatrists’ limited availability, we enrolled a convenience sample which may have introduced selection bias. Based upon the higher ESI and admission rates, the enrolled cohort may have had higher severities of illness. Though this may have introduced spectrum bias, the DTS and bCAM’s diagnostic performances did not appreciably change in the subgroup analysis of admitted patients (Table 3). Ideally, a subgroup analysis of discharged patients should have also been performed to fully ascertain the presence of spectrum bias. However, there were insufficient cases of delirium (n = 5) in this subgroup to perform a meaningful analysis. Because delirium can rapidly fluctuate and psychoactive medications (e.g. opioid medications) are frequently given in the ED, the allotted 3-hour time interval may have caused some discordant observations between the research team and psychiatrists’ assessments. However, the bias was most likely bidirectional as these discordant observations can lead to an underestimation or overestimation of the DTS and bCAM’s diagnostic accuracy. The reliability of the psychiatrist’s DSM-IV-TR assessment was not tested, because having a second psychiatrist perform a comprehensive evaluation would have placed undue burden on the patient. We used consultation-liaison psychiatrists who had a wealth of experience in diagnosing delirium to minimize misclassification. They also performed a rigorous and comprehensive evaluation of the patient, surrogate, and medical record to arrive at a diagnosis. This study was performed in a single ED located at an urban, academic hospital in patients who were 65 years and older. Our findings may not be generalizable in other settings. More importantly, the bCAM and DTS may have different diagnostic performances in patients less than 65 years of age. We chose to focus on older patients, because they are disproportionately affected by delirium. Future multi-centered investigations are needed to determine the DTS and bCAM’s diagnostic accuracies in other settings and across the entire age spectrum. Lastly, the DTS and bCAM were performed by research personnel. The diagnostic accuracies may be lower when used in real world settings and we were unable measure the impact of these assessments on delirium recognition. Future studies are needed to determine how non-research personnel can perform these delirium assessments and determine if their routine use improves delirium recognition.

Discussion

Delirium is missed at an alarmingly high rate because healthcare professionals do not screen for it.^{2, 8–13} This compromise in the quality and safety of care occurs¹⁵ because brief (<1 minute) and easy to use delirium assessments are not readily available. This investigation provides a novel and simple two-step approach to delirium surveillance that is reliable, valid, and could significantly improve patient care and health outcomes. The DTS (spell “LUNCH” backwards) and bCAM (recite the months of the year backwards), when used sequentially, showed a very good sensitivity and excellent specificity. The DTS was highly sensitive with a negative likelihood ratio of 0.04; a negative DTS essentially ruled-out delirium. The bCAM was highly specific and those with a positive bCAM were 20 to 25 times more likely to have delirium than those with a negative bCAM. Furthermore, both these assessments can be reliably performed by healthcare professionals regardless of clinical background and experience. Though formal validation studies are needed, our subgroup analysis suggests that the DTS and bCAM may also have utility in the in-hospital setting where delirium is similarly missed; their diagnostic performances were similar in a subgroup of hospitalized non-ICU patients.

The DTS is analogous to D-dimer (highly sensitive) which was intended to reduce computed tomography pulmonary angiography (highly specific) use for pulmonary embolism diagnosis. A negative DTS can potentially reduce the number of formal delirium assessments needed by 50%, while a positive DTS would require a more specific rule-in test. We developed the bCAM to be a brief (<1 minute) rule-in test. If performed in an algorithmic fashion (i.e. stopping if the patient can recite the months backwards with ≤ 1 errors), then the bCAM would take even less time which may be appealing for many clinical environments. The bCAM, however, missed subtler forms of inattention, and these patients required a more extensive psychiatrist evaluation than usually needed. The CAM, which has revolutionized delirium diagnosis for non-psychiatrists, can also potentially serve as the rule-in test for delirium. A recent meta-analysis reported it to be 86% sensitive and 93% specific.²¹ However, the CAM may take over 5 minutes to complete,^{41, 42} require significant training, and may have decreased diagnostic accuracy when used by non-physicians.^{22, 23}

In addition to the dramatic effect that these instruments could have on the routine clinical practice, the availability of feasible assessments will open a new frontier to additional delirium investigations. As with ST-segment elevation myocardial infarction and severe sepsis care, we envision establishing similar paradigms for delirium care in which early identification and intervention in the ED of delirious patients at higher risk is emphasized.^{43, 44} However, a multi-faceted investigative approach must be taken to achieve this goal. We must determine if our two-step approach captures patients at highest risk for adverse outcomes and those who would benefit most from intervention. Several delirium multi-component interventions exist for hospitalized patients, but their efficacy has been equivocal.⁴⁵ Many of these studies, however, enrolled delirious patients across the entire spectrum of severity 24 to 48 hours after admission. Conceivably, performing these interventions in high-risk delirious patients early in the ED course may cost-effectively improve short- and long-term outcomes. This hypothesis should be tested in future investigations using randomized-control trial methodology.

Our study has several strengths. The DTS and bCAM achieved a NLR of 0.04 and PLR of 20, respectively. These assessments strongly influence clinical decision making.⁴⁶ The two-step approach to delirium surveillance also serves as a conceptual model and can be tailored to each institution; one or both components can be used. In addition, each component can reliably performed by healthcare professionals regardless of clinical background and at different times during the ED course, further adding to this approach’s flexibility. These instruments test for inattention which is considered the cardinal feature of delirium and is core to the definition of delirium in the upcoming DSM revision (5^th edition).¹ Consequently, the DTS and bCAM will remain valid with the updated DSM-V definition of delirium. We also enrolled 406 patients which is one of the larger delirium assessment cohorts to be examined, especially in the ED setting.²¹

In conclusion, this study provides a novel two-step approach to delirium surveillance that is brief and may enable healthcare professionals of all backgrounds to screen for this under-recognized form of acute brain dysfunction. A negative DTS essentially rules-out delirium and reduces the number of formal delirium assessments needed, increasing screening efficiency. The bCAM balances brevity with diagnostic accuracy, and is an effective rule-in delirium assessment. These assessments have the potential to ameliorate a patient safety issue and improve delirium recognition in the ED and potentially other clinical environments. Larger, multi-centered trials are needed to confirm these findings and to determine the impact of these assessments on delirium recognition in the ED.