A Quick and Easy Delirium Assessment for Non-Physician Research Personnel

Abstract

Objectives

Delirium in the emergency department (ED) is an emerging field of research. Most ED research infrastructures utilize lay personnel to collect data, but delirium assessments that can be reliably performed by non-physicians are lacking. We evaluated the diagnostic performance of the modified Brief Confusion Assessment Method (modified bCAM) for this purpose.

Methods

This was a secondary analysis of a prospective observational study than enrolled ED patients 65 years or older. The original bCAM was a brief (< 2 minutes) delirium assessment that assessed for inattention by asking the patient to recite the months backwards from December to July. It was modified by adding the Vigilance A (“squeeze my hand when you hear the letter ‘A’”) to the inattention assessment. The elements of the modified bCAM were performed by a research assistant (RA) and emergency physician. The reference standard for delirium was a psychiatrist assessment performed within 3 hours using Diagnostic and Statistical Manual of Mental Disorder, Fourth Edition, Text Revision (DSM-IV-TR) criteria. All assessors were blinded to each other. Sensitivities and specificities with their 95% confidence intervals (95%CI) were calculated for the RA and emergency physician.

Results

Of the 406 patients enrolled, 50 (12%) were delirious. The modified bCAM was 82.0% (95%CI: 71.4% – 92.6%) sensitive and 96.1% (95%CI: 94.0% – 98.1%) specific when performed by the RA. The emergency physician’s modified bCAM exhibited similar diagnostic performance.

Conclusions

The modified bCAM may be a feasible, accurate, and reliable method for non-physicians to assess for delirium. Future studies are needed to confirm these findings.

INTRODUCTION

Delirium is an acute loss of cognition characterized by the presence inattention, altered level of consciousness, and disorganized thinking.¹ This form of acute brain failure occurs in 8% to 10% of older emergency department (ED) patients, affecting approximately 1.9 million older ED patients each year in the US alone.^2–5 This number exceeds the annual number of patients with acute coronary syndromes in the US in all ages, which is a syndrome with comparable mortality and morbidity.^6,7 In the hospital setting, the deleterious consequences of delirium are well documented. It is associated with higher long-term mortality,^8–10 accelerated functional and cognitive decline,^10–15 prolonged hospital length of stays,¹⁶ and increased health care costs.¹⁷

However, the epidemiology of delirium in the ED is unclear. Compared with the hundreds of in-hospital studies published over the past two decades, the number of studies conducted in the ED is sparse. Hospital based studies have limited generalizability to the ED patient population since it excluded those who were discharged from the ED. Additionally, many of these studies enrolled patients 24 to 48 hours after admission and the patient’s delirium status at the time of enrollment may not have reflected his or her delirium status in the ED. Because the ED is the nexus of the health system and the gateway for a significant proportion of hospital admissions, improving our understanding of delirium in the ED is crucial to improving the quality of care delivered to the geriatric patient.¹⁸ As a result, there has been an increased push to perform more delirium investigations in the ED by funding agencies.^19–21

One significant barrier to conducting such investigations is lack of feasible delirium assessments available for the dynamic ED environment. Many delirium assessments can take up to 30 minutes to complete.^22,23 Prolonged assessments are not feasible to perform in the ED, because large number of eligible patients can present at one time and interruptions frequently occur. Additionally, many ED research infrastructures utilize non-physicians for data collection. Because most delirium assessments are subjective, their diagnostic accuracy may be reduced in non-physicians.^24,25 Hence, brief (< 2 minutes) delirium assessments that maximize efficiency, minimize burden to the patient, and can be reliably performed by non-physicians are needed. The Brief Confusion Assessment Method (bCAM) is a delirium assessment that possesses these desirable characteristics and takes less than two minutes to perform.²⁶ Though highly specific in older ED patients, its sensitivity was 78% when performed by research assistants. We sought to modify the bCAM and to improve its sensitivity for research assistants while minimally affecting specificity.²⁶ The objective of this study was to evaluate the diagnostic accuracy and reliability of the modified bCAM.

METHODS

Study Design and Setting

This was a secondary analysis of prospective observational study that investigated the validity of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU), Brief Confusion Assessment Method (bCAM), and the Delirium Triage Screen (DTS) in older ED patients.^26,27 The results of this particular analysis has not been previously published. This study was conducted at a tertiary care, academic ED with an annual census of approximately 57,000 visits.

Study Population

A convenience sample of patients was enrolled from July 2009 to February 2012, Monday through Friday between 8AM and 4PM. The enrollment window was based upon the delirium reference rater’s (consultation-liaison psychiatrists) availability. Because of the extensiveness of their evaluations, patient enrollment was limited to one per day. The local institutional review board (IRB) reviewed and approved this study. Informed consent was obtained from the patient or an authorized surrogate whenever possible. The IRB granted a waiver of consent for patients who were both unable provide consent and without an authorized surrogate available in the ED or by phone. This waiver was provided because they determined this study to have minimal risk to the patient and that the potential benefits of participating in our study far outweighed the risks.

Patients were included if they were 65 years and older, were in the ED for less than 12 hours at the time of enrollment, and were not in a hallway bed. The 12-hour cutoff was arbitrarily set to help include patients who presented in the evening and early morning hours, yet minimized extraneous factors which would artificially cause new-onset delirium from prolonged exposure to known delirium precipitants (e.g. opioid or benzodiazepines). Patients in the hallway beds were not included because there was a high level of ambient noise. Performing a lengthy psychiatric evaluation in the hallway would have been difficult. Patients were excluded if they refused consent, were non-English speaking, were previously enrolled, were deaf or blind, were comatose, were non-verbal or were unable to follow simple commands prior to their acute illness, or did not complete all the study assessments. Comatose patients were excluded because a patient must be arousable to verbal stimuli in order to assess for delirium.²⁸ Patients who were unable to follow simple commands or were non-verbal prior to their acute illness were considered to have end-stage dementia, and this was determined by surrogate interview or medical record review. They were excluded because diagnosing delirium in patients with end-stage dementia can be challenging even for a psychiatrist. Patients who did not complete the study were those who refused the psychiatrist assessment or were discharged from the ED prior to the data collection being completed.

Measurements

The bCAM was originally adapted from the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU),²⁹ both of which were based on the Confusion Assessment Method (CAM) algorithm developed by Inouye et al.¹ The bCAM consisted of four features: 1) altered mental status or fluctuating course, 2) inattention, 3) altered level of consciousness, and 4) disorganized thinking. A patient was considered to be bCAM positive if altered mental status or fluctuating course (feature 1) and inattention (feature 2), and either altered level of consciousness (feature 3) or disorganized thinking (feature 4) were present. Altered mental status and altered level of consciousness represent two different, yet inter-related constructs. Altered mental status refers to an acute change in the patient’s global mentation, while altered level of consciousness reflects a specific change in the patient’s level of arousal or wakefulness.

We modified the bCAM to further improve its sensitivity, while maintaining its specificity prior to the analysis using expert opinion. The original bCAM assessed feature 2 (inattention) by asking the patient to recite the months backwards from December to July; the task was stopped if the patient paused at specific month for 15 seconds. The modification added the Vigilance “A” test from the CAM-ICU; for this task, the patient had to squeeze the rater’s hand every time he or she heard the letter “A”.³⁰ A series of 10 letters (“SAVEAHAART”) were given every 3 seconds. If the patient made >1 error on either the month backwards or Vigilance A tasks, then the patient was considered to be inattentive or feature 2 positive. This cut-off was also set a priori using expert opinion. If they were unable or refuse to perform these tasks, then they were considered to be positive for inattention.

Like the bCAM, feature 1 was considered positive if the patient had altered mental status or a fluctuating course. This feature was primarily determined by interviewing a family member or caregiver in the ED or by phone. If the patient was from a skilled nursing facility, then the skilled nursing facility was contacted. If the patient lived alone at home and no collateral history was available, then the medical record was reviewed to help determine the patient’s mental status baseline. If there was no information about the patient’s baseline mental status and the patient was feature 2 (inattention) positive and either feature 3 (altered level of consciousness) or 4 (disorganized thinking) positive, then it was assumed that the patient was feature 1 positive. Altered level of consciousness (feature 3) was determined by the Richmond Agitation Sedation Scale (RASS) which is an arousal scale that ranges from −5 (comatose) to +4 (combative).³¹ If the patient had a RASS other than 0 (alert, normal level of consciousness), then the patient was considered to have altered level of consciousness. Disorganized thinking (Feature 4) was assessed for by asking the patient four yes/no questions and a simple command. If the patient made any error in the disorganized thinking assessment, then he/she was considered to be positive for this feature.

The reference standard for delirium was a consultation-liaison psychiatrist’s assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition criteria, Text Revision (DSM-IV-TR).³² Details of their evaluation have been previous reported.^26,27 In summary, these psychiatrists had an average of 11 years of clinical experience and diagnosed delirium routinely as part of their clinical practice. They used all means of patient evaluation and testing, as well as data gathered from those who best understood the patient’s current mental status (e.g., the patient’s surrogates, physicians, and nurses). As part of their routine evaluation, an extensive battery of cognitive tests and a focused neurological examination were performed at the patient’s bedside.

Data Collection

The research assistants and physician performed the components of the modified bCAM after the patient was enrolled. Research assistants had a wide range of clinical backgrounds and consisted of college graduates, emergency medical technicians, and paramedics. Prior to study onset, the research assistants received training that included didactic lectures, studying training manuals, and practicing these delirium assessments on simulated and actual patients. The physician was a board-certified emergency physician who had prior experience in performing the CAM-ICU, but received no formal training in assessing the cognition or mental status of elders.

To assess reliability, one person performed the assessment while the other person observed the patient. This method of inter-observer reliability testing was used rather than two separate evaluations (test-retest) to avoid learning on the part of the patient. This also avoids other inherent problems with test-retest reliability comparisons, such as a change in patient mental status if significant time occurred between ratings.³³ A consult-liaison psychiatrist then performed the reference assessment within 3-hours of the modified bCAM. All assessors were blinded to each other.

Medical Record Review

Using medical record review, the Charlson Comorbidity Index was calculated; this weighted index is a measure of comorbidity burden that takes into account the number and seriousness of preexisting comorbid conditions.³⁴ To quantify severity of illness, the Emergency Severity Index was recorded from the ED nurse’s triage note. The Acute Physiology Score (APS) portion of the Acute Physiology and Chronic Health Evaluation II was also calculated to quantify severity of illness.³⁵ The APS is a continuous variable based upon the initial values of 12 routine physiologic measurements; higher scores indicate higher severities of illness. Dementia was also collected from the electronic medical record.

Statistical Methods

Measures of central tendency and dispersion for continuous variables were reported as medians and interquartile ranges (IQR). Categorical variables were reported as absolute numbers and proportions. The modified bCAM’s sensitivities, specificities, positive likelihood ratios, and negative likelihood ratios with their 95% confidence intervals (95%CI) were calculated for both the research assistant and emergency physician using the psychiatrist’s DSM-IV-TR assessment as the reference standard. Kappa statistics were also calculated to measure inter-observer reliability between the research assistant and emergency physician. All statistical analyses were performed using open source R statistical software, version 2.15.1 (http://www.r-project.org/).

RESULTS

A total of 953 older ED patients were screened between July 2009 and February 2012 and of these, 406 were enrolled (Figure 1). The characteristics of those enrolled can be seen in Table 1. The median age was 73.5 years old (IQR: 69, 80), 202 (49.8%) were female, 57 (14.0%) were non-white race, and 50 (12.3%) patients were diagnosed with delirium by the psychiatrist. The median (IQR) time between the research assistant/physician and psychiatrist assessments was 70 minutes IQR (31, 120) minutes.

Figure 1.

Enrollment Flowsheet.

Table 1.

Patient Characteristics

Patient Characteristics	Enrolled patients (n = 406)
Median Age (IQR)	73.5 (69, 80)
Female Gender	202 (49.8%)
Non-white race	57 (14.0%)
Residence
Home	366 (90.1%)
Assisted Living	24 (5.9%)
Rehabilitation/Post-acute	5 (1.2%)
Nursing Home	11 (2.7%)
Education
Elementary or below	9 (2.2%)
Middle School	48 (11.8%)
High School	163 (40.2%)
College	118 (29.1%)
Graduate School	67 (16.5%)
Missing	1 (0.3%)
Dementia in Medical Record	24 (5.9%)
Median Charlson (IQR)	2 (1, 4)
Median APS (IQR)	2 (1, 4)
ED Chief Complaint
Abdominal pain	17 (4.2%)
Altered mental status	23 (6.2%)
Chest pain	67 (16.5%)
General weakness	40 (9.9%)
Shortness of breath	46 (11.3%)
Syncope	23 (5.7%)
Admitted to the hospital	294 (72.4%)

Patient characteristics and demographics of enrolled patients. IQR, interquartile range; APS, Acute Physiology Score, ED, emergency department.

The modified bCAM’s diagnostic performance in the research assistants and emergency physician with their interobserver reliabilities can be seen in Table 2. Using the psychiatrist’s DSM-IV-TR assessment as the reference standard, the modified bCAM’s sensitivities were 82.0% (95%CI: 71.4% – 92.6%) and 86.0% (95%CI: 76.4% – 95.6%) when performed by the research assistants and emergency physician, respectively. The modified bCAM’s specificity was 96.1% (95%CI 94.0% – 98.1%) for the research assistants and 94.7% (95%CI: 92.3% – 97.0%) for the emergency physician. A positive modified bCAM strongly increased the likelihood of delirium while a negative test moderately decreased the likelihood of delirium for both raters. The kappa between the research assistants and emergency physician was 0.87 indicating very good inter-observer reliability.

Table 2.

Diagnostic performance of the Modified Brief Confusion Assessment Method (bCAM)

Rater	Sensitivity (95%CI)	Specificity (95%CI)	PLR (95%CI)	NLR (95%CI)
RA	82.0% (71.4% – 92.6%)	96.1% (94.0% – 98.1%)	20.85 (12.28 – 35.41)	0.19 (0.10 – 0.34)
Physician	86.0% (76.4% – 95.6%)	94.7% (92.3% – 97.0%)	16.11 (10.26 – 25.31)	0.15 (0.07 – 0.29)
	Kappa = 0.87

The modified bCAM performed by the research assistants (RA) and emergency physician. Sensitivities, specificities, positive likelihood ratios (PLR) and negative likelihood ratios (NLR) are reported with their 95% confidence intervals (95%CI). The reference standard for delirium was a psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria.

DISCUSSION

The number of older ED patients affected with delirium exceeds the number patients with acute coronary syndromes in US EDs each year,^2–7 yet there is a dearth of delirium investigations in the ED. Delirium assessments that are specifically tailored for the ED are lacking and remain a significant barrier to conducting research in this challenging setting. We modified the bCAM and found that this modification may improve its sensitivity with a minimal decrease in specificity for delirium compared with a consultation-liaison psychiatrist’s reference assessment. The modified bCAM is brief enough where it can easily be performed in the dynamic clinical ED environment. Additionally, it can be reliably performed by non-physicians who are typically at the forefront of data collection in many ED research infrastructures.

Several other delirium assessments validated for the ED exists. The CAM-ICU has been validated in the ED and was found to be 68% sensitive and 99% specific when performed by an research assistant.²⁶ The bCAM is a modification of the CAM-ICU and the sensitivity improved to 78%, while maintaining a specificity of 97%.²⁶ The CAM-ICU and bCAM have adequate sensitivity for clinical purposes, but for research, its sensitivity may be inadequate especially when performed by non-physicians. Hence, bCAM was further modified, and these modifications increased the sensitivity to 82% when performed by the research assistant while being 96% specific for delirium.

The CAM is the most widely studied and used delirium assessment for non-psychiatrists,^1,22 and is the progenitor of the CAM-ICU and bCAM. Wong et al. performed a meta-analysis and found that the CAM’s pooled sensitivity was 86% and pooled specificity was 93%.²² However, the CAM may be difficult to perform in the ED for several reasons. It can take 5 to 10 minutes to complete,^23,36 and may be difficult to perform in the ED where a large number of eligible patients can be in the ED at once and interruptions are frequent. Because the CAM uses the rater’s subjective clinical judgment, it may also have decreased diagnostic accuracy when performed by non-physicians.^22,24,37,38 As a result, it may not be a practical solution for many ED-based research infrastructures that utilize nurses, college graduates, graduate students, paramedics, and emergency medical technicians for ED data collection. Lastly, to perform the CAM requires extensive training to perform accurately and reliably.^37,38 Providing training and reliability testing can be resource intensive and may not feasible if the number of raters that require training is large (i.e. multi-centered trials). Several brief delirium assessments have been recently developed and also have promise. The 4AT is a 2 minute delirium assessment that is 90% sensitive and 84% specific for delirium in older hospitalized patients.³⁹ The 3-minute diagnostic interview for delirium using the Confusion Assessment Method (3D-CAM) is 95% sensitive and 94% specific in older hospitalized patients.⁴⁰ However, their validity in older ED patients are unknown.

The modified bCAM may facilitate and advance our knowledge of delirium in the ED. First, the risk factors for delirium in the ED are unknown. One ED study identified dementia, hearing impairment, and functional impairment increased the risk of delirium, but they used the CAM-ICU as the reference standard.⁴ Second, the effect of delirium in the ED still remains unclear. Based upon a limited number of studies, delirium in the ED appears to be associated with higher mortality and prolonged hospitalizations,^41–44 but does not appear to be associated with accelerated functional decline.⁴⁵ Additional studies are needed to better elucidate these relationships. Third, the natural course of delirium in the ED is unknown. If vast majority of delirium in the ED may resolve rapidly (<24 hours), then identifying these patients in the ED may not be important. Conversely, identifying patients at high risk for a protracted delirioum may help identify patients who may benefit from early intervention. Lastly, the optimal management for delirious patients is still unknown. Antipsychotic medications have been traditionally used to manage delirium, but such practice is based upon studies that were limited by their non-blinded or non-randomized study design, small sample sizes, and the lack of a placebo group.⁴⁶ Several multi-component, non-pharmacologic delirium interventions have been developed for hospitalized patients,^47,48 but their efficacy is equivocal.⁴⁹ Many of these delirium intervention studies enrolled patients 24 to 48 hours after admission and intervening earlier in the ED may improve these interventions’ effectiveness.

LIMITATIONS

This study has several notable limitations. While the modification was made a priori and based upon expert opinion, this study was still a secondary analysis. Though we enrolled one of the largest delirium validation cohorts to be enrolled, the precision of the modified bCAM’s sensitivity estimate was relatively wide. We were also unable to ascertain how much the modified bCAM diagnostic accuracy improved over the bCAM with any statistical certainty. We also used DSM-IV-TR criteria for delirium as our reference standard. Recently, the DSM-5 was published and may have a more restrictive definition for delirium thus possibly altering the modified bCAM’s sensitivity and specificity. These findings will need to be confirmed prospectively using a larger cohort and DSM-5 criteria.⁵⁰

Because of the limited availability of the psychiatrists, we enrolled a convenience sample which introduced selection bias. Because delirium can fluctuate and the ED patient is frequently exposed to deliriogenic medications (i.e. opioid medications), time may have caused some discrepant observations between the research team and psychiatrist’s assessments. Patients who were initially non-delirious may have become delirious and patients who were initially delirious may have become non-delirious during the ED course. Lastly, this study was performed in a single ED located at an urban, tertiary care, academic hospital in patients who were 65 years and older; our findings may not be generalizable to other settings and especially in those who are less than 65 years of age.

CONCLUSIONS

The modified bCAM may be a useful method to assess for delirium in ED research studies. It has very good sensitivity and excellent specificity, and can be reliably performed by non-physicians and physicians alike. Additional studies with larger samples and conducted in other centers are needed to confirm these findings.