A Tale of Two Methods: Chart and Interview Methods for Identifying Delirium

Abstract

Background

Interview and chart-based methods for identifying delirium have been validated. However, relative strengths and limitations of each method have not been described, nor has a combined approach (using both interviews and chart), been systematically examined.

Objectives

To compare chart and interview-based methods for identification of delirium.

Design, Setting and Participants

Participants were 300 patients aged 70+ undergoing major elective surgery (majority were orthopedic surgery) interviewed daily during hospitalization for delirium using the Confusion Assessment Method (CAM; interview-based method) and whose medical charts were reviewed for delirium using a validated chart-review method (chart-based method). We examined rate of agreement on the two methods and patient characteristics of those identified using each approach. Predictive validity for clinical outcomes (length of stay, postoperative complications, discharge disposition) was compared. In the absence of a gold-standard, predictive value could not be calculated.

Results

The cumulative incidence of delirium was 23% (n= 68) by the interview-based method, 12% (n=35) by the chart-based method and 27% (n=82) by the combined approach. Overall agreement was 80%; kappa was 0.30. The methods differed in detection of psychomotor features and time of onset. The chart-based method missed delirium in CAM-identified patients laacking features of psychomotor agitation or inappropriate behavior. The CAM-based method missed chart-identified cases occurring during the night shift. The combined method had high predictive validity for all clinical outcomes.

Conclusions

Interview and chart-based methods have specific strengths for identification of delirium. A combined approach captures the largest number and the broadest range of delirium cases.

INTRODUCTION

Delirium, defined as an acute decline in attention and cognition, is common among hospitalized older patients, occurring in 6 to 56% of general medical populations and 15 to 53% of patients postoperatively.¹ Research studies often use interview-based methods such as the Confusion Assessment Method (CAM).² Interview methods have high sensitivity and specificity for identification of delirium compared to ratings by psychiatrists specializing in geriatric psychiatry. However, fluctuation is a defining characteristic of delirium and chart-based methods may capture delirium that was not present during the limited duration of bedside interviews but that developed at a different time of day (e.g., during the night shift).

A chart-based method for identifying delirium was validated in 2005, with overall agreement with the CAM of 82% and kappa = 0.41,³ and has been used subsequently in many studies.^4-7 However, the profile of patients identified using chart and interview-based methods have not been systematically compared. We hypothesize that the complementary strengths of the two methods suggest that a combined approach that defines delirium based on evidence from either the chart or interview-based methods may provide an important additional approach for detection of delirium.

The aim of this paper is to compare a chart-based method with an interview-based method using the CAM for identification of delirium, considering both overall delirium diagnosis, detection of specific features and timing of onset. In addition we examine the associations of delirium, identified using chart, interview and combined (which defines delirium based on evidence from either the chart or interview-based methods) methods, with key hospital outcomes, including post-operative complications, length of stay and discharge disposition.

METHODS

Setting and Patients

This study was conducted in the first 300 patients enrolled in a prospective observational study of older patients scheduled for major scheduled surgery, the Successful Aging after Elective Surgery (SAGES) study, described in detail previously.⁸ Written informed consent was obtained from the patient according to procedures approved by the institutional review board of all participating institutions and participants received a small stipend for participation.

Chart-Based Delirium Instrument

The chart-based delirium instrument was developed to maximize sensitivity for identification of delirium,³ was not based on ICD-9 codes or discharge diagnoses and included information on acute changes in mental status, time and duration of such episodes, evidence of agitation and reversibility or improvement of the acute confusion. Chart abstractors were nurses or physicians with training in delirium who worked independently of the hospital interviewers, and who were blinded to the results of the CAM delirium ratings. Chart abstractors obtained a baseline mental status from pre-operative visits (including pre-operative anesthesia evaluation and pre-surgical notes), previous discharge summaries and outpatient visit notes.

Change in mental status during hospitalization was obtained through review of the entire chart and focusing on admission and daily nursing notes, progress notes (nurse or physician), notes from a specialty consult and the discharge summary. Abstractors were provided with ‘trigger words’ or phrases that may be used to indicate delirium, such as mental status change’, ‘disoriented/re-oriented’, ‘unresponsive’, ‘agitated’, and that prompted the rater to look for details of episodes that might indicate delirium. All chart-based cases of delirium were adjudicated by a geriatrician (SKI) and a neuropsychologist (BW), both with extensive training in delirium assessment. Any discrepancies were resolved during a consensus meeting.

Interview-Based Delirium Assessment

Trained interviewers assessed patients daily during hospitalization for the development of delirium using the CAM, a widely used, standardized method for identification of delirium that has high sensitivity, specificity and inter-rater reliability.^{2, 9} The CAM was rated based on information from patient interviews including a brief cognitive screen described previously,⁸ the Delirium Symptom Interview (DSI)¹⁰ and information from nurses or family members. The CAM diagnostic algorithm requires the presence of acute change or fluctuating course, inattention, and either disorganized thinking or an altered level of consciousness to fulfill criteria for delirium.

Definitions of Study Variables

From the baseline assessment, vision impairment was defined as corrected binocular near vision worse than 20/70 on the bedside Jaeger vision test¹¹ and hearing impairment was defined as hearing correctly six or fewer of 12 numbers out of both ears on the Whisper test.¹² Global cognitive function was assessed by the General Cognitive Performance (GCP) composite, which was created for the SAGES study and has been validated and shown to be sensitive to cognitive decline^{8, 13} and by the Modified Mini-Mental State Examination (3MS).¹⁴ Race and ethnicity were self-reported by the patient. A baseline interview was conducted with a patient’s family member to obtain informant ratings of mild cognitive impairment and dementia based on the Informant Questionnaire on Cognitive Decline in the Elderly (IQ-CODE) short-form.¹⁵ Charlson Comorbidity Index, the Acute Physiology and Chronic Health Evaluation II (APACHE II),^{16, 17} post-operative length of stay (in days), major post-operative complications and discharge disposition were abstracted from the medical record.

Statistical Analysis

Since we regarded the CAM and chart methods as providing distinct yet equally important information about a delirium episode, we did not choose either one to serve as the reference standard for this study. Rather, our goal was to report the agreement/disagreement between the chart and interview-based methods and to describe the characteristics of participants identified by one method as compared with the other.

We compared the predictive validity of the three approaches to identification of delirium (identification of delirium based on: 1. CAM only; 2. Chart only; and 3. CAM or Chart identification) and their association with post-operative LOS, number of post-operative complications and discharge disposition (discharge to a post-acute care facility vs. other location) using Poisson and logistic regression adjusted for age, sex, race, surgery type, comorbidity burden, severity of illness (APACHE-II) and IQCODE score.

The population attributable risk (PAR), measuring the proportion of each clinical outcome that could be attributed to delirium, was calculated as the product of a function of the relative risk (RR) of the outcome among patients with delirium ([RR – 1}/RR) and the prevalence of delirium. The PAR provides the strongest indication of the clinical impact of delirium in the study population, since it incorporates both prevalence and relative risk. All analyses were performed using Stata Version 11.2 (Statacorp, College Station, TX).

RESULTS

The sample consisted of 300 hospitalized patients with a mean age of 77 years; sample characteristics are reported in Table 1.

Table 1.

Sample Characteristics (N=300)

Characteristic
Age (at Surgery) [Mean (SD)]	76.9 (5.0)
Age 80+, n(%)	81 (27)
Female, n(%)	166 (55)
Non-white/Hispanic, n(%)	21 (7)
English as Second Language (ESL), n(%)	20 (7)
Married/lives with partner, n(%)	185 (62)
Education, n(%)
0-12 Years	86 (29)
13-16 Years	129 (43)
17+ Years	85 (28)
Surgical Type, n(%)
Orthopedic	253 (84)
Vascular	16 (5)
Gastrointestinal	31 (11)

Incidence of Delirium

The cumulative incidence of delirium was 22.7% (n= 68) according to CAM ratings by trained interviewers and 11.7% (n=35) according to ratings from the chart-based method.(Table 2) Of the 68 participants rated as delirious by the CAM, 21 were also identified as delirious using the chart-based method. Among the 232 participants rated as non-delirious using the CAM, 218 were also identified as non-delirious by the chart, yielding an overall agreement of 80% between the CAM and chart-based methods and kappa of 0.30.

Table 2.

Correlates of Delirium Identified by Chart or CAM (N=300)

		Chart Based			CAM Based
Variable	Total N=300	Delirium n=35	No Delirium	P	Delirium n=68	No Delirium	P
Patient Characteristics
Age 80+, n(%)	81 (27)	9 (26)	72 (27)	1.00	18 (26)	63 (27)	1.00
MCI or dementia by proxy IQCODE (%)	62 (21)	10 (29)	53 (20)	0.27	14 (22)	49 (21)	1.00
3MS Score [Median (IQI)]	94.0 (90.0-98.0)	94.0 (88.0-99.0)	94.0 (90.5-97.0)	0.86	93.0 (89.0-97.0)	95.0 (91.0-98.0)	0.07
General Cognitive Performance (Pre-Op) [Median (IQI)]	57.4 (52.8-61.6)	55.7 (51.3-61.6)	57.6 (53.0-61.5)	0.21	54.8 (51.2-59.7)	57.9 (53.4-62.8)	0.01
Charlson Comorbidity Index (Pre-Op) [Median (IQI)]	1.0 (0.0-2.0)	2.0 (0.0-3.0)	1.0 (0.0-2.0)	0.01	1.0 (1.0-3.0)	1.0 (0.0-2.0)	0.01
Visual Impairment, n(%)	3 (1)	0 (0)	3 (1)	1.00	2 (3)	1 (0)	0.13
Hearing Impairment, n(%)	120 (40)	12 (34)	108 (41)	0.58	28 (41)	92 (40)	0.89
Post-operative Apache II score [Median (IQI)]	12.0 (10.0-14.0)	14.0 (11.0-16.0)	12.0 (10.0-14.0)	0.001	13.0 (11.0-15.0)	12.0 (10.0-14.0)	0.01
Post-Operative MDAS [Median (IQI)]	2.3 (1.5-4.0)	4.3 (3.0-7.8)	2.3 (1.3-3.8)	<0.001	6.0 (3.9-7.8)	2.0 (1.3-3.0)	<0.001
CAM Delirium Variables
CAM Delirium, n(%)	68 (23)	21 (60)	47 (18)	<0.001	–	–	–
CAM Feature Counta (number present) [Median (IQI)]	2.0 (1.0-4.0)	5.0 (3.0-6.0)	2.0 (1.0-3.0)	<0.001	5.0 (4.0-6.0)	2.0 (1.0-3.0)	<0.001
Psychomotor agitation ever present, n(%)	35 (12)	14 (40)	21 (8)	<0.001	16 (24)	19 (8)	0.002
Psychomotor retardation ever present, n(%)	90 (30)	19 (54)	71 (27)	<0.001	46 (68)	44 (19)	<0.001
Inappropriate behavior ever present, n(%)	17 (6)	8 (23)	9 (3)	<0.001	12 (18)	5 (2)	<0.001
CAM Delirium Positive Days [Median (IQI)	0.0 (0.0-0.0)	1.0 (0.0-2.0)	0.0 (0.0-0.0)	<0.001	-	-	-

^aCount of non-psychomotor Confusion Assessment Method (CAM) features ever present during a patient’s hospital stay;

IQCODE = Informant Questionnaire of Cognitive Decline in the Elderly; Apache II = Acute Physiology and Chronic Health Evaluation II; MDAS = Memorial Delirium Assessment

Table 2 presents correlates of chart and CAM delirium status, including patient characteristics and delirium variables. Patients identified as delirious by either the CAM or the chart method had significantly higher comorbidity and illness severity scores, compared with non-delirious patients. Patients rated as delirious by the CAM had significantly lower scores on baseline measures of global cognitive function, however, no such difference in baseline performance on cognitive measures was found for chart identified cases.

Factors Associated with Delirium Ratings using Chart vs. CAM Methods

Among participants rated as delirious according to the chart-based method (n=35), 60% (n=21) were also rated as delirious according to the CAM. (Appendix Table 1) Among those chart-identified delirium patients where the delirium was documented on the night shift, the CAM was more likely to be negative (64% vs. 19% with CAM positive, p = 0.01).

Among CAM positive cases, the chart-based method was more likely to miss patients with lower disease severity by the APACHE-II score, as well as patients who did not demonstrate either psychomotor agitation or inappropriate behavior. (Appendix Table 2)

Predictive Validity of Chart-Based, CAM and Combined Methods

We examined the association between the three methods for identification of delirium-- CAM method alone (n=68 cases of delirium), chart-based method alone (n=35 cases) and combined CAM and chart-based methods (n=82 cases)--and three clinically relevant outcomes of postoperative delirium: hospital length of stay, post-operative complications and discharge to a post-acute rehabilitation facility, using regression models and population attributable risk (PAR). In multivariable adjusted models, CAM-rated delirium was associated with a 17% (Incident Rate Ratio (IRR)= 1.17; 95% CI 1.04-1.31) (Table 3) longer LOS compared to participants rated as non-delirious on the CAM, chart-based delirium with a 27% (IRR= 1.27; 95% CI 1.10-1.47) longer LOS and the combined approach with a 18% (IRR= 1.18; 95% CI 1.06-1.32) longer post-operative LOS. The population attributable risk estimates associated with LOS in Table 3 can be interpreted as the post-operative LOS attributable to delirium. That is, approximately 3% of LOS can be attributed to delirium identified using the CAM or chart-based method and 4% to delirium identified by the combined method, thus the combined method accounts for 33% additional LOS attributed to delirium than either method alone.

Table 3.

Relationship of Delirium Identification Method and Clinical Outcomes. (N=300)

		Post-operative Length of Staya		Post-operative Complicationsa		Post-acute Facility Dischargeb
Delirium Definition	Delirium Cases	IRR 95% CI	Population Attributable Risk, %c	IRR 95% CI	Population Attributable Risk, %	OR 95% CI	Population Attributable Risk, %
Delirium by Chart	35	1.27 (1.10-1.47)	2.50	1.94 (1.42-2.65)	5.65	14.69 (3.78-57.03)	4.91
Delirium by CAM	68	1.17 (1.04-1.31)	3.23	1.50 (1.13-1.99)	7.56	4.78 (2.22-10.33)	7.82
Combined Delirium	82	1.18 (1.06-1.32)	4.16	1.53 (1.17-2.01)	9.49	6.16 (2.95-12.84)	10.74

Note: Adjusted for age, female sex, nonwhite race, surgical type, pre-operative Charlson comorbidity index, pre-operative proxy Informant Questionnaire of Cognitive Decline in the Elderly (IQCODE), and post-operative Acute Physiology and Chronic Health Evaluation II (APACHE II) score;

^aPoisson Model

^bLogistic Regression Model;

^cPopulation attributable risk percentage is the product of a function of the relative risk (RR) of the outcome among those with delirium ([RR – 1])/RR) and the prevalence of delirium.

IRR = Incident rate ratio; 95% CI = 95% Confidence Interval; OR = Odds Ratio; CAM = Confusion Assessment Method

CAM-identified delirium was associated with a 50% increase in the number of post-operative complications (IRR= 1.50; 95% CI 1.13-1.99) (Table 3), the chart-based with nearly a doubling of the number of post-operative complications (IRR= 1.94; 95% CI 1.42-2.65); and the combined CAM/chart method with a 53% increase (IRR= 1.53; 95% CI 1.17-2.01). Approximately 8% of post-operative complications can be attributed to delirium identified by the CAM and nearly 6% to delirium identified by the chart-based method and more than 9% of complications can be attributed to delirium identified by the combined method.

Participants rated as delirious on the CAM had a five-fold increased risk for discharge to a post-acute care facility (Odds Ratio (OR)= 4.78; 95% CI 2.22-10.33) while chart-based ratings were associated with more than a 15-fold increase (OR= 14.69; 95% CI 3.78-57.03). The combined CAM/chart ratings were associated with an approximately six-fold risk (OR= 6.16 (95% CI 2.95-12.84). Approximately 8% of post-acute care discharges can be attributed to delirium identified by the CAM, 5% to delirium identified by the chart-based method and the combined approach is associated with an additional 50-100% of risk of post-acute care associated with delirium compared to either method alone (combined method PAR=11%).

DISCUSSION

We compared chart and interview-based methods for identifying delirium in older surgical patients. We found that the chart-based method is more likely to identify patients whose delirium occurs at night and patients with hyperactive delirium, while the CAM (interview)-based method is more likely to identify patients with hypoactive delirium and delirium in the setting of poor cognitive performance that may be missed by nursing staff. The combined approach (using both CAM and chart-based methods) maximizes sensitivity with a more complete capture of cases of delirium with respect to psychomotor features and time of onset. Delirium identified using the combined approach had strong predictive validity for clinically-relevant outcomes (hospital length of stay, post-operative complications and discharge to a post-acute care facility) and was associated with a higher attributable risk for these outcomes than either method alone.

Our findings are strengthened by the rigorous chart-review method applied, which included review of physician and nursing notes as well as admission, pre-operative, discharge and outpatient visit summaries. Our study sample is well characterized with respect to disease severity and overall health, and we were therefore able to compare cases of delirium identified using the CAM and chart-based methods on factors that are contained in the medical record (e.g., Charlson or APACHE II scores) and those that are not (e.g., pre-admission level of cognitive function).

Several limitations of this study are important to mention. A true gold-standard (e.g., continuous bedside monitoring of the patient) was not feasible. Instead, we used an outcomes validation approach and found that the combined approach had the strongest association with clinical outcomes. The chart method lacks sufficient information to determine type of delirium (hyperactive vs. hypoactive) in all cases and there is currently no measure of delirium severity that can be obtained using the chart method alone. The CAM is only one of several interview-based screening instruments for delirium; other instruments include the Memorial Delirium Assessment Scale,¹⁸ the Delirium Rating Scale Revised-98,¹⁹ and the Delirium Observation Screening Scale.²⁰ Although the CAM has several limitations, such as the lack of a validated severity score and the need for structured training and administration of a cognitive test in order to achieve high sensitivity, trained non-clinical staff can administer the CAM and it has been recommended as the best available bedside delirium screening instrument.²¹ Information from the family may have been taken into account by the research staff when scoring the CAM and by the clinical staff when charting, thus the methods may not be truly independent in all cases. It is important to note that there was no independent reference standard for delirium in this study; however, we did not consider this a limitation since both methods utilized have been extensively validated,^{2, 3, 9} all diagnoses were adjudicated by an expert panel, and validating the methods was not a goal of this study. The majority of patients were undergoing orthopedic surgery, which has high rates of postoperative delirium, therefore results may not be generalizable to other surgical populations. Finally, our study included patients from 2 hospitals in a single geographical area who were cognitively in-tact prior to surgery, relatively well-educated, primarily Caucasian and who were healthy enough to undergo elective surgery, limiting the potential generalizability of our findings.

The combination of interview and chart-based methods for identifying delirium have been used in several previous studies.^22-25 These studies, however, used the chart-based method to augment the interview-based method and did not directly compare characteristics of patients with delirium identified by chart and interviewer based methods. We did not use either method as a gold-standard, rather we examined the relative strengths of each method for identifying delirious patients. For example, the chart method identified fewer cases than the interview-based method but had the distinct added value of capturing cases of delirium occurring during the night shift. The combined chart/interview approach is designed to maximize sensitivity for diagnosis of delirium and may, however, sacrifice specificity. However, the results from our predictive validity analyses suggest that the combined approach performs better than either method alone in predicting important clinical outcomes.

A combined approach would be useful in both research and clinical settings. In research settings, where daily CAM administration is becoming relatively standard, a chart review could be added to pick up additional delirium cases. In clinical settings, where the chart is routinely reviewed by the clinical team, a daily CAM interview, taking less than 3 minutes, could be systematically included to increase delirium detection.

CONCLUSIONS

Delirium is common among hospitalized older patients and is increasingly recognized for its association with poor clinical, functional and quality of life outcomes.²⁶ It is of critical importance that delirium detection methods identify as many delirious patients as possible in order to promote optimal clinical care and enhance the study of delirium. A method that combines interview and chart-based methods maximizes identification of cases of delirium and should be considered in future work.

Appendix Table 1.

Characteristics of Chart-Identified Delirium Stratified by CAM Delirium Status (N=35)

Characteristic	Delirium by CAM, n=21	No Delirium by CAM, n=14	p-value
Age 80+, n(%)	5 (24)	4 (29)	1.00
MCI or dementia by proxy IQCODE, n(%)	4 (19)	6 (43)	0.15
3MS Raw Score [Median (IQI)]	94.0 (90.0-99.0)	94.0 (88.0-97.0)	0.54
General Cognitive Performance (Pre-Op) [Median (IQI)]	54.8 (52.8-57.5)	56.3 (49.9-63.5)	0.50
Post-operative Apache II score [Median (IQI)]	15.0 (13.0-17.0)	13.0 (10.0-14.0)	0.04
Charlson Comorbidity Index (Pre-Op) [Median (IQI)]	2.0 (0.0-3.0)	1.0 (0.0-2.0)	0.14
Visual Impairment, n(%)	0 (0)	0 (0)	-
Hearing Impairment, n(%)	7 (33)	5 (36)	1.00
Post-Operative MDAS [Median (IQI)]	6.6 (4.5-8.8)	2.7 (1.8-4.0)	<0.001
Chart Delirium Variables, n(%)
Evidence of reversibility	19 (90)	11 (79)	0.63
Agitation associated with charted confusion	12 (57)	8 (57)	1.00
Delirium documented at night shift	4 (19	9 (64)	0.01
CAM Delirium Variables, n(%)
Psychomotor agitation ever present	9 (43)	5 (36)	0.73
Psychomotor retardation ever present	16 (76)	3 (21)	0.002
Inappropriate behavior ever present	7 (33)	1 (7)	0.11

Note: IQI = Interquartile Interval; CAM = Confusion Assessment Method; IQCODE = Informant Questionnaire of Cognitive Decline in the Elderly; Apache II = Acute Physiology and Chronic Health Evaluation II; MDAS = Memorial Delirium Assessment

Appendix Table 2.

Characteristics of CAM-Identified Delirium Stratified by Chart Delirium Status (N=68)

Characteristic	Delirium by Chart,	No Delirium by Chart,	p-value
Age 80+, n(%)	5 (24)	13 (28)	1.00
MCI or dementia by proxy IQCODE, n(%)	4 (19)	10 (23)	1.00
3MS Raw Score [Median (IQI)]	94.0 (90.0-99.0)	92.0 (89.0-96.0)	0.16
Pre-operative General Cognitive Performance [Median (IQI)]	54.8 (52.8-57.5)	54.8 (50.3-59.8)	0.88
Preoperative Charlson Comorbidity Index [Median (IQI)]	2.0 (0.0-3.0)	1.0 (0.0-2.0)	0.09
Visual Impairment, n(%)	0 (0)	2 (4)	1.00
Hearing Impairment, n(%)	7 (33)	21 (45)	0.43
Post-Operative MDAS [Median (IQI)]	6.6 (4.5-8.8)	5.4 (3.8-7.3)	0.13
Post-operative Apache II score [Median (IQI)]	15.0 (13.0-17.0)	12.0 (10.0-14.0)	0.001
CAM Delirium Variables
Psychomotor agitation ever present, n(%)	9 (43)	7 (15)	0.03
Psychomotor retardation ever present, n(%)	16 (76)	30 (64)	0.41
Inappropriate behavior ever present, n(%)	7 (33)	5 (11)	0.04
CAM Feature Count (First 8, ever present) [Median (IQI)]	6.0 (5.0-6.0)	5.0 (4.0-5.0)	0.003

Note: IQI = Interquartile Interval; CAM = Confusion Assessment Method; IQCODE = Informant Questionnaire of Cognitive Decline in the Elderly; Apache II = Acute Physiology and Chronic Health Evaluation II; MDAS = Memorial Delirium Assessment