Diagnosing Delirium in Hospitalized Elderly Patients with Dementia: Adapting the Confusion Assessment Method to ICD-10 Diagnostic Criteria

Abstract

Background

The Confusion Assessment Method (CAM) performs well in DSM-IV delirium screening, but the ICD-10 classification utilizes more delirium symptoms.

Objectives

To compare performance characteristics of the CAM algorithm for screening and delirium diagnosis with ICD-10 and DSM-IV delirium criteria in high-risk patients.

Design

Prospective cohort study

Setting

Academic geriatric hospital

Participants

102 patients, aged 80–100 years, hospitalized for acute medical illness.

Measurements

Complete CAM instrument (9 items), scored using the 4-item CAM diagnostic algorithm. Gold standard classification of delirium was rated independently by expert consensus, based on DSM-IV and ICD-10 criteria for delirium.

Results

In 79 hospitalized patients, the CAM performed well for delirium screening (delirium prevalence of 24% by DSM-IV and 14% by ICD-10). Of all CAM features, acute onset and fluctuating course are most important for diagnosis (area under the curve, AUC, 0.92 in DSM-IV and 0.83 in ICD-10). Compared with the DSM-IV reference standard, the CAM diagnostic algorithm had a sensitivity of 0.74, specificity of 1.0, and AUC of 0.88; compared with ICD-10, which had a sensitivity of 0.82, specificity of 0.91, and AUC of 0.85. Adding psychomotor change to the CAM algorithm improves specificity to 97% but sensitivity is reduced to 55% for ICD-10 (AUC 0.96). Alternatively, applying psychomotor change sequentially to only the group identified with no delirium by the CAM algorithm improves sensitivity to 91% with specificity of 85% (AUC 0.95).

Conclusion

While the CAM diagnostic algorithm performs well against a DSM-IV reference standard, adding psychomotor change to the CAM algorithm improves specificity and diagnostic value against ICD-10 criteria over all in aged individuals with dementia, and improves sensitivity and screening performance when applied sequentially in CAM-negative individuals.

INTRODUCTION

Delirium has been shown to be a devastating syndrome that is associated with increased morbidity and mortality¹ This holds true especially for acutely ill older patients with dementia. Increased diagnostic effort is therefore warranted to minimize negative consequences. However, delirium prevalence and detection rates differ with respect to the classification system used². While DSM-IV criteria³ focus on rapid onset, attentional impairment, and organic cause, ICD-10 criteria⁴ include changes in psychomotor behaviour and disturbance in sleep-wake cycle and are more stringent than those of the DSM-IV classification. The Confusion Assessment Method (CAM)⁵ with its 4-item diagnostic algorithm is the most widely used screening test worldwide.^6,7 Developed on the basis of DSM-III-R, it is now predominantly used for DSM-IV delirium screening. In most countries other than the United States, however, ICD-10 is used for clinical diagnoses. Thus, the performance of the CAM algorithm as a diagnostic test for ICD-10 delirium detection should be assessed.

When delirium screening is routinely performed, the CAM has been widely used since the test is short and can be readily administered. Other common delirium screening tests⁸ are more complicated and time consuming⁷, or limited to postoperative settings. CAM, moreover, is the only delirium screening instrument that has been validated in Germany for acutely ill, elderly patients⁹.

In routine clinical practice, delirium diagnosis is often based exclusively on the results of the CAM diagnostic algorithm without any additional verification^6,10–14 although this practice is not supported by the articles originally presenting the CAM. Indeed, psychogeriatric specialists are not always available to validate the diagnosis clinically. Thus, validating CAM for purposes of diagnosing delirium in the face of a high-risk patient group with cognitive impairment represents an important area of investigation. This validation should involve both reference standards for delirium, DSM-IV and ICD-10 diagnostic criteria.

Older patients with dementia represent a high-risk group in which delirium is particularly difficult to diagnose,¹ especially with acute illness. In this setting, delirium often presents as a hypoactive syndrome that is easily overlooked and acute changes in cognition are often subtle and fluctuating. Thus, enhanced multi-professional effort may be required to detect them.¹⁵

It is useful to evaluate the performance of delirium diagnostic tests in high-risk groups that include false-positive challenges such as dementia and multi-morbidity, and in whom detection rates are low. The concept of a patient’s vulnerability¹⁶ suggests that the oldest-old and dementia patients bear a high delirium risk during acute illnesses requiring hospitalization. We systematically screened all patients older than 80 years who were admitted to a geriatric hospital in Germany in acute condition. With respect to European health care systems, this naturalistic setting provided a well-characterized sample of frail elderly and gave highly comparable patient groups of demented patients without delirium, demented patients with delirium, and cognitively intact controls.

The specific aims were:

1. To determine whether the CAM is valid for delirium screening in a naturalistic high-risk group of acutely ill and often demented geriatric patients. Screening validity of the CAM was compared to a multidisciplinary consensus diagnosis based on DSM-IV or ICD-10 criteria.

2. To determine whether the CAM can serve as a proxy for diagnosis in clinical settings requiring ICD-10 criteria. CAM’s performance in diagnosing delirium was assessed in comparison to consensus based DSM-IV and ICD-10 criteria.

3. To determine whether the diagnostic performance of the CAM criteria can be improved for ICD-10 delirium.

METHODS

Subjects

Between October 2003 and March 2004 acutely ill patients admitted to Bethanien-Hospital, the Geriatric Center at the University of Heidelberg, were recruited to this prospective study by systematic sampling. All patients (n=102) aged 80 and older, admitted on Tuesdays and Fridays were screened and approached. Exclusion criteria included global aphasia (n=3), and terminal condition (n=6). Fourteen patients refused to participate. Thus, a total of 79 patients were studied. All study participants were screened for capacity to consent by an independent consultant. Those able to consent provided written informed consent. For patients with impaired decisional capacity, written informed consent was obtained from the legal guardian. The study was conducted in accordance with the Declaration of Helsinki and was approved by the Heidelberg University ethics review board (No.255/2003). To avoid confounding by the stress associated with the hospital admission process, the study was conducted on the third day after admission. The time for completion of all assessments was less than 4 hours in 73 (92.4%) patients; and all were completed within 6 hours.

Procedures

Patients were evaluated regarding their demographic characteristics, overall disease¹⁷ and medication burden (see table 1). Delirium-associated factors and common risk factors known to increase delirium incidence in medical patients were summarized as a simple sum of the number of risk factors present (range 1–13, see table 1 for details). Number of medications with psychoactive or anticholinergic side effects, considered “delirogenic,” were calculated by using a list derived from the literature (corticosteroids, antibiotics, psychotropic medication, and furosemide)¹⁸. Primary caregivers were asked about acute cognitive and behavioral changes, sleeping patterns, and completed the Informant Questionnaire of Cognitive Decline (IQCODE)¹⁹, which detects long-term changes in cognitive and instrumental abilities for dementia assessment.

Table 1.

Sample Characteristics

	ALL PATIENTS	SUBGROUPS CONSENSUS DIAGNOSIS BASED ON DSM-IV
		Cognitively Intact Elderly	Dementia Without Delirium	Dementia with Delirium
N	79	20	38	21
Age (range 80–100 y)	84.1 ± 5.9	83.9 ± 5.8	84.5 ± 6.2	83.5 ± 5.4
Women	57 (72%)	18 (90%)	24 (63 %)	15 (71 %)
# of diagnoses	2.6 ± 1.4 (3.0; 0–6)	1.8 ± 0.9 ***	2.7 ± 1.4	3.3 ± 1.3
CIRS‡ ( range 0–56)	29.8 ± 5.2	26.5 ± 4.8 **	30.5 ± 4.4	31.8 ± 5.5
# of Medications	5.6 ± 2.6 (5.0; 1–13)	5.2 ± 2.6	5.6 ± 2.6	5.8 ± 2.7
# of Delirogenic medications	1.97 ± 1.5 (2.0; 0–7)	2.00 ± 2.1	1.81 ± 1.2	2.24 ± 1.4
# of Risk factors§	2.0 ± 1.5 (2.0; 0–5)	1.5 ± 1.4	2.3 ± 1.4	2.1 ± 1.6
Delirium Index¶	2.0 ± 1.5 (2.0; 0–5)	0.8 ± 1.0 ***	5.9 ± 3.9	8.9 ± 4.9 ††
IQCODE # (norm <3.3)	4.0 ± 0.5 (4.0; 1–5)	3.2 ± 0.2***	4.2 ± 0.6	4.2 ± 0.6
MMSE5‡‡ (norm >28)	17.0 ± 6.5 (2.0; 1–30)	28.0 ± 2.6 ***	17.4 ± 6.9	13.8 ± 6.3†††

Mean ± SD are depicted; median and range are given in parentheses. Significant differences in chi² tests are depicted by *** (= p<0.001), ** (p<0.01) and * p<0.05. In the dementia group comparison, significant differences are indicated by ^††† for p<0.001 and ^†† for p< 0.01.

^‡CIRS = Cumulative Illness Rating Scale¹⁷

^§#of risk factors ( range 0–13) includes delirium-associated factors (anemia, cachexia, dehydration, acute infections, oxygen saturation, metabolic disturbances, and diabetes mellitus) and common risk factors (cognitive deficits, sensory impairment, and restricted mobility)

^¶Delirium Index²⁰ (range 0–21)

^# IQCODE = Informant Questionnaire on Cognitive Decline of the Elderly¹⁹

^‡‡MMSE = Mini Mental State Examination²⁹

The CAM and Delirium Index²⁰ were rated by a physician-in-training or gerontologist based on brief cognitive testing, which included the Mini-Mental State Examination (MMSE), logic questions, and brief interview with nurse about acute onset of symptoms and sleep.⁹ Cognitive testing was performed independently (blinded to the CAM ratings) by a psychologist or geriatrician, included digit spans and verbal memory and was used to secure the consensus diagnoses.

Delirium Diagnosis According to DSM-IV Criteria

Delirium was diagnosed by a neuropsychiatric-geriatric consensus panel according to DSM-IV criteria. The panel used all available information from charts, clinical course, psychiatric consultation, neuropsychological testing, and proxy questionnaires. The consensus panel then assigned the patients to the following three groups: 21 patients with delirium and dementia (D+D), 38 demented patients (DP), and 20 cognitively intact (CI) patients (table 1).

Three patients with delirium were not previously diagnosed with dementia; however, their proxy IQCODE¹⁹ ratings resulted in a score higher than 3.4, indicating pre-existing cognitive deficits suggestive of dementia. We therefore combined all delirious patients into one group (D+D).

Delirium Diagnosis According to ICD-10 Criteria

A geropsychiatrist interviewed all patients independently, reviewed their charts (on day 3), and completed the checklist for fulfilment of the research criteria of the International Classification of Diseases (ICD-10). The ICD-10 checklist result was reassessed after discharge by the consensus panel, using all available information to confirm diagnosis according ICD-10 criteria and no patients were reclassified.

Statistical Analysis

Demographic variables and group differences were compared by Kruskal-Wallis ANOVAs with post hoc testing by the Multiple Comparisons Test. Categorical variables were assessed in contingency tables using the Chi-square test.

To assess performance of all 9 CAM items and the CAM algorithm, we calculated sensitivity and specificity, prevalence-based positive and negative predictive value (PPV, NPV), and likelihood ratio (LR) for DSM-IV and ICD-10 delirium in the entire group (n=79).

The ability of CAM items to improve diagnostic validity for ICD-10 delirium was evaluated by logistic regression analyses. All possible subset logistic regression was performed (i.e., 511 combinations) to detect the model with the largest area under the curve (AUC) for comparison. Models were excluded when individual items showed perfect predictability in the logistic regression. Secondly, the effect of adding additional items to the 4-item CAM diagnostic algorithm was compared (i.e., adding all five remaining items of the CAM⁵: disorientation, memory impairment, perceptual disturbances, abnormal psychomotor activity, and altered sleep-wake cycle, see online appendix for all nine CAM items), thus meeting all ICD-10 research criteria). Formal model testing was performed using the likelihood ratio test. To compare all models we used Akaike’s and Bayesian information criteria (AIC and BIC). All statistical analysis was carried out using Stata Version 10.1.

RESULTS

Clinical Characteristics of the Study Population

The study population was comprised of acutely ill elderly individuals, as characterized in Table 1. Hospital admission was initiated because of falls (n=16, 20%), infections (n=15; 19%), cardiopulmonary problems (n=11; 14%), psychiatric disorders with medical comorbidity (n=10; 13%), cerebrovascular conditions (n=6; 8%) metabolic disease (n=2; 3%) or other reasons (22%). Mean length of hospital stay was 17.4±10.3 days; six patients died (7.5%).

Subgroups did not differ with respect to the demographic, medication and risk variables that are depicted in Table 1. However, significantly higher morbidity was observed in the two dementia groups and, as expected, in cognitive measures.

CAM for Screening in DSM-IV and ICD-10 Delirium

Prevalence of DSM-IV delirium was 28% in all elderly and 36% in demented patients. ICD-10 delirium was diagnosed in about half of these patients, resulting in rates of 14% and 19%, respectively. The validity of screening based on the 4-item CAM diagnostic algorithm in this high-risk group is depicted in Table 2. For screening properties, a high sensitivity and a high positive predictive value (PPV) are desirable. Six patients with DSM-IV delirium were missed by the CAM algorithm (71% sensitivity). All but one of the DSM-IV delirium patients who screened negative by the CAM algorithm had an acute onset, but did not show a positive score for any other CAM item. One scored positively only for inattention and psychomotor abnormality. The prevalence-dependent PPV reached 1.0, supporting the usefulness of the CAM for screening purposes.

Table 2.

Validity of the CAM Algorithm in DSM-IV and ICD-10 Delirium Classification

Frequency tables	DSM-IV Delirium	ICD-10 Delirium
	(+) (−)	(+) (−)
	CAM (+) 15 0	CAM (+) 9 6
	CAM (−) 6 58	CAM (−) 2 62
Correct Class.	0.92 (0.87–0.98)	0.90 (0.83 – 0.97)
Sensitivity	0.74 (0.50–0.86)	0.82 (0.51 – 0.96)
Specificity	1.00 (0.92–1.00)	0.91 (0.82 – 0.96)
Positive PV	1.00 (1.00–1.00)	0.60 (0.35 – 0.82)
Negative PV	0.91 (0.84–0.98)	0.97 (0.93 – 1.00)
LR for positive Test	74*	9.27 (4.11 – 20.91)
LR for negative Test	0.28 (0.15–0.56)	0.20 (0.06 – 0.70)
AUC	0.88	0.85

Frequency tables and validity measures of delirium detection performance of the algorithm in the two main classification systems. Confidence intervals are shown in brackets.

Correct Class. = correct classification, PV = Predictive Value, LR = Likelihood Ratio.

^*= calculated using a specificity of 99%.

AUC= area under the Receiver Operating Characteristic curve

Taking ICD-10 delirium diagnosis as the reference standard, sensitivity was even better (9/11 delirium patients) but PPV declined to 0.6 as delirium prevalence was lower and six false-positive diagnoses were made. These patients lacked abnormal psychomotor features as a mandatory ICD-10 delirium criterion in all but one case. The favorable screening performance was also confirmed in the subgroup of demented patients (sensitivity 71%). Applying psychomotor change sequentially to the group identified with no delirium by the CAM algorithm improves overall screening performance in reference to ICD-10 delirium (sens 91%, spec 85%, AUC 0.95), see Figure 1. In comparison, the MMSE alone shows an AUC of 0.17 for ICD-10 delirium and 0.22 for DSM-IV delirium, and demonstrates high specificity only in cognitively intact elderly.

Figure 1. Diagnostic Validity of the CAM Algorithm, Sequential and Additive Model for ICD-10 Delirium Diagnosis.

Original 4-item CAM algorithm (DSM-based), sequential application (original CAM algorithm, with sequential addition of the psychomotor change item only to the CAM-negative group) and additive model i.e., adding psychomotor change to the original CAM algorithm are depicted. All models are validated against the ICD-10 delirium diagnosis as reference standard. The area under the ROC curve adapted with maximum likelihood method is depicted. Sensitivity, specificity, positive and negative predictive values (PPV and NPV) as well as correct classification percentages are given.

The performance of the individual items that comprise the 9-item CAM instrument differed in their diagnostic performance with reference to the disease classification systems. In single-item ROC (receiver operating characteristics) analyses, item 1 (acute onset and fluctuating course) is especially meaningful in both diagnostic systems, reaching a high screening test quality as indicated by the AUC. For DSM-IV delirium, disorganized thinking and, to a lesser degree, clouding of consciousness are important items, while in ICD-10 delirium psychomotor changes play a prominent role (see online Appendix for details).

Clinical Delirium Diagnosis on the Basis of the CAM

CAM does not include all the diagnostic features of delirium required by the current classification systems and therefore cannot be used to rate all of these features. To justify the frequent use of CAM algorithm as a diagnostic tool, especially in geriatric settings, diagnostic test performance for ruling out disease such as specificity and PPV must be assessed.

When DSM-IV criteria for delirium are applied, the CAM algorithm performs with a specificity of 100% and a PPV of 1.0. Thus, a diagnostic conclusion is justified. However, performance measures are inferior when ICD-10 delirium criteria represent the gold standard and the CAM may not have sufficient accuracy to rule out a delirium diagnosis by ICD-10 criteria.

Extending the CAM Algorithm to Improve Diagnostic Accuracy for ICD-10 Delirium

To improve the diagnostic performance of CAM for ICD-10 delirium, our goal was to identify a set of CAM items that provided maximal diagnostic accuracy and were practical for clinical use (Figure 1). Thus, we evaluated a two-step approach. Estimated by logistic regression analysis, the performance of the CAM algorithm (logLH -19.0, AUC 0.86) was compared to several extended algorithms, including item 8 (psychomotor change) or 9 (sleep disturbance). The AUC was 0.93 when psychomotor change was added to the CAM algorithm (logLH -17.1); but lower when including sleep-wake-disturbance (logLH -18.2, AUC 0.89) as were other combinations. Models were then compared by information criteria AIC and BIC. The optimal diagnostic performance is obtained by adding abnormal psychomotor activity to the CAM algorithm, (Sens. 54.6, Spec. 97.1, PPV 0.75). The trade off for better diagnostic validity, however, is lower sensitivity.(original CAM algorithm: 82%; extended CAM 55%). Using psychomotor change sequentially to the group identified with no delirium by the CAM algorithm improves overall screening performance in reference to ICD-10 delirium (sens 91%) at the expense of specificity (85%). Summarizing, ICD-10 and DSM-IV criteria are especially different in evaluating specific symptoms, such as psychomotor changes and sleep-wake cycle disturbances. Thus, adding psychomotor changes (either hyperactive or hypoactive) to the CAM algorithm substantially increases diagnostic value for ICD-10 delirium with improved specificity (97%) but decreased sensitivity (55%). When psychomotor change is added sequentially to the subgroup of patients who are delirium negative by the original CAM algorithm, the screening performance is improved with a sensitivity of 91% and specificity of 85%.

The ROC graphs in Fig. 1 depict sensitivity and the false-negative rate of the CAM algorithm, the sequential application of psychomotor activity to patients with a negative CAM algorithm and the additive model to improve diagnostic specificity to illustrate their diagnostic performance (Fig. 1).

DISCUSSION

The validated German translation of the Confusion Assessment Method⁹ was used to evaluate delirium detection in high-risk oldest-old patients in a European setting. Applying DSM-IV criteria, point prevalence of delirium in our naturalistic cohort of acutely ill, hospitalized elderly individuals was 27%, while the more rigid ICD-10 criteria classified 14% of the elderly as being delirious. Investigations in medical cohorts in various countries have revealed comparable delirium prevalence rates.^2,21

Screening Properties of CAM in DSM-IV and ICD-10 Delirium

In comparison to recent reviews^6,7 reporting an overall sensitivity of 94% (95%CI: 91–97%) and specificity of 89% (95% CI: 85–94%), the German CAM performs appropriately even in the highest risk patients, who present with a high prevalence of neuropsychiatric disease (84%). Sensitivity for DSM-IV delirium is lower in our study (74%) than in most reports. This might be due to the high rate of dementia (74%) and depression (34%), which have not been well-examined in previous validation studies. Very old age and acute disease in our sample could also account for the slightly inferior performance of the original CAM algorithm^5,6 As specificity is high, our rigorous operationalization using standardized tasks for attention and disorganized thinking assessment, as is recommended routinely for all uses of the CAM^5,6 (See CAM Training Manual available at <www.hospitalelderlifeprogram.org>), and performing the MMSE and a nurses’ interview might have reduced the sensitivity for DSM-IV delirium. This is even more likely as performance characteristics for ICD-10 delirium screening are higher than previously reported²². Moreover, for the high-risk delirium population assessed, the MMSE is inadequate as it only detected delirium in previously cognitively intact subjects. Serial cognitive testing might improve performance, but they require additional time and effort²³.

The CAM Algorithm Result as a Proxy for Delirium Diagnosis

In geriatric hospitals, rehabilitation units, and nursing homes, the CAM algorithm is often used as a diagnostic instrument ^6,10–14 although the authors advise caution in this regard.⁵

For DSM-IV diagnosis, the CAM algorithm reached a high specificity, justifying its use as a proxy for delirium diagnosis in demented patients. Our methods were optimized by including MMSE, formal testing of attention and disorganized thinking, nurse interviews, and in-depth training of raters.⁶ Others have reported lower specificity for DSM-IV delirium and for ICD-10 delirium,²² but they did not consistently apply formalized tests such as MMSE as well as interviews with nurses to score the CAM ratings. To obtain high levels of performance, training and formal cognitive assessment are highly recommended for optimal use of the CAM.^5–7,9

Selecting a model for ICD-10 delirium diagnosis from CAM items

When ICD-10 criteria are utilized, however, diagnostic performance of the CAM algorithm was not adequate to rule out delirium. The sequential addition of other CAM items was examined to determine whether this step might help to improve results. To improve screening properties, the original CAM algorithm should be employed initially, and then extended by adding psychomotor activity changes to the original CAM algorithm. This new approach demonstrated a favourable diagnostic performance, and therefore, was selected as the ideal model on the basis of Akaike’s test. Psychomotor change is a mandatory criterion in ICD-10 and hypoactivity was frequently found to carry a poor prognosis.^14,24,25 Furthermore, the hypoactive subtype of delirium is easily overlooked²⁶ and therefore warrants special attention¹. Thus, the extended CAM not only detects ICD-10 delirium with high accuracy (AUC = 0.96) but also heightens awareness to a clinically relevant delirium syndrome.²⁷ Moreover, apart from the diagnosed ICD-10 delirium patients, two subgroups with high delirium probability are better characterized, (1) patients scoring positively on the CAM but who do not show any psychomotor changes and (2) patients revealing psychomotor changes but scoring negatively on the CAM algorithm. These can be treated as subgroups that require a more thorough or advanced delirium evaluation. These patients might be considered as subsyndromal ICD-10 delirium cases that are also known to have a poor prognosis and a high risk for developing full delirium.²⁸ With this approach, the extended CAM not only improves ICD-10 diagnosis approximation accuracy, but also delirium detection and care.

Strengths of our prospective study include the naturalistic patient group of high-risk elderly with acute disease that is very well characterized with respect to risk factors and comorbidity. State-of-the-art measures for assessing delirium were used. Limitations include the lack of a subgroup of delirious patients without prior cognitive impairment and the overall sample size, which was limited because of time and resource restrictions.

Conclusions

We assessed both the screening properties and the diagnostic performance of the validated CAM instrument⁹ for DSM-IV and ICD-10 delirium classification.

Few studies have applied ICD-10 criteria in delirium research. In many countries, however, ICD-10 criteria are mandatory for documenting health care and cost and, therefore, screening and diagnostic properties must be evaluated. We were able to demonstrate that the CAM is valid for delirium screening according to the ICD-10 criteria. To increase correct classification and specificity in ICD-10 delirium, we propose an extended 5-item CAM algorithm, the I-CAM (I for ICD-10), which utilizes abnormal psychomotor activity as an additional item. The 5-item I-CAM provides a useful diagnostic and screening tool for ICD-10 delirium and, moreover, calls attention to the important and often unrecognized syndrome of hypoactive delirium. Use of the new algorithm must still be confirmed in various other patient groups, in different settings, and across delirium aetiologies.