The CAM-ICU-7 Delirium Severity Scale: A Novel Delirium Severity Instrument for Use in the Intensive Care Unit

Abstract

Objective

Delirium severity is independently associated with longer hospital stays, nursing home placement and death in patients outside the intensive care unit (ICU). Delirium severity in the ICU is not routinely measured because the available instruments are difficult to complete in critically ill patients. We designed our study to assess the reliability and validity of a new ICU delirium severity tool, the Confusion Assessment Method (CAM)-ICU-7 delirium severity scale.

Design

Observational cohort study.

Setting

Medical, surgical and progressive ICUs of three academic hospitals.

Patients

518 adult (≥18 years) patients.

Interventions

None.

Measurements and Main Results

Patients received the CAM-ICU, Richmond Agitation-Sedation Scale (RASS), and Delirium Rating Scale-Revised (DRS-R)-98 assessments. A 7-point scale (0-7) was derived from responses to the CAM-ICU and RASS items. CAM-ICU-7 showed high internal consistency (Cronbach's alpha=0.85) and good correlation with DRS-R-98 scores (correlation coefficient=0.64). Known-groups validity was supported by the separation of mechanically ventilated and non-ventilated assessments. Median CAM-ICU-7 scores demonstrated good predictive validity with higher odds (OR=1.47; 95% CI=1.30-1.66) of inhospital mortality, and lower odds (OR=0.8; 95% CI=0.72-0.9) of being discharged home after adjusting for age, race, gender, severity of illness, and chronic comorbidities. Higher CAM-ICU-7 scores were also associated with increased length of ICU stay (p=0.001).

Conclusion

Our results suggest that CAM-ICU-7 is a valid and reliable delirium severity measure among ICU patients. Further research comparing it to other delirium severity measures, its use in delirium efficacy trials, and real life implementation is needed to determine its role in research and clinical practice.

Introduction

Delirium, sometimes referred to as acute brain failure, is characterized by altered consciousness with a reduced ability to focus, sustain, or shift attention that develops quickly and fluctuates over the course of the day.[1] Delirium is highly prevalent among critically ill patients;[2-6] and is associated with greater lengths of intensive care unit (ICU) and hospital stays, mortality, and cost of care.[3-5] Delirium severity has also been associated with adverse patient outcomes including higher risks of nursing home placement and mortality.[7-9] Delirium severity scores have been used in clinical settings and for conducting research outside the ICU,[10] but their use in the ICU have been restricted mostly to research. Measuring delirium severity in the ICU may not only fulfill a prognostic role for patients with delirium, but could also serve as a guide for successful therapeutic interventions.

Delirium Rating Scale-Revised-98 (DRS-R-98) is a widely used delirium severity scale advocated in the ICU setting given its strong psychometric properties.[11] Unfortunately DRS-R-98 use in the ICU has been limited by the inherent difficulty in implementing and interpreting it among mechanically ventilated patients due to the structure of its questions. Hence assessments performed through DRS-R-98 results in missing data, thus decreasing its clinical applicability. It also requires significant administration time and expert judgment on several of its items, thereby further restricting its use by untrained busy clinicians. An ICU delirium severity tool that can overcome these limitations would be ideally suited for the complex, chaotic ICU environment.

Indiana University Center for Aging Research conducted a randomized, clinical trial the “Pharmacological Management of Delirium (PMD)”[12] in which every enrolled patient underwent daily sedation, delirium and delirium severity assessments. The presence of these assessments allowed us to undertake the current study with the objective to develop a new delirium severity tool and to assess its reliability and validity.

Methods

The Research Compliance Administration of Indiana University-Purdue University approved the study (Protocol#1010002428). Informed consent was obtained from patients’ legally authorized representatives.

Study Setting

Patients enrolled in the PMD trial and admitted to the ICU services of three Indianapolis hospitals (Wishard Memorial Hospital (WMH) now known as Eskenazi Health, University Hospital, and Methodist Hospital) from March 2009-January 2015 were included in the study. PMD is a National Institutes of Health funded clinical trial[12] testing the effectiveness of a multi-component intervention to reduce delirium duration and severity in the ICU. The details of the trial have been published elsewhere.[12] WMH is a 457-bed, university-affiliated, public hospital with three ICU units, an 8-bed surgical ICU (SICU), a 14-bed medical ICU (MICU), and a 29-bed progressive (step-down) ICU (PICU). University hospital is a 257-bed tertiary care hospital with 36 MICU and SICU beds. Methodist hospital is an 802-bed tertiary care center with a 65-bed MICU/SICU.

Inclusion and Exclusion Criteria

Inclusion criteria: 1) admitted to the ICUs of WMH, University Hospital and Methodist Hospital; 2) age ≥18 years; and 3) had delirium based on Confusion Assessment Method for the ICU (CAM-ICU).[13] Exclusion criteria: 1) not English speaking; 2) hearing impaired; 3) legally blind; 4) admitted with alcohol intoxication; 5) prisoners; 6) having an Axis 1 Psychiatric disorder; or 7) pregnant/nursing.

Assessments

Richmond Agitation-Sedation Scale (RASS)[14] and the CAM-ICU[13] were used to assess patients’ sedation and delirium respectively. RASS has excellent inter-rater reliability (inter-class correlation coefficient=0.956; k=0.73, 95% CI=0.71-0.75) and high validity.[14] CAM-ICU has high criterion validity (sensitivity=97%, specificity=98%, accuracy=98.4%) and high inter-rater reliability (k=0.96, 95% CI=0.92-0.99).[13] Trained research assistants performed twice-daily RASS/CAM-ICU assessments. Patients with a RASS score of −4 (no response to voice, but movement or eye opening to physical stimulation) or −5 (no response to voice or physical stimulation) were ineligible for CAM-ICU assessments. Patients were considered delirious if they had a RASS greater than or equal to −3 (any response to verbal stimulation) and a positive CAM-ICU result, achieved by showing signs of acute change in mental status or fluctuating course, displaying features of inattention, and either disorganized thinking or altered level of consciousness.[13] Research assistants administered DRS-R-98 twice daily to assess delirium severity covering 24-hour period using information from family, nurses, doctors and medical charts.[11] DRS-R-98 is a 16-item scale with 13 severity items; each rated from 0-3 with a maximum of 39 points with higher scores indicating greater delirium severity. DRS-R-98 assesses symptoms such as impairments in attention, short and long-term memory, visuospatial ability and orientation, perceptual and sleep-wake cycle disturbances, abnormalities of language, thought process and content, motor agitation/retardation, and mood lability. It has excellent inter-rater reliability (intra-class correlation=0.97) and internal consistency (Cronbach's α=0.94).[11] All the research assistants had bachelor's degree and one was an MD. Dr. Paula Trepacz, the developer of DRS-R-98 trained the research assistants on DRS-R-98 administration. The initial training consisted of didactics followed by as-needed consultations. Afterwards, Dr. Malaz Bosutani, an expert dementia and delirium researcher oversaw the training and quality control for DRS-R-98 administration and scoring.

Development of CAM-ICU-7 Delirium Severity Scale

A 7-point rating scale (0-7) was derived from the CAM-ICU and RASS assessments. The CAM-ICU items were further categorized as shown in Table 1. The scoring method was adapted from a prior study validating CAM-S as a delirium severity instrument outside the ICU setting.[10] CAM-ICU-7 maintained the same scoring scheme of CAM-S, but the scores were objectively derived based on the CAM-ICU and RASS items (Table 1). For acute onset, we could only create a binary outcome based on the definition. For inattention, disorganized thinking and altered level of consciousness, we conducted regression models with DRS-R-98 as the dependent variable and chose the cut-off points in each domain to maximize the correlation with DRS-R-98. The final CAM-ICU-7 score ranges from 0-7 with 7 being most severe. CAM-ICU-7 scores were further categorized as 0-2: no delirium, 3-5: mild to moderate delirium, and 6-7: severe delirium.

Table 1.

The CAM-ICU-7 Delirium Severity Scale

CAM-ICU
Items	Grading	Score
1. Acute Onset or Fluctuation of Mental Status Is the patient different than his/her baseline mental status? OR Has the patient had any fluctuation in mental status in the past 24 hours as evidenced by fluctuation on a sedation/level of consciousness scale (i.e., RASS/SAS), GCS, or previous delirium assessment?	0 absent 1 present
2. Inattention Say to the patient, “I am going to read you a series of 10 letters. Whenever you hear the letter ‘A,’ indicate by squeezing my hand.” Read letters from the following letter list in a normal tone 3 seconds apart. SAVEAHAART (Errors are counted when patient fails to squeeze on the letter “A” and when the patient squeezes on any letter other than “A”)	0 absent (correct ≥ 8) 1 for inattention (correct 4-7) 2 for severe inattention (correct 0-3)
3. Altered Level of Consciousness Present if the Actual RASS score is anything other than alert and calm (zero)	0 absent (RASS 0) 1 for altered level (RASS 1, −1) 2 for severe altered level (RASS >1, < −1)
4. Disorganized Thinking Yes/No Questions 1. Will a stone float on water? 2. Are there fish in the sea? 3. Does one pound weigh more than two pounds? 4. Can you use a hammer to pound a nail? Errors are counted when the patient incorrectly answers a question. Command: Say to patient “Hold up this many fingers” (Hold two fingers in front of patient). “Now do the same with the other hand” (Do not repeat number of fingers) An error is counted if patient is unable to complete the entire command.	0 absent (correct ≥ 4) 1 for disorganized thinking (correct 2, 3) 2 for severe disorganized thinking (correct 0, 1)
Total Score

CAM-ICU: Confusion Assessment Method for the Intensive Care Unit; RASS: Richmond Agitation Sedation Scale; SAS: Sedation-Agitation Scale; GCS: Glasgow Coma Scale

Other Data and Clinical Outcomes

Baseline demographics such as age, gender, and race were collected. Patients’ chronic comorbidities were assessed using the Charlson comorbidity index.[15] The severity of acute illness was assessed using the Acute Physiology and Chronic Health Evaluation (APACHE) II scale.[16] Length of ICU and hospital stay, and in-hospital mortality data were collected from electronic medical records.

Statistical Analyses

Internal consistency-reliability was assessed using Cronbach's alpha. Pearson correlation coefficient was used to assess correlations between the CAM-ICU-7 and DRS-R-98 in the overall sample as well as in specific subgroups. We used Wilcoxon-Rank Sum tests to compare CAM-ICU-7 severity across known subgroups based on their mechanical ventilation status and age. Logistic regression analysis was used to assess the relationship of in-hospital mortality and discharge status with the CAM-ICU-7 (median, maximum) and DRS-R-98 (median) summary scores as well as delirium duration after adjusting for age, race, gender, Charlson comorbidity index and severity of illness. For assessments with missing items on the DRS-R-98, we imputed the total DRS-R-98 score if at least 50% of the thirteen scale items were completed. We calculated the total score by taking the mean of the completed items and multiplying by the total number of items on the DRS-R-98. Due to the skewed nature of the ICU length of stay (LOS) outcome, we used linear regression with the log (ICU LOS+1) for associations with CAM-ICU-7 severity measures adjusting for age, race, gender, Charlson comorbidity index and severity of illness. All analyses were conducted using SAS 9.4 software (SAS Institute Inc., Cary, NC, USA).

Results

We included 518 delirious patients in the study. The mean age of the patients was 60.2 years (SD 16.1), 55% were females, 45% were African-Americans and 59% were mechanically ventilated (Table 2).

Table 2.

Patients’ Characteristics

Patients’ Characteristics*	(n: 518)
Age	60.2 (16.1)
Female n (%)	286 (55.2)
African-American n (%)	232 (45)
Mechanical ventilation n (%)	304 (58.7)
Education in years	11.4 (2.4)
Katz ADLa Scale	5.4 (1.4)
Lawton IADLb Scale	6.1 (2.6)
IQCODEc	3.2 (0.5)
APACHEd II	20 (8.2)
Charlson comorbidity Index	3.2 (2.8)
Delirium duration in days	3 (4.3)
Mechanical ventilation duration in days	4.0 (6.7)
Intensive care unit length of stay in days	14.4 (15.2)
Hospital length of stay in days	26.4 (34.1)
Discharged Home n (%)	189 (36.5)
Mortality n (%)	57 (11)

^*Data presented as means (SD) unless otherwise specified.

^aActivities of daily living

^bInstrumental activities of daily living

^cInformant questionnaire on cognitive decline in elderly

^dAcute physiology and chronic health evaluation score

Internal Consistency-Reliability

We found high internal consistency-reliability of the CAM-ICU-7 scales (Cronbach's alpha=0.85). We performed sensitivity analyses by examining possible effects of race, gender, age, and mechanical ventilation. Cronbach's alpha was consistently high in various subgroups: 0.85 in African-Americans, 0.85 in Caucasians; 0.86 in females, 0.85 in males; 0.86 in patients age <65 years, 0.83 in patients ≥65 years, and 0.83 among mechanically ventilated compared to 0.82 among non-ventilated.

Correlation of CAM-ICU-7 with DRS-R-98 (Construct Validity)

We completed 8,056 RASS and CAM-ICU assessments on 518 patients. Out of 8,056 RASS and CAM-ICU assessments, there were 5,120 assessments where DRS-R-98 scores were available (3,709 completed assessments, 1,411 assessments with at least seven completed items where the DRS-R-98 scores were imputed). CAM-ICU-7 scores correlated well with the 5,120 DRS-R-98 scores with a correlation coefficient of 0.64, hence demonstrating construct validity. The correlation coefficient was 0.67 for assessments with all completed DRS-R-98 items and 0.56 for assessments with imputed DRS-R-98 scores. The scores also correlated among mechanically ventilated (r=0.40) and non-ventilated assessments (r=0.66), although ventilated patients had higher DRS-R-98 missing values. The correlation coefficient was 0.66 for patients <65 years of age and 0.57 for those ≥65 years. Supplemental Digital Content – Figure 1 shows the average CAM-ICU-7 scores for number of items completed on DRS-R-98, demonstrating an inverse relationship between missing DRS-R-98 values and CAM-ICU-7 scores.

CAM-ICU-7 scores by clinical sub-groups (Known-Groups Validity)

CAM-ICU-7 scores were higher in assessments among mechanically ventilated patients [median=5 (IQR=2-7)], compared to non-ventilated assessments [median=0 (IQR=0-3)] (p<0.001). CAM-ICU-7 scores also increased with increasing age, median: [<50 years: 0 (0-3), 50-64 years: 1 (0-4), ≥65 years: 2 (0-5)] (p<0.001).

Association of CAM-ICU-7 with clinical outcomes (Predictive Validity)

The median CAM-ICU-7 score from each patient during hospitalization was associated with an odds ratio (OR) of 1.47 (95% CI=1.30-1.66) (AUC=0.785) of in-hospital mortality after adjusting for age, race, gender, severity of illness, and chronic comorbidities. Similar results were obtained using the highest CAM-ICU-7 scores [OR=1.32 (1.11-1.57)] (AUC=0.731). In contrast, the logistic models using median DRS-R-98 scores or delirium duration provided lower AUCs [DRS-R-98=0.727 (p=0.06); delirium duration=0.685 (p=0.003)] for in-hospital mortality compared to using median CAM-ICU-7. For patients who did not die during the hospitalization (n=461), higher median CAM-ICU-7 scores during hospitalization were associated with lower odds [OR=0.8 (CI=0.72-0.9)] (AUC=0.747) of being discharged home after adjusting for age, race, gender, severity of illness, and chronic comorbidities. Similarly highest CAM-ICU-7 scores were associated with lower odds of discharge to home [OR=0.78 (0.71-0.86)] (AUC=0.764). Table 3 shows the odds of in-hospital mortality and discharge to home associated with delirium severity (measured by CAM-ICU-7 and DRS-R-98), and delirium duration. Supplemental Digital Content – Table 1 shows the logistic regression models for median CAM-ICU-7 scores associated with mortality and discharge to home compared with the median DRS-R-98 scores and delirium duration. Supplemental Digital Content – Figure 2 shows the receiver operating characteristics (ROC) for the median CAM-ICU-7 and delirium duration for mortality (2a) and discharge to home (2b) respectively. The median CAM-ICU-7 scores (p=0.001; partial r=0.145) and highest CAM-ICU-7 scores (p<0.001; partial r=0.327) were also associated with longer length of ICU stay.

Table 3.

Odds ratios (OR) of in-hospital mortality and discharge destination associated with delirium severity measured through CAM-ICU-7 and DRS-R-98, and delirium duration.

Variables		In-Hospital Mortality
		OR (95% CI)+	AUCc
Delirium Severity	CAM-ICU-7a (n=518)*	1.47 (1.30 – 1.66)	0.785
	DRS-R-98b (n=465)#	1.11 (1.04 – 1.17)	0.727
Delirium Duration (days) (n=518)*		1.05 (0.98 – 1.11)	0.685

		Discharged Home
		OR (95% CI)	AUC
Delirium Severity	CAM-ICU-7 (n=461)*	0.80 (0.72 – 0.90)	0.747
	DRS-R-98 (n=434)#	0.92 (0.88 – 0.97)	0.743
Delirium Duration (days) (n=461)*		0.87 (0.82 – 0.93)	0.765

^aCAM-ICU-7: Confusion Assessment Method for the Intensive Care Unit-7

^bDRS-R-98: Delirium Rating Scale Revised

^cAUC: Area under the curve

^*Data from the whole cohort.

^#Data from patients in whom DRS-R-98 was calculated.

⁺Models adjusted for age, gender, race, severity of illness, and chronic co-morbidities

Sub-categorization of CAM-ICU-7 Scores

We categorized the CAM-ICU-7 scores as 0-2: no delirium, 3-5: mild to moderate delirium, and 6-7: severe delirium. After adjusting for age, race, gender, severity of illness and chronic comorbidities, patients with severe delirium had significantly higher odds of death (OR=2.92; CI=1.17-7.26, p=0.02) compared to those with mild to moderate delirium.

Discussion

Our results suggest that CAM-ICU-7 delirium severity scale is a valid, reliable and practical delirium severity measure that correlates with the currently available, validated delirium severity scale, the DRS-R-98. Further more, delirium severity as measured by the CAM-ICU-7 scores significantly predicts the clinical outcomes of in-hospital mortality, discharge destination and length of ICU stay. Derived from the widely used RASS and CAM-ICU clinical tools, the CAM-ICU-7 delirium severity scale showed good test characteristics with a higher predictive validity for in-hospital mortality over delirium severity measured through the DRS-R-98 and over delirium duration.

In addition to its association with relevant clinical outcomes, the structure of the CAM-ICU-7 offers certain practical elements that may allow easy incorporation into busy clinical practice. First and foremost is the absence of additional data collection. The data to calculate CAM-ICU-7 are already generated through the RASS and CAM-ICU assessments. The other advantage includes an objective ordinal score that could be followed over time to assess the efficacy of therapeutic measures in controlling delirium symptoms. Our project did not address the questions of implementation of the CAM-ICU-7 into the ICU and the efficacy of interventions to reduce delirium severity. Although with an increase in research to reduce delirium burden in the ICU, incorporation of a valid and practical delirium severity measure such as the CAM-ICU-7 will help in answering the aforementioned observations. Also use of short, practical tools in research studies will produce results that could be quickly and efficiently translatable to the clinical setting.

Currently, measurement of delirium severity in the ICU has been limited to research and is not a standard clinical practice. As mentioned above, this is largely due to the lack of brief, practical delirium severity scales along with absence of efficacious therapeutic agents for delirium symptoms. DRS-R-98 is a valid and reliable instrument for measurement of delirium severity[11] that has been extensively used for research. Although it has strong psychometric properties[11] and covers the breadth of delirium symptoms, its use in the ICU has been limited. This is due to the amount of time required for administration, extensive training requirements, and the ICU specific clinical factors including severity of illness and mechanical ventilation that renders it difficult to complete DRS-R-98 assessments. This was evident in our patient population where mechanically ventilated patients had a large number of missing DRS-R-98 assessments. We found higher CAM-ICU-7 scores among patients with missing DRS-R-98 data, raising the question of underestimating severe delirium when symptoms cannot be assessed due to the inability to complete DRS-R-98 (Supplemental Digital Content – Figure 1). We also assessed whether deep sedation may artificially inflate delirium severity as measured through the CAM-ICU-7 but found similar distributions of higher delirium severity among both sedated and agitated assessments (Supplemental Digital Content – Figure 3).

Besides DRS-R-98, Delirium Detection Score (DDS),[17] Nursing Delirium Screening Scale (Nu-DESC),[18,19] and Intensive Care Delirium Screening Checklist (ICDSC)[20,21] have been used in critical care settings to assess delirium severity. These scales consist of items depicting various symptoms of delirium, which together form an overall score with higher scores representing higher severity.[17-21] DDS and Nu-DESC do not capture inattention, one of the cardinal features of delirium, whereas DDS has poor sensitivity, making it less desirable as a delirium screening tool.[19] Heavy workflow in the ICU with limited time for evaluation and documentation makes it impractical to use separate scales for assessment of delirium and its severity. The ICDSC captures inattention making it a suitable scale for both delirium identification and severity.[20, 21] ICDSC also evaluates sleep-wake cycle disturbances not evaluated by the CAM-ICU-7. Evaluating additional constructs is an advantage, but the information to generate ICDSC scoring is collected over 24 hours, which could lead to recall bias and over estimation of delirium severity.[22] CAM-ICU administration time of less than one minute[23] allows for more frequent administrations along with direct interaction with patients. This provides a higher reproducibility especially among the mechanically ventilated, as CAM-ICU-7 does not rely on observation alone. Future work comparing CAM-ICU-7 to ICDSC will help clarify which of the two instruments has the greatest utility to measure and follow delirium.

Both ICDSC and Nu-DESC can identify patients with subsyndromal delirium,[18,21] characterized by presence of one or more symptoms of delirium and associated with adverse clinical outcomes.[18,21] A critique of the CAM-ICU is that its dichotomous approach of detecting delirium and absence of ordinal grading of delirium severity symptoms could miss patients with lower delirium severity that may benefit from early interventions. This could potentially be mitigated by use of the CAM-ICU-7 that provides a graded scale for delirium severity assessment. As seen in our study, clinical outcomes vary between mild to moderate delirium and severe delirium. This is in contrast to ICDSC, which plateaus at the threshold of clinical delirium and does not provide further predictive discrimination. As our data were limited to delirious patients only, we were not able to identify subsyndromal delirium. Studies with both delirious and non-delirious patients will be able to clarify assessment of subsyndromal delirium using the CAM-ICU-7.

Limitations: A) Inability of the CAM-ICU-7 to capture the entire symptom spectrum of delirium severity, compromising its construct validity. This should be evaluated in the context of feasibility versus validity. CAM-ICU-7 is easy to implement as it takes the same time as CAM-ICU that has been adopted internationally and is the most widely used delirium assessment scale in the ICU. In addition, we believe that CAM-ICU-7 captures the core cognitive constructs of delirium. B) We compared the CAM-ICU-7 with the DRS-R-98 and not the gold-standard psychiatrist based assessment of delirium or with a validated ICU delirium severity scale such as the ICDSC. Although not validated specifically in the ICU, the DRS-R-98 has been utilized in the critical care setting[24] and is highly reliable and valid.[11] C) CAM-ICU-7 and DRS-R-98 assessments were performed by the same research assistants that could have led to a higher correlation. D) Research assistants performed the DRS-R-98, an instrument originally designed for psychiatrists. E) The timeframe between identifying delirious patients and study enrollment lasted up to 48 hours in some cases that could have resulted in missing highest severity assessments.

Our study has several strengths. We have a large and diverse sample with half of the patients being females and African Americans. Presence of both mechanically ventilated and non-ventilated patients belonging to different age groups provide known-groups validity to the CAM-ICU-7 assessments. Highly trained research assistants performed twice-daily CAM-ICU and DRS-R-98 assessments. Patients were recruited from three different hospitals with different case-mixes. We gave equal severity weight to both the hyper and hypoactive delirium based on RASS assessments. This mitigates the concern of mislabeling an intervention efficacious when it converts hyperactive agitated delirium to hypoactive delirium. This aspect will be beneficial both for clinical monitoring and conducting future research interventions.

Conclusions

The CAM-ICU-7 delirium severity scale is a valid, reliable, and practical delirium severity measure among ICU patients that can be easily calculated and is associated with meaningful clinical outcomes. This practical tool could improve the ability to correlate delirium severity with long-term complications, including cognitive impairment and healthcare resource utilization. Additionally, the CAM-ICU-7 may facilitate evaluation of delirium severity as an outcome of clinical trials attempting to reduce the burden of delirium in the ICU.