Serial Administration of a Modified Richmond Agitation and Sedation Scale for Delirium Screening

Abstract

Objectives

Because delirium is common and frequently unrecognized, this study sought to design a brief screening tool for a core feature of mental status and to validate the instrument as a serial assessment for delirium.

Design

Prospective cohort

Setting

Tertiary VA Hospital in New England

Participants

100 Veterans admitted to the medical service

Methods

A consensus panel developed a modified version of the Richmond Agitation and Sedation Scale (RASS) to capture alterations in consciousness. Upon admission and daily thereafter, patients were screened with the modified RASS and independently, underwent a comprehensive mental status interview by a geriatric expert, who determined if the criteria for delirium were met. The sensitivity, specificity, and positive likelihood ratio (LR) of the modified RASS for delirium are reported.

Results

As a single assessment, the modified RASS had a sensitivity of 64% and a specificity of 93% for delirium (LR=9.4). When used to detect change, serial modified RASS assessments had a sensitivity of 74% and a specificity of 92% (LR=8.9) in both prevalent and incident delirium. When prevalent cases were excluded, any change in the modified RASS had a sensitivity of 85% and a specificity of 92% for incident delirium (LR=10.2)

Conclusion

When administered daily, the modified RASS has good sensitivity and specificity for incident delirium. Given the brevity of the instrument (approximately 15 seconds), consideration should be given to incorporating the modified RASS as a daily screening measure for consciousness and delirium.

Introduction

A vital sign is an objective, non-invasive, easily reproducible measurement of a vital function.¹ As indicators of vital function, vital signs are a fundamental component of the physical exam and serve key diagnostic and monitoring purposes for hospitalized patients. The brain is as vital to life as the cardiovascular system (blood pressure, pulse), the respiratory system (respiratory rate), and the immune/ thermoregulatory system (temperature), yet currently no vital sign exists that would allow rapid, reliable, and easily reproducible assessment of cognition.² As a result, acute mental status changes due to delirium frequently go undetected and untreated.^4–5 Given this lack of critical clinical information, perhaps it is time to consider adoption of a clinical tool that would systematically and reliably measure mental status — a mental status vital sign.

Mental status is a broad term that includes thought content, cognitive processing, attention, memory, language, and executive functioning.⁶ Delirium is defined as an acute change in attention with fluctuations in cognition, thought, and/or consciousness throughout the course of the day.⁷ Because delirium in older patients is common and is associated with increased morbidity, mortality, functional decline, and costs,^{8, 9, 11}, development and validation of a rapid, objective screening assessment could be utilized by nursing staff to identify patients at high-risk for delirium.

Current recommendations for inpatient monitoring for delirium, usually involve daily cognitive screening with a standardized screening instrument, such as the Mini Mental Status Exam, followed by completion of a diagnostic algorithm, such as the Confusion Assessment Method (CAM).⁸ This combination provides a good screening assessment of acute mental status, but is time-consuming to implement (8–12 minutes). Thus, most patients do not get routine screening. To facilitate clinical implementation, we focused on developing a brief (30 seconds or less) inpatient screening measure of a feature of mental status which could be administered serially. The purpose of this study was to a) develop a brief screening tool for a core feature of mental status and b) to validate this screening tool for delirium in an older inpatient population.

Methods

Consensus Panel

In June 2009, the Veterans Administration sponsored a conference entitled “Identifying and Treating Delirium” which included sessions intended to solicit input from interdisciplinary participants on identifying the most targetable components of delirium and discussing potential clinical instruments. This information was considered by the members of the consensus panel of 10 representatives from internal medicine, geriatrics, nursing, psychiatry, and psychology. A modified Delphi method was utilized to target characteristic features of delirium and identify instruments that could best capture mental status change. While the group came to consensus that inattention was the core cognitive feature of delirium, capturing the acute onset and fluctuating nature of delirium was better suited as a vital sign. To meet these criteria, the group modified the Richmond Agitation Sedation Scale (RASS).^{13, 14}

The RASS is an observational instrument that has been validated in the ICU setting for objectively determining level of sedation. It has been shown to be highly reliable and associated with delirium.^14–15 The RASS is a quick, objective scale of consciousness with a scoring system that captures both hyperactive and hypoactive levels of consciousness. A disadvantage of using the RASS includes its limited attention assessment. The Consensus panel modified the RASS to improve its assessment of attention, using a brief open-ended question that was asked prior to scoring (”Describe how you are feeling today”) (Figure 1).

Figure 1.

Modified Richmond Agitation and Sedation Scale

Validation Study

Participants

For this prospective validation study, we recruited 100 medical patients 65 and older admitted to a tertiary VA hospital. Participants provided written informed consent. Patients were excluded if they did not meet age requirements, if they reported that they were expecting to leave the hospital within one day, or if they were experiencing visual impairment that would prevent their ability to complete informed consent forms and cognitive screening tools. Patients unable to complete the informed consent process due to cognitive dysfunction were excluded.

Mental Status Assessments

Following enrollment, three study staff members visited each participant independently. First the trained research assistant obtained informed consent, completed demographic, cognitive, and functional assessments. The Mini-Mental State Exam was administered to provide a baseline measure of cognitive function at the time of admission.²¹ A nurse-interviewer later administered the modified RASS separately. Finally, a delirium expert performed an independent comprehensive 30–45 minute mental status interview including assessments of attention, executive function, memory, and mood. Delirium was diagnosed by the delirium expert according to DSM-IV criteria.⁷ Each investigator was blinded to the others’ ratings. Following the initial assessments, each participant was visited daily throughout the hospitalization by a modified RASS assessor and, independently, by the delirium expert.

To determine inter-rater reliability, sixty participants were evaluated with the modified RASS by the trained research assistant and a nurse-interviewer simultaneously. The modified RASS was scored independently and the assessors were blinded to each others’ ratings.

Statistics

The paired modified RASS-delirium assessments were analyzed in three ways: a) as single-day independent assessments; b) longitudinally as a change from baseline including prevalent delirium; and c) longitudinally as a change from baseline, excluding prevalent delirium cases. We examined a 1-point change on the modified RASS and a 2-point change on the modified RASS from baseline. These dichotomous cut points allowed us to examine the impact on the comparison statistics to determine the most appropriate cut point for clinical use. The expert’s diagnosis was considered the reference standard for delirium. Sensitivity, specificity, positive and negative predictive values, likelihood ratios, and inter-rater reliability were calculcated.

Results

Characteristics of the study population are presented in Table 1. Because this was a VA population, the vast majority (94%) of participants were male with mean age 80.8 years (range 66–96 years) and 90% were white. This population had a high Charlson Comorbidity Index (mean 3.9 ±2.4), mean AUDIT of 2.5 (±3.1), and 80% were prior smokers (8% current). These risk factors were reflected in functional assessments, with 35% reporting difficulty with at least one ADL and 55% reporting difficulty with at least one IADL. Despite the age and high comorbidity of the population, delirium prevalence was low 10% (n=10) and incidence was 14% (n=13). Inter-rater reliability of the modified RASS between the research assistant and the nurse practitioner yielded 98% agreement with a weighted kappa of 0.48 (p<.001).

Table 1.

Baseline Characteristics of the Population (n=100)

Characteristic	Mean (SD) n (%)
Age, years	80.8 (7.4)
Male Sex	94 (94%)
Race, White	90 (90%)
Charlson Comorbidity Index	3.9 (2.4)
Body Mass Index, kg/m2	27.4 (6.2)
Mini Mental State Examination	24.6 (4.0)
AUDIT	2.5 (3.1)
Tobacco Use, pack-years	54 (56)
Current	8 (8%)
Never	17 (17%)
Prior	72 (72%)
Functional Impairment
ADL	35 (35%)
IADL	55 (55%)

AUDIT – Alcohol Use Disorders Identification Test; ADL – Activities of Daily Living; IADL – Independent Activities of Daily Living

When the modified RASS was analyzed as a single day independent assessment, any abnormal score (i.e. a score ≠ 0) had a sensitivity of 64% and a specificity of 93% for delirium relative to the expert evaluation (Table 2). Using the sensitivity and specificity, we calculated a likelihood ratio (LR+) of 9.4 and a positive predictive value of 70%. When we defined an abnormal modified RASS as ≥2 or ≤-2, the sensitivity fell to 34% while the specificity increased to 99.6%. In this scenario the LR+ increased to 86 and the positive predictive value increased to 96% while the negative predictive value decreased to 86%.

Table 2.

Criteria	Modified RASS	Sensitivity	Specificity	LR+	LR−
Single-Day Independent Assessments
	Any Abnormal	63.9% 51.9, 76.0	93.2% 90.3, 96.4	9.4	0.4
	RASS ≥2 or ≤-2	34.4% 22.5, 46.3	99.6% 98.8, 100	86	0.7
Longitudinal Assessments
Any delirium	Any Change	73.9% 56.0, 91.9	91.7% 85.3, 98.1	8.9	0.3
	Change 2 points	21.7%	100%		0.8
Incident Delirium	Any Change	84.6% 65.0, 100.0	91.7% 85.3, 98.1	10.2	0.2
	Change 2 points	23.1%	100%		0.8

LR+ – postitive likelihood ratio; LR− – negative likelihood ratio; NPV – negative predictive value; PPV – positive predictive value; RASS – Richmond Agitation and Sedation Scale

When the modified RASS was used longitudinally to detect change in delirium during the hospital stay among all participants, the modified RASS had a sensitivity of 72% and specificity of 92% for any change. Increasing the stringency of the criteria by looking at a change of 2 or more modified RASS points, lowered the sensitivity (22%) and increased the specificity (100%).

To capture the potential of the modified RASS administered on a longitudinal basis as a diagnostic aid, the prevalent cases of delirium were excluded. In this analysis, any change in the modified RASS had a sensitivity of 85% and a specificity of 92% for incident delirium. With more stringent criteria of a change of 2 points, the sensitivity was 23% and the specificity was 100%.

Discussion

In this study, we developed a modification of the RASS for serial mental status assessment. While a single measurement of the modified RASS has modest sensitivity and good specificity for delirium, longitudinal measurement increased the sensitivity with no loss in specificity. Importantly, the 15 seconds (or less) required for the modified RASS could be incorporated into daily workflow and provides an objective measure of consciousness. As such, we believe the modified RASS can potentially serve as a longitudinal measure of consciousness – much like a vital sign for mental status.

In delirium, altered consciousness is a clinical and diagnostic feature of delirium.^{22, 23} Furthermore, fluctuation in mental status is a diagnostic feature of delirium. As such, a screening instrument able to quantify the level of consciousness longitudinally and allow comparison to prior and subsequent determinations has face validity as a delirium screening instrument..

The modified RASS has other features that make it appropriate for serial measurement in a manner similar to a vital sign. First, it objectively described consciousness on a scale, which is an improvement relative to many of the subjective descriptions clinicians often use. Consistent with other studies of the RASS^{13, 14}, the modified RASS has good inter-rater reliability, meaning that healthcare team members will get similar readings. These factors allow a common language to be used to describe level of consciousness across healthcare settings that can become the basis for a systematic and standardized monitor of cognitive change, improving continuity of care and communication between providers. It can be further used to objectively establish a patient’s baseline and monitor change longitudinally.

The current study is limited by the lack of diversity and small size of the study population, which limits external validity (generalizability). Additional studies evaluating the utility of the modified RASS by a variety of healthcare team members in a larger, more ethnically, racially diverse, and heterogeneous population should be completed before we can determine if it can perform as a mental status vital sign, and if it is associated with better patient outcomes. Additionally, this study selected patients who were physically and cognitively capable of enrolling. Patients who were significantly cognitively impaired were unable to provide consent to participate. Likewise, those with severe sensory or physical deficit (i.e. blindness or severe tremor) that prevented their completion of the requisite forms were also excluded. Thus, some of the sickest, frailest, and most cognitively impaired patients were excluded. Unfortunately, this study therefore excluded a population significantly more vulnerable to the development of delirium.

Because a change in mental status (delirium) is common, morbid, and costly, a brief tool that can reliably and effectively assess mental status is needed. The modified RASS used in this study provides an objective measurement of consciousness, a key component of mental status, and was demonstrated to reliably screen for presence or absence of delirium when administered longitudinally. Further study in diverse populations with administration by a variety of healthcare team members is needed to determine if the modified RASS can accurately serve as a mental status vital sign. If adopted widely, the modified RASS could be used alongside the traditional vital signs to establish patient baselines, monitor change, improve provider communication, and potentially improve patient outcomes.