‘A quality improvement program to increase nurses’ detection of delirium on an acute medical unit

1. Introduction

Delirium, an acute decline in cognition and attention and disturbance of consciousness and perception,¹ is a common clinical syndrome in hospitalized older adults affecting up to 56% of older medical patients and 61% of older surgical patients; more than half of the cases develop after hospital admission.² Delirium in hospitalized older adults is associated with increased mortality rates, longer lengths of stay, more frequent complications, increased need for nursing surveillance, greater hospital costs, functional decline and higher admission rates to nursing homes.² Despite its prevalence and adverse consequences, delirium is frequently unrecognized by clinicians and is documented in less than 5% of older delirious patients’ medical records.^2–7 Recognizing delirium is difficult, due in part to its fluctuating nature, its clinical overlap with dementia, its multi-factorial etiology, and frequency of the hypoactive form that presents in older adults.² Under recognition has also been attributed to physicians’ and nurses’ under-appreciation of delirium’s clinical consequences, lack of knowledge and lack of standardized or structured cognitive assessment.^2,8–13 Nurses are crucial to early detection and follow-up of delirium because of their frequent contact with the patients. Unfortunately, nurses typically do not receive formal training in standardized cognitive or delirium assessment.^3,11,14–16

There are numerous studies, conducted under rigorously controlled conditions, that demonstrate the validity of bedside assessment tools for delirium assessment in acute and critical care settings.^17,18 Real world application by nurses of a standardized assessment tool for delirium as part of usual practice has been examined more recently in intensive care units^19–28 and less frequently in acute non-critical care units.^29–33 Findings among ICUs studies varied in a) the degree to which nurses used a bedside assessment tool after introduction to the unit, b) nurses’ accuracy of use, and c) nurses’ perceptions of the tool’s usefulness and/or ease of use. A consistent finding was the decrease in sensitivity of the delirium assessment tool when completed by nurses as compared to research personnel. The range of success in nurses’ uptake of a structured delirium assessment as part of usual practice may be explained by type of unit (e.g., medical versus surgical ICU; teaching versus non-teaching institution), type, strength and duration of implementation strategies, choice of bedside assessment tool, and unit and team culture. Only one of the studies involving non-ICU units reported positive acceptance and use by nurses.³³ This was a short pilot study examining staff nurses’ acceptance and use of a structured bedside delirium assessment tool on two medical units in a teaching hospital. The implementation process consisted of active nurse manager support, educational sessions, and integration with work flow processes and documentation forms. Day and associates provide a detailed report of a hospital implementation process, but provided no results of the efforts.²⁹

At the desire of staff nurses on a newly created Acute Care for the Elderly (ACE) unit, we implemented a multi-component quality improvement (QI) project on an acute medical unit with the overall goal of establishing nurses’ use of a structured bedside assessment tool for delirium as part of their usual practice. The multi-component QI strategy included staff nurse participation and decision making, knowledge and skill training, electronic health record (EHR) integration, and ongoing, regular evaluation and feedback. We measured the impact of the QI on nurse frequency of use of the delirium assessment tool, accuracy of nurses’ assessments, frequency of nurse-physician communication, and physicians’ actions associated with nurse communications.

2. Methods

2.1. Design

This was a quality improvement project. The quality improvement protocol was reviewed and approved by the Vanderbilt University Institutional Review Board. Waiver of informed consent was granted. No nurse-specific information was collected and all nurses were aware of the QI project through planning meetings, staff meetings and an information letter. No patient health protected information was collected from medical records. For the subgroup of patients for whom investigators conducted a formal and separate delirium assessment, verbal permission was obtained.

2.2. Setting and context

The study was conducted on a 28-bed general medical unit at Vanderbilt University Hospital, an 815-bed academic medical center in Nashville, TN. This unit was newly designated as an Acute Care for the Elderly (ACE) Unit to provide interdisciplinary care to older adult patients. As a new ACE Unit, no specific ACE protocols were in place at the start of this QI. The unit was not exclusively restricted to admission of geriatric patients. Patients of varying ages and medical and surgical conditions were admitted to the unit based on bed availability. At the time of this QI project, approximately 45% of the patients were 65 years or older.

As an open unit, multiple medical and surgical teams of attending and resident physicians and nurse practitioners rounded daily on their patients. For older adults on the geriatric services, rounds included a geriatrician, geriatric nurse practitioner, clinical pharmacist, social worker and when available, a physical or occupational therapist. Nurse staffing was one registered nurse per 4–5 patients and 2–3 non-licensed nursing personnel per shift. As part of usual nursing care, licensed staff nurses assessed and documented patients’ cognition once each 12-h shift and more frequently as needed. The neurocognitive assessment consisted of the Glasgow Coma Scale,³⁴ orientation, and level of consciousness using drop down menus in the electronic health record. In addition, nurses could write in comments, such as ‘acute confusion’ or ‘acute change in mental status’, within text boxes. Notification of the resident physicians regarding the patient’s cognition was at the discretion of the primary nurse.

2.3. Nurses’ assessment of delirium: quality improvement elements and activities

2.3.1. Planning stage: involving staff nurses

The first ACE protocols nurses desired to address were delirium assessment and communication. Staff nurse acceptance and willingness to test a new assessment screening tool were considered major factors in the success of the project.^35–38

To engage the nurses, we held several staff meetings where various delirium screening tools were presented and discussed, allowing the nurses to decide which standardized assessment tool they would incorporate as part of their usual nursing care and documentation. The delirium assessment tools discussed were the Confusion Assessment Method (CAM)³⁹; the CAM-ICU⁴⁰ which was developed and validated at Vanderbilt University Hospital; and the Nursing Delirium Screening Scale.^30,41 The first two tools are designed to be diagnostic of delirium while the last is designed solely as a screening instrument. After much discussion, the nurses opted for the Nu-DESC based on concerns of time constraints of the nurse on a medical/surgical floor and perceived ease of use.

2.3.2. Nursing delirium screening scale (Nu-DESC)

The Nu-DESC was designed to have sensitivity as the key property and feasible, that is, easily incorporated into nurses’ routine assessments.^30,41 The Nu-DESC consists of five items that reflect a number of the DSM-IV criteria: disorientation and/or misperception, inappropriate behavior (e.g., pulling at tubes), inappropriate communication (nonsensical speech, incoherent, non-communicative), illusions/hallucinations, and psychomotor retardation. The tool developers kept this last item separate from the hyperactive inappropriate behavior to enhance nurses’ recognition of the hypo-alert delirium. Each item has a range of values from 0 (no symptom), 1 (present but mild) to 2 (present and pronounced/intense). Item scores are summed to provide a theoretical range of 0–10. Scores of 2 or greater are considered a positive screen. Against the CAM, the Nu-DESC had 85.7% sensitivity and 86.8% specificity, a correlation of 0.71 and an AUC of 0.92. Repeat studies demonstrated sensitivity of 95–96% and specificity of 79–87%.

2.3.3. Electronic health record

The investigators collaborated with personnel from the Biomedical Informatics Department to develop the Nu-DESC documentation within the parameters of the electronic documentation system, e.g., no line item may have more than 18 characters. The Nu-DESC was placed in the same section of the nurses’ documentation as the other neuromental status evaluations. The EHR documentation was designed to minimize nurse efforts by simple pull down menus and button tabs for entry. The Nu-DESC was automatically summed for the nurse. Last, nurses discussed their concerns that physician residents often seemed indifferent or ignored the nurses’ notifications when patients experienced an acute change in mental status. To standardize nurses’ notification and documentation of the notification, an additional item was added to prompt the nurse to notify the physician whenever a) the Nu-DESC had a score of 2 or greater, b) a worsening score of 2 or more points, and c) an option to allow the nurse to indicate that the resident had already been notified.

2.3.4. Educational intervention

At the beginning of the intervention period all 39 of the unit nurses received a 1-h educational intervention which included a delirium pre-test and education on delirium assessment techniques. The education included symptom recognition of delirium using a web-based video,⁴² instruction on completing the Nu-DESC in the EHR on each shift, practice session using case vignettes,⁴³ and process to follow to implement the ACE unit Delirium Protocol. Bi-weekly rounds (see below) were used as an opportunity to reinforce didactic content and address any questions or concerns of either the standardized assessment and/or use of the EHR.

Internal medicine residents were informed of the project’s aims to enhance nurses’ assessment of delirium on the ACE unit and communication of the patient’s status to the primary physician. They were alerted to the Nu-DESC form and its location in the Adult Nursing Flow sheet section of the EHR. Last, a similar, voluntary informational session was also conducted with medical faculty.

2.3.5. Unit rounds

The geriatrician or one of the advanced practice nurses made twice weekly rounds on varying days throughout the intervention phase. Rounds were conducted within 1 h of the bedside nurse’s assessment rounds (i.e., between 7 and 8 AM or 7–8 PM). The investigators independently assessed and rated patients, randomly selected, on the Nu-DESC during each visit for a total of 10 paired ratings per week. During these rounds, investigators re-educated staff nurses in the event of Nu-DESC discrepancies to maintain ≥90% agreement.

Two of the geriatric experts independently rated 119 patients with the CAM using a structured standardized cognitive assessment of brief interview, Mini-Mental State Exam,⁴³ and the Digit Span test.⁴⁴ Pre-existing cognitive impairment (none, dementia, psychiatric diagnosis (e.g., schizophrenia), neurologic disease (e.g., acute stroke), or mental retardation) was noted.

2.4. Performance measures for the quality improvement project

2.4.1. Data collection

Data were collected for two time periods: for three months (April 1–Jun 30, 2009) pre-implementation (baseline) and for five months during implementation (January 18–June 30, 2010). Additional time was provided in the implementation phase to allow for uptake of the new practice.³⁸ Data were abstracted from medical records of all older adults (≥65 years) admitted to the study unit during the two time periods.

Several measures were used to evaluate the success of the QI project. Nurse frequency of documentation was measured as (number of documented Nu-DESC entries/total number of patient shifts) × 100. In addition, any text documentation specifically of ‘acute mental status change’, ‘delirium’, or ‘acute confusion’ was noted separate from notation of abnormal Glasgow Scale, disorientation, ‘confusion’ or altered level of consciousness. Physician notification was determined by nurse documentation at baseline (text notes) and implementation (Nu-DESC notification). Accuracy of nurses’ assessment was examined by the subgroup assessment between the investigators’ Nu-DESC assessments with the nurses’ Nu-DESC assessments. Investigators conducted their assessments prior to determining the nurses’ assessments.

Physician actions associated with nurse notification was determined as the presence of orders or progress notes of efforts to determine potential underlying etiology of delirium, such as specific orders for blood or urine cultures, imaging tests, blood chemistry tests or pharmacologic treatment and presence of nurses’ documentation of notification within the preceding 24 h. Pharmacologic treatment could be for the underlying etiology (e.g., infection) or for agitated behavior. Registered nurses (RN), who were not part of the ACE staff and blinded to the comparison aim, were trained by the physician investigator using a standardized procedural manual to determine whether physicians’ orders and notations were deemed related to a delirium work up or treatment and presence of nurses’ communication of delirium in the preceding 24 h. If the RN had any question whether the physician’s notes reflected an assessment or treatment for delirium, the medical record was reviewed by the physician investigator.

2.4.2. Accuracy of the Nu-DESC

Last, because the CAM³⁹ is considered the gold standard, an additional measurement was made comparing the Nu-DESC assessment with a CAM assessment. This was determined by the geriatrician and advanced practice nurses during their rounds on the unit. Patients were approached, using a table of random numbers, informed of the quality improvement project, and asked permission whether they would be willing to undergo a separate cognitive assessment. Most agreed.

2.4.3. Data analysis

Data were entered into REDCap, a secure web-based application for building and managing online databases.⁴⁵ Data were downloaded to SPSS v. 18 for analyses. Descriptive statistics were conducted on all variables. Between group comparisons were made using chi square test for categorical variables and Mann Whitney U tests for non-normally distributed continuous variables. Nurse accuracy as compared to investigators was examined using percent agreement. Nu-DESC accuracy as compared to the CAM was examined using sensitivity and specificity measures.

3. Results

3.1. Patient profile

During the baseline pre-implementation phase, there were 187 patients who were 65 years and older with at least one day stay on the unit (Table 1). Patients had a mean age of 78.2 (±8.3). Unit length of stay ranged from 1 to 16 days with a mean of 3.82 and median of 3.0 days. During the implementation phase, 287 patients were admitted to the unit. Patients had a mean age of 77.6 (±7.9) with a median age of 77 years. Length of stay ranged from 1 to 15 days with one outlier of 29 days, mean 3.8 (±2.6), median 3.0 days.

Table 1.

Patient profile pre- and post-implementation of Nu-DESC.

Patient characteristics	Pre-implementation 3 months	Post-implementation 5 months	P- valuea
Patients over 65 years old	n = 187	n = 287
Female	121 (65%)	175 (61%)	.235
Average age	78.2 (±8.3)	77.6 (±7.9)	.392
Average length of stay	3.8 (+2.9) days	3.8 (+2.6) days	.353

^aMann Whitney U tests.

3.2. Nurses’ frequency of documentation

During the pre-implementation phase, at least one neuro-cognitive assessment was documented for 177 of the patients (95%). Of those, an abnormal neurocognitive status was identified in 68 (38%). No notation was made specifically of ‘delirium’, ‘acute confusion’ or ‘acute change in mental status’. Some type of cognitive assessment was documented for 70% of the nursing shifts.

During the post-implementation phase, Nu-DESC was documented for 283 of the 287 patients (99%). Nurses noted a positive Nu-DESC screen for 38 (13%) patients. Notation of any type of cognitive impairment, including the Nu-DESC, was present for 65 (23%) patients. Some type of cognitive assessment was documented for 77% of the nursing shifts.

3.3. Nurse accuracy

Investigator–nurse paired Nu-DESC assessments (n = 111) showed that although there was a 93% agreement for the paired assessments, none of the nurses identified a positive Nu-DESC as compared to 8 positive Nu-DESC ratings by the investigators.

3.4. Physician actions associated with nurse notification

There were 35 (19%) patients with a documented delirium work up or treatment pre-implementation. Of these, 19 (54%) occurred after nurse notification of abnormal cognition; 3 (9%) patients had abnormal cognition noted but not reported; and 13 (37%) had normal cognitive assessments recorded by the nurse. During the post-implementation stage, 31 (11%) patients had a documented delirium work up or treatment. Of these, 28 (90%) occurred after nurse notification, 2 (6%) had an abnormal cognitive assessment but no notification and 1 had normal cognitive assessments recorded by the nurse (Table 2).

Table 2.

Frequency of nurses’ documentation and physician work up for delirium pre- and post-implementation of a quality improvement program.

Quality improvement measures	Pre-implementation 3 months	Post-implementation 5 months	X2	P-value
Shifts with neurocognitive assessment	826/1185 (70%)	1288/1679 (77%)	17.29	P < .001
Patients with neurocognitive assessment	177/187 (95%)	283/287 (99%)	4.87	P = .027
Shifts with documentation of ‘delirium’, ‘acute confusion’, ‘acute change in mental status’	0/177	38/283(13%)	21.04	P < .001
Patients with documentation of any abnormal cognitive assessment	68/177 (38%)	65/283 (23%)	11.91	P < .001
Nurse notified physician of patient’s cognitive status/behavior	21/68 (31%)	31/65 (48%)	12.44	P < .001
Proportion of physician delirium work up or treatment in response to nurse notification	19/35 (54%)	28/31 (90%)	8.73	P = .003

3.5. Accuracy of the Nu-DESC

Investigator assessment for 119 patients (see Table 3) revealed a positive CAM for 7 (6%) patients and a positive Nu-DESC for 10 (8%) patients of which 7 agreed with the positive CAM finding, resulting in a sensitivity of 1.0 (95% CI: 0.56–1.0) and specificity of 0.97 (95% CI: 0.92–0.99).

Table 3.

Experts’ comparison of Nu-DESC with CAM (N = 119).

Nu-DESC	Diagnosed delirium using the confusion assessment method
	Delirium present	Delirium absent
Positive results	7	3
Negative results	0	109

Nu-DESC: Nurses Delirium Screening Tool; CAM: Confusion Assessment Method.

4. Discussion

We demonstrated that a multi-component quality improvement program can result in nurses’ implementation and adoption of a structured bedside delirium assessment tool. Adoption of an additional assessment and documentation form every shift proved feasible as evidenced by the completion rate by patient and by nursing shift. The QI evaluation was not designed to discern which component (staff nurse involvement, electronic health record aligned with current documentation processes, didactic education, regular rounding and reinforcement) led to the success of the adoption. However, most quality improvement frameworks span multiple implementation strategies based on the complexity of changing practice behavior^35,38 and do not seek to test implementation strategies head to head.

Despite the frequency of completion, nurse accuracy of assessment was problematic. A subgroup of paired comparisons of the Nu-DESC between clinical experts and staff nurses showed that nurses could accurately identify the absence of delirium, but had difficulty recognizing the presence of delirium. This was in spite of a mandatory education session, use of video demonstration and case-based educational approach, and ongoing weekly reinforcement with select patients. ICU studies have reported good agreement of nurses’ assessments with expert ratings (>90% of paired ratings) after the implementation of a bedside delirium assessment tool, but several have also reported that despite good agreement, nurses continue to have difficulty in accurately identifying delirium (i.e., low sensitivity).^21,22,27 Rolfson and colleagues reported on nurses’ low accuracy of delirium assessment in a post-operative recovery unit³² and Lemiengre and colleagues³¹ reported similar results from an adult medical unit. Thus, it does not appear that the specialization of nurses plays a role. Potential explanations include background training and skills of physicians versus nurses³²; possibility of short cuts taken by the nursing staff in their assessments⁴⁶; insufficient components in a bedside assessment tool to accurately identify the presence of delirium, such as the capture of fluctuating events or inattention⁴⁶; the need for a more intense exam in covert cases (e.g., hypoactive form)^10,47; and inadequate training and implementation strategies.^10,46,47 Our education and training approach was a combination of didactic, demonstration and hands on experiences. Others have also reported on educational strategies needed to change clinician practice that require didactic sessions but also capitalize on interactive sessions.⁴⁸ Further research is needed to determine the most effective ways to introduce new knowledge and practices into the practice setting. Finally, our program used the Nu-DESC which does not include the feature ‘inattention’; this may have contributed to nurses’ underestimation of delirium. However, when skilled clinicians compared the Nu-DESC to the CAM, Nu-DESC overestimated the presence of delirium, a finding similar to the original development and validation studies.^30,41

Early recognition of delirium is crucial in the treatment of hospitalized elders. Efficient and effective education of nurses on recognizing delirium in hospitalized elders is important as is the communication of the detection to physicians. During the planning phase, our nurses voiced concerns of lack of physician response. Soja and colleagues²⁶ similarly reported on nurses’ perception of physician indifference to notification of patient delirium. We found, however, that a standardized process for reporting positive screens resulted in a significantly higher association of nurse notification with subsequent physician work up and treatment of delirium. Prior to the implementation, nurses indicated normal cognitive assessments for 37% of the patients who received delirium work up or treatment. After implementation, this only occurred in 6% of the patients who received work up or treatment. Thus, although nurses had lower rates of detection versus clinical experts, they did improve either their ability to discern more subtle signs or their awareness of the importance of documenting these signs.

There are several limitations to note. First, the Nu-DESC does not assess for “inattention”, a required feature of delirium.¹ Second, Nu-DESC is a screening tool, not a diagnostic tool as evidenced by the overestimation of skilled clinicians’ identification of delirium. Nevertheless, we found a standardized approach to nurses’ assessment specifically for potential delirium resulted in more targeted assessments and documentation of the condition in the EHR. There was an increase of nurse notification prior to physician work up and treatment of delirium. Further study could determine whether increased nurse screening and communication of results to physicians ultimately reduce the duration of delirium, complications of delirium, and extended care needs and expenses.