Development and Validation of an ICU Delirium Playbook for Provider Education

OBJECTIVES:

Although delirium detection and prevention practices are recommended in critical care guidelines, there remains a persistent lack of effective delirium education for ICU providers. To address this knowledge-practice gap, we developed an “ICU Delirium Playbook” to educate providers on delirium detection (using the Confusion Assessment Method for the ICU) and prevention.

DESIGN:

Building on our previous ICU Delirium Video Series, our interdisciplinary team developed a corresponding quiz to form a digital “ICU Delirium Playbook.” Playbook content validity was evaluated by delirium experts, and face validity by an ICU nurse focus group. Additionally, focus group participants completed the quiz before and after video viewing. Remaining focus group concerns were evaluated in semi-structured follow-up interviews.

SETTING:

Online validation survey, virtual focus group, and virtual interviews.

SUBJECTS:

The validation group included six delirium experts in the fields of critical care, geriatrics, nursing, and ICU education. The face validation group included nine ICU nurses, three of whom participated in the semi-structured feedback interviews.

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

The 44-question quiz had excellent content validity (average scale-level content validity index [S-CVI] of individual items = 0.99, universal agreement S-CVI = 0.93, agreement κ ≥ 0.75, and clarity p ≥ 0.8). The focus group participants completed the Playbook in an average (sd) time of 53 (14) minutes, demonstrating significant improvements in pre-post quiz scores (74% vs 86%; p = 0.0009). Verbal feedback highlighted the conciseness, utility, and relevance of the Playbook, with all participants agreeing to deploy the digital education module in their ICUs.

CONCLUSIONS:

The ICU Delirium Playbook is a novel, first-of-its-kind asynchronous digital education tool aimed to standardize delirium detection and prevention practices. After a rigorous content and face validation process, the Playbook is now available for widespread use.

KEY POINTS

Question: What is the content and face validity of a novel ICU Delirium Playbook digital education module for critical care providers?

Findings: A panel of delirium experts found that the Playbook had excellent content validity and clarity. A subsequent focus group of ICU nurses completing the Playbook yielded significant improvements in pre-post quiz scores (74% vs 86%), with positive feedback regarding the module’s conciseness, utility, and relevance.

Meaning: The ICU Delirium Playbook is a novel digital educational module for delirium detection and prevention and is now available for widespread use.

Delirium is a syndrome of “acute brain failure” affecting up to 50% of all ICU patients and 80% of those receiving mechanical ventilation (1, 2). A devastating syndrome, delirium is independently associated with various adverse consequences, including short- and long-term cognitive, physical, and mental health impairments as well as longer ICU and hospital length of stay (3–9).

Given the prevalence and consequences of delirium, the 2018 Society of Critical Care Medicine Clinical Practice Guidelines for Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption recommend screening all ICU patients daily using validated tools such as the Confusion Assessment Method for the ICU (CAM-ICU), and addressing modifiable risk factors (e.g., benzodiazepine infusions) using multicomponent strategies (2, 10). Although the CAM-ICU was validated more than 20 years ago, requires no equipment and takes less than 2 minutes to perform, many ICU providers struggle to consistently and correctly detect delirium, thus hindering prevention practices. Specifically, CAM-ICU completion rates can be as low 38%, with up to 30% of documented assessments inappropriately scored “unable-to-assess” despite a patient being noncomatose (10–12). Delirium assessments can be particularly challenging in the setting of hypoactivity or neurologic diagnoses (e.g., dementia, stroke), heightening the chances of incorrect documentation and under-detection (13, 14).

Although consistent delirium detection and prevention practices can benefit both patients (e.g., improved outcomes) and providers (e.g., lower burden), the majority of bedside providers report receiving insufficient delirium education (15–21). Prior efforts attempted to fill this knowledge gap via one-time lectures and webinars, intensive workshops, bedside simulations, paper-based self-learning, and e-learning modules, but were often limited in scale and sustainability, and/or used nonvalidated knowledge assessment tools (22–30). To address this issue, our interdisciplinary team built on our prior ICU Delirium Video Series by developing and validating a corresponding quiz, thus forming the first-of-its-kind “ICU Delirium Playbook” as described in this article.

METHODS

ICU Delirium Playbook Development

Motivated by consistent ICU nurse reports regarding a lack of prior and ongoing delirium education, our interdisciplinary team (of nurse educators, physicians, bedside nurses, implementation scientists, and research staff) sought to fill this gap by developing an ICU Delirium Video Series. Notably, this seven-video series included teaching on the indications for delirium screening and details regarding the CAM-ICU assessment such as demonstrations and considerations (e.g., assessing non-English-speaking patients and those with neurologic deficits). This series was later pilot tested in a group of 20 critical care nurses, who provided important feedback and confirmation of knowledge acquisition via completion of a publicly available CAM-ICU quiz (31). Furthermore, the pilot feedback motivated video refinements and the addition of an eighth video emphasizing the benefits of detection and interdisciplinary prevention strategies including sedation minimization and family engagement (2, 31).

Following the pilot group, the interdisciplinary team identified a need to develop and validate a quiz that reflected the content of the updated ICU Delirium Video Series. Using the eight videos, the team outlined one to four learning objectives within each of the seven content domains (Table 1), and leveraged established guidelines and methodologies to draft a multiple-choice quiz, taking care to include plausible distractors, minimize misleading stems, and avoid complex answer choices (17, 32, 33) (Fig. 1). The videos and quiz were combined in Qualtrics (Provo, UT), forming the first version of the “ICU Delirium Playbook” (www.icudeliriumplaybook.com).

TABLE 1.

ICU Delirium Playbook Content Domains and Learning Objectives

Content Domain	Order No.	Length (min:s)	Learning Objectives
CAM-ICU Feature 1: Acute change in mental status	1	03:35	Describe sources of information that may identify whether the patient’s mental status has altered from baseline in the past 24 hr (e.g., patient “handoff,” family, electronic medical record)
			Evaluate whether Feature 1 (acute change or fluctuating course of mental status) is positive/present or negative/absent
			Know that proceeding with the CAM-ICU is necessary if Feature 1 (acute change or fluctuating course of mental status) is positive/present
CAM-ICU Feature 3: Altered level of consciousness (RASS)	2a	04:59	Use the RASS correctly
			Understand that the RASS can be used to evaluate all critically ill patients (e.g., mechanically ventilated and/or sedated)
			Evaluate Feature 3 (altered level of consciousness) as either positive/present or negative/absent based on the RASS score
			Know to proceed with the CAM-ICU if Feature 3 (altered level of consciousness) is positive/present
CAM-ICU Feature 2: Inattention	3a	03:41	Perform the S-A-V-E-A-H-A-A-R-T assessmentb correctly
			Evaluate Feature 2 (Inattention) as either positive/present or negative/absent based on how many errors the patient makes
			Recognize when adaptations to Feature 2 (inattention) are needed
			Know to proceed with the CAM-ICU if Feature 1 (acute change or fluctuating course of mental status) and Feature 2 are both positive/present
CAM-ICU Feature 4: Disorganized thinking	4	03:58	Perform the “Fingers” command and Yes/No questions assessment correctly
			Evaluate whether Feature 4 (disorganized thinking) is positive/present or negative/absent based on the “Fingers” command and the Yes/No questions assessment
			Recognize when adaptations to Feature 4 (disorganized thinking) are needed
			Know how to proceed with the CAM-ICU if Feature 4 (disorganized thinking) is positive/present
Delirium and the CAM-ICU: Pearls and pitfalls	5	01:57	Know when to document the CAM-ICU as “unable to assess”
			Explain indications and contraindications for performing regular delirium assessments on a patient
			List online resources for CAM-ICU translations and modifications
Performing the CAM-ICU: Examples numbers 1 and 2	6	07:53c	Demonstrate assessment competency with the CAM-ICU based on assessments of changes in mental status (Feature 1), inattention (Feature 2), level of consciousness (Feature 3), and disorganized thinking (Feature 4) (if needed)
CAM-ICU positive: My patient is delirious, now what?	7	08:43	Explain the importance of routinely screening and monitoring patients for delirium
			Identify interventions (e.g., sedation minimization) to manage and prevent delirium
			Recognize patients eligible for early mobilization interventions
			Understand the importance of implementing environmental and non-pharmacological interventions to synchronize sleep/wake rhythms

CAM-ICU = Confusion Assessment Method for the ICU, RASS = Richmond Agitation-Sedation Scale.

^aFeature 3 precedes Feature 2 as required physical contact for Feature 2 can invalidate verbal responsiveness evaluation portion of Feature 3.

^bInvolves reading a series of 10 letters (e.g., S-A-V-E-A-H-A-A-R-T or C-A-S-A-B-L-A-N-C-A) and having the patient squeeze the assessor's hand when hearing the letter “A.”

^cIncludes two videos.

Figure 1.

Flowchart of ICU Delirium Playbook development. ^aRefers to the original ICU Delirium Video Series (31). ^bOne item removed based on expert comments. ^cAll items removed based on expert comments. CAM-ICU = Confusion Assessment Method for the ICU, v = version.

Content Validation

Delirium Expert Panel and Validation Survey

With guidance from an experienced nurse scientist (J.E.D.), our team followed an established content validation methodology to conduct an expert Playbook review using a Qualtrics survey (34). To recruit experts in delirium detection and prevention education, our Playbook team received recommendations from leaders within the American Delirium Society. Several experts were contacted via email, and the first six who agreed were selected to participate. All experts received a small token of appreciation for participation.

First and second survey hyperlinks were distributed via email (35, 36). The survey prompted delirium experts to view each domain’s video content and evaluate corresponding quiz items (questions), rating each for 1) relevance (Likert scale; 1 = “Not Relevant” to 4 = “Highly Relevant”), 2) clarity (yes/no), and 3) importance (Likert scale; 1 = “Not Essential” to 3 = “Essential”) (32), along with 4) domain and Playbook content comprehensiveness (yes/no) (32, 35–38). The first round of quiz content validation included relevance, clarity, and importance ratings on multiple-choice items as well as comprehensiveness ratings on domains and the entire Playbook. The second round included relevance, clarity, and importance ratings only on edited or new items. In both rounds, expert ratings and comments informed revisions to videos and quiz items, along with item elimination and creation.

Content Validity and Psychometric Properties

Seven properties were evaluated from the validation survey responses: 1) the item-content validity index (I-CVI), equaling the proportion of experts rating an item as “moderately relevant” or “very relevant,” with an I-CVI score of greater than or equal to 0.8 indicating item relevancy, between 0.70 and 0.79 as an item needing revision, and less than or equal to 0.69 as an item being eliminated (32, 34, 38, 39); 2) the modified kappa statistic (κ) to account for change agreements on I-CVIs, with greater than or equal to 0.75 indicating excellent agreement, between 0.6 and 0.74 good, 0.4 and 0.59 fair, and less than or equal to 0.39 poor (32); 3) the scale-level CVI by average (S-CVI/Ave; the average of all I-CVIs) and 4) the scale-level CVI by universal agreement (S-CVI/UA; the proportion of I-CVIs equaling 1), with S-CVI/Ave greater than or equal to 0.9 and S-CVI/UA greater than or equal to 0.8 indicating excellent content validity (32, 40); 5) a proportion (P) of expert agreement on clarity (yes) ratings, with greater than or equal to 0.8 interpreted as appropriate and less than or equal to 0.79 as an item needing major revision (37); 6) the content validity ratio (CVR), a measure of agreement in expert importance ratings, with less than or equal to 0.99 indicating item nonessentialness (CVR not used to inform item revisions or eliminations in this study) (32, 41) and 7) a proportion (P) of expert agreement on comprehensiveness (yes) ratings, with less than or equal to 0.79 indicating a need for additional quiz items to cover a domain’s content (32).

Face Validation

As critical care nurses are the first-line providers in delirium detection and prevention, we ensured the Playbook was designed to meet the qualifications and educational needs of this primary end-user audience. Via an “ICU Delirium Playbook” focus group teleconference, critical care nurse participants entered individual breakout rooms to evaluate the Playbook. First, participants received a hyperlink and were instructed to complete the new content-validated quiz. Subsequently, participants reentered the main teleconference room where they received a hyperlink for the complete Playbook, which included the videos and identical quiz. In real-time, completed pre- and post-video quizzes were evaluated, with pre-video items flagged if distractors were chosen by greater than 50% of participants and post-video items flagged if answered correctly by less than 80% of participants (42, 43).

Following Playbook completion, participants reconvened and engaged in an open-ended group debrief, which included evaluation of flagged items. Each flagged item was examined for issues such as inadequate supporting education or unclear question wording; participants were then invited to propose revisions and provide verbal feedback regarding the entire Playbook (32). Following the focus group, participants were invited to complete a voluntary survey, which included a post-Playbook delirium detection and prevention competency self-evaluation. As the focus group was limited to 2 hours based on participant availability, any missed flagged items were later evaluated in semi-structured interviews with focus group participants.

Analysis

Categorical variables were summarized as proportions and continuous variables as means and sds. After verifying normality with a Shapiro-Wilk test, completion times were reported as means with sds, and pre-post quiz score changes were evaluated using a paired two-sided Student t test. Verbal feedback was analyzed using Braun and Clarke’s thematic analysis approach (44). All analyses completed using Stata version 17.0 (StataCorp, College Station, TX), with p value of less than 0.05 representing statistical significance. As this effort involved standard-of-care bedside practice education, it was certified as not qualifying as human subjects research by the UC San Diego Institutional Review Board (IRB) and therefore not requiring IRB review (IRB No. 201442QI; August 24, 2020). All delirium experts and focus group nurses assented to participation before initiating their modules.

RESULTS

Content Validation Round 1

The videos and initial 39-item quiz were evaluated by a panel of six delirium detection and prevention experts, which included four ICU nurse researchers/educators, one critical care physician, and one geriatrician from five U.S. academic hospitals. Thirty-eight of 39 items (97%) had appropriate relevance (I-CVI ≥ 0.8) and excellent agreement (κ ≥ 0.75), with the entire instrument demonstrating strong content validity (S-CVI/Ave = 0.97 and S-CVI/UA = 0.87 [≥ 0.9 and ≥ 0.8, respectively]). Additionally, 34 of 39 items (89%) had appropriate clarity (p ≥ 0.8), thus requiring minor or zero revisions. Therefore, a single irrelevant item (I-CVI ≤ 0.69) was eliminated, and five unclear items (p ≤ 0.79) underwent revision (Table 2; and Supplemental Table 1, http://links.lww.com/CCX/B215).

TABLE 2.

Content Validation of ICU Delirium Playbook Quiz (n = 6 Experts)

Content-Domain	Items Reviewed	n (%)
		Eliminated	Revised	No Edits
First round validation (39-item quiz)
Feature 1: Acute change in mental status	3	0 (0)	1 (33)	2 (67)
Feature 2: Inattention	6	1 (17)a	1 (17)	4 (66)
Feature 3: Altered level of consciousness (RASS)	6	0 (0)	1 (17)	5 (83)
Feature 4: Disorganized thinking	6	1 (17)	1 (17)	4 (66)
Delirium and the CAM-ICU: Pearls and pitfalls	3	0 (0)	0 (0)	3 (100)
Performing the CAM-ICU: Examples 1 and 2	8	0 (0)	0 (0)	8 (100)
CAM-ICU positive: My patient is delirious, now what?	7	0 (0)	0 (0)	7 (100)
First round total	39	2 (5)	4 (10)	33 (85)
Second round validation (four revised and 11 new items)b
Feature 1: Acute change in mental status	5	0 (0)	0 (0)	5 (100)
Feature 2: Inattention	1	0 (0)	0 (0)	1 (100)
Feature 3: Altered level of consciousness (RASS)	5	0 (0)	0 (0)	5 (100)
Feature 4: Disorganized thinking	2	0 (0)	0 (0)	2 (100)
CAM-ICU positive: My Patient is delirious, now what?	2	0 (0)	0 (0)	2 (100)
Second round total	15	0 (0)	0 (0)	15 (100)
Final “ICU Delirium Playbook” quiz	48	4 (8)a	0 (0)	44 (92)

CAM-ICU = Confusion Assessment Method for the ICU, RASS = Richmond Agitation-Sedation Scale.

^aQuestion eliminated based on expert comments rather than relevance or clarity.

^bZero items required reviewing during this round in the “Delirium and the CAM-ICU: Pearls and Pitfalls” and “Performing the CAM-ICU: Examples 1 and 2.”

Two of seven (29%) Playbook content domains did not meet criteria for appropriate comprehensiveness (p ≥ 0.8), prompting addition of four quiz items each to the “CAM-ICU Feature 1” and “CAM-ICU Feature 3” domains. Overall quiz comprehensiveness was p = 0.67, prompting addition of one and two items to the “CAM-ICU Feature 4” and “CAM-ICU Positive” domains, respectively. Overall, while 33 of 39 items (85%) had appropriate relevance and clarity, one irrelevant item was eliminated, one unclear item was eliminated based on expert comments, and 11 items were added for comprehensiveness, yielding 48 items (Table 2; and Supplemental Table 1, http://links.lww.com/CCX/B215).

Content Validation Round 2

All 15 items (100%) evaluated met criteria for relevancy (I-CVI ≥ 0.8), agreement (κ ≥ 0.75), and clarity (p ≥ 0.8), thus requiring minor or zero revisions (Table 2; and Supplemental Table 1, http://links.lww.com/CCX/B215). While all 48 items (100%) had appropriate relevance and clarity, four items were eliminated based on expert comments, yielding 44 questions with excellent content validity (S-CVI/Ave = 0.99 and S-CVI/UA = 0.93).

Face Validation

Nine critical care nurses from four academic hospitals were recruited. This group included bedside nurses (n = 5), clinical nurse specialists (n = 1), nurse scientists (n = 1), and nurse practitioners (n = 2) who worked primarily in medical-surgical (n = 5), neurologic (n = 2), and cardiothoracic surgery/cardiovascular (n = 2) ICUs. Six nurses had over 5 years of bedside ICU experience, three had no formal delirium detection and prevention training, and one reported English as a second language.

Among four pre- (distractor chosen by > 50%) and 12 post-video items (under 80% correct) deemed concerning, the focus group felt edits were required for zero pre-and 5 post-video items. In follow-up interviews, the remaining seven post-video items were assessed, with only one requiring edits. No items were removed or added to the 44-item quiz.

On the pre- and post-video quizzes, the mean (sd) scores were 74% (12%) and 86% (9%; pre-post difference p = 0.0009), with significant improvements observed on the “CAM-ICU Feature 4” and “Performing the CAM-ICU” domains (Table 3). The ICU Delirium Playbook (eight videos and 44-item quiz) required an average (sd) of 53 (14) minutes to complete, with all participants requiring less than 70 minutes.

TABLE 3.

ICU Delirium Playbook Quiz Performance (n = 8 Nurses)^a

Content Domain	Questions, n	Mean % Correct (sd)		Difference, % (sd)	p b
		Before Video	After Video
Feature 1: Acute change in mental status	6	79 (15)	85 (17)	6 (9)	0.08
Feature 2: Inattention	5	63 (23)	70 (24)	8 (28)	0.5
Feature 3: Altered level of consciousness (Richmond Agitation-Sedation Scale)	9	76 (21)	86 (18)	10 (26)	0.3
Feature 4: Disorganized thinking	6	52 (29)	81 (11)	29 (29)	0.03
Delirium and the CAM-ICU: Pearls and pitfalls	2	44 (32)	81 (26)	38 (44)	0.05
Performing the CAM-ICU: Examples numbers 1 and 2	8	75 (22)	92 (13)	17 (16)	0.02
CAM-ICU positive: My patient is delirious, now what?	8	97 (6)	97 (6)	0 (7)	1
Entire examination	44	74 (12)	86 (9)	13 (6)	0.0009

CAM-ICU = Confusion Assessment Method for the ICU.

^aWhile the focus group included nine ICU nurses, quiz scores include data from only eight nurses, as one experienced technical difficulties during the Qualtrics quiz survey.

^bCalculated using paired Student t test.

Compared to before completing the Playbook, six of eight (75%) focus group participants reported feeling more comfortable performing the CAM-ICU on patients with stroke or dementia and managing critically ill patients with delirium (Table 4).

TABLE 4.

Self-Reported Competency After Completing the ICU Delirium Playbook (n = 8 Nurses)^a

Rating	n (%)
Ability to perform Confusion Assessment Method for the ICU on patients who are:
Critically ill
Much more comfortable than before	4 (50)
Somewhat more comfortable as before	2 (25)
As comfortable as before	2 (25)
Moderately to deeply sedated
Much more comfortable than before	5 (63)
Somewhat more comfortable as before	2 (25)
As comfortable as before	1 (12)
Ventilated and semi-awake
Much more comfortable than before	5 (63)
Somewhat more comfortable as before	1 (12)
As comfortable as before	2 (25)
Encephalopathic
Much more comfortable than before	4 (50)
Somewhat more comfortable as before	1 (12)
As comfortable as before	3 (38)
History of stroke
Much more comfortable than before	5 (63)
Somewhat more comfortable as before	1 (12)
As comfortable as before	2 (25)
History of dementia
Much more comfortable than before	5 (63)
Somewhat more comfortable as before	1 (12)
As comfortable as before	2 (25)
Ability to manage critically ill patients with delirium
Much more comfortable than before	5 (63)
Somewhat more comfortable as before	1 (12)
As comfortable as before	2 (25)

^aWhile the focus group included nine ICU nurses, competency ratings include data from only eight nurses, as one experienced technical difficulties with the Qualtrics survey.

Finally, verbal feedback from nine focus group participants highlighted three major themes: length and structure (e.g., “short” and “videos weren’t too long”); utility (e.g., “I was able to get what I needed from the videos”); and clinical relevance (e.g., “more real life than…other training” and “highlighted my own personal bias”) (Table 5). Notably, one participant stated that the CAM-ICU feature 3 (altered level of consciousness) domain identified gaps in their own knowledge and highlighted personal sedation biases. Last, all participants agreed they would deploy the ICU Delirium Playbook in their ICUs.

TABLE 5.

Verbal Feedback on the ICU Delirium Playbook (n = 9 Nurses)^a

Theme
Length and structure
“Short...and to the point”
“The videos weren't too long compared to other training videos”
“The written material…portions are too quick”
“I had to pause and go back to read the written slides”
Utility
“I really liked the videos”
“I loved having the vignettes”
“I was able to get what I needed from the videos”
“It's important we have education on this”
“Assessments were excellent”
Clinical relevance
“More real life than the other training I've had”
“I feel like we tend to keep our patients, even if orders say light sedation...at moderate sedation, so it highlighted my own personal bias…and lack of knowledge”
“I think it really highlights the difference between RASS”
“I think it's good…we do have a lot of patients with weakness on one side”
Did the videos provide sufficient education to answer the questions?
Consensus answer was “yes”
Would you deploy the ICU Delirium Playbook in your ICU?
Consensus answer was “yes”
Did we miss anything?
Consensus answer was “no”

RASS = Richmond Agitation-Sedation Scale.

^aFeedback collected from all nine focus group ICU nurses.

Feedback was analyzed using Braun and Clarke’s thematic analysis approach.

DISCUSSION

Motivated by multiple ICU nurses reporting a lack of prior delirium education, this article describes the interdisciplinary development and validation of a first-of-its-kind “ICU Delirium Playbook” (www.icudeliriumplaybook.com), comprised of a series of videos and a corresponding quiz in a scalable, sustainable, and adaptable asynchronous digital platform. Our evaluation demonstrated that the final 44-question quiz had appropriate relevance and clarity, strong content validity, and excellent inter-rater agreement. Importantly, when evaluated by critical care nurses, the Playbook required an average of 53 minutes to complete and yielded significant pre-post quiz score improvements. Following Playbook completion, nurses reported high levels of comfort in detecting and documenting delirium correctly.

To our knowledge, the ICU Delirium Playbook is the first digital education tool to emphasize not only the importance of delirium but provide detailed instruction on correct and consistent documentation of all four features of the CAM-ICU in both routine and challenging (e.g., patients receiving sedation and/or mechanical ventilation, or with pre-existing neurocognitive impairments) clinical situations. While the Playbook covers familiar concepts, it also includes expert tips (e.g., evaluating hard of hearing and non-English speaking patients) and addresses common misconceptions, particularly that the CAM-ICU has been validated to be performed in mechanically ventilated and sedated patients, with “unable-to-assess” reserved primarily for comatose patients (5). Assessing challenging patients and employing necessary adaptations are vital for correct delirium detection and a key step in designing and developing large-scale prevention efforts.

While prior educational interventions often did not include validated assessments, our team was fortunate to include experts with extensive experience in quiz development and validation who could maximize Playbook rigor, yielding a final 44-item quiz with excellent content validity as well as end-user feedback who found the Playbook had strong face validity (34). Furthermore, to optimize familiarity, scalability, and sustainability, we successfully adapted the Playbook to be completed in 60 minutes or less, employ a familiar digital video and quiz format, and be widely accessible via an institutional e-learning system.

Following Playbook completion by a focus group of nine critical care nurses, the majority reported increased levels of comfort in correctly performing the CAM-ICU, with most posting at 9% or higher pre-post quiz improvement, comparable with other prior delirium improvement efforts (22, 24, 45). Overall, this interdisciplinary effort confirmed Playbook validity and associated knowledge acquisition, thus maximizing its educational value. A logical next step would be to use a rigorous approach (e.g., an established implementation framework) to adopt the Playbook in real-world ICU settings while evaluating its sustainability and impact on important patient (e.g., length of stay), family (e.g., satisfaction), and provider (e.g., knowledge) outcomes.

Despite the strengths and novelty of the ICU Delirium Playbook, several limitations require discussion. First, as our delirium experts and nurses represented multiple institutions, differences in local ICU delirium policies and practices yielded occasional disagreements. To address this issue, the interdisciplinary team settled disagreements during virtual meetings and, if needed, adjudicated differences using the CAM-ICU training manual, thus preserving Playbook generalizability (46). Second, given the busy schedules of ICU providers, the interdisciplinary team could not facilitate more than two rounds of content validation and more than 2 hours of face validation. However, this meeting frequency and duration was sufficient, as all questions addressed in the second round were deemed satisfactory for the final Playbook. Third, despite multiple translations of the CAM-ICU, the ICU Delirium Playbook provides education only in English, thus limiting its reach. To address this limitation, our team has expanded to develop a Spanish language version of the ICU Delirium Playbook, which is currently being translated and culturally adapted before undergoing validation and evaluation. Methods used for the Spanish language version will subsequently be applied to develop the Playbook in other languages. Fourth, due to logistical and scheduling constraints, we were unable to recruit a group with broader clinical (e.g., more neurologic ICU nurses) and diverse representation (e.g., more non-English speakers from outside the United States). Future projects in this area, including one led by an interdisciplinary team from Puerto Rico to evaluate the Spanish language version of the ICU Delirium Playbook, will engage a larger and more diverse group. Fifth, as a video and quiz-based platform, the ICU Delirium Playbook does not include mechanisms for hands-on teaching (e.g., simulated or real patients). Given its expandable digital platform, we anticipate future versions of the Playbook will include novel additions such as game-like simulations, dissemination and implementation strategies for ICU leadership, patient testimonials, and detailed input regarding prevention efforts (e.g., early mobilization).

As the validated ICU Delirium Playbook is now available publicly (www.icudeliriumplaybook.com and @icuplaybook), how can ICUs use this module at the local level? First, ICU stakeholders must evaluate institutional delirium education practices including barriers and facilitators to early and correct delirium detection at the local level (47). Second, prior to ICU Delirium Playbook rollout, stakeholders must consider an active implementation approach with necessary leadership buy-in, engaged local champions and super users, a disciplined rollout approach, staff incentivization, and audit-and-feedback mechanisms (48). For example, ICU leaders and educators can consider requiring the Playbook for all providers and incentivize completion by awarding learners with continuing education units and opportunities for performing delirium-related projects (49). Establishing a minimum baseline level of knowledge among staff can jumpstart small- and large-scale efforts to evaluate, sustain and optimize ICU delirium practices.

CONCLUSIONS

Following a rigorous content and face validation process, the ICU Delirium Playbook is now available for widespread use at www.icudeliriumplaybook.com. Simple and scalable, this digital ICU delirium education module can be completed in less than 60 minutes. This first-of-its-kind module aims to standardize delirium detection and prevention practices and motivate larger-scale efforts to improve critical care practices and patient outcomes.

ACKNOWLEDGMENTS

We are grateful to the many critical care nurses, physicians, scientists, and researcher personnel who participated in development of the “ICU Delirium Playbook.”