Checklist design and implementation: critical considerations to improve patient safety for low-frequency, high-risk patient events

Abstract

Purpose: This pilot project describes the development and implementation of two specialised aviation-style checklist designs for a low-frequency high-risk patient population in a cardiac intensive care unit. The effect of the checklist design as well as the implementation strategies on patient outcomes and adherence to best practice guidelines were also explored. The long-term objective was to improve adherence to accepted processes of care by establishing the checklists as standard practice thereby improving patient safety and outcomes.

Methods: During this project, 10specialised crisis checklists using two specific aviation-style designs were developed. A quasiexperimental prospective pre-post repeated measure design including surveys along with repetitive simulations were used to evaluate self-confidence and self-efficacy over time as well as the perceived utility, ease of use, fit into workflow and benefits of the checklists use to patients. Performance, patient outcomes and manikin outcomes were also used to evaluate the effectiveness of the crisis checklists on provider behaviours and patient outcomes.

Results: Overall self-confidence and self-confidence related to skills and knowledge while not significant demonstrated clinically relevant improvements that were sustained over time. Perceptions of the checklists were positive with consistent utilisation sustained over time. More importantly, use of the checklists demonstrated a reduction in errors both in the simulated and clinical setting.

Conclusion: Recommendations from this study consist of key considerations for development and implementation of checklists including: utilisation of stakeholders in the development phase; implementation in real and simulated environments; and ongoing reinforcement and training to sustain use.

Introduction

During high-stress situations, healthcare workers are vulnerable to forgetting facts and data, to skipping crucial steps due to high cognitive burden, distraction, crisis or fatigue, and to fixating on a single issue when there may be several potential causes of a problem.^1–4 This potential for human failures in stressful situations has been well recognised in the aviation industry where both preflight checklists and emergency checklists are diligently used to assist with evolving inflight crises.^{3 5–7} For these reasons, checklists have been recommended as a methodology to improve the quality and safety of medical care by supporting human memory and standardising critical and often complex processes.2 3 6–9 Although designed with good intention, these checklists often lack usability and differ significantly from aviation-style checklists.^{6 7} The purpose of this pilot project was to evaluate the development and implementation of specialised aviation-style checklists on perceptions of self-confidence, checklist relevance and utilisation, patient outcomes and adherence to best practice guidelines for patients requiring the Impella left ventricular assist device (LVAD) in a cardiac intensive care unit (CICU).

This pilot project sought to fill an important gap in patient safety literature related to the development, design and implementation of aviation-style checklists targeted at a low-frequency, high-risk patient populations. Both the design and the utility of the checklists for use with patients requiring the Impella LVAD device as well as the adherence to accepted safety processes that would decrease the likelihood of serious harm were examined. The long-term objective was to improve adherence to accepted processes of care by establishing checklists as standard practice for this low-frequency, high-risk invasive life support device.

Literature review

Checklists

Checklists have long been accepted in other high-risk industries (eg, aviation, military and nuclear power) as a tool to aid performance during rare and unpredictable critical events.6 7 10 11 More recently, these tools have been embraced by the medical community to improve patient safety and patient care.5–7 12–14 The evidence supporting checklist utilisation has also grown with numerous examples of their positive influence on patient safety and outcomes in a variety of clinical settings.5 10 15–24 For example, in several high-risk work environments such as the intensive care unit,^25–27labour and delivery,^{18 28} emergency centre,¹⁶ operating room8 10 11 29–40 and radiology,^{21 41} checklists have clearly demonstrated positive results for facilitating complex multistep processes leading to error reduction. Checklist utilisation has also been linked to benefits beyond traditional morbidity and mortality outcomes including cost savings, improved team dynamics and a creation of a culture of safety through the adherence to patient safety processes.10 15 17 42–45

The exact mechanism by which checklists improve outcomes is not well understood.1 5 6 15 29 46 However, using a safety checklist can improve situational awareness by ensuring that all relevant data are assessed and comprehended, that problems are anticipated and that the correct course of action is selected.^{6 43} The likelihood of error is also reduced with checklist utilisation through enhanced communication, teamwork and compliance with established evidence-based protocols.8 11 15 31 38 43 47 48 In addition, the use of a checklist reduces the need to rely on memory, especially in rare, highly dynamic and unpredictable critical situations where time is of essence.^{6 10} Using a checklist in these situations draws attention to crucial steps, or steps in a process that, if omitted, could adversely affect patient safety and ultimately outcomes.6 7 10 49

In summary, while the benefits associated with the implementation of checklists have been well studied, the effect of specialised aviation-style checklists on provider behaviour in high-risk, critically ill patients has been largely untested. In addition, there is a paucity of studies describing the actual process of checklist development.5–7 15 46 48 50 51 Therefore, numerous questions still remain on the best methodology for developing, designing and adapting checklists in acute care settings.

Methods

Design and setting

A quasi-experimental prospective pre-post repeated measure design was used for this pilot project to evaluate the development and implementation of specialised aviation style checklists for patients requiring the Impella LVAD (table 1).

Table 1.

Symbolic representation of the prospective pre-post repeated measure design

Preintervention	Evidence-based intervention Phase I	Evidence-based intervention Phase II	Evidence-based intervention Phase III	Evidenced-based intervention Phase IV
	EB Didactic Impella Education Sessions March 2014.	Impella Patient Event-Based Simulation June–July 2014.	Impella Checklist Introduction September 2014. Impella Patient Event-Based Simulation September–December 2014.	Impella Self-Confidence and Comfort Survey February 2017. Impella Simulation-Based Education February–March 2017.

Preintervention surveys completed by participants and preintervention organisational and patient-level observations	Postintervention survey completed by participants	Postintervention survey completed by participants	Postintervention survey completed by participants and organisational and patient level observations	Postintervention survey completed by participants
Impella Self-Confidence and Comfort Survey October–November 2013. Patient Safety Quality Incident Reports (PSQI) for Impella Patients January–September 2014.	Impella Self-Confidence and Comfort Survey April–May 2014.	Impella Self-Confidence and Comfort Survey July–August 2014.	Impella Self-Confidence and Comfort Survey July–August 2014. Impella Checklist Design Questionnaire December 2014–January 2015. Patient Safety Quality Incident Reports for Impella Patients September 2014–May 2015. Mannequin Outcomes Simulation Video Analysis May–June 2015.	Impella Self-Confidence and Comfort Survey June–July 2017.

A convenience sample of CICU nurses identified as Impella LVAD superusers were recruited for this project and were followed betewen March 2014 and June 2017. Prior to recruitment, approval was obtained from the hospital Institutional Review Board HIC #2014–173.

Development phase

During this pilot project, 10 crisis checklists using two specific aviation-style designs (‘Read and Verify’ and ‘Read and Do’) were developed in collaboration with a checklist development team. The checklist development team included clinical experts, bedside CICU nurses identified as Impella LVAD superusers (defined as those who have received specialised training with the Impella LVAD), an Abiomed clinical representative and a national patient safety expert.⁴⁶ Items for the checklists were identified through a literature review, consultation with Abiomed representatives and with Impella LVAD superusers, as well as a review of Patient Safety Quality Incident Reports (PSQI) reflecting common errors or omissions in care. Based on this information, the selected areas for the checklists encompassed patient admission, patient management and common alarms. Particular attention was paid to aspects of the checklist design based on human factors research and experience from the aviation industry.

Design phase

The first checklist design was a ‘Read and Verify’ checklist meant specifically for situations involving routine, frequently accomplished tasks in which the steps in the process are well known and learnt through repetition and thus often completed from memory⁶ (see figure 1). A key concept of the ‘Read and Verify’ checklist design required that a series of steps would be first completed by memory and then followed by a read back and verification from the checklist.^{6 46} This design was intended for use by two individuals, with one reading the challenge on the left and the other person replying with the response verifying the task was completed. The requirement for two users (one to read and one to verify) has been shown to reduce the likelihood of human error.⁶ A series of five crisis checklists were created using this ‘Read and Verify’ design for use during routine care activities that would be done consistently and repeatedly with every patient with the Impella LVAD.^{6 7} Specific intent was given to maintaining a simple, easy-to-read format that was both clear and concise allowing ease of use and long term adoption. To supplement the read and verify checklist, several ‘expanded’ checklists were also created to provide additional support and guidance, while the series of tasks related to general patient management were being committed to memory.^{6 7 52}

Figure 1.

Read and Verify Impella Admission Checklist. This checklist provided superusers with guidance on the routine tasks that must be completed with all Impella patients on admission to the cardiac intensive care unit (CICU).

The second checklist design used during this project was a ‘Read and Do’. This type of checklist design is intended for highly critical or emergent situations where the expectation was to not rely on memory to manage the situation.^{6 7 27} Unlike the format of ‘Read and Verify’ checklists where challenges (nouns) are listed on the left side and responses (verbs in the past tense or the condition of a system) on the right, ‘Read and Do’ checklists present actions to be taken step by step in real time and can have a variety of formats.^{6 27 46} This ‘Read and Do’ checklist design was also presented in a decision tree type format with ‘yes’ or ‘no’ type questions posed to the end user managing the situation, with different courses of actions for each. The goal was to provide support during critical patient situations and emergencies where the appropriate response may be dependent on the presence or absence of specific circumstances.^{6 27}

The concepts driving this type of checklist design were especially important as the patient conditions can change rapidly, requiring specific responses and actions to ensure patient safety.⁶ The stress from this type of emergency situation may negatively affect memory and the ability to assess and critically think, both of which contribute to a reduction in situational awareness as the situation unfolds.⁶ The ‘Read and Do’ checklists provided guidance and direction ensuring that the assessments were focused on the correct items and that the actions taken were suitable for the conditions experienced.^{6 7 52} For this reason, the checklists created using this design were not intended to be memorised and were also intended for two individuals to use^{6 46} (see figure 2). A series of five crisis checklists were created using this ‘Read and Do’ design for use during critical patient situations.⁶

Figure 2.

Read and Do Impella Catheter Position Verification Checklist. This checklist provided support to the super-users during critical patient situations and emergencies where the appropriate response may be dependent on the presence or absence of specific circumstances.

Validation phase

Validation and evaluation of the usability of the checklist designs occurred in both the simulated and actual clinical setting by the checklist development team and Impella LVAD superusers. After initial review and validation, revisions were made based on the feedback.^{19 46} A total of three rounds of revisions were made to the initial checklist designs with the entire process from conceptualisation through the final revisions and implementation with all superusers taking approximately 12 months.

Implementation phase

In addition to the development of the aviation style checklists, this project also integrated a multifaceted educational approach to introduce the checklists including three high-fidelity simulation-based learning experiences (SBLEs). The superusers participated in the educational activities in a sequential manner (eg, web-based learning modules and didactic curriculum prior to participation in the SBLE) at 3-month intervals.⁵³ The crisis checklists were first introduced to all superusers and were available for use after an initial SBLE featuring an Impella LVAD admission with cardiogenic shock in the hospital simulation centre.⁵³ Prior to the second SBLE, superusers were introduced to the checklists in a traditional in-service on the unit and were then able to use the checklists during the follow-up SBLE. A third SBLE was offered 2 years after the second SBLE. SBLE sessions lasted approximately 20 min followed by a 45 min debriefing session using the Debriefing for Meaningful Learning framework.⁵⁴ The patient scenario did not change between the first and third simulation sessions.⁵³

Evaluation measures

Impella Self-Confidence and Overall Comfort Survey

This survey measured the perceptions of participants related to self-confidence and comfort with management of an Impella LVAD patient. Questions were adapted from the National League for Nursing (NLN) Satisfaction and Self-Confidence with Learning Tool.⁵⁵ The tool developed for this study consisted of 17 items developed to measure self-confidence (10 items) and comfort (7 items) for the Impella LVAD and focused in basic management and patient situations. The tool used a 5-point Likert scale (1=strongly disagree to 5=strongly agree). The survey was administered to all CICU Impella LVAD superusers via an email survey link as a baseline measurement prior to conducting the SBLE. It was readministered to all superusers at six different time points between March 2014 and June 2017 to ascertain if there was a change from baseline after traditional education with online modules and didactic education, SBLEs and the introduction of the Impella LVAD checklists.

Impella Checklist Design Questionnaire

This survey assessed participant perceptions of the Impella LVAD crisis checklists designs immediately after they were first exposed to the checklists during the second SBLE session. This questionnaire consisted of 10 items designed to measure satisfaction with each of the crisis checklists using a 5-point Likert scale (1=strongly disagree to 5=strongly agree). The Impella Checklist Design Questionnaire was sent one time via an email survey link to all superusers who had participated in the SBLE. Two additional questions exploring checklist design were added to the original Impella Self-Confidence and Overall Comfort Surveys in 2017 to assess ongoing perceptions of the Impella LVAD crisis checklist designs.

Nursing performance, patient and manikin outcomes

Manikin outcomes (eg, vital signs and haemodynamic parameters) and attainment of critical performance markers (eg, device configuration, verification of placement and anticoagulation, and alarm management) from the SBLE were evaluated by the primary investigator with the use of the crisis checklists and video review of the second SBLE session offered in 2014–2015. Ability to obtain key orders (eg, fluid resuscitation for suction alarms) or perform critical patient safety steps (eg, verification of anticoagulation), with the use of the crisis checklists as well as the patient’s final haemodynamic status (eg, heart rate, blood pressure and cardiac output) were examined during this analysis. In addition, all Impella LVAD patients admitted to the CICU were followed by the research team during 2014–2015 to identify patient safety issues related to critical processes of care.

Data analysis

Data were analysed using SPSS V.22.0. Descriptive statistics, including mean, frequencies and SDs were calculated to describe the study sample and report survey responses. One-way analysis of variance was performed to determine statistically significant differences in pre/post survey responses over time.

Results

Demographic characteristics

Twenty (77%) of the CICU nurses (n=26) completed online Impella LVAD learning modules, and 19 (79%) attended a didactic educational session in March 2014. In June and July 2014, 23 of the 26 (88%) identified superusers participated in nine SBLEs. The crisis checklists were then introduced approximately 3 months later during nine additional SBLEs held between September and December 2014. These sessions were attended by 21 of the 26 (81%) superusers. In February 2017, additional SBLEs were offered for 48 CICU nurses identified as superusers for the Impella LVAD device. Prior to participating in the study, the majority of the superusers (74%) had never cared for an Impella LVAD patient in the clinical setting. By the conclusion of the study (2017), over half of the participants (57.1%) had managed more than one patient with an Impella LVAD device (table 2).

Table 2.

Experience and educational characteristics of participants

	October 2013 (n=19)	May 2014 (n=19)	September 2014 (n=17)	January 2015 (n=14)	February 2017 (n=26)	July 2017 (n=21)
CICU experience
Less than 6 months	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)
6–11 months	0% (0)	5.3% (1)	0% (0)	0% (0)	0% (0)	0% (0)
1–2 years	0% (0)	5.3% (1)	5.9% (1)	0% (0)	19.23% (5)	14.29% (3)
3–7 years	5.3% (1)	10.5%(2)	5.9% (1)	14.3% (2)	11.54% (3)	9.52% (2)
8–12 years	5.3% (1)	21.1% (4)	23.5%(4)	35.7% (5)	11.54% (3)	19.05% (4)
13–20 years	52.6% (10)	36.8% (7)	47.1% (8)	35.7% (5)	26.92% (7)	19.05% (4)
21 or more years	36.8% (7)	21.1% (4)	17.6% (3)	14.3% (2)	30.77% (8)	38.10% (8)
Nursing experience
Less than 6 months	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)
6–11 months	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)
1–2 years	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)	0% (0)
3–7 years	5.3% (1)	15.8% (3)	5.8% (1)	7.1% (1)	19.23% (5)	14.29% (3)
8–12 years	10.5% (2)	10.5% (2)	11.8% (2)	14.3% (2)	0% (0)	0% (0)
13–20 years	31.6%(6)	31.6% (6)	41.2% (7)	50% (7)	23.08% (6)	19.05% (4)
21 or more years	52.6% (10)	42.1% (8)	41.2% (7)	28.6% (4)	57.69% (15)	66.67% (14)
Education level
Associate degree	26.3% (5)	26.3% (5)	35.3%(6)	28.6%(4)	23.08% (6)	23.81% (5)
Bachelor’s degree	68.4% (13)	63.2% (12)	58.8% (10)	64.3% (9)	73.08% (19)	76.19% (16)
Master’s degree	5.3% (1)	10.5% (2)	5.9% (1)	7.1% (1)	3.85% (1)	0% (0)

CICU, cardiac intensive care unit.

Impella Self-Confidence and Comfort Survey and Checklist Design

Participant mean scores of overall self-confidence, self-confidence related to skills and knowledge, not missing necessary tasks, admitting an Impella LVAD patient, performing ongoing monitoring and alarm management were not statistically significant. However, all areas explored related to self-confidence and comfort with management of an Impella LVAD patient demonstrated clinically relevant improvements between May 2014 and January 2015 when the SBLEs and Impella LVAD crisis checklists were introduced (table 3). Confidence in having the skills and required knowledge to perform the necessary tasks improved from a mean score of 3.79 to 3.94 (possible range 1–5) between November 2013 (n=19) and January 2015 (n=13). Likewise, confidence in not missing any necessary tasks improved from a mean score of 3.37–4.08, confidence in determining causes for alarms improved from a mean score of 3.58–4.23 and confidence in ability to implement interventions for alarms improved from a mean score of 3.47–4.31 between the fall of 2013 (baseline assessment) and January of 2015 after implementation of the SBLEs and Impella crisis checklists (possible range on all scores 1–5).

Table 3.

Self-confidence and comfort with Impella catheter

Answer options	Rating average Fall 2013 (n=19)	Rating average May 2014 (n=19)	Rating average September 2014 (n=17)	Rating average January 2015 (n=13)	Rating average February 2017 (n=22)	Rating average July 2017 (n=20)
I am confident that I have the skills and required knowledge to perform the necessary tasks for managing a patient with an Impella catheter.	3.79	3.89	3.94	4.23	3.77	3.74
I feel confident that I will not miss any necessary tasks (eg, echo, transfer to standard configuration, catheter centimetre marking, pressure tubing set up) when admitting a patient with an Impella catheter.	3.37	3.63	3.76	4.08	3.64	3.79
I am confident that I will not miss any required ongoing general patient monitoring and care assessments for a patient with the Impella catheter.	3.63	4.00	4.00	4.31	3.91	3.84
I do not feel confident that I am able to determine the proper placement signal and motor current waveforms for the Impella catheter.	2.26	2.32	2.12	1.62	2.41	2.26
I am confident that I can determine if the Impella catheter is properly positioned and functioning based on my assessments of the Impella flow rate.	3.89	3.95	4.06	4.15	3.82	3.79
I do not feel confident that I can locate the amount of purge solution delivered and purge pressure on an hourly basis.	2.32	2.11	1.94	1.54	1.91	1.89
I am confident that I can determine potential causes for alarms.	3.58	4.00	3.94	4.23	3.73	3.53
I feel confident that I can identify the correct interventions for an alarm on the Impella device.	3.47	3.89	3.88	4.31	3.73	3.58
I do not feel confident that I can manage/troubleshoot alarms (eg, suction, low purge flow pressure, wrong position or flow reduced) for the Impella catheter.	2.53	2.05	1.94	2.00	2.27	2.21
I am confident that I know what to do with the Impella device in the event of a CPR.	3.68	4.05	4.06	4.31	4.05	3.84
When admitting a patient with the Impella catheter, my comfort level with completing all of the necessary tasks (specific to the device) is:	2.89	3.26	3.53	3.77	3.23	3.32
While performing the ongoing general patient monitoring and care (eg, vital signs, circulation checks, haemodynamic profiles and physical assessment) for a patient with the Impella catheter, my comfort level is:	3.74	3.68	3.88	4.23	3.41	3.58
My comfort level with ongoing monitoring of waveforms to verify placement of the Impella catheter is:	3.63	3.68	3.65	4.00	3.41	3.53
My comfort level with ongoing monitoring of the Impella flow rate is:	3.42	3.74	3.65	4.15	3.41	3.63
My comfort level in monitoring the purge flow rate and purge pressure is:	3.47	3.63	3.59	4.08	3.45	3.58
When I am troubleshooting a problem or alarms with the Impella catheter, my comfort level is:	2.79	3.26	3.24	3.54	3.18	3.21
During a CPR, my comfort level in knowing what to do with the Impella device is:	3.26	3.53	3.76	4.15	3.5	3.63

Note: possible range on all scales is 1–5. Ratings (using a Likert scale) of overall self-confidence all demonstrated clinically relevant improvements over time between November 2013 and July 2017.

CPR, Cardiopulmonary Resuscitation.

Follow-up surveys after additional education in February 2017 and June 2017 yielded similar findings; however, the mean scores in both of the follow-up surveys 2017 were lower than those in 2015 at the end of phase III for this project. For example, having the skills and required knowledge to perform necessary tasks was 4.23 in Janury 2015 (n=13) compared with 3.77 in Feburary of 2017 (n=22), and then 3.74 in July of 2017 (n=20) (possible range on all scores 1–5). Additionally, between January 2015 and February 2017, there was turnover of superusers and addition of new superusers, which may have impacted the perceptions of self-confidence (table 3).

Perceptions of the checklist utility and relevance were also very positive. A majority of the participants (n=10) indicated that the use of the checklist should continue with a mean rating of 4.5 (possible range 1–5). Respondents also indicated that the checklists were easy to use (mean=4.40), they facilitated a feeling of being prepared to manage Impella LVAD patient (mean=4.60) and that the checklists assisted in completing critical tasks safely (mean=4.50) (possible range 1–5) (table 4). These findings remained consistent over time with respondents again indicating that the checklists were still perceived as easy to read (mean=4.0 to mean=4.2) in January (n=26) and then again in July of 2017 (n=21) during the follow-up SBLE and surveys. Additionally, over 80% of the respondents in January and July 2017 identified ongoing utilisation of the Impella LVAD checklists for assisting with management of these high-risk patients.

Table 4.

CICU Impella LVAD Checklist Design Questionnaire

Answer options	Rating average (M) January 2015 (n=10)	Rating average (M) January 2017 (n=26)	Rating average July 2017 (n=21)
The checklist helped me feel better prepared during the Impella patient scenario.	4.60
The checklist was easy to use.	4.40
The checklist was easy to understand/follow.	4.50
The font was clear and easy to read.	4.50
The checklist(s) helped me carry out the necessary patient tasks in a safe manner.	4.50
The length of the checklist(s) was appropriate for the tasks.	4.50
I would use this checklist if I were presented with this clinical patient situation in real life.	4.50
If I were a patient with an Impella device, I would want this checklist to be used.	4.50
The use of the Impella checklists should continue.	4.50
The use of checklists should be expanded to other low frequency high-risk devices in the ICU (eg, CRRT, IABP, PA catheters).	4.40
I found the Impella checklist tools for managing/troubleshooting the Impella catheter easy to read (new 2017 question).		4.0	4.2
I found the Impella checklist tools for managing/troubleshooting the Impella catheter easy to apply/use for my clinical practice while caring for a patient with this device. (new 2017 question)		4.0	4.2

Note: possible range on all scales is 1–5. Rating average is displayed.

CRRT, Continous Renal Replacement Therapy; IABP, Intra-Aortic Balloon Pump; ICU, intensive care unit; PA, Pulmonary Artery.

Nursing performance, patient and manikin outcomes

Video review of the SBLEs in 2014–2015 revealed clinically relevant improvements in critical processes of care. Nurses were less likely to omit key practice guidelines for patients (eg, verification of anticoagulation, device location, identification and resolution of critical alarms) during the second simulation. During the first SBLE, critical care processes were missed 55%–100% of the time. These errors were reduced to 0%–78% during the second SBLE. For example, 100% of the groups (n=9) verified the Impella LVAD placement on admission during the second SBLE with the crisis checklists compared with only 89% of the groups (n=9) completing this critical task during the first simulation sessions without the crisis checklists. The average time to complete this task was also reduced by nearly 4 min between the first and second SBLE (average of 5 min 32 s during the first SBLE [n=9] to an average of 1 min and 35 s during the second SBLE [n=9]). Verification and changing of the anticoagulation purge solution also demonstrated improvement with only 22% of the groups (n=9) missing this critical task during the second SBLE with the crisis checklists compared with 100% of the groups (n=9) missing this critical task during the first SBLE without the crisis checklists. One potential explanation for not attaining 100% in this task or other critical patient care tasks during the second SBLE with the crisis checklists is that the superusers may not have fully used the crisis checklists as intended. It is possible that those participants who missed critical tasks during the second SBLE continued to rely on memory versus using the checklist as designed for use in a ‘Read and Verify’ format with one reading the information and the other nurse verifying completion of the task (table 5).

Table 5:

E1–E3 nursing performance during simulation

Table E1: nursing completion of key practice guidelines for impella patient during simulation sessions (n=9)
Task	Complete	Incomplete
Two-dimensional (2D) echo verification of placement
Simulation #1	11% (1)	89% (8)
Simulation #2	100% (9)	0% (0)
CM marking verified
Simulation #1	44% (4)	55% (5)
Simulation #2	100% (9)	0% (0)
Tightening of Toughy Borst valve
Simulation #1	0% (0)	100% (9)
Simulation #2	78% (7)	22% (2)
Transfer to standard configuration
Simulation #1	89% (8)	11% (1)
Simulation #2	89% (8)	11% (1)
Purge solution verified and hung
Simulation #1	0% (0)	100% (9)
Simulation #2	78% (7)	22% (2)

Table E2: average time to complete critical tasks for Impella patient during simulation (n=9)
Activity	Simulation 1	Simulation 2
2D echo verification	5 min 32 s	1 min 32 s
CM marking	5 min 13 s	3 min 55 s
Standard configuration	4 min 3 s	4 min 20 s
Purge solution verified	No groups fully completed*	2 min 18 s
Purge solution hung	No groups fully completed*	3 min 13 s

Table E3: nursing recognition and management of alarms for impella patient during simulation sessions (n=9)
	Recognised alarm (suction, aorta/ventricle)	Correctly identified cause of alarm	Correct identification of action(s) to resolve alarm	Notified physician with recommendation	Notified physician without recommendation
Simulation #1	100% (9)	33% (3) (suction alarms only)	33% (3) (suction alarms only)	33% (3) (suction alarms only)	67% (6)
Simulation #2	100% (9)	100% (9)	100% (9)	100% (9)	0% (0)

*During simulation #1, all groups partially completed the task of verifying the purge solution change to dextrose plus heparin by simply hanging the solution. However, none of groups changed the information in the Impella console; therefore, the console remained programmed as having only dextrose infusing as the purge solution and would not recognise or calculate the amount of heparin in use and delivered to the patient.

At the same time, and perhaps even more importantly, a reduction in actual patient errors (eg, verification of catheter placement, waveform recognition and alarm management; missed verification of anticoagulation) was noted for patients with the Impella LVAD in the CICU. Between January and September of 2014, the patient error rate for CICU patients with an Impella was 72% (n=7). Between September 2014 and May 2015, the number of CICU Impella patient errors decreased to 40% (n=10). Unfortunately, continued reporting of patient safety errors and tracking of this metric was not sustained between May of 2015 and July of 2017.

Limitations

At the time of this project, there was a paucity of literature on the ongoing management of Impella LVAD patients outside the cardiac catheterisation laboratory as well as on the design and development of medical checklists.^{5 46} As a result, there was limited literature to support evidence-based strategies for the superusers managing Impella LVAD patients and on important design features or considerations for a crisis checklist.⁴⁶ Due to the lack of literature, some items on the crisis checklists lack an extensive evidence base; however, all elements have demonstrated clinical face validity.⁵⁰

Similarly, due to the highly specialised nature of an Impella LVAD device, there was a limited patient population meeting the criteria for the use of this medical device resulting in a low number of actual patient encounters to inform the development, revisions, implementation and ongoing sustainability of the checklists.46 50 51 53 Additionally, this project took place at a single site using a convenience sample limited to only those CICU nurses who were considered superusers. This yielded a low sample size initially (n=26) as the checklists were first developed and introduced also contributing to limited exposure and feedback on the crisis checklists.^{50 51 53}

Further adding to the challenge of recruiting a larger sample size of superusers was the limited exposure of the superusers to actual Impella LVAD patients. Although the small sample size limits application of the findings from this pilot project, this project could be replicated in the future with other low-frequency, high-risk devices or procedures. Consideration for a multicentre study with the current crisis checklists for Impella LVAD patients may also provide a larger sample size as well as increase the number of patient encounters.

In addition, during the course of this project, there was ongoing turnover and training of superusers, which may have negatively impacted perceptions of self-confidence, adoption of the checklists and overall perceptions of the checklist designs (particularly for those not involved in the development and validation phases).^{46 51} At the start of the project in 2013, there were 26 superusers. Of these original 26, five were no longer caring for Impella LVAD patients by January of 2017. Furthermore, between January 2014 and January 2017, there were a total of 27 new superusers trained to care for Impella LVAD patients. As a result, although all superusers were required to attend a training session (web-based learning module and/or didactic curriculum) prior to caring for an Impella patient, not all superusers attended the SBLEs. This ongoing turnover and addition of new superusers underscores the need to provide continued support and training in both the clinical and simulated environment to sustain the utilisation of the crisis checklists into routine clinical practice.

Finally, sustainability and full adoption of the crisis checklists may have been negatively impacted by lack of ongoing educational support for the project.^{46 51} Between 2015 and 2017, there were limited educational opportunities, and no further SBLEs offered to further support the utilisation of the crisis checklists. In addition, there was a lack of reporting of patient safety errors during this same timeframe, which may not accurately reflect changes in patient safety, but rather changes in the monitoring and reporting patterns for these patients.⁵⁶ This suggests that further emphasis on sustained educational and administrative support are critical to the success of implementation and adherence to patient safety processes such as the utilisation of a crisis checklist for low-frequency, high-risk patients. Future studies should include a long-term plan to sustain educational and administrative support to promote full adoption of crisis checklists into practice.

Discussion

The findings from this project suggest that the aviation style design of the crisis checklists for the Impella LVAD were perceived as a useful adjunct to the traditional clinical resources. Including key stakeholders in the initial development phase proved to be a valuable resource in assisting with identification of frequently omitted care processes, critical content for inclusion, readability, usability with current workflow and determination of the scope of the checklists.5 6 46 48 Likewise, inclusion of key stakeholders in this phase reduced the need to revise the checklists after implementation.

Implementation of the checklists in both the real and simulated environments also proved to be a valuable component potentially contributing to the successful utilisation 2 years after the initial implementation.^{19 46 51} Implementation success was likely attributed to introduction of the checklists with rationale for development and then use in an intentional, segmental manner prior to roll out.^{46 51} Likewise, providing multiple SBLEs also permitted the nurse superusers to firsthand experience the benefits of using the checklists in a realistic but safe setting with an Impella LVAD patient case similar to those managed on the unit.^{46 48 51}

The timing of the checklist project similarly proved to be a valuable asset to the adoption and ongoing utilisation of the checklists.^{46 48 51} The project was initiated in late 2013 due to both end user and administration request as the first Impella LVAD patients began to arrive in the CICU. As these patients were low frequency and new on the unit, there was a perceived need for education and support by the end users, which may have contributed to the adoption and subsequent high rate of utilisation. End users also were engaged in the development of the checklist from the beginning, providing valuable feedback on the design, integration into workflow and identification of processes where errors were most likely to occur.^{6 46} This facilitated a synergistic effect in terms of the perceived need and value that the checklists were providing to the superuser and to the patient.

Recommendations for practice

1. Included key stakeholders in the development and design of crisis checklists for low-frequency, high-risk patients to promote usability and adoption of the checklists into practice.

2. Consider validation and implementation of crisis checklists in both the simulated and clinical environments to assist with proper utilisation and facilitate adoption into practice.

3. Provide ongoing training and booster sessions including SBLEs routinely (every 3–6 months) after implementation to facilitate proficiency in managing a low-frequency, high-risk patient and provide opportunities to embed utilisation of a crisis checklist into routine practice when caring for these patients.

Conclusion

This project explored the perceptions of the checklist designs, the impact on superuser confidence in managing these high-risk patients, as well as the superuser adherence to critical safety processes with a goal of reduced errors and improved patient safety. The results from this pilot project suggest that careful attention to the design and development of crisis checklists is a key factor in their adoption and long-term utilisation. Validation and implementation in both the simulated and actual clinical setting was equally as important to facilitate initial adoption and adherence to critical safety practices for low-frequency, high-risk patients. However, ongoing education including simulation booster sessions should be considered to further support long-term adoption. Additional exploration is warranted with a larger sample size and on application to other low-frequency, high-risk patient conditions.