Stress Testing the Resuscitation Room: Latent Threats to Patient Safety Identified During Interprofessional In Situ Simulation in a Canadian Academic Emergency Department

Abstract

Objectives

Emergency department (ED) resuscitation is a complex, high‐stakes procedure where positive outcomes depend on effective interactions between the health care team, the patient, and the environment. Resuscitation teams work in dynamic environments and strive to ensure the timely delivery of necessary treatments, equipment, and skill sets when required. However, systemic failures in this environment cannot always be adequately anticipated, which exposes patients to opportunities for harm.

Methods

As part of a new interprofessional education and quality improvement initiative, this prospective, observational study sought to characterize latent safety threats (LSTs) identified during the delivery of in situ, simulated resuscitations in our ED. In situ simulation (ISS) sessions were delivered on a monthly basis in the EDs at each campus of a large tertiary care academic hospital system, during which a variety of scenarios were run with teams of ED health care professionals. LSTs were identified by simulation facilitators and participants during the case and debriefing and then grouped thematically for analysis.

Results

During the study period, 22 ISS sessions were delivered, involving 58 cases and reaching 383 ED health care professionals. 196 latent safety threats were identified through these sessions (mean = 3.4 LSTs per case) of which 110 were determined to be “actionable” at a system level. LSTs identified included system/environmental design flaws, equipment problems, failures in department processes, and knowledge/skill gaps. Corrective mechanisms were initiated in 85% of actionable cases.

Conclusions

Effective quality improvement and continuing education programs are essential to translate these findings into more resilient patient care. ISS, beyond its role as a training tool for developing intrinsic and crisis resource management skills, can be effectively used to identify system issues in the ED that could expose critically ill patients to harm.

Emergency department (ED) resuscitations are complex, dynamic interactions involving a diverse team of health care professionals (e.g., nurses, physicians, respiratory therapists, patient care aides) directed at the rapid stabilization, diagnostic workup, and treatment of critically ill patients before transfer for ongoing or definitive care. For medical teams working in this high‐stakes context, tightly coordinated and interdependent action is essential to ensure patient safety and optimize outcomes.1 However, effective performance in this environment is threatened by the ad hoc nature of the teams brought together, knowledge and skill gaps among individual team members, breakdowns in communication, and design flaws in the system and working environment2, 3, 4 Each of these challenges, in turn, risks exposing the patient to medical error, morbidity, and adverse outcomes. At the very least, these challenges threaten to thwart our efforts in delivering high‐quality care.

Simulation‐based education (SBE) is a training modality that uses sophisticated mannequins and other techniques (e.g., task trainers, trained actors) to replicate clinical encounters, providing the opportunity for clinicians to develop competency in high‐stakes clinical skills in a safe environment.5 SBE is particularly well suited to high‐acuity cases where opportunities for deliberate practice are limited6 and has been shown to be an effective training method for medical trainees in diverse specialties,7 as well as for interprofessional education.8 The aim of such programs is to develop proficiency in crisis resource management (CRM)—both the technical (i.e., clinical, procedural) and the intrinsic (i.e., interpersonal, teamworking) skills necessary for effective navigation of a medical crisis.9 Traditionally, SBE has been delivered in a laboratory or theatre setting, mocked up to resemble the clinical environment.

A more contemporary use of simulation is its application in the real patient care environment (a.k.a., “in situ simulation” [ISS]) as a means to heighten realism, relevance, and retention among simulation participants10, 11 while bringing together groups of health care professionals that work together day to day as clinical teams. ISS has been proposed as a means to engage practicing health care professionals in interprofessional team training and offers an opportunity for on‐the‐job education.12 Further, ISS creates opportunities to integrate SBE within the broader health care system, as a tool for optimizing patient care and safety.13 In settings outside the ED resuscitation environment, such ISS interventions have been shown to enhance technical performance, reinforce team behaviors, and improve objective clinical outcomes, especially in the context of cardiac arrest and trauma resuscitation.14, 15, 16

In situ simulation has also been proposed as a mechanism to identify latent safety threats (LSTs) in the clinical environment. In a systems‐centered approach to understanding patient safety in health care,17 errors are recognized as generally being attributable to misalignments between the individual(s), equipment, and working environment rather than a fault on the part of individual health care practitioners alone—the “Swiss Cheese Model” of failure pioneered by Reason.17 Through this system lens, LSTs are recognized as significant modifiable threats to patient safety or quality of care that result from challenges with equipment, processes, training, or other system breakdowns that typically lie dormant.18 These unrecognized risks embedded within clinical systems are usually only elucidated under stressed conditions when existing defences or workaround processes within the organization fail. ISS can therefore play a critical role, where SBE is implemented in the real clinical environment with the real clinical team that is working, using real medications and equipment to discover LSTs without risking harm to a patient19 Examples of LSTs include cultural barriers to effective teamwork, flaws in the design of working environments and/or equipment, issues with maintenance or upkeep of systems, and training or knowledge gaps among clinical staff. When such threats are unmasked during a crisis, they can interfere with effective team functioning or safe and timely delivery of care and can ultimately lead to patients suffering harm—through delays, misdiagnosis, or more grievous errors.

Research on this topic to date has been largely focused on operating room, inpatient, and pediatric contexts,20, 21, 22, 23, 24 where the literature has demonstrated that ISSs can be effective tools for imparting the necessary stress to a health care system to effectively unmask LSTs before a real patient encounter. Limited examples in the domains of pediatric emergency medicine and trauma resuscitation have shown similar results,25, 26 but to date little published evidence exists for its use in adult emergency medicine.27 In this paper we report on the use of an ISS program in our tertiary care academic ED for detecting LSTs.

Methods

We prospectively evaluated the impact of an ED‐based interprofessional ISS program on the identification of LSTs at our institution from January 2015 to December 2016. A needs assessment was conducted through consultation with departmental leadership, nurse educators, and simulation faculty and from incident reviews, patient safety reports, and morbidity and mortality rounds. From this needs assessment, we developed our curriculum, teaching model, and selected cases relevant to the resuscitation environment. The research protocol was approved by the institutional research ethics board.

We delivered ISS sessions in 4‐hour blocks, once per month in the EDs at two campuses of an academic tertiary care hospital system and regional referral center for trauma, cardiovascular and cardiac arrest, stroke, cancer, and critical care in Ottawa, Canada (combined patient volume ~180,000 visits/year). On each date, three teams of on‐shift ED staff (typically attending physician, emergency medicine resident, medical student, nurses, respiratory therapist, patient care assistant) were recruited on a voluntary basis to participate in an approximately 45‐minute simulation session. Informed consent was obtained for participation in the study.

A resuscitation bay in each ED was allocated for a 4‐hour period at the discretion of the charge nurse on duty that day. The charge nurse also maintained discretion to cancel simulation sessions if department pressures were too great. Four‐hour blocks were identified to maximize efficient use of resources (i.e., simulation technologists, equipment, facilitators, clinical space) and timed to coincide with historically lower‐volume periods to minimize impact on departmental flow and crowding. Funding was allocated from the departmental academic budget to provide physician educators and from institutional nursing education budgets to staff additional nurses for the 4‐hour period. The supplementary staff allowed for flexibility among the on‐shift ED team to leave assignments for the sake of participation. The explicit objectives of the sessions were to:

1. Enhance the use of CRM, specifically: situational awareness, closed‐loop communication, summarizing, task assignment, leadership, and followership;

2. Practice the management of critical incidents in the clinical setting; and

3. Identify latent threats to quality and safety in the clinical environment.

Each session followed a standard structure: a 5‐minute scripted prebriefing on confidentiality, the goals of this session (both team training and system audit), and an orientation to the simulation mannequin (Data Supplement S1, Appendix S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10422/full); a 15‐minute simulated medical resuscitation scenario using a high‐fidelity mannequin (SimMan 3G) or standardized patient, depending on case; a 20‐minute facilitated debriefing following the PEARLS framework28 to address both technical and intrinsic skill performance of the team; and 5 minutes for session feedback from participants. Cases were selected from our local emergency medicine simulation case bank of over 50 previously developed clinical scenarios, with slight adaptations (e.g., to narrative or procedural needs) made at facilitator discretion to optimize educational value for all members of the interprofessional team. Real clinical equipment was used, and this waste was considered acceptable from an operational and budgetary perspective by department leadership. An experienced nurse‐confederate was involved in each case and responsible for maintaining "boundaries" of the simulation—for example, blood bank activation would lead to a simulated "massive transfusion box" made available and the blood bank not actually contacted (rather, a mock phone call was made by the confederate). Emergency physician faculty members active in the departmental simulation program and experienced in debriefing oversaw delivery of cases and codebriefed the interprofessional teams with the full‐time nurse educator at each campus. The debriefing focused on highlighting success, closing performance gaps, and meeting the session objectives including CRM principles and the discovery of LSTs.

Data Collection and Analysis

At each simulation session, a nurse research assistant with experience in emergency medicine and simulation‐based medical education was present to assist with recruitment of participants, collection of consent and feedback forms, and LST documentation. LSTs were identified primarily by simulation participants and facilitators, either during the case or during the debriefing, and were recorded by the research assistant on a standardized data collection form (Data Supplement S1, Appendix S2). The research assistant was also responsible for capturing subjective quotes relevant to the curriculum objectives. The case facilitators (nurse educator and simulation faculty member) reviewed the compiled list of LSTs identified at the end of each case to ensure that no items were missed based on their own observations. Participants completed a standardized, anonymous feedback form (Data Supplement S1, Appendix S3) rating their experience participating in the simulation, reactions to the case, any LSTs they had identified, reflections on their own learning, and feedback on the program. Data were then entered into a master spreadsheet (Microsoft Excel, 2011) and grouped thematically by two authors (GM and CP) for analysis. Due to the qualitative, exploratory nature of this study, only descriptive statistics were calculated. A report was generated at the end of each 4‐hour session summarizing the LSTs identified (example in Data Supplement S1, Appendix S4); this report was circulated to key stakeholders including nursing and departmental leadership, unit managers, nurse educators, and the departmental clinical practice, quality, and safety committee to develop and implement mitigation strategies for the LSTs identified.

Results

Over our 2‐year intervention period we successfully delivered 22 simulation sessions between two campuses, comprising 58 simulated resuscitations on a range of clinical topics (Table 1) and reaching 383 total ED health care professionals (Table 2). Only one planned simulation session was canceled due to departmental overcrowding and high patient volume. Cases and participant demographics are described in Tables 1 and 2, respectively.

Table 1.

Cases Run at ISS

Case	No.
Septic shock	7
Unstable trauma	7
Pediatric cardiac arrest	7
Unstable bradyarrythmia	7
Pediatric sepsis	4
Excited delirium	3
Acute myocardial infarction	3
Pulseless electrical activity arrest	3
Massive gastrointestinal bleed	4
Rapid atrial fibrillation	2
Acute stroke	1
Ventricular fibrillation arrest	4
Challenging an authority figure	1
Peri‐mortem cesarean section	2
Crashing congestive heart failure	3

ISS = in situ simulation.

Table 2.

Demographics of Participants at ISS

Participant	No.
Physicians	162
Attending MD	56
Fellows	5
Resident physician	72
Medical student	29
Nurses	153
Respiratory therapists	45
Patient care aides and other	23

ISS = in situ simulation.

During these sessions we identified and reported on 196 LSTs—yielding an average of 8.9 (95% confidence interval [CI] = 7.7 to 10.1) LSTs identified per session and 3.4 (95% CI = 3.1 to 3.7) per case. These findings included safety threats attributable to system/environmental design flaws, equipment problems, failures in department processes, and knowledge/skill gaps (Figure 1).

Figure 1.

Proportions of LSTs identified by category. LST = latent safety threat.

System design flaws identified issues with physical space and ergonomics of the resuscitation room working environment. Examples of safety threats in this domain included the physical locations of medication and cardiac arrest carts (outside the room, necessitating that team members needed to leave the case frequently with subsequent degradation in situational awareness), organization of equipment carts (making it difficult to locate needed equipment in a timely fashion), latency in the hospital paging system when calling for help from consultants, and the absence of a labeling/ID system to identify various team members in a busy environment. Process failures pertained to medication administration errors, difficulties in accessing or applying clinical protocols, handover deficiencies, and issues with hospital‐level activations (e.g., massive transfusion protocol, trauma team activation). Equipment threats identified unrecognized problems with specific equipment required to deliver effective resuscitative care—such as defective parts in several laryngoscopes or expended batteries in a transvenous pacemaker generator as well as restocking issues for some frequently used devices (e.g., sterile ultrasound probe sheath covers, PEEP valves).

The remainder of the LSTs—comprising half of all those identified—were categorized as “knowledge/skill gaps”; that is, breakdowns in team functioning attributable to gaps or deficiencies in technical or intrinsic skills within the team. These gaps were further subcategorized by type, including breakdowns in team situational awareness, loss of team “shared mental model,” errors in the performance of clinical procedures, errors of omission, and inappropriate action taken for the clinical problem. Examples are provided in Table 3.

Table 3.

Knowledge/Skill Gap Examples

Knowledge/Skill Gap	Examples
Team situational awareness	Delayed recognition of changes in patient condition
Team “shared mental model”	Communication breakdowns, misunderstanding of clinical problems by members of the team
Incorrect performance of clinical procedures	Poor CPR quality, Incorrect administration of fluids/drugs, difficulties using intraosseous infusion lines
Errors of omission	Failure to use established protocols, ignoring airway compromise
Inappropriate action for the clinical problem	Inappropriate medications given, CPR not started when necessary

Of the 196 LSTs identified, 110 (56%) were found to be “actionable” at a systemic level. For example:

• Layout of physical spaces and equipment carts has been through several revisions, informed by the findings of the ISS program;

• Restocking problems have been addressed with supply managers;

• Medication references and infusion charts have been better itemized with binders made more accessible to staff; and

• Education campaigns were disseminated to address recurring knowledge gaps.

In the study period, 93 of the actionable LSTs (85%) have been addressed through process improvement, system change, or enhanced education efforts. The remainder of LSTs generally pertained to individual knowledge gaps and teamwork breakdowns that were addressed explicitly during the debriefing.

Discussion

In this study, implementation of a system‐integrated, longitudinal ISS program allowed us to identify a high prevalence of LSTs in an academic tertiary hospital ED environment. The nature of tertiary care emergency medicine dictates that teams rarely work together on more than one occasion, training and skill levels are heterogeneous, and working environments are often organized by accretion rather than through detailed planning, testing, and revision. While resuscitation in the ED is common, in a large department such as ours it represents an example of “high acuity, low opportunity” for individual health care professionals, limiting opportunities to identify targets for individual, team, or system improvement. Moreover, in a busy ED environment, dedicated time for debriefing is rarely forthcoming. Without such time, dangers or problems encountered (and usually mitigated) during major cases are often dismissed or forgotten about by staff who must urgently turn their attention to the next patient, rather than engaging in the cumbersome process of reflection and subsequent system change.

As such, ISS in our setting has been an effective tool in creating dedicated opportunities and time for interprofessional ED teams to train and improve together: fostering increased understanding of concepts in CRM while simultaneously providing a forum to reflect on and identify latent threats to patient safety that need attention. In the course of running the ISS program, a recurring theme reported by staff was that many identified LSTs had “always been an issue”—suggesting that they accepted a status quo and learned mitigation strategies rather than reporting on the issues or advocating for change, which aligns with the vast body of literature on system failures and LSTs. This program has thus proven useful not only in identifying LSTs, but in advancing a culture of patient safety, disseminating knowledge among our group of health care professionals on the influence of human factors and system design in medical error, and promoting the value of addressing identified problems “upstream.” This has been reflected repeatedly in both formal and informal feedback from participants stating that “we need to do this more often,” that “these sessions are extremely important,” and that they have “more confidence speaking up if they saw an issue” with the medical care or working environment. A cornerstone of the success for our program has been an adept and responsive group of managers and leaders interested in supporting the program delivery with operating funds and an eagerness to act on closing LST gaps emerging from the ISS program.

Interestingly, while our methodology followed a model similar to that of Patterson et al.25 in combining participant and facilitator observations to identify LSTs, we identified LSTs at a much higher frequency than that in studies by either Patterson et al. or Couto et al.27 We favored the approach of Patterson et al. to LST identification as it allowed for a perspective from the front‐line team, where LSTs are likely often quietly mitigated by the team during the case and only raised after, in the debrief. In contrast to the study by Patterson et al., where a hard 10‐minute cap was used for both scenario delivery and debriefing, we allowed our cases to run longer and allotted more time for a fulsome debrief. Our experienced facilitators endeavored to allow a free‐flowing dialogue by the participants, with an organic identification of LSTs, although they also raised their observations in the discussion. In combination, these differences in technique may have allowed more time for participants to reflect on and relay the range of LSTs encountered.

A particular highlight of our program is its success despite the busy and chaotic environment of a tertiary care, urban academic ED. Hypothetically "unannounced" simulations might have been more naturalistic and could have elucidated LSTs during stressed operating conditions. However, this is extraordinarily challenging to accomplish in the ED where overcrowding, irregular and unpredictable arrivals of critically ill patients, and high patient volumes are commonplace. To facilitate the delivery of these sessions, careful planning and coordination with on‐shift leadership was necessary. Despite these factors, our program was able to run with high engagement and a very low cancellation rate. As such, our article perhaps illustrates the pragmatism necessary to balance the priorities of idealized human factors testing against real‐world operational issues that challenge feasibility.

Limitations

Our study has other limitations. First, as a single‐institution trial in a Canadian academic ED, the findings may not be generalizable to other centers, particularly smaller hospitals and those in countries with different health care delivery models. Second, this study was bolstered by tremendous institutional buy‐in, enabling us to run ISS sessions with little resistance from leadership or front‐line staff despite occasional personnel shortages and ED overcrowding. Third, our timing of the simulations to coincide with lower‐volume periods in the ED, chosen for convenience and to minimize the probability of cancellation, may have led to underreporting of LSTs related to stressors encountered during higher‐volume times. Fourth, this study relied on self‐identification of LSTs by health care professionals involved in the cases and as such could have missed LSTs that might have been detected by, for example, formal human factors analysis. Fourth, as an observational study, we cannot objectively demonstrate that ISS improved LST detection, and it is possible that other patient safety interventions or QA processes could have had a similar effect on detection and subsequent mitigation. Similarly, our study set out explicitly to find LSTs, and our system‐integrated approach to case selection meant that in some instances we deliberately set out to impart stress, through the cases selected, on areas of suspected vulnerability; as such, our findings suffer from confirmation bias in influencing the identification of LSTs by team members. Finally, and importantly, the detection and reporting of LSTs has not been shown in any simulation‐based study to definitively improve upon real patient‐oriented outcomes (e.g., process improvements leading to objectively more efficient/effective care, morbidity and mortality, changes in departmental safety issue reporting rates), so while intuitively the increased interception of LSTs should improve clinical care, more work needs to be done to justify the considerable investment of time and resources into such a program.

Conclusion

We found that in our tertiary care, academic EDs, in situ simulation has been effective as a “stress test” of staff, space, and systems, revealing a high frequency of actionable latent gaps that endanger safe patient care. Further research is still required to understand if the actionable changes in fact decrease the frequency of latent safety threats occurring in real clinical care and whether this modality is more efficient in uncovering latent safety threats and linking these to improved patient outcomes. Our findings add to the growing body of evidence that in situ simulation can be used as a tool to enhance quality and improve the care ED teams are able to provide for their sickest patients.

Supporting information

Data Supplement S1. Supplemental material.

AEM Education and Training 2020;4:254–261