Simulated airway drills as a tool to measure and guide improvements in endotracheal intubation preparation in the paediatric emergency department

Abstract

Introduction

Emergent paediatric intubation is an infrequent but high-stakes procedure in the paediatric emergency department (PED). Successful intubations depend on efficient and accurate preparation. The aim of this study was to use airway drills (brief in-situ simulations) to identify gaps in our paediatric endotracheal intubation preparation process, to improve on our process and to demonstrate sustainability of these improvements over time in a new staff cohort.

Method

This was a single-centre, simulation-based improvement study. Baseline simulated airway drills were used to identify barriers in our airway preparation process. Drills were scored for time and accuracy on an iteratively developed 16-item rubric. Interventions were identified and their impact was measured using simulated airway drills. Statistical analysis was performed using unpaired t-tests between the three data collection periods.

Results

Twenty-five simulated airway drills identified gaps in our airway preparation process and served as our baseline performance. The main problem identified was that staff members had difficulty locating essential airway equipment. Therefore, we optimised and implemented a weight-based airway cart. We demonstrated significant improvement and sustainability in the accuracy of obtaining essential airway equipment from baseline to postintervention (62% vs 74%; p=0.014), and postintervention to sustainability periods (74% vs 77%; p=0.573). Similarly, we decreased and sustained the time (in seconds) required to prepare for a paediatric intubation from baseline to postintervention (173 vs 109; p=0.001) and postintervention to sustainability (109 vs 103; p=0.576).

Conclusions

Simulated airway drills can be used as a tool to identify process gaps, measure and improve paediatric intubation readiness.

what is already known on this subject.

• Emergent endotracheal intubation is a complex and infrequent critical procedure performed in the paediatric emergency department.

• Development of paediatric emergent intubation proficiency and ongoing skills retention is often challenging for emergency providers given the rare occurrence in the clinical practice.

What this study adds.

• We demonstrated that brief, in-situ simulated airway assessments can be utilised as an effective tool to assess paediatric intubation readiness, identify process gaps and improve paediatric emergent intubation preparation.

• Through integration of reliable interventions such as a weight-based airway equipment cart, educational outreach and in-situ airway simulation drills, our study shows significant process improvements and sustainability in the accuracy of obtaining essential airway equipment and in the time required to prepare for an emergency paediatric intubation.

Background

Emergent paediatric endotracheal intubation is an infrequent, high-risk procedure and carries a risk of significant morbidity and mortality.^1–4 Known peri-intubation adverse events include bradycardia, oesophageal intubation and hypoxemia.^2–5 While some of these adverse events may be related to technical skill, there are other more complex issues that can impact patient outcomes. Suboptimal communication, varying personnel experience and teamwork breakdowns may contribute in significant process variation resulting in adverse clinical care.^{2 6–8} Given the time sensitive nature of the procedure, preparing accurately and efficiently has been shown to contribute to the safety of this life saving procedure.⁹

The relative rarity of occurrence of emergent paediatric intubations in the clinical setting makes training, development of proficiency and maintenance of skill sets challenging for emergency providers.^{1 3 10} As such, alternative strategies are critical to ensure both individual and team-based competence in paediatric advanced airway management. There are no known protocols that alone reduce the risk of intubation-associated adverse events, though available data stress the importance of standardising the processes in one’s practice setting.^{2 7}

When faced with rare, high stakes events, medical simulation, especially when run ‘in-situ’ or within the actual practice setting, plays an important role by mimicking realistic environments. As such, simulation education has been used in multiple specialties with studies purporting its benefit to learning and for uncovering latent safety concerns.^11–16

Paediatric emergent intubation requires immediate access to age-specific equipment and timely airway emergency response can be delayed by the failure to rapidly obtain the essential equipment.¹⁷ Previous studies reported that consolidation of appropriate emergency airway supplies into a specialised cart has been shown to improve efficacy and response time in managing paediatric airway emergencies.¹⁸

Areas for improvement in our paediatric emergency intubation process were perceived in our pediatric emergency department (PED). These included disorganisation of essential airway equipment and staff educational gaps around airway adjuncts, both of which negatively impacted our preparation for an airway emergency.

As such, the aim of this study was to use simulated airway drills to objectively identify gaps in our paediatric endotracheal intubation preparation process, to improve preparation for intubation and to demonstrate sustainability of our improvements both over time and among a dynamic emergency department team.

Methods

Setting

The study took place at the Yale-New Haven Children’s Hospital paediatric emergency department (PED) between March 2018 and January 2020. Study time periods were as follows. The baseline period was in May 2018; the postintervention period was December 2018 and the sustainability period was December 2019.

Ethical considerations

This study was exempt from IRB review as it met institutional criteria for a quality improvement project. Participation was voluntary and all data were anonymised.

Participants

Study participants were Yale-New Haven Children’s Hospital paediatric emergency medicine attendings, fellows and technical associates (technicians). At our institution, it is their shared responsibility to prepare equipment for intubations.

Assessment

Our paediatric endotracheal intubation preparation process was assessed using brief in-situ simulation assessments, referred to as ‘airway drills’. Each airway drill began with a mock patch from an Emergency Medical Services colleague reporting the age of the patient and the need for immediate management of respiratory failure on arrival to the PED (online supplemental appendix A). The drill asked participants to gather what they deemed essential airway equipment. The simulated airway drill ended when the participant stated ‘I’m ready to intubate’. Airway drill performance was evaluated for accuracy based on an iteratively developed 16-item rubric and for time in seconds (online supplemental appendix A). The 16-itemacy rubric was adapted from previously published paediatric emergency medicine peri-intubation checklists.^{7 19 20} Additionally, raters sketched the pathways participants took during baseline drills to create process maps (figure 1). Postintervention airway drills were conducted and process maps were again generated to quantify and visually depict the impact of the interventions after 6 months. A third series of simulated airway drills were completed a year later to test if our improvements sustained over time, despite the rarity of real clinical practice and despite several personnel changes in the PED team composition.

Figure 1.

(A–D) Baseline assessment pathway map. Examples of pathways team members took in the resuscitation room while gathering essential paediatric airway equipment. Numbered yellow boxes indicate the item searched for and arrows mark the path.

Interventions

Interventions were identified and selected by the interdisciplinary team (paediatric emergency medicine faculty, fellows, nurses and technicians), who openly discussed the local challenges through analysis of our baseline results. Interventions were developed and tracked through creation of a key driver diagram that prioritised change strategies of high reliability (figure 2).

Figure 2.

Identified performance gaps and interventions diagram.

Our first and most significant intervention was the optimisation and implementation of a weight-based airway equipment cart. Faculty and staff were educated regarding the contents and organisation of this cart through staff meetings, educational emails and simulations outside of those performed in this improvement work. Subsequently, we embarked on targeted educational initiatives focusing on our airway adjunctive equipment through one-on-one individual trainings and distribution of short educational videos. Furthermore, staff was provided updates on our progress in our weekly staff newsletter and at staff meetings.

Statistical analysis

Descriptive data were used to report on our subjects and how the composition of the study participants changed over time. Primary outcomes were the means and SD and were compared via non-paired t-tests. Similarly, postintervention and sustainability sample outcomes were compared via non-paired t-tests.

Results

Twenty-five in-situ simulated airway drills were completed in the baseline period to establish our baseline process performance. The makeup of the participants during each assessment period is reflected in table 1. New participants or those completing in their first ‘airway drill’ made up 25% (5/20) of the postintervention sample and 35% (7/20) of the sustain intervention sample.

Table 1.

Participant characteristics during each assessment periods

	Baseline N (%)	Postintervention N (%)	Sustainability N (%)
Role
Attending MD	11 (44)	9 (45)	8 (40)
Fellow MD	4 (16)	3 (15)	5 (25)
ED technician	10 (40)	8 (40)	7 (35)
Total	25 (100)	20 (100)	20 (100)

Baseline assessment

Overall, the accuracy of completing all required tasks on the 16-point airway preparation rubric was 62% (SD 18%) and the average time to prepare for intubation was 173 s (SD 70 s). Process maps demonstrated how participants navigated the resuscitation room while gathering essential paediatric airway equipment (figure 1).

Baseline analysis

We identified two main challenges to our airway management preparation process:

1. As noted in the SD of our primary outcomes and depicted on the baseline process maps, variable preparation practices were evident among our participants in both time to readiness and accuracy. We addressed these findings by using the baseline ‘drills’ as educational moments for the individual provider and integrating the use of the airway cart into our monthly multidisciplinary in-situ case simulation series. Additionally, the study team educated PED staff through brief instructional videos, including ‘airway tips of the week’ in our departmental weekly newsletter and providing airway updates at our monthly PED staff meetings.

2. Locating essential airway equipment was inefficient. Again, as evident from our process mapping (figure 1), key equipment was stocked in multiple locations within the resuscitation room. We addressed this challenge by reorganising essential airway equipment into a weight-based, colour-coded equipment cart, which allowed most necessary equipment to be ready immediately when the appropriate drawer was opened. Contents of the drawer included weight-based unopened endotracheal tubes (two sizes available), stylet, nasal and oral airway adjuncts, suction catheter, nasogastric tube, laryngeal mask airway and back-up direct (straight and curved) laryngoscope blades (figure 3).

Figure 3.

Airway cart drawer contents.

Post-intervention assessment

Six months after this baseline assessment phase, we again used simulated airway drills to measure the effects of our intervention. In the postintervention period, 20 drills were performed and accuracy improved to 74%, a 12% improvement from baseline (p=0.014). The time to prepare for intubation was 109 s, a 37% improvement (p=0.000). Furthermore, the variation in performance among participants improved. We noted a decrease in the accuracy SD from 18% to 13% and a decrease in the time to prepare SD from 70 to 33 s.

Regarding locating essential equipment, all participants took a more focused path directly to the airway cart where all equipment was found eliminating the variety of pathways we had seen from the baseline assessment (figures 1 and 4).

Figure 4.

Postassessment pathway map. Focused path directly to the airway cart after intervention.

Sustainability assessment

In the sustain–intervention period, 20 drills were performed and accuracy remained consistent to postintervention performance at 77% (p=0.573). The time to prepare for intubation was also stable at 103 s (p=0.576). Additionally, the improvements in variation among participants in accuracy and time to prepare equipment remained similar to postintervention measurements.

Results of accuracy and time for intubation preparation during the three assessment periods are shown in figure 5A and B, respectively. Additionally, we have found that syringe and correct size of a non-rebreather mask were the most commonly omitted airway equipment during each simulated airway assessment period (table 2).

Figure 5.

(A and B) Accuracy and time to endotracheal intubation preparation.

Table 2.

Airway equipment deficiency during each assessment periods

	Baseline N (%)	Postintervention N (%)	Sustainability N (%)
Role
Attending MD	11 (44)	9 (45)	8 (40)
Fellow MD	4 (16)	3 (15)	5 (25)
ED technician	10 (40)	8 (40)	7 (35)
Total	25 (100)	20 (100)	20 (100)

ED, Emergency Department; MD, Medical Doctor; PED, Pediatric Emergency Department.

Discussion

This study aimed to improve our intubation preparation process as measured by the accuracy of essential equipment gathered and the time needed to gather the equipment. Importantly, we also desired to demonstrate that without additional interventions or explicit training, we had created a sustainable process improvement that both lasted over time and worked with new teammates, key factors in the field of emergency medicine.

Our study has three main results. We were able to (1) improve the accuracy of obtaining essential airway management equipment by 12%, (2) decrease the time required to prepare for a paediatric intubation by 37% and (3) sustain these improvements.

First, over the course of the study, the accuracy of obtaining essential airway management equipment improved significantly. In recent years, healthcare simulation has been shown to be an excellent tool for effective education and training, especially when performed in-situ.^{21 22} This particular type of simulation can serve as an assessment tool for process development and uncover latent safety issues.¹⁶ Melzer et al described the utilisation of a paediatric emergent airway kit and implementation with simulation significantly reduced the mean time of otolaryngology resident arrival by 47% and mean time of successful intubation by 50%.⁹ Similarly, we were able to demonstrate that short in-situ simulations helped prepare our team for paediatric intubations by uncovering gaps in preparedness. Furthermore, by reviewing trends of the most common airway equipment omitted during the simulation assessment, we identified areas of recurring equipment deficiency which will improve future paediatric intubations. Future simulated airway assessments incorporating a peri-intubation checklist and ongoing airway cart modification based on recurring equipment deficiencies may further advance our performance.

Second, the time to preparedness decreased significantly by 37%. Emergent paediatric intubations are a time-sensitive procedure with varying first-pass success rates—especially when intubations are performed by trainees.⁵ Preparing for intubations efficiently and effectively having the correct equipment available more quickly could allow the person intubating to focus on the intubation task. Further studies may inquire how improvements in preparation correlate with first pass success rates.

Third, the improvement in accuracy of equipment preparation and time to preparation was shown to be sustained. This shows that interventions with high reliability, such as our airway cart, directly addressed the uncovered problem of not centrally locating essential equipment. Further, these interventions created a system that was easy for established staff to use and new staff to learn. Frankel et al suggested that process and system reliability can be achieved by applying the fundamental principles of standardisation and simplification to minimise non-patient-specific variation.²³ Our team simplified the preparation of essential airway equipment, making it less time consuming and facilitating staff assimilation to our airway process. Had we tested the same cohort all three times, we likely would have seen continued improvements in our outcomes. However, we purposely assessed the varied cohorts as this is reality in emergency care teams and speaks to the strength of our new system. Furthermore, the decreased variation in the SD in time to prepare may further support the strength of this intervention, signifying that a more reliable process incorporated into our emergency department.

Finally, we discovered the value of drills to prepare for the rare but critical event.²⁴ Brief drills allowed for uncovering of process deficits, individual teaching opportunities and engaging with learners in spaced out, deliberate practice over the study period.^{25 26} There are several paediatric emergency medicine events that meet the criteria of low frequency, high risk. Future studies may apply elements of our approach to situations like disaster preparedness or severe trauma management.

Limitations

This study has several limitations. This is a simulation-based trial and we did not assess improvements in preparation for real paediatric intubations in the PED. In the future, preparation accuracy and time to preparation need to be assessed in real patients for a better understanding of the translation of in-situ airway drills to clinical practice. This could be accomplished by having an observer for each intubation that is prepared for—even if the actual procedure is not executed. Our main methodologic limitation is that we did not pilot-test our simulation case or validate the 16-point accuracy scoring rubric. Our simulation-cases only consisted of one sentence and we therefore decided that pilot testing was not necessary. The accuracy rubric is based on published peri-intubation checklists used in several PEDs.^{19 20} The main bias of our study is the Hawthorne effect. Our participants knew they were being assessed. This could be overcome by routinely assessing real intubation preparations rather than in-situ simulations or randomly running drills on a regular basis. Furthermore, despite having postintervention results, our team did not implement additional changes to the airway cart prior to the sustainability phase assessment. We theorised that the accuracy outcome could have further improved during the sustainability phase if additional modifications were applied to the airway cart based on postintervention results. Finally, our single institution has a unique shared-responsibility for equipment preparation among ED technicians, attending and fellow physicians. The results of this study may be difficult to generalise to clinical settings where a different division of responsibility is used.

Conclusions

Through process analysis from in-situ simulation drills, we objectively validated our perceptions of the need for improvements for paediatric intubation preparation in our PED. Integrating a reliable intervention, such as a weight-based airway equipment cart, combined with individual teaching, sharing of progress data and integrating drills into our practice, improved our preparation process. A year later, even with several new team members, we demonstrated sustainability of these improvements. To further improve performance, future directions include iterative changes to our airway equipment cart, continued use of intubation preparation drills and integration of a preintubation checklist

Data availability statement

Data are available upon reasonable request. All of the individual deidentified participant data collected during the study are available immediately available after publication.

Ethics statements

Patient consent for publication

Not required.