Characterization of teamwork and guideline compliance in prehospital neonatal resuscitation simulations

Abstract

Aim

Neonatal cardiopulmonary arrests are rare but serious events. There is limited information on compliance to best-practice guidelines due to rarity, but deviations can have dire consequences. This research aimed to characterize compliance with and deviations from Neonatal Resuscitation Program (NRP) guidelines and their association with teamwork.

Methods

We observed Emergency Medical Service (EMS) teams responding to standardized neonatal resuscitation simulations following a precipitous home delivery. A Clinical expert evaluated teamwork during simulations using the Clinical Teamwork Scale (CTS™). A neonatologist evaluated technical performance in blinded video review according to NRP guidelines. We report the types, counts, and severity of observed deviations. Logistic regression tested the association of CTS™ factors with the occurrence of deviations.

Results

Forty-five (45) teams of 265 EMS personnel from fire and transport agencies participated in the simulations. Eighty-seven percent (39/45) of teams were rated as having good teamwork according to CTS™. Nearly all teams (44 of 45) delayed or did not perform one or more of the initial steps of dry, warm, or stimulate; delayed bag-valve mask ventilation (BVM); or performed continuous compressions instead of the recommended 3:1 compression-to-ventilation ratio. Logistic regression revealed an 82% (p < 0.04) decrease in the odds of airway errors for each level of improvement in teams’ decision-making.

Conclusion

Drying, warming, and stimulating, and ventilation tailored to the physiologic needs of infants continue to be top priorities in neonatal care for out-of-hospital settings. EMS teamwork is good and higher quality of decision-making appears to decrease the odds of ventilation errors.

Introduction

Every year, approximately 4 million babies are born in the United States and 64,000 infants are born out of the hospital.1., 2. Out-of-hospital births are at increased risk for morbidity and mortality.3., 4. Ten percent of babies born will require some form of resuscitation.5., 6., 7., 8. According to a recent National Academy of Medicine report, the number of out-of-hospital births is growing. Thus, it is important to understand and optimize the quality of neonatal resuscitation efforts out-of-hospital.

The transition from intrauterine to extrauterine life involves major physiological adjustments that must be supported in a timely and effective manner. Neonatal Resuscitation Program (NRP) guidelines⁹ were established by the American Academy of Pediatrics (AAP), American Heart Association (AHA), and International Liaison Committee on Resuscitation (ILCOR) to provide lifesaving care when emergencies occur and improve birth outcomes.10., 11., 12., 13.

Observational studies of NRP in the hospital setting have identified important gaps in care that need improvement: assessment; initial steps of drying, warming, and stimulating; monitoring and communication of heart rate; and awareness of clinical deterioration.14., 15., 16., 17. Chart reviews of hospital records have found that paramedics frequently encounter complications due to maternal comorbidities and extreme preterm births. 18., 19., 20., 21. Furthermore, inconsistent and absent documentation suggests that key procedures, such as warming, are not performed.²⁰ It has been recognized that many paramedics do not receive training in neonatal resuscitation and the rarity of cases prevents acquisition and maintenance of skills.¹⁸

There is limited information on the quality of out-of-hospital NRP care. Recognizing that exposures to suboptimal care have the potential for lifelong consequences, it is important to understand care and identify opportunities to support Emergency Medical Service (EMS) agencies in providing care to newborn infants. Therefore, we employed simulations to observe the performance of EMS teams in standardized scenarios and identify opportunities for improving knowledge, skills, and resources in this infrequent but critical event.

Methods

We conducted a secondary observational analysis of teamwork and technical performance in NRP simulations with EMS responders. This is an extension of an earlier study, where we evaluated performance of the first critical steps of NRP: dry, warm, stimulate, and bag-valve mask ventilation.²² This paper focuses on the effects of teamwork on compliance with NRP guidelines across the entire resuscitation. This study was funded by the US National Institutes of Child Health and Human Development (NICHD R01HD062478) and Agency for Healthcare Research and Quality (AHRQ F32HS025590), and it was approved by Oregon Health & Science University’s Institutional Review Board (IRB# 00006942). We obtained voluntary informed consent, which included disclosure of video release and confidentiality agreement. After consent, participants filled out a questionnaire regarding demographic information and NRP experience. They were then oriented on the functionality of the manikin but not the details of the scenario they would be responding to.

Simulation scenario

EMS teams were dispatched to a home for an unplanned birth in progress. Teams arrived in their own ambulances and fire trucks and used their own equipment. They arrived to a father holding a cyanotic and apneic newborn manikin that was wet and covered in vernix, umbilical cord cut; also in the room was the mother (Gaumard Noelle® manikin) on a bed, placenta delivered. Professional actors played the role of parents and a high-fidelity patient simulator, Newborn HAL® from Gaumard, served as the newborn patient.

The newborn was presented as limp, with peripheral cyanosis, and inadequate respirations. Optimal management was defined as performing the initial steps of dry, warm, stimulate, and BVM ventilation within the first minute of care. The manikin was programmed to improve if the appropriate care was delivered and go into cardiac arrest if not. This simulation was designed to last 10 minutes and the full details of the scenario are presented in the supplemental materials.

Teamwork evaluation

A clinical expert with extensive experience in using the Clinical Teamwork Scale (CTS™)²³ directly observed simulations in real time and measured teamwork using this instrument. The CTS™ instrument rated teamwork overall and across 4 major domains for communication, situational awareness, decision-making, and role responsibility using a 10-point Likert scale: 0 = unacceptable, 1–3 = poor, 4–6 = average, 7–9 = good, and 10 = perfect.

NRP technical performance evaluation

A neonatologist independently reviewed videos of the teams and evaluated technical performance using a predetermined structured intake form.²² The intake form captured information about NRP tasks such as, time performed, if they were done correctly, and attributes of performance. For example, we would track that a team incorrectly performed bag-valve mask ventilation (BVM) at 1:20 (mm:ss) using an adult-sized Ambu® bag, pushing full volume at a slow rate. Attributes that could not be measured directly, such as volume of air ventilated and rate of compressions, were subjectively rated by the neonatologist. Established criteria identified and rated deviations based on whether tasks were neglected, delayed, or performed incorrectly. See supplemental materials for technical performance data collection forms and criteria.

A taxonomy was developed prior to analysis to classify harms. A harm was coded as mild if an action could cause temporary adverse effects and severe for permanent effects, including death. For example, immediately providing bag-valve mask ventilation (BVM) for a hypoxic newborn would cause no harm, but delaying for 1 minute would cause mild harm and >2 minutes severe harm. The rules were grouped according to the therapeutic intent of the various tasks: assessment; dry, warm, and stimulate; airway; cardiopulmonary resuscitation (CPR); and medications. See supplemental materials for rules.

Statistical analysis

We used logistic regression in R 4.0.0 (https://www.r-project.org) to test the association of teamwork scores on deviations from NRP guidelines. The scores for each of the CTS™ domains (overall teamwork, communication, situational awareness, decision-making, role responsibility) were collapsed to three-levels: low = 0–4, medium = 5–7, and high = 7–10. This was done to avoid the assumption that the intervals between the integer performance scores are equal across the scales range. The dependent variables, deviations from NRP guidelines, were grouped according to potential harm (mild, severe, or any harm). We coded each harm group as a binomial variable indicating 0 = no deviation observed or 1 = the occurrence of one or more deviations.

A total of 15 models were constructed, one for each of the five resuscitation procedures (assessment; dry, warm, and stimulate; airway; CPR; and medications) and for each of the three categories of potential harm (mild, severe, and any). The best-subset strategy was used to select the “best” models according to the Akaike Information Criterion (AIC).

Results

Table 1 shows participant demographics, where we recruited 265 EMS providers organized into 45 teams. Each team responded to an NRP scenario. Teams were composed fire and transport crews who were working on the days of the simulations. This enhanced fidelity of the simulations because it reflects the dual-response system providers work in. There was a mean (±SD) of 4 (±1) fire members and 2 (±1) transport members per team, and each team had at least 1 member with paramedic training. One-half (133) of the providers had paramedic training while the other half (128) had Emergency Medical Technician (EMT) training. Approximately half of the responders reported having NRP training within the past 2 years and 26% reported never having NRP training.

Table 1.

Individual and team level characteristics.

Individual Characteristics	No. (%)
Total participants	265
Age, mean (SD), years*	37 (9)
Gender male, N (%)	225 (85)
Race white, N (%)	226 (85)
Level of training, N (%)
EMT-Intermediate	26 (10)
EMT	101 (38)
Advanced EMT	2 (1)
Paramedic	133 (50)
No answer	3 (1)
Years worked at current level of training*	9 (7)
Years worked in EMS*	12 (8)
Time since NRP training, N (%)
≤2 years	128 (48)
>2 years	36 (14)
No training	69 (26)
No answer	32 (12)
Team Characteristics
Total teams	45
Composition
Fire crew members	4 ± 1
Transport crew members	2 ± 1

^*Mean and SD for those who answered the question. All % are computed with 265 as denominator.

Teamwork

Overall, EMS teams were rated as having relatively good teamwork. Fig. 1 shows the mean value was 6 (high average ± 2 SD) across all scales. There was a strong positive correlation among the raw values of all scales, ranging from 0.54 to 0.90. Collapsing the values into categories of low, medium, and high, shows that a majority of the teams were assigned high scores.

Fig. 1.

Distribution of CTS™ scores across domains of teamwork.

NRP technical performance

Table 2 summarizes our evaluation of technical performance, where we observed a total of 431 deviations from NRP guidelines across all teams. Among them 284 (66%) were judged to have potential for mild harm and 147 (34%) a potential for severe harm. Almost all teams, 98% (44/45), had suboptimal performance with warming, BVM timing, and CPR technique. For CPR, 51% (23/45) of teams performed continuous compressions with asynchronous ventilation instead of the recommended 3:1 coordinated compression-to-ventilation ratio.

Table 2.

Deviations from NRP guidelines.

Error	Mild	Severe	Overall
	n/N (%)	n/N (%)	n/N (%)
Assessment
Verbalize heart rate <100	8/45 (18)	NA	8/45 (45)
Attach pulse oximeter	26/45 (58)	NA	*26/45 (58)
Attach ECG	27/45 (60)	NA	*27/45 (60)
Measure length/weight	21/45 (47)	NA	21/45 (47)
Use pediatric guide	7/45 (16)	NA	7/45 (16)
Subtotal	89/270 (33)	NA	89/270 (33)
Dry, warm, stimulate
Dry	13/45 (29)	14/45 (31)	*27/45 (60)
Warm	10/45 (22)	34/45 (76)	**44/45 (98)
Stimulate	6/45 (13)	31/45 (69)	*37/45 (82)
Subtotal	29/135 (21)	79/135 (56)	108/135 (80)
Airway
First airway (BVM or blow-by oxygen)	NA	1/45 (2)	1/45 (2)
BVM rate	29/45 (64)	NA	*29/45 (64)
BVM technique	NA	12/45 (27)	12/45 (27)
BVM timing	39/45 (87)	5/45 (11)	**44/45 (98)
Assess adequate ventilation	17/45 (38)	NA	17/45 (38)
Intubation	2/30 (7)	2/30 (7)	2/30 (7)
Verbalize tube depth	23/30 (77)	3/30 (10)	*26/30 (87)
Verbalize tube size	20/30 (67)	NA	20/30 (67)
Subtotal	130/225 (58)	21/195 (11)	151/315 (48)
CPR
CPR technique	21/45 (47)	23/45 (51)	**44/45 (98)
CPR swap	8/45 (18)	21/45 (47)	*29/45 (64)
Subtotal	29/90 (32)	44/90 (49)	73/90 (81)
Drugs
Verbalize epinephrine	4/15 (27)	NA	4/15 (27)
Administer epinephrine	2/11 (18)	3/11 (28)	5/11 (45)
Subtotal	6/25 (24)	3/11 (27)	9/25 (36)
Totals	284/745 (38)	147/431 (34)	431/835 (52)

Note, N < 45 for some rules because the preconditions were not met, e.g. intubation and epinephrine were not always performed.

^**Error present in >90% of teams.

^*Error present in >50% of teams.

A majority of teams had difficulty in other areas as well, including attaching pulse oximetry 58% (26/45) and electrocardiograph monitors 60% (27/45) in a timely manner, drying 60% (27/45) and stimulating 82% (37/45), ventilating at an appropriate rate 64% (29/45), verbalizing intubation details 87% (26/30), and swapping chest compression roles every two minutes 64% (29/45). Teams had good performance for some tasks, such as recognizing a heart rate <100 beats per minute, supporting airway with BVM first, and keeping intubation attempts below 2. Less than a quarter (24% or 11 of 45) of teams administered epinephrine when indicated, and almost half of those who administered epinephrine (45% or 5 of 11) administered an incorrect dose.

The effect of teamwork on deviations from NRP guidelines

Logistic regression models were used to explore associations between NRP deviations and teamwork. There was a significant association for decision-making level and deviations in airway procedures that had the potential to cause severe harm. The odds of committing a severe procedural error was 88% less if teams scored higher in clinical decision-making (OR = 0.12, 95% C.I. = 0.02–0.087, p < 0.04). The coefficients of the covariates for overall teamwork and communication were not significant, however their inclusion improved explanatory power according to the AIC diagnostics (AIC = 63.1). Aside from this instance, there was no relation between other NRP deviations and any CTS™ factor.

Discussion

EMS teams deviated from NRP guidelines despite having good teamwork. Teams delayed or neglected to perform the initial steps of dry, warm, and stimulate within 5 minutes of arrival, with rates of 31%, 75%, and 69% respectively. These steps are essential because they help maintain normothermia and initiate respirations. Drying reduces heat loss from the evaporation of amniotic fluid. Dry towels help reduce heat loss when exposed to the surrounding air. If these tasks are not performed, the baby undergoes vasoconstriction to conserve heat, which limits tissue oxygenation and increases risks of mortality.²⁴

The time to perform BVM was also delayed by up to a minute in most teams 87% (39/45) but this could be attributed to getting oriented to the scenario. Importantly, teams had some difficulty pushing the smaller volumes of air because they only carried pediatric- and adult-sized bags, which is an important threat to patient safety. First, larger bags make it difficult to accurately push air into infant-sized lungs. Second, the masks may be too big to form a proper seal on the infant’s face. Air can escape and providers may squeeze the bags more to compensate. This is very important as studies suggest that excessive volume may be more important in causing lung injury than pressure.²⁵ Also, teams ventilated at a slower rate than was appropriate for a newborn, bagging at a rate more consistent with a larger child or adult. Together, these problems can worsen underlying symptoms and impede resuscitation efforts.

For CPR, many teams performed continuous compressions with asynchronous ventilations instead of at the recommended 3:1 ratio. Continuous compressions are recommended in adult patients because their arrest is often cardiac in origin and interruptions decrease perfusion pressure.²⁶ In neonates and pediatric patients, cardiac arrest is driven by hypoxia. Blood continues to flow and can deplete the pulmonary oxygen reservoir, thus competing with ventilations.26., 27. Experts currently do not recommend continuous compressions for non-intubated pediatric or neonatal patients.27., 28.

We used logistic regression to test the relationship between teamwork factors and deviations from NRP guidelines. The results of this study suggest that compliance with NRP guidelines were largely unaffected by teamwork. Among teamwork elements, only decision-making was associated with deviations from NRP guidelines for airway management. Teams exhibited good decision-making when they recognized the hypoxic origin of the newborn’s condition and prioritized airway accordingly. It is perhaps more surprising that teams exhibited good communication, situational awareness, and role responsibility, but still deviated significantly from NRP guidelines. This suggests that mechanisms to increase knowledge about steps in providing NRP care in the out-of-hospital setting are important, particularly the initial steps of drying, warming, and stimulating.

Similar to reports in hospital medicine, our findings agree that the initial steps of neonatal resuscitation are often neglected or not performed to a sufficient level of quality.8., 9., 11., 12. We have not determined why this occurs, but it could be attributed to a variety of reasons including lack of knowledge, focus on other tasks, or lapse in decision-making. Other tasks may be preferred because they directly address more prominent symptoms and may align with other resuscitation algorithms that are more familiar, such as Pediatric Advanced Life Support (PALS) or Advanced Cardiac Life Support (ACLS).

CPR quality for neonates in the out-of-hospital setting is a topic that, to the best of our knowledge, does not appear to be addressed in the literature. The NRP guidelines recommend a 3:1 compression-to-ventilation ratio. Almost all teams performed continuous compressions, but this technique has only been supported in the presence of sustained inflations.²⁹ We suspect that this behavior is carried over from adult algorithms, where this practice is the norm. This raises the implication that paramedics and EMTs may fall back on familiar training if they lack specialized knowledge for neonates.

This study has several limitations. First, it is based on simulation and not patient outcomes. Second, although we used a high-fidelity manikin, manikins by design lack several key features that serve as cues to perform critical tasks, such as skin temperature. We attempted to give as many cues as possible by turning on blue LED lights in the manikin’s face to indicate onset of hypoxia, creating wet simulated vernix to give a clue for the need to dry, and having the confederate actors provide details and hints. Lastly, some of our evaluation criteria depended on subjective interpretation. This evaluation was conducted by an experienced neonatologist and NRP instructor who regularly assesses clinicians for NRP certification.

We have observed that, similar to the hospital setting¹⁴, deviations from NRP guidelines are frequent and have identified several promising areas for improvement. The importance of warming and ventilation can be emphasized and CPR technique refined for these situations. The challenge will be in finding room for improvement in a population that is saturated with training requirements.

Conclusion

The prehospital setting is unique and important in the initial resuscitation of newborns. The rarity of neonatal cardiorespiratory arrests can make it difficult to maintain the appropriate knowledge and skills, which can increase deviations from NRP guidelines and potentially lead to patient harm. We identified several areas of care that could be improved: emphasis on the initial steps of dry, warm, stimulate; use of appropriate-sized equipment; and performing 3:1 compressions. A simple solution would be to provide more training and decision tools, but it is likely that even more is needed to ensure that the system is resilient to inevitable human errors and deterioration in knowledge.

Funding source

Supported by the National Institute of Child Health and Human Development grant R01HD062478 awarded to Jeanne-Marie Guise, MD and the Agency for Healthcare Research and Quality grant F32HS025590 awarded to Nathan Bahr, PhD.

Role of funder/sponsor

The funder/sponsor did not participate in this work.

Financial disclosure

The authors have no financial relationships relevant to this article to disclose.

Contributors’ statement

Nathan Bahr, PhD, participated in data collection and analysis as well as writing and editing the manuscript.

Trang Huynh, MD, participated in evaluation of the simulation videos, provided content expertise, and editing the manuscript.

William Lambert, PhD, provided statistical expertise and participated in editing the manuscript.

Jeanne-Marie Guise participated in the design and conception of the study, evaluation of teamwork, provided content expertise, and edited the manuscript.

All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Appendix A. Supplementary material

The following are the Supplementary data to this article: