Effect of a positive pressure ventilation-refresher program on ventilation skill performance during simulated newborn resuscitation

Abstract

Aim

Clinical staff highly proficient in neonatal resuscitation are essential to ensure prompt, effective positive pressure ventilation (PPV) for infants that do not breathe spontaneously after birth. However, it is well-documented that resuscitation competency is transient after standard training. We hypothesized that brief, repeated PPV psychomotor skill refresher training would improve PPV performance for newborn care nurses.

Methods

Subjects completed a blinded baseline and post PPV-skills assessment. Data on volume and rate for each ventilation was recorded. After baseline assessment, subjects completed PPV-Refreshers over 3 months consisting of psychomotor skill training using a newborn manikin with visual feedback. Subjects provided PPV until they could deliver ≥30 s of PPV meeting targets for volume (10−21 mL) and rate (40–60 ventilations per minute [vpm]). Baseline and post assessments were compared for total number PPV delivered, number target PPV delivered (volume 10−21 mL), mean volume and mean rate (Wilcoxon signed-rank test, median[IQR]).

Results

Twenty-six subjects were enrolled and completed a baseline assessment; 24 (92%) completed a post-assessment; 2 (8%) were lost to follow-up. Over 3 months, a mean 3.2 (range 1–6) PPV-Refreshers/subject were completed. Compared to baseline, subjects demonstrated significant improvement for total (57 [36–74] vs. 33 [26–46]; p = 0.0007) and target PPV (23 [13–23] vs. 11 [5–21]; p = 0.024), and a significant change in mean volume (mL) (11.5 [10.2–13] vs. 13.4 [11–16]; p = 0.02) and mean rate (vpm) (54 [45–61] vs. 40 [28–49]; p = 0.019).

Conclusions

A PPV-Refresher program with brief, repeated psychomotor skill practice significantly improved PPV performance with the greatest improvement in total PPV and target PPV. Additional investigation is warranted to determine optimal PPV-Refresher frequency.

Registered at ClinicalTrials.gov #NCT02347241

Introduction

Globally, it is estimated that approximately 10% of all newborns do not breathe on their own or respond to the initial steps of stimulation after birth.[1], [2], [3] Newborn healthcare providers must be well-trained and skilled to initiate the delivery of adequate and effective positive pressure ventilation (PPV) during this transition period to ensure newborn survival and reduce long-term morbidity.[4], [5], [6] If this respiratory support is not performed in a timely or adequate manner, the progression of birth asphyxia to cardiovascular collapse and death may ensue.[4], [7] Unfortunately, exposure to neonatal resuscitation and training can be variable among front-line providers and it is well documented that resuscitation training retention is poor, with decline of psychomotor skill performance reported as soon as 2–12 weeks after training.[8], [9] Additionally, provider anxiety, operator position, challenges with appropriate use of equipment, and the ability to implement appropriate corrective strategies if the initial rescue breaths are not effective are significant barriers to the timely and adequate delivery of effective PPV.¹⁰

Just-in-time and just-in-place[11], [12], [13], [14] simulation training has been shown to improve acquisition and retention of resuscitation skills.[1], [4], [15], [16], [17] Furthermore, various studies have reported that implementation of a training program consisting of high-frequency, short-duration, psychomotor skill “Refreshers” can improve resuscitation skills when utilized as a supplement to standard resuscitation education.[18], [19], [20], [21], [22], [23] Therefore, our objective was to assess the effectiveness of a neonatal resuscitation simulation educational program featuring brief, repeated PPV-Refresher psychomotor skill practice to improve PPV performance among front-line nurses caring for newborns. We hypothesized that neonatal staff nurses would have improved PPV skill performance after participation. Secondary objectives were to determine if there is an association of PPV skill performance with number of PPV-Refresher sessions completed.

Methods

This prospective, educational intervention was conducted in the Labor and Delivery Unit (LDU) and Intensive Care Nursery (ICN) in a tertiary academic hospital. The LDU has 14 beds and three obstetric operating rooms with >4000 deliveries per year. Standard deliveries in the LDU are attended by at least two obstetricians and at least one obstetric registered nurse, also responsible for assessment and care of low-risk newborns. The 36-bed level III Intensive Care Nursery (ICN) has >800 admissions per year. A neonatal resuscitation team, consisting of at least two neonatal medical providers (attending, fellow, or resident) and an ICN nurse, is called to attend all term cesarean-section deliveries and vaginal deliveries pre-identified as high risk or that experience unanticipated complications during or after delivery.

Staff and standard training

Nurses from the ICN (neonatal specialization) and LDU (obstetric specialization) were invited to participate in the study. All providers in these units are required to maintain active Neonatal Resuscitation Program (NRP) status and complete an NRP training program every two years.⁴ Nurses in both the ICN and LDU are offered supplementary NRP skills trainings or mock code simulation trainings every 1–2 months and are completed as allowed by staffing and census. Standard PPV training for nurses includes delivery of PPV as a lone provider, as they are often the first responder in the LDU and ICN.

PPV assessments

Nurses completed a PPV skills assessment upon enrollment (Baseline) and three months after the baseline assessment. All assessments were completed using a validated prototype Resusci® Baby QCPR (Laerdal Medical, Stavanger, Norway) (Fig. 1) to allow for comprehensive data collection. Specific data on PPV inflation volume (mL) and ventilation rate (vpm) was recorded by the manikin software system, but was not visible to the providers during the assessment. The manikin airway system was comprised of an internal rubber tube connecting the mouth-lung compartment which was leak-free. By design, if the manikin’s head tilt is incorrect, the tubing will occlude, simulating an obstructed airway. The manikin airway system had a tidal volume of approximately 25 mL, and chest rise could be noted at approximately 10−12 mL. For the assessment, a standard flow-inflating bag and infant mask (Model E191, Anesthesia Associates, San Marcos, CA) with a visible built-in pressure manometer was used.

Fig. 1.

PPV Assessments: Image of infant ventilation-recording manikin (count, volume, rate), flow-inflating bag, and infant mask placed on a standard infant warmer. Each simulation session was 2 min in duration however, the last 90 s of the scenario was analyzed for PPV performance. Providers did not receive feedback on their performance during or after their assessment.

Each evaluation session began with an overview of the PPV assessment and an orientation to the resuscitation equipment and manikin. Subjects were then read a scripted resuscitation scenario featuring a full-term 3-kg newborn. The simulation scenario was two minutes in duration and PPV data collection commenced after the first 30 s, that is, the simulation session would begin, but data was not recorded for the first 30 s to allow for the subjects to provide the standard NRP stimulation and assessment. Subjects did not receive feedback on their performance during or after the assessment in order to minimize influence of the assessment on the PPV-Refresher intervention.

Intervention: PPV-Refreshers

After the Baseline assessment, nurses participated in PPV-Refreshers over three months of their regular schedule, in addition to any standard training (as described above). The mobile PPV-Refresher cart consisted of a Newborn Lung Simulator (NLS) connected to NeoNatalie™ (Laerdal Medical, Stavanger, Norway), a low-cost inflatable manikin designed to teach basic neonatal resuscitation skills, and a standard flow-inflating bag with infant mask (Fig. 2). While ventilating the manikin, a NLS console provided visual feedback of delivered PPV volume and pressure and chest rise was visible if delivery was appropriate. PPV-Refreshers were standardized in that each participant was first oriented to the PPV-Refresher cart and instructed to provide PPV to the manikin “according to NRP recommendations” until PPV competence (defined as delivered PPV volume of 10−21 mL and rate of 40–60 ventilations/minute¹ [1 vent/1–1.5 sec) could be demonstrated for at least 30 s. Each session was facilitated by a member of the study team (DN) or a designated NRP-trained educator. Facilitators also provided coaching (placement of mask, reminder of rate, volume and pressure recommendations, etc.) during the session and remediation after the session, as needed, until the provider was independently able to achieve competence.

Fig. 2.

PPV-Refreshers: Image of infant training manikin with Newborn Lung Simulator (NLS), flow-inflating bag and infant mask on a mobile cart. The console provides visual feedback on PPV volume (mL, center cylinder) and pressure (mmHg, right analogue dial). Providers practiced until PPV competence (defined as PPV volume 10-21 mL and rate 40-60 vpm) could be demonstrated for at least 30 s. Providers were coached and remediated, as needed, during their PPV-Refresher.

Nurses that were available and volunteered for training completed PPV-Refresher sessions in the hallway of the LDU (outside patient rooms) and within the ICN (aside infant warmers/beds). PPV-Refresher sessions typically lasted less than four minutes and were available at least three times per week during daytime and evening shifts. PPV-Refresher participation was tracked although specific data on PPV performance during the PPV-Refreshers was not recorded.

Analysis

Standard descriptive statistics (means and standard deviations or median and ranges for continuous variables; frequencies and percentages for categorical variables) were reported. Ventilation data during the PPV assessments was recorded for 90 s and summarized using the manikin’s standard software (SkillReporter® and SessionViewer v.6.2.5000, Laerdal Medical, Stavanger, Norway). Data included volume (mL) and rate (ventilations per minute: vpm) for each ventilation delivered during the assessment. “Target” PPV was defined as any ventilation that met a volume of 10−21 mL [4−6 mL/kg²⁴ ± 15%]). Due to system limits, ventilations with volume less than 7 mL were not recorded. PPV rate for each PPV attempt was calculated by determining the time between ventilation peaks divided by 60 s.

For statistical testing of pre-post effects, Wilcoxon paired signed-rank test was used for count and continuous variables and McNemar's test was used for categorical variables. Modeling was used to estimate the effect of the pre-post change in terms of overall change, and broken down into first PPV-Refresher and additional PPV-refreshers.²⁵ Estimates, their 95% confidence intervals (CI⁹⁵), and p-values were reported where appropriate. Differences between ICN and LDU nurses were explored by conducting the analysis stratified by these two groups, although sample size is limited for this sub analysis. A p-value of <0.05 was considered statistically significant. SAS 9.4 was used to conduct analyses.

This study was reviewed by the Institutional Review Board of the University of Pennsylvania and met criteria for exemption from IRB oversight (45 CRF §46.101, Category 1). Data collection procedures were in compliance with the Health Insurance and Portability and Accountability Act to ensure subject confidentiality. This study is registered at ClinicalTrials.gov (NCT02347241).

Results

Fig. 3 displays the enrollment and study flow diagram and demographic information is presented in Table 1. Overall, the majority of nurses reported 5 years or less experience; 50% in the ICN and 67% in the LDU. Over 3 months, 75 PPV-Refresher session were completed. All subjects completed at least one PPV Refreshers over the course of the 3 months (median 3.2; range 1–6).

Fig. 3.

Enrollment and study flow diagram.

Table 1.

Demographic information.

		ALL (n = 24)	ICN (n = 9)	LDU (n = 15)
Female		24 (100)	9 (37)	15 (63)
Experience:	1−5 years	15 (63)	5 (56)	10 (67)
	≥6 years	9 (37)	4 (44)	5 (33)
Months since NRP training		6 [4−16.5]	5 [3.5−14.5]	6.5 [4.8−15.8]

Data presented as counts and frequencies (n (%)) except for months since NRP training which is presented as median [IQR].

There were no statistically significant differences between ICN and LDU-groups.

Boxplots of PPV performance at the Baseline and Post assessments are displayed in Fig. 4. There was a significant difference in PPV skills for all performance metrics at the post assessment. Compared to baseline, subjects demonstrated significant improvement for total number of PPV delivered (57 [36–74] vs. 33 [26–46]; p = 0.0007) and for number of Target PPV delivered (23 [13–23] vs. 11 [5–21]; p = 0.024), There was also a significant change compared to the baseline assessment in mean volume (11.5 [10.2–13] mL vs. 13.4 [11–16] mL; p = 0.02) yet performance remained within a median Target range of 10−21 mL. PPV rate also changed significantly 54 [45–61] vpm vs. 40 [28–49] vpm; p = 0.019). Although the median values also remained within target range of 40–60 vpm, the performance variability was reduced at the post assessment.

Fig. 4.

Boxplot diagrams demonstrating PPV performance summaries at Baseline and Post PPV Refresher assessments.

Table 2 presents estimates for generalized estimating equations with covariance assumed between the two timepoints (baseline-post). After participating in PPV-Refreshers (model 1), total number of PPV was significantly higher at post assessment; rate ratio 1.65 (CI⁹⁵ 1.33, 2.06; p < 0.0001). Number of Target PPV was significantly higher at post: rate ratio 1.87 (CI⁹⁵ 1.25, 2.80; p = 0.002). Mean PPV volume was significantly lower at the post assessment: mean difference -2.09 mL (CI⁹⁵ -3.92, -0.27; p = 0.025). Mean PPV rate was significantly higher at the post assessment: mean difference +13.0 vpm (CI⁹⁵ +3.34, +22.57; p = 0.008).

Table 2.

Estimates from Generalized Estimating Equations with covariance assumed between the two timepoints (baseline-post).

		Total Number of PPV*		Total Number in Target Volume*		Mean PPV Volume (ml)**		Mean PPV Rate (vpm)**
		Rate Ratio (CI95)	P-value	Rate Ratio (CI95)	P-value	Mean Difference (CI95)	P-value	Mean Difference (CI95)	P-value
Model 1	Post PPV Refresher relative to Baseline	1.65 (1.33, 2.06)	<0.0001	1.87 (1.25, 2.80)	0.002	−2.09 (−3.92, −0.27)	0.025	12.96 (3.34, 22.57)	0.008
Model 2	First PPV Refresher relative to Baseline	1.68 (1.12, 2.52)	0.013	3.51 (1.73, 7.12)	0.0005	2.35 (-0.13, 4.82)	0.064	9.55 (−7.42, 26.51)	0.27
	Additional PPV Refresher beyond First***	0.99 (0.83, 1.18)	0.938	0.72 (0.54, 0.95)	0.023	−2.09 (−3.08, −1.10)	<0.0001	1.61 (−5.0, 8.21)	0.634

^*Estimated using Negative Binomial Distribution.

^**Estimated using Normal Distribution.

^***Additional PPV Refreshers was treated as a continuous variable, and therefore assumes a linear relationship.

When comparing the effect of the First PPV-Refresher vs. subsequent PPV-Refreshers (model 2), completion of the First PPV-Refresher was associated with an increase in total number of PPV: rate ratio 1.68 (CI⁹⁵ 1.12, 2.52; p = 0.013) and for number of Target PPV: rate ratio 3.51 (CI⁹⁵ 1.73, 7.12; p = 0.0005). However, the first PPV-Refresher was not associated with a significant difference in mean PPV volume (p = 0.064) or mean PPV rate (p = 0.27) (Table 2). In determining the effect of the subsequent PPV-Refresher sessions beyond the first, there was no significant difference for total number of PPV (p = 0.938). Yet, Target PPV trended significantly lower for each additional PPV-Refresher: rate ratio 0.72 (CI⁹⁵ 0.54, 0.96; p = 0.023) and each additional PPV-Refresher beyond the first was also associated with a significantly lower volume: mean difference -2.09 mL (CI⁹⁵ -3.08, -1.10; p < 0.0001). However, mean PPV rate was not significantly different with additional PPV-Refreshers (p = 0.634) (Table 2).

Discussion

This study has examined the effectiveness of a brief, educational intervention using high-frequency, short-duration, psychomotor skill training to optimize PPV performance for front-line neonatal nurses. Not surprisingly, we found that PPV performance as measured by number of PPV meeting target volume criteria (Target-PPV) improved significantly after PPV-Refresher training. Although there was no association between change in PPV performance and number of PPV-Refreshers completed, we found that there was a significant change with completion of the first PPV-Refresher session versus subsequent PPV-Refreshers.

Various studies have demonstrated deterioration of skills before certification expiration across a variety of disciplines and time periods, although overall knowledge may be retained much longer.[9], [26], [27] In addition, provider anxiety and challenges with appropriate use of equipment and the ability to implement appropriate corrective strategies if the initial rescue breaths are not effective can exacerbate the delay the onset of effective PPV.¹⁰ In an audit of video-recorded infant resuscitations, Skåre et al. reported that weakest link – the provider that had the most undue interruptions during PPV – seemed to be the “initial responders” responsible for initiating resuscitation in unexpected compromised newborns.²⁸ Additionally, the optimal volume, pressure and flow to achieve functional residual capacity in asphyxiated infants have yet to be established¹ which may hinder provider confidence. In our study, even though providers were compliant with all required training, at their baseline assessment, providers delivered only a median of 33 (IQR 26–46) PPV over course of the 90-second assessment with only a third (11; IQR 5–21) meeting “Target” PPV volume criteria between 10−21 mL. However, after implementation of the PPV-Refresher program, we found that the median number of total PPV delivered increased by 73% (57; IQR 36–74) with a greater than doubling of the number of Target PPV (23; IQR 13–32). It should be noted, that as the assessment recording system was unable to collect data on ventilations less than 7 mL, there may have been a number of ventilations in this range that would have added to the total count. The significance of ventilations with these small tidal volumes (<7 mL) during the initial steps of resuscitation is unknown.[29], [30], [31]

Expert consensus recommends resuscitation psychomotor skill training and validation at regular time intervals in order to improve performance and mitigate recognized barriers.[32], [33], [34] As such, we targeted our training intervention to the relevant population of nurses who were most likely to provide PPV during the critical initial moments of birth asphyxia. There were significant improvements in all performance metrics. However, summary results for mean PPV volume and mean PPV rate were within target ranges both before and after PPV-Refresher participation. Notably, although results remained within target ranges, variance in performance among providers was reduced at the post assessment. We postulate that the ability for the subjects to have brief, repeated, one-on-one practice, provided an essential psychomotor skill training supplement to traditional curriculum which significantly improved not only number of delivered and Target PPV, but narrowed the variability of PPV volume and rate.

The PPV-Refresher program utilized a deliberate practice model (repetitive performance of psychomotor skills in a focused domain; rigorous skills assessment that provides learners specific, informative feedback, which results in increasingly better skills performance in a controlled setting).³⁵ We previously studied a refresher program built on a deliberate practice model for cardiopulmonary resuscitation (CPR) psychomotor skills and reported that a single, less than 2-minute CPR Refresher training session resulted in nearly 60% of providers performing excellent CPR at the 3-month evaluation. However, at 6 months, excellent CPR performance was not different from baseline (OR, 0.93; 95% CI, 0.22–3.9; p = 0.92).²¹ In another investigation, we found when in-hospital providers were retrained every 1–3 months using American Heart Association absolute chest compression rate and depth guideline targets, CPR skill retention was improved.[18], [36] In this study, we translated the novel Refresher training model to focus on newborn PPV-psychomotor skill performance; a similar rare, high-stakes, critical event. The PPV-Refreshers were designed to be brief, completed in-situ (near bedside), utilize the same PPV equipment used in real resuscitations, and to concentrate on PPV psychomotor skills with timely facilitator feedback and error correction.

There has yet to be consensus as to what the optimal frequency of refresher training interval should be and in what manner they are best implemented.[21], [37], [38] A study conducted at a rural referral hospital in Northern Tanzania by Mduma et al. found the success of brief (3-minute), frequent (weekly) newborn resuscitation simulation skills training most successful when combined with monthly training sessions over the course of a 1-year study period.²³ After the intervention, they observed a significant change in routine clinical practice yielding a 40% decrease in mortality. In this study, although offered multiple times every week, completion of PPV-Refreshers was dictated by provider scheduling, availability, and patient acuity, therefore it was not always possible for subjects to complete a PPV-Refresher when offered. Since we did not collect data on actual PPV-Refresher performance, we cannot determine if those that completed more PPV-Refreshers had better or worse performance during their PPV-Refresher sessions. However, we speculate that those more challenged with PPV competence may have had internal motivation to complete more PPV-Refreshers. Likewise, more skilled providers may have felt that they did not needed the additional sessions. However, since the sample size was small, when analyzing the number of PPV-Refresher completed we were unable to determine an association of any PPV variable with performance beyond the first PPV-Refresher. Additional investigation is warranted to determine association of optimal performance with participation frequency in a PPV-Refresher program and the impact of a single PPV-Refresher versus multiple (mastery versus spaced deliberate practice).

Limitations

This simulation manikin training study has several limitations. First, there was an additional cohort consisting of first-year residents but was not included in this analysis because of concerns with difference in population (education, training and experience) and PPV-Refresher training schedule (conducted over 1-month). Due to the study scheduling, we did not have a control group for ICN and LDU staff nurses as there was no ability to separate a control group from the intervention; nurse scheduling was varied and PPV-Refreshers were completed out in the open. Along with not assessing facemask leak, there are other inherent limitations of the manikin measurement system used during the pre-post assessment sessions: the recording sensitivity (unable to record ventilations less than 7 mL) may lead to potential ascertainment bias, and the biofidelity used in this study may not provide true representation of the subjects’ PPV performance on a real newborn with fluid-filled lungs. Additionally, the manikin used during the PPV-Refreshers was a different system than the one used for the recorded pre-post assessments, therefore the skills developed during the PPV-Refreshers may not have translated to the recording manikin at the post-assessments. Importantly, the appropriate number and volume of PPV to be delivered during various phases of resuscitation is unknown, thus, delivering PPV outside recommendations does not necessarily equate to inadequate resuscitation. Finally, the translation of PPV skill performance on real newborns with birth asphyxia was not evaluated.

Conclusion

Implementation of a brief PPV-Refresher psychomotor skill program significantly improved the total number of PPV delivered and number of Target PPV (volume 10−21 mL) on a manikin. This suggests that a frequent, brief PPV-Refresher program may improve the capability of front-line providers to deliver effective PPV during actual newborn stabilization and resuscitation. Although we were unable to determine an optimal PPV-Refresher frequency to improve performance, our data suggests that completing one PPV-Refresher session quarterly may be sufficient to improve and maintain the number and quality of PPV delivered.

Funding

Supported by the Laerdal Foundation for Acute Medicine and an R18 grant from the Agency for Healthcare Research and Quality, #HS022469. The funders had no involvement in the study design, collection, analysis or interpretation of the data, nor in the writing of the report or in the decision to submit the paper for publication.

Conflicts of interest

The authors acknowledge the following potential conflicts of interest: Dana Niles received partial salary support for this work by the Laerdal Foundation for Acute Medicine. Additional funding for partial salary support for Dana Niles, Elizabeth Foglia, Elena Insley, Courtney Cines, Anne Ades, Michael Posencheg, and Vinay Nadkarni was provided by an Agency for Healthcare Research and Quality R18 grant. The authors report no other relationships or activities that have influenced this work.

CRediT authorship contribution statement

Dana E. Niles: Conceptualization, Methodology, Investigation, Project administration, Data curation, Formal analysis, Writing - original draft, Writing - review & editing, Funding acquisition. Christiane Skåre: Conceptualization, Methodology, Writing - review & editing. Elizabeth E. Foglia: Methodology, Project administration, Writing - review & editing. Elena Insley: Investigation, Data curation. Courtney Cines: Investigation, Data curation. Theresa Olasveengen: Supervision, Writing - review & editing. Lance S. Ballester: Formal analyses, Writing - review & editing. Anne Ades: Methodology, Writing - review & editing. Michael Posencheg: Supervision, Writing - review & editing. Vinay M. Nadkarni: Conceptualization, Supervision, Writing - review & editing. Jo Kramer-Johansen: Methodology, Supervision, Writing - review & editing.