Low-dose, high-frequency simulation with respiratory function monitor feedback enhances face mask ventilation across neonatal resuscitation devices: A prospective study in neonatology fellows

Abstract

Face mask ventilation (FMV) is a critical skill in neonatal resuscitation, yet structured assessment of neonatology fellow competence has remained limited. This study evaluated whether low-dose, high-frequency simulation with real-time respiratory function monitor (RFM) feedback improved FMV performance across commonly used ventilation devices.

A prospective, single-center study was conducted over nine months involving neonatology fellows who performed simulated FMV using a T-piece resuscitator, self-inflating bag, and flow-inflating bag. Each session included blinded and unblinded RFM conditions. Primary outcomes included tidal volume (TV) within target range, mask leak, peak inspiratory pressure (PIP), positive end-expiratory pressure (PEEP), respiratory rate, and self-reported proficiency. Mixed-effects models accounted for repeated measures.

TV within goal range was higher with RFM feedback (80.3% vs 62.3%, p < 0.0001) and mask leak was lower with feedback across all three devices. No independent effect of feedback was observed for PIP or respiratory rate. Increasing session number improved respiratory rate, PIP, and PEEP but did not significantly improve TV or mask leak, suggesting limited skill improvement without feedback. Performance varied by device modestly, but higher performance was associated with RFM feedback and practice sessions across all devices. Self-reported confidence improved significantly across all devices, though clinical FMV frequency did not change.

These findings support the use of RFMs in neonatal training regardless of device type while highlighting the need for strategies to improve long-term skill retention.

Background

Procedural competence in neonatology fellowship training is an area of focus, with most discussions centered on intubation competency. To our knowledge, however, no studies have evaluated fellow competence in Face Mask Ventilation (FMV). FMV is a critical skill for neonatal fellows to develop as approximately 10% of infants will require assisted ventilation during initial resuscitation.¹ Data is limited on how often FMV is administered to infants admitted to the Neonatal Intensive Care Unit (NICU), but we know apneic episodes occur in nearly all infants born below twenty-eight weeks gestation.2, 3 Since mask ventilation is the mainstay of therapy for apneic episodes unresponsive to stimulation, it stands to reason that FMV is occurring frequently in the NICU.

At many large institutions, FMV is often performed by respiratory therapists (RTs). However, outside of tertiary or quaternary care centers, the availability of an experienced neonatal RT is inconsistent, leaving nurses and doctors to provide this essential intervention. In our experience, doctors have little opportunity during their training to provide effective FMV and troubleshoot ineffective FMV. Furthermore, depending on practice setting, different ventilation devices are more commonly used than others, such as the T-piece resuscitator (TPR) in high resource and the self-inflating bag (SIB) globally.4, 5 This limits practice on less frequently used devices in the trainees setting, such as the flow inflating bag (FIB). In one study, only RTs were able to effectively deliver consistent ventilation with an FIB.⁶

Respiratory Function Monitors (RFMs) have the potential to improve the quality of FMV delivered and a proven ability in FMV training.7, 8, 9 One study used a low dose, high frequency (LDHF) training with an RFM and found that competence as defined by adequate mask seal in >95% and avoidance of high inflating pressures in >95% of ventilation breaths delivered could be achieved in 6 and 5 sessions over 9 months, respectively.¹⁰ While the effectiveness of RFMs as an educational tool is well established, there have not been any studies to date analyzing the effectiveness of RFMs across all ventilatory devices.7, 8, 9, 10, 11, 12

We developed an LDHF simulation curriculum incorporating real-time RFM feedback to evaluate its effect on FMV performance across devices. We hypothesized that practice with RFM feedback would improve FMV performance equally well across all devices as measured by mask leak, tidal volume (TV), respiratory rate (RR), peak inspiratory pressure (PIP), and positive end expiratory pressure (PEEP). Our null hypothesis was that RFMs do not result in improved performance equally well for all devices.

Materials and methods

Study design

This was a prospective, single-center study for FMV training implementation and simulation program development which used an RFM and an LDHF model as an educational intervention for over nine months. We concurrently employed a repeated-measures experimental study design that randomized order of devices utilized within each practice session to determine if the educational intervention was different across the three most used FMV devices. All neonatology fellows in the Baylor College of Medicine department were voluntarily recruited and consented to be a part of our study (IRB H-55749).

RFM practice

Participants completed a pre-study survey about their current utilization of FMV, perceived barriers, and confidence with each device on a Likert scale. Participants were oriented and trained on the RFM device. An investigator was present at all sessions. During each session, participants performed FMV using a T piece, flow-inflating bag, and self-inflating bag in randomized order determined via random number generator. Each device was used first blinded (no RFM feedback visible) and then unblinded (RFM feedback visible). Performance during blinded use was interpreted as skill retention since last session. The following unblinded use was interpreted as utility of direct RFM feedback. Participants were instructed to practice for at least 30 s but encouraged to practice for as long as needed until they believed they had delivered at least 30 s of “adequate” FMV.

Participant frequency was self-determined. Training platform was made available daily. However, after initial low frequency use noted (less than 2 times per month), incentives were added to encourage more frequent practice. Each participant then received a nominal incentive of $10 gift card for enrolling and an additional $10 for every two sessions completed.

Equipment

A mobile training cart was created to allow the practice sessions to move to wherever learners were present (Fig. 1). An AV cart on locking wheels was selected that was a similar height as beds in the NICU.

Fig. 1.

Mobile simulation cart contents (clockwise from bottom left): Respiratory Function Monitor (Monivent AB), Premature Anne mannequin (Laerdal), Flow-inflating bag, Self-inflating bag, T-piece resuscitator, aquarium bubbler, and AV cart.

The Monivent Neo Training device (Monivent AB, Gothenburg, Sweden) was selected as our RFM. It provides feedback information on TV, PIP, PEEP, mask leak, and respiratory rate. PIP, PEEP, mask leak, and rate are displayed as numerical values on the tablet’s screen. TV is displayed as a graphic using a cylinder which is filled with green when delivering TV in range (4–6 mL/kg) and red when out of range. Display is updated with each delivered breath in real time.

We utilized the Laerdal Premature Anne manikin. This manikin was selected due to its realistic lung compliance (0.6 mL/cmH₂O),¹² as previous studies estimate mechanically ventilated premature infant lungs to have a compliance of approximately 0.4 ml/cmH₂O.¹³ Additionally, this manikin generated realistic TV at a given PIP and estimated fetal weight of 1350 g when utilizing our chosen RFM. Only one manikin was used to minimize confounding variables.

The same premature infant mask was used across all devices. A Fisher and Paykel T-piece resuscitator was used and set to a PEEP of 5 and PIP of 25 prior to all sessions. For the self-inflating bag, a new Ambu Infant Spur II was used. For the flow-inflating bag, a new Ambu Hyperinflation bag was used and set pressure in the closed system to approximately 10 before each session. Participants were encouraged to change settings as they deemed fit for each device during practice. An AQAA aquarium air pump was utilized and set to 10 L/min for every session as the gas source.

Data analysis

Outcomes were divided into measured mastery evaluated by the quality of FMV delivered during training sessions and perceived mastery as evaluated by reported confidence performing FMV on a Likert scale and the frequency of performing FMV in clinical practice. Our study had a convenience sample size of 18 fellows at our large neonatal-perinatal fellowship program of 19 fellows. The first author was a fellow during the investigation and, therefore, ineligible to participate. A sample size was not calculated for this implementation and educational intervention or for the comparison of FMV devices.

Data was extracted from the automatic logs for percent TV in goal range, average RR, and mask leak generated for each session by the RFM. For PIP and PEEP, data was manually extracted from raw data exported from the RFM in Excel format.

To accommodate the clustering of repeated sessions within participants, mixed effects linear models were fit to examine the effect of feedback (yes or no), FMV device (TPR, SIB, or FIB) and session number (1–23 sessions) on outcomes while treating participant as a random effect. For each outcome, initially a full model was fit that included the “FMV device × feedback”, “FMV device × session” and “feedback × session” interaction terms, in addition to the three main effects. If an interaction term was statistically significant, the effect of the two factors could not be generalized because the effect of one factor depends on the level of the other factor. Therefore, a stratified analysis was performed for factors with a significant interaction. Otherwise, non-significant interaction terms were removed, and a reduced regression model was fit with only the main effects and any significant interaction terms from the full model. Statistical analysis for survey responses used Mann-Whitney U test for non-parametric testing. R version 4.5.2 (R Foundation for Statistical Computing, Vienna, Austria) was used for data analysis.

Continuous outcomes are reported as mean (±standard deviation) in tables to reflect the summary statistics used by the mixed-effects models, whereas figures display boxplots (median, interquartile range, and range) to illustrate the distribution and spread of raw session-level values, given that tidal volume in goal range and mask leak in particular exhibited skewed distributions not fully captured by mean and standard deviation.

Results

Recruitment

Of 18 eligible fellows, 17 completed the pre-study survey. Fourteen completed at least two sessions for inclusion in analysis and, of these, 12 completed the post-study survey.

Effect of RFM feedback

The presence of RFM feedback significantly increased TV in goal range after controlling for device and session number (80.3% vs 62.3%, p < 0.0001) and TV in goal range was significantly increased across all three devices after controlling for feedback and session number (Fig. 2A). Mask leak was also reduced with RFM feedback and remained significant across each device despite an interaction between feedback and FMV device. After controlling for session number, TPR leak was reduced from 42.6% to 20.9% (p < 0.0001), SIB leak reduced from 27.3% to 14.0% (p < 0.0001), and FIB leak reduced from 24.0% to 13.5% (p = 0.0009, Fig. 2B). PEEP average was significantly higher when feedback was given compared to when it was not given (p = 0.004). No significant independent effect of RFM feedback was observed for respiratory rate (p = 0.89) or PIP (p = 0.16).

Fig. 2.

Respiratory function monitor (RFM) parameters with and without feedback: Box and whisker plots showing Tidal volume (TV) in target range (A) and Mask leak (B) percent with and without RFM feedback. RFM feedback significantly improved both parameters across all device types (TV in goal range *p < 0.0001 for all devices) (mask leak % *p < 0.0001 for T-piece and self-inflating bag, *p = 0.0009 for flow-inflating bag). T-piece was more often in TV goal range (^#p = 0.04 for T-piece vs self-inflating bag, ^#p = 0.01 for T-piece vs flow-inflating bag) and more mask leak compared to self-inflating (^#p = 0.0004 without feedback, ^#p = 0.003 with feedback) and flow-inflating bag (^#p < 0.0001 without feedback, ^#p = 0.001 with feedback). N = no and Y = yes.

Effect of repeated practice

Increasing session number decreased respiratory rate by 0.55 breaths per minute per session (p < 0.0001) and after controlling for device and feedback resulted in rate that was more often in goal range (Fig. 3A). PEEP increased significantly with an estimated increase of 0.068 cmH₂O for each additional session after controlling for FMV device and feedback (p = 0.005, Fig. 3B). PIP increased with practices sessions across devices despite a significant interaction between FMV device and session number. TPR PIP average increased by 0.23 cmH₂O (SE = 0.04, p < 0.0001) per session, SIB PIP average increased by 0.20 cmH₂O (SE = 0.06, p = 0.0008), and FIB PIP average increased by 0.15 cmH₂O (SE = 0.06, p = 0.017, Fig. 3C). TV percent in goal range trended higher with increasing session number (0.38 percentage points per session, SE = 0.23) after controlling for device and feedback but was not significant (p = 0.09). Similarly, mask leak did not change with increasing practice (p = 0.23).

Fig. 3.

Respiratory function monitor (RFM) parameters by practice session number: Scatter plots with average line for Respiratory Rate (A), positive end expiratory pressure (PEEP) (B), and peak inspiratory pressure (PIP) (C) by increasing practice sessions before visible RFM feedback. Dotted line represented recommended range for respiratory rate (40–60) and recommended PEEP of 5. Practice significantly improved all three parameters in all three devices used. Adjusted mean respiratory rate was significantly higher when using the self-inflating bag than the T-piece or flow-inflating bag (60.9 ± 2.8, 53.9 ± 2.8, and 52.4 ± 2.8 respectively, mean ± SE, p < 0.0001). Peep was significantly higher using the t-piece than the flow-inflating bag (4.2 ± 0.2 vs 3.2 ± 0.2, mean ± SE, p < 0.001).

Device comparison

After controlling for feedback and session number, learners were more often in TV target range using the TPR than on SIB (p = 0.04) or FIB (p = 0.01, Table 1), but SIB and FIB were not significantly different from each other (p = 0.06). This was true even though learners had higher mask leak using the TPR than either the SIB (without feedback p = 0.0004, with feedback p = 0.003) or FIB (without feedback p < 0.0001, with feedback p = 0.001). TPR PEEP was significantly higher than FIB (p < 0.001, Table 1). SIB was not evaluated for PEEP as SIBs do not generate consistent PEEP. After controlling for feedback and session number, the estimated mean rate was significantly higher for SIB compared to TPR (p < 0.001) but the rates for TPR versus FIB were not significantly different (p = 0.11). FMV device did not show a significant interaction with feedback (p = 0.32) for average PIP and thus was not stratified for statistical comparison.

Table 1.

Respiratory Device Comparision by Respiratory Funcrtion Monitor feedback.

RFM data element	T-piece resuscitator		Self-inflating bag		Flow-inflating bag
	No feedback	Feedback	No feedback	Feedback	No feedback	Feedback
TV (% in goal)	66.8 ± 33.8	86.5 ± 19.6	63.2 ± 31.0	80.1 ± 22.2	61.9 ± 33.7	79.4 ± 26.8
Mask leak (%)	40.9 ± 32.4	19.5 ± 17.0	27.7 ± 26.4	14.2 ± 15.5	24.2 ± 30.0	13.7 ± 20.1
Rate (per min)	52.4 ± 9.9	52.8 ± 9.7	60.6 ± 13.5	58.8 ± 12.5	50.7 ± 14.3	51.8 ± 13.5
PIP (mmHg)	23.6 ± 2.5	24.2 ± 2.3	24.7 ± 3.9	24.6 ± 3.6	23.0 ± 3.7	23.3 ± 3.3
PEEP (mmHg)	4.1 ± 1.3	4.4 ± 1.4	N/A	N/A	3.0 ± 2.3	3.4 ± 2.5

Respiratory function monitor data by face mask ventilation device type, with or without feedback. Values are unadjusted mean ± std dev, n = 127–130 combined sessions from all participants.

Survey results

Self-reported confidence improved significantly across all FMV devices (Fig. 4). However, reported clinical frequency of FMV use did not change, reporting a median of 3.5 (IQR 2.1–5.5, n = 16) times in 3 months in the pre-survey and a median of 4 (IQR 3.0–5.0, n = 12) in the post-survey. The most cited barrier to performing FMV in clinical setting was the presence of respiratory therapists. One hundred percent of learners who completed the post-intervention survey reported that the curriculum was a good use of their time.

Fig. 4.

Perceived proficiency in performing face mask ventilation by device: Use of T-piece improved from a mean (±std dev) of 3.9 (±0.6)–4.7 (±0.5), use of flow-inflating bag improved from a mean of 2.5 (±1.1)–4.1 (±0.7), and use of self-inflating bag improved from a mean of 2.9 (±0.8)–4.3 (±0.5). Likert scale 1 = Not at all proficient, 2 = Slightly proficient, 3 = Somewhat proficient, 4 = Mostly proficient, 5 = Completely proficient. *p < 0.05 comparison pre- to post-survey, Mann-Whitney U test. Bars are mean ± std error.

Discussion

Our study demonstrated that training with an RFM can improve FMV with respect to PIP, PEEP, RR, TV in goal range, and mask leak; however, the skills of improved mask leak and TV in goal range did not carry over to subsequent practice sessions. Several studies have demonstrated the effectiveness of RFMs as a teaching device for effective facemask ventilation. Previous studies emphasized its effectiveness in decreasing mask leak, improving tidal volume and improving respiratory rate during a single session training event.8, 9, 14, 15, 16 Our study is consistent with these findings that use of an RFM does improve mask leak and TV. Another study demonstrated that mask leak and appropriate TV were not skills that were retained on blinded follow-up testing.¹⁷ While we did not formally assess retention by blinded follow up testing, our lack of improvement during blinded portions of repeated practice is consistent with a lack of retention.

Our results differ from other published reports in several respects. Notably, one study demonstrated retained skills three months after the initial training session defined by TV and mask leak.¹⁸ However, at their follow-up session, participants were allowed to practice with RFM visible prior to their recorded sessions with the RFM blinded and then unblinded. This differs from our study where participants were blinded to an RFM with their initial practice, which may account for the difference in results. Another study demonstrated the retention in appropriate TV delivered, but mask leak remained >20% for all devices and time points.⁸ Our study also differed from some of the available literature with respect to RR and PIP. A previous study demonstrated improved performance with TV, mask leak, PEEP, PIP, and RR when RFM was visible, while in our study RFM feedback did not independently improve RR or PIP.¹⁵

Our results also point to a discordance between perceived FMV skill and demonstrated proficiency. Participants ranked themselves as mostly to completely proficient at the end of our study period, while quantitative measures of proficiency such as TV in range and mask leak remained sub-optimal without the RFM feedback. In another study, 78% of graduating pediatric residents felt that they were prepared to perform FMV without supervision in a neonatal resuscitation while only 13% and 43.5% demonstrated actual proficiency on a FIB and TPR respectively.¹⁹ With RFM feedback, proficiency is improved. But without RFM feedback, even after multiple training sessions, our data did not demonstrate quantitative measures to support such high perceived confidence.

There are several key findings that are novel in our study. Previous studies only evaluated RFM effectiveness on one or two devices per study, typically a TPR and SIB, whereas ours studied all three common FMV devices.8, 16 Our study also demonstrated that some components of FMV skill (TV and mask leak) were not improved when blinded to the RFM even with multiple sessions over several months. A possible explanation is that the manikin chosen had substantial chest rise even when leak was high and TV was low, likely leading participants to believe, when blinded to the RFM, that adequate FMV was being delivered. Other components of FMV skill (PIP, PEEP, and respiratory rate) can be incrementally improved over time with practice using RFMs. Most importantly for our hypothesis, all directions of change and factors that affected the change trended in the same direction across the three FMV devices. We found no evidence of a differential effect of RFM feedback by FMV device, however, interaction terms were likely underpowered given the sample size. We, therefore, cannot conclude with certainty that the effects of RFM feedback are the same between FMV devices.

There are several limitations to our study. First, our sample was a convenience sample of neonatology fellows and, thus, limits the generalizability of our results to all healthcare providers. The study was designed as a feasibility and educational intervention study rather than a confirmatory efficacy trial, and the findings should be interpreted as hypothesis generating pending replication in a larger sample. Second, our study may be limited by the fact that unblinded sessions always follow blinded sessions on the same device. The fixed blinded to unblinded order means the reported feedback effect cannot be fully separated from a short interval practice or warm up effect, and that a counterbalanced or randomized order would be needed to isolate the feedback effect in future work. Third, the self-determined frequency of practice sessions and addition of mid-study incentivization may have introduced selection bias as more motivated participants would be more likely to complete more practice sessions. The mixed-effects model that adjusted for the participants’ session number should mitigate some of this effect. Fourth, skill retention was not tested as the focus was on the effects of the training intervention on skill acquisition for the three FMV devices. Future research could assess whether skills were retained similarly on all three devices or if learning decays at different rates for each device. Fifth, the presence of an investigator at every session may have introduced a Hawthorne effect. However, given the casual nature of the sessions, and that the investigator present most often was a peer, it is also possible that the opposite is true, and participants may have performed worse. Finally, as this is a simulation study, usefulness across FMV devices in clinical practice is unknown. This is consistent with ILCOR’s conditional recommendation of RFMs as an adjunct in simulation,²⁰ even as a recent systematic review continues to highlight insufficient benefit for clinical use.²¹

Conclusion

RFM feedback increased tidal volume in goal range and decreased mask leak across multiple devices. However, skill retention in the absence of ongoing feedback remained limited. Incorporation of RFMs into training curricula is supported no matter the FMV device used. Further work is needed to optimize long-term competency and skill retention.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work, the authors used Grok (xAI) for manuscript critique and title ideas. The authors did not use AI to generate any portion of the writing in the manuscript. AI critiques were used by the authors to make revision of the manuscript before submission. The authors reviewed and edited the output as needed and take full responsibility for the content of the published article.

CRediT authorship contribution statement

Samantha J. Carlton: Writing – original draft, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization. Joseph L. Hagan: Writing – review & editing, Methodology, Formal analysis, Data curation. Nicole K. Neveln: Writing – review & editing, Visualization, Methodology. Daniel S. Lemke: Writing – review & editing, Supervision, Methodology. Nathan C. Sundgren: Writing – review & editing, Validation, Supervision, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.