Evaluation of “Real BVM Help” for Improving Manual Ventilation Quality in the Prehospital Setting: A Before-After Manikin Study

Abstract

Background

Manual ventilation is a critical skill for emergency medical service (EMS) members. However, it is challenging in terms of correct ventilation rates and tidal volumes, with potentially severe adverse effects of hypo- and hyperventilation. Measuring the quality and involving real-time feedback may be effective in optimizing of manual ventilation.

Methods

Data acquired retrospectively from a quality management project in 143 advanced emergency medical technicians were included. They performed bag ventilations on an intubated adult manikin for two minutes without any feedback system, and then another two minutes with the Real BVM Help^® device. Ventilation rates and volumes and their allocation in correct/recommended ranges were determined.

Results

With the feedback device, correctly applied ventilation rates increased by 21% (63.6% in the correct range without vs 84.6% with the feedback device; p<0.001), and ventilation volumes improved by 41% (27% in the correct range without vs 68% with the feedback device; p<0.001). Without the device, the average ventilation rate was 10.5 ±3.1/minute, compared to 9.5 ±1.9/minute with the device. Ventilation volumes amounted to 370.6 ±84 mL without Real BVM Help^®, while when using it, 415.5 ±33.1 mL was noted.

Conclusion

Our data demonstrate significant improvements in ventilation rates and volumes when using a ventilation feedback device. This manikin study suggests a ventilation feedback device being beneficial for the use by EMS members, but our findings must be further validated in real-life conditions.

Plain Language Summary

For various emergency situations in the out-of-hospital setting, it is important for ambulance personnel to be able to perform manual ventilations using a ventilation bag on patients. This critical skill is, however, difficult to acquire and maintain. Ventilating too little or too much are common challenges. Thus, measuring (with a so-called “feedback device”) how much air is given and how many times per minute may support paramedics.

We thus conducted a data analysis of paramedics ventilating manikins with and without a feedback device, and we observed the allocation in correct/recommended ranges.

Correctly applied ventilations increased significantly with the use of a feedback device. Our data thus suggests a ventilation feedback device being beneficial for the use by paramedics, but our findings must be further validated with real patients instead of manikins.

Introduction

Manual ventilation including bag-valve-mask (BVM) ventilation is a critical skill that emergency medical service (EMS) members, irrespective of whether in a physician-based or non-physician-based system, must master to provide appropriate care to patients in acute emergencies.¹ However, BVM ventilation is known to be challenging in terms of correct ventilation rates and tidal volumes. During cardiopulmonary resuscitation (CPR), ventilations are currently recommended in a 30 chest compression to two ventilations ratio or (if the patient is intubated) with a ventilation rate of once per six seconds with around 500 mL, or until a visible chest rise.² However, even highly trained rescuers are often not able to perform this correctly:³ Literature reports ventilation rates of up to 50/minute^4–10 and a 20% variability among breaths,¹¹ and subsequent consequences of hyperventilation include increased intrathoracic pressures, a reduced cerebral blood flow, impaired coronary perfusion pressures, barotrauma, and gastric insufflation with a risk of aspiration.^5,12,13 Thus, parallel to general feedback tools proving to be beneficial in CPR training,¹⁴ measuring the quality of manual ventilation, especially if involving real-time feedback, may be effective in preventing hyperventilation and achieving ventilation values and -goals.² Real-time ventilation parameter measurement is possible and has been shown to be feasible,^15–17 making it a research focus.¹⁸ As a real-time ventilation feedback system is already used throughout the EMS Vienna, we aimed to elucidate whether it improves manual BVM ventilation in EMS personnel at a high training level.

Methods

EMS Setting

Austria operates a multi-tiered EMS system consisting of basic and advanced emergency medical technician (EMT) units, as well as physician-staffed units. According to their training level, EMTs typically provide basic life support (BLS) or advanced life support (ALS). In urban regions such as Vienna, EMS is coordinated centrally and dispatches EMT crews alongside emergency physicians when indicated. All EMTs undergo structured training and certification, and standardized protocols guide on-scene interventions until further advanced support arrives. Our study setting reflects typical urban Austrian EMS structures, with emphasis on rapid response and close physician-EMT collaboration.

Study Design and Participants

The study was carried out as a retrospective data analysis. The Real BVM Help^® system is already routinely used by the EMS Vienna in all ventilation situations, and quality assurance measurements are conducted frequently. Data were collected during such a quality management project to assess the ventilation competency of EMTs: Regular refresher training sessions that all NKI EMTs (= EMTs with the special emergency qualification to intubate and ventilate, the highest medical qualification which EMTs can achieve in Austria, “besondere Notfallkompetenz Intubation und Beatmung” or “NKI”) of the EMS Vienna must complete to maintain their professional certification had been used: During 18 such recertification sessions in June 2022, the EMTs had to provide ventilation to an adult manikin for two minutes. This was conducted first without the ventilation feedback system, and then for another two minutes with the system in place, to show the potential benefits of the system directly to the users. The ventilation feedback system used was the Real BVM Help^® by ZOLL^®: It consists of a sensor attached to the top of the BVM system and a part that measures ventilation frequency and tidal volume in real-time, then providing feedback on a compatible monitor.¹⁹ An intubation manikin (Airway Management Trainer manikin by Laerdal^®) with visible plastic lungs and already correctly intubated with an endotracheal tube size 7.0, as well as an adult ventilation bag (The Bag II, Laerdal^®) with a volume of 1650 mL and with a one-hand-delivered tidal volume of 830 mL were used. The system was monitored using the corresponding monitor/defibrillator to record and depict data in real-time. After the quality management project, the idea to analyse the collected data was born. The inclusion criteria for the analysis at hand were thus having taken part in one of the described recertification sessions, and ventilation data being available. Exclusion criteria consisted of, for homogeneity reasons, not being employed by the EMS Vienna, or datasets containing too few datapoints to analyse. The described sequential approach naturally avoided crossover or randomization but reflects real-world implementation.

Objective

The study objective was to evaluate whether the use of a real-time ventilation feedback device improves the quality of manual ventilation in intubated adult manikins performed by NKI EMTs compared to manual BVM ventilation performed without the feedback system, as measured by correct ventilation rates and tidal volumes. We assumed that the ventilation system was closed without any leaks and that inspiration equalled expiration. The standard for the ventilation rate was set at 8–12/minute, and the tidal volume range at 400–500 mL, as per international guidelines.² Ethical approval for this study was waived by the Ethical Committee of the Karl Landsteiner University of Health Sciences Krems, Austria (waiver 03/11/2024).

Statistical Analysis

The two independent variables use of a BVM feedback device and no use of the device and the dependent variable percentage of correctly administered ventilations were assessed. Data was collected using a pre-designed data collection form and entered into a spreadsheet for analysis. Descriptive statistics were used to summarize participant characteristics and outcome measures. The sample size of 143 cases provided a power of 80% to detect an increase of 7% (assuming 10% of discordant pairs). Power analysis is based on an alpha level of 5% two sided using a McNemar test. Descriptive characteristics are expressed using counts, percentages, means, and standard deviations (SD). Categorical data are expressed with absolute and relative frequencies. For metric variables, medians with interquartile ranges (IQR) are given. To verify the assumption of a normal distribution of the parameters, Q–Q plots were generated. For comparison of groups, Chi-square or Fisher exact tests were used. The Wilcoxon rank-sum test was used to analyse differences in numerical data. All tests were two-sided and p-values of <0.05 were considered statistically significant. For analyses, Microsoft Excel Version 16.9 and the software R (RStudio Version 1.1456, RStudio Inc., Boston, MA, U.S.A) were used.

Results

Study Population

Of a total of 340 eligible EMTs, 120 had to be excluded because not being a member of the EMS Vienna, and another 12 because not having taken part in the specific training with the feedback device. Another 65 records were excluded due to missing data (Figure 1). Finally, we were able to include data from 143 EMTs, each participating in two rounds (without and with use of the feedback device) of the practice scenario. Information about age and sex of the participants was not available due to organisational regulations and the retrospective nature of the study. Figure 2 depicts the main study findings.

Figure 1.

Study inclusion flowchart. NKI = Austrian advanced paramedic training in airway management.

Figure 2.

Summary of the main findings.

Ventilation Rates and Volumes

Examination of the ventilation rate outcomes respective of the use of the Real BVM Help^® system showed that, without its deployment, a total of 91 (64%) were within the correct range. With utilization of the Real BVM Help^®, 121 participants (85%) attained correct ventilation rates (p<0.001). Without BVM Help^®, in 39 (27%) cases, a correct tidal volume was achieved, whereas with its use, this increased to 97 (68%) times (p<0.001). The mean ventilation rate within the entire cohort, both with and without the feedback device, was 10.03 (±2.6)/minute (without the device: 10.53 ±3.1/minute; with the device: 9.54 ±1.9/minute; absolute difference: 0.99/minute; relative difference: 9.9%; p<0.001). Ventilation volumes in the entire cohort amounted to 393.07 ±68 mL (without Real BVM Help^®: 370.62 ±84 mL; with Real BVM Help^®: 415.53 ±33 mL; absolute difference: 44.92 mL; relative difference: 11.43%; p<0.001). Table 1 and Table 2 as well as Figure 3 provide an overview.

Table 1.

Details of Ventilation Rates and Volumes Respective of the Use of the Real BVM Help^® Feedback Device

N=143	Ventilation Rate Without Feedback Device (Per Minute)	Ventilation Rate with Feedback Device (Per Minute)	Ventilation Volume Without Feedback Device (mL)	Ventilation Volume with Feedback Device (mL)
Mean	10.5	9.5	370.6	415.5
SD	3.1	1.9	84	33.1
Minimum	4	6	169	328
Maximum	20	18	653	525

Table 2.

Comparison of Ventilation Rates and Volumes Respective of the Use of the Real BVM Help^® Feedback Device

N=143	Without Feedback Device	With Feedback Device	p-value
Correct ventilation rate, n (%)	91 (64)	121 (85)	<0.001
Correct ventilation volume, n (%)	39 (27)	97 (68)	<0.001

Figure 3.

Distribution of ventilation rates and volumes as histograms (A) and boxplots (B).

Abbreviations: VR, ventilation rate; VT, ventilation volume/tidal volume.

Discussion

Our study aimed to evaluate the efficacy of the Real BVM Help^® system in improving manual BVM ventilation conducted by specifically trained EMTs of the EMS Vienna. In summary, the utilization of the Real BVM Help^® resulted in a substantial enhancement of ventilation efficiency, regarding both ventilation rate and volume.

Improved Guideline Adherence

Previous research findings highlight the frequently seen excessively high ventilation rates, for instance of 30 per minute (15–49).^5,20 A previous resuscitation study involving Viennese paramedics also showcased hyperventilation during CPR, with a median ventilation rate of 18 (14–24)/minute. Remarkably, only 18% (192 of 1065 assessed minutes of CPR) of the CPR duration adhered to the correct ventilation rate of 12–15/minute at that time.²¹ In comparison with our findings, the situation seems to have improved (10.5 (4–20)/minute even without a feedback system) but is still not optimal. However, with the feedback system, 85% of ventilations were within the recommended range and showed a better interquartile range (9.5 (6–18)/minute). Concerning ventilation volume, a significant variability of up to 20%, affecting the tidal volume has been previously reported.¹¹ In our context, the use of a feedback device could reduce the interquartile range of delivered tidal volumes from 169 to 653 mL to 328–525 mL. This constitutes a clear indication that the ventilations were executed more consistently and with reduced variability. With few exceptions,¹¹ literature predominantly supports our findings: Real-time ventilation feedback in newborn and general paediatric resuscitation/ventilation was shown to be beneficial,^22–24 and also for adults, respective results have been published both on manikins^{15–17,25–28} and real-life patients.²⁹ Assessing the ZOLL feedback system used in OHCA patients in a before–after study, Drennan et al showed an increased guideline compliance.²⁹ Improved guideline adherence when utilizing ventilation feedback systems has also been stressed before,^17,26,30 and should probably the ultimate goal.

Implications for EMT Training and Clinical Practice

Our analysis offers valuable insights into the effectiveness of specialized ventilation training for EMTs. While the ventilation rate was being applied reasonably confidently, particular attention should be paid to the applied volumes in order to prevent overcompensation counteracting hyperventilation (probably due to the EMTs knowing about the potentially harmful effects of hyperventilation): Our findings of a mean ventilation volume of 371 mL actually average below the recommended amount. The feedback system significantly improved this, with then 68% of ventilations falling within the correct range of 400 to 500 mL. This demonstrates the value of immediate feedback as an educational tool, not only for achieving technical accuracy but also for reinforcing physiological understanding of ventilation targets. Such systems may assist EMTs in better internalizing appropriate volume and rate combinations, thus tackling both hypo- and hyperventilation, which is particularly relevant in high-stress settings like cardiac arrest where cognitive overload may impair performance. The successful use of CPR feedback devices for compressions supports the idea that ventilation feedback could be an equally valuable standard component of resuscitation training programs. General CPR feedback devices, mostly focussing on chest compressions, are already recommended and have been shown to improve CPR skill acquisition and retention in training^14,31,32 – why not close the loop and introduce automatic feedback on all vital CPR parameters? However, further development of feedback devices, for instance integrating audio feedback,³³ should be evaluated, and the optimal transformation from ventilation-centred training settings to complex scenarios is still a knowledge gap.³⁴

Looking beyond the training environment, implications emerge for real-life clinical application: An integration of real-time ventilation feedback into prehospital care settings—either through defibrillator-integrated systems or stand-alone airway feedback devices—could help standardize ventilation quality during actual cardiac arrests. Such systems could also serve as post-event debriefing tools, enabling structured feedback on airway management and helping identify performance gaps.

While simulation-based research is essential for foundational insights, observational or interventional studies assessing actual ventilation performance—with and without feedback—are needed to assess generalizability and outcome relevance. Linking device data with clinical outcomes would provide high-quality evidence for regulatory approval, protocol integration, and widespread adoption.

In summary, real-time ventilation feedback has the potential to enhance both the training and practice of airway management during CPR. Realizing this potential will require a concerted effort to validate, adapt, and scale these tools across diverse clinical and educational contexts.

Limitations and Applicability

Our study cohort is – despite having the same official qualification level – heterogenous in terms of additional trainings, duration of professional experience, frequency of emergency calls attended, and the number of emergency incidents requiring manual ventilation. Due to a lack of respective data, it was not possible to adjust for all these factors. Additionally, variations in the EMTs’ everyday roles including duties like driver, team leader, dispatcher, and involvement in training activities and field supervising, may have contributed to differences in proficiency levels. Of importance, the findings are only partly applicable to other EMS organisations than the EMS Vienna, as organisations vary in terms of percentages of voluntary and fully employed paramedics, organisational rules, and standard operating procedures. For instance, in Austria, the training of “NKI” is still considered a rarity. The Austrian emergency medical system is predominantly volunteer-based, especially in rural areas. Many EMTs with basic training are deployed, having had little training in the realm of ventilation. Furthermore, their operational experience differs significantly due to just a few 12-hour shifts per month for volunteers on the one-, and a full-time professional commitment on the other hand. Additionally, the density of emergency incidents varies markedly throughout the country. Consequently, a substantial research gap remains concerning the representation of current ventilation metrics under reduced training and operational experience, and whether these metrics can indeed be improved as effectively as demonstrated in our study through the implementation of the feedback system.

There were also several further potential sources of bias: First, the measurement of only inspiratory tidal volume may not be an accurate representation of the actual delivered tidal volume. Moreover, the use of a simulated setting with a manikin may not fully replicate the stress and distractions during a real-life emergency. Last, we had to deal with a considerable amount of incomplete datasets and data we had to exclude; this was due to the retrospective nature of the study. A potential prospective repetition of the study for individual participants would, however, have been counterproductive, as this could have introduced practice effects. The applicability of our results to real operational conditions and the representativeness of the studied group are factors that must be carefully considered when evaluating the practical feasibility of the Real BVM Help^®. Future research should assess its use in real-time conditions and on various training levels. Also, further research should re-assess the sample size needed to demonstrate respective effects.

Conclusion

Our findings underscore the potential of the Real BVM Help^® by ZOLL^® to significantly enhance manual BVM ventilation executed by specifically trained EMTs. The observed changes in ventilation parameters have implications for both training and patient care. These insights contribute to the existing body of literature on ventilation feedback devices, together forming a robust argument for their utilization and further scientific evaluation. Our data should be validated in real-life cardiac arrest patients.

Funding Statement

There is no funding to report.

Data Sharing Statement

Data can be inquired from the corresponding author upon reasonable request. This is in accordance with national law.

Ethics and Consent

Ethical approval for this study was waived by the Ethical Committee of the Karl Landsteiner University of Health Sciences Krems, Austria (waiver 03/11/2024). The study protocol complies with the Declaration of Helsinki.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

SS has in the past received a research grant from the ZOLL foundation. However, ZOLL was not involved in the design or analysis of the present study. The authors report no other conflicts of interest in this work.