Newborn self-inflating manual resuscitators: precision robotic testing of safety and reliability

Abstract

Aim

A controlled bench test was undertaken to determine the performance variability among a range of neonatal self-inflating bags (SIB) compliant with current International Standards Organisation (ISO).

Introduction

Use of SIB to provide positive pressure ventilation during newborn resuscitation is a common emergency procedure. The United Nations programmes advocate increasing availability of SIB in low-income and middle-income nations and recommend devices compliant with ISO. No systematic study has evaluated variance in different models of neonatal SIB.

Methods

20 models of SIB were incrementally compressed by an automated robotic device simulating the geometry and force of a human hand across a range of precise distances in a newborn lung model. Significance was calculated using analysis of variance repeated measures to determine the relationship between distance of SIB compression and delivered ventilation. A pass/fail was derived from a composite score comprising: minimum tidal volume; coefficient of variation (across all compression distances); peak pressures generated and functional compression distance.

Results

Ten out of the 20 models of SIB failed our testing methodology. Two models could not provide safe minimum tidal volumes (2.5–5 mL); six models exceeded safety inflation pressure limit >45 cm H₂O, representing 6% of their inflations; five models had excessive coefficient of variation (>30% averaged across compression distances) and three models did not deliver inflation volumes >2.5 mL until approximately 50% of maximum bag compression distance was reached. The study also found significant intrabatch variability and forward leakage.

Conclusion

Compliance of SIBs with ISO standards may not guarantee acceptable or safe performance to resuscitate newborn infants.

What is already known on this topic?

• Self-inflating bags (SIBs) are the most common device used to provide positive pressure ventilation during resuscitation at birth.

• The United Nations programmes are advising increasing availability of SIB in resource-poor nations.

• International standards guide clinicians, regulators and manufacturers as to safety and performance of SIB.

What this study adds?

• Some SIBs brands do not deliver safe reproducible ventilation suitable for use during newborn resuscitation.

• Non-delivery of volume at any compression distance as a consequence of poor SIB patient valve design and batch variability is undetectable by the operator.

• Stated compliance to International Standards Organisation does not guarantee safety and efficacy of delivered ventilation when using SIB.

Introduction

Resuscitation at the time of birth is a common emergency procedure.¹ The transition from in utero to an independent ex utero state can be complicated by failure to establish adequate respiration with bradycardia or asystole. The self-inflating bag (SIB) is the most commonly used device worldwide to provide positive pressure ventilation (PPV). It is well suited to resource-constrained countries given that it is a manual device, which can be used without a compressed gas source.^{2 3}

In 2012, the United Nations (UN) Commission on Life-Saving Commodities for Women and Children identified the need to increase access to neonatal resuscitation devices in 50 of the world’s poorest countries.⁴ The provision of SIBs to birth attendants for all newborns requiring resuscitation in the low-income and middle-income countries has been a recent major world health initiative.^{5 6} These initiatives have identified the need for a significant boost in the production of SIBs in order to meet the global demand for the device. In addition to the production of SIBs, training, education, device maintenance and distribution are crucial elements in this health initiative.

A programme funded by USAID, the ‘Programme for Appropriate Technology in Health’ (PATH), examined a range of internationally available SIBs focusing on cost and manufacturing aspects with a view to supplying the device in resource-constrained birth settings.⁷ No data on biomechanical performance were available for the PATH review. Instead, the recommendations relied on statements of compliance outlined by the International Standards Organisation (ISO) and American Section of the International Association for Testing Materials.^{8 9} The WHO document on technical specifications for neonatal resuscitation devices published in 2016 recommends SIB devices for newborn resuscitation that are compliant with the current ISO standard but advises: ‘There is a gap between physiology and available equipment specifications, and discrepancies among existing standards and professional association guidelines'.¹⁰

Manufacturers of SIBs are guided by international standards as to the ventilation performance required to provide effective ventilation at birth. The current International Standard Organisation (ISO) document for manual resuscitators ISO 10651.4-2002 (reviewed 2013), advises that SIB ventilation performance (section 6.7), for ≤5 kg body weight should be able to deliver a minimum 20 mL volume into a test lung (compliance 1 mL/cm H₂O, resistance 400 cm H₂O/L/s), at I:E ratio 1:1, 60 inflations per minute and provide a safety pressure relief at 45 cm H₂O. It also warns that delivered volume may bypass the patient when bag is compressed. This is termed forward leakage of the patient control valve (PCV).

A poor response to initial resuscitation with SIB compression should prompt the resuscitator to first adjust the way the mask is held in position on the face to avoid or eliminate potential mask leak, reposition airway and assess the airway for obstruction. Only then should bag compression be increased to generate higher pressures to increase the delivered ventilation.^{2 11} Too vigorous bag compression may cause lung injury (pneumothorax, chronic lung disease in the preterm) due to excessive pressures and tidal volumes (VT), overventilation and hypocarbia.^12–15 There is a general assumption that when using SIBs there is a consistent relationship between distance compressed and delivered VT for any given lung compliance.

A recent review by Hooper et al proposes a three-phase sequence of interdependent physiological changes in the transitional process at birth, and that the type of respiratory support provided should vary to optimise the underlying physiological lung state at each phase: 1) liquid clearance, possibly aided by sustained inflation (SI), 2) liquid re-entry/temporary accumulation, aided by continuous positive airway pressure and 3) respiratory support/gas exchange, aided by PPV with positive end-expiratory pressure. They also state that current resuscitation approach is ‘unidimensional’ and focuses on ‘strategies applicable for an already aerated lung that is not water-logged'.¹⁶ Term animal studies have found very low compliance levels (<0.1 mL/cm H₂O/kg) in fluid-filled lungs.^{17 18} Ersdal et al have recently presented measured compliance data from 1053 apnoeic term and near-term newborns resuscitated in Tanzania. The measured average static compliance of <1 mL/cm H₂O at up to 2 min of life supports the Hooper hypothesis, and the need for manual SIB devices providing PPV to be sufficiently responsive to the operator reacting to ventilation breath by breath.¹⁹

There is currently no data on the evaluation of differences in mechanical performance, delivered tidal ventilation and intrabatch variance in different brands of single use and reusable SIB resuscitators designed for use on newborns. We aimed to examine the dynamic ventilation performance of SIBs in a low compliant leak free lung model. This model reflects term asphyxiated infants in the first minutes of life who are apnoeic with fluid-filled low compliance lungs¹⁹ and surfactant-deficient preterm infants.^{17 18} To provide a stable, precise and repeatable testing method, a robotic ‘hand’ replicating the geometry and force delivered by a standard human hand was developed.²⁰

Methods

A computer-controlled two-armed robotic mechanism simulating ‘standard hand compressions’ provided incremental compressions to SIBs across a precise range of distances. The relationship between distance of bag compression, delivered VT and airway pressure were examined. Twenty different models of SIB (n=173) resuscitators were examined with an average of nine units per model summarised in online supplementary table S1. Some manufacturers produced multiple SIB models with differing part numbers and physical characteristics. A test lung (Drager Lubeck, Germany) of known compliance (0.5 mL/cm H₂O) and resistance (50 cm H₂O/L/s) was attached to the SIBs with a respiratory function monitor (RFM) (Acutronics, Medical Systems, Zug, Switzerland). All SIB units were functionally checked according to each manufacturer’s insert instructions, and tested for compliance to ventilation performance as specified in the current ISO standard²⁰ before our testing model was applied, all devices complied.

The robotic mechanism was programmed to mimic forces generated by a human hand compressing an SIB at an inspiratory time of 0.5 s, I:E ratio 1:1 and an inflation rate of 60 per minute (IPM) as recommended in the ISO standard at all distances tested (see online supplementary data: robothand.pdf). The programme sequence was designed to provide 40 compressions at each defined distance starting at 15 mm at a rate of 60 IPM. The robotic mechanism automatically increased compression distance in increments of 5 mm (±0.03 mm) after the previous run of 40 inflations. This sequence continued to a maximum distance of 60 mm (which corresponded to approximately 82% of maximal bag compression distance averaged across the models (see online supplementary table S1). Overpressure valves were left in their normal operating position during testing.

The RFM hot wire pneumotach and pressure sensor line were sited between the SIB under test and the test lung where the airway pressures and flows were measured. System was calibrated with an external syringe of known volume, and pressure/flow via a traceable reference ventilator analyser (PF300, IMT Medical, Buchs, Switzerland). Analogue signals output from the RFM were collected and digitised at 200 Hz with data acquisition and analysis software (Spectra, Grove Medical, London UK). The pneumotach was re-zeroed before each SIB was tested.

Our primary outcome was a ‘pass’ or ‘fail’ assessed as passing all four criteria determined a priori: safe minimum average VT for the smallest expected newborn infant between 5 and 10 mL/kg²¹ (2.5–5 mL); <30% average coefficient of variation (CV) in volume delivered over compression distance per device and SIB model; 99% of delivered peak inflation pressures (PIPs) are <45 cm H₂O as recommended by the ISO standards and a functional compression distance where the device starts delivering average tidal inflations ≥2.5 mL before 50% (≤30 mm) of total bag compression is reached.

Data analysis

Analysis was conducted using Stata (V.13 MP, StataCorp, College Station, Texas, USA). The measured parameters included the mean, %CV, maximum PIP and VT. Analysis of variance (ANOVA) for repeated measures was used to determine differences between models and units for VT delivered at each distance. ANOVA were reported with p values adjusted F test using Box’s conservative epsilon, p values of <0.05 were considered to be statistically significant.

Results

A total of 67 540 compressions from 20 different models (173 individual SIB units) provided 48 493 compressions with volumes ≥2.5 mL, and 2191 compressions with volumes <2.5 mL. There were 16 856 compressions where the SIB failed to deliver a measurable volume when compressed. Based on the a priori ‘composite pass’ criteria, 10 of 20 models passed on all four predetermined characteristics (table 1). Of those models passing all criteria, 30 963 (81.6 %) compressions provided VT≥2.5 mL. This was compared with 17 530 (59.2%) compressions of models failing one or more criteria. Variations in delivered PIPs are detailed in online supplementary figure S1.

Table 1.

Composite scores of SIBs tested sorted by a priori category pass and %CV

A priori categories	Minimum delivered tidal volume* (a)	Batch test variance (n)† (b)	High PIP fail‡ (c)	Start distance§ (d)	Category pass/fail
Model (n=units) Part no Type RU/SU	Mean volume¶	% CV mean of distances¶	% of inflations>45 cm H2O	mm	a, b, c, d
1. Laerdal (10) 845031 SU	4.9	7.0	0.7	25	✓✓✓✓
2. Portex (9) 8527MPB SU	4.1	7.7	0	25	✓✓✓✓
3. Galemed (10) SHMC 2203 SU	3.9	8.6	0	25	✓✓✓✓
4. Vital signs (10) 7902R SU	3.2	9.0	0	20	✓✓✓✓
5. Laerdal (10) 8560533 Vertical RU	3.8	9.0	0	20	✓✓✓✓
6. Laerdal (10) 850050 RU	3.1	9.3	0	20	✓✓✓✓
7. Ambu (10) SPUR II SU	2.5	10.2	0	15	✓✓✓✓
8. Portex (9) 8528M SU	3.2	11.0	0	25	✓✓✓✓
9. Portex (9) 8527MP SU	3.2	12.3	0	25	✓✓✓✓
10. Headstar (10) HP9333FN SU	4.0	20.5	0.1	25	✓✓✓✓
11. Besmed (11) BE-1703 SU	3.1	9.7	7.1	20	✓✓✗✓
12. Mercury (4) 1055299 SU	3.6	10.7	6.4	20	✓✓✗✓
13. Hsiner (10) PH60103 SU	3.1	12.2	7.7	20	✓✓✗✓
14. Hsiner (7) 60152 SU	3.2	15.3	4.6	20	✓✓✗✓
15. Hsiner (10) 60 113 P SU	2.9	48.9	0	30	✓✗✓✓
16. Hudson RCI (9) 5364 SU	8.8	7.1	0	40	✗✓✓✗
17. Besmed (10) BE-2701 SU	2.7	34.2	0	40	✓✗✓✗
18. Hsiner (5) 60113 SU	3.4	46.4	2.9	30	✓✗✗✓
19. Zeal (5) RSB 1001 RU	2.5	47.3	3.7	30	✓✗✗✓
20. Meditrin (5) Infant 250 mL RU	6.3	37.2	0	40	✗✗✓✗

*Pass criteria; deliver minimum usable volume of 2.5–5 mL.

†Pass criteria; average CV of delivered volume <30% (all distances).

‡Pass criteria; <1% of inflations>45 cm H₂O.

§Pass criteria; average start distance 50% (≤30 mm) of total compression distance.

¶Tidal volumes and CV all significantly different ANOVA repeated measures, p<0.001.

ANOVA, analysis of variance; CV, coefficient of variation; RU, reusable; SIB, self-inflating bags; SU, single use.

Minimum average tidal volumes

Meditrin and Hudson SIBs could not provide the minimum safe volumes for a 500 g infant. Of note, start volumes for these models were at >50% total compression distance (table 1 and online supplementary table S1). Detailed results of VT per compression distance per SIB model batch are presented in online supplementary table S2. Figure 1 shows box and whisker plots of delivered VT per SIB model batch at each compression distance.

Figure 1.

Box and whisker plots of delivered tidal volumes per self-inflating bags model batch at each compression distance with Lowess regression lines. RU, reusable; SU, single use.

Average coefficient of variation

Five models had average %CV exceeding 30% ranging from 34% to 46.7%. The overall mean VTs were 2.0 mL lower for those models with %CV>30% than not (95% CI −2.2 to −1.9 mL) (table 1 and Supplementary data table S2.

Peak inflation safety limit

Eight hundred twenty-one inflations exceeded the safety limit of 45 cm H₂O (mean 48, range 45.1–53.0 cm H₂O) in six models (table 1). This represented approximately 6% of all inflations of these models. Six models were found to be unsatisfactory by exceeding the peak inflation safety limit criteria (figure 2).

Figure 2.

Box and whisker plot inflations >45 cm H₂O for high peak inflation pressure (PIP) fail models. RU, reusable; SU, single use.

Functional compression distance

Three models did not deliver measurable inflation volumes until 40 mm compression distance corresponding to approximately 67% of maximum bag compression (table 1). These three models also failed on at least two a priori criteria. The Meditrin exhibited a variance in starting distance from 40 to 55 mm and one unit was unable to deliver measurable volume at any compression distance.

Intrabatch variability

The Hsiner 60 113P SIB was notable in that 5 of 10 units did not deliver a minimum usable volume until the 55 mm starting distance was reached in contrast with the 3 units that started at 25 mm compression distance (figure 3).

Figure 3.

Poor batch variance self-inflating bags (SIB) Hsiner (part no60113 P). Box and whisker plots of delivered tidal volumes per SIB unit at each compression distance with Lowess regression lines.

Discussion

This is the first study to examine relationships between the compression distance of a SIB and delivered volume. The robotic bag compression and pass/fail scoring methodology described is an innovative, independent and repeatable method of assessment. It gives a new level of rigour beyond that used to set current ISO standard for manual ventilation performance for infants<5 kg. Given the ventilation performance, model and intrabatch variance shown, it is crucial to accurately define mechanical performance necessary for the provision of safe life-saving manual ventilation for infants. Such a system was proposed as long ago as 1975 by Dick and Ahnefeld but not implemented in subsequent ISO standards.²²

Limitations of this study are the use of a low compliance test lung model of ill newborn infants. Many of the SIB models tested are rated for use up to 10 kg body weight, which reflects infant/paediatric use (see online supplementary table S1).²³ The performance issues found in this study are similar to those found by Kain et al in a paediatric SIB bench study using higher test lung compliance (10 mL/cm H₂O).²⁴ A potential limitation is the generalisability of the two-armed robotic mechanism compared with a human 2–5 fingered hand compression. However, Basinni et al have shown no difference in delivered volume between 2/3/4 and 5 finger hand compressions.²⁵ The inflation time would be <0.5 s of compression time when forward leakage was present.

The inability of the safety pressure limiting valve to prevent PIP in excess of 45 cm H₂O and the wide intrabatch variation (CV) in delivered volumes seen in some models is of serious concern. In vivo, potential exists for SIBs with faulty safety pressure limiting valves to allow higher PIPs than seen in this simulation study under uniform compression force.²⁶

Our results indicate compliance to the ISO standard by SIB manufacturers may not guarantee the efficacy of the device to deliver a safe or uniform volume of gas when the SIB is compressed. Non-delivery of volume to the patient connection when the SIB is compressed is termed forward leakage in the ISO standard.²⁰ Forward leakage cannot be routinely detected by the operator during resuscitation (figure 4 and  online supplementary video1.mpg).

Figure 4.

Cross-section of self-inflating bags patient control valve illustrating forward leakage. (Original artwork US Patent 5722394A, modified and illustrated by M Hinder).

Munford and Wishaw described adult SIB failure due to forward leakage characteristics of the PCV.²⁷ The PCV valve did not move fully into the inspiratory position and much of the VT bypassed the patient. Importantly, it was noted that users may lack the experience to detect failure to inflate the lungs. Kain et al in 1993 examined the performance of six different brands of both reusable and disposable paediatric SIB and also observed that the PCV can be placed inadvertently in an intermediate position giving rise to life-threatening hypoventilation during resuscitation.²⁴

The current ISO standard warns: ’If forward leakage is a design feature of a resuscitator, this should be disclosed so that the user does not confuse this leakage with a malfunction’ (page 20 Rationale B.6.5).²⁰ Notwithstanding, we found no testing methodology to detect or quantify forward leakage described in the standard, and there was no disclosure in any of the product literature supplied with the devices examined in this study. Forward leakage does not appear to have any beneficial characteristics that we can determine.

Resende et al in 2006 examined 10 experienced neonatologists ventilating intubated preterm lambs with SIB (Hudson Lifesaver) showing great variation in delivered PIP and excess VT/kg (median 17.8 mL/kg IQR 14.1–22.4).²⁸ Resende et al concluded their results are explained by operator variation. Our results of the same model of Hudson Lifesaver SIB alternatively suggest forward leakage of PCV as a major potential contributing factor.

The ability of some of the SIB models tested to safely deliver ventilation to a low compliant newborn lung (term apnoeic or preterm <1 kg) is questionable. As all units in this study passed the ventilation performance testing detailed in the current ISO standard (section 6.7<5 kg),²⁰ the results suggest the standard may be inadequate to detect models of SIB that have manufacturing issues causing serious deficiencies in biomechanical performance. The estimated number of SIBs required for implementation of UN programmes in eight identified low-resource countries is approximately 400 000 units alone.^{29 30} Most of these countries are without an established regulatory authority to oversee the quality and safety of medical devices. The likelihood of the distribution of inferior devices with poor performance and unacceptable intrabatch variation is great.

In the absence of evidence from in vivo studies, the results of this study of SIB device performance provide valuable data for those making decisions regarding the brands and models that are most suitable for resource-constrained countries in the effort to reduce neonatal mortality.^{4 5}

A major concern arising from this study is that significant forward leakage via the PCV can lead to a complete lack of VT reaching the patient. The operator being unaware of this issue would likely react to poor clinical response by increasing the compression distance. At some point, the PCV closes resulting in delivery of a potentially excessive, harmful VT to the patient.

Conclusion

Half the SIBs examined in this study failing our testing method exhibited unacceptable performance variation between units despite all models passing current ISO standard criteria for delivered ventilation <5 kg. We conclude that the current ISO standard 10 651.4–2002 (reviewed 2013) for life-saving manual inflation devices specific to SIBs may need substantial revision. SIBs that were shown in this robotic simulation study to fail our testing methodology may potentially be unsafe for use with vulnerable newborn infants.