Evaluation of N95 respirators, modified snorkel masks and low‐cost powered air‐purifying respirators: a prospective observational cohort study in healthcare workers

Summary

Disposable N95 respirator masks are the current standard for healthcare worker respiratory protection in the COVID‐19 pandemic. In addition to shortages, qualitative fit testing can have low sensitivity for detecting poor fit, leading to inconsistent protection. Multiple groups have developed alternative solutions such as modified snorkel masks to overcome these limitations, but validation of these solutions has been lacking. We sought to determine if N95s and snorkel masks with attached high‐efficiency filters provide consistent protection levels in healthcare workers and if the addition of positive pressure via an inexpensive powered‐air purifying respirator to the snorkel mask would provide enhanced protection. Fifty‐one healthcare workers who were qualitatively fitted with N95 masks underwent quantitative mask fit testing according to a simulated workplace exercise protocol. N95, snorkel masks with high‐efficiency filters and snorkel masks with powered‐air purifying respirators were tested. Respiratory filtration ratios were collected for each step and averaged to obtain an overall workplace protocol fit factor. Failure was defined as either an individual filtration ratio or an overall fit factor below 100. N95s and snorkel masks with high‐efficiency filters failed one or more testing steps in 59% and 20% of participants, respectively, and 24% and 12% failed overall fit factors, respectively. The snorkel masks with powered‐air purifying respirators had zero individual or overall failures. N95 and snorkel masks with high‐efficiency filter respirators were found to provide inconsistent respiratory protection in healthcare workers.

The burden of severe acute respiratory syndrome‐coronavirus‐2 (SARS‐CoV‐2) infections among healthcare workers continues to exceed that of the general community [1, 2, 3]. This could be due to knowledge gaps regarding the route of transmission and infectivity [4] or failure of personal protection systems.

Single‐use half‐face respirators (N95 in North America; FFP2 and FFP3 in Europe; and KN95 in China) are widely used for protection of healthcare workers from respiratory pathogens including SARS‐CoV‐2. These devices and standards are broadly equivalent. An N95 and FFP2 have a protection factor of 10, while an FFP3 has a protection factor of 20, which corresponds to 95% and 99% filtration capacities, respectively [5]. However, most international literature to date focuses specifically on N95s. Recent meta‐analyses suggest that N95s have failed to significantly reduce infections among healthcare workers compared with standard surgical masks [6, 7]. Available studies also suggest that N95s may not provide consistent performance, even after standardised fitting and training [8, 9]. Taken together, this literature suggests that N95 and comparable respirators do not provide adequate protection against droplets and close‐range aerosols.

The performance of N95s and other negative‐pressure respirators primarily depends on a stable face seal to prevent entrainment of unfiltered air [10]. Consistency of this seal is essential to protect healthcare workers, particularly anaesthetists. Anaesthetists are highly involved in the management of patients with COVID‐19, serving on rapid response teams, as well as caring for patients in the operating theatre and critical care units, all roles that involve exposure to aerosol‐generating procedures and increased SARS‐CoV‐2 transmission risk [2].

Limited global supply of N95 and comparable disposable and re‐usable respirators has led to the development of alternative methods of respiratory protection, including altering commercially available masks designed for other industries and applications, potentially resulting in inconsistent protection. Snorkel masks originally designed for underwater use have been widely repurposed during the current pandemic and modified with attached high‐efficiency filters because they are widely available and offer full‐face protection [11] . However, snorkel masks, like N95s, require a stable facial seal. In the absence of rigorous testing, snorkel masks may provide a false sense of security to healthcare workers [12].

Powered air‐purifying respirators (PAPRs) offer the highest level of protection when dealing with virulent airborne pathogens [13], were utilised extensively by the Chinese airway teams and may have led to the low healthcare worker infection rate seen in Wuhan [14]. Unfortunately, their limited availability has precluded their widespread use in the current pandemic. Further modification of snorkel masks with a small inexpensive fan to provide filtered positive air pressure and convert this into a PAPR could potentially overcome the requirement of a consistent facial seal and may provide an accessible option with superior protection.

We sought to determine the failure patterns and filtration efficacy of snorkel masks with a high‐efficiency filter, and snorkel masks equipped with a PAPR in comparison to qualitatively fit‐tested single‐use N95s in healthcare workers.

Methods

After obtaining institutional review board approval, a convenience sample of 54 healthcare workers were recruited who had been qualitatively fit tested and trained on disposable N95 mask usage by our institution’s occupational health department within the previous year. This was achieved via a hospital‐wide email to the peri‐operative departments asking for volunteers over the first 2 weeks of April 2020.

After informed consent, participants were quantitatively assessed for mask fit and filtration ratios using the aerosol condensation nuclei counter quantitative fit testing method (AccuFIT 9000, AccuTech Analytics, Tulsa, OK, USA) under three conditions – N95 respirators; snorkel masks with high‐efficiency filters; and snorkel masks equipped with a PAPR. The Canadian Standards Association Z94.4 simulated workplace exercise protocol for respirator selection was followed [15]. The Z94.4 standard utilises the same seven exercises as outlined by the International Organization of Standards 16975‐3 protocol which governs respirator fit [16]. The Z94.4 protocol quantitatively assesses the particulate count inside the respirator and compares that with the outside particulate count while the wearer participates in seven exercises: quiet breathing; deep breathing; moving the head side to side; moving the head up and down; speaking; bending forward and backward; and quiet breathing again. For each exercise, particle concentrations of 0.2–1 µm were measured inside the respirator and compared with ambient concentrations, and an individual filtration fraction of outside/inside concentration was generated. Individual filtration fraction measures for the seven exercises were then harmonically averaged to provide the simulated workplace exercise protocol factor. The Canadian Standards Association simulated workplace exercise protocol factor minimum standard for N95 (i.e. the minimum level of protection required when dealing with airborne pathogens) is 100 [14]. Thus, an individual filtration fraction of < 100 was deemed a ‘failure'.

Participants were first tested using a new, recently institutionally fit‐tested model of N95s. The simulated workplace protocol was then repeated for the snorkel mask with high‐efficiency filter and the snorkel masks with PAPR attachment.

The snorkel mask (Aria Ocean Reef, Vista, CA, USA) utilised air‐guard filters (Model 179‐0000, Intersurgical, Wokingham, UK; filter efficiency > 99.9%) connected via an adaptor similar to previously published designs [11].

The PAPR was constructed utilising 18‐volt rechargeable batteries (Ryobi, Hiroshima, Japan), and an off‐the‐shelf brushless motor fan (UTUO Brushless Radial Blower, UTUO Group, Wathai, China) capable of generating an average of 2 cmH₂O pressure inside the mask. Polyurethane enclosures and adapters were cast using custom‐designed 3D‐printed moulds created by our team. Two air‐guard filters were connected in parallel to the blower and standard 22‐mm anaesthetic tubing and one‐way valves were used for connections (Fig. 1). This device is not approved by any health regulatory agency at present. All components, including design files, detailed specifications and testing data are released under the CERN‐Open Hardware License 2.0‐Strongly Reciprocal and archived in a public repository (https://github.com/tgh‐apil/PAPR) to facilitate knowledge sharing and further development. The authors have no exclusive financial interest in the device. By virtue of the open hardware license, any member of the public has comparable rights to the authors to seek regulatory approval to manufacture and market the device.

Figure 1.

Modified snorkel mask with added powered‐air purifying filter. Masks were connected via standard 22‐mm anaesthetic tubing with a one‐way valve to a case housing a 12‐volt brushless fan powered by an 18‐volt battery. High‐efficiency filters are housed inside the case, which allows easy replacement.

Participants were asked to rate the following on a 1–5 scale: comfort, ‘how comfortable was the respirator to wear’; ease of donning and doffing, ‘how easy was it to put on and take off the respirator’; and the overall usability and practicality, ‘how practical do you think using this respirator for the day would be?’, of each respirator device. Data were analysed with non‐continuous tests (Friedman) after normality was assessed, or chi‐squared as appropriate, with the threshold for significance set at 0.05 (GraphPad Software, La Jolla, CA, USA).

Results

We enrolled 54 subjects, 51 of which completed the testing. Three subjects withdrew from the study due to time constraints and were not included in the analysis. A total of 357 testing steps were performed on each device. The characteristics of participants are shown in Table 1.

Table 1.

Baseline characteristics of participants in this study. Values are mean (SD) or number (proportion).

	Participants n = 51
Age; y	46.4 (11.5)
Sex; male	24 (47%)
Occupation
Physician	21 (41%)
Nurse	21 (41%)
Other	9 (18%)
BMI; kg.m−2	25.8 (4.3)
Ethnicity
Caucasian	32 (63%)
Asian	12 (23%)
South‐east Asian	3 (6%)
African American	4 (8%)

The average fit factor of the snorkel mask with attached PAPR (12,177) and snorkel mask with high‐efficiency filter (2939) was significantly higher than N95s (144) (Fig. 2). There was a significantly increased failure rate with N95s and the snorkel mask with the high‐efficiency filter compared with the PAPR (p < 0.01). On N95s, 59% of participants failed an individual step and 35% failed the overall test. On the snorkel mask with the high‐efficiency filter, 20% of participants failed an individual step and 8% failed overall. For the averaged overall test, 24% and 12% of participants passed on their disposable N95 or snorkel mask with the high‐efficiency filter, respectively, despite failing individual steps. The addition of a PAPR to the snorkel mask eliminated both individual and overall failures. The pattern and distribution of individual step and overall test results are presented in Fig. 3. It is important to note that although the standard criteria for an overall pass permits failure on up to two individual steps, such failures still represent a vulnerability for the individual in question. The addition of a small amount of positive pressure appears to eliminate this vulnerability.

Figure 2.

Average simulated workplace protection factor for 51 participants wearing N95, snorkel masks with high‐efficiency filters (SM‐HEF), and snorkel masks with powered air‐purifying respirators (SM‐PAPR). *, p < 0.05 when compared with N95.

Figure 3.

Individual results for 51 participants wearing (a) N95; (b) snorkel masks with a high‐efficiency filter; and (c) snorkel masks with a powered air‐purifying respirator undergoing a seven‐step workplace exercise protocol. Results are harmonically averaged to provide an overall simulated workplace protection factor (SWEP‐F). Green boxes represent filtration ratios ≥ 100 (pass), red boxes represent filtration ratios < 100 (fail).

Mean (SD) comfort was significantly greater with the PAPR compared with the disposable N95 or snorkel mask with high‐efficiency filter (3.9 (1.4), 3.1 (1.2) and 2.9 (1.2), respectively; p < 0.001). Mean (SD) usability and practicality of the PAPR was greater than both N95 and snorkel masks with high‐efficiency filters (3.1 (1.2), 2.5 (1.0), 2.6 (1.0), respectively; p = 0.005). There were no significant differences between the PAPR, N95 or snorkel mask with a high‐efficiency filter in ease of donning or doffing (3.6 (1.0), 3.5 (1.1), 3.6 (1.0), respectively; p = 0.91).

Discussion

The main finding of this study is that qualitatively fit‐tested disposable N95s and snorkel masks with high‐efficiency filters have significant failures and do not provide consistent protection in trained and tested healthcare workers.

Current mask fit testing protocols can be broadly divided into qualitative and quantitative methods. Qualitative methods rely on detection of a bitter substance such as Bitrex, while quantitative fit testing calculates an internal to external particulate ratio, which is then mathematically averaged to provide a ‘pass’.

In this study, 35% of participants who had previously passed the institutional qualitative fit testing and been trained on the use of N95s within the previous 12 months (most within the previous month), failed the overall quantitative fit test.

Given that viral exposure can occur when a mask fails an individual fit exercise point, our observations suggest that current qualitative respirator fit protocols, as well as quantitative protocols that focus on averaged values (such as the Canadian Z94.4 protocol and International Organization for Standardization (ISO)) have limited relevance when dealing with biological agents. Currently, a ‘pass’ relies on exceeding an average predetermined value, instead of ensuring all individual steps exceed this value. However, 24% of participants who passed the overall quantitative test still failed one or more individual steps, such as talking or bending over. This suggests that qualitatively fit‐tested disposable N95s may not provide consistent protection against respiratory aerosols in healthcare workers.

The snorkel masks with high‐efficiency filters demonstrated improved consistency compared with N95s but were also not completely reliable. While the overall test failure rate was much lower (8%), 12% of participants using snorkel masks with high‐efficiency filters still passed the overall test despite failing an individual step. The risk posed by seal breaches in snorkel masks may be greater than that posed by seal breaches in N95s. A study by Zhu et al. suggests that when utilised in the healthcare environment, an unnoticed seal leak in an elastomeric respirator may paradoxically result in worse protection than an N95 [10]. Together, these findings suggest that significant caution should be exercised when considering the use of repurposed snorkel masks for high‐risk clinical situations. However, our data show that the risks associated with modified snorkel masks can be significantly mitigated with the addition of an inexpensive fan and filter providing a small amount of positive pressure.

A limitation of this study is its single‐centre design and small sample size. Our sample was recruited by asking for volunteers, which may potentially lead to selection bias, though we attempted to control for this by ensuring a diverse ethnic, age and occupation sample that is broadly reflective of the local population. To achieve the National Institute for Occupational Safety and Health standards, a morphometric fit panel is usually required, and ideally, the mask could be tested on the ISO digital head forms [17]. As we did not have access to the morphometric fit panel, we sought to ensure a diverse ethnic sample and used a sample size that exceeds what has been previously recommended (35–40) for respirator fit test panels [18] to mitigate this risk. Further, due to limited commercial supply, we were unable to test re‐usable half‐face respirators.

The cost of snorkel masks with a high‐efficiency filter was approximately £58 (€63, $100 CAD). We estimate the production cost of the snorkel mask with attached PAPR to be about £144 (€157, $250 CAD) per unit for a run of 1000 units. This is equivalent to 50–100 disposable N95 masks that cost between £1 and £3 (€1–3, $2–5 CAD), suggesting roughly equivalent costs with long‐term use.

The goals of personal protective equipment are to protect healthcare workers and to minimise the risk of infection. Current negative pressure respirators such as N95s and snorkel masks demonstrated significant protection limitations in the current investigation. The optimal PPE strategy for aerosolised vectors minimises potential faults in mask fit, allowing consistent protection, which is critical when dealing with a biological vector where even a small number of particles may result in infection. Quantitative fit testing that focuses on passing all steps, as opposed to passing overall, can help identify individual fit faults and guide N95 respirator choice to allow the achievement of consistent fit.

Our findings suggest that respirators with positive pressure are likely to provide more consistent respiratory protection. Further design and manufacturing optimisation, as well as testing to ensure compliance with technical and regulatory standards, are required for cost‐effective devices to be brought into routine use in high‐risk healthcare settings.