Predictive value of the user seal check in determining half-face respirator fit

Summary

Guidelines issued by the Centers for Disease Control and Prevention and the World Health Organization state that healthcare workers should wear N95 masks or higher-level protection during all contact with suspected cases of severe acute respiratory syndrome. Before use, the manufacturer recommends performing a user seal check to ensure that the mask is fitted correctly. This study aimed to test the ability of the user seal check to detect poorly fitting masks. This study is a retrospective review of a mask-fitting programme carried out in the intensive care unit of the Prince of Wales Hospital in Hong Kong. In this programme, all staff were tested with two types of N95 mask and one type of N100 mask. The results of the documented user seal check were then compared with the formal fit-test results from a PortaCount. Using a PortaCount reading of 100 as the criterion for a correctly fitted mask, the user seal check wrongly indicated that the mask fitted on 18–31% of occasions, and wrongly indicated that it did not fit on 21–40% of occasions. These data indicate that the user seal check should not be used as a surrogate fit test. Its usefulness as a pre-use test must also be questioned.

Introduction

Hong Kong Department of Health figures show that 22% of the cases of severe acute respiratory syndrome (SARS) in Hong Kong occurred in healthcare workers (http://www.info.gov.hk/dh/diseases/ap/eng/infected.htm). The Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO) currently recommend the use of N95 masks or higher-level protection to prevent the transmission of SARS to staff in these areas (http://www.cdc.gov/ncidod/sars/infectioncontrol.htm and http://www.who.int/csr/sars/infectioncontrol/en/).

In unfitted masks, the average penetration by ambient aerosol was found to be 33%, compared with 4% in fitted masks.¹ Due to the unreliability of an unfitted respirator, the National Institute for Occupational Safety and Health (NIOSH) has made fit testing of N95 respirators mandatory for tuberculosis prevention.² Both the CDC and the WHO recommend that fit testing should be carried out prior to use of N95 masks for SARS prevention. In the context of a SARS epidemic, however, fit testing a sufficient number of staff may cause logistic difficulties.

Prior to use of a respirator, the manufacturers recommend that the user should carry out a user seal or fit check to exclude gross leaks. It has been suggested that this check might be used as a surrogate for formal fit testing. We carried out this study to determine the false-positive and false-negative rates of a user seal check in determining the fit of disposable N95 and N100 respirators.

The NIOSH standards do not apply in Europe. In a healthcare setting, masks meeting the FFP2 standard are similar to N95 masks, and FFP3 masks are similar to masks meeting the N100 standard.

Methods

This was a retrospective analysis of data collected during an occupational safety programme for SARS and tuberculosis prevention for nurses working in the intensive care unit (ICU) of the Prince of Wales Hospital in Hong Kong.

All nurses were fit tested using a PortaCount Plus (TSI Incorporated, St Paul, Minnesota, USA) according to the protocol described in the US regulation, 29 CFR 1910.134.³ The PortaCount measures the number of ambient dust particles inside and outside the respirator, and calculates a fit factor that is a ratio of the two measurements. The machine runs in two modes; the N99/N100 mode and the N95 mode. In the N99/N100 mode, the device counts all particles sized between 0.02 and 1 μm diameter. In the N95 mode, only particles with a diameter of 0.04 μm are counted. One N100 mask (8233) and two N95 masks (1860s and 9210) (3M, St Paul, Minnesota, USA) were tested. Prior to carrying out each fit test, the nurse was asked to perform a user seal check, and to state whether or not she could detect a leak. The mask was considered to have passed the user seal check if no leak was detected.

Following the manufacturer's recommendation, the two N95 masks were tested using the N95 mode, and the N100 mask was tested using the N99/N100 mode.

All staff were already familiar with the 1860s and 8233 masks, as they had used them during the epidemic. Most staff had not used the 9210 mask previously. Prior to testing all of the masks, the staff were instructed in their use.

One modification was made to the PortaCount. The re-usable tubing supplied by the manufacturer was replaced with 150 cm of disposable PVC tubing of the same internal diameter to minimize any risk of cross-infection. As this tubing was longer than the tubing usually used in the N95 mode, the ambient purge time in this mode was increased to 15 s to compensate for the additional length. This time was found in separate testing to be 5 standard deviations greater than the average time required to purge this length of tubing.

To ensure an adequate ambient particle count throughout the testing, the 8026 Particle Generator (TSI Incorporated, St Paul, Minnesota, USA) was used to generate saline particles throughout the testing procedures.

The accuracy of the user seal check was scored against the quantitative fit test. Following the NIOSH guidelines, a fit factor of 100 on this test was used as the pass mark for each of the respirators.

The number of staff fitting each mask and the difference between the number of males and females were compared using the Chi-squared test (EPI-INFO v6, CDC). A P value <0.05 was considered to be significant.

Results

All nurses were of Chinese descent. The 1860s (N95) mask was tested in 82 female nurses and two male nurses, the 9210 (N95) mask in 81 females and 12 males, and the 8233 (N100) mask in 79 females and 12 males.

The user seal check was correct on 71–75% of occasions. Detailed results are shown in Table I . Fit factors for masks that had been incorrectly passed were 19–87 for the 1860s (N95) mask, 7.8–92 for the 9210 (N95) mask, and 12–91 for the 8233 (N100) mask. The user seal check was no more accurate in men compared with women; 50% vs 76% for the 1860s (N95) mask, 66% vs 74% for the 9210 (N95) mask, and 75% vs 70% for the 8233 (N100) mask (no significant differences). The 50% failure rate with the 1860s mask in men appears high; however, only two men were tested on this mask.

Table I.

Results of user seal checks in three masks

Mask	Number of staff	Fit-test pass rate (%)	User seal check correct (%)	User seal check incorrectly failed mask (% failures)	User seal check incorrectly passed mask (% passes)
1860s	84	69	75	40	19
9210	93	55	71	24	31
8233	91	70	73	45	18

The smaller proportion of staff fitting the 9210 (N95) mask, compared with the other two masks, approached significance (P=0.052).

Discussion

Data from the CDC indicate that for any given N95 respirator, the percentage of subjects who achieve an adequate fit ranges from 0 to 88%.¹ As a result, the CDC recommend that fit testing should be carried out on each individual to find a brand of respirator that achieves an adequate fit. To perform a quantitative fit test on one N95 mask using a PortaCount takes at least 10 min. Qualitative tests take even longer. As our data indicate that a given mask may fit only 70% of staff, many staff will need to be tested on more than one mask.

During an epidemic, these time constraints may make it difficult to fit test all staff before they need to use the respirators in actual clinical duties. It would be useful to have an alternative test, should it not be possible to carry out fit testing in all staff before their exposure to a potentially airborne pathogen. Unfortunately, our data indicate that the user seal check is incapable of performing this function as it has unacceptably high incorrect pass and failure rates. In some cases, the user felt that the mask fitted when the observed fit factor was very low.

The manufacturer's recommendations are that the user should perform a seal check after donning the respirator to test for gross leaks. As the masks were tested immediately after the user seal check was performed, our data indicate that this check seems to be of limited value in detecting such leaks. This is consistent with the findings of Delaney et al. ⁴ for full-face respirators.

We chose to use the PortaCount for the fit-testing procedure because it was quicker than NIOSH-approved methods such as aerosol testing. The results from Bitrex aerosol are similar to those from the PortaCount, so we believe that it is likely that the user seal check would perform similarly poorly when compared with this test. Saccharin aerosol has been shown to be less good at detecting induced leaks than Bitrex or the PortaCount, so we do not use this test in our institution. As all staff members had the seal of their masks visually checked by the fit-test operator prior to performing the test, a visual check by a colleague does not seem to be a good surrogate for formal fit testing.

The weaknesses of this study are that only three different masks were tested and most of the subjects were Chinese females. It is conceivable, although unlikely, that different results might be obtained with different masks, male subjects or subjects of a different race.

Retrospective studies may have problems with selection bias, or protocols not being followed properly. All of the staff in the ICU were tested, so we do not believe that we have selected a particular subset. The testing was performed towards the end of the SARS epidemic, so our staff had a significant incentive not to deviate from our written protocol. It is possible, however, that as this study was performed retrospectively, biases of this nature could have been introduced.

During the epidemic, it was difficult to identify brands of masks that fitted staff members. The three masks tested were chosen because they have been found previously to fit the greatest number of staff. Although the aim of the study was not to look at the pass rate of different N95 masks, we did note that these masks still had a relatively high failure rate. It is possible that this is because our subjects had smaller or different-shaped faces to the population used when the masks were designed. Similar testing carried out in North America, however, also showed a high failure rate.¹

In conclusion, the user seal check appears to be limited in its use as a surrogate for mask fit testing. If inadequate time is available to test staff using a full protocol, consideration could be given to using a shortened protocol such as that described by Sreenath et al. ⁵ The best strategy at present is to test staff routinely, prior to the outbreak of a respiratory epidemic.