Selecting models for a respiratory protection program: What can we learn from the scientific literature?

Abstract

Background

An unbiased source of comparable respirator performance data would be helpful in setting up a hospital respiratory protection program.

Methods

The scientific literature was examined to assess the extent to which performance data (respirator fit, comfort and usability) from N95 filtering facepiece respirator (FFR) models are available to assist with FFR model selection and procurement decisions.

Results

Ten studies were identified that met the search criteria for fit, whereas 5 studies met the criteria for comfort and usability.

Conclusion

Analysis of these studies indicated that it is difficult to directly use the scientific literature to inform the FFR selection process because of differences in study populations, methodologies, and other factors. Although there does not appear to be a single best fitting FFR, studies demonstrate that fit testing programs can be designed to successfully fit nearly all workers with existing products. Comfort and usability are difficult to quantify. Among the studies found, no significant differences were noted.

Implementing an effective respiratory protection program is important. According to U.S. federal regulations enforced by the Occupational Safety and Health Administration (OSHA), the respirator program must be overseen by a qualified administrator and include written procedures governing respirator use at that site.¹ In addition to implementing respiratory protection programs to reduce health care worker (HCW) exposure to routine infectious diseases (eg, tuberculosis), hospitals are purchasing and stockpiling respirators (typically filtering facepiece respirators [FFRs]) in preparation for future possible public health emergencies (eg, respiratory pathogen outbreak, pandemic).² Health care compliance with the OSHA’s respiratory protection program requirements is mixed.³ For example, a recent study evaluating a hospital respiratory protection program in California during the 2009–10 H1N1 influenza pandemic found that only 1 of 16 programs would be considered complete.⁴

One of the key components of a respiratory protection program is that all workers that need to wear a tight-fitting respirator must be qualified via a fit test to wear ≥1 specific models.¹ Lee et al described a process to minimize the number of FFR models necessary to successfully qualify a large HCW population.⁵ Using fewer models can simplify inventory management, training, and fit testing programs and minimize worker confusion as to which FFR to wear. The suggested procedure involves a team composed of both respiratory protection program management and potential FFR users. This bilateral approach is critical for 2 important reasons. First, employee participation and acceptance have been recognized as important to the success of any safety and health program for many years.⁶ Second, it is clear that even well-fitting FFR can protect users only if they are worn for the entire exposure period.⁷ HCWs have finite tolerance for the subjective discomfort and job interference FFRs can cause.^8,9 Therefore, FFR comfort assessment by potential users increases the likelihood of success.

However, this best practice process still requires selecting specific respirator models from 1 or more vendors (eg, respirator manufacturers, distributors). The general parameters for selection and procurement are typically established by management and often consider availability from the supplier and ability to fit employees.³ The OSHA requires that respirators be certified by the National Institute for Occupational Safety and Health (NIOSH). However, there are hundreds of NIOSH-certified respirator models available on the market today. Even for respirators designed for use in surgical settings where clearance by the Food and Drug Administration (FDA) as a medical device is important, there are still over a dozen different FFR manufacturers, with some manufacturers offering multiple models of varying features and styles. Although NIOSH certification is required by the OSHA and FDA clearance in certain situations is needed, these federal agencies only provide general pass or fail information about the performance of the products they regulate.

Although vendors are often willing and highly capable of assisting in the selection process, it would be advantageous to have an unbiased source of comparable respirator performance data. There is no third party clearing house or Consumer Reports-type publication that compares respirator performance data among brands and models. The need for this type of information has been discussed.^3,5,10 In its 2010 report, the Institute of Medicine discussed progress made in the area of personal protective equipment for HCWs and identified future research needs.² One of the recommendations for future activities was “To improve consumer and purchaser information on fit capabilities, NIOSH should establish a website to disseminate fit test results for specific respirator models on an anthropometric (NIOSH) test panel, where such data exists.” In addition to fit on anthropometric panels, respiratory protection program administrators may benefit from test results for specific models from any well-designed study involving human test subjects or actual workers.

This work was undertaken to assess the scientific literature for respirator performance data to assist with FFR model selection and procurement decisions. The review was focused on NIOSH-certified N95 class FFRs because this type is widely used in health care.

METHODS

Assessment of fit test methods

FFRs must fit properly to provide maximum protection. Mandatory procedures for assuring proper fit can be found in the OSHA Respiratory Protection standard 29 CFR 1910.134.¹ The methods applicable to the N95 FFRs include qualitative fit tests using either saccharin or denatonium benzoate (Macfarlan Smith Ltd, Edinburgh, U.K.) aerosols. Adequate fit is indicated if the test subject does not detect the sweet or bitter taste of saccharin or Bitrex, respectively, while performing a series of 7 specified test exercises. Alternatively, the OSHA permits a quantitative fit test using ambient aerosols to numerically estimate how well the FFR fits the user by measuring the aerosol particle ratio outside (C_o) and inside (C_i) the device. The only instrument currently sold to make these measurements with N95 FFR is the PortaCount Pro+ Fit Tester 8038 (TSI, St Paul, MN) or its predecessor, the PortaCount 8020 and N95-Companion (TSI, St Paul, MN). Both instruments are hereafter referred to as the N95-Companion. The harmonic mean of C_o:C_i ratios measured during individual test exercises is known as a quantitative fit factor. The OSHA defines a fit factor of ≥100 as acceptable for FFRs. It is also important to recognize that the 2 qualitative fit tests were developed to screen for a minimum fit factor of 100.^11,12

Validation of these fit test methods has been done using a generated aerosol quantitative fit test.^11–13 They are therefore considered equivalent to one another for fit testing FFRs. Interestingly, they do not always produce identical pass or fail results.^14–16 Nonetheless, workplace protection factor studies demonstrate that workers fit tested with each method receive expected levels of protection.^17–21

Therefore, only Bitrex, saccharin, and the N95-Companion fit test passing rates were considered in the analysis. Laboratory results gathered to compare the efficacy of various fit test methods or in the development of new fit test methods were not considered because these methods have not been validated. Studies that pre-screened test subjects to eliminate those that could not pass a fit test were excluded because they potentially skew the true pass and fail rate.

Assessment of comfort and usability test methods

The OSHA does not have a comfort or usability requirement. Furthermore, the NIOSH and FDA do not assess these parameters as part of their certification and clearance processes. One promising respirator evaluation tool considers comfort, aesthetics, and somatic impact.²² Some objective physiologic data (eg, heart rates, air and skin temperatures, humidity levels) exist to compare FFR models,^23–27 but studies to correlate these data with comfort and tolerability are just emerging. Unlike fit or human physiologic data, assessment of comfort and usability is almost entirely subjective. Test subjects are typically asked to rate comfort using a visual or numerical scale. The ends of the scale are identified with terms such as very comfortable and very uncomfortable. No standardized criteria exist by which subjects are to rate comfort, and with current methodology, an FFR that one subject finds comfortable may be uncomfortable to another. However, trends across respirator types (eg, FFRs vs elastomeric half-mask air purifying respirators) appear consistent across studies. Evaluating FFR usability presents similar challenges. No performance standards exist, and acceptable usability of an FFR is largely defined by the user and work environment.

Despite these challenges, our analysis considered articles in which multiple NIOSH-certified N95 FFR models were assessed or compared against each other using some type of standardized questionnaire administered to human test subjects either during or immediately following respirator use. This criterion excludes studies in which only a single N95 FFR model was identified. Because there is no standard way of testing for these parameters or generally agreed on acceptable or unacceptable levels, it is not possible to compare findings across these types of studies.

Search criteria

The internet search engine Google Scholar was used to identify articles published in the peer-reviewed literature that identified individual NIOSH-certified N95-class FFR models by name and included possible comparable performance data using the fit and comfort and usability tests previously described. Published technical reports from government or end-user organizations were also given consideration. Numerous search terms were used, including N95, FFR, filtering facepiece respirator, and facemask, as were the names of common respirator models used in health care. We also searched the Web sites of journals likely to publish articles describing implementation of a respiratory protection program in health care, including Infection Control and Hospital Epidemiology, American Journal of Infection Control, Journal of Hospital Infection, Annals of Occupational Hygiene, Journal of Occupational and Environmental Hygiene, and Journal of the International Society for Respiratory Protection. Citation lists from articles meeting eligibility requirements were also reviewed as a possible source of material. Because manufacturers sometimes update their products without changing model numbers, to increase the relevancy of the data, we limited the search to the years 2003–2013. When assessing articles for eligibility, no attempt was made to confirm data with the investigators of the study.

RESULTS

Over 100 full-text articles were found using the search terms and Web sites selected. One of the authors reviewed the articles and assessed each for eligibility. No articles meeting the eligibility requirements were excluded.

Summary of articles with fit test results

Table 1 summarizes the results of the literature survey. Ten studies met our eligibility criteria. There were 3 laboratory studies that used test subjects who were not HCWs. Seven studies used HCWs or student nurses for test subjects, which we refer to as applied studies. Salient points from the articles are subsequently summarized for completeness.

Table 1.

Summary of fit test studies involving NIOSH-certified N95 FFRs

Study	Fit test method(s)	Study population	N95 models tested: passes/no. of tests	Comments
Coffey et al (2004)14	Bitrex Saccharin PortaCount	25 U.S. subjects, LANL panel	MSA Affinity Plus: 35/50 3M 8210, 8110S: 25/50 3M 1860, 1860S: 25/50 Moldex 2200N95/2201N95: 21/50 Willson 9510F: 22/50	18 models from 9 manufacturers* Passing rates are the sum of Bitrex and N95-Companion passes for the 5 best fitting FFRs
Lawrence et al (2006)16	Bitrex Saccharin PortaCount	25 U.S. subjects, various facial sizes	3M 9210: 25/75 Gerson 3945: 25/75 Gerson 1730: 27/75 3M 8515: 22/75 Moldex 2600, 2601: 16/75	15 models from 10 manufacturers* Passing rates are the sum of passes for all 3 fit tests for the 5 best fitting FFRs
Berry Ann (2010)28	PortaCount	40 U.S. subjects, NIOSH bivariate panel	Lot1: 22/40 Lot2: 25/40	2 manufacturing lots were tested; pass defined as at least 1 fit factor >100 out of 3 trials per subject
Derrick et al (2005)29	PortaCount	84–93 Chinese health care workers	3M 1860S: 58/84 3M 9210: 51/93	Study also included the 3M 8233 N100 FFR
Lam et al (2011)30	PortaCount	204 Chinese nursing students	3M 1860S: 125/204	Study also included the 3M 1862 (European P2)
Lam et al (2011)31	PortaCount	349 Chinese nursing students	3M 1860S: 227/349	Study also included the 3M 1862 (European P2)
Lee et al (2008)32	Bitrex	43 Canadian health care workers	3M 8210: 32/43 3M 8110S: 10/11 3M 9210: 1/1	Initial fit test results after training
Winter et al (2010)33	Saccharin	50 Australian hospital staff volunteers	Kimberly-Clark PFR95 N95: 14/50	Study also included 3M 8822 and 9320 (Australian P2)
Lee et al (2004)5	PortaCount	Pilot: 40 U.S. health care workers. Workplace: 1,850 U.S. health care workers	Pilot: Aearo Pleats Plus N95 (M/S): 38/40 3M 1860, 1860S: 30/40 Willson N9510F: 27/40 Survivair 1930: 18/40 MSA FR200: 3/40 Workplace: Aearo Pleats Plus N95 (M/S): 1,793/1,830 3M 1860, 1860S: 50/57	Pilot study was used to select the FFRs for the workplace study; 99.6% of the worker population fitted with 1 of these 2 FFRs
McMahon et al (2008)34	Bitrex	1,271 Canadian health care workers	3M 1870: 1,115/1,271 3M 1860, 1860S: 119/156

FFR, filtering facepiece respirator; LANL, Los Alamos National Laboratory; NIOSH, National Institute for Occupational Safety and Health.

^*See citation for complete list of NIOSH-certified N95 FFRs tested.

Laboratory studies

Coffey et al,¹⁴ Lawrence et al,¹⁶ and Berry Ann²⁸ reported fit test passing rates using laboratory test subjects. Coffey et al and Lawrence et al evaluated passing rates using both qualitative and quantitative fit tests, whereas Berry Ann used only the N95-Companion quantitative fit test. The former 2 studies measured passing rates for multiple FFR models and also compared rates for each fit test method. Berry Ann tested a single FFR model.

Coffey et al found passing rates averaging approximately 30% overall for the 3 test methods. Individual FFR pass rates ranged from 13%–55% for Bitrex, 0%–44% for Saccharin, and 0%–88% for the N95-Companion. Lawrence et al reported overall passing rates of 11% for Bitrex and saccharin and 22% for the N95-Companion. For individual devices the ranges were 0%–36% for Bitrex, 0%–32% for saccharin, and 0%–60% for the N95-Companion.

Berry Ann²⁸ reported the results of an NIOSH evaluation of fit test passing rates for the 3M 8000 FFR (3M, St. Paul, MN). The FFR investigated had been reported to fit none of approximately 20 HCWs tested with Bitrex by a large health care employer, and subsequent N95-Companion testing of 20 more workers found a passing rate of 40%. The passing rates Berry Ann found (55% and 62%) were deemed to be within the expected range for similar FFRs.

Applied studies

Studies by Derrick et al²⁹ and Lam et al^30,31 evaluated the ability of user seal checks to predict acceptable fit for HCWs or student nurses. To address that objective, they reported fit test passing rates for respirators commonly found in Hong Kong health care settings using the N95-Companion. Passing rates for specific respirator models ranged from 55%–69% and were comparable among the 3 studies. The studies also demonstrated that user seal checks are not an acceptable surrogate for fit testing.

Two articles reported fit testing results before and after test subjects were trained in proper respirator donning.^32,33 Both studies used qualitative fit tests and found that subject training improved passing rates. Nonetheless, Lee et al³² reported fit test failure rates of 25%–50% in follow-up fit tests conducted at 3 and 14 months for HCWs who did not regularly use the respirators. Fit tests among regular FFR users were higher, leading the authors to conclude that regular FFR use is necessary to see the benefits of fit test programs.

Lee et al⁵ and McMahon et al³⁴ described procedures for fitting large populations of HCWs with quantitative or qualitative fit testing, respectively. Lee et al used a pilot study involving both management and HCWs to choose respirator models for testing the entire HCW population. Each respirator was available in 2 sizes. The primary FFR model was tested first on each worker; the secondary model was used only if the fit test with the primary model failed. McMahon et al also used a primary and secondary model approach for testing the HCW population. If the fit test failed with the primary model, the test conductor selected a standard or small-size secondary device based on the perceived facial dimensions of the subject. Additional FFR models were available if the second fit test also failed. Both Lee et al and McMahon et al were able to successfully fit >99% of the workers tested.

Summary of comfort and usability studies

Only 5 articles in the peer-reviewed literature were found that met our search criteria. Unfortunately, none of these studies measured usability directly; however, 1 study did assess ease of donning (a component of usability) in addition to comfort. The 4 other studies evaluated comfort.

In addition to fit testing, Lee et al asked each of the 40 subjects in their pilot phase to give a simple yes or no rating for comfort at the end of each respirator fit test.⁵ Interestingly, comfort rates varied in proportion to fit test pass rates. The 2 best fitting models, Aearo Pleats Plus N95 (Aearo, Southbridge, MA) and 3M 1860 (3M, St. Paul, MN), also had the 2 highest comfort rates (77% and 72%, respectively).

Bryce et al³⁵ collected questionnaires from 137 HCWs (deemed to be frequent users of N95 FFRs) in Canada. The following FFRs were used in the study: 3M 1860, 3M 1870, and Kimberly-Clark PFR95 N95 (Kimberly-Clark, Roswell, GA). Respondents (77% women) compared the respirators with surgical masks using a 6-point Likert scale on factors such as comfort, shortness of breath, claustrophobia, and whether it caused dizziness. No significant differences in mean comfort scores were observed.

Viscusi et al³⁶ collected participants’ responses to several subjective criteria in a study of effects of decontamination on 6 FFR models. The following FFRs were used in the study: 3M 8000, 3M 8210, Moldex 2200N95 (Moldex-Metric, Culver City, CA), 3M 1860, 3M 1870, and Kimberly-Clark PFR95-270 (Kimberly-Clark, Roswell, GA). Eighteen subjects (50% women) provided responses for 1–6 of the FFRs on which they had first been successfully fit tested. The subjective characteristics rated included comfort, ease of donning, and odor before and after 3 decontamination treatments. Only small numerical differences in the ratings of these characteristics were found among the FFRs tested, and their meaning is not clear. The investigators also collected subjects’ comments about each device, again with no clear significant differences seen among them.

Roberge et al²⁶ reported subjective data from 20 subjects (13 men, 7 women) every 5 minutes during 1 hour of low to moderate work rate treadmill exercise. Four models were evaluated: Moldex 2200N95 and 2300N95 (Moldex-Metric, Culver City, CA) and 3M 9210 and 9211 (3M, St. Paul, MN). These 4 models were selected to compare 2 shapes (cup [Moldex devices] vs flat fold [3M devices]) and 2 configurations (exhalation valve [Moldex 2300N95 and 3M 9211] vs no exhalation valve [Moldex 2200N925 and 3M 9210]). Each subjects’ rating of perceived exertion was measured using a Borg scale, whereas rating of heat perception was measured using the Frank scale. Overall, the differences across products were small. The average exertion ranged from 9.6–10.6 (fairly light exertion), whereas average heat perception varied from only 5.8–6.1 (neutral to slightly hot). Unfortunately, the study design called for statistical analysis to compare the measured parameters versus the control (no respirator) and not to compare model versus model. Compared with the control, only the 3M 9211 was not statistically different. The other 3 models resulted in higher levels of exertion and heat perception compared with wearing no respirator.

Shenal et al³⁷ reported self-perceived discomfort using a visual analog scale and perceived exertion using the Borg scale from 27 HCWs (aged 24–65 years, 55% women) wearing different types of respirators and facial protection equipment during an 8-hour shift. Three N95 FFRs were included: 3M 1860 (cup shaped), 3M 8511 (cup shaped with exhalation valve, 3M, St. Paul, MN), and Kimberly-Clark PFR95170 (duckbill, Kimberly-Clark, Roswell, GA). Other configurations included in the study included respirators with a surgical mask overlay and 2 reusable respirators. Unfortunately, the focus of this article was on comparisons across respirator types and effects of the surgical mask overlay, and there was no statistical analysis comparing the N95 FFR models. However, analysis of the data available from the 2 figures in the article indicates that after 8 hours, the N95 FFR model with the exhalation valve had the lowest average perceived exertion (~10.5) and perceived discomfort level (~3.5). The average perceived exertion levels at the 2-hour point reported in this study (~9–10.5) are in the same range as those reported by Roberge et al²⁶ for 1 hour of low to moderate exercise.

DISCUSSION

Among the performance characteristics considered in the study, only fit testing can be measured using mature and standard test methods that permit trend analysis. Fit testing is a necessary component of a respiratory protection program. Unlike fit, comfort and usability assessment are not specific respirator program requirements, and the development of standardized test methods is still emerging. However, poor comfort and usability have been identified as causes of poor compliance.^2,8,9,27,38

Analysis of Table 1 suggests that only generalizations can be made regarding best fitting respirator models. That is, an FFR that fits a large percentage of those tested in a particular study may not necessarily perform as well in every situation. For example, in the most extreme difference found, the Aearo Pleats Plus N95 was successfully fitted on 98% of workers (1,793/1,830) tested by Lee⁵ but fit only 2 of 25 test subjects (8%) in the study reported by Coffey et al.¹⁴ These kinds of performance differences could be caused by changes made to the respirator between the studies, different test populations (workers vs test subjects, ethnicity, sex, etc), slight differences in methodology (eg, how the respirator was selected, number of retries allowed), skill level of the person doing the fit test or being fit tested, or other unknown factors. Nonetheless, FFRs that have fit higher percentages of those tested in ≥1 studies are logical choices to include when starting a fit testing program.

Interestingly, the highest reported fit test pass rates (>80%) were from actual workplaces with >1,000 employees. This is consistent with other studies that did not meet our search criteria because they did not identify pass or fail rates for specific models but instead included pass or fail rates for the overall program. For example, in a report published by Workers Compensation Board of British Columbia, 784 participants were assessed for initial respirator fit to be enrolled in a multiyear respirator study.³⁹ Of those, 86% (674) were able to pass both a Bitrex and a N95-PortaCount fit test on either the 3M 1860, 1860S, or 1870. They also observed that regular usage of FFRs resulted in a lower fit test failure rate and increased the proficiency of FFR donning skills. Another study that did not meet our search criteria involved 3,554 female and 969 male Australian HCWs.⁴⁰ Five different FFRs were chosen for testing based on their common usage in health care settings within the region: 3M 1870, Smith & Nephew ProShield (medium and small, Victoria, Australia), and Kimberly-Clark PFR95 N95 (medium and small). Fit test operators chose the most appropriate respirator based on each test subject’s facial characteristics, appearance of fit, and a screening measurement with the N95-Companion. Using this procedure 82.9% of successful fits were achieved with the first FFR, 12.3% succeeded with the second model, and 4.8% needed ≥3 tests. The overall test population success rate was 89.6% for men and 91% for women. The 3M 1870 accounted for 60.9% of the successful fits. Another important observation in this study was that fit test passing rates increased over time, suggesting that the skill of the person doing the fit test may have an effect.

There are few comfort and usability studies. Those summarized have not shown significant differences among FFR models; however, 2 studies did observe that an N95 FFR with an exhalation valve was slightly more comfortable. Because comfort and usability are individual judgments, HCWs should be given as much input as possible in the FFR selection process. Potential users should have ≥2 models of FFRs available to allow them to select the one they find most comfortable. The NIOSH and the Veterans Health Administration are working on a Project Better Respiratory Equipment Using Advanced Technologies for Healthcare Employees that seeks to promote better, more comfortable respirators for use in health care settings. Part of that effort includes development of the B95 standard, which would include testing for comfort.³⁸ The development of standardized test methods assessing comfort and usability will likely spur additional research in this area, leading to increased publications for future trend analysis.

There were several challenges identified in using the scientific literature as a possible tool for model selection. Many articles do not identify the respirators tested by make and model. Although there are certainly reasons not to identify commercial products by name, respiratory protection program administrators are denied a potential source of unbiased information. Another complication was that respirators are certified in different counties (NIOSH, European Union, Japan, Australia, etc). Manufacturers can make very similar models but use a different model number for each certification body. Without specific information from the manufacturer, data collected by researchers in one country cannot always be compared directly with models available in other parts of the world. There are global initiatives to develop international respirator standards that would help in this regard.

CONCLUSIONS

The primary conclusion is that it is difficult to use the scientific literature directly to inform the respirator selection process. Differences in methodology, study populations, skill level of the fit tester, and inherent errors in fit testing contribute to the variation seen when looking for trends in fit test pass rates across studies. Although there does not appear to be a single best fitting FFR, several studies demonstrate that fit testing programs can be designed to successfully fit nearly all workers with existing FFR. Program factors, such as routine usage and the skill of the fit tester, are likely important. The specific FFR models that are best in a given workplace likely depend on worker characteristics that include facial dimensions and ethnicity. Other worker characteristics that may affect fit include age and sex; however, studies are inconclusive.

Comfort and usability are nebulous terms that do not lend themselves to objective assessment. Among the FFRs studied, no significant differences in these traits have yet been described in the literature. Nonetheless, in every case worker input should be sought in setting up the respiratory protection program, and emphasis should be placed on selecting the most subjectively comfortable and usable devices.