A General Framework to Test and Evaluate Filtering Facepiece Respirators Considered for Crisis Capacity Use as a Strategy to Optimize Supply

Abstract

During a public health emergency, respirator shortages can have a profound impact on the national response, such as for the current coronavirus disease 2019 (COVID-19) pandemic. Due to a severe shortage of respirators (particularly filtering facepiece respirators [FFRs]), there may be contexts in which understanding the performance of FFRs that are approved for use as part of a crisis capacity strategy is desired. This includes FFRs that are not covered under the National Institute for Occupational Safety and Health (NIOSH) Respirator Approval Program because they have been stored past their designated shelf life, have been decontaminated, or are approved by international certification bodies other than NIOSH. The purpose of this document is to provide a general framework to assess the performance of FFRs that are only being used as a crisis capacity strategy. The intended audience are those who are responsible for managing large amounts of FFRs. This framework includes a four-step process consisting of: 1) defining the population of FFRs to be sampled; 2) providing sampling strategy options; 3) inspecting and testing the sampled units; and 4) evaluating the results. In addition to the four-step process, we provide an example of how NIOSH recently evaluated the quality of FFRs sampled from ten U.S. stockpiles.

INTRODUCTION

Personal protective equipment (PPE), such as respirators, gloves, and gowns, are critical to ensuring the health and safety of healthcare personnel (HCP), especially during pandemics and widespread public health emergencies. During these events, the increased need for PPE, such as filtering facepiece respirators (FFRs), may lead to shortages. During the COVID-19 response, and as part of crisis capacity strategies¹, numerous methods of supplementing the supply chain of FFRs have been used—e.g., the release of FFRs from stockpile facilities, extended use and limited reuse of FFRs (CDC, 2020a), the use of FFRs past their recommended shelf life (CDC, 2020a; CDC, 2020b), reuse following decontamination (CDC, 2020c), and the use of FFRs imported from other countries² (CDC, 2020d). For any of these strategies, assessing the filtration performance of those FFRs supports crisis capacity strategy implementation efforts.

Stockpiles are an important way to prepare for public health emergencies, and are common at the hospital, local, state, regional, and national levels (Pyrek, 2014; Swaminathan et al., 2007; Hashikyra & Kizu, 2009; Carias et al., 2015; Abramovich et al., 2017; Radonovich et al., 2009; WHO, 2007; Courtney et al., 2009; Esbitt, 2003). Given that FFRs are not typically designed to be stored for long periods and are not required to establish one for NIOSH certification³, manufacturers commonly assign a five-year shelf life.⁴ If shelf lives are designated as part of the respirator approval (e.g., on the user instructions or part of the respirator packaging), respirators that are past their designated shelf life are no longer considered NIOSH-approved as all conditions of use set by the approval holder must be met.⁵ During pandemics, millions of FFRs that may have been stored for a prolonged period may become available for use; however, their viability may be questioned since some FFRs may be beyond their designated shelf life. Given the significant investment in procurement and storage, along with projected demands for respirators during public health emergencies, discarding large quantities of potentially viable respirators is not an attractive option. In addition to stockpiling respirators, other strategies such as extended use and limited reuse, use after decontamination, and use of imported respirators are currently being utilized to optimize respirator supply during the current COVID-19 pandemic.

An adequate evaluation of FFR performance typically requires destructive testing to assess its protective properties. While testing a large number of samples will result in higher confidence in population estimates, depleting the stock of critical assets through excessive testing is not ideal. Further, given the associated expense of testing FFRs, a performance evaluation should balance costs of testing and FFR attrition with a level of accuracy sufficient to evaluate protection. Instead, representative samples should be taken and evaluated to ensure that the population of FFRs provide the expected level of protection to the end user. In addition, if testing is required in a time-sensitive situation, sampling and testing should be designed to be quickly performed. A general framework to verify performance, however, is not readily available (Greenawald et al., 2020; Greenawald et al., unpublished).

A pressing need, therefore, is a general framework on testing and evaluating FFRs subjected to the strategies noted above in order to understand their viability. In this paper, we aim to provide a general approach regarding the process of testing and evaluating FFRs through a structured assessment; this general approach is separate from the formal evaluation approach as part of NIOSH’s comprehensive Respirator Approval Program. A four-step process is proposed and consists of: 1) defining the population of FFRs to be sampled; 2) providing sampling strategy options; 3) inspecting and testing the samples; and 4) evaluating the results. This framework may be used during implementation of crisis capacity strategies to quantify the performance of FFRs that would otherwise not be used in U.S. healthcare settings. The methods described can be applied to FFRs approved through international certification bodies, those that may be beyond the shelf life designated by the manufacturer, and those that may have undergone sterilization or decontamination procedures. This framework does not apply to approval holders that are formally part of NIOSH’s Respirator Approval Program⁶, which includes the comprehensive certification of respirators set forth in 42 Code of Federal Regulations (CFR) Part 84 (NIOSH, 2020b).⁷ This paper employs sampling methods and quality assessment approach from NIOSH’s previous works: Yorio et al. (2019) and Greenawald et al. (2020).

GENERAL FRAMWORK

Overview of a General Framework to Test and Evaluate the Viability of FFRs That are Used as Crisis Capacity Strategies

This general framework is intended to inform those responsible for large amounts of FFRs (e.g., stockpile and inventory managers, PPE purchasers, policy makers, or entities that may be accepting donated respirators, mass-purchasing respirators approved by international certification bodies, or decontaminating large amounts of respirators) to determine the performance viability of N95 FFRs, or those with similar or equivalent protection, that are used as crisis capacity strategies. Wthin this framework, a general four-step process to verify FFR performance is first presented, followed by an example of how the general steps were used by NIOSH to test and evaluate stockpiled FFRs. In addition, information to verify the current approval status of NIOSH-approved respirators is provided.

Step 1: Defining the Population of FFRs to be Sampled

First, the population of FFRs must be defined. The population must be carefully chosen and defined so that inferences made using the testing results are properly assigned. As a general rule for assessing FFR performance, the population should be the smallest suspected unit of consistent quality. In practice, properly defining and accounting for the entire population of FFRs that quality inferences are desired for may be easy or challenging. When FFR models are assigned a lot number corresponding to a specific manufacturing batch, the most conservative population for a study of FFR performance quality may be the production lot of a given model (Yorio et al., 2019). Respirators purchased from international certification bodies may or may not be assigned a manufacturing lot number, and careful consideration should be made toward defining the population as the smallest suspected unit of consistent quality.

If quality information is desired for a single FFR model with a few lots, designing a study that allows for quality inferences to be made to the entire FFR model (i.e., the study population) could be easily accomplished by randomly sampling FFRs from each lot. There may be numerous contexts around the globe where the FFR lot numbers may not be able to be readily identified, though. In these instances, groups of FFRs can be delineated and categorized as the population for which a sample will be taken and inferences will be made. For example, in the absence of lot numbers, a single FFR model within a shipment may be the most prudent method to define population. Still, every effort should be made to assign the boundaries around a population that corresponds to the lowest level for which quality is expected to be homogenous.

When many FFR models and numerous lots are considered, as in large storage contexts or in stockpiles, the process of properly defining study populations can be quite challenging. Inventory management, therefore, is an essential part of the process of properly defining the study’s population and facilitating random sampling within lots.

The information that managers of stockpiles or inventory should strive to maintain includes, but is not limited to:

1. Manufacturer/approval holder

2. Model/part number

3. Lot number

4. Location of the lots within the storage facility

5. Type and condition of packaging (e.g. in a bag, box, crate, or pallet)

6. Quantity of product case, box, and respirators within the case⁸

7. Shelf life status (manufactured date and shelf life date [if applicable])

Additionally, when long-term storage is considered (e.g., stored in stockpile facilities), the storage conditions should be monitored and controlled to align with those conditions outlined by the manufacturer or approval holder. Storage information, such as temperature and percent relative humidity, for each facility should be maintained so that the quality of the respirators can be assessed according to the conditions in which they were stored.

Step 2: Sampling strategies

Once the population is defined, a sample of FFRs can be taken. When an FFR sample is taken from the population, a random sampling process should be used. Random sampling means that each unit within a lot has an equal probability of being selected for testing. Random sampling ensures that the results of testing the sample are an unbiased representation of the remainder of the population, and thus the estimates of quality derived from the sample are representative of the population that was defined. This means that, although the estimate from a single sample can be different from the true quality level of the population, the average of repeated samples should approach the true quality level that exists in the population that was defined.

Samples that are not randomly selected may bias the test results. For example, if only the top respirator is tested out of a selected package, conditions that affected only respirators at the bottom will not be represented. Likewise, if only a single box is sampled, then the specific conditions that the box experienced will be overrepresented. Truly independent sampling can be performed by assigning unique identifiers to each FFR, for example case, carton, and/or position, and then randomly selecting the identifiers using a computer program. A spreadsheet (e.g., Microsoft Excel) or a purpose-built program, many of which are freely available online, can readily perform this task.

A practical decision that needs to be made when implementing a plan for testing and evaluating the viability of respirators is the sample size that will be used. One way of simplifying the task is to seek a fixed sample size. Step 4: Evaluating the Test Results provides information on how to evaluate the results of a sample when sample sizes of 10, 18, 24, and 40 are considered.

When lot information is available, testing and evaluation may take the form of a Lot Quality Assurance Sampling (LQAS) technique. LQAS is a quality assurance process that may be most suitable when manufacturing lot information is available and FFRs are stored in large quantities, as in U.S. stockpiles. This technique originated in the manufacturing industry for quality control purposes and has been adapted to numerous quality assurance contexts since its inception (Dodge & Roming, 1959; Robertson & Valadez, 2006). LQAS is consistent with a stratified random sample of items to estimate the quality of the production lot instead of testing each item within the lot. LQAS is designed to facilitate decisions to accept or reject the entire lot based on the results of single or multiple samples, an approach that keeps sampling costs to a minimum (Hoshaw-Woodard, 2001; Lemeshow et al., 1990). Given that evaluating FFR quality may be applicable to numerous lots of distinct manufacturer models, the LQAS approach of randomly sampling small numbers of units within each lot is an economical approach (Yorio et al., 2019). When lot information is not provided, the LQAS method may be adapted to include taking samples from groups of 100, or some other number deemed appropriate given the number of FFRs available, and treating these groups as lots.

Figure 1 depicts a hypothetical stratified random sampling design for N95 FFRs. This technique divides the entire population of N95 FFRs into nonoverlapping subpopulations of lots that are expected to share quality commonalities.

Figure 1.

Respirator sampling depiction based on the LQAS technique.

[source: Adapted from Yorio et al., 2019]

The random sampling process integrates the principles of statistical inference, which allow one to generalize the results from sample data to the wider population from which the sample was drawn. Sampling done at the lowest level of consistent quality implies that the test results will generalize to that level; in other words, a specific inference can be made to a specific subpopulation. For example, in Figure 1, statistical inference from Sample 7 should be made to Manufacturer 2 Model 1 – Lot A subpopulation (or, for example, the group of FFRs delineated in the absence of manufacturing lots), not the entire population of respirators within the stockpile/shipment or even the entire population of Manufacturer 2 respirators within the stockpile/shipment.

Sampling done at the lowest level of consistent quality implies that the test results will generalize to this level; however, information at each level can be combined and inferences may be made at higher levels (Yorio et al., 2019). For example, the estimates of quality for all units that share a common manufacturer model can be made by combining the results of samples corresponding to particular subpopulations (Yorio et al., 2019). In order to make higher level inferences, testing multiple subpopulations is recommended to investigate for between sample variability.

Step 3: Inspecting and Testing the Sample

After sampling is done, the units can be inspected and tested for quality. Filtering facepiece respirators protect the user by filtering potentially contaminated air, requiring the filtering media to be effective at capturing particulates while fitting snugly to create an effective seal with the user’s face (D’Alessandro & Cichowicz, 2020). The FFR’s unique components (including the filtering media, straps, and nose clip) may degrade over time, such as by environmental and/or storage conditions (Viscusi et al., 2009). Changes to any of these components may affect the filtration performance or fit of the respirators, potentially offering less protection to the end user. Quality assessment metrics include visual inspection and performance tests using widely accepted test methods for the sampled respirators.

Visual Inspection of FFRs

An example visual inspection checklist for individual FFRs is provided in Table I. Damage to the straps, nose clip, nose foam (if present), or general deformation of the facepiece may affect fit, whereas general deformation and the presence of dust or mold may affect the filtration performance (CDC, 2020e; 3M, 2020). Note that this checklist is for demonstration purposes only; it should be tailored to the requirements of the entity evaluating the respirators.

Table I.

Example Respirator Visual Inspection Checklist

Visual Inspection Items	Evaluation
1. Does the packaging have any signs of damage?	□ YES □ NO □ NA
2. Does the respirator have an odor?	□ YES □ NO □ NA
3. Does the respirator have a presence of dust?	□ YES □ NO □ NA
4. Is the respirator deformed in any way?	□ YES □ NO □ NA
5. Is the nose clip cracked, corroded, or detached?	□ YES □ NO □ NA
6. Is the nose foam flaking, or does it appear damaged?	□ YES □ NO □ NA
7. Does the respirator appear moldy?	□ YES □ NO □ NA
8. If applicable, are the staples attaching the straps cracked, corroded, or rusty?	□ YES □ NO □ NA
9. Are the straps detached from the respirator?	□ YES □ NO □ NA
10. Are the straps damaged in any way?	□ YES □ NO □ NA

Standard Testing Procedures (STPs) Relevant to N95 FFRs

NIOSH applies test conditions that represent near “worst case” scenarios when testing FFRs submitted for approval (Krah & Shaffer, 2016). Outside of the NIOSH Respirator Approval Program, N95 FFRs may be tested in accordance with NIOSH performance requirements defined by regulations and standards. The following links provide standard testing procedures (STPs) used by NIOSH to evaluate the performance of N95 FFRs:

1. TEB-APR-STP-0003 (Determination of Exhalation Resistance)

   https://www.cdc.gov/niosh/npptl/stps/pdfs/TEB-APR-STP-0003-508.pdf

2. TEB-APR-STP-0004 (Determination of Exhalation Valve Leakage Test)⁹

   https://www.cdc.gov/niosh/npptl/stps/pdfs/TEB-APR-STP-0004-508.pdf

3. TEB-APR-STP-0007 (Determination of Inhalation Resistance)

   https://www.cdc.gov/niosh/npptl/stps/pdfs/TEB-APR-STP-0007-508.pdf

4. TEB-APR-STP-0059 (Determination of Particulate Filter Efficiency Level for N95 Series Filters Against Solid Particulates for Non-Powered, Air-Purifying Respirators)

   https://www.cdc.gov/niosh/npptl/stps/pdfs/TEB-APR-STP-0059–508.pdf

STPs for FFRs with other levels of protection (i.e., N99, N100, R95, R99, R100, P95, P99, P100) can be found at https://www.cdc.gov/niosh/npptl/stps/apresp.html.

The performance tests presented above are part of the evaluation process that NIOSH employs to approve FFRs. Within the scope of providing a general framework for entities to test and evaluate their own products, and considering limited sample sizes, STP-0059 (or equivalent STPs for other levels of protections) is an STP that should be considered the primary performance test to evaluate the viability of the FFRs being tested. This STP provides the minimum performance requirements for percent particle penetration—for N95 FFRs, particulate penetration is required to be at or below 5.0% as measured using STP-0059 (NIOSH, 2019d). If the FFRs being tested have exhalation valves, STP-0004 may be considered to check the viability of the valve. Additionally, it should be noted that, for purposes outside of NIOSH’s Respirator Approval Program, a modified sample size may be used instead of the sample sizes outlined in NIOSH’s STPs to assess the viability of the FFRs. For example, while STP-0059 requires 20 FFRs to be tested, a sample size of 10 FFRs within one production lot within one model may be tested for particulate penetration efficiency in order to leverage the LQAS sampling plan. Tests STP-0003 and STP0007 assess the difficulty of drawing breath through the respirator. Unlike filter penetration, changes in breathing resistance do not directly affect the protection that is expected to be provided by the FFR and may better be detected by wearers.

Step 4: Evaluating the Test Results

After a sample of FFRs is tested, the results can be evaluated. Visual inspections will result in a pass/fail designation assigned to each unit, which can then be aggregated (i.e., how many units in the sample failed the visual inspection). Other variables such as the percent particle penetration from the filtration test can be analyzed in its continuous variable form or converted to a pass/fail attribute. The cutoff which differentiates between a passed/failed filtration test may be chosen based on NIOSH’s filtration performance requirements for N95 respirators (i.e., <5.0% particle penetration is a pass).

When filtration performance metrics are evaluated in their continuous variable form, the average level of filtration for the sample along with a 95% or 99% confidence interval may be generated. These statistics may then be evaluated in relation to the cutoffs that differentiate passing and failing FFRs in the NIOSH STPs. For example, the performance of the population may be deemed adequate if the upper bound of the 95% or 99% confidence interval for particle penetration is below the 5% line used to evaluate N95 respirators.

When attributes such as pass/fail are considered, the number of units in the sample that failed the test(s) can be added and a fail rate computed. For example, if a random sample of 10 units were taken from a lot of decontaminated FFRs and visually inspected and 4 failed the visual inspection, the fail rate would be 40% (4/10). As an additional example, if a random sample of 18 FFRs were tested for filtration and 5 failed the filtration test, the fail rate would be 27.78% (5/18). The quality of the population (i.e., the entire lot being studied) can then be evaluated based on the number of failures observed within the random sample. This is done by determining the probability that the observed number of failures came from a lot with a chosen, desired level of quality. The probability of obtaining any number of failures in a sample of any size can be statistically computed by using the probability density function for the binomial distribution. Although not exhaustive of all possibilities, Table II shows the probabilities that a sample of 10, 18, 24, and 40 will result in a specific number of failures when the true number of defective units in the lot is 10%, 5%, and 2.5%.

Table II.

Probability that a Sample of n FFRs Will Result in a Specific Number of Failures when the True Number of Defective Units in the Lot is 10%, 5%, and 2.5%

Sample Size	Number of Failures	Probability (in %) for a lot with 10% Defective Units	Probability (in %) for a lot with 5% Defective Units	Probability (in %) for a lot with 2.5% Defective Units
10	0	34.9	59.9	77.6
	1	38.7	31.5	19.9
	2	19.4	7.5	2.3
	3	5.7	1.0	-
	4	1.1	0.1	-
	5	0.1	-	-
	6	-	-	-
18	0	15.0	39.7	63.4
	1	30.0	37.6	29.3
	2	28.4	16.8	6.4
	3	16.8	4.7	0.9
	4	7.0	0.9	0.1
	5	2.2	-	-
	6	0.5	-	-
	7	0.1	-	-
	8	-	-	-
	9	-	-	-
24	0	8.0	29.2	54.5
	1	21.3	36.9	33.5
	2	27.2	22.3	9.9
	3	22.1	8.6	1.9
	4	12.9	2.4	0.3
	D	5.7	0.5	-
	6	2.0	0.1	-
	7	0.6	-	-
	8	0.1	-	-
	9	-	-	-
	10	-	-	-
40	0	1.5	12.9	36.3
	1	6.6	27.1	37.3
	2	14.2	27.8	18.6
	3	20.0	18.5	6.0
	4	20.6	9.0	1.4
	5	16.5	3.4	0.3
	6	10.7	1.0	-
	7	5.8	0.3	-
	8	2.6	0.1	-
	9	1.0	-	-
	10	0.4	-	-
	11	0.1	-	-
	12	-	-	-

[source: based on calculations from Yorio et al. (2019)]

Table II is colored white for instances in which the probability of obtaining that number of failures in a given sample size or a given desired quality level is greater than 10%. Table II is shaded gray when the probabilities lie between approximately 1% and 10%. Table II is colored black when the probabilities are less than 1%. This 3-tier colored tool suggests that results within a white cell may be deemed acceptable according to the previously specified desired lot quality level; results within a gray cell may require additional testing and evaluation; and results within a black cell suggest that there is a high probability that the quality level for a given lot is less than the selected level of quality identified. The breakdown of numbers used for decision making in Table II is only proposed as a suggestion. Those who are responsible for assessing the viability of FFRs may consider alternative decision-making criteria to suit their organization’s needs.

For illustration purposes, consider the situation that 3 out of a sample of 18 FFRs failed a performance test. Table II can then be used to evaluate if the quality level of the entire population had fewer than 5% poor performing FFRs. Table II suggests a sample of 18 respirators from a lot in which 5% of the units are bad will result in 0, 1, or 2 failures a little over 94% of the time (i.e., the sum of the probabilities of observing exactly 0, 1, and 2 failures). The likelihood of observing 3 failures out of 18 respirator items is relatively small—only 4.7% of the time. Using this information, one may conclude that there is a low likelihood that the sample of 18 units came from a population in which there were 5% defective units or less and, therefore, it would be reasonable to conclude that the number of defects in the population exceeds 5%. This cell within Table II is colored gray to indicate that another random sample of units within that population may be beneficial to further assess the quality of the lot. For further discussions, see Yorio et al. (2019). Table II represents a simple tool that can be used to determine when the proportion of defects in a population is likely to have exceeded the maximum number that decision makers are willing to accept.

AN EXAMPLE: NIOSH’s RESEARCH STUDY for TESTING and EVALUATING STOCKPILED RESPIRATORS

Prior to NIOSH’s research study, there were limited published data to understand the viability of respirators that have undergone long-term storage under various storage conditions. Given the limited information and the interest by stockpile managers and policy makers, NIOSH obtained samples of air-purifying respirators (including FFRs) from geographically dispersed stockpiles with varied storage conditions. A portion of this dataset is used as an example here to apply the LQAS sampling strategy; the study summary and findings related to testing and evaluating stockpiled respirators are also described below.

Study Methods

NIOSH researchers visited ten stockpile facilities to sample respirators that were stored since 2003 (Greenawald et al., 2020). This included eleven N95 FFR models and one P95 particulate filter model to be used with an elastomeric half mask respirator. A total of 93 production lots (i.e., populations) were sampled, where 43 respirators were sampled from each production lot—40 respirators were tested for particulate filtration performance and 3 respirators were tested for inhalation and exhalation resistance¹⁰. Two production lots were sampled from each manufacturing year, where possible. These individual lots were sampled within each facility where disparate conditions were observed, where possible. For example, one case/production lot was sampled at the top of the metal shelving racks where lights and fans were in close proximity to the respirator pallet, and another case/production lot was sampled from the bottom rack. For filtration performance, ten respirator units were sampled from four different boxes dispersed within the product case (e.g., one box from each side of the case). For inhalation/exhalation resistance, one respirator unit was sampled from three of these four boxes.

At each facility, NIOSH used checklists to document site and packaging (i.e., pallet, case, and box) conditions that may impact respirator performance, such as: 1) the PPE packaging, including the presence of dust, chemicals, and moisture; 2) exposure to sunlight and other strong light; 3) proximity to fans, windows, doors, and ventilation systems; 4) damage to pallet shrink wrap and product packaging; and 5) location of pallet on storage shelving rack (e.g., top, bottom), and location of PPE product on pallet (e.g., top/not load-bearing, bottom/load-bearing). Furthermore, NIOSH collected facility temperature and relative humidity (%RH) data from either the facility-generated data provided by the stockpile personnel or, where monitoring did not exist, by placing at least one data logger in the facility.¹¹ A total of 3,971 respirators were tested. For each respirator tested, a visual inspection was conducted using similar criteria as shown in Table I. Respirators were tested for inhalation and exhalation resistance and filtration efficiency performance in accordance with the NIOSH STPs. No respirators identified or sampled had exhalation valves.

Summary of Findings

Overall, less than 2% of the total stockpiled respirators that were visually inspected had visual inspection concerns, which included some respirators’ nose foam sticking to adjacent respirators within the box, crushing, appearance of mold, and damage to straps. However, no trends were observed between respirators with visual inspection concerns and performance—i.e., all 66 respirator units associated with visual inspection concerns met NIOSH’s minimum performance requirements. It should be noted that fit of these respirators, which may be affected by those visual inspection concerns identified, was not assessed in these reports. Moreover, these concerns would likely lead to discarding these identified respirators.

Furthermore, NIOSH evaluated the inhalation resistance (NIOSH STP-0007) and exhalation resistance (NIOSH STP-0003) for a total of 276 stockpiled and 36 control respirators. All stockpiled and control respirators from each model passed these tests. NIOSH also evaluated the filtration efficiency (STP-0053 for P95 filters or STP-0059 for N95 FFRs) for 3,695 stockpiled respirators and 240 control respirators (i.e., 20 for each of the 12 respirator models). Seventy-six N95 FFRs exceeded the filter penetration allowed under the NIOSH test procedure STP-0059, accounting for 2% of the total respirators tested—i.e., 98% of the total stockpiled respirators passed NIOSH performance requirements. These 76 failing respirators came from three different stockpiles and three different N95 FFR models.¹²

The summary of NIOSH’s findings on stockpiled respirators’ inhalation and exhalation resistance and filtration performance in each facility is summarized in Appendix A.¹³ Note that the results may not be applicable to other stockpiles, models, and/or under different storage conditions. Since the inhalation and exhalation resistance and filtration effectiveness may differ by models, production lots, and stockpile facilities, it is highly recommended to get facility-specific data on stockpiled respirators and evaluate their quality.

Applying the LQAS to NIOSH’s Research Study Related to Stockpiled Respirators

The findings from NIOSH’s study were compared to the LQAS sampling plan in Table II. No failures were identified for exhalation or inhalation resistance (STPs-0003 and −0007, respectively), therefore the comparison is only for units that failed solid particulate filtration efficiency (STP-0059). Table III shows a summary of the failing units from three different stockpiles.

Table III.

NIOSH’s Identified Filtration Performance Failures

Stockpile Facility	Failing Respirator Models (manufacturer [mfr.] year)	Number of Failing Units (Filtration Performance Only14)	Total Number of Respirators Tested from this Lot for Filtration Performance
#3	3M 1860 Lot A (mfr. 2006)	1	40
#3	3M 1860 Lot B (mfr. 2009)	1	40
#4	Kimberly Clark (KC) 46827 Lot A (mfr. 2007)	9	40
#4	KC 46827 Lot B (mfr. 2007)	25	40
#7	KC 46727 Lot A (mfr. 2007)	40	40

These data can be used as an example of how Table II may be used to assess the quality of the population in lieu of the performance results derived from the sample.

• For the 3M 1860 models tested from Stockpile 3, 40 respirators were sampled and 1 failure was identified in each lot. The probability of one failure in a lot with 10%, 5%, and 2.5% defective units is 6.6%, 27.1%, and 37.3%, respectively. This finding suggests most of the population of FFRs from which the sample of 40 was tested would also be expected to perform well.

• For the KC 46827 model from Stockpile 4, 40 respirators were sampled from Lot A and 9 failures were identified. The probability of detecting 9 failures in a population with 10% defective units is 1.0%; approximately 0% for a population with 5% defective units; and also 0% for a population with 2.5% defective units. This finding suggests that it is likely that the population from which the sample of 40 was tested has more than 10% poor performing units. The same conclusion would be drawn for the second lot of KC 46827 FFRs (Lot B) in which 25 out of 40 FFRs failed.

• For the KC 46727 model from Stockpile 7, all 40 of the sample of FFRs failed. In this instance, Table II suggests that the probability of the population having less than 10% of the units perform well is approximately 0%.

The findings presented coincide with CDC/NIOSH’s decision that, based on the empirical data collected, the KC 46827 and KC 46727 models that were past their designated shelf life may not provide the expected level of protection to the wearer and were recommended to not be released from the stockpiles (CDC, 2020b). It is important to note that this manufacturer currently has a five-year shelf life for these models and specifies that units past their designated shelf life should be disposed.

CONCLUSIONS AND ADDITIONAL RESOURCES

During a public health emergency, FFR shortages may occur. In response, unique strategies may be widely utilized to optimize the supply of FFRs where knowledge of the performance viability is limited – e.g., FFRs that have undergone long-term storage, FFRs that have been decontaminated, and imported FFRs. A general approach to evaluating FFRs that are not part of the NIOSH Respirator Approval Program is provided. It includes defining the FFR population to be sampled, sampling the FFRs based on desired lot quality, inspecting and testing the FFRs using well-established performance standards, and interpreting the results. The sampling approach is applicable to understanding the viability of FFRs from varying population sizes, and, the simple sampling strategies include a fixed sample size regardless of the population size. An example of applying the LQAS sampling strategy is also provided, using data from a recent NIOSH study.

Entities that have the proper test equipment can perform the respirator testing themselves to assess the respirators against NIOSH’s minimum performance requirements; the equipment used can be found in the STPs in the previously provided links. Additionally, a third-party laboratory may be used.

To further learn about the approval status of NIOSH-approved respirators, product information (such as storage conditions and shelf life information) should be requested from the manufacturer/approval holder. Additionally, the NIOSH Certified Equipment List (CEL) provides a comprehensive list of the current approval status of all NIOSH-approved respirators (NIOSH, 2016), and those approvals specific to FFRs can be found here: https://www.cdc.gov/niosh/npptl/topics/respirators/disp_part/respsource.html

Lastly, NIOSH’s Listserv provides email notifications relevant to respirators. The listserv can be found here: https://www.cdc.gov/niosh/npptl/sub-NPPTL.html.

Disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention (CDC). Product and company names are provided for identification purposes only and do not imply endorsement by the CDC.

Appendix A. Summary of NIOSH Research Study Related to Stockpiled Respirators

Summary of Inhalation and Exhalation Resistance and Filtration Performance Results for Stockpiled Respirators from Ten Facilities

Facility	Results
	Inhalation and Exhalation Resistance	Participate Filtration Performance
Facility 1, DHHS region 2	All 24 stockpiled respirators from 3 different models passed these tests	All 320 stockpiled respirators from 3 different models passed this test
Facility 2, DHHS region 2	All 21 stockpiled respirators from 2 different models passed these tests	All 280 stockpiled respirators from 2 different models passed this test
Facility 3, DHHS region 4	All 18 stockpiled respirators from 2 different models passed these tests	Out of 240 stockpiled respirators, 2 individual respirators tested (same model, 2 different lots) exceeded the 5.0% maximum allowable penetration. A statistically significant difference was observed between controls (i.e., new respirators of the same model) and each of these two stockpiled lots
Facility 4, DHHS region 9	All 39 stockpiled respirators from 6 different models passed these tests	Out of 535 stockpiled respirators, 34 individual respirators (same model, 2 different lots) exceeded the 5.0% maximum allowable penetration. A statistically significant difference was observed between controls and stockpiled respirators for this lot
Facility 5, DHHS region 9	All 42 stockpiled respirators from 6 different models passed these tests	All 560 stockpiled respirators from 6 different models passed this test
Facility 6, DHHS region 10	All 27 stockpiled respirators from 4 different models passed these tests	All 360 stockpiled respirators from 4 different models passed this test
Facility 7, DHHS region 1	All 39 stockpiled respirators from 6 different models passed these tests	Out of 520 stockpiled respirators, 40 individual respirators (same model, same lot) exceeded the 5.0% maximum allowable penetration. A statistically significant difference between the controls and stockpiled respirators was observed for this lot with the 40 failing units (Table III, Stockpile 7)
Facility 8, DHHS region 8	All 24 stockpiled respirators from 5 different models passed these tests	All 320 stockpiled respirators from 5 different models passed this test
Facility 9, DHHS region 8	All 12 stockpiled respirators from 2 different models passed these tests	All 160 stockpiled respirators from 2 different models passed this test
Facility 10, DHHS region 6	All 30 stockpiled respirators from 5 different models passed these tests	All 400 stockpiled respirators from 5 different models passed this test

[source: Greenawald et al. (2020)]