Masking strategy to protect healthcare workers from COVID-19: An umbrella meta-analysis

Abstract

Background

The burden of severe disease and death due to SARS-CoV-2 (COVID-19) pandemic among healthcare workers (HCWs) worldwide has been substantial. Masking is a critical control measure to effectively protect HCWs from respiratory infectious diseases, yet for COVID-19, masking policies have varied considerably across jurisdictions. As Omicron variants began to be predominant, the value of switching from a permissive approach based on a point of care risk assessment (PCRA) to a rigid masking policy needed to be assessed.

Methods

A literature search was conducted in MEDLINE (Ovid platform), Cochrane Library, Web of Science (Ovid platform), and PubMed to June 2022. An umbrella review of meta-analyses investigating protective effects of N95 or equivalent respirators and medical masks was then conducted. Data extraction, evidence synthesis and appraisal were duplicated.

Results

While the results of Forest plots slightly favoured N95 or equivalent respirators over medical masks, eight of the ten meta-analyses included in the umbrella review were appraised as having very low certainty and the other two as having low certainty.

Conclusion

The literature appraisal, in conjunction with risk assessment of the Omicron variant, side-effects and acceptability to HCWs, along with the precautionary principle, supported maintaining the current policy guided by PCRA rather than adopting a more rigid approach. Well-designed prospective multi-centre trials, with systematic attention to the diversity of healthcare settings, risk levels and equity concerns are needed to support future masking policies.

Introduction

Respiratory infectious diseases pose occupational risks to healthcare workers (HCWs) who are not adequately protected, adding to the risks HCWs incur from household and community exposures. The SARS-CoV-2 (COVID-19) pandemic has emphasized the need to protect HCWs, both because of their crucial role in taking care of those who fall ill, as well as to respect the right of all workers to a safe working environment. As of May 2021, 6633 HCW deaths were reported to World Health Organization (WHO), and an extrapolation of mortality rates and infection rates estimated the number of HCW deaths worldwide as between 80,000 and 180,000 [1]. In addition to droplet transmission, mounting evidence supporting the potential for airborne transmission of the SARS-CoV-2 virus [2,3] led to calls to equip all front-line HCWs with N95 or equivalent respirators (e.g. FFP2, FFP3) to achieve a higher protective effect than afforded by medical masks [3,4]. However, much controversy exists over implementing such a policy. Nonetheless, as the pandemic has continued, guidelines regarding how and when to use N95 respirators thus merited review.

Current masking policies in many jurisdictions worldwide are based on whether patient care activities involve aerosol generating medical procedures (AGMPs) – activities carried out on a patient that can produce aerosols and droplet nuclei, which, if they contain viable virus, could increase the possibility of airborne transmission of viruses that generally are not airborne. Examples of AGMPs are autopsies of respiratory tissues, some types of intubation and extubation procedures, open airway suctioning, etc. [5] This contrasts with activities that involve direct contact with patients or entering patient care areas but which do not involve AGMPs. The distinction between aerosolizing processes and direct patient care, however, is not clear-cut. For example, rapid-sequence intubation does not result in aerosolization, and, conversely, a coughing patient may indeed produce substantial aerosols [6]. In addition, patients are grouped into COVID-19 not-suspected, suspected or confirmed, with masking guidance falling into either continuous or targeted use based on a patient's presumed COVID-19 status.

In British Columbia (BC), Canada, the BC Centre for Disease Control (BCCDC), has recommended that HCWs continuously wear medical masks when in patient care areas since 2020 [7]. Respirators are recommended for AGMPs on patients with suspected respiratory infection, based on a point-of-care risk assessment (PCRA) by the HCW, for suspect/confirmed airborne pathogens (e.g. COVID-19, tuberculosis, measles, etc.) [8]. On November 4, 2021, all HCWs were mandated to wear a medical mask in all patient care settings at all times regardless of their immunization status. The communique also stated that a HCWs should access a respirator when they determine there to be an elevated risk of COVID-19 aerosolization or other pathogen exposure based on the PCRA [9]. This message had previously been emphasized for other respiratory infectious agents, for example during the 2009 influenza pandemic [10]. The BCCDC's COVID-19 PCRA [11] states that a respirator is recommended in the following scenarios: “1) risk of airborne transmission from an airborne infectious agent (e.g., pulmonary tuberculosis); 2) risk of airborne transmission from a procedure (e.g., aerosol generating medical procedures on a patient with suspected or confirmed COVID-19); and 3) based on organizational guidance as determined by an organizational risk assessment (e.g., in areas with poor or unknown ventilation).” It noted that a respirator will be provided if “there is an elevated risk of COVID-19 transmission, as determined by the PCRA”. Masking policies from international and Canadian jurisdictions are listed in Appendix A.

Regardless of the selection strategy, all medical masks and respirators must be certified to comply with international and regional standards. In addition, HCWs should be trained to use a medical mask and a respirator correctly, including donning (i.e. putting on) and doffing (i.e. taking off) procedures, and when to change facial personal protective equipment (PPE). HCWs must be fit-tested for the most suitable respirator before the first use and should perform a defect and a seal check each time of use.

The prevalent Omicron variant is characterized by substantially higher transmissibility than the original SARS-Cov-2 virus, which may be due to increased binding affinity between the mutated spike protein of the virus and the angiotensin converting enzyme 2 (ACE2 “receptor”) on the cell surface [12]. In addition, higher asymptomatic infection rate and immune escape likely contribute to widespread transmission of this variant [13,14]. Consequently, there is a strong rationale for assessing whether a stricter masking policy for HCWs is needed. Specifically, we are interested primarily in whether there is evidence to support a policy of wearing respirators (N95 or equivalent or higher level) instead of medical masks when providing patient care; and secondarily, whether wearing such respiratory protection universally and continuously in the patient care environment should be adopted in light of new variants.

To address these research questions, the scientific evidence underpinning masking policies was assessed by conducting an umbrella review of meta-analyses. In our discussion of the results of our analysis and their implications for decision making, we are guided by the precautionary principle and published research literature about the risk (likelihood and severity of outcome) associated with Omicron variants compared to previous variants, in addition to side-effects, healthcare practical concerns and wider issues including global considerations.

Methods

A systematic literature review was conducted to identify evidence syntheses of protective effects of N95 or equivalent respirators and medical masks and an umbrella review was then conducted to appraise the evidence synthesized in these meta-analyses.

Design

As the use of medical masks for suspected/confirmed COVID-19 patients is the minimum requirement across all scanned jurisdictions (see Appendix A), the two interventions evaluated in the Population, Intervention, Comparison, Outcome (PICO) framework [15] are: 1) N95 respirator or equivalent or higher level vs. medical masks for routine patient care, and 2) continuous masking vs. targeted masking (i.e. wearing respiratory PPE regardless of the patients’ COVID-19 status, versus only wearing respiratory PPE when interacting with COVID-19 suspected or confirmed patients). In addition, given that most of the meta-analyses were based on studies conducted before the COVID-19 pandemic, we expanded the diseases of interest in the PICO to include both respiratory viral infections (laboratory confirmed) and clinical respiratory illness. All studies searched were limited to meta-analyses. The PICO framework is presented in Table 1 .

Table 1.

The PICO framework of the research question.

Population	Healthcare workers (HCWs)
Intervention vs. Comparison	1) Wearing N95 respirator or equivalent or higher level vs. wearing medical masks
	2) Continuous masking vs. targeted masking
Outcome	Infected with respiratory viral diseases or clinical respiratory illness

Literature sources and search strategy

Four health science databases, namely MEDLINE (Ovid platform), Cochrane Library, Web of Science (Ovid platform), and PubMed, were selected for literature searching (as of June 2022). The exact keywords, including SARS-CoV-2, COVID-19, mask, masks, respirator, respirators, healthcare workers, health care workers, HCWs, healthcare personnel, healthcare professionals, healthcare providers, were used for all databases. The search strategy for MEDLINE and Cochrane library databases used keywords and MESH terms “exploded” search, and the search strategy for Web of Science and PubMed databases used advanced search for keywords in All Fields. All the searches were limited to articles with a meta-analysis. No language restriction was applied in searches.

Selection criteria

The articles selected met the following criteria: 1) A meta-analysis article; and 2) Relevant to all four components of the PICO framework.

Literature appraisal tools

The certainty of each selected meta-analysis was assessed by a methodological framework adopted from the Assessment of Multiple Systematic Reviews.

(AMSTAR) 2 tool (a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both) [16] and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group [17].

The GRADE approach rates the certainty of evidence into four levels (high, moderate, low, and very low). Initially, a meta-analysis was assigned with a high-certainty evidence level, given that it usually has higher evidence quality compared to a single study (randomized control trial or observational study). Six components, including study design, risk of bias (RoB), inconsistency, indirectness, imprecision and publication bias, were evaluated to maintain or downgrade the evidence certainty level. The evidence could be downgraded by each component up to two levels and by all components to very low quality.

The study design was assessed by selecting questions in the AMSTAR 2 tool's checklist [16], covering PICO, methodology, and disclosure. RoB was rated upon the RoB assessment in each meta-analysis. Inconsistency was assessed based on the individual paper's discussion of heterogenicity. Indirectness was considered according to the relevance between the meta-analysis and the project objectives. Imprecision was evaluated on confidence intervals (CIs) of outcomes in the meta-analysis. The discussion of publication bias in each meta-analysis was reviewed.

Criteria of evidence grading

The study design was rated based on the following eight questions intended to specifically assess study design (quoted from the AMSTAR 2 tool's checklist).

• 1.Did the research questions and inclusion criteria for the review include the components of PICO?
• 2.Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review, and did the report justify any significant deviations from the protocol?
• 3.Did the review authors explain their selection of the study designs for inclusion in the review?
• 4.Did the review authors use a comprehensive literature search strategy?
• 5.
  Did the review authors perform study selection in duplicate?

  • •
    at least two reviewers independently agreed on the selection of eligible studies and achieved consensus on which studies to include
  • •
    OR two reviewers selected a sample of eligible studies and achieved good agreement (at least 80 percent), with the remainder selected by one reviewer.
• 6.
  Did the review authors perform data extraction in duplicate?

  • •
    at least two reviewers achieved consensus on which data to extract from included studies
  • •
    OR two reviewers extracted data from a sample of eligible studies and achieved good agreement (at least 80 percent), with the remainder extracted by one reviewer.
• 7.Did the review authors provide a list of excluded studies and justify the exclusions?
• 8.Did the review authors describe the included studies in adequate detail?

The rating and downgrading criteria of all six components of the appraisal approach are listed in Table 2 .

Table 2.

The rating and downgrading criteria of the appraisal approach components.

Component	Levels	Rating criteria	Downgrading criteria
Study design	serious/not serious	Rate “serious” if any of the eight questions is answered “NO”.	Downgrade one level of evidence certainty if rated “serious”.
Risk of Bias	serious/not serious	Rate “serious” if more than half of the original studies had “high” or “some” risk of bias.	Downgrade one level of evidence certainty if rated “serious”.
Inconsistency	serious/not serious	Rate “serious” if more than half of the relevant results had high heterogeneity.	Downgrade one level of evidence certainty if rated “serious”.
Indirectness	very serious/serious/not serious	Rate “serious” if any one of the following descriptions were not met, and rate “very serious” if more than two descriptions were not met. 1.The outcome is COVID-19 infection. 2.The original studies were conducted in healthcare settings. 3.Comparison is N95 respirator vs. medical mask.	Downgrade one level of evidence certainty if rated “serious”, and two levels if rated “very serious”.
Imprecision	serious/not serious	Rate “serious” if more than half of the relevant results had confidence intervals across 1.	Downgrade one level of evidence certainty if rated “serious”.
Publication bias	serious/not serious	Rate “serious” if the authors mentioned concerns about publication bias.	Downgrade one level of evidence certainty if rated “serious”.

Data extraction and synthesis

Studies identified were screened by two reviewers independently based on the titles, abstracts, and full texts. Then, two reviewers independently appraised the selected meta-analyses. Any disagreements were resolved by a panel discussion with other reviewers.

Data analysis

Forest plot analysis was performed by statistical software R version 4.2.2 with the package forestplot version 3.1.1. The studies were pooled using a random-effect model to combine the odds ratios (OR) and CIs. Subgroup analysis was used to explore possible effects within different study designs.

Results

Literature selection

Database searches identified a total of 553 records by June 2022. After removing duplicates (194), the remaining 359 articles were screened for relevance through title and abstract. Irrelevant articles (e.g., studies that exclusively focused on downsides of masking or people's attitude towards N95 respirators, etc.) were excluded from full-text review. Among 41 articles screened for full-text evaluation, those that did not meet the criteria of the PICO statement were excluded (e.g., studies that only assessed the protective effect of face masks without specifying types compared to no mask; studies only for on general public or non-HCWs, etc.). As reported in the diagram (Fig. 1 ), 10 meta-analyses were included for the umbrella review.

Fig. 1.

Diagram illustrating the flow of studies through review selection.

Evidence synthesis

Out of ten included meta-analyses, four were based solely on randomized controlled trials (RCTs) [[18], [19], [20], [21]], two exclusively consisted of observational studies [22,23], and four were a mix of RCTs and observational studies [[24], [25], [26], [27]]. The number of original studies related to the PICO framework in each meta-analysis ranged from 4 to 32 (mean = 14, median = 9).

One meta-analysis directly compared the protective effect from SARS-CoV-2 infection between N95 respirators (and equivalents) and surgical masks, returning an nonsignificant result [27]. Four meta-analyses conducted the same comparison but on preventing respiratory viral infections (RVIs) and clinical respiratory illnesses. Among these, two favoured N95 respirators over medical/surgical masks in HCWs [25,26] with the 95% CI not including the null; meta-estimates from the other two were not as precisely estimated as 95% CIs included the null value [18,19]. Three meta-analyses compared the protective effect of respiratory PPE (including respirators and medical/surgical masks) with no mask. Among them, one indicated that although masking provided protection from contracting SARS-CoV-2 compared to not wearing a mask, N95 respirators did not offer a better protective effect at a 95% level of confidence [23]. Finally, using a distinct study design, the network meta-analysis conducted by Tran et al. [20] showed that wearing N95 respirators offered a protective effect on respiratory infections compared with medical masks at a 95% level of confidence.

In the other network meta-analysis, which is the only review to answer the research question of wearing manners, Yin et al. [21] showed that continuous wearing N95 respirators is the most effective combination of respiratory PPE (N95 respirator, surgical mask, or cloth mask) and wearing manner (continuous or targeted) to protect HCWs from laboratory-confirmed RVIs by using the surface under the cumulative ranking curve analysis (SUCRA). However, compared with targeted wearing, continuous wearing the same respiratory PPE did not provide significant protection from laboratory-confirmed RVIs at a 95% level of confidence, although continuous wearing had a favorable, but not statistically significant, protective effectiveness compared with targeted wearing in terms of N95 respirators (network odds ratio = 0.62, 95% credibility interval = 0.30–1.50). Characteristics of included studies are summarized in Table 3 .

Table 3.

characteristics of included meta-analyses.

#	Reference	n	Original Study Type	Relevant Objective	Population and Settings	Outcome	Relevant Intervention and Comparison	Relevant Results
1	Bartoszko et al., 2020 [18]	4	RCTs	Review of studies examining the protective effect between N95 respirators and medical masks for preventing HCWs from laboratory-confirmed RVIs and clinical respiratory illnesses	HCWs; health-care settings	Laboratory-confirmed RVIs and clinical respiratory illnesses	N95 respirators vs. Medical masks	Compared with N95 respirators, medical masks were not associated with an odds of increased laboratory-confirmed RVIs (adjusted OR = 1.06, 95% CI = 0.90–1.25) and clinical respiratory illnesses (adjusted OR = 1.49, 95% CI = 0.98–2.28)
2	Chen et al., 2022 [24]	31	Observational studies and RCTs	Review of studies examining the protective effect of masks for preventing RVIs	HCWs and non-HCWs	laboratory-confirmed RVIs	rPPE (all types) vs. no masks	N95 respirators were effective in preventing RVIs (Case-Control studies: OR = 0.27, 95% CI = 0.14–0.54, Cohort Studies: RR = 0.30, 95% CI = 0.16–0.58); Surgical masks were effective in preventing RVIs (Cohort Studies: RR = 0.05, 95% CI = 0.00–0.97, RCTs: RR = 0.65, 95% CI: 0.48–0.89).
3	Chu et al., 2020 [22]	10	Observational studies	Review of studies examining the protective effect of N95 respirators (or similar) and surgical masks (or similar, e.g., 12-16-layer cotton masks) for preventing RVIs	HCWs and non-HCWs; health-care settings and non-health-care settings	RVIs	Surgical masks or similar (e.g., 12-16-layer cotton masks) vs. no mask; Health-care settings: N95 respirators or similar vs. no mask	Surgical masks or similar associated with a significant reduction in RVIs odds compared with no masks (adjusted OR = 0.33, 95% CI = 0.17–0.61); N95 respirators or similar associated with a significant reduction in RVIs odds compared with no mask (adjusted OR = 0.04, 95% CI = 0.004–0.30)
4	Collins et al., 2021 [25]	8	Observational studies and RCTs	Review of studies examining the protective effect between N95 respirators and surgical masks for preventing HCWs from RVIs	HCWs; health-care settings	RVIs	N95 respirators vs. Surgical masks	N95 respirators associated with significant decreased RVIs risk compared with surgical masks (RVIs (RR = 0.73, 95% CI = 0.65–0.82), SARS-CoV 1 and 2 virus infection (RR = 0.17, 95% CI = 0.06–0.49), and laboratory-confirmed RVIs (RR = 0.75, 95% CI = 0.66–0.84))
5	Jefferson et al., 2020 [19]	4	RCTs	Review of studies examining the protective effect between N95 respirators and medical masks for preventing HCWs from RVIs and clinical respiratory illnesses	HCWs; health-care settings	RVIs and clinical respiratory illness	N95 respirators vs. Medical masks	N95 respirators did not associate with significant decreased RVIs and clinical respiratory illnesses risks compared with medical masks (clinical respiratory illness (RR = 0.70, 95% CI = 0.45–1.10), influenza-like illness (RR = 0.81, 95% CI = 0.59–1.11), and laboratory-confirmed influenza (RR = 1.05, 95% CI = 0.79–1.40))
6	Li et al., 2021b [26]	32	Observational studies and RCTs	Review of studies examining the protective effect between N95 respirators and medical masks for preventing HCWs from RVIs and clinical respiratory illnesses	HCWs; health-care settings	RVIs and clinical respiratory illnesses	N95 respirators vs. Medical masks	N95 respirators did not associate with significant decreased RVIs risk compared with medical masks (laboratory-confirmed RVIs (RR = 0.99, 95% CI = 0.86–1.13), clinical respiratory illnesses (RR = 0.89, 95% CI = 0.45–1.09), influenza-like illness (RR = 0.75, 95% CI = 0.54–1.05), and pandemic H1N1 for laboratory-confirmed RVIs (OR = 0.92, 95% CI = 0.49–1.70)) N95 respirators associated with a significant reduction in beta coronaviruses caused infection odds compared with medical masks (adjusted OR = 0.43, 95% CI = 0.20–0.94)
7	Li et al., 2021a [23]	6	Observational studies (case–control)	Review of studies examining the protective effect of rPPE for preventing SARS-CoV-2 virus infection	HCWs and non-HCWs	SARS-CoV-2 virus infection	rPPE (type not reported) vs. no masks; subgroup: N95 respirators vs. no masks	rPPE associated with a significant reduction in SARS-CoV-2 virus infection odds compared with no masks (adjusted OR = 0.19, 95% CI = 0.09–0.38), with stronger association in HCWs subgroup (adjusted OR = 0.18, 95% CI = 0.09–0.34); N95 respirators associated with a significant reduction in SARS-CoV-2 virus infection odds compared with no masks (adjusted OR = 0.2, 95% CI = 0.09–0.44)
8	Kunstler et al., 2022 [27]	21	Observational studies and RCTs	Review of studies examining the protective effect between N95 respirators or equivalents and surgical masks for preventing HCWs from SARS-CoV-2 virus infection	HCWs; health-care settings	SARS-CoV-2 virus infection	N95 respirators or equivalents vs. Surgical masks	N95 respirators or equivalents did not associate with a significant reduction in SARS-CoV-2 virus infection odds compared with surgical masks (OR = 0.85, 95%CI = 0.72–1.01)
9	Tran et al., 2021 [20]	16	RCTs	Review of studies examining the efficacy of rPPE in preventing respiratory infections	HCWs and non-HCWs; health-care settings and non-health-care settings	Respiratory infections	Network comparison among different types of rPPE (N95 respirator, surgical mask, cloth mask)	Wearing medical masks associated with significant increased respiratory infection risk compared with fit-tested respirators (RR 1.26 95% CI = 1.01–1.56)
10	Yin et al., 2021 [21]	6	RCTs	Review of studies examining the most suitable type of rPPE and manner of wearing to prevent HCWs from RVIs	HCWs; health-care settings	Laboratory-confirmed RVIs	Network comparison among different types of rPPE (N95 respirator, surgical mask, cloth mask) and manners of wearing (continuous, targeted)	Continuous wearing N95 respirators associated with significant decreased RVIs odds compared with other combinations of rPPE and wearing manners (SUCRA = 85.4)

Note: n =Number of original studies for the relevant objectives; CI =confidence interval; CrI =credibility interval; HCWs =Healthcare workers; OR = Odds Ratio; rPPE =respiratory Personal Protective Equipment; RCTs =Randomized controlled trials; RR =Relative Risk/Risk Ratio; RVIs =Respiratory Viral Infections; SARS-CoV =severe acute respiratory syndrome coronavirus; SUCRA = the Surface Under the Cumulative Ranking curve Analysis.

Forest plots (Fig. 2, Fig. 3, Fig. 4 ) examine the association between the use of N95s compared with medical masks, with an outcome of respiratory illness. Referring to the studies in Table 3, Study 2 [24], 3 [22] and 7 [23] were excluded from the analysis as they compared N95 use to no mask (or medical mask use to no mask). Subset analyses within each meta-analyses were excluded if they were not mutually exclusive from the primary result, maintaining statistical independence between elements in the forest plot. In the Forest plots, each square and its horizontal line, respectively, represent the OR and its 95% CI for each individual systematic review (the size of the square corresponds to the weight of that review in the meta-analysis of the umbrella review); the diamond and its horizontal diagonal show the combined overall OR and its 95% CI, respectively. Forest plot (Fig. 2) of studies using either RCT or observational study designs shows that the odds of respiratory illness is 0.83 times lower in those using N95 respirators (OR = 0.83; 95% CI = 0.74, 0.94; p = 0.003; I² = 6.4%); sub-group of meta-analyses with only RCT studies (Fig. 3) indicates that the odds of respiratory illness is 0.79 times lower with N95 respirator use (OR = 0.79; 95% CI = 0.64, 0.98; p = 0.031; I² = 13.4%); and sub-group of meta-analyses with only HCWs working in healthcare settings (Fig. 4) indicates that the odds of respiratory illness is 0.84 times lower with N95 respirator use (OR = 0.84; 95% CI = 0.73, 0.96; p = 0.013; I² = 13.7%).

Fig. 2.

Forest plot of meta-analyses using RCT and/or observational study designs. The OR as 0.83 indicates that the odds of respiratory illness is 0.83 times lower in those using N95 respirators (OR = 0.83; 95% CI = 0.74, 0.94; p = 0.003; I² = 6.4%).

Fig. 3.

Forest plot of RCT-only meta-analyses. The OR as 0.79 indicates that the odds of respiratory illness is 0.79 times lower in those using N95 respirators (OR = 0.79; 95% CI = 0.64, 0.98; p = 0.031; I² = 13.4%).

Fig. 4.

Forest plot of meta-analyses including only HCWs working in healthcare settings. The OR as 0.84 indicates that the odds of respiratory illness is 0.84 times lower in those using N95 respirators (OR = 0.84; 95% CI = 0.73, 0.96; p = 0.013; I² = 13.7%).

Literature appraisal

The appraisal showed 8 out of 10 selected meta-analyses to have very low certainty, and 2 meta-analyses have low certainty. Study design and publication bias are “not serious” in all papers, while indirectness is rated “serious” for 5 papers, “very serious” for 4 papers, and “not serious” for 1 paper. Risk of bias, inconsistency, and imprecision have similar “serious” to “not serious” ratios (6:4, 6:4, 5:5, respectively). See Table 4 for details of the assessment.

Table 4.

Evidence certainty assessment of selected meta-analysis.

Evidence certainty assessment								Certainty
#	Reference	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Publication bias
1	Bartoszko et al., 2020 [18]	not serious	serious	serious	serious	serious	not serious	⨁◯◯◯ VERY LOW
2	Chen et al., 2022 [24]	not serious	not serious	not serious	very serious	not serious	not serious	⨁⨁◯◯ LOW
3	Chu et al., 2020 [22]	not serious	not serious	serious	very serious	not serious	not serious	⨁◯◯◯ VERY LOW
4	Collins et al., 2021 [25]	not serious	serious	serious	serious	not serious	not serious	⨁◯◯◯ VERY LOW
5	Jefferson et al., 2020 [19]	not serious	not serious	not serious	serious	serious	not serious	⨁⨁◯◯ LOW
6	Li et al., 2021b [26]	not serious	serious	not serious	serious	serious	not serious	⨁◯◯◯ VERY LOW
7	Li et al., 2021a [23]	not serious	not serious	serious	very serious	not serious	not serious	⨁◯◯◯ VERY LOW
8	Kunstler et al., 2022 [27]	not serious	serious	serious	not serious	serious	not serious	⨁◯◯◯ VERY LOW
9	Tran et al., 2021 [20]	not serious	serious	serious	very serious	not serious	not serious	⨁◯◯◯ VERY LOW
10	Yin et al., 2020 [21]	not serious	serious	not serious	serious	serious	not serious	⨁◯◯◯ VERY LOW

Discussion and conclusion

The approach we used was systematic and evidence-based. Six out of ten systematic reviews analyzed consisted of meta-analyses [[20], [21], [22],25,26] that favoured N95 respirators over medical masks, while the other four [18,19,23,24,27] did not show that N95 respirators have superior protection effect compared with medical masks at a 95% CI [18,27]. Further analysis with Forest plots for meta-analyses directly comparing N95 respirators with medical masks indicated that N95 respirators offered slightly better protection than medical masks for respiratory illnesses. Sub-group analyses for RCT-only meta-analyses, and for meta-analyses including only HCWs working in healthcare settings, gave similar results. That being said, the meta-analyses appraisal showed that the certainty of evidence from the included literature is very low, mostly due to the indirectness of the papers. Almost all original studies included in these meta-analyses were conducted before the COVID-19 outbreak, leading to the “serious” indirectness in every meta-analysis. Four papers have “very serious” indirectness because some of the original studies were conducted in non-healthcare settings and/or N95 respirators were compared not with medical masks but to non-masked controls. In addition, no meta-analyses included discussion of the setting (routine patient care versus AGMPs) in the policy scan, affecting the robustness of the final conclusion of this umbrella review. Risk of bias, inconsistency, and imprecision downgraded the certainty ratings of some papers as well.

One COVID-19 Scientific Advisory Group, in a rapid evidence review on the masking guidance for HCWs in 2021, stated that N95 respirators did not show superior protective effectiveness compared to surgical masks in real-world data and concluded that the current masking policy should be retained [28]. On the other hand, Griswold et al. [29] conducted an umbrella review of systematic reviews and reported that N95s and equivalents may have higher protective effectiveness compared with surgical masks. However, this conclusion was only based on Chu et al.‘s [22] study as the later studies included in our present study were not available at the time. Of note, “very low” certainty was assigned to the study by Chu et al. because 1) it did not compare respirators with surgical masks directly, 2) the outcome was not related to COVID-19 infection, and 3) the findings had high heterogeneity.

The umbrella review of meta-analyses presented thus suggests the possibility of greater protection with the use of N95 respirators, particularly if continuously used. However, the research underpinning this conclusion lacks the rigor to support a definitive policy change with respect to protection against the SARS-CoV-2 Omicron variant. A study recently published, and therefore not included in the systematic reviews assessed, did indeed directly compare surgical masks to respirators for HCWs caring for known or suspected COVID-19 patients; it showed no significant difference in the outcome of COVID-19 infection [30].

Beyond the evidence presented here and in previous studies, policymakers need to take other factors into account, including the results of a full risk assessment of the likelihood and severity of COVID-19 Omicron variants in specific contexts, considerations of healthcare practice, side-effects, acceptability, cost and availability of N95 respirators, as well as broader equity issues and the hierarchy of controls.

Given the lower morbidity and higher risk of omicron transmission in the community mandating universal use of N95, even where they have not historically been indicated, may not necessarily result in reduced infection rate in HCWs, given that community exposure remains the major risk factor for contracting COVID-19 [[31], [32], [33], [34], [35]] and a broader approach to HCW protection is needed.

Importantly, the absence of robust scientific evidence should not be the reason for withholding the best possible controls to mitigate potential risks (to the large population of HCWs in our context), as stated in the definition of the precautionary principle [36]. Nonetheless, donning the most protective gear is not always the best control measure if one takes all considerations into account. For example, given PPE supply and logistical constraints, overusing N95 respirators in one setting or jurisdiction may have the effect of creating scarcity in other settings or jurisdictions with attendant inequities and global health consequences [37]. Furthermore, using the costliest PPE may not necessarily be the best way to effectively use scare healthcare resources for protecting HCWs. Also, if HCWs in low- and middle-income countries who need N95s most cannot get them because of overuse in high-income countries, the pandemic may be perpetuated with new variants emerging putting HCWs globally more at risk.

Moreover, practical considerations include the linked logistics of mask policies in healthcare settings, manners of masking (targeted versus continuous), and nature of PCRA guidance. Many jurisdictions recommend that HCWs conduct PCRA before every patient interaction to choose the appropriate respiratory PPE (N95 respirator or medical mask) and decide how to wear it (targeted or continuous). Only one paper focused on continuous versus targeted use, with findings suggesting that continuously wearing an N95 respirator may have the highest effectiveness against clinical symptoms of respiratory infection among all combinations of respiratory PPE and wearing manner; however, continuous wearing did not offer better protection than targeted wearing at a 95% confidence level regarding the same respiratory PPE [21]. This single study therefore cannot justify the necessity of adopting a mandatory continuous masking policy. It is important to consider the preferences and acceptability of recommendations by HCWs to prevent rejection of recommendations or less desirable work-arounds to avoid unpleasant side-effects of respirators. For example, prolonged N95 respirator wearing (over 1 h) has been reported to cause significant physiological effects, including headache, increased breathing burden, and nervous system and cardiovascular system changes (e.g., reduced cognition, decreased cardiac contractility), due to the elevated carbon dioxide (CO₂) concentration [38]. Kunstler et al. [27] reported that HCWs who wore respirators over a prolonged period experienced significantly more headaches, respiratory distress, facial irritation, and pressure-related injuries (OR = 4.39, 95% CI = 2.37–8.15, which is the greatest odds ratio among all findings) compared with HCWs who only wore surgical masks. In terms of sweating, attention deficit or disorder, and erythema, there were no significant differences between N95 respirators and surgical masks. The authors also indicated that risk of bias (e.g., participation bias) and confounders (e.g., fit-testing status, comorbidities, mitigation strategies for adverse effects, etc.) may influence the results (e.g., unfit-tested respirators could cause higher adverse effects rate). Bakhit et al. [39] conducted a systematic review and meta-analysis on the downsides of face masks, showing that participants adhered to medical masks significantly better than N95 respirators.

A limitation of this study is that it did not include qualitative studies, which may contain useful information that might support different conclusions. Also, evidence of side-effects and cost of N95 respirator were not systematically appraised. Moreover, all of the studies included in the umbrella review were not conducted during the period when the current Omicron variant was most prevalent. Finally, the downgrade criteria of indirectness may be overstated if the findings regarding non-SARS-CoV-2 respiratory viral infections are more directly applicable to the SARS-CoV-2 and Omicron variant.

Risk of bias (e.g., selection bias, reporting bias, recall bias, participant bias) and confounders (e.g., vaccination status, community-acquired infection, workplace setting, other infection controls) possibly hindered researchers from conducing more rigorously designed studies regarding respiratory PPE effectiveness, therefore limited the number of meta-analyses from our systematic search. Different literature selection and appraisal criteria (e.g., study design, population, intervention vs. comparison, types of effect, etc.) contributed to the variability of the findings in our meta-analyses. Notably, the RCTs [[40], [41], [42], [43]] selected by Collins et al. [25] were identical with RCTs included in Bartoszko et al.‘s review [18] and were selected by Jefferson et al. [19] as well. However, Collins et al. [25] concluded differently as there were also observational studies in the pool.

Despite the limitations of the current state of knowledge, the evidence appraisal, in combination with a risk assessment, practical considerations, and application of the precautionary principle, supports maintaining the current policy whereby HCWs have the option of accessing N95 respirators after practicing a PCRA. Further, due to the limitations of the current scientific evidence to support changing the current masking policies, either to mandated continuous universal respirator wearing or to medical mask-wearing only, a flexible mask wearing policy is recommended over rigid policies one way or the other.

Future studies regarding respiratory PPE effectiveness should stratify by healthcare settings and variants of concern at the time. A good example is the prospective multicentre cohort study by Haller et al. [44] comparing respirators with surgical masks on protecting HCWs from COVID-19 in non-AGMPs settings. In addition, adequately powered RCTs comparing different PPE strategies to protect HCWs from respiratory viruses are required.

In the current as well as future pandemics of respiratory infectious disease, the state of scientific evidence supporting masking policies/strategies should be reviewed with a broad view when applying the precautionary principle. Systematic attention should be devoted to whether the risk is increased or decreased compared to previous pathogens or variants, acceptability of masking policy and side-effects for HCWs, as well as wider equity concerns. While properly designed RCTs are urgently needed, it is essential that their results be interpreted with a fulsome understanding of the hierarchy of controls including an appreciation of the global connectedness that drives the risks to HCWs during a pandemic.

Authorship statement

YL, CM, and AY substantially contributed to conceptualization and design. YL drafted the initial manuscript and revised it critically for important intellectual content. YL and AO conducted database searches and literature appraisal. All authors contributed to the manuscript revision and approved the submission.

Funding

CM received funding via a Michael Smith Health Research BC Scholar Award and from WorkSafeBC.

AY received funding as a Tier 1 Canada Research Chair from the Canada Research Council.

Ethics

Ethics approval and informed consent was not required as this was an umbrella review of previously published meta-analyses.

Conflict of interest

The authors do not have any conflict of interest to declare.

Appendix A. Supplementary data

The following is the Supplementary data to this article.