Comparative efficacy of respiratory personal protective equipment against viral respiratory infectious diseases in healthcare workers: a network meta-analysis

Abstract

Objective

With the epidemic of coronavirus disease 2019 (COVID-19), the healthcare workers (HCWs) require proper respiratory personal protective equipment (rPPE) against viral respiratory infectious diseases (VRIDs). It is necessary to evaluate which type of mask and manner of wearing is the best suitable rPPE for preventing the VRID.

Study design

A Bayesian network meta-analysis was performed to comprehensively analyze the protective efficacy of various rPPE.

Methods

This network meta-analysis protocol was registered in an international prospective register of systematic reviews (CRD42020179489). Electronic databases were searched for cluster randomized control trials (RCTs) of comparing the effectiveness of rPPE and wearing manner in preventing HCWs from VRID. The primary outcome was the incidence of laboratory-confirmed viral respiratory infection reported as an odds ratio (OR) with the associated 95% credibility interval (CrI). The secondary outcome was the incidence of clinical respiratory illness (CRI) reported as an OR with the associated 95% CrI. Surface under the cumulative ranking curve analysis (SUCRA) provided a ranking of each rPPE according to the primary outcome and the secondary outcome as data supplement.

Results

Six studies encompassing 12,265 HCWs were included. In terms of the incidence of laboratory-confirmed viral respiratory infection, the continuous wearing of N95 respirators (network OR, 0.48; 95% CrI: 0.27 to 0.86; SUCRA score, 85.4) showed more effective than the control group. However, in terms of reducing the incidence of CRI, there was no rPPE showing superior protective effectiveness.

Conclusions

There are significant differences in preventive efficacy among current rPPE. Our result suggests that continuous wearing of N95 respirators on the whole shift can serve as the best preventive rPPE for HCWs from the VRID.

Introduction

Respiratory personal protective equipment (rPPE) is critical to reducing the risk of spreading respiratory pathogens in the current coronavirus disease 2019 (COVID-19) epidemic.¹ There are three main types of rPPE currently used among healthcare workers (HCWs): cloth mask, surgical mask, and N95 respirator. Cloth mask, although it has been replaced by the disposable surgical mask, is still used repeatedly in the area wherein the shortage of rPPE. The surgical mask has traditionally been used by HCWs to avoid hand-to-face contact and prevent respiratory droplet transmission, although it may not be reliable for preventing aerosol transmission.² The N95 respirator is designed to prevent HCWs from inhaling small airborne particles when treating patients with suspected viral respiratory infectious diseases (VRIDs).³ In the epidemic of COVID-19, the N95 respirator is strongly recommended to use in HCWs from occupationally acquired infections through droplet or airborne spread.¹

Although N95 respirator has been thought to be superior to other masks (e.g. surgical mask) in preventing the COVID-19 and the other common VRIDs (adenoviruses, influenza, respiratory syncytial virus, metapneumovirus, parainfluenza virus, rhinovirus enterovirus, coronavirus, coronaviruses parainfluenza viruses, severe acute respiratory syndrome–associated coronavirus, adenoviruses and human bocavirus, coxsackie/echoviruses), the existing evidence is still controversial.^{4 , 5} This may be due to that the diagnosis of VRIDs replies on laboratory tests based on nucleic acids or antibodies. It could be false negative results generated from nucleic acid and antibody based on lab tests. To minimize the risk of missing specific targets and increase the sensitivity, clinical respiratory illness (CRI) is an important supplemental method to diagnose VRIDs. We include the data from laboratory tests, as well as clinical presentations (coryza, fever [temperature >37.8°C], lymphadenopathy, tachypnea [respiratory rate >25/min]) and symptoms (arthralgias/myalgias/body aches, chills, cough, diarrhea, dyspnea, fatigue, headache, malaise, other gastrointestinal systems, sore throat, sputum production, sweats, vomiting/nausea),⁶ in this network meta-analysis (NMA) to better evaluate and compare the protective effects of rPPE.

It is noteworthy that asymptomatic infections existed in many VRIDs. Therefore, the wearing manner is also important for HCWs who were facing the suspectable patient.7, 8, 9 There are two types of wearing manners including continuous wearing and targeted wearing. Continuous wearing refers to wearing of the rPPE during the whole working shift, whereas targeted wearing refers to wearing it only on performing high-risk procedures (e.g. endotracheal intubation) or when in high-risk situations (e.g. entering an isolation room or barrier nursing of a patient).¹⁰ The previous study also found out that different wearing manners affected the protective efficacy of rPPE.¹⁰ Recently, Bartoszko et al.⁵ provided a negative result in comparing surgical masks and N95 respirators in COVID-19 epidemics. However, the manner of wearing might take consideration in preventive efficacy in rPPE. Hence, we conducted this network meta-analysis to examine which type of mask and manner of wearing is the best suitable rPPE for preventing the VRID.

Methods

Study design

In this Bayesian network meta-analysis, we compared the efficacy of various rPPE in preventing VRIDs in HCWs.

Data sources and search strategy

This NMA protocol was registered in a prospective register of systematic reviews (CRD42020179489). This NMA was conducted following guidelines in the preferred reporting items for systematic reviews and meta-analyses report, the extension of network meta-analyses.¹¹ PubMed, the Cochrane Library, Web of Science, and EMBASE were searched by computer to collect cluster randomized control trials of comparing the effectiveness of rPPE and wearing manner in preventing HCWs from VRIDs. The retrieval time limit was from Jan 1, 1970, to Dec 31, 2019. Simultaneously, the research and related systematic evaluation references that have been included in the manual retrieval are conducted to supplement and obtain relevant literature. Through PubMed, the search strategy is determined, and the search is carried out with a combination of subject words and free words. English search terms include the randomized controll trials (RCT), mask, face mask, respirator trace effects, respirator masks, N95 respirator masks, virus, and so on.

Study selection and eligibility criteria

Inclusion criteria were cluster RCTs comparing the effectiveness of rPPE and wearing manner in preventing HCWs from VRIDs. The outcome includes the incidence of laboratory-confirmed viral respiratory infection and the incidence of CRI (Table 1 ). Exclusion criteria were non-RCT experiments; incomplete or repeated publication of relevant data; non-human studies; and reviews, study protocols, comments, case reports, and letters.

Table 1.

Characteristics of the included studies.

Study ID	Country	Study size	Sex (male/female)	Age (years)	(Allocated numbers in arms)	Laboratory-confirmed viral respiratory infection	Clinical respiratory illness
Loeb18 2008-09	Canada	446	26/420	36.15 ± 10.59	Targeted wearing of surgical mask (225) vs targeted wearing of N95 respirator masks (221)	Parainfluenza, influenza viruses A and B, respiratory syncytial virus, metapneumovirus, rhinovirus enterovirus, coronavirus	Body temperature 38 °C or greater; new or worsening cough; shortness of breath
MacIntyre16 2008-09	China	1922	N/A	N/A	Continuous wearing of N95 respirator masks (949) vs continuous wearing of surgical masks (492) vs control (481)	Adenoviruses, human metapneumovirus, coronaviruses, parainfluenza viruses, influenza viruses A and B, respiratory syncytial viruses A and B, or rhinovirus A/B	N/A
MacIntyre17 2008-09	China	1441	142/1299	33.63 ± 9.56	Continuous wearing of surgical mask (492) vs continuous wearing of N95 respirator masks (949)	Adenoviruses, human metapneumovirus, coronavirus, parainfluenza viruses 1, 2 and 3, influenza viruses A and B, respiratory syncytial virus A and B, rhinovirus A⁄ B and coronavirus	At least two respiratory symptoms (cough, sneezing, runny nose, shortness of breath, sore throat) or one respiratory symptom and one systemic symptom (including fever, headache, and lethargy).
MacIntyre10 2009-10	China	1669	243/1426	33.1 ± 9.61	Continuous wearing of surgical masks (572) vs targeted wearing of N95 respirator masks (516) vs continuous wearing of N95 respirator masks (581)	Adenoviruses, human metapneumovirus, coronaviruses parainfluenza viruses, influenza viruses A and B, respiratory syncytial viruses A and B, or rhinoviruses A/B	At least two respiratory symptoms (cough, sneezing, runny nose, shortness of breath, sore throat); one respiratory symptom and one systemic symptom (including fever, headache, and lethargy).
MacIntyre15 2011	Vietnam	1607	357/1250	35.65 ± 10.39	Continuous wearing of surgical masks (580) vs continuous wearing cloth masks (569) vs control (458)	Respiratory syncytial virus (RSV) A and B, human metapneumovirus (hMPV), influenza A and B, parainfluenza viruses, influenza C, rhinoviruses, severe acute respiratory syndrome (SARS)–associated coronavirus, adenoviruses and human bocavirus (hBoV)	At least two respiratory symptoms (cough, sneezing, runny nose, shortness of breath, sore throat); one respiratory symptom and one systemic symptom (including fever, headache, and lethargy).
Radonovich6 2011-15	America	5180	798/4382	43 ± 11.55	Targeted wearing of N95 respirator masks (2512) vs targeted wearing of surgical masks (2668)	Coxsackie/echoviruses, coronavirus, human metapneumovirus, human rhinovirus, influenza A and B, parainfluenza virus, respiratory syncytial virus	At least 1 sign or 2 symptoms listed, representing a change from baseline. Sign: coryza, fever (temperature >37.8 °C), lymphadenopathy, tachypnea (respiratory rate >25/min); Symptoms: arthralgias/myalgias/body aches, chills, cough, diarrhea, dyspnea, fatigue, headache, malaise, other gastrointestinal systems, sore throat, sputum production, sweats, vomiting/nausea

Data extraction and quality assessment

Two authors (X.Y. and X.W.) independently extracted relevant data parameters. In case of disagreement, the arbitration shall be conducted by the corresponding author. The following data extraction parameters were extracted: name of the primary author, country of study, number of HCWs, number of participants per arm, HCW age (mean or median and standard deviation [SD] or range, if available), the gender of HCWs, quality information included in the study, the efficiency of the incidence of laboratory-confirmed viral respiratory infection in the rPPE arm and control arm, the efficiency of the incidence of CRI in the rPPE arm and control arm.

The study quality was assessed by two authors (X.Y. and X.W.) according to Cochrane Collaboration's tool. It includes six aspects: sequence generation, allocation consideration, blinding, incomplete outcome data, no selective outcome reporting, other sources. RevMan software (v 5.3) was only used for the risk of bias summary. In case of disagreement, the arbitration will be conducted by the corresponding author.

Outcomes

The prespecified primary outcome was the incidence of laboratory-confirmed viral respiratory infection reported as an odds ratio (OR) value and 95% credibility interval (CrI). The OR value was calculated by taking the odds of laboratory-confirmed viral respiratory infection in a specific rPPE group and dividing this value by odds of the control. The prespecified secondary outcome was the incidence of CRI reported as an OR with the associated 95% CrI. The OR was calculated by taking the odds of CRI in a specific rPPE group and dividing this value by odds of the control. The protective effect of rPPE was defined as an OR (including the associated 95% CrI) falling under unity (1.0).

Statistical analysis

The Stata15 SE was used for network diagrams. ADDIS software (version1.16.8) was used for network meta-analysis; all analyses use a random model by default. Node-split analysis was used to test the consistency between direct and indirect comparisons. If the P value > 0.05, a consistency type was used; otherwise, an inconsistency type was used. If node-split analysis could not be applied, both type data were reported.¹² Potential scale reduction factor (PSRF) analysis was used to determine the model convergence; when the PSRF value was 1,¹³ approximate convergence had been reached. Network OR value, and 95% CrIs were used as the effect magnitude, output ranks, and the surface under the cumulative ranking curve analysis (SCURA) value.

R software (version 3.6.1) was used for heterogeneity analysis and sensitivity analysis. According to the Cochrane handbook, Q-values less than the degree of freedom (DF), P values greater than 0.10, and I² values between 0% and 40% suggested no significant heterogeneity. If the Q-value was greater than the DF, the P value was less than 0.10, and the I² value was between 75% and 100%, the data were considered heterogeneous.¹⁴ Sensitivity analysis was conducted by changing the random model to the fixed model; if the results show no significant change, the sensitivity was low, and the results are relatively stable and reliable. No publication bias analysis was conducted in this NMA as only 10 studies were included.¹⁴

Results

Characteristics of the included studies

The search produced 745 citations, and 21 eligible articles were retrieved in full text. Six cluster RCTs were included after screening (Fig. 1 ).^{6 , 10 ,} 15, 16, 17, 18 In total, 12,265 HCWs were analyzed with a network meta-analysis, including 3 two-arm studies and 3 three-arm studies (Table 1). In the included literature, we studied five kinds of interventions and one control group. The average age of the HCWs was 38.66 ± 11.65 years (10,343 HCWs, five articles included, one article not reported¹⁶), and the sex ratio was 0.18 (10,343 HCWs, 1,566 men, 8,777 women, five articles included, one article not reported¹⁶).

Fig. 1.

PRISMA process. PRISMA, preferred reporting items for systematic reviews and meta-analyses.

Risk of bias and quality of evidence assessment

The research included in this study used RevMan software (version 5.3) for the risk of bias summary (Fig. 2 ). All of them were designed as cluster randomized controlled studies. All studies used random allocation concealment. Two studies were reported as double blinded.^{6 , 18}

Fig. 2.

Risk of bias summary. IORV, without control arm.

The incidence network meta-analysis for the laboratory-confirmed viral respiratory infection.

A total of six cluster RCTs were included in this NMA. There were 3 two-arm studies and 3 three-arm studies. A total of six nodes were included in this NMA, with each node representing a different rPPE and a wearing manner; the analysis results are shown in Fig. 3 . The size of each node represents the included number of HCWs for the intervention. The width of each line represents the number of direct comparisons between interventions (Fig. 3A). The most studied interventions were continuous wearing of surgical masks (4 RCTs) and continuous wearing of N95 respirators (3 RCTs).

Fig. 3.

Network meta-analysis for laboratory-confirmed viral respiratory infection. (A) The network plot shows a comparison of the incidence of the laboratory-confirmed viral respiratory infection between nodes (blue circles). Each node represents a unique rPPE and wearing manner or control; the size of each node represents the included HCWs for the intervention. The width of each line represents the number of direct comparisons between interventions. The connecting line noted the number of trial-level comparisons between the two nodes. (B) The forest plot of the network meta-analysis comparing the VRID of each rPPE group against the control group. (C) Schematic detailing the most efficacious rPPE class in terms of reducing laboratory-confirmed viral respiratory infection according to the surface under the cumulative ranking curve analysis (SUCRA). HCWs, healthcare workers; OR, odds ratio; CrI, credibility interval; rPPE, respiratory personal protective equipment; VRID, viral respiratory infectious disease. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

In the NMA, node splitting analysis shows P-value is 0.96 (Table 2 ), so we used the consistency type to analyze data. After 100,000 simulation iterations, the PSRF value is 1, indicating that approximate convergence has been reached. Pooled network OR values indicate that continuous wearing of N95 respirators (network OR, 0.48; 95% CrI: 0.27 to 0.86) showed significant superiority over the control group (Fig. 3C). Forest plot of the network meta-analysis comparing differences of the efficacy of each rPPE class against the control group (Fig. 3B). SUCRA analysis provided a ranking of each rPPE and a wearing manner according to its efficacy in reducing the incidence of laboratory-confirmed viral respiratory infection (Fig. 3C). The top-ranked rPPE was the continuous wearing of N95 respirators (SUCRA score, 85.4; Fig. 3C).

Table 2.

Node splitting analysis for incidence of laboratory-confirmed viral respiratory infection.

Name	Direct effect	Indirect effect	Overall	P-Value
Continuous wearing of N95 respirator, control	0.76 (0.01, 1.60)	0.72 (−0.26, 1.77)	0.74 (0.16, 1.31)	0.96

Heterogeneity analysis shows no significant heterogeneity (Q-value = 1.32 < 4 (DF), P-value = 0.86, I²-value = 0%) (Fig. 3B). The sensitivity analysis was conducted, and results showed that there is no significant change in fixed models (Supplementary Fig. 1A and B). Hence, the sensitivity was low, and the results were stable.

The incidence network meta-analysis for CRI

A total of five cluster RCTs were included in this NMA. There were 3 two-arm studies and 3 three-arm studies. A total of six nodes were included in the efficiency of the incidence of CRI network meta-analysis, with each node representing a different rPPE and a wearing manner; the analysis results are shown in Fig. 4 . The size of each node represents the included HCWs for the intervention. The width of each line represents the number of direct comparisons between interventions (Fig. 4A). The most studied interventions were continuous wearing of surgical masks (3 RCTs) and targeted wearing of N95 respirators (3 RCTs). In the NMA, because the node splitting analysis can not run, we provide both consistency type and inconsistency type data (Supplementary Fig. 2A and B). Pooled network OR values indicate that no intervention was significantly superior over the control group (Fig. 4C). Forest plot of the network meta-analysis comparing differences of CRI of each rPPE class against control group (Fig. 4B). SUCRA analysis provided a ranking of each rPPE and wearing manner according to its incidence of CRI (Fig. 4C). Although all classes were equivalent to controls, the top-ranked rPPE was the continuous wearing of N95 respirators (SUCRA score, 79.5; Fig. 4C).

Fig. 4.

Network meta-analysis for clinical respiratory illness. (A) Network plot showing comparisons of the incidence of clinical respiratory illness between nodes (blue circles), each representing a unique rPPE and wearing manner or control; the size of each node represents the included HCWs for the intervention. The width of each line represents the number of direct comparisons between interventions. The connecting line noted the number of trial-level comparisons between the two nodes. (B) Forest plot of the network meta-analysis comparing the CRI of each rPPE group against the control group. (C) Schematic detailing the most efficacious rPPE classes in terms of reducing CRI according to the surface under the cumulative ranking curve analysis (SUCRA). HCWs, healthcare workers; OR, odds ratio; CrI, credibility interval; rPPE, respiratory personal protective equipment; CRI, clinical respiratory illness. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

Heterogeneity analysis shows no significant heterogeneity (Q-value = 0.73 < 2 (DF), P-value = 0.69, I²-value = 0%) (Fig. 4B). The sensitivity analysis was conducted, and the results showed that the 95% CrI has a significant change in fixed and random models (Supplementary Fig. 2C and D). Hence, the sensitivity was high, and the results were unstable.

Discussion

COVID-19 is predominantly transmitted by contact or droplet. Airborne transmission may occur if the patient had respiratory symptoms such as coughing or HCWs performing high-risk procedures such as incubation.¹⁹ Preventing VRID transmission by rPPE is highly recommended. But current guidelines for the use of rPPE in HCWs in the hospital setting are based on limited evidence-based studies.²⁰ In this NMA of 6 RCTs consisting of 11,828 HCWs, we compared the protective effect of three types of rPPE. In addition, we focused on the wearing manner for further assessment. Results of NMA showed that continuous wearing of N95 respirators on the whole shift may have better protection against VRIDs, whereas there is no significant difference in the CRI.

Appropriate rPPE use is critical to decreasing the infectious risk for HCWs. However, previous RCTs showed inconsistent results in different rPPE.^{21 , 22} Our finding supports that the N95 respirator is superior to the surgical mask and the cloth mask. Furthermore, continuous wearing showed an increasingly protective effect against VRIDs. In the medical setting such as in the emergency medicine department, patients with VRIDs are not able to be screened or confirmed by serological tests or medical imagining. HCWs who are exposed to such an environment will face a higher risk of VRIDs. In the COVID-19 epidemic, asymptomatic carriers had been proved to be contagious,⁸ which also became a potential risk for HCWs. Hence, the continuous wearing of the N95 respirator during the whole shift might provide more consistent and reliable protection for HCWs.

The unexpected result is that the targeted wearing of surgical masks showed better efficacy than the continuous wearing of it. It could be prolonged and continuously wearing time leads to moist condensation to the inner layer of mask which decrease filtration rate and its efficacy.²³ Reusable cloth mask, which is widely used in the underserved area, showed only marginal protection against VRIDs. Lack of proper guidelines and equipment to decontaminate reusable cloth mask could contribute to this because the airborne pathogen can survive on the mask surface for days. Besides, the cloth mask showed lower filtration capacity than disposable masks.²⁴

The sensitivity analysis of the incidence of CRI is unstable (Supplementary Fig. 2). Therefore, we assumed there is no rPPE superior to the control group in preventing VRIDs (Fig. 3C). We speculated that factors that drive other biases are difficult to quantify. Therefore, our finding of the aforementioned rPPE in preventing VRID by the incidence of CRI should be cautiously interpreted.

There are some other limitations to our analysis. Firstly, the consistency evaluation for the protective efficacy of rPPE may vary in different studies. Different medical settings might have different risks of infection. For example, HCWs in emergency settings are more susceptible to VRIDs due to the crowded environment comparing with a well-controlled surgical/operation setting. To better evaluate the protection level of rPPE, it is preferred to evaluate rPPE under the same working environment and treating similar patient groups. Secondly, there is a limited quantity of studies that focus on rPPE have been published during this COVID-19 epidemic. Therefore, the reliability of selection in this study is relatively lacking. The network meta-analysis for the incidence of CRI failed to pass the sensitivity analysis. Some unknown biases may exist. Therefore, the results should be dealt with some cautions. Thirdly, as no other studies could be found based on the inclusion criteria, the retrieval time was set from 1970 in search strategy. Finally, our results were largely based on previous studies about other VRIDs such as influenza. Although the World Health Organization has recommended using N95 respirators to prevent COVID-19 in HCWs, our results should be interpreted with caution. More COVID-19 RCTs need to be performed to further support our results.

Conclusion

This NMA showed that continuous wearing of N95 respirators on the whole shift may have the best protection against VRIDs. Surgical mask, on the contrary, needs to be replaced frequently for better efficacy. In terms of cloth mask, although it is still being used, it only provides marginal protection against VRIDs. Further analysis should include more RCTs during this COVID-19 epidemic.

Author statements

Ethical approval

Ethical approval was not required for this study.

Funding

None declared.

Competing interests

The authors have no conflicts of interest to disclose.

Author contributions

C.H. conceptualization (lead) and in writing, reviewing, and editing the manuscript (equal).; X.Y contributed in writing methodology of the study (lead); software (lead); formal analysis (lead); and writing, reviewing, and editing the manuscript (equal). X.W. writing the original draft (lead) and writing, reviewing, and editing the manuscript (equal). S.X. contributed in writing, reviewing, and editing the manuscript (equal).

Appendix A. Supplementary data

The following are the Supplementary data to this article: