Revisiting the evidence for physical distancing, face masks, and eye protection

We read with great interest the results of the systematic review¹ on the effect of personal protective equipment (PPE) to prevent SARS-CoV-2 infection, predominantly based on evidence from other betacoronaviruses. As this work raised many more questions than it answered, and because its implications are far-reaching, we highlight several salient concerns.

To evaluate the association of mask use with viral infection, the Derek Chu and colleagues completed a meta-analysis of adjusted odds ratios (aORs). However, Seto and colleagues² reported only unadjusted ORs, whereas three other investigator groups adjusted for different sets of covariates.3, 4, 5 Thus, the reported effect sizes are not comparable, and it might not be appropriate to combine them.⁶ Furthermore, Seto and colleagues² reported results for both N95 respirators and surgical masks, but it is unclear why Chu and colleagues only included the N95 respirator data and excluded the surgical mask data. Given that there were no severe acute respiratory syndrome (SARS) infections in the surgical mask group, exclusion of the group underestimates the benefit of surgical masks. Also, Alraddadi and colleagues’ findings⁴ were inappropriately included as in their comparison of N95 respirator versus no mask, 95% of the no-mask group were health-care workers (HCWs) who wore face masks more often than “not always”.⁴

Our close evaluation also revealed two key instances of data duplication. First, a case-control study by Nishiura and colleagues of PPE use in a hospital in Vietnam.⁷ A second report of the outbreak from the same hospital in the same month of the year was described by Nishiyama and colleagues⁸ and included the same subjects as in the initial report by Nishiura and colleagues.⁷ Only one of these studies should have been included. Second, a study by Liu and colleagues⁹ is an English language duplicate of the earlier study by Ma and colleagues³ as strictly stated by the authors.⁹ Two questions arise: why did Chu and colleagues¹ include both studies of the same dataset as separate entities, and why does the subgroup analysis of N95 respirators only include data from Ma and colleagues, but the subgroup analysis of surgical masks only includes data from Liu and colleagues? Moreover, conducting an overall meta-analysis while there are repeated (ie, not independent) data is inappropriate—an alternative methodology, such as a network meta-analysis, would be preferred.

Beyond the issues with duplicated studies, might it even appropriate to combine data from health-care and non-health-care settings, for example, Lau and colleagues¹⁰ and Wu and colleagues,¹¹ as the transmission dynamics are very different between the two settings? In addition, in the study by Lau and colleagues, the only reported analyses on mask use were for hospital visitors by household members of infected patients; in these analyses, any hospital visit was associated with a higher odds of infection, with a trend towards higher odds if neither patient nor visitor wore a mask.¹¹ It was therefore surprising that an aOR of 0·32 associated with this study was reported in figure 5 of the Article.¹

Having done an unsystematic manual search of references of the included studies and a rapid literature search, we identified two additional studies that could have been eligible for inclusion in the systematic review: one by Pei and colleagues¹² and one by Loeb and colleagues.¹³ Both studies included the risk of SARS acquisition associated with the use of surgical masks without apparent duplication of data with others.

We respectfully disagree with the authors of the linked Comment, who called for a review of all guidelines that recommend a medical mask for HCWs caring for COVID-19 patients.¹⁴ This recommendation was based on a seriously flawed analysis of low-certainty evidence that should be interpreted with extreme caution, and we believe the conclusion should be contrary: there is currently no evidence that N95 respirators are more effective than surgical masks for HCW protection.

Finally, the major limitation, acknowledged by the authors, was comparing apples (in this case, SARS-CoV-2) and oranges (other betacoronaviruses). Indeed, both their clinical courses and transmission settings (community vs health-care settings) were very different. The protection of HCWs from the nosocomial acquisition of respiratory viruses is critical. However, as we are making important decisions regarding PPE, it is even more critical that we include the appropriate studies, avoid data duplication, and synthesise the data appropriately.

We declare no competing interests.