Unpacking Cochrane’s Update on Masks and COVID-19

Recently, the Cochrane Library released its anticipated update on physical interventions to control the spread of respiratory viruses, including masks to contain the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).¹ The update was widely read and cited, becoming a point of controversy in the public debate about the efficacy of face masks, as it appeared to contradict both public health guidance² and research.³ The appearance of controversy was in part owing to the methodological approach of Cochrane reviews, which allows inclusion of only randomized controlled trials (RCTs).

The authors added 11 new RCTs and cluster RCTs, of which six were conducted during the COVID-19 pandemic and evaluated various interventions for hygiene, including face masks and hand washing. Only two of the six studies compared use of face masks with no use of masks: one from Denmark, the DANMASK-19 RCT,⁴ and one from Bangladesh.⁵ But even with these limited, additional data, the appearance of disagreement between the Cochrane review results and public health guidance disappears if infectious disease models are applied, because the models calibrate quite well to the new Cochrane data and, when extrapolated, show that masks can reduce respiratory infections significantly.

TWO NEW COCHRANE REVIEW STUDIES

The DANMASK-19 study had several flaws: it was underpowered; was not able to evaluate the impact of masks as source control (i.e., filtering viral particles directly from the source, the infected wearer); used SARS-CoV-2 antibody testing to detect infection instead of antigen testing, which is used to identify acute illness (so that infection could have happened at any time in the past, not necessarily during the study period); and was conducted at a time of low SARS-CoV-2 circulation.^6,7

The cluster RCT in Bangladesh was a large study, with more than 340 000 participants, that demonstrated that villages receiving the intervention had increased mask use.⁵ Although the study was not designed to demonstrate mask efficacy in reducing infections, it found that increases in mask use correlated with lower SARS-CoV-2 seroprevalence. Because evidence is weighted by the number of participants, the most heavily weighted data for face masks during the COVID-19 pandemic come from the RCT conducted in Bangladesh, representing more than 95% of the new data related to the pandemic that was used in the Cochrane update.

The Cochrane review mentioned many of its own limitations and weaknesses, particularly with regard to face masks and its limited number of robust studies; it, therefore, cautioned against drawing any strong conclusions. Given the strong opinions expressed about the study, Cochrane further clarified that their review should not be used as evidence against mask efficacy per se, noting that the data were not definitive and that masks might be effective at preventing respiratory virus infection.⁸

MASKS AND MATERIALS ENGINEERING

From the standpoint of workplace safety and materials engineers, the debate on masks is something of an enigma: the utility of wearing masks should be obvious. Viruses like SARS-CoV-2 populate the respiratory tract.⁹ During talking, singing, coughing, and sneezing, viruses are expelled into the ambient air in small droplets and aerosols.^10,11 Tight-fitting masks of various weaves and fiber content filter the droplets and aerosols from the air we breathe with various efficiencies.¹² Susceptible uninfected people are protected when the infectious, potentially asymptomatic shedder wears a mask (source control) or when wearing a mask themselves (wearer protection¹³). Every step in this causal chain of reasoning has been researched and documented and has been verified in studies of household transmission of SARS-CoV-2.^14,15

The exact efficiency of transmission and filtration in each of the stages described can be measured, analyzed, and debated, but it is certainly not zero. From an engineering and materials standpoint, then, the question is not “Do masks work?” but “Do masks work as well as predicted, and if not, why not?” Of course, from immunological, epidemiological, and medical standpoints, we know that there are a host of modifiers that degrade face mask efficacy, including but not limited to the precise relationship between viral shedding and attack rates (i.e., the exact mathematical function connecting number of shed viral particles to number of secondary infections), mask contamination (e.g., wearers touching their masks and then rubbing their eyes), poor fitting around the nose and mouth, compensatory behaviors (mask wearers taking more risks because they think they are better protected than they are), and failure to maintain or use masks properly or at all (which has been a problem in most epidemiological studies of masks). Many of these modifiers contain a component of wearer training and practice, and these suggest that education about proper mask selection, use, and fit are important for improving public protection, as they are directly related to mask efficacy.

Although a detailed look at the Cochrane review demonstrates that the bias, methodological variations, and low adherence to interventions during the studies that were included preclude making firm conclusions about the effects of face masks, modeling the impact of mask wearing on transmission can make the case for masks even if we take the data added to the Cochrane review at face value. In the DANMASK-19 RCT, the authors estimated that no more than 5% of the general population used masks at the time of the study, thus masks were not a significant contribution to source control in the community.⁴ The study was not powered to detect a wearer protection efficacy of less than 50%⁷ and estimated a confidence interval (CI) ranging from a 46% reduction to a 23% increase in infections for the masked group, so that the effect was not statistically significant. Gurbaxani et al.³ predicted an approximately 28% to 32% decrease in infections in the masked group, which corroborates the DANMASK-19 measured (but nonsignificant) decrease, although the modeling study assumptions do not closely align with the conditions of the DANMASK-19 study (e.g., the model assumed masks were worn indoors and more widely used in general).

Considering other limitations of the study beyond those discussed, for example, only wearing masks outdoors (where there is much less transmission because of better ventilation), low positive predictive value of testing given low prevalence of SARS-CoV-2 at the time of the study, and potential problems with adherence,⁶ the fact that no statistically significant effects were observed for mask wearing was to be expected.

The other new RCT included in the Cochrane review that examined mask wearing to prevent SARS-CoV-2 infection, the Bangladesh RCT,⁵ did find a statistically significant, but small, effect for mask wearing. About two orders of magnitude larger than the DANMASK-19 study, the Bangladesh RCT was powered to discern a small effect size and found an 11.5% (95% CI = 6.5%, 17%) reduction in symptomatic illness and 9.5% (95% CI = 1%, 19%) reduction in seropositivity in the masked group compared with the unmasked group.⁵ It is notable that some symptomatic individuals did not consent to blood draws, reducing the seropositivity value. The study investigators were able to achieve a 42% adoption of surgical mask (medical procedure mask) wearing in the intervention communities versus 13% in the control communities (with ∼180 000 people in each group), which correlated with reductions in seroprevalence.⁵

MODEL CONSISTENCY

The Centers for Disease Control and Prevention (CDC) developed a detailed model that can predict the impact that various levels of masking would have for different types of masks, having measured the filtration efficacy of several different mask types in National Institute for Occupational Safety and Health laboratories.^3,12 The model includes the impact of both symptomatic and asymptomatic transmission; varying degrees of isolation for detected spreaders, including a Bayesian calculation for how well both symptomatic and asymptomatic people are detected; age-structured contact rates; and different levels of masking in each of those compartments.

Plugging the mask adoption rates for the intervention and control communities of the Abaluck et al. study into the model, the model results for reduced infections attributable to mask use are aligned with the Abaluck et al. results.⁵ Depending on whether you assume the ancestral virus, or Alpha variant, circulating in Bangladesh at the time of the study (November 2020–April 2021), predictions are for an 8% to 15% drop in infections in the intervention communities (Figure 3 in Gurbaxani et al.³; Figure 1 herein). Although some of the parameters used as a default in the CDC model may or may not match those of the Abaluck et al. study (e.g., the contact rates in the POLYMOD study¹⁶), the calibration points are in the approximate effect sizes we see in both the DANMASK-19 and the Bangladesh RCT studies. The CDC model also predicts a much higher impact of better-quality masks (e.g., KN95 and KF94 respirators) when used by more than 70% of the population (Gurbaxani et al.³; Figure 1), which supports general mask use during times of high transmission to ensure a high population-level impact.

FIGURE 1—

Modeled Reduction in SARS-CoV-2 Infections Among the General Population, by Mask Type Relative to No Mask: United States, November 2020–April 2021

Note. The figure shows the percentage reduction in cumulative infections after six months of simulation, relative to no mask use in the population, as mask use varies in the general, susceptible population for different types of face masks (using the model from Gurbaxani et al.³). Contact rates between age groups were taken from the POLYMOD study and therefore apply to the US population as of 2017, but other parameters used in the model were taken from a variety of sources³ and could easily apply to Bangladesh as well as the United States in the 2020–2021 timeframe. Mask source control parameters were fixed according to estimates for the given types, and wearer protection efficacy was assumed to be half of the source control efficacy. Younger susceptible persons were assumed to use masks at 70% of the rate of persons aged 65 years or older. Known infected people aged 65 years or older were masked at a 90% rate, with younger persons at 70%. All parameters were kept the same as similar figures in Gurbaxani et al.,³ except the baseline basic reproductive number (R₀, the average number of secondary infections resulting from each primary infection) in the absence of mask use was assumed to be 4.0, consistent with the Alpha variant of SARS-CoV-2. Vertical (gray) lines show mask prevalence for medical procedure (surgical) masks in the intervention and control groups of the Abaluck et al.⁵ study, and corresponding horizontal lines show the model-predicted reduction in infections over the six-month study period.

A study by Chikina et al.¹⁷ has suggested that the Bangladesh RCT had an ascertainment bias, which could explain the weak positive result as an artifact of the experiment, given that nearly all of the differences in symptomatic rates between treatment and control groups was attributable to sample size. It is not clear how differences in enrollment and consent at the start of the trial create a significant bias when the outcome is symptomatic seroprevalence at the trial’s end, the ratios of which (seropositive to symptomatic) were equal between treatment and control groups. Both Chikina et al.¹⁷ and their publicly available reviewers suggest some possible mechanisms, but these are far from proven. Alternatively, it is quite possible, as Abaluck et al. suggest,⁵ that the greater enrollment in the treatment group simply reflected that group’s motivation to obtain more masks and the treatment group’s surveillance workers’ enthusiasm to distribute them. Also, the Chikina et al. article did not address the 23% and 35%, respectively, decreases in symptomatic seroprevalence among the groups aged 50 to 59 years and aged 60 years and older in the intervention group, which cannot be explained by differences in sample size alone and would be expected according to a generally higher symptomatic prevalence for SARS-CoV-2 in those age groups.

Moreover, mask studies inherently suffer from a lack of validation of proper or consistent mask use and of measures of fit and filtration, which would tend to bias the results toward the null hypothesis that masks do not work. Insufficient mask use has been measured even in places where compliance is emphasized and monitored.¹⁸ Either way, neither positive nor negative sources of bias were supported in the Abaluck et al. study. If the Abaluck et al. study proves to be an accurate calibration point for the model, then the widespread use of high-quality, well-fitting masks during times of high transmission shows promise in slowing transmission and reducing the effective reproductive number (R_e).

One of the important criticisms of the Cochrane review is that the role of masks as source control—beyond their role of wearer protection (personal protective equipment)—is an effect that the large RCTs that Cochrane analyzes are not good at estimating.¹⁹ Many types of masks are more effective as source control than they are as wearer protection,^3,12 and, although some have disparaged the distinction,²⁰ modeling can elucidate the relative impact of source control. In particular, source control is critically important when a respiratory virus is transmitted largely asymptomatically, as seen with SARS-CoV-2.²¹

A modeling study by Glasser et al.,²² which fit high-quality national serological survey data to a metapopulation model of the spread of the virus, estimated the efficacy of nonpharmaceutical interventions (a combination of masking and social distancing) at 31% in the fall of 2020 (before vaccines were available). Overfitting is not a concern in this estimate, given that the effect of nonpharmaceutical interventions was the only parameter fit to the data in that metapopulation model (see the first table in Glasser et al.²² for the origin of all the model parameters). This estimate is also in the ballpark of what would be expected given the percentage of cloth and medical procedure mask use in the general population at the time.

MASK EPIDEMIOLOGY BEYOND MODELS

Beyond these modeling validations of the new data added to the Cochrane review of face masks, there are more than a dozen excellent observational epidemiological studies that demonstrate the positive effect of masking, with very sound data and statistical methods, that did not meet the inclusion criteria of the Cochrane framework, which favors large RCTs. Although RCTs are considered ideal, they are not without limitations. Therefore, considering many other data sources along with their strengths and weaknesses is necessary for informed policymaking.²³

Examples of other studies include that of Donovan et al.,²⁴ who looked at schools in adjacent school districts in Arkansas—some of which had mask mandates in place, some of which had partial mask mandates, and some of which had none at all—and observed that the strength of the reduction in COVID-19 cases depended on the strength of the mandate, and the results were statistically significant. Other studies of mask mandates showed similar results.²⁵ Other types of studies, which include controlled laboratory-based experimental studies, epidemiological investigations, and population-level community studies, are detailed in Table A (available as a supplement to the online version of this article at http://www.ajph.org) and merit consideration in assessing the effect of mask use in reducing SARS-CoV-2 transmission.

MORE IS MORE AND BETTER IS BETTER

The science of masking and its impact on SARS-CoV-2 transmission is complicated. Observational studies present valuable data that warrant consideration in informing policy with a full understanding of the utility of mask use in a variety of settings. The Cochrane review did not include a large body of evidence, and that resulted in a biased conclusion. If all types of studies are considered, it is clear that well-fitting, properly used masks do have a measurable and significant effect on reducing transmission when properly worn by the vast majority of the population during times of high community transmission.³ Although the data in the two new studies included in the Cochrane update on masks are accurate, modeling studies correctly predict the small effect sizes that those studies observed; furthermore, the models predict that the effect size would be much larger with better masks more widely and correctly used. Taken together, these and other studies strongly indicate that masking is an effective intervention to reduce transmission of SARS-CoV-2 (source control) and should be considered to protect those most vulnerable from severe COVID-19 illness (wearer protection) as a general nonpharmaceutical intervention during times of high transmission.