Association between COVID-19 outcomes and mask mandates, adherence, and attitudes

Abstract

We extend previous studies on the impact of masks on COVID-19 outcomes by investigating an unprecedented breadth and depth of health outcomes, geographical resolutions, types of mask mandates, early versus later waves and controlling for other government interventions, mobility testing rate and weather. We show that mask mandates are associated with a statistically significant decrease in new cases (-3.55 per 100K), deaths (-0.13 per 100K), and the proportion of hospital admissions (-2.38 percentage points) up to 40 days after the introduction of mask mandates both at the state and county level. These effects are large, corresponding to 14% of the highest recorded number of cases, 13% of deaths, and 7% of admission proportion. We also find that mask mandates are linked to a 23.4 percentage point increase in mask adherence in four diverse states. Given the recent lifting of mandates, we estimate that the ending of mask mandates in these states is associated with a decrease of -3.19 percentage points in mask adherence and 12 per 100K (13% of the highest recorded number) of daily new cases with no significant effect on hospitalizations and deaths. Lastly, using a large novel survey dataset of 847 thousand responses in 69 countries, we introduce the novel results that community mask adherence and community attitudes towards masks are associated with a reduction in COVID-19 cases and deaths. Our results have policy implications for reinforcing the need to maintain and encourage mask-wearing by the public, especially in light of some states starting to remove their mask mandates.

Introduction

As of December 2020, SARS-CoV-2, the virus responsible for COVID-19 has infected at least 66 million people worldwide and caused more than 1.5 million deaths [1]. Numerous studies have analyzed the role played by masks during the COVID-19 pandemic [2–6]: masks have been associated with a reduction in the infection rate among health care workers in a large hospital network [7], mask mandates have helped reduce the number of cases in the United States and in Germany [8–10], and simulations have shown that wearing a mask can protect against droplet infection by preventing the spread of viral particles [11–16]. Despite this evidence, 6 states recently lifted their mask mandates even though they have some of the lowest vaccination rates in the country.

A large body of literature already exists and is ongoing about the drivers behind mask usage: studies have shown strong relationships between mask usage and different types of messaging, gender and country mandatoriness [17], masculinity [18], perceptions of masks [19], political ideology and risk attitudes [20], gender, age and location [21, 22], rhetoric [23, 24] amongst many other individual, community and societal attributes [25]. Similarly, there has been work looking at the effect of masks on the economy [26], at the drivers behind the early mask shortages [27, 28] and, more generally, a large amount of work on how to optimally cause behavior change in individuals and populations [29–34]. Here, we focus on the downstream effect of masks on COVID-19 outcomes and aim to strengthen the case that masks have a positive effect on the pandemic.

To do so, we extend previous studies by investigating an unprecedented breadth and depth of outcomes, geographical resolutions, types of mask mandates, control variables, early versus later waves and controlling for other government interventions (such as stay-at-home orders). For example, it has been shown that state-level mask mandates have a positive effect on cases [8] using the same methodology as ours, the event study framework. Building on this work, we investigate the effect of mandates not only on cases, but also on hospitalizations and deaths, adding new controls for individual mobility, temperature and precipitation, COVID-19 test rates, and also investigating the effect of county-level mask mandates. Similarly, mask mandates have been found to be beneficial in a study of cases in Germany [10], but without looking at hospitalization and deaths, and not controlling for critically important temporally evolving variables such as mobility and testing rate (additionally, we expand our investigation to 69 countries). The work closest to ours investigates causal impact of mask policies [35] but does not look at hospitalizations, county-level mandates and outcomes, different types of mandates, earlier vs. later affected states, etc. However, we base our approach of how to control for voluntary individual behavioral responses to information on their causal graphical model.

Overall, our work makes the following novel contributions relative to recent studies: we look at a much larger number of counties (covering 77% of the US population) and all 50 states and the District of Columbia controlling for a number of temporally evolving and geography-specific variables compared to [9]; we investigate longer time horizon treatment effects and expand beyond hospital populations compared to [7]; we include states beyond New York and Washington compared to [11]; we include outcomes beyond hospitalization, look at all states and investigate county-level effects compared to [36]; we investigate the temporal evolution of mask mandates before and after their introduction compared to [6]; and we analyze much larger populations and include the temporal evolution and geographical heterogeneity of treatment effects compared to [37].

In addition to strengthening evidence of the effect of mask mandates on COVID-19 outcomes, this work introduces two novel contributions to the research of masks: we investigate the effect of mask mandates on mask adherence (the percentage of people who wear masks) and the effect of mask adherence on COVID-19 cases and deaths. Secondly, using a novel dataset of survey responses in 69 countries around the world across months-long waves of surveys containing 847 thousand responses in 51 languages, we investigate the effect of community mask adherence (how many people in a community wear masks) and community mask attitudes (how many people in a community believe that masks are an important for COVID-19) on country-level cases and deaths. This is especially important as similar community and adherence drivers behind mask avoidance [38] might impact the on-going deployment of vaccines.

This paper is structured as follows: in the next section, we describe our results: the main result is that mask mandates are associated with a significant improvement in COVID-19 outcomes (corresponding to 14% of the highest recorded number of daily new cases, 13% of deaths, and 7% of hospital admissions). As described in the Materials and Methods section, our analysis covers all 50 states (including the District of Columbia) and 857 counties (representing 77% of the US population) from Feb 2, 2020 to September 27, 2020, and we include a large number of robustness tests.

In support of our main result, we also observe that mask mandates are associated with a 23.4 percentage point increase in mask adherence in four geographically, culturally and politically diverse states—Hawaii, Iowa, North Dakota and New Hampshire—and that mask adherence is itself associated with a significant decrease in COVID-19 cases and deaths in all 50 states and D.C. We also observe that the recent lifting of state mask mandates is associated with decreases in adherence and increases in cases. Finally, using a large novel dataset [39], we observe that community mask adherence and attitudes towards masks are associated with fewer cases and deaths in 69 countries, controlling for population density, human development and mobility. We end by discussing the limitations and implications of this work.

Taken together, these results strongly suggest the positive effect of mask mandates, mask adherence and mask attitudes on COVID-19 cases, hospitalizations and deaths.

Results

Mask mandates and COVID-19 outcomes

As show in Fig 1, mask mandates were introduced at day zero, shown by the red vertical line. Because we are using an event study model, the y-axis represents the treatment effect associated with mask mandates on the COVID-19 outcome for each day before or after the introduction of a mask mandate relative to the day before the introduction of a mask mandate.

Fig 1. Effect of state-level mask mandates on COVID-19 outcomes.

Shaded area represents 95% confidence interval, with standard errors clustered at the state level. Red vertical line represents the start of the mask mandate, and the red horizontal line represents the line of zero treatment effect. We observe the treatment effect to start near zero when mask mandates are introduced (red vertical line) and to gradually increase (i.e. the treatment effect becomes more negative) reaching, 40 days after the introduction of mask mandates, -0.49 standard deviations (95% confidence interval is [-0.61,-0.37], adjusted R² is 0.632 and p < 0.001) for new cases (a), -2.38 percentage points ([-2.99,-1.78], adjusted R² is 0.698 and p < 0.001) for the proportion of daily hospitalization admissions (b) due to COVID-19, and 0.52 standard deviations (95% confidence interval is [-0.69,-0.35], adjusted R² is 0.537 and p < 0.001) for deaths (c). These effects are large, corresponding to 14% of the highest recorded number of cases, 13% of deaths, and 7% of admission proportion.

For all three outcomes, we observe the treatment effect (y-axis) to be negative which means that mask mandates are associated with a decrease in cases, hospitalizations and deaths. Specifically, we observe this treatment effect to start near zero when mask mandates are introduced and to gradually increase (i.e. the treatment effect becomes more negative) reaching, 40 days after the introduction of mask mandates, -0.49 standard deviations (95% confidence interval is [-0.61,-0.37], adjusted R² is 0.632 and p < 0.001) for new cases, -2.38 percentage points ([-2.99,-1.78], adjusted R² is 0.698 and p < 0.001) for the proportion of daily hospitalization admissions due to COVID-19, and 0.52 standard deviations (95% confidence interval is [-0.69,-0.35], adjusted R² is 0.537 and p < 0.001) for deaths.

In real terms, these treatment effects are large, corresponding to an associated decrease of 3.55 cases per 100,000 people (or 14% of the highest recorded number of new cases per 100,000 people during our observation period), 0.13 deaths per 100,000 people (or 13% of the highest recorded number of new deaths), and a 7% decrease compared to the highest recorded proportion (34%) of COVID-19 related hospitalization admissions during our observation period. As detailed in the specification (Eq 1) of the model in the Methods section, we control for many confounders such as other NPIs (stay-at-home and non-essential business closures), mobility, weather, new test rates and various aspects of behavior change.

Mask mandate scope: Business employees vs. the public

We conduct a number of checks to test the robustness of our results. First, we investigate changing how mask mandates are defined. As noted earlier, so far, we have defined the start of a mask mandate as the earlier date between mandates for all employees or mandates for all members of the public.

Here, we consider two alternative specifications in which we only consider mask mandates requiring everyone to wear a mask outside, and another specification where we only consider mandates requiring only business employees to wear masks. The former specification is more stringent than our original specification (which used the earlier of the two types of mandates) as these public mandates generally came later in the year than those focusing on business employees only, and more states still do not have the public mandates.

As shown in Fig 2, the trajectory and magnitude of the treatment effect associated with mask mandates under either types of mandates is consistent with our earlier estimates (shown in Fig 1) which suggests that our treatment effects are not sensitive to our definition of mask mandates.

Fig 2. Effect of different types of mask mandates.

Robustness check of main event study estimation where, here, we only consider mask mandates that require business employees (right) vs. the public (left) to wear masks. The trajectory and magnitude of the treatment effect is consistent with our previous main result, and do not differ significantly between each other, which demonstrates that our main result is not sensitive to our definition of mask mandate start dates. Full regression results are shown in S5 and S6 Tables in S1 File.

Early wave vs. later wave states

So far, we have always looked at all 50 states and the District of Columbia all together. Here we investigate the heterogeneity of treatment effect between earlier and later-wave states as it is possible that the effect of mask mandates on states that were part of the COVID-19 pandemic’s early wave might be different from states that were affected later. For example, mask shortages were present during the early stages of the pandemic [27, 40, 41] which might affect the proportion of people who wear masks (mask adherence) following a mask mandate for early states compared to later states.

Therefore, we look separately at the first 15 states to have the highest number of cases per 100K during the month of April (namely, NY, NJ, MA, CT, MI, DC, RI, IL, WA, PA, GA, VT, MD, FL, LA) compared to the rest of the states (i.e. later wave states). As shown in Fig 3, the effect of masks mandates was more gradual and noisier in earlier states whereas they had a faster effect on later states. This is consistent with the literature of behavior change whereby asking people to change their habits (here, mask-wearing) is much harder when the behavior is still very novel [29, 31]. Because the trajectory and magnitude of the treatment effect overall is consistent with our previous main result, this suggests that our specification is not sensitive to subsets of earlier wave, later wave or all states.

Fig 3. Effect of mask mandates on earlier-hit or later-hit states.

Robustness check of main event study results (section Fig 1) where we compare the effect of mask mandates on cases, deaths and hospitalization for earlier wave states only (i.e. states who had early spikes of COVID-19 peaks in the pandemic, namely, NY, NJ, MA, CT, MI, DC, RI, IL, WA, PA, GA, VT, MD, FL and LA) vs later wave states. Masks mandates took more time to have an effect on earlier states whereas they had a faster effect on later states. Full regression results are shown in S9 Table in S1 File.

Testing rate exogeneity

We also investigate if the decreases in COVID-19 outcomes we observe might be because the number of COVID-19 tests each day done might be decreasing following the introduction of mask mandates leading us to observe, erroneously, a lower rate of COVID-19 outcomes. Although we already control for the testing rate in our event study specification, here, we also investigate if testing rates were themselves exogenously decreasing during the period of observation. We do so in two ways: first, we look at the per-state and mean (over all states) testing rate (daily new tests per 100K) every day. As shown in S2 Fig in S1 File (left), it does not decrease but instead increases over our period of investigation. As a further test, we conduct an event study estimation (Eq 2) to investigate a confounding treatment effect associated with mask mandates on testing rate. As shown in S2 Fig in S1 File (right), there is no associated treatment effect of mask mandates on testing rate. Therefore, it is unlikely that our estimates of decreasing COVID-19 outcomes after the introduction of a mask mandate are due to decreased detection of COVID-19 outcomes.

Controlling for confirmed cases

We can estimate our treatment effect of mandates on the number of deaths and the proportion of hospitalizations when controlling for the number of cases. This specification allows us to investigate whether when we keep the number of cases constant (by controlling for cases), we observe that hospitalizations and deaths still decrease following the introduction of mandates. As shown in S6 Fig in S1 File, controlling for the number of cases has little effect on the trajectory of the effect of mask mandates on deaths and hospitalization which suggest that the treatment effect of mask mandates is strong even when controlling for cases.

Political affiliation, demographics and trends through time and unit fixed effects

Because we control for time and unit (state or county) fixed effects, we are implicitly controlling for a range of factors that did not change geographically and temporally. For example, including unit fixed effects allows us to control for political affiliation and including the 2018 election vote proportion by political party for each state (or county) correctly did not alter our estimation. The same is true for demographic factors such as age distributions and economic inequality. Similarly, including time fixed effects allows the model to disentangle the particular trajectory in outcome numbers that each state (or county) had over time.

Behavioral change endogeneity and pre-trends

A potential issue with the event study framework is that policy adoption might be endogenous which here might mean that people might be changing their behavior before the policy was introduced, or that they changed their exposure to the virus for other endogenous reasons unrelated to but coinciding with the timing of the introduction of mask mandates. For example, people might be going out less around the time that the mandate was introduced due to fear of infection, had started wearing masks before the mandate was introduced, or that the mandate did not increase mask adherence (how much people wear masks).

We first focus on the issue of pre-trends, which is the fact that people might have anticipatorily started wearing masks even before the introduction of mask mandates. For all three outcomes, the pre-treatment effect is statistically indistinguishable from zero (because 95% confidence interval still contains the zero-effect horizontal zerp line). We do observe a slight upward trend but further investigation using the standard methods for pre-trend robustness checks [42] shows that this pre-trend is still minimal: as show in S5 Fig in S1 File, omitting time and unit fixed effects and using a longer pre-treatment window of 20 days (we cannot use values earlier than -20 as it leads to some state no longer having enough data as many states implemented mandates very soon after COVID-19 outcome data was available) shows that the anticipatory behavior is indistinguishable from zero for cases and hospitalization, and small for deaths.

Secondly, because we control for six types of mobility (i.e. mobility at the residential, workplace, transit, parks, grocery and pharmacy and retail and recreation) and include the squared values of each mobility indicator as a proxy for social contacts [43, 44], our estimate of the effect of mask mandates has precluded (theoretically, they have been expected out) the effect of people’s endogenous mobility and contact behavior. Similarly, we include controls for the two broadest and far-reaching NPIs, stay-at-home and non-essential business closures, which again precludes the potential confounding effect of other NPIs. Another potential source of endogeneity is the fact that people might be wearing masks (or not wearing them) irrespective of the mask mandates, i.e. the mask mandates themselves did not cause behavior changes. We investigate this in the next section where we show that mask mandates lead to a 23.4 percentage point increase in mask adherence (how many people wear masks) following a mask mandate, and that mask adherence itself are directly associated with improved case and death outcomes in the US, and in 67 other countries.

As another control for behavioral changes, we leverage the causal graphical model of [35] which shows that declines COVID-19 outcomes are not only due to the introduction of policies but are also a result of private behavioral responses to observations of increased outcome numbers. A consequence of their very general causal model is that conditioning on the delayed outcome and the growth rate of the outcome with an approximately 2 week lag enables the confounding path due to behavior change on the outcome to be blocked (see [45] for a review of causal D-separation). As seen in S1 Fig in S1 File, not controlling for the delayed outcome and growth rate has little effect on the overall trajectory and magnitude of the treatment effect, suggesting that our main model estimation is robust to behavioral endogeneity.

County-level heterogeneity

So far, our analysis has been at the state level. There is significant heterogeneity at the county and city level as to when these smaller units introduced mask mandates [9]. We therefore use a dataset of county-level mask mandates [46] on county-level outcomes and controls to investigate the effect of mask mandates at the county-level. We make sure to only use county-level mask mandates (as opposed to county and state-level mandates) for location which had both, and we do not include data that happened after a state-mandate was also introduced. As shown in S4 Fig in S1 File, we again observe that mask mandates are associated with a helpful effect on cases, hospitalizations and deaths. However, the treatment effect is noisier as is expected due to a number of noise signals when using data at the county level such as peer-effects of neighboring counties’ mandates due to the underlying interdependence between county mobility patterns [47], as well as the fact that people living in one county often have to travel to a different county to get medical care, resulting in inconsistent accounting of COVID-19 health outcomes at the county level [48].

Overall, our analysis shows that the introduction of mask mandates at the state and county level is associated with a statistically significant and large decrease of cases, hospitalizations, and deaths.

Mask mandate and mask adherence

Here, as a supporting result, we investigate whether the introduction of mask mandates is associated with a significant increase in mask adherence i.e., do mask mandates lead to more people wearing masks? Although there are significant on-going efforts to estimate mask adherence at a population level such as by leveraging computer vision on social media [49–52], data that tracks on-the-ground mask usage is not currently available. On the other hand, there are a large number of studies that have created instruments to measure mask adherence based on online [53, 54] and telephone [2] surveys, and numerous studies have shown that self-reported COVID-19 attributes (e.g. symptoms [55, 56], social distancing [57]) correlate strongly with ground truth. We therefore use survey-based self-reported mask adherence as a proxy for on-the-ground mask usage in this work.

Here, we use the CMU Delphi project’s survey [58] which, as of September 8, 2020, estimates the percentage of people who wore a mask most or all of the time while in public in the past 5 days by asking the question “In the past 5 days, how often did you wear a mask when in public?” in an on-going Facebook online survey ran on millions of people in the US. We therefore use their weighted state-level estimates of mask adherence as our dependent variable. Their weighting strategy to obtain representative samples is described briefly in the Data section, and comprehensively described in their technical report [59].

During the period starting September 8 2020 (when adherence data started being available), only 4 states have enacted new state level requirements: Hawaii and Iowa on November 16 2020, North Dakota on November 14 2020 and New Hampshire on November 20 2020. Although some of these states had county-level mask mandates, had memos encouraging mandates at county or state-level, or had mask requirements in certain businesses, they did not have a state-wide requirement with implementation until the above dates. We run a similar event study estimation as earlier but this time using mask adherence as the outcome variable and for 13 days after mandate introduction (because this is the longest time horizon we have simultaneously for all four states).

As shown in Fig 4, there is a flat pre-treatment trend up to 8 days before the introduction of a mask mandate followed by an uptick in mask adherence after the introduction of mask mandates with a maximum increase in adherence of 23.4 [12.0,34.8] percentage points (adjusted R² is 0.925 and p < 0.001) 13 days after mask mandate introduction. We observe a delay of about 4 days before we observe an upward trend in mask adherence which is consistent with behavior change literature [29, 31].

Fig 4. Effect of mask mandates on mask-adherence.

Treatment effect of mask mandates on mask adherence in the only four states (Hawaii, Iowa, North Dakota and New Hampshire) which had late mask mandates (Nov 14, 16, and 20) during the period we have daily state-level mask adherence data. We find that there is a flat pre-treatment trend up to 13 days before the introduction of a mask mandate followed by a 23.4 [12.0,34.8] percentage points increase in mask adherence following introduction (adjusted R² is 0.925 and p < 0.001).

As a robustness test, we investigate if our estimation changes significantly if we remove testing rate, and cases and deaths as controls which we included as a confounders because, perhaps, the more case and deaths people see, the more they might be likely to wear masks. As seen in S7 Fig in S1 File, our estimation is robust to these controls. Unfortunately, we cannot replicate this result at the county level because no county mask mandate datasets currently exist with county-level mask mandates starting during the period that mask adherence data is available. Our results are consistent with previous work [53] although they do not control for time and unit fixed effects which prevents the disentanglement of people already increasing their mask wearing irrespective of the mask mandate.

Although we only estimate the effect of mandates on adherence in these 4 states due to the fact that they are the only ones that implemented mask mandates late enough that we have adherence data for them, we expect this result to support our main result that mask mandates are associated with a significant decrease in COVID-19 outcomes, especially due to the fact that these four states (Hawaii, Iowa, North Dakota and New Hampshire) are very geographically, culturally and politically diverse.

As an additional result, because we have a limited window of time where we have both COVID-19 outcomes (only cases and deaths, no hospitalization data as access ended in September 2020) and mask adherence data, we can use it to investigate if mask adherence is more directly associated with a positive effect on COVID-19 outcomes. To do so, we implement a multi-linear regression of COVID-19 outcomes on compliance, controlled, as before, by a number of factors as specified in Eq 3. Note that we use all states (including D.C.) here, not just the four states that had mandates after Sep. 8th. As shown in S16 Table in S1 File, an increase of 1% in mask adherence leads to a decrease of -1.63 [-1.76, -1.50] new confirmed cases per 100K (adjusted R² is 0.502 and p < 0.001), and -0.016 [-0.015, -0.018] new deaths per 100K (adjusted R² is 0.343 and p < 0.001), controlling for test rates, weather and mobility. For reference, the rate of COVID-19 outcomes was 25.4 new daily cases per 100K and 0.95 daily new deaths per 100K, suggesting that even a 1% increase in masking adherence can have a significant positive effect.

Lifting mask mandates

Previously, we estimated the positive effect of the introduction of mask mandates on COVID-19 outcomes. In the last 3 months, 6 states have lifted their state-level mask mandates: North Dakota on January 18 2021, Iowa on February 7 2021, Montana on February 12 2021, Texas on March 10 2021 and Wyoming on March 16 2021, Arkansas on March 30 2021. In this section, we estimate the associated effect of lifting mask mandates on, first, mask adherence, and, second, on COVID-19 outcomes.

An additional control of this study will be daily vaccination rates for each state normalized by population. This control is included both to control for its effect on COVID-19 outcomes—as we want to estimate the marginal effect of masks on outcomes controlling for the benefit due to vaccines—and to account for behavior changes: perhaps people wear masks less following getting vaccinated. Since vaccination data are only available since relatively recently, we consider the period between January 1 2021 and March 30 2021 for this section. Vaccine data were obtained from the Our World in Data initiative [60].

Mandate lifting on mask adherence

Using the same public data source (CMU Delphi project [58]) as in the previous section for daily state-level mask adherence values, here we estimate the associated effect of the lifting of mask mandates on mask adherence.

As shown in Fig 5 below, we observe a generally flat pre-treatment period during the 25 days prior to the lifting of the mask mandate indicating that mask adherence did not significantly decrease before the lifting of mask mandate. After the lifting, we observe mask adherence to be stable up to almost 30 days, but then a statistically significant decrease of up to -3.19 [-5.85,-0.53] percentage points 40 days after the lifting.

Fig 5. Effect of lifting mask mandates on adherence.

We observe a flat pre-trend before the lifting of state-level mask mandates (red vertical line at zero) followed by a statistically significant decrease. Full regression results in S15 Table in S1 File.

Although noisy due to limited data from the very recent states mandate lifting, this result suggests that the lifting of mask mandates resulted in a decrease in mask adherence in these 6 states. Note that this result includes controlling for previous controls (e.g. cases and deaths, testing rate, mobility, weather) and a new control for vaccination. A robustness test without these controls is included in S8 Fig in S1 File.

Mandate lifting on COVID-19 outcomes

Here, we reproduce the analysis from the first section, but this time examining the effect of lifting mask mandates on COVID-19 outcomes (e.g. other NPIs, mobility, test rate, mobility and delayed outcomes and growth), and introducing vaccinations as a control.

As can be seen from Fig 6, the lifting of mask mandates has a statistically significant effect on daily new cases: we observe a flat pre-treatment up to 45 days before the lifting, followed by an increase in cases after a delay of about 9–15 days, which is consistent with the behavior change literature [29, 31]. We observe a maximum increase in cases of 0.55 [0.41,0.71] standard deviations 45 days after the lifting. In real terms, this corresponds to an increase of 12 cases per 100K, which amounts to 13% of the highest recorded number of daily new cases (91 per 100K) in our observation period. Although noisy due to limited data from the very recent states mandate lifting, this result suggest that the lifting of mandates is associated with an increase in cases.

Fig 6. Effect of lifting mask mandates on COVID-19 outcomes.

We observe a significant increase in daily new cases, but no effect on hospitalizations and deaths. Full regression results in S12 Table in S1 File.

On the other hand, we do not observe a statistically significant effect associated with mask mandate lifting on hospitalization proportions and deaths. This could be due to a number of factors: noise due to limited data from the very recent states which mandated lifting, positive effect of vaccines on the more severe, downstream hospitalization and death effects of the virus [61, 62], or symptom management and relief practices [63–65].

International 69-country mask adherence and attitudes

We expand the scope of our analysis and look at the effect of mask adherence and attitudes towards mask wearing internationally. We do so by using the novel survey-based dataset ‘COVID-19 Beliefs, Behaviors & Norms Survey’ which asks more than 150 questions (see [39] for the complete survey instrument) about COVID-19 to 847 thousand online Facebook survey respondents in 51 languages from 69 countries. Because the 69 countries we include in our analysis are from different parts of the world and were in various stages of the pandemic (first wave, second waves, before mandates, after mandates, etc.) during the 18 waves over 9 months of observation, we expect that our estimates are unbiased by such confounders.

We focus on the two questions most relevant to the effect of masks on COVID-19 outcomes: a question about mask adherence: “Out of 100 people in your community, how many do you think wear a face mask or covering when they go out in public?”; and a question about mask norms: “Out of 100 people in your community, how many do you think believe the following because of COVID-19: People should wear a face mask or covering when out in public?”. The first question is a self-reported sample estimate of the percentage of a respondent’s community that wears masks in public, and the second question provides a sample estimate of the percentage of the community that believes masks to be important to wear in public.

We use a weighted survey regression approach to regress each country’s new daily deaths per 100K and cases per 100K (both smoothed) against the self-reported mask adherence and mask attitude with countries as survey strata and individual anonymized survey responses as survey clusters. Samples are expected to be representative because we use a unique weight for each sample which correct for a variety of biases including demographics (age bracket and sex) of the respondents compared to the census data in each country and compared to Facebook’s online population (through post-stratification), and for response and non-response drivers (the estimated design effects was below four as detailed in [39]). The dataset’s weighting strategy to obtain representative samples is described more in the Data section, and comprehensively described in their technical report [39]. We also control for mobility, new test rate, population density and the country’s human development index. The regression specification is described in Eq 4.

As shown in Fig 7, the associated effect coefficients suggest that a 1% increase in mask adherence is associated with a decrease of -0.45 [-0.70,-0.29] cases per 100K and a decrease of -0.042 [-0.046,-0.037] deaths per 100K. Similarly, we find that a 1% increase in attitude about the importance of wearing masks leads to a decrease of -0.53 [-0.64,-0.42] cases per 100K and a decrease of -0.035 [-0.038,-0.032] deaths per 100K. For reference, the current statistics in the world as of December 14, 2020 is 10 daily new cases per 100K and 0.17 daily deaths per 100K, which suggest that even a 1% increase in mask adherence and attitudes is associated with a strong positive impact on COVID-19 outcomes worldwide.

Fig 7. Impact of community mask adherence and attitudes.

Using a novel survey-based dataset of 847 thousand online Facebook survey respondents in 51 languages from 69 countries, we estimate that a 1% increase in community mask adherence is associated with a decrease of -0.61 [-0.82,-0.39] cases per 100K and a decrease of -0.035 [-0.040,-0.029] deaths per 100K. Similarly, we find that a 1% increase in community attitude about the importance of wearing masks leads to a decrease of -0.56 [-0.80,-0.32] cases per 100K and a decrease of -0.036 [-0.042,-0.030] deaths per 100K. For reference, the current COVID-19 outcomes in the world as of December 14, 2020 is 10 daily new cases per 100K and 0.17 daily deaths per 100K, which suggest that mask adherence and attitudes are associated with a strong positive impact on COVID-19 outcomes worldwide. Samples are expected to be representative because we use a unique weight for each sample which corrects for a variety of biases.

As a robustness check, we perform the same weighted survey regression but this time dis-aggregating by survey waves. As shown in S12 Fig in S1 File, the regression coefficients of mask adherence and attitudes on deaths and cases are negative or indistinguishable from zero, which supports our results that mask adherence and attitudes have a positive impact on COVID-19 cases and deaths across waves.

Discussion

We extend previous studies by investigating an unprecedented breadth and depth of outcomes, geographical resolutions, types of mask mandates, early versus later waves, and control variables such as mobility, weather, test rate, and other government interventions (such as stay-at-home orders). We find that, overwhelmingly, mask mandates are associated with a decrease in cases, hospitalizations and deaths. Additionally, we observe that mask mandates are associated with an increase in mask adherence in the 4 states where adherence data is present before and after the introduction of a mask mandate. Using a novel dataset of surveys from 69 countries across a variety of pandemic phases and pre/post mask mandate stages, we also observe that community mask adherence (how many people in a community wear masks) and community mask attitudes (how many people in a community believe that masks are an important for COVID-19) are associated with a decrease in cases and deaths.

Although different studies report different metrics and use different methodologies over different timescales, our results are comparable to previous work on the effect of masks. One influential study which also uses an event study design observes a 2.0 percentage points decrease [8] in cases following state mask mandate introductions. Their methodology is different in that they calculate changes in cases relative to a reference period while we first Z-normalize and population-scale our outcomes and then compare them relative to a reference period to prevent some states such as New York that had a much higher COVID-19 incidence than other states to dominate estimates. Over the same time period as them, we observe a decrease of -0.31 standard deviations [-.54,-0.07] equivalent to 8% [2,15%] of the highest recorded numbers. The range of our estimate is higher perhaps due to the fact that we analyze a longer time period which had far higher cases: they examine cases between March and May 2020 while we examine cases between February and the end of September 2020. Two other influential studies estimated the changes in slope before and after mask mandates and observe a net slope change of approximately 0.19 cases per 100K per day between counties with and those without mask mandates [9] and 1.65% before and after implementation of universal masking [7]. In comparison, we observe a corresponding decrease of 0.09% [0.07,0.11%] cases per day. Our estimates might be lower due to the fact that they do not control for a range of variables that almost certainly affected their observed decreases (and perhaps led to overestimates) such as decreases in mobility and implementation of other NPIs. Finally, one study observes mask mandates to have led to a decrease in cases between 15% and 75% in Germany [10]. We observe mask mandates in the USA to lead to a decrease of 14% of highest recorded number. Their higher estimates might be explained by the much higher and faster adherence of masks in Germany that the authors observed.

Even though our results, taken together, strongly suggest the positive effect of mask mandates, mask adherence and mask attitudes on COVID-19 cases, hospitalizations and deaths, an important open question is the investigation of the optimal incentives and regulatory mechanisms to increase mask-adherence [22, 66–70] especially in a climate where conspiracy theories are emerging to discourage mask-adherence [71]. More research is also needed to probe improvements in public health communication strategies to encourage mask-adherence, including harm-reduction frameworks [72] and policies tailored to help disadvantaged communities’ lived experiences [73]. Lessons can be taken from successful approaches in other public health crises such as HIV [74–77].

Such research into incentives, regulation, and messaging is especially important in light of our observation that the lifting of mandates is associated with a decrease in mask adherence and in increasing cases. An on-going debate is whether people would have voluntarily started wearing masks without requiring masks [78] and if mask wearing would voluntarily continue even in the absence of mask mandates. Our results suggest that mask mandates were effective and should be maintained.

Our results suffer from the usual limitations of econometrics-based studies: although we control for many confounders such as other NPIs (stay-at-home and non-essential business closures), mobility, weather and test rate, it is possible that unexplained exogeneity and endogeneity persists. For example, it is possible that our treatment effects are underestimates due to the peer effects caused by neighboring states’ mask mandates affecting other states [47]. Once outcome data disaggregated by demographics (e.g. age, race, gender) is available at the state or county level, an interesting question for future work is to investigate the differential impact of mask mandates over these demographic groups. In our study of mask mandates on mask adherence, ongoing efforts using computer vision to estimate mask adherence from social media images and videos [49–52] should soon provide more detailed data on mask adherence, hopefully starting from the beginning of the pandemic. Another limitation is that although our survey-based sample estimates of mask adherence and mask attitudes in the 69-country dataset include weights to correct for a number of biases, it is possible that there is some bias remains although this is unlikely due to their small estimated instrument design effect [39].

We leave to future work the important design and estimation around these limitations but expect that our findings will still provide new insights into the long-term effect of mask mandates on mask adherence, COVID-19 cases, hospitalizations and deaths, and introduces the importance of community mask adherence and community mask attitudes.

Our findings have policy implications for reinforcing the need to maintain and encourage mask-wearing by the public. These results are especially significant in light of some states starting to remove their mask mandates and attempts by state governments to prevent local governments from implementing mask orders, even though they are associated with a minimal economic impact [26]. Generally, our results are relevant to the ongoing effort to determine how to best design, implement and sustain the adoption of non-pharmaceutical interventions (NPIs) to curb the spread of the COVID-19 pandemic [79–84].

Materials and methods

We obtain data from a large variety of publicly available sources which we detail in the Data section below. We collected no data for this study. The study was approved under MIT’s Committee on the Use of Humans as Experimental Subjects (IRB) protocols E-2578 and E-2859.

The Analysis section next describes how we produced our three main results. To ensure the reproducibility of our results, we have shared the code and data used to perform data processing, analysis and plotting at https://github.com/d-val/Mask_Mandates_Adherence_Attitudes.

Data

Mask mandate start and end dates and types

To identify the start and, if any, the end of state-level mask mandates, we used two publicly available state-level mask mandate datasets [85, 86] and followed their references to official state government memos to perform accuracy checks. We found the datasets to be highly accurate even though the specifics of how and when mandates were instituted varies strongly between states. We investigated three specifications of mandates to use as our treatment: mandates requiring only business employees to wear masks, mandates for the general public, or whichever came first between mandates for business employees and the public. For the state-level analysis, we select only mandates that are at the state-level that require mask use as opposed to county-level mask mandates or using mandates that merely encouraging people to wear masks. The table of mask mandate start and end dates is shown in S3 Table in S1 File.

In order to investigate county-level mandate effect heterogeneity within a state, we investigate the effect of county-level mask mandates. We use a publicly available dataset [46] which we similarly verify for accuracy.

Control NPIs: Stay-at-home and non-essential business closures intervention

Mask mandates were often introduced around the same time that other non-pharmaceutical interventions (NPI) such as stay-at-home orders and non-essential business closures were implemented. In order to estimate the effect of mask mandates without the bias of other NPI’s, we use the same public datasets [85, 86] to obtain the start and end dates of stay-at-home and non-essential business closures (the two broadest and far-reaching NPIs), which we also verify for accuracy.

COVID-19 cases, hospitalizations and deaths data

State and county-level COVID-19 outcome data was obtained from the CMU Delphi project [58]. Specifically, we use as outcomes the daily new confirmed cases per 100K population, new confirmed deaths per 100K population, percentage of new hospital admissions with COVID-associated diagnoses, and the percentage of outpatient doctor visits primarily about COVID-related symptoms.

It is important to note that we only observed the proportion of hospitalizations, which is the number of new hospitalizations due to COVID-19 relative to the number of total admissions. Unfortunately, this proportion is a possibly noisy proxy for the number of hospitalizations per 100K due to COVID-19 (that we would like to observe) as it is well documented that hospital capacity was deliberately increased to accommodate surges in the number of hospitalizations and that fewer people went to the hospital as regular admissions were discouraged and elective treatments postponed during the pandemic [87–90] which affects the denominator of the hospitalization proportion. However, even though possibly noisy, we investigate the effect of mandates on the proportion of hospitalization as it still provides a useful perspective of the effect of mask mandates on COVID-19 outcomes.

We use the smoothed outcomes measures using 7-day averages and with systematic day-of-week effects removed. Our outcome data cover all 50 states in addition to the District of Columbia and run from Feb 2, 2020 to September 27, 2020 (we cannot obtain data past September for some outcomes such as hospitalization). 43 states and the District of Columbia had a required state-level mask mandates during this period of data. For the county-level dataset, we have 857 counties for which we have outcome and control variable data. This is because county-level datasets are currently not available due to their higher resolution. However, our 857 counties cover a large portion of the country—77% of the US population as shown in S3 Fig in S1 File—and are therefore representative.

We make sure not to use any outcome data that was after a state or county’s mask mandate was lifted to prevent end-of-treatment effects from biasing our treatment estimates, and we only use the first mask mandate start and end dates in cases where there were multiple waves of mandates over the year. Overall, this dataset contains 10,078 data points used for the state-level event study (described in the next section) and 77,930 data points for the county-level event study.

Mask adherence data

State-level mask adherence data (how much people are wearing face masks) became available on September 8, 2020 from the Delphi project [58]. These data were collected by them via online surveys of more than 1 million U.S. residents and we use their weighted state-level estimates as our indicator of state-level mask adherence. Their survey weighing [59] accounts for various factors including adjusting estimates so that they are representative of the US population according to a state-by-age-gender stratification, adjusting for the US Facebook user population, and adjusting for the propensity of a Facebook user to take the survey. For the regression of the effect of mask mandates on adherence, we have 323 data points for the four states, and for the regression of adherence on COVID-19 cases and deaths, we have 3,841 data points for all states (including D.C.). County-level mask mandate data is only available until August 4, 2020 which is before mask adherence data starts (September 8, 2020), making the use and investigation of county level adherence data not possible.

International mask norm and adherence data

International mask norm and adherence data was obtained from the COVID-19 Beliefs, Behaviors and Norms Survey [39] which collected 847 thousand online Facebook survey responses in 51 languages, from 69 countries. The survey was deployed in 18 waves starting on July 7, 2020 and ending in March 2021. Each survey response comes with a weight that allows for representativeness of estimates at the country and wave-level. Weights are calculated to account for a variety of biases including demographics (age bracket and gender) of the respondents compared to the census data in each country (through post-stratification), and response or non-response behaviors. The reported design effects due to both the non-response and post-stratification weighting was below four. In our analysis, we only use survey responses where the full survey was completed (including the demographic part) and where the mask-related questions we use in our analysis were completed. Overall, we have 847,398 data points covering 18 waves of survey deployment. The survey dataset (de-identified) is available to academic and non-profit researchers upon completion of the Facebook Data Use Agreement at https://dataforgood.fb.com/docs/preventive-health-survey-request-for-data-access/.

Control variables

We control for a large number of variables in our analyses: we use restaurant and recreation, grocery and pharmacy, parks, transit stations, workplace and residential percentage change from baseline (using the median day-of-the-week value from the first 5 weeks of January and February) mobility data from Google’s COVID-19 Community Mobility Reports [91]; new tests per 100K population from the COVID Tracking Project [92], and temperature and precipitation data from NOAA’s Weather and Climate Toolkit [93]. For international mask norm and adherence, we obtain COVID-19 country-level confirmed cases and confirmed deaths from the Our World in Data initiative [60].

Because we also include the squared values of each mobility indicator as a proxy for social contacts [43, 44], we first re-scale the Google mobility index [91] as such: re-scaled mobility = (mobility + 100)/100. This is done to prevent the issue of Google’s the square of negative mobility indicator values to be the same as the positive values.

Analysis

Event study framework

We build on recent work using event study designs to estimate treatment effects in the context of the COVID-19 pandemic: the effect of NPIs (e.g. business closures and stay-at-home orders) on the volume of online searches related to unemployment [94], on social distancing in the US [95], on the airline industry [96], on stock markets worldwide [97–100], and on COVID-19 cases at the county level [101].

An event study design allows us to estimate the treatment effect associated with mask mandates on COVID-19 outcomes on each day following the introduction of the mandate relative to the day prior to its introduction. Our geographical unit of analysis are states to minimize peer-effects of neighboring counties’ mandates due to the underlying interdependence between county mobility patterns [47], as well as the fact that people living in one county often have to travel to a different county to get medical care, resulting in inconsistent accounting of COVID-19 health outcomes at the county level [48]. By contrast, patients rarely cross state borders to get medical help as health insurance in the US is predominantly state-based. We include a robustness check by running our analysis at the county-level using a dataset of county-level only mask mandates [46] to investigate county-level heterogeneity.

The event study framework is well-suited for our purposes as it allows for the same state to be used both as treatment and control based on the timing of a mandate, which is critical because states introduced mask mandates at different dates over the course of the pandemic [94]. While the first mandates were implemented in April, the latest ones occurred in November, some states still having had no mandate as of this writing (see S3 Table in S1 File).

Event study specification

Our specification is:

$$\begin{array}{c}\hfill C_{o,t,s}\sim \sum _{\gamma =-10}^{40}{\gamma }_{\tau }\times \mathbb{1}(r_{s,t}=\tau )+{\alpha }_s+{\alpha }_t+{\beta }_{s}t+X_{t,s}+C_{o,t-\delta ,s}+C_{o,t,s}^{\prime }+{ϵ}_{t,s}\end{array}$$ (1)

The outcome is C_o,t,s (o∈ {new deaths due to COVID-19, new confirmed cases of COVID-19, hospitalization proportion due to COVID-19}) for each day t in state (or county when investigating county-level mandates) s. We make sure to only use new cases, hospital admissions and deaths each day, as opposed to cumulative numbers. Because some states such as New York had a much higher COVID-19 incidence than other states, we used population-scaled (per hundred thousand people) outcomes, and we further Z-normalize the daily value of each cases and deaths using state-specific means and standard deviations (hospitalization is already a percentage and therefore does not require normalization). Due to these transformations, our treatment effect estimates are expressed in units of standard deviations. We also ensured that all our variables have been smoothed and de-trended (for day of the week variations). ϵ_t,s represent the model residuals.

The goal is to estimate treatment effects γ_τ where τ is the number of days before or after the start of a mask mandate. $\mathbb{1}$ is the delta function such that γ_τ is non-zero only when the number of days r_s,t relative to the start of mask implementation is τ. As is standard in an event study framework [94], we normalize γ_{τ = −1} = 0 which means that our estimated treatment effects are to be interpreted relative to the day before the introduction of a mask mandate. We set the range of days r_s,t before or after the introduction of mask mandates to be from -10 to 40 because choosing values earlier than -10 leads to some state no longer having enough data (as they implemented mandates very soon after COVID-19 outcome data was available) and 40 to account for reporting delays and the pathogenesis characteristics of the virus as the interquartile range of the time between symptom onset and death is 8–26 days [102–109]. The clock time interval for our data is February 1 to September 27, 2020, the time during which data for all outcomes and controls was available.

α_s and α_t are state and time fixed effects, β_s t is a state-specific trend coefficient multiplying the time trend variable to allow for the fact that different states had different outcome trajectories (see [110] for more details). We report standard errors clustered at the state-level.

X_t,s are our controls for a number of confounding factors: (a) indicators of human mobility [91] (number of visits to recreational areas, grocery stores, pharmacies, parks, transit stations, workplace and residential areas) to control for variations in the spread of the virus due to variations in mobility and as a proxy for other NPIs such as stay-at-homes (as they impact mobility); (b) daily new test rate [92] to control for variations in the number of tests being administered; (c) the start and end dates of stay-at-home and non-essential business closures (the two broadest and far-reaching NPIs) to estimate the effect of mask mandates without the bias of other NPI’s; and (d) temperature and precipitation [93] to control for weather-induced behaviors.

We also include a time-delayed (δ = two weeks) outcome C_o,t−δ,s and a growth rate (over the past two weeks) $C_{o,t,s}^{\prime }$ because conditioning on the delayed outcome and growth rate results in the confounding path due to behavior change to be blocked based on the general causal graphical model of [35]. A delay of two weeks was chosen based on both our sensitivity analysis and that the estimated behavioral lag in the estimation of the causal graphical model of [35]. Additionally, as will be shown in the results section, shorter or longer delays would be at odds with the pathogenesis characteristics of the virus [102–109]. As a robustness check, we explore the effect of including or not delays on the event study estimation in S1 Fig in S1 File.

While political affiliation has been shown to correlate the propensity to wear a mask [111], this effect should captured in the state fixed effect since we do not expect political affiliation to change significantly during our observational period.

We use the R package’s lfe’s regression function felm to run this specification as it uses the method of alternating projections to speed up regression on our large datasets [112].

As a robustness check, we repeat employ a similar event study specification to verify that the decrease in COVID-19 outcomes we observe is not due to a decrease in test rate (per 100K of population. We use the following specification:

$$\begin{array}{c}\hfill T{R}_{t,s}\sim \sum _{\gamma =-5}^{20}{\gamma }_{\tau }\times \mathbb{1}(r_{s,t}=\tau )+{\alpha }_s+{\alpha }_t+{\alpha }_s^{\prime }t+X_{t,s}^{\prime }+{ϵ}_{t,s}\end{array}$$ (2)

where TR_t,s is the daily new test rate, and all other variables are similar to above, with $X_{t,s}^{\prime }$ being our controls without test rate (since it is now the dependent variable).

Mask mandate and mask adherence

Using daily mask adherence data from September 8 to November 30, 2020, we estimate a similar event study specification to the above using mask adherence A_s,t as the dependent variable. Additionally, we control for deaths and confirmed cases because we want to control for the fact that the more case and deaths people see, the more they might be likely to wear masks (although we do not observe much of a difference when omitting them). Again, we control for a number of factors: testing rate due to its potential effect on mask usage as more people might wear masks if they are more regularly tested, and mobility as we want to control for the fact that perhaps the more people leave their house, the more they might wear masks.

We implement a multi-linear regression of COVID-19 outcomes on compliance, controlled by a number of factors, as specified below:

$$\begin{array}{c}\hfill C_{o,t,s}\sim A_{s,t}+X_{t,s}^{\prime \prime }+{ϵ}_{t,s}\end{array}$$ (3)

where C_o,t,s are the daily t COVID-19 outcomes o∈ {deaths, cases} and $X_{t,s}^{\prime \prime }$ are the mobility, new test rate and weather controls.

International mask adherence and norms

Here, we use multi-linear regression to estimate the effect of mask norms or mask adherence on COVID-19 case and death outcomes for each wave w:

$$\begin{array}{c}\hfill C_{o,t,k}\sim Q_{t,k}+X_{t,s}^{\prime \prime \prime }+{ϵ}_{t,s}\end{array}$$ (4)

where C_o,t,k are the daily t COVID-19 outcomes o∈ {deaths, cases} for each country k, for question q, controlling for mobility and new test rate $X_{t,s}^{\prime \prime \prime }$. One separate model estimation is run for each wave w, each outcome o and each question q∈ {mask norms, mask adherence}.

More details about survey weights and the dataset can be found in the section. We also verify that the weighted average mask adherence, weighted average mask attitude and total number of responses did not change significantly over time during the survey as shown in S9-S11 Figs in S1 File.

We use a complex survey regression (using R’s survey package) with countries as survey strata and individual anonymized survey responses as clusters. We only select responses that finished the whole survey (including several attention check questions), and we remove responses with missing values for the questions under consideration, and also those with missing weights (<1% of responses).

The 69 countries in our dataset are: United Kingdom, Poland, Italy, Germany, Japan, Argentina, France, Turkey, Mexico, Colombia, United States, Pakistan, Romania, Indonesia, Philippines, Egypt, Malaysia, Bangladesh, Nigeria, Brazil, Thailand, Vietnam, India, Netherlands, Azerbaijan, Australia, South Africa, Canada, Estonia, Senegal, Afghanistan, Tanzania, Angola, Ecuador, Georgia, Mongolia, Peru, Algeria, Mozambique, Bolivia, Portugal, Iraq, Cameroon, Morocco, South Korea, Uruguay, Honduras, Nepal, Sri Lanka, United Arab Emirates, Spain, Cote d’Ivoire, Myanmar, Chile, Venezuela, Guatemala, Trinidad & Tobago, Sudan, Kenya, Jamaica, Ghana, Uganda, Ukraine, Taiwan, Singapore, Cambodia, and Kazakhstan.

Supporting information

S1 File

(PDF)

Data Availability

Data and code are available on GitHub (https://github.com/d-val/Mask_Mandates_Adherence_Attitudes).

Funding Statement

The author(s) received no specific funding for this work.