The Immediate Effect of COVID-19 Policies on Social-Distancing Behavior in the United States

Abstract

Objective

Although anecdotal evidence indicates the effectiveness of coronavirus disease 2019 (COVID-19) social-distancing policies, their effectiveness in relation to what is driven by public awareness and voluntary actions needs to be determined. We evaluated the effectiveness of the 6 most common social-distancing policies in the United States (statewide stay-at-home orders, limited stay-at-home orders, nonessential business closures, bans on large gatherings, school closure mandates, and limits on restaurants and bars) during the early stage of the pandemic.

Methods

We applied difference-in-differences and event-study methodologies to evaluate the effect of the 6 social-distancing policies on Google-released aggregated, anonymized daily location data on movement trends over time by state for all 50 states and the District of Columbia in 6 location categories: retail and recreation, grocery stores and pharmacies, parks, transit stations, workplaces, and residences. We compared the outcome of interest in states that adopted COVID-19–related policies with states that did not adopt such policies, before and after these policies took effect during February 15–April 25, 2020.

Results

Statewide stay-at-home orders had the strongest effect on reducing out-of-home mobility and increased the time people spent at home by an estimated 2.5 percentage points (15.2%) from before to after policies took effect. Limits on restaurants and bars ranked second and resulted in an increase in presence at home by an estimated 1.4 percentage points (8.5%). The other 4 policies did not significantly reduce mobility.

Conclusion

Statewide stay-at-home orders and limits on bars and restaurants were most closely linked to reduced mobility in the early stages of the COVID-19 pandemic, whereas the potential benefits of other such policies may have already been reaped from voluntary social distancing. Further research is needed to understand how the effect of social-distancing policies changes as voluntary social distancing wanes during later stages of a pandemic.

In the absence of antiviral drugs and vaccines to contain the coronavirus disease 2019 (COVID-19) pandemic, social-distancing policies have been adopted by various affected countries.^1,2 These attempts have been made, largely, to keep the peak infection level below the resource capacity of health care systems and to buy time for possible drug and vaccine development.³

A decrease in the social contact rate during pandemic outbreaks is caused by a combination of voluntary actions by people and businesses driven by social awareness^4,5 and an array of nonpharmaceutical interventions (NPIs) implemented at the national, state, or local level. Social distancing played a substantial role in containing the first wave of the COVID-19 outbreak in China,^6,7 and evidence indicates the effectiveness of such policies in several European countries⁸ and some US states.^9-13 However, the relative effect of voluntary actions versus policy interventions on the decrease in the social contact rate is yet to be determined. Determining which interventions have the greatest effect on lowering the contact rate beyond what can be achieved via awareness mechanisms is important from a health policy perspective. An evaluation of social-distancing policies could provide valuable lessons to policy makers to respond efficiently to future pandemics or subsequent waves of COVID-19.

Although most countries have followed a central policy scheme during the COVID-19 pandemic, the United States left decisions about NPI to individual states, which created a natural experiment resulting from a high level of variation in the type, location, and timing of such policies^14-16 (Supplementary Figure S1 available at https://tinyurl.com/y3zyv2uj). Although evidence for reduced social contact in the United States is strong, not all decreases in social contact can be attributed to NPIs: mobility data show that people in most states had already started to reduce the time they spent outside their homes before any NPI was implemented (Figure 1; Supplementary Figure S2 available at https://tinyurl.com/y3zyv2uj). For some states such as Idaho, Missouri, Wyoming, and the District of Columbia, people’s presence at home had already increased and had reached a steady level before any social-distancing policies went into effect, which suggests the implementation of voluntary social-distancing activities. These trends suggest that attributing the entire reduction in social interaction to policy measures is misleading. Our study attempts to disentangle the direct effect of NPIs from voluntary decisions that are driven by factors such as changes in awareness about the disease and possible geographic spillover effects.

Figure 1.

Aggregate trend in presence at home relative to the start date of the first social-distancing policy implemented in each state during the coronavirus disease 2019 pandemic, using Google community mobility data, United States, February 15–April 25, 2020. The x-axis shows the number of days relative to implementation of the first social-distancing policy. The y-axis shows changes in presence at home relative to the baseline period (January 3–February 6, 2020). The vertical line indicates the day the first social-distancing policy went into effect in the state.

On the other hand, stricter policies such as stay-at-home orders may reduce the rates of COVID-19 spread,¹³ hospitalizations, and deaths.¹⁷ However, the underlying mechanisms of such effects are not fully understood, especially because the disease dynamic is simultaneously affected by other precautionary behaviors, such as the prevalence of wearing face masks, which has been changing since the beginning of the outbreak. To quantify the effectiveness of NPIs, we focused on the intended mechanism and studied their effects on reducing social interaction. We used daily state-level variations in adoption of the 6 most common intervention policies: statewide stay-at-home orders, limited stay-at-home orders, nonessential business closures, bans on large gatherings, school closure mandates, and restaurant and bar limits. We also used Google-released geolocation data for 6 categories of locations. We focused on the early stage of the outbreak (February 15–April 25, 2020), which is the period from the early rise of the outbreak until the date at which several states started reversing social-distancing policies.

Methods

Data Sources

We used publicly available Google-released aggregated, anonymized daily location data on movement trends over time by state, across 6 location categories from February 15 through April 25, 2020.¹⁸ The data illustrated how the frequency and duration of visits from several places and the length of stay changed relative to the baseline period, defined as the median value, for the corresponding day of the week, during the 5-week period January 3 through February 6, 2020. The data included mobility trends for 6 location categories: retail and recreation, grocery stores and pharmacies, parks, transit stations, workplaces, and residences.¹⁹ Because we used de-identified, publicly available data, institutional review board review was not required.

We collected data on COVID-19–related policies and their effective dates for all 50 states and the District of Columbia, beginning with the report of the first positive case of COVID-19 in the United States. Because of discrepancies in policy start dates among data sets available in third-party sources, we used the original documents issued by state governments, collected by the Kaiser Family Foundation,²⁰ to determine the type and date of each state policy. We considered the effective date as the first date on which the policy in question was in full effect. We used the dates on which policies were effective that are consistent with other published studies on the topic.^13,17,21 To control for the effect of temperature variation, humidity, and wind speed on human mobility and spread of disease, we constructed average daily temperature (in degrees Fahrenheit), humidity (in percentage), and wind speed (in miles per hour) for each state by aggregating daily data for the top 5 biggest cities in each state (supplementary material available at https://tinyurl.com/y3zyv2uj).

Statistical Analysis

We used a difference-in-differences methodology, which is a quasi-experimental approach used in social science to evaluate the effectiveness of policies. The methodology was first developed in its simple form in 1849²² to study the cause of the cholera outbreak in London, and the findings resulted in policy adoptions that played an important role in ending the outbreak. We compared the outcome of interest in states that adopted various COVID-19–related policies with states that did not adopt such policies, before (January 3–February 6, 2020) and after (February 15–April 25, 2020) these policies took effect.

The validity of this approach hinges on the assumption of parallel trends in the outcome of interest absent the policies, an assumption that we empirically tested using an event-study approach. The event-study analysis is a more flexible version of the difference-in-differences method, which breaks down the timing of the estimated policy effect by period for states that adopt the policy. It is also a more reliable method than the ordinary difference-in-differences method when prepolicy trends occur in the outcome variables. We used a linear regression model when investigating the effect of COVID-19 policies on mobility because the related outcomes were percentage changes in the duration of visits relative to the baseline. In the regression model, we controlled for average daily state-level temperature (F), humidity (%), and wind speed (miles per hour) in addition to the 6 COVID-19–related policies (statewide stay-at-home orders, limited stay-at-home orders, nonessential business closures, bans on large gatherings, school closure mandates, and restaurant and bar limits). We also controlled for fixed effect of the day (71 indicator variable) and state (50 indicator variable). We clustered standard errors at the state level to account for nonindependence of mobility measures in a given state over time.²³

In our regression analysis, we used a 2-tailed t test to conduct hypothesis testing, with significance set at α = .05. We conducted all analyses using Stata version 16 MP (StataCorp). We also conducted a number of supplementary analyses to ensure the robustness of results:

1. We included state-specific day-of-week variables (a variable for each day of the week interacted by state fixed effects) in each model.

2. We conducted a permutation test in which we dropped each state from the sample, one at a time, and then estimated the effect of each policy on the remaining states.

3. We tested the results with state-specific linear and quadratic trends.

4. We examined results of the effect of “early” and “late” statewide stay-at-home orders. We considered the orders early if they were adopted by March 26, 2020, and late if they were adopted after March 26, 2020.

Results

Effect of Policies on Human Mobility

On average, presence at home increased during the sample period (February 15–April 25, 2020) relative to the baseline period (January 3–February 6, 2020), whereas other mobility in other location categories, including retail and recreation, grocery stores and pharmacies, and transit stations, declined. Mobility in parks did not show any significant change (Table 1). In addition, the mean for statewide stay-at-home orders was smaller than the mean for other policies (except limited stay-at-home orders).

Table 1.

Google community mobility data related to social-distancing policies implemented during the COVID-19 pandemic (N = 3621 observations), United States, February 15–April 25, 2020

Variable	Mean % (SD)
Mobility data, by locationa
Presence at home	9.55 (9.05)
Grocery stores and pharmacies	−3.51 (13.84)
Parks	13.87 (35.69)
Retail and recreation	−19.95 (23.98)
Transit stations	−21.57 (25.52)
Workplaces	−23.72 (22.12)
COVID-19 policies (proportion of days)
Statewide stay-at-home order	0.35 (0.48)
Limited stay-at-home order	0.07 (0.26)
Nonessential business closure	0.50 (0.50)
Ban on large gatherings	0.50 (0.50)
School closure mandate	0.55 (0.50)
Limits on restaurants and bars	0.54 (0.50)
Additional covariates
Mean daily temperature, °F	45.62 (14.25)
Average humidity, %	63.82 (16.72)
Average wind speed, mph	9.15 (3.84)

Abbreviations: COVID-19, coronavirus disease 2019; SD, standard deviation.

^aMobility data are percentage changes in visits to the locations during the study period as compared with baseline (January 3–February 6, 2020).

Statewide stay-at-home orders significantly increased the measure associated with presence at home by 2.45 percentage points or 15.2% compared with the day before implementation of the statewide stay-at-home policy (Table 2). Limits on restaurants and bars had a positive effect on presence at home (1.38 percentage points or 8.5%), although the effect size was smaller than what was observed for statewide stay-at-home orders. We did not observe any significant effects for limited stay-at-home orders, bans on large gatherings, and school closures.

Table 2.

Effect of coronavirus disease 2019 social-distancing policies on community mobility, United States, February 15–April 25, 2020 (N = 3621 observations)^a

Variable	Location
	Presence at home	Groceries and pharmacies	Parks	Retail and recreation	Transit stations	Workplaces
Mean outcome variables 1 day before implementation of statewide stay-at-home orders, %b	16.2	−6.2	7.3	−36.9	−40.9	−40.5
Weather condition variables
Mean daily temperature	0.006 (0.013)	0.011 (0.017)	−0.009 (0.123)	−0.018 (0.028)	0.013 (0.033)	−0.011 (0.015)
Average humidity	0.048c (0.003)	−0.103c (0.006)	−0.966c (0.066)	−0.121c (0.007)	−0.109c (0.010)	−0.040c (0.005)
Average wind speed	0.069c (0.018)	−0.134c (0.037)	−1.837c (0.296)	−0.142c (0.039)	−0.141c (0.060)	−0.033 (0.031)
COVID-19–related policies
Statewide stay-at-home order	2.452c (0.351)	−6.850c (0.825)	−10.434c (4.870)	−4.652c (1.304)	−7.617c (1.594)	−5.342c (0.593)
Limited stay-at-home order	−0.552 (0.618)	−0.077 (1.825)	0.250 (5.746)	2.231 (2.283)	−0.255 (2.465)	0.419 (1.419)
Nonessential business closure	0.753 (0.394)	−0.270 (0.566)	−3.263 (6.726)	−1.264 (0.893)	−1.910 (2.058)	−1.124 (0.818)
Ban on large gatherings	−0.072 (0.269)	0.073 (0.976)	1.285 (2.692)	−0.030 (0.628)	0.044 (1.243)	−0.307 (0.648)
School closure mandate	−0.283 (0.325)	−1.374c (0.808)	4.578 (2.893)	−0.803 (0.797)	−0.315 (1.666)	0.432 (0.920)
Limits on restaurants and bars	1.382c (0.301)	−1.969c (0.740)	−11.874c (3.985)	−3.964c (0.876)	−6.908c (1.417)	−2.672c (0.641)
R 2	0.973	0.917	0.612	0.969	0.952	0.978

^aEach column reports regression coefficients from a linear regression model, weighted by state population in 2018. In addition to the listed variables, models control for state and day-of-the-month fixed effects for each regression. Standard errors (SEs) are clustered at the state level using a 2-tailed t test. All values are coefficient (SE), except where noted.

^bNegative means suggest a decline in those outcomes before implementation of that policy relative to the baseline (January 3–February 6, 2020).

^cSignificant at P < .05.

Of the coefficients for the 6 COVID-19–related policies, coefficients for statewide stay-at-home orders had the largest effect on reducing out-of-home mobility (2.45 percentage points; Table 2). Policies such as bans on large gatherings had a small and nonsignificant effect on keeping people at home (−0.07 percentage points) and a positive effect on presence at transit stations (ie, increased presence at transit stations; 0.04 percentage points). We also did not find any significant effect for school closures on presence at home.

Before statewide stay-at-home orders were implemented, no large and significant changes occurred in presence at home (Figure 2). However, immediately after implementation of stay-at-home orders, presence at home significantly increased, and this trend continued for the rest of the study period. Among other policies, except for limits on bars and restaurants and, to some extent, nonessential business closure, we found no effects on presence at home (Figure 3). We found no preexisting trends in presence at home before the adoption of policies such as limited stay-at-home orders, nonessential business closure, and school closure.

Figure 2.

Effect of implementation of statewide stay-at-home policy on presence at home during the coronavirus disease 2019 pandemic, using Google community mobility data, United States, February 15–April 25, 2020. Gray areas highlight 95% CIs. The x-axis shows the number of days relative to implementation of the first social-distancing policy. The y-axis shows changes in presence at home relative to the baseline period (January 3–February 6, 2020). The horizontal line indicates zero estimated coefficient.

Figure 3.

Effect of implementation of social-distancing policies on presence at home during the coronavirus disease 2019 pandemic, United States, February 15–April 25, 2020. Gray areas highlight 95% CIs. The x-axis shows the number of days relative to implementation of the first social-distancing policy. The y-axis shows changes in presence at home relative to the baseline period (January 3–February 6, 2020).

These results help us better interpret other results. For example, the estimated coefficient for nonessential business closure was small (0.75 percentage points; Table 2), which is consistent with the corresponding event study graph (Figure 3), suggesting a slight increase in presence at home 1 week after implementation of the policy. However, we found no significant downward or upward trends for statewide stay-at-home orders and limits on bars and restaurants before they took effect; trends in presence at home were flat before these policies took effect. We observed the largest effect on presence at home through statewide stay-at-home orders. The effect of limited stay-at-home orders was small and nonsignificant. We found no change in presence at home before and after implementation of the bans on large gatherings, suggesting that this policy was ineffective in changing people’s behavior (ie, staying at home). Moreover, event-study results showed a decline in activities outside of the home after implementation of statewide stay-at-home orders. However, we found evidence of a decline in presence at work and transit stations before these policies were implemented (supplementary information available at https://tinyurl.com/y3zyv2uj).

Including the state-specific day-of-week variables (a variable for each day of the week interacted by state fixed effects) in each model did not markedly change the results (supplementary material, https://tinyurl.com/y3zyv2uj). Moreover, the results were stable after the inclusion of linear and quadratic trends (supplementary information available at https://tinyurl.com/y3zyv2uj). The state exclusion permutation test shows that the effects were not driven by any particular state (Figure S4, available at https://tinyurl.com/y3zyv2uj). Finally, the analysis of the early versus late stay-at-home orders demonstrates that early orders (21 states) had a larger effect on increasing presence at home (3.89 percentage points, P < .001) than late orders (1.52 percentage points, P < .001; supplementary information available at https://tinyurl.com/y3zyv2uj).

Discussion

Our findings show the scale of effectiveness for 6 social-distancing policies on reducing out-of-home social interaction during the early stage of the COVID-19 pandemic. Reductions in out-of-home social interactions were driven by a combination of policy, as quantified by our analysis, and voluntary social distancing, as evidenced by changes in mobility before implementation of any social-distancing policies in many states. Our results indicate that during the early stages of the pandemic, much of the potential benefits of certain social-distancing policies in reducing human mobility had already been reaped by voluntary social distancing. Evidence for the dominant role of voluntary social distancing for certain policies is particularly evident from the event-study analysis graphs, which did not show any significant upward trends in presence at home after implementation of moderate social-distancing policies. It is, however, worth noting that although voluntary social distancing is evident from trends in geographic mobility, patterns of pretrends in the event-study analysis do not directly translate into evidence for voluntary social distancing in such graphs, because this approach cancels out the preexisting variations in presence at home that were common between states with and without social-distancing policies.

Results of our robustness tests provide further support for our main findings and demonstrate that the estimated effects of social-distancing policies were not driven by any single state and did not depend on any state-specific or day-of-week–specific trends in presence at home. In addition, we provide evidence that our results are valid for states with both early and late stay-at-home orders, although the magnitude of the effect was larger for states with early stay-at-home orders.

From a policy perspective, our results show that at the early stages of the pandemic, when social awareness about the disease outbreak was high, policies were implemented that substantially reduced human mobility beyond what could be achieved through voluntary measures. This finding is evident by the strong effect of statewide stay-at-home orders and the more moderate effect of limits on bars and restaurants and nonessential business closure. Furthermore, consistent with Dave et al,¹⁰ our findings indicate that early adoption of the stay-at-home order would be more effective in reducing mobility than later adoption of the stay-at-home order and that it matters when these policies take effect. Dave et al also studied a similar research question by using a similar method and SafeGraph mobility data. Their results suggest a comparable effect of stay-at-home orders on presence at home.

Limitations

Our study had several limitations. First, the Google database is not based on the universe of all smartphone users and only includes data on people who have enabled the Location History setting on their account. However, because most users keep their location services on,²⁴ our estimates are not likely underrepresented. Second, the data are imperfect because they do not include people who do not have smartphones and people who do not carry their cell phones to certain places. However, this factor should not affect changes in recorded behavior and likely had little effect on our results. Third, certain social-distancing policies were adopted within a short period of time in the early stages of the pandemic, a factor that made it harder to distinguish between the effect of one policy from another. Finally, no study of the effect of social-distancing policies is complete without quantifying other affected socioeconomic factors and associated tradeoffs involved. Although several recent studies have reported notable models and recommendations in this direction,^25,26 in the long run, and with the availability of more data, policy makers need more comprehensive studies on various factors that are affected by the pandemic and related social-distancing policies.

Conclusion

We studied the effectiveness of various NPIs by examining the effect they had on reducing out-of-home human mobility. Studying the effect of policies on intended mechanisms rather than disease dynamics is crucial because the disease dynamic is affected by a combination of factors related to social distancing and other precautionary measures, such as the prevalence of wearing a mask in various states. We demonstrated the significant effect of statewide stay-at-home policies on both measures and found that most of the expected effects of other social-distancing policies were already reaped from nonpolicy mechanisms such as voluntary actions.

Our results indicate the strong role of voluntary mechanisms in reducing human mobility at the early stage of the COVID-19 outbreak. However, we need to be cautious when generalizing the results from this early stage of the pandemic to the later stages and possible future waves of the pandemic. Specifically, our results do not suggest that lenient social-distancing policies such as school closures or bans on large gatherings are never causally effective in reducing social interaction. Although most of the social-distancing capacities of such policies were already absorbed in non–policy-driven mechanisms—possibly as a result of public awareness—it is expected that as the pandemic continues, voluntary social-distancing measures will start to wane, making such policies (individually or in combination) more effective in later stages of the pandemic than they were in the early stages of the pademic.^27,28

ORCID iD

Rahi Abouk, PhD https://orcid.org/0000-0002-8920-4055