Association between state stay-at-home orders and risk reduction behaviors and mental distress amid the COVID-19 pandemic

Abstract

The 2019 Coronavirus Disease (COVID-19) pandemic has triggered stay-at-home orders in 43 states since March 19, 2020. Evidence is limited on how these orders affect peoples' behaviors and mental distress. We used a nationally representative survey of 1094 American adults collected between March 19 and March 31, 2020 to compare risk-reduction behaviors and mental distress in states with and without orders.

Risk reduction behaviors included hand washing, wearing face mask and social distancing, and the mental distress was assessed by the four-item version of the Patient Health Questionnaire (PHQ-4). We predicted the probability of a person adopting risk reduction behaviors and the mental distress status using the number of days since their state issued the order relative to those in the non-order states, controlling for COVID-19 prevalence in the state, self-reported symptoms and demographic characteristics. The analysis was conducted in April 2020.

Results show that the probability of adopting risk reduction behaviors increased between 8 (avoidance of people with high risk, 87% to 95%) and 27 (use of face mask, 18% to 45%) percentage points in the response period. Mental distress increased by 1.0 point on the PHQ-4 score (from 2.4 to 3.5 point) in the first week and started to drop afterwards.

In summary, stay-at-home orders were associated with a differential increase in risk-reduction behaviors. People's mental distress rose in the first week under order and dropped afterwards. While in need for confirmation in longitudinal data, these results suggest that residents are responsive to orders.

Highlights

• •State stay-at-home orders are associated with increasing risk reduction behaviors.
• •The difference spills over to behaviors not included in the orders.
• •Orders are associated with mental distress that peaks within a week and then declines.

1. Introduction

Since the first confirmed case of the 2019 Coronavirus Disease (COVID-19) in the U.S. in Washington State on February 21, 2020, the Centers for Disease Control and Prevention (CDC) and other health authorities have promoted risk reduction behaviors. The early recommendations focused on reduced travel, frequent hand hygiene, cover one's cough and stay home if sick, and soon expanded to maintaining a physical distance of six feet or more apart in social or public context (Miller, 2020; Patel and Jernigan, 2020). As the number of confirmed cases and deaths increased in states such as Washington, New York, and California, state governments took more drastic measures.

California became the first state that issued an order to shut down non-essential public services and businesses, including schools, restaurants and bars, gyms, and stores, effective March 19 (Arango and Cowan, 2020), followed by other severely affected states, such as Illinois, New Jersey, New York, and Washington. By April 7, 43 out of 50 states and the District of Columbia had a similar stay-at-home or shelter-in-place orders in place (Mervosh et al., 2020). Though the details of the guidelines and the level of enforcement varied, they all entailed restrictions to maximize social-distancing: closure of non-essential business and services unless transformed to an online platform; restrictions on non-essential travel, as well as activities and social contact outside of one's residence.

While those orders sent a strong signal about the seriousness of the pandemic, it is not known how they have affected risk reduction behaviors. In addition, there is limited insight into its downside effects on stress and anxiety due to resulting job losses (Fowler, 2020), isolation and disruption of daily routines (Armitage and Nellums, 2020; Wright, 2020). In this study, we use cross-sectional data from a nationally representative survey to provide initial insights into the association between state orders and risk reduction behaviors as well as mental distress.

2. Methods

2.1. Participants

Our data came from participants in the Understanding America Study (UAS), a nationally representative panel housed in the Center for Economic and Social Research at the University of Southern California. The panel has been recruited using address-based sampling between 2014 and 2019, and includes U.S. residents aged 18 and older. Respondents answer survey questions on a computer, tablet, or smart phone, and received a tablet and broadband Internet if needed. Details on the panel have been documented in prior publications (Understanding America Study, 2020).

Between March 10 and March 31, all active English-speaking respondents were invited to participate in a survey on their perceptions and responses to the COVID-19 pandemic. Among 8502 who were invited, 6884 completed the survey for an overall completion rate of 81%. Our study focuses on the subset of 1094 respondents who participated during March 19 and March 31, the period during which one or more states had a stay-at-home order in effect. Because respondents took the survey at their own schedule and had no prior knowledge about the content of the survey, we consider them a random sample from all panel participants.

2.2. Survey instrument

2.2.1. Risk reduction behaviors

One primary outcome of interest is whether respondents had practiced preventive behaviors in the past seven days to keep themselves safe from the coronavirus in addition to what they normally did. The behaviors were partly derived from the prevention guidelines published by the CDC to combat the COVID-19 pandemic (Centers for Disease Control and Prevention, 2020), for instance, on hand hygiene and social distancing. Other behaviors include those people often practiced during the pandemic, though not necessarily included in the guideline (e.g., avoiding restaurants or stockpiling food). The behaviors used in the analysis include: wearing a face mask; canceling a doctor's appointment; washing hands with soap or using hand sanitizers several times a day; canceling or postponing air travel for work, or for pleasure; canceling or postponing work or school activities, as well as social activities; avoiding public spaces, gatherings, and crowds; avoiding contact with people who could be high risk; avoiding eating at restaurants or Chinese restaurants; working or studying at home; visiting a doctor; stockpiling food or water; and praying. Respondents answered yes, no or unsure to each behavior, and the low percentage of unsure responses were coded as negative answers.

2.2.2. Mental distress

Mental distress is another outcome of interest, assessed by the four-item version of the Patient Health Questionnaire (PHQ-4) that has been validated in the general population (Kroenke et al., 2009; Löwe et al., 2010). The instrument asks about the frequencies in the past two weeks of the following: feeling nervous, anxious, or on edge; not being able to stop or control worrying; feeling down, depressed, or hopeless; and having little interest or pleasure in doing things. Response options include not at all (0), several days (1), more than half the days (2), and nearly every day (3). While the first two items assess anxiety and the latter two assess depression, the sum score over all four items is considered to measure the overall mental distress.

2.2.3. Covariates

The survey asked about a broad range of physical symptoms the respondents experienced in the past seven days, such as cough, fever, shortness of breath and fatigue. While COVID-19 cannot be diagnosed based on symptoms alone, we considered a combination of self-reported shortness of breath and fever or chills, the two cardinal symptoms of the disease, as indicative of a respondent potentially having the disease.

Periodic data collection was conducted on the panelists' socio-demographic characteristics, including gender, age, race/ethnicity, education and household income. These variables were also used in sampling and recruitment, as well as creation of the weights against the overall U.S. population. Demographic data used in this study were collected between December 2019 and March 2020.

2.2.4. State order and COVID-19 prevalence

We used the New York Times' archive on state orders (Mervosh et al., 2020), and their time-series database on the total number of confirmed COVID-19 cases per state (The New York Times, 2020), which are both publicly accessible. To create the disease prevalence measure, we divided the number of confirmed cases by the one-year estimate of the population size based on the 2018 American Community Survey (U.S. Census Bureau, 2020). Because different state orders went in effect on different dates and the survey responses came in continuously across the fielding period, we constructed the number of days under state order and the corresponding prevalence for each state on the date a participant responded.

2.2.5. Statistical analyses

We used multivariate logistic regression to predict the probability of a person adopting a risk reduction behavior as a function of the number of days since their state issued the order relative to those in the non-order states. We controlled for COVID-19 prevalence in the state, self-reported symptoms and demographic characteristics. The standard errors were clustered at the state level to account for the additional dependency among those who resided in the same state. Because the analysis was performed on individual behavior, we used Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995) to adjust for the inflated false discovery rate due to multiple hypothesis testing. To study mental distress, we used multiple regression to predict the PHQ-4 score, with the same set of predictors and covariates. To capture the differences over the number of days under state order, we included both a linear term and the quadratic term to predict all outcomes. The quadratic term was used to capture potential nonlinear upward or downward trends. Analyses were conducted using STATA 16.

3. Results

Our sample consists of a total of 1094 adult Americans, who completed the survey between March 19 and March 31, 2020. As shown in Table 1 , more than half of the respondents were non-Hispanic white (58%) and female (61%), about three quarters were between 18 and 59 years old (77%), and less than half reported having a Bachelor degree or higher (40%). About half (51%) had a household income less than $60,000. We compared our analytic sample to the excluded sample who responded prior to March 19 and found the demographic composition to be similar except that our sample had more Hispanic and less non-Hispanic whites, and tended to be slightly younger. A comparison between included and excluded respondents can be found in the Appendix.

Table 1.

Characteristics of the survey sample responded between March 19 and March 31, 2020 (n = 1094).

	n (%)
Race/ethnicity
White	634 (58%)
Black	89 (8%)
Hispanic	261 (24%)
Other race/ethnicity	108 (10%)
Age
18–39	383 (35%)
40–49	243 (22%)
50–59	213 (19%)
60+	255 (23%)
Education
High school or less	266 (24%)
Some college	401 (37%)
Bachelor or higher	427 (39%)
Household income
<$30,000	271 (25%)
$30,000–$59,999	288 (26%)
$60,000–$99,999	266 (24%)
>$100,000	265 (24%)
Gender
Male	423 (39%)
Female	671 (61%)
State order
Yes	530 (48%)
No	564 (52%)

Among the analytic sample, about half (52%) resided in a state in which a state order to stay home was in effect at the time of their survey response. Although 31 out of 50 states and the District of Columbia had issued an order by March 31, we considered the rate reasonable as some responses were collected before their state orders went into effect. For those who were under state order at the time of response, the median days exposed to an order was 5 days with an interquartile range between 2 and 7 days.

Table 2 contrasts the outcome measures and the covariates by being under a state order or not. Those who had a state order were significantly more likely to adopt risk reduction behaviors, and had higher mental distress. They were also more likely to be race/ethnic minorities and younger. The state, in which they resided, tended to be more heavily impacted by the COVID-19, with an average of 96 confirmed cases per 100,000 population, compared to an average of 32 confirmed cases for those, who had no state order (p-value<0.001).

Table 2.

Comparison of respondents exposed and not exposed to a Stay-at-Home order at the time of survey (n = 1094).

	Without order (n = 564)	With order (n = 530)	p-value
Risk reduction behavior, n (%)a
Wear a face mask	88 (16)	169 (32)	<0.001
Wash your hands with soap or use hand sanitizer several times per day	527 (95)	512 (98)	0.012
Cancel a doctor's appointment	203 (37)	240 (46)	0.002
Cancel or postpone air travel for work	186 (34)	245 (47)	<0.001
Cancel or postpone air travel for pleasure	310 (56)	365 (70)	<0.001
Cancel or postpone work or school activities	280 (50)	337 (64)	<0.001
Cancel or postpone social activities	431 (78)	470 (90)	<0.001
Avoid public spaces, gatherings, or crowds	472 (85)	488 (93)	<0.001
Avoid contact with people who could be high-risk	460 (83)	480 (92)	<0.001
Avoid eating at restaurants	430 (77)	459 (87)	<0.001
Avoid eating at Chinese restaurants	229 (41)	243 (47)	0.071
Work or study at home	264 (48)	312 (60)	<0.001
Visit a doctor	74 (13)	72 (14)	0.854
Stockpile food or water	243 (44)	277 (53)	0.003
Pray	385 (69)	339 (65)	0.094
Race/ethnicity, n (%)
White	404 (72)	230 (43)	<0.001
Black	65 (12)	24 (5)
Hispanic	55 (10)	206 (39)
Other race/ethnicity	39 (7)	69 (13)
Age, N (%)
18–39	168 (30)	215 (41)	0.001
40–49	132 (23)	111 (21)
50–59	128 (23)	85 (16)
60+	136 (24)	119 (22)
Education, n (%)
High school or less	151 (27)	115 (22)	0.092
Some college	207 (37)	194 (37)
Bachelor or higher	206 (37)	221 (42)
Household income, n (%)
<$30,000	142 (25)	129 (24)	0.990
$30,000–$59,999	149 (26)	139 (26)
$60,000–$99,999	137 (24)	129 (24)
>$100,000	135 (24)	130 (25)
Gender, n (%)
Male	213 (38)	210 (40)	0.529
Female	351 (62)	320 (60)
Fever/chills and shortness of breath, n (%)b
Yes	<10 (1)	<10 (1)	0.420
No	>555 (99)	>520 (99)
Number of COVID-19 cases per 100,000 population in the state, median (IQR)	1 (0–4)	2 (0–5)	<0.001
Patient Health Questionnaire (PHQ-4), median (IQR)	14.73 (7.77–35.47)	47.62 (18.76–100.22)	<0.001

^aAssessed by the question: Which of the following have you done in the last seven days to keep yourself safe from coronavirus in addition to what you normally do?

^bAssessed by the question: Have you experienced any of the following symptoms in the past seven days?

The coefficient estimates on the quadratic trend term were not statistically significant in models predicting the probability of adopting a risk reduction behavior. Thus, we report the linear model here. Table 3 includes the estimated odds ratio on risk reduction behaviors corresponding to a one-day increase in time under state order, controlling for the COVID-19 prevalence in the state at the time of the response, self-reported symptoms and demographics. After Benjamini-Hochberg adjustment for an overall false discovery rate of 0.05, a total of 8 out of 15 behaviors were significantly increased when exposure to a state order increased by one day.

Table 3.

Adjusted odds ratio of changing risk reduction behaviors for each day of exposure to Stay-at-Home order and predicted probability (n = 1094).

	Adjusted odds ratioa			Predicted probability
	Estimate (95% CI)	p-value	Adjusted significanceb	No order	13th days under order
Wear a face mask	1.11 (1.09,1.13)	<0.001	*	18%	45%
Cancel a doctor's appointment	1.07 (1.04,1.10)	<0.001	*	37%	58%
Cancel or postpone air travel for pleasure	1.07 (1.03,1.10)	<0.001	*	60%	77%
Avoid public spaces, gatherings, or crowds	1.13 (1.03,1.24)	0.009	*	88%	97%
Avoid eating at restaurants	1.07 (1.02,1.13)	0.010	*	81%	91%
Avoid contact with people who could be high-risk	1.08 (1.02,1.16)	0.012	*	87%	95%
Work or study at home	1.04 (1.01,1.07)	0.016	*	52%	63%
Cancel or postpone social activities	1.10 (1.02,1.20)	0.020	*	82%	94%
Cancel or postpone work or school activities	1.04 (0.99,1.09)	0.098
Cancel or postpone air travel for work	1.03 (0.99,1.07)	0.219
Avoid eating at Chinese restaurants	1.01 (0.98,1.05)	0.337
Wash your hands with soap or use hand sanitizer several times per day	1.04 (0.95,1.15)	0.375
Pray	0.99 (0.97,1.02)	0.530
Visit a doctor	0.99 (0.96,1.03)	0.637
Stockpile food or water	1.00 (0.98,1.03)	0.849

^aOdds ratio was adjusted using multivariate logistic regression, for age, gender, race/ethnicity, education, household income, having symptoms of fever/chills and shortness of breath, and number of COVID-19 cases per 100,000 population in the state, with the standard errors clustered at the state level.

^bSignificance adjusted using the Benjamini-Hochberg procedure for an overall false discovery rate of 0.05.

One additional day under state order predicted a higher likelihood to adopt social distancing measures including cancellation of air travel for pleasure (adjusted odds ratio AOR = 1.06, 95% CI = 1.03, 1.10), cancellation of day-to-day social activities (AOR = 1.10, 95% CI = 1.02, 1.20), working or studying at home (AOR = 1.04, 95% CI = 1.01, 1.07), avoidance of public spaces or gatherings (AOR = 1.13, 95% CI = 1.03, 1.24), avoidance of eating at restaurants (AOR = 1.07, 95% CI = 1.02, 1.13), and avoidance of people who could be high risk (AOR = 1.08, 95% CI = 1.02, 1.16). These were consistent with stay-at-home recommendation by states. Longer time under state order was also associated with behaviors that were not specified in state orders, such as wearing a face mask (OR = 1.11, 95% CI = 1.09, 1.13) and cancellation of a doctor's appointment (OR = 1.07, 95% CI = 1.04, 1.10). These estimated odds ratios translate into cumulative odds ratios between 1.3 and 2.4 for a seven-day increase in exposure time. Fig. 1 demonstrates the monotonically increasing trend between the probability of adopting a behavior and the number of days under state order. The cumulative increase in predicted probability ranges from 8% to 28% across these behaviors.

Fig. 1.

Predicted probability of exhibiting risk reduction behaviors over time of exposure to Stay-at-Home order, with 95% CI.^a

^a Models were adjusted for age, gender, race/ethnicity, education, household income, having symptoms of fever/chills and shortness of breath, and number of COVID-19 cases per 100,000 population in the state, with the standard errors clustered at the state level.

The number of days under state order was predictive of the PHQ-4 score in linear and quadratic form, while also controlling for the disease prevalence, self-reported symptoms of having fevers and shortness of breath, as well as demographics. As shown in Table 4 , one more day under state order was associated with 0.26 (95% CI = 0.14, 0.37) point increase in the PHQ-4 score, which was compensated by a negative association in the squared day under order with −0.02 (95% CI = -0.03, −0.01) coefficient. Fig. 2 illustrates the joint interpretation of the linear and the quadratic terms. On average, one's mental distress increased when his or her state moved into a new phase under the stay-at-home order. The level of distress peaked in the first week of exposure, with a total increase of 1.0 point (from 2.4 to 3.4 point) on the PHQ-4 score and started to decrease afterwards.

Table 4.

Adjusted difference in the Patient Health Questionnaire (PHQ-4) score for each day of exposure to Stay-at-Home order (n = 1094).

Predictor	Adjusted difference (95% CI)a	p-value
Number of days under state order	0.26 (0.14,0.37)	<0.001
Squared number of days under state order	−0.02 (−0.03,-0.01)	0.001

^aDifference was adjusted using multiple regression, for age, gender, race/ethnicity, education, household income, having symptoms of fever/chills and shortness of breath, and number of COVID-19 cases per 100,000 population in the state, with the standard errors clustered at the state level.

Fig. 2.

Predicted score on Patient Health Questionnaire (PHQ-4) over time of exposure to Stay-at-Home order, with 95% CI.^a

^a Model was adjusted for age, gender, race/ethnicity, education, household income, having symptoms of fever/chills and shortness of breath, and number of COVID-19 cases per 100,000 population in the state, with the standard errors clustered at the state level.

4. Discussion

The current restrictions on public activities in response to the COVID-19 pandemic are unprecedented in the history of the United States. To our knowledge, this study provides one of the first assessments on how the statewide stay-at-home orders predicted risk reduction behaviors and mental health, which could provide useful insight in understanding in what ways state orders reduced COVID-19 mortality and rate of growth in hospitalization (Lyu and Wehby, 2020).

Several patterns emerge from the data. First, changes in social activities were more strongly associated with state orders than changes in work-related activities. Widely communicated risk reduction behaviors, such as avoidance of public spaces, social activities, restaurants, leisure travel and high-risk individuals showed the strongest associations with the time, during which respondents were exposed to a stay-at-home order. The probability of taking up such behaviors reached almost 100% after around two weeks under state order, corresponding to increases of 8 to 12 percentage points. Self-reported hand hygiene did not change but had already reached near-perfect compliance with recommendations in all respondents. Conversely, only 63% of respondents reported that they had started working from home. Cancellation of air travel for work and reduction of work and school activities was not associated with exposure to state orders. Presumably, many employers had already reacted prior to the state orders by eliminating non-essential travel, shifting work to home offices and furloughing or firing staff (Thorbecke, 2020). Second, even selected behaviors changed that were not explicitly spelled out in the stay-at-home orders at the time of the survey, such as a wearing face mask and canceling doctor's appointments. Lastly, behaviors with limited risk reduction effect, such as stockpiling food or water, visiting a doctor, and praying, were not increasingly adopted.

Parallel to changes in risk reduction behaviors, people reported elevated mental distress, which peaked in the first week under state orders and then declined towards baseline levels, whereas risk reduction behaviors continued to increase. This temporary increase in mental distress seems to suggest that people started coping with changed circumstances after experiencing a full weekly cycle.

5. Limitations

Our study provides initial evidence for the association between the state order and risk reduction behavior as well as mental distress. However, the findings are subject to a few limitations. First, our analysis was based on a subsample of the panel. Although the subsample was not selectively drawn from the full panel, the included and excluded portions differed slightly on race/ethnicity and age. In addition, the data only covered the first 13 days since the first state order came into effect. In this time window, many states barely stepped into the new status, and several states had not issued orders at all. Moreover, the assessment of the preventive behaviors referred to a one-week period prior to the survey time and the assessment of the mental distress referred to a two-week period prior to survey time, whereas the data collection ended less than two weeks after the first state order became effective. The fact that some of the assessed time windows were prior to the state order might attenuate the true effect when the observations would have been taken fully after the state order became effective. Furthermore, the data were based on a cross-sectional observation, and the difference over time was estimated based on average of different people. Future work to collect more data would allow a fuller observation on more states and people, and tracking people over time could create a true time series of change. Finally, the behavioral data were self-reported, which is inferior than direct observation. Studies using sensor-based data, such as the social distancing data collected by smartphone's GPS sensor (Dave et al., 2020), could complement our work and provide additional insights on people's behaviors.

6. Conclusions

Our results suggest that people had been increasingly practice risk reduction behaviors when their states had a stay-at-home order, especially with respect to social activities. Effects on mental distress seemed temporary.

CRediT authorship contribution statement

Ying Liu: Conceptualization, Data curation, Methodology, Formal analysis, Writing - original draft. Soeren Mattke: Conceptualization, Methodology, Writing - review & editing.

Appendix A.

Comparison of demographic composition between included and excluded samples (n = 6884).

	Included sample (n = 1904)	Excluded sample (n = 5790)
	n (%)	n (%)	p-value
Race/ethnicity
White	634 (58)	3873 (67)	<0.001
Black	89 (8)	446 (8)
Hispanic	261 (24)	866 (15)
Other race/ethnicity	108 (10)	586 (10)
Age
18–39	383 (35)	1737 (30)	<0.001
40–49	243 (22)	990 (17)
50–59	213 (19)	1137 (20)
60+	255 (23)	1926 (33)
Education
High school or less	266 (24)	1262 (22)	0.165
Some college	401 (37)	2152 (37)
Bachelor or higher	427 (39)	2376 (41)
Household income
<$30,000	271 (25)	1393 (24)	0.667
$30,000–$59,999	288 (26)	1472 (25)
$60,000–$99,999	266 (24)	1409 (24)
>$100,000	265 (24)	1503 (26)
Gender
Male	423 (39)	2429 (42)	0.043
Female	671 (61)	3361 (58)