COVID-19 Vaccination and Mental Health: A Difference-In-Difference Analysis of the Understanding America Study

Abstract

Introduction

Mental health problems increased during the COVID-19 pandemic. The knowledge that one is less at risk after being vaccinated may alleviate distress, but this hypothesis remains unexplored. This study tests whether psychological distress declined in those vaccinated against COVID-19 in the U.S. and whether changes in anticipatory fears mediated any association.

Methods

A nationally representative cohort of U.S. adults (N=8,090) in the Understanding America Study were interviewed regularly from March 2020 to June 2021 (28 waves). Difference-in-differences regression tested whether vaccination reduced distress (Patient Health Questionnaire 4 scores), with mediation analysis used to identify potential mechanisms, including perceived risks of infection, hospitalization, and death.

Results

Vaccination was associated with a 0.04-SD decline in distress (95% CI= −0.07, −0.02). Vaccination was associated with a 7.77–percentage point reduction in perceived risk of infection (95% CI= −8.62, −6.92), a 6.91-point reduction in perceived risk of hospitalization (95% CI= −7.72, −6.10), and a 4.68-point reduction in perceived risk of death (95% CI= −5.32, −4.04). Including risk perceptions decreased the vaccination–distress association by 25%. Event study models suggest that vaccinated and never vaccinated respondents followed similar Patient Health Questionnaire 4 trends before vaccination, diverging significantly after vaccination. Analyses were robust to individual and wave fixed effects and time-varying controls. The effect of vaccination on distress varied by race/ethnicity, with the largest declines observed among American Indian and Alaska Native individuals (β= −0.20, p<0.05, 95% CI= −0.36, −0.03).

Conclusions

COVID-19 vaccination was associated with declines in distress and perceived risks of infection, hospitalization, and death. Vaccination campaigns could promote these additional benefits of receiving the COVID-19 vaccine.

INTRODUCTION

On May 13, 2020, the UN warned that although coronavirus disease 2019 (COVID-19) was primarily an infectious disease, it was also sowing the seeds of a major mental health crisis.¹ Surveys in the U.S. reveal elevated levels of psychological distress, anxiety, and suicidal ideation since the onset of the pandemic.2, 3, 4, 5, 6 Several factors have contributed to these findings, including loss of income and work, food insecurity, social isolation, caregiving burdens, substance use, and racialized discrimination.7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 One recent study points to anticipatory fears, with perceived risk of infection and mortality explaining 20.7% of the increased distress between March 2020 and June 2020.¹⁹ A recent meta-analysis found that after peaking in March 2020–April 2020, population distress decreased to prepandemic levels by mid-2020.²⁰

Yet, despite population-level declines in distress, recent research points to persistent depressive symptoms among those reporting an accumulation of COVID-19‒induced stressors between March 2020–April 2020 and March 2021–April 2021.²¹ Many individuals may also continue to experience anticipatory fears associated with the pandemic, which have been shown to have adverse effects on mental health.^{19 , 22} As of July 4, 2021, cumulative deaths attributed to COVID-19 had surpassed 600,000 in the U.S., but only 67.1% of adults had received ≥1 dose of a COVID-19 vaccine. A consistent body of evidence now supports the safety of COVID-19 vaccines as well as their efficacy in preventing infection, serious illness, and death.²³ One critical gap in research is whether being vaccinated reduces COVID-19‒related anticipatory fears by reducing the risk of these severe outcomes. If vaccination alleviates fears associated with COVID-19, this may also have salutary spillover effects for mental health. Furthermore, these dynamics may be more pronounced among vulnerable groups whose communities have experienced disproportionate COVID-19‒related suffering and loss of life.

Are there individual and social benefits of vaccination beyond preventing infection? One hypothesis, so far unexplored, is whether vaccination improves mental health by reducing anticipatory fears of infection, hospitalization, and death. This study uses a difference-in-difference method with nationally representative longitudinal data to test whether vaccination for COVID-19 reduces psychological distress and, if so, whether lower perceived risk mediates this association. Finally, this study tests whether the effects of vaccination vary across sociodemographic groups.

METHODS

Study Sample

This study used data from the Understanding Coronavirus in America study,²⁴ an extension of an Internet-based, nationally representative longitudinal survey. A total of 28 survey waves were examined, between March 2020 and June 2021. Surveys were fielded every 2 weeks until February 16, 2021 (Wave 24), then every 4 weeks from Feburuary 17, 2021 (Wave 25) onwards. The primary analysis was restricted to those who participated in ≥2 survey waves with nonmissing values for perceived risk measures, mental distress, and covariates (N=8,090). Appendix Figure 1 (available online) shows a flowchart with these sample inclusion criteria, and Appendix Table 1 (available online) shows dates and sample sizes for each study wave. Appendix Table 2 (available online) shows the descriptive characteristics for never vaccinated and vaccinated respondents in the analytic sample. The analyses of deidentified secondary data are considered exempt from IRB approval according to the University of New Hampshire Office of Research, Economic Engagement and Outreach.

Measures

The primary exposure was vaccination status, coded as 1 beginning in the first wave in which the respondent answered yes to the question Have you gotten vaccinated for the coronavirus? and imputed as 1 thereafter. It was coded as 0 for no and imputed as 0 in each period before Wave 21 (December 23, 2020 to January 18, 2021), the first time this question was asked.

Psychological distress was assessed using the Patient Health Questionnaire-4 (PHQ-4) developed by Kroenke et al.²⁵ and validated by Löwe et al.²⁶ Scores on the scale range from 0 to 12, with higher scores indicating greater distress. The main analyses used total PHQ-4 scores standardized to have a mean of 0 and SD of 1. Indicator variables for moderate distress (PHQ-4 ≥6) and severe distress (PHQ-4 ≥9) were also used in supplementary analyses.²⁵

Participants were asked: On a scale from 0 to 100%, what is the chance that you will get the coronavirus in the next three months? Then: If you do get the coronavirus, what is the percent chance you will be hospitalized (spend at least one night in the hospital) from it? Finally, perceived infection–fatality risk was assessed by asking: If you do get infected with the coronavirus, what is the chance you will die from it? Responses were also recorded as 0%–100%.

In addition to individual and study wave fixed effects, the main analyses were adjusted for several time-varying, self-reported covariates. Specifically, we control for receiving Supplemental Nutrition Assistance Program benefits in the month before the survey, receiving unemployment insurance in the past 14 days, whether the respondent had been diagnosed with COVID-19, and employment status at the time of the survey.

Statistical Analysis

This study used 2-way fixed effects models to assess the association between receiving a COVID-19 vaccination and changes in mental health. These models take the form:

$${\gamma }_{it}=\alpha +Vaccinate{d}_{it}+{\mu }_i+W_t+X_{it}+{\epsilon }_{it},$$

where ${\gamma }_{it}$ denotes psychological distress for individual i at wave t; ${\mu }_i$ and $W_t$ are individual and wave fixed effects, respectively; and $X_{it}$ represents time-varying controls. Individual fixed effects adjust for all unobserved time-stable factors, whereas wave fixed effects removes secular changes that have the same effect on all units, such as economic shocks or national COVID-19 rates.²⁷ $Vaccinate{d}_{it}$ is an indicator variable that switches to 1 in the first wave in which the respondent answered yes to the question Have you gotten vaccinated for the coronavirus? and remains as 1 thereafter. To assess how the outcomes of interest changed over time before and after vaccination, this study uses event study models. Further details of these event study models are shown in the Appendix (available online). All analyses are weighted to produce nationally represenative estimates.

RESULTS

Appendix Figure 2 (available online) shows a gradient in the prevalence of reporting ≥1 dose of vaccination between December 2020 and June 2021 by age, income, and education, with higher rates of vaccination observed among older respondents and those at the higher end of the SES spectrum. Among racial and ethnic groups, Asian and Pacific Islanders had the highest rates of vaccination (78%), followed by Whites (64%), mixed-race respondents (58%), American Indian and Alaska Natives (AI/ANs) (58%), Hispanic individuals (59%), and Black respondents (54%).

Appendix Figure 3 (available online) shows the secular trends in distress scores for never vaccinated respondents and respondents who were vaccinated during the study period. Distress scores increased for both groups at the outset of the pandemic, peaking in Wave 2 (April 1, 2020–April 27, 2020), and then declined steadily until Wave 7 (June 10, 2020–July 6, 2020). Notwithstanding some fluctuations, distress scores remained relatively stable for both groups until Wave 26 (March 17, 2021–April 27, 2021), after which distress declined slightly for respondents who became vaccinated and increased slightly for never vaccinated respondents. Wave 26 corresponds to the median wave of vaccination in this sample.

Appendix Figure 4 (available online) shows the secular trends in risk perceptions for never vaccinated respondents and vaccinated respondents during the study period. In this case, although respondents who became vaccinated exhibited slightly higher levels of risk perceptions for most of the study period, both groups shared similar trends until Wave 25 (February 17, 2021–March 29, 2021), corresponding to the wave before the median wave of vaccination. Risk perceptions then declined for vaccinated respondents, falling below the mean risk perception levels of unvaccinated respondents.

Table 1 shows the difference-in-difference estimates for the association between vaccination and perceived risk of infection in Model 1, perceived risk of hospitalization in Model 2, and perceived risk of death in Model 3. In this case, vaccination was associated with a 7.77–percentage point reduction in perceived risk of infection (95% CI= −8.62, −6.92), a 6.91-point reduction in perceived risk of hospitalization (95% CI= −7.72, −6.10), and a 4.68-point reduction in perceived risk of death (95% CI= −5.32, −4.04). These effects correspond to a 0.36-SD decrease in perceived risk of infection, a 0.29-SD decrease in perceived risk of hospitalization, and a 0.19-SD decrease in perceived risk of death.

Table 1.

Two-Way Fixed Effects Models With Perceived Risk Factors Regressed on Vaccination Status, April 2020–June 2021

Variables	Model 1: risk of infection,	Model 2: risk of hospitalization,	Model 3: risk of death,
	coefficient (95% CI)	coefficient (95% CI)	coefficient (95% CI)
Received vaccination (ref: no)	−7.77** (−8.62, −6.92)	−6.91** (−7.72, −6.10)	−4.68** (−5.32, −4.04)
Received UI (ref: no)
Yes	0.63 (−0.38, 1.63)	0.27 (−0.65, 1.19)	0.54 (−0.16, 1.25)
Unsure	2.04* (0.29, 3.79)	0.79 (−1.50, 3.08)	0.85 (−1.15, 2.86)
Received SNAP (ref: no)
Yes	0.48 (−0.58, 1.54)	0.07 (−0.94, 1.08)	−0.37 (−1.25, 0.52)
Unsure	−0.98 (−2.41, 0.44)	−1.19 (−2.66, 0.27)	−0.49 (−2.00, 1.02)
Currently working (ref: no)	0.78 (−0.42, 1.99)	0.11 (−0.93, 1.15)	−0.33 (−1.12, 0.46)
Diagnosed with COVID-19 (ref: no)	16.00** (12.71, 19.30)	0.98 (−1.49, 3.45)	−2.75** (−4.18, −1.32)
Individual fixed effects	Yes	Yes	Yes
Wave fixed effects	Yes	Yes	Yes
Cases, n	8,090	8,090	8,090

Note: Boldface indicates statistical significance (*p<0.05, **p<0.001).

SEs are clustered at the individual level. The study period covers April 2020–June 2021.

SNAP, Supplemental Nutrition Assistance Program; UI, Unemployment Insurance.

Table 2 shows the primary difference-in-difference analyses. In Model 1, receiving vaccination was associated with a −0.04 decrease in standardized distress scores (p<0.001, 95% CI= −0.07, −0.02), and this relationship remained unchanged after adjusting for time-varying covariates in Model 2. Model 3 adjusts for risk perceptions. A 10–percentage point increase in perceived risk of infection was associated with a 0.01 increase in standardized distress scores (p<0.001, 95% CI=0.01, 0.02), and similarly, a 10-point increase in perceived risk of death was associated with a 0.01 increase in standardized distress scores (p<0.05, 95% CI=0.00, 0.01). Perceived risk of hospitalization was not independently associated with distress in this model. Adjusting for risk perceptions in Model 3 reduced the coefficient for vaccination by 25% (β= −0.03, p<0.05, 95% CI= −0.05, −0.00).

Table 2.

Two-Way Fixed Effects Models With Standardized Distress Scores (PHQ-4) Regressed on Vaccination Status and Perceived Risk

Variables	Model 1,	Model 2,	Model 3,
	coefficient (95% CI)	coefficient (95% CI)	coefficient (95% CI)
Received vaccination (ref: no)	−0.04** (−0.07, −0.02)	−0.04** (−0.07, −0.02)	−0.03* (−0.05, −0.00)
Received UI (ref: no)
Yes		0.04 (−0.01, 0.09)	0.04 (−0.01, 0.09)
Unsure		0.06 (−0.06, 0.17)	0.05 (−0.07, 0.17)
Received SNAP (ref: no)
Yes		0.00 (−0.05, 0.04)	0.00 (−0.05, 0.04)
Unsure		−0.10* (−0.18, −0.02)	−0.10* (−0.18, −0.02)
Currently working (ref: no)		0.05 (−0.00, 0.11)	0.05 (−0.00, 0.11)
Diagnosed with COVID-19 (ref: no)		0.19** (0.13, 0.26)	0.17** (0.11, 0.24)
Risk of infection			0.01** (0.01, 0.02)
Risk of hospitalization			0.00 (−0.00, 0.01)
Risk of death			0.01* (0.00, 0.01)
Individual fixed effects	Yes	Yes	Yes
Wave fixed effects	Yes	Yes	Yes
Cases, n	8,090	8,090	8,090

Note: Boldface indicates statistical significance (*p<0.05; **p<0.001).

SEs are clustered at the individual level. Coefficients for perceived risk factors are expressed as a 10-percentage point increase. The study period covers April 2020–June 2021.

SNAP, Supplemental Nutrition Assistance Program; UI, Unemployment Insurance.

To test whether the vaccination–distress association is transmitted through risk perceptions, the suest command in Stata, version 16.1, was used to combine the variance–covariance matrices from models testing the association between vaccination and distress without mediators (total effect in Model 2) and then with mediators (direct effect in Model 3). The lincom command in Stata was then used to test the null hypothesis that the difference between the total and direct effects (the indirect effect) is equal to 0. This test indicated a statistically significant indirect effect of vaccination on distress by perceived risk factors (β= 0.013, p<0.001, 95% CI=0.010, 0.018). Taken together, these models suggest that receiving the COVID-19 vaccination reduces distress and that this effect is partially transmitted through declines in perceived risk of infection and death.

Event study analyses in Figures 1 and 2 provide additional support for the findings in Table 2. Figure 1 shows that vaccinated and never vaccinated respondents followed similar trends in distress before vaccination and that these diverged afterward, with vaccinated respondents experiencing significant declines in distress. The magnitude of the effect at the wave in which vaccination was first reported (β= −0.02, 95% CI= −0.05, 0.00) increased in magnitude until 4 waves after vaccination (β= −0.06, 95% CI= −0.11, −0.02), which suggests that the estimates in Table 2 underestimate the true effect. Figure 2 shows slight differences in risk perceptions between vaccinated and never vaccinated respondents in the pretreatment period, although these were stable over time. After vaccination, these trends diverged, with vaccinated individuals experiencing large reductions in risk perceptions than those who were never vaccinated in this sample. Intervals on the x-axis represent 2-week periods up to Wave 24 (February 2021), and 4-week periods from Wave 25 onwards. Thus, the effects of vaccination on risk perceptions and distress persist for at least 8 weeks.

Figure 1.

Difference-in-differences estimates of the association between receiving the COVID-19 vaccine and psychological distress.

Note: Each point estimate refers to the change in distress between vaccinated and never vaccinated individuals, compared with their baseline differential in the wave immediately before vaccination. Models control for individual and wave fixed effects, receiving SNAP benefits in the month before the survey, whether the respondent received unemployment insurance in the past 14 days, whether the respondent has been diagnosed with COVID-19, and employment status at the time of the survey. Intervals between waves reflect 2–week periods up to February 16, 2021 (Wave 24), and 4–week periods from February 17, 2021 (Wave 25) onwards.

PHQ-4, Patient Health Questionnaire-4; SNAP, Supplemental Nutrition Assistance Program.

Figure 2.

Difference-in-differences estimates of the association between receiving the COVID-19 vaccine and perceived risk factors.

Note: Each point estimate refers to the change in risk perceptions between vaccinated and never vaccinated individuals, compared with their baseline differential in the wave immediately before vaccination (reference line on the x-axis). Models control for individual and wave fixed effects, receiving SNAP benefits in the month before the survey, whether the respondent received unemployment insurance in the past 14 days, whether the respondent has been diagnosed with COVID-19, and employment status at the time of the survey. Intervals between waves reflect 2–week periods up to February 16, 2021 (Wave 24), and 4–week periods from February 17, 2021 (Wave 25) onwards.

SNAP, Supplemental Nutrition Assistance Program.

To facilitate interpretation of effect sizes, additional models estimated the effects of health behaviors (alcohol use and exercise) and perceived financial risk on distress in Appendix Table 3 (available online). In this case, a 1-day increase in alcohol use per week was associated with a 0.01-SD increase in distress (p<0.001), and a 10–percentage point increase in perceived risk of running out of money in the next 3 months was associated with a 0.03-SD increase in distress (p<0.001). The effect size for the association between vaccination and distress was thus slightly larger than a 10–percentage point increase in perceived financial risk and was comparable with a 4-day increase in alcohol use per week.

Supplementary analyses also tested whether the effects of vaccination were heterogeneous across sociodemographic groups, first using stratified analysis to plot coefficients (Appendix Figures 5–9, available online) and then by fitting models with statistical interactions between vaccination and each group. Statistically significant interactions were found between vaccination and race/ethnicity but not in other sociodemographic groups. In this case, the largest effect of vaccination was observed among non-Hispanic AI/AN respondents (β= −0.20, p<0.05, 95% CI= −0.36, −0.03), followed by non-Hispanic respondents who identified with ≥2 race categories (β= −0.16, p<0.05, 95% CI= −0.30, −0.01) and Hispanic/Latino respondents (β= −0.07, p=0.06, 95% CI= −0.16, 0.003).

Post hoc contrasts after models including interactions revealed statistically significant differences. Larger declines in distress after vaccination were observed for non-Hispanic AI/AN respondents (p<0.01), non-Hispanic mixed-race respondents (p<0.05), Hispanic/Latinos (p<0.05), and non-Hispanic White respondents than for non-Hispanic Asian and Pacific Islander respondents. Non-Hispanic AI/AN respondents also reported marginally larger declines in distress than non-Hispanic White (p=0.05) and non-Hispanic Black individuals (p=0.09). Coefficient plots for the effect of vaccination on distress stratified by race and ethnicity are shown in Appendix Figure 9 (available online).

Appendix Tables 4–8 (available online) show how the patterns described in the main analyses remained robust when restricted to the Understanding America Study national sample, restricting models to respondents aged ≥65 years, including state-by-wave fixed effects, and when using an indicator for moderate (PHQ-4 scores ≥6) and severe (PHQ-4 scores ≥9) distress as the dependent variable. With respect to categorical indicators of PHQ-4 scores, vaccination was associated a 0.8–percentage point reduction in moderate distress and a 1–percentage point reduction in severe distress. The latter estimate corresponds to a 25% relative decline in severe distress from baseline prevalence in the wave before vaccination.

Finally, Appendix Table 9 (available online) shows the characteristics of the national sample at Wave 1 versus the characteristics of respondents in their final wave in the analytic sample. As Appendix Table 9 (available online) shows, these characteristics remained highly similar, suggesting that any attrition across waves was random.

DISCUSSION

This study has several important findings. First, vaccination was associated with a 0.04-SD reduction in PHQ-4 distress scores, with this effect increasing to 0.06 SD by 4 waves after vaccination. Vaccination was also associated with an 8–percentage point decrease in perceived risk of infection, a 7-point decrease in perceived risk of hospitalization, and a 5-point reduction in perceived risk of death. Adjusting for risk perceptions decreased the association between vaccination and distress by 25%, and the indirect effect of vaccination on mental distress through these anticipatory fears was statistically significant. Event study models suggest that vaccinated and never vaccinated respondents shared similar trends in perceived risk and distress before vaccination and that these trends diverged significantly after vaccination.

This study also found important variations in the effect of vaccination on distress by self-reported race/ethnicity. The largest effect was observed among AI/AN individuals, who reported a 0.2-SD decline in distress after vaccination or 5 times the average effect of vaccination in this sample. This comparatively stronger effect of vaccination may be the result of several factors. Longstanding structural inequities rooted in federal neglect and marginalization contributed to heightened vulnerability to COVID-19 among Native American communities, resulting in disproportionate rates of COVID-19 infection, hospitalization, and death.^{28 , 29} The Indian Health Service and Native nations across the U.S. mounted highly effective vaccination campaigns in response to the crisis, with community leaders heavily involved in outreach, education, and vaccine distribution.³⁰ Scholars have noted that an emphasis on community well-being and protection likely contributed to high rates of vaccine uptake in many AI/AN communities.³¹ Given the disproportionate COVID-19‒related illness and death experienced by AI/AN individuals, the reduction of risk through vaccination, both at the individual and community level, may have provided a particularly salient sense of relief.

Much work remains to be done to promote vaccine equity and uptake in America. As of July 4, only 67.1% of U.S. adults had been vaccinated,³² falling short of President Biden's target of 70% by the same date, and daily vaccination rates have fallen sharply.³³ Although side effects and safety top the list of concerns of those not vaccinated, lack of information and access remain barriers for vulnerable individuals, particularly Black Americans.³⁴ Many people of color who are not yet vaccinated express vaccine hesitancy but also a high perceived risk from COVID-19 infection.35, 36, 37 Meanwhile, although the poor and those facing food and housing insecurity are less likely to be vaccinated, many, especially those with children, want a vaccine.^{36 , 38}

Limitations

As with all observational analyses, this study has clear limitations. First, measurements of both the distress and vaccination rely on self-report, which may be biased. Second, the web-based sample may not be truly representative of the U.S. population and individuals from under-represented racial and ethnic groups. Finally, the average effect of vaccination on distress in the overall sample was relatively small, which is not necessarily surprising given research showing persistent COVID-19‒related stress exposures from March–April 2020 to March–April 2021.²¹ These exposures may dampen the effect of vaccination for many individuals. Furthermore, vaccination is likely to have myriad positive spillover impacts on mental health that are not captured during this study period. Mental health may further improve as friends and family become vaccinated, as the economy rebounds, as community prevalence of virus falls, and as fewer people suffer major illness or death. The exposure in this study was receiving ≥1 dose of vaccine, and thus the mental health impacts of full vaccination may not yet be captured. It is thus noteworthy that this study found a statistically significant impact of vaccination on distress that persists and becomes stronger for at least 8 weeks after vaccination.

Notwithstanding these limitations, this study has several key strengths. First, this is the first study to find large reductions in anticipatory fears after COVID-19 vaccination, with beneficial spillover effects for mental health. Second, the main findings are robust to time-varying controls, individual and wave fixed effects, and several alternative modeling strategies. Importantly, the analyses were robust to state-by-wave fixed effects, ruling out confounding because of time-varying factors at the state level, such as rates of infection or policy implementation. Finally, vaccination rates in this sample mirror national estimates described elsewhere. In this sample, a total of 63% of adults had received ≥1 dose of a COVID-19 vaccine between December 2020 and June 2021. This maps closely to rates reported by the Kaiser Family Foundation published on June 30, 2021.³⁴

CONCLUSIONS

These results have important public health and policy implications. Murphy et al.³⁹ suggest that messages tailored to vaccine-hesitant or vaccine-resistant individuals could emphasize the personal benefits of vaccination against COVID-19. At a time when there are emerging evidence that prolonged and sustained lockdowns may pose risks to mental health,⁴⁰ it is especially important to renew efforts to effectively communicate the benefits, both physical and mental, associated with vaccination .

Beyond these individual benefits, eliminating barriers to vaccination for vulnerable groups must be a key priority. Reducing collective risk requires equitable distribution and access to vaccines both locally and globally. These efforts must be coupled with urgent action to promote equitable access to testing, improved ventilation in schools and workplaces, paid sick leave, and indoor masking. The rise of new COVID-19 variants is an ongoing reminder that no one is safe until everyone is safe.