Vaccination rates and COVID outcomes across U.S. states

Abstract

Rates of COVID deaths, hospitalizations, and cases differ markedly across U.S. states, as do rates of vaccination. This study uses cross-state regressions to assess impacts of vaccinations on COVID outcomes. A number of familiar issues arise concerning cross-sectional regressions, including omitted variables, behavioral responses to vaccination, and reverse causation. The benefits from a field context and from the broad range of observed variations suggest the value from dealing with these issues. Results reveal sizable negative effects of vaccination on deaths, hospitalizations, and cases up to early December 2021, although vaccine efficacy seems to wane over time. The findings for deaths apply to all-cause excess mortality as well as COVID-related mortality. The estimates imply that one expected life saved requires 248 additional doses, with a marginal cost around $55000, far below typical estimates of the value of a statistical life. Results since December 2021 suggest smaller effects of vaccinations on deaths and, especially, hospitalizations and cases, possibly because of diminished effectiveness of vaccines against new forms of the virus, notably the omicron variant. A further possibility is that confidence engendered by vaccinations motivated individuals and governments to lessen non-pharmaceutical interventions, such as masking and social distancing.

Vaccination rates against COVID-19 differ markedly across U.S. states. For example, based on data reported in Chetty, et al. (2022) and as shown in Table 1, the rate of “full” vaccination¹ over a recent period of roughly three months, 2/11/22–5/8/22, averaged 64% with a standard deviation of 9%. These rates varied from 51% in Alabama to 81% in Rhode Island. If vaccinations are effective at reducing infections and deaths, these differences should map into differences in COVID-related deaths, hospitalizations, and cases.

Table 1.

Means and Standard Deviations of Variables

Variable	Mean	s.d.	Natl Avg
COVID cumulative deaths per capita (change per year)
3/19/21–6/13/21	.00067	.00031	.00077
6/13/21–9/6/21	.00058	.00043	.00061
9/6/21–12/1/21	.00175	.00092	.00168
12/1/21–2/25/22	.00207	.00066	.00202
2/25/22–5/22/22	.00086	.00044	.00082
COVID cumulative excess mortality per capita (change per year)
3/20/21–6/12/21 (week-ending dates)	.00072	.00052	.00075
6/12/21–9/4/21	.00172	.00108	.00173
9/4/21–12/4/21	.00295	.00123	.00263
12/4/21–2/26/22	.00202	.00106	.00212
COVID cumulative hospitalizations per capita (change per year)
3/19/21–6/13/21	.0360	.0169	.0398
6/13/21–9/6/21	.0480	.0297	.0536
9/6/21–12/1/21	.0773	.0284	.0727
12/1/21–2/25/22	.1107	.0338	.1115
2/25/22–5/22/22	.0248	.0092	.0247
COVID cumulative cases per capita (change per year)
3/19/21–6/13/21	.049	.022	.049
6/13/21–9/6/21	.077	.044	.082
9/6/21–12/1/21	.134	.057	.110
12/1/21–2/25/22	.389	.069	.387
2/25/22–5/22/22	.056	.029	.054
“Full” vaccinations per capita
3/5/21–5/30/21 (data start 3/5/21)	.237	.031	.228
5/30/21–8/23/21	.450	.071	.448
8/23/21–11/17/21	.548	.079	.553
11/17/21–2/11/22	.607	.083	.614
2/11/22–5/8/22	.641	.089	.650
Booster vaccinations per capita
8/23/21–11/17/21 (data start 10/20/21)	.025	.008	.024
11/17/21–2/11/22	.199	.053	.196
2/11/22–5/8/22	.290	.075	.290
Fraction over age 25 with completed high school, 2019	.901	.027	.886
Population fraction 65 and older, 2020	.173	.020	.169
Life expectancy at birth, 2018	78.8	1.8	79.3
Population fraction black, 2020	.110	.101	.124
Urbanization rate, 2010	.741	.149	.809
Fraction of votes Republican, 2020 Presidential election	.492	.120	.469
Population fraction 75 and older, 2020	.068	.010	.067
Fraction over age 25 with completed college, 2019	.327	.065	.331
Per capita personal income ($1000s), 2020	57.7	9.4	59.6
Maximum temperature, December 1-February 25	25.2	12.7	51.0
Maximum temperature, September 6-December 1	66.2	8.9	69.7
Maximum temperature, June 13-September 6	84.7	6.5	85.0
Maximum temperature, March 19-June 13	69.6	8.1	71.6
Maximum temperature, February 25-May 22	63.0	9.3	66.0

Notes: COVID-related deaths, hospitalization, and cases are differences in cumulative values per person for dates shown (corresponding to cumulations of new deaths, hospitalizations, and cases), expressed at annual rates. Data for cumulative deaths and cases per person are from Chetty, et al. (2022). Values for deaths and cases are adjusted in accordance with n. 11. (Data for cumulative hospitalizations are given in the downloadable file provided in Chetty, et al. (2022). Adjustments in the hospitalization numbers do not seem to be necessary.) The changes in cumulative values were divided by state population in 2020. Full and booster vaccinations are averages per person over periods shown. The averages apply to dates at the start, end, and middle of each period, with the middle value getting double weight. Vaccination data are adjusted in accordance with n. 11. Maximum temperature is average high temperature in degrees Fahrenheit over dates shown. Underlying values are monthly for largest city in each state.

Table 1 shows that data on reported COVID-related deaths, hospitalizations, and cases also varied substantially across the states. For example, for 2/25/22–5/22/22 (14 days after the period used for vaccinations), the change in cumulative deaths per person—corresponding to cumulations of new deaths over the period—averaged 0.0009 with a standard deviation of 0.0004.² The range was from 0.0002 for the District of Columbia to 0.0020 for Kentucky. Over the same period, the change in cumulative hospitalizations per person averaged 0.025 with a standard deviation of 0.009 and a range from 0.010 for Wyoming to 0.051for Delaware. Cumulative cases per person averaged 0.056 with a standard deviation of 0.029 and a range from 0.017 for Nebraska to 0.127 for Vermont.

Table 1 shows comparable statistics for earlier periods. For vaccinations, the data start at 3/5/21, corresponding to the beginning of CDC information on full vaccinations.³ Each of the five periods considered covers roughly three months (86 days). Note that the mean of full vaccination rates rose from 0.24 in 3/5/21–5/30/21 to 0.45 in 5/30/21–8/23/21, 0.55 in 8/23/21–11/17/21, 0.61 in 11/17/21–2/11/22, and 0.64 in 2/11/22–5/8/22. The table also shows national averages, which differ to a minor extent from the means of values across the states.

For the COVID-related outcomes, the periods in Table 1 are each 14 days subsequent to the periods for vaccinations. Note that the mean of COVID deaths per person was 0.0007 in 3/19/21–6/13/21 and 0.0006 in 6/13/21–9/6/21, then rose to 0.0018 in 9/6/21–12/1/21 and 0.0021 in 12/1/21–2/25/22, before falling to 0.0009 in 2/25/22–5/22/22 (all measured at annual rates). Similar patterns apply over time for hospitalizations and cases. Note, however, that reported cases are particularly subject to measurement error because they depend on the volume of testing, which has varied substantially over time and across states.⁴

The objective of this study is to use cross-sectional regressions for the U.S. states to attempt to assess the effects of vaccinations on COVID-related outcomes. The regression framework takes as dependent variables the outcomes over the five periods shown in Table 1. That is, each dependent variable is the number of deaths or hospitalizations or cases per person cumulated over periods of roughly three months. The corresponding explanatory variables related to levels of vaccinations are averages over periods lagged 14 days compared to the dependent variables.⁵ The idea is that, at any point in time, the probabilities of infection, hospitalization, and death depend, with some lag, on the fraction of the population vaccinated.

The use of cross-sectional regressions involves a tradeoff between familiar econometric issues versus the large amount of information contained in the cross sections. A common procedure effectively discards the cross-sectional information by employing cross-sectional fixed effects (in the present case, state fixed effects) in the regressions. In contrast, the perspective of the present study is that the econometric issues can be handled satisfactorily, and the benefits from retaining the cross-sectional information make it advisable not to follow the common fixed-effects procedure.

The empirical framework can be viewed as a reduced form of SIR—susceptible-infected-recovered (and died)—models constructed by epidemiologists and, more recently, economists to study the evolution of contagious disease. This work began with Kermack and McKendrick (1927) and has been pursued recently by, among others, Atkeson (2020); Eichenbaum, Rebelo, and Trabandt (2021); and Acemoglu, Chernozhukov, Werning, and Whinston (2021). In this type of model, vaccination makes an individual less likely to catch the disease through contact with an infected person. Vaccination also lowers the likelihood of a person’s disease becoming severe, thereby lowering the probability of hospitalization, conditional on infection. Finally, vaccination lowers an individual’s probability of death, conditional on severe disease. At the community level, by lowering the overall rate of infection, a higher vaccination rate reduces the chance that an individual will be infected in a random encounter. The full equilibrium encompasses the direct inverse effects of vaccination on adverse outcomes along with the effects that work through contagion at the community level. Overall, a higher vaccination rate in a community associates with a reduction in the rates of infection (cases), severe disease (proxied by hospitalizations), and mortality. In the empirical analysis, the relevant community is taken to be a U.S. state, so that the spread of disease across state borders is neglected. In a planned extension, a community will be associated with a county, so that cross-border transmission will be more important.

I. Issues with Cross-Sectional Regressions

As is well known, inferences from cross-sectional regressions may be difficult to draw. Because of these problems, detailed below, many researchers have moved increasingly away from these types of regressions, preferring instead to rely on randomized control trials (RCTs) or natural experiments. Although RCTs are important for assessing the efficacy of vaccines, including those recently developed for COVID-19, it is more difficult to evaluate impacts on cases, hospitalizations, and deaths in the “field.” As far as I know, there are no RCTs applicable to field results connecting COVID vaccinations to COVID outcomes.⁶ In some cases, natural experiments—such as regression-discontinuity designs applied to state borders—have been used successfully in the context of COVID-19. For example, this approach has been applied to facemask mandates by Goolsbee and Syverson (2021), who consider economic impacts, and Hansen and Mano (2021a), who assess health outcomes.⁷

There are also important advantages of cross-sectional regressions. In particular, they apply to the field context and can exploit the large observed cross-sectional variations in the variables of interest, especially differences across U.S. states in vaccination uptake. Because of these major benefits, it seems worthwhile to pursue the cross-sectional regression approach in the context of COVID vaccinations and outcomes.

One issue in interpreting results from cross-state regressions is that vaccination take-up may be correlated with other variables that influence COVID outcomes. If these other variables are omitted from the regressions, the estimated coefficient on the vaccination rate may proxy for the influences of these other variables. For example, if older people are more susceptible to COVID infection and, especially, death, they are likely to be vaccinated more frequently (and earlier). In this case, the observed associations between vaccine take-up and COVID deaths, hospitalizations, and cases may be positive. This issue is handled by including as explanatory variables a set of major socio-economic variables—specifically, the fraction of the state population aged 65 and over in 2020, state life expectancy at birth in 2018, the fraction of the state adult population with education of four years of high school or more in 2019, the fraction of the state population classified by the U.S. Census as black in 2020, and the urbanization rate in 2010. To deal with possible seasonal effects, the analysis also includes differences in average temperature across states at different times of the year. Inclusion of some other variables—population share aged 75 and over in 2020, per capita personal income in 2020, population density in 2020, and college education in 2019—do not materially affect the results.⁸

Another issue is that persons vaccinated may alter their behavior in ways that impact probabilities of COVID infection, hospitalization, and death. For example, a vaccinated person may feel protected against infection and, especially, death conditional on infection and therefore react by engaging in more social interactions or other risky behaviors. A general discussion of this issue in the context of insurance is in Ehrlich and Becker (1972). Their ideas were applied to seatbelt use in Peltzman (1975), who argued that a person who used a seatbelt (perhaps because of a legal mandate) is likely to drive faster. These kinds of mitigating actions may not arise in clinical trials (particularly if persons do not know their vaccination status) but would apply in the field. Moreover, the nature and extent of these actions may vary over time as empirical evidence grows about the nature of the disease and the effects of vaccinations and non-pharmaceutical interventions. People may also change their behavior in response to cumulated “fatigue” from past isolation.⁹

In the regression analysis, the estimated effects of vaccinations on COVID outcomes comprise direct effects combined with any mitigation behavior. In some contexts, these combined effects are the objects of interest—e.g. overall effects of vaccinations on deaths (or of seatbelt use on automobile fatalities). In other contexts, there would be more interest in the effects of vaccinations, holding fixed the behavioral variables.¹⁰ In any event, the regression results apply to the combined effects in various periods.

The final issue concerns reverse causation. Higher vaccination rates likely reduce COVID deaths, hospitalizations, and cases, and these are the effects that we seek to isolate. However, in addition, higher probabilities of death, hospitalization, and infection likely encourage people to get vaccinated (and motivate governments to mandate or subsidize vaccinations and to support the creation and distribution of vaccines). The first channel, whereby vaccination reduces probabilities of adverse outcomes, tends to generate negative associations between vaccination rates and rates of death, hospitalization, and infection, whereas the second channel tends to generate positive associations. If the second channel is not held constant, the observed associations tend to underestimate the magnitude of the (negative) effects from vaccination.

A common way to deal with reverse causation is to use instrumental variables that explain a substantial part of the variation in the explanatory variable, in the present context the vaccination rate, but do not enter directly as determinants of COVID outcomes. That is, the instrument matters for outcomes only through the channel of affecting the frequency of vaccination. The present analysis uses as an instrument a variant of the variable proposed by Welsch (2020, Section 3.2)—the Trump (Republican) share of the 2020 Presidential vote.¹¹ Welsch (Table 2) used the 2016 value of this variable as an instrument for facemask usage, measured in July 2020 in a survey conducted by The New York Times.

Table 2.

Regressions for COVID Deaths per Capita

	(1)	(2)	(3)	(4)
Estimation method	SUR	SUR	Instruments	Instruments
Period I: 3/19/21–6/13/21
vaccination rate	−.0010 (.0014)	−.0011 (.0014)	−.0057 (.0086)	−.0057 (.0086)
Period II: 6/13/21–9/6/21
vaccination rate	−.0042*** (.0008)	−.0041*** (.0008)	−.0041*** (.0012)	−.0041*** (.0012)
Period III: 9/6/21–12/1/21
vaccination rate	−.0092*** (.0016)	−.0100*** (.0017)	−.0093*** (.0021)	−.0096*** (.0021)
vaccination rate, older	--	−.0060* (.0032)	--	−.0063 (.0047)
joint p-value		0.000		0.000
p-value for equal coeffs		0.24		0.44
Period IV: 12/1/21–2/25/22
vaccination rate	−.0023** (.0010)	−.0023** (.0012)	−.0025* (.0014)	−.0031** (.0015)
vaccination rate, older	--	−.0023 (.0015)	--	.0016 (.0017)
joint p-value		0.08		0.13
p-value for equal coeffs		0.99		0.39
Period V: 2/25/22–5/22/22
vaccination rate	−.0002 (.0007)	.0000 (.0007)	.0004 (.0010)	.0006 (.0009)
vaccination rate, older	--	−.0012 (.0012)	--	.0010 (.0014)
joint p-value		0.58		0.44
p-value for equal coeffs		0.32		0.23
R-squared	.28 .65 .65 .65 .60	.28 .65 .66 .65 .61	.16 .65 .65 .65 .59	.16 .65 .66 .65 .60
s.e.	.0003 .0003 .0006 .0004 .0003	.0003 .0003 .0006 .0004 .0003	.0003 .0003 .0006 .0004 .0003	.0003 .0003 .0006 .0004 .0003

Notes: Sample is 50 U.S. states plus District of Columbia. Sample dates shown in the left-most column refer to the dependent variable. This variable is the change in cumulative reported COVID-related deaths per capita over each period (values expressed per year). Vaccination rate in columns 1 and 3 is the fraction of the population fully vaccinated against COVID-19 (not counting booster shots). This variable is lagged 14 days from the dependent variable and is entered as an average over each period, as described in Table 1. In columns 2 and 4, vaccination rate is the fraction of the population fully vaccinated over roughly the last 6 months plus the fraction fully vaccinated earlier who have received booster shots. In these columns, “vaccination rate, older” is the fraction fully vaccinated roughly 6 or more months in the past less the fraction who have received booster shots. Other explanatory variables, shown in Table 1, are fraction of population aged 65 and over in 2020, life expectancy at birth in 2018, fraction of population aged 25 and over who completed high school or more in 2019, fraction of population black in 2020, urbanization rate in 2010, and average maximum temperature over periods corresponding to the dependent variable. Coefficients on these variables, constant terms, and the vaccination rates differ across periods. Standard errors of coefficient estimates are in parentheses. SUR (seemingly-unrelated regression) allows for a different error variance in each period and for correlation of the error terms across periods. s.e. is the standard error of each regression. In columns 1 and 3, instrumental estimation (three-stage least-squares) uses as the excluded instrument the fraction of the population voting in 2020 that voted Republican (as shown in Table 1). In columns 2 and 4, the instrument list also includes the difference in the vaccination variable for the current period from that in the period roughly 6 months earlier.

^***Significant at 1%,

^**significant at 5%,

^*significant at 10%.

Perhaps surprisingly, the Trump variable has a great deal of explanatory power for vaccination rates across states, even after holding constant key socio-economic variables, such as those mentioned before—old-age share, life expectancy, education, fraction black, and urbanization. That is, the Trump variable does not matter for vaccine take-up because it proxies for these kinds of socio-economic factors. Therefore, from the standpoint of having a lot of independent explanatory power for vaccination rates, the Trump variable is a good candidate as an instrument. In effect, the 2020 Presidential voting pattern sorts people (and states) into bins for vaccine attitudes in a manner that is largely orthogonal to socio-economic characteristics.

A reasonable concern is that the Trump variable would matter for COVID outcomes in ways that do not work entirely through vaccination status. For example, Welsch (2020, Appendix Table A1) found that the Trump vote share was inversely related to facemask usage in the New York Times survey. Consistent with Welsch’s findings, for the period 3/16/20–2/1/21, which precedes major distribution of vaccines, the presence of a facemask mandate at the state level is significantly negatively related to the Trump vote share.¹² However, a combination of the estimated negative effect of the Trump vote variable on facemask mandates with the Hansen and Mano (2021a) estimated negative effect of facemask mandates on COVID deaths yields a very small implied positive effect of the Trump vote on COVID deaths, compared with the effects estimated below that work through vaccinations. Similarly, in the RCT of Abaluck, et al. (2022), the significantly negative effect of facemask usage on COVID infection corresponds to a minor likely impact of facemask usage on COVID deaths, probably around two-to-three lives saved out of a total sample of 300,000. Therefore, from a quantitative standpoint, the Trump variable may be a satisfactory instrument for vaccination rates even though this variable has influences on COVID outcomes that work through facemask mandates and usage or other forms of non-pharmaceutical interventions.

II. Data and Empirical Setup

Data on COVID-related deaths, hospitalizations, and cases, measured relative to state population, are reported by the CDC and provided by Opportunity Insights, Economic Tracker (see Chetty, et al. [2022]).¹³ The data used in this study are for the 50 U.S. states plus the District of Columbia.

The three measures of COVID outcomes enter as dependent variables in the regressions and are examined over the five periods noted before. The starting date, March 19, 2021, is 14 days after the beginning of data on vaccination rates (fully vaccinated persons relative to state population), also coming from the CDC and Opportunity Insights.¹⁴ The first three periods, shown in Table 1, are 3/9/21–6/13/21, 6/13/21–9/6/21, and 9/6/21–12/1/21. These periods are of equal length (86 days) and extend to the rough date of onset of the omicron variant in the United States. The two most recent periods, 12/1/21–2/25/22 and 2/25/22–5/22/22, are the same length as the first three. For each period, COVID-related deaths or hospitalizations or cases are the changes in the cumulative per capita numbers, expressed at annual rates.

III. Regression Results

A. COVID-related deaths

Regression results in Table 2 are for COVID-related deaths per capita, observed over the five periods of 86 days: Period I (3/19/21–6/13/21), Period II (6/13/21–9/6/21), Period III (9/6/21–12/1/21), Period IV (12/1/21–2/25/22), and Period V (2/25/22–5/22/22).¹⁵ The first two columns are for seemingly-unrelated regressions, which use a least-squares procedure but compute standard errors of estimated coefficients when allowing for different error-term variances in each period and for correlation of the error terms across periods. The first column has on the right-hand side the average of the full vaccination rate over periods lagged 14 days relative to the dependent variable.¹⁶ Note that, whereas the dependent variable is the change in cumulative deaths per person over the periods shown, the independent variable is the cumulative level of full vaccinations per person (with a 14-day lag compared to the dependent variable).

To allow for a possible waning effectiveness of the vaccine, the specification in column 2 of Table 2 includes two measures of vaccination rates—one for full vaccinations that occurred roughly within the last six months and the other covering full vaccinations from six or more months in the past. In this specification, booster shots, for which CDC information starts on 10/20/21,¹⁷ are viewed as effectively converting an old full vaccination into a recent one. That is, when combined with the remaining efficacy of a full vaccination from six months ago, a booster is modeled as generating efficacy equal to that of a recent full vaccination. The inclusion of booster shots applies only to the three most recent periods in Table 2; that is, no boosters existed and none of the full vaccinations were “old” up to roughly September 2021.

The regressions include on the right-hand sides the socio-economic variables mentioned before—old-age fraction, life expectancy, high school education, fraction black, and urbanization. Also included is the historical average maximum temperature over the relevant period (computed from monthly data for the largest city in each state). In the estimation, separate coefficients are estimated for each period for each independent variable, including the constant term, which absorbs variations over time in aggregate COVID outcomes.

In column 1, the estimated coefficients on the (roughly) contemporaneous vaccination rate are negative and significant at the 1% level for Periods II and III (6/13/21–9/6/21 and 9/6/21–12/1/21), negative and significant at the 5% level for Period IV (12/1/21–2/25/22), and roughly zero for the other two periods. To assess the magnitudes of the estimated responses, consider period III, for which the estimated coefficient is the largest in magnitude, −0.0092. Over this interval, the mean of the vaccination-rate variable is 0.548 with a standard deviation of 0.079. Therefore, a one-standard-deviation increase in the vaccination rate, a rise by 14.4%, is estimated to lower the death rate by 0.00073, compared to the mean death rate of 0.00175. That is, the death rate falls by 41.7%. The implied elasticity of response is the ratio of −41.7 to 14.4, which equals −2.9. The estimated elasticity is similar for Period II (−3.2) but smaller in magnitude for Period IV (−0.7).¹⁸

When the two measures of vaccination rates are included in column 2, the results for period III (9/6/21–12/1/21) suggest that recent vaccinations are roughly twice as effective against deaths as older vaccinations; point estimates of coefficients are −.0100 and −.0058, respectively. Each of these estimated coefficients is statistically significant at least at the 5% level, and the two estimated coefficients differ from each other with a p-value of 0.22. Period IV (12/1/21–2/25/22) shows no indication of a waning influence of vaccinations, and Period V (2/25/22–5/22/22) provides essentially no information.

The small size of the estimated coefficient for Period I (3/19/21–6/13/21) may reflect reverse causation from COVID deaths to vaccination propensity. This effect is likely to be powerful during the early stages of vaccination rollout, when the places most adversely impacted are likely to have large rollouts of vaccinations.

Another way to interpret the estimated effects of vaccinations on COVID deaths comes from the literature on the value of a statistical life.¹⁹ The point estimates for Period III from Table 2 imply that the coefficient −0.0100 applies to full vaccination rates over the first six months and the coefficient −0.0058 applies over the next six months. If vaccinations are ineffective after 12 months, a quantity V of full vaccinations would be expected to reduce deaths by V∙(.5∙.0100 + .5∙.0058) = .0079∙V. Therefore, to expect to save one life, one needs 1/.0079 = 127 full vaccinations, which correspond to 254 shots for a two-dose regime. (This analysis could also be applied to booster shots.)

The marginal cost of COVID-19 vaccinations has several components. First, the Department of Health and Human Services (2021, Table 18) estimates that the direct cost of each dose is $20 and the cost per dose from vaccine administration averages $20. The time required per dose for persons receiving shots is estimated to average 2 hours, with an hourly value of time of $20.55, implying a cost per dose of $41. The Occupational Safety and Health Administration (2021, p. 61480) estimates that the average worker time lost due to adverse reactions to the shots is 0.36 days for 2 doses, corresponding, if a work day consists of 8 hours, to 2.9 hours for 2 doses. With an hourly value of time of $20.55, the implied average cost for 2 doses due to adverse reactions is $60. Combining the various terms, the overall marginal cost for a two-dose regime is $222.

The value $222 implies that it costs about $56000 at the margin to expect to save one life through added two-dose vaccinations. Usual estimates of the value of a statistical life for the average person in the United States are much larger than $56000; for example, Robinson, Sullivan, and Shogren (2021) assume values as high as $10 million. Therefore, the present empirical results indicate that vaccinations against COVID-19 were a great bargain in late 2021. The results are less powerful with the smaller magnitudes of coefficients estimated for other periods. For example, with the coefficients estimated for 12/1/21–2/25/22 in Table 2, it requires 426 full vaccinations or 852 shots or about $189,000 to expect to save one life. Even this higher magnitude suggests that vaccinations were a great deal.²⁰

The instrumental estimation treats the vaccination rates as endogenous. The instrument list includes the 2020 Republican vote share for President, along with the other explanatory variables mentioned before. That is, the Trump vote share is the one excluded instrument.²¹ Table 3 shows first-stage regressions, with the vaccination-rate variable over the various periods as the dependent variable. The remarkable aspect of these results is the strong explanatory power of the Republican vote share in the 2020 election (Trump vote), especially for the four most recent periods. The important point is that a higher Trump vote share strongly associates with a lower vaccination rate even when the other explanatory variables are held fixed. An increase by 0.12 in this vote share (which has a mean of 0.49 and a standard deviation of 0.12) associates in Period V, 2/11/22–5/8/22, with a decline by 0.073 in the vaccination rate (which has a mean in this period of 0.64). The results are similar for the three preceding periods but are weaker for Period I, 3/5/21–5/3/21.

Table 3.

First-Stage Regressions for Vaccination Rates

	(1)	(2)	(3)	(4)	(5)
Periods for vaccination rates	3/5/21–5/3/21	5/3/21–8/23/21	8/23/21–11/17/21	11/17/21–2/11/22	2/11/22–5/8/22
Constant	.02 (.29)	.10 (.36)	.22 (.37)	.17 (.40)	.20 (.42)
Over-65	.24 (.19)	.75*** (.23)	.81*** (.24)	.71*** (.26)	.65** (.26)
Life expectancy	.0020 (.0036)	.0043 (.0046)	.0072 (.0047)	.0098** (.0051)	.0105** (.0053)
High School Education	.12 (.17)	.18 (.20)	−.08 (.21)	−.14 (.22)	−.17 (.23)
Black	−.131*** (.050)	−.247*** (.062)	−.213*** (.063)	−.203*** (.068)	−.211*** (.072)
Urban	−.046 (.036)	−.037 (.045)	−.025 (.045)	−.041 (.049)	−.039 (.051)
Average Maximum Temperature	.0001 (.0005)	.0001 (.0004)	−.0001 (.0002)	.0000 (.0002)	.0001 (.0003)
Trump vote	−.095** (.044)	−.475*** (.054)	−.531*** (.055)	−.561*** (.059)	−.606*** (.063)
R-squared	.42	.82	.84	.84	.84
s.e.	.026	.033	.034	.036	.038

Notes: Sample is 50 U.S. states plus District of Columbia. Dependent variables, over the periods shown in the top row, are the averages of full vaccination rates, as used in Table 2. Over-65 is the fraction of the population in 2020 that was aged 65 or more. Life expectancy at birth is for 2018. High School Education is fraction of the population in 2019 aged 25 or more that had completed four years of high school or more. Black is the fraction of the population in 2020 classified as black. Urban is the fraction of the population urbanized in 2010. Trump vote is the fraction of votes for President in 2020 that went Republican. Estimation is by seemingly-unrelated regression, which allows for a different error variance in each period and for correlation of the error terms across periods. Standard errors of estimated coefficients are in parentheses. s.e. is the standard error of each regression.

^***Significant at 1%,

^**significant at 5%,

^*significant at 10%.

The results from instrumental estimation are in columns 3 and 4 of Table 2. For Periods II and III, where the estimated effects from vaccinations on COVID deaths were strongest, the estimated coefficients from instrumental estimation are still highly significant. For Period I, 3/19/21–6/13/21, the extent of the change in the point estimate of the coefficient is much larger under instrumental estimation, and this estimated value is now in the ballpark of those found for Periods II and III. However, the standard error of the coefficient estimate blows up, likely because the excluded instrument—the Trump vote variable—is only marginally significant for explaining the vaccination rate in this period (Table 3). That is, the instrument is weak.

For Period IV, 12/1/21–2/25/22, the instrumental estimate in column 3 of Table 2, which includes only one vaccine variable, is close to that found before. In column 4, the results do not clearly distinguish the effect from recent vaccinations (including boosters) to that from older vaccinations. In any event, the main inference is that vaccinations had less effect overall against COVID deaths, compared to that in periods that preceded the rise of the omicron variant in early December 2021. For Period V, 2/25/22–5/22/22, the instrumental estimates differ insignificantly from zero.

B. All-Cause Excess Mortality

A number of measurement issues arise for the CDC data on COVID-related deaths. One concern is that the dates entered refer to report dates, rather than the timing of deaths. However, this issue may not be of major consequence for data averaged over periods of substantial length, such as the roughly three months used for each period in the regressions.

A likely more serious issue involves the classification of causes of death between COVID and alternatives. Specifically, this assignment may be sensitive to the degree of testing for COVID—as mentioned before (n. 4), this testing has varied substantially over time. A high level of testing has a direct positive impact on the number of reported cases, but an increased tendency to test hospitalized patients for COVID may also raise the numbers of hospitalizations and deaths assigned to COVID.

A possible way to deal with this measurement problem is to use excess all-cause mortality, rather than deaths specifically ascribed to COVID-19, to construct the dependent variable. One downside of this procedure is that it introduces a lot of noise into the dependent variable by including in deaths the large numbers unrelated to COVID and, therefore, unlikely to be related to vaccination against COVID. As discussed in the Appendix, estimates of excess all-cause mortality are provided by state on a weekly basis by the CDC. However, the CDC indicates that there is a lag of more than eight weeks for the all-cause mortality data to be nearly complete.

Death rates constructed from excess all-cause mortality are highly correlated with those based on COVID-related mortality for Periods II and III of Table 2 but less strongly correlated for Periods I and IV. The correlations between the two alternative concepts of the dependent variable are 0.32 for Period I, 0.89 for Period II, 0.88 for Period III, and 0.50 for Period IV. (This comparison was not conducted for Period V because of the lag in reporting of data on all-cause excess mortality.)

The estimated regression coefficients based on all-cause excess mortality, shown in Table A1, are broadly similar to those shown for COVID deaths in Table 2. Most importantly, the estimated effects of vaccinations on mortality still appear weaker in Periods I and IV, compared to those in Periods II and III. In addition, for all-cause excess mortality, there is no indication of a negative effect from vaccines on mortality in Period IV. Overall, the main differences in the results are that the regression fits are poorer and the standard errors of coefficient estimates are higher when all-cause excess mortality is used instead of COVID-related mortality.

C. COVID-related hospitalizations

Table 4 has regression results with COVID-related reported hospitalizations per capita as the dependent variable. This setting parallels that in Table 2 for COVID deaths. Results on hospitalizations in Table 4 for Periods II and III (6/13/21–9/6/21 and 9/6/21–12/1/21) roughly parallel those for COVID deaths. To evaluate the magnitudes of the estimated responses for hospitalizations, consider Period III, for which the estimated coefficient on the vaccination rate in column 1 is −0.28. As noted before, the mean of the vaccination rate over this period is 0.548 with a standard deviation of 0.079, so that a one-standard-deviation increase in the vaccination rate, a rise by 14.4%, is estimated to lower the hospitalization rate by 0.022, compared to the mean of 0.077. That is, the hospitalization rate falls by 28.6%. The implied elasticity of response is the ratio of −28.6 to 14.4, which equals −2.0 (compared to −2.9 for deaths).

Table 4.

Regressions for COVID Hospitalizations per Capita

	(1)	(2)	(3)	(4)
Estimation method	SUR	SUR	Instruments	Instruments
Period I: 3/19/21–6/13/21
vaccination rate	−.003 (.058)	.048 (.061)	−.085 (.380)	−.085 (.380)
Period II: 6/13/21–9/6/21
vaccination rate	−.261*** (.053)	−.275*** (.053)	−.257*** (.073)	−.257*** (.073)
Period III: 9/6/21–12/1/21
vaccination rate	−.274*** (.046)	−.312*** (.046)	−.245*** (.068)	−.272*** (.065)
vaccination rate, older	--	−.098 (.083)	--	−.108 (.145)
Period IV: 12/1/21–2/25/22
vaccination rate	.008 (.052)	−.059 (.053)	.081 (.082)	−.011 (.085)
vaccination rate, older	--	.135** (.066)	--	.237** (.095)
Period V: 2/25/22–5/22/22
vaccination rate	.032 (.019)	.020 (.018)	.056* (.026)	.043 (.026)
vaccination rate, older	--	.092** (.029)	--	.133*** (.040)
R-squared	.49 .70 .64 .54 .24	.49 .70 .68 .60 .33	.46 .70 .63 .52 .23	.46 .70 .67 .59 .31
s.e.	.013 .018 .018 .025 .009	.013 .017 .018 .023 .008	.013 .017 .019 .025 .009	.013 .017 .018 .024 .008

Notes: See notes to Table 2. The only difference is that the dependent variable is based on COVID-related reported hospitalizations per capita.

^***Significant at 1%,

^**significant at 5%,

^*significant at 10%.

Results in Table 4 for Period I, 3/19/21–6/13/21, are also parallel to those for deaths in the sense that the vaccination rate does not have statistically significant effects on hospitalizations. These results may again reflect reverse causation in this period—the point estimate of the coefficient on the vaccination rate is negative and larger in magnitude in the instrumental estimation, but the standard error blows up.

The hardest results to interpret for COVID hospitalizations are for Periods IV and V, 12/1/21–2/25/22 and 2/25/22–5/22/22, which cover the rise of the omicron variant. There is no indication in these periods that vaccinations reduce COVID-related hospitalizations and some indications of positive effects. Positive effects might arise if vaccinations induce reductions in protective practices such as social distancing and masking.

D. COVID-related cases

Table 5 has regression results with COVID-related reported cases per capita as the dependent variable. This setting parallels that in Tables 2 and 4 for COVID deaths and hospitalizations, respectively. The results for cases in Table 5 for Periods II and III (6/13/21–9/6/21 and 9/6/21–12/1/21) roughly parallel those for deaths and hospitalizations. To evaluate the magnitudes of the estimated responses for cases, consider Period III, for which the estimated coefficient on the vaccination rate in column 1 is −0.41. As noted before, the mean of the vaccination rate over this period is 0.548 with a standard deviation of 0.079, so that a one-standard-deviation increase in the vaccination rate, a rise by 14.4%, is estimated to lower the case rate by 0.032, compared to the mean of 0.134. That is, the case rate falls by 23.9%. The implied elasticity of response is the ratio of −23.9 to 14.4, which equals −1.7 (compared to −2.0 for hospitalizations and −2.9 for deaths).

Table 5.

Regressions for COVID Cases per Capita

	(1)	(2)	(3)	(4)
Estimation method	SUR	SUR	Instruments	Instruments
Period I: 3/19/21–6/13/21
vaccination rate	−.015 (.091)	.031 (.092)	−.684 (.711)	−.684 (.711)
Period II: 6/13/21–9/6/21
vaccination rate	−.462*** (.083)	−.459*** (.083)	−.509*** (.117)	−.509*** (.117)
Period III: 9/6/21–12/1/21
vaccination rate	−.397*** (.089)	−.503*** (.086)	−.328*** (.121)	−.390*** (.114)
vaccination rate, older	--	.075 (.156)	--	.460* (.254)
Period IV: 12/1/21–2/25/22
vaccination rate	.294* (.165)	.147 (.181)	.272 (.220)	.200 (.242)
vaccination rate, older	--	.541** (.222)	--	.394 (.273)
Period V: 2/25/22–5/22/22
vaccination rate	.253*** (.044)	.242*** (.046)	.317*** (.060)	.307*** (.061)
vaccination rate, older	--	.303*** (.075)	--	.379*** (.095)
R-squared	.43 .65 .71 .16 .61	.43 .65 .76 .17 .61	-.16 .66 .71 .17 .61	-.16 .66 .74 .19 .61
s.e.	.018 .028 .033 .068 .020	.018 .028 .030 .068 .020	.025 .028 .033 .067 .020	.025 .028 .032 .067 .020

Notes: See notes to Table 2. The only difference is that the dependent variable is based on COVID-related reported cases per capita.

^***Significant at 1%,

^**significant at 5%,

^*significant at 10%.

Results in Table 5 for Period I, 3/19/21–6/13/21, also parallel those for deaths and hospitalizations in the sense that the vaccination rate does not have a statistically significant effect on cases. These results may again reflect reverse causation—the point estimate of the coefficient on the vaccination rate is negative and much larger in magnitude in the instrumental estimation, but the standard error blows up.

The hardest results to interpret are again for Periods IV and V, 12/1/21–2/25/22 and 2/25/22–5/22/22, which cover the rise of the omicron variant. As with hospitalizations, there is no indication in these periods that vaccinations reduce COVID cases. Moreover, there is a stronger indication of positive coefficients. As noted before, this pattern might reflect negative influences of vaccination on protective actions such as social distancing and masking.

E. Effects of Prior Infection on COVID Outcomes

A common view is that COVID infection provides natural immunity against future mortality, partly by reducing infection risk and partly by lowering mortality probability conditional on infection. One way to assess this possibility is to add measures of lagged COVID cases per capita to the regressions for COVID deaths per capita in Table 2. This analysis is challenging because, as noted before, case numbers are subject to substantial measurement error. Moreover, there is a lack of instruments to use to isolate exogenous variations in cases. The present analysis uses 6-month lags of case rates as explanatory variables and also includes these variables on the instrument lists.

Table 6 has results when 6-month lags of COVID cases per capita are added to the regressions for COVID deaths per capita from columns 2 and 4 of Table 2. These regressions also include, as before, recent and older vaccination rates. Column 1 of Table 6 has estimates for seemingly-unrelated regressions, and column 2 has instrumental estimates when the instruments are those used in Table 2, along with the lagged values of the case rates.

Table 6.

Regressions for COVID Deaths per Capita, Including Lagged Cases

	(1)	(2)
Estimation method	SUR	Instruments
Period I: 3/19/21–6/13/21
vaccination rate	−.0010 (.0014)	−.0044 (.0076)
lagged case rate	.0011 (.0019)	.0005 (.0021)
Period II: 6/13/21–9/6/21
vaccination rate	−.0049*** (.0009)	−.0057*** (.0014)
lagged case rate	−.0037** (.0018)	−.0043** (.0020)
Period III: 9/6/21–12/1/21
vaccination rate	−.0132*** (.0017)	−.0166*** (.0030)
vaccination rate, older	−.0083*** (.0030)	−.0156*** (.0056)
lagged case rate	−.0124*** (.0038)	−.0170*** (.0051)
Period IV: 12/1/21–2/25/22
vaccination rate	−.0034** (.0014)	−.0056** (.0025)
vaccination rate, older	−.0040** (.0018)	−.0052* (.0030)
lagged case rate	−.0037 (.0030)	−.0064 (.0044)
Period V: 2/25/22–5/22/22
vaccination rate	.0001 (.0009)	.0020 (.0018)
vaccination rate, older	−.0013 (.0013)	.0006 (.0021)
lagged case rate	−.0001 (.0019)	.0030 (.0029)
R-squared	.28 .68 .71 .66 .61	.22 .67 .67 .66 .57
s.e.	.0003 .0003 .0005 .0004 .0003	.0003 .0003 .0006 .0004 .0003

Note: These results extend columns 2 and 4 of Table 2 to include the lagged cumulative number of COVID cases per capita. The lagged case rates are for 12/23/20 for period I, 3/19/21 for period II, 6/13/21 for period III, 9/6/21 for period IV, and 12/1/21 for period V. These lagged values are included on the instrument lists in column 2.

^***Significant at 1%,

^**significant at 5%,

^*significant at 10%.

For periods II and III, the estimated coefficients on the lagged case rate in Table 6 are significantly negative, with larger magnitudes under instrumental estimation. As an example, the results for period III imply elasticities of response of COVID death rates to prior COVID cases at the sample means of −1.2 for the SUR estimation and −1.6 under instrumental estimation. The estimated coefficients on the vaccination rates are larger in magnitude than those found in Table 2, when the lagged case rate was omitted. There is also less indication than before of an attenuation of vaccine efficacy for older vintages.

For other periods, the coefficients on the lagged case rate are significantly negative for period II and negative but not statistically significant for period IV. For Period V, there is no indication of a negative effect from previous infection on current mortality. This result goes along with the lack of effect from vaccination.

F. Summing Up

Regression results reveal clear negative effects from vaccination on COVID-related deaths up to the rise of the omicron variant in early December 2021. There is some evidence that the impact of vaccinations attenuates over time, but vintages over six months old retain some efficacy. The influence of vaccination on deaths weakens by early 2022 and disappears in the period from February to May 2022. This pattern likely reflects reduced effectiveness of vaccination against new variants of the virus. Hospitalizations show a similar overall pattern, though the evidence for weakening effects appears earlier than for deaths. This pattern appears in a stronger form for cases, and there is some indication in recent periods of coefficients with the “wrong” sign; that is, higher vaccination associating with more cases. This pattern can arise if vaccinations motivate individuals to engage in more risky behavior, such as increased social interaction. There is also some indication up to December 2021 of a negative effect on mortality from prior infection. As with vaccination, there is no evidence in the period from February to May 2022 that prior infection reduces mortality.

IV. Observational Studies of COVID Vaccinations

A number of observational studies exist that can be compared with the present regression findings. First are the regular CDC reports on COVID outcomes in relation to vaccination status.²² On October 17, 2021, data from 31 jurisdictions indicated that COVID cases per capita for unvaccinated persons were 4.3 times those for fully vaccinated and 15.4 times for those with a booster shot. The respective ratios fell to 2.2 and 3.4 on January 2, 2022, 3.2 and 1.5 on April 24, 2022, and 3.5 and 1.7 on May 15, 2022. The higher case rates for those boosted compared to those only fully vaccinated in April-May 2022 suggest a selection issue related to those who chose to become boosted. In any event, case rates for vaccinated had become closer to those for unvaccinated, when compared to the outcomes from October 2021.

For COVID deaths, covering 30 jurisdictions, the ratio on October 17, 2021 for unvaccinated versus fully vaccinated was 14.3, whereas that for unvaccinated versus boosted was 30.6. The respective ratios fell to 6.7 and 23.5 on January 2, 2022 and to 5.6 and 6.6 on April 24, 2022. Thus, compared to October 2021, death rates for unvaccinated became closer to those for vaccinated, but a substantial gap remained in April 2022, with boosted still doing better than fully vaccinated.

Overall, the CDC reports on outcomes in relation to vaccination status seem consistent with the present regression findings, which indicate weaker effects of vaccinations on COVID deaths and, especially, cases since early December 2021. It is worth keeping in mind that the CDC analysis is subject to issues analogous to those that apply to the cross-state regressions. For example, if less healthy people are more likely to die from COVID, for given vaccination status, and more likely to be vaccinated, then the association between vaccination and death would tend to understate the beneficial effects from vaccination. Similar effects arise if older people are more likely to die for given vaccination status and more likely to be vaccinated, although the CDC indicates that its statistics adjust for age. However, the CDC analysis does not adjust for other socio-economic variables or for vintage of vaccination.

Another group of large-scale observational studies assesses the efficacy of the Pfizer-Biontech vaccine in Israel. As a recent example of this research, Arbel, et al. (2022) studied the effects on COVID-19 mortality in a 40-day period in early 2022 from a second booster shot of the Pfizer-Biontech vaccine. The sample consisted of the 563,000 members of a large healthcare organization who had previously received a first booster shot, were aged 60 and above, and who satisfied some other criteria. The rate of uptake of second booster shots in this sample was 58%. The regression analysis held fixed age and gender and other socio-economic characteristics, as well as an array of existing health conditions. The main finding (Arbel, et al. [2022, Table 3]) was that the second booster shot significantly lowered COVID-related mortality, particularly among the oldest group. However, as with the regression results in the first two columns of Table 2 in the present study, the analysis did not take account of the endogeneity of vaccine uptake (aside from the relationships with the observable variables that were included in the regressions).

V. Speculative Thoughts and Research Plans

The results in Tables 2 and 4-6 reveal substantial negative effects of vaccinations on COVID deaths, hospitalizations, and cases up to roughly the emergence of the omicron variant of the virus in early December 2021. Results on deaths (Table 2) suggest that the power of vaccines wanes over time but still remains effective even after about six months. This waning influence is offset by the introduction of booster shots. In comparison to the findings from earlier periods, the results since early December 2021 indicate that vaccinations have a weaker effect in reducing COVID deaths and may no longer reduce COVID hospitalizations and cases.

There are a number of possible explanations for the apparent reduction in the effectiveness of vaccinations in the cross-state analysis for the period since early December 2021 (Tables 2, 4, and 5). One is waning efficacy of vaccinations over time, though the regression analysis attempted to take account of this channel by considering the vintages of vaccinations and allowing for the introduction of booster shots. Another factor is diminishing effectiveness of existing vaccines against new forms of the virus, notably the omicron variant. A further possibility is that confidence engendered by vaccinations may have motivated individuals and governments to lessen non-pharmaceutical interventions, such as masking and social distancing. These responses may have been reinforced by “COVID fatigue,” which raised the perceived benefits from social interactions compared to the costs attached to health risks. Of course, this response need not be irrational; that is, the benefits from heightened social interactions may, in fact, more than offset the costs from increases in deaths, hospitalizations, and cases.

More narrowly, in terms of research plans, the first idea is to carry out the analysis at the county level. This change will sharply raise the available number of cross-sectional observations. However, the county-level data introduce new concerns about measurement error and about the connection between location of vaccination and location of outcome. There are also likely to be important spillover effects of disease transmission from one county to others.

Second, a key issue in the estimation involves the instrumental variables employed. Even if the Trump 2020 vote is viewed as an appropriate instrument, there are difficulties in extending the analysis to allow for more than one endogenous variable on the right-hand side of the regressions. This issue arises, for example, in attempting to distinguish the impact of recent from older vaccinations and in isolating effects from the lagged case rate. Relatedly, this analysis involves the role of booster shots. At this stage, it is unclear what additional instruments will be useful. Possibilities include the measures of pre-existing locations of retail pharmacies used by Hansen and Mano (2021b) and Brownstein, et al. (2022).

Highlights.

Changes made in response to referee suggestions are shown in the highlighted text.

Appendix

The analysis for COVID-related deaths per capita was redone with deaths calculated from all-cause excess mortality rather than COVID-related mortality. The underlying information for calculating excess mortality is given by the CDC in Provisional COVID-19 Death Counts by Week Ending Date and State, updated June 28, 2022, available at data.cdc.gov. Data are from columns K (total deaths) and L (percent of expected deaths). The number of expected deaths is calculated by the CDC from the observed average number of deaths for the corresponding weeks in 2017–2019. The computed all-cause excess mortality rate equals total deaths divided by state population multiplied by (percent of expected deaths – 100)/(percent of expected deaths). State population is taken to be the value for 2020 from the U.S. Census Bureau. This measure of the all-cause excess mortality rate is used to calculate the dependent variables in Table A1. The CDC indicates that there is a substantial lag in computing nearly complete data for total deaths—“data are generally at least 75% complete within 8 weeks of when the death occurred.” Therefore, these data are used only for regressions in Table A1 that go through late February 2022.

Table A1.

Regressions for All-Cause Excess Mortality per Capita

	(1)	(2)	(3)	(4)
Estimation method	SUR	SUR	Instruments	Instruments
Period I: 3/20/21–6/12/21
vaccination rate	−.0024 (.0018)	−.0012 (.0021)	.0012 (.0125)	.0012 (.0125)
Period II: 6/12/21–9/4/21
vaccination rate	−.0082*** (.0019)	−.0078*** (.0020)	−.0057* (.0031)	−.0057* (.0031)
Period III: 9/4/21–12/4/21
vaccination rate	−.0089*** (.0018)	−.0094*** (.0018)	−.0052* (.0031)	−.0062* (.0032)
vaccination rate, older	--	-.0052 (.0039)	--	-.0005 (.0041)
joint p-value		0.000		0.012
p-value for equal coeffs		0.29		0.027
Period IV: 12/4/21–2/26/22
vaccination rate	−.0002 (.0023)	.0001 (.0026)	.0026 (.0030)	.0023 (.0034)
vaccination rate, older	--	−.0006 (.0032)	--	.0031 (.0038)
joint p-value		0.98		0.68
p-value for equal coeffs		0.84		0.82
R-squared	.36 .60 .60 .36	.36 .60 .61 .36	.37 .59 .59 .34	.37 .59 .56 .34
s.e.	.0004 .0007 .0008 .0009	.0004 .0007 .0008 .0009	.0004 .0007 .0008 .0009	.0004 .0007 .0009 .0009

Note: The specification is the same as in Table 2 except that the death rate is measured by all-cause excess mortality per capita and Period V is excluded.