Effect of mandating vaccination on COVID-19 cases in colleges and universities

Abstract

Background

With the introduction of COVID-19 vaccines, many colleges and universities decided to mandate vaccination for all students and employees. The objective of this paper is to empirically investigate the effect of the mandate policy on Fall 2021 COVID-19 cases in institutions of higher education.

Method

We construct a unique dataset of a sample of 94 colleges and universities in the east and southeast regions of the United States, 41 of which required vaccination prior to Fall 2021. A difference-in-differences analysis is conducted, considering vaccine requirement as a policy implemented only in a sub-group of these institutions. We control for several factors, including state-level case per capita and student population.

Results

Our analysis shows that mandatory vaccination substantially decreased cases in institutions of higher education by 1,473 cases per 100,000 student population (95 CI: 132, 2813).

Conclusions

The results suggest that a COVID-19 vaccine requirement is an effective policy in decreasing cases in such institutions, leading to a safer educational experience.

Introduction

Colleges and universities have continuously faced a tension between keeping students safe from COVID-19 and offering a pleasant and interactive educational service (Frazier et al., 2022; Ghaffarzadegan et al., 2021; Paltiel, 2021). With the advent of vaccines, there was increasing hope that institutions of higher education could return to their in-person mode of operation in Fall 2021 and offer a more rewarding educational experience (Paltiel & Schwartz, 2021). While no one expected that all students and employees would voluntarily take the vaccines (Poehling et al., 2012), it was hoped that a large majority would be fully vaccinated by the beginning of the Fall semester. Many institutions encouraged students to become fully vaccinated, and several went even further and instituted stricter policies mandating vaccination (Hodge et al., 2022).

The institutional risks and benefits of a vaccine mandate policy were unclear at the time. Several state institutions in the United States decided to “encourage” rather than “require” the vaccines (Moody, 2021); others made the requirement contingent on the FDA's full approval (Moody, 2021) of the vaccines (they were initially given “emergency authorization” for use by the FDA), which delayed vaccination by several months for many students and staff. While studies have looked at institutional factors that explain whether or not universities adopted the policy to mandate (Gostin et al., 2021; Hodge et al., 2022), an important question is whether mandates decreased Fall 2021 cases of COVID-19.

While there have been a number of case-based modeling studies of the spread of the virus in higher education settings (Frazier et al., 2022; Ghaffarzadegan, 2021; Gressman & Peck, 2020; Yu et al., 2021), a clear answer to the question requires a large-scale examination beyond a single institution. Our study addresses this question with a sample of 94 colleges and universities in the east and southeast regions of the United States. Our study shows that mandating vaccination substantially decreased COVID-19 cases by an average of 1,473 per 100,000 student population (95% CI: 132, 2813).

Methods

We study the effect of mandates on cumulative cases over a full semester normalized to student population. A difference-in-differences analysis is conducted considering a vaccine requirement as a policy implemented only in a sub-group of institutions. The method is chosen due to its simplicity and flexibility, and the fact that it helps to get closer to exploring causal effects. By comparing the two groups of treatment and control (in our study, institutions that mandate COVID-19 vaccination versus others), and controlling for cross-observational variations, the method explores the difference between the groups, before and after policy implementation. Furthermore, to address the limitations of our data and study design (limitations are discussed at the end of the paper) we supplement the primary analysis with several other statistical models for robustness check.

Data

Since there are no comprehensive data available, we construct a dataset from multiple sources. We focus on 10 distinct states in the U.S. east and southeast regions to minimize the effects of regional heterogeneities in our dataset. These include Georgia, Maryland, North Carolina, Ohio, Pennsylvania, South Carolina, Tennessee, Virginia, West Virginia, and the District of Columbia. A research assistant, blind to the research hypothesis, was asked to select about 100 colleges and universities with two main constraints: they should have student populations of 1,000 or more and publicly accessible COVID-19 dashboards tracking cases within their student population.

Student populations were found by an online search of the most recent available data, often from institution websites, U.S. News & World Report rankings, and Wikipedia pages. For the vaccine requirement policy, we used archival data on all college and university websites and their date of implementation to differentiate institutions that required vaccination for registration for Fall 2021. Of our final sample of 94 institutions, 41 required vaccination prior to attending Fall 2021 semester classes, and 9 institutions required vaccination after the FDA's full approval of the Pfizer vaccine, which resulted in delaying full vaccination until the middle of the semester or later.

We constructed a dataset for this list of institutions from different sources. For Fall 2021 case data, we used the colleges’ and universities’ COVID-19 dashboards. Several universities remove or update their dashboard after each semester. Thus, for previous semesters, we used the New York Times online tracking of higher education cases in 2020–211 . Sources for New York Times data are universities. However, to assure consistency, we also manually compared past cases of universities with available archival data with the New York Times dataset. The data were consistent. Cumulative cases (over a semester) per 100,000 student population was used as the dependent variable. For several of the institutions, but not all, total tests for Fall 2021 were also available; these were included in our dataset.

More details about the sample, including our method for determining institutions’ vaccination requirement policies, are provided in the online supplementary. Table S1 in the supplementary reports summary statistics.

Approach

The main analysis in this study is a difference-in-differences analysis that follows equation 1:

$$I_{i,j}={\beta }_0+{\beta }_1·F{21}_j+{\beta }_2·R_i+{\beta }_3·\left(F{21}_j·R_i\right)+{\beta }_4·C_{i,j}$$ (1)

where $I_{i,j}$ is the number of cases in institution $i$, over semester $j$, per 100,000 student population, and where $j=\left\{\text{Spring}2021,\text{Fall}2021\right\}$. In other words, since the number of cases in colleges and universities of different populations naturally differ, our dependent variable is cumulative cases (over a full semester) per 100,000 student population. $F{21}_j$ is equal to 1 for Fall 2021 datapoints, and zero otherwise, and $R_i$ is 1 for an institution $i$ if it mandated a vaccine prior to the start of Fall 2021. In this model, our main focus is on ${\beta }_3$ which will represent the effect of requiring vaccination while ${\beta }_2$ captures other differences between the two groups of institutions, and ${\beta }_1$ controls for the secular trend. In this equation, $C_{i,j}$ includes all covariates, such as state-level case per capita and student population. Additional analyses are offered to check the robustness of the results.

Results

Our initial investigation confirms that COVID-19 vaccination rates were higher in institutions that mandated it. Among universities that reported their vaccination rate on their COVID-19 dashboards, the average rate for ones that mandated vaccination was 0.95 (standard deviation =0.05). In contrast, average vaccination rate of those that did not mandate it reached 0.75 (0.12) by the end of the Fall semester (0.77 (0.13) if we include ones that mandated around the middle of the semester). This is not a surprise given that in universities that mandated vaccination, students that did not provide evidence for religious or medical exemptions were often disenrolled. For example, Virginia Tech and University of Virginia disenrolled 134 and 238 unvaccinated students in Fall 2021 respectively.2

Average Spring and Fall 2021 cases (per capita) in institutions of higher education that required vaccination (the treatment group) were 3,642 and 2,947, respectively. Among others (the control group), the same measures were 3,344 and 4,171 cases. We compared cases (per 100,000 student population) of the control and treatment groups in Spring 2021 (prior to the mandate) and, as expected, observed no statistically significant difference.

Table 1 shows the results of our analysis using equation 1 with different control variables. Model 1 depicts that the coefficient of interest is large and statistically significant: about 1,522 (per 100,000 student population) fewer cases in universities that mandated vaccination. The results are robust controlling for institutional factors (model 2). Multicollinearity is not a concern in the model (F21, VIF = 2.6; R, VIF = 2.3; F21*R, VIF=3.1; F20, VIF=2.2; student population, VIF=2.1). Using different measures to control for variation across the states, the results remain statistically significant and large (p=.032). Interestingly, in model 3 state cases are not associated with university cases. Overall, as models 1–3 consistently show, mandatory vaccination substantially decreased the number of cases.

Table 1.

Regression results for the impact of required vaccination on COVID-19 cases at colleges and universities. DV: Cumulative cases per 100,000 student population over a full semester (Spring 2021 and Fall 2021)

		Model 1	Model 2	Model 3
Main independent variables
	F21 (Fall 2021)	827*	827*	727
		(468)	(424)	(697)
	R (Required Vaccination)	298	830*	1,333**
		(501)	(466)	(538)
	F21*R (the coefficient of interest)	-1,522**	-1,522**	-1,473**
		(708)	(642)	(679)
Institution-level control
	Historical Cases (F20)		0.33***	0.26***
			(0.05)	(0.06)
	Student Population		-0.03***	-0.02*
			(0.01)	(0.01)
State-level control
	State cases per capita			0.04
				(0.24)
	State Fixed Effect			Yes
Intercept		3,344***	2,241***	2,005***
		(331)	(412)	(773)
R-Squared		0.03	0.22	0.31
Adjusted R-squared		0.02	0.19	0.25
F		2.18*	10.00***	4.91***
Observations		188	188	188

Note: *p<0.1; **p<0.05; ***p<0.01; Standard errors are in parentheses below coefficient estimates; DV: Dependent variable; DV is cumulative cases per 100,000 student population.

In addition, cases per student population declined by student population, possibly indicating that, on average, larger institutions implemented more effective policies and thus experienced relatively lower case rates.

Robustness check

Additional analyses show that the policy effect becomes statistically and size-wise stronger if we include the marginal effect of the policy in institutions of higher education that required vaccination by mid-semester, after the FDA's full approval. To that end, we consider the fact that some colleges and universities required vaccination by the middle of the semester or later. This can potentially influence the results, and we expect to see a stronger effect of mandating vaccination. To test, we run a difference-in-differences analysis with continuous treatment. The model is similar to equation 1, replacing $R$ with $R\prime$, the latter being “duration of the vaccine requirement” with the unit of a semester. Specifically, $R\prime$ for institutions that required vaccination before the beginning of the semester was 1; $R\prime$ for those that set their due date for after the semester began (often mid-October or mid-November which is around the middle of the Fall semester) was set at 0.5; and for the rest, $R\prime$ was set at 0. The results reported in Table 2 are consistent with and even stronger (-1,709 cases; 95% CI: -374, -3044) than the main analysis. They are also statistically more significant (p=.012), which was expected.

Table 2.

Regression results for the impact of required vaccination on COVID-19 cases at colleges and universities (per 100,000 student population), considering those that required vaccination after the beginning of the Fall semester.

		Model 1	Model 2	Model 3
Main independent variables
	F21 (Fall 2021)	1,023**	1,023**	857
		(500)	(452)	(683)
	R’ (Duration of vaccine requirement)	435	951**	1,600***
		(521)	(482)	(535)
	F21*R’ (the coefficient of interest)	-1,777**	-1,777***	-1,709**
		(737)	(667)	(676)
Institution-level control
	Historical Cases (F20) per population		0.33***	0.26***
			(0.05)	(0.06)
	Student Population		-0.03***	-0.02*
			(0.01)	(0.01)
State-level control
	State cases per capita			0.07
				(0.23)
	State Fixed Effect			Yes
Intercept		3,264***	2,146***	1,720**
		(353)	(425)	(759)
R-Squared		0.04	0.22	0.32
Adjusted R-squared		0.02	0.20	0.26
F		2.51*	10.36***	5.45***
Observations		188	188	188

Note: *p<0.1; **p<0.05; ***p<0.01; Standard errors are in parentheses below coefficient estimates; DV: Dependent variable. DV is cumulative cases per 100,000 student population. R’ is the duration of vaccine requirement (unit: a semester; R’={0, 0.5, 1}).

Furthermore, in a supplementary analysis, we compare college and university cases and control for test frequency across the universities. While we expect that variation across institutions is mainly captured by the covariates of R (Required) and Fall 2020 cases (per 100,000 student population), controlling for the test frequency (per 100,000 student population) helps make sure that the observed effect of requiring vaccination is not due to a change in the number of tests.

We conduct two examinations. First, we compare the average number of tests (per 100,000 student population) among colleges and universities that require vaccination with ones that did not require it. We find that tests were more frequent in institutions that required vaccination (p<.05), and thus a decline in case rates is not due to lower test rates. We then control for test frequency (per 100,000 student population) in a regression model of cumulative cases over a full semester. Since test numbers were available for only 69 institutions and only for Fall 2021, we can only check the association between requiring vaccination in institution $i$ and Fall 2021 cases per 100,000 population ($I_{i,F21}$), controlling for the number of tests per 100,000 student population ($T_{i,F21}$). Specifically, we run the following regression:

$$I_{i,F21}={\beta }_0+{\beta }_1·I_{i,F20}+{\beta }_2·I_{i,S20}+{\beta }_3·R_i+{\beta }_4·T_{i,F21}+{\beta }_5·C_i$$ (2)

In this model, all cumulative case values are per 100,000 student population. Here, we are mainly interested in ${\beta }_3$. Table 3 shows the results, and the coefficient of interest is statistically significant (p=.016). Furthermore, the Fall 2021 test rate (per 100,000 student population) is positively correlated with the case rate (p<.01), and thus more tests did result in a greater number of (identified) cases. The results show that even though colleges and universities that required vaccination had a higher test frequency too, the observed decline in their cases is not due to the effect of more tests. Note that in this supplementary analysis, the sample size drops to include only institutions that have reported their test figures for Fall 2021.

Table 3.

Robustness of results to controlling number of tests in Fall 2021. DV: COVID-19 cases per 100,000 student population in Fall 2021.

	Model 1	Model 2
F20 Cases per population	.19***	.17**
	(.07)	(.07)
S21 Cases per population	.17**	.23***
	(.08)	(.09)
R (Required Vaccination)	-1,014**	-1,119**
	(405)	(450)
Student Population	-.029*	-.035**
	(.015)	(.016)
F21 Tests per population		.0019***
		(.0005)
Intercept	3,115***	2,629***
	(440)	(475)
R-Squared	.28	.43
Adjusted R-squared	.25	.39
Observations	94	69

Note: *p<0.1; **p<0.05; ***p<0.01; Standard errors are in parentheses below coefficient estimates; DV: Dependent variable.

Finally, we would like to make sure that the effect of the policy on cases is via increasing vaccination rate. As stated, the data confirm that vaccination rate was higher among schools that mandated vaccination (p<.01). To examine whether the effect of the mandate on cases is via increasing vaccination rates, we run a similar test to model 1 of Table 3 (repeated as model 1 of Table 4 for the purpose of comparison), once replacing the mandate policy variable with the vaccination rate (model 2 in Table 4) and once with including the rate and the policy together (model 3 in Table 4). We expect that as the policy's effect should be via increasing vaccination, in model 2 vaccination rate should demonstrate a negative effect on cases, and in model 3 it should wipe-out the effect of the policy. The results confirm our expectations. In model 2 the effect of vaccination rate is significant (p=.003). In addition, in model 3 the effect of vaccination rate remains strong and statistically significant (p=.07) while as expected the effect of the policy disappears after controlling for the vaccination rate (p=.32). Overall, even though the sample size is declined to the ones that report vaccination rates, the observed pattern confirms that the effect of the policy on cases is via increasing vaccination rates.

Table 4.

Robustness of results to controlling vaccination rate in Fall 2021. DV: COVID-19 cases per 100,000 student population in Fall 2021.

	Model 1	Model 2	Model 3
F20 Cases per population	.19***	.18*	.18*
	(.07)	(.10)	(.10)
S21 Cases per population	.17**	.09	.10
	(.08)	(.10)	(.10)
R (Required Vaccination)	-1,014**		-641
	(405)		(694)
Vaccination rate		-6495 ***	-4948*
		(2060)	(2657)
Student Population	-.029*	-.012	-.016
	(.015)	(.021)	(.022)
Intercept	3,115***	7956***	6994***
	(440)	(1874)	(2146)
R-Squared	.28	.39	.40
Adjusted R-squared	.25	.33	.33
Observations	94	49	49

Note: *p<0.1; **p<0.05; ***p<0.01; Standard errors are in parentheses below coefficient estimates; DV: Dependent variable.

Discussions

Abundant evidence exists for the effectiveness of COVID-19 vaccines and the need for continuous investment in vaccines (Gupta et al., 2021; Paltiel, Schwartz, et al., 2020). However, vaccines are effective only if people take them and, from a policy standpoint, we should look for ways to increase vaccination rates. This study provides empirical evidence for the effect of mandating vaccines in colleges and universities. The findings of this study suggest that the incremental increase in vaccination rate that institutions of higher education are gaining from mandating vaccination can substantially decrease the number of cases and consequently result in a safer educational environment. Our findings imply that more institutions should require COVID-19 vaccination and boosters in the upcoming semesters.

Overall, this report provides the first empirical evidence for several past simulation studies (Paltiel & Schwartz, 2021; Ghaffarzadegan, 2022) showing that achieving a high vaccination rate is an important policy tool for the administrations of institutions of higher education. While institutions of higher education have benefited from several other policies (Larson et al., 2020; Nourinejad et al., 2021) including higher test frequencies (Paltiel et al., 2020; Schultes et al., 2021), and it is difficult to separate the effect of the mandates from other policies in correlational studies, our difference-in-differences method controls for the historical trends for each university and isolates Fall 2021 policy changes from other policies that were in place before Fall 2021. Moreover, it is likely that institutions with such high vaccination levels had a lower risk perception, leading to more interactions; thus, our estimation points to a lower bound effect of mandating vaccination.

This paper has several limitations that are avenues for future research. Obtaining individual-level data and a larger sample of universities, gathering test frequency data before and after the intervention, and tracking cases during summer semesters to improve cross-university comparison are some of the potential extensions. Future studies should also examine several related behavioral factors that influence individuals’ change in risk perception and willingness to vaccinate (Rahmandad et al., 2022). Common methodological assumptions such as consistent trends across universities, while consistent with our intuitions, are still strong assumptions, and given the lack of data on summer semesters are hard to fully confirm. In addition, policy changes in institutions, after the Spring semester and before the Fall semester, that might be hidden to the researcher, may have influenced the results. However, our robustness analyses provide additional supports for the main argument of the paper, and especially show that the observed effect of the policy is very likely to be via increasing vaccination rates. Nevertheless, we invite other methodological approaches (e.g., Darabi & Hosseinichimeh, 2020) including studies that directly work with stakeholders (here students, faculty, staff, and administration) (Wilkerson et al., 2020) to shed more lights on this institutional, health policy challenge.

Keeping students and employees safe from COVID-19 is going to be a major responsibility in higher education in the years to come (Paltiel, 2021), and operational challenges will continue. Several institutions will continue to deal with the tradeoff between health and quality education and will prefer to maintain in-person instruction. Waning immunity (Goldberg et al., 2021) and changes in seasons and thus temperature (Xu et al., 2021) can lead to higher rates of the spread of the virus; consequently, proactive institutional policies should be in place. The results of this study support a policy framework of “requiring,” rather than simply “encouraging,” vaccination at colleges and universities.

Declaration of Competing Interest

The author declares that he has no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.