Risk benefit analysis to evaluate risk of thromboembolic events after mRNA COVID-19 vaccination and COVID-19

Abstract

We compared the risks and benefits of COVID-19 vaccines using a causal pathway analysis to weigh up possible risk factors of thromboembolic events post-vaccination. The self-controlled case series (SCCS) method examined the association between thromboembolic events and vaccination while a case-control study assessed the association between thromboembolic events and COVID-19, addressing under-reported infection data issues. The net vaccine effect was estimated using results from SCCS and case-control studies. We used electronic health record data from Corewell Health (16,640 subjects in SCCS and 106,143 in case-control). We found increased risks of thromboembolic events post-vaccination (incidence rate ratio: 1.19, 95% CI: [1.08, 1.31] after the first dose; 1.22, 95% CI: [1.11, 1.34] after the second dose). Vaccination attenuated infection-associated thromboembolic risks (odds ratio: 4.65, 95% CI: [4.18, 5.17] in unvaccinated vs 2.77, 95% CI: [2.40, 3.24] in vaccinated). After accounting for vaccine efficacy and protection against infection-associated thromboembolic events, vaccination decreases thromboembolic event risk, especially during high infection rate periods.

Introduction

The Coronavirus Disease 2019 (COVID-19) pandemic prompted a race to develop and distribute effective vaccines. Approximately 81.4% of the US population have been vaccinated with at least one dose, and 69.5% have completed the primary series of COVID-19 vaccination¹. While the benefits of vaccination are widely acknowledged, concerns have emerged regarding the development of thromboembolic events after vaccination². Phase 3 clinical trials were not statistically powered to identify rare adverse events³. The risks of new vaccines were not fully known during regulatory approval, particularly for mRNA-based vaccines (mRNA-1273 or BNT162b2), which were under authorized emergency use. Therefore, it is important to conduct post-marketing safety surveillance of the vaccines. More specifically, cases of venous thromboembolism following a mRNA-based vaccination were reported in 2022 after COVID-19 vaccines were administered in the US and some other countries^4–7, drawing attention to the potential risk of thromboembolic events after the first vaccination dose. One study confirmed an increased risk of thromboembolism, ischemic stroke, and cerebral venous sinus thrombosis after the first dose of BNT162b2⁸, and another retrospective cohort study found an increased risk of cerebral venous thrombosis and portal vein thrombosis after any mRNA-based vaccination⁹. Moreover, a recent systematic review¹⁰ has shown that thromboembolism is the most frequent cardiovascular complication following a mRNA-based vaccination. Despite those findings, vaccination is still recommended to reduce the likelihood of COVID-19, hospitalization, and mortality^8,11. Furthermore, COVID-19 itself substantially increases the risk of thromboembolic events^12–18, with a more prolonged and significant threat compared to vaccine-associated risks⁸. Therefore, studying the risk of thromboembolic events after COVID-19 vaccination should incorporate the protective effect of vaccines against COVID-19 severity and hence COVID-19-associated thromboembolic events.

Several studies have reported a positive correlation between thromboembolic events and mRNA-based vaccines, with reported incidence rate ratios (IRRs) between 1.04 and 1.22^8,19–22. These studies used the self-controlled case series²³ (SCCS) design, which is a standard approach to studying adverse events of vaccines. The same design was used to evaluate the risk of thromboembolic events after COVID-19, with reported IRRs between 6.18 and 63.52^8,11,14. However, since a thromboembolic event typically requires a hospital visit (emergency visit or hospital admission), subjects with a thromboembolic event are subject to a higher rate of COVID-19 testing, and so at a lower likelihood of misclassification as uninfected compared to subjects without an event. Hence, the SCCS design is subject to some risks of bias²⁴, which we would expect to inflate the SCCS estimated relative risk (RR) of thromboembolic events after COVID-19.

The objective of this study is to evaluate whether the overall effect of the COVID-19 vaccination is to increase or decrease the risk of thromboembolic events. To do so, we first quantified the risk of thromboembolic events after mRNA-based vaccination using the SCCS method. Secondly, we evaluated the association between thromboembolic events and COVID-19 using a case-control study, avoiding the misclassification bias associated with the SCCS method. Finally, we conducted a risk-benefit analysis by comparing the magnitude of the increased risk through the direct effect of the COVID-19 vaccination with the reduced risk through the indirect pathway via protection against infection-associated thromboembolic events.

Methods

Data

Our studies used electronic health record (EHR) data from the Corewell Health East (CHE, formerly known as Beaumont Health) and Corewell Health West (CHW, formerly known as Spectrum Health) healthcare systems, which includes demographics, mortality, hospital admissions, and COVID-19 testing. We obtained accurate COVID-19 vaccination records (vaccine types, dates, and doses) by linking EHR data at Corewell Health with the Michigan Care Improvement Registry (MCIR), giving more complete data for individuals who received the COVID-19 vaccines outside the healthcare system. We included all patients aged ≥ 18-years-old and were registered with a primary care physician within 18 months before Jan 1st, 2021.

We identified thromboembolic events based on ICD-10 (International Classification of Diseases version 10) codes from a hospital visit (emergency visit or hospital admission). These ICD-10 codes represent diagnoses for venous thromboembolism, arterial thrombosis, cerebral venous sinus thrombosis, ischemic stroke, and myocardial infarction (Supplementary Table 1). We also used patients with physical injury at a hospital visit (list of ICD-10 codes in Supplementary Table 2) to identify potential bias related to the misclassification and further leveraged them as a control group to estimate the effect of COVID-19 on thromboembolic events.

Estimate effect of mRNA-vaccination on thromboembolic events

We used the SCCS design to examine the association of thromboembolic events and the first two doses of mRNA-based COVID-19 vaccines (mRNA-1273 or BNT162b2) from December 1st, 2020, to August 31st, 2022. The SCCS method compares the incidence rate of thromboembolic events before and after vaccination. In this method, subjects are under their own control, and comparisons are made within subjects, thus avoiding any time-invariant confounding. We included subjects who had a thromboembolic event and received at least one dose of the primary series of mRNA-based vaccines in the study period. The control period was defined from December 1st, 2020, to 28 days before the first dose of vaccination, excluding the period of 28 days prior to vaccination to avoid bias due to contra-indications²⁵. Two separate risk periods for the first and second doses were defined until 28 days after vaccination, death, or August 31st, 2022, whichever occurred first (Supplementary Fig. 1). We also excluded subjects who had COVID-19 within 90 days before a thromboembolic event to remove the confounding effect of infection on that event. We used a conditional Poisson regression²² with an offset for the length of each period to estimate the IRRs of dose one and dose two simultaneously. Specifically, the model has an independent variable of the period with three categories (control periods, and two risk periods after the first and second dose). Using the control period as the reference, we derived the IRRs for the two doses. As Poisson regression assumes the independence between recurrent events, therefore, we considered only events that occurred at least one year after the previous events.

Estimate effect of COVID-19 on thromboembolic events

In an initial analysis of the association between thromboembolic events and COVID-19, we used the SCCS design and included patients who had at least one positive COVID-19 test (PCR or antigen) and a thromboembolic event at a hospital visit during the same period as in the previous study of vaccination. However, due to the missing infection data in patients who did not have any hospital visits for thromboembolic events or other reasons, the SCCS design resulted in a biased estimate of the association between thromboembolic events and COVID-19. Patients visiting the hospital, almost always received a COVID-19 (PCR or antigen) test, especially early in the pandemic, while patients who did not visit the hospital were subject to underreporting infection data. This underreporting (or misclassification of infected as uninfected) led to an inflated IRR of thromboembolic events after COVID-19.

We proposed a simple and efficient method to quantify the association between thromboembolic events and COVID-19 while dealing with the misclassification issue. The main idea is to select a subset of control (i.e., subjects without thromboembolic events) who had a hospital visit for reasons independent of COVID-19 and therefore had complete infection data. To this end, we used patients who had a diagnosis code for physical injury (see Supplementary Table 2) at a hospital visit as the control group, since we would not expect any causal association between physical injury and COVID-19. We used a case-control design, in which patients with a thromboembolic event are considered as cases, and patients with a physical injury are considered as controls. If an individual had multiple hospital visits for thromboembolic events or physical injuries, we considered only the first visit. As physical injuries can be risk factors for thromboembolic events^26,27, we therefore excluded patients who experienced both events at the same visit. We determined the COVID-19 status based on the COVID-19 test results during the 28 days prior to the date of the event (Supplementary Fig. 2). If an individual had a positive test result, this subject was classified as exposed to COVID-19, otherwise, unexposed. We compared the odds of infection (exposed) vs no infection (unexposed) in the cases (with thromboembolic events) vs controls (with physical injury) using a logistic regression model adjusted for age, race, gender, Charlson comorbidity index (CCI), number of visits, and prior vaccination status (yes/no). Patients who had any COVID-19 vaccine between the date of the positive COVID-19 test and the date of the event were removed. The number of visits was fit with a natural spline with three degrees of freedom. The CCI was obtained using the R package comorbidity and categorized into four categories, ‘0’, ‘1–2’, ‘3–4’, and ‘≥5’^28,29. Analyses were done after excluding patients with incomplete covariate data.

Estimate the net effect of mRNA-vaccination on thromboembolic events: a risk-benefit analysis

COVID-19 vaccines are protective against COVID-19 and COVID-19 severity^30–32, and so can indirectly decrease the likelihood of experiencing a thromboembolic event. Hence, we conducted a risk-benefit analysis to estimate the net RR of thromboembolic events after vaccination by considering the role of vaccination in preventing infection-associated thromboembolic events. Figure 1 illustrates the direct and indirect effect of the COVID-19 vaccination on the occurrence of thromboembolic events while considering vaccine efficacy (VE). As presented in the diagram, the association between thromboembolic events and COVID-19 vaccination is described by two paths, the direct association between thromboembolic events and vaccination, and the indirect association between thromboembolic events and vaccination via potential reduction in the risk of thromboembolic events through decreasing the risk of COVID-19. We estimated the overall influence of vaccination on the occurrence of thromboembolic events by considering both direct and indirect paths.

Fig. 1. Direct and indirect associations between thromboembolic events and mRNA vaccines.

COVID-19 (I), individuals with COVID-19. COVID-19 vaccination (V), individuals with COVID-19 vaccines. Thromboembolic events (Y), individuals with thromboembolic events. V → I indicates vaccine effect (VE) in preventing COVID-19, V → Y indicates the risk of thromboembolic events after COVID-19 vaccination, I → Y indicates the risk of thromboembolic events after COVID-19, V → Y (via I) indicates the risk of thromboembolic events after vaccination accounting for vaccine effect in reducing infection-associated thromboembolic events.

Let $\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)$ and $\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)$ be the probability of COVID-19 ($\mathrm{I})$ in vaccinated ($\mathrm{V}$) and unvaccinated ($\stackrel{\circledR }{\mathrm{V}}$) subjects, respectively. Let $\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{V}},\overline{\mathrm{I}}\right),\mathrm{P}\left(\mathrm{Y}\mid \mathrm{V},\overline{\mathrm{I}}\right),\mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\overline{\mathrm{V}}\right),$ and $\mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\mathrm{V}\right)$ be the probability (or risk) of thromboembolic events ($\mathrm{Y})$ in unvaccinated and uninfected, vaccinated and uninfected, unvaccinated and infected, and vaccinated and infected subjects, respectively.

With the above notations, for a vaccinated subject, the total risk of thromboembolic events is $\mathrm{P}\left(\mathrm{Y}\mid \mathrm{V},\overline{\mathrm{I}}\right)+\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)\times \mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\mathrm{V}\right)$, where the product $\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)\times \mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\mathrm{V}\right)$ is the indirect risk calculated by multiplying the risk of COVID-19 of a vaccinated subject and the risk of thromboembolic events given a COVID-19 in the vaccinated group. Similarly, the overall risk of thromboembolic events for an unvaccinated subject is given by $\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{V}},\overline{\mathrm{I}}\right)+\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times \mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\overline{\mathrm{V}}\right)$. Hence the net RR (${\mathrm{RR}}_{\mathrm{Net}}$) of thromboembolic events for a vaccinated subject compared to an unvaccinated subject is

$$\begin{array}{c}{\mathrm{RR}}_{\mathrm{Net}}=\frac{\mathrm{P}\left(\mathrm{Y}\mid \mathrm{V},\overline{\mathrm{I}}\right)+\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)\times \mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\mathrm{V}\right)}{\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{V}},\overline{\mathrm{I}}\right)+\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times \mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\overline{\mathrm{V}}\right)}\hfill \\ =\frac{\frac{\mathrm{P}\left(\mathrm{Y}\mid \mathrm{V},\overline{\mathrm{I}}\right)}{\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{V}},\overline{\mathrm{I}}\right)}+\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)\times \frac{\mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\mathrm{V}\right)}{\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{I}},\overline{\mathrm{V}}\right)}}{1+\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times \frac{\mathrm{P}\left(\mathrm{Y}\mid \mathrm{I},\overline{\mathrm{V}}\right)}{\mathrm{P}\left(\mathrm{Y}\mid \overline{\mathrm{I}},\overline{\mathrm{V}}\right)}}\hfill \\ =\frac{{\mathrm{RR}}_{\mathrm{V}}+\mathrm{P}\left(\mathrm{I}\mid \mathrm{V}\right)\times {\mathrm{RR}}_{\mathrm{IV}}}{1+\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times {\mathrm{RR}}_{\mathrm{I\mid }\overline{\mathrm{V}}}}\hfill \end{array}$$ 1

The terms ${\mathrm{RR}}_{\mathrm{V}}$ is the RR of thromboembolic events comparing vaccinated versus unvaccinated in subjects without COVID-19, and ${\mathrm{RR}}_{\mathrm{I|}\stackrel{\circledR }{\mathrm{V}}}$ is the RR of thromboembolic events comparing subjects with and without COVID-19 in the unvaccinated group. The term ${\mathrm{RR}}_{\mathrm{IV}}$ is the RR of thromboembolic events in subjects who have both vaccination and infection, compared to the group of subjects who do not have any exposures.

We further defined VE as $\mathrm{VE}=1-\mathrm{P}\left(\mathrm{I|V}\right)/\mathrm{P}\left(\mathrm{I|}\stackrel{\circledR }{\mathrm{V}}\right)$, then plugged VE into Eq. (1) to obtain

$${\mathrm{RR}}_{\mathrm{Net}}=\frac{{\mathrm{RR}}_{\mathrm{V}}+\left(1-\mathrm{VE}\right)\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times {\mathrm{RR}}_{\mathrm{IV}}}{1+\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)\times {\mathrm{RR}}_{\mathrm{I\mid }\overline{\mathrm{V}}}}$$ 2

If ${\mathrm{RR}}_{\mathrm{Net}}$ is smaller than one, COVID-19 vaccination offers protection against thromboembolic events, with a lower ${\mathrm{RR}}_{\mathrm{Net}}$ implying a stronger protection.

Statistical analyses were performed in R 4.3.0. We reported odds ratio (OR) and IRR with 95% CIs and p-values from the two-sided test. We generated a figure for ${\mathrm{RR}}_{\mathrm{Net}}$ over a range of VE values based on the estimates of ORs and IRRs.

We used de-identified EHR data, the use of which was approved by the Institutional Review Board of Corewell Health.

Results

Study population

During the study period from December 1st, 2020, to August 31st, 2022, there were 747,070 subjects at Corewell Health who received mRNA-based vaccines, among which 279,229 (37.38%) had the primary series of mRNA-1273 and 467,841 (62.62%) took BNT162b2. Overall, the number of fully vaccinated patients was 711,460 (95.23%), and 35,610 (4.77%) patients received only one dose. The median age was 57 (with interquartile range [IQR]: 40–69), and 59.81% of patients were female. There were 367,105 patients taking at least one COVID-19 test (antigen or PCR), among which 78,568 (21.4%) patients received positive results. The median age was 52 (with interquartile range [IQR]: 34–67), and 61.44% of patients were female.

In the study cohort of vaccination exposure, there were 16,640 patients who had at least one thromboembolic event and had the first dose of either mRNA-1273 or BNT162b2 vaccine. Patient demographics are presented in Table 1. We identified 2724 events in the control period, 722 events within 28 days after the first dose, and 786 events within 28 days after the second dose.

Table 1.

SCCS cohort to study COVID-19 vaccination exposure on thromboembolic events

	Patient characteristics in SCCS study
	Dose 1, (N = 16,640)	Dose 2, (N = 15,744)
AGE
18–30	204 (1.23%)	179 (1.14%)
31–50	1505 (9.04%)	1379 (8.76%)
51–65	4216 (25.34%)	3973 (25.24%)
>65	10,715 (64.39%)	10,213 (64.87%)
GENDER
Female	8071 (48.50%)	7631 (48.47%)
Male	8569 (51.50%)	8113 (51.53%)
RACE
White race	14,803 (80.16%)	12,641 (80.29%)
Black race	2432 (14.62%)	2257 (14.34%)
Other	896 (5.38%)	846 (5.37%)
CCI
0	6579 (39.54%)	6203 (39.40%)
1–2	5457 (32.79%)	5178 (32.89%)
3–4	2790 (16.77%)	2654 (16.86%)
≥5	1814 (10.90%)	1709 (10.85%)
NUM. THROMBOEMBOLIC EVENTS IN RISK PERIODS
	722	786

Characteristics of patients who experienced at least one thromboembolic event and had at least one dose of mRNA-based vaccination from December 1st, 2020 to August 31st, 2022.

In the study cohort of COVID-19 exposure, there were 18,004 patients who had a thromboembolic event (cases) and 88,139 patients who had a physical injury (controls) at a hospital visit. 16.96% of cases and 1.48% of controls had COVID-19 within 28 days before the event. Demographics of patients are presented in Table 2.

Table 2.

Case-control cohort to study COVID-19 exposure on thromboembolic events

Patient characteristics in case-control study
	Thromboembolic events, N = 18,004 (16.96%)	Physical injury, N = 88,139 (83.04%)	Overall, N = 106,143
AGE
18–30	344 (1.91%)	15,334 (17.40%)	15,678 (14.77%)
31–50	2164 (12.02%)	21,711 (24.63%)	23,875 (22.49%)
51–65	5098 (28.32%)	20,526 (23.29%)	25,624 (24.14%)
>65	10,398 (57.75%)	30,568 (34.68%)	40,966 (38.60%)
GENDER
Female	8703 (48.34%)	50,795 (57.63%)	59,498 (56.05%)
Male	9301 (51.66%)	37,344 (42.37%)	46,645 (43.95%)
RACE
White Race	14,159 (78.64%)	68,640 (77.88%)	82,799 (78.01%)
Black Race	2764 (15.35%)	13,071 (14.83%)	15,835 (14.92%)
Other	1081 (6.00%)	6428 (7.29%)	7509 (7.07%)
CCI
0	8008 (44.48%)	50,342 (57.12%)	58,350 (54.97%)
1–2	5584 (31.02%)	22,328 (25.33%)	27,912 (26.30%)
3–4	2674 (14.85%)	9077 (10.30%)	11,751 (11.07%)
≥5	1738 (9.65%)	6392 (7.25%)	8130 (7.66%)
NUMBER OF HOSPITAL VISITS (IQR)
Median (IQR)	7 (4–11)	7 (4–12)	7 (4–12)
COVID-19 VACCINATION BEFORE INFECTION
NO	7989 (44.37%)	44,730 (50.75%)	52,719 (49.67%)
YES	10,015 (55.63%)	43,409 (49.25%)	53,424 (50.33%)
INFECTION WITHIN 28 DAYS PRIOR TO THE DATE OF THE EVENT
NO	16,875 (93.73%)	86,838 (98.52%)	103,713 (97.71%)
YES	1129 (6.27%)	1301 (1.48%)	2430 (2.29%)

Characteristics of patients who experienced a thromboembolic event or hospitalized injury event from December 1st, 2020 to August 31st, 2022.

Estimate effect of mRNA-vaccination on thromboembolic events

Based on the SCCS analysis, we found an increased risk of thromboembolic events 28 days after the first dose (IRR = 1.19, 95% confidence interval (CI): [1.08, 1.31], p-value < 0.001), and after the second dose (IRR = 1.22, 95% CI: [1.11, 1.34], p-value < 0.001) of the mRNA-based vaccines.

We studied the risk of thromboembolic events in a 28-day window after vaccination based on prior research⁸. An event that occurs in a short period (such as 28 days) is more likely to be attributable to the vaccines. We also conducted a sensitivity analysis using a 60-day window after vaccination. The conclusions remained the same with slightly lower IRRs (IRR = 1.13, 95% CI: [1.03, 1.24] after the first dose, and IRR = 1.14, 95% CI: [1.05, 1.3] after the second dose).

Supplementary Figs. 3 and 4 show the IRRs for subgroup analyses by age (“18–31”, “31–50”, and “≥51”) and gender (female/male). We found that the effects of vaccination on thromboembolic events were similar between age groups and gender groups.

Estimate effect of COVID-19 on thromboembolic events

Naïve SCCS analysis showed a very large increased risk of thromboembolic events associated with COVID-19 (IRR = 19.36, 95% CI: [17.64, 21.26], p-value < 0.001). However, a similar analysis using the physical injury as an event also derived a large increased risk (IRR = 3.31, 95% CI: [3.10, 3.54], p-value < 0.001), indicating misclassification bias as COVID-19 should not substantially increase the risk of physical injury. In the case-control analysis with controls having a physical injury, we found that COVID-19 increased the risk of thromboembolic events but with a much smaller magnitude than the risk in the SCCS analysis (although it is still larger than the vaccination exposure). Moreover, the degree of the increased risks was modified by vaccination status (Fig. 2). The reported OR for the unvaccinated group was 4.65 (95% CI: [4.18, 5.17], p-value < 0.001) compared to 2.77 (95% CI: [2.40, 3.24], p-value < 0.001) for the vaccinated group. We observed the increased risks of thromboembolic events after COVID-19 in both groups, but vaccination appears to confer some protection against infection-associated thromboembolic events, given the lower OR. Alternatively, we divided the vaccinated group into four categories based on the time to the last vaccination (“≥365 days”, “180–365 days”, “90–180 days”, and “<90 days”). The effects of COVID-19 on thromboembolic events were similar across the four vaccinated groups. The results are in Supplementary Fig. 5.

Fig. 2. Forest plot for the case-control study for the association between thromboembolic events and COVID-19.

OR is denoted by a solid circle and a 95% CI is represented by a line. The x-axis is plotted on the natural log scale. CCI Charlson comorbidity index. Infection or non-infection refers to COVID-19.

We also conducted two sensitivity analyses. In the first analysis, rather than adjusting for the CCI, we adjusted individual risk factors that might be related to a thromboembolic event. These are congestive heart failure, peripheral vascular disease, cerebrovascular disease, chronic pulmonary disease, diabetes with complications, cancer, moderate or severe liver disease, and metastatic solid tumors. We included the above eight risk factors (present or absent) in the logistic regression model. The effect of COVID-19 on the outcome of thromboembolic events was similar to the analysis with CCI. Results can be found in Supplementary Fig. 6.

We assumed that patients who visited hospitals were routinely tested for COVID-19, especially during the early pandemic. Based on Corewell Health’s policy, patients who visited the healthcare system before March 1st, 2022, were tested for COVID-19. In our study cohort, 74.05% of participants had a hospital visit before March 1st, 2022. We conducted a sensitivity analysis using only these patients and the conclusions remained the same. See results in Supplementary Fig. 7.

Estimate the net effect of mRNA-vaccination on thromboembolic events: a risk-benefit analysis

Our analysis in the previous sections gave an IRR of 1.22 as the measure of the association between thromboembolic events and the second dose of COVID-19 vaccination, therefore, we set ${\mathrm{RR}}_{\mathrm{V}}$ = 1.22. We also obtained odd ratios ${\mathrm{OR}}_{\mathrm{I|}\stackrel{\circledR }{\mathrm{V}}}$ = 4.65 and ${\mathrm{OR}}_{\mathrm{IV}}$ = 2.82 from the analysis using the case-control design. Since the RR is very close to the OR when the event is rare, we therefore set ${\mathrm{RR}}_{\mathrm{I|}\stackrel{\circledR }{\mathrm{V}}}$ = 4.65 and ${\mathrm{RR}}_{\mathrm{IV}}$ = 2.82, as the thromboembolic events are rare³³. Hence, plugging these estimators into Eq. (2), the ${\mathrm{RR}}_{\mathrm{Net}}$ becomes

$${\mathrm{RR}}_{\mathrm{Net}}=\frac{1.22+2.82\times \left(1-\mathrm{VE}\right)\mathrm{P}\left(\mathrm{I}\mid \overline{\mathrm{V}}\right)}{1+4.65\times \mathrm{P}\left(\mathrm{I\mid }\overline{\mathrm{V}}\right)}$$

Figure 3 illustrates the ${\mathrm{RR}}_{\mathrm{Net}}$ of thromboembolic events after COVID-19 vaccination as a function of VE. As VE increases from 0 to 1, ${\mathrm{RR}}_{\mathrm{Net}}$ decreases and reaches a point where vaccine benefits outweigh the harms. Specifically, vaccines with higher VE offer higher protection against thromboembolic events. For example, the effectiveness of mRNA-based COVID-19 vaccines against infection was 61% during the Delta period and 46% during the Omicron period^34–36. Given an infection rate of 0.08 among unvaccinated subjects, the risk of thromboembolic events was decreased by 4.62% in the Delta period, which is higher than 2.07% in the Omicron period. Moreover, vaccines offer stronger protection during periods with higher infection rates. For example, with the infection rate of 0.1 in unvaccinated subjects, the reduction of the risk of thromboembolic events was higher (by 9.19% in Delta and 6.23% in the Omicron period), compared to the scenario when the infection rate was 0.08.

Fig. 3. Net RR of thromboembolic events comparing vaccinated and unvaccinated subjects.

The x-axis is VE, and the y-axis is the net RR of thromboembolic events.

The list of ICD-10 codes for thromboembolic events is based on a previous publication⁸, including old myocardial infarction (I252). Old myocardial infarction (I252) reports for any myocardial infarction described as older than four weeks. However, our study cohort removed subjects with an I252 code who had any thromboembolic event with ICD-10 codes listed in Table S1 in the prior year. Therefore, we can consider observing I252 in the study period as a new incidence. There were 20,002 (18.84%) patients with a hospital visit associated with the I252 code. We conducted a sensitivity analysis by excluding these patients and the conclusions did not change. The estimated IRRs of thromboembolic events are 1.16 and 1.17 after vaccine dose 1 and dose 2, respectively, which are slightly smaller than the original results including the I252 code (IRRs were 1.19 and 1.22 after the first and second dose). The association between COVID-19 and thromboembolic events is higher in the unvaccinated group (OR = 5.77 without I252 and OR = 4.65 with I252) and similar in the vaccinated group (OR = 2.80 without I252 and OR = 2.77 with I252). Hence, given the same infection rate and VE, vaccination offered a stronger protection, compared to the analysis with the I252 codes. For example, given an infection rate in the unvaccinated population of 0.08 and a VE of 0.8, vaccination lowers the risk of thromboembolic events by 17.14% without I252, compared to 6.67% in the analysis with I252. Detailed results are in Supplementary Figs. 8 and 9. We considered the analysis that includes the I252 code as the main analysis to represent more conservative results.

Discussion

We found that both COVID-19 vaccination and COVID-19 increase the risk of thromboembolic events. However, evidence implies that the likelihood of experiencing a thromboembolic event after COVID-19 is much higher than after vaccination. Our analysis agrees with previous research, indicating that COVID-19 is a more dangerous risk factor for thromboembolic events than vaccination^8,11–14.

Different from existing work, we evaluated the association between thromboembolic events and COVID-19 using a case-control study, avoiding the misclassification issue associated with the SCCS design. We also studied the effect of prior vaccination on reducing infection-associated thromboembolic events. Moreover, we included both COVID-19 vaccination and COVID-19 in the analysis of the risk of thromboembolic events and conducted a risk-benefit analysis by comparing the magnitude of the increased risk through the direct effect of COVID-19 vaccination with the reduced risk through the indirect pathway via protection against severe diseases. Our analysis provides evidence that COVID-19 vaccination directly increases the risk of thromboembolic events, but indirectly reduces the risk of infection-associated events. Results show that the indirect benefit of preventing infection-associated thromboembolic events outweighs the direct harm if the VE and infection rate reaches certain levels. Moreover, COVID-19 vaccination may have additional benefits in preventing thromboembolic events associated with COVID-19, as a higher rate of vaccination increases the overall level of immunity in the population, reducing the spread of the virus and conferring collective protection against infection-associated thromboembolic events and other health risks associated with COVID-19.

There are several limitations to this study. First, using ICD-10 codes to identify thromboembolic events may be subject to phenotype errors. Second, Corewell Health has 22 hospitals, and the catchment area for these hospitals is across many counties, hence patients may seek care at other facilities outside the Corewell Health system, leading to missing data such as infection data. To deal with the missing infection data, we used the case-control study. Moreover, the use of a prior number of hospital visits as covariates in the regression model mitigates the bias due to differing degrees of interaction with the Corewell Health system between infected and control subjects. However, patients with a hospital visit due to injuries may not be the perfect control group, but it is clearly better than a control group of patients without thromboembolic events. Therefore, we may not totally correct the bias, but we reduce it. Finally, the study population for vaccine doses 1 and 2 are different. If a subject had a thromboembolic event after the first vaccine dose, this subject is unlikely to receive the second dose, therefore, the population who received the second dose only includes subjects who did not have a thromboembolic event after the first dose.

Despite these limitations, our study makes a critical contribution to quantifying the net risk of thromboembolic events associated with COVID-19 vaccination. It accounts for both the direct effects of vaccination and the indirect effects of protection against COVID-19 and severe diseases. The dual consideration is vital for a comprehensive understanding of the risk-benefit profile. The mechanism of vaccination is to simulate the immune response the body has against infection using a dead/attenuated virus or mRNA, which can lead to side effects similar to those of the virus, albeit in a less severe form (e.g., thromboembolic events, myocarditis³⁷, acute kidney injury^38,39). Our finding highlights the necessity of evaluating both the indirect benefits and direct harms of vaccination to provide a complete and accurate assessment of vaccine safety. This comprehensive approach ensures a balanced understanding of the risks and the benefits, reinforcing the overall safety and efficacy of vaccination programs.

Our risk-benefit analysis was conducted on the population level. This analysis can also be stratified by patient groups of interest. For example, the risk-benefit of vaccination might be different between older and younger populations. Moreover, our findings are for a broad range of thromboembolic conditions, so more research is needed on the specific biological mechanisms connecting COVID-19 and mRNA vaccination to these events, both to establish causality and help identify a more specific set of conditions or risk factors.

Author contributions

H.N.Q.T.: manuscript writing, study design, statistical analysis, and data preparation. M.R.: manuscript writing and study design. G.B.: clinical advice and study design. L.Z.: manuscript writing, method development, study design, and statistical analysis.

Data availability

The datasets analyzed during the current study are not publicly available due to privacy or ethical restrictions.

Code availability

Code for this study is available from the corresponding author on request.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41541-024-00960-7.