Skip to main content
PMC home page
JAMA Network logoLink to JAMA Network
. 2022 Apr 14;5(4):e227038. doi: 10.1001/jamanetworkopen.2022.7038

Surveillance of Safety of 3 Doses of COVID-19 mRNA Vaccination Using Electronic Health Records

Michiel J M Niesen 1, Colin Pawlowski 1, John C O’Horo 2,3, Doug W Challener 2, Eli Silvert 1, Greg Donadio 1, Patrick J Lenehan 1, Abinash Virk 2, Melanie D Swift 4, Leigh L Speicher 5, Joel E Gordon 6, Holly L Geyer 7, John D Halamka 8, A J Venkatakrishnan 1, Venky Soundararajan 1,✉, Andrew D Badley 2,9
PMCID: PMC9011130  PMID: 35420661

This cohort study assesses adverse events associated with a third dose of mRNA COVID-19 vaccine using data from electronic health records (EHRs).

Key Points

Question

Is a third dose of US Food and Drug Administration–authorized COVID-19 mRNA vaccines safe?

Findings

This cohort study of electronic health record data for 47 999 individuals receiving 3-dose mRNA COVID-19 vaccines found no significant increase in the reporting of severe adverse events (ie, anaphylaxis, cerebral venous sinus thrombosis, myocarditis, and pericarditis) after the third vaccine dose compared with before vaccination and after prior doses. Significantly increased reporting was found for low-severity adverse events (ie, fatigue, lymphadenopathy, nausea, and headache).

Meaning

These findings suggest that third-dose vaccination with COVID-19 mRNA vaccines may be safe.

Abstract

Importance

Recent reports on waning of COVID-19 vaccine–induced immunity have led to the approval and rollout of additional doses and booster vaccinations. Individuals at increased risk of SARS-CoV-2 infection are receiving additional vaccine doses in addition to the regimen that was tested in clinical trials. Risks and adverse event profiles associated with additional vaccine doses are currently not well understood.

Objective

To evaluate the safety of third-dose vaccination with US Food and Drug Administration (FDA)–approved COVID-19 mRNA vaccines.

Design, Setting, and Participants

This cohort study was conducted using electronic health record (EHR) data from December 2020 to October 2021 from the multistate Mayo Clinic Enterprise. Participants included all 47 999 individuals receiving 3-dose COVID-19 mRNA vaccines within the study setting who met study inclusion criteria. Participants were divided into 2 cohorts by vaccine brand administered and served as their own control groups, with no comparison made between cohorts. Data were analyzed from September through November 2021.

Exposures

Three doses of an FDA-authorized COVID-19 mRNA vaccine, BNT162b2 or mRNA-1273.

Main Outcomes and Measures

Vaccine-associated adverse events were assessed via EHR report. Adverse event risk was quantified using the percentage of study participants who reported the adverse event within 14 days after each vaccine dose and during a 14-day control period, immediately preceding the first vaccine dose.

Results

Among 47 999 individuals who received 3-dose COVID-19 mRNA vaccines, 38 094 individuals (21 835 [57.3%] women; median [IQR] age, 67.4 [52.5-76.5] years) received BNT162b2 (79.4%) and 9905 individuals (5099 [51.5%] women; median [IQR] age, 67.7 [59.5-73.9] years) received mRNA-1273 (20.6%). Reporting of severe adverse events remained low after the third vaccine dose, with rates of pericarditis (0.01%; 95% CI, 0%-0.02%), anaphylaxis (0%; 95% CI, 0%-0.01%), myocarditis (0%; 95% CI, 0%-0.01%), and cerebral venous sinus thrombosis (no individuals) consistent with results from earlier studies. Significantly more individuals reported low-severity adverse events after the third dose compared with after the second dose, including fatigue (2360 individuals [4.92%] vs 1665 individuals [3.47%]; P < .001), lymphadenopathy (1387 individuals [2.89%] vs 995 individuals [2.07%]; P < .001), nausea (1259 individuals [2.62%] vs 979 individuals [2.04%]; P < .001), headache (1185 individuals [2.47%] vs 992 individuals [2.07%]; P < .001), arthralgia (1019 individuals [2.12%] vs 816 individuals [1.70%]; P < .001), myalgia (956 individuals [1.99%] vs 784 individuals [1.63%]; P < .001), diarrhea (817 individuals [1.70%] vs 595 individuals [1.24%]; P < .001), fever (533 individuals [1.11%] vs 391 individuals [0.81%]; P < .001), vomiting (528 individuals [1.10%] vs 385 individuals [0.80%]; P < .001), and chills (224 individuals [0.47%] vs 175 individuals [0.36%]; P = .01).

Conclusions and Relevance

This study found that although third-dose vaccination against SARS-CoV-2 infection was associated with increased reporting of low-severity adverse events, risk of severe adverse events remained comparable with risk associated with the standard 2-dose regime. These findings suggest the safety of third vaccination doses in individuals who were eligible for booster vaccination at the time of this study.

Introduction

Although clinical trials1,2 and early studies3,4 have found that mRNA-based COVID-19 vaccines were associated with high effectiveness in prevention of SARS-CoV-2 infection and decreased COVID-19 severity, several recent studies suggest that vaccine effectiveness against milder disease may be waning.5,6,7,8,9 This is likely associated with waning immunity, as well as poorer immune response in certain groups with increased risk, such as individuals who are immune compromised.10,11 This has prompted health policy discussions on the need for additional or booster vaccine doses.

Initial studies indeed found that third-dose vaccination was associated with improved protection against SARS-CoV-2 infection.12,13 An additional vaccine dose may be associated with particularly beneficial outcomes among individuals at increased risk of breakthrough infections or severe COVID-19, such as patients who are immunocompromised, as suggested by measurement of antibody levels in adults with solid tumors.14 However, despite these potential benefits, it is essential to monitor the safety of additional vaccine doses beyond the primary series as they are administered to the general public.

Extensive research has found that the standard 2-dose regimen of mRNA-based COVID-19 vaccines is associated with relatively safe outcomes. Although severe adverse events, such as anaphylaxis,15 myocarditis,16,17 and blood clotting,18,19,20,21,22 have been reported after COVID-19 vaccination, these outcomes are rare, and the benefit of vaccination is deemed to outweigh the potential risks. The most common adverse reactions occur immediately after vaccination23 and are relatively mild, including headache, fatigue, pains, low-grade fever, and nausea. Results from initial studies, including clinical trials,13,24 self-reporting of adverse events in a small cohort of individuals receiving third vaccine doses,25 and analysis of voluntary reports in the Centers for Disease Control and Prevention v-safe health-check-in application,26 suggest that additional vaccination doses may also be safe, although further study in a broader study population is needed.

In addition to existing studies, evidence from clinical practice extracted from electronic health records (EHRs) may help evaluate the safety of additional COVID-19 vaccination doses. The advantage of this method for collecting adverse event reports is that a large and diverse cohort of individuals can be readily included, with fewer barriers associated with study enrollment or self-reporting. However, successful extraction of vaccine-associated adverse events from EHR data is challenging owing to the vast amount of data involved and difficulty in verifying that mention of a symptom corresponds with the patient experiencing the symptom at a specific time.

In this study, we aimed to investigate the safety of additional vaccine doses for individuals who previously received the standard 2-dose regimen of a mRNA-based COVID-19 vaccine. We evaluated vaccine-associated symptoms reported in cohorts of individuals who received additional vaccination with BNT162b2 (Pfizer-BioNTech) or mRNA-1273 (Moderna).

Methods

This cohort study was reviewed by the Mayo Clinic Institutional Review Board as a minimal-risk study and determined to be exempt (45 CFR 46.104d, category 4). The IRB approved waiver of Health Insurance Portability and Accountability Act of 1996 (HIPAA) authorization in accordance with applicable HIPAA regulations. Participants were excluded if they did not have a research authorization on file. Further information on the Mayo Clinic Institutional Review Board and adherence to basic ethical principles underlying the conduct of research and ensuring that rights and well-being of potential research participants are adequately protected are available at the Mayo Clinic website.27 This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Study Population

This is a retrospective study of individuals within the Mayo Clinic Enterprise who were vaccinated with exactly 3 doses of Food and Drug Administration (FDA)–approved mRNA-based COVID-19 vaccines between December 1, 2020, and October 17, 2021. Individuals who had specifically opted out of inclusion of electronic medical records in research were excluded. Inclusion and exclusion criteria were defined as follows: (1) aged 18 years or older at date of initial COVID-19 vaccination; (2) received the first 2 doses of BNT162b2 or mRNA-1273 per emergency use authorization protocol (per-protocol BNT162b2 vaccination was defined as 2 BNT162b2 doses administered 18-28 days apart and per-protocol mRNA-1273 vaccination was defined as 2 mRNA-1273 doses administered 25-35 days apart); (3) received a third dose of an mRNA-based COVID-19 vaccine at least 90 days after the second dose; (4) third COVID-19 vaccine dose was of the same type as the original 2 doses; (5) did not have more than 3 doses of an mRNA-based COVID-19 vaccine on record; (6) did not previously receive doses of a non–mRNA-based COVID-19 vaccine (eg, Ad26.COV2.S [Janssen]); and (7) had at least 14 days of follow-up after third vaccine dose. We excluded 887 individuals who received mixed vaccine brands and 76 individuals who received a second dose of Ad26.COV2.S.

Study participants were divided into 2 cohorts for analysis by vaccine type administered, specifically, cohorts of 38 094 individuals with 3 BNT162b2 vaccine doses and 9905 individuals with 3 mRNA-1273 vaccine doses (Figure 1A). Demographic and clinical characteristics of the cohorts are shown in the Table, and information on timing of vaccine doses is shown in eFigure 1 in the Supplement. Race and ethnicity were determined from EHRs. See eAppendix in the Supplement for details on race and ethnicity response options as they appeared in the EHR database. Categories were grouped in this study as Asian, Black or African American, American Indian, Native Hawaiian or Pacific Islander, White or Caucasian, other, and unknown for race and Hispanic or Latino, not Hispanic or Latino, and unknown for ethnicity. These grouping were used to match categories used by the US Census Bureau for race28 and ethnicity.29 The category other included response options other and unable to provide. Race and ethnicity were assessed to determine the applicability of our findings to underrepresented groups given that the makeup of our analysis cohort was different from that of the general population and this is needed to frame the limitations of applicability in this study.

Figure 1. Schematic Depiction of Study Design .

Figure 1.

Open in a new tab

A, The study population consisted of 47 999 adult patients at Mayo Clinic Health System who met study inclusion criteria. This population was subdivided into cohorts by vaccine dose received. B, To investigate adverse events associated with COVID-19 vaccination, clinical symptoms were quantified as reported in electronic health record (EHR) notes during 14-day periods starting from the date of each vaccine dose (orange), as well as a control 14-day period before the first vaccine dose (blue).

Table. Demographic Characteristics and Tracked Comorbidities of Study Participants.

Characteristic Participants, No. (%) (N = 47 999)
3-Dose BNT162b2 (n = 38 094) 3-Dose mRNA-1273 (n = 9905)
Sex
Women 21 835 (57.3) 5099 (51.5)
Men 16 253 (42.7) 4806 (48.5)
Unknown 6 (<0.1) 0
Age, median (IQR), y 67.4 (52.5-76.5) 67.7 (59.5-73.9)
Race
Asian 1199 (3.1) 167 (1.7)
Black or African American 582 (1.5) 278 (2.8)
American Indian 72 (0.2) 39 (0.4)
Native Hawaiian or Pacific Islander 24 (<0.1) 4 (<0.1)
White or Caucasian 35 226 (92.5) 9188 (92.8)
Other 586 (1.5) 124 (1.3)
Unknown 405 (1.1) 105 (1.1)
Ethnicity
Hispanic or Latino 877 (2.3) 289 (2.9)
Not Hispanic or Latino 36 269 (95.2) 9,386 (94.8)
Unknown 948 (2.5) 230 (2.3)
Elixhauser index score, mean (SD) 7.6 (10.9) 12.2 (12.7)
Comorbidity
Immunosuppressed within 1 y 4693 (12.3) 2820 (28.5)
Cancer 8262 (21.7) 3378 (34.1)
HIV or AIDS 153 (0.4) 88 (0.9)
Open in a new tab

Extracting Adverse Event Sentiments From EHR Data Using Augmented Curation

A bidirectional encoder representations from transformers (BERT)–based30 classification model was used to investigate the sentiment of adverse event phenotypes mentioned in clinical notes. This model has been previously used to investigate signs and symptoms associated with COVID-19,31 short-term and long-term complications associated with COVID-19,32 and adverse events associated with mRNA-based COVID-19 vaccines.23 Given a sentence that includes a phenotype, this model outputs 1 of the following labels: yes for confirmed diagnosis, maybe for possible diagnosis, no for ruled out diagnosis, or other for none of the above (eg, family history of diagnosis). This model was trained on a data set of 18 490 sentences from clinical notes in the Mayo Clinic that included more than 250 different phenotypes and achieved an out-of-sample accuracy of 93.6% and precision and recall values of more than 95%.31

For this analysis, the previously described classification model was applied to clinical notes for individuals in the study population for the following time periods: 15 days to 1 day prior to the first vaccine dose, 0 to 14 days after the first vaccine dose, 0 to 14 days after the second vaccine dose, and 0 to 14 days after the third vaccine dose (Figure 1B). We considered 19 adverse event phenotypes, including anaphylaxis, arthralgia, cerebral venous sinus thrombosis, chills, diarrhea, erythema, facial paralysis, fatigue, fever, headache, local pain, local swelling, lymphadenopathy, myalgia, myocarditis, nausea, pericarditis, soreness, and vomiting. For each pair of adverse event and time period, individuals with at least 1 clinical note labeled yes by the model with greater than 90% confidence were counted as having the adverse event. For a select set of rare severe adverse events (ie, anaphylaxis, facial paralysis, myocarditis, and pericarditis), additional manual curation was performed (J.C.O. and D.W.C.) to confirm that patients identified by the model did experience the adverse events during the period of interest and that the adverse event was not associated with another known factor (ie, anaphylaxis associated with allergic reaction to a known nonvaccine allergen).

Extracting Comorbidity and Immunosuppressant Medication Data From EHRs

For each patient, Elixhauser comorbidities33 for HIV and AIDS and cancer were determined using International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes in the 5-year period before the first COVID-19 vaccine dose. In addition, overall Elixhauser comorbidity scores were computed using the van Walraven method.34 Patients who had taken immunosuppressant medications in the previous year in their medical history were identified by querying the EHR database for the list of medications in the World Health Organization Anatomical Therapeutic Chemical Classification System drug class LO4A.35

Statistical Analysis

We used previously described augmented curation models to extract sentiments of adverse events from EHRs.23,31 Specifically, for each individual in a cohort, we investigated whether positive sentiments for vaccine-associated adverse events were present in clinical notes during a specific 14-day period relative to their date of vaccination (Figure 1). Risk of an adverse event was then reported as the percentage of individuals receiving vaccines in a cohort with a positive sentiment for that adverse event. This risk was compared with the baseline risk in the cohort, taken as the risk associated with the adverse event in a 14-day period before the first COVID-19 vaccine dose. To account for the paired nature of reporting data (eg, patients reporting an adverse event before and after a vaccine dose), in analyses in which we quantified increased reporting of adverse events after the third dose compared with after the second dose and baseline incidence using risk difference (RD), RDs were calculated by a paired comparison of adverse event reports for single patients during the 2 time intervals being considered. The RD of an adverse event was defined as the difference between the percentage of the cohort that reported the adverse event after dose 3 and the percentage of the cohort that reported the adverse event during the control period (ie, before dose 1 or after dose 2). Reported 95% CIs were calculated using bootstrap resampling (with 10 000 samples), and P values were determined using a paired t test via the scipy.stats package in the Python programming language version 3.9 (Python Software Foundation). P values were 2-sided, and P < .05 was considered statistically significant. Data were analyzed from September through November 2021.

Results

Among 47 999 individuals who received 3-dose COVID-19 mRNA vaccines, 38 094 individuals (21 835 [57.3%] women; median [IQR] age, 67.4 [52.5-76.5] years) received BNT162b2 (79.4%) and 9905 individuals (5099 [51.5%] women; median [IQR] age, 67.7 [59.5-73.9] years) received mRNA-1273 (20.6%). Among participants receiving BNT162b2, there were 1199 Asian individuals (3.1%), 582 Black individuals (1.5%), 72 American Indian individuals (0.2%), 24 Native Hawaiian or Pacific Islander individuals (<0.1%), 35 226 White individuals (92.5%), 586 individuals with other race (1.5%), and 405 individuals with unknown race (1.1%); there were 877 Hispanic or Latino individuals (2.3%). Among participants receiving mRNA-1273, there were 167 Asian individuals (1.7%), 278 Black individuals (2.8%), 39 American Indian individuals (0.4%); 4 Native Hawaiian or Pacific Islander individuals (<0.1%), 9188 White individuals (92.8%), 124 individuals with other race (1.3%), and 105 individuals with unknown race (1.1%); there were 289 Hispanic or Latino individuals (2.9%). The BNT162b2 and mRNA-1273 cohorts differed in potentially confounding factors, including the relative proportion of individuals with immunosuppression (4693 individuals [12.3%] vs 2820 individuals [28.5%]) (Table).

We find no significant difference in the reporting of severe adverse events and a significant increase in reporting for most low-severity adverse events after the third COVID-19 vaccine dose compared with earlier doses. The most common adverse events reported after the third vaccine dose, with significant increases vs after the second dose, were fatigue (2360 individuals [4.92%] vs 1665 individuals [3.47%]; P < .001), lymphadenopathy (1387 individuals [2.89%] vs 995 individuals [2.07%]; P < .001), nausea (1259 individuals [2.62%] vs 979 individuals [2.04%]; P < .001), headache (1185 individuals [2.47%] vs 992 individuals [2.07%]; P < .001), arthralgia (1019 individuals [2.12%] vs 816 individuals [1.70%]; P < .001), myalgia (956 individuals [1.99%] vs 784 individuals [1.63%]; P < .001), diarrhea (817 individuals [1.70%] vs 595 individuals [1.24%]; P < .001), fever (533 individuals [1.11%] vs 391 individuals [0.81%]; P < .001), vomiting (528 individuals [1.10%] vs 385 individuals [0.80%]; P < .001), and chills (224 individuals [0.47%] vs 175 individuals [0.36%]; P = .01). In addition, common adverse events after the third dose with no significant increase vs after the second dose were erythema (480 individuals [1.00%] vs 436 individuals [0.91%]; P = .14) and soreness (174 individuals [0.36%] vs 171 individuals [0.36%]; P = .87) (eFigure 2 in the Supplement). Median times between vaccine dose and reporting of an adverse event are listed in eTables 1 and 2 in the Supplement. We found that, compared with after the second dose, there was increased incidence of reporting of most common adverse events for BNT162b2 and mRNA-1273 after dose 3. Overall, patients had significantly increased incidence of reporting fatigue (RD = 1.45 percentage points [95% CI, 1.21-1.68 percentage points]; P < .001), lymphadenopathy (RD = 0.82 percentage points [95% CI, 0.64-1.00 percentage points]; P < .001), nausea (RD = 0.58 percentage points [95% CI, 0.40-0.76 percentage points]; P < .001), headache (RD = 0.40 percentage points [95% CI, 0.23-0.58 percentage points]; P < .001), arthralgia (RD = 0.42 percentage points [95% CI, 0.26-0.59 percentage points]; P < .001), myalgia (RD = 0.36 percentage points [95% CI, 0.20-0.52 percentage points]; P < .001), diarrhea (RD = 0.46 percentage points [95% CI, 0.32-0.60 percentage points]; P < .001), fever (RD = 0.30 percentage points [95% CI, 0.18-0.41 percentage points]; P < .001), vomiting (RD = 0.30 percentage points [95% CI, 0.18-0.41 percentage points]; P < .001), and chills (RD = 0.10 percentage points [95% CI, 0.03-0.18 percentage points]; P = .01) (Figure 2). RDs are also reported per vaccine brand (Figure 2). No vaccine brand–specific adverse events were found.

Figure 2. Risk Difference for Common Adverse Events.

Figure 2.

Open in a new tab

The risk difference is shown for individuals receiving 3-dose BNT162b2 (A) or 3-dose mRNA-1273 (B). Error bars indicate 95% CIs.

aA significant difference in prevalence was found after the third dose compared with before the first dose in paired t test. P values for BNT162b2 were <.001 for fatigue, <.001 for lymphadenopathy, <.001 for nausea, <.001 for headache, <.001 for arthralgia, <.001 for myalgia, <.001 for diarrhea, <.001 for erythema, <.001 for fever, .008 for vomiting, <.001 for chills, and .90 for soreness. P values for mRNA-1273 were <.001 for fatigue, <.001 for lymphadenopathy, <.001 for nausea, .001 for headache, .26 for arthralgia, .02 for myalgia, <.001 for diarrhea, .003 for erythema, <.001 for fever, .02 for vomiting, .04 for chills, and .01 for soreness.

bA significant difference in prevalence was found after the third dose compared with after second dose in paired t test. P values for BNT162b2 were <.001 for fatigue, <.001 for lymphadenopathy, <.001 for nausea, <.001 for headache, <.001 for arthralgia, <.001 for myalgia, <.001 for diarrhea, .25 for erythema, .002 for fever, <.001 for vomiting, .03 for chills, and .61 for soreness. P values for mRNA-1273 were <.001 for fatigue, <.001 for lymphadenopathy, <.001 for nausea, .002 for headache, .02 for arthralgia, .054 for myalgia, .007 for diarrhea, .27 for erythema, <.001 for fever, .003 for vomiting, .18 for chills, and .23 for soreness.

Reporting of severe adverse events was rare after the third dose and was not significantly increased compared with the frequency of reporting after the second dose (eFigure 2 in the Supplement). After the third vaccine dose, 4 patients reported pericarditis (0.01% [95% CI, 0%-0.02%]; P = .18), 2 patients reported anaphylaxis (0% [95% CI, 0%-0.01%]; P = .41), and 1 patient reported myocarditis (0% [95% CI, 0%-0.01%]; P = .32). We find no significant increase in risk of these adverse events after the third dose compared with after the second dose (eFigure 2 in the Supplement). Additionally, we assessed the rate of emergency department visits within 2 days of each vaccine dose and found a significant increase after the third BNT162b2 dose, with visits among 0.29% (95% CI, 0.24%-0.35%) of recipients compared with 0.2% (0.15%-0.24%, 95% CI) of recipients after the second dose (P = .02) (eFigure 3 in the Supplement). No significant difference in emergency department visits was found among individuals receiving mRNA-1273. Individuals who received exactly 1 dose of an mRNA-based COVID-19 vaccine had greater increases in incidence of adverse events after dose 1, compared with baseline, than what was observed in the 3-dose cohort (eFigure 4 in the Supplement).

Discussion

Results from this cohort study suggest that a third dose of the same type of vaccination after a BNT162b2 or mRNA-1273 primary series is associated with safe outcomes. Although we observed an increase in early postvaccination adverse events after the third dose compared with earlier doses, these outcomes were symptoms of low concern (ie, fatigue, lymphadenopathy, nausea, and diarrhea). We observed no significant increase in EHR reporting of severe adverse events after the third dose compared with after the second dose, with incidence comparable with that found in previous literature.16,17 The observed increase in adverse events compared with earlier doses could be associated with a stronger response elicited by the third dose, comparable to what was observed for the second dose compared with the first dose.1,2,36 Further studies are needed to explore whether the third vaccine dose is associated with a stronger immune response.

Limitations

This study has several limitations. To control for patient-specific covariates that may be associated with symptoms experienced after vaccination, we have considered only cohorts of patients who received exactly 3 doses of mRNA-based COVID-19 vaccines. No comparison between adverse events reported in the BNT162b2 and mRNA-1273 cohorts should be made given that these cohorts differed in potentially confounding factors, including in the relative proportion of individuals with immunosuppression. The cohorts are also less likely to include individuals who had strong adverse reactions to earlier doses of mRNA-based COVID-19 vaccines given that such individuals would be more likely to opt out of additional vaccination.15 Indeed, analysis of individuals who received exactly 1 dose of an mRNA-based COVID-19 vaccine found a greater increase in incidence of adverse events after dose 1, compared with baseline, than what was observed in the 3-dose cohort. Additionally, a large proportion in the 3-dose cohort was immunosuppressed and older, potentially associated with reduced immune reaction to vaccination and associated adverse events and decreased prevalence of adverse events that are more prevalent in younger individuals (eg, myocarditis). This suggests that our conclusions on the safety of additional-dose vaccination with BNT162b2 or mRNA-1273 may apply specifically to individuals who are included in these cohorts, opted to receive an additional COVID-19 mRNA vaccine dose of the same type, and opted to report symptoms to allow EHR capture. Therefore, these results may not be generalizable to individuals who are healthier. Furthermore, we have no way of accounting for variation in the likelihood of individuals reporting outcomes; for example, if individuals were more likely to report certain outcomes after a first vaccine dose but are more likely to dismiss them after a third dose without contacting their clinicians, we would not be able to detect that likelihood with our data. Our eligible study population included too few individuals with mixed vaccine brands (887 individuals) or with second doses of Ad26.COV2.S (76 individuals) to reach meaningful conclusions, and these populations were therefore not included in this report. Further study on larger, more general populations may therefore find additional evidence of adverse events and will be needed to reach meaningful conclusions on the prevalence of rare adverse events.

In this study, we used augmented curation of EHR notes to quantify clinical symptoms experienced by individuals receiving vaccines. The augmented curation process involves defining a list of symptoms and subsequently quantifying positive sentiments associated with these symptoms in EHR notes. Given that symptoms that were not explicitly included would not be quantified, the study design is not suitable for discovering adverse events that have not previously been associated with COVID-19 vaccination. Identification of positive sentiments for vaccine adverse events using augmented curation is not perfect; however, previous studies have found excellent accuracy associated with the use of these augmented curation algorithms for related tasks.31

Extraction of adverse events from EHR notes is complementary with clinical trials and self-reporting approaches used in previous studies.13,24,25,26 Barriers associated with self-reporting of adverse events (ie, via a survey or device) may be removed, and all adverse events for which an individual seeks medical attention may be counted. This association with reduced barriers to data collection allowed us to analyze a larger cohort of individuals receiving 3-dose vaccines than in previous studies, without necessary selection for individuals willing to self-report. However, we detected only symptoms that individuals reported to clinicians and that were captured in EHR notes. This is likely associated with the low rate of common but nonsevere adverse events found (ie, fatigue, local swelling, and local redness) compared with previous studies24,26 given that individuals may not seek medical attention for expected low-severity adverse events after vaccination.

Conclusions

This study provides further evidence suggesting that third-dose vaccination with the same type of COVID-19 mRNA vaccine as used in the primary series is associated with safe outcomes in eligible populations. Together with previous studies of the safety13,25,26 and effectiveness12,13,24 associated with booster doses, our study suggests that third-dose mRNA COVID-19 vaccination may be appropriate for at-risk populations.

Supplement.

eAppendix. Race and Ethnicity Database Options

eFigure 1. Distribution of Time Between Vaccine Doses

eFigure 2. Prevalence of Vaccine-Associated Adverse Events During 14-Day Periods Before and After Vaccination With 3 Doses

eFigure 3. Emergency Department Visits Within 2 Days of Vaccine Dose

eFigure 4. Prevalence of Vaccine-Associated Adverse Events During 14-Day Periods Before and After Vaccination With 1 Dose

eTable 1. Tracked Adverse Events by Time Window for 3-Dose BNT162b2 Cohort

eTable 2. Tracked Adverse Events by Time Window for 3-Dose mRNA-1273 Cohort

Click here for additional data file. (1.4MB, pdf)

References

  • 1.Polack FP, Thomas SJ, Kitchin N, et al. ; C4591001 Clinical Trial Group . Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine. N Engl J Med. 2020;383(27):2603-2615. doi: 10.1056/NEJMoa2034577 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Baden LR, El Sahly HM, Essink B, et al. ; COVE Study Group . Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384(5):403-416. doi: 10.1056/NEJMoa2035389 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Pawlowski C, Lenehan P, Puranik A, et al. FDA-authorized mRNA COVID-19 vaccines are effective per real-world evidence synthesized across a multi-state health system. Med (N Y). 2021;2(8):979-992.e8. doi: 10.1016/j.medj.2021.06.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Thompson MG, Burgess JL, Naleway AL, et al. Interim estimates of vaccine effectiveness of BNT162b2 and mRNA-1273 COVID-19 vaccines in preventing SARS-CoV-2 infection among health care personnel, first responders, and other essential and frontline workers—eight U.S. locations, December 2020-March 2021. MMWR Morb Mortal Wkly Rep. 2021;70(13):495-500. doi: 10.15585/mmwr.mm7013e3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Puranik A, Lenehan PJ, Silvert E, et al. Comparative effectiveness of mRNA-1273 and BNT162b2 against symptomatic SARS-CoV-2 infection. Med (N Y). 2022;3(1):28-41.e8. doi: 10.1016/j.medj.2021.12.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Lopez Bernal J, Andrews N, Gower C, et al. Effectiveness of COVID-19 vaccines against the B.1.617.2 (Delta) variant. N Engl J Med. 2021;385(7):585-594. doi: 10.1056/NEJMoa2108891 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Thompson MG, Stenehjem E, Grannis S, et al. Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings. N Engl J Med. 2021;385(15):1355-1371. doi: 10.1056/NEJMoa2110362 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Grannis SJ, Rowley EA, Ong TC, et al. ; VISION Network . Interim estimates of COVID-19 vaccine effectiveness against COVID-19-associated emergency department or urgent care clinic encounters and hospitalizations among adults during SARS-CoV-2 B.1.617.2 (Delta) variant predominance—nine states, June-August 2021. MMWR Morb Mortal Wkly Rep. 2021;70(37):1291-1293. doi: 10.15585/mmwr.mm7037e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Self WH, Tenforde MW, Rhoads JP, et al. ; IVY Network . Comparative effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) vaccines in preventing COVID-19 hospitalizations among adults without immunocompromising conditions–United States, March-August 2021. MMWR Morb Mortal Wkly Rep. 2021;70(38):1337-1343. doi: 10.15585/mmwr.mm7038e1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Puranik A, PJ Lenehan, JC O’Horo, et al. Durability analysis of the highly effective BNT162b2 vaccine against COVID-19. bioRxiv. Preprint posted online September 7, 2021. doi: 10.1101/2021.09.04.21263115 [DOI] [PMC free article] [PubMed]
  • 11.Israel A, Merzon E, Shäffer AA, et al. Elapsed time since BNT162b2 vaccine and risk of SARS-CoV-2 infection in a large cohort. medRxiv. Preprint posted online August 5, 2021. doi: 10.1101/2021.08.03.21261496 [DOI] [PMC free article] [PubMed]
  • 12.Bar-On YM, Goldberg Y, Mandel M, et al. Protection of BNT162b2 vaccine booster against COVID-19 in Israel. N Engl J Med. 2021;385(15):1393-1400. doi: 10.1056/NEJMoa2114255 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Pfizer . Pfizer and BioNTech announce phase 3 trial data showing high efficacy of a booster dose of their COVID-19 vaccine. Accessed March 8, 2022. https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-announce-phase-3-trial-data-showing
  • 14.Shroff RT, Chalasani P, Wei R, et al. Immune responses to two and three doses of the BNT162b2 mRNA vaccine in adults with solid tumors. Nat Med. 2021;27(11):2002-2011. doi: 10.1038/s41591-021-01542-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Centers for Disease Control and Prevention . Possible Side Effects After Getting a COVID-19 Vaccine. Accessed March 8, 2022. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/expect/after.html
  • 16.Mevorach D, Anis E, Cedar N, et al. Myocarditis after BNT162b2 mRNA vaccine against COVID-19 in Israel. N Engl J Med. 2021;385(23):2140-2149. doi: 10.1056/NEJMoa2109730 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Witberg G, Barda N, Hoss S, et al. Myocarditis after COVID-19 vaccination in a large health care organization. N Engl J Med. 2021;385(23):2132-2139. doi: 10.1056/NEJMoa2110737 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Muir K-L, Kallam A, Koepsell SA, Gundabolu K. Thrombotic thrombocytopenia after Ad26.COV2.S vaccination. N Engl J Med. 2021;384(20):1964-1965. doi: 10.1056/NEJMc2105869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Sadoff J, Davis K, Douoguih M. Thrombotic thrombocytopenia after Ad26.COV2.S vaccination—response from the manufacturer. N Engl J Med. 2021;384(20):1965-1966. doi: 10.1056/NEJMc2106075 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Schultz NH, Sørvoll IH, Michelsen AE, et al. Thrombosis and thrombocytopenia after ChAdOx1 nCoV-19 vaccination. N Engl J Med. 2021;384(22):2124-2130. doi: 10.1056/NEJMoa2104882 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Greinacher A, Thiele T, Warkentin TE, Weisser K, Kyrle PA, Eichinger S. Thrombotic thrombocytopenia after ChAdOx1 nCov-19 vaccination. N Engl J Med. 2021;384(22):2092-2101. doi: 10.1056/NEJMoa2104840 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Scully M, Singh D, Lown R, et al. Pathologic antibodies to platelet factor 4 after ChAdOx1 nCoV-19 vaccination. N Engl J Med. 2021;384(23):2202-2211. doi: 10.1056/NEJMoa2105385 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.McMurry R, Lenehan P, Awasthi S, et al. Real-time analysis of a mass vaccination effort confirms the safety of FDA-authorized mRNA COVID-19 vaccines. Med (N Y). 2021;2(8):965-978.e5. doi: 10.1016/j.medj.2021.06.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Pfizer . Pfizer and BioNTech initiate rolling submission of supplemental biologics license application to U.S. FDA for booster dose of Comirnaty in individuals 16 and older. Accessed March 8, 2022. https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-initiate-rolling-submission
  • 25.Mofaz M, Ychezkel M, Guan G, et al. Self-reported and physiological reactions to the third BNT162b2 mRNA COVID-19 (booster) vaccine dose. bioRxiv. Preprint posted online September 22, 2021. doi: 10.1101/2021.09.15.21263633 [DOI]
  • 26.Hause AM, Baggs J, Gee J, et al. Safety monitoring of an additional dose of COVID-19 vaccine United States, August 12-September 19, 2021. MMWR Morb Mortal Wkly Rep. 2021;70(39):1379-1384. doi: 10.15585/mmwr.mm7039e4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Mayo Clinic . Institutional review board (IRB). Accessed March 9, 2022. https://www.mayo.edu/research/institutional-review-board/overview
  • 28.US Census Bureau . About the topic of race. Accessed March 18, 2022. https://www.census.gov/topics/population/race/about.html
  • 29.US Census Bureau . About the Hispanic population and it's origin. Accessed March 18, 2022. https://www.census.gov/topics/population/hispanic-origin/about.html
  • 30.Devlin J, Chang MW, Lee K, and Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv.org. Accessed March 8, 2022. https://arxiv.org/abs/1810.04805
  • 31.Wagner T, Shweta F, Murugadoss K, et al. Augmented curation of clinical notes from a massive EHR system reveals symptoms of impending COVID-19 diagnosis. Elife. 2020;9:e58227. doi: 10.7554/eLife.58227 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Venkatakrishnan AJ, Pawlowski C, Zemmour D, et al. Mapping each pre-existing condition’s association to short-term and long-term COVID-19 complications. NPJ Digit Med. 2021;4(1):117. doi: 10.1038/s41746-021-00484-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. doi: 10.1097/00005650-199801000-00004 [DOI] [PubMed] [Google Scholar]
  • 34.van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6):626-633. doi: 10.1097/MLR.0b013e31819432e5 [DOI] [PubMed] [Google Scholar]
  • 35.World Health Organization Collaborating Centre for Drug Statistics Methodology . ATC/DDD index. Accessed March 8, 2022. https://www.whocc.no/atc_ddd_index/?code=L04A&showdescription=no
  • 36.Arunachalam PS, Scott MKD, Hagan T, et al. Systems vaccinology of the BNT162b2 mRNA vaccine in humans. Nature. 2021;596(7872):410-416. doi: 10.1038/s41586-021-03791-x [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eAppendix. Race and Ethnicity Database Options

eFigure 1. Distribution of Time Between Vaccine Doses

eFigure 2. Prevalence of Vaccine-Associated Adverse Events During 14-Day Periods Before and After Vaccination With 3 Doses

eFigure 3. Emergency Department Visits Within 2 Days of Vaccine Dose

eFigure 4. Prevalence of Vaccine-Associated Adverse Events During 14-Day Periods Before and After Vaccination With 1 Dose

eTable 1. Tracked Adverse Events by Time Window for 3-Dose BNT162b2 Cohort

eTable 2. Tracked Adverse Events by Time Window for 3-Dose mRNA-1273 Cohort

Click here for additional data file. (1.4MB, pdf)

Articles from JAMA Network Open are provided here courtesy of American Medical Association

RESOURCES