Near real-time surveillance of safety outcomes in US COVID-19 vaccine recipients aged 12 to 64 years

Abstract

Background

Active monitoring of safety outcomes following COVID-19 vaccination is critical to understand vaccine safety and can provide early detection of rare outcomes not identified in pre-licensure trials. We present findings from an early warning rapid surveillance system in three large commercial insurance databases including more than 16 million vaccinated individuals.

Methods

We evaluated 17 outcomes of interest following COVID-19 vaccination among individuals aged 12–64 years in Optum, HealthCore, and CVS Health databases from December 11, 2020, through January 22, 2022, January 7, 2022, and December 31, 2021, respectively. We conducted biweekly or monthly sequential testing and generated rate ratios (RR) of observed outcome rates compared to historical (or expected) rates prior to COVID-19 vaccination.

Findings

Among 17 outcomes evaluated, 15 did not meet the threshold for statistical signal in any of the three databases. Myocarditis/pericarditis met the statistical threshold for a signal following BNT162b2 in two of three databases (RRs: 1.83–2.47). Anaphylaxis met the statistical threshold for a signal in all three databases following BNT162b2 vaccination (RRs: 4.48–10.86) and mRNA-1273 vaccination (RRs: 7.64–12.40).

Discussion

Consistent with published literature, our near-real time monitoring of 17 adverse outcomes following COVID-19 vaccinations identified signals for myocarditis/pericarditis and anaphylaxis following mRNA COVID-19 vaccinations. The method is intended for early detection of safety signals, and results do not imply a causal effect. Results of this study should be interpreted in the context of the method’s utility and limitations, and the validity of detected signals must be evaluated in fully adjusted epidemiologic studies.

1. Introduction

Four vaccines are currently available in the United States (US) to prevent Coronavirus Disease 2019 (COVID-19). The Pfizer-BioNTech vaccine (BNT162b2) is licensed for persons 12 years and older and authorized under emergency use authorization (EUA) for those aged 6 months to 11 years (BNT162b2). The Moderna (mRNA-1273) vaccine is licensed for persons 18 years and older and is authorized under EUA for persons aged 6 months to 17 years. The Janssen (Ad26COV2.S) vaccine is authorized for persons 18 years and older and the Novavax (NVX-CoV2373) vaccine is authorized for persons 12 years and older. The safety of these vaccines has been reported in clinical trials [1], [2] and is currently being monitored [3], [4], [5]. We report on the safety among individuals aged 12 to 64 years following vaccination with three COVID-19 vaccines (BNT162b2, mRNA-1273, or Ad26.COV2.S) by examining medical and pharmacy claims data from three data sources. Over 16 million privately insured persons were assessed using near real-time safety monitoring for 17 adverse events of special interest (AESI). This study was conducted by the US Food and Drug Administration (FDA) Biologics Effectiveness and Safety (BEST) Initiative, which is an active surveillance program for post-market surveillance of biologic products including vaccines.

This study significantly expands the safety information on available COVID-19 vaccines by rapidly monitoring many health outcomes in three large health plans. The large population size can be especially important for monitoring rare outcomes such as myocarditis/pericarditis, Guillain-Barré Syndrome (GBS), thromboses with thrombocytopenia (TTS), transverse myelitis and others. The method of real-time surveillance is designed to be sensitive enough for rapid identification of potential safety signals. However, results do not imply a causal association due to methodologic limitations, including limited adjustments for confounding. Accordingly, further epidemiological studies are needed for evaluation of any potential signals.

In this report, we summarize the results for 17 outcomes based on biweekly and monthly descriptive monitoring and sequential testing of claims data submitted to large commercial insurance organizations. This extensive study enables signals to be rapidly identified for public health purposes, which can provide preliminary scientific evidence about vaccine safety to clinicians, regulators, and public health officials and prompt further in-depth investigation on this topic.

2. Methods

2.1. Study design

This study utilized sequential testing to detect potential safety signals following vaccination by comparing observed rates of adverse events following vaccination to historical background rates. This is referred to as a rapid cycle analysis (RCA) design, a form of near real-time surveillance that conducts multiple tests as data on vaccine exposures accrue.

2.2. Data sources

This study used data from Optum (through OptumServe), HealthCore (through the HealthCore Integrated Research Database, HIRD), and CVS Health (Aetna Enterprise Data Warehouse through CVS Health Clinical Trial Services, CTS). These databases contain longitudinal medical and pharmacy claims data, and they are supplemented with immunization information system (IIS) vaccination data (Appendix A) [6]. The Optum, HealthCore, and CVS Health databases contain approximately 14 million, 24 million, and 22 million persons per year, respectively.

2.3. Study period and population

The study included health plan members aged 12 to 64 years who received a COVID-19 vaccine from December 11, 2020, the date of emergency use authorization of the BNT162b2 COVID-19 vaccine, through January 22, 2022 for Optum, January 7, 2022 for HealthCore, and December 31, 2021 for CVS Health plan enrollees. Individuals must have been enrolled with medical insurance coverage on the vaccination date and be continuously enrolled during an AESI-specific pre-vaccination clean window, defined as the interval prior to vaccination where a patient must not have had the AESI.

2.4. Exposure and follow-Up

Exposure was defined as receipt of BNT162b2, mRNA-1273, or Ad26.COV2.S COVID-19 vaccination, identified using brand- and dose-specific Current Procedural Terminology (CPT)/ Healthcare Common Procedure Coding System (HCPCS) codes or National Drug Codes (NDC) (eTable 1). Dose number was assigned based on the chronological order in which vaccinations were observed, as pharmacy claims do not have corresponding administration codes. If a patient was administered doses exceeding the recommended number for that brand during the study period, these vaccinations were assigned as “unknown dose.” Doses occurring within 3 days of a previous dose were considered invalid and excluded from the study. The primary analysis included all observed doses (including third/booster doses) for mRNA vaccinations and first doses for Ad26.COV2.S. Additional dose-specific analyses following dose 1 and dose 2 were conducted for mRNA vaccinations. All analyses were conducted separately by brand.

Follow-up time included all person-time accrued during the pre-specified post-vaccination risk windows. Risk windows were censored at subsequent vaccination, death, disenrollment, end of risk window, or end of study period, whichever came first.

2.5. Adverse events of special interest

Claims-based AESI algorithms, including pre-vaccination clean windows, and post-vaccination risk windows were developed based on comprehensive literature review and consultation with clinical experts for 18 AESI (eTable 2) [7]. Of the 18 AESI, 17 were included in sequential testing and 1 outcome (multisystem inflammatory syndrome) was only monitored descriptively due to limited observability during historical periods. Sequential testing for anaphylaxis was pre-specified as a secondary analysis because it is a well-documented adverse reaction to vaccines where medical interventions are available [8], [9], [10]. All AESIs were identified using International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes. Claims from inpatient facilities (IP), emergency department encounters in outpatient facilities (OP-ED), and all outpatient facilities and individual providers or professionals (OP/PB) were used to identify AESI. Medical records for a sample of myocarditis/pericarditis cases were reviewed to confirm the outcomes.

2.6. Descriptive analyses

Near real-time surveillance data on exposures and outcomes were updated every other week for the Optum database and every month for the HealthCore and CVS Health databases. Descriptive analyses of vaccines were tabulated by database and vaccine brand, including individuals that received multiple vaccine brands. Descriptive statistics of vaccinations and AESI counts by age category, sex, Health and Human Services (HHS) Region [11], and urban or rural residency status were calculated. Race and ethnicity data are frequently missing within the databases and therefore were excluded from the analysis.

2.7. Sequential testing

Poisson Maximized Sequential Probability Ratio Test (PMaxSPRT) was used to detect increased AESI risk following vaccination compared to a historical baseline for 17 AESI in each of the three databases (eTable 3) [12], [13], [14]. This method is also referred to as RCA or sequential monitoring in the vaccine surveillance field. Sequential testing was conducted with each data update.

2.8. Comparator rate selection

The observed number of AESI in the COVID-19 vaccinated population was compared to an expected number of AESI based on the background rate of the AESI drawn from the same data source primarily from the pre-pandemic period [7]. The comparator rate was standardized by age group and sex by calculating rates within the strata of the standardization variables during a time period of 2017–2019 for HealthCore, 2019 for Optum, and 2019–2020 for CVS Health. To account for delayed observation of adverse events due to claims processing, the comparator rate was adjusted based on the data lag for each outcome estimated from events in 2019 [3], [5]. The comparator population was selected as either a general population or an influenza-vaccinated population depending on the background rates observed in these historical cohorts. If rates in the historical period differed from each other, as generally assessed by comparing the 95% confidence intervals (CI), we selected the lowest rate in order to enhance sensitivity of this monitoring; otherwise, the median annual rate was selected. For two outcomes (appendicitis and narcolepsy, both in the CVS database) where the peri-COVID rate (June - December 2020) did not return to pre-COVID rates, the peri-COVID rate was selected as the comparator (eTable 3).

2.9. PMaxSPRT sequential testing

Sequential testing for each AESI commenced when a minimum of three cases accrued, and the surveillance length for each AESI was pre-specified based on the expected vaccinations to be administered over the course of the vaccination campaign. One-tailed tests were used with a null hypothesis that the observed rate was no greater than the historical comparator beyond a pre-specified test margin with an overall alpha of 1%. The alpha level was selected to address the large number of tests and to reduce Type 1 error. The test margin was selected for each AESI based on expert guidance to avoid minimal risk increases that were unlikely to be clinically relevant. The expected number of events accrued under the null hypothesis was used as the upper limit on the number of tests. A statistical signal occurred if the log likelihood ratio exceeded the critical value [12], [13], [14]. A rate ratio (RR) was obtained by comparing the incidence of observed events among the study population to the adjusted historical background rates; confidence intervals were not calculated for the RRs due to the design of the PMaxSPRT. Sequential testing continued until a signal was observed or the pre-specified surveillance length was reached, after which descriptive monitoring of post-vaccine outcome rates continued until the end of the study period.

2.10. Statistical software

Analyses were conducted using R 4.0.2 (R Foundation for Statistical Computing, Vienna, Austria) and the package Sequential 3.3.1. Analyses conducted by Optum and CVS Health data additionally used SAS v. 9.4 (SAS Institute Inc., Cary, NC, United States), and analyses of HealthCore data used SAS Enterprise Guide v. 8.3 (SAS Institute Inc., Cary, NC, United States).

2.11. Ethical considerations

This surveillance activity was conducted as part of the US FDA public health surveillance mandate. This study was exempt from institutional review board approval because these databases contain deidentified data.

3. Results

3.1. Descriptive analyses

Among individuals aged 12 to 64 years, a total of 9,604,918 doses were observed in 5,070,372 recipients in the Optum database, 14,146,413 doses were observed in 7,445,051 recipients in the HealthCore database, and 8,003,958 doses were observed in 4,326,594 recipients in the CVS Health database (data not shown). In the Optum database, there were 6,139,808 BNT162b2 doses, 3,164,993 mRNA-1273 doses, and 299,886 Ad26.COV2.S doses; in HealthCore, there were 8,738,079 BNT162b2 doses, 4,901,100 mRNA-1273 doses, and 507,120 Ad26.COV2.S doses; in the CVS Health database, there were 5,072,078 BNT162b2 doses, 2,691,167 mRNA-1273 doses, and 240,595 Ad26.COV2.S doses.

Table 1 shows patient characteristics of COVID-19 vaccine recipients overall and by vaccine brand (mRNA-1273, BNT162b2, Ad26.COV2.S, and multiple) in Optum, HealthCore, and CVS Health databases, respectively. Characteristics of the vaccinated populations were largely similar across databases. All databases had similar age and sex distributions among vaccinated individuals, with slightly more females than males among the vaccinated populations. The HealthCore database contained a higher proportion of vaccinated individuals living in rural areas than in Optum and CVS Health databases (11.6% vs. 5.4% and 4.5%, respectively).

Table 1.

Characteristics of Health Plan Members Aged 12–64 years Receiving COVID-19 Vaccines Overall and Stratified by Vaccine Brand in the Optum, HealthCore, and CVS Health Databases.

Patient Characteristicsa	Optum (data through 1/22/2022)					HealthCore (data through 1/7/2022)					CVS Health (data through 12/31/2022)
	Total	mRNA-1273	BNT162b2	Ad26.COV2.S	Multiple	Total	mRNA-1273	BNT162b2	Ad26.COV2.S	Multiple	Total	mRNA-1273	BNT162b2	Ad26.COV2.S	Multiple
	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%	N/%
Total	5,070,372	1,558,811	3,100,326	220,434	190,801	7,445,051	2,429,049	4,386,627	376,968	252,407	4,326,594	1,373,442	2,631,526	186,743	134,883
Age at First Dose (Years)
12–17	8.5%	0.0%	13.9%	0.0%	1.0%	8.5%	0.1%	14.3%	0.0%	1.2%	8.1%	0.2%	13.1%	0.0%	1.8%
18–25	11.6%	11.7%	11.4%	13.8%	11.9%	12.9%	12.7%	13.0%	14.9%	10.2%	12.9%	13.0%	12.7%	15.7%	11.8%
26–35	19.0%	20.1%	18.5%	18.9%	19.6%	17.2%	17.9%	16.9%	16.4%	17.8%	19.7%	20.5%	19.3%	18.7%	20.3%
36–45	20.7%	22.2%	19.7%	21.7%	24.0%	19.2%	20.5%	18.3%	19.8%	22.5%	19.6%	20.7%	18.8%	20.3%	23.0%
46–55	21.1%	23.4%	19.6%	24.6%	23.9%	21.0%	23.4%	19.1%	25.0%	24.4%	19.9%	21.9%	18.5%	23.1%	22.3%
56–64	19.0%	22.5%	17.0%	21.0%	19.6%	21.1%	25.4%	18.4%	23.8%	23.9%	19.8%	23.7%	17.6%	22.1%	20.7%
Sex
Female	52.3%	52.4%	52.9%	43.7%	51.0%	52.6%	53.2%	53.1%	44.4%	50.9%	52.8%	52.9%	53.4%	44.5%	50.7%
Male	47.7%	47.6%	47.1%	56.3%	49.0%	47.4%	46.8%	46.9%	55.6%	49.1%	47.2%	47.1%	46.6%	55.5%	49.2%
Urban/Rural
Rural	5.4%	7.9%	4.1%	8.5%	4.0%	11.6%	16.2%	8.7%	16.3%	8.8%	4.5%	6.3%	3.3%	7.3%	3.7%
Urban	94.4%	91.9%	95.8%	91.4%	95.9%	88.4%	83.8%	91.2%	83.7%	91.2%	95.1%	93.3%	96.2%	92.1%	96.2%
HHS Region
Region 1	5.0%	5.2%	4.8%	4.9%	6.9%	7.2%	7.4%	7.0%	7.6%	8.6%	5.7%	5.7%	5.6%	6.0%	8.8%
Region 2	7.5%	7.4%	7.6%	6.4%	6.2%	7.2%	7.6%	7.0%	6.2%	6.3%	16.9%	17.6%	16.8%	14.5%	16.1%
Region 3	7.9%	8.5%	7.5%	8.1%	8.7%	9.0%	9.8%	8.8%	8.0%	6.1%	13.4%	14.1%	13.0%	12.6%	12.8%
Region 4	19.6%	20.5%	19.1%	21.4%	17.2%	18.2%	19.8%	17.7%	17.9%	11.5%	14.6%	14.4%	14.9%	16.1%	9.9%
Region 5	18.6%	17.4%	19.0%	20.7%	20.4%	20.0%	17.8%	21.0%	22.4%	21.1%	15.3%	14.1%	15.3%	17.7%	23.0%
Region 6	13.9%	14.7%	13.7%	13.8%	11.5%	4.4%	4.3%	4.6%	4.2%	3.1%	9.9%	10.2%	9.8%	10.7%	6.9%
Region 7	6.5%	5.6%	7.0%	5.5%	6.1%	3.6%	3.4%	3.8%	3.3%	3.0%	3.3%	2.9%	3.5%	3.1%	2.6%
Region 8	5.1%	5.0%	5.2%	4.9%	5.4%	4.6%	4.4%	4.6%	4.5%	7.6%	2.8%	2.7%	2.8%	2.8%	2.6%
Region 9	12.5%	12.6%	12.6%	10.7%	13.6%	24.3%	24.1%	24.0%	24.6%	31.4%	14.2%	14.2%	14.3%	12.0%	14.1%
Region 10	3.3%	2.9%	3.4%	3.5%	3.9%	1.5%	1.3%	1.5%	1.5%	1.2%	3.6%	3.8%	3.6%	4.0%	3.1%

Abbreviations: COVID-19, coronavirus disease of 2019; HHS, United States Department of Health and Human Services.

^a Characteristics of the population were assessed at the time of first vaccine dose.

^b Beneficiaries who receive >1 brand of COVID-19 vaccination are classified as “multiple.”

Missing/Unknown rows not shown.

Fig. 1 shows the weekly number of COVID-19 vaccinations by sex and by vaccine brand. Vaccine doses peaked between April 3 and April 24, 2021 across all vaccine brands and sexes in all databases. Fig. 2 shows the weekly number of COVID-19 vaccinations by age category and by vaccine brand. Across databases, mRNA-1273 and BNT162b2 doses were administered earlier for older individuals, with peaks observed for the 56–64 year-old population in early April 2021 and for the 12–17 year-old age group with BNT162b2 in early May through mid-June 2021. Timing of Ad26.COV2.S vaccine administration was similar across age groups and databases with peaks in mid-March and mid-April 2021.

Fig. 1.

Number of COVID-19 Vaccine Doses Administered, by Sex and Vaccine Brand for Each Week of Sequential Testing in the Optum, HealthCore, and CVS Health Databases.

Fig. 2.

Number of COVID-19 Vaccine Doses Administered, by Age Category and Vaccine Brand for Each Week of Sequential Testing in the Optum, HealthCore, and CVS Health Databases.

Seven cases of multisystem inflammatory syndrome were identified through descriptive monitoring across all databases. Four of seven cases were identified in the population aged 12–17 years.

3.2. Historical comparisons using PMaxSPRT

Among the 17 AESIs evaluated in sequential testing, 15 did not meet the threshold for a statistical signal in any of the three databases. Only two AESI met the threshold for statistical signal in one or more databases. Myocarditis/pericarditis signaled in the all-dose analysis following BNT162b2 in the HealthCore and CVS Health databases (HealthCore RR = 1.83, N = 322; CVS Health RR = 2.47, N = 243) but did not meet the threshold for a signal in the Optum database (RR = 1.73; N = 264) (Table 2 ). In the analysis stratified by vaccine dose number and brand, myocarditis/pericarditis met the threshold for signal following dose 2 of BNT162b2 in the HealthCore and CVS Health databases (HealthCore RR = 1.95; N = 189; CVS RR = 2.73, N = 143), and myocarditis/pericarditis met the threshold for a signal following dose 1 of BNT162b2 in the CVS Health database only (RR = 2.18; N = 87) (Supplementary Table 4).

Table 2.

Sequential Testing Results in Each Database for Persons 12–64 Years of Age by AESI and Vaccine Brand Following All Doses in the Optum, HealthCore, and CVS Health Databases.

AESI	Vaccine Brand	Optum				HealthCore				CVS Health
		Number of Doses	Observed AESI	Observed Person-Time (Years)	RR	Number of Doses	Observed AESI	Observed Person-Time (Years)	RR	Number of Doses	Observed AESI	Observed Person-Time (Years)	RR
Acute Myocardial Infarction	BNT162b2	5,019,206	287	286,238	0.94	7,071,320	402	395,810	0.89	3,924,085	253	212,161	1.08
	mRNA-1273	2,563,618	214	181,507	1.03	3,959,087	314	277,351	0.91	2,061,250	191	136,166	1.15
	Ad26.COV2.S	244,144	22	17,097	1.05	410,337	47	28,898	1.18	182,559	23	12,830	1.41
Deep Vein Thrombosis	BNT162b2	5,008,608	554	330,015	0.88	7,056,415	784	471,740	0.84	3,914,571	510	257,894	0.97
	mRNA-1273	2,557,370	372	181,145	0.90	3,949,117	622	284,292	0.90	2,055,345	373	144,691	1.05
	Ad26.COV2.S	243,754	43	17,080	1.08	409,584	66	29,015	0.89	182,160	44	12,987	1.34
Pulmonary Embolism	BNT162b2	5,012,070	491	330,248	1.05	7,061,704	703	480,890	1.28	3,917,508	419	263,292	1.33
	mRNA-1273	2,559,512	287	181,298	0.93	3,952,765	484	291,066	1.19	2,057,241	290	148,429	1.36
	Ad26.COV2.S	243,960	41	17,093	1.39	409,983	53	29,172	1.23	182,303	42	13,061	2.14
Disseminated Intravascular Coagulation	BNT162b2	5,023,766	<11	331,025	0.68	7,077,841	20	454,338	1.04	3,928,054	<11	246,883	0.75
	mRNA-1273	2,566,651	<11	181,806	1.13	3,963,914	<11	270,691	0.67	2,064,003	<11	136,647	0.78
	Ad26.COV2.S	244,438	0	17,127	0.00	410,855	<11	28,770	2.04	182,820	<11	12,854	0.00
Non-hemorrhagic Stroke	BNT162b2	5,021,535	155	330,879	0.84	7,074,745	203	463,405	0.86	3,926,134	133	246,760	1.04
	mRNA-1273	2,565,297	102	181,710	0.82	3,961,715	163	277,911	0.91	2,062,693	92	136,557	1.03
	Ad26.COV2.S	244,335	<11	17,120	0.33	410,643	25	28,933	1.24	182,699	<11	12,846	0.59
Hemorrhagic Stroke	BNT162b2	5,023,270	49	330,993	0.93	7,077,241	65	463,570	1.09	3,927,632	50	252,624	1.29
	mRNA-1273	2,566,330	39	181,783	1.13	3,963,522	46	278,041	1.05	2,063,645	20	140,920	0.75
	Ad26.COV2.S	244,425	<11	17,126	0.86	410,812	<11	28,945	1.21	182,798	<11	12,946	0.77
Immune Thrombocytopenia	BNT162b2	5,021,182	107	430,376	0.81	7,073,917	192	624,983	1.26	3,925,794	107	342,468	1.28
	mRNA-1273	2,565,158	89	234,123	1.12	3,961,662	124	375,130	1.21	2,062,548	77	191,084	1.49
	Ad26.COV2.S	244,341	13	25,382	1.53	410,664	<11	43,453	0.81	182,739	11	19,418	2.03
Myocarditis/Pericarditis	BNT162b2	5,021,652	264	430,415	1.73	7,074,944	322	625,077	1.83*	3,926,219	243	342,505	2.47*
	mRNA-1273	2,565,543	125	234,160	1.33	3,962,191	191	375,181	1.62	2,062,979	118	191,126	1.92
	Ad26.COV2.S	244,356	14	25,382	1.27	410,677	25	43,455	1.63	182,728	<11	19,417	1.44
Guillain-Barré Syndrome	BNT162b2	5,023,855	13	430,610	1.21	7,078,019	11	499,735	1.11	3,928,151	<11	282,110	1.05
	mRNA-1273	2,566,689	<11	234,266	1.40	3,963,994	<11	343,807	0.85	2,064,026	<11	179,364	1.34
	Ad26.COV2.S	244,448	<11	25,392	6.53	410,867	<11	42,672	8.53	182,825	<11	19,176	2.31
Bell's Palsy	BNT162b2	5,538,066	422	474,536	0.88	7,758,783	601	684,770	0.98	4,399,969	360	382,934	1.10
	mRNA-1273	2,844,137	259	259,242	0.86	4,338,960	415	410,068	1.01	2,321,435	241	214,305	1.17
	Ad26.COV2.S	268,766	49	27,862	1.49	446,543	74	47,119	1.46	206,493	25	21,824	1.13
Encephalo- myelitis/ Encephalitis	BNT162b2	5,541,351	18	474,821	1.48	7,763,335	18	640,440	1.43	4,402,681	11	347,757	1.67
	mRNA-1273	2,846,033	<11	259,415	0.90	4,341,907	13	374,979	1.82	2,323,255	<11	187,781	1.89
	Ad26.COV2.S	268,944	<11	27,881	1.26	446,815	<11	46,229	6.48	206,652	<11	21,177	0.00
Anaphylaxis	BNT162b2	6,076,878	26	32,601	4.48*	8,627,389	42	46,920	7.50*	4,989,398	39	27,296	10.86*
	mRNA-1273	3,135,659	20	16,852	7.64*	4,836,013	33	26,284	11.88*	2,644,637	20	14,465	12.40*
	Ad26.COV2.S	297,441	<11	1,525	4.05	500,220	<11	2,580	10.47	236,326	<11	1,234	20.41
Transverse Myelitis	BNT162b2	5,023,831	<11	430,608	0.70	7,078,019	<11	511,896	0.81	3,928,148	<11	326,988	1.26
	mRNA-1273	2,566,683	<11	234,265	0.89	3,963,962	<11	354,221	0.70	2,064,016	<11	179,776	1.30
	Ad26.COV2.S	244,445	<11	25,392	5.05	410,867	<11	42,968	7.41	182,822	<11	19,184	3.82
Narcolepsy	BNT162b2	5,020,198	133	430,293	0.74	7,072,100	237	624,826	1.07	3,924,967	132	342,395	1.35
	mRNA-1273	2,564,481	83	234,066	0.78	3,960,148	143	374,990	1.02	2,061,955	78	191,032	1.36
	Ad26.COV2.S	244,273	12	25,374	1.01	410,465	16	43,433	0.94	182,663	<11	19,410	1.63
Appendicitis	BNT162b2	5,016,516	617	429,981	1.09	7,068,850	744	598,204	1.01	3,922,860	449	326,528	1.32
	mRNA-1273	2,563,415	295	233,967	0.95	3,959,186	428	354,296	1.08	2,061,316	219	179,531	1.27
	Ad26.COV2.S	244,128	51	25,358	1.39	410,355	50	42,933	0.97	182,566	37	19,156	1.84
Common Thromboses with Thrombocytopenia	BNT162b2	5,022,754	86	330,958	1.03	7,076,575	94	473,091	1.08	3,927,012	81	258,722	1.31
	mRNA-1273	2,566,085	48	181,765	0.86	3,962,989	63	285,291	0.97	2,063,270	49	145,252	1.15
	Ad26.COV2.S	244,393	<11	17,124	1.46	410,783	13	29,098	1.80	182,766	<11	13,029	0.98
Unusual Site Thromboses (Broad) with Thrombocytopenia	BNT162b2	5,023,838	11	331,030	1.42	7,077,950	11	463,616	1.34	3,928,087	<11	212,417	0.96
	mRNA-1273	2,566,664	<11	181,806	0.99	3,963,936	<11	278,070	0.68	2,063,992	<11	136,353	0.92
	Ad26.COV2.S	244,449	<11	17,127	1.95	410,861	<11	28,949	4.46	182,821	<11	12,848	2.34

*indicates the AESI for the specific brand has signaled in the data.

Abbreviation: AESI, adverse event of special interest; RR, rate ratio.

Results presented for Optum data through 1/22/2022, HealthCore data through 1/7/2022, CVS data through 12/31/2021.

The number of sequential tests conducted varied by database and AESI.

Cell sizes 1–10 were masked for confidentiality.

In the secondary analysis assessing the risk of anaphylaxis, anaphylaxis met the statistical threshold for a signal in the all-dose analysis in all three databases following BNT162b2 vaccination (Optum RR = 4.48, N = 26; HealthCore RR = 7.50, N = 42; CVS Health RR = 10.86, N = 39) and mRNA-1273 vaccination (Optum RR = 7.64, N = 20; HealthCore RR = 11.88, N = 33; CVS Health RR = 12.40, N = 20) (Table 2). Similar results were observed in a stratified analysis by vaccine dose number and brand (Supplementary Table 4). No signals were observed for any of the other AESI in the all-dose or dose-stratified analyses.

4. Discussion

Our near real-time surveillance or RCA of 17 AESI following over 32 million COVID-19 vaccine doses among more than 16 million persons aged 12 to 64 years identified no safety signals for 15 AESIs and signals for myocarditis/pericarditis and anaphylaxis after mRNA vaccination. These signals have been reported in previous studies [8], [9], [10], [15], [16], [17], [18]. Myocarditis/pericarditis following a BNT162b2 vaccination met the pre-specified statistical threshold for a signal in HealthCore and CVS Health but not in the Optum database. Anaphylaxis following both BNT162b2 and mRNA-1273 met the pre-specified statistical threshold for a signal in all three databases, although the analysis was pre-specified as secondary as it has been identified as a signal for COVID-19 mRNA vaccines previously.

Aligned with results of this study, an elevated risk of myocarditis/pericarditis following mRNA vaccines (BNT162b2 and mRNA-1273), especially among younger males (under 40 years old), has been reported in spontaneous reporting systems [15], [16], [17], [18] and several observational studies [19], [20]. In conjunction with this study, we conducted ad-hoc analyses comparing observed myocarditis incidence rates to expected incidence rates by age group, vaccine dose, and sex, including a meta-analysis of results from all databases [21], [22]. The results showed an elevated risk of myocarditis/pericarditis following COVID-19 mRNA vaccines, and the highest risk was observed in males aged 18–25 years.

Safety signals were not detected following vaccination with Ad26.COV.2.S in our study. The much smaller number of Ad26.COV.2.S doses than mRNA vaccine doses in our surveillance system reflects the distribution of brand-specific vaccine coverage in the US and may have limited the power of this study to detect potential safety signals for extremely rare outcomes following vaccination with Ad26.COV.2.S. Passive surveillance systems including the US Vaccine Adverse Event Reporting System (VAERS) and case series reported risks of TTS [23], [24], [25], GBS [26], [27], and immune thrombocytopenia (ITP) [28] following vaccination with Ad26.COV.2.S. Though limited by potential underreporting and the absence of a direct comparator group, VAERS serves as a complementary early warning system for safety monitoring. The lack of detection of signals for TTS, GBS, and ITP after receipt of Ad26.COV2.S vaccines in our study is consistent with interim results from a RCA conducted in another active surveillance system in the US, Vaccine Safety Datalink (VSD), using a similar sequential testing framework [4], [29], [30]. Further analysis from VSD found an elevated risk of GBS following Ad26.COV.2.S compared to historical rates [30]. While our analysis found a relatively large rate ratio associated with GBS following Ad26.COV2.S, the test did not meet the statistical threshold for a signal and reflected substantial uncertainty with very few observed events.

There are several strengths in this near real-time safety surveillance of COVID-19 vaccines. In this large study of more than 16 million persons 12–64 years, we were able to detect an increase in the relative risk of rare outcomes for vaccines that may not be captured in pre-authorization clinical trials. The near real-time safety surveillance study included a large, geographically diverse, commercially insured population from three US health insurance administrative claims databases using the same protocol, analysis, and programming specifications. The study rapidly screened for elevated risks in multiple outcomes (AESI) simultaneously following COVID-19 vaccination and enabled early detection of potential safety signals as exposures accrued in the population.

There are several limitations in this study. First, this study used a rapid monitoring method designed for early detection of a potentially increased risk of an adverse event, crossing a pre-specified threshold. The results of such a study do not establish a causal relationship between an exposure and an outcome. The signal may not persist in a fully adjusted epidemiologic study as the comparator group was selected from a historical patient population without selection criteria to ensure comparability with the COVID-19 vaccinated treatment group. Moreover, the statistical analyses did not adjust for confounding factors other than age and sex. This study design is intended as a screening method and an alert system for further investigation. Therefore, the results of this study should be interpreted in the context of its methods utility and limitations, and the validity of detected signals must be evaluated in a fully adjusted epidemiologic study. Second, observed events were identified by reimbursement codes in claims databases, which are subject to coding errors and do not necessarily reflect the presence of a true case. Medical record review is ongoing to validate a subset of claims-based outcome definitions. Also, only conditions that triggered a health encounter were captured by claims data. Therefore, AESI that did not require medical attention would be absent in the claims databases and lead to potential outcome misclassification and false negative results. Third, the AESI risk intervals were pre-specified based on literature review and clinical input, but these may be subject to misclassification or may be incorrect for the COVID-19 vaccines. True signals may not be identified if the AESI risk is delayed or if the true AESI risk intervals are longer. Fourth, although the overall population in RCA provides greater precision to detect rare adverse events, AESI risk of certain subgroups may be masked. Fifth, differences in the health plan enrollee population for each health plan may result in different findings even though a common protocol and analysis were used. This difference may explain our detection of a signal for myocarditis/pericarditis in two of three and not all three data sources in the study. Sixth, COVID-19 vaccination administration data are not fully captured in claims data particularly with respect to COVID-19 vaccines although efforts are underway linking the three claims databases to the Immunization Information Systems (IIS) across the country where COVID-19 vaccine administration to a large proportion of the US population is captured. Lower capture of vaccine exposure can limit the power of the study and cause AESI following vaccinations not to be captured. Seventh, the sequential testing statistics require a wide range of pre-specified parameters, and their misspecification could result in false positive or false negative results. Eighth, while some uptake of booster doses in late 2021 was captured in the all-dose mRNA vaccine analyses, this study did not include third or booster dose specific analyses. Further study of booster doses with an extended study period is ongoing. Lastly, the results of this surveillance in commercially insured populations may not be generalizable to those who are uninsured or covered by other health insurance plans.

In conclusion, in our rapid safety surveillance of 17 AESI following COVID-19 vaccination in three large commercial insurance databases, we identified signals for myocarditis/pericarditis after BNT162b2 vaccination and anaphylaxis after both BNT162b2 and mRNA-1273 mRNA vaccines in vaccine recipients aged 12–64 years. Both signals have been reported by other studies and surveillance systems. We continue to monitor the safety of COVID-19 vaccines in general, which includes an ongoing RCA in the pediatric population. The FDA BEST Initiative plays a major role as part of a larger US federal surveillance effort to monitor and enhance safety profile of COVID-19 vaccines and contribute to regulatory decision-making that further protects public health during the current pandemic.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: John Seeger reports a relationship with Optum Inc that includes: employment and equity or stocks. Kandace Amend reports a relationship with Optum Inc that includes: employment and equity or stocks. Jennifer Song reports a relationship with Optum Inc that includes: employment and equity or stocks. Robin Clifford reports a relationship with Optum Inc that includes: equity or stocks. Cheryl N. McMahill-Walraven reports a relationship with CVS Health Clinical Trial Services LLS that includes: employment. Djenaba Audrey Djibo reports a relationship with CVS Health Clinical Trial Services LLS that includes: employment. Charlalynn Harris reports a relationship with CVS Health Clinical Trial Services LLS that includes: employment. Jennifer L. Pigoga reports a relationship with CVS Health Clinical Trial Services LLS that includes: employment.

Appendix A. Supplementary data

The following are the Supplementary data to this article: