Contextualising adverse events of special interest to characterise the baseline incidence rates in 24 million patients with COVID-19 across 26 databases: a multinational retrospective cohort study

Summary

Background

Adverse events of special interest (AESIs) were pre-specified to be monitored for the COVID-19 vaccines. Some AESIs are not only associated with the vaccines, but with COVID-19. Our aim was to characterise the incidence rates of AESIs following SARS-CoV-2 infection in patients and compare these to historical rates in the general population.

Methods

A multi-national cohort study with data from primary care, electronic health records, and insurance claims mapped to a common data model. This study's evidence was collected between Jan 1, 2017 and the conclusion of each database (which ranged from Jul 2020 to May 2022). The 16 pre-specified prevalent AESIs were: acute myocardial infarction, anaphylaxis, appendicitis, Bell's palsy, deep vein thrombosis, disseminated intravascular coagulation, encephalomyelitis, Guillain- Barré syndrome, haemorrhagic stroke, non-haemorrhagic stroke, immune thrombocytopenia, myocarditis/pericarditis, narcolepsy, pulmonary embolism, transverse myelitis, and thrombosis with thrombocytopenia. Age-sex standardised incidence rate ratios (SIR) were estimated to compare post-COVID-19 to pre-pandemic rates in each of the databases.

Findings

Substantial heterogeneity by age was seen for AESI rates, with some clearly increasing with age but others following the opposite trend. Similarly, differences were also observed across databases for same health outcome and age-sex strata. All studied AESIs appeared consistently more common in the post-COVID-19 compared to the historical cohorts, with related meta-analytic SIRs ranging from 1.32 (1.05 to 1.66) for narcolepsy to 11.70 (10.10 to 13.70) for pulmonary embolism.

Interpretation

Our findings suggest all AESIs are more common after COVID-19 than in the general population. Thromboembolic events were particularly common, and over 10-fold more so. More research is needed to contextualise post-COVID-19 complications in the longer term.

Funding

None.

Research in context.

Evidence before this study

During the rollout of the COVID-19 vaccinations, regulatory authorities paid special attention to a pre-specified list of adverse events of particular concern (AESIs). Some of the pre-specified AESIs have since been related with vaccinations. Not only are some AESIs potentially related with COVID-19 vaccinations, but also with the virus itself. To comprehend the benefit-risk of COVID-19 vaccinations, we must be aware of the projected COVID-19 infection rates. A literature search was conducted up until March 31, 2022 in PubMed to determine what evidence has previously been published on the occurrence of AESI after COVID-19 infection. Included in the search criteria were “COVID-19” and each of the 16 AESIs evaluated (the original 15 AESIs plus thrombosis with thrombocytopenia). There were 63 publications discovered and evaluated. However, the investigations were conducted on a single database or a small number of databases, and the majority of the papers concerned AESIs after immunization, with no mention of post-COVID occurrences.

Added value of this study

We demonstrate that all 16 studied AESIs are more common after COVID-19 than expected in the general population. Thromboembolic events were particularly common, and over 10-fold more so. These findings highlight the need for more research to contextualize post-COVID-19 complications in the longer term.

Implications of all the available evidence

Our study shows considerable database heterogeneity across the AESIs, suggesting individual study estimates should be interpreted with caution. Comparing the ‘Subjects with COVID-19’ to the ‘Pre-Pandemic Background Population’ shows a fairly consistent elevated rate in experiencing an AESI within 90-days after index. The results of this work are of public health importance as they help to put into perspective the risk of the AESIs post vaccination versus post SARS-CoV-2 infection.

Introduction

Since coronavirus disease 2019 (COVID-19) was first reported until July 2022, over 554 million cases and 6.3 million deaths have been reported worldwide.¹ COVID-19 vaccines have demonstrated to reduce COVID-19 hospitalisations and deaths, both in randomised controlled trials and in real-world observational studies.^2,3 As of 30th May 2022, more than 5 billion people (approximately 67% of the global population) have received at least one dose of a COVID-19 vaccine.⁴ At least 39 vaccines have been authorised or approved for use by at least one country, of which 11 have been granted emergency use by the World Health Organization (WHO). Although vaccine safety has been rigorously monitored in clinical trials, rare adverse events can go undetected during vaccine development due to limited number of trial participants, a relatively short follow-up duration, and insufficient generalisability to the broader population.

Throughout the roll-out of the COVID-19 vaccines, adverse events of special interest (AESIs) have been pre-specified to be monitored by medicines regulatory authorities. To provide context for these safety investigations, Li et al.,⁵ in a multinational network cohort study, reported heterogeneity in the background incidence rates of 15 AESIs across age and sex stratification as well as various administrative claims and electronic health records (EHR) databases. Against this backdrop, some vaccine-related adverse events have gone on to be associated with the vaccines; two within the originally pre-specified 15 AESIs (myocarditis and pericarditis⁶ and Guillain-Barré syndrome (GBS)⁷), and one newly-identified AESI (thrombosis with thrombocytopenia syndrome⁸).

It is, however, important to note that some AESIs are not only potentially associated with the COVID-19 vaccines, but with COVID-19 itself. And in order to understand the benefit risk of COVID-19 vaccines it is important to contextualise this with expected rates following COVID-19. To address this problem, the Observational Health Data Sciences and Informatics (OHDSI)⁹ community carried out a network study using observational data from 26 databases across 11 countries with the aim of estimating the incidence rates of 16 vaccine related AESIs among people who had COVID-19, as compared to the background population during the pre-pandemic phase.

Methods

Data sources

Rates were obtained from 26 databases, which included 8 administrative claims databases, 12 EHRs, 1 EHR with a registry, and 5 general practitioner (GP) databases. These databases represented 11 countries: Belgium, Estonia, France, Germany, Japan, the Netherlands, Serbia, Spain, Turkey, the United Kingdom (UK), and the United States of America (US). All of these databases represent subsets of the total population from which they originate.

The administrative claims databases were: IBM® MarketScan® Commercial Claims and Encounters Database (IBM_CCAE); IBM® MarketScan® Multi-State Medicaid Database (IBM_MDCD); IBM® MarketScan® Medicare Supplemental and Coordination of Benefits Database (IBM_MDCR); IQVIA LRxDX Open Claims (IQVIA_OPENCLAIMS); IQVIA PharMetrics Plus (IQVIA_PharMetrics); JMDC¹⁰; Optum De-Identified Clinformatics® Data Mart Database–Socio-Economic Status (SES) (OPTUM_SES); and University of Tartu (U_OF_TARTU). The EHR systems were: Health Data Warehouse of Assistance Publique–Hopitaux de Marseille (APHM); University of Colorado Anschutz Medical Campus- Health Data Compass (CU_AMC); Columbia University Irving Medical Center (CUIMC); Fundación para la Investigación e Innovación Biosanitaria en Atención Primaria COVID-19 (FIIBAP); Health Informatics Centre (HIC); Parc de Salut Mar Barcelona Information System (IMASIS); Istanbul Faculty of Medicine, Istanbul University (IU); Medaman Hospital Data (MHD); Optum® de-identified Electronic Health Record Dataset (OPTUM_EHR); STAnford medicine Research data Repository-OMOP (STARR)¹¹; University Clinical Center of Serbia (UCCS); and the University of California Health Data Warehouse (UCHDW). The EHR paired with a registry was the UK Biobank (UK_BIOBANK).¹² Finally, the GP databases were: Clinical Practice Research Datalink AURUM (CPRD_AURUM); Integrated Primary Care Information (IPCI)¹³; IQVIA Disease Analyzer France (IQVIA_FRANCE_DA)¹⁴; IQVIA Disease Analyzer Germany (IQVIA_GERMAN_DA)¹⁴; and The Information System for Research on Primary Care (SIDIAP).¹⁵

Table 1 provides a high-level overview of the databases. All datasets were mapped to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) V5.3.1 or higher⁹ which is maintained by OHDSI. See Appendix 1 for database details.

Table 1.

Database characteristics.

Name	Country	Data provenance	Dates covered YYYY/MM	Total persons	% F	Ages covered mean age (IQR)	Patients with COVID-19 #	Pre-pandemic background population #	Measurements with values
APHM	FR	EHR	1998/11–2021/07	2,465,265	51.9	40 (21–60)	11,431	673,031	N
CPRD_AURUM	UK	GP	1995/01–2021/03	39,879,547	51.8	31 (18–44)	587,886	14,094,032	Y
CU_AMC	US	EHR	2011/01–2022/04	4,795,392	54.0	39 (22–56)	72,648	830,579	Y
CUIMC	US	EHR	1985/03–2021/08	6,808,470	55.8	38 (21–56)	28,044	1,197,983	Y
FIIBAPa	ES	EHR	2001/03–2021/10	292,305	54.7	41 (23–58)	7138	78	N
HIC	SC	EHR	2005/01–2021/12	1,254,464	50.4	45 (28–66)	11,813	885,236	Y
IBM_CCAE	US	Claims	2000/01–2021/07	159,440,276	51.1	31 (17–46)	983,089	23,483,191	N
IBM_MDCD	US	Claims	2006/01–2020/12	32,806,887	56.2	24 (5–38)	196,997	11,810,505	N
IBM_MDCR	US	Claims	2000/01–2021/07	10,356,249	55.3	72 (65–77)	41,542	1,467,963	N
IMASIS	ES	EHR	1990/02–2021/07	976,524	47.4	38 (23–54)	9330	198,012	Y
IPCI	NL	GP	2006/01–2021/06	2,529,355	51.2	37 (18–55)	91,759	1,329,674	Y
IQVIA_FRANCE_DA	FR	GP	2016/07–2021/06	3,767,012	52.3	38 (18–56)	2859	1,394,912	N
IQVIA_GERMANY_DA	DE	GP	2011/04–2021/03	30,780,239	55.7	44 (25–62)	45,508	9,040,531	N
IQVIA_OPENCLAIMS	US	Claims	2000/01–2021/10	306,000,000	52.6	34 (14–52)	17,848,443	306,000,000	N
IQVIA_PHARMETRICS	US	Claims	2013/01–2021/09	166,422,594	50.6	38 (19–50)	1,593,578	46,947,246	N
IU	TR	EHR	2018/01–2021/10	899,515	53.0	35 (18–52)	6194	619	Y
JMDC	JP	Claims	2005/01–2021/03	12,541,088	48.6	32 (19–46)	17,564	6,680,196	N
MHD	BE	EHR	2015/07–2021/12	117,131	50.9	52 (29–70)	203	23,754	N
OPTUM_EHR	US	EHR	2007/01–2021/03	99,454,715	53.3	37 (19–56)	693,334	41,281,147	Y
OPTUM_SES	US	Claims	2000/05–2021/06	90,285,937	50.5	36 (19–52)	899,986	17,212,611	Y
SIDIAP	ES	GP	2003/01–2021/06	8,022,374	50.1	35 (17–51)	495,237	5,934,449	Y
STARR	US	EHR	2008/01–2022/04	3,475,673	53.6	36 (18–54)	31,928	1,118,549	Y
U_OF_TARTUb	EE	Claims	2021/01–2021/02	386,557	53.2	39 (21–57)	84,957	376,842	N
UCCS	RS	EHR	2018/10–2021/03	823,962	54.1	51 (35–67)	16,764	49,643	Y
UCHDWa	US	EHR	2012/01–2022/05	316,119	53.8	37 (12–54)	61,037	240,831	Y
UK_BIOBANK	UK	EHR + Registry	1970/02–2020/07	502,504	54.4	58 (40–69)	1717	458,889	Y
Total	–	–	–	945,520,607	–	–	23,840,986	492,730,503	–

APHM = Health Data Warehouse of Assistance Publique - Hopitaux de Marseille, BE = Belgium, CPRD_AURUM = Clinical Practice Research Datalink AURUM, CU_AMC = University of Colorado Anschutz Medical Campus- Health Data Compass, CUIMC = Columbia University Irving Medical Center, DE = Germany, EE = Estonia, EHR = Electronic Health Record, ES = Spain, F = Female, FIIBAP = Fundación para la Investigación e Innovación Biosanitaria en Atención Primaria COVID19, FR = France, GP = General Practitioner, HIC = Health Informatics Centre, IBM_CCAE = IBM® MarketScan® Commercial Claims and Encounters Database, IBM_MDCD = IBM® MarketScan® Multi-State Medicaid Database, IBM_MDCR = IBM® MarketScan® Medicare Supplemental and Coordination of Benefits Database, IMASIS = Parc de Salut Mar Barcelona Information System, IPCI = Integrated Primary Care Information, IQR = Interquartile Range, IQVIA_FRANCE_DA = IQVIA Disease Analyzer France, IQVIA_GERMAN_DA = IQVIA Disease Analyzer Germany, IQVIA_OPENCLAIMS = IQVIA LRxDX Open Claims, IQVIA_PHARMETRICS = IQVIA Pharmetrics, IU = Istanbul Faculty of Medicine, Istanbul University, JP = Japan, MHD = Medaman Hospital Data, N = No, NL = The Netherlands, OPTUM_EHR = Optum® de-identified Electronic Health Record Dataset, OPTUM_SES = Optum De-Identified Clinformatics® Data Mart Database - Socio-Economic Status, RS = Serbia, SC = Scotland, SIDIAP = The Information System for Research on Primary Care, STARR = STAnford medicine Research data Repository-OMOP, TR = Turkey, U_OF_TARTU = University of Tartu (U_OF_TARTU), UCCS = University Clinical Center of Serbia, UCHDW = University of California Health Data Warehouse, UK = United Kingdom, UK_BIOBANK = UK Biobank, US = United States, Y = Yes.

^aCOVID-19 only subset.

^bCOVID-19 + Controls.

The use of APHM was approved by the health data access commission (CADS). The use of CPRD_AURUM was approved by the Research Data Governance Process, protocol 20_000211. The use of IPCI database was approved by its governance board (2020-04). For UK_BIOBANK, ethical approval was provided from the UKB Access Review Board, reference 58,356 “Defining and redefining human disease at scale: an atlas of the human phenome.” COVID-19 research using UK Biobank data has been outlined here: https://www.ukbiobank.ac.uk/learn-more-about-uk-biobank/covid-19-hub. Participant data for this project are available directly from the UKB following a protocol review and contractual agreements, more information can be found on the UKB website and participants that had withdrawn consent were excluded from the study. The use of de-identified STARR-OMOP data and contribution of aggregate results has been approved (IRB Panel #8 RB-53248, registered to Lelan Stanford Junior University) under the Stanford Human Research Protection Program (HRPP). The following databases had institutional review board or ethics committee approval: CU_AMC (IRB COMIRB #21-2869), CUIMC (IRB protocol AAAO7805), FIIBAP (through Hospital Universitario 12 de Octubre, project code: 22/295), HIC (through Caldicott Guardian(s), reference number IGTCAL9397), IMASIS (CEIC Parc de Salut Mar, ref. Code 2021/9975), IU (ref. No 2021/2190), MHD (approved 7 FEB 2022), SIDIAP (project code: 21/052-PCV), U_OF_TARTU (protocol 330/T-10), and UCCS (Decision 410-9).

The New England Institutional Review Board (IRB) has determined that studies conducted in IBM_CCAE, IBM_MDCD, IBM_MDCR, OPTUM_SES, and OPTUM_EHR are exempt from study-specific IRB review, as these studies do not qualify as human subjects research. Research activity associated with access to the UC-wide UCHDW COVID Research Data Set (CORDS) Limited Data Set does not involve human subjects as defined by the federal regulations summarized in 45 CFR 46.102(e), and hence does not require IRB review. Based on Ethical Guidelines for Epidemiological Research issued by the Japanese Ministry of Health, Labor and Welfare, ethics approval and informed consent were not applicable for use of JMDC. For use of IQVIA_FRANCE_DA, IQVIA_GERMAN_DA, IQVIA_OPENCLAIMS, and IQVIA_PAHRMETRICS, no patient permission was necessary because all patient data are deidentified for research purposes.

Study participants

We determined the incidence of AESI within two target populations: a ‘Pre-Pandemic Background Population’ from 2017 to 2019 and ‘Patients with COVID-19’ from 2020 to 2022. This work focuses on the ‘Patients with COVID-19’ using the ‘Pre-Pandemic Background Population’ to understand the occurrence of those conditions in the absence of either COVID-19 or the vaccines.

We defined the ‘Pre-Pandemic Background Population’ cohort as patients who were observed in a database on the index dates 1 January 2017, 1 January 2018, or 1 January 2019 (i.e., the patient was active in the database on one or any of those dates they would be included in this cohort). The entry event was the date observed and patients were only included if they had at least 365 days of observable time prior to the index date. We used the time prior to the pandemic, 2017 till 2019, as a way to identify patients that we are confident have not been exposed, had, or were experiencing COVID-19. Once the COVID-19 pandemic started it would be hard to be confident a patient did not or does not have the disease.

The ‘Patients with COVID-19’ cohort was defined as persons with a positive laboratory test for SARS-CoV-2 (any SARS-CoV-2 laboratory testing method) or an observed diagnosis code for COVID-19 (without a negative test on or within 3 days after index). The index date, or entry event, was set to the first of these and did not have to occur on a specific month or day. Additional inclusion rules included keeping patients that had at least 365 days of observable time prior to the index date and having no diagnosis of COVID-19 or SARS-CoV-2 prior to index. The calendar time for this cohort included all time available within each database after 1 December 2019, see Table 1 for specifics (e.g., CPRD_AURUM last date of capture is March 2021). An additional ‘Patients with COVID-19’ variation cohort was created for sensitivity analysis; it was a similar cohort except with the additional inclusion rule that cohort entry had to be prior to the COVID-19 vaccines becoming available (i.e., index before 1 January 2021); this cohort was called ‘Patients with COVID-19 prior to 2021’. Patients were not excluded from the COVID-19 cohorts if they were also in the ‘Pre-Pandemic Background Population’. The codes for this COVID-19 definition are listed in the protocol appendix¹⁶ and have been used in prior studies.17, 18, 19, 20

All databases contributed to both study cohorts (‘Pre-Pandemic Background Population’ and ‘Patients with COVID-19’), except for FIIBAP and IU, which only contributed to the ‘Patients with COVID-19’ cohort as these databases only contained COVID-19 patients and not a general population.

This study uses coded data that already exist in an electronic database. Confidentiality of patient records was always maintained. All study reports contain only aggregate data and do not identify individual patients or physicians. At no point during the study was identifying information about the subjects used.

Adverse events of special interest (AESIs)

We studied 16 AESIs. This list includes the original 15 prespecified by regulatory authorities and the addition of thrombosis with thrombocytopenia syndrome (TTS) which was an adverse event discovered when COVID-19 vaccinations started. The 16 outcomes include: GBS, facial nerve (Bell's) palsy, anaphylaxis, encephalomyelitis, narcolepsy, appendicitis, non-haemorrhagic stroke, haemorrhagic stroke, acute myocardial infarction (AMI), myocarditis and pericarditis, deep vein thrombosis (DVT), pulmonary embolism, disseminated intravascular coagulation (DIC), immune thrombocytopenia (ITP), transverse myelitis, and the cooccurrence of thrombosis with thrombocytopenia (TWT) as a proxy to TTS. All AESI definitions were used in previous published studies⁵ and were based on the FDA's Center for Biologics Evaluation and Research protocol.²¹ The AESIs were reviewed within each data partner to ensure the outcome could be reliably captured and if not, that AESI would be excluded for that particular data partner (Appendix 2 provides further detail). The definitions are fully specified in the protocol.¹⁶

Statistical analysis

We defined time at risk as 1–90 days after the study participant index date. Patients contributed time-at-risk from the index date until the earliest of 90 days after the index, they leave the database, or the start date of the AESI event. Death was not explicitly used as a censoring criteria, as the capture of death varies within each database, including some databases that do not capture death at all. We do censor patients at the end of observation, whether that be death or other reason for loss-to-follow-up.

We also excluded individuals for a specific outcome if a prior event had occurred during the clean window for that outcome. A clean window is the minimum time between event occurrences to be considered a new event. We used an outcome-specific clean window, within which an outcome is not considered incident. The clean window was 365 days for all outcomes except 30 days for anaphylaxis and 183 days for facial nerve palsy and encephalomyelitis.²¹ The clean window was set different for anaphylaxis, facial nerve palsy, and encephalomyelitis to better define incident cases of these AESI.

Incidence rates were calculated as the total number of events divided by the person time at risk. The incidence rates were stratified by age and sex subgroups for each database. The age subgroups were (in years): 0–5, 6–17, 18–34, 35–54, 55–64, 65–74, 75–84, and ≥85. The age-specific rates for the AESIs were pooled across the databases using a random effects meta-analysis, with the DerSimonian-Laird method to estimate between database variation. The 95% prediction intervals were calculated using the R package “meta”. Prediction intervals reflect the expected uncertainty if an estimated rate from another study were included in the meta-analysis.

Indirect standardisation²² was used to account for differences between the age subgroups and sex distribution in the COVID-19 disease cohort and the ‘Pre-Pandemic Background Population’. Within each database, the pre-pandemic cohort was used as the demographic reference, such that an expected incidence rate could be computed using COVID-19-specific incidence rates weighted to reflect the ‘Pre-Pandemic Background Population’ demographics. COVID-19 cohort observed and expected rates were compared using standardised incidence ratios (SIR) with corresponding exact 95% confidence intervals.

Negative control outcomes were also used to evaluate potential bias in incidence ratio estimates. Twenty negative control outcomes were selected as diseases with no a priori evidence of a causal relationship with COVID-19, based on literature review and clinician adjudication (Appendix 3). Incidence rate and SIRs for these negative control outcomes were estimated in the same manner as for the 16 AESIs.

Meta-analytic age- and sex-specific rates were classified using the WHO Council for International Organizations of Medical Sciences thresholds: very common (≥10%), common (>1%–<10%), uncommon (≥0.1%–<1%), rare (≥0.01%–<0.1%), and very rare (<0.01%).²³

All analyses were performed in R. The study protocol and analysis code are available at https://github.com/ohdsi-studies/Covid19SubjectsAesiIncidenceRate.

Role of funding source

There was no funding source for this study. All authors approved the final version, had access to the data, and accept responsibility to submit the final version for publication.

Results

We included 23,840,986 ‘Patients with COVID-19’ from 26 databases representing a diverse set of care settings from North America, Europe, and Asia (Table 1). The percentage of females found across the databases ranged from 47.4% to 56.1%; all databases had slightly more females than males, except IMASIS and JMDC. The average age of persons in each database ranged from 24 to 72 years, reflecting the differing patient populations covered across the databases (e.g., IBM_MDCD has a large child and childbearing-aged female population, while IBM_MDCR represents retired individuals). The ‘Pre-Pandemic Background Population’ represented more than 492,730,503 person records. Characterisation results produced by CohortDiagnostics.²⁴ can be found on an interactive web app (https://data.ohdsi.org/Covid19SubjectsAesiIncidenceRate/).

Fig. 1 plots the incidence rates for the ‘Patients with COVID-19’ population, stratified by age group (x-axis), sex (half the x-axis represents females and the other half males), and database (color-coded by database type: claims, EMR, and GP centric data), trellised by the 16 AESIs. Similar age trends across databases were observed. Some AESIs had a clear increase in incidence rate with age: AMI, non-haemorrhagic stroke, DVT, PE, haemorrhagic stroke, Bell's Palsy, TWT, ITP, DIC, and GBS. In contrast, some AESIs had a clear decrease in incidence rate with age: appendicitis and anaphylaxis. Finally, some age trends were less clear: myocarditis and pericarditis, narcolepsy, encephalomyelitis, and transverse myelitis. Fig. 1 also shows substantial database heterogeneity within individual AESIs. For example, for DIC in the 90 days after COVID-19 diagnosis, males aged 35–54 in JMDC experienced the event 2417 per 100,000 person-years as compared to CPRD which experienced the event 5 per 100,000 person-years. Additionally, the incidence rates for ‘Patients with COVID-19’ were compared to the ‘Pre-Pandemic Background Population’ in Appendix 4. For most outcomes, across all database/age group/sex combinations, the incidence rates are higher for the ‘Patients with COVID-19’ compared to the ‘Pre-Pandemic Background Population’.

Fig. 1.

For ‘patients with COVID-19’, age group and sex stratified incidence rates of the 16 adverse events of special interest for 24∗ databases.∗ FIIBAP and IU did not contribute to the ‘Pre-Pandemic Background Population’ and thus and IR could not be calculated. APHM = Health Data Warehouse of Assistance Publique - Hopitaux de Marseille, CPRD_AURUM = Clinical Practice Research Datalink AURUM, CU_AMC = University of Colorado Anschutz Medical Campus- Health Data Compass, CUIMC = Columbia University Irving Medical Center, F = Female, FIIBAP = Fundación para la Investigación e Innovación Biosanitaria en Atención Primaria COVID19, HIC = Health Informatics Centre, IBM_CCAE = IBM® MarketScan® Commercial Claims and Encounters Database, IBM_MDCD = IBM® MarketScan® Multi-State Medicaid Database, IBM_MDCR = IBM® MarketScan® Medicare Supplemental and Coordination of Benefits Database, IMASIS = Parc de Salut Mar Barcelona Information System, IPCI = Integrated Primary Care Information, IQVIA_FRANCE_DA = IQVIA Disease Analyzer France, IQVIA_GERMAN_DA = IQVIA Disease Analyzer Germany, IQVIA_OPENCLAIMS = IQVIA LRxDX Open Claims, IQVIA_PHARMETRICS = IQVIA Pharmetrics, IU = Istanbul Faculty of Medicine, Istanbul University, M = Male, MHD = Medaman Hospital Data, OPTUM_EHR = Optum® de-identified Electronic Health Record Dataset, OPTUM_SES = Optum De-Identified Clinformatics® Data Mart Database–Socio-Economic Status, SIDIAP = The Information System for Research on Primary Care, STARR = STAnford medicine Research data Repository-OMOP, U_OF_TARTU = University of Tartu (U_OF_TARTU), UCCS = University Clinical Center of Serbia, UCHDW = University of California Health Data Warehouse, UK_BIOBANK = UK Biobank.

To help give an overview of these results, Fig. 2 shows the pooled estimated age- and sex-stratified incidence rates per 100,000 person years (PY) for AESI events in the 90-days after index for ‘Patients with COVID-19’. Nevertheless, the incidence of several AESIs differed greatly between age and sex strata. For example, pulmonary embolism was uncommon for patients less than 35 then becoming common for older individuals. The pattern was similar for AMI, non-haemorrhagic stroke, and DVT (with the exception of AMI and non-haemorrhagic stroke becoming common at an older age of 55). The incidence rate of haemorrhagic stroke and TWT was primarily uncommon becoming common later in life (75 years and 65 years respectively). The other AESIs were primarily uncommon with ages between 18 and 74, experiencing the event rarely. For comparison, we additionally produced this figure for the ‘Pre-Pandemic Background Population’ which can be found Appendix 5.

Fig. 2.

Pooled estimated age and sex stratified incidence rates per 100,000 person years (with 95% prediction intervals), calculated from meta-analyses for ‘Patients with COVID-19’.

Fig. 3 reports the meta-analytic estimates of SIRs comparing the ‘Patients with COVID-19' to the ‘Pre-Pandemic Background Population’. Twelve of 16 AESIs had ratios above 2 and 7 of 16 AESIs had ratios above 5. Pulmonary embolism had the highest SIRs (11.7 [95% confidence interval 10.1–13.7]), suggesting that the observed incidence of pulmonary embolism in the 90 days after COVID-19 diagnosis was over 11 times higher than expected in a 90-day period for the background population in the pre-pandemic period. The database specific results are reported in Appendix 6. The negative control outcome estimates are provided in Appendix 7 and reveal consistent positive bias in estimated SIR values. SIRs ranged from 0.9 to 5.0 with a majority of generated SIRs above 1.7. The meta-analysis was repeated for ‘Patients with COVID-19 prior to 2021’ to see if inclusion of COVID-19 vaccinated patients may have impacted the results however the results found were similar (Appendix 8).

Fig. 3.

Standardised incidence ratios forest plot with meta analysis comparing the ‘patients with COVID-19' to the ‘pre-pandemic background population’. CI = confidence interval.

Discussion

To our knowledge this is the largest study to date on the descriptive epidemiology of AESIs among the COVID-19 population. In this study we estimated age- and gender-specific incidence rates of 16 AESIs among COVID-19 patients using 26 observational data health sources from around the world. To contextualise our findings, we reported on the age- and gender-SIR, comparing the IR among COVID-19 cohort to the ‘Pre-Pandemic Background Population’ of the databases. Our findings suggest a consistent trend of an increased risk for multiple AESIs among the COVID-19 cohort when compared to the database ‘Pre-Pandemic Background Population’, though the magnitude of the risk should be interpreted with caution.

We found considerable heterogeneity in the IR among the COVID-19 cohort reflected by the wide prediction intervals of the pooled age- and sex-specific IR, suggesting that caution is needed when these estimates are used. We also observed considerable variability with age and some with sex groupings, emphasising the need for age- and sex-stratification when assessing risks and benefits of COVID-19 vaccines.

The observed magnitude of heterogeneity across sources within age and sex subgroups suggests that residual differences are present. The remaining heterogeneity may be related to differences in healthcare systems, settings, data capture processes, or true differences in subpopulations. A limitation of this work is we have not furthered stratified by comorbidities related to either COVID-19 or outcome (such as type 2 diabetes or obesity) or socio-economic status. These differences may also be due to systematic error, selection bias, or differential outcome measurement error between databases.²⁵ In a similar analysis across 13 databases, Li et al.⁵ reported that AESI background rates varied by database and regions. Consistent to our findings, the authors also found considerable differences in the incidence rates by age and sex, suggesting caution is needed when incidence rates are compared across populations.⁵

Among COVID-19 patients, the majority of COVID-19 AESIs were found to be “uncommon” with few “rare” among all age and gender groups investigated. Thrombotic events such as AMI, strokes, DVT, and pulmonary embolism were more frequent compared to other AESIs and were “common” in older COVID-19 patients. These findings are consistent with prior studies suggesting that cardiovascular and thrombotic complications in particular are relatively common post COVID-19,^26,27 especially among older patients. In most databases, the risk of these thrombotic events was higher among COVID-19 patients when compared to the ‘Pre-Pandemic Background population’ of the databases with a pooled SIR above 3. In particular, the SIR for pulmonary embolism had an elevated trend away from the null in all databases but one. It is well established that COVID-19 is associated with a hypercoagulable state, and older patients with additional risk factors have a worse outcome. Consistent with our results, clinical evidence suggests that COVID-19 is particularly related to pulmonary embolism²⁸ especially among those with pneumonia. However, until now, the precise incidence of thrombosis in patients with COVID-19 has not been determined, mainly because multiple studies have reported conflicting estimates.²⁹

The Centers for Disease Control and Prevention (CDC) lists myocarditis and pericarditis, thrombosis with thrombocytopenia syndrome (TTS), presented in this work as TWT, and GBS as three serious types of adverse events following COVID-19 vaccination, with evidence that suggests, although rare, a link to certain types of COVID-19 vaccinations.³⁰ These three adverse events are also listed on the label of at least one COVID-19 vaccine authorised for use in the US and Europe. Our findings suggest that, while still rare among COVID-19 patients, the risk of these three events may be higher in the COVID-19 patients when compared to the ‘Pre-Pandemic Background Population’. Since the estimated incidence ratios are unadjusted and are not presented for causal inference, future investigation may be warranted to fully contextualise the risk benefit balance of the COVID-19 vaccines as it relates to myocarditis and pericarditis, TWT, and GBS.

Multiple studies have concluded that COVID-19 disease may lead to neurological complications leading to GBS, and some studies also highlighted differences in the presentation of the disease, with greater severity of symptoms in GBS associated with COVID-19.31, 32, 33 Previous studies have also found that myocarditis and pericarditis may be part of the wide spectrum of cardiovascular sequelae of COVID-19 disease.^32,34,35 A CDC network retrospective cohort study³⁴ using data electronic medical records across multiple databases in the US, reported that among 814,524 COVID-19 patients the incidence of myocarditis or pericarditis varied by age and gender and ranged between 17.6 and 114.0 per 100,000 for males and between 10.8 and 61.7 per 100,000 for females in a 21 day risk window. The same study reported that the incidences of myocarditis or pericarditis after SARS-CoV-2 infection were higher than after the mRNA COVID-19 vaccination for both males and females in all age groups. To our knowledge, our study is the first to report on TWT incidence among ‘Patients with COVID-19’. However, prior work has highlighted the potential misclassification error related to identifying TTS (also known as vaccine-induced thrombotic thrombocytopenia [VITT]) in observational data.³⁶ The incidence rates of TWT reported in this study were higher among men of older ages which is inconsistent with known trends of TTS/VITT. Thus, these rates should be interpreted with caution.

Our findings suggest that COVID-19 disease itself must be considered when assessing the relationship between COVID-19 vaccines and the AESI. To be specific, our findings suggest that COVID-19 disease may be associated with at least some of the AESIs and may consequently exert a confounding or intermediating effect in the observed association between the vaccines and the AESI. When conducting observational studies investigating the association between the vaccines and the AESI, it is important to control for COVID-19.

A particular strength of this study is that it includes a large number of databases from around the world, covering a sizable study population with diversity in geographical location, underlying populations, pandemic status, health systems, and data types. Our analysis enabled a comprehensive and standardised assessment of incidence rates of AESIs among ‘Patients with COVID-19’ across multiple settings. This was possible due to the use of the OMOP CDM, which enabled use of the same study design and analytical code in all databases and to gather results from participating data partners rapidly and without transferring patient level data. All outcome definitions, clinical codes, and phenotype algorithms have been made open source and are available online for review and to maximise reproducibility and reuse.

A limitation of this analysis was we did not differentiate for the multiple variants of COVID-19, and we did not consider recurrent COVID-19. This means we cannot categorise or compare the differences in AESIs associated with the different variants or in patients who had COVID-19 multiple times. The incidence of the AESIs may change depending on the variant or number of infections, so this work cannot describe the incidence of an AESI for a given variant, be representative of future variants, or describe what happens for patients who had multiple infections of COVID-19.

An additional limitation of the analysis of the SIRs is that they do not fully account for confounding and were shown to exhibit bias with negative controls and therefore should not be interpreted as causal effect estimates. Also, the use of a historical population as a comparator is known to be associated with an elevated type 1 error, which may bias the results away from the null³⁷ (e.g., individuals with ‘Patients with COVID-19’ disease my tend to receive more clinical attention or examination than individuals in the ‘Pre-Pandemic Background Population’). Results from negative control outcomes in this work show a consistent positive bias (away from the null). These findings indicate that most associations observed in our study are potentially larger than the magnitude of the bias observed using negative controls. Another limitation of this study is that all outcomes and the COVID-19 definition itself are subject to measurement error. While most of the outcome definitions have been used in prior studies and were reviewed by clinicians and data experts, they were mainly based on the presence of specific diagnostic codes and were not validated further. There was significant heterogeneity in COVID-19 test and disease ascertainment across databases. COVID-19 severity is an important factor that was outside the scope of this analysis but may have affected the risk of AESIs and probably varied by database. When defining the ‘Pre-Pandemic Background Population’, we used data from 2017 to 2019 of all people in each database with more than 365 days of observation indexed on 1 January. The impact of these design decisions, in particular the index date (anchoring effect) has been shown to influence rate estimates, however the effect of season has been shown to be moderate.²⁵

Finally, there are some limitations related to participating databases. Information on hospital admissions was not available in all primary care databases used (Table 1) resulting in inpatient events not being captured. EHR databases are subject to incomplete capture of medical events that may occur but are recorded outside the participating health system. The bias of incomplete information was partially mitigated by including only those patients who had at least one year of continuous observation but defining continuous observation can be challenging across disparate databases. Administrative claims databases offered a potentially complete data capture but lacked some important data elements such as laboratory test results. We mitigated some of these limitations by providing within database incidence rate comparison. Additionally, the repeated influxes of vast numbers of critically ill COVID-19 cases suddenly overwhelmed hospitals and we cannot exclude changes in the organisation and thus the quality of coding in code-based administrative database. Finally, all our databases represented subsets of the population in which they originate, which poses a risk of selection bias.

Our study assessed the descriptive epidemiology of the occurrence of AESIs in the 90-days after COVID-19 disease. These results found large variations in the rates of AESIs in ‘Patients with COVID-19’ across age groups and sex, showing the need for stratification. Considerable database heterogeneity was found across the AESIs, suggesting individual study estimates should be interpreted with caution. Comparing the ‘Patients with COVID-19’ to the ‘Pre-Pandemic Background Population’ showed a fairly consistent elevated rate in experiencing an AESI within 90-days after index. This elevated risk we see both consistently across the database stratified results as well as the meta-analysis. The results of this work are of public health importance as they help put into perspective the risk of the AESIs post vaccination versus post SARS-CoV-2 infection.

Contributors

Using the CRediT, Contributor Roles Taxonomy³⁸ here are how the co-authors participated in this work. For conceptualization, or formulation of the research goals, the following authors participated: EAV, AS1 (Shoaibi), TA, AGS, GR, DRM, TDS, KN, CR, EPM, NHS, JDP, MTGM, DB, HCJ, ADH, MPJ, PSB, NGB, RV, AJB, AP2 (Patel), LD, CH, FA, AO, FN, GH, MAS, DPA, PRR, PBR. For data curation, or those who worked with research data, the following authors participated: EAV, CB, SvS, TLC, SH, DRM, JMR, JMRA, MAM, MdW, LHJ, TDS, AP1 (Pistillo), RK, FM, SD, VP, MGK, KN, CR, AS2, EPM, NHS, JDP, MTGM, DB, HCJ, ADH, MPJ, NGB, SS, AYU, BE, LB2 (Belmans), GB, DJD, ZC, RV, AJB, AP2, LD, CH, GH, DPA, PRR, PBR. For formal analysis, or those who analysed or synthesised study data, the following authors participated: EAV, AS1, LYHL, CB, AGS, GR, CF, JMRA, MAM, LHJ, AP1, FM, MGK, EPM, SS, AYU, BE, LB2, GB, AO, FN, DPA, MJS, PBR. For investigation or conducting of the research or data/evidence collection, the following authors participated: KH, DRM, JMRA, MAM, RK, SS, AYU, BE. Design of the methodology was performed by the following individuals: EAV, AS1, TA, AGS, GR, DRM, MGK, AS2, EPM, FA, AO, FN, GH, MAS, DPA, PRR, MJS, PBR. Project administration was performed by the following individuals: EAV, CB, MGK, EPM, NHS, GB, RV, AJB, AP2, LD, CH, PBR. Resources, or provision of computing resources or analysis tools, was performed by the following: DRM, JMR, JMRA, MAM, TDS, MGK, AS2, DPA, PRR. Software, or program development, was performed by: EAV, AGS, SH, LHJ, MGK, EPM, JDP, MAS, MJS, PBR. Supervision, or oversight, was performed by: LB1 (Boyer), DRM, TDS, AJB, AP2, LD, CH, MAS, DPA, PRR, MJS, PBR. Validation, or verification of results, was performed by: RM, LB1, SH, MdW, MGK, PBR. Visualization, or creation of visualizations or data presentation, was performed by: EAV, LYHL, CB, FB, PBR. Writing of the original draft was performed by: EAV, AS1, LYHL, TA, TDS, ER, FM, SD, VP, MTGM, DB, HCJ, ADH, MPJ, PSB, NGB, SS, AYU, BE, DJD, ZC, AO, FN, PBR. Editing and reviewing the original writing was performed by: EAV, AS1, LYHL, CB, TA, RM, KH, AGS, GR, SvS, CF, LB1, TLC, DRM, JMR, JMRA, MAM, LHJ, TDS, ER, AP1, FM, SD, MGK, KN, CR, AS2, EPM, NHS, JDP, MTGM, DB, HCJ, ADH, MPJ, PSB, NGB, SS, AYU, BE, LB2, GB, MLNGM, DJD, ZC, RV, AJB, AP2, LD, CH, FB, FA, AO, FN, GH, MAS, DPA, PRR, MJS, PBR. All authors approved the final version, had access to the data, and accept responsibility to submit the final version for publication. No competing interests. All authors approved the final version, had access to the data, and accept responsibility to submit the final version for publication.

Data sharing statement

The results of the analysis can be both found at https://data.ohdsi.org/Covid19SubjectsAesiIncidenceRate/ and an export of the incidence rates can be found in Appendix 4.

Some of the datasets used within this study are available via license. The data that support the findings of this study are available to license from IBM (IBM_CCAE, IBM_MDCD, IBM_MDCR), Optum (OPTUM_EHR, OPTUM_SES), JDMC, IQVIA (IQVIA_FRANCE_DA, IQVIA_GERMANY_DA, IQVIA_OPENCLAIMS, IQVIA_PHARMETRICS). Data are available from IBM at https://www.ibm.com/products/marketscan-research-databases, from Optum at https://www.optum.com/business/solutions/life-sciences/real-world-data.html, from JMDC at https://www.jmdc.co.jp/en/jmdc-claims-database/, and from IQVIA at https://www.iqvia.com/solutions/real-world-evidence/real-world-data-and-insights.

Outside the license data previously described, this study was performed as a federated network study, meaning the data remained with the data partner. Individual organisations would need to be contacted in order to gain access to those data assets.

Declaration of interests

EAV, AS, CB, RM, KH, AGS, GR, MJS, and PBR are employees of Janssen Research and Development LLC and shareholders of Johnson & Johnson (J&J) stock. Many data partners (including JMR, JMRA, MAM, MdW, LHJ, GB, PRR) on this project have received funding from the Innovative Medicines Initiative (IMI) 2 Joint Undertaking (JU) under grant agreement No 806968. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. The funding source had no role in writing the manuscript or the decision to submit it for publication. Odysseus Data Services, where SvS is employed, is a beneficiary of the mentioned IMI 2 JU project. DPA has received support from the EMA and Innovative Medicines Initiative. DPA's research group has received grant support from Amgen, Chesi-Taylor, Novartis, GSK, UCB Biopharma and J&J. DPA's department has received advisory/consultancy fees from AstraZeneca, Amgen, Astellas, J&J, and UCB Biopharma. DPA's department has received fees for speaker services from Amgen and UCB Biopharma. DPA's has other financial or non-financial interests in Janssen, on behalf of the IMI-funded EHDEN and European Medical Information Framework (EMIF) consortiums, and Synapse Management Partners have supported training programmes organised by DPA's department that was open for external participants. TDS received financial support from the Institute de Salud Carlos Ill (ISCIII; Miguel Servet 2021: CP21/00023) which is co-funded by the European Union. The funding source had no role in writing the manuscript or the decision to submit it for publication. PRR, MdW, LHJ work for a research group who received/receives unconditional research grants from Chiesi, Amgen, UCB, J&J, European Medicines Agency (EMA), and IMI. GH provided funding for use of the CUIMC database and the OHDSI infrastructure. GH received support through a grant to Columbia University (NIH R01 LM006910). MAS receives grants from the US Department of Veterans Affairs within the scope of this work, and grants and contracts from the US National Institutes of Health, US Food & Drug Administration, and Janssen Research & Development outside the scope of this work. EPM receives salary support from the Department of Medicine at University of Calgary. EPM receives consulting fees from Janssen Research & Development for work that is outside the scope of the current study. FM's main employment is for Heliant d.o.o, an EHR provider in Serbia. JDP received salary support through the Universidad de Norte. DJD and ZC are full-time employees of CPRD, which provides contract research services for government and related healthcare authorities, and the pharmaceutical industry, including Janssen. FN holds some AstraZeneca shares and was employed there until 2019. RV, AJB, AP2, LD, and CH received partial funding for this work by the University of California Health System and its Center for Data-Driven Insights and Innovation. MGK received support for the manuscript via institutional (internal funding) only. AJB's research has been funded by NIH, Peraton (as the prime on an NIH contract), Genentech, Johnson and Johnson, FDA, Robert Wood Johnson Foundation, Leon Lowenstein Foundation, Intervalien Foundation, Priscilla Chan and Mark Zuckerberg, the Barbara and Gerson Bakar Foundation, and in the recent past, the March of Dimes, Juvenile Diabetes Research Foundation, California Governor’s Office of Planning and Research, California Institute for Regenerative Medicine, L’Oréal, and Progenity. AJB receives royalty payments through Stanford University, for several patents and other disclosures licensed to NuMedii and Personalis. AJB is a co-founder and consultant to Personalis and NuMedii; consultant to Mango Tree Corporation, and in the recent past, Samsung, 10x Genomics, Helix, Pathway Genomics, and Verinata (Illumina); has served on paid advisory panels or boards for Geisinger Health, Regenstrief Institute, Gerson Lehman Group, AlphaSights, Covance, Novartis, Genentech, Merck, and Roche. AJB has received honoraria and travel reimbursement for invited talks from J&J, Roche, Genentech, Pfizer, Merck, Lilly, Takeda, Varian, Mars, Siemens, Optum, Abbott, Celgene, AstraZeneca, AbbVie, Westat, many academic institutions, medical or disease specific foundations and associations, and health systems. AJB is a shareholder in Personalis and NuMedii; is a minor shareholder in Apple, Meta (Facebook), Alphabet (Google), Microsoft, Amazon, Snap, 10x Genomics, Illumina, Regeneron, Sanofi, Pfizer, Royalty Pharma, Moderna, Sutro, Doximity, BioNtech, Invitae, Pacific Biosciences, Editas Medicine, Nuna Health, Assay Depot, Vet24seven, and several other non-health related companies and mutual funds. DRM has grants or contracts from Wellcome Trust Clinical Research Fellowship (214588/Z/18/Z). FB has a leadership or fiduciary role as the program chair of j-ISBA.