A scoping review of active, participant centred, digital adverse events following immunization (AEFI) surveillance of WHO approved COVID-19 vaccines: A Canadian immunization Research Network study

ABSTRACT

This scoping review examines the role of digital solutions in active, participant-centered surveillance of adverse events following initial release of COVID-19 vaccines. The goals of this paper were to examine the existing literature surrounding digital solutions and technology used for active, participant centered, AEFI surveillance of novel COVID-19 vaccines approved by WHO. This paper also aimed to identify gaps in literature surrounding digital, active, participant centered AEFI surveillance systems and to identify and describe the core components of active, participant centered, digital surveillance systems being used for post-market AEFI surveillance of WHO approved COVID-19 vaccines, with a focus on the digital solutions and technology being used, the type of AEFI detected, and the populations under surveillance. The findings highlight the need for customized surveillance systems based on local contexts and the lessons learned to improve future vaccine monitoring and pandemic preparedness.

Introduction

Post-market surveillance for adverse events following immunization (AEFI) from COVID-19 vaccines is a key priority amongst public health stakeholders and policy makers, with emphasis placed on the necessity for adequate, comprehensive, and adaptable population level safety monitoring.^1–3 Broadly speaking, AEFI surveillance can either be passive (unprompted, spontaneous reporting of events)^4,5 or active (deliberate prompting of participants and/or active case seeking to solicit event reporting),⁶ with data typically sourced from either healthcare providers, vaccinees, or both to monitor a population for safety signals.⁴ An increasingly recognized and emerging form of AEFI monitoring is active, participant-centered surveillance, which collects solicited health and/or reactogenicity information from vaccinees.⁵ In addition to classic analog approaches to active, participant-centered AEFI surveillance, such as health diary cards and interviews, a number of systems are employing digital solutions and technology, such as e-mail and short-message-system (SMS).⁵

Digital solutions have been utilized in many facets of public health measures one of which is pandemic planning and responses.^7–10 The technology has been implemented for some AEFI surveillance for monitoring the safety of vaccines, for example the CDC’s V-Safe app.¹¹ As the COVID-19 pandemic subsides there is an opportunity to learn from the implementation of the various digital solutions implemented in multiple jurisdictions. This information can be valuable for future pandemic as well as non-pandemic settings. Digital systems have various advantages. The major advantage is that they facilitate more real time AEFI surveillance which allows for more rapid detection of AEFI signals. Another advantage is that digital systems can be more easily standardized, which is key in implementing a “gold-standard” that is translatable across the international community, and they can capture large volumes of data and information.¹² A disadvantage surrounding digital systems is data privacy concerns. Patients may have concerns regarding how their health information is digitally handled which could hinder patient trust in the system.¹³ Digital systems also exclude individuals who may not be fluent with technology, such as older adults in long-term care. Finally, they are more expensive to implement and require more maintenance.^13–15

To assist in this regard, we conducted a scoping review to better understand the role of different technological and digital approaches to active, participant-centered, AEFI surveillance of COVID-19 vaccines during the early stages of the pandemic.

Methods

Our objectives were

1. To identify the published research (describe the extent, range, and nature of research activity)^12–14 of digital solutions and technology used for active, participant centered, AEFI surveillance of novel COVID-19 vaccines approved by the World Health Organization (WHO).¹⁵

2. To identify gaps in literature surrounding digital, active, participant centered AEFI surveillance systems

3. To identify and describe the core components of active, participant centered, digital surveillance systems being used for post-market AEFI surveillance of WHO approved COVID-19 vaccines, with a focus on the digital solutions and technology being used, the type of AEFI detected, and the populations under surveillance.

For the purpose of this review, “digital” was defined as any tool that used electronic technology for capturing and processing data through digital signals. “Active, participant-centered, AEFI surveillance” was defined as an approach which proactively searched for AEFIs and included purposeful solicitation of health events and/or symptom information specifically from vaccinees following immunization, where clear prompting for and elicitation of data occurred, with cases actively sought out.

Methodological approach

This scoping review followed a detailed and structured approach, informed by PRISMA Extension for Scoping Review (PRISMA-ScR) guidelines to identify, plot, and describe the peer-reviewed literature landscape within the area of active, participant centered, digital AEFI surveillance for WHO approved COVID-19 vaccines.¹⁴

Information sources

Three bibliographic databases (Embase Classic + Embase, OVID-Medline, and EBM Review – Cochrane Central Register of Controlled Trials) were searched for published, peer-reviewed literature ranging from January 1^st, 1946 to December 15^th, 2022. The search strategy was created in collaboration with, and executed by, an experienced medical librarian. A detailed description of the search strategy, including the specific search terms selected and conventions applied, is found in Appendix 1. The final search result records were uploaded to Covidence, where additional deduplication automatically occurred. Screening was conducted by four independent investigators (DS, DZ, MS, and NK). Grey literature was searched for and accessed in order to provide additional contextual information for the identified digital solutions extracted from included records.

Selection of sources of evidence

Pre-determined inclusion and exclusion criteria (Table 1) were first applied to all titles and abstracts by two independent investigators (DZ, DS, NK, and MS) followed by full text screening completed independently in duplicate, with a third-party (BB and KW) resolving decision conflicts. Studies that were included in the scoping review underwent data extraction by one investigator (NK), with a second performing verification (MS).

Table 1.

Applied inclusion and exclusion criteria.

Inclusion Criteria (Included Studies Must Satisfy All the Following to Be Included)
The vaccine under surveillance must be approved by the World Health Organisation (Novavax (NVX-CoV2373), COVOVAX (Novavax formulation), Moderna (mRNA-1273), Pfizer/BioNTech (BNT162b2), Janssen (Ad26.COV2.S), Oxford/AstraZeneca (AZD1222), Covishield, Covaxin, Sinopharm (BBIBP-CorV -Vero Cells), Sinovac (CoronaVac)15
The surveillance method described must include a digital solution/technology, such as an app, e-mail, SMS, website, and/or e-questionnaire (a mix of digital and non-digital was allowed as long as results were separated out, with digital outcomes specifically reported on, such as response rate)
The study describes post-market AEFI surveillance (Phase IV studies, post-market reports, etc.)
The manuscript is primary research
The AEFI surveillance type is “active,” defined as being directly instructed/prompted to respond (solicited AEFI reports) and is patients/participants/vaccinees centered
English language full text, manuscripts are available (matching language proficiencies of investigators)
Study must report the following: Software platform used (e.g., REDCap, Google Forms) Participant Response Rates Type of AEFI detected
Exclusion Criteria (Record Excluded if Any of the Following)
The vaccine under surveillance is not approved by the World Health Organization (mix was allowed if results were separated out with specific outcomes reported for approved vaccines)
The publication is not primary research (narrative review, editorial, letter, comment, opinion piece)
The study does not describe post-market AEFI surveillance
Abstract and/or poster only available which do not provide sufficient detail for interpretation and data extraction
Surveillance reports came from manufacturers, healthcare providers, and/or non-vaccinee/patient/participant sources
Passive surveillance or administrative data studies (mix was allowed if results were separated out with specific outcomes for active surveillance components)
Non-digital solution/technology used for AEFI surveillance, including phone interviews, assisted calling, diary cards, paper forms, etc. (Mix was allowed if results were separated out with specific outcomes for digital AEFI surveillance components)
Full text, in English to meet investigator language proficiency, is not available and/or accessible
The method of surveillance and the components of digital surveillance were unclear, and interpretation of the approach taken was not possible due to an absence of description and details regarding the methods applied.

Data charting process & items

Data was collected from included records and inputted into tables with prespecified categories. Extraction endpoints included reference details (authorship and publication year), study design, surveillance approach details (period of data collection, population(s) under surveillance, technology and digital solutions used for AEFI data collection, and reporting schedule, any formal system name (e.g., V-Safe etc.), the type of COVID-19 vaccine(s) under surveillance, types of adverse events reported (local, systemic, serious, or severe), and management approach(es) for serious events.

Synthesis and presentation of results

Characteristics of included studies (authorship, publication year, country, study design, data collection period, and sample size), characteristics and components of the digital surveillance (population(s) monitored, vaccine(s) covered, response rate(s), participant communication methods, data collection methods, and AEFI surveillance timing) and human resources required to carry out surveillance (human follow-up approaches, operating costs, and associated public health agencies) were summarized in table format.

Results

Selection of sources of evidence and included studies

The applied search strategy, after initial deduplication by the medical librarian using referencing software, identified 3796 records. After additional automatic deduplication by Covidence (n = 1),¹⁶ title and abstract screening excluded 3443 records, from which an additional 296 were subsequently excluded after full-text review. A detailed description of the screening process is presented in a PRISMA flow-chart (Figure 1). 56 studies were included in the present scoping review (Table 2). Two of these publications, one by Zhang et al. (2021) and the other by Zhu et al. (2021), performed their analyses using the same dataset; accordingly, we included 56 papers from 55 unique studies.

Figure 1.

PRISMA chart.

PRISMA Flow-Chart (COVID-19 Vaccine Active, Participant Centered, Digital AEFI Surveillance)

Table 2.

Study characteristics.

Reference Number First Author and Country study conducted	Population Monitored & Data Collection Date	Vaccines Included	Primary Communication Method	Study Design & Data Collection Method	Name of Surveillance System(s) or Software Used	Timing of AEFI Reporting After Vaccination
Alhowaymel et al17– Saudi Arabia	Adults received AstraZeneca in Riyadh -March – May 2021	Oxford-AstraZeneca	Not specified	Cross-sectional-Online survey link	Google Forms	Not specified
Amodio et al18– Italy	Vaccinated persons Palermo University Hospital-January – April 2021	Pfizer-BioNTech	WhatsApp	Cohort-Online survey link	Google Forms	7 days after 1st and 2nd dose
Angkasekwinai et al.19 – Thailand	Healthy Thai HCW’s Siriraj Hospital-February – July 2021	Sinovac and Oxford-AstraZeneca	Not specified	Cohort-Online survey link	Google Forms	7 days after each vaccination
Azzolini et al20 – Italy	4156 HCW’s tertiary care hospital Northern Italy-December 2020 – April 2021	Pfizer-BioNTech	Not specified	Cohort-Online questionnaire	Online questionnaire	Following each dose
Beatty et al21 – United States	18 years + with internet access and smartphone-March 2020 – May 2021	Pfizer-BioNTech	Phone number	Cohort-Mobile app/web-based software	Eureka	Daily, weekly, monthly surveys
Bettinger et al22 – Canada	Vaccinated Canadians from 7 provinces and territories-December 2020 – February 2022	Pfizer, Moderna, Oxford-AstraZeneca	Email	Cohort-Online questionnaire	CANVAS	7 days after each vaccination
Briggs et al23 – USA	Multiple sclerosis patients registered in the iConquer MS research network-March 22, 2021 – June 9, 2021	Pfizer-BioNTech, Moderna, Johnson & Johnson, Oxford-AstraZeneca	iConquer MS web portal	Cross-sectional -Digital invitation to the web-based survey was sent to subjects via the iConquer MS web portal	iConquerMS	Not specified
Bsoul et al24 – USA	University of Texas Health San Antonio dentistry community-January – March 2021	Pfizer-BioNTech	Email	Cross-sectional- Online link	Qualtrics	47 days after
Chalermphanchai et al.25 – Thailand	Participants 18+ years completed 2 doses of 3 week Sinovac in Lampang-August – September 2021	Sinovac	Not specified	Cohort-Online survey	Online side effect monitoring survey	Day 1, 7, 30 after injection
Cuschieri et al.26– Italy	Healthcare workers in the state hospital in Malta, Italy-March 29, 2021 – April 9, 2021	Pfizer-BioNTech	Email	Cross-sectional-Online survey link was sent to subjects via e-mail	Google Forms	Not specified
D’Arminio Monforte et al.27 – Italy	HCW’s two large hospitals Milan, Italy-January – February 2021	Pfizer-BioNTech	Email	Cohort-Online questionnaire	Online questionnaire	Just before second dose, two weeks after second dose
Deng et al28 – Australia	16 years + vaccination sites Australia-February – August 2021	Oxford-AstraZeneca, Pfizer-BioNTech	Email, SMS	Cohort-Link to online survey	AusVaxSafety	0–3 days, 4–7 days after vaccination
Ebinger et al.29 – USA	Healthcare workers at Cedar-Sinai Medical Centre in the USA-Dec 17, 2020 – Feb 10, 2021	Pfizer-BioNTech	Not specified	Cohort-Not specified	REDCap	Participants were prospectively instructed to complete the survey between 8–21 days after each vaccine dose
Figueroa30 – Mexico	HCW’s High Specialty Regional Hospital in Yucatan-January – February 2021	Pfizer-BioNTech	Email	Cohort-Online link	SurveyMonkey	Not specified
Lai et al31 – Hong Kong	Recruited 16+ years receiving 1st dose Sinovac or Pfizer at community vaccination centers Hong Kong-February – July 2021	Sinovac, Pfizer-BioNTech	SMS	Cohort-Online survey	Qualtrics	Up to 14 days after vaccine
Gepner et al32 – Israel	Not found to be COVID positive receive 2nd dose Pfizer January – March 2021	Pfizer-BioNTech	Not specified	Cohort-Mobile application	PerMed	15 days after
Goldlin et al33 – India	Vaccinees tertiary teaching hospital Tamil Nadu-January – February 2021	Oxford-AstraZeneca	Mobile number	Cohort-Mobile phone	Mobile phone	2 weeks after
Guan et al34 – Israel	Israel 2nd or 3rd BNT dose-January – September 2021	Pfizer-BioNTech	Mobile phone	Cohort-Mobile phone questionnaire	Smartwatch/smartphone questionnaire	Up to 14 days after vaccination
Hammad et35 – Egypt	Healthcare workers Zagazig Faculty of Medicine Egypt -June 2021 – March 2022	Oxford-AstraZeneca	Not specified	Cohort-Online survey link	Google Forms	Monitored after each vaccine
Hyun et al36 – Korea	Korean HCW’s Gangnam Severance Hospital-March – August 2021	Oxford-AstraZeneca	Not specified	Cohort-Online survey link	Google Forms	14 days after vaccination
Inoue et al37 – Japan	Medical staff Yamagata University Hospital-March – August 2021	Pfizer-BioNTech	Not specified	Cross-sectional-Online survey link	Google Forms	Not specified
Javed et al38 – Saudi Arabia	HCW’s Maternity and Children Hospital Buraidah-March – April 2021	Oxford-AstraZeneca	WhatsApp, Email	Cohort-Online survey link	Google Forms	15–20 days post vaccination
Jeon et al.39 – South Korea	Healthcare workers aged <65 years at a teaching hospital in South Korea-Not specified	Oxford-AstraZeneca	SMS	Cohort-Daily web-based survey links were sent to subjects via SMS	MVAERS	Twice daily from days 0–7 after vaccination
Kim et al40 – South Korea	Healthcare workers in 3 referral teaching hospitals in South Korea-April, 2021	Pfizer-BioNTech	SMS/e-mail	Cross-sectional-Online survey link was sent to subjects via SMS/e-mail	Google Forms	Range: 5–41 days after vaccination
Lee et al41 – South Korea	HCW’s at Hanyang University Hospital who received 2 doses-March 2021 – May 2021	Oxford-AstraZeneca	Not specified	Cohort-Online survey	Online survey	7 days after each dose
Levy et al42 – Israel	HCW’s vaccinated with Pfizer Sheba Medical Centre-December 2020 – April 2021	Pfizer-BioNTech	SMS	Cohort-Online questionnaire	Text message questionnaire	7 days after each dose
Lim et al43 – Singapore	Healthcare workers at the National University Hospital in Singapore-February 8, 2021 – April 12, 2021	Pfizer-BioNTech	Not specified	Cross-sectional-FormSG web-based survey platform	FormSG	7 days after vaccination
Lotan et al44 – Israel	Multiple sclerosis patients at Rabin Medical Centre, Israel-March 15, 2021 – April 17, 2021	Pfizer-BioNTech	Email	Cross-sectional-Online survey link was sent to subjects via e-mail	REDCap	Not specified
Low et al45 – Singapore	Lactating HCW’s Singapore-February – March 2021	Pfizer-BioNTech	BNT	Cohort-FormSG web-based survey platform	FormsSG	28 days after
Maruyama et al.46 – Japan	HCW’s vaccinated with BNT in Japan-March – July 2021	Pfizer-BioNTech	Not specified	Cohort-Website	Website	8 days after injection
Mofaz et al47 – Israel	Participants who received more than 1 BNT Israel-November 2020 – September 2021`	Pfizer-BioNTech	Not specified	Cohort-Mobile application	PerMed mobile application	Monitored 37 days, 7 days before vaccination
Nachtigall et al48 – Germany	Employees of hospitals of Helios group-May – June 2021	Pfizer-BioNTech, Moderna, Oxford-AstraZeneca	Email	Cross-sectional-Online survey	Online survey	More than 5 days after injection
Nittner-Marszalska et al.49 – Poland	Medical students and professionals at Wroclaw University in Poland -44,229	Pfizer-BioNTech	Email	Cross-sectional-Online survey link was sent to subjects via e-mail	Google Forms	Variable – vaccination dates varied, and survey was distributed on one day only
Okumura et al.50 – Japan	Individuals at Keio University School of Pharmacy-June 2021 – June 2022	Moderna	Email	Cohort-Online survey link	Google Forms	Day 1, 3, 7 after each dose questionnaire administered online
Park et al.51 – South Korea	Hospital staff at a university hospital in Daegu, South Korea-June 2, 2021 – June 18, 2021	Pfizer-BioNTech	SMS	Cross-sectional-Online survey link was sent to subjects via SMS	NAVER Form	3 SMS prompts from June 2–18, 2021
Pellegrino et al.52 – Italy	IBD patients at University of Campania ‘Luigi Vanvitelli’-April 2021 – January 2022	Pfizer-BioNTech	Not specified	Cohort-Online questionnaire	Online questionnaire	9 ± 2 days after vaccination
Presby et al.53 – USA	Individuals wearing WHOOP device Boston MA-44317	Oxford-AstraZeneca, Johnson, Pfizer-BioNTech, Moderna	Biometric device	Cross-sectional-Survey	Wearable biometric device	1 week before and after
Rahmani et al.54 – Italy	Resident physicians at University of Genoa-January 11, 2021 – March 16, 2021	Pfizer-BioNTech	Email	Cross-sectional-Online survey link was sent to subjects via e-mail	LimeSurvey	3 reminder e-mails within 7 days after vaccination
Rolfes et al.55 – Netherlands	People vaccinated in Dutch immunization program-March 2021 – May 2021	Pfizer-BioNTech, Oxford-AstraZeneca, Johnson & Johnson, Moderna	Not specified	Cohort-Online questionnaire	Lareb Intensive Monitoring System	First survey sent 7 days after vaccination, 6 questionnaires sent over period of 6 months
Sadarangani et al.56 – Canada	Pregnant women Canada-December 2020 - November 2021	Pfizer-BioNTech, Moderna, AstraZeneca	Email, telephone number	Cohort-Online survey	REDCap	7 days after vaccination
Sen et al.57 – Various countries	COVAD study multiple countries-April – September 2021	Multiple vaccines	Not specified	Cross-sectional-Online survey	Online survey	7 days after vaccination
Shimamura et al.58– Japan	Healthcare workers in hospital in Japan-March 2021 – January 2022	Pfizer-BioNTech	SMS	Cross-sectional- Online survey link was sent to subjects via SMS	Microsoft Forms	After each vaccine, for two days
Song et al.59 – South Korea	Healthcare workers aged 20–64 years at Inje University Ilsan Paik Hospital in South Korea-March 17, 2021 – March 21, 2021	Oxford-AstraZeneca	SMS	Cross-sectional-Online survey link was sent to subjects via text	Google Forms	Range: 5–9 days after vaccination
Supangat et al.60– Indonesia	Medical students in clerkship programs at Soebandi General Hospital in Indonesia-February, 2021	Sinovac	WhatsApp	Cross-sectional-Online survey link was sent to subjects via WhatsApp after each vaccine dose	Google Forms	7 days after each dose of the vaccine
Tani et al.61 – Japan	HCW’s who received 3 Pfizer doses at Fukuoka City Hospital-March 2021 – January 2022	Pfizer-BioNTech	Not specified	Cohort-Web-based questionnaire	Web-based questionnaire	7 days after
Tawinprai et al.62 – Thailand	Individuals at Chulabhorn Hospital in Bangkok, Thailand 18+ years negative for anti-SARS-CoV2 antibody were eligible-March 31 2021 – May 5 2021	Oxford-AstraZeneca	SMS	Cohort-Online questionnaire through SMS	SMS questionnaire	Day 1 and 7 post-vaccination
Toussia-Cohen et al.63 – Israel	Pregnant who received 2 doses BNT to pregnant women receiving 3 doses-January – November 2021	Pfizer-BioNTech	Not specified	Cohort-Digital questionnaire	Digital questionnaire	2–4 wks after vaccination
Vigezzi et al.64 – Italy	Hospital staff to San Raffaele Hospital-January 4, 2021 – April 27, 2021	Pfizer-BioNTech	Email	Cross-sectional-Online survey link was sent to subjects via e-mail	SurveyMonkey	Not specified
Walmsley et al.65 – Canada	Persons receiving vaccine at Ontario vaccine distribution centers-May – July 2021	Pfizer-BioNTech, Moderna, Oxford-AstraZeneca	Email	Cohort-Electronic questionnaire	Electronic questionnaire	7 days after each dose
Warkentin et al.66 – USA	Individuals at primary care practices or vaccination centers in Bavaria, Germany-April 2021 – August 2021	Pfizer-BioNTech, Moderna, Oxford-AstraZeneca	Email	Cohort-Web-based survey	REDCap	14–19 and 40–59 days after vaccination
Wei et al.67 – China	Staff Guizhou Provincial Staff Hospital-January 2021 – January 2022	Sinopharm, Oxford-AstraZeneca	Mobile phone	Cross-sectional-Mobile phone questionnaire	Mobile phone	3 days after vaccination
Yamazaki et al.68 – Japan	HCW’s Chiba University Hospital Comirnaty vaccinees-March – April 2021	Pfizer-BioNTech	Email/Mobile phone	Cross-sectional-Respon:sum	Respon:sum	14 days after
Yechezkel et al.69 – Israel	Retrospective from Maccabi Health Services and Prospective from PerMed study-December 2021 – July 2022	Pfizer-BioNTech	Mobile phone	Cohort-Mobile phone/smartwatch	Smartwatch/mobile application	42 days after vaccination
Zhang et al.70 – China	Hospital staff in a tertiary hospital in Taizhou, China-February 24, 2021 – March 7, 2021	Sinovac	WeChat/e-mail	Cross-sectional-Online survey link was sent to subjects via WeChat/e-mail	Wen-Juang-Xing platform	Not specified
Zhu et al.71 – China	Hospital staff in a tertiary hospital in Taizhou, China-February 24, 2021 – March 7, 2021	Sinovac	WeChat/e-mail	Cross-sectional-Online survey link was sent to subjects via WeChat/e-mail	Wen-Juang-Xing platform	Not specified

Characteristics of studies

The studies included came from 19 unique countries. 7 of the studies were conducted in Italy, 7 in Israel, 6 in the United States, 6 in South Korea, 6 in Japan, and 3 from Canada.

There were various study designs implemented. As expected, all studies were observational in nature and can be further classified as cross sectional or cohort in nature. It was found that 22 studies were classified as cross-sectional, and 33 studies were classified as cohort.

Populations examined included healthcare workers which accounted for 55.3% of all studies (N = 31/56), the general population at 33.9% (N = 19/56), patients with various illnesses at 5.4% (N = 3/56), and pregnant people at 3.6% (N = 2/56).

Multiple different COVID-19 vaccines were examined in these studies and some studies had multiple vaccines. The most common was the Pfizer-BioNTech (BNT162b2) vaccine, which was found in 39 studies, followed by the Oxford-AstraZeneca (AZD1222) vaccine, which was found in 20 studies, the third most common was the Moderna (mRNA-1273) vaccine which was found in 9 different studies. Other vaccines that were included in these studies were the Sinovac (CoronaVac) vaccine, the Johnson & Johnson (Ad26.COV2.S) vaccine, and the Sinopharm (BBIBP-CorV) vaccine.

Digital AEFI surveillance solutions

There were two broad categories of digital solutions identified. A small percentage of publications (N = 5.6%; N = 3/56) employed specifically designed AEFI digital surveillance systems (either purpose built or adapted from publicly available software) such as CANVAS, CANIM, Voxiva, TeleWatch, or SmartVax. However, most of the papers reported using publicly available software for data capture. The most frequently used software platforms were Google Forms (21.4%; N = 12/56) and REDCap (N = 7.1%; N = 4/56). These two categories overlap as the first set of solutions may leverage publicly available software. A comprehensive list of digital technologies used for surveillance, and their attributes, can be seen in Table 3. Crucially, many studies that were excluded from our review did not provide necessary details concerning their digital surveillance tool (N = 54) to determine what was used.

Table 3.

Response rates.

	Any reaction N(%)	Severe Reaction N(%)	Medically attended	Follow-up	Gender/Sex N(%)	Age	Race/Religion/Cultural Group N(%)	Vaccine type N(%)
Alhowaymel et al.17 - Saudi Arabia	174/222(78.4)	N/A	N/A	N/A	M–165(74.3) F–57(25.7)	N(%) 18–29: 66(29.8) 30–40: 68(30.6) 41–51: 60(27.0) >52: 28(12.6)	Saudi-138(62.2) Non-Saudi-84(37.8)	AZ- 222(100)
Amodio et al.18 - Italy	242/293(82.6)	N/A	N/A	Seven questionnaires sent for a week following first and second dose	F − 134/293(45.7) M − 159/293(54.3)	Median(IQR) 36(29–52)	N/A	Pfizer 293(100)
Angkasekwinai et al.19 - Thailand	CoronaVac v ChAdOx1 1st dose 152/180(84.4) vs. 119/180(66.1) CoronaVac v ChAdOx1 2nd dose 136/180(75.6) vs. 109/180(60.6)	0(0)	N/A	7 days after vaccination	F − 303(84.2) M − 57(15.8)	Median(IQR) 35(29–44)	N/A	CoronaVac 180(50) ChAdOx1 180(50)
Azzolini et al.20 - Italy	1621/4156	N/A	8/2211 events(0.36)	10 days after 2nd dose	M − 1589(38) F − 2567(62)	Median 37 IQR 27–48	N/A	Pfizer 4156(100)
Beatty et al.21 - United States	1 dose of BNT162b2 or mRNA-1273 5629/8680(64.9) 2 doses of BNT162b2 or mRNA-1273 or 1 dose of JNJ-78436735 8947/11140(80.3)	1 dose of BNT162b2 or mRNA-1273 26/8680(0.3) 2 doses of BNT162b2 or mRNA-1273 or 1 dose of JNJ-78436735 27/11140(0.2)	N/A	Monthly surveys from January 14 – May 19, 2021	M − 6024/19586(30.9) F – 13,281/19586(68.1) Transgender- 46/19586(0.23) Genderqueer-110/19586(0.6) Other-60/19586(0.3)	Median (IQR) 54 (38–66)	American Indian or Alaska Native 286(1.5) Asian 1506(7.8) Black 443(2.3) Native Hawaiian/Pacific Islander 87(0.4) White 17294(89.4) Other/Unknown 617(3.2) Hispanic 1476(7.6)	Did not separate out for each
Bettinger et al.22 - Canada	234174/683847(34.2)	2055/683847(0.3)	10092/683847(1.5)	8 days after each of 3 doses sent questionnaire	M − 291144/683847(42.6) F − 391528/683847(57.3) Intersex/Decline − 1175/683847(0.17)	20–29: 72750(10.6) 30–39: 111493(16.3) 40–49: 66067(9.7) 50–64: 138524(20.3) 65–79: 160767(23.5) 80+:23246(3.4)	Black 2947/369351(0.8) Asian 9204/369351(2.5) Indigenous 1909/369351(0.5) Latino 3512/369351(0.9) Arabic 3871/369351(1.0) Indian/Pakistani 6188/369351(1.7) Southeast Asian 3503/369351(0.9) White 215626/369351(58.4) Mixed 5491/369351(1.5) Other/Unknown 112262/369351(30.4) Declined 4838/369351(1.3)	Pfizer 369406(54) Moderna 201314(29.4) AZ 113127(16.5)
Briggs et al.23 - USA	1st dose 459/719(63.8) 2nd dose 327/442(74.0)	1st dose 122(16.9) 2nd dose 99(22.4)	N/A	N/A	1st dose F − 608(84.6) M − 111(15.4) 2nd dose F- 371(83.9) M-71(16.1)	Mean(SD) 1st dose 53.0 (SD 11.8) 2nd dose 53.5 (SD 12.2)	1st dose White − 677(94.2) Non-white − 32(4.4) Unknown − 10(1.4) 2nd dose White − 419(94.6) Non-white − 21(4.8) Unknown − 3(0.7)	1st dose Pfizer-409(56.9) Moderna-258(35.9) Johnson-31(4.3) AZ-20(2.8) Other-1(0.1) 2nd dose Pfizer-269(60.9) Moderna-166(37.6) Johnson-0(0) AZ-6(1.4) Other-1(0.2)
Bsoul et al.24 - USA	296(78)	30(8)	N/A	N/A	F − 241(64) M- 134(35) Prefer not to answer − 4(1)	18–24:56(15)25–34:101(27) 35–44:51(13)45–54:58(15)55+:113(30)	Asian:62(16) Black:10(3)Hispanic:124(33) Other:12(3)White:171(45)	Pfizer 379(100)
Chalermphanchai et al.25 - Thailand	20/42(47.6)	0(0)	N/A	Followed for 30 days	M − 12(28.3) F − 30(71.4)	Mean 48 Range 23–62	N/A	Pfizer 42(100)
Cuschieri et al.26 - Italy	34–1316/1480(2.3–88.9) across several symptoms	2–186/1480(0.1–12.6) across several symptoms	N/A	Not specified	M − 493(33.3) F − 987(66.7)	18–24 144(9.7) 25–34 474(32) 35–44 291(19.7)45–54 328(22.2)55–64 235(15.9)65+ 80(5.4)	N/A	Pfizer 4885(100)
D’Arminio Monforte et al.27 - Italy	First dose 1836/3078(59.6) Second dose 2238/3049(73.4)	0(0)	0(0)	Two weeks after 2nd dose	F − 1980/3078(64.3) M − 1098/3078(35.7)	Median(IQR) 47(34–56)	Italian 2856(92.8) Other 222(7.2)	Pfizer 3078(100)
Deng et al.28 - Australia	Pfizer Dose 1–483 003/1 346 308(35.9) Pfizer Dose 2–521 748/953 704(54.7) AZ Dose 1 - 228 685/433 427(52.8) AZ Dose 2 - 66 726/302 544(22.0)	N/A	Pfizer Dose 1–8699/1 346 308(0.65) Pfizer Dose 2–13 073/953 704(1.4) AZ Dose 1–5260/433 427(1.2) AZ Dose 2–1266/302 544(0.42)	N/A	Pfizer Dose 1 M: 160 764/565 158(28.4) F: 320 712/777 187(41.3) Other: 782/1700 (46) Pfizer Dose 2 M: 181 950/399 392(45.6) F: 338 101/551 535(61.3) Other: 690/1021(68) AZ Dose 1 M: 93 652/199 643(46.9) F: 133 113/230 019(57.9) Other: 199/285(70) AZ Dose 2 M: 23 052/136 689(16.9) F: 43 174/163 831(26.4) Other: 40/92 (44)	Median (IQR) Pfizer Dose 1 42 (33–49) Pfizer Dose 2 44 (37–49) AZ Dose 1 61 (52–68) AZ Dose 2 62 (54–70)	Pfizer Dose 1 Indigenous7443/20 245(36.8) Non-indigenous 467 856/1 303 080(35.9) Pfizer Dose 2 Indigenous 6447/12 228(52.7) Non-indigenous 498 268/910 202(54.7) AZ Dose 1 Indigenous 2230/4 551(49.0) Non-indigenous 219 938/416 314(52.8) AZ Dose 2 Indigenous 625/3019(20.7) Non-indigenous 62 624/284 443(22.0)	Pfizer Dose 1 1 346 308/3 035 983(44.3) Pfizer Dose 2 953 704/3 035 983(31.4) AZ Dose 1 433 427/3 035 983(14.3) AZ Dose 2 66 726/3 035 983(2.2)
Ebinger et al.29 - USA	After dose 1 614/1032(60.0) After dose 2 752/1032(73.6)	N/A	N/A	N/A	F − 691(67.4) M − 341(32.6)	Average Age (SD) 43.3(12.6)	Non-Hispanic Asian 280(27.1) Non-Hispanic Black 33(3.2) Non-Hispanic White 493(47.8) Hispanic/Latinx 126(12.2) Other 100(9.7)	Pfizer 1032(100)
Figueroa et al.30 - Mexico	First dose 68/79(86) Second dose 64/79(81)	First dose 0(0) Second dose 0(0)	N/A	N/A	F − 51(64.6) M- 28(35.4)	Median 42 years (IQR 35–46)	Mexican 79(100)	Pfizer 79(100)
Lai et al.31 - Hong Kong	80/160(50)	N/A	N/A	N/A	F − 90/160(56.25) M − 70/160(43.75)	21–78 Median 40	N/A	Pfizer 160(100)
Gepner et al.32 - Israel	422/1323(31.9)	1/625(0.16)	2/625(0.32) Hospitalization	Followed up at the end of 1st and 2nd week after vaccination	M − 676(51.1) F − 647(48.9)	<30: 239/422(56.6) 30–59: 169/422(39.8) 60+: 14/422(3.6)	N/A	Covishield 1323(100)
Goldlin et al.33 - India	Second vaccine 102/355(30.4) Third vaccine 404/1179(34.2)	Second vaccine 52(15.6) Third vaccine 120(10.2)	N/A	N/A	Second vaccine M − 149 (42.1) F − 206(57.9) Third vaccine M − 512(43.4) F − 667(56.6)	Second vaccine Average 51.8 Third vaccine 50.0	N/A	Second vaccine 355(100) Third vaccine 1179(100)
Guan et al.34 - Israel	212/255(83.1)	0(0)	0(0)	Followed for 6 months after completing vaccine schedule	F − 144(56.5) M − 111(43.5)	Mean(SD) 40.7(11.4)	N/A	AZ 255(100)
Hammad et al.35 - Egypt	First dose 199/232(85.78) Second dose 136/232(58.62)	N/A	N/A	N/A	M − 26(11.21) F − 206(88.79)	Average 39(SD 9.97)	N/A	AZ 232(100)
Hyun et al.36 - Korea	First dose 1450/1586(91.4) Second dose 1194/1306(91.4)	First dose 0(0) Second dose 0(0)	N/A	N/A	First dose M − 522/1586(32.9) F − 1064/1586(67.1) Second dose M − 388/1306(29.7) F − 918/1306(70.3)	First dose 20–29546 (34.4%) 30–39402 (25.3%) 40–49336 (21.2%) 50–59220 (13.9%) 60–82 (5.2%) Second dose 20–29 427 (32.7%) 30–39 321 (24.6%) 40–49 288 (22.1%) 50–59 197 (15.1%) 60- 73 (5.6%)	N/A	Pfizer First dose 1586(100) Second dose 1306(100)
Inoue et al.37 - Japan	1st dose 975/994(98.1) 2nd dose 661/727(90.9)	1/994(0.1)	1st dose 13/994(1.3) 2nd dose 5/727(0.7)	7 days following vaccination	1st dose F − 762(76.7) M − 232(23.4) 2nd dose F − 559(76.9) M − 168(23.1)	1st dose Mean 35.7 Range 19–63 2nd dose Mean 36.7 Range 20–63	N/A	AZ 1384(100)
Javed et al.38 - Saudi Arabia	324/564(57.4)	0(0)	N/A	15–20 days Google Forms after vaccination	M − 210(37.2) F − 354(62.8)	25 and below:108(18.4) 26–35:288(51.1) 36–45:110(19.5) 45+:62(11)	N/A	AZ 564(100)
Jeon et al.39 - South Korea	Pfizer Dose 1 Local reactions: 996/1406(70.8) Systemic reactions: 850/1406(60.5) Pfizer Dose 2 Local reactions: 849/1168(72.7) Systemic reactions: 1010/1168(86.5) AZ Local reactions: 1195/1679(71.2) Systemic reactions: 1501/1679(89.4)	N/A	Pfizer Dose 1 26/1406(1.8) Pfizer Dose 2 38/1168(3.3) AZ 143/1679(8.5)	Not specified	F − 3216/4253(75.6) M − 1037/4253(24.4)	20–29 1448(34) 30–39 1184(27.8) 40–49 903(21.2)50–59 561(13.2)60+ 157(3.7)	N/A	Pfizer 2500/6385(39.2) AZ 3885/6385(60.8)
Kim et al.40 - South Korea	Pfizer 801/969(82.7) CoronaVac 543/1129(48.1)	Adjusted odds ratios (severe allergic reaction) for those who received CoronaVac vs Pfizer Odds ratio (95% confidence interval) 0.62 (0.36–1.06) Adjusted odds ratios (severe allergic reaction) from the second dose compared with the first dose Odds ratio (95% confidence interval) CoronaVac 1.15 (0.62–2.15) Pfizer 2.01 (1.21–3.33), p < .05	N/A	Followed up 2 weeks after 2nd dose	Pfizer M − 498/969(51.4) F − 471/969(48.6) CoronaVac M − 527/1129(46.7) F − 602/1129(53.3)	Pfizer Mean(SD) 43.13(16.54) CoronaVac 46.49(24.42)	N/A	Pfizer 869/1998(43.5) CoronaVac 1129/1998(56.5)
Lee et al.41 - South Korea	434/447(97.1)	206(46.1)	N/A	7 days after 2 injections	F − 388(86.8) M − 59(13.2)	Mean(SD) 40.6(10.9)	N/A	AZ 447(100)
Levy et al.42 - Israel	1st dose 711/831(85.6) 2nd dose 673/738(91.2)	N/A	N/A	Seven days after each dose, received text message	F − 627/831(75.5) M − 204/831(24.5)	Mean 46.5(SD 11.8)	N/A	Pfizer 831(100)
Lim et al.43 - Singapore	Dose 1 3–975/1704(0.2–57.2) Dose 2 13–1195/1704(0.8–70.1)	0(0)	196/1704(11.5)	1 week after both doses	F − 1340(78.6) M − 364(21.4)	Median (Range) 35(18–76)	N/A	Pfizer 6101(100)
Lotan et al.44 - Israel	136/239(56.9)	N/A	8/36(22.2)	Week following vaccination	F- 199/262(75.9) M − 63/262(24.0)	Median(range) 42(22–79)	N/A	Pfizer 425(100)
Low et al.45 - Singapore	57/88(68.4)	0(0)	0(0)	N/A	F − 88(100)	Mean 33.2(SD 3.3)	Chinese77(87.5) Malay 6 (6.8) Indian 2 (2.3) Others 3 (3.4)	Pfizer 88(100)
Maruyama et al.46 - Japan	348/374(93.0)	N/A	N/A	8 days after vaccination for both doses	F − 225(60.2) M − 149(39.8)	Mean(SD) 42.44(12.71)	N/A	Pfizer 374(100)
Mofaz et al.47 - Israel	1st dose 217/1609(13.5) 2nd dose 586/1609(36.4) 3rd dose 637/1609(39.6)	N/A	N/A	N/A	F − 854(53.08) M − 755(46.92)	Median 52 Range 18–88	N/A	Pfizer 1609(100)
Nachtigall et al.48 - Germany	12084/16207(74.6)	9/16207(0.05)	N/A	Not specified	F − 6131/8269(74.1) M − 2138/8269(25.9)	18–30: 1096/8269(13.3) 31–40: 1817/8269(30.0) 41–50: 2044/8269(24.7) 51–60: 2515/8269(30.4) >61:627/8269(7.6 Invalid: 170/8269(2.1)	N/A	Pfizer-Pfizer 4179/8246(50.7) Moderna-Moderna 207/8246(2.5) AZ-AZ 748/8246(9.1) AZ-Pfizer 1465/8246(17.8) AZ-Moderna 284/8246(3.4) Missing information 1363/8246(16.5)
Nittner-Marszalska et al.49 - Poland	1st dose 1571/1707(92.03) 2nd dose 1587/1707(92.97)	1st dose 0(0) 2nd dose 0(0)	1st dose Pharmacological intervention 128(7.5) Medical consultation 6(0.35) 2nd dose Pharmacological intervention 567(33.2) Medical consultation 64(3.7)	Not specified	M − 356(20.85) F- 1351(79.15)	20–29: 585(34.27) 30–39: 554(32.45) 40–49: 264(15.47) 50–59: 160(9.37)60+: 144(8.44)	N/A	Pfizer 1707(100)
Okumura et al.50 - Japan	252/301(83.7)	0(0)	First dose Day 0: 1/301(0.3) Day 1: 0 (0) Day 3: 1/196(0.5) Day 7: 0(0) Second dose Day 0: 1/179(0.6) Day 1: 0(0) Day 3: 1/138(0.7) Day 7: 0(0) Third dose Day 0: 0(0) Day 1: 1/38(2.6) Day 3: 0(0) Day 7: 0(0)	1 week follow-up after each dose	M − 128/301(42.5) F- 172/301(57.1)	18–29 238/301(79.1) 30–69 63/301(20.9)	N/A	Moderna 301(100)
Park et al.51 - South Korea	AZ 1st dose 4–205/299(1.3–68.6) across several symptoms 2nd dose 1–174/304(0.3–57.2)across several symptoms Pfizer 1st dose 0–13/19(0–68.4) across several symptoms 2nd dose 0–15/22(0–68.2) across several symptoms	Severe interference with work 161/603(26.7) Severe interference with daily life 195/644(30.3)	AZ 1st dose 28/299(9,4) 2nd dose 3/299(0.1) Pfizer 1st dose 1/19(5.3) 2nd dose 1/22(4.5)	N/A	AZ 1st dose M-80(26.8) F-219(73.2) 2nd dose M-80(26.3) F-224(73.7) Pfizer 1st dose M-2(10.5) F-17(89.5) 2nd dose M-2(9.1) F-20(90.9)	AZ 1st dose 20–29:96(32.1) 30–39:56(18.7) 40–49:54(18.1) 50–59:81(27.1) 60+:12(4.0) AZ 2nd dose 20–29: 87(28.6) 30–39: 56(18.4) 40–49: 66(21.7) 50–59: 83(27.3) 60+:12(3.9) Pfizer 1st dose 20–29:11(57.9) 30–39:4(21.1) 40–49:1(5.3) 50–59: 3(15.8) 60+:0(0) Pfizer 2nd dose 20–29:10(45.5) 30–39:7(31.8) 40–49: 3(13.6) 50–59:2(9.1) 60+: 0(0)	N/A	AZ 368/395(93.2) Pfizer 27/395(6.8)
Pellegrino et al.52 - Italy	after 1st, 2nd, 3rd dose Local 26.25% (21/80), 58.75% (47/80), and 28.37% (21/74) Systemic 52.2% (42/80), 48.75% (39/80), and 43.24% (32/74)	0(0)	N/A	N/A	M − 42/80(52.5) F − 36/80(37.5)	Median 47.5	N/A	Pfizer 80(100)
Presby et al.53 - USA	AZ First Dose 2915/3547(84.3) AZ Second Dose 191/325(58.7) Johnson 3751/4584(81.8) Moderna First Dose 10763/17632(61.0) Moderna Second Dose 14963/16987(88.1) Pfizer First Dose 14825/29366(50.5) Pfizer Second Dose 19854/27084(73.3)	N/A	N/A	N/A	M − 65334/99435(65.7) F − 34101/99435(34.3)	18–29 28107(28.3) 30–39 38928(39.1) 40–54 26164(26.3) 55+ 6236(6.27)	N/A	AZ 3782(3.8) Johnson 4584(4.6) Moderna 34619(34.8) Pfizer 56450(56.8)
Rahmani et al.54 - Italy	Dose 1 2–285/296(0.7–96.3) across several local and systemic reactions Dose 2 6–257/275(2.2–93.5) across several local and systemic reactions	Dose 1 0–6/296(0–2.0) across several local and systemic reactions Dose 2 0–17/275(0–6.2) across several local and systemic reactions	Dose 1 0(0) Dose 2 0(0)	7 days after both doses	F − 272(53.2) M − 240(0.47)	Mean(SD) 28.9(2.7)	N/A	Pfizer 512(100)
Rolfes et al.55 - Netherlands	13959/22184(62.9)	N/A	N/A	6 months after vaccination	M − 3199/13959(22.9) F − 10760/13959(77.1)	0–50 6419(46) 51–60 3090(22.1) 61–79 3539(25.3) 80+ 911(6.5)	N/A	Pfizer 10724/22590(47.5) AZ 8778/22590(38.9) Johnson 1508/22590(6.7) Spikevax 1508/22590(6.7) Unknown 72/2590(2.8)
Sadarangani et al.56 - Canada	Dose 1 Not pregnant 10950/174765(6.3) Pregnant 226/5597(4.0) Dose 2 Not pregnant 10254/91131(11.3) Pregnant 227/3108(7.3)	Dose 1 Not pregnant 733/174765(0.4) Pregnant 31/5597(0.6) Dose 2 Not pregnant 343/91131(0.4) Pregnant 19/3108(0.6)	Y for severe reactions	N/A	Dose 1 Not pregnant Woman:170674(97.7) Man:819(0.5) Non-binary:2380(1.4) Two-spirit:148(0.1) Other: 139(0.1) Unknown:605(0.3) Pregnant Woman: 5579(99.7) Man: 3(0.1) Non-binary:12(0.2) Two-spirit:0(0) Other:1(<0.1) Unknown:2(<0.1) Dose 2 Not pregnant Woman:89176(97.9) Man:341(0.4) Non-binary:1254(1.4) Two-spirit:49(0.1) Other:69(0.1) Unknown:242(0.3) Pregnant Woman:3091(99.5) Man:1(<0.1) Non-binary:13(0.4) Two-spirit:0(0) Other:0(0) Unknown: 3(0.1)	Dose 1 Not pregnant 15–29: 59263(33.9) 30–49: 115502(66.1) Pregnant 15–29:1417(25.3) 30–49:4180(74.7) Dose 2 Not pregnant 15–29: 26324(28.9) 30–49:64807(71.1) Pregnant 15–29:716(23.0) 30–49:2392(77.0)	Dose 1 Not pregnant White: 47539(27.2) Black:1155(0.7) East Asian:3367(1.9) South Asian:1977(1.1) Southeast Asian:1552(0.9) Indigenous:699(0.4) Middle Eastern:1352(0.8) Latino:1392(0.8) Mixed:2714(1.6) Unknown or Other:113018(64.7) Pregnant White: 1818(32.5) Black:18(0.3) East Asian:117(2.1) South Asian:95(1.7) Southeast Asian:47(0.8) Indigenous:22(0.4) Middle Eastern:44(0.8) Latino:48(0.9) Mixed :92(1.6) Unknown or Other:3296(58.9) Dose 2 Not pregnant White:50779(55.7) Black:1169(1.3) East Asian:3471(3.8) South Asian:2041(2.2) Southeast Asian:1587(1.7) Indigenous:703(0.8) Middle Eastern:1361(1.5) Latino: 1460(1.6) Mixed:2800(3.1) Unknown or Other:25758(28.3) Pregnant White:1778(57.2) Black:22(0.7) East Asian:106(3.4) South Asian:104(3.3) Southeast Asian:45(1.4) Indigenous:21(0.7) Middle Eastern:44(1.4) Latino:50(1.6) Mixed:88(2.8) Unknown or Other:850(27.3)	Dose 1 Not pregnant Pfizer 107121/180362(59.4) Moderna 67644/180362(37.5) Pregnant Pfizer 3414/180362(1.9) Moderna 2183/180362(1.2) Dose 2 Not pregnant Pfizer 53077/94239(56.3) Moderna 38054/94239(40.4) Pregnant Pfizer 1892/94239(2.0) Moderna 1216/94239(1.3)
Sen et al.57 - Various countries	8573/10900(79)	351/10900(3)	38/10900(0.3)	N/A	M − 2834/10900(26) F − 8066/10900(74)	Median(IQR) 42(30–55)	Caucasian 4972(45) African American 83(0.7) Asian 2018(18) Hispanic 1193(11) Native American/Indigenous/Pacific Islander 342(3) Do not wish to disclose 449(4) Other 865(8) Unanswered 1672(15)	Pfizer 4339(39) AZ 1456(13) Johnson 95(1) Moderna 910(8) Novavax 14(0.1) Covishield 1194(11) Covaxin 248(2) Sputnik 204(2) Sinopharm 1821(17) Not sure 62(0.5) Others 563(5)
Shimamura et al.58 - Japan	40–1910/1990(2.0–96.0) across several local and systemic reactions	4/1990(0.2)	N/A	2 days following each of 3 doses	M − 418(21) F − 1572(79)	Median 32	N/A	Pfizer 1990(100)
Song et al.59 – South Korea	809/998(81.1)	0(0)	N/A	N/A	F − 779(78.1) M − 219(21.9)	20–29: 380(38.1 30–39: 216(21.6) 40–49: 185(18.5) 50–59: 180(0.18) 60–64: 37(3.7)	N/A	AZ 998(100)
Supangat et al.60 - Indonesia	68/144(47.2)	N/A	N/A	Followed for 1 week after both doses	M − 38/144(26.4) F − 106/144(73.6)	Average age range 21–25	N/A	CoronaVac 144(100)
Tani et al.61 - Japan	278/281(98.9)	N/A	N/A	Data collected until 7 days after booster dose	F − 204/281(72.6) M − 77/281(27.4)	Median(IQR) 41(33–50)	N/A	Pfizer 281(100)
Tawinprai et al.62 - Thailand	322/538(59.9)	0–41(7.62) across various local and systemic reactions	N/A	Data collected until 7 days after vaccination	F − 517/794(65.1)	Median(IQR) 40(30–57)	N/A	AZ 794(100)
Toussia-Cohen et al.63 - Israel	Second vaccination 73/78(93.6) Third vaccination 61/84(72.6)	Second vaccination 0(0) Third vaccination 0(0)	Second vaccination 0(0) Third vaccination 0(0)	N/A	Second vaccinationF-78(100) Third vaccinationF-84(100)	Second vaccination Mean 32.85(SD3.49) Third vaccination 33.23(3.95)	N/A	Second vaccination Pfizer-78(100) Third vaccination Pfizer-84(100)
Vigezzi et al.64 - Italy	Male 376/720 (52.2) Female 1,286/1,939(66.3)	285/2,659 (10.7)	N/A	N/A	F − 2600/5668(45.9) M − 3068/5668(54.1)	Median 42 Range 19–76	N/A	Pfizer 5668(100)
Walmsley et al.65 - Canada	First dose 26/37(0.70) Second dose 906/955(94.9)	N/A	N/A	38(4%) reported persistent adverse events thought related to the vaccine at month 1, decreasing to 10(1%) at month five.	F or non-binary 760/1193(63.7) M 433/1193(36.3)	30–50 41[36, 45] 70+ 73[71, 76]	Arab/West Indian 10/1193(0.8) Black 20/1193(1.7) Indigenous 5/1193(0.4) Latin American 7/1193(0.6) South Asian 15/1193(1.3) Southeast Asian 32/1193(2.7) White 1045/1193(87.6) Other 52/1193(4.4)	2 doses Pfizer 733(61.4) 2 doses Moderna 131(11.0) 1 dose Pfizer/1 dose Moderna 201(16.8) 1 dose AZ/1 dose Pfizer or Moderna 69(5.8) Other or unknown 36(3.0)
Warkentin et al.66 - USA	ChAdOx1/ChAdOx1 475/552(86) ChAdOx1/mRNA 1382/2383(58) mRNA/mRNA 4721/6212(0.76)	N/A	ChAdOx1/ChAdOx1 69/462(14.9) ChAdOx1/mRNA 287/1638(17.5) mRNA/mRNA 796/5004(15.9)	Followed up to 56 days after vaccination	F − 4661/8145(57.2) M − 3483/8145(42.8) Diverse - 1/8145(0.01)	ChAdOx1/ChAdOx1 Mean(SD) 55.87(15.3) ChAdOx1/mRNA 47.6(13.89) mRNA/mRNA 45.87(15.14)	N/A	ChAdOx1/ChAdOx1 487/8145(6.0) ChAdOx1/mRNA 1943/8145(23.9) mRNA/mRNA 5715/8145(70.2)
Wei et al.67 - China	Homologous Vero Cell Booster Group 62/635(9.8) Homologous CHO Cell Booster Group 13/75(17.3) Heterologous Mixed Vaccines Booster Group 17/82(20.7)	N/A	N/A	N/A	F − 597/792 (75.4) M − 195/792(24.6)	Ages 18–60	N/A	Homologous Vero Cell Booster Group 635/792(80.1) Homologous CHO Cell Booster Group 75/792(9.5) Heterologous Mixed Vaccines Booster Group 82/792(10.4)
Yamazaki et al.68 - Japan	1st dose 2134/2406(88.7) 2nd dose 2168/2347(92.4)	1st dose 27(1.1) 2nd dose 154(6.6)	1st dose 9(0.4) 2nd dose 13(0.6)	N/A	1st dose M − 921(38.3) F − 1485(61.7) 2nd dose M-898(38.3) F-1449(61.7)	1st dose 20–29:657(27.3) 30–39:772(32.1) 40–49:551(22.9) 50–59:335(13.9) 60+:91(3.8) 2nd dose 20–29: 627(26.7)30–39: 750(32.0)40–49: 541(23.1)50–59: 339(14.4)60+: 90(3.8)	N/A	Pfizer 1st dose 2406(100) 2nd dose 2347(100)
Yechezkel et al.69 - Israel	Retrospective Cohort 2–203(<1–1.1)	Retrospective Cohort 0–170 (0–1)	N/A	N/A	Prospective First booster M: 866/1785(48.5) F: 919/1785(51.5) Unspecified: 0/1785(0) Second booster M: 348/699(48.8) F: 350/699(50.2) Unspecified: 1/699(<1) First and Second Booster M: 215/446(48.2) F: 231/446(51.8) Unspecified: 0/446(0) Retrospective First booster M: 45208/94169(48.0) F: 48961/94169(52.0) Unspecified: 0/94169(0) First and Second Booster M: 8879/17814(49.8) F: 8935/1781(50.2) Unspecified: 0/1781(0)	Prospective Median(IQR) First booster 52(34–61) Second booster 62(53–68) First and second booster 64(57–70) Retrospecitve First booster 47(31–61) First and second booster 69(62–76)	Prospective First booster Jewish 1678(94) Arab 18(1.0) Unspecified89(5.0) Second Booster Jewish 672(96.1) Arab 1(<1) Unspecified26(3.7) First and second booster Jewish 434(97.3) Arab 0(0) Unspecified12(2.7) Retrospective First booster Jewish 90106(95.7) Arab 4046(4.3) Unspecified17(<1) First and second booster Jewish 17513(98.3) Arab 300(1.7) Unspecified1(<1)	Pfizer 254698(100)
Zhang et al.70 - China	Female 0–124/1107(0–11.2) across several systemic and local reactions Male 0–25/290(0–8.6) across several systemic and local reactions	N/A	N/A	1 week following 2 doses	F − 1107/1397(79.2) M − 290/1397(20.8)	F – Mean (SD) 34.7(8.6) M – Mean (SD) 38.7(9.9)	N/A	CoronaVac 3013(100)
Zhu et al.71 - China	Female 0–124/1107(0–11.2) across several systemic and local reactions Male 0–25/290(0–8.6) across several systemic and local reactions	N/A	N/A	1 week following 2 doses	F − 1107/1397(79.2) M − 290/1397(20.8)	F – Mean (SD) 34.7(8.6) M – Mean (SD) 38.7(9.9)	N/A	CoronaVac 3013(100)

A variety of communication mediums were used when reaching out to individuals within the various studies. Email was the most frequent (26.8%; N = 15/56), followed by SMS (23.2%; N = 13/56), cell-phone app notification (7.1%; N = 4/56), web-portal notification (1.7%; N = 1/56, or a combination of methods (8.9%; N = 3/56). 17 studies did not clearly specify how participants were communicated with (27.1%; N = 17/56), although it appears that prospective instructions were given to participants in person in some instances.

Response rates

There was a wide range of the response rates in the studies reflecting the heterogeneity of the technologies and study designs (See Table 3. for more information). In some instances, there is a higher response rate reported for specific genders or age groups. For example, in Vigezzi et al, females had a higher response rate (66.3%, N = 1286/1939) (p < .01) compared to males (52.2%, N = 376/720) (p < .01).

Discussion

This scoping review provides an overview of published research on the digital technologies used for active, participant-centered AEFI surveillance of COVID-19 vaccines approved by the World Health Organization (WHO) during the early stages of the pandemic. Our review provides a sample of the breadth of programs that were utilized during the pandemic. We observed a diversity of programs, with some appearing to be more specifically built for AEFI surveillance and others identifying existing software that could facilitate this function. There was a diversity in data collected among programs, methods of communicating with participants and participant response rates. We limited the search to this time period as we wanted to examine AEFI reporting in the context of the COVID-19 pandemic which was a highly unusual event and atypical situation for standard AEFI reporting. Future studies could expand this review to examine AEFI reporting of COVID-19 vaccines beyond the initial pandemic release of vaccines.

We also noted variability on the level of detail reported on these systems, particularly with respect to evaluation criteria and a substantial difference in response rates. Future research would benefit from further exploration of the best strategies to ensure optimal reporting of AEFI’s. Ultimately, however, surveillance systems need to be custom built for the local environment in which they will be implemented. Federal jurisdictions face challenges with respect to the collection of public health data from regional governments which unitary states do not.⁷² There is also a diversity of challenges for high income countries versus low- and middle-income countries.⁷³ In many low- and middle-income countries (LIMCs), there is a lack of a formal vaccine safety monitoring system. Vaccines are often used without extensive post-licensure experience.⁷⁴ For example, vaccines that target novel threats such as Lassa and Nipah viruses are employed in such environments. In response, sentinel sites, which are designed healthcare facilities, are provided the tools and resources to collect data from individuals who experience an adverse event post-vaccination.⁷⁴ This approach has been successful in Mali and Niger when evaluating a new meningococcal vaccine.

The difference between females and males regarding AEFI response rate is still to be understood fully. This could be due to selection bias or behavior in terms of who response to online surveys or biological differences that may influence AEFI occurrence.⁷⁵ The COVID-19 pandemic and subsequent vaccine roll-out demonstrated the need for AEFI surveillance systems and the value of digital technologies in supporting these systems. The rapid roll-out of a multitude of new vaccines, some using novel platforms, required post-market surveillance systems to ensure both the safety and effectiveness of these vaccines. COVID vaccines approve for use on an emergency basis further emphasized the need for robust post-market surveillance. AEFI surveillance systems were critical as they identified the risk of vaccine-induced immune thrombotic thrombocytopenia (VITT) with the ChAdOx1 CoV-19 vaccine and the risk of myocarditis from mRNA vaccines, quantified these risks and guided vaccine recommendations.^76,77

This review can guide public health AEFI surveillance. Robust AEFI surveillance systems need to be in place in anticipation of future pandemic vaccines as well as to enhance monitoring of existing vaccine programs and the roll-out of novel vaccines.⁷⁴ Standardization of AEFI surveillance and reporting of these systems is a priority of the WHO.⁷⁸ For example, we observed that many studies found within this review would have benefitted from having a comparison group to serve as a control. Having a comparison group that is representative of the vaccinated population would allow the studies examined to have a more accurate assessment of AEFI risks and benefits.

The international community should prioritize the adoption of standardized definitions for events, using established frameworks such as those provided by the Brighton Collaboration.⁷⁹ To ensure global consistency and facilitate seamless integration across digital systems, it is imperative to implement a WHO standard. This involves the development of an Adverse Events Following Immunization (AEFI) reporting framework that incorporates standardized forms or templates for comprehensive data collection, covering essential information such as patient demographics, vaccination details, and a detailed description of AEFI.^79,80

In collaboration with the World Health Organization (WHO), the international community could further enhance this framework by developing a recognized system for coding AEFI events, akin to established medical coding systems like the International Classification of Diseases (ICD) or the Diagnostic and Statistical Manual of Mental Disorders (DSM).⁷⁸ This holistic approach, combining standardized definitions, digital system integration, and a universally accepted coding system, would significantly contribute to the global effort in ensuring the safety of vaccines and streamlining the reporting and analysis of vaccine safety data.

AEFI data extracted through digital surveillance technologies

The studies included in this scoping review clearly defined the type of AEFI being detected in their respective participant populations. All studies reported local and systemic events (N = 56), although there was less consistency with respect to defining and reporting severe events, serious adverse events (SAEs) and medically attended adverse events (MAEs). Twenty-two studies report medically attended events and seven of the twenty-two studies report that participants experienced ‘severe’ events, although the term is not defined. Further information is provided in Table 3. including follow-up protocols for SAE surveillance where applicable.

Limitations

The protocol that was generated internally and used to conduct this scoping review was not registered. This scoping review did not conduct an environmental scan, thus it only included peer reviewed published articles and did not search the gray literature which encompasses non-published materials, such as newspaper articles, policy documents, conference abstracts, reports and any other forms of unpublished research. Due to investigator language proficiency, only records that were available in English could be included, which presents an issue due to the global scope of our study. Further, this scoping review only included articles up to December 31^st, 2022, therefore, limiting our study inclusion and analyses to approximately the first two waves of the COVID-19 pandemic which encompassed the vaccine rollout of the primary vaccine and a 2^nd booster in Canada. Our intent, however, was to examine AEFI systems for the release of the emerging vaccines during the pandemic period which would largely have occurred by this time period. AEFI reporting during the post-pandemic phase of COVID-19 would be similar to other AEFI reporting which we have previously reported on.⁸¹ Due to the rapidly evolving nature of the pandemic, newly emerging, COVID-19 vaccines, and changing landscape of active, participant-centered AEFI surveillance systems in response to these innovations, future studies should incorporate longer-term follow-up and continued evaluation of these surveillance systems as the pandemic progresses.

Conclusion and future directions

The scoping review has explored the different approaches and digital solutions for AEFI surveillance during the early stages of the COVID-19 pandemic. The rapid creation or repurposing of AEFI surveillance systems was a major challenge for public health systems during the pandemic. Learnings from each other experience can allow these systems to be better prepared for future pandemics as well as further augment their existing AEFI surveillance systems.

Appendix 1. Search Strategies

Embase Classic+Embase <1947 to 2022 December 15>

Ovid MEDLINE(R) ALL < 1946 to December 15, 2022>

EBM Reviews - Cochrane Central Register of Controlled Trials <November 2022>

1. COVID-19 Vaccines/36188

2. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw,kf.63512

3. ((mRNA or messenger RNA) adj3 vaccin*).tw,kf.14674

4. (BNT162b2 or BNT 162b2).tw,kf.8949

5. pfizer vaccin×.tw,kf.660

6. moderna vaccin×.tw,kf.773

7. astra zeneca vaccin×.tw,kf.46

8. (AZD1222 or azd 1222).tw,kf.1310

9. (mRNA-1273 or mRNA1273).tw,kf.3897

10. johnson vaccin×.tw,kf.154

11. Vaxzevria.tw,kf.638

12. astrazenica.tw,kf.70

13. Covishield.tw,kf.680

14. Spikevax.tw,kf.510

15. BNT162b1.tw,kf.63

16. ChAdOx1-S.tw,kf.367

17. or/1-16 76,261

18. (adverse event* or side effect*).tw,kf.1493402

19. Adverse Drug Reaction×.tw,kf.56363

20. exp “Drug-Related Side Effects and Adverse Reactions”/752180

21. ((local or systemic) adj2 reaction*).tw,kf.36448

22. reactogenicity.tw,kf. or ae.fs. or aefi.tw,kf.3505587

23. risk/or risk factors/or patient safety/or “drug-related side effects and adverse reactions”/2873320

24. or/18-23 7,147,377

25. 17 and 2419156

26. Vaccines, Synthetic/ae and COVID-19/95

27. 25 or 26 19,156

28. product surveillance, postmarketing/or pharmacovigilance/27593

29. Adverse Drug Reaction Reporting Systems/13099

30. (pharmacovigilance or monitor* or drug evaluation*).tw,kf.2412654

31. Adverse Drug Reaction Reporting Systems/13099

32. Drug Evaluation/247384

33. surveillance.mp.650989

34. Self Report/190401

35. ((self or patient) adj2 report*).tw,kf.728881

36. survey×.mp.3025822

37. questionnaire×.mp.2282175

38. or/28–37 7,685,377

39. or/28–38 7,685,377

40. 27 and 395422

41. 40 use medall1920

42. limit 41 to dt = 20211209–202212161196

43. exp SARS-CoV-2 vaccine/46143

44. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw.62869

45. ((mRNA or messenger RNA) adj3 vaccin*).tw.14094

46. (BNT162b2 or BNT 162b2).tw.8752

47. pfizer vaccin×.tw.627

48. moderna vaccin×.tw.737

49. astra zeneca vaccin×.tw.44

50. (AZD1222 or azd 1222).tw.1271

51. (mRNA-1273 or mRNA1273).tw.3784

52. johnson vaccin×.tw.147

53. Vaxzevria.tw.615

54. astrazenica.tw.70

55. Covishield.tw.640

56. Spikevax.tw.471

57. BNT162b1.tw.61

58. ChAdOx1-S.tw.359

59. or/43–58 77,013

60. vaccination reaction/or exp adverse drug reaction/752180

61. (adverse event* or side effect*).tw.1475630

62. AEFI.tw.1385

63. ((local or systemic) adj2 reaction*).tw.36238

64. reactogenicity.tw.7637

65. or/60–64 2,100,903

66. 59 and 65 11,633

67. exp SARS-CoV-2 vaccine/ae7833

68. exp SARS-CoV-2 vaccine/and (risk/or risk factor/or patient safety/)2331

69. 66 or 67 or 68 17,452

70. postmarketing surveillance/or drug surveillance program/or active surveillance/46398

71. pharmacovigilance/7477

72. (surveillance or pharmacovigilance or monitor* or drug evaluation*).tw.2844349

73. drug screening/254360

74. self report/190401

75. ((self or patient) adj2 report*).tw.720219

76. (survey* or questionnaire*).mp.4344459

77. or/70–76 7,600,425

78. 69 and 774992

79. 78 use emczd3113

80. limit 79 to dc = 20211209–202212162439

81. COVID-19 Vaccines/36188

82. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw,kw.68854

83. ((mRNA or messenger RNA) adj3 vaccin*).tw,kw.14268

84. (BNT162b2 or BNT 162b2).tw,kw.8874

85. pfizer vaccin×.tw,kw.655

86. moderna vaccin×.tw,kw.771

87. astra zeneca vaccin×.tw,kw.46

88. (AZD1222 or azd 1222).tw,kw.1306

89. (mRNA-1273 or mRNA1273).tw,kw.3866

90. johnson vaccin×.tw,kw.147

91. Vaxzevria.tw,kw.634

92. astrazenica.tw,kw.70

93. Covishield.tw,kw.673

94. Spikevax.tw,kw.508

95. BNT162b1.tw,kw.63

96. ChAdOx1-S.tw,kw.367

97. or/81–96 80,358

98. (adverse event* or side effect*).tw,kw.1515106

99. Adverse Drug Reaction×.tw,kw.82184

100. 100exp “Drug-Related Side Effects and Adverse Reactions”/752180

101. 101((local or systemic) adj2 reaction*).tw,kw.36275

102. 102reactogenicity.tw,kw. or ae.fs. or aefi.tw,kw.3505562

103. 103risk/or risk factors/or patient safety/or “drug-related side effects and adverse reactions”/2873320

104. 104or/98–103 7,175,226

105. 10597 and 104 19,496

106. 106Vaccines, Synthetic/ae and COVID-19/95

107. 107105 or 106 19,496

108. 108product surveillance, postmarketing/or pharmacovigilance/27593

109. 109Adverse Drug Reaction Reporting Systems/13099

110. 110(pharmacovigilance or monitor* or drug evaluation*).tw,kw.2396986

111. 111Adverse Drug Reaction Reporting Systems/13099

112. 112Drug Evaluation/247384

113. 113surveillance.mp.650989

114. 114Self Report/190401

115. 115((self or patient) adj2 report*).tw,kw.722922

116. 116survey×.mp.3025822

117. 117questionnaire×.mp.2282175

118. 118or/108–117 7,670,074

119. 119107 and 1185458

120. 120119 use cctr141

121. 121limit 120 to yr=“2022”66

122. 12242 or 80 or 1213701

123. 123remove duplicates from 1222717

Ovid MEDLINE(R) ALL <1946 to December 15, 2022>

1. COVID-19 Vaccines/16941

2. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw,kf.29013

3. ((mRNA or messenger RNA) adj3 vaccin*).tw,kf.6420

4. (BNT162b2 or BNT 162b2).tw,kf.3648

5. pfizer vaccin×.tw,kf.234

6. moderna vaccin×.tw,kf.314

7. astra zeneca vaccin×.tw,kf.14

8. (AZD1222 or azd 1222).tw,kf.369

9. (mRNA-1273 or mRNA1273).tw,kf.1358

10. johnson vaccin×.tw,kf.59

11. Vaxzevria.tw,kf.177

12. astrazenica.tw,kf.3

13. Covishield.tw,kf.214

14. Spikevax.tw,kf.130

15. BNT162b1.tw,kf.18

16. ChAdOx1-S.tw,kf.139

17. or/1-16 33,413

18. (adverse event* or side effect*).tw,kf.493062

19. Adverse Drug Reaction×.tw,kf.19521

20. exp “Drug-Related Side Effects and Adverse Reactions”/129577

21. ((local or systemic) adj2 reaction*).tw,kf.12741

22. reactogenicity.tw,kf. or ae.fs. or aefi.tw,kf.1954820

23. risk/or risk factors/or patient safety/or “drug-related side effects and adverse reactions”/1117659

24. or/18-23 3,175,642

25. 17 and 247348

26. Vaccines, Synthetic/ae and COVID-19/94

27. 25 or 267348

28. product surveillance, postmarketing/or pharmacovigilance/10588

29. Adverse Drug Reaction Reporting Systems/8665

30. (pharmacovigilance or monitor* or drug evaluation*).tw,kf.966031

31. Adverse Drug Reaction Reporting Systems/8665

32. Drug Evaluation/42048

33. surveillance.mp.277616

34. Self Report/41786

35. ((self or patient) adj2 report*).tw,kf.279327

36. survey×.mp.1217430

37. questionnaire×.mp.913528

38. or/28–37 2,803,321

39. or/28–38 2,803,321

40. 27 and 391920

Embase Classic+Embase <1947 to 2022 December 15>

1. exp SARS-CoV-2 vaccine/27938

2. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw.32738

3. ((mRNA or messenger RNA) adj3 vaccin*).tw.7628

4. (BNT162b2 or BNT 162b2).tw.5040

5. pfizer vaccin×.tw.399

6. moderna vaccin×.tw.434

7. astra zeneca vaccin×.tw.30

8. (AZD1222 or azd 1222).tw.867

9. (mRNA-1273 or mRNA1273).tw.2376

10. johnson vaccin×.tw.91

11. Vaxzevria.tw.431

12. astrazenica.tw.53

13. Covishield.tw.424

14. Spikevax.tw.354

15. BNT162b1.tw.38

16. ChAdOx1-S.tw.204

17. or/1-16 42,179

18. vaccination reaction/or exp adverse drug reaction/618761

19. (adverse event* or side effect*).tw.792306

20. AEFI.tw.776

21. ((local or systemic) adj2 reaction*).tw.20492

22. reactogenicity.tw.2959

23. or/18-22 1,286,882

24. 17 and 237196

25. exp SARS-CoV-2 vaccine/ae4332

26. exp SARS-CoV-2 vaccine/and (risk/or risk factor/or patient safety/)1940

27. 24 or 25 or 26 10,427

28. postmarketing surveillance/or drug surveillance program/or active surveillance/41029

29. pharmacovigilance/4339

30. (surveillance or pharmacovigilance or monitor* or drug evaluation*).tw.1590371

31. drug screening/199586

32. self report/145953

33. ((self or patient) adj2 report*).tw.390578

34. (survey* or questionnaire*).mp.2650343

35. or/28–34 4,521,044

36. 27 and 353113

EBM Reviews – Cochrane Central Register of Controlled Trials <November 2022>

1. COVID-19 Vaccines/210

2. ((coronavirus or 2019 ncov or 2019-ncov or covid or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or COVID-19 or COVID-19 virus or coronavirus disease 19 or coronavirus disease 2019 or coronavirus disease 2019 virus or coronavirus disease-19 or sars cov 2 or sars coronavirus 2 or sars-cov-2 or sars2) adj3 (vaccin* or immuni*)).tw,kw.1462

3. ((mRNA or messenger RNA) adj3 vaccin*).tw,kw.342

4. (BNT162b2 or BNT 162b2).tw,kw.165

5. pfizer vaccin×.tw,kw.13

6. moderna vaccin×.tw,kw.13

7. astra zeneca vaccin×.tw,kw.1

8. (AZD1222 or azd 1222).tw,kw.58

9. (mRNA-1273 or mRNA1273).tw,kw.109

10. johnson vaccin×.tw,kw.3

11. Vaxzevria.tw,kw.20

12. astrazenica.tw,kw.14

13. Covishield.tw,kw.25

14. Spikevax.tw,kw.11

15. BNT162b1.tw,kw.6

16. ChAdOx1-S.tw,kw.20

17. or/1–161627

18. (adverse event* or side effect*).tw,kw.229879

19. Adverse Drug Reaction×.tw,kw.29100

20. exp “Drug-Related Side Effects and Adverse Reactions”/3842

21. ((local or systemic) adj2 reaction*).tw,kw.3114

22. reactogenicity.tw,kw. or ae.fs. or aefi.tw,kw.140676

23. risk/or risk factors/or patient safety/or “drug-related side effects and adverse reactions”/31571

24. or/18–23 356,402

25. 17 and 24664

26. Vaccines, Synthetic/ae and COVID-19/0

27. 25 or 26664

28. product surveillance, postmarketing/or pharmacovigilance/123

29. Adverse Drug Reaction Reporting Systems/95

30. (pharmacovigilance or monitor* or drug evaluation*).tw,kw.108677

31. Adverse Drug Reaction Reporting Systems/95

32. Drug Evaluation/5750

33. surveillance.mp.9469

34. Self Report/2662

35. ((self or patient) adj2 report*).tw,kw.56917

36. survey×.mp.70305

37. questionnaire×.mp.163449

38. or/28–37 329,913

39. 27 and 38141

Disclosure statement

KW is Chief Scientists of CANImmunize Inc and has served as a member of the independent data safety advisory board for Medicago and Moderna. KAT receives research support from the Coalition of Epidemic Preparedness Innovations for vaccine safety studies. During the conduct of this work, D. B. F. worked for the University of Ottawa and had academic appointments at the Children’s Hospital of Eastern Ontario Research Institute and ICES; she is currently employed by Pfizer.

Financial support

This work was supported by the Public Health Agency of Canada and Canadian Institute of Health Research through the Canadian Immunization Research Network (FRN#151944).