Abstract
This paper aims to summarize through meta-analyses the overall vaccine effectiveness of the BNT162b2 mRNA vaccine from observational studies. A systematic literature search with no language restriction was performed in electronic databases to identify eligible observational studies which reported the adjusted effectiveness of the BNT162b2 mRNA vaccine to prevent RT-PCR confirmed COVID-19. Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR) and pooled incidence rate ratio (IRR) at 95% confidence intervals, and the vaccine effectiveness was indicated as (pooled HR − 1)/HR or (pooled IRR − 1)/IRR. Nineteen studies were included for this meta-analysis. The meta-analysis revealed significant protective effect against RT-PCR confirmed COVID-19 ≥ 14 days after the first dose, with vaccine effectiveness of 53% (95% confidence interval 32–68%), and ≥ 7 days after the second dose, with vaccine effectiveness of 95% (95% confidence interval: 96–97%). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval. There have been limited data about vaccine effectiveness among immunocompromised patients; thus, the vaccine should be used cautiously in this patient population.
Keywords: BNT162b2, COVID-19, Real world, SARS-CoV-2, Vaccine
Introduction
The global rollout of vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) offers a glimmer of hope toward ending the coronavirus disease 2019 (COVID-19) pandemic. As of the time of writing, there have been with more than 1.7 billion people worldwide received at least one dose of any COVID-19 vaccine, and over 790 million people worldwide are fully vaccinated (Our World in Data 2021).
The phase 3 randomized controlled trial of the BNT162b2 mRNA vaccine against SARS-CoV-2 demonstrated the efficacy of 95% in preventing symptomatic COVID-19, which has led to the emergency conditional approval of the vaccine in many countries (Polack et al. 2020). However, it should be noted that the clinical trial was performed in a highly controlled setting that may not simulate the real-world mass rollout of COVID-19 vaccines.
Therefore, it is imperative to determine the population-level vaccine effectiveness from the mass vaccination campaigns and to report data on the safety aspects of vaccines. This paper aims to summarize through meta-analyses the overall effectiveness of the BNT162b2 mRNA vaccine from large observational studies, which could be important to inform the development of the public health policy related to mass vaccination.
Methods
A systematic literature search with no language restriction was performed in electronic databases, including PubMed, Google Scholar, Scopus, and preprint servers (medRxiv, Research Square, SSRN), to identify eligible studies published up to June 05, 2021. The search strategy was built based on the following keywords and MeSH terms: “BNT162b2”, “Pfizer”, “BioNTech”, “mRNA vaccine”, “mRNA vaccination”, and “effectiveness”. The reference lists of relevant articles were also reviewed to retrieve additional studies. Two investigators (CSK and SSH) independently performed the literature screening to identify eligible studies.
Studies eligible for inclusion were observational studies of any design (case–control, case–cohort, and prospective cohort), which reported the effectiveness of the BNT162b2 mRNA vaccine to prevent reverse transcription–polymerase chain reaction (RT-PCR) confirmed COVID-19 (through comparison between vaccinated and unvaccinated individuals) and adjusted for covariates. For two or more studies which utilized the same data source for their investigations on vaccine effectiveness, we included the one that performed analysis on the latest record. We excluded randomized trials, studies that reported unadjusted effectiveness estimates, studies that reported only non-specific outcomes such as COVID-19-related mortality or COVID-19-related hospitalization, studies where RT-PCR did not use to confirm the diagnosis of COVID-19, and studies that reported vaccine effectiveness against a specific variant(s) of SARS-CoV-2.
Our outcome of interest, namely vaccine effectiveness, is defined as a relative reduction in RT-PCR risk confirmed COVID-19 in vaccinated individuals compared with unvaccinated individuals (Weinberg and Szilagyi 2010). Each included study was independently evaluated by two investigators (CSK and SSH), who also extracted the study characteristics. Study characteristics extracted had the first author’s surname, study design, country, sample population, number of participants, the incidence of COVID-19 in both vaccinated and unvaccinated individuals, and adjusted vaccine effectiveness estimates and covariates adjusted. Two investigators (CSK and SSH) assessed the quality of included observational studies using the Newcastle–Ottawa Scale, with a score of > 7 indicating high quality (Wells et al. 2013).
Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR), pooled incidence rate ratio (IRR), or pooled odds ratio (OR) at 95% confidence intervals, comparing the incidence of RT-PCR confirmed COVID-19 in vaccinated participants relative to unvaccinated participants, when there were three or more studies reporting the same type of effect measure (either HR, IRR, or OR). The vaccine effectiveness was indicated as (pooled HR − 1)/HR, (pooled IRR − 1)/IRR or (pooled OR − 1)/OR, together with a 95% confidence interval. We examined the heterogeneity between studies using the I2 statistics and the χ2 test, with significant heterogeneity set at > 50% and P < 0.10. All analyses were performed using Meta XL, version 5.3 (EpiGear International, Queensland, Australia).
Results
Our literature search yielded 712 abstracts. After deduplication and application of the eligibility criteria, 38 relevant articles were shortlisted for inclusion through full-text examination (Fig. 1). Of these, 19 studies were excluded since they either did not report vaccine effectiveness, reported non-specific outcomes such as COVID-19-related mortality and COVID-19-related hospitalization, or reported unadjusted effectiveness estimates. Therefore, 19 studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021) were included for this meta-analysis; 12 studies (Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Regev-Yochay et al. 2021) were retrospective in design with seven database reviews (Dagan et al. 2021; Emborg et al. 2021; Glampson et al. 2021; Haas et al. 2021; Mason et al. 2021; Monge et al. 2021; Swift et al. 2021), three retrospective case–control studies (Chung et al. 2021; Gras-Valentí et al. 2021; Lopez Bernal et al. 2021), and two retrospective cohort studies (Fabiani et al. 2021; Regev-Yochay et al. 2021); the remaining seven studies (Björk et al. 2021; Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were prospective cohort studies (n = 6) (Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) and prospective database review (n = 1) (Björk et al. 2021). The included studies (Björk et al. 2021; Dagan et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Haas et al. 2021; Hall et al. 2021; Mason et al. 2021; Menni et al. 2021; Monge et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were originated from 8 countries: the United Kingdom (n = 6) (Glampson et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Pritchard et al. 2021; Shrotri et al. 2021), the United States (n = 2) (Swift et al. 2021; Thompson et al. 2021), Canada (n = 1) (Chung et al. 2021) Sweden (n = 1) (Björk et al. 2021), Israel (n = 4) (Angel et al. 2021; Dagan et al. 2021; Haas et al. 2021; Regev-Yochay et al. 2021), Italy (n = 1) (Fabiani et al. 2021), Denmark (n = 1) (Emborg et al. 2021), and Spain (n = 3) (Cabezas et al. 2021; Gras-Valentí et al. 2021; Monge et al. 2021). Study characteristics are depicted in Table 1. The included studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021) are deemed moderate-to-good quality with Newcastle–Ottawa Scale ranging from 7 to 8.
Fig. 1.
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram of process of study selection
Table 1.
Characteristics of included studies
| Study, country | Design | Sample | Total number of participants | Incidence/frequency of COVID-19 | |||||
|---|---|---|---|---|---|---|---|---|---|
| Unvaccinated | ≥ 14 days after dose 1 | Adjusted estimate | Unvaccinated | ≥ 21 days after dose 1 | Adjusted estimate | ||||
| Hall et al., UK | Prospective multicenter | Adults (aged ≥ 18 years) working in publicly funded hospitals in the United Kingdom | 23,324 | 137.5 per 100,000 person-days | 98.6 per 100,000 person-days | HR = 0.44 (0.34–0.57) | 137.5 per 100,000 person-days | 79.6 per 100,000 person-days | HR = 0.44 (0.31–0.63) |
| Mason et al., UK | Retrospective database review |
Vaccinated: Individuals aged 80–83 who were not residents of care homes and had no prior history of COVID-19 Unvaccinated: Individuals aged 76–79 who were not yet eligible for vaccination |
301,462 | 34.0 per 100,000 persons-days | 28.2 per 100,000 persons-days | IRR = 0.83 (0.63–0.91) | 30.0 per 100,000 persons-days | 13.4 per 100,000 persons-days | IRR = 0.45 (0.34–0.59) |
| Björk et al., Sweden | Prospective database review | Individuals aged 18–64 years residing in Skåne county, Sweden, on 27 December 2020 when vaccinations started | 805,741 | 42.0 per 100,000 persons-days | 24.3 per 100,000 persons-days | IRR = 0.58 (0.37–0.86) | 42.0 per 100,000 persons-days | 16.7 per 100,000 persons-days | IRR = 0.40 (0.19–0.73) |
| Dagan et al., Israel | Retrospective database review | Individuals insured in Clalit Health Services | 1,760,152 | – | – | IRR = 0.54 (0.41–0.60) | – | – | IRR = 0.40 (0.34–0.47) |
| Pritchard et al., UK | Prospective cohort study | Randomly selected individuals aged ≥ 16 years | 373,402 | – | – | – | – | – | OR = 0.33 (0.28–0.39) |
| Glampson et al., UK | Retrospective database review | Adults aged ≥ 16 years and registered with a general practitioner, or with a resident postcode, in the North West London catchment area | 2,183,939 | – | – | HR = 0.42 (0.36–0.50) | – | – | HR = 0.22 (0.18–0.27) |
| Monge et al., Spain | Retrospective database review | Residents aged ≥ 65 years and residing in elderly homes | 296,093 | 188.5 per 100,000 persons-day | 92.4 per 100,000 persons-day | HR = 0.49 (0.48–0.50) | 155.8 per 100,000 persons-day | 59.3 per 100,000 persons-day | HR = 0.38 (0.37–0.39) |
| Fabiani et al., Italy | Retrospective cohort study | Frontline health-care personnel employed at the local health unit that serves the entire province of Treviso in the Veneto region | 9878 | 103.0 per 100,000 persons-day | 16.0 per 100,000 persons-day | HR = 0.16 (0.04–0.60) | 28.0 per 100,000 persons-day | 27.0 per 100,000 persons-day | HR = 0.15 (0.02–1.35) |
| Haas et al., Israel | Retrospective database review | Residents of Israel (ie, the census population) aged 16 years and older | 154,648 | 91.5 per 100,000 persons-day | 34.1 per 100,000 persons-day | IRR = 0.42 (0.40–0.45) | – | – | – |
| Swift et al., US | Retrospective database review | Actively employed health-care personnel at the Mayo Clinic | 71,152 | – | – | IRR = 0.22 (0.18–0.27) | – | – | – |
| Gras–Valentí et al., Spain |
Retrospective Case–control study |
Healthcare personnel at the Department of Health of General University Hospital of Alicante |
268 | n = 31/91 (34.1%) | n = 39/177 (22.0%) | OR = 0.47 (0.23–0.99) | – | – | – |
| Lopez Bernal et al., UK | Retrospective test negative case–control study | Adults aged 70 years or older in England who reported having symptoms and tested for COVID-19 | 80,545 | n = 37,320/126697 (29.5%) | n = 811/3285 (24.7) | OR = 0.84 (0.77–0.91) | n = 37,320/126697 (29.5%) | n = 367/2036 (18.0%) | OR = 0.61 (0.54–0.69) |
| Angel et al., Israel | Retrospective cohort study | Healthcare workers at Tel Aviv Sourasky Medical Center | 6710 | – | – | – | – | – | – |
| Chung et al., Canada | Retrospective test negative case–control study | Community-dwelling adults aged ≥ 16 years who were tested for SARS-CoV-2 and had COVID-19 symptoms | 310,880 | n = 51,220/302761 (16.9) | n = 636/8119 (7.8%) | OR = 0.41 (0.38–0.45) | – | – | – |
| Shrotri et al., UK | Prospective cohort study | Care home residents aged ≥ 65 years from 310 long-term care facilities | 4274 | 213.9 per 100,000 persons-day | 282.6 per 100,000 persons-day | HR = 0.77 (0.37–1.58) | 213.9 per 100,000 persons-day | 266.7 per 100,000 persons-day | HR = 0.94 (0.50–1.79) |
| Regev-Yochay et al., Israel | Retrospective cohort study | Healthcare workers at Sheba Medical Center | 9650 | – | – | – | – | – | – |
| Emborg et al., Denmark | Retrospective database review | 5 priority groups: Individuals living in long-term care facilities; ≥ 65 years living at home requiring practical help and personal care; individuals aged 85 and older; frontline health-care workers; individuals with high risk of severe COVID-19 | 864,096 | – | – | HR = 0.93 (0.85–1.01) | – | – | HR = 0.58 (0.50–0.67) |
| Thompson et al., US | Prospective cohort study | Healthcare personnel, first responders, and other essential and frontline workers in eight locations | 5969 | 121.9 per 100,000 persons–day | 16.2 per 100,000 persons-day | HR = 0.20 (0.10–0.40) | – | – | – |
| Cabezas et al., Spain | Prospective cohort study | Nursing home residents | 28,191 | 266.2 per 100,000 persons-day | 175.8 per 100,000 persons-day | HR = 0.77 (0.69–0.86) | – | – | – |
| Nursing home staff | 26,075 | 138.6 per 100,000 persons-day | 121.1 per 100,000 persons-day | HR = 0.80 (0.68–0.93) | – | – | – | ||
| Healthcare workers in nursing home | 47,106 | 103.2 per 100,000 persons-day | 98.9 per 100,000 persons-day | HR = 0.85 (0.77–0.95) | – | – | – | ||
| Study, country | Incidence/frequency of COVID-19 | Covariates adjustment/matching | NOS | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Adjusted estimate | Unvaccinated | ≥ 7 days after dose 2 | Adjusted estimate | Unvaccinated | ≥ 14 days after dose 2 | Adjusted estimate | |||
| Hall et al., UK | HR = 0.44 (0.31–0.63) | 137.5 per 100,000 person-days | 42.9 per 100,000 person-days | HR = 0.19 (0.07–0.51) | 137.5 per 100,000 person-days | 39.5 per 100,000 person-days | HR = 0.14 (0.06–0.34) | Age, sex, ethnicity, comorbidities, job role, frequency of contact with COVID-19 patients, employed in a patient-facing role, and occupational exposure, period | 7 |
| Mason et al., UK | IRR = 0.45 (0.34–0.59) | – | – | – | – | – | – | Sex, area of residence, small area deprivation, ethnic group, health status, living arrangements, seasonal influenza vaccine history since April 2020, emergency hospital stays in the previous two months | 7 |
| Björk et al., Sweden | IRR = 0.40 (0.19–0.73) | 42.0 per 100,000 persons-days | 6.0 per 100,000 persons-days | IRR = 0.14 (0.06–0.28) | – | – | – | Age, sex | 7 |
| Dagan et al., Israel | IRR = 0.40 (0.34–0.47) | – | – | IRR = 0.08 (0.05–0.12) | – | – | – | Age, sex, sector, neighborhood of residence, history of influenza vaccination during the preceding 5 years, total number of coexisting conditions | 8 |
| Pritchard et al., UK | OR = 0.33 (0.28–0.39) | – | – | OR = 0.28 (0.21–0.36) | – | – | – |
Age, sex, ethnicity, index of multiple deprivation, working in a care-home, having a patient-facing role in health or social care, presence of long-term health conditions, household size, multigenerational household, rural–urban classification, direct or indirect contact with a hospital or care-home, smoking status, mode of travel to work, work location, visit frequency, geographic area |
8 |
| Glampson et al., UK | HR = 0.22 (0.18–0.27) | – | – | – | – | – | – | Age, sex, ethnicity, index of multiple deprivation, vaccination manufacturer | 8 |
| Monge et al., Spain | HR = 0.38 (0.37–0.39) | – | – | – | Follow-up day, previous COVID-19 (before beginning of follow-up), daily-varying 7-day SARS-CoV-2 cumulative incidence specific to the province, its quadratic term, the empirical reproduction number for that province on that date | 7 | |||
| Fabiani et al., Italy | HR = 0.15 (0.02–1.35) | 19.0 per 100,000 persons-day | 27.0 per 100,000 persons-day | HR = 0.05 (0.01–0.38) | - | - | - | Age group, sex, professional category, work context, starting week of exposure | 7 |
| Haas et al., Israel | – | 91.5 per 100,000 persons-day | 3.1 per 100,000 persons-day | IRR = 0.05 (0.04–0.06) | 91.5 per 100,000 persons-day | 2.1 per 100,000 persons-day | IRR = 0.04 (0.03–0.05) | Age group, sex, calendar week | 8 |
| Swift et al., US | – | – | – | – | – | – | IRR = 0.03 (0.02–0.05) | Age, sex, job type, geographic location | 7 |
| Gras–Valentí et al., Spain | – | – | – | – | – | – | – | Vaccination status, reason for COVID-19 testing, job role, department | 7 |
| Lopez Bernal et al., UK | OR = 0.61 (0.54–0.69) | n = 37,320/126697 (29.5%) | n = 31/245 (12.7) | OR = 0.26 (0.18–0.39) | n = 37,320/126697 (29.5%) | n = 42/714 (5.9) | OR = 0.17 (0.12–0.23) | Age, period, sex, region, ethnicity, care-home, index of multiple deprivation fifth | 7 |
| Angel et al., Israel | – |
Symptomatic: 149.8 per 100,000 persons-day Asymptomatic: 67.0 per 100,000 persons-day |
Symptomatic: 4.7 per 100,000 persons-day Asymptomatic: 11.3 per 100,000 persons-day |
Symptomatic: IRR = 0.03 (0.01–0.06) Asymptomatic: IRR = 0.14 (0.07–0.31) |
Symptomatic: 146.3 per 100,000 persons-day Asymptomatic: 69.9 per 100,000 persons-day |
Symptomatic: 2.1 per 100,000 persons-day Asymptomatic: 4.2 per 100,000 persons-day |
Symptomatic: IRR = 0.02 (0.01–0.06) Asymptomatic: IRR = 0.06 (0.02–0.22) |
Age, sex, employment sector, exposure risk, number of PCR tests for each health-care worker in the time frame under observation | 7 |
| Chung et al., Canada | – | n = 51,220/302761 (16.9) | n = 51/3326 (1.5%) | OR = 0.09 (0.07–0.12) | – | – | – | Age, sex, public health unit region, biweekly period of test, number of SARS-CoV-2 tests in the 3 months prior to 14 December 2020, presence of any comorbidity that increase the risk of severe COVID-19, receipt of influenza vaccination in current or prior influenza season, neighborhood income, essential worker, persons per dwelling, proportion of persons employed as non-health essential workers, self-identified visible minority quintiles | 8 |
| Shrotri et al., UK | HR = 0.94 (0.50–1.79) | – | – | – | – | – | – | Age, sex, local monthly infection incidence, bed capacity | 7 |
| Regev-Yochay et al., Israel | – | 81.9 per 100,000 persons-day | 29.8 per 100,000 persons-day | IRR = 0.25 (0.18–0.34) | 81.9 per 100,000 persons-day | 9.4 per 100,000 persons-day | HR = 0.12 (0.08–0.17) | Intensity of exposure | 7 |
| Emborg et al., Denmark | HR = 0.58 (0.50–0.67) | – | – | HR = 0.18 (0.16–0.21) | – | – | – | Age, sex, comorbidities, hospital admission, calendar time | 7 |
| Thompson et al., US | – | – | – | – | 121.9 per 100,000 persons-day | 2.5 per 100,000 persons-day | HR = 0.07 (0.02–0.22) | Age, sex, race, ethnicity, health status, comorbidities, medications, household characteristics, influenza vaccination history, study week, local virus circulation, study location, occupation, number of hours worked in contact with patients or the public, number of hours in direct contact with someone with known or suspected COVID-19, percent of time wearing personal protective equipment during each of those exposure categories | 7 |
| Cabezas et al., Spain | – | – | – | – | – | – | – | Age, sex | 7 |
| – | – | – | – | – | – | – | Age, sex | ||
| – | – | – | – | – | – | – | Age, sex | ||
COVID-19 coronavirus disease 2019 HR hazard ratio IRR incidence rate ratio Newcastle–Ottawa Scale OR odds ratio
The meta-analysis of eight studies (Cabezas et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021; Thompson et al. 2021) which presented effect measure as HR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled HR = 0.58; 95% confidence interval: 0.45–0.75; Fig. 2), where pooled estimate indicates vaccine effectiveness of 42% (95% confidence interval 25%–55%). Similarly, the meta-analysis of five studies (Björk et al. 2021; Dagan et al. 2021; Haas et al. 2021; Mason et al. 2021; Swift et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled IRR = 0.47; 95% confidence interval: 0.32–0.68; Fig. 3), where pooled estimate indicates vaccine effectiveness of 53% (95% confidence interval 32%–68%).
Fig. 2.
Pooled hazard ratio (HR) of the incidence of COVID-19 14 as well as 21 days post first dose of vaccine (A) and 7 as well as 14 days post second dose of vaccine (B) relative to no vaccination
Fig. 3.
Pooled incident rate ratio (IRR) of the incidence of COVID-19 14 as well as 21 days post first dose of vaccine (A) and 7 as well as 14 days post second dose of vaccine (B) relative to no vaccination
Even higher vaccine effectiveness was observed 21 days or more after the first dose of BNT162b2 mRNA vaccine, where the meta-analysis of six studies (Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021) which presented effect measure as HR reported pooled HR of 0.42 (95% confidence interval: 0.31–0.57; Fig. 2), and thus vaccine effectiveness of 58% (95% confidence interval: 43%–69%). Likewise, the meta-analysis of three studies (Björk et al. 2021; Dagan et al. 2021; Mason et al. 2021) which presented effect measure as IRR reported pooled IRR of 0.41 (95% confidence interval: 0.36–0.47; Fig. 3), and thus vaccine effectiveness of 59% (95% confidence interval: 53–64%).
The recipient of the second dose of the BNT162b2 mRNA vaccine further boosted the vaccine effectiveness. The meta-analysis of three studies (Emborg et al. 2021; Fabiani et al. 2021; Hall et al. 2021) which presented effect measure as HR reported pooled HR of 0.18 (95% confidence interval: 0.16–0.20; Fig. 2) 7 days or more after the second dose, and thus vaccine effectiveness of 82% (95% confidence interval: 80–84%). Similarly, the meta-analysis of five studies (Angel et al. 2021; Björk et al. 2021; Dagan et al. 2021; Haas et al. 2021; Regev-Yochay et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 7 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.09; 95% confidence interval: 0.05–0.19; Fig. 3), where pooled estimate indicates vaccine effectiveness of 91% (95% confidence interval 81%–95%). The findings from the meta-analysis of three studies (Chung et al. 2021; Lopez Bernal et al. 2021; Pritchard et al. 2021) which presented effect measure as OR are also consistent (pooled OR = 0.19; 95% confidence interval 0.09–0.40) and show vaccine effectiveness of 81% (95% confidence interval 60%-91%) 7 days or more after the second dose of BNT162b2 mRNA vaccine. The meta-analysis of three studies (Hall et al. 2021; Regev-Yochay et al. 2021; Thompson et al. 2021) which presented effect measure as HR reported pooled HR of 0.12 (95% confidence interval: 0.08–0.16; Fig. 2) 14 days or more after the second dose, and thus vaccine effectiveness of 88% (95% confidence interval: 84%–92%). Likewise, the meta-analysis of three studies (Angel et al. 2021; Haas et al. 2021; Swift et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.04; 95% confidence interval: 0.03–0.05; Fig. 3), where pooled estimate indicates vaccine effectiveness of 96% (95% confidence interval 95–97%).
Discussion
The findings of the meta-analyses align with the phase 3 randomized controlled trial (Polack et al. 2020) of BNT162b2 mRNA vaccine, though with a lower protective rate: 82% after the first dose (versus overall vaccine effectiveness of 48–55% [14–21 days or more] after the first dose in the current study; Fig. 2) and 95% (7 days or more) after the second dose (versus overall vaccine effectiveness of 86–94% [7–14 days or more] after the second dose in the current study; Fig. 3). Variability in the protective rate between clinical trial and real-world studies could stem from the difference in the definition of confirmed COVID-19; confirmed COVID-19 was defined in the clinical trial as the presence of symptoms and positive RT-PCR test for SARS-CoV-2; while the included studies of our meta-analyses, confirmed COVID-19 was defined as positive RT-PCR test for SARS-CoV-2 regardless of the presence of symptoms.
In addition, individuals with comorbidities (e.g., hypertension, diabetes, and obesity) who are predisposed to severe COVID-19 constituted only about one-fifth of the study population in phase 3 randomized controlled trial (Polack et al. 2020) of BNT162b2 mRNA vaccine. Individuals with comorbidities (e.g., hypertension, diabetes, and obesity), especially those with old age, are often prioritized in the real-world mass vaccination campaign. Therefore, this could explain the lack of reproducible vaccine efficacy reported from the highly controlled clinical research settings compared to the real-world settings since these individuals with comorbidities mainly constituted the real-world study population. Indeed, elderly individuals with comorbidities often have diminished immune responses to vaccines (Kwetkat and Heppner 2020).
Nevertheless, with up to 59% of real-world protective rate after the administration of the first dose of the BNT162b2 mRNA vaccine, it seems reasonable to delay the administration of the second dose in an attempt to allow vaccination in a higher proportion of individuals to reduce the risk of transmission of COVID-19 to an acceptable level. Our study was limited by the fact that included studies were originated in only a few countries. Therefore, the generalizability of our findings is unknown, especially to the countries where variants of concern of SARS-CoV-2 are circulating. Future studies should aim to investigate the vaccine effectiveness against different variants of concern of SARS-CoV-2 and with longer follow-ups to determine the duration of protection against COVID-19. Furthermore, the effectiveness of the BNT162b2 mRNA vaccine among immunocompromised individuals as well as individuals who receive treatment with immunosuppressive therapy should also be investigated since they had been excluded from the participation of phase 3 randomized controlled trial (Polack et al. 2020). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Declarations
Conflict of interest
The authors report no conflict of interest.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- Angel Y, Spitzer A, Henig O, Saiag E, Sprecher E, Padova H, Ben-Ami R (2021) Association between vaccination with BNT162b2 and incidence of symptomatic and asymptomatic SARS-CoV-2 infections among health care workers. JAMA 325:2457–2465 [DOI] [PMC free article] [PubMed]
- Björk J, Inghammar M, Moghaddassi M, Rasmussen M, Malmqvist U, Kahn F (2021) Effectiveness of the BNT162b2 vaccine in preventing COVID-19 in the working age population-first results from a cohort study in Southern Sweden. Preprint. medRxiv 2021.04.20.21254636
- Cabezas C, Coma E, Mora-Fernandez N, Li X, Martinez-Marcos M, Fina-Aviles F, Fabregas M, Hermosilla E, Jover A, Contel JC, Lejardi Y (2021) Effects of BNT162b2 mRNA Vaccination on COVID-19 Disease, Hospitalisation and Mortality in Nursing Homes and Healthcare Workers: a Prospective Cohort Study Including 28,594 Nursing Home Residents, 26,238 Nursing Home Staff, and 61,951 Healthcare Workers in Catalonia. Preprint. SSRN ssrn.3815682 [DOI] [PMC free article] [PubMed]
- Chung H, He S, Nasreen S, Sundaram M, Buchan S, Wilson S, Chen B, Calzavara A, Fell D, Austin PC, Wilson K (2021) Effectiveness of BNT162b2 and mRNA-1273 COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe COVID-19 outcomes in Ontario, Canada. Preprint. medRxiv 2021.05.24.21257744 [DOI] [PMC free article] [PubMed]
- Dagan N, Barda N, Kepten E, Miron O, Perchik S, Katz MA, Hernán MA, Lipsitch M, Reis B, Balicer RD. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021;384:1412–1423. doi: 10.1056/NEJMoa2101765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Emborg H, Valentiner-Branth P, Schelde AB, Nielsen KF, Gram MA, Moustsen-Helms IR, Chaine M, Seidelin UH, Nielsen J (2021) Vaccine effectiveness of the BNT162b2 mRNA COVID-19 vaccine against RT-PCR confirmed SARS-CoV-2 infections, hospitalisations and mortality in prioritised risk groups. Preprint. medRxiv 2021.05.27.21257583
- Fabiani M, Ramigni M, Gobbetto V, Mateo-Urdiales A, Pezzotti P, Piovesan C. Effectiveness of the Comirnaty (BNT162b2, BioNTech/Pfizer) vaccine in preventing SARS-CoV-2 infection among healthcare workers, Treviso province, Veneto region, Italy, 27 December 2020 to 24 March 2021. Euro Surveill. 2021;26:2100420. doi: 10.2807/1560-7917.ES.2021.26.17.2100420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glampson B, Brittain J, Kaura A, Mulla A, Mercuri L, Brett S, Aylin P, Sandall T, Goodman I, Redhead J, Saravanakumar K (2021) North West London Covid-19 Vaccination Programme: Real-world evidence for Vaccine uptake and effectiveness. Preprint. medRxiv 2021.04.08.21254580 [DOI] [PMC free article] [PubMed]
- Gras-Valentí P, Chico-Sánchez P, Algado-Sellés N, Jiménez-Sepúlveda NJ, Gómez-Sotero IL, Fuster-Pérez M, Cartagena-Llopis L, Sánchez-Valero M, Cerezo-Milán P, Martínez-Tornero I, Tremiño-Sánchez L, Nadal-Morante V, Monerris-Palmer M, Esclapez-Martínez A, MorenodeArcos-Fuentes E, Escalada-Martín I, Escribano-Cañadas I, Merino-Lucas E, Rodríguez-Díaz JC, Sánchez-Payá J. Efectividad de la primera dosis de vacuna BNT162b2 para prevenir la COVID-19 en personal sanitario [Effectiveness of the first dose of BNT162b2 vaccine to preventing covid-19 in healthcare personnel.] Rev Esp Salud Publica. 2021;95:e202104070. [PubMed] [Google Scholar]
- Haas EJ, Angulo FJ, McLaughlin JM, Anis E, Singer SR, Khan F, Brooks N, Smaja M, Mircus G, Pan K, Southern J, Swerdlow DL, Jodar L, Levy Y, Alroy-Preis S. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. Lancet. 2021;S0140–6736(21):00947–948. doi: 10.1016/S0140-6736(21)00947-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hall VJ, Foulkes S, Saei A, Andrews N, Oguti B, Charlett A, Wellington E, Stowe J, Gillson N, Atti A, Islam J, Karagiannis I, Munro K, Khawam J, Chand MA, Brown CS, Ramsay M, Lopez-Bernal J, Hopkins S, SIREN Study Group COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. Lancet. 2021;397:1725–1735. doi: 10.1016/S0140-6736(21)00790-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kwetkat A, Heppner HJ. Comorbidities in the elderly and their possible influence on vaccine response. Interdiscip Top Gerontol Geriatr. 2020;43:73–85. doi: 10.1159/000504491. [DOI] [PubMed] [Google Scholar]
- Lopez Bernal J, Andrews N, Gower C, Robertson C, Stowe J, Tessier E, Simmons R, Cottrell S, Roberts R, O’Doherty M, Brown K, Cameron C, Stockton D, McMenamin J, Ramsay M. Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on covid-19 related symptoms, hospital admissions, and mortality in older adults in England: test negative case-control study. BMJ. 2021;373:n1088. doi: 10.1136/bmj.n1088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mason TF, Whitston M, Hodgson J, Watkinson RE, Lau YS, Abdulrazeg O, Sutton M (2021) Effects of BNT162b2 mRNA vaccine on Covid-19 infection and hospitalisation among older people: matched case control study for England. Preprint. medRxiv 2021.04.19.21255461 [DOI] [PMC free article] [PubMed]
- Menni C, Klaser K, May A, Polidori L, Capdevila J, Louca P, Sudre CH, Nguyen LH, Drew DA, Merino J, Hu C, Selvachandran S, Antonelli M, Murray B, Canas LS, Molteni E, Graham MS, Modat M, Joshi AD, Mangino M, Hammers A, Goodman AL, Chan AT, Wolf J, Steves CJ, Valdes AM, Ourselin S, Spector TD. Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study. Lancet Infect Dis. 2021;S1473–3099(21):00224–233. doi: 10.1016/S1473-3099(21)00224-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Monge S, Olmedo C, Alejos B, Lapena M, Sierra MJ, Limia A (2021) Direct and indirect effectiveness of mRNA vaccination against SARS-CoV-2 infection in long-term care facilities in Spain. Preprint. medRxiv 2021.04.08.21255055 [DOI] [PMC free article] [PubMed]
- Moustsen-Helms IR, Emborg HD, Nielsen J, Nielsen KF, Krause TG, Molbak K, Moeller KL, Berthelsen AS, Valentiner-Branth P. Vaccine effectiveness after 1st and 2nd dose of the BNT162b2 mRNA Covid-19 Vaccine in long-term care facility residents and healthcare workers–a Danish cohort study. Preprint. medRxiv 2021.03.08.21252200
- Our World in Data (2021) Statistics and Research: Coronavirus (COVID-19) Vaccinations https://ourworldindata.org/covid-vaccinations. Accessed 29 Apr 2021
- Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Pérez Marc G, Moreira ED, Zerbini C, Bailey R, Swanson KA, Roychoudhury S, Koury K, Li P, Kalina WV, Cooper D, Frenck RW, Jr, Hammitt LL, Türeci Ö, Nell H, Schaefer A, Ünal S, Tresnan DB, Mather S, Dormitzer PR, Şahin U, Jansen KU, Gruber WC, C4591001 Clinical Trial Group Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603–2615. doi: 10.1056/NEJMoa2034577. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pritchard E, Matthews PC, Stoesser N, Eyre DW, Gethings O, Vihta KD, Jones J, House T, VanSteenHouse H, Bell I, Bell JI, Newton JN, Farrar J, Diamond I, Rourke E, Studley R, Crook D, Peto TEA, Walker AS, Pouwels KB (2021) Impact of vaccination on new SARS-CoV-2 infections in the United Kingdom. Nat Med. 10.1038/s41591-021-01410-w [DOI] [PMC free article] [PubMed]
- Regev-Yochay G, Amit S, Bergwerk M, Lipsitch M, Leshem E, Kahn R, Lustig Y, Cohen C, Doolman R, Ziv A, Novikov I (2021) Decreased infectivity following BNT162b2 vaccination. Preprint. SSRN ssrn.3815668 [DOI] [PMC free article] [PubMed]
- Shrotri M, Krutikov M, Palmer T, Giddings R, Azmi B, Subbarao S, Fuller C, Irwin-Singer A, Davies D, Tut G, Bernal JL, Moss P, Hayward A, Copas A, Shallcross L (2021) Vaccine effectiveness of the first dose of ChAdOx1 nCoV-19 and BNT162b2 against SARS-CoV-2 infection in residents of long-term care facilities in England (VIVALDI): a prospective cohort study. Lancet Infect Dis. 10.1016/S1473-3099(21)00289-9 [DOI] [PMC free article] [PubMed]
- Swift MD, Breeher LE, Tande AJ, Tommaso CP, Hainy CM, Chu H, Murad MH, Berbari EF, Virk A (2021) Effectiveness of mRNA COVID-19 vaccines against SARS-CoV-2 infection in a cohort of healthcare personnel. Clin Infect Dis ciab361 [DOI] [PMC free article] [PubMed]
- Thompson MG, Burgess JL, Naleway AL, Tyner HL, Yoon SK, Meece J, Olsho LEW, Caban-Martinez AJ, Fowlkes A, Lutrick K, Kuntz JL, Dunnigan K, Odean MJ, Hegmann KT, Stefanski E, Edwards LJ, Schaefer-Solle N, Grant L, Ellingson K, Groom HC, Zunie T, Thiese MS, Ivacic L, Wesley MG, Lamberte JM, Sun X, Smith ME, Phillips AL, Groover KD, Yoo YM, Gerald J, Brown RT, Herring MK, Joseph G, Beitel S, Morrill TC, Mak J, Rivers P, Harris KM, Hunt DR, Arvay ML, Kutty P, Fry AM, Gaglani M (2021) Prevention and Attenuation of COVID-19 by BNT162b2 and mRNA-1273 Vaccines. Preprint. medRxiv 2021.06.01.21257987
- Weinberg GA, Szilagyi PG. Vaccine epidemiology: efficacy, effectiveness, and the translational research roadmap. J Infect Dis. 2010;201:1607–1610. doi: 10.1086/652404. [DOI] [PubMed] [Google Scholar]
- Wells G, Shea B, O’Connell D, Peterson J, Welch V, Losos M, Tugwell P (2013) The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed 29 Apr 2021