Real-world effectiveness of mRNA COVID-19 vaccines in the elderly during the Delta and Omicron variants: Systematic review

Abstract

BACKGROUND

As of 31 December 2022, there were over 6.6 million coronavirus disease 2019 (COVID-19) deaths and over 651 million cases across 200 countries worldwide. Despite the increase in vaccinations and booster shots, COVID-19 cases and deaths continue to remain high. While the effectiveness of these vaccines has already been established by different manufacturers, the fact remains that these vaccines were created quickly for global emergency use, tested under controlled clinical conditions from voluntary subjects and age groups whose general characteristics may differ from the actual general population.

AIM

To conduct a systematic review to determine the real-world effectiveness of mRNA COVID-19 vaccines in the elderly during the predominance of Delta and Omicron variants in preventing COVID-19 related infection, hospital, intensive care unit (ICU) admission and intubation, and death.

METHODS

A combination of Medical Subject Headings and non–Medical Subject Headings was carried out to identify all relevant research articles that meets the inclusion and exclusion criteria from PubMed, Cochrane, CINAHL, Scopus, ProQuest, Embase, Web of Science, and Google Scholar databases, as well as qualified research studies from pre–print servers using medRxiv and Research Square, published from January 1, 2021 – December 31, 2022.

RESULTS

As per the inclusion and exclusion criteria, the effectiveness of Pfizer-BioNTech and Moderna vaccines were evaluated from an estimated total study population of 26,535,692 using infection, hospital, ICU admission and intubation, and death as outcome measures from studies published between 2021 and 2022, conducted in New York, Finland, Canada, Costa Rica, Qatar, Greece, and Brazil. The risk of bias was evaluated using risk of bias in nonrandomized studies of interventions (ROBINS-I) tool for cohort, case-control, and cross-sectional studies. While clinical trial data on Pfizer-BioNTech and Moderna vaccines demonstrated 94% vaccine effectiveness in the elderly, the results in this study showed that vaccine effectiveness in real-world settings is marginally lower against infection (40%–89%), hospitalization (92%), ICU admission and intubation (98%–85%), and death (77%–87%) with an indication of diminished effectiveness of vaccine over time. Furthermore, 2 doses of mRNA vaccines are inadequate and only provides interim protection.

CONCLUSION

Because of the natural diminishing effectiveness of the vaccine, the need for booster dose to restore its efficacy is vital. From a research perspective, the use of highly heterogeneous outcome measures inhibits the comparison, contrast, and integration of the results which makes data pooling across different studies problematic. While pharmaceutical intervention like vaccination is important to fight an epidemic, utilizing common outcome measurements or carrying out studies with minimal heterogeneity in outcome measurements, is equally crucial to better understand and respond to an international health crisis.

Core Tip:

This systematic review investigates the real-world effectiveness of mRNA coronavirus disease 2019 (COVID-19) vaccines in reducing morbidity and mortality in the elderly during the predominance of Delta and Omicron variants. This study found that the effectiveness of mRNA COVID-19 vaccines in the elderly against the Delta and Omicron variants is marginally lower than what was suggested in clinical trial data. Vaccine efficacy also diminishes over time, indicating the need for a booster dose to restore its effectiveness. Furthermore, to better understand and respond to an epidemic, studies should utilize common outcome measurements or minimize heterogeneity in outcome measures to facilitate data comparison and integration of results.

INTRODUCTION

As of December 31, 2022, there were over 6.6 million coronavirus disease 2019 (COVID-19) deaths and over 651 million cases across 200 countries worldwide^[1]. During the second half of 2021, COVID-19 cases and deaths were predominantly influenced by the Delta and Omicron variants, wreaking havoc even in countries with tough COVID-19 restrictions^[2,3]. Epidemiological studies have shown that the contagious and highly transmissible nature of the Delta and Omicron variants has even put the elderly population in a more disadvantaged position, accounting roughly 14% of all COVID-19 cases and 70% of all COVID-19 deaths as of December 31, 2022^[4–7]. While there is no broad consensus on the age at which a person can be considered elderly, the approved cutoff age as per the United Nations is 60+ years^[8].

As a response to the extensive impact of COVID-19, which has become a public health concern and an international health crisis, the Centers for Disease Control and Prevention rolled out a global strategy response framework which outlined a combination of non-pharmaceutical and pharmaceutical interventions^[9–11]. While the primary method of epidemic control has been non-pharmaceutical measures, pharmaceutical intervention, like vaccine, is expected to be the only effective, long-term defense against infection and death^[12,13]. Vaccination is critical since the epidemic is still challenging to control due to the dormant symptoms and contagious nature of the virus especially during the incubation period which triggers late detection of infection^[12,13].

Of the 356 vaccine candidates, over 12 billion vaccine doses have been administered by 34 different vaccines approved under Emergency Use Authorization^[1,14]. Despite the increase in vaccinations and booster shots, COVID-19 cases and deaths continue to remain high^[1]. While the effectiveness of these vaccines has already been established by different manufacturers, the fact remains that these vaccines were created quickly for global emergency use, tested under controlled clinical conditions from voluntary subjects and age groups whose general characteristics may differ from the actual general population^[15–17]. In spite of the many observational studies providing data on the effectiveness of vaccination in various populations, this study aims to compile the disparate data through systematic review^[18–29]. This study carefully examines the effectiveness of COVID-19 vaccines in real-world settings in the elderly during the predominance of Delta and Omicron variants.

MATERIALS AND METHODS

The systematic review was designed according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards to ensure a comprehensive and methodical approach^[30].

Search strategy and selection criteria

The review searched for qualified studies using a combination of Medical Subject Headings (MeSH) and non–Medical Subject Headings from PubMed, Cochrane, CINAHL, Scopus, ProQuest, Embase, Web of Science, and Google Scholar databases, as well as qualified research studies from pre–print servers using medRxiv and Research Square, published from January 1, 2021 – December 31, 2022. The search was independently performed by a single researcher using the following keywords and search terms (Supplementary Table 1: Keywords and Search Terms using PICO): (1) covid-19; covid 19; covid19; SARS CoV 2*; SARS-CoV-2*; SARS Coronavirus 2 Infection; sars virus; 2019 Novel Coronavirus*; nCoV; 2019-nCoV*; COVID-19 Pandemic*; COVID-19 Virus*; Coronavirus; Coronavirus Disease*; Severe Acute Respiratory Syndrome Coronavirus 2 Infection; CV-19; CV19; (2) covid 19 vaccine*; covid-19 vaccine*; Pfizer-BioNTech vaccine; Comirnaty; BNT162b2; Bnt-162b2; Bnt162b2; Tozinameran; Tozinameran [INN]; UNII-5085ZFP6SJ; Moderna vaccine; mRNA-1273; MRNA-1273; Spikevax; CX-024414; Elasomeran; Elasomeran [INN]; M-1273; Moderna covid-19 vaccine rna; TAK-919; UNII-EPK39PL4R4; Covid 19 booster; Covid-19 booster; SARS-CoV-2 vaccine; SARS-CoV-2 booster; vaccinated; inoculat*; immuni*; post-vaccination; antibody; protected; (3) unvaccinated; uninoculated; uninoculated; unimmunized; unprotected; susceptible; (4) reduce incidence*; reduce admission*; reduce infection*; reduce hospitalization*; reduce morbidity*; reduce mortality*; reduce death*; lessen infection*; lessen admission*; lessen hospitalization*; lessen morbidity*; lessen mortality*; lessen death*; prevent incidence*; prevent infection*; prevent admission*; prevent hospitalization*; prevent morbidity*; prevent mortality*; prevent death*; minimize incidence*; minimize admission*; minimize infection*; minimize hospitalization*; minimize morbidity*; minimize mortality*; minimize death*; control incidence*; control admission*; control infection*; control hospitalization*; control morbidity*; control mortality*; control death*; combat incidence*; combat admission*; combat infection*; combat hospitalization*; combat morbidity*; combat mortality*; combat death*; eliminate incidence*; eliminate admission*; eliminate infection*; eliminate hospitalization*; eliminate morbidity*; eliminate mortality*; eliminate death*; diminish incidence*; diminish admission*; diminish infection*; diminish hospitalization*; diminish morbidity*; diminish mortality*; diminish death*; solve incidence*; solve admission*; solve infection*; solve hospitalization*; solve morbidity*; solve mortality*; solve death*.

Eligibility standards: inclusion and exclusion criteria

In accordance to the inclusion criteria, the systematic review identified relevant English-published observational studies, which examined the effectiveness of COVID-19 vaccines among the (1) elderly populations who were ≥ 60 years old; (2) recipient of at least 2 doses of mRNA (Pfizer-BioNTech and Moderna) vaccines; (3) during the predominance of Delta (B.1.617.2) or Omicron (B.1.1.529/BA); and (4) studies which examined subjects as COVID-19 positive based on a positive Reverse Transcription Polymerase Chain Reaction (RT-PCR or PCR) tests as well as studies which compared and examined the incidence of COVID-19, infection, hospitalization, admission to intensive care unit (ICU), intubation, and death. This systematic review, however, will not include (1) systematic review and meta-analysis studies, case reports, case series, reviews, editorials, conference papers, letters, and correspondence; (2) studies on animals; (3) studies with mathematical modelling analysis; (4) studies with insufficient data to calculate the prevention rate of COVID-19; (5) studies with immunocompromised subjects; (6) studies which did not have an unvaccinated subjects to compare; (7) studies that did not use SARS-CoV-2 vaccination as the exposure; (8) duplicate studies or studies with overlapping participants; and (9) studies that did not explain how COVID-19 subjects were determined (Figure 1).

Figure 1.

Flow diagram of study identification and selection process.

Data extraction and outcomes

The review process underwent 4 stages: (1) All the papers found within the identified databases were examined and the publication year, study titles, authors, and abstracts were imported into an Excel spreadsheet; (2) the records were managed, screened and duplicates were eliminated manually by assessing the study title, authors, and abstract for inclusion; (3) only those with titles and abstracts that match the inclusion criteria were retrieved and carefully evaluated for full text review; and (4) using a separate Excel spreadsheet, a data extraction sheet was developed to independently extract the general study characteristics (author and publication year, study design, location, purpose, study population, including age of study population, variant of concern, vaccine type, number of doses received, outcome measures, vaccine effectiveness, and results). All qualified studies for systematic reviews were imported, stored, and managed in EndNote20.

The studies included in the review were assessed based on: (I) age of study population; (II) variant of concern; (III) type of vaccine used, and (IV) effectiveness of vaccines based on outcome measures. The effectiveness of COVID-19 mRNA vaccines in reducing morbidity and mortality were examined by comparing the following outcomes amongst the selected studies:

1. Effectiveness of COVID-19 mRNA vaccines to reduce morbidity in terms of infections, hospitalization, admission to ICU and intubation;

2. Effectiveness of COVID-19 mRNA vaccines to reduce mortality or deaths.

This study did not require the approval of an ethical committee or an Institutional Review Board since data collection and synthesis were gathered from already published studies in which proper consent or approvals would have been obtained by the researchers.

Quality assessment and data synthesis

The methodological quality of these observational studies was assessed through the risk of bias using ROBINS-I tool (risk of bias in non-randomized studies of interventions) and were analyzed using a narrative synthesis method which gathered the information from several sources and employed words and text to summarize and explain the findings since meta-analysis is not practical due to significant heterogeneity between the studies^[31,32].

RESULTS

Study selection process and study characteristics

After searching 9 different databases, 1,021 studies were identified from PubMed (n = 454), Cochrane (n = 22), CINAHL (n = 54), Scopus (n = 20), Embase (n = 58), Web of Science (n = 10), medRxiv (n = 312), Research Square (n = 37), and Google Scholar (n = 54). From the preliminary review, 20 duplicates, 626 unrelated, and 12 unretrieved studies were excluded, leaving 363 studies were moved for title and abstract screening. As per the inclusion and exclusion conditions in the eligibility criteria, 354 studies were excluded for the following reasons: (1) Irrelevant sample population and studies (n = 89); (2) mathematical modelling, systematic reviews, and case studies (n = 47); (3) no matched control group (n = 7); (4) studied different vaccine and variant of concern (n = 168); (4) did not have sub-analysis for elderly population (n = 43); and (5) overlap in study population (n = 1). As a result, only 8 studies were included for systematic review^[33–40]. PRISMA Flow Diagram summarized the literature selection process (Figure 1).

Among these studies, 3 were published and 5 were published on the pre–print platforms^[33–40]. All of the 8 studies used observational study designs such as cohort, case control, and cross-sectional studies^[33–40]. These studies reported the effectiveness of Pfizer-BioNTech (n = 8) and Moderna (n = 4) vaccines, with 7 studies examining 2 doses, 2 studies examining 2nd and booster doses, and 1 study examining booster dose in reducing COVID-19 morbidity and mortality during the prevalence of Delta (B.1.617.2) and Omicron (B.1.1.529/BA) variants^[33–40]. Study locations were in New York, Finland, Canada, Costa Rica, Qatar, Greece, and Brazil published between 2021 (n = 3) and 2022 (n = 5)^[33–40]. The studies compared an estimated total sample size of 8,740,562 vaccinated elderly people and 9,658,245 unvaccinated elderly cohorts from an estimated total study population of 26,535,692 which evaluated the effectiveness of mRNA vaccines of an adult population including elderly cohorts who were 50 years old and older^[33–40]. Although the goal of the study is to focus on elderly subjects who were 60 years old and older, some of the selected studies in this review, grouped the elderly subjects from 50 years old to include 60 years old and older subjects^[35–38]. The largest sample size of vaccinated elderly people was 3,479,102 and 813,8482 unvaccinated elderly cohorts while the smallest sample size of vaccinated and unvaccinated elderly people was 45,345 and 1,272, respectively^[34,35,39]. The outcome measures used by the selected studies defined morbidity as infection (n = 5), hospitalization (n = 6), admission to ICU and intubation (n = 2) and mortality or death (n = 4) as outcome measurements^[33–40]. The characteristics of included studies are shown in Tables 1 and 2.

Table 1.

Characteristic of studies included for vaccine effectiveness

Ref.	Study design	Location	Purpose	Age of study group	Vaccine type	Number of doses received	Variant of concern
Baum et al[33], 2022	Cohort study	Finland	To estimate VE against severe COVID-19 among the elderly	Adult Population including ≥ 70 yr old	Pfizer-BioNTech	2 doses	Omicron
Grewal et al[34], 2022	Case control design	Canada	To estimate vaccine effectiveness of mRNA vaccines among aged ≥ 60 yr who were tested for SARS-CoV-2	≥ 60 yr old	Pfizer-BioNTech and Moderna	Booster	Omicron
Rosenberg et al[35], 2021	Cohort study	USA (NY)	To describe vaccine efficacy in NY	Adult Population including ≥ 50 yr old	Pfizer-BioNTech and Moderna	2 doses	Delta
Rosero-Bixby[36], 2021	Cross-sectional study	Costa Rica	To estimate the dose-dependent effectiveness of coronavirus disease (COVID-19) vaccines to prevent severe illness in real-world conditions	Adult Population including ≥ 58 yr old	Pfizer-BioNTech	2 doses	Delta
Rane et al[37], 2022	Case control study	USA (NY)	To monitor changes in vaccine effectiveness against COVID-19 outcomes for various vaccine products in different population subgroups	Adult Population including ≥ 50 yr old	Pfizer-BioNTech	2 doses	Delta
Chemaitelly et al[38], 2021	Case control study	Qatar	To estimate vaccine effectiveness against any SARS-CoV-2 infection and against any severe, critical, or fatal case of COVID-19	Adult Population including ≥ 50 yr old	Pfizer-BioNTech	2 doses	Delta
Lytras et al[39], 2022	Cohort study	Greece	To estimate COVID-19 effectiveness against disease and death	Adult Population including ≥ 60 yr old	Pfizer-BioNTech and Moderna	2 doses and booster	Delta
Ranzani et al[40], 2022	Case control study	Brazil	To evaluate vaccine effectiveness against symptomatic COVID-19 and severe COVID-19 (hospital admission or deaths)	Adult Population including ≥ 70 yr old	Pfizer-BioNTech	2 doses and booster	Omicron

COVID-19: Coronavirus disease 2019; SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2; USA (NY): United States of America (New York); VE: Vaccine effectiveness.

Table 2.

Characteristic of participants included for vaccine effectiveness

Ref.	Study population (n)	Age of study population	mRNA vaccinated elderly participants with 2 doses	mRNA vaccinated participants with 2 doses according to vaccine type		mRNA vaccinated elderly participants with 2 doses according to age		Unvaccinated elderly participants	Unvaccinated elderly participants according to age
				Pfizer-BioNTech	Moderna	Pfizer-BioNTech	Moderna
Baum et al[33], 2022	897932	Adult population including ≥ 70 yr old	241630	-	-	70–79 yr old: 171816; 80–89 yr old: 57024; 90–115 yr old: 12790		747486	70–79 yr old: 480532; 80–89 yr old: 223267; 90–115 yr old: 43687
Grewal et al[34], 2022	46849	≥ 60 yr old	45345	-	-	-	-	1272	-
Rosenberg et al[35], 2021	8834604	Adult population including ≥ 50 yr old	3479102	-	-	50–64 yr old: 846664; ≥ 65 yr old: 984464	50–64 yr old: 624226; ≥ 65 yr old: 1023748	976536	50–64 yr old: 606411; ≥ 65 yr old: 370125
Rosero-Bixby[36], 2021	3670000	Adult population including ≥ 58 yr old	741474	-	-	741474		58887	58887
Rane et al[37], 2022	1058493	Adult population including ≥ 50 yr old	143104	-	-	50–59 yr old: 64806; 60–69 yr old: 48260; 70–79 yr old: 22997; ≥ 80 yr old: 7041		27362	50–59 yr old: 14936; 60–69 yr old: 8352; 70–79 yr old: 3066; ≥ 80 yr old: 1008
Chemaitelly et al[38], 2021	-	Adult population including ≥ 50 yr old	1402622	907763	494859	-	-	8043	50–59 yr old: 6350; 60–69 yr old: 1326; ≥ 70 yr old: 367
Lytras et al[39], 2022	9200000	Adult population including ≥ 60 yr old Adult	2380402	2128913	251492	-		-	8138482
Ranzani et al[40], 2022	1417149	population including ≥ 60 yr old	306883	-	-	60–79 yr old: 258306; ≥ 80 yr old: 48577		306,588	60–79 yr old: 265073; ≥ 80 yr old: 41515

Risk of bias

The risk of bias was evaluated by following ROBINS-I tool (risk of bias in non-randomized studies of interventions)^[31]. All of the 8 observational studies were rated to have moderate risk of bias mainly due to lack of control for confounders such as comorbidities or socioeconomic status like age and occupation, outbreak data such as location and time of test, and other risk-taking behavior modification^[33–40]. Due to the dependence in surveillance data which were subject to incomplete information, 5 studies received a moderate risk of bias score because of missing data, while 3 studies due to misclassification of measurement of outcomes, were rated with moderate bias^[35–40]. Table 3 shows the results of ROBINS-I risk of bias assessment of observational studies.

Table 3.

ROBINS-I risk of bias assessment of observational studies

Ref.	Confounding	Selection of participants	Classification of interventions	Deviations from interventions	Missing data	Measurement of outcomes	Reported result	Overall bias
Baum et al[33], 2022	Moderate	Low	Moderate	Moderate	Low	Low	Low	Moderate
Grewal et al[34], 20223	Moderate	Moderate	Low	Low	Low	Low	Low	Moderate
Rosenberg et al[35], 2021	Moderate	Moderate	Low	Low	Moderate	Low	Low	Moderate
Rosero-Bixby[36], 2021	Moderate	Low	Low	Low	Moderate	Moderate	Low	Moderate
Rane et al[37], 2022	Low	Low	Low	Low	Moderate	Low	Low	Low
Chemaitelly et al[38], 2021	Moderate	Low	Low	Low	Moderate	Low	Low	Moderate
Lytras et al[39], 2022	Moderate	Low	Low	Low	Moderate	Moderate	Low	Moderate
Ranzani et al[40], 2022	Low	Low	Low	Low	Low	Moderate	Low	Moderate

Low risk: Comparable to a well-performed randomised trial.

Moderate risk: Sound for a non-randomised study but cannot be compared to a well performed randomised trial.

Serious risk: Study has some important problems.

Critical risk: Study is too problematic to provide any useful evidence.

Synthesis of results

Vaccine effectiveness against infection:

5 of the 8 studies (36%) reported the effectiveness of vaccines using infection as an outcome measure^{[34,35,37,38,40]}. Among these, 2 of the studies used booster dose to evaluate vaccine effectiveness against asymptomatic and symptomatic infections while 3 studies assessed the effectiveness of 2 doses of vaccines^{[34,35,37,38,40]}. The findings from these studies revealed that 2 doses of mRNA vaccines offer 83%–89% protection against infection, while other studies revealed vaccine’s protection level against infection at 40%–63%, marginally lower for 65 years old when compared to Moderna, Pfizer-BioNTech vaccine was reported to have slightly lower efficacy against infections with indications of declining protection over a period of time^{[34,35,37,38,40]}.

Vaccine effectiveness against hospitalization:

3 of the 8 studies (21%) reported the effectiveness of vaccines using hospitalization as an outcome measure which demonstrated 92% efficacy for older people^[33,35,36]. Similarly, when compared to Moderna, Pfizer-BioNTech vaccine have lower marginal protection against hospitalization with indication of waning effectiveness against hospitalization, 6 months after the 2^nd dose^[33,35].

Vaccine effectiveness against ICU admission and intubation:

2 out of the 8 studies (14%) reported the effectiveness of vaccines using admission to ICU and intubation as outcome measures^[33,39]. The study on ICU admissions revealed that Pfizer-BioNTech vaccine’s protection waned from 98% down to 85% after 6 months among 70 years old and older^[33]. Similar findings was observed when intubation was used as an outcome measure, which revealed diminished vaccine effectiveness from 96.9% down to 86%, in 6 months among 60 years old and older populations but was restored at 97.6% by booster dose^[39].

Vaccine effectiveness against death:

4 out of the 8 studies (29%) reported the effectiveness of vaccines using death with hospitalization as outcome measures^[34,38–40]. The findings showed that although 2 doses of mRNA vaccine can prevent death, it offers a marginally limited protection against death among 75 years old and older with indications of diminishing protection which was only restored by a booster dose^[38–40]. Additionally, the finding showed that Pfizer-BioNTech has marginally higher protection level against death at 87% when compared to Moderna at 77%^[34]. The outcomes of included studies for vaccine effectiveness are shown in Table 4.

Table 4.

Outcomes of studies included for vaccine effectiveness

Ref.	Outcome measurements					Vaccine effectiveness
	Infection (n/%)	Hospitalization (n/%)	ICU admission (n/%)	Intubation (n/%)	Death (n/%)
Baum et al[33], 2022	-	2 doses of Pfizer (within 3 mo): 30/5.64%; 2 doses of Pfizer (within 6 mo): 193/36.28%; 2 doses of Pfizer (≥ 6 mo): 148/27.82%; 3 doses of Pfizer (within 3 mo): 95/17.86%; 3 doses of Pfizer (≥ 6 mo): 66/12.41%	2 doses of Pfizer (within 3 mo): 5/8.33%; 2 doses of Pfizer (within 6 mo): 24/40%; 2 doses of Pfizer (≥ 6 mo): 14/23.33%; 3 doses of Pfizer (within 3 mo): 9/15%; 3 doses of Pfizer (≥ 6 mo): 8/13.33%	-	-	Hospitalization: 2 doses of Pfizer (within 3 mo): 90%; 2 doses of Pfizer (within 6 mo): 85%; 2 doses of Pfizer (≥ 6 mo): 72%; 3 doses of Pfizer (within 3 mo): 95%; 3 doses of Pfizer (≥ 6 mo): 88%; ICU Admission: 2 doses of Pfizer (within 3 mo): 98%; 2 doses of Pfizer (within 6 mo): 95%; 2 doses of Pfizer (≥ 6 mo): 82%; 3 doses of Pfizer (within 3 mo): 98%; 3 doses of Pfizer (≥ 6 mo): 85%
Grewal et al[34], 2022	Infection: 3 doses of Pfizer: 2691/42.43%; 3 doses of Moderna: 3651/57.57%; Symptomatic Infection: 3 doses of Pfizer: 395/38.69%; 3 doses of Moderna: 626/61.31%	3 doses of Pfizer: 214/58.79%; 3 doses of Moderna: 150/41.21%	-	-	3 doses of Pfizer: 214/58.79%; 3 doses of Moderna: 150/41.21%	Infection: 3 doses of Pfizer: 31%; 3 doses of Moderna: 51%; Symptomatic Infection: 3 doses of Pfizer: 61%; 3 doses of Moderna: 73%; Hospitalization or Death: 3 doses of Pfizer: 87%; 3 doses of Moderna: 77%
Rosenberg et al[35], 2021	≥ 65 yr old (2 doses of Pfizer): 5302/61.70%; ≥ 65 yr old (2 doses of Moderna): 3291/38.30%	≥ 65 yr old (2 doses of Pfizer): 972/64.07%; ≥ 65 yr old (2 doses of Moderna): 545/35.93%	-	-	-	Infection: ≥ 65 yr old (2 doses of Pfizer): 83.0%; ≥ 65 yr old (2 doses of Moderna): 89.2%; Hospitalization: ≥ 65 yr old (2 doses of Pfizer): 91.9%; ≥ 65 yr old (2 doses of Moderna): 95.7%
Rosero-Bixby[36], 2021		40–57 yr old: 37/0.006%; ≥ 58 yr old: 65/0.009%	-	-	-	Hospitalization: 40–57 yr old: 94%; ≥ 58 yr old: 92%
Rane et al[37], 2022	60–69 yr old: 3232/5.8%; 70–79 yr old: 1221/2.2%; ≥ 80 yr old: 468/0.8%	-	-	-	-	Infection: 51–64 yr old: 60%; 65 80 yr old: 55%; ≥ 80 yr old: 51%
Chemaitelly et al[38], 2021	2 doses of Pfizer (within 1 mo): 2915/2.51%; 2 doses of Pfizer (within 2 mo): 1450/1.28%; 2 doses of Pfizer (within 3 mo): 800/0.71%; 2 doses of Pfizer (within 4 mo): 492/0.44%; 2 doses of Pfizer (within 5 mo): 548/0.49%; 2 doses of Pfizer (within 6 mo): 460/0.41%; 2 doses of Pfizer (≥ 7 mo): 135/0.12%	2 doses of Pfizer (within 1 mo): 32/0.78%; 2 doses of Pfizer (within 2 mo): 23/0.56%; 2 doses of Pfizer (within 3 mo): 17/0.42%; 2 doses of Pfizer (within 4 mo): 10/0.25%; 2 doses of Pfizer (within 5 mo): 0/0; 2 doses of Pfizer (within 6 mo): 8/0.20%; 2 doses of Pfizer (≥ 7 mo): 6/0.15%			2 doses of Pfizer (within 1 mo): 32/0.78%; 2 doses of Pfizer (within 2 mo): 23/0.56%; 2 doses of Pfizer (within 3 mo): 17/0.42%; 2 doses of Pfizer (within 4 mo): 10/0.25%; 2 doses of Pfizer (within 5 mo): 0/0; 2 doses of Pfizer (within 6 mo): 8/0.20%; 2 doses of Pfizer (≥ 7 mo): 6/0.15%	Infection: 2 doses of Pfizer (within 1 mo): 75.8%; 2 doses of Pfizer (within 2 mo): 69.7%; 2 doses of Pfizer (within 3 mo): 63.7%; 2 doses of Pfizer (within 4 mo): 39.1%; 2 doses of Pfizer (within 5 mo): 11.4%; 2 doses of Pfizer (within 6 mo): 9.2%; 2 doses of Pfizer (≥ 7 mo): −4.4%; Hospitalization and Death: 2 doses of Pfizer (within 1 mo): 95.9%; 2 doses of Pfizer (within 2 mo): 96.3%; 2 doses of Pfizer (within 3 mo): 93.4%; 2 doses of Pfizer (within 4 mo): 80.8%; 2 doses of Pfizer (within 5 mo): 100%; 2 doses of Pfizer (within 6 mo): 81.8%; 2 doses of Pfizer (≥ 7 mo): 44.1%
Lytras et al[39], 2022				2 doses of Pfizer: 548/90.28%; 2 doses of Moderna: 35/5.77%; 3 doses of Pfizer: 24/0.04%	2 doses of Pfizer: 1629/64.60%; 2 doses of Moderna: 42/2.44%; 3 doses of Pfizer: 51/5.96%	Intubation: 60–79 yr old (2 doses): 96.9%; ≥ 80 yr old (2 doses): 94.4%; ≥ 80 yr old (2 doses, within 6 mo): 86.0%; ≥ 80 yr old (3 doses): 97.6%; Death: 60–79 yr old (2 doses): 94.6%; ≥ 80 yr old (2 doses): 91.0%; ≥ 80 yr old (2 doses, within 6 mo): 84.1%; ≥ 80 yr old (3 doses): 98.4%
Ranzani et al[40], 2022	Symptomatic Infection: 60–74 yr old (2 doses): 34077/86.93%; ≥ 75 yr old (2 doses): 15539/79.84%; 60–74 yr old (3 doses, within 2 mo): 15053/64%; 60–74 yr old (3 doses, after 2 mo): 3273/66.36%; ≥ 75 yr old (3 doses, within 2 mo): 116955/89.86%; ≥ 75 yr old (3 doses, after 2 mo): 64495/99.36%				60–74 yr old (2 doses): 2035/63.3%; ≥ 75 yr old (2 doses): 2750/52.81%; 60–74 yr old (3 doses, within 2 mo): 50/68.50%; 60–74 yr old (3 doses, after 2 mo): 51/96.01%; ≥ 75 yr old (3 doses, within 2 mo): 511/90.41%; ≥ 75 yr old (3 doses, after 2 mo): 2964/72.40%	Infection: 60–74 yr old (2 doses): 63.4%; ≥ 75 yr old (2 doses): 40.7%; 60–74 yr old (3 doses, within 2 mo): 88.4%; 60–74 yr old (3 doses, after 2 mo): 90.4%; ≥ 75 yr old (3 doses, within 2 mo): 77.3%; ≥ 75 yr old (3 doses, after 2 mo): 78.5%; Hospitalization or death: 60–74 yr old (2 doses): 63.4%; ≥ 75 yr old (2 doses): 40.7%; 60–74 yr old (3 doses, within 2 mo): 88.4%; 60–74 yr old (3 doses, after 2 mo): 90.4%; ≥ 75 yr old (3 doses, within 2 mo): 77.3%; ≥ 75 yr old (3 doses, after 2 mo): 78.5%

DISCUSSION

While clinical trial data on Pfizer-BioNTech and Moderna vaccines demonstrated 94% effectiveness among the elderly, the results in this study showed that the effectiveness of mRNA vaccines in real-world settings is marginally lower against COVID-19 infection, hospitalization, ICU admission and intubation, and deaths during the predominance of Delta and Omicron variants^[33–40].

The results in this systematic review further strengthen and supplement the increasing evidence on the real-world effectiveness of mRNA vaccines. While the inclusion and exclusion criteria of this review limits a variety of similar studies in the data analysis, for discussion purposes these studies echoed similar findings. A study conducted in United Kingdom revealed that vaccine effectiveness for ≥ 60 years old is 42.3%^[41]. The same observed pattern is reported for ≥ 75 years old in a study conducted in Israel, in a case-control study conducted among US military personnel and in a test-negative design study conducted in Malaysia^[42–44]. Using random-effects model on 15 observational studies to estimate the pooled vaccine effectiveness (VE) with 95% confidence intervals for each vaccine type against each variant, the systematic review and meta-analysis conducted by Zhang et al^[45] revealed a limited vaccine effectiveness among ≥ 65 years old. The result in this study also align with the result in our study and with the findings of other studies focusing on vaccine effectiveness in the elderly during the predominance of Delta and Omicron variants^[46–48]. Utilizing the same research model, a contrasting result was reported by Li et al^[49] when they evaluated the effectiveness of vaccine in over 30,000 participants aged 60 years and older. This systematic review and meta-analysis however, largely focused on randomized controlled trials which may have skewed the outcomes. Given that clinical trials on COVID-19 vaccines are conducted under controlled clinical conditions from volunteer subjects of targeted age groups, these studies are not able to take into account the abilities of COVID-19 to mutate and evade the vaccine^{[46,47,50–53]}. Therefore, the tangible effect of vaccines can be substantially different from the real-world which may not necessarily illustrate the authentic effectiveness of vaccines. Furthermore, although interventional studies such as clinical trials are more methodologically sound, observational studies are more reliable since they produce practical and realistic results that are grounded from real-world experiences.

By focusing on Delta and Omicron variants, we hypothesize that much of the previous research on vaccine effectiveness only included earlier variants which may have skewed the results for newer and more dominant variants like Delta and Omicron. We also aim to provide value in understanding the effectiveness of mRNA vaccines by comparing their effectiveness in real-world settings. While the results in this study reported a marginal difference in effectiveness between Moderna and Pfizer-BioNTech vaccines, the minor difference on an absolute scale can be significant when considering world-wide population for vaccination^[54].

Furthermore, the observed waning effectiveness of vaccine in this study supports the findings of other studies which suggested that the diminishing effectiveness of vaccine is due to the extensive abilities of COVID-19 virus to evolve and generate new variants which allow them to avoid the effects of the vaccines^[51–53]. The ability of Delta and Omicron variants to elude sensitivity to antibody neutralization was observed to decline over time making the vaccine less effective^[41,55–58]. Consistent to the findings of this study, this imply that 2 doses of mRNA vaccines is inadequate and only provides interim protection against COVID-19 infection, hospitalization, ICU admission and intubation, and deaths^[58–61]. Because of the vaccine’s natural diminishing effectiveness, the importance of booster dose to restore its efficacy is vital in providing additional protection against emerging variants^{[33,34,39,40,49,62–64]}. This positions is in line with the study conducted in an elderly long-term care facility where the effectiveness of vaccine was observed to only have improved after the second dose, along with other studies pointing out that booster dose can provide significant protection and is the most effective approach to COVID-19 prevention^[59,63,65].

Strengths and limitations

This study provides useful information on the effectiveness of mRNA vaccines in the real-world settings which are not under a regulated condition of clinical trials. Specifically, the strengths of this study made use of an inexpensive design that is reproducible since it rests on an organized search strategy, strong procedure with the inclusion of literature from pre-print servers. It also included a broad range of possible outcome measures to include as many studies as possible that can provide relevant information on the topic. Furthermore, this systematic review included research studies from different parts of the world that have relatively large representatives of elderly population with longer follow up which is useful in minimizing selection bias.

The findings of this study should be cautiously interpreted due to certain limitations. First, the included literature were observational studies which have restrictions in statistical power. Second, since there is limitation to access the data used by the research studies, this study has a risk in information bias. Third, due to the missing data and estimation of some studies, a degree of misclassification further delimits this study. Fourth, in exchange of large sample sizes, this study sits on a potential bias of unmeasured confounder such as comorbidities or socioeconomic status like age and occupation, outbreak data such as location and time of test, and other risk-taking behavior modification. Finally, the use of heterogenous outcome measures creates limitation due to potential classification errors.

CONCLUSION

As a response to the rapidly evolving COVID-19 outbreak, many research studies were organized and carried out resulting in highly heterogeneous outcome measurements. From a research perspective, this heterogeneity inhibits the comparison, contrast, and integration of the results which makes data pooling across different studies problematic. Therefore, this systematic review suggests that, while pharmaceutical intervention like vaccination is important to fight an epidemic, utilizing common outcome measurements or carrying out studies with minimal heterogeneity in outcome measurements, is equally crucial to better understand and respond to an international health crisis. Notwithstanding these limitations, the consistent findings of this review indicated waning of vaccine effectiveness over time, implying that a large proportion of the vaccinated population, particularly the elderly, may lose protection unless booster doses are rolled out to restore the effectiveness of the vaccine (Supplemental Table 1).

ARTICLE HIGHLIGHTS.

Research background

Although there has been a rise in the administration of vaccinations and booster shots, coronavirus disease 2019 (COVID-19) infections and fatalities continue to persist at a significant level. The effectiveness of these vaccines has been confirmed by multiple manufacturers and were rapidly developed for emergency use with testing conducted in controlled clinical conditions and on voluntary participants, whose characteristics may differ from those of the broader population.

Research motivation

The COVID-19 pandemic has caused an unprecedented global health crisis resulting in millions of deaths and infections worldwide. Despite the availability of COVID-19 vaccines and the administration of booster shots, the number of cases and deaths remains high. The development and clinical trials of these vaccines were conducted in controlled environments with volunteers which may not fully represent the general population. Therefore, there is a need to determine the real-world effectiveness of mRNA COVID-19 vaccines in the elderly during the predominance of Delta and Omicron variants in preventing COVID-19-related infections, hospitalizations, intensive care unit (ICU) admission and intubation, and death.

Research objectives

This study aimed to conduct a systematic review of available research articles to evaluate the effectiveness of Pfizer-BioNTech and Moderna vaccines on the elderly using infection, hospitalization, ICU admission and intubation, and death as outcome measures.

Research methods

The study utilized a combination of Medical Subject Headings (MeSH) and non-MeSH to identify relevant research articles from various databases and pre-print servers.

Research results

While clinical trial data on Pfizer-BioNTech and Moderna vaccines demonstrated high vaccine effectiveness in the elderly, the results of this study showed that vaccine effectiveness in real-world settings is marginally lower against infection, hospitalization, ICU admission and intubation, and death, with an indication of diminished effectiveness of the vaccine over time. Furthermore, 2 doses of mRNA vaccines are inadequate and only provide interim protection, emphasizing the need for booster doses to restore its efficacy.

Research conclusions

Continued monitoring and research to improve the effectiveness of vaccines and combat the virus effectively is important to evaluate vaccine efficacy in real-world settings, especially as new variants emerge. In addition, the use of highly heterogeneous outcome measures poses a challenge in comparing and integrating the results, and standardized outcome measures or minimal heterogeneity in outcome measurements are essential to better understand and respond to a global health crisis.

Research perspectives

Future research should continue to evaluate the real-world effectiveness of COVID-19 vaccines, including the efficacy of booster shots and the effectiveness of vaccines against new variants. Additionally, efforts should be made to standardize outcome measures to enable better comparisons across studies and facilitate the integration of findings. Ultimately, such research will be crucial in guiding public health policies and interventions aimed at controlling the spread of COVID-19 and in mitigating its impact on public health.