Assessing the real-world effectiveness of five SARS-CoV-2 vaccines in a cohort of Mexican pensioners: a nationwide nested test-negative design study

Summary

Background

Despite the extensive distribution of COVID-19 vaccines across Latin America, research on their real-world performance remains limited. We aimed to evaluate the effectiveness of five vaccines (BNT162b2, AZD1222, CoronaVac, Gam-COVID-Vac, and Ad5-nCoV) in a cohort of 2,559,792 pensioners covered by the Mexican Institute of Social Security.

Methods

We conducted a nested test-negative design study on 28,271 individuals tested for SARS-CoV-2 infection between April and November 2021, accounting for 29,226 separate episodes. We used mixed-effects logistic regression models to estimate the vaccine effectiveness (VE) in fully vaccinated individuals for symptomatic infection, hospitalization, severe disease, and death.

Findings

The median age of the study population was 70 years (interquartile range 65–76) and 76.4% (21,598/28,271) were male. VE rates were 56.3%, 75.3%, 79.7%, and 79.8% against symptomatic infection (95% confidence interval [CI]: 53.5–59.0), hospitalization (95% CI: 73.4–77.0), severe disease (95% CI: 78.0–81.3), and death (95% CI: 78.1–81.4), respectively. When evaluating vaccines individually, all showed moderate to high VE, with the best being BNT162b2 (symptomatic infection, 69.8%, 95% CI: 67.3–72.0; hospitalization, 84.1%, 95% CI: 82.5–85.6; severe disease, 88.2%, 95% CI: 86.7–89.5; and death, 88.3%, 95% CI: 86.9–89.6) and Gam-COVID-Vac (symptomatic infection, 70.0%, 95% CI: 64.8–74.4; hospitalization, 86.8%, 95% CI: 83.7–89.3; severe disease, 91.9%, 95% CI: 89.4–93.9; and death, 92.0%, 95% CI: 89.5–93.9).

Interpretation

All five SARS-CoV-2 vaccines available for this population showed moderate to high levels of protection against COVID-19 and its progression to severe outcomes.

Funding

Fundación IMSS, México.

Research in context.

Evidence before this study

Despite the widespread distribution of COVID-19 vaccines across Latin America, research on their real-world performance remains limited. We searched MEDLINE (via Ovid), Embase (via Ovid), and Web of Science for studies evaluating the effectiveness of COVID-19 vaccines in Mexico. We limited our search to studies published between December 24, 2021 (when COVID-19 vaccination began in Mexico) and December 31, 2022, with no language restrictions. We used two different search strategies: (1) (“COVID-19” OR “SARS-CoV-2” OR “coronavirus”) AND (“vaccine effectiveness” OR “vaccine efficacy”) AND (“Mexico”), and (2) (“COVID-19” OR “SARS-CoV-2” OR “coronavirus”) AND (“efectividad” OR “eficacia”) AND (“Mexico”). Our search identified only three relevant studies, all of which found that COVID-19 vaccines were effective against SARS-CoV-2 infection, hospitalization, and death. However, these studies either focused on specific subpopulations, such as non-essential workers or individuals hospitalized because of COVID-19, or relied exclusively on self-reported vaccination information, which may compromise the accuracy of critical details, such as immunization dates. There is an urgent need for accurate evaluations of the real-world effectiveness of COVID-19 vaccines, as this information is critical for guiding public health policy, building confidence in vaccination, and informing future research and development.

Added value of this study

The main aim of this study was to evaluate the effectiveness of five SARS-CoV-2 vaccines (Pfizer-BioNTech's BNT162b2, Gamaleya Research Institute's Sputnik V Gam-COVID-Vac, Oxford-AstraZeneca's AZD1222, Sinovac's CoronaVac, and CanSino's Ad5-nCoV) in a cohort of 2,559,792 pensioners covered by the Mexican Institute of Social Security (IMSS), using the most reliable sources of information available in Mexico. We conducted a test-negative design study, nested within the cohort of IMSS pensioners, in which we included all symptomatic individuals who were tested for SARS-CoV-2 infection between April and November 2021. According to our results, all five COVID-19 vaccines available for this population provided moderate to high levels of protection against COVID-19 and its progression to severe outcomes.

Implications of all the available evidence

Our research confirms findings from previous studies conducted in Mexico and underscores the importance of vaccinating all individuals, especially those who are partially vaccinated or hesitant. Importantly, our estimates of vaccine effectiveness are still lower than those reported in previous randomized clinical trials and observational studies involving older adults in high-income settings. A possible explanation for this discrepancy could be the presence of biases, such as misclassification bias, due to incorrect or missing records in vaccine registries. However, it is important to emphasize that our estimates are consistent with results from studies of older adults in other Latin American countries, including Mexico. Factors contributing to differences in vaccine effectiveness between high-income and low- and middle-income countries (LMICs) may include differences in demand dynamics and vaccine availability, and the inherent challenges associated with vaccine storage, cold chain management, and distribution, which may be more pronounced in LMICs than in high-income settings. Further research is needed to identify the drivers of variation in COVID-19 vaccine effectiveness across regions, as this information could guide future policy decisions. Similarly, additional studies are urgently needed to assess the effectiveness of booster doses in Mexico, especially as new variants of concern, such as Omicron, continue to emerge.

Introduction

Globally, over 661 million confirmed cases of coronavirus disease 2019 (COVID-19) and nearly 6.7 million related deaths have been reported since the first patient was identified in Wuhan, China.¹ A key strategy in the fight against COVID-19 has been vaccination. According to recent estimates, vaccines alone prevented around 14.4 million deaths worldwide (95% credible interval: 13.7–15.9) between December 8, 2020 and December 8, 2021.² To enhance coverage and accessibility, various technologies were used to develop over 176 vaccine products against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection at an unprecedented speed.³ However, the certainty of the evidence supporting their use varies widely, with most evaluations involving pre-licensure randomized controlled trials (RCTs).^3,4

Despite their crucial role in pandemic response, vaccination programs have had differing implementation (and possibly effectiveness). In Mexico, the vaccination strategy was developed and executed by the Ministries of Foreign Affairs (procurement), Treasury (financing), Welfare (logistic and registry), and Health (store, distribution, cold chain, application and adverse event surveillance) in collaboration with local governments. More than 15 vaccines received emergency use authorization; the most widely used were BNT162b2 (Pfizer–BioNTech; United States/Germany), AZD1222 (University of Oxford–AstraZeneca; England), Gam-COVID-Vac (also known as “Sputnik V”; Gamaleya National Research Institute of Epidemiology and Microbiology; Russia), CoronaVac (Sinovac; China), and Ad5-nCoV (CanSino; China).⁵ Since the beginning, vaccines were provided at no individual cost to everyone through a national rollout that prioritized those at the highest risk for infection and death.^6,7 Allocation began with healthcare workers and followed the general population, targeting those aged 60 and older first. Eligibility gradually expanded by decreasing 10-year age cohorts and also included pregnant women, schoolteachers, residents in some border cities, adolescents, and children. Program rollout was overseen by the armed forces and personnel from the Secretariat of Welfare, which verified eligibility, registered participants, and provided accountability for all vaccines, and by the state public health departments, which provided trained health personnel, secured a cold chain, and monitored for adverse reactions.^6,7 From January 2021 to February 2023, nearly 225 million vaccine doses were administered.⁸

Although COVID-19 vaccines have been widely distributed across Latin America, research on their real-world effectiveness remains limited.9, 10, 11, 12 It is crucial to understand how vaccines perform outside of RCTs, since these studies might not provide a complete picture of their effects under real-world conditions. For example, factors such as low participation rates from low- and middle-income countries (LMICs) or certain minority groups, including the elderly population,¹³ along with nonclinical factors, such as challenges related to vaccine supply, storage, demand dynamics, distribution logistics, and cold chain management, may lead to lower vaccine effectiveness (VE).^11,13 Acquiring a deeper understanding of real-world VE is especially important in Latin America, given that the pandemic was particularly severe there.¹⁴ In Mexico, only a few studies have evaluated VE, and these studies have either focused on specific populations, such as non-essential workers^15,16 or individuals hospitalized because of COVID-19,¹⁷ or relied exclusively on self-reported vaccination information, potentially limiting the accuracy of critical details, such as immunization dates.⁹

Accurate information on real-world VE is needed to guide public health decisions, build public trust in vaccinations, and guide research and development. We aimed to address this gap by assessing the real-world VE of the five most common vaccines in a cohort of 2,559,792 pensioners covered by the Mexican Institute of Social Security (IMSS, Instituto Mexicano de Seguro Social).

Methods

Study design, setting, and participants

We conducted a nested test-negative design (TND) study¹⁸ using a sample of 28,271 IMSS pensioners tested for SARS-CoV-2 infection between April 1, 2021 (when the second dose of the vaccine was introduced for this population) and November 30, 2021 (when the booster was introduced for this population), accounting for 29,226 separate episodes. Eligibility was defined as any individual recognized as a pensioner by the IMSS and included in its January pensioners’ payroll list (for more detailed definitions, please refer to Supplementary Table S1). IMSS is a tripartite entity to which workers, private sector employers, and the federal government contribute. By law, IMSS provides five types of insurance: Work Risks, Illness and Maternity, Disability and Life, Retirement or Unemployment at an Advanced Age and Old Age, and Nurseries and Social Benefits. Through its medical system, IMSS also provides medical services to workers in the formal sector and their families and workers who made their contributions and were entitled to obtain a pension upon fulfilling the legal conditions.¹⁹

We first identified all episodes in which a pensioner was evaluated for COVID-19–like symptoms (Supplementary Table S1) during the study period and received a valid SARS-CoV-2 diagnostic test at an IMSS medical facility, defined as either a nucleic acid amplification test (NAAT) or a rapid antigen diagnostic test (RADT) taken no more than two days before or up to 14 days after symptom onset. Individuals were included multiple times (episodes), provided their symptom onset dates were separated by at least eight days. The exposure of interest was receiving a full COVID-19 vaccination schedule, defined as either one dose of Ad5-nCoV or two doses of BNT162b2, AZD1222, CoronaVac, or Gam-COVID-Vac, with at least 15 days having passed since completion. Therefore, we excluded partially vaccinated individuals and those whose symptom onset occurred within 14 days of completing vaccination.

The outcomes assessed included symptomatic infection and its progression to hospitalization, severe disease (defined as either admission to the intensive care unit or death), and death alone (Supplementary Table S1). Episodes with a positive SARS-CoV-2 test result (either NAAT or RADT), were classified as test-positive cases; episodes with only negative test results were considered test-negative controls. In other words, to analyze VE against symptomatic infection, symptomatic individuals who tested positive were classified as cases, and those testing negative were controls. The same classification procedure was applied to the analyses of VE against hospitalization, severe disease, and death. Finally, VE was estimated by comparing the odds of vaccination between cases and controls (see statistical methods for further details on the analyses).

Data sources

The study dataset was compiled by linking the following five administrative and clinical data sources (linkage methods are described below).

• 1)The register of IMSS retirees and pensioners, which contains demographic information for 2,559,792 individuals (the target population), including age, sex, location, and the monthly pension amount received.
• 2)The IMSS Unified Epidemiological Surveillance System (SINOLAVE; “Sistema Único de Información de Vigilancia Epidemiológica”), which has information from 5,529,656 episodes of medical evaluations due to suspected SARS-CoV-2 infection, occurring at multiple IMSS testing and healthcare facilities between February 2020 and December 2021. It includes individual-level data on COVID-19–like symptoms, self-reported medical history (including comorbidities, vaccination status, and dates), SARS-CoV-2 diagnostic tests (including test type, dates, and results), and information on COVID-19–related hospitalizations and deaths within IMSS medical facilities.
• 3)The Mexican COVID-19 vaccination registry “Mi vacuna,”²⁰ which has data that include the type of vaccine administered, batch number, and dates of administration.
• 4)The Mexican Epidemiological and Statistical System of Deaths (SEED; “Sistema Epidemiológico y Estadístico de las Defunciones”), a comprehensive registry that gathers data from death certificates. This includes information on the causes of death, classified according to the Tenth Revision of the International Classification of Diseases (ICD-10).²¹ The data covers deaths occurring in all locations across Mexico.
• 5)The IMSS hospital discharge records database, which has data on hospital discharges occurring at any IMSS facility, including admission and discharge dates and diagnoses, classified by ICD-10; it identifies 6,781,821 discharges between February 2019 and December 2022.

We linked data using both probabilistic and deterministic methods. Data from the register of IMSS retirees and pensioners, Mi vacuna, the IMSS hospital discharge records database, and SEED were linked exclusively through deterministic methods, as two unique identifiers were available in these datasets: the Population Registry Identification Code (CURP, “Clave Única de Registro de Población”) and Social Security number. In contrast, SINOLAVE data were linked using a hierarchical approach. Initially, this database was linked deterministically with the other data sources using CURP. Then, to account for potential false negative matches from typographical errors in SINOLAVE, a probabilistic linkage algorithm was applied using the “fastLink” package (version 0.6.0)²² in R. Among the 29,226 separate episodes comprising the final dataset, 93.1% (27,208) were identified using deterministic methods and 6.9% (2018) using probabilistic methods. All data processing and linkage procedures were conducted in a secure location within IMSS facilities to ensure the privacy and security of sensitive data.

Statistical methods

In TND, assuming that the vaccine under study does not affect an individual's probability of presenting with another disease or infection that causes COVID-19–like symptoms, the case-status odds ratio (OR) comparing vaccinated and unvaccinated individuals provides an unbiased estimate of the conditional risk ratio for medically attended symptomatic COVID-19 (or its progression to hospitalization or death) comparing vaccinated and unvaccinated individuals.^18,23 Therefore, VE in TND studies can be estimated using the formula VE = [1–OR] x 100%.^18,23

We used mixed-effects logistic regression models to estimate the case-status OR and VE against medically attended symptomatic infection and its progression to hospitalization, severe disease, or death using the lme4 package (version 1.1.34)²⁴ in R. The models used case status and vaccination status as the outcome and explanatory variables, respectively. All models were adjusted for sex, standardized age modeled through splines, zone (geographical location), standardized monthly pension income, Charlson comorbidity index²⁵ and relevant comorbidities not included in the index (i.e., asthma, smoking status, obesity, hypertension). We included random intercepts for epidemiological weeks (grouped by four to prevent convergence issues).

To estimate the VE of being fully vaccinated with any of the five vaccines (BNT162b2, AZD1222, CoronaVac, Gam-COVID-Vac, and Ad5-nCoV), we used the entire study population as the analytical sample. However, to estimate VE for each vaccine product, we included both those who received the specific vaccine and unvaccinated individuals. To assess the duration of protection, we estimated VE by restricting the analysis to unvaccinated individuals and those vaccinated who had ≤90, ≥91 to ≤140, and ≥141 days between the completion of their vaccination schedule and symptom onset. These cutoffs were selected to provide an optimal division in terms of the number of episodes in the three strata. Last, we assessed whether sex modified the effect of vaccination on the four outcomes by conducting a sex-stratified analysis. Data on ethnicity were not available, which precluded an assessment of effect measure modification by this variable.

Given the minimal amount of missing data, all analyses were conducted using complete cases. The proportion of missing data for each variable was as follows: diabetes (0.36%), obesity (0.36%), hypertension (0.36%), chronic obstructive pulmonary disease (0.37%), asthma (0.37%), ever smoked (0.37%), and cancer (0.37%) (Table 1 and Supplementary Table S2). All statistical analyses were conducted using R (version 4.2.2, R Foundation for Statistical Computing, Vienna, Austria, 2022).

Table 1.

Baseline characteristics of the study population (April–November 2021), stratified by case status (test-positive cases and test-negative controls) and outcome of interest.

	Test-negative controls	Test-positives cases
		Symptomatic infection	Hospitalization	Severe Covid	Death
Sociodemographicinformation
Males, n/N (%)	12,540/17,014 (73.7)	9765/12,212 (80.0)	5356/6287 (85.2)	3678/4238 (86.8)	3649/4205 (86.8)
Age (median [IQR])	70 [65, 76]	70 [66, 76]	72 [67, 79]	73 [68, 79]	73 [68, 79]
Geographical locationa, n/N (%)
Centre	6176/17,014 (36.3)	3711/12,212 (30.4)	1723/6287 (27.4)	1133/4238 (26.7)	1120/4205 (26.6)
West-Centre	3516/17,014 (20.7)	2474/12,212 (20.3)	1416/6287 (22.5)	944/4238 (22.3)	940/4205 (22.4)
North	5006/17,014 (29.4)	3464/12,212 (28.4)	1958/6287 (31.1)	1280/4238 (30.2)	1270/4205 (30.2)
South-Southeast	2316/17,014 (13.6)	2563/12,212 (21.0)	1190/6287 (18.9)	881/4238 (20.8)	875/4205 (20.8)
Monthly pension, MXN (median [IQR])	3502 [3,020, 5512]	3459 [3,011, 5620]	3317 [3,011, 5349]	3317 [3,011, 5287]	3317 [3,011, 5281]
Medicalhistory
CCI (median [IQR])	3 [2, 4]	3 [2, 4]	3 [3, 4]	3 [3, 4]	3 [3, 4]
Diabetes, n/N (%)	4891/17,014 (28.7)	3946/12,212 (32.3)	2456/6287 (39.1)	1654/4238 (39.0)	1639/4205 (39.0)
NA	78/17,014 (0.5)	27/12,212 (0.2)	7/6287 (0.1)	3/4238 (0.1)	3/4205 (0.1)
Obesity, n/N (%)	1731/1,7014 (10.2)	1434/12,212 (11.7)	824/6287 (13.1)	587/4238 (13.9)	580/4205 (13.8)
NA	78/17,014 (0.5)	26/12,212 (0.2)	6/6287 (0.1)	2/4238 (0.0)	2/4205 (0.0)
Hypertension, n/N (%)	7212/17,014 (42.4)	5531/12,212 (45.3)	3198/6287 (50.9)	2169/4238 (51.2)	2149/4205 (51.1)
NA	78/17,014 (0.5)	28/12,212 (0.2)	8/6287 (0.1)	4/4238 (0.1)	4/4205 (0.1)
COPD, n/N (%)	1207/17,014 (7.1)	548/12,212 (4.5)	377/6287 (6.0)	240/4238 (5.7)	238/4205 (5.7)
NA	78/17,014 (0.5)	29/12,212 (0.2)	8/6287 (0.1)	5/4238 (0.1)	5/4205 (0.1)
Asthma, n/N (%)	326/17,014 (1.9)	165/12,212 (1.4)	82/6287 (1.3)	56/4238 (1.3)	56/4205 (1.3)
NA	78/17,014 (0.5)	29/12,212 (0.2)	8/6287 (0.1)	5/4238 (0.1)	5/4205 (0.1)
Ever smoking, n/N (%)	1695/17,014 (10.0)	992/12,212 (8.1)	642/6287 (10.2)	405/4238 (9.6)	399/4205 (9.5)
NA	78/17,014 (0.5)	30/12,212 (0.2)	9/6287 (0.1)	5/4238 (0.1)	5/4205 (0.1)
Cancer, n/N (%)	267/17,014 (1.6)	128/12,212 (1.0)	96/6287 (1.5)	67/4238 (1.6)	67/4205 (1.6)
NA	78/17,014 (0.5)	29/12,212 (0.2)	8/6287 (0.1)	5/4238 (0.1)	5/4205 (0.1)
Past COVID-19, n/N (%)	125/17,014 (0.7)	4/12,212 (0.0)	0/6287 (0.0)	0/4238 (0.0)	0/4205 (0.0)
Episode-relatedvariables
Epi-week (median [IQR])	35 [29, 41]	34 [31, 38]	34 [31, 38]	34 [31, 38]	34 [31, 38]
NAAT (positives), n/N (%)	0/17,014 (0.0)	4466/12,212 (36.6)	3851/6287 (61.3)	2477/4238 (58.4)	2456/4205 (58.4)
NA	12,714/17,014 (74.7)	7623/12,212 (62.4)	2327/6287 (37.0)	1712/4238 (40.4)	1701/4205 (40.5)
RADT (positives), n/N (%)	0/17,014 (0.0)	9772/12,212 (80.0)	4230/6287 (67.3)	2988/4238 (70.5)	2966/4205 (70.5)
NA	1119/17,014 (6.6)	1255/12,212 (10.3)	1008/6287 (16.0)	686/4238 (16.2)	682/4205 (16.2)
Vaccinated, n/N (%)	14,349/17,014 (84.3)	8947/12,212 (73.3)	3818/6287 (60.7)	2383/4238 (56.2)	2359/4205 (56.1)
Days vaccinated (median [IQR])	117 [75, 160]	113 [86, 145]	116 [90, 149]	116 [91, 148]	116 [91, 148]
Type of vaccine received, n/N (%)
AstraZeneca–AZD1222	3846/17,014 (22.6)	2383/12,212 (19.5)	996/6287 (15.8)	630/4238 (14.9)	623/4205 (14.8)
CanSino–Ad5-nCoV	727/17,014 (4.3)	774/12,212 (6.3)	391/6287 (6.2)	263/4238 (6.2)	262/4205 (6.2)
Pfizer - BNT162b2	5184/17,014 (30.5)	2208/12,212 (18.1)	878/6287 (14.0)	491/4238 (11.6)	483/4205 (11.5)
Sinovac–CoronaVac	3437/17,014 (20.2)	3103/12,212 (25.4)	1393/6287 (22.2)	923/4238 (21.8)	916/4205 (21.8)
Sputnik V–Gam-COVID-Vac	1155/17,014 (6.8)	479/12,212 (3.9)	160/6287 (2.5)	76/4238 (1.8)	75/4205 (1.8)
Unvaccinated	2665/17,014 (15.7)	3265/12,212 (26.7)	2469/6287 (39.3)	1855/4238 (43.8)	1846/4205 (43.9)

Abbreviations: CCI, Charlson comorbidity index; COPD, Chronic Obstructive Pulmonary Disease; Epi-week, epidemiological week; MXN, Mexican pesos; IQR, interquartile range; NA, not available; NAAT, Nucleic Acid Amplification Test; RADT, Rapid Antigen Diagnostic Test.

^aFor definitions, please refer to Supplementary Table S1.

Ethics approval

This study is based on a retrospective analysis of routine observational data collected by IMSS, which was deemed public health surveillance and did not require Institutional Review Board approval. Procedures were implemented to protect individuals’ confidentiality, and a signed confidentiality agreement bound all researchers with access to personal data.

Role of the funding source

This research was financially supported by the IMSS. Nonetheless, IMSS's involvement was restricted solely to the participation of the affiliated authors and did not influence the study design, analysis, writing, or decision to submit the manuscript.

Results

Study population

The IMSS pensioners’ cohort comprised 2,559,792 individuals. The median age was 69.0 years (interquartile range [IQR]: 65.0–75.0), and 73.1% (1,870,303/2,559,792) were male. Seventy percent of the cohort were fully vaccinated by June 29, 2021 (Supplementary Figure S1). Most had completed their vaccination schedule before that date, regardless of the type of vaccine received. During the study period, 36,414 (1.4%) of the pensioners had a valid COVID-19 diagnostic test at IMSS facilities, resulting in 37,860 separate episodes (Fig. 1), of which 22.8% (8634/37,860) were excluded: 97.5% (8417/8634) due to individuals being fully vaccinated less than 14 days before symptom onset or partially vaccinated and 2.5% (217/8634) because they received other vaccines, such as Janssen, Moderna, Sinopharm, or unknown. The final study population comprised 28,271 individuals and 29,226 separate episodes. As shown in Fig. 2 and Supplementary Figure S2, most episodes were identified between June and November 2021, a period dominated by the Delta variant in Mexico.

Fig. 1.

Study population flowchart. Abbreviations: COVID-19, Coronavirus Disease 2019; IMSS, Instituto Mexicano del Seguro social. Note: “Episodes” refers to medical evaluations due to the presence of COVID-19-like symptoms, each separated by at least eight days in relation to the previous symptom onset date and coupled with a valid SARS-CoV-2 test. A valid test was defined as either a nucleic amplification test or a rapid antigen test performed between 2 days before and 14 days after the onset of symptoms.

Fig. 2.

Number of test-positive cases and test-negative controls per day identified during the study period (April–November 2021).

A comparison of the study population's characteristics with the entire cohort of IMSS pensioners is presented in Supplementary Table S2. Individuals included in this study were slightly older (median age 70 years, IQR: 65–76 years, vs. 69 years, IQR: 65–75 years, in the complete cohort), were more often male (76.4% [21,598/28,271] vs. 73.1% [1,870,303/2,559,792] in the complete cohort), received a lower median pension income (3481 MXN, IQR: 3020–5549 MXN, vs. 3855 MXN, IQR: 3055–8660 MXN in the complete cohort), and more frequently resided in the south-southeast (16.8% [4738/28,271] vs. 12.7% [326,059/2,559,792] in the complete cohort).

VE against symptomatic SARS-CoV-2 infection

Table 1 presents the characteristics of the study subjects, classified into test-negative controls and test-positive cases. The analysis of receiving a full schedule of any of the five vaccines against symptomatic infection included 12,212 cases and 17,014 controls; the VE was 56.3% (95% CI: 53.5–59.0). Detailed results from all models can be found in Supplementary Material Section 2. After stratifying the analysis by days since full vaccination, we observed a slight decrease in VE estimates: from 62.8% (95% CI: 59.5–65.9) for ≤90 days to 53.8% (95% CI: 49.2–58.0) for ≥141 days (Table 2 and Fig. 3). For individual vaccines, the highest VE rates were shown by Gam-COVID-Vac (70.0%; 95% CI: 64.8–74.4]), BNT162b2 (69.8%; 95% CI: 67.3–72.0]), and AZD1222 (56.8%; 95% CI: 53.2–60.1]). Conversely, the lowest VE rates were shown by CoronaVac (39.1%; 95% CI: 34.1–43.7) and Ad5-nCoV (30.5%; 95% CI: 21.2–38.6]). All vaccines showed a slight decrease in VE over time.

Table 2.

COVID-19 vaccine effectiveness against symptomatic SARS-CoV-2 infection, hospitalization, severe COVID-19, and death among IMSS pensioners, either of any vaccine or by vaccine type, overall and stratified by days since full-vaccination: ≤90 days, 91–140 days, and ≥141 days, April–November 2021.

	Symptomatic Infection			Hospitalization			Severe COVID-19			Death
	Positive tests, n	Negative tests, n	VE, % (95% CI)	Positive tests	Negative tests	VE, % (95% CI)	Positive tests	Negative tests	VE, % (95% CI)	Positive tests	Negative tests	VE, % (95% CI)
Unvaccinated
Unvaccinated	3265	2665	–	2469	2665	–	1855	2665	–	1846	2665	–
Anyvaccine
Overall	8947	14,349	56.3 [53.5, 59]	3818	14,349	75.3 [73.4, 77]	2383	14,349	79.7 [78, 81.3]	2359	14,349	79.8 [78.1, 81.4]
≤90 days	2585	4895	62.8 [59.5, 65.9]	977	4895	80.3 [78.2, 82.3]	592	4895	84.7 [82.7, 86.4]	586	4895	84.8 [82.8, 86.5]
91–140 days	3875	4318	53.9 [50.1, 57.5]	1677	4318	74 [71.4, 76.3]	1083	4318	78.9 [76.6, 81]	1075	4318	79 [76.7, 81.1]
≥141 days	2487	5136	53.8 [49.2, 58]	1164	5136	71.2 [67.8, 74.2]	708	5136	75.4 [72.1, 78.4]	698	5136	75.6 [72.3, 78.5]
BNT162b2 (Pfizer-BioNTech)
Overall	2208	5184	69.8 [67.3, 72]	878	5184	84.1 [82.5, 85.6]	491	5184	88.2 [86.7, 89.5]	483	5184	88.3 [86.9, 89.6]
≤90 days	433	1502	74.3 [70.5, 77.6]	155	1502	87.5 [84.9, 89.7]	90	1502	90.4 [87.8, 92.4]	90	1502	90.3 [87.7, 92.4]
91–140 days	982	1558	70.1 [66.6, 73.2]	376	1558	85.3 [83.1, 87.2]	216	1558	89.5 [87.6, 91.1]	215	1558	89.5 [87.6, 91.1]
≥141 days	793	2124	66.9 [62.7, 70.7]	347	2124	80.5 [77.4, 83.2]	185	2124	85.3 [82.3, 87.8]	178	2124	85.7 [82.8, 88.2]
AZD1222 (Oxford-AstraZeneca)
Overall	2383	3846	56.8 [53.2, 60.1]	996	3846	76.2 [73.8, 78.4]	630	3846	80.3 [77.9, 82.4]	623	3846	80.4 [78.1, 82.5]
≤90 days	1200	1774	61.2 [56.9, 65.1]	443	1774	81.1 [78.4, 83.5]	274	1774	85.3 [82.8, 87.5]	271	1774	85.4 [82.9, 87.6]
91–140 days	764	1140	57.8 [52.4, 62.6]	358	1140	73.7 [69.5, 77.3]	238	1140	77.1 [72.9, 80.7]	235	1140	77.3 [73.2, 80.9]
≥141 days	419	932	51.8 [43.5, 58.8]	195	932	70.5 [64.2, 75.8]	118	932	73.5 [66.5, 79.1]	117	932	73.6 [66.6, 79.2]
CoronaVac (Sinovac)
Overall	3103	3437	39.1 [34.1, 43.7]	1393	3437	63.8 [60.4, 67]	923	3437	68.4 [64.9, 71.5]	916	3437	68.5 [65.1, 71.6]
≤90 days	726	1087	47 [39.6, 53.5]	307	1087	69.8 [64.5, 74.2]	192	1087	75.4 [70.3, 79.7]	190	1087	75.6 [70.5, 79.8]
91–140 days	1496	1083	36.1 [28.7, 42.7]	658	1083	63.8 [58.9, 68.1]	443	1083	69.8 [65.2, 73.8]	439	1083	70 [65.3, 74]
≥141 days	881	1267	34.9 [26.2, 42.6]	428	1267	57.8 [51, 63.6]	288	1267	59.2 [51.6, 65.5]	287	1267	59.2 [51.6, 65.5]
Ad5-nCoV (CanSino)
Overall	774	727	30.5 [21.2, 38.6]	391	727	52.1 [44.5, 58.6]	263	727	59.2 [51.8, 65.6]	262	727	59.2 [51.8, 65.6]
≤90 days	103	172	29 [5.3, 46.7]	43	172	58.2 [39, 71.3]	24	172	69.4 [50.6, 81]	24	172	69.1 [50, 80.8]
91–140 days	411	207	14.3 [−3.5, 29.1]	207	207	42.9 [29.3, 54]	147	207	52.1 [39.3, 62.1]	147	207	51.9 [39, 62]
≥141 days	260	348	43.6 [32.1, 53.1]	141	348	58.8 [48.7, 66.9]	92	348	63.8 [53.2, 72]	91	348	64.1 [53.6, 72.3]
Gam-COVID-Vac Sputnik V (Gamaleya Research Institute)
Overall	479	1155	70 [64.8, 74.4]	160	1155	86.8 [83.7, 89.3]	76	1155	91.9 [89.4, 93.9]	75	1155	92 [89.5, 93.9]
≤90 days	123	360	68.5 [59.5, 75.5]	29	360	90.4 [85.5, 93.7]	12	360	94.8 [90.5, 97.2]	11	360	95.2 [91, 97.4]
91–140 days	222	330	66.2 [58.2, 72.7]	78	330	84.6 [79.5, 88.4]	39	330	90.5 [86.4, 93.4]	39	330	90.5 [86.3, 93.4]
≥141 days	134	465	75.5 [69.1, 80.6]	53	465	87.1 [82.2, 90.6]	25	465	91.8 [87.3, 94.7]	25	465	91.8 [87.3, 94.7]

Abbreviations: 95% CI, 95% confidence interval; VE, vaccine effectiveness.

Fig. 3.

COVID-19 vaccine effectiveness against symptomatic SARS-CoV-2 infection, hospitalization, severe COVID-19, and death among IMSS pensioners, either of any vaccine or by vaccine type, stratified by days since full-vaccination: ≤90 days, 91–140 days, and ≥141 days, April–November 2021.

VE against COVID-19–related hospitalization

The analysis of VE against hospitalization included 6287 cases and 17,014 controls. The VE of full vaccination with any of the five vaccines was 75.3% (95% CI: 73.4–77.0). We observed a slight decrease in VE estimates after stratifying the analysis by days since full vaccination, from 80.3% (95% CI: 78.2–82.3) for ≤90 days to 71.2% (95% CI: 67.8–74.2) for ≥141 days. The highest VE was shown by Gam-COVID-Vac (86.8%; 95% CI: 83.7–89.3]), BNT162b2 (84.1%; 95% CI: 82.5–85.6), and AZD1222 (76.2%; 95% CI: 73.8–78.4]). In contrast, the lowest VE was shown by CoronaVac (63.8%; 95% CI: 60.4–67.0]) and Ad5-nCoV (52.1%; 95% CI: 44.5–58.6]). Most vaccines demonstrated a slight downward trend in VE.

VE against severe disease

The analysis of VE against severe disease included 4238 cases and 17,014 controls. The VE of full vaccination with any of the five vaccines was 79.7% (95% CI: 78.0–81.3). A decrease in VE estimates over time was also observed, from 84.7% (95% CI: 82.7–86.4) for ≤90 days to 75.4% (95% CI: 72.1–78.4) for ≥141 days. The highest VE rates were observed with Gam-COVID-Vac (91.9%; 95% CI: 89.4–93.9), BNT162b2 (88.2%; 95% CI: 86.7–89.5), and AZD1222 (80.3%; 95% CI: 77.9–82.4). In contrast, the lowest VE rates were shown by CoronaVac (68.4%; 95% CI: 64.9–71.5) and Ad5-nCoV (59.2%; 95% CI: 51.8–65.6). Most vaccines demonstrated a slight downward trend in VE.

VE against COVID-19–related death

The analysis of VE against death included 4205 cases and 17,014 controls. The VE of full vaccination with any of the five vaccines was 79.8% (95% CI: 78.1–81.4). We also observed a slight decrease in VE estimates after stratifying the analysis by days since full vaccination, from 84.8% (95% CI: 82.8–86.5) for ≤90 days to 75.6% (95% CI: 72.3–78.5) for ≥141 days. The highest VE rates were shown by Gam-COVID-Vac (92.0%; 95% CI: 89.5–93.9), BNT162b2 (88.3%; 95% CI: 86.9–89.6]), and AZD1222 (80.4%; 95% CI: 78.1–82.5). The lowest VE rates were shown by CoronaVac (68.5%; 95% CI: 65.1–71.6) and Ad5-nCoV (59.2%; 95% CI: 51.8–65.6). Most vaccines demonstrated a slight downward trend in VE.

VE stratified by sex

The results of the sex-stratified analyses are presented in Supplementary Table S3. Overall, VE estimates were higher for women than men. Among women, receipt of any vaccine had the following VE rates: symptomatic infection (62.5%; 95% CI: 56.7–67.4), hospitalization (84.8%; 95% CI: 81.9–87.2), severe disease (88.9%; 95% CI: 86.3–91.0), and death (88.9%; 95% CI: 86.3–91.0). In contrast, men had the following VE rates: symptomatic infection (54.7%; 95% CI: 51.4–57.9), hospitalization (72.5%; 95% CI: 70.2–74.6), severe disease (77.2%; 95% CI: 75.1–79.2), and death (77.4%; 95% CI: 75.3–79.3). These differences persisted when all five vaccines were evaluated separately.

Discussion

In this nationwide TND study nested within a cohort of IMSS pensioners, we found that a full vaccination schedule with any of the five evaluated SARS-CoV-2 vaccines—Pfizer–BioNTech's BNT162b2, Gamaleya Research Institute's Gam-COVID-Vac, Oxford–AstraZeneca's AZD1222, Sinovac's CoronaVac, or CanSino's Ad5-nCoV—provided moderate to high levels of protection against COVID-19 and its progression to severe outcomes. VE rates were the following: symptomatic infection (56.3%; 95% CI: 53.5–59.0), hospitalization (75.3%; 95% CI: 73.4–77.0), severe disease (79.7%; 95% CI: 78.0–81.3), and death (79.8%; 95% CI: 78.1–81.4). Importantly, these estimates only decreased slightly over the study period.

This study, which focused on a population composed predominantly of older adults in Mexico, provides important insights into the impact of SARS-CoV-2 vaccines on this age group. Understanding that impact is paramount due to their increased vulnerability to severe complications, including hospitalization and death.²⁶ Trials including older adults have proved that COVID-19 vaccines are efficacious against infection.²⁷ VE estimates from these studies include 90.9% (95% CI: 86.3–94.2) for BNT162b2 in those over 55,²⁸ 91.8% (95% CI: 67.1–98.3) for Gam-COVID-Vac in those over 60,²⁹ 83.5% (95% CI: 54.2–94.1) for AZD1222 in those over 65,³⁰ and 53.3% (95% CI: 0.9–78) for Ad5-nCoV in those over 60.³¹ Some of these estimates are comparable to those found in real-world studies of VE in high-income settings. For BNT-162b2, a TND study reported a VE of 96.2% (95% CI: 95.5–96.9) against infection among the U.S. veterans’ healthcare system (median age of 61 years),³² and a cohort study performed in Israel reported VE of 98% (95% CI: 90–100) against symptomatic infection for those aged ≥70.³³

Although our estimates of VE are lower than those reported in RCTs and studies on older adults in high-income settings, they are similar to those found in other Latin American countries, including in a study conducted in Mexico. For example, in Brazil, a TND study of adults aged ≥70 reported that CoronaVac had a VE against symptomatic infection of 46.8% (95% CI: 38.7–53.8).¹⁰ In Colombia, a cohort study of adults aged 60+ found that VE in preventing hospitalization without death was 61.6% (95% CI: 58.0–65.0) and death after hospitalization was 79.8% (95% CI: 78.5–81.1) for those who had received BNT162b2, AZD1222, CoronaVac, or Ad26. COV2–S.³⁴ Finally, a cohort analysis in Mexico using self-reported vaccination information yielded similar estimates of VE against symptomatic infection as ours: BNT162b2 (66.58%), Gam-COVID-Vac (53.95%), AZD1222 (59.13%), CoronaVac (48.03%), and Ad5-nCoV (70.04%).⁹

In addition to the biases discussed below, several factors could explain the differences in VE estimates between high-income countries and LMICs. These include differences in demand dynamics and vaccine availability and the inherent challenges associated with vaccine storage, cold chain management, and distribution, which may be more pronounced in LMICs than in high-income settings.¹¹ For instance, BNT162b2 and Gam-COVID-Vac require an extensive and stringent cold chain that was not available at the local level in routine immunization programs in Mexico and thus had to be established. Specifically, BNT162b2 must be stored at ultralow temperatures (between −90 °C and −60 °C) or alternatively, it can be kept at 2 °C and 8 °C for only 10 weeks.³⁵ Consistently maintaining this throughout the entire supply chain, from manufacturing and distribution to storage and administration, was challenging. A secondary analysis shown in Supplementary Table S4 supports the possibility of logistical challenges across regions, as states with higher marginalization levels had lower VE estimates against all outcomes; further research is needed to identify the primary drivers of this variation, which will be instrumental in devising effective interventions to enhance the impact of vaccination campaigns in low- and middle-income countries across Latin America.

It is also important to evaluate VE by sex, as differences in clinical and immune responses, including a higher incidence of adverse events in women, have been recognized.^36,37 Nevertheless, sex-stratified data are notably lacking in VE studies. For example, one systematic review found that only 8.8% of post-approval evaluations provided such results.³⁸ These findings are supported by another review, which showed that only 24% of experimental and observational studies of COVID-19 VE reported sex-stratified primary outcome data.³⁷ The heterogeneity of study populations and approaches to estimating VE, and the variety of vaccines evaluated, prevented authors from such studies meta-analyzing VE estimates.

We found that, overall, women had higher VE estimates than men for the four evaluated outcomes. Several potential explanations for these differences may deserve further investigation. First, as mentioned, biological differences in the immune response could result in a more robust immune response in women and therefore potentially greater effectiveness. For example, studies have shown that women may develop higher antibody titers to some influenza vaccines.^37,39 However, clinical trials evaluating COVID-19 vaccines and reporting sex-stratified effects did not show differences in immunogenicity or efficacy by sex.³⁷ A second possible explanation could be differences in unmeasured factors between the sexes, such as social and behavioral characteristics. For example, higher health literacy and more proactive preventive behaviors in women could lead to higher vaccine uptake, lower risk of infection, and therefore higher VE estimates.³⁷ Finally, our results may also be explained by other factors, such as differential misclassification bias of relevant variables, such as better reporting of comorbidities in women leading to less residual confounding.

Our study has both strengths and limitations. To begin with the strengths, we used a TND, potentially reducing outcome misclassification and confounding by healthcare-seeking behaviors (HSBs)—well-known factors that can bias estimates of VE.¹⁸ Second, given that our study was nested within a well-defined homogeneous cohort, we were able to adjust for pertinent factors, such as age, sex, geographical location, and monthly pension. Additionally, our analyses incorporated the possibility of varying VE estimates over time by using mixed-effects models with random intercepts for epidemiological week. Third, we leveraged the best information available, drawing on multiple data sources to enhance the reliability of our results. Last, a notable strength is the substantial number of cases and controls derived from a national sample of IMSS pensioners. IMSS is the largest social security institution in Mexico and one of the largest in Latin America, and this considerable sample size enabled us to evaluate the real-world VE within this population.

As for the limitations, our study population was the first targeted by the national vaccination campaign, resulting in some subjects having missing or inaccurate information about vaccination dates due to the early stages of the rollout and vaccine registry implementation. We sought to ameliorate this issue by limiting our analyses to individuals with a complete vaccination schedule, since more accurate information was available for them. Second, assuming no effect measure modification by HSBs,¹⁸ our study may at best represent the population of IMSS pensioners during the period when the Delta variant was predominant in Mexico. However, the probability of no effect modification by HSBs is low, as several differences may exist in relevant factors, such as socioeconomic status, between our study population and those who did not seek care for COVID-19 symptoms. Such differences could potentially limit the generalizability of our findings.

Third, we included subjects irrespective of the type of test used (NAAT vs. RADTs); NAATs were more commonly performed on unvaccinated individuals in our study population (44.9% vs. 26.7% of episodes). Predicting the direction and magnitude of the bias accurately is challenging without a formal quantitative bias analysis. Nonetheless, misclassification biases may have been reduced by using TND and including only symptomatic individuals, since symptom status is known to increase the sensitivity of both NAATs and RADTs (the specificity for both types is high for diagnosing acute infection regardless of the presence of symptoms).^40,41 Fourth, the saturation of COVID-19 testing sites at IMSS could have prevented some potential cases from participating, potentially biasing VE estimates. Similarly, in the context of the TND, bias could also arise if HSBs differ based on vaccination status. For instance, if vaccinated patients are more proactive in seeking healthcare for mild symptoms compared to unvaccinated people, this could lead to an overrepresentation of test-positive cases among the vaccinated, making COVID-19 vaccines appear less effective than they actually are.⁴² Nonetheless, the potential for this bias might have been diminished in our study by requiring a common clinical definition for both test-positive cases and test-negative controls, thereby increasing the comparability of HSBs between groups. Last, it is essential to underscore that our study is observational, meaning that we cannot establish definitive causal relationships.

Conclusions

Our nationwide TND study, nested within a cohort of IMSS pensioners, found that all evaluated vaccines (Pfizer–BioNTech's BNT162b2, Gamaleya National Research Institute of Epidemiology and Microbiology's Gam-COVID-Vac, Oxford–AstraZeneca's AZD1222, Sinovac's CoronaVac, and CanSino's Ad5-nCoV) were moderately to highly effective against symptomatic infection and its progression to hospitalization, severe disease, and death. VE only slightly decreased for most vaccines over the study period. Our research underscores the importance of vaccinating all individuals, especially those who remain hesitant, and ensuring completion for those partially vaccinated. Urgent research is necessary to identify the primary factors driving the differences in VE estimates between high- and low-middle-income settings. Last, additional studies are required to evaluate the effectiveness of booster shots against COVID-19 and its progression to severe outcomes, particularly as new variants of concern, such as Omicron, continue to emerge.

Contributors

MHA contributed to the conceptualization, study design, data analysis, data collection, original draft writing, review, editing, and supervision. EOB was involved in the conceptualization, study design, data analysis, data collection, original draft writing, review, and editing. MTO participated in the conceptualization, study design, data collection, data analysis, original draft writing, review, editing, and validation. RZT contributed to the conceptualization, study design, data collection, data analysis, original draft writing, visualization, and validation. HGD was responsible for conceptualization, visualization, study design, data curation, and formal analysis. DBSC handled writing the original draft, review, and editing. WVR, GARO, MMA, and AHG were all engaged in review, editing, and study design. Each author has reviewed and approved the final version of the manuscript and agreed to be accountable for their respective roles in the project.

Data sharing statement

An anonymized dataset may be shared upon reasonable request.

Declaration of interests

All authors declare no conflicts of interest.

Appendix ASupplementary data

Supplementary Material