SARS-CoV-2 Vaccine Effectiveness in a High-Risk National Population in a Real-World Setting

Data are urgently needed on the real-world effectiveness of the currently administered COVID-19 vaccines. This case–control study evaluated the short-term effectiveness of currently administered vaccines in preventing confirmed SARS-CoV-2 infection among veterans receiving care in the U.S. Department of Veterans Affairs health care system.

Visual Abstract. SARS-CoV-2 Vaccine Effectiveness in a High-Risk National Population.

Data are urgently needed on the real-world effectiveness of the currently administered COVID-19 vaccines. This case–control study evaluated the short-term effectiveness of currently administered vaccines in preventing confirmed SARS-CoV-2 infection among veterans receiving care in the U.S. Department of Veterans Affairs health care system.

Abstract

Background:

With the emergency use authorization of multiple vaccines against SARS-CoV-2 infection, data are urgently needed to determine their effectiveness in a real-world setting.

Objective:

To evaluate the short-term effectiveness of vaccines in preventing SARS-CoV-2 infection.

Design:

Test-negative case–control study using conditional logistic regression.

Setting:

U.S. Department of Veterans Affairs health care system.

Participants:

All veterans who had testing for SARS-CoV-2 infection between 15 December 2020 and 4 March 2021 and no confirmed infection before 15 December 2020.

Intervention:

SARS-CoV-2 vaccination with either the BNT-162b2 (Pfizer–BioNTech) or mRNA-1273 (Moderna) vaccine as part of routine clinical care.

Measurements:

Effectiveness of vaccination against confirmed SARS-CoV-2 infection.

Results:

Among 54 360 persons who tested positive and 54 360 propensity score–matched control participants, the median age was 61 years, 83.6% were male, and 62% were White. Median body mass index was 31 kg/m² among those who tested positive and 30 kg/m² among those who tested negative. Among those who tested positive, 9800 (18.0%) had been vaccinated; among those who tested negative, 17 825 (32.8%) had been vaccinated. Overall vaccine effectiveness 7 or more days after the second dose was 97.1% (95% CI, 96.6% to 97.5%). Effectiveness was 96.2% (CI, 95.5% to 96.9%) for the Pfizer–BioNTech BNT-162b2 vaccine and 98.2% (CI, 97.5% to 98.6%) for the Moderna mRNA-1273 vaccine. Effectiveness remained above 95% regardless of age group, sex, race, or presence of comorbidities.

Limitations:

Predominantly male population; lack of data on disease severity, mortality, and effectiveness by SARS-CoV-2 variants of concern; and short-term follow-up.

Conclusion:

Currently used vaccines against SARS-CoV-2 infection are highly effective in preventing confirmed infection in a high-risk population in a real-world setting.

Primary Funding Source:

None.

In randomized, placebo-controlled clinical trials, vaccines developed by Pfizer–BioNTech (BNT-162b2) and Moderna (mRNA-1273) have shown 94% to 95% efficacy in preventing symptomatic SARS-CoV-2 infection (1, 2). The effectiveness against infection is more difficult to estimate and was lower in the earlier phase 3 clinical trials. A recent report from Israel showed 92% effectiveness of the BNT-162b2 vaccine in preventing documented infection and severe disease 7 or more days after the second dose (3). In another recent report from Qatar, the BNT-162b2 vaccine was 89.5% effective in preventing infection with the Alpha (previously known as the B1.1.7) variant and 75% effective in preventing infection with the Beta (previously known as the B1.351) strain (4). Other reports from select population groups have noted a sharp decrease in infections among vaccinated persons, although vaccine effectiveness is often not explicitly stated (5–7). Real-world data from large, national populations are urgently needed to provide reassurance about the effectiveness of current vaccines.

The veterans enrolled in care in the U.S. Department of Veterans Affairs (VA) health care system are at higher risk for severe clinical consequences due to older age, higher burden of comorbidities, and higher prevalence of social vulnerability factors (8–10). Therefore, the VA health care system provides a unique real-world setting to study natural history, disease outcomes, and effectiveness of new therapies in a high-risk population (11). We sought to determine the short-term effectiveness of currently administered vaccines in preventing confirmed SARS-CoV-2 infection among veterans receiving care in the VA health care system.

Methods

VA COVID-19 Shared Data Resource

In response to the SARS-CoV-2 pandemic, the VA rapidly created a national VA COVID-19 Shared Data Resource. Using case definitions and data mapping, which were validated collaboratively across the VA, the resource contains information on all veterans with a confirmed laboratory diagnosis of SARS-CoV-2 infection within the VA and those who were tested outside the VA with a VA clinical note confirming the diagnosis. The VA COVID-19 Shared Data Resource is updated regularly and contains extensive demographic, clinical, pharmacologic, laboratory, vital sign, and clinical outcome information derived from multiple validated sources, including the VA Corporate Data Warehouse and the VA electronic medical record.

Study Population and Data Set Creation

For the current study, we identified all veterans in the VA COVID-19 Shared Data Resource who had SARS-CoV-2 testing between 15 December 2020 and 4 March 2021 and no confirmed infection before 15 December 2020. For each person who tested positive, we identified a propensity score–matched control participant who tested negative, matched by age, sex, race, body mass index, Charlson Comorbidity Index score, and geographic location. We used nearest-neighbor matching with a caliper of 0.25 SD. The Charlson Comorbidity Index is a validated and widely used score that identifies persons at the highest risk for mortality over a period of time based on the presence and severity of comorbidities (12–14). For geographic location, we used the VA facility where the index test was performed to account for geographic variation in disease incidence and testing patterns. Data retrieved included demographic characteristics, clinical diagnoses, presence of symptoms at presentation, anthropometric measurements and vital signs, all pharmacologic interventions, and select laboratory results. Data on vaccine administration date and the type of vaccine used were also retrieved. Comorbidities were defined according to the definitions used in the VA Corporate Data Warehouse, which uses the International Classification of Diseases (Ninth or 10th Revision, as appropriate) for classifying comorbidities. Baseline diagnoses included all diagnoses recorded in the 2 years before the index date.

Statistical Analysis

We used a test-negative case–control design to evaluate the effectiveness of vaccination against confirmed SARS-CoV-2 infection. This is a widely accepted design for determining vaccine effectiveness in a population after the introduction of a vaccine (15–17). We used conditional logistic regression to calculate the odds of testing positive among the vaccinated group versus the unvaccinated group (18, 19). Vaccine effectiveness was calculated by using the following formula:

1 − Odds(T+|Vaccinated) / Odds(T+|Nonvaccinated)

Our primary outcome of interest was overall vaccine effectiveness 7 or more days after the second dose. We also assessed vaccine effectiveness among those who received only 1 dose and effectiveness anytime after receipt of the second dose. We evaluated effectiveness separately among those who received the Pfizer–BioNTech BNT-162b2 vaccine and those who received the Moderna mRNA-1273 vaccine.

Finally, we performed subgroup analyses by age group, sex, race, and comorbidity burden to determine vaccine effectiveness in various population subgroups. Effectiveness was calculated for each subgroup, and P values were calculated to test the significance of the difference within each subgroup. A P value less than 0.05 was considered significant. For all point estimates of vaccine effectiveness, we calculated 95% CIs (19, 20).

Ethical Considerations

The study was approved by the Institutional Review Board at the VA Pittsburgh Healthcare System. A waiver of informed consent was granted for the study.

Role of the Funding Source

There was no external funding source for this study.

Results

We identified 54 360 matched pairs of veterans who were tested for SARS-CoV-2 infection between 15 December 2020 and 4 March 2021 (1:1 ratio of positive and negative results). Baseline characteristics of the study population are shown in Table 1. The median age was 62 years among those who tested negative and 61 years among those who tested positive; in both groups, 83.6% of participants were male and 62% were White. Median body mass index was 31 kg/m² among those who tested positive and 30 kg/m² among those who tested negative. Among those who tested positive, 9800 (18.0%) had been vaccinated, and among those who tested negative, 17 825 (32.8%) had been vaccinated. The overall vaccine effectiveness was 97.1% (95% CI, 96.6% to 97.5%) 7 or more days after the second dose. Effectiveness was 96.2% (CI, 95.5% to 96.9%) for the Pfizer–BioNTech BNT-162b2 vaccine and 98.2% (CI, 97.5% to 98.6%) for the Moderna mRNA-1273 vaccine. The median follow-up after the first dose was 60 days (interquartile range [IQR], 49 to 69 days) for recipients of the Moderna mRNA-1273 vaccine and 56 days (IQR, 47 to 64 days) for recipients of the Pfizer–BioNTech BNT-162b2 vaccine. Among those who received 2 doses, median follow-up after the second dose was 30 days (IQR, 21 to 41 days) for recipients of the Moderna mRNA-1273 vaccine and 34 days (IQR, 26 to 43 days) for recipients of the Pfizer–BioNTech BNT-162b2 vaccine.

Table 1.

Baseline Characteristics of Persons Who Tested Positive and Propensity Score–Matched Control Participants Who Tested Negative

Among those who received only 1 dose, effectiveness was 85.0% (CI, 84.2% to 85.8%) overall, 84.0% (CI, 82.7% to 85.1%) for the Pfizer–BioNTech BNT-162b2 vaccine, and 85.7% (CI, 84.6% to 86.8%) for the Moderna mRNA-1273 vaccine. Effectiveness against infection anytime after the second dose was 95.3% (CI, 94.7% to 95.9%) overall, 94.3% (CI, 93.4% to 95.2%) for the Pfizer–BioNTech BNT-162b2 vaccine, and 96.6% (CI, 95.7% to 97.2%) for the Moderna mRNA-1273 vaccine (Table 2).

Table 2.

Vaccine Effectiveness, Calculated Using the Test-Negative Case–Control Design (Conditional Logistic Regression Analysis)

Vaccine effectiveness was numerically similar (though statistically significantly higher) among persons aged 70 years or older compared with those younger than 70 years (97.5% and 96.4%, respectively; P = 0.032). Effectiveness was similar between Blacks and Whites (97.4% and 96.9%, respectively; P = 0.20 overall for race) and between men and women (97.2% and 95.4%, respectively; P = 0.25). Vaccine effectiveness was 98.1% among persons with a Charlson Comorbidity Index score less than 2 and 96.9% among those with a score of 2 or higher (P = 0.025) (Table 3).

Table 3.

Vaccine Effectiveness, by Population Subgroup

Discussion

In a large national health care system, we found the currently used SARS-CoV-2 vaccines to be more than 95% effective in preventing confirmed infection. Veterans are at particularly high risk given their older age and greater burden of comorbidities compared with the general population (8, 9), and the high level of vaccine effectiveness we observed is therefore reassuring. Our results are similar to those of a recent study at a national level in Israel, which found effectiveness of 92% for infection prevention with the Pfizer–BioNTech BNT-162b2 vaccine (3). Although these 2 studies had different designs and assessed different populations, the remarkably similar results provide further reassurance of the effectiveness of the vaccines in preventing infection in various real-world settings.

Vaccine effectiveness was 97.5% in persons aged 70 years or older and 96.4% among those younger than 70 years. Although this difference was statistically significant, the numerical similarity implies similarly high rates of effectiveness. Slightly higher effectiveness in older persons may be related to lower mobility and consequently a lower likelihood of exposure to infection. Older age has been associated with reduced mobility, which is in turn associated with increases in health care use, nursing home admissions, and mortality (21–23). There were no differences based on sex or race, with very high effectiveness among all subgroups. However, effectiveness was slightly lower among persons with comorbidities compared with those without comorbidities. Previous studies have shown a higher risk for infection as well as poorer clinical outcomes among persons with comorbidities. Our results are consistent with prior studies and the current understanding of SARS-CoV-2 infection.

Although we clearly show the effectiveness of the current vaccines in preventing infection, we did not assess their effectiveness in preventing severe disease and death. A cohort design is better suited to answer this question, as shown by the aforementioned study in Israel (3). Further studies are warranted to confirm this, although limited data from other health care systems suggest a high level of effectiveness in preventing these serious outcomes (3).

Strengths of our study include the large national study population at high risk for infection and use of the widely accepted test-negative case–control design. However, several limitations must be acknowledged, including a predominantly male population and a lack of data on disease severity and mortality. We did not have information on the reasons for testing, and we were unable to test the effect of SARS-CoV-2 variants of concern on vaccine effectiveness. Finally, we determined vaccine effectiveness over a relatively short period, and longer follow-up may be needed to determine long-term effectiveness.

In conclusion, the SARS-CoV-2 vaccines currently used in the VA health care system provided a high level of protection against confirmed infection. Further studies are needed to determine vaccine effectiveness in other populations (such as women and younger persons) and the effect of vaccination on other serious outcomes.