In a target trial emulation study that included nearly 6 million predominately male patients receiving care in the U.S. Department of Veterans Affairs health care system, those receiving messenger RNA vaccines against SARS-CoV-2 were matched 1:1 to unvaccinated controls according to demographic, clinical, and geographic characteristics and followed for SARS-CoV-2 infection or SARS-CoV-2–related death to determine vaccine efficacy.
Visual Abstract. COVID-19 Vaccination Effectiveness Against Infection or Death.
In a target trial emulation study that included nearly 6 million predominately male patients receiving care in the U.S. Department of Veterans Affairs health care system, those receiving messenger RNA vaccines against SARS-CoV-2 were matched 1:1 to unvaccinated controls according to demographic, clinical, and geographic characteristics and followed for SARS-CoV-2 infection or SARS-CoV-2–related death to determine vaccine efficacy.
Abstract
Background:
Little is known about real-world COVID-19 vaccine effectiveness (VE) in racially and ethnically diverse, elderly populations with high comorbidity burden.
Objective:
To determine the effectiveness of messenger RNA COVID-19 vaccines.
Design:
Target trial emulation study comparing newly vaccinated persons with matched unvaccinated controls.
Setting:
U.S. Department of Veterans Affairs health care system.
Participants:
Among persons receiving care in the Veterans Affairs health care system (n = 5 766 638), those who received at least 1 dose of the Moderna or Pfizer–BioNTech COVID-19 vaccine from 11 December 2020 to 25 March 2021 (n = 2 099 871) were matched to unvaccinated controls in a 1:1 ratio according to demographic, clinical, and geographic characteristics.
Intervention:
Follow-up for SARS-CoV-2 infection or SARS-CoV-2–related death, defined as death within 30 days of infection, began after the vaccination date or an identical index date for the matched unvaccinated controls and continued until up to 30 June 2021.
Measurements:
Vaccine effectiveness against SARS-CoV-2 infection or SARS-CoV-2–related death.
Results:
Vaccinated and unvaccinated groups were well matched; both were predominantly male (92.9% vs. 93.4%), had advanced age (mean, 68.7 years in both groups), had diverse racial and ethnic distribution (for example, Black: 17.3% vs. 17.0%, Hispanic: 6.5% vs. 6.1%), and had substantial comorbidity burden. Vaccine effectiveness 7 or more days after the second vaccine dose was 69% (95% CI, 67% to 70%) against SARS-CoV-2 infection and 86% (CI, 82% to 89%) against SARS-CoV-2–related death and was similar when follow-up was extended to 31 March versus 30 June. Vaccine effectiveness against infection decreased with increasing age and comorbidity burden.
Limitation:
Predominantly male population and lack of data on SARS-CoV-2 variants.
Conclusion:
In an elderly, diverse, high-comorbidity population, COVID-19 VE against infection was substantially lower than previously reported, but VE against death was high. Complementary infection mitigation efforts remain important for pandemic control, even with vaccination.
Primary Funding Source:
U.S. Department of Veterans Affairs.
The U.S. Food and Drug Administration issued emergency use authorization for COVID-19 vaccines manufactured by Pfizer–BioNTech (BNT162b2) on 11 December 2020, by Moderna (mRNA-1273) on 18 December 2020, and by Janssen (JNJ-78436735) on 27 February 2021. The 2 messenger RNA (mRNA) vaccines, BNT162b2 and mRNA-1273, were shown to have very high efficacy of 95% and 94.1%, respectively, against symptomatic SARS-CoV-2 infection in phase 3 randomized controlled trials (1–3).
Real-world studies also suggest that these vaccines are effective. A prospective study of 23 234 adult health care workers from 104 public hospitals in England followed with biweekly SARS-CoV-2 polymerase chain reaction (PCR) testing suggested that BNT162b2 had 85% (95% CI, 74% to 96%) vaccine effectiveness (VE) for preventing symptomatic or asymptomatic infection 7 or more days after the second dose (4). A study of 3950 health care personnel, first responders, and other frontline workers in 8 U.S. locations tested for SARS-CoV-2 weekly showed 90% effectiveness 14 or more days after the second dose of BNT162b2 (5). A study of 596 618 persons vaccinated with BNT162b2 in Israel and matched to unvaccinated controls reported that VE at 7 or more days after the second dose was 92% for documented infection (6). A study of the entire population of Israel aged 16 years or older reported that VE at 7 or more days after the second BNT162b2 dose was 95.3% for asymptomatic infection, 97.0% for symptomatic infection, 97.5% for SARS-CoV-2–related hospitalization, and 96.7% for SARS-CoV-2–related death (7).
The real-world effectiveness of the 2 mRNA vaccines in ethnically and racially diverse populations across the entire United States is not well characterized, especially in more vulnerable populations, such as elderly persons with high comorbidity burden. The U.S. Department of Veterans Affairs (VA) health care system is the largest national, comprehensive health care system in the United States. It includes a large proportion of older adults with multimorbidity. A recent test-negative, case–control study of VE using VA data reported an extremely high VE against infection of 97.1% (CI, 96.6% to 97.5%) 7 or more days after the second vaccine dose (8). However, this study did not match case patients to controls by date of testing and did not select cases and controls with similar disease severity (9), which could lead to overestimation of VE. In addition, the case–control study design does not allow estimation of VE against SARS-CoV-2–related death. Accurate estimates of VE in high-risk, real-world populations are critical for guiding policy in this area and determining whether complementary infection mitigation efforts (for example, physical distancing, masking, and screening) remain important for pandemic control even after vaccination. We used a target trial emulation design (10) to estimate the effectiveness of the 2 mRNA vaccines in the VA health care system against both SARS-CoV-2 infection and death.
Methods
Study Setting and Data Source
The VA provides care at 171 medical centers and 1112 outpatient clinics throughout the country. It uses a nationwide electronic health records system, enabling accurate ascertainment of relevant baseline characteristics and potential confounders. We used data from the VA's Corporate Data Warehouse, a relational database of VA enrollees' comprehensive electronic health records, including the VA COVID-19 Shared Data Resource, which includes analytic variables provisioned by the VA Informatics and Computing Infrastructure on all VA enrollees who were tested for or vaccinated against SARS-CoV-2 (11). We also used linked Medicare data obtained through the VA Information Resource Center (12) to identify any additional VA enrollees diagnosed with or vaccinated against COVID-19 through Medicare-covered services.
The VA Corporate Data Warehouse, COVID-19 Shared Data Resource, and VA Information Resource Center Medicare data were last accessed on 13 September 2021 to allow adequate time for outcomes extending to 31 June 2021 to be electronically recorded. The study was approved by the VA Puget Sound Institutional Review Board (#01886).
Study Population and Ascertainment of Type and Date of COVID-19 Vaccination
We created a cohort of all VA enrollees aged 18 years or older who were alive as of 11 December 2020 (the date of emergency use authorization for BNT162b2) and had an inpatient or outpatient encounter in the VA health care system in the preceding 12 months or until 26 March 2021 (n = 5 766 638). We excluded 153 116 persons with evidence of SARS-CoV-2 infection before their vaccination date (or the index date for unvaccinated matched controls [see section Vaccination Effectiveness: Target Trial Emulation]) because they have a high rate of protection against reinfection (13), thereby masking the effect of vaccination. We excluded 60 185 persons living in VA long-term care facilities because they had very high early vaccination rates and thus no appropriate persons to match as unvaccinated controls, as well as 37 445 persons who received vaccination before emergency use authorization or had irregular vaccine dosing; this left 5 515 892 persons who could serve as vaccine recipients or matched unvaccinated controls in our analyses. Among these, we identified 2 103 790 persons who received at least 1 mRNA vaccine dose between 11 December 2020 and 25 March 2021, of whom 2 099 871 (1 166 019 [55.5%] Moderna and 933 852 [45.5%] Pfizer–BioNTech) vaccine recipients who could be matched to appropriate unvaccinated controls (see section Vaccination Effectiveness: Target Trial Emulation) were included in the vaccine recipient group of the current analysis. We identified both vaccinations done within the VA, identified through pharmacy records (n = 1 782 691 [84.9%]), and vaccinations done outside the VA (n = 317 180 [15.1%]), confirmed by documentation of type and date of vaccination entered into VA or Medicare records.
Only 12 620 veterans received the Janssen vaccine. These persons were considered as potential controls, and those who matched were censored at the time they received the Janssen vaccine.
Vaccination Effectiveness: Target Trial Emulation
We designed this observational study to emulate a target trial of COVID-19 vaccination versus placebo (10). Vaccine recipients were matched in a 1:1 ratio to unvaccinated VA enrollees who had the same inclusion criteria (that is, were alive, had received VA care in the prior 12 months, and had no evidence of prior SARS-CoV-2 infection as of their matched vaccinated person's index date). To facilitate identification of matching cohorts of vaccinated and unvaccinated persons anchored around the same calendar date (14, 15), we divided the observation period into 14 one-week periods. In each week, we identified all persons who were alive, uninfected, and unvaccinated as of the first day of the week. Among this “at-risk” cohort, we identified all treated persons who could potentially be included in our target trial, defined as those who were vaccinated during the week, and all potential controls, defined as those who were not vaccinated at the start of the week. We used the baseline characteristics of all persons at the beginning of the week to identify the single best unvaccinated matched control person for each vaccinated person using a combination of exact and coarsened exact matching and propensity score matching, implemented by STATA's kmatch command (16) (StataCorp). Entropy balancing of means in all matching characteristics was included as a refinement in the matching process. We used exact matching by Veterans Integrated Services Network (the administrative regions of VA) to account for geographic variability in vaccination distribution and risk for infection. We used coarsened exact matching by age (2-year buckets) and Charlson Comorbidity Index (CCI) (2-point buckets)—the 2 characteristics most strongly associated with both receipt of vaccination and development of SARS-CoV-2 infection or death in VA enrollees (17–20). The characteristics used in the propensity score logistic regression model estimating the probability of undergoing vaccination during the week were determined a priori and are listed in the next section. STATA's kmatch procedure assigned to each vaccinated person 2 unvaccinated persons who had the same Veterans Integrated Services Network, age bucket, and CCI bucket and the nearest-neighbor propensity score within a caliper of 0.01. Controls were matched without replacement within the week but were eligible to be matched to another vaccinated person in a subsequent week. Among these 2 persons, we then identified the 1 person with the nearest propensity score to the vaccine recipient who was still alive, unvaccinated, and uninfected as of the actual day of the week their matched vaccine recipient was vaccinated. Each vaccinated–unvaccinated matched pair was then followed from the date of vaccination (or index date for the unvaccinated control) until the date of the earliest of the following events: occurrence of an outcome event, death unrelated to COVID-19, vaccination (for unvaccinated controls), vaccination of the matched control (for vaccine recipients), or the end of the follow-up period (30 June 2021). Newly vaccinated persons were eligible for inclusion in the study even if they had previously been selected as a control.
This entire process was repeated 14 times for each of the 14 one-week periods. Vaccine recipients without a matched unvaccinated control (n = 3919 [0.2%]) were omitted from all analyses.
Selection of Covariates for Inclusion in the Propensity Score Model and for Additional Adjustments
Covariates included in the propensity score model were known to be associated with vaccination and SARS-CoV-2 infection or SARS-CoV-2–related death in the VA population (17, 18, 20, 21). These included age; sex; self-reported race/ethnicity; urban or rural residence (based on ZIP codes, using data from the VA Office of Rural Health [22], which uses the secondary rural–urban commuting area for defining rurality); geographic location (categorized according to the 19 Veterans Integrated Services Networks [23]); and the following comorbid conditions: diabetes, congestive heart failure, chronic obstructive pulmonary disease, and chronic kidney disease, defined by appropriate International Classification of Diseases, 10th Revision, codes (Table 1). We calculated the 2-year CCI (24, 25) as a measure of comorbidity burden. We used the Care Assessment Need (CAN) score, a validated measure of 1-year mortality in VA enrollees calculated using sociodemographic characteristics, clinical diagnoses, vital signs, medications, laboratory values, and health care use data from the VA's national electronic health record. The CAN score is an independent predictor of COVID-19–related death (17, 26), while also capturing health care use. It is calculated only in VA enrollees who have an assigned VA primary care provider; therefore, a “missing” CAN score is informative as an indicator of VA enrollees who did not have a primary care provider. Finally, we used the body mass index, calculated using measured weight and height.
Table 1.
Baseline Sociodemographic and Clinical Characteristics of Persons Who Received COVID-19 Vaccination With Messenger RNA Vaccines Between 11 December 2020 and 25 March 2021 in the VA Health Care System and Matched Unvaccinated Controls*
Definition of SARS-CoV-2 Infection and SARS-CoV-2–Related Death
The COVID-19 Shared Data Resource captures all veterans who tested positive for SARS-CoV-2 within the VA system on the basis of PCR tests as well as those with such tests done outside the VA system but documented in VA records. In addition, we used linked Medicare claims to identify any additional cohort members who were diagnosed with COVID-19 (12) on the basis of International Classification of Diseases, 10th Revision, code U07.1 (Medicare records do not include the results of SARS-CoV-2 tests). The earliest date of a documented positive test result or diagnosis of COVID-19 in Medicare claims was taken as each patient's date of infection. Most (56.8%) incident infections were found in VA data, 33.3% in Medicare data, and 9.9% in both data sources.
Similar to prior studies, we defined SARS-CoV-2–related death as death from any cause within 30 days of a positive test result or COVID-19 diagnosis (17, 18, 20, 21). Deaths occurring both within and outside the VA are comprehensively captured in the Corporate Data Warehouse from various VA and non-VA sources, including VA inpatient files, the VA Beneficiary Identification Records Locator Subsystem, U.S. Social Security Administration death files, and the U.S. Department of Defense (27).
Statistical Analysis: Estimates of VE and Primary Outcome
We used Cox proportional hazards regression to compare vaccinated persons and their matched controls with respect to time to development of SARS-CoV-2 infection or SARS-CoV-2–related death, with follow-up for these outcomes extending to 30 June 2021. We calculated the incidence of each outcome and the hazard ratio, unadjusted and adjusted for all of the covariates listed in the preceding section, comparing vaccinated persons with unvaccinated matched controls during each of the following time periods: 14 to 20 days (for Pfizer–BioNTech) or 14 to 27 days (for Moderna) after the first vaccination dose, 21 to 27 days (Pfizer–BioNTech) or 28 to 35 days (Moderna) after the first dose, and 7 or more days after the second dose. Vaccine effectiveness (%) was calculated as 100 × (1 − adjusted hazard ratio).
The study's primary outcome was VE 7 or more days after the second vaccine dose, in agreement with prior studies (1, 6, 7). We did separate analyses with follow-up extending until either 31 March or 30 June 2021 to evaluate whether VE declined over time.
We did the following supplementary analyses: We calculated VE as 100 × (1 − risk ratio), where cumulative risk was calculated as 1 minus the unadjusted Kaplan–Meier estimate, and we used only documented PCR-positive SARS-CoV-2 tests to define incident infection and excluded the cases diagnosed using International Classification of Diseases, 10th Revision, codes in Medicare data.
Role of the Funding Source
The study was supported by the VA Office of Research and Development. The funding sources had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.
Results
Baseline Characteristics of Vaccinated and Matched Unvaccinated Groups
The absolute standardized mean differences and variance ratios of baseline characteristics between vaccinated and unvaccinated persons were calculated for the raw and matched data and showed balance after matching (Appendix Figure, top), as did the distribution of the propensity scores (Appendix Figure, bottom). All baseline characteristics were well balanced between vaccinated persons and their matched unvaccinated controls (n = 2 099 871 in each group) (Table 1); characteristics before matching are shown in Supplement Table 1. The unvaccinated controls consisted of 1 635 948 unique persons, of whom 1 254 709 were matched to a single vaccinee, 310 152 to 2 vaccinees, 60 770 to 3 vaccinees, 9157 to 4 vaccinees, 1047 to 5 vaccinees, 106 to 6 vaccinees, and 7 to 7 vaccinees. A total of 469 472 persons were initially recruited as unvaccinated controls and re-recruited as vaccinees at the time of vaccination.
Appendix Figure. Distribution of baseline characteristics and propensity scores in vaccinated and unvaccinated persons before and after matching.
BMI = body mass index; CAN = Care Assessment Need score. Top. Absolute standardized mean differences and variance ratios of baseline characteristics between vaccinated and unvaccinated persons for the raw and matched data show balance after matching. Bottom. Probability distribution (kernel density) of propensity score between vaccinated and unvaccinated persons for the raw and matched data show balance after matching.
Both vaccinated and unvaccinated groups were predominantly men (92.9% and 93.4%, respectively), had advanced age (mean, 68.7 years in both groups), had diverse racial and ethnic distribution (for example, Black: 17.3% and 17.0%, Hispanic: 6.5% and 6.1%), had substantial comorbidity burden (mean CCI, 2.8 in both groups), and had similar CAN scores (56.6 and 55.7). Major comorbid conditions, such as diabetes, congestive heart failure, chronic obstructive pulmonary disease, and chronic kidney disease, were common and nearly equally distributed in the 2 groups.
Among persons who received both doses of Moderna or Pfizer–BioNTech vaccines, 94.5% received the second dose within 4 days before or after the recommended date. Among persons who received the first vaccine dose, only 2.6% did not receive the second dose within 42 days for Pfizer–BioNTech and 3.0% for Moderna.
Vaccine Effectiveness
During a mean follow-up of 101 days that extended up to 6.5 months to 30 June 2021, a total of 29 141 SARS-CoV-2 infections and 1488 SARS-CoV-2–related deaths were documented. The Figure illustrates the much lower cumulative incidence of SARS-CoV-2 infection (top) and SARS-CoV-2–related death (bottom) in vaccinated versus matched unvaccinated cohorts.
Figure. Kaplan–Meier curve with 95% confidence bands showing cumulative incidence (%) of SARS-CoV-2 infection and SARS-CoV-2–related death in persons who received COVID-19 vaccination versus matched unvaccinated controls, with number at risk and cumulative number of infections or deaths shown at 28-d intervals.
The slight separation in Kaplan–Meier curves between days 0 and 7 is likely related to the standard screening questions before vaccination and postponing vacination in persons who feel sick, including those who may have COVID-19, hence resulting in an artificially slightly lower rate of infection and death in vaccinated persons immediately after vaccination (incident infections or deaths within 0 to 14 d after vaccination were not used in any of the analyses). Top. SARS-CoV-2 infection. Bottom. SARS-CoV-2–related death.
Vaccine effectiveness against infection increased from 31% (CI, 26% to 35%) at 14 to 20 days (for Pfizer–BioNTech) or 14 to 27 days (for Moderna) after the first dose, to 46% (CI, 41% to 50%) at 21 to 27 days (Pfizer–BioNTech) or 28 to 35 days (Moderna) after the first dose, to 65% (CI, 63% to 68%) for the period extending from 7 days after the second dose until 31 March 2021, with little change in VE (69% [CI, 67% to 70%]) when the follow-up extended further to 30 June 2021 (Table 2). Vaccine effectiveness against documented infection 7 days after the second dose seemed to decline with increasing age (75% in persons aged 18 to 64 years, 72% in persons aged 65 to 74 years, and 61% in persons aged ≥75 years) and less so with increasing CCI category (72% in CCI 0 to 1, 69% in CCI 2 to 4, and 65% in CCI ≥5) (Table 2).
Table 2.
Comparison of the Incidence of SARS-CoV-2 Infection in Persons Who Received COVID-19 Vaccination Between 11 December 2020 and 25 March 2021 Versus Matched Unvaccinated Controls and Estimation of VE
Vaccine effectiveness against SARS-CoV-2–related death was 55% (CI, 42% to 64%) at 14 to 20 days (for Pfizer–BioNTech) or 14 to 27 days (for Moderna) after the first dose, 55% (CI, 39% to 67%) at 21 to 27 days (Pfizer–BioNTech) or 28 to 35 days (Moderna) after the first dose, and 89% (CI, 84% to 92%) for the period extending from 7 days after the second dose until 31 March 2021, with little change in VE (86% [CI, 82% to 89%]) when the follow-up extended to 30 June 2021 (Table 3). Vaccine effectiveness against SARS-CoV-2–related death 7 days after the second dose seemed to decline with increasing CCI category (94% in CCI 0 to 1, 88% in CCI 2 to 4, and 83% in CCI ≥5), with no obvious trends by age category (Table 3).
Table 3.
Comparison of the Incidence of SARS-CoV-2–Related Death in Persons Who Received COVID-19 Vaccination Versus Matched Unvaccinated Controls and Estimation of VE
Secondary analyses in which VE was calculated as 100 × (1 − risk ratio) (Supplement Tables 2 and 3) showed minor numerical differences from those with VE calculated on the basis of adjusted hazard ratios. Calculating VE using only SARS-CoV-2 PCR tests with positive results documented in VA records resulted in slightly greater estimates of VE against infection at 7 or more days after the second dose (71% [CI, 69% to 72%]) and almost identical VE against SARS-CoV-2–related death (86% [CI, 82% to 90%]) (Supplement Tables 4 and 5).
Discussion
Our target trial emulation study in the national VA health care system among 2 099 871 persons vaccinated with mRNA COVID-19 vaccines between 11 December 2020 and 25 March 2021 and matched 1:1 to unvaccinated controls estimated that VE at 7 or more days after the second vaccine dose was 69% (CI, 67% to 70%) for SARS-CoV-2 infection and 86% (CI, 82% to 89%) for SARS-CoV-2–related death during follow-up extending to 30 June 2021. Vaccine effectiveness did not decline when follow-up was extended from 31 March to 30 June 2021. Vaccine effectiveness against SARS-CoV-2 infection decreased with increasing age and comorbidity burden. This study advances our understanding of “real-world” mRNA COVID-19 VE by focusing on an older, racially and ethnically diverse, high-comorbidity population; applying target trial emulation methods that strengthen causal inferences; and examining whether VE declines over longer follow-up.
Our estimate of mRNA COVID-19 VE against documented infection (69%) is lower than what has been reported in phase 3 clinical trials; some population-based observational studies (6, 7, 28); and a recent test-negative, case–control study that was also done with VA data from December to March 2021 (8). However, that study did not match cases and controls by the date of testing and did not account for the temporal bias introduced by the fact that SARS-CoV-2 incidence decreased during the observation period (there were 47 357 positive results on SARS-CoV-2 tests in the VA in December 2020 but only 9257 in March 2021) in parallel with a rapid increase in the likelihood of having been vaccinated (Supplement Table 6), creating a spurious association between receipt of vaccination and protection from infection and inflating observed VE. Also, case patients and test-negative controls were not limited to persons who manifested similar degrees of disease severity (for example, symptoms suggestive of COVID-19). Mandatory screening of persons who were not at risk for COVID-19 before medical interventions, procedures, and hospitalizations results in a test-negative population that is not comparable with the test-positive persons.
Adherence to receipt and timing of the second vaccination dose was excellent in our study and, thus, unlikely to have resulted in lower VE. It is likely that the advanced age and high comorbidity burden of the VA population contributed to lower VE. This is supported by the lower VE that we saw in subsets of our population with age 75 years or older or CCI of 5 or greater. Recent studies suggested that only a minority of solid-organ transplant recipients develop antibodies against the spike protein after 2 doses of mRNA vaccines (29, 30), and blunted humoral responses to COVID-19 vaccines have also been reported in patients receiving hemodialysis (31) and patients with hematologic malignancies (32).
Vaccine effectiveness against SARS-CoV-2–related death could not be assessed in the phase 3 randomized controlled trials or even in prior observational cohort studies because of the small number of deaths observed. The only prior cohort study we are aware of that reported VE against SARS-CoV-2–related death evaluated the entire population of Israel aged 16 years or older from January to April 2021 and reported an overall effectiveness of 96.5% (7). However, that study did not attempt to match vaccinated to unvaccinated persons using appropriate target trial emulation methods. The large number of SARS-CoV-2–related deaths that occurred in our cohort enabled relatively precise estimates of VE. Our results confirm the very high effectiveness of mRNA vaccines against SARS-CoV-2–related death, estimated at 86% at least 7 days after the second dose, with follow-up extending to 30 June 2021, even in an elderly population (mean age, 68.7 years) with high comorbidity burden.
We found that VE against infection or death did not decrease when the follow-up was extended from 30 March 2021 (mean follow-up, 36 days) to 30 June 2021 (mean follow-up, 101 days). Our study did not have data on SARS-CoV-2 variants and did not extend beyond 20 June 2021, when the B.1.617.2 (Delta) variant became predominant in the United States (33). Future VA studies could determine if VE declines with even longer follow-up since vaccination, including the period after 30 June 2021 when the Delta variant was predominant.
Our study is limited by a predominantly male population and lack of regular SARS-CoV-2 screening during the observation period. Therefore, asymptomatic or minimally symptomatic infections were not routinely captured. If persons who had vaccination were more likely than unvaccinated persons to be tested even for asymptomatic or minimally symptomatic SARS-CoV-2 infection, then this bias would tend to attenuate the observed effectiveness against documented infection. Although we identified infections documented in the VA as well as Medicare records, positive results from tests done elsewhere may not be captured unless they were recorded in VA or Medicare records. We identified all vaccinations done within the VA health care system, vaccinations done outside the VA that were documented in VA records, and vaccinations of VA enrollees recorded in Medicare records. If additional vaccinations were done in our matched unvaccinated group during the observation period, this would tend to attenuate the observed VE. Although not seen in our results, waning of efficacy over time may reflect declining vaccine protection or result from bias due to a depletion of susceptibles, such that risk in the unvaccinated declines over time due to immunity from unrecognized infection or other reasons, thereby reducing differences in risk between vaccinated and unvaccinated.
In conclusion, in an elderly population of U.S. veterans with high comorbidity burden, COVID-19 VE against infection was substantially lower than previously reported but effectiveness against death was very high.
Supplementary Material
Footnotes
This article was published at Annals.org on 21 December 2021.
References
- 1. Polack FP , Thomas SJ , Kitchin N , et al; C4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603-2615. [PMID: ] doi: 10.1056/NEJMoa2034577 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Jackson LA , Anderson EJ , Rouphael NG , et al; mRNA-1273 Study Group. An mRNA vaccine against SARS-CoV-2—preliminary report. N Engl J Med. 2020;383:1920-1931. [PMID: ] doi: 10.1056/NEJMoa2022483 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Baden LR , El Sahly HM , Essink B , et al; COVE Study Group. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403-416. [PMID: ] doi: 10.1056/NEJMoa2035389 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hall VJ , Foulkes S , Saei A , et al; SIREN Study Group. COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. Lancet. 2021;397:1725-1735. [PMID: ] doi: 10.1016/S0140-6736(21)00790-X [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Thompson MG , Burgess JL , Naleway AL , et al. Interim estimates of vaccine effectiveness of BNT162b2 and mRNA-1273 COVID-19 vaccines in preventing SARS-CoV-2 infection among health care personnel, first responders, and other essential and frontline workers—eight U.S. locations, December 2020-March 2021. MMWR Morb Mortal Wkly Rep. 2021;70:495-500. [PMID: ] doi: 10.15585/mmwr.mm7013e3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Dagan N , Barda N , Kepten E , et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021;384:1412-1423. [PMID: ] doi: 10.1056/NEJMoa2101765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Haas EJ , Angulo FJ , McLaughlin JM , et al. Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. Lancet. 2021;397:1819-1829. [PMID: ] doi: 10.1016/S0140-6736(21)00947-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Butt AA , Omer SB , Yan P , et al. SARS-CoV-2 vaccine effectiveness in a high-risk national population in a real-world setting. Ann Intern Med. 2021;174:1404-1408. [PMID: ] doi: 10.7326/M21-1577 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Chua H , Feng S , Lewnard JA , et al. The use of test-negative controls to monitor vaccine effectiveness: a systematic review of methodology. Epidemiology. 2020;31:43-64. [PMID: ] doi: 10.1097/EDE.0000000000001116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Hernán MA , Robins JM . Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183:758-64. [PMID: ] doi: 10.1093/aje/kwv254 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. U.S. Department of Veterans Affairs. Corporate Data Warehouse. Accessed at www.hsrd.research.va.gov/for_researchers/vinci/cdw.cfm on 22 March 2021.
- 12. U.S. Department of Veterans Affairs. VA Information Resource Center (VIReC) Medicare Data. Accessed at https://vaww.virec.research.va.gov/VACMS/Medicare/Data.htm on 30 May 2021.
- 13. Hall VJ , Foulkes S , Charlett A , et al; SIREN Study Group. SARS-CoV-2 infection rates of antibody-positive compared with antibody-negative health-care workers in England: a large, multicentre, prospective cohort study (SIREN). Lancet. 2021;397:1459-1469. [PMID: ] doi: 10.1016/S0140-6736(21)00675-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Arterburn DE , Olsen MK , Smith VA , et al. Association between bariatric surgery and long-term survival. JAMA. 2015;313:62-70. [PMID: ] doi: 10.1001/jama.2014.16968 [DOI] [PubMed] [Google Scholar]
- 15. Li YP, Propert KJ, Rosenbaum PR. Balanced risk set matching. J Am Stat Assoc. 2001;96:870-82.
- 16. Jann B. KMATCH: Stata module for multivariate-distance and propensity-score matching, including entropy balancing, inverse probability weighting, (coarsened) exact matching, and regression adjustment. Statistical Software Components S458346, Boston College Department of Economics, revised 19 September 2020. Accessed at https://ideas.repec.org/c/boc/bocode/s458346.html on 21 May 2021.
- 17. Ioannou GN , Green P , Fan VS , et al. Development of COVIDVax model to estimate the risk of SARS-CoV-2-related death among 7.6 million US veterans for use in vaccination prioritization. JAMA Netw Open. 2021;4:e214347. [PMID: ] doi: 10.1001/jamanetworkopen.2021.4347 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Ioannou GN , Locke E , Green P , et al. Risk factors for hospitalization, mechanical ventilation, or death among 10 131 US veterans with SARS-CoV-2 infection. JAMA Netw Open. 2020;3:e2022310. [PMID: ] doi: 10.1001/jamanetworkopen.2020.22310 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Fan VS , Dominitz JA , Eastment MC , et al. Risk factors for testing positive for severe acute respiratory syndrome coronavirus 2 in a national United States healthcare system. Clin Infect Dis. 2021 Nov 2;73:e3085-e3094. [PMID: ] doi: 10.1093/cid/ciaa1624 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. King JT Jr, Yoon JS, Rentsch CT, et al.. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the Veterans Health Administration COVID-19 (VACO) Index. PLoS One. 2020;15:e0241825. [PMID: ] doi: 10.1371/journal.pone.0241825 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Ioannou GN , O'Hare AM , Berry K , et al. Trends over time in the risk of adverse outcomes among patients with SARS-CoV-2 infection. Clin Infect Dis. 2021. [PMID: ] doi: 10.1093/cid/ciab419 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Veterans Health Administration, Office of Rural Health. Accessed at https://vaww.vashare.vha.va.gov/sites/ruralhealth/GSOD/default.aspx on 28 March 2021.
- 23.List of Veterans Affairs medical facilities. Accessed at https://en.wikipedia.org/wiki/List_of_Veterans_Affairs_medical_facilities on 7 May 2021.
- 24. Deyo RA , Cherkin DC , Ciol MA . Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613-9. [PMID: ] [DOI] [PubMed] [Google Scholar]
- 25. Charlson ME , Pompei P , Ales KL , et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-83. [PMID: ] [DOI] [PubMed] [Google Scholar]
- 26. Osborne TF , Veigulis ZP , Arreola DM , et al. Automated EHR score to predict COVID-19 outcomes at US Department of Veterans Affairs. PLoS One. 2020;15:e0236554. [PMID: ] doi: 10.1371/journal.pone.0236554 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Sohn MW , Arnold N , Maynard C , et al. Accuracy and completeness of mortality data in the Department of Veterans Affairs. Popul Health Metr. 2006;4:2. [PMID: ] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Lopez Bernal J, Andrews N, Gower C, et al.. Effectiveness of the Pfizer–BioNTech and Oxford–AstraZeneca vaccines on Covid-19 related symptoms, hospital admissions, and mortality in older adults in England: test negative case–control study. BMJ. 2021;373:n1088. [PMID: ] doi: 10.1136/bmj.n1088 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Werbel WA , Boyarsky BJ , Ou MT , et al. Safety and immunogenicity of a third dose of SARS-CoV-2 vaccine in solid organ transplant recipients: a case series [Letter]. Ann Intern Med. 2021;174:1330-1332. [PMID: ] doi: 10.7326/L21-0282 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Kamar N , Abravanel F , Marion O , et al. Three doses of an mRNA Covid-19 vaccine in solid-organ transplant recipients [Letter]. N Engl J Med. 2021;385:661-662. [PMID: ] doi: 10.1056/NEJMc2108861 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Ducloux D , Colladant M , Chabannes M , et al. Humoral response after 3 doses of the BNT162b2 mRNA COVID-19 vaccine in patients on hemodialysis [Letter]. Kidney Int. 2021;100:702-704. [PMID: ] doi: 10.1016/j.kint.2021.06.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Maneikis K , Šablauskas K , Ringeleviciute U , et al. Immunogenicity of the BNT162b2 COVID-19 mRNA vaccine and early clinical outcomes in patients with haematological malignancies in Lithuania: a national prospective cohort study. Lancet Haematol. 2021;8:e583-e592. [PMID: ] doi: 10.1016/S2352-3026(21)00169-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Bajema KL , Dahl RM , Prill MM , et al; SUPERNOVA COVID-19. Effectiveness of COVID-19 mRNA vaccines against COVID-19-associated hospitalization—five Veterans Affairs medical centers, United States, February 1-August 6, 2021. MMWR Morb Mortal Wkly Rep. 2021;70:1294-1299. [PMID: ] doi: 10.15585/mmwr.mm7037e3 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.