Messenger RNA Vaccine Effectiveness Against Coronavirus Disease 2019 Among Symptomatic Outpatients Aged ≥16 Years in the United States, February–May 2021

Abstract

Evaluations of vaccine effectiveness (VE) are important to monitor as coronavirus disease 2019 (COVID-19) vaccines are introduced in the general population. Research staff enrolled symptomatic participants seeking outpatient medical care for COVID-19–like illness or severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) testing from a multisite network. VE was evaluated using the test-negative design. Among 236 SARS-CoV-2 nucleic acid amplification test-positive and 576 test-negative participants aged ≥16 years, the VE of messenger RNA vaccines against COVID-19 was 91% (95% confidence interval, 83%–95%) for full vaccination and 75% (55%–87%) for partial vaccination. Vaccination was associated with prevention of most COVID-19 cases among people seeking outpatient medical care.

With the test-negative design used to evaluate protection from messenger RNA vaccines, vaccine effectiveness against coronavirus disease 2019 among outpatients aged ≥16 years was 91% (95% confidence interval, 83%–95%) for full vaccination and 75% (55%–87%) for partial vaccination.

Randomized controlled trials and real-world effectiveness studies have demonstrated high coronavirus disease 2019 (COVID-19) vaccine effectiveness (VE) against severe outcomes and symptomatic illness among priority groups for vaccination, including healthcare workers and persons aged ≥65 years [1–5]. Following the Advisory Committee on Immunization Practice’s recommendations for COVID-19 vaccine allocation to target populations, states expanded vaccine availability to the general public aged ≥16 years starting in the spring of 2021 [6]. Given the more common clinical presentation of mild to moderate illness compared to severe outcomes, data are needed on VE for the prevention of COVID-19 among persons seeking care for COVID-19–like illness (CLI) in outpatient settings [7].

Since 2008, the US Influenza Vaccine Effectiveness Network (US Flu VE Network) has provided influenza VE estimates annually. The strength of this long-standing active surveillance network includes coupling of clinical and epidemiological data in thousands of patients annually to generate VE estimates midway through each influenza season. These estimates provide decision makers with real-time data to assess VE in the current season and contribute to informing global annual vaccine strain selection decisions. Investigations of VE in outpatient settings can enhance our understanding of protection among persons seeking care for mild or moderate illness, contribute to estimating the averted healthcare burden attributed to COVID-19, and inform community mitigation policies as vaccine coverage continues to increase among adults and adolescents in the United States. We used the robust surveillance platform of the US Flu VE Network to estimate VE against laboratory-confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection among persons aged ≥16 years with COVID-19–like symptoms seeking outpatient care or clinical SARS-CoV-2 testing.

METHODS

We used the test-negative design to evaluate messenger RNA (mRNA) VE against outpatient COVID-19 by comparing vaccine receipt in persons testing positive or negative for SARS-CoV-2 infection [8]. Beginning in March 2020, participating health systems offering outpatient medical care at 5 study sites for the US Flu VE Network in Michigan, Pennsylvania, Texas, Washington, and Wisconsin began active surveillance for COVID-19.

Research staff screened persons who sought outpatient medical care (ie, telehealth, primary care, urgent care, and emergency departments) or clinical SARS-CoV-2 testing using a standard case definition for CLI of an acute onset of fever or feverishness, cough, or loss of taste or smell with symptom duration <10 days [9]. Research staff contacted potentially eligible outpatients by telephone or electronic message to confirm eligibility and enroll consenting participants. In addition to meeting the CLI definition, eligible participants had a clinical or research respiratory specimen collected for SARS-CoV-2 molecular testing within 10 days of illness onset. Standardized questionnaires collected demographic information, general health status, self-reported COVID-19 vaccination, and history of individual respiratory, gastrointestinal, and systemic symptoms experienced during acute illness, as well as potential risk factors for contracting COVID-19, such as working in a healthcare setting and having contact with a person with laboratory-confirmed COVID-19. SARS-CoV-2 nucleic acid amplification test results were used to classify SARS-CoV-2–positive cases and test-negative controls. Research testing, or testing for the purpose of this study, was performed if clinical results were unavailable for study use.

For this analysis, we included participants with illness onset on or after 1 February 2021, for those aged ≥65 years, and on or after 22 March 2021, for those aged 16–64 years; beginning dates of inclusion in analyses varied by site according to local COVID-19 vaccination policies for all persons aged ≥65 or ≥16 years (Supplementary Table 1). We determined vaccination status through participant interviews and verified vaccination based on participant-provided vaccination record cards, documentation of vaccination in electronic medical records, or state immunization information systems. Fully vaccinated participants were defined as those who received 2 doses of an mRNA vaccine (Pfizer-BioNTech BNT162b2 or Moderna mRNA-1273) ≥14 days before illness onset [2, 4]. Partially vaccinated participants were defined as those who received ≥1 dose of an mRNA vaccine ≥14 days before illness onset but who were not fully vaccinated. Those who did not report vaccine receipt and had no documentation of any COVID-19 vaccine before illness were defined as unvaccinated. Participants who received their first dose <14 days before illness (n = 100), had been vaccinated with Johnson & Johnson’s Janssen (JNJ-784367350) vaccine (n = 22), or self-reported vaccine receipt without documentation (n = 35) were excluded.

For each category of COVID-19 vaccination, VE was calculated as 1 − odds ratio of vaccination among SARS-CoV-2 test-positive participants versus test-negative participants (controls), using multivariable logistic regression. Models were adjusted a priori for study site, age in years (continuous), and enrollment period (natural cubic spline with 3 percentile knots of interval between 1 January 2021 and illness onset date). We evaluated sex, race and Hispanic ethnicity, and having had a SARS-CoV-2–positive contact as additional covariates and included race/ethnicity and positive contact in the final models. We also performed sensitivity analyses comparing VE using plausible self-report with documented vaccination, where plausibility was determined by ability to report credible location of vaccination. Statistical analyses were conducted using SAS software, version 9.4. This activity was reviewed by the institutional review boards of the Centers for Disease Control and Prevention (CDC) and other participating institutions and was conducted consistent with applicable federal law and CDC policy.

RESULTS

Between 1 February and 28 May 2021, 27% of outpatients who were contacted for screening and enrollment agreed to participate. Among 812 enrolled participants aged ≥16 years with CLI, 236 (29%) tested positive for SARS-CoV-2. During the study period, 36 positive SARS-CoV-2 specimens from the US Flu VE Network were sequenced, of which 56% were identified as the alpha (B.1.1.7) variant. SARS-CoV-2 positivity was higher among male participants, those identifying as non-Hispanic black, those aged <65 years, and participants enrolled from the Michigan and Pennsylvania study sites (Table 1). Within the enrollment period, SARS-CoV-2 positivity peaked during the second week of April.

Table 1.

Characteristics of Enrolled Participants by Severe Acute Respiratory Syndrome Coronavirus 2 Status, US Influenza Vaccine Effectiveness Network, 1 February to 28 May 2021

Characteristic	Participants, No. (Column %)		P Valuea
	SARS-CoV-2 Positive (Cases) (n = 236)	SARS-CoV-2 Negative (Controls) (n = 576)
Age group, y
16–64	200 (85)	455 (79)	.06
≥65	36 (15)	121 (21)
Study site
Michigan	87 (37)	55 (10)	<.01
Pennsylvania	57 (24)	77 (13)
Texas	18 (8)	124 (22)
Washington	53 (22)	221 (38)
Wisconsin	21 (9)	99 (17)
Sex
Female	145 (61)	408 (71)	<.01
Male	91 (39)	168 (29)
Race/ethnicityb
Black non-Hispanic	52 (23)	49 (9)	<.01
Hispanic	4 (2)	46 (8)
Other non-Hispanic	19 (8)	74 (13)
White non-Hispanic	156 (68)	405 (71)
Underlying conditionc
No	155 (67)	351 (63)	.42
Yes	78 (33)	209 (37)
Contact with COVID-19 case
No/unknown	147 (62)	517 (90)	<.01
Yes	89 (38)	59 (10)
Healthcare workerd
No	209 (91)	469 (85)	.06
Yes	20 (9)	85 (15)
Prior infectione
No	214 (91)	521 (92)	.86
Yes	20 (9)	48 (8)
Time from illness onset to specimen collection, d
0–3	114 (48)	305 (53)	.19
4–6	79 (33)	194 (34)
7–10	43 (18)	77 (13)

Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

^aP values calculated with χ2 test, with differences considered statistically significant at P < .05.

^bInformation on race/ethnicity was missing for 7 participants (5 SARS-CoV-2–positive case patients and 2 test-negative controls).

^cUnderlying conditions (eg, heart disease, lung disease, diabetes, cancer, liver or kidney disease, immune suppression, or high blood pressure) were self-reported, with underlying condition status missing for 19 participants (3 SARS-CoV-2–positive case patients and 16 test-negative controls) .

^dWork in healthcare settings was self-reported, with data missing for 3 participants; participants <18 years old were not asked this question.

^ePrior infection was self-reported, with prior infection status missing for 9 participants (2 SARS-CoV-2–positive case patients and 7 test-negative controls).

Across all vaccinated participants included in the analysis, 226 (62%) received Pfizer-BioNTech, and 138 (38%) received the Moderna vaccine. Among the 236 SARS-CoV-2–positive case patients, 37 (16%) received ≥1 dose of an mRNA COVID-19 vaccine (Table 2). Seventeen (46%) of the 37 case patients who received any vaccine dose were considered fully vaccinated, of whom 15 received Pfizer-BioNTech and 2 received Moderna; 20 (54%) of the 37 were partially vaccinated, of whom 15 received Pfizer-BioNTech and 5 received Moderna. In comparison, 327 (57%) SARS-CoV-2–negative controls were vaccinated with ≥1 dose, of whom 231 (71%) were fully vaccinated and 96 (29%) were partially vaccinated (Table 2).

Table 2.

Estimates of Messenger RNA Vaccine Effectiveness Against Laboratory-Confirmed Coronavirus Disease 2019 Among Outpatients, Using Vaccine Doses Verified by Immunization Documentation

Vaccination Status	Outpatients, No. Vaccinated/Total (% Vaccinated)		VE (95% CI), %
	SARS-CoV-2 Positive (Cases)	SARS-CoV-2 Negative (Controls)	Unadjusted	Adjusteda
Full vaccination	17/216 (8)	231/480 (48)	91 (84–95)	91 (83–95)
Partial vaccination	20/219 (9)	96/345 (28)	74 (56–84)	75 (55–87)

Abbreviations: CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; VE, vaccine effectiveness.

^aVE adjusted for study site, age in years (continuous), enrollment period (natural cubic spline with 3 percentile knots), race/ethnicity, and contact with a SARS-CoV-2–positive person.

The effectiveness of mRNA vaccines against laboratory-confirmed COVID-19 in outpatient settings was 91% (95% confidence interval, 83%–95%) among fully vaccinated and 75% (55%–87%) among partially vaccinated participants (Table 2). VE was similar when using documentation, plausible self-report, or both to classify vaccination status (Supplementary Table 2). In addition, VE including participants who received the Johnson & Johnson’s Janssen vaccine was similar to that of mRNA vaccines.

Discussion

During February–May 2021 when the alpha (B.1.1.7) variant was the predominant circulating strain in the United States, vaccination reduced laboratory-confirmed symptomatic illness by 91% among those fully vaccinated and 75% among those partially vaccinated in a multisite outpatient network evaluating COVID-19 VE [10]. These findings add to evidence from clinical trials of efficacy against symptomatic illness and observational studies of VE across the continuum of illness severity in multiple countries [1–5, 11–13].

Monitoring VE against COVID-19 in outpatient settings is relevant for 3 reasons. First, outpatient settings may better capture younger age groups, which account for an increasing proportion of COVID-19 cases and are likely to present with moderate symptoms, necessitating outpatient care rather than hospitalization [14]. Furthermore, vaccine coverage is lower in younger adults and adolescents, and thus a higher proportion of cases may occur in this age group. Second, people with mild and moderate COVID-19 are rarely hospitalized and are more likely to seek care in outpatient facilities. Thus, when considering logistics of monitoring VE, planning for enrollment and sample size, evaluating duration of protection, and assessing protection against variants of concern in real time is more feasible in outpatient settings. In addition, outpatient networks have the capability to evaluate possible long-term effects of mild and moderate COVID-19 through follow-up surveys and extraction of related postenrollment medical visits and hospitalizations.

The third rationale for monitoring VE against COVID-19 in outpatient settings is that past experiences from the annual influenza vaccination program have demonstrated success of test-negative studies for monitoring VE among people seeking outpatient medical care for respiratory illness. In turn, VE studies in outpatient settings can be conducted in a timely manner and have less potential for confounding than traditional case-control and cohort studies [8]. Thus, implementing VE studies in outpatient settings to monitor vaccine protection could be valuable for informing policy decisions, such as community mitigation strategies and strain selection for vaccines or booster doses.

These results were subject to several limitations. First, because the uptake of Johnson & Johnson’s Janssen vaccine in the general population was limited during the study period, participants receiving the Janssen vaccine were not included in this analysis. Including the 22 vaccinated participants who received the Janssen vaccine (4 case patients and 18 controls) resulted in similar estimates of VE (Supplementary Table 2). Second, surveillance populations at the study sites are not representative of the US population, and this analysis did not evaluate VE by race and Hispanic ethnicity. Additional evaluation of VE among racial/ethnic groups disproportionately affected by the pandemic and among other specific populations, such as persons with underlying health conditions, are needed.

As a third limitation, because we relied on vaccine documentation to determine vaccination status, we may have missed unrecorded vaccine doses. However, estimates including self-reported doses, without documentation, showed similar VE (Supplementary Table 2). Fourth, selection bias may occur in test-negative studies if vaccinated people were more likely than unvaccinated people to seek care or agree to participate in the study, but that would be expected to underestimate VE. Finally, this analysis measured VE against symptomatic COVID-19 caused by SARS-CoV-2 viruses that circulated during the study period, before emergence of the delta (B.1.617.2) variant in the United States. Further enrollment in the US Flu VE Network will evaluate COVID-19 vaccine protection against delta (B.1.617.2) and emerging variants.

As of 29 August 2021, 62% of the US population had received ≥1 dose of a COVID-19 vaccine [15]. A growing number of VE studies have provided evidence that mRNA vaccines confer similar protection against COVID-19 in real-world conditions as in clinical trials, reducing risk of infection and related severe outcomes by ≥90% among those fully vaccinated [1, 2, 12]. In this study, receipt of mRNA vaccines was associated with prevention of most mild to moderate COVID-19 in outpatients seeking medical care or testing in the United States.

Studies should continue to monitor COVID-19 VE against symptomatic illness over time and against variant SARS-CoV-2 viruses to inform vaccination strategies. With the high VE against mild to moderate COVID-19 observed during the study period, early community vaccination strategies likely had a marked impact on disease burden. Efforts to increase vaccination coverage are warranted as the primary prevention strategy, in addition to use of masking, social distancing, and community mitigation strategies for schools, workplaces and gatherings.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. The authors acknowledge additional contributions from Hannah Berger, Joshua Blake, Keegan Brighton, Gina Burbey, Deanna Cole, Linda Heeren, Erin Higdon, Lynn Ivacic, Julie Karl, Sarah Kopitzke, Erik Kronholm, Jennifer Meece, Nidhi Mehta, Vicki Moon, Cory Pike, Carla Rottscheit, Jackie Salzwedel, Marshfield Clinic Research Institute, Marshfield, Wisconsin; Alanna Peterson, Linda Haynes, Erin Bowser, Louise Taylor, Karen Clarke, Krissy Moehling Geffel, Todd M. Bear, Klancie Dauer, Heather Eng, Monika Johnson, Donald B. Middleton, Jonathan M. Raviotta, Theresa Sax, Miles Stiegler, Joe Suyama, Alexandra Weissman, and John V. Williams, University of Pittsburgh Schools of the Health Sciences and University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania; Adam Lauring, Joshua G. Petrie, Lois E. Lamerato, E. J. McSpadden, Caroline K. Cheng, Rachel Truscon, Samantha Harrison, Armanda Kimberly, Anne Kaniclides, Kim Beney, Sarah Bauer, Michelle Groesbeck, Joelle Baxter, Rebecca Fong, Drew Edwards, Weronika Damek Valvano, Micah Wildes, Regina Lehmann-Wandell, Caitlyn Fisher, Luis Gago, Marco Ciavaglia, Kristen Henson, Kim Jermanus, and Alexis Paul, University of Michigan, Ann Arbor, and Henry Ford Health System, Detroit, Michigan; Eric Hoffman, Martha Zayed, Marcus Volz, Kimberly Walker, Arundhati Rao, Manohar Mutnal, Michael Reis, Lydia Requenez, Amanda McKillop, Spencer Rose, Kempapura Murthy, Chandni Raiyani, Natalie Settele, Jason Ettlinger, Courtney Shaver, Elisa Priest, Jennifer Thomas, Alejandro Arroliga, and Madhava Beeram, Baylor Scott & White Health, Temple, Texas; C. Hallie Phillips, Erika Kiniry, Stacie Wellwood, Brianna Wickersham, Matt Nguyen, Rachael Burganowski, and Suzie Park, Kaiser Permanente Washington Research Institute, Seattle, Washington.

Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Vaccination data from Pennsylvania were supplied by the Bureau of Health Statistics & Registries, Pennsylvania Department of Health, Harrisburg. The Pennsylvania Department of Health specifically disclaims responsibility for any analyses, interpretations, or conclusions.

Financial support. This work was supported by the US Centers for Disease Control and Prevention (cooperative agreements U01IP001034–U01IP001039) and the National Institutes of Health (grant UL1TR001857).

Potential conflicts of interest. M. P. N. reports grants from Merck, outside the submitted work. R. K. Z. reports grants from Sanofi Pasteur, outside the submitted work. G. K. B. reports grants from Merck, outside the submitted work, and consulting fees from New World Medical. E. T. M. reports grants from Merck, outside the submitted work, and consulting fees from Pfizer. A. S. M. reports consulting fees from Sanofi Pasteur and Seqirus. L. E. L. reports grants from Xcenda, eMAXHealth, AstraZeneca, Pfizer, and Evidera, outside the submitted work. M. L. J. reports grants from Sanofi Pasteur. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.