Abstract
Evidence from early observational studies suggested negative vaccine effectiveness (
) for the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Omicron variant. Since true 
is unlikely to be negative, we explored how differences in contact among vaccinated persons (e.g., potentially from the implementation of vaccine mandates) could lead to observed negative 
. Using a susceptible-exposed-infectious-recovered (SEIR) transmission model, we examined how vaccinated-contact heterogeneity, defined as an increase in the contact rate only between vaccinated individuals, interacted with 2 mechanisms of vaccine efficacy: vaccine efficacy against susceptibility (
) and vaccine efficacy against infectiousness (
), to produce underestimated and in some cases, negative measurements of 
. We found that vaccinated-contact heterogeneity led to negative estimates when 
, and especially 
, were low. Moreover, we determined that when contact heterogeneity was very high, 
could still be underestimated given relatively high vaccine efficacies (0.7), although its effect on 
was strongly reduced. We also found that this contact heterogeneity mechanism generated a signature temporal pattern: The largest underestimates and negative measurements of 
occurred during epidemic growth. Overall, our research illustrates how vaccinated-contact heterogeneity could have feasibly produced negative measurements during the Omicron period and highlights its general ability to bias observational studies of 
.
Keywords: bias, contact heterogeneity, COVID-19, SARS-CoV-2, transmission model, vaccine effectiveness, vaccine efficacy
Abbreviations
- SARS-CoV-2
severe acute respiratory syndrome coronavirus 2
- SEIR
susceptible-exposed-infectious-recovered
- SIR
susceptible-infectious-recovered
-
vaccine effectiveness
-
vaccine efficacy against susceptibility
-
vaccine efficacy against infectiousness
Within 4 weeks of the emergence and in the context of rising cases of Omicron, population-based studies in Canada (1), Denmark (2), and the United Kingdom (3) had reported “negative vaccine effectiveness” against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Vaccine effectiveness (
) is calculated by comparing the rates of infection between vaccinated and unvaccinated individuals. Thus, an observed negative 
measurement suggests that vaccinated individuals were acquiring infections at higher rates than unvaccinated individuals. One potential explanation for the increased infection was that the vaccine increased biological susceptibility, for example, if the virus had evolved to spread faster in vaccinated individuals (4). However, 
measurements are calculated using observational data and thus subject to various biases, including but not limited to differences in testing/detection and exposures among vaccinated and unvaccinated populations (5). Differential exposures by vaccination status could stem from contact heterogeneity.
Contact heterogeneity refers to different levels of contact among and between population subgroups. Increased contact between vaccinated persons, potentially arising due to policies that restrict certain spaces to vaccinated individuals (e.g., vaccine mandates), is one type of contact heterogeneity (hereafter, vaccinated-contact heterogeneity). In this study, we tested: 1) whether vaccinated-contact heterogeneity could lead to observed negative 
measurements; 2) how this relationship is affected by 2 components of vaccine efficacy related to transmissibility: vaccine efficacy against susceptibility (
) and vaccine efficacy against infectiousness (
) (6); and, if negative measurements can be produced, 3) how this mechanism can be identified. As negative measurements of 
are an example of an underestimate, we also explore how vaccinated-contact heterogeneity interacts with vaccine efficacies to influence the degree to which 
is underestimated. We adopt both 
and 
as they are both part of a vaccine’s benefit against transmission, with 
reflecting the reduced probability of vaccinated recipients acquiring infection and 
reflecting the reduced infectiousness of vaccinated individuals if a breakthrough infection occurs. We hypothesize that both vaccinated-contact heterogeneity and the levels of 
and 
contribute to producing measurements of negative 
.
METHODS
To model the dynamics underlying measurements of 
we built a simple compartmental susceptible-exposed-infectious-recovered (SEIR) transmission dynamics model based on Shim and Galvani (7) that included vaccinated and unvaccinated individuals and assumed an all-or-nothing vaccine type ((8); Web Appendix 1 and Web Figure 1, available at https://doi.org/10.1093/aje/kwad055). We also created a complementary susceptible-infectious-recovered (SIR) transmission dynamics model to evaluate how the removal of a latency period (i.e., exposed state) affects measurements of 
(Web Appendix 2). To explicitly account for potential contact differences, the transmission models contained both within-group contact rates for unvaccinated, 
, and vaccinated individuals, 
, as well as between-group contact rates for unvaccinated with vaccinated, 
, and vaccinated with unvaccinated, 
.
In all simulations, we assumed 75% vaccination coverage. We explored 2 different contact scenarios: homogeneous contact, where vaccinated and unvaccinated individuals have equal contacts with random (“proportionate”) mixing, and vaccinated heterogeneous contact where vaccinated individuals have increased within-group contact. In the homogenous contact scenario, we assumed 6 daily contacts per person, reflecting approximate contact rates from the United States and United Kingdom during the pandemic (9), and thus defined 
= 
= 4.5 and 
= 
=1.5. In the vaccinated heterogeneous contact scenario, contacts between vaccinated were increased by 50% compared with the homogeneous contact scenario (
= 6.75), with all other parameter values unchanged. We set the recovery rate to be 1/10 (10), the rate of progression from exposed to infectious (the reciprocal of the incubation period) to be 1/4 (11), and the probability of transmission to be 0.01, such that 
in a fully unvaccinated population with random mixing. Given the uncertainty surrounding vaccine efficacies, we focused our analyses on 2 different baseline values of 
and 
(0.1, 0.5) but also explored the dynamics of higher levels of vaccine efficacies (0.7, 0.9) to determine their effects on 
measurements.
We conducted sensitivity analyses, varying 
and 
from 0.1 to 1 and increasing 
by 0% to 100% from the homogeneous contact scenario rates (
= 4.5–9) to explore their effects on the production of observed negative 
To generalize beyond negative 
, we also conducted additional sensitivity analyses exploring how different levels of vaccinated-contact heterogeneity and vaccine efficacies influence the maximum degree of 
underestimation (i.e., the difference between the true 
and the minimum measured 
per scenario). In our sensitivity analyses, the analyses focused on the minimum 
recorded to illustrate the maximum bias that would be observed in each given scenario. To start our simulations, we introduced 1 infected vaccinated and 1 infected unvaccinated individual into our population.
Following Haber (12), we measured 
as 
, with RR(t) defined as:
 |
(1) |
where 
and 
are the cumulative incidences for vaccinated and unvaccinated groups at time 
, respectively, and 
and 
are the total numbers of vaccinated and unvaccinated individuals, respectively. As 
and 
are from the same simulated population that includes vaccinated and unvaccinated individuals, 
should capture the direct effects of vaccination (i.e., 
) (6, 13). We also tracked how differences in the depletion of the proportion of susceptible vaccinated, 
, and unvaccinated, 
, interacted with 
to influence measurements of 
over time.
RESULTS
Different contact patterns by vaccination status and levels of vaccine efficacies influenced the existence and degree of bias in measurements of 
. First, scenarios of homogeneous contact according to vaccination status never led to underestimated or negative 
and instead, after a short initial period (due to our initial conditions of equal vaccinated and unvaccinated cases), produced accurate measurements of 
(
). Second, scenarios of heterogeneous contact according to vaccination status consistently produced underestimated 
, but resulted in negative 
only in the context of lower vaccine efficacies (
, 
, and 
, 
; Figure 1A). Third, while increased levels of vaccine efficacies reduced the effect of vaccinated-contact heterogeneity on 
measurements, moderately high vaccine efficacies (
) led to 
underestimates when vaccinated-contact heterogeneity was high (100% higher contact; Figure 1A and Web Figure 2).
Figure 1.
Simulation results illustrating how vaccine effectiveness (1 – relative risk) and infection dynamics are influenced by contact patterns and vaccine efficacies. Homogeneous contact rates (equal contacts among vaccinated and unvaccinated individuals) and heterogeneous contact rates (vaccinated have higher contact between vaccinated) interact with vaccine efficacy against susceptibility (
) and vaccine efficacy against infectiousness (
) to influence measurements of vaccine effectiveness over time (A) and the proportion of infected (exposed or infectious) individuals over time (B). Measurements of negative vaccine effectiveness became positive once the proportion of susceptible unvaccinated individuals became lower than the proportion of susceptible vaccinated individuals combined with the level of 
(gray vertical lines; Web Appendix 3). Note that here the heterogeneous contact scenarios assume 50% higher contact between vaccinated individuals compared with the homogenous contact scenarios.
Vaccinated-contact heterogeneity caused measurements of 
to vary across time. In the heterogeneous contact scenarios, the highest underestimates, and the measurements of negative 
, occurred only during periods of epidemic growth (Figure 1A–B). For the scenarios where negative 
was produced, 
became positive only once the proportion of susceptible unvaccinated was lower than the combined proportion of susceptible vaccinated with the proportion immune due to vaccination (i.e., the level of 
Web Figure 3 in Web Appendix 3). Both SEIR and SIR transmission dynamics models produced largely consistent infection and 
dynamics. The main influence of the latency period was on the timing of the negative 
periods: SIR models with heterogeneous contact scenarios resulted in consistently earlier 
crossovers from negative to positive compared with the SEIR models that accounted for a 4-day incubation period (day 25 vs. 51 for 
; day 41 vs. 75 for 
and 
; Figure 1 and Web Figure 4).
In the sensitivity analyses, we found that the maximum negative 
recorded for a given scenario was moderately influenced by 
, and it was strongly influenced by the levels of 
and the contact between vaccinated individuals (Figure 2). For example, when 
was less than 0.15 and 
was 100% higher than the homogeneous contact scenario, 
was strongly negative (less than −0.5). 
was less influential on negative 
, but high levels of 
(>0.9), could still help prevent negative 
even at low 
(0.1). As the production of negative 
is a more extreme example of an underestimate, 
, 
, and the amount of contact between vaccinated individuals also influenced the degree to which 
was underestimated. In general, increasing contact between vaccinated individuals led to larger underestimates, especially when paired with lower 
and, especially, 
(e.g., 
maximum underestimate = 0.7; Web Figure 5). Vaccine efficacies strongly mediated the effect of this heterogeneous contact, with moderate levels of 
and 
resulting in smaller underestimates (e.g., 
, maximum underestimate = 0.07) and very high levels of 
(0.9) resulting in accurate 
measurements (even with 100% higher contact; Web Figure 5).
Figure 2.
Simulation results illustrating the sensitivity of negative vaccine effectiveness (
) measurements to vaccine efficacy (VE) against infectiousness (
), vaccine efficacy against susceptibility (
), and the degree of higher contact between vaccinated individuals. The existence and degree of observed negative 
, measured using the relative risk, was influenced by 
(A), the % increase in contact between vaccinated individuals (i.e., vaccinated-contact heterogeneity bias) (B), and 
(A and B). Colors indicate the maximum negative 
measurement (the minimum 
) observed for a given simulation with >0 indicating a nonnegative measurement. Similar patterns also emerge when measuring the effect of 
, 
, and the vaccinated-contact heterogeneity bias on 
underestimates (Web Figure 5).
DISCUSSION
Our results demonstrated how vaccinated-contact heterogeneity, defined as higher contact levels between
vaccinated individuals, could lead to observed measurements of negative 
. We also identified how these heterogeneous contact patterns could produce underestimated 
when the degree of contact heterogeneity was lower and/or when vaccine efficacies were higher. Thus, we illustrate different plausible scenarios where vaccines can be perceived to be either less beneficial or even harmful despite providing benefits to a population (vaccine efficacies of >0).
Vaccinated-contact heterogeneity can negatively bias measurements of 
. In general, a negative measurement of 
will be observed only when the underestimate (i.e., the degree of downward bias) is larger than the true vaccine benefit. When vaccinated-contact heterogeneity was present, different levels of bias in 
were produced depending on the levels of vaccine efficacies (Figure 2; Web Figure 5), with this bias disappearing when vaccine efficacies were high (e.g., Web Figure 2). As both vaccine efficacies, through their influence on epidemic dynamics, mediate the effect and temporal pattern of the contact heterogeneity bias (Figure 1), observing negative measurements requires the underlying vaccine efficacies to be lower—in particular, lower 
. As 
against Omicron has been found to be consistently lower compared with those recorded for other variants (e.g., Tseng et al. (14)), higher vaccine efficacies and their ability to mediate the effects of vaccinated-contact heterogeneity could explain how this bias could be present before Omicron despite the absence of prior negative measurements.
Beyond testing vaccinated-contact heterogeneity feasibility as a mechanism of bias, we also identified a temporal signature in 
measurements that indicates when this mechanism could be the cause of observed negative 
The only major effect of including/excluding a latency period was the specific timing when negative 
were found. These similarities in temporal signature patterns are due to both SEIR and SIR models generating similar epidemic curves, with the differences in the timing of observed negative values attributable to the stretch factor applied to SEIR epidemic curves that arises due to the added incubation period (15). In the context of vaccinated-contact heterogeneity, negative measurements for both models only occurred during epidemic growth when the proportion of susceptible unvaccinated was higher than the proportion of susceptible vaccinated (mediated by 
; Web Figure 3). In each of the empirical studies, the negative 
measurements coincided with Omicron’s epidemic growth stage (1–3). If measurements of 
are consistently updated and found to change direction later in an epidemic, this would suggest the negative measurement may have been the result of vaccinated-contact heterogeneity. Similarly, if in the future, positive but low measurements of 
become notably higher following the peak of an epidemic, it could signal that vaccinated-contact heterogeneity might be causing 
to be underestimated.
Vaccinated-contact heterogeneity is one possible cause of negative 
, but other biases, such as selection bias via testing access or health-seeking behavior (5), as well as higher immunity among unvaccinated from prior infection could also potentially cause observed negative measurements. Moreover, our analysis focused on an all-or-nothing vaccine type for simplicity, but a leaky vaccine type (16) could impart a different temporal pattern for the vaccinated-contact heterogeneity bias. Additionally, assuming a leaky vaccine type may also result in a different mediating effect of vaccine efficacies on the vaccinated-contact heterogeneity bias, as in this scenario, all vaccinated individuals could potentially become infected given enough exposures. Important next steps for researchers will include exploring other potential biases that may lead to negative 
and including how assumptions surrounding leaky versus all-or-nothing vaccine type may influence 
measurements over time.
Although our study was designed to explain potential mechanisms, and not to specify which values of 
, 
, and contact differences most likely caused observed negative measurements, the findings have important implications for the conduct and interpretation of observational studies measuring 
. When conducting observational studies, researchers should attempt to address vaccinated-contact heterogeneity when measuring 
or at a minimum acknowledge its potential existence and consequences. While differences in exposure risk according to vaccination status have long been recognized as an important source of bias (17); identifying and addressing these differences is and will remain challenging (18). Contact surveys (e.g., POLYMOD (19); B.C. Mix COVID-19 Survey (20)) are an important tool to help identify differences in contact patterns across segments of the population. These types of surveys can be used to address vaccinated-contact heterogeneity specifically by not only recording survey respondents’ numbers of contacts per day but also their vaccination status and the vaccination status of their primary contacts. When paired with mathematical transmission models, which can also be used to estimate vaccine efficacies and effectiveness (7), differences in contact patterns across vaccinated and unvaccinated groups could be explicitly accounted for, thereby eliminating these biases from the estimates. If it is not possible to assess whether vaccinated-contact heterogeneity is present, then reports and public communication should ensure that the interpretation of 
particularly if it is negative or low, includes the possibility of this bias.
Failing to acknowledge vaccinated-contact heterogeneity and other biases that cause observed negative 
can have implications for the management of coronavirus disease 2019 (COVID-19) infection. Specifically, biases that produce apparent negative 
can amplify vaccine mistrust (21) as well as affect vaccine benefit/risk assessments, resulting in less recognition of vaccines as a valid control measure. Hence, in the worst-case-scenario, these biases could inadvertently lead to overall higher COVID-19 transmission and a greater potential for epidemic outbreaks.
Here we have highlighted one possible pathway for 
to be underestimated and even appear negative when vaccines are beneficial. Moreover, we have also outlined how vaccinated-contact heterogeneity can be mediated by both 
and 
and how its presence can be identified via a key temporal signature. Overall, our findings not only illustrate a potential mechanism for negative 
measurements found for the Omicron variant, but also provide a potential explanation for observed negative 
in future studies.
Supplementary Material
ACKNOWLEDGMENTS
Author affiliations: MAP Centre for Urban Health Solutions, Unity Health Toronto, Toronto, Ontario, Canada (Korryn Bodner, Jesse Knight, Mackenzie A. Hamilton, Sharmistha Mishra); Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada (Jesse Knight, Sharmistha Mishra); Division of Epidemiology, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada (Sharmistha Mishra); Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada (Sharmistha Mishra); and Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Ontario, Canada (Sharmistha Mishra).
This work was funded by a Public Health Agency of Canada COVID-19 Immunology Task Force COVID-19 Hot Spots Competition Grant (grant 2021-HQ-000143). S.M. is supported by a Tier 2 Canada Research Chair in Mathematical Modeling and Program Science. J.K. is supported by the Natural Sciences and Engineering Council of Canada (NSERC) Doctoral Award.
R code to perform all simulations and analyses is available on Github (https://github.com/kbbodner/contact_and_NegVE.git).
We thank Drs. Sarah A. Buchan and Jeffrey C. Kwong for their engaging discussions on biases in measurements of vaccine effectiveness as well as Cedric B. Hunter and Winston Irwin for their continued support.
A preprint of this article has been published online. Bodner K, Knight J, Hamilton MA, Mishra S. Higher contact among vaccinated can be a mechanism for negative vaccine effectiveness. medRxiv. 2022. (https://doi.org/10.1101/2022.04.25.22274266).
Conflict of interest: none declared.
REFERENCES
- 1. Buchan SA, Chung H, Brown KA, et al. Effectiveness of COVID-19 vaccines against Omicron or Delta infection [preprint]. medRxiv. 2022. 10.1101/2021.12.30.21268565. Accessed January 27, 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hansen CH, Schelde AB, Moustsen-Helm IR, et al. Vaccine effectiveness against SARS-CoV-2 infection with the Omicron or Delta variants following a two-dose or booster BNT162b2 or mRNA-1273 vaccination series: a Danish cohort study [preprint]. medRxiv. 2021. 10.1101/2021.12.20.21267966. Accessed February 20, 2022. [DOI] [Google Scholar]
- 3. UK Health Security Agency . COVID-19 vaccine surveillance report: 6 January 2022 (week 1).(Research and analysis: COVID-19 vaccine monthly surveillance reports (weeks 39 to 48, 2021 to 2022) (UKHSA gateway number GOV-10975). https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1045329/Vaccine_surveillance_report_week_1_2022.pdf. Accessed January 15, 2022.
- 4. Day T, Kennedy DA, Read AF, et al. The evolutionary epidemiology of pathogens during vaccination campaigns [preprint]. arXiv 2021. 10.48550/arXiv.2109.13680. Accessed January 27, 2022. [DOI] [Google Scholar]
- 5. Weinberg GA, Szilagyi PG. Vaccine epidemiology: efficacy, effectiveness, and the translational research roadmap. J Infect Dis. 2010;201(11):1607–1610. [DOI] [PubMed] [Google Scholar]
- 6. Halloran ME, Longini IM, Struchiner CJ. Design and interpretation of vaccine field studies. Epidemiol Rev. 1999;21(1):73–88. [DOI] [PubMed] [Google Scholar]
- 7. Shim E, Galvani AP. Distinguishing vaccine efficacy and effectiveness. Vaccine. 2012;30(47):6700–6705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Smith PG, Rodrigues LC, Fine PEM. Assessment of the protective efficacy of vaccines against common diseases using case-control and cohort studies. Int J Epidemiol. 1984;13(1):87–93. [DOI] [PubMed] [Google Scholar]
- 9. Liu CY, Berlin J, Kiti MC, et al. Rapid review of social contact patterns during the COVID-19 pandemic. Epidemiology. 2021;32(6):781–791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Walsh KA, Spillane S, Comber L, et al. The duration of infectiousness of individuals infected with SARS-CoV-2. J Infect. 2020;81(6):847–856. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Public Health Ontario . COVID-19 Omicron (B.1.1.529) variant of concern and communicability—what we know so far. https://www.publichealthontario.ca/-/media/Documents/nCoV/COVID-WWKSF/2022/01/wwksf-omicron-communicability.pdf. Published January 17, 2022. Updated January 2023. Accessed May 4, 2022.
- 12. Haber M. Estimation of the direct and indirect effects of vaccination. Stat Med. 1999;18(16):2101–2109. [DOI] [PubMed] [Google Scholar]
- 13. Elizabeth Halloran M, Struchiner CJ, Longini IM. Study designs for evaluating different efficacy and effectiveness aspects of vaccines. Am J Epidemiol. 1997;146(10):789–803. [DOI] [PubMed] [Google Scholar]
- 14. Tseng HF, Ackerson BK, Luo Y, et al. Effectiveness of mRNA-1273 against SARS-CoV-2 Omicron and Delta variants. Nat Med. 2022;28(5):1063–1071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Heng K, Althaus CL. The approximately universal shapes of epidemic curves in the susceptible-exposed-infectious-recovered (SEIR) model. Sci Rep. 2020;10(1):1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Lewnard JA, Tedijanto C, Cowling BJ, et al. Measurement of vaccine direct effects under the test-negative design. Am J Epidemiol. 2018;187(12):2686–2697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. World Health Organization . Evaluation of COVID-19 Vaccine Effectiveness: Interim Guidance 17 March 2021. Geneva, Switzerland: World Health Organization; 2021. [Google Scholar]
- 18. Lewnard JA, Cobey S. Immune history and influenza vaccine effectiveness. Vaccines (Basel). 2018;6(2):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Mossong J, Hens N, Jit M, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008;5(3):0381–0391. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Adu P, Binka M, Mahmood B, et al. Quantifying contact patterns: development and characteristics of the British Columbia COVID-19 population mixing patterns survey. Int J Infect Dis. 2022;116:S30–S31. [Google Scholar]
- 21. Miller A. Canadian COVID-19 vaccine study seized on by anti-vaxxers—highlighting dangers of early research in pandemic. CBC Radio-Canada. https://www.cbc.ca/news/health/covid-19-vaccine-study-omicron-anti-vaxxers-1.6315890. Accessed February 20, 2022. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.