Retrospective, Observational Studies for Estimating Vaccine Effects on the Secondary Attack Rate of SARS-CoV-2

Abstract

Coronavirus disease 2019 (COVID-19) vaccines are highly efficacious at preventing symptomatic infection, severe disease, and death. Most of the evidence that COVID-19 vaccines also reduce transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is based on retrospective, observational studies. Specifically, an increasing number of studies are evaluating vaccine effectiveness against the secondary attack rate of SARS-CoV-2 using data available in existing health-care databases or contact-tracing databases. Since these types of databases were designed for clinical diagnosis or management of COVID-19, they are limited in their ability to provide accurate information on infection, infection timing, and transmission events. We highlight challenges with using existing databases to identify transmission units and confirm potential SARS-CoV-2 transmission events. We discuss the impact of common diagnostic testing strategies, including event-prompted and infrequent testing, and illustrate their potential biases in estimating vaccine effectiveness against the secondary attack rate of SARS-CoV-2. We articulate the need for prospective observational studies of vaccine effectiveness against the SARS-CoV-2 secondary attack rate, and we provide design and reporting considerations for studies using retrospective databases.

Abbreviations

COVID-19

coronavirus disease 2019

PCR

polymerase chain reaction

SAR

secondary attack rate

SARS-CoV-2

severe acute respiratory syndrome coronavirus 2

VE

vaccine effectiveness

Coronavirus disease 2019 (COVID-19) vaccines were developed primarily to prevent symptomatic infection and, most importantly, severe disease and death. They have shown high efficacy against these endpoints in experimental and observational studies (1–13). Evidence suggests that these vaccines also prevent infection (5, 14–18) and potentially reduce transmission (19–23), albeit with smaller effects against the highly transmissible Omicron variant compared with wildtype severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and earlier variants (24–26).

While randomized controlled trials play a key role in understanding efficacy against COVID-19 disease, they are inefficient for capturing asymptomatic SARS-CoV-2 infection or transmission, and so most evidence for COVID-19 vaccine effects against these endpoints comes from observational studies. Recent literature has evaluated study designs and analytical approaches for estimating vaccine efficacy against infection (27–30). Here, we focus on another key component of a vaccine’s effect on the overall burden of the epidemic: reducing secondary transmission (30–33).

The secondary attack rate (SAR)—the rate of transmission to susceptible contacts within a transmission unit over a well-defined time period (34)—is a common measure of transmission risk. Vaccine efficacy against the SAR is 1 minus the relative SAR for infected vaccinated individuals versus the SAR for infected unvaccinated individuals (31, 34, 35). Evaluating vaccine efficacy against the SAR of SARS-CoV-2 is challenging in the experimental setting, particularly in the context of an ongoing pandemic (28, 36). It requires frequent surveillance of trial participants to detect all incident infections, even those that are asymptomatic, and enrollment and surveillance of all contacts of infected participants to detect secondary transmission. Phase III COVID-19 vaccine efficacy trials generally captured asymptomatic infections only through periodic SARS-CoV-2 serology, and infectiousness was assessed through the proxy of SARS-CoV-2 viral load in symptomatic individuals; transmission events from infected trial participants were not captured (2, 3, 5–8, 33, 37–40). Phase III efficacy trials also provide limited evidence on variants of concern because the blinded follow-up was largely complete before their emergence.

Given these challenges, most evidence on COVID-19 vaccine effects against the SARS-CoV-2 SAR come from retrospective observational, postlicensure studies, where systematic diagnostic testing is often absent (33). In particular, studies are often done using linked national registries of vaccination and infection data based on household addresses (21–23), and of contact-tracing databases (19, 20). Observational studies can be used to evaluate vaccine effectiveness (VE) against the SARS-CoV-2 SAR. VE against the SAR is influenced not only by biological effects on transmission but also by potential behavioral effects such as behavioral disinhibition and by effects of “real world” vaccine delivery such as variation in timing of dose administration (41, 42). We illustrate the limitations of retrospective, observational study designs for studying VE against the SARS-CoV-2 SAR. First, we highlight the main classes of study designs. Next, we discuss challenges in the ascertainment of infections and transmission events using these designs. We derive an analytical formula for the bias in VE against the SAR introduced by common diagnostic testing strategies in these designs. Finally, we articulate the need for well-designed prospective observational studies of VE against the SARS-CoV-2 SAR, provide study design and reporting considerations for retrospective designs, and highlight areas for methodological innovation that may enable more accurate estimation of VE of the SARS-CoV-2 SAR when retrospective databases are the only option.

RETROSPECTIVE STUDY DESIGNS FOR EVALUATING VE AGAINST THE SARS-COV-2 SAR

Existing studies use 2 primary retrospective designs to evaluate VE against the SARS-CoV-2 SAR: record-linked database studies and contact-tracing database studies. We discuss the features of these designs with a focus on identifying transmission units and transmission events (see Table 1 for examples).

Table 1.

Statistical Designs of Example Studies of COVID-19 Vaccine Efficacy Against SARS-CoV-2 Transmission^a

First Author, Year (Reference No.)	Design	Testing Strategy	Transmission Unit and Susceptible Contacts	Strategy to Exclude Community Transmission	Strategy to Restrict to Transmissions from Primary Subject Only
Eyre, 2022 (19)	Analysis of contact-tracing database.	Event-prompted testing for primary subjects. Only close contacts that received a test were included.	Relative risk of infection in close contacts that received a PCR test within 1–10 days of the index case, comparing index cases who were unvaccinated, partially vaccinated, and fully vaccinated, controlling for vaccination status of contact.	N/A: No predefined transmission unit.	Included only test results from contacts tested within 1–10 days of index case
de Gier, 2021 (20)	Analysis of contact-tracing database. Person can be both a contact and an index.	Event-prompted testing for primary subjects. Encouraged contacts of index case to get tested as soon as possible, and on the fifth day after last exposure.	Relative risk of infection in close contacts of unvaccinated adults compared with adults with confirmed infection who have received partial or full vaccination in the Netherlands. Both pooled and stratified analyses by vaccination status of the contact.	N/A: No predefined transmission unit.	Excluded indexes where their most likely source of infection was in the home. Included only test results from contacts tested within 1–14 days of index case. Required quarantining of contacts after exposure.
Harris, 2022 (21)	Analysis of health-care databases with linked records for households.	Event-prompted testing for primary subject and contacts.	Relative risk of infection in unvaccinated household contacts of vaccinated individuals compared with unvaccinated individuals who also test positive.	Only include contact infections that occurred within 2–14 days of index.	Exclude households with more than 1 person testing positive within 2 days.
Prunas, 2021 (22)	Analysis of health-care databases with linked records for households. Does not assign an index case; models acquisition and transmission events within households over time by imputing infection times.	Event-prompted testing for primary subject and contacts.	Relative risk of infection in household contacts of vaccinated subjects compared with unvaccinated subjects, controlling for vaccination status of contact.	Allow for baseline risk of community-acquired infection in modeling.	Imputation of infection times allows for within-household sources of infections that are not the primary subject in modeling.

Table 1.

Continued

First Author, Year (Reference No.)	Design	Testing Strategy	Transmission Unit and Susceptible Contacts	Strategy to Exclude Community Transmission	Strategy to Restrict to Transmissions from Primary Subject Only
Lyngse, 2022 (23)	Analysis of health-care databases with linked records for households.	Event-prompted testing for primary subjects. Substantial effort put into contact tracing.	Relative risk of infection in household contacts (controlling for vaccination status) of fully vaccinated and boosted individuals who test positive compared with partially or unvaccinated individuals who also test positive. Paper specifically focuses on comparing this relative risk between households with primary Omicron infection versus primary Delta infection.	Only include contact infections that occurred within 1–7 days of index.	Exclude households with more than 1 person testing positive on same day.

Abbreviations: COVID-19, coronavirus disease 2019; N/A, not applicable; PCR, polymerase chain reaction; SAR, secondary attack rate; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

^a Studies were selected as examples to illustrate key elements of the retrospective designs discussed. This is not an exhaustive list of retrospective studies of vaccine efficacy against the SARS-CoV-2 SAR.

Identifying transmission units in retrospective designs

Key to studying SARS-CoV-2 transmission is identifying susceptible contacts of infected individuals. The group of individuals that includes an infectious person and their susceptible contacts is commonly referred to as a transmission unit (34). We use the term index case for the first detected case and primary case for the first infected person in the transmission unit (34, 43). Transmission units may be defined based on a characteristic at the group level, such as a shared residence, or in reference to an individual (e.g., close contacts identified through contact tracing). Identifying susceptible contacts requires defining “susceptible.” Given data suggesting that breakthrough SARS-CoV-2 infections (44–46) and reinfections (47–50) are not uncommon, there are arguments for considering all contacts to be susceptible.

Health-care record linkage to infer transmission units

An important class of retrospective designs defines a household as the transmission unit and identifies SARS-COV-2 transmission events within households by leveraging databases of diagnostic test results; vaccination status, type, and timing; and household addresses. Prunas et al. (22) used data from household members in a database from a health maintenance organization covering 2.5 million individuals in Israel that includes demographic characteristics, health-care utilization data, vaccination data, and SARS-CoV-2 test results. Lyngse et al. (23) used unique identification numbers for residents of Denmark to link reverse-transcription polymerase chain reaction (PCR) and antigen test results from the Danish Microbiology Database with vaccination records in the Danish Vaccination Register and identified transmission units based on residential address. Similarly, Harris et al. (21) linked the national immunization database in England to a data set with all laboratory-confirmed SARS-CoV-2 infections and identified persons sharing the same address. Transmission units captured via record linkage may incorporate some degree of measurement error due to changes in households over the course of the study or nontraditional household structures (51, 52).

Contact-tracing databases to infer transmission units

An alternative retrospective design links existing SARS-CoV-2 contact-tracing databases that include diagnostic test results of contacts of confirmed cases with databases of vaccination records. One example is de Gier et al. (20), where close contacts of confirmed COVID-19 cases were sought out for PCR testing by the Municipal Health Services contact monitoring in the Netherlands, and the test results were combined with the national infectious disease notification registry that contains vaccination records. Similarly, Eyre et al. (19) used data from England’s National Health Service Test and Trace Service, which performs a similar contact monitoring service for individuals who test positive in England. Vaccination status was obtained from the National Immunization Management Service. Transmission units inferred from contact-tracing databases are specific to the definitions, strategies, and function of the tracing service and may be incompletely captured if tracing services are overly conservative in defining contacts of infected individuals, if infected individuals are reticent to disclose their contacts, or if contacts cannot be contacted or tested. Incomplete capture of contacts is a pervasive challenge for contact-tracing systems (53–55). Mutually exclusive transmission units, which are ideal for direct estimation of the SAR, may be difficult to define using contact-tracing databases, where “strings” of infections and contacts are identified.

IMPACT OF TESTING STRATEGY IN RETROSPECTIVE DESIGNS

Retrospective designs, by definition, rely on testing designed for clinical diagnosis or management as opposed to research. One common challenge is “event-prompted” SARS-CoV-2 diagnostic testing, where testing is triggered by occurrence of an event, such as onset of symptoms, potential SARS-CoV-2 exposure, or mandate by social institutions or circumstances, such as travel, employment, or medical procedures (33). Another challenge is infrequent testing (29). In this section, we describe the pitfalls of event-prompted and infrequent diagnostic testing and illustrate their potential biases. In what follows, we consider tests with no potential for false positives for people in the transmission unit. We revisit this issue in the discussion.

Limitations of event-prompted testing

Symptom-prompted testing is common and guaranteed to miss many SARS-CoV-2 infections, given the large burden of asymptomatic infection (56, 57). The proportion of infections that are asymptomatic varies with age, comorbidities, and preexisting immunity (56, 58); method for ascertaining symptoms (59); and infecting variant (60). The extent to which existing databases capture primarily symptomatic testing may vary over time and geography as routine asymptomatic testing of individuals becomes more common or is mandated by social institutions. Importantly, many studies have no information on what prompted the tests captured in the database, which prevents exploration of the impact of the missed infections on VE estimates.

In addition to missed asymptomatic infections, symptom-prompted testing poses challenges in inferring transmission chains because infections are not detected until (and if) symptoms occur, typically several days after acquisition of infection. The incubation period for SARS-CoV-2—the time between onset of infection and symptoms—has been estimated as 6 days on average for ancestral strains, shorter with new variants, and is variable across individuals (61–64).

Limitations of infrequent testing

Even with routine testing, infections may be missed if testing occurs infrequently. The duration of SARS-CoV-2 positivity by diagnostic testing can be as short as 1 day (65–67), especially when vaccinated or previously infected. And yet, even individuals who shed for a short period may still have transmission potential if shedding large amounts of virus (68). Daily PCR testing is likely necessary to capture all SARS-CoV-2 infections, and twice-weekly PCR testing likely captures a supermajority of infections (69–72).

SARS-CoV-2 serology is another tool for capturing current or past infections based on periodic blood collection. While commercial assays detecting antibodies against the nucleocapsid protein (anti-N antibodies) have demonstrated high sensitivity and specificity (73), and anti-N antibody responses are reported to be durable (74) and not elicited by COVID-19 vaccines that target the spike protein, serology has limited value for studies of transmission because it does not inform on timing of infection. Furthermore, serological assays may be insensitive to past infection in vaccinated individuals (75).

Potential impact of event-prompted or infrequent testing on estimation of VE

To illustrate the potential impact of event-prompted or infrequent testing on estimation of VE against transmission, we compare the “target” estimand—the expectation of the statistical estimate of VE against the SARS-CoV-2 SAR—with the “actual” estimands under imperfect testing. We assume a range of scenarios around SARS-CoV-2 infection and transmission (see Web Appendix 1, available at https://doi.org/10.1093/aje/kwad046) that illustrate the issues of symptom-prompted testing (see Web Appendix 2) and infrequent testing (see Web Appendix 3) in isolation; in practice there may be several issues at play.

As shown in Figure 1, under the scenarios we consider, for both symptom-prompted testing and infrequent testing, the actual estimand is no larger than the target estimand. The target and actual estimands agree when either vaccinated individuals have no transmission potential (VE = 1) or when asymptomatic and symptomatic individuals have the same transmission potential. As the transmission potential of breakthrough infections increases, the difference between the estimands increases. This relationship is more pronounced for larger reductions in transmission for asymptomatic infections (Figure 1A). The implication is that, since asymptomatic individuals likely transmit less frequently than symptomatic individuals (76), testing only symptomatic infections may tend to underestimate VE against the SARS-CoV-2 SAR.

Figure 1.

Analytical comparison of actual (y-axis) and target (x-axis) vaccine efficacy against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) transmission estimands. The dashed line indicates equality. A) Difference between target and actual estimands under symptom-prompted testing under varying reduction in transmission potential for asymptomatic versus symptomatic infections. B) Difference between target and actual estimands under infrequent testing under varied frequency of testing. Note the different scales of the 2 x- and y-axes. In (A), the color indicates percent reduction in the secondary attack rate comparing asymptomatic to symptomatic infected people: A value of 100% indicates that asymptomatic infected people cannot transmit severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and a value of 0% indicates that the transmission potential for asymptomatic and symptomatic people is the same. In (B), the color indicates interval between subsequent tests (e.g., a value of 7 means that individuals were tested once every 7 days). “Actual” estimand refers to the estimand that is available given the sampling design, and is derived analytically (see Web Material).

Under infrequent testing, the difference between estimands is a function of the testing interval: When the testing interval is longer than the minimum duration of infection (1 day), the actual estimand is smaller than the target estimand (Figure 1B). In Figure 1B, we show only testing intervals that are shorter than the maximum duration of infection in the vaccinated group (15 days). Generally, the relationship between the testing interval and the difference between the actual and target estimands is not monotonic (see Web Figure 1), and when the interval is longer than the longest duration of infection, the difference between the estimands does not depend on the testing interval. The implication is that infrequent testing would tend to yield an underestimate of VE against the SARS-CoV-2 SAR.

In Figure 1, we assumed correct classification of who infected whom. Figure 2 illustrates how irregular and infrequent testing for SARS-CoV-2 may also misclassify transmission events. In the hypothetical transmission units shown (Figure 2A), both symptom-prompted testing (Figure 2B) and infrequent testing (Figure 2C) underestimate the number of transmission events and result in misclassifying the primary case in at least 1 of the transmission units. Under symptom-prompted testing (Figure 2B), the misclassification is due to a longer presymptomatic period for the primary case vs. the secondary infection. The impact of misclassification of transmission events will depend on the extent and nature of the misclassification; the actual estimand may be higher or lower than the target estimand.

Figure 2.

True and inferred transmission chains for 2 hypothetical transmission units, each with 1 primary case and 3 susceptible contacts. Black dots represent true or inferred transmission events. A) True infection and transmission histories. B) Inferred transmission events based on symptom-prompted testing. C) Inferred transmission events based on infrequent testing. D) Inferred transmission events if community acquisition events are incorrectly included as transmission events. E) Inferred transmission events if contact-to-contact transmissions are incorrectly included as transmission events.

Impact of incomplete testing of transmission unit

Some individuals in a transmission unit may “opt out” of testing entirely or during certain time periods, or there may be limited testing availability. Such incomplete testing is likely nearly universal and is not always reported. Two out of 3 of the household studies listed in Table 1 did not report testing rates; Lyngse et al. (23) reported that about 10% of household members in their study were not tested within 1 week after the index case tested positive. For records-based studies, the most common approach is to assume that individuals not tested are SARS-CoV-2 negative. However, the outcome for untested individuals is missing, resulting in incomplete capture of both primary case and secondary transmission events. For contract-tracing studies, the number of susceptible contacts—the denominator of the SAR—may also be underestimated due to incomplete testing. Without being able to identify whether missingness is “completely at random” (77) or depends on factors such as vaccination or demographic characteristics, it is not possible to determine the direction of the induced bias. We revisit this issue when discussing analytical strategies.

Strategies for confirming potential transmission events

Correct estimation of the SAR requires knowledge of transmission from a primary case to susceptible contacts in the transmission unit. To confirm an infection in a contact as a transmission event, acquisition of infection from the community and contact-to-contact transmission within a transmission unit need to be ruled out. Including community acquisition or contact-to-contact transmission events generally biases the calculation of the SAR upwards (78). The direction and magnitude of the bias in the VE against the SAR will depend on the extent to which contacts of vaccinated vs. unvaccinated primary cases acquire infection from the community and from other transmission unit members.

Community-acquired infections in susceptible contacts.

Figure 2D illustrates the issue with community-acquired infections: We may classify infections diagnosed after the index case as transmission events when in fact the infections are community-acquired. As an example, 2 people may be infected from the same source outside of a household but test positive at different times, so we incorrectly infer that the second infection was a within-household transmission event. One way to account for community acquisition is through modeling, with assumptions around the latent and infectious periods for both vaccinated and unvaccinated individuals (78–81), as applied to SARS-CoV-2 by Prunas et al. (22). Households without any infections other than the index case contribute information about community transmission under this estimation strategy. An alternative approach is to leverage additional data to distinguish between community-acquired and transmission events. One source of information is the timing of diagnosis; for example, de Gier et al. (20) and Harris et al. (21) include only infected contacts who developed symptoms within 2–14 days of the index case. Viral genetic sequence data may also assist in ruling out potential transmission events (82, 83). High-quality SARS-CoV-2 genome sequencing data and epidemiologic data have been used during outbreak investigations (84, 85). Studies of infection clustering have used viral genetic testing to construct and infer phylogenetic trees, including studies of SARS-CoV-2 transmission among university students (86) and US Marine recruits (87). Finally, epidemiologic data can assist in restricting attention to likely transmission events based on close contact. For example, Sikkens et al. (88) used test results and behavioral data to rule out cases of health-care worker–to-patient transmission. Note that these data on viral sequences and epidemiologic data are rarely available in records-linkage studies.

The choices of transmission unit and study duration can also assist in distinguishing between community acquisition events and transmission events. For household studies, the within-household SAR for SARS-CoV-2 during Omicron has been estimated at 50%–70% (24). In contrast, the likelihood of community-acquired infection over a short time period, even in the context of a local outbreak, is generally much smaller. Even during the peak of the Omicron epidemic in New York City, with nearly 700,000 recorded incident infections in January 2022, about 8% of residents were diagnosed with infection (89). In general, the greater the ability of the study to restrict attention to populations with tightly interacting transmission units over the duration of infectiousness for the primary case, the better the chances are that community acquisition events can be excluded. However, this may limit generalizability of the estimates of VE.

Contact-to-contact transmission within the same transmission unit.

A susceptible contact may acquire a SARS-CoV-2 infection from a different contact within the transmission unit, other than the primary case (Figure 2E). Given that a transmission unit is selected as a group of individuals who have frequent, close physical contact, it is difficult to rule out the possibility of contact-to-contact transmission even with timing of diagnosis, viral genetic data, and behavioral data. Failing to account for contact-to-contact transmission may introduce bias in the estimation of the SARS-CoV-2 SAR, and thus bias in the estimation of VE against the SAR (78). One strategy is a model-based approach such as that explored by Prunas et al. (22), where infection times and infection durations are simulated to infer pairwise transmission risk (79). These strategies may help limit misclassification of transmission events.

The SARS-CoV-2 viral load in the index case and infected contact, which is a likely surrogate for transmission potential (90–93), may assist in excluding contact-to-contact transmission events. Prospectively designed studies have the important merit that they can capture full viral load curves, thus greatly enhancing accurate timing of acquisition in the contact and of infectiousness of the index. However, retrospective studies seldom capture viral load in indexes or contacts, as this is not a routine measurement in clinical practice. Some qualitative PCR assays report cycle threshold (Ct) as a surrogate of viral load; however, there is considerable variability across assays, platforms, and specimen types in terms of the reference range and reliability of this measure (91, 92). Even when measured, retrospective studies generally only have the cycle threshold measurement at a single time point—at infection diagnosis—which greatly limits the utility of the measure for classifying transmission events.

DISCUSSION

More than 2 years into the pandemic, SARS-CoV-2 remains a major global public health challenge. As more transmissible variants have arisen, in the context of high rates of vaccine hesitancy in some populations and scarce access to vaccine in other populations, definitive evidence on the overall ability of COVID-19 vaccines to reduce onward transmission of infection is needed.

While there remain arguments for randomized trial designs in settings without access to COVID-19 vaccines (94, 95) and for randomized rollout of vaccines in settings where vaccines are starting to be distributed (33, 36, 96, 97), future studies with randomized designs and clinical endpoints are likely to be few. Furthermore, because capturing transmission events through prospective and frequent testing is resource-intensive, future studies of COVID-19 vaccine effects on transmission will likely be observational.

Observational designs that follow potential transmission units prospectively provide the most rigorous answers. This may be accomplished by prospective enrollment and testing of entire transmission units, prior to any infection diagnosis, and follow-up of all members over a long period spanning an outbreak (33, 98, 99). Cohen et al. initiated such a design in the context of seasonal influenza (100) and pivoted to study SARS-CoV-2 at the onset of the pandemic (101), where individuals within household transmission units performed twice-weekly nasal swabs for 13 months to permit PCR testing regardless of symptoms. Alternatively, a “case-ascertained” approach enrolls transmission units with an incident infection, all of whose members are subsequently tested for a short period (98, 102). Clifford et al. (103) implemented a case-ascertained approach in which household contacts of index cases in the United Kingdom were tested for SARS-CoV-2 on days 1, 3, and 7 after enrollment. Prospective testing of contacts will provide the most accurate capture and classification of transmission events but is resource-intensive.

While health-care record–linkage and contact-tracing designs often leverage large databases that are rich sources of data, they incorporate limitations due to their retrospective nature and the fact that the associated data are collected for nonresearch purposes. Infrequent and irregular testing is likely to miss infections and to selectively capture longer-duration and symptomatic infections (29). Misclassification of the primary cases and secondary transmission events is also possible. There is likely to be incomplete testing of both indexes and contacts, as well as a limited understanding of drivers of testing utilization. Even with correct identification of primary cases, there may be bias due to the inclusion of community-acquired infections and contact-to-contact transmission. We did not consider the potential for false positives, which could occur for a variety of reasons (104). The false-positive rate of PCR (where a false positive is defined as positive for only a single gene at a cycle threshold of >35) has been estimated to be around 0.5% (105). The impact of false positives on estimation of SAR will depend on the underlying incidence of SARS-CoV-2 circulating and the rate of testing.

Finally, factors that are associated with both risk of acquisition or transmission of SARS-CoV-2 and with uptake and timing of COVID-19 vaccination, or with both receipt of SARS-CoV-2 tests and vaccination, are potential cofounders. Behavioral factors that influence pathogen exposure and transmission, uptake of vaccination, and testing strategies are likely not captured in retrospective databases and may lead to unmeasured confounding.

Considerations for statistical analysis

Improved statistical analyses may help in addressing some of the challenges encountered in retrospective studies of VE against the SAR for SARS-CoV-2 and other respiratory pathogens. The fundamental challenge encountered for both record-linked and contact-tracing databases is informatively missing data: missed primary cases and transmission events and misclassified transmission events. Given additional information or knowledge on the mechanism of missingness, it is possible to employ statistical corrections. If drivers of missingness are known or can be inferred from the data, statistical models may be leveraged to model the probability of missing data as a function of measured variables, and inverse-probability weighted (IPW)—or more efficient “augmented” IPW—analyses may be performed to attempt to account for the informative missingness (106, 107). Latent variable analyses that allow for the modeling of unobserved variables may also be helpful (108). Sensitivity analyses are critical tools for evaluating the extent to which study conclusions depend on potential errors introduced by missing or mismeasured primary cases and transmission events (109, 110).

Recommendations for study design

We recommend leveraging studies in populations that have controlled diagnostic testing prompts over the duration of study, such as health-care workers or university students, as this can mitigate the effects of infrequent and event-driven testing. Performing studies in homogeneous study populations, with tightly interacting transmission units, will assist in evaluating the relative likelihood of community acquisition vs. within-unit transmission. Authors should collect contextual information on factors influencing testing uptake and timing, such as relevant public health guidelines and institutional policies and practices, as well as factors influencing transmission, such as guidelines on masking, social distancing, isolation, and quarantine. Contextual data are helpful for interpreting study reliability and generalizability, even if only available at the population level. Finally, using epidemiologic and sequence data in addition to data on the timing of infection diagnosis, will improve the ability to classify transmission events.

Recommendations for reporting results

STROBE guidelines for observational studies (111, 112), and RECORD guidelines for observational routinely collected health data (113) should be followed. Authors should describe the testing program in place during the study, including drivers of testing utilization and timing and the extent of testing completion (e.g., percent of household members receiving a test within 1 week of the index case, see Lyngse et al. Figure 1A (23)), and summarize how these drivers vary by vaccination status. This may include performing sensitivity analyses around factors related to the testing program to describe directions of bias in vaccine efficacy estimates, as in our Figure 1. Authors should provide details about how transmission events were ascertained, the expected level of community transmission that occurred during the study, and how the authors accounted for potential contact-to-contact transmission. In addition to describing which variables were used to control for confounding, we recommend including a discussion of which unmeasured variables may contribute to residual confounding given the study population, time, and local epidemic dynamics. We advise that authors provide careful interpretation of the VE estimand that was estimated given the testing program. Employing these strategies will assist readers in interpreting the VE parameters in context and in gauging the reliability and generalizability of the results.