Guidance for design and analysis of observational studies of fetal and newborn outcomes following COVID-19 vaccination during pregnancy

Abstract

COVID-19 vaccines are now being deployed as essential tools in the public health response to the global SARS-CoV-2 pandemic. Pregnant individuals are a unique subgroup of the population with distinctive considerations regarding risk and benefit that extend beyond themselves to their fetus/newborn. As a complement to traditional pharmacovigilance and clinical studies, evidence to comprehensively assess COVID-19 vaccine safety in pregnancy will need to be generated through observational epidemiologic studies in large populations. However, there are several unique methodological challenges that face observational assessments of vaccination during pregnancy, some of which may be more pronounced for COVID-19 studies. In this contribution, we discuss the most critical study design, data collection, and analytical issues likely to arise. We offer brief guidance to optimize the quality of such studies to ensure their maximum value for informing public health decision-making.

1. Introduction

COVID-19 vaccines are now being deployed globally. Although pregnant individuals were excluded from pre-licensure trials of COVID-19 vaccines [1], the first randomized controlled trial (RCT) of COVID-19 vaccination in pregnancy began recruitment in February 2021 [2]. Current recommendations in several countries allow for vaccination during pregnancy for those who fall into other priority groups, such as health care workers [3], [4], and many pregnant individuals have already opted to receive a COVID-19 vaccine [5]. Inadvertent vaccination in early pregnancy will also likely occur, since several priority groups contain a high proportion of reproductive-age individuals. Given the current absence of evidence on COVID-19 vaccination during pregnancy, safety monitoring is essential from the outset of large-scale deployment. In addition to traditional pharmacovigilance approaches, comprehensive safety assessment in pregnancy will need to be generated through observational epidemiologic studies in large populations, such as those using routinely-collected data sources like health administrative databases and registries.

There are some unique methodological challenges for observational studies of adverse fetal and newborn outcomes following vaccination during pregnancy that, if not properly addressed, can lead to inaccurate conclusions [6], [7], [8], [9]. In this Short Communication, we discuss the most critical study design, data collection, and analytical issues likely to be faced by studies of COVID-19 vaccination during pregnancy. We draw on lessons learnt from influenza vaccination studies and invoke a hypothetical “target” RCT [10] as a conceptual foundation for considering methodological issues. Finally, we conclude with guidance for optimal design and analysis of observational studies of COVID-19 vaccination during pregnancy to inform public health decision-making.

2. Temporal issues in observational studies of vaccination during pregnancy

2.1. Time-varying exposures

In a hypothetical RCT, follow-up would begin immediately following assignment to vaccination/control group (at time zero). In an observational study, however, timing of vaccination and initiation of follow-up may not temporally align, meaning that an individual’s vaccination status can be “unexposed” (unvaccinated) for part of the pregnancy and “exposed” (vaccinated) for the remainder. If vaccine exposure is classified as a binary variable based on final vaccination status at the time of delivery, the period of pregnancy prior to vaccination becomes “immortal”, because had an adverse pregnancy outcome occurred, it would have been attributed to the unvaccinated group [10]. Since the opportunity to be vaccinated during pregnancy increases with the length of gestation, those who deliver at term have more opportunity to be vaccinated than those delivering earlier, making vaccination appear protective of earlier events (e.g., preterm birth). As illustration, consider the scenario in Fig. 1 panel A in which individuals 1 and 2 were vaccinated at 6 and 8 months’ gestation, respectively, while individual 3 remained unvaccinated throughout the full-term pregnancy. Individual 4, whose pregnancy ended at 6 months’ gestation, had less opportunity to become vaccinated even if it was intended. An analysis that treats individuals 1 and 2 as vaccine-exposed based on their final vaccination status at delivery (shown by solid line) would misclassify the time prior to vaccination, leading to underestimation of the risk of adverse pregnancy outcomes in the exposed group [11]. Panel B depicts the correct approach for classifying vaccination status; follow-up time before vaccination is classified as “unexposed” (dashed line) and time after vaccination is classified as “exposed”. Time-varying exposure classification for COVID-19 vaccines has added complexity, since several require a two-dose schedule. Fig. 1 panel C illustrates how exposure for two-dose vaccines could be accommodated as a three-level time-varying exposure. Although other approaches are possible [11], the most common method for preventing immortal time bias in a cohort study is through the use of Cox proportional hazards regression with a time-varying exposure variable [11], [12] (see list of illustrative studies in eTable 1). Failure to apply such techniques in studies of preterm birth has been empirically shown to bias estimates in favour of vaccination—by up to 10–26% [12], [13]. It is worth noting, however, that studies specifically assessing the relationship between first trimester vaccination and preterm birth, or any other outcome that cannot occur during the first trimester, would not be impacted by immortal time bias.

Fig. 1.

(A) Illustration of immortal time for individuals 1 and 2 (time prior to vaccination is incorrectly classified as “exposed” based on final vaccination status at delivery). (B) Corrected classification of time for individuals 1 and 2 (time prior to vaccination is classified as “unexposed” [dashed line] and exposure status changes to “exposed” [solid line] at time of vaccination). (C) Illustration of possible exposure classification scenarios for two-dose COVID-19 vaccines, showing different gestational ages and intervals between doses. Solid lines represent “exposed”time (following dose 2 of a two-dose vaccine). Dashed lines represent “unexposed”time. Dotted lines representtime after dose 1 but before dose 2 of a two-dose vaccine. Note that Individual 8 in Panel C received the first dose prior to pregnancy and the second dose during early pregnancy (perhaps even before pregnancy recognition), requiring one change in exposure status from post-dose 1 to post-dose 2 (there would be no vaccine-unexposed time). Figure adapted from Platt et al.[11].

2.2. Gestational and calendar time

It is important to ensure that vaccinated and unvaccinated pregnant individuals being compared have similar opportunities for vaccination, based on vaccine availability/distribution, and have similar exposure to background pandemic factors; both were found to temporally confound assessments of vaccination and fetal and newborn outcomes in influenza studies [7], [12]. For example, during the 2009 A/H1N1 influenza pandemic, analyses of preterm birth that did not account for timing of conception nor exclude pregnancies completed before the pandemic A/H1N1 vaccine became available produced estimates that were biased downward (making vaccination appear protective), particularly for assessment of third trimester vaccination [12]. Whereas randomization would ensure similar calendar time alignment of pregnancies in vaccination/control groups in an RCT, an observational study of a vaccine that is available only at particular times (e.g., during a period of intense vaccine deployment) will have variable gestational timing of vaccination that depends strongly on when the pregnancy occurs [12]. Given changes in COVID-19 vaccine supply and allocation, evolving recommendations for use in pregnancy, and variable disease transmission dynamics, these temporal issues are likely to be even more pronounced during this pandemic.

Fig. 2 illustrates a hypothetical scenario that could develop with a period of intensive COVID-19 vaccination that includes pregnant individuals. Those with pregnancies conceived earlier would have a higher probability of being vaccinated in the third trimester once vaccines are available, while pregnancies conceived later would be more likely to be vaccinated in the first or second trimester simply because of when the pregnancy occurs relative to vaccine availability and distribution. Alignment of vaccine-exposed and vaccine-unexposed pregnancies with respect to calendar time can be accomplished using approaches such as matching pregnancies on estimated date of conception, excluding pregnancies having no gestational overlap with time periods when the vaccine is available, and/or including timing of conception in propensity score models that estimate the probability of vaccination [12] (see examples in eTable 1).

Fig. 2.

Timing of pregnancies relative to hypothetical outbreak dynamics and periods of limited and widespread COVID-19 vaccination. Horizontal grey bars depict full-term pregnancies occurring throughout the COVID-19 pandemic time period. Line graph portrays outbreak dynamics. Blue shaded zones illustrate potential vaccine delivery time periods.

3. Confounding in observational studies of vaccination during pregnancy

In an RCT, randomization would ensure that, on average, participants would have an equal likelihood of vaccination as well as an equal distribution of baseline variables that may be associated with the outcome. Conversely, in observational studies, participants typically differ in their probability of being vaccinated and in their probability of experiencing the outcome, resulting in potential confounding bias [6], [14]. Depending on the characteristics of a confounding factor (e.g., prevalence in the vaccine-exposed and vaccine-unexposed groups and strength/direction of association with the outcome), estimates can be biased toward or away from the null value, with the potential to dilute, obscure, or reverse true associations.

Confounding bias has been well-documented in observational studies of influenza vaccination; healthier older adults are more likely to be vaccinated, which exaggerates beneficial effects of vaccination and creates a “healthy vaccinee bias” [15]. In the obstetrical population, the influence of confounding factors is less predictable and may depend on local vaccine recommendations/practices or other factors that affect uptake. For instance in some influenza vaccination studies, uptake was higher among pregnant individuals with pre-existing medical comorbidities that may increase the risk of adverse fetal or neonatal outcomes [16]. Conversely, other studies have documented higher vaccine uptake by pregnant individuals with higher socio-economic status and fewer comorbidities or other risk factors [17].

In principle, emulating the randomization feature of an RCT in an observational study requires information on all confounding factors [14]. In practice, all such factors are never known; even for those that are known or suspected, information may not be available, especially in administrative databases. Health awareness and proclivity to seek vaccination may well be predictive of pregnancy outcome, but challenging to measure and address. Studies should aim to incorporate as much information as possible on covariates known to be associated with vaccination and pregnancy outcomes, as well as proxy variables for unmeasured factors associated with both [18]. These variables must be used in statistical analyses to reduce confounding bias, achievable via different methods including adjustment in regression models. Propensity score based methods, such as inverse probability of treatment weighting and matching, are commonly used in this area of research; while their application does not guarantee removal of all confounding, they have been shown to reduce bias in vaccine studies using large administrative databases [19]. The ability to view and empirically assess the balance of baseline variables by vaccination status after inverse probability of treatment weighting or propensity score matching is an additional advantage (eTable 1).

4. Measurement of priority variables

We recommend that measurement of several variables be prioritized by studies of COVID-19 vaccination during pregnancy; these variables can be classified as: (i) those related to vaccination; (ii) those necessary for measuring outcomes; and (iii) those required to account for confounding (details in eTable 2).

Priority variables in the first category include type of COVID-19 vaccine (as considerations for use in pregnancy may vary by vaccine product [20]), dates of vaccination for all doses received, number of doses, and gestational age in weeks when each dose was received (or equivalently, date of vaccination plus date of conception, or date of vaccination plus date and gestational age at delivery). This information should also be obtained for any doses administered before pregnancy to enable accurate characterization of exposure for individuals who have recently initiated or completed a COVID-19 vaccination series (see earlier discussion in Section 2.1). During the 2009 A/H1N1 influenza pandemic, some surveillance programs only collected information on whether a pandemic vaccine was administered during pregnancy, with no information on gestational or calendar timing of vaccination [7]. Information on both time axes is essential for addressing the temporal issues discussed herein; namely, analyzing vaccination as a time-varying exposure (especially in assessments of outcomes defined by, or highly associated with time, such as preterm birth and stillbirth [11]), and ensuring calendar time alignment of vaccine-exposed and vaccine-unexposed pregnancies [7], [12]. While an in-depth discussion of potential sources of data on vaccination is beyond the scope of this Short Communication, it should be noted that COVID-19 vaccines are being administered in many non-traditional settings (e.g., workplaces), which may not generate a claim in a health administrative database. Studies relying on such databases should recognize this important potential source of exposure misclassification and explore options for procuring alternate data sources, such as vaccine registers maintained by ministries of health and/or public health agencies.

While identifying priority variables for measuring fetal and neonatal outcomes will depend on the specific research question, some general guidance is useful. Although not an exhaustive list of outcomes, we recommend assessment of those proposed by the Brighton Collaboration’s Global Alignment of Immunisation Safety Assessment in Pregnancy (GAIA) initiative (eTable 3) [21]. Since GAIA’s case definitions were developed for clinical studies, their measurement using routinely-collected data sources may be challenging due to a lack of clinical details and/or a gestational age threshold (e.g., ≥20 weeks) for systematic capture of conceptions by some databases and registries [9]. Spontaneous abortion is an important outcome for vaccine safety assessments; however, measurement using health administrative databases is particularly challenging since these data sources can only identify those who were aware of their pregnancy and sought care (e.g., physician visit or emergency department visit for clinical management of early pregnancy loss). This will introduce ascertainment bias if individuals who opt for vaccination are also more likely to seek medical care for a spontaneous abortion. Studies should distinguish between early and later fetal losses (e.g., <12 weeks, ≥12 weeks) if the former are likely to be incompletely ascertained in the data source. For all outcomes, it is imperative that investigators clearly document their study definitions, particularly noting any relevant gestational age thresholds or use of reference standards, since these details are critical for interpretation/appraisal of study findings as well as for future systematic reviews. Heterogeneous and poorly documented definitions of perinatal outcomes have been previously noted in a systematic review of influenza vaccination during pregnancy [22].

Finally, collection/acquisition of data on known confounding variables (or their proxies) is crucial to ensure an unbiased assessment of fetal and neonatal outcomes following COVID-19 vaccination in pregnancy. At the individual level, factors likely to be associated with the exposure and outcomes include sociodemographic factors (e.g., income, education, occupational characteristics, race/ethnicity, maternal age, location/type of residence), health status/behaviors (e.g., smoking, pre-pregnancy obesity, pre-existing medical comorbidities such as hypertension and diabetes, prenatal care attendance), and pregnancy-related factors (e.g., parity, multifetal gestation, history of adverse fetal/neonatal outcome); most health administrative databases and registries traditionally used in vaccination in pregnancy studies typically contain information on the majority of these individual variables. Where possible (e.g., if primary data are being collected or other data sources are available), occupation-related details (e.g., employed during the pandemic, whether a frontline healthcare or other essential worker, able to work from home during pandemic) would be useful as they are likely to be correlated with the propensity to be vaccinated and may also predict pregnancy outcome. Previous history of COVID-19 illness should also be collected as a potential effect measure modifier. At the population level, it is also important to collect setting-specific temporal information about vaccine approval and supply/distribution, recommendations for use pregnancy, as well as other relevant information about the pandemic, such as outbreak dynamics.

5. Conclusions

In addition to traditional methodological challenges faced by all observational epidemiologic studies, those assessing vaccination during pregnancy face additional challenges [9], several of which are likely to be amplified in the context of the COVID-19 pandemic. While the topics addressed in this Short Communication are not exhaustive, application of this brief guidance will help optimize the quality of observational studies of COVID-19 vaccination during pregnancy and, in turn, ensure their maximum value for informing public health decision-making. A final recommendation is that studies of COVID-19 vaccination in pregnancy should involve researchers experienced in both vaccine evaluation and perinatal epidemiology from the beginning to ensure optimal study design and execution [6].

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: [Consulting fees unrelated to this article have been received from Vanderbilt University Medical Center, GLG Market Research, Pfizer, Foundation for Influenza, and Sequirus. Research support unrelated to this article has been paid to his research unit from US NIH, Pfizer, PATH, and WHO.].

Appendix A. Supplementary data

The following are the Supplementary data to this article: