Skip to main content
. 2021 Jun 1;39(30):4013–4024. doi: 10.1016/j.vaccine.2021.05.099

Table 1.

Types of Observational Studies to Measure COVID-19 Vaccine Effectiveness [3].

Type of Observational Study Strengths Weaknesses Resource requirement Comment
Cohort Studies (prospective or retrospective)
  • Results easily communicated to policy makers and stakeholders

  • Can estimate burden of COVID-19 in a population and potentially measure the impact of vaccination

  • Easier to interpret when done early when limited vaccine supply

  • Can potentially be used to study asymptomatic or mildly symptomatic infections

  • Vaccination status difficult to determine in retrospective cohorts without good vaccination records

  • Rt if outcome of interest is uncommon such as severe COVID 19

  • May be expensive, especially if prospective

  • If prospective, possible ethical dilemma in following unvaccinated persons who are recommended for vaccination

High Could be undertaken in certain situations such as among healthcare workers, in institutionalized settings, Health Maintenance Organizations or sentinel hospitals with electronic medical records, or in well circumscribed outbreaks
Case-Control (CaCo) Studies
  • Efficient as requires smaller sample size, as focus on identifying cases rather than following a large population with few cases

  • Less expensive than cohort studies

  • Most people familiar with case-control design

  • Need to choose controls to reflect the population from which cases arise, in terms of exposure to virus and vaccination coverage

  • Vaccinated persons may be more likely to seek, or have access to, health care and become cases, biasing towards reduced VE

  • Misclassification of vaccination status greater compared to cohort studies, especially prospective cohort studies

Moderate Controls should be enrolled at same time as case enrolled in changing incidence setting.
Test-Negative Design (TND) Case-Control Studies
  • Reduces bias of differences in healthcare seeking behavior and access by vaccine status

  • All cases and controls seek care at same facilities, potentially decreasing differences in access to vaccines and community-level confounders

  • Vaccination status often obtained before results of laboratory tests available, minimizing diagnostic bias

  • Can use existing surveillance platforms, such as those for influenza

  • Logistics are simplified, less resource intensive

  • False negative misclassification more likely than CaCo as both cases and controls have COVID-19-like illness.

  • Test-negative controls more likely to be tested for exacerbation of an underlying illness (e.g., COPD), that is an indication for COVID-19 vaccination leading to increased VE.

  • Cases and controls need to be matched or the analysis needs to be adjusted by time

  • Does not remove confounding from common predictors of vaccination and exposure to infection, such as being in a priority group by age or occupation

Moderate Probably most efficient and least biased study design for VE studies of COVID-19 disease in most settings.
Screening Method
  • Markedly reduced expenses since relies on available coverage data and leverages ongoing disease surveillance

  • Do not have to collect data among non-cases since uses vaccine coverage surveys

  • Estimation of expected number of cases who are vaccinated (I.e., breakthrough cases)

  • Coverage survey data may not be representative of population from which cases are being collected (e.g. differences in healthcare access and healthcare seeking behavior)

  • Vaccination status may come from administrative data rather than surveys raising concerns about validity of coverage estimate

  • Must have vaccine status of all reported cases

  • Unable to adjust for individual level covariates

Minimal Rapid rollout makes coverage estimate moving target; disaggregation of coverage data by target populations is difficult. Could be used to determine expected number of cases among vaccinated.
Regression Discontinuity Design
  • Minimizes selection bias as vaccine allocation is based on programmatic criterion

  • Minimizes temporal and geographic trends among the groups

  • Defining the ”neighborhood” around cut-off value for vaccination can be challenging

  • Potentially small sample size

  • Spillover vaccination among those outside cut-off

  • Herd protection among unvaccinated

  • Age cut-offs for vaccination may change rapidly depending on vaccine availability.

Moderate