Table 2.

Potential biases of COVID-19 vaccine effectiveness studies [3].

Bias	Description	Designs affected*	Typical Magnitude	Direction on VE estimate	Outcomes / subgroups in which VE affected	Methods to minimize bias	Comments
Care-seeking behavior/access to care	Those more likely to get vaccine seek care more, thus more likely to be cases	CaCo, cohort	Large	Decrease	Non-severe more than severe disease	Use TND; enroll only severe patients.	TND partially addresses, but can create collider bias [32]
Care-seeking based on vaccine status	Vaccinated persons less likely to seek care/testing due to COVID-19-like illness due to perception of protection	All	Small-moderate	Increase in CaCo and cohort; decrease in TND, if vaccine confers some protection.	Non-severe more than severe disease	Smaller magnitude in TND	Might partially offset care-seeking behavior/better access bias
Collider bias [32]	Health-seeking and SARS-CoV-2 infection both lead to testing	TND	unknown	Unknown, depends on how health-seeking and infection affect testing	Non-severe more than severe disease	Limit to severe patients; limit to older adults
Confounding other than by factors mentioned above	Occurs when there are common causes of receipt (or lack of receipt) of vaccine and risk of SARS-CoV-2 exposure	All	Unknown	Unknown (depends on direction risk of vaccination and exposure are affected)	All	Stratification, regression adjustment, or matching for potential confounders (e.g., HW occupation)	It is important to collect high quality data on potential confounding factors, particularly adherence to NPI. Example of healthy vaccinee effect
Diagnostic bias	HWs more likely to test unvaccinated persons for COVID-19	All	Varies on setting	Increases	Non-severe more than severe disease	Test all persons or a systematic random sample meeting protocol-specified case definitions
Misclassification of the outcome	False negatives (persons with COVID-19 disease who test negative)	TND > CaCo, cohort [23]	Small	Decrease	Severe disease more affected due to later presentation for testing	Use a highly sensitive test; limit to illness onset ≤ 10 days; exclude TND controls with COVID-19-specific symptoms (e.g. loss of taste)	Rapid tests currently have lower sensitivity than PCR; If vaccination shortens shedding time, could lead to increased estimate of VE.
Misclassification of the outcome	False positives (persons without COVID-19 disease who test positive)	TND > CaCo, cohort	Small	Decrease	All	Limit to illness onset ≤ 10 days, use highly specific test, use of clinical case definition for enrollment.	Possible chronic shedder/persistent PCR positive who is ill from another cause, but likely rare; could be more problematic when incidence is high.
Misclassification of the exposure	Vaccine effect may start before/after specified cutoff for considering individual vaccinated	all	Large but can be nearly eliminated by design	Decrease	All	Exclude from primary analysis outcomes occurring in periods of ambiguous vaccine effect, e.g. 2 weeks after first dose	Particular concern for COVID-19 when rollout is fast and large proportion of follow-up time and cases will occur soon after vaccination.
Nonspecific vaccine effect	Vaccine prevents diseases for which controls seek care	TND	Small (has not been shown)	Either; depends if vaccine increases or decreases other diseases	All	Exclude controls with diseases possibly affected by COVID-19 vaccines [33]	E.g., adenovirus-vector vaccines might prevent adenovirus illness
Prior infection	If known prior SARS-CoV-2 infection, less likely to get vaccinated	All	Small-moderate (depends on seroprevalence / past incidence of infection)	Decrease	All	Sensitivity analysis excluding those with prior SARS-CoV-2 by history or lab	Assumes prior infection confers immunity. Asymptomatic prior infection could occur in risk group targeted for early vaccine (e.g. HWs)
Spurious waning	Unvaccinated individuals become immune through natural infection faster than vaccinated [34]	All	Small soon after vaccine campaign, large with increasing time since campaign	Decreases with time since vaccination	VE of duration of protection	Do VE study soon after vaccine introduction; anchoring in time of cases and controls	Occurs with “leaky” vaccine that partially protect against infection and there is high incidence of infection [35]
Survivorship	Unvaccinated more likely to die of COVID-19	All	Small	Decrease	Severe disease; high-risk mortality groups	Quantify percent of COVID-19 deaths in non-study population who were vaccinated. If conducting inpatient evaluation, attempt to enroll fatal cases	Refers to deaths of person before they would have chance to be enrolled in study

^*Designs include traditional case-control (CaCo), test-negative design case control (TND), and cohort studies