Estimated Impact of the US COVID-19 Vaccination Campaign—Getting to 94% of Deaths Prevented

In their modeling study, Steele et al¹ seek to quantify the number of lives saved by COVID-19 vaccination in the US compared with a hypothetical scenario in which no vaccine was ever created. The authors conclude that the US vaccination campaign averted 58% of deaths that might have otherwise occurred during the study period. As we pass 1 million individuals in the US dead from COVID-19, the study¹ also makes clear that more must be done to reach those who remain unvaccinated—a challenge that will require substantial investments into national data infrastructure and research into traditionally underfunded areas. How can we improve vaccine access and uptake to save more lives? This is a critical research question, and one that is difficult to answer with currently available knowledge, data, and tools.

To estimate the impact of the vaccination campaign, Steele et al¹ rely on incomplete national data, assumptions about how COVID-19 hospitalizations relate to incident infections and severe outcomes, and imprecise measures of vaccine effectiveness. The piecemeal nature of COVID-19 data highlights an urgent need to develop a comprehensive national data collection infrastructure. Nationally representative data on infections, hospitalizations, deaths, and vaccination are needed to prepare for the next pandemic. A national data infrastructure should be paired with a repository of state and local policies to improve interpretability of quantitative inputs. By increasing transparency, reproducibility, and reliability, a comprehensive national data resource might also help with widespread erosion of trust in government, including its statistics—a problem that has impeded the US vaccination campaign’s reach.

In the absence of a curated national data repository, Steele et al¹ rely on estimates from multiple sources—many of which are themselves estimated, sometimes from the output of other synthetic models. The model’s assumptions affect the study’s conclusions in ways that are difficult to parse and that may also obscure important regional differences in vaccination policy and impact. Like all models, the outputs of this model and its findings are limited by the quality of its inputs and by the limitations of current methods for estimating key variables, such as vaccine effectiveness.

Vaccine Effectiveness Estimates

One of the model’s key parameters is vaccine effectiveness for preventing severe disease and death. Unfortunately, this is also one of the estimates with the most uncertainty. Key design differences between preauthorization randomized clinical trials lead to some noncomparability even among the most robust studies.² Furthermore, although vaccines are initially evaluated in randomized clinical trials, many questions about their effectiveness in the general population are answered by analysis of observational data. Care is required to account for the many biases and confounders that can affect vaccine effectiveness estimates in observational frameworks.³ One of the more common methods of evaluation—test negative designs—are increasingly limited by prior infections in the control group, which can lead to differential misclassification and underestimates of vaccine effectiveness.⁴ Changes in circulating viral variants, the proportion of the population that has already been infected, policy changes, and changes in behavioral response to the pandemic over time mean that vaccine effectiveness is better conceived as a dynamic rather than a static entity. Thus, it cannot be modeled accurately with a point estimate.

Hospitalization Metrics

Steele et al¹ estimate the vaccination campaign impact from hospitalization data from COVID-NET, a network of more than 250 acute care hospitals located in 99 counties in 14 states. Although the COVID-NET surveillance area is generally similar to the US population by demographic characteristics, it is not a representative sample of the US population. In addition, COVID-NET data are not weighted to be representative. COVID-Net defines a COVID-19 hospitalization as any hospitalization within 14 days of a positive COVID-19 test result. However, screening practices for inpatients vary at the hospital and state levels. Some states require universal screening of all inpatients at the time of admission, whereas others do not. Differences in screening practices affect the distribution of severe cases vs nonsevere cases identified.⁵ For example, Massachusetts requires screening of all hospitalized patients and reports both COVID-19 hospitalizations and COVID-19 hospitalizations that required treatment; with universal screening, 30% to 70% of COVID-19 hospitalizations are not primarily for management of COVID-19 respiratory illness, and unvaccinated patients are far more likely to be admitted with severe disease than vaccinated patients. Thus, extrapolating from hospitalization data without accounting for differences in disease severity may underestimate the impact of vaccination for preventing deaths. For similar reasons, extrapolating from hospitalization data likely obscures important regional differences in pandemic responses from variation in vaccine uptake and distribution.

Improving Reach and Impact

We are unlikely to ever know the exact number of people saved by the nationwide vaccination campaign, but we do know that vaccination is our most powerful tool for preventing severe disease and death. Steele et al¹ estimate that vaccination prevented 58% of expected deaths in adults. If that is the case, then the remaining 42% of expected deaths not prevented leaves a lot of room for improvement. The question, then, is how we can do better.

We need evidence-based practices to improve vaccine uptake and to rebuild trust, which will require substantial research investments into new strategies for vaccine promotion.⁶ Educational messaging about the benefits and safety of the vaccine and the dangers of the disease are classic strategies to promote vaccine uptake among vaccine-hesitant parents; however, the effectiveness of this approach is unclear. When Nyhan et al⁷ randomly assigned parents to different vaccine promotion educational materials, they found that none of the educational strategies tested led to increases in parental intention to vaccinate a future child, even when vaccine misinformation was successfully refuted. Furthermore, materials intended to highlight the dangers of disease lead to increases in beliefs about the harms of vaccination—the opposite of the expected outcome. Nyhan et al conclude that “current public health communications about vaccines may not be effective.”⁷ However, public health approaches to vaccine promotion remain rooted in theories of change, such as the health belief model and theory of reasoned action, which were developed in the 1950s and 1960s.⁸ We too often assume that people are vaccine hesitant because they do not have the correct information. However, health communication research suggests that “knowledge is important but not enough to change behavior.”⁹

Some studies suggest paths forward but also highlight major challenges. In a recent study¹⁰ of US veterans, factors associated with vaccine uptake included talking with trusted others, ease of access, and perceptions of family and societal benefits. In at least some cases, the problem is not lack of information but rather lack of trust in the information or in the information delivery system. In the era of communication bubbles, social media, and political polarization, we cannot count on most vaccine-hesitant people to chance upon a trusted friend who promotes vaccination. Research investments are needed to understand the influence of social media, misinformation and disinformation online, and social determinants of vaccine hesitancy. In addition, evidence-informed health promotion strategies are needed to address widespread mistrust in the public health system. The marriage of politics and the pandemic has been a public health disaster that will have consequences for population health that reach far beyond the direct protection of vaccinated individuals.

With 1 in 3 Americans still unvaccinated nationally and wide variation among age groups and by region, we need new strategies. Rebuilding trust is essential, and we need evidence-informed strategies for achieving these critical goals. Identifying and implementing effective interventions will require substantial and sustained investments in health communications research; an understanding of the influence of social media and information bubbles on information spread and impact; public health infrastructure; and boots-on-the-ground, targeted, and individualized approaches. Without strategies that are evidence informed, we may find ourselves continuing to wonder why the same COVID-19 vaccines that can reduce risk of death by up to 94% only managed to prevent 58% of deaths.