COVID-19 Vaccine Effectiveness

To the Editor:

I am often asked variants of the following questions: “Are the various COVID-19 vaccines really good?” and “Is one better than the others?” My typical response is, “Yes, from reading what has been published, they appear to work, and they may be comparable.” Notice that I use several qualifying words—appear and may be. In many reports and briefings, we are told that several vaccines are 94% to 95% effective. For example, in the study by Baden et al¹ of the Moderna messenger RNA 1273 vaccine, the authors concluded that the vaccine “showed 94.1% efficacy at preventing COVID-19 illness.” My first concern is that this and similar statements omit a very important modifying phrase: “in the population studied.” In this same Moderna study, one of the inclusion criteria indicates that volunteers were from “locations or circumstances that put them at an appreciable risk of SARS-CoV-2 infection, a high risk of severe COVID-19, or both.” SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) is the virus that causes COVID-19 illness. However, in this and other trials, we do not know whether study volunteers were actually in close enough contact with persons who were ill from COVID-19 or whether those exposed were exposed to persons with comparable SARS-CoV-2 viral loads or even the same mutations of this virus.

Why do I raise this issue? Shouldn't the large samples sizes and the use of randomization obviate these concerns? Perhaps, but do they? In 2020, the population of the United States was approximately 330 million persons.² Also in 2020, approximately 30.9 million cases of COVID-19 infection were identified and recorded.³ This rate of 9.36% is likely to be an underestimate, given that not all cases are identified and reported. It would not be surprising to find something close to this rate in the placebo-vaccinated cohorts in trials of potential vaccines. In the Moderna trial, symptomatic COVID-19 infection occurred in 185 of the 15,210 placebo-vaccinated participants (1.22%).¹ In the BioNTech/Pfizer trial, infection occurred in 162 of 18,325 placebo recipients (0.88%).⁴ This finding suggests that the samples in these studies were at lower risk of infection than the general population, and thus estimates of vaccine efficacy may not be completely reliable.

Unfortunately, we do not know what features might differentiate those who become infected from those who do not. For both vaccines, a small number of actively treated patients became infected. For the Moderna vaccine, 11 individuals became symptomatic (0.07%), whereas for the BioNTechPfizer vaccine, the comparable figure was 8 of 18,198 (0.04%). Was there anything different about these 19 individuals that made them more vulnerable, or did they receive the vaccine after they were already infected? It is possible that viral load may have been below the limits of detection at the time of screening, resulting in false-negative reporting.⁵

Prevention trials are not the same as treatment trials. Were it ethical to do so, the only way to firmly establish the real efficacy of a preventive vaccine would be to expose both cohorts to a defined amount of the offending infection and then see how many people are prevented from becoming ill (ie, a challenge trial). These trials have been used successfully in diseases such as cholera and dengue fever but pose serious logistical and ethical challenges. Even then there might be unanswered questions. For example, are there identifiable risk factors for vulnerability yet to be discovered?

Currently, we do not know how long induced immunity will last, whether periodic booster shots may be required, or whether new mutations will arise that are resistant to currently available vaccines. We also need to find ways to protect immunocompromised patients. To use a baseball analogy, we have gotten on base safely with the currently available vaccines. However, that is not the same as hitting a homerun or winning the game. Continuing investment is needed to protect us all from these and other viruses that may leap from animals to humans.⁶

The currently available messenger RNA vaccines for preventing SARs-CoV-2 viral infections appear to be effective, even though some important questions remain unanswered. Strong real-world evidence from health care workers and a large population study of Israeli patients confirms the benefits of vaccination.7, 8, 9

Everyone who can safely take these vaccines should do so—to protect both themselves and others. Those who are hesitant or fearful should be helped by professionals and community and religious leaders to overcome their resistance. Only then can we be more sure of being on the path to health and safety.