Re: Subramanian and Kumar. Vaccination rates and COVID-19 cases

Subramanian and Kumar’s (2021) correspondence outlines an exploratory group-level analysis between population-level vaccination proportions (in several dozen countries and a few thousand U.S. counties) and increases in rates of cases of SARS-CoV-2 over a seven-day period, compared to a prior seven-day period [1]. This correspondence, while an interesting exercise in analyzing group-level data, is incomplete in several ways, presented without caveats and clarifications crucial for readers to consider when critically evaluating this type of analysis [2]. Clarity in research is especially important during a time of heightened misuse/misinterpretation of statistics regarding vaccination efficacy and effectiveness, when incomplete and/or misinterpreted information may be co-opted for purposes other than authors’ intentions, potentially leading to further vaccine hesitancy within individuals and/or communities, hindering public health efforts to lower infection rates.

One tenet to understanding the health of (a) population(s) outlines that causal differences in health outcomes across populations are not necessarily the same as those within populations, necessitating a multilevel understanding of an issue [3]. The authors’ first sentence in their Findings states, “At the country-level, there appears to be no discernable relationship between percentage of population fully vaccinated and new SARS-CoV-2 cases in the last 7 days.” Training in epidemiologic (or general health research) methods teaches us of inherent issues (e.g. potential for introduction of biases) when using group-level data alone when making causal inferences from group to individual levels. One introductory text explains that when community characteristics [e.g. group-level variables] are ascribed to individuals in that community, validity of these findings is often open to questions [4]. The reason being that group-level associations are sometimes not seen at individual levels or, in some cases, may show a reverse trend. Put simply, this correspondence’s authors, through omission of key information, seem to ascribe effects of a contextual variable of exposure (i.e. vaccination rates) to infer biologic or individual-level causal association. Group-level analyses require clear caveats, something absent from this correspondence. Also, authors’ statements related to trends in country-level data are based only on visual interpretations of the trend line, without inclusion of any statistical tests, which could have been included in the published graphic had a test (e.g. for positive linearity) been performed. They also do not make mention of estimated thresholds required for herd immunity, which are population-level characteristics necessitating consideration when comparing group-level vaccination rates and their effects on incidence of acquisition, hospitalization, and/or death.

In the few instances when individual-level findings from other regions are highlighted (e.g., real-world effectiveness in Israel), the authors do so sparingly and incorrectly. For instance, they state, “Even though vaccinations offers [sic] protection to individuals against severe hospitalization and death, the Centers for Disease Control (CDC) reported an increase from 0.01 to 9% and 0 to 15.1% (between January to May 2021) in the rates of hospitalizations and deaths, respectively, amongst the fully vaccinated.” The representation of these statistics is confusing and, upon closer inspection, incorrect. First, what do authors mean by “severe hospitalization”? When pursuing cited material for more information, interpretations do not refer to “severe hospitalizations”. Further, the “15.1% statistic” represents proportion of deaths among those who were both hospitalized and vaccinated, not among only the vaccinated, as suggested. And, as shown within the same cited source, individual-level findings presented just prior to statistics used by authors in their correspondence outline that vaccination is associated with an 8-fold reduction in disease incidence, a 25-fold reduction in hospitalization incidence, and a 25-fold reduction in incidence of deaths, when comparing vaccinated individuals to the unvaccinated [5]. Additionally, there is a focus on Israel’s vaccine effectiveness without considering higher effectiveness seen in other countries (e.g. Canada, England/Scotland). These extra summaries of individual-level data and context, left out of this correspondence, paint more of a complete picture of protective effects of vaccines against symptomatic infection and detrimental health outcomes that should not be excluded from the limited contextual analysis done by Subramanian and Kumar here.

Finally, additional factors need to be considered with this analysis, including differences in timing and changes in population proportions of vaccinated and unvaccinated individuals, rates of active (i.e. prevalent) cases within these areas during time periods under consideration, changes and evolutions of public health recommendations, policies, and practices (such as mask mandates, physical distancing levels, and timing of lifting of lockdown measures) in these countries/counties, and other population-level characteristics such as population density, and age distributions within populations. While useful for making suggestions about future directions of research (e.g. suggesting individual-level research to appreciate a multi-level understanding of issues), this type of analysis needs to be presented (and accepted for publication) using a critical eye, with the limitations of such an analysis clearly stated. The release of this correspondence, in the hands of those without training to understand and critically appraise these data appropriately, can potentially be harmful when not presented with these caveats.

Declarations

Conflict of interest

The authors have no competing financial or non-financial interests related to the publication of this letter.