Evaluation of protection by COVID-19 vaccines after deployment in low and lower-middle income countries

Summary

The availability and use of vaccines for the coronavirus disease 2019 (COVID-19) in low and middle-income countries (L/MICs) lags far behind more affluent countries, and vaccines currently used in L/MICs are predominantly of lower efficacy. As vaccines continue to be rolled out in L/MICs, successful control of COVID-19 by vaccines requires monitoring both of vaccine protection of vaccinees (effectiveness) and of the entire targeted populations, including vaccine herd protection of non-vaccinees (impact). To be of greatest relevance to L/MICs, there is the need to address the distinctive medical and demographic features of populations, health systems, and demography that may greatly affect vaccine performance in these settings. We identified 58 published studies that included 85 evaluations of the effectiveness of different COVID-19 vaccines globally. Only three were done in L/MICs, and no impact studies were identified in these settings. Post-deployment studies of the protection by COVID-19 vaccines rolled out in L/MICs constitute an important but currently neglected global priority.

Introduction

The global coronavirus disease (COVID-19) pandemic has infected over 200 million and killed over 4 million since its inception in late 2019.¹ Although appearing to have been less intense in the world's poorest countries in early waves, COVID-19 has now caused substantial morbidity, mortality, and economic disruption in all countries irrespective of location or level of economic development.¹ COVID-19 vaccine development has proceeded with unprecedented speed. As of this writing six demonstrably safe and efficacious vaccines have received World Health Organization (WHO) “Emergency Use Listing” (EUL), and an additional 17 vaccines have received emergency use authorizations by national regulatory authorities.² Roll out of approved vaccines has occurred rapidly in many of the world's affluent countries. As well, ambitions for achieving high vaccine coverage globally led to creation of the COVAX facility by Gavi (the Vaccine Alliance), CEPI (Coalition for Epidemic Preparedness Innovations) and WHO to ensure that at least 20% of countries’ COVID-19 vaccine needs would be covered by the end of 2021 (∼2 billion doses). However, to date roll out of vaccines in the world's poorer countries has been slow, and, with the exception of a few countries, has not come close to meeting the targeted coverage.³

Recognizing the divergence of vaccine protection that can occur when vaccines are deployed in public health practice in comparison with that measured in idealized, individually randomized, Phase 3 efficacy trials for licensure, and that such trials cannot address all pertinent questions about vaccine protection that arise in practice, in March 2021 the WHO issued guidance for the design and conduct of non-randomized effectiveness studies of vaccine protection for vaccines used in lower middle and lower income (L/MIC) countries. These “effectiveness” studies encompass studies of vaccine protection against COVID-19 in persons who are vaccinated in real life public health programs.⁴ In this Review we consider the rationale for conducting effectiveness studies of COVID-19 vaccines deployed in L/MICs, and then summarize studies that have been conducted up to August 1, 2021. We conclude with a discussion of the implications of these findings, as well as how evaluation of vaccines after deployment might be expanded to provide important evidence for COVID-19 vaccine policy.

Why are effectiveness studies in L/MICs necessary?

Phase 3 trials for licensure of COVID-19 vaccines have been designed as individually randomized trials to ensure that necessary data are generated as rapidly and efficiently as possible, while maintaining the highest scientific and ethical standards. Notably, most Phase 3 trials of efficacy conducted to date have been done in upper and upper middle-income countries. Once vaccines are deployed in real-life public health programs, however, several changes occur that may affect vaccine protection: the range of vaccine recipients may broaden beyond those eligible for trials; vaccines may be given with incomplete or mixed vaccine regimens; dosing intervals may vary; vaccines may not be correctly stored and administered; vaccinees may receive concomitant drugs that are disallowed for vaccine trial participants; and vaccine immunity may wane over time; and genetic variants of COVID-19 may arise. Rare outcomes such as severe disease or death, and functional outcomes such as long haul post-COVID symptoms, may require measurement, as they may not be adequately addressed in trials.⁴ Effectiveness studies of vaccines are needed to provide evidence that addresses these public health realities.

The need for effectiveness studies specifically done in L/MICs is underscored by several additional considerations. First, the existing deployment of COVID-19 vaccines globally has not only been slow for L/MICs but also highly segmented, with high efficacy messenger ribonucleic acid (mRNA) vaccines predominating in many affluent countries, and moderate efficacy adenovirus-vectored and killed whole virion vaccines in poorer countries. Second, vaccine supply constraints and difficulties maintaining recommended dosing intervals in these settings will require post-deployment effectiveness assessments of mix and match vaccine regimens not encountered in wealthier settings as well as vaccine performance with less-than-ideal vaccine regimens. Third, several distinctive features of poor countries and their populations make vaccine protection in practice even more uncertain. Many L/MICs have younger age structures and share a double burden of non-communicable diseases and communicable diseases, as well as malnutrition and HIV infection, all of which might impact vaccine protection. Vaccine protection may be altered in the sprawling, densely populated slums seen throughout the urban developing world, and, conversely, in remote, sparsely settled rural areas with limited ability to meet the thermal storage requirements of several current COVID-19 vaccines. These features may not only affect the generalizability of studies of vaccine effectiveness in L/MICs, but also require consideration as potential confounding variables for evaluations in these settings. Fourth, some L/MIC countries, such as Bangladesh, have extremely high seroprevalence rates of naturally acquired anti-COVID immunity, which may modify vaccine induced immunity and protection.⁵ Fifth, unchecked transmission of COVID-19 in poor countries may facilitate development of new genetic variants with altered susceptibility to vaccine-induced immunity. Delayed recognition of these variants in resource-constrained settings may result in continued use of vaccines conferring poor protection.

We were interested to provide a glimpse of effectiveness studies, using controlled designs recommended by WHO ⁴, that have been undertaken to date. Our interest was not to evaluate the methodological quality or synthesize the findings of the studies, but to appraise the extent to which relevant questions about vaccine protection after introduction have been addressed. We assembled candidate publications published on or before August 1, 2021, using multiple databases and web searches.

Effectiveness evaluation of COVID-19 vaccines: the available evidence

As shown in Table 1, we identified 58 papers that included 85 evaluations of the effectiveness of different COVID-19 vaccines (some of the papers simultaneously evaluated more than one vaccine).6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63 There were no published studies of vaccines that had not received WHO EUL at the time of the survey. Only three studies evaluated vaccines deployed in L/MICs (all three in India), and 79 (93%) of the evaluations were done in high income countries. Reflecting the upper income country bias, 63 (74%) evaluations addressed mRNA vaccines and a further 18 (21%) evaluated the Astra-Zeneca adenovirus-vectored vaccine (a vaccine that has been distributed to countries at all economic levels). We found no effectiveness evaluations of two WHO EUL vaccines (Johnson and Johnson adenovirus-vectored; Sinopharm inactivated whole virion).

Table 1.

Effectiveness evaluations of WHO's emergency use listed vaccines in different country settings^*.

Vaccine evaluated	High-income countries+	Upper-middle income countries	Lower-middle income countries	Low-income countries
BNT162b2(BioNTech/Pfizer), mRNA Vaccine	48	0	0	0
mRNA-1273 (Moderna); mRNA Vaccine	15	0	0	0
ChAdOx1 nCoV-19/AZD1222 (AstraZeneca/ University of Oxford); Viral Vector Vaccine	15	0	0	0
CoronaVac (Sinovac Biotech), Inactivated Vaccine	1	3	0	0
Covishield™(Serum Institute of India), Viral Vector Vaccine	0	0	3	0
Ad26.COV2.S (Johnson & Johnson); Viral Vector Vaccine	0	0	0	0
BBIBP-CorV (Sinopharm/ Beijing Institute of Biological Products); Inactivated vaccine	0	0	0	0

^⁎Table presents 85 vaccine effectiveness evaluations in 58 identified studies (several studies evaluated more than one vaccine simultaneously).

⁺Countries are categorized by income status as defined by the World Bank. High income countries included USA, UK, Canada, Israel, Spain, Italy, Chile, Qatar, and Denmark. The upper-middle income country was Brazil and Lower-middle income country was India.

We were also interested to assess the extent to which the studies assessed modification vaccine effectiveness by factors commonly cited as being of relevance to more affluent countries, but also of importance in the world's poorer countries. Because COVID-19 vaccines had only been in use for a relatively short period of time at the time of our literature search, no study evaluated long term (>6 months) vaccine protection, currently an important issue. As well, we encountered no studies that examined vaccine effectiveness in pregnancy, with mixed regimens of different vaccines, or in combination with use of face masks or other mitigating measures. Table 2 examines selected additional issues examined by these effectiveness studies. Almost all studies used cohort or test-negative (a type of case-control) designs. Forty-eight (56%) evaluations studied vaccine protection in the elderly (60 years and above), eighteen (21%) measured protection in persons with comorbid conditions regarded as placing them at higher risk of severe disease, thirty-three (39%) evaluated protection in healthcare workers, and eleven (13%) assessed protection by ethnic group. Thirty-three (39%) assessed protection in all vaccinees regardless of the number of doses received, and 52 (61%) examined protection by completeness of the vaccine regimen. Twenty (24%) measured protection against asymptomatic infection, and 45 (53%) evaluated protection against severe disease. Fifty-two (61%) examined protection against variants of concern; 26 (31%) for alpha, 8 (9%) for beta, 9 (11%) for gamma, and 9 (11%) for delta, differences likely reflecting the temporal sequence of emergence and geographic span of the variants.

Table 2.

Summary of selected features of effectiveness evaluations of WHO's emergency use listed vaccines irrespective of country economic status.

Vaccine	Study Design			Effectiveness by Host Factors				Effectiveness by Regimen		Effectiveness by Outcome
	Cohort	Test negative case control	Screening method	Health Care Workers	Elderly	Comorbidities	Ethnic groups	One or more doses	Single Dose Vs Two doses	Asymptomatic	Severe disease	Variant of Concern*
												Alpha	Beta	Delta	Gamma
BNT162b2 (BioNTech/Pfizer), mRNA Vaccine (n = 48)	33	14	1	21	29	9	5	21	27	13	23	13	3	4	3
mRNA-1273 (Moderna); mRNA Vaccine (n = 15)	6	9	–	6	7	5	3	3	12	5	9	4	3	1	2
ChAdOx1 nCoV-19/AZD1222 (AstraZeneca/ University of Oxford); Viral Vector Vaccine (n = 15)	8	6	1	1	9	1	2	8	7	2	10	8	1	4	1
CoronaVac (Sinovac Biotech), Inactivated Vaccine (n = 4)	2	2	–	2	2	2	1	1	3	–	1	1	1	–	3
Covishield (Serum Institute of India), Viral Vector Vaccine (n = 3)	2	1	–	3	1	1	–	–	3	–	2	–	–	–	–

^⁎Studies estimating effectiveness against more than one variant were counted individually.

Evidence Gap in effectiveness evaluation in LMICS

Our literature survey identified a large number of published effectiveness studies of COVID-19 vaccines. However, this aggregate of studies is highly skewed, addressing primarily COVID-19 vaccines deployed in the world's more affluent countries, with little attention to vaccines used in L/MICs. The slow roll out of vaccines in L/MICs may partially explain this disparity in evaluations, but this explanation seems incomplete, as vaccines have often been deployed in poor countries in a phased fashion, creating subpopulations for whom vaccine coverage would be high enough to evaluate effectiveness, as has been done in three studies in India.^13,17,24 Moreover, our review found no studies of the effectiveness of the 17 vaccines not yet approved by WHO, which to a large extent have been purchased by or allocated to poorer countries. Without effectiveness studies, the basis for using many of these vaccines may remain unclear. Studies of the effectiveness of vaccines actually being deployed in L/MICs are urgently needed.

The specific questions about vaccine effectiveness currently addressed (Table 2) will need to be expanded to address many distinctive features of L/MICs that may modify vaccine protection. At the same time, our survey identified several gaps in published effectiveness studies that apply to countries of all levels of economic development. Moreover, as the pandemic evolves, all countries will need data on vaccine effectiveness addressing new challenges such as waning vaccine protection, the emergence of new variants, and the need for use of concomitant mitigation measures such as facial masks.

Is effectiveness the only measure of importance in evaluating protection by deployed COVID-19 vaccines?

Effectiveness measures vaccine protection only in persons who are vaccinated. While important, this measure does not fully capture vaccine “impact”, the effect of vaccination on disease in an entire population targeted for vaccination, which reflects not only vaccine effectiveness but also the effect of vaccination in further preventing COVID-19 outcomes by reducing transmission of COVID-19, also known has vaccine herd protection.⁶⁴ Reduction of transmission can lead to protection of non-vaccinees and enhanced protection of vaccinees, amplifying the direct protection of vaccinees resulting only from vaccine-elicited immune responses in these individuals. Vaccine herd protection often becomes first appreciated in the context of real-life mass immunization programs, as individual randomization used in Phase 3 trials is designed to isolate measured direct vaccine protection from vaccine herd effects. Although there is a common misconception that vaccine herd protection is an all-or-none phenomenon, resulting in a complete halt to person-to-person transmission, partial reductions of transmission can result from lower levels of vaccine coverage than those required to stop person-to-person transmission, also known as the herd immunity threshold.⁶⁴

For the foreseeable future, L/MICs will receive predominantly vaccines conferring moderate efficacy, allocated in quantities unlikely to attain high vaccine coverage in at-risk populations. A key question for vaccine impact in the near term is thus how to do more with less. Vaccine impact studies will be critical to designing and evaluating efficient and impactful vaccination strategies that achieve this goal. The importance of measuring population impact of vaccines with moderate efficacy for developing countries is well illustrated by the recent experience with inactivated oral cholera vaccines (OCVs), which provide only moderate (ca. 65%) direct protection against cholera. This moderate efficacy led to skepticism about their public health value until measurement of their impact was undertaken after licensure.⁶⁵ These impact studies demonstrated that the OCVs can provide a high level of indirect protection against cholera to non-vaccinees as well has enhanced protection to vaccinees.66, 67, 68 Application of these findings from field epidemiological studies to dynamic, population-based models of cholera in Bangladesh predicted that vaccine coverage of only about 60% with this moderately efficacious vaccine could nearly extinguish cholera transmission in hyperendemic sites such as Bangladesh (Figure 2).⁶⁹ Importantly, these findings on vaccine impact were critical in influencing decisions to create and fund a global OCV stockpile in 2013, which has deployed over 87 million doses to the world's poorest countries since the stockpile's inception.⁷⁰ At present the WHO strongly endorses the conduct of the effectiveness of COVID-19 vaccines in L/MICs, but due to the greater demands of impact studies, limits its recommendation for impact studies to selected settings.⁴

Figure 2.

Predictions of a dynamic population model of cholera occurrence in Matlab, Bangladesh calibrated to the results of field evaluation of oral cholera vaccine herd protection. Reproduced with permission from Longini et al. ⁶⁹.

Available study designs for measuring impact

Two studies, both undertaken in high income studies, have evaluated the post-deployment impact of having a vaccinated member of the household in reducing transmission to other household members.71, 72, 73 While providing important information on the potential of vaccines to reduce transmission, however, these studies do not measure the overall impact of vaccination on the entire targeted population. Epidemiological models can estimate impact, though without field-based measurements of vaccine herd protection to calibrate these models, the credibility of model predictions may be uncertain. Before versus after, or interrupted time series, studies evaluate the secular trend of COVID incidence relation to introduction of COVID vaccines in the population, though the unpredictable trend of COVID incidence can make such studies difficult to interpret. Our search identified two examples of this approach, applied to national data from Israel.^74,75 Another approach would measure concurrent rates of COVID disease in populations that have been or have not been phased into vaccination programs, which, when phasing is done in a randomized fashion, is called the stepped-wedge design. Such designs may be constrained by rapid rollout of COVID vaccines, which can limit the time interval for measurement of impact. This approach was used in conjunction with the time trend approach to evaluate vaccine impact, including herd protection of children too young to be vaccinated, in Israel.⁷⁶ Notably, no field evaluations of vaccine impact in L/MICs were identified during the time frame for our literature search.

Application of Geographical Information Systems (GIS) to epidemiological studies offers another approach to measuring vaccine herd protection and impact, which, although not yet used for COVID-19 vaccines, has been used for field measurement of herd protection by OCVs, as cited earlier.^77,78 With this approach vaccine herd protection and impact can be estimated by evaluating the occurrence of COVID-19 in each individual in the population in relation to the vaccine coverage of a cluster of surrounding persons demarcated by GIS. This approach can be incorporated into traditional studies of vaccine effectiveness, such as cohort and case-control studies, thus providing information not only about vaccine effectiveness but also about vaccine herd protection of non-vaccinees and enhanced protection of vaccinees. As with other observational study designs, care must be taken to mitigate bias through tactics in design, execution, and analysis.^4,79

Conclusion

In efforts to address the COVID-19 pandemic countries must aim to assure that deployment of vaccines serves their stated aims of pandemic control. To date, these aims have often remained implicit, but can vary widely, from reducing deaths and hospitalizations, to reducing all illnesses.⁸⁰ Whatever the aims, studies of vaccine effectiveness are essential to, but will not be sufficient for guiding vaccination strategies and assessing progress. Also needed will be studies of vaccine impact to address herd protective effects, including protection of non-vaccinees, as well as direct protection of vaccinees. As vaccines in limited quantity and of lower efficacy will likely continue to be distributed to L/MICs in the near future, both types of studies will be crucial to assuring the best possible use of this limited supply for successful COVID-19 control in these settings. Provision of resources and strategic selection of countries for these studies constitute high global priorities.

Search strategy and selection criteria

We assembled candidate publications published up to and including August 1, 2021, by searching databases including PubMed/Medline, Embase, and the Cochrane Database for Systematic Reviews, Global Index Medicus; websites of Centers for Disease Control (CDC) and Governments; preprint databases (bioRxiv, medRxiv); and free-text web searches using Google Scholar without restricting the study period and language. We also employed a “snowball” search strategy to scan useful references used in similar articles retrieved through the initial search strategy. Keywords used for the search, alone and combined were “COVID-19,” “SARS-CoV-2,” “COVID-19 vaccine,” “Effectiveness,” “Protection,” “Real-world,” “Impact,” and specific COVID-19 vaccine names. These keywords were combined using the AND/OR Boolean logic and free text searches. In addition to free text and wildcard searches, Medical Subject Headings (MeSH) terms were used to search databases.

We sought studies done on any COVID-19 vaccine deployed after authorization (national or WHO) but identified only studies on vaccines authorized for emergency use by the WHO (mRNA vaccines by Pfizer and Moderna; adenovirus vectored vaccines by Astrazeneca, Serum Institute of India, and Johnson and Johnson; and inactivated whole virion vaccines by Sinopharm and Sinovac). Next, titles and abstracts of all the articles retrieved through the search strategy were reviewed by two authors according to the inclusion and exclusion criteria presented in Table 3. During the screening process, articles with a conflicting decision to include or exclude were assessed by a third reviewer as a “tie breaker”. For inclusion, articles should fulfill at least one inclusion criteria and none of the exclusion criteria. Once articles were selected for full text review, full texts were retrieved through PubMed, WHO GIFT access, HINARI, or Institutional websites. Full texts were similarly reviewed by two independent authors including review by a third reviewer in cases of conflicting decisions to retain for extraction or exclude. Names of authors from articles in the search results were not blinded for abstract or full-text review. Information on type of vaccine, the methods used for vaccine effectiveness evaluation, the setting where the studies were conducted, and other relevant data were abstracted for each article in a data form.

Table 3.

Inclusion and exclusion criteria for search strategy.

Inclusion Criteria: any of the following

•Effectiveness evaluation of any of the COVID-19 vaccines. •Studies evaluating impact following deployment of any of the Covid-19 vaccines •All types of articles except reviews. Reviews were screened for relevant citations.

Exclusion Criteria: none of

•Articles reporting pre-authorization individually randomized trials. •Studies involving only modeling with no primary data analysis. •Articles presenting only non-vaccine interventions.

We finally performed data extraction in a pretested data extraction form developed in Microsoft Excel 2019. Data extracted from the articles included article information (PMID, first author, year of publication, and country), study design (effectiveness design, setting – country), effectiveness evaluation by subgroups, type of vaccine regimen, non-vaccine cointerventions, and types COVID-19 outcomes. Details of the search can be found in the Figure 1.⁸¹

Figure 1.

Diagram showing the disposition of retrieved citations for assembly of vaccine effectiveness evaluations.

Contributors

JDC, conceived the idea, drafted the Manuscript, reviewed, and summarized the data. ABA, BTT, SSK, FM and JHK contributed to the initial discussion of concept, edited the Manuscript, reviewed, and summarized the data.

Declaration of interests

No conflicts of interest to declare.