Evaluating COVID-19 vaccines in the real world

The effectiveness of the mRNA vaccines in preventing COVID-19 disease progression in 2021 set new expectations about the role of prevention interventions for the disease. Efficacy observed in the trials was more than 90%.1, 2 The efficacy of other vaccines evaluated in large randomised trials, such as the Oxford–AstraZeneca (70%) and Sputnik V (91%) vaccines, have been criticised for elements of the trial conduct and concerns about safety.3, 4 For that reason, mRNA vaccines have been most widely distributed in wealthier settings while other vaccines, such as Sinopharm and Sinovac, with some exceptions,³ have been provided in low-income and middle-income countries. The opportunity for head-to-head clinical trials evaluating vaccine comparative effectiveness no longer exists because so many people have received at least one dose of a vaccine. What happens in a clinical trial might also differ from the experiences of actual vaccine programmes and roll-outs. Until now, the real-world effectiveness of these lower profile vaccines has not been well established.

In The Lancet, Analia Rearte and colleagues⁵ report a real-world evaluation of the effectiveness of the rAd26-rAd5 (Sputnik V), ChAdOx1 nCoV-19 (Oxford–AstraZeneca), and BBIBP-CorV (Sinopharm) vaccines. The large country-wide study in Argentina involved 1 282 928 individuals (693 170 [54·0%] women) aged 60 years or older. The two viral vector vaccines, rAd26-rAd5 (93·1% [95% CI 92·6–93·5]) and ChAdOx1 nCoV-19 (93·7% [93·2–94·3]), were both more effective at preventing death due to COVID-19 than the inactivated virus BBIBP-CorV vaccine (85·0% [84·0–86·0]) after two doses. Even with only one dose of any of the three vaccines, the effectiveness of preventing death was more than 70%.

The work by Rearte and colleagues⁵ is an important contribution that has ramifications for many low-income and middle-income countries because the choice of vaccines available in a country is rarely dependent on the effect size observed in the primary clinical trials, but rather on procurement negotiations with various manufacturers early on in the pandemic. Therefore, the findings are reassuring that these lower profile vaccines offer important prevention benefits, including in people older than 60 years who had not been adequately recruited in primary studies of the vaccines. The findings reinforce the advantages of additional dosing of the vaccines to improve prevention and reduce mortality; however, they also show that a single dose offers a large prevention benefit, supporting the widespread implementation of single dosing until other doses are available.

Rearte and colleagues used a test-negative design, frequently applied in real-world influenza vaccine effectiveness studies.⁶ Test-negative studies are a form of case-control study in which those who test positive for a disease—in this case COVID-19—are compared with controls undergoing the same tests for the same reasons but who tested negative.⁶ Strengths of the test-negative approach in this study include the very large sample size, efforts to reduce confounding by taking into consideration programmatic knowledge of vaccine scale up, and known differences related to health-care seeking or access. Real-world evaluations cannot capture elusive behaviours that affect vaccine uptake and increased exposure, largely driven by social and cultural inclinations that are not part of a standard electronic medical record.⁶ Still, assuming consistent application of the methods described in this Article by Rearte and colleagues,⁵ all future estimates will be subject to the same sources of potential bias, thus providing a reliable tool for continuous monitoring of vaccine-induced immunity in the population, along with the changing of the seasons, and dominant variants.

Evaluating the real-world effectiveness of vaccines for any disease is challenging, but even more so with COVID-19 because the roll-out of vaccines occurred with unprecedented speed over divergent social and geographical environments. These vaccines were developed during the early period of the pandemic when SARS-CoV-2 variants were poorly understood. The roll-out in Argentina occurred when the lambda (C.37) variant was the dominant subtype in the continent of South America.⁵ Argentina is now experiencing a surge in cases, most likely linked to omicron (B.1.1.529).⁷ Unlike other continents, the delta (B.1.617.2) variant has not developed much of a foothold in the region, but the emergence of omicron as a highly transmissible variant now necessitates careful monitoring to determine the spread of it in South America and whether new mutations weaken the morbidity associated with infection. Evidence from North America reported in early 2022 indicated that two doses of vaccine may be less effective than hoped at protecting against omicron infection, but that a third vaccine dose with an mRNA vaccine offered some protection (82%) that wanes over time.⁸ The definition of fully vaccinated now is reasonably three doses.

The many questions answered by Rearte and colleagues are important to every country that is choosing which vaccines to distribute. Evidence that the rAd26-rAd5 and ChAdOx1 nCoV-19 vaccines offer slightly more protection than BBIBP-CorV should be balanced with the costs and availability of these vaccines. An important factor to consider is emerging evidence that indicates mixing vaccines offers favourable outcomes and supports distribution of any vaccine that is readily available.⁹

© 2022 Marcos Brindicci/Stringer/Getty Images

EJM is employed by Platform Life Sciences, which is a private company providing consultancy on global health trial design and methods. Platform Life Sciences is not involved with any work on COVID-19 vaccines. EJM has previously been employed by Cytel, which does contract research and software development. GR is the founder and an employee of Cardresearch, which leads the Brazilian TOGETHER trial of repurposed therapies for SARS-CoV-2.