Abstract
Several vaccines are being developed as part of the COVID‐19 pandemic. The results of clinical trials for these vaccines were published with efficacy values of more than 90%, using mainly relative risk (RR). In this paper, we decided to reanalyse the data using the different validated methods of risk expression. Using main publications, absolute risks (AR), AR reduction (ARR), number needed to treat (NNT) were calculated for five COVID‐19 vaccines (tozinameran Comirnaty®, Moderna, Vaxzevria®, Janssen, and Sputnik V vaccines). AR, ARR, NNT, and RR values varied according to COVID‐19 vaccines. The order of the different vaccines was not the same according to the chosen efficacy parameters. This is a further example of the need to express results of clinical trials, using not only RR, but also AR, ARR, and NNT in order to clearly present the clinical interest of drugs.
Keywords: absolute risk, COVID‐19 vaccines, Janssen (Ad26.COV2.S) vaccine, number needed to treat, tozinameran, Vaxzevria®
1. INTRODUCTION
Several vaccines are being developed as part of the COVID‐19 pandemic. The first vaccines to be marketed were the Pfizer tozinameran Comirnaty® [1] and Moderna [2] mRNA vaccines, followed by the adenovirus vaccines from AstraZeneca (Vaxzevria®) [3] and Janssen (Ad26.COV2.S) [4, 5]. Today, the Russian vaccine, another adenovirus vaccine, is not marketed in Europe or United States [6].
The results of clinical trials for these different vaccines were published in leading medical journals as well as in the media as “very active” with efficacy values of more than 90% [7]. As clinical pharmacologists, we were interested by a clear quantification of their efficacy. Thus, we decided to reanalyse the data using the different methods of risk expression.
2. METHODS
After the extraction of the main data from the published papers concerning the five COVID‐19 vaccines [1, 2, 3, 4, 5], we calculated different validated expressions of infection risk:
Absolute risks (AR), that is, the risks in the exposed group (patients receiving vaccine) and in the unexposed group (control);
Absolute risk reduction (ARR), that is, the arithmetic difference between the risk in the treatment group and the risk in the control group;
Number needed to treat (NNT) calculated as the reciprocal of the ARR;
Relative risk (RR) with its 95% confidence interval (CI) defined as the ratio of the cumulative incidence of the outcome in the exposed group to the cumulative incidence in the control group [8, 9].
We also calculated the same parameters for influenzae [10] and Ebola vaccines [11] as controls.
3. RESULTS
Table 1 shows the different evaluation parameters for the different vaccines. The order of the different vaccines was not the same according to the chosen efficacy parameters. AR values vary between 0.04 for tozinameran to 0.34 for Janssen vaccine in exposed patients. Large differences were found in ARR between 0.67 for Janssen and 1.20 for Vaxzevria® or Sputnik V vaccines. Thus, NNT varies between 83 for Vaxzevria® and Sputnik V vaccines and 149 for the Janssen vaccine. Lower NNT values were found for influenzae and Ebola vaccines. RR was between 0.05 for tozinameran and 0.34 for Janssen vaccine.
TABLE 1.
Risk of infections expressed as absolute risk (AR), absolute risk reduction (ARR), number needed to treat (NNT), and relative risk (RR) with its 95% confidence interval (CI) with the four COVID‐19 vaccines (V), influenzae vaccine (V), and Ebola vaccine (V)
| Results in exposed patients | Results in control patients | AR in exposed patients | AR in control patients | ARR | NNT | RR | 95% CI | |
|---|---|---|---|---|---|---|---|---|
| Pfizer vaccine tozinameran Comirnaty® | 8/21 720 | 162/21 728 | 0.04% | 0.74% | 0.71% | 141 | 0.05 | 0.02–0.10 |
| Moderna vaccine | 11/15 210 | 185/15 210 | 0.07% | 1.20% | 1.13% | 91 | 0.06 | 0.03–0.11 |
| AstraZeneca vaccine | 30/5807 | 101/5829 | 0.50% | 1.70% | 1.20% | 83 | 0.30 | 0.19–0.44 |
| Sputnik V vaccine | 16/14 964 | 62/4902 | O.10% | 1.30% | 1.20% | 83 | 0.09 | 0.05–0.14 |
| Janssen vaccine | 66/19 306 | 193/19 178 | 0.34% | 1.01% | 0.67% | 149 | 0.34 | 0.26–0.45 |
| Influenzae vaccine | 221/18 797 | 357/13 095 | 1.18% | 2.73% | 1.56% | 64 | O.43 | 0.35–0.50 |
| Ebola vaccine | 91/91 492 | 880/92 262 | 0.10% | 9.50% | 9.4% | 11 | 0.11 | 0.08–0.11 |
4. DISCUSSION
The present work aims to present the different modes of expression of the efficacy of COVID‐19 vaccines. Several measures are used to report benefits and harms of drugs or treatments. They widely differ in their clinical relevance because they do not equally reflect the benefits in clinical terms. The most widely used measures RR (or RRR) are of limited clinical interest, because they fail to take into account baseline risks and tend to exaggerate the positive results of studies. ARR and NNT are better expressions of risk from a clinical point of view [7, 8]. In this study, we did not calculate the number needed to harm (NNH) because safety data about COVID‐19 vaccines today are limited and mainly concern “nonserious” adverse events.
Authors should mention that absolute risk estimates require a time frame to be correctly understood, that is, how long patients were treated or exposed to the drug (vaccine) or observed.
Analysis of these different benefit values allows three major conclusions. First, RR and NNT values give different approaches of the COVID‐19 vaccines efficacy. Second, the magnitude of the influenzae or Ebola vaccines' effect seems to be more important than that of COVID‐19 ones. Third, no major differences seem to appear between the different COVID‐19 vaccines; even NNT values for tozinameran or Janssen vaccines are a little bit lower than those found with the three other COVID‐19 vaccines. An indirect comparison is not possible because exposed and control populations were not strictly comparable. In fact, it was not the purpose of this paper to perform an indirect comparison of the different vaccines or to discuss the interest of COVID‐19 vaccines, but only to better illustrate, from a medical point of view, the different efficacy parameters, using validated methods [8, 9].
The paper has other limitations. We used data from scientific international publications and not regulatory data packages. As underlined above, a strict comparison between the different vaccines was not possible because different evaluation criteria were used in different published papers, even if we tried to minimize this bias. Finally, limits of NNT use are well known: no account taken of adverse events or different clinical forms of the disease, lack of reliable confidence intervals, and so forth. Some authors recommend the use of ARR values instead [12]. However, to be correctly understood, AR estimates require a time frame, that is, taking into account how long patients were treated or exposed to the drug (vaccine) or observed.
In conclusion, because RR measures do not take into account baseline risks and tend to exaggerate benefits of treatments, we believe that presentation of results in large international clinical trials should also include not only RR but also AR, ARR, and NNT in order to help prescribers (but also health authorities) truly represent the clinical interest of therapeutic interventions and make the best choice for patients and society. This conclusion is true not only for drugs in general but also for vaccines.
AUTHOR CONTRIBUTIONS
JLM and AS designed the study. JLM extracted the data and performed the statistical analysis. JLM, PB, and AS analysed the data. JLM wrote the paper. The three authors reviewed the successive versions of the manuscript and approved the final version.
CONFLICT OF INTEREST
None.
ETHICS STATEMENT
The study, which only involved review of published data without direct access to patients, was performed according to international ethics guidelines.
FUNDING INFORMATION
The work was performed during the university research time of the authors. There were no funding sources. The authors certify that they have not received any funding from any institution, including personal relationships, interests, grants, employment, affiliations, patents, inventions, honoraria, consultancies, royalties, stock options/ownership, or expert testimony for the last 48 months.
Montastruc J‐L, Biron P, Sommet A. Efficacy of COVID‐19 vaccines: Several modes of expression should be presented in scientific publications. Fundam Clin Pharmacol. 2022;36(1):218-220. 10.1111/fcp.12715
REFERENCES
- 1. Polack F, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid‐19 vaccine. N Engl J Med. 2020;38:2603‐2615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Baden LR, el Sahly HM, Essink B, et al. Efficacy and safety of the mRNA‐1273 SARS‐CoV‐2 vaccine. N Engl J Med. 2021;384(5):403‐416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Voysey M, Clemens SAC, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV‐19 vaccine (AZD1222) against SARS‐CoV‐2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet. 2021;397(10269):99‐111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Sadoff J, Gray G, Vandebosch A, et al. Safety and efficacy of single‐dose Ad26.COV2.S vaccine against Covid‐19. Nejm. 2021;384(23):2187‐2201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Vaccines and Related Biological Products Advisory Committee Meeting . February 26, 2021. FDA Briefing Document Janssen Ad26.COV2.S Vaccine for the Prevention of COVID‐19 https://www.fda.gov/media/146217/download (accessed May 5, 2021)
- 6. Lagunov DL, Dolzhikova IV, Shcheblyakov DV, Tukhvatulin AI, et al. Safety and efficacy of an rAd26 and rAd5 vector‐based heterologous prime‐boost COVID‐19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet. 2021;397(10275):887‐897. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Deplanque D, Launay O. Efficacy of COVID‐19 vaccines: from clinical trials to real life. Therapie. 2021, in press;76. 10.1016/j.therap.2021.05.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Strom BL, Kimmel SE, Hennessy S. Pharmacoepidemiology. 5th ed. Wiley‐Blackwell; 2012. 953pp. [Google Scholar]
- 9. Akobeng AK. Communicating the benefits and harms of treatments. Arch Dis Child. 2008;93(8):710‐713. [DOI] [PubMed] [Google Scholar]
- 10. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta‐analysis. Lancet Infect Dis. 2012;12(1):36‐44. [DOI] [PubMed] [Google Scholar]
- 11. Anonym . Ebola vaccine rVSV‐Zebov Erbevo® in an outbreak of Ebola virus disease. Prescr Int. 2021;30:33‐37. [Google Scholar]
- 12. Hutton JL. Misleading statistics: the problems surrounding number needed to treat and number needed to harm. Pharm Med. 2010;24(3):145‐149. [Google Scholar]