COVID-19 vaccine trials should seek worthwhile efficacy

Three issues are crucial in planning COVID-19 vaccine trials: (1) whether to demand not only proof of some vaccine efficacy but also proof of worthwhile efficacy; (2) whether the initial trials of vaccine against placebo should prioritise not only single-vaccine trials but also a multivaccine trial; and (3) whether to assess safety, protection against severe disease, and duration of protection by continuing blinded follow-up of the vaccine and placebo groups after definite evidence of short-term efficacy has emerged, but before an effective vaccine has been deployed locally in the general population.

The world needs efficient, speedy, and reliable evaluation of many candidate vaccines against COVID-19. There is a danger that political and economic pressures for rapid introduction of a COVID-19 vaccine could lead to widespread deployment of a vaccine that is in reality only weakly effective (eg, reducing COVID-19 incidence by only 10–20%), perhaps because of a misleadingly promising result from an underpowered trial. Deployment of a weakly effective vaccine could actually worsen the COVID-19 pandemic if authorities wrongly assume it causes a substantial reduction in risk, or if vaccinated individuals wrongly believe they are immune, hence reducing implementation of, or compliance with, other COVID-19 control measures. Deployment of a marginally effective vaccine could also interfere with the evaluation of other vaccines, as subsequent vaccines would then have to be compared with it rather than with a placebo. For a vaccine superior to the weakly effective vaccine, the increased sample size required could delay recognition of its efficacy. More importantly, if the weak vaccine is compared against an even weaker vaccine, the statistical criteria used to analyse non-inferiority trials could well endorse the even weaker vaccine as being non-inferior (so-called bio-creep).¹

The criteria used to define a successful vaccine in the initial clinical trials of vaccination versus placebo should therefore be strict enough to protect against the risk of a weakly effective vaccine being deployed, especially since there are already many candidate vaccines against COVID-19 to be tested,² providing many chances to overestimate efficacy. Hence, the initial trials comparing COVID-19 vaccines versus placebo should seek reliable evidence not only of some efficacy but of worthwhile efficacy.

WHO recommends that successful vaccines should show an estimated risk reduction of at least one-half,³ with sufficient precision to conclude that the true vaccine efficacy is greater than 30%. This means that the 95% CI for the trial result should exclude efficacy less than 30%. Current US Food and Drug Administration guidance includes this lower limit of 30% as a criterion for vaccine licensure.⁴ As an example of a result that would just satisfy these two criteria, an evenly randomised trial with 50 cases arising in those vaccinated and 100 cases arising in those given placebo would have a 95% CI that just excludes 30%, but would suggest 50% short-term efficacy. A vaccine that has 50% efficacy could appreciably reduce incidence of COVID-19 in vaccinated individuals, and might provide useful herd immunity. Hence, although efficacy far greater than 50% would be better, efficacy of about 50% would represent substantial progress.

In comparison with individual trials for each of the many different vaccines, a global multivaccine trial with a shared control group could provide more rapid and reliable results. Additionally, its continuous use of established clinical trial infrastructure could save time and effort, accelerating the needed discovery of several safe and effective vaccines. High enrolment rates facilitated by flexible trial design and hundreds of study sites in high-incidence locations could yield results on short-term efficacy for each vaccine within just a few months of including that vaccine.

Reliable evidence is also needed about longer-term efficacy, vaccine safety, and protection against severe COVID-19. Trials of sufficient size and duration are needed to provide this, and to determine whether the vaccine can make COVID-19 more hazardous (so-called disease enhancement).5, 6 Trials that assess only immunological endpoints cannot provide this evidence, and human challenge studies in young, otherwise healthy, adult volunteers might not provide sufficient evidence of safety or efficacy in other populations. Assessments of safety in multivaccine trials can determine directly whether particular vaccines have adverse effects not shared by other vaccines. Evaluation of multiple COVID-19 vaccines with standardised methodology will facilitate regulatory and deployment decisions.⁷ Unless such decisions are informed by reliable randomised evidence, the effect on public acceptance of COVID-19 vaccines could adversely affect COVID-19 control and the uptake of vaccines against other diseases.⁸

The WHO Solidarity Vaccines Trial⁹ (figure ) aims to evaluate efficiently and rapidly (within 3–6 months of each vaccine's introduction into the study) the efficacy of multiple vaccines,¹⁰ helping to ensure that weakly effective vaccines are not deployed. The trial seeks to achieve rapid, reliable results by the simplicity of the trial design plus real-time checks on the quality of the limited amount of data sought, facilitating high recruitment rates. A major challenge with vaccine trials at fixed study sites is that unexpectedly low attack rates can delay progress. The WHO trial will mitigate this by geographical diversity, recruiting in many high-incidence countries through fixed and mobile (pop-up) research sites in localities where there are substantial COVID-19 attack rates at the time of enrolment.

Figure.

Selected design features of the WHO Solidarity Vaccines Trial

The primary outcome is laboratory-confirmed symptoms >14 days after vaccination is completed. Analyses of each vaccine after about 40, 70, and 100 primary outcomes occur in the placebo group will report success if they show ≤10 versus 40, ≤30 versus 70, or ≤50 versus 100 outcomes. The third analysis is reported regardless of its findings. In all cases placebo-controlled follow-up continues until at least month 12 (or local deployment of an effective vaccine) to assess safety, disease severity, and duration of protection.

For a one-dose or two-dose vaccine that halves risk the main result on short-term efficacy should emerge within 3–6 months, unless definite results for a highly effective vaccine emerge in interim analyses. Placebo-controlled follow-up then continues until at least month 12, or until an effective vaccine is deployed locally. This approach increases the reliability of the evidence on younger and older adults, duration of protection, efficacy against severe disease, and any disease enhancement.

Funders, vaccine developers, researchers, and government institutions¹¹ have signed an international statement of collaboration in vaccine research. Several of these developers and more than 250 research sites intend to join the WHO Solidarity Vaccines Trial in the hope of bringing forward the time when the world will move beyond the widespread disease, death, and disruption from the COVID-19 pandemic. The trial costs will be a fraction of the societal costs of COVID-19, and this global collaboration could rebut detrimental vaccine nihilism and vaccine nationalism.