Improving vaccine trials in infectious disease emergencies

SUMMARY

Unprecedented global effort is underway to facilitate testing of countermeasures in infectious disease emergencies. Better understanding of the various options for trial design, as well as preliminary global agreement on the most suitable designs for the various scenarios, are needed now—in advance of outbreaks. What would enhance, then, the speed, validity, and ethics of clinical studies of such countermeasures? Focusing on studies of vaccine efficacy and effectiveness in emergencies, we highlight three needs: for formal randomized trials—even in most emergencies; for individually-randomized trials—even in many emergencies; and for six areas of innovation in trial methodology. These needs should inform current updates of protocols and roadmaps.

Preparedness for an epidemic of a new or reemerging disease requires the ability to test whether candidate treatments and vaccines against that disease are safe, efficacious, and effective. During the 2014–5 West African Ebola outbreak, one rate-limiting step in this process was that vaccine candidates that had shown promise in animal models had never been tested in humans. The entire sequence of safety and dose-ranging (Phase I), immunogenicity (Phase IIa), and efficacy/effectiveness (Phase IIb and III) trials had to be performed while the outbreak was underway. To expedite future emergency responses, the new Coalition for Epidemic Preparedness Innovations (CEPI), addresses this challenge by funding proof of concept and safety trials, in anticipation of outbreaks, for vaccines against infections considered at high risk of emergence (1). Unlike efficacy and effectiveness trials, early-phase trials on healthy volunteers can be completed before the relevant infection is present in populations, and outside outbreak areas.

But the Ebola outbreak also illustrated another obstacle to testing candidate countermeasures. At the height of the epidemic, disputes flared about the scientific validity, feasibility, speed, and ethics of competing efficacy trial designs (2, 3). To consider these trial design questions outside the pressure-cooker of real-time responses and to keep that response expedient and trusted by the public, methodological and ethical deliberation and agreement are needed now. World Health Organization (WHO) work on a Blueprint for Research and Development for Action to Prevent Epidemics includes efforts to create “roadmaps,” protocols, and decision-making tools for trials of vaccines against certain priority pathogens (4). A new report by the US National Academies of Science, Engineering and Medicine recommends expanding such activities to include developing generic research protocols and advance arrangements for legal and administrative details that could save time when an emergency occurs (5).

We concur and would endorse an even more ambitious agenda for research and consensus-building about testing countermeasures in anticipation of future outbreaks. First, advance planning is especially urgent for those areas of trial design that remain contentious. Disputes about trial ethics, in particular, proved a bottleneck to vaccine efficacy tests during the 2014–5 Ebola outbreak (2, 3). During response to earthquakes, blasts, floods, and some other disasters, rescue workers already triage patients with algorithms that have been settled in advance (6), both to expedite decisions and to avoid corrosive ethical disputes (7). In an infectious disease emergency, especially in research, protecting public trust is key—as is advance agreement on trial design.

Second, excellent trial design is a subtle and complex art. Advance deliberations would allow input from all disciplines involved, stakeholder involvement, and external expert advice, with ample time for back and forth on the intricacies of statistical method, philosophy, and local circumstance and culture of various designs and scenario types.

This perspective identifies three specific needs of trials in emerging disease emergencies that such advance agreement ought to include. For brevity, we focus on vaccine trials, and discuss diagnostics and therapeutics only briefly. Our first point is that formal randomized trials will nearly always be needed before a vaccine can be rolled out, even in an emergency. Second, even in outbreaks of highly lethal emerging diseases, randomization of individual participants to different arms is likely to remain scientifically desirable, and, notwithstanding recent skepticism, ethically permissible. Third, scientific and methodological innovation in six areas would help keep evaluations of candidate vaccines both rapid and credible.

Needed: Randomized Trials

During the 2014–5 West African Ebola outbreak, some questioned whether randomized efficacy trials should be conducted: once vaccines, for example, had been shown safe and immunogenic in humans, and protective of animals upon challenge with the virus, wouldn’t it be unnecessary and unethical to conduct a randomized trial for vaccine candidates, instead of rolling them out while monitoring the effects? Experimental though they remained, vaccine candidates held out at least the possibility of protection from Ebola (3).

This reasoning may hold initial appeal, but it fails for nearly all epidemic situations (8, 9). First, the historical record shows repeated instances where vaccines that seemed highly promising prior to Phase III were not proven effective, or were even proven harmful, to everyone or to determinate subgroups. Multiple vaccines against the respiratory syncytial virus (RSV) have met this fate (10). A vaccine that was efficacious against herpes simplex virus type 2 in serodiscordant couples failed to show efficacy when tested in a more representative population (11). Vaccines against HIV (12) and against Staphylococcus aureus (13) that were protective in animal challenge models and immunogenic in humans were not found effective in a wider population and worsened some participants’ outcomes during their respective Phase III efficacy trials. A dengue vaccine, although beneficial overall, was found during Phase III (14) to be harmful to certain identifiable recipients (15). Remarkably, while not always due to trial outcomes, a third to a half of vaccines that enter Phase III fail to submit for regulatory approval (16, 17).

Phase III trials for candidate vaccines are not mere formalities, then. They are performed both to quantify the degree of protection offered by the vaccine, and to test the real possibility that the vaccine, despite promising preclinical and safety results, may be ineffective or harmful in humans exposed to the infection. Indeed, in several cases where vaccines were judged harmful to some in Phase III, the harm was an increase in the probability of infection upon exposure, or the probability of severe disease upon infection—safety flaws undetectable in safety trials, which usually take place in areas where exposure is very rare (10, 12–14).

During outbreaks, the waste and potential for toxicity might initially be thought less crucial than the public health need for a chance at protection. But skipping Phase III trials could lead to recalls and even medical harm, undermining public trust in researchers and responders, crucial during emergencies.

The 2014–5 Ebola outbreak exemplified another consideration: that several promising vaccine candidates can exist. One point of randomized controlled trials is precisely to establish which if any is worthiest of the advance investment in large-scale production and roll-out.

Despite these considerations, one can envision scenarios so urgent that an experimental vaccine should be rolled out before proven effective. In an outbreak of a new disease combining the lethality of severe acute respiratory syndrome (SARS) with the transmission characteristics of pandemic influenza, the months required for vaccine trials might cost millions of lives and risk a breakdown of trust. Such circumstances may warrant replacing advance randomized trials by observational studies during rollout, notwithstanding the notoriously-challenging methodology for observational vaccine studies (18). Further development of approaches to combining rollout with randomized evaluations, either through individual randomization (19) or through cluster-randomization (20–22), could help expand the options for such extreme scenarios. While for most disease outbreaks, advance randomized trials would be preferable, the precise conditions warranting immediate rollout and plans for evaluating interventions in such circumstances should also be identified in advance.

Needed: The Option for Individually-Randomized Trials

The central debate on vaccine trial design during Ebola concerned the ethics of individual randomization (2, 3). Is it permissible to randomize individuals not to receive a study vaccine, and/or to receive placebo? Humanitarian workers, public health leaders and ethicists who accepted the urgency of randomized trials insisted that it would be unethical to conduct them by enrolling individual participants and then randomizing them to receive either the experimental vaccine or a control substance (either a vaccine against a different infection, or placebo).

Many espoused instead a form of cluster-randomized trials in which everyone is offered the experimental vaccine during the trial, yet certain groups receive it earlier than others. Vaccine impact is then assessed by comparing incidence between the early and the late groups. Two such designs were considered: 1) a stepped-wedge design in which different communities are offered the vaccine in a sequence determined at random, and the incidence is compared between those who have already received the vaccine and those who have yet to receive it; and 2) a ring-vaccination design, implemented for the first time by the 2015 Ebola ça Suffit vaccine trial in Guinea, which randomized “rings” of persons at high risk of infection from a confirmed Ebola case, to receive either immediate or delayed vaccination; incidence in immediately-vaccinated rings was compared to incidence in rings with delayed vaccination. It was argued that these two designs were ethically superior to individual control because no participant was denied access to the vaccine throughout the trial (23).

Whatever the other virtues of the stepped-wedge and ring-vaccination designs, they are not ethically superior to individual randomization (24, 25). Just as an individually-randomized trial would withhold the experimental vaccine from members of the comparator group, the stepped-wedge and ring-vaccination designs withhold the experimental vaccine from members of the delayed group, albeit temporarily (25). More generally, randomizing participants into arms for any trial design creates a disparity in their prospects whenever: 1) the experimental intervention has shown enough promise in earlier trial phases to get to Phase III; 2) no proven alternative intervention exists; and 3) to get the experimental intervention early is better than to get it late if it works. These conditions will be met in many or in most Phase III trials for vaccines against emerging infections, including Ebola in 2014-5. Participants randomized (individually or as a group) to receive the intervention (or to receive it early) have better prospects than other participants, other things equal. Fair or unfair, that disparity is inevitable whether control is temporal or spatial or by type of intervention or based on the difference between intervention and placebo. We have termed it the “near-inevitable disparity within all randomized controlled trials of new interventions.” Accepting randomized trials—as one should—is accepting this disparity (24).

Sometimes, doses are initially insufficient regardless of study method and it is impossible to offer the candidate vaccine to all participants at risk immediately. This may seem to support a stepped-wedge design, which can roll out vaccine as it becomes available. However, an individually-randomized design with stepped rollout (19) offers multiple scientific advantages over stepped-wedge while delivering vaccine as it becomes available.

Finally, individually-randomized designs can offer the experimental vaccine to members of the control group after data are collected—once data analysis suggests efficacy or, if so desired, earlier. Inasmuch as individual randomization reaches adequate sample size earlier than the cluster-randomized alternatives (24), experimental vaccines could therefore become available to controls in individually-randomized trials earlier than to ones in the cluster-randomized alternatives. Figure 1 captures this possibility. The figure compares an individually-randomized control trial with two arms (bottom) and a stepped wedge trial (top). Green designates having been vaccinated. Both trials begin at point A in time, and once data collection is over at the individually-randomized trial, at point B, access to the vaccine candidate can be offered to all its participants, at point C. That takes place past data collection in that trial, but chronologically before the vaccine candidate reaches all participants of the stepped wedge trial, at point D (24).

Fig. 1. Comparing stepped wedge and individual randomization for vaccine protection of trial participants.

Top: Stepped-wedge designs offer vaccine candidates to all eligible trial participants at some point during the trial (dark shading) if they remain uninfected and alive. Bottom: Individually randomized trials, where control participants do not receive the experimental vaccine during data collection, ordinarily take less time or fewer participants or fewer disease cases than other designs to achieve a given degree of statistical power, so data collection can end sooner (vertical dashed line). If control participants in such a trial are offered the vaccine at the end of data collection (light shading), this can permit all trial participants to have access to the vaccine earlier than all stepped-wedge participants (24).

In short, not only are individually-randomized vaccine trials typically better than cluster-randomized alternatives in the critical terms of efficiency (because for any result deemed compelling enough for population rollout, the former require fewer participants and/or fewer disease cases to reach that result). There is also a strong preliminary case that individually-randomized trials are otherwise ethically no worse than cluster-randomized ones.

Needed: Design and Analysis Innovations

As diseases continue to emerge, new design challenges surface, calling for novel approaches to trial design and analysis (22). An excellent example is the Ebola ça Suffit trial, which addressed the challenge of dwindling cases with a novel design based on vaccinating individuals at high risk of infection from known cases (23). A similar design might be in some future emergencies (26), depending on the circumstances. But many challenges remain in vaccine testing during outbreaks, and opportunities for methodological innovation abound.

One challenge for investigators of many emerging disease outbreaks, including Ebola (27–29) and Zika (30), is that incidence is patchy and (at least initially) unpredictable in space and time. Deploying a trial in an area with uncertain future incidence might expend experimental vaccine doses and trial effort without reaching a conclusive result. Designs that place the trial in the vicinity of known cases (as did Ebola ça Suffit (31)), or in areas predicted by modeling to have high later incidence (22, 29), should be further studied to characterize their suitability to particular situations.

A second challenge is to create trial designs and analysis methods that can deal with the extreme urgency of testing vaccines in emergencies. Initially, this urgency arises from the approximately exponential growth of a typical infectious disease outbreak (Fig. 2, left), making each week of delay miss more prevention opportunities than the preceding week. Following effective public health interventions or exhaustion of the susceptible population, an epidemic begins to wane (Fig. 2, right). Then, a new form of time-sensitivity takes over: the need to test countermeasures while there remain enough cases for statistically-meaningful results. This transition can be seen in Fig. 2, where early growth of cases gave way by spring 2015 to a situation where were too few cases to perform vaccine efficacy trials in Sierra Leone and Liberia; to test a vaccine at end of the epidemic in Guinea (32), a special design (ring-vaccination) had to be invented “on the fly” (23). Zika vaccine trials may face similar challenges, largely because those areas most suitable for transmission (e.g. due to high vector density) will be protected by herd immunity resulting from transmission in prior seasons. As a result, it will be hard to identify populations still at-risk for substantial outbreaks with enough cases to test vaccines (33).

Figure 2. Time-sensitivity of testing countermeasures at different stages of the epidemic, illustrated with the epidemic curve of Ebola cases in Guinea and Sierra Leone.

During the ascending phase of an epidemic, case numbers accumulate at an accelerating rate, meaning that each added week of delay in identifying effective prevention or treatment measures brings a larger number of new infections. Once an epidemic is brought under control dwindling case numbers threaten trial power, because lower disease incidence ordinarily makes more trial participants necessary for statistically robust results. In sum, a race to identify countermeasures to stop the epidemic in the early stage is replaced in the late stage by a race to test the countermeasures before a research opportunity for developing countermeasures for future outbreaks is missed.

A third challenge is designing trials of vaccines for infections that are sometimes asymptomatic or subclinical, such as Nipah (34) and Zika (35). Because for Zika, at least, even asymptomatic infections may contribute to birth defects in the fetuses of infected pregnant women (36), and to transmission (37), it is important to measure the ability of vaccines to prevent asymptomatic infections. To do so, one might repeatedly test healthy trial participants for asymptomatic viral infection or, alternatively, test serum samples at the end of follow-up to detect whether participants were infected. Either strategy risks substantially increasing study cost and complexity, and methodological work on streamlining is needed.

A fourth challenge is to apply to outbreaks the latest advances in adaptive designs, in which pre-set rules govern intra-trial adjustments of the number of participants enrolled, the duration of follow-up, the proportion randomized to different arms, and other parameters (38). These adjustments occur in response to experience in the trial; for example, a vaccine trial may increase enrolment if incidence in the enrolled participants has not yet provided enough cases to achieve statistical power, or a treatment trial may increase the proportion of participants randomized to treatments that show greatest initial promise. Another form of adaptive design combines more than one of the phases of traditional trials (Phase I, II and III) to achieve greater efficiency and speed (39). The Ebola ça Suffit trial (23) and other proposed vaccine trial designs (19) contained adaptive elements. Further work in this direction could expand the options available for trials in future epidemics.

Fifth, new technologies in other areas can also help improve the precision of vaccine trials. The increasing speed and declining cost of pathogen genome sequencing may allow detecting not only the infection but also each participant’s likely infector. That would enable greater precision in estimating the effects of prevention measures (40), vaccines included.

Finally, host responses to vaccination can now be characterized in exquisite detail, with each individual’s response to vaccination measured at the levels of transcription, protein production, cellular proliferation, and antigenic specificity (41). To date, these measurements have not been deployed on a large scale in clinical trials during emergency. Their deployment could help to elucidate the mechanisms and correlates of protection (42), useful for many decisions about vaccine use in emergencies, e.g. whether to use fractional doses of a scarce vaccine so as to extend supply (43, 44).

Summary

Consensus is arising that the time to debate trial designs for vaccine candidates against infections under various scenarios is between epidemics, not during them. Exploring in advance intricate questions in trial ethics and design is especially important. Toward this end, we have defended the need for randomized efficacy trials prior to rollout of new interventions even in most disease outbreaks, the need for individual randomization, and the need for continued development of innovative designs.

This perspective focuses on vaccines. Trials of diagnostics are often simpler and can be performed on blood and other specimens that need not be tested in real time. Treatment trials may enable adaptive designs (45), more than vaccine trials, because many adaptive designs can work only if outcomes are known for earlier participants before assigning treatments to later participants and that is more common in trials of therapeutics than in ones of vaccines. On the other hand, individual control, placebo control, and other design elements are more ethically challenging for therapeutics than they are for vaccines, because only in therapeutic efficacy trials are participants already infected with the relevant serious disease. For all these categories of countermeasures, investing now in innovation and consensus-building is likely to save time and ultimately lives.