Evaluation of Alternative Endpoints for ZIKV Vaccine Efficacy Trials

Abstract

Zika virus (ZIKV) infection during pregnancy is associated with microcephaly and other birth defects, collectively termed Congenital Zika Syndrome (CZS). During the epidemic in 2015-16, ZIKV spread through the Americas and quickly joined the list of other known teratogenic pathogens, TORCH. Multiple ZIKV vaccines have been developed for protection of pregnant women and women of childbearing age. However, ZIKV infection incidence has since waned substantially, and adverse birth outcomes are rare outcomes of infection. Studying a vaccine’s protective efficacy against CZS in a large phase III clinical trial may be infeasible in such times of low incidence. Should trials be initiated, researchers may resort to alternative clinical endpoints.

In this study, we simulate a variety of vaccine clinical trial scenarios to evaluate the feasibility of the CZS endpoint in vaccine studies and compare CZS to other potential outcomes: ZIKV infection detected through weekly, biweekly, or monthly testing and laboratory-confirmed, symptomatic Zika Virus Disease. We compare the sample size required for 80% statistical power to detect vaccine efficacy and trial duration for each scenario. Our results show the feasibility of CZS clinical endpoints depends on the timing of simulated clinical trials in the course of a seasonal epidemic, due to CZS risk varying with trimester of infection. This result highlights additional considerations needed when designing vaccine efficacy trials of protection against teratogenic pathogens.

1. Introduction

Zika virus (ZIKV) was first isolated in Uganda in 1947 and associated with human illness in 1953 [1]. Sporadic outbreaks followed until 2015, when the virus was identified in Brazil and linked to an alarming increase in the incidence of microcephaly and other adverse birth outcomes [1–4]. Researchers began to list ZIKV with other infectious teratogens, exposures that harm a developing embryo or fetus, which include pathogens known by the acronym TORCH—Toxoplasmosis, Other agents, Rubella, Cytomegalovirus, and Herpes Simplex [5].

ZIKV spread to over fifty countries and territories after its introduction to the Americas [6]. During that time, its teratogenic effects led the World Health Organization (WHO) to declare a Public Health Emergency of International Concern and call for the development of vaccines to protect pregnant women and women of childbearing age [7]. WHO removed the emergency designation in November 2016, but ZIKV remains a significant threat during pregnancy, with an extensive list of potential adverse birth outcomes collectively termed Congenital Zika Syndrome (CZS) [8]. In response, multiple ZIKV vaccines were developed and entered early phase clinical trials. Before these vaccines are made available to the public, large phase III trials must be completed to evaluate their efficacy, the reduction in risk in the vaccinated compared to the unvaccinated [9].

As a phase III clinical trial’s power to detect vaccine efficacy depends in part on the number of cases observed, a successful trial will depend on the extent of ZIKV transmission in the study area. While ZIKV incidence has fortunately waned substantially since the initial outbreak, this poses challenges to the successful identification of a safe and efficacious vaccine. The Zika Modeling and Projections for Vaccination Trials collaboration recommended increasing the number of study sites in various geographical regions, along with recruiting larger numbers of participants and extending the duration of trial participant follow-up to adequately power a vaccine efficacy trial [10]. Furthermore, though WHO called for vaccines that protect pregnant women and women of childbearing age, CZS is a rare outcome of ZIKV infection and may not be a feasible clinical trial endpoint in a time of low incidence [11]. Other possible endpoints are ZIKV infection and symptomatic Zika Virus Disease, both of which occur at higher frequency and would allow recruitment from a larger population of both men and women.

Choosing either infection or symptomatic disease as primary endpoints in a ZIKV vaccine trial can only be justified, however, if protection against them reasonably predicts protection against more severe complications, like CZS, that are the greatest health burdens associated with the virus [11]. This is challenging for symptomatic disease endpoints, as disease has not yet been associated with higher viral load or increased risk of maternal-fetal transmission [12]. Conversely, vaccines that prevent maternal infection may be expected to prevent subsequent maternal-fetal transmission, and research suggests that complete sterilizing immunity may be required to protect against CZS [13]. In practice, testing for ZIKV infection relies on viral RNA detection [11,14,15], and the tests used have a lower limit of detection that may be above the viral load necessary for maternal-fetal transmissions [13,16]. In this scenario, trials can show efficacy against detectable viral load, when infection may still have occurred and sterilizing immunity was not produced. For simplicity, we continue to refer to this clinical endpoint as infection-based, though we recognize that it may be impossible to show actual sterilizing immunity.

It is clear that the choice of clinical endpoint has far-reaching consequences for the feasibility of a ZIKV vaccine trial and the ultimate conclusions about the vaccine’s protective efficacy against the most severe outcomes of infection. These consequences include immediate effects on trial participant recruitment and statistical power to detect vaccine efficacy. We simulate a variety of clinical trial scenarios in the present study, to identify important patterns to consider when evaluating the feasibility of the following ZIKV clinical endpoints: ZIKV infection detected through weekly, biweekly, or monthly testing, laboratory-confirmed Zika Virus Disease, and CZS.

2. Methods

We simulated ZIKV vaccine efficacy trials to explore three main clinical endpoints: infection, symptomatic disease, and CZS. Within infection-based endpoints, we additionally varied the time between laboratory tests. In all cases, simulations were completed in four steps. First, we obtained observed ZIKV incidence data for six countries and scaled the data to meet a generic 1% cumulative annual incidence rate while maintaining the seasonal pattern observed. Second, we simulated trial populations of various sizes within those regions, and assigned weekly risk according to seasonal patterns, layering individual heterogeneity in infection risk on top of these patterns. Third, we modeled time until infection, time until symptom onset, and time until loss of detectable viral RNA in blood samples for a range of assumed vaccine efficacies in trials with ZIKV infection and symptomatic Zika Virus Disease endpoints. For CZS endpoint trials, we additionally generated CZS outcomes diagnosed at birth. Finally, we analyzed simulated trial data to assess the statistical power of each trial scenario, and compared sample size and trial duration for trials achieving 80% power. We used R version 3.4.2 for all simulations and analyses [17].

2.1. Scaling weekly country-level incidence rates

The Pan American Health Organization publishes periodic updates on ZIKV cases in Latin America and the Caribbean. These updates include tables of cumulative case counts and are often accompanied by graphs of the ZIKV epidemic curve in each country [6]. There are notable errors in the case count tables, which are fortunately not reflected in the graphs of epidemic curves [18]. Rather than use the tabular data and risk including these errors in our simulation, we obtained the graph data of weekly incidence in multiple countries in 2016 and 2017 that was previously digitized by the Andersen Lab at the Scripps Research Institute [18]. For this study, we obtained the digitized data for eight countries previously identified as having sufficient infrastructure for a large clinical trial [10]. Of these, six countries had eighty-two weeks of observed incidence data at the time of our study: Colombia, Costa Rica, Ecuador, Mexico, Panama, and Peru. The two remaining countries were excluded for having missing observations (Brazil) or many weeks with 0 cases (Dominican Republic), which made the data unscalable. We scaled the digitized incidence data to meet a generic national yearly cumulative incidence rate, maintaining the seasonal pattern of Zika transmission in our simulation (Figure 1). A generic cumulative incidence rate of 1% of the population per year was chosen, not to reflect actual epidemic values but to better illustrate qualitative patterns between trial endpoints. To imitate realistic trial site selection, we additionally modeled a selection process so that four countries would ultimately be chosen, and that selection would be based upon monitored ZIKV infection incidence. To simulate this process, we randomly selected four of the six countries at the beginning of each simulation iteration. To examine how incidence at the start of a trial affected feasibility, we set trial start dates independently within each of the four countries according to three possible starting rules: (1) when that country first reached a weekly ZIKV infection rate of 25 cases per 100,000 persons in the scaled data, (2) when it reached a weekly rate of 50 cases per 100,000 persons, or (3) one week following the peak of a local epidemic in the first 40 weeks (approximately half) of the scaled data.

Figure 1: Scaled person-week infection rates.

Scaled person-week infection rates for the six countries included in the study. Data was scaled from observed cases reported by the Pan American Health Organization. Week 1 corresponds to epidemiological week 1 of 2016, or January 3, 2016. Week 82 is dated July 23, 2017.

2.2. Simulating trial populations

Each participant’s weekly ZIKV infection risk was modeled as the product of the country-level weekly incidence rate and an individual-level risk factor. Individual-level infection risk factors were drawn from the lognormal distribution (log mean = 0, log sd = 1) to simulate heterogeneity in infection risk that was non-negative. We assumed independence in infection risk, as trial populations are likely to be a small proportion of each country’s total population if participants are recruited from many countries. Though only six countries were used in this study, recruiting from a large number of total countries is recommended for ZIKV vaccine trials [11].

In each simulation, we varied the start time of the trial according to one of three possible starting rules, discussed above. Each trial participant was then randomized to either the control or vaccination arm. We considered vaccine efficacies of 50%, 70%, and 90%, and assumed an immune ramp-up period lasting one month before achieving the assumed efficacy. Efficacy was simulated as a reduction in each vaccinated individual’s overall weekly risk. We simulated a point recruitment process such that all individuals were randomized and, if applicable, vaccinated at the trial start date.

2.3. Generating clinical outcomes

We generated time until infection for each participant using the exponential distribution, so that a participant’s risk of infection in a given week was independent of previous weeks. We assumed viral RNA would be detectable in blood two days after infection. We then used reported estimates for the mean time to viral clearance to model an RNA detection interval using the flexible Weibull distribution [19,20]. The detection interval described the time from first positive samples until the virus could no longer be detected in blood.

For trials with ZIKV infection as the primary endpoint, we created a set of laboratory testing days that began at the start of the trial and occurred weekly, biweekly, or monthly thereafter. Cases were identified when these predetermined testing days fell within an individual’s RNA detection interval. In simulated trials with laboratory-confirmed ZVD as the primary endpoint, we determined which infected individuals would develop disease based on the approximate 20-25% symptomatic rate [11]. A median incubation period of 5.9 days was used to generate time to symptom onset [19]. Viral RNA was assumed to be detectable upon symptom onset. The duration of detection time in samples was generated as above. Rather than record all ZVD outcomes as events, cases were identified when weekly testing dates fell within a symptomatic individual’s detection interval. Weekly testing was used as it was assumed not all individuals would report to a study clinic immediately, but may be expected to do so within a short timeframe given an intense surveillance protocol as suggested by WHO [11]. For those cases identified by this protocol, we recorded time until symptom onset as time until infection plus the incubation period.

For trial simulations in which CZS was the primary endpoint, we assumed an entirely female study population. Furthermore, we focused on populations of women who self-identify as trying to conceive (TTC). Women who first attempt to conceive have a 30% probability of conception within a menstrual cycle, and this probability decreases to approximately 5% in women who have not conceived by the end of the first year [21]. In the CZS endpoint scenarios, all the women in the trial were given an initial 21.3% probability of conception in the first month of the trial. This probability was calculated as the mean of the first six months of exponentially decaying conception probability, to account for women entering the trial with different histories of conception attempts, and decreased every four weeks of the trial. Time until pregnancy was drawn from the exponential distribution. We set CZS risk according to trimester of ZIKV infection: infection in the first trimester only (15%), infection in multiple trimesters, including the first (21%), or infection in the second or third trimesters only (2.27%) [22]. The CZS outcome was considered binomial, as either present or absent at birth, and so was generated from the binomial distribution.

In sensitivity analyses, we included women regardless of desire to conceive. Given the contraceptive use rate of 73% in Latin America [23] we used the complement of this rate as the assumed proportion of trial participants who were TTC. While this most likely overestimates the proportion of women who are sexually active and actually attempting to conceive, none of these trials achieved adequate statistical power in simulations and were not examined further.

We varied trial population sizes from 200 to 30,000 individuals for non-CZS endpoints and from 200 to 120,000 individuals for CZS endpoints. We simulated 250 runs for each scenario (i.e. set of parameters).

2.4. Trial analysis and power calculations

All trial designs were assumed to be group sequential up to four interim analyses before a final analysis. Time-to-infection and time-to-symptoms were the outcomes of interest for ZIKV infection and Zika Virus Disease scenarios, respectively. We analyzed these trials with Cox proportional hazards models. We analyzed simulated CZS trial data using a logistic regression model with CZS diagnosis present at birth as the endpoint. We calculated statistical power as the proportion of scenario iterations in which the vaccine efficacy p-value was less than the significance level of 0.05. Mean trial duration was calculated for all trials with power of 80% or more.

3. Results

3.1. CZS-based clinical endpoints

For trials of vaccine efficacy against CZS outcomes, the relationship between size of the recruited population and power varied with the timing of the trial in the course of the epidemic (Table 2, Figure 2). Of the three starting rules, those trials following rule 1 began when a country reached a weekly rate of 25 cases per 100,000 persons and achieved 80% statistical power at the lowest sample size. Interestingly, trials begun a week after the peak of an epidemic (rule 3) required an intermediate sample size, while those trials beginning when a country reached a higher weekly infection rate of 50 cases per 100,000 persons (rule 2) required the largest sample sizes to reach 80% power (Table 2, Figure 2). For all starting rules, increasing assumed vaccine efficacy from 70% to 90% resulted in lower participant numbers. No trial in the 50% assumed efficacy scenarios achieved adequate power with sample sizes of up to 120,000 participants.

Table 2:

Minimum required sample size and study duration for trial scenarios with 80% power

	Assumed Efficacy: 50%		Assumed Efficacy: 70%		Assumed Efficacy: 90%
Trial Scenario Endpoint	N	Duration (days)	N	Duration (days)	N	Duration (days)
Infection Based
Weekly Testing	8357	296	3825	287	2013	294
Biweekly Testing	11076	306	4732	307	2466	313
Monthly Testing	21046	307	9263	301	4732	321
Symptom-based
Symptomatic ZVD	35094	316	15608	310	7451	324
CZS-based
Start at 25/100,000 pw rate	-	-	79451	598	44071	626
Start at 50/100,000 pw rate	-	-	110585	507	62468	530
Start after Peak	-	-	93603	515	52562	546

Figure 2: Statistical power to detect vaccine efficacy by number of participants recruited.

Statistical power to detect vaccine efficacy by number of recruited participants for seven different clinical trial scenarios and three levels of assumed vaccine efficacy. Trials with infection-based endpoints, with laboratory tests spaced weekly, biweekly, or monthly, achieved 80% power for fewer participants than trials with symptomatic disease or Congenital Zika Syndrome endpoints. Trials with CZS-based endpoints that were initiated earlier in the course of an epidemic or following an epidemic peak achieved 80% power for fewer participants than similar trials that began at an intermediate average start time.

To better understand the effect of starting conditions on sample size for trials with CZS-based endpoints, we plotted the number of infections that occurred before completion of the month-long ramp-up period, after which the vaccine was considered protective (Figure 3, top). These infections are removed in the simulation to accurately compare the vaccine and control groups. High infection rates in the first month of the trial appear to lead to more infections being excluded from the remainder of the trial analysis. Subsequently, as women are more likely to conceive at the beginning of the trial, the number of first-trimester infections is also lower (Figure 3, bottom). Trial simulations following starting rule 2 appear to have more infections removed during the pre-immunity period, and so also have fewer first-trimester infections recorded.

Figure 3: Number of infections occurring in period before vaccine immunity is achieved, and number of first trimester infections in CZS scenarios.

Top: Number of infections occurring in the first four weeks of simulated trials, before the vaccine was assumed to become protective, by number of recruited participants, for CZS-based endpoint scenarios with three different starting rules. Bottom: number of infections during the first trimester of pregnancy, by number of recruited participants.

3.2. ZIKV infection- and symptomatic disease-based endpoints

For trials of vaccine efficacy against ZIKV infection or symptomatic Zika Virus Disease, the timing of the trial in the course of the epidemic did not alter sample size requirements, and so data for only one starting rule was analyzed further. In all cases, trial scenarios achieved 80% power for far fewer participants than the number required for the CZS endpoint (Table 2, Figure 2). For ZIKV infection, we also studied the effect of lengthening the time between routine testing visits, from weekly to biweekly or monthly. As expected, trial scenarios with a ZIKV infection endpoint identified through weekly testing were adequately powered with the fewest participants in all simulation scenarios. Doubling the time between routine tests to biweekly increased the required sample size by 26%, averaged over assumed vaccine efficacies. Approximately 2.4-times as many participants as in trials with weekly testing were needed when we increased the testing interval to monthly. For symptomatic Zika Virus Disease, sample sizes were 64% greater, on average, than those required for infection-based endpoints with monthly testing.

The length of trials, in terms of shortest mean study duration in simulations, changed little between infection and disease outcomes (Table 2). Assuming that trials recruit the minimum sample size found in simulations, extending the time between testing or moving to the symptomatic Zika Virus Disease endpoint did not appear to dramatically increase the length of the overall study period. For example, within trial scenarios with assumed vaccine efficacy of 70%, moving from weekly testing to biweekly testing increased the minimum sample size by 907 participants, but only increased the mean study period by twenty days. In trials with monthly testing, the mean duration was six days less than in trials with biweekly testing.

4. Discussion

CZS is the most severe outcome of ZIKV infection. As an endpoint for a ZIKV vaccine trial, it ensures that vaccine efficacy is shown against the most serious outcome of infection. While ZIKV incidence rates are low, however, CZS endpoints may not be a feasible option for vaccine efficacy trials.

Perhaps surprisingly, simulated trials with CZS endpoints in our study did achieve adequate statistical power to detect vaccine efficacy in some scenarios. These particular trials were those simulated with a start date earlier in the epidemic or a week after the peak of an epidemic. Sample sizes were greater at the intermediate start time, when incidence was highest. This result is tied to the nature of CZS risk, which varies with the trimester of infection. Probability of conception is greatest in the first months of regular sexual activity, and it was assumed women were within the first six months of conception attempts at the start of the trial. This indicates that the majority of trial pregnancies will begin near the trial start date. When this is also a period of high ZIKV transmission, infection in the first trimester and, subsequently, CZS outcomes are more likely. However, we assumed the vaccine would not yet be protective in the first four weeks of a trial, and all infections occurring in those weeks were removed from the final dataset. Figure 3 illustrates the consequences of these dynamics in our simulation. Those trials that began at the intermediate start time, when a country reached a 50/100,000 person-weeks infection rate, have a greater number of infections removed during the pre-immunity period, and fewer first trimester infections recorded.

Comparing the feasibility of endpoints is thus further complicated by the nature of CZS risk. As with other TORCH pathogens, the risk of adverse outcomes varies with trimester of maternal infection. For example, maternal cytomegalovirus infection is less likely to be transmitted to a fetus in early pregnancy but, given fetal infection, more likely to lead to disabilities or other symptoms at birth [24]. The complex relationships between trimester of infection and overall risk must be considered when selecting congenital infection outcomes as trial endpoints.

All CZS-endpoint simulations relied on several important assumptions, such as the ability to identify women trying to conceive, the point process with all individuals recruited and, if applicable, vaccinated at the start of the trial—thereby ensuring all women entered the trial at a period of similar ZIKV incidence—and the determination of CZS outcomes at birth. Delayed manifestations of other congenital syndromes, namely congenital rubella syndrome, are well documented [25]. Mounting evidence suggests CZS may also manifest some time after birth [26,27]. It would be necessary to extend follow-up time in vaccine trials to avoid missing delayed CZS cases.

We focused simulations on populations of women who were not already pregnant at the time of recruitment. We assumed this because of the potential delay in recognizing pregnancy, as well as the month we assumed for immune ramp-up before the vaccine was protective. These factors prevent inclusion of the entire first trimester, when CZS risk is greatest. We recognize that recruiting women who are planning on becoming pregnant presents serious ethical concerns. Choosing this population for our simulation study, however, demonstrates the complex nature of congenital infection that should be carefully considered when designing trials of vaccine efficacy against teratogenic pathogens.

We worked with eighty-two weeks of seasonal risks and assumed all pregnancies and infections must occur by the final week, regardless of how we varied the starting time of the trial. As a result, some trial scenarios ran for the entirety of the eighty-two weeks, while others ran for a shorter period. It can be expected that sample sizes would be greater for trials of shorter duration, as we artificially ended ZIKV transmission at the final week despite of any pregnancies that were not completed by that time. This eliminated any chance of infection for some pregnancies, as well as chance of multiple infections for others. Fewer pregnancies may be expected in later trial weeks, however, as we assumed all women entered the trial at an average probability of conception, and this probability declined significantly by the end of the simulated trials.

Despite these limitations, simulation results revealed a critical relationship between ZIKV incidence at the beginning of a trial and the feasibility of a CZS clinical endpoint. In the event a trial with CZS endpoints remains infeasible regardless, infection- and disease-based endpoints are alternatives. Trial duration varied little between the infection endpoint scenarios, regardless of time between laboratory tests (Table 2). Trials that extend the time between sampling, though recruiting more individuals, halve or quarter the laboratory tests required per participant. This decreases participant inconvenience and potentially improves protocol adherence. These endpoints also avoid the need for intense trial surveillance to identify mild disease cases, which is recommended for trials that rely on symptomatic Zika Virus Disease endpoints [11]. Disease endpoints also required an additional 64% increase in participant numbers over monthly-testing ZIKV infection trial scenarios.

In terms of laboratory testing, we did not vary the sensitivity of tests to compare urine versus blood testing methods. Case identification was assumed to rely only on performing the test within the period of detectable RNA. We assumed that trials will follow current recommended laboratory protocols, which for ZIKV diagnosis currently requires paired blood and urine testing [11,14,15]. As the difference in the RNA detection interval between these sample types remains debatable, we used blood testing parameters to simplify the simulation. These tests have lower limits of detection that may be higher than the viral load required for maternal-fetal ZIKV transmission, so we recommend caution in interpreting maternal infection-based trial results in terms of CZS risk.

In all simulations, we assumed the vaccine would be protective after one month. However, this period was kept constant in all simulations and did not vary between endpoints. Similarly, as we relied on an assumed annual cumulative incidence rate, the qualitative patterns between trial scenarios will be more robust than sample sizes or trial durations provided by the simulation (Table 2). Actual values will depend on the vaccine under study and on incidence during the trial period.

In conclusion, though laboratory-confirmed symptomatic Zika Virus Disease and infection-based endpoints are perhaps more feasible, we found CZS clinical trial endpoints were possible in several simulated trials due to the relationship between CZS risk at trimester of infection and the timing of the trial in the course of an epidemic. This result highlights additional considerations needed when designing studies with pregnancy-related clinical endpoints, including studies involving other teratogenic pathogens.

Table 1:

Simulation Parameters

Parameter	Value	Reference
Incubation period in days, Median (25%, 75%)	5.9 (4.6, 7.6)	[19]
Persistence of viremia in days, Mean (25%, 75%)	9.9 (5.8, 12.7)	[19]
CZS risk, 1st trimester infection	0.15	[22]
CZS risk, 2nd/3rd trimester infection	0.0227	[22]
CZS risk, infection during multiple trimesters, including 1st	0.21	[22]
Contraceptive use, proportion of Latin American women aged 15-49 years	73%	[23]
Recruited population size, assumed vaccine efficacy, conception probability, start date within scaled epidemic data	Varied