In Pursuit of an HIV Vaccine: Designing Efficacy Trials in the Context of Partially Effective Nonvaccine Prevention Modalities

Abstract

The HIV prevention landscape is evolving rapidly, and future efficacy trials of candidate vaccines, which remain the best long-term option for stemming the HIV epidemic, will be conducted in the context of partially effective nonvaccine prevention modalities. It is essential that these trials provide for valid and efficient evaluation of vaccine efficacy and immune correlates. The availability of partially effective prevention modalities presents opportunities to study their interactions with vaccines to maximally reduce HIV incidence. This article proposes an approach for conducting future vaccine efficacy trials in the context of background use of partially effective nonvaccine prevention modalities, and for conducting future vaccine efficacy trials that provide nonvaccine prevention modalities in one or more of the randomized study groups. Strategies are discussed for responding to emerging evidence on nonvaccine prevention modalities during ongoing vaccine trials. Next-generation HIV vaccine efficacy trials will almost certainly be more complex in their design and implementation but may become more relevant to at-risk populations and better suited to the ultimate goal of reducing HIV incidence at the population level.

Introduction

Since 2010, the field of HIV prevention has been energized by the results of a number of clinical trials providing evidence that oral tenofovir, tenofovir vaginal gel, or oral tenofovir plus emtricitabine (TDF-FTC, Truvada), administered to HIV-negative persons as preexposure prophylaxis (PrEP) agents, are partially effective for HIV prevention.^1–4 These results are encouraging, even while other trials have failed to find the efficacy of these same interventions,^5–10 suggesting substantial heterogeneity in PrEP efficacy across at-risk populations, potentially due to differences in adherence patterns. Provision of antiretrovirals (ARVs) to the HIV-positive partner in a serodiscordant relationship has also proven highly effective for preventing heterosexual transmission to HIV-uninfected partners.¹¹ A number of additional nonvaccine HIV prevention efficacy trials, or open-label extensions thereof, are still underway.¹²

An effective HIV vaccine remains the best long-term option for stemming the HIV epidemic, and well-designed vaccine efficacy trials will accelerate this development. The RV144 “Thai Trial” has provided the only evidence to date of even partial efficacy of an HIV vaccine, with an estimated 31% reduction in HIV incidence.¹³ HIV Vaccine Trials Network (HVTN) 505 was stopped in April 2013 due to lack of efficacy.¹⁴ Several other efficacy trials are in the planning stages.¹⁵ It is essential that future efficacy trials address the most pertinent questions in vaccine evaluation, while taking into account the larger HIV prevention landscape. Attention is needed to ensure that, in the context of nonvaccine prevention modalities (NVPMs), vaccine clinical trial designs provide for valid and efficient assessment of vaccine efficacy¹⁶ and evaluation of immune correlates of risk and protection.^17–21

Oral PrEP is likely to be used at an increasing rate by some subset of vaccine trial participants. The U.S. Food and Drug Administration recently expanded the approved indications of TDF-FTC to include reduction of the risk of sexually acquired HIV infection in adults at high risk of acquiring HIV infection. TDF-FTC is the first ARV to be approved for prevention of HIV. This is an important advance for the field but numerous questions remain to be answered. Issues include how to measure and optimize adherence, how to quantify the dependence of intervention efficacy on subject characteristics, assessment of long-term toxicities, and further study of drug resistance that may develop among those who acquire HIV while taking PrEP. Novel delivery mechanisms, formulations, and dosing strategies of prophylactic ARVs are being explored to improve adherence, acceptability, and potentially the durability of response.^12,22 In coming years, additional manufacturers will likely apply for approval of their products for prevention indications and uptake of prophylactic ARVs will increase.

The existence of partially effective biomedical interventions working by different mechanisms provides the opportunity to evaluate combination prevention strategies for optimal reduction of HIV incidence.²³ Theoretically, combining oral or topical PrEP with vaccines may produce additive, or even synergistic effects,^16,24,25 especially beneficial in contexts where viral exposure is high. Synergy could occur if (1) provision of PrEP during the vaccination phase (which may range from 3 to 12 months, depending on the regimen) provides protection until the vaccine-induced protective immune response has matured; (2) PrEP during the vaccination and immediate postvaccination phases protects activated CD4⁺ cells from infection if exposure occurs during this critical period; (3) a partially effective vaccine provides protection from acquisition during periods of lower adherence to PrEP or for systemic PrEP regimens that are not taken daily; (4) having ARV levels intracellularly at the site of infection (e.g., vaginal or rectal compartment) increases the likelihood of abortive infection; native viral antigen expression might then add to the immunogenicity that the vaccine elicits^24,25; or (5) the threshold for virus escape is increased, requiring mutations in both the reverse transcriptase gene and in the target epitopes of the CD8⁺ cytotoxic T cells thought to control disease progression.

Future trials involving partially effective NVPMs, either in combination with or in comparison to vaccine regimens, will be most valuable if they provide information on the effectiveness of prevention programs based on these interventions. Following Schaper et al.²⁶ we define the effectiveness of a prevention program to be its total effect on the spread of HIV in the population. Program effectiveness is influenced by both the biological and behavioral effects of an intervention, with the latter effects potentially influenced by imperfect adherence, and risk compensation^27,28 if the intervention is unblinded. While randomized clinical trials can only partly address program effectiveness, given that interventions are evaluated in the artificial setting of a clinical trial, they can be designed to capture behavioral effects and/or the combination of both biological and behavioral effects. These effects are particularly important to understand for NVPMs, such as daily oral PrEP, that rely on participants' sustained adherence to the prevention strategy.

This article proposes an approach to HIV vaccine clinical research that addresses the new challenges and research questions that are arising in the context of partially effective NVPMs. Specifically, it first discusses the design of vaccine efficacy trials to account for participants' use of NVPMs that are not provided through the study, called “background” NVPM use. Second, it describes and evaluates candidate designs for efficacy trials that provide NVPMs in one or more of the randomized study groups. Proposed strategies for responding to emerging evidence on NVPMs during ongoing vaccine efficacy trials are described in the Appendix.

Materials and Methods

We define an NVPM as a biomedical intervention that is not currently standard of prevention and not part of the HIV prevention package provided to all vaccine efficacy trial participants. We assume that the NVPM has been found to be partially efficacious in a previous efficacy trial; one exception to this is an NVPM codeveloped with a vaccine that would presumably not have been tested on its own in a prior efficacy trial. Examples of potential NVPMs include oral ARVs for pre- or postexposure prophylaxis (PrEP or PEP), microbicides (topical PrEP), vaginal rings, and male circumcision. Because only individual-level randomized vaccine efficacy trials that do not enroll sexual partners are planned to evaluate current vaccine candidates, we focus exclusively on NVPMs that are provided to individual participants rather than to HIV-infected partners (e.g., treatment as prevention) or to communities (e.g., test and treat strategies). For simplicity we also assume that there is a single NVPM of interest, but the concepts and methods generalize to multiple NVPMs.

Our focus is at-risk populations in the United States and abroad. The standard of HIV prevention, namely the best possible set of real-world prevention strategies, clearly will vary among countries and populations. We operate under the principle that it is this local standard of prevention that informs the choice of research strategy.

Results

Efficacy trials with background use of NVPMs

A prototype HIV vaccine efficacy trial is randomized and double-blinded: high-risk HIV-negative participants are assigned Vaccine or Vaccine-placebo and followed for incident HIV infection with diagnostic testing performed every 2–6 months. Postinfection endpoints such as viral load, CD4⁺ T cell count, and sequences of HIV viral isolates are measured.

Vaccine efficacy trials must be designed to take into account trial participants' background use of partially efficacious NVPMs that are not provided as part of the study intervention (a later section will discuss trial designs that provide an NVPM). Because the trial design is randomized and double-blinded, the extent of NVPM use is expected to be balanced across the Vaccine and Vaccine-placebo arms, alleviating concerns about bias in the assessment of vaccine efficacy. However, background use of the NVPM will lower the HIV incidence in the placebo group, and this must be taken into account at the design stage.

HVTN 505, a Phase 2b efficacy trial evaluating a multiclade HIV-1 DNA plasmid vaccine plus multiclade HIV-1 recombinant adenoviral type 5 vector vaccine in adenovirus 5-seronegative, circumcised, men and male-to-female transgender persons who have sex with men, is an example. Following the initial announcement of the iPrEx study results demonstrating the partial efficacy of PrEP² in a study population similar to that assessed in HVTN 505, the vaccine trial was redesigned to anticipate a modest amount of PrEP use during the trial. In particular, the design modification allowed up to 20% of the total person-years at risk to be periods of PrEP use. At the extremes, this 20% could arise due to 20% of the participants using PrEP throughout the follow-up period or to all participants using PrEP 20% of the time; the reality would be somewhere in between and the sample size calculations below give the same answer for any of the ways that yield the 20% figure.

Powering trials to account for background NVPM use

In general, accounting for background use of partially effective NVPMs will necessitate larger vaccine trials, since power for evaluating vaccine efficacy (VE), typically measured as one minus the relative risk of HIV infection in Vaccine vs. Vaccine-placebo recipients, is lower with reduced placebo group HIV incidence. We illustrate the impact on trial size using a Phase 2b vaccine efficacy trial design very similar to that described by Gilbert et al.¹⁵ While this design may be used to assess multiple HIV vaccine regimens in the same trial, we focus on just one vaccine regimen and Stage 1 of the two-stage design (Stage 2 evaluates durability of vaccine efficacy). Participants are enrolled over 18 months, randomized to Vaccine or Vaccine-placebo, and tested bimonthly for HIV infection for 18 months per participant. Without NVPM use, the HIV incidence in the placebo group is anticipated to be 4% annually. In total, 3,800 participants are required (N=1,900 per arm) for 90% power to reject the null hypothesis that VE ≤0% under the design alternative that vaccine efficacy is 40%. Figure 1 shows the increase in sample size that results under increasing fractions of participants using the NVPM and under increasing NVPM efficacy, assuming that the NVPM does not affect vaccine efficacy. For example, if 50% (or 100%) of the person-years at risk is under NVPM use and the NVPM has 60% efficacy, then the required sample size increases to N=5,300 (N=9,200).

FIG. 1.

Sample sizes for a two-arm vaccine efficacy trial in the context of background nonvaccine prevention modality (NVPM) use. Participants are enrolled over 18 months, with halved accrual in the first 6 months, randomized with equal probability to Vaccine or Vaccine-placebo, and tested bimonthly for HIV infection for 18 months per participant. The placebo group incidence, absent NVPM use, is assumed to be 4% annually. A total of 10,000 trials are simulated and each design is sized to ensure that the median number of infections yields 90% power to detect 40% vaccine efficacy using a one-sided 0.025-level log-rank test (rejecting the null hypothesis that vaccine efficacy is less than 0%), allowing for 5% annual dropout. Calculations assume that the NVPM does not affect vaccine efficacy. (A) Total trial size is shown as a function of the efficacy of the NVPM (TE-NVPM), for different fractions of total person-years at risk that are periods of NVPM use. (B) Total trial sizes of less than 5,000 participants are shown, where the impact of low fractions of total person-years of NVPM use can be evaluated.

Interim monitoring of the overall incidence of HIV infection in the trial, as part of an operational futility monitoring plan, can be used to allow for the possibility that NVPM use or NVPM efficacy is higher than was anticipated at the design stage. Specifically, criteria based on low incidence, absent a large estimated VE, may be prespecified to allow the sample size to be increased or the trial to be terminated for operational futility if the primary objective to assess vaccine efficacy cannot be addressed in a timely manner.

Measuring NVPM use among trial participants

Measuring NVPM use among trial participants is necessary for addressing the following study objectives: (1) assessing VE among participants using/not using the NVPM near the time of acquisition, and assessing NVPM modification of VE; (2) assessing whether NVPM use near acquisition modifies the vaccine effect on postinfection endpoints; (3) assessing the impact of NVPM use on vaccine immunogenicity; and (4) assessing immune correlates of risk and protection while correcting for potential confounding effects of NVPM use, as well as assessing whether the correlates differ between NVPM users and nonusers. Typically, efficacy trials are underpowered to address these objectives unless NVPM effects are large. Moreover, these objectives are better addressed in studies where NVPMs are provided (and controlled) as part of the study intervention. Therefore any findings serve to motivate follow-up clinical studies.

A basic component of any plan to measure NVPM use involves obtaining self-reported use data from all participants at specific study visits. However, given the limited accuracy of self-report data and poor precision in capturing the level/frequency of use, biological specimens (plasma and/or mucosal samples—the latter particularly in the case of topical PrEP) are also tested in selected participants to assess NVPM use and nature of use. For example, the HVTN 505 redesign includes monitoring by self-report of PrEP/PEP use as well as plasma ARV drug level testing.

We propose a two-phase sampling design for NVPM biological specimen testing. Two-phase designs are well-established tools for measuring inexpensive “proxy” data on the entire trial population and expensive “gold-standard” data on a selected subsample.^29–36 Here the Phase 1 proxy data are self-reported NVPM use and any other subject characteristics that predict actual use, and the Phase 2 data are the biological specimen test results measured retrospectively on a subset of participants as follows: Samples are tested for participants who become HIV-infected (cases) at and just prior to HIV diagnosis. At each visit where testing is performed, uninfected participants (controls) who self-report NVPM use are tested along with a random sample of those not self-reporting use. Participants in the immunogenicity cohort, the subset of trial participants in whom vaccine-induced immune responses are assessed, are also tested at the primary time point for measuring immunogenicity. Samples collected at enrollment may be tested for all cases, to allow for a case-only assessment of modification of VE by baseline NVPM use.^37–39 In some settings it may be cost-effective to test a larger set of subjects at baseline in order to use baseline NVPM information as a predictor of risk of HIV infection for increasing the efficiency of VE estimation.⁴⁰ An important feature of this proposal is that the two-phase design and analysis exploits the self-reported NVPM use data to increase statistical efficiency; this strategy has not routinely been employed in PrEP efficacy trials.^2,3,5

Specialized two-phase statistical methods are needed to evaluate VE by NVPM use, since gold standard specimen test results are available only for the Phase 2 subjects.^32,34,35 To evaluate NVPM effects on postinfection endpoints and on vaccine immunogenicity, NVPM use data are available for all subjects in the analysis, so that two-phase methodology is not needed.

Efficacy trials with NVPMs as study interventions

As NVPMs become part of the standard of prevention for at-risk populations, it will become appropriate to consider vaccine efficacy trial designs that include NVPMs as part of the study intervention. We consider the five most compelling design options (Designs A–E) shown in Table 1. Interventions are randomized and double-blinded. Primary analyses are intention-to-treat (ITT), reflecting the priority placed on evaluating the effectiveness of the interventions and on assessing causal effects of treatment assignment; important secondary analyses assess treatment effects in adherent subgroups (see, e.g., Dai et al.⁴¹). The treatment efficacy (TE) specified for a given intervention measures the HIV incidence among those assigned the intervention relative to those assigned a placebo version of that intervention, and therefore this parameter aggregates biological and behavioral (i.e., adherence) effects of being assigned the intervention. The designs are illustrated using the Phase 2b efficacy trial design described above.¹⁵ For concreteness the statistical calculations consider TDF-FTC administered as oral PrEP as the NVPM of interest, assuming PrEP efficacy of 60%, which represents the midpoint of the positive efficacy estimates to-date (these range from 44% to 75%).^2–4 Designs are sized to have at least 90% power to evaluate all primary efficacy objectives.

Table 1.

Potential Vaccine Efficacy Trial Designs Incorporating a Nonvaccine Prevention Modality as a Study Intervention

Primary objectives	Study design	Contexta	Comments
1. Evaluate TE-Vaccine+NVPM 2. Evaluate TE-Vaccine 3. Compare TE-Vaccine vs. TE-Vaccine+NVPM	Design A • Vaccine+NVPM • Vaccine+NVPM-placebo • Vaccine-placebo+NVPM • Double-placebo	Expect moderate at best TE-NVPM or NVPM is not available to the trial population, so that double-placebo is justified. NVPM-alone arm is of interest.	The following secondary objectives can also be addressed: 1. Evaluate TE-NVPM 2. Compare TE-Vaccine vs. TE-NVPM 3. Assess vaccine-NVPM synergy/antagonism
1. Evaluate TE-Vaccine+NVPM 2. Evaluate TE-Vaccine 3. Compare TE-Vaccine vs. TE-Vaccine+NVPM	Design B • Vaccine+NVPM • Vaccine+NVPM-placebo • Double-placebo	Expect moderate at best TE-NVPM or NVPM is not available to the trial population, so that double-placebo is justified. NVPM-alone arm is not of interest.	This is Design A without an NVPM-alone arm, and it cannot address the secondary objectives listed above for Design A.
1. Evaluate TE-Vaccine in the context of NVPM	Design C • Vaccine+NVPM • Vaccine+NVPM-placebo • Vaccine-placebo+NVPM	NVPM is available and expect moderate-to-high TE-NVPM, so that double-placebo is unwarranted. Questions remain regarding NVPM use, e.g., unknown long-term safety profile, so that Vaccine-alone arm is justified.	The Vaccine-alone arm is useful for exploratory analysis, e.g., to compare TE-Vaccine vs. TE-NVPM when the data are combined with an estimate of TE-NVPM from a separate NVPM efficacy trial.
1. Evaluate TE-Vaccine in the context of NVPM	Design D • Vaccine+NVPM • Vaccine-placebo+NVPM	Double-placebo is unwarranted because NVPM is available and is approaching standard of prevention, high TE-NVPM is expected, or target population is highly interested in using the NVPM. Vaccine-alone arm is not justified.	This is Design C without a Vaccine-alone arm.
1. Evaluate TE-Vaccine+NVPM	Design E • Vaccine+NVPM • Double-placebo	Double-placebo is warranted but Vaccine-alone and NVPM-alone arms are not, since Vaccine and NVPM will always be provided together.	Primarily of interest when the vaccine and NVPM have been codeveloped.

^aImplicit in all contexts is that the efficacy of combination prevention (TE-Vaccine+NVPM) is of interest.

Designs target evaluation of the vaccine, NVPM, or combination prevention (vaccine+NVPM) treatment efficacy.

NVPM, nonvaccine prevention modality; TE, treatment efficacy.

A fundamental question involved in the choice of study design is whether a Double-placebo arm (Vaccine-placebo+NVPM-placebo) is justified. Ethicists and community members will be essential for informing this choice, and will also be needed for choosing the general research strategy. We select designs under the assumption that a Double-placebo is warranted only if low-to-moderate NVPM treatment efficacy is anticipated or the NVPM is not available to the trial population. Low-to-moderate TE-NVPM may be attributable to low biological efficacy, low participant adherence, or a combination of both; and lack of availability may be due to high cost of the intervention, lack of infrastructure to deliver it, or inadequate product availability. On the other hand, use of a Double-placebo may not be justified if the NVPM is available to the trial population and is nearing standard of prevention, expected to have high efficacy, or of high interest to participants. In general, this implies that all trial participants should be provided the NVPM. Another working assumption is that if moderate-to-high TE-NVPM is expected but sufficient questions remain regarding NVPM use, for example, an unknown long-term safety profile, a Double-placebo may not be justified but a Vaccine-alone arm may be of interest.

Biological plausibility of synergy or antagonism between the vaccine and NVPM is another important design consideration. We define statistical evidence of biological synergy/antagonism as an interaction between the vaccine and NVPM on a multiplicative scale. Specifically, suppose treatment efficacy is measured by one minus the relative risk, TE=1 – RR. Then, rejecting the null hypothesis of additivity that RR-Vaccine+NVPM=RR-Vaccine×RR-NVPM in favor of the alternative hypothesis RR-Vaccine+NVPM<RR-Vaccine×RR-NVPM provides evidence of synergy, and the reverse inequality indicates antagonism. We assume henceforth that vaccine-NVPM additivity or synergy is anticipated; in the unlikely event that antagonism is expected and the vaccine warrants further study, a design that compares the two interventions head-to-head (but not together) may be of interest.

Supplementary Fig. S1 (Supplementary Data are available online at www.liebertpub.com/aid) shows the decision tree we used to identify the most appropriate study design, under these basic tenets.

Objectives and associated study designs

The four-arm Design A of Table 1, which randomizes participants to Vaccine+NVPM, NVPM-alone, Vaccine-alone, or Double-placebo, is most informative for teasing apart the vaccine and NVPM contributions to HIV prevention. This design is appropriate when a Double-placebo arm is justified and an NVPM-alone arm is of interest. In addition to addressing the primary objectives of evaluating Vaccine and Vaccine+NVPM treatment efficacy and comparing Vaccine vs. Vaccine+NVPM efficacy, the inclusion of an NVPM-alone arm allows three secondary objectives to be addressed: (1) evaluate NVPM efficacy, (2) compare Vaccine vs. NVPM efficacy, and (3) assess synergy/antagonism of the vaccine and NVPM. The three-arm Design B, which is Design A excluding the NVPM-alone arm, addresses the same primary objectives without addressing the three secondary objectives, with a 25% smaller sample size. Under equal treatment allocation, the sizes of these designs are functions of the smallest design alternative, 40% vaccine efficacy (TE-Vaccine) in our illustrations (Fig. 2). In the example with oral PrEP as NVPM, N=7,600 participants in total (N=1,900 per arm) are required for Design A and N=5,700 (N=1,900 per arm) for Design B. These sample sizes ensure adequate power to (1) detect 40% vaccine efficacy, rejecting the null hypothesis of less than 0% vaccine efficacy; (2) detect 76% efficacy of the combination prevention (TE-Vaccine+NVPM), rejecting the null hypothesis that TE-Vaccine+NVPM is less than 30%; and (3) compare the efficacy of the vaccine vs. combination prevention, rejecting the null hypothesis that TE-Vaccine=TE-Vaccine+NVPM. The Vaccine+NVPM design alternative of 76% is based on the assumption that the vaccine and NVPM effects are additive [1 – (1 – 0.4)×(1 – 0.60)=0.76] and the null of 30% efficacy has been used to design the majority of PrEP efficacy trials.^2,3,5 As shown in Table 2, the power is low-to-moderate for addressing the secondary objectives of Design A.

FIG. 2.

Sample sizes for Designs A–E of Table 1 as a function of NVPM treatment efficacy (TE-NVPM), for a vaccine efficacy trial in which participants are enrolled over 18 months, with halved accrual in the first 6 months, randomized with equal probability to each treatment arm, and tested bimonthly for HIV infection for 18 months per participant. The HIV incidence is assumed to be 4% annually, absent Vaccine or NVPM assignment. For each design, 10,000 trials are simulated and the design is sized to ensure that the median number of infections yields 90% power to address the primary objectives shown in Table 1, under 40% vaccine efficacy, TE-NVPM equal to the value on the x-axis, and the corresponding level of combination prevention efficacy (TE-Vaccine+NVPM), assuming Vaccine and NVPM efficacy additivity. Treatment arms are compared using one-sided 0.025-level log-rank tests with no adjustment for multiple hypothesis tests. Increasing TE-NVPM yields lower underlying HIV incidence for Designs C and D and higher TE-NVPM and TE-Vaccine+NVPM design alternatives for Designs A, B, and E.

Table 2.

Power for Addressing Secondary Objectives of Design A

	True TE parameters			Null hypotheses tested
Scenario	TE-Vaccine (%)	TE-NVPM (%)	TE-Vaccine+NVPM (%)	TE-Vaccine ≤0%	TE-NVPM ≤30%	TE-Vaccine+NVPM ≤30%	TE-Vaccine=TE-NVPM	TE-Vaccine=TE-Vaccine+NVPM	Vaccine and NVPM Additive
Additivity	40	60	76	90	90	100	53	99	5
Synergy	40	60	88	90	90	100	53	100	53
Antagonism	40	60	64	90	90	96	53	72	31
Additivity	40	70	82	90	100	100	91	100	5
Synergy	40	70	91	90	100	100	91	100	45
Antagonism	40	70	73	90	100	100	91	96	27
Additivity	40	80	88	90	100	100	100	100	5
Synergy	40	80	94	90	100	100	100	100	33
Antagonism	40	80	82	90	100	100	100	100	22

Power (%) for evaluating the treatment efficacy of the combination intervention (TE-Vaccine+NVPM) and the NVPM alone (TE-NVPM), comparing active treatment arms head-to-head, and testing for vaccine and NVPM synergy/antagonism under Design A of Table 1. Under TE-NVPM=60%, 70%, or 80%, Design A requires 7,600 participants (1,900 per treatment arm) for 90% power to detect TE-Vaccine=40% and the level of TE-Vaccine+NVPM shown assuming Vaccine and NVPM additivity, and to compare TE-Vaccine vs. TE-Vaccine+NVPM, rejecting the null hypotheses: TE-Vaccine ≤0%, TE-Vaccine+NVPM ≤30%, and TE-Vaccine=TE-Vaccine+NVPM using one-sided 0.025-level log-rank tests, respectively. Power is shown under vaccine and NVPM additivity, synergy (a 50% decrease in relative risk), and antagonism (a 50% increase in relative risk). Evidence for synergy/antagonism is evaluated using a two-sided 0.05-level likelihood ratio test under the Cox proportional hazards model.

TE, treatment efficacy.

Consider, in contrast, settings where a Double-placebo arm is not justified, but a vaccine-alone arm is of interest. Design C, which randomizes participants to Vaccine+NVPM, NVPM-alone, or Vaccine-alone, may be suitable. Under this design, the primary objective assesses vaccine efficacy in the context of the NVPM. Design C is large as a consequence of the reduced underlying HIV incidence across all arms (Fig. 2). In the example with oral PrEP as NVPM, Design C requires 13,800 participants (4,600 per arm) to detect 40% vaccine efficacy in the context of the NVPM.

If neither a Double-placebo nor a Vaccine-alone arm is justified, the two-arm Design D that randomizes participants to Vaccine+NVPM or NVPM-alone may be appropriate. Design D shares with Design C the primary objective of evaluating TE-Vaccine in the context of the NVPM. This objective is addressed without the Vaccine-alone arm and the design is therefore 33% smaller. For the efficacy trial example with oral PrEP as NVPM, N=9,200 participants in total (N=4,600 per arm) are required for Design D to be adequately powered to detect 40% vaccine efficacy.

A final option, appealing for its simplicity, is Design E that compares the combination prevention strategy (Vaccine+NVPM) to Double-placebo. This design is generally warranted only if the vaccine and NVPM will always be provided together, for example, because the prevention strategies were codeveloped. As shown in Fig. 2, Design E is small assuming that the combination prevention has high efficacy. In the example with oral PrEP as NVPM, N=1,400 participants in total (N=700 per arm) are required to detect 76% efficacy of Vaccine+NVPM with adequate power, rejecting the null hypothesis that TE-Vaccine+NVPM is less than 30%.

An important element of these designs is the choice of the null hypothesis regarding the treatment efficacy of each prevention strategy; the larger the null TE value, the larger the size of the trial. For testing Vaccine-alone versus Placebo, the null hypothesis of 0% vaccine efficacy is appropriate for a Phase 2b design,^15,42 given the goal to screen-in vaccine candidates with any efficacy at all. A higher null value, perhaps TE-Vaccine=25% or 30%, would be warranted for a Phase 3 licensure trial. For testing an NVPM alone versus Placebo, a null of 30% efficacy has been used for the majority of PrEP trials,^2,3,5 and accordingly we use TE-NVPM=30% as an example here. However, if, for example, the NVPM was previously found to have 60% efficacy in a similar setting, the appropriate null hypotheses may be that TE-NVPM and TE-Vaccine+NVPM are less than 60%.

Note that under Designs A–E, there may be additional use of the NVPM by participants that is not provided through the study. As described in the section on background NVPM use, the impact of this is to reduce the underlying HIV incidence, and the designs can take into account such usage through the choice of the HIV incidence parameter.

As shown in Fig. 2, Designs C and D tend to require the largest sample sizes whereas Design E is the smallest. While Designs C, D, and E address only one primary objective each, Designs A and B are most informative in that they address three primary objectives. Moreover, Designs D and E pose challenges for breadth of vaccine licensure indication given that the vaccine is studied only in combination with NVPM. Designs A and B are intermediate in size and allow for dissecting the individual components of combination prevention efficacy, and therefore we favor these designs. Design A increases the sample size of Design B by 33% and allows three additional secondary objectives to be addressed, albeit with low-to-moderate power for some without additional sample size increase.

Establishing noninferiority

An alternative primary objective for future trials is to establish noninferiority of the vaccine. Typically, noninferiority is considered when an existing intervention has proven efficacy.^43–45 We do not consider establishing noninferiority of Vaccine relative to a partially effective NVPM to be a compelling primary objective. However, establishing noninferiority of Vaccine relative to Vaccine+NVPM may be appealing, given the reduced complexity and expense of the Vaccine without NVPM. Because the combination prevention would not have proven efficacy in advance of the trial, including a Double-placebo arm would be important to ensure valid assessment of TE-Vaccine+NVPM in comparison to TE-Vaccine. Designs A and B that include Vaccine, Vaccine+NVPM, and Double-placebo arms could potentially be used to address both superiority and noninferiority objectives.

Defining noninferiority requires specifying a “margin” within which Vaccine is not clinically inferior to Vaccine+NVPM. A common approach is to define this margin to be a fraction of TE-Vaccine+NVPM, say 50%, measured on the log relative risk (RR) scale. This translates into declaring Vaccine to be noninferior to Vaccine+NVPM if RR-Vaccine <1.65×RR-Vaccine+NVPM, where 1.65=exp(0.5). In our illustrations with a design alternative of 40% vaccine efficacy [RR-Vaccine=0.6] and assuming Vaccine and NVPM additivity [RR-Vaccine+NVPM=RR-Vaccine×RR-NVPM], Vaccine would be noninferior to combination prevention if TE-NVPM <39% [RR-Vaccine+NVPM=0.6×RR-NVPM >0.6/1.65]. Therefore establishing noninferiority of Vaccine relative to Vaccine+NVPM may be justified as a primary objective only in settings where low TE-NVPM is anticipated.

Use of an NVPM-placebo

To further probe the effectiveness of the NVPM in Design A, the NVPM-placebo may be replaced with no further intervention. Provision of an NVPM-placebo may also raise ethical issues in settings where high efficacy has been observed in previous studies. If the NVPM-placebo is eliminated, the effect is to unblind participants as to NVPM assignment, while maintaining the blind as to Vaccine assignment. The parameters TE-NVPM and TE-Vaccine+NVPM then capture the impact of participants' knowledge of NVPM assignment on HIV incidence. Note that this strategy does not have the same appeal with Designs B, C, or E because for these designs knowledge of whether the NVPM has been assigned informs some participants as to whether they have been assigned Vaccine or Vaccine-placebo.

Nonrandomized comparisons

Whereas all comparisons between study arms discussed elsewhere are based on comparing groups of participants enrolled concurrently and randomized to different interventions, and therefore provide an unbiased assessment of intervention effects, there are additional comparisons that may be of interest but that require data from separate NVPM efficacy trials. For example, in Designs B and C, TE-Vaccine can be compared with TE-NVPM given external estimates of TE-NVPM. These comparisons are challenging to interpret, however, because they are subject to bias due to differences in baseline HIV incidence or NVPM efficacy between study populations. For this reason, we view such comparisons as exploratory.

Two-stage randomization

When adherence to the NVPM is anticipated to be a challenge, implementing Design A with a two-stage randomization strategy may be scientifically advantageous. Specifically, at baseline, participants are first randomized to Vaccine or Vaccine-placebo and then participants willing to be randomized to the NVPM are randomized to NVPM or NVPM-placebo (Fig. 3A). This strategy is useful because participants who are randomized to NVPM-placebo (Groups 2 and 4 in Fig. 3) as well as NVPM-unwilling participants (Groups 2a and 4a) can be included in the estimation of TE-Vaccine. This allows the design to achieve its objectives with the smallest possible sample size. Using oral PrEP as an example, the N=7,600 participant Design A could be reduced to N=6,900 participants, while maintaining 90% power for addressing its primary objectives, under the assumption that two-thirds of participants are willing to be randomized to PrEP. This two-stage randomization also applies to Design B where participants randomized to Vaccine-placebo and who are NVPM-willing are all assigned NVPM-placebo.

FIG. 3.

Two potential two-stage randomization schemes for Design A of Table 1. (A) Participants are first randomized to Vaccine or Vaccine-placebo, and at the second stage participants willing to be randomized to the NVPM are randomized to NVPM or NVPM-placebo while the remaining participants receive no further intervention. (B) At the second stage, participants desiring to use the NVPM are provided it, participants unwilling to use the NVPM are provided no further intervention, and the remaining participants are randomized to NVPM or NVPM-placebo. Squares denote points of participant decisions and circles indicate points of randomization.

An alternative randomization strategy for Design A queries participants at stage two to identify those who desire to use the NVPM (and therefore are provided it), those who are unwilling to use the NVPM (and therefore are not provided it), and those who are willing to be randomized to NVPM versus NVPM-placebo (and therefore are randomized) (Fig. 3B). This strategy maximizes the number of eligible participants; however the sample size is larger than that required under Fig. 3A given the inclusion of participants desiring the NVPM (Groups 1a and 3a) who do not contribute to primary analyses. Data from these participants are used for secondary analyses comparing Vaccine+NVPM vs. NVPM efficacy, and for assessing separately the effect of being randomized and blinded to NVPM receipt (Groups 3 vs. 4) versus choosing to be provided the NVPM (Groups 3a vs. 4a), albeit with a smaller sample size for the latter comparisons.

At the other extreme, a simple potential approach is to include only NVPM-unwilling participants in the study, and to randomize these participants to Vaccine or Vaccine-placebo. This design has the advantage of maintaining the size of current-generation vaccine efficacy trial designs, assuming minimal background NVPM use. However, it poses a potential challenge for recruiting participants and is powered only to assess vaccine efficacy; it provides no information on the vaccine in combination with the NVPM.

Discussion

With increased proliferation and uptake of partially effective HIV prevention strategies, vaccine trials will become more complex. Several additional design issues emerge.

Background use of PrEP/PEP may have implications for study blinding. Vaccine trials use specialized HIV diagnostic algorithms and counsel participants to avoid HIV testing outside the study because vaccination can cause seropositivity absent HIV infection; this is called vaccine-induced seropositivity (VISP).⁴⁶ Increased uptake of prophylactic ARVs and of home-based HIV tests may raise the likelihood that trial participants are knowingly or unknowingly being tested for HIV outside the study. In addition to its adverse effects on trial participants, VISP can unblind participants to treatment assignment. It is therefore critical to ensure that providers prescribing PrEP/PEP are knowledgeable about VISP and are linked with the vaccine research sites to ensure appropriate referrals for HIV testing. Vaccine trial participants are mentioned specifically in the package inserts for home HIV test kits, and the Risk Evaluation Mitigation Strategy that is developed with the FDA for licensure of these tests⁴⁷ would ideally include these considerations. Vaccine efficacy trials should continue to collect data on participant perceptions of treatment assignment in order to assess unblinding.

While the recent results demonstrating partial efficacy of the Thai vaccine regimen¹³ and other NVPMs^1–4 have shifted the focus toward HIV acquisition as the primary endpoint in next-generation vaccine trials, postinfection endpoints such as sequences of viral isolates and viral load remain important for informing on the mechanism of any vaccine effect and the vaccine's impact on postinfection disease course. However, treatment of HIV-infected patients is increasingly being initiated early after infection diagnosis, given evidence of improved potency of ARVs and tolerability of regimens, greater appreciation for the non-AIDS-related sequelae of untreated HIV viremia, and high efficacy of ARVs for preventing transmission of HIV to uninfected partners.^11,48 Early postinfection initiation of ARVs represents another type of nonvaccine intervention that poses challenges for vaccine evaluation. In future trials, fewer vaccine trial participants who become HIV infected will have postinfection endpoints measured beyond the acute and early phase, therefore making it difficult to assess vaccine effects on these endpoints.

Accurately measuring NVPM use among study participants will be an ongoing challenge. The approach proposed here allows for limitations in the self-reported data by augmenting these with NVPM use measured by biological specimen testing. Errors in the specimen test result, which may occur if the assay is able to detect NVPM use only within a short time window of sampling, are more concerning. For example, current generation assays can detect ARV use in plasma only within the past 14 days.⁴⁹ For analyses involving NVPM use data that are available for all subjects in the analysis, e.g., studying postinfection endpoints, existing methods can be employed to account for measurement error in the NVPM use data.^50,51 However, for analyses requiring two-phase methodology, existing methods are not well-developed to deal with measurement error in the Phase 2 data; this is an area requiring future research.

HIV prevention efficacy trials are typically implemented with interim monitoring for early evidence of harm, nonefficacy, or high efficacy of study interventions. These monitoring plans have been developed for evaluating vaccine candidates¹⁵ and their impact on trial size is typically minimal, given that the stopping boundaries are well away from the current Phase 2b design alternative of 40% vaccine efficacy. However, for designs that include vaccines and NVPMs as study interventions, interim monitoring strategies need to be reevaluated. If it is deemed appropriate to do more aggressive monitoring for NVPM or Vaccine+NVPM arms given prior evidence of NVPM partial efficacy, this monitoring may have a larger impact on trial size. Another design issue that will require more attention in future research is how to design the trials to account for potential time variations in intervention efficacy that may result from time variations in vaccine-induced immunity, NVPM adherence, or other factors over time.

Other prevention modalities, HIV-1 neutralizing monoclonal antibodies (mAbs), are currently being investigated for their potential use as HIV prevention agents. Strategies include injecting a recombinant vector with insert encoding mAbs and repeated injections of mAbs. Here, synergy between interventions is possible, and perhaps even likely: two mAbs may be combined to increase the breadth of the immune response, potentially blocking infection by more variants. For these agents, there is much stronger motivation for adequately powering a comparative study to detect synergy.

For NVPMs for which adherence is an issue, or for which there are other substantial behavioral effects, assessing intervention effectiveness is key for future trials and we recommend consideration of designs that assess the effect of unblinded assignment to the NVPM. For the four-arm Design A, a two-stage randomization scheme (first to Vaccine, then to NVPM) can be used to maximize the number of eligible participants and achieve the design's objectives with the smallest possible sample size. Additional inclusion of participants desiring to use the NVPM allows these subjects' data to contribute to secondary analyses such as those comparing NVPM effectiveness versus efficacy.

The intersection of prevention technologies creates new opportunities to conduct future HIV vaccine studies in the background of NVPM use and to evaluate combination prevention strategies to assess interactions between NVPMs and a vaccine. These trials also present opportunities to study the effectiveness of NVPMs and combination prevention strategies in comparison to vaccines for which participant adherence is less of an issue. Although future efficacy trials will be more complex in their design, study implementation, and evaluation of endpoints, they may become more relevant and applicable to diverse populations and better suited to the ultimate goal of reducing HIV incidence at a population level.

Appendix: A Case Study on Responding to Evidence Regarding NVPMs During Ongoing Vaccine Efficacy Trials: HVTN 505

As evidence of efficacious NVPMs emerges during an ongoing vaccine efficacy trial, the study team will face the questions of whether and how to modify the trial design. Assuming for simplicity that the ongoing trial uses a traditional two-arm Vaccine vs. Vaccine-placebo design, the options for responding to the emerging evidence are as follows, using labels consistent with those in Table 1.

• Option 0: Continue with the current design, but monitor for NVPM use and potentially increase the size of the trial to counter the decreased placebo group HIV incidence

• Option A: Add NVPM and Vaccine+NVPM arms

• Option B: Add a Vaccine+NVPM arm

• Option C: Add NVPM and Vaccine+NVPM arms, rolling the Vaccine-placebo recipients into the NVPM arm

• Option D: Add the NVPM to both study arms

• Option E: Add the NVPM to the Vaccine arm

Under Options A–C, going forward, the randomization fractions require adjustment to ensure equal-sized treatment arms after including the already enrolled participants. These design options are to be compared with respect to the scientific questions that can be addressed, study size, duration, and, for Options A–E, how the data collected up until the time of design modification can be utilized. The approach taken by the HVTN 505 team in reaction to the iPrEx study results is illustrative.

In 2010 iPrEx released results suggesting that use of daily oral TDF-FTC reduced the risk of HIV acquisition by 44% (95% CI: 15–63%) in an international men who have sex with men (MSM) population.² The RV144 Thai vaccine efficacy trial had also recently been published with the result of an estimated 31% VE.¹³ At the time, HVTN 505 was enrolling and randomizing N=1,350 participants in the United States to DNA prime/Ad5-boost vaccine or placebo. Enrollment was restricted to adenovirus-5-seronegative and fully circumcised MSM and male-to-female transgender persons who had sex with men (MTF). Roughly half of the participants had been enrolled. The primary endpoint was setpoint viral load (VL), and the study was not powered to assess the effect of the vaccine on HIV acquisition. The study team considered the following options (see Supplementary Table S1):

• - Continue with the current design, and add monitoring for PrEP/PEP use (Option 0)

• - Add acquisition as a coprimary endpoint, which would require an increase in the sample size, and add monitoring for PrEP/PEP use (Enlarged Option 0)

• - Modify the design to a four-arm trial to evaluate the efficacy of the Vaccine, PrEP, and Vaccine+PrEP in the U.S. adenovirus-5-seronegative, circumcised, MSM/MTF population (Option A)

Under Option A, participants would be randomized to Vaccine or Vaccine-placebo first, and then offered randomization to PrEP or PrEP-placebo. To maximize the number of participants already enrolled who could be included in the primary analysis, only those originally randomized to Vaccine and subsequently randomized to PrEP would be excluded, since the provision of Vaccine and PrEP would not be concurrent. To control for differences between the existing participants and newly enrolled participants, the final analysis would compare Vaccine and Vaccine-placebo groups stratified on the enrollment period and PrEP provision (PrEP vs. PrEP-placebo vs. refusal of second randomization).

The HVTN 505 redesign options were compared with respect to the scientific questions that could be addressed, with Option A being the most informative, allowing the Vaccine, PrEP, and Vaccine+PrEP effects on both viral load and acquisition to be evaluated. Option 0 was determined to have essentially no impact on study size or duration, whereas Expanded Option 0 increased the sample size by 1,850 participants and accrual by 17 months, and Option A increased the size by 2,250 participants and accrual by 20 months. The final choice, made based on input from the community, trial participants, site investigators, study team, and funding agency, was a scaled-down version of Expanded Option 0. HIV acquisition was added as a coprimary endpoint, but the sample size was set to N=2,200 to power the trial to detect VE=50% instead of VE=40%. The trial was stopped for efficacy futility in April 2013.⁸

Acknowledgments

The project described was supported by Grants AI-069470, RR-024156, and UM1AI068635 from the National Institutes of Health (National Institute of Allergy and Infectious Diseases and National Center for Research Resources). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Allergy and Infectious Diseases or the National Institutes of Health.

Author Disclosure Statement

No competing financial interests exist.