Methodologies for evaluating HIV prevention intervention (populations and epidemiologic settings)

Abstract

Purpose of the review

There have been over 30 HIV-prevention trials of which only four reported evidence of efficacy. The reasons for these failures may be due to ineffective interventions, but in part reflect the inappropriate selection of study populations and epidemiologic settings.

Recent findings

Three trials showed that male circumcision reduces HIV acquisition in men by 50–60%. In contrast, seven out of eight trials of bacterial and viral sexually transmitted infection (STI) control, and multiple microbicide trials show no efficacy for HIV prevention. Several microbicide trials found vaginal irritation and microulceration, which may increase HIV risk. Three vaccine trials failed to show efficacy and one trial suggested increased HIV risk in a subgroup of uncircumcised men.

Summary

The failure of most prevention trials reflects inadequate pretrial screening of potentially efficacious interventions, insufficient information on background HIV incidence, selection of high-risk populations with poor compliance and lack of generalizability, and treatment interruption during pregnancy all of which compromises power.

Introduction

The development of HIV-prevention interventions is a complex process. In general, observational studies, pretrial screening, in vitro and animal studies are the hypothesis generating investigations, which provide the rationale for hypothesis testing trials. Randomized trials provide evidence of efficacy and there have been over 30 HIV-prevention trials of which only four reported reduction in HIV acquisition [1]. Here, we review examples of sexually transmitted disease (STD) control, microbicide, male circumcision and vaccine HIV-prevention trials.

Individually randomized trials and study populations

The basic design of individually randomized HIV-prevention trials is to enroll HIV-uninfected individuals who are then randomized to a preventive intervention or a control condition, and the end point is based on a comparison of HIV incidence between randomized arms during follow up. Such trials have been used for assessment of T-cell [2,3••] and humoral HIV vaccines [4,5], microbicides [5–7,8•,9], diaphragm [10•], sexually transmitted infection (STI) treatment [11–16], and male circumcision [17,18••,19••]. The limitation of individually randomized trials is that an uninfected individual receives the intervention or control condition, but in most cases, there is no randomization of their potentially HIV-infected partners. Thus, most individually randomized trials only assess intervention effects on HIV susceptibility, but cannot assess the effects of the intervention on HIV infectivity. An exception to this is in trials, which treat an HIV-infected individual to prevent transmission to initially uninfected partners in an HIV-discordant relationship. Such discordant couples trials have been designed to assess the effects of herpes type 2 (HSV-2) suppressive therapy [20••,21••] or antiretroviral therapy of HIV-positive partners [1] to evaluate HIV transmission to their uninfected partners. Trials have also assessed the effects of HSV-2 suppression in HIV-infected persons on HIV genital tract shedding and plasma viral load, which are considered to be intermediate markers affecting infectivity [22,23].

The main determinant of the size and duration of an individually randomized HIV-prevention trial is the anticipated number of HIV-incident events. Higher incidence requires a smaller sample size and, as a consequence, many trials enroll high-risk populations to maximize incident events and thus reduce costs and trial duration. However, high-risk individuals such as commercial sex workers (CSWs) may be atypical of the general populations at risk of HIV, which can limit the external validity or generalizability of the trial findings. A particular difficulty arises with interventions that must be used with each act of intercourse such as microbicides or the diaphragm, because high-risk individuals tend to have frequent sexual contacts. Therefore, an intercourse-related intervention will require frequent use, which reduces compliance, and potentially adds to toxicity if frequent product use leads to adverse effects [10•]. Such problems have emerged with vaginal irritation and microulceration associated with microbicide use [7,8•,9]. Another disadvantage of high-risk populations is that they are often mobile and hard to follow up, so poor retention rates may compromise the trial results, particularly if losses to follow up are differential between study arms. Many studies lack information on pretrial HIV incidence, which is either estimated from prevalence data or inferred from behavioral characteristics. However, such estimates can be inaccurate, and the trial may ultimately be underpowered, if the observed incidence is lower than expected [3••,8•,24]. For example, a vaccine trial enrolled ‘high risk women’, who were assumed to have high-HIV incidence, but only one incident case was actually observed, and the study could not assess efficacy in women [3••].

In summary, where possible, it is highly desirable to enroll participants who are representative of the general population in which the intervention will ultimately be used, should it prove efficacious? This enhances the external validity of the trial, which is important for policy decisions. Also, general populations tend to be more compliant than high-risk populations, so the additional costs entailed by a larger sample size requirement may be offset by higher retention rates and better adherence. In addition, it is critical to have pretrial empirical data on HIV incidence in order to ensure the study has adequate power.

HIV-discordant couples may provide a relatively stable high-risk population with high-transmission rates. However, discordant couples are more difficult to identify and enroll. This is particularly the case if HIV-status disclosure is an eligibility criterion. Thus, discordant couples studies may encounter slow and prolonged study enrollment extending the trial duration and costs. Also, if trial eligibility requires couple disclosure of their discordant-HIV status, the rates of HIV infection are likely to decline due to a reduction in risk behaviors (e.g., increased condom use or sexual abstinence) following disclosure, which may reduce power. Moreover, knowledge of HIV-discordant status can lead to marital disruption, which then censors observation of the couple and compromise study power.

Another consideration is whether to include women at risk of pregnancy in a trial. If the intervention confers a risk of adverse pregnancy outcomes (e.g., efavirenz), then clearly women at risk of conception should be excluded from the trial, or be required to use reliable contraception, as a trial eligibility criterion [24]. However, in many trials contraception is not required and treatment is interrupted during pregnancy or lactation [7,8•,10•]. Such pregnancy-related treatment interruptions are common in microbicide trials, despite the fact that there is no a priori evidence of risk to the pregnant woman or her unborn child.

Community randomized trials

In certain circumstances communities rather than individuals are randomized and HIV-infected and uninfected participants resident in the communities are enrolled [25]. Community randomized trials have been used to assess STI control for HIV prevention in several African countries [11,12,14,15]. Whole communities were randomized to provide total population coverage of the intervention in order to maximize effects on STI prevalence and thus potentially on HIV acquisition and transmission. The rationale is that by treating whole populations, one can reduce STI exposures throughout sexual networks and hopefully reduce HIV infectivity and susceptibility. If individuals within a population were randomized to STI treatment or a control condition, there would be a risk of contamination if a treated participant had sexual relations with an untreated individual. Thus, community randomized trials allow estimation of the effects of the intervention in initially uninfected participants, as well as the effects that may accrue from reducing infectivity of HIV-positive participants [25].

Community randomized trials are more complex in design and require a larger sample size than individually randomized trials [26,27]. The complexity in design arises from the likelihood that risk behaviors and HIV incidence differ between communities, and lack of comparability of community characteristics can compromise the validity of comparisons [25,27]. To overcome this problem many community randomized trials use matched designs and randomized within matched groupings. The sample size requirements for community-randomized trials depend both on the background incidence of HIV and the degree to which HIV incidence or risk behaviors differ between communities [28]. Sample sizes need to be inflated to compensate for this community-level clustering by estimating the ‘design effect’ based on the interclass correlation or coefficient of variation in community-specific HIV incidence. In general, trials, which select more homogeneous communities, require a smaller sample size (i.e., smaller design effect) to achieve a specified power [27,28]. However, several community randomized trials selected nonhomogeneous clusters and this may have compromised study power [11,14] In addition, since the unit of analysis is the community cluster rather than the individuals within the randomized communities, it is preferable to have a larger number of randomized communities to maximize the degrees of freedom for efficacy analyses [26].

Approaches to analysis of trials

Randomized trials commonly use an intent-to-treat analysis, which compares HIV incidence between randomization arms irrespective of whether individual participants did or did not actually receive the intervention [29]. Such analyses are least biased but efficacy may be undermined, if compliance is poor. For example, persons randomized to receive a microbicide or a diaphragm may use it infrequently or not all and thus would not benefit from the intervention [10•]. Similarly, in trials of male circumcision, some men randomized to receive surgery may decline the procedure, or conversely, some men randomized to the control arm may obtain circumcision from nontrial sources [17,18••,19••]. These ‘crossovers’ would dilute the estimates of efficacy based on an intention-to-treat analysis. To assess whether compliance affected the study outcome, investigators use an ‘as treated’ or ‘per protocol analysis,’ whereby efficacy is assessed among those who actually received or did not receive the intervention. The problem with ‘as treated’ analyses is that the most compliant individuals are likely to be highly self-selected and atypical, and their risk behaviors may differ from the less compliant participants. Therefore, an as treated analysis may have only limited external validity and may not reflect the likely efficacy in a less selective programmatic setting.

Examples of HIV-prevention trials

We review the findings from HIV-prevention trials of STI control, male circumcision, microbicides and vaccines.

Trials of sexually transmitted infection control for HIV prevention

There was, compelling evidence that STIs are associated with HIV acquisition and transmission, and this provided the rationale for trials of STI control to prevent HIV infection.

Five community randomized trials of bacterial STI control trials were conducted in sub-Saharan Africa [12–15]. The trials were similar in size but differed in background HIV incidence, and in the interventions used for STI control (syndromic management versus presumptive treatment). One Tanzanian trial reported a 38% reduction of HIV incidence [11], whereas the other trials found no effect of STI control on community-level HIV incidence. All trials observed some decrease in STI prevalence, but none accomplished complete STI control. An individually randomized trial of CSWs in Nairobi assessed monthly presumptive azithromycin treatment and found no effect on HIV acquisition despite marked reductions in bacterial STIs [16]. Two individually randomized trials of HSV-2 suppression therapy in HSV-2 infected/HIV-negative individuals failed to observe a reduction in HIV acquisition [20••,21••].

Why did seven out of eight trials of STI control for HIV prevention fail to show efficacy despite compelling observational studies? A major weakness of the observational studies is that STIs and HIV are both acquired and transmitted by sexual intercourse. Thus, the associations between STIs and HIV infections may be mediated by high-risk sexual behaviors, rather than a causal effect of STIs as cofactors for HIV acquisition. Also, STIs are more infectious than HIV and are likely to be acquired prior to HIV, so the fact that STIs precede HIV infection may not indicate a causal association, but rather reflect differential infectivity [30]. Finally, although STIs may be causally linked to HIV susceptibility or infectivity in individuals, their contribution to HIV transmission at a population level will depend on the prevalence of STIs in the population and the magnitude of the risk of HIV acquisition/transmission in the presence of STIs. This combination of risk factor prevalence and magnitude of cofactor risk effects can be estimated from the ‘population attributable fraction’ (PAF), and the PAF in the community randomized STI control trials may have been too small to be detected [31]. Models suggest that the PAF of HIV incidence due to STIs is likely to decline in mature generalized HIV epidemics [32]. With the exception of the Tanzanian trial in which HIV prevalence was 4%, all the other community randomized trials of STI control were in mature generalized HIV epidemic settings where the PAF of incident HIV due to STIs is likely to have been low. In summary, the study settings may have been inappropriate for testing the STI control hypothesis.

Male circumcision for HIV prevention

There is an inverse correlation between the prevalence of male circumcision and the prevalence of HIV in sub-Saharan Africa and multiple observational studies suggested that circumcision is associated with a reduced prevalence and incidence of HIV in men [33]. However, the evidence is not entirely consistent, there was a possibility of confounding and it was necessary to conduct trials of circumcision for HIV prevention.

Three trials in South Africa, Kenya and Uganda randomized uncircumcised men to either receive immediate circumcision (the intervention) or circumcision delayed for 21–24 months (controls) [17,18••,19••]. Despite differences in background HIV incidence, eligible age groups, urban/rural settings and surgical procedures, all three trials showed a 50–60% efficacy of circumcision for HIV prevention in men. These findings led WHO/UNAIDS to recommend circumcision for HIV prevention in men, and programs are now being scaled up in several African countries. Circumcision also reduced male genital ulcer disease (GUD), HSV-2 and HPV infections [34••].

In Uganda, a parallel trial assessed the effects of circumcision of HIV-infected men on HIV transmission to their initially uninfected female partners and found no reduction in female HIV acquisition. There was a possible increased risk of female HIV infection if circumcised HIV-positive men initiated intercourse prior to complete wound healing [35]. However, male circumcision reduced genital ulcer disease (GUD), trichomonas and bacterial vaginosis in female partners [36].

Microbicide trials

Female controlled HIV-prevention methods are urgently needed and this has motivated a large number of trials to assess surfactant and buffering microbicide agents [7,8•,37,38]. One trial evaluated use of the diaphragm with a gel [10•]. No trial demonstrated statistically significant efficacy in an intention to treat analysis, some showed increased risk of HIV acquisition, and many observed vaginal irritation and microulceration. One trial of 0.5% PRO 2000/5 gel showed 30% efficacy, which was of borderline statistical significance (P = 0.06) in an intention to treat analysis [39].

Why did these trials fail to show efficacy? Most microbicide trials were conducted in high-risk populations practicing frequent intercourse and thus frequent product use, which probably contributed to the adverse vaginal effects. Compliance with microbicide use was generally poor. In most trials, microbicide use was interrupted during pregnancy because safety in pregnancy had not been established in prior animal studies and women of reproductive potential were not required to use contraception. Treatment interruptions due to pregnancy substantially reduced the power of these studies [24]. Finally, in-vitro screening of microbicides for HIV inhibition was done in the absence of seminal fluid, but studies showed that the presence of seminal plasma diluted or abolished HIV inhibition [40].

Vaccine trials

Three large scale phase 3 HIV vaccine trials have been conducted and none showed efficacy [2,3••,4]. This suggests that evidence of immunogenicity derived from phase 1 or 2 trials may not reflect actual protection from infection. However, inappropriate study populations may have also played a role. For example, the STEP trial of the Merck vaccine enrolled 1844 men and 1135 ‘high risk’ women, but only one female incident infection was observed and female efficacy could not be assessed [3••]. Thus, the trial essentially wasted substantial effort and resources on a noninformative population. Pretrial incidence data were not available and the investigators assumed that due to their self-reported sexual behaviors, these female participants were at high risk of HIV, but this assumption proved incorrect because despite their risk behaviors, the women were not exposed to HIV due to the low prevalence of HIV infection in their male partners.

Conclusion

With the exception of male circumcision, HIV-prevention trials have been largely disappointing or inconclusive. This suggests that the pretrial screening of potential interventions, particularly vaccines and microbicides is inadequate. However, in many cases, trials have also been compromised by inappropriate selection of populations and HIV-epidemic settings.