Performance of a validated risk score to predict HIV-1 acquisition among African women participating in a trial of the dapivirine vaginal ring

To the Editor

Young women living in Eastern and Southern Africa continue to be at high risk of acquiring HIV-1, highlighting the urgent need for access to effective HIV-1 prevention interventions [1]. Recent World Health Organization (WHO) guidelines recommend that daily oral pre-exposure prophylaxis (PrEP) be offered as a priority prevention intervention for populations with an HIV-1 incidence of approximately 3% per year or higher [2]. However, it is difficult to prospectively identify individuals who may be considered part of such populations, especially in generalized epidemics where risk may be heterogeneous despite a high background HIV-1 prevalence. Empiric HIV risk assessment tools have been developed for several groups, including African heterosexual HIV-1 serodiscordant couples [3] and men who have sex with men (MSM) in the United States [4, 5], in an attempt to discern those at highest risk within these populations. These tools have been utilized in PrEP demonstration projects and as part of national guidelines to help guide PrEP implementation efforts [6]. In order to attempt to identify women at higher risk of HIV-1 acquisition, we developed and validated a risk score using data from women who participated in several HIV-1 prevention trials [7]. The score was derived from women who participated in MTN 003/VOICE, a phase IIB safety and effectiveness trial of oral and vaginal tenofovir-based PrEP [8]. The VOICE risk score showed modest performance in identifying women at highest risk for HIV-1 acquisition over the course of one year and having a risk score ≥3 correlated with an HIV-1 incidence of >3% per year. The VOICE risk score may represent a valuable tool to assist with targeted PrEP delivery in countries, like South Africa and Kenya, that are in the process of scaling-up access to PrEP [9]. However, further validation of the risk score is important for understanding the tool’s performance when applied to different populations of women at risk for HIV-1 in these regions.

To further assess VOICE risk score performance, we used data from women enrolled in MTN-020/ASPIRE, a randomized, double blind, placebo-controlled, phase III trial of the safety and effectiveness of the dapivirine vaginal ring for HIV-1 prevention (Clinicaltrials.gov NCT01617096). Detailed methods and results for the trial have been published [10]. Briefly, healthy, sexually active, HIV-1 uninfected women aged 18–45 years from Malawi, South Africa, Uganda, and Zimbabwe were enrolled and were tested monthly for HIV-1 using rapid antibody tests for a minimum of one year. Participants provided written informed consent, and applicable local and national ethical and regulatory authorities approved the study protocol.

The risk score was applied to data collected from ASPIRE participants at enrollment. The VOICE risk score is calculated based on taking the sum of the point values that correspond to the following factors assessed at enrollment: age <25 years=2 points; unmarried or not living with partner=2 points; partner does not provide financial or material support=1 point; primary partner has other partners (yes or don’t know)=2 points; alcohol use in the past 3 months=1 point; having a curable sexually transmitted infection (STI) at baseline=1 point; herpes simplex virus type 2 (HSV-2) status=2 points. In ASPIRE, data were not collected on living with primary partner, partner provides financial support, or HSV-2 status; therefore, we were unable to allocate points for those factors. Participants were included in the analysis if they had complete data for all of the other baseline factors assessed in the risk score. We calculated the HIV-1 incidence over the first year after enrollment and evaluated the predictive ability of the total score by generating receiver operating characteristic (ROC) curves and calculating area under the curve (AUC). We calculated additional performance characteristics (sensitivity and specificity) using risk score cut-points that corresponded to an HIV incidence of approximately ≥3% per year. Our primary analysis included all enrolled women; however, since the intervention significantly reduced HIV-1 incidence compared to the placebo arm, we conducted a sensitivity analysis repeating risk score evaluation stratified by study arm. Additional exploratory analyses were performed restricted to women enrolled in South Africa since South Africa contributed the largest proportion of participants to the study. All analyses were conducted using Stata v14.2 (StataCorp, College Station, TX).

Among 2,629 women enrolled in ASPIRE, 2,539 had complete data at baseline for factors included in the risk score. The majority of women (58%) were not married, 39% were <25 years of age, 21% reported that their partner had other partners and 57% did not know if their partner had other partners, 12% reported any alcohol use in the past three months, 21% had a curable STI detected at baseline and 54% were from South Africa. There were 95 seroconversions during the first year of follow-up (2,566 person-years of follow-up; HIV-1 incidence=3.70%; 95% confidence interval [CI] 3.03, 4.52). HIV-1 incidence was highest in South Africa (5.40%), followed by Malawi (2.61%), Uganda (1.59%), and Zimbabwe (1.48%). The HIV-1 incidence by risk score value is presented in Figure 1. The AUC was 0.69 (95% CI 0.64, 0.74) indicating modest predictive ability. There were no infections among women with risk scores below two (7% of women), and women with a score of two had an incidence <2% per year. Sixty-five percent of women had a risk score of ≥3 and the HIV-1 incidence among women in this category was 5.21% (95% CI 4.22, 6.44) compared to 0.98% (95% CI 0.51, 1.88) among women with a risk score <3. Using a cut-off of ≥3, risk score sensitivity and specificity were 91% and 36%, respectively. Results were similar when the risk score was separately evaluated by study arm (data not shown). Exploratory analyses restricted to women enrolled in South Africa showed similar predictive performance of the overall score, with age <25 years, being unmarried, and having a curable STI at baseline being the strongest predictors of HIV-1 acquisition within the adapted score.

Figure 1. HIV incidence and 95% confidence interval by risk score* value.

*VOICE risk score includes the following factors = age <25 years=2 points; unmarried or not living with partner=2 points; partner does not provide financial or material support=1 point; primary partner has other partners (yes or don’t know)=2 points; alcohol use in the past 3 months=1 point; having a curable sexually transmitted infection at baseline=1 point; herpes simplex virus type 2 (HSV-2) status=2 points. Data on partner provides financial support and HSV-2 status were not collected in ASPIRE; therefore, no points were allocated for these factor. Data were only collected on marital status (no data on living with partner), therefore points were only allocated is participants reported being unmarried.

An adaption of the VOICE risk score had modest performance in predicting HIV acquisition among women participating in ASPIRE. Consistent with previous applications of the risk score among women from other HIV-1 prevention trials [2], having a score ≥3 in this simplified scoring tool correlated with HIV incidence >3%, highlighting the utility of the risk score in different populations of women at risk of acquiring HIV-1. Recent WHO guidelines recommend that oral PrEP should be offered as part of combination HIV-1 prevention as an additional prevention choice for individuals at substantial risk of HIV-1 infection, where “substantial risk” is defined as populations with an HIV-1 incidence >3% per year [2]. As countries in Eastern and Southern Africa continue to scale-up access to oral PrEP, risk scoring tools could be used to improve efficiency in identifying women who will benefit most from priority access to PrEP and other novel HIV-1 prevention interventions, such as the dapivirine vaginal ring [10, 11]. For example, even among the very high-risk population participating in ASPIRE, the annual HIV-1 incidence was zero among the 7% of women with the lowest risk scores, and an additional 28% with scores of two had an incidence <2% per year.

Our findings should be interpreted in the context of several limitations. First, we were unable to fully assess performance of the complete VOICE risk score, as several factors were not assessed in ASPIRE. However, using the subset of factors that were assessed, risk score performance was similar in ASPIRE (AUC=0.69) as compared to VOICE (AUC=0.69), HPTN 035 (AUC=0.70), and FEM-PrEP (AUC=0.58) [7]. Both VOICE and ASPIRE did not include large numbers of adolescents (in ASPIRE, only 19% of participants were between 18–21); therefore, the performance of the risk score in this population could be limited. There may be other factors that are important to consider in assessing HIV-1 risk among adolescents and additional work to identify such factors is paramount for efficient implementation of HIV-1 prevention interventions in this high-risk population. In addition, few women enrolled in VOICE or ASPIRE reported transactional sex or use of injection drugs. Women who engage in transactional sex or who inject drugs are already considered to be at high risk, thus use of the VOICE risk score in these populations would likely not provide any additional benefit.

An adaptation of the VOICE risk score predicted HIV-1 acquisition over one year among women participating in ASPIRE. Our findings demonstrate the robustness of our risk scoring tool and the consistency of external validation results. The VOICE risk score should further be considered as a tool to inform scale-up of HIV-1 prevention strategies for women living in Eastern and Southern Africa to identify those with an anticipated HIV-1 incidence of >3%, which represents a priority group for access to PrEP and other HIV-1 prevention interventions.