Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: AIDS Behav. 2020 Jan;24(1):274–283. doi: 10.1007/s10461-019-02616-3

Development and Validation of the Personalized Sexual Health Promotion (SexPro) HIV Risk Prediction Model for Men Who Have Sex with Men in the United States

Hyman Scott 1, Eric Vittinghoff 2, Risha Irvin 3, Albert Liu 4, LaRon Nelson 5, Carlos Del Rio 6, Manya Magnus 7, Sharon Mannheimer 8, Sheldon Fields 9, Hong Van Tieu 10, Irene Kuo 11, Steve Shoptaw 12, Beatriz Grinsztejn 13, Jorge Sanchez 14, Steven Wakefield 15, Jonathan D Fuchs 16, Darrell Wheeler 17, Kenneth H Mayer 18, Beryl A Koblin 19, Susan Buchbinder 20
PMCID: PMC6954959  NIHMSID: NIHMS1535903  PMID: 31352633

Abstract

Accurate HIV risk assessment among men who have sex with men (MSM) is important to help providers assess risk, and target HIV prevention interventions. We sought to develop an evidence-based HIV risk assessment tool for US MSM that is inclusive of Black MSM. Data from four large longitudinal cohorts of MSM were used to develop (EXPLORE), and validate (VAX004, HPTN061, and HVTN505). These data included visits in which participants self-reported HIV risk behavior and underwent HIV testing. We developed a pooled logistic model for incident HIV infection based on self-reported risk behaviors during the 6 months before each study visit. A total of 4,069 MSM were used for the development cohort, and 8,047 MSM in the three validation cohorts through 2013. The final model includes age (<35, ≥35), Black race and Latino ethnicity, numbers of HIV-negative anal sex partners; number of insertive or receptive anal intercourse episodes; having 1 HIV-negative partner only; self-reported substance use; and bacterial sexually transmitted infection diagnosis. The model showed good discrimination in internal validation (C-statistic=79.5). The external validation cohorts also showed good discrimination, with C-statistics of 73.1, 71.0, 71.9 in VAX004, HPTN061, and HVTN505 respectively, and acceptable calibration. We developed and validated an HIV risk assessment tool for MSM, which showed good predictive ability, including among the largest cohort of HIV-uninfected Black MSM in the US. This tool is available online (mysexpro.org) and can be used by providers to support targeting of HIV prevention interventions such as pre-exposure prophylaxis for MSM.

Keywords: HIV Risk Assessment, MSM, HIV Prevention

Introduction

Men who have sex with men (MSM) are disproportionally impacted by HIV in the United States (US), accounting for two thirds of new HIV diagnoses in US, while only making up approximately 3% of the US population (1, 2). There are important racial/ethnic disparities among MSM, with increased HIV prevalence and incidence among Black and Latino MSM despite similar or lower reported HIV risk behavior (38). Highlighting these disparities, the Centers for Disease Control and Prevention (CDC) recently estimated the lifetime risk of HIV infection to be 1 in 6 among MSM overall, but 1 in 2 and 1 in 4 for Black MSM and Latino MSM, respectively (9).

Despite this elevated lifetime HIV risk, self-perceived HIV risk is relatively low among MSM. In a CDC web-based HIV surveillance study, approximately one third of MSM who had never been tested for HIV reported low perceived HIV risk as the reason for not testing (10). However, these men often reported high levels of HIV risk behavior: 56% reported more than two male anal sex partners in the past 12 months, and 37% reported condomless anal intercourse. Low perceived HIV risk is even more pronounced among young MSM. In the Young Men’s Study, nearly 75% of participants in this study perceived themselves at low lifetime risk for HIV, despite 32% reporting four or more partners in the prior six months, and 21% having been diagnosed with a recent sexually transmitted infection (STI) (11).

Prior HIV risk assessment tools have been developed for clinicians, without a focus on improving self-assessment of HIV risk among MSM. Menza and colleagues developed an HIV risk assessment tool from a clinical cohort of 1,900 MSM who were repeat HIV testers in Seattle, Washington (12). Based on the answers to four questions about sexual risk behavior, popper or methamphetamine use, and previous STI diagnosis, patients were stratified into one of four categories of HIV seroconversion risk from low (<5%) to very high (>14%). Another tool, the HIV Incidence Risk Index for MSM (HIRI-MSM), was developed using two longitudinal cohorts of MSM in the US by the CDC, and is intended to support targeting of HIV prevention interventions for MSM (13). Finally, the San Diego Early Test (SDET) score was developed in a clinical cohort of acute and early HIV infected patients in San Diego (14). This tool was developed with a focus on MSM who may be at risk for early or acute HIV infection.

These HIV risk tools, primary designed for use by clinicians, have similar ability to identify MSM at risk for HIV using a short behavioral questionnaire. However, they were developed in clinical or longitudinal cohorts of mostly White MSM in the US, and have not been validated in more contemporary cohorts, or cohorts that include large numbers of Black MSM, the group most disproportionally impacted by HIV in the US. A recent study found that these risk tools were poor at predicting HIV seroconversion among a cohort of young Black MSM (15).

Given the limitations of the currently available HIV risk tools, the goal of this project was to develop an evidence-based HIV risk assessment tool for use by providers and individuals at risk, and validate it in a large, contemporary, and diverse cohort of MSM to assess applicability to Black MSM. This Sexual Health Promotion (SexPro) tool could assist providers in appropriately targeting HIV prevention options with their patients, and by MSM to evaluate their HIV risk and support making more informed decisions regarding HIV risk reduction options. Finally, development onto a web-based platform would allow providers and patients to complete the HIV risk assessment on a user-friendly interface in both clinical and non-clinical settings.

Methods

EXPLORE was used to develop the SexPro model, and data from VAX004 was used for initial external validation (16, 17). Two more recent, large, longitudinal HIV prevention trials – HIV Vaccines Trials Network (HVTN) 505 and HIV Prevention Trials Network (HPTN) 061 – were used for additional validation (1820).

Development Cohort

EXPLORE enrolled 4,295 MSM in six US cities from 1999–2001 into a randomized clinical trial comparing an intensive ten-session behavioral intervention to standard risk reduction counseling (16). Participants were followed every six months until study completion in 2003. EXPLORE enrolled HIV-negative men who reported anal intercourse with at least one male partner in the past 12 months, excluding those who had been in mutually monogamous relationships for two or more years with a male partner known to be HIV negative. Because the study did not demonstrate efficacy against HIV acquisition in the behavioral intervention arm, all participants were included in the development of the SexPro model.

Validation Cohorts

Initial external validation of the model was conducted using data from the VAX004, a longitudinal HIV prevention cohort study of MSM in the US. The VAX004 trial enrolled 4,643 MSM in 57 sites in the U.S., Canada, and the Netherlands (only participants from US sites were used for this analysis) from 1998–1999 in a randomized placebo-controlled efficacy trial of a recombinant gp120 HIV-1 vaccine (17). Participants were followed at six-month intervals for 36 months and the study was completed in 2002. VAX004 enrolled HIV-negative men who reported anal intercourse with at least one male partner in the past 12 months, and excluded men who had been in mutually monogamous relationships for the previous year or reported injection drug use (IDU) at baseline. This study failed to show efficacy against HIV acquisition in the vaccine arm, so all participants were used in this analysis.

The HIV Prevention Trials Network (HPTN) 061 study was a multi-site cohort study designed to determine the feasibility and acceptability of a multi-component (behavioral and structural) intervention for Black MSM (19, 20). The study was conducted in six US cities from 2009–2013, and enrolled 1,553 Black MSM, including 1,164 men initially HIV-negative men who were eligible for follow-up for this analysis. Study participants were followed up at 6 months and one year.

Finally, HIV Vaccine Trials Network (HVTN) 505 enrolled 2,504 MSM and transgender women in 18 US cities from 2009–2013 in a randomized placebo-controlled trial of a DNA/rAd5 vaccine regimen (18). Participants were followed at one- to three-month intervals for 24 months. The vaccinations were stopped in 2013 by the Data Safety Monitoring Board (DSMB) for lack of efficacy. Accordingly, this analysis used data collected by the time of the DSMB meeting for all placebo and vaccine participants.

Demographics

Participants in all study cohorts were asked to self-report race and ethnicity at baseline. Participants could report both race and Latino ethnicity in all cohorts except VAX004. Since age categories were available in EXPLORE, exact ages were categorized to ≤ 35 or >35 years for the validation cohorts (VAX004, HPTN 061, and HVTN 505).

HIV risk behavior

Self-reported sexual risk behavioral data were collected via audio-computer-assisted self interview (EXPLORE, HPTN 061, and HVTN 505) or structured counselor-administered interview (VAX004), and included sexual behaviors in the past six (EXPLORE, HPTN 061, VAX004) or three months (HVTN 505), including receptive and insertive anal intercourse episodes without a condom (ncRAI and ncIAI), as well as receptive anal episodes with a condom (cRAI), all by perceived partner HIV-serostatus. Other self-reported behavioral risk factors collected in all cohorts included use of poppers and methamphetamine, heavy alcohol use (≥ 4 drinks daily, or ≥6 drinks on any occasion), and bacterial STIs such as gonorrhea, chlamydia, and syphilis.

Initial model development

We developed a pooled logistic model for incident HIV infection at each follow-up EXPLORE visit based on self-reported HIV risk behaviors during the preceding 6 months. Each participant contributed multiple observations, one for each semi-annual visit up until the time of HIV infection or the end of study follow-up. In addition to sexual risk behaviors, we considered included age, race/ethnicity, any use of methamphetamines and/or poppers, heavy alcohol use, and indicators for STIs. To allow for non-linearity, numbers of sexual risk episodes and numbers of sexual partners were modeled allowing different effects above and below counts of 10. Candidate models were compared in terms of the C-statistic, or area under a ROC curve, a measure of discrimination. To help avoid selecting an over-fitted model, the C-statistic was estimated using 10-fold cross-validation. Models with 6–12 different predictors were considered. The selected model had a cross-validated C-statistic near the maximum and satisfactory goodness of fit.

Adaptation of the prediction model for online self-assessment

We converted risk predictions based on the final SexPro model to a 1–20 scale, with 20 denoting low HIV infection risk. Positive (or gain) frame messaging has been used extensively to promote health behaviors, so was used in the design of the SexPro score (21, 22). This was motivated to forestall misinterpretation of small risk probability estimates, and to focus attention instead on changes in the scores resulting from actual or potential changes in risk behavior. Because age and race/ethnicity were important independent risk factors, young men, particularly younger Black and Latino men, were classified as having relatively high HIV risk even if they reported no sexual or drug use risk behaviors. Accordingly, we reset their scores to the highest score of 20 (lowest HIV risk) in the absence of any reported sexual risk behavior. For the same reason, drug use was not allowed to lower the SexPro score below 20 among MSM reporting no sexual risk behaviors. The model has since been implemented on a website (mysexpro.org) to provide access to MSM for self-assessment and to providers to support targeting of HIV prevention interventions.

External validation

The model coefficients estimated in EXPLORE for the final SexPro model were used to calculate predicted risks for each participant in the validation cohorts. These risks were then used to calculate the C-statistic. To assess calibration, average predicted 6-month risks were compared to observed 6-month HIV infection rates, stratified by categories of the SexPro score. The HIRI-MSM risk tool was evaluated in the three validation cohorts.

Results

In EXPLORE, 22,577 visits and 217 HIV infections during study follow-up were used to develop the SexPro model (Table 1). For external validation, a total of 433 HIV infections were observed over 25,427, 1,692, and 10,652 visits from VAX004, HPTN 061, and HVTN 505, respectively (Table 1). Participants under 35 years were the majority in EXPLORE and HVTN 505, but made up approximately half of VAX004 and HPTN 061 cohorts. All participants in HPTN 061 were Black, while the majority of participants in the other cohorts were White. A higher proportion of participants in VAX004 (43.8%) reported having an HIV-positive partner compared with EXPLORE (28.4%), HPTN 061 (14.6%), or HVTN 505 (18.9%).

Table I:

Baseline demographic and HIV Risk Characteristics of Participants in EXPLORE, VAX004, HPTN 061, and HVTN 505 Cohort Studies.

Variable EXPLORE N=4069 VAX004 N=4878 HPTN 061 N=973 HVTN 505 N=2196
(%) (%) (%) (%)
Demographics
Age ≤ 35 years 60.9 48.8 44.8 68.3
Black 7.4 3.4 100 18.3
Latino ethnicity 14.8 0.7 7.7 8.5
Risk Behavior and Characteristics
Any HIV positive anal sex partners* 28.4 43.8 14.6 18.9
Number of HIV negative anal sex partners*
0 34.6 43.2 31.6 30.9
1 18.2 16.3 21.0 26.1
2–5 31.7 29.3 34.0 35.7
6–9 5.7 4.5 7.3 3.9
≥10 9.8 6.7 6.2 3.4
Number of ncRAI episodes with HIV positive or unknown status partners*
0 72.2 79.2 67.6 75.6
1 9.8 7.2 9.2 9.6
2–5 11.0 7.8 12.9 10.3
6–9 2.8 1.6 2.4 1.6
≥10 4.2 4.1 8.0 2.9
Number of cRAI episodes with HIV positive or unknown status partners*
0 57.9 55.9 76.4 70.6
1 9.9 10.2 7.5 10.4
2–5 18.5 17.6 9.8 13.0
6–9 5.6 4.6 2.5 3.0
≥10 8.2 11.9 3.8 2.9
Number of ncIAI episodes with HIV positive or unknown status partners*
0 63.5 68.1 45.8 66.2
1 11.4 8.9 14.8 10.5
2–5 15.2 11.5 18.4 15.5
6–9 3.8 2.7 5.8 3.2
≥10 6.2 8.7 15.2 4.6
Heavy alcohol use * 10.2 10.7 40.4 15.2
Methamphetamine use* 12.8 9.1 9.3 5.7
Popper use* 36.7 32.8 10.4 24.7
STI* 6.5 9.7 4.8 4.7
Outcomes and events
Participant visits (n) 22,577 25,427 1,692 10,652
HIV infections (n) 217 343 25 65

HPTN – HIV Prevention Trials Network

HVTN – HIV Vaccine Trials Network

ncRAI – Receptive anal intercourse without a condom

cRAI – Receptive anal intercourse with a condom

ncIAI – Insertive anal intercourse without a condom

Heavy alcohol use was defined as ≥ 4 drinks daily, or ≥6 drinks on any occasion STI was defined as gonorrhea, syphilis, or chlamydia diagnosis by self-report

*

Self-report in the prior 6 months

Final SexPro model

The final SexPro model includes the following variables: age (<35, ≥35), Black race and Latino ethnicity, the number of ncRAI contacts with HIV-positive partners or unknown status partners (with a leveling of risk after 10 partners), numbers of ncIAI and cRAI episodes with HIV-positive or unknown status partners, as well as numbers of HIV-negative anal sex partners, and having 1 HIV-negative partner only (Table 2). Other HIV risk variables included heavy alcohol use, use of methamphetamine and poppers, and self-report of gonorrhea, syphilis, chlamydia. Coefficient estimates for the final SexPro model variables are listed in Table 2.

Table II:

Coefficients Used in the Final SexPro Multivariate Pooled Logistic Model.*

Variable Coefficient estimate
Constant −5.799
Age <=35 0.2703
Black 0.8749
Latino 0.4284
Number of ncRAI episodes with HIV positive or unknown status partners
 Per episode, up to 10 episodes 0.2109
 Per episode, after 10 episodes 0.0024
Number of cRAI episodes with HIV positive or unknown status partners 0.0322
Number of ncIAI episodes with HIV positive or unknown status partners 0.0053
Number of HIV negative anal sex partners 0.0985
1 HIV negative anal sex partner only −1.0955
Heavy alcohol use 0.6450
Methamphetamine use 0.7663
Popper use 0.1575
Gonorrhea, syphilis, or chlamydia diagnosis** 0.3570

ncRAI – Receptive anal intercourse without a condom

cRAI – Receptive anal intercourse with a condom

ncIAI – Insertive anal intercourse without a condom

Heavy alcohol use was defined as ≥ 4 drinks daily, or ≥6 drinks on any occasion

*

The predicted outcome is HIV infection diagnosed at a follow-up visit.

**

Self-report

The SexPro model showed good discrimination in the EXPLORE internal validation (C-statistic=79.5). The external validation cohorts showed good discrimination, with C-statistics of 73.1, 71.0, 71.9 in VAX004, HPTN 061, and HVTN 505, respectively. Figure 1 shows the predicted and observed HIV infection risk in the development and validation cohorts, suggesting acceptable calibration in EXPLORE, HPTN 061, and HVTN 505, but under-predicted risk in VAX004. The area under the receiver operator curve (AUC) was greater than 0.70 for both internal and external validation cohorts (Figure 2). The HIRI-MSM score showed similar discrimination in VAX004, HPTN 061 and HVTN 505, with C-statistics of 74.0, 75.1 and 73.5 respectively.

Figure 1:

Figure 1:

Predicted and Observed HIV Infection Risk in EXPLORE, VAX004, HIV Prevention Trials Network 061, and HIV Vaccine Trials Network 505 Cohort Studies.*

* C-statistics for internal validation in the development cohort EXPLORE=79.5; and external validation cohorts VAX004=73.1, HPTN 061=71.0, and HVTN 505=71.9. The Sex Pro Score risk strata of 16–20 for HPTN 061 is blank because all participants in this cohort had predicted and observed Sex Pro scores of <16.

Figure 2:

Figure 2:

Area under Receiver Operator Curve (AUC) for Explore, VAX004, HIV Prevention Trials Network 061, and HIV Vaccine Trials Network 505 Cohort Studies.*

* AUC for internal validation in the development cohort EXPLORE=0.80; and external validation cohorts VAX004=0.73, HPTN 061=0.71, and HVTN 505=0.72.

Table 3 shows sensitivities and specificities of SexPro scores dichotomized at each possible value from 1 to 20, by cohort. Based on these results, a SexPro score cut off of 16 could be used to identify MSM at increased risk for HIV infection, and maximize sensitivity of the score for Black MSM. SexPro scores of 16 or lower had sample sensitivities for HIV acquisition of 81.1%, 64.4%, 100%, and 75.4%, in EXPLORE, VAX004, HPTN 061, and HVTN 505, respectively, with corresponding specificities of 59.6%, 67.4%, 0%, and 51.8%. The sensitivity and specificity of a HIRI-MSM score of ≥10 was calculated in EXPLORE (sensitivity 86.6%, specificity 35.7%), VAX004 (sensitivity 82.2%, specificity 45.8%), HPTN 061 (sensitivity 80%, specificity 61.2%), and HVTN 505 (sensitivity 86.2%, specificity 43.2%).

Table III:

Sensitivity and Specificity of SexPro Scores In EXPLORE, VAX004, HIV Prevention Trials Network 061, and HIV Vaccine Trials Network 505 Cohort Studies.*

EXPLORE VAX004 HPTN 061 HVTN 505
Score Cutoff** Sens. Spec. Sens. Spec. Sens. Spec. Sens. Spec.
1 34.6 94.9 19.2 95.9 52.0 78.5 26.2 95.1
2 34.6 94.5 20.4 95.5 56.0 77.1 26.2 94.6
3 36.4 94.0 21.3 95.2 60.0 75.3 27.7 94.1
4 39.2 93.5 22.4 94.9 64.0 72.5 29.2 93.6
5 39.2 92.9 23.6 94.5 68.0 69.9 29.2 92.7
6 40.1 92.3 24.2 94.1 72.0 66.1 30.8 91.8
7 41.9 91.6 26.5 93.8 76.0 63.6 30.8 90.9
8 45.6 90.6 28.6 93.1 76.0 60.6 30.8 89.9
9 49.8 89.1 31.5 92.2 80.0 55.9 36.9 88.3
10 52.1 87.6 33.5 91.3 80.0 51.3 36.9 86.6
11 56.2 85.6 35.9 89.9 80.0 46.3 43.1 84.6
12 60.8 82.5 39.9 87.7 80.0 41.0 47.7 80.4
13 63.1 79.6 44.6 84.5 80.0 34.3 53.8 74.9
14 70.0 75.0 49.0 80.6 92.0 22.5 58.5 69.1
15 74.2 69.5 55.1 75.3 92.0 16.1 64.6 61.9
16 81.1 59.6 64.4 67.4 100 0 75.4 51.8
17 92.2 45.9 80.5 51.5 100 0 90.8 38.6
18 96.3 25.7 97.4 190 100 0 100 8.8
19 99.1 15.3 100 0 100 0 100 0
20 100 0 100 0 100 0 100 0

HPTN – HIV Prevention Trials Network

HVTN – HIV Vaccine Trials Network

*

The predicted outcome is HIV infection diagnosed at a follow-up visit.

**

Sex Pro scores range from 1–20, with 20 representing the lowest HIV risk.

Discussion

The SexPro HIV self-assessment risk tool was developed and validated using four longitudinal cohort studies of MSM in the US, including HPTN 061, the largest cohort study of Black MSM. Developed to support accurate HIV risk self-assessment for MSM in clinical and non-clinical settings, SexPro showed good sensitivity in predicting HIV risk similar to the HIRI-MSM, based on a small number of easily assessed demographic and behavioral variables. SexPro can help MSM get a more accurate assessment of their risk for HIV, and also assist providers in targeting HIV prevention services, in particular PrEP. The SexPro self-assessment tool is available at mysexpro.org.

The SexPro model reflects established understanding of the demographic and behavioral risk factors for HIV acquisition, in particular ncRAI, numbers of partners, heavy alcohol and drug use, and STI, and well as youth and race/ethnicity (2326). The substantial inverse weight put by the model on having a single monogamous partner is consistent with CDC clinical practice guidelines against PrEP use by MSM in mutually monogamous relationships with a recently tested HIV-negative man (27). Similar to other HIV risk prediction scores, substance use was an important predictor of HIV risk in the SexPro model. Methamphetamines were associated with the largest increase in HIV risk, consistent with prior literature (23, 25). In the model developed by Menza from the clinical cohort in Seattle, Washington, methamphetamine or popper use contributed more than twice as many score points (11 points) as the next highest risk factor, reporting an STI (4 points) (12). Methamphetamine use is a well described driver of HIV infection among US MSM, although there is regional variability in use (23, 25). In contrast to the Menza finding, the HIRI-MSM, developed in a cohort in San Diego, did not find non-injection stimulant use (inclusive of methamphetamines) to be an independent risk factor for early HIV infection in their multivariable model (14).

None of the three earlier HIV risk assessment tools for MSM – Menza et al., SDET or HIRI –included race or ethnicity as variables in their scoring algorithms since race and ethnicity were not perceived as modifiable and therefore could not be targeted for risk reduction (1214). However, given the important racial/ethnic disparities in HIV risk in the US, often despite similar or lower levels of individual risk behavior, we feel it is important to include these demographic variables in the SexPro risk prediction model (28). First, a risk score that does not include race and ethnicity will underestimate the HIV risk of Black and Latino MSM, and likely lead to reduced targeting of efficacious HIV prevention options such as PrEP. This was shown by Lancki et al who found that currently available risk scores have low sensitivity or AUC (HIRI-MSM) among a cohort of young Black MSM with an HIV incidence of 8.5/100 person years (15). Without accurate risk assessment, these men will be less likely to meet risk criteria for these interventions, exacerbating the already widening HIV disparities (28). Second, race is a social construct, and the increased risk of HIV infection among Black and Latino MSM is driven by social (i.e., higher HIV prevalence among sexual networks) and structural (i.e., incarceration policies, poverty, etc.) factors disproportionally experienced by these men (29, 30). So, the social and structural factors that are driving the disparities are potentially “modifiable” at the social or structural level (i.e., incarceration policy changes) and race/ethnicity serves as proxy for these variables (29).

Furthermore, the latest, updated version of the SexPro tool also incorporates U=U, supporting that HIV-positive MSM who are virally-suppressed have negligible risk of transmitting HIV to their sexual partners. Data from several large cohorts covering thousands of condomless anal sex acts among MSM, have not demonstrated any linked HIV transmissions among their HIV-negative partners (3133). This message has been supported by many policy, research, and public health organizations including the CDC (34). While the required data on HIV-positive partner monogamy, treatment, and viral suppression were unfortunately unavailable in the four cohorts used in this report, we have since incorporated the concept of U=U into the SexPro score by adjusting the risk of a virally suppressed HIV-positive partner to the same as a HIV-negative partner. Currently, no other risk tools incorporate the compelling data on the impact of viral suppression of HIV-positive partners on HIV transmission risk.

The SexPro model showed good model fit in all cohorts, but did have a lower specificity at the selected cut off of ≤16 in HPTN 061 compared with the other cohorts, in part reflecting systematically lower scores for Black MSM. The cut-off of 16 was chosen to emphasize sensitivity, as appropriate for a screening tool used for targeting of HIV prevention interventions (13). For example, using HIRI-MSM with a recommended score cut-off ≥10 had a lower sensitivity in HPTN 061 compared with the other validation cohorts. Our decision to recommend the same cut-off for Black MSM, despite evidence for reduced specificity in this group, was motivated by the disproportionate impact of HIV in this group. SexPro is being used as a screening tool in the clinical trial of injectable cabotegravir as PrEP (HPTN 083) which will provide implementation data in clinical settings (35).

There are important limitations to consider for this study. The initial cohort included MSM from the early combination antiretroviral (cART) era. Since these studies have been completed, there have been advances in HIV treatment recommendations and increases in viral suppression among people living with HIV. However, our prior work demonstrated similar per-contact risk for HIV infection among MSM in the pre- and early cART era (26). The SexPro model was also validated with good model fit in two contemporary longitudinal cohorts of MSM in the US from 2009–2013. Another limitation for the initial development of the SexPro score was the limited racial/ethnic diversity of the EXPLORE study. The EXPLORE study sample was only 7% Black and 15% Latino, and VAX004 had even lower proportions of these men. In an effort to ensure applicability to a diverse sample of men, we chose additional validation cohorts with more diverse study participants; it was reassuring that SexPro model fit well in these cohorts. Finally, while the SexPro model requires only a few demographic and sexual risk behavior questions, the calculations are nonetheless complicated, requiring its implementation as a website rather than as a simple checklist with additive score.

Conclusions

The SexPro tool represents a novel HIV risk self-assessment tool for MSM to assist with targeting HIV prevention strategies. SexPro scores ≤16 predicted incident HIV infections with good sensitivity and acceptable specificity in large contemporary, diverse cohorts of MSM. This tool can be used by MSM to improve self-assessment of HIV risk, and by providers in targeting and improving uptake of HIV prevention strategies such as PrEP.

Acknowledgment

The HIV Prevention Trials Network (HPTN) is funded by the National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Child Health and Human Development (NICH/HD), National Institute of Drug Abuse (NIDA) and the National Institute of Mental Health (NIMH), Office of AIDS Research, National Institutes of Health (NIH), Department of Health and Human Services. This work was also supported by the National Institutes of Health [K23MH104116 to H.M.S]. The authors would like to thank the study participants; and HPTN 061 Protocol Co-Chairs: Beryl Koblin, PhD, Kenneth Mayer, MD, and Darrell Wheeler, PhD, MPH; Emory University (Ponce de Leon Center & Hope Clinic Clinical Research Sites): Carlos del Rio, Paula Frew, Christin Root, Jermel L. Wallace; Fenway Institute at Fenway Health: Kenneth Mayer, Benjamin Perkins, Kelvin Powell, Benny Vega; George Washington University School of Public Health and Health Services: Manya Magnus, Alan Greenberg, Jeanne Jordan, Irene Kuo, Gregory Phillips II, Christopher Watson; Harlem Prevention Center: Sharon Mannheimer, Avelino Loquere Jr.; New York Blood Center: Beryl Koblin, Krista Goodman, Hong Van Tieu; San Francisco Department of Public Health: Susan P. Buchbinder, Michael Arnold, Chadwick Campbell, Mathew Sanchez; University of California Los Angeles (UCLA): Steven J. Shoptaw, Christopher Hucks-Ortiz; HPTN Coordinating and Operations Center (CORE), FHI 360; Sam Griffith, Erica Hamilton, LaShawn Jones, Georgette King, Jonathan Paul Lucas, Teresa Nelson; HPTN Network Laboratory, Johns Hopkins Medical Institute: Sue Eshleman, Vanessa Cummings; HPTN Statistical and Data Management Center, Statistical Center for HIV/AIDS Research and Prevention (SCHARP): Lei Wang, Corey Kelly, Ting-Yuan Liu; Division of AIDS (DAIDS) at the U.S. National Institutes of Health (NIH): Jane Bupp, Vanessa Elharrar; Additional HPTN 061 Protocol Team Members: Darrell Wheeler (co-chair), Sheldon Fields, Kaijson Noilmar, Steven Wakefield; Other HPTN 061 Contributors: Black Gay Research Group, HPTN Black Caucus, Kate MacQueen, Leo Wilton. We thank the HVTN 505 study participants and protocol team, the NIH National Institute of Allergy and Infectious Diseases (NIAID), and the NIAID-funded HIV Vaccine Trials Network for providing the clinical trial data from HVTN 505.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

Contributor Information

Hyman Scott, Bridge HIV, San Francisco Department of Public Health, San Francisco, California, USA.

Eric Vittinghoff, University of California, San Francisco, San Francisco, California, USA.

Risha Irvin, Johns Hopkins School of Medicine, Baltimore, Maryland, USA.

Albert Liu, Bridge HIV, San Francisco Department of Public Health, San Francisco, California, USA.

LaRon Nelson, University of Rochester, Rochester, New York, USA.

Carlos Del Rio, Emory University, Atlanta, Georgia, USA.

Manya Magnus, George Washington University, District of Columbia, USA.

Sharon Mannheimer, Columbia University, New York, New York, USA.

Sheldon Fields, New York Institute of Technology, Old Westbury, New York, USA.

Hong Van Tieu, New York Blood Center, New York, New York, USA.

Irene Kuo, George Washington University, District of Columbia, USA.

Steve Shoptaw, University of California, Los Angeles, Los Angeles California, USA.

Beatriz Grinsztejn, Instituto de Pesquisa Clinica Evandro Chagas (IPEC), Rio de Janeiro, Brazil.

Jorge Sanchez, Asociacion Civil Impacta Salud y Educacion, Lima, Peru.

Steven Wakefield, Fred Hutchinson Cancer Research Center, Seattle, Washington, USA.

Jonathan D. Fuchs, Center for Learning Innovation, San Francisco Department of Public Health, San Francisco, California, USA.

Darrell Wheeler, University at Albany - SUNY, Albany, New York, USA.

Kenneth H. Mayer, Fenway Health, Boston, Massachusetts, USA.

Beryl A. Koblin, New York Blood Center, New York, New York, USA.

Susan Buchbinder, Bridge HIV, San Francisco Department of Public Health, San Francisco, California, USA.

References

  • 1.Lieb S, Fallon SJ, Friedman SR, Thompson DR, Gates GJ, Liberti TM, et al. Statewide estimation of racial/ethnic populations of men who have sex with men in the U.S. Public Health Rep. 2011;126(1):60–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Centers for Disease Control and Prevention. HIV Surveillance Report, 2014; vol. 26. http://www.cdc.gov/hiv/library/reports/surveillance/. Published November 2015 Accessed November 30, 2015.
  • 3.Millett GA, Flores SA, Peterson JL, Bakeman R. Explaining disparities in HIV infection among black and white men who have sex with men: a meta-analysis of HIV risk behaviors. AIDS. 2007;21(15):2083–91. [DOI] [PubMed] [Google Scholar]
  • 4.Mimiaga MJ, Reisner SL, Cranston K, Isenberg D, Bright D, Daffin G, et al. Sexual mixing patterns and partner characteristics of black MSM in Massachusetts at increased risk for HIV infection and transmission. J Urban Health. 2009;86(4):602–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Maulsby C, Millett G, Lindsey K, Kelley R, Johnson K, Montoya D, et al. HIV among Black men who have sex with men (MSM) in the United States: a review of the literature. AIDS Behav. 2014; 18(1):10–25. [DOI] [PubMed] [Google Scholar]
  • 6.Sullivan PS, Peterson J, Rosenberg ES, Kelley CF, Cooper H, Vaughan A, et al. Understanding racial HIV/STI disparities in black and white men who have sex with men: a multilevel approach. PLoS One. 2014;9(3):e90514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Irvin R, Vallabhaneni S, Scott H, Williams JK, Wilton L, Li X, et al. Examining levels of risk behaviors among black men who have sex with Men (MSM) and the association with HIV acquisition. PLoS One. 2015;10(2):e0118281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Sullivan PS, Rosenberg ES, Sanchez TH, Kelley CF, Luisi N, Cooper HL, et al. Explaining racial disparities in HIV incidence in black and white men who have sex with men in Atlanta, GA: a prospective observational cohort study. Ann Epidemiol. 2015;25(6):445–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Hess K, Hu X, Lanksy A, Mermin J, Hall HI. Estimating the Lifetime Risk of a Diagnosis of HIV Infection in the United States. CROI 2016. Boston, MA. Abstract#52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Mackellar DA, Hou SI, Whalen CC, Samuelsen K, Sanchez T, Smith A, et al. Reasons for not HIV testing, testing intentions, and potential use of an over-the-counter rapid HIV test in an internet sample of men who have sex with men who have never tested for HIV. Sexually transmitted diseases. 2011;38(5):419–28. [DOI] [PubMed] [Google Scholar]
  • 11.MacKellar DA, Valleroy LA, Secura GM, Behel S, Bingham T, Celentano DD, et al. Perceptions of lifetime risk and actual risk for acquiring HIV among young men who have sex with men. AIDS Behav. 2007;11(2):263–70. [DOI] [PubMed] [Google Scholar]
  • 12.Menza TW, Hughes JP, Celum CL, Golden MR. Prediction of HIV acquisition among men who have sex with men. Sexually transmitted diseases. 2009;36(9):547–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Smith DK, Pals SL, Herbst JH, Shinde S, Carey JW. Development of a clinical screening index predictive of incident HIV infection among men who have sex with men in the United States. J Acquir Immune Defic Syndr. 2012;60(4):421–7. [DOI] [PubMed] [Google Scholar]
  • 14.Hoenigl M, Green N, Mehta SR, Little SJ. Risk Factors for Acute and Early HIV Infection Among Men Who Have Sex With Men (MSM) in San Diego, 2008 to 2014: A Cohort Study. Medicine (Baltimore). 2015;94(30):e1242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Lancki N, Almirol E, Alon L, McNulty M, Schneider JA. Preexposure prophylaxis guidelines have low sensitivity for identifying seroconverters in a sample of young Black MSM in Chicago. AIDS. 2018;32(3):383–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Koblin B, Chesney M, Coates T, Team ES. Effects of a behavioural intervention to reduce acquisition of HIV infection among men who have sex with men: the EXPLORE randomised controlled study. Lancet. 2004;364(9428):41–50. [DOI] [PubMed] [Google Scholar]
  • 17.Bartholow BN, Goli V, Ackers M, McLellan E, Gurwith M, Durham M, et al. Demographic and behavioral contextual risk groups among men who have sex with men participating in a phase 3 HIV vaccine efficacy trial: implications for HIV prevention and behavioral/biomedical intervention trials. J Acquir Immune Defic Syndr. 2006;43(5):594–602. [DOI] [PubMed] [Google Scholar]
  • 18.Hammer SM, Sobieszczyk ME, Janes H, Karuna ST, Mulligan MJ, Grove D, et al. Efficacy trial of a DNA/rAd5 HIV-1 preventive vaccine. N Engl J Med. 2013;369(22):2083–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Koblin B, Mayer K, Eshleman S, Wang L, Mannheimer S, et al. Correlates of HIV Acquisition in a Cohort of Black MenWho Have Sex with Men in the United States: HIVPrevention Trials Network (HPTN) 061. PLoS ONE. 2013;8(7):e70413. doi: 10.1371/journal.pone.0070413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Mayer KH, Wang L, Koblin B, Mannheimer S, Magnus M, del Rio C, et al. Concomitant socioeconomic, behavioral, and biological factors associated with the disproportionate HIV infection burden among Black men who have sex with men in 6 U.S. cities. PLoS One. 2014;9(1):e87298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Rothman AJ. Toward a theory-based analysis of behavioral maintenance. Health Psychol. 2000;19(1 Suppl):64–9. [DOI] [PubMed] [Google Scholar]
  • 22.Gallagher KM, Updegraff JA. Health message framing effects on attitudes, intentions, and behavior: a meta-analytic review. Ann Behav Med. 2012;43(1):101–16. [DOI] [PubMed] [Google Scholar]
  • 23.Buchbinder SP, Vittinghoff E, Heagerty PJ, Celum CL, Seage GR, Judson FN, et al. Sexual risk, nitrite inhalant use, and lack of circumcision associated with HIV seroconversion in men who have sex with men in the United States. J Acquir Immune Defic Syndr. 2005;39(1):82–9. [DOI] [PubMed] [Google Scholar]
  • 24.Woody GE, Donnell D, Seage GR, Metzger D, Marmor M, Koblin BA, et al. Non-injection substance use correlates with risky sex among men having sex with men: data from HIVNET. Drug Alcohol Depend. 1999;53(3):197–205. [DOI] [PubMed] [Google Scholar]
  • 25.Koblin BA, Husnik MJ, Colfax G, Huang Y, Madison M, Mayer K, et al. Risk factors for HIV infection among men who have sex with men. AIDS. 2006;20(5):731–9. [DOI] [PubMed] [Google Scholar]
  • 26.Scott HM, Vittinghoff E, Irvin R, Sachdev D, Liu A, Gurwith M, et al. Age, race/ethnicity, and behavioral risk factors associated with per contact risk of HIV infection among men who have sex with men in the United States. J Acquir Immune Defic Syndr. 2014;65(1): 115–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.CDC. Pre-Exposure Prophylaxis for Prevention of HIV Infection in the United States- 2014: A clinical practice guideline. May 2014. Available at: http://www.cdc.gov/hiv/guidelines/preventing.html.
  • 28.Wejnert C, Hess KL, Rose CE, Balaji A, Smith JC, Paz-Bailey G, et al. Age-Specific Race and Ethnicity Disparities in HIV Infection and Awareness Among Men Who Have Sex With Men-20 US Cities, 2008–2014. J Infect Dis. 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Baral S, Logie CH, Grosso A, Wirtz AL, Beyrer C. Modified social ecological model: a tool to guide the assessment of the risks and risk contexts of HIV epidemics. BMC Public Health. 2013;13:482. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Poundstone KE, Strathdee SA, Celentano DD. The social epidemiology of human immunodeficiency virus/acquired immunodeficiency syndrome. Epidemiol Rev. 2004;26:22–35. [DOI] [PubMed] [Google Scholar]
  • 31.Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Rodger AJ, Cambiano V, Bruun T, Vernazza P, Collins S, Degen O, et al. Risk of HIV transmission through condomless sex in serodifferent gay couples with the HIV-positive partner taking suppressive antiretroviral therapy (PARTNER): final results of a multicentre, prospective, observational study. Lancet. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Rodger AJ, Cambiano V, Bruun T, Vernazza P, Collins S, van Lunzen J, et al. Sexual Activity Without Condoms and Risk of HIV Transmission in Serodifferent Couples When the HIV-Positive Partner Is Using Suppressive Antiretroviral Therapy. JAMA. 2016;316(2): 171–81. [DOI] [PubMed] [Google Scholar]
  • 34.US CDC News. Dear colleague: national gay men’s HIV/AIDS awareness day. (27 September 2017) https://www.cdc.gov/hiv/library/dcl/dcl/092717.html. 2017.
  • 35.ClinicalTrials.gov Safety and Efficacy Study of Injectable Cabotegravir Compared to Daily Oral Tenofovir Disoproxil Fumarate/Emtricitabine (TDF/FTC), For Pre-Exposure Prophylaxis in HIV-Uninfected Cisgender Men and Transgender Women Who Have Sex With Men. Bethesda, MD: National Library of Medicine; Identifier: NCT02720094. Available from: https://clinicaltrials.gov/ct2/show/NCT02720094 [Google Scholar]

RESOURCES