Diagnostic performance of HIV risk assessment tools for identifying pre-exposure prophylaxis candidates: a systematic review and meta-analysis

Summary

Background

To support the implementation of HIV pre-exposure prophylaxis (PrEP), we conducted a systematic review and meta-analysis evaluating the diagnostic performance of HIV risk assessment tools in predicting HIV infection.

Methods

We searched MEDLINE, Embase, and CINAHL for observational studies published between January 1, 1998, and May 13, 2024 that reported on the diagnostic performance of HIV risk assessment tools. We calculated pooled area under the curve (pAUC) values using inverse variance methods, with sensitivity and specificity reported at common cutoffs (PROSPERO registration number: CRD42024543975).

Findings

Of 3704 publications, 27 met our criteria. Twelve studies on men who have sex with men (MSM) assessed nine tools, with four extensively validated, predominantly in U.S. populations. SexPro exhibited the highest performance (pAUC: 0.75), while HIRI-MSM (pAUC: 0.69), Menza (pAUC: 0.63), and SDET (pAUC: 0.66) demonstrated moderate predictive ability, with considerable heterogeneity. For cisgender women, twelve African studies evaluated six tools, with VOICE being the only extensively validated tool (pAUC: 0.65 for adult females; 0.62 for adolescent and young women). Although additional tools were available for subgroups within Africa, there were no tools for cisgender women outside Africa. Among other populations, DHRS demonstrated good discrimination for general U.S. adults (pAUC: 0.80), as did the HIV Prevalence Risk Score for African mixed populations (AUC: 0.70), Kahle for heterosexual serodiscordant couples in Africa (pAUC: 0.73), and ARCH-IDU for people who use drugs in the U.S. (pAUC: 0.72). Sensitivity and specificity varied by cutoffs. Tool items fell into six domains: sexual activities, substance use, clinical factors, demographics, reproductive health, and other factors, with complexity differing by population and context.

Interpretation

Validated tools can help identify HIV risk in some populations, but tools are still needed to promote equitable PrEP access for subpopulations such as cisgender women outside Africa. Public health programs and clinicians should consider incorporating up-to-date, local data to enhance the relevance and effectiveness of existing tools.

Funding

This work was supported by the Canadian Institutes of Health Research (Grant number PCS – 183410). DHST is supported by a Tier 2 Canada Research Chair in Biomedical HIV/STI Prevention.

Research in context.

Evidence before this study

Pre-exposure Prophylaxis (PrEP) prevents human immunodeficiency virus (HIV) infection effectively, but identifying candidates remains challenging because individuals often underestimate their risk and clinicians miss screening opportunities. HIV risk assessment tools can improve equitable PrEP access, especially for marginalized groups, yet most tools are population-specific, and lack clear predictive accuracy. Previous reviews (AVAC 2018; Luo et al., 2023) have not systematically evaluated the performance of clinically relevant tools across diverse populations.

Added value of this study

We systematically searched MEDLINE, EMBASE, and CINAHL (1980–2024), and identified 27 studies spanning men who have sex with men (MSM), cisgender women, general adults, heterosexual HIV-1 serodiscordant couples, and people who inject drugs. Validated MSM tools with moderate accuracy included HIRI-MSM, Menza, SDET, and SexPro, mostly from U.S.-based studies. The VOICE tool showed moderate performance but only among African cisgender women. Tool items fell into six key domains: sexual activities (most commonly higher number of sex partners, condomless receptive anal sex, group sex), substance use (most commonly methamphetamine use, inhaled nitrate use and heavy alcohol use), clinical factors (most commonly history of STI), demographics (age, race/ethnicity), reproductive health, and other social factors such as intimate partner violence.

Implications of all the available evidence

This review fills an important gap by quantitatively synthesizing HIV risk tool performance across populations and regions. Strong tools exist for MSM, but critical gaps remain for women outside Africa, transgender populations, and other subgroups. Revising cutoffs and incorporating local epidemiological data will improve tool relevance and support broader PrEP implementation.

Introduction

Pre-exposure prophylaxis (PrEP) is a highly effective method for reducing the risk of human immunodeficiency virus (HIV) infection, involving the consistent use of antiretroviral medications.1, 2, 3, 4, 5, 6 The recent emergence of long-acting PrEP formulations, including the two-monthly injectable cabotegravir and six-monthly injectable lenacapavir, represent pivotal advancements in efforts to end the epidemic. Despite these innovations, PrEP remains underutilized and global HIV infections continue to rise.⁷

Some current guidelines recommend offering PrEP to anyone who shows interest.^8,9 This approach is an important advance in expanding PrEP access, recognizing that stigma, discrimination, embarrassment and other factors may limit some people's ability to discuss specific indications for PrEP with their healthcare providers. However, many individuals underestimate their own risk of HIV, often leading them to decline or fail to seek out PrEP, even when they could benefit from it.^10,11 Additionally, busy clinicians face challenges in identifying the most suitable candidates for PrEP, despite the existence of national and regional guidelines.¹² Many front-line clinicians who are well-positioned to provide PrEP,^13,14 are unfamiliar with it and struggle to identify who would benefit the most from it.¹⁵ Risk-based tools incorporating a small set of key questions can help these clinicians quickly identify potential candidates and facilitate discussions about PrEP in routine care.^16,17

A comprehensive and up-to-date review of the performance of such tools across diverse demographic and geographic contexts could enhance their application in routine practice, improve clinicians’ ability to effectively scale up PrEP delivery to appropriate candidates, and identify populations which are not served by available tools. A 2018 review by the AIDS Vaccine Advocacy Coalition (AVAC) on risk assessment tools did not evaluate the performance of these tools in predicting HIV risk but instead provided a qualitative overview of key eligibility criteria from PrEP delivery programs.¹⁸ Another review in 2023 focused on HIV infection prediction models specifically for men who have sex with men (MSM), rather than evaluating the broader range of HIV risk tools used in clinical practice.¹⁹ However, simple HIV risk tools for clinical practice exist and their overall effectiveness in predicting HIV infection is not well-established.

To address this need, we conducted a systematic review of the current global evidence on HIV risk assessment tools which are often used for recommending PrEP. Our primary objective was to summarize HIV infection predictive performance characteristics (sensitivity, specificity, and area under the receiver operating characteristic curve or AUC) of these tools, as well as their validation in predicting HIV infection. A key motivation for this review was to support ongoing efforts to update the Canadian HIV PrEP guidelines by synthesizing evidence on the performance of HIV risk assessment tools, which may help inform strategies for identifying individuals who could potentially benefit from PrEP.²⁰

Methods

Search strategy and data extraction

We searched MEDLINE, EMBASE, and CINAHL databases for articles published between January 1, 1980, and May 13, 2024, that reported the diagnostic performance of one or more HIV risk assessment tools in predicting either incident or prevalent HIV infection. We used search terms related to HIV acquisition, risk assessment/scores or clinical decision rules, and PrEP eligibility, without restrictions on mode of HIV transmission, language or geography (Table S1). We also reviewed the reference lists of included articles to identify additional relevant references. The search strategy was developed and peer-reviewed by two independent information specialists following the Peer Review of Electronic Search Strategies (PRESS) guideline.²¹ The study protocol has been published elsewhere,²² and registered with the International Prospective Register of Systematic Reviews (PROSPERO) under the registration number CRD42024543975. The reporting of this review followed PRISMA guidelines.²³

We first screened relevant articles based on their titles and abstracts, followed by a full text review to identify eligible studies. Original research articles on observational studies that reported the performance characteristics (AUC, sensitivity, specificity) of HIV risk assessment tools (sometimes described as clinical prediction rules, indices, or scores) designed to predict incident HIV infection in adults and/or adolescents (≥12 years) were included. In order to maximize usefulness for identifying PrEP candidates, we also included tools predicting prevalent HIV infection. We excluded studies focusing on optimizing HIV testing strategies or studies evaluating purely statistical models and/or machine learning algorithms.

From the included studies, we extracted AUC values along with sensitivity and specificity values either at the optimal cutoff defined within each study or at cutoffs used by other studies evaluating the same tool. From longitudinal studies, we extracted HIV incidence rates (IR) per 100 person-years (PY) and 95% confidence intervals (CI) or extracted the number of HIV infections, total PY, and/or median follow-up duration when the IRs and 95% CIs were not available. From cross-sectional studies, we extracted data on HIV prevalence, including the number of HIV-positive participants and the final eligible sample size. Additionally, we extracted key characteristics of each risk tool, such as the number of variables included, modifications to the original tool, and how tools were validated (internal or external validation). We also recorded details about study participants (e.g., population, age, race/ethnic composition) and study characteristics (e.g., design, location, study year, data sources, and years of data collection). We referenced the tools by their originally reported names and if no name was provided, we used the last name of the first author. Screening (MMO and DHST) and data extraction (MMO and MR) were conducted by two independent reviewers. Any discrepancies were resolved through consensus.

Data analysis

To estimate tool-level performance stratified by population, we calculated the pooled AUC (pAUC) within each specific tool, using random-effects inverse-variance methods. We applied the Hartung-Knapp adjustment to correct the variance of the combined effect size, providing a more conservative estimate of confidence intervals and test statistics, especially given the limited number of studies and significant heterogeneity.²⁴ Between-study heterogeneity was assessed using Higgins & Thompson's I² statistic, categorized as low (<25%), moderate (25%–50%), or substantial (>50%).²⁵ We constructed forest plots to display individual and overall estimates, along with heterogeneity values, for each tool within each population.

We pooled HIV incidence rates per 100 PY using inverse variance methods, and presented these results in forest plots. For studies not reporting incidence rates, we calculated them from available PY or estimated them based on total or median follow-up durations when PY were not provided. These incidence rates were used for the contextual interpretation of the performances of these tools against the World Health Organization's threshold value of 3 per 100 PY to recommend PrEP for HIV prevention.²⁶ Prevalence values were not pooled due to an insufficient number of studies reporting this information.

To illustrate the distribution of variables and domains across tools by population, we summarized common variables across domains for each population. We assessed the risk of bias in included studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool, appraising four domains: patient selection, index tests, reference standard, and flow/timing.²⁷ Risk of bias assessments were conducted for descriptive purposes only and did not influence the decision to include or exclude studies from the review. Following the Cochrane Collaboration recommendation, we did not assess publication bias in this systematic review, as further methodologic research in their accuracy is needed.²⁸ All analyses were performed in R, version 4.4.1, using the meta package.²⁹

Role of the funding source

The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Patient consent for publication

Not applicable.

Role of the funding source

The funder had no role in the design, data collection, data analysis, and reporting of this study.

Results

We identified 3704 potential publications, screened 1997 titles and abstracts after removing 1707 duplicates, and excluded 1948 articles as non-relevant. Of 49 full-text articles retrieved, 27 met our eligibility criteria and were included in the final review (Fig. 1).30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53 Based on the QUADAS-2 tool, four studies exhibited some risk of bias in flow and timing due to their cross-sectional design (Figure S1).^36,42,45,53 However, the overall risk of bias was low, with no major concerns regarding patient selection, index tests, or reference standards.

Fig. 1.

PRISMA Flowchart. HIV, human immunodeficiency virus; AI, artificial intelligence; ML, machine learning; MSM, men who have sex with men; AGYW, adolescent girl and young women.

Characteristics of included studies

Among the 27 articles included in the review, twelve focused on men who have sex with men (MSM), another twelve on cisgender women, and one each on general adults, heterosexual HIV-1 serodiscordant couples, and people who inject drugs (PWID) (Table 1). Four studies used cross-sectional designs,^{33,36,42,45,53} while the majority were cohort studies or randomized controlled trials (RCTs). Eight out of twelve studies among MSM were conducted in the United States (U.S.). One of these studies specifically focused on young Black MSM⁴⁰ while the other seven included a diverse ethno-racial composition including Black participants (from 9% to 47%) and Hispanic participants (from 5% to 32%).^32,36, 37, 38^,40,41,48,49 However, all studies on cisgender women originated from Africa: five focused on adult women,^{31,33,39,43,54} four on adolescent girls and young women (AGYW),^30,35,42,46 two on pregnant and/or lactating women,^44,45 and one on female sex workers.⁵² The data used to derive and validate these tools spanned from 1988 to 2018.

Table 1.

Characteristics of included studies.

Characteristic	Overall N = 27	MSM N = 12 (44%)	Cisgender women N = 12 (44%)
Country
U.S.	10 (37.0%)	8 (66.7%)	–
South Africa	6 (22.2%)	–	6 (50.0%)
Kenya	4 (14.8%)	2 (16.7%)	2 (16.7%)
Africa	4 (14.8%)	–	3 (25.0%)
China	1 (3.7%)	1 (8.3%)	–
Malawi	1 (3.7%)	–	1 (8.3%)
Netherlands	1 (3.7%)	1 (8.3%)	–
Study design
Cohort	13 (48.2%)	9 (75.0%)	3 (25.0%)
Cross-sectional	4 (14.8%)	1 (8.3%)	2 (16.7%)
RCT	10 (37.0%)	2 (16.7%)	7 (58.3%)
Setting
Community	10 (37.0%)	6 (50.0%)	3 (25.0%)
Trial	10 (37.0%)	1 (8.3%)	2 (16.7%)
Primary health care	3 (11.1%)	2 (16.7%)	7 (58.3%)
STD clinic	3 (11.1%)	1 (8.3%)	–
Research clinic	1 (3.8%)	2 (16.7%)	–
Study population
MSM	12 (42.9%)	12 (100%)
AGYW	4 (14.3%)	–	4 (33.3%)
Adult women	5 (17.9%)	–	5 (41.8%)
Pregnant women	1 (3.6%)	–	1 (8.3%)
Young pregnant/lactating women	1 (3.6%)	–	1 (8.3%)
Female sex workers	1 (3.6%)	–	1 (8.3%)
General adult	1 (3.6%)	–	-
Serodiscosrdant couples	1 (3.6%)	–	-
PWID	1 (3.6%)	–	-
African mixed populationa	1 (3.6%)	–	-
Years of data (Range)	1988–2018	1998–2018	2007–2018

MSM, men who have sex with men; U.S., United States of America; RCT, randomized control trial; STD, sexually transmitted disease; AGYW, adolescent girls and young women; HIV, human immunodeficiency virus; PWID, people who inject drug.

^aPrEPVacc registration cohort (Kansiime 2023) included the following in the prevalence risk score development: female bar workers and FSW in Dar es Salaam and Mbeya, Tanzania; men who have sex with men (MSM), FSW and other at risk individuals from the general population in Maputo, Mozambique; and the general population in areas of known high HIV incidence in Durban, South Africa.

HIV incidence and prevalence

Eleven MSM studies reported 2970 new HIV infections over 263,392 person-years (PY) of follow-up in 76,397 participants, resulting in a pooled incidence rate of 2.59 per 100 PY (95% CI: 1.83–3.36) (Figure S2). The young Black MSM cohort⁴⁰ in the U.S. had the highest incidence rate at 8.45 followed by two Kenyan studies^50,51 at 7.50 and 7.04, respectively. A 2017 U S. study³⁸ reported a high incidence rate of 4.62 among Black MSM compared to 1.32 among White MSM.

In ten studies of African cisgender women, 26,679 participants experienced 1053 new infections over 26,217 PY, resulting in a pooled incidence rate of 3.85 per 100 PY (95% CI: 2.61–5.09) (Figure S2). All studies among adult females reported incidence rates above 3, with the CAPRISA004 cohort³³ having the highest rate at 9.04, followed by VOICE⁵⁴ at 6.05 and ECHO⁴³ at 5.43.

Incidence rates among AGYW, pregnant women, and female sex workers were all below 3. For the remaining populations, studies reported an incidence rate of 1.84 (95% CI: 1.59–2.09) among PWID in the ALIVE study⁴⁸ and a pooled rate of 3.05 (95% CI: 2.45–3.66) among serodiscordant couples.⁵⁵ In three cross-sectional studies, HIV prevalence was 13% in South African AGYW,⁴² 0.5% and 0.7% in general U.S. adults³⁶ attending a STD clinic and an emergency department, respectively, and 6% in Chinese MSM.⁵³

Performance of HIV risk tools

We identified nine risk assessment tools for MSM: six from the U.S. (Menza⁴¹ 2009, HIV Incidence Risk Index [HIRI-MSM]⁵⁶ 2012, San Diego Early Test [SDET]³⁷ 2015, Beymer³² 2017, Seattle PrEP Score⁴⁹ and SexPro⁴⁷ 2020), two from Kenya (CDRSS⁵⁰ 2013 and Wahome⁵¹ 2018), and one from China (HIV Risk Assessment Tool⁵³ 2020). HIRI-MSM, Menza and SDET were externally validated, while SexPro was validated on separate cohorts within its derivation study (Table 2). SexPro had the highest pooled AUC of 0.75 (95% CI: 0.70–0.81). HIRI-MSM, Menza, and SDET had pooled AUCs of 0.69 (95% CI: 0.66–0.73), 0.63 (95% CI: 0.57–0.70), and 0.66 (95% CI: 0.58–0.74), respectively (Fig. 2). We observed low-to-high levels of between-study heterogeneity (I²: 0%–88%). Seattle PrEP score, Beymer, and Wahome lacked external validation (Table 2). The Cohort-derived Risk Screening Score (CDRSS)⁵⁰ had a high AUC of 0.85 for symptom-based risks, and Zhang's HIV Risk Assessment Tool, derived using the Delphi method, had an AUC of 0.83.

Table 2.

Derivation and validation of HIV risk tools.

Tool name	Derivation study	Internal validationa	External validationa
MSM
Beymer	Beymer 2017	Beymer 2017	–
CDRSSb	Wahome 2013	–	–
HIRI-MSM	Smith 2012	Smith 2012	Smith 2012; Jones 2017; Lancki 2018; Tordoff 2020;
HIV Risk Assessment Toolc	–	–	Zheng 2020
Menza	Menza 2009	–	Menza 2009; Jones 2017; Tordoff 2020;
SDET	Hoenigl 2015	–	Hoenigl 2015; Jones 2017; Dijkstra 2020; Tordoff 2020;
Seattle PrEP Score	Tordoff 2020	Tordoff 2020	–
SexPro	Scott 2020	Scott 2020	Scott 2020;
Wahome	Wahome 2018	–	–
Cisgender women
VOICE (Adult & AGYW)	Balkus 2016	Balkus 2016	Balkus 2016; Ayton 2020d; Balkus 2018; Castor 2022; Giovenco 2019d; Kansiime 2023; Peebles 2020; Rosenberg 2020d;
Peebles (Adult & AGYW)	Peebles 2020	Peebles 2020	–
Moyo (AGYW)	Moyo 2023	–	Moyo 2023
Pintye (Pregnant women)	Pintye 2017	Pintye 2017	–
Ramraj (Young pregnant/lactating women)	Ramraj 2022	Ramraj 2022	–
Willcox (Female sex workers)	Willcox 2021	–	–
General adult
DHRS	Haukoos 2012	Haukoos 2012	Haukoos 2012
Heterosexual serodiscordant couple
Kahle	Kahle 2013	Kahle 2013	Kahle 2013
People who inject drug
ARCH-IDU	Smith 2015	Smith 2015	–
Mixed population (Africa)
HIV Prevalence Risk Score	Kansiime 2023	Kansiime 2023	–

HIRI-MSM, HIV Incidence Risk Index for MSM; SDET, Sandiego Early Test; PrEP, pre-exposure prophylaxis; CDRSS, cohort-derived risk screening score; HIV, human immunodeficiency virus; VOICE, Vaginal and Oral Interventions to Control the Epidemic; DHRS, Denver HIV Risk Score; ARCH-IDU, Assessing the Risk of Contracting HIV in injection drug users.

^aInternal validation means validating a tool using a subset of the same dataset from which it was developed, whereas external validation means testing the tool with a separate dataset that was not used in its development.

^bSymptom-based HIV risk screening tool.

^cLi et al. (2017) developed this tool using the Delphi method, but did not report its operating characteristics; thus its derivation paper (Li 2017) was excluded from our review.

^dValidated in AGYW population.

Fig. 2.

Area under the curve values of HIV risk tools among men who have sex with men. The X-axis represents the discrimination power of the area under the curve (AUC) statistic, with higher values meaning better performance. The vertical dotted line at 0.5 is equivalent to a coin toss. HIV, human immunodeficiency virus; AUC, area under the curve; CI, confidence interval; D, derivation cohort; V, validation cohort; CDRSS, cohort-derived risk screening score; MSM, men who have sex with men; YBMSM, young Black MSM; HIRI-MSM, HIV Incidence Risk Index for MSM; SDET, Sandiego Early Test; PrEP, pre-exposure prophylaxis; MSM, men who have sex with men; LGBT, lesbian, gay, bisexual and transgender people.

Due to inadequate reporting of necessary data, we were unable to pool sensitivity and specificity values, but instead summarized available values using medians and interquartile ranges. Sensitivity and specificity varied by score cutoffs (Table 3). At their commonly-used cutoffs (Table 3), HIRI-MSM, SDET, Menza and SexPro showed a balance of sensitivity (77%–82%) and specificity (41%–56%), Seattle PrEP Score had the lowest sensitivity (54%) and Wahome the lowest specificity (17%).

Table 3.

Summary performance of HIV risk tools by study population.

Tool name	Number of items	Cutoff	Sensitivitya	Specificitya	Pooled AUC (95% CI)b
MSM
Beymer	11	≥5	75%	50%	NR
CDRSSc	6	≥2	81%	76%	0.85 (NR)
HIRI-MSM	7	≥10	82% (76%–85%)	44% (37%–51%)	0.69 (0.66–0.73)
HIV Risk Assessment Toolsd	8	≥0.916	78%	75%	0.83 (NR)
Menza	4	≥1	79% (65%–87%)	41% (32%–41%)	0.63 (0.57–0.70)
SDET	4	≥1	77% (75%–79%)	48% (48%–49%)	0.66 (0.58–0.74)
Seattle PrEP Score	4	≥2	54% (50%–58%)	69% (69%–69%)	0.65 (0.56–0.74)
SexPro	11	≥16	78% (73%–86%)	56% (39%–62%)	0.75 (0.70–0.81)
Wahome	5	≥1	98%	17%	0.76 (0.71–0.80)
Cisgender women
VOICE (Adult)	7	≥3	98% (91%–99%)	15% (8%–30%)	0.65 (0.60–0.69)
VOICE (Adult)	–	≥5	93% (83%–95%)	23% (6%–46%)	–
VOICE (AGYW)	–	≥3	65% (50%–79%)	35% (18%–52%)	0.62 (0.57–0.67)
VOICE (AGYW)	–	≥5	85% (43%–89%)	32% (19%–64%)	–
Peebles	7	≥5	64% (56%–71%)	57% (50%–64%)	0.65 (0.62–0.69)
Moyo (AGYW)d	13	≥2.43	71%	60%	0.76 (NR)
Pintye (Pregnant women)	7	≥8	79%	84%	0.80 (0.70–0.90)
Ramraj (Pregnant women)d	7	≥31	73% (66%–80%)	57% (51%–64%)	0.71 (0.70–0.72)
Willcox (Female sex workers)	3	≥1	53%	76%	0.67 (0.52–0.82)
General adult
DHRSd	8	–	NR	NR	0.80 (0.70–0.90)
Heterosexual HIV serodiscordant couples
Kahle	6	≥5	73%	67%	0.73 (0.71–0.76)
People who inject drug
ARCH-IDU	7	≥46	86%	42%	0.72 (NR)
Mixed population (Africa)
HIV Prevalence Risk Score	7	≥6	70%	63%	0.70 (0.66–0.74)

HIV, human immunodeficiency virus; AUC, area under the curve; CI, confidence interval; NR, not reported; HIRI-MSM, HIV Incidence Risk Index for MSM; SDET, Sandiego Early Test; PrEP, pre-exposure prophylaxis; CDRSS, cohort-derived risk screening score; HIV, human immunodeficiency virus; VOICE, Vaginal and Oral Interventions to Control the Epidemic; DHRS, Denver HIV Risk Score; ARCH-IDU, Assessing the Risk of Contracting HIV in injection drug users.

^aSensitivity and specificity values are summarized into median values and interquartile ranges (raw values were shown for tools with a single study). Meta-analysis was not performed because many studies did not report raw data nor 95% CI values to derive standard errors for pooling.

^bAUC and 95% CI values were pooled using generic inverse variance methods (metagen function in {meta} R package). Standard errors were derived from 95% CIs.

^cSymptom-based HIV risk screening tool.

^dIncluded 4 studies with cross-sectional design (Haukoos 2012, Moyo 2023, Ramraj 2022, Zheng 2020).

For cisgender women, we identified six tools (Table 2): VOICE⁵⁴ (2016) and Peebles⁴³ (2020) for adults and adolescent girls and young women (AGYW), Moyo⁴² (2023) for AGYW, Pintye⁴⁴ (2017) and Ramraj⁴⁵ (2022) for pregnant/lactating mothers, and Willcox⁵² (2021) for female sex workers. VOICE was more-extensively validated than other tools, with pooled AUC values of 0.66 (95% CI: 0.61–0.71) for adults and 0.62 (95% CI: 0.57–0.67) for AGYW, showing high heterogeneity (I² = 87%) among adults (Fig. 3). Peebles and Moyo demonstrated similar performance to VOICE, but lacked validation (Table 2). Pintye⁴⁴ had an AUC of 0.80 (95% CI: 0.70–0.90) and Ramraj⁴⁵ an AUC of 0.71 (95% CI: 0.70–0.72) for pregnant and lactating women. Willcox⁵² had an AUC of 0.67 for female sex workers. At commonly used cutoffs of ≥3 and ≥ 5 (Table 3), VOICE showed high sensitivity (98% and 93%) but low specificity (15% and 23%), while other tools varied in sensitivity (53%–79%) and specificity (57%–84%).

Fig. 3.

Area under the curve values of HIV risk tools among cisgender women. The X-axis represents the discrimination power of the area under the curve (AUC) statistic, with higher values meaning better performance. The vertical dotted line at 0.5 is equivalent to a coin toss. HIV, human immunodeficiency virus; AUC, area under the curve; CI, confidence interval; AGYW, adolescent girls and young women; D, derivation cohort; V, validation cohort.

For other populations, four tools were available: DHRS³⁶ for the general adult population, ARCH-IDU⁴⁸ for U.S.-based PWID, Kahle⁵⁵ for heterosexual serodiscordant couples, and Kansiime's HIV Prevalence Risk Score (Tables 2 and 3). DHRS, developed and externally validated in 2012 for U.S. adults attending an STD clinic and urban emergency department, had the highest AUC of 0.8 (95% CI: 0.70–0.90) (Fig. 4). ARCH-IDU⁴⁸ had an AUC of 0.72 with high sensitivity but modest specificity. Kahle⁵⁵ and HIV Prevalence Risk Score both showed high AUC values (0.72 and 0.70, respectively), with balanced sensitivity and specificity. Detailed performance metrics are provided in Table S2.

Fig. 4.

Area under the curve of HIV risk tools among other populations. The X-axis represents the discrimination power of the area under the curve (AUC) statistic, with higher values meaning better performance. The vertical dotted line at 0.5 is equivalent to a coin toss. HIV, human immunodeficiency virus; AUC, area under the curve; CI, confidence interval; D, derivation cohort; V, validation cohort; DHRS; Denver HIV Risk Score; ARCH-IDU, Assessing Risk of Contracting HIV in Injection Drug Users; STD; sexually transmitted diseases; ED UCMC, emergency department of the University of Cincinnati Medical Center.

Common variables across tools by study population

The number of variables varied across tools, averaging six items (range: 4–11) for MSM and seven items (range: 3–13) for cisgender women (Table 3). We categorized these items into six domains: sexual activities, substance use, clinical factors, demographics, reproductive health, and other factors (Tables S3 and S4, and Figure S3). MSM tools primarily focused on sexual activities and substance use, identifying higher HIV risk with a higher number of sex partners, condomless receptive anal sex, group sex, recreational drug use, and heavy alcohol use. A history of STI diagnosis was an HIV risk factor for both MSM and cisgender women. In both populations, demographic questions associated HIV risk with younger age and having age disparities with partners. Black race and Hispanic ethnicity were considered as risk factors in MSM tools from the U.S., where these populations are disproportionately impacted due to structural inequities, while marital status and socioeconomic factors were emphasized for African women. Local factors such as high HIV-1 prevalence and geography were also included in tools for African women, with reproductive health questions identifying higher HIV risk among pregnant or lactating women. Notably, studies using the VOICE tool frequently modified questions due to data gaps or definitional changes during validation, resulting in multiple versions of the original VOICE model⁴⁶ (Table S5).

Discussion

This systematic review evaluated the diagnostic performance of risk tools for predicting HIV infection and identifying suitable candidates for PrEP. We identified four extensively validated tools with good performance for MSM, in primarily US-based populations: HIRI-MSM, Menza, SDET, and SexPro. For cisgender women, VOICE was the only extensively validated tool, showing promising performance in African women, but no tools were available for women outside Africa. Across all risk indices, commonly recommended cutoffs tended to prioritize sensitivity (generally ranging from 14% to 98%) over specificity (4%–97%). Multiple studies reported HIV incidence rates exceeding the WHO's “substantial risk” threshold of 3 infections per 100 PY, justifying PrEP recommendations.²⁶ Among six identified domains, questions regarding sexual activities and substance use were common for MSM, while demographics and reproductive health were emphasized for cisgender women, and a history of STIs was common in both.

Among the four extensively validated tools for MSM, HIRI-MSM, Menza, and SDET demonstrated moderate discrimination, comparable to the CHADS2 score⁵⁷ for stroke risk in atrial fibrillation (AUC: 0.64–0.67), a widely cited model that has historically guided anticoagulation decisions despite moderate accuracy. However, they generally lagged behind SexPro which demonstrated superior performance comparable to established clinical prediction tools in other fields of medicine, such as the Framingham Risk Score⁵⁸ for cardiovascular risk (AUC: 0.72), the APACHE score⁵⁹ for severe disease or death in intensive care (AUC: 0.75–0.82), and the Ottawa ankle rules⁶⁰ for foot bone fracture (AUC: 0.71). Since the same data from EXPLORE and VAX004 RCTs were used for the development of SexPro, HIRI-MSM, and Menza, we speculate that the improved performance of SexPro could be attributable to its more balanced incorporation of variables across four domains, including race/ethnicity (Black or Latino), heavy alcohol use, and a history of STI, in addition to sexual activities and substance use. It is also important to note that SexPro was validated at the same time in the same study as it was developed, and there are no other studies externally validating this tool at the time of our review.

The observed heterogeneity in the pooled performance of the MSM tools, as quantified by the AUC values, may stem from two main factors. First, many were developed using predominantly White cohorts in the U.S., but validated in more diverse groups including Black, Latino, or Hispanic participants.^{32,38,40,41,49} For example, the HIRI-MSM, Menza, and SDET tools performed less effectively in Black MSM and young Black MSM compared to their white counterparts.^38,40 Our pooled results provide an overall summary of tool performance, and may particularly be useful for programs and providers caring for a diversity of patients, but users of these tools should bear this important source of variability in mind. Second, the tools often focus on sexual behaviours and substance use that may not fully capture the complexities of the HIV epidemic in certain MSM populations. Additionally, the substance use addressed in the tools was restricted to methamphetamine or inhaled nitrates, limiting their relevance in areas where these substances are not the main drivers of HIV transmission.³⁸

Some papers have identified race to be a strong predictor of HIV risk.^32,38,47 However, we argue that framing the issue this way overlooks the broader context of systematic disparities, and may unintentionally contribute to systemic racism. Racial disparities in HIV rates highlight underlying systemic factors such as poverty, access to healthcare, and socioeconomic challenges that disproportionately affect marginalized communities.61, 62, 63 Rather than treating race as a risk factor, it may be more useful to integrate social and structural drivers such as housing instability, poverty, migration, and high prevalence networks, particularly among Black and Latino MSM, into the context-specific clinical application of existing tools, or into the development of novel tools. Additional factors such as country of origin, impulse control, and specific geographic contexts⁶⁴ such as high HIV prevalence in rural U.S. South settings, should also be incorporated.

VOICE is the only extensively validated tool for cisgender adult and young women, demonstrating comparable performance with MSM tools. It predicted incident HIV infections in adults slightly better than in AGYW, as it was originally developed using data from the VOICE trial of daily oral tenofovir disoproxil fumarate and 1% vaginal tenofovir gel as PrEP.⁵⁴

Unfortunately, no HIV risk tools were identified for cisgender women outside Africa. This poses a particular challenge in high-income countries like the U.S., Canada, and Europe, where HIV incidence rates in women are generally lower than the WHO's recommended threshold, but where gender-based inequities in PrEP uptake are worsening.65, 66, 67 In the U.S., cisgender women represented 18% of new HIV infections in 2022,⁶⁸ while in Canada, women accounted for half of new diagnoses, with rates on the rise.⁶⁹ Although general adult population tools such as the DHRS could potentially be used among women to predict HIV risk, the cohort from which it was developed predominantly consisted of men (85%), making its variables more MSM-focused. There is a clear and pressing need for tools specifically tailored to the needs of cisgender women outside Africa as well as transgender populations.

Despite injection drug use being the “most hidden face of the HIV epidemic”⁷⁰ amid rising substance use trends in North America and Australia, ARCH-IDU⁴⁸ is the only tool specifically designed to assess HIV risk in this population. In addition, ARCH-IDU concentrated solely on injection drugs, while other forms of recreational drug use have also been linked to driving the HIV epidemic.^71,72 Given that DHRS already includes a variable for injection drug use, expanding its scope of substance use questions to encompass other recreational drug use and drug-related behaviours (e.g., chemsex) could be helpful.

Following basic epidemiological principles, tool sensitivity and specificity varied across cutoffs, with higher cutoffs (indicating higher risk) corresponding to lower sensitivity and higher specificity. Many recommended cutoffs were selected somewhat arbitrarily, generally prioritizing sensitivity over specificity.^{37,41,47,49,54,56} For any risk index, the most appropriate cutoff value depends on the goal of its use. When applied to PrEP implementation, lowering the selected cutoff would increase the number of people for whom PrEP is recommended, which may be increasingly appropriate as PrEP costs decline (e.g., as drugs go off-patent), and as more PrEP products become available (ensuring more people will find at least one suitable PrEP option). In contrast, a higher cutoff might be motivated by cost considerations,⁷³ which may be increasingly relevant with the emergence of costly new long-acting PrEP technologies, or could be appropriate in subpopulations approaching micro-elimination of HIV.⁶⁴

A shortcoming of the identified literature is that most validation studies did not reassess the optimal cutoff for their specific contexts. Ideally, public health programs and clinical practices should incorporate up-to-date local epidemiology to tailor use of these tools to maximize impact. Optimal cutoffs should serve as guidance rather than strict criteria, to prevent them from becoming barriers. For example, jurisdictions facing higher HIV incidence rates might select lower cutoff scores for recommending PrEP, and/or may recommend additional criteria for PrEP based on local epidemiology.

The findings of this study should be interpreted in light of its strengths and limitations. First, the included studies generally did not provide the raw data for true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), which are essential for using more robust meta-analysis approaches such as hierarchical models. These newer models could be beneficial for test accuracy meta-analyses in the future if such data become available. Second, our analysis focused on studies reporting risk tools or clinical prediction scores, excluding those reporting purely statistical and/or machine learning models without the development of a specific risk scoring tool. While such models may yield better AUC values,¹⁹ they generally require extensive patient-specific data to be available, and we prioritized practical tools that front-line clinicians can use in real-world settings. Third, concerns arise from the heterogeneity observed in the pooled AUC measures across different tools, a consideration that motivated us to use Hartung-Knapp adjustment in our meta-analysis methods. While diverse data sources and populations may have introduced variability in performance, extensive validation in real-world scenarios enhanced their overall applicability. Fourth, the absence of reported confidence intervals, particularly for sensitivity and specificity values, in several studies limited our ability to derive standard errors and conduct a more comprehensive meta-analysis inclusive of all available data points. Fifth, although we quantified the overall frequency with which individual predictor variables appear across risk tools in Figure S3, we did not analyze their individual contributions to variance in HIV risk, an approach which could potential further refine risk tools. Sixth, we summarized HIV incidence rates in included studies to provide context for our findings; these rates should not interpreted as representative of the populations studied. Finally, our analysis concentrated on the broader applicability of these risk tools in clinical and public health practice, rather than examining their development processes and methodological approaches as addressed by Luo et al. (2023).¹⁹ Therefore, we did not follow the guidance of the critical appraisal and data extraction for systematic reviews of prediction modeling studies CHARMS (Moons et al., 2014) as it is not relevant to our analysis.⁷⁴

Individual risk profiles, including sexual and substance use behaviours, can change over time,^75,76 and it is crucial to offer HIV prevention support to those at elevated risk. Validated HIV tools, when used regularly, may assist clinicians in identifying individuals who could benefit from further risk evaluation and discussions about prevention options, including PrEP. As these tools typically assess behaviours over the past three to six months, periodic reassessment may be useful in both clinical and programmatic settings.

In conclusion, HIV risk assessment tools demonstrated potential for supporting the identification of individuals at increased risk of HIV acquisition. While PrEP access should remain inclusive and client-driven, the context-specific use of existing tools, along with the development of new tools for understudied populations, may complement broader efforts to tailor prevention strategies and enhance equitable PrEP uptake without excluding those who seek it.

Contributors

Conceptualization—DHST.

Protocol—MMO & DHST.

Screening, data extraction—MMO, MR, DHST.

Accessed and verified the data—MR, DHST.

Analysis—MMO, MR, DHST.

1st draft of manuscript—MMO, MR.

Critical review of manuscript—all authors.

Data sharing statement

The data supporting the findings of this study are available within the article and its supplementary materials. Additional data, including extracted datasets and analytic code, can be made available upon reasonable request to the corresponding author, subject to ethical and privacy considerations.

Declaration of interests

DHST's institution has received support from Gilead for investigator-initiated research grants, and from Glaxo Smith Kline and ViiV for participation in industry-sponsored clinical research. CS is a CIHI principle investigator and co-investigator in multiple investigator-initiated clinical research studies involving HIV/STI prevention clinical care, as well as Gilead Sciences-supported Investigator initiated implementation science work investigating HIV prevention medication uptake among patients engaged in substance use programs. He also received support from both Gilead Sciences and ViiV for speaking engagements on HIV prevention risk assessment and a sponsorship to attend 2024 Glasgow HIV Therapeutics Conference from Gilead Sciences. He is also Chief Medical Officer of PurposeMed (Freddie)'s HIV prevention and treatment service operating in Canada and USA.