HIV risk assessment tools for identifying individuals who could benefit from pre-exposure prophylaxis: a systematic review protocol

Abstract

Background

Pre-exposure prophylaxis (PrEP) is a highly effective, safe and acceptable intervention for preventing HIV infection. However, identifying individuals who could best benefit from PrEP remains a significant challenge. Existing HIV risk assessment tools vary in performance depending on context. This systematic review aims to synthesise evidence on their diagnostic performances to predict incident HIV infection.

Methods and analysis

This protocol is informed and reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Protocols. We will search MEDLINE (Ovid), Embase (Ovid) and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases (January 1998–May 2024) for observational and relevant interventional studies assessing the diagnostic performance of HIV risk tools to predict incident HIV for PrEP eligibility. There will be no restrictions on study language or location. Two reviewers will conduct the search, data extraction and risk of bias assessment using the Johanna Briggs Institute Critical Appraisal Checklist for Diagnostic Studies. Standardised templates will be used in Covidence for data extraction. We will conduct a meta-analysis if appropriate, otherwise, a narrative review. We will use the PRISMA guidelines to guide reporting.

Ethics and dissemination of research

Ethical approval is not required as data is publicly available. This review will inform updates to Canadian HIV PrEP guidelines and guide healthcare professionals in using HIV risk assessment tools for identifying PrEP candidates. Findings will be presented at guideline panel meetings and submitted for publication in a peer-reviewed journal and conferences.

PROSPERO registration number

CRD42024543975.

STRENGTHS AND LIMITATIONS OF THE STUDY.

• This systematic review will address a global gap in HIV pre-exposure prophylaxis (PrEP) guidelines by comprehensively synthesising evidence on the diagnostic characteristics of HIV risk assessment tools for PrEP eligibility.

• The review will identify the most effective tools used across diverse settings and populations, evaluating their performance alongside potential risks and benefits.

• This review’s findings will directly inform updates to Canadian HIV PrEP guidelines and further guide healthcare professionals worldwide seeking to use these tools for identifying suitable PrEP candidates.

• This review protocol follows the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols guidelines with transparency regarding the methods and processes that will be used.

• This review will focus exclusively on HIV risk tools for PrEP eligibility, excluding studies solely on optimising population-level HIV testing strategies.

Background

Pre-exposure prophylaxis (PrEP) involves the regular use of antiretroviral medications by HIV-negative persons for the purpose of reducing the risk of HIV acquisition.¹,⁶ PrEP is a highly effective standard of care intervention for individuals at elevated risk of HIV acquisition (reducing the risk from sex by about 99% and from injecting drug use by at least 74%), yet it is underused as a prevention strategy.⁷,⁹ New HIV infections continue to rise among key populations in Canada and globally, signifying ongoing barriers to broader PrEP uptake.¹⁰

Individuals often underestimate their risk of acquiring HIV. Even if they could greatly benefit from taking it, they may not seek out their healthcare providers for PrEP.^{11 12} National and regional guidelines for HIV PrEP exist, but effectively identifying people who could benefit most from PrEP remains a challenge for patients, clinicians and the healthcare system as a whole.^{13 14} This is, in part, a key reason why newer PrEP guidelines in the USA and elsewhere emphasize that people who self-identify as PrEP candidates should be prescribed it and should not be reliant on clinicians’ judgements regarding their risk.^{15 16}

However, simple and accurate HIV risk assessment tools can empower busy clinicians to identify potential PrEP users and integrate PrEP discussions into routine patient encounters.^{17 18}

Several risk assessment tools exist to help clinicians identify individuals at high HIV risk.¹⁹,²⁴ These tools may be particularly helpful to front-line clinicians with only limited familiarity with HIV and related issues of sexual health and harm reduction.²⁵ However, they have important limitations. The tools are often population- and context-specific and may therefore not be suitable for accurately assessing HIV risk in different clinical scenarios.²⁶ Key performance characteristics such as sensitivity and specificity may change over time, even within the same population and geographic context, as the expansion of HIV treatment and other prevention tools changes the epidemiology of HIV and, hence, the risk of infection. Scoring and administration may also be tedious, restricting their suitability for busy clinical settings.²⁷ Consequently, a systematic review of the literature is needed to evaluate these tools, their components and their performances across various populations and settings.

Canadian guidelines on HIV PrEP and post-exposure prophylaxis (PEP) were first developed and published in 2017.²⁸ These guidelines are currently in the process of revision, and an important clinical question to be addressed in the updated guideline relates to how clinicians should be advised to use these tools to assess patients for potential PrEP use. There has also been a notable gap in comprehensive, up-to-date evaluations of these tools’ performance across diverse demographic contexts and geographic settings, which may hinder optimal PrEP delivery and effectiveness. Hence, we are undertaking a systematic review to synthesise current global evidence on HIV risk assessment tools, their performance and validations in predicting HIV acquisition in any population. Specifically, this systematic review seeks to answer the following questions: (1) What clinical tools are available to identify individuals who could benefit from PrEP? (2) Do they accurately predict incident HIV infection? (3) What are their additional implementation characteristics?

Our findings will inform the guideline panel’s recommendations on strategies for identifying individuals who could benefit from PrEP. To prevent duplication of reviews, a preliminary search of similar protocols or reviews was conducted between April and May 2024 in MEDLINE, Google Scholar and the International Prospective Register of Systematic Reviews (PROSPERO) databases. No review protocol or systematic review on this topic published in the last decade was identified.

Objectives

Our primary objective is to evaluate the diagnostic characteristics of HIV risk assessment tools in terms of sensitivity, specificity and the area under the receiver operating characteristic curve (AUC) in predicting incident HIV infections in adults and/or adolescents for the purpose of prescribing PrEP.

Secondary objectives are to describe the following characteristics of identified risk tools:

1. Characteristics of the HIV risk tool (ie, number of items and type of questions);

2. Settings where the tool is used (eg, sexually transmitted infections [STI] clinic, outpatients, primary health clinic and emergency department);

3. External validation (ie, testing the performance of the tool in groups of individuals who are not the same as the development cohort or pilot testing);

4. Methods that were used for developing and validating these tools.

Methods

This systematic review was designed to describe the HIV risk assessment tools, their implementation characteristics and performance to predict incident HIV infection. We plan to conduct this study between 1 May 2024 and 31 December 2024. This protocol has been developed in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) guidelines (online supplemental file 1).²⁹ Reporting of the synthesised findings will be informed by PRISMA guidelines.³⁰ This protocol has been registered in the PROSPERO (registration number CRD42024543975). Important amendments to this protocol will be published along with the results of the systematic review.

Eligibility criteria

To be eligible for this review, studies will need to report on the operating characteristics of HIV risk assessment tools (eg, risk index and risk score) in HIV-negative individuals aged ≥12 years for predicting incident (primary outcome of interest) or prevalent (secondary outcome of interest) HIV infection. We anticipate that the bulk of such studies will be observational in design. Since relevant interventional studies prospectively evaluating the performance of such tools may also exist, we will include both types of studies. There is no restriction on the region, key population and/or HIV risk behaviours studied. Although the key operating characteristics of interest are sensitivity, specificity and AUC, some studies may instead report on positive and/or negative predictive value, from which the sensitivity and specificity can be calculated if the local HIV prevalence is known. Hence, any study reporting on any of these values will be eligible for inclusion. Most included studies will likely be descriptions of the derivation of a risk assessment tool, but some articles may present the results of a validation study, and would also be eligible. Systematic reviews, letters, editorials, duplicated results from the same study, studies that assessed the diagnostic performances of HIV risk tools for population-level HIV testing strategies and studies that reported on the performance of artificial intelligence or machine learning algorithms to predict HIV risk will be excluded.

Information sources

We will electronically search (Ovid) MEDLINE, EMBASE and (EBSCO) CINAHL databases from 1998 to the present. This timeframe was selected because the years since 1997 represent the modern era of highly active antiretroviral therapy, which has radically transformed the clinical course of HIV infection into a fully treatable condition at the individual level and thus affected the risk of HIV acquisition at the population level. We have also searched the PROSPERO registration database to identify ongoing or unpublished systematic reviews. Conference proceedings will not be searched due to time constraints. The reference lists of eligible studies will also be used to identify others of potential relevance. There will be no restrictions imposed on publication language. If studies are identified in languages other than English, the reports will be translated in conjunction with a native speaker of the language, or using Google Translate or a similar online translation tool.

Search strategy

A literature search strategy was developed by an experienced information specialist (IS) (TK), in consultation with the research team. The search will be conducted using subject headings and keywords related to the two main concepts: HIV acquisition and risk assessment tools. The Population, Intervention, Comparator and Outcome (PICO) model, as recommended by the Cochrane Handbook, will be used to develop a search grid for this review. The search grid with identified PICO concepts is presented in table 1.

Table 1. Search grid with identifiable PICO concepts.

PICO concepts
Participants	HIV-negative adults or adolescents at risk of HIV acquisition.
Interventions	Use of any type of clinical tool for assessing the risk of incident HIV infection, which may be labelled by the authors as a risk index, risk score, risk stratification tool or similar term.
Comparators	None
Outcomes	Diagnostic performance characteristics of the tool (sensitivity, specificity and AUC), or PPV and NPV if local HIV prevalence is known.

AUCarea under the receiver operating characteristic curveNPVnegative predictive valuePICOPopulation, Intervention, Comparator and OutcomePPVpositive predictive value

Using the identified PICO concepts, a three-step search strategy was used to identify relevant studies. First, keywords for PICO concepts were brainstormed by reviewers and the IS before the IS drafted an initial search strategy in MEDLINE. Second, the search strategy was reviewed by a second IS using the Peer Review of Electronic Search Strategies checklist for improving the accuracy, quality and comprehensiveness of our search strategy.³¹ Third, a final search of the MEDLINE, EMBASE and CINAHL databases will be conducted using the identified keywords and subject headings. Boolean operators such as ‘OR’ and ‘AND’ will be applied when combining similar search terms and different search terms, respectively. A detailed search strategy is presented in online supplemental file 2.

Selection process

Following the search, all identified citations will be collated and uploaded into the Covidence online systematic review tool as recommended by the Cochrane Handbook.³² This tool is designed to help reviewers remove duplicates, screen abstracts and the full texts of identified articles, assess the risk of bias in included articles and perform data extraction. Abstracts of the relevant full texts will be assessed for eligibility by two reviewers (MMO and DHST) independently. Full-text articles for the selected titles will be further reviewed independently by these reviewers. Disagreements will be resolved by consensus where possible or by a third reviewer (MH) as needed.

Assessment of risk of bias and quality of evidence

Following this selection, methodological quality of each included study will be independently assessed by two reviewers using the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Diagnostic Studies.³³ This appraisal tool includes 10 criteria (rating: yes, no, unclear and not applicable), respectively, with an overall appraisal decision and a narrative form for decision-making. Two reviewers will assign each study a JBI score ranging from 1 to 10, with higher scores indicating higher quality. Resulting quality assessments will be included in the results of the review for descriptive purposes only rather than impacting decisions about eligibility for inclusion in the review. Any disagreements that arise among the reviewers at each stage of the study selection process will be resolved through discussions to reach a consensus. All screening, coding and data abstraction forms will be pilot-tested and revised as necessary. Risk of bias, consistency, directness, precision and publication bias will be included as domains within the tool to be considered, and additional domains may be added as appropriate. The quality will be determined to be high, moderate or very low depending on the certainty of the estimate of the effect.

Data extraction

Data will be extracted by MMO using the customised data extraction form in Covidence (online supplemental file 3).³² Another reviewer will also extract the key data related to our primary objective (sensitivity, specificity and AUC) independently and verify the remaining extracted data. Any discrepancies between reviewers will be resolved through discussion, and a third reviewer will be consulted if necessary to achieve consensus. Extracted data will include study title, author, publication year, study design, study population, location, setting, risk tool, reference standard and data regarding the diagnostic validity of the tool. The key data of interest will be the diagnostic performance data (ie, sensitivity, specificity and AUC). When appropriate, we will extract percentages (sensitivity and specificity), AUC values, cut-off score points, number of incident HIV infections (or prevalent infection if incident number is not reported) and other discriminatory values of the tool (eg, Youden’s index and diagnostic ORs). For any study that lacks information required for proper assessment, the reviewers will attempt to contact the study authors up to three times.

Data synthesis

We will present a systematic narrative synthesis of the articles included, divided into three sections. First, we will provide a preliminary synthesis of findings that will involve a short textual description of each included study, including study design, population, comparative diagnostic data, reference standards and measures of diagnostic validity (sensitivity, specificity and AUC values). These summary measures will also be stratified by target population, country and incident or prevalent HIV as well as by various cut-off points used in these studies. Meta-analysis will only be performed if the data permit and we observe a low level of heterogeneity among risk tools, questions/items within the tools, study populations and study settings. Reasons for data heterogeneity will be explored by considering study design, population, comparable diagnostic data and incident or prevalent HIV infection. We will assess between-study heterogeneity using Higgins & Thompson’s I² statistic, with thresholds for low (25%), moderate (50%) or substantial (75%) heterogeneity. Forest plots will illustrate individual and overall estimates, including heterogeneity values for each tool within each population.

Second, we will explore relationships within the data through thematic analysis, to identify any common themes/trends among HIV risk assessment tools within the included studies. Third, the robustness of the data of included studies will be contextualised using the critical appraisal tool in a narrative format.

Ethics and dissemination of research

This study will not require any formalised ethical review because we will only use data that are publicly available. The findings of this review will be presented to the panel of experts updating the Canadian Guidelines for PrEP and PEP and submitted for presentation at relevant conferences. The findings of this review will also be submitted for publication in a peer-reviewed journal.

Conclusion

Despite established guidelines, identifying the most suitable candidates for HIV PrEP remains challenging. While it is important to empower people to self-identify as PrEP candidates, individuals often underestimate their risk of acquiring HIV, so it is important to support clinicians in proactively identifying individuals who could benefit from PrEP through HIV risk assessment. However, there has been a notable gap in comprehensive, up-to-date evaluations of these tools’ performance across diverse demographic contexts and geographic settings, which may hinder optimal PrEP delivery and effectiveness. Our systematic review will help clinicians determine which tools to use, depending on population and context, so as to accurately identify individuals at high risk, guide PrEP discussions and prevent more HIV infections.