Machine Learning for Human Immunodeficiency Virus Prevention in Rural Africa: The SEARCH for Sustainability

(See the Major Article by Balzer et al on pages 2326–33.)

With highly effective human immunodeficiency virus (HIV) treatment and preexposure prophylaxis (PrEP), our generation is poised to end the HIV pandemic. However, as the US Preventive Services Task Force noted in its recent grade A recommendation for PrEP, successful implementation of these HIV-prevention strategies will rely on the identification of people at high risk of HIV acquisition [1]. Unfortunately, HIV risk assessment is no easy task. Although HIV risk prediction tools have been developed over the past 3 decades, most require manual collection of potentially sensitive data, have limited ability to predict HIV acquisition, and are not widely used.

In this issue of Clinical Infectious Diseases, Balzer et al used data from a large-scale study in rural Kenya and Uganda to construct HIV risk scores and assess their ability to predict HIV acquisition during follow-up [2]. The data source was the intervention arm of the SEARCH study, a randomized trial of a community-level intervention to promote universal HIV testing and treatment during 2013–2017 [3]. Using sociodemographic, health, and household data collected at the time of HIV testing, the authors aimed to identify study participants who were at high risk of HIV acquisition and might benefit from more intensive interventions, including PrEP.

As the authors note, several HIV risk scores had been previously developed and validated in eastern and southern Africa [4–7]. Balzer et al built on this prior work in 2 important ways. Their HIV risk scores were developed using data from a general population rather than a specific risk group, such as men who have sex with men, which could enhance disseminability. They also used machine learning, which can identify unanticipated relationships in complex data and potentially improve prediction by relying on fewer modeling assumptions [8].

Although machine learning has been used to identify potential PrEP candidates in healthcare settings in the United States [9–11], its application to HIV prevention has been limited overall. To understand the added value of machine learning for predicting HIV acquisition, Balzer et al compared the performance of 3 approaches. First, they evaluated an approach based on belonging to a known HIV risk group, creating an HIV risk score by summing the number of groups to which each person belonged. Second, they evaluated a model-based approach that used standard regression methods, creating HIV risk scores from model coefficients. Finally, they used a machine learning approach called Super Learner, an ensemble method that builds a weighted combination of algorithm-specific predictions from a library of candidate algorithms [12].

What advantages does machine learning provide? A standard metric for prediction models is the area under the receiver-operator characteristic curve (AUC), also known as the C-statistic, which represents the probability that a randomly drawn case (ie, a person who acquired HIV) from the study population will have a higher model-generated risk score than a randomly drawn noncase [13]. The AUC for the machine learning approach was only slightly higher than for the model-based approach (0.73 vs 0.70, with 1.0 signifying perfect model performance). However, in contrast to sensitivity, positive predictive value, and number needed to evaluate, the AUC tells us relatively little about how a model will perform if implemented, especially in the setting of a rare outcome such as HIV acquisition [14].

When the authors evaluated measures of model efficiency that incorporated these additional metrics, they found substantial advantages with machine learning. For example, to correctly classify a fixed 50% of new HIV cases as high-risk (ie, 50% sensitivity), the machine learning approach would need to identify only 18% of the population as candidates for PrEP compared with 27% for the model-based strategy and 42% for the risk group strategy. Thus, the machine learning approach could maximize PrEP offerings to those who are most likely to benefit, with a lower number needed to evaluate to prevent 1 new HIV infection. As HIV risk prediction tools continue to be developed, validated, and optimized, researchers should consider using similar efficiency metrics to evaluate model performance and guide future implementation.

Can machine learning algorithms for HIV prevention be implemented in resource-constrained settings? In a previously published report [15], the SEARCH investigators deployed their prediction algorithm in 5 SEARCH communities and offered PrEP to all individuals who had high HIV risk scores or who self-identified as being at risk for HIV acquisition. The accomplishment of collecting sociodemographic and HIV testing data on many thousands of individuals, computing their HIV risk scores at the point of care, and using that information to inform PrEP offerings in rural Africa is remarkable. Nevertheless, only 11% of individuals who were offered PrEP because of their risk score initiated PrEP within 30 days, in contrast to 39% of those who self-identified as being at high risk, suggesting that many people will decline PrEP if they do not already perceive themselves to be at risk of HIV infection.

There may be additional challenges to realizing public health gains from HIV prediction models in rural African settings. The prediction algorithm in SEARCH was feasible to implement as part of a community-based trial, but it is unknown if implementation would be sustainable using only local public health resources, given a need for ongoing collection and integration of the sociodemographic and health data needed to inform model predictions. The algorithm generated by Balzer et al might be particularly difficult to implement outside of a research study because a key predictor was having a partner living with HIV. This variable was ascertained by linking HIV test results between household partners, which might be impractical under routine public health operations.

Machine learning holds promise to optimize PrEP provision in resource-constrained settings, but research is needed to establish sustainability. Implementation science could explore ways to deploy and sustain machine learning algorithms in routine care with locally available resources, including by quantifying the cost savings made possible by more efficient PrEP provision. Studies would also be needed to understand how to communicate the results of risk prediction models to individuals in culturally tailored ways that engender trust and motivate people to adopt PrEP, even if they otherwise perceive themselves to be at low risk for HIV infection. Because HIV and PrEP may be stigmatized, the clinical impact of prediction algorithms could also be improved by communication strategies that build trust in both HIV risk assessments and PrEP itself.

Balzer et al have demonstrated that machine learning is a step forward in HIV risk prediction in rural eastern and southern Africa compared with existing tools. In their prior work, the SEARCH team also demonstrated that machine learning can improve PrEP provision at the point of care in the context of a large research study and the external resources that it brings. The question that remains is whether local public health infrastructure in rural African settings can sustain machine learning strategies for identification of PrEP candidates. In this sense, now that the SEARCH study has ended, the search for sustainability begins.

Notes

Financial support. This work was supported by the National Institute of Allergy and Infectious Diseases (grant K01 AI122853 to J. L. M.), the Eleanor and Miles Shore Faculty Development Awards Program at Harvard Medical School, and the Harvard Pilgrim Healthcare Institute.

Potential conflicts of interest. J. L. M. has consulted for Kaiser Permanente Northern California on a research grant from Gilead Sciences. D. S. K. has participated in research supported by grants from Gilead Sciences to Fenway Health; received honoraria for developing continuing medical education content for Medscape, MED-IQ, and DKBMed; and has received royalties for authoring medical education content for Uptodate, Inc. Both authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.