Development and validation of a predictive model for new HIV infection screening among persons 15 years and above in primary healthcare settings in Kenya: a study protocol

Abstract

Introduction

This study seeks to determine incidence, comorbidities and drivers for new HIV infections to develop, test and validate a risk prediction model for screening for new cases of HIV.

Methods and analysis

The study has two components: a cross-sectional study to develop the prediction model using the HIV dataset from the Kenya AIDS and STI Control Programme and a 15-month prospective study for the validation of the model. Inferential analysis will be conducted using algorithms that perform best in disease prediction: Extreme Gradient Boosting (XGBoost) and Multilayer Perceptron. Model sensitivity and specificity will be examined using the receiver operating characteristic curve, and performance will be evaluated using metrics: accuracy, precision, recall and F1 score.

Ethics and dissemination

The study obtained ethical approval (JKU/ISERC/02321/1421) from the Jomo Kenyatta University of Agriculture and Technology Ethical and Research Board and a research licence (NACOSTI/P/24/414749) from the National Commission for Science, Technology and Innovation.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Existing HIV risk-scoring functions are developed using historical data with a myriad of quality challenges.

WHAT THIS STUDY ADDS

• This study proposes a cross-sectional study to develop the risk-scoring function and a prospective study for the validation of the model considering data quality, performance and generalisability of the model.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• This study aims to identify gaps in HIV screening and health systems to inform targeted interventions.

Introduction

HIV remains one of the most life-threatening infectious diseases worldwide.¹ A recent report by WHO shows that 85.6 million people have been infected with HIV globally. Sub-saharan Africa bears the greatest burden of HIV, with a prevalence of 25.6 million people living with HIV. In Kenya, 1.4 million people are reported to be infected with HIV, and 13 000 adults aged 15 years and above were newly infected with HIV during the year 2023.²

Screening for HIV cases is an effective method for the identification of new HIV infections.³ Community-based screening, despite the associated cost, is the most effective method for the identification of new infections of HIV. Universal screening, which involves subjecting all persons seeking primary healthcare services, seems to be a possible alternative to community-based screening.⁴ However, HIV testing faces challenges such as inadequate personnel and resource limitations.^{5 6}

HIV incidence stands at about 8.5% and 13% for males and females, respectively. Predictors for new HIV infection include level of education, economic capability, use of alcohol, age when a person first had sex, gender and relationship to the head of the family.⁷ Also, the presence of morbidities such as oral candidiasis, lymphadenopathy and persistent fever is more common among cases of new HIV infections.⁸

A recent review⁹ identified a few studies that attempted to develop HIV risk-scoring functions using retrospective study designs. One¹⁰ compared the performance of eight machine learning algorithms, and results had limitations such as a small sample size comprised of imbalanced classes. Also,^{7 11} utilised large secondary datasets (87 000 and 83 000, respectively) characterised by a high degree of missing data¹² and model performance was inconclusive due to the fact that the dataset was self-reported.

This study seeks to determine the incidence, comorbidities and drivers for new HIV infections among persons 15 years and above attending outpatient clinics at selected hospitals in Kenya and develop, test and validate a risk-scoring function for screening for new HIV infections. Even though this study uses a retrospective dataset for the development of the ML model, we are planning to ensure that the dataset used in this study is both representative and of good quality. Unbalanced data results in skewed over-fitted models that are ungeneralisable, thus resulting in high recall. Therefore, our study aims to bridge these gaps in order to develop a predictive model for HIV screening that enhances performance while minimising low recall.

Methods and analysis

Study design

The first component of this study is a cross-sectional study for the development of a risk prediction model using the HIV dataset from the Kenya AIDS and STI Control Programme (NASCOP). The second component is a 15-month prospective study comprised of 1308 participants who meet the inclusion criteria for the validation of the predictive model.

Study population

Development study

The study population for the development of the HIV risk prediction model is comprised of records of patients who had sought HIV services in Kenya.

Validation study

The study population for the validation study will include all persons attending outpatient clinics at selected County Referral hospitals in Kenya. Study participants will be required to provide information such as social demographics, social and family health history.²

Eligibility criteria

The sampled records will be balanced to ensure that male and female clients aged 15 years and above with no more than 10% missing information are sufficiently represented.

Inclusion criteria for the validation study are as follows:

1. Participant must be 15 years of age and above.

2. Participants must be able to communicate in English or Kiswahili.

3. Participants must have been a resident of the respective county for the past 12 months

4. Verbal assent from adolescents aged 15–17 years old.

5. Informed written consent from adult participants and caregivers to adolescents.

Exclusion criteria for the validation study are as follows:

1. Do not give consent to the study.

2. Previous use of antiretroviral therapy.

3. Previous use of pre-exposure prophylaxis or post-exposure prophylaxis services.

4. Inability to participate in the interview because of sickness or cognitive impairment.

Sample size calculation

Development study

The appropriate sample size will be determined considering the desired level of precision, confidence and variability. Therefore, this study will consider all the 88 284 records in the HIV dataset of clients 15 years of age and above.

Validation study

Considering the prevalence of HIV in Kenya, a sample size of 1308 participants will be gathered from 10 counties: Kisumu, Nairobi, Homabay, Siaya, Migori, Nakuru, Mombasa, Kakamega, Kiambu and Kisii, which contribute 57% of new HIV infections in Kenya.²

Sampling technique

A stratified sampling of male and female participants will be considered to ensure that the dataset is sufficiently balanced. For the development study, the dataset will be split as 80% for training the prediction function and 20% for testing the trained model.

Data collection

Development study

Data for the development of the risk prediction model have been obtained from NASCOP after making a formal request. The data have been shared in three comma-separated value files.

Validation study

Study participants will be subjected to a 10-min interview by research assistants using interviewer-administered questionnaire. The prospective validation study flow diagram is as shown in figure 1.

Figure 1. The prospective validation study flow diagram.

Data pre-processing

Electronic medical records are commonly characterised by incompleteness and the presence of noise.¹³ Consequently, data entry errors, missing values and outliers will be handled appropriately. Thereafter, data will be subjected to rescaling using z-score normalisation.¹⁴

Data analysis

Descriptive analysis to determine measures of central tendency, dispersion and frequency tables. Feature selection, which supports the reduction of the number of input variables to features that are most useful to the model using the Sequence Forward Selection (SFS).¹⁵ SFS is a bottom-up search strategy that begins with an empty set and gradually adds features one after another while optimising each addition to improve the targeted metrics.¹⁶ Thereafter, inferential analysis will be conducted using two methods: Extreme Gradient Boosting (XGBoost), a tree-based ensemble Machine Learning (ML) algorithm and a Multilayer Perceptron, a type of Artificial Neural Network algorithms that perform best in disease prediction models¹³ with an expectation of achieving high sensitivity and minimised recall. The sensitivity and specificity will be determined with the use of receiver operating characteristic curve, which is the curve obtained by plotting the likelihood of a true positive result against the probability of a false positive result. The performance of the model will also be evaluated using metrics such as accuracy, precision, recall and F1 score.¹⁷

Pathways to impact

The output of this study will be used in the identification of potential new cases for HIV in primary healthcare, thus reducing costs that come with the universal screening of HIV. The results from this study will be submitted for publication in a peer-reviewed journal.