Utility of the International HIV Dementia Scale for HIV-Associated Neurocognitive Disorder

Abstract

Background:

We assessed the utility of the International HIV Dementia Scale (IHDS) in detecting HIV-associated neurocognitive disorder (HAND) in Uganda in antiretroviral (ART)-naïve and ART-experienced adults.

Setting:

Longitudinal observational cohort study in Rakai, Uganda.

Methods:

399 HIV+ ART-naïve adults underwent neurological, functional status, and neuropsychological assessments including the IHDS. 312 participants who initiated ART were re-evaluated after two years. HAND stages (asymptomatic neurocognitive impairment [ANI], minor neurocognitive disorder [MND], HIV-associated dementia [HAD]) were determined based on Frascati criteria using local normative data. Sensitivity, specificity, and area under the ROC curve (AUC) were determined for various IHDS thresholds (≤9, ≤ 9.5, and ≤10).

Results:

At baseline, participants’ mean age was 35 years (SD ± 8), 53% were male, and 84% had less than a high school education. At baseline, sensitivity for detecting any HAND stage, symptomatic HAND [MND, HAD], and HAD alone was maximized at IHDS ≤10 (81%, 83%, 92%, respectively). Among 312 individuals who returned for the two-year follow-up and had initiated ART, a score of ≤10 provided a lower or equal sensitivity for detecting different stages of HAND (all HAND: 70%; symptomatic HAND: 75%; HAD: 94%). The AUC was higher for ART-experienced versus ART-naïve individuals.

Conclusion:

The IHDS is a potentially useful screening tool for neurocognitive impairment in rural Uganda for both ART-naïve and ART-experienced adults. A cutoff ≤10 demonstrates higher sensitivity for more severe HAND stages compared to less severe HAND. Future studies should focus on potential modifications to the IHDS to improve its specificity.

INTRODUCTION

Prior to 1996 and the discovery of antiretroviral therapy (ART), HIV-associated dementia (HAD) was invariably fatal. The initiation of ART allowed patients with HAD to live longer resulted in reduced incidence of HAD.^[1] However, HAD is the most severe stage of a spectrum of neurocognitive impairment known as HIV-associated neurocognitive disorder (HAND). HAND also includes asymptomatic neurocognitive impairment (ANI) and mild neurocognitive disorder (MND) that are now seen more frequently (15–55%) in the ART-experienced HIV+ population.^{[2, 3]} Identifying even milder stages of HAND is important as these patients have higher rates of co-morbidities, longer hospital stays, and a higher likelihood of developing more significant neurocognitive impairment over time than those without neurocognitive impairment.^{[4, 5]}

The greatest burden of HAND worldwide is likely in sub-Saharan Africa where almost 70% of the global HIV+ population resides, and studies have shown that those with HIV infection in this region have worse neurocognitive performance that those without HIV infection.^[6–13] More specifically, a study in Uganda with 81 participants found 49% had HAND (20% with MND and 18% with HAD).^[14] However, most resource-limited settings in sub-Saharan Africa are unable to complete a comprehensive evaluation for HAND, including formal neuropsychological testing, and HAND is often underdiagnosed in these settings. As such, a validated screening tool applicable to resource-limited settings and multiple cultures is essential.

The International HIV Dementia Scale (IHDS) is a clinical screening tool designed to detect HAD first developed and validated in urban Uganda.^[15] The simplicity and brevity of the screen has allowed it to be used in several resource-limited countries, and administered by personnel untrained in neurology.^{[14, 16–18]} However, limited research has been conducted on the efficacy of the IHDS in rural sub-Saharan Africa and for less severe HAND stages. We assessed the utility of the IHDS for detecting milder HAND stages in a large population of ART-naïve and ART-experienced HIV+ adults in Rakai, Uganda.

METHODS

Study Participants.

The participants in this study were enrolled from the Rakai Community Cohort Study, an open, community-based cohort of adults residing in 50 communities in Rakai District, Uganda, and from individuals presenting for routine care or for HIV testing to HIV clinics supported by the Rakai Health Sciences Program. Participants were ART-naive HIV+ adults ≥ 20 years old with either advanced immunosuppression (CD4 count ≤ 200 cells/μL) or moderate immunosuppression (CD4 count 350–500 cells/uL) at the time of enrollment. Exclusion criteria included severe systemic or central nervous system illness, inability to provide informed consent, physical disability resulting in an inability to travel to the main Rakai Health Sciences Program clinic, and plans to leave Rakai District within two years of enrollment. Participants were evaluated at baseline and after two years. All participants were referred for ART initiation, provided free of charge, upon enrollment.

Study Procedures.

A longitudinal observational cohort study was conducted between July 2013 and July 2015. All participants completed a sociodemographic and behavioral interview, depression screen (Center for Epidemiologic Studies Depression Scale (CES-D)),^[19] functional status assessments (Patient Assessment of Own Functioning Inventory (PAOFI)),^[20] an instrumental activities of daily living inventory,^[21] and Karnofsky Performance Status,^[22] and a neurocognitive assessment (World Health Organization (WHO) University of California – Los Angeles (UCLA) verbal learning test, WHO UCLA verbal delayed recall, timed gait, finger tapping, dominant and non-dominant grooved pegboard, symbol digit, color trails parts 1 and 2, animal naming, and digit span forward and backward).^[23] A medical officer, a physician who completed internship but no neurology training, performed a neuromedical evaluation on all participants after receiving training from study neurologists (DS, NS). The IHDS (Figure 1) and all other neurocognitive assessments were administered by two study nurses who received training in their administration from study investigators (D.S., N.S., K.R., N.N.). The neurocognitive battery took approximately one hour to complete, was performed in the participant’s primary language (e.g. English or Luganda – the predominant local language), and any signs of fatigue and effort level were noted by the examiner for each participant. Peripheral blood draw was performed for confirmation of HIV status and CD4 count. After two years, study participants returned for a follow-up evaluation where the same protocol was performed.

Figure 1.

The International HIV Dementia Scale (IHDS).

Standard protocol approvals, registrations, and patient consents.

Written informed consent was obtained from all study participants. This study was approved by the Western Institutional Review Board, the Uganda Virus Research Institute Research and Ethics Committee, and the Uganda National Council for Science and Technology.

Statistical Analysis.

HAND stage (i.e. ANI, MND, and HAD) was determined using Frascati criteria^[2] and normative neurocognitive data collected from 400 HIV-negative age- and gender-matched adults from the Rakai Cohort Study as described previously.^[23] HAND stage was then dichotomized as follows: (1) abnormal (ANI, MND and HAD) versus unimpaired; (2) symptomatic HAND (MND and HAD) versus unimpaired and ANI; and (3) dementia (HAD) versus unimpaired, ANI, and MND. Depression was defined as a CES-D score ≥ 16.

All participants were included in the analysis of baseline data which is reported as the ART-naïve cohort data. Only participants who had initiated ART by the time of their two-year follow-up were included in this follow-up analysis, and these data are reported as the ART-experienced cohort. Thus, both baseline and follow-up data is included in these analyses, and the ART-experienced cohort consists of a subset (78%) of the ART-naïve cohort. As such, the data presented reflects IHDS performance from the same cohort at two different time points – once before and once after ART initiation.

Diagnostic utility of the IHDS was assessed at different score cut-offs (≤9, ≤9.5, ≤10) for each category of neurocognitive impairment using sensitivity, specificity, positive and negative predictive values, percent of cases correctly classified, positive and negative likelihood ratios, receiver operating characteristic (ROC) curves, and area under the ROC curves (AUC). STATA version 14 (StataCorp, College Station, TX) was used to perform these analyses.

RESULTS

Of the 399 ART-naïve participants evaluated at baseline, approximately half (53%) were men with mean age 35 ± 8 years and mean education of 5 ± 3 years (Table 1). Half of participants had CD4 count ≤ 200 cells/μL at baseline, and nearly one-quarter of participants met criteria for depression. 333 (83%) participants returned for their two-year follow-up evaluation, and 312 of these had initiated ART (94%). Of those initiating ART, >90% had been on ART for at least one year prior to follow-up, and 85% had an undetectable plasma viral load. There were no significant demographic differences between participants who did and did not initiate ART by their follow-up visit (data not shown). Heavy alcohol use occurred in 12% of the ART-naïve and 8% of the ART-experienced cohort while drug use was substantially lower (2% in ART-niave and 1% of ART-experienced participants). Compared to baseline, participants who had initiated ART by their follow-up visit had higher body mass indexes (mean 21.8 ± 3.5 vs. 22.8 ± 3.5, p<0.001) and lower rates of depression (24% vs 8%, p<0.001). When comparing the baseline IHDS scores of participants on ART at follow-up to those lost to follow up (n=66), the score was higher in those who returned to follow-up (9.17 ± 2 vs 8.45 ± 2, p=0.008).

Table 1.

Demographic characteristics of study participants at baseline and follow-up.

	Baseline Visit [ART-naïve participants] (n=399)	Follow-Up Visit [ART-experienced participants] (n=312)	P*
Male Sex [n (%)]	212 (53%)	158 (51%)	0.62
Age (years) [mean (SD)]	35 (8)	37 (8)	0.002
Education (years) [mean (SD)]	5 (3)	6 (3)	0.22
CD4 count [mean (SD)]	257 (171)	412 (195)	<0.001
Plasma viral load [median (IQR)]	52525 (9080, 164040)	0 (0, 0)	< 0.001
Undetectable viral load [n (%)]	0 (0%)	46 (85%)	< 0.001
BMI [mean (SD)]	21.8 (3.5)	22.8 (4)	<0.001
Depression (CES-D Score ≥ 16) [n (%)]	96 (24%)	25 (8%)	<0.001
Heavy alcohol use+ [n (%)]	46 (12%)	25 (8%)	0.12
HAND Stage [n (%)]
Unimpaired	163 (41%)	150 (48%)	< 0.001
ANI	24 (6%)	42 (13%)
MND	152 (38%)	105 (34%)
HAD	60 (15%)	15 (5%)

^*Student’s t-tests were used to compare means, and chi-square tests were used to compare proportions.

⁺Heavy alcohol use was defined as ≥ 2 ‘yes’ answers to the following questions: Have you ever felt you need to cut down on your drinking? Have people annoyed you by criticizing your drinking? Have you ever felt guilty about your drinking? Have you ever felt you needed a drink first thing in the morning to steady your nerves or to get rid of a hangover?

Abbreviations: ANI: asymptomatic neurocognitive impairment; ART: antiretroviral; BMI: body mass index; CES-D: Center for Epidemiologic Studies Depression; HAD: HIV-associated dementia; HAND: HIV-associated neurocognitive disorder; MND: minor neurocognitive disorder; SD: standard deviation;

Diagnostic utility of the IHDS for identifying different levels of neurocognitive impairment in ART-naïve HIV+ patients is shown for three different IHDS score cutoffs (≤ 9, ≤ 9.5, and ≤ 10) in Table 2. An IHDS score ≤9 had the highest AUC at all levels of neurocognitive impairment but was highest for diagnosing HAD (AUC=0.673). The sensitivity, specificity, and negative predictive value of the IHDS were also highest for individuals with HAD with an IHDS cutoff of ≤ 9 demonstrating 82% sensitivity, 53% specificity and a negative predictive value of 94% for this group. However, this cutoff score correctly classified only 57% of participants compared to 61–63% of individuals with any HAND stage and symptomatic HAND. For each level of neurocognitive impairment, using a higher IHDS score cutoff maximized sensitivity but resulted in marked reductions in specificity.

Table 2.

Diagnostic utility of the International HIV Dementia Scale (IHDS) in antiretroviral therapy (ART)-naïve and ART-experienced individuals in rural Rakai, Uganda.

	Any HAND Stage vs. Uninimpaired*			Symptomatic vs. Asymptomatic+			HAD vs. No Dementia=
	IHDS ≤ 9	IHDS ≤ 9.5	IHDS ≤ 10	IHDS ≤ 9	IHDS ≤ 9.5	IHDS ≤ 10	IHDS ≤ 9	IHDS ≤ 9.5	IHDS ≤ 10
ART-Naïve Individuals (n=399)
Sensitivity	63%	67%	81%	65%	69%	83%	82%	82%	92%
Specificity	63%	55%	36%	62%	54%	36%	53%	46%	29%
Positive Predictive Value	71%	68%	65%	66%	63%	59%	23%	21%	19%
Negative Predictive Value	54%	54%	57%	61%	61%	64%	94%	93%	95%
Correctly Classified	63%	62%	63%	63%	62%	61%	57%	51%	39%
Positive Likelihood Ratio	1.7	1.5	1.3	1.7	1.5	1.3	1.7	1.5	1.3
Negative Likelihood Ratio	0.594	0.595	0.53	0.573	0.579	0.49	0.346	0.4	0.286
Area Under the ROC Curve	0.628	0.611	0.585	0.632	0.613	0.591	0.673	0.638	0.604
ART-Experienced Individuals (n=312)
Sensitivity	47%	50%	70%	51%	55%	75%	67%	64%	93%
Specificity	79%	76%	58%	75%	73%	55%	68%	65%	45%
Positive Predictive Value	70%	69%	64%	56%	56%	51%	11%	10%	9%
Negative Predictive Value	58%	58%	64%	71%	72%	78%	98%	97%	99%
Correctly Classified	62%	62%	64%	66%	66%	63%	67%	64%	48%
Positive Likelihood Ratio	2.18	2.07	1.7	2.05	2.05	1.7	2	1.8	1.7
Negative Likelihood Ratio	0.681	0.665	0.52	0.65	0.61	0.45	0.5	0.52	0.13
Area Under the ROC Curve	0.626	0.628	0.639	0.632	0.642	0.651	0.668	0.653	0.694

^*Any HAND stage (asymptomatic neurocognitive impairment (ANI), minor neurocognitive disorder (MND) and HIV-associated dementia (HAD)) versus unimpaired cognition.

⁺Symptomatic HAND (minor neurocognitive disorder (MND) and HIV-associated dementia (HAD)) versus asymptomatic HAND (asymptomatic neurocognitive impairment (ANI) and unimpaired cognition).

⁼HAD (HIV-associated dementia) versus no dementia (unimpaired cognition, asymptomatic neurocognitive impairment (ANI), and minor neurocognitive disorder (MND))

ROC: receive operating characteristic

In ART-experienced participants assessed at follow-up, an IHDS score ≤10 was shown to have the greatest AUC at all levels of neurocognitive impairment but was greatest for diagnosing HAD (AUC=0.694). For persons with HAD, a score of ≤10 had a sensitivity of 93% and negative predictive value of 99%. However, a score of ≤10 had the lowest specificity (46%) for detecting HAD and correctly classified the smallest proportion of participants (48%). The same threshold (i.e. score ≤10) correctly classified a higher proportion (63–64%) of ART-experienced individuals with less severe HAND stages (ANI and MND). When compared to less severe HAND categories, the sensitivity, specificity, and negative predictive value of IHDS ≤ 9 was also highest for detecting HAD,. However, this cutoff correctly classified only 57% of participants compared to 61–63% of individuals with any HAND stage and symptomatic HAND. Similarly to the ART-naïve participants, for each level of neurocognitive impairment, using a higher IHDS score cutoff maximized sensitivity but resulted in marked decreases in specificity.

DISCUSSION

In this study of the diagnostic utility of the IHDS in 399 HIV+ ART-naïve and 312 ART-experienced individuals in rural Uganda, the IHDS demonstrated moderate sensitivity as a screening tool and had the highest utility in ART-naïve compared to ART-experienced populations. It was most sensitive for detecting more severe stages of HAND (e.g. HAD) than for detecting milder HAND stages. In the HIV+ ART-naïve participants, the IHDS score of ≤10 provided much higher sensitivity for the less severe HAND stages (ANI 81%, MND 83%) compared to ART-experienced individuals (ANI 69%, MND 75%). However, an IHDS score of ≤10 had high sensitivity in both the ART-naïve and ART-experienced population for detecting HAD (92%, 94% respectively).

The IHDS was originally validated in the USA and Uganda in patients living in urban areas with 11.8–13.8 years of education.^[15] This study found that at a cut-off of ≤10, the sensitivity and specificity were 80% and 50% in the US and 80% and 55% in the Ugandan population. However, the original study only included ART-naïve participants. When comparing our patient population to the original cohort, it is notable that participants resided in rural areas and had markedly fewer years of education (5–6 years). However, despite these differences, the sensitivity and specificity of the IHDS in this cohort was approximately comparable to the original study with 81% sensitivity and 36% specificity in ART-naïve individuals and 70% sensitivity and 58% specificity in ART-experienced individuals. This suggests that the IHDS may have clinical utility to detect neurocognitive impairment in different populations despite varying levels of education and urbanization in sub-Saharan Africa.

Sensitivity is a measure of the proportion of individuals with a disease who are correctly identified by a given test. As a result, lowering the sensitivity of a test results in more ‘false negatives.’ For clinical purposes, a commonly acceptable threshold for screening tests is that it should have a low false negative rate with a sensitivity ≥ 80% to be considered reliable.^[24] Using an 80% threshold, for ART-naïve individuals, a score of ≤10 provides sufficient sensitivity to be used as a screening tool for all stages of HAND. In ART-experienced individuals a score of ≤10 showed 93% sensitivity in identifying HAD but only 75% and 70% sensitivity when detecting MND and ANI. Findings in Brazil, Cape Town, rural Southeastern China and Berlin have demonstrated the utility of the IHDS in detecting HAD or the spectrum of HAND diseases, with a sensitivity for detecting any HAND stage ranging from 73.7–94% and a specificity from 24.7–69.7%.^[16–18] However, none of these assessments included the IHDS’ efficacy in detecting a specific diagnosis within HAND, such as ANI and MND.

Specificity is a measure of the proportion of people without a disease who a test correctly identify as being disease-free. In other words, the lower the specificity of a test, the higher the number of ‘false positive’ results. The major limitation of the IHDS in this cohort was its specificity, which was better amongst ART-experienced participants (ranged from 45% to 79% depending on the HAND stage and score cutoff being investigated) than ART-naïve participants (range: 29–63%). If implemented on a large scale, these low- to moderate specificities would result in relatively large numbers of individuals being referred for further diagnostic investigation which could overwhelm an already resource-limited setting. As such, the IHDS would likely be most effective in a situation in which its high sensitivity could be utilized to identify most people with HAND while the effect of needing to formally evaluate many unimpaired individuals for HAND would not have significant negative consequences.

The IHDS has been noted in recent studies to demonstrate even poorer specificity. A prospective cohort study of IHDS utility conducted in Kenya, Tanzania and Uganda reported a 91% sensitivity but 17% specificity for detecting any HAND stage which would result in 1.18 false positive cases for every correctly identified case.^[25] The authors concluded implementation of the IHDS would lead to excess resources being utilized in an already resource-limited setting. This study was restricted to HIV+ individuals on ART and the IHDS was used for detecting neurocognitive impairment (i.e. all stages of HAND) as well as a more severe definition of neurocognitive impairment. Functional outcomes were not included in this study to define stages of HAND, so the outcomes between this study and our study are not directly comparable. These differences could at least partially account for the discrepant results between this study and our current study. However, further research to identify which population-level factors are driving differing IHDS utility between cohorts would also be useful in determining appropriate target populations for effective implementation.

Study strengths include the large sample size, use of locally developed normative data for the neuropsychological tests, the use of functional assessments to define HAND, and data on the longitudinal performance of the IHDS from pre-ART to after ART initiation. However, there were several limitations, including our inability to rule out other causes of neurocognitive impairment unrelated to HIV infection such as opportunistic infections, perinatal injury, alcohol use and cerebrovascular disease. Many of these conditions would require brain imaging to confirm, and this was unavailable in our setting. Lacking access to such imaging, we may have overestimated the proportion of participants with HAND. In addition, 86 participants (22% of the total participants) were lost to follow-up at the two-year visit, and they were more likely to have lower baseline IHDS scores than those who returned for follow-up. This could lead to biased results when comparing ART-naïve to the ART-experienced group, especially if the neurocognitive status of those who did not return was a factor in their lack of follow up. The neurocognitive battery we employed in this study, while validated in prior studies in Uganda, was relatively brief and may have missed mild deficits that would have been detected by a more extensive battery. In addition, it was skewed towards detection of motor deficits which may be less prominent in ART-experienced individuals. These factors may have resulted in lower rates of overall HAND and potentially milder HAND stages than a more extensive battery with less weight on motor function. If this is the case, this ultimately would have negatively impacted the sensitivity and specificity of the IHDS in this cohort.

In conclusion, the IHDS is a potentially useful screening tool for neurocognitive impairment in rural Uganda for both ART-naïve and ART-experienced adults. It demonstrates higher sensitivity for detecting more severe than less severe HAND stages in both ART-naïve and ART-experienced HIV+ adults. It is more sensitive for less severe HAND stages in ART-naïve than ART-experienced patients. Tthe IHDS generally demonstrated adequate sensitivity for all groups studied, so it would be useful as a screening tool in that it would result in few false positives. However, its low specificity in many subgroups may limit widespread use due to high rates of false positive results which would result in many individuals with normal cognition being referred for more extensive evaluations. Future studies could focus on potential modifications to the IHDS to improve its specificity or understanding population-level characteristics that drive its variable utility in different populations.