Abstract
Objective
To conduct a preliminary evaluation of the utility and reliability of a diagnostic tool for HIV-associated dementia (HAD) for use by primary health care workers (HCW) which would be feasible to implement in resource-limited settings.
Background
In resource-limited settings, HAD is an indication for anti-retroviral therapy regardless of CD4 T-cell count. Anti-retroviral therapy, the treatment for HAD, is now increasingly available in resource-limited settings. Nonetheless, HAD remains under-diagnosed likely because of limited clinical expertise and availability of diagnostic tests. Thus, a simple diagnostic tool which is practical to implement in resource-limited settings is an urgent need.
Methods
A convenience sample of 30 HIV-infected outpatients was enrolled in Western Kenya. We assessed the sensitivity and specificity of a diagnostic tool for HAD as administered by a primary HCW. This was compared to an expert clinical assessment which included examination by a physician, neuropsychological testing, and in selected cases, brain imaging. Agreement between HCW and an expert examiner on certain tool components was measured using Kappa statistic.
Results
The sample was 57% male, mean age was 38.6 years, mean CD4 T-cell count was 323 cells/µL, and 54% had less than a secondary school education. Six (20%) of the subjects were diagnosed with HAD by expert clinical assessment. The diagnostic tool was 63% sensitive and 67% specific for HAD. Agreement between HCW and expert examiners was poor for many individual items of the diagnostic tool (K = .03–.65). This diagnostic tool had moderate sensitivity and specificity for HAD. However, reliability was poor, suggesting that substantial training and formal evaluations of training adequacy will be critical to enable HCW to reliably administer a brief diagnostic tool for HAD.
Introduction
HIV-associated dementia (HAD) is an indication for initiating antiretroviral therapy regardless of CD4 T-cell count according to World Health Organization (WHO) guidelines (a WHO Stage IV diagnosis [1]. However, HAD is likely under-diagnosed in routine clinical practice in resource-limited settings [2]. Furthermore, HAD is still a clinically important disorder in resource-limited settings where many individuals present with advanced HIV disease. In research studies from sub-Saharan Africa, the prevalence of HAD ranges from 2.5%–54% [3]–[10]; these estimates vary widely likely due to differences in the sampled populations and methods for assessment of cognitive impairment. In contrast, the incidence of HAD in more developed regions has decreased dramatically since ART became widely available [11]–[14].
Focusing on the diagnosis of HAD as opposed to milder forms of HIV-associated neurocognitive disorders (HAND) is critical in settings where decisions about whether to initiate ART are frequently made based on WHO criteria alone and where ART treatment is available only to those with the greatest need—individuals with very low CD4 T-cell counts and/or WHO Stage III and IV diagnoses. Since HAD typically improves with ART [15]–[17], and availability of ART is increasing even in resource-limited settings, identification of individuals with HAD can be of great importance to improve health outcomes. However, the diagnosis of HAD remains a challenge in HIV outpatient primary care settings in resource-limited regions [2]. Potential reasons for this include: a lack of specialized personnel and diagnostic tests, and the inherent difficulties in making a clinical diagnosis of a complex disorder.
Several brief screening tests, like the International HIV Dementia Scale (IHDS), were developed to identify individuals with HAND in resource-limited settings. The IHDS has been demonstrated to be useful in detecting HAND in settings where screening is conducted by trained physicians and referral to a specialist is an option [9]. However, specialized personnel are rare in low income countries. In Kenya the median number of neurologists per 100,000 population is 0.03 as compared to 2.96 in high-income countries [18]; similarly, there are only 0.14 physicians per 1,000 population in Kenya as compared to the United States where the ratio is 2.56 [19].
In addition to highly specialized clinical expertise, in developed regions, the diagnosis of HAD often entails imaging of the brain, lumbar puncture, and in milder cases, neuropsychological testing. Diagnostic tests such as Computed Tomography (CT) of the head are widely available but unaffordable in Kenya, while magnetic resonance imaging (MRI) is even less affordable and only available in the capital city, Nairobi [20]. There are only two neuropsychologists in Kenya (both in private practice) [20]. Thus, neuropsychological testing is essentially unavailable, and, while some culturally specific normative data has recently become available for other sub-Saharan African populations, no published normative data exists for Kenyans [21], [22].
In summary, in resource-limited settings, non-physician health care workers (HCW) are the first and often the only health care provider available for a majority of HIV-infected individuals [23] and diagnostic tests are very limited. Therefore, our goal was to develop a simple diagnostic tool to identify individuals with HAD—individuals who should be started on ART regardless of their CD4 T-cell count. Furthermore, we felt it was important to develop a tool which would be feasible to implement in decentralized resource-limited settings. The diagnostic tool should not require referral to a specialist or additional diagnostic tests, should only require minimal additional training, and should be reliably administered by primary HCW. We created a diagnostic tool designed to guide a HCW through a clinical assessment for HAD; the tool includes a review of cognitive symptoms, a neurological examination to screen out individuals with focal neurological findings, cognitive tests to identify individuals with impairment, and an assessment of functional status. Here we report on challenges and lessons learned during pilot testing of this diagnostic tool for HAD designed for use by non-physician primary HCW in Nyanza province, Western Kenya.
Methods
Objectives
The objective of this study was to conduct a preliminary evaluation of the utility and reliability of a diagnostic tool for HAD designed for use by non-physician primary HCW.
Participants
A convenience sample of 30 adults was recruited from an outpatient HIV care and treatment program run by Family AIDS Care and Education Services (FACES) in Kisumu, a city in Western Kenya, between September and October 2009. Potential participants were invited to participate when they presented for clinical care. One participant who had time to participate in the study (study activities took about 3 hours) was chosen each day by clinic staff and referred to study staff. In addition, patients with neurological findings were oversampled in order to assess the neurological examination portion of the tool—we enrolled any individual with neurological findings referred to us by study staff. Participants who could not speak English, Dholuo, or Kiswahili well enough to participate in the neuropsychological testing were excluded from the study.
Procedures
A diagnostic tool was administered by a HCW. The results of this tool were compared to an expert clinical assessment performed by study staff which included: examination by a physician, a neuropsychological test battery administered by expert study staff, and computed tomography of the head when clinically indicated. The results of the expert clinical assessment were used to identify patients meeting a modification of the Frascati criteria for HAD [24]. The diagnostic tool was also administered by trained staff during the expert assessment to assess reliability.
Diagnostic tool (administered by HCW)
A non-physician HCW administered the diagnostic tool (Appendix S1) which consisted of a brief history and neurological examination, cognitive assessment based on the International HIV Dementia Scale (IHDS) [9] and assessment of functional status. The HCW were blinded to the results of the expert clinical assessment performed by study staff. To assess reliability, all components of the cognitive testing portion of the diagnostic tool (C4–C8) were also administered by expert study staff. Since the tool was administered by HCW during the course of their usual clinical duties, the timing of tool administration was planned in such a way as to minimize impact to clinic flow. The expert clinical assessment was conducted after the HCW administered the diagnostic tool in 19/30 cases.
The eighteen non-physician HCW who participated in this study were either clinical officers or nurses on the staff of FACES. Clinical officers receive three to four years of post-secondary school education and are comparable to physician's assistants in the United States. Nurses receive two to four years of post-secondary school education. Since we sought to explore the utility of this diagnostic tool in a setting we felt mimicked “real-life” practice settings for primary HCW, we did not provide additional training specific to this tool to participating HCW. All the HCW who participated in the study have received HIV-specific training and ongoing mentorship through FACES. At the time of the study, one investigator (ACM) had been providing training and mentorship in HIV Neurology for six months at the study site and had conducted several trainings on the neurological examination to clinicians and nurses at our study site using a format quite similar to the neurological examination contained in the diagnostic tool. Most of the tests included in the cognitive assessment are part of the Kenyan National Guidelines, and in theory, all HCW had received training on their administration.
Expert clinical assessment (administered by study team)
Each participant underwent a standardized clinical examination, a neuropsychological test battery, and computed tomography of the head when clinically indicated, and classification of dementia.
Examination
Each participant underwent a standardized examination which included: a detailed history; full medical examination; subjective and objective examination of mental state and cognition using the Patients Assessment of Own Functioning Inventory (PAOFI) [25] and the Mini-Mental State Examination [26]; detailed examination of all cranial nerves except the olfactory nerve; fundoscopy; motor examination and strength testing in 20 different muscle groups; examination of light touch, pinprick and temperature at 14 predefined areas, as well as assessment of distal joint position and vibration sense; six predefined tests of coordination and motor function; assessments of five deep tendon reflexes, the plantar reflex, four frontal release reflexes; gait exam with seven specific observations, and Romberg test. The two non-neurologists (DC and JK) who performed the neurologic examinations underwent extensive training including numerous full observed examinations by the principal investigator (AM). Computerized tomography (CT) of the brain was performed in four participants with likely HAD or focal findings on neurological exam to rule out a mass lesion, infarct, or other process which might mimic the clinical and neuropsychological findings of HAD. Several measures of functional status were used including the PAOFI, Activities of Daily Living [27], Instrumental Activities of Daily Living [28], and the Physical Function, Role Function, and Role limitations due to Physical Health of the Medical Outcomes Study [29]. Most of these have not been formally tested in a Kenyan population, so the answers to all these questions were integrated by the physician or neurologist in assigning a Karnovsky Performance Scale score [30] intended to represent limitations in function due only to cognitive impairment.
Neuropsychological test battery
The neuropsychological test battery was administered by expert study staff and assessed attention/working memory (Digit Span [31]), category fluency (Animals, First Names [32]) executive function (Block Design [31], Color Trails 2 [33]), speed of information processing (Digit Symbol [31], Color Trails 1 [33]), verbal and visual learning and memory (WHO/UCLA Auditory Verbal Learning Test [AVLT] [32], [33] and Revised Brief Visuospatial Memory Test [BVMT] [34]), and psychomotor speed (Grooved Pegboard [35], Finger Tapper [36], Timed Gait [37]). In addition, two established screening tests, the MMSE [26] and the International HIV Dementia Scale (IHDS) [9] were administered. As described previously, the MMSE was administered by the physician as part of the examination. Neuropsychological testing was performed in the patient's preferred language (either English, Kiswahili, or Dholuo). The study staff member administering neuropsychological tests was fluent in the preferred language.
As no published normative data was available from Kenyan or East African populations, normative values from general United States populations were used to convert raw scores to standardized scores (Z-scores) [31], [34]–[40]. The exception was for category fluency in which normative data from an African American U.S. population was used as data from a general population was not available [38]. Z-scores were calculated so that negative Z scores always represented worse performance on a test. A composite standardized score was generated by averaging the mean standardized score for each individual test.
Classification of Dementia
The Frascati criteria for HAD were modified slightly and implemented as follows [24]: for an individual to be classified with HAD, they had to have (1) neuropsychological test scores >2 standard deviations (SD) below the mean in at least two different cognitive domains, and (2) major functional decline due to cognitive impairment in the opinion of the examining physician. We omitted a second criterion of impairment on neuropsychological tests (a score >1 SD mean in one cognitive domain and a score of >2.5 SD below the mean in another) because we did not have a culturally appropriate normative sample and nearly all of our participants scored >1SD below the mean in at least one cognitive domain. For an assessment of the performance of neuropsychological tests we divided our sample into two additional categories. Individuals who did not meet the above criteria for abnormal neuropsychological tests were classified as normal. The remaining individuals who met the neuropsychological test criteria for cognitive impairment as described above did not have no major impairments in functional status were grouped and classified as Asymptomatic Neurocognitive Impairment (ANI)/Mild Neurocognitive Disorder (MND). Subjects were classified by a physician and a neurologist and final diagnoses were decided by consensus. The physician and neurologist were blinded to the results of the diagnostic tool administered by the HCW.
Analytic Approach and Statistical Methods
Descriptive statistics were generated for demographic and clinical characteristics. Student's T-tests were used to compare means and Fisher's exact test to compare proportions between individuals with and without HAD. We also compared raw and standardized scores on neuropsychological tests across categories of HAND and between individuals with and without moderate to severe depression. Sensitivity, specificity, receiver operator characteristic (ROC) curves were generated for the diagnostic tool. Quadratic-weighted kappa statistics were generated for the overall score and individual items on the cognitive portion of the diagnostic tool (Section C4–C8) to compare agreement between the HCW and expert study staff.
Statistical analyses were performed using Stata10.0 (StataCorp, College Station, Texas). A significance level of p≤.05 was used.
Ethics Statement
The study protocol was approved by the University of California San Francisco (UCSF) Committee on Human Research and the National Ethical Review Committee (ERC) of the Kenya Medical Research Institute (KEMRI). Written informed consent was obtained from each participant.
Results
Expert Clinical Assessment
The mean age of the sample was 39 years and 57% were male. The mean CD4 T-cell count was 323 cells/µL, 60% had WHO Stage 3 or 4 disease, and 50% had less than a secondary school education (Table 1). Nearly 23% (7/30) of participants had moderate or severe depression using the Patient Assessment of Health Questionnaire (PHQ9) [41]. In general, functional status measures demonstrated mild impairments. There were no significant differences in demographic, clinical, functional status or psychiatric characteristics between individuals with and without HAD though this should be interpreted with caution given small sample sizes.
Table 1. Baseline characteristics of study participants comparing individuals with and without HIV-associated dementia.
| Overall (n = 30) | No dementia (n = 24) | HIV-associated dementia (n = 6) | |
| Demographic Characteristics | |||
| Age [mean (SD)] | 39 (10) | 47 (6.8) | 37 (10) |
| Male [% (n)] | 57% (17) | 50% (12) | 83% (5) |
| Marital Status [% (n)] | |||
| Never Married | 13% (4) | 17% (4) | 0% |
| Married | 63% (19) | 54% (13) | 83% (5) |
| Widowed | 20% (6) | 21% (5) | 17% (1) |
| Divorced/separated | 4% (1) | 4% (1) | 0% |
| Highest level of education achieved [% (n)] | |||
| Less than primary | 23% (7) | 21% (5) | 33% (2) |
| At least primary | 27% (8) | 29% (7) | 17% (1) |
| Secondary | 37% (11) | 42% (10) | 17% (1) |
| More than secondary | 14% (4) | 12% (2) | 33% (2) |
| Socio-economic status | |||
| Electricity in dwelling [% (n)] | 37% (11) | 38% (9) | 33% (2) |
| Running water in dwelling [% (n)] | 13% (4) | 13% (3) | 17% (1) |
| Selected clinical characteristics | |||
| WHO Stage currently [% (n)] | |||
| Stage 1 | 13% (4) | 13% (3) | 17% (1) |
| Stage 2 | 27% (8) | 29% (7) | 17% (1) |
| Stage 3 | 37% (11) | 33% (8) | 50% (3) |
| Stage 4 | 23% (7) | 25% (6) | 17 (1) |
| Nadir CD4 cell count [mean (SD)] | 198 (150) | 215 (157) | 127 (103) |
| Current CD4 cell count [mean (SD)] | 323 (211) | 337 (227) | 268 (134) |
| Moderate to severe depression [%(n)] * | 23% (7) | 21% (5) | 33% (2) |
| Functional Status Measures | |||
| Karnovsky Performance Scale [mean (SD)]Range (0–100) | 92 (20) | 92 (22) | 91 (4.1) |
| Activities of Daily Living (ADL)[mean (SD)] Range (0–4) | 4 (0) | 4 (0) | 4 (0) |
| Independent ADL [mean (SD)] Range (0–8) | 7.8 (0.7) | 7.8 (0.7) | 8 (0) |
| Physical Function[mean (SD)] Range (0–100) | 89(19) | 97 (4) | 87 (201) |
| Role Limitation due to Physical Function [mean (SD)] Range (0–100) | 82 (5.1) | 80 (31) | 89 (16) |
| Patient assessment of own functioning inventory (PAOFI)† [mean (SD)] Range (0–165) | 12 (14) | 13 (15) | 12 (5.9) |
Score ≥10 on Patient Health Questionnaire (PHQ9);
PAOFI- a score of 0 reflects no impairment.
Among our pilot sample, 20% (6/30) were diagnosed with HAD based on the expert clinical assessment. An additional 33% (10/30) were classified as normal. The remaining 47% (14/30) individuals were grouped and classified as mild neurocognitive impairment. Among the individuals classified as normal, there were five individuals with abnormal neurological examinations potentially localizing to the central nervous system. Findings included: (1) unsteady gait and asymmetric ataxia (left worse than right); (2) multiple cranial neuropathies (III, IV, V, VI, VII, IX, X) with subtle left sided weakness and left cerebellar signs; (3) parkinsonism; (4) a large left retro-orbital mass, and (5) a history of right-sided weakness with no objective findings on examination. Of these individuals, three individuals were referred for computed tomography (CT) of the head. The CT findings were as follows: (1) Marked cerebellar volume loss including the middle cerebellar peduncle and pons but sparing the medullary pyramids; (2) periventricular hypothalamic lesion and marked edema, likely CNS lymphoma, (3) diffuse volume loss and small caudate heads. The individual with a left retro-orbital mass had a CT scan previously that demonstrated a sizeable mass in the retro-orbital space. A fourth individual who had markedly poor performance on neuropsychological tests and who was ultimately classified as having HAD, also underwent a CT scan which was normal.
Neuropsychological tests and HAND
We then reviewed the performance of our neuropsychological test battery, excluding the four individuals listed above with abnormal neurological examinations. As the level of cognitive impairment increased, mean raw scores worsened on nearly all of the individual neuropsychological tests chosen for this test battery (Table 2). Similarly, individuals with higher levels of cognitive impairment had worse performance on the MMSE and IHDS. The one exception was Category Fluency- First Names where mean raw scores among individuals with mild neurocognitive impairment were higher than scores among normal individuals. Mean standardized scores also worsened on all tests except Category Fluency-First Names as the level of cognitive impairment increased (Figure 1). The composite score, an average of the standardized scores for all the tests, decreased from -0.6 [SD: 0.76] among normal individuals to -2.16 [SD: 1.63] among individuals with HAD. The greatest magnitude differences on standardized scores between normal individuals and those with HAD were on the tests of Motor Skills and Color Trails 1. Of note, among individuals classified as normal, the mean standardized score was ≤−1 in Block Design, Digit Symbol, and all the tests of Motor Skills.
Table 2. Comparison of mean raw scores on individual tests comprising neuropsychological test battery by level of cognitive impairment among HIV-infected individuals.* .
| Normal | ANI, MND | HAD | ||||
| (n = 6) | (n = 14) | (n = 6) | ||||
| Mean | SD | Mean | SD | Mean | SD | |
| Category Fluency | ||||||
| Animals | 20.8 | (5.5) | 16.6 | (3.8) | 14.8 | (4.1) |
| First Names | 23 | (4.3) | 23.1 | (5.4) | 19.3 | (5.5) |
| Executive Function | ||||||
| Color Trails 2† | 118.8 | (28.9) | 159.2 | (46.5) | 169.5 | (42.9) |
| Block Design | 16.2 | (7.3) | 11.8 | (2.5) | 7.3 | (3.3) |
| Speed of Information Processing | ||||||
| Digit Symbol | 51.8 | (12) | 34.5 | (10.9) | 23 | (14.0) |
| Color Trails 1† | 51.2 | (9.3) | 76.3 | (29.0) | 99.3 | (28.9) |
| Attention/Working Memory | ||||||
| Digit Span | 13.8 | (2.1) | 12.2 | (3.2) | 11.2 | (4.7) |
| Visual and Verbal Learning | ||||||
| AVLT Total Score | 52.8 | (3.2) | 45.1 | (8.0) | 41.3 | (6.7) |
| AVLT Trial V | 12.5 | (1.4) | 11.1 | (2.0) | 9.3 | (2.7) |
| BVMT Total Recall | 21.8 | (4.8) | 17.4 | (8.2) | 13.2 | (6.9) |
| Visual and Verbal Memory | ||||||
| AVLT Delayed Recall | 12.2 | (1.7) | 9.4 | (2.9) | 7.7 | (2.3) |
| BVMT Delayed Recall | 8.8 | (1.8) | 6.3 | (3.6) | 5.7 | (2.9) |
| Motor Skills | ||||||
| Finger Tapper (D) | 44.7 | (5.7) | 38.3 | (5.7) | 34.4 | (10.1) |
| Finger Tapper (ND) | 38.2 | (6.4) | 33.8 | (4.1) | 33.5 | (7.7) |
| Grooved Pegboard (D)† | 71.2 | (10.3) | 91.7 | (25.6) | 111.3 | (27.8) |
| Grooved Pegboard (ND)† | 82.5 | (12.3) | 93.9 | (14.5) | 132.5 | (30.2) |
| Timed Gait† | 11 | (1.5) | 13.3 | (2.3) | 13.5 | (2.8) |
| Batteries | ||||||
| International HIV Dementia Scale | 11 | (1.7) | 9.9 | (1.9) | 8.5 | (1.8) |
| Mini-Mental State Examination | 27.5 | (1.9) | 27.1 | (2,1) | 26.5 | (2.2) |
Four individuals with non-HIV related neurologic diagnoses that affected performance on neuropsychological tests were dropped from this analysis.
Higher scores indicate worse performance. Abbreviations: ANI: Asymptomatic Neurocognitive Impairment; D = Dominant; HAD: HIV-associated dementia; MND: Mild Neurocognitive Disorder; ND = Non-dominant.
Figure 1. Comparison of mean standardized scores on individual neuropsychological tests comprising neuropsychological test battery and a composite measure by level of cognitive impairment among HIV-infected individuals.
Neuropsychological tests and depression
Individuals with moderate to severe depression on the PHQ-9 instrument were not more likely to have a score of ≤22 on our novel diagnostic tool using Fisher's exact (.39). They were more likely to have a score of ≤10 on the IHDS using Fisher's exact (.02). In general, individuals with moderate to severe depression had significantly lower mean standardized scores on all the neuropsychological tests from the Psychomotor domain (Grooved Pegboard (Dom/ND), Finger Tapper (Dom/ND), Timed Gait) as well as the verbal learning and memory domain (AVLT Total Score, AVLT Score Trial V, AVLT DR) (data not shown). The remainder of the tests did not demonstrate significant differences in mean standardized scores between with individuals with and without moderate to severe depression: Category fluency (animals and names), Color Trails 1 & 2, Block Design, Digit Symbol, Digit Span, BVMT, BVMT-DR (data not shown).
Novel Diagnostic tool and HAD
Among this sample, the diagnostic tool determined that 53% (16/30) of our sample had HAD. An ROC curve was generated to assess the utility of the diagnostic tool in identifying individuals with HAD. A cut-off score of ≤22 (maximum = 26) resulted in a sensitivity of 63%, specificity of 67% and accuracy of 63%. Kappa statistics were used to evaluate agreement between the neuropsychological tests administered by a HCW as compared to expert study staff in Section C (cognitive portion) of the diagnostic tool. An expert examiner administered the cognitive components of the diagnostic tool after the HCW in 19/30 cases; in the remaining 11 cases, the expert examiner administered the diagnostic tool first. As shown in Table 3, moderate agreement was observed for C6: Serial Operations and C7: Category Fluency. Fair agreement was observed for C8: Recall and the total neuropsychological test score (C score). There was poor agreement for C4: Finger tapping and C5: Luria sequence. Despite poor agreement on 2 of 3 items taken from the IHDS, there was fair agreement on the overall score of the IHDS. Similarly there was poor agreement between Section B (neurological examination portion) of the diagnostic tool and the neurological examination conducted by the study physician; four individuals found to have an abnormal neurological exam by the study physician were not similarly classified when examined by the HCW administering the diagnostic tool.
Table 3. Agreement between non-physician health care worker and a expert examiner on cognitive tests in the diagnostic tool.
| Neuropsychological test Items from diagnostic tool (Section C) | Quadratic weighted Kappa |
| C4: Finger Tapping | 0.03 |
| C5: Luria sequence | 0.05 |
| C6: Serial Operations | 0.47 |
| C7: Category fluency (animals) | 0.65 |
| C8: Recall | 0.25 |
| C score (Sum of C1–C8) | 0.32 |
| International HIV Dementia Scale total score (Sum of C4, C5,C8) | 0.26 |
Discussion
Expert Clinical Assessment
This convenience sample of individuals presenting for routine clinical care in Kenya was comparable to larger community-based samples from sub-Saharan Africa [42]–[44]. In addition, during the month that we were recruiting for this study, we encountered several individuals with severe neurological disorders—highlighting the importance of improving neurological training for primary HIV healthcare workers in resource-limited settings.
Neuropsychological Test Battery and HAND
Mean raw and standardized scores (Z-scores) on many tests in our neuropsychological test battery demonstrated large magnitude differences between individuals with and without HAND. Our results demonstrated the typical pattern of HIV associated cognitive impairment with one notable exception. Our results were similar to prior studies in that we observed large differences in scores in the domains of executive function (Block Design, Color Trails 2,) speed of information processing (Digit Symbol, Color Trails), and motor/psychomotor function (Grooved Pegboard, Finger Tapper, and Timed Gait) [45]–[49].
In contrast to previous studies, tests of motor or psychomotor performance in the non-dominant hand did not demonstrate larger magnitude differences in mean raw or standardized scores and thus may not be more sensitive and reliable in diagnosing HAND [10], [22]. This may be related to the small sample size or cultural differences in interpreting test instructions. For example, we noted that many study subjects did not work as quickly as they could after being given the instruction “as fast as you can” and that additional explanation was necessary to elicit maximal effort on timed tests. These differences warrant further exploration with a larger sample size and culturally specific normative data before any conclusions can be drawn.
We found large differences in mean raw and standardized scores for AVLT and BVMT-R among individuals with and without HAND although difficulty with verbal learning and memory are thought to be less characteristic of HAND [50]. These findings were comparable to those reported from a Ugandan study which used AVLT as part of their test battery [22]. Whether this represents a different pattern of cognitive deficits unique to sub-Saharan Africa, is related to literacy levels, or is a function of overlapping cognitive domains (e.g. low scores on AVLT could be caused by poor attention) remains to be seen.
One important finding is that among individuals classified as normal, performance on several tests was substantially lower than our U.S. normative population. Mean standardized scores on Block Design, Digit Symbol, and all the tests of motor skills were less than or equal to one standard deviation below the normative mean. Strictly applying the AAN Frascati criteria [24] could result in a diagnosis of mild cognitive impairment at this level of performance, again highlighting the need for a culturally appropriate normative sample.
Novel Diagnostic Tool and HAD
As administered in this pilot sample, the novel diagnostic tool had lower sensitivity and specificity than studies using similar tools—the IHDS in Uganda [9] and a culturally adapted version of the MMSE in Zambia [3]. This study differed from those studies in several important ways. First, in this study, a non-physician HCW administered the diagnostic tool without additional training while in the Ugandan study, a highly trained physician administered the IHDS [9]. Second, we introduced a novel assessment tool while in the Zambian study, the Zambian version of the MMSE was familiar to the primary HCW from their clinical training [3]. In the same study, the HIV Dementia Scale (HDS) [51] was also administered by trained HCW. Even with training, frequent errors were reported in the test which may have been due in part to poor administration by HCW [42].
As described in the methods section, the HCW participating in this study had received some general training in the neurological examination and in HIV care according to Kenya National Guidelines which includes many of the cognitive tests contained in the diagnostic tool. In this study, we chose not to provide additional training specific to the tool for two reasons: (1) to assess training needs for this tool, and (2) to assess the feasibility of implementation in a decentralized health care system where most care is provided by non-physician HCW and wide-scale comprehensive training is not feasible. However, the lack of specific training likely contributed to the poor sensitivity of the tool. For example, there was poor agreement between the non-physician HCW and the study physician on the neurological examination. In addition there was poor agreement on many of the cognitive items comprising the diagnostic tool between HCW and expert study staff.
These findings suggest that substantial training and assessments of inter-rater agreement between physicians and non-physician HCW will be critical prior to the use of this diagnostic tool in routine clinical settings. Providing ongoing training is a significant challenge in resource limited settings where staff turn-over is high. Alternatively, choosing an assessment tool, or components of an assessment tool, that are simpler to administer or more familiar to primary HCW may improve diagnostic utility in routine clinical settings. To maximize the applicability of HAD screening tools in a decentralized system one may have to trade diagnostic sophistication and precision—measuring the cognitive domains most affected in HAD—to develop a tool that is easy to teach and easy to use.
Another potential reason for the lower utility of the diagnostic tool is inclusion of individuals with co-morbid depression who may have been misclassified as having HAD because of poor performance on cognitive assessments [52]. Furthermore, the symptom (Section C, items 1 and 2) and functional status questions (Section D) on the tool showed little variation and likely added little to the ability of the tool to discriminate between those with and without HAD (data not shown). Finally, although the patient populations were demographically similar, cultural differences between our study site in Kenya and those in Zambia and Uganda may have contributed to the lower utility in our population.
Strengths and Limitations
A strength of our study is that it was performed in an outpatient HIV care setting typical of others in sub-Saharan Africa. In addition, the diagnostic tool was administered by non-physician HCW which improves the generalizability of our results. However, this study was limited by the small sample size and the lack of training for non-physician HCW administering the tool. Also, the neuropsychological test instructions were not standardized but were interpreted by study staff fluent in those languages. In addition, the expert clinical assessment by study staff did not include a lumbar puncture to evaluate for other infections of the central nervous system or MRI to look for other focal lesions of the brain. In place of these procedures, which are not available in our setting, we performed a detailed medical and neurological examination. We also performed CT of the head in individuals with HAD or an abnormal neurological examination that localized to the central nervous system.
It is important to note that a major limitation of this study was the lack of Kenyan normative data. Since performance on many neuropsychological tests is sensitive to cultural, linguistic, educational, gender and age differences, it is not known if (or to what extent) the tests scores of an HIV uninfected Kenyan normal population would differ from this HIV infected population. Our study demonstrates that many normal individuals receive scores in the impaired range on several of the neuropsychological tests, suggesting that normative data from the U.S. is not appropriate, and highlighting the need for culturally appropriate normative data.
Conclusion
Pilot testing suggests that substantial training and formal evaluations of training adequacy will be critical to enable primary HCW to reliably administer a brief diagnostic tool for HAD that contains cognitive tests and a neurological examination. Alternatively, selecting the simpler or more familiar components from more sophisticated screening or diagnostic tools may improve the diagnostic utility in routine clinical settings in a decentralized system of care. Finally, normal individuals frequently have standardized scores in the impaired range using our neuropsychological test battery. Thus, culturally specific normative values for neuropsychological tests are critical prior to widespread clinical or research use for the diagnosis of HIV-associated neurocognitive disorders.
Supporting Information
HIV Dementia Diagnostic Tool.
(DOC)
Acknowledgments
We would like to thank the medical officers, clinical officers, nurses, clinic/community health assistants, staff and patients at Family AIDS Care and Education Services for their participation. We would also like to thank the Director of the Centre for Microbiological Research, the Director of Kenya Medical Research Institute (KEMRI), and the Ministry of Public Health and Sanitation, Nyanza Province, Government of Kenya, for their support. In addition we would like to thank Dr Prabhakar Reddi (Aga Khan Hospital Kisumu) and Alisa D. Gean, Professor of Radiology and Biomedical Imagaing, University of California, San Francisco, for their assistance interpreting computed tomography scans of the head. Finally, we would like to thank the anonymous reviewers for their insightful comments which greatly improved this manuscript. This manuscript is submitted with permission to publish from the Director KEMRI.
Footnotes
Competing Interests: The authors have declared that no competing interests exist.
Funding: This study was supported by the Fulbright African Regional Research Program, American Academy of Neurology Foundation Practice Research Training Fellowship, and Doris Duke International Clinical Research Fellowship. In addition this study was supported by the Fogarty International Clinical Research Fellowship (5 R24 TW00798; 3 R24 TW00798-02S1) from the National Institutes of Health, Fogarty International Center through Vanderbilt University, the National Cancer Institute, the National Institute on Drug Abuse, Office of the Director, National Institutes of Health and the National Institute of Mental Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
References
- 1.World Health Organization. WHO Case definitions of HIV for surveillance and revised clinical staging and immunological classification of HIV-related disease in adults and children. Geneva 2007 [Google Scholar]
- 2.Meyer A. 2009. Prevalence of HIV associated encephalopathy using clinical database: Unpublished data.
- 3.Kvalsund M, Haworth A, Murman D, Velie E, Birbeck G. Short Report: Closing Gaps in Antiretroviral Therapy Access: Human Immunodeficiency Virus-Associated Dementia Screening Instruments for Non-Physician Healthcare Workers. Am J Trop Med Hyg. 2009;80:1054–1059. [PubMed] [Google Scholar]
- 4.Maj M, Satz P, Janssen R, Zaudig M, Starace F, et al. WHO Neuropsychiatric AIDS study, cross-sectional phase II. Neuropsychological and neurological findings. Arch Gen Psychiatry. 1994;51:51–61. doi: 10.1001/archpsyc.1994.03950010051007. [DOI] [PubMed] [Google Scholar]
- 5.Njamnshi AK, Djientcheu Vde P, Fonsah JY, Yepnjio FN, Njamnshi DM, et al. The International HIV Dementia Scale is a useful screening tool for HIV-associated dementia/cognitive impairment in HIV-infected adults in Yaounde-Cameroon. J Acquir Immune Defic Syndr. 2008;49:393–397. doi: 10.1097/qai.0b013e318183a9df. [DOI] [PubMed] [Google Scholar]
- 6.Howlett WP, Nkya WM, Mmuni KA, Missalek WR. Neurological disorders in AIDS and HIV disease in the northern zone of Tanzania. AIDS. 1989;3:289–296. doi: 10.1097/00002030-198905000-00007. [DOI] [PubMed] [Google Scholar]
- 7.Perriens JH, Mussa M, Luabeya MK, Kayembe K, Kapita B, et al. Neurological complications of HIV-1-seropositive internal medicine inpatients in Kinshasa, Zaire. J Acquir Immune Defic Syndr. 1992;5:333–340. [PubMed] [Google Scholar]
- 8.Clifford DB, Mitike MT, Mekonnen Y, Zhang J, Zenebe G, et al. Neurological evaluation of untreated human immunodeficiency virus infected adults in Ethiopia. J Neurovirol. 2007;13:67–72. doi: 10.1080/13550280601169837. [DOI] [PubMed] [Google Scholar]
- 9.Sacktor NC, Wong M, Nakasujja N, Skolasky RL, Selnes OA, et al. The International HIV Dementia Scale: a new rapid screening test for HIV dementia. AIDS. 2005;19:1367–1374. [PubMed] [Google Scholar]
- 10.Wong MH, Robertson K, Nakasujja N, Skolasky R, Musisi S, et al. Frequency of and risk factors for HIV dementia in an HIV clinic in sub-Saharan Africa. Neurology. 2007;68:350–355. doi: 10.1212/01.wnl.0000252811.48891.6d. [DOI] [PubMed] [Google Scholar]
- 11.Portegies P, de Gans J, Lange JM, Derix MM, Speelman H, et al. Declining incidence of AIDS dementia complex after introduction of zidovudine treatment. BMJ. 1989;299:819–821. doi: 10.1136/bmj.299.6703.819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Sacktor N, Lyles RH, Skolasky R, Kleeberger C, Selnes OA, et al. HIV-associated neurologic disease incidence changes:: Multicenter AIDS Cohort Study, 1990–1998. Neurology. 2001;56:257–260. doi: 10.1212/wnl.56.2.257. [DOI] [PubMed] [Google Scholar]
- 13.d'Arminio Monforte A, Cinque P, Mocroft A, Goebel F-D, Antunes F, et al. Changing incidence of central nervous system diseases in the EuroSIDA cohort. Ann Neurol. 2004;55:320–328. doi: 10.1002/ana.10827. [DOI] [PubMed] [Google Scholar]
- 14.Bhaskaran K, Mussini C, Antinori A, Walker AS, Dorrucci M, et al. Changes in the incidence and predictors of human immunodeficiency virus-associated dementia in the era of highly active antiretroviral therapy. Ann Neurol. 2008;63:213–221. doi: 10.1002/ana.21225. [DOI] [PubMed] [Google Scholar]
- 15.Sacktor NC, Lyles RH, Skolasky RL, Anderson DE, McArthur JC, et al. Combination antiretroviral therapy improves psychomotor speed performance in HIV-seropositive homosexual men. Multicenter AIDS Cohort Study (MACS). Neurology. 1999;52:1640–1647. doi: 10.1212/wnl.52.8.1640. [DOI] [PubMed] [Google Scholar]
- 16.Sacktor N, Nakasujja N, Skolasky R, Robertson K, Wong M, et al. Antiretroviral therapy improves cognitive impairment in HIV+ individuals in sub-Saharan Africa. Neurology. 2006;67:311–314. doi: 10.1212/01.wnl.0000225183.74521.72. [DOI] [PubMed] [Google Scholar]
- 17.Sacktor NC, Skolasky RL, Lyles RH, Esposito D, Seines OA, et al. Improvement in HIV-associated motor slowing after antiretroviral therapy including protease inhibitors. J Neurovirol. 2000;6:84–88. doi: 10.3109/13550280009006385. [DOI] [PubMed] [Google Scholar]
- 18.World Health Organization, World Federation of Neurology. Atlas: Country Resources for Neurologic Disorders. Geneva. 2004:1–62. [Google Scholar]
- 19.World Health Organization. The world health report: working together for health. 2006. http://www.who.int/whr/2006/whr06_en.pdf, Accessed 2010 September 1.
- 20.Jowi J. Provision of Care to People with Epilepsy in Kenya. East African Medical Journal. 2007;84:97–99. [PubMed] [Google Scholar]
- 21.Singh D, Joska J, Goodkin K, Lopez E, Myer L, et al. Normative scores for a brief neuropsychological battery for the detection of HIV-associated neurocognitive disorder (HAND) among South Africans. BMC Research Notes. 2010;3:28. doi: 10.1186/1756-0500-3-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Robertson KR, Nakasujja N, Wong M, Musisi S, Katabira E, et al. Pattern of neuropsychological performance among HIV positive patients in Uganda. BMC Neurol. 2007;7:8. doi: 10.1186/1471-2377-7-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.World Health Organization. 2008. Task shifting: rational redistribution of tasks among health workforce teams: global recommendations and guidelines: http://www.who.int/healthsystems/TTR-TaskShifting.pdf, Accessed: 2010 September 1.
- 24.Antinori A, Arendt G, Becker J, Brew B, Byrd D, et al. Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 2007;69:1789–1799. doi: 10.1212/01.WNL.0000287431.88658.8b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Chelune G, Heaton R, Lehman R. Neuropsychological and personality correlates of patients' complaints of disability. In: Goldstein G, editor. Advances in clinical neuropsychology. New York: Plenium Press; 1986. pp. 95–126. [Google Scholar]
- 26.Folstein M, Folstein S, McHugh P. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res Nov. 1975;12(3):189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 27.Katz S, Ford A, Moskowitz R, Jackson B, Jaffe M. Studies of Illness in the Aged. The Index of ADL: A standardized measure of biological and psychosocial function. JAMA. 1963;185:914–919. doi: 10.1001/jama.1963.03060120024016. [DOI] [PubMed] [Google Scholar]
- 28.Lawton M, Brody E. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9:179–186. [PubMed] [Google Scholar]
- 29.RAND Health Appendix A Core Survey Instrument: The Medical Outcomes Study 116 item core set of measures of functioning and well-being. http://www.rand.org/content/dam/rand/www/external/health/surveys_tools/mos/mos_core_survey.pdf. Accessed: 2008 May 25.
- 30.Schag C, Heinrich R, Ganz P. Karnofsky Performance Status Revisited: Reliability, Validity and Guidelines. Journal of Clinical Oncology. 1984;2:187–193. doi: 10.1200/JCO.1984.2.3.187. [DOI] [PubMed] [Google Scholar]
- 31.Wechsler D. 1997. WAIS-III: Administration and Scoring Manual: The Psychological Corporation.
- 32.Lezak MD, Howieson DB, Loring DW, Hannay HJ, JS F, editors. Neuropsychological Assessment. 4th ed. New York: Oxford University Press; 2004. pp. 422–429. [Google Scholar]
- 33.Maj M, Janssen R, Starace F, Zaudig M, Satz P, et al. WHO Neuropsychiatric AIDS study, cross-sectional phase I. Study design and psychiatric findings. Arch Gen Psychiatry. 1994;51:39–49. doi: 10.1001/archpsyc.1994.03950010039006. [DOI] [PubMed] [Google Scholar]
- 34.Benedict RHB, editor. Brief Visual memory Test-Revised. 4th ed. Florida: Psychological Assessment Resources, Inc; 1997. [Google Scholar]
- 35.Lafayette I. Grooved Pegboard User's Manual. Lafayette, IN: Lafayette Instrument Company, Inc; 2003. [Google Scholar]
- 36.PAR. Finger Tapper User's Guide. Florida: Psychological Assessment Resources, Inc; 1992. [Google Scholar]
- 37.Robertson KR, Parsons TD, Sidtis JJ, Hanlon Inman T, Robertson WT, et al. Timed Gait test: normative data for the assessment of the AIDS dementia complex. J Clin Exp Neuropsychol. 2006;28:1053–1064. doi: 10.1080/13803390500205684. [DOI] [PubMed] [Google Scholar]
- 38.Gladsjo JA, Schuman CC, Evans JD, Peavy GM, Miller SW, et al. Norms for letter and category fluency: demographic corrections for age, education, and ethnicity. Assessment. 1999;6:147–178. doi: 10.1177/107319119900600204. [DOI] [PubMed] [Google Scholar]
- 39.Monsch AU, Bondi MW, Butters N, Salmon DP, Katzman R, et al. Comparisons of verbal fluency tasks in the detection of dementia of the Alzheimer type. Arch Neurol. 1992;49:1253–1258. doi: 10.1001/archneur.1992.00530360051017. [DOI] [PubMed] [Google Scholar]
- 40.Strauss E, Sherman E, Spreen O. A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary. New York: Oxford University Press Inc; 2006. [Google Scholar]
- 41.Monahan P, Shacham E, Reece M, Kroenke K, On'gor W, et al. Validity/Reliability of PHQ-9 and PHQ-2 Depression Scales Among Adults Living with HIV/AIDS in Western Kenya. Journal of General Internal Medicine. 2008;24:189–197. doi: 10.1007/s11606-008-0846-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Birbeck GL, Chomba E, Kvalsund M, Bradbury R, Mang'ombe C, et al. Antiretroviral adherence in rural Zambia: the first year of treatment availability. Am J Trop Med Hyg. 2009;80:669–674. [PubMed] [Google Scholar]
- 43.Jarvis Joseph N, Lawn Stephen D, Vogt M, Bangani N, Wood R, et al. Screening for Cryptococcal Antigenemia in Patients Accessing an Antiretroviral Treatment Program in South Africa. Clin Infect Dis. 2009;48:856–862. doi: 10.1086/597262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Liechty CA, Solberg P, Were W, Ekwaru JP, Ransom RL, et al. Asymptomatic serum cryptococcal antigenemia and early mortality during antiretroviral therapy in rural Uganda. Trop Med Int Health. 2007;12:929–935. doi: 10.1111/j.1365-3156.2007.01874.x. [DOI] [PubMed] [Google Scholar]
- 45.Sacktor N. The epidemiology of human immunodeficiency virus-associated neurological disease in the era of highly active antiretroviral therapy. J Neurovirol. 2002;8:115–121. doi: 10.1080/13550280290101094. [DOI] [PubMed] [Google Scholar]
- 46.Saykin A, Janssen R, Sprehn G, Kaplan J, Spira T, et al. Longitudinal evaluation of neuropsychological function in homosexual men with HIV infection: 18 month follow up. J Neuropsychiatry Clin Neurosci. 1991;3:286–298. doi: 10.1176/jnp.3.3.286. [DOI] [PubMed] [Google Scholar]
- 47.McArthur JC, Haughey N, Gartner S, Conant K, Pardo C, et al. Human immunodeficiency virus-associated dementia: an evolving disease. J Neurovirol. 2003;9:205–221. doi: 10.1080/13550280390194109. [DOI] [PubMed] [Google Scholar]
- 48.Heaton RK, Grant I, Butters N, White DA, Kirson D, et al. The HNRC 500–neuropsychology of HIV infection at different disease stages. HIV Neurobehavioral Research Center. J Int Neuropsychol Soc. 1995;1:231–251. doi: 10.1017/s1355617700000230. [DOI] [PubMed] [Google Scholar]
- 49.Tozzi V, Balestra P, Lorenzini P, Bellagamba R, Galgani S, et al. Prevalence and risk factors for human immunodeficiency virus-associated neurocognitive impairment, 1996 to 2002: results from an urban observational cohort. J Neurovirol. 2005;11:265–273. doi: 10.1080/13550280590952790. [DOI] [PubMed] [Google Scholar]
- 50.Dunbar N, Brew B. Neuropsychological Dysfunction in HIV Infection: A Review. J NeuroAIDS. 1997;1:73–102. doi: 10.1300/j128v01n03_05. [DOI] [PubMed] [Google Scholar]
- 51.Power C, Selnes O, Grim J, McArthur J. HIV Dementia Scale: a rapid screening test. Journal of Acquired Immunodeficiency Syndrome and Human Retrovirology. 1995;8:273–278. doi: 10.1097/00042560-199503010-00008. [DOI] [PubMed] [Google Scholar]
- 52.Castaneda A, Suvisaari J, Marttunen M, Perala J, Saarni S, et al. Cognitive functioning in a population-based sample of young adults with a history of non-psychotic unipolar depressive disorders without psychiatric comorbidity. J Affect Disord. 2008;110:36–45. doi: 10.1016/j.jad.2007.12.239. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
HIV Dementia Diagnostic Tool.
(DOC)