Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Apr 1.
Published in final edited form as: Clin Neuropsychol. 2013 Jan 21;27(3):437–454. doi: 10.1080/13854046.2012.759627

Exploring the Utility of the Montreal Cognitive Assessment to Detect HIV-Associated Neurocognitive Disorder: The Challenge and Need for Culturally Valid Screening Tests in South Africa

Reuben N Robbins 1, John A Joska 2, Kevin GF Thomas 3, Dan J Stein 2, Teboho Linda 2, Claude A Mellins 1, Robert H Remien 1
PMCID: PMC3631431  NIHMSID: NIHMS432303  PMID: 23336183

Abstract

There is a strong need in South Africa for neuropsychological tests that can help detect HIV-associated neurocognitive disorder (HAND) in the country’s 5.6 million people living with HIV. Yet, South African neuropsychologists are challenged to do so, as few neuropsychological tests or batteries have been developed or adapted for, and normed on, South Africa’s linguistically, culturally, educationally, and economically diverse population. The purpose of this study was to explore the utility of the Montreal Cognitive Assessment to detect HIV-associated neurocognitive impairment among a sample of HIV+ and HIV-Black, Xhosa-speaking South Africans. HIV+ participants performed significantly worse overall and specifically in the domains of visuospatial, executive, attention, and language (confrontation naming). Regression analysis indicated that HIV status and education were the strongest predictors of total scores. Floor effects were observed on cube drawing, rhinoceros naming, serial 7’s, and one abstraction item, suggesting those items might not be useful in this population. While the Montreal Cognitive Assessment holds promise to help detect HAND in South Africa, it will likely need modification before it can be normed and validated for this population. Findings from this study may help neuropsychologists working with similar populations.

Keywords: HIV, neuropsychology, screening, cross-cultural, South Africa, low-literacy, low-education

Introduction

One of the most challenging aspects of clinical neuropsychology is trying to determine quickly whether an individual is impaired or not, particularly if that person is from a population or group for whom few or no neuropsychological screening tests or batteries have been developed, adapted, or normed. Yet, neuropsychologists throughout the world are often required to do so, creating an urgent need for tests in settings with few or very limited neuropsychological tests for its populations (Robertson, Liner & Heaton, 2009).

Neuropsychologists in South Africa face a particularly daunting task, as this country has an extremely diverse population with 11 national languages and multiple ethnic groups with distinct cultural heritages (e.g., Nguni, Sotho, Indo-Malay, Indian, British, and Afrikaans). It also has wide variations in socioeconomic and educational backgrounds, with about half the population living in poverty (Armstrong, Lekezwa, & Siebrits, 2008; Statistics South Africa, 2003). A third of the population are functionally illiterate (Aitchison & Harley, 2006), and only one-fifth have completed 12 years of education (Statistics South Africa, 2003). Many are educated in substandard schools with dirt floors, no electricity or running water and consistently poor educational outcomes (van der Berg, 2008). Few screening tests or batteries have been specifically developed for or culturally adapted and normed for this diverse and largely disadvantaged population, making the determination of neuropsychological impairment rather challenging.

HIV care and research in South Africa is in urgent need of neuropsychological tests that are able to detect HIV-associated neurocognitive disorder (HAND). South Africa has the largest population of people living with HIV (PLWH) in the world with approximately 5.6 million (UNAIDS, 2009), and HAND is one of the most common clinical conditions of HIV (Heaton et al., 1995). Being diagnosed with HAND increases one’s risk of mortality (Vivithanaporn et al., 2010), and often leads to poor functional outcomes, such as suboptimal antiretroviral therapy (ART) adherence, employment difficulties, driving problems, and impaired activities of daily living (Gorman, Foley, Ettenhofer, Hinkin, & van Gorp, 2009; Heaton et al., 2004; Hinkin et al., 2002; Marcotte et al., 2004; van Gorp et al., 2007). Recent estimates in South Africa suggest HAND may be present in as many as 70% of ART-naïve adults with late stage HIV under 40 years of age (Joska et al., 2010), and may have a prevalence of as high as 80% among adults with documented ART adherence difficulties and low CD4 counts (Robbins, Remien, Mellins, Joska, & Stein, 2011). If these studies are at all reflective of the larger population of PLWH in South Africa, then millions of PLWH are at risk for HAND and all its associated functional impairments.

Currently, only two neuropsychological screening tests for HAND have been validated for use in South Africa: the International HIV Dementia Scale (IHDS; Sacktor et al., 2005; Joska et al., 2011; Singh, Sunpath, John, Eastham, & Gouden, 2008) and the HIV Dementia Scale (HDS; Power et al., 1995; Ganasen, Fincham, Smit, Seedat, & Stein, 2008). Concerns have been raised that these tests may (a) under-report or over-report impairment depending on the population (Joska et al., 2010; Robbins et al., 2011), (b) contain culturally inappropriate items (Ganasen et al., 2008), and (c) only screen for the most severe and least prevalent form of HAND (viz., HIV-associated dementia; Simioni et al., 2010; Joska et al., 2010; Robbins et al., 2011). Anecdotal reports suggest that some HDS tasks, such as three-dimensional cube drawing and timed alphabet writing, may be too difficult for some South African individuals (Ganasen et al., 2008). Furthermore, the IHDS fist-edge-palm task may also be too demanding for populations with low educational attainment and high rates of illiteracy (Nitrini, Caramelli, Herrera, Charchat-Fichman, & Porto, 2005).

The Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005), a screening tool designed to detect mild neurocognitive impairment, may hold promise to assist in the detection of HAND, including its less severe forms, in South Africa. The MoCA, which takes only approximately 10 minutes to administer, assesses many of the neurocognitive domains most affected by HIV, including executive functioning, attention/concentration, and memory. Although originally developed for use in North America with older adults at risk for Alzheimer’s disease, it has been validated for use as a screening tool for mild neurocognitive impairment related to other disease processes (e.g., Parkinson’s and Huntington’s Diseases; Bourdeau et al., 2005; Ismail, Rajji, & Shulman, 2010; Videnovic et al., 2010; Zadikoff et al., 2008). Furthermore, Koski et al. (2011) found the MoCA holds promise as a means of detecting milder forms of HIV-associated neurocognitive impairment. Although the MoCA has been studied and validated for use in several countries, such as, Japan, Egypt, Korea, and Portugal (Fujiwara et al., 2010; Lee et al., 2008; Rahman & El Gaafary, 2009; Wong et al., 2009), little is known about its utility for use with South African populations to detect HAND. This lack of knowledge is particularly critical because some MoCA tasks may not be appropriate for some South African populations.

The purpose of this study was to explore the utility of the MoCA as a brief screening tool for HAND among one of South Africa’s culturally distinct populations. First, we examined whether overall MoCA scores could distinguish between the neurocognitive functioning of HIV+ individuals and their demographically and psychiatrically matched HIV− counterparts. Second, we examined how individual test items performed in this population. Third, we examined to what extent demographic factors influenced test performance. Because no local norms for the MoCA exist, we examined how the HIV− participants’ performance compared to the MoCA’s published norms for normal, mild cognitively impaired, and Alzheimer’s disease groups. Finally, we present an interim, modified MoCA with possible cut-off scores.

Method

Participants

Seventy-eight Xhosa-speaking Black South Africans (39 HIV+ and 39 HIV−) participated. HIV+ participants were recruited from two Cape Town City government run health clinics. These clinics provide HIV care to the surrounding township communities (Khayelitsha and Imizamo Yethu). HIV− participants were recruited from proximally sited voluntary counseling and testing (VCT) centers also serving those communities. All participants self-identified as Black South African citizens with Xhosa as their primary language spoken at home. Data were collected from March 2009-December 2010.

HIV− participants received a confirmatory serological test to ensure HIV status. HIV+ participants were either currently on ART with HIV status confirmed through medical records, or had recently received a positive rapid and confirmatory serological HIV test and were being considered for ART. HIV− participants were purposively recruited to match the demographic characteristics of age, educational attainment, cultural background (Black, Xhosa-speaking), and current psychiatric conditions (mood, anxiety, and alcohol dependence disorders) of the HIV+ participants.

Individuals were not eligible for participation if they had a current psychotic disorder, severe cognitive impairment such that they did not have capacity to consent, history of head injury with a loss of consciousness of 30 minutes or longer, and, for HIV−, if they did not meet the matching criteria. All individuals who met study eligibility criteria and who agreed to participate in the study provided written informed consent. The Human Research Ethics Committee of the Faculty of Health Sciences at the University of Cape Town, the Institutional Review Board of the New York State Psychiatric Institute, and the health authorities of the City of Cape Town’s Department of Health provided approval for the conduct of this study.

Measures and Procedure

All measures were available in English and Xhosa and were administered in the participants’ choice of language by trained research staff. All participants chose the Xhosa-language administration. All assessment measures were forward and back translated by University of Cape Town translators, and were also reviewed by our Xhosa-speaking staff to ensure that the language would be acceptable and readily understood by the participant population. The measures were administered by bilingual (Xhosa and English) research staff who underwent training and confirmation in the administration and scoring of the measures by a US-based clinical psychologist trained in both psychiatric and neuropsychological assessment. Research staff were furthermore regularly supervised and monitored on-site by a South African-based psychologist trained in psychiatric and neuropsychological assessment.

Demographics and psychiatric measures

All participants completed questionnaires assessing demographic characteristics, and were administered a medical screen that asked about previous head injuries. Psychiatric status was assessed with the Mini Neuropsychiatric Inventory (MINI; Sheehan et al., 1998), a brief structured interview that assesses individuals across the major DSM-IV Axis I disorders (American Psychiatric Association, 1994) and that has been used as a gold standard in psychiatric studies around the world, including HIV+ patients in South Africa (Smit, van der Berg, Bekker, Seedat, & Stein, 2006).

Montreal Cognitive Assessment (MoCA)

The MoCA assesses participants across six broad domains of ability and neurocognitive function (Nasreddine et al., 2005). (1) Visuospatial abilities are assessed via a three-dimensional cube drawing task and a clock drawing task. (2) Executive functioning is assessed via an untimed, alternating trail making test, a two-item abstraction (similarities-type) task, and a phonemic fluency task. (3) Short-term memory is assessed via a five-item word list recall task. This task is comprised of two immediate recall trials, which are not scored, and a 5-min delayed free recall trial. (4) Attention/ working memory are assessed via three subtests: a digit span task (forward and backward), a tapping test, and a serial 7 subtraction task (serial 7’s). (5) Language is assessed via sentence repetition, phonemic fluency, and confrontation naming tasks. (6) Orientation is assessed via six items asking participants to name the current date, month, year, and day, as well as their current location and city.

Given the high rates of illiteracy in this population and from our experience administering the MoCA during training sessions with Xhosa-speaking practice examinees, as well as in discussions with our expert panel of neuropsychologists and psychometrists (including four of the authors), the phonemic fluency task and sentence repetition task were deemed problematic. Specifically, given high rates of illiteracy in South Africa and low levels of education among our target sample, we decided that the phonemic fluency task was too difficult, which is consistent with clinical experience and research with similar populations (Ratcliff et al., 1998; Strauss, Sherman, and Spreen, 2006). Furthermore, there is no readily available reference describing the frequency and distribution of phonemes in the Xhosa language. Hence, we could not determine which phonemes would be equivalent to the letter F. Additionally, Regarding the sentence repetition task, we did not have the resources to conduct any research into what types of sentences would be considered equivalent in terms of grammar, word count, syllabic count, and meaning salience, as the Xhosa language uses different consonant sounds (e.g., clicks) and grammatical structure (Pinnock, 1994). Thus, we substituted a semantic fluency task (animal naming) for phonemic fluency, a strategy also used in the Korean version of the MoCA (Lee et al., 2008), and we dropped the sentence repetition task. Whereas the original total MoCA score ranges from 0 to 30, the total maximum score for this modified version was 28. Furthermore, because of this modification we did not use the original cutoff score of < 25 in any analyses.

Statistical Analysis

Independent-samples t-tests and chi-square tests were conducted to examine how well the HIV− and HIV+ groups were matched. To test whether the MoCA could discriminate between HIV+ and HIV− participants’ neurocognitive function, we conducted a series of independent samples t-tests using MoCA total score and each individual domain score as dependent variables. We used Cohen’s d as an estimate of the effect size associated with each between-group comparison. To examine the cultural suitability of test items, we conducted a chi-square test of contingency (on group status) for each test item. We used phi (Φ) to estimate the effect size in each case. To examine the influence of demographic and disease factors on MoCA performance, we performed a hierarchical multiple regression using HIV status, gender, age in years, and education in years to predict the MoCA total score.

Finally, to test whether the published norms for the MoCA were “exportable” to South African populations such as that represented by the current sample, we conducted independent samples t-tests comparing our HIV− participant mean scores on the trail making, cube and clock drawing, confrontation naming, memory, digit span, tapping, serial 7’s, abstraction, and orientation tests to those from the published North American MoCA normative data. Here, the more conservative Welch’s t-test was used, as we did not want to assume equal variance. Because of the exploratory nature of this study, the dearth of studies on and the substantive need for neuropsychological tests for South African populations, we decided not to use any correction methods for multiple comparisons, in order to identify useful and problematic items for future use and research on the MOCA.

Results

Sample Characteristics

As Table 1 shows, the HIV+ and HIV− groups were well matched with regard to demographic and psychiatric characteristics: there were no significant between-group differences on any of the matching variables. Overall, the sample had a mean age of 29.62 years (SD = 5.75; range = 19 to 46), and 70.5% were female. Participants had an average of 10.81 years of education (SD = 1.38; range = 7 to 13); just over 65% did not complete high school. Only 21.8% (n = 17) of the sample reported being employed full- or part-time. Among the HIV+ participants, recent CD4 counts were available for 37 participants. The mean CD4 count was 297.22 (SD = 235.15), and ranged from 43 to 1200 with 65% (n = 24) having CD4 counts below 300. Thirty-three HIV+ participants were currently on antiretroviral therapy; all had documented adherence problems.

Table 1.

Participant Characteristics by HIV Status

HIV−
(n=39)
HIV+
(n=39)
Mean
or
Percent
SD or
N
Mean
or
Percent
SD or
N
t or
χ 2
P Cohen’s d or φ
Age 28.62 6.16 30.62 5.19 −1.60 0.13 0.35
Female 64% 25 77% 30 1.54 0.21 0.14
Education (years completed) 11.10 1.23 10.51 1.47 0.86 0.06 0.44
Employed 23% 9 21% 8 0.08 0.58 0.03
Current Psychiatric Disorder
Major Depressive Episode 3% 1 0% 0 1.01 0.31 0.11
Manic Episode 0% 0 0% 0 -- -- --
Hypomanic Episode 0% 0 0% 0 -- -- --
Panic Disorder 3% 1 5% 2 0.35 0.56 0.07
Post-traumatic Stress
Disorder
5% 2 10% 4 0.72 0.40 0.10
Generalized Anxiety
Disorder
3% 1 13% 5 2.89 0.09 0.19
Alcohol Dependence
Disorder
0% 0 0% 0 -- -- --

MoCA Scores

Regarding MoCA total scores, HIV+ participants performed significantly more poorly than HIV− participants (see Table 2). Regarding MoCA domain-specific scores, HIV+ participants performed significantly more poorly than HIV− participants on tests of visuospatial and executive, attention, and language (see Table 2). There were no other significant between-group differences.

Table 2.

Between groups comparison on MoCA domain and total scores

HIV−
(n=39)
HIV+
(n=39)
Max.
Score
Mean SD Mean SD t p Cohen’s d
Visuospatial (clock and cube drawing) 4 2.92 0.58 2.10 1.19 3.88 <0.001 0.88
Executive (trail making, abstraction and fluency)
Memory
4 2.82 0.51 1.74 1.09 5.58 <0.001 0.84
  Short-term (5-minute delayed recall) 5 3.72 0.97 3.92 1.46 −0.73 0.47 −0.16
  5-Word Learning Trial 1 5 4.26 0.68 2.49 1.39 7.13 <0.001 1.62
  5-Word Learning Trial 2 5 4.85 0.37 3.67 1.69 4.28 <0.001 .97
  Percent Retained 100 76.92 19.86 50.47 34.42 4.56 <0.001 .94
Attention (tapping, serial 7’s, and number span) 6 4.18 1.21 3.36 1.46 2.70 0.009 0.61
Language (confrontation naming) 3 2.33 0.74 1.79 1.08 2.57 0.01 0.58
Orientation 6 5.69 0.52 5.69 0.66 0.71 1.00 0
Total Score 28 21.67 2.00 18.62 4.39 3.95 <0.001 0.89

Note: The total maximum score for this modified MoCA was 28; Learning Trials 1 and 2, as well as Percent Retained are not scored in the original MoCA.

Though learning scores are not a part of the conventional scoring of the MoCA, we were interested in whether the MoCA could also be used to assess this neurocognitive domain, as learning deficits have been strongly associated with HNI (Grant, 2008; Heaton et al., 1995). Although the groups performed similarly on the delayed recall task, our analyses indicated that the HIV+ group performed significantly more poorly on both learning trials of the word list (see Table 2). Furthermore, we computed the amount of information retained from the learning trials to the delayed recall trial (delayed recall total divided by highest value from learning trials 1 and 2) and found that the HIV+ participants performed significantly more poorly (50% retained versus 77% retained). These results are also presented in Table 2.

MoCA Individual Item Comparisons

To examine whether individual MoCA items could discriminate neurocognitive performance between groups and whether certain items may be inappropriate for this population, we conducted chi-square tests for each of the dichotomously scored test items (serial 7’s was recoded such that 3-4 correct responses was coded as correct, and < 3 correct responses was coded as incorrect). Table 3 presents the results of these analyses. HIV− participants were significantly more likely than HIV+ participants to complete the trail making test correctly, draw the contour of the clock and depict requested time, name a line drawing of a lion and camel, repeat a three-digit string of numbers backwards, make 4 to 5 correct subtractions on serial 7’s, name more than 11 animals in 1 minute, and describe how a train and bicycle are alike. There were no significant between-group differences on any other items.

Table 3.

Between groups comparison of categorically scored MoCA items (percent correctly answered)

HIV−
(n=39)
HIV+
(n=39)
% N % N χ 2 p φ
Visuospatial Cube Copy 21% 8 13% 5 0.83 0.36 0.10
Clock Contour 100% 39 82% 32 7.69 0.01 0.31
Clock Numbers 87% 34 64% 25 5.64 0.02 0.27
Clock Hands 85% 33 51% 20 9.95 0.002 0.36
Executive Trail Making 90% 35 51% 20 13.87 <0.001 0.42
Semantic Fluency 100% 39 49% 19 26.90 <0.001 0.59
Train-Bicycle 87% 34 64% 25 5.64 0.02 0.27
Watch-Ruler 5% 2 10% 4 0.72 0.40 0.10
Attention Digits Forward 69% 27 72% 28 0.06 0.80 0.03
Digits Backward 72% 28 49% 19 4.34 0.04 0.24
Tapping 92% 36 92% 36 0 1.00 0
Serial 7’s 31% 12 10% 4 5.03 0.02 0.35
Language Lion 100% 39 80% 31 8.91 0.003 0.34
Rhinoceros 49% 19 38% 15 0.83 0.36 0.10
Camel 85% 33 62% 24 5.28 0.02 0.26

Note: Serial 7’s scored 0 – 3; analysis indicates that the highest score (3) significantly differs between HIV groups.

Across both groups’ performance, there were floor effects on several items. Both groups performed equally poorly on cube drawing, naming a rhinoceros, and describing how a watch and a ruler are alike. Although the HIV− group made significantly more full credit responses (4 to 5 correct subtractions) on the serial 7’s task, it is important to note that both groups had poor overall performance on this task. Only 31% of the HIV− group and 10% of the HIV+ group made 4-5 correct subtractions. Hence, asking individuals from this population to subtract 7’s from 100 might not be an appropriate cognitive task with this population.

Demographic Influences on MoCA Performance

Table 4 summarizes results from the hierarchical multiple regression using HIV status and demographic variables to predict MoCA total score. As the table shows, in the final model only HIV status (B = −2.51, SE = .74, p < .01) and years of education (B = 1.10, SE = .27, p < .001) significantly predicted MoCA total score.

Table 4.

Summary of hierarchical regression analysis for demographic variables predicting MoCA total score

Model 1 Model 2 Model 3 Model 4
Variable B SE B β B SE B β B SE B β B SE B β
HIV status −3.05 .77 −41*** −2.97 .78 −40*** −3.10 .80 −42*** −2.51 .74 −.34**
Gender .64 .86 .08 .60 .86 .07 .92 .79 .11
Age in years .06 .07 .10 .11 .06 .17
Education in years 1.10 .27 .41***
R 2 .17 .18 .19 .34
F for change in R2 15.61*** .55 .81 16.49***

Note: HIV status coded as 0 = negative, 1 = positive; Gender coded as 0 = female, 1 = male.

*

p<.05

**

p<.01

***

p<.001

Comparison of HIV Control Performance to Published MoCA Norms

It is important to note that the MoCA normative data are based on samples of older adults (mean age > 70 years) with either no cognitive impairment, mild cognitive impairment, or Alzheimer’s disease, from memory clinics in North America. Although clearly not the ideal comparison group for our HIV− sample, these are, to our knowledge, the only published norms for the MoCA, and it is these norms upon which the conventional MoCA impairment cut-off score is based.

Because our HIV− sample was much younger than the MoCA normative sample, we hypothesized that our participants would perform similarly or better. Contrary to those expectations, the HIV− participants did not perform significantly better than the MoCA normal controls on any of the subtests, though they did perform similarly to the MoCA normal controls on 4 of the 10 subtests: trail making, clock drawing, delayed memory, and tapping/attention (see Table 4). The HIV− participants performed, on average, significantly better than the mild cognitive impairment and Alzheimer’s disease groups on the trail making, clock drawing, and delayed tests. Interestingly, although our HIV− sample performed more poorly than the MoCA normal controls and mild cognitive impairment groups on the cube drawing, confrontation naming, digit span, serial 7’s, and abstraction subtests, they performed no differently on these tests than the Alzheimer’s group. On the orientation subtest, the HIV− sample performed more poorly than the normal control group, but significantly better than the mild cognitive impairment and Alzheimer’s disease groups.

Interim, Modified MoCA with Proposed Cut-Off Scores

Because options for neuropsychological testing and screening are so limited in South Africa, we present an interim, modified MoCA that may assist neuropsychologists in detecting neurocognitive impairment among Xhosa-speaking PLWH. Our modified MoCA used only those items best able to distinguish between HIV groups, including the two word-list learning trials. Although serial 7’s was able to discriminate between groups, we decided to drop it from this modified version because even among the HIV− participants, scores tended to be very low. While we retained digits backwards in this interim, modified version, we dropped digits forward, as it did not demonstrate any ability to distinguish between HIV groups. The modified test yields a total maximum score of 19, and still assesses individuals across multiple domains, including visuospatial, executive, attention/working memory, and language. While short-term memory is dropped, learning is added. Specifically, the modified test includes trail making, all components of clock drawing, digits backward, two items from confrontation naming (lion and camel), the first item from abstraction, and the additional total score from both word-list learning trials.

Proposed cut-off scores of ≤ 15, ≤ 14, and ≤ 13 were examined. Using a cut-score of ≤ 15 (approximately 2 standard deviations below the HIV− mean) indicated that 77% of the HIV+ participants would be classified as impaired, while 13% of the HIV− participants would be classified as such. Using a cut-score of ≤ 14 indicated that 72% of HIV+ participants would be classified as impaired and 3% of HIV− individuals would also be classified as impaired. Using a cut-score of ≤ 13 indicated that 56% of HIV+ participants would be classified as impaired, while none (0%) of HIV− participants would be classified as such. Table 6 presents normative data for this interim, modified test for both HIV groups.

Table 6.

Interim, Modified MoCA Normative Data

HIV−
(n=39)
HIV+
(n=39)
Max.
Score
Mean SD Mean SD
Visuospatial (cube drawing: contour, number, hands) 3 2.72 0.51 1.97 1.06
Executive (trail making, abstraction and fluency) 3 1.77 0.49 1.15 .78
Learning (Sum of 5-Word Learning Trials 1 and 2) 10 9.10 0.91 6.15 2.78
Attention (digit span backwards) 1 1.64 .54 1.41 .64
Language (confrontation naming) 2 1.85 .37 1.41 0.79
Total Score 19 17.15 1.29 11.67 4.91

Discussion

In our sample of demographically and psychiatrically matched HIV− and HIV+ South Africans, the HIV+ group had significantly poorer overall performance on the MoCA. More specifically, the HIV+ group had significantly worse scores in the domains of visuospatial, executive, and attention/working memory; participants in that group also had lower scores on both word-list learning trials and retained a significantly lower percentage of the words over a 5-minute delay. This pattern of performance has been observed in HAND previously (Grant, 2008; Heaton et al., 1995, 2011; Martin et al., 2001; Peavy et al., 1994). When we used HIV status and demographic variables to predict MoCA total score, HIV status significantly predicted total score over and above age, gender, and education (i.e., being HIV+ predicted lower total scores). However, education was also a significant predictor of MoCA performance over and above HIV status.

This pattern of data suggests that the MoCA can grossly discriminate between the neurocognitive performance of HIV+ and HIV− Black, Xhosa-speaking South Africans. Because of the minor modifications we made to the MoCA (e.g., switching phonemic for semantic fluency) and the fact that no reliability and validity data exist for its use in South Africa, we did not have a validated cut-off score to use in classifying mild impairment versus no impairment. However, it is worthwhile to note that mean total scores in both groups were well below 23, which is 2 points less than the established cut-off score from North America of ≤25, and reflective of our removal of the sentence repetition task where the highest possible score on our modified MoCA was 28. Using this benchmark cut-score, most participants, regardless of their HIV status, would be classified as impaired.

Item-by-item analyses indicated that several MoCA tasks exhibited strong floor effects and thus may be inappropriate for use in this population. For example, most participants in both groups could not copy the cube correctly; this piece of data provides empirical support for previous anecdotal reports (Ganasen et al., 2008). It appears that complex drawing tasks are not appropriate for this population, though simpler ones might be. Similarly, most participants had difficulty with the confrontation naming task, particularly with the rhinoceros item. Despite the fact that the rhinoceros is indigenous to South Africa, common responses to the item were elephant, buffalo, and hippopotamus. Both groups also performed poorly on the serial 7’s task, and on one item of the abstraction task (watch and ruler). Because the HIV− group also performed poorly on these items, we suspect it was due to non-disease-related factors and may therefore reflect a possible source of test bias, though additional research is needed to confirm this finding and to speculate on potential reasons for this pattern of test performance.

We suspect these floor effects are due to educational opportunities or the lack thereof for these participants. For example, without early and regular exposure to drawing, more complex drawing, such as the Necker cube, many of these participants may simply not know how certain figures are drawn. Hence, when selecting test items for different populations, clinicians and researchers must evaluate which items are appropriate based on the specific cultural and educational backgrounds of the participants.

When we compared the performance of the HIV− participants to the MoCA normative samples (no impairment, mild cognitive impairment, and Alzheimer’s) with North Americans aged 70 years and above we found that, on average, they performed most similarly to the MoCA Alzheimer’s disease group on several tasks (cube drawing, confrontation naming, digit span, serial 7’s, and abstraction) and similarly to the normal group on other tasks (trail making, clock drawing, short-term memory, and tapping). Interestingly, they did not perform significantly better than North Americans aged 70-and-above on any of the items. These findings are a clear indication that the normative data from the MoCA samples is likely wholly inappropriate for this population. Using the normative data from the MoCA, from which the conventional cut-off score was generated, may lead to misclassification of healthy individuals as impaired in populations similar to our sample. Further research is needed to establish locally appropriate normative data and to determine the most sensitive and specific cut-off scores. Most participants in this study, regardless of HIV status, would be classified as impaired when compared to the MoCA normative sample means and standard deviations.

Finally, our proposed, interim, modified MoCA yielded a test with a maximum score of 19. Although we were unable to validate the psychometric properties of this modified version, including the proposed cut-off scores, we nonetheless present this version to offer some help to those clinicians in South Africa who work with HIV+ Xhosa-speakers and need some gauge of neurocognitive functioning. We think that the proposed cut-off scores of ≤13 and ≤14 are the most clinically useful, as these cut-offs suggest that 56% or 72%, respectively, of the sample could be impaired. Research in South Africa indicates that as many 42% of Xhosa-speaking PLWH may have mild HIV-associated neurocognitive impairment, and as many as 25% may have HIV-associated dementia (Joska et al., 2010). Hence, the proposed cut-off scores are consistent with this previous research, where as many as 67% of Xhosa-speaking PLWH have clinically significant impairment. The latter two proposed cut-off scores also minimize the number of HIV− participants being classified as impaired. Because this interim, modified MoCA and its proposed cut-off scores have not been validated, and because it is not clear what the true sensitivity and specificity of this version are, we caution against use before further validation efforts. Although the research gives us an estimate of what the base rate of neurocognitive impairment among Xhosa-speaking PLWH is, we do not have a similar estimate for the general, HIV− Xhosa population.

Although our findings shed important light on the utility of the MoCA in South Africa to detect mild neurocognitive impairment, there are several limitations to our study. First, we had a small, convenience sample, which may have limited our ability to (a) detect age-, gender-, and education-related differences, and (b) generalize to the larger South African population. Second, we performed multiple comparisons and we run the risk of having a high familywise error rate in this study; significant differences that emerged may be false discoveries. Bonferroni corrections were considered to reduce the error rate, however, we decided against using them because of this study’s exploratory nature, the relatively small sample, and the substantive need to begin to identify promising tools. At this stage, we did not want to overlook any possible differences due to being under powered. We recognize that given the multiple comparisons we ran, we could have a high Type I error rate and hope that future studies with larger sample sizes can help support these findings.

Third, based on the limited exclusionary criteria of this sample, several important confounders were not controlled for, which could offer alternative explanations to our findings, particularly among the HIV− participants. Although we attempted to control for some causes of neurocognitive disorder (e.g., head trauma with loss of consciousness greater than 30 minutes), without brain imaging or lumbar punctures, we cannot ascertain with certainty that any neurocognitive impairment was due only to HIV, or to what extent our HIV− participants did not have any neurological illness or any other disorder that could cause cognitive impairment. Fourth, although our groups were well matched on years of education completed, we were unable to control for quality of education. Given that many of the individuals in this sample were educated during Apartheid and in rural areas, the quality of their education was likely substandard. The quality education for someone who completed 11 years of school in a rural area with no electricity or running water and with dirt floors (a not so uncommon set of conditions) during Apartheid is vastly different from someone educated in the past decade at an urban school with computers and writing utensils. Future research must take this into consideration in understanding neuropsychological test performance for these populations. However, since all of the participants in this study were Black and from very low socioeconomic status, it is likely that quality of education did not vary much across groups.

Fifth, it is important to note that among this convenience sample, there were very low rates of depression among the HIV+ participants, much less than would be expected based on previous research (Olley, Seedat, & Stein, 2006). Hence, future studies will need to recruit more representative samples, as well as account for common comorbidities among PLWH, such as depression, in their analyses.

Finally, given the absence of any assessment of the MoCA’s reliability and validity in South Africa and that we did not have a comparison of MoCA scores to a gold standard neuropsychological test or test battery in this study, low scores cannot be assumed to reflect neurocognitive impairment. Future research needs to compare performance of our interim, modified MoCA to currently validated screening test for HAND in South Africa, such as the IHDS and HDS.

To our knowledge, this is the first study to systematically and empirically explore the utility and cultural appropriateness of the MoCA for HIV+ populations in South Africa and to generate data and ideas about how the MoCA performs in a very understudied, albeit very important, population (i.e., Black, Xhosa-speaking South Africans). This is also the first study to propose an interim, modified MoCA to assist neuropsychologists and clinicians in screening for HAND in South Africa. More research is needed to: examine how well the MoCA can differentiate among PLWH, and to answer the question of whether it can discriminate between high and low CD4 cell counts; develop locally derived norms; and offer proper validation of the MoCA (or a modified MoCA) for use in the South African clinical setting.

Table 5.

Comparison of HIV− (n=39) performance to MoCA control and clinical norms

Normal
(n=90)
Mild Cognitive
Impairment
(n=94)
Alzheimer’s Disease
(n=93)
t df t df t df
Trail Making 0.49 79 4.75*** 111 9.25*** 102
Cube −6.13*** 81 −3.00** 86 −0.50 75
Clock 0.66 91 4.76*** 111 8.90*** 124
Confrontation −4.42*** 46 −2.34* 58 0.96 79
Naming
Memory
−0.05 93 11.75*** 106 16.98*** 75
Digits Span −3.46*** 52 −3.57*** 51 −0.63 66
Tapping −1.06 53 −0.20 69 3.84*** 118
Serial 77#x2019;s −6.91*** 45 −5.03*** 55 0.16 88
Abstraction −11.22*** 74 −5.25*** 112 −0.66 124
Orientation −3.57** 39 1.41 111 8.95*** 122

Note: positive t-values indicate better mean performance among HIV− controls compared to MoCA samples, whereas negative t-values indicate worse mean performance among HIV− controls compared to MoCA samples.

*

p<.05

**

p<.01

***

p<.001

Acknowledgments

The work was supported by the National Institute of Mental Health [R34-MH082654 to R.H.R., T32-MH19139 Behavioral Sciences Research in HIV Infection; to T. Sandfort] and a scholar grant from the Mount Sinai Institute for NeuroAIDS Disparities to R.N.R. [National Institute of Mental Health, R25MH080663 to S. Morgello].

References

  1. Aitchison JJ, Harley A. South African illiteracy statistics and the case of the magically growing number of literacy and ABET learners. Journal of Education. 2006;39:89–112. [Google Scholar]
  2. American Psychiatric Association . Diagnostic and statistical manual of mental disorders: DSM-IV. 4th ed. American Psychiatric Association; Washington, DC: 1994. [Google Scholar]
  3. Armstrong P, Lekezwa B, Siebrits K. Poverty in South Africa: A profile based on recent household surveys. University of Stellenbosch; Matieland, South Africa: 2008. [Google Scholar]
  4. Bourdeau I, Bard C, Forget H, Boulanger Y, Cohen H, Lacroix A. Cognitive function and cerebral assessment in patients who have Cushing’s syndrome. Endocrinol Metab Clin North Am. 2005;34(2):357–369. doi: 10.1016/j.ecl.2005.01.016. ix. doi: 10.1016/j.ecl.2005.01.016. [DOI] [PubMed] [Google Scholar]
  5. Bruer E, Stoloff K, Myer L, Seedat S, Stein DJ, Joska J. Reliability of the lay adherence counsellor administered Substance Abuse and Mental Illness Symptoms Screener (SAMISS) and the International HIV Dementia Scale (IHDS) in a primary care HIV clinic in Cape Town, South Africa. AIDS & Behavior. 2011 doi: 10.1007/s10461-011-0067-z. in press. Epub ahead of print retreived December, 11, 2011. doi:10.1007/s10461-011-0067-z. [DOI] [PubMed] [Google Scholar]
  6. Fujiwara Y, Suzuki H, Yasunaga M, Sugiyama M, Ijuin M, Sakuma N, Shinkai S. Brief screening tool for mild cognitive impairment in older Japanese: Validation of the Japanese version of the Montreal Cognitive Assessment. Geriatrics & Gerontology International. 2010;10(3):225–232. doi: 10.1111/j.1447-0594.2010.00585.x. doi: 10.1111/j.1447-0594.2010.00585.x. [DOI] [PubMed] [Google Scholar]
  7. Ganasen KA, Fincham D, Smit J, Seedat S, Stein D. Utility of the HIV Dementia Scale (HDS) in identifying HIV dementia in a South African sample. Journal of the Neurological Sciences. 2008;269(1-2):62–64. doi: 10.1016/j.jns.2007.12.027. doi:10.1016/j.jns.2007.12.027. [DOI] [PubMed] [Google Scholar]
  8. Gorman A, Foley J, Ettenhofer M, Hinkin C, van Gorp W. Functional Consequences of HIV-associated Neuropsychological Impairment. Neuropsychology Review. 2009;19(2):186–203. doi: 10.1007/s11065-009-9095-0. doi: 10.1007/s11065-009-9095-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Heaton RK, Grant I, Butters N, White DA, Kirson D, Atkinson JH, the HNRC Group The HNRC 500: Neuropsychology of HIV infection at different disease stages. Journal of the International Neuropsychological Society. 1995;1(3):231–251. doi: 10.1017/s1355617700000230. doi: 10.1017/S1355617700000230. [DOI] [PubMed] [Google Scholar]
  10. Heaton RK, Marcotte TD, Mindt MR, Sadek J, Moore DJ, Bentley H, the HNRC Group The impact of HIV-associated neuropsychological impairment on everyday functioning. Journal of the International Neuropsychological Society. 2004;10(3):317–331. doi: 10.1017/S1355617704102130. doi: 10.10170S1355617704102130. [DOI] [PubMed] [Google Scholar]
  11. Hinkin CH, Castellon SA, Durvasula RS, Hardy DJ, Lam MN, Mason KI, Stefaniak BA. Medication adherence among HIV+ adults: Effects of cognitive dysfunction and regimen complexity. Neurology. 2002;59(12):1944–1950. doi: 10.1212/01.wnl.0000038347.48137.67. doi: 10.1212/01.WNL.0000038347.48137.67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Joska J, Westgarth-Taylor J, Myer L, Hoare J, Thomas K, Combrinck M, Flisher AJ. Characterization of HIV-associated Neurocognitive Disorders Among Individuals Starting Antiretroviral Therapy in South Africa. AIDS and Behavior. 2010:1–7. doi: 10.1007/s10461-010-9744-6. doi: 10.1007/s10461-010-9744-6. [DOI] [PubMed] [Google Scholar]
  13. Joska JA, Westgarth-Taylor J, Hoare J, Thomas KGF, Paul R, Myer L, Stein DJ. Validity of the International HIV Dementia Scale in South Africa. AIDS Patient Care and STDs. 2011;25(2):95–101. doi: 10.1089/apc.2010.0292. doi: doi:10.1089/apc.2010.0292. [DOI] [PubMed] [Google Scholar]
  14. Koski L, Brouillette MJ, Lalonde R, Hello B, Wong E, Tsuchida A, Fellows AK. Computerized testing augments pencil-and-paper tasks in measuring HIV-associated mild cognitive impairment*. HIV Medicine. 2011 doi: 10.1111/j.1468-1293.2010.00910.x. no-no. doi: 10.1111/j.14681293.2010.00910.x. [DOI] [PubMed] [Google Scholar]
  15. Lee J-Y, Lee Dong Woo, Cho S-J, Na DL, Jeon Hong Jin, Kim S-K, et al. Brief Screening for Mild Cognitive Impairment in Elderly Outpatient Clinic: Validation of the Korean Version of the Montreal Cognitive Assessment. Journal of Geriatric Psychiatry and Neurology. 2008;21(2):104–110. doi: 10.1177/0891988708316855. doi: 10.1177/0891988708316855. [DOI] [PubMed] [Google Scholar]
  16. Marcotte TD, Wolfson T, Rosenthal TJ, Heaton RK, Gonzalez R, Ellis RJ, the HNRC Group A multimodal assessment of driving performance in HIV infection. Neurology. 2004;63(8):1417–1422. doi: 10.1212/01.wnl.0000141920.33580.5d. doi: 10.1212/01.wnl.0000141920.33580.5d. [DOI] [PubMed] [Google Scholar]
  17. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Cherktow H. The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment. Journal of the American Geriatrics Society. 2005;53(4):695–699. doi: 10.1111/j.1532-5415.2005.53221.x. doi: http://dx.doi.org/10.1111/j.1532-5415.2005.53221.x. [DOI] [PubMed] [Google Scholar]
  18. Nitrini R, Caramelli P, Herrera EJ, Charchat-Fichman H, Porto CS. Performance in Luria’s Fist-Edge-Palm Test According to Educational Level. Cognitive and Behavioral Neurology. 2005;18(4):211–214. doi: 10.1097/01.wnn.0000195292.48422.d5. [DOI] [PubMed] [Google Scholar]
  19. Olley BO, Seedat S, Stein DJ. Persistence of psychiatric disorders in a cohort of HIV/AIDS patients in South Africa: a 6-month follow-up study. Journal of Psychosomatic Research. 2006;61(4):479–484. doi: 10.1016/j.jpsychores.2006.03.010. [DOI] [PubMed] [Google Scholar]
  20. Pinnock PS. Xhosa: A cultural grammar for beginners. African Sun Press; Cape Town, South Africa: 1994. [Google Scholar]
  21. Rahman TTA, El Gaafary MM. Montreal Cognitive Assessment Arabic version: Reliability and validity prevalence of mild cognitive impairment among elderly attending geriatric clubs in Cairo. Geriatrics & Gerontology International. 2009;9(1):54–61. doi: 10.1111/j.1447-0594.2008.00509.x. doi: 10.1111/j.1447-0594.2008.00509.x. [DOI] [PubMed] [Google Scholar]
  22. Ratcliff G, Ganguli M, Chandra V, Sharma S, Belle S, Seaberg E, Pandav R. Effects of literacy and education on measures of word fluency. Brain and Language. 1998;61:115–122. doi: 10.1006/brln.1997.1858. [DOI] [PubMed] [Google Scholar]
  23. Robertson K, Liner J, Heaton R. Neuropsychological assessment of HIV-infected populations in international settings. Neuropsychology Review. 2009;19(2):232–249. doi: 10.1007/s11065-009-9096-z. doi: 10.1007/s11065-009-9096-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Robbins RN, Remien RH, Mellins CA, Joska J, Stein D. Screening for HIV-associated dementia in South Africa: The potentials and pitfalls of task-shifting. AIDS Patient Care and STDs. 2011;25(10):587–593. doi: 10.1089/apc.2011.0154. doi: 10.1089/apc.2011.0154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Sacktor NC, Wong M, Nakasujja N, Skolasky RL, Selnes OA, Musisi S, Katabira E. The International HIV Dementia Scale: A new rapid screening test for HIV dementia. AIDS. 2005;19(13):1367–1374. doi: 10.1097/01.aids.0000180790.77379.3a. [PubMed] [Google Scholar]
  26. Simioni S, Cavassini M, Annoni J-M, Rimbault Abraham A, Bourquin I, Schiffer V, Du Pasquier RA. Cognitive dysfunction in HIV patients despite long-standing suppression of viremia. AIDS. 2010;24(9):1243–1250. doi: 10.1097/QAD.0b013e3283354a7b. doi: 1210.1097/QAD.1240b1013e3283354a3283357b. [DOI] [PubMed] [Google Scholar]
  27. Singh D, Sunpath H, John S, Eastham L, Gouden R. The utility of a rapid screening tool for depression and HIV dementia amongst patients with low CD4 counts-a preliminary report. African Journal of Psychiatry. 2008;11(4):282–286. [PubMed] [Google Scholar]
  28. Smit J, van der Berg CE, Bekker L-G, Seedat S, Stein DJ. Translation and cross-cultural adaptation of a mental health battery in an African setting. African Health Sciences. 2006;6(4):215–222. doi: 10.5555/afhs.2006.6.4.215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Statistics South Africa . Census 2001. Census in Brief. Statistics South Africa; Pretoria, South Africa: 2003. [Google Scholar]
  30. Strauss E, Sherman EM, Spreen O. A compendium of neuropsychological tests: Administration, Norms and Commentary. 3rd ed. Oxford University Press; New York: 2006. [Google Scholar]
  31. UNAIDS Global Report: UNAIDS Report on the Global AIDS Epidemic: 2010. 2010 Retrieved August 15, 2011, from http://www.unhcr.org/refworld/docid/4cfca9c62.html.
  32. van der Berg S. How effective are poor schools? Poverty and educational outcomes in South Africa. Studies In Educational Evaluation. 2008;34(3):145–154. doi: 10.1016/j.stueduc.2008.07.005. [Google Scholar]
  33. van Gorp WG, Rabkin JG, Ferrando SJ, Mintz J, Ryan E, Borkowski T, et al. Neuropsychiatric predictors of return to work in HIV/AIDS. Journal of the International Neuropsychological Society. 2007;13(1):80–89. doi: 10.1017/S1355617707070117. doi: http://dx.doi.org/10.1017/S1355617707070117. [DOI] [PubMed] [Google Scholar]
  34. Videnovic A, Bernard B, Fan W, Jaglin J, Leurgans S, Shannon KM. The Montreal Cognitive Assessment as a screening tool for cognitive dysfunction in Huntington’s disease. Movement Disorders. 2010;25(3):401–404. doi: 10.1002/mds.22748. doi: 10.1002/mds.22748. [DOI] [PubMed] [Google Scholar]
  35. Vivithanaporn P, Heo G, Gamble J, Krentz HB, Hoke A, Gill MJ, Power C. Neurologic disease burden in treated HIV/AIDS predicts survival. Neurology. 2010;75(13):1150–1158. doi: 10.1212/WNL.0b013e3181f4d5bb. doi: 10.1212/WNL.0b013e3181f4d5bb. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Wong A, Xiong YY, Kwan PWL, Chan AYY, Lam WWM, Wang K, Mok VC. The Validity, Reliability and Clinical Utility of the Hong Kong Montreal Cognitive Assessment (HK-MoCA) in Patients with Cerebral Small Vessel Disease. Dementia and Geriatric Cognitive Disorders. 2009;28(1):81–87. doi: 10.1159/000232589. doi: 10.1159/000232589. [DOI] [PubMed] [Google Scholar]
  37. Zadikoff C, Fox SH, Tang-Wai DF, Thomsen T, de Bie RMA, Wadia P, Marras C. A comparison of the mini mental state exam to the montreal cognitive assessment in identifying cognitive deficits in Parkinson’s disease. Movement Disorders. 2008;23(2):297–299. doi: 10.1002/mds.21837. doi: 10.1002/mds.21837. [DOI] [PubMed] [Google Scholar]

RESOURCES