Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Mar 28.
Published in final edited form as: Appl Neuropsychol Adult. 2018 Sep 28;27(2):173–180. doi: 10.1080/23279095.2018.1510403

Home- versus acquired-language test performance on the Hopkins Verbal Learning Test-Revised among multilingual South Africans

Travis M Scott 1,5, Hetta Gouse 2, John Joska 2, Kevin G F Thomas 3, Michelle Henry 4, Anna Dreyer 2, Reuben N Robbins 5
PMCID: PMC6438773  NIHMSID: NIHMS1512474  PMID: 30265567

Abstract

The Hopkins Verbal Learning Test-Revised (HVLT-R) has been adapted for use in many different languages and in low- and middle-income countries. However, few adaptations have evaluated performance differences between home- and acquired-language administrations. The present study examined performance on an adapted HVLT-R between multilingual South Africans who chose to be tested in a home or acquired language. The HVLT-R was administered to 112 multilingual, isiXhosa as home language, Black South African adults (49% men) with no major medical, neurological, or psychiatric problems. Sixty-one preferred to take the test in isiXhosa and 51 preferred English. We examined between-language differences in word equivalency, primary scores, learning indices, and serial position effects. We also examined language, age, education, and gender on test performance. English-examinees were significantly younger and more educated than isiXhosa-examinees (p’s<.05). Although isiXhosa words had more letters and syllables than English words (p’s <.001), there were no significant differences between groups on HVLT-R performance or serial recall (p’s>.05). More education and being a woman predicted better Total and Delayed Recall (p’s<.05). Performance on this modified HVLT-R appears similar between English and isiXhosa administrations among South African isiXhosa first language speakers, which makes comparisons between preferred language administrations appropriate.

Keywords: Verbal list-learning, cognitive testing, cross-cultural, multilingual, South Africa

There is an urgent need for neuropsychologists to competently and accurately assess linguistically diverse individuals (Rivera Mindt, Byrd, Saez, & Manly, 2010). Unique challenges arise when assessing multilingual individuals, especially in low- and middle-income countries (LAMICs) with very limited neuropsychological resources. In many LAMICs, little or no normative data exist for neuropsychological tests adapted for home languages, and norms that do exist are usually for a commonly used but acquired language (e.g., English; Robertson, Liner, & Heaton, 2009). Furthermore, although objective measures of language proficiency (e.g., verbal fluency tests) are vital in determining the most appropriate language for neuropsychological testing, there is limited availability of such standardized and normed objective measures in LAMICs (Rivera Mindt et al., 2008).

South Africa is a multicultural, multilingual LAMIC that is a prime example of a context within which assessing linguistically diverse individuals is of great importance. The country has 11 official languages and most citizens speak their first learned or home language and at least one other language (Posel & Zeller, 2016; Statistics South Africa, 2012). In the Western Cape province of South Africa (where the current study was conducted), isiXhosa is the second most commonly spoken language with over 8 million total native speakers (Statistics South Africa, 2012). It is a tonal language that was modernized to use a Latin alphabet where the letters c, x, and q each represent different clicking sounds (Austin, 2008). Primary school in isiXhosa communities is typically taught in isiXhosa and English. In fact, the South African Constitution guarantees that children have the right to be educated in their home language, but after grade 3, their remaining education usually occurs in English. Few South Africans are truly monolingual as there is not only a need to speak and understand English (the predominant language of business and government), but proficiency in other languages is often required (Posel & Zeller, 2016).

Verbal list-learning tests are widely used to assess learning and memory, and are a key component of many comprehensive neuropsychological batteries. Because performance on these tests are highly influenced by language proficiency, and because the most commonly used verbal list-learning tests were developed and normed on English-speaking samples, these tests must be translated, modified, and/or adapted when used to assess non-English speaking individuals. Determining in which language to administer these tests is challenging in the absence of objective measures of language proficiency (Nell, 2012; Robbins et al., 2013; Robertson et al., 2009), and thus test administrators often rely on examinees’ stated language preference. It is therefore vital to evaluate performance between multilingual individuals taking the test in an acquired language and those taking the test in their home language. Furthermore, evaluating whether there are performance differences between multiple language versions of the same verbal memory test is an important step prior to norming the test (Ardila, 2005; Rivera Mindt et al., 2010; Uzzell, Ponton, & Ardila, 2013). The creation of separate norms for various language adaptations of a verbal list-learning test requires tremendous resources and is an onerous process, especially in settings where people speak more than one language. However, the creation of separate language-based norms may be warranted if there are significant differences in test performance between language versions (Hofmann, 2017; Nell, 2012).

The Hopkins Verbal Learning Test-Revised (HVLT-R; Benedict, Schretlen, Groninger, & Brandt, 1998) is one of the more commonly used verbal list-learning tests in neuropsychological practice. This test has been translated and adapted for use in numerous non-English speaking LAMIC populations (e.g., Brazil, India, Malawi, Peru, Thailand, Zimbabwe; Hestad et al., 2016; Robertson et al., 2016), including an isiXhosa version (Joska et al., 2011). The isiXhosa version was originally used in studies characterizing the prevalence of HIV-Associated Neurocognitive Disorders (HAND) among HIV+ South African adults (Joska et al., 2011; Joska et al., 2012). More recently, the isiXhosa version of the HVLT-R has been used as part of a neuropsychological test battery to compare with a new tablet-based screening test to detect neurocognitive impairment in this population (Robbins et al., 2018). However, most adaptations of the HVLT-R, including the isiXhosa variant, have not reported on multiple, important linguistic and demographic characteristics known to influence test performance.

For example, a verbal list-learning test in a language that has longer words and more syllables in each word might be more challenging than the same version in a language with shorter words with less syllables. In fact, word equivalency (i.e., number of letters and syllables) can influence performance, and thus should be considered when comparing performance between different language groups (Agranovich & Puente, 2007; Lim et al., 2009). Additionally, the primacy effect (i.e., the high likelihood of words from the beginning of a word list being recalled) and the recency effect (i.e., the high likelihood of words from the end of a word list being recalled) has been used as evidence of similar verbal list-learning test performance across different languages (Lim et al., 2009; Murdock, 1962). Finally, several studies have found that being a woman and/or more years of education impact HVLT-R performance (Cherner et al., 2007; Friedman, Schinka, Mortimer, & Borenstein Graves, 2002; Hester, Kinsella, Ong, & Turner, 2004; Vanderploeg et al., 2000), but few studies have examined demographic factors (e.g., age, years of education, and gender) in non-English versions of the HVLT-R.

The purpose of this study was to compare performance characteristics between preferred language administrations of an adapted version of the HVLT-R (Joska et al., 2011; Witten, Thomas, Westgarth-Taylor, & Joska, 2015) among South Africans who share isiXhosa as their home language. We explored test performance between language administrations across HVLT-R Trial 1, Total and Delayed Recall, Percent Retention, Recognition Discrimination, learning indices (e.g., serial and semantic clustering), word equivalency (i.e., word length and syllable count), and serial position effects. We also examined demographic effects (i.e., age, education, and gender) on verbal list-learning performance in the entire sample.

Methods

Participants

As part of a larger research study (Robbins et al., 2018), 112 Black South African adult isiXhosa first language speakers (49% men) were recruited from a peri-urban primary healthcare clinic in Cape Town, South Africa. Inclusion criteria were: 1) HIV-negative status; 2) age 18 years or older; 3) isiXhosa-speaking; and 4) capacity for informed consent. To determine HIV status, interested participants were required to bring proof of a recent (within 7 days) HIV-negative test result. Exclusion criteria were: 1) history of loss of consciousness for more than 15 minutes, or hospitalization, as a result of traumatic brain injury; 2) current psychotic or substance use disorder (with the latter determined using the Drug Use Disorders Identification Test (DUDIT; Berman, Bergman, Palmstierna, & Schlyter, 2003) and the Alcohol Use Disorders Identification Test (AUDIT; Saunders, Aasland, Babor, De la Fuente, & Grant, 1993); 3) current mild-moderate or severe depressive symptomatology (i.e., Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996) score ≥ 17; 4) positive urine toxicology result for any illicit substance; 5) or anything preventing informed consent or participation based on clinical judgment of providers (e.g., severe cognitive impairment).

Procedures

After providing informed consent, participants indicated their language of preference for testing and a trained neuropsychology technician administered the HVLT-R in English or isiXhosa. Fifty-one participants preferred English administration, and 61 preferred isiXhosa administration. The Faculty of Health Sciences Human Research Ethics Committee, University of Cape Town, and the New York State Psychiatric Institute gave ethical approval for the study.

Measures

Demographic and neuromedical data.

Demographic and medical history data were collected by a neuropsychology technician who administered a neuromedical questionnaire prior to administering the cognitive tests.

Modified HVLT-R.

The present study examined a modified form A of the original HVLT-R (Joska et al., 2011). The content and instructions of this form of the instrument are identical to the original, with one notable exception: the test was modified to be more culturally appropriate by replacing all words indicating gemstones with words indicating items of clothing. These latter words were taken from form D of the original. Additionally, on the recognition trial, the word penny was replaced with cent to be consistent with South African currency. For more details on the adaptation process, see Joska et al. (2011). The test was administered using standard procedures for participants taking the test in English or isiXhosa. Participants were first asked to verbally state as many of the words as they could remember. This process was repeated twice for a total of three learning trials. Participants were then asked to spontaneously recall the words after a 20–25-minute delay. A recognition trial immediately followed the delayed recall trial where participants were presented with a list in random order of the 12 learned target words and 12 non-target words, 6 of which were drawn from the same semantic categories as the target words. Then participants were asked to indicate whether or not each of the 24 words were from the target list.

Statistical analyses

We used SPSS (version 22.0) to analyze the data, with a p level of .05 determining statistical significance.

Deriving outcome variables.

For the purpose of the present study, primary outcome measures for this modified HVLT-R included: Trial 1 performance, Total Recall (i.e., total words recalled on trials 1–3), Delayed Recall, Percent Retention (i.e., Delayed Recall / higher score of Trials 2 or 3), and Recognition Discrimination (i.e., true positive responses minus related and unrelated false positive responses). HVLT-R learning indices included: Average Learning Slope, Semantic Clustering Ratio, and Serial Clustering Ratio. We defined Average Learning Slope based on previous research, which calculated the average number of new words (i.e., in trials 2 and 3) correctly recalled across the learning trials (Delis, Kramer, Kaplan, & Thompkins, 1987; Noll, Weinberg, Ziu, & Wefel, 2016). Semantic Clustering Ratio was calculated by dividing the total number of semantic clusters by the total number of words correctly recalled on the learning trials. The total number of semantic clusters was derived from the number of times a participant correctly recalled two consecutive words within the same category (i.e., pants then skirt) across the three trials (Noll et al., 2016). Serial Clustering Ratio was calculated by dividing the total number of serial clusters by the total number of words correctly recalled on the learning trials. The total number of serial clusters was derived from the number of times a participant correctly recalled two consecutive words in the same position in the word list as presented during the original trial (i.e., words 1, 2, 4, 5, 3, 8, 10 would be two serial clusters) across the three trials (Noll et al., 2016). Similar to prior research, we defined primacy as the percentage of items recalled from the primacy region (i.e., first 4 words) for trials 1–3 (i.e., Total Recall) and Delayed Recall, and recency as the percentage of items recalled from the recency region (i.e., last 4 words) for trials 1–3 and Delayed Recall (Schrijnemaekers, de Jager, Hogervorst, & Budge, 2006).

Primary analyses.

Two independent sample t-tests were conducted to examine word equivalency characteristics (i.e., word length and syllable count) between test language groups (English or isiXhosa). To examine demographic differences in the sample, independent samples t-tests compared age and years of education between participants tested in English and isiXhosa. A chi-square analysis was used to determine the association between gender and test language.

Independent samples t-tests compared primary HVLT-R outcome measures (i.e., Trial 1, Total, Delayed Recall, Percent Retention, and Recognition Discrimination), HVLT-R learning indices (i.e., Average Learning Slope, Semantic Clustering Ratio, and Serial Clustering Ratio), and primacy and recency effect differences between and the test language groups. Four multiple linear regressions were computed to predict primary outcome measures based on age, years of education, gender, and test language.

Results

The resulting sample included 112 participants, 51% female, aged 18 to 64 years old (M = 35.4, SD = 12.0), who had obtained between 7 and 13 years of education (M = 10.5, SD = 1.4). The entire sample (i.e., both language groups combined) recalled an average of 6.41 words on trial 1 (SD = 1.54), 24.88 words on trials 1–3 (i.e., HVLT-R Total Recall; SD = 4.00), and 8.56 words on the delayed recall trial (SD = 2.01). The sample recalled an average of 86% (SD = 16%) of the words on trials 2 or 3 compared to delayed recall (i.e., HVLT-R Percent Retention), and the average Recognition Discrimination score was 10.61 (SD = 1.37).

Comparisons between word composition

In all but one word (tiger/ingwe), isiXhosa words had more syllables and letters than English words. Overall, isiXhosa words had, on average, 3.42 (SD = 2.31) more letters and 2.33 (SD = 0.89) more syllables than English words. An independent samples t-test comparing the number of letters indicated that isiXhosa words had significantly more letters (M = 7.92, SD = 1.93) than English words (M = 4.50, SD = 0.90; t(22) = −5.56, p < .001, d = 2.37). An independent samples t-test comparing the number of syllables per word between languages also indicated that isiXhosa words had significantly more syllables (M = 3.58, SD = 0.67) than English words (M = 1.25, SD = 0.45; t(22) = −10.01, p < .001, d = 4.26).

Between language group comparisons

Table 1 presents demographic and HVLT-R performance differences between English- and isiXhosa-examinees.

Table 1.

Demographic and HVLT-R performance differences across language groups (N = 112).

English M (SD) or % (n) isiXhosa M (SD) or % (n) t or χ2 Cohen’s d or Cramer’s V
Age* 30.5 (10.5) 39.5 (11.7) 4.25 .81
Education (in years)* 11.0 (1.3) 10.2 (1.5) 2.93 .57
Gender (% women) 45% (23) 56% (34) 1.26 .11
HVLT-R primary measures
Trial 1 6.4 (1.7) 6.4 (1.4) 0.01 <.01
Total Recall Raw 25.2 (4.3) 24.7 (3.7) 0.66 .12
Delayed Recall Raw 8.7 (2.0) 8.4 (2.0) 0.69 .13
Percent Retention 87.1 (16.5) 85.6 (15.9) 0.47 .09
Discrimination Index 10.7 (1.4) 10.6 (1.4) 0.42 .08
HVLT-R learning process indices
Average Learning Slope 2.1 (0.9) 1.9 (1.0) 0.90 .21
Semantic Clustering Ratio 0.28 (0.10) 0.28 (0.10) 0.13 <.01
Serial Clustering Ratio 0.10 (0.08) 0.10 (0.07) 0.32 <.01
Open in a new tab

Notes. HVLT-R = Modified Hopkins Verbal Learning Test-Revised

*

p <.01

Demographics.

English- and isiXhosa-examinees did not differ in terms of gender composition (p = .26). English-language examinees were younger than isiXhosa-language examinees (30.5 vs. 39.5 years, respectively; p < .001) and had more years of education (11.0 vs. 10.2 years, respectively; p = .004).

HVLT-R primary outcome measures.

English- and isiXhosa-examinees did not differ in performance on any primary HVLT-R outcome measure. Analyses detected no significant between-group differences on HVLT-R Trial 1 (p = .995), HVLT-R Total Recall (p = .51), Delayed Recall (p = .49), Percent Retention (p = .64), or Recognition Discrimination (p = .68).

HVLT-R learning indices.

English- and isiXhosa-examinees also did not differ in performance on any secondary HVLT-R outcome measure. Analyses detected no significant between-group differences on Average Learning Slope (p = .37), Semantic Clustering Ratio (p = .90), or Serial Clustering Ratio (p = .75).

Serial recall effect.

Figures 1 and 2 present the comparisons between English- and isiXhosa-administrations based on primacy and recency response type for HVLT-R Total Recall (Figure 1) and Delayed Recall (Figure 2). Analyses detected no significant primacy or recency effects between language administration groups for either the Total Recall or Delayed Recall outcome variables. Specifically, for HVLT-R Total Recall, the English-examinees recalled a similar percentage of words from the primacy and recency regions as the isiXhosa-examinees (t(110) = 1.22, p = .22; t(110) = −0.81, p = .42, respectively). Likewise, for HVLT-R Delayed Recall, English-examinees recalled a similar percentage of words from the primacy and recency regions as the isiXhosa-examinees (t(110) = 0.38, p = .71; t(110) = 1.13, p = .26, respectively).

Figure 1.

Figure 1.

Open in a new tab

Comparisons between English- and isiXhosa-examinees for serial position effect on HVLT-R Total Recall.

Figure 2.

Figure 2.

Open in a new tab

Comparisons between English- and isiXhosa-examinees for serial position effect on HVLT-R Delayed Recall.

Demographic factors and test language predicting HVLT-R performance in combined sample

Four multiple regressions were performed to determine whether demographic factors (i.e., age, education, and gender) or test language impacted primary HVLT-R outcome measures (i.e., HVLT-R Total Recall, Delayed Recall, Percent Retention, and Recognition Discrimination) in the combined sample. A significant regression was found for Total Recall (R2 = .15, F(4, 107) = 4.66, p<.01) and Delayed Recall (R2 =.11, F(4, 107) = 3.17, p = .02). More education (B = .56, p = .04; B = .34, p = .01, respectively) and being a woman (B = −2.77, p < .01; B = −.98, p = .01, respectively) predicted better HVLT-R Total and Delayed Recall performance. Neither age nor test language were associated with HVLT-R Total or Delayed Recall performance (p’s > .05). No demographic factor or test language predicted HVLT-R Percent Retention (R2=.04 F(4, 107) =0.99, p=.42) or Recognition Discrimination (R2 =.05 F(4, 107) =1.30, p =.27). There were no significant interaction effects for test language by education or gender by education (p’s > .05) in any of the above regressions.

Discussion

Results of the present study provide evidence that this South African version of the HVLT-R (Joska et al., 2011; Witten et al., 2015) performs similarly when administered to multilingual South Africans who share isiXhosa as their home language, but prefer to take the test in either English or isiXhosa. English-language examinees were younger and more educated than isiXhosa-language examinees, which is not surprising given that younger, more educated South Africans might be likely to prefer English over another home or acquired language. Despite English-language examinees being younger and with more years of education and isiXhosa words having significantly more letters and syllables, HVLT-R performance did not significantly differ between participants who were administered the test in English or isiXhosa. Participants also did not differ in learning (i.e., learning slope, semantic or serial clustering) or recall strategy used (i.e., primacy and recency effects). In the combined sample, more years of education and being a woman, but not age, predicted better Total Recall and Delayed Recall scores.

Performance on this adapted version of the HVLT-R is consistent with previous research on the English HVLT-R and other verbal list-learning tests (e.g., California Verbal Learning Test, Rey Auditory Verbal Learning Test) in that individuals with more years of education and women perform better than those with less education and men (Friedman et al., 2002; Norman, Evans, Miller, & Heaton, 2000; Van Der Elst, Van Boxtel, Van Breukelen, & Jolles, 2005).

No study to our knowledge has compared performance between different language administrations of the HVLT-R among multilingual individuals who share the same home language. Unlike most of the prior studies that have compared performance of two or more language groups on an adapted verbal list-learning test (Agranovich & Puente, 2007; Gonzalez, Mungas, Reed, Marshall, & Haan, 2001; Maj et al., 1994), the present study demonstrated similar performance between language administration groups not only in traditional outcome measures (e.g., total and delayed recall), but also by item level and in learning and recall strategy used. Furthermore, given the generalizability of the HVLT-R compared to other novel verbal list-learning tests (e.g., Lim et al., 2009), similar performance between English- and isiXhosa-examinees provides avenues for future studies to conduct more rigorous psychometric testing to establish HVLT-R test equivalence (e.g., construct validity, test-retest reliability properties) within these and other languages. Future psychometric studies examining the performance of different language versions of the HVLT-R are essential to provide the foundation to advance cross-cultural and cross-linguistic neuropsychology research and test development. As the present study only reported outcomes for form A of this adapted HVLT-R, future studies might also adapt and establish equivalency for and between additional forms of this test to aid in repeated testing and longitudinal studies with repeated measures. While evaluating the psychometric properties of this adapted version of the HVLT-R remains a goal, the findings from the present study have practical implications for clinicians in South Africa.

Specifically, results from the present study suggest that multilingual South Africans who identify their home language as isiXhosa and are proficient in English and/or isiXhosa can be administered this version of the HVLT-R in their language of choice. This result is important for the following reasons: 1) these findings provide much needed evidence to suggest that a commonly used neuropsychological test can be successfully modified for multilingual individuals; 2) given the widespread use of modified versions the HVLT-R in HIV research, our study provides a future avenue for establishing equivalency and validity of modified versions of this test in patients at risk for neurologic illness (e.g., HIV) who speak different languages; 3) it may be appropriate to compare HVLT-R performance between individuals regardless of whether the test was administered in English or isiXhosa; and 4) at least in South Africa, it may be appropriate to generate normative data for this version of the HVLT-R in bi- and/or multi-lingual communities, as opposed to generating separate language based norms at a much greater cost. To inform future clinical practice, it will be vital for research to further demonstrate that other modified neuropsychological tests perform similarly to well-established English versions of these tests and similarly across multilingual groups taking the test in different languages.

Our study provides the foundation for which normative data can be derived for multilingual South Africans with isiXhosa as their home language. As research groups from other African countries are currently using the same procedures for test modification as the present study (e.g., Hestad et al., 2016; Kabuba, Anitha Menon, Franklin, Heaton, & Hestad, 2017), our results also provide support for current applications of this particular version of the HVLT-R. Validating this modified HVLT-R in other African countries and around the world is an important next step to establish test equivalency in both healthy individuals and people living with neurological disorders. Additionally, the creation of new verbal list-learning tests specifically created for use with African and other languages might remain a goal for future studies.

It is important to note several limitations to this study that constrain the generalizability and applicability of our findings. First, though participants had the preference of receiving the test in either English or isiXhosa, formal language proficiency assessment was not administered. Second, normative performance for multilingual South Africans with isiXhosa as their home language has not been established and use of this test for diagnostic purposes is not yet warranted. Hence, future research must establish appropriate general normative data (i.e., data that corrects for education and gender) for this test. Third, we did not include several demographic variables (i.e., quality of education and premorbid intelligence) that appear to impact neuropsychological test performance (Manly et al., 1999). Future research should consider these variables as possible demographic predictors of neuropsychological performance in multilingual groups. Finally, it is possible that the significant findings observed in this study may be spurious due to multiple comparisons. Hence, these findings should be considered with caution.

In conclusion, this is the first study to compare multiple aspects of performance (e.g., word equivalency, serial position effects) on a commonly used verbal-list learning test across multilingual individuals taking the test in different languages. Similar performance between English- and isiXhosa-examinees on a modified version of the HVLT-R might provide the framework for the creation of normative data for this test, and the creation of such normative data would permit use of the test, regardless of language of administration, for clinical diagnostic purposes. However, will be especially important for future studies to replicate these findings in other language groups to determine if performance on this modified HVLT-R is similar between English and additional languages. For now, our study provides support for continued use of this modified HVLT-R administered to isiXhosa first language speakers who prefer to take this test in either English or isiXhosa.

Acknowledgements

The authors wish to acknowledge the hard work of our research staff: Michelle Henry, Veronica Jonah, Thandeka Mbonambi, and Tandiwe Mngxuma, and study participants. We would like to thank the City of Cape Town Department of Health for their support.

Funding

This work was supported by the National Institute of Child Health and Human Development under Grant R21-HD084197; National Institute of Nursing Research under Grant R21-NR015009; National Institute of Mental Health under Grant R01-MH09557; National Institute of Mental Health to the HIV Center for Clinical and Behavioral Studies at NY State Psychiatric Institute and Columbia University under Grant P30-MH43520; and the Columbia University Global Mental Health Scholars Program.

Footnotes

Disclosure statement

The authors report no conflicts of interest.

References

  1. Agranovich AV, & Puente AE (2007). Do Russian and American normal adults perform similarly on neuropsychological tests? Preliminary findings on the relationship between culture and test performance. Archives of Clinical Neuropsychology, 22(3), 273–282. 10.1016/j.acn.2007.01.003 [DOI] [PubMed] [Google Scholar]
  2. Ardila A (2005). Cultural values underlying psychometric cognitive testing. Neuropsychology Review, 15(4), 185–195. 10.1007/s11065-005-9180-y [DOI] [PubMed] [Google Scholar]
  3. Austin P (2008). One thousand languages: Living, endangered, and lost. Los Angeles, CA: University of California Press. [Google Scholar]
  4. Beck AT, Steer RA, & Brown GK (1996). Beck Depression Inventory-II (BDI-II). San Antonio, TX: Psychological Corporation. [Google Scholar]
  5. Benedict RHB, Schretlen D, Groninger L, & Brandt J (1998). Hopkins Verbal Learning Test – Revised: Normative data and analysis of inter-form and test-retest reliability. The Clinical Neuropsychologist, 12(1), 43–55. 10.1076/clin.12.1.43.1726 [DOI] [Google Scholar]
  6. Berman AH, Bergman H, Palmstierna T, & Schlyter F (2003). The Drug Use Disorders Identification Test: Manual. Stockholm, Sweden: Karolinska Institutet. [Google Scholar]
  7. Cherner M, Suarez P, Lazzaretto D, Fortuny LA, Mindt MR, Dawes S, … Heaton R (2007). Demographically corrected norms for the Brief Visuospatial Memory Test-Revised and Hopkins Verbal Learning Test-Revised in monolingual Spanish speakers from the U.S.-Mexico border region. Archives of Clinical Neuropsychology, 22(3), 343–353. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Delis DC, Kramer JH, Kaplan E, & Thompkins BAO (1987). CVLT: California Verbal Learning Test-adult version: Manual. San Antonio, TX: Psychological Corporation. [Google Scholar]
  9. Friedman MA, Schinka JA, Mortimer JA, & Borenstein Graves A (2002). Hopkins Verbal learning test-revised: Norms for elderly African Americans. The Clinical Neuropsychologist, 16(3), 356–372. 10.1076/clin.16.3.356.13857 [DOI] [PubMed] [Google Scholar]
  10. Gonzalez HM, Mungas D, Reed BR, Marshall S, & Haan MN (2001). A new verbal learning and memory test for English- and Spanish- speaking people. Journal of the International Neuropsychological Society, 7, 544–555. [DOI] [PubMed] [Google Scholar]
  11. Hestad K, Menon J, Serpell R, Kalungwana L, Mwaba S, Kabuba N, … Heaton R (2016). Do neuropsychological test norms from African Americans in the United States generalize to a Zambian population? Psychological Assessment, 28(1), 18–38. 10.1037/pas0000147 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hester RL, Kinsella GJ, Ong B, & Turner M (2004). Hopkins Verbal Learning Test: Normative data for older Australian adults. Australian Psychologist, 39(3), 251–255. 10.1080/00050060412331295063 [DOI] [Google Scholar]
  13. Hofmann SG (2017). Clinical psychology: A global perspective. Hoboken, NJ: John Wiley & Sons. [Google Scholar]
  14. Joska JA, Westgarth-Taylor J, Hoare J, Thomas KGF, Paul R, Myer L, & Stein DJ (2012). Neuropsychological outcomes in adults commencing highly active anti-retroviral treatment in South Africa: A prospective study. BMC Infectious Diseases, 12(1), 39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Joska JA, Westgarth-Taylor J, Myer L, Hoare J, Thomas KGF, Combrinck M, … Flisher AJ (2011). Characterization of HIV-associated neurocognitive disorders among individuals starting antiretroviral therapy in South Africa. AIDS and Behavior, 15(6), 1197–1203. [DOI] [PubMed] [Google Scholar]
  16. Kabuba N, Anitha Menon J, Franklin DR, Heaton RK, & Hestad KA (2017). Use of Western neuropsychological test battery in detecting HIV-Associated Neurocognitive Disorders (HAND) in Zambia. AIDS and Behavior, 21(6), 1717–1727. 10.1007/s10461-016-1443-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lim YY, Prang KH, Cysique LAJ, Pietrzak RH, Snyder PJ, & Maruff P (2009). A method for cross-cultural adaptation of a verbal memory assessment. Behavior Research Methods, 41(4), 1190–1200. 10.3758/BRM.41.4.1190 [DOI] [PubMed] [Google Scholar]
  18. Maj M, Satz P, Janssen R, Zaudig M, Starace F, D’elia L, … Ndetei D (1994). WHO Neuropsychiatric AIDS Study, cross-sectional phase II: Neuropsychological and neurological findings. Archives of General Psychiatry, 51(1), 51–61. [DOI] [PubMed] [Google Scholar]
  19. Manly JJ, Jacobs DM, Sano M, Bell K, Merchant CA, Small SA, & Stern Y (1999). Effect of literacy on neuropsychological test performance in nondemented, education-matched elders. Journal of the International Neuropsychological Society, 5(3), 191–202. [DOI] [PubMed] [Google Scholar]
  20. Murdock BB (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64(5), 482–488. 10.1037/h0045106 [DOI] [Google Scholar]
  21. Nell V (2012). Cross-cultural neuropsychological assessment: Theory and practice. New York, NY: Psychology Press Routledge. [Google Scholar]
  22. Noll KR, Weinberg JS, Ziu M, & Wefel JS (2016). Verbal learning processes in patients with glioma of the left and right temporal lobes. Archives of Clinical Neuropsychology, 31(1), 37–46. 10.1093/arclin/acv064 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Norman MA, Evans JD, Miller WS, & Heaton RK (2000). Demographically corrected norms for the California Verbal Learning Test. Journal of Clinical and Experimental Neuropsychology, 22(1), 80–94. [DOI] [PubMed] [Google Scholar]
  24. Posel D, & Zeller J (2016). Language shift or increased bilingualism in South Africa: Evidence from census data. Journal of Multilingual and Multicultural Development, 37(4), 357–370. 10.1080/01434632.2015.1072206 [DOI] [Google Scholar]
  25. Rivera Mindt M, Arentoft A, Germano KK, D’Aquila E, Scheiner D, Pizzirusso M, … Gollan TH (2008). Neuropsychological, cognitive, and theoretical considerations for evaluation of bilingual individuals. Neuropsychology Review, 18(3), 255–268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Rivera Mindt M, Byrd D, Saez P, & Manly J (2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Robbins RN, Gouse H, Brown HG, Ehlers A, Scott TM, Leu CS, … Joska JA (2018). A mobile app to screen for neurocognitive impairment: Preliminary validation of neuroscreen among HIV-infected South African adults. Journal of Medical Internet Research, 20(1). 10.2196/mhealth.9148 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Robbins RN, Joska JA, Thomas KGF, Stein DJ, Linda T, Mellins CA, & Remien RH (2013). Exploring the utility of the Montreal Cognitive Assessment to detect HIV-associated neurocognitive disorder: The challenge and need for culturally valid screening tests in South Africa. The Clinical Neuropsychologist, 27(3), 437–454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Robertson K, Jiang H, Evans SR, Marra CM, Berzins B, Hakim J, … Walawander A (2016). International neurocognitive normative study: Neurocognitive comparison data in diverse resource-limited settings: AIDS Clinical Trials Group A5271. Journal of NeuroVirology, 22(4), 472–478. 10.1007/s13365-015-0415-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Robertson K, Liner J, & Heaton R (2009). Neuropsychological assessment of HIV-infected populations in international settings. Neuropsychology Review, 19(2), 232–249. 10.1007/s11065-009-9096-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Saunders JB, Aasland OG, Babor TF, De la Fuente JR, & Grant M (1993). Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption‐II. Addiction, 88(6), 791–804. 10.1111/j.1360-0443.1993.tb02093.x [DOI] [PubMed] [Google Scholar]
  32. Schrijnemaekers A, de Jager C, Hogervorst E, & Budge M (2006). Cases with mild cognitive impairment and Alzheimer’s disease fail to benefit from repeated exposure to episodic memory tests as compared with controls. Journal of Clinical and Experimental Neuropsychology, 28(3), 438–455. 10.1080/13803390590935462 [DOI] [PubMed] [Google Scholar]
  33. Statistics South Africa. (2012). Census 2011 - Census in brief. World Wide Web. http://doi.org/ISBN978-0-621-41388-5
  34. Uzzell BP, Ponton M, & Ardila A (2013). International handbook of cross-cultural neuropsychology. Mahwah, New Jersey: Psychology Press. [Google Scholar]
  35. Van Der Elst WIM, Van Boxtel MPJ, Van Breukelen GJP, & Jolles J (2005). Rey’s verbal learning test: Normative data for 1855 healthy participants aged 24–81 years and the influence of age, sex, education, and mode of presentation. Journal of the International Neuropsychological Society, 11(3), 290–302. [DOI] [PubMed] [Google Scholar]
  36. Vanderploeg RD, Schinka JA, Jones T, Small BJ, Borenstein Graves A, & Mortimer JA (2000). Elderly norms for the Hopkins Verbal Learning Test-Revised. The Clinical Neuropsychologist, 14(3), 318–324. [DOI] [PubMed] [Google Scholar]
  37. Witten JA, Thomas KGF, Westgarth-Taylor J, & Joska JA (2015). Executive dyscontrol of learning and memory: Findings from a clade C HIV-positive South African sample. The Clinical Neuropsychologist, 29(7), 956–984. 10.1080/13854046.2015.1108455 [DOI] [PubMed] [Google Scholar]

RESOURCES