Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: J Neurovirol. 2016 Oct 13;23(1):134–140. doi: 10.1007/s13365-016-0487-7

Evaluation of Practice Effect on Neuropsychological Measures among Persons With and Without HIV Infection in Northern India

Raymond L Ownby *,1, Drenna Waldrop-Valverde *,2, Deborah L Jones *, Sunil Sharma +, Ritu Nehra +, Adarsh M Kumar *, Sudesh Prabhakar +, Amarilis Acevedo 3, Mahendra Kumar *
PMCID: PMC5330814  NIHMSID: NIHMS823097  PMID: 27739034

Abstract

An evaluation of the effects of HIV infection on neurocognition over time is important for understanding disease progression. Changes in cognitive function can be evaluated longitudinally using neuropsychological testing at repeated intervals. The assessment of change over time, however, is complicated by the potentially confounding influence of learning on repeated test administrations, often referred to as practice effect. In this study, we present data on testing of persons with or without HIV infection on a battery administered at study baseline and repeated one year later. Results suggest that practice effects may be diminished in persons with HIV infection compared to without it. This appears to be true even among those with relatively intact immune functioning as measured by CD4 count.

Keywords: neurocognitive, practice effect, longitudinal assessment, India

Introduction

The ability to evaluate disease-related changes in neurocognitive function is essential in carrying out longitudinal research on neurologic diseases. In order to create and validate models of the underlying disease process and to appraise its course over periods that may include years, it is necessary to assess patients’ cognitive functioning at multiple time points. Change in neurocognitive function over time in those with diseases affecting the brain can thus be evaluated via serial administrations of neuropsychological test batteries but the interpretation of scores on batteries repeated over several administrations is complicated by the observation that most persons’ scores on tests of cognitive function improve over repeated administrations. This improvement results from learning across test administrations and is often termed practice effect (Bartels et al., 2010; Calamia et al., 2012). Information on practice effects for individuals without neurologic disease is available for many neurocognitive measures in various Western countries (Calamia et al., 2013; Strauss et al., 2006); however, no readily identifiable report on practice effects in India could be located.

HIV-1 infection has clear effects on neurocognitive tests. Infection with HIV-1 has been shown in the West to continue to have neurocognitive effects even in the context of effective antiretroviral treatment (Heaton et al., 2010). HIV-1 infection in Western countries, however, primarily results from clade B virus (Gannon et al., 2011), while in India it primarily results from clade C (Robertson et al., 2010). Since there are differences in the structures of clade B and C virus that can affect each clade’s neurotropism and neurovirulence (Tyor et al., 2013), it is not clear that research on neuropsychological performance of affected individual in the West can be validly generalized to persons in India. This is especially important given findings that HIV-1 infection in India is associated with neurocognitive deficits. These studies, however, are limited and their results have been inconsistent (Gupta et al., 2007; Yepthomi et al., 2006). Other researchers (Tyor et al., 2013) have argued that ascertainment of cognitive impairment in India may be biased due to participant selection methods and the lack of adequate normative data for diverse groups and languages (Waldrop-Valverde et al., 2015).

Studies in India of neurocognition in HIV/AIDS have been completed with relatively small numbers of participants, most often in southern regions of the country in which language and culture differ in important ways from Northern India. In the north, studies have only rarely been done. Studies that examine the effects of HIV/AIDS on neurocognition in persons living in the northern area of India are not readily available. We previously reported normative data for control and HIV-infected individuals evaluated in a study at the Post Graduate Institute of Medical Education & Research (PGIMER) in Chandigarh, a city in northwest India (Waldrop-Valverde et al., 2015). In this paper, we report results of a study of practice effect on the same neuropsychological battery administered on two occasions over one year to healthy individuals and persons with HIV infection.

Methods

Participants

The Institutional Review Boards of both the Post Graduate Institute of Medical Education & Research (PGIMER), Chandigarh, India and the University of Miami reviewed and approved this protocol before study initiation. Participants included men and women with or without HIV-1 infection who lived in the area served by PGIMER in Chandigarh. The sample consisted of 89 persons with HIV infection (42 men and 47 women; mean age 30.4 years, SD = 6.2) and 51 persons without infection (33 men and 18 women; mean age 28.9 years, SD = 6.0 years). Persons without infection were recruited as control subjects in a longitudinal study of persons with HIV infection who received ongoing care at PGIMER. Exclusion criteria for both groups were (1) history of head injury, (2) drug or alcohol dependence assessed by self-report, (3) age less than 18 or greater than 45 years.

All data were collected between October, 2007 and August, 2009. During this period in India, only individuals with CD4 counts less than 350 were provided with antiretroviral medication. At the time of the first visit, no participants with HIV infection had counts less than this value. At the time of the second visit, 8 of the participants with HIV infection had CD4 counts less than 350.

Measures

Measures used in this study were chosen to evaluate cognitive domains known to be affected by HIV infection based on studies in the US and other western countries. All measures were administered in Hindi, the language most commonly spoken by persons in Chandigarh and its vicinity. Measures were translated into Hindi in a process that first involved a general meeting of investigators from the US and PGIMER. Test content was not modified in the translation process, that is, materials were translated literally from English to Hindi, with the following exceptions. The team judged that several words on the Hopkins Verbal Learning Test-Revised (HVLT-R; items representing the gemstones “opal” and “ruby”) would be unfamiliar to research participants. The Hindi words “moonga” and “pukhraj,” also gemstone names but more likely to be familiar to participants, were substituted in the HVLT-R. Other words were literally translated. Letters used in the Controlled Oral Word Association Test (COWA) were changed from F, A, and S to P, A, and R as these letters occur in Hindi at a frequency similar to the letters used in the English version. No other modifications were made. Administration instructions were translated by PGIMER staff and, the translations’ accuracies were verified by the study team.

Participants thus completed an extensive neurocognitive battery administered and scored according to published techniques. Tests included in the battery:

Hopkins Verbal Learning Test-Revised, or HVLT-R (Benedict et al., 1998) requires the person assessed to learn and then recall a list of words, first immediately after presentation and then 20 minutes later. Recognition memory is assessed when the examiner presents a list of 24 words that includes both words from the original presentation and foils; the person tested sates whether the words were on the original list.

Brief Visuospatial Memory Test-Revised, or BVMT-R (Benedict, 1997) evaluates both immediate and delayed recall of simple geometric designs presented on cards. After presentation, the person tested is asked to draw as many designs as he or she saw. This process is repeated two more times. After a delay interval, the person is asked again to produce the designs. Recognition is also assessed in this measure as the ability of persons to identify whether designs had previously been presented to them.

Digit Symbol-Coding, is a subtest from the Wechsler Adult Intelligence Scale-III or WAIS-III (Wechsler, 1997a), that taps attention and psychomotor speed. The person tested is asked to fill in an array of blank boxes according to a key at the top of the page that indicates pairing of numbers and symbols. The test is timed and performance is scored as the number of number-design pairing correctly produced in the timed interval.

Symbol Search, also a subtest of the WAIS-III (Wechsler, 1997a), evaluates attention and psychomotor speed as well. The person tested scans groups of symbols on a piece of paper and must decide if symbols in the groups match a provided target.

The Grooved Pegboard Test (Lafayette Instrument, 2002) evaluates fine motor coordination and psychomotor speed. Persons tested are asked to put 25 metal pegs into holes on a 3″ × 3″ metal board. All pegs have the same shape and have a ridge on one side which corresponds to a notch in each hole on the board requiring an appraisal of the peg’s correct orientation as well as the motor skill to place it in the correct orientation.

The Color Trails Test 1 & 2, or Color Trails, (D’Elia et al., 1996) uses colored circled numbers and differently colored circled numbers to measure mental flexibility and speeded visual search. The CTT was developed by the World Health Organization to reduce the cultural bias of similar measures such as the Trail Making test that require familiarity with the letters of the Roman alphabet.

The Spatial Span subtest of the Wechsler Memory Scale, 3rd edition (Wechsler, 1997b) assesses a person’s attention span for a sequence of positions on a board with randomly placed blocks. It is interpreted as a visual form of digit span tests that assess auditory attention span. Persons assessed observe the examiner tap on a sequence of blocks and are asked to reproduce the positions tapped.

The Controlled Oral Word Association Test or COWA (Borkowski et al., 1967) assesses mental speed and the ability to generate words beginning with one of three letters. The standard letters were replaced with P, A, and R based on their relative frequency of appearance in words in Hindi.

The Category Fluency Test (Borkowski et al., 1967) is similar to the COWA but in this instance persons assessed are asked to generate examples of items fitting a semantic category. In this study, participants were asked to say the names of as many animals as possible in 60 seconds.

The Stroop Color Word Test (Golden, 1978) assesses a person’s ability to inhibit a standard response (reading a word aloud) when confronted with a novel task (stating the color of ink that the word is written in). The measures has three parts in which the person evaluated is asked to (1) reading color name words (“red,” “blue,” “green”); (2) say aloud the colors of small squares on a piece of paper in the same colors; and (3) saying the colors of the ink in which competing words are printed, such as the word “red” printed in blue ink. In this case, the person must inhibit his or her tendency to read the word as “red” and state the color in which it is printed, “blue.”

The Wisconsin Card Sort Test, or WCST (Kongs et al., 2000) tests a person’s ability to learn to sort cards containing colored markers of various shapes and various number of shapes. The person is asked to match cards sequentially to a standard conforming to a rule (match the color no matter the shape or number of items on the card). Persons assessed are only provided with feedback on whether their response is correct. After the person correctly responds to 10 items based on the color rule, the rule is changed without the person being told that it has changed. The person is then told his or her responses are correct when their choices match shapes and then the number of items. Test scores reflect the individual’s mental flexibility and learning capacity.

Neuropsychological tests were administered by a post-doctoral fellow at PGIMER under the supervision of an Indian trained and licensed neuropsychologist. All testing was completed in one session lasting approximately 3 hours. Frequent breaks were given to limit fatigue. Participants were compensated for each visit the equivalent of approximately three US dollars.

Data Analyses

Evaluation of practice effect was complicated by the possibility that the effect might differ between those with and without HIV infection. In addition to the possibility that the levels of performance for each group might differ, practice effect might differ as well. We therefore first assessed each group’s performance at each test administration with basic descriptive statistics and t-tests assessing between-group differences. Plots of each group’s mean performance at each time point were then inspected. Following the procedure recommended by (Chelune, 2003) we used regression analyses to assess the extent to which participants’ performance on the second administration of the measures was related to their performance on the first, taking serostatus, age, gender, education, and time between administrations into account. Finally, the extent to which group performance over time was the same for those with and without HIV infection was evaluated by testing whether the slopes of regression lines were the same for each group (see Figure, below, illustrating slopes for the Color Trails 2).

Figure 1.

Figure 1

Open in a new tab

Results

A total of 89 persons (42 men and 47 women) with and 51 (33 men and 18 women) without HIV infection completed the test battery on two occasions. Data on participants’ education was collected as presented in Table 1. Analyses indicated an association between education and group membership (χ2 [df = 5] = 21.45, p = 0.001). Data in Table 1 show that overall control group participants had completed more years of education than had those with HIV infection. Control participants were somewhat more likely to have been men than women, with genders approximately equally divided in the group with HIV infection and men relatively more frequent in the group without infection (χ2 [df = 1] = 4.00, p = 0.046).

Table 1.

Education and Gender of Participants

HIV+ HIV−
Frequency Percent Frequency Percent
Education
Primary 14 15.7 7 13.7
Middle school 26 29.2 20 39.2
High school 32 36.0 16 31.4
Senior secondary 10 11.2 5 9.8
Graduate 6 6.7 3 5.9
Post Graduate 1 1.1 51 100.0
Totals 89 100.0 100.0
Gender
Men 42 47.0 33 65.0
Women 47 53.0 18 35.0
Totals 89 51
Open in a new tab

Other descriptive data for the sample are provided in Table 2. Participants in both groups were approximately the same age, but those with HIV infection had lower CD4 and higher CD8 counts. The time difference between assessments did not differ between groups. The table also includes the average viral load for participants with HIV infection.

Table 2.

Description of Participants for Continuous Variables

HIV + HIV −
Mean SD Mean SD t df p
Age 30.47 6.15 28.92 5.95 1.45 138 0.15
Time between assessments (days) 392.97 58.39 387.31 37.17 0.62 138 0.54
CD4 611.88 160.98 866.22 326.94 6.14 137 < 0.001
CD8 1,435.44 860.41 725.40 659.99 5.09 137 < 0.001
Viral Load 471,005 1,075,259
ln load 11.29 2.41
Open in a new tab

Table 3 presents the unadjusted means and standard deviations for each group at both assessments, the difference between the two assessments, and a standardized estimate of observed practice effect, if any. The estimate of practice effect was calculated as the difference between performance at the two times divided by its standard deviation, yielding a standard or z score.

Table 3.

Performance of each group on neuropsychological measures
Within groupa
(T1 vs T2)		 Between groupsb
(HIV+ vs HIV−)
Time 1 Mean (SD) Time 2 Mean (SD) Difference (SD) Effect size t p t p Predictors of time 2 performancec
Hopkins Verbal Learning Test Total
HIV+ 21.87 (4.52) 22.11 (4.14) 0.25 (4.93) 0.05 0.47 0.64 0.09 0.93 Baseline; Education; Interactiond
HIV− 22.90 (4.46) 23.08 (3.98) 0.18 (3.61) 0.05 0.35 0.73
Hopkins Verbal Learning Test Delayed Recall
HIV+ 7.43 (2.24) 7.54 (1.91) 0.11 (2.37) 0.05 0.45 0.66 0.09 0.93 Baseline; Education; Interaction
HIV− 7.80 (1.78) 7.88 (1.82) 0.08 (1.48) 0.05 0.38 0.71
Brief Visual Motor Test–Revised Total
HIV+ 14.48 (9.09) 16.44 (9.55) 1.96 (6.15) 0.32 3.00 0.004 0.99 0.32 Baseline; Education
HIV− 13.90 (7.66) 17.04 (9.03) 3.14 (7.74) 0.41 2.89 0.006
Brief Visual Motor Test–Revised Delayed Recall
HIV+ 6.15 (3.69) 6.81 (3.86) 0.66 (2.57) 0.26 2.43 0.02 1.31 0.19 Baseline; Age; Education
HIV− 5.71 (3.28) 7.02 (3.63) 1.31 (3.25) 0.40 2.89 0.006
WAIS-III Digit Symbol
HIV+ 45.62 (15.26) 49.43 (15.62) 3.81 (10.40) 0.37 3.45 0.001 1.35 0.18 Baseline; Education; Time p = 0.10
HIV− 49.94 (15.76) 56.16 (12.95) 6.22 (9.74) 0.64 4.56 < 0.001
WAIS-III Symbol Search
HIV+ 20.54 (6.15) 21.92 (7.00) 1.38 (5.41) 0.26 2.41 0.02 0.97 0.33 Baseline; Age p = 0.050; Education
HIV− 22.67 (7.36) 25.02 (7.14) 2.35 (6.14) 0.38 2.74 0.009
Grooved Pegboard Time
HIV+ 76.85 (14.76) 75.45 (14.47) 1.41 (17.32) 0.07 0.77 0.45 1.20 0.23 Baseline; Age; Education
HIV− 74.29 (12.29) 69.73 (6.56) 4.47 (9.99) 0.45 3.27 0.002
Colored Trails 1
HIV+ 53.01 (17.07) 49.34 (17.69) 2.67 (18.44) 0.14 1.37 0.18 2.09 0.04 Baseline; Education; Interaction
HIV− 57.00 (30.83) 45.84 (13.91) 11.16 (29.65) 0.38 2.69 0.01
Colored Trails 2
HIV+ 114.87 (44.31) 117.42 (46.72) 2.55 (43.39) 0.06 0.55 0.58 2.19 0.03 Baseline; Education; Time diff p = 0.06; Interaction
HIV− 113.55 (58.49) 98.57 (30.83) −14.98 (49.05) 0.31 2.18 0.03
WMS-III Spatial Span
HIV+ 13.45 (2.51) 14.03 (4.44) 0.58 (2.64) 0.22 2.09 0.04 0.50 0.62 Baseline; Gender; Age, Education
HIV− 14.75 (2.95) 15.10 (2.39) 0.35 (2.70) 0.13 0.94 0.35
COWA
HIV+ 16.66 (6.74) 16.92 (7.05) 0.26 (6.68) 0.04 0.37 0.72 0.56 0.58 Baseline; Education; Interaction
HIV− 19.82 (6.39) 19.43 (8.15) 0.39 (6.57) 0.06 0.43 0.67
Category Fluency
HIV+ 12.33 (3.02) 11.40 (2.75) −0.92 (3.67) 0.25 2.37 0.02 2.84 0.005 Baseline; Time diff; Interaction p = 0.053
HIV− 11.80 (3.19) 12.65 (3.01) 0.84 (3.28) 0.26 1.83 0.07
Stroop Color Word score
HIV+ 29.94 (11.51) 25.12 (7.44) −4.82 (11.95) 0.40 3.81 < 0.001 1.38 0.17 Baseline not a predictor; Education
HIV− 30.43 (8.81) 28.35 (6.67) 2.08 (10.11) 0.21 1.47 0.15
WCST
HIV+ 32.62 (10.87) 32.26 (11.70) 0.35 (11.81) 0.03 0.28 0.78 0.10 0.92 Baseline; Education; Interaction
HIV− 36.57 (10.84) 36.43 (10.87) 0.14 (11.48) 0.01 0.09 0.93
Open in a new tab
a

Tests of the difference between time 1 and time 2 performance for each group; all df = 88 for HIV+ and 50 for HIV−.

b

Tests of the difference between change in performance over time for each group; all df = 138.

c

Significant predictors of time 2 performance; see text and data supplement for additional explanation and complete results

d

Significant interaction of participant serostatus by time 1 performance in prediction of time 2 performance. A significant interaction implies that the slope of the regression line for each group is different (also see Figure).

Table 3 also includes results of t tests comparing performance at each time for each group (within group tests; df = 88 for HIV+ and df = 50 for HIV−) as well as group differences in change (between groups tests; df = 138 for all tests). Finally, in its rightmost column, Table 3 presents regression model predictors of time 2 performance for each measure and whether the interaction of serostatus and time 1 performance was significant. This interaction can be interpreted as indicating that the slopes of regression lines for each group were different, implying that patterns of change over administrations were different for the two groups.

Neither group evidenced substantial improvement on verbal memory tasks (HVLT total and delayed recall) but did so on visual memory tasks (BVMT-R total and delayed recall) of approximately 0.30 standard deviation units for those with infection and 0.40 units for those without. Practice effects did not differ significantly for the two groups. The performance of both groups improved across test administrations on the WAIS-III Digit Symbol subtest but effects did not differ between groups. Both groups also improved on the WAIS-III Symbol Search, but again effects did not differ between groups.

On three measures, Grooved Pegboard dominant hand time, Colored Trails 1 and Colored Trails 2, only the control group showed significant improvement over time. While the difference between groups for the Grooved Pegboard was not significant, effects differed between groups for both parts of Color Trails (see Figure, below).

The average score of both groups improved on the WMS-IV Spatial Span subtest, but the improvement was significantly only for those with HIV infection and group differences in improvement were not significant. Neither group’s performance on the COWA changed substantially over administrations. On Category Fluency, the performance of individuals with HIV infection actually declined significantly over administrations while the performance of controls improved at a level that approached significance. For this measure, the change in scores was significantly different between groups. The average performance of those with HIV infection significantly declined over administrations on the Stroop Color Word task although the size of this improvement (lower scores represent improved performance) was not significantly greater than that of controls. Finally, neither group’s performance on the WCST changed substantially over repeated administrations. The average practice effect across groups and measures was roughly 0.30, about one third of a standard deviation. Control participants, in general, showed larger improvements than did those with HIV infection, although in most cases this difference was not statistically significant.

Given these observed group differences and the possible influence of education, age, gender and time difference between assessments, regression models were used to assess the effect of these factors as well as baseline performance and serostatus on performance at the second time point. Performance on the second assessment was predicted by participants’ initial performance, age, education, gender, and serostatus. The difference in the relation between baseline performance and second performance based on serostatus was assessed by including an interaction term in regression models testing whether the slopes of regression equations were the same for each group.

Significant predictors in these models are listed in Table 3. Results indicate that baseline performance on each measure, except for the Stroop Color Word score, was a significant predictor of subsequent performance on the same measure. This finding suggests that any observed improvements in performance may at least partly been the result of previous performance, supporting the interpretation of changes as resulting from practice effect. In 13 of 14 cases, level of education was also a positive predictor of performance on the second administration of the measure emphasizing the possible role that education may have in determining performance on these measures. For only three measures, the time difference between assessments either reached or approached significance as a predictor. For WAIS-III Digit Symbol, there was a modest negative effect of time between assessments that approached significance (i.e., a longer time between assessment was related to poorer subsequent performance), while for Colored Trails 2, there was a modest positive effect that again only approached significance (i.e., a longer time between assessments was related to better subsequent performance). For Category Fluency, the time difference between assessments had a significant negative impact on subsequent performance.

The slope of regression lines for CTT2 differed for the two groups. As illustrated in Figure 1, the performance of control participants significantly improved over test administrations (on this measure, better performance means fewer seconds taken to complete the task) while the performance of persons with HIV infection actually was slightly worse.

Discussion

The purpose of this study was to investigate practice effect on common neuropsychological measures among HIV infected and uninfected individuals in northern India. It was hoped that results would provide information that could assist in interpreting data on repeated assessments in this population. Results of this study show that in many cases, both groups showed improved scores over time, even with an interval of more than one year. Baseline performance for most measures was a significant predictor of later performance even after taking age, gender, education, and time difference between assessments into account, supporting our interpretation of observed differences as indicating practice effect. Data provide an approximate guide of the magnitude of likely practice effect on these measures for both groups, allowing researchers and clinicians to use this information in interpreting observed changes in neuropsychological test performance over time.

Group differences were found in improvement over time, with those without HIV infection often improving more, though not statistically significantly better, than those with infection. The pattern of differences also is noteworthy, since group differences in improvement over time were observed in measures sensitive to the effects of HIV infection, such as psychomotor speed. On measures such as Digit Symbol, the Grooved Pegboard, and Colored Trails 1, control participants improved substantially more than did persons with HIV infection, as evidenced by the size of standardized differences between administrations. On one measure of executive function (Category Fluency), persons with infection actually significantly declined in level of function while controls participants improved. The difference between groups was also significant. A similar pattern was seen for CTT2, where the performance of persons with infection declined while the performance of controls improved significantly. Again, the difference in change was significant.

These findings are in many respects similar to reports by others who investigated practice effect in controls and persons with HIV infection (Duff et al., 2007; Reger et al., 2002). One important exception is our current failure to find substantial change for either group on verbal learning measures. The reason for failing to find an improvement across administrations is unclear but merits further study.

Measures most sensitive to cerebral dysfunction in HIV (Woods et al., 2009) showed the greatest differences, with Color Trails 2 and category fluency actually showing poorer performance on the second administration. This finding highlights the potential significance of HIV infection for cognitive function, even in persons with intact immune systems (as assessed by CD4 and CD8 counts) but who were not virally suppressed (as indicated by detectable viral loads).

Limitations of this study: The initial aim of this study was to evaluate practice effect on repeated administrations of a neuropsychological battery in persons in northern India with and without HIV infection. In this connection, results show that over a period averaging slightly more than one year, both those with and without HIV infection evidence modest increases in their baseline performance on a number of cognitive measures. In most instances, persons without HIV infection improved their performance to a greater degree than did those with infection. This observation, however, must be tempered by the fact that uninfected participants had somewhat higher levels of education and that education by itself was associated with change in performance on several measures.

In conclusion, the present study provides information on practice effects and potential cognitive decline for use in northern India among Hindi speaking individuals. The findings have particular relevance for use with individuals with HIV-1 clade C infection, as data on the neurocognitive impact of infection with this clade are limited. Moreover, as HIV/AIDS continues to affect persons across the globe and as treatment becomes more widely available, thus extending the lives of those infected the ability of HIV care providers to accurately detect cognitive dysfunction will be critical for care and treatment planning throughout patients’ lives. In addition, these data will be useful to assess the residual neurocognitive deficits among HIV-1 infected individuals as antiretroviral interventions are more widely available in norther India.

Acknowledgments

Supported by: NIH grant # RO1 NS-055653 (M. Kumar, P.I.)

Footnotes

Conflict of Interest Statement:

The authors declare that they have no conflict of interest.

References

  1. Bartels C, Wegrzyn M, Wiedl A, Ackermann V, Ehrenreich H. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neurosci. 2010;11:118. doi: 10.1186/1471-2202-11-118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Benedict RHB. Brief Visuospatial Memory Test—Revised. Psychological Assessment Resources, Inc.; Odessa, FL: 1997. [Google Scholar]
  3. Benedict RHB, Schretlen B, Groninger L, Brandt J. Hopkins Verbal Learning Test – Revised: Normative data and analysis of enter-form and test-retest reliability. 1998;12:43–55. [Google Scholar]
  4. Borkowski JG, Benton AL, Spreen O. Word fluency and brain damage. 1967;5:135–140. [Google Scholar]
  5. Calamia M, Markon K, Tranel D. Scoring higher the second time around: meta-analyses of practice effects in neuropsychological assessment. Clin Neuropsychol. 2012;26:543–570. doi: 10.1080/13854046.2012.680913. [DOI] [PubMed] [Google Scholar]
  6. Calamia M, Markon K, Tranel D. The robust reliability of neuropsychological measures: meta-analyses of test-retest correlations. Clin Neuropsychol. 2013;27:1077–1105. doi: 10.1080/13854046.2013.809795. [DOI] [PubMed] [Google Scholar]
  7. Chelune GJ. Assessing reliable neuropsychological change. In: Franklin RD, editor. Prediction in forensic and neuropsychology: Sound statistical practices. Lawrence Erlbaum; Mahwah, NJ: 2003. pp. 123–147. [Google Scholar]
  8. D’Elia LF, Satz P, Uchiyama CL, White R. Color Trails Test. Psychological Assessment Resources; Odessa, FL: 1996. [Google Scholar]
  9. Duff K, Beglinger LJ, Schultz SK, Moser DJ, McCaffrey RJ, Haase RF, Westervelt HJ, Langbehn DR, Paulsen JS. Practice effects in the prediction of long-term cognitive outcome in three patient samples: a novel prognostic index. Arch Clin Neuropsychol. 2007;22:15–24. doi: 10.1016/j.acn.2006.08.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gannon P, Khan MZ, Kolson DL. Current understanding of HIV-associated neurocognitive disorders pathogenesis. Curr Opin Neurol. 2011;24:275–283. doi: 10.1097/WCO.0b013e32834695fb. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Golden CJ. Stroop Color Word Test. Stoelting; Chicago: 1978. [Google Scholar]
  12. Gupta JD, Satishchandra P, Gopukumar K, Wilkie F, Waldrop-Valverde D, Ellis R, Ownby R, Subbakrishna DK, Desai A, Kamat A, Ravi V, Rao BS, Satish KS, Kumar M. Neuropsychological deficits in human immunodeficiency virus type 1 clade C-seropositive adults from South India. J Neurovirol. 2007;13:195–202. doi: 10.1080/13550280701258407. [DOI] [PubMed] [Google Scholar]
  13. Heaton RK, Clifford DB, Franklin DR, Jr, Woods SP, Ake C, Vaida F, Ellis RJ, Letendre SL, Marcotte TD, Atkinson JH, Rivera-Mindt M, Vigil OR, Taylor MJ, Collier AC, Marra CM, Gelman BB, McArthur JC, Morgello S, Simpson DM, McCutchan JA, Abramson I, Gamst A, Fennema-Notestine C, Jernigan TL, Wong J, Grant I. HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology. 2010;75:2087–2096. doi: 10.1212/WNL.0b013e318200d727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kongs SK, Thompson LL, Iverson GL, Heaton RK. Wisconsin Card Sort Test-64 Card Version: Professional Manual. Psychological Assessment Resources; Odessa, FL: 2000. [Google Scholar]
  15. Lafayette Instrument. Grooved Pegboard Test User Instructions. Lafayette Instrument; Lafayette, In: 2002. [Google Scholar]
  16. Reger M, Welsh R, Razani J, Martin DJ, Boone KB. A meta-analysis of the neuropsychological sequelae of HIV infection. J Int Neuropsychol Soc. 2002;8:410–424. doi: 10.1017/s1355617702813212. [DOI] [PubMed] [Google Scholar]
  17. Robertson K, Liner J, Hakim J, Sankale JL, Grant I, Letendre S, Clifford D, Diop AG, Jaye A, Kanmogne G, Njamnshi A, Langford TD, Weyessa TG, Wood C, Banda M, Hosseinipour M, Sacktor N, Nakasuja N, Bangirana P, Paul R, Joska J, Wong J, Boivin M, Holding P, Kammerer B, Van RA, Ive P, Nath A, Lawler K, Adebamowo C, Royal W, III, Joseph J. NeuroAIDS in Africa. J Neurovirol. 2010;16:189–202. doi: 10.3109/13550284.2010.489597. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Strauss E, Sherman EMS, Spreen O. A compendium of neuropsychological tests: Administration, norms, and commentary. Oxford University Press, Inc.; New York, NY: 2006. [Google Scholar]
  19. Tyor W, Fritz-French C, Nath A. Effect of HIV clade differences on the onset and severity of HIV-associated neurocognitive disorders. J Neurovirol. 2013;19:515–522. doi: 10.1007/s13365-013-0206-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Waldrop-Valverde D, Ownby RL, Jones DL, Sharma S, Nehra R, Kumar AM, Prabhakar S, Kumar M. Neuropsychological test performance among healthy persons in northern India: development of normative data. J Neurovirol. 2015;21:433–438. doi: 10.1007/s13365-015-0332-4. [DOI] [PubMed] [Google Scholar]
  21. Wechsler D. Manual for the Wechsler Adult Intelligence Scale–Third Edition (WAIS-III) Pearson Assessment; San Antonio TX: 1997a. [Google Scholar]
  22. Wechsler D. Manual for the Wechsler Memory Scale. 3rd. Pearson Assessment; San Antonio, TX: 1997b. [Google Scholar]
  23. Woods SP, Moore DJ, Weber E, Grant I. Cognitive neuropsychology of HIV-associated neurocognitive disorders. Neuropsychol Rev. 2009;19:152–168. doi: 10.1007/s11065-009-9102-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Yepthomi T, Paul R, Vallabhaneni S, Kumarasamy N, Tate DF, Solomon S, Flanigan T. Neurocognitive consequences of HIV in southern India: a preliminary study of clade C virus. 2006:424–430. doi: 10.1017/s1355617706060516. 2006/08/15. [DOI] [PubMed] [Google Scholar]

RESOURCES