Performances on the CogState and Standard Neuropsychological Batteries Among HIV Patients Without Dementia

Abstract

HIV-associated neurocognitive disorders (HAND) remain prevalent but challenging to diagnose particularly among non-demented individuals. To determine whether a brief computerized battery correlates with formal neurocognitive testing, we identified 46 HIV-infected persons who had undergone both formal neurocognitive testing and a brief computerized battery. Simple detection tests correlated best with formal neuropsychological testing. By multivariable regression model, 53% of the variance in the composite Global Deficit Score was accounted for by elements from the brief computerized tool (p<0.01). These data confirm previous correlation data with the computerized battery, yet illustrate remaining challenges for neurocognitive screening.

Introduction

With the development of combination antiretroviral therapy (cART), the prevalence of HIV-associated dementia (HAD) has declined but less severe HIV-associated neurocognitive disorders (HAND) have become prominent [1]. With the transformation of HIV into a chronic medical illness, advanced age and HIV infection may act synergistically to increase the prevalence of HAND [2]. Recent cohort studies have reported the prevalence of HAND to range from 39% to 69% of subjects on cART [3, 4]. The need for brief and psychometrically sound methods to evaluate neurocognitive function is important to identify HAND and limit progressive impairment in one’s capacity to adhere to medical regimens, safely operate a motor vehicle, complete basic activities of daily life, and maintain employment [5–8].

Some available screening tools used to diagnose neurocognitive impairment, such as the International AIDS Dementia Screen or the Mini-Mental Status Exam, are insensitive to early manifestations of functional impairment [9–11]. Conversely, formal neurocognitive testing is time-consuming and requires special training and thus, cannot realistically be used in the typical outpatient setting, where the necessary time, staffing, space, and funding are generally not available. CogState, a commercially available product (http://www.cogstate.com), is a computerized cognitive test battery designed to measure psychomotor performance, attention, memory, and executive functioning: domains frequently impaired in persons with early neurocognitive disorders [12, 13].The battery consists of brief tasks in the form of card games to minimize language and cultural differences. It has previously been validated in persons with HAD [13] and been used in clinical cohort studies to measure change in neurocognitive function and performance [14]. In this analysis, we sought to determine the correlation between CogState and formal neuropsychological testing to detect neurocognitive impairment in a subset of healthy, autonomous HIV-infected persons who are followed in two cohort studies at the Washington University Outpatient HIV Clinic.

Methods

Washington University in St. Louis is a site for two prospective cohort studies evaluating complications of HIV and HIV therapy including cognitive function, one with a validated traditional neurocognitive testing battery and the other with CogState. To determine the utility of the CogState computerized battery as a screening tool, subjects enrolled in both studies were identified to compare the two neurocognitive batteries. Subjects were eligible for participation in this study if they completed an assessment in CHARTER within 6 months of the baseline SUN assessment. Enrollment criteria for the studies have been outlined previously [15, 16]. Both studies were approved by the Washington University IRB and all subjects provided written informed consent.

SUN Study

The Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) is a CDC-funded multi-site prospective observational cohort monitoring complications of antiretroviral therapy and HIV [15]. At baseline and each six-month visit thereafter, participants are evaluated for neurocognitive ability using CogHealth© CogState Ltd., Melbourne, Australia [13]. The computerized battery requires 12–15 minutes to complete and consists of 6 individual tests: two tests of detection evaluating simple reaction times (DET1 and DET2) assess psychomotor function and speed of processing domains; a test of identification as a choice reaction time (IDN) assesses the domains of visual attention and vigilance; the one back test (ONB) assesses the domains of attention and working memory; the monitoring test is a measure of divided attention (MON) that assesses the domain of attention; and the associate learning test (ASSL) assesses the domain of visual learning and memory. Each test is scored based on time to complete the task (speed) and error rate (accuracy).

CHARTER Study

The CHARTER study is an NIH-funded multi-site cohort to explore HIV neurological complications in the context of emerging antiviral treatments such as cART [16, 17]. Participants receive comprehensive neuromedical, neurocognitive, and laboratory examinations. The neurocognitive battery performed in CHARTER requires approximately 1 hour to assess the following domains: attention/psychomotor speed (Trailmaking Test Part A, Symbol Digit Test and Symbol Search Test from WAIS-III, Paced Auditory Serial Addition Task); fine motor speed skills (Dominant and Non-dominant Hand Pegboard Test); learning and memory (Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Figure Memory Learning Test and Story Memory Learning test); executive functioning including working memory (Wisconsin Card Sorting Test, Letter-Number Sequence from WAIS-III), fluency (Controlled Oral Word Association Test and Animal Category Test), and set shifting/response inhibition (Trailmaking Test Part B) [18–30].

Statistical Analysis

Data from the computerized battery were evaluated for normality of data distribution; reaction time measures were log₁₀ transformed due to a positive skew of the distribution and accuracy measures were transformed using Arcsine-root transformation [31]. The raw scores from the neurocognitive tests from CHARTER were converted into T scores corrected for demographic data to minimize the impact of education, age, race/ethnicity, and sex. The Global Deficit Score (GDS) is a composite score calculated by converting the T scores from the CHARTER battery into one summary deficit score ranging from 0 (normal) to 5 (severe neurocognitive impairment) [32]. For the evaluation of construct validity, a correlation matrix was created to determine the correlation between the CogState measures and the CHARTER Study standard neuropsychological measures, including the GDS, using Pearson’s correlation coefficient. To determine the potential value of the CogState battery as a screening tool, we performed a stepwise linear regression analysis with GDS (log transformed to approximate a normal distribution) as the dependent variable and the CogState measures as independent variables. The stepwise regression was performed using the default selection method (forward in, backward out) in SAS. The stepwise regression was validated using 1,000 bootstrapping replicates. The significant Cogstate variables from the stepwise regression were used as independent variables in a multivariable regression model with GDS as the dependent variable to determine the model R².

Results

Forty-six subjects enrolled in both cohort studies were eligible for the present analysis. Table 1 outlines the clinical parameters of the subjects at the time of baseline CogState evaluation between 2004 and 2006. Median CD4 count was 424.5 c/mm³ with 74% of the cohort on cART and 61% with HIV viral load < 400 cp/mL. The median GDS, as calculated from the CHARTER study, for the cohort was 0.47 (range 0.00–2.79), with 24 subjects (52%) having normal function (GDS<0.5) and 22 (48%) having mild to moderate impairment (GDS≥0.5).

Table 1.

Baseline Demographics of Study Participants, the Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) and the CNS HIV Antiretroviral Therapy Effects Research (CHARTER) study.

Characteristic	n=46 n (%)
Gender
Male	33 (72%)
Female	13 (28%)
Race
Caucasian	19 (41%)
African American	26 (57%)
Hispanic	1 (2%)
Median age (range)	40 (21–62)
Median years since HIV diagnosis	5.5 (0.–23)
(range)
Education level
Less than HS	6 (13%)
HS/GED	22 (48%)
Some college	12 (26%)
College graduate	5 (11%)
unknown	1 (2%)
Current substance use
Alcohol	26 (57%)
Marijuana	3 (7%)
Cocaine	2 (4%)
IVDU	1 (2%)
Median nadir CD4 count (range)	255 (0–1020)
Nadir CD4 count < 200c/mm3	15 (33%)
Median current CD4 count (range)	424.5 (79–1300)
On cART	34 (74%)
Median current HIV VL (range)	136.5 (<50 –309,000)
VL range, copies/mL
<400	28 (61%)
400–999	1 (2%)
1000–9,999	6 (13%)
10,000–99,999	10 (22%)
>100,000	1 (2%)
Global Deficit Score
Normal	24 (52%)
Mild impairment	20 (43%)
Moderate impairment	2 (4%)

HIV=human immunodeficiency virus; HS=high school; GED=general educational development; IVDU=intravenous drug user; VL=viral load.

The correlation matrix showing the Pearson product-moment correlation coefficient between each COGSTATE index and the dependent variables from the CHARTER clinical exam, including GDS is shown in Table 2. The speed measures for both simple detection tests (DET1, DET2) and the identification task (IDN), a more complex reaction time task, correlated with the GDS and had the highest levels of correlation across the tests performed in the CHARTER battery. The accuracy components of the CogState battery generally correlated poorly with the individual neurocognitive tests in the CHARTER battery. The accuracy measures for the complex tasks of One-Back Memory (ONB) and the Measure of Divided Attention (MON) were the only accuracy measures that correlated with the GDS derived from the CHARTER testing.

Table 2.

Correlation Matrix between CogState Tests and CHARTER Neurocognitive Tests^§

		SUN Study CogState Tests
		Detection1		Detection2		Identification		One Back		Monitoring		Associate Learning
		Speed	Accuracy	Speed	Accuracy	Speed	Accuracy	Speed	Accuracy	Speed	Accuracy	Speed	Accuracy
CHARTER Domain Measure	Median Test Score‡ (IQR)
Attention/Psychomotor Speed
Trailmaking Test Part A	−0.5 (−1.4,0.1)	−0.42**	0.31	−0.42**	0.26	−0.38*	0.34	−0.16	0.33	−0.12	0.23	−0.05	0.17
Digit Symbol Test (WAIS-III†)	0.15 (−1.1,1.0)	−0.44**	0.22	−0.47**	−0.03	−0.43*	0.16	−0.31	0.40*	−0.11	0.11	−0.13	−0.02
Symbol search test (WAIS-III)	−0.45 (−1.3,0.3)	−0.34*	−0.04	−0.31	0.12	−0.21	0.02	−0.37*	0.31	−0.23	0.01	−0.06	0.06
PASAT-50† Number correct	−0.8 (−1.5,0.2)	−0.53**	0.15	−0.53**	−0.08	−0.49**	−0.04	−0.22	0.45**	−0.18	0.25	−0.07	0.00
Fine Motor Speed Skills
Pegboard Dominant Hand	−0.3 (−1.2,0.2)	−0.46**	0.14	−0.47**	0.09	−0.29	0.40*	−0.15	0.36*	−0.18	0.27	−0.12	0.15
Pegboard non-dominant Hand	−0.7 (−1.3,−0.1)	−0.52**	0.10	−0.33**	−0.09	−0.28	0.21	−0.22	0.35*	−0.12	0.24	−0.23	0.02
Learning and Memory
BVMT† Total Learning	−0.2 (−1.2, 0.6)	−0.34	0.08	−0.11	0.21	−0.23	0.14	−0.32	0.32	−0.21	0.37*	−0.07	0.26
BVMT Delayed Recall	−0.55 (−1.3,0.4)	−0.33	0.10	−0.08	0.15	−0.22	0.17	−0.27	0.28	−0.12	0.38*	−0.12	0.19
HVLT† Total Learning	−0.1 (−1.4,0.9)	−0.42**	0.15	−0.44**	−0.11	−0.20	0.42**	−0.03	0.08	−0.13	0.18	0.04	0.11
HVLT Delayed Recall	−0.1 (−1.5,1.4)	−0.42**	0.16	−0.44**	−0.11-	−0.23	0.36*	0.04	−0.10	−0.05	0.13	−0.02	−0.01
Figure memory learning	−0.65 (−1.4,0.2)	−0.40*	−0.04	−0.35*	−0.09	−0.41*	0.20	−0.34*	0.17	−0.09	0.37*	−0.21	0.15
Figure memory delayed recall	0.0 (−0.3,0.4)	−0.22	0.03	−0.19	−0.22	−0.37*	0.15	−0.24	−0.01	−0.01	−0.02	−0.23	0.05
Story memory learning	−0.85 (−1.8,0.4)	−0.36	−0.04	−0.27	−0.26	−0.10	0.06	0.02	0.09	−0.12	0.17	−0.05	−0.10
Executive Systems Function
Working Memory
WCST† Perseverative Responses	−0.8 (−1.6,0.1)	−0.22	0.04	−0.21	−0.23	0.45**	0.05	−0.15	0.16	−0.08	0.15	−0.23	−0.08
Letter-Number Sequence (WAIS)	−0.6 (−1.3, 0.0)	−0.31	−0.12	−0.15	−0.23	−0.27	0.06	−0.09	0.06	−0.20	0.21	−0.27	−0.30
Fluency
COWAT † Total Correct Word	−0.45 (−1.2,0.0)	−0.30	0.21	−0.28	−0.01	−0.10	0.28	0.00	−0.06	−0.05	0.13	−0.12	−0.29
Animal Category	−0.25 (−1.2,0.6)	−0.16	0.15	−0.24	0.28	0.03	0.27	0.02	0.04	−0.18	0.04	0.06	−0.05
Set shifting/response inhibition
Trailmaking Test Part B	−0.2 (−1.3,0.4)	−0.40*	0.05	−0.54**	−0.01	−0.47*	0.27	−0.13	0.27	0.03	0.15	−0.08	0.06
Global Deficit Score	0.47 (0.26,0.89)	0.52**	−0.22	0.48**	−0.05	0.40*	−0.35	0.16	−0.35*	0.16	−0.40*	0.03	−0.15

^*Denotes p value <0.05.

^**Denotes p value<0.01.

^§Pearson’s product moment correlation was performed.

^†Wechsler adult intelligence scale, Paced Auditory Serial Addition Task, Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Wisconsin Card Sorting Test, Controlled Oral Word Association Test

^‡All tests scores are reported as T scores except the GDS which is a summary score of the entire test battery with higher scores indicating greater impairment on a scale from 0 to 5 with 0−0.5 considered normal cognition.

The stepwise regression analysis identified the following potential independent correlates with GDS: the accuracy and speed of the two simple detection tests (DET1lmn, DET2acc) and 3 more complex tasks: the accuracy of the associate learning (ASSLacc), the accuracy of monitoring tasks (MONacc), and the accuracy of the One Back test (ONBacc). A regression model using GDS as the dependent variable and these measures as independent variables yielded a model R² of 0.53 (p<0.0001) indicating that approximately 53% of the variance in the GDS is explained by these five CogState variables. In the validation analyses using results from the stepwise regression analysis of 1,000 bootstrap replicates MONacc appeared in 96% of the replicates, DET1lmn appeared in 76% of the replicates, DET2acc appeared in 71% of the replicates, ASSLacc appeared in 57% of the replicates and ONBacc appeared in 51% of the replicates. A regression model using GDS as the dependent variable and MONacc, DETlmn and DET2acc (the three variables that appear in more than 70% of the replicates) yields a model R² of 0.39 (p=0.0002).

Discussion

In this study, we compared a brief, self-administered computerized screening battery with formal neurocognitive assessment. With the exception of reaction tests that evaluate functional speed, individual tests in the computerized battery correlated poorly with formal neurocognitive testing. Cognitive slowing is a prominent feature of HAND, and thus, these results are consistent with previous comparisons of the CogState battery to formal neurocognitive functioning although the associations are not as robust for less severe cognitive impairment [13].

While these results illustrate that a single brief screening test may be insensitive to identifying people with neurocognitive impairment, particularly those with mild impairment, the modeling of several parameters from the CogState battery against the Global Deficit Score highlights that there is utility to a brief computerized neurocognitive screening tool. Future prospective studies are needed to determine the sensitivity of the CogState battery to identify HAND. If it proves to be a sensitive tool with established valid cutpoints, this battery could be used to identify persons with mild cognitive impairment. However, in the absence of cutpoints that denote clinical significance, the battery will not achieve optimal clinical utility.

We recognize the limitations of this analysis. It consists of a small group of individuals and lacks a control group. No persons with advanced cognitive imparirment were included. Another limitation to these findings was the lack of correlation between the Associate Memory test and the measures of learning and memory. The CogState battery evaluated here was insensitive to these measures and thus may impact the ability of Cogstate to identify persons with impaired learning ability. Additionally, there was a lack of correlation between CogState indices and the Trailmaking Part B test, which is widely used to reflect executive function with important implications regarding one’s ability to perform activities of daily living independently [33]. Our data fail to confirm the previous work by Cysique et al, in which the strongest correlations were with the Trail Making tests. However, their work focused on persons with HAD. While the Trailmaking Part B test does not necessarily capture all of the components of executive control important for daily living, our findings suggest that CogState may not provide critical information related to early decline in functional independence, though it should be noted that executive function is a heterogeneous construct.

In summary, we found that a compilation of the tests from a brief computerized screening tool for neurocognitive function was correlated to traditional neurocognitive testing among HIV-infected persons. These findings confirm previous reports of correlation between brief computerized CogState battery and standard neuropsychological examination [13], especially for identifying cognitive slowing, a central feature of HIV-associated neurocognitive disorders, although there were differences regarding different domains potentially related to differences in the severity of impairment in the studies [34]. Nevertheless, additional research is needed to fully evaluate the utility of this battery.