Reaction Time Variability in HIV-Positive Individuals

Abstract

Progression of HIV/AIDS is frequently associated with frontal/subcortical dysfunction and mean reaction time (RT) slowing. Beyond group means, within-subject variability of RT has been found to be particularly sensitive to frontal/subcortical dysfunction in other populations. However, the possible relevance of RT variability to HIV/AIDS patients remains unknown. This study evaluated the relationships between RT variability and indicators such as neurocognitive, behavioral, and immunological status. A total of 46 HIV-positive adults on antiretroviral medication regimens were included in this study. Overall performance of this sample was poorer than normative means on measures of RT latency, RT variability, and traditional neurocognitive domains. Results demonstrated that the measures of RT variability were associated with global cognition, medication adherence rates, and peak immunological dysfunction, above and beyond the effects of RT latency. These preliminary findings suggest that measures of RT variability may provide enhanced sensitivity to neurocognitive disease burden in HIV/AIDS relative to more traditional measures of mean RT or cognitive function.

Studies have documented that mean reaction time (RT) slowing commonly occurs in HIV/AIDS patients, with one meta-analysis using Brinley plots estimating an 11.7% overall increase in RT latency among symptomatic HIV-positive adults when compared with HIV-negative controls (Hardy & Hinkin, 2002). Along with impairments in information processing speed, motor function, executive function, attention, and memory, RT slowing in this population appears to be the result of frontal-subcortical dysfunction associated with the progression of illness (Cysique, Maruff, & Brew, 2006a; Heaton et al., 1995). Additionally, a number of studies have found slower mean RT to be associated with immunological compromise (i.e., lower CD4 count; Gonzalez et al., 2003; Ogunrin, Odiase, & Ogunniyi, 2007).

Beyond mean RT latency, many investigators have suggested that intra-individual variability of RT (i.e., RT inconsistency) may also be an important supplemental indicator of neurocognitive functioning (see, e.g., Bunce et al., 2007; Stuss, Murphy, Binns, & Alexander, 2003). Results from studies of aging, for example, have demonstrated higher RT variability among older adults than that among younger adults (MacDonald, Hultsch, & Dixon, 2003; West, Murphy, Armilio, Craik, & Stuss, 2002) and lower RT variability among nonimpaired older adults than that among older adults with identified cognitive impairment (Dixon et al., 2007; Strauss, Bielak, Bunce, Hunter, & Hultsch, 2007). Considering the psychometric overlap that exists between indicators of RT latency and RT variability (i.e., greater latency is typically associated with greater variability), questions have been raised about the incremental utility of RT variability beyond mean level of RT (Christensen et al., 2005; Salthouse & Berish, 2005). However, in studies of RT variability in aging, significant relationships between age (or age-related diagnoses of cognitive impairment) and RT variability have often been shown to persist even after controlling for differences in RT latency (Dixon et al., 2007; MacDonald, Hultsch, & Bunce, 2006; MacDonald et al., 2003; Strauss et al., 2007), and factor analytic studies have found that the RT latency and variability diverge across a wide range of clinical populations (Kelly, 2000; Kremen, Seidman, Faraone, Pepple, & Tsuang, 1992; Mirsky, Anthony, Duncan, Ahearn, & Kellam, 1991; Pogge, Stokes, & Harvey, 1994), including those with HIV (Levine et al., 2008). Increased intra-individual RT variability has also been demonstrated among adults with traumatic brain injury (Stuss, Pogue, Buckle, & Bondar, 1994; Stuss et al., 1989) and multiple sclerosis (Bruce, Bruce, & Arnett, 2010).

Overall, evidence regarding the neuroanatomical mechanisms of RT variability remains rather limited. However, magnetic resonance imaging studies in normal aging populations have demonstrated that elevated RT variability is associated with reduced white matter volume (Walhovd & Fjell, 2007) and a greater proportion of white matter hyperintensities (Bunce et al., 2007), independently of age. Interestingly, Bunce and colleagues (2007) found that RT variability was related only to hyperintensities in frontal white matter, and not to those in temporal, parietal, or occipital areas. These results were consistent with those of Stuss and colleagues (2003), who found that among individuals with focal brain lesions of mixed etiology, only frontal lesions were associated with increased RT variability.

Considering findings suggesting that the integrity of frontal and subcortical brain areas may be crucial to maintaining consistency of performance and minimizing RT variability in other populations, it is surprising that RT variability has not been more extensively investigated in HIV/AIDS patients. Specifically, frontal/subcortical dysfunction is implicated in HIV-associated cognitive decline (Cysique et al., 2006a; Heaton et al., 1995), and as such the value of RT variability as an incremental marker of neurocognitive disease burden in HIV/AIDS patients beyond RT latency warrants investigation. The only existing study to our knowledge, conducted within our laboratory, employed exploratory factor analysis and found evidence for a latent cognitive factor that was heavily weighted by RT variability (Levine et al., 2008). This factor was independent from RT latency and was uniquely associated with medication adherence. However, the heterogeneous sample and exploratory methods used for the previous study necessitate further investigation. For example, the degree to which individual indicators of RT latency and RT variability might tap aspects of cognition that are related to fluctuations in medication adherence and immunocompetence remains unclear. Potentially, these RT measures may be of value in exploring mechanisms of HIV-related neurocognitive decline and in detecting its possible onset.

In the current study, we hypothesized that RT variability would be related to global cognitive ability, indicators of immune status, and behavioral functioning (antiretroviral medication adherence) in a sample of HIV-positive individuals. Parallel relationships were also examined for mean RT latency and indicators of cognitive ability. On the basis of our previous findings in HIV/AIDS patients (Levine et al., 2008) and those in other populations (Dixon et al., 2007; MacDonald et al., 2003; Strauss et al., 2007), we expected that measures of RT variability would be significantly and incrementally associated with indicators of behavioral and immunological status relative to mean RT latency.

Materials and Methods

Participants

Study participants were 46 HIV-positive adults who were prescribed self-administered highly active antiretroviral therapy (HAART) at the time of participation. Recruitment was conducted from community agencies in the Los Angeles area that specialize in providing services to HIV-infected individuals, and using fliers posted in infectious disease clinics at university-affiliated medical centers. Individuals with current (initial evaluation or follow-up) alcohol or drug dependence, psychotic disorder, history of stroke, seizure, anoxic injury, traumatic brain injury with loss of consciousness >30 min, or other neurological illnesses were excluded. Baseline demographic and health characteristics of the sample are presented in Table 1.

Table 1.

Participant demographic and health characteristics

Characteristic		Valid N
Women, n (%)	9 (19.6%)	46
Age, in years, M (SD)	41.52 (7.34)	46
Education, in years, M (SD)	13.35 (1.90)	46
Estimated verbal IQ, M (SD)	106.64 (9.67)	46
Past alcohol dependence, n (%)	13 (28.3%)	46
Past substance dependence, n (%)	28 (60.9%)	46
BDI-II score, M (SD)	11.48 (8.91)	46
Hepatitis C, n (%)	8 (17.4%)	46
CD4 count (most recent), M/median (SD)	492.18 / 378 (306.20)	45
HIV viral load (most recent), median (IQR)	116.50 (0–1644)	42
Meet criteria for AIDS, n (%)	31 (67.4%)	46
% medication adherence, M (SD)	75.61 (16.94)	46
Ethnicity/race
Hispanic or Latino/a, n (%)	10 (21.7%)
Native American, n (%)	0 (0%)
Asian/Pacific Islander, n (%)	2 (4.3%)
African American/Black, n (%)	25 (54.3%)
Caucasian/White, n (%)	7 (15.2%)
# Multiracial (%)	2 (4.3%)

Note: BDI-II = Beck Depression Inventory-II.

Measures

Reaction Time Latency and Variability

Participants completed the Conners' Continuous Performance Test, Second Edition (CPT-II; Conners, 2000). The CPT-II is a computerized vigilance test that requires examinees to press the space bar as quickly as possible whenever a target (any letter other than ‘X’) appears on the screen, and to inhibit this response in the presence of the letter ‘X’. Ninety percent (90%) of letters presented are targets. Each letter is presented for 250 ms, with 1, 2, or 4 s between letters (the inter-stimulus interval [ISI]). Full-test administration includes a 1-min practice block and approximately 14 min of testing divided into six blocks. Each block can be divided into three sub-blocks of 20 trials for which the ISI is set to 1, 2, or 4 s, respectively.

CPT-II outcome variables are t-scores (standardized by age and gender; Conners, 2000), with lower values representing better scores (faster or more consistent performance). Hit RT is the only indicator of RT central tendency (i.e., mean RT latency); all other CPT-II variables chosen for analysis represent one or more aspects of trial-to-trial RT variability. Hit SE represents the standard error of the participant's Hit RT across all test trials, whereas Hit SE Variability represents the standard deviation of Hit SE across trial sub-blocks. Hit SE Block Change represents the overall slope of change in Hit SE across blocks, with higher values indicating an increase in variability of Hit RT over the course of the test.

Cognitive Ability

Participants completed a comprehensive fixed battery of neuropsychological instruments that was designed to evaluate those cognitive domains most sensitive to HIV/AIDS, including attention, processing speed, learning and memory, motor function, and executive function (see Appendix for individual indicators within each domain and sources of normative data). Raw test scores were converted to demographically corrected t-scores using published normative data. Domain t-scores were computed as the average of individual measured t-scores grouped by cognitive domain for each participant, and global t-scores were computed as the average of the domain t-scores.

Medication Adherence

Adherence to an antiretroviral medication was tracked prospectively over approximately 6 months using the Medication Event Monitoring System (MEMS [Aprex, Union City, CA]), which employs a pressure-activated microprocessor in the medication bottle cap to record the date, time, and duration of dosing events. These data were later downloaded from the bottle cap using a personal computer. In order to maximize the accuracy of the MEMS data, participants were instructed not to use pill organizers or take “pocket doses” (e.g., removing multiple pills for later use), and to open the MEMS cap only when they are taking a dose. For use in data analysis, MEMS adherence was calculated as total doses taken/total doses prescribed.

Additional Measures

During participants' initial visit, substance use modules from the Structured Clinical Interview for DSM-IV were administered to assess for alcohol or substance abuse or dependence. The Beck Depression Inventory-II (BDI-II) was administered as an indicator of current depressive symptomatology. Estimated premorbid Verbal IQ was obtained using total score from the American Version of the National Adult Reading Test (AMNART). Immunological variables were provided by participants from medical records during the initial visit, including most recent CD4 count and viral load (each obtained M = 2.43 [SD = 1.55] months previously), lowest CD4 count (obtained M = 39.93 [SD = 36.55] months previously), and highest viral load (obtained M = 39.13 [SD = 31.31] months previously).

Procedure

All research methods and procedures were approved by IRB panels at UCLA and the West Los Angeles VA Medical Center. After providing written informed consent, participants completed a detailed demographic questionnaire and structured psychiatric interview, and fixed neuropsychological evaluations were administered by trained psychometrists under the supervision of a board-certified neuropsychologist. Participants then received instruction in how to use the MEMS caps and were scheduled to return at intervals of approximately 1 month for six follow-up visits. Participants completed the CPT-II during a follow-up visit, M = 2.47 (SD = 1.58) months from the baseline visit.

Analysis

The SPSS 16.0 statistical package was used for all data analyses. Neurocognitive variables were screened for extreme outliers (z > 4.0 or z < −4.0), which were truncated to prevent the undue influence of single scores on linear models and reduce Type I and Type II error rate. These truncations were necessary only for CPT-II variables, which included selected values reflecting unusually poor performance (2.7% of CPT-II data points total). All variables used in primary analyses were screened for violations of normality, and HIV viral load variables were log₁₀-transformed for the purposes of normalization. However, despite transformation, the Kolmogorov–Smirnov test demonstrated significant departure from normality for most recent viral load (Z = 1.50, p = .02). Therefore, nonparametric statistics were used for all relevant analyses of this variable.

Results

Participant demographic and health characteristics are presented in Table 1. No demographic or psychiatric characteristics were significantly related to indicators of RT latency or RT variability. Descriptive statistics of CPT-II variables and neurocognitive domains are presented in Table 2. Data from HIV-negative controls were not available from which to make within-study comparisons of performance. However, mean performance of study participants was .54–1.10 SD poorer than demographically adjusted normative means for neurocognitive domain scores, and was .41 to .94 SD poorer than demographically adjusted normative means for CPT-II variables of interest. Considering the previously documented effects of HIV/AIDS on neurocognitive abilities (Cysique et al., 2006a; Heaton et al., 1995) along with the mean estimated pre-morbid verbal IQ in this sample of 106.64 (SD = 9.67), it is unlikely that this pattern of performance can be attributed to pre-morbid cognitive weakness.

Table 2.

Descriptive statistics of neurocognitive variables

Variable	t-Score, M (SD)
CPT-II Hit RTa	59.42 (12.47)
CPT-II Hit SEa	58.77 (14.11)
CPT-II Hit SE variabilitya	56.86 (12.93)
CPT-II Hit SE block changea	54.14 (12.84)
Global cognitive abilityb	42.75 (5.37)
Domain score: Attentionb	44.49 (6.74)
Domain score: Processing speedb	44.58 (7.04)
Domain score: Learning and memory b	43.10 (10.74)
Domain score: Motor functionb	39.05 (9.61)
Domain score: Executive functionb	42.52 (7.79)

Note: n = 46 for all analyses. CPT-II = Conners' Continuous Performance Test-II; RT = reaction time; SE = standard error.

^aDemographically standardized t-scores; lower values indicate better performance.

^bDemographically standardized t-scores; higher values indicate better performance.

As shown in Table 3, associations with indicators of interest varied between mean RT latency, measures of intra-individual RT variability, and traditional neurocognitive domains. RT latency (Hit RT) was significantly associated with global cognitive ability (r = −.30, p = .04) and indicators of recent immunological status (most recent CD4 count r = −.32, p = .03; most recent viral load r_s = .36, p = .02). Similarly, measures of RT variability were also associated with global cognitive ability (Hit SE r = −.37, p = .01; Hit SE variability r = −.44, p = .003) and most recent CD4 count (Hit SE r = −.32, p = .03). However, measures of RT variability showed additional associations with medication adherence (Hit SE r = −.36, p = .02; Hit SE variability r = −.38, p = .01; Hit SE block change r = −.29, p = .05), lowest CD4 count (Hit SE block change r = −.31, p = .04), and highest viral load (Hit SE variability r = .35, p = .03). By comparison, among traditional indicators of cognitive ability, only learning/memory was associated with antiretroviral medication adherence (r = −.39, p = .01), and no cognitive indicators were significantly associated with CD4 count (lowest or most recent) or HIV viral load (highest or most recent) in this sample.

Table 3.

Correlations between cognition, adherence, and immunological status

Cognitive variable (characteristics)	Global cognitive ability	Medication adherence	Most recent CD4 count	Lowest CD4 count	Most recent viral loada,b	Highest viral loada
Valid N	46	46	45	45	42	40
Global cognitive abilityc	1	.23	−.12	.00	−.04	.04
Domain score: Attentionc	.53**	−.01	−.18	−.22	−.03	−.06
Domain score: Processing speedc	.76**	.16	.13	.13	−.07	.03
Domain score: Learning/memoryc	.62**	.39*	.09	.11	−.12	.10
Domain score: Motor functionc	.53**	−.02	−.27	−.01	−.01	−.03
Domain score: Executive functionc	.80**	.15	−.16	−.09	.09	.07
CPT-II Hit RTd (mean RT latency)	−.30*	−.24	−.32*	−.17	.36*	.16
CPT-II Hit SEd (trial variability)	−.37*	−.36*	−.32*	−.27	.20	.22
CPT-II Hit SE variabilityd (trial and sub-block variability)	−.44**	−.38**	−.20	−.20	.13	.35*
CPT-II Hit SE block changed (trial and block trend variability)	−.24	−.29†	−.27	−.31*	.02	.09

Note: Pearson correlations used, except where specified. CPT-II = Conners' Continuous Performance Test-II; RT = reaction time; SE = standard error.

*p < .05.

**p < .01.

†p = .05.

^aLog₁₀-transformed for purposes of normalization.

^bNon-parametric Spearman correlation used.

^cDemographically standardized t-scores; higher values indicate better performance.

^dDemographically standardized t-scores; lower values indicate better performance.

To evaluate the incremental relationships between RT variability and clinical variables beyond their associations with mean RT latency, hierarchical regressions were then performed for each significant clinical correlation with RT variability. Each of these regressions included RT latency (Hit RT) as a predictor in the first block, with the relevant indicator of RT variability added in the second block. Results of these regression analyses are provided in Table 4. For global cognitive ability, Hit SE variability (B = −.39, p = .02) accounted for an additional 10% of variance beyond that accounted for by RT latency (B = −.09, p = .60), whereas the incremental association of Hit SE was non-significant (Hit SE B = −.36, p = .13; Hit RT B = −.01, p = .96; Δr² = .05). In analyses of medication adherence, Hit SE variability (B = −.63, p = −.04) accounted for an additional 9% of variance beyond RT latency (B = −.04, p = .80), whereas Hit SE (B = −.46, p = .06; Hit RT B = .13, p = .59; Δr² = .08) and Hit SE block change (B = −.24, p = .12; Hit RT B = −.16, p = .30; Δr² = .05) each fell short of significance. Indicators of RT variability were not able to account for significant incremental variance in most recent CD4 count (Hit SE B = −.19, p = .42; Hit RT B = −.18, p = .45; Δr² = .01), and incremental associations with lowest CD4 count fell slightly short of significance (Hit SE block change B = −.29, p = .07; Hit RT B = −.10, p = .50; Δr² = .08). However, RT variability (B = .37, p = .048) predicted an additional 10% variance in highest viral load beyond that accounted for by RT latency (B = −.03, p = .87).

Table 4.

Incremental predictive value of RT variability beyond RT latency

Dependent variable	Block	df	ΔF	Δr2	p	CPT-II predictorsb	B	p
Global cognitiona	1	1, 44	4.35	.09	.04	Hit RT	−.30	.04
	2	1, 43	2.40	.05	.13	Hit SE	−.36	.13
						Hit RT	−.01	.96
	2	1, 43	5.57	.10	.02	Hit SE variability	−.39	.02
						Hit RT	−.09	.60
Medication adherence	1	1, 44	2.63	.06	.11	Hit RT	−.24	.11
	2	1, 43	3.79	.08	.06	Hit SE	−.46	.06
						Hit RT	.13	.59
	2	1, 43	4.44	.09	.04	Hit SE variability	−.63	.04
						Hit RT	−.04	.80
	2	1, 43	2.47	.05	.12	Hit SE block change	−.24	.12
						Hit RT	−.16	.30
Most recent CD4 count	1	1, 43	4.94	.10	.03	Hit RT	−.32	.03
	2	1, 42	.68	.01	.42	Hit SE	−.19	.42
						Hit RT	−.18	.45
Lowest CD4 count	1	1, 43	1.34	.03	.25	Hit RT	−.17	.25
	2	1, 42	3.59	.08	.07	Hit SE block change	−.29	.07
						Hit RT	−.10	.50
Highest viral loadc	1	1, 38	1.05	.03	.31	Hit RT	.16	.31
	2	1, 37	4.18	.10	.048	Hit RT variability	.37	.048
						Hit RT	−.03	.87

Note: Standardized Beta weights shown. CPT-II = Conners' Continuous Performance Test-II; RT = reaction time; SE = standard error.

^aDemographically standardized t-scores; higher values indicate better performance.

^bDemographically standardized t-scores; lower values indicate better performance.

^cLog₁₀-transformed for purposes of normalization.

Discussion

Consistent with research in other populations demonstrating the value of intra-individual RT variability as an indicator of neurocognitive status (Bruce et al., 2010; Dixon et al., 2007; MacDonald et al., 2003; Strauss et al., 2007), results of this study of HIV-positive individuals demonstrated that elevated RT variability was associated with poorer overall cognitive ability. Higher RT variability in this study was also associated with indicators of recent immunological status (i.e., recent CD4 count and viral load), peak immunological dysfunction (i.e., nadir CD4 count and highest viral load), and, consistent with preliminary indications from our previous study (Levine et al., 2008), poorer adherence to antiretroviral medication regimens. This latter finding is of particular interest, as medication adherence represents an important indicator of functional ability in this population. Recent findings suggest that medication adherence and cognitive dysfunction may be reciprocally related in HIV/AIDS (Ettenhofer, Foley, Castellon, & Hinkin, 2010). However, it was not possible to evaluate the degree to which the current findings might represent the potential for RT variability to predict individuals' ability to adhere, versus any possible protective effect of antiretroviral adherence on RT variability.

In comparison with RT variability, mean RT latency was associated with a relatively narrower set of indicators, including recent immunological status and global cognitive ability, but not medication adherence, lowest CD4 count, or highest viral load. Follow-up analyses demonstrated that RT variability accounted for additional variance in global cognition, medication adherence, and peak immunological dysfunction beyond that explained by RT latency. Moreover, the relationship between RT latency and global cognition was no longer significant when RT variability was included in the model. Beyond demonstrating its incremental value, this finding suggests that RT variability may also be more fundamentally related to global cognition than RT latency in this population.

Among indicators of cognitive ability, only learning/memory was associated with medication adherence, and no significant associations between cognitive ability and immunological status were found in this sample. Considering our modest sample size, these findings should not be interpreted as evidence for the absence of relationships between the variables in question. For example, significant relationships between medication adherence and executive function have been demonstrated in previous research (Hinkin et al., 2002). Instead, the current pattern of results is best interpreted as preliminary evidence that RT variability may be more sensitive to poor medication adherence and recent immunological dysfunction than indicators of processing speed, attention, executive functions, and motor functioning, and may be more sensitive to poor medication adherence and peak immunocompromise than RT latency.

The potential significance of this latter finding is highlighted further by recent research suggesting that peak immunosupression may be more strongly related to neurocognitive decline than day-to-day fluctuation in immunological status (Cysique, Maruff, & Brew, 2006b). Neither neuroanatomical data nor a suitable HIV-negative control groups were available in this study to determine whether RT variability was specifically related to HIV-related frontal/subcortical disturbance. However, our participants' above-average mean estimated premorbid verbal IQ suggests that the poorer-than-average mean performance demonstrated on neurocognitive tasks and indicators of RT latency and variability probably represent true weaknesses in this population. Therefore, in combination with the previously documented relationships between RT variability and neuropathology in aging and mixed neurological samples (Bunce et al., 2007; Stuss et al., 2003; Walhovd & Fjell, 2007), preliminary findings from this study suggest that RT variability might also provide enhanced sensitivity to important aspects of HIV-associated neurocognitive decline.

Future research on this topic may be strengthened by improved measurement of neurological, behavioral, and immunological status. For example, additional measures of neurological status (e.g., neuroimaging) and behavioral functioning (e.g., driving capacity) would provide valuable points of reference for the neuroanatomical and functional significance of elevated RT variability in HIV/AIDS. Likewise, relationships between RT variables and “most recent” immune indicators in this study may have contracted or otherwise changed during the time frame between the collection of information about immune status and the administration of RT measures. Therefore, a more systematic approach to measurement of immune status may be useful in examining the links between RT variability and immunocompromise in HIV/AIDS.

Additionally, sample size limited the degree to which statistical comparisons could be made between the different aspects of RT variability assessed in this study. Comprehensive examination of different forms of RT variability in future investigations may provide additional information regarding the utility of these novel measures and possible mechanisms of impairment in HIV/AIDS. Finally, in future research, it will be important to compare RT variability and other neurocognitive indicators between HIV-positive and HIV-negative groups in order to more directly verify the extent of elevations in RT variability that may be present.

In conclusion, this study provides a window into the possible relevance of RT variability as an indicator of neurocognitive burden in HIV/AIDS patients, and the potential value of RT variability measures as an adjunct to other methods of neuropsychological assessment. Although neuropsychologists typically focus on mean level of performance rather than consistency of performance, data from this study suggest that by doing so we may be overlooking important sources of information. It is possible that, like the proverbial ‘canary in the coal mine’, increased RT variability may prove to be a harbinger of incipient declines in level of performance. However, the neurobiological mechanisms of RT variability in HIV/AIDS remain unclear. Additional research will be needed to evaluate the degree to which the sources of RT variability in HIV/AIDS may parallel those suggested by research in other populations, and to specify elements of this variability that may be most fundamentally related to various aspects of neurological, immunological, and behavioral functioning.

Funding

This investigation was supported by National Institutes of Health Grant R01 DA13799, and the National Institutes of Health, Ruth L. Kirschstein National Research Service Award T32 MH19535.

Conflict of interest

None declared.

Appendix: Neuropsychological tests by domain, with normative source utilized

Domain/test	Normative source
Processing speed
Trail Making Test, part A, time	Heaton, Grant, and Matthews (1991)
Digit Symbol Coding, total score	Wechsler (1997)
Symbol Search, total score	Wechsler (1997)
Attention
Paced Auditory Serial Addition Test, series 1, total correct	Stuss, Stethem, and Pelchat (1988)
Digit Span, total score	Wechsler (1997)
Letter-Number Sequencing, total score	Wechsler (1997)
Executive function
Trail Making Test, part B, time	Heaton and colleagues (1991)
Stroop Color-Word Test—interference trial, total correct	Selnes and colleagues (1991)
Wisconsin Card Sorting Test—64 card version, total errors	Kongs, Thompson, Iverson, and Heaton (2000)
Letter Fluency (FAS) Total	Selnes and colleagues (1991)
Learning and memory
CVLT-II Immediate Recall Total	Delis, Kramer, Kaplan, and Ober (2000)
CVLT-II Short Delay Free Recall Total	Delis and colleagues (2000)
CVLT-II Long Delay Free Recall Total	Delis and colleagues (2000)
BVMT-R Immediate Recall Total	Benedict (1997)
BVMT-R Delayed Recall Total	Benedict (1997)
Motor Function
Grooved Pegboard Dominant Hand, Time	Heaton and colleagues (1991)
Grooved Pegboard Non-dominant Hand, Time	Heaton and colleagues (1991)

Note: CVLT-II = California Verbal Learning Test-II; BVMT-R = Brief Visuospatial Memory Test-Revised.