Construct Validity of the Item-Specific Deficit Approach to the California Verbal Learning Test (2^nd ed) in HIV Infection

Abstract

Impairment in list learning and recall is prevalent in HIV-infected individuals and is strongly predictive of everyday functioning outcomes. Consistent with its predominant frontostriatal pathology, the memory profile associated with HIV infection is best characterized as a mixed encoding/retrieval profile. The Item-Specific Deficit Approach (ISDA) was developed by Wright et al. (2009) to elicit indices of Encoding, Consolidation, and Retrieval from the well-validated California Verbal Learning Test (CVLT; Delis et al., 1987; 2000). The current study evaluated construct validity of the ISDA for the CVLT-II in 40 persons with HIV-associated neurocognitive disorders (HIV+/HAND+), 103 HIV-infected persons without HAND (HIV+/HAND−), and 43 seronegative comparison subjects (HIV−). Results provided mixed support for the construct validity of ISDA indices. HIV+/HAND+ individuals performed significantly more poorly than persons in the HIV+/HAND− and HIV− groups on ISDA Encoding, Consolidation, and Retrieval deficit indices, which demonstrated adequate classification accuracy for diagnosing HIV+/HAND+ participants and evidence of both convergent (e.g., episodic memory) and divergent (e.g., motor skills) correlations in the HIV+/HAND+ participants. However, highly intercorrelated ISDA indices and traditional CVLT-II measures showed comparable between-groups effect sizes, classification accuracy, and correlations to other memory tests, thereby raising uncertainties about the incremental value of the ISDA approach in clinical neuroAIDS research.

Episodic memory deficits are a common feature of infection with the human immunodeficiency virus (HIV; Carey, et al., 2006; Delis et al., 1994). Episodic memory impairments are detectable in between 40 and 60% of HIV-infected individuals (Heaton et al., 2010), and typically escalate in prevalence and severity as the disease progresses (Reger, Welsh, Razani, Martin, & Boone, 2002). Episodic memory is also among the most sensitive indicators of HIV-associated neurocognitive disorders (HAND; Carey et al., 2004). Most commonly assessed by testing learning and delayed recall of word lists, auditory passages, and visual designs, memory impairments are strongly predictive of poorer daily functioning outcomes, including unemployment (Heaton et al., 1994), poorer driving ability (Marcotte et al., 2000), medication non-adherence (Woods et al., 2009), and lower health-related quality of life (Tozzi et al., 2003). Taken together, the high prevalence and adverse functional impact of HIV-associated episodic memory impairments have clear clinical implications for the assessment of HAND.

In HIV, episodic memory impairment is most often manifested as a mixed encoding and retrieval deficit (e.g., Delis et al., 1994; Woods, Delis, Scott, Kramer, & Holdnack, 2006). Specifically, individuals with HIV typically demonstrate impaired free recall, characterized by limited acquisition and decreased use of high level encoding strategies such as semantic clustering (Peavy et al., 1994), particularly in HAND (Gongvatana et al., 2007). Some research suggests that individuals with HIV also tend to show increased repetition errors during list-learning tasks (e.g., Peavy et al., 1994). It is widely held that consolidation is largely spared (e.g., Delis et al., 1994), except perhaps in persons with frank HIV-associated dementia (Scott et al., 2006). Individuals with HIV typically perform better on recognition relative to recall trials, suggesting that a partial retrieval deficit in HIV may normalize upon recognition testing (Delis et al., 1994; Woods et al., 2006). Consistent with the proportionally greater effects on frontostriatal systems in HIV (Ellis, Calero, & Stockin, 2009), the typical pattern of impairments is similar to that of other frontostriatal systems disorders (e.g., Huntington’s disease) and distinct from posterior cortical disorders (e.g., Alzheimer’s disease; Delis et al., 2005). However, considerable heterogeneity is observed in learning and memory profiles among individuals with HIV (Murji et al., 2003), perhaps due to differential involvement of the prefrontal-subcortical circuitry. For example, one recent study showed decreased bilateral hippocampal activation during encoding and increased activation during delayed recognition in a small sample of HIV-infected persons (Maki et al., 2009).

The California Verbal Learning Test (CVLT and CVLT-II; Delis, Kramer, Kaplan, & Ober, 1987; Delis, Kramer, Kaplan, & Ober, 2000) is a reliable and well-validated measure of episodic verbal learning and memory that readily distinguishes between healthy adults and individuals with various neuropsychological conditions, including HIV (e.g., Delis et al., 1994; Peavy et al., 1994). The CVLT is one of the five most common neuropsychological tests administered in North America (Rabin, Barr, & Burton, 2005), and has been recommended for neuroAIDS research by a National Institutes of Health (NIH) working group in Frascati (Antinori et al., 2007). The CVLT is a multidimensional task that yields a large volume of clinically useful quantitative indices that enable one to identify research-based memory profiles associated with various patient populations and draw inferences about component cognitive processes (e.g., encoding, retention, and retrieval; Delis et al., 1991; 2000). Recently, Wright and colleagues (2009) developed a novel approach to the quantification of the CVLT that was hypothesized to provide improved diagnostic identification of process-level deficits by removing the effects of inattention and considering list learning at the item level. Labeled the Item-Specific Deficit Approach (ISDA), this scoring method differs from traditional CVLT indices in that it aims to account for patterns of performance in order to enable separation between closely related memory constructs (e.g., consolidation and retrieval). The method yields three unitary deficit indices for commonly employed process units of episodic memory; namely, Encoding, Consolidation, and Retrieval. It is notable that the ISDA’s labeling of single scores as representing putative components of memory differs from the approach recommended for interpreting the CVLT (Delis et al., 2000), which provides descriptive, task-specific performance indices (e.g., Trial 1, learning slope, semantic and serial clustering, serial position proportions) that are intended to be interpreted in combination when drawing inferences about broader memory constructs (e.g., encoding). Another noteworthy divergence of the ISDA approach is that the retrieval index does not consider recognition performance, citing the conceptual complexity of the concept and some mixed evidence for the construct validity of using the traditional approach of examining performance discrepancies on free recall and recognition discriminability (see Delis et al., 2000).

The construct validity of the ISDA was originally tested in a mixed sample of neurologically compromised individuals that included persons with HIV infection and traumatic brain injury (TBI; Wright et al., 2009). Results of this investigation showed acceptable internal consistency of the ISDA Encoding, Consolidation, and Retrieval indices(α range = .58–.77), and demonstrated evidence of construct separation as indicated by the lack of a significant correlation between the Consolidation and Retrieval indices. Classification accuracy of individuals as neurologically compromised was comparable to traditional CVLT indices, and ISDA indices were associated with verbal memory performance on the Logical Memory subtest of the Wechsler Memory Scale-Third Edition (WMS-III; Weschler, 1997). A second study by Wright and colleagues provided additional support for the ISDA methodology in persons with severe TBI (Wright, Schmitter-Edgecombe, & Woo, 2010). Specifically, ISDA Encoding and Consolidation deficits accounted for a substantial proportion of the variance in long delay free recall in TBI. While level of acquisition (high vs. low) contributed to Consolidation deficits, participants exhibited Encoding deficits above and beyond that which could be explained solely by problems in Consolidation (Wright et al., 2010). Although these results suggest that ISDA indices may explain significant variance in relevant outcome measures, several limitations warrant further investigation of the ISDA as it applies to HIV. While individuals with HIV were included in the initial neurologically compromised sample (Wright et al., 2009), the study did not include analyses of the ISDA effects specific to HIV, and the authors provided limited information on important sample characteristics, such as the prevalence and effects of HAND. In addition, the ISDA has not yet been utilized with the most recent, second edition of the CVLT (i.e., CVLT-II).

Although memory profiles of HIV have been well categorized using traditional CVLT indices in the era before combined antiretroviral treatment (cART, e.g., Delis et al., 1995; Peavy et al., 1994), the potential incremental validity of the ISDA warrants its investigation in the age of widespread cART use. Accordingly, the current study aims to expand upon the initial Wright et al. studies (2009, 2010) by evaluating the construct validity of the ISDA indices in HIV with particular attention to their ability to classify HAND, their correlations with convergent (e.g., episodic memory) and divergent (e.g., motor skills) constructs in HAND, and their possible incremental value relative to traditional CVLT-II measures.

Method

Participants

A total of 186 individuals between the ages of 19 and 74 were drawn from a larger National Institute of Mental Health (NIMH)-funded R01 cohort (N = 221) based on the following criteria. Individuals with histories of severe psychiatric, medical, or neurological conditions were excluded, as were individuals with diagnoses of substance-related disorders within six months of evaluation as determined by the Composite International Diagnostic Interview (CIDI version 2.1). To confirm current substance abstinence, participants underwent a urine toxicology test for illicit substances on the day of testing. HIV status was determined by enzyme linked immunosorbent assays and confirmed by a Western Blot test. Participants were classified into three groups: HIV− (n = 43, ages 19–74), HIV+/HAND− (n = 103, ages 18–70), and HIV+/HAND+ (n = 40, ages 25–71). Using the current nomenclature recommended by the Frascati group (Antinori et al., 2007), 65% of the HIV+HAND+ participants were classified as asymptomatic neurocognitive impairment (ANI; i.e., global neurocognitive impairment without evidence of major functional disability), 27.5% met criteria for mild neurocognitive disorder (MND; global neurocognitive impairment with mild-to-moderate functional disability), and 2.5% met diagnosed with HIV-associated dementia (HAD; i.e., moderate-to-severe global neurocognitive impairment with at least moderate functional disability). HAND was diagnosed via multidisciplinary case conference according to the Antinori et al. (2007) criteria and was based on the results of a comprehensive neuropsychological (excluding the CVLT-II), psychiatric, and neuromedical assessment (see Woods et al., 2007). As this investigation involved diagnostic accuracy of CVLT-II ISDA scoring methods, the CVLT-II was excluded from the determination of HAND diagnoses for this study.

Table 1 displays the demographic, HIV disease, and psychiatric characteristics of the study samples. The three groups were similar in terms of age, education, ethnicity, and lifetime histories of substance use disorders (ps > .10). The HIV+ samples were well matched on HIV disease (e.g., current and nadir CD4 count, viral load, and disease duration) and treatment (i.e., the proportion prescribed antiretroviral therapies) factors. Statistically significant differences between samples were as follows. The HIV− group had a significantly lower proportion of men, and the HIV+/HAND+ group had a slightly lower estimated verbal IQ score as measured by the Wechsler Test of Adult Reading (ps < .05). As compared to the two HIV+ groups, participants in the HIV− sample also reported slightly less current affective distress on the Profile of Mood States (POMS; p < .10) and were significantly less likely to have met lifetime diagnostic criteria for Major Depressive Disorder (p < .05).

Table 1.

Demographic, HIV Disease, and Psychiatric Characteristics of Study Participants (n=186)

Participant characteristic	HIV− (n = 43)	HIV+ HAND− (n = 103)	HIV+ HAND+ (n = 40)	p	Pairwise comparisons
Demographic
Age (years)	43.3 (12.5)	45.6 (8.2)	45.1 (9.2)	.6141	-
Education (years)	14.6 (2.2)	14.1 (2.2)	14.1 (2.6)	.4050	-
Sex (% men)	53.5%	89.3%	82.5%	<. 0001**	HIV− < HAND− = HAND+
Ethnicity (% Caucasian)	53.5%	63.1%	55.0%	.2423	-
Est. verbal IQa	105.7 (11.2)	107.1 (11.0)	99.6 (12.8)	.0042*	HIV− = HAND− > HAND+
HIV Disease
Est. duration of infection (years)	-	16.1 (7.1, 20.6)	10.6 (6.9, 21.8)	.4174	-
Current CD4 Count	-	525.5 (318.5, 751.0)	566.5 (372.8, 931.0)	.1475	-
Nadir CD4 Count	-	140.0 (35.0, 288.0)	185.5 (50.5, 357.5)	.5542	-
HIV RNA plasma (log10)	-	1.7 (1.7, 2.2)	1.7 (1.7, 2.7)	.7755	-
AIDS (%)	-	64.0%	55.0%	.3166	-
Proportion on HAART	-	82.5%	72.5%	.0839	-
Psychiatric Characteristics
Substance dependence (lifetime)	35.7%	50.5%	55.3%	.1656	-
Major depressive disorder (lifetime)	35.7%	50.5%	60.5%	.0791	-
Generalized anxiety disorder (lifetime)	7.1%	3.8%	10.5%	.3393	-
POMS Total mood disturbance	45.05 (28.54)	64.76 (37.59)	51.58 (33.41)	.0421*	HIV− = HAND− < HAND+

Note.

^aas measured by the WTAR. HAART = highly active antiretroviral therapy. POMS = Profile of Mood States.

^*p < .05,

^**p < .001.

Materials and Procedure

All participants provided informed consent prior to completing the neurocognitive test battery, which included the CVLT-II administered according to standardized procedures (see Delis et al., 2000). The CVLT-II is a well-validated test of verbal memory that consists of five learning trials, a distractor list trial, and both free and cued recall trials after a short delay and a long delay. Finally, participants are administered a yes/no recognition trial, in which they are asked to indicate whether or not each of a series of words was included in the original list. Consistent with prior research, Total Trials 1–5, Short Delay Free Recall, Long Delay Free Recall, and Recognition Discriminability (d′) indices were calculated by the CVLT-II program software (Delis et al., 2000). In addition, we selected a few key supplementary measures from the CVLT-II, including semantic clustering and recall consistency, as well as several contrast measures (e.g., LDFR vs. Trial 5).

ISDA indices were calculated according to the procedure originally described by Wright et al. (2009). Specifically, responses on the CVLT-II were first coded at the item level to indicate the presence/absence of each List A word on all of the individual learning and delayed recall trials. The ISDA Encoding Index was calculated by determining the number of individual words on the CVLT-II List A that were recalled two or fewer times over the five learning trials, such that higher scores indicate worse performance (range = 0 to 16). The ISDA Consolidation Index was calculated as the number of items recalled at least once during list learning, but not at all during delayed recall trials. This value was divided by the total number of words recalled during learning trials in order to control for disparities in total word acquisition (range = 0 to 1). The ISDA Retrieval Index was calculated as the number of words on the CVLT-II List A that were recalled during learning trials, but inconsistently recalled after a delay (recalled on at least one but not all delay trials). This value was also divided by the total words recalled during learning trials to control for varying levels of acquisition (range = 0 to 1). As ISDA indices are deficit indices rather than performance indices, higher deficit scores indicate poorer performance.

Finally, well-validated tests of learning and cognitive functioning were extracted from the neuropsychological battery in order to examine the convergent validity of ISDA indices with tests that were expected to demonstrate similar patterns of performance in the HIV+HAND+. These tests included the Weschler Memory Scale’s Logical Memory subtest (WMS-III; Weschler, 1997), Rey Complex Figure Test – Boston Qualitative Scoring System (BQSS; Stern et al., 1999), Tower of London – Drexel Version (TOL-DX; Culbertson & Zilmer, 2001), and Trail Making Test, Part B (TMT-B; Reitan & Wolfson, 1985). Tests that were not expected to correlate strongly with ISDA indices were chosen in order to assess the divergent validity of the ISDA, including visuoconstruction as assessed by the Copy Trial of the Rey Complex Figure (Boston Qualitative Scoring System; Stern et al., 1999) and motor skills as measured by the Grooved Pegboard (Kløve, 1963).

Data Analyses

The primary study hypothesis was evaluated using a series of nonparametric Kruskal-Wallis tests, as all primary ISDA variables of interest were found to be non-normally distributed according to the Shapiro-Wilk test and to have unequal variance between groups according to the Levene test (ps < .05). A planned series of follow-up Wilcoxon signed-rank tests was conducted in order to investigate pairwise differences between groups. Where possible, results are supported by unbiased Cohen’s d values. In accordance with conventional interpretations, effect sizes of .2, .5, and .8 correspond to small, medium and large effect sizes, respectively. Follow-up regressions were planned in order to investigate the effect of HAND on ISDA indices in the context of variables that differed between groups (i.e., gender, POMS total, lifetime Major Depressive Disorder, and WTAR verbal IQ), and Wilcoxon signed-rank tests were planned in order to examine the relative ISDA deficits of the three groups. Receiver-operating characteristic (ROC) curves and follow-up descriptive classification accuracy statistics were generated for ISDA and CVLT-II indices (between HIV− and HAND+ individuals) in order to assess the diagnostic value of ISDA variables for HAND. Areas under the ROC curves for ISDA and traditional indices were then compared to one another using the procedures for non-independent samples described in Hanley & McNeil (1983). Spearman’s rank correlation coefficients (ρ) were used to examine the associations between ISDA measures and a priori selected traditional CVLT indices in the HIV+/HAND+ group. Spearman’s ρ correlational analyses were also conducted in order to investigate the relationships between ISDA and traditional indices and between CVLT performance indices and neuropsychological performance on other cognitive tasks. A critical alpha level of .05 was selected because: 1) the specific tests included for analysis were chosen on an a priori basis; 2) we balanced our null hypothesis significance testing with consideration of effect sizes; and 3) the relatively small HIV+/HAND+ sample for the correlational analysise (n = 40).

Results

Descriptive data, between-group differences, and effect sizes for ISDA indices, as well as the primary and supplementary traditional CVLT-II scores, in the HIV−, HIV+/HAND−, and HIV+/HAND+ groups are displayed in Table 2. Overall, nonparametric Kruskal-Wallis tests showed significant between-group differences on ISDA Encoding, Consolidation, and Retrieval indices (ps < .05). In three separate multiple regressions, the association between HAND and Encoding (adjusted R² = .21 F (4, 186) = 6.93, p < .0001), Consolidation (adjusted R² = .11 F (4, 186) = 3.83, p < .001), and Retrieval (adjusted R² = .24 F (4, 186) = 8.36, p < .0001) remained significant (ps < .05) when the confounding factors on which the study groups differed (i.e., gender, POMS total, and WTAR verbal) were included in the model. Follow-up Wilcoxon signed-rank tests indicated that the ISDA Encoding Deficit index was significantly greater in HIV+/HAND+ than in the HIV+/HAND− and in HIV− samples (ps < .05), while the latter two groups did not significantly differ from each other (p > .10). Follow-up Wilcoxon signed-rank tests demonstrated that the Consolidation index was also significantly higher in HIV+HAND+ individuals (p < .01), and did not differ significantly between HIV+/HAND− and HIV− individuals (p > .10). Finally, Wilcoxon signed-rank tests indicated that the ISDA Retrieval Deficit index followed this pattern as well, with HIV+/HAND+ individuals obtaining significantly higher scores (p < .001) than HIV+/HAND− and HIV− individuals, who did not significantly differ from each other (p > .10).

Table 2.

CVLT-II Performance in the Three Study Samples (n = 186)

CVLT-II Variables	HIV− (n = 43)	HIV+ HAND− (n = 103)	HIV+ HAND+ (n = 40)	pa	db (HIV− vs HAND+)	db (HAND− vs HAND+)
Item Specific Deficit Indices
Encoding Index	4.0 (2.0, 6.0)	5.0 (3.0, 7.0)	7.0 (5.0, 9.0)	.0024	−0.79	−0.49
Consolidation Index	0.1 (0.1, 0.2)	0.1 (0.1, 0.3)	0.2 (0.1, 0.3)	.0368	−0.55	−0.37
Retrieval Index	0.4 (0.3, 0.6)	0.4 (0.2, 0.6)	0.6 (0.5, 0.7)	.0005	−0.64	−0.73
Standard Indices (raw)
Trials 1–5	56.0 (46.0, 62.0)	49.0 (41.0, 59.0)	44.0 (40.0, 51.8)	.0045	0.77	0.44
Short Delay Free Recall	11.0 (9.0, 14.0)	11.0 (8.0, 13.3)	9.0 (6.0, 10.0)	.0006	0.94	0.63
Long Delay Free Recall	12.0 (9.0, 14.0)	11.0 (8.0, 14.0)	9.0 (5.3, 11.0)	.0010	0.89	0.58
Total Recognition (d′)	3.1 (2.6, 3.7)	3.3 (2.5, 3.8)	2.7 (2.1, 3.3)	.0085	0.61	0.48
Semantic Clustering	0.9 (0.2, 2.6)	1.0 (−0.2, 3.5)	0.1 (−0.3, 1.0)	.0147	0.52	0.55
Recall Consistency	85.0 (79.0, 92.0)	83.0 (76.0, 90.0)	78.5 (73.0, 83.0)	.0007	0.82	0.59
SDFR vs. Trial 5	−10.0 (−27.3, 0.0)	−11.8 (−30.8, 0.0)	−23.1 (−30.6, −13.0)	.0063	0.69	0.49
LDFR vs. Trial 5	−6.7 (−18.8, 0.0)	−8.3 (−25.6, 0.0)	−20.7 (−37.2, −9.1)	.0105	0.68	0.45
Total Recog Disc vs. LDFR	33.3 (14.8, 55.0)	37.0 (17.6, 65.9)	55.6 (26.5, 90.7)	.0548	−0.32	−0.30

Note. Data represent medians and interquartile ranges.

^aIn all significant pairwise comparisons, HIV− = HAND− > HAND+

^bbias corrected Hedges d effect size estimate.

Examination of the area under the ROC curves indicated that the ISDA and traditional indices all performed significantly better than chance when discriminating HIV+/HAND+ individuals from HIV− individuals (ps < .05), but not from HIV+/HAND− individuals (ps > .10). The ROC curves were then used to generate descriptive classification statistics for each index (see Table 3) in order to compare their classification accuracy in discriminating HIV+/HAND+ from HIV− and HIV+/HAND-individuals. In all cases, cut points were selected using Youden’s J, which determines the maximum value for (Sensitivity + Specificity – 1). ISDA indices performed comparably to traditional CVLT indices in terms of specificity (55–61% in ISDA and 63–76% in traditional CVLT-II indices), and sensitivity (43–78% in ISDA and 60–78% in traditional CVLT-II indices). Rates of Type I error (false-positives = 20–23% in ISDA, 12–19% in CVLT-II) and Type 2 error (false-negatives = 6–16% in ISDA, 14–19% in CVLT-II) were similar between ISDA and traditional CVLT-II indices. Areas under ROC curves did not significantly differ between traditional and ISDA indices (all ps > .67).

Table 3.

Classification Accuracy of ISDA and Traditional CVLT indices for Predicting HAND vs. HIV− (n = 186)

CVLT-II Variable	AUC	Standard error	p-value	Sensitivity	Specificity
Item Specific Deficit Indices
Encoding	.719	.083	<.003*	.78	.61
Consolidation	.659	1.66	<.018*	.68	.61
Retrieval	.693	1.01	<.007*	.88	.56
Standard Indices
Total trials 1–5	.710	.023	<.002*	.60	.76
Short delay free recall	.745	.085	<.0003**	.78	.63
Long delay free recall	.730	.077	<.0005**	.70	.65
Total recognition (‘d’)	.684	.304	<.010*	.75	.63

Note. AUC = Area Under Curve.

^*p <.05,

^*p<.001. Cutpoints were selected based on the optimal balance between sensitivity and specificity.

Between-group differences on traditional CVLT indices between HIV−, HIV+/HAND−, and HIV+/HAND+ groups are also presented in Table 2. These results demonstrated similar patterns across the three groups when standard CVLT indices of encoding, consolidation, and retrieval were examined. Significant omnibus differences were observed between the HIV−, HIV+/HAND−, and HIV+/HAND+ groups on well-validated CVLT measures (ps < .01). The association between HAND and Trials 1–5 Total (adjusted R² = .23 F (4, 186) = 7.81, p < .0001), Short Delay Free Recall (adjusted R² = .24 F (4, 186) = 8.27, p < .0001), Long Delay Free Recall (adjusted R² = .23 F (4, 186) = 7.82, p < .0001), and Total Recognition Discriminability (adjusted R² = .21 F (4, 186) = 7.01, p < .0001) remained significant (ps < .05) in four separate regressions that included potentially confounding factors on which the three groups differed (i.e., gender, POMS total, and WTAR verbal IQ). Planned comparisons showed that HIV+/HAND+ individuals performed significantly more poorly relative to HIV+/HAND− and HIV− on Total recall for Trials 1–5 (p < .01), Short Delay Free Recall (p < .01), Long Delay Free Recall (p < .01), and Total Recognition Discriminability (p < .01). HIV+/HAND− and HIV− individuals did not differ significantly from each other on Trials 1–5, Short Delay Free Recall, Long Delay Free Recall, or Total Recognition (all ps > .08) indices. Similar results were observed for the supplementary CVLT-II measures displayed in Table 2.

Correlations between the ISDA indices and standard CVLT measures of encoding, consolidation, and retrieval in the HIV+/HAND+ group are presented in Table 4. These data show that ISDA Encoding was significantly associated with Total Trials 1–5, Trials 1–5 Semantic Clustering, and Across-Trial Recall Consistency (ps < .001, rho range = −.51 to −.92). ISDA Consolidation was significantly associated with Long-Delay Free Recall, Short-Delay Free Recall, Short-Delay Free Recall vs. Trial 5, and Long Delay Free vs. Trial 5 (ps < .05, rho range = −.39 to −.71. ISDA Retrieval was significantly associated with Total Recognition Discriminability vs. LD Free and Total Recognition Discriminability (ps < .001, rho range = −.64 to −.68). Age was not significantly correlated with any of the ISDA or CVLT-II indices in the HIV+HAND+ group.

Table 4.

Correlations Between ISDA, Traditional CVLT-II, and Other Measures in HAND (n = 40)

	Item-Specific Deficit Approach Variable
Variable	Encoding	Consolidation	Retrieval
Encoding	-	.60**	.63**
Total Trials 1–5s	−.92**	−.62**	−.69**
Trials 1–5 Semantic Clustering	−.51**	−.55**	−.44*
Across-Trial Recall Consistency	−.52**	−.34*	−.51**
Total Learning Slopes	.03	−.15	−.18
Consolidation	.60**	-	.56**
Long-Delay Free Recall	−.75**	−.71**	−.86**
Short-Delay Free Recall	−.72**	−.68**	−.84
Short Delay Free vs. Trial 5a	−.35*	−.39*	−.58**
Long Delay Free vs. Trial 5	−.52**	−.58**	−.70**
Retrieval	.63**	.56**	-
Total Recognition Disc vs. LD Free	.59**	.46	−.64**
Total Recognition Discriminabilitya	−.50**	−.57**	−.68**

Note. Raw scores were used in generating these Spearman’s ρ values.

^aIndices cited as evidence for construct validity in Wright et al. (2009).

^*p <.05,

^**p<.001.

Correlations between CVLT-II ISDA and traditional indices and performance on other cognitive tasks in the HIV+/HAND+ group are presented in Table 5. Significant correlations of comparable strength were observed between ISDA and traditional CVLT indices and performance on the WMS-III Logical Memory subtest, the BQSS, Trail Making Test B, and Tower of London (ps < .05). ISDA indices did not correlate with measures of visuoconstruction or motor skills (ps > .10).

Table 5.

Correlations Between Other Neurocognitive Variables and ISDA and Traditional Indices in HAND (n = 40)

Index	Item-Specific Deficit Approach			Traditional
	Encoding	Consolidation	Retrieval	Trials 1–5	SDFR	LDFR	Recognition (d′)
WMS-III
LM-I	.49*	.42*	.54*	.55*	.53*	.56*	.48*
LM-II	.53*	.44*	.59*	.58*	.54*	.62*	.58*
LM-II Rec.	.49*	.38*	.50*	.53*	.47*	.52*	.53*
BQSS
Immediate	.35*	.25*	.38*	.39*	.36*	.39*	.36*
Delay	.37*	.24*	.41*	.40*	.38*	.40*	.33*
TMT, Part B	.33*	.20*	.39*	.33*	.37*	.38*	.32*
ToL Total Moves	.18*	.07*	.23*	.19*	.23*	.23*	.19*
BQSS Copy	−.15	.13	−.27	.15	.12	.21	.16
Pegboard Dominant Time	−.12	−.15	−.10	.07	.15	.06	−.15
Pegboard Non-dominant Time	.01	.20	−.05	−.07	.05	.04	.00
BNT Spontaneous Correct	−.11	−.20	−.53**	.04	.08	.15	.28
Famous Faces	.00	−.07	−.21	.18	.35*	.35*	.52**

Note. LMI = Wechsler Memory Scale Logical Memory Subtest % retrieval, LMII = Wechsler Memory Scale Logical Memory Subtest II, LMII Rec = Wechsler Memory Scale Logical Memory Subtest II Recognition, BQSS Rey-Osterrieth Complex Figure Delay, TMT = Trail Making Test, ToL = Tower of London.

^*p <.05.

Discussion

Given the conceptual attractiveness of the ISDA to the CVLT-II and preliminary evidence for its construct validity (Wright et al., 2009; 2010), we sought to examine its usefulness in identifying and characterizing HAND. Results of the current study provided mixed support for the construct validity of the ISDA indices in HAND. Examination of these novel measures in HAND revealed a pattern of impaired learning and recall broadly paralleling the mixed encoding/retrieval profile that is typically observed in HIV infection using conventional CVLT indices (e.g., Delis et al., 1994). Specifically, HIV+/HAND+ individuals demonstrated significantly higher ISDA Encoding and Retrieval indices relative to HIV+/HAND− and HIV− individuals. Unsurprisingly, effect sizes between HIV− and HIV+/HAND+ groups were larger in magnitude than effect sizes between HIV+/HAND− and HIV+/HAND+ groups (moderate and mild-to-moderate, respectively). These patterns of verbal memory performance are consistent with the well-established abnormalities of prefrontostriatal loops in HIV (e.g., Ellis et al., 2009), as well as in other frontostriatal conditions (e.g., Huntington’s disease; Delis et al., 2005).

In contrast, the finding of a HAND-associated deficit on the ISDA Consolidation index is a bit more challenging to interpret, as recent data regarding rapid forgetting in HIV have been somewhat contradictory. The ISDA Consolidation index revealed significantly higher deficits in HIV+/HAND+ individuals relative to the HIV+/HAND− and HIV− participants. This finding is contrary to the commonly reported profile of learning and recall difficulties with spared retention in HIV (e.g., Delis et al., 1994; White et al., 1997). However, a closer review of the literature reveals some indications of mildly deficient consolidation in HIV (Peavy et al., 2004; Woods et al., 2005), particularly in the setting of dementia (e.g., Scott et al., 2006). Moreover, postmortem degeneration of hippocampal regions (crucial for consolidation) is associated with antemortem neuropsychological impairment in HIV (Moore et al., 2006). Recent neuroimaging findings show hippocampal activation during encoding and increased activation during retrieval in HIV+ women (Maki, et al., 2009). Thus the ISDA Consolidation index finding may reflect a genuine deficit, especially considering the parallel finding for CVLT-II Long Delay Free Recall vs. Trial 5, which was also correlated with the ISDA Consolidation index. However, the consolidation effect size was smaller than the observed encoding and retrieval deficits, suggesting that the former deficit may be relatively minor as compared to the latter.

Although ISDA indices performed similarly to traditional CVLT-II measures in many respects, evidence of incremental validity is needed to justify the personnel time and cost of hand-calculating each index. Relative to traditional CVLT-II indices, effect sizes for ISDA indices were of comparable magnitude and had overlapping effect size confidence intervals. The ISDA indices were also highly correlated with traditional CVLT-II measures, which on one hand suggest a favorable correspondence between the new indices and the gold standard indices of the CVLT-II. On the other hand, these associations (e.g., as high as .9 in some cases) may suggest possible collinearity between the two approaches. The current study was unable to test incremental validity directly (by the inclusion of ISDA and traditional indices in the same regression) due to these high intercorrelations. Further investigations may wish to clarify if other disease populations (e.g., Alzheimer’s disease) demonstrate more varied patterns of association between the ISDA and traditional CVLT indices (see Delis et al., 2003).

Although ISDA indices significantly discriminated HAND from HIV− individuals, diagnostic classification rates were broadly similar to those provided by traditional CVLT-II variables. For both ISDA and CVLT-II indices, the area under the ROC curve was not significant when discriminating HAND from HIV+/HAND− participants. ISDA indices yielded slightly lower specificity values overall (range = .41 to .55), but marginally higher sensitivity on average than traditional CVLT-II indices. However, sensitivity and specificity values did not significantly differ between ISDA and traditional CVLT-II indices (ps > .10). Taken together, these results suggest that despite the previous reports of enhanced ISDA diagnostic accuracy in mixed and TBI samples (Wright et al., 2009; 2010), diagnostic accuracy was comparable to traditional CVLT metrics in this study. Similar sensitivity (.69) and specificity (.86) for HAND were previously found using the Hopkins Verbal Learning Test - Revised (Carey et al., 2004). Given that classification accuracy using these learning tests alone was modest, assessment of multiple domains (and multiple tests per domain) is recommended to maximize diagnostic accuracy in HAND (Antinori et al., 2007).

Correlations between ISDA indices and neuropsychological measures of episodic memory and executive functions were found in the magnitude and direction consistent with the established profile of HAND. Moreover, evidence of divergent validity was seen in small, non-significant correlations between ISDA and tests of visuoconstruction and motor skills. Overall, correlations between each ISDA and traditional index and the other neuropsychological tests were of a similar magnitude. These findings seem to offer further support to the construct validity of the ISDA, but the large correlations between ISDA measures themselves (Spearman’s rho values between .56 and .63) raise questions regarding whether the indices truly represent independent constructs. Post-hoc analyses showed that in the HIV+ sample as a whole (n = 146), Spearman’s rho values for ISDA and CVLT-II correlations were even higher (range = between .65 and .71). These data also suggest that the intended separation between constructs may not have been sufficiently achieved in this population, which raises questions about the ISDA’s incremental validity.

Several limitations of the current study should temper the reader’s interpretation of these results. First, this study was conducted in a fairly healthy, predominantly Caucasian, well-educated sample. As ethnicity and education are associated with CVLT performance (Norman, Evans, Miller, & Heaton, 2000), it is not yet clear whether similar patterns would be observed in a more diverse sample. This is particularly relevant as some of the fastest-growing demographic groups of HIV-infected individuals are non-Caucasian (Espinoza, et al., 2007). Regarding the ISDA itself, one should remain mindful that assuming a one-to-one correspondence between a single index and the broad constructs of encoding, consolidation, and retrieval may be problematic. For example, there are complex interactions between attentional resources, task-generic resources, and task-specific resources underlying the successful encoding of episodic memories (Uncapher & Rugg, 2008), and that both content and context should be considered (Morgan, et al., 2009). Therefore, the clinical interpretation of an encoding deficit will usually rely on multiple indicators with (and across) tests of learning and memory (e.g., our interpretation of the consolidation findings above was greatly enhanced by considering multiple indicators). Relatedly, the ISDA Retrieval index does not take into account recognition performance, which although not without its controversies, commonly provides valuable information regarding the relative improvement when self-initiated memory search and retrieval demands are minimized (Delis et al., 2000). Despite these limitations, ISDA findings are sufficiently interesting that tuture investigations might consider examining the incremental ecological relevance of ISDA indices, for example in relation to performance-based (e.g., automobile driving, medication adherence), naturalistic (e.g., employment), and self-report (e.g., instrumental activities of daily living) measures of everyday functioning. Future work may also explore the neural substrates of the ISDA indices in HIV by incorporating measurements of critical disease biomarkers (e.g., tau), neuroimaging (e.g., diffusion tensor imaging), and various neuropathological (e.g., HIV encephalitis) processes.