The Item-Specific Deficit Approach to evaluating verbal memory dysfunction: Rationale, psychometrics, and application

Abstract

In the current study, we introduce the Item-Specific Deficit Approach (ISDA), a novel method for characterizing memory process deficits in list-learning data. To meet this objective, we applied the ISDA to California Verbal Learning Test (CVLT) data collected from a sample of 132 participants (53 healthy participants and 79 neurologically compromised participants). Overall, the ISDA indices measuring encoding, consolidation, and retrieval deficits demonstrated advantages over some traditional indices and indicated acceptable reliability and validity. Currently, the ISDA is intended for experimental use, although further research may support its utility for characterizing memory impairments in clinical assessments.

Verbal memory is often the focus of neuropsychological studies, in which descriptive indices are frequently extracted from list-learning data. These are used to identify memory process disruptions. Such indices are easily derived from list-learning tests and can be utilized when experimental manipulations may not be feasible. Unfortunately, the verbal memory profiles derived using these indices often lead to mixed results across studies and/or disagree with results from studies using experimental manipulations and measures of list-learning characteristics (e.g., human immunodeficiency virus, HIV: White et al., 1997 vs. Scott et al., 2006; traumatic brain injury, TBI: Vanderploeg, Crowell, & Curtiss, 2001 vs. DeLuca, Schultheis, Madigan, Christodoulou, & Averill, 2000; Parkinson’s disease: Beatty et al., 2003 vs. Faglioni, Saetti, & Botti, 2000). The objective of the current study is to introduce and evaluate a new set of memory process indices that can be applied to list-learning data and account for weaknesses in previously used measures.

Psychologists have often employed a three-stage model when discussing episodic memory (Ellis & Hunt, 1983; but see Howe, 1988). Here, memory is conceptualized as three distinct process units. The first is encoding, where information is taken in and transformed into a format that can be stored in the brain. The second stage is consolidation, where the transformed information is stored in the brain for later use. The third is retrieval, or extraction of the stored information for use. While both cognitive and neuropsychological studies have shown that the encoding, consolidation, and retrieval processes overlap, there is evidence that some distinctions do exist among them (e.g., Bernard, Desgranges, Platel, Baron, & Eustache, 2001; Greicius et al., 2003; Nyberg, Cabeza, & Tulving, 1996; Tulving & Markowitsch, 1997; Tulving & Osler, 1968; Tulving & Thompson, 1973). That said, the constructs of encoding, consolidation, and retrieval remain useful for discussing temporally graded memory processes that are, to some degree, subserved by different neuroanatomical networks (see Bernard et al., 2001; Fuster, 2000; Greicius et al., 2003; Nielsen-Bohlman & Knight, 1994; Nyberg et al., 1996; Nyberg, Forkstam, Petersson, Cabeza, & Ingvar, 2002). Therefore, this terminology is used in the following discussion.

Encoding

Taken together, functional imaging and lesion studies have implicated the left prefrontal cortex (namely, the left inferior frontal gyrus) and the left medial temporal regions (primarily the parahippocampal gyrus) in the encoding of verbal memories (Alessio et al., 2004; Ariza et al., 2006; Bor, Cumming, Scott, & Owen, 2004; Buckner, Kelly, & Petersen, 1999; Dolan & Fletcher, 1999; Fernandez et al., 1998; Gimranov & Mal’tseva, 2005; Habib, Nyberg, & Tulving, 2003; Karlsgodt, Shirinyan, van Erp, Cohen, & Cannon, 2005; Strange, Otten, Josephs, Rugg, & Dolan, 2002; Vingerhoets, Miatton, Vonck, Seurinck, & Boon, 2006). Verbal memory encoding ability is often operationally defined in terms of the learning slope: the increase in recall across consecutive learning trials (Delis, Kramer, Kaplan, & Ober, 2000). Performance on Trial 1 is typically contrasted to performance on subsequent learning trials, with higher scores on later trials reflecting greater encoding. Unfortunately, attention deficits, which are common in many neurological and psychiatric conditions (Cohen, Malloy, Jenkins, & Paul, 2006), are likely to reduce list-learning performances, especially on Trial 1 (Wiegner & Donders, 1999), and an increased slope would indicate initial attention difficulties that are reduced with repetitive learning rather than encoding abilities. The encoding skills of participants with inattention might be better characterized by contrasting scores on Trial 2 to subsequent learning trials, but such indices would likely lead to an inaccurate representation of encoding in participants with intact attentional abilities (e.g., healthy controls), and therefore this alteration to the learning slope would be of limited utility for group comparisons. Another approach to assessing encoding ability is to sum the number of items recalled across learning trials (Delis et al., 2000). Although this approach is less reliant on initial learning, it can still be negatively impacted by poor Trial 1 performance, making it susceptible to the confounding effects of inattention.

Consolidation

A rather large body of work indicates that medial temporal lobe structures—namely, the hippocampus—play a major role in the consolidation of memories, although with the passage of time memory storage appears to come under the control of the neocortex (Squire, 1980, 1994; Squire & Zola, 1998; Tulving & Markowitsch, 1998). Data from a number of studies indicate that verbal memory consolidation tends to be controlled by the left hemisphere (Alessio et al., 2004; Ariza et al., 2006; Gimranov & Mal’tseva, 2005; Vingerhoets et al., 2006). At the behavioral level, consolidation deficits are often indexed by forgetting/retention calculations (Lezak, Howieson, Loring, Hannay, & Fischer, 2004). These measures typically compare recall from the last learning trial to that from a delayed-recall trial (Delis et al., 2000). Here, a significant decrease in delayed recall is considered to indicate rapid forgetting or a consolidation deficit. This method assumes that lower delayed recall reflects poor consolidation rather than retrieval difficulties. Unfortunately, this assumption makes forgetting calculations a somewhat nonspecific indicator of memory storage. It should be noted that this index is often considered in conjunction with discrepancies between recall and recognition scores (suggested to reflect retrieval deficits when recognition is superior to recall), although this index suffers from psychometric issues that are discussed below.

An alternative approach is the assessment of proactive interference (PI). PI refers to the adverse effect of older learning on new learning (Postman, 1971). This is a normal phenomenon and is suggested to reflect ongoing consolidation of the previously presented information. In a list-learning paradigm, PI can be introduced by presenting a second to-be-learned list following the learning trials of the first list (e.g., List A Learning Trials 1, 2, 3, …, k, List B Learning Trial 1; Delis et al., 2000). This procedure is common to many standard list-learning tests—for example, Rey Auditory–Verbal Learning Test (RAVLT), California Verbal Learning Test (CVLT). Here, PI is assessed by comparing the recall performance from Trial 1 of the first list (List A) and performance on the second list (List B). A decrement in recall of the second list (List A recall > List B recall) is suggested to reflect PI and hence ongoing consolidation of the first list. Given this framework, low PI can be used as a marker of consolidation difficulties. However, as noted above, initial recall performances during list learning (List A Trial 1, List B Trial 1) are likely subject to the confounding effects of inattention. Therefore, PI is probably an invalid index of consolidation for most neuropsychological investigations.

Retrieval

While encoding of verbal memories appears to be supported by the left prefrontal cortex and hippocampus, research suggests that the retrieval of verbal memories is primarily under the control of the right prefrontal cortex and, in some cases, the hippocampus (Dolan & Fletcher, 1999; Greicius et al., 2003; Habib et al., 2003; Karlsgodt et al., 2005; Nyberg, Cabeza, & Tulving, 1996). Retrieval ability has been routinely assessed via recognition and free-recall discrepancies. This method assumes that greater recognition in comparison to recall is suggestive of partial retrieval deficits (Delis et al., 2000), in addition to possible disruptions in encoding or consolidation (Lezak et al., 2004; Squire, 1980). Furthermore, converging lines of evidence suggest that recall–recognition discrepancies may be an imprecise indicator of retrieval ability. For example, Wilde, Boake, and Sherer (1995) showed that recognition–recall discrepancies did not agree with other hypothesized indices of retrieval problems (e.g., increased recall via semantic cueing, inconsistent recall across learning trials) calculated from the CVLT in a sample of TBI participants. This research met with some criticism due to the choice of retrieval indices used and the fact that an external verbal memory criterion was not utilized to validate the retrieval indices in the study (Delis et al., 2000). Nevertheless, the use of recognition–free-recall discrepancies has resulted in mixed findings in other studies as well (see Riefer & Batchelder, 1991). It has been suggested that results based on such discrepancies can be misleading, as they reflect multiple processes. Indeed, a growing body of literature suggests that recall is a product of recollection, whereas recognition performances are comprised of both recollection and familiarity, which each have differing neural substrates (see Aggleton & Brown, 2006).

The Item-Specific Deficit Approach

Several memory process indices have been proposed and utilized widely in clinical practice and research (e.g., Beatty et al., 2003; Delis et al., 2000; Delis, Peavy, Heaton, & Butters, 1995; Peavy, Jacobs, Salmon, & Butters, 1994; Vanderploeg et al., 2001; Weschler, 1997; White et al., 1997). Although these indices reflect aspects of encoding, consolidation, and retrieval, their precision could be improved upon. One possible means of improvement would be to consider list-learning performances at the item level while accounting for patterns of performance on learning and delayed-recall trials. This way the confounding effects of inattention, as well as adequate index separation between different constructs (e.g., consolidation and retrieval), might be achieved. The Item-Specific Deficit Approach (ISDA) was developed within this rational framework. The ISDA encoding deficit index is a reflection of low acquisition across learning trials. It is derived by counting the number of items on a list-learning test that are not recalled on more than half of the learning trials (for example, on a task with five learning trials, it would be the number of items that were recalled on two or fewer learning trials; see Appendix for ISDA worksheet). The greater this value (maximum value = total number of list items), the greater the encoding difficulties are presumed to be. This encoding deficit index is not particularly reliant on any given learning trial, so it is less likely to be affected by inattention than are previously used encoding/acquisition indices.

APPENDIX.

ISDA Worksheet

List Learning Measure: Examinee Name/ID: Date:
	Item recalled during list learning (yes = 1, no = 0)?	Number of times item was recalled during list learning.	Items recalled less than X* during learning (yes = 1, no = 0)?	Item not recalled on any delayed recall trial (yes = 1, no = 0)? Only count items that were recalled during learning.	Item recalled inconsistently on delay recall trials (yes = 1, no = 0)? Only count items that were recalled during learning.
List items** 1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
	A. Sum =		B. Sum =	C. Sum =	D. Sum =

ISDA Indices

Encoding Deficit Index (B) =

Acquisition Adjusted Consolidation Deficit Index (C/A) =

Acquisition Adjusted Retrieval Deficit Index (D/A) =

Note. The ISDA can only be fully applied to list learning measures that include multiple learning and delayed recall trials.

^*X = (number of learning trials +1)/2; this includes items not recalled during list learning.

^**Add or omit rows as necessary.

The ISDA consolidation deficit index is calculated by summing the individual items that were recalled during list learning but not recalled on subsequent delayed-recall trials (multiple delayed recall trials are necessary to calculate this index). For comparative purposes, this value can be divided by the sum of the individual items recalled during list learning (i.e., the total items recalled at least once during learning; maximum value = total number of list items) to control for differing acquisition levels between groups. Since the loss of previously learned items is accounted for across delayed-recall trials, the ISDA consolidation deficit index is more likely to reflect storage difficulties than retrieval problems.

The ISDA retrieval deficit index is calculated by summing the individual items that were recalled during list learning but inconsistently recalled across delayed-recall trials (multiple delayed-recall trials are necessary to calculate this index). Similar to the consolidation deficit index, this value can be divided by the sum of the individual items recalled during list learning to control for acquisition level. Additionally, since the consistency of individual list items is tracked across delayed-recall trials, the ISDA retrieval deficit index is more likely to reflect retrieval problems than consolidation difficulties.

In the current study we applied the ISDA to CVLT data obtained from healthy and neurologically compromised (HIV+ and TBI) participants. We chose to include HIV+ and TBI participants in our neurologically compromised group, as both populations routinely evidence verbal memory impairments (Heaton et al., 1995; Levin & Goldstein, 1986). We assessed the ISDA’s relationship with traditional memory process indices, its reliability, and its ability to classify neurologically compromised participants, as well as its capacity to predict poor verbal memory performances on an external criterion.

METHOD

Participants

Data from 132 participants (53 healthy participants and 79 neurologically compromised participants—37 HIV+ participants and 42 TBI participants) were analyzed (see Table 1 for sample characteristics). Table 1 provides detailed information regarding the study sample. All participants consented to voluntary participation in institutional review board (IRB) approved studies conducted in California (Los Angeles area; all HIV+ participants and 12 healthy participants), Tennessee (Memphis area; 18 TBI and 10 healthy participants), and Washington (Spokane/Pullman area; 24 TBI and 31 healthy participants). All participants were free of current psychiatric illness and substance abuse. All neurologically compromised participants were free of any other central nervous system disorder or disease (e.g., cerebrovascular accident, neurosyphilis, epilepsy) other than their index condition. All TBI participants had estimates of posttraumatic amnesia and/or loss of consciousness that indicated severe brain injury (Williamson, Scott, & Adams, 1996). Additionally, all TBI survivors suffered from acceleration–deceleration injuries and were at least one year post injury (range = 1–27 years). For the HIV+ participants, HIV infection was confirmed with serologic testing for HIV antibody (enzyme-linked immunosorbent assay, ELISA, followed by Western blot), and of the 34 HIV+ participants with recent CD4 counts, 26% met the Center for Disease Control’s criteria for an AIDS diagnosis. As can be seen in Table 1, our sample was diverse in terms of ethnic background, although these ethnic differences were not equally distributed across each subgroup. Previous work has shown that ethnic minority group status is associated with lower CVLT performances (Norman, Evans, Miller, & Heaton, 2000), an effect that is likely driven by inequalities in educational experiences (Manly, Byrd, Touradji, & Stern, 2004; Manly, Jacobs, Touradji, Small, & Stern, 2002). Given this, we analyzed our data for ethnic group differences on CVLT indices displayed in Table 1. We found no differences between Caucasians (n = 80) and non-Caucasians (n = 52), Fs ≤ 0.05, ps ≥ .83. Similarly, we found no differences between Caucasians and African Americans (n = 41), Fs ≤ 1.19, ps ≥ .28.

TABLE 1.

Sample characteristics

		Neurologically compromised participants (n = 79)
Demographics	Healthy participants (n = 53)	HIV+ (n = 37)	TBI (n = 42)	Difference	Effect size	Additional detail
Age (years)	35.81(9.43)	41.19(5.83)	32.93(9.34)	F(2,129) = 9.40**	ηp2 = .13	HIV+ > Hlthy, HIV+ > TBI
Education (years)	14.66(2.00)	13.54(1.56)	13.57(2.10)	F(2,129) = 5.24*	ηp2 = .13	Hlthy>HIV+, Hlthy>TBI
% Male	70 (n = 37)	70 (n = 26)	67 (n = 28)	χ2(2, N=132) = 0.15	Øc = .03	—
% Caucasian	66 (n = 35)	13 (n = 5)	95 (n = 40)	χ2(2, N = 132) = 56.12**	Øc = .65	Hlthy: 12–AfA, 35–C, 6–O HIV+: 27–AfA, 1–AsA, 5–C, 2–MA, 1–NA, 1–O TBI: 2–AfA, 40–C
Injury/disease characteristics	—	Mean CD4 (n = 34) = 398.71 (310.18), AIDS by CD4 = 9	Mean PTA in days (n = 41) = 74.41(91.07), mean LOC in hrs (n = 41) = 837.20 (883.82); MVA = 36, Hit by Auto = 2, Long Falla = 4	—	—	—
Verbal Memory (CVLT)b
Total learning	52.89(9.64)	27.00(9.03)	20.00(13.34)	F(2,129)=123.24**	ηp2=.66	Hlthy>HIV>TBI
Short-delay free recall	0.38(0.95)	−2.46(0.84)	−3.38(1.29)	F(2,129)=169.98**	ηp2=.73	Hlthy>HIV>TBI
Long-delay free recall	0.18(0.83)	−2.22(1.11)	−2.95(1.41)	F(2,129)=102.81**	ηp2=.61	Hlthy>HIV>TBI
Recognition discriminability	0.00(0.44)	−0.86(0.82)	−1.57(1.40)	F(2,129)=33.00**	ηp2=.34	Hlthy>HIV>TBI

Note. Table depicts the means and standard deviations (SD in parentheses) for years of age, years of education, and CVLT performances for the healthy and neurologically compromised participants. Differences on these variables were assessed via 3 (group) × 1 (test variable) analyses of variance (ANOVAs). Significant ANOVAs were followed up with Tukey tests. Participant sex and racial/ethnic data are displayed as percentages. Differences in these categorical variables were tested by chi-square analyses. Disease and injury characteristics are also displayed for the neurologically compromised participants. HIV+ = human immunodeficiency virus-1 participants, TBI = traumatic brain injury participants, Hlthy = healthy participants, AfA = African American, AsA = Asian American, C = Caucasian, MA = Mexican American, NA = Native American, O = Other, AIDS = acquired immune deficiency syndrome, PTA = post traumatic amnesia, LOC = loss of consciousness, MVA = motor vehicle accident, CVLT = California Verbal Learning Test.

^aA long fall was defined by a fall of the participant’s height or greater (see Abou-Hamden et al., 1997).

^bGiven group differences in demographic characteristics, age-corrected standard scores (T-score for total learning and z-scores for all other values) generated by the CVLT scoring program were used to make group comparisons.

^*p < .01.

^**p < .001.

Procedure

All participants were administered the CVLT, a list-learning test that was used to derive conventional memory process indices, as well as the ISDA indices. A subset of participants were administered the Digit Span subtest from the Wechsler Adult Intelligence Scale–Revised (WAIS–R; Weschler, 1981; 50 neurologically compromised and 12 healthy) and the Logical Memory subtest from the Wechsler Memory Scale–Third Edition (WMS-III; Weschler, 1997; 20 neurologically compromised and 31 healthy) in order to assess attentional and memory factors independent of the list-learning task. All tests were administered and scored in accordance with standard instructions. The CVLT and the WMS-III were scored and normed via the software packages provided by the test manufacturers.

CVLT

The CVLT (Delis, Kramer, Kaplan, & Ober, 1987) is a standardized verbal list-learning test composed of 16 words that can be grouped into four different semantic categories. The list is presented orally to participants over five learning trials. Subsequently, a distractor list is presented, and participants are asked to recall the distractor items. Following the distractor trial, participants are administered a short-delay free-recall test, a short-delay cued-recall test, a long-delay (20-minute) free-recall test, a long-delay cued-recall test, and recognition trial.

The ISDA was applied to CVLT data to derive deficit-focused indices of encoding, consolidation, and retrieval. In order to accomplish this, CVLT performances were coded at the item level, indicating whether or not each word was recalled on each trial. The ISDA encoding deficit index was calculated by counting the items on the CVLT (out of a possible maximum of 16) that were recalled two or fewer times across the CVLT’s five learning trials. The ISDA consolidation deficit index was calculated by adding the individual items that were recalled during list learning, but not recalled on any subsequent recall trial.¹ The ISDA retrieval index was derived by summing the individual items that were recalled during list learning, but inconsistently recalled across delayed-recall trials.

In addition to the ISDA indices, other proposed indicators of encoding ability—that is, learning slope and total recall across learning trials (total learning), consolidation (recall differences between the last learning trial and short-delay free recall (forgetting), and retrieval (differences between recognition hits and long-delay free recall (recognition–recall discrepancy)—generated by the CVLT scoring program were retained.

WMS-III Logical Memory

The Logical Memory subtest of the WMS-III is a verbal memory test for two orally presented stories (Wechsler, 1997). The first story is presented once followed by an immediate recall trial. The second story is read to examinees twice, with a recall trial following each presentation. Following this, there is a 25–35-minute retention interval. After the retention interval, free recall of both stories is obtained. Finally, examinees are administered a recognition test for the elements of each story. Immediate- and delayed-recall performances on the Logical Memory subtest were normed via the WMS-III scoring program, converted to T-scores, and then transformed to deficit scores per the method developed by Heaton and colleagues (see Carey et al., 2004), where T-scores below 40 are considered to reflect deficit performances.

WAIS–R Digit Span

Finally, Digit Span from the WAIS–R (Wechsler, 1981) is a measure of auditory attention. This test is composed of trials where examinees are read strings of digits and are asked to repeat them back aloud. The length of the digit strings is incrementally increased over successive trials. On the first set of trials, an examinee is asked to repeat the digits back in the same order in which they were presented. On the second set of trials, the examinee is required to repeat back the digits in reverse order. Performances on both sets of trials are summed for an overall indicator of auditory attention.

Analysis plan

Differences in continuous variables (demographics and CVLT performances) were determined via one-way analyses of variance (ANOVAs). Categorical differences were assessed via chi-square analyses. Pearson product–moment correlations were carried out on the Digit Span, ISDA encoding index, learning slope, and total learning to determine any possible associations. Additional correlational analyses were conducted on the ISDA and traditional indices to determine how well they represented different aspects of memory processing. The internal consistency/reliability of the ISDA indices were evaluated with coefficient alpha or the KR-20 statistic, depending on the nature of the index (continuous vs. dichotomous; Pedhazur & Schmelkin, 1991). Both the traditional memory process indices and the ISDA indices were submitted to linear discriminant function analyses to determine how well they discriminated between neurologically compromised and healthy participants. Finally, a second set of discriminant function analyses were conducted to assess how well the ISDA and traditional memory process indices could predict deficit performances on Logical Memory. A p < .05 was set for all analyses.

RESULTS

As can be seen in Table 1, the neurologically compromised participants evidenced verbal memory deficits in comparison to the healthy participants as expected. Correlational analyses were used to determine the degree that the proposed encoding indices were related to each other and whether they might be confounded by inattention. In the participants that completed WAIS–R Digit Span (n = 62), the ISDA encoding index was highly related to CVLT total learning (r = −.96, p < .001) and moderately associated with learning slope (r = .33, p < .05). However, only CVLT total learning was significantly associated with WAIS–R Digit Span (r = .29, p < .05); neither learning slope nor the ISDA encoding deficit index (r = −.21) was correlated with Digit Span (r = −.14). To determine whether these indices behaved differentially as a function of inattention, as hypothesized, we reran the analysis in a subgroup of participants (n = 53)—45 neurologically compromised and 8 healthy; 72% male; 20 Caucasians, 28 African Americans, 2 Mexican Americans, 1 Asian American, 1 Native American, 1 “other;” mean age = 39.79 years (SD = 7.21); mean years of education = 13.68 (SD = 1.82)—that exhibited inattention on WAIS–R Digit Span (T < 40; see Carey et al., 2004). As can be seen in Table 2, the pattern of associations is very different for participants with attention deficits. The ISDA encoding deficit index did not correlate with either of the traditional indices (rs ≤ −.17) or Digit Span (r = −.10). However, CVLT total learning was correlated with Digit Span (r = .33) and learning slope (r =.30); the relationship between learning slope and Digit Span approached significance (r = −.25, p = .07).With regard to the other memory process index correlations (see Table 3), numerous significant intercorrelations were found. However, it should be noted that forgetting and recognition–recall discrepancies were moderately associated (r = −.45), while the ISDA consolidation deficit index was unrelated to the ISDA retrieval deficit index (r = .03). Also, the ISDA retrieval deficit index was not associated with the recognition–recall discrepancies index (r = .03). Additionally, although total learning and the ISDA encoding index were highly correlated in the sample as a whole, these indices were shown to be functionally distinct in a group of inattentive participants (see Table 2).

TABLE 2.

Correlations between Digit Span and indices of encoding in inattentive participants

	1	2	3	4
1. Digit Span	—
2. Total learning	.33*	—
3. Learning slope	−.25	.30*	—
4. ISDA encoding deficit index	−.10	−.17	−.07	—

Note. Correlation matrix for Digit Span data (n = 53), the ISDA encoding deficit index, and learning slope and total learning generated by the CVLT scoring program. Note that greater ISDA index values represent greater impairment. ISDA = item-specific deficit approach, CVLT = California Verbal Learning Test.

^*p < .05.

**p < .001.

TABLE 3.

Correlations between memory process indices

	1	2	3	4	5	6	7
1. Total learning	—
2. Learning slope	.33**	—
3. Forgetting	.63**	.23*	—
4. Recognition–recall discrepancy	−.52**	−.25*	−.45**	—
5. ISDA encoding deficit index	−.96**	−.38**	−.61**	.50**	—
6. ISDA consolidation deficit index	−.58**	−.39**	−.54**	.44**	.58**	—
7. ISDA retrieval deficit index	−.34**	−.10	−.44**	.03	.34**	.03	—

Note. Table displays correlation matrix for the traditional and ISDA memory process indices (n = 132). Values generated by the CVLT scoring program were used for total learning, learning slope, forgetting (difference in recall between Learning Trial 5 and short-delay free-recall test), and recognition–recall discrepancy (difference between recognition hits and long-delay free recall). Raw values were used in this analysis. Note that greater ISDA index values represent greater impairment. ISDA = item-specific deficit approach, CVLT = California Verbal Learning Test.

^*p < .01.

^**p < .001.

The ISDA encoding index provides dichotomous outcomes for each list item (recalled vs. not recalled), so its internal consistency was evaluated with the KR-20 statistic. In contrast, the acquisition-adjusted consolidation and retrieval deficit indices, which are adjusted for the number of items gained during learning, are not composed of dichotomous values, so they were evaluated with coefficient alpha. While high internal consistency coefficients (> .80) are routinely expected of indices used to make important decisions about level of impairment (Anastasi, 1988; Urbina, 2004), indices that are used to characterize these impairments are not judged by the same standard since they are descriptive (not diagnostic) in nature (Kline, 2005). The internal consistency estimates of such “process” indices are often decreased by two factors: a small number of items and heterogeneity in item responses due to individual differences in test-taking approach (Cronbach, 1984; Kehoe, 1995; Pedhazur & Schmelkin, 1991). Heterogeneity in item responses is indeed an issue in memory test performances, as common individual differences in list-learning approaches (levels of self-initiated rehearsal, intensity of study) differentially impact immediate- and delayed-recall patterns (e.g., Geiselman, Woodward, & Beatty, 1982). It has been suggested that moderate internal consistency estimates (.50–.70) are acceptable for experimental measures and/or descriptive indices comprised of a relative small number of items (Kehoe, 1995; Kline, 2005).

As can be seen in Table 4, ISDA indices of encoding, consolidation, and retrieval deficits showed acceptable internal consistency (.58–.77). In order to obtain internal consistency estimates that were less influenced by individual differences, we analyzed data from a subset of 67 participants that utilized a consistent list-learning strategy (indicated by a z-score of ≤1 on the CVLT’s semantic or serial clustering indices). This led to higher internal consistency estimates (.64–.84; see Table 4). Finally, to determine whether the number of items for each of the indices was impacting their internal consistency, we artificially doubled the responses for each index and calculated internal consistency coefficients for the whole sample as well as the subset that indicated a consistent learning strategy. This analysis showed that the number of items did influence our internal consistency estimates as the coefficients increased dramatically (.80–.89 and .82–.92, respectively; see Table 4).

TABLE 4.

Internal consistency coefficients for the ISDA as applied to the CVLT

			Simulated coefficients
	Overall sample (n = 132; 16 items)	Strategy users (n = 67; 16 items)	Overall sample (n = 132; 32 items)	Strategy users (n = 67; 32 items)
Encoding deficit index	.77	.84	.89	.92
Consolidation deficit indexa	.58	.64	.80	.82
Retrieval deficit indexa	.60	.66	.81	.84

Note. Table shows internal consistency coefficients for the ISDA indices and indicates adequate internal consistency. The acquisition adjusted indices were evaluated with coefficient alpha, while the encoding index was assessed with the KR-20 statistic as it is composed of dichotomous outcomes. Strategy users were individuals that indicated a z-score of ≥1 on the CVLT’s semantic or serial clustering indices. Item responses were artificially doubled for simulated coefficients. ISDA = item-specific deficit approach, CVLT = California Verbal Learning Test.

^aAcquisition-adjusted index.

Linear discriminant function analysis was employed to determine how well the ISDA indices could predict neurological status. The ISDA indices significantly discriminated between neurologically compromised and healthy participants: Wilks’s λ= 0.33, χ³(3) = 143.45, p < .001; canonical r(cr) =.82, η² = .67. The encoding deficit index was the main contributor to this discriminant function (cr = .71), followed by the retrieval deficit index (cr = .54) and the consolidation deficit index (cr = .45). Overall, the ISDA indices correctly classified 93.2% of the participants as either neurologically compromised or intact, with 92.4% sensitivity and 94.3% specificity.

A follow-up linear discriminant function analysis was conducted to determine how well traditional memory process indices (i.e., learning slope,² forgetting, recognition-recall discrepancy) could predict neurological compromise. The traditional indices significantly discriminated between neurologically compromised and healthy participants: Wilks’s λ = 0.60, χ² (3) = 64.31, p < .001; cr =.63, η² = .40. The traditional consolidation index (forgetting) was the main contributor to this discriminant function (cr = .83), followed by the encoding index (learning slope; cr =.48) and the retrieval index (recognition-recall discrepancy; cr = −.13). Overall, the traditional memory process indices correctly classified 82.2% of the participants as either neurologically compromised or intact, with 90.6% sensitivity and 78.9% specificity.

Discriminant function analysis was conducted on a subsample of participants (n = 51)—20 neurologically compromised and 31 healthy; 82% male; 44 Caucasians, 1 African American, 6 “other;” mean age = 34.45 years (SD = 9.39); mean years of education =14.29 (SD = 2.18)—that had completed the Logical Memory subtest of the WMS-III, a measure of verbal learning and memory. Impaired performance on Logical Memory was determined by converting norm-derived scaled scores for total story units recalled in Logical Memory I and Logical Memory II to T-scores and then transforming these to deficit scores (see Carey et al., 2004). Performances resulting in a deficit score of 1 or greater (i.e., T < 40) on either Logical Memory I or Logical Memory II were considered to reflect impaired verbal memory. The ISDA indices significantly predicted impaired verbal memory on Logical Memory: Wilks’s λ = 0.65, χ²(3) = 26.41, p < .001; cr = .59, η² = .35. This function was primarily driven by the consolidation deficit index (cr =.89), while the encoding (cr =.14) and retrieval deficit (cr =.20) indices played a minor role. It should be noted that the chief role of the consolidation deficit index in this function is not particularly surprising, given that meaningful prose provides a good deal of encoding and retrieval support (e.g., Jefferies, Lambon Ralph, & Baddeley, 2004). This discriminant function correctly classified 82.4% of the Logical Memory performances, with 74.0% sensitivity and 89.3% specificity.

An additional discriminant function analysis was carried out to determine how well traditional memory process indices (i.e., learning slope, forgetting, recognition-recall discrepancy) predicted performance deficits on the Logical Memory subtest. The traditional indices also predicted Logical Memory deficits: Wilks’s λ = 0.79, χ²(3) = 11.25, p = .01, cr =.46, η² = .21. This function was primarily driven by forgetting (cr =.67), followed by recognition-recall discrepancies (cr = −.53) and learning slope (cr = −.11). This discriminant function correctly classified 72.5% of the Logical Memory performances, with 93.0% sensitivity and 48.0% specificity.

Given the satisfactory psychometric findings regarding the ISDA as applied to the CVLT, we further analyzed the healthy participants’ (n = 53) data in order to derive useful normative references for future research. It should be noted that we used acquisition-corrected indices of consolidation and retrieval deficits for these analyses. Correlational analyses among age, education, and the ISDA indices indicated a positive association between age and the encoding deficit index (r = .48, p < .001). Neither the consolidation deficit nor the retrieval deficit indices were related to age (r =.15, r = .07, respectively). None of the ISDA indices was associated with years of education (rs = −.09 to −.11).

To further explore the relationship between age and the encoding deficit index, we stratified the healthy sample into three age groups (18–28 years, n = 12; 29–39 years, n = 19; 40–50 years, n = 22) and conducted an Age Group × Encoding Deficit Index ANOVA. The ANOVA revealed a significant main effect of age, F(2, 50) = 8.40, p = 0.001, η_p² = .25. Tukey post hoc tests indicated that 40–50-year-olds had greater encoding deficits than the other two younger age groups. The 18–29- and the 29–39-year-olds did not differ with respect to the ISDA encoding deficit index. Based on these results, we present preliminary normative data for the ISDA indices stratified by two age groups (18–39 and 40–50 years) in Table 5.

TABLE 5.

Age corrected experimental normative data for the ISDA as applied to the CVLT

	Group
ISDA index	18–39 years (n = 31)	40–50 years (n = 22)	Total (n = 53)
Encoding deficit index	1.94 (1.50)	3.59 (1.76)	2.63 (1.80)
Consolidation deficit index	0.06 (0.07)	0.10 (0.09)	0.08 (0.08)
Retrieval deficit index	0.17 (0.13)	0.18 (0.11)	0.17 (0.12)

Note. Table displays means and standard deviations (SDs in parentheses) for healthy adult participants as a whole and stratified by age. Age was only found to correlate with the ISDA encoding deficit index. Years of education was not related to any of the ISDA indices. The consolidation and retrieval deficit indices were corrected for acquisition by dividing them by the total items gained on the CVLT. Please note that higher index scores indicate worse performance. ISDA=item-specific deficit approach, CVLT=California Verbal Learning Test.

DISCUSSION

Traditional verbal memory process indices are routinely generated from list-learning data (total learning, learning slope, forgetting/retention, recognition–recall discrepancy) for clinical use and research purposes. These measures have yielded mixed results, have disagreed with other methods, and are suggested to be relatively nonspecific estimates of encoding, consolidation, and retrieval. Improved methods for discerning specific deficits in encoding, consolidation, and retrieval from list-learning data are of clinical and experimental importance. Indeed, the cognitive rehabilitation task force assembled by the European Federation of Neurological Societies indicated that better clinical and pathological characterizations are needed in intervention studies (Cappa et al., 2003). Better memory process indicators would not only improve diagnostic accuracy via fractionation of amnestic samples into subgroups, but they could also suggest optimal targets for interventions. Further, such psychometric improvements would aid investigators in more thoroughly evaluating the utility of behavioral and pharmacological interventions for memory impairment such that modest, mixed, or partially beneficial effects might be better explained (e.g., Evans et al., 2000; Rund & Borg, 1999; Stip, Sepehry, & Chouinard, 2007; Whitehead et al., 2004).

In the current study, we evaluated a novel item analytic approach (the ISDA) to assess memory process breakdowns in a mixed sample of healthy and neurologically compromised participants. We found that the ISDA demonstrated several strengths when applied to CVLT data. First, the ISDA encoding deficit index showed no association with impaired auditory attention (Digit Span), while a more commonly used index of encoding/acquisition (i.e., total words recalled from Trials 1–5 on the CVLT) did. This suggests that the ISDA encoding deficit index is less susceptible to the confounding effects of inattention. Second, the ISDA consolidation deficit index was shown to be unrelated to the ISDA retrieval deficit index. This contrasts with traditional indices of consolidation (forgetting/retention) and retrieval (recognition–recall discrepancies) that are moderately associated. Our data suggest that the ISDA produces functionally distinct indices of consolidation and retrieval, while traditional indices overlap. The overlap between traditional consolidation and retrieval indices is problematic for clinical and research use, as consolidation can contaminate retrieval measures and vice versa.

Since the ISDA indices of encoding, consolidation, and retrieval deficits are composed of itemlevel responses, we were able to evaluate the internal consistency of each index. Overall, the ISDA indices indicated moderate internal consistency (.58–.77). Such coefficients are considered to reflect adequate reliability for descriptive (not diagnostic) indices/scales comprised of a small number of items (Kehoe, 1995; Kline, 2005; Pedhazur & Schmelkin, 1991). That said, these estimates were likely reduced by differences in strategy use, as such differences are associated with disparate patterns of immediate and delayed recall (e.g., Geiselman et al., 1982), and such individual disparities often translate into greater item variances (Pedhazur & Schmelkin, 1991). Indeed, the ISDA indices reliability estimates increased (.64–.84) when individual differences in learning strategy use were partially controlled for. Beyond differences in strategy use, the internal consistency estimates of the ISDA indices were likely lowered by the relatively low number of items comprising each index and our sample size. Future work employing test–retest reliability estimates might better ascertain the reliability of the ISDA indices.

With regard to validity, discriminant function analyses revealed that the ISDA indices had a slight advantage over traditional indices in terms of classifying participants as neurologically compromised or intact: ISDA, 93.2 % classified correctly (92.4% sensitivity, 94.3% specificity), versus traditional indices, 82.2% classified correctly (90.6% sensitivity, 78.9% specificity). Additional discriminant function analyses conducted with a subset of participants (n = 51) showed that the ISDA outperformed traditional indices in predicting low memory performances on the Logical Memory subtest of the WMS-III. Specifically, the ISDA correctly predicted 82.4% of poor performances on Logical Memory, with 74.0% sensitivity and 89.3% specificity, while the traditional indices correctly predicted 72.5% of deficit performances on Logical Memory, with 93.0% sensitivity and 48.0% specificity. It should be noted that the ISDA’s discriminant function for Logical Memory deficits was primarily driven by the ISDA consolidation deficit index. Although this was not predicted, it was not particularly surprising, since meaningful prose, such as that provided in the Logical Memory stories, provides a good deal of encoding and retrieval support (e.g., Erber, Galt, & Botwinick, 1985; Jefferies et al., 2004; Spilich & Voss, 1982). This reduces any variance that could be accounted for by the ISDA encoding and retrieval deficit indices. In future research, another list-learning measure (e.g., RAVLT) should be used to provide an external criterion for the ISDA as applied to the CVLT.

In summary, while traditional memory process indices have yielded mixed results, have disagreed with experimental manipulations, and have been suggested to suffer from methodological difficulties; the ISDA was shown to be a reliable and valid method when applied to the CVLT. The current study suggests that the ISDA provides a decided advantage over traditional indices for experiments identifying the specific deficits that are responsible for memory impairments. However, additional work is necessary to determine the clinical utility of the ISDA. Future work should focus on alternative approaches to assessing the reliability (e.g., test–retest) and validity (e.g., experimental manipulations) of the ISDA across populations and using other list-learning measures. Applications of the ISDA to the CVLT-II and RAVLT, for example, have yet to be investigated. Additionally, studies are needed to determine how well the ISDA can discriminate memory deficit profiles and patterns of brain pathology. Nevertheless, the ISDA is a new analytic approach that has strengths over previously used methods to dissociate memory deficits. Furthermore, the ISDA may prove to be a useful clinical tool if future research continues to indicate that it has good psychometric properties and can meaningfully distinguish memory-disordered groups.