Performance Variability During a Multitrial List-Learning Task as a Predictor of Future Cognitive Decline in Healthy Elders

Abstract

Introduction

In clinical settings, neuropsychological test performance is traditionally evaluated with total summary scores (TSS). However, recent studies demonstrated that indices of intraindividual variability (IIV) yielded unique information complementing TSS. This 18-month longitudinal study sought to determine whether IIV indices derived from a multitrial list-learning test (the Rey Auditory Verbal Learning Test) provided incremental utility in predicting cognitive decline in older adults compared to TSS.

Method

Ninety-nine cognitively intact older adults (aged 65 to 89 years) underwent neuropsychological testing (including the Rey Auditory Verbal Learning Test) at baseline and 18-month follow-up. Participants were classified as cognitively stable (n = 65) or declining (n = 34) based on changes in their neuropsychological test performance. Logistic regression modeling tested the ability of baseline TSS indices (Sum of Trials 1-5, Immediate Recall, and Delayed Recall) and IIV indices (Lost Access and Gained Access) to discriminate between stable and declining individuals.

Results

Higher values of both Lost Access and Gained Access at baseline were associated with an increased risk for decline at 18-month follow-up. Further, the IIV indices provided predictive utility above and beyond the TSS indices.

Conclusion

These results highlight the value of analyzing IIV in addition to TSS during neuropsychological evaluation in older adults. High levels of IIV may reflect impairment in anterograde memory systems and/or executive dysfunction that may serve as a prognostic indicator of cognitive decline.

Introduction

Neuropsychological assessments typically evaluate performance using total summary scores (TSS), such as the number of items correctly recalled or mean reaction times. However, this approach may neglect potentially important and unique information that can be derived from examining measures of intraindividual variability (IIV). As an example, MacDonald et al. (2003) observed that high baseline IIV on four reaction time measures was associated with subsequent declines in cognitive functioning at 3-year and 6-year follow-up intervals. Further, individuals who exhibited cognitive decline at follow-up also demonstrated increased reaction time variability. These results remained consistent after correcting for mean reaction time. Another study (Lovden, Li, Shing, & Lindenberger, 2007) demonstrated that older adults who had greater variability on a simple reaction time task at baseline were more likely to exhibit declines over a 13-year interval on tests of processing speed and category fluency. These studies suggest that IIV may serve an important role in identifying older adults who are at the greatest risk for future cognitive decline.

While the aforementioned studies (Lovden et al., 2007; MacDonald et al., 2003) examined reaction time variability with regard to subsequent decline, less work has been dedicated to variability in performance accuracy. One study (Gamaldo, An, Allaire, Kitner-Triolo, & Zonderman, 2012) observed that fluctuating performance on common neuropsychological measures across annual evaluations was associated with a higher rate of eventual cognitive impairment. However, no studies to date have analyzed recall accuracy variability within a single cognitive evaluation as a predictor of future decline.

Analysis of IIV in the context of episodic memory measures may be particularly useful for studying risk for future cognitive decline because episodic memory impairment may reflect the earliest stages of dementia (Backman, Small, & Fratiglioni, 2001). One commonly used episodic memory measure is the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1958). The RAVLT requires individuals to learn a list of 15 words over five trials. The primary interpretive indices are TSS indices: the total number of words recalled across Trials 1-5, Immediate Recall (following an interference trial), and Delayed Recall (following a 30-minute delay). However, two measures of performance variability can also be calculated across trials two through five: Lost Access and Gained Access. Lost Access is defined as the sum of words that are not recalled on trial n but are recalled on trial n-1 and represents the rapid forgetting of learned information. Gained Access is defined as the sum of words recalled on trial n but not on trial n-1 and represents benefit from repetition. Greater amounts of Lost and/or Gained Access reflect IIV across the five trials. Lost Access appears to increase across the lifespan while Gained Access decreases (Dunlosky & Salthouse, 1996). Further, individuals with Alzheimer's disease exhibit higher Lost Access (and lower Gained Access) compared to cognitively intact controls (Woodard, Dunlosky, & Salthouse, 1999). Whether IIV measures provide additional useful information relative to TSS in predicting future cognitive decline in cognitively intact elders is unknown.

The current study sought to determine whether IIV indices derived from the RAVLT might be useful in predicting cognitive decline after an 18-month follow-up period in elders who were cognitively intact at baseline. We compared the prognostic utility of the IIV variables to TSS indices: Trials 1-5, Immediate Recall, and Delayed Recall. We hypothesized that IIV measures would provide important and unique information complementing traditional TSS indices in predicting future cognitive decline, and that higher values of both Lost and Gained Access would represent a greater risk for declines.

Method

Participants

The current sample is drawn from a longitudinal study designed to identify neuropsychological and neuroimaging markers predictive of future cognitive decline in elders (aged 65 to 89 years) who were deemed cognitively intact at entry. Participants were recruited from the Milwaukee, WI area via newspaper advertisement. Half the sample was recruited based on a family history of dementia in a first-degree relative in order to enrich the sample with persons at genetic risk for dementia. A more detailed description of recruitment procedures and sample characteristics is available in our previous publications (Woodard et al., 2010).

All participants were deemed cognitively intact at baseline. None of the participants performed greater than 1.5 standard deviations below age-appropriate means on the RAVLT Trials 1-5, Delayed Recall, and Long Term Percent Retention scores and Mattis Dementia Rating Scale-2 Total Score (DRS-2; Jurica, Leitten, & Mattis, 2001). The normative data for the RAVLT were based on a sample of 87 adults in the Milwaukee area (separate from the current sample) ranging in age from 65 to 89 years old, inclusive. Data are available from the authors upon request. The normative data used for the DRS-2 were from the test manual (Jurica et al., 2001). Although the Mini-Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975) was not part of the entry criteria, all participants scored 25 or higher. All but two participants scored at 27 or higher on the MMSE, and the other two participants (scores of 25 and 26) met our entry criteria based on scoring within normal limits on the RAVLT and DRS-2. All participants denied current psychiatric disturbance or impairments in activities of daily living. Further, participants were excluded if they reported a history or evidence of neurological illness or any medical conditions that could affect brain functioning.

Ninety-nine older adults (Mean age at study entry = 72.8 years, SD = 4.9 years) met study criteria and served as participants (Table 1). Informed consent was obtained consistent with the Declaration of Helsinki and institutional guidelines established by the Medical College of Wisconsin Human Subjects Review Committee. All participants received financial compensation.

Table 1.

Mean demographic and neuropsychological test results for stable and declining participants. Standard deviations are provided in parentheses.

	Groups
Variable	Stable (n=65)	Declining (n=34)	p	η2
Age at baseline (years)	72.2 (5.0)	73.9 (4.7)	.087	.030
Education (years)	15.0 (2.5)	14.7 (3.0)	.548	.004
Test-Retest Interval (days)	549.7 (39.2)	560.9 (41.4)	.202	.017
Ethnicity	100% Caucasian	97% Caucasian, 3% Black	.343+	-
Family History Status	57% Dementia positive	18 Dementia positive	.832=	-
APOE allele status	20 ε4 positive	20 ε4 positive	.001+	-
Gender	47F/18M	24F/10M	1.000+	-
Neuropsychological Measures
MMSE
Baseline	29.4 (0.8)	28.8 (1.3)	.015	.061
Follow-up	29.5 (1.0)	29.0 (1.3)	.034	.046
DRS-2 Total
Baseline	140.4 (3.0)	139.4 (4.0)	.158	.021
Follow-up	139.4 (2.1)*	135.9 (4.9)*	<.001	.143
RAVLT Trials 1-5
Baseline	50.0 (8.7)	45.8 (8.2)	.022	.053
Follow-up	49.9 (8.5)	39.9 (7.3)*	<.001	.274
RAVLT Immediate Recall
Baseline	9.9 (2.3)	8.5 (2.7)	.009	.069
Follow-up	10.3 (2.6)	7.3 (2.3)*	<.001	.267
RAVLT Delayed Recall
Baseline	9.9 (2.3)	9.0 (2.8)	.123	.025
Follow-up	10.2 (2.6)	6.3 (2.4)*	<.001	.360
RAVLT Lost Access
Baseline	9.4 (3.4)	11.4 (3.2)	.005	.078
Follow-up	9.2 (3.4)	12.6 (3.9)	<.001	.162
RAVLT Gained Access
Baseline	12.8 (3.0)	14.7 (3.3)	.006	.078
Follow-up	13.0 (2.7)	13.7 (3.7)	.367	.008

APOE = Apolipoprotein E; DRS-2 = Mattis Dementia Rating Scale-2; RAVLT = Rey Auditory Verbal Learning Test; p-values and η² are for independent samples t-tests comparing the two groups.

⁺Fisher's exact test used in place of independent samples t-test;

^*paired samples t-test revealed significant decrease (p < .05) from baseline to follow-up.

Neuropsychological Assessment

Participants underwent neuropsychological testing at baseline and at 18-month follow-up. The measures administered included the MMSE, DRS-2, and RAVLT. The MMSE is a 30-point measure of cognitive status. The DRS-2 is a measure of cognitive functioning in five domains with a maximum score of 144.

The RAVLT contains a list of 15 words that are aurally presented to participants over five trials. After each trial, participants are instructed to recall as many words as possible. Following the presentation of a second distractor list of words, participants must then recall as many words as possible from the original list (Immediate Recall). Thirty minutes later, participants must recall the list again (Delayed Recall). Outcome measures include the total number of words correctly recalled across the Trials 1-5 and at Immediate Recall and Delayed Recall. Additionally, Gained and Lost Access are calculated on a trial-by-trial basis and summed across Trials 2-5. Alternate forms of the RAVLT (Schmidt, 1996) were used at follow-up.

Definition of Cognitive Decline

Cognitive decline was defined as exhibiting a >1 SD reduction from baseline to 18-month follow-up on at least one of three outcome indices (DRS-2 Total Score, RAVLT Sum of Trials 1-5, and RAVLT Delayed Recall). We computed standardized residual change scores for each of the three measures to adjust for baseline performance, practice effects, and regression to the mean (McSweeny, Naugle, Chelune, & Luders, 1993). For each participant, if any change scores were -1.0 or lower, that individual was classified as cognitively declining. The remaining participants were classified as stable. Note that our definition of cognitive decline was purposely liberal to identify larger than average changes in cognitive function that can occur in persons who are cognitively intact at baseline but have a genetic risk for dementia. The goal was not to identify clinically significant declines in performance leading to a diagnosis of MCI or dementia.

Data Analysis

Logistic regression analyses were conducted to determine which baseline RAVLT variables were the strongest predictors of cognitive outcome (Stable or Declining) at follow-up. Variables included the TSS indices (Trials 1-5, Immediate Recall, and Delayed Recall) as well as the IIV measures (Lost and Gained Access). If IIV indices were significant predictors in models including TSS, this finding would indicate that intertrial fluctuation provides predictive utility beyond measures of total score alone.

For each model, the ability to distinguish between groups was evaluated with the Nagelkerke R² and concordance (C) index. The Nagelkerke R² assesses the importance of the predictors in a given model relative to a “perfectly fitting” null model (Nagelkerke, 1991). The C index reflects the proportion of all possible pairs of declining and stable participants in which the stable participant has a lower predicted probability of decline compared to the declining participant (Harrell, 2001). Thus, a model that provides no information regarding classification would have a C Index of .50. As such, the C Index is less susceptible to the relative sizes of the groups and the designated cut point for classification. Bayesian Information Criterion values were statistically compared to evaluate relative model fits.

Results

Demographics and Neuropsychological Scores

Thirty-four of the 99 participants were classified as cognitively Declining. The remaining 65 participants were classified as Stable. Demographics and neuropsychological test results are displayed in Table 1. At baseline, although all participants performed within normal limits, the Declining group scored significantly lower on the MMSE, RAVLT Trials 1-5, and RAVLT Immediate Recall. The Declining group also had significantly higher Lost Access and Gained Access and contained a higher proportion of APOE ε4 carriers. No other between-groups differences were observed for demographic variables. At follow-up, the Declining group demonstrated clear reductions on the DRS-2 and TSS variables for the RAVLT. Although the Stable group demonstrated a significant reduction on the DRS-2, no participant had a residual change score of < -1.0.

RAVLT Predictors of Cognitive Decline

Baseline Models

Table 2 presents the results of logistic regression analyses in which the outcome variable was cognitive status at follow-up (Stable or Declining). Five baseline models were conducted to establish the predictive ability of the RAVLT interpretive indices. Baseline Lost Access and Gained Access were both significant predictors of future cognitive status. For both indices, higher values represented a greater predicted probability of decline. Out of the three TSS indices, only baseline Trials 1-5 and Immediate Recall were significant predictors of cognitive decline.

Table 2.

Results of logistic regression models with baseline variables predicting cognitive outcome (Stable/Declining) at 18-month follow-up. Positive regression weights indicate that higher values of the variable represent an increased predicted probability of decline.

Predictive Ability of IIV Indices
	C = .68	R2 = .10				C = .68	R2 = .12
Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p
Int	-2.55	0.78	-3.28	.001	Int	-3.58	1.11	-3.21	.001
Lost	0.18	0.07	2.62	.009	Gained	0.21	0.08	2.74	.006
Predictive Ability of Traditional Variables
	C = .62	R2 = .07				C = .65	R2 = .10				C = .56	R2 = .04
Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p
Int	2.07	1.23	1.68	.093	Int	1.63	0.87	1.87	.061	Int	0.55	0.78	0.70	.483
1-5	-0.06	0.03	-2.21	.027	IR	-0.25	0.09	-2.64	.008	DR	-0.13	0.08	-1.57	.116
Traditional Variables plus Lost Access
	C = .68	R2 = .14	ΔR2	p= .022*		C = .69	R2 = .14	ΔR2	p = .092+		C = .68	R2 = .10	ΔR2	p= .022#
Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p
Int	-0.14	1.62	-0.09	.932	Int	-0.34	1.48	-0.23	.816	Int	-2.30	1.55	-1.49	.137
1-5	-0.04	0.03	-1.63	.104	IR	-0.18	0.10	-1.68	.092	DR	-0.02	0.10	-0.19	.850
Lost	0.16	0.07	2.20	.028	Lost	0.13	0.08	1.65	.099	Lost	0.17	0.08	2.19	.029
Traditional Variables plus Gained Access
	C = .70	R2 = .17	ΔR2	p = .006*		C = .73	R2 = .21	ΔR2	p = .004+		C = .71	R2 = .16	ΔR2	p = .002#
Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	p	Effect	B	SE	Wald Z	P
Int	-0.77	1.73	-0.45	.655	Int	-1.19	1.39	-0.85	.393	Int	-2.25	1.29	-1.75	.080
1-5	-0.05	0.03	-1.94	.052	IR	-0.26	0.10	-2.56	.010	DR	-0.16	0.09	-1.85	.064
Gained	0.20	0.08	2.56	.010	Gained	0.21	0.08	2.67	.008	Gained	0.23	0.08	2.87	.004

Int = Intercept, Lost = Lost Access, Gained = Gained Access, 1-5 = Sum of Trials 1-5, IR = Immediate Recall, DR = Delayed Recall.

^*significance for total model fit compared to 1-5 alone.

⁺significance for total model fit compared to IR alone.

^#significance for total model fit compared to DR alone.

Addition of Lost and Gained Access

Three models tested whether Lost Access could provide incremental predictive ability above and beyond TSS indices. The models containing Trials 1-5 and Delayed Recall (as well as Lost Access) fit the data significantly better than the respective models that included only the TSS variable. The model containing both Immediate Recall and Lost Access did not fit the data significantly better than the model containing Immediate Recall alone.

Three additional models tested the incremental predictive ability of Gained Access compared to TSS indices. All three models containing Gained Access fit the data significantly better than the respective models with the TSS variables alone. These three models also generated higher C and Nagelkerke R² values than the three alternative models in which Lost Access was substituted for Gained Access. Among the three TSS indices, only Immediate Recall was a significant predictor of cognitive decline in the models that also included Gained Access.

Discussion

Our results indicate that embedded measures of IIV derived from the RAVLT are predictive of future declines in cognitive performance in healthy elders who were cognitively intact at baseline. Higher values of both Lost Access and Gained Access were significantly associated with cognitive decline after an 18-month interval. Further, the IIV indices complemented traditional RAVLT indices by explaining unique sources of variance that improved the prediction of cognitive decline. Our findings suggest that IIV indices should be used in conjunction with TSS in evaluating the cognitive functions of older adults.

It is important to note that the purpose of the current study was not to identify individuals with a clinically significant level of cognitive decline, i.e., individuals meeting diagnostic criteria for MCI or dementia. Rather, our definition of cognitive decline was purposely liberal in order to examine elders who are in the very early stages of cognitive decline and exhibit subtle but meaningful changes in cognitive performance. Even though the decline may not be at a clinically significant level, our previous work, using the same criteria and follow-up period as the current study, has shown that declining elders have smaller hippocampal volumes at baseline, differential patterns of fMRI activation in response to semantic and episodic memory tasks at baseline, and are more likely to carry the APOE ε4 allele (Hantke et al., 2013; Seidenberg et al., 2013; Woodard et al., 2010; Woodard et al., 2012). The current work serves to add to this literature by demonstrating that IIV can also serve as a prognostic indicator of future cognitive decline.

High levels of IIV might represent impairment in anterograde memory systems. Thus, analysis of IIV may serve as an additional method for studying episodic memory abilities, which have been strongly linked to risk for cognitive decline (Backman et al., 2001). High IIV might also reflect difficulty with cognitive organization and storage of multiple items, which could be indicative of poor executive control. Executive dysfunction could partially account for the prognostic relationship observed in the current work. Older adults typically display greater IIV compared to younger adults for tasks that place high demands on executive abilities such as ignoring distractors or maintaining multiple items in working memory (West, Murphy, Armilio, Craik, & Stuss, 2002). Thus, older adults who exhibit above-average IIV may exhibit difficulties with executive functioning, which in turn can be a prognostic indicator of impending cognitive decline (Chen et al., 2001).

One counterintuitive finding was that higher values for both Lost Access and Gained Access were associated with an increased predicted probability of cognitive decline. Lost Access is designed to represent rapid forgetting, and loss of information between trials could logically be indicative of cognitive dysfunction. On the other hand, Gained Access is intended to measure learning and benefit from repetition. Theoretically, greater Gained Access should be protective against cognitive decline. However, Gained Access is a complex index, as there are situations where poorer performance can result in greater Gained Access. For example, losing words (Lost Access) provides an opportunity to regain those words (Gained Access) on subsequent trials. We observed a positive correlation between baseline Lost and Gained Access in our sample (r = .66, p < .01), which reflects how the two indices are related. Additionally, a low score on Trial 1 provides more opportunity to have high Gained Access, as more words are available to gain on subsequent trials. This possibility is supported by a negative correlation between Trial 1 scores and Gained Access (r = -.37, p < .01). In contrast, we did not observe a significant correlation between Trial 1 and Lost Access (r = -.05, p = .604)

Due to the high correlation between Lost and Gained Access, we did not include regression models in which both variables served as predictors to avoid multicollinearity. In the models presented in Table 2, no predictor had a variance inflation factor greater than 1.4, suggesting an absence of multicollinearity. Lost Access significantly (p < 0.01) correlated with Trials 1-5, Immediate Recall, and Delayed Recall (r's = -.30, -.50, and -.53, respectively). In contrast, Gained Access was not significantly correlated with the TSS variables (r's = -.13, -.07, and .07, respectively). The lack of significant correlations between Gained Access and the TSS indices may account for why Gained Access was the better predictor of cognitive decline than Lost Access. In other words, Gained Access independently explained more information than Lost Access. We ran additional models containing a combined Lost Access plus Gained Access score that supported this finding. Three models containing this score combined with the TSS indices did not predict significantly more variance than the corresponding three models with Gained Access presented in the bottom row of Table 2.

In the current work, Lost and Gained Access appear to represent a similar underlying construct. This finding stands in contrast to previous research indicating that the two measures represent separate processes. For example, older adults exhibit more Lost Access and less Gained Access than younger adults (Dunlosky & Salthouse, 1996). Likewise, individuals with Alzheimer's disease demonstrate more Lost Access and less Gained Access than cognitively intact older adults (Woodard et al., 1999). These studies suggest that Lost Access represents rapid forgetting while Gained Access represents a benefit from repetition and strong cognitive abilities. Based on these studies, greater Gained Access could be hypothesized to represent superior cognitive functioning and in turn, reduced risk for cognitive decline. Our results seemingly contradict previous findings in suggesting that the two variables are similar in how they predict subsequent decline on episodic recall.

However, there are two crucial differences between the two aforementioned studies and the current work. First, the earlier studies compared two groups with different levels of global cognition, whereas the current study examined a relatively homogenous sample of cognitively intact individuals at baseline. They each reported that the group with greater Lost Access and lesser Gained Access also demonstrated substantially lower Trials 1-5 performance and appeared to have poorer global cognition. Second, our study was longitudinal rather than cross-sectional in nature. We determined to what extent Lost and Gained Access could predict subsequent decline, whereas the previous studies used these indices to distinguish between two groups. In our results, both variables predicted subsequent decline in a similar manner. However, monitoring the change in the IIV indices over the two time points suggested that the two variables indeed represent separate constructs. Specifically, individuals who declined between baseline and 18-month follow-up on Immediate Recall and Delayed Recall also demonstrated increased Lost Access across time points, as demonstrated by inverse correlations between residualized change scores on the indices (r's = -.43 and -.47 for Immediate and Delayed Recall, respectively, p's < .001). In contrast, change in Gained Access across time points was not significantly correlated with change on Immediate or Delayed Recall (r's = -.05 and .09, respectively, p's > .36). Thus, when examining test performance across the 18-month interval the two indices appeared to reflect different constructs, with increased Lost Access being associated with cognitive decline.

A potential criticism of this study involves criterion contamination because two predictors (Trials 1-5 and Delayed Recall) were also used in defining cognitive decline. However, the cognitive decline variable was established from residualized change scores, which factors out interindividual differences in baseline performance. Thus, the change scores solely represent change from baseline to follow-up, and the models detail which variables are most predictive of change. It is important to note that the IIV indices were significant predictors of cognitive outcome and outperformed the two variables that would be the most susceptible to the criterion contamination.

We did not include APOE ε4 allele status in the prediction models, even though this variable is significantly associated with cognitive decline in this longitudinal cohort (Woodard et al., 2010; Woodard et al., 2012). We chose to omit these models in order to primarily focus on the predictive value of IIV. It is notable that when added to the 11 prediction models in Table 2, APOE ε4 allele status was a significant predictor in every model (p's < .04). However, the addition of APOE allele status did not influence whether the RAVLT predictors were statistically significant. Thus, APOE allele status and RAVLT variables predicted separate sources of variance and the inclusion of APOE did not impact whether the RAVLT variables contributed significantly to the model.

A limitation of the current work is the possible lack of generalizability of the findings to the general population of older adults. We purposely enriched our sample with a greater proportion of elders who have a family history of AD and possess one or both APOE ε4 alleles than those in the general population to specifically study the effects of genetic risk factors for AD. Therefore, we suspect that our mean rate of decline is greater than what would be observed in the general population over an 18-month interval. A second limitation of the study is the homogeneity of the sample with regard to its demographic characteristics. The sample was mostly Caucasian and well educated. It is unknown whether our results would generalize to other samples with different characteristics.

In conclusion, the current work suggests that IIV derived from an episodic list-learning task may predict subsequent declines in cognitive performance in elders who were intact at baseline. These findings suggest that utilization of IIV indices may be valuable in interpreting the clinical status of older adults. Relying solely on total performance metrics may overlook valuable and unique information regarding the process by which individuals achieve the total score. Extended longitudinal studies are needed to determine whether IIV indices are predictive of the conversion to clinically diagnostic conditions (e.g., mild cognitive impairment or Alzheimer's disease).