Short-term repeat cognitive testing and its relationship to hippocampal volumes in older adults

Abstract

Background:

Practice effects are improvements in cognitive test scores due to repeated exposure to testing materials. If practice effects provide information about Alzheimer’s disease pathology, then they could be useful for clinical trials enrichment. The current study sought to add to the limited literature on short-term practice effects on cognitive tests and their relationship to neuroimaging biomarkers.

Methods:

Twenty-five, non-demented older adults (8 cognitively intact, 17 with mild cognitive impairment) received magnetic resonance imaging and two testing sessions across one week to determine practice effects on seven neuropsychological test scores. A series of correlations examined if hippocampal volume was associated with baseline, one-week, or practice effects scores on these tests. Next, a series of stepwise multiple regression models examined which of the three test scores best predicted hippocampal volumes.

Results:

In the correlation analysis, baseline scores on 5 of the 7 tests were significantly associated with hippocampal volumes, one week scores were significantly related for 7 of the 7 tests, and practice effects scores were significantly correlated for 4 of the 7 tests. In the stepwise regression models, 5 of the 7 tests indicated that one-week scores best predicted hippocampal volumes. For the other models, baseline score and practice effects score each best predicted hippocampal volume.

Conclusions:

These results add to the growing body of evidence suggesting that diminished practice effects on short-term repeat testing is related to neuroimaging biomarkers of Alzheimer’s disease and may serve as a screening tool for clinical practice and to enrich samples for research trials.

Introduction

Practice effects are improvements in cognitive test performance that occur with repeated exposure to the same or similar test materials [1]. These improvements are frequently observed in cognitively intact individuals, but their magnitude can be influenced by age [2], intellect [3], cognitive domain assessed [4], and clinical condition [4–6]. Although these improvements are typically viewed as a source of error in repeated assessments, practice effects may provide clinically useful information. For example, intact elders tend to show larger practice effects than those with mild cognitive impairments [7–10]. As such, smaller practice effects might be diagnostically useful. Practice effects across shorter periods of time predict cognitive outcomes across longer intervals [11–13]. For example, reduced practice effects across one week might be used prognostically to predict greater cognitive decline across one year. Practice effects have also been indicative of treatment response in older adults with cognitive impairments and patients with schizophrenia [14–17], with patients with smaller practice effects showing less improvement to interventions.

These improvements in test scores due to repeated assessment also seem to offer information about disease pathology associated with Alzheimer’s disease. For example, Mormino et al. [18] reported smaller practice effects across yearly visits in cognitively normal older adults with either beta-amyloid deposition or neurodegeneration on brain imaging compared to older adults without either type of pathology. Across only one week, Duff et al. [19] also observed that smaller practice effects were associated with higher amyloid plaque burden on brain imaging. In this same cohort [20], brain metabolism on FDG-PET was significantly correlated with practice effects across one week. Despite these encouraging results, more work is needed to better examine the utility of using practice effects as a screening measure of brain pathology in Alzheimer’s disease.

Therefore, the purpose of this study was to examine if hippocampal volumes determined by magnetic resonance imaging (MRI) were related to and best predicted by baseline cognitive test scores, one-week cognitive test scores, or practice effects between baseline and one week. It was hypothesized that baseline cognitive test scores would be related to and best predict hippocampal volumes, but that one-week test scores and practice effects would add to this prediction. If short-term repeat testing and/or practice effects are predictive of hippocampal volumes, then they could be used as a brief, affordable, and clinically useful screening method to identify individuals who are likely to have Alzheimer’s related brain pathology.

Methods

Participants.

Twenty-five older adults (19 females/6 males; mean age=77.5 years, SD=6.5; mean education=16.0 years, SD=2.9) were enrolled in this study. These individuals were all recruited from senior centers and independent living facilities to participate in studies on memory and aging. All participants reported to be functionally independent in activities of daily living, and this was corroborated by a knowledgeable informant. Based on objective cognitive testing, the minority of these individuals were classified as cognitively intact (n=8), with the remainder characterized as MCI (n=17) [21], exhibiting at least an amnestic profile. Exclusion criteria for this study included: history of neurological disease known to affect cognition (e.g., stroke, head injury with loss of consciousness of >30 minutes, seizure disorder, demyelinating disorder, etc.); dementia based on DSM-IV criteria; current or past major psychiatric illness (e.g., schizophrenia, bipolar affective disorder); 30-item Geriatric Depression Score >15; history of substance abuse; current use of cholinesterase inhibitors, other cognitive enhancers, antipsychotics, or anticonvulsant medications; history of radiation therapy to the brain; history of significant major medical illnesses, such as cancer or AIDS; and currently pregnant.

Procedures.

The local institutional review board approved all procedures and all participants provided informed consent before data collection commenced. As part of a larger study, all participants completed a neuropsychological battery designed to characterize their functioning on tests of memory and processing speed. The following battery was given during a baseline visit.

• Hopkins Verbal Learning Test - Revised (HVLT-R) is a verbal memory task. In this task, an individual is aurally presented a list of 12 words (4 words from 3 semantic categories). After hearing the list, the individual attempts to freely recall as many of the words as she/he can, in any order. There are 2 additional learning trials, each followed by free recall. Correct responses across these 3 learning trials are summed for the Total Recall score (range=0 – 36). After a 20 – 25-minute delay, free recall of the words is again attempted by the individual. The Delayed Recall score is the correct responses on this trial (range=0 – 12). For all HVLT-R scores, higher values indicate better performance.

• Brief Visuospatial Memory Test - Revised (BVMT-R) is a visual memory task. In this task, an individual is visually presented a card that contains 6 geometric designs in 6 locations on the card. After viewing the card for 10 seconds, the individual attempts to freely draw as many of the designs as he/she can, attempting to place them in their correct location on the page. Points are given for accurate drawing of the design and accurate placement on the page (1 point for each). There are 2 additional learning trials, each followed by free recall. Correct responses across these 3 learning trials are summed for the Total Recall score (range=0 – 36). After a 20 – 25-minute delay, free recall of the designs and locations is again attempted by the individual. The Delayed Recall score is the correct responses on this trial (range=0 – 12). For all BVMT-R scores, higher values indicate better performance.

• Symbol Digit Modalities Test (SDMT) is a divided attention and psychomotor speed task. In this task, an individual uses a key to correctly complete as many symbol and digit pairs as she/he can in 90 seconds. The score is the correct responses on this task (range=0 – 110), with higher values indicating better performance.

• Trail Making Test Parts A and B (TMT-A, TMT-B) are tests of visual scanning/processing speed and set shifting, respectively. In TMT-A, an individual attempts to correctly connect 25 circles as quickly as possible in numerical order. In TMT-B, an individual attempts to correctly connect 25 circles as quickly as possible, alternating between numbers and letters. The score for each part of this test is the time to complete the task, with higher values indicating poorer performance.

• Wide Range Achievement Test – 4^th edition (WRAT-4 Reading) is used as an estimate of premorbid intellect. In this task, an individual attempts to read as many irregular words as possible. The score is the correct responses on this task, and is standardized (M=100, SD=15) compared to age-matched peers, with higher values indicating better performance.

• Geriatric Depression Scale (GDS) is a 30-item screening measure of depressive symptoms in the elderly. For this task, an individual endorses each symptom as yes/no over the past week. Higher scores indicate more depressive symptoms.

After approximately one week (M=7.1 days, SD=0.9, range=6 – 11), the HVLT-R, BVMT-R, SDMT, TMT-A, and TMT-B were repeated. The same form of each test was used to maximize practice effects.

MRI was acquired on a Siemens Trio 3.0T scanner with a standard head coil (Siemens, Erlangen, Germany). The imaging protocol was a sagittal 3D magnetization prepared rapid acquisition gradient-echo (MPRAGE) T1-weighted acquisition (inversion time=1000 ms, echo time=2.08 ms, repetition time=2400 ms, flip angle=8 degrees, field of view=224 mm, slice thickness=0.7 mm, 256 slices). All MRI scans were processed on the same workstation using FreeSurfer image analysis suite v5.3.0 (http://surfer.nmr.mgh.harvard.edu/) to estimate total intracranial and hippocampal volumes. Technical details are described previously [22–24]. Hippocampal volumes were normalized to total intracranial volume.

Statistical analyses.

Three test scores were considered for each cognitive test: 1) raw scores at baseline, 2) raw scores at one-week follow-up, and 3) standardized practice effects scores. Practice effects scores were generated based on the regression formulae developed on a large sample of older adults tested twice across one week [25], which were subsequently validated in cohort of patients with amnestic MCI [26]. These practice effects scores are z-scores, with positive values indicating larger than expected score changes and negative values indicating smaller than expected changes.

Two sets of primary analyses were planned. First, to examine if hippocampal volumes were related to any of the three test scores, Pearson correlations were calculated between normalized bilateral hippocampal volume and baseline, one-week, and practice effects scores. Second, to see which of the three test scores best predicted hippocampal volumes, a stepwise multiple regression model examined if the normalized bilateral hippocampal volume was best predicted by baseline scores, one-week scores, and/or practice effects scores. Separate correlations and regression models were calculated for each of the cognitive scores that were repeated across one week (BVMT-R Total Recall, BVMT-R Delayed Recall, HVLT-R Total Recall, HVLT-R Delayed Recall, SDMT, TMT-A, and TMT-B). An alpha value of 0.05 was used for these comparisons.

Results

The mean bilateral hippocampal volumes normalized to intracranial volume was 0.005 (SD=0.001, range=0.003 – 0.007, which equate to M=6,653.64 mm³, SD=1,205.37, range=4,802.30 – 8,713.10). On average, subjects classified as cognitively intact had significantly greater normalized hippocampal volumes than those classified as MCI (intact: M=0.005, SD=0.001; MCI: M=0.004, SD=0.001, t[23]=2.33, p=0.03).

For the entire group, premorbid intellect was at the upper end of the average range (WRAT-4 Reading: M=109.2, SD=11.1), and minimal depressive symptoms were endorsed (GDS: M=3.2, SD=3.9). The mean performances on the cognitive battery at baseline and one-week follow-up, which are presented in the Table 1, tend to be in the low average to average range, although there was some variability. Across the seven cognitive test scores, baseline and one-week scores correlated between 0.83 – 0.93 (all p’s<0.001). Additionally, statistically significant improvements were observed between baseline and one-week on BVMT-R (Total and Delayed Recall) and HVLT-R (Total and Delayed Recall), but not on the other measures. Practice effects z-scores are also reported in Table 1. Overall, the current sample shows less improvement across one week compared to the participants in Duff (2014).

Table 1.

Cognitive performances at baseline and one week and practice effects

Cognitive measure	Baseline	One-week	Practice effects	r12
HVLT-R Total Recall	21.36 (6.99)	24.72 (8.90)*	−0.36 (1.22)	0.91
HVLT-R Delayed Recall	5.92 (4.71)	7.44 (4.57)*	−0.69 (1.39)	0.93
BVMT-R Total Recall	13.40 (7.73)	18.76 (10.96)*	−0.71 (1.07)	0.92
BVMT-R Delayed Recall	5.08 (3.94)	6.96 (3.71)*	−0.33 (0.92)	0.91
SDMT	38.08 (11.96)	38.68 (14.59)	−0.47 (1.37)	0.92
TMT-A	42.08 (20.36)	39.88 (15.90)	0.31 (0.92)	0.83
TMT-B	133.25 (79.07)	118.33 (73.50)	0.19 (1.30)	0.84

Note. HVLT-R = Hopkins Verbal Learning Test – Revised, BVMT-R = Brief Visuospatial Memory Test – Revised, SDMT = Symbol Digit Modalities Test, TMT = Trail Making Test. Baseline and One-week are raw scores. Practice effects are z-scores based on change across one week using Duff (2014). For Baseline, One-week, and Practice effects scores, means are listed, with standard deviations in the parentheses.

^*= paired sample t-test between baseline and one-week scores were statistically significantly different at p<0.001. r₁₂ = correlation between baseline and one-week scores and all are statistically significant at p<0.001.

In the first set of analyses, the Pearson correlations between normalized hippocampal volumes and the three test scores (baseline, one-week, and practice effects) are presented in Table 2. For the baseline scores, hippocampal volume was significantly related to 5 of the 7 cognitive variables. For the one-week scores, hippocampal volume was related to all 7 variables. Finally, for the practice effects scores, hippocampal volume was related to 4 of the 7 variables.

Table 2.

Correlations between normalized hippocampal volumes and baseline, one-week, and practice effects scores.

Cognitive measure	Baseline	One-week	Practice effects
HVLT-R Total Recall	0.54**	0.56**	0.46*
HVLT-R Delayed Recall	0.59**	0.67**	0.65**
BVMT-R Total Recall	0.58**	0.65**	0.57**
BVMT-R Delayed Recall	0.65**	0.62**	0.26
SDMT	0.39	0.62**	0.73**
TMT-A	−0.35	−0.54**	−0.28
TMT-B	−0.45*	−0.49*	−0.36

Note. HVLT-R = Hopkins Verbal Learning Test – Revised, BVMT-R = Brief Visuospatial Memory Test – Revised, SDMT = Symbol Digit Modalities Test, TMT = Trail Making Test. Baseline and One-week are raw scores. Practice effects are z-scores based on change across one week using Duff (2014).

^*= p<0.05.

^**= p<0.01.

To further inform about which test score was most predictive of hippocampal volumes, a series of stepwise regression models were calculated, with normalized hippocampal volume as the criterion variable and the three test scores as the predictor variables. In these analyses, 5 of the 7 regression models showed that bilateral hippocampal volume was best predicted by the one-week scores: BVMT-R Total Recall (R²=0.42, F[1,24]=16.83, p<0.001), HVLT-R Total Recall (R²=0.32, F[1,24]=10.72, p=0.003), HVLT-R Delayed Recall (R²=0.45, F[1,24]=18.90, p<0.001), TMT-A (R²=0.23, F[1,24]=6.61, p=0.017), and TMT-B (R²=0.24, F[1,24]=6.85, p=0.016). For the BVMT-R Delayed Recall model, the baseline score best predicted hippocampal volume (R²=0.42, F[1,24]=16.79, p<0.001). In the SDMT model, practice effects best predicted hippocampal volumes (R²=0.54, F[1,24]=26.47, p<0.001). For all models, there was only a single statistically significant predictor variable.

Discussion

The current study sought to examine if hippocampal volumes determined by MRI were related to and best predicted by baseline cognitive test scores, those obtained after one week, or a calculation of practice effects between baseline and one week. In line with expectations, hippocampal volumes were significantly related to 5 of the 7 baseline test score variables in the initial correlation analysis. However, one-week repeat testing scores were also significantly related to hippocampal volumes, on all 7 of the cognitive test scores. Practice effects scores were significantly related to 4 of the 7 cognitive test score variables in this study. The relationship between baseline test scores and hippocampal volumes are consistent with multiple prior studies [27–29]. To our knowledge, prior studies have not examined if short-term repeat cognitive testing or practice effects were related to this brain structure. Existing studies have found relationships between practice effects and other brain imaging modalities in non-demented older adults [19, 20].

In the second set of analyses, hippocampal volumes were best predicted by one-week follow-up scores on 5 of the 7 cognitive tests, which included immediate verbal and visual memory, delayed verbal memory, visual scanning, and set shifting. Conversely, only baseline performance on a visual delayed memory test was the best predictor of volumes of this brain structure. Lastly, practice effects on a divided attention test best predicted hippocampal volumes in these non-demented older adults. These results are in line with limited existing literature on repeat testing/practice effects and Alzheimer’s related brain pathology [18–20]. Whereas prior studies have used PET scans (amyloid or FDG) as the marker of pathology, the current study extended that work to MRI, which may indicate that practice effects also track with milder structural changes in the brains of these older individuals.

To our knowledge, no other studies have directly examined this link between hippocampal volume on MRI and practice effects/repeat cognitive testing in individuals at risk for Alzheimer’s disease. In two recent studies [18, 30], investigators categorized older adults as having evidence of neurodegeneration on the basis of hippocampal atrophy and/or hypometabolism on FDG PET. In both studies, evidence of neurodegeneration was associated with diminished practice effects across 24 – 30 months. Although these two prior studies have similar findings to the current one, there are some notable differences. For example, the current study examined repeated testing across one week, whereas the other two used much longer retest intervals. It is suspected that the clinical and research utility of a shorter retest interval would be much higher. The current study included participants with a range of cognitive abilities (i.e., intact to MCI), whereas the prior studies used only cognitively normal individuals. Understanding the relationship between practice effects and imaging biomarkers would appear more useful if it applied to a cognitively heterogeneous sample compared to a narrowly focused one. Admittedly, the sample sizes in the prior studies were much larger (166 – 190 subjects) than the current study (25 subjects), which makes the current results preliminary and in need of replication and extension. Nonetheless, these three studies (and others) point to the value of examining repeat cognitive testing and biomarkers of Alzheimer’s disease.

Numerous studies have reported that hippocampal volumes are related to baseline memory scores in patients at higher risk for Alzheimer’s disease [27–29]. Few have examined if this brain structure is more related to repeat cognitive testing. The potential value of repeat testing, whether across short or long intervals, may be that it provides more information about an individual’s trajectory. For example, for individuals with baseline hippocampal volumes that are relatively small (or average or large), some individuals will remain stable and others will show further reduction in the size of this brain structure. Repeat imaging after 12, 24, or 36 months will reveal the trajectory of the brain integrity. However, if repeat cognitive testing informs us about the changes in the hippocampi (and other brain structures), then this allows clinicians to make more informed decisions about the care of their patients (e.g., more frequent follow-up visits for patients likely to decline). Such information can also allow researchers to enrich clinical trials with individuals who may be more likely to decline (e.g., poorer practice effects and smaller hippocampi). In this way, short-term repeat cognitive testing can be a low cost, non-invasive, and practical method of tracking changes in patients and research participants between imaging sessions.

For the memory measures in the current battery, one week repeat testing on the HVLTR (Total and Delayed Recall) and BVMT-R (Total Recall) was significantly related to normalized hippocampal volumes in these regression models. These results are consistent with prior studies examining neuroimaging and baseline memory testing [27–29], as well as theories about the role of the hippocampus on memory functioning [31]. Surprisingly, the baseline score on the Delayed Recall trial on the BVMT-R was most related to hippocampal volumes, and neither the one-week score nor the practice effects scores on this same test significantly added to this prediction. In prior studies with similar cohorts and study designs, practice effects on the BVMTR was predictive of cognitive decline [12], amyloid deposition [19], and brain hypometabolism [20]. It is possible that the relationship between Delayed Recall on the BVMT-R and brain functioning is most noticeable early in the development of cognitive decline (e.g., cognitive changes, amyloid deposits), and the association is diminished later in the course (e.g., atrophy of brain structures).

Repeat testing after one week and short-term practice effects were most strongly related to all three processing speed measures in the current battery: SDMT, TMT-A, and TMT-B. The connection between these speeded measures that also tap into executive functioning are consistent with prior studies linking baseline processing speed/executive functioning scores and other biomarkers in Alzheimer’s disease (e.g., amyloid-beta in plasma and cortical thinning [32]; amyloid-beta in cerebrospinal fluid [33]; amyloid-beta determined by PET imaging [34]). Interestingly, only SDMT utilized practice effects across one-week in best predicting hippocampal volumes, contributing 54% of the variance. SDMT (and other tests similar to it) seem to be particularly sensitive to brain dysfunction in a range of neuropsychiatric conditions, and practice effects on this test have been observed in clinical trial of patients with multiple sclerosis [35].

Despite the potentially useful findings, some limitations of this study should be acknowledged. First, these results should be viewed cautiously as the sample size was small. As noted above, larger studies with a wider range of cognitive functioning would better test this hypothesis. Secondly, the sample was relatively homogeneous (e.g., all Caucasian, highly educated, mostly female, healthy enough to complete an MRI scan), and the ability to generalize these findings to a more diverse group is unknown. Third, other biomarkers of Alzheimer’s disease (e.g., APOE, cerebrospinal fluid markers, etc.) were not part of this research protocol, and they could be examined in future studies. Fourth, our sample was quite mild in their cognitive dysfunction, with a mixture of cases classified as cognitively intact and amnestic MCI. It is unclear if these findings would replicate in more impaired samples. However, since clinical trials appear to be moving towards earlier points in the disease spectrum, these findings might be less relevant to more advanced cases. Regardless of these limitations, the current study found that hippocampal volume was best predicted by short-term repeat testing and practice effects on multiple cognitive tests in non-demented community-dwelling older adults. Future examination of practice effects as a screening tool in preventative clinical trials in Alzheimer’s disease is warranted.