Short-Term Practice Effects and Amyloid Deposition: Providing Information Above and Beyond Baseline Cognition

Abstract

BACKGROUND

Practice effects, which are improvements in cognitive test scores due to repeated exposure to testing materials, may provide information about Alzheimer’s disease pathology, which could be useful for clinical trials enrichment.

OBJECTIVES

The current study sought to add to the limited literature on short-term practice effects on cognitive tests and their relationship to amyloid deposition on neuroimaging.

PARTICIPANTS

Twenty-seven, non-demented older adults (9 cognitively intact, 18 with mild cognitive impairment) received amyloid imaging with 18F-Flutemetamol, and two cognitive testing sessions across one week to determine practice effects.

RESULTS

A composite measure of 18F-Flutemetamol uptake correlated significantly with all seven cognitive tests scores on the baseline battery (r’s = −0.61 – 0.59, all p’s<0.05), with higher uptake indicating poorer cognition. Practice effects significantly added to the relationship (above and beyond the baseline associations) with 18F-Flutemetamol uptake on 4 of the 7 cognitive test scores (partial r’s = −0.45 – 0.44, p’s<0.05), with higher uptake indicating poorer practice effects. The odds ratio of being “amyloid positive” was 13.5 times higher in individuals with low practice effects compared to high practice effects.

CONCLUSIONS

Short-term practice effects over one week may be predictive of progressive dementia and serve as an affordable screening tool to enrich samples for preventative clinical trials in Alzheimer’s disease.

Introduction

Practice effects are improvements in cognitive test performance that occur with repeated evaluation using the same or similar test materials (1). These improvements are routinely observed in cognitively intact individuals, but their magnitude can be influenced by age (2), intellect (3), cognitive domain assessed (4), and clinical condition (4–6). Traditionally viewed as a source of error in repeated assessments, practice effects may provide clinically useful information. For example, differences in practice effects separate intact elders from those with milder cognitive impairments (7–10). Prognostically, practice effects across shorter periods of time predict cognitive outcomes across longer intervals (11–13). Practice effects have also been positively correlated with treatment response in older adults with cognitive impairments and patients with schizophrenia (14–17).

These improvements in test scores due to repeated assessment may also offer information about disease pathology. For example, three studies have recently examined practice effects and brain pathology associated with Alzheimer’s disease. Mormino et al. (18) reported diminished practice effects across yearly visits in cognitively normal older adults with either beta-amyloid deposition or neurodegeneration on brain imaging compared to older adults without either type of pathology. In another study of older adults with varying levels of cognitive impairment (19), practice effects across one week on a single visual memory test were negatively correlated with β-amyloid neuritic plaque density utilizing PET imaging, with lower practice effects associated with higher amyloid plaque burden. In this same cohort (20), brain metabolism on FDG-PET was significantly correlated with practice effects across one week, even after controlling for baseline cognition. Despite these encouraging results, more work is needed to better examine the utility of using practice effects as a screening measure of brain pathology in Alzheimer’s disease.

Therefore, the purpose of this study was to examine the relationship between short-term practice effects on a broad battery of cognitive tests and amyloid deposition via PET imaging in a cohort of non-demented older adults. It was hypothesized that increased amyloid deposition would be negatively correlated with both baseline cognitive scores and short-term practice effects in this sample. If practice effects were predictive of amyloid deposition, then they could be used as an affordable screening method to identify individuals who are likely to be amyloid positive, which could enrich samples for preventive clinical trials (e.g., Anti-Amyloid Treatment in Asymptomatic AD [A4] study).

Methods

Participants

Twenty-seven older adults (20 females/7 males, mean age=77.5 [6.4] years, mean education=16.3 [3.0] years) were enrolled in this study. These individuals were all recruited from senior centers and independent living facilities to participate in studies on memory and aging. All participants reported to be functionally independent in activities of daily living, and this was corroborated by a knowledgeable informant. Based on objective cognitive testing, the minority of these individuals were classified as cognitively intact (n=9), with the remainder characterized as MCI (n=18) (21), exhibiting at least an amnestic profile. Exclusion criteria for this study included: history of neurological disease known to affect cognition (e.g., stroke, head injury with loss of consciousness of >30 minutes, seizure disorder, demyelinating disorder, etc.); dementia based on DSM-IV criteria; current or past major psychiatric illness (e.g., schizophrenia, bipolar affective disorder); 30-item Geriatric Depression Score >15; history of substance abuse; current use of cholinesterase inhibitors, other cognitive enhancers, antipsychotics, or anticonvulsant medications; history of radiation therapy to the brain; history of significant major medical illnesses, such as cancer or AIDS; and currently pregnant.

Procedures

The local institutional review board approved all procedures and all participants provided informed consent before data collection commenced. As part of a larger study, all participants completed a neuropsychological battery designed to characterize their functioning on tests of memory and processing speed. The following battery was given during a baseline visit.

• Hopkins Verbal Learning Test - Revised (HVLT-R) is a verbal memory task. In this task, an individual is aurally presented a list of 12 words (4 words from 3 semantic categories). After hearing the list, the individual attempts to freely recall as many of the words as she/he can, in any order. There are 2 additional learning trials, each followed by free recall. Correct responses across these 3 learning trials are summed for the Total Recall score (range = 0 – 36). After a 20 – 25 minute delay, free recall of the words is again attempted by the individual. The Delayed Recall score is the correct responses on this trial (range = 0 – 12). For all HVLT-R scores, higher values indicate better performance.

• Brief Visuospatial Memory Test - Revised (BVMT-R) is a visual memory task. In this task, an individual is visually presented a card that contains 6 geometric designs in 6 locations on the card. After viewing the card for 10 seconds, the individual attempts to freely draw as many of the designs as he/she can, attempting to place them in their correct location on the page. Points are given for accurate drawing of the design and accurate placement on the page (1 point for each). There are 2 additional learning trials, each followed by free recall. Correct responses across these 3 learning trials are summed for the Total Recall score (range = 0 – 36). After a 20 – 25 minute delay, free recall of the designs and locations is again attempted by the individual. The Delayed Recall score is the correct responses on this trial (range = 0 – 12). For all BVMT-R scores, higher values indicate better performance.

• Symbol Digit Modalities Test (SDMT) is a divided attention and psychomotor speed task. In this task, an individual uses a key to correctly complete as many symbol and digit pairs as she/he can in 90 seconds. The score is the correct responses on this task (range = 0 –110), with higher values indicating better performance.

• Trail Making Test Parts A and B (TMT-A, TMT-B) are tests of visual scanning/processing speed and set shifting, respectively. In TMT-A, an individual attempts to correctly connect 25 circles as quickly as possible in numerical order. In TMT-B, an individual attempts to correctly connect 25 circles as quickly as possible, alternating between numbers and letters. The score for each part of this test is the time to complete the task, with higher values indicating poorer performance.

• Wide Range Achievement Test – 4th edition (WRAT-4 Reading) is used as an estimate of premorbid intellect. In this task, an individual attempts to read as many irregular words as possible. The score is the correct responses on this task, and is standardized (M = 100, SD = 15) compared to age-matched peers, with higher values indicating better performance.

• Geriatric Depression Scale (GDS) is a 30-item screening measure of depressive symptoms in the elderly. For this task, an individual endorses each symptom as yes/no over the past week. Higher scores indicate more depressive symptoms.

After approximately one week (M = 7.1 days, SD = 0.9, range = 6 – 11), the HVLT-R, BVMT-R, SDMT, TMT-A, and TMT-B were repeated. The same form of each test was used to maximize practice effects.

Participants received 18F-Flutemetamol imaging as described previously (22). 18F-Flutemetamol was produced under PET cGMP standards and the studies were conducted under an approved Federal Drug Administration Investigational New Drug application. Imaging was performed 90 minutes after the injection of 185 mBq (5 mCi) of 18F-Flutemetamol. Emission imaging time was approximately 30 minutes. A GE ST PET/ CT scanner was used for 18F-Flutemetamol imaging in this study, which possessed the full width at half maximum spatial resolution at 5.0 mm. The field of view for reconstruction was set to 25.6 cm on the scanner to generate a pixel size of 2.0 mm × 2.0 mm (image matrix size 128 × 128). The native slice thickness was 3.27 mm. Volumes of interest were automatically generated by the CortexID Suite analysis software (GE Healthcare) with Z axis dimensions substantially larger than the slice thickness. 18F-Flutemetamol binding was analyzed using a regional semi-quantitative technique described by Vandenberghe et al. (23) and refined by Thurfjell et al. (24). In this technique, semi-quantitative regional (prefrontal, anterior cingulate, precuneus/posterior cingulate, parietal, mesial temporal, lateral temporal, occipital, sensorimotor, cerebellar grey matter, and whole cerebellum) and global composite standardized uptake value ratios (SUVRs) in the cerebral cortex were generated automatically and normalized to the pons using the CortexID Suite software (25). This software uses a threshold z score of 2.0 to indicate abnormally increased regional amyloid burden that corresponds to a composite SUVR of 0.59 when normalized to the pons, providing a 99.4% concordance with visual assessment (24). For 18F-Flutemetamol amyloid imaging, there is no specific age-related “normal” level of binding in the CortexID Suite database to assess age-matched normality. Therefore, the study images were compared to the intrinsic software database control group as a whole to calculate the z-scores compared to clinically negative amyloid scans.

Statistical analyses

Raw scores on each test in the cognitive battery were used in all analyses. To generate a practice effects score on each test, a standardized regression-based prediction formula was developed using an independent sample of 167 non-demented older adults who were also administered this cognitive battery at baseline and one-week (26). This methodology, which allowed us to predict one-week scores from observed baseline scores and demographic information (e.g., age, education, gender), is frequently used in the assessment of neuropsychological change (27). The resulting z-score, which reflects how much the predicted one-week score deviated from the observed one-week score for each participant, was used as the practice effects score in all analyses. Two primary analyses were planned. In the first, Pearson correlations were calculated between the 18F-Flutemetamol global composite and the observed baseline raw scores for each cognitive variable. In the other primary analysis, partial correlations were calculated between the 18F-Flutemetamol global composite and the practice effects z-score for each cognitive variable, controlling for the baseline cognitive scores. An alpha value of 0.05 was used for these comparisons. In secondary analyses, we examined the odds ratio of having elevated amyloid deposition based on the level of practice effects across one week.

Results

No adverse events were reported during the injection, uptake time, or imaging studies with the investigational imaging agent 18F-Flutemetamol. The mean global composite of SUVRs normalized to cerebellar cortex was 0.63 (SD=0.18, range=0.41 – 1.09). Of the 27 scans, 13 were categorized as “positive” for 18F-Flutemetamol uptake, using a cutoff of z-score greater than or equal to 2.0. Of these 13 scans with notable 18F-Flutemetamol uptake, 3 were from the 9 participants categorized as cognitively intact (33%) and 10 were from the 18 participants categorized as MCI (56%).

For the entire group, the mean performances on the cognitive battery at baseline and one-week follow-up were in the low average to average range, although there was some variability. Practice effects on these measures also fell in the low average to average range, again with some variability. Relevant cognitive scores are presented in the Table 1.

Table 1.

Cognitive performances and correlations with 18F-Flutemetamol global composite

Cognitive measure	Baseline	One-week	Practice effect	r of 18F-Flutemetamol and baseline cognition	partial r of 18F-Flutemetamol and practice effects
HVLT-R Total Recall	21.7 (6.9)	25.2 (8.8)	−0.3 (1.2)	−0.53 p=0.005	−0.22 p=0.28
HVLT-R Delayed Recall	6.2 (4.7)	7.6 (4.5)	−0.7 (1.3)	−0.58 p=0.002	0.03 p=0.90
BVMT-R Total Recall	13.9 (8.0)	19.6 (11.0)	−0.7 (1.1)	−0.60 p=0.001	−0.39 p=0.04
BVMT-R Delayed Recall	5.3 (4.0)	7.3 (3.7)	−0.3 (1.0)	−0.61 p=0.001	−0.45 p=0.02
SDMT	38.6 (11.7)	39.4 (14.3)	−0.4 (1.3)	−0.53 p=0.004	−0.43 p=0.03
TMT-A	41.4 (19.7)	38.8 (15.7)	0.2 (0.9)	0.59 p=0.001	0.09 p=0.67
TMT-B	130.0 (76.8)	114.9 (71.9)	0.1 (1.3)	0.50 p=0.01	0.44 p=0.03

Note. HVLT-R = Hopkins Verbal Learning Test – Revised, BVMT-R = Brief Visuospatial Memory Test – Revised, SDMT = Symbol Digit Modalities Test, TMT = Trail Making Test. Baseline and One-week are raw scores. Practice effects are z-scores based on standardized regression-based change formula, with M=0, SD=1.

In the primary analyses, the correlations between 18F-Flutemetamol uptake and 1) the observed baseline raw score, and 2) the practice effects z-score controlling for the baseline score are presented in Table 1. In this analysis, 18F-Flutemetamol uptake was significantly correlated with all seven cognitive test scores at baseline (r’s = −0.61 – 0.59, all p’s<0.05). In the second analysis, after controlling for baseline cognition, 18F-Flutemetamol uptake continued to be significantly related to practice effects z-scores for 4 of the 7 measures (partial r’s = −0.45 – 0.44, p’s<0.05).

In secondary analyses, to further examine practice effects as an enriching strategy for clinical trials requiring amyloid positivity as an entrance criterion, each value was dichotomized. Individuals with a global composite of 18F-Flutemetamol SUVR z-score of ≥2.0 were labeled as amyloid positive and those with composites <2.0 were amyloid negative (24). For practice effects, individuals with a mean practice effects z-score across all seven measures of ≤−0.50 were labeled as low practice effect and those with mean z-scores >−0.50 were high practice effects (19). Using this dichotomization strategy, 13 individuals were amyloid positive (48%) and 14 were amyloid negative (52%); 11 had low practice effects (41%) and 16 had high practice effects (59%). As can be seen in Table 2, of those with notable 18F-Flutemetamol uptake (i.e., positive amyloid scans), 9 had low practice effects (69%) and 4 had high practice effects (31%). Of those without notable 18F-Flutemetamol uptake (i.e., negative amyloid scans), 2 had low practice effects (14%) and 12 had high practice effects (86%). These differences are statistically significant (χ2[1]=8.4, p=0.004). Based on these two dichotomous groupings, the odds ratio of having a positive amyloid scan was 13.5 times higher if the individual had low practice effects compared to high practice effects.

Table 2.

Practice effects and 18F-Flutemetamol global composite

Increased 18F-Flutemetamol Uptake	Low practice effects	High practice effects
Yes	9	4
No	2	12

Note. Notable 18F-Flutemetamol uptake: Yes = 18F-Flutemetamol global composite z-score ≥2.0, No = 18F-Flutemetamol global composite z-score <2.0; Low practice effects = reliable change index ≤−0.50: High practice effects = reliable change index >−0.50

Discussion

The current study sought to examine the relationship between short-term practice effects on a battery of cognitive tests and amyloid deposition via PET imaging in a sample of non-demented older adults. To rigorously inspect the association between practice effects and amyloid uptake, it seemed important to also consider the relationship between baseline cognition and amyloid deposition. Consistent with expectations, the amount of 18F-Flutemetamol uptake was negatively correlated with baseline cognitive scores in this cohort, with greater amyloid uptake being associated with lower performances on all seven cognitive test scores. These results are in line with existing literature (22, 28, 29).

More relevant to the purpose of the study, one-week practice effects were also significantly related to amyloid deposition, after controlling for baseline cognition. For memory measures, practice effects scores on the BVMT-R (Total and Delayed Recall) were significantly related to amyloid deposition in these partial correlations. These results are consistent with those reported by Duff et al. (19), who also showed that practice effects on this visual memory measure seem to provide valuable information about brain pathology. Conversely, practice effects on the HVLT-R, the verbal memory test, did not significantly correlate with amyloid uptake after controlling for baseline scores on this test. There may be multiple reasons for this discrepancy. For example, in our experience with both measures, the BVMT-R appears more challenging, as subjects need to learn 6 shapes in 6 locations on a page, which is a less familiar task than HVLT-R’s list learning format. Additionally, in prior studies with similar samples, practice effects on the BVMT-R tend to be larger (9) and tell us more about future cognition (30) than practice effects on the HVLT-R.

For the processing speed measures, practice effects on the SDMT and TMT-B were significantly related to amyloid uptake, above and beyond baseline scores on these tests. The connection between these speeded measures that also tap into executive functioning are consistent with prior studies linking baseline processing speed/executive functioning scores and amyloid, whether assessed via plasma (31), cerebrospinal fluid (32), or PET imaging (29). However, to our knowledge, this is the first study examining practice effects on these tests and amyloid deposition via PET.

As clinical trials are moving towards preventative studies in prodromal individuals who are biomarker positive for Alzheimer’s disease (33), methods are needed to enrich samples with those most likely to be biomarker positive or these trials will deplete valuable resources (34). Our results suggest that baseline cognitive testing and one-week practice effects may be an affordable and non-invasive first-line screening option to enrich preventive trials. For example, in our secondary analyses, when practice effects and amyloid status were dichotomized, those having greater 18F-Flutemetamol uptake/positive amyloid scan were 13.5 times more likely to have low practice effects compared to high practice effects. In the Anti-Amyloid Treatment in Asymptomatic Alzheimer’s Disease (A4) trial, Sperling et al. (35) indicated needing 1,000 healthy older individuals who show elevated amyloid accumulation. Based on their estimate that ~30% of older individuals will have sufficient amyloid accumulation, they will need to scan 3,333 individuals to get their 1,000 amyloid positive subjects. Using a more conservative odds ratio of 6, the A4 trial would only need to recruit 2,000 subjects with low practice effects to get their 1,000 amyloid positive subjects, which would be a reduction in sample size of 40%. In addition to the reduced sample size, there are other cost savings from using practice effects to enrich this trial. For example, if an amyloid scan costs $5,000, then it would cost about $16.6 million to obtain scans on 3,333 subjects. However, using the practice effects-enriched sample of 2,000 subjects, it would cost $10 million to obtain these amyloid scans (or 40% savings in scanning costs). There would be additional costs of collecting practice effects data, but these would be minimal compared to amyloid imaging.

Compared to our prior study examining this similar topic (19), the odds ratio of amyloid positivity with low practice effects was notably higher in the current study (5 vs. 13.5, respectively). Although both of these studies should be viewed as preliminary, one reason for the higher odds ratio in the current study might be due to the greater percentage of MCI patients in the current study compared to Duff et al. (67% vs. 40%, respectively). Given the potential of differential relationships between practice effects and amyloid deposition at different points along the Alzheimer’s disease continuum, future studies should examine more cognitively diverse samples. However, if these findings are replicated in larger samples, then practice effects have the potential to reduce financial costs, personnel time, and participant involvement in those trials.

Despite the potentially useful findings, some limitations of this study should be acknowledged. First, these results should be viewed cautiously as the sample size was small. As noted above, larger studies with a wider range of cognitive functioning (including practice effects) would better test this hypothesis. Secondly, the sample was relatively homogeneous (e.g., all Caucasian, highly educated, mostly female, healthy enough to complete a PET scan), and the ability to generalize these findings to a more diverse group is unknown. Third, other biomarkers of Alzheimer’s disease (e.g., APOE, cerebrospinal fluid markers, etc.) were not part of this research protocol, and they could be examined in future studies. Fourth, our sample was quite mild in their cognitive dysfunction, with a mixture of cases classified as cognitively intact and amnestic MCI. The current study focused on those who were “biomarker positive” for a suspected Alzheimer’s disease pathology, which is not the same thing as having Alzheimer’s disease dementia. As noted earlier, it is unclear if these findings would replicate in more impaired samples. However, since clinical trials appear to be moving towards earlier points in the disease spectrum, the relevance of these findings to more advanced cases might be less. Regardless of these limitations, the current study found notable relationships between amyloid-binding with 18F-Flutemetamol and practice effects in non-demented community-dwelling older adults, and future examination of practice effects as a screening tool in preventative clinical trials in Alzheimer’s disease seems warranted.