Abstract
Background:
The Learning Ratio (LR) is a new learning score created to examine the proportion of information learned over successive learning trials relative to information available to be learned. LR factors in the opportunity for future learning that will vary depending on performance during the initial learning trial. Validation of this novel LR score is warranted to understand its sensitivity to Alzheimer’s disease (AD) pathology.
Method:
One-hundred twenty-three participants across the AD continuum underwent memory assessment, quantitative brain imaging, and genetic analysis. LR scores were calculated from the HVLT-R, BVMT-R, RBANS List Learning, and RBANS Story Memory, and compared to total hippocampal volumes, 18F-Flutemetamol composite SUVR uptake, and APOE ε4 status.
Results:
Lower LR scores were consistently associated with smaller total hippocampal volumes, greater cerebral β-amyloid deposition, and APOE ε4 positivity. This LR score outperformed a traditional learning slope calculation – raw improvement subsequent to the initial learning trial – in all analyses, and were generally equivalent across measures using different trial length and trial number.
Conclusions:
LR is sensitive to AD pathology along the AD continuum – more so than the traditional raw learning score – and reducing the competition between the first trial and subsequent trials can better depict learning capacity.
Keywords: Learning, Learning Slope, Alzheimer’s disease, Mild Cognitive Impairment, MRI, Amyloid-PET, APOE ε4
INTRODUCTION
Quantifying learning curves for memory tasks reflects the rate of learning (Jones et al., 2005). Differences in learning slopes are frequently observed in individuals with normal cognition, Mild Cognitive Impairment (MCI), or Alzheimer’s disease (AD). Weaker learning curves signify less potential to benefit from repeated exposure to information over multiple trials, which is associated with greater severity of cognitive impairment (Gifford et al., 2015). Although the relationship between episodic memory and AD biomarkers like hippocampal volume, cerebral β-amyloid deposition, and Apolipoprotein ε4 (APOE ε4 allele) carrier status has been well studied, limited investigation has been undertaken to date between learning slopes and AD biomarkers.
To date, the focus on learning slopes and AD biomarkers has been related to hippocampal volumes. Gifford and colleagues (2015) investigated structural imaging markers (hippocampal volume and cortical thickness across a variety of brain regions) and a variety of learning slopes for a commonly administered memory task. Among patients with MCI, they observed that stronger learning slopes (denoting greater learning) were related to larger hippocampal volumes and both greater ventrolateral prefrontal cortex and greater parahippocampal thickness. No significant relationships were observed between learning and structural imaging markers in normal control or AD samples. Additionally, Bender and colleagues (Bender et al., 2020) found that the interaction of greater hippocampal volume and limbic white matter microstructure predicted larger verbal learning rates in a population-based sample of older adults, although hippocampal volume did not predict learning on its own. Further, Bonner-Jackson and colleagues (Bonner-Jackson et al., 2015) observed positive associations between bilateral hippocampal volumes and performance on both the Hopkins Verbal Learning Test – Revised (HVLT-R; Brandt & Benedict, 1997) and the Brief Visual Memory Test – Revised (BVMT-R; Benedict, 1997). In a sample of memory clinic patients, they observed that BVMT-R learning slope – using what will be described below as the raw learning score – possessed a stronger positive relationship with hippocampal volumes than HVLT-R learning slope.
Conversely, investigation of relationships between learning slopes and other AD biomarkers has been minimal. No research has been reported examining the association between learning slope and cerebral β-amyloid deposition. Only one study has been reported investigating learning slope and APOE ε4 status. Specifically, Wehling and colleagues (Wehling et al., 2007) observed that while APOE ε4 status was associated with a variety of cognitive measures – including initial learning in a memory clinic sample, carrier status did not impact the slope of the learning curve.
Recently, Spencer and colleagues (Spencer et al., 2020) developed the Learning Ratio (LR) in an effort to overcome the inherent competition between the first trial – or Trial One – performance and performance on subsequent trials of a multi-trial learning task in most learning slope calculations. Traditionally, performance at Trial One impacts the slope of the learning curve, as more items learned at Trial One leaves fewer opportunities to improve in future trials. For example, consider two patients being assessed on a 12-item learning test over multiple trials, with Patient One recalling 1 word at Trial One and 4 words at the Final Trial, and Patient Two recalling 9 words at Trial One and 12 words at the Final Trial. Traditional learning slope calculations tend to consider learning slope – or raw learning score (RLS) – as performance on the Final Trial minus performance on Trial One (see RLS equation in the Methods), therefore both patients would be shown to obtain a learning slope score of 3. This comparable learning slope score is counter to the apparent disparity in learning efficiency between these individuals.
LR, conversely, is calculated as the number of items learned after Trial One divided by the number of items still yet to be learned after Trial One (see LR equation in the Methods; Spencer et al., 2020). Consequently, LR represents the proportion of information learned over successive trials relative to information available to be learned, and factors in the opportunity for future learning that will vary depending on Trial One performance. From our example above, because Patient One learned only 1 item on Trial One of a 12-item word list, (s)he has the potential to learn 11 more words on subsequent trials. In contrast, because Patient Two learned 9 items on Trial One from the 12-item word list, (s)he has the opportunity to learn only three more items on future trials. Both patients learned 3 more words at the Final Trial relative to Trial One, therefore Patient One’s LR score would be 0.27 (or 27%; 3/11), whereas Patient Two’s LR score would be 1.00 (or 100%; 3/3). These LR differences reflect the intuitive difference in learning ability between the two individuals.
Preliminary validation of this LR equation for the List Learning and Story Memory tasks of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 2012) was established by Spencer et al. (2020) for a sample of 289 older veterans from an outpatient memory clinic. Their findings suggested that this LR equation, and the associated Aggregate score, possessed superior correlations with traditional measures of memory when compared with RLS. The LR composite score also better discriminated between patients with and without a neurocognitive diagnosis. Hammers and colleagues (Hammers et al., In Press) subsequently showed validity for this LR learning slope when calculated from performances on the HVLT-R and BVMT-R in an independent sample of 56 memory clinic patients. Lower LR scores were associated with smaller hippocampal volumes using quantitative brain imaging, along with worse performances on traditional memory measures. Exploratory analyses showed that LR scores were smaller for patients with AD than those with MCI. Further, all findings in Hammers et al. indicated greater effects for the LR calculation than for traditional learning slope scores.
The current study sought to expand the limited research on the relationship between learning slopes and AD biomarkers to include LR metrics calculated from the HVLT-R, BVMT-R, RBANS List Learning, and RBANS Story Memory. Specifically, we investigated this novel LR calculation and its associations with total hippocampal volume, cerebral β-amyloid deposition, and APOE ε4 status using an independent sample of well-characterized older adults along the AD continuum. This study represents an improvement over Hammers and colleagues’ recent work by using more robust sample sizes and incorporating a larger set of AD biomarkers (i.e., Hammers et al. only evaluated hippocampal volumes). LR is hypothesized to be positively associated with hippocampal volumes and negatively associated with β-amyloid deposition, such that worse learning ratios will be related to smaller hippocampal volumes and greater β-amyloid deposition in our sample. It is additionally anticipated that lower LR performance will be related to positive APOE ε4 status. Overall, should our hypotheses be correct, our results would provide further support for use of LR score to assess learning slope in participants along the AD continuum, and suggest that this learning slope metric is sensitive to AD pathology.
METHODS
Sample
Our sample included 123 older adults recruited either from a memory disorders clinic at a university medical center (49%) or through the community (51%). The mean age of our total sample of participants was 74.1 years old (SD = 5.7; range = 65 – 91) and the mean education was 16.1 years (SD = 2.4; range = 12 – 20). Please see Table 1. The mean premorbid intellectual functioning was in the average range (Standard Score Mean = 109.5; SD = 8.5; range = 88 – 134) according to the Wide Range Achievement Test – Fourth Edition Reading subtest (WRAT-4; Wilkinson & Robertson, 2006). Approximately half the sample was female (58.5%) and most participants were white (98.4%).
Table 1.
Variable | Total Sample | Normal Cognition | MCI | AD |
---|---|---|---|---|
n | 123 | 52 | 37 | 34 |
Age | 74.1 (5.7) | 72.5 (4.9) | 73.4 (5.0) | 77.2 (6.4) |
Education | 16.1 (2.4) | 16.7 (2.1) | 15.3 (2.7) | 16.0 (2.3) |
Sex (% female) | 58.5% | 61.5% | 56.8% | 55.6% |
Race (% Caucasian) | 98.4% | 100.0% | 94.6% | 100.0% |
WRAT | 109.5 (8.5) | 110.6 (7.4) | 107.3 (9.9) | 110.3 (8.3) |
RBANS Total Scale | 91.0 (22.1) | 110.9 (13.4) | 82.8 (12.0) | 69.7 (14.0) |
RBANS Immediate Memory Index | 87.8 (22.5) | 107.5 (13.4) | 79.0 (15.7) | 67.3 (13.9) |
RBANS Delayed Memory Index | 82.0 (29.4) | 110.6 (12.1) | 69.9 (20.0) | 51.7 (11.3) |
HVLT-R Total Recall | 41.8 (15.1) | 54.9 (8.7) | 36.7 (11.0) | 27.2 (8.4) |
HVLT-R Delayed Recall | 37.5 (16.8) | 53.9 (7.8) | 28.8 (11.5) | 21.9 (6.5) |
BVMT-R Total Recall | 41.6 (16.2) | 56.8 (8.9) | 34.8 (11.5) | 25.8 (5.2) |
BVMT-R Delayed Recall | 40.9 (17.6) | 57.5 (8.0) | 35.0 (12.9) | 21.9 (3.8) |
Note: MCI = Mild Cognitive Impairment, AD = Alzheimer’s Disease, WRAT = Wide Range Achievement Test – Fourth Edition Reading Subtest, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, HVLT-R = Hopkins Verbal Learning Test – Revised, BVMT-R = Brief Visuospatial Memory Test – Revised. WRAT score and RBANS scores listed as a Standard Score, and HVLT-R and BVMT-R scores are listed as T Scores. All values are Mean (Standard Deviation) unless listed otherwise.
Categorization of participants into diagnostic groups was based on a two-step approach. First, for participants recruited from the clinic – which included 100% of the AD dementia participants (hereafter referred to as “AD” participants) and approximately 85% of the MCI participants – information was obtained from multi-disciplinary consensus clinical diagnosis (from neurological consultations, comprehensive neuropsychology evaluations, and brain MRI/FDG-PET/Amyloid-PET imaging) prior to the study to inform diagnosis. Second, the classification battery developed by ADNI (ADNI2, 2020) was used to confirm diagnosis. These ADNI measures included the Clinical Dementia Rating Scale (CDR; Morris, 1993), the Mini Mental State Examination (MMSE; Folstein et al., 1975), and the Wechsler Memory Scale-Revised (WMS-R; Wechsler, 1987) Logical Memory II Paragraph A. Recruitment of participants from the community – which included 100% of the Normal Cognition participants and approximately 15% of the MCI participants –relied solely on the application of ADNI2 criteria. Of note, no data from the HVLT-R, BVMT-R, or RBANS – nor learning slope indices – were used to inform diagnosis either from the pre-study consensus diagnostic evaluations or from the current study in order to reduce circularity between diagnosis and LR calculations, which has hindered previous learning slope research (Hammers et al., In Press; Spencer et al., 2020). Diagnostic composition of participants after applying clinical consensus and ADNI2 criteria included Normal Cognition (n = 52), MCI (n = 37), and mild AD (n = 34). Table 1 displays the significant differences between diagnostic groups that can be observed for age, F(2, 120) = 8.52, p < .001, and education, F(2, 120) = 4.00, p = .02. AD participants were older than both the Normal Cognition (p < .001) and MCI groups (p = .01), and Normal Cognition participants possessed higher education than MCI participants (p = .02). No differences can be observed between groups for premorbid intellect or other demographic variables (all ps > .05). As can be expected based on diagnosis, the AD group performed significantly worse on all immediate and delayed memory measures (HVLT-R, BVMT-R, and RBANS Immediate/Delayed Memory Indexes) than the MCI group (all ps < .005), who performed worse than the Normal Cognition group (all ps < .001). Expanded results (e.g., F values, effect sizes) are available upon request.
Study exclusion criteria involved being < 65 years of age, not having a knowledgeable collateral source available to comment on the participant’s cognition and daily functioning, and medical comorbidities likely to affect cognition (including neurological conditions like stroke, epilepsy/seizure, brain tumor, and TBI; current severe depression; substance abuse; and major psychiatric conditions). The mean total score for the Modified Hachinski Ischemic Scale (Rosen et al., 1980) for the sample was 1.34 (SD = 1.3), and the means for the Charlson Comorbidity Index Total Score (Charlson et al., 1987) was 1.21 (SD = 1.4) and the Chronic Disease Score (Von Korff et al., 1992) was 3.69 (SD = 3.7). Participants with a past history of cancer whose course or treatment was not felt to impact cognition were included, which was relevant for 21 participants. A total of 18 participants self-reported possessing a diagnosis of diabetes mellitus II in the current study. Additional exclusion criteria included the inability to complete Magnetic Resonance Imaging (MRI) or Positron Emission Tomography (PET) imaging, being enrolled in a clinical drug trial related to anti-amyloid agents, scoring greater than 5 on the 15-item Geriatric Depression Scale (GDS; Yesavage et al., 1982), or having moderate or severe dementia as indicated by a CDR score of 2 or greater or a MMSE score of less than 20.
Procedure
Procedures were approved by the local Institutional Review Board before participants enrolled. Following informed consent, participants underwent neuropsychological testing. Approximately 10 days later (M = 10.7 days; SD = 19.2 days) participants received an MRI of the brain, and approximately 20 days after cognitive testing (M= 19.5 days; SD = 16.9 days) participants returned to receive amyloid-PET imaging of the brain using 18F-Flutemetamol and receive a blood draw that assessed for APOE ε4 status.
Neuropsychological and Behavioral Measures
In addition to the diagnostic classification measures, all participants underwent neuropsychological testing as follows:
The WRAT-4 (Wilkinson & Robertson, 2006) Reading subtest was administered to assess premorbid intellect. This task requires participants to accurately pronounce 55 irregular words presented to them on a list. The number of correct responses is summed and normative comparisons are used to create standard scores corrected for age (M=100, SD=15). Higher scores indicate higher premorbid intellectual functioning.
The HVLT-R (Brandt & Benedict, 1997) is a verbal memory task with 12 words learned over three trials, with the correct words summed for the Total Recall score (range = 0 – 36). The Delayed Recall score is the number of correct words recalled after a 20 – 25-minute delay (range = 0 – 12).
The BVMT-R (Benedict, 1997) is a visual memory task with 6 geometric designs in 6 locations on a card learned over three trials, with correct designs and locations summed for the Total Recall score (range = 0 – 36). The Delayed Recall score is the number of correct designs and locations recalled after a 25-minute delay (range = 0 – 12).
The RBANS (Randolph, 2012) is a brief neuropsychological testing battery that comprises 12 subtests used to calculate 6 index scores in the domains of immediate memory, visuospatial/constructional, attention, language, delayed memory, and a composite score of global neuropsychological functioning. Two of the subtests germane to the current analyses include List Learning (four learning trials of 10 words) and Story Memory (two learning trials of 12 items).
Of note, for all HVLT-R, BVMT-R, and RBANS List Learning and Story Memory scores, raw score values were used in correlational analyses, as described below. Memory measures additionally utilized age-corrected normative comparisons for descriptive statistics. Specifically, HVLT-R and BVMT-R normative comparisons generated T Score values (M = 50, SD = 10), whereas RBANS Index normative comparisons generated Standard Score values (M = 100, SD = 15). Learning slope performances were evaluated by raw data from individual trials of each of the memory measures. For both raw scores and T/Standard Scores, higher values indicate better performance.
Calculation of Learning Slopes
For the HVLT-R, BVMT-R, and RBANS List Learning and RBANS Story Memory tests, RLS scores were computed as the number of points earned on the Final Trial relative to Trial One. The aggregated RLS score was computed as the sum of the RLS for each test. The LR score is represented as a proportion where differences in performance between the Final Trial and Trial One is in the numerator, and the difference between a maximum score for a trial and performance on Trial One serves as the denominator. The aggregated LR score was computed as the combined difference between Trial One and the Final Trial for both tests, divided by the difference between the combined total points available for a trial for both tests and the sum of Trial One from both tests. The formulas for RLS, LR, and their respective aggregates (HVLT-R and BVMT-R, or RBANS List Learning and RBANS Story Memory) are as follows:
MR Imaging
MRI was acquired on a 3.0-T Siemens Prisma scanner with a standard head coil. Structural data was acquired using an MP2RAGE sequence (TR = 5000, TE = 2.93, acquired sagittally, resolution=1 × 1 × 1 mm) to obtain high quality whole brain 1mm isotropic T1 images with improved signal homogeneity in ~7 minutes. All MRI scans were examined for the presence of common artifacts, including motion, susceptibility, and distortion, and were determined to be of sufficient quality for quantitative analysis. All data were processed on the same workstation using FreeSurfer image analysis suite v6.0 (http://surfer.nmr.mgh.harvard.edu/) to estimate total intracranial and hippocampal volumes. Technical details are described previously (Fischl & Dale, 2000; Fischl et al., 2002; Fischl et al., 2004). To address head size differences, hippocampal volumes have been adjusted by estimated total intracranial volume.
Amyloid Imaging
Amyloid imaging was performed using 18F-Flutemetamol which is a radioactive diagnostic agent indicated for PET imaging of the brain to estimate beta-amyloid neuritic plaque density in adult patients with cognitive impairment. 18F-Flutemetamol was produced under PET cGMP standards and conducted under an approved FDA Investigational New Drug application (IND). Twenty minutes of emission imaging was performed 90 minutes after the injection of approximately 185 mBq (5 mCi) of 18F-Flutemetamol. A GE Discovery PET/CT 710 (GE Healthcare) was used in this study. This PET/CT scanner has full width at half-maximum spatial resolution of 5.0 mm and excellent performance characteristics (Sunderland & Christian, 2015; Yester et al., 2014). Volumes of interest were automatically generated by using the CortexID Suite analysis software (GE Healthcare). 18F-Flutemetamol binding was analyzed using a regional semi-quantitative technique described by Vandenberghe et al. (2010) and refined by Thurfjell et al. (2014) The CortexID Suite software generates, semi-quantitative regional (prefrontal, anterior cingulate, precuneus/posterior cingulate, parietal, mesial temporal, lateral temporal, occipital, sensorimotor, cerebellar grey matter, and whole cerebellum) standardized uptake value ratios (SUVRs) normalized to the pons. A composite standardized uptake value ratio (SUVR) in the cerebral cortex was generated automatically and normalized to the pons using the CortexID Suite software (Lundqvist et al., 2013).
APOE Genotyping
Polymerase Chain Reaction and Fluorescence Monitoring using hybridization probes for APOE genotyping was conducted using whole blood samples. Results were recorded and participants were categorized as being APOE ε4 allele carriers (both hetero- and homozygous) or non-carriers.
Data Analysis
For characterization of LR learning slope performances among diagnostic groups, multiple analysis of covariance (MANCOVA) were conducted correcting for age and education, with one-way analyses of covariance (ANCOVA) analyses conducted for individual memory measures within the omnibus test. Bonferroni post-hoc comparisons were subsequently calculated among diagnostic group performances for significant ANCOVA results. Additionally, bivariate Pearson correlation coefficients were calculated among the LR learning slope metrics for HVLT-R, BVMT-R, RBANS List Learning, and RBANS Story Memory, and then separately among the RLS learning slope metrics, Trial One performances, and total learning score performances.
For the primary biomarker analyses, partial correlation coefficients were calculated comparing HVLT-R, BVMT-R, RBANS List Learning, and RBANS Story Memory learning slope metrics (along with Trial One and Total Recall scores) and both total-intracranial-volume-adjusted hippocampal volumes and cerebral β-amyloid deposition as measured by the cerebral 18F-Flutemetamol composite uptake (SUVR), controlling for age, education, and premorbid intellect. Additionally, MANCOVA was conducted comparing APOE ε4 status and learning slope performances, also using age, education, and premorbid intellect as covariates. To determine appropriateness of covariates, bivariate correlation coefficients were calculated between continuous demographic variables (e.g., age, education, premorbid intellect) and (1) LR scores, (2) total hippocampal volumes, and (3) 18F-Flutemetamol uptake. Relatedly, one-way analyses of variance (ANOVA) were calculated for the categorical demographic variables (e.g., sex and ethnicity) and (1) LR scores, (2) total hippocampal volumes, and (3) 18F-Flutemetamol composite SUVR uptake. For covariates pertaining to APOE ε4, ANOVA were conducted for APOE ε4 and continuous demographic variables, and Chi Square analyses were conducted for APOE ε4 and categorical demographic variables.
Measures of effect size were expressed as r2 (correlational analyses) and partial eta squared (η2; MANCOVA/ANCOVA) values, and comparisons between r values were examined using Fisher r to z transformations. To protect against multiple comparisons, a two-tailed alpha level was set at .01 for all primary analyses.
RESULTS
Learning Slope Characteristics
Please see Table 2 for total sample and diagnostic group values on the LR, RLS, and Trial One performance. As can be seen, the mean values for LR for the 123 participants in the current study ranged from 0.41 to 0.51, which equates to the sample, on average, learning 40% to 51% of the available information after Trial One for these memory measures. Additionally, statistically significant differences in LR learning slope performances were observed between Normal Cognition, MCI, and AD diagnostic groups, Wilk’s Lambda = .332, F(12, 226) = 13.84, p < .001, η2 = .42. Specifically, group differences existed across all LR metrics (all ps < .001, η2s = .39 to .57). Scores were lower on all LR metrics for both the AD and MCI groups relative to the Normal Cognition group (all ps < .001 for each group). Similarly, LR performance was significantly worse for the AD group relative to MCI group on BVMT-R LR, Aggregated HVLT-R/BVMT-R LR, RBANS List Learning LR, and Aggregated RBANS LR metrics (all ps < .006), but not for HVLT-R LR and RBANS Story Memory LR (p = .14 and p = .02, respectively).
Table 2.
Variable | Total Sample | Normal Cognition | MCI | AD |
---|---|---|---|---|
n | 123 | 52 | 37 | 34 |
HVLT-R LR | 0.44 (0.3; −0.12 – 1.00) | 0.67 (0.3; 0.00 – 1.00) | 0.32 (0.2; 0.00 – 0.75) | 0.21 (0.2; −0.12 – 0.58) |
BVMT-R LR | 0.41 (0.3; −0.20 – 1.00) | 0.67 (0.2; 0.00 – 1.00) | 0.32 (0.2; −0.10 – 1.00) | 0.09 (0.2; −0.20 – 0.33) |
Aggregated HVLT-R/BVMT-R LR | 0.42 (0.3; −0.12 – 1.00) | 0.67 (0.2; 0.17 – 1.00) | 0.32 (0.2; 0.00 – 0.83) | 0.15 (0.1; −0.12 – 0.43) |
RBANS List Learning LR | 0.51 (0.3; −0.20 – 1.00) | 0.75 (0.2; 0.20 – 1.00) | 0.41 (0.3; −0.20 – 1.00) | 0.24 (0.2; 0.00 – 0.50) |
RBANS Story Memory LR | 0.49 (0.3; −0.20 – 1.00) | 0.75 (0.3; 0.00 – 1.00) | 0.39 (0.3; −0.20 – 1.00) | 0.23 (0.2; 0.00 – 0.67) |
Aggregated RBANS LR | 0.50 (0.3; −0.20 – 1.00) | 0.75 (0.2; 0.18 – 1.00) | 0.40 (0.2; −0.20 – 1.00) | 0.24 (0.1; 0.05 – 0.50) |
HVLT-R RLS | 2.7 (1.9; −1.00 – 8.00) | 3.4 (1.8; 0.00 – 8.00) | 2.5 (1.6; 0.00 – 6.00) | 2.0 (1.9; −1.00 – 7.00) |
BVMT-R RLS | 2.7 (2.1; −2.00 – 9.00) | 3.9 (1.9; 0.00 – 9.00) | 2.7 (1.7; −1.00 – 6.00) | 1.1 (1.6; −2.00 – 3.00) |
Aggregated HVLT-R/BVMT-R RLS | 5.5 (3.1; −2.00 – 13.00) | 7.3 (2.8; 2.00 – 13.00) | 5.1 (2.2; 0.00 – 9.00) | 3.1 (2.6; −2.00 – 9.00) |
RBANS List Learning RLS | 2.8 (1.6; −1.00 – 8.00) | 3.7 (1.3; 1.00 – 8.00) | 2.5 (1.5; −1.00 – 5.00) | 1.7 (1.1; 0.00 – 4.00) |
RBANS Story Memory RLS | 2.6 (1.8; −1.00 – 8.00) | 2.9 (1.9; 0.00 – 7.00) | 2.8 (1.8; −1.00 – 8.00) | 2.0 (1.6; 0.00 – 6.00) |
Aggregated RBANS RLS | 5.4 (2.6; −2.00 – 12.00) | 6.7 (2.4; 2.00 – 12.00) | 5.3 (2.6; −2.00 – 9.00) | 3.7 (2.0; 1.00 – 8.00) |
HVLT-R Trial One | 4.9 (2.6; 0.00 – 10.00) | 6.7 (1.7; 3.00 – 10.00) | 4.3 (2.3; 0.00 – 9.00) | 2.5 (1.7; 0.00 – 8.00) |
BVMT-R Trial One | 3.7 (2.8; 0.00 – 11.00) | 6.0 (2.3; 1.00 – 11.00) | 2.6 (2.1; 0.00 – 9.00) | 1.5 (1.2; 0.00 – 4.00) |
RBANS List Learning Trial One | 4.1 (1.4; 1.00 – 8.00) | 4.9 (1.2; 2.00 – 8.00) | 3.8 (1.3; 1.00 – 6.00) | 3.1 (1.0; 1.00 – 5.00) |
RBANS Story Memory Trial One | 5.7 (2.9; 0.00 – 12.00) | 7.9 (2.3; 4.00 – 12.00) | 4.8 (2.0; 1.00 – 9.00) | 3.2 (2.0; 0.00 – 7.00) |
Note: MCI = Mild Cognitive Impairment, AD = Alzheimer’s Disease, HVLT-R = Hopkins Verbal Learning Test – Revised, LR = Learning Ratio, BVMT-R = Brief Visuospatial Memory Test – Revised, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, RLS = Raw Learning Score. All values are Mean (Standard Deviation; range) unless listed otherwise.
Additionally, Table 3 reflects the correlational coefficients between learning slopes among HVLT-R, BVMT-R, RBANS List Learning, RBANS Story Memory, and associated Aggregates. As observed in the Table, correlational coefficients among LR values were strongly correlated with each other (all ps < .001), and universally stronger than those among RLS values according to Fisher r to z transformations (ps = .0001 to .007). Correlational coefficients among LR values were consistent with those among Trial One performances (Fisher r to z transformations ps > .05), but smaller than those among five of six Total Recall values (Fisher r to z transformations ps = .0001 to .001).
Table 3.
1 | 2 | 3 | 4 | 5 | 6 | |
---|---|---|---|---|---|---|
1. HVLT-R LR | - | |||||
2. BVMT-R LR | .59 | - | ||||
3. Aggregated HVLT/BVMT LR | .86 | .91 | - | |||
4. RBANS List Learning LR | .61 | .71 | .74 | - | ||
5. RBANS Story Memory LR | .58 | .61 | .67 | .66 | - | |
6. Aggregated RBANS LR | .65 | .73 | .77 | .90 | .90 | - |
1 | 2 | 3 | 4 | 5 | 6 | |
1. HVLT-R RLS | - | |||||
2. BVMT-R RLS | .21 | - | ||||
3. Aggregated HVLT/BVMT RLS | .75 | .81 | - | |||
4. RBANS List Learning RLS | .20 | .28 | .31 | - | ||
5. RBANS Story Memory RLS | .20 | .10 | .18 | .22 | - | |
6. Aggregated RBANS RLS | .25 | .23 | .31 | .74 | .82 | - |
1 | 2 | 3 | 4 | |||
1. HVLT-R Trial One | - | |||||
2. BVMT-R Trial One | .69 | - | ||||
3. RBANS List Learning Trial One | .59 | .50 | - | |||
4. RBANS Story Memory Trial One | .72 | .68 | .64 | - | ||
1 | 2 | 3 | 4 | |||
1. HVLT-R Total Recall | - | |||||
2. BVMT-R Total Recall | .84 | - | ||||
3. RBANS List Learning Total Score | .83 | .81 | - | |||
4. RBANS Story Memory Total Score | .83 | .82 | .84 | - |
Note: HVLT-R = Hopkins Verbal Learning Test – Revised, LR = Learning Ratio, BVMT-R = Brief Visuospatial Memory Test – Revised, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, and RLS = Raw Learning Slope. All LR, Trial One, and Total Recall/Score bivariate correlation coefficients were significant at p < .001. RLS bivariate correlation coefficients greater than .22 were significant at p < .01.
Demographic Analyses for Covariation
Bivariate correlation coefficients between LR values and age were significant, rs = −.25 to −.35, ps =.001 to .005, for five of the six comparisons, as were correlation coefficients between LR metrics and education, rs = .19 to 0.25, ps = .007 to .04. Four of six LR metrics possessed significant correlation coefficients with premorbid intellect, rs = .26 to 0.33, ps = .001 to .004. One-way ANOVA indicated that LR values were not significantly associated with sex (ps = .197 to .75), nor ethnicity (ps = .32 to .47). Additionally, hemispheric hippocampal volumes were significantly and positively correlated with each other, r = .90, r2 = .81, p < .001, therefore they were combined into a single “total hippocampal volume” variable by summing the left and right hemispheres together. Bivariate correlation coefficients between total hippocampal volume and age were significant, r = −.32, p = .001, as were coefficients between 18F-Flutemetamol SUVR and age, r = −.18, p < .05. Correlation coefficients between total hippocampal volume and education were also significant, r = .19, p < .05, as were coefficients between 18F-Flutemetamol SUVR and education, r = −.18, p < .05. One-way ANOVAs indicated that total hippocampal volumes and 18F-Flutemetamol composite SUVR values were not significantly associated with sex (ps = .06 to .79), nor ethnicity (ps = .38 to .78). Finally, one-way ANOVAs indicated that APOE ε4 status was not significantly associated with age (p = .49) nor education (p = .51), and Chi Square analyses indicated that APOE ε4 status was not associated with sex (p = .36) nor ethnicity (p = .37). As a result of these analyses, age, education, and premorbid intellect were used as covariates in the subsequent analyses.
AD Biomarker Analyses
When examining the relationships between learning slope and total hippocampal volume after controlling for age, education, and premorbid intellect, a statistically significant positive association was observed between all LR metrics and total hippocampal volume, rs = .37 to .46, r2 = .14 to .21, ps < .001. See Table 4. LR performances increased as a function of greater hippocampal volumes. Similarly, significant positive associations were observed between most RLS metrics and total hippocampal volume, rs = .21 to .33, r2 = .04 to .21, ps = .001 to .03. Also, significant positive associations were observed between Trial One performances for the associated memory measures and total hippocampal volume, rs = .25 to .35, r2 = .06 to .12, ps = .001 to .008, as well as between total learning score performances and total hippocampal volume, rs = .36 to .42, r2 = .13 to .18, ps < .001. Although correlation coefficients with hippocampal volumes tended to be larger for LR values relative to RLS values, none of these values were significant based on Fisher r to z transformations (ps = .04 to .22). Inspection of Table 4 also shows that correlation coefficients with hippocampal volume were generally comparable between LR values and Trial One performances (Fisher r to z transformations ps = .10 to .43) and total score performances (Fisher r to z transformations ps = .65 to .93).
Table 4.
Measure | Total Hippocampal Volume | 18F-Flutemetamol SUVR | ||||
---|---|---|---|---|---|---|
r | r2 | p value | r | r2 | p value | |
HVLT-R LR | .39 | .15 | <.001 | −.35 | .12 | <.001 |
BVMT-R LR | .37 | .14 | <.001 | −.48 | .23 | <.001 |
Aggregate HVLT-R/BVMT-R LR | .45 | .20 | <.001 | −.49 | .24 | <.001 |
RBANS List Learning LR | .40 | .16 | <.001 | −.44 | .19 | <.001 |
RBANS Story Memory LR | .41 | .17 | <.001 | −.50 | .25 | <.001 |
Aggregate RBANS LR | .46 | .21 | <.001 | −.53 | .28 | <.001 |
HVLT-R RLS | .27 | .07 | .006 | −.01 | .01 | .92 |
BVMT-R RLS | .24 | .06 | .01 | −.31 | .10 | .001 |
Aggregate HVLT-R/BVMT-R RLS | .33 | .11 | <.001 | −.22 | .05 | .03 |
RBANS List Learning RLS | .31 | .10 | .001 | −.27 | .07 | .005 |
RBANS Story Memory RLS | .21 | .04 | .06 | −.18 | .03 | .06 |
Aggregate RBANS RLS | .32 | .10 | .001 | −.28 | .08 | .003 |
HVLT-R Trial One | .32 | .10 | .001 | −.59 | .35 | <.001 |
BVMT-R Trial One | .35 | .12 | <.001 | −.47 | .22 | <.001 |
RBANS List Learning Trial One | .25 | .06 | .008 | −.44 | .19 | <.001 |
RBANS Story Memory Trial One | .28 | .08 | .004 | −.53 | .28 | <.001 |
HVLT-R Total Recall | .42 | .18 | <.001 | −.61 | .37 | <.001 |
BVMT-R Total Recall | .42 | .17 | <.001 | −.58 | .34 | <.001 |
RBANS List Learning Total Score | .38 | .15 | <.001 | −.57 | .32 | <.001 |
RBANS Story Memory Total Score | .36 | .13 | <.001 | −.62 | .38 | <.001 |
Note: HVLT-R = Hopkins Verbal Learning Test – Revised, LR = Learning Ratio, BVMT-R = Brief Visual Memory Test – Revised, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, and RLS = Raw Learning Score. Significance values reflect p value relationship between learning score and either total hippocampal volume or 18F-Flutemetamol composite SUVR uptake. Effect Sizes were measured using r2 values. A two-tailed alpha level was set at .01 for all analyses. The total hippocampal volume variable was additionally adjusted for total intracranial volume.
When examining the relationships between learning slope and cerebral β-amyloid deposition after controlling for age, education, and premorbid intellect, a statistically significant negative association was observed between all LR metrics and 18F-Flutemetamol composite SUVR uptake, rs = −.35 to −.53, r2 = .12 to .28, ps < .001 (see Table 4). LR performances decreased as a function of greater 18F-Flutemetamol uptake (i.e., greater β-amyloid burden). Similarly, significant negative associations were observed between some RLS metrics and 18F-Flutemetamol composite SUVR, including for BVMT-R RLS, r = −.31, r2 = .10, p = .001, RBANS List Learning RLS, r = −.27, r2 = .07, p = .005, and Aggregated RBANS RLS, r = −.28, r2 = .08, p = .003. Further, significant positive associations were observed between all Trial One performances for the associated memory measures and 18F-Flutemetamol uptake, rs = −.44 to −.59, r2 = .19 to .35, ps <.001, as well as between total learning score performances and 18F-Flutemetamol uptake, rs = −.57 to −.62, r2 = .32 to .38, ps < .001. Correlation coefficients with 18F-Flutemetamol composite SUVR tended to be larger for LR values relative to RLS values, with Fisher r to z transformations being significant for HVLT-R, Aggregated HVLT-R/BVMT-R, RBANS Story Memory, and Aggregated RBANS (ps = .002 to .009). Inspection of Table 4 also shows that correlation coefficients with 18F-Flutemetamol composite SUVR uptake were generally comparable between LR values and both Trial One performances (Fisher r to z transformations ps = .28 to .50) and total score performances (Fisher r to z transformations ps = .17 to .28) with the exception of HVLT-R, which was of stronger magnitude for both Trial One and total learning performances than LR (ps = .008).
When comparing LR learning slope performances between individuals based on APOE ε4 status, statistically significant differences were observed between groups, Wilk’s Lambda = .763, F(6, 111) = 5.74, p < .001, η2 = .24, after controlling for age, education, and premorbid intellect. As seen in Table 5, the APOE ε4 positive group performed significantly worse on all LR metrics than the APOE ε4 negative group (all ps < .002). Conversely, when comparing RLS learning slope performances between individuals based on APOE ε4 status, no statistically significant difference was observed between APOE ε4 positive and APOE ε4 negative groups, Wilk’s Lambda = .930, F(4, 113) = 2.13, p = .08, η2 = .07, after controlling for age, education, and premorbid intellect. Additionally, when comparing Trial One performances between individuals based on APOE ε4 status, statistically significant differences were observed between groups, Wilk’s Lambda = .775, F(4, 113) = 8.19, p < .001, η2 = .23, after controlling for age, education, and premorbid intellect. Specifically, the APOE ε4 positive group performed significantly worse on all Trial One performances than the APOE ε4 negative group (all ps < .003). Finally, when comparing total score performances between individuals based on APOE ε4 status, statistically significant differences were observed between groups, Wilk’s Lambda = .775, F(4, 113) = 8.21, p < .001, η2 = .23, after controlling for age, education, and premorbid intellect. Specifically, the APOE ε4 positive group performed significantly worse on all total learning performances than the APOE ε4 negative group (all ps < .001).
Table 5.
APOE ε4 status Mean (SD) |
F value | p value | η 2 | ||
---|---|---|---|---|---|
APOE ε4 Positive | APOE ε4 Negative | ||||
HVLT-R LR | 0.35 (0.2) | 0.55 (0.4) | 18.97 | <.001 | .14 |
BVMT-R LR | 0.30 (0.3) | 0.51 (0.3) | 19.80 | <.001 | .15 |
Aggregate HVLT-R/BVMT-R LR | 0.33 (0.2) | 0.54 (0.3) | 28.15 | <.001 | .20 |
RBANS List Learning LR | 0.43 (0.3) | 0.61 (0.3) | 12.18 | .001 | .10 |
RBANS Story Memory LR | 0.41 (0.3) | 0.62 (0.3) | 17.30 | <.001 | .13 |
Aggregate RBANS LR | 0.41 (0.3) | 0.61 (0.3) | 21.07 | <.001 | .15 |
HVLT-R RLS | 2.59 (1.5) | 3.03 (2.1) | 2.83 | .10 | .02 |
BVMT-R RLS | 2.52 (2.3) | 3.05 (1.7) | 1.71 | .19 | .02 |
Aggregate HVLT-R/BVMT-R RLS | 5.11 (2.9) | 6.09 (3.0) | 3.95 | .05 | .03 |
RBANS List Learning RLS | 2.54 (1.6) | 3.10 (1.5) | 4.02 | .05 | .03 |
RBANS Story Memory RLS | 2.37 (1.7) | 2.93 (1.9) | 3.85 | .05 | .03 |
Aggregate RBANS RLS | 4.90 (2.7) | 6.03 (2.4) | 6.38 | .01 | .05 |
HVLT-R Trial One | 4.06 (2.4) | 5.76 (2.5) | 14.71 | <.001 | .11 |
BVMT-R Trial One | 2.57 (2.2) | 5.05 (2.8) | 31.98 | <.001 | .22 |
RBANS List Learning Trial One | 3.73 (1.3) | 4.48 (1.4) | 10.27 | .002 | .08 |
RBANS Story Memory Trial One | 4.87 (2.8) | 6.6 (2.8) | 12.40 | .001 | .10 |
HVLT-R Total Recall | 16.63 (7.3) | 22.57 (7.9) | 22.35 | <.001 | .16 |
BVMT-R Total Recall | 12.06 (7.9) | 20.18 (9.0) | 32.61 | <.001 | .22 |
RBANS List Learning Total Score | 20.75 (6.3) | 25.71 (6.6) | 23.11 | <.001 | .17 |
RBANS Story Memory Total Score | 12.00 (5.6) | 16.14 (5.0) | 21.50 | <.001 | .16 |
Note: APOE ε4 = Apolipoprotein ε4 allele, η2 = partial eta squared, HVLT-R = Hopkins Verbal Learning Test – Revised, LR = Learning Ratio, BVMT-R = Brief Visual Memory Test – Revised, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, and RLS = Raw Learning Score. Significance values reflect p value of learning score differences between patients with positive or negative APOE ε4 status. Effect Sizes were measured using η2 values. A two-tailed alpha level was set at .01 for all analyses.
DISCUSSION
The current study was undertaken to examine the relationships between LR – a novel method to assess learning slope that controls for initial trial learning – and AD biomarkers in a sample of 123 well-characterized older adults along the AD continuum. Our results indicated that LR performance was consistently associated with AD biomarkers. Lower performances on each of the LR metrics in the current study were significantly associated with smaller total hippocampal volumes (or greater hippocampal atrophy). These results are consistent with our past research (Hammers et al., In Press) observing a relationship between the LR score calculated from BVMT-R and total hippocampal volume (r = .37, r2 = .14 currently, versus r = .35, r2 = .12 for Hammers et al.). However, our current correlations for HVLT-R LR with total hippocampal volume appeared to be stronger than Hammers et al.’s HVLT-R LR findings (r = .39, r2 = .15 for the current study, compared to r = .12, r2 = .01 in Hammers et al.), with a similar difference between studies for the Aggregated HVLT-R/BVMT-R LR score correlation. The magnitude of relationships seen in the current study may have been stronger than Hammers et al.’s because of the larger, better characterized, and “cleaner” sample of patients along the AD continuum currently used, whereas Hammers et al. incorporated a convenience sample of patients seen in a memory disorders clinic with a variety of etiologies. Additionally, Hammers et al. used a conventional MPRAGE whereas the current study utilized an MP2RAGE structural MRI, which has been shown to demonstrate better gray/white matter contrasts in deep gray matter regions (Okubo et al., 2016).
These results are additionally consistent with other research examining the relationship of learning slopes and hippocampal volumes in the literature – although none assessed LR, specifically. For example, Gifford and colleagues (Gifford et al., 2015) observed that a variety of learning slope metrics were associated with hippocampal volumes (and also ventrolateral prefrontal cortex and parahippocampal thickness) in a sub-sample of MCI participants, though no relationships were observed within normal cognition or AD sub-samples (or the combined sample). Similarly, Bonner-Jackson and colleagues (Bonner-Jackson et al., 2015) found that BVMT-R learning slope possessed a strong positive relationship with hippocampal volumes – more so than the relationship between HVLT-R learning slope and hippocampal volume. It is possible that use of LR in these studies could have enhanced their respective findings, given the magnitude of our observed effects, and that LR may illuminate future learning-biomarker research. Additionally, the relationship between reduced episodic memory performance and smaller hippocampal volumes has repeatedly been observed in the literature, including for cognitively intact individuals (Choi et al., 2016; Pohlack et al., 2014) and those with cognitive impairment associated with MCI (Choi et al., 2016; Jack et al., 2000) or probable AD (Choi et al., 2016; Jack et al., 1998; Mizuno et al., 2000; Mungas et al., 2002). These results, consequently, appear to further extend this established research to this novel learning slope calculation, and serve as the first study to establish criterion validity of the RBANS List Learning and Story Memory LR metrics with AD hippocampal volumes.
Additionally, lower scores on each of the LR metrics in the current study were significantly associated with greater cerebral β-amyloid deposition using the radio ligand 18F-Flutemetamol composite SUVR. Although limited research exists regarding learning slopes and β-amyloid deposition in the literature, this current result is not necessarily surprising given β-amyloid’s role in AD pathology (Beach et al., 2016; Wu et al., 2012). Also, research has consistently suggested associations between immediate and delayed memory performance on the RBANS and 18F-Flutemetamol uptake. For example, Hammers and colleagues (Hammers et al., 2017) observed that worse performances on RBANS indexes of Immediate Memory and Delayed Memory were strongly related to increased 18F-Flutemetamol composite uptake in a small sample of participants with MCI, and Duff and colleagues (Duff et al., 2013) observed similar findings for RBANS Delayed Memory (but not Immediate Memory). Additionally, Duff and colleagues (Duff et al., 2017) have observed associations between increased 18F-Flutemetamol composite uptake and decreased performances on both HVLT-R and BVMT-R Total Recall and Delayed Recall. The current results observed between amyloid accumulation and learning slope are especially relevant because amyloid deposition is proposed to occur early in the development of AD pathology (Jack et al., 2013; Jack et al., 2010), further suggesting that poor LR performance may be sensitive to early AD pathology. This is especially relevant since the high cost of amyloid-PET imaging and constraints related to its appropriate use (Johnson et al., 2013) have led to it being either cost prohibitive or restricted in many clinical settings.
Further, our results indicated that participants who were APOE ε4 positive performed worse than those who were APOE ε4 negative for all LR metrics in the study. Specifically, the APOE ε4 negative participants tended to gain between 51 – 62% of the available information after Trial One for these memory measures (LR scores of 0.51 to 0.62), whereas the APOE ε4 positive participants gained 30 – 41% (LR scores of 0.30 to 0.52). Research examining learning slopes and APOE ε4 status appears to be in its relative infancy, as these findings reflect the first research to show criterion validity of this LR measure with APOE ε4 status, and the only other literature found to examine learning slopes and APOE ε4 was a null result (Wehling et al., 2007). When considering relationships between APOE ε4 status and cognitive functioning more globally, the results tend to be somewhat ambiguous. For example, a number of studies (Caselli, 2014; Jorm et al., 2007; Small et al., 2000) failed to find a clear relationship between APOE ε4 status and cognition, and in the Gifford et al. study referenced above, they observed no changes to the relationship between learning slope and hippocampal volumes when APOE ε4 status was removed from the model (Gifford et al., 2015). Conversely, APOE ε4 has been shown to be significantly related to associative recognition in older adults with MCI (Troyer et al., 2012), and with more rapid cognitive decline on a cognitive composite in an AD sample (Cosentino et al., 2008). Further, meta-analytic findings suggest that non-carriers performed better than ε4 carriers in the domains of global cognitive functioning, episodic memory, and executive functioning (Small et al., 2004). Together, our results coincide with the findings for hippocampal volumes and cerebral β-amyloid deposition, and suggest that the LR metric is sensitive to AD pathology.
It is of note that the magnitude of these relationships observed between learning slope and each of the AD biomarkers appeared to be generally stronger for the LR metric than the RLS metric across tasks, and that the magnitudes were generally consistent with those observed for both Trial One learning and total score learning. These results are consistent with previous work supporting the use of LR over RLS. When comparing learning slope metrics with traditional immediate memory measures, both Hammers et al. (In Press) and Spencer et al. (2020) observed stronger relationships for LR than RLS. Spencer et al. also observed greater effects for LR than for Trial One performance, as was observed in our study. Similarly, counter to their aforementioned significant findings with LR and total hippocampal volume, Hammers et al. did not observe a significant relationship between RLS and hippocampal volume. These differential findings between LR and RLS may be because RLS does not consider the competition between Trial One and subsequent learning trials for calculation of the learning slope. Incorporating “information left to learn” after Trial One of a learning task reduces this competition, and appears to lead to stronger associations with both AD biomarkers and traditional measures of memory. Conversely, the lack of differential findings between LR and either Trial One or total learning score performance may suggest that LR provides little incremental utility beyond a respective measure’s Trial One or immediate recall score when predicting biomarker status. However, this overlooks other key roles that the LR metric could play in evaluating learning. Specifically, use of LR metrics allows for a more accurate and nuanced conceptualization of trial-by-trial learning than either a total learning score or the currently-used RLS calculations, which in turn may permit more personalized treatment recommendations for individual patients.
Given our strong associations between AD biomarkers and this novel learning slope metric, it is therefore not surprising that participants in our sample with AD or MCI performed significantly worse than participants with Normal Cognition (see Table 2). Additionally, participants with AD performed worse than participants with MCI for four of the six LR metrics calculated, and non-significant trends were observed for the other two LR metrics (RBANS Story Memory LR, p = .02; HVLT-R LR, p = .14). This is consistent with previous research suggesting that the BVMT-R is more adept at discriminating impairment in the AD continuum than the HVLT-R (Bonner-Jackson et al., 2015; Duff et al., 2018). Overall, when collapsing across measures, the Normal Cognition group tended to learn around 70% of available information after Trial One, the MCI group tended to learn around 35%, and the AD group tended to learn around 20%. These results are consistent with Spencer and colleagues’ (2020) observations that LR calculations from the RBANS discriminated veterans diagnosed with major/minor neurocognitive disorder from those without. These findings are also consistent with the preponderance of research suggesting that early and predominant anterograde episodic/semantic memory deficits are a hallmark cognitive marker of AD due to entorhinal cortex and hippocampal/medial temporal lobe degeneration (Braak & Braak, 1991; Hyman et al., 1990; Van Hoesen et al., 1991). When combined with the current results for total hippocampal volume, cerebral β-amyloid deposition, and APOE ε4 status, these results support the notion that LR performance is influenced by AD pathology.
Finally, our findings suggest that there was no appreciable difference in the magnitude of correlations between LR metrics and AD biomarkers as a function of the number of learning trials present in the measure. Specifically, correlations between RBANS List Learning or RBANS Story Memory LR performances were generally comparable for both total hippocampal volume (rs = .40 and .41 for RBANS List Learning LR and RBANS Story Memory LR, respectively) and 18F-Flutemetamol composite SUVR (rs = −.44 and −.50 for RBANS List Learning LR and RBANS Story Memory LR, respectively), despite the former having a format of 10 words presented over 4 trials and the latter having a format of 12 details presented over 2 trials. Similarly, HVLT-R and BVMT-R both assessed learning of 12 items over 3 trials, yet the correlations between LR and total hippocampal volume were rs = .37 and .39 for total hippocampal volume, respectively, and rs = −.35 to −.48 for 18F-Flutemetamol composite SUVR, respectively. Table 5 also indicated that the proportion of information learned in subsequent trials relative to APOE ε4 status was generally comparable across measures (e.g., 30 – 43% for APOE ε4 positive participants and 50 – 62% for APOE ε4 negative participants). By using an external benchmark like AD pathology and independence between LR and group classification, these findings appear to provide evidence that trial length and trial number have limited impact on the LR metric.
Limitations regarding the current study should be highlighted. First, this sample was relatively homogenous in regard to premorbid intellect, ethnicity, and education, therefore it is unclear how the results from this sample would generalize to a sample more reflective of the greater population. Future research should consider generalizing these results to samples that are not mostly Caucasian, well-educated, and with high premorbid intellect. However, the current study’s proportion of highly educated females appear to reflect long-standing trends in research participation. Specifically, it has been observed that women tend to volunteer more than men across all age groups (United States Bureau of the Census, 2015), which reaches a difference of upwards of 30% (U.S. Bureau of Labor Statistics, 2016). Individuals with higher education and Caucasians also consistently volunteer at greater levels (United States Bureau of the Census, 2015). Second, these results are specific to the LR metric calculated by Spencer and colleague’s methods and applied to the HVLT-R, BVMT-R, RBANS List Learning, and RBANS Story Memory, therefore future research is needed to understand if these results will generalize to other learning slope calculations or particular memory measures. Third, an LR value of 1.00 (100% learned) was assigned in the rare scenario when participants learned all available items on Trial One of any measure. This was consistent with Spencer et al., and necessary because a “perfect” score on Trial One would result in a value of zero in the denominator (Total Points Available For a Single Trial – Trial One), which represents an undefined mathematical expression. Additionally, as the calculation of LR does not incorporate intrusion errors, perseveration errors, and paraphasic substitutions from these learning tasks, it will be important to consider these meaningful clinical errors separately from the LR metric. Fourth, our sample was not uniform across diagnostic groups, which may have influenced the associations observed in the current study. Relatedly, severely impaired participants were excluded from our sample (e.g., very low MMSE values or high CDR scores), which may have impacted the current findings and restricts generalizability with severe AD samples. Further, the focus on this manuscript was on participants with presentations primarily related to memory impairment (e.g., AD and amnestic MCI), therefore results may differ for samples characterized by other MCI subtypes. Fifth, as previous research has suggested that ADNI-2 criteria may be prone to elevated false-positive rates of MCI diagnosis relative to other diagnostic criteria (like the actuarial Jak/Bondi criteria for MCI; Bondi et al., 2014; Edmonds et al., 2015), future research on LR should consider other methods of diagnosis when considering MCI samples. However, as the primary biomarker analyses in the current study came from a pooled sample of diagnoses along the AD spectrum, the use of ADNI-2 criteria was not felt to adversely affect the current primary results. Sixth, cognitive assessment and AD biomarker status was obtained within a short time-frame of each other, therefore additional research is necessary to understand if LR has any potential to predict future biomarker changes (e.g., hippocampal atrophy or cerebral β-amyloid deposition). Seventh, the possibility exists that other cognitive domains like executive functioning (Miyake et al., 2000) or mood-related issues (Eysenck et al., 2007) may moderate learning slope capacity, therefore these factors should be incorporated into future work on learning slope. This may be particularly true because participants with elevated depression on the GDS were excluded from the parent study. Finally, while these results appear to shed light on this LR metric in response to AD pathology, these results are still generally experimental in nature and are not directly applicable in clinical settings. To permit greater use clinically, future research is needed to develop normative data on these LR learning slopes in cognitively intact or “clean” individuals (Goodwill et al., 2019; Harrington et al., 2017) for commonly used memory measures.
Despite these limitations, the current study provides support for Spencer et al.’s (2020) LR learning slope metric across a variety of AD biomarkers. This LR score tended to outperform the traditional learning slope calculation – the RLS – in all analyses, and were generally equivalent across measures using different trial length and trial number. Together, these results provide evidence that this LR score is sensitive to AD pathology along the AD continuum – more so than the traditional raw learning score – and that reducing the competition between Trial One and subsequent trials can better inform learning capacity.
Acknowledgments
Funding:
The project described was supported by research grants from the National Institutes on Aging: 5R01AG045163. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.
References
- ADNI2. (2020). Alzheimer’s Disease Neuroimaging Initiative: ADNI2 Procedures Manual. Retrieved May 21 from https://adni.loni.usc.edu/wp-content/uploads/2008/07/adni2-procedures-manual.pdf
- Beach TG, Thal DR, Zanette M, Smith A, & Buckley C (2016, Mar 31). Detection of Striatal Amyloid Plaques with [18F]flutemetamol: Validation with Postmortem Histopathology. J Alzheimers Dis, 52(3), 863–873. 10.3233/JAD-150732 [DOI] [PubMed] [Google Scholar]
- Bender AR, Brandmaier AM, Duzel S, Keresztes A, Pasternak O, Lindenberger U, & Kuhn S (2020, Apr 14). Hippocampal Subfields and Limbic White Matter Jointly Predict Learning Rate in Older Adults. Cereb Cortex, 30(4), 2465–2477. 10.1093/cercor/bhz252 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Benedict R (1997). Brief Visuospatial Memory Test-Revised. Psychological Assessment Resources, Inc. [Google Scholar]
- Bondi MW, Edmonds EC, Jak AJ, Clark LR, Delano-Wood L, McDonald CR, Nation DA, Libon DJ, Au R, Galasko D, & Salmon DP (2014). Neuropsychological criteria for mild cognitive impairment improves diagnostic precision, biomarker associations, and progression rates. J Alzheimers Dis, 42(1), 275–289. 10.3233/JAD-140276 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bonner-Jackson A, Mahmoud S, Miller J, & Banks SJ (2015, Oct 15). Verbal and non-verbal memory and hippocampal volumes in a memory clinic population. Alzheimers Res Ther, 7(1), 61. 10.1186/s13195-015-0147-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Braak H, & Braak E (1991). Neuropathological stageing of Alzheimer-related changes. Acta Neuropathol, 82(4), 239–259. http://www.ncbi.nlm.nih.gov/pubmed/1759558 [DOI] [PubMed] [Google Scholar]
- Brandt J, & Benedict R (1997). Hopkins Verbal Learning Test-Revised. Psychological Assessment Resources, Inc. [Google Scholar]
- Caselli RJ (2014, Oct). Does an Alzheimer’s disease susceptibility gene influence the cognitive effects of cancer therapy? Pediatr Blood Cancer, 61(10), 1739–1742. 10.1002/pbc.24768 [DOI] [PubMed] [Google Scholar]
- Charlson ME, Pompei P, Ales KL, & MacKenzie CR (1987). A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis, 40(5), 373–383. 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
- Choi MH, Kim HS, Gim SY, Kim WR, Mun KR, Tack GR, Lee B, Choi YC, Kim HJ, Hong SH, Lim DW, & Chung SC (2016, May 4). Differences in cognitive ability and hippocampal volume between Alzheimer’s disease, amnestic mild cognitive impairment, and healthy control groups, and their correlation. Neurosci Lett, 620, 115–120. 10.1016/j.neulet.2016.03.044 [DOI] [PubMed] [Google Scholar]
- Cosentino S, Scarmeas N, Helzner E, Glymour MM, Brandt J, Albert M, Blacker D, & Stern Y (2008, May 6). APOE epsilon 4 allele predicts faster cognitive decline in mild Alzheimer disease. Neurology, 70(19 Pt 2), 1842–1849. 10.1212/01.wnl.0000304038.37421.cc [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duff K, Anderson JS, Mallik AK, Suhrie KR, Atkinson TJ, Dalley BCA, Morimoto SS, & Hoffman JM (2018, Nov). Short-term repeat cognitive testing and its relationship to hippocampal volumes in older adults. J Clin Neurosci, 57, 121–125. 10.1016/j.jocn.2018.08.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duff K, Foster NL, Dennett K, Hammers DB, Zollinger LV, Christian PE, Butterfield RI, Beardmore BE, Wang AY, Morton KA, & Hoffman JM (2013, Nov). Amyloid deposition and cognition in older adults: the effects of premorbid intellect. Arch Clin Neuropsychol, 28(7), 665–671. 10.1093/arclin/act047 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duff K, Hammers DB, Dalley BCA, Suhrie KR, Atkinson TJ, Rasmussen KM, Horn KP, Beardmore BE, Burrell LD, Foster NL, & Hoffman JM (2017). Short-Term Practice Effects and Amyloid Deposition: Providing Information Above and Beyond Baseline Cognition. J Prev Alzheimers Dis, 4(2), 87–92. 10.14283/jpad.2017.9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Edmonds EC, Delano-Wood L, Clark LR, Jak AJ, Nation DA, McDonald CR, Libon DJ, Au R, Galasko D, Salmon DP, Bondi MW, & Alzheimer’s Disease Neuroimaging I. (2015, Apr). Susceptibility of the conventional criteria for mild cognitive impairment to false-positive diagnostic errors. Alzheimers Dement, 11(4), 415–424. 10.1016/j.jalz.2014.03.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eysenck MW, Derakshan N, Santos R, & Calvo MG (2007, May). Anxiety and cognitive performance: attentional control theory [Review]. Emotion, 7(2), 336–353. 10.1037/1528-3542.7.2.336 [DOI] [PubMed] [Google Scholar]
- Fischl B, & Dale AM (2000, Sep 26). Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc Natl Acad Sci U S A, 97(20), 11050–11055. 10.1073/pnas.200033797 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, van der Kouwe A, Killiany R, Kennedy D, Klaveness S, Montillo A, Makris N, Rosen B, & Dale AM (2002, Jan 31). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355. 10.1016/s0896-6273(02)00569-x [DOI] [PubMed] [Google Scholar]
- Fischl B, van der Kouwe A, Destrieux C, Halgren E, Segonne F, Salat DH, Busa E, Seidman LJ, Goldstein J, Kennedy D, Caviness V, Makris N, Rosen B, & Dale AM (2004, Jan). Automatically parcellating the human cerebral cortex. Cereb Cortex, 14(1), 11–22. 10.1093/cercor/bhg087 [DOI] [PubMed] [Google Scholar]
- Folstein MF, Folstein SE, & McHugh PR (1975, Nov). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res, 12(3), 189–198. 10.1016/0022-3956(75)90026-6 [DOI] [PubMed] [Google Scholar]
- Gifford KA, Phillips JS, Samuels LR, Lane EM, Bell SP, Liu D, Hohman TJ, Romano RR 3rd, Fritzsche LR, Lu Z, Jefferson AL, & Alzheimer’s Disease Neuroimaging I. (2015, Jul). Associations between Verbal Learning Slope and Neuroimaging Markers across the Cognitive Aging Spectrum. J Int Neuropsychol Soc, 21(6), 455–467. 10.1017/S1355617715000430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goodwill AM, Campbell S, Henderson VW, Gorelik A, Dennerstein L, McClung M, & Szoeke C (2019, May). Robust norms for neuropsychological tests of verbal episodic memory in Australian women. Neuropsychology, 33(4), 581–595. 10.1037/neu0000522 [DOI] [PubMed] [Google Scholar]
- Hammers DB, Atkinson TJ, Dalley BC, Suhrie KR, Beardmore BE, Burrell LD, Horn KP, Rasmussen KM, Foster NL, Duff K, & Hoffman JM (2017, Apr). Relationship between (18)F-Flutemetamol uptake and RBANS performance in non-demented community-dwelling older adults. Clin Neuropsychol, 31(3), 531–543. 10.1080/13854046.2016.1278039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hammers DB, Gradwohl BD, Kucera A, Abildskov T, Wilde EA, & Spencer RJ (In Press). Preliminary validation of a measure for learning slope for the HVLT-R and BVMT-R in older adults. Cognitive and Behavioral Neurology. [DOI] [PubMed]
- Harrington KD, Lim YY, Ames D, Hassenstab J, Rainey-Smith S, Robertson J, Salvado O, Masters CL, Maruff P, & Group AR (2017, Mar 1). Using Robust Normative Data to Investigate the Neuropsychology of Cognitive Aging. Arch Clin Neuropsychol, 32(2), 142–154. 10.1093/arclin/acw106 [DOI] [PubMed] [Google Scholar]
- Hyman BT, Van Hoesen GW, & Damasio AR (1990, Nov). Memory-related neural systems in Alzheimer’s disease: an anatomic study. Neurology, 40(11), 1721–1730. http://www.ncbi.nlm.nih.gov/pubmed/2234428 [DOI] [PubMed] [Google Scholar]
- Jack CR Jr., Knopman DS, Jagust WJ, Petersen RC, Weiner MW, Aisen PS, Shaw LM, Vemuri P, Wiste HJ, Weigand SD, Lesnick TG, Pankratz VS, Donohue MC, & Trojanowski JQ (2013, Feb). Tracking pathophysiological processes in Alzheimer’s disease: an updated hypothetical model of dynamic biomarkers. Lancet Neurol, 12(2), 207–216. 10.1016/S1474-4422(12)70291-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jack CR Jr., Knopman DS, Jagust WJ, Shaw LM, Aisen PS, Weiner MW, Petersen RC, & Trojanowski JQ (2010, Jan). Hypothetical model of dynamic biomarkers of the Alzheimer’s pathological cascade. Lancet Neurol, 9(1), 119–128. 10.1016/S1474-4422(09)70299-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jack CR Jr., Petersen RC, Xu Y, O’Brien PC, Smith GE, Ivnik RJ, Boeve BF, Tangalos EG, & Kokmen E (2000, Aug 22). Rates of hippocampal atrophy correlate with change in clinical status in aging and AD. Neurology, 55(4), 484–489. 10.1212/wnl.55.4.484 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jack CR Jr., Petersen RC, Xu Y, O’Brien PC, Smith GE, Ivnik RJ, Tangalos EG, & Kokmen E (1998, Oct). Rate of medial temporal lobe atrophy in typical aging and Alzheimer’s disease. Neurology, 51(4), 993–999. 10.1212/wnl.51.4.993 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson KA, Minoshima S, Bohnen NI, Donohoe KJ, Foster NL, Herscovitch P, Karlawish JH, Rowe CC, Carrillo MC, Hartley DM, Hedrick S, Pappas V, & Thies WH (2013, Mar). Appropriate use criteria for amyloid PET: a report of the Amyloid Imaging Task Force, the Society of Nuclear Medicine and Molecular Imaging, and the Alzheimer’s Association. J Nucl Med, 54(3), 476–490. 10.2967/jnumed.113.120618 [DOI] [PubMed] [Google Scholar]
- Jones RN, Rosenberg AL, Morris JN, Allaire JC, McCoy KJ, Marsiske M, Kleinman KP, Rebok GW, & Malloy PF (2005, Jul-Sep). A growth curve model of learning acquisition among cognitively normal older adults. Exp Aging Res, 31(3), 291–312. 10.1080/03610730590948195 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jorm AF, Mather KA, Butterworth P, Anstey KJ, Christensen H, & Easteal S (2007, Jan). APOE genotype and cognitive functioning in a large age-stratified population sample. Neuropsychology, 21(1), 1–8. 10.1037/0894-4105.21.1.1 [DOI] [PubMed] [Google Scholar]
- Lundqvist R, Lilja J, Thomas BA, Lotjonen J, Villemagne VL, Rowe CC, & Thurfjell L (2013, Aug). Implementation and validation of an adaptive template registration method for 18F-flutemetamol imaging data. J Nucl Med, 54(8), 1472–1478. 10.2967/jnumed.112.115006 [DOI] [PubMed] [Google Scholar]
- Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, & Wager TD (2000, Aug). The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: a latent variable analysis. Cogn Psychol, 41(1), 49–100. 10.1006/cogp.1999.0734 [DOI] [PubMed] [Google Scholar]
- Mizuno K, Wakai M, Takeda A, & Sobue G (2000, Feb 1). Medial temporal atrophy and memory impairment in early stage of Alzheimer’s disease: an MRI volumetric and memory assessment study. J Neurol Sci, 173(1), 18–24. 10.1016/s0022-510x(99)00289-0 [DOI] [PubMed] [Google Scholar]
- Morris JC (1993, Nov). The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology, 43(11), 2412–2414. 10.1212/wnl.43.11.2412-a [DOI] [PubMed] [Google Scholar]
- Mungas D, Reed BR, Jagust WJ, DeCarli C, Mack WJ, Kramer JH, Weiner MW, Schuff N, & Chui HC (2002, Sep 24). Volumetric MRI predicts rate of cognitive decline related to AD and cerebrovascular disease. Neurology, 59(6), 867–873. 10.1212/wnl.59.6.867 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Okubo G, Okada T, Yamamoto A, Kanagaki M, Fushimi Y, Okada T, Murata K, & Togashi K (2016, Jan). MP2RAGE for deep gray matter measurement of the brain: A comparative study with MPRAGE. J Magn Reson Imaging, 43(1), 55–62. 10.1002/jmri.24960 [DOI] [PubMed] [Google Scholar]
- Pohlack ST, Meyer P, Cacciaglia R, Liebscher C, Ridder S, & Flor H (2014, Jan). Bigger is better! Hippocampal volume and declarative memory performance in healthy young men. Brain Struct Funct, 219(1), 255–267. 10.1007/s00429-012-0497-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Randolph C (2012). Repeatable Battery for the Assessment of Neuropsychological Status. The Psychological Corporation. [Google Scholar]
- Rosen WG, Terry RD, Fuld PA, Katzman R, & Peck A (1980, May). Pathological verification of ischemic score in differentiation of dementias. Ann Neurol, 7(5), 486–488. 10.1002/ana.410070516 [DOI] [PubMed] [Google Scholar]
- Small BJ, Graves AB, McEvoy CL, Crawford FC, Mullan M, & Mortimer JA (2000, Jun 13). Is APOE--epsilon4 a risk factor for cognitive impairment in normal aging? Neurology, 54(11), 2082–2088. 10.1212/wnl.54.11.2082 [DOI] [PubMed] [Google Scholar]
- Small BJ, Rosnick CB, Fratiglioni L, & Backman L (2004, Dec). Apolipoprotein E and cognitive performance: a meta-analysis. Psychol Aging, 19(4), 592–600. 10.1037/0882-7974.19.4.592 [DOI] [PubMed] [Google Scholar]
- Spencer RJ, Gradwohl BD, Williams TF, Kordovski VM, & Hammers DB (2020, Jul 11). Developing learning slope scores for the repeatable battery for the assessment of neuropsychological status. Appl Neuropsychol Adult, 1–7. 10.1080/23279095.2020.1791870 [DOI] [PubMed] [Google Scholar]
- Sunderland JJ, & Christian PE (2015, Jan). Quantitative PET/CT scanner performance characterization based upon the society of nuclear medicine and molecular imaging clinical trials network oncology clinical simulator phantom. J Nucl Med, 56(1), 145–152. 10.2967/jnumed.114.148056 [DOI] [PubMed] [Google Scholar]
- Thurfjell L, Lilja J, Lundqvist R, Buckley C, Smith A, Vandenberghe R, & Sherwin P (2014, Oct). Automated quantification of 18F-flutemetamol PET activity for categorizing scans as negative or positive for brain amyloid: concordance with visual image reads. J Nucl Med, 55(10), 1623–1628. 10.2967/jnumed.114.142109 [DOI] [PubMed] [Google Scholar]
- Troyer AK, Murphy KJ, Anderson ND, Craik FI, Moscovitch M, Maione A, & Gao F (2012, Dec). Associative recognition in mild cognitive impairment: relationship to hippocampal volume and apolipoprotein E. Neuropsychologia, 50(14), 3721–3728. 10.1016/j.neuropsychologia.2012.10.018 [DOI] [PubMed] [Google Scholar]
- U.S. Bureau of Labor Statistics. (2016). Volunteering in the United States, 2015 https://www.bls.gov/news.release/volun.nr0.htm
- United States Bureau of the Census. (2015). Current Population Survey, September 2014: Volunteer Supplement Inter-university Consortium for Political and Social Research. 10.3886/ICPSR36154.v1 [DOI]
- Van Hoesen GW, Hyman BT, & Damasio AR (1991, Jan). Entorhinal cortex pathology in Alzheimer’s disease. Hippocampus, 1(1), 1–8. 10.1002/hipo.450010102 [DOI] [PubMed] [Google Scholar]
- Vandenberghe R, Van Laere K, Ivanoiu A, Salmon E, Bastin C, Triau E, Hasselbalch S, Law I, Andersen A, Korner A, Minthon L, Garraux G, Nelissen N, Bormans G, Buckley C, Owenius R, Thurfjell L, Farrar G, & Brooks DJ (2010, Sep). 18F-flutemetamol amyloid imaging in Alzheimer disease and mild cognitive impairment: a phase 2 trial. Ann Neurol, 68(3), 319–329. 10.1002/ana.22068 [DOI] [PubMed] [Google Scholar]
- Von Korff M, Wagner EH, & Saunders K (1992, Feb). A chronic disease score from automated pharmacy data. J Clin Epidemiol, 45(2), 197–203. 10.1016/0895-4356(92)90016-g [DOI] [PubMed] [Google Scholar]
- Wechsler D (1987). Manual for the Weschler Memory Scale - Revised. The Psychological Corporation. [Google Scholar]
- Wehling E, Lundervold AJ, Standnes B, Gjerstad L, & Reinvang I (2007, Oct 31). APOE status and its association to learning and memory performance in middle aged and older Norwegians seeking assessment for memory deficits. Behav Brain Funct, 3, 57. 10.1186/1744-9081-3-57 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilkinson GS, & Robertson GJ (2006). WRAT 4: Wide Range Achievement Test: Professional Manual. Psychological Assessment Resources, Incorportated [Google Scholar]
- Wu L, Rowley J, Mohades S, Leuzy A, Dauar MT, Shin M, Fonov V, Jia J, Gauthier S, & Rosa-Neto P (2012). Dissociation between brain amyloid deposition and metabolism in early mild cognitive impairment. PLoS One, 7(10), e47905. 10.1371/journal.pone.0047905 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey M, & Leirer VO (1982). Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res, 17(1), 37–49. http://www.ncbi.nlm.nih.gov/pubmed/7183759 [DOI] [PubMed] [Google Scholar]
- Yester M, Al-Senan R, & White S (2014). NEMA testing of GE Discovery 710 PET scanner compared to a simplified protocol for routine testing of PET scanners. Journal of Nuclear Medicine, 55, 2157. [Google Scholar]