Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Feb 1.
Published in final edited form as: J Int Neuropsychol Soc. 2022 Feb 28;29(2):230–234. doi: 10.1017/S135561772200008X

NIH Toolbox® Episodic Memory Measure Differentiates Older Adults with Exceptional Memory Capacity from those with Average-for-Age Cognition

Tatiana Karpouzian-Rogers 1,2,*, Beth Makowski-Woidan 2, Alan Kuang 3, Hui Zhang 3, Angela Fought 4, Janessa Engelmeyer 2, M-Marsel Mesulam 1,5, Sandra Weintraub 1,2, Emily Rogalski 1,2
PMCID: PMC9439674  NIHMSID: NIHMS1831240  PMID: 35225214

Abstract

Objective:

Older adults with exceptional memory function, designated “SuperAgers,” include individuals over age 80, with episodic memory at least as good as individuals ages 50s-60s. The Northwestern University SuperAging cohort is defined by performance on an established test of verbal memory. The purpose of this study was to determine if superior verbal memory extends to nonverbal memory in SuperAgers by examining differences in the NIH Toolbox® (NIHTB) between older adults with exceptional memory and those with average-for-age cognition.

Method:

SuperAgers (n=46) and cognitively average-for-age older adults (n=31) completed a comprehensive neuropsychological battery and the NIHTB Cognition module. Multiple linear regressions were used to examine differences on subtests between groups.

Results:

There was a significant effect of group on the Picture Sequence Memory score, (p=0.007), such that SuperAgers had higher scores than cognitively average-for-age older adults. There were no other group effects across other non-episodic memory NIHTB Cognition measures.

Conclusions:

Findings from this study demonstrated stronger performance on the memory measure of the NIHTB in SuperAgers compared to cognitively average-for-age older adults demonstrating superior memory in not only verbal but also nonverbal episodic memory in this group. Additionally, this study adds to the literature validating the NIHTB in older adults, particularly in a novel population of adults over age 80 with exceptional memory.

Keywords: SuperAgers, normal aging, NIH Toolbox, memory, cognition, dementia

Introduction

Decline in memory functions is often accepted as part of “normal” aging, with mild changes beginning in mid-life and more accelerated changes occurring over the age of 60 (Nyberg et al., 2012). However, at the Northwestern Mesulam Center for Cognitive Neurology and Alzheimer’s Disease, we have identified a group of individuals that we designated “SuperAgers,” who are over age 80 and able to maintain superior memory performance compared to their same age peers and at a level that is at least “average” for 50- and 60-year-olds (Rogalski et al., 2013). Longitudinal follow-up of these individuals suggests that superior memory performance can be maintained over time, providing additional support for their resistance to the typical age-related decline (Gefen et al, 2014; Rogalski et al., 2018). With respect to psychological factors, SuperAgers report greater levels of social relationships compared to cognitively average-for-age peers (Cook Maher et al., 2017). Neuroimaging studies have demonstrated greater cortical integrity and slowed rates of atrophy compared to cognitively average age-matched peers and thicker anterior cingulate cortex compared to 50- to 65-year-olds (Harrison et al., 2012; Cook et al., 2017). Further, post-mortem studies suggest a lower frequency of Alzheimer neuropathology and higher density of von Economo neurons in the anterior cingulate compared to cognitively average older adults and individuals with amnestic mild cognitive impairment (Gefen et al., 2015).

The operationalization of memory capacity in SuperAgers was defined on the basis of scores on the Rey Auditory Verbal Learning Test (RAVLT), a difficult 15-item list-learning test of verbal episodic memory, which is widely used and has good psychometric properties. However, performance on tests of other episodic memory measures, including memory tests that place less emphasis on verbal abilities, has not been systematically investigated in this population. A recent research tool that was designed to measure cognitive functions in adults is the NIH Toolbox® for Assessment of Neurological and Behavioral Function (NIHTB; Gershon et al., 2013; Weintraub et al., 2014). Traditionally, the evaluation of cognitive abilities in older adults have included either brief cognitive screening measures or lengthy neuropsychological batteries that often require clinical expertise and these batteries frequently differ across studies, making direct comparisons difficult to conduct. The NIHTB is a computerized suite of tests that measure cognitive, emotional, motor and sensory domains in individuals aged 3–85, and was designed to be used across a variety of settings, particularly in longitudinal research studies so that findings across studies could be conveniently compared. The Cognition Module includes a test of episodic memory, the Picture Sequence Memory test, that relies less heavily on verbal abilities, requiring participants to recall sequences of pictured actors and actions in the order they were originally learned over several trials.

The present study examined differences across all subtests of the Cognition module of the NIHTB between older adults with exceptional memory and those who are cognitively average-for-age. Our particular focus was to determine if the NIHTB episodic memory test specifically, which is less reliant on verbal abilities in comparison to our gold-standard of memory capacity, would be sensitive to differences between SuperAgers and “normal” agers. This is important as it would extend the contexts in which SuperAgers display superior episodic memory performance and opens the possibility of using the NIHTB as an efficient tool for future identification of SuperAgers. This study also the first characterization of NIHTB Cognition module in an established cohort of older adults with exceptional memory. Given that SuperAgers display superior memory capacity, we hypothesized that the SuperAger group would demonstrate greater performance on the episodic memory test of the NIHTB.

Methods

Participants

Participants 80 years or older were recruited through the Mesulam Center and Northwestern’s Alzheimer’s Disease Research Center (ADRC) Clinical Core, community lectures, and/or word of mouth. SuperAgers were referred on the basis of high memory scores and the absence of impairment in any other cognitive domain but were not necessarily superior in non-memory domains. Inclusion criteria for SuperAgers included: 1) score at or above the average level for 50 to 65-year-olds (equivalent to the Superior range for their own age) on the delayed recall condition of the Rey Auditory Verbal Learning Test (RAVLT; Schmidt, 1996), a 15-word list-learning memory test; and 2) performance within one standard deviation of the average range for their age on nonmemory measures including the Trail Making Test Part-B, Category Fluency Test, and 30-item Boston Naming Test according to published normative data (Heaton, 2004, Randolph et al., 1998, Mack et al., 1992). Inclusion criteria for Cognitively Average-for-Age Older Adults included: performance within the average-for-age normative range on the RAVLT and on all non-memory tests administered in the study. Full scale IQ was measured using the Wechsler Adult Intelligence Scale, Third Edition (WAIS-III). Additional inclusion criteria for both groups were that all participants maintained their cognitive status (as measured by neuropsychological battery described above) from their visit to the time the NIH Toolbox® was administered to maintain the integrity of our SuperAger sample. The administration of the NIHTB and collection of the neuropsychological battery occurred no more than three months apart. Additionally, all participants were required to have preserved activities of daily living. Participants with significant neurologic or psychiatric illnesses were excluded. All participants provided written informed consent. The Institutional Review Board at Northwestern University approved all study procedures. Research was completed in accordance with the Helsinki Declaration.

Study Measures

As described in previous studies (Gefen et al., 2014), all participants underwent a neuropsychological battery, including measures of attention, executive functions, language, and episodic memory. Participants completed the Cognition module of the NIH Toolbox® as part of the biyearly standardized battery. The Cognition Battery consists of tests assessing Executive Function and Attention (Dimensional Change Card Sort Test and Flanker Inhibitory Control and Attention Test), Episodic Memory (Picture Sequence Memory Test), Language (Oral Reading Recognition Test and Picture Vocabulary Test), Processing Speed (Pattern Comparison Processing Speed Test), and Working Memory (List Sorting Working Memory Test) (Weintraub et al., 2014). In addition to individual test scores, Cognitive Function, Fluid Cognition, and Crystallized Cognition composite scores are computed. In the Picture Sequence Memory Test, participants are shown a series of pictures depicting a sequence of events, e.g., playing in the park. Then, the pictures are assembled in the center of the screen and participants are asked to reproduce the spatial placement of the previously demonstrated sequence of pictures. For additional details on these modules, refer to the original publications (Weintraub et al., 2013; Gershon et al., 2010; Gershon et al., 2013) and the NIHTB website (nihtoolbox.org).

Statistical Analyses

Differences in participant demographics were assessed using two-sample t-tests. NIH Toolbox® scores were summarized using frequencies and percentages for categorical variables or mean and standard deviation for continuous variables. Histograms and scattered plots of each NIH Toolbox® measure were examined to explore the shape of distributions and identify potential outliers. For the Cognition Module, computed scores were calculated for Flanker, Dimensional Change Card Sort, Pattern Comparison, and Picture Sequence Memory subtests, theta scores were calculated for Reading and Vocabulary, and raw scores were used for List Sorting. For details regarding computation of theta and computed scores, refer to NIHTB Scoring and Interpretation Guide (http://www.healthmeasures.net/images/nihtoolbox/Training-Admin-Scoring_Manuals/NIH_Toolbox_Scoring_and_Interpretation_Manual_9-27-12.pdf). Composite scores were calculated by averaging the normalized scores of each measure, and then deriving scale scores based on this new distribution. For each NIH Toolbox® measure, multiple linear regressions were used to examine differences between groups. Covariate adjustments included WAIS-FSIQ, sex, age, and education. Linear regression model fit was assessed using measures of collinearity and non-linearity, including residuals versus fits plots, histograms, Q-Q plots of residuals, and Dfbeta statistics. Adjusted R2 values were used to summarize variability explained in the linear models. All analyses were conducted in R 3.5.3 software.

Results

Groups did not differ with respect to age, race, years of education, or sex (ps > .05). SuperAgers had a higher WAIS-FSIQ than cognitively average-for-age older adults (p < 0.001). Across all linear models, r-squared values ranged from 0.03–0.34. Colinearity was not a concern, with pairwise correlations ranging from 0 to 0.56 for model covariates. Multicollinearity was not a concern, with variance inflation factor ranging from 1.03 to 1.79. Dfbeta statistics indicated there are some observations of influences. In order to improve linearity for measures that were positively skewed, scores were log-transformed, which included scores on the Picture Sequence Memory test.

Within the Cognition Module, there was an effect of group on Picture Sequence Memory scores (F (1,63) = 7.7, p = .007, β = 0.10), such that the SuperAger group had higher scores than the Cognitively Average-for-Age Older Adults. The effects of sex, age, education level, and FSIQ were not significant in this model. There were no other effects of group across all other NIHTB Cognition tests of non-memory domains or on Composite Scores (all ps > 0.05).

Discussion

This study sought to extend the superiority of episodic memory compared with other cognitive domains in the Northwestern University SuperAging cohort by comparing SuperAgers with cognitively average-for-age older adults on the tests of the NIHTB Cognition Battery. Findings of this study demonstrated greater performance on a test reliant on nonverbal episodic memory, the Picture Sequence Memory Test, in the SuperAging group compared to cognitively average-for-age older adults. Performance across all other measures of cognition on the NIHTB were comparable between groups. These findings confirm the exceptional episodic memory in SuperAgers.

The criteria for inclusion in the Northwestern University SuperAging research program involve completion of neuropsychological measures examining multiple aspects of cognition, with particular emphasis on memory abilities. SuperAging status requires performance at or above the average level of 50 to 65-year-olds on the RAVLT, a well-established measure of verbal episodic memory shown to be sensitive to early changes in memory and structural brain changes in Alzheimer’s disease (e.g., Estévez‐González et al., 2003). Longitudinal studies of SuperAgers have demonstrated that superior memory performance tends to be stable, suggesting that exceptional memory capacity is not necessarily a function of superior premorbid cognitive abilities, but rather a resistance to age-related cognitive changes (Gefen et al., 2014; Rogalski et al., 2018). The finding that performance on the NIHTB Picture Sequence Memory, a measure of episodic memory that places less emphasis on verbal memory abilities, also differentiates the SuperAging group and cognitively average-for-age older adults is further confirmation of the memory superiority in SuperAgers. This suggests that superior memory capacity in SuperAgers may not be specific to the list learning of the RAVLT, but is more general to episodic memory. Additionally, scores on all other NIHTB measures were similar between groups, which mirrors our criteria that SuperAgers may score in at least the average range on all other measures of cognition, including object naming, semantic fluency, and executive attention. Although it is unclear why there were no differences at the group level across other NIHTB subtests, it is possible there may be nuanced profiles at the individual level; this is similar to what was observed in recent work from the SuperAging Research Program, which demonstrated significant intragroup variability on multiple cognitive domains (Maher et al., 2021). Additional explanations include differences in the specific domains assessed by our neuropsychological measures versus the NIHTB (I.e. verbal fluency), as well as differences in tests used in this study compared to the neuropsychological tests used to validate the NIHTB (Weintraub et al., 2013).

The NIHTB has been validated in older adults without cognitive impairment and has also been investigated in older adults with varying degrees of cognitive impairment. In particular, one study examined the psychometric properties of the NIHTB cognition module in cognitively intact older adults and found acceptable test-retest reliability over a one-year period, as well as a relationship between NIHTB Fluid Composite and cerebral volumes, and a strong correlation between Fluid and Crystallized Composites with their respective gold standard composites (Scott et al., 2019). In a study of older adults with subjective decline, mild cognitive impairment, or mild dementia, performance on the NIHTB Cognition module was consistent with performance on traditional neuropsychological tests and had greater discriminative ability when supplemented with RAVLT delayed recall performance (Hackett et al., 2018). Further, neuroimaging studies have demonstrated relationships between NIHTB performance and hippocampal volume and tau deposition in older adults (O’Shea et al. 2016; Snitz et al., 2020). One important limitation is that the sample was a predominantly white, well-educated group, and therefore replication with a more diverse sample is needed. The present study is one of the first, to our knowledge, to examine performance in the NIHTB in an established cohort of adults over age 80 with exceptional memory and adds to the utility of using the NIHTB to measure cognitive functioning in the oldest of old age groups.

Figure 1.

Figure 1.

Open in a new tab

SuperAgers perform significantly better than cognitively average-for-age 80+ year-olds on the NIH Toolbox® Picture Sequence Memory.

**p < .01

Table 1.

Study Sample Characteristics and NIHTB Subtest Scores

SuperAgers (n=46) Cognitively Average-for-Age 80+ year-olds (n=31) Test Statistic (t or χ2)
Demographics and Estimated IQ
Age, Mean (SD) 84.2 (3.3) 84.0 (3.9) 0.3
Sex, M:F 10:21 16:30 0.1
Race, CA:AA 43:3 24:7 0.3
Years of Education, Mean (SD) 17.2 (2.1) 16.5 (2.0) 1.3
WAIS-FSIQ, Mean (SD) 133.0 (12.2) 118.7 (15.2) 4.5***
Performance on Standardized Neuropsychological Measures used for Study Inclusion
RAVLT Delay Raw, Mean (SD) 11.0 (1.8) 5.2 (1.4) 1.3***
BNT-30 Raw, Mean (SD)a 28.2 (1.6) 26.3 (2.8) 3.5**
Animal Fluency Raw, Mean (SD)b 22.7 (4.4) 19.8 (5.3) 2.6**
Trails B Raw (sec), Mean (SD) 78.9 (28.3) 104.0 (42.9) 2.9**
NIH Toolbox Performance
Oral Reading Recognition (Language), Mean (SD) 7.4 (1.3) 6.2 (2.1) 0.03
Picture Vocabulary (Language), Mean (SD) 7.8 (1.7) 6.9 (2.0) 0.1
Flanker Inhibitory Control (Executive Attention), Mean (SD) 7.4 (0.6) 7.2 (0.9) 0.5
Dimensional Card Sort, (Executive Switching), Mean (SD) 7.5 (0.7) 7.4 (0.8) 0.5
Pattern Comparison (Processing Speed), Mean (SD) 35.5 (9.4) 33.5 (8.0) 0.0
List Sorting (Working Memory), Mean (SD) 15.5 (2.4) 14.6 (3.1) 0.1
Picture Sequence Memory (Episodic Memory), Mean (SD) 441.2 (65.8) 388.0 (58.9) 7.7**
Open in a new tab
*

p<.05

**

p<.01

***

p<.001

SD = Standard Deviation; M = Male; F = Female; CA = Caucasian; AA = African American.

*

Note two participants (1 control, 1 SuperAger) had low BNT scores at the time of NITB but met qualifying criteria at their initial visit. Four participants (3 Controls, 1 SuperAger) had consistently low BNT scores, for one English was a second language and may have contributed to lower scores. These participants were retained in all analysis as there were no other objective or subjective reports of difficulty with language.

Acknowledgments

The authors have no conflicts of interest to disclose.

Research reported in this publication was supported, in part, by the following National Institutes of Health (NIH): the National Institute on Aging [NIA; award numbers R01AG045571, R56AG045571, R01AG067781, E. Rogalski; P30 AG13854, R. Vassar), the Institutional National Research Service Award (T32 AG020506, R. Vassar, trainee: T. Karpouzian-Rogers), and the National Center for Advancing Translational Science (NCATS; award number U54 NS092089, R. D’Aquila). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

References

  1. Cook Maher A, Kielb S, Loyer E, Connelley M, Rademaker A, Mesulam MM, … & Rogalski E (2017). Psychological well-being in elderly adults with extraordinary episodic memory. PloS one, 12(10), e0186413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Cook AH, Sridhar J, Ohm D, Rademaker A, Mesulam MM, Weintraub S, & Rogalski E (2017). Rates of cortical atrophy in adults 80 years and older with superior vs average episodic memory. Jama, 317(13), 1373–1375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Estévez‐González A, Kulisevsky J, Boltes A, Otermín P, & García‐Sánchez C (2003). Rey verbal learning test is a useful tool for differential diagnosis in the preclinical phase of Alzheimer’s disease: comparison with mild cognitive impairment and normal aging. International journal of geriatric psychiatry, 18(11), 1021–1028. [DOI] [PubMed] [Google Scholar]
  4. Gefen T, Peterson M, Papastefan ST, Martersteck A, Whitney K, Rademaker A, … & Geula C (2015). Morphometric and histologic substrates of cingulate integrity in elders with exceptional memory capacity. Journal of Neuroscience, 35(4), 1781–1791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Gefen T, Shaw E, Whitney K, Martersteck A, Stratton J, Rademaker A, … & Rogalski E (2014). Longitudinal neuropsychological performance of cognitive SuperAgers. Journal of the American Geriatrics Society, 62(8), 1598–1600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Gershon RC, Cella D, Fox NA, Havlik RJ, Hendrie HC, & Wagster MV (2010). Assessment of neurological and behavioural function: the NIH Toolbox. The Lancet Neurology. [DOI] [PubMed] [Google Scholar]
  7. Gershon RC, Wagster MV, Hendrie HC, Fox NA, Cook KF, & Nowinski CJ (2013). NIH toolbox for assessment of neurological and behavioral function. Neurology, 80(11 Supplement 3), S2–S6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Hackett K, Krikorian R, Giovannetti T, Melendez-Cabrero J, Rahman A, Caesar EE, … & Isaacson RS (2018). Utility of the NIH Toolbox for assessment of prodromal Alzheimer’s disease and dementia. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 10, 764–772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Harrison TM, Weintraub S, Mesulam MM, & Rogalski E (2012). Superior memory and higher cortical volumes in unusually successful cognitive aging. Journal of the International Neuropsychological Society: JINS, 18(6), 1081. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Heaton RK (2004). Revised comprehensive norms for an expanded Halstead-Reitan Battery: Demographically adjusted neuropsychological norms for African American and Caucasian adults, professional manual. Psychological Assessment Resources. [Google Scholar]
  11. Mack WJ, Freed DM, Williams BW, & Henderson VW (1992). Boston Naming Test: shortened versions for use in Alzheimer’s disease. Journal of gerontology, 47(3), P154–P158. [DOI] [PubMed] [Google Scholar]
  12. Maher AC, Makowski-Woidan B, Kuang A, Zhang H, Weintraub S, Mesulam MM, & Rogalski E (2021). Neuropsychological Profiles of Older Adults with Superior versus Average Episodic Memory: The Northwestern “SuperAger” Cohort. Journal of the International Neuropsychological Society, 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Nyberg L, Lövdén M, Riklund K, Lindenberger U, & Bäckman L (2012). Memory aging and brain maintenance. Trends in cognitive sciences, 16(5), 292–305. [DOI] [PubMed] [Google Scholar]
  14. O’Shea A, Cohen R, Porges EC, Nissim NR, & Woods AJ (2016). Cognitive aging and the hippocampus in older adults. Frontiers in aging neuroscience, 8, 298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Randolph C (1998). Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). San Antonio, TX: Psychological Corporation. [Google Scholar]
  16. Rogalski E, Gefen T, Mao Q, Connelly M, Weintraub S, Geula C, … & Mesulam MM (2019). Cognitive trajectories and spectrum of neuropathology in Super Agers: The first 10 cases. Hippocampus, 29(5), 458–467. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Rogalski EJ, Gefen T, Shi J, Samimi M, Bigio E, Weintraub S, … & Mesulam MM (2013). Youthful memory capacity in old brains: anatomic and genetic clues from the Northwestern SuperAging Project. Journal of cognitive neuroscience, 25(1), 29–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Schmidt M (1996). Rey auditory verbal learning test: A handbook (p. 1996). Los Angeles, CA: Western Psychological Services. [Google Scholar]
  19. Scott EP, Sorrell A, & Benitez A (2019). Psychometric Properties of the NIH Toolbox Cognition Battery in Healthy Older Adults: Reliability, Validity, and Agreement with Standard Neuropsychological Tests. Journal of the International Neuropsychological Society, 25(8), 857–867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Snitz BE, Tudorascu DL, Yu Z, Campbell E, Lopresti BJ, Laymon CM, … & Cohen AD (2020). Associations between NIH Toolbox Cognition Battery and in vivo brain amyloid and tau pathology in non‐demented older adults. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 12(1), e12018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Weintraub S, Dikmen SS, Heaton RK, Tulsky DS, Zelazo PD, Bauer PJ, … & Fox NA (2013). Cognition assessment using the NIH Toolbox. Neurology, 80(11 Supplement 3), S54–S64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Weintraub S, Dikmen SS, Heaton RK, Tulsky DS, Zelazo PD, Slotkin J, … & Gershon R (2014). The cognition battery of the NIH toolbox for assessment of neurological and behavioral function: validation in an adult sample. Journal of the International Neuropsychological Society, 20(6), 567–578. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES