Cognition assessment using the NIH Toolbox

Abstract

Cognition is 1 of 4 domains measured by the NIH Toolbox for the Assessment of Neurological and Behavioral Function (NIH-TB), and complements modules testing motor function, sensation, and emotion. On the basis of expert panels, the cognition subdomains identified as most important for health, success in school and work, and independence in daily functioning were Executive Function, Episodic Memory, Language, Processing Speed, Working Memory, and Attention. Seven measures were designed to tap constructs within these subdomains. The instruments were validated in English, in a sample of 476 participants ranging in age from 3 to 85 years, with representation from both sexes, 3 racial/ethnic categories, and 3 levels of education. This report describes the development of the Cognition Battery and presents results on test-retest reliability, age effects on performance, and convergent and discriminant construct validity. The NIH-TB Cognition Battery is intended to serve as a brief, convenient set of measures to supplement other outcome measures in epidemiologic and longitudinal research and clinical trials. With a computerized format and national standardization, this battery will provide a “common currency” among researchers for comparisons across a wide range of studies and populations.

Cognition is 1 of the 4 domains of behavioral and neurologic health assessed in the NIH Toolbox for the Assessment of Neurological and Behavioral Function (NIH-TB). All domain measures were intended to be freely accessible, to be usable with individuals from 3 to 85 years of age, with each domain battery not to exceed 30 minutes in duration. Expert surveys were conducted and panels of research scientists and clinicians consulted in an iterative manner to rank cognitive subdomains in order of their perceived importance for health. Information was requested from experts (N = 102) who reported sufficient familiarity with cognition to make recommendations for specific subdomains of importance. The 2 top-ranked subdomains were Executive Function (EF) (95%) and Episodic Memory (93%), followed by Language (55%), Processing Speed (52%), and Attention (50%). Many (57%) also listed a “Global Score” as important. Other cognitive subdomains were excluded because of lower priority in the rankings, coupled with the stringent time constraints on the length of the battery.

The rationale for specific cognitive constructs within subdomains and instrument selection was based on a systematic review of the literature, including evidence of the known biological associations of each. The EF subdomain was deemed to include several distinct constructs, including Switching/Set Shifting, Inhibitory Control and Attention, and Working Memory. Because of the heavy weighting of EF by respondents, these 3 constructs were considered separate subdomains, with single instruments addressing each. Cognitive subdomains and specific constructs selected for measurement follow.

EF, also called “cognitive control,” refers to the top-down cognitive modulation of goal-directed activity. Development of EF in childhood parallels the development of prefrontal, anterior cingulate, and parietal cortex, and the basal ganglia, as well as the growth of connections between these regions and others.¹ EF emerges in infancy² and grows rapidly between the ages of 2 and 5 years³ with more gradual changes continuing into adolescence and early adulthood. EF is very vulnerable to aging,⁴ and comparisons across the lifespan yield an inverted U-shaped pattern, with early age-related improvement followed by later age-related decline.⁵ Based on factor-analytic work, there is an emerging consensus that EF can be divided into 3 partially independent subcomponents: set shifting, inhibitory control, and updating/working memory.⁶ These distinctions are clearest in middle childhood and beyond, and far less distinct in children younger than age 6.⁷ There is also evidence that prefrontal activation during the performance of EF tasks becomes increasingly focal and differentiated in the course of development.⁸

The set-shifting component of EF consists of the ability to shift responses based on rules or contingencies. It is measured in the Cognition Battery by a paradigm initially developed for children, the NIH-TB Dimensional Change Card Sort Test.⁹ This aspect of EF is supported by a distributed neuroanatomical network involving lateral prefrontal, anterior cingulate, and inferior parietal regions.

The ability to focus, sustain, and shift attention is a prerequisite for performing most conscious cognitive operations frequently tested experimentally or clinically.¹⁰ The developmental syndrome of attention deficit hyperactivity disorder has been associated with poor outcomes in academic achievement and adult life adaptation, including increased risk of accidents.¹¹ Visual spatial attention, critical at many developmental time points and important for safety in a variety of environments, is mediated by a well-studied, distributed large-scale neuroanatomical network composed of the frontal eye fields, the posterior parietal cortex, and the anterior cingulate area and their interconnections with one another and with subcortical structures in the thalamus and basal ganglia.^12,13 A measure of visuospatial inhibitory attention, the NIH-TB Flanker Inhibitory Control and Attention Test, was chosen for the Cognition Battery.

Working memory (WM) refers to a limited-capacity storage buffer that becomes overloaded when the amount of information exceeds that capacity. Conceptually, WM refers to the ability to 1) process information across a series of tasks and modalities, 2) hold the information in a short-term buffer, 3) manipulate the information, and 4) hold the products of that manipulation in the same short-term buffer. Cortical networks associated with spatial and nonspatial WM include prefrontal and posterior parietal regions.¹⁴ WM has been studied extensively across the lifespan.^15,16 Its integrity has been linked to scholastic development¹⁷ and letter knowledge,¹⁸ and its impairment to reading disabilities.¹⁹ WM improves significantly as children develop and WM span is thought to double in capacity between the ages of 5 and 10.²⁰ WM is relatively stable throughout adulthood. A reduction in performance in older adults may be attributable to a reduction in processing speed, rather than to changes in WM per se.²¹ The test chosen to measure this construct is the NIH-TB List Sorting Working Memory Test.

Memory is composed of different systems of information storage and retrieval. The memory construct selected for the NIH-TB was episodic memory, a system involved in storage of unique events or experiences encoded in a time-specific manner. Episodic memory is fragile, sensitive to decay and interference and to both “normal” aging and many brain diseases. Episodic memory provides the building blocks for cognitive growth during development and is the system we rely on to update reality. Its absence, as in the historic case of the patient H.M., results in an existence in which there is only the present.²² Episodic memory has a protracted course of development, with pronounced changes throughout the first 2 decades of life.^23,24 It is one of the first cognitive functions to show age-related decline and the most susceptible to developmental disorders,²⁵ brain trauma, and neurodegenerative diseases such as Alzheimer disease.²⁶ The large-scale neuroanatomical network that supports episodic memory in addition to the hippocampus includes the hypothalamus, thalamus, medial temporal regions, cingulate cortex, and prefrontal cortex.^22,27 The NIH-TB Picture Sequence Memory Test is the measure of episodic memory in the Cognition Battery.

Language is a system of conventional symbols for communication, linked to a large-scale neuroanatomical network in the left cerebral hemisphere.²⁸ Developmental disorders of language and communication (e.g., autism, dyslexia) and limited opportunities to acquire literacy have a significant impact on academic achievement and life adaptation. Language scores can predict occupational attainment and performance.²⁹ Many acquired conditions can impair language in adulthood, including aphasia due to stroke and neurodegenerative brain disease. After much deliberation considering the various language components that the NIH-TB could test, 2 measures were designed: a single-word oral reading test, the NIH-TB Oral Reading Recognition Test, and a single-word vocabulary comprehension test, the NIH-TB Picture Vocabulary Test.

Reading was selected because it is a proxy for a broad range of cognitive, educational, and socioeconomic factors. The ability to pronounce low-frequency words with irregular orthography has been used as an estimate of overall intelligence.³⁰ Single-word reading recognition tasks are strong predictors of health and cognitive outcomes across the lifespan, and performance on these tasks is also an estimate of the quality of education, accounting for some of the racial/ethnic differences on neuropsychological test performance seen in older adults.^31,32

Vocabulary represents the lexical component of language and is highly associated with general measures of “crystallized intelligence,” or “g_c,”³³ overall cognitive functioning, and success in school and work.^29,34 Single-word auditory comprehension is a fundamental language skill that children learn very early, even before they are able to speak. Infants may have a repertoire of as many as 50 words they can understand before age 1.³⁵ Syntactic proficiency is equally important for development,^36,37 but is more challenging to measure and to translate across different languages than single-word processing.

The final subdomain, Processing Speed (PS), is defined as either the amount of time it takes to process a set amount of information, or the amount of information that can be processed within a certain unit of time.³⁸ Simple PS tasks require a simple motor response to a target stimulus. Measures of complex PS, in contrast, require more concentration, as well as some mental manipulation.

The greatest growth in PS is observed relatively early and becomes more attenuated during childhood and adolescence.³⁹ Performance declines in young adulthood and steadily as people age.⁴⁰ PS measures are among the most sensitive indicators of cerebral dysfunction,⁴¹ and slowed PS has been demonstrated in traumatic brain injury, multiple sclerosis, Parkinson disease, symptomatic HIV, chronic fatigue syndrome, dementia, and schizophrenia.⁴² Slowed PS has been associated with changes in neurotransmitter activity (e.g., reduced cholinergic function, reduced numbers of D₂ dopamine receptors, and altered glutamate activity), white matter integrity, glucose metabolism, and nerve conduction velocities (e.g., as measured by evoked potentials, event-related potentials, and EEG).⁴² For the Cognition Battery, the NIH-TB Pattern Comparison Processing Speed Test was chosen to measure PS.

The data reported in this article are derived from the validation study of the Cognition Battery. Results are reported for test-retest reliability, the effects of age on performance, and convergent and discriminant construct validity. More extensive details of test design and administration and scoring are available for the pediatric portion of the sample (ages 3–15),⁴³ and similar details will be presented for the adult sample (ages 20–85) in future publications.

METHODS

Although the entire battery is computerized and includes automated scoring, it is necessary for an examiner to present task instructions, monitor compliance, and ensure valid results. For accessibility, all instructions are administered visually on the screen and also presented orally.

NIH-TB Cognition Battery tests.

NIH-TB Flanker Inhibitory Control and Attention Test (Executive/Attention).

This test is a version of the Eriksen flanker task derived from the Attention Network Test.⁴⁴ It tests the ability to inhibit visual attention to irrelevant task dimensions. On each trial, a central directional target (fish for children younger than 8, arrows for ages 8 and older) is flanked by similar stimuli on the left and right. The task is to indicate the direction of the central stimulus. On congruent trials, the flankers face the same direction as the target. On incongruent trials, they face the opposite direction. A scoring algorithm integrates accuracy, a suitable measure in early childhood, and reaction time, a more relevant measure of adult performance on this task, yielding scores from 0 to 10. There are 40 trials and the average time to complete the task is 4 minutes.

NIH-TB Dimensional Change Card Sort Test (Executive/Shifting).

The NIH-TB Dimensional Change Card Sort Test,⁹ originally designed for children, was adapted for adults to assess the set-shifting component of EF. A target visual stimulus must be matched to 1 of 2 choice stimuli according to shape or color. Participants younger than 8 years receive a block of trials in which only 1 dimension is relevant and then a second block (switch) in which the other dimension is critical. Those who succeed following the switch also receive a mixed block, in which color is relevant on the majority of trials with occasional, unpredictable shifts to shape. Participants 8 years and older receive only the mixed block. The relevant criterion word, “color” or “shape,” appears on the screen and for young children is also delivered orally via the computer. Scoring is similar to that for the flanker task, with an algorithm that weights accuracy for children and reaction time for adults. A total of 40 trials require 4 minutes.

NIH-TB List Sorting Working Memory Test (Working Memory).

This task is an adaptation of Mungas' List Sorting task from the Spanish and English Neuropsychological Assessment Scales.^45,46 A series of stimuli is presented on the computer screen visually (object) and orally (spoken name), 1 at a time. Participants are instructed to repeat the stimuli to the examiner in order of size, from smallest to largest. In 1 condition, all stimuli come from 1 category. In the second, stimuli are presented from 2 categories, following which the participant must report first all stimuli from 1 category, then from the other, in order of size within each. The number of items in each series increases from one trial to the next and the test is discontinued when 2 trials of the same length are failed. The prototype task has been previously validated in an elderly sample.^45,47 The List Sorting task takes approximately 7 minutes to administer. Test scores consist of total items correct across all trials.

NIH-TB Picture Sequence Memory Test (Episodic Memory).

The NIH-TB Picture Sequence Memory Test is a new measure derived from imitation-based tasks (elicited and deferred imitation) used in research with infants and young children.^48–50 The original stimuli were 3-dimensional props used to produce action sequences that the infant or child imitates. For the NIH-TB, the stimuli are pictured objects and activities, thematically related but with no inherent order. For each trial, pictures appear in the center of the computer screen and then are moved 1 at a time into a fixed spatial order, as an audio file simultaneously describes the content of each (e.g., “Plant the tomatoes”), until the entire sequence is displayed on the screen. Then the pictures return to the center of the screen in a random display and the participant must move them into the sequence demonstrated. The score is derived from the cumulative number of adjacent pairs of pictures remembered correctly over 3 learning trials. Based on pilot testing, level of task difficulty was adjusted for the various age groups. Thus, for ages 3 to 4 years, 6 pictures were administered; 5 to 6 years, 9 pictures; 8 years, 12 pictures; 9 to 60 years, 15 pictures; and 65 to 85 years, 9 pictures. Administration time is approximately 10 minutes.

NIH-TB Oral Reading Recognition Test (Language).

This test measures the ability to pronounce single printed words and/or to recognize letters. An English item bank, controlled for frequency of word use, complexity of letter-sound relationships, and orthographic typicality, was developed with an initial set of item response theory calibrations. Letters and other “prereading” items are included. Items are presented on the computer screen one by one and the participant is asked to read them aloud. Items are administered by computer adaptive testing (CAT) and participant responses are entered by the examiner. The CAT item bank in final form will contain approximately 250 items, although only 30 to 40 will be presented, depending on performance. Average administration time is 4 minutes.

NIH-TB Picture Vocabulary Test (Language).

Single words are presented via an audio file, paired simultaneously with 4 screen images of objects, actions, and/or depictions of concepts. The task is to pick the picture that matches the spoken word. The test is CAT administered, which reduces the amount of time to identify performance level. The test does not require speaking and can be performed by individuals who are preliterate and illiterate. Items were recalibrated and final parameter estimates were obtained after norming for optimal CAT administration. Total administration time is approximately 5 minutes.

NIH-TB Pattern Comparison Processing Speed Test (Processing Speed).

This test is modeled after Salthouse's Pattern Comparison Task,⁵¹ an extensively researched assessment of choice reaction time, easily adapted for computerized administration. Participants are asked to identify whether 2 visual patterns are the “same” (“Yes” button) or “not the same” (“No” button). Children younger than 8 years indicate these choices with a “smiley” or “frowny” face button. Type, complexity, and number of stimuli are varied to ensure adequate variability of performance across the age spectrum from 3 to 85 years. The NIH-TB Pattern Comparison Processing Speed Test requires 3 minutes to administer and the score is the number of correct items (of a possible 130) completed in 90 seconds.

Subjects.

The sample (N = 476) was based on a stratification plan to include adequate numbers of individuals within age bands, level of education, and racial/ethnic backgrounds. A marketing research firm assisted in recruitment of community-dwelling individuals. Testing was completed at Northwestern University (Chicago) and 5 additional sites: Emory University (Atlanta), the University of Minnesota (Minneapolis), the University of Washington (Seattle), NorthShore University HealthSystem (Evanston, IL), and Kessler Foundation Research Center (West Orange, NJ). Eligible participants were 3 to 85 years of age living in the community. See table 1 for age, sex, race, and education strata. Not all ages were sampled in this study. Education levels in the table are defined as actual years of education completed by adult participants (ages 20+ years) and highest parental education for children (ages 3–15 years). One-third of the sample was randomly selected to repeat testing 7 to 21 days later to assess test-retest reliability.

Table 1.

Initial validation sample demographics (N = 476)^a

Analyses.

This initial report includes results from analyses of test-retest reliability, associations of test scores with age, and convergent and discriminant construct validity. Age associations reflect the validity of the tests for measuring cognitive development during childhood and age-related cognitive decline during adulthood. Convergent and discriminant validity results provide evidence that the Cognition Battery is measuring the intended constructs.

Pearson correlation coefficients between age and test performance were calculated separately for children and adults. Intraclass correlation coefficients were calculated to evaluate test-retest reliability. Convergent validity was assessed by correlations between each NIH-TB measure and a well-established “gold standard” measure of the same construct; evidence of discriminant validity was assessed by correlations with gold standards of a different cognitive construct. Gold standard measures for each NIH-TB instrument are listed in table 2.

Table 2.

Convergent and discriminant validity (“gold standard”) measures used for ages 8 to 85 years

Gold standard measures were scarce for 3- to 6-year-olds, and we were unable to identify well-established measures to test convergent or discriminant validity in this age group for the measures of attention, episodic memory, EF, and PS. Thus, in this age group, only convergent validity was measured between the Cognition Battery measures and a measure of general cognitive ability (i.e., “g”) obtained by averaging z scores of the Wechsler Preschool and Primary Scale of Intelligence–3rd edition Block Design subtest⁵² and the Peabody Picture Vocabulary Test–4th edition.⁵³ More detailed psychometric information on individual measures and on challenges related to testing for construct validity in very young children is detailed in Zelazo et al. (in press).⁹

RESULTS

Test-retest reliability.

Test-retest reliability was strong for the entire sample and separately for children (ages 3–15 years) and adults (ages 20–85 years; table 3). Intraclass correlation coefficients for the entire sample on the NIH Toolbox measures ranged from 0.78 for the Picture Sequence Memory Test to 0.99 on the Oral Reading Recognition Test, with most other values falling above 0.90.

Table 3.

Test-retest reliability of instruments in the NIH-TB Cognition Battery

Age effects.

All cognitive abilities are expected to improve during childhood, and most are expected to show some age-related decline during adulthood, with the exception of language skills and other aspects of “crystallized” intelligence. Therefore, correlations between age and NIH-TB test performance were conducted separately for children (ages 3–15 years) and adults (ages 20–85 years). Table 4 presents these results.

Table 4.

Correlations between NIH-TB cognition subdomain scores and age

All NIH-TB Cognition Battery measures showed robust associations between test performance and age in the child group (r = 0.58–0.87), where scores improved with age. With the exception of the language measures (Vocabulary and Reading, r = 0.15 and −0.02, respectively), age and test scores (r = −0.46 to −0.65) on the remaining NIH-TB Cognition Battery measures were negatively associated, with lower scores at higher age levels. Thus, on the Picture Sequence Memory Test, performance improved during childhood and early adolescence, with gradual decline in scores across adult age ranges beginning in the 30s (figure, A). In contrast, the NIH-TB Picture Vocabulary Test showed gradual, linear improvement with age until the mid 50s and then stabilized, whereas Oral Reading Recognition showed a much sharper increase until early grade-school years (age 7–8) and then followed the same pattern of more gradual improvement and then stability in the older age groups (figure, B).

Figure. Episodic memory scores vs language scores across age.

(A) NIH Toolbox Picture Sequence Memory Test scores show improvement into early adulthood and then decline from the 50s on. Administration set size varied by age group as follows: ages 3 to 4 years, 6 pictures; 5 to 6 years, 9 pictures; 8 years, 12 pictures; 9 to 60 years, 15 pictures, and 65 to 85 years, 9 pictures. (B) Results for NIH Toolbox Reading and Vocabulary scores (reported as a “theta,” or individual ability score, based on item response theory analyses) show improvement sustained into adulthood. The data points in both A and B represent the mean score ± standard error.

Convergent and discriminant validity.

In children from 3 to 6 years of age, all NIH-TB Cognition Battery measures were significantly correlated (ranging from r = 0.54 to r = 0.74) with our measure of general cognitive ability (“g”), indicating that they are sensitive to a range of different cognitive ability levels within this age cohort. Table 5 shows results for convergent and discriminant validity for ages 8 to 85 years. For all NIH-TB CB instruments, correlations for convergent validity measures ranged from r = 0.48 to r = 0.93 (all p < 0.0001), suggesting that the NIH-TB measures are tapping the desired constructs. Correlations for discriminant validity measures ranged from r = 0.05 to r = 0.30, indicating lack of, or relatively weak, relationship with measures that tap different constructs.

Table 5.

Convergent and discriminant validity of NIH-TB cognitive function battery measures

DISCUSSION

This article introduces the NIH Toolbox Cognition Battery, a brief series of cognitive tests for the purpose of supplementing measures in epidemiologic and longitudinal studies to constitute a “common currency” among researchers. The Cognition Battery has 7 computerized instruments that measure 6 ability subdomains important for cognitive health from the ages of 3 to 85 years. Data are presented for 208 normal children (age 3–15 years) and 268 normal adults (age 20–85 years) on 3 important psychometric characteristics: test-retest reliability, sensitivity to cognitive growth during childhood and age-related decline during adulthood, and construct validity.

The subject sample for this study deliberately emphasized representation of ethnic minorities (almost 50%) and the oldest and youngest groups (3–6 years and 65–85 years; together, 48%) to ensure that the tests would perform as needed in these important segments of the population. Thus, the participants in this study (or their parents, in the case of children) tended to be rather highly educated, particularly in the youngest and oldest groups (see table 1). A more representative population-based sampling strategy was implemented for the NIH-TB norming study.

Adequate test-retest reliability was considered essential for the NIH-TB Cognition Battery, particularly because of its anticipated use in longitudinal studies. The results suggest that test-retest reliability of all NIH-TB measures is good to excellent across a large age range. Composite scores, which have higher reliability than individual test scores, are being developed to increase the potential for use of the battery in clinical trials and other longitudinal research.

Evidence of test validity can take many forms, and derives from both clinical and nonclinical subject samples. We presented the relationship of Cognition Battery performance with age in cognitively normal children and adults. The Reading and Vocabulary scores showed the expected associations with age, growing through adolescence and stabilizing in older adulthood. Not surprisingly, Reading showed an especially steep improvement from age 3 to the early school years, when both formal and informal educational experiences ideally promote such development. The Language subdomain tests, as expected, are experience-based and peaked somewhat later than measures of other cognitive subdomains, and then remained relatively stable even into the ninth decade of life. The nonlanguage subdomain tests in the battery, in contrast, conformed to the pattern expected with cognitive ability measures in that they peaked in early adulthood and then declined in later adulthood at different rates, depending on the measure.

Another validity measure we included in this report expresses how well the tests in the battery measure the intended constructs (convergent validity) as opposed to different cognitive constructs (discriminant validity). The “gold standard” tests related to the Cognition Battery instruments in the expected ways for participants across a wide age band (ages 8–85 years), demonstrating both convergent and discriminant validity. Evaluating construct validity in young children (ages 3–6 years) was challenging because of the absence of specific gold standard measures of targeted constructs appropriate for these ages. (See Zelazo et al., in press,⁵⁴ for discussion.) The lack of such measures may reflect the fact that different subdomains of cognition become more differentiated with experience and development.⁵⁵ The correlations between the NIH-TB measures and general cognitive ability, our index of convergent validity in young children (ages 3–6 years), were high (0.54–0.74), possibly supporting such a notion.

The NIH-TB Cognition Battery was designed as a brief, diverse, accessible, and psychometrically sound set of instruments that will be broadly applicable in research studies of normal and abnormal groups across a wide age range. The current results regarding age effects, test-retest reliability, and construct validity are promising. The next phase of development established normative standards for the NIH-TB measures using a large, demographically diverse sample, including a Spanish-language version of the measures. More detailed information will also be available about associations of test performances with various aspects of everyday functioning (e.g., school performance in the child sample) and relationships with additional demographic characteristics (educational level/socioeconomic status, sex, and ethnicity). The NIH-TB Cognition Battery was not developed as a clinical measure to either screen for cognitive impairment or to substitute for a full, competent neuropsychological evaluation. However, future studies with clinical populations are expected to generate another source of validation of the Cognition Battery as a sound set of measures of a broad range of normal and abnormal cognitive functioning, with implications for brain health in large-scale research studies.

GLOSSARY

CAT

computer adaptive testing

CB

Cognition Battery

EF

executive function

NIH-TB

NIH Toolbox for the Assessment of Neurological and Behavioral Function

PS

processing speed

WM

working memory

AUTHOR CONTRIBUTIONS

Sandra Weintraub: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data. Sureyya Dikmen: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, statistical analysis, study supervision. Robert Heaton: drafting/revising the manuscript, study concept or design, analysis or interpretation of data. David Tulsky: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data, statistical analysis, study supervision. Philip Zelazo: drafting/revising the manuscript, study concept or design, analysis or interpretation of data. Patricia Bauer, Noelle Carlozzi, and Jerry Slotkin: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data, study supervision. David Blitz: analysis or interpretation of data, statistical analysis, study supervision. Kathleen Wallner-Allen: drafting/revising the manuscript, study concept or design, analysis or interpretation of data. Nathan Fox: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, study supervision. Jennifer Beaumont: analysis or interpretation of data, statistical analysis. Dan Mungas: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, statistical analysis. Cindy Nowinski: drafting/revising the manuscript, study concept or design, study supervision. Jennifer Richler: drafting/revising the manuscript, study concept or design, acquisition of data. Joanne Deocampo and Jacob Anderson: study concept or design, analysis or interpretation of data, contribution of vital reagents/tools/patients, acquisition of data, statistical analysis, study supervision. Jennifer Manly: drafting/revising the manuscript, acquisition of data. Beth Borosh: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data, study supervision. Richard Havlik: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data. Kevin Conway: drafting/revising the manuscript, study concept or design, study supervision. Emmeline Edwards: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, member of the NIH Project team managing the Tool Box Contract that produced data for this manuscript. Lisa Freund: study concept or design, study supervision, obtaining funding. Jonathan King and Claudia Moy: drafting/revising the manuscript. Ellen Witt: drafting/revising the manuscript, analysis or interpretation of data. Richard Gershon: drafting/revising the manuscript, study concept or design, analysis or interpretation of data, acquisition of data, statistical analysis, study supervision, obtaining funding.

STUDY FUNDING

This study is funded in whole or in part with Federal funds from the Blueprint for Neuroscience Research, NIH, under contract no. HHS-N-260-2006-00007-C.

DISCLOSURE

S. Weintraub is funded by NIH grants R01DC008552, P30AG013854, and the Ken and Ruth Davee Foundation, and conducts clinical neuropsychological evaluations (35% effort) for which her academic-based practice clinic bills. S. Dikmen receives research grant funding from NIH R01 NS058302 and R01HD061400, NIDRR H133A080035, NIDRR H133G090022, and NIDRR, H133A980023, and DoD W81XWH-0802-0159. R. Heaton is funded by NIH grants P30MH062512, P50DA026306, P01DA012065, R01MH060720, R01MH073433, R01MH058076, R01MH078748, R01MH078737, U01MH083506, R01MH083552, and R01MH081861. D. Tulsky is funded by NIH contracts H133B090024, H133N060022, H133G070138, B6237R, cooperative agreement U01AR057929, and grant R01HD054659. He has received consultant fees from the Institute for Rehabilitation and Research, Frazier Rehabilitation Institute/Jewish Hospital, Craig Hospital, and Casa Colina Centers for Rehabilitation. P. Zelazo serves on the editorial boards of Child Development, Development and Psychopathology, Frontiers in Human Neuroscience, Cognitive Development, Emotion, and Developmental Cognitive Neuroscience, and Monographs of the Society for Research in Child Development. He is a Senior Fellow of the Mind and Life Institute and President of the Jean Piaget Society. He receives research funding from the Canadian Institute for Health Research (grant 201963), NIDDK/NICHD (1699-662-6312), and the Baumann Foundation. P. Bauer is funded by NIH grant HD067359. N. Carlozzi is funded by NIH grant R03NS065194 and by contracts H133B090024, B6237R, and H133G070138; she previously received funding from NIH grant H133A070037-08A and a grant from the NJ Department of Health and Senior Services. J. Slotkin, D. Blitz, and K. Wallner-Allen report no disclosures. N. Fox is funded by NIH grants R37HD017899, MH074454, U01MH080759, R01MH091363, P50MH078105, and P01HD064653. He serves on the scientific board of the National Scientific Council for the Developing Child. J. Beaumont served as a consultant for NorthShore University HealthSystem, FACIT.org, and Georgia Gastroenterology Group PC. She received funding for travel as an invited speaker at the North American Neuroendocrine Tumor Symposium. D. Mungas is funded by research grants from the National Institute on Aging and a grant from the California Department of Public Health California Alzheimer's Disease Centers program. C. Nowinski receives or has received research support from the NIH (contracts HHSN265200423601C, HHSN260200600007C, and HHSN267200700027C), the Department of Veteran's Affairs, the Analysis Group, Novartis, and Teva Pharmaceuticals. She has also received honoraria for writing and updating an article for Medlink. J. Richler is funded by NIH/NCRR grant UL1RR025761. J. Deocampo and J. Anderson report no disclosures. J. Manly is funded by NIH grants R01AG028786 and R01AG037212; she previously received funding from NIH grant R01AG016206 and a grant from the Alzheimer's Association (IIRG 05-14236). B. Borosh, R. Havlik, and K. Conway report no disclosures. E. Edwards is the Director of the Division of Extramural Research at NCCAM. Dr. Edwards declares that except for income received from her primary employer, no financial support or compensation has been received from any individual or corporate entity over the past 3 years for research or professional service and there are no personal financial holdings that could be perceived as constituting a potential conflict of interest. L. Freund reports no disclosures. J. King is the NIA Project Scientist for the NIH cooperative agreements U01AG014289, U01AG014276, U01AG14260, U01AG14282, and U01AG014263 (comprising the ACTIVE clinical trial). C. Moy and E. Witt report no disclosures. R. Gershon has received personal compensation for activities as a speaker and consultant with Sylvan Learning, Rockman, and the American Board of Podiatric Surgery. He has several grants awarded by NIH: N01-AG-6-0007, 1U5AR057943-01, HHSN260200600007, 1U01DK082342-01, AG-260-06-01, HD05469; NINDS: U01 NS 056 975 02; NHLBI K23: K23HL085766 NIA; 1RC2AG036498-01; NIDRR: H133B090024; OppNet: N01-AG-6-0007. Go to Neurology.org for full disclosures.

DISCLAIMER

The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of NIH or any of the sponsoring organizations, agencies, or the U.S. government.