Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Mar 20.
Published in final edited form as: J Int Neuropsychol Soc. 2015 Mar 20;21(3):231–238. doi: 10.1017/S1355617715000144

An FMRI-Compatible Symbol Search Task

Spencer W Liebel 1, Uraina S Clark 2, Xiaomeng Xu 3, Hannah H Riskin-Jones 4, Brittany E Hawkshead 1, Nicolette F Schwarz 1, Donald Labbe 4, Beth A Jerskey 4, Lawrence H Sweet 1,4
PMCID: PMC4454461  NIHMSID: NIHMS693192  PMID: 25794263

Abstract

Our objective was to determine whether a Symbol Search paradigm developed for functional magnetic resonance imaging (FMRI) is a reliable and valid measure of cognitive processing speed (CPS) in healthy older adults. As all older adults are expected to experience cognitive declines due to aging, and CPS is one of the domains most affected by age, establishing a reliable and valid measure of CPS that can be administered inside an MR scanner may prove invaluable in future clinical and research settings. We evaluated the reliability and construct validity of a newly developed FMRI Symbol Search task by comparing participants’ performance in and outside of the scanner and to the widely used and standardized Symbol Search subtest of the Wechsler Adult Intelligence Scale (WAIS). A brief battery of neuropsychological measures was also administered to assess the convergent and discriminant validity of the FMRI Symbol Search task. The FMRI Symbol Search task demonstrated high test–retest reliability when compared to performance on the same task administered out of the scanner (r = .791; p<.001). The criterion validity of the new task was supported, as it exhibited a strong positive correlation with the WAIS Symbol Search (r = .717; p<.001). Predicted convergent and discriminant validity patterns of the FMRI Symbol Search task were also observed. The FMRI Symbol Search task is a reliable and valid measure of CPS in healthy older adults and exhibits expected sensitivity to the effects of age on CPS performance.

Keywords: Cognitive processing speed, Aging, Neuropsychological assessment, Cognitive decline, Construct validity, Reliability

INTRODUCTION

The proportion of older adults in the population of industrialized countries is rising substantially due to falling birth rates and increased longevity (Bloom, 2011). For instance, by 2050 it is estimated that there will be nearly twice as many older adults over 65 than children under 15 in the United States (Centers for Disease Control and Prevention, 2013; Cohen, 2003). Unfortunately, increased longevity does not ensure preserved quality of life, as all older adults are expected to experience age-associated cognitive decline, neuronal deterioration, and compromised white matter integrity (Gunning-Dixon, Brickman, Cheng, & Alexopoulos, 2009; Park & Reuter-Lorenz, 2009; Wen & Sachdev, 2004). The cognitive impairments experienced by older adults are associated with compromised white matter integrity, and they include poorer performance on tasks of cognitive processing speed (CPS), memory, and executive functions (Gunning-Dixon & Raz, 2000).

Deterioration of brain tissue often manifests as white matter hyperintensities (WMH) on T2-weighted magnetic resonance images (MRI; DeCarli et al., 2005; Rabbitt et al., 2007; Söderlund et al., 2006). WMH are present in 11–21% of adults aged 64, and this proportion increases to 94% by age 82 (Debette & Markus, 2010). Additionally, prior research has shown that, in otherwise healthy older adults, WMH are linked to poorer performance on many tests of cognitive functioning, including measures of CPS (Papp et al., 2014; van den Heuvel et al., 2006). Due to its prevalence and significance as a neurocognitive marker, quantifying and tracking WMH volume has been a valuable clinical and research outcome measure (Schmidt et al., 2004).

An extensive body of research literature shows that cognitive functioning declines consistently with age (e.g., Rabbitt, 2002, for review). Gradual age-related declines are most robust in the cognitive domains of working memory, long-term memory, and CPS (Gunning-Dixon & Raz, 2000; Salthouse, 2000). Simply considered a consequence of aging, these decreases in performance occur in the absence of obvious disease or trauma. For instance, in healthy adults, raw scores on Symbol Search, a subtest of the Wechsler Adult Intelligence Scale (WAIS) that is used to assess CPS, decline by more than 50% between the ages of 25 and 65 (Wechsler, 2008). CPS is also of particular interest to aging research because it is a fundamental component of many of the brain’s other functions (Rypma & Prabhakaran, 2009; Salthouse, 1996). Indeed, age-related slowing of CPS is a significant contributor to declines in test scores in other cognitive domains (Finkel et al., 2004; Salthouse & Coon, 1993; Whiting & Smith, 1997). Because CPS is a fundamental component of many cognitive functions, this accurate assessment may be particularly useful as a sensitive predictor of changes in higher-order cognitive abilities, and an early marker of brain dysfunction (Duering et al., 2014; Eckert, 2011; Salthouse & Ferrer-Caja, 2003). Therefore, measures of CPS, such as Symbol Search, are frequently used to detect both abnormal and normal age-related changes in brain integrity.

Over the past 20 years, functional magnetic resonance imaging (FMRI) has demonstrated great value in improving assessments of brain function associated with age-related cognitive decline. Clinical applications for FMRI have also been emerging, such as presurgical mapping, and have contributed to presymptomatic diagnostics of a broad range of diseases (Matthews, Honey, & Bullmore, 2006). Many of these advances in aging research have relied on accurate and concurrent assessments of behavioral performance during FMRI. A fundamental assumption of the technique is that the brain activity elicited is a response to a carefully controlled behavioral challenge of a well-defined neurocognitive construct. Yet, the cognitive tests used by FMRI investigators in the scanner are not typically normed or standardized across sites. Indeed, some have not been validated at all. Therefore, it is imperative that valid FMRI paradigms are developed that also have demonstrated generalizability and sensitivity to cognitive decline in older adults.

Relatively few CPS paradigms have been adapted for administration during FMRI, and those that have been, have not been systematically examined for reliability and validity. Moreover, the study of functional neuroimaging correlates of CPS has lagged behind other neurocognitive functions, such as working memory and attention. Indeed, the use of externally validated FMRI paradigms is rare in any neurocognitive domain. When standardized measures (e.g., neuropsychological tests) have been adapted for FMRI, they often undergo extensive changes to make them feasible in the scanning environment or subsequent data analyses (e.g., Langeneker, Nielson, & Rao, 2004; Leavitt, Wylie, Genova, Chiaravalloti, & Deluca, 2012; Phelps, Hyder, Blamire, & Shulman, 1997). Given the major challenges of the setting, many of these alterations are necessary and, nevertheless, have yielded valuable findings. For example, typical major alterations to tasks include nonverbal responding during a verbal task (Langeneker et al., 2004), visual presentation of an auditory task (Staffen et al., 2002), requiring a motor response instead of an oral response (Leavitt et al., 2012), or standardized-pacing of a self-paced test (Langenecker et al., 2004). More questionable is the less common, and now rare, strategy of assessing behavioral performance outside of the scanner instead of during the scan. While these strategies have provided valuable insights into neurocognitive functioning, modifying these measures calls into question their generalizability to the original measure and their underlying neurocognitive constructs. Thus, establishing validated behavioral tasks for FMRI assessments would allow for greater reliability, generalizability, and clinical utility (Cohen & Sweet, 2011).

The overall goal of this study was to determine if a Symbol Search paradigm developed for FMRI is a reliable and valid behavioral challenge of CPS in healthy older adults. Specifically, our aim was to determine if performance on a newly developed FMRI Symbol Search task demonstrated test–retest reliability when given in and out of the scanner, and whether it exhibited criterion validity (i.e., correlated strongly with the WAIS-III Symbol Search subtest). Construct validity was further assessed by examining convergent and discriminant validity (Cronbach & Meehl, 1955). Associations between the FMRI Symbol Search paradigm and other measures of the CPS construct, age, and WMH volume were examined to determine convergent validity. We examined discriminant validity by assessing the relationship between the FMRI Symbol Search task and measures that are known to exhibit weak relationships to CPS. In sum, we predicted that the FMRI Symbol Search task would exhibit high reliability and strong construct validity as a measure of CPS in healthy older adults.

METHODS

Participants

A community sample of 45 healthy older adults (31 female) over the age of 50 were recruited via newspaper ads and flyers (Table 1). Ages ranged from 50 to 85 years (M age = 63.09; SD = 8.44). Mean level of education was 15.75 years (SD = 2.26). WMH volume ranged from 1.9 to 45.5 mL (M = 7.74 mL; SD = 7.17 mL). Participants were monetarily compensated. The inclusion criteria were that the individuals be right-handed English-speakers with normal or corrected vision at the time of testing. Potential participants were excluded if they were diagnosed with significant heart problems (e.g., surgery, infarct), neurological disease (e.g., history of stroke, multiple sclerosis), traumatic brain injury (with loss of consciousness), history of substance abuse that resulted in hospitalization, diagnosis of any current psychiatric illness, or any MRI contraindications (e.g., metal implants). The study was approved by hospital and university Internal Review Boards and conformed to the Helsinki Declaration.

Table 1.

Participant demographics, neuropsychological test battery data, and medications taken

M SD
Age 63.09 8.44
Level of education 15.75 2.26
WMH volume 7.74 7.17
Neuropsychological measures M SD
WAIS Symbol Search 30.67 6.92
D-KEFS Trails 2 33.89 9.92
D-KEFS Trails 3 34.53 12.96
D-KEFS CWIT (color naming) 29.78 5.62
D-KEFS CWIT (word reading) 21.84 4.21
RBANS: Coding 48.04 8.36
PPT: dominant hand 12.76 2.13
RBANS: Picture Naming 9.80 0.46
RBANS: List Learning Recognition 19.47 0.94
RBANS: Figure Copy 17.76 2.47
RBANS: Line Orientation 17.27 2.68
WTAR Total Score 42.98 5.73
MMSE Total Score 29.36 1.26
Medications taken M SD
Total medication count 4.07 4.65
Medication classes Number of participants Proportion of sample
Cardiovascular 25 55.56%
Psychiatry 16 35.56%
Neurology 4 8.89%
Analgesics 5 11.12%
Dermatology 1 2.23%
Ear, nose, and throat 2 4.45%
Gastroenterology 6 13.34%
Hematology 2 4.45%
Immunology 0 0%
OB/Gyn 4 8.89%
Oncology 1 2.23%
Ophthalmology 2 4.45%
Pulmonary 5 11.12%
Urology 1 2.23%
Open in a new tab

Procedures

Participants completed a neuropsychological and MRI assessment during two separate visits. The neuropsychological assessment was supervised by a licensed clinical neuropsychologist. It took place before the MRI assessment in a quiet room and included the WAIS-III Symbol Search subtest. Administration of the 2-min Symbol Search subtest from the WAIS occurred approximately 40 min after the start of the 2-hr neuropsychological assessment battery. Responses were collected via paper and pencil as per the WAIS-III administration manual (Wechsler, 1997). The MRI assessment was 1 hr long and the FMRI Symbol Search paradigm was presented approximately 25 min after the start of the scanning session. Stimuli were presented onto a projection screen that was visible to the participant while in the scanner. The number of correct responses during a 2-min period of time was determined based on yes/no button presses on a MRI compatible response box. The FMRI Symbol Search paradigm was also presented out of the scanner approximately 20 min before the FMRI scan.

Measures

FMRI Symbol Search

The FMRI Symbol Search paradigm was administered with minor adaptations according to the instructions in the WAIS-III manual (Wechsler, 1997) and a prior in-scanner version (Sweet et al., 2005). Task instructions were the same as those provided before administration of the WAIS-III Symbol Search subtest. The task was presented using E-Prime 2.0 (http://www.pstnet.com/). Although FMRI data were collected concurrently with the in-scanner Symbol Search task, these data were not analyzed as part of the current study.

The task consisted of two, four-cycle imaging runs of 6 min each. Each of the four cycles consisted of a 30-s control task block followed by a 30-s Symbol Search task block. Participants responded to as many items as possible during the time allotted. To avoid laterality effects due to visual presentation, the exemplar figures were placed directly above the five target figures (Figure 1). Instead of drawing a line through their desired response, participants used a button box to identify whether either exemplar was present in the target symbol group using a “YES” or “NO” button. A new item appeared immediately following button press. A visuomotor control task was also administered. The control task asked participants to respond “YES” or “NO” to indicate whether a particular symbol was present anywhere on the screen. The target symbol participants were asked to identify during the control task remained the same for the duration of the task. The control task was self-paced. The number of correct items during each block was recorded using E-Prime. The symbols presented during the FMRI Symbol Search task were comprised of syllabic characters (excluding ideograms) from the Mycenaean alphabet Linear B, which has not been in use for over 3200 years.

Fig. 1.

Fig. 1

Open in a new tab

Sample FMRI Symbol Search and visuomotor control task items. This figure demonstrates how the FMRI Symbol Search task and visuomotor control task were presented to participants, including instructions before task items.

Measure of Reliability

The same FMRI Symbol Search task was presented both inside and outside of the scanner to examine reliability. The outside-of-scanner administration occurred approximately 45 min before the in-scanner version was presented using a computer running E-Prime 2.0 and a connected response box. When administered outside of the scanner, the FMRI Symbol Search task included the same instructions, presentation, and response format as given in the scanner. High reliability, indicated by a high correlation coefficient between performances on the FMRI Symbol Search task administered in and out of the scanner, would support the generalizability of the FMRI Symbol Search task despite the uniqueness of the scanning environment (e.g., interactions with individual differences, measurement error).

Measure of Criterion Validity

WAIS Symbol Search

Symbol Search is a subtest of the WAIS (Wechsler, 2008) and is often included in clinical neuropsychological assessments to measure visuospatial attention and CPS. This is a self-paced task during which examinees are allotted 2 min to complete as many items as possible. The task requires rapid comparisons to determine if a set of five target geometric designs include one of two exemplars, which are positioned just to the left of the target designs. If one of the exemplars is present in the set of five target designs participants draw a line through the target design; if neither of the two exemplars is found in the group of target designs a line is drawn through a “NO” box. Participants are instructed to work as quickly and accurately as possible until they are told to stop. It was predicted that a high correlation coefficient between performance on the WAIS Symbol Search and the FMRI Symbol Search would represent strong criterion validity.

Measures of Convergent Validity

The following measures were examined to assess convergent validity. All available CPS measures from the neuropsychological battery were included in our analyses. They were expected to exhibit strong associations with the FMRI Symbol Search task. Because these measures are sensitive to the effects of age and the conversion from raw scores to normed scores takes into account participant age, raw scores were used in all analyses

Delis-Kaplan Executive Functioning System (D-KEFS) Trails Making Test

The D-KEFS Trails Making Test (Delis, Kaplan, & Kramer, 2001) is a test frequently used as a measure of visuo-motor scanning, executive functioning, and CPS in neuropsychological test batteries (Lezak, Howieson, Bigler, & Tranel, 2012). For this study, Trails 2 (number sequencing) and 3 (letter sequencing) were used because they are considered measures of CPS. Administration and scoring of Trails 2 and 3 were completed according to the D-KEFS administration manual (Delis et al., 2001).

Delis-Kaplan Executive Functioning System (D-KEFS) Color-Word Interference Test

The D-KEFS Color-Word Interference Test (CWIT; Delis et al., 2001) consists of four parts: color naming, word reading, inhibition, and inhibition/switching. The present study examined performance on color naming and word reading because these are routinely used as a measure of CPS in neuropsychological test batteries. For color naming, participants were presented with a page containing a series of red, green, and blue squares. The participants were asked to say the names of the colors as quickly as possible without making mistakes. For word reading, the participants were presented with a page containing the words “red”, “green”, and “blue” printed in black ink. Participants were asked to read the words aloud as quickly as possible without making mistakes. Participant performance was measured by completion time on each trial.

Repeated Battery for the Assessment of Neuropsychological Status (RBANS) Coding

RBANS Coding (Randolph, Tierney, Mohr, & Chase, 1998) is a frequently used measure of CPS in neuropsychological assessments of older adults. Briefly, participants were instructed to fill in missing digits corresponding to shapes in a key located at the top of the page as quickly as possible in 90 s. Administration and scoring followed the instructions provided in the RBANS manual.

The Purdue Pegboard Test (PPT)

The PPT (Model 32020; Lafayette Instrument Co., Lafayette, IN) is a functional assessment tool of hand dexterity and psychomotor processing speed. It consists of 30 holes arranged in two identical columns and pegs located in four cups at the top of the board. Subjects were instructed to start the test after a verbal cue from an examiner and the examiner timed the test with a stopwatch. Subjects were allotted 30 s to fill the holes with pegs using their dominant hand, then 30 s to fill them with their non-dominant hand, and finally with both hands simultaneously. Each subtest was repeated three times to obtain an average. In this study, only dominant hand data were used as a measure of CPS. The test scores equaled the number of correctly placed pegs.

WMH quantification

WMH volume was quantified from high resolution (1 mm3) fluid attenuated inversion recovery (FLAIR) and T1-weighted MRI sequences as described in detail elsewhere (Riskin-Jones, Xu, Clark, Labbe, & Sweet, 2014). Briefly, high-resolution whole-brain T1-weighted images were individually segmented into white and gray matter using Free-Surfer following established procedures (Fischl et al., 2002). The segmentations were aligned to high-resolution whole-brain FLAIR images in native space using AFNI-SUMA alignment tools (Cox, 1996; Saad, 2004). An iterative region growing algorithm was applied to the white matter segmentation to identify WMH. Initially, seeds were identified as voxels with intensities of at least 25% above the median intensity. The iterative algorithm searched within a 27-voxel rectangle around each voxel in a seed region for adjacent voxels that fall within 5% of the seed mean. Those neighboring voxels were then added to the total seed region volume used in the next iteration. The algorithm continued until no new voxels were added. Total WMH volume was the sum of the voxels included in this iterative process.

Measures of Discriminant Validity

The following measures are known to be weakly related to the CPS construct and age. Thus, their associations with the FMRI Symbol Search task were predicted to be relatively weak and substantially accounted for by age.

RBANS Picture Naming

RBANS Picture Naming (Randolph et al., 1998) is a test of language, specifically confrontation naming, which includes 10 line drawings of objects that must be named by the examinee. The examinee is given up to 20 s to name each object. This test is not considered to consist of a strong CPS component.

RBANS List Learning Recognition

RBANS List Learning Recognition (Randolph et al., 1998) is a measure of delayed verbal memory that requires examinees to accurately identify items from a previously learned word list. Examinees respond either “YES” or “NO” to words presented from the list or 10 distractors. This subtest yields a score between zero and 20.

RBANS Figure Copy

RBANS Figure Copy (Randolph et al., 1998) is a measure of visuospatial and constructional skills. Examinees are asked to copy a complex geometric figure. The figure consists of 10 components and yields a maximum score of 20.

RBANS Line Orientation

RBANS Line Orientation (Randolph et al., 1998) is a test of visuospatial abilities. The task presents examinees with an array of 13 lines, fanning out from the same origination point but in different directions. For each item, two target lines were shown beneath the array and subjects identified which lines match within the array. There are 10 items, each containing two lines to be matched, for a total maximum score of 20.

Wechsler Test of Adult Reading (WTAR)

The WTAR (Wechsler, 2001) assesses the participants’ familiarity with irregularly pronounced words and is regularly used for clinical and research purposes to assess premorbid IQ. Administration and scoring followed prescribed methods from the WTAR manual. Briefly, examinees were presented with a page of words and asked to read each one aloud. This is a self-paced test, and the number of correctly read words constituted the participants’ score.

Mini-Mental State Examination (MMSE)

The MMSE (Folstein, Folstein, & McHugh, 1975) is the most commonly used geriatric cognitive screening tool in the United States, Canada, and United Kingdom (Shulman et al., 2006). It is a very brief instrument, but it has been shown to reliably screen for cognitive deficit in older adults. Administration and scoring followed guidelines prescribed in the test manual.

Statistical Procedures

To examine the reliability and validity of the newly adapted FMRI Symbol Search paradigm, the following statistical procedures were conducted. To assess reliability, the association between the same FMRI Symbol Search task that was administered in and out of the scanner was examined by calculating a Pearson correlation coefficient. To measure construct validity, we first assessed criterion validity and then convergent and discriminant validity. Criterion validity was measured using a Pearson correlation between the FMRI Symbol Search task and raw WAIS Symbol Search subtest scores. To measure convergent validity, Pearson correlation coefficients between the FMRI Symbol Search task and raw scores of other well-established measures of CPS were examined. Absolute correlation coefficients less than .3 were considered poor, .3–.6 adequate, and .6 or greater were good to very good. These values differ from the widely known “conventions” suggested by Cohen (1988, 1992) and represent an effort to place more stringent criteria on the data (Hemphill, 2003). Discriminant validity was measured by calculating correlation coefficients between the FMRI Symbol Search task and raw scores on measures that should not be strongly related to CPS; poor correlations lower than .3 were considered evidence of discriminant validity. Significance threshold for all analyses was set at p<.05, two-tailed. Statistical Package for the Social Sciences (SPSS; version 21) was used for all analyses.

RESULTS

The FMRI Symbol Search task demonstrated good to very good test–retest reliability with a significant positive correlation when compared to performance on the same task administered out of the scanner (r = .791; p<.001). Mean performance on the task administered outside of the scanner improved compared to in-scanner performance (t(44) = −3.257; p<.002). Criterion validity analysis revealed that the FMRI Symbol Search task exhibited significant positive correlation with the WAIS Symbol Search (r = .717; p<.001), which corresponded to the good to very good range. Correlations between the FMRI Symbol Search and other measures of CPS are shown in Table 2 (i.e., convergent validity). Before controlling age, the FMRI Symbol Search task was significantly associated with all measures. Associations ranged from adequate to good to very good. Notably, there was a strong expected inverse relationship between the FMRI Symbol Search task and age. Age was controlled in subsequent analyses to explore the underlying relationships between these other measures of CPS independent of the known effects of age. Therefore, partial correlations between FMRI Symbol Search and the other CPS measures were thought to provide an extremely stringent representation of the underlying CPS construct by controlling extraneous variables associated with age. After controlling age, two measures remained in the good to very good range (RBANS Coding and D-KEFS Trails 3), with the remaining measures falling at or near the adequate cutoff. Table 3 contains the results of discriminant validity analyses. Before controlling age, the FMRI Symbol Search task showed predicted weak correlation (r<.3) with five of the six non-CPS measures falling in the poor range. After controlling for age, all six non-CPS measures demonstrated predicted weak correlations (r<.3). These results suggest that the FMRI Symbol Search task exhibits only weak associations with the measures of cognitive domains chosen for discriminant validity.

Table 2.

Convergent validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age

Other CPS measure Pearson correlations
Partial correlations controlling age
r p Partial r p
Age −.411 .005 — —
D-KEFS Trails 2 (number sequencing) −.320 .032 −.295 .068
D-KEFS Trails 3 (letter sequencing) −.713 .001 −.740 .001
D-KEFS CWIT (color naming) −.403 .006 −.303 .046
D-KEFS CWIT (word reading) −.348 .019 −.248 .104
RBANS: Coding .678 .001 .602 .001
PPT: dominant hand .474 .001 .254 .024
WMH Volume (corrected) −.448 .004 −.290 .073
Open in a new tab

Table 3.

Discriminant validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age

Non-CPS measure Pearson correlations
Partial correlations controlling age
r p Partial r p
RBANS: Picture Naming −.005 .975 .091 .594
RBANS: List Learning Recognition −.053 .728 .105 .535
RBANS: Figure Copy .183 .228 .030 .858
RBANS: Line Orientation .075 .629 .081 .635
WTAR Total Score .276 .070 .202 .231
MMSE Total Score .506 .001 .187 .268
Open in a new tab

DISCUSSION

The FMRI Symbol Search task was developed to provide clinicians and researchers a reliable and valid measure of CPS that would generalize to the WAIS Symbol Search subtest. Findings strongly support the reliability and construct validity of our FMRI-adapted Symbol Search task as a measure of CPS in healthy older adults. Indeed, the reliability of in- and out-of-scanner results was good to very good (r = .791; Hemphill, 2003).When compared to the WAIS Symbol Search, a widely accepted and carefully standardized measure of CPS, our FMRI Symbol Search task also showed strong criterion validity that was in the good to very good range. To our knowledge, this is the first systematic demonstration that a CPS measure may be adapted for use in the FMRI environment while maintaining high reliability and generalizability.

These data also suggest that the FMRI Symbol Search task demonstrated excellent convergent validity when compared to the other measures of CPS. In fact, the FMRI Symbol Search task showed adequate to good to very good Pearson correlation coefficients with age and all CPS measures (Hemphill, 2003). These effects remained despite controlling age, which is a well-documented correlate of the WAIS Symbol Search and other measures of CPS. Age was controlled to examine residual effects (i.e., the underlying CPS construct) without the influence of this potential extraneous variable confound (i.e., third variable effects associated with age). Three of the other measures of CPS exhibited adequate to good to very good correlation with the FMRI Symbol Search task when age was controlled. The four that did not yield strong correlation coefficients after age correction, nevertheless, yielded effects that were very close to the adequate cutoff of r = .3. It is hypothesized that these measures no longer reached the adequate cutoff due to a reduction in power when age was controlled. Overall, these findings provided robust support for convergent validity because controlling common variance in all CPS measures that was associated with age likely yields an underestimation of the true relationships among the measures.

The FMRI Symbol Search task demonstrated excellent discriminant validity as well. It was only weakly associated (i.e., not adequate) with measures that do not include a strong CPS component. Overall, these correlations became even weaker after controlling the effects of age. The discriminant validity results suggest, as expected, that the FMRI Symbol Search task is not associated with non-CPS constructs (e.g., memory, visuospatial skills, or language).

These findings are important because they support the utility of this FMRI-compatible measure of CPS. The development of reliable FMRI paradigms with demonstrated construct validity and generalizability is crucial to understanding CPS and the other cognitive domains affected by the aging brain. With an increasing geriatric population, FMRI paradigms that assess cognitive domains important to aging will be in greater demand. Due to its high level of association with the WAIS Symbol Search, the FMRI Symbol Search task may find use in future studies as an efficient proxy of CPS in this population. Furthermore, the format of the FMRI Symbol Search task relies less on motor coordination abilities than does the WAIS Symbol Search. This is often an area of difficulty for older persons and individuals with clinical conditions that result in motor coordination dysfunctions. Additionally, the potential exists for this FMRI measure to be incorporated into cognitive batteries used to assess patients in which CPS is known to be one of the cardinal markers of cognitive deficit (e.g., survivors of traumatic brain injury or individuals with multiple sclerosis). Because routine clinical care of these patients often includes separate cognitive and MRI assessments, a standardized CPS assessment could be shifted to the MRI session to yield concurrent acquisition of complementary behavioral and functional data. Validation of our FMRI Symbol Search measure allows clinicians and researchers to link clinically relevant behavioral performance to routinely assessed structural MRI indices and adds novel information about concurrent brain function. This is likely to enhance diagnostics, prognostics, and outcome monitoring.

The interpretation of the results of this study must, however, be considered in light of some limitations. The reliability and validity of the FMRI-adapted Symbol Search paradigm were only examined in an older adult sample with above average levels of education. While the greatest expected CPS decline and variance was expected in this sample, administering the task to younger adults with a greater range of educational achievement is necessary to support its validity across the lifespan and levels of education.

There is a great need for standardized FMRI assessments in both existing research settings and emerging clinical applications. The FMRI Symbol Search task provides researchers and clinicians such a test and represents the first measure of CPS adapted for use in FMRI. Because clinical applications for FMRI are emerging, behavioral measures adapted for use inside the scanning environment will be useful to increase generalizability to the normed clinical measure about which much is already known. Indeed, there are many additional neuropsychological measures that can be adapted for use with FMRI so that clinicians and researchers may benefit from the advantages and strengths of the behavioral measure and FMRI technology to reveal neural correlates.

Acknowledgments

This work was supported by grants from the National Institutes of Health Heart, Lung and Blood Institute (L.H.S., R01 HL084178) and the National Institutes of Health (U.S.C., K23 MH096628).

Footnotes

The authors do not report any conflicts of interest.

References

  1. Bloom DE. 7 billion and counting. Science. 2011;333(6042):562–569. doi: 10.1126/science.1209290. [DOI] [PubMed] [Google Scholar]
  2. Centers for Disease Control and Prevention. The state of aging and health in America. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services; 2013. [Google Scholar]
  3. Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
  4. Cohen J. A power primer. Psychological Bulletin. 1992;112(1):155–159. doi: 10.1037/0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
  5. Cohen JE. Human population: The next half century. Science. 2003;302(5648):1172–1175. doi: 10.1123/science.1088665. [DOI] [PubMed] [Google Scholar]
  6. Cohen RA, Sweet LH. Brain imaging in behavioral medicine and clinical neuroscience: Synthesis. In: Cohen RA, Sweet LH, editors. Brain imaging in behavioral medicine and clinical neuroscience. New York: Springer; 2011. pp. 383–394. [DOI] [Google Scholar]
  7. Cox RW. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  8. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin. 1955;52(4):281–302. doi: 10.1037/h0040957. [DOI] [PubMed] [Google Scholar]
  9. Debette S, Markus HS. The clinical importance of white matter hyperintensities on brain magnetic resonance imaging: Systematic review and meta-analysis. British Medical Journal. 2010;341:c3666. doi: 10.1136/bmj.c3666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. DeCarli C, Massaro J, Harvey D, Hald J, Tullberg M, Au R, Wolf PA. Measures of brain morphology and infarction in the Framingham heart study: Establishing what is normal. Neurobiology of Aging. 2005;26(4):491–510. doi: 10.1016/j.neurobiolaging.2004.05.004. [DOI] [PubMed] [Google Scholar]
  11. Delis DC, Kaplan E, Kramer JH. Delis Kaplan Executive Function System. San Antonio, TX: The Psychological Corporation; 2001. [Google Scholar]
  12. Duering M, Gesierich B, Seiler S, Pirpamer L, Gonik M, Hofer E, Dichgans M. Strategic white matter tracts for processing speed deficits in age-related small vessel disease. Neurology. 2014;84:1946–1950. doi: 10.1212/WNL.0000000000000475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Eckert MA. Slowing down: Age-related neurobiological predictors of processing speed. Frontiers in Neuroscience. 2011;5:25. doi: 10.3389/fnins.2011.00025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Finkel SI, Mintzer JE, Dysken M, Krishnan KRR, Burt T, McRae T. A randomized, placebo-controlled study of the efficacy and safety of sertraline in the treatment of the behavioral manifestations of Alzheimer’s disease in outpatients treated with donepezil. International Journal of Geriatric Psychiatry. 2004;19:9–18. doi: 10.1002/gps.998. [DOI] [PubMed] [Google Scholar]
  15. Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Hasselgrove C, Dale AM. Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain. Neuron. 2002;33(3):341–355. doi: 10.1016/s0896-6273(02)00569-x. [DOI] [PubMed] [Google Scholar]
  16. Folstein MF, Folstein SE, McHugh PR. “Minimental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12(3):189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  17. Gunning-Dixon FM, Brickman AM, Cheng JC, Alexopoulos GS. Aging of cerebral white matter: A review of MRI findings. International Journal of Geriatric Psychiatry. 2009;24(2):109–117. doi: 10.1002/gps.2087. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gunning-Dixon FM, Raz N. The cognitive correlates of white matter abnormalities in normal aging: A quantitative review. Neuropsychology. 2000;14:224–232. doi: 10.1037//0894-4105.14.2.224. [DOI] [PubMed] [Google Scholar]
  19. Hemphill JF. Interpreting the magnitudes of correlation coefficients. American Psychologist. 2003;58(1):78–79. doi: 10.1037/0003-066X.58.1.78. [DOI] [PubMed] [Google Scholar]
  20. Langenecker SA, Nielson KA, Rao SM. fMRI of healthy older adults during Stroop interference. Neuroimage. 2004;21:192–200. doi: 10.1016/j.neuroimage.2003.08.027. [DOI] [PubMed] [Google Scholar]
  21. Leavitt VM, Wylie G, Genova HM, Chiaravalloti ND, Deluca J. Altered effective connectivity during performance of an information processing speed task in multiple sclerosis. Multiple Sclerosis. 2012;18(4):409–417. doi: 10.1177/1352458511423651. [DOI] [PubMed] [Google Scholar]
  22. Lezak MD, Howieson DB, Bigler ED, Tranel D. Neuropsychological assessment. 5. New York: Oxford University Press; 2012. [Google Scholar]
  23. Matthews PM, Honey GD, Bullmore ET. Applications of fMRI in translational medicine and clinical practice. Nature Reviews, Neuroscience. 2006;7(9):732–744. doi: 10.1038/nrn1929. [DOI] [PubMed] [Google Scholar]
  24. Papp KV, Kaplan RF, Springate B, Moscufo N, Wakefield DB, Guttmann CR, Wolfson L. Processing speed in normal aging: Effects of white matter hyperintensities and hippocampal volume loss. Aging, Neuropsychology, and Cognition. 2014;21(2):197–213. doi: 10.1080/13825585.2013.795513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Park DC, Reuter-Lorenz P. The adaptive brain: Aging and neurocognitive scaffolding. Annual Review of Psychology. 2009;60:173–196. doi: 10.1146/annurev.psych.59.103006.093656. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Phelps EA, Hyder F, Blamire AM, Shulman RG. FMRI of the prefrontal cortex during overt verbal fluency. Neuroreport. 1997;8:561–565. doi: 10.1097/00001756-199701200-00036. [DOI] [PubMed] [Google Scholar]
  27. Rabbitt P. Aging and cognition. In: Pashler H, editor. Steven’s handbook of experimental psychology. 3. New York: John Wiley & Sons; 2002. [Google Scholar]
  28. Rabbitt P, Scott M, Lunn M, Thacker N, Lowe C, Pendelton N, Jackson A. White matter lesions account for all age-related declines in speed but not in intelligence. Neuropsychology. 2007;21(3):363–370. doi: 10.1037/0894-4105.21.3.363. [DOI] [PubMed] [Google Scholar]
  29. Randolph C, Tierney MC, Mohr E, Chase TN. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology. 1998;20(3):310–319. doi: 10.1076/jcen.20.3.310.823. [DOI] [PubMed] [Google Scholar]
  30. Riskin-Jones HH, Xu X, Clark US, Labbe DR, Sweet LH. New tool for quantification of white matter hyperintensities. Poster presented at the meeting of the Cognitive Aging Conference; Atlanta, Georgia. 2014. Apr, [Google Scholar]
  31. Rypma B, Prabhakaran V. When less is more and when more is more: The mediating roles of capacity and speed in brain-behavior efficiency. Intelligence. 2009;37(2):207–222. doi: 10.1016/j.intell.2008.12.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Saad ZS. SUMA: An interface for surface-based intra- and inter-subject analysis with AFNI. Biomedical Imaging: Nano to Macro. 2004;2:1510–1513. doi: 10.1109/ISBI.2004.1398837. [DOI] [Google Scholar]
  33. Salthouse TA. The processing-speed theory of adult age differences in cognition. Psychological Review. 1996;103(3):403–428. doi: 10.1037/0033-295X.103.3.403. [DOI] [PubMed] [Google Scholar]
  34. Salthouse TA. Aging and measures of processing speed. Biological Psychology. 2000;54:35–54. doi: 10.1016/S0301-0511(00)00052-1. [DOI] [PubMed] [Google Scholar]
  35. Salthouse TA, Coon VE. Influence of task-specific processing speed on age differences in memory. Journal of Gerontology. 1993;48:245–255. doi: 10.1093/geronj/48.5.p245. [DOI] [PubMed] [Google Scholar]
  36. Salthouse TA, Ferrer-Caja E. What needs to be explained to account for age-related effects on multiple cognitive variables? Psychology and Aging. 2003;18(1):91–110. doi: 10.1037/0882-7974.18.1.91. [DOI] [PubMed] [Google Scholar]
  37. Schmidt R, Scheltens P, Erkinjuntti T, Pantoni L, Markus HS, Wallin A, Fazekas F. White matter lesion progression: A surrogate endpoint for trials in cerebral small-vessel disease. Neurology. 2004;63(1):139–144. doi: 10.1212/01.wnl.0000132635.75819.e5. [DOI] [PubMed] [Google Scholar]
  38. Shulman KI, Herrmann N, Brodaty H, Chiu H, Lawlar B, Ritchie K, Scanlan JM. IPA survey of brief cognitive screening instruments. International Psychogeriatrics. 2006;18(2):281–294. doi: 10.1017/S1041610205002693. [DOI] [PubMed] [Google Scholar]
  39. Söderlund H, Nilsson LG, Berger K, Breteler MM, Dufouil C, Fuhrer R, Launer LJ. Cerebral changes on MRI and cognitive function: The CASCADE study. Neurobiology of Aging. 2006;27(1):16–23. doi: 10.1016/j.neurobiolaging.2004.12.008. [DOI] [PubMed] [Google Scholar]
  40. Staffen W, Mair A, Zauner H, Unterrainer J, Neiderhofer H, Kutzelnigg A, Ladurner G. Cognitive function and fMRI in patients with multiple sclerosis: Evidence for compensatory cortical activation during an attention task. Brain. 2002;125:1275–1282. doi: 10.1093/brain/awf125. [DOI] [PubMed] [Google Scholar]
  41. Sweet LH, Paskavitz JF, O’Connor MJ, Browndyke JN, Wellen JW, Cohen RA. FMRI correlates of the WAIS-III Symbol Search subtest. Journal of the International Neuropsychological Society. 2005;11:471–476. doi: 10.1017/S1355617705050575. [DOI] [PubMed] [Google Scholar]
  42. van den Heuvel DM, ten Dam VH, de Craen AJ, Admiraal-Behloul F, Olofsen H, Bollen EL, van Buchem MA. Increase in periventricular white matter hyperintensities parallels decline in mental processing speed in a non-demented elderly population. Journal of Neurology, Neurosurgery, and Psychiatry. 2006;77(2):149–153. doi: 10.1136/jnnp.2005.070193. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Wechsler D. Wechsler Adult Intelligence Scale. 3. New York: Pearson; 1997. [Google Scholar]
  44. Wechsler D. Wechsler Test of Adult Reading. New York: Pearson; 2001. [Google Scholar]
  45. Wechsler D. Wechsler Adult Intelligence Scale. 4. New York: Pearson; 2008. [Google Scholar]
  46. Wen W, Sachdev P. The topography of white matter hyperintensities on brain MRI in healthy 60- to 64-year-old individuals. Neuroimage. 2004;22:144–154. doi: 10.1016/j.neuroimage.2003.12.027. [DOI] [PubMed] [Google Scholar]
  47. Whiting WL, IV, Smith AD. Differential age-related processing limitation in recall and recognition tasks. Psychology and Aging. 1997;12(2):216–224. doi: 10.1037/0882-7974.12.2.216. [DOI] [PubMed] [Google Scholar]

RESOURCES