Abstract
Cognitive reserve is a hypothetical construct that has been used to inform models of cognitive aging, and is presumed to be indicative of life experiences that may mitigate the effects of brain pathology. The purpose of this study was to evaluate the construct validity of cognitive reserve by examining both its convergent and discriminant validity across three different samples of participants using structural equation modeling. The cognitive reserve variables were found to correlate highly with one another (thereby providing evidence of convergent validity) but demanding tests of discriminant validity indicated that, in two of the samples, the cognitive reserve construct was highly related to an executive functioning construct.
Keywords: Construct validity, cognition, cognitive reserve, executive functioning, neuropsychology, structural equation modeling
The concept of reserve emerged as a mechanism to explain why there is not a perfect relationship between brain pathology post-mortem and the severity of a clinical condition, such as vascular dementia (VaD) or Alzheimer dementia. Research has indicated that individuals with the same amount of brain pathology may have very different clinical manifestations of dementia. Increased reserve is hypothesized to be associated with protection against clinical expression of dementia because those individuals with higher reserve may be more efficient or better able to compensate for the pathological changes accumulating in the brain. An initial model of reserve indicated the existence of two classifications of reserve- brain reserve and cognitive reserve. Both brain reserve and cognitive reserve are hypothetical constructs that have been used to inform models of aging. Our purpose here is to examine whether it is meaningful to refer to cognitive reserve as a distinct construct that represents a unique dimension of individual differences.
Although brain reserve is an important theoretical construct in its own right, in this paper we are primarily concerned with the concept of cognitive reserve that, Stern (2002) has argued, is an active model of reserve comprised of two main components- neural reserve and neural compensation. The neural reserve component of cognitive reserve refers to the efficiency or capacity of the brain prior to pathological damage. The neural compensation component of cognitive reserve refers to the recruitment of brain areas not typically used by individuals without brain damage to compensate for brain pathology.
Cognitive reserve is typically operationalized with variables that are reflective of lifetime experiences that may provide individuals with a set of skills to better manage pathological brain damage, as seen in diseases like AD and VaD. To that end, education level, literacy, occupational attainment, and performance on tests of knowledge (such as vocabulary) have been used as surrogates of cognitive reserve.
Evidence that greater educational attainment is associated with a reduced relative risk of developing AD has been demonstrated in the United States (e.g., Evans et al., 1997; Stern et al., 1994; White et al., 1994), China (Zhang et al., 1990), France (Letenneur et al., 1994), and Sweden (Qiu et al., 2001). An inverse relationship between occupational attainment and incident dementia has been found in a number of studies as well (see Valenzuela & Sachdev, 2005 for a review).
Research has indicated that those individuals who have greater cognitive reserve tend to decline more quickly once diagnosed with AD. This finding supports the idea that at any given level of clinical severity, patients with higher cognitive reserve have more pathological brain damage (Stern et al., 1995).
Although cognitive reserve is typically invoked as an important concept in dementia research, there is also evidence that cognitive reserve may play a protective role against normal age-related declines. Age-related declines in measures of memory, processing speed, reasoning, and spatial ability have been well documented both cross-sectionally (e.g., Salthouse, 2004) and longitudinally (e.g., Christensen, 2001). More education has been linked with slower declines in both cognition and function in cognitively healthy older adults over short intervals of 1 year (Farmer et al., 1995) or 2 to 2.5 years (e.g., Albert et al., 1995) using general measures of cognition such as the Mini-Mental State Exam; (MMSE; Folstein et al., 1975) and a composite created from several neuropsychological tests. In a prospective study of older adults, Christensen et al. (1997) found that education was inversely related to the rate of decline in measures of language and knowledge (i.e., crystallized ability), but not in measures of memory, processing speed, or reaction time. Likewise, Salthouse (2006) reported that in a cross-sectional study individuals in a high cognitive stimulation group (i.e., individuals who engaged in mentally stimulating activities) were not significantly different from a low cognitive stimulation group on measures of visual-spatial ability, memory, and processing speed. However, declines with age were different across the low and high cognitive stimulation groups for a vocabulary test such that scores decreased less for individuals in the high cognitive stimulation group. Together, these findings suggest that high cognitive reserve (as measured by education or leisure activity) may be protective against declines in measures of crystallized ability, but not protective against declines in other cognitive (fluid) domains. This finding is not surprising considering that vocabulary, for example, is sometimes used as a measure of cognitive reserve.
The validity of referring to cognitive reserve as a coherent construct has not been tested. In order to continue to refer to cognitive reserve as an independent construct it is important to establish that the variables typically used as measures of cognitive reserve are correlated with one another, and that they represent a unique dimension of individual differences that are distinct from other cognitive constructs.
Both convergent validity and discriminant validity are essential to establishing construct validity. Convergent validity can be evaluated by examining whether the variables hypothesized to represent a latent construct have significant variance in common, as determined by assessing the magnitudes of the factor loadings in a confirmatory factor analysis. The loadings of the observed cognitive reserve variables on the latent construct (i.e., cognitive reserve) should be moderately large in magnitude, and significantly different from zero. Discriminant validity is evaluated by examining whether the construct of interest is distinct from other non-target constructs, as indicated by the inter-factor correlations, which should be significantly less than 1.0. Using an individual differences approach, we evaluated the convergent and discriminant validity of the cognitive reserve construct with a set of analytic models designed to assess construct validity (Salthouse et al., 2003; Salthouse et al., 2004; Siedlecki et al., 2005).
In this project, the construct validity of the cognitive reserve construct was evaluated with four structural equation models described in detail in Salthouse et al. (2003). The models represent progressively more demanding tests of construct validity. Model A assesses only convergent validity by evaluating whether the cognitive reserve variables have significant variance in common. Model B assesses discriminant validity by examining the correlations among the cognitive reserve construct and non-target cognitive constructs (represented by verbal memory, processing speed, and executive functioning/fluid ability in the current study). Model C extends Model B by allowing the non-target constructs to be related to each of the cognitive reserve variables, one at a time. If the relationship between a non-target construct and an observed cognitive reserve variable results in a significantly improved fit of the overall model (as determined by ΔX2/Δ df) then that relationship is retained in the final Model C. Model D is the most demanding test of construct validity because each of the non-target constructs is allowed to be related to each of the observed cognitive reserve variables simultaneously. Convergent validity is assessed in Model D by examining whether there is significant residual variance among the cognitive reserve variables, after allowing them to be related to the non-target cognitive constructs. Discriminant validity is assessed by examining the magnitude of the relations among the cognitive reserve variables and the non-target constructs.
It should be noted that Salthouse et al. (2003) found that after examining constructs hypothesized to represent different aspects of executive functioning with the four construct validity models, there was only weak evidence that these constructs represented distinct dimensions of individual differences. An executive functioning construct comprised of a set of neuropsychological variables was found to lack convergent and discriminant validity. Specifically, the executive functioning variables were highly related to measures of fluid ability. In the current project, executive functioning is included as one of the non-target constructs (in Samples 1 and 2). In terms of individual differences we consider executive functioning and fluid ability to be highly related to one another, representing a common dimension of individual differences but because they are different both practically and semantically, we will continue to refer to them as separate constructs. Also, the source population for Samples 1 and 2 is one for which executive function measures have been helpful in discriminating those with vascular cognitive dysfunction (Wright et al., 2008).
The purpose of this project was to evaluate the construct validity of cognitive reserve using structural equation modeling (SEM) in a population-based clinically stroke-free sample of cognitively-healthy older adults and to examine the generalizability of the findings by evaluating the construct validity of the cognitive reserve in two additional samples of healthy adults.
Method
Participants
Three samples were selected for the current project: the primary sample and two replication samples. All participants had to be cognitively-normal at baseline as measured by a Clinical Dementia Rating of zero. The data included in this manuscript were obtained in compliance with regulations of the local Institutional Review Board.
Participants in Sample 1 and 2 were part of an MRI substudy recruited from the larger Northern Manhattan Study (NOMAS), a prospective, population-based study designed to study stroke incidence, risk factors, and prognosis in a sample from a multi-ethnic, urban community. Details regarding recruitment into NOMAS and the substudy have been documented elsewhere (Sacco et al., 2001; Prabhakaran et al. 2008). Sample 1 participants were native English speakers and Sample 2 participants were native Spanish speakers. Demographic details of the primary sample and the two replication samples are presented in Table 1. Sample 2 had significantly lower mean education, age, and MMSE scores than the participants in the primary sample.
Table 1.
Demographic Characteristics of the Three Samples
| Demographics | ||||||
|---|---|---|---|---|---|---|
| Sample 1 | Sample 2 | Sample 3 | ||||
| Mean (SD) | Range | Mean (SD) | Range | Mean (SD) | Range | |
| N | 365 | 431 | 136 | |||
| Age | 73.82 (8.56) | 53 – 95 | 69.41 (7.16) | 54 – 97 | 38.27 (19.62) | 20 – 81 |
| Educ(years) | 13.47 (3.63) | 1 – 20 | 7.40 (4.45) | 0 – 20 | 16.35 (1.87) | 12 – 22 |
| MMSE | 28.07 (2.22) | 18 – 30 | 26.13 (3.44) | 11 – 30 | _ | _ |
| % Female | 55.30% | 61.5% | 49.30% | |||
|
| ||||||
| Ethnicity | ||||||
| White | 143 (39.2%) | 1 (0.2%) | 78 (57.4%) | |||
| Black | 168 (46.0%) | 0 (0.0%) | 23 (16.9%) | |||
| Other | 25 (6.8%) | 0 (0.0%) | 24 (17.6%) | |||
| Hispanic | 29 (7.9%) | 430 (99.8%) | 11 (8.1%) | |||
Sample 3 was comprised of individuals who were not NOMAS subjects but had participated in other brain imaging studies at Columbia University Medical Center and had completed tests of cognitive reserve. The individuals in Sample 3 were significantly younger than those in either the primary sample or Sample 2.
Neuropsychological Evaluation for Samples 1 and 2
Participants in Sample 1 and Sample 2 were given identical neuropsychological evaluations with the exception of the Wide Range Achievement Test (WRAT; Wilkinson, 1993) which was only administered to the primary sample and Word Accentuation Test (WAT; Del Ser et al., 1997) which was only administered to Sample 2. The language of administration was different between the two groups; English in Sample 1, Spanish in Sample 2.
Memory
Memory was assessed with a modified California Verbal Learning Test-II (CVLT; Delis et al., 2000) in which participants heard a pre-recorded list of 12 unrelated nouns across five trials and were asked to recall the words at the end of each trial. Participants were scored on the total number of words remembered across the five trials (CVLT-total), the total number of words recalled after a 20 minute delay (CVLT-recall), and the total number of correctly recognized words in a recognition test comprised of 12 words and 12 foils (CVLT-recognition).
Processing speed
Processing speed was assessed with the Grooved Pegboard task (Matthews & Klove, 1964) that measures the total time it takes the participant to place 25 small metal grooved pegs into an array of holes with different orientations, requiring the subject to rotate the peg correctly in order to insert it into the pegboard, using both dominant and non-dominant hands.
Executive functioning
Executive functioning was assessed with the Wechsler Adult Intelligence Scale – III (WAIS-III; Wechsler, 1997) Letter Number Sequencing subtest, the Odd-Man-Out task (Flowers & Robertson, 1985), and a difference score calculated from the Color Trails Test (D’Elia et al., 1996).
In the Letter Number subtest1 participants are presented verbally with strings of intermixed letters and numbers (that become increasingly longer on a trial basis) and are asked to repeat the items back in a specific sequence.
The Odd-Man-Out task consists of four sets of subtests and involves the participant selecting which item in a set of three does not belong with the other items. Scores of each of the subtests were summed to create an Odd-Man-Out total score.
The Color Trails test (CTT) requires participants to connect numbers (Form A) or numbers alternating in color (Form B) in numerical order as quickly as possible. The time it takes the participants to complete Trails B minus the time to complete Trails A (Trials diff) is thought to be indicative of cognitive flexibility (ability to switch between sets) irrespective of general speed or ability to perceive and respond to stimuli and, as such, was used as the measure in this study.
Cognitive Reserve
Three variables were hypothesized to represent the cognitive reserve construct- education (as defined in terms of level of educational attainment in which 20 is the highest level, representing a doctoral degree), performance on the Peabody Picture Vocabulary Test-third edition (PPVT; Dunn & Dunn, 1997) and performance on the reading subtest of either the WRAT or the WAT. Each of these variables presumably reflects life experiences, above and beyond that of age, that have the potential to provide protection against clinical manifestation of disease in the brain, and has been used in the literature as a proxy for cognitive reserve.
The PPVT is comprised of 204 black-and-white line drawings of items that participants are required to name. We used the total number of correct responses as the measure. The WRAT reading subtest (completed by the participants in Sample 1) is comprised of 55 words that the participant must read out loud to the examiner. The WAT (completed by the participants in Sample 2) is comprised of 30 infrequent Spanish words written without an accentuation mark. The participants are instructed to read the words out loud. The total number pronounced correctly was used as the measure in both tests.
Neuropsychological Evaluation for Sample 3
The three non-target constructs included in these analyses were speed, memory and fluid ability (Gf). The speed and memory constructs in this sample were comprised of different, albeit similar, variables as compared to Sample 1 and 2.
Memory
Memory was assessed with three subscores of the Selective Reminding Test (SRT; Buschke & Fuld, 1974) in which participants were read a list of 12 words and asked to recall the words after each of six trials. After each recall attempt, participants were reminded of the words they failed to recall. SRT-total refers to the total number of words out of a possible 72 that the participant remembered. SRT-delayed refers to the number correct that the participant recalled after a 15-minute delay. SRT-recognition is the total correctly recognized words in a recognition test in which each of the 12 words are presented with three distracters.
Processing Speed
Processing speed was assessed with the WAIS-R (Wechsler, 1981) Digit Symbol subtest and the Trail Making test (Reitan & Wolfson, 1993). The Digit Symbol test involves writing symbols that correspond with single digit numbers, as provided by a key at the top of the test form, as quickly as possible. The time to complete Trails A (numbers only) was also included as a speed measure.
Fluid Ability
The construct of fluid ability (Gf) was comprised of the WAIS-III Letter Number Sequencing subtest (described above), as well as the Wisconsin Card Sorting Test (WCST; Heaton et al., 1993) and the Matrix Reasoning test (Raven, 1962). In the WCST participants must sort cards into one of four piles based on one dimension of the cards (e.g., color, number, etc.). After sorting the cards correctly for 10 trials, the rule for sorting changes without the subject being informed, and the rule may change as many as six times. The total number of correct responses was the measure used. The Matrix Reasoning task involves determining which pattern out of a set of eight possible patterns best completes the missing cell of a matrix.
Cognitive Reserve
Cognitive reserve was assessed with the National Adult Reading Test (NART; Nelson, 1982), WAIS-R Vocabulary subtest (Wechsler, 1981), and years of education. The NART requires participants to pronounce words that do not follow normal grapheme and phoneme rules (e.g., superfluous, epitome). The Vocabulary subtest requires participants to define words out loud to the examiner. Education was defined as the total number of years of formal, classroom education.
Modeling procedure
Structural equation modeling (SEM) was used to examine the construct validity of the cognitive reserve construct. The fit of the models were evaluated with several fit statistics. These fit statistics include the chi-square (X2), the critical ratio (X2/df), and the root mean square error of approximation (RMSEA), for which numbers closer to zero indicate a better fit, and Bentler’s comparative fit index (CFI) for which values closer to 1.0 indicate a better fit (Hu & Bentler, 1999).
Full-information maximum likelihood estimation was used to deal with missing data. Amos 5.0 (Arbuckle, 2003) was used for all SEM analyses.
Results
Means and standard deviations of each variable for the primary sample and Sample 2 are presented in Table 2. Inspection of the table indicates that each of the memory and executive functioning variables were significantly negatively correlated with age. The two speed variables were significantly positively correlated with age and given that smaller values indicate faster performance, increased age was associated with slower processing speed. The relationship between the cognitive reserve variables and age was mixed. Education was significantly negatively correlated with age in both samples, whereas the WRAT/WAT was not significantly related to age in either sample. The PPVT was negatively correlated with age in Sample 2 only. Participants in Sample 2 performed significantly worse than participants in Sample 1 on all measures except the speed measures (as determined with independent samples t-tests, p value <.01 for each significant comparison).
Table 2.
Means (and Standard Deviations) of Cognitive Variables for Samples 1 and 2
| Variable | Mean (SD) | Range | Age r | Mean (SD) | Range | Age r |
|---|---|---|---|---|---|---|
| Sample 1 | Sample 2 | |||||
| Memory | ||||||
| CVLT-total | 30.40 (8.62) | 8–55 | −.38*** | 28.38 (6.26) | 11–49 | −.23*** |
| CVLT-recall | 6.36 (2.71) | 0–12 | −.31*** | 5.83 (2.26) | 0–12 | −.17** |
| CVLT-recognition | 10.94 (1.51) | 4–12 | −.30*** | 10.37 (1.63) | 4–12 | −.15** |
| Processing Speed | ||||||
| Grooved Pegboard, dom | 108.24 (27.07) | 62–150 | .50*** | 109.41 (25.78) | 60–150 | .43*** |
| Grooved Pegboard, non-dom | 115.23 (25.85) | 64–150 | .45*** | 113.46 (24.73) | 70–150 | .38*** |
| Executive Function | ||||||
| Odd-Man-Out total | 31.04 (6.55) | 8–40 | −.32*** | 26.32 (6.11) | 7–40 | −.16** |
| Trails difference | 80.70 (46.32) | −6 – 235 | .21*** | 108.54 (52.05) | −41–245 | .21*** |
| Letter Number | 8.12 (3.21) | 0–19 | −.29*** | 5.77 (2.25) | 2–13 | −.26** |
| Cognitive Reserve | ||||||
| PPVT | 175 (23.22) | 84–203 | −.11 | 98.99 (22.39) | 14–200 | −.16** |
| Education | 13.47 (3.63) | 1 – 20 | −.14** | 7.40(4.46) | 0–20 | −.15** |
| WRAT/WAT | 44.41 (6.23) | 21–55 | −.02 | 13.37 (6.48) | 0–28 | .01 |
Note. Age r = age correlation; CVLT= California Verbal Learning Test; PPVT= Peabody Picture Vocabulary Test; WRAT= Wide Range Achievement Test; WAT= Word Accentuation Test;
p <.05.
p <.01.
p <.001.
Prior to evaluating the construct validity of the cognitive reserve construct, a three-factor model comprised of the non-target cognitive constructs was examined, in the primary sample. This three-factor model fit the data well (X2= 44.20, df = 17; X2/df = 2.60; CFI = .98; RMSEA = .07) and inspection of Table 3 indicates that the loadings of each variable on its respective construct was moderate to large, and significantly different from zero.
Table 3.
Standardized Coefficients from the Confirmatory Factor Analysis of the Three-Factor Model of Non-Target Constructs
| Factor |
|||
|---|---|---|---|
| Variable | Memory | Speed | Executive Function |
| CVLT-total | 0.91 | ||
| CVLT-recall | 0.87 | ||
| CVLT-recognition | 0.57 | ||
| Grooved Pegboard, dom | 0.94 | ||
| Grooved Pegboard, non-dom | 0.88 | ||
| Odd-Man-Out total | 0.69 | ||
| Trails difference | −0.63 | ||
| Letter Number | 0.74 | ||
| Factor Correlation | |||
| Memory | - | ||
| Speed | −0.48 | - | |
| Executive Function | 0.71 | −0.63 | - |
Note. All values are significantly different from zero at the p <.001 level; CVLT= California Verbal Learning Test.
Construct Validity
Initial construct validity analysis of the cognitive reserve construct consisted of examining a simple one-factor model comprised of the three hypothesized target variables and the latent construct (i.e., Model A). The results of this model are summarized in the top half of column A in Table 4. The overall fit of this model cannot be ascertained because there were no degrees of freedom, but the three variables hypothesized to represent cognitive reserve all had significant loadings on the latent construct. This finding suggests that the variables hypothesized to represent cognitive reserve have convergent validity.
Table 4.
Statistics for Construct Validity Models for the Variables Hypothesized to Reflect Cognitive Reserve in the Primary Sample
| Model |
||||
|---|---|---|---|---|
| Variable | A | B | C | D |
| Fit statistics | ||||
| X2 | 0 | 81.35 | 73.34 | 72.558 |
| df | 0 | 38 | 35 | 32 |
| X2/df | 2.14 | 2.10 | 2.27 | |
| CFI | 1 | 0.98 | 0.98 | 0.98 |
| RMSEA | 0.056 | 0.055 | 0.059 | |
|
| ||||
| Loadings on CR construct | ||||
| CR -> PPVT | .92*** | .93*** | .92*** | .55*** |
| CR -> Education | .59*** | .59*** | .48*** | .33*** |
| CR -> WRAT | .88*** | .87*** | .92*** | .56*** |
|
| ||||
| Correlations with other constructs | ||||
| CR <-> Memory | .45*** | .46*** | ||
| CR <-> Speed | −.34*** | −.37*** | ||
| CR <-> Exec Function | .77*** | .78*** | ||
|
| ||||
| Memory | Speed | Exec Function | ||
|
|
||||
| Loadings on other constructs | ||||
| Model C | ||||
| PPVT | ||||
| Education | .14 | |||
| WRAT | .08 | −.01 | ||
| Model D | ||||
| PPVT | −.15 | .17* | .93*** | |
| Education | −.02 | .05 | .54*** | |
| WRAT | −.15 | .25** | .91*** | |
Note. CFI= Comparative Fit Index; RMSEA= Root Mean Square Error of Approximation; CR = Cognitive Reserve; PPVT= Peabody Picture Vocabulary Test; WRAT= Wide Range Achievement Test.
p <.05.
p <.01.
p <.001. In the top part of the table, each column corresponds to a Model (A– D). Values of goodness-of-fit indices are reported in the first five rows. The values in the second set of rows correspond to the loadings of each target variable on the cognitive reserve construct (when a loading is significant it means that the value is significantly greater than zero). The values in the third set of rows correspond to the inter-factor correlations between the cognitive reserve construct and each of the non-target constructs. In the bottom part of the table each column corresponds to each of the of non-target constructs included in the analyses. The values reported in the rows are the loadings from each of the cognitive reserve variables to the non-target constructs. For Model C these are the loadings that significantly improved the fit of the model when included in the model. In Model D each of the cognitive reserve variables loads simultaneously on each of the non-target constructs (as well as on the cognitive reserve construct).
In Model B the cognitive reserve construct is examined in the context of the three non-target constructs (memory, speed, and executive functioning) to evaluate discriminant validity. This model, depicted in Figure 1, fit well (see column B of Table 4) and inspection of the estimated correlation coefficients suggest that although the cognitive reserve construct was only moderately correlated with memory and speed, it was fairly highly correlated with executive functioning (r = .77). However, the 95% confidence interval (.69–.84) did not include 1.0, indicating the correlation was significantly less than 1.0.
Figure 1.
Four-factor Model B of the construct validity model for Sample 1 and Sample 2. Double-headed arrows connecting the latent variables (represented by circles) represent correlations among the constructs. The observed variables are represented by rectangles. The single-headed arrows from the latent ability constructs to the observed variables represent the loading of each task on the latent variable. The latent variables labeled “e” represent the unique and error variance associated with each observed variable.
In Model C, each of the non-target constructs is allowed to be related to each of the target cognitive reserve variables one at a time. If including the path in the model significantly improves the fit of the model (as determined by ΔX2/Δ df) then that relation is retained in the final Model C. Three additional loadings were retained in Model C, as compared to Model B. As can be seen in the bottom half of Table 4, the WRAT variable loaded on the speed and executive functioning constructs and the education variable also loaded on the executive functioning construct. Although the cognitive reserve variables were allowed to be related to other constructs, the three variables still loaded significantly on the cognitive reserve construct providing additional evidence of convergent validity. In addition, the correlations among the cognitive reserve construct and the non-target variables were still moderate, or in the case of executive functioning, large but still significantly less than 1.0.
Model D is the most demanding test of both convergent and discriminant validity. In Model D the correlations among the cognitive reserve construct and the non-target constructs are no longer estimated. Instead, in this model, each of the non-target constructs is allowed to be related to each of the observed cognitive reserve variables simultaneously. Inspection of column D in Table 4 indicates that although the loadings of the observed cognitive reserve variables on the cognitive reserve construct are still significant, the cognitive reserve variables also all load significantly on the executive functioning construct. In addition, the PPVT and WRAT load significantly on the speed construct as well.
Construct validity: Sample 2
All the same tasks were completed in Sample 2 as in the primary sample, except in this sample the participants completed the WAT in Spanish (instead of the WRAT). The results of the construct validity analyses are presented in Table 5.
Table 5.
Statistics for Construct Validity Models for the Variables Hypothesized to reflect Cognitive Reserve in Sample 2
| Model |
||||
|---|---|---|---|---|
| Variable | A | B | C | D |
| Fit statistics | ||||
| X2 | 0 | 64.17 | 50.519 | 48.161 |
| df | 0 | 38 | 34 | 32 |
| X2/df | 1.69 | 1.49 | 1.51 | |
| CFI | 1 | 0.98 | 0.99 | 0.99 |
| RMSEA | 0.040 | 0.034 | 0.034 | |
|
| ||||
| Loadings on CR construct | ||||
| CR -> PPVT | 0.81*** | .86*** | .92*** | .37*** |
| CR -> Education | 0.69*** | .70*** | .70*** | .29*** |
| CR -> WAT | 0.88*** | .83*** | 1.49*** | .62*** |
|
| ||||
| Correlations with other constructs | ||||
| CR <-> Memory | .51*** | .51*** | ||
| CR <-> Speed | −.54*** | −.50*** | ||
| CR <-> Exec Function | .84*** | .90*** | ||
|
| ||||
| Memory | Speed | Exec Function | ||
|
|
||||
| Loadings on other constructs | ||||
| Model C | ||||
| PPVT | −.17 | −.77 | ||
| Education | ||||
| WAT | −.14 | −.21 | ||
| Model D | ||||
| PPVT | .12 | −.05 | .63*** | |
| Education | .01 | .09 | .70*** | |
| WAT | .11 | .05 | .64*** | |
Note. CFI= Comparative Fit Index; RMSEA= Root Mean Square Error of Approximation; CR = Cognitive Reserve; PPVT= Peabody Picture Vocabulary Test; WAT= Word Accentuation Test.
p <.05.
p <.01.
p <.001. See Table 4 note for a detailed description of what the values in each row and column represent.
In Model A the loadings on the cognitive reserve construct are all significant and large, suggesting that the variables share a large amount of common variance. Model B fits the data well, despite the large correlation (r = .84), between the cognitive reserve and the executive functioning construct. Model B provides some evidence of discriminant validity because although the correlation with executive functioning is very large it is significantly less than 1.0. In Model C the inclusion of the paths to PPVT from the speed and executive functioning constructs and the paths to the WAT variable also from the speed and executive functioning constructs significantly improved the fit of the model. With the inclusion of these four additional relations in the model, there is still significant common variance among the observed cognitive reserve variables. In Model C the 95% confidence interval around the correlation estimate (r = .90) between the cognitive reserve and executive functioning constructs includes 1.0. Evaluation of Model D indicates that each of the observed cognitive reserve variables load significantly on the cognitive reserve construct, and they also all load significantly on the executive functioning construct. The loadings on the executive functioning construct are all the same general magnitude, or larger, than the loadings of the observed cognitive reserve variables on the cognitive reserve construct.
Construct Validity: Sample 3
As described in the Method section, the tasks completed by the participants in Sample 3 were not identical to those tasks completed by the participants in the primary sample and in Sample 2. Means and standard deviations for each variable are presented in Table 6.
Table 6.
Means (and Standard Deviations) of Measures Obtained from Sample 3
| Variable | Mean (SD) | Range | Age r |
|---|---|---|---|
| Memory | |||
| SRT-total | 57.91 (7.84) | 31–72 | −.48*** |
| SRT-delayed recall | 10.04 (1.88) | 4–12 | −.33*** |
| SRT-recognition | 11.82 (1.06) | 1–12 | −.01 |
| Processing Speed | |||
| Digit Symbol | 63.20 (14.29) | 0–93 | −.39*** |
| Trails A | 29.02 (12.33) | 1–87 | .30** |
| Gf | |||
| Letter Number | 13.94 (5.58) | 5–59 | −.25** |
| WCST | 49.84 (8.92) | 28–66 | −.35** |
| Matrix Reasoning | 19.72 (6.97) | 5–71 | −.39*** |
| Cognitive Reserve | |||
| Vocabulary | 56.22 (9.79) | 21–70 | −.02 |
| NART | 118.13 (13.45) | 9–148 | .23* |
| Education | 16.35 (1.87) | 12–22 | .19* |
Note. Age r = age correlation; NART= National Adult Reading Test; SRT= Selective Reminding Test; WCST= Wisconsin Card Sorting Test.
p <.05.
p <.01.
p <.001.
Evaluation of Model A indicates that the three observed cognitive reserve variables have significant variance in common (see Table 7). Model B fits the data very well and the correlation between the cognitive reserve construct and the non-target constructs are only moderate in magnitude, providing evidence for discriminant validity. Model C was identical to Model B since no path between the observed cognitive reserve variables and the non-target constructs significantly improved the fit of the model. Model D also fit the data well and most striking in this sample is that none of the loadings of the observed cognitive reserve variables on the non-target constructs were significant. In addition, there was substantial common residual variance among the three target variables in Model D.
Table 7.
Statistics for Construct Validity Models for the Variables Hypothesized to Reflect Cognitive Reserve in Sample 3
| Model |
||||
|---|---|---|---|---|
| Variable | A | Ba | C | D |
| Fit statistics | ||||
| X2 | 0.00 | 37.12 | 37.12 | 32.584 |
| df | 0 | 39 | 39 | 33 |
| X2/df | 0.95 | 0.95 | 0.99 | |
| CFI | 1.00 | 1.00 | 1.00 | 1 |
| RMSEA | 0.154 | 0.000 | 0.000 | 0.000 |
|
| ||||
| Loadings on CR construct | ||||
| CR -> Vocab | .89*** | 1.00*** | 1.00*** | .91*** |
| CR -> Educ | .38** | .34*** | .34*** | .30*** |
| CR -> NART | .44** | .40*** | .40*** | .48*** |
|
| ||||
| Correlations with other constructs | ||||
| CR <-> Memory | .33*** | .33*** | ||
| CR <-> Speed | −.35** | −.35** | ||
| CR <-> Gf | .34*** | .34*** | ||
|
| ||||
| Memory | Speed | Gf | ||
|
|
||||
| Loadings on other constructs | ||||
| Model C | ||||
| Vocab | ||||
| Educ | ||||
| NART | ||||
| Model D | ||||
| Vocab | .21 | −.09 | .19 | |
| Educ | .14 | .16 | .19 | |
| NART | .01 | .10 | .13 | |
Note. CFI= Comparative Fit Index; RMSEA= Root Mean Square Error of Approximation; CR = Cognitive Reserve; NART= National Adult Reading Test.
p >.05.
p <.01.
p <.001. See Table 4 note for a detailed description of what the values in each row and column represent.
Discussion
Cognitive reserve is a theoretical construct that has been used to inform models of cognitive aging and is presumed to be indicative of life experiences that may help in the management of brain pathology. The purpose of this project was to evaluate the appropriateness of referring to cognitive reserve as reflecting a distinct dimension of individual differences.
Analyses in the current study indicated that the variables hypothesized to reflect cognitive reserve have moderate to large loadings on the latent construct, providing evidence of convergent validity across three samples. The finding that the cognitive reserve variables are correlated with one another is not surprising when one considers that most cognitive variables are correlated with one another, a finding first reported by Spearman (1904). To further establish construct validity it is therefore necessary to also establish discriminant validity: that is, that the construct hypothesized to represent a unique dimension of individual differences (i.e., cognitive reserve) is only moderately correlated with other non-target constructs. We examined discriminant validity across three progressively more demanding models. Because the models are increasingly stringent in their test of discriminant validity it is possible to find evidence of discriminant validity in Models B and C, but not Model D, for example. Ideally, the magnitude of the loadings on the cognitive reserve construct from the observed variables should be the same or larger than the correlations among the target and non-target constructs. Across all three samples we found that this was generally the case; the magnitude of the cognitive reserve loadings was similar or larger than the correlation between the cognitive reserve construct and the non-target constructs (see Models B and C). These findings from Models B and C provide evidence of discriminant validity.
However, examination of the results of Model D suggest that, in Samples 1 and 2, the cognitive reserve variables are highly related to the executive functioning variables. Thus the findings reported for Model D in Samples 1 and 2 fail to provide support for discriminant validity of the cognitive reserve construct. Furthermore, in Sample 2 the correlation estimate between the cognitive reserve and executive functioning construct was not significantly less than 1.0 suggesting that the two constructs were not distinct from one another, at least in terms of individual differences. However, there was still significant positive residual variance among the cognitive reserve variables in Model D, which provides additional evidence for convergent validity. Stereotypical executive functioning tasks typically capture the ability to use flexible strategies to solve problems. The construct of cognitive reserve presumably reflects the cumulative effect of life experiences to provide individuals with a set of skills to better manage pathological damage through the use of alternative or compensatory strategies. Some aspects of executive function, such as flexible cognition, may even be considered part of cognitive reserve such that it may influence life experiences. It is therefore not surprising that these two constructs would be highly related to one another (in Samples 1 and 2). Furthermore, previous work by Salthouse et al. (2003) has suggested that the construct of executive functioning may not represent a unique dimension of individual differences. It therefore may not be realistic to expect the cognitive reserve variables to demonstrate evidence of discriminant validity in Model D (the most stringent test) from a set of variables that have been shown previously to lack discriminant validity.
This was not the case for Model D in Sample 3; evaluation of these results indicate that there was no significant overlap between the cognitive reserve and Gf constructs (the Gf construct being similar to the executive functioning construct examined in the other samples). In this sample, discriminant validity of the cognitive reserve construct was strongly supported. The cognitive reserve variables were not significantly related to any of the non-target constructs, and the cognitive reserve variables still had substantial and significant residual variance in common after being allowed to be related to the non-target constructs.
To summarize, results of Models A through C provided evidence of both convergent and discriminant validity of cognitive reserve across all three samples. Results from Model D, the most demanding test of construct validity, failed to provide evidence for discriminant validity for Samples 1 and 2 but did provide additional evidence of discriminant validity in Sample 3.
The cognitive reserve constructs in this study were fairly narrowly defined (for example, occupation was not included in the construct) and therefore, in many ways, the cognitive reserve construct was akin to what cognitive psychologists refer to as crystallized ability (Gc; Cattell, 1987). It is well-established that Gc represents a unique dimension of individual differences and researchers often partition intelligence (g) into Gc and Gf components (e.g., Cattell, 1987; Horn, 1982). Consequently, the finding of strong construct validity in a model in which there are constructs closely related to both Gc and Gf (Sample 3) is not surprising. However, although some of the cognitive reserve tasks (used in the current project, and also in the literature at large) are often considered measures of semantic knowledge (i.e., the PPVT) the theoretical concept of reserve refers to variables that are reflective of lifetime experiences that promote cognitive flexibility (Stern, in press). Each of the variables in this project presumably provides an indication of these lifetime experiences.
Collectively, these findings suggest that it may be reasonable to refer to cognitive reserve as a distinct construct (based on strong convergent validity and moderate discriminant validity) as long as it is acknowledged that the cognitive reserve variables are highly related to executive functioning. Another possible consequence of these findings is that the concept of cognitive reserve should be reconsidered to include measures of executive functioning. However, because prior research has indicated that executive functioning does not likely reflect a distinct dimension of individual differences (Salthouse et al., 2003) it is probably unrealistic to expect strong evidence of construct validity when an executive construct is included in the models. In fact, the results of the analyses in Sample 3 suggest that without the inclusion of an executive functioning construct the cognitive reserve construct demonstrates strong discriminant validity, even in Model D.
For the most part, the same general pattern was found across the primary sample (which consisted of cognitively-healthy community dwelling English-speakers with an average age of 74 years and an average education of 13.5 years), Sample 2 (which consisted of Spanish speakers who were significantly younger and less educated than the primary sample), and Sample 3 (who were much younger and more educated than the other 2 samples). Our findings suggest that the main results of our study can be replicated across samples that differ along age, language, and education levels. This suggests that the findings presented here are fairly generalizable, at least in terms of healthy, non-demented, adults.
Supplementary Material
Acknowledgments
We would like to thank the staff of the Northern Manhattan Study (NOMAS), especially Yeseon Moon for compiling the data set and Janet DeRosa, the project manager. This work is supported by the NINDS (R01 NS 29993; K02 NS059729). We acknowledge the support of the NIMH (KLS, T32MH020004-09) and the Evelyn F. McKnight Brain Research Foundation (CBW, RLS).
Footnotes
Note: This manuscript has been accepted for publication in the Journal of the International Neuropsychological Society. Copyright belongs to Cambridge University Press, http://journals.cambridge.org. This article may not exactly replicate the final version published.
Not all of the participants completed the Letter Number subtest because it was removed from the test battery. Fifty-one percent of the English speakers and 33.2% of the Spanish speakers completed the task. Maximum-likelihood estimation was used to deal with the missing data in the SEM analyses.
References
- Albert MS, Jones K, Savage CR, Berkman L, Seeman T, Blazer D, Rowe JW. Predictors of cognitive change in older persons: MacArthur studies of successful aging. Psychology and Aging. 1995;10:578–589. doi: 10.1037//0882-7974.10.4.578. [DOI] [PubMed] [Google Scholar]
- Arbuckle JL. Amos 5.0. Chicago, IL: SPSS; 2003. [Google Scholar]
- Buschke H, Fuld PA. Evaluating storage, retention, and retrieval in disordered memory and learning. Neurology. 1974;24:1019–1025. doi: 10.1212/wnl.24.11.1019. [DOI] [PubMed] [Google Scholar]
- Cattell RB. Intelligence: Its structure, growth and action. Amsterdam, The Netherlands: Elsevier; 1987. [Google Scholar]
- Christensen H. What cognitive changes can be expected with normal ageing? Australian and New Zealand Journal of Psychiatry. 2001;35:768–775. doi: 10.1046/j.1440-1614.2001.00966.x. [DOI] [PubMed] [Google Scholar]
- Christensen H, Korten AE, Jorm AF, Henderson AS, Jacomb PA, Rodgers B, MacKinnon AJ. Education and decline in cognitive performance: compensatory but not protective. International Journal of Geriatric Psychiatry. 1997;12:323–330. doi: 10.1002/(sici)1099-1166(199703)12:3<323::aid-gps492>3.0.co;2-n. [DOI] [PubMed] [Google Scholar]
- D’Elia LF, Satz P, Uchiyama CL, White T. Color Trails Test. Professional manual. Odessa, FL: Psychological Assessment Resources; 1996. [Google Scholar]
- Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test – second edition: Adult version. Manual. San Antonio, TX: Psychological Corporation; 2000. [Google Scholar]
- Del Ser T, González-Montalvo J, Martínez-Espinosa S, Delgado-Villapalos C, Bermejo F. Estimation of pre-morbid intelligence in Spanish people with the Word Accentuation Test and its application to the diagnosis of dementia. Brain and Cognition. 1997;33:343–356. doi: 10.1006/brcg.1997.0877. [DOI] [PubMed] [Google Scholar]
- Dunn LM, Dunn LM. Examiner’s manual for the Peabody Picture Vocabulary Test- Third Edition. Circle Pines, MN: American Guidance Service; 1997. [Google Scholar]
- Evans DA, Hebert LE, Beckett LA, Scherr PA, Albert MS, Chown MJ, Pilgrim DM, Taylor JO. Education and other measures of socioeconomic status and risk of incident Alzheimer disease in a defined population of older person. Archives of Neurology. 1997;54:1399–1405. doi: 10.1001/archneur.1997.00550230066019. [DOI] [PubMed] [Google Scholar]
- Farmer ME, Kittner SJ, Rae DS, Bartko JJ, Regier DA. Education and change in cognitive function: The epidemiologic catchment area study. Annals of Epidemiology. 1995;5:1–7. doi: 10.1016/1047-2797(94)00047-w. [DOI] [PubMed] [Google Scholar]
- Folstein M, Folstein S, McHugh P. ”Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- Flowers KA, Robertson C. The effect of Parkinson’s disease on the ability to maintain a mental set. Journal of Neurology, Neurosurgery, and Psychiatry. 1985;48:517–29. doi: 10.1136/jnnp.48.6.517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heaton RK, Chelune GJ, Talley JL, Kay GG, Curtiss G. Wisconsin Card Sorting Test manual – Revised and expanded. Lutz, Fl: Psychological Assessment Resource; 1993. [Google Scholar]
- Horn JL. The aging of human abilities. In: Wolman BB, editor. Handbook of developmental psychology. Englewood Cliffs, NJ: Prentice Hall; 1982. pp. 847–870. [Google Scholar]
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. [Google Scholar]
- Letenneur L, Commenges D, Dartigues JF, Barberger-Gateau P. Incidence of dementia and Alzheimer’s disease in elderly community residents of south-western France. International Journal of Epidemiology. 1994;23:1256–1261. doi: 10.1093/ije/23.6.1256. [DOI] [PubMed] [Google Scholar]
- Matthews CG, Klove H. Instruction Manual for the Adult Neuropsychology Test Battery. University of Wisconsin Medical School; Madison, WI: 1964. [Google Scholar]
- Nelson HE. National Adult Reading Test. Windsor, UK: NFER-Nelson; 1982. [Google Scholar]
- Prabhakaran S, Wright CB, Yoshita M, Delapaz R, Brown T, DeCarli C, Sacco RL. Prevalence and determinants of subclinical brain infarction: The Northern Manhattan Study. Neurology. 2008;70:425–430. doi: 10.1212/01.wnl.0000277521.66947.e5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Qiu CX, Backman L, Winblad B, Aguero-Torres H, Frtiglioni L. The influence of education on clinical diagnosed dementia: Incidence and mortality data from the Kungsholmen Project. Archives of Neurology. 2001;58:2034–2039. doi: 10.1001/archneur.58.12.2034. [DOI] [PubMed] [Google Scholar]
- Raven J. Advanced progressive matrices, Set II. London: H.K. Lewis; 1962. [Google Scholar]
- Reitan RM, Wolfson D. The Halstead–Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation. 2. Tuscon, AZ: Neuropsychology Press; 1993. [Google Scholar]
- Sacco RL, Boden-Albala B, Abel G, Lin IF, Elkind M, Hauser WA, Paik MC, Shea S. Race-ethnic disparities in the impact of stroke risk factors: The Northern Manhattan Stroke Study. Stroke. 2001;32:1725–1731. doi: 10.1161/01.str.32.8.1725. [DOI] [PubMed] [Google Scholar]
- Salthouse TA. What and when of cognitive aging. Current Directions in Psychological Science. 2004;13:140–144. doi: 10.1177/0963721414535212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Salthouse TA. Mental exercise and mental aging. Perspectives on Psychological Science. 2006;1:68–87. doi: 10.1111/j.1745-6916.2006.00005.x. [DOI] [PubMed] [Google Scholar]
- Salthouse TA, Atkinson TM, Berish DE. Executive functioning as a potential mediator of age-related cognitive decline in normal adults. Journal of Experimental Psychology: General. 2003;132:566–594. doi: 10.1037/0096-3445.132.4.566. [DOI] [PubMed] [Google Scholar]
- Salthouse TA, Berish DE, Siedlecki KL. Construct validity and age sensitivity of prospective memory. Memory & Cognition. 2004;32:1133–1148. doi: 10.3758/bf03196887. [DOI] [PubMed] [Google Scholar]
- Siedlecki KL, Salthouse TA, Berish DE. Is there anything special about the aging of source memory? Psychology and Aging. 2005;20:19–32. doi: 10.1037/0882-7974.20.1.19. [DOI] [PubMed] [Google Scholar]
- Spearman C. “General intelligence” objectively determined and measured. American Journal of Psychology. 1904;15:8–20. [Google Scholar]
- Stern Y. What is cognitive reserve? Theory and research application of the reserve concept. Journal of the International Neuropsychological Society. 2002;8:448–460. [PubMed] [Google Scholar]
- Stern Y. Cognitive reserve. Neuropsychologia in press. [Google Scholar]
- Stern Y, Tang MX, Denaro J, Mayeux R. Increased risk of mortality in Alzheimer’s disease patients with more advanced educational and occupational attainment. Annals of Neurology. 1995;37:590–595. doi: 10.1002/ana.410370508. [DOI] [PubMed] [Google Scholar]
- Stern Y, Gurland B, Tatemichi TK, Tang MX, Wilder D, Mayeux R. Influence of education and occupation on the incidence of Alzheimer’s disease. Journal of the American Medical Association. 1994;271:1004–1010. [PubMed] [Google Scholar]
- Valenzuela MJ, Sachdev P. Brain reserve and dementia: a systematic review. Psychological Medicine. 2005;35:1–14. doi: 10.1017/S0033291705006264. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler Adult Intelligence Scale- Revised. New York: The Psychological Corporation; 1981. [Google Scholar]
- Wechsler D. Wechsler Adult Intelligence Scale-Third Edition. San Antonio, TX: The Psychological Corporation; 1997. [Google Scholar]
- White L, Katzman R, Losonczy K, Salive M, Wallace R, Berkman L, Taylor J, Fillenbaum G, Havlik R. Association of education with incidence of cognitive impairment in three established populations for epidemiologic studies of the elderly. Journal of Clinical Epidemiology. 1994;47:363–74. doi: 10.1016/0895-4356(94)90157-0. [DOI] [PubMed] [Google Scholar]
- Wilkinson GS. The Wide Range Achievement Test: Manual. 3. Wilmington, DE: Wide Range; 1993. [Google Scholar]
- Wright CB, Festa JR, Paik MC, Schmiedigen A, Brown TR, Yoshita M, DeCarli C, Sacco R, Stern Y. White matter hyperintensities and subclinical infarction: associations with psychomotor speed and cognitive flexibility. Stroke. 2008;39:800–805. doi: 10.1161/STROKEAHA.107.484147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang M, Katzman R, Salmon D, Jin H, Cai G, Wang Z, Qu G, Grant I, Yu E, Levy P, Klauber MR, Liu WT. The prevalence of dementia and Alzheimer’s disease in Shanghai, China: Impact of age, gender and education. Annals of Neurology. 1990;27:428–437. doi: 10.1002/ana.410270412. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.