Within-person across-neuropsychological test variability and incident dementia

Abstract

Context

Neuropsychological tests are used to predict and diagnose dementia. However, no studies to date examined whether within-person across-neuropsychological test variability predicts dementia.

Objective

To examine whether within-person across-neuropsychological test variability predicted future dementia

Design

The Einstein Aging Study (EAS) is a population-based longitudinal study of aging and dementia located in the Bronx county, NY. Cox proportional hazards models using age as the time scale estimated hazard ratios for performance on individual neuropsychological tests (Free and Cued Selective Reminding Test, Digit Symbol, Vocabulary) and for within-person across-neuropsychological test variability as predictors of incident dementia. Analyses were stratified by gender, and controlled for education, and medical illness.

Setting and participants

A total of 1797 participants (age ≥70 years) enrolled in the EAS between October, 1993 and December, 2007. Participants seen for the baseline visit only (n=750), prevalent dementia cases (n=72) and those with missing information (n=78) were excluded. A total of 897 individuals were included in this investigation. Participants had follow-up visits every 12 to 18 months.

Main Outcome Measure(s)

Diagnosis of dementia

Results

Sixty one incident dementia cases were identified during follow-up (M=3.3±2.4 years). Within-person across-neuropsychological test variability predicted incident dementia (hazard ratio for one point difference in variability, 3.926, 95% CI, 2.040 - 7.558) even after adjusting for level of performance on individual neuropsychological tests (hazard ratio, 2.098, 95% CI, 1.041 - 4.225). The significant partial likelihood ratio test (p=0.04) comparing Cox models using neuropsychological tests with and without within-person across-neuropsychological test variability provided further evidence that the former improved the prediction of incident dementia. Finally, within-person across-neuropsychological test variability significantly increased the sensitivity for predicting dementia within one year compared to neuropsychological tests alone (McNemar’s test p=0.014).

Conclusions

Within-person across-neuropsychological test variability is a novel predictor of incident dementia.

Developing strategies to improve the prediction and diagnosis of dementia has paramount therapeutic and public health implications¹. Neuropsychological tests support the diagnosis of dementia^{2, 3},the transitional stages that precede it, such as mild cognitive impairment^{4, 5}, and the delineation of cognitive changes over time⁶. Level of performance on tests of memory^{7, 8} and executive function^{9, 10} has been reported to predict future dementia. Further, memory begins to decline more rapidly up to seven years prior to diagnosis of dementia^{11, 12}.

When neuropsychological tests are used for diagnostic purposes, an individual’s level of performance on specific tests is measured against healthy normative samples to determine cognitive impairment^{13, 14}. This approach, however, does not take into account intra-individual variability in cognitive function. The taxonomy of intra-individual variability, considered as inconsistency in cognitive performance within a person, includes the following definitions: A) Variability on the same cognitive task across multiple assessments over a long period of time. Such variability may be high in healthy individuals across the life span¹⁵. B) Variability on repeated trials of a single cognitive task administered on one occasion or over short periods of time. Cross-sectional studies show that this form of variability is increased in aging¹⁶ and dementia ^{17, 18} but has shown variable results in subjects with mild cognitive impairment^{19, 20}. C) Variability in performance across neuropsychological tests administered in a single session. Such variability is expected in the normal population^{21, 22} including non-demented older adults²³ reflecting the individual’s relative cognitive strengths and weaknesses. This study focused on within-person across-neuropsychological test variability as it can be estimated directly from standardized clinical neuropsychological procedures.

Increased within-person across-neuropsychological test variability has been related to poor cognitive function in normal older adults at cross section²³. Accordingly, we examined whether increased within-person across-neuropsychological test variability: a) predicted future dementia; b) provided incremental prediction after taking into account level of performance on individual neuropsychological tests; and c) improved sensitivity for the prediction of dementia.

Methods

Study population

Participants were enrolled in the Einstein Aging Study (EAS), a longitudinal study of aging and dementia located at the Albert Einstein College of Medicine in Bronx County, New York. The study design, recruitment and follow-up methods were previously described^{24, 25}. Briefly, EAS has used telephone-based screening procedures to recruit and follow a community-based cohort since 1993. The primary aim of the EAS is to identify risk factors for dementia. Eligibility criteria require that participants be 70 years of age and older, reside in Bronx, and speak English. Exclusion criteria include severe audiovisual disturbances that would interfere with completion of neuropsychological tests, inability to ambulate even with a walking aid or in a wheelchair, and institutionalization. Potential participants over age 70 from the Center for Medicaid Medicare Services population lists of Medicare eligible individuals were first contacted by letter, then by telephone, explaining the purpose nature of the study. The telephone interview included verbal consent, a brief medical history questionnaire, and telephone-based cognitive screening tests²⁴. Following the interview, an age-stratified sample of subjects who matched on a computerized randomization procedure was invited for further evaluation at the medical center. This procedure was implemented to ensure that the individuals included in the EAS represented the Bronx population at that age stratum; and that those who agreed to participate were not different from non-responders in terms of key demographic characteristics. Written informed consents were obtained at clinic visits according to study protocols and approved by the Committee on Clinical Investigation (CCI; the institutional review board of the Albert Einstein College of Medicine).

A total of 1797 participants were enrolled in the EAS between October, 1993 and December, 2007. Participants seen for the baseline visit only (n=750), prevalent dementia cases (n=72) and those with missing information (n=78) were excluded. Hence, a total of 897 participants were included in this investigation. Participants had follow-up visits every 12 to 18 months, at which they underwent detailed neurological and neuropsychological evaluations.

Algorithmic diagnosis of dementia

To address the issue of diagnostic circularity we derived algorithmic diagnosis of dementia that was independent of the neuropsychological tests performance. Two conditions had to be met. First, participants had to make 8 or more errors on the Blessed Information-Memory-Concentration test (BIMC, best score 0 errors, worst possible score, 32 errors)²⁶. This test has high test-retest reliability (0.86) and correlates well with the pathology of Alzheimer’s disease^{27, 28}. The Blessed test is used to screen for cognitive impairment on the baseline telephone and in-person interviews in the EAS, but the scores are not utilized when assigning dementia diagnosis in consensus case conferences²⁴. Second, impairments on basic activities on the Lawton-Brody activities of daily living²⁹ secondary to cognitive impairment had to be documented using previously described procedures³⁰. Incident dementia using the above algorithmic procedures was used as the main outcome measure in the primary analyses reported. There was 96 percent agreement between the algorithmic and clinical diagnosis of dementia in this cohort.

Clinical diagnosis of dementia

EAS participants suspected to have dementia received a complete diagnostic workup as previously described for this cohort³¹. Diagnoses of dementia were assigned according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV)³², at case conferences attended by at least one study neurologist, a neuropsychologist, and a geriatric nurse or social worker. Alzheimer’s disease was diagnosed according to the criteria for probable disease detailed by the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association³. The State of California Alzheimer’s Disease Diagnostic and Treatment Centers criteria was used to assign diagnoses of probable, possible, or mixed vascular dementia ³³. In subjects diagnosed with dementia, neuroimaging was used to help allocate the diagnosis of “probable” Alzheimer’s disease or “probable” vascular dementia. We have reported good agreement between clinical diagnoses of Alzheimer’s disease,³⁴ vascular dementia,³¹, and dementia with Lewy bodies,³⁵ and pathological findings in our study. For the purpose of this paper clinical diagnosis of dementia subtypes was used only in secondary analyses to explore whether associations with variability varied as a function of diseases processes. Hence only “pure” subtypes (Alzheimer’s and Vascular) but not mixed were considered.

Neuropsychological assessment

The tests included in this battery have been validated for use in the aging population in our and other aging studies^{8, 36, 37}. Our recent studies reveal that factor analysis of the neuropsychological test battery consistently yielded three empirically derived and statistically orthogonal cognitive domains including Verbal IQ, Attention/Executive function and Memory^{38, 39}. For the purpose of this study we identified three tests that represented the three cognitive domains mentioned above, and were available for all 897 participants included in this study. The Vocabulary (total score) subtest of the Wechsler Adult Intelligence Scale-revised (WAIS-R⁴⁰), considered a “hold” test in that it is not sensitive to the effect of aging and age-related diseases, represented the Verbal IQ domain. The Free and Cued Selective Reminding Test (FCSRT, free recall) is commonly used to assess verbal memory and is sensitive to dementia⁴¹. The WAIS-R Digit Symbol Substitution subtest⁴⁰ is commonly used to assess attention and executive function.

Estimate of within person variability across neuropsychological tests

The method used herein to estimate within person across tests variability was described in other studies as well ^{23, 42}. The distribution of scores on the FCSRT, Vocabulary, and Digit Symbol tests were carefully examined to ascertain that assumptions of normality were met. Second, the raw scores of each test were Z transformed on the basis of the distribution of the entire sample (n=897). Then, the Z transformed test scores were used to calculate within person variability across the three tests using this equation:$\text{Variability}=\sqrt{{(Z_{ik}-A_i)}^2∕(K-1)}$, where Z_ik is the k th cognitive test score for i th subject. Here, k = 1, ..., K, K =3 (FCSRT, Vocabulary, and Digit Symbol) and $A_i={\sum }_{k=1}^K{Z}_{ik}$ is the individual’s mean z transformed score based on the three tests.

Illness Index

Consistent with our previous studies ^{38, 39, 43} dichotomous rating (present or absent) of diabetes, chronic heart failure, arthritis, hypertension, depression, stroke, Parkinson’s disease, chronic obstructive lung disease, angina, and myocardial infarction was used to calculate an illness index summary score (range 0-10). Medical history was obtained from multiple sources including significant others and family physicians when available. Trained research assistants used structured clinical interview, and the study physician obtained medical history during the neurological examination independently of the structured clinical interview. Information gathered from all the above sources was carefully scrutinized prior to data entry.

Statistical Analyses

First, to examine whether level of performance on the neuropsychological tests at baseline predicted the development of dementia we used Cox proportional-hazards regression analysis to estimate hazard ratios with 95 percent confidence intervals. The second Cox analysis determined the effect of within-person across-neuropsychological test variability on the development of dementia. To assess whether within-person across-neuropsychological variability provided incremental prediction of incident dementia above and beyond what was predicted by the absolute level of performance on the neuropsychological tests a third Cox proportional-hazards regression analysis was conducted controlling for performance differences on Vocabulary, FCSRT and Digit Symbol. The partial likelihood test is specifically designed to compare the goodness of fit of nested Cox proportional hazard models⁴⁵. Herein, we compared a model with the three individual neuropsychological tests to a model that added within-person across-neuropsychological test variability using the partial likelihood ratio test to examine whether the latter improved significantly the prediction of dementia. Finally, using the McNemar’s test we examined whether within-person across-neuropsychological test variability significantly improved sensitivity for the prediction of new dementia cases within one year.

Age served as the time-scale because it is considered more appropriate than follow-up time in cohort studies⁴⁴. When age serves as the time-scale, the hazard function can be directly interpreted as the age-specific incidence function and age is accounted for in the non-parametric term of the hazard function providing a more flexible and effective control of age⁴⁴. Time to event was from age at the baseline assessment accounting for the left truncation at study inclusion to age at which diagnosis of dementia was ascertained or to final study contact for non-demented participants. In addition, all multivariate analyses were stratified by gender and controlled for education, and medical illness. For all primary analyses incident dementia was defined using the algorithmic procedures described earlier. Proportional hazards assumptions of the models were examined analytically and graphically and were adequately met.

Finally, Cox proportional-hazards regression analysis with age as the time scale were used to examine whether within person across neuropsychological tests variability predicted incident dementia subtypes (e.g. Vascular and Alzheimer’s dementia) defined by consensus diagnosis as previously described. Given the low number of incident cases for of Alzheimer’s dementia and Vascular dementia as well as concerns for potential diagnostic circularity these analyses were considered exploratory. All analyses were performed using SAS 9.1 (SAS Institute Inc., Cary, N.C) and S-Plus 8.0 (Insightful Corp.) The significance level was set at 0.05 and all tests were two-sided.

Results

Demographics

Participants were community residents who were relatively healthy and had normal global mental functioning based on BIMC scores that were within the normal range (Table 1).

Table 1.

Summary of sample characteristics, per quartile of within-person across-neuropsychological test variability at baseline

CHARACTERISTICS	Entire sample (N=897)	Quartile 1 variability (N=225)	Quartile 2 Variability (N=224)	Quartile 3 Variability (N= 224)	Quartile 4 Variability (N=224)
Age Mean (SD), y	78.6 (5.3)	78.3 (5.1)	78.8 (5.6)	78.4 (5.2)	79 (5.3)
Education Mean (SD), y	13.3 (3.6)	13.6 (3.2)	13.4 (3.3)	13.2 (3.8)	12.9 (3.9)
Women, No, (%)	538 (60)	134 (60)	146 (65)	129 (58)	129, (58)
Blessed: Mean (SD)	2.6 (2.5)	2.2 (2.1)	2.5 (2.5)	2.4 (2.2)	3.5 (2.9)
FCSRT: Mean (SD)	30.0 (6.8)	30.8 (4.9)	30.9 (5.7)	30.4 (5.9)	27.8 (9.3)
Vocabulary: Mean (SD)	46.9 (13.9)	49.9 (10.9)	48.8 (11.7)	47.4 (14.2)	42.6 (17.3)
DSYM: Mean (SD)	34.5 (11.4)	35.9 (8.6)	35.7 (10.5)	33.3 (11.7)	32.9 (14.1)
Variability: Mean (SD)	0.74 (0.40)	0.29 (0.11)	0.56 (0.07)	0.82 (0.09)	1.29 (0.26)
Incident dementia cases, No (%)	61 (7)	3 (1)	19 (8)	13 (6)	26 (12)
Illness Index: Mean (SD)	1.2 (1.0)	1.2 (1.0)	1.1 (1.0)	1.2 (1.0)	1.1 (1.1)
Depression, No, (%)	83 (9)	28 (12)	23 (10)	14 (6)	18 (8)
Angina, No, (%)	90 (10)	22 (10)	22 (10)	25 (11)	21 (9)
Arthritis, No, (%)	51 (6)	13 (6)	12 (5)	15 (7)	11 (5)
Diabetes, No, (%)	120 (13)	29 (13)	24 (11)	31 (14)	36 (16)
Chronic obstructive lung disease, No, (%)	35 (4)	11 (5)	9 (4)	8 (4)	7 (3)
Hypertension, No, (%)	464 (52)	114 (51)	117 (52)	121 (54)	112 (50)
Myocardial infarction, No, (%)	90 (10)	25 (11)	18 (8)	26 (12)	21 (9)
Chronic heart failure, No, (%)	12 (1)	2 (1)	4 (2)	4 (2)	2 (1)
Parkinson’s Disease, No, (%)	12 (1)	4 (2)	2 (1)	2 (1)	4 (2)
Stroke, No, (%)	86 (10)	22 (10)	17 (8)	25 (11)	22 (10)

Variability denotes within-person across-neuropsychological test variability

Increased variability denotes worse cognitive function

FCSRT: Free and Cued Selective Reminding Test

DSYM: Digit Symbol Substitution subtest of the WAIS-R

Vocabulary: Vocabulary subtest of the WAIS-R

Of the 897 participants, 61 (6.8%) incident dementia cases, defined by the algorithmic procedure described earlier, were identified during the follow-up period (M=3.3±2.4 years). On the basis of the consensus clinical diagnostic procedures 47 participants developed incident dementia of the Alzheimer’s type and 18 participants developed incident Vascular dementia.

Prediction of Dementia

All Cox proportional-hazards regression models used age as the time scale, were stratified by gender, and adjusted for education and medical illness. The first model revealed that higher scores on FCSRT (hazard ratio for one point difference on the test, 0.872, 95% CI, 0.840 - 0.906, p<0.001) and Digit Symbol Substitution (hazard ratio for one point difference on the test, 0.971, 95% CI, 0.943 - 0.999, p=0.04) predicted lower risk of developing dementia. Level of performance on the Vocabulary test did not predict future dementia (hazard ratio for one point difference on the test, 1.005, 95% CI, 0.981 - 1.030, p=0.67).

The second Cox proportional-hazards regression analysis showed that within-person across-neuropsychological test variability was a significant predictor of incident dementia defined algorithmically (hazard ratio for one point difference in variability, 3.926, 95% CI, 2.040-7.558, p<0.001). This association was in the expected direction with the risk of dementia increasing as a function of greater within person variability.

The third Cox proportional-hazards regression analysis examined whether within-person across-neuropsychological test variability provided incremental prediction of incident dementia. Similar to the previous analysis, the results revealed that higher scores on the FCSRT (hazard ratio for one point difference on the test, 0.886, 95% CI, 0.853 - 0.921, p<0.001) and Digit Symbol Substitution (hazard ratio for one point difference on the test, 0.973; 95% CI, 0.943 - 1.000, p=0.04) predicted lower risk of incident dementia. As in the previous analysis the Vocabulary test did not predict future dementia (hazard ratio, 1.003 for one point difference on the test, 95% CI, 0.980 - 1.026, p=0.81). However, even after taking level of neuropsychological performance into account, within-person across-neuropsychological test variability remained a significant predictor of incident dementia (hazard ratio for one SD difference in variability, 2.098; 95% CI, 1.041 - 4.225, p=0.03). The reduction in the hazard ratio of variability was expected when adjusting for performance level on the individual tests. However, the low correlations between variability and performance on the neuropsychological tests (FCSRT, r= -0.21, p<0.001; Digit Symbol Substitution, r= -0,115, p<0.001; Vocabulary, r= -0.179, p<0.001) although significant were not suggestive of collinearity. The lower, but significant, hazard ratio for within-person across-neuropsychological test variability in the third compared to the second Cox analysis, along with hazard ratios for level of performance on the individual neuropsychological tests that were similar in the two analyses, are evidence that part, but not all, of the effect of variability is mediated through changes in levels of cognitive performance.

The partial likelihood ratio test⁴⁵ examined whether adding within-person across-neuropsychological test variability as a predictor to level of performance on the neuropsychological tests (third Cox analysis) improved prediction of dementia compared to using performance level on the neuropsychological tests alone (first Cox analysis). The results were significant (χ² =4.19, p=0.04) indicating that prediction of incident dementia was improved by adding within-person across-neuropsychological test variability. It was of further interest to examine whether within-person across-neuropsychological test variability improved the sensitivity for the prediction of dementia within one year. In a model that included the three individual neuropsychological tests and controlled for age, sex, education, and disease illness the cut-score that resulted in 80% specificity yielded 83% sensitivity for predicting dementia within one year. Including within-person across-neuropsychological test variability in this model significantly increased the sensitivity for predicting dementia within one year to 88% (p=0.014, McNemar’s test).

Kaplan-Meier Curves delineated the cumulative risk of incident algorithmic dementia using age as the time scale. Within person across neuropsychological tests variability, dichotomized into the highest (ie., worst) quartile versus the rest, revealed that consistent with the previous results higher variability was associated with increased risk of incident dementia (hazard ratio, 2.253; 95% CI, 1.317 - 3.855) (Figure 1a).

Figure 1.

a: Kaplan-Meier Curves for the Cumulative Risk of incident algorithmic dementia as a function of within-person across-neuropsychological test variability (highest quartile illustrated with a blue line versus the remaining three quartiles illustrated with a black line). Variability denotes within-person across-neuropsychological test variability; Variability > Q3 denotes highest (i.e., worst) quartile; Variability <= Q3 denotes remaining lower 3 quartiles

b: Kaplan-Meier Curves for the Cumulative Risk of incident algorithmic dementia as a function FCSRT performance (highest quartile illustrated with a black line versus the remaining three quartiles illustrated with a blue line). FCSRT denotes Free and Cued Selective Reminding Test; FCSRT > Q1 denotes highest (i.e., best) quartile; FCSRT <= Q1 denotes remaining lower 3 quartiles

c: Kaplan-Meier Curves for the Cumulative Risk of incident algorithmic dementia as a function of performance on the Digit Symbol Substitution test of the WAIS-R (highest quartile illustrated with a black line versus the remaining three quartiles illustrated with a blue line). DSYM denotes Digit Symbol Substitution Test; DSYM > Q1 denotes highest (i.e., best) quartile; DSYM <= Q1 denotes remaining lower 3 quartiles

d: Kaplan-Meier Curves for the Cumulative Risk of incident algorithmic dementia as a function of performance on the Digit Symbol Substitution tests (highest quartile illustrated with a black line versus the remaining three quartiles illustrated with a blue line). VOCAB denotes Vocabulary subtest of the WAIS-R VOCAB > Q1 denotes highest (i.e., best) quartile; VOCAB <= Q1 denotes remaining lower 3 quartiles

Additionally, Kaplan-Meier curves for the FCSRT (hazard ratio, 4.772; 95% CI, 2.798-8.1), Digit Symbol Substitution (hazard ratio, 2.428; 95% CI, 1.440-4.093), and Vocabulary tests ((hazard ratio, 1.505; 95% CI, 0.880-2.574) dichotomized into the highest (i.e., best) quartile versus the rest are illustrated in Figure 1b -d, respectively.

Exploratory analyses using clinical consensus diagnosis of dementia subtypes

Cox proportional-hazards regression analysis with age serving as the time scale adjusting for sex, education, and illness index examined whether associations between variability and clinical diagnosis of dementia using consensus case conference procedures varied as a function of dementia subtype. Variability predicted development of both incident dementia of the Alzheimer’s type (n=47; hazard ratio, 3.626; 95 percent confidence interval, 1.786 to 7.365), and incident vascular dementia (n=18; hazard ratio, 5.258; 95 percent confidence interval, 1.696 to 16.303). The disease processes underlying “pure” Alzheimer’s and Vascular dementias are different. Thus, it appears that variability is sensitive to dementia irrespective of disease subtype.

Discussion

Intra-individual variability is concerned with the study of within person differences in cognitive function and its use as a marker of pathology⁴⁶. The taxonomy of intra-individual variability offers several operational definitions for this construct⁴⁷. To our knowledge, this is the first study to demonstrate that within-person across-neuropsychological test variability predicts dementia in a population-based cohort age 70 and older even after adjusting for level of performance on each individual neuropsychological test. The potential clinical and diagnostic utility of within-person across-neuropsychological test variability was further substantiated by its contribution to increased sensitivity in the prediction of dementia within one year of the baseline assessment.

The premise guiding the study was that variability or inconsistency in performance across neuropsychological tests would increase in the pre-clinical stages of dementia compared to normal aging. This, in part, is attributed to the decline in memory function years before the diagnosis of dementia¹¹. As opposed to summary scores or intra-individual indices that estimate function within one cognitive domain, within-person across-neuropsychological test variability may be conceptualized as a single representation of variability across multiple domains subserved by a several cortical regions and networks. In this context, this form of cognitive variability maybe considered a signature of decline in cerebral integrity in the early stages of dementing illnesses.

In choosing the individual tests used to calculate within-person across-neuropsychological test variability we aimed to maximize the number of participants, represent different cognitive domains, and increase generalizability by including a few easily administered tests that are commonly used in other studies and by clinicians assessing cognitive impairment in the elderly. Within-person across-neuropsychological test variability can be computed using tests other than those used in this study. Further, this form of cognitive variability can be estimated using standard and widely used clinical neuropsychological assessment procedures that are typically given in one testing session. Hence, the potential clinical utility of this aspect of cognitive function is quite appealing as it requires no changes to standard assessment procedures in aging studies or assessment of cognitive disorder in clinic settings. It is noteworthy that variability on sub-domains of the MMSE⁴⁸ was related to short term decline on a global measure of cognitive function in a small cohort of centenarians⁴². Hence, examining across cognitive domain variability in screening measures including those currently used to support the diagnosis of mild cognitive impairments⁴⁹ is of interest.

Research and theories concerning within person variability on single measures, although relatively recent, suggest that increased inconsistency represents impaired top down executive control processes⁵⁰ subserved by frontal regions^{51, 52}. However, at present, little is known about within person across cognitive tests variability, the theoretical and neuroanatomical basis for this putative construct. Our findings suggest that even though substantial in non-demented older adults²³ within person across neuropsychological tests variability is pathological when a certain threshold is exceeded. It is noteworthy that variability was sensitive to both Alzheimer and Vascular dementia subtypes, which vary in terms of their etiology and cognitive profile^{33, 53, 54}. Therefore, as a signature of early decline in global cerebral integrity, variability may capture the summation of differing sensitivities of brain regions and networks to various disease processes as opposed to estimating the effect of disease on a single brain region. The low correlations between variability and the level of performance on the individual neuropsychological tests appear to support this notion.

The limitations of the study should be considered. First, the sample although representative of the Bronx, consisted of volunteers who resided in the community and who were relatively healthy and willing to travel to the medical center. Hence, the generalizability of the findings may potentially be limited. Second, the diagnosis of incident dementia was based on an algorithmic procedure that was independent of neuropsychological test scores and consensus diagnosis. Hence, diagnostic circularity is less likely to confound the findings. However, optimally, within-person across-neuropsychological test variability should be derived from tests that are not part of the clinical diagnostic procedures to examine longitudinal associations between this form variability and clinical diagnosis of dementia. Third, assignment of subtypes of dementia is fallible. Although the diagnoses were made according to standardized criteria, some misclassification is inevitable. Further, differential associations between variability and dementia subtypes may be demonstrated, perhaps when using neuropsychological tests that are more sensitive to either Vascular or Alzheimer’s diseases^{33, 53, 54}. Fourth, attrition is a major issue of concern in any longitudinal study. A significant number of individuals enrolled in the EAS were not eligible to participate in this investigation because they had only the baseline evaluation and were awaiting the next yearly visit. However, the subsample included in this study was not different from the entire EAS cohort in terms of age, gender distribution and level of education. Nonetheless the potential selection bias should be considered. Finally, we emphasize that a single variability measure cannot replace a complete neuropsychological examination nor do we advocate that three neuropsychological tests are sufficient to provide adequate assessment of the individual’s cognitive function. Instead, we propose that measures of within person across tests variability should be viewed as complementary to standard assessment procedures that are used to predict future risk of dementia.

It is of further interest to examine whether and the extent to which different aspects of intra-individual variability in cognitive functions are related in terms of the theory, underlying mechanism and utility in predicting outcomes of interest. For instance, research suggests that individuals are not consistently diagnosed with MCI across multiple sessions^{55, 56}. Although some fluctuations may be attributed to error in measurement and limitations of this construct, it is of interest to examine the relationship between, across and within testing session variability and whether the latter improves understanding and prediction of transitional stages in cognitive aging.