Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Feb 1.
Published in final edited form as: J Clin Exp Neuropsychol. 2011 Dec 13;34(2):209–219. doi: 10.1080/13803395.2011.630652

Normative data of a brief neuropsychological battery for Spanish individuals older than 49

Montserrat Alegret 1, Ana Espinosa 1, Georgina Vinyes-Junqué 1, Sergi Valero 1,2, Isabel Hernández 1, Lluís Tárraga 1, James T Becker 3,4,5, Mercè Boada 1,6
PMCID: PMC3264764  NIHMSID: NIHMS345362  PMID: 22149440

Abstract

There is an increasing need for standardized assessment of cognition in older patients that is relatively brief, easy to administer, and has normative data adjusted for age and educational attainment. We tested 332 literate, cognitively normal, Spanish persons older than 49 years from the Memory Clinic of Fundació ACE, Institut Català de Neurociències Aplicades (Barcelona, Spain) with measures of cognitive information processing speed, orientation, attention, verbal learning and memory, language, visuoperception, praxis, and executive functions. Several of the tests were affected by age, education, and/or gender, but the language of administration (i.e., Spanish or Catalan) did not affect the test scores. Standardized scores and percentile ranks were calculated for each age and/or education group for use by clinical neuropsychologists.

Keywords: Neuropsychological assessment, Aging, Educational effects, Verbal learning, Cognition

The population in developed countries is aging, and with increased age comes an increased risk for age-associated neurodegenerative disorders. Early and accurate detection of changes in cognitive status are critical, so that primary and secondary prevention strategies can be put in place. Neuropsychological assessment is a core component in the differential diagnosis of clinical dementia syndromes; however, it is critical that the assessment not only be sensitive to alterations in cognition above and beyond that expected for an individual’s age and education, but it must also be efficient (i.e., optimal time length). Thus, clinical practitioners require brief, easy-to-administer, neuropsychological batteries that may be useful in making the discrimination between normal and pathological aging, and also between different patterns of cognitive impairment.

In Spain, there have been several projects that provided normative data for a range of cognitive and functional instruments used to assess and diagnose dementia, such as the Alzheimer’s Disease Assessment Scale (ADAS) (Peña-Casanova, Aguilar, et al., 1997), Barcelona Neuropsychological Test–Shortened Version (Peña-Casanova, Guardia, et al., 1997), Boston Naming Test (Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Aguilar, et al., 2009; Rami et al., 2008), Stroop Color–Word Interference Test (Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Quintana, et al., 2009), Trail Making Test (Peña-Casanova, Quiñones-Úbeda, Quintana-Aparicio, et al., 2009), Visual Object, Space Perception Battery–Abbreviated (Peña-Casanova, Quintana-Aparicio, et al., 2009), Group for the Study and Multicenter Registry of Incident Cases of Dementia in Spain (GERMCIDE) neuropsychological protocol (Manubens et al., 2005), Blessed Dementia Rating Scale (Peña-Casanova et al., 2005), and Rapid Disability Rating Scale–2 (Monllau et al., 2006). However, there is relatively less information regarding visual–perceptual and verbal list learning abilities, which are frequently impaired early in dementia syndromes.

Although normative data have been provided for some memory tests, such as the Rey–Osterrieth Complex Figure Test and the Free and Cued Selective Reminding Test (Peña-Casanova, Gramunt-Fombuena, et al., 2009), there is a lack of information regarding the normal range of performance on verbal learning and retention, especially if recognition cues are utilized. Further, existing normative data—for example, from the Wechsler Memory Scale–Third Edition (Wechsler, 1997b) using Spanish individuals—found that a score of zero was into the normal range among those older than 74 years, rendering these normative data unusable in a dementia clinic.

The purpose of this paper was to describe the neuropsychological battery used in Fundació ACE (NBACE) that was created specifically to assess cognitive functions in approximately 45 minutes—a reasonable amount of time in routine clinical practice. The test battery includes common neuropsychological instruments whose utility have been demonstrated in research studies. In addition, we used the 15-Objects Test because it is easy to administer and provides information regarding the overall severity of cognitive impairment and visual perception abilities (Alegret et al., 2009; Alegret et al., 2010). We report here the psychometric characteristics of these tests, as well as the normative ranges and percentile scores for specific age and/or education subgroups.

METHOD

Subjects

Fundació ACE, Institut Català de Neurociències Aplicades is a nonprofit Alzheimer’s center that provides diagnostic, treatment, and patient management services to the Catalan Public Health Service (Xarxa Hospitalària d’Utilització Pública, XHUP). The patients are usually referred to the Memory Clinic of Fundació ACE by primary care physicians or medical specialists because the patients, their family, or their physician felt that they could have a memory problem.

The data included in this analysis were drawn from 332 individuals (118 men, 214 women) who visited Fundació ACE between January 2006 and May 2010. We selected for study those who were classified as cognitively normal with preserved performance on each of two cognitive screening tests—the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) ≥24 and the Clock Test (Del Ser, Sánchez, et al., 2004) ≥5—and had a Clinical Dementia Rating (CDR; Morris, 1993) of zero, were older than 49 years of age, and were functionally literate. From an initial sample of 513 individuals, the medical records were reviewed to exclude those individuals who did not meet the general inclusion criteria, or had a psychiatric or neurological disease, a focal lesion on brain computed tomography (CT) imaging, a history of alcohol or other substance abuse, and severe auditory or visual abnormalities including glaucoma and cataracts. From this group, a total of 332 healthy individuals were studied.

The subjects included in this study had no evidence of functional impairment secondary to decline in cognition and had at least minimal writing abilities. However, among older individuals living in Spain, especially women, there were reduced opportunities for educational experience due to the disruption caused by the Spanish Civil War and the Second World War. All of the participants lived in Barcelona, and their native language was either Spanish or Catalan. The neuropsychological assessment was administered in their native language: 204 Catalan, and 128 Spanish. It needs to be mentioned that participants were assessed in the language that they felt more comfortable with, but many of them spoke Catalan and Spanish equally well.

All of the data included in this report were obtained in compliance with the regulations of industrial assay, and the study followed the Declaration of Helsinki guidelines. Written informed consent was obtained from all participants prior to any research evaluations.

Neurobehavioral assessment

All of the participants received an extensive neurobehavioral evaluation including medical history and physical examination, neurological history and exam, a semistructured psychosocial interview conducted by a social worker, and a neuropsychological assessment. The clinical classification of each individual was made initially by the neurologist based on the clinical evaluation, and this was later reviewed by the study team including neurologists, neuropsychologists, and social workers at a consensus diagnostic conference.

The neurological exam included cranial nerve testing, motor tone, abnormal movements, strength, deep tendon reflexes, release signs, plantar response and clonus, cerebellar testing, primary sensory testing, gate, and postural stability. The neurologist also completed the Tinetti balance and gait scale (Tinetti, Williams, & Mayewski, 1986) and the Hachinski Ischemia Scale (Hachinski et al., 1975). Each participant also completed the Spanish version of the MMSE (Blesa et al., 2001), the Spanish version of the short form of the Neuropsychiatric Inventory (NPI; Boada, Cejudo, Tàrraga, López, & Kaufer, 2002), and the Blessed Dementia Rating Scale (Blessed, Tomlinson, & Roth, 1968). In addition, all of the subjects completed the Spanish version of the Clock Test (Del Ser, García de Yébenes, et al., 2004), with a scoring range of 0–7, as a measure of global cognition.

Neuropsychological assessment

As detailed previously (Alegret et al., 2009, Alegret et al., 2010; Seshadri et al., 2010), the neuropsychological battery in use in Fundació ACE (NBACE) from January 2006 onward includes tests sensitive to cognitive information processing speed, orientation, attention, verbal learning and memory, language, visuoperception, praxis, and executive functions, including the following tests: Temporal, Spatial and Personal Orientation; Digit Span Forwards and Backwards, Block Design (abbreviated so that Items 6 to 9 were scored only for accuracy, 1 point, without a time bonus), and Similarities (abbreviated to the first 10 items) subtests of the Wechsler Adult Intelligence Scale–Third Edition (WAIS–III; Wechsler, 1997a); The Word List Learning test from the Wechsler Memory Scale–Third Edition (WMS–III; without using the interference list; Wechsler, 1997b); Verbal Comprehension (to correctly execute two simple, two semicomplex, and two complex commands extracted from the ADAS-cognitive subscale (ADAS-cog); Rosen, Mohs, & Davis, 1984; and the Barcelona test battery; Peña-Casanova, 1991); an abbreviated 15-item naming test from the Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1983); the Poppelreuter test (Della Sala, Laiacona, Trivelli, & Spinnler, 1995); Luria’s Clock Test (Golden, 1980); the Automatic Inhibition subtest of the Syndrom Kurtz Test (SKT; Erzigkeit, 1989); Phonetic Verbal Fluency (words beginning with “P” in one minute; Artiola, Hermosillo, Heaton, & Pardee, 1999); and Semantic Verbal Fluency (“animals” in one minute; Goodglass & Kaplan, 1983). A subsample of 119 individuals also completed the 15-Objects Test (Pillon et al., 1989), which focuses on visual perceptual functions, but also provides information regarding their overall severity of cognitive impairment (Alegret et al., 2009, Alegret et al., 2010). The complete list of tests is shown in Table 1.

TABLE 1.

Description and descriptive statistics for NBACE tests in the 332 participants

Function/Test Test variable Range Sp + CM (SD) Spanish M (SD) Catalan M (SD)
Orientation
Temporal (T) Total score 0–5 4.93 (0.25) 4.95 (0.21) 4.92 (0.28)
Spatial (S) Total score 0–5 5.00 (0.00)a 5.00 (0.00) 5.00 (0.00)
Personal (P) Total score 0–5 4.99 (0.09) 5.00 (0.00) 4.99 (0.12)
Global orientation T + S + P 0–15 14.93 (0.26) 14.95 (0.21) 14.91 (0.28)
Verbal learning and memory
Learning Trial 1 WMS–III Correct responses 0–12 4.48 (1.38) 4.34 (1.48) 4.57 (1.32)
Trial 2 WMS–III Correct responses 0–12 6.94 (1.63) 6.90 (1.69) 6.96 (1.60)
Trial 3 WMS–III Correct responses 0–12 8.45 (1.70) 8.31 (1.73) 8.54 (1.68)
Trial 4 WMS–III Correct responses 0–12 9.58 (1.72) 9.55 (1.83) 9.59 (1.66)
Learning (total) 1 + 2 + 3 + 4 trials 0–48 29.44 (5.43) 29.09 (5.58) 29.67 (5.33)
Delayed Recall WMS–III Correct responses 0–12 7.27 (2.38) 7.30 (2.33) 7.26 (2.41)
Forgetting Trial 4–Delayed Recall 0–12 2.30 (1.80) 2.25 (1.72) 2.33 (1.86)
Recognition memory Correct responses 0–24 22.86 (1.37) 22.77 (1.44) 22.92 (1.32)
False memories 0–12 0.25 (0.61) 0.25 (0.65) 0.25 (0.59)
Attention and working memory
Digit Span Forward WAIS–III Correct responses 0–16 7.84 (1.94) 7.69 (2.00) 7.94 (1.89)
Digit Span Backwards WAIS–III Correct responses 0–14 4.98 (1.65) 4.89 (1.54) 5.03 (1.72)
Praxis
Ideomotor (I) Correct responses 0–4 3.88 (0.42) 4.00 (0.00) 4.00 (0.07)
Block Design WAIS–III (B) Correct responses 0–4 4.00 (0.05) 3.73 (0.61) 3.80 (0.53)
Imitation (Im) Correct responses 0–4 3.77 (0.57) 3.84 (0.44) 3.90 (0.40)
Global praxis I + B + Im 0–12 11.65 (0.74) 11.57 (0.80) 11.70 (0.69)
Language
Comprehension Correct responses 0–6 5.97 (0.21) 5.98 (0.15) 5.96 (0.24)
Repetition Correct responses 0–4 4.00 (0.05) 4.00 (0.00) 4.00 (0.07)
Visual Naming (15-BNT) Correct responses 0–15 14.63 (0.79) 14.49 (0.94) 14.72 (0.66)
Visual perception
Poppelreuter test Correct responses 0–10 9.85 (0.40) 9.86 (0.39) 9.84 (0.40)
Number or errors ≥0 0.28 (0.79) 0.33 (1.06) 0.25 (0.56)
The 15-Objects Testb Correct responses 0–15 13.55 (1.44) 12.98 (1.67) 13.90 (1.14)
Number of errors ≥0 1.43 (1.48) 1.91 (1.67) 1.12 (1.26)
Luria’s Clock test Correct responses 0–4 3.52 (0.63) 3.50 (0.64) 3.54 (0.63)
Executive functions
Automatic Inhibition SKT Time in seconds >0 27.11 (7.85) 28.70 (8.52) 26.11 (7.24)
Number of errors ≥0 0.85 (1.54) 0.98 (1.74) 0.77 (1.40)
Phonetic Verbal Fluency Correct responses ≥0 14.17 (4.67) 14.38 (4.86) 14.04 (4.55)
Semantic Verbal Fluency Correct responses ≥0 18.24 (4.53) 18.23 (4.73) 18.25 (4.42)
Similarities WAIS–III Total score with the first 10 items 0–15 11.31 (1.95) 11.15 (2.03) 11.41 (1.90)
Open in a new tab

Note. NBACE = the neuropsychological battery used in Fundació ACE; Sp = Spanish; C = Catalan; WMS–III = Wechsler Memory Scale, Third Edition; WAIS–III = Wechsler Adult Intelligence Scale, Third Edition; 15-BNT = the abbreviated Boston Naming Test with 15 items; SKT = Syndrom Kurtz Test.

a

It is practically a constant.

b

In a subsample of 119 subjects.

All of the neuropsychological testing was carried out in the Diagnostic Unit of Fundació ACE by one of the three neuropsychologists (M.A., A.E., G.V.J.) and was completed in the patient’s language of choice (Spanish or Catalan). The results of each examination were reviewed by all of the neuropsychologists, and scoring was arrived at by consensus to ensure reliability of the data.

Statistical analysis

Statistical analysis was performed using SPSS 18 (SPSS Inc., Chicago, IL). All the data were examined for normality, skew, and restriction of range. Regression models were used to evaluate the effects of age, education, sex, and language of administration for each of the variables. The eta-square values were reported as a measure of effect size.

Each neuropsychological variable was regressed on sex, age (three categories), and educational level (three categories). If one of these independent variables was statistically significant, a descriptive table including means and standard deviations (SDs) of the categories was reported for such variables. When two of these factors were significant, the same descriptive table was reported, but including the combination of the categories of these factors. When gender was also found to be statistically significant, the analysis was adjusted by gender. No post hoc comparisons between categories were analyzed because the aim of this study was to provide a descriptive profile of the variables and not to estimate differences between groups. If none of the predictors had a significant effect, a mean and a standard deviation of the overall sample were provided. For all the analyses, an effect was considered significant when p < .05. Finally, percentile scores were provided to show the relative rank of each raw score.

RESULTS

The mean age of participants was 68.89 years (SD = 8.22, range: 50–89). A total of 59.3% of subjects had elementary or high-school education (from 7 to 11 years of formal education), 24.4% had a Bachelor’s degree (from 12 years of formal education), and 16.3% were literate with less than elementary school (that is, from 1 to 6 years of formal education). The mean score on the MMSE was 28.95 (SD = 1.16, range: 25–30), and the mean score on the Clock test was 6.86 (SD = 0.44, range: 5–7). The means and standard deviations of the NBACE test variables are detailed in Table 1. Percentile scores were also calculated (see Table 2). Table 3 presents the effects of age, education, and gender for each of the neuropsychological variables.

TABLE 2.

Percentile scores on the NBACE tests

Function/test Percentiles
10 25 50 75 90
Orientation
Global orientationa,b 15 15 15 15 15
Verbal learning and memory Total learning (Trials 1 + 2 + 3 + 4) 22 26 30 33 37
21 25 29 32 37
22 26 30 33 37
Delayed Recall WMS–III 4 5 7 9 10
4 5 7 9 11
4 6 7 9 10
Forgetting (Trial 4–Delayed Recall)b 4 3 2 1 0
Recognition verbal memory 21 22 23 24 24
20 22 23 24 24
21 22 23 24 24
Attention and working memory
Digit Span Forward WAIS–IIIb 6 6 8 9 10
Digit Span Backwards WAIS–IIIb 3 4 5 6 7
Praxis
Global praxis (I + B + Im)b 11 12 12 12 12
Language
Comprehensiona,b 6 6 6 6 6
Repetitiona,b 4 4 4 4 4
Visual Naming (15-BNT) 14 15 15 15 15
13 14 15 15 15
14 15 15 15 15
Visual perception
Poppelreuter testb 9 10 10 10 10
The 15-Objects Test (correct responses) 12 13 14 15 15
11 12 13 14 15
12 13 14 15 15
The 15-Objects Test (mistakes) 3 2 1 0 0
4 3 1 1 0
3 2 1 0 0
Luria’s Clock testb 3 3 4 4 4
Executive functions
Automatic Inhibition SKT (s) 38 31 26 21 18
41 33 27 22 18
35 30 25 21 17
Automatic Inhibition SKT (errors) 3 1 0 0 0
3 2 0 0 0
2 1 0 0 0
Phonetic verbal fluencyb 9 11 14 17 20
Semantic verbal fluency 13 15 18 21 25
12 15 18 21 25
13 15 18 21 25
Similarities WAIS–IIIb 9 10 11 13 14
Open in a new tab

Note. Values in regular type correspond to all the sample (n = 332), values in bold correspond to the Spanish subsample (n = 128), and values in italics correspond to the Catalan subsample (n = 204). NBACE = the neuropsychological battery used in Fundació ACE; WMS–III = Wechsler Memory Scale, Third Edition; WAIS–III = Wechsler Adult Intelligence Scale, Third Edition; I = Ideomotor; B = Block Design WAIS–III; Im = Imitation; 15-BNT = the abbreviated Boston Naming Test with 15 items; SKT = Syndrom Kurtz Test.

a

It is practically a constant.

b

Spanish and Catalan subjects have the same percentile scores.

TABLE 3.

Effects of age, education, and gender on the neuropsychological variables

Function/test Age
 Education
 Gender
F(2, 326) ε2 F(2, 326) ε2 F(1, 326) ε2
Orientation
Global orientation 0.17 .01 1.54 .01 0.75 .01
Verbal learning and memory
Trial 1 WMS–III 7.01** .04 15.23*** .08 2.49 .01
Trial 2 WMS–III 13.42*** .08 13.81*** .08 8.12** .02
Trial 3 WMS–III 14.61*** .08 12.15*** .07 9.09** .03
Trial 4 WMS–III 11.24*** .06 10.66*** .06 14.83*** .04
Learning (Trials 1+2+3+4) 17.30*** .10 18.15*** .10 12.50*** .04
Delayed Recall WMS–III 15.62*** .09 2.43 .01 12.41*** .04
Forgetting (Trial 4–Recall) 3.85* .02 1.36 .01 0.92 .01
Recognition memory 6.03** .04 0.08 .01 3.78 .01
Attention and working memory
Digit Span Forward 2.83 .02 11.00*** .06 0.15 .01
Digit Span Backwards 1.12 .01 23.10*** .12 1.14 .01
Praxis
Global praxis 0.45 .01 5.92** .03 0.57 .01
Language
Comprehension 0.10 .01 2.34 .01 0.57 .01
Repetition 1.35 .01 0.31 .01 2.06 .01
Visual Naming (15-BNT) 3.12* .02 6.12** .04 2.15 .01
Visual perception
Poppelreuter test (responses) 3.06* .02 2.06 .01 0.30 .01
Poppelreuter test (mistakes) 1.86 .01 4.28* .03 0.00 .01
The 15-Objects Test (responses) 2.51 .04 1.85 .03 1.06 .01
The 15-Objects Test (mistakes) 4.90** .08 3.19* .05 1.02 .01
Luria’s Clock test 2.20 .01 1.40 .01 14.80*** .04
Executive functions
Automatic Inhibition SKT (s) 8.68*** .05 27.06*** .14 6.57* .02
Automatic Inhibition SKT (error) 3.24* .02 7.24** .04 1.94 .01
Phonetic verbal fluency 6.98** .04 29.02*** .15 1.85 .01
Semantic verbal fluency 22.23*** .12 17.39*** .10 0.00 .01
Similarities WAIS–III 3.82* .02 28.71*** .15 0.01 .01
Open in a new tab

Note. WMS–III = Wechsler Memory Scale, Third Edition; WAIS–III = Wechsler Adult Intelligence Scale, Third Edition; 15-BNT = the abbreviated Boston Naming Test with 15 items; SKT = Syndrom Kurtz Test.

*

p < .05.

**

p ≤ .005.

***

p ≤ .0005.

The language in which the battery was administrated (204 Catalan and 128 Spanish) did not produced statistically significant effects in any of the test performances, so all scores are presented without regard to language. Statistically significant differences were found between age, education, and gender groups on several measures, but in most cases they were not considered clinically relevant (see Table 3). Nevertheless, means and standard deviations are presented in Table 4, stratified into three age groups and three educational levels for the four trials of the Word List from the WMS–III, Phonetic and Semantic Verbal Fluency, the Automatic Inhibition subtest of the SKT, and Abstract Reasoning from the WMS–III. Verbal Long-Term Memory, Forgetting, and Recognition scores in the Word List from the WMS–III are shown stratified by age. All the values reported in Table 4 were adjusted for gender.

TABLE 4.

Test battery

Test Age group (years)
<65 65–74 >75
WMS–III
First trial WMS–III on the Word List subtest
 Less than elementary school 3.78 (0.97) 3.68 (1.31) 3.71 (1.53)
 Elementary or high school 4.75 (1.04) 4.67 (1.40) 3.91 (1.15)
 Bachelor’s degree 5.32 (1.22) 4.95 (1.58) 4.67 (1.45)
Second trial WMS–III on the Word List subtest
 Less than elementary school 7.11 (1.05) 6.32 (1.33) 5.88 (1.41)
 Elementary or high school 7.00 (1.31) 7.09 (1.48) 6.21 (1.62)
 Bachelor’s degree 8.36 (1.81) 7.66 (1.60) 6.40 (1.80)
Third trial WMS–III on the Word List subtest
 Less than elementary school 8.11 (1.76) 7.75 (1.58) 7.53 (1.37)
 Elementary or high school 8.75 (1.47) 8.64 (1.65) 7.61 (1.70)
 Bachelor’s degree 9.54 (1.40) 9.32 (1.68) 7.80 (1.52)
Fourth trial WMS–III on the Word List subtest
 Less than elementary school 8.89 (1.62) 9.04 (1.45) 8.06 (2.04)
 Elementary or high school 10.12 (1.46) 9.74 (1.40) 9.11 (1.74)
 Bachelor’s degree 10.43 (1.53) 9.92 (2.12) 9.00 (1.77)
Long-term memory, Forgetting (Fourth trial–long-term memory), and Recognition on the Word List subtest from the WMS–III
 Long-term memory 7.84 (0.25) 7.22 (0.20) 6.02 (0.25)
 Forgetting 1.91 (0.20) 2.22 (1.16) 2.64 (0.20)
 Recognition verbal memory 23.06 (0.15) 22.87 (0.12) 22.39 (0.15)
Phonetic verbal fluency
 Less than elementary school 12.11 (4.20) 11.32 (3.41) 11.29 (2.95)
 Elementary or high school 14.46 (4.73) 14.20 (3.82) 12.25 (3.92)
 Bachelor’s degree 18.43 (5.36) 14.00 (4.99) 14.93 (3.33)
Semantic verbal fluency
 Less than elementary school 17.89 (3.22) 15.64 (3.47) 13.53 (3.26)
 Elementary or high school 20.03 (5.18) 18.01 (3.61) 16.58 (3.76)
 Bachelor’s degree 22.54 (4.15) 19.74 (4.15) 17.40 (3.79)
SKT (time in s)
 Less than elementary school 32.78 (9.09) 34.39 (8.19) 32.29 (6.48)
 Elementary or high school 24.66 (8.48) 26.86 (6.28) 29.82 (7.31)
 Bachelor’s degree 21.21 (4.26) 22.95 (5.27) 26.40 (7.36)
Abstract Reasoning from the WMS–III
 Less than elementary school 10.67 (1.58) 9.79 (1.87) 10.00 (2.18)
 Elementary or high school 11.42 (2.01) 11.23 (1.69) 10.84 (1.71)
 Bachelor’s degree 13.18 (1.59) 12.03 (1.42) 12.40 (1.96)
Open in a new tab

DISCUSSION

In this report, we describe the results of the analysis of neuropsychological test data obtained from a 45-minute battery of tests, the NBACE, administered to cognitively normal individuals attending a memory diagnostic clinic in central Barcelona. We found no significant differences in test performance between individuals who were assessed in Spanish and those who were examined in Catalan. Thus, we were able to utilize data from all 332 individuals in order to estimate the effects of age, education, and gender on test performance as well as documenting percentile ranks for each of the tests administered.

The NBACE was designed specifically for use in elderly individuals who were being evaluated for dementia, although it could certainly be used in other contexts. It has the advantage of being brief, and it can be completed in the context of a single-day assessment within a neurobehavioral clinic. The test data obtained are particularly relevant to the identification of neurodegenerative diseases in that they focus heavily on measures of memory and learning, visual perception, and executive functions, all of which are affected early in the course of most degenerative disorders (Alegret et al., 2009; Amieva et al., 2005; Bennett et al., 2002; Espinosa et al., 2009). However, additional research will be needed to determine whether the tests have sufficient sensitivity to identify and characterize mild cognitive impairment (Petersen et al., 1999).

There have been several previous neuropsychological test batteries designed, standardized, and normalized on native speakers in Spain (i.e., Brief Assessment in Cognition in Schizophrenia: Segarra et al., 2011; a short neuropsychological protocol in Spanish called PRO-NEURO: Adrián, Hermoso, Buiza, Rodríguez-Parra, & González, 2008; GERMCIDE: Manubens et al., 2005; ADAS: Peña-Casanova, Aguilar, et al., 1997; Barcelona Neuropsychological Test–Shortened Version: Peña-Casanova, Guardia, et al., 1997). These have the advantage of being comprehensive and useful across a wide age range. By contrast, the present battery was specifically designed in order to be useful in clinical and research settings that focus on neurobehavioral disorders of the elderly (i.e., Alzheimer’s disease and related dementias), including visual perceptual and verbal list learning tests, which allows measurement of verbal learning, delayed free recall, and recognition memory abilities.

In order to facilitate the integration of the neuropsychological testing into a single day clinic visit, and to reduce patient fatigue related to the testing, the battery was intentionally limited to 45–55 minutes duration. This required some streamlining of the test battery, but this was done with the full knowledge that additional information would be available from the neurological exam (completed by a behavioral neurologist) and psychosocial interview (social worker). Additional information regarding the patient’s cognitive and behavioral health was obtained from the caregivers who accompanied the patient to the clinic. Thus, while an abbreviated battery such as the one described here would be less effective in a stand-alone neuropsychology clinic, it is optimally detailed for use in a multidisciplinary clinic as exists in Fundació ACE.

It is well known that verbal learning and memory are affected by age (Gómez-Pérez & Ostrosky-Solís, 2006; Lezak, Howieson, & Loring, 2004; Manubens et al., 2005; Wechsler, 1997b; Norman, Evans, Miller & Heaton, 2000; Peña-Casanova, Gramunt-Fombuena, et al., 2009), education (Ardila, Ostrosky-Solis, Rosselli & Gómez, 2000; Lezak et al., 2004; Norman et al., 2000; Peña-Casanova, Gramunt-Fombuena, et al., 2009), and gender (Lezak et al., 2004; Norman et al., 2000). In the present study, the results of the analysis of the neuropsychological test scores revealed that measures of the verbal list learning test were the most sensitive to the effects of age and education. In particular, the total number of words learned over the course of four trials had the largest effect sizes for age, education, and sex, and the verbal delayed recall score was particularly affected by age and gender.

In accordance with previous studies (Ardila et al., 2000; Gómez-Pérez & Ostrosky-Solís, 2006), measures of attention and working memory (Digit Span Forward and Backward) were uniquely associated with education. Peña-Casanova and coworkers (Peña-Casanova, Quiñones-Úbeda, Quintana-Aparicio, et al., 2009), in a sample similar to ours (ranging from 50 to 90 years), found that attentional abilities were also related to age (Peña-Casanova, Quiñones-Úbeda, Quintana-Aparicio, et al., 2009), but they reported that this effect increased with the complexity of the test, with small effects (<7% of the variance) on simple tests, such as those used in our test battery (Digit Span Forward and Backwards), and larger effects on more complex tests, such as the Symbol Digit Modalities Test (23% of the variance). Finally, in contrast to other studies (Wechsler, 1997b; Wilde, Strauss, & Tulsky, 2004), we did not find effects of age on simple attentional and working memory tasks likely because our study had a more restricted range of age—that is, our study ranged from 50, while others ranged from younger ages, such as 35 (Wechsler, 1997b) or even 16 (Wechsler, 1997a; Wilde et al., 2004).

With regard to the measure of visual confrontation naming, and consistent with previous studies, performance was found to be mainly affected by education (Gómez-Pérez & Ostrosky-Solís, 2006; Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Aguilar, et al., 2009; Pineda et al., 1998; Rami et al., 2008), but also by age (Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Aguilar, et al., 2009; Pineda et al., 1998). In terms of visuoperceptual abilities, as reported previously (Peña-Casanova, Quintana-Aparicio, et al., 2009), age and education were found significantly and discretely related to performance on the object perception tests.

Measures of executive function were, generally speaking, most affected by age and level of education. This was most apparent on the time measure from the Automatic Inhibition subtest of the SKT, the measures of phonetic and semantic verbal fluency, and to a lesser extent on the measure of verbal similarities. It is of interest to note that the two fluency measures, the Automatic Inhibition subtest, Digit Span Backwards, and the Similarities subtest had the largest effect sizes related to education. In accord with our results, previous studies also found a significant effect of education (Gómez-Pérez & Ostrosky-Solís, 2006; Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Quintana-Aparicio, et al., 2009; Rami, Serradell, Bosch, Villar, & Molinuevo, 2007; Snitz et al., 2009) and age (Gómez-Pérez & Ostrosky-Solís, 2006; Peña-Casanova, Quiñones-Úbeda, Gramunt-Fombuena, Quintana-Aparicio, et al., 2009) on semantic and phonetic verbal fluency, and abstract reasoning.

In spite of the relatively large size of the study sample, it was difficult to provide sufficient data within an individual age by gender by sex breakdown to provide appropriate standardized scores. Therefore, we utilized percentile scores for each of the relevant variables, which will be useful for the clinician in determining how the examinee is functioning relative to his/her reference group. As reported previously (Manubens et al., 2005; Peña-Casanova, Guardia, et al., 1997), we found that the measures of orientation, repetition, and verbal comprehension were sufficiently easy that any error on any of the three components was abnormal.

Ideally, normative data such as these would be obtained from community-dwelling volunteers who were recruited using various epidemiological sampling methods. What is less useful are individuals who are recruited into research studies through referral clinics where there may be a significant bias with regards to the level of education and socioeconomic strata, thus restricting the generaliz-ability of the findings. In the present study, the data were obtained from family members of patients evaluated in the memory clinic, as well as from individuals who were self-referred but found to be cognitively normal. The subjects included in this study had no evidence of functional impairment secondary to decline in cognition, and they had at least minimal writing abilities. Because of the social disruptions in the years 1935–1955, 16% of the participants were only functionally literate, having completed fewer than 4 years of formal education. Thus, the data from these cognitively normal subjects can be appropriately compared to the patient samples, as they are demographically similar.

Creating truly normative data for any kind of variable assumes that the distribution of the scores approximates normal. However, neuropsychological tests, with few exceptions, violate this assumption. This is due to the fact that these measures are generally designed to measure the performance of patients known (or suspected) to have impaired cognitive functions. If test performance is in the normal range, it is useful for diagnosis and management only to the extent that it may provide a comparison with scores that are outside of the normal range. It is usually more important for the clinician to observe which scores are abnormal and exactly how far they are below the expected range. In addition, because the focus of the testing is on patients, the tests must be within their capabilities. As a consequence, many tests have ceiling effects, because the alternative (a floor effect) would render the test generally useless in clinical practice. Consequently, using procedures such as those suggested by Crawford and colleagues (Crawford, Garthwaite, & Slick, 2009) is difficult, because the raw data that are used to create the percentile scores, for example, assume normality of the data—an assumption that is often not met. In the current project, we did inspect the distributions of the test scores and, with the exceptions of the two fluency variables, none of the measures was statistically normal. Consequently, rather than use techniques that focus on z-transformations, we instead opted for examining the distributions of the data themselves for determining the percentile ranks.

The data reported here are the first step in developing a set of diagnostic cutoffs and decision-making algorithms for use in identifying elderly individuals with cognitive impairment. Fundació ACE enrolls more than 1,000 new patients every year, all of whom are evaluated using the NBACE. Subsequent studies will evaluate the utility of this neuropsychological test battery in discriminating among patient groups, as well as setting the relevant cutoff scores for impaired function for different levels of education and age.

Acknowledgments

Preparation of this manuscript was supported in part by the University of Pittsburgh Alzheimer’s Disease Research Center (AG05133) and by funds from the Fulbright Commission to J.T.B.

References

  1. Adrián JA, Hermoso P, Buiza JJ, Rodríguez-Parra MJ, González M. Estudio piloto de la validez, fiabilidad y valores de referencia normativos de la escala PRO-NEURO en adultos mayores sin alteraciones cognitivas [Pilot study of the validity, reliability and normative data for PRO-NEURO scale in adult-elderly people without cognitive impairments] Neurología. 2008;23:275–287. [PubMed] [Google Scholar]
  2. Alegret M, Boada-Rovira M, Vinyes-Junqué G, Valero S, Espinosa A, Hernández I, et al. Detection of visuoperceptual deficits in preclinical and mild Alzheimer’s disease. Journal of Clinical and Experimental Neuropsychology. 2009;31:860–867. doi: 10.1080/13803390802595568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Alegret M, Vinyes-Junqué G, Boada M, Martínez-Lage P, Cuberas G, Espinosa A, et al. Brain perfusion correlates of visuoperceptual deficits in mild cognitive impairment and mild Alzheimer’s disease. Journal of Alzheimer’s Disease. 2010;21:557–567. doi: 10.3233/JAD-2010-091069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Amieva H, Jacqmin-Gadda H, Orgogozo JM, Le Carret N, Helmer C, Letenneur L, et al. The 9 year cognitive decline before dementia of the Alzheimer type: A prospective population-based study. Brain. 2005;128:1093–1101. doi: 10.1093/brain/awh451. [DOI] [PubMed] [Google Scholar]
  5. Ardila A, Ostrosky-Solis F, Rosselli M, Gómez C. Age-related cognitive decline during normal aging: The complex effect of education. Archives of Clinical Neuropsychology. 2000;15:495–513. [PubMed] [Google Scholar]
  6. Artiola L, Hermosillo D, Heaton R, Pardee RE. Manual de normas y procedimientos para la batería neuropsicológica en español [Manual of norms and procedures for the Spanish neuropsychological battery] Tucson, AZ: M Press; 1999. [Google Scholar]
  7. Bennett DA, Wilson RS, Schneider JA, Evans DA, Beckett LA, Aggarwal NT, et al. Natural history of mild cognitive impairment in older persons. Neurology. 2002;59:198–205. doi: 10.1212/wnl.59.2.198. [DOI] [PubMed] [Google Scholar]
  8. Blesa R, Pujol M, Aguilar M, Santacruz P, Bertran-Serra I, Hernández G, et al. Clinical validity of the “mini-mental state” for Spanish speaking communities. Neuropsychologia. 2001;39:1150–1157. doi: 10.1016/s0028-3932(01)00055-0. [DOI] [PubMed] [Google Scholar]
  9. Blessed G, Tomlinson BE, Roth M. The association between quantitative measures of dementia and of senile change in the cerebral grey matter of elderly subjects. British Journal of Psychiatry. 1968;114:797–811. doi: 10.1192/bjp.114.512.797. [DOI] [PubMed] [Google Scholar]
  10. Boada M, Cejudo JC, Tàrraga L, López OL, Kaufer D. Neuropsychiatric Inventory Questionnaire (NPI-Q): Validación española de una forma abreviada del Neuropsychiatric Inventory (NPI) [Neuropsychiatric inventory questionnaire (NPI-Q): Spanish validation of an abridged form of the Neuropsychiatric Inventory (NPI)] Neurología. 2002;17:317–323. [PubMed] [Google Scholar]
  11. Crawford JR, Garthwaite PH, Slick DJ. On percentile norms in neuropsychology: Proposed reporting standards and methods for quantifying the uncertainty over the percentile ranks of test scores. Clinical Neuropsychologist. 2009;23:1173–1195. doi: 10.1080/13854040902795018. [DOI] [PubMed] [Google Scholar]
  12. Della Sala S, Laiacona M, Trivelli C, Spinnler H. Poppelreuter-Ghent overlapping figures test. Its sensitivity to age, and its clinical use. Archives of Clinical Neuropsychology. 1995;10:511–534. doi: 10.1016/0887-6177(94)00049-v. [DOI] [PubMed] [Google Scholar]
  13. Del Ser T, García de Yébenes MJ, Sánchez F, Frades B, Rodríguez A, Bartolomé MP, et al. Evaluación cognitiva del anciano. Datos normativos de una muestra poblacional española de más de 70 años [Cognitive assessment in the elderly Normative data of a Spanish population sample older than 70 years] Medicina Clínica. 2004;122:727–740. doi: 10.1157/13062190. [DOI] [PubMed] [Google Scholar]
  14. Del Ser T, Sánchez F, García de Yébenes MJ, Otero A, Zunzunegui MV, Muñoz DG. Versión española del Test de los 7 Minutos. Datos normativos de una muestra poblacional de ancianos de más de 70 años [Spanish version of the 7 Minute screening neurocognitive battery. Normative data of an elderly population sample over 70] Neurología. 2004;19:344–358. [PubMed] [Google Scholar]
  15. Erzigkeit H. The SKT: A short cognitive performance test as an instrument for the assessment of clinical efficacy of cognitive enhancers. In: Berenger M, Reisberg B, editors. Diagnosis and treatment of senile dementia. Berlin, Germany: Springer Verlag; 1989. pp. 164–174. [Google Scholar]
  16. Espinosa A, Alegret M, Boada M, Viñas G, Valero S, Martínez-Lage P, et al. Ecological assessment of executive functions in mild cognitive impairment and mild Alzheimer’s disease. Journal of the International Neuropsychological Society. 2009;15:751–757. doi: 10.1017/S135561770999035X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state.” A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  18. Golden CJ. In reply to Adams “In search of Luria’s battery: A false start”. Journal of Consulting and Clinical Psychology. 1980;48:517–521. doi: 10.1037//0022-006x.48.4.517. [DOI] [PubMed] [Google Scholar]
  19. Gómez-Pérez E, Ostrosky-Solís F. Attention and memory evaluation across the life span: Heterogeneous effects of age and education. Journal of Clinical and Experimental Neuropsychology. 2006;28:477–494. doi: 10.1080/13803390590949296. [DOI] [PubMed] [Google Scholar]
  20. Goodglass H, Kaplan E. The assessment of aphasia and related disorders. 2. Philadelphia, PA: Lea and Febiger; 1983. [Google Scholar]
  21. Hachinski VC, Iliff LD, Zilhka E, Du Boulay GH, McAllister VL, Marshall J, et al. Cerebral blood flow in dementia. Archives of Neurology. 1975;32:632–637. doi: 10.1001/archneur.1975.00490510088009. [DOI] [PubMed] [Google Scholar]
  22. Kaplan EF, Goodglass H, Weintraub S. The Boston Naming Test. Philadelphia, PA: Lea & Febiger; 1983. [Google Scholar]
  23. Lezak MD, Howieson DB, Loring DW. Neuropsychological assessment. 4. New York, NY: Oxford University Press; 2004. [Google Scholar]
  24. Manubens JM, Barandiarán M, Martínez-Lage P, Francés I, Martínez C, García de la Rocha ML, et al. Valores del protocolo neuropsicológico GERMCIDE en una muestra de sujetos normales [Values of GERMCIDE neuropsychological protocol in a sample of normal subjects] Neurología. 2005;20:174–179. [PubMed] [Google Scholar]
  25. Monllau A, Aguilar M, Peña-Casanova J, Böhm P, Blesa R, Sol JM, et al. Estudio de la Escala de Evaluacíón Rápida de Discapacidad-2 en la enfermedad de Alzheimer: Datos del proyecto NORMACODEM [Rapid Disability Rating Scale-2 in Alzheimer’s disease: NORMACODEM project data] Neurologia. 2006;21:282–288. [PubMed] [Google Scholar]
  26. Morris JC. The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology. 1993;43:2412–2414. doi: 10.1212/wnl.43.11.2412-a. [DOI] [PubMed] [Google Scholar]
  27. Norman MA, Evans JD, Miller WS, Heaton RK. Demographically corrected norms for the California Verbal Learning Test. Journal of Clinical and Experimental Neuropsychology. 2000;22:80–94. doi: 10.1076/1380-3395(200002)22:1;1-8;FT080. [DOI] [PubMed] [Google Scholar]
  28. Peña-Casanova J. Programa Integrado de Exploración Neuropsicológica Test Barcelona [Integrated Program of Neuropsychological Assessment Barcelona Test] Barcelona, Spain: Masson; 1991. [Google Scholar]
  29. Peña-Casanova J, Aguilar M, Santacruz P, Bertran-Serra I, Hernández G, Sol JM, et al. Adaptación y normalización españolas de la Alzheimer’s Disease Assessment Scale (ADAS) (NORMACODEM) (y II) [Adaptation and normalization of the Alzheimer’s disease Assessment Scale for Spain (NORMACODEM) (II)] Neurología. 1997;12:69–77. [PubMed] [Google Scholar]
  30. Peña-Casanova J, Gramunt-Fombuena N, Quiñones-Ubeda S, Sánchez-Benavides G, Aguilar M, Badenes D, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for the Rey–Osterrieth Complex Figure (Copy and Memory), and Free and Cued Selective Reminding Test. Archives of Clinical Neuropsychology. 2009;24:371–393. doi: 10.1093/arclin/acp041. [DOI] [PubMed] [Google Scholar]
  31. Peña-Casanova J, Guardia J, Bertran-Serra I, Manero RM, Jarne A. Versión abreviada del test Barcelona (I): Subtests y perfiles normales [Normalization of cognitive and functional assessment instruments for dementia (NORMACODEM) (I): Objectives, content and population] Neurología. 1997;12:99–111. [PubMed] [Google Scholar]
  32. Peña-Casanova J, Monllau A, Böhm P, Aguilar M, Sol JM, Hernández G, et al. Valor diagnóstico y fiabilidad test-retest de la escala de demencia de Blessed (BDRS) para la enfermedad de Alzheimer: datos del proyecto NORMACODEM [Diagnostic value and test–retest reliability of the Blessed Dementia Rating Scale for Alzheimer’s disease: Data from the NORMACODEM project] Neurologia. 2005;20:349–355. [PubMed] [Google Scholar]
  33. Peña-Casanova J, Quiñones-Úbeda S, Gramunt-Fombuena N, Aguilar M, Casas L, Molinuevo JL, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for Boston Naming Test and Token Test. Archives of Clinical Neuropsychology. 2009;24:343–354. doi: 10.1093/arclin/acp039. [DOI] [PubMed] [Google Scholar]
  34. Peña-Casanova J, Quiñones-Úbeda S, Gramunt-Fombuena N, Quintana-Aparicio M, Aguilar M, Badenes D, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal fluency tests. Archives of Clinical Neuropsychology. 2009;24:395–411. doi: 10.1093/arclin/acp042. [DOI] [PubMed] [Google Scholar]
  35. Peña-Casanova J, Quiñones-Úbeda S, Gramunt-Fombuena N, Quintana M, Aguilar M, Molinuevo JL, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for the Stroop-Word Interference Test and the Tower of London–Drexel. Archives of Clinical Neuropsychology. 2009;24:413–429. doi: 10.1093/arclin/acp043. [DOI] [PubMed] [Google Scholar]
  36. Peña-Casanova J, Quiñones-Úbeda S, Quintana-Aparicio M, Aguilar M, Badenes D, Molinuevo JL, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for Verbal Span, Visuospatial Span, Letter and Number Sequencing, Trail Making Test, and Symbol Digit Modalities Test. Archives of Clinical Neuropsychology. 2009;24:321–341. doi: 10.1093/arclin/acp038. [DOI] [PubMed] [Google Scholar]
  37. Peña-Casanova J, Quintana-Aparicio M, Quiñones-Úbeda S, Aguilar M, Molinuevo JL, Serradell M, et al. Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for the Visual Object and Space Perception Battery–Abbreviated, and Judgment of Line Orientation. Archives of Clinical Neuropsychology. 2009;24:355–370. doi: 10.1093/arclin/acp040. [DOI] [PubMed] [Google Scholar]
  38. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment. Clinical characterization and outcome. Archives of Neurology. 1999;56:303–308. doi: 10.1001/archneur.56.3.303. [DOI] [PubMed] [Google Scholar]
  39. Pillon B, Dubois B, Bonnet AM, Esteguy M, Guimaraes J, Vigouret JM, et al. Cognitive slowing in Parkinson’s disease fails to respond to levodopa treatment: The 15-Objects Test. Neurology. 1989;39:762–768. doi: 10.1212/wnl.39.6.762. [DOI] [PubMed] [Google Scholar]
  40. Pineda DA, Mejía SE, Rosselli M, Ardila A, Romero MG, Pérez C. Variability of the Boston test for the diagnosis of aphasia in active working adults. Revista de Neurología. 1998;154:962–970. [PubMed] [Google Scholar]
  41. Rami L, Serradell M, Bosch B, Caprile C, Sekler A, Villar A, et al. Normative data for the Boston Naming Test and the Pyramids and Palm Trees Test in the elderly Spanish population. Journal of Clinical and Experimental Neuropsychology. 2008;30:1–6. doi: 10.1080/13803390701743954. [DOI] [PubMed] [Google Scholar]
  42. Rami L, Serradell M, Bosch B, Villar A, Molinuevo JL. Valores normativos de tests de función cognitiva frontal para la población mayor de 60 años [Standard values of frontal cognitive functioning tests for those over 60 years of age] Revista de Neurología. 2007;45:268–271. [PubMed] [Google Scholar]
  43. Rosen WG, Mohs RC, Davis KL. A new rating scale for Alzheimer’s disease. American Journal of Psychiatry. 1984;141:1356–1364. doi: 10.1176/ajp.141.11.1356. [DOI] [PubMed] [Google Scholar]
  44. Segarra N, Bernardo M, Gutiérrez F, Justicia A, Fernadez-Egea E, Allas M, et al. Spanish validation of the Brief Assessment in Cognition in Schizophrenia (BACS) in patients with schizophrenia and healthy controls. European Psychiatry. 2011;26:69–73. doi: 10.1016/j.eurpsy.2009.11.001. [DOI] [PubMed] [Google Scholar]
  45. Seshadri S, Fitzpatrick AL, Ikram MA, DeStefano AL, Gudnason V, Boada M, et al. Genome-wide analysis of genetic loci associated with Alzheimer disease. JAMA: The Journal of the American Medical Association. 2010;303:1832–1840. doi: 10.1001/jama.2010.574. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Snitz BE, Unverzagt FW, Chang CH, Bilt JV, Gao S, Saxton J, et al. Effects of age, gender, education and race on two tests of language ability in community-based older adults. International Psychogeriatrics. 2009;21:1051–1062. doi: 10.1017/S1041610209990214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Tinetti ME, Williams TF, Mayewski R. Fall risk index for elderly patients based 524 on number of chronic disabilities. American Journal of Medicine. 1986;80:429–434. doi: 10.1016/0002-9343(86)90717-5. [DOI] [PubMed] [Google Scholar]
  48. Wechsler D. WAIS–III Wechsler Adult Intelligence Scale–Third Edition Technical manual. San Antonio, TX: The Psychological Corporation; 1997a. [Google Scholar]
  49. Wechsler D. WMS–III Wechsler Memory Scale–Third Edition Administration and scoring manual. San Antonio, TX: The Psychological Corporation; 1997b. [Google Scholar]
  50. Wilde NJ, Strauss E, Tulsky DS. Memory span on the Wechsler scales. Journal of Clinical and Experimental Neuropsychology. 2004;26:539–549. doi: 10.1080/13803390490496605. [DOI] [PubMed] [Google Scholar]

RESOURCES