Demographically-Adjusted Norms for Selected Tests of Verbal Fluency: Results from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project

Abstract

Objective:

Verbal fluency tests are sensitive to various disorders affecting the central nervous system and are commonly included in neuropsychological evaluations. We aimed to develop normative data for two verbal fluency tests in a sample of native Spanish-Speakers living in the US-Mexico border region.

Method:

Participants included 254 adults from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project (Age: range=19–60; Education: range=0–20, 59% female). Participants completed two verbal fluency tests (i.e., letter [PMR] and semantic/category fluency [Animal Naming]) as part of a larger neuropsychological test battery. We examined linear and nonlinear effects of demographic factors (age, education, and gender) on verbal fluency raw scores, and developed T-scores using fractional polynomial equations controlling for demographics. We also calculated the rates of “impairment” (T-scores <40) that would be obtained by applying the newly developed norms and available norms for non-Hispanic English-speakers on comparable tests.

Results:

There were positive small effects of age and medium effects of education on verbal fluency raw scores. The normalized distribution of T-scores with the new norms showed expected psychometric properties. However, rates of impairment for both letter and semantic fluency were significantly higher when applying non-Hispanic White norms, and significantly lower when applying non-Hispanic Black norms.

Conclusions:

We provide norms for Spanish speakers living along the US-Mexico border region for two verbal fluency tests that are co-normed with a more extensive neuropsychological battery. These regional norms will improve interpretation of verbal fluency test performance in Spanish speakers living in the US-Mexico borderland.

Introduction

Verbal fluency is the ability to retrieve and express words, and is essential for optimal communication (Wysokiński et al., 2010). Language production and knowledge are cognitive functions assessed by verbal fluency tests. Yet, performance on these tests is also impacted by other cognitive abilities, including speed of information processing, executive control, attention, working memory, cognitive flexibility, response inhibition, and semantic memory (Henry & Crawford, 2004; Miyake & Friedman, 2012; Miyake et al., 2000; Rende, Ramsberger, & Miyake, 2002). Tests of verbal fluency are often used in neuropsychological assessments performed in the context of clinical practice and research, due to their ease of administration and sensitivity to a variety of conditions impacting the central nervous system. There are extensive data suggesting performance on tests of verbal fluency is impacted in a number of neurocognitive and neuropsychiatric conditions, such as stroke, traumatic brain injury, dementia and mood disorders (Henry & Crawford, 2004; Stolwyk, Bannirchelvam, Kraan, & Simpson, 2015; Van Den Berg, Jiskoot, Grosveld, Van Swieten, & Papma, 2017).

Verbal fluency tests typically consist of tasks of letter fluency, phonemic fluency, and semantic or category fluency. In letter fluency tasks, examinees are given one letter at a time (typically a total of three letters), and are asked to produce as many words as possible that begin with that letter, typically within one minute. Phonemic fluency tasks are similar, except examinees are asked to say words that begin with specific phonemes (sounds). In semantic verbal fluency tasks, examinees are asked to produce words within a given category, also generally within the span of one minute (Strauss, Sherman, & Spreen, 2006). The letters F, A, and S are commonly used with English-speaking samples for letter fluency tasks (Strauss et al., 2006). However, different combinations of letters are often used in different languages to assess letter fluency. For example, Artiola, Hermosillo, Heaton, and Pardee (1999) proposed the letters P, M, and R instead of F, A and S for Spanish-speakers, an adaptation made to minimize language effects based upon frequency of words within respective language (Artiola et al., 1999; Casals-Coll et al., 2013; Ordóñez et al., 2016). Similarly, other studies have used letters W, R, and G in Arabic populations (Khalil, 2010) and X (Chi), Σ (Sigma), and A (Alpha) in Greek populations (Kosmidis, Vlahou, Panagiotaki, & Kiosseoglou, 2004). In semantic fluency tasks, some of the most common categories are: animals, fruits and vegetables, occupations, grocery store items, kitchen items, and furniture (Casals-Coll et al., 2013; Da Silva, Petersson, Faísca, Ingvar, & Reis, 2004; Gocer March & Pattison, 2006; Piatt, Fields, Paolo, & Tröster, 1999; Price et al., 2012; Woods et al., 2005).

Certain demographic characteristics, such as age and education, have been shown to impact performance on verbal fluency tests. In general, verbal fluency performance increases curvilinearly with age, suggesting a gradual improvement from ages 6 to 12 and through early adulthood, to a subsequent slow decline in performance starting in the early 60s for both letter and categorical fluency tasks (Auriacombe, Fabrigoule, Lafont, Jacqmin-Gadda, & Dartigues, 2001; Gustavson et al., 2018; Harris & Deary, 2011; Miranda et al., 2012; Van Der Elst, Van Boxtel, Van Breukelen, & Jolles, 2006). In regards to education, persons with more years of formal education tend to achieve higher scores on both tasks (Guerrero-Berroa et al., 2016; Olabarrieta-Landa et al., 2015; Rosselli, Tappen, Williams, Salvatierra, & Zoller, 2009; Strauss et al., 2006; Van Der Elst et al., 2006). Gender is another common demographic characteristic that has been examined in terms of performance on verbal fluency tasks. However, data addressing the effect of gender are inconsistent. Some studies have found women outperformed men (Burton, Henninger, & Hafetz, 2005; Capitani, Barbarotto, & Laiacona, 2005; Halari et al., 2006; Munro et al., 2012; Wallentin, 2009; Weiss et al., 2003), or men outperform women (Kempler, Teng, Dick, Taussig, & Davis, 1998; Olabarrieta-Landa et al., 2015; Tombaugh, Kozak, & Rees, 1999; Van Der Elst et al., 2006), while others have found no significant gender differences (Gladsjo et al., 1999; Mathuranath et al., 2003; Olabarrieta-Landa et al., 2015; Polunina, Bryun, Sydniaeva, & Golukhova, 2018). These demographic variables’ well-studied influence on verbal fluency test performance underscores the need for normative data that considers the impact of these factors’ effects in order to more accurately detect underlying brain dysfunction.

Language plays an instrumental role in verbal fluency tasks (Vygotsky, 1962, 1989). Different languages are different in their grammar, phonology, pragmatics, lexicon, and reading system, which may affect language test performance. The majority of norms on verbal fluency tests have been developed in English-speaking populations in the United States of America (U.S.) and Canada (Henrich, Heine, & Norenzayan, 2010; Ruff, Light, Parker, & Levin, 1996; Tombaugh et al., 1999), but normative data on verbal fluency tests are also available in a number of other languages and countries [i.e., Cantonese (Chan et al., 2002; Chiu et al., 1997), Dutch (Van der Elst, Hurks, Wassenberg, Meijs, & Jolles, 2011; Van Der Elst et al., 2006), Italian (Costa et al., 2014), Greek (Konstantopoulos, Vogazianos, & Vayanos, 2014; Kosmidis et al., 2004); English/Afrikaans (de Picciotto & Friedland, 2001); Japanese (Abe et al., 2004), Portuguese (Brocki & Bohlin, 2004; Esteves et al., 2015; Machado et al., 2009) Swedish (Tallberg, Ivachova, Jones Tinghag, & Östberg, 2008), and Arabic (Khalil, 2010)]. Among Spanish-speakers, norms have been developed in Spain (Artiola et al., 1999; Benito-Cuadrado, Esteba-Castillo, Böhm, Cejudo-Bolivar, & Peña-Casanova, 2002; Buriel, Gramunt, Bohm, Rodes, & Pena-Casanova, 2004; Casals-Coll et al., 2013; Contador et al., 2016; Nieto, Galtier, Barroso, & Espinosa, 2008; Villodre et al., 2006; Wysokiński et al., 2010), Latin America [i.e., Mexico, Guatemala, Colombia, Venezuela, Perú, Argentina] (Ardila & Rosselli, 1994; Butman, Allegri, Harris, & Drake, 2000; Labos, Trojanowski, del Rio, Zabala, & Renato, 2013; Morlett Paredes, 2018; Olabarrieta-Landa et al., 2015; Ostrosky-Solís, Ardila, & Rosselli, 1999; Pontón et al., 1996; Rivera et al., 2019; Sosa et al., 2009) and in the U.S. (Artiola et al., 1999; O’Bryant et al., 2018; Pontón et al., 1996; Taussig, Henderson, & Mack, 1992).

There are over 37 million Spanish-speakers in the U.S. (Ryan, 2013) making the U.S. the second country in the world with the largest number of Spanish-speakers. Spanish-speakers in the U.S. are quite heterogenous, tracing their origin to various Spanish-speaking countries around the world, and having varying degrees of English proficiency (Ryan, 2013). Among U.S. dwelling Spanish-speakers, letter fluency normative data are available for the letters P, M and R (Artiola et al., 1999), and F, A and S (O’Bryant et al., 2018; Pontón et al., 1996; Taussig et al., 1992). Semantic fluency normative data are available for the following categories: animals (Acevedo et al., 2000; González, Mungas, & Haan, 2005; LaRue, Romero, Ortiz, Chi Lang, & Lindeman, 1999; O’Bryant et al., 2018; Taussig et al., 1992), vegetables (Acevedo et al., 2000), fruits (Acevedo et al., 2000), and supermarket items (Taussig et al., 1992). Most of these normative efforts include samples of older adults (Acevedo et al., 2000; González et al., 2005; LaRue et al., 1999; O’Bryant et al., 2018; Taussig et al., 1992), and some primarily middle-aged adults (Artiola et al., 1999; Pontón et al., 1996), and most typically include adults with wide ranges of education (0–12+ years of education; (Acevedo et al., 2000; González et al., 2005; LaRue et al., 1999; O’Bryant et al., 2018; Pontón et al., 1996). None of the published studies to date with Spanish-speakers in the U.S. provide norms for both the PMR and a category fluency test, along with being co-normed together with a battery of neuropsychological tests.

The main purpose of the present study was to provide norms for two verbal fluency tests, i.e., letter fluency (P, M, & R) and semantic fluency (Animal Naming) in a sample of Spanish-Speakers living in the US-Mexico border region, who also completed a larger battery of neuropsychological tests as part of their participation in the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project. In order to examine whether existing norms for other cultural/linguistic groups might apply to the current sample of Spanish-speakers, we compared the rates of “impairment” classifications obtained by the newly established norms to those based on the application of existing published norms for English-speaking non-Hispanic Whites and Blacks in the U.S. for similar tests.

Methods

Participants

Two-hundred and fifty-four native Spanish speakers living in or near the U.S.-Mexico border region of San Diego, California (n=152) and Tucson, Arizona (n=102) completed measures of letter (i.e., PMR) and semantic (Animal Naming) fluency as part of their participation in the NP-NUMBRS Project. Participants were recruited via flyers in the community and in-person presentations by study staff in Hispanic serving community and health-care organizations in San Diego, CA and Tucson, AZ. Inclusion criteria were: being primarily Spanish-speaking and residing in or near San Diego (California) and Tucson (Arizona). Potential participants were screened for exclusionary conditions using a structured interview. Exclusion criteria were: central nervous system disorder or other medical condition (e.g., chronic obstructive pulmonary disease, or significant peripheral injuries or disabilities) that may influence test performance; serious psychiatric conditions; and any lifetime substance dependence. Less serious conditions (e.g., hypertension, metabolic disorders, or certain peripheral injuries) were reviewed for inclusion by the senior investigators on a case-by-case basis. Age of the sample ranged from 19 to 60 years, education ranged from 0 to 20 years, and a little over half of the sample was female (Table 1). See Cherner, Marquine and colleagues (this issue) for further details regarding participants.

Table 1.

Demographic Characteristics of the Study Cohort (N=254)

Demographic Characteristic	Descriptive Statistic
Age
M (SD)	37.32 (10.24)
n (%)
19–29	65 (25.59)
30–39	82 (32.28)
40–49	71 (27.95)
50–60	36 (14.17)
Education
M (SD)	10.67 (4.34)
n (%)
≤6	58 (22.84)
7–10	56 (22.05)
11–12	64 (25.19)
≥13	76 (29.92)
Female, n (%)	149 (58.66)
Right handed, n (%)a	222 (97.37%)

Procedures

Procedures followed in the NP-NUMBRS project were approved by the University of California San Diego (UCSD) Human Subjects Committee Internal Review Board. Participants completed a comprehensive neuropsychological test battery assessing multiple domains in addition to verbal fluency (the focus of the present report) as follows: speed of information processing (Rivera Mindt, Marquine et al, this issue; Suárez, Diaz Santos, Marquine et al., this issue), attention/working memory (Gooding et al., this issue; Scott et al., this issue), executive functioning (Morlett Paredes, Carrasco et al, this issue; Suárez Diaz Santos, Marquine et al., this issue; Marquine, Yassai Gonzales et al, this issue), learning and memory (Díaz-Santos et al., this issue), visuospatial skills (Scott et al., this issue), and fine motor skills (Heaton et al., this issue). Please see Cherner, Marquine et al (this issue) for a list of specific tests by domain. Participants were tested in Spanish by bilingual (Spanish/English) study staff. Data for the NP-NUMBRS project were collected in two waves: from 1998 to 2000 (Cohort 1: n=183) and from 2006 to 2009 (Cohort 2: n=71; see Cherner, Marquine et al, this issue, for further details on study procedures.)

The PMR letter fluency test was administered following procedures described in Artiola I. Fortuny and colleagues (Artiola I. Fortuny et al., 1999; please see manual for specific instructions). Briefly, participants were asked to say all the words that they could think of that began with the letters “P”, “M” and “R”, with 60 seconds allotted for each letter. Proper names and plurals, and other versions of the same word were not allowed. Animal Naming was administered following procedures from Gladsjo and colleagues (1999). Participants were asked to produce the names of animals within a span of 60 seconds. Test instructions were adapted to Spanish, and reviewed and approved by a group of bilingual neuropsychologists and staff familiar with the practice of neuropsychology (see Appendix for instructions in Spanish, and Cherner, Marquine et al, this issue, for further details on procedures followed for the adaptation to Spanish).

Demographically corrected T-Scores are reported for the following scores:

• PMR Total Correct: Sum of correct words generated in the PMR test for each letter, excluding intrusions, proper names, repetitions and variation of the same word (e.g., if the participant said “pedir, pediste, pidió”, only the first word was scored as correct).

• Animal Naming Total Correct: Sum of correct words generated in the Animal Naming task, excluding intrusions, repetitions and variations of the same word (e.g., if the participant said “vaca, vaquita”, only the first word was scored as correct). Both the general category and specific animal in a given category were scored as correct (e.g. If the participant said: “reptil, vívora”, both words were scored as correct).

We also provide percentile scores for the following scores:

• PMR Perseverative Responses: Sum of all repeated words for letters P, M and R.

• PMR Intrusion Errors: Words not beginning with the specified letter for each trial and proper nouns (i.e., “P” in the P trial, “M” in the M trial, “R” in the R trial).

• Animal Naming Perseverative Responses: Sum of repeated animal names in the Animal naming task.

• Animal Naming Intrusion Errors: Sum of words that were not the name of animals in the Animal Naming task.

A subset of participants completed self-report questionnaires, which collected information on educational, social, and language use background. Current performance-based language fluency was assessed via administration of the Controlled Oral Word Association Test with letters F-A-S in English (Strauss, et al., 2006) and P-M-R in Spanish. A language fluency ratio was computed based on words produced in the FAS task to total words in both tasks (language fluency ratio = FAS/[FAS+PMR]), and used to classify participants as Spanish dominant (scores ≤ 0.33), English-dominant (scores ≥ 0.67) and bilingual (scores: 0.34 to 0.66; Miranda et al., 2016). Cherner, Marquine and colleagues (this issue) provides further details on the collection of educational, social, and language factors.

Statistical Analyses

We computed descriptive statistics of demographic factors (e.g., age, education, gender) and raw scores on PMR and Animal Naming Total Correct, Intrusions and Perseverative Responses. We examined the distribution of verbal fluency raw scores via Shapiro-Wilk tests. We then examined the linear and non-linear association of age and education with verbal fluency raw scores via a series of univariable linear regression analyses, and the association between gender and verbal fluency raw scores via independent sample t-tests (or Wilcoxon Rank Sum tests for variables with skewed distributions). To investigate interaction effects of demographics on verbal fluency total raw scores we ran a series of linear regression analyses with two-way interaction terms of demographics as predictors (i.e., age X education, age X sex, education X sex) on PMR and Animal Naming Total Scores.

Raw scores were converted to normalized Scaled Scores (mean of 10 and SD of 3) by measuring standardized quantiles of the raw scores and scaling them to have a mean of 10 and standard deviation (SD) of 3, with higher values correspond to better performance. T-Scores for measures with adequate distribution were obtained by applying the fractional polynomial equations to regress Scaled Scores on age, education, and gender, and rescaling the resulting residuals to have a mean of 50 and SD=10. The fractional polynomial method allowed for the consideration of linear and non-linear effects for age and education, and selected the best curve (p<0.05) from various options: linear, quadratic, logarithmic, and other combinations of fractional polynomials of first (e.g. x^m) and second degree (e.g. x^m1 + x^m2) with powers (m_i) ranging from −2 to +3. Further details can be found in Cherner, Marquine and colleagues (this issue). We then examined the descriptive characteristics of the resulting T-Scores and their distribution via Shapiro-Wilk tests, and investigated the association of age and education with the newly developed T-Scores via Pearson product moment correlation coefficients, and the association of gender with T-Scores via independent samples t-tests. We also examined site and cohort effects on T-Scores via a series of independent samples t-tests, as well as the effect of bilingualism (as defined by a ratio of letter fluency in English to total letter fluency: FAS/[FAS+PMR]) on T-Scores via Pearson product moment correlation coefficients in a subset of the sample with available data on both PMR and FAS.

To examine whether norms developed for other cultural/linguistic groups might be applicable to the current sample of Spanish-speakers, we then calculated T-Scores for the raw scores of both verbal fluency tests based on published norms for English-speaking non-Hispanic Whites and non-Hispanic Blacks/African Americans in the U.S. (Heaton, Miller, Taylor, & Grant, 2004). Of note, for the letter fluency test, norms for the letters F, A, and S were used based on their availability in English-speakers. Rates of “impairment” (T-Score < 40) were computed utilizing the published norms and the newly developed norms. We ran McNemar’s tests to compare rates of impairment in the current sample when utilizing current Spanish-speaking norms and previously published non-Hispanic English-speaking norms. We also examined the association of demographic factors with T-Score (continuous) based on non-Hispanic norms via Pearson product moment correlation coefficients and independent samples t-tests.

Results

Educational, social and language characteristics of the study sample

Table 2 provides educational, social, and language background characteristics of a subset of participants in the study sample. The majority of participants completed most of their formal education in their country of origin (rather than in the U.S.), and approximately half attended a large school. Almost a third of the sample had to stop attending school to work. Parental education was on average 6 years. Participants lived most of their lives in their country of origin. Most participants self-reported their childhood socioeconomic status as middle class, with over a third reported having been poor/very poor. Approximately half of participants reported working for money during childhood, and approximately two-thirds of participants were gainfully employed at the time of their participation in the present study. All but one of the participants reported that Spanish was the first language they learned. Average ratings of language used in various everyday activities indicated that Spanish was the predominant language used in daily life. Nearly two-thirds of the sample was monolingual Spanish-speaking/strongly Spanish dominant, with the remaining third being bilingual.

Table 2.

Demographic, Educational, Social, and Language Background Characteristics of the Study Sample

	Descriptive Characteristics
	M(SD), %	n
Educational Background
Years of education in country of origin	8.53 (4.80)	227
Years of education in the U.S.	2.53 (4.73)	227
Proportion of education by country		227
More years of education in country of origin	84.14%	191
More years of education in the U.S.	14.98%	34
Equal number of years of education in both countries	0.88%	2
Type of school attendeda		243
Large	55.56%	135
Regular	39.92%	97
Small	4.53%	11
Number of students in the class		247
Less than 21	15.39%	38
21 to 30	39.27%	97
31 to 40	24.29%	60
40+	21.05%	52
Had to stop attending school to work		224
Yes	28.57%	64
Social Background
Mother’s years of education	5.76 (3.65)	180
Father’s years of education	6.79 (5.06)	163
Years lived in country of origin	26.41 (12.49)	245
Years living in the U.S.	10.69 (10.85)	245
Childhood SESb		251
Very poor	5.98%	15
Poor	27.09%	68
Middle class	58.17%	146
Upper class	8.77%	22
Worked as a child		248
Yes	52.82%	131
Reason to work		131
Help family financially	38.17%	50
Own benefit	61.83%	81
Age started working as a child	12.98 (3.18)	131
Currently Gainfully Employed		224
Yes	68.75%	154
Language
First Language		250
Spanish	98.40%	246
English	0.40%	1
Both	1.20%	3
Current Language Use Ratingc		251
Radio or TV	2.37 (1.03)	--
Reading	2.24 (1.18)	--
Math	1.54 (1.05)	--
Praying	1.26 (0.72)	--
With family	1.56 (0.89)	--
Performance-based language fluency		203
Spanish dominant	62.07%	126
English dominant	0.00%	0
Bilingual	37.93%	77

Note. M: mean; SD: standard deviation. SES: socioeconomic status

^aType of school attended: large=school that had many classrooms and room to play; regular=a school of regular size that had at least one classroom per grade and room to play; small= small school with less than one classroom per grade.

^bChildhood SES was assessed by the following question and response options: “As a child, your family was: (1) Very Poor; (2) Poor; (3) Middle Class); (4) Upper Class”.

^cRatings for each activity ranged from 1 (Always in Spanish) to 5 (Always in English), with a rating of 3 being “Similarly in English and Spanish”.

Raw Scores to Scaled Scores Conversions

Table 3 shows the descriptive summary characteristics of raw scores on PMR and Animal Naming Total Correct, Intrusion Errors and Perseverative Responses. Although the range of Intrusions for PMR was relatively large, only one participant had 15 intrusions and another one had 7 intrusions. All intrusions for these two participants were proper names. PMR Total Correct raw scores were normally distributed, but PMR Intrusion Errors and Perseverative responses and all the Animal Naming raw scores were not.

Table 3.

Mean, standard deviation, and range of the PMR and Animal Naming

	Mean (SD)	Range
PMR Total Correct	39.01 (12.25)	9 – 82
PMR Perseverative Responses	1.05 (1.42)	0–10
PMR Intrusion Errors	0.60 (1.38)	0–15
Animal Naming Total Correct	20.37 (4.65)	8 – 37
Animal Naming Perseverative Responses	0.62 (0.89)	0–6
Animal Naming Intrusion Errors	0.11 (0.33)	0–2

Figures 1a–d show the association of age and education to Total Correct raw scores for both PMR and Animal Naming. There were small linear effects of age on PMR raw scores, nonlinear effects of age on Animal naming raw scores, and medium linear effects of education on both verbal fluency test raw scores. There were no significant gender differences on either PMR (p=.55, Cohen’s d=0.07) or Animal Naming (p=.93, Cohen’s d=−0.03) Total Correct raw scores, with comparable performances between men (M=38.50, SD=12.21; and M=20.44, SD=4.89, respectively) and women (M=39.36, SD=12.30; and M=20.33, SD=4.48, respectively) on these tasks. There were no significant two-way interaction effects of demographic characteristics on PMR or Animal Naming Total Correct raw scores.

Figure 1.

Association of PMR Total correct raw scores and age (Fig 1a; PMR = 32.91 + 0.16*age, p = 0.03, R^2 = 1.9%), PMR Total correct raw scores and education (Fig 1b; PMR = 26.78 + 1.15*education, p < 0.001, R^2 = 16.5%), Animal Total Correct raw scores and age (Fig 1c; Animal Naming = 17.61 + 0.00066*agê3 – 0.00016*log(age)*agê3, p< 0.01, R^2 = 3.8%), and Animal Total Correct raw scores and education (Fig 1d; Animal Naming = 16.42 + 0.37*education, p < 0.001, R^2 = 12.0%).

Table 4 shows the raw-to-scaled score conversions for PMR and Animal Naming Total Correct scores. Of note, although Animal Naming total correct raw scores were not normally distributed, their Scaled Scores conversions corrected this. Given the highly skewed distribution and limited range of intrusions and perseverations for the PMR and Animal Naming, we only provide raw to percentile conversions for these scores (Table 5).

Table 4.

Raw-to-Scale Score Conversions

Scaled	PMR Total Correct	Animal Naming Total Correct
19	170 – 240	59 – 80
18	77 – 169	34 – 58
17	69 – 76	31 – 33
16	63 – 68	30
15	59 – 62	28 – 29
14	54 – 58	26 – 27
13	49 – 53	25
12	45 – 48	23 – 24
11	41 – 44	21 – 22
10	37 – 40	20
9	33 – 36	18 – 19
8	29 – 32	17
7	26 – 28	16
6	22 – 25	15
5	19 – 21	13 – 14
4	16 – 18	12
3	14 – 15	10 – 11
2	5 – 13	5 – 9
1	0 – 4	0 – 4

Table 5.

Raw Scores to Percentiles Conversions for Perseverations and Intrusions for the PMR and Animal Naming Tests

Raw Score	Percentile
	PMR Perseverative Responses	PMR Intrusion Errors	Animal Naming Perseverative Responses	Animal Naming Intrusion Errors
0	≥57th	≥33rd	≥43rd	≥10th
1	26th	12th	13th	1st
2	10th	5th	4th	--
3	4th	3rd	1st	--
4	2nd	2nd	--	--
5 or more	1st	1st	--	--

T-Scores Equations

Table 6 shows the T-Scores equations used to compute individual T-Scores. Please see supplemental material with digital calculator to facilitate the computation of T-Scores based on these formulas. The resulting T-Scores were normally distributed, with a mean of 50 and a SD of 10 (Figures 2a–2b). T-Scores ranged from 27 to 80 for PMR Total Correct and from 21 to 85 for Animal Naming Total Correct. There were no significant effects of demographic factors on any of the T-Scores (ps>.44). There were also no differences on PMR Total Correct T-Scores by site (Arizona: M=49.17, SD=8.99; California: M=50.17, SD=10.77; p=.51; Cohen’s d=0.10) or cohort (Cohort 1: M=49.62, SD=9.80; Cohort 2: M=50.91, SD=10.51; p=.37, Cohen’s d=0.13), as well as no significant differences on Animal T-Scores by site (Arizona: M=51.47, SD=9.94; California: M=49.73, SD=10.87; p=.27; Cohen’s d=0.17) or cohort (Cohort 1: M=50.69, SD=10.37; Cohort 2: M=48.22, SD=8.83; p=.06, Cohen’s d=0.25). Bilingualism was not significantly associated with demographically adjusted T-Scores derived from the present norming project on tests of verbal fluency in Spanish (PMR: r=−0.04, p=.63; Animal Naming: r=0.12, p=.12).

Table 6.

T-Score Equations

Measure	Equation
PMR Total Correct	$10\times \left(\frac{\text{SS PMR Total Correct}–\left(5.06325+4.24915*\frac{\mathit{\text{age}}}{100}+2.92761*\frac{\left(\text{edu}+1\right)}{10}–0.25399*\text{gender}\right)}{2.69214}\right)+50$
Animal Naming Total Correct	$10\times \left(\frac{\text{SS Animal Naming Total Correct}–\left(5.0559–66.24736*{\left(\frac{\mathit{\text{age}}}{100}\right)}^3–117.74239*{\left(\frac{\mathit{\text{age}}}{100}\right)}^3*\text{log}\left(\frac{\mathit{\text{age}}}{100}\right)+2.55247*\frac{\left(\text{edu}+1\right)}{10}–0.06000*\text{gender}\right)}{2.79774}\right)+50$

Note. These formulas should be applied to education level ranges from 0–20 and age 19–60. Using values outside these ranges might result in extrapolation errors.

Gender: Male=1; Female=0

Edu=years of education

Age= years of age

The combined effect of age, education, and gender on PMR Total Correct is estimated by adjusted R²=0.18 and on Animal Naming Total Correct is adjusted R²=0.16.

Figure 2.

Distribution of PMR (Fig 2a) and Animal (Fig 2b) Total Correct T-Scores based on applying current norms (blue), non-Hispanic White norms (orange) and non-Hispanic Black norms (green)

Applications of Existing Norms

Figures 2a and 2b present the distributions of T-Scores on verbal fluency tests based on applying current norms and published norms for English-speaking non-Hispanic Whites and Blacks (Heaton et al., 2004) to the present sample of native Spanish-speakers. While the distribution of T-Scores shifted based on the set of norms used, the resulting T-Scores for both verbal fluency tests were normally distributed for all norms. Utilizing non-Hispanic White norms, we found there was a significant positive association between older age and higher T-Scores on letter (r=0.27, p<.001) and category fluency (r=0.32, p<.001), and between higher education and higher PMR T-Scores (r=0.15, p=.02), with no other significant demographic effects on the verbal fluency T-Scores based on non-Hispanic White norms (ps>.51). When utilizing non-Hispanic Black norms, older age was also significantly associated with better T-scores on letter (r=0.23, p<.001) and animal fluency (r=0.19, p<.01), and females had higher T-Scores on Animal Naming (M=56.62, SD=8.54) than males (M=54.02, SD=9.16; p=.02), with no other significant demographic effects (ps>.16).

Figure 3 shows rates of impairment (T-Scores<40) based on newly developed and published norms (Heaton et al., 2004). Given the normalized distribution of T-Scores, 15–16% of the sample would be expected to fall in the impaired range utilizing the population-specific norms. As shown in Figure 1, results of McNemar’s test indicate that norms based on English-speaking non-Hispanic Whites significantly overestimated rates of impairment in the current sample for both verbal fluency tests (ps ≤.003), while norms based on English-speaking non-Hispanic Blacks significantly underestimated impairment for both fluency tests (ps ≤ .0001).

Figure 3.

Rates of impairment based on published norms for non-Hispanic (NH) Whites and NH Blacks (Heaton et al), and newly developed norms in verbal fluency tests. Impairment was defined as T-Score <40.

Discussion

The present study developed normative data for two verbal fluency tests, i.e., letter (PMR) and semantic fluency (Animal Naming), in a sample of native Spanish-speakers living in the US-Mexico border region of Arizona and California. We found age and education were significantly associated with raw performance on these tests. We developed demographically adjusted T-scores, which performed as psychometrically expected, with no significant differences based on where participants resided (Arizona or California) or when the data were collected (Cohort 1 or 2), and no significant effects of bilingualism on the demographically-adjusted T-scores. When we applied published normative data for English-speaking non-Hispanics to our sample of Spanish-speakers, we found that these normative data significantly misclassified impairment. In addition, the published norms for English-speakers did not fully correct for demographics in the current Spanish-speaking sample, whereas the new population-specific norms did.

Similar to prior studies (Guerrero-Berroa et al., 2016; Olabarrieta-Landa et al., 2015; Rosselli et al., 2009; Strauss et al., 2006; Van Der Elst et al., 2006), we found that more years of formal education were associated with better performance on both verbal fluency tests. There are likely various reasons for these associations. They might include factors associated with language ability (i.e., formal education may increase vocabulary knowledge), as well as factors related to neuropsychological test performance more broadly, such as the fact that school provides contents and processes frequently included in cognitive testing [e.g., timed tests; Ardila, Ostrosky-Solis, and Mendoza (2000)], and that exposure to cognitively stimulating experiences in early life via educational experiences can enhance brain development and impact cognitive ability later in life (Britto et al., 2017; Fox, Levitt, & Nelson III, 2010; Hutton et al., 2015; Noble et al., 2015; Thompson, 2001). Thus, many factors can influence educational attainment, including socioeconomic status and lack of access to education, among many others.

We found small positive effects of age on PMR Total Correct raw scores (i.e., older age was associated with better performance), and a non-linear association between age and Animal naming Total Correct raw scores, such that there was a small positive effect of age on Animal naming raw scores until the late 40s, and then raw scores declined slightly. These findings in Animal naming are consistent with those of prior studies, showing a curvilinear effect of age on verbal fluency (Auriacombe et al., 2001; Gustavson et al., 2018; Harris & Deary, 2011; Van Der Elst et al., 2006). However, present results of no curvilinear effect of age on PMR are somewhat inconsistent with these prior findings. This may be at least partly explained by the age range of our study sample (19 to 60 years), as most previous studies show a notable improvement in verbal fluency performance until age 16–17 years and a decline starting at age 60. While some prior studies have found gender effects on verbal fluency tests, others have not. Our results are in line with these latter findings and lend support to the lack of gender effects on verbal fluency tests.

Intrusions were relatively infrequent in this sample of healthy Spanish-speakers, with the exception that a small number of participants had a notable number of intrusions on PMR. When this occurred, the type of intrusion errors was almost exclusively due to the production of proper names in this task. Instructions for the PMR utilized in the present study did not include re-orienting the examinees after they produced intrusions, as it is the case in some versions of verbal fluency tests. Instructions used in the present study might have made it more likely for some examinees to produce a large number of incorrect responses.

The T-Scores resulting from the norms developed in the present study performed as expected psychometrically (i.e., they were normally distributed, had a range of 50 and SD of 10, were not significantly associated with demographic variables, and resulted in 15–16% impairment based on T-Scores<40). In contrast, applying norms developed for English-speaking non-Hispanics resulted in notable misclassification of impairment in the present Spanish-speaking sample. This finding illustrates the pitfalls of applying norms developed for other cultural/linguistic groups to interpret test performance. Neuropsychological test performance can be impacted by factors that are not the result of underlying brain impairment. Our normative approach adjusted for demographic characteristics (age, education, sex) that have consistently been found to influence test performance. Yet, these demographic effects often differ across cultural/racial groups, as evidenced in present findings showing significant effects of demographics on test performance when norms for other groups were applied. There are also several other potential variables whose effect on test performance are more difficult to quantify, including familiarity with testing; stereotype threat, and other sociocultural and linguistic factors. This is where normative data developed for cultural/linguistic groups similar to those of the examinee could improve diagnostic accuracy for brain dysfunction (Marquine, Rivera-Mindt et al., This Issue).

It is unlikely that language alone explained findings of differential rates of impairment when different normative adjustments where applied, given that utilizing norms for non-Hispanic Whites resulted in significant overestimation of impairment, while utilizing norms for non-Hispanic Blacks resulted in significant underestimation of impairment in the overall sample. The fact that norms for the letter fluency task were based on different letters (F, A and S instead of P, M, and R) might explain, at least in part, misclassification of impairment on this task. However, our findings showing that similar patterns of misclassification were observed in the semantic fluency task, are in line with the notion that the use of different letters by language do not fully account for the misclassifications. The reasons for differences in classification rates of neurocognitive impairment across norms are likely varied, and might include demographic differences across norming samples. We found that education and age were linked to our sample’s letter fluency T-Scores based on non-Hispanic White norms, suggesting that these norms do not accurately account for the effect of these factors on verbal fluency tests among Spanish-speakers. Relatedly, the sample in which English-speaking norms were developed (Heaton et al., 2004) had on average higher years of education (M=14.4, SD=2.6 for non-Hispanic Whites and M=13.3, SD=2.4 for non-Hispanic Blacks) than the current sample (M=10.7, SD=4.4), underscoring the potential impact of the limited representation of persons with very low levels of education in the norming sample of non-Hispanic Whites. Further, females had higher T-Scores on Animal Naming than males when using norms for non-Hispanic Blacks, indicating that gender effects might differ across cultures.

Other reasons for different rates of impairment when using different norms might include the varying degrees of use of other languages (i.e., bilingualism, multilingualism) and the exposure to testing situations that resemble neuropsychological testing differ across ethnic/racial groups in the US, along with other sociocultural differences. Data on some language use and other culturally-relevant factors were collected in a subset of the current sample (see Table 2). The examination of the impact of bilingualism on neuropsychological test performance in the entire NP-NUMBRS Project battery is examined in another manuscript in this issue (Suarez, Marquine, Diaz-Santos, Gollan et al, this issue). In the present study, we found that among the subset of participants with available data, degree of bilingualism (as defined by a ratio of English letter fluency to total letter fluency) did not significantly contribute to variance in verbal fluency test performance above and beyond the standard demographic adjustments for age, education, and gender developed in the present project. While degree of English fluency may need to be considered when interpreting performance on some tests (see Suarez, Marquine et al., this issue), the normative adjustments provided by the NP-NUMBRS project appear applicable for tests of verbal fluency in primarily Spanish speakers with a range of English language ability. In interpreting these findings it is worth considering that the current sample included only participants who were primarily Spanish-speaking (both monolingual and bilingual), but excluded primarily English-speaking bilinguals. For further discussion of the effects of bilingualism on the entire NP-NUMBRS battery, please see Suarez, Marquine and colleagues (this issue). We are also currently investigating how the complex and interrelated sets of factors presented in Table 1 might impact performance in the entire battery (including verbal fluency tests) in our sample of primarily Spanish-speakers living in the US-Mexico border region. The educational, social, and language characteristics of the study sample described in the present report, might aid clinicians and researchers in deciding whether the normative data presented here might apply to their patients and population of study. Future studies investigating whether different sets of norms developed for Spanish-speakers in the U.S. work similarly across populations, would be helpful to better aid clinicians and researchers in identifying the normative sets that might best applied to the populations they serve.

The present study has some important limitations and strengths that merit discussion. One limitation of the present study is that we included only native Spanish-speakers living in the U.S. (Arizona and California) and close to the US-Mexico border, who were primarily of Mexican origin descent. Participants were recruited in or near Tucson (Arizona) and San Diego (California) and thus findings are not necessarily representative of the entire population living along the U.S.-Mexico border region. Caution is warranted when applying the norms developed in the present study to other Spanish-speaking subgroups based on country of origin and region of residency within the US, given findings that neurocognitive test performance might vary across these dimension (González et al., 2014; Marquine et al., 2018). Hispanics are a highly heterogeneous group, and include persons from various origins, language use, level of acculturation, place of birth, among other factors, and the full effect of these variables on neurocognitive test performance is not fully understood at this time. Future studies, including large and diverse groups of Hispanics/Latinos living in and outside the U.S., would be best suited for the examination of these factors. It is also worth noting that our sample included persons aged 19–60 years, and thus the utilization of the present norms in persons outside of these age ranges is generally discouraged, and should be done only after careful consideration of other available norms.

Also of note, the data for the present study were gathered a number of years ago, raising the question as to whether more current cohorts of Spanish-speakers might be showing a Flynn effect (Flynn, 2007). Relatedly, changes in characteristics of the population in the border region over time might impact the applicability of the current norms. While we cannot rule out that this might be the case, our findings indicate that there were no significant differences in test performance between Cohorts 1 and 2, which would be inconsistent with this notion. Future studies examining the applicability of the norms presented here to current samples in the region will be able to best answer this question. Of note, most normative data on verbal fluency tests available for Spanish-speakers in the U.S. were also collected before or around the same time that the current data were (Acevedo et al., 2000; Artiola et al., 1999; González, Mungas, & Haan, 2005; LaRue et al., 1999; Pontón et al., 1996; Taussig et al., 1992), with the exception of those from the Texas Mexican American Adult Normative studies (O’Bryant et al., 2018), which included participants 40 years and over. Despite the limitations of the present study, it is imperative to note that this is the first published study to provide norms for both the Spanish language PMR and a category fluency test, and to offer multiple verbal fluency tests that were co-normed together with a comprehensive battery of neuropsychological tests among Spanish-speakers living in the U.S. with a wide age range (19 – 60 years).

Lastly, it is critical to note that these demographically-adjusted norms for verbal fluency tests are not a panacea for the challenge of conducting valid, meaningful neuropsychological evaluations and data interpretations with culturally/linguistically diverse populations. To this point, Manly and Echemendia (2007) offer a useful overview of the appropriate use of demographically-adjusted normative data to improve diagnostic validity that is consistent with the goals of this study. Moreover, this study does not espouse the use of demographically-adjusted norms as the sole approach to address current assessment and diagnostic challenges when working with diverse populations. Instead, readers are encouraged to utilize evidence-based approaches to integrating normative data with more granular sociocultural data approaches (please see Rivera Mindt et al., 2019, Rivera Mindt et al., 2010, Rivera Mindt et al., 2008).

Overall, the present study has implications for neurocognitive clinical practice and research. The normative data developed in the present study represent a useful tool for clinicians and researchers interested in detecting underlying brain impairment based on the use of neurocognitive tests among Spanish-speaking adults living in the U.S. (particularly those of Mexican origin/descent and living close to the U.S.-Mexican border, ages 19–60 years). To that end, these normative data are intended to be utilized in concert with more granular sociocultural assessments (e.g., language/bilingualism) also known to impact neurocognitive test performance. In addition, the fact that the normative data presented was obtained as part of a larger project allowed for co-norming, and thus will facilitate comparisons of neurocognitive performance across tests within a single individual and group of individuals (Rivera-Mindt, Marquine, Aghvinian, Morlett Paredes et al., This Issue). While much work remains to be done regarding our understanding of sociocultural factors that might impact neurocognitive test performance among Spanish-speakers in the U.S., the present study adds to the accumulating literature in the field on the development of neuropsychological test norms in an effort to improve diagnostic validity of commonly used verbal fluency tests within this population.

Appendix

Animal Naming Instructions

Examiner: “Ahora vamos a hacer algo diferente. Esta vez me gustaría que dijera todos los animales que pueda. No importa con qué letra empiecen. Simplemente diga todos los animales que pueda. Tiene alguna pregunta?”

(“Now we are going to do something different. This time I would like you to say as many animals as possible. It does not matter which letter they start with. Just say the names of as many animals as you can. Do you have any questions?”)

• If the examinee did not comprehend the task, the examiner can repeat the instructions and/or paraphrase them, without providing further assistance or strategies as to how to best complete the task. Then say “Listo/a? Comience” (“Ready? Begin”)

• If the examinee does not have questions, say “Listo/a? Comience” (“Ready? Begin”)

Start counting the time. The examiner must write down all the answers on the answer sheet. The examinee should not be interrupted, nor asked to go slower. The examiner can repeat the instructions if the examinee is confused or if he/she forgets the task at hand. After 60 seconds the examiner should say “Pare” (“Stop”).