Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Feb 1.
Published in final edited form as: Clin Neuropsychol. 2020 Feb 20;35(2):396–418. doi: 10.1080/13854046.2020.1713400

Demographically-Adjusted Norms for the Grooved Pegboard and Finger Tapping Tests in Spanish-Speaking adults: Results from the Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) Project

Anne E Heaton 1, Amanda Gooding 1, Mariana Cherner 1, Anya Umlauf 1, Donald R Franklin 1, Monica Rivera Mindt 2,3, Paola Suárez 4, Lidia Artiola i Fortuni 5, Robert K Heaton 1, María J Marquine 1
PMCID: PMC7438231  NIHMSID: NIHMS1555743  PMID: 32077791

Abstract

Objective:

We developed demographically-corrected norms for Spanish-speakers from the U.S.-Mexico border regions of California and Arizona on two tests of motor skills - the Grooved Pegboard Test (Pegboard) and Finger Tapping Test (Tapping) - as part of a larger normative effort.

Method:

Participants were native Spanish-speakers from the Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) Project (Pegboard: N=254; Tapping: N=183; age: 19–60 years; education: 0–20 years; 59% women). We examined the association of demographics (age, education and gender) with raw scores. Raw test scores were then converted to demographically-corrected T-scores via fractional polynomial equations. We also examined rates of impairment (T-score<40) based on the current norms and on previously published norms for English-speaking non-Hispanic Whites and Blacks.

Results:

Having more years of education was associated with better raw test score performance on both tests (p<.001), and increased age was associated with worse performance on Pegboard (p<.001). Men outperformed women on Tapping, and older age was associated with lower raw scores in men only on the Tapping non-dominant hand trial (p=.02). The normed T-scores were confirmed to be normally distributed and free from demographic influences, and resulted in expected rates of impairment. Applying existing norms for English-speaking non-Hispanic Whites and Blacks to the raw scores of Spanish-speakers generally yielded lower than expected impairment rates (2–13%), with one exception: non-dominant Pegboard, for which non-Hispanic White norms overestimated impairment (23%).

Conclusions:

Present findings underscore the importance of appropriate, population-specific normative data, even for tests of motor ability.

Keywords: motor functioning, normative data, Spanish-speakers, cross-cultural assessment

Introduction

Assessment of fine motor functioning is usually part of comprehensive neuropsychological evaluations. The characterization of motor skills provides information that can be relevant to functional outcomes (Anstey et al., 2001; Haaland et al., 1994). In addition, it often plays an important role in making inferences about central nervous system focal lesion lateralization and the functional integrity of each cerebral hemisphere (Goldstein, 1974; Reitan, 1966). Deficits in this domain have been implicated in a host of neurological conditions, including traumatic brain injury, stroke, epilepsy, multiple sclerosis, HIV infection, movement disorders, and others (Arneson et al., 2018; Li et al., 2003; Johansson & Häger, 2019; Heaton et al., 2015; Hernandez, et al., 2002; Montoya et al., 2019; Rivera Mindt et al., 2003). While several motor tasks exist, two of the most commonly administered measures include the Finger Tapping Test (Halstead, 1947; Reitan & Wolfson, 1993; Spreen & Strauss, 1998), and the Grooved Pegboard Test (Klove, 1963).

Demographic differences (i.e., age, education, gender, race/ethnicity) can influence neuropsychological test performance and need to be accounted for in determining normal expectations and criteria for classifying disease-related impairment (Alley et al., 2007; Artiola i Fortuny et al.,1999; Brewster et al., 2014; Díaz-Venegas, et al., 2016; Flores et al., 2017; Heaton et al., 2004a; Heaton, Ryan, & Grant, 2009; Nell, 2000; Puente & Agranovich, 2003; Pontón et al., 1996; Rivera Mindt, Byrd, Saez, & Manly, 2010; Rosselli & Ardila, 2003). Normative data that correct for these effects decrease the likelihood of misclassifying impairment in groups and individuals with diverse demographic and background characteristics. Heaton, Miller, Taylor & Grant (2004b) have published normative data on the Finger Tapping and Grooved Pegboard tests for English-speaking non-Hispanic White (NHW) and non-Hispanic African American/Black (NHB) populations in the U.S. In both of these racial groups, performance on Finger Tapping was driven in large part by age and sex differences, with moderate education effects. Younger age and more years of education were associated with better performance, and men outperformed women on this test (Heaton et al., 2004b). Performance on the Grooved Pegboard was most affected by age, with a much lower influence of sex and education (Heaton, et al., 2004b). Despite being comparable on age, education, and gender, NHWs performed better than NHBs on both Finger Tapping and Grooved Pegboard tests, raising concerns that similar findings might emerge in other U.S. minority groups.

Members of ethnic/racial groups who value speed, competitiveness, and success might perform better on timed assessments, compared to those from cultures that value accuracy over speed (Heaton et al., 2009; Mulenga et al., 2001; Nell, 2000; Puente & Agranovich, 2003). Evidence suggests Spanish-speakers perform differently than English-speaking NHWs on many neuropsychological tests, including those involving both verbal and non-verbal skills (Arango-Lasprilla et al., 2015; Ardila, 2005; Ardila & Moreno, 2001; Arnold et al., 1994; Artiola i Fortuny et al., 1998; Benson et al., 2014; Boone et al., 2007; Buré-Reyes et al., 2013; Casaletto et al., 2016; Cherner et al., 2007; Flores et al., 2017; Gasquoine, 2001; Matute et al., 2000; Mulenga et al., 2001; O’Bryant et al., 2018; Ojeda, 2010; Ostrosky-Solis, Efron, R., & Yund, E.W., 1991; Puente & Salazar, 1998; Polubinski & Melamed, 1986; Pontón et al., 1996; Rivera Mindt et al., 2010; Rosselli & Ardila, 2003; Rosselli, Ardila, Bateman, & Guzman, 2001; Wang et al., 2015). Although Rosselli and colleagues (2001) published norms for Spanish-speaking Colombian children on Finger Tapping and Grooved Pegboard tests, normative data on motor tests for Spanish-speakers living in the U.S. are rather limited. Previous studies (Pontón et al., 1996) have generated normative data for Spanish-speakers living in the U.S. on the Pin Test, which is similar to the Grooved Pegboard Test, but to our knowledge normative data have not been generated for this population on the Finger Tapping or Grooved Pegboard tests. Similarly, the extant literature has not specifically investigated the influence of demographic factors such as age, education and gender on fine motor functioning among Spanish-speakers.

Given the limitations to existing normative datasets for linguistic minorities, this study aimed to develop and provide demographically-corrected norms for the Finger Tapping and Grooved Pegboard tests among U.S.-dwelling native Spanish-speakers who reside in the U.S.-Mexico border regions of California and Arizona. In addition, this study aimed to investigate the impact of applying existing norms for English-speaking NHWs and NHBs on the classification of impairment on fine motor tests among these Spanish-speakers.

Methods

Participants

Participants were part of a larger normative effort called the Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) project. This norming project combined participants from two cohorts recruited at different timepoints to increase the size of the normative sample. Participants from the first cohort (Cohort 1, N=183) were recruited between 1998 and 2000 from or near the regions of Tucson, Arizona (n=102) and San Diego, California (n=81). Participants from the second cohort (Cohort 2, N=71) were recruited between 2006–2009 from or near San Diego only. Two-hundred and fifty-four adults completed the Grooved Pegboard Test (all participants from both cohorts) and a subset of 183 participants completed the Finger Tapping Test (all from Cohort 1) as part of a comprehensive neuropsychological battery. Multiple papers in this issue on the NP-NUMBRS project present demographically-corrected norms for all cognitive domains in this battery, i.e., verbal fluency (Marquine et al., 2020a), speed of information processing (Rivera Mindt et al., 2020a; Suárez et al., 2020a), attention/working memory (Gooding et al., 2020), executive functioning (Marquine et al., 2020b; Morlett Paredes et al., 2020b; Suárez et al., 2020a), learning and memory (Díaz-Santos et al., 2020), visuospatial skills (Scott et al., 2020), and fine motor skills, which is the focus of the current report. For additional information on this NP-NUMBRS project and the state of norms for Spanish-speakers in this U.S.-Mexico border region, see the introduction paper (Cherner et al., 2020a) and review paper (Morlett Paredes et al., 2020a) in this issue. Finally, a paper by Kamalyan et al. (2020) begins to address the validity of the current norms for most tests in the NP-NUMBRS battery (including the Pegboard) for detecting central nervous system disorders, by applying the norms to test results of Spanish-speaking, U.S.-Mexico border region residents with HIV infection.

All participants resided in the U.S. at least part of the time, had Spanish as their native and currently dominant language, and were tested in Spanish by bilingual (Spanish-English) trained psychometrists. Motor test formats were not changed from the English versions, and translations of intructions were conducted in a manner consistent with published guidelines (van de Vijver & Hamleton, 1996; Hambleton, Merenda, & Spielberger, 2005; International Test Commision, 2005, 2017); instructions for both measures were back-translated and examined for fidelity with the English versions (see Appendices A and B for test administration instructions in Spanish). Potential participants for both cohorts were screened based on similar inclusion and exclusion criteria using structured interviews. Participants were excluded for any condition or illness that may influence test performance, such as a central nervous system disorder, medical condition, serious psychiatric condition, or peripheral injury. Less serious conditions or disabilities were reviewed on a case-by-case basis by senior investigators. Although participants with reading glasses were told to bring them on the day of testing, there was no vision test performed prior to testing and visual acuity was not specifically tested. For additional details on study recruitment, participants, procedures, and data collection, see Cherner et al. (2020b).

Motor Assessments

The Finger Tapping Test is a measure of simple motor speed. Participants are instructed to place the index finger of one hand on a counting device similar to a telegraph key and are asked to use it to tap as many times as possible in 10-second trials. The participants perform five consecutive 10-second trials, first with their dominant hand, and then their nondominant hand. A total score generated for each hand is the mean of 5 consecutive trials (i.e., five trials within a 5-point range). See Reitan & Wolfson (1993) for more details on test administration, including the addition of more trials in the case of excessive variability across the first 5 trials.

The Grooved Pegboard Test measures manipulative dexterity and perceptual-motor speed. The pegboard contains 25 grooved holes (5 rows with 5 holes each) that match the grooves of the pegs. The grooved holes on the pegboard are randomly oriented, such that the pegs must be rotated to fit into each groove. The participants are instructed to use only one hand (first their dominant hand, then their non-dominant hand) to insert all of the 25 pegs in rows from top to bottom as fast as they can by matching the groove of the pegs with grooved holes in the pegboard. Scores are generated for the time of each trial, the number of pegs that are correctly inserted (25 if completed), and the number of pegs unintentionally dropped for each trial. See the Trites (1977) and Lafayette Instrument Company (2002) manuals for more detail on test administration.

Demographically-corrected T-scores (mean=50, SD=10) are reported for the following scores:

  • Grooved Pegboard Dominant Hand Total Score: Total time to completion (in seconds) using only the dominant hand.

  • Grooved Pegboard Non-dominant Hand Total Score: Total time to completion (in seconds) using only the non-dominant hand.

  • Finger Tapping Dominant Hand Total Score: Mean number of taps produced in 5, 10-second trials using only the index finger on the dominant hand.

  • Finger Tapping Non-dominant Hand Total Score: Mean number of taps produced in 5, 10-second trials using only the index finger on the non-dominant hand.

Statistical Analyses

The distribution of raw scores was examined via Shapiro-Wilk tests. The univariable associations between demographic characteristics (age, education, and sex) and raw test scores were examined with Pearson product moment correlation coefficients (or Spearman ρ) for age and education, and independent sample t-tests or Wilcoxon rank-sum tests for sex. The interactive effect of demographic variables on raw test scores was investigated via separate linear regression models with terms for the two-way interaction of demographics (i.e., age X education, age X sex, education X sex).

Normed scores were calculated by first converting raw scores to normalized scaled scores (SS; mean=10, SD=3). Scaled scores were then converted to age-, education-, and sex-corrected T-scores via a series of fractional polynomial equations, which allowed for the consideration of linear and nonlinear effects of demographics factors. Please see Cherner et al. (2020b) for further details on norming procedures. We examined the distribution of the resulting T-scores via Shapiro-Wilk tests and their association with demographic factors via Pearson product moment correlation coefficients for age and education, and an independent sample t-test for sex. We also compared the resulting T-scores by cohort and site via independent sample-tests. Lastly, we calculated T-scores based on published norms for English-speaking NHWs and NHBs (Heaton et al., 2004b) on the present sample, and computed rates of impairment (T-scores < 40) utilizing both the published norms and the current norms for Spanish-speakers. McNemar’s tests were then used to compare rates of impairment applying our newly-developed Spanish-speaking norms and published norms for English-speaking NHWs and NHBs (Heaton et al., 2004b).

Results

Demographic characteristics of the norming sample

Demographic characteristics of the norming sample with data on the Grooved Pegboard and the Finger Tapping tests are summarized in Table 1a. The sample was comprised of adults ages 19 to 60, who had an average education of less than high school, and with a little over half being female. There were no significant differences on age (p=.12), education (p=.40) or gender (p=.97) between participants who completed both tests and those who completed only the Grooved Pegboard test. Tables 1b and 1c show demographic characteristics stratified by education group (≤6 years, 7–10 years, 11–12 years, and ≥13 years of education) for the Pegs and Tapping, respectively. Table 2 includes educational, social, and language background characteristics of our cohort stratified by test. Unfortunately, not all participants had data available on these measures, so the table does not include all participants who completed the tests. The majority of participants lived most of their lives and completed most of their formal education in their country of origin (rather than in the U.S.). Almost a third of the sample had to stop attending school to work, and approximately half of participants reported working for money during childhood. All but one of the participants reported that Spanish was the first language they learned. Average ratings of language used in various everyday activities indicated that Spanish was the predominant language used in daily life (with ratings for each activity ranging from 1 “Always in Spanish” to 5 “Always in English”, with 3 being “similarly in English and Spanish”). Nearly two-thirds of the sample was monolingual Spanish-speaking/strongly Spanish dominant, with the remaining third being bilingual. In addition to those self-report measures of language preference, the Controlled Oral Word Association Test (Borkowski, Benton, & Spreen, 1967) was administered as an objective, performance-based measure of word fluency. Both the English version with letters F-A-S (Strauss, Sherman, & Spreen, 2006) and the Spanish version with letters P-M-R (Artiola i Fortuny, Hermosillo Romo, Heaton, & Pardee, 1999; Strauss et al., 2006) were administered to create an estimated English fluency rating from the ratio of scores [(FAS)/(FAS + PMR)] (Suárez et al., 2014). Participants with a ratio of scores ≥ .67 (i.e., 2/3 of all words in English) were classified as English-dominant, and excluded from analyses. Participants with scores ≤ .33 were classified as monolingual Spanish-speakers, and those in between were considered bilingual. See Suárez et al. (2020b) for further detail.

Table 1a.

Demographic characteristics of the normative sample for the Grooved Pegboard test and Finger Tapping test.

Grooved Pegboard (N=254) Finger Tapping (N=183)
Age (years),
M (SD) 37.32 (10.24) 37.15 (9.58)
 Range 19–60 20–55
Education (years),
M (SD) 10.67 (4.34) 9.93 (4.20)
 Range 0–20 0–20
N (%)
  ≤ 6 22.84% (58) 27.32% (50)
  7–10 22.05% (56) 25.14% (46)
  11–12 25.20% (64) 24.59% (45)
  ≥13 29.92% (76) 22.95% (42)
% Female 58.66% 57.92%

Table 1b.

Demographic characteristics of the normative sample for the Grooved Pegboard Test stratified by years of education (N=254)

≤ 6 (n=58) 7–10 (n=56) 11–12 (n=64) ≥13 (n=76)
Age (years), M (SD) 39.71 (9.86) 36.95 (9.54) 35.14 (10.34) 37.61 (10.69)
Education (years), M (SD) 4.72 (1.55) 8.59 (0.91) 11.81 (0.39) 15.79 (1.67)
% Female 62.07% 55.36% 65.53% 52.63%

Table 1c.

Demographic characteristics of the normative sample for the Finger Tapping Test stratified by years of education (N=183)

≤ 6 (n=50) 7–10 (n=46) 11–12 (n=45) ≥13 (n=42)
Age (years), M (SD) 38.38 (9.41) 36.74 (9.35) 35.04 (9.82) 38.40 (9.68)
Education (years), M (SD) 4.68 (1.52) 8.61 (0.86) 11.84 (0.37) 15.57 (1.58)
% Female 62.00% 56.52% 62.22% 50.00%

Table 2.

Educational, Social, and Language Background Characteristics of the Normative Sample for the Grooved Pegboard and Finger Tapping Tests

Characteristics Grooved Pegboard (N=254) Finger Tapping (N=183)
M(SD), % n M(SD), % n
Educational Background
 Years of education in country of origin 8.53 (4.879 227 8.41 (4.42) 170
 Years of education in the U.S. 2.53 (4.73) 227 1.72 (3.45) 170
 Proportion of education by country 227 170
  More years of education in country of origin 84.14% 191 87.65% 149
  More years of education in the U.S. 14.98% 34 11.18% 19
  Equal number of years of education in both countries 0.88% 2 1.18% 2
 Type of school attended+ 243 181
  Large 55.56% 135 51.93% 94
  Regular 39.92% 97 43.09% 78
  Small 4.53% 11 4.97% 9
 Number of students in the class 247 181
  Less than 21 15.39% 38 8.84% 16
  21 to 30 39.27% 97 39.23% 71
  31 to 40 24.29% 60 27.62% 50
  40+ 21.05% 52 24.31% 44
 Had to stop attending school to work 224 180
  Yes 28.57% 64 31.77% 54
Social Background
 Mother’s years of education 5.76 (3.65) 180 5.76 (3.65) 180
 Father’s years of education 6.80 (5.06) 163 6.80 (5.06) 163
 Years lived in country of origin 26.41 (12.50) 245 27.86 (11.87) 181
 Years living in the U.S. 10.69 (10.85) 245 9.16 (9.56) 181
 Childhood SES 251 182
  Very poor 5.98% 15 6.04% 11
  Poor 27.09% 68 26.37% 48
  Middle class 58.17% 146 58.24% 106
  Upper class 8.77% 22 9.34% 17
 Worked as a child 248 181
  Yes 52.82% 131 59.12% 107
   Reason to work 130 112
    Help family financially 38.46% 50 34.82% 39
    Own benefit 61.54% 80 65.18% 73
   Age started working as a child 12.98 (3.18) 127 13.41 (2.98) 106
 Currently gainfully employed 224 179
  Yes 68.75% 154 65.36% 117
Language
 First Language 250 180
  Spanish 98.40% 246 98.89% 178
  English 0.40% 1 0.00% 0
  Both 1.20% 3 1.11% 2
 Current language use rating* 251 181
  Radio or TV 2.37 (1.03) -- 2.30 (1.01) --
  Reading 2.24 (1.18) -- 2.08 (1.11) --
  Math 1.54 (1.05) -- 1.41 (0.92) --
  Praying 1.26 (0.72) -- 1.17 (0.53) --
  With family 1.56 (0.90) -- 1.43 (0.74) --
 Performance-based language fluency^ 203 170
  Spanish dominant 62.07% 126 60.00% 102
  English dominant 0.00% 0 0.00% 0
  Bilingual 37.93% 77 40.00% 68

Note. M: mean; SD: standard deviation; SES: socioeconomic status.

+

Types of school attended: Large=many classrooms per grade and room to play, Regular=at least one classroom per grade and room to play, Small=less than one classroom per grade.

*

Current language use ratings: Via self-report questionnaires, participants rated their current language use for common daily activities, ranging from 1 (Always in Spanish) to 5 (Always in English).

^

Performance-based language fluency: The Controlled Oral Word Association test was administered in English (FAS) and Spanish (PMR) to create an estimated English fluency rating from the ratio of scores.

There were no significant differences between participants tested in Arizona and California on any demographic factors in the overall cohort (ps>.26) or among participants from Cohort 1 (ps>.16). However, participants in Cohort 1 had significantly fewer years of education than Cohort 2 (p<.001), and analyses showed similar findings within participants from California only (n=152; p<.001). This difference in years of education may be due to the specific recruitment goals for Cohort 1 to represent the full range of educational attainment (Cherner et al., 2020b).

Raw scores to scale scores conversions

Table 3 shows descriptive characteristics of raw scores for the Grooved Pegboard and Finger Tapping total scores for dominant (Dom) and non-dominant (NDom) hands. Table 4 shows the univariable association of raw test scores with demographic variables. There were significant effects of education on both the Grooved Pegboard and Finger Tapping total raw scores (Dom and NDom), of age on Grooved Pegboard total raw scores, and large effects of gender (favoring men) on Finger Tapping total raw scores. There was a significant age and gender interaction on Finger Tapping NDom raw scores (p=.03), indicating that increasing age was associated with lower raw scores in males (n=77, p=.02) but not in females (n=106, p=.67). There were no other significant two-way interactions among demographic variables on Finger Tapping or Grooved Pegboard raw scores.

Table 3.

Mean, standard deviation, and range of the Grooved Pegboard and Finger Tapping total raw scores for dominant and non-dominant hands.

N Mean (SD) Range
Grooved Pegboard Total Score (Dominant) 254 63.41 (11.47) 42 – 136
Grooved Pegboard Total Score (Non-dominant) 254 72.74 (12.72) 49 – 124
Finger Tapping Total Score (Dominant) 183 49.47 (6.79) 25.4 – 62.6
Finger Tapping Total Score (Non-dominant) 183 45.67 (5.91) 25.2 – 61.1

Table 4.

Association between raw test scores and demographic characteristics.

N Agea Educationa Genderb
Male Female p Cohen’s d
Grooved Pegboard Total Score (Dominant) 254 0.23* −0.22* 64.73 (12.76) 62.48 (10.42) .20 0.15
Grooved Pegboard Total Score (Non-dominant) 254 0.19* −0.23* 73.71 (13.87) 72.05 (11.84) .64 0.11
Finger Tapping Total Score (Dominant) 183 −0.09 0.35* 53.16 (5.75) 46.79 (6.23) <.001 1.06
Finger Tapping Total Score (Non-dominant) 183 −0.14 0.27* 48.35 (5.67) 43.73 (5.32) <.001 0.84

Note. Based on results from Pearson product moment correlation coefficients or Spearman ρa, and independent sample t-tests or Wilcoxon rank-sum testsb.

*

p<.001

Table 5 shows the raw-to-normalized scaled score conversions for the Grooved Pegboard and Finger Tapping Total Scores (Dom and NDom hands).

Table 5.

Raw-to-scale score conversions.

Scaled Grooved Pegboard Raw Score (Dominant) Grooved Pegboard Raw Score (Non-dominant) Finger Tapping Raw Score (Dominant) Finger Tapping Raw Score (Non-dominant)
19 65.0 64.9 – 65.0
18 < 47 < 51 62.4 – 64.9 60.2 – 64.8
17 47 – 48 51 – 52 61.8 – 62.3 57.2 – 60.1
16 49 – 50 53 – 56 61.4 – 61.7 56.6 – 57.1
15 51 57 – 58 58.8 – 61.3 54.3 – 56.5
14 52 – 53 59 57.8 – 58.7 52.8 – 54.2
13 54 – 55 60 – 62 55.5 – 57.7 50.7 – 52.7
12 56 – 57 63 – 65 52.9 – 55.4 48.8 – 50.6
11 58 – 59 66 – 68 50.6 – 52.8 46.6 – 48.7
10 60 – 62 69 – 71 48.6 – 50.5 44.5 – 46.5
9 63 – 65 72 – 77 46.5 – 48.5 42.5 – 44.4
8 66 – 68 78 – 80 44.5 – 46.4 41.1 – 42.4
7 69 – 72 81 – 87 41.6 – 44.4 39.0 – 41.0
6 73 – 78 88 – 94 40.1 – 41.5 38.1 – 38.9
5 79 – 91 95 – 104 36.6 – 40.0 36.7 – 38.0
4 92 – 107 105 – 108 32.2 – 36.5 32.6 – 36.6
3 108 – 119 109 – 118 25.8 – 32.1 25.8 – 32.5
2 120 – 225 119 – 220 1.1 – 25.7 1.0 – 25.7
1 226 – 301 221 – 301 0.0 – 1.0 0.0 – 0.9

T-Scores Equations

Table 6 shows the equations used to compute individual T-scores. As expected, the resulting T-scores had a mean of 50 and a SD of 10 (Table 7). Similarly, for the subset of participants who had data on both tests, the Grooved Pegboard Dom hand T-scores had a mean of 49.5 and a SD of 9.7, and Grooved Pegboard NDom hand T-scores had a mean of 49.8 and a SD of 10.3. Pearson product moment correlations showed no significant effect of age or education on any T-scores (p’s>96), and there were no significant gender differences by t-tests (p’s> .97), including when considering only participants who had data on both tests.

Table 6.

T-Score equations.

Measure Equation
Grooved Pegboard Total Score (Dominant)a 10×(SS Grooved Pegboard Dominant(3.8852822.03585logage1009.15441(logage100)2+1.66513(edu+1)100.75294gender)2.7217)+50
Grooved Pegboard Total Score (Non-dominant)a 10×(SS Grooved Pegboard Nondominant(10.48135 5.78302age100+1.61742(edu+1)100.48245gender)2.86059)+50
Finger Tapping Total Score (Dominant)b 10×(SS Finger Tapping Dominant(6.07174+1.28165age100+2.21826((edu+1)10)32.8893log(edu+1)10((edu+1)10)3+2.85531gender)2.46635)+50
Finger Tapping Total Score (Non-dominant)b 10×(SS Finger Tapping Nondominant(8.46996 3.043age100+1.58865(edu+1)10+2.20988gender)2.6906)+50

Note.

a

These formulas should be applied to education level ranges from 0–20 and age 19–60.

b

These formulas should be applied to education level ranges from 0–20 and age 20–55. Using values outside these ranges might result in extrapolation errors. Gender: Male=1; Female=0

Edu=years of education

Age= years of age

Table 7.

Comparisons of T-scores when raw data is normed utilizing different sets of norms.

Test Measure N Current Norms M (SD) Range Non-Hispanic White Norms for English-speakers Non-Hispanic Black Norms for English-speakers
M (SD)
Range
Cohen’s d M (SD)
Range
Cohen’s d
Grooved Pegboard
(Dominant)
254 49.98 (10.00)
25–82
51.28 (10.21)
19–83
−0.13* 58.52 (10.21)
27–91
−0.85*
Grooved Pegboard
(Non-dominant)
254 49.98 (10.00)
21–77
47.31 (9.76)
21–73
0.27* 56.64 (10.76)
30–88
−0.64*
Finger Tapping
(Dominant)
183 50.02 (10.02)
22–73
52.45 (8.87)
25–76
−0.26* 50.59 (7.81)
27–73
−0.06
Finger Tapping
(Non-dominant)
183 49.99 (10.00)
20–73
53.95 (8.96)
29–74
−0.42* 51.45 (8.09)
28–73
−0.16*

Note.

*

p<0.0001 based on dependent sample t-tests between T-scores based on current norms and non-Hispanic norms for English speakers

Group comparisons

Independent sample t-tests on Grooved Pegboard T-scores between Cohort 1 (Dom hand: M=49.49, SD=9.66; NDom hand: M=49.76, SD=10.33) and Cohort 2 (Dom hand: M=51.24, SD=10.79; NDom hand: M=50.54, SD=9.15) showed no significant differences (p’s>.23). Similar analyses comparing participants who completed testing in Arizona (n=102) and those who did so in California (n=152) showed no significant differences on T-scores on the Finger Tapping test for either hand (p’s>.38) or on Grooved Pegboard Dom hand T-scores (p=.21). There was a statistically significant difference on Grooved Pegboard NDom hand T-scores, such that T-scores were somewhat lower in Arizona (n=102; M=48.39, SD=10.36) than California (n=152, M=51.04, SD=9.65), p=.042, but with a small effect size (Cohen’s d=.27).

Comparisons across Normative Sets

Table 7 shows T-scores based on the newly-developed norms for Spanish-speakers and T-scores when norms for native English-speaking NHWs and NHBs were applied to the current raw data. A series of dependent sample-tests showed that, compared to T-scores based on current norms, T-scores based on non-Hispanic norms were significantly higher (with generally large effect sizes) except for the following two instances: for Grooved Pegboard NDom hand, T-scores were lower when utilizing NHW norms, and for Finger tapping Dom hand there were no significant differences when NHB norms were applied.

Figure 1 shows the percent of impairment classifications (using 1 SD cut-off) on the tests when the respective norms were applied. Results from McNemar analyses showed that for most test scores, norms based on English-speaking NHWs and NHBs significantly underestimated rates of impairment in the current sample. The only exception was Grooved Pegboard Test NDom hand scores, for which NHW norms overestimated impairment.

Figure 1.

Figure 1.

Rates of impairment on the current sample of Spanish-speakers based on published norms for English-speaking non-Hispanic (NH) Whites and NH Blacks (Heaton et al. 2004), and current population specific Spanish-speaking norms. Impairment was defined as T-score <40. Asterisks denote significant differences based on McNemar’s test in comparison to rates of impairment based on Spanish-speaking norms; *p<.005, **p<.0001.

To further explore possible reasons for the apparently anomalous finding that Spanish-speakers scored worse only on NDom Pegboard when NHW norms for English-speakers were applied, we examined the possible effects of dropping pegs on this test. Fifty percent of the Spanish-speaking participants (119 of the 238 with available data on drops) had at least one drop, and 41 had more than one drop (range=2–11). Of those with drops, 23 individuals dropped pegs with both hands, 38 dropped only with Dom hand, and 58 dropped with only with the NDom hand; thus, more people had drops with the NDom hand (68% vs. 51% for Dom hand). Within the total sample, males were more likely than females to drop pegs on any trial (58% of males, 44% of females; p=.036), especially for NDom hand trials (43% vs. 28%; p=.013).

Individuals with drops on NDom hand trials had lower T-scores (M=47.85, SD=9.93) than those without drops on that trial (M=51.25, SD=9.93; p=.012), reflecting slower performance when drops occurred. To further investigate this finding, a matched pairs test was conducted at the individual level, comparing trials for individuals who dropped on one trial but not the other. The T-scores for trials in which pegs were dropped (M=48.64, SD=10.09) were significantly lower than T-scores for trials without drops (M=51.69, SD=9.45; p=.003), confirming that drops result in slower times.

Discussion

The present study aimed to develop normative data for commonly used tests of motor skills in a group of native Spanish-speaking adults in the U.S.-Mexico border region. The Finger Tapping and Grooved Pegboard tests have been previously normed in Spanish-speaking Colombian children (Rosselli et al., 2001), but the current study is the first to do so with adult Spanish-speakers in the U.S.-Mexico border region.

Findings from the present study of Spanish-speakers show small to medium effects of education on raw scores for both trials of the Finger Tapping and Grooved Pegboard tests. These education effects were lower than those reported in prior studies of NHW and NHB English-speakers (Heaton, et al., 2004b). This discrepancy might be at least partly due to differences in education range and variability, and differential factors underlying years of education completed across ethnic/racial groups. While we do not have consistent data on access to education and/or reasons for terminating schooling, it is possible that lack of access and/or need to work at an early age notably contributed to discontinuing formal education at an early age for some in this sample of Spanish-speakers in the United States (Childfund, 2013; Cigna, 2016; Coley & Baker, 2013; Wolf, Magnuson, & Kimbro, 2018). Unlike much of the prior literature on the Finger Tapping test in which performance decreased with age (Bornstein, 1985; Da Silva et al., 2012; Villardita, Cultrera, Cupone, & Mejía, 1985), we generally found no significant effects of age, except for the NDom hand trial of Finger Tapping, in which performance was worse with increasing age in male individuals only. However, the limited age range of our Spanish-speaking sample may account for the disparity. In our Spanish-speaking sample, males outperformed females on the Finger Tapping Test, which has been consistently demonstrated in the literature in both Spanish- and English-speakers on this test (Bornstein, 1985; Grice et al., 2003; Heaton et al., 2004b; Polubinski & Melamed, 1986; Pontón et al., 1996; Rosselli et al., 2001; Wang et al., 2015). In our sample, there were no gender differences on the Grooved Pegboard test raw scores. Although sex differences on Grooved Pegboard have not been found consistently in the literature, most studies on the Grooved Pegboard (and its variations) have reported that females outperformed males (Da Silva et al., 2012; Grice et al., 2003; Polubinski & Melamed, 1986; Rosselli et al., 2001; Wang et al., 2015).

When norms for English-speaking NHWs and NHBs were applied to our data of Spanish-speaking Hispanics, resulting impairment rates were largely lower for both tests, with the exception of the NDom trial for the Grooved Pegboard test. These results suggest that Spanish-speaking individuals from this population would have an increased risk of being incorrectly deemed “unimpaired” on these motor tests if existing norms for English-speakers are applied. Such applications of norms based upon other populations could lead to undiagnosed impairment in motor function. This could have significant clinical importance, since manual speed and dexterity are important for the performance of daily tasks and those involved in many occupations (Wang, et al., 2015); such skills may be especially needed in the kinds of manual jobs that tend to be most readily available in the U.S. for individuals with lower education levels and a lack of English proficiency (Childfund, 2013; Coley & Baker, 2013; Pandya, 2012). Loss of dexterity has been correlated with worse job proficiency, disease progression, and cognitive impairment (Bezdicek et al., 2014; Lundergan, Soderststrom, & Chambers, 2007; Price, 2014; Yancosek & Howell, 2009). In addition, the lack of appropriate motor evaluation and diagnosis may further increase the disparity in health outcomes that already exists between English- and Spanish-speakers (AHRQ, 2015; Cigna, 2016; Marquine et al., 2016; Martinez, 2008). For information regarding the clinical application of these norms and cognitive impairment in a population of individuals living with HIV in the U.S., see the paper by Kamalyan et al. (2020) in this issue.

Most prior research has demonstrated an over-classification of impairment when English-speaking norms were applied to Spanish-speakers (Casaletto et al., 2016; Cherner et al., 2007). That is, Spanish-speakers, similar to most other cultural minorities in the U.S., have typically been found to perform worse that English-speaking NHWs, although there are exceptions. For example, Hedden et al. (2002) found that Chinese participants located in China significantly outperformed North American participants located in the U.S. on verbal fluency and the Digit Span test, as well as multiple findings that East Asian American children exhibit better-developed motor skills (writing names and numerals) and more advanced mathematic abilities than European American children (Huntsinger, Jose, Liaw, & Ching, 1997; Zupei, Hose, Huntsinger, & Pigott, 2007). However, unlike the aforementioned assessments of motor skills that may benefit those with more formal education (e.g., paper-and-pencil tests), Pegs and Tapping don’t require skills learned in school, suggesting that all participants are equally unfamiliar with these tasks regardless of educational attainment. It has been suggested that speed is typically more valued in mainstream societies in the U.S., and that individuals from less “competitive cultures,” such as the Hispanic/Latino cultures, may perform at a slower pace on neuropsychological tasks that require speed (Ardila, 2005; Ardila & Moreno, 2001; Ardila, Rodríquez-Mendénez, & Rosselli, 2002; Arnold et al., 1994; Benson et al., 2014; Boone et al., 2007; Ojeda, 2010; Puente & Agranovich, 2003; Puente & Salazar, 1998; Rosselli & Ardila, 2003). In contrast, our data largely indicate the opposite for tests of fine motor skills; i.e., on those of the four speed-based scores considered here, the Spanish-speakers outperformed expectation based upon norms for English-speakers. The instruction of going “as fast as you can” for both the Grooved Pegboard test and Finger Tapping test might be sufficient to minimize the potential impact of cultural differences in any general value placed on speed. It might also be the case that different types of occupations (with more or less emphasis on motor speed) across racial/ethnic groups could be playing a role in current findings. While occupation data are not consistently available for the present sample, this may well be a relevant variable to consider in future studies investigating factors explaining differences in cross-cultural neuropsychological test performance.

It is unclear why the current sample of Spanish-speakers performed somewhat below expectations based upon published, NHW norms for English-speakers, only on the NDom hand trial of the Pegboard. The Pegboard test differs from Finger Tapping in that it requires fine manipulation of small metal pegs, and therefore has the possibility of “errors” (dropped pegs). Our follow-up analyses revealed that pegs were more likely to be dropped by the NDom hand (especially by males), and that drops resulted in slower times on this test. Although individuals were excluded from the study if they had any self-reported significant hand or wrist injury, it is possibile that more minor effects of prior injuries may have affected Pegboard performance. It is also quite possible that people in the current Spanish-speaking population (especially the males) had worked in manual-labor occupations more often than their better-educated, English-speaking counterparts who contributed to the published norms for NHWs. It has been suggested that non-dominant hands and fingers are more prone to injury when working with tools with one’s dominant hand while using the non-dominant hand to maintain the object being manipulated. This could increase the risk of hand or finger injuries on the non-dominant hand, such as being smashed while hammering a nail, or a finger being cut while preparing food. Therefore, such workers could potentially incur decreased dexterity due to minor finger injuries from years of working with their hands, also relating to more drops with non-dominant hands. Unfortunately, we do not have specific information about the manual requirements of jobs our participants have held, so this remains a speculative (albeit plausible) possibility.

Collecting data on minor hand/finger/joint abnormalities or disabilities could help confirm our speculations in future normative studies of motor function. Additionally, including an assessment of grip strength could contribute to other mechanisms. For example, it has been suggested that the grip strength of the dominant hand of healthy adults averages 10% greater for the dominant hand, potentionally due to higher resistence to fatigue than the non-dominant hand (Farina et al., 2003; Harris & Eng, 2006). Multiple studies have found differences in hand dominance and function to be relevant in the assessment of orthopedic and neurological conditions, including lateralized brain damage (Mack, 1969). Harris & Eng (2006) described clinical findings in Osteoarthritis (Caspi et al., 2001) and Parkinson’s Disease (Nutt et al., 2000), in which functional changes related to hand dominance. Bravi et al. (2017) described studies in the literature suggesting that precision and variability of movements in tasks of timing coordination were related to somatosensory feedback (Spencer, Zelaznik, Diedrichsen, & Ivry, 2003). Hammond (2002) explored the functional interplay of motor lateralization and structural brain asymmetries, often correlated with handedness, highlighting further clinical implications related to motor function. Bagesteiro & Sainburg (2002) suggested that the dominant arm is better at predictive and dynamic features of movement, while the non-dominant arm is specialized for stabilizing tasks.

Another unexpected finding about our sample’s NDom hand performances on Pegboard is that there was a small Arizona versus California site effect only on this trial of this test: the Arizona cohort had somewhat lower T-scores on the current norms than the California cohort (p=.042, Cohen’s d=.27). Although the prevalence of peg-dropping was not different at the two sites, the site difference in speed-based T-scores disappeared when NDom trials with peg drops were excluded. Again, while peg dropping appears to importantly affect results on this test, we are limited by inadequate information about participants’ work histories and prior hand injuries. Future examinations using this test should more carefully inquire about such histories, especially for interpreting the meaning of peg drops and associated time penalties.

Normative data for Spanish-speakers along the U.S.-Mexico border region were successfully generated, but it is worth noting that a majority of the study participants lived in California and not other border states. However, California continues to have the largest population of Hispanics in the U.S., and 63% of all Hispanics in the U.S. are of Mexican origin/descent (Flores, 2017; U.S. Census Bureau, 2017). Another limitation is that we do not have consistent information about potentially important aspects of participants’ backgrounds, such as levels of acculturation, country of origin, and occupation, among others. Only a subset of participants have data related to acculturation and bilingualism available, and in this subsample, we investigated the association of these factors with test performance in Suárez et al. (2020b). Additionally, our sample does not include Spanish-speaking adults over the age of 60 years old (55 years old for the cohort with data on the Finger Tapping test), and thus the current norms should be used with great caution (if at all) with Spanish-speakers over age 60.

Normative standards from other groups are not a good fit for interpreting motor test performance in the Spanish-speaking population, which in the current instance would have increased risks for misdiagnosing fine motor impairment. These findings underscore the importance of appropriate, population-specific normative data, even for tests of motor ability that have a minimal verbal/linguistic demand. Despite the study limitations, these norms have clinical relevance and the potential to improve patient care through more accurate diagnostic evaluation of U.S.-Mexico borderland Spanish-speakers. However, due to our modest sample size, lower age range, and limited data on cultural background information, these norms should be considered specific to the young to middle-aged, Spanish-speaking, adult population in the U.S.-Mexico border region of California and Arizona. Additional studies are warranted to assess generalizability of these findings in similar populations and extend to more diverse Spanish-speaking groups in the U.S. Also, the data on which the current norms are based were collected 10 to 20 years ago. Since we do not have more current data for Spanish-speakers on these tests, we cannot examine whether there may be a Flynn or cohort effect with individuals from present day. However, when significant differences between the current cohorts (tested 10 years apart) were investigated in another paper in this issue (Cherner et al., 2020b), few were found, and there were no statistical differences between cohorts on Pegs T-scores (Tapping data from Cohort 1 only). Nevertheless, due to the length of time since these studies ended, revalidation of these norms with more contemporary data would be needed to clarify or rule out any generational effect. Further research also is needed to better understand the many factors that may influence test performance between different populations, particularly in the diverse and heterogeneous Spanish-speaking population, where differences exist regarding country of origin, acculturation status, educational background, and linguistic preference/fluency. While we have some “culturally-relavant” data available for our normative sample, their associations with neuropsychological performance will be investigated in an upcoming paper. Due to the complexity of the interactions of educational and social characteristics with both corrected and uncorrected scores, the significant associations of these background characteristics with domains in this NP-NUMBRS issue will be described in a separate, upcoming paper. Such insights with these and other tests are likely to be important for informing clinical evaluations and for improving overall health outcomes for Spanish-speakers in the U.S.

For more information on the NP-NUMBRS norming project and next steps, see the paper by Rivera Mindt et al. (2020b). Also, for the interested reader a user-friendly digit calculator will be available for clinicians to generate T-scores for the Grooved Pegboard and Finger Tapping tests to implement in the assessment of motor function based on the current new norms for this particular population.

Acknowledgements

All authors attest that there are no conflicts of interest with this work. This manuscript was supported by NIH grants P30MH62512, R01MH57266, K23MH105297, P30AG059299, the UCSD Hispanic Center of Excellence and T32 MH019934.

Appendix A. Grooved Pegboard Instructions

“Aquí tenemos un tablero con unas ranuras y estas son unas clavijas.”

The examiner points out each and then picks up one of the pegs and continues.

“Todas las clavijas son idénticas. Todas tienen un lado plano y un lado redondo, igual que las ranuras del tablero. Lo que usted debe hacer es colocar correctamente el lado plano de la clavija con el lado plano de la ranura y colocarlas en el tablero así.” (insert first row of pegs.)

Start with their dominant hand.

DOMINANT HAND:

“Cuando yo diga “Adelante” comience aquí (point to the top left groove) y coloque las clavijas en el tablero lo más rápido que pueda, usando solamente su mano derecha. Llene la hilera de arriba completamente de este lado a éste. No deje de llenar ninguna ranura, y llene cada hilera de la misma manera en que completó la primera. ¿Alguna pregunta? ¿Listo/a? Lo más rápido que pueda. Adelante.”

NON-DOMINANT HAND:

“Cuando diga “Adelante” comience aquí (point to the top right groove) y coloque las clavijas en el tablero lo más rápido que pueda, usando solamente su mano izquierda. Llene la hilera de arriba completamente de este lado a éste. No deje de llenar ninguna ranura, y llene cada hilera de la misma manera en que completó la primera. ¿Alguna pregunta? ¿Listo(a)? Lo más rápido que pueda. Adelante.”

Discontinue test if subject cannot complete task in 301 seconds (5 minutes, 1 second).

Appendix B. Finger Tapping Instructions

“Ahora vamos hacer un examen para ver que rápido puede dar palmaditas.”

The examiner demonstrates the use of the key.

“Ahora vamos hacer un examen para ver que rápido puede dar palmaditas. Usaremos esta llavesita (demonstrate the use of the key to the subject) y quiero que dé palmaditas lo más rápido que pueda usando el dedo índice (point to the forefinger of the subject) de su mano (derecha/izquierda). Cuando lo haga, asegúrese usar no más que el dedo. No mueva su mano entera o su brazo. Cuando dé palmaditas, recuerdese que la llave tiene que subir para arriba y dar golpear cada vez o el número no cambiará.”

The examiner demonstrates how the key operates and how it should be allowed to “click” etc.

“Ahora, mueva la tabla a una posición que sea cómodo para su mano y practique”

After a brief practice period:

“Recuerdese de hacerlo lo más rápido que pueda. ¿Entiende? ¿Listo/a? ¡Empiece!”

References

  1. Agency for Healthcare Research and Quality: Improving Patient Safety Systems for Patients With Limited English Proficiency. Content last reviewed September 2012. Agency for Healthcare Research and Quality, Rockville, MD: https://www.ahrq.gov/health-literacy/systems/hospital/lepguide/index.html. [Google Scholar]
  2. Alley D, Suthers K, Crimmins E (2007). Education and Cognitive Decline in Older Americans: Results from the AHEAD Sample. Research on Aging, 29(1): 73–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Anstey KJ, Luszcz MA, Giles LC, & Andrews GR (2001). Demographic, health, cognitive, and sensory variables as predictors of mortality in very old adults. Psychology and Aging, 16, 3–11. [DOI] [PubMed] [Google Scholar]
  4. Arango-Lasprilla JC, Rivera D, Garza MT, Saracho CP, Rodríguez W, Rodríguez-Agudelo Y, Aguayo A, Schebela S, Luna M, Longoni M, Martínez C, Doyle S, Ocampo-Barba N, Galarza-Del-Angel J, Aliaga A, Bringas M, Esenarro L, García-Egan P, Perrin PB (2015). Hopkins Verbal Learning Test- Revised: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37(4): 699–718. [DOI] [PubMed] [Google Scholar]
  5. Ardila A (2005). Cultural Values Underlying Psychometric Cognitive Testing. Neuropsychology Review, 15(4): 185–95. [DOI] [PubMed] [Google Scholar]
  6. Ardila A.l., & Moreno S (2001). Neuropsychological test performance in Aruaco Indians: An exploratory study. Journal of International Neuropsychological Society, 7: 510–515. [DOI] [PubMed] [Google Scholar]
  7. Ardila A, Rodríquez-Mendénez G, Rosselli M (2002). Current issues in neuropsychological assessment with Hispanics/Latinos In: Ferraro FR, editor. Minority and cross-cultural aspects of neuropsychological assessment. Lisse, Netherlands: Swets & Zeitlinger; pp. 161–179. [Google Scholar]
  8. Arneson D, Zhang G, Ying Z, Zhuang Y, Byun HR, Ahn IS, Gomez-Pinilla F, & Yang X (2018). Single cell molecular alterations reveal target cells and pathways of concussive brain injury. Nature Communications, 9(1), 3894. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Arnold BR, Montgomery GT, Castaneda I, Longoria R (1994). Acculturation and Performance of Hispanics on selected Halstead-Reitan neuropsychological tests. Assessment, 1(3), 239–248. [Google Scholar]
  10. Artiola i Fortuny L, Hermosillo RD, Heaton RK, Pardee RE III. Manual de normas y procedimientos para la bateria neuropsicologica en Espanol. Tucson, AZ: 1999. [Google Scholar]
  11. Artiola i Fortuny L, Heaton RK, Hermosillo D (1998). Neuropsychological comparisons of Spanish-speaking participants from the U.S.-Mexico border region versus Spain. Journal of International Neuropsychological Society, 4(4): 363–379. [PubMed] [Google Scholar]
  12. Artiola i Fortuny L, Hermosillo Romo D, Heaton RK, & Pardee RE (1999). Manual de Normas y Procedimientos para la Bateria Neuropsicologica en Espanol. Tucson, AZ: m Press. [Google Scholar]
  13. Benson G, de Felipe J, Xiaodong, Sano M (2014). Performance of Spanish-speaking community-dwelling elders in the United States on the Uniform Data Set. Alzheimers Dementia, 10(0): S338–S343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Bezdicek O, Nikolai T, Hoskovcová M, Stochl J, Brozová H, Dusek P, Zárubová K, et al. (2014). Grooved pegboard predicts more of cognitive motor involvement in Parkinson’s disease. Assessment, 21(6), 723–30. [DOI] [PubMed] [Google Scholar]
  15. Boone KB, Victor TL, Wen J, Razani J, & Pontón M (2007). The association between neuropsychological scores and ethnicity, language, and acculturation variables in a large patient population. Archives of Clinical Neuropsychology, 22: 355–365. [DOI] [PubMed] [Google Scholar]
  16. Borkowski JG, Benton AL, & Spreen O (1967). Word fluency and brain damage. Neuropsychologia, 5,135–140. [Google Scholar]; Gladsjo JA, Schuman CC, Evans JD, Peavy GM, Miller SW, & Heaon RK (1999). Norms for letter and category fluency: Demographic corrections for age, education, and ethnicity. Assessment, 6, 147–178. [DOI] [PubMed] [Google Scholar]
  17. Bornstein MH (1985). Habituation of attention as a measure of visual information processing in human infants: Summary, systematization, and synthesis In Gottlieb G & Krasnegor NA (Eds.), Measurement of audition and vision in the first year of postnatal life: A methodological overview (pp. 253–300). Westport, CT, US: Ablex Publishing. [Google Scholar]
  18. Brewster PW, Melrose RJ, Marquine MJ, Jonson JK, Napoles A, MacKay-Brandt A, Farias S, Reed B, & Mungas D (2014). Life experience and demographic influences on cognitive function in older adults. Neuropsychology, 28(6): 846–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Buré-Reyes A, Hidalgo-Ruzzante N, Vilar-López R, Gontier J, Sánchez L, Pérez-García M, & Puente AE (2013). Neuropsychological test performance of Spanish speakers: Is performance different across different Spanish-speaking subgroups? Journal of Clinical and Experimental Neuropsychology, 35(4): 404–412. [DOI] [PubMed] [Google Scholar]
  20. Casaletto KB, Umalauf A, Beaumont J, Gershon R, Slotkin J, Akshoomoff N, Heaton RK (2016). Demographically Corrected Normative Standards for the English Version of the NIH Toolbox Cognition Battery. Journal of International Neuropsychological Society, 21(5): 378–391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Caspi D, Flusser G, Farber E, et al. (2001). Clinical, radiologic, demographic, and occupational aspects of hand osteoarthritis in the elderly. Seminars in Arthritis Rheumatism, 30, 321–31. [DOI] [PubMed] [Google Scholar]
  22. Cherner M, Suarez P, Lazzaretto D, Fortuny LA, Mindt MR, Dawes S, Marcotte T, Grant I, Heaton R, & the HNRC Group (2007). Demographically corrected norms for the Brief Visuospatial Memory Test-revised and Hopkins Verbal Learning Test-revised in monolingual Spanish speakers from the U.S.-Mexico border region. Archives of Clinical Neuropsychology, 22(3): 343–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Cherner M, Rivera Mindt M, Marquine MJ, Artiola i Fortuni L, Suárez P, & Heaton RK (2020a). Introduction to the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project. The Clinical Neuropsychologist, 34, XX–XX. [Google Scholar]
  24. Cherner M, Marquine MJ, Umlauf A, Morlett Paredes A, Rivera Mindt M, Suárez P, Yassai-Gonzalez D, Kamalyan L, Scott T, Heaton AE, Díaz-Santos M, Gooding A, Artiola i Fortuni L, & Heaton RK (2020b). Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) Project: Methodology and Sample Characteristics. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Childfund (2013). The Effects of Poverty on Education in the United States. Available at: https://www.childfund.org/Content/NewsDetail/2147489206/.
  26. Cigna (2016). Health Disparities in Hispanic Population. Retrieved from: https://www.cigna.com/health-care-providers/resources/hispanic-health-disparities.
  27. Coley RJ, & Baker B (2013). Poverty and Education: Finding the Way Forward. Retrieved from the Education Testing Service, Center for Research on Human Capital and Education website: https://www.ets.org/s/research/pdf/poverty_and_education_report.pdf.
  28. Da Silva FN, Irani F, Richard J, Brensinger CM, Bilker WB, Gur RE, Gur RC (2012). More than just tapping: index finger-tapping measures procedural learning in schizophrenia. Schizophrenia Research, 137(1–3): 234–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Díaz-Santos M, Suárez PA, Umlauf A, Marquine MJ, Rivera Mindt M, Artiola i Fortuny L, Heaton RK, Cherner M, & the HNRC group (2020). Updated Demographically Adjusted Norms for the Brief Visuospatial Memory Test Revised and Hopkins Verbal Learning Test-Revised in Native Spanish Speakers from the U.S.-Mexico Border Region: NP-NUMBERS Project. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Díaz-Venegas C, Downer B, Langa KM, & Wong R (2016). Racial and ethnic differences in cognitive function among older adults in the USA. International Journal of Geriatric Psychiatry, 31(9): 1004–1012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Farina D, Merletti R, & Enoka RM(2003). The extraction of neural strategies from the surface EMG. Journal of Applied Physiology, 96, 1486–1495. [DOI] [PubMed] [Google Scholar]
  32. Flores I, Casaletto KB, Marquine MJ, Umlauf A, Moore DJ, Mungas D, Gershon RC, Beaumont JL, Heaton RK (2017). Performance of Hispanics and Non-Hispanic Whites on the NIH Toolbox Cognition Battery: the roles of ethnicity and language backgrounds. Clinical Neuropsychology, 31: 783–797. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Flores A (2017). How the US Hispanic Population is changing. Retrieved from the Pew Research Center website: http://www.pewresearch.org/fact-tank/2017/09/18/how-the-u-s-hispanic-population-is-changing/.
  34. Gasquoine PG (2001). Research in Clinical Neuropsychology With Hispanic American Perticipants: A Review. The Clinical Neuropsychologist, 15(1): 2–12. [DOI] [PubMed] [Google Scholar]
  35. Grice KO, Vogel KA, Le V, Mitchell A, Muniz S, & Vollmer MA (2003). Adult Norms for a Commercially Available Nine Hole Peg Test for Finger Dexterity. American Journal of Occupational Therapy, 57(5), 570–3. [DOI] [PubMed] [Google Scholar]
  36. Goldstein G (1974). The use of clinical neuropsychological methods in the lateralization of brain lesions In Dimon SJ & Beaumont JG (Eds.), Hemisphere function in the human brain. New York: Halsted Press. [Google Scholar]
  37. Gooding A, Seider T, Marquine MJ, Suárez P, Umlauf A, Rivera Mindt M, Heaton RK, Artiola i Fortuni L, & Cherner M (2020). Demographically-Adjusted Norms for the Paced Auditory Serial Addition Task and Letter Number Sequencing Test in Spanish Speakers from the US-Mexico Border Region. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Haaland KY, Temkin N, Randahl G, & Dikmen S (1994). Recovery of simple motor skills after head injury. Journal of Clinical and Experimental Neuropsychology, 16, 448–456. [DOI] [PubMed] [Google Scholar]
  39. Halstead WC (1947). Brain and intelligence. Chicago: University of Chicago Press. [Google Scholar]
  40. Hambleton RK, Merenda P, & Speilberger C (Eds.) (2005). Adapting educational and psychological tests for cross-cultural assessment. Hillsdale, NJ: Lawrence S. Erlbaum Publishers. [Google Scholar]
  41. Heaton RK, Marcotte TD, Rivera Mindt M, Sadek J, Moore DJ, Bentley H, McCutchan JA, Reicks C, Grant I & the HNRC Group (2004a). The impact of HIV-associated neuropsychological impairment on everyday functioning. Journal of International Neuropsychological Society, 10(3): 317–331. [DOI] [PubMed] [Google Scholar]
  42. Heaton RK, Miller SW, Taylor MJ, Grant I (2004b). Revised comprehensive norms for an expanded Halstead-Reitan Battery: Demographically adjusted neuropsychological norms for African American and Caucasian adults. Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  43. Heaton RK, Ryan L, & Grant I (2009). Demographic influences and use of demographically corrected norms in neuropsychological assessment In Grant I & Adams KM (Eds.). Neuropsychological Assessment of Neuropsychiatric and Neuromedical Disorders. New York, NY: Oxford University Press, 127–155. [Google Scholar]
  44. Heaton RK, Franklin DR Jr., Deutsh R, Letendre S, Ellis RJ, Casaletto K, Marquine MJ, … Grant I & the CHARTER Group (2015). Neurocognitive change in the era of HIV combination antiretroviral therapy: the longitudinal CHARTER study. Clinical Infectious Diseases, 60(3): 473–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Hedden T, Park DC, Nisbett R, Ji LJ, Jing Q, & Jiao S (2002). Cultural variation in verbal versus spatial neuropsychological function across the life span. Neuropsychology 16, 65–73. 10.1037/0894-4105.16.1.65. [DOI] [PubMed] [Google Scholar]
  46. Hernandez MT, Sauerwein HC, Jambaqué I, De Guise E, Lussier F, Lortie A, Dulac O, & Lassonde M (2002). Deficits in executive functions and motor coordination in children with frontal lobe epilepsy. Neuropsychologia, 40, 384–400. [DOI] [PubMed] [Google Scholar]
  47. Huntsinger CS, Jose PE, Liaw FR, & Ching WD (1997). Cultural Differences in Early Mathematics Learning: Comparison of Euro-American, Chinese-American, and Taiwan-Chinese Families. International Journal of Behavioral Development, 21, 371–388. [Google Scholar]
  48. International Test Commission (2005). International Guidelines on Test Adaptation. [www.intestcom.org]
  49. International Test Commission (2017). The ITC Guidelines for Translating and Adapting Tests (Second edition). [www.intestcom.org]
  50. Johansson GM & Häger CK (2019). A modified standardized nine hole peg test for valid and reliable kinematic assessment of dexterity post-stroke. Journal of NeuroEngineering Rehabil, 16: 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Kamalyan L, Hussain MA, Días MM, Umlauf A, Franklin DR, Cherner M, Rivera Mindt M, Artiola i Fortuny L, Grant I, Heaton RK, & Marquine MJ (2020). Neurocognitive Impairment in Spanish-speaking Latinos Living with HIV in the US: Application of the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS). The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Klove H (1963). Clinical neuropsychology In Foster FM (Ed.), The medical clinics of North America. New York, NY: Saunders. [PubMed] [Google Scholar]
  53. Lafayette Instrument Company, Inc. (2002). Grooved Pegboard Test User Instructions. Lafayette, IN: Lafayette Instrument Company, Inc. [Google Scholar]
  54. Li S, Latash ML, Yue GY, Siemionow V, & Sahgal V (2003). The effects stroke and age on finger interaction in multi-finger force production tasks. Clinical Neurophysiology, 114(9), 1646–1655. [DOI] [PubMed] [Google Scholar]
  55. Lundergan WP, Soderstrom EJ, & Chambers DW (2007). Tweezer dexterity aptitude of dental students. Journal of Dental Education, 71, 1090–1097. [PubMed] [Google Scholar]
  56. Mack JL (1969). Validity of the Purdue Pegboard as a screening test for brain damage in a psychiatric population. Perceptual and Motor Skills, 28, 832–4. [DOI] [PubMed] [Google Scholar]
  57. Marquine MJ, Sakamoto M, Dufour C, Rooney A, Fazeli P, Umlauf A, Gouaux B, Franklin D, Ellis R, Letendre S, Cherner M, Heaton RK, Grant I, Moore DJ, & the HNRP Group (2016). The impact of ethnicity/race on the association between the Veterans Aging Cohort Study (VACS) Index and neurocognitive function among HIV-infected persons. Journal of Neuroviroly, 22(4): 442–454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Marquine MJ, Morlett Paredes A, Madriaga C, Blumstein Y, Umlauf A, Kamalyan L, Rivera Mindt M, Suárez P, Artiola i Fortuni L, Heaton RK, & Cherner M (2020a). Demographically-Adjusted Norms for Selected Tests of Verbal Fluency in a Spanish-Speaking Adult Population: Results from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Marquine MJ, Yassai-Gonzalez D, Perez-Tejada A, Umlauf A, Kamalyan L, Morlett Paredes A, Suárez PA, Rivera Mindt M, Franklin DR, Artiola i Fortuny L, Cherner M, & Heaton RK (2020b). Demographically-Adjusted Normative Data for the Wisconsin Card Sorting Test-64 Item: Results from the Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) Project. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Martinez G (2008). Language-in-healthcare policy, interaction patterns, and unequal care on the U.S.-Mexico border. Lang Policy, 7: 345–363. 10.1007/s10993-008-9110-y. [DOI] [Google Scholar]
  61. Matute E, Leal F, Zarabozo D, Robles A, Cedillo C (2000). Does literacy have an effect on stick construction tasks? Journal of International Neuropsychological Society, 6(6): 668–672. [DOI] [PubMed] [Google Scholar]
  62. Montoya JL, Campbell LM, Paolillo EW, Ellis RJ, Letendre SL, Jeste DV, & Moore DJ (2019). Inflammation Contributes to Poorer Complex Motor Performance among Adults Living with HIV on Suppressive Antiretroviral Therapy. JAIDS, 80(1): 15–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Morlett Paredes A, Gooding A, Artiola i Fortuni L, Rivera Mindt M, Heaton RK, Cherner M, & Marquine MJ (2020a). The State of Neuropsychological Test Norms for Spanish-Speaking Adults in the United States. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Morlett Paredes A, Carrasco J, Kamalyan L, Cherner M, Umlauf A, Rivera Mindt M, Suárez PA, Artiola i Fortuni L, Franklin DR, Heaton RK, & Marquine MJ (2020b). Demographically Adjusted Normative Data for the Halstead Category Test in a Spanish-Speaking Adult Population: Results from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS). The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Mulenga K, Ahonen T, & Aro M (2001). Performance of Zambien children on the NEPSY: A pilot study. Developmental Neuropsychology, 20, 375–384. [DOI] [PubMed] [Google Scholar]
  66. Nell V (2000). Cross-cultural neuropsychological assessment: Theory and practice. Mahwah, NJ: Erlbaum. [Google Scholar]
  67. Nutt JG, Lea ES, Van Houten L, Schuff RA, & Sexton GJ (2000). Determinants of tapping speed in normal controls and subjects with Parkinson’s disease; differing effects of brief and continued practice. Movement Disorders, 15, 843–9. [DOI] [PubMed] [Google Scholar]
  68. O’Bryant SE, Edwards M, Johnson L, Hall J, Gamboa A, & O’Jile J (2018). T Texas Mexican American adult normative studies: Normative data for commonly used clinical neuropsychological measures for English- and Spanish-speakers. Dev Neuropsychol, 43(1): 1–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Ojeda CA (2010). Neuropsychological testing Spanish speakers: The challenge of accurately assessing linguistically and culturally diverse individuals. Unpublished doctoral dissertation, University of North Carolina Wilmington, U.S.A. http://dl.uncw.edu/etd/2010-1/ojedac/carlosojeda.pdf.
  70. Ostrosky-Solis F, Efron R, Yund EW (1991). Visual detectability gradients: Effect of literacy. Brain Cognition, 17(1): 42–51. [DOI] [PubMed] [Google Scholar]
  71. Pandya C (2012). Limited English Proficient Workers and the Workforce Investment Act: Challenges and Opportunities. Retrieved from the Migration Policy Institute: https://www.migrationpolicy.org/article/limited-english-proficient-workers-and-workforce-investment-act-challenges-and-opportunities.
  72. Price RC (2014). Determinants of Manual Dexterity in Young Adults Integrative Physiology Graduate Thesis and Dissertations, 38, University of Colorado. [Google Scholar]
  73. Polubinski JP & Melamed LE (1986). Examination of the Sex Difference on a Symbol Digit Substitution Test. Perceptual and Motor Skills, 62(3): 975–982. [Google Scholar]
  74. Pontón MO, Satz P, Herrera L, Ortiz F, Urrutia CP, Young R, … & Namerow N (1996). Normative data stratified by age and education for the Neuropsychological Screening Battery for Hispanics (NeSBHIS): Initial report. JINS, 2(2): 96–104. [DOI] [PubMed] [Google Scholar]
  75. Puente AE, & Agranovich AV (2003). The cultural in crosscultural neuropsychology In Hersen M, Goldstein G, & Beers SR (Eds.), The handbook of psychological assessment (Vol. 1; pp. 31–332). Wiley, New York. [Google Scholar]
  76. Puente AE, & Salazar GD (1998). Assessment of minority and culturally diverse children In Prifitera A & Saklofske DH (Eds.), WISC-III clinical use and interpretation: Scientist-practitioner perspectives (pp. 227–248). San Diego, CA: Academic Press. [Google Scholar]
  77. Reitan RM (1966). A research program on the neuropsychological effects of brain lesions in human beings In Ellis NR (Ed.), International review of research on mental retardation. New York: Academic Press. [Google Scholar]
  78. Reitan RM & Wolfson D (1993). The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation (2nd ed.). Tucson, AZ: Neuropsychological Press. [Google Scholar]
  79. Rivera Mindt M, Byrd D, Saez P, & Manly J (2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Rivera Mindt M, Cherner M, Marcotte TD, Moore D, Bentley H, Esquivel MM, Lopez Y, Grant I, Heaton RK, and the HNRC Group (2003). The functional impact of HIV-associated neuropsychological impairment in Spanish-speaking adults: A pilot study. The Journal of Clinical and Experimental Neuropsychology, 25, 122–132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Rivera Mindt M, Marquine MJ, Aghvinian M, Scott TM, Cherner M, Morlett Paredes A, Kamalyan L, Taylor MJ, Umlauf A, Suárez P, Artiola i Fortuny L, & Heaton RK (2020a). Demographically-Adjusted Norms for the Processing Speed Subtests of the WAIS-III in a Spanish-Speaking Adult Population Living in the U.S.-Mexico Border Region. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Rivera Mindt M, Cherner M, Marquine MJ, Umlauf A, Suárez PA, Artiola i Fortuny L, Morlett Paredes A, Kamalyan L, Scott TM, Heaton AE, Díaz-Santos M, Gooding A, & Heaton RK (2020b). Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project: Summary and Next Steps. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Rosselli M, Ardila A (2003). The impact of culture and education on non-verbal neuropsychological measurements: A critical review. Brain and Cognition, 52, 326–333. [DOI] [PubMed] [Google Scholar]
  84. Rosselli M, Ardila A, Bateman JR, Guzman M (2001). Neuropsychological Test Scores, Academic Performance, and Developmental Disorders in Spanish-Speaking Children. Developmental Neuropsychology, 20(1), 355–373. [DOI] [PubMed] [Google Scholar]
  85. Scott TM, Morlett Paredes A, Taylor MJ, Umlauf A, Artiola i Fortuny L, Heaton RK, Cherner M, Marquine MJ, & Rivera Mindt M (2020). Demographically-Adjusted Norms for the WAIS-R Block Design and Arithmetic Subtests: Results from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  86. Spencer RMC, Zelaznik HN, Diedrichsen J, & Ivry RB (2003). Disrupted timing of discontinuous but not continuous movements by cerebellar lesion. Science, 300, 1437–9. [DOI] [PubMed] [Google Scholar]
  87. Spreen O & Strauss E (1998). A compendium of neuropsychological tests (2nd ed.) New York: Oxford University Press. [Google Scholar]
  88. Strauss E, Sherman EM, & Spreen O (2006). A compendium of neuropsychological tests: Administration, norms, and commentary. American Chemical Society. [Google Scholar]
  89. Suárez PA, Gollan TH, Heaton R, Grant I, Cherner M, & the HNRP Group (2014). Second-language fluency predicts native language stroop effects: evidence from Spanish-English bilinguals. J of Int Neuropsychol Soc, 20(3), 342–348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Suárez PA, Díaz-Santos M, Marquine MJ, Rivera Mindt M, Umlauf A, Heaton RK, Grant I, & Cherner M (2020a). Demographically Adjusted Norms for the Trail Making Test in Native Spanish-speakers from the U.S.-Mexico Border Region. The Clinical Neuropsychologist, 34, XX–XX. [DOI] [PMC free article] [PubMed] [Google Scholar]
  91. Suárez PA, Díaz-Santos M, Marquine MJ, Gollan T, Artiola i Fortuny L, Heaton RK, Cherner M, & the HNRC group (2020b). Role of English Fluency on Verbal and Non-Verbal Neuropsychological Tests in Native Spanish Speakers from the U.S.-Mexico Border Region using Demographically Corrected Norms. The Clinical Neuropsychologist, 34, XX–XX. [Google Scholar]
  92. Trites R (1977). Neuropsychological Test Manual. Ottawa, ON, Canada: Royal Ottawa Hospital; available from Lafayette Instrument Company. [Google Scholar]
  93. U.S. Census Bureau (2017). Facts for Features: Hispanic Heritage Month 2017. Retrieved from https://www.census.gov/newsroom/facts-for-features/2017/hispanic-heritage.html.
  94. Van de Vijver FJR, & Hambleton RK (1996). Translating tests: Some practical guidelines. European Psychologist, 1(2), 89–99. [Google Scholar]
  95. Villardita C, Cultrera S, Cupone V, & Mejía R (1985). Neuropsychological test performance and normal aging. Archives of Gerontology and Geriatrics, 4, 311–319. [DOI] [PubMed] [Google Scholar]
  96. Wang YC, Bohannon RW, Kapellusch J, Garg A, Gershon RC (2015). Dexterity as measured with the 9-Hole Peg Test (9-HPT) across the age span. Journal of Hand Therapy, 28: 53–60. [DOI] [PubMed] [Google Scholar]
  97. Wolf S, Magnuson KA, Kimbro RT (2018). Family poverty and neighborhood poverty: Links with children’s school readiness before and after the Great Recession. Children and Youth Services Review, 79:368–384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  98. Yancosek KE, & Howell D (2009). A Narrative Review of Dexterity Assessments. Journal of Hand Therapy, 22(3), 258–270. [DOI] [PubMed] [Google Scholar]
  99. Zupei L, Jose PE, Huntsinger CS, & Pigott TD (2007). Fine motor skills and mathematics achievement in East Asian American and European Amircan kindergartners and first graders. British Journal of Developmental Psychology, 25, 595–614. [Google Scholar]

RESOURCES