Updated Demographically-Adjusted Norms for the Brief Visuospatial Memory Test-Revised and Hopkins Verbal Learning Test-Revised in Spanish-Speakers from the U.S.-Mexico Border Region: The NP-NUMBRS Project

Abstract

Objective:

We generated demographically-adjusted norms for the Brief Visuospatial Memory Test-revised (BVMT-R) and the Hopkins Verbal Learning Test-revised (HVLT-R) for Spanish-speakers from the U.S.-Mexico border region as part of a larger normative project.

Methods:

Healthy native Spanish-speakers (n=203; Age: 19–60 years; Education: 0–20 years, 59% women) living in Arizona (n=63) and California (n=140) completed the BVMT-R and the HVLT-R as part of the larger Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) project. Raw scores were converted to T-scores utilizing fractional polynomial equations, which considered linear and non-linear effects of demographic variables (age, education, sex). To demonstrate the benefit of employing our population-specific norms, we computed the proportion of our participants whose test performance fell below one standard deviation (T-score<40) when applying published norms from non-Hispanic English-speakers, compared to the base rate derived from the new normative sample.

Results:

The resulting demographically-adjusted T-scores showed the expected psychometric properties and corrected the misclassification in rates of impairment that were obtained when applying norms based on the English-speaking sample. Unexpectedly, participants in Arizona obtained slightly lower HVLT-R T-scores than those in California. This site effect was not explained by available sociodemographic or language factors. Supplementary formulas were computed adjusting for site in addition to demographics.

Conclusions:

These updated norms improve accuracy in identification of learning and memory impairment among Spanish-speaking adults living in the U.S.-Mexico border region. It will be important to generate additional data for elders, as the present norms are only applicable to adults age 60 and younger.

Spanish is the fourth most commonly spoken language in the world and the second in the United States (Ethnologue, 2019; U.S. Census Bureau, 2017). Native Spanish-speakers living in the United States vary across many factors, such as their country of origin, immigration history, their governing educational laws impacting the range of educational attainments before migrating to the U.S., and their current geographical location within in the U.S. Spanish-speakers from the U.S./Mexico border are a growing population with unique socio-cultural and linguistic factors driven by regular exposure to two countries with their respective cultures, traditions, customs, educational policies and laws. As age, sex and educational attainment are known factors impacting aspects of neuropsychological performance (Heaton, Grant & Matthews, 1991; Heaton, & Taylor, 2004; Norman et al., 2011), normative demographic-corrections are a common accepted practice for improving diagnostic accuracy. Given the heterogeneity among Spanish speaking groups, current best practices call for regional population-specific norms to improve the utility of neuropsychological assessments.

There have been a few efforts to generate norms for native Spanish speakers, which vary in their applicability to the heterogenous populations living in the U.S. Morlett-Paredes et al. (2020, this Issue) reviewed the available norms for Spanish-speaking adults living in the U.S. as well international efforts of other colleagues in generating normative data for adults in Spanish-speaking countries. With regard to memory assessment, these include the Rey-Osterreith Complex Figure Recall (Ostrosky-Solís et al., 1994; Ponton et al., 1996; Ostrosky-Solís et al., 1999), the WHO-UCLA AVLT – Spanish version (Ponton et al., 1996), an adaptation of the California Verbal Learning Test (Artiola i Fortuny et al., 1999), and the Verbal Selective Reminding Test (Campo & Morales, 2003; Morales et al., 2010). Norms for these tests also vary with regard to the selection of demographic adjustments and how they are deployed.

Our group previously published demographically adjusted norms for adaptations of the Brief Visuospatial Memory Test-revised (BVMT-R; Benedict, 1997) and the Hopkins Verbal Learning Test-revised (HVLT-R; Brandt & Benedict, 2001) for use with Spanish-speakers from the U.S.-Mexico border region (Cherner et al., 2007). We adapted the verbal stimuli and test instructions to account for language and cultural idiosyncrasies with the goal of producing an instrument that was linguistically neutral across Spanish-speaking groups. Details about the adaptation procedures appear in the Methods section. Our original sample included a modest 127 native Spanish speaking adults of Mexican descent from the U.S.-Mexico border regions of Arizona and California. Results showed that applying the published interpretive standards generated in non-Hispanic English-speakers (Brandt & Benedict, 2001; Benedict et al., 1998) resulted in inflated classification of impairment in our native Spanish speaking sample from the U.S.-Mexico border region, which was most salient at low levels of education. The updated norms we present now include a larger sample (n=203), which improves reliability and representation of cases across the age and education ranges and have updated statistical modeling (including cohort and recruitment site analyses; more information in the methods section). We also provide substantial detail about the demographic characteristics of the sample to better assist the clinician in discerning the appropriateness of the norms for their own patients. Importantly, the current BVMT-R and HVLT-R normative data have the added benefit of being co-normed with a larger battery of tests, whereas our 2007 normative data were published in isolation.

Norms for the full battery of tests is presented in this TCN Special Issue on the Neuropsychological Norms for the U.S.-Mexico Border Region in Spanish (NP-NUMBRS) project. The battery covers multiple domains typically assessed in a comprehensive neuropsychological evaluation, adding to the toolkit available to facilitate direct comparisons of demographically corrected test performance across instruments; i.e., verbal fluency (Marquine et al., 2020), speed of information processing (Rivera Mindt et al., 2020; Suárez et al., 2020), attention/working memory (Gooding et al., 2020; Scott et al., 2020), executive functioning (Morlett Paredes, Carrasco, et al., 2020; Suárez et al., 2020, Marquine et al., 2020), learning and memory (Díaz-Santos et al., Current Paper), visuospatial skills (Scott et al., 2020), and fine motor skills (A. Heaton et al., 2020). To illustrate the utility of the newly generated normative data, we also apply the existing BVMT-R and HVLT-R norms for English-speakers to our Spanish-speaking sample to quantify the degree of diagnostic misclassification that would result from applying mismatched normative standards.

Method

Participants

The full NP-NUMBRS sample (N = 254) had a mean age of 37.32 (SD = 10.24), mean education of 10.64 (SD = 4.34), with 58.66% women (Cherner, Marquine et al., 2020). The sample for this study was composed of the 127 participants included in the original normative study by Cherner et al. (2007) with an additional 76 participants recruited later as healthy controls in a research study using identical methods and eligibility criteria. Of the total sample of 203 healthy, native Spanish speakers, 202 had valid data on the BVMT-R and 201 on the HVLT-R, with 200 participants having complete data on both tests. All participants in the present study were part of the larger NP-NUMBRS project that provides demographic adjustments for a co-normed battery of tests. A detailed description of the recruitment methods and characteristics of the sample appear in this Special Issue in the methodology paper by Cherner, Marquine, and colleagues (2020). Briefly, participants were recruited from two U.S.-Mexico borderland regions: Tucson, Arizona (n = 63) and San Diego, California (n = 140) in two study waves (Cohort 1 [1998–2000]: n=132, Cohort 2 [2006–2009]: n=71). The first cohort was recruited specifically to develop neuropsychological test norms for healthy Spanish-speaking adults living in the U.S.-Mexico border region. The second cohort comprised healthy control participants from a study of cognitive and functional outcomes among Spanish speakers living with HIV in San Diego. Inclusion and exclusion criteria for healthy participants in both studies were identical. To be eligible, participants had to be men and women between the ages of 19 and 60 years old who were native Spanish-speakers and lived in the U.S. at least part of the time. All potential participants completed a comprehensive structured interview screening for central nervous system injury or disease, developmental disability, serious psychiatric conditions (e.g., psychosis, bipolar disorder, severe depression), lifetime substance abuse or dependence, or any other serious medical condition. Potential participants with less severe medical conditions (e.g., hypertension, metabolic disorders), or peripheral injuries were carefully reviewed by the respective senior investigator (Heaton, Artiola i Fortuny, or Cherner) before determining eligibility.

As detailed in the methodology article (Cherner, Marquine et al., 2020), potential participants were also excluded if English was their dominant language. This was determined with a two-pronged approach, using both subjective and objective measures of language use. Specifically, we obtained self-report of language preference across a number of daily life activities (e.g., listening to the radio, watching TV, reading, speaking with family and friends, praying). As detailed in the article by Suárez et al. (2020, this issue), participants also completed the Controlled Oral Word Association Test with letters F-A-S in English (Strauss, Sherman, & Spreen, 2006) and P-M-R in Spanish (Artiola i Fortuny, Hermosillo Romo, Heaton, & Pardee, 1999; Strauss et al., 2006). We estimated English fluency as the ratio of FAS to total words in both languages [FAS/(FAS+PMR)] (Suárez et al., 2014) and classified as English-dominant those with scores ≥ .67 (i.e., greater than or equal to 2/3 of all words in English). Any participants in this range were excluded from the study. Participants with scores less than or equal to .33 (i.e., fewer than one third of words in English) were considered Spanish-dominant, and those in between were considered bilingual. We recognize that this is not a comprehensive method for determining language ability. Our intent was simply to exclude subjects whose better language is English. Varying degrees of second language skills are expected in a borderland population, and this makes our normative sample more representative of Spanish speakers in the U.S. than norms collected in the country of origin. It should be noted that all participants learned Spanish as their first language and preferred to be tested in Spanish. In this Special Issue, Suárez et al. (2020) explores the effect of English fluency on performance across all the tests in the NP-NUMBRS battery. They conclude that the demographic corrections we generated for the learning and memory tests sufficiently correct for any differences between Spanish-dominant and bilingual individuals (defined in the way we mention above). As such, we do not additionally correct for degree of English fluency relative to Spanish fluency in the normative adjustments. Please refer to Cherner, Marquine et al. (2020) for additional details about the overall methodology and statistical approach, including participant recruitment, complete demographic characteristics of the NP-NUMBRS sample, and comparisons by study cohort and study site.

Procedure

All NP-NUMBRS participants completed a comprehensive neuropsychological test battery in Spanish assessing multiple domains (verbal fluency, speed of information processing, attention/working memory, executive function, visuospatial skills and fine motor skills) in addition to the tests of verbal and visual learning and memory described in the present paper. Table 2 shows the educational, social and linguistic background information for those with valid data for the BVMT-R and HVLT-R.

Table 2.

Educational, Social, and Language Background Characteristics of Participants (N =203) with Data on Tests of Learning and Memory from the Neuropsychological Norms for the US-Mexico Border Region in Spanish (NP-NUMBRS) Project

Characteristics	Descriptives
	M (SD), %	n
Educational Background
Years of education in Mexico	8.51 (4.81)	185
Years of education in the U.S.	2.52 (4.83)	185
Proportion of education by country	--	185
More years of education in Mexico	85.41%	158
More years of education in the U.S.	13.51%	25
Equal number of years of education in both countries	1.08%	2
Type of school attendeda	--	193
Large	55.96%	108
Regular	38.86%	75
Small	5.18%	10
Number of students in the class	--	197
Less than 21	17.26%	34
21 to 30	36.55%	72
31 to 40	22.84%	45
40+	23.35%	46
Had to stop attending school to work	--	182
Yes	30.77%	56
Social Background
Mother’s years of education	5.73 (3.85)	129
Father’s years of education	6.87 (5.23)	119
Years lived in Mexico	26.77 (12.58)	195
Years living in the U.S.	10.79 (11.12)	195
Childhood SESb	--	201
Very poor	5.97%	12
Poor	29.85%	60
Middle class	53.73%	108
Upper class	10.45%	21
Worked as a child	--	198
Yes	52.02%	103
Reason to work	--	101
Help family financially	41.58%	42
Own benefit	58.42%	59
Age started working as a child	12.66 (3.27)	99
Currently Gainfully Employed	--	176
Yes	68.18%	120
Language
First Language	--	200
Spanish	98.50%	197
English	0.50%	1
Both	1.00%	2
Current Language Use Ratingc
Radio or Television	2.36 (1.03)	201
Reading	2.23 (1.18)	201
Math	1.52 (1.03)	199
Praying	1.28 (0.76)	193
With family	1.56 (0.90)	196
Performance-based language fluencyd	--	161
Spanish dominant	62.73%	101
English dominant	0.00%	0
Spanish-English Bilingual	37.27%	60

Note. M: mean; SD: standard deviation; SES: socioeconomic status; US: United States

^aType of school attended: ‘large’ refers to large school that had many classrooms and room to play; ‘regular’ refers to a school of regular size that had at least one classroom per grade and room to play; and small school refers to a small school with less than one classroom per grade.

^bChildhood SES was assessed by the following question and response options: “As a child, your family was: (1) Very Poor; (2) Poor; (3) Middle Class; (4) Upper Class”.

^cRatings for each activity ranged from 1 “Always in Spanish” to 5 “Always in English”, with 3 being “similarly in English and Spanish”.

^dAssessed via administration of the Controlled Oral Word Association Test with letters F-A-S in English and P-M-R in Spanish (Artiola i Fortuny et al., 1999; Strauss et al., 2006). Based on the ratio of words produced in the FAS task to total words in both tasks (English fluency ratio = FAS/[FAS+PMR]; Miranda et al., 2016; Suárez et al., 2014), we classified participants as Spanish dominant (scores ≤ 0.33), or bilingual (scores:0.34 to 0.66), with scores ≥ 0.67 considered English-dominant (ineligible).

Adaptation of BVMT-R and HVLT-R Stimuli and Instructions.

The original article by Cherner et al. (2007), which provided the first Spanish language norms for the BVMT-R and HVLT-R, details the systematic modifications that were deployed to account for language and cultural idiosyncrasies in the adaptation of verbal stimuli and test instructions. The original English language versions of these tests were translated into Spanish and adapted to make them as culturally and linguistically neutral as possible across Spanish-speaking regions to maximize their generalizability. To this end, standard forward and back translations of the stimuli and instructions were carried out in an iterative manner with review by psychology professionals who were native Spanish-speakers from a number of regions (Argentina, Colombia, Cuba, Mexico, Puerto Rico, and Spain). This process resulted in a few word substitutions, kept within the same semantic category, on the various forms of the HVLT-R compared to a literal translation of the original version. Substitutions were based on whether a translated item had different names in different countries or to disambiguate stimuli that resulted in a word with multiple meanings (i.e., within and outside the semantic categories of the word list). Of note, for these reasons, the resulting stimuli differ slightly from the Spanish language version that is currently available from the test publisher (https://www.parinc.com/). As we detail in our original article, in Form A, because the word “tent” has various translations depending on country, we replaced it with “mansion.” In Form B, we used “tequila” instead of “bourbon” because that spirit is uncommon in the lexicon and the word is not translated when used, except as “Borbón” in reference to the royal house. At the time of our adaptation, we were unaware of any other Spanish language translation.

BVMT-R and HVLT-R Administration and Scoring Procedures.

In the current study, the administration and scoring protocols were faithful to the original English language version of these tests (PAR, Inc; Brandt & Benedict, 2001; Benedict, Schretlen, Groninger, & Brandt, 1998). Participants in Cohort 1 were randomly assigned to one the first two alternate test forms. Cohort 2 participants were part of a parent study that administered the first form of each test. Thus, approximately two-thirds of the sample received the first form and one-third received the second form of each test (BVMT-R:138 Form 1, 64 Form 2; HVLT-R: 139 Form A, 61 Form B [missing form data for one participant]). Testing was performed in Spanish by trained bilingual psychometrists using standard procedures. Scoring was done according to the guidelines detailed in the published manual for each measure. An independent examiner verified the scoring of the test protocol prior to entry into a database.

Statistical Analyses

Descriptive Characteristics and Distribution of Raw Scores.

We computed descriptive statistics for raw test scores. BVMT-R variables include learning Trials 1 through 3, Total Recall (also called immediate recall or total learning = sum of trials 1–3), Delayed Recall, Recognition True Positives, Recognition False Positives, Discrimination, Response Bias, and Copy scores. HVLT-R variables included Trial 1, Trial 2, Trial 3, Total Recall (sum of learning trials 1–3), Delayed Recall, Percent Retained, Recognition True Positives, Recognition False Positives, and Discrimination scores. We examined the distribution of BVMT-R and HVLT-R raw scores via Shapiro-Wilk tests. We then examined univariable association of age and years of education with BVMT-R and HVLT-R raw scores via a series of Pearson product moment correlation coefficients (or Spearman correlation coefficients for variables with skewed distributions), and the association between sex and raw scores via independent sample t-tests (or Wilcoxon rank-sum tests for variables with skewed distributions). Testing and selection of linear or non-linear terms for effects of age and education was performed utilizing an automated procedure available in the R package mfp (function mfp). The procedure uses Likelihood ratio tests with significance level of 0.05. Full details of the procedure is described in Sauerbrei et al. (2006).

We also computed a series of separate linear regression models to explore potential two-way interactions between demographic variables (i.e., age, education, and sex) on Total Recall and Delayed Recall measures. Beta values and r-square values are reported for the linear-regression models. We additionally tested for cohort effects via independent samples t-tests by comparing the resulting T-scores between participants in Cohort 1 and Cohort 2, as well as between participants tested in Arizona and California.

Generation of normative T-scores.

The NP-NUMBRS project used a uniform procedure for the generation of T-scores (M = 50; SD = 10) as the metric for demographically adjusted test performance. Fully adjusted population-specific T-scores represent a person’s level of cognitive functioning compared to expectations for healthy individuals with the same combination of age, education, and sex, coming from a similar cultural and linguistic background.

As detailed in the methodology paper is this issue (Cherner, Marquine, et al., 2020), raw scores were first converted to normal quantiles and transformed into normally distributed scaled scores (M = 10; SD = 3). Higher scaled scores correspond better performance. Next, the scaled scores were regressed on age, education, and sex using in fractional polynomial multiple regression equations (Royston & Altman, 1994). This method allows examination of linear and non-linear effects for numeric/continuous predictors (i.e., age and education), selecting the best curve (p < 0.05) from several options: linear, quadratic, logarithmic and other combinations of fractional polynomials of first (e.g. x^m) and second degree (e.g. x^m1 + x^m2) with powers (m_i) ranging from −2 to +3. Residuals, the calculated differences between the recorded scaled scores and the scores predicted by the mfp model, were then standardized and scaled to obtain the normative T-scores, which have a mean of 50 and SD = 10.

Cherner, Marquine et al. (2020) describe in detail the sensitivity analyses performed for each test in the NP-NUMBRS. Briefly, we followed the recommendations by Royson & Sauerbrei (2009) to ensure the stability of the fractional polynomial multiple regression equations. To facilitate the information for the reader, we reproduce the text in that article as a footnote here¹. For both tests, we report scaled scores and demographically adjusted T-scores for Total Recall and Delayed Recall, which met the requisite distributional properties. For those outcomes that did not (i.e., those with limited range or very skewed raw score distribution), we report percentile ranges. From the BVMT-R, these included Trials 1, 2, and 3, Recognition True Positives, Recognition False Positives, Recognition Discrimination, Response Bias, and Copy scores. From the HVLT-R, these were Trials 1, 2, and 3, Percent Retained, Recognition True Positives, Recognition False Positives, and Recognition Discrimination scores.

Comparisons between published and newly derived norms.

T-scores were calculated for BVMT-R and HVLT-R Total Recall and Delayed Recall based on published norms for English-speaking NH Whites (Benedict et al., 1998; Brandt & Benedict, 2001) and NH Blacks/African Americans in the U.S. (Norman et al., 2011). To illustrate the utility of using population-specific norms, we calculated the proportion of our participants whose performance would fall below one SD using these mismatched normative standards. The −1 SD cutpoint was selected based on data that classifying impaired performances as those below 1 SD results in the best tradeoff between sensitivity and specificity for discriminating between healthy and neurologic illness groups (Heaton et al., 2004). We compared the performance of our resulting regional norms to the existing published norms using McNemar’s tests.

In order to examine whether demographic adjustments derived from NH Whites and Blacks/African Americans (Norman et al., 2011) adequately corrected for effects of demographic characteristics (age, education, and sex) on test performance in the current sample of Spanish-speakers, we examined the univariable association between these demographic factors and T-scores calculated using existing norms for NH Whites and Blacks/African Americans. These univariable associations were tested via a series of Pearson product moment correlation coefficients and independent sample t-tests.

Results

Sample Characteristics

Table 1 shows age, sex, and education of the study sample stratified by years of education in order to give the reader a sense of the sampling distribution across education, age, and sex. Note that years of age and education were used as continuous variables in the normative equations. There were slightly more women than men in the sample as a whole, with comparable proportions across levels of education. Table 2 summarizes the educational, social and language background characteristics of the subset of the sample with available data on these variables. The majority of participants completed more years of formal education in Mexico than in the U.S., and nearly a third of the sample reported that they had to stop attending school to work. On average, education completed by both parents was 5–6 years. Participants lived a majority of their lives in their country of origin (Mexico). Approximately half of participants described their childhood socioeconomic status as middle class, with about a third reporting having been poor or very poor. Half of participants reported working for money during childhood, and over two-thirds of participants were currently gainfully employed. A separate manuscript (in preparation at the time of writing) examines the influence of sociodemographic background on test performance in the NP-NUMBRS sample. Self-ratings of language use in everyday activities indicated that Spanish was the predominant language used in daily life. As mentioned, based on performance-based verbal fluency measures, almost two-thirds of the sample was monolingual Spanish-speaking or strongly Spanish dominant, with the remaining third considered bilingual.

Table 1.

Demographic characteristics of the normative sample stratified by years of education (N =203)

	Years of Educationa
	≤ 6 (n=46)	7–10 (n=44)	11–12 (n=51)	≥13 (n=62)
Age (years), M (SD)	40.28 (9.90)	37.91 (9.22)	35.33 (10.64)	37.79 (10.88)
Education (years), M (SD)	4.61 (1.63)	8.61 (0.92)	11.82 (0.39)	15.91 (1.71)
% Female	58.69%	56.81%	66.67%	54.84%

Note. M: mean; SD: standard deviation.

^aNorms use years of education as a continuous variable; here we show education groups for descriptive purposes.

Descriptive Characteristics and Distribution of Raw Scores

Table 3 shows the descriptive summary characteristics of raw scores on BVMT-R and HVLT-R variables. Results from Shapiro-Wilk tests indicated that none of the variables were normally distributed in raw scores. Table 4 shows the association of raw scores with demographic variables. Examination of univariable relationships between raw scores and demographic characteristics revealed a robust association between higher education and better performance on most measures of verbal and non-verbal learning and memory. Increasing age correlated with worse performance on the BVMT-R Total Recall, Delayed Recall and Copy raw score, as well as the HVLT-R Total Recall and Discrimination raw scores. Two-way interactions with demographic variables (i.e., age, education, and sex) showed significant age by sex interactions on HVLT-R Total Recall (p = .04) and Delayed Recall (p = .01) scores. Separate stratified models by sex showed that older age was significantly associated with worse performance on these scores among women (p < .01 for Total Recall and Delayed Recall), but not men (p = .88 and p = .64, respectively). There were no other significant demographic interactions.

Table 3.

Mean, standard deviation (SD), and range of raw scores on the Brief Visuospatial Memory Test-Revised (BVMT-R; N = 202) and the Hopkins Verbal Learning Test-Revised (HVLT-R; N = 201)

BVMT-R	Mean (SD)	Range
Trial 1	5.39 (2.62)	0 – 12
Trial 2	8.18 (2.81)	1 – 12
Trial 3	9.51 (2.51)	0 – 12
Total Recall	22.99 (7.16)	3 – 36
Delayed Recall	9.26 (2.69)	1 – 12
Recognition True Positives	5.74 (0.57)	3 – 6
Recognition False Positives	0.23 (0.59)	0 – 4
Recognition Discrimination	5.51 (0.93)	1 −6
Response Bias	0.49 (0.13)	0.17 – 0.90
Copy Trial	11.39 (0.85)	9 – 12
HVLT-R
Trial 1	6.60 (1.76)	3 – 11
Trial 2	9.07 (1.71)	5 – 12
Trial 3	9.97 (1.54)	5 – 12
Total Recall	25.64 (4.37)	16 – 35
Delayed Recall	8.79 (2.11)	3 – 12
Percent Retained	86.14 (14.86)	38 – 129
Recognition True Positives	11.44 (0.94)	8 – 12
Recognition False Positives	0.84 (0.99)	0 – 5
Recognition Discrimination	10.59 (1.50)	5 – 12

Table 4.

Association between raw scores and demographic characteristics for Brief Visuospatial Memory Test-Revised (BVMT-R; N =202) and Hopkins Verbal Learning Test-Revised (HVLT-R; N = 201)

	Agea	Educationa	Sexb
			Men	Women
BVMT-R
Total Recall	−0.26***	0.52***	23.73 (7.19)	22.51 (7.15)
Delayed Recall	−0.25***	0.44***	9.61 (2.61)	9.03 (2.73)
Discrimination	−0.08	0.42***	5.56 (0.84)	5.48 (0.98)
Response Bias	−0.13	−0.08	0.49 (0.11)	0.48 (0.14)
Copy	−0.26***	0.09	11.41 (0.83)	11.38 (0.87)
HVLT-R
Total Recall	−0.18*	0.39***	25.01 (4.45)	26.05 (4.27)
Delayed Recall	−0.14	0.40***	8.57 (2.22)	8.86 (2.18)
Percent Retained	−0.04	0.29***	85.67 (15.73)	85.72 (16.39)
Discrimination	−0.20**	0.34***	10.39 (1.67)	10.70 (1.37)

Note. Based on results from Spearman ρ^a and Wilcoxon rank-sum tests^b.

^*p<.05,

^**p<.01,

^***p<.001

Table 5 shows the raw-to-scaled score conversions for BVMT-R and HVLT-R Total Recall and Delayed Recall. Given limited ranges and highly skewed distributions of the other outcomes, in Tables 6a and 6b we provide percentile conversions for individual learning trials and Recognition Discriminability for both tests, as well as the BVMT-R Copy Trial and HVLT-R Percent Retained scores (Delayed Recall score divided by the higher of trial 2 or 3 × 100).

Table 5.

Raw-to-scale score conversions on the Total Recall and Delayed Recall from the Brief Visuospatial Memory Test-Revised scores (BVMT-R; N =202) and the Hopkins Verbal Learning Test-Revised Scores (HVLT-R; N =201)

Scaled	BVMT-R		HVLT-R
	Total Learning	Delayed Recall	Total Learning	Delayed Recall
19	--	--	36	--
18	--	--	35	--
17	36	--	--	--
16	34–35	--	34	--
15	33	--	32–33	12
14	32	12	31	--
13	30–31	--	30	11
12	28–29	--	29	10
11	26–27	11	27–28	--
10	23–25	10	26	9
9	21–22	9	24–25	8
8	17–20	8	22–23	--
7	14–16	7	21	7
6	12–13	5–6	19–20	6
5	9–11	4	--	5
4	5–8	3	17–18	4
3	4	2	--	--
2	1–3	1	3–16	2–3
1	0	0	0–2	0–1

Table 6a.

Raw scores to percentile conversions for individual learning trials, discrimination and copy on the Brief Visuospatial Memory Test-Revised (BVMT-R; N = 202)

Percentile	Trial 1	Trial 2	Trial 3	Discrimination	Copy
1st	0	1	1	2	<9
2nd	1	2	3	3	9
5th	--	3	5	--	--
9th	2	4	6	4	10
16th	--	5	7	--	--
25th	3	6	8	5	--
37th	4	8	9	--	11
50th	5	9	10	--	--
63d	6	--	--	--	--
75th	7	10	11	--	--
84th	8	11	--	--	--
91st	9	--	--	--	--
95th	10	--	--	--	--
98th	11	--	--	--	--
99th	12	12	12	6	12

Note. BVMT-R raw scores of individual learning trials, discrimination and copy did not meet sensitivity criteria due to the skewness of the raw data and/or the limited range of the data. We provide percentile scores to assist clinicians in their decision making while evaluating its utility, and applicability to their respective clinical needs.

Table 6b.

Raw scores to percentile conversions for individual learning trials, percent retained and discrimination on the Hopkins Verbal Learning Test-Revised Scores (HVLT-R; N = 201)

Percentile	Trial 1	Trial 2	Trial 3	% Retained	Discrimination
1st	3	5	<6	40%	6
2nd	--	6	6	50%	--
5th	--	--	7	60%	7
9th	4	--	--	70%	8
16th	--	7	8	73%	9
25th	5	--	9	75%	10
37th	6	8	--	82%	--
50th	--	9	10	89%	--
63d	7	--	--	91%	11
75th	--	10	11	--	--
84th	8	--	--	--	--
91st	9	11	--	100%	--
95th	10	--	--	--	--
98th	--	--	--	--	--
99th	11	12	12	>100%	12

Note. HVLT-R learning trials, percentage retained, and discriminations raw scores did not meet sensitivity criteria due to the skewness of the raw data and/or the limited range of the data. We provide percentile scores to assist clinicians in their decision making while evaluating its utility, and applicability to their respective clinical needs.

T-score Equations

Table 7 details the fractional polynomial equations to compute T-scores for Total Recall and Delayed Recall of the BVMT-R and HVLT-R, which were weighted for demographic characteristics (see online supplement for a digital calculator). The resulting T-scores were normally distributed and had a mean of 50 and a SD of 10. T-scores ranged from 25–75 for BVMT-R Total Recall, and 23–73 on BVMT-R Delayed Recall, and 25 to 82 on HVLT-R Total Recall, 26–71 on HVLT-R Delayed Recall. As expected, Pearson product moment correlation coefficients showed no significant relationships of age or education with any of the T-scores, and there were no significant sex effects. There were no significant age × sex interactions on HVLT-R Total Recall (p = .17) or Delayed Recall T-scores (p = .09). There were no differences in T-scores based on form used for the BVMT-R (Form 1: n = 138; Form 2: n = 64; Total Recall: p = .46, Delayed Recall: p = .63) or the HVLT-R (Form A: n = 139; Form B: n = 61 [data missing on form for one case]; Total Recall: p = .65; Delayed Recall: p = .63).

Table 7.

T-score equations for the Total Recall and Delayed Recall Trials of the Brief Visuospatial Memory Test-Revised (BVMT-R; N = 202)) and the Hopkins Verbal Learning Test-Revised (HVLT-R; N = 201).

BVMT-R	$10\times \left(\frac{\text{SS\hspace{0.17em}BVMT}-\text{R Total Recall }-\left(\mathbf{8.2332}-\mathbf{6.3184}*\frac{\mathit{\text{age}}}{100}+\mathbf{3.4264}*\frac{(\text{edu}+1)}{10}+\mathbf{0.2493}*\text{sex}\right)}{\mathbf{2.3370}}\right)+50$
Total
Recall
BVMT-R	$10\times \left(\frac{\text{SS\hspace{0.17em}BVMT}-\text{R Delayed Recall }-\left(\mathbf{8.3379}-\mathbf{6.1433}*\frac{\mathit{\text{age}}}{100}+\mathbf{3.0829}*\frac{(\text{edu}+1)}{10}+\mathbf{0.4407}*\text{sex}\right)}{\mathbf{2.5717}}\right)+50$
Delayed
Recall
HVLT-R	$10\times \left(\frac{\text{SS\hspace{0.17em}HVLT}-\text{R Total Recall }-\left(\mathbf{8.5385}-\mathbf{4.0261}*\frac{\mathit{\text{age}}}{100}+\mathbf{2.8295}*\frac{(\text{edu}+1)}{10}-\mathbf{0.9471}*\text{sex}\right)}{\mathbf{2.7131}}\right)+50$
Total
Recall
HVLT-R	$10\times \left(\frac{\text{ SS HVLT }-\text{ R Delayed Recall }-\left(\mathbf{8.1527}-\mathbf{3.1480}*\frac{\mathit{\text{age}}}{100}+\mathbf{2.8320}*\frac{(\text{edu}+1)}{10}-\mathbf{0.6092}*\text{ sex }\right)}{\mathbf{2.7093}}\right)+50$
Delayed
Recall

Note. These formulas should be applied to education level ranges from 0–20 and age 19–60. Using values outside these ranges might result in extrapolation errors. Sex: Male=1; Female=0 SS=observed scaled score; Edu=years of education; Age=years of age.

There were no significant cohort effects on any of the T-scores (p values > .20). There were also no site effects on the BVMT-R Total Recall (p = .22) or Delayed Recall (p = .99) T-scores. Unexpectedly, however, there was an effect of testing site on the HVLT-R. Total Recall T-scores from Arizona (M = 44.77, SD = 8.34) were significantly lower than those from California (M = 52.35, SD = 9.84, p <.001), with a similar pattern on Delayed Recall (Arizona: M = 46.11, SD = 9.36; California: M = 51.76, SD = 9.83; p <.001). In order to understand better the factors driving these differences, in follow-up analyses we investigated potential site-based differences in demographic characteristics, as well as social, educational and language background via a series of independent sample t-tests and Chi-Square tests (see Supplemental Table 1). Results showed no significant differences by site in age, years of education, or sex distribution. With regard to other background characteristics, participants in Arizona were less likely than those from California to have attended a large size school and less likely to report that they had to stop attending school to work. They also reported higher childhood SES, and more years of parental education. While there were no significant site differences on the percent of participants who reported working as a child, among those who did, participants in Arizona were older when they did so, and were less likely to report that they worked in order to help their family financially. Furthermore, participants in Arizona were more likely to be bilingual.

We then tested a multivariable linear regression model for HVLT-R Total Recall T-score including testing site and variables that differed across groups (i.e., type of school, stopped attending school to work, childhood SES, and bilingualism) as predictors. We did not include parental education in these initial analyses, given significant missing data on this variable among participants in California, and we did not include the reason for working as a child and at what age, since those variables were relevant only for the subset of the sample who reported working as a child. Results from this multivariable model continued to show that participants in Arizona had lower T-score than participants in California (Beta = −7.66, SE = 1.84, p < .001), even after adjusting for significant covariates. Analyses on HVLT-R Delayed Recall yielded similar findings (Beta = −5.69, SE = 1.90, p < .01). In the subset of participants with data on parental education, adjusting for mother and father’s education (in addition to other significant covariates included in the prior model), resulted in similar findings. The effect of study site remained significant among participants who worked as children, when adjusting for the reason to work as a child and the age at which they started. As a result, we created additional T-score formulae for HVLT-R Total Recall and Delayed Recall also adjusting for site (Table 8).

Table 8.

Hopkins Verbal Learning Test-Revised (HVLT-R) Total Recall and Delayed Recall T-score formula including site (California, Arizona) as a predictor

HVLT-R	$10\times \left(\frac{\text{SS}+\text{HVLT}-\text{R Total Recall }-\left(\mathbf{7.2650}-\mathbf{4.0909}*\frac{\mathit{\text{age}}}{100}+\mathbf{2.7952}*\frac{\left(\text{edu}+1\right)}{10}-\mathbf{1.1726}*\text{sex}+2.0687\times \text{site }\right)}{\mathbf{2.5409}}\right)+50$
Total
Recall
HVLT-R	$10\times \left(\frac{\text{SS\hspace{0.17em}HVLT}-\text{R Delayed Recall }-\left(\mathbf{7.2024}-\mathbf{3.1964}*\frac{\mathit{\text{age}}}{100}+\mathbf{2.8065}*\frac{(\text{edu}+1)}{10}-\mathbf{0.7775}*\text{sex}+1.5436\times \text{site }\right)}{\mathbf{2.6147}}\right)+50$
Delayed
Recall

Note. These formulas should be applied to education level ranges from 0–20 and age 19–60. Using values outside these ranges might result in extrapolation errors. Sex: Male=1; Female=0. Site: CA=1; AZ=0. California (CA)=1; Arizona (AZ)=0. California N = 139; Arizona N = 62. SS=observed scaled score; Edu=years of education; Age=years of age.

Testing the Performance of the NP-NUMBRS Norms Compared to Existing Norms for Non-Hispanic (NH) White and Non-Hispanic (NH) African American/Black Populations

As Figure 1 indicates, compared to rates of “impaired” performance (using a −1 SD cutpoint; T-score < 40) based on our newly derived norms (from Table 7), the English language NH White norms resulted in significant higher rates of impairment in the BVMT-R and HVLT-R. Impairment rates utilizing NH Black norms were generally comparable to those obtained with our norms for Spanish-speakers (p > .05), except that rates of impairment were significantly lower on the BVMT-R Delayed Recall (p = .03).

Figure 1.

Proportion of cases with T-scores below 1 standard deviation (−1SD; TS<40) on the Brief Visuospatial Memory Test-Revised (BVMT-R; N = 202) and the Hopkins Verbal Learning Test-Revised (HVLT-R; N = 201) Total Recall and Delayed Recall using the newly derived NP-NUMBRS norms vs. demographically adjusted norms for non-Hispanic Black and White English-speakers in the U.S. Based on the normal distribution, the expected base-rate is 15 to 16%. Asterisks denote a significant difference based on McNemar’s tests compared to proportions obtained with the Spanish-speaking borderland norms. *p <.05; **p < .0001

NH = Non-Hispanic

NP-NUMBRS = Neuropsychological Norms for the US-Mexico Border Region in Spanish

Unlike raw test scores, because T-scores include demographic adjustments, it is expected that they will no longer be associated with the demographic factors for which they correct. We tested whether the demographic adjustments conferred by the norms from the English-speaking samples were unrelated to (i.e., corrected for) age, education, and sex effects in our Spanish-speaking sample. Results from univariable analyses examining the effects of demographics on T-scores derived from norms for NH Whites in the present sample of Spanish-speakers, showed that there were significant negative effects of age on HVLT-R Total Recall T-Scores (r = −0.15, p = .03) and BVMT-R Delayed Recall T-Scores (r = −0.18, p = .01), significant negative effects of education on HVLT-R Total Recall T-Scores (r = −0.24, p <.001) and HVLT-R Delayed Recall T-Scores (r = −0.22, p < .01) and significant positive effects of education on BVMT-R Total Learning Recall T-Scores (r = .27, p < .001) and Delayed Recall T-Score (r = .27, p < .001). There were also significant effects of sex on HVLT-R Delayed Recall T-Scores (p < .01) and BVMT-R Total Recall T-Score (p < .001), with women obtaining lower T-Score than men on both measures. Similar analyses using norms for NH Blacks, showed a significant positive effects of age on HVLT-R Delayed Recall T-Score (r = 0.15, p = .04), and significant positive effects of education on BVMT-R Total Learning Recall T-Score (r = 0.21, p < .01) and BVMT-R Delayed Recall T-Score (r = 0.21, p <.01). There were also significant sex effects on BVMT-R Total Learning Recall T-Scores (p < .001) and BVMT-R Delayed Recall T-Scores (p < .01), with women obtaining lower T-Scores than men. These results show that T-scored generated with these mismatched norms do not confer appropriate demographic adjustments.

Discussion

The purpose of this study was to update our earlier demographically adjusted norms for Spanish language versions of the BVMT-R and the HVLT-R (Cherner et al., 2007) with a larger sample of native Spanish speaking adults from the U.S.-Mexico border region, and as part of a co-normed comprehensive battery of tests. We present raw-to-scaled score conversions and normative equations to arrive at demographically adjusted T-scores for learning and delayed recall measures, as well as percentile ranks for retention and discriminability measures, whose distributional properties did not allow for T-score calculations.

As expected, and consistent with prior findings in other groups (Artiola i Fortuny et al., 1999; Heaton et al., 2004; Norman et al., 2011) most raw scores were influenced by years of education. Higher formal education showed moderate associations (r’s between .29 and .52) with better test performances on the majority of outcomes. In our sample of young-to-middle age adults (≤ 60), we did observe some weak but significant associations (r’s between −.18 and −.26) between older age and worse performance on both the BVMT-R and HVLT-R in learning and delayed recall. We also observed an age by sex interaction on HVLT-R learning and delayed recall, such that older age was associated with worse performance only among women. Considering that our sample did not include people over age 60, this interaction could reflect differences in performance between pre- and post- menopausal women, as estrogen changes among women have been shown to impact overall cognition, and in particular, verbal memory (Barth et al., 2015; Epperson et al., 2013; Fuh et al., 2006; Greendale et al., 2010; Gur & Gur, 2002; Maki et al., 2010; Randolph et al., 2011).

Our results cannot be directly compared with the only other published HVLT-R norms for Spanish speakers (Guàrdia-Olmos, Peró-Cebollero, Rivera & Arango-Lasprilla, 2015) due to differences in the methods and norming approach. Some similarities, however, can be gleaned. Here we focus on the Mexican sample for the Guàrdia-Olmos et al. study, as it is the most similar to the NP-NUMBRS participants. The sample comprised an impressive 1300 Mexicans residing in Mexico. The proportion of women was a little larger than ours (67% vs 59%). No effects of sex were reported in that sample. The mean age and age range of the sample was higher than ours (52.5 [20.5] vs. 37.32 [10.24]), as their recruitment included elders to age 95, so it is not surprising that the authors found a stronger influence of age on test performance than we did in our study. The proportion of participants who completed some higher education was similar between their sample (22%) and ours (30%), and both studies found a positive effect of education on test performance. Importantly, though, in our study we modeled a full range of formal education (0–20) as a continuous predictor, while Guàrdia-Olmos et al. (2015) grouped their sample according to whether they completed up to 12 years vs. exceeded 12 years. Given the field’s understanding of the effects of formal education on test performance and based on our results showing highly significant linear associations between education and HVLT-R scores, we feel that our approach provides substantially greater granularity to account for effects of education.

In examining the performance of the newly derived T-scores, we found unexpected site-related differences in the HVLT-R scores that we were unable to ascribe to known demographic, educational, social or language differences between the Arizona and California samples. As a result, we offer additional T-score equations including site as a predictor, and it is left to the clinician to decide how to best represent a specific patient when calculating scores. Information regarding demographic, educational, social and language background characteristics of the samples at each location might be helpful in this regard (Cherner, Marquine, et al., 2020). Great efforts were taken to train staff and assure the accuracy and consistency of data collected at both sites. Thus, these sites differences are not thought to be explained by differences in administration of the HVLT-R or collection of the data. Our finding showing no site effects on the BVMT-R are supportive of this interpretation. It is possible that other geographically relevant information not collected in the present study would shed light on this finding.

We also showed that published demographically adjusted norms for English-speaking NH Whites over-estimated impairment on both the BVMT-R and HVLT-R in our sample, while norms for NH Black/African Americans from the San Diego region tended to yield rates of impairment that were closer to the those of the new Spanish language norms. The fact that patterns of rates of impairment were consistent across verbal and visual learning and memory tests within the different sets of norms suggests that differences in rates of impairment are not completely driven by language. Importantly, there were significant effects of demographic variables on T-scores derived from norms developed for NH White and Black/African Americans in our sample, indicating that these effects are not adequately accounted for when norms for other groups are applied. This is important to consider particularly in light of findings showing generally comparable rates of overall impairment when T-scores based on norms for NH Blacks/African Americans were used. It would be misleading to assume that these norms are adequate for our Spanish speakers, as the significant effect of demographics on these T-scores indicates that such norms would not perform similarly across age and education levels, or by sex.

While the current Spanish language norms provide a much-needed tool to assess learning and memory in the largest group of U.S.-dwelling Spanish-speaking adults, our study had a number of important limitations. First, our sample size remains rather modest for norms generation. There were also unequal proportions of participants from Arizona and California, which is also home to large numbers of borderland Spanish speakers of Mexican origin. These limitations notwithstanding, we enrolled participants with a full range of formal education (0–20 years) and a range of age from early adulthood to middle age (19–60), and we used statistical methods that took advantage of the continuous nature of these demographic predictors, rather than forcing them into categorical variables. The current norms will not be applicable to pediatric or older populations. As part of this Special Issue, Morlett-Paredes and colleagues (2020) detail other norms that may be better suited to different populations, including older adults. Our colleagues Kamalyan et al., (2020) also applied the current NP-NUMBRS norms to HIV+ Spanish-speaking adults to examine their diagnostic accuracy with a patient group. Marquine, Rivera Mindt et al. (2020) discuss important future directions, including data collection on older adults and validation in other clinical samples.

As with other existing norms for many commonly used tests, cohort effects may also be a point of consideration for clinicians weighing the applicability of the NP-NUMBRS norms for a specific patient. Almost 22 years have passed since the beginning of sample collection, and changes in the demographics in the Mexico-US border might be expected, as immigration patterns have continued to evolve, including individuals immigrating to the U.S. and to Mexico from multiple Latin American countries. Similarly, demographics of the U.S. population in general have also evolved, and so many of our current norms for English-speakers may also be subject to cohort effects. In the NP-NUMBRS project, we examined test performance based on our two recruitment waves and did not find cohort effects (although these were only 6–11 years apart). Clinicians are encouraged to carefully consider the applicability of the current NP-NUMBRS norms for their respective patient and the other tools/norms currently available.

Additionally, degree of literacy and quality of education, important predictors of neuropsychological performance on both verbal and non-verbal tasks, may have not be captured by self-reported years of formal schooling. Research shows that measures of this type can account for differences in test performance between and within racial and ethnic groups (Lopez-Arango & Uriel-Mendoza, 1998; Manly et al., 1999, Manly et al., 2002; Manly, Touradji, Tang, & Stern, 2003; Matute et al., 2000; Ostrosky-Solis, Ardila, Rosselli, Lopez-Arango, & Uriel-Mendoza, 1998; Ostrosky-Solis, Ramirez, & Ardila, 2004; Rivera Mindt et al., 2008). In our study, we did not administer a measure of literacy, but we demonstrate that years of formal education was the strongest predictor of test performance; thus, adjusting for years of education provides a significant improvement in the interpretation of the results (i.e., less probability of over-estimating of impairment). Finally, other demographic factors inherently present in our sample, such as degree of acculturation and socioeconomic status, among others, may have also contributed to test performance. We did not examine the association of these factors with performance on out memory tests because of missing data and the potentially complex interrelationships among these factors, although ongoing work is investigating the impact of the complex set of educational, social and language factors on test performance in the entire NP-NUMBRS battery. Another article in this issue (Suárez, et al., 2020) details the effects of bilingualism (quantified as degree of English fluency) on test battery performances in the subset of our sample with available data. Given these considerations, we provide the additional details we have available about the educational and sociodemographic background of our sample (Table 2) to assist the clinician in determining the suitability of our norms for a specific patient. Additional culturally-relevant factors not collected in the present study may affect test performance on both verbal and non-verbal neuropsychological tests, including stereotype threat, and comfort with the testing situation (Ardila, 2002; Ardila, 2005; Ardila et al., 2010; Ardila, Rodriguez-Menendez, & Rosselli, 2002; Artiola I Fortuny & Mullaney, 1997; Brickman, Cabo, & Manly, 2006; Byrd, Jacobs, Hilton, Stern, & Manly, 2005; Cagigas & Manly, 2015; Coffey, Marmol, Schock, & Adams, 2005; Greenfield, 1997; Helms, 2005; Judd et al., 2009; Manly, 2006; Manly et al., 1999; Marmol, Schock, & Adams, 2005; Manly et al., 1998; Ostrosky-Solis, Ramirez, & Ardila, 2004; Perez-Arce, 1999; Ponton & Leon-Carrion, 2001; Rivera Mindt, Saez, & Byrd, 2010; Romero et al., 2009; Rosselli & Ardila, 2003; Steele, 1997; Thames et al., 2013; Wong, 2006). Future normative studies are highly encouraged to gather educational, social and language background information to assess their potential influence in neuropsychological performance. If any of these socio-cultural and linguistic factors predict performance above and beyond the expected effects from standard demographic information (i.e., age, sex, years of education), these factors might be modeled in the development of future norms.

In summary, our study presents regional norms for Spanish-speakers living in the southwestern United States and highlights the importance of accounting for demographic factors that affect neuropsychological test performance in both verbal and non-verbal learning and memory tasks. Future work ought to focus on identifying additional sources of variance in test performance that can explain differences in test with greater granularity than broad geographic and linguistic groupings.