Comparing Vocabulary Knowledge Conceptualizations Among Spanish–English Dual Language Learners in a New Destination State

Abstract

Purpose

Despite the unprecedented growth of Spanish–English dual language learners (DLLs) in new destination states—where DLLs have not been historically served—empirical understanding of their language and literacy skills is scant. Drawing on scientific understandings of bilingual language acquisition, we compare monolingually scored and bilingually scored vocabulary knowledge of Spanish–English DLLs in a new destination state.

Method

Participants included second- and fourth-grade Spanish–English DLLs (N = 60) in a large urban school district in a new destination state. Students were considered DLLs if their parents indicated that Spanish was spoken at home to some extent. We utilized monolingually scored (Spanish-only and English-only) and bilingually scored (conceptual and total) conceptualizations of vocabulary knowledge. Descriptive analysis and Bonferroni-adjusted multiple comparisons were used to compare vocabulary knowledge by assessment approach (i.e., monolingual or bilingual), grade (i.e., second and fourth), and English proficiency status (i.e., limited English proficient and English proficient).

Results

Findings revealed that (a) DLLs, across grade levels and English proficiency status, demonstrated higher bilingual vocabulary scores compared to monolingual scores and stronger receptive vocabulary performance than expressive vocabulary performance and (b) DLLs' response patterns varied depending on the bilingual assessment approach used, with DLLs in limited English–proficient and English-proficient groups evidencing similar response patterns.

Conclusions

Bilingual scoring of vocabulary knowledge provides a more holistic understanding of elementary-aged DLLs' language skills. Results represent an important step toward shifting school-based assessment practices to incorporate comprehensive and equitable ways to conceptualize and measure elementary-aged Spanish–English DLLs' vocabulary knowledge.

Students from homes in which a language other than English is used represent the fastest growing segment of the elementary-aged population in the United States (McFarland et al., 2019). The majority of these students come from Spanish-speaking, low-income homes and have immigrant parents (Park et al., 2018). This population of students is commonly known as English learners, and a subset of these are formally designated as limited English proficient (LEP). In this study, we thus use the broad term Spanish–English dual language learners (DLLs) to refer to students from households in which Spanish is used to some degree, whether they are English proficient (EP) or LEP.

Spanish–English DLLs' growing national presence and persistent concerns about their language and literacy achievement (Grimm et al., 2019; McFarland et al., 2019) demand increased attention to these skills. Specifically, it is often reported that DLLs tend to struggle with vocabulary knowledge to a greater extent than their non-DLL peers, leaving them particularly vulnerable to reading difficulties (Spencer & Wagner, 2017). Additionally, even when Spanish–English DLLs' vocabulary knowledge is assessed in both Spanish and English, many studies find DLLs to perform substantially below normative expectations in both languages (Gross et al., 2014; Hoff, 2018). The growing presence of DLLs in U.S. schools raises questions of educational equity, which calls for increased attention to how DLLs' vocabulary knowledge is assessed (Sugarman & Villegas, 2020). A foundational first step in this critical line of research is to ensure that the measures used with DLLs are based on scientific understandings of bilingual language acquisition.

We explore the following interrelated goals that remain understudied for elementary-aged Spanish–English DLLs, both EP and LEP, in a new destination state: (a) to describe and compare their achievement on monolingually scored and bilingually scored vocabulary assessments, receptively and expressively, and (b) to examine patterns of responses on the bilingually scored vocabulary assessments.

DLLs in New Destination States

DLLs—although not a new student population in the United States—are a rapidly growing student group in U.S. schools, with one in three elementary-aged students coming from homes in which the primary language is not English (Child Trends, 2019). In particular, new immigrant destination states in the South that have not historically served DLLs, such as Tennessee, are experiencing unprecedented growth of DLLs (Gándara & Mordechay, 2017; McFarland et al., 2017). Mirroring national trends, the majority of DLLs in the South are U.S.-born from Spanish-speaking, low-income households (McFarland et al., 2019; Park et al., 2018). For new destination states, the fast demographic changes in schools signal a call to action, as DLLs are entering schools that have historically conceptualized “minority” education in terms of race rather than linguistic diversity(Marrow, 2011) and have limited experience in supporting DLLs' unique linguistic needs (Takanishi & Le Menestrel, 2017). Empirical understandings of DLLs' language skills in new destination states are understandably scant; few studies to date have been conducted in school settings that are not traditional immigrant destinations (e.g., Mancilla-Martinez et al., 2019; Manis et al., 2004). Existing research does offer insight into DLLs' language and reading skills (e.g., Mancilla-Martinez & Lesaux, 2017; Pearson et al., 1993), but we must be cautious in assuming that previous findings hold true for DLLs in new destination states, where they are often much less likely to receive formal instructional support in Spanish, for example (Gándara & Mordechay, 2017). Consequently, there is a critical need to empirically understand the vocabulary skills—a key element to successful literacy outcomes—of DLLs in this new context, which is what the this study addresses.

Beyond Monolingual Yardsticks to Assess DLLs' Vocabulary

Given that Spanish–English DLLs hear and use two languages to varying degrees and have vocabulary knowledge distributed across their languages (Mancilla-Martinez et al., 2011; Pearson et al., 1995), the stubborn reliance on English-only assessments to measure DLLs' achievement is concerning. DLLs have more than one linguistic resource, and scholars have long cautioned against framing a bilingual as two monolinguals in one person or expecting a bilingual to demonstrate vocabulary knowledge in each language on par with monolinguals (Grosjean, 1989). Yet, monolingually normed assessments in general, and English-only assessments specifically, remain standard practice in classrooms (e.g., Arias & Friberg, 2017; Caesar & Kohler, 2007). Dependence on English-only assessments raises important educational equity concerns (Sugarman & Villegas, 2020), as these assessments only capture knowledge in one language (i.e., English). Considering that high-stakes educational decisions are made based on assessment outcomes—such as diagnosing language-related difficulties for special education services—it is imperative that the measures used are anchored on scientific understandings of bilingual language acquisition.

Previous research on bilingual scoring of DLLs' vocabulary knowledge have commonly examined DLLs' understanding and use of translation equivalents (e.g., Pearson et al., 1995; Umbel et al., 1992). Translation equivalents (e.g., perro and dog)—also referred to in the literature as doublets—reflect the same underlying conceptual content in each language (i.e., the word for a concept is known in both Spanish and English). In contrast, singlets refer to words a DLL understands or knows in either Spanish or English, but not in both (e.g., a child can identify perro but not dog, or vice versa). In examining bilingual vocabulary knowledge, researchers have typically examined Spanish–English DLLs' responses on bilingually scored vocabulary assessments in four ways: (a) correct in both Spanish and English (i.e., doublet), (b) correct only in Spanish (i.e., Spanish singlet), (c) correct only in English (i.e., English singlet), and (d) incorrect in both Spanish and English (e.g., Pearson et al., 1995; Peña et al., 2002; Umbel et al., 1992). This type of analysis allows for the examination of the distributed nature of DLLs' bilingual word knowledge at a much finer grain. Beyond simply documenting that DLLs' vocabulary knowledge may be distributed between their languages (e.g., Spanish and English), we can glean a greater,more nuanced understanding of how the knowledge is distributed (e.g., to what extent are singlets and doublets distributed in DLLs' knowledge base?). For example, in a study with 105 Spanish–English bilingual first graders, Umbel et al. (1992) found that, for a significant portion of the translation equivalent words, children reliably knew some words only in English but not in Spanish and did not show much overlap in their two languages (i.e., doublets). In contrast, in a study with 27 developing Spanish–English bilinguals from birth to 2 years of age, Pearson et al. (1995) did find children's knowledge of doublets—at an average of 30% of all words coded in both languages—as reported by their parents. Mancilla-Martinez et al. (2011) similarly reported between 21% and 32% of doublets among their sample of 70 Spanish–English bilingual toddlers from low-income homes. Furthermore, in a study with 44 Spanish–English bilingual preschoolers, Peña et al. (2002) reported doublet knowledge, but noted that children generated more unique singlet responses (i.e., Spanish-only or English-only) than doublet responses. The gap between the two response types was considerable, at 68.40% of the total items for single-language responses versus 28.20% for both Spanish and English responses.

Conceptual and Total Vocabulary Scoring

Despite efforts to account for both languages, the use of measures designed for and normed on monolingual speakers of either Spanish or English inevitably leads to partial, and often misguided, conclusions about DLLs' language skills (Bedore et al., 2005; Hammer et al., 2014). Given findings on the distributed nature of Spanish–English DLLs' vocabulary knowledge during the early childhood years (Oller & Pearson, 2002; Pearson et al., 1993), a growing body of research at the elementary level has examined the integrated use of Spanish and English in assessing DLLs' vocabulary knowledge. Findings support the use of an integrated, bilingual account of DLLs' vocabulary skills—mainly conceptual and total scoring—compared to monolingual vocabulary approaches (Bedore et al., 2005; Goodrich & Lonigan, 2018; Hwang et al., 2019).

Conceptual vocabulary simultaneously attends to DLLs' both languages, such that DLLs receive credit for known concepts, rather than for the language (i.e., label) of the correct response. Typically, scores are derived by summing all the words a student knows in Spanish and English and then subtracting translation equivalents (e.g., students do not receive double credit for knowing both perro and dog). Studies have often utilized conceptual vocabulary as an index of DLLs' language ability, with mounting evidence on its utility for understanding Spanish–English DLLs' language skills (Hwang et al., 2019; Peña & Halle, 2011). Additionally, studies have found conceptual scoring to reduce often-documented vocabulary gaps between monolingual and bilingual children (Bedore et al., 2005; Mancilla-Martinez & Vagh, 2013; Pearson et al., 1993) and to be useful for assessing DLLs with limited exposure to their second language upon school entry (Goodrich & Lonigan, 2018). However, previous measures of DLLs' conceptual vocabulary have been typically adapted from monolingual measures (e.g., Core et al., 2013; Mancilla-Martinez & Vagh, 2013), offering no bilingually normed information to evaluate DLLs' performance.

Total scoring also integrates both Spanish and English. However, while conceptual scoring gives a single credit for knowing both perro and dog (because the focus is on conceptual knowledge and not on the language-specific label), total scoring gives DLLs double credit if they know both perro and dog. In other words, total vocabulary scores are derived by summing all the words known in Spanish and English without subtracting translation equivalents. Total vocabulary encompasses both word forms and meaning, which has also been referred to as lexical vocabulary (Pearson et al., 1993). Core et al. (2013) argued that total vocabulary helps address the fact that conceptual vocabulary puts word form at the peripheral, as the focus of conceptual scoring is the concepts known. Indeed, word knowledge involves both meaning (i.e., semantic representations or concepts) and form (i.e., phonological representations; McGregor et al., 2002; Storkel, 2001), and total vocabulary thus taps both dimensions by prompting both languages for each item.

Building on scientific understandings of bilingual language acquisition, we adopt the revised hierarchical model (Kroll & Stewart, 1994), which postulates a centralized conceptual system between two languages (e.g., Spanish and English), where asymmetrical strength of the relations between first language, second language, and concepts vary as a function of an individual's fluency in the second language. Similarly, we draw on psycholinguistic evidence that suggest bilinguals' unified lexico-semantic reservoir that is comparable to monolinguals (e.g., Dijkstra & van Hueven, 2002; Kroll & Sunderman, 2003). These scientific insights of bilingual development support the use of bilingual scoring for DLLs' vocabulary knowledge. However, the extent to which conceptual or total vocabulary serves as a better proxy of DLLs' vocabulary knowledge remains inconclusive. Furthermore, research in this area has largely concentrated at the early childhood level. In fact, only recent studies have attended to elementary-aged Spanish–English DLLs, but this work has focused on conceptual and not total vocabulary (e.g., Hwang et al., 2019; Mancilla-Martinez et al., 2019).

Emerging Evidence of a Receptive–Expressive Vocabulary Gap

Studies on DLLs' vocabulary knowledge have found discrepancies between the receptive (i.e., comprehension) and expressive (i.e., production) domains, domestically (e.g., Gibson et al., 2012, 2018) and internationally (e.g., Keller et al., 2015; Yan & Nicoladis, 2009). The “receptive–expressive gap” refers to a pattern in which DLLs perform better receptively than expressively (e.g., Sheng et al., 2011; Yan & Nicoladis, 2009). Empirical evidence on the receptive–expressive gap is growing among DLLs in English-only instructional contexts (e.g., Gibson et al., 2012; Gross et al., 2014), but most work has focused on the early childhood level (e.g., Bedore et al., 2005; Mancilla-Martinez et al., 2018; Windsor & Kohnert, 2004). For example, a study with kindergarten Spanish–English DLLs found that the receptive–expressive vocabulary gap persisted regardless of levels of English exposure, with a larger gap in Spanish compared to English (Gibson et al., 2012). Likewise, a receptive–expressive gap in vocabulary knowledge has been found in Spanish and English among Spanish-speaking kindergarten DLLs (Hammer et al., 2008). More recently, Lonigan and Milburn (2017) presented a contradicting finding, suggesting no distinction between receptive and expressive language comprehension skills among students from preschool to fifth grade. However, their study was conducted with English monolinguals. Further research with elementary-aged Spanish–English DLLs in an English-only instructional context—a predominant learning environment for this population—is warranted. A better understanding of the extent to which there is or is not a receptive–expressive vocabulary gap among elementary-aged Spanish–English DLLs can be of significant practical value. This knowledge base can help guide the selection of vocabulary measures for assessment as testing time is at a premium (e.g., assessing only receptive vocabulary and not both receptive and expressive vocabulary if receptive vocabulary is found to be the area of most concern). Furthermore, and inextricably intertwined with assessment implications, this knowledge base can also help guide instructional efforts.

Notwithstanding the valuable contributions of previous studies, research has mostly focused on the early childhood level (e.g., Mancilla-Martinez et al., 2011; Pearson et al., 1995) and has often attended to the receptive vocabulary domain (e.g., Oller et al., 2007; Umbel et al., 1992). This leaves open questions about the overall utility of bilingually scored vocabulary assessments for the large and growing population of school-age DLLs, including questions about patterns of performance by language status (i.e., EP and LEP). Previous work suggests that bilingually scored vocabulary assessments paint a more comprehensive understanding of DLLs' vocabulary compared to reliance on single-language measures, but it is unclear whether those findings extend to older DLLs and how findings compare by DLLs' language status. Perhaps more importantly, studies to date have not attended to DLLs' patterns of responses on the two most common scoring methods, conceptual and total vocabulary, which would lend unique insight into the utility of these approaches to inform the selection of measures. Our study goal is unique in not only contributing to the existing body of research by examining both the receptive and expressive vocabulary domains of elementary-aged DLLs in a new destination state who are EP and LEP, but also analyzing within-child response patterns on the two most common bilingual scoring approaches.

Current Study

This study focuses on the vocabulary knowledge of elementary-aged Spanish–English DLLs in a new destination state by attending to the following: (a) monolingual and bilingual conceptualizations of vocabulary knowledge, (b) receptive and expressive vocabulary domains, (c) conceptual and total vocabulary scoring approaches, and (d) performance by grade and language status. We emphasize that previous studies and measures of elementary-aged DLLs' conceptual vocabulary have been adapted from monolingually normed measures, meaning measures that were not designed for and normed on DLLs. Our study addresses this measurement gap by using vocabulary measures designed for and normed on Spanish–English DLLs in the United States. Furthermore, to our knowledge, studies to date have not examined total vocabulary among elementary-aged DLLs. The following research questions guide this study:

1. How do Spanish–English DLLs' vocabulary performance compare on receptive and expressive monolingually scored (i.e., Spanish-only and English-only) and bilingually scored (i.e., conceptual and total) assessments? How do these outcomes compare by grade and English language proficiency status (i.e., EP vs. LEP)?

2. In the bilingually scored vocabulary assessments, what are Spanish–English DLLs' patterns of responses (i.e., Spanish-only, English-only, both languages, or neither)? How do these patterns of responses compare between conceptual and total scoring? How do these patterns compare by grade and English language proficiency status (i.e., EP vs. LEP)?

Method

Participants

As a part of a 3-year longitudinal study focused on DLLs' conceptual vocabulary development, the data for the current study come from the last wave of the larger study, when total vocabulary was assessed. Spanish–English DLLs (N = 60) for this study were recruited from three elementary schools in a large urban school district in Tennessee, a new destination state experiencing unprecedented growth of DLLs. Students were considered DLLs if their parents indicated that Spanish was spoken at home, to some extent. Of the 60 students (34 in second grade; 26 in fourth grade), 63% of the students (n = 38) were formally identified as LEP by their schools. Family demographic information was collected as a part of the larger research project (n = 54), and 90% of DLLs were born in the United States, with the rest born outside the United States (6% in Honduras, 2% in Mexico, and 2% in Cuba).

Measures

Monolingually Scored Vocabulary Assessments

English receptive vocabulary was measured with the Peabody Picture Vocabulary Test–Fourth Edition (Dunn & Dunn, 2007), which requires the child to point to a picture that corresponds to a target word read by an examiner. The publisher reports internal consistency reliabilities of .96–.97 for ages 7 and 8 years (an average age range for second graders) and .94–.96 for ages 9 and 10 years (an average age range for fourth graders). The parallel test in Spanish, the Test de Vocabulario en Imagenes Peabody (Dunn et al., 1986), was used to assess students' Spanish receptive vocabulary. The publisher reports the split-half reliability coefficients of .94 for ages 7 and 8 years and .91–.94 for ages 9 and 10 years. English expressive vocabulary was measured with the Picture Vocabulary subtest of the Woodcock-Muñoz Language Survey-III (Woodcock et al., 2017). In each session, an examiner points to a picture and asks the child to label the target item. Students also completed the Spanish version of the subtest, Vocabulario sobre Dibujos (Woodcock et al., 2017). Cronbach's alphas for the subtests ranged from .77 to .79 for ages 7–10 years.

Spanish–English Bilingually Scored Vocabulary Assessments

The Receptive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition (ROWPVT-4: SBE; Martin, 2013a) and the Expressive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition (EOWPVT-4: SBE; Martin, 2013b) were used to assess receptive and expressive vocabulary, respectively, and to generate conceptual and total vocabulary scores. Conceptual vocabulary was measured using the standardized protocol established by the publishers of the ROWPVT-4: SBE and EOWPVT-4: SBE. For the receptive measure, an examiner provides the target item in Spanish or English to the child, based on their language dominance as reported by parents. Likewise, for the expressive measure, the child was presented a picture and asked to label the item in Spanish or English (i.e., “What is this?” or “¿Qué es esto?”). Designed as bilingual measures of vocabulary knowledge, the ROWPVT-4: SBE and EOWPVT-4: SBE count a response as correct regardless of the language of a response (i.e., Spanish or English), as long as the concept is accurately identified. The publisher reports the median internal consistency reliability coefficient as .95 for both tests.

Total vocabulary also considered both languages, but scores were researcher generated as there are no standardized measures for total vocabulary. To generate total vocabulary scores, we used nonstandard administration of the ROWPVT-4: SBE and the EOWPVT-4: SBE that immediately followed the standardized conceptual vocabulary administration. In contrast to conceptual scoring that began at the age-recommended item per the standardized protocol, total scoring began with the first test item of the ROWPVT-4: SBE and EOWPVT-4: SBE. The examiner prompted the child in both languages from Item 1 of the test up to the test ceiling (i.e., where the conceptual testing ended, per the publishers' protocols), and in either Spanish or English for items that were not prompted in the other language during the standardized conceptual administration (i.e., students were not prompted in the other language if they provided the correct response at the first prompting). This allowed us to measure whether students knew the labels in both languages, in only one, or in neither language (i.e., they did not know the label in Spanish and did not know the label in English) for all tested items. In other words, the ROWPVT-4: SBE and EOWPVT-4: SBE were not administered twice. Instead, the total vocabulary administration entailed ensuring that items that were not already probed in both languages during the standardized conceptual vocabulary protocol were probed. Because total vocabulary scores were a researcher-generated sum of correct answers and do not have associated norms, standardized scores could not be generated. The use of raw scores for total vocabulary might limit generalizations of our findings. However, given the absence of empirical data about this bilingual scoring approach with elementary-aged DLLs, valuable insight might be gleaned on bilingual vocabulary knowledge among this growing population of learners across the nation.

Analysis Plan

To answer our research questions, we conducted descriptive analyses to obtain a detailed picture of vocabulary knowledge across both EP and LEP Spanish–English DLLs in a new destination state. To address the first research question comparing DLLs' vocabulary knowledge—and specifically those of EP and LEP DLLs—across the monolingual and bilingual measures, we generated descriptive statistics for the receptive and expressive domains for each of the four vocabulary conceptualizations (i.e., Spanish-only, English-only, conceptual, and total). We compared DLLs' achievement to national norms provided by assessment publishers, with the exception of the total vocabulary scores due to the absence of normative scores. We also conducted pairwise group comparisons to examine whether differences existed between DLLs' receptive and expressive vocabulary, as well as by grade levels and English language proficiency status. To reduce Type I error given the multiple comparisons, Bonferroni adjustments were made. We first compared receptive and expressive vocabulary scores within Spanish-only, English-only, conceptual, and total conceptualizations, given emerging findings that suggest DLLs' larger receptive knowledge compared to expressive knowledge (e.g., Gross et al., 2014; Mancilla-Martinez et al., 2020; Sheng et al., 2011). Next, we compared scores across languages (i.e., Spanish-only and English-only) to understand how bilingual vocabulary knowledge might be described for DLLs in a new destination state. Finally, we conducted comparisons by grade and LEP status.

To address the second question on students' patterns of responses on the bilingually scored measures, we first calculated percentages of each response type (i.e., Spanish-only, English-only, both, or neither) by student. Then, overall response patterns per response type were generated by averaging the values across all students. In doing so, we aimed to gain greater insight into the distribution of DLLs' bilingually scored vocabulary knowledge when conceptual compared to total scoring is used, which is unknown simply through reporting a conceptual and total score. This analysis was also conducted by grade and LEP status.

Results

Research Question 1: Monolingually and Bilingually Scored Vocabulary Knowledge

Table 1 displays students' average raw and standard scores (except for total vocabulary) across vocabulary conceptualizations for the full sample, by grade (i.e., second and fourth), and by English proficiency status (i.e., EP and LEP). For the full sample, DLLs performed below the standard mean of 100 on the Spanish and English monolingual vocabulary measures, with English receptive vocabulary scores falling within the low to low-average range. However, DLLs' monolingual English expressive vocabulary scores fell over 1.5 SDs below national norms. DLLs' Spanish receptive and expressive vocabulary evidenced a similar pattern to that of monolingual English receptive and expressive performance, such that the average receptive vocabulary scores were higher than the average expressive scores. The key difference was that DLLs' monolingual Spanish vocabulary fell about 1–3 SDs below national norms. In stark contrast, the receptive and expressive conceptually scored vocabulary standard scores indicated that DLLs performed in the average to above-average range. In addition, between the two grade levels, the fourth-grader DLLs showed greater knowledge for all vocabulary conceptualizations compared to second-grade DLLs, except for expressive Spanish-only vocabulary.

Table 1.

Sample means on monolingually scored and bilingually scored vocabulary by full sample, grade, and English proficiency status.

Variables	Total			Second grade			Fourth grade			EP			LEP
	Raw	Standard		Raw	Standard		Raw	Standard		Raw	Standard		Raw	Standard
	M (SD)		n	M (SD)		n	M (SD)		n	M (SD)		n	M (SD)		n
Spanish-only vocabulary
Receptive	58.22 (16.11)	83.98 (18.98)	58	53.91 (16.08)	86.13 (19.48)	32	63.54 (14.77)	81.35 (18.37)	26	61.50 (18.70)	86.86 (21.51)	22	56.22 (14.22)	82.22 (17.33)	36
Expressive	17.86 (5.06)	50 (14.59)	58	17.19 (5.26)	51.09 (15.99)	32	18.69 (4.76)	48.65 (12.83)	26	17.05 (5.74)	48.46 (11.37)	22	18.36 (4.61)	50.94 (16.33)	36
English-only vocabulary
Receptive	130.98 (19.96)	92.56 (12.43)	59	124.27 (14.91)	94.06 (9.18)	33	139.5 (22.48)	90.65 (15.62)	26	140.77 (21.79)	98.32 (14.03)	22	125.16 (16.49)	89.14 (10.09)	37
Expressive	28.97 (4.38)	73.72 (13.84)	58	27.47 (4.61)	74.03 (15.07)	32	30.81 (3.32)	73.34 (12.44)	26	30.90 (3.57)	80.14 (11.23)	22	27.78 (4.45)	69.81 (13.94)	36
Conceptual vocabulary
Receptive	113.81 (19.15)	118.22 (14.95)	59	108.18 (16.38)	120.12 (11.86)	33	120.96 (20.31)	115.81 (18.11)	26	122.73 (17.49)	124.73 (13.58)	22	108.51 (18.31)	114.35 (14.54)	37
Expressive	84.88 (13.82)	111.76 (13.95)	59	80.82 (10.95)	104.85 (10.97)	33	90.04 (15.51)	107.85 (16.40)	26	88.78 (14.81)	114.73 (12.84)	22	82.57 (12.85)	110.00 (14.45)	37
Total vocabulary
Receptive	192.27 (35.67)	—	59	179.21 (28.84)	—	33	208.85 (37.09)	—	26	205.09 (38.00)	—	22	184.65 (32.35)	—	37
Expressive	122.34 (24.35)	—	59	113.52 (17.10)	—	33	133.54 (27.72)	—	26	126.82 (25.00)	—	22	119.68 (23.90)	—	37

Note. Spanish-only and English-only receptive vocabulary were measured with the Peabody Picture Vocabulary Test–Fourth Edition and the Test de Vocabulario en Imagenes Peabody. Spanish-only and English-only expressive vocabulary were measured with the Picture Vocabulary subtest and Vocabulario sobre Dibujos subtest of the Woodcock-Muñoz Language Survey-III. Conceptual and total vocabulary were measured with the Receptive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition and the Expressive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition. The em dashes indicate that “standard scores” do not apply to the receptive and expressive types of Total vocabulary. EP = English proficient; LEP = limited English proficient.

Furthermore, EP and LEP groups generally showed a similar pattern to the full sample. Both groups performed below the standard mean of 100 for monolingually normed Spanish and English assessments, to varying degrees between the two groups. For monolingually normed receptive English vocabulary, both EP and LEP DLLs performed within 1 SD below national norms. Likewise, both groups' expressive English vocabulary fell below the national norms. More specifically, DLLs who were also LEP showed expressive English vocabulary scores that fell over 2 SDs below the national norms and more variability (M = 69.81, SD = 13.94) compared to their EP peers (M = 80.14, SD = 11.23). Spanish receptive and expressive vocabulary revealed a similar pattern for both groups, where the average scores fell within the low to low-average range. Similar to English receptive scores, both groups' scores were approximately within 1 SD below the national norm (M = 86.86, SD = 21.51 for EP; M = 82.22, SD = 17.33 for LEP). Furthermore, in line with the full sample average, DLLs' expressive Spanish vocabulary fell more than 3 SDs below the national average for both groups, with the LEP group showing greater variability (M = 50.94, SD = 16.33) compared to the EP group (M = 48.46, SD = 11.37). However, and again in line with the full sample results, on the bilingually scored receptive and expressive measures, both EP and LEP groups performed in the average to above-average range (see Table 1).

Next, we conducted Bonferroni-corrected multiple comparisons to examine potential differences among the various vocabulary conceptualizations. Results confirmed the significant differences across domains in each vocabulary conceptualization. DLLs—both by grade level and by LEP status—performed comparatively better receptively than expressively across all conceptualizations of vocabulary (see Figure 1). Additionally, likely given our participants' English-only educational context, the sample mean of English-only vocabulary was higher than that of Spanish-only vocabulary. More specifically, students demonstrated more English-only vocabulary—both receptively, t(56) = 27.39, p < .0001, and expressively, t(57) = 11.68, p < .0001—compared to Spanish vocabulary. Additionally, for bilingually scored vocabulary, DLLs' receptive and expressive total vocabulary scores were significantly higher than their receptive and expressive conceptual vocabulary, t(58) = 30.65, p < .0001 and t(58) = 18.24, p < .0001, respectively. Grade-level comparisons also revealed a similar pattern with the full sample, where both second- and fourth-grade DLLs showed significantly greater English-only vocabulary scores than Spanish-only vocabulary scores and greater total vocabulary scores than conceptual vocabulary scores (see Figure 1).

Figure 1.

Sample means in raw scores of monolingually scored and bilingually scored vocabulary conceptualizations by full sample, grade, and English proficiency status.

Note. Bonferroni adjustments were applied to all comparisons. *p < .05. **p < .01. ***p < .001. LEP = limited English proficient; Receptive Spanish = Test de Vocabulario en Imagenes Peabody; Expressive Spanish = Woodcock-Muñoz Language Survey-III Vocabulario sobre Dibujos; Receptive English = Peabody Picture Vocabulary Test–Fourth Edition; Expressive Vocabulary = Woodcock-Muñoz Language Survey-III Picture Vocabulary; Receptive Conceptual and Receptive Total = Receptive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition; Expressive Conceptual and Expressive Total = Expressive One-Word Picture Vocabulary Test–Fourth Edition: Spanish–Bilingual Edition.

Furthermore, we conducted Bonferroni-corrected multiple comparisons to compare the vocabulary knowledge achievement between the EP and LEP subgroups. Results revealed that, similar to the patterns found in the full sample, both EP and LEP DLLs performed comparatively better receptively than expressively across all conceptualizations of vocabulary (see Figure 1). Furthermore, as found with results with the full DLL sample, the English-only receptive sample means were significantly higher than those of Spanish-only vocabulary, for both EP, t(21) = 14.80, p < .01, and LEP, t(34) = 27.04, p < .01 groups. Likewise, English-only expressive scores were also higher than those of Spanish-only expressive scores for EP, t(21) = 10.17, p < .01, and LEP, t(35) = 7.75, p < .01, DLLs. Of note, the magnitude of differences between Spanish-only and English-only expressive scores was higher for the EP DLL group than those of the LEP group, which is to be expected given that the EP group, by definition, would have more English word knowledge. Additionally, for bilingually scored vocabulary, receptive total scores were higher than receptive conceptual scores regardless of LEP designation, t(21) = 16.45, p < .01, for EP; t(36) = 27.38, p < .01, for LEP. Comparison of bilingually scored expressive vocabulary scores also revealed the same pattern, where total expressive scores were higher than conceptual expressive scores, t(21) = 10.01, p < .01, for EP; t(36) = 15.39, p < .01, for LEP.

Although findings should be interpreted with caution given the relatively small sample size per group (see Table 1), results revealed that EP students outperformed their LEP peers in receptive and expressive English-only vocabulary, receptive conceptual vocabulary, and receptive total vocabulary. It is interesting to note that, perhaps expectedly, EP (i.e., English proficient) students showed greater knowledge in both domains of English vocabulary knowledge. However, we found no differences in Spanish-only vocabulary between the two groups, t(56) = 1.22, p = .23, which may be explained by the fact that both EP and LEP DLLs in our sample are not only educated in an English-only political context where instructional support for Spanish is unavailable, but are also still in the process of developing their skills in both languages (hence dual language learners), although to different extents. Of note, we did not find any significant differences in expressive conceptual and total vocabulary scores between EP and LEP groups, t(57) = 2.20, p = .10 and t(57) = 1.09, p = .30, respectively, but did find slightly higher receptive conceptual and total scores for EP students, t(57) = 2.93, p <.01 and t(57) = 2.20, p < .05, respectively.

Research Question 2: Response Patterns on Bilingually Scored Vocabulary Measures

To address our second research question, we examined DLLs' response patterns (i.e., Spanish-only, English-only, both Spanish and English, and neither) on bilingually scored vocabulary knowledge (see Figure 2). To note, for receptive vocabulary, the “both Spanish and English” response (i.e., the child provides correct answers in both languages simultaneously) only applies to receptive total vocabulary (i.e., the child is prompted in both Spanish and English to point to a picture). Based on the standardized protocol for receptive conceptual vocabulary, the child would not be prompted in the other language if they correctly point to the target item at the first prompting (i.e., in English or Spanish). However, this response type does apply to expressive vocabulary via both conceptual and total scoring.

Figure 2.

Dual language learners' response patterns on bilingually scored assessments by full sample, grade, and English proficiency status.

Note. EP = English proficient; LEP = Limited English proficient.

The number of correct responses for tested items varied widely, ranging from one to 58 responses per item. Given that students reached a ceiling at different test items, we calculated proportions of correct responses per response type up to each student's test ceiling and averaged the proportions per response type across the full sample to index overall patterns of response. First, based on the standardized procedure required for conceptual scoring, results showed that DLLs, on average, provided their receptive responses mostly in English (46%), followed by Spanish (44%), none in both languages (note that we would not expect responses in both languages in receptive conceptual testing, but it could have spontaneously occurred for expressive testing, as illustrated below for expressive conceptual vocabulary), and 10% in neither language. Expressively, DLLs provided 50% of their responses only in English, 24% only in Spanish, 2% in both languages, and 23% in neither language.

To examine the response patterns in a more detailed way, we then examined the response patterns by grade and LEP status (see Figure 2). Overall, the results by grade and by LEP status revealed similar response patterns. Using the same calculation rule as above, for receptive conceptual vocabulary, the second-grade DLLs provided 46% of their responses in Spanish, followed by English (43%), and neither language (11%). In contrast, expressively, they provided more than half of their responses in English (55%), followed by Spanish (24%), and neither language (23%). Fourth-grade DLLs showed a slightly different response rate for receptive conceptual vocabulary, in that they provided most of their responses only in English (48%), followed by Spanish (42%), and neither language (10%). For expressive conceptual vocabulary, their response pattern mirrored that of second-grade DLLs.

Moreover, the EP group provided the majority of their receptive responses only in English (45%), followed by 44% only in Spanish, and 10% in neither language. Likewise, their expressive responses were also predominantly in English (53%), then in Spanish (24%), none in both languages, and 23% in neither language. The LEP group also showed a similar pattern, where they provided 45% of their receptive responses only in English, 44% only in Spanish, and 11% in neither language. Expressively, the LEP group provided 50% of their responses only in English, 24% only in Spanish, 2% in both languages, and 24% in neither language. As shown in Figure 2, both EP and LEP groups demonstrated relatively similar response patterns or conceptually scored receptive and expressive vocabulary.

As expected, for total vocabulary—scored via our research team's long administration protocol (see Method)—DLLs provided a noticeably higher percentage of responses in both languages, although more of their monolingual responses were still English-only than Spanish-only. Receptively, DLLs identified 76% of the concepts in both languages, far outweighing the proportion of English-only (11%), Spanish-only (6%), and neither-language (6%) responses. Expressively, DLLs identified 47% of the concepts in both languages, compared to 27% only in English, 1% only in Spanish, and 16% in neither language.

Following the pattern of the full sample, second- and fourth-grade groups mostly responded in both Spanish and English to demonstrate their vocabulary knowledge, both receptively and expressively. Receptively, second- and fourth-grade DLLs provided 75% and 78% of their responses, respectively, in both languages, followed by noticeably smaller percentages for English-only (12% and 10%), Spanish-only (7% and 6%), and neither language (7% and 6%). Expressively, second and fourth graders evidenced a high proportion of both Spanish and English responses (47% for both grades), but more English-only knowledge compared to receptive total vocabulary (27% and 26%, respectively), followed by Spanish-only (7% and 6%) and neither -language (7% and 6%) responses.

Similarly, the EP group gave receptive responses predominantly in both languages (75%), on average, compared to 11% only in English, 8% in Spanish, and 6% in neither language. Their expressive total vocabulary responses also showed a similar pattern, where the majority of responses were in both languages (49%), compared to 25% only in English, 1% only in Spanish, and 16% in neither language. Likewise, the LEP group provided the majority of receptive total vocabulary responses in both languages (77%), far outweighing the percentage of English-only (12%), Spanish-only (5%), and neither-language (7%) responses. Their expressive responses also showed a similar pattern, where 45% of the test items were identified in both languages, compared to 28% only in English, 1% only in Spanish, and 16% in neither language.

Discussion

In this study, we compared different conceptualizations of vocabulary knowledge among elementary-aged Spanish–English DLLs in a new destination state. By doing so, we aimed to advance knowledge in an understudied area of research to help inform assessment practices in the service of best supporting DLLs' academic achievement. Three key findings emerged, all underscoring the utility of bilingually scored vocabulary assessments during the school-age years. First, results revealed within- and across-language differences among DLLs' vocabulary, where receptive knowledge was greater than expressive knowledge and bilingual scores were higher than monolingual scores. Second, DLLs' response patterns varied depending on the bilingual assessment approach used (i.e., conceptual and total). Finally, findings based on an examination of DLLs' performance by grade level and English proficiency status (i.e., EP and LEP) revealed many similarities to that of the full sample of DLLs, with some noteworthy exceptions centered on English-only vocabulary and on bilingually scored receptive vocabulary by LEP status. We discuss these interrelated findings and their implications in the sections that follow.

Across- and Within-Language Differences Among DLLs' Vocabulary Knowledge

Across-language comparisons revealed that students demonstrated significantly higher vocabulary knowledge in English as compared to Spanish, both receptively and expressively. Even though DLLs from Spanish-speaking homes are exposed to and often use Spanish at home or with their peers to some extent, this cross-linguistic gap might be related to the English-only educational context of our participants or the general language shift that DLLs often experience (Lutz, 2008; Tse, 2001). Especially in new destination states where English-only instruction is the norm and formal support in DLLs' home language is unavailable, opportunities to hear and use Spanish become more limited (Hammer et al., 2014), leading to increased possibility of home language loss (Castilla-Earls et al., 2019). Likewise, studies find Spanish–English DLLs to demonstrate lower receptive and expressive vocabulary in Spanish with increased formal schooling in English and English proficiency (Anderson, 2012; Sheng et al., 2013). Thus, students' limited Spanish-only expressive vocabulary—at more than 3 SDs below national norms (see Table 1)—might be at least partly attributable to DLLs' increased exposure to and use of English at school, leading to DLLs' shift to English as their primary and preferred language, as often observed among DLLs (Castilla-Earls et al., 2019). Of note, as expected, fourth graders generally outperformed second graders (see Figure 1). Furthermore, despite the slight difference between two grade levels for receptive Spanish vocabulary, we found no difference for expressive Spanish vocabulary, likely reflecting lack of opportunities to use Spanish under the English-only instruction context of our study.

Comparisons within the four vocabulary conceptualizations (i.e., Spanish-only, English-only, conceptual, total) revealed significant domain gaps. This receptive–expressive gap aligns with extant evidence at the early childhood level (e.g., Mancilla-Martinez et al., 2018), especially with studies on DLLs in English-only instructional contexts (e.g., Gross et al., 2014), like our sample. This might be attributable to the unique context of our sample, in which students are receiving instruction and learning concepts only in English; hence, we can expect DLLs' larger knowledge base for identifying and producing labels in English. In line with previous findings on the receptive–expressive gap, DLLs in our sample demonstrated noticeably higher vocabulary knowledge receptively than expressively, including both conceptual and total vocabulary (i.e., the bilingually scored vocabulary knowledge).

Importantly, we found no differences on expressive conceptual and expressive total scores by language status. This finding is expected given that bilingually scored measures should attenuate vocabulary score differences that are common when monolingual measures are used. Although we did find significant differences in receptive conceptual and receptive total vocabulary by language status, recent empirical findings find the expressive domain to be particularly important for understanding Spanish–English DLLs' language and literacy over receptive skills (e.g., Mancilla-Martinez et al., 2019; Ribot et al., 2018). In other words, bilingual scoring of DLLs' expressive vocabulary (i.e., the domain found to be noteworthy for DLLs' later reading outcomes) did not penalize students based on their language status while monolingual measures revealed that DLLs' vocabulary knowledge was below the national norms. Given emerging findings on the utility of expressive vocabulary as a predictor of reading comprehension, at a practical level, bilingually scored expressive vocabulary may be especially valuable in light of testing time constraints in schools.

DLLs' Response Patterns on Bilingually Scored Assessments

Our examination of DLLs' response patterns on the bilingually scored vocabulary measures provided unique insight into how we might best capture DLLs' full range of linguistic resources. Comparisons between conceptual and total vocabulary responses revealed that DLLs demonstrated more comprehensive word knowledge with total scoring. This is unsurprising, given that total scoring ensures DLLs have opportunities to respond in both languages for every item until their test ceiling, which is not the case with conceptual scoring. For responses known in both Spanish and English, total vocabulary revealed a larger percentage of known words both receptively and expressively (76% and 47%, respectively), compared to conceptual vocabulary (0% and 2%, respectively).

Comparing response patterns provided unique insight into the way in which different bilingually scored vocabulary measures represent DLLs' Spanish and English vocabulary knowledge. By prompting DLLs in both languages for each item, total scoring helped depict a more comprehensive profile of DLLs' bilingual vocabulary knowledge. It is also noteworthy that the proportion of unknown concepts was significantly smaller for both receptive and expressive total vocabulary compared to receptive and expressive conceptual vocabulary, highlighting total scoring as a potentially more thorough and accurate means of gauging what DLLs do and do not know.

Overall, our results are in line with emerging research evidence that suggests a greater proportion of DLLs' knowledge of singlets (i.e., English-only word knowledge in our case) than doublets (i.e., Spanish and English word knowledge), especially in the expressive domain (e.g., Peña et al., 2002) and for this age group. To our knowledge, the current study is the first to investigate this phenomenon with elementary-aged students from Spanish-speaking homes, which provides needed insight into developmental trends. This finding is understandable in light of our sample's distinct home and school environments. That is, while Spanish is used to some extent in DLLs' home environments, only English is used in their formal schooling environment. Follow-up item-level analyses by our research team will investigate the semantic groups of words DLLs know and how the item traits might be related to whether DLLs know each item in only Spanish, only English, or both languages. Our emerging findings thus contribute to the scant body of research on bilingual approaches to measuring vocabulary knowledge, extending growing research efforts for equitably assessing, and understanding the language skills of this historically underserved population of learners.

Additionally, bilingual-driven scores helped draw a more accurate understanding of DLLs' language skills, revealing their rich linguistic knowledge that is on par with, and even above, the national average. Hence, our findings can serve as a comprehensive indicator of DLLs' linguistically rich knowledge base and potentially prevent “dumbing down” of curriculum for DLLs, which is a source of instructional concern due to the risk of diminishing classroom content (Haggis, 2007). We argue that, by recognizing DLLs' linguistic resources in a language other than English, there is much possibility for drawing on this knowledge in the classroom in the service of supporting English academic growth.

Given findings that suggest a link between lower expectations and improper special education placement or low academic trajectories due to misunderstood language skills of DLLs (for a review, see Klingner et al., 2005), our results offer an important step toward better understanding elementary-aged DLLs' language skills and appropriately identifying their potential to meet their academic needs. We found that accounting for DLLs' both languages, a notion grounded in scientific understandings of bilingual development such as the revised hierarchical model (Kroll & Stewart, 1994) and asset-oriented accounts of DLLs' language skills, may help shift language stigma by presenting more comprehensive information of what DLLs bring to their learning and thus improve the quality of instruction they receive. This does not imply that English vocabulary support, especially in the expressive domain in light of our findings, should be ignored. On the contrary, our results reveal that many DLLs evidence English (namely, expressive) vocabulary skills that can be expected to significantly compromise their ability to comprehend texts in English. Indeed, it remains the case that English language skills are nonnegotiable for academic success in U.S. schools, where (high-stakes) assessments are in English. Thus, as we have underscored, it is imperative that DLLs' English (expressive) vocabulary be supported in a concerted way.

However, an especially promising implication of our results is that use of bilingual vocabulary measures have the potential to reframe misinterpretations that may inadvertently stem from reliance on English-only assessments. Reliance on English-only assessments may erroneously lead to assumptions that there are differential learning abilities at play between DLLs and their English monolingual peers. We argue that what may be at play are differential “opportunities to learn” (Aguirre-Muñoz & Amabisca, 2010; Gee, 2003) for DLLs compared to their English monolingual peers, such that it may be more a matter of “opportunity gaps” (Milner, 2012). For example, and as noted earlier, DLLs' home and schooling contexts tend to be rather distinct, with Spanish spoken to some extent at home and English used almost exclusively for instructional purposes. Without attention to bilingual vocabulary assessments, understanding which concepts DLLs may already know in the home language (i.e., Spanish) and subsequently capitalizing on their (home language) conceptual knowledge base to help them acquire the corresponding label in English would be severely limited.

We are also fully cognizant of the human resource limitations U.S. schools face. There is a well-documented shortage of bilingual teachers to work with DLLs (National Academies of Sciences, Engineering, and Medicine, 2017) in general and an even bigger shortage of bilingual speech-language pathologists (Thordardottir, 2020). The current context makes it challenging to ensure it is feasible for DLLs, even those from commonly spoken home languages (e.g., Spanish), to be assessed bilingually. In our view, however, the current context should not place a limit on the future. We maintain that advocating for the training and recruitment of a more linguistically diverse educator workforce that better aligns with the demographics of U.S. school-aged population is a necessary step, and a step that the results of our study supports.

Limitations and Future Directions

Our study investigated diverse conceptualizations of DLLs' vocabulary knowledge—a key contributor of successful literacy outcomes (August et al., 2005; Lesaux et al., 2010)—and specifically what they reveal about vocabulary knowledge of DLLs in a new destination state in the U.S. South. However, findings must be interpreted within their limitations, such as that our study focused on Spanish–English DLLs who received English-only instruction in Tennessee. Furthermore, although we found significant differences by grade and LEP status, given that the subsample sizes were small and different (second grade = 34, fourth grade = 26, EP = 38, LEP = 22), these significant differences should be interpreted with caution. Nonetheless, our findings offer preliminary insight into trends to be further investigated with a larger sample with balanced subsamples by grade and LEP status, to draw a clearer landscape and wider generalizations of DLLs' vocabulary knowledge. We also did not account for patterns of language use and exposure at home and school. Given that contextual factors influence DLLs' language outcomes (Takanishi & Le Menestrel, 2017), future research should investigate how DLLs' language and literacy achievement might be linked to these factors. Nonetheless, our examination of the diverse vocabulary conceptualizations offers ways to better assess and understand Spanish–English DLLs' vocabulary skills, particularly in a new destination state where educators and schools may be less familiar with meeting DLLs' unique linguistic needs (Durgunoğlu & Hughes, 2010; Harper & de Jong, 2009).

Furthermore, although our findings suggest the benefit of bilingual scoring for DLLs, there is a lack of consensus on a systematic method for generating and interpreting bilingually driven scores (Anaya et al., 2018; Thordardottir et al., 2006). As a start, we recommend the use of bilingual vocabulary measures from which normative scores can be derived (i.e., conceptual vocabulary) given that standardized total vocabulary assessments remain unavailable and are more time consuming. Related to total vocabulary, our use of raw scores for total vocabulary might limit generalizations of our findings. In addition, conceptual and total vocabulary comparisons should be interpreted with caution, as the administration protocols differed given the limited availability of vocabulary measures geared toward Spanish–English DLLs. Relatedly, we acknowledge that our findings are limited given the reliance on a single bilingually normed assessment (i.e., ROWPVT-4: SBE, Martin, 2013a; and EOWPVT-4: SBE, Martin, 2013b). However, few bilingually normed Spanish–English language assessments exist (Peña & Halle, 2011). Exceptions include the Bilingual English–Spanish Assessment (BESA; Peña et al., 2018), designed for children from ages 4 to 6 (thus younger than our sample), and BESA–Middle Elementary, an extended version of BESA for older students between ages 7 and 9;11 (years;months; Peña et al., 2018). Although the testing age range for BESA–Middle Elementary does include lower elementary grades, 30% (n = 18) of our sample were older than the recommended age limit of 9;11, and the measure has not yet been published for clinical use. Therefore, to our knowledge, the EOWPVT-4: SBE and ROWPVT-4: SBE are the only publicly available measures normed on Spanish–English DLLs in the United States across the life span and thus appropriate to use with our elementary-aged sample. In summary, our findings show that drawing on DLLs' Spanish and English linguistic resources is a promising assessment approach that reveals a rich body of linguistic knowledge that could be leveraged in the classroom, to help reshape deficit-based and misinformed orientations toward DLLs' language skills in new destination states.

Funding Statement

This research was supported by a grant from the Eunice Kennedy Shriver National Institute of Child Health and Human Development awarded to Vanderbilt University (P20 HD075443).