Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Jun 13.
Published in final edited form as: Learn Disabil Res Pract. 2010 Feb 7;25(1):48–58. doi: 10.1111/j.1540-5826.2009.00300.x

Using Nonword Repetition Tasks for the Identification of Language Impairment in Spanish-English Speaking Children: Does the Language of Assessment Matter?

Vera F Gutiérrez-Clellen 1, Gabriela Simon-Cereijido 2
PMCID: PMC3374488  NIHMSID: NIHMS377825  PMID: 22707854

Abstract

The purpose of this study was twofold: (a) to evaluate the clinical utility of a verbal working memory measure, specifically, a nonword repetition task, with a sample of Spanish-English bilingual children and (b) to determine the extent to which individual differences in relative language skills and language use had an effect on the clinical differentiation of these children by the measures. A total of 144 Latino children (95 children with typical language development and 49 children with language impairment) were tested using nonword lists developed for each language. The results show that the clinical accuracy of nonword repetition tasks varies depending on the language(s) tested. Test performance appeared related to individual differences in language use and exposure. The findings do not support a monolingual approach to the assessment of bilingual children with nonword repetition tasks, even if children appear fluent speakers in the language of testing. Nonword repetition may assist in the screening of Latino children if used bilingually and in combination with other clinical measures.

Over the past two decades, the United States has witnessed a significant growth in the number of children from Spanish-speaking backgrounds and for whom English is their second language (U.S. Department of Commerce, 2001). Latino children are now the largest minority group in the United States and among English language learners, these children are the most likely to experience academic delays compared to other minority students of similar socioeconomic backgrounds (National Center for Education Statistics, 2004). Furthermore, many of these children are under-represented or overrepresented in special education (Zehler, Fleischman, Hopstock, Stephenson, Pendzick, & Sapru, 2003) and as a result, they do not receive appropriate services. Although the patterns of misidentification are prevalent throughout the school years, it is during the early grades that Latino children are most likely to be misclassified. A recent analysis of the Early Childhood Longitudinal Study-Kindergarten Cohort indicates that these children are underrepresented in special education in kindergarten and first grade and overrepresented in third grade (Samson & Lesaux, 2009).

One of the reasons Latino children are not identified appropriately is the lack of valid assessment criteria used to differentiate low reading abilities related to limited English proficiency from limitations related to a learning disability. Available tests are normed on monolingual English speakers. Yet, children who are in the process of learning two languages are likely to exhibit performance differences across a variety of achievement and cognitive tests compared to their monolingual peers. This is particularly evident when the assessment measures rely on vocabulary (such as in tests of reading or listening comprehension) or on conceptual knowledge. Performance on these tasks will vary depending on the child’s relative language skills and use of the language(s). For example, a study of the Structured Photographic Expressive Language Test: Third Edition (SPELT-3: Dawson, Stout, & Eyer, 2003) comparing Latino bilingual children and monolingual English-speaking children found a moderate effect size based on the child’s bilingual status (Perona, Plante, & Vance, 2005). The bilingual children had lower language scores compared to the monolinguals. The same patterns have been described when Spanish tests are used. For example, Restrepo and Silverman (2001) reported that bilingual Spanish-English children scored 1 standard deviation below the mean for both receptive and expressive language scores on the Spanish Preschool Language Scale (PLS-3: Zimmerman, Steiner, & Pond, 1993). The percentage of children with language impairment classified as impaired (i.e., sensitivity) by the Spanish Structured Photographic Expressive Language Test (SPELT II: Werner & Kresheck, 1989) is low. Only 65% of Spanish-speaking children with language impairment were correctly identified by this measure (Restrepo, 1998). Similar classification rates are reported for the Spanish Clinical Evaluation of Language Functions test (CELF Preschool-2 Spanish: Wiig, Secord, & Semel, 1992). Plante and Vance (1994) suggested that, for a diagnostic measure to be fair, its sensitivity and specificity should be at or above 80%.

In a large study conducted in Miami, bilingual children performed below monolinguals across a series of standardized oral and written language measures in both English and Spanish, after controlling for differences in socioeconomic status (Oller & Eilers, 2002). One of the reasons for the lower performance of bilingual children is the fact that children’s learning may be distributed across the two languages. For example, a child may know vocabulary items in one language and not in the other. In fact, when the Peabody Picture Vocabulary Test (PPVT: Dunn & Dunn, 1981) and the Test de Vocabulario en Imágenes Peabody (TVIP: Dunn, Padilla, Lugo, & Dunn, 1986) were administered to bilingual children, a large proportion of vocabulary items shared by both test versions did not overlap within children’s responses, lowering their total scores (Umbel, Pearson, Fernandez, & Oller, 1992). There are great individual differences within and between the two languages of bilingual children and current assessment instruments are not designed to differentiate differences from true disabilities in these children.

A second complicating issue affecting the early identification of bilingual children is the need for assessment measures that can help diagnose children at risk for learning or reading difficulties without penalizing for individual differences in literacy experience. A measure of clinical predictive validity is a nonword repetition task. Dollaghan and Campbell (1998) compared the performance of African American and Caucasian children on several standardized tests commonly used in clinical assessments and on the repetition of nonwords of increasing length as a measure of verbal working memory. Because the repetition of nonwords does not require vocabulary knowledge, it was predicted that nonword repetition task performance would not be vulnerable to differences in literacy experience. The African American children had lower scores on the tests than their Caucasian peers but there were no differences between the groups for the nonword repetition task. The clinical potential of the nonword repetition task in assessing children from diverse backgrounds was also supported by a large study of 581 second grade English speakers (Ellis Weismer, Tomblin, Zhang, Buckwalter, Chynoweth, & Jones, 2000). In this study, the language tests and nonword repetition scores of 88 African American children were compared to those of 493 Caucasian children. Although the tests yielded lower scores for the African American children, the groups had similar nonword repetition scores. In addition, the nonword repetition scores for the two groups had similar distributions.

The available research indicates that verbal working memory measures such as the nonword repetition task could reduce bias in assessment while helping identify children with disabilities. In English, the nonword repetition task has been proposed as a clinical marker for inherited language impairment (Bishop, North, & Donlan, 1996) with promising diagnostic differentiation (e.g., Conti-Ramsden, Botting, & Faragher, 2001; Conti-Ramsden & Hesketh, 2003; Gray, 2003). Researchers in the field have found that verbal working memory is affected in groups of children with learning disabilities (for a review, Alloway & Archibald, 2008), attention deficits (Alloway, Rajendran, & Archibald, 2009), reading disabilities (e.g., Kibby, Marks, Morgan, & Long, 2009; Swanson & Beebe-Frankenberger, 2004), and language impairments (Ellis Weismer et al., 2000). In verbal working memory tasks, the focus is on assessing the child’s ability to simultaneously store and process information for a brief period of time. If children have verbal working memory deficits, learning may be disrupted. In fact, there is documented evidence for the association between working memory deficits and learning difficulties in the areas of reading and math (e.g., Bull & Scerif, 2001; Gathercole, Alloway, Willis, & Adams, 2006; Gersten, Jordan, & Flojo, 2005; Swanson & BeebeNonword Frankenberger, 2004). In addition, the nonword repetition task may be particularly useful for the assessment of second language learners. Performance on this task has been linked to the ability to learn new words (Gathercole, 2006) and it has been used to predict children’s learning of English as a second language (e.g., Service, 1992).

There is evidence that a nonword repetition task has clinical significance in other languages. For example, both children with language impairment and children with a familial risk of dyslexia performed more poorly than control children on this measure in Dutch (de Bree, Rispens, & Gerrits, 2007). This research suggested that nonword repetition performance was a marker of Dutch language impairment and could help predict reading deficits in that language. Research examining the Spanish nonword repetition task (SNWRT) performance of Spanish-speaking children is very limited. Available studies are based on Spanish speakers sampled in Spain and, to our knowledge, only two SNWRT studies are based on bilingual children in the United States. Aguado Alonso and his colleagues compared the nonword repetition performance of Spanish children (ages 5;0 to 7;0 years) with and without language impairment using a list of nonwords that included two, three, four, and five syllables constructed to resemble Spanish words (Aguado Alonso, Cuetos Vega, Domezain, & Pascual, 2006). Their research indicated significant differences between the two groups, but the clinical utility of the SNWRT was not investigated. A more recent study with a group of school age children (ages 8;3 to 10;11 years) also sampled in Spain suggested that the measure had high diagnostic differentiation using a cutoff score of 50% correct production (Girbau & Schwartz, 2007). However, these classification results were based on a small sample of children (11 children with typical language development, 11 children with language impairment). In addition, the children were exposed to Catalan. The children’s relative proficiency in the two languages was not reported and as a result, it is not known whether differences in language skills were controlled. Performance on the SNWRT is expected to vary depending on the child’s level of development in the language.

There is a critical need for examining the clinical application of this measure with Spanish-speaking children who are English learners or bilingual. A recent study using the English nonword repetition task (ENWRT) developed by Dollaghan and Campbell (1998) with bilingual children found that their scores were lower than their monolingual English peers. Kohnert, Windsor, and Yim (2006)) examined the performance of 100 children (ages 7;10 to 13;11 years) distributed in three groups: monolingual English-speaking with specific language impairment, monolingual English-speaking with typical language development, and Spanish-English bilingual with typical language development. The classification accuracy of the measure was based on a cutoff score of less than or equal to 76% correct production. For the monolingual groups, scores equal to or less than 76% correct production were 10.7 times likely to come from children with language impairment (i.e., an intermediate high likelihood ratio). However, when the bilingual children with typical language development were added to the comparison, the analysis resulted in a lower degree of sensitivity. A lower cutoff score had to be used to maximize differentiation (i.e., a score equal to or less than 72% correct production). Using this score, children with language impairment were only 5.07 times likely to be classified correctly (still an intermediate high likelihood ratio, however). The scores of the bilingual children were found to be higher than those of the children with language impairment but lower than their English-only peers, affecting the diagnostic accuracy of the measure.

Although the classification rates of the bilingual children were not reported, Kohnert and colleagues (2006) noted that performance on the ENWRT does not provide sufficient diagnostic accuracy for separating bilingual children with typical language development from monolingual children with language impairment. Since the study did not examine the nonword repetition performance of the bilingual children in their first language (i.e., Spanish), it is not known if the results might be related to individual differences in language dominance and second language development. Previous research using measures of verbal working memory developed for the two languages with bilingual children, found no differences between the bilinguals and English-only or Spanish-only children (Gutiérrez-Clellen, Calderón, & Ellis Weismer, 2004). This research emphasized the need to examine performance across the two languages.

In order to evaluate the nonword repetition task with bilingual learners, it is critical to address potential differences in language dominance or language skills. A recent study compared the SNWRT performance of 11 children with specific language impairment and 11 age-matched children (ages 7:6 to 10; 10 years) who were described as living in the U.S. and using Spanish at home (Girbau & Schwartz, 2008). The measure included one, two, three, four, and five-syllable nonwords. The group comparisons on their SNWRT performance indicated significant differences and adequate diagnostic differentiation. However, no external measures of language competence or language use were reported and the tests used to identify the affected children were based on monolingual norms. Language tests normed on monolingual children do not accurately identify a language disorder in bilingual children.

The purpose of this study was to evaluate the clinical utility of a verbal working memory measure such as the nonword repetition task with a sample of Spanish-English speaking children with and without language impairment. Current research shows that children with language impairment have a high risk for persistent reading difficulties (Puranik, Petscher, Otaiba, Catts, & Lonigan, 2008) and the studies reported earlier suggest that the measure may help predict children’s learning difficulties. The first goal of the study was to replicate previous findings based on monolingual speakers with language impairment and determine if performance on the nonword repetition task can differentiate typical bilingual learners from bilingual children with language impairment. The second goal was to assess the extent to which individual differences in language skills and use had an effect on the clinical differentiation of these children. As these children acquire the two languages in different contexts and with different levels of English and Spanish exposure, we expected that the clinical accuracy of the measure would vary depending on the language(s) tested. As a secondary analysis, we compared the diagnostic differentiation of the measure when either one or both languages were considered as well as when the classification rates were based on the child’s dominant language.

Method

Participants

There were ninety-five children with typical language development (TLD) and 49 children with language impairment (LI). The mean age of the full sample was 72 months or 6;0 years (SD = 9 months, range: 47 to 94 months). Age was not significantly different across the language ability groups (TLD mean age = 73 months, SD = 9 months; LI mean age = 71 months, SD = 11 months), t(142) = .843, p = .401. There were 50 girls and 45 boys in the typical group and 16 girls and 33 boys in the affected group, χ2(1, N = 144) = 5.197, p = .023, reflecting the higher incidence of language impairment in boys than in girls (Leonard, 1998).

The children were sampled from schools in districts in Southern California. At the time of the study children were attending pre-kindergarten, kindergarten, and elementary (1st and 2nd grade) classrooms (see Table 1). There was no significant difference in the number of children attending different grades across groups, χ2(2, N = 144) = 2.014, p = .365. The majority of the children attended bilingual classrooms with Spanish-speaking aides. Table 1 describes the percentage of the children’s families according to the level of education in the home, eligibility to free/reduced or regular school lunch as a metric for income level, and dialect spoken at home. The education level of the home was based on maternal education, since there were no differences between the number of years of education of the parents and those of other adults living with the child. Eligibility to free/reduced or regular school lunch was used as an indirect estimate of family income because each school determined lunch program qualification status based on family income and the number of occupants in the household. More than half (55.5 %) of the children were eligible for reduced or free lunch and 21% of the children’s mothers had fewer than 12 years of schooling. The education level in the home was comparable across the two ability groups, χ2(2, N = 129) = 1.225, p = .542, as was the families’ eligibility for school lunch program, χ2(1, N = 99) = 1.213, p = .271. The majority of the children (98%) were of Mexican-American descent.

Table 1.

Percentage of Families of the Children With Typical Language Development (TLD) And Language Impairment (LI) In Each Category Of Educational Level, Eligibility To Lunch Program, and Dialect Spoken.

Characteristic TDL
n = 95		 LI
n = 49
Education level in the home
    Primary and Some Secondary 18% 26%
    High School Grad 41% 39%
    Some College and College Grad 41% 35%
Eligibility to Lunch Program
    Free or Reduced 48% 60%
    Regular 52% 40%
Regional Dialect
    Mexican Spanish 97% 100%
    American English 100% 100%
Grade
    Pre-kindergarten 19% 28%
    Kindergarten 33% 33%
    Elementary (1st and 2nd grade) 48% 39%
Open in a new tab

Note: 12 TDL & 3 LI have incomplete questionnaires

Description of the measures used to identify children with language impairment

Given the paucity of developmentally, culturally, and linguistically valid standardized tests to identify bilingual children with language impairment, it was deemed inappropriate to use available published tests to assign children to ability groups. Therefore, the children were identified using procedures and measures previously validated and recommended with this population (Gutiérrez-Clellen, Restrepo, & Simon-Cereijido, 2006; Gutiérrez-Clellen, Restrepo, Bedore, Peña, & Anderson, 2000; Gutiérrez-Clellen & Simon-Cereijido, 2007; Restrepo, 1998). The assessment protocol included evidence of clinical concern, evidence of ungrammatical spontaneous language, and below performance on the English-Morphosyntax Test and the Spanish-Morphosyntax Test of the Bilingual English-Spanish Assessment (BESA) (Peña, Gutiérrez-Clellen, Iglesias, Goldstein, & Bedore, n.d.). Restrepo (1998) found that a parent interview in conjunction with language sample analyses discriminated with 90% accuracy between children with language disorders and children with typical development. That research indicated that the proportion of ungrammatical utterances in the child’s spontaneous language samples in conjunction with parent concern regarding the child’s language problems were the most accurate diagnostic indicators.

The children with language impairment had to demonstrate above 20% ungrammatical utterances (Gutiérrez-Clellen et al., 2000; Restrepo, 1998) in both of their languages based on narrative samples obtained using wordless picture books: “Frog on His Own” (Mayer, 1973) and “Frog Goes to Dinner” (Mayer, 1974) for Spanish; “Frog, Where Are You?” (Mayer, 1969) and “One Frog Too Many” (Mayer, 1975) for English. The languages were tested in two separate administrations of the narrative procedure on different days. Narratives were transcribed word by word, and coded for the presence of grammatical errors in each language using the Systematic Analysis of Language Transcripts computer program (Miller & Chapman, 2000) by a bilingual research assistant. Spanish–English mixed utterances and utterances with unintelligible words were excluded from the analysis. Above 90% transcription reliability and grammatical code reliability were obtained with two bilingual transcribers who independently transcribed and coded 20% of the audiotapes in each language. Any discrepancies were resolved by consensus with a third bilingual transcriber. The narrative samples from the children with typical language development had higher grammatical accuracy both in Spanish, t(88) = −3.451, p = .001, d = .79, and in English, t(109) = −6.829, p < .001, d = 1.37, than the narrative samples from children with language impairment. Table 2 lists the children’s percentages of ungrammatical utterances in Spanish and English across the two ability groups.

Table 2.

Mean and standard deviation (SD) of measures of Spanish and English exposure, use, proficiency, and ungrammaticality for all participants by language dominance and language ability

English dominant Spanish dominant Fluent in both
TLD LI TLD LI TLD LI
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Percentage of Spanish exposure at home 37.20 18.35 41.26 22.94 77.34 19.93 81.86 18.51 73.78 22.64 70.36 23.85
Percentage of English exposure at home 62.80 18.35 58.74 22.94 22.66 19.93 18.14 18.51 26.22 22.64 29.64 23.85
Parents' rating of Spanish use 2.76 1.18 2.28 1.44 3.97 0.16 3.95 0.22 3.86 0.38 4.00 0.00
Parents' rating of English use 3.96 0.21 3.78 0.52 2.29 1.18 2.76 0.94 3.14 1.21 3.33 0.58
Percentage of Spanish exposure at school 33.97 35.91 30.79 31.10 56.93 29.71 48.00 29.13 54.29 24.40 60.00 34.64
Percentage of English exposure at school 59.97 38.11 69.21 31.10 43.07 29.71 52.00 29.13 45.71 24.40 40.00 34.64
Teachers’ rating of Spanish use 1.61 1.45 1.24 1.30 3.22 1.11 2.35 1.40 3.19 0.79 3.56 0.77
Teachers’ rating of English use 3.12 1.25 3.35 0.97 2.09 1.29 2.26 1.20 2.90 0.71 1.78 1.07
Percentage of Spanish ungrammatical utterances 26.93 17.20 28.48 17.93 9.96 4.61 26.64 17.06 9.47 5.76 38.64 30.49
Percentage of English ungrammatical utterances 16.11 9.97 38.90 14.24 37.76 17.99 48.15 10.18 21.08 11.78 64.31 9.31
Open in a new tab

Children’s ability status was also established using the English-Morphosyntax Test and the Spanish-Morphosyntax Test of the Bilingual English-Spanish Assessment (BESA) specifically developed for this population (Peña, Gutiérrez-Clellen, Iglesias, Goldstein, & Bedore, n.d.). The BESA Morphosyntax scores represent the percentage of correct responses. Recent research indicated that children whose scores fell below 60% were accurately identified as children with language impairment (Gutiérrez-Clellen et al., 2006; Gutiérrez-Clellen & Simon-Cereijido, 2007). These cut-off scores were used to determine the ability status of the participants.

The children with typical language development met criteria based on the same measures as the children with language impairment. These children had higher BESA Morphosyntax scores (Mean = 86%, SD = 10%) than the children with language impairment (Mean = 53%, SD = 20%), t(61.241) = 11.041, p < .001, d = 2.32. None of the children had hearing impairments, mental retardation, emotional disturbance, motor difficulties, or neurological deficits, according to parent report and school records.

None of the children with language impairment had cognitive delays. Clinically, these children would be considered to have specific language impairment. However, because no independent measure of nonverbal IQ was available for many of the study participants, children with atypical language development could not be formally determined to have specific language impairment. For this reason, in the current study the term children with language impairment will be used to characterize this group.

Procedures for establishing English and Spanish exposure, use, and dominance

Current language proficiency tests may not be useful to establish language dominance in bilingual children because they may overestimate a child’s English skills. For example, the vocabulary subtests of the Woodcock Language Proficiency Battery – Revised (Woodcock, 1991) and the Woodcock Language Proficiency Battery-Revised: Spanish Form (Woodcock & Muñoz-Sandoval, 1995) evaluate the same items in English and Spanish. However, direct comparisons to establish relative dominance in the languages using this measure cannot be made because children may only know items in one language, and this language may not be their most dominant language (Gutiérrez-Clellen & Simon-Cereijido, submitted).

In order to verify bilingual status and establish the child’s level of language use, exposure, and dominance we used parent and teacher questionnaires (Gutiérrez-Clellen & Kreiter, 2003). In previous research children’s language performance in Spanish and English was found to significantly correlate with the estimates of the use of the languages indicated by the parents and teachers (e.g., parents’ rating of Spanish use and Spanish performance: r = .75, p < .0001; teachers’ rating of English use and English performance: r = .61, p < .0001). Multiple regression analyses revealed that exposure to the languages at home was only significant for Spanish, accounting for 26% of variance of the children’s Spanish language skills. The percentages of English exposure at home and at school were not significantly related to the children’s English language skills (Gutiérrez-Clellen & Kreiter, 2003).

Based on this research, parents were asked to rate the spoken English and Spanish use of their child using a 5-point rating scale for each measure (0 representing no use and 4 representing use all the time). They also reported the number of hours the child interacted with each member of the household and the language spoken during those interactions. The children's teachers were also given a questionnaire to rate the participants’ use of English and Spanish using the same 5-point scale. In addition, they provided an estimate of the percentage of time that the child was exposed to English and Spanish as a measure of language exposure at school. Teachers were experienced bilingual Spanish-English speakers and were deemed to be reliable informants. Table 2 describes the English and Spanish exposure and use of the children based on the questionnaire data.

Children were determined to be English dominant if they had (a) a minimum parent and/or teacher rating of 3 for English use; and (b) some spoken Spanish. Children were identified as Spanish dominant if they had (a) a minimum parent and/or teacher rating of 3 for Spanish use and (b) some spoken English. Using these criteria, parents and teachers agreed on the bilingual status of the participants. However, some discrepancies between parents and teachers were observed for the classification of some children. Some children had high ratings of use by the parent or teacher for one of the languages but their scores on the diagnostic assessments demonstrated limited skills. Therefore, questionable cases were verified with the scores obtained on the BESA morphosyntax subtests in each language.

For such cases we compared the two scores in the two languages and used a greater than 10% score difference to establish a dominant language. This cutoff difference was based on previous research with these tests with a national sample of children. There were 10 children (3 children with language impairment and 7 children with typical language development) whose questionnaires did not favor a specific language because their morphosyntax scores were less than 10% points apart. These children were considered to have a balanced dominance and were excluded from the comparisons related to dominance. There were 63 Spanish dominant children (22 with language impairment and 41 with typical language development) and 71 English dominant children (24 with language impairment and 47 with typical language development).

Procedures

English nonword repetition task (ENWRT)

For the English nonword repetition task (ENWRT) we used Dollaghan and Campbell’s (1998) list of nonwords. This task consists of a set of 16 nonwords ranging from one to four syllables in length. These word lengths are representative of the English language. The list was carefully designed to include sounds that are learned early and syllables that occur in low frequency in the language. None of the syllables correspond to English lexical items. For a detailed description of the development and design criteria of this task, see Dollaghan and Campbell (1998). A native female speaker of English recorded the list of nonwords that was administered to the children via headphones.

Spanish nonword repetition task (SNWRT)

To test nonword repetition in Spanish, a list of Spanish nonwords was developed making sure that the nonwords complied with the prosodic and phonological characteristics of words in the language. In contrast to the common practice of translating items from English when developing test measures in other languages, the Spanish nonwords were not based on the same sounds as the ENWRT. Instead, the nonwords included only Spanish consonants and vowels found in the language. In addition, it was important to ensure that the nonwords represented the most common word lengths of Spanish. Therefore, the nonwords only included two-, three-, and four-syllables. Words that resembled English words were excluded in order to control for crosslinguistic transfer. An original pool of 113 Spanish nonwords was created using syllables that had limited frequency in that position of the word (for additional details, Calderón, 2003). This approach is similar to that used by Dollaghan and Campbell (1998) for the development of the English NWRT. Thus, the SNWRT was assumed to parallel the English test and measure the same construct.

In order to make sure that errors in repetition were not due to speech production difficulties, only two of the late developing phonemes in Spanish (/ r, s /) were included. The production of these sounds was not penalized if children had difficulties producing these sounds in their spontaneous language. This initial work resulted in a pool of 66 two-syllable, 35 three-syllable and 12 four-syllable words. To ensure that the nonwords did not resemble real words, the nonwords were rated by a panel of native Spanish speakers. The panel was instructed to mark those nonwords that could be perceived as a real word. All nonwords that were judged to be wordlike were eliminated. The final set of nonwords was selected based on a series of item analyses conducted with a separate sample of children (Calderón, 2003). This process resulted in a set of 20 nonwords comprised of four 2-syllable, five 3-syllable, and eleven 4-syllable words (see Appendix). Nine words were stressed in the last syllable of the word, nine in the penultimate syllable, and two in the antepenultimate syllable. The audiotape of the nonwords was recorded by a different female who was a native speaker of Spanish.

Test administration and scoring

All children were tested by bilingual research assistants in a quiet room at the child’s school. Each language was tested on different days and the order of presentation of the two languages varied across children. Children were told they were going to listen to some made-up words and they had to repeat them exactly the way that they heard them. The nonwords were presented only once via audiotape and the child’s responses were recorded for later transcription and analysis. The nonwords were scored on a phoneme-by-phoneme basis following Dollaghan and Campbell’s (1998) procedures. Articulation errors due to a child’s difficulty producing sounds in their spontaneous speech were not penalized. This approach was followed because previous research has shown that children with speech impairment have no difficulties with nonword repetition tasks when their phonological errors are taken into account (Munson, Edwards, & Beckman, 2005). Only phoneme omissions or substitutions that were clearly within the child’s phonological repertoire and not repeated correctly were scored as incorrect. Spanish-influenced errors were not penalized when scoring the English task. There were no English-influenced errors on the Spanish task. The number of phonemes repeated correctly was divided by the total number of target phonemes and then multiplied by 100 to obtain the Total Percentage of Phonemes Correct (TPPC). The nonword repetition task responses of one half of the children were independently re-scored by a second bilingual research assistant. Item by item reliability checks yielded above 91% agreement between the two judges across the participants.

Statistical analyses

Group differences on the total percentage of phonemes correct (TPPC) in each language were compared using a t-test with an alpha error set at <.05. Effect sizes for these comparisons were calculated using Cohen’s d.

To evaluate the extent to which the nonword repetition task could be used to rule in or rule out language impairment, the sensitivity (i.e., the proportion of children with language impairment that scored below a cutoff score) and specificity (i.e., the proportion of children with typical language development that scored above a cutoff score) for each score and in each language were examined using Receiver Operating Characteristic (ROC) curves. The ROC curve plots the true positive rate (sensitivity) against the false positive rate (1-specificity) for the observed nonword repetition task scores and is used to determine the “best” cutoff score for the measure (i.e. the score that gives the highest sensitivity and specificity) (Gray, 2004; Sackett, Haynes, Guyatt, & Tugwell, 1991). The area under a ROC curve provides a measure of test accuracy. It is the percentage of randomly drawn pairs of subjects (one with typical language development, one with language impairment) that the test identifies accurately. In other words, it is the probability that a randomly selected child with typical language development will score higher than a randomly selected child with language impairment (Fawcett, 2004; Tape, 2008). An area of 1 represents a perfect test; an area of .90 to 1 indicates excellent accuracy; an area of .80 to .90 represents good accuracy; an area of .70 to .80 denotes fair accuracy; an area of .60 to .70 indicates poor accuracy; and an area of .50 represents a worthless test (Tape, 2008).

In addition, likelihood ratio (LR) analyses were conducted using presence/absence of language impairment as the gold standard. These metrics are considered to be robust to variations in the characteristics of the sample (Dollaghan, 2004). A positive LR (i.e., LR+ > 1) indicates the likelihood that a child has a language impairment. The likelihood that children with typical language development fail the measure should be below 1 (i.e., LR− < 1). The LR for a given test score is calculated based on sensitivity and specificity rates. The positive LR is sensitivity divided by (1- specificity). Likelihood ratios were calculated based on the cutoff scores generated by the ROC curve analyses. The ROC curves indicated that a TPPC score of 70% or lower was the score that maximized differentiation in each language to rule in a language impairment. A score above 70% was used to rule out language impairment.

Results

There were significant group differences between the two language ability groups (i.e., children with typical language development, children with language impairment). For English, the children with typical language development scored significantly higher (Mean = 79%, SD = 10%) than the children with language impairment (Mean = 63%, SD = 14%), t(73.895) = 6.134, p < .001, d = 1.06. Similar results were found for the Spanish nonword repetition task: the children with typical language development scored significantly higher (Mean = 77%, SD = 16%) than the children with language impairment (Mean = 63%, SD = 16%), t(142) = 4.987, p < .001, d = .86

When the ROC curves were examined, the area under the ROC curve for the ENWRT was found to have fair test accuracy (area = .756, S.E. = .04, C.I. 95% = .67 – .84) and was significantly greater than chance (area = .05), p <.001. Similarly, the area under the ROC curve for the SNWRT also had fair test accuracy (area = .76, S.E. = .04, C.I. 95% = .68 – .84) and was significantly greater than chance (area = .05), p < .001. Table 3 lists the sensitivity and specificity of different ENWRT scores including the scores corresponding to percentiles typically used in clinical practice and the score of 70% based on the ROC curve analysis. Although the ENWRT score had moderate specificity (i.e., .82), its sensitivity was poor (i.e., .55). About 45% of the children with language impairment were misclassified because their scores were higher than 70%. Table 4 lists the sensitivity and specificity of the SNWRT for the percentile scores and the 70% score derived from the ROC curve. Again, although the specificity of the SNWRT was moderate (i.e., .82), the sensitivity rate was not adequate (i.e., .61).

Table 3.

Sensitivity and Specificity of English NWRT Scores Using Different Clinical Thresholds

Percentile ENWRT Score Sensitivity Specificity
10th 59% .265 .989
16th 63% .347 .937
Of interest 70% .551 .821
Open in a new tab

Table 4.

Sensitivity and Specificity of Spanish NWRT Scores Using Different Clinical Thresholds

Percentile SNWRT Score Sensitivity Specificity
10th 48% .143 .916
16th 55% .265 .884
Of interest 70% .612 .821
Open in a new tab

Neither the ENWRT nor the SNWRT in isolation appeared to help rule in or rule out language impairment probably because children have varying levels of language skills across the two languages. As a result, a child with typical language development may fail in one language and not in the other. In contrast, children with language impairment are expected to have low scores in both languages. Thus, the next step was to calculate likelihood ratios based on the children’s performance in the two languages. Table 5 shows the number of children in each group (typical language development, language impairment), the proportion of children who failed both languages, and the proportion of children who failed only one language or passed both languages. When the two languages were considered, the specificity increased to .95, a classification rate considered to be good (Plante & Vance, 1994). Given that there were more boys than girls with language impairment, separate likelihood ratios for each gender are also included in Table 5.

Table 5.

NWRT Performance Rates and Likelihood Ratios for the Total Sample.

LI (n = 49) TLD (n = 95) Likelihood
Ratio
Number Proportion Number Proportion
Failed both 20 0.408 4 0.042 9.71
Failed one or passed both 29 0.592 91 0.958 0.62
Girls LI (n = 16) TLD (n = 50) Likelihood
Ratio
Number Proportion Number Proportion
Failed both 6 0.375 2 0.040 9.375
Failed one or passed both 10 0.625 48 0.960 0.65
Boys LI (n = 33) TLD (n = 45) Likelihood
Ratio
Number Proportion Number Proportion
Failed both 14 0.424 2 0.040 10.600
Failed one or passed both 19 0.576 43 0.956 0.60
Open in a new tab

As was described earlier, the positive likelihood ratio (LR) is the true positive rate (or sensitivity) divided by the false positive rate (or (1-specificity)). A clinically useful LR should be as high as possible. Based on the cutoff score of 70% in each language, the measure had an intermediate high positive LR of 9.71. This means that a score of 70% or lower in both SNWRT and ENWRT was more than nine times likely to come from a child with language impairment. A closer look at the data indicated individual differences in nonword repetition task performance across the two languages, regardless of the child’s language dominance. Some children who were English dominant (2 children with language impairment, 4 with typical language development) passed the SNWRT (see Table 6) and some children who were Spanish dominant (3 children with language impairment, 5 with typical language development) passed the ENWRT (see Table 7). Clearly, these results show that testing only in the child’s dominant language would lead to a greater rate of misclassification.

Table 6.

NWRT Performance Rates for the Sample of English-Dominant Children

LI (n = 24) TLD (n = 47)
Number Proportion Number Proportion
Fail both 8 0.333 1 0.021
Fail Spanish only 5 0.208 7 0.149
Fail English only 2 0.083 4 0.085
Open in a new tab

Table 7.

NWRT Performance Rates for the Sample of Spanish-Dominant Children

LI (n = 22) TLD (n = 41)
Number Proportion Number Proportion
Fail both 10 0.454 3 0.073
Fail Spanish only 3 0.136 5 0.122
Fail English only 5 0.227 8 0.195
Open in a new tab

Discussion

The purpose of this study was to examine the clinical usefulness of the nonword repetition task with bilingual Latino children. As a measure that does not rely on vocabulary or conceptual knowledge we predicted that the nonword repetition task would be useful in differentiating children with language impairment in clinical assessments. The results of this study replicate findings reported by research using this measure with English speakers (Conti-Ramsden et al., 2001; Conti-Ramsden & Hesketh, 2003; Ellis Weismer et al., 2000; Gray, 2003) and Spanish speakers (Girbau & Schwartz, 2007). Our sample of bilingual children with language impairment had a significant deficit in nonword repetition task compared to their peers with typical language development. This is consistent with current proposals about the nature of specific language impairment which emphasize limitations in children’s capacity to analyze, encode and form phonological and morphological representations necessary for the storage and retrieval of word forms (for a discussion, Ellis Weismer & Edwards, 2006).

The second goal of this study was to examine the impact of individual differences in language skills on the clinical differentiation of these children. We predicted that the clinical accuracy of the measure would vary depending on the language(s) tested. The results showed that nonword repetition task performance on a single language (i.e., Spanish or English) resulted in inadequate sensitivity and specificity rates. A considerable proportion of children with typical language development did not obtain passing scores in both languages, probably because of individual differences in use and skills in one of the languages tested. Using a monolingual approach in clinical assessments of Latino children is not likely to reduce assessment bias, even with measures that purportedly do not rely on vocabulary knowledge. Our findings are comparable to those reported by Kohnert and colleagues (2006). In that study, ENWRT cutoff scores derived from a monolingual English sample resulted in lower specificity rates for the bilingual children tested. In the present study, the nonword repetition task cutoff scores were not based on monolingual norms but were directly derived from the bilingual Latino sample. Thus, this procedure allowed for performance comparisons that were not likely to be confounded by differences in ethnicity or socioeconomic background.

Discrepancies in performance between the two languages may be related to differences in the strength of linguistic representations as children vary in their exposure and use of the languages. Children with greater exposure to a given language may find it easier to draw analogies with the phonological forms of real words in that language. However, exposure and use are not stable across contexts and they vary over time. Mainela-Arnold and Evans (2005) argued that working memory capacity is not distinct from language knowledge and that performance on these tasks emerges from an interaction between frequency from language input and factors related to the individual speaker.

The sensitivity rates found in the present study may also be reduced because our gold standard did not rely on treatment status. Many of the participating children were not receiving intervention at the time of this study. The participants with language impairment demonstrated a wide range of severity (from mild to severe) which probably explained why some children were not receiving services. Inclusion in treatment was the gold standard used in Dollaghan and Campbell’s (1998) study and in fact, Ellis Weismer and colleagues (2000) speculated that increased diagnostic accuracy with the nonword repetition task can be obtained when treatment status is taken into account.

The study also showed that the clinical differentiation of children with and without language impairment was improved when both languages were evaluated. This is in contrast with the prevalent practice of selecting English as the language of testing in assessments of bilingual children (e.g., Caesar & Kohler, 2007; Ortiz, 2001). The results also demonstrated that selecting the child’s dominant language may result in less accurate assessments. A child’s language dominance may not be consistent across tasks or across direct observations of the child’s language use, and it varies over time as well. The findings of this study suggest that a bilingual procedure may provide greater clinical accuracy than monolingual testing.

The likelihood ratio analyses based on the two languages combined indicated that the measure could help identify language impairment with an intermediate high likelihood (i.e., 9.71). This level of accuracy was lower than the likelihood ratios of the ENWRT reported by Dollaghan and Campbell (1998) (i.e., 25), but it was higher than those reported by Kohnert et al. (i.e., 5.07), Ellis Weismer et al. (i.e., 2.60), and Thal, Miller, Carlson and Vega (2005) (i.e., 3.33). The LR obtained for this bilingual sample was similar to the LR obtained in a study with bilingual Spanish speakers using a SNWRT (i.e., 9.00) (Girbau & Schwartz, 2008). However, it is important to note that the classification accuracy of nonword repetition tasks varies greatly across clinical samples and nonword lists. A recent meta-analysis based on 23 English nonword repetition task studies comparing children with and without specific language impairment found large differences in effect sizes across studies depending on the nonword repetition lists used (Estes, Evans, & Else-Quest, 2007). For example, the Children’s Test of Nonword Repetition (CNRep: Gathercole & Baddeley, 1996), yielded a high level of sensitivity (i.e., the percentage of true positives that were correctly identified by the measure was 95%) and high specificity (i.e., the percentage of true negatives that were correctly identified was 100%) (Gray, 2003). Archibald and Gathercole (2006) found that affected children obtained lower scores on the CNRep than on the Nonword Repetition Test used by Dollaghan and others. These differences were interpreted as potentially related to the greater articulatory demands imposed by the inclusion of consonant clusters (e.g., “cl”, “pr”) and unstressed syllables with limited saliency in the CNRep. Both the ENWRT and the SNWRT used in the present study were comparable in word length but had nonwords with fewer syllables and less complexity than the CNRep nonword list and the SNWRT used by Girbau and Schwartz. Measures with longer and more complex nonwords may promote greater clinical differentiation. However, it is important to note that children vary significantly in the manifestation of their language learning difficulties or in the role of different underlying causal processes. A single cognitive process (such as verbal working memory) is not likely to fully explain their learning deficits and much more research will be needed to develop measures that will help understand the nature of their difficulties (for a discussion, Ellis Weismer & Edwards, 2006).

The findings of the present study suggest that a verbal working memory task such as the nonword repetition administered in the two languages should be used only in combination with other assessment measures in both languages (Simon-Cereijido & Gutierrez-Clellen, 2007). Research is currently underway to determine which combination of measures may help improve the diagnostic classification of Spanish-English speaking children in clinical assessments. There is evidence that for English, the combination of a nonword repetition task and the use of past tense –ed verb marking result in a sufficiently high diagnostic accuracy rate to be considered a significant identifier of young children with specific language impairment (Conti-Ramsden, 2003). Similarly, Bortolini and colleagues (2006) suggested the use of the nonword repetition task in combination with a grammatical measure to identify Italian specific language impairment.

The available research with the nonword repetition task indicates that these models may help provide greater identification accuracy. In addition, studies focused on the predictive validity of these measures will be needed to establish whether performance on verbal working memory tasks such as the nonword repetition tasks can contribute to differences in language learning over time. Current research has shown that the presence of a language impairment in kindergarten or during the early grades is an important indicator of reading difficulties later in school (Catts, Bridges, Little, & Tomblin, 2008). Language screenings in both languages using the nonword repetition task in combination with other measures may help identify at risk Latino children early enough to prevent their academic delays.

Biographies

Dr. Vera Gutierrez-Clellen is Professor and Coordinator of the Bilingual Speech-Language Pathology Certificate in the School of Speech, Language, and Hearing Sciences at San Diego State University. Her research has focused on the development of identification measures as well as a language intervention curriculum for Latino children with language impairments.

Gabriela Simon-Cereijido is an assistant professor in the Department of Communication Disorders, California State University, Los Angeles.

Appendix

The Spanish Nonword Repetition Task

2-syllable 3-syllable 4-syllable
t i n ɾu duɾbiɛpos xi tʃɛɾu pia xɛn bur naɪpos
g au ß ɛ ɾ ʧɛɾu gua xɛn tʃu faɪt i n M aʊteɪp ɔɪt i n
mɛɾgui t in tʃau ßɛl buɾdaɪgui pos x i ɾu tʃɛp ia
m ɛɾf a s r u tʃɛtua Xi ɾu tʃɛpos t i ŋ k aʊm ie pos
xus naɪʧɛ mɛɾxan tʃu t i n xi tʃɛɾu pos
k ui maɪ xɛn pos
Open in a new tab

Note: Bolded sections denote stressed syllables.

Contributor Information

Vera F. Gutiérrez-Clellen, San Diego State University

Gabriela Simon-Cereijido, San Diego State University, University of California, San Diego.

References

  1. Aguado Alonso G, Cuetos Vega F, Domezain MJ, Pascual B. Repetición de pseudopalabras en niños españoles con trastorno específico del lenguaje: marcador psicolingüístico [Repetition of pseudowords by Spanish children with specific language impairment: a psycholinguistic marker] Revista de Neurologíal. 2006;43(S01):S201. [PubMed] [Google Scholar]
  2. Alloway TP, Archibald LMD. Working memory and learning in children wiht Developmental Coordination Disorder and Specific Language Impairment. Journal of Learning Disabilities. 2008;41:151–162. doi: 10.1177/0022219408315815. [DOI] [PubMed] [Google Scholar]
  3. Alloway TP, Rajendran G, Archibald LMD. Working memory in children with developmental disorders. Journal of Learning Disabilities. 2009;20:1–11. doi: 10.1177/0022219409335214. [DOI] [PubMed] [Google Scholar]
  4. Archibald LMD, Gathercole SE. Nonword repetition: A comparison of tests. Journal of Speech, Language and Hearing Research. 2006;49(5):970–983. doi: 10.1044/1092-4388(2006/070). [DOI] [PubMed] [Google Scholar]
  5. Bishop DVM, North T, Donlan C. Nonword repetition as a behavioural marker for inherited language impairment: Evidence from a twin study. Journal of Child Psychology and Psychiatry and Allied Disciplines. 1996;37(4):391–403. doi: 10.1111/j.1469-7610.1996.tb01420.x. [DOI] [PubMed] [Google Scholar]
  6. Bortolini U, Arfe B, Caselli CM, Degasperi L, Deevy P, Leonard LB. Clinical markers for specific language impairment in Italian: The contribution of clitics and non-word repetition. International Journal of Language and Communication Disorders. 2006;41(6):695–712. doi: 10.1080/13682820600570831. [DOI] [PubMed] [Google Scholar]
  7. Bull R, Scerif G. Executive functioning as a predictor of children's mathematics ability: Shifting, inhibition, and working memory. Neurodevelopmental Neuropsychology. 2001;19:273–293. doi: 10.1207/S15326942DN1903_3. [DOI] [PubMed] [Google Scholar]
  8. Caesar LG, Kohler PD. The state of school-based bilingual assessment: Actual practice versus recommended guidelines. Language, Speech and Hearing Services in Schools. 2007;38(3):190–200. doi: 10.1044/0161-1461(2007/020). [DOI] [PubMed] [Google Scholar]
  9. Calderón J. Working Memory in Spanish-English Bilinguals with Language Impairment. San Diego, CA: University of California, San Diego/San Diego State University; 2003. [Google Scholar]
  10. Catts HW, Bridges MS, Little TD, Tomblin JB. Reading achievement growth in children with language impairments. Journal of Speech, Language and Hearing Research. 2008;51(6):1569–1579. doi: 10.1044/1092-4388(2008/07-0259). [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Conti-Ramsden G. Processing and linguistic markers in young children with specific language impairment (SLI) Journal of Speech, Language and Hearing Research. 2003;46(5):1029–1037. doi: 10.1044/1092-4388(2003/082). [DOI] [PubMed] [Google Scholar]
  12. Conti-Ramsden G, Botting N, Faragher B. Psycholinguistic markers for Specific Language Impairment (SLI) Journal of Child Psychology and Psychiatry and Allied Disciplines. 2001;42:741–748. doi: 10.1111/1469-7610.00770. [DOI] [PubMed] [Google Scholar]
  13. Conti-Ramsden G, Hesketh A. Risk markers for SLI: a study of young language-learning children. International Journal of Language & Communication Disorders. 2003;38(3):251–263. doi: 10.1080/1368282031000092339. [DOI] [PubMed] [Google Scholar]
  14. Dawson JI, Stout CE, Eyer JA. Structured Photographic Expressive Language Test: Third Edition. DeKalb, IL: Janelle Publications; 2003. [Google Scholar]
  15. de Bree E, Rispens J, Gerrits E. Non-word repetition in Dutch children with (a risk of) dyslexia and SLI. Clinical Linguistics & Phonetics. 2007;21(11):935–944. doi: 10.1080/02699200701576892. [DOI] [PubMed] [Google Scholar]
  16. Dollaghan C. Evidence-based practice in communication disorders: what do we know, and when do we know it? Journal of Communication Disorders. 2004:391–400. doi: 10.1016/j.jcomdis.2004.04.002. [DOI] [PubMed] [Google Scholar]
  17. Dollaghan C, Campbell T. Nonword repetition and child language impairment. Journal of Speech and Hearing Research. 1998;41(5):1136–1146. doi: 10.1044/jslhr.4105.1136. [DOI] [PubMed] [Google Scholar]
  18. Dunn L, Dunn L. Peabody picture vocabulary test - revised (PPVT) Circle Pines, MN: American Guidance Service; 1981. [Google Scholar]
  19. Dunn L, Padilla E, Lugo D, Dunn L. Test de Vocabulario en Imágenes Peabody - Adaptación Hispanoamericana [Peabody Picture Vocabulary Tests - Latin American adaptation] Cicle Pines, MN: American Guidance Service; 1986. [Google Scholar]
  20. Ellis Weismer S, Edwards J. The role of phonological storage deficits in specific language impairment: A reconsideration. Applied Psycholinguistics. 2006;27:556–562. [Google Scholar]
  21. Ellis Weismer S, Tomblin J, Zhang X, Buckwalter P, Chynoweth J, Jones M. Nonword repetition performance in school-age children with and without language impairments. Journal of Speech, Language, and Hearing Research. 2000;43:865–878. doi: 10.1044/jslhr.4304.865. [DOI] [PubMed] [Google Scholar]
  22. Estes KG, Evans JL, Else-Quest NM. Differences in the nonword repetition performance of children with and without specific language impairment: A meta-analysis. Journal of Speech, Language, and Hearing Research. 2007;50(1):177–195. doi: 10.1044/1092-4388(2007/015). [DOI] [PubMed] [Google Scholar]
  23. Fawcett T. ROC Graphs: Notes and Practical Considerations for Researchers. Palo Alto, CA: HP Laboratories; 2004. [Google Scholar]
  24. Gathercole SE. Nonword repetition and word learning: The nature of the relationship. Applied Psycholinguistics. 2006;27:513–543. [Google Scholar]
  25. Gathercole SE, Alloway TP, Willis CS, Adams AM. Working memory in children with reading disabilities. Journal of Experimental Child Psychology. 2006;93:265–281. doi: 10.1016/j.jecp.2005.08.003. [DOI] [PubMed] [Google Scholar]
  26. Gathercole SE, Baddeley AD. The Children's Test of Nonword Repetition. London: The Psychological Corporation; 1996. [Google Scholar]
  27. Gersten R, Jordan NC, Flojo JR. Early identification and interventions for students with mathematics difficulties. Journal of Learning Disabilities. 2005;38:293–304. doi: 10.1177/00222194050380040301. [DOI] [PubMed] [Google Scholar]
  28. Girbau D, Schwartz RG. Non-word repetition in Spanish-speaking children with specific language impairment (SLI) International Journal of Language & Communication Disorders. 2007;42(1):59–75. doi: 10.1080/13682820600783210. [DOI] [PubMed] [Google Scholar]
  29. Girbau D, Schwartz RG. Phonological working memory in Spanish-English bilingual children with and without specific language impairment. Journal of Communication Disorders. 2008;41(2):124–145. doi: 10.1016/j.jcomdis.2007.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Gray GE. Concise guide to evidence-based psychiatry. Washington, DC: American Psychiatric Publishing; 2004. [Google Scholar]
  31. Gray S. Diagnostic accuracy and test-retest reliability of nonword repetition and digit span tasks administered to preschool children with specific language impairment. Journal of Communication Disorders. 2003;36:129–151. doi: 10.1016/s0021-9924(03)00003-0. [DOI] [PubMed] [Google Scholar]
  32. Gutiérrez-Clellen V, Restrepo MA, Simon-Cereijido G. Evaluating the discriminant accuracy of a grammatical measure with Spanish-speaking children. Journal of Speech, Language, and Hearing Research. 2006;49:1209–1223. doi: 10.1044/1092-4388(2006/087). [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Gutiérrez-Clellen V, Simon-Cereijido G. Using language sampling in clinical assessments with bilingual children: Challenges and future directions. doi: 10.1055/s-0029-1241722. (submitted). [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Gutiérrez-Clellen VF, Calderón J, Ellis Weismer S. Verbal working memory in bilingual chilldren. Journal of Speech, Language, and Hearing Research. 2004;47:863–876. doi: 10.1044/1092-4388(2004/064). [DOI] [PubMed] [Google Scholar]
  35. Gutiérrez-Clellen VF, Kreiter J. Understanding child bilingual acquisition using parent and teacher reports. Applied Psycholinguistics. 2003;24:267–288. [Google Scholar]
  36. Gutiérrez-Clellen VF, Restrepo MA, Bedore L, Peña E, Anderson R. Language sample analysis in Spanish-speaking children: Methodological considerations. Language, Speech and Hearing Services in Schools. 2000;31:88–98. doi: 10.1044/0161-1461.3101.88. [DOI] [PubMed] [Google Scholar]
  37. Gutiérrez-Clellen VF, Simon-Cereijido G. The discriminant accuracy of a grammatical measure with Latino English-speaking children. Journal of Speech, Language, and Hearing Research. 2007;50(4):968–981. doi: 10.1044/1092-4388(2007/068). [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Kibby MY, Marks W, Morgan S, Long C. Specific impairment in developmental reading disabilities: A working memory approach. Journal of Learning Disabilities. 2009;37:349–363. doi: 10.1177/00222194040370040601. [DOI] [PubMed] [Google Scholar]
  39. Kohnert K, Windsor J, Yim D. Do language-based processing tasks separate children with primary language impairment from typical bilinguals? Journal of Learning Disabilities Research & Practice. 2006:19–29. [Google Scholar]
  40. Leonard LB. Children with specific language impairment. Cambridge, Mass: The MIT Press; 1998. [Google Scholar]
  41. Mainela-Arnold E, Evans JL. Beyond capacity limitations: Determinants of word recall performance on verbal working memory span tasks in children with SLI. Journal of Speech, Language, and Hearing Research. 2005;48(4):897–909. doi: 10.1044/1092-4388(2005/062). [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mayer M. Frog, where are you? New York, NY: 1969. [Google Scholar]
  43. Mayer M. Frog on his own. New York, NY: 1973. [Google Scholar]
  44. Mayer M. Frog goes to dinner. New York, NY: 1974. [Google Scholar]
  45. Mayer M. One frog too many. New York, NY: 1975. [Google Scholar]
  46. Miller JF, Chapman R. Systematic analysis of language transcripts. (Version 6.1) Madison, WI: Language Analysis Laboratory; 2000. [Computer software]. [Google Scholar]
  47. Munson B, Edwards J, Beckman ME. Relationships between nonword repetition accuracy and other measures of linguistic development in children with phonological disorders. Journal of Speech, Language, and Hearing Research. 2005;48(1):61–78. doi: 10.1044/1092-4388(2005/006). [DOI] [PubMed] [Google Scholar]
  48. Oller DK, Eilers RE. An integrated approach to evaluating effects of bilingualism in Miami school children: The study design. In: Oller DK, Eilers RE, editors. Language and Literacy in Bilingual Children. Clevedon, England: Multilingual Matters; 2002. pp. 22–41. [Google Scholar]
  49. Ortiz SO. Assessment of cognitive abilities in Hispanic children. Seminars in Speech and Language. 2001;22:17–37. doi: 10.1055/s-2001-13869. [DOI] [PubMed] [Google Scholar]
  50. Peña ED, Gutiérrez-Clellen VF, Iglesias A, Goldstein B, Bedore LM. Bilingual English Spanish Assessment (BESA) (n.d.). [Google Scholar]
  51. Perona K, Plante E, Vance R. Diagnostic Accuracy of the Structured Photographic Expressive Language Test: Third Edition (SPELT-3) Language, Speech, and Hearing Services in Schools. 2005;36:103–115. doi: 10.1044/0161-1461(2005/010). [DOI] [PubMed] [Google Scholar]
  52. Plante E, Vance R. Selection of preschool language tests: A data-base approach. Language, Speech, and Hearing Services in the Schools. 1994;25:15–24. [Google Scholar]
  53. Puranik CS, Petscher, Otaiba SA, Catts HW, Lonigan CJ. Development of oral reading fluency in children with speech or language impairments: A growth curve analysis. Journal of Learning Disabilities. 2008;41:545–560. doi: 10.1177/0022219408317858. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Restrepo MA. Identifiers of predominantly Spanish-speaking children with language impairment. Journal of Speech, Language, and Hearing Research. 1998;41:1398–1411. doi: 10.1044/jslhr.4106.1398. [DOI] [PubMed] [Google Scholar]
  55. Restrepo MA, Silverman S. Validity of the Spanish Preschool Language Scale-3 for use with bilingual children. American Journal of Speech Language Pathology. 2001;10:382–393. [Google Scholar]
  56. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2nd ed. Boston, MA: Little, Brown; 1991. [Google Scholar]
  57. Samson JF, Lesaux NK. Language minority learners in special education: Rates and predictors of identification for services. Journal of Learning Disabilities. 2009;42:148–162. doi: 10.1177/0022219408326221. [DOI] [PubMed] [Google Scholar]
  58. Service E. Phonology, working memory, and foreign language learning. The Quarterly Journal of Experimental Psychology. 1992;45 A(1):21–50. doi: 10.1080/14640749208401314. [DOI] [PubMed] [Google Scholar]
  59. Simon-Cereijido G, Gutierrez-Clellen VF. Spontaneous language markers of Spanish language impairment. Applied Psycholinguistics. 2007;28:317–339. [Google Scholar]
  60. Swanson HL, Beebe-Frankenberger M. The relationship between working memory and mathematical problem solving in children at risk and not at risk for match disabilities. Journal of Education Psychology. 2004;96:471–491. [Google Scholar]
  61. Tape TG. Interpreting diagnostic tests. [Retrieved December 14 2008];2008 from http://gim.unmc.edu/dxtests/Default.htm. [Google Scholar]
  62. Thal DJ, Miller S, Carlson J, Vega MM. Nonword repetition and language development in 4-year-old children with and without a history of early language delay. Journal of Speech, Language, and Hearing Research. 2005;48(6):1481–1495. doi: 10.1044/1092-4388(2005/103). [DOI] [PubMed] [Google Scholar]
  63. U.S. Department of Commerce. [Retrieved November 28, 2006];The Hispanic Population. 2001 from http://www.census.gov/prod/2001pubs/c2kbr01-3.pdf.
  64. Umbel VM, Pearson BZ, Fernandez MC, Oller DK. Measuring bilingual children's receptive vocabularies. Child Development. 1992;63:1012–1020. [PubMed] [Google Scholar]
  65. Werner EO, Kresheck JD. Spanish Structured Photographic Expressive Language Test-Preschool. Sandwich, IL: Janelle Publications; 1989. [Google Scholar]
  66. Wiig EH, Secord WA, Semel E. Clinical Evaluation of Language Fundamentals, Second Edition Preschool Spanish (CELF Preschool-2 Spanish) San Antonio, TX: The Psychological Corporation; 1992. [Google Scholar]
  67. Woodcock RW. Woodcock Language Proficiency Battery - Revised. Itasca, IL: Riverside Publishing; 1991. [Google Scholar]
  68. Woodcock RW, Muñoz-Sandoval AF. Woodcock Language Proficiency Battery-Revised: Spanish Form. Itasca, IL: Riverside Publishing; 1995. [Google Scholar]
  69. Zehler AM, Fleischman HL, Hopstock PI, Stephenson TG, Pendzick ML, Sapru S. Descriptive Study of Services to LEP Students and LEP Students with Disabilities. Arlington, VA: Development Associates, Inc.; 2003. Submitted to U.S. Department of Education, OELA. [Google Scholar]
  70. Zimmerman IL, Steiner VG, Pond RE. Preschool Language Scale-3: Spanish Edition. San Antonio, TX: Psychological Corporation; 1993. [Google Scholar]

RESOURCES