Identifying Developmental Language Disorder in School age Bilinguals: Semantics, Grammar, and Narratives

Abstract

Children with Developmental language disorder (DLD) have particular difficulty learning language despite otherwise general normal development. When school age bilingual children struggle with language, a common question is if the difficulties they present reflect lack of ability or lack of language experience. To address the question of identification of DLD in the context of bilingualism, we explore the diagnostic accuracy of measures administered in two languages. The Bilingual English Spanish Assessment Middle Extension (BESA-ME) assesses semantics and morphosyntax and the Test of Narrative Language (TNL) assesses comprehension and production of narratives. These measures were administered to 112 second graders (19 with DLD) and 64 fourth graders (7 with DLD). We explored the classification accuracy of each of these measures alone and in combination using receiver operator characteristic (ROC) curve analysis. The ROC curve illustrates diagnostic classification of a measure at various cutpoints. We compared the extent to which measures in English, Spanish, or best language account for area under the (ROC) curve. Discriminant function analysis using the best indicator (Spanish, English, best language) from each type of measure (semantics, morphosyntax, narrative) in combination demonstrate classification accuracy above 80%. Morphosyntax in the best language was the strongest predictor of DLD for second grade children. In fourth grade, the three measures contributed more equally in predicting DLD.

Educators share the responsibility for ensuring the students in their classrooms have the best possible educational outcomes. Students who are English learners (ELs) are at risk for poor educational outcomes when they lack the linguistic foundations needed for literacy and to gain access to the academic content presented across subject areas. For 70% of EL children in U.S. schools who come from Hispanic backgrounds, this risk may translate to reduced likelihood of high school completion and associated consequences related to employment opportunities and longer-term health and adjustment. Language skills are key to strong academic performance and attention to foundational skills at school age is a step toward maximizing achievement and minimizing risks.

When bilingual children struggle academically, it is challenging to disentangle the reasons; whether the source of difficulty arises from language differences associated with bilingualism or from a developmental language disorder (DLD). In the category of developmental language disorder, we consider children who have language skills that are below the expected range given their age (and in the case of bilingual children, below expectations given their language experience) (Bishop & Leonard, 2000; Leonard, 2014). We are especially concerned about children who demonstrate language difficulties but do not have conditions known to cause language disorder such as autism or hearing impairment (Bedore & Peña, 2008). Language difficulties in absence of any associated conditions may be due to language disorder or to language differences leading to uncertainty in accurate diagnostic decision-making. This uncertainty has been associated with both under- and over- identification of disabilities in Hispanic children (Sullivan & Bal, 2013). A recent best evidence synthesis highlights that when individual level achievement is accounted for, Hispanic children are less likely to receive special education support than their white peers (Morgan et al., 2018). Level of bilingualism was not specifically documented in the source data in the Morgan study, but the Hispanic children were more likely to be bilingual than their white peers. In contrast to the larger group, children who were enrolled in ESL services were more likely to be overrepresented with special education diagnoses. The solution in either case is to apply information about bilingual language development and the manifestation of DLD at school age and to deploy this knowledge using measures that accurately capture the expected language skills of English language learners.

Bilingual Children’s Language Development

It is challenging to make predictions about the expected developmental trajectories of bilingual children at school age. For example, as children from Spanish-speaking homes transition to increased use of English at school and with their peers, the quantity of Spanish to which they are exposed logically decreases. Variation in exposure to and knowledge of English and Spanish in bilingual first and third graders was documented by Bedore et al. (2016), demonstrating that children’s scores on language screeners were associated with their experiences at each grade. Spanish performance was more variable than English performance and children demonstrated higher Spanish performance when they regularly used the language. The importance of maintaining use of a language was also illustrated by Rojas and Iglesias (2013) tracking of narrative performance in Spanish English bilinguals from Kindergarten through the end of second grade. Children showed a general upward trajectory in narrative performance. At the same time, there were dips in English mean length of utterance and the number of different words produced in narrative samples. These changes were associated with summer breaks, which were reported to be periods of decreased English use and increased Spanish use. These studies illustrated that language experience impacts how children will perform in each of their languages at any point in time and this is likely to impact how they will perform on norm referenced measures which are often foundational to educational decisions around children’s language knowledge.

Variation in children’s exposure to each of their languages is associated with variation in their performance in each language. Low performance or proficiency in the second language is expected during second language acquisition, but without documentation of first language proficiency and exposure, it is difficult to whether low proficiency indicates a language learning difficulty. To understand the importance of children’s dual language profile, Collins, et al. (2014) studied the predictive relationship between profiles from kindergarten to second grade for 163 Spanish-English children at two timepoints: kindergarten and second grade. The profile was based on the children’s Spanish and English scores on the Woodcock Johnson Oral Language Battery. Children were divided into five profile groups (English proficient, Spanish proficient, dual proficient had Oral Language Composites above 85 in the corresponding language(s); borderline profiles had composites between 85 and 70 in both languages, and limited proficient with scores below 70 in both languages). At kindergarten, 63% of children’s profile scores were not yet in the proficient range in either language, while 64% were in the proficient range in one or both languages by second grade. Of interest were children in the borderline and low proficient groups. Children in the low proficiency group were particularly likely to persist in the low group although they had language experiences comparable to those children who demonstrated increased proficiency. The investigators obtained information about children’s referral to speech and language services. Less than half of the children in the low proficient group received speech and language services and even those children did not shift out of the low proficiency group. The authors highlight the need to take a more proactive stance in identifying bilingual children who may have language-learning difficulties such as DLD.

Assessments of School-Age Bilingual Children

There is growing awareness across health related and educational fields that the evidence base needs to be considered in the development of assessments. Here the evidence base should include information on the nature of the population to be assessed. Additionally, the nature and purpose of the instruments should be considered early in the development phase (Cohen et al., 2016). We know that assessments for identification of DLD in bilingual children need to account for the full range of dominance patterns observed and variability in knowledge associated with divided input using culturally and linguistically appropriate tasks that load on clinical markers of language impairment (Castilla-Earls et al., 2019). Clinical markers are forms that are likely to be produced correctly by children with typical bilingual skills and not produced or produced in error by children with DLD. Use of these markers is likely to lead to acceptable classification accuracy. Specifically, for clinical decision making, measures that identify DLD should have at least 80% sensitivity and 80% specificity (Plante & Vance, 1994).

There is a growing set of measures and databases for preschool and early school age children that yield the quality data needed to make reliable clinical decisions. Available standardized measures focus mainly on Spanish-English bilinguals. These include the BESA (Peña, Gutiérrez-Clellen, et al., 2018), Preschool Language Scale-5 Spanish edition (Zimmerman, et al., 2012) and the Clinical Evaluation of Language Fundamentals-Preschool Spanish edition (Wiig, et al., 2009). These tests target children in the three to six year age range. The Clinical Evaluation of Language Fundamentals-4 Spanish (Wiig et al., 2006), has a broad age range (5–21). Most studies of this measure examine performance in younger children. For example, a recent study evaluated performance of bilingual, low income children with and without DLD between the ages of 5;0 and 7;11 on the CELF-4 Spanish (Barragan et al., 2018). Results of this study indicated that for Spanish-English bilinguals in English-only schools, the cut score suggested by the examiners manual overidentified typical children as DLD. An empirically derived cut score of 78 (lower than the suggested cut score of 85) most accurately classified this group of children. This demonstrates the need to consider level of children’s exposure to each language in making clinical decisions.

At school age, the manifestation of DLD starts to shift in monolingual English speakers. In the oral domain, children start to move into the expected accuracy range in regards to production of grammatical elements while difficulties in complex syntax are likely to become more evident. Even when children demonstrate knowledge of common vocabulary they may lack the depth and breadth of their peers and demonstrate word finding difficulties. This profile may be entangled with the written domain as children’s oral language difficulties interfere with children’s reading and the opportunities to build general language knowledge based on literacy experience. Tasks that remain challenging for school age bilinguals include the production of complex syntax, production of behaviors that demonstrate breadth and depth of semantic knowledge such as definitions, analogies, and repeated associations. These difficulties are particularly evident in the comprehension and production of narratives because it requires the integration of semantic and syntax to convey precise information. In bilingual children, this profile is likely to be complicated by the fact that they may still be learning their second language while their first language plateaus. Further, bilingual children may exhibit mixed dominance profiles when tested across different linguistic areas (Bedore et al., 2016; Lugo-Neris et al., 2015). In this scenario, it is important to determine what clinical markers will be most informative for school-aged bilinguals.

Although there are relatively few studies of DLD in school-age bilinguals, available work suggests that the clinical markers evident at early school-age hold promise for differentiating children with and without DLD into the elementary years. Jacobson (2012) evaluated the production of direct object clitics and verb tense marking in the Spanish of bilingual children with and without DLD between first and sixth grade. Although both groups of children demonstrated errors in the production of these forms in a sentence repetition task, the children with DLD demonstrated the same kinds of errors as their TD peers. In the semantic domain, bilingual children with DLD also demonstrate differences in their productions of repeated associations. Sheng et al. (2012) compared the productions of repeated associations of children ages 7; 0 to 9;11 with high and low Spanish experience who did or did not have DLD. Children with DLD showed less semantic depth across their two languages than did their TD- language, experience, and age matched peers.

While there is considerable work looking at narrative development in younger children, there is less information about narrative in school-age children. Squires et al. (2014) considered the narrative difficulties of bilingual children at kindergarten and first grade with and without DLD who were matched on the basis of age and language experience. Narrative samples in Spanish and English were scored for macro and microstructure. TD children’s narrative skills increased in both languages over the two year period but children with DLD did not demonstrate comparable gains. Cross-language transfer was evident for macrostructure but not microstructure. This may be related to the need for language-specific input to learn the nuances of microstructure. Assuming that these difficulties persist into school age, measures that focus on the various aspects of narrative skills are likely to also differentiate children with and without DLD.

An additional factor to consider when assessing school-age bilingual children, is how to combine or interpret the information obtained across multiple measures of language ability (e.g. semantics, morphosyntax, and narratives) in order to best differentiate TD and DLD groups when testing in more than one language. Empirically deriving cut-scores and examining the classification accuracy of each individual measure is informative for providing clinical guidelines for test use. Combining measures in a discriminant functions analysis provides insights about how to best weight scores for clinical decision-making. An additional way to interpret dual-language results includes selecting the “best language score,” which refers to the language in which the child scored highest on a particular measure (Peña & Bedore, 2011). Peña, Bedore and Kester (2016) for example found that comparing a child’s coordinate score (the higher standard score in Spanish or English) on the semantics subtest of the BESA yielded the best classification accuracy in bilingual children when compared to single language or total language scoring. This approach has been implemented with children up to age 6;11 reported in (Peña, Gutiérrez-Clellen, et al., 2018). We do not know whether using the best language scores will yield high classification accuracy for an older school-age sample, particularly for other linguistic domains (e.g. morphosyntax, narratives). Nor do we know the effects of combining scores across both language and domain.

Purpose

The purpose of the current study is to explore the classification accuracy of various measures of language development designed for school age Spanish-English bilingual children including the Semantics and Morphosyntax subtests of the Bilingual English Spanish Assessment Middle Extension (BESA-ME) and the English and Spanish versions of the Test of Narrative Language (TNL) (Gillam & Pearson, 2004; Gillam et al., n.d.). Our specific research questions were:

1. What is the classification accuracy of English, Spanish, and Best Language scores for each of the target measures for a sample of Spanish-English bilingual second and fourth graders?

2. What is the best combination of scores across all three measures (BESA-ME Morphosyntax, BESA-ME Semantics, TNL) that results in highest classification accuracy for each grade (second and fourth)?

Method

Participants

Participants were 175 Spanish-English bilingual children with typical language development (n = 149; 85%) or who met the criteria for developmental language disorder (n = 26; 15%) in the second or fourth grades. Participants were selected from a larger pool of 1,696 bilingual elementary school-aged children recruited from public schools in the Central Texas region for a longitudinal study of bilingual language development when they were in preschool, first, or third grade (Bedore, Peña, et al., 2016). From this larger pool, 334 children were recruited to the longitudinal phase of the study if they identified as Latino and had exposure to Spanish and/or English. Children who scored within the risk range for DLD in their better language on the language screener were invited to the longitudinal part of the study (i.e., a score at least one standard deviation below the mean on either of two linguistic domains, semantics or morphosyntax, in the better language). Additionally, at least three children who scored above the risk range were invited to the study for every child scoring below the risk cut off. These children varied in the screener scores (from low average, average, and high average ranges) and matched the risk children for age, age of first English exposure, current exposure to English, sex, and parental socioeconomic status. All children were tested one year after initial screening to verify DLD status. Testing included non-verbal IQ using the Universal Nonverbal Intelligence Test (Bracken & McCallum, 2016), administration of the index measures, and completion of parent and teacher interviews used as reference measures.

From the group of 334, children were included in the current analysis if they were enrolled in the second or fourth grade at the time of testing and if they had complete parent, teacher, screening, and testing data in both languages. These criteria resulted in 160 children. We included an additional 15 children who had all but one data point if the missing data point was: (a) either one parent or one teacher concern indicator (n = 13), or (b) a test variable in the child’s weaker language, as determined by all remaining test variables (n = 2). Children at risk for language disorder were over-sampled given the purposes of the larger study (Bedore et al., 2016), resulting in a greater proportion of children with DLD in our sample than in the general population (Tomblin et al., 1997).

Table 1 shows participant demographics. Children were between the ages of 7;2 and 11;6 (M = 8;7, SD = 1;0) and 49% of the sample was female. There were 111 second graders representing 63% of the sample and 64 fourth graders representing 37% of the sample. The average maternal education level was 2.7 (SD = 1.6), corresponding with a middle school (2) or partial (3) high school education (Hollingshead, 1975). As reported by their parents, 87% of the children were Hispanic of Mexican descent, 4% were Hispanic of non-Mexican descent (e.g., Salvadorean), 1% were non-Hispanic, and the remaining 8% were unreported. On average at the time of testing, children had 50% English input/output (SD = 22%) per week.

Table 1.

Participant Demographics, Screening, and Test Scores (n = 175)

			Second Grade				Fourth Grade
			TD n = 92		DLD n = 19		TD n = 57		DLD n = 7
			M	SD	M	SD	M	SD	M	SD
	Age in Months		94.42	4.17	95.58	5.90	117.72	5.75	116.57	4.47
	Age of First English Exposure		2.82	1.83	3.18	1.74	2.22	1.89	2.50	2.07
	Current Input/Output	English	47.05	22.66	49.89	25.22	53.67	18.59	59.57	22.35
		Spanish	52.95	22.66	50.11	25.22	46.33	18.59	40.43	22.35
	Maternal Education		2.53	1.52	2.94	1.57	2.74	1.74	3.17	1.60
	UNIT		101.64	13.05	102.16	15.49	102.40	12.11	88.71	10.32
Reference Measures	Parent Concern (Best Language)		102.87	11.62	80.25	25.68	103.89	11.50	79.88	25.82
	Teacher Concern (Best Language)		105.11	9.20	77.73	18.64	103.02	11.50	83.03	12.67
	BESOS Morphosyntax	English	82.9511	21.19	59.24	11.01	94.79	21.33	76.94	13.46
		Spanish	93.3902	19.72	61.73	7.57	81.44	18.31	69.05	13.78
	BESOS Semantics	English	84.9043	16.42	70.59	11.66	97.07	32.24	65.45	15.67
		Spanish	91.7815	18.91	71.51	13.11	86.57	32.26	58.17	7.13
Index Measures	BESA-ME Morphosyntax	English	85.64	19.83	59.58	11.73	95.77	16.56	77.14	14.38
		Spanish	91.08	19.99	59.84	9.49	81.65	18.43	62.71	14.00
	BESA-ME Semantics	English	104.21	20.28	82.32	19.47	113.88	15.59	96.14	25.25
		Spanish	94.92	19.13	73.42	15.15	90.33	17.51	73.43	11.27
	TNL	English	83.46	15.93	67.63	10.31	86.32	14.32	69.57	11.28
		Spanish	97.51	18.50	77.58	15.68	98.70	17.27	87.29	12.87

Note. TD = Typically Developing; DLD = Developmental Language Disability; UNIT = Universal Nonverbal Intelligence Test; BESOS = Bilingual English Spanish Oral Screener; BESA-ME = Bilingual English Spanish Assessment – Middle Extension; TNL = Test of Narrative Language.

Demographic information was obtained via phone interviews of participant’s parents and included questions related to race, ethnicity, parent education and occupation. Socioeconomic status information was derived using Hollingshead scores based on mother education.

Language history was obtained using the BIOS Bilingual Input Output Survey (BIOS; Peña, et al, 2018), which is a detailed interview in which parents and/or teachers provide information regarding the child’s history of exposure and use of English and Spanish. Parent interviews were completed by phone. Teacher interviews were conducted in person. Age of acquisition (AoA) as well as percentage estimate of current input and output were calculated based on parent and teacher responses.

Test Methods

Data collection methods were planned as part of the study design to include reference and index measure independently administered with a community sample using a prospective design. A set of reference measures were used to identify DLD for the purpose of this analysis. The index measures are the measures under study and these were evaluated for their sensitivity and specificity relative to the reference standard as a first step in establishing classification accuracy (Cohen et al., 2016).

Reference Measures

The reference standard employed in this study is a combination of screener data using the Bilingual Spanish English Oral Screener (BESOS, Peña, Bedore, Gutiérrez-Clellen, et al., 2010) combined with parent and teacher concern. Both these measures have fair to good sensitivity and specificity independently for bilingual Spanish-English speaking children. Because the BESOS is a screener, it is quick to administer in both languages (approximately 15 minutes). Parent and teacher concern was employed here because this is typically a first step to making a referral for assessment of DLD (Hendricks, et al., 2019). Parent and teacher report is shown to correlate with child performance on language tasks (Bedore, Peña, Joyner, et al., 2011; Wittke et al., 2013). All raw scores were converted to standard scores with a mean of 100 and a standard deviation of 15 allowing direct comparison.

The 1st or 3rd grade versions of the BESOS were used to screen participants depending on their grade. Spanish and English subtests target semantics and morphosyntax. Items included in the BESOS include items from the BESA-ME try-out study that had high differentiation across TD and DLD groups. Of the 66 items on the BESOS 1^st grade screener, there are 15 items in common with the longer (159 item) version of the BESA-ME. The 3^rd grade BESOS screener has 67 items, and of these, 18 overlap with the BESA-ME. The BESOS semantics subtests contain in Spanish and English items that assessed semantic relationships such as categories, descriptions, and comparisons. The specific items are not translations of each other so that each language version tests similar concepts with different items. The morphosyntax subtests included cloze and sentence repetition items. These are comprised of items that are difficult for children with DLD in the target language (e.g. past tense -ed, third person present tense -s, in English; articles, direct object clitics, and subjunctive in Spanish). Preliminary analysis demonstrates that for first grade the BESOS had 93% and specificity of 92% using a cutscore of −1 SD across both languages (reported in Peña, Bedore, Shivabasappa, et al., 2018). Similarly, preliminary data for the third grade BESOS demonstrates a sensitivity of 80% and specificity of 94% using a cutscore of 85 (−1 SD below the mean across both languages).

Parents and teachers also completed the Instrument to Assess Language Knowledge (ITALK; Peña, Gutiérrez-Clellen, et al., 2018). This questionnaire was completed in an interview format. Parents and teachers were asked to rate children’s skills in English and Spanish in terms of vocabulary, speech, sentence production, grammar, and comprehension on a scale of 0 to 5. There are descriptors and examples for each point of the scale so that parents and teachers can select the description which best matches their child’s ability in each language. Scores across the five areas were then averaged resulting in a parent and teacher concern score for each language and converted to standard scores.

Index Measures

We administered three index measures in English and Spanish. Specifically, we employed a measure of morphosyntax and one of semantics (from the experimental version of the Bilingual English Spanish Assessment - Middle Extension BESA-ME; (BESA-ME; Peña, Bedore, Iglesias, et al., 2010). Additionally, we employed a measure of narrative language, the Test of Narrative Language (TNL; Gillam & Pearson, 2004; Gillam, Peña, Bedore, & Pearson, et al., n.d.)

The BESA-ME (Peña, Bedore, Iglesias, et al, 2010) is an experimental test developed for Spanish-English bilingual children ages 7;0–9;11 years of age. Similar to the BESA (Peña, Gutiérrez-Clellen, et al., 2018), it consists of semantics and morphosyntax subtests in both languages. The BESA-ME Morphosyntax is a measure of children’s grammatical knowledge, and contains cloze and sentence repetition items. The Morphosyntax subtest has a coefficient alpha of 0.877 in Spanish and 0.880 in English based on preliminary validity studies. The BESA-ME Semantics is a measure of both semantic breadth and depth and contains receptive and expressive items that assess naming words and categories, functions, definitions, analogies, associations, and similarities/differences. Preliminary data indicate a coefficient alpha of .788 for Spanish Semantics and .898 for English Semantics. Standard Scores were derived for each of the BESA-ME subtests in each language.

Participants completed the English and Spanish versions of the Test of Narrative Language (Gillam & Pearson, 2004; Gillam, Peña, Bedore & Pearson, n.d.). The TNL measures story comprehension and story narration. It includes story elicitation and comprehension tasks in three formats: (1) with no picture cues, (2) with sequenced pictures, and (3) with a single picture. For monolingual English speakers, sensitivity for the TNL is .92 and specificity is .87. The examiners manual indicates that internal consistency is good with alpha levels of .76 for narrative comprehension subtest scores and 0.87 for oral narration subtest scores In a study of bilingual Spanish-English bilingual first graders the TNL English had a sensitivity of .86 and a specificity between .76 (comprehension) and .78 (oral narration) with a cut score −1.33 SD from the mean (Gillam, Peña, Bedore, Bohman, et al., 2013).

The Spanish TNL (TNL-S) contains stories that are parallel in structure and type to the English version (i.e., 3 comprehension and 3 oral narration tasks). Preliminary normative data is based on 216 children between the ages of 6;0 (years; months) and 9;11. Means and standard deviations were derived for each age group (by years) and standard scores were calculated for each age. Preliminary analyses indicate an alpha level of .888 for narrative comprehension subtest scores and 0.931 for oral narration subtest scores. Preliminary analysis of a subset of 90 children demonstrates that on average, those with TD (raw story score M = 8.61) score significantly higher on the subtests of the TNL-S compared to those with DLD (raw story score M = 4.44). In both languages, standard administration and scoring procedures were followed yielding a standard score, the Narrative Language Ability Index (NLAI) for each language.

Procedure

Participants were screened at grades 1 and 3 using the BESOS by a team of bilingual examiners including certified speech-language pathologists (SLPs) and trained research assistants. Research assistants held at least a B.A. degree in communication sciences and disorders or related degree, and were fluent Spanish-English bilinguals. Members of the team were all trained on test administration including elicitation of responses and scoring. Certified SLPs supervised administration of the screener to ensure fidelity.

Participants who entered the longitudinal study completed a battery of testing in the year following screening when they were in second and fourth grade. Among the battery of measures, they completed the BESAME and the TNL in English and Spanish. Parents and teachers were interviewed using the BIOS and the ITALK. As before, these measures were administered by bilingual SLPs and by trained research assistants.

Identification of Risk for DLD

Table 1 displays the group and grade means for the reference measures. We classified children as at risk for developmental language disorder (DLD) if they received a standard score below 85 on the average of the following four risk indicators: (a) best BESOS morphosyntax score, (b) best BESOS semantics score, (c) highest standardized parent concern rating, and (d) highest standardized teacher concern rating. “Best” scores represented the score from the language (either English or Spanish) in which the child tested higher on that BESOS measure. “Highest” concern ratings represented the language (either English or Spanish) in which the parents and teachers scored children’s abilities the highest.

Results

Analysis

We aimed to assess the classification accuracy of the BESA-ME morphosyntax subtest, BESA-ME semantics subtest, and TNL. We used receiver operating characteristic (ROC) curves to determine optimal cut-point values when analyzing children’s English scores, Spanish scores, or the better of the two scores (i.e., “best” score) for each of the above three measures. The optimal cut-point values represented the scores at which sensitivity and specificity were maximized. Using these values, we calculated positive likelihood ratios (LR+; the likelihood that an DLD child was correctly classified as DLD divided by the likelihood that a TD child was incorrectly classified as DLD) and negative likelihood ratios (LR−; the likelihood that an AR child was incorrectly classified as TD divided by the likelihood that a TD child was correctly classified as TD). The ROC analyses also estimated the area under the curve (AUC), a measurement indicating the probability that a child drawn at random from the TD group would rank higher than a child drawn at random from the AR group. AUC values between .70 and .80 are considered acceptable, values between .80 and .90 are considered excellent, and values above .90 are considered outstanding (Rice & Harris, 2005). Finally, we combined the best discriminator (English, Spanish, or best) for each of the three measures (BESA-ME morphosyntax, BESA-ME semantics, and TNL) in two exploratory discriminant function analyses, one for each grade level.

Second Grade

Table 2 shows the ROC results and classification accuracy for the second grade TD and DLD groups. The highest AUC for each measure was from the best language scores of the BESA-ME morphosyntax subtest (AUC = .95, p < .001); BESA-ME semantics subtest (AUC = .83, p < .001); and TNL (AUC = .85, p < .001). These AUCs are in the excellent to outstanding range (see Figure 1, upper panel).

Table 2.

Receiver Operating Characteristic (ROC) Curve Results – Second Grade

	BESA-ME Morphosyntax			BESA-ME Semantics			TNL
	English	Spanish	Best	English	Spanish	Best	English	Spanish	Best
AUC	.86***	.89***	.95***	.78***	.81***	.83***	.78***	.80***	.85***
Standard Error	.04	.03	.03	.06	.04	.05	.05	.05	.04
Asymptotic 95% CI	[.77, .95]	[.82, .95]	[.89, 1.00]	[.67, .89]	[.72, .89]	[.73, .93]	[.69, .89]	[.71, .89]	[.76, .93]
Optimal Cut-Point	62.5	64.5	80.0	93.0	87.0	100.5	68.5	89.5	93.0
Sensitivity  (True Positive)	.84 (16/19)	.84 (16/19)	.90 (17/19)	.74 (14/19)	.79 (15/19)	.79 (15/19)	.68 (13/19)	.74 (14/19)	.84 (16/19)
(False Negative)	(3/19)	(3/19)	(2/19)	(5/19)	(4/19)	(4/19)	(6/19)	(7/19)	(3/19)
Specificity  (True Negative)	.80 (74/92)	.87 (80/92)	.89 (82/92)	.73 (66/91)	.70 (64/91)	.71 (65/91)	.81 (72/89)	.72 (64/89)	.73 (65/89)
(False Positive)	(18/92)	(12/92)	(10/92)	(25/91)	(27/91)	(26/91)	(17/89)	(25/89)	(24/89)
Positive Likelihood Ratio	4.20	6.46	8.18	2.74	2.63	2.72	3.58	2.64	3.11
Negative Likelihood Ratio	.20	.18	.11	.36	.30	.30	.40	.36	.22

Note. AUC = Area Under the Curve; CI = Confidence Interval; BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language.

^***p < 0.001.

Figure 1.

Receiver Operating Characteristic (ROC) Curve Results by Test and Grade. BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language

We entered these three measures in a discriminant function analysis. Equality of covariance matrices between the TD and DLD groups as tested by Box’s M was met (p = .870) and the log determinants were similar (TD = 15.302, DLD = 15.548). The chi-square test was significant, Wilks’ λ = .533, χ² = 72.874, df = 3, canonical correlation = .702, p < .001. These results indicated that there was a significant association between groups that explained 49.23% of the between-groups variability. The combination of measures classified 89.23% of children correctly and yielded a sensitivity of .89, specificity of .89, signifying high classification accuracy. The LR+ of 8.23 is suggestive to likely that a positive result is true (Dollaghan, 2007). The LR− of .12 indicates that a negative result is very to almost certainly true. The standardized discriminant coefficients show the unique contributions of each variable to the discriminant function. Using .3 as a cut off, results (see Table 3) indicated that the best language BESA-ME morphosyntax was the strongest predictor at .979, accounting for most of the variance. Independently, all three predictors were positively and significantly correlated to the discriminant function.

Table 3.

Correlations of Predictor Variables of the Discriminant Functions at Second Grade

	Structure Matrix	Standardized Canonical Discriminant Function Coefficients
BESA-ME Morphosyntaxa	0.988	0.979
TNLa	0.550	0.179
BESA-ME Semanticsa	0.529	−0.124

Note. BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language.

^aBest language.

Fourth Grade

Table 4 shows the ROC results and classification accuracy for the fourth grade TD and DLD groups. For this analysis, there were inconsistencies between the AUC and highest classification accuracy when Spanish, English, or best language were compared. This result is likely due to the small number of children in the DLD group. We thus ran a stepwise discriminant function analysis for each of the measures (BESA-ME morphosyntax, BESA-ME semantics, and TNL) to determine which of the three options (Spanish, English, or best language) contributed most to the discrimination between groups. Results of the stepwise discriminant analysis are displayed in Table 5. Results indicate that for each of the three analyses, the best language score accounted for all of the variance in the discriminant function.

Table 4.

Receiver Operating Characteristic (ROC) Curve Results – Fourth Grade

	BESA-ME Morphosyntax			BESA-ME Semantics			TNL
	English	Spanish	Best	English	Spanish	Best	English	Spanish	Best
AUC	.82**	.76*	.85**	.73*	.80*	.74*	.81**	.73*	.80**
Standard Error	.06	.09	.06	.13	.06	.13	.09	.09	.09
Asymptotic 95% CI	[.70, .94]	[.59, .93]	[.73, .97]	[.48, .98]	[.68, .92]	[.49, .99]	[.64, .98]	[.55, .91]	[.62, .99]
Optimal Cut-Point	90.0	65.5	89.5	108.5	82.0	108.5	80.5	90.0	90.0
Sensitivity  (True Positive)	.86 (6/7)	.86 (6/7)	.71 (5/7)	.71 (5/7)	.86 (6/7)	.71 (5/7)	.86 (6/7)	.71 (5/7)	.71 (5/7)
(False Negative)	(1/7)	(1/6)	(2/7)	(2/7)	(1/7)	(2/7)	(1/7)	(2/7)	(2/7)
Specificity  (True Negative)	.74 (42/57)	.75 (43/57)	.77 (44/57)	.67 (38/57)	.72 (41/57)	.68 (39/57)	.59 (33/56)	.75 (42/56)	.84 (47/56)
(False Positive)	(15/57)	(14/57)	(13/57)	(19/57)	(16/57)	(18/57)	(23/56)	(14/56)	(9/56)
Positive Likelihood Ratio	3.31	3.44	3.09	2.15	3.07	2.22	2.1	2.84	4.44
Negative Likelihood Ratio	.19	.19	.38	.43	.19	.43	.24	.39	.35

Note. AUC = Area Under the Curve; BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language.

^*p < .05

^**p < .01.

Table 5.

Comparison of Predictor Variables of Stepwise Discriminant Functions for Morphosyntax, Semantics, and TNL at Fourth Grade

	Structure Matrix			Standardized Canonical Discriminant Function Coefficients
	English	Spanish	Best language	English	Spanish	Best language
BESA-ME Morphosyntax	0.924	0.301	1.00	N/A	N/A	1.00
BESA-ME Semantics	0.969	0.283	1.00	N/A	N/A	1.00
TNL	0.501	0.794	1.00	N/A	N/A	1.00

Note. BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language; N/A = did not enter into the discriminant function

We then entered these three measures into a discriminant function analysis. Equality of covariance matrices between the TD and DLD groups as tested by Box’s M was met (p = .135) and the log determinants were similar (TD = 14.831, DLD = 16.519). The chi-square test was significant, Wilks’ λ = .768, χ² = 15.992, df = 3, canonical correlation = .482, p < .001. There was a significant association between groups that explained 23.23% of the between-groups variability. The combination of measures classified 78.1% of children correctly and yielded a sensitivity of .77.2 and specificity of .85.7 indicating fair classification accuracy. A LR+ of 3.76 indicates that a positive result is a suggestive but not a certain indicator of DLD, and a LR− of .19 indicates that a negative result is highly likely to be true. Using .3 as a cut off, results (see Table 6) followed by the best language TNL score at .532. Independently, all three predictors were positively and significantly correlated to the discriminant function.

Table 6.

Correlations of Predictor Variables of the Discriminant Functions at Fourth Grade

	Structure Matrix	Standardized Canonical Discriminant Function Coefficients
BESA-ME Morphosyntaxa	0.839	0.629
BESA-ME SemanticsA	0.702	0.162
TNLA	0.615	0.532

Note. BESA-ME = Bilingual English-Spanish Assessment – Middle Extension; TNL = Test of Narrative Language.

^aBest language.

Discussion

In the current study, we sought to explore methods for determination of DLD in school-age Spanish-English bilingual children with a range of exposure to each language. There are some available procedures for making this determination for younger children (specifically preschool through first grade) but almost none for older elementary age children. Additionally, while current recommendations for assessment of bilingual children are to test in both languages there continues to be a lack of empirically derived methods for combining test results across both languages. We evaluated bilingual children’s performance in second and fourth grade in order to extend what it is we know about the diagnostic accuracy of measures beyond preschool and early elementary age. Our target measures of morphosyntax, semantics, and narrative had both Spanish and English versions. This allowed us to compare classification by language, as well as evaluating the contribution of the three best measures together.

Using Best Language Scores

We were interested in obtaining information on whether each language or a best language score would yield higher classification on individual measures for bilingual children in second and fourth grades. For bilingual children in second and fourth grade, the best language score (either the higher English or higher Spanish score determined individually) accounted for the most AUC for all three measures (second grade) or those that accounted for the greatest variance (fourth grade). These findings are consistent with previous studies examining semantics using the BESA (Peña, Bedore, & Kester, 2016). In this study, we examined preschool, kindergarten, and first grade bilingual children and found that using the higher BESA semantics score (when comparing standard scores across Spanish and English) yielded higher classification accuracy than Spanish alone or English alone. The current study replicates this finding and extends it to evaluation of narratives and morphosyntax.

Combining Test Tasks

In evaluation for DLD, clinicians must combine test results to make a clinical diagnostic decision. There are few guidelines for combining test results across languages. From previous work (Bedore, Peña, Summers, et al., 2012; Bedore, Peña, Gillam, et al., 2010; Lugo-Neris et al., 2015) it is known that preschool and early school age children present with “mixed” dominance profiles. That is they may demonstrate higher scores in one language in morphosyntax, and higher scores in their other language in semantics. The test manual of the BESA guides clinicians to combine the highest score across each domain (phonology, semantics, and morphosyntax) to yield a language composite. This language composite provides the best classification accuracy on this measure for children between 4;0 and 6;11. In this study, we extend these findings to older children. Specifically, we examine how to combine assessment results in the domains of semantics, morphosyntax, and narratives in second and fourth grade children. We found that the best language scores for each domain combined to yield good classification accuracy.

For second graders, results of morphosyntax testing accounted for the most variance in the discriminant function. For the fourth grade children however, morphosyntax and narratives together accounted for most of the variance. The findings that morphosyntax contributed significantly to the variance at both ages is consistent with the notion that morphosyntax is a robust of clinical markers of DLD. Narrative comprehension and use, as systematically assessed via the TNL, is demanding in that it requires the child to demonstrate comprehension of and use precise vocabulary and syntax in connected discourse in order to achieve high levels of performance. It may be that by grade 4 the differences between children with and without DLD to integrate these skills helps to further differentiate them.

To better understand the patterns of best language by domain, we examined the extent to which children in the study demonstrated higher performance in one language or the other across the three tasks. Across age, approximately two-thirds of the children demonstrated a mixed dominance profile. At the individual level 64% of the second graders, and 69% of the fourth graders had one score that was better in one language, while their other two scores were better in the other language. In second grade 25% of the children performed higher in Spanish across all three domains, and 11% of the children performed higher in English. These patterns were inverted for the older children. Here, 22% scored higher across the board in English, while 9% demonstrated higher scores in the three domains in Spanish. This pattern of mixed dominance is consistent with previous findings for younger children (Bedore Peña, Summers, et al., 2012; Bedore, Peña, Gillam, et al., 2010; Bedore, Peña, Griffin, et al., 2016; Lugo-Neris, et al, 2015). This demonstrates the importance of assessment in both languages and comparing performance measure by measure, not assuming that dominance will be consistent across specific linguistic areas. This also validates the use of a bilingual coordinate approach for clinical decision-making (Anaya et al., 2016).

Implications for Selecting Assessment Measures

Comparison of findings across the two grades has important implications for selection of appropriate measures in differential diagnosis and highlight the importance of assessment in both languages. For the second graders, the best language BESA-ME morphosyntax was the strongest predictor, and the best language TNL and BESA-ME Semantics were weaker predictors. This means that in early elementary grades, bilingual children with DLD may have most difficulty with morphosyntactic structures. Selection of assessment measures that do not tap into morphosyntax or rely on semantics or narrative structure alone may result in underidentification. It is important to note that all three predictors were strongly correlated with the discriminant function, suggesting that all three measures provide important and clinically valid information.

In the fourth grade group, best language BESA-ME Morphosyntax and best language TNL together accounted for most of the variance in the discriminant function. As with the younger children, all three measures were significantly associated with DLD. For these older children, tasks that include morphosyntax in addition to examining the ability to form cohesive and grammatically accurate narratives will help to make a determination of DLD.

Limitations

The current study provides preliminary insights about what kinds of measures might work to differentiate bilingual children with and without DLD. We expected that the best scores for each measure would maximally differentiate children with risk and this was borne out in the study. While we found good classification accuracy with a combination of the three measures for the second grade children, but classification accuracy for the fourth grade children was less accurate. This reduced accuracy may be due to the small n particularly for fourth grade children with DLD, limiting generalizability of the findings.

Future Directions

These findings highlight the importance of assessment in both languages. Examination of scores relative to children’s level of exposure by domain may help to elucidate how exposure and DLD status influences performance, particularly for children the majority of whom consistent with previous reports present with mixed-dominance profiles across different language domains. In addition, the BESA-ME and TNL-S are still in development. Item analysis will be used to select items that best differentiate children with and without DLD that maximizes classification for bilinguals in this age range. Nevertheless, the fair to good classification accuracy for these measures holds promise for informing clinical diagnostic decision making.