Neural processing of speech in children is influenced by bilingual experience

Abstract

Language experience fine-tunes how the auditory system processes sound. For example, bilinguals, relative to monolinguals, have more robust evoked responses to speech that manifest as stronger neural encoding of the fundamental frequency (F0) and greater across-trial consistency. However, it is unknown whether such enhancements increase with increasing second language experience. We predict that F0 amplitude and neural consistency scale with dual-language experience during childhood, such that more years of bilingual experience leads to more robust F0 encoding and greater neural consistency. To test this hypothesis, we recorded auditory brainstem responses to the synthesized syllables ‘ba’ and ‘ga’ in two groups of bilingual children who were matched for age at test (8.4+/−0.67 years) but differed in their age of second language acquisition. One group learned English and Spanish simultaneously from birth (n=13), while the second group learned the two languages sequentially (n=15), spending on average their first four years as monolingual Spanish speakers. We find that simultaneous bilinguals have a larger F0 response to ‘ba’ and ‘ga’ and a more consistent response to ‘ba’ compared to sequential bilinguals. We also demonstrate that these neural enhancements positively relate with years of bilingual experience. These findings support the notion that bilingualism enhances subcortical auditory processing.

Introduction

Acquisition of a second language enhances how sound is processed both cortically [19] and subcortically [14]. While bilingualism’s influence on cortical areas has been extensively evaluated [see 5 for a review] its effects on subcortical auditory processing is a recent topic. Subcortical assessments have revealed that bilingual adolescents demonstrate greater across-trial neural consistency and encode the fundamental frequency (F0) of speech more robustly than monolinguals [14, 16]; however, whether the degree of these enhancements is dependent on the extent of second language experience is unknown. The subcortical auditory response of newborns is not biased to the native language of their parents [10] but young adults show enhanced subcortical processing of native language features [10, 13], implicating an emergence of spoken-language dependent tuning of the auditory brainstem during childhood. Therefore, we hypothesize that second language learning during childhood leads to additional structural and/or functional changes in the neural circuitry underlying auditory communication, with the amount of plasticity being commensurate with the amount of bilingual experience. This leads to the prediction that among age-matched bilinguals, the encoding of the F0 of speech and the consistency of the response will be greater in children who learned their second language earlier in life. To test this prediction, the current study compared F0 encoding strength and response consistency across two groups of bilingual children who differed in their age of second language acquisition.

Methods

Participants

Electrophysiological responses to the synthesized syllables ‘ba’ and ‘ga’ were collected in 27 school-aged children (8.4+/−0.67 years, 17 female) recruited from Los Angeles, California. All children were Spanish-English bilinguals from predominately low socioeconomic backgrounds as measured by maternal education, which has previously been used as a reliable index of socioeconomic status in children [26, 28] (high-school or less: n=23; some college or beyond: n=4). Two sequential bilinguals were born outside the United States (Honduras, Mexico) and moved to the U.S. at age 3. All participants had normal hearing (< 20 dB HL at octaves ranging from 125 Hz to 8000 Hz, ANSI, 2009) and normal click-evoked auditory brainstem responses based on lab-internal normative data [25] (80 dB SPL, 31.1/s). All participants had normal IQ (simultaneous=102.83+12.7, sequential=96.67+12.2, t = 1.278, p=0.213, Wechsler Abbreviated Scale of Intelligence, WASI), were right-handed, and had no reported diagnosis of language, learning, neurological or attention impairment. Parental ratings of language proficiency and language exposure have previously been found to be reliable measures of a child’s first and second language experience [3, 8]; and so, parental ratings of a child’s language knowledge and exposure were used in the current study. Based on these parental reports, all participants were rated to be highly proficient in speaking and understanding Spanish and English on a scale of 1 (lowest) to 10 (highest).

Using standard grouping criteria to define the participants as simultaneous or sequential bilinguals [e.g., see 2, 24], the children were divided into two groups based on the parental report of when the child began learning English. The simultaneous group (n=13, 10 female, age 8.36+0.53 years, bilingual experience 8.3+0.64 years) comprised children exposed to both English and Spanish in the home since birth. Children in the sequential group (n=15, 7 female, age 8.44+0.79 years, bilingual experience 4.24+1.1 years) were exposed to Spanish since birth but did not begin learning English until pre-school or Kindergarten (mean age of English exposure=4.1 years). For each child, extent of bilingual experience was quantified by subtracting the child’s age of English acquisition from age at test. Parent ratings of English and Spanish proficiency were matched between the simultaneous and sequential groups (English: simultaneous=9.88+0.3, sequential=9.50+0.9, t=1.603, p=0.121; Spanish: simultaneous=7.80+2.0, sequential=7.80+2.3, t=−0.038, p=0.97). The two groups were sex- (t(26)=1.656, p=0.110) and age-matched (t(26)=−0.332, p=0.742), however, given their influence on the cABR [15, 25] all analyses were run co-varying for both factors. Prior to testing, all participants provided English informed assent and parents gave informed consent in their preferred language. All procedures were approved by the Internal Review Board of Northwestern University.

Stimuli

The syllables ‘ba’ and ‘ga’ were synthesized with a Klatt-based synthesizer (Klatt 1980). Each syllable is 170 ms, consisting of an initial stop-consonant burst followed by a 50 ms transition between the burst and sustained vowel. During the transition the first, second, and third formants linearly change (F1=400–720 Hz; F2(ba)=900–1240 Hz; F2(ga)=2480–1240 Hz; F3=2580–2500 Hz) while the fundamental frequency (F0), fourth, fifth, and sixth formants remain level (F0=100 Hz, F4=3300 Hz; F5=3750 Hz; F6 =4900 Hz). The F0 and formants are constant during the vowel (50–170 ms). These syllables were constructed to be neither Spanish-like nor English-like, but to minimally differ in the acoustic properties that distinguish them as ‘ba’ or ‘ga’ (i.e., F2 trajectory during the transition). These phonemes were chosen because they are present in both Spanish and English [32] allowing us to focus on how bilingual experience modulates the processing of sounds that are common to both languages. Moreover, we selected two syllables, instead of just one, to assess the generalizability of the bilingual neural enhancement across stimuli.

Electrophysiological recording

Subcortical electrophysiological responses were recorded using the SmartEP cABR module (Intelligent Hearing Systems). During the recording, the child sat in a comfortable chair and watched a movie in English on a portable DVD player (Sony Corporation, Minato, Tokyo, Japan). cABRs were collected using three Ag/AgCl electrodes applied in a vertical montage (CZ–active, right ear–reference, forehead–ground). Stimuli were presented in alternating blocks (i.e., ‘ba’, ‘ga’, ‘ba’, ‘ga’ or ‘ga’, ‘ba’, ‘ga’, ‘ba’) to the participant’s right ear through an insert earphone at 4.35 Hz (60 ms interstimulus interval) and 80 dB SPL. The left ear remained unoccluded so the participant could hear the movie soundtrack at a level that did not mask the stimulus (< 40 dB SPL). For each stimulus, 6000 responses were collected over two 3000-trial blocks (1500 of each stimulus polarity). Responses were digitized at 13,333 Hz, and filtered from 50–3000 Hz (6 dB/octave roll off). Epoching (−40ms to 190ms), artifact rejection (+35μV), and averaging were performed on-line.

Analyses

Spectral Encoding

In MATLAB (Mathworks, Inc.) a fast-Fourier transform was performed for the formant transition (20–60 ms) and steady-state response (60–180 ms) and average spectral amplitudes were calculated over 40-Hz wide frequency bins, centered on the stimulus F0 (100 Hz) and harmonics H2–H10 (200–1000 Hz). A composite of harmonic amplitude was calculated by averaging H2-H10 [22]. Spectral amplitudes over the formant transition and vowel were analyzed using a 2 (language group: simultaneous, sequential) x 2 (stimulus: ‘ba’, ‘ga’) x 2 (frequency range: F0, harmonics) repeated measures ANOVA covarying for sex and age (RMANCOVA). Significant interactions were explored using independent-samples t-tests.

Response Consistency

Consistency was calculated for each stimulus over the formant transition (20–60 ms) and vowel (60–180 ms) by correlating an average of the first 3000 trials (i.e., block 1) to an average of the last 3000 trials [i.e., block 2; 9, 16]. An r value of 1 indicates perfect morphological consistency between the blocks while an r-value of 0 represents no consistency. R-values are used for graphical purposes and to report group means, but were normalized by a Fisher z-transform for statistical analyses. Consistency of the response to the formant transition and vowel were analyzed using a 2 (language group: simultaneous, sequential) x 2 (stimulus: ‘ba’, ‘ga’) RMANCOVA. Significant interactions were explored using independent-samples t-tests.

Results

Simultaneous bilinguals had a larger evoked response to the F0 and tended to have greater across-trial consistency that was specific to the response to the vowel relative to the sequential bilinguals. Across groups, these neural measures related to amount of bilingual experience.

Spectral Encoding

The groups differed in their F0, but not harmonic, encoding in response to the vowel (60–180 ms) of both stimuli (Fig. 1A,B RMANCOVA statistics and effect sizes in Table 1; post-hoc t-test for ‘ba’ F0: t(26)=2.916, p=0.007; ‘ga’ F0: t(26)=2.771, p=0.01; ‘ba’ and ‘ga’ harmonics p’s > 0.05). The mean F0 amplitude for the simultaneous bilinguals was 0.064μV + 0.029μV (‘ba’) and 0.065μV + 0.026μV (‘ga’). For sequential bilinguals, the mean F0 amplitude was 0.041μV + 0.012μV (‘ba’) and 0.044μV + 0.012μV (‘ga’). The two groups did not differ in F0 or harmonic encoding during the formant transition (all p’s > 0.1, Table 1).

Figure 1.

Average neural responses, spectral amplitude, and response consistency comparisons between simultaneous and sequential bilinguals. The evoked brainstem responses to the syllables ‘ba’ (A) and ‘ga’ (B) are plotted in the time domain (Ai, Bi) and frequency domain (transition (20–60 ms): Aii, Bii, steady-state (60–180 ms): Aiv, Biv). Simultaneous bilinguals (black) show a larger representation of the fundamental frequency (F0, 100 Hz) of the vowel of ‘ba’ and ‘ga’ relative to the sequential bilinguals (gray). Neural response consistency to ‘ba’ in the transition (Aiii) and steady-state (Av) and ‘ga’ for the transition (Biii) and steady-state (Bv) are also plotted. The simultaneous bilinguals (black) have greater across-trial consistency in response to the vowel of ‘ba’, but not ‘ga’, relative to sequential bilinguals (gray), but the groups do not differ over the formant transition of either syllable. C) Relations between neural processing and years of bilingual experience. Response consistency for ‘ba’ (Ci) and ‘ga’ (Cii) are plotted on the y-axis, with years of bilingual experience for the simultaneous (black) and sequential (gray) bilinguals plotted on the x-axis. The consistency to ‘ba’ relates with years of second language experience, while the consistency to ‘ga’ does not. F0 encoding for ‘ba’ (Ciii) and ‘ga’ (Civ) are plotted on the y-axis, with years of bilingual experience plotted on the x-axis. Both measures of F0 encoding relate to the number of years of experience the child has speaking two languages.

Table 1.

RMANOVA statistics for main effects and interactions. For spectral amplitudes, the RMANOVA compared language group language group (simultaneous v. sequential bilinguals), stimulus (‘ba’ v. ‘ga’), and frequency range (F0 v. harmonics) in response to the formant transition and the steady-state vowel. The two groups did not differ in their encoding of the formant transition, but did differ in their encoding of the F0 of both ‘ba’ and ‘ga’ over the vowel. For response consistency, the RMANOVA compared language group (simultaneous v. sequential bilinguals) and stimulus (‘ba’ v. ‘ga’) in response to the formant transition and the steady-state vowel. Although there was a significant stimulus x group interaction in the consistency with which the stimuli were encoded during the formant transition, post hoc tests demonstrated no group differences. In response to the vowel, the simultaneous group tended to have more consistent responses than the sequential bilingual, especially in response to the vowel of ‘ba’.

Spectral Amplitudes		F	p	η2
Formant Transition	Language Group	2.265	0.145	0.086
	Stimulus	2.016	0.168	0.077
	Frequency Range	0.257	0.617	0.011
	Stimulus x Language Group	1.942	0.176	0.075
	Frequency Range x Language Group	1.258	0.273	0.05
	Stimulus x Frequency Range	1.596	0.219	0.062
	Stimulus x Frequency Range x Language Group	2.534	0.124	0.096
Vowel	Language Group	11.16	0.003***	0.317
	Stimulus	0.04	0.843	0.002
	Frequency Range	0.017	0.896	0.001
	Stimulus x Language Group	0.441	0.513	0.018
	Frequency Range x Language Group	8.955	0.006**	0.272
	Stimulus x Frequency Range	0.04	0.843	0.002
	Stimulus x Frequency Range x Language Group	0.216	0.647	0.009
Response Consistency
Formant Transition	Stimulus	0.028	0.869	0.001
	Language Group	1.424	0.244	0.056
	Stimulus x Language Group	5.933	0.023*	0.198
Vowel	Stimulus	2.118	0.158	0.081
	Language Group	5.349	0.03*	0.182
	Stimulus x Language Group	7.889	0.01**	0.247

Response Consistency

Compared to sequential bilinguals, simultaneous bilinguals had more consistent responses to the vowel ‘a’ in response to ‘ba’ (Fig. 1A,B; RMANCOVA statistics in Table 1, post hoc t-test: t(26)=2.539, p=0.017), but not ‘ga’ (Fig. 1A,B; RMANCOVA statistics in Table 1, post hoc t-test: t(26)=1.176, p=0.25). The mean response consistency for the simultaneous bilinguals was r=0.73+0.15 for ‘ba’ and r=0.71+0.13 for ‘ga’. For the sequential bilinguals, mean response consistency was r=0.59+0.17 for the vowel portion of ‘ba’ and r=0.64+0.17 for ‘ga’. Although there was a significant language group x stimulus interaction (Table 1), the bilingual groups did not differ in response consistency over the formant transition (post-hoc t-test for ‘ba’: t(26)=1.488, p=0.149; ‘ga’: t(26)=−0.152, p=0.880).

Relations to Bilingual Experience

Correlations were run relating years of bilingual experience and the neural measures. F0 encoding positively related to years of bilingual experience for both ‘ba’ (Fig. 1C; r=0.497, p=0.01) and ‘ga’ (Fig. 1C; r=0.489, p=0.011). Consistency of the response to ‘ba’ related to amount of bilingual experience (Fig. 1C; r=0.585, p=0.002), while these two measures were not related for ‘ga’ (Fig. 1C; r=0.199, p=0.329).

Discussion

Previously, we found that adolescent bilinguals had larger and more consistent auditory brainstem responses to speech relative to adolescent monolinguals [14, 16]. Now, when comparing younger bilingual children who differed in their amount of bilingual experience, we find that children who spoke both languages since birth had greater F0 encoding of ‘ba’ and ‘ga’ and more consistent responses to ‘ba’ than age-matched peers who spent half as many years using two languages. Moreover, we find that amount of dual-language experience positively relates to these neural enhancements. Together, these findings support the argument that bilingualism shapes auditory processing to a degree that is commensurate with the child’s amount of bilingual experience.

Enhanced neural consistency has been linked to heightened language-based skills [9]; thus, a bilingual’s more consistent response may provide a platform upon which skills important for second-language abilities, such as enhanced F0 encoding, can develop. Though the role of the F0 in pitch perception, tracking an auditory object, and attending to a target talker in noise make the F0 an essential cue for all listeners to attend to during communication, it may provide additional aid to bilingual speakers. Indeed, bilinguals modulate the F0 of their voice when switching between languages [1], suggesting that the F0 acts as a language cue for an interlocutor when communicating in a bilingual environment. Additionally, that bilinguals, but not monolinguals, use the F0 for phonemic identification [17], offers further evidence of a heightened role of the F0 for bilinguals.

Though the biological mechanisms of bilingualism-driven neural plasticity are not fully understood, mounting evidence suggests that bilingualism increases gray matter density in areas of the brain underlying communication. For example, bilinguals, relative to monolinguals, have larger Heschl’s gyrus volume [23] and higher gray matter density in the left inferior parietal cortex [20]. This latter finding correlates with age of second-language acquisition, where individuals who acquired their second language at an earlier age have higher gray matter density [20]. Additionally, in bilinguals, gray matter density in the left pars opercularis (i.e., left inferior frontal gyrus comprising Broca’s area) positively relates to lexical efficiency in a second language [7]. Therefore, one possible mechanism underlying bilingual enhancements in subcortical auditory processing is increased gray matter density in the generators of this evoked response. Support for this comes from work showing that speech-sign interpreters have enhanced gray matter density in the inferior colliculus, the putative generator of the responses recorded in the current study [4], relative to non-interpreters [6]. Bilingual enhancements in gray matter density in the inferior colliculus could foster greater neural consistency and enhance the encoding of important features of the signal, such as the F0. This imprinting of bilingualism on subcortical auditory function may be maximized when second language experience occurs during childhood.

Early childhood has been described as a sensitive period, or developmental time window when learning has a maximal influence on neural function [31]. Although the auditory brainstem had been described as mature by age three [e.g., see 11], it has recently been shown that subcortical auditory processing is enhanced during childhood (~5–11 years) compared to adults [25]. This enhancement suggests there is a sensitive period for auditory brainstem processing during childhood (likely resulting from an overshoot in gray matter density [12]), which endows heightened neural plasticity [25]. Therefore, if childhood represents a period of heightened auditory system plasticity, then the subcortical enhancements in F0 encoding and response consistency seen in bilinguals who learned their second language early in life may not extend to second language learning that occurs outside of this period. While longitudinal studies have found that gray matter volume in the left inferior frontal gyrus [21, 27] and cortical thickness in the superior temporal gyrus [18] increase as one learns a second language during adulthood, it remains to be determined whether subcortical neuroplasticity in older second language learners can reach the same level seen in early bilinguals. Future studies should compare early and late bilinguals to determine whether second-language learning later in life produces similar neural profiles to those demonstrated here in bilingual children. In doing so, the role of age of acquisition could be better dissociated from the influence of number of years of bilingual experience on these physiological enhancements.

In our previous work on bilingualism we tested adolescents [14, 16] who were early, high-proficiency bilinguals; and, in this adolescent population, neural differences were not observed between simultaneous and sequential bilinguals. Rather, both types of bilinguals demonstrated enhanced F0 encoding and greater response consistency relative to monolinguals. Moreover, while the neural enhancements in the younger children in the current study relate to amount of experience, for the adolescent bilinguals, the degree of the enhancement tracked with the subject’s dual-language proficiency, not amount of experience. This may suggest that when bilingualism begins early in life there is a saturation point – an age or amount of time over which the sequential bilinguals “catch up” to the simultaneous bilinguals. After the neural enhancements develop, maintenance of this enhancement may then become dependent on continued use and increased proficiency of the two languages. Though the differences in F0 encoding and ‘ba’ response consistency between our simultaneous and sequential bilinguals suggest this may be the case, this theory cannot be directly tested with our dataset because (1) children were matched on proficiency in both English and Spanish, and (2) we did not include a monolingual group for comparison. Moreover, comparisons with our existing adolescent monolingual population are not feasible given the effect of age on the cABR [11, 25] and differences in the recording equipment between this study and its predecessors. To further characterize the time course of bilingual neural enhancements and their relationship to both experience and proficiency, future research could track the simultaneous and sequential bilinguals and a group of age- and SES-matched monolinguals longitudinally to determine how the relationship between neural enhancements and both amount of experience and level of proficiency change across development.

Given that both groups in the current study are bilingual, it is not surprising that they did not differ on every measure that comprises the bilingual neural signature. Whereas the sequential bilinguals represented ‘ba’ with less consistency than the simultaneous bilinguals, both groups demonstrated high consistency in their response to ‘ga’. This raises the possibility that that the sequential group may have once (i.e., at an earlier age point) differed from the simultaneous group on ‘ga’ response consistency but have now caught up to their more experienced bilingual peers. Furthermore, given that the simultaneous and sequential bilinguals are matched on the consistency of their response to ‘ga’ suggests that increased consistency in the auditory response is among the first neural changes to emerge with bilingual experience. Thus, the consistency of the sequential bilingual’s response to ‘ba’ may soon reach the same level as the simultaneous bilinguals. To provide conclusive support for this idea, future research should adopt longitudinal designs to track the emergence of neural enhancements. A second (not mutually exclusive) possibility, is that the lack of a difference between the two groups on ‘ga’ but not ‘ba’ results from the similarity with which ‘ga’ is produced across Spanish and English and the dissimilarity of ‘ba’ across these two languages [32]; however, this explanation is less likely given that the synthesized stimuli were not created to be English-like or Spanish-like.

Just as bilinguals demonstrate specific enhancements relevant for communicating across two languages, experience playing a musical instrument leads to selective enhancements in the neural encoding of auditory features important for musicians to attend to, such as the sound’s timbre [29]. For both musicians and bilinguals, these enhancements depend on the extent of the experience playing music or speaking two languages, as evidenced by correlations between these neural measures and years of music practice [30] and proficiency in [16] or amount of experience communicating in two languages (current study). These relationships between neural variables and extent of experience, and the fact that different types of experience result in different selective enhancements, highlights the influence of experience on neural plasticity.

Conclusions

Bilingual experience during childhood can foster plasticity in the neural encoding of sound. Bilingual children who learned their two languages simultaneously from birth had enhanced encoding of the F0 and more consistent evoked responses to ‘ga’ compared to bilingual children who learned their two languages sequentially. These findings suggest that enhanced F0 encoding and neural consistency emerge with increasing experience communicating in two languages during childhood and support the notion that bilingualism enhances auditory processing of select acoustic aspects of speech. However, confirmation of this interpretation requires that sequential and simultaneous bilinguals be compared to an age-matched and SES-matched monolingual population. Additionally, measurement of dual language experience using objective assessments of language knowledge may further support the notion that these neural enhancements relate to bilingual experience and proficiency. Future research should delineate the time course over which these enhancements take place and further separate the influences of age of acquisition and years of second language experience.

Highlights.

1. Auditory neural plasticity emerges as a child gains bilingual experience

2. Neural consistency and spectral encoding track with amount of bilingual experience

3. Bilingualism enhances auditory processing of select acoustic aspects of speech