Impact of associative word learning on phonotactic processing in 6-month-old infants: A combined EEG and fNIRS study

Abstract

During early language development native phonotactics are acquired in a ‘bottom-up’ fashion, relying on exquisite auditory differentiation skills operational from birth. Since basic lexico-semantic abilities have been demonstrated from 6 months onwards, ‘top-down’ influences on phonotactic learning may complement the extraction of transitional probabilities in phonotactic learning. Such a bidirectional acquisition strategy predicts, that familiarization with (proto)words should affect processing of untrained word-forms of similar phonological structure. We investigated 6-month-old infants undergoing an associative training to establish a pseudoword-pseudoobject link. Comparison between pre- and post-training responses to trained and untrained items allowed investigating training effects. Additionally phonotactic status (50% legal, 50% illegal with regard to German) allowed investigating influences of previous language experience. EEG and functional near-infrared spectroscopy (fNIRS) provided measures of electrophysiological and hemodynamic responses. We find evidence for a robust effect of associative training on pseudoword processing when presented in isolation. This transferred to untrained items. Previous linguistic experience showed a much weaker effect. Taken together the results suggest that sensitivity to phonotactic contrasts is present at 6 months, but that acceptance as lexical candidates is rapidly modulated when word forms following non-native phonotactics become potentially meaningful due to repeated exposure in a semantic context.

1. Introduction

One pillar of linguistic competence is lexico-semantic knowledge. Apart from modality-independent semantic knowledge about objects and concepts this requires a lexicon containing word-forms and a tight link between both representations (Gupta and Tisdale, 2009). While word-forms are generally arbitrary, they show quite specific regularities within a given language, in that languages differ substantially with regard to their phoneme inventory and the way in which phonemes are combined to yield well-formed lexical candidates (Chomsky and Halle, 1965, Trask, 1996). As an example neither the English word/NECK/nor the Slovak word/KRK/have any inherent relation to the respective body part but while the corresponding German word/HALS/could qualify as a lexical candidate in English,/KRK/cannot. It violates the ‘phonological grammar’ (Jacquemot et al., 2003) of English (and German). Such phonotactic constraints regulate lexical access (Vitevitch et al., 1999, Vitevitch, 2003), aid segmentation of the auditory stream (Brent and Cartwright, 1996, McQueen, 1998) and have been shown to play an important role in language acquisition in infants (Friederici and Wessels, 1993, Jusczyk and Aslin, 1995, Jusczyk, 1999, Gervain and Mehler, 2010).

The acquisition of native phonotactic rules can be conceived to follow the trajectory of increasingly sophisticated sensitivity to subtle differences in the auditory input. At birth infants are exquisite ‘differentiators’: they are sensitive to basic acoustic features of speech (Telkemeyer et al., 2011), prefer their mother’s over other female voices (DeCasper and Fifer, 1980) and native over non-native stress patterns (Mehler et al., 1988); newborns attend more to speech than to non-speech sounds (Vouloumanos and Werker, 2007), show greater cerebral activation for forward versus reversed speech (Dehaene-Lambertz et al., 2002, Pena et al., 2003), distinguish phonemes (Mahmoudzadeh et al., 2013) and may even perceive phonotactic universals pertaining to sonority profiles (Gomez et al., 2014). Within the first year of life the endowment with such universal, ‘inborn’ discriminative capacities sets the basis for a gradual perceptual attunement to specific features of the native language(s) (Naatanen et al., 1997). There is ample evidence for such ‘perceptual narrowing’, extending beyond the linguistic domain (for a review see Maurer and Werker, 2014). With regard to phonetic attunement, vowel (Cheour et al., 1998) and consonant (Werker and Tees, 1984) differentiation have been shown to increasingly narrow down to those relevant in the native language between 4 and 10 months. Notably this seems to include both, a decrease in sensitivity to non-native and an increase in sensitivity to native contrasts (Kuhl et al., 2006). Regarding phonotactics behavioral work has shown an impact of native regularities on word segmentation at 9 months (Friederici and Wessels, 1993), but sensitivity to a native/non-native contrast may be present even earlier (Obrig et al., 2010).

In sum, sensitivity to phonotactic and prosodic regularities in the incoming speech stream helps the infant to segment word-forms in a ‘bottom-up’ fashion (Gervain and Mehler, 2010). This supplies lexical candidates to be linked to conceptual knowledge, traditionally assumed to emerge after the 1st birthday. Notably however, increasing lexical knowledge in turn fosters statistical learning of phonotactic frequency distributions in the native language. Recent work suggests that such top-down cues for novel word-form learning may be available quite early. Evidence for ‘true word knowledge’ has been reported at 6 months (Bergelson and Swingley, 2012), and training of word-object pairs may establish ‘proto-words’ already at 3 months (Friedrich and Friederici, 2015). Irrespective of what qualifies as a ‘true word’ (Nazzi and Bertoncini, 2003), even very basic lexical knowledge can support segmentation in a ‘top-down’ fashion at 6 months (Bortfeld et al., 2005). Instead of assuming acquisition of substantial word-form knowledge a prerequisite to proceed to lexico-semantic acquisition, it therefore seems more intuitive to postulate mutual interaction between both capacities during language acquisition. As yet work on the interaction between ‘bottom-up’ statistical phonological learning and the influence of emerging lexical knowledge on the acquisition of native phonotactic constraints is scarce (Swingley and Aslin, 2007, Yeung et al., 2014). The present study targets this as a part of a larger project1 investigating different age groups and different training scenarios.

Proceeding from previous work in adults (Rossi et al., 2011a, Rossi et al., 2013) we here investigate how training impacts on phonotactic processing in 6-month-old infants. Specifically we ask to what extent establishing an associative link between novel objects and novel word-forms will modulate processing of trained word-forms, and will generalize to untrained items with the same phonotactic structure. To this end we measured electrophysiological and hemodynamic responses to monosyllabic pseudowords prior to and after a short training. The focus is on three aspects of modulation: (i) During training only half of the pseudowords were repeatedly paired to novel pseudoobjects, establishing an associative word-object link. The comparison of trained versus untrained items yields a measure of generalization: similar response modulation for trained and untrained items would suggest that infants acquired some knowledge on the trained sound structure. (ii) Half of the pseudowords started with consonant clusters attested in the infants’ ambient native language (German) while the other half violated these constraints. This addresses the influence of prior language experience: large differences between the modulation of attested versus unattested pseudowords would suggest that previously acquired phonotactics strongly guide lexical candidate acceptance; conversely no or little differences would speak for a dominant influence of the training-induced associative-semantic link (top-down effect on phonotactic processing). (iii) The experiment was repeated on 3 consecutive days. The assessment of short- and long-term modulation (pre-/post-training and day1/day2/day3), taps into the question, to what extent comparatively brief familiarization with linguistic regularities serves as a basis for emerging word knowledge. Previous research on ‘proto-word’ learning suggests very brittle overnight retention at 6 months (Friedrich and Friederici, 2011). Therefore we address the question whether more extensive training over three days targeting phonological structure may show effects of more stable retention.

2. Materials and methods

2.1. Participants

Fourty 6-month-old healthy infants from monolingual German families with no exposure to Slavic languages participated in the present study. Recruited from a databank of the Institute2 and by advertisement, infants showed the following demographics providing the mean and [range]: age: 181 days [168–194]; gestational age: 38–42 weeks; birth length 50.6 cm [48–55]; birth weight: 3576.8 g [2794–4460]; head circumference at birth: 34.7 cm [32–39.5]. Infants with known neurological, visual, acoustic, or developmental disorders were not included. After recruitment, data from some infants were excluded due to inability to participate in all session on 3 consecutive days or high degree of left-handedness and/or speech disorders in the family reported in a parental questionnaire. If data were overly contaminated by motion or technical artifacts in one modality (EEG or fNIRS) the respective data were not included in the analysis. After these exclusions data of 36 infants (21 female) entered the fNIRS data analyses and those of 25 infants (13 female) the EEG analyses. All infants included in the EEG analysis were also part of the fNIRS-analysis cohort.

2.2. Experimental material

The Material comprised (i) audio traces of spoken pseudo-words and (ii) visually presented pictures. (i) The 36 pseudowords were spoken by a German/Slovak early bilingual female speaker with no foreign accent in either language in a soundproof booth and recorded digitally with 16 bits at a sampling rate of 44 kHz in infant-directed-speech mode (IDS). IDS is characterized by an exaggerated pitch, longer duration, and high phonological clarity (Soderstrom, 2007). All pseudowords were monosyllabic and had a consonant-consonant-vowel-consonant (CCVC) structure. With regard to the reference language (German) 18 of these were phonotactically legal (e.g.,/brop/), while the other 18 were phonotactically illegal (e.g.,/bzop/). Illegal onset clusters are not attested in German but are attested in Slovak, granting natural pronunciation. Slovak was chosen due to its larger repertoire of attested bi-consonant onset clusters compared to German (Hanulikova et al., 2010). Additionally, controlling for the language of the illegal clusters allowed for a priori exclusion of participants exposed to Slavic languages. The auditory material represents a subset of a larger corpus used in previous studies (Rossi et al., 2011a, Rossi et al., 2013). (ii) Visual stimuli consisted of 18 colored pictures of pseudo-objects (stimulus images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition, Carnegie Mellon University, www.tarrlab.org). These ‘fribbles’ are complex, organically shaped, colorful objects, which do not share obvious features with typical depictions of real objects, but could represent toys. Thus the experimental material consists of 36 novel words and 18 novel objects. Apart from the experimental stimuli, silent cartoons without any meaningful content and suitable for infants were used during those parts of the experimental sessions not including image presentations (pre- and post-tests).

2.3. Design

Participants underwent 3 experimental sessions on three consecutive days. Each session comprised a pretest, a training period, and a posttest. During pre- and posttests only the acoustic experimental material (pseudowords) was presented, while the silent movie was played to support the infants alertness (Fig. 1A). The shapes and ‘characters’ in these cartoons shared no features with the pseudo-objects and their appearance/movement was temporally unrelated to the auditory stimuli. Since only half of the pseudowords were trained (Fig. 1B) 4 categories can be differentiated: LEG^untr: legal pseudowords which were not trained during the training phase; LEG^train: legal pseudowords used in the training phase; ILL^untr: illegal untrained items and ILL^train: illegal trained items. Each category included 9 pseudowords which were repeated twice during pre- and posttest. Thus, in total 72 pseudowords were acoustically presented via loudspeakers at a volume of 70 dB.

Fig. 1.

Design of the study. (A) illustrates the pre- and post-test scenario, in which all pseudowords (n = 36) were presented twice in a randomized order (stimulus onset asynchrony of 8 s (mean) with a jitter of 6–10 s). The video on the screen was shown to keep infants attention. (B) During training, performed between pre- and post-test on each day, half of the legal (n = 9) and half of the illegal (n = 9) pseudowords were presented with the pseudoobjects (tight temporal co-occurrence; attention grabber to alert the infant). (C) To establish an associative link during training a given pseudoword was presented 6 times with a specific pseudoobject and 6 times with variable other objects. (D) The analysis was performed for EEG and fNIRS data only during pre- and post-tests. Four factors were analyzed: short term effect of training immediately prior to and after training (PREPOST) and the long term effect over the three consecutive days (DAY); the factor TRAIN differentiates between trained/untrained pseudowords (train/untr) while the factor LEGALITY (not illustrated) differentiates phonotactically legal from illegal pseudowords.

During pre- and posttest pseudowords were presented in an event-related fashion. The mean SOA was 8 s with a jitter of 6–10 s. The design respects the sluggishness of the hemodynamic response and seeks to attenuate contamination by low frequency oscillations in the fNIRS signal (Obrig et al., 2000). Additionally, 8 null events (20 s without stimulus) were inserted in a randomized order. The design represents a compromise between EEG and fNIRS constraints since the former largely requires a high number of trials, while the latter needs to respect constraints of the sluggish and hence temporally overlapping hemodynamic responses.

Training was realized as an associative learning paradigm (Fig. 1C). Pseudoobjects and −words were combined following the principle of statistical learning: a specific pseudoword co-occurred 6 times with the picture of a specific pseudoobject and 6 times with a random selection of 6 other pseudoobjects. Applied to 9 legal and 9 illegal pseudowords (LEG^train and ILL^train), each training comprised an overall 216 trials. This well-established design for novel word learning (Breitenstein and Knecht, 2002) creates an associative semantic relation between a novel word and a novel object broadly mimicking naturally occurring implicit word learning3 Since it is essential that infants attend to the monitor during a pseudoword/-object pairing, a trial started with a ringing tone directing the infants’ attention to the loudspeakers at the monitor. Additionally objects appeared in an animated fashion (for 200 ms) before settling in the center of the screen for an additional 1800 ms. The acoustic presentation of the pseudoword started as soon as the object was static. To attenuate potential sequencing effects, 5 different pseudo-randomizations were created for pretest, training, and posttest respectively.

The design allows for the analysis of 4 different effects: (i) effect of the training session (factor PREPOST) comparing response to all pseudowords prior and after the associative training across all three days; (ii) effect of training, which compares pseudowords only presented during pre- and post-test to those repeatedly presented in the associative training (factor TRAIN, i.e. untrained vs. trained items); (iii) the influence of preexisting phonotactic status comparing pseudowords complying with native (German) phonotactics to those which do not (factor LEGALITY, i.e. legal vs. illegal pseudowords); and (iv) the changes which developed over the 3 consecutive days (factor DAY, i.e. differences between day1 vs. day2 vs. day 3). PREPOST, TRAIN, and DAY are illustrated in Fig. 1D and the statistical model is detailed below (Section 2.5. Data analyses).

2.4. Recording of electrophysiological and hemodynamic response (EEG/fNIRS)

2.4.1. EEG

EEG was recorded with 24 Ag/AgCl active electrodes (Brainproducts, Munich, Germany) placed in a commercially available elastic EEG cap (EasyCap, Herrsching, Germany) at the following positions: F7/8, F3/4, Fz, FT7/8, FC3/4, T7/8, C3/4, Cz, CP3/4, P3/4, and Pz (nomenclature based on Sharbrough et al., 1991) (Fig. 2). The ground electrode was positioned at AFz; online reference was the left mastoid TP9; TP10 (right mastoid) was additionally recorded to be used for off-line re-referencing to averaged bilateral mastoids. The vertical electrooculogram was recorded from FP2 and the horizontal from F10. Impedance was kept below 10 kΩ, and EEG signals were digitized on-line with 1000 Hz, and filtered before digitalization by means of the analog/digital converter with an upper cut-off of 250 Hz (30 db/oct) to prevent aliasing.

Fig. 2.

Layout of the measurement locations. The upper graph shows the 19 electrode positions. 15 of these were used in the further analysis. Fp2 and F10 were used for vEOG and hEOG. ROIs, comprising 2 electrodes were defined for the lateral electrodes. The lower graph shows the layout of the fNIRS probe array. Light-emitter and −detector positions and the resulting sampling volumes are illustrated only on the left hemisphere. From the overall 16 sampling volumes four ROIs were defined on each hemisphere (frontal, fronto-temporal, temporal and temporo-parietal) including averages across 2 NIRS channels. The image on the right shows the co-registration in a 6-month-old participant (parental permission obtained).

2.4.2. fNIRS

Cortical oxygenation changes were measured by near infrared spectroscopy (fNIRS). The method relies on the relative transparency of biological tissue to light in the near-infrared (600–900 nm), and differential absorption of oxygenated and deoxygenated hemoglobin in this spectral range. Based on a modified Beer-Lambert law (Cope et al., 1988) changes in the concentrations of both hemoglobins ([oxy-Hb]/[deoxy-Hb]) can be assessed in a sampling volume reaching to the cerebral cortex. Physiologically the principles of neurovascular coupling predict that enhanced neuronal signaling in a brain region elicits an increase in regional cerebral blood flow which overcompensates the increase in oxygen demand (Fox and Raichle, 1986). Thus the expected fNIRS signature of a cortical activation is an increase in [oxy-Hb] and a decrease in [deoxy-Hb], the latter the major source of BOLD-contrast increases as measured by fMRI (for more details see Obrig and Villringer, 2003, Steinbrink et al., 2006).

We used a NIRScout system (NIRx Medizintechnik GmbH, Berlin, Germany) measuring light attenuation at 760 & 850 nm in a cw-mode with a sampling rate of 12.5 Hz 6 light emitters and 8 light detectors were arranged in two grids covering bilateral fronto-temporo-parietal brain areas. The distance between light emitters and detectors was 2.5 cm. The setup yields 16 measurement volumes (henceforth channels), that is 8 over each hemisphere (Fig. 2).

A modified EEG cap (http://nirx.net/nirscaps/) allowed for simultaneous EEG and fNIRS recordings with comparatively low demands regarding montage on the participant’s head, an example is shown in Fig. 2.

2.5. Data analyses

Data analyses were performed for 6 different time points, that is pre- and posttest on each of the 3 consecutive days (Fig. 1D). Since only half of the pseudowords were trained and these two cohorts were further divided into legal and illegal pseudowords, four conditions result for each of the 6 time points: LEG^train; ILL^train; LEG^untr; ILL^untr. For analysis the resulting 24 values were organized in 4 factors: (i) PREPOST assesses short term modulation between pre- and posttest by semantic training; (ii) DAY investigates long term consolidation from day1 to day2 to day3; (iii) the factor TRAIN captures effects for trained versus untrained items targeting generalization, while (iv) LEGALITY captures the influence of preestablished phonotactic rules. A full factorial 2 × 3 × 2 × 2 ANOVA (PREPOST × DAY × TRAIN × LEGALITY) was computed for each ROI. Next, if the main effect DAY or any interaction reached significance (p < 0.05), subsequent post-hoc t-tests were performed applying a Bonferroni correction. Note that main effects for 2-level factors PREPOST, TRAIN and LEGALITY do not require post-hoc testing. A correction according to Greenhouse and Geisser (1959) was applied and reported as the corrected significance whenever the degrees of freedom exceeded 1 (Table 1). Since we performed separate ANOVAs in each ROI correction for multiple comparisons is required. Bonferroni correction yields a criterion of p < 0.005 for the 9 ROIs of the EEG analysis and p < 0.006 for the 8 ROIs of the fNIRS analysis. For EEG this criterion was only reached for the PREPOST comparison, while for fNIRS the criterion was reached for PREPOST, TRAIN and LEGALITY. For the factor DAY the criterion was not reached in any of the analyses. For a more complete picture we also report on the results using an uncorrected criterion (p < 0.05) in the Results section; these uncorrected results are indicated by the superscript $ in Table 1.

Table 1.

Results of the overall repeated-measures ANOVAs and post-hoc tests. Upper table provides EEG-results in the early- (250–450 ms), mid- (450–650 ms) and late- (650–850 ms) latency windows. Lower table lists the results for fNIRS; changes in [deoxy-Hb] and [oxy-Hb] are listed separately. The factors analyzed were: PREPOST: comparison between pre- and post-test; TRAIN: trained versus untrained items; LEGALITY: differences between legal and illegal pseudowords, and DAY: changes over the three consecutive days of training. Since separate ANOVAs were performed for the different ROIs Bonferroni correction requires a p < 0.005 for the 9 ROIs in the EEG analysis and a p < 0.006 for the 8 ROIs analyzed for fNIRS. Effects surviving this are marked by an asterisk *, those significant only at the uncorrected level (p < 0.05) are indicated by ^$. For illustrations of the main effects and significant post-hoc comparisons see Fig. 3, Fig. 4 (EEG) and Fig. 5, Fig. 6 (fNIRS). The upper row in each cell provides the degrees of freedom, the F- and the p-values of the respective main effect or interaction: (df):F-value/p-value. The lower row indicates the result of the post-hoc testing; n.s. indicates that the interaction was significant but that post-hoc testing showed no effect or did not survive Bonferroni correction (for the 3-level factor DAY).

EEG and fNIRS data capture different aspects of the brain response. ERP-components allow for a temporal analysis in the range of hundreds of milliseconds while fNIRS captures the relatively sluggish hemodynamic response peaking ∼ 5 s after stimulus presentation. Therefore prior to entering the statistical analyses data were differently processed in either modality, as is detailed below.

2.5.1. EEG data analysis

EEG data were filtered offline with a 45-Hz low-pass Butterworth zero-phase filter (high cutoff: 45 Hz; slope: 12 dB/oct). Data were then segmented from −200 ms to 1200 ms with 0 ms representing the onset of the pseudoword. Trials overly contaminated by artifacts were excluded from further analyses. This was performed on a visual inspection and manual rejection of each trial. Infants in whom less than 10 trials per electrode and per condition survived this procedure were not included in the analysis. Similarly if less than 3 sessions or less than 10 out of 19 electrodes were artifact-free, infants were excluded from the final set. In the next step, segments were averaged for each condition and each subject yielding event-related brain potentials (ERPs) with a pre-stimulus baseline of 200 ms. Three time-windows of the ERPs entered the above ANOVAs: 250–450 ms, 450–650 ms, and 650–850 ms (early-, mid-, and late-latency). These were chosen based on literature, on visual inspection of the grand averages, and by a 50-ms-analysis. The latter represents a univariate ANOVA with the factor TIME including the 6 levels pre1, post1, pre2, post2, pre3, and post3 on each electrode and each condition in consecutive 50-ms-windows from 100 to 1000 ms. The analysis yielded significant differences for TIME in consecutive 50ms-windows before 650 and after 650 ms for legal trained, before 450 and after 450 ms for legal untrained, from 650 ms onward for illegal trained, and between 400 and 600 ms for illegal untrained. The choice of time windows therefore relies on visual inspection (see inset of Fig. 3), a statistical analysis, and is in line with previous analysis schemes in comparable approaches (e.g. Kooijman et al., 2013, Friedrich and Friederici, 2015, Friedrich and Friederici, 2005). The final statistical analysis was performed on 6 regions of interest (ROIs). The following ROIs were defined for statistical analyses: left anterior lateral (F7, FT7), right anterior lateral (F8, FT8), left anterior medial (F3, FC3), right anterior medial (F4, FC4), left posterior (CP3, P3), right posterior (CP4, P4) (Fig. 2). Midline electrodes (Fz, Cz, Pz) were analyzed separately.

Fig. 3.

EEG results (time courses). The large graph provides the grand average time courses for pre- and posttest (PREPOST) from −200 to 1000 ms with regard to stimulus onset (bold asterisk * indicates corrected, small asterisks * indicate uncorrected comparison regarding the 9 ROIs). The inset illustrates the 3 different analysis windows: early-, mid-, and late-latency (250–450/450-650/650–850 ms). For the factors TRAIN, LEGALITY, and DAY only those ROIs which yielded significant main effects are shown. For a comprehensive illustration of all effects (including interactions) please refer to bar plots in Fig. 4 and Table 1. Negative polarity is upwards.

We additionally performed an overall ANOVA additionally including the factor ROI (9 levels). This 2 × 3 × 2 × 2 × 9 repeated measure ANOVA (PREPOST × DAY × TRAIN × LEGALITY × ROI) yielded main effects for ROI (df 8,96; F = 31.4/25.5/11.2 yielding a p< 0.001 for all time windows), LEGALITY (df 1,12; F = 6.3 p< 0.027 for early time window), and PREPOST (df 1,12; F = 5.8; p = 0.03 for late time window). Additionally the 2-way interactions PREPOSTxROI (df 8,96; F = 2.9/3.7/3.0 with p = 0.024/0.010/0.025 for the respective time windows) and DAYxROI (df 16,192; F = n.s./3.5/2.3 with p = n.s./0.006/0.049 for medium and late time windows) were significant. Finally a 3-way interaction LEGALITY × TRAINING × ROI (df 8,96; F = 2.0 with p = 0.03 for the early time window) was significant. The factor ROI was not analyzed further to not overly complicate the results and because localization of the ERPs is only a weak indicator of the underlying brain structures generating the response. In the results section we therefore only report results of the ANOVAs performed in each ROI separately.

2.5.2. fNIRS data analyses

fNIRS data of each participant were screened manually. Artifacts (abrupt changes > 2σ of the variance) were removed by a linear interpolation approach. A 0.3-Hz low-pass filter (Butterworth, third order) was applied to attenuate high-frequency artifacts mainly arising from the heart beat. Next a model including the 4 different conditions as separate boxcar-predictors was convolved with a canonical hemodynamic response function peaking at 5 s (Boynton et al., 1996, Wobst et al., 2001). Data were then fed into a general linear model approach to obtain beta values for each condition and each of the two hemoglobins.

Statistical analyses were performed on the beta values of both [oxy-Hb] and [deoxy-Hb]. Although the typical response pattern over an activated area consists of a decrease in [deoxy-Hb] accompanied by an increase in [oxy-Hb] we report both hemoglobins separately as has been regularly done in the infant literature (Lloyd-Fox et al., 2010). Localization of NIRS probes without extensive, time consuming referencing is rough. Therefore we report on 4 ROIs on each hemisphere allowing for some anterior-posterior differentiation of the responses: henceforth the NIRS-ROIs are referred to as: FRONT: frontal; FRTMP: fronto-temporal, TEMP: temporal; TPAR: temporo-parietal (Fig. 2).

3. Results

Results are reported separately for EEG and fNIRS. The factor ROI was not further analyzed (see Methods 2.5.). The statistical model, a 4factorial ANOVA including the factors PREPOST(2), DAY(3), TRAIN(2), and LEGALITY(2) was applied separately to the 3 time-windows of the ERP and to [deoxy-Hb] and [oxy-Hb]. The latter two parameters result from the GLM used for the fNIRS analysis.

3.1. EEG results

The overall repeated-measures ANOVA resulted in a robust effect for PREPOST at Fz in the mid and late time window and more broadly distributed effects when no correction for the separate testing in 9 ROIs was applied. At such an uncorrected level the factor TRAIN showed effects in the early time window and LEGALITY showed a main effect in the late and an interaction-based effect in the early time window. DAY showed main effects (uncorrected) but post hoc tests did not survive correction. Fig. 3 illustrates ERP time courses. It shows the comparison between pre- and posttest responses (PREPOST) for all electrodes analyzed. For the other three factors (TRAIN, LEGALITY and DAY) only the electrodes of ROIs yielding a significant main effect (uncorrected) are shown. Fig. 4 provides all results including the differences resulting from interactions (Bonferroni-corrected post-hoc testing). The statistically significant main effects and interactions are numerically listed in Table 1 (upper part).

Fig. 4.

EEG results (bar plots). The figure illustrates mean ERP amplitudes in three different time windows: 250–450 ms (early latency), 450–650 ms (mid latency), and 650–850 ms (late latency). All comparisons shown, yielded a significant main effect or an interaction for the overall ANOVA comprising the factors PREPOST, TRAIN, LEGALITY, and DAY (see Table 1,upper part for details). PREPOST: in the frontal ROIs post-tests showed more negative values than pre-tests, mostly in mid- and late latency windows. TRAIN: trained elicited more negative values than untrained items only for the early-latency window. LEGALITY: legal items showed less negative values when compared to illegal items in central and right parietal ROIs. This applied to early and late time window respectively. POST/untr in the inset indicates that this comparison results from an interaction and includes values from post-test for untrained items. DAY: the factor DAY was involved in three significant interactions, however post hoc testing did not survive multiple comparison corrections. Descriptively, however, negativity increased from day 1 over day 2 to day 3 in bilateral frontotemporal ROIs. In the left parietal ROI the pattern was inversed. Negative polarity is upwards.

The factor PREPOST yielded more negative values for the post- when compared to the pre-test (or a decrease in positivity after training). At Fz this held for all time windows whereas in bilateral frontal and frontotemporal ROIs (F4FC4, F3FC3, F8FT8 & F7FT7) it was significant in the mid- and late-latency time window (450–650 & 650–850 ms). On the contrary, the factor TRAIN was significant only in the early time window (250–450 ms) with a bilateral frontal and central distribution (F4FC4, F3FC3, Cz). Trained pseudowords elicited more negative values when compared to untrained items. The factor LEGALITY showed a main effect for the late time window in the right parietal ROI (C4CP4), and was involved in an interaction (LEGALITY x TRAIN x PREPOST) at Cz. Post hoc testing of the latter interaction yielded a significant effect only for the post-tests of untrained items. In both cases legal pseudowords elicited more positive values when compared to illegal items. The factor DAY yielded main effects in bilateral frontotemporal and left parietal ROIs. None of the post-hoc tests survived correction. Descriptively, however, ERPs became increasingly more negative in the mid-latency time window over the bilateral frontotemporal ROIs, whereas the right parietal effect showed the inverse modulation from day 1 to day 3 (increase in positivity).

To sum up, robust increases in negativity were seen from pre- to post-test. Similarly trained elicited more negative responses than untrained words, which was only significant at an uncorrected level. On the contrary, legal showed more positive ERPs than illegal pseudowords, an effect which was less robust. The long term modulation over the three consecutive days did not reach reliable significance and showed variable modulation direction. For a more detailed illustration of all time windows and all conditions which showed significant main effects please refer to Supplemental Material.

3.2. fNIRS results

The analysis of oxygenation changes as a marker of the hemodynamic response yielded partially corresponding results. Note that an increase in [oxy-Hb] and a decrease in [deoxy-Hb] are expected over an activated area. For the comparison pre- versus posttest (PREPOST) Fig. 5 provides time courses of oxy-Hb and deoxy-Hb in all ROIs while Fig. 6 provides bar-plots of all statistically significant main effects and significant post-hoc results from the interactions. The factor DAY yielded neither a significant main effect nor did post-hoc testing of interactions survive correction. Table 1 (lower part) numerically lists the statistical results and includes statistically significant interactions, not all of which are illustrated in Fig. 6.

Fig. 5.

fNIRS results (time courses). Grand average time courses across all participants for pre- and post-tests are depicted (pre: light, dashed lines; post: solid dark lines). Increases in [oxy-Hb] and decreases in [deoxy-Hb] signal activation of the underlying brain area. Note that the here illustrated PREPOST comparison only yielded statistically significant effects for the decrease in [deoxy-Hb], although in some ROIs (e.g. left FRTMP) [oxy-Hb] shows the expected mirror response. Large bold * denotes main effect for PREPOST over right frontal ROI (FRONT); smaller * on left hemispheric ROIs (FRONT, FRTEMP and TEMP) denotes that post-hoc tests of interactions showed the PREPOST effect (see Table 1 and Fig. 6 for details). The inset illustrates the GLM-based analysis: the stimulus (modeled as a 2 s event) was convolved with the canonical hrf, yielding predictors for all conditions. Using these predictors the GLM provides β-values for each condition. The shaded areas sketch the best fit of the predictor to the measured data. For bar-plot representations of all significant results please refer to Fig. 6 and Table 1 (lower part).

Fig. 6.

fNIRS results (bar plots). Statistically significant differences for the factors PREPOST, TRAIN and LEGALITY in [oxy-Hb] (red) and [deoxy-Hb] (blue). The factor DAY was involved in significant interactions (see Table 1, lower part) but did not yield statistically significant results in the post-hoc tests. Note that a larger increase in [oxy-Hb] and a larger decrease in [deoxy-Hb] are expected over an activated cortical area. PREPOST (upper row): after the training pseudowords elicited larger amplitudes for the [deoxy-Hb] responses. An inverse response direction was seen for [deoxy-Hb] over the left temporal ROI. Overall, the effect was left lateralized. TRAIN: bilateral ROIs showed larger response amplitude for trained when compared to untrained items, an effect partially reflected in the [oxy-Hb] responses. LEGALITY: a less robust difference between legal when compared to illegal items was demonstrated over left temporal ([oxy-Hb]) and right fronto-temporal ([deoxy-Hb]) ROIs. FRONT: frontal ROI; FRTMP: fronto-temporal ROI; TEMP: temporal ROI; TPAR: temporo-parietal ROI. train, LEG, POST, day3 in the bar-plot insets indicate that these comparisons result from interactions and show the comparison for the respective subset of data. Light shaded bars indicate values for pretest, untrained and illegal; right-sided bars show post-test, trained and legal. Scale for [oxy-Hb] is twice larger than that for [deoxy-Hb]. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

For the factor PREPOST [deoxy-Hb] and [oxy-Hb] largely showed greater response amplitude for post- when compared to pretest (more negative values for [deoxy-Hb]). An inverse pattern was seen for [deoxy-Hb] over left temporal ROI (TEMP). The effect was left lateralized. In conjunction with the more negative ERP values after training the changes from pre- to post-test consistently indicate an increase in neuronal processing in response to the pseudowords after the training. Time courses of [oxy-Hb] suggest a differential response in some ROIs (e.g. left FRTMP) but these differences were not statistically significant. Overall [oxy-Hb] was the less robust marker of differences most likely due to a larger proportion of noise and variance across subjects (see error bars in Fig. 6).

The effect of training (factor TRAIN, i.e. trained vs. untrained items) also yielded a robust difference with regard to the hemodynamic response. This differential activation was not lateralized and [deoxy-Hb] decreases consistently indicated that stronger cortical activation was elicited by trained when compared to untrained items. Again, this is in line with the ERP findings showing more negative going ERPs for the same comparison.

The factor LEGALITY yielded less robust effects. Left temporal and right fronto-temporal ROIs showed a statistically significant difference for the activation elicited by legal when compared to illegal pseudowords. Notably legal pseudowords yielded larger activation compared to illegal items. This is in line with the effect seen in adults (Rossi et al., 2011a), however contrary to the above two effects it is inversely related to the ERP findings which showed less negative ERPs for the legal when compared to illegal items. This will be discussed below (Discussion).

The factor DAY showed no main effect in [oxy-Hb] or [deoxy-Hb], but was involved in two 2-way and one 3-way interactions (see Table 1, lower part). However, similar to the ERP results, post hoc testing did not show a statistically robust change in activation over the three days.

4. Discussion

Infants enter the linguistic world with sophisticated discriminatory auditory skills. Within the first year of life they tune in to the phonology of their native language(s) (Cheour et al., 1998, Ortiz-Mantilla et al., 2013). Following this developmental trajectory it has been assumed that knowledge about word-forms is established in a ‘bottom-up’ fashion to be linked to meaning at a later stage of development. Indeed infants provide unequivocal evidence of emerging lexico-semantic competence by uttering their first words around their 1st birthday and show substantial increases in their active vocabulary only later (Goldfield and Reznick, 1990). Recent work, however, suggests surprisingly early passive word-knowledge at 6 months (Bergelson and Swingley, 2012). Moreover training-induced associations between a word-form and an object may be established even earlier (Friedrich and Friederici, 2015). Thus knowledge about word-forms, rough, semantic categories and familiar individuals (Tincoff and Jusczyk, 1999), plus a link between both representations may be available to the infant very early during linguistic development. This renders an interactive evolution of word-form and (proto)semantic knowledge more plausible. If infants use very rudimentary word knowledge to segment speech (Bortfeld et al., 2005) this knowledge should in turn support the establishment of phonological regularities in their native language(s). In 18 months old infants there is indeed evidence that the likelihood of a novel label to be established in a short training period depends on phonological dis/similarity to known words (Swingley and Aslin, 2007). Moreover regarding phonetic learning it was shown that 9 months old learnt non-native lexical tone discrimination better if the different tones were consistently paired to objects during familiarization (Yeung et al., 2014). These studies suggest that knowledge on word-forms and (proto)semantic knowledge interact already at a very early age (Werker and Yeung, 2005). The present study addressed such interactive influences on phonotactic processing at 6 months using an associative training paradigm (novel word ↔ novel object) over three days. Our hypothesis is that cerebral processing of novel word forms presented in isolation is altered by a training in which these word forms are associatively linked to images of novel objects. Similar effect on trained and untrained items would signal generalization; moreover in using both phonotactically legal and illegal pseudowords we addressed the question to what extent prior exposure to the native ‘phonological grammar’ would impact on training induced changes.

Pertaining to these research questions we find: (i) short-term training effects for all pseudowords, (ii) less robust differences between trained novel words and untrained items of highly similar phonological structure; (iii) a weak effect of previous linguistic experience (legal vs. illegal pseudowords) and, (iv) little evidence for consolidation over the three training sessions performed on consecutive days.

• (i)Short-term effect of associative training on processing of all pseudowords (trained ± untrained).

After training, all pseudowords showed larger amplitudes of the oxygenation response (fNIRS) and more negative ERPs mostly in the mid- and late-latency time windows (see main effects in Table 1 for the time windows 450–650 & 650–850 ms). This PREPOST main effect applied to both, pseudowords only presented during pre- and post-test without an associative semantic context, and those additionally multiply repeated during the associative training. Taken together these results indicate that cerebral processing of all pseudowords was enhanced after training. Such an effect could be explained by increased attention to basic auditory features of the stimuli since all pseudowords were produced by the same speaker in infant directed speech mode and shared the monosyllabic CCVC structure. However, left-lateralization of the changes as evidenced by fNIRS supports the notion of a more linguistic processing (Minagawa-Kawai et al., 2007, Obrig et al., 2010). Moreover, EEG effects at different ERP latencies have previously been interpreted to signal familiarization with novel word-forms in infants. At 10 months, words familiarized in isolation elicited a reduced positivity (or increased negativity) when presented in sentences during the test phase (Kooijman et al., 2005). In the same vein an increase in frontolateral negativity was suggested to indicate familiarity with the word form at 6 (Friedrich and Friederici, 2011) and even at 3 months (Friedrich and Friederici, 2015); notably the latter experiments used a pseudoword ↔ pseudoobject pairing paradigm similar to our present design. Later effects (up to 1000 ms) may represent an immature marker for emerging word knowledge at 12 months (Friedrich and Friederici, 2005) probably representing a precursor of the N400, established as a neurophysiological signature of lexico-semantic access in adults (Van Petten and Luka, 2012). We are cautious to interpret mid- and late-latency effects to represent differential training effects along the depth of lexcio-semantic encoding, but are confident that combined ERP-fNIRS results indicate an enhanced sensitivity due to familiarization with the overall phonological structure of the pseudowords.

It should be noted that untrained pseudowords were repeated during pre- and post-tests on each day. Such a repeated exposure may additionally contribute to the altered processing during post- compared to pretests. Although we cannot rule out this effect, it should be kept in mind that untrained pseudowords were presented much less frequently and outside any semantic context. Therefore we consider ‘generalization’ of the basic phonological structure the more probable candidate for the PREPOST effect in all pseudowords irrespective of their training status.

• (ii)Effects of training on the processing of the pseudowords (trained vs. untrained).

While the effect of the training applied to both trained and untrained pseudowords in mid- and late-latency windows, the early-latency window of the ERP analysis showed evidence for more negative responses to trained when compared to untrained pseudowords. This main effect TRAIN was confirmed by bilateral increases in hemodynamic response amplitude as measured by fNIRS. Additionally interactions involving the factors PREPOST and TRAIN showed that trained items elicited larger responses than untrained items only in the post-test (Cz for early latency ERP and deoxy-Hb over left frontal ROI (note that these effects did not survive multiple comparison correction for the 9 ROIs tested separately). Since we did not assess the response to congruent versus incongruent pseudoword ↔ pseudoobject pairings after the training, we can only speculate as to whether this indicates more effective familiarization due to repeated exposure or whether a very basic lexico-sematic link was established during training. Support for an influence of training-induced proto-word representation comes from experiments in which 6- and even 3-months old infants underwent a similar but even shorter training protocol and showed evidence for associative pseudoword ↔ pseudoobject pairing after very few simultaneous presentations (Friedrich and Friederici, 2011, Friedrich and Friederici, 2015). Additionally an experiment in adults performed with largely identical material and design (Rossi et al., 2013) showed clearly differential effects on novel word processing depending on whether mere passive familiarization or a categorization task was applied during the training phase. Not least, preliminary results of two parallel experiments in 6-month-old infants in which we used the current protocol, but training was either replaced by mere familiarization or consisted of an associative pairing with real objects, support the notion that changes in word-form processing strongly depend on the degree of (proto)semantic context provided. We therefore believe that the present results suggest both, a familiarization with phonological structure, which is generalized to phonologically similar items (overall differences between pre- and post-test) and a deeper encoding of word forms which allows for a basic form of lexico-semantic mapping due to the associative training (trained vs. untrained pseudowords).

• (iii)Impact of previous linguistic experience.

In adults, the impact of native phonotactic rules on the processing of novel word forms is well established (e.g., Dupoux et al., 1999, Steinberg et al., 2011, Wagner et al., 2012). Our own research using a larger corpus of the present material showed that passive listening to illegal items elicited smaller N400-amplitudes when compared to legal pseudowords (Rossi et al., 2011a) and such differential processing likely relies on areas including the middle temporal gyrus (Obrig et al., 2016). These findings indicate an implicit activation of lexico-semantic search to be triggered by attested but not unattested word forms. However, at 6 months lexico-semantic knowledge may be emerging but must be considered small. There is evidence that older infants (18/19 months) prefer phonotactically legal over illegal word forms, in that they learn pseudoobject ↔ pseudoword associations when the word form is phonotactically legal but not if it is illegal (Graf Estes et al., 2011, Graf Estes and Bowen, 2013). In younger infants (12 months) pairing of known object and novel words provided evidence for sensitivity to the contrast legal/illegal by differences in a familiarity related ERP component. On the contrary, differences in lexico-semantic integration − evidenced by variation in the N400 amplitude − were present at 19 months only (Friedrich and Friederici, 2005). Since categorical differentiation based on phonotactic status is present at 9 months (Mattys et al., 1999), and potentially emerges substantially earlier (Obrig et al., 2010, Rossi et al., 2011b) the small effect of phonotactic status on word form processing in our current experiment may rather indicate that infants are still exquisite learners of phonological structure at this age. Indeed, previous work indicates rapid adaptation to novel phonotactic rules in infants at 10 and 16 months (Chambers et al., 2003, Chambers et al., 2011) and very basic phonotactic rule acquisition is operational at 10 and even 4 months (Seidl et al., 2009). We propose that at 6 months infants can apply native phonotactic rules to differentiate between lexical candidates, but when semantic context is supplied during repeated exposure, such ‘meaningful’ training elicits rapid adaptation to previously illegal phonotactic rules. This notion is supported by the fact that for ERP-analyses the factor LEGALITY was involved in a large number of interactions with the other training-related factors. Regarding the ERP results pertaining to this factor one finding requires discussion: legal generally elicited more positive ERPs than illegal. This is contrary to the expectation that increases in negativity correlate to enhanced cortical processing. There are two potential explanations: (i) illegal pseudowords indeed elicited enhanced demands, due to relative unfamiliarity; alternatively (ii) the inverse polarity of the ERP component signals an immature precursor of the response. Such polarity reversals with age have been described for related ERP-components (Kooijman et al., 2013, Mannel and Friederici, 2013). Although phonotactic status only weakly affected response magnitude in both methodologies the larger fNIRS response to legal compared to illegal items speaks for the latter interpretation.

• (iv)Consolidation of training effects over the 3 consecutive days.

To serve as useful lexical candidates, familiarity to novel word-forms needs to survive short term training: the family dog and its name will not be exposed to the infant repetitively in one session, but its name (and later potentially the category ‘dog’) will occur repetitively over an extended period, including longer time intervals without exposure. Although training scenarios, including the one used in the present study, are far from being ‘natural’, the association between a specific word form and a specific object (or even category) should become increasingly stable over consecutive sessions. At a strict statistical level, the present results speak against such a consolidation over three consecutive days in 6-month-old infants. However, ERPs showed a main effect for the factor DAY. Although not surviving multiple comparison correction, there was a consistent increase in negativity over bilateral frontotemporal and a decrease in negativity over left parietal ROIs. We suggest that this indicates a weak long-term consolidation effect. Notably fNIRS data support this in showing interactions between LEGALITY x DAY over right frontotemporal ROIs (see Table 1). Note that the phonotactic status (legal/illegal) represents a long-term rule familiarization due to previous native language experience. Moreover, it should be noted that comparisons including the factor DAY suffer from additional noise due to novel montage of the EEG and fNIRS probes on the three consecutive days. Overall we speculate that infants did indeed show a weak long term consolidation, which might be disclosed if more rigid probe localization could be realized, or more salient word-object pairs will be used. Regarding previous work such a view is in line with weak but existent overnight consolidation shown in 6-month- and even 3-month-old infants (Friedrich and Friederici, 2011, Friedrich and Friederici, 2015). Furthermore, recently it was shown that a short ‘nap’ may lead to an even broader ‘conceptual’ representation of novel word forms, establishing a prerequisite for true word knowledge (Friedrich et al., 2015).

5. Conclusions

In the current study we investigated the flexibility of phonotactic rule representation in 6-month-old infants. Complementing previous work assessing sensitivity to native versus non-native phonotactics in different age groups, our study is embedded in the overall research goal to assess plasticity of the language network devoted to phonological analysis during early language development. We find strong modulatory effects of the associative, ‘proto-semantic’ training on trained pseudowords, and evidence for generalization to the processing of phonologically similar word-forms. Previous linguistic experience yielded a lesser modulation, which we interpret to show high plasticity rather than imperfect discrimination afforded by the network at this age. In line with previous research there was little evidence for longer term consolidation over 3 days.

Conflict of interest

None.

Appendix A. Supplementary data

The following is Supplementary data to this article: