Spoken words activate native and non-native letter-to-sound mappings: Evidence from eye tracking

Abstract

Many languages use the same letters to represent different sounds (e.g., the letter P represents /p/ in English but /r/ in Russian). We report two experiments that examine how native language experience impacts the acquisition and processing of words with conflicting letter-to-sound mappings. Experiment 1 revealed that individual differences in nonverbal intelligence predicted word learning and that novel words with conflicting orthography-to-phonology mappings were harder to learn when their spelling was more typical of the native language than less typical (due to increased competition from the native language). Notably, Experiment 2 used eye tracking to reveal, for the first time, that hearing non-native spoken words activates native language orthography and both native and non-native letter-to-sound mappings. These findings reveal high interactivity in the language system, illustrate the role of orthography in phonological learning and processing, and demonstrate that experience with written form changes the linguistic mind.

1. Introduction

Experience with language has lasting effects on further learning and on the mind (Marian, 2023). Native language (L1) experience in particular hones the brain to features and regularities of language input (Ellis, 2006; Schmitt, 2008). The present research investigated how experience with L1 orthography-to-phonology mappings influences non-native word processing. We taught participants vocabulary with non-native letter-sound correspondences to examine how individual differences (such as nonverbal IQ) and linguistic characteristics (such native language typicality) influence the learning of new mappings (Experiment 1) and whether native and non-native mappings are co-activated during non-native auditory word processing (Experiment 2).

Oftentimes, two related languages share a subset of their orthographic and phonemic inventories, but utilize conflicting letter-sound correspondences. For example, the letter J, is pronounced /dʒ/ in English (a voiced palato-alveolar affricate) but /x/ or /h/ in Spanish (a voiceless velar fricative or voiceless glottal fricative), and the phoneme /v/ is represented by the letter V in English but W in German. These differences can compound at the word level, as illustrated by the diverging pronunciations for the English/French cognate “destination,” (English /dEstIneIʃən/ and French /dEstinasjɔ̃/) and the false cognates “champ” (meaning field in French, and pronounced /tʃæmp/ or /ʃɑ̃/). Likewise, the English and Russian interlingual homograph веер is pronounced /bip/ in English, but /v^jeir/ (meaning paper fan) in Russian. Because phonological drift occurs more rapidly than orthographic changes over the historical evolution of typologically related languages (Marian, Bartolotti, Chabal, & Shook, 2012), differences in orthography-to-phonology correspondences accumulate over time, with potential consequences for word learning and processing.

Orthography-to-Phonology Mapping

Conflicting orthography-to-phonology correspondences are not unique to bilinguals. English monolinguals, for example, must learn that the letter C can take the /s/ or /k/ sound in different contexts, and likewise the phoneme /dʒ/ can be expressed either by G or J. This type of within-language variability in letter-to-sound mappings is more common in some languages (e.g., English) than in others (e.g., Italian), which can contribute to differences in how speakers of different languages learn and process words. Children who speak languages with more consistent letter-sound mappings (i.e., high orthographic transparency, e.g., Italian, Greek) have been shown to exhibit more advanced reading skills than speakers of less transparent languages (e.g., English) (Ellis & Hooper, 2001; Spencer & Hanley, 2003; see Ziegler & Goswami, 2005 for review). Likewise, adult second language (L2) learners benefit from greater orthographic consistency in the L2 when learning novel words (Burt & Blackwell, 2008; Taylor, 2011; see Meade, 2020 for review). A match in L1 and L2 orthographic transparency facilitates L2 word learning and phonological decoding (Hamada & Koda, 2008; Ijalba & Obler, 2015), demonstrating that knowledge and strategies developed for the native language influence later acquisition and processing of non-native words.

Furthermore, while individual differences (e.g., in phonological awareness) have been shown to predict children’s reading skills for both high- and low- transparency scripts (Bar-Kochva & Breznitz, 2014; Torgesen et al., 1997), the relationship between individual differences and reading development can vary depending on orthographic transparency. For instance, Ziegler et al. (2010) found that individual differences in phonological awareness were especially predictive of phonological decoding for children learning less transparent languages (e.g., English), while vocabulary knowledge was more predictive for those learning more transparent languages (e.g., Finnish). Such findings reveal that linguistic characteristics and individual differences interact to influence acquisition of native language letter-sound mappings. Relatively less is known, however, about the variables that predict the acquisition and activation of new mappings once the native language is fully established. The present study therefore investigates the linguistic and cognitive factors that moderate how adult native speakers of English (a low transparency language) learn and process words with letter-to-sound mappings that are consistent within the novel vocabulary but conflict with those of the native tongue.

Word Learning

Learning non-native vocabulary as an adult presents several challenges that distinguish the task from first language vocabulary acquisition. Foremost is the fact that the adult learner already commands a fully functioning linguistic system which influences subsequent word learning and processing. In addition, whereas one’s L1 is first acquired phonologically and only later orthographically, non-native input is often experienced as a combination of written and spoken words. This early availability of cross-modal information may have unique consequences for novel word learning. For instance, the successful acquisition of the Russian word веер for a native English speaker requires the ability to manage conflicting orthography-to-phonology mappings in the two languages. As such interlingual homographs demonstrate, non-native written word decoding is a convergence point for the challenges of L1 interference and cross-modal integration. In the current study, we investigate this intersection by exploring how orthography and phonology interact during early stages of adult non-native vocabulary acquisition.

One’s native language has both positive and negative effects on second language learning and processing. Linguistic features that are perceived by the learner to be similar in both languages can result in positive transfer, facilitating L2 acquisition (Ionin et al., 2008; Jarvis & Odlin, 2000). Non-overlapping features can also have negative effects on learning when structures in the L1 interfere with nascent L2 knowledge (Bhela, 1999; Birdsong, 2014; MacWhinney, 2007). The way that the L1 and L2 intertwine, however, cannot be comprehensively expressed as an accumulation of independent positive and negative interactions. For example, cognate words, which overlap in both form and meaning across languages, are generally easier to learn than non-cognates (De Groot & Keijzer, 2000; Lotto & De Groot, 1998) due to scaffolding on the existing L1 framework. However, in proficient L2 users’ speech production, cognate words are typically more heavily accented compared to non-cognates, because of their direct link to the native language (Amengual, 2012; Goldrick et al., 2014). Activation of the L1 can be both an early advantage and late disadvantage for the L2 learner. Here, we examine whether learning words with non-native letter-to-sound mappings is more challenging when they look more like L1 words than when they have less familiar orthographic forms.

Second language learners who are already literate in their native language experience persistent L1 interference during phonological decoding in an L2 (Hamada & Koda, 2008; Wang, Koda, & Perfetti, 2003). In fact, when letter-sound mappings in the L2 conflict with the native language, presenting words’ written forms in addition to their spoken forms disrupts learning compared to auditory presentation alone (Kaushanskaya & Marian, 2008, 2009). To determine what drives the difficulty posed by conflicting letter mappings during novel word learning, the challenges associated with learning new letter-to-sound mappings must be disentangled from those stemming from the acquisition of new letters or sounds. In most natural language pairs, the second language learner learns not only new letter-sound correspondences, but often new phonemes or graphemes as well. Categorical perception of L1 phonemes can distort perceptual discrimination and speech production in late L2 learners (Iverson et al., 2003; Kuhl & Iverson, 1995), with consequences for second language processing and representation (Baker & Trofimovich, 2005; Sebastian-Gallés et al., 2005). This perceptual mechanism may mask the related, but distinct effect of conflicting letter-to-sound mappings between the L1 and L2 on word learning. The present study disentangles the effects of new perceptual learning and learning of new mappings by keeping the same letters and sounds in the native and novel vocabulary and manipulating only the mappings between orthography and phonology.

To examine how conflicting orthography-to-phonology mappings and native language similarity impact vocabulary acquisition, learners were taught artificial words with letters and sounds that were not new to the learner but existed in their native language. Critically, the mappings between the letters and sounds in the novel vocabulary differed from those in L1 and had orthographic forms that were either typical or atypical of L1 words.

In addition, to examine the processes engaged during vocabulary acquisition, we assessed whether word learning and alphabetic knowledge was predicted by individual differences in cognitive abilities (inhibitory control, nonverbal IQ) and linguistic skills and knowledge (phonological working memory, English reading proficiency, English vocabulary size). Domain-general cognitive functions such as cognitive control could be expected to support word learning through more effective inhibition of competing native language orthography-to-phonology mappings (Blumenfeld et al., 2016; Linck, Schwieter, & Sunderman, 2012). Nonverbal fluid intelligence (Brooks et al., 2006) and verbal working memory (Misyak & Christiansen, 2012), on the other hand, could facilitate the extraction and encoding of non-native letter-sound regularities, which could deepen alphabetic knowledge about the novel vocabulary. Lastly, while native language proficiency may be associated with a general aptitude for word learning (Baddeley, et al., 1985; Braze et al., 2007), it may also interfere with the acquisition of words with conflicting orthography-to-phonology mappings, particularly when their orthographic wordforms resemble those of the native tongue.

Word Processing

The second, critical aim of the present study was to examine whether, following the acquisition of novel words in Experiment 1, conflicting native and non-native mappings are co-activated when processing the auditory forms of the newly learned words. Prior work with bilinguals has shown that knowledge of two languages with shared orthographies that map onto different phonologies can affect visual word processing. In visual go/no-go tasks, participants take longer when responding to written homographs (which share form but not meaning across languages), suggesting that orthography and semantics are activated in both languages (Dijkstra, Timmermans, & Schrieger, 2000). While this finding demonstrates that direct exposure to a written homograph (e.g., the written word веер) can co-activate competing phonological (/bip/ and /v^jeir/) and semantic representations (a tone and a paper fan), it remains an open question whether interference from overlapping orthography can be driven by non-overlapping auditory input during spoken language processing (e.g., the spoken word /bip/ activating the written form веер and corresponding English and Russian pronunciations and meanings).

Previous studies using eye tracking have revealed co-activation of phonological (e.g., Alopenna, Magnuson, Tanenhaus, 1998; Huettig, Rommers, & Meyer, 2011; Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995) and orthographic (e.g., Salverda & Tanenhaus, 2010) competitors in monolinguals, and co-activation of both languages in bilinguals (e.g., Marian & Spivey, 2003a,b; Shook & Marian, 2019; Spivey & Marian, 1999). For instance, studies utilizing the Visual World Paradigm (VWP) have shown that when bilinguals are presented with a visual array of objects (e.g., a marker, a marble, a stamp, and a shoe) and asked to click on one of the objects (e.g., the “marker”), they will often make visual fixations to objects whose labels overlap with the target, both within the same language (e.g., the marble), as well as across languages (e.g., the stamp, which translates to marka in Russian). Whether the orthography-to-phonology mappings of native and non-native vocabulary are co-activated, however, is yet unknown. The present research broaches this question by indexing co-activation of native language orthography-to-phonology mappings during novel spoken word processing using eye movements in the VWP.

The Present Study

In sum, the aims of the current research were two-fold. The first aim was to examine the influence of individual differences and linguistic characteristics on the initial acquisition of novel orthography-to-phonology mappings. The second aim was to investigate whether native and non-native orthography-to-phonology mappings are co-activated during non-native auditory word processing. Participants completed two experiments in different phases of a single study session, one focused on word learning and one on word processing. Experiment 1 examines how orthography-to-phonology mappings interact with individual differences and native-language typicality to impact new vocabulary learning. Specifically, we ask whether conflicting orthography-to-phonology mappings interfere with novel word learning and whether this interference is (1) predicted by individual differences in cognitive and linguistic skills and (2) exacerbated for words with more typical L1 spellings (i.e., greater interference for novel words with typical English forms, e.g., HANE, relative to those with more atypical forms, e.g., RAKO). Word learning was assessed using three tasks. First, in word learning: orthography, participants learned to match the written and spoken forms of pseudowords that mapped letters to sounds differently from the native language (e.g., the novel word pronounced /gufɔ/ (IPA transcription) would be orthographically represented as HANE rather than the expected-based-on-English-mappings GOOFA). Second, in word learning: meaning, they learned to associate the novel words with picture referents. Third, in word learning: letter knowledge, participants completed two additional tests of their word learning, which assessed how well they generalized their knowledge of novel orthography-to-phonology mappings to untrained words.

The aim of Experiment 2 was to examine how, following the acquisition of novel words in Experiment 1, conflicting orthography-to-phonology mappings impact the processing of recently-learned vocabulary and the extent to which the two modalities (visual in the case of orthography and auditory in the case of phonology) and the two patterns of letter-sound mappings (native and non-native) interact and are co-activated during spoken word comprehension. In other words, we ask whether after learning novel vocabulary in Experiment 1, a participant asked to “click on the /gufɔ/” from an array of objects would make visual fixations to an English cross-linguistic competitor that overlaps with the target’s orthographic form (e.g., a picture of a cane via activation of the target’s written form HANE). The activation of orthography during spoken word processing was assessed in a cross-linguistic interference task using the Visual World Paradigm (see Appendix for full instructions of each task).

2. Experiment 1: Word Learning

To investigate cross-modal effects on learning, we taught English monolinguals artificial words containing familiar English graphemes and phonemes, but novel orthography-to-phonology mappings that differed from English (e.g., a word pronounced /gufɔ/ and spelled HANE). Successful acquisition required learners to minimize interference from English mappings in order to form correct associations for the non-native vocabulary. Non-native vocabulary varied in wordform similarity to English based on English bigram probabilities and the number of English orthographic neighbors. We predicted that novel words that used conflicting orthography-to-phonology mappings, but superficially resembled English and thus increased activation of the native language (e.g., HANE), would be more difficult to learn than words that used conflicting mappings but looked less like typical English words (e.g., RAKO). To further explore the processes underlying the acquisition of conflicting mappings, we examined whether successful acquisition was predicted by individual differences in cognitive skills (e.g., nonverbal IQ) and linguistic knowledge (e.g., English vocabulary size).

2.1. Methods

2.1.1. Participants

Twenty monolingual English speakers (16 women) participated after providing informed consent in accordance with the university’s Institutional Review Board. Eye tracking data for one participant was unavailable due to equipment malfunction. Language background was assessed using the Language Experience and Proficiency Questionnaire (LEAP-Q, Marian, Kaushanskaya, & Blumenfeld, 2007). Participants had minimal second language proficiency (<= 3 out of 10). Cognitive and linguistic assessments were used to identify the effect of individual differences on learning and cross-linguistic competition, including non-verbal IQ (Wecshler Abbreviated Scale of Intelligence, WASI, block design and matrix reasoning subtests, PsychCorp, 1999), English vocabulary size (Peabody Picture Vocabulary Test III, PPVT, Dunn, 1997), self-reported English reading proficiency (LEAP-Q, Marian, Blumenfeld, & Kaushanskaya, 2007), phonological working memory (Comprehensive Test of Phonological Processing, CTOPP, digit span and nonword repetition subtests, Wagner, Torgesen, & Rashotte, 1999), and inhibitory control (colored squares Simon task, Simon & Rudell, 1967). Standard scores for each measure are included in Table 1.

Table 1.

Participant characteristics

Measure	M	SD
Age in years	22.2	2.35
WASI performance IQ	109.2	10.16
PPVT-III vocabulary	111.65	6.77
CTOPP Digit Span raw score	16.9	2.07
CTOPP Nonword Repetition raw score	13.35	2.99
LEAP-Q English reading proficiency	9.65	0.61

Note. Performance IQ is from the Weschler Abbreviated Scale of Intelligence, block design and matrix reasoning subtests (PsychCorp, 1999). English vocabulary size is from the Peabody Picture Vocabulary Test III (Dunn & Dunn, 1997). Digit Span and Nonword Repetition are from the Comprehensive Test of Phonological Processing (Wagner, Torgesen, & Rashotte, 1999). English reading proficiency is from the Language Experience and Proficiency Questionnaire (Marian, Blumenfeld, & Kaushanskaya, 2007).

2.1.2. Materials

A set of nonwords was created using 13 mismatching English graphemes and phonemes (4 vowels and 9 consonants). To enhance task engagement, participants were informed that they would be learning words from a language called Colbertian. This name was chosen to honor comedy show wordsmith and Northwestern University School of Communication alumnus Stephen Colbert (as in Bartolotti & Marian, 2012). The letter-sound mappings in Colbertian were designed to maximally differ from English. For example, whereas the English letter B corresponds to /b/, a voiced bilabial stop, the Colbertian letter B corresponds to the phoneme /s/, a voiceless alveolar fricative. English vowels were mapped to Colbertian vowels (e.g., the letter A would be pronounced /u/ in Colbertian) and English consonants were mapped to Colbertian consonants (e.g., the letter H would be pronounced /g/ in Colbertian). Because different characteristics are typically used to describe vowels (height, backness, rounding) and consonants (place of articulation, manner of articulation, voicing), mapping sounds within each class across languages enabled us to select Colbertian sounds that explicitly differed along each dimension within that class.

The inventory of 13 letters was used to create 24 bisyllabic words in Colbertian, each composed of four letters (all orthography-to-phonology mappings and Colbertian words are provided in the Appendix). The novel words were designed to vary in how closely their orthographic forms adhered to English lexical patterns, based on a wordlikeness metric comprising English neighborhood size (i.e., the number of English words that differed from the novel word by substitution, deletion, or addition of a single letter) and bigram probability (i.e., the average frequency of each pair of letters in the novel word). Twelve High-Englishlike words (e.g., HANE) had both high English neighborhood sizes (M = 16.08, SD = 3.48) and bigram probabilities (M = .028, SD = .008). Twelve Low-Englishlike words (e.g., RAKO) had both smaller neighborhoods (M = 6.58, SD = 5.25, t(22) = 5.23, p < .001) and bigram probabilities (M = .013, SD = .005, t(22) = 16.01, p < .001). Phonological stimuli for each Colbertian word were recorded by a female speaker of Standard American English. Stimuli presentation throughout the experiment was controlled by the experimental software MATLAB with the Psychophysics Toolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997).

2.1.3. Procedure

Word learning: Orthography.

First, participants were exposed to the 24 Colbertian words, one at a time. A single word was presented auditorily over headphones while its spelling was presented in the center of the computer screen; participants repeated the word out loud and clicked the mouse to advance. Then participants completed individualized training regimes to master the language (Figure 1A). A single training block included 24 trials with each word as a target once. In each trial, participants viewed four written words on the screen and heard the target over headphones. After making a selection by clicking on one of the written words, accuracy was recorded and the correct answer was provided as feedback so that participants could improve over time. Additional training blocks were repeated until the participant achieved a performance criterion of 90% accuracy on two consecutive blocks.

Figure 1.

Word learning procedure. A) During orthographic word learning, participants heard a target word in the new language and selected its matching written form. Feedback was provided after each trial to reinforce the correct association. Participants repeated blocks of 24 trials until they achieved a learning criterion of 90% accuracy on two consecutive blocks. B) During word-meaning learning, participants heard and saw a Colbertian target and selected the matching picture. Corrective feedback was provided. As in orthographic word learning, training continued until the performance criterion of 90% accuracy on two consecutive blocks was achieved.

Word learning: Meaning.

After learning auditory words’ spellings, participants learned to associate the words they had just acquired with picture meanings. First, participants were shown all 24 pairings. A Colbertian word was presented visually and auditorily along with four pictures, and the target was indicated by a red box. Then, as in word-form learning, participants completed individualized training regimes to master the pairings (Figure 1B) using the same performance criterion (90% accuracy on two consecutive testing blocks). In each trial, the Colbertian target word was presented visually and auditorily, and four pictures were shown in the corners of the screen (in order to reduce novelty effects in the subsequent cross-linguistic interference task, competitors and fillers were used as foils during learning, but were paired with different targets). After selecting a picture, the correct answer was provided as feedback so that participants could reinforce the correct association.

Word learning: Letter knowledge.

Two phonetic generalization tasks probing recognition and production were used to assess how well participants learned Colbertian’s underlying letter-to-sound mappings. In the recognition task, four new Colbertian written words were presented on the screen. The auditory target was presented over headphones, and the participant selected the matching word. In 24 Low Similarity trials, knowledge of a single letter was sufficient to identify the target, because all four words contained unique letters at each position. (e.g., Target /suzɔ/ spelled BAPE does not have any letters in the same position as Foils KOVI, VEDO, or RINA). In 24 High Similarity trials, each foil partially overlapped with the target, and thus an accurate response required knowledge of multiple Colbertian letters (e.g., Target /wɔtʃæ/ spelled KEDI shares letters with foils KOVA, NADO, and BERI). Low and High Similarity trials were intermixed during testing.

In the production task, participants viewed 48 novel written Colbertian words (of ConsonantVowelConsonantVowel format) in sequence and verbally produced the phonological form. Responses were phonetically transcribed by hand. Word accuracy was the percentage of novel words produced correctly. For each participant, letter accuracy was calculated across all novel words as a measure of their knowledge of word-embedded letter-to-sound mappings.

2.1.4. Data Analysis

The primary outcome of interest was accuracy over time on the word learning: orthography task – that is, the rate and extent to which participants successfully learned to associate Colbertian spoken words with their orthographic forms. In order to capture the maximal learning window across participants, we analyzed performance up to the average criterion point (i.e., blocks 1–10). The change in accuracy over time in orthographic word learning was analyzed using growth curve analysis (Mirman, Dixon, & Magnuson, 2008; Mirman, Magnuson, Graf Estes, & Dixon, 2008), a technique specifically designed to assess change over time. Growth curve analysis is a form of multilevel regression that simultaneously estimates the effects of individuals (e.g., differences in mean accuracy and rate of learning across participants) and of experimental manipulations (e.g., Englishlikeness) on timecourse data. The curve or best-fit line characterizing each participant’s trajectory of learning over time was first estimated in a base level-1 model using second-order orthogonal polynomials (i.e., intercept, linear, and quadratic time terms). The intercept captures overall levels of accuracy, the linear term characterizes the slope or rate of learning over time, and the quadratic term describes the rise and fall in accuracy around a central inflection point in the learning curve.

Effects of word-level factors (Englishlikeness) or participant-level factors (cognitive profile, word generalization skill) on each of these parameters were then assessed in level-2 models. For instance, an effect of Englishlikeness on the intercept would reflect differences in the average height of the curve (e.g., if overall accuracy was higher for low-Englishlike than high-Englishlike words). Effects on the linear and quadratic terms would each reflect changes in the rate of learning over time, with the linear term indexing the overall slope of the learning curve across the full time window (e.g., if the overall change in accuracy from Block 1 to Block 10 was greater for low- than high-Englishlike words) and the quadratic term reflecting periods of accelerated and decelerated learning toward the middle of the window (e.g., if rapid gains were made earlier in the task for low-, but not high-Englishlike words). The base model included all time terms and random effects of participant on all time terms. The effect of Englishlikeness was examined by comparing the model fit of the base model with one that included a fixed effect of Englishlikeness plus interactions with all time terms.

After examining the effect of Englishlikeness, we assessed whether accuracy on the orthographic word learning test was further predicted by individual difference measures of cognitive and linguistic abilities (i.e., non-verbal IQ, Simon effect, Simon inhibition, English reading proficiency, English vocabulary size, phonological working memory) and Colbertian letter knowledge (i.e., performance on the novel word recognition and production tasks). Effects of individual difference measures were evaluated in separate models that included fixed effects of the individual difference measure, Englishlikeness, the time terms, and all two-way interactions, plus random effects of participant on all time terms. Models were fit using maximum likelihood estimation (MLE), which seeks to maximize the likelihood that the estimated parameters would produce the observed data (represented by the likelihood function; L(β)). Model fit can be determined by taking twice the negative log of the likelihood function (i.e., −2LogLikelihood), with smaller values indicating better model fit (common metrics of model fit such as the corrected Akaike’s Information Criterion (AICc) and the Bayesian Information Criterion (BIC) are estimated using −2LogLikelihood). The significance of each fixed effect was assessed using a Chi-square test on −2LogLikelihood change in model fit (between a model with and without the fixed effect), and parameter-specific p-values were estimated by using a normal approximation, treating the t-value from the model as a z-value (See Barr, Levy, Scheepers, & Tily, 2013).

2.2. Results

Participants reached the 90% accuracy criterion in the word learning: orthography task after an average of ten blocks (M = 10.10, SD = 7.40, Range [2, 31]), and in the word learning: meaning task after three blocks (M = 3.05, SD = 0.69, Range [2, 4]). The training was designed to equate participants on knowledge of the specific trained words, and accordingly, accuracy was high in the subsequent cross-linguistic interference task (Experiment 2), for which participants were asked to identify visual objects corresponding to previously-learned spoken words (M = 99.6%, SD = 0.8%, Range [97.9, 100]).

Orthographic word learning was affected by the Englishlikeness manipulation and by nonverbal IQ, assessed by improvements to model fit for the growth curve model of word learning over blocks 1–10. The Englishlikeness factor significantly improved model fit compared to the base model (ΔLL = 5.89, X²(1, 13) = 11.77, p < .01), with an effect on the intercept (Estimate = −0.035, SE = .011, z = 3.22, p < .01) driven by decreased overall accuracy for the high-Englishlike compared to the low-Englishlike items across training (Figure 2). Of the individual difference measures, nonverbal IQ improved model fit compared to the base + Englishlikeness model (ΔLL = 5.89, X²(4, 17) = 11.78, p < .01), with effects on the intercept (Estimate = 0.121, SE = .035, z = 3.39, p < .001) and quadratic terms (Estimate = −0.051, SE = .022, z = 2.30, p < .05). These effects reflected both higher overall accuracy and an accelerated learning pace for higher IQs; a one SD increase in the standard score (i.e., 15 points) corresponded to an increase in accuracy of 1.2% over training blocks 1–10 (Figure 3, left). Lastly, while the addition of phonological working memory (digit span) only marginally improved model fit compared to the base + Englishlikeness model (ΔLL = 4.26, X²(4, 17) = 8.52, p = .074), parameter estimates revealed a significant effect of working memory on the linear slope term (Estimate = −0.10, SE = 0.04, z = 2.42, p < .05), reflecting a shallower learning curve among those with higher working memory. Specifically, higher working memory was associated with greater accuracy at the beginning of the task, but relatively smaller gains over the course of orthographic word learning (Fig. 3, right). Neither nonverbal IQ nor the digit span measure of working memory interacted with Englishlikeness, and none of the other individual difference measures (nonword repetition, English reading proficiency, English vocabulary size, Simon effect, Simon inhibition) approached significance (ps > .1).

Figure 2.

Orthographic word learning and English similarity. Englishlikeness of the novel words had a significant effect on the intercept term of a growth curve model of word learning. Colbertian words that were highly Englishlike (solid line) were responded to less accurately over the course of training compared to low-Englishlike words (dashed line), suggesting greater native-language interference during learning.

Figure 3.

Left: Orthographic word learning and non-verbal IQ based on a median split (median WASI standardized score = 109). A continuous measure of non-verbal IQ had significant effects on the intercept and quadratic terms of a growth curve model of word learning. Higher non-verbal IQ was associated with greater accuracy and more accelerated rates of learning during training. Right: Orthographic word learning and verbal working memory based on a median split (median digit span = 11.5). A continuous measure of verbal working memory had a significant effect on the linear term of a growth curve model of word learning. Higher working memory was associated with a shallower learning curve, resulting from superior performance at the beginning of the task combined with relatively modest gains over the course of training.

Given the role of nonverbal reasoning in pattern learning (Brooks et al., 2006; Kempe, Brooks, & Kharkhurin, 2010), the superior word learning among those with higher nonverbal IQ may be attributed to more successful extraction of individual letter-to-sound mappings. Support for this interpretation was found in the relationship between nonverbal IQ and Colbertian letter knowledge. There was wide variability in participants’ knowledge of Colbertian letter-sound correspondences, as indexed by performance in the two word learning: letter knowledge tasks. In the novel word recognition task, increasing the similarity of the foils (and thus the letter-sound knowledge necessary to identify the target) significantly impacted accuracy, from 92.1% in low similarity (SD = 7.6%, Range = [66.7, 100]) down to 75.2% in high similarity (SD = 19.1, Range = [41.7, 100], t(19) = 4.55, p < .001, 95% CI = [0.09, 0.25], d = 1.19). In the novel word production task, word accuracy ranged from 0 to 87.5%, M = 11.7, SD = 21.0. Letter accuracy within participants ranged from 0 to 94.8%, M = 45.6, SD = 26.3 (Figure 4). A linear model regressing novel word recognition onto similarity condition (low, high), nonverbal IQ, working memory (digit span), inhibitory control, English reading proficiency, and English vocabulary size revealed that, in addition to significantly greater accuracy in the low than high similarity condition (p < .001; see above), higher nonverbal IQ was associated with significantly better recognition of untrained words (Estimate = 0.06, SE = 0.03, t = 2.22, p < .05). Likewise, a model regressing novel word production onto the individual difference measures revealed that higher nonverbal IQ was associated with significantly more accurate production of untrained words, both when accuracy was scored based on production of individual letters (Estimate = 0.14, SE = .06, t = 2.22, p < .05) and of the whole word (Estimate = 0.13, SE = .06, t = 2.16, p < .05). No other individual difference measure affected novel word recognition or production.

Figure 4.

Mean accuracy on the orthographic word learning and letter knowledge tasks, including low- and high-similarity novel word recognition, and letter and whole word production of untrained words. Data points represent mean accuracy for individual participants.

To more directly examine the relationship between letter knowledge and orthographic word learning, four models were constructed to assess whether the fit of the base + Englishlikeness orthographic word learning model was significantly improved by adding a fixed effect of 1) high-similarity recognition accuracy, 2) low-similarity recognition accuracy, 3) letter production accuracy or 4) whole word production accuracy. Accuracy in the more difficult High-Similarity condition of the novel word recognition task significantly improved model fit (ΔLL = 5.79, X²(3, 16) = 11.59, p < .01), with an effect on the intercept term (Estimate = 0.131, SE = .033, z = 3.96, p < .001). In other words, the ability to spell new (untrained) words based on their phonological forms was associated with overall greater accuracy on the initial word learning task. Similarly, letter accuracy in novel word production improved model fit (ΔLL = 6.02, X²(3, 16) = 12.05, p < .01), with an effect on the intercept term (Estimate = 0.125, SE = .034, z = 3.62, p < .01) -- that is, the ability to produce new (untrained) words based on their written forms was associated with better orthographic word learning. Further, whole word production accuracy improved model fit (ΔLL = 8.05, X²(3, 16) = 16.11, p < .01), with a significant effect on the intercept (Estimate = 0.130, SE = .034, z = 3.85, p < .001) and a marginal effect on the linear term (Estimate = −0.088, SE = .044, z = 2.00, p = .059). In each case, better Colbertian letter knowledge was associated with higher overall accuracy during the critical early window of orthographic word learning comprising blocks 1–10; whole word production accuracy was also associated with a shallower slope due to reaching ceiling performance earlier during orthographic word learning¹.

2.3. Discussion

Experiment 1 examined how native language typicality and individual differences in cognitive and linguistic abilities influence the acquisition of novel vocabulary that conflict with L1 orthography-to-phonology mappings. All Colbertian vocabulary utilized English letters and phonemes in order to isolate the effect of orthography-to-phonology mappings on learning, independent of perceptual learning. Furthermore, the novel words varied in their similarity to English lexical patterns, based on a wordlikeness metric consisting of English neighborhood size and bigram frequency.

First, we observed that the challenges associated with learning novel vocabulary with non-native letter-to-sound mappings are compounded for words that orthotactically resemble the native language. Language users automatically generate the corresponding orthographic or phonological forms for novel words based on phonological rules (Johnston, McKague, & Pratt, 2004). Speakers more readily adopt non-native pronunciation patterns for words that are perceived to be of foreign origin, such as city names (Fitt, 1995), and here we show that this pattern of L1 similarity also affects non-native phonological decoding during learning.

Novel written or spoken words with high L1 form overlap are generally easier to learn than those with low form overlap when orthography-to-phonology conflict is not manipulated (Roodenrys & Hinton, 2002; Storkel et al., 2006; Thorn & Frankish, 2005). For instance, Storkel et al. (2006) observed that after English-speaking adults were presented with pairs of nonwords and novel objects embedded in a story, nonwords with a higher number of phonological English neighbors were recalled with significantly greater accuracy than words with fewer neighbors. Likewise, studies utilizing story-based paired-associates have found that higher neighborhood density and biphone frequency facilitates children’s acquisition of nonword object labels (Storkel, 2001, 2003, 2004; Storkel & Maekawa, 2005; Storkel & Rogers, 2000). Neighborhood density and biphone frequency have also been shown to influence serial recall, with superior memory for nonwords with more native-like forms (e.g., Gathercole et al., 1999; Roodenrys & Hinton, 2002; Thorn & Frankish, 2005). The advantages of native language typicality are likely to stem from the increased activation of L1 lexical and/or sublexical representations. For instance, it has been proposed that activating a greater number of L1 neighbors may facilitate the process of integrating novel words into the existing lexico-semantic network (Storkel et al., 2006), as well as the ability to reconstruct degraded memory traces based on existing lexical and sublexical knowledge (Thorn & Frankish, 2005). Notably, however, the benefits of greater L1 activation are likely to be contingent on the extent to which existing representations and rules can be applied to the novel language. When orthography-to-phonology mappings of non-native vocabulary conflict with that of the native tongue, as in the current study, the opposite pattern emerges – Colbertian words with more Englishlikeness were harder to learn. This discrepancy can be attributed to interference caused by conflicting L1 letter-sound correspondences and the stronger activation of L1 knowledge for words that resemble the native language.

Proficient bilinguals routinely use sublexical cues like letter and bigram frequencies as indicators of language membership, which helps inform how the written form should be represented phonologically (Oganian et al., 2015). Similarly, the novel language learner may initially use statistical cues to language membership as a way to mark whole word exceptions from L1 phonological rules. This kind of whole-word association is particularly common in the case of loan words. For example, common English words of French origin, including gourmet, bouquet, and ballet, retain French-like spellings and pronunciations that contrast with English phonological rules. Because of Colbertian’s modest vocabulary of 24 words, it was possible to learn the novel vocabulary as whole-word exceptions to English phonological rules. However, because this whole-word approach does not scale well with increased vocabulary size, eventual learning of L2 alphabetic mappings is essential. If participants were given a training vocabulary larger than the 24 words used in the current study, whole-word learners’ acquisition rates may start to decline as they reach their vocabulary capacity, followed by a transition period of slow growth during which they pick up the necessary letter-sound mappings. In contrast, the alphabetic learners who have mastered individual letter-sound mappings are expected to adhere to a consistent rate of vocabulary growth over time.

Speaking to the potential impact of whole-word vs. alphabetic learning on acquisition rates, we observed that individual differences in verbal working memory, nonverbal intelligence, and alphabetic knowledge had distinct effects on the trajectory of word learning over time. Higher verbal working memory was associated with better word learning immediately following the initial exposure phase, but did not yield benefits beyond the first few blocks of training or for the acquisition of more fine-grained alphabetic knowledge. It may therefore be the case that the early advantages of higher working memory resulted from better encoding and maintenance of whole words rather than of individual letter-sound mappings.

In contrast, we found that higher scores on a nonverbal intelligence test were associated with increasing gains over the course of testing, and superior generalization to untrained words. Similar effects of nonverbal IQ on word learning have been observed in prior studies utilizing paired-associates learning paradigms (e.g., de Jong et al., 2000; Krishnan, Watkins, & Bishop, 2017), as well as on pattern-based learning of orthographic wordforms (Hung, 2012; Ricketts et al., 2011) and grammatical categories (Brooks et al., 2006, 2017; Kempe et al., 2010). There is additionally evidence that nonverbal reasoning among children with language impairments is a strong predictor of language development (Bottling, 2005; Phelps, 2000; Stevens et al., 2000; Stothard et al., 1998; Tomblin, Freese, & Records, 1992) and that better nonverbal reasoning among adult cochlear implant users is associated with superior word and sentence recognition (e.g., Knutson et al., 1991; Mattingly, Castellanos, & Moberly, 2018; Moderly & Reed). Though the dynamic relationship between language abilities and nonverbal IQ is not yet fully understood, it has been proposed that individuals with higher nonverbal IQ may be better able to compensate for language difficulties (e.g., Snowling, Bishop & Stothard, 2000; Stanovich, 1993; Stevens et al., 2000). In the context of the present study, the superior orthographic word learning among those with better nonverbal reasoning may have resulted from more efficient extraction of non-native patterns and stronger encoding of associations between individual letters and sounds.

Further support for the important role of alphabetic knowledge was found in the close relationship between orthographic word learning and performance on the word generalization tests. Better recognition and production of untrained words were each associated with superior performance on the orthographic word learning task. Each learned letter in the novel vocabulary facilitated acquisition of other letters by reducing the amount of unknown information presented in each trial. Note that at no point in training were participants told to attend to individual letters or that they would later be asked to generalize to new words, indicating that letter knowledge may have been an emergent property of word learning, to a varying degree across individual learners. Future studies assessing participants’ strategy-use and metacognitive evaluations of alphabetic knowledge may clarify the explicit vs. implicit nature of learning letter-to-sound mappings during non-native vocabulary acquisition.

The ability to generalize language knowledge to new exemplars is critical to expanding vocabulary and incidental word learning. The current study illustrates the challenges encountered by learners when their experience is with two languages that actively conflict, as is sometimes the case for mappings between orthography and phonology. Though participants in the current study all mastered the Colbertian vocabulary to criterion, individual differences were observed in their ability to generalize to novel wordforms. This finding suggests that learners do not necessarily obtain knowledge of conflicting mappings in the course of learning and that in some cases, explicit instruction may be necessary and beneficial (Bitan & Karni, 2003; Brennan & Booth, 2015; Brennan, 2014).

Because many world languages share orthographic units but differ in how they use orthography to represent phonological information, for the language learner, navigating these differences is an important step in achieving L2 proficiency. Identifying how previous experience with letter-to-sound mappings in the native language impacts acquisition and processing of novel letter-sound mappings provides valuable insights into the architecture of the language system.

3. Experiment 2: Cross-linguistic Co-activation

To investigate cross-modal effects on novel word processing, we tracked participants’ eye movements using a visual world search task conducted with the newly-learned vocabulary. After mastering the novel vocabulary, participants listened to auditory Colbertian words and identified their matching pictures on a visual display. In competitor trials, the English name of one of the pictures overlapped orthographically with the target (e.g., competitor picture of a CANE, which overlaps with the target /gufɔ/, spelled HANE (High-Englishlike) or competitor picture of a RAKE, which overlaps with the target /huwi/, spelled RAKO (Low-Englishlike)). The goal was to measure whether participants fixate L1 orthographic competitors more than non-overlapping pictures in a display when hearing L2 words, in order to examine whether non-native spoken words can simultaneously activate orthography in the native language.

3.1. Methods

3.1.1. Participants

The same participants took part in Experiment 2 as in Experiment 1.

3.1.2. Materials

Because each Colbertian word was created by substituting one letter of an English word, each had an English orthographic neighbor with no phonological overlap. For example, the Colbertian word HANE, pronounced /gufɔ/, is an orthographic neighbor of the English word CANE, pronounced /keIn/. This high degree of orthographic overlap with no phonological similarity is rare in natural language pairs, and allowed us to isolate the effect of dual-language orthography on auditory processing in a visual world task. A Colbertian word (target), its neighbor (orthographic competitor), and two non-overlapping filler words comprised a single test set; black and white line drawings were selected to pair with each word in a set. Pictures were highly recognizable, with English naming consistencies above 80% in either the International Picture Naming Project database (Bates et al., 2003), or production norms (N = 20) collected from university students and Amazon’s Mechanical Turk (https://www.mturk.com). English picture names of the three object types (target, competitor, filler) did not differ in neighborhood size (CLEARPOND, Marian et al., 2012), lexical frequency (SUBTLEXUS, Brysbaert & New, 2009), or concreteness, imageability, or familiarity (MRC Psycholinguistic Database, Coltheart, 1981) (all ps > .05).

3.1.3. Procedure

The visual world search task used eye tracking to assess co-activation of native and non-native orthography during Colbertian spoken word processing. Eye movements in the visual world task were recorded with an SR Eyelink 1000 eye tracker, at 1000 Hz. Each trial began with a 1000ms fixation cross in the center of the screen. Then four pictures were displayed in the corners, and after a 500ms delay the Colbertian target was played over headphones (the orthographic form was never shown in the task). Participants clicked the matching target as quickly and accurately as possible with no feedback provided. In 24 experimental trials (Figure 5), one of the pictures was an English orthographic competitor (e.g., competitor CANE for the target /gufɔ/, spelled HANE). Note that competitors did not overlap phonologically with the target, allowing us to isolate the effect of orthographic overlap during spoken word processing. Experimental trials were intermixed with 24 (Competitor-Absent) Control trials used to mask the experimental manipulation, in which no picture names overlapped orthographically or phonologically with the target.

Figure 5.

Cross-linguistic interference procedure. In each trial, participants heard a Colbertian word (no orthography was presented) and selected the matching picture. In 24 competitor trials, the English written form of one of the pictures (e.g., Colbertian target /gufɔ/, spelled HANE, meaning ruler) overlapped with the orthographic form of the target (in this case, the English word CANE, /keIn/).

3.1.4. Data Analysis

Eye tracking fixations were also analyzed using growth curve analysis, which allowed us to examine the overall proportion of visual fixations to competitor and filler objects in the display, as well as differences in the rise and fall of fixations over time (Mirman, Dixon, & Magnuson, 2008; Mirman, Magnuson, Graf Estes, & Dixon, 2008). For each participant, we began by calculating the proportion of fixations that were made to competitor and filler objects at each 50ms time bin between 250ms pre word onset (at which point participants began fixating objects in the display) until 750ms post word onset (at which point fixations to the target peaked, corresponding to final target selection). Fixation proportions were calculated by summing the number of fixations that were made to competitors or fillers within each time bin and dividing by the total number of fixations that were made during that time. Fixation proportions for fillers were averaged across the two filler objects in the display. The base linear mixed-effects regression model included fourth-order orthogonal polynomials that each characterized distinct components of the fixation curve. These included an intercept, which captured the average proportion of visual fixations to the competitor and filler objects in the display, a linear term, capturing the overall slope of fixations across the entire time window, a quadratic term characterizing the rise and fall of fixations around a central inflection point in the middle of the window, and cubic and quartic terms, respectively capturing asymmetric and symmetric effects toward the tail ends of the window. Additional models added a fixed effect of object type (Competitor vs Filler) to each time variable in turn, and change in model fit was assessed using a Chi-square test. Parameter-specific p-values were obtained using the Satterthwaite approximation for degrees of freedom. All models included random effects of participant on all time terms.

3.2. Results

The visual world search task assessed native and non-native orthographic interactions during Colbertian auditory word processing. Visual fixations to all objects in the display increased 250ms after display onset as participants scanned the scene prior to onset of the auditory word (the auditory target was announced 500ms after display onset) (Figure 6). Fixations to the target picture during this preview period increased more than to filler pictures, a consequence of expectations that may have been developed during training (although participants viewed target, filler, and competitor pictures during picture-word learning, Colbertian word labels were only learned for target pictures).

Figure 6.

Cross-linguistic interference. Visual fixations to target, competitor, and filler pictures in the visual world task from 500ms pre-word onset to 2000ms post-word onset. Fixations to competitors and fillers in the display were analyzed using growth curve analysis from −250ms pre-word onset, where fixation curves began to diverge, to 750ms post-word onset, the time of peak target fixation indicating target selection.

The effects of target viewing and spoken word comprehension on activation of L1 orthographic competitors were assessed using growth curve analysis. Visual fixations to competitor and filler objects in the display were analyzed from −250ms pre word onset to 750ms post word onset (Figure 7, left). The base fourth-order polynomial time model was significantly improved by adding an effect of Object Type (orthographic competitor vs filler) ΔLL = 11.43, X²(5, 41) = 21.42, p < .001). Object Type had significant effects on the intercept (Estimate = 0.024, SE = 0.005, z = 4.60, p < .001), linear (Estimate = −0.071, SE = 0.027, z = −2.58, p < .05) and cubic terms (Estimate = 0.066, SE = 0.021, z = 3.12, p < .01). These results indicate that participants viewed competitor pictures more than fillers, and that competitor fixations peaked early, and then decreased to the filler baseline in the latter half of the window. The difference in competitor and filler fixations prior to word onset suggests that the visual stimuli alone activated orthography in both English and Colbertian.

Figure 7.

Fixations to competitor and filler locations from −250ms to 750ms when the competitor was present (left) and absent (right). When competitors were present, participants were more likely to fixate the locations of cross-linguistic orthographic competitors (e.g., competitor CANE for target /gufɔ/ spelled HANE, solid line and black circles) than non-overlapping filler pictures (dashed line and white circles) in a display. In contrast, when competitors were absent (control trials), participants made similar fixations to controls and fillers. Circles mark observed data, lines are best fit fourth-order growth curve models, and zero time indicates target word onset.

To ensure that the difference in competitor and filler fixations was not affected by visual characteristics of the stimuli, we conducted the same model comparisons in the Competitor-Absent Control trials. Control trials had the same structure as experimental trials, except that pictures of competitor items were presented in trials where they did not overlap with the target either orthographically or phonologically. In this analysis, Object Type did not affect model fit ΔLL = 3.47, X²(5, 41) = 6.94, p > .1), indicating that participants fixated competitor and filler pictures similarly when there was no orthographic overlap present among pictures in the display (Fig. 7, right).

To examine whether the effects of Object Type were moderated by the wordlikeness of the targets’ orthographic forms, an additional set of analyses were conducted on fixations of High- and Low-Englishlike items. The base fourth-order polynomial time model included fixed effects of all time terms plus random effects of item on all time terms. The base model was significantly improved by adding fixed effects of Object Type (orthographic competitor vs filler), Englishlikeness (high vs low), and their interaction (ΔLL = 13.30, X²(15, 51) = 26.61, p < .05). Removing the effect of Object Type from the full model significantly reduced model fit (ΔLL = 9.50, X²(10, 51) = 19.08, p < .05), indicating that competitors were fixated more than fillers when controlling for Englishlikeness. Removing the effect of Englishlikeness from the full model led to a marginal reduction in model fit (ΔLL = 8.50, X²(10, 51) = 17.12, p = .072). Parameter estimates revealed a significant Object Type x Englishlikeness interaction on the quadratic time term (Estimate = −0.13, SE = 0.05, t = −2.45, p < .05). Separate analyses of trials with High- and Low-Englishlike targets revealed no significant effects of Object Type for targets with High-Englishlike written labels (ps > .05), but significant effects of Object Type on the intercept (Estimate = 0.032, SE = 0.015, z = 2.17, p < .05) and quadratic terms (Estimate = −0.117, SE = 0.042, z = −2.82, p < .01) for targets with Low-Englishlike written labels (Fig. 8). These findings suggest that activation of spoken words’ written forms varies as a function of initial word learning. Because Low-Englishlike words were learned better than High-Englishlike words during initial acquisition, fixations to Low-Englishlike competitors were greater than to High-Englishlike competitors during subsequent spoken word processing.

Figure 8.

Fixations to competitor and filler locations from −250ms to 750ms in response to targets with high-Englishlike (left) and low-Englishlike (right) orthographic forms. Competitor (solid line and black circles) and control (dashed line and white circles) fixations did not significantly differ for targets with high-Englishlike orthographic forms (e.g., HANE). In contrast, participants were more likely to fixate the locations of cross-linguistic orthographic competitors than non-overlapping filler pictures in response to targets with low-Englishlike orthographic forms (e.g., ROKA). Circles mark observed data, lines are best fit fourth-order growth curve models, and zero time indicates target word onset.

3.3. Discussion

In a spoken word comprehension task using eye tracking, we observed cross-linguistic orthographic interference from native language orthography-to-phonology mappings when listening to non-native words. Pictures in a visual display led to co-activation of corresponding English and Colbertian word forms. For example, auditory-only presentation of the novel word /huwi/, which is spelled RAKO in Colbertian, resulted in increased visual fixations to a picture of a rake in the display (because of the orthographic overlap between the Colbertian RAKO and English RAKE and despite different pronunciations), even though no orthographic input was present. Because RAKE does not overlap with any properties of the auditory input itself, this pattern can only be observed if the spoken word activates its corresponding non-native orthographic form, which then spreads activation to similarly-spelled words in the native language.

Despite differences in orthography-to-phonology mappings across native and non-native vocabulary in the current study, orthographic forms in both English and Colbertian became activated, resulting in increased eye movement fixations to English competitors. This finding is consistent with previous research reporting that orthography within and across languages influences spoken word comprehension at early stages of processing (Hoshino & Thierry, 2011; Perre & Ziegler, 2008). Notably, we found that competitor fixations exceeded filler fixations even prior to the onset of the spoken target, indicating that both native and non-native orthographic labels can be co-activated in response to objects’ visual forms.

Interestingly, we observed stronger activation of novel words’ written forms when they were orthotactically less typical of native language words. This effect of Englishlikeness may be due to the superior learning of Low-Englishlike words during the acquisition stage (Experiment 1). While overt exposure to native-like written words is likely to increase L1 activation (e.g., from HANE to CANE), potentially resulting in greater interference and slower learning, the activation of a spoken word’s orthography based on novel letter-sound mappings (e.g., from /gufɔ/ to HANE) is likely to be contingent on how well the non-native vocabulary was learned.

In sum, the observed pattern of results suggests high interactivity across lexicons and modalities during non-native word processing and demonstrates that native language orthography and orthography-to-phonology mappings are activated during the spoken processing of non-native vocabulary. Finding co-activation both across lexicons (native and non-native) and across modalities (auditory and visual) reveals a language system that is highly interactive and suggests a highly interconnected linguistic architecture.

4. Limitations and Future Directions

Languages often differ from each other in aspects ranging from sublexical properties to pragmatic use. Laboratory-created linguistic stimuli provide a useful means of isolating variables of interest with a level of precision that cannot be obtained with natural language stimuli. While evidence suggests that similar cognitive and neural processes are recruited to process natural and artificial language input, including nonwords as in the present study (Ettlinger et al., 2016; Friederici, Steinhauer & Pfeifer, 2002; see Hayakawa, Ning, & Marian, 2020 for a review), we note that the generalizability of the present findings should be confirmed in future studies incorporating more complex, naturalistic stimuli.

For instance, while the present study was designed to test for the impact of conflicting orthography-to-phonology mappings while controling for difficulties associated with the acquisition of novel letters and sounds, it is likely that native language activation and interference during natural language processing would be moderated by familiarity with the novel words’ sublexical information (see Ziegler & Goswami, 2005). Additionally, unlike the artificial vocabulary used in the present study (for which the sounds of each letter maximally differed from that of the native language), natural language pairs include both distinct and overlapping features. The inability to globally disregard existing language knowledge may therefore contribute to even greater interference during the acquisition of natural languages. The results of the present study are therefore likely to provide a conservative estimate of native language influence during non-native vocabulary acquisition. Indeed, our finding that the challenges associated with conflicting mappings are exacerbated for words that orthotactically resemble the native language speaks to the variable competition that can emerge depending on the extent of native and non-native similarity. The fact that evidence of native language co-activation and interference was found despite the potentially easier task of globally suppressing L1 mappings speaks to the degree of interactivity within the linguistic system.

Similarly, there is reason to expect that the present findings obtained with English speakers may reflect a lower level of native language interference than might be found with speakers of other languages. According to dual-process models of visual word recognition, a written word’s meaning can be accessed both through direct activation of semantics from orthography, as well as via phonological mediation, whereby graphemes activate corresponding phonological representations before further lexical processing (Coltheart et al., 2001, Harm and Seidenberg, 2004). Neurophysiological correlates of grapheme-phoneme conversion (e.g., the N320 ERP component; Bentin et al., 1999) indicate that speakers of more orthographically-transparent languages are more likely to rely on sublexical phonological mediation during written word processing than speakers of less transparent languages (Simon et al., 2006). Given that English is characterized by a notably low degree of orthographic transparency, the interference caused by L1 grapheme-phoneme conversion is likely to be greater for speakers of more transparent languages. This hypothesis could be evaluated in future studies by observing whether speakers of more orthographically transparent languages experience greater difficulty learning words with conflicting letter-to-sound mappings. ERP measures may additionally shed light on the underlying mechanisms, for instance by confirming whether native language interference is predicted by individual and cross-linguistic differences in grapheme-phoneme conversion (indexed by the N320) at different stages of learning. Evidence of greater grapheme-phoneme conversion during initial exposure to non-native words is likely to be correlated with the activation of competing native language phonemes, and in turn, slower acquisition of the novel words and mappings. Greater grapheme-phoneme conversion at later stages of learning, on the other hand, may reflect phonological activation based on either native or non-native rules. To the extent that deeper non-native alphabetic knowledge facilitates word learning, we should find that increased phonological mediation is associated with slower learning in the case of the former (i.e., activation based on native language mappings), and faster learning in the case of the latter (i.e., activation based on non-native mappings).

The use of electrophysiological measures in future research could inform our understanding of the mechanisms underlying the superior acquisition of words with less typical native language forms. One possibility is that the advantage for less Englishlike words stemmed from reduced activation of English language knowledge (i.e., its letter-to-sound mappings). Alternatively, it may be the case that reduced similarity to English minimized reliance on English language strategies. Simon et al. (2006) found that Arabic-French bilinguals relied on grapheme-phoneme conversion to a greater extent when making lexical decisions in French (a language with relatively consistent letter-sound mappings) than in Arabic (for which the same letter often has multiple different pronunciations). In other words, while the relative lack of orthographic transparency in Arabic may have initiated more lexically-based processing of Arabic words (wherein semantic information is accessed directly from orthographic wordforms), individuals switched to a more phonologically-mediated strategy when processing more orthographically transparent French words. Given that, unlike English words, the Colbertian vocabulary had consistent letter-to-sound mappings, a reduction in orthotactic similarity to English may have facilitated Colbertian grapheme-phoneme conversion (as opposed to whole word processing). Greater phonological decoding based on non-native mappings could have contributed to deeper alphabetic knowledge and superior word learning. To the extent that the low-typicality advantage stems from reduced activation of conflicting L1 letter-to-sound mappings, we should see that better learning of less nativelike words is mediated by a reduction in grapheme-phoneme conversion (i.e., smaller N320 amplitudes). If the low-typicality advantage instead results from reduced reliance on inefficient native language strategies (e.g., lexically-based processing of low transparency words), we should observe that better learning of less nativelike, orthographically-transparent words will be mediated by increased grapheme-phoneme conversion (i.e., larger N320 amplitudes).

5. Conclusions

Using carefully designed artificial vocabulary and eye-tracking methodology, we were able to take a closer look at the interactivity in the language system, including the variables that moderate the extent of native language interference during non-native word learning and processing. Our findings reveal that individual differences and linguistic characteristics predict acquisition of novel orthography-to-phonology mappings. Critically, the results demonstrate for the first time that both native and non-native mappings are activated during auditory processing of non-native words.

In sum, the study generated two sets of main findings. The first (from Experiment 1) is that conflicting L1 letter-to-sound mappings impacted non-native word learning, to varying degrees depending on individual differences in cognitive abilities and on characteristics of the word stimuli. Novel words with more native-language-like orthographic forms were harder to learn, likely due to increased competition from conflicting L1 letter-to-sound mappings. Word learning was predicted by individual differences in nonverbal intelligence. Based on prior findings that nonverbal intelligence facilitates extraction of novel patterns and regularities, we propose that the word learning advantage associated with higher IQ likely stems from more efficient acquisition of non-native orthography-to-phonology mappings. Support for this interpretation comes from our finding that individuals with higher IQ were also more successful at generalizing non-native mappings to the recognition and production of previously unseen novel words, which was in turn associated with superior acquisition during formal training. Individual differences in nonverbal reasoning may therefore be especially likely to translate to individual differences in pattern-based learning of novel linguistic features. While we found that individual differences in verbal working memory predicted learning during the earliest stages of acquisition, the advantages of better short-term memory declined over the course of training and did not extend to knowledge of individual letter-sound pairings (as assessed by the generalization tasks). We therefore suggest that nonverbal IQ and verbal working memory have distinct effects on the trajectory of word learning.

The second, most novel set of findings (from Experiment 2) is that both native and non-native orthography-to-phonology mappings were co-activated when hearing the non-native words, as well as when viewing the visual objects prior to the onset of the spoken word. We conclude that the cross-linguistic co-activation of native language orthography when hearing non-native words indicates simultaneous activation of both native and new orthography-to-phonology mappings during spoken word processing. These findings suggest a cascading sequence of spreading activation -- from phonology to orthography and across the two orthographic systems – and demonstrate that spoken word recognition is a dynamic process, where lexical alternatives that are several steps removed from the auditory input can affect target word processing (Marian, 2023). Such cascading activation is a testament to the dynamic interactivity across languages and modalities in the human mind.