The Slow Development of Real-Time Processing: Spoken-Word Recognition as a Crucible for New Thinking About Language Acquisition and Language Disorders

Abstract

Words are fundamental to language, linking sound, articulation, and spelling to meaning and syntax; and lexical deficits are core to communicative disorders. Work in language acquisition commonly focuses on how lexical knowledge—knowledge of words’ sound patterns and meanings—is acquired. But lexical knowledge is insufficient to account for skilled language use. Sophisticated real-time processes must decode the sound pattern of words and interpret them appropriately. We review work that bridges this gap by using sensitive real-time measures (eye tracking in the visual world paradigm) of school-age children’s processing of highly familiar words. This work reveals that the development of word recognition skills can be characterized by changes in the rate at which decisions unfold in the lexical system (the activation rate). Moreover, contrary to the standard view that these real-time skills largely develop during infancy and toddlerhood, they develop slowly, at least through adolescence. In contrast, language disorders can be linked to differences in the ultimate degree to which competing interpretations are suppressed (competition resolution), and these differences can be mechanistically linked to deficits in inhibition. These findings have implications for real-world problems such as reading difficulties and second-language acquisition. They suggest that developing accurate, flexible, and efficient processing is just as important a developmental goal as is acquiring language knowledge.

Words are fundamental to language, linking sound, articulation, and spelling to meaning and syntax. Because of this centrality, lexical-processing deficits may underlie higher-level language disorders (Nation, 2014; Stothard et al., 1998). Consequently, vocabulary acquisition is a central topic in the study of language development (Samuelson & McMurray, 2017). The majority of this work emphasizes knowledge: How do children learn the meanings of words given the ambiguity in word-learning situations?

But learning is only half the challenge. Even someone with a robust vocabulary must engage active processes to use that knowledge in the moment. Words have multiple meanings and must be interpreted relative to the context in which they are used (Elman, 2009). Even at the level of sound, words are briefly ambiguous. The onset of “wizard” (“wi-”), for example, is consistent with many words (“wizard,” “whistle,” “window,” etc.). Listeners must cope with this temporary ambiguity to efficiently process language. Children must solve these problems to effectively use the rich knowledge that words bind together (e.g., sound, spelling, meaning). Thus, becoming a skilled language user requires learners to acquire both word knowledge (sound patterns and meanings) and real-time skills to use that knowledge in the moment.

Our own work has addressed the question of how children acquire these real-time skills by examining processing of highly familiar words in school-age children. Studying this age group, rather than infants and toddlers, who have been the focus of much work in lexical development, makes it possible to use adult psycholinguistic methods that go beyond speed of processing to measure dynamically unfolding lexical representations more directly. Even in this context, it is difficult to dissociate differences in processing from differences in knowledge. A given child may have less efficient or less flexible real-time skills than another because the child’s skills are not fully developed (even though the child’s linguistic knowledge is) or because the child’s representations are incomplete in some way. Our argument is that changing the emphasis to processing raises critical questions about how children become skilled language users that are not addressed by standard approaches that focus on acquiring word knowledge.

Spoken-Word Recognition

The process of mapping sound to meaning is served by competition (Dahan & Magnuson, 2006). When listeners hear the onset of a word (e.g., the “wi-” in “wizard”), they activate a range of candidates (e.g., “whisper,” “window”), which compete (Fig. 1a). This process does not passively reflect the unfolding input. Competition dynamics are biased by word frequency and inhibition from neighbors, and words without a strict temporal match are activated—listeners might activate “lizard” after hearing “wizard,” or “pack” after hearing “cap.” Thus, competition weighs and combines evidence for words to flexibly cope with ambiguity.

Fig. 1.

Examples of lexical competition. The graph in (a) shows activations from the TRACE model (McClelland & Elman, 1986) after the model hears the target word “wizard.” Over time, TRACE activates a wide range of candidate words, which start from a resting level of 0, and many of these words are ultimately inhibited (activation below 0 by the end of processing). Competitors can include cohorts (which overlap with the target word at onset), rhymes (which overlap with the target at offset), and other types of words. The diagram in (b) illustrates a trial in a typical study using the visual world paradigm (Allopenna et al., 1998); participants hear a target word (here, “wizard”) in the context of pictures representing that word, competitors (a cohort competitor, “whistle,” and a rhyme competitor, “lizard”), and an unrelated baseline word (“beaver”). The graph in (c) shows an example time course of fixations in this paradigm as a trial unfolds (data from the typically developing group of McMurray et al., 2010). Early in the trial, listeners fixated both the target (“wizard”) and the cohort competitor (“whistle”). Later, they began to suppress the cohort competitor, but also fixated a rhyme competitor (“lizard”). By around 1 s, they were fully committed to the target.

This view is supported by measures that trace the time course of competition in real time. In the visual world paradigm (VWP; Fig. 1b; Allopenna et al., 1998), listeners hear a word and select its referent from an array of pictures including the target and competitors. To locate the correct picture, participants make several fixations to the various pictures while lexical processing unfolds. Critically, fixations reflect processing at the point when they are launched. Thus, at the onset of “wizard” (“wi-”), listeners may fixate the pictures of the target and cohort competitor (“whistle”); however, as they hear more of the word, fixations on the picture of the cohort competitor decrease, even as a rhyme competitor is considered (Fig. 1c).

The Development of Real-Time Word Recognition

The first developmental studies of real-time lexical processing used a simplified VWP with toddlers (e.g., Fernald & Marchman, 2012). The researchers found that development in the speed of word recognition predicts later language ability. Our work built on this by studying older children (9- and 16-year-olds) using more sophisticated VWP tasks that more fully characterize the dynamics of word recognition (Rigler et al., 2015). We found clear evidence of development: Sixteen-year-olds converged on the target and resolved competition more rapidly than 9-year-olds (Figs. 2a and 2b; see also Sekerina & Brooks, 2007). At the time of this study, speech and word recognition were thought to develop largely during the infant and toddler years (Werker & Curtin, 2005) and to be relatively stable afterward. By that standard, the time course of development demonstrated in this study was shockingly protracted.

Fig. 2.

Comparison of school-age children’s fixations to targets and competitors as a function of time. The first two columns show the proportion of fixations to targets and cohort competitors in (a, b) typically developing 9- and 16-year-olds (Rigler et al., 2015) and (c, d) adolescents with and without developmental language disorder (McMurray et al., 2010). The column on the right shows (e) the proportion of fixations to targets in prelingually deaf school-age children who wore a cochlear implant and age-matched control children with normal hearing and (f) the difference between the proportion of fixations to cohort competitors and unrelated words in the same two groups (McMurray et al., 2017).

Ongoing development through adolescence was highlighted by a study examining even lower-level speech perception skills (McMurray et al., 2018). Children’s sensitivity to the structure of speech categories was tested with the VWP using continua spanning words differing only in the initial consonant (e.g., “bear” vs. “pear”). The auditory stimuli varied in small uniform steps of voice-onset time (VOT), an acoustic cue that contrasts “b” and “p.” (A second set of continua spanning a different contrast was also tested, but we focus here on the “b”–“p” continua.) The children heard the auditory stimuli and clicked on the matching pictures while their eye movements were monitored. We analyzed their ultimate choices (mouse clicks) to construct categorization functions showing how their decision changed with changes in VOT (Fig. 3a). These functions showed a steep slope indicating a clear delineation between categories even among 7-year-olds, as well as a small (though significant) increase in steepness over development (see also Hazan & Barrett, 2000).

Fig. 3.

The development of speech categorization. The graph in (a) shows the proportion of “p” responses (mouse clicks on the picture showing something referred to by a word beginning with “p”) as a function of voice-onset time (VOT) step in typically developing 7- to 8-year-olds, 11- to 12-year-olds, and 17- to 18-year-olds. VOT is expressed relative to each participant’s own boundary, such that 0 is the participant’s boundary, −1 is one step to the left, and so forth. The graphs in (b) through (e) show fixation bias at four different time points (300 ms, 500 ms, 700 ms, and 1,700 ms) for the same three age groups. Fixation bias was computed from the relative proportions of looking to objects beginning with “b” versus objects beginning with “p.” A score of −1 indicates full commitment to “b” (all fixations were on the “b” object), whereas a bias of +1 indicates complete commitment to “p.” The graphs in (f) and (g) show the proportion of looks to the competitor as a function of distance from the category boundary, for trials in which the child chose the “correct” phoneme; results for typically developing children are shown in (f), and results for adolescents with and without developmental language disorder are shown in (g). Data for (a) through (f) are from McMurray et al. (2018), and data for (g) are from McMurray et al. (2014).

More dramatic developmental changes were seen earlier in processing by using the fixation record to construct categorization functions at different points in time prior to the final response. For example, the fixations 500 ms into the trials (Fig. 3c) were used to create a similar measure of categorization at an earlier moment before the final decision was made. These categorization functions revealed the relative commitment to one phoneme or the other as processing unfolded in time (Figs. 3b–3e; for animations, see the files available at OSF, at http://osf.io/w5bqg). At 300 ms after the onset of the word (Fig. 3b), 7- to 8-year-old children showed a relatively flat response across the continuum, which suggests that by this moment, they had not yet begun to categorize the sounds. Older children showed a steeper curve, even as there were differences between 12- to 13-year-old and 17- to 18-year-old children. At later times (Figs. 3c–3e), the categorization functions expanded, but developmental differences were evident at all time points.

Moreover, we also used the fixation data to investigate whether children track fine-grained changes within a phonemic category (i.e., do they respond differently to changes in VOT, even for VOTs within the same category?). We found that the 11- to 12-year-olds and 17- to 18-year-olds showed gradient sensitivity to VOT: The amount of competition fell off as VOT departed from the category boundary at 0. The younger children showed reduced or even no such sensitivity (Fig. 3f). These results suggest increasing attunement to acoustic and phonetic detail between the ages of 7 and 18 (McMurray et al., 2018) and contrast with standard perceptual-narrowing accounts (Maurer & Werker, 2014), which posit that children lose access to within-category differences as they develop.

Again, the ongoing development of these speech perception processes during school age and even adolescence is surprisingly protracted. Obviously, the youngest children in these studies (i.e., 7- to 9-year-olds; McMurray et al., 2018; Rigler et al., 2015) have the relevant knowledge. A typical 7-year-old knows thousands of words (and these experiments used simple words, such as “bees” and “ship”), and categories such as the sounds “b” and “p” are in place by the age of 4 months (Galle & McMurray, 2014). Yet the results show development in the ability to use this knowledge.

Why do these real-time processes develop so slowly? Word recognition (and much of language) requires learners to simultaneously be accurate, efficient, and flexible. Listeners could wait until the end of a word to begin lexical access, bypassing temporary ambiguity and maximizing accuracy. However, this would be slow, as downstream processes would have to wait for the end of the word. Conversely, if listeners commit to a target and suppress competitors too rapidly, this would be inflexible—an early misperception would be hard to overcome when later information arrives. A goal of development may be to tune processing to the demands of efficiency and flexibility.

Individual Variation in Language Ability (Language Disorders)

The standard view is that language disorders derive from incomplete “knowledge” (e.g., number of words known, the depth of lexical knowledge), driven by differences in learning (Ullman et al., 2020). In contrast, we hypothesized that if real-time lexical competition plays a functional role in language ability, then language difficulties may derive from difficulties achieving efficiency and flexibility. Our first study examined developmental language disorder (DLD; previously termed specific language impairment). DLD affects 12% to 15% of children (Tomblin et al., 1997) and is characterized by poor language ability without obvious cause. DLD may not be a distinct disorder, but rather may simply be the low end of the ability range (Reilly et al., 2014). Indeed, it is diagnosed with continuous measures scored against a normative sample of corresponding age. Thus, DLD is a convenient proxy for variation in language ability.

This first study (McMurray et al., 2010) used the same VWP task whose results are summarized in Figures 2a and 2b (Rigler et al. (2015), but with older adolescents with and without DLD. Children with lower language ability were as quick as typical children to initially activate candidates (Figs. 2c and 2d). However, later in the trial (when the proportions of fixations reached asymptote), target fixations were lower among the children with DLD than among those without DLD, and competitors were never fully suppressed among the children with DLD, even as they chose the correct object. This profile is consistent with studies using other paradigms (Dollaghan, 1998; Helenius et al., 2009), and was not related to nonverbal cognitive ability.

Such a deficit could relate to overall language ability via multiple routes. If a child keeps multiple candidates active for each word, this could affect downstream sentence processing. Alternatively, given that competition is thought to underlie every level of language, the deficits observed in this study could be a biomarker of poor competition resolution throughout the system.

Dimensions of Individual Differences in Processing

These studies suggest that variation in language ability over development has a profile distinct from that of variation in language ability within an age (DLD). Over development (Figs. 2a and 2b), lexical competition becomes initiated and resolved increasingly quickly, whereas DLD is associated with lingering unresolved competition (Figs. 2c and 2d). Children with DLD are not just behind; they cope with temporary ambiguity differently and less effectively than other children. We suggest that these profiles reflect two latent traits: activation rate and competition resolution. Though we focus our discussion on lexical processing, the fact that language processing at every level may use activation/competition mechanisms means that this framing could be extended to many levels of language.

Activation rate

Activation rate is the rate at which information in the inputs affects lexical activation. This could be determined in part by the fidelity of perceptual encoding—greater perceptual fidelity provides more information to discriminate words. This possibility is supported by our finding that children become increasingly sensitive to fine-grained acoustic detail across development. As shown in Figure 3f, among 17- to 18-year-olds, competition between two candidates along an acoustic continuum falls off as the stimuli depart further in either direction from the boundary at the middle of the continuum. This link between activation rate and perceptual encoding is also seen in children with hearing impairment who use cochlear implants (McMurray et al., 2017) and in adults in challenging listening conditions (Hendrickson et al., 2020; McMurray et al., 2017). In these situations, the time course of word recognition can be delayed by as much as 250 ms (Fig. 2e) as each “slice” of the input contains less information than when hearing is good or when listening conditions are less challenging. This results in less consideration of cohorts (Fig. 2f), a pattern colloquially termed wait and see. Thus, activation rate is not simply processing speed; variation in activation rate has complex consequences, affecting how much different candidates are considered.

Over development, changes in activation rate likely derive from changes internal to the lexical system. Computational modeling suggests that activation rate is affected by the strength of connections between phonemes and words (McMurray et al., 2010) or by the degree to which irrelevant connections between phonemes and words are pruned (McMurray et al., 2012). In such models, phonological representations of words are stored in these connections. Thus, improvements in activation rate could reflect refinement in the stored sound patterns of words. However, there may be across-the-board individual differences reflecting a latent trait; that is, across all words, some children could have higher or lower activation rates (more or less robust connections) than other children. Either way, improving this pathway would speed up both the initial commitment to candidates (activating “wizard” and “window” after “wi-”) and the elimination of competitors (because disconfirming input has a larger impact).

Competition resolution

Competition resolution reflects the degree to which competitors are ultimately suppressed when the system stabilizes. Incomplete resolution is typically maladaptive (see Gernsbacher, 1997, for related ideas regarding higher-level aspects of language). However, in cases of phonetic ambiguity or hearing difficulty, it may be useful to keep competitors available in case revisions are needed.

Several mechanisms may contribute to resolution. Simulations in one study (McMurray et al., 2010) link resolution to decay of activation (see also empirical evidence in Helenius et al., 2009). When the target word cannot maintain its activation, it inhibits competitors less, giving rise to effects such as those illustrated for adolescents with DLD in Figures 2c and 2d. Lateral inhibition among words within the lexicon (Dahan et al., 2001) may also contribute: As one word becomes more activated than its neighbors, it suppresses them. This form of inhibition is distinct from domain-general cognitive control (and not correlated with it: Kapnoula & McMurray, 2021), as well as conceptions of inhibition from classic learning theory (e.g., pruning connections). Lateral inhibition cannot have effects until words become somewhat activated; thus, its effects arise late, as a sort of cleanup operation.

This framing suggests two predictions for DLD. First, researchers have proposed that DLD has a perceptual locus (Goswami et al., 2016). If so, the activation rate of individuals with DLD should be weakened, as perceptual deficits affect the ability to extract information from the signal. In one study (McMurray et al., 2014), we tested for a perceptual deficit in adolescents with DLD by investigating whether they showed different levels of sensitivity to within-category differences compared with typically developing adolescents (e.g., flatter functions, as shown for 7- to 8-year-olds in Fig. 3f, which presents results for a similar paradigm). They did not, despite showing greater competition across all continuum steps compared with the typically developing group (Fig. 3g). Second, this framing suggests that DLD is associated with impairment in lexical processes, such as inhibition. We recently tested for lexical inhibition using a paradigm in which the target word’s acoustics were manipulated to boost a competitor (which would be expected to inhibit the target). In this paradigm, participants with DLD showed no evidence for inhibition (McMurray et al., 2019). Together, these results implicate a distinctly lexical deficit in DLD that is isolated to competition resolution.

A Nonlinguistic Locus?

A critical question is whether the differences in language processing associated with DLD or the differences in language processing among children compared with adults reflect differences in lexical processing or differences in task-specific factors (e.g., eye movement control) or domain-general skills (e.g., working memory, cognitive control). Converging evidence suggests that differences in nonlinguistic processing are not the whole story. Effects of DLD are not due to differences in eye movement control (McMurray et al., 2010), and they mirror effects found in other paradigms (Dollaghan, 1998; Helenius et al., 2009). Moreover, effects of DLD on competitor fixations are significant even after unrelated fixations are subtracted or partialed out (McMurray et al., 2014, 2018; Rigler et al., 2015). Thus, task-specific factors cannot be the sole explanation. With respect to domain-general cognitive differences, our DLD studies have shown no effects of DLD on nonverbal cognitive skills (McMurray et al., 2010). Additionally, although our work comparing cochlear-implant users with normal-hearing peers shows robust differences in auditory word recognition between these two groups (Figs. 2e and 2f), we found no such differences in a completely nonspeech analogue to the VWP (McMurray et al., 2017). It is unclear whether some of the age-related differences shown in Figures 2 and 3 derive in part from general cognitive development (e.g., cognitive control), and ongoing longitudinal work may achieve a more definitive answer.

Implications and Future Directions

This work suggests that research on language development must move beyond questions of how a child “knows” words to investigate what the child can do with them in the moment. We have focused on one process: accumulating auditory input in order to recognize a word form. However, ambiguity can arise at multiple levels. Even if a child knows the word “chicken,” the child has to deal with phonetically reduced forms of the word (“chi’n”), temporary ambiguity of the word itself, and ambiguity in the intended meaning (the food? the animal?). The real-time mechanisms that sort this out are as important to language competency as is knowledge.

Consideration of real-time processing changes the developmental story

The canonical story is that speech and word recognition develop during infancy and toddlerhood (Werker & Curtin, 2005): Infants first master the sounds of language and then master higher-level skills. However, an examination of real-time processing suggests that these skills develop slowly—perhaps concurrently with higher-level skills. This perspective offers new solutions to the classic problem of how children learn the sounds of language. Infants know few words, cannot produce speech, and do not get feedback on their perception. If speech sounds are acquired early, the only available mechanism for such acquisition may be some form of perceptual learning that works without feedback (Werker & Curtin, 2005). In contrast, if speech development is slow, learning could take advantage of rich sources of feedback, such as the later-developing lexicon, speech production skills, richer social interactions, and reading instruction.

Going beyond the questions of how children acquire language, one might ask if the real-time processes involved in spoken-word recognition are plastic. Recent work suggests that lexical inhibition is amenable to training (Kapnoula & McMurray, 2016): Brief training on tasks that encourage competing words to be coactive (e.g., discriminating similar words) can strengthen inhibitory pathways and help participants recover from interference. It is unclear if other aspects of real-time processing are amenable to training. However, such plasticity may be how resolution is tuned to maximize flexibility and efficiency, and it could pave the way to remediation approaches for DLD.

Finally, how do children learn to balance efficiency, flexibility, and accuracy? School may be a catalyst for this, as after children enter school, they are exposed to more words, to reading, and to a more diverse set of talkers. The development of real-time language skills may also hinge on mechanisms that tune the system to minimize the cost of an error (as does the development of many other skills). Consequently, listeners may need experience mishearing words in order to develop this kind of flexibility. The fact that mistakes are likely infrequent may explain why development is protracted. Moreover, the richer diversity of speakers and more challenging input of the school setting may offer opportunities for mishearing, and thus broader experience to push the system.

Other domains

Our broader hypothesis is that between-individuals differences in activation rate reflect differences in development, or skill acquisition, whereas between-individuals differences in resolution reflect more firmly established differences in processes that stabilize representations later in processing. The constructs of activation rate and resolution may be useful in other domains as well. For example, written-word recognition relies on competition mechanisms similar to those that spoken-word recognition relies on (Rastle, 2018). Thus, these constructs may prove similarly diagnostic for understanding variation in reading development or reading disorders (cf. Roembke et al., 2021). Second-language acquisition may present similar challenges, as learners must recognize words with poor representations and manage competition from two lexica. Diagnostics of activation rate and resolution can reveal whether poor performance is due to a lack of practice or difficulties managing competition (Sarrett et al., in 2022).

Knowledge Versus Processing

At a neural level, representation (knowledge) may seem dissociable from processing. Neural mechanisms that contribute to learning representations (e.g., forming, pruning, or strengthening synapses) differ from those involved in processing (activation flow, lateral inhibition). Yet representation and processing may each be shaped by the other: Synaptic changes are widely believed to be proportional to the amount of activity; processing takes place over learned connections. This makes it difficult to dissociate representation from processing, and we are not advocating for a strong distinction.

However, a focus on processing poses new questions concerning problems (that children must solve) that are entirely different from those that motivate typical learning-based accounts of lexical development. Moreover, conceptualizing language acquisition along both dimensions has practical value. Teachers may want to teach children new words or determine whether they “know” a word (knowledge), but might also want to assess how efficient their processing is. Nonetheless, mechanistic theories that can relate learning and processing are needed.

Some researchers would argue that learning and processing are isomorphic. In connectionist accounts, knowledge is stored in the connections over which processing occurs (Elman, 2009)—the sound pattern of “bug” is stored in its connections to “b,” “u,” and “b.” Consequently, tuning these connections (over development) improves both knowledge and processing. Even in traditional information-processing accounts (Perfetti, 2007; Yap et al., 2009), lexical knowledge is a continuum of quality, or integrity, along multiple dimensions (semantic, orthographic, phonological).¹ Under both views, relatively poor real-time processing derives from learning that is incomplete (at early stages of development) or impaired (in the case of DLD; Ullman et al., 2020). However, it is unclear in this kind of account why two groups with equivalently poor states of knowledge (young children, children with DLD) exhibit different profiles of processing. Thus, these accounts may need to be developed so that even within the phonological level, lexical quality is not a single dimension: Different aspects of lexical representation must differentially predict activation rate or resolution. Competition models offer some leverage for disentangling mechanisms of representation and processing (McClelland & Elman, 1986; McMurray et al., 2012). In such systems, some connections (e.g., between phonemes and words) reflect stored representations, whereas other connections (inhibition and feedback) enable flexible or efficient processing. Thus, the type and function of the pathways being formed or tuned may help distinguish processing from knowledge. In practice, dissociating processing and knowledge may be challenging. For example, stronger bottom-up pathways (“knowledge” of word forms) could boost activation of competitors and create more interference, even if inhibitory connections are stable. Future work may disentangle such effects by using more sophisticated experimental designs. It may also be possible to leverage the match between the VWP and competition models (Allopenna et al., 1998) by varying specific parameters controlling the strength of bottom-up activation, inhibition, and decay in the model and relating this variation to variation in VWP performance (Kapnoula & McMurray, 2016; McMurray et al., 2010).

The functional differences we have established (activation rate vs. resolution) are meaningful regardless of their precise instantiation. Our focus on processing raises new questions. In a knowledge-centered view, good processing derives solely from robustly learned representations. However, in our framework, processing skills could develop on their own to balance the demands of accuracy, efficiency, and flexibility (Kapnoula & McMurray, 2016). Real-time processing could also influence what is learned: Better resolution of competition may help learners achieve more precise word-referent mappings (Apfelbaum & McMurray, 2017).

It is easy to dismiss the developmental effects on spoken-word recognition that we have described here as differences in mere speed of processing: Who cares if a child is a bit slow to identify words? This attitude belies the rapid pace of language use. If the child is to understand sentences in real time, sentence processing cannot wait for word recognition to reach asymptote at 1,500 ms; word recognition is likely based on the state of lexical competition much earlier. Thus, poor sentence processing may derive from lexical-processing skills that are not yet fully developed. Consequently, the development of real-time processing may be a primary determinant of how well children can translate language knowledge into real-world language behavior.