Statistical language learning in infancy

Abstract

Research to date suggests that infants exploit statistical regularities in linguistic input to identify and learn a range of linguistic structures, ranging from the sounds of language (e.g., native-language speech sounds, word boundaries in continuous speech) to aspects of grammatical structure (e.g., lexical categories like nouns and verbs, basic aspects of syntax). This article presents a brief review of the infant statistical language learning literature, and raises broader questions concerning why infants are sensitive to statistical regularities. In doing so, we consider the relationship between statistical learning, prediction, and uncertainty reduction in infancy.

Over the past two decades, a substantial body of research has focused on statistical learning: the ability to detect patterns and regularities in the environment. While statistical learning has been investigated in participants across a range of ages, it is arguably most useful early in life; infants have the most to learn, and the least prior knowledge to guide that learning. The literature to date suggests that infants are sensitive to statistical regularities across an array of domains, including speech, music, actions, and visuo-spatial patterns (for a recent overview of infant statistical learning across domains, see Saffran & Kirkham, 2018). Because statistical learning has been investigated most extensively in the domain of language, this review will focus primarily on considerations of the role of statistical learning in language development. Note, however, that while this review is focused on first language learning by human infants, much of the ensuing discussion is also relevant to statistical learning research at other developmental time points and in other domains and species.

Infant statistical language learning

Early studies of infant statistical learning were designed to address a fascinating problem facing novice language learners: how do infants discover words in fluent speech? This is a notoriously difficult challenge because unlike written text, there are no spaces between words in fluent speech. Listen to a speaker of a language you don’t know (or watch a speaker of a sign language you don’t know), and it becomes quickly clear that pauses and other acoustic cues do not reliably indicate word boundaries (even in infant-directed speech or sign). Jusczyk and Aslin (1995) were the first to demonstrate that infants could recognize English words after hearing them embedded in sentences. This classic study provided the first evidence that infants could pick out the sounds of words from continuous speech stream.

The results of Jusczyk and Aslin’s (1995) studies, along with other research in the mid-1990s, provided clear evidence that infants can detect words in fluent speech. A number of possible cues might facilitate this process for infants, including acoustic cues (e.g., allophonic alternations), prosodic cues (e.g., lexical stress), and other boundary cues (e.g., words in isolation, presence of familiar words). Another possibly informative type of cue resides in the statistical regularities in speech. Syllables that are part of the same words tend to co-occur together more reliably than syllables that span word boundaries.

Saffran, Aslin, & Newport (1996) tested the hypothesis that infants are sensitive to the probabilities with which syllables co-occur as a way to break into the speech stream to find words. Eight-month-old infants were first exposed to a 2 minute stream of syllables, presented in a monotone, with no pauses or other auditory cues to word boundaries. The infants were then tested to determine whether they could distinguish the words in the speech stream from syllable sequences spanning word boundaries. The results suggested that infants were sensitive to the statistical regularities in the speech stream, distinguishing between words (in which syllables co-occurred with high probabilities) and sequences spanning word boundaries (marked by lower syllable co-occurrence probabilities). Notably, subsequent studies revealed that infants’ statistical sensitivity is not limited to simple artificial languages; infants can also use statistical regularities to detect word-like units in more ecologically-valid natural speech input (Pelucchi, Hay, & Saffran, 2009a, 2009b).

Infants are sensitive to a wealth of different types of statistical regularities in linguistic input (for an extensive theoretical review, see Erickson & Thiessen, 2015). The studies reviewed in the previous paragraph focused on conditional statistics such as transitional probabilities: the likelihood that one event predicts another (for example, the syllable “pre” tends to be followed by “ty”, “tend”, or “dict” in English). Infants are sensitive to these regularities in both directions; that is, they not only detect that “pre” tends to be followed by “ty”, but that “ty” is frequently preceded by “pre” (Pelucchi et al., 2009b). These types of patterns help infants to discover word-like units in fluent speech, generating candidate words that are available for mapping to meaning (Graf Estes et al., 2007; Hay et al., 2011). They can also be used to find non-adjacent relationships between words when other units intervene, at least under some circumstances (e.g., Gomez, 2012).

Notably, infants are not limited to tracking conditional statistics. Infants are also sensitive to distributional statistics. One language-relevant example pertains to determining whether a particular continuum of speech sounds corresponds to one or two phoneme categories. Consider the continuum of sounds between /ra/ and /la/. Some languages divide this continuum of sounds into two categories (e.g., English). Other languages treat the same continuum of sounds as members of a single category (e.g., Japanese and Korean). How do infants determine whether they are hearing a language with one or two phoneme categories represented in a single continuum? One solution to this problem is for infants to detect whether there are one or two peaks in the histogram – bumps in frequency of occurrence – along the continuum, which corresponds to the number of underlying speech categories (e.g., Maye et al., 2002). This type of statistical regularity is particularly important for acquiring the native language phonemic repertoire. Distributional statistics help to inform infants about how their language breaks up speech sounds into phoneme categories.

Infants also exploit statistical information in the context of mapping labels to referents. Figuring out which words refer to which objects or events in the world is a notoriously challenging problem: novel words frequently occur in the context of many possible referents. One potential solution is to track the correlations between the co-occurrences of labels and their potential referents. For example, consider the word pineapple. The first time a baby hears pineapple, it may be in the context of an array of unfamiliar tropical fruits, making it unclear which fruit pineapple refers to. With subsequent occurrences of the word pineapple, an infant can potentially track the correlation between the word and the particular items in her visual field, eventually narrowing down the mapping between pineapple and the correct fruit. While any individual situation may be ambiguous with respect to label-object pairs, infants can use cross-situational statistics to cope with widespread ambiguity in the mappings between words and the world (e.g., Smith & Yu, 2008).

Distributions of words can also help infants to discover lexical categories by tracking which words tend to co-occur with one another (e.g., Mintz, 2003). Lexical categories like nouns and verbs are not transparently signaled in the input; infants must exploit regularities in the speech that they hear (or signs that they see) to determine which items belong to which lexical categories. Statistical regularities in word sequences are highly informative for identifying these categories. For example, nouns in English are frequently preceded by words like “a” or “the”, while verbs are never preceded by words like “a” or “the”. Infants in artificial language learning studies are sensitive to these types of regularities when acquiring lexical categories (e.g., Lany & Saffran, 2010).

Perhaps most strikingly, infants can detect statistical relationships between lexical categories, suggesting that they are able to track patterns of abstract elements (Saffran et al., 2008). Grammatical regularities in natural languages frequently live in the relationships between lexical categories; as skilled language users, we can determine the grammaticality of novel sentences because we are familiar with the patterns that connect lexical categories in our language. Saffran et al. (2008) demonstrated that 12-month-old infants were sensitive to the grammaticality of sentences from an artificial language in which the pertinent regularities lay at the level of lexical categories, not individual words. While it is far from clear that infants can exploit statistical patterns to learn complex aspects of syntax, particularly those for which the evidence in the input is either infrequent or nonexistent (Han et al., 2016; Lidz et al., 2003), it is important nevertheless that infants appear to be able to learn relationships between lexical categories/abstract elements in lab learning tasks.

To summarize, infants appear to be sensitive to a wealth of different statistical regularities, at least in lab tasks. That said, there are still many outstanding questions about the nature of these mechanisms, including their neural mechanisms, their domain specificity, their commonalities and differences, and their relationship with other aspects of cognition, especially memory systems (for a recent theoretical approach to these issues, see Thiessen, 2017). In the next section, we turn to one such issue, concerning the factors that lead infants to track statistical regularities.

Why do infants track statistics?

As the evidence supporting the potential importance of statistical learning for theories of language development has mounted, questions have emerged concerning why human infants learn in this way. Infants are not obligated to track statistical regularities. They are not instructed to do so either in the lab or in the wild. And there are no explicit rewards in most statistical learning tasks; infants detect these regularities in the absence of any external motivation to do so.

Why, then, do infants track statistical regularities? One possible explanation, at least in the domain of language, is that infants track linguistic patterns because they want to figure out how to communicate with their caregivers. While motivation to communicate undoubtedly plays an important role in infant language development, it seems unlikely to fully explain infants’ sensitivity to statistical structure. Infants track many of the same types of patterns in both communicative and non-communicative domains (e.g., musical tones, geometric shapes, computer alert sounds; for review, see Saffran & Kirkham, 2018). Even newborn infants detect statistical regularities in speech, long before they are plausibly engaged in communicatively interactions (Fló et al., 2019). And non-human animals are also sensitive to some of the same statistical regularities as human infants (for review, see Santolin & Saffran, 2018).

Below, we explore one hypothesis concerning why infants track statistical regularities: to generate expectations/predictions about their environments. We end with a related hypothesis: learning itself is motivating, and infants are driven to attempt to reduce uncertainty – a process for which statistical information would be invaluable. These issues have been widely studied in both human adult and non-human animal research, but have received scant attention in developmental science, particularly in the domain of language.

Statistical learning and prediction

The prediction literature is largely distinct from the statistical learning literature. Infants and young children engage in predictive behavior across numerous domains. For example, young children viewing images on a screen will use the semantics of the other words in the sentence to anticipate which word is likely to come next: hearing the word “pirate” leads to the expectation that “boat” will occur downstream (e.g., Borovsky et al., 2012). Similarly, toddlers who speak a language with grammatical gender (like Spanish or French) can use the gender of the article preceding the target noun to anticipate the likely next word, shifting their gaze to the referent that is of the correct grammatical gender before hearing the referent (e.g., Lew-Williams & Fernald, 2012). These behaviors can be considered to be predictive because the child is generating and acting on an expectation about what noun is likely to follow the word or words they just heard. Note, however, that these behaviors are likely not governed solely by statistical regularities. Most examples of incremental language processing in childhood, like those described above, are not statistical per se. In the examples above, listeners can exploit semantic information (e.g., the connection between pirate and boat) and grammatical information (e.g., whether a word is masculine, feminine, or neuter).

Language is a particularly informative domain in which to consider relationships between statistical learning and predictive processes: natural languages contain rich statistical structure, and linguistic input unfolds in time. One way in which statistical regularities help infants to generate expectations is in the realm of sound sequences. When infants are familiarized with a particular type of sound-pattern regularity, they are subsequently better at segmenting word forms consistent with that regularity from fluent speech (e.g., Saffran & Thiessen, 2003; Sahni et al., 2010). Sound pattern regularities also facilitate mapping sounds to meanings. For example, infants familiarized with word forms that contain a specific phonotactic structure – regularities about which sounds precede and follow which other sounds – are subsequently more successful at mapping novel word forms to referents when the novel forms are consistent with the familiarized forms (Breen et al., 2019). These data suggest that infants are generating expectations about likely word forms based on the regularities to which they have been exposed.

Another source of evidence suggesting a relationship between statistical language learning and expectations comes from studies where infants are trained on novel label-object pairs following exposure to fluent speech streams (Graf Estes et al., 2007; Hay et al., 2011). Infants are more likely to successfully map labels that were words from the fluent speech (strong internally-cohesive statistical regularities) to novel referents than labels containing weaker statistical regularities. These data – in tandem with word learning studies using labels that vary in their native-language statistical cohesion (Graf Estes & Bowen, 2013; Graf Estes et al., 2011) – support the view that statistical patterns influence infants’ expectations about likely labels for objects.

Infant word learning is also influenced by the predictability of the events within which label-object pairings are situated. There appears to be a trade-off such that a label attached to an extremely unpredictable event – like a ball floating in the air – enhances word learning (Stahl & Feigenson, 2017), whereas a label attached to a moderately unpredictable event – like a violation of a visual sequence – hinders word learning (Benitez & Saffran, 2018). Thus, the statistical regularities of the environment – whether they are derived from real-world experience or within-lab training – impact infants’ expectations about what is likely to occur next, influencing the ease with which infants map labels onto objects participating in those events.

This body of literature suggests that infants’ linguistic expectations are influenced by the statistics of their environments. A question for future research concerns the degree to which infants’ predictive behaviors are informed by the statistics of their linguistic environments. Researchers working in the visual domain have demonstrated that infants’ visual behavior is influenced by the statistical structures to which they have been exposed. For example, infants use the previous locations of visual events to generate visual expectations about subsequent events (Romberg & Saffran, 2013; Tummeltshammer & Kirkham, 2013). Interestingly, their skill in doing so is related to native language vocabulary attainment, suggesting some potential relationships between non-linguistic statistical learning and language attainment (Reuter et al., 2018). Toddlers can also make visual predictions based on the sequential statistics of human actions (Monroy et al., 2017). This literature raises important questions about analogous situations in language learning. For example, do sequential statistics – word combinations – influence infants’ predictions about what word(s) are likely to occur next? The extant literature about children’s incremental language processing, discussed at the beginning of this section, suggests that infants can use semantic and grammatical information to make these predictions, but can they use statistical information either in lieu of or in addition to these other types of cues? And can infants learn from incorrect linguistic predictions, detecting their errors and updating their expectations, as observed with their visual predictions (Romberg & Saffran, 2013)? Addressing these types of questions will help to integrate the research fields of statistical learning and prediction/expectation.

Statistical learning and uncertainty

A related issue concerns how infants decide what to learn about. Researchers in the area of cognitive development have recently begun to focus on the potential role of uncertainty as a motivator for learning, providing new ways to think about abstract behaviors such as curiosity. Children are motivated by uncertainty as they make decisions about how to allocate their attention to different aspects of their environments. For example, children’s play behavior is influenced by opportunities to reduce uncertainty. preschoolers preferentially select toys where the potential causes of an event are confounded, using the opportunity to isolate variables to discover underlying causes (e.g., Schulz & Bonawitz, 2007). In this situation, preschoolers’ exploratory play suggests a preference for uncertainty, with subsequent uncertainty-reducing behaviors. In learning tasks, infants prefer stimuli that are neither too redundant nor too random, but that maximize opportunities for uncertainty reduction (e.g., Kidd et al., 2012). Parallel literatures in developmental robotics and neuroscience support the efficacy of curiosity-driven learning procedures (e.g., Kidd & Hayden, 2015; Oudeyer & Smith, 2016). The emerging developmental literature suggests a view of infants as active learners who sample from their environment in order to maximize learning outcomes (e.g., Sim & Xu, 2017; Smith et al., 2018). Oudeyer & Smith (2016) summarize as follows:

These theoretical advances lead to a definition of curiosity as an epistemic motivational mechanism that pushes an organism to explore activities for the primary sake of gaining information (as opposed to searching for information in service of achieving an external goal like finding food or shelter).

(p. 493)

There is a clear role for statistical learning in considerations of uncertainty as a motivator for attention and behavior. Uncertainty is related to experience with the statistics of the environment. And, given infants’ intense interest in linguistic stimuli, curiosity-based learning is potentially highly relevant to language acquisition. Indeed, several sources of evidence point to infants’ active role in caregiver interactions, whereby infants influence their own language input via the language-relevant behavior that they elicit from their caregivers. For example, infants’ pointing gestures signal their interest in learning: 19-month-olds are better able to learn labels for objects they had previously pointed to than for objects that had not been the object of their pointing gestures (Lucca & Wilbourn, 2018). Similarly, infants’ babbling behavior, when directed towards objects, increases caregivers’ contingent responsiveness (Albert et al., 2017). These types of findings suggest that infants take an active role in shaping their language learning environment. They do not, however, provide evidence regarding the factors that motivate infants’ choices of things to learn about, which likely range from novelty to salience to affective significance.

We hypothesize that statistical regularities influence infants’ decisions about which stimuli to sample from their environments. These regularities can be used to locate islands of uncertainty – permitting infants to subsequently take an active role in reducing that uncertainty. Despite the clear conceptual links between uncertainty, statistical learning, and language acquisition, these relationships have not been investigated. Part of the reason for this lacuna is methodological: how can infants’ interest in learning about different types of stimuli be directly assessed? We have recently developed infant-controlled eye-tracking methods that allow infants to select items for additional exposure during word-learning tasks. Our goal is to harness these methods to determine whether infants preferentially sample items about which they are more uncertain. More generally, we hope to explore the hypothesis that infants are motivated to track statistical regularities to identify and remediate uncertainty. These early processes may be among the general-purpose mechanisms at the roots of the development of curiosity (Kidd & Hayden, 2015).

Concluding remarks

More than two decades have passed since the first investigations of infant statistical language learning. Over the ensuing years, statistical learning perspectives have been integrated into many different approaches to the study of language development, as well as in myriad other aspects of child development, adult cognition and psycholinguistics, cognitive neuroscience, and cross-species comparative cognition. In this brief review, we have pointed to areas that have been well-studied (e.g., the use of statistical regularities to discover word boundaries in fluent speech) as well as other aspects of language development where statistical learning approaches are becoming entrenched.

In the latter part of the review, we connected statistical learning approaches to two other areas of intense current interest in the area of language and cognitive development: prediction and uncertainty. Now that statistical learning approaches have matured to the point that we can begin to ask “why” questions, it is crucial to engage in theory development that integrates insights from other related areas of study. In particular, it is evident from the contemporary literature that infants are not passive “sponges” soaking up regularities in their environments. Instead, infants are actively engaging with the world to gather information about things that are interesting to them, thereby shaping their own environments (e.g., Lucca & Wilbourn, 2018).

In addition to expanding the theoretical reach of statistical learning accounts, it is also important to expand the methodological basis of research in this area. While experiments with highly simplified artificial languages have been hugely informative in initiating research on statistical learning, it is crucial for theory-building that researchers expand their methodological armamentarium to use richer stimuli (for a recent critical review, see Frost et al., 2019). It is also increasingly important to develop studies that tap multiple levels of language concurrently. Just as infants are not tasked with learning just a single level of linguistic input at a time, our studies should connect learning at multiple levels. Language learning is a dynamic process; what a child can learn at any given moment depends on what she has already learned. With a richer base of experimental and computational results from which to draw, it will be possible to more clearly delineate both the promise and the limits of statistical learning approaches to language development.