The Emergence of Richly Organized Semantic Knowledge from Simple Statistics: A Synthetic Review

Abstract

As adults, we draw upon our ample knowledge about the world to support such vital cognitive feats as using language, reasoning, retrieving knowledge relevant to our current goals, planning for the future, adapting to unexpected events, and navigating through the environment. Our knowledge readily supports these feats because it is not merely a collection of stored facts, but rather functions as an organized, semantic network of concepts connected by meaningful relations. How do the relations that fundamentally organize semantic concepts emerge with development? Here, we cast a spotlight on a potentially powerful but often overlooked driver of semantic organization: Rich statistical regularities that are ubiquitous in both language and visual input. In this synthetic review, we show that a driving role for statistical regularities is convergently supported by evidence from diverse fields, including computational modeling, statistical learning, and semantic development. Finally, we identify a number of key avenues of future research into how statistical regularities may drive the development of semantic organization.

Introduction

We constantly draw upon our knowledge about the world to accomplish a variety of everyday feats of cognition, including using language, reasoning, planning for the future, and navigating through the environment. Our ability to use our prior knowledge for these feats depends on the fact that it is not merely a collection of stored facts, but instead functions as an interrelated, semantic network of concepts (e.g., Estes, Golonka, & Jones, 2011; Fisher, Godwin, Matlen, & Unger, 2015b; Gobbo & Chi, 1986; Heit, 2000; Kutas & Federmeier, 2000; López, Gelman, Gutheil, & Smith, 1992; Mack & Eckstein, 2011; McClelland & Rogers, 2003; Moores, Laiti, & Chelazzi, 2003; Nation & Snowling, 1999; Osherson, Smith, Wilkie, Lopez, & Shafir, 1990). For example, our knowledge of chickens does not exist in isolation, but can instead be related to our knowledge of eggs, corn, cows, and pigeons. How do our experiences with the world around us organize our semantic knowledge about the world?

To date, attempts to understand what aspects of experience drive the emergence of semantic organization in development have focused on learning from the sensorimotor aspects of concepts, such as learning that chickens and pigeons are related by observing that they share visual features (e.g., wings, feathers; Madole & Oakes, 1999; McClelland & Rogers, 2003; Quinn & Eimas, 2000; Rosch, Mervis, Gray, Johnson, & Boyes-Braem, 1976; Sloutsky, 2010; Smith & Heise, 1992). Sensorimotor experiences likely indeed represent one vital driver of semantic development. However, sensorimotor experience alone may be insufficient for developing richly organized semantic knowledge. First, knowledge is populated by many concepts for which we do not have direct access to sensorimotor features, such as abstract concepts (e.g., fun). Second, throughout development, knowledge is rich in semantic relations between concepts that do not share sensorimotor features, such as chicken and egg (e.g., Fenson, Vella, & Kennedy, 1989; Lin & Murphy, 2001; Nelson, McEvoy, & Schrieber, 1998; Unger, Savic, & Sloutsky, 2020a). Moreover, in spite of access to substantially different sensorimotor input, the organization of semantic knowledge is similar in sighted and congenitally blind individuals, even for visual concepts (e.g., visual actions such as peer and glance, Bedny, Koster-Hale, Elli, Yazzolino, & Saxe, 2019; Kim, Elli, & Bedny, 2019).

In this synthetic review, we therefore cast a spotlight on another, often overlooked, source of information ubiquitous in the environment that may complement the contributions of sensorimotor experience. Specifically, we examine the potentially fundamental contributions to semantic development of exposure to the rich statistical co-occurrence regularities that are ubiquitous in our everyday environments. According to this perspective, semantic relations may form as children observe the regularities with which concept labels or real-world referents co-occur.

This review consists of five parts. First, to ground our discussion of semantic development, we describe semantic relations that come to play key roles in organizing semantic knowledge: (1) Relations between concepts whose labels or referents are Associated with each other in the environment (e.g., chicken - egg), and (2) Relations between concepts belonging to the same stable, Taxonomic category (e.g., chicken - pigeon). Second, we highlight how sensitivity to co-occurrence regularities may powerfully contribute to the development of both types of semantic relations. In support of the proposal that co-occurrence regularities contribute to the development of semantic organization, we review evidence that much of the relations between concepts in human semantic organization are latent in co-occurrence regularities in language and visual input. Specifically, we highlight evidence that distributional semantics models, which form semantic representations from exposure to recorded real-world input rich in co-occurrence regularities, predict many semantic phenomena observed in humans (Asr, Willits, & Jones, 2016; Frermann & Lapata, 2015; Huebner & Willits, 2018; Jones & Mewhort, 2007; Jones, Willits, & Dennis, 2015; Lund & Burgess, 1996; Rohde, Gonnerman, & Plaut, 2004; Sadeghi, McClelland, & Hoffman, 2015). Thus, exposure to co-occurrence regularities may also contribute to the development of semantic organization in humans. Third, we underline the plausibility of co-occurrence as a driver of semantic development by reviewing evidence that developing humans are sensitive to co-occurrence regularities in multiple domains, including emerging evidence that this sensitivity contributes to semantic development. Fourth, we discuss how contributions of co-occurrence regularities to the formation of semantic relations may account for much of the developmental changes in semantic organization observed in prior research. Finally, we highlight key avenues for future research into the contributions of co-occurrence regularities to semantic development.

Semantic Relations

Over the past several decades, researchers have classified the relations that organize concepts and their labels in semantic knowledge into a variety of different groups denoted by a variety of terms. However, in spite of their differences, these classifications broadly divide semantic links into two main types that we describe in this section: Associative and Taxonomic¹.

Associative Relations

One fundamental type of relation that contributes to the organization of concepts in semantic knowledge is Association. In this review, we use this term to refer to relations between concepts whose real-world counterparts or labels are reliably experienced together in the physical or linguistic environment, such as chicken and egg.. (Note that this definition focuses on relations that capture associations in input, and is therefore distinct from the use of the term “association” to refer to patterns of responses on free association tasks, whose origins are ambiguous; Hutchison, 2003; McRae, Khalkhali, & Hare, 2012)

Much of prior research into the development of semantic organization has downplayed the importance of associative relations between concepts in the organization of semantic knowledge. Specifically, prior developmental accounts have either overlooked the development of associative relations (Gelman & Markman, 1986; Madole & Oakes, 1999; Quinn & Eimas, 2000; Sloutsky, 2010; Smith & Heise, 1992) or treated the development of associative relations as an immature phase of development that becomes overwritten by more mature relations between concepts with age (Inhelder & Piaget, 1964; Nelson, 1988; Nelson & Nelson, 1990). However, there is growing evidence that learned associative relations play a key role in the organization of semantic knowledge, and the vital cognitive processes that semantic knowledge supports. First, numerous findings provide evidence that associative relations are a key facet of semantic organization because they are activated automatically (see, e.g., Estes et al., 2011; Hutchison, 2003, for reviews). Moreover, associative relations between known concepts can support inferences that generalize knowledge, such as inferring that if a chicken is infected with a disease, its egg may be also (Coley, 2012; Lin & Murphy, 2001). Associative relations can also support learning new words from linguistic context, such as learning that dax refers to an animal when it is heard in the context of words associated with animal such as furry, zoo, etc. (Sloutsky, Yim, Yao, & Dennis, 2017). In addition, associative relations contribute to visual search, such as more readily locating and recognizing an egg when it is near a chicken (Mack & Eckstein, 2011; Moores et al., 2003). Finally, associative relations play a key role in language comprehension. For example, numerous findings (Altmann & Kamide, 1999; DeLong, Urbach, & Kutas, 2005; Kutas & Federmeier, 2000) suggest that during language comprehension, people anticipate upcoming words associated with the words they have heard or read thus far, such as anticipating “egg” upon hearing “I went to the coop this morning, and saw that the chicken had laid an…”..

Taxonomic Relations

Another key type of semantic relation that contributes to organizing semantic knowledge consists of relations between concepts (e.g., chicken, pigeon) that belong to the same category whose membership is stable over time (e.g., birds). Correspondingly, words for these concepts are therefore similar in meaning. In this review, we use the term Taxonomic to refer both to concepts belonging to the same stable category, and words for these concepts that are similar in meaning.

Taxonomic relations have long been identified as playing a key role in semantic development (Deák & Bauer, 1996; Gelman & Coley, 1990; Gelman & Markman, 1986; Markman & Hutchinson, 1984; Nelson, 1988; Sloutsky, 2010) because they are of particular utility for generalizing knowledge: I.e., extending information that we have gained from prior experience to new situations (Fisher et al., 2015b; Heit, 2000; Keil, Smith, Simons, & Levin, 1998; Kemp & Tenenbaum, 2009; López et al., 1992; Osherson et al., 1990; Rips, 2001; Sloman, 1993). For example, Taxonomic relations among familiar concepts such as chicken, pigeon, eagle, etc. can help us recognize a new entity we have never seen before, such as a sage grouse, as a bird. This same knowledge can also help us generalize knowledge about one bird, such as that pigeons have hollow bones, to both familiar and new birds. By the same token, this support for generalization can help us make inferences about how birds will behave, such as that a bird walking on the ground may take flight when we approach.

This contribution of taxonomic relations also plays a key role in language fluency (e.g., Borovsky, Ellis, Evans, & Elman, 2016; Borovsky & Elman, 2006; Hadley, Dickinson, Hirsh‐Pasek, & Golinkoff, 2019; Kaefer & Neuman, 2013; Neuman & Dwyer, 2011). One vital contribution of taxonomic relations is that we are able to produce and understand a potentially unlimited variety of meaningful utterances in part because words similar in meaning can be substituted with each other in utterances. For example, even if we have only ever heard utterances about “chickens” laying eggs, taxonomic relations between “chicken” and words similar in meaning such as “pigeon” allow us to understand and produce utterances about other birds laying eggs (Deák & Bauer, 1996; Fisher, 2010; Fisher, Godwin, & Matlen, 2015a; Fisher et al., 2015b; Fisher, Matlen, & Godwin, 2011; Gelman & Coley, 1990; Gelman & Markman, 1986; López et al., 1992).

In contrast with association, these contributions of taxonomic links to semantic organization and the cognitive processes that semantic organization supports have been the focus of extensive empirical (e.g., Bauer & Mandler, 1989; Deák & Bauer, 1996; Fenson, Cameron, & Kennedy, 1988; Fisher et al., 2015a; Fisher et al., 2015b; Gelman & Coley, 1990; Gelman & Markman, 1986; Tversky, 1985; Waxman & Namy, 1997) and theoretical work (e.g., Fisher, 2015; Inhelder & Piaget, 1964; Madole & Oakes, 1999; McClelland & Rogers, 2003; Nelson, 1988; Quinn & Eimas, 2000; Sloutsky, 2010; Smith & Heise, 1992).

Summary

Both Associative and Taxonomic relations play important roles in supporting a variety of everyday feats of cognition by organizing our knowledge about the world. Therefore, a key facet of understanding cognition is understanding how these relations are formed during development. The apparent differences between these semantic relations have been taken to suggest that they comprise distinct semantic systems (Mirman, Landrigan, & Britt, 2017) and must be learned in fundamentally different ways. Indeed, accounts of the emergence of semantic organization have largely treated these relations as entirely separate, and focused on explaining how taxonomic relations in particular are formed (Gelman & Coley, 1990; Madole & Oakes, 1999; McClelland & Rogers, 2003; Nelson, 1988; Sloutsky, 2010; Smith & Heise, 1992; Waxman & Gelman, 1986). In contrast, in this review, we highlight the possibility that the same source of information ubiquitous in the environment may vitally contribute to both types of relations: Regularities with which words or entities co-occur more reliably with each other than with others.

Potential Co-Occurrence Contributions to Learning Semantic Relations

In this section, we outline how co-occurrence regularities may play a powerful role in forming both Associative and Taxonomic relations. As we will review in depth below, this proposal is motivated by and grounded in evidence that language is rich in co-occurrence regularities from which semantic relations can in principle be learned (Asr et al., 2016; Frermann & Lapata, 2015; Griffiths, Steyvers, & Tenenbaum, 2007; Harris, 1954; Huebner & Willits, 2018; Jones & Mewhort, 2007; Jones et al., 2015; Landauer & Dumais, 1997; Lund & Burgess, 1996; Mikolov, Chen, Corrado, & Dean, 2013; Miller & Charles, 1991; Rohde et al., 2004; Rubenstein & Goodenough, 1965; Sadeghi et al., 2015).

First, sensitivity to co-occurrence may directly foster Associative relations. For example, sensitivity to the regular co-occurrence between eat and apple or chicken and egg (i.e., co-occurrence of the words that denote these concepts and/or their real-world counterparts) may directly foster the formation of an Associative relation between them. In contrast, entities or words from the same taxonomic category may not reliably co-occur. However, this does not mean that learning Taxonomic relations necessarily requires separate learning processes. Instead, relations between entities or labels from the same taxonomic category may be derived from the regularity with which they share each other’s patterns of co-occurrence with other entities or labels (this relationship is emphasized in models of word meaning derived from co-occurrence regularities discussed below, as well as in, e.g., Borovsky & Elman, 2006; Li, Farkas, & MacWhinney, 2004). For example, members of the same taxonomic category of foods, such as apple and spaghetti, both consistently co-occur with eat. For clarity, we refer to regularities with which entities or labels co-occur with each other as “direct” co-occurrence regularities, and regularities with which entities or labels share each other’s patterns of co-occurrence as “shared” co-occurrence regularities. A schematic depiction of the semantic relations that may be learned through sensitivity to these regularities is presented in Figure 1.

Figure 1.

Left: Schematic depiction of direct and shared co-occurrence regularities. Right: Examples of associative and taxonomic links that can form from direct and shared co-occurrence regularities in language (top) or visual (bottom) input.

These potentially powerful contributions of sensitivity to co-occurrence regularities are not merely speculative. Instead, this possibility is grounded in extensive evidence that much of the relations between concepts in human semantic knowledge can in principle be learned from co-occurrence regularities that are ubiquitous in the environment.

Co-Occurrence Regularities Capture Semantic Relations

In this section, we review evidence that the environment contains co-occurrence regularities from which semantic relations can be learned. First, a number of analyses have found that the regularity with which words either directly co-occur or share each other’s patterns of co-occurrence in language corpora predict human semantic phenomena, such as responses on free association tasks (Hofmann, Biemann, Westbury et al., 2018; Spence & Owens, 1990), semantic priming (Roelke, Franke, Biemann et al., 2018; Willits, Amato, & MacDonald, 2015, Experiment 2A), reading behavior (McDonald & Shillcock, 2003), language processing (Arnon & Snider, 2010), and neural responses evoked during language processing (Frank & Willems, 2017). These findings support the possibility that co-occurrence regularities are a key source of input available from the environment that can contribute to the development of semantic relations.

The availability of co-occurrence regularities that can contribute the development of semantic organization is further highlighted by substantial computational modeling evidence. Specifically, beginning with the seminal work of Landauer and Dumais (1997) and Lund and Burgess (1996), numerous distributional semantics models have been developed in which simple mechanisms sensitive to co-occurrence regularities successfully capture much of the relations between concepts in human semantic knowledge. The success with which these models capture semantic relations underlines the availability in the environment of co-occurrence regularities that can contribute to the formation of semantic relations in developing humans. Distributional semantics models derive representations of words or objects from the regularities with which they co-occur with other words or objects in corpora of language or visual scenes (Asr et al., 2016; Frermann & Lapata, 2015; Huebner & Willits, 2018; Jones & Mewhort, 2007; Jones et al., 2015; Lund & Burgess, 1996; Rohde et al., 2004; Sadeghi et al., 2015). Based on these regularities, representations of items can become similar both when they directly co-occur, such as juicy and apple, and when they share each other’s patterns of co-occurrence, such as apple and pear. The similarities between representations of items formed in this manner correspond closely to the semantic relations between concepts in human semantic knowledge. For example, when used to simulate semantic priming phenomena (e.g., faster decisions that apple is a word when preceded by juicy versus furry), these models produce semantic priming behavior similar to that observed in humans (e.g., Hare, Jones, Thomson, Kelly, & McRae, 2009; Jones & Mewhort, 2007; Lund & Burgess, 1996). Importantly, recent evidence (Asr et al., 2016; Frermann & Lapata, 2015; Huebner & Willits, 2018) suggests that this correspondence between the similarity of representations formed from distributional regularities and semantic relations also emerges when models are given recorded child-directed speech as input (MacWhinney, 2000; for ongoing research that may illuminate the presence of semantically-relevant co-occurrence regularities in visual input to children, see Smith, Yu, Yoshida, & Fausey, 2015).

These bodies of research provide evidence that, in principle, co-occurrence regularities in the environment can serve as a vital source of input for learning semantic relations between concepts in development. Moreover, although the majority of this research has focused on co-occurrence regularities in language (likely due in part to the availability of language corpora), emerging evidence suggests that the environment also contains rich co-occurrence regularities between objects in the world (Sadeghi et al., 2015). However, it is important to note that many of the distributional semantics models reviewed here were not explicitly designed as models of human semantic development. Therefore, it is beyond the scope of this review to evaluate whether the mechanisms embodied in these models capture learning processes that give rise to semantic organization in developing humans (see, e.g., Jones et al., 2015; Lenci, 2018, for reviews of model mechanisms). Instead, the success with which these models capture human semantic phenomena underlines co-occurrence regularities as a ubiquitous and informative source of input that may contribute to the development of semantic organization (for a similar perspective on the importance of regularities in input, see Louwerse, 2011).

Importantly for insight into the development of human semantic organization, some recent research (e.g., Sloutsky et al., 2017) has begun to derive predictions about how mechanisms that form semantic relations from co-occurrence regularities may drive the development of semantic organization. Specifically, this research has investigated how the representations built by distributional semantics models change as they receive increasing amounts of exposure to recorded language input to children (MacWhinney, 2000). One key prediction produced by this research is that Associative relations that can be formed from direct co-occurrence should emerge early in development because they are fostered directly from input itself. For example, experiencing an instance of direct co-occurrence between “juicy” and “apple” can contribute to an Associative relation to the corresponding concepts. In contrast, shared co-occurrence-based Taxonomic relations should emerge more gradually, because learners must first form the overlapping direct co-occurrence-based Associative relations from which shared co-occurrence can be derived. In sum, these findings predict that Associative relations should emerge early, and become gradually supplemented by Taxonomic relations that are derived (at least in part) from them.

In the following sections, we evaluate the possibility that sensitivity to co-occurrence regularities represents a key driver of human semantic organization. Specifically, we assess whether: (1) Humans are sensitive to direct and shared co-occurrence regularities during development, and (2) Developmental changes in semantic organization are consistent with predictions from models that build semantic representations from co-occurrence regularities.

Evidence for Sensitivity to Co-Occurrence Regularities in Development

For co-occurrence regularities to be a viable driver of semantic development, developing humans must be sensitive to co-occurrence regularities. As described in the following sections, there is extensive evidence for the emergence in infancy of sensitivity to direct co-occurrence regularities in multiple domains (Aslin, Saffran, & Newport, 1998; Bulf, Johnson, & Valenza, 2011; Fiser & Aslin, 2002; Gómez & Maye, 2005; Gomez, 2002; Kirkham, Slemmer, & Johnson, 2002; Kirkham, Slemmer, Richardson, & Johnson, 2007; Pelucchi, Hay, & Saffran, 2009; Saffran, 2001; Saffran, Aslin, & Newport, 1996; Saffran, Johnson, Aslin, & Newport, 1999), including some recent evidence within the semantic domain (Matlen, Fisher, & Godwin, 2015; Wojcik & Saffran, 2015). Moreover, decades of research provide robust evidence that adults are sensitive to direct co-occurrence in many domains (for reviews of learning and mechanisms, see, e.g., Le Pelley, 2004; Le Pelley, Mitchell, Beesley, George, & Wills, 2016; Wasserman & Miller, 1997). In contrast, the development of sensitivity to shared co-occurrence-has been the subject of only limited research. However, emerging evidence suggests that sensitivity to shared co-occurrence may be present in some form starting in infancy (Yermolayeva & Rakison, 2016), and may become increasingly robust over the course of childhood and into adolescence (Bauer & San Souci, 2010; Schlichting, Guarino, Schapiro, Turk-Browne, & Preston, 2017).

Sensitivity to Direct Co-Occurrence Regularities

Evidence for early-emerging sensitivity to direct co-occurrence comes in part from extensive statistical learning research. Specifically, numerous statistical learning studies have revealed the emergence in infancy of sensitivity to direct co-occurrence between inputs including speech sounds (as early as within the first 6 months of life and improving over the course of infancy to, e.g., detect direct co-occurrence between non-adjacent sounds, Aslin et al., 1998; Estes, Evans, Alibali, & Saffran, 2007; Gómez & Maye, 2005; Gomez, 2002; Johnson & Tyler, 2010; Pelucchi et al., 2009; Saffran, 2001; Saffran et al., 1996; Thiessen & Saffran, 2003), acoustic sounds (from at least 8 months, Saffran et al., 1999), and visual objects (possibly from birth and improving over the course of infancy, Bulf et al., 2011; Fiser & Aslin, 2002; Kirkham et al., 2002; Kirkham et al., 2007) (for reviews, see, e.g., Lany & Saffran, 2013; Saffran & Kirkham, 2018).

Recent studies suggest that sensitivity to direct co-occurrence between words also emerges early in development, and may contribute to semantic relations between the concepts that words denote. First, a study conducted by Wojcik and Saffran (2015) provided evidence that 26–28-month-old toddlers perceived pairs of novel words (e.g., “coro” and “blicket”) as related after only 12 presentations of sentences in which the novel words directly co-occurred (e.g., “The coro broke the blicket”). Similarly, Bannard and Matthews (2008) provided evidence that 2–3-year-old children’s word knowledge stores sets of words that regularly directly co-occur in real-world language input, such as “sit in your chair”. A handful of findings suggest that this sensitivity to direct co-occurrence between words may contribute to semantic relations between corresponding concepts in children. First, a series of studies conducted by Fisher and colleagues (Fisher et al., 2011; Matlen et al., 2015) provides evidence that direct co-occurrence between familiar words contributes to semantic relations between the corresponding familiar concepts in young (four-year-old) children. In these studies, four-year-old children were presented with a semantic generalization task in which they were told that a “target” entity denoted by a word, such as a whale, possessed a novel property, such as plaxium blood. Children were then prompted to choose to generalize this property to either a close taxonomic match, such as a dolphin, or a more distant taxonomic match, such as a seal. Children only reliably chose the close taxonomic match when its label directly co-occurred with the label for the target either in corpora of child-directed speech (Fisher et al., 2011), or when the labels directly co-occurred in a speech stream played to children prior to the task (Matlen et al., 2015). Convergent with this evidence that direct co-occurrence contributes to semantic relations between concepts in children, emerging evidence (Savic, Unger, & Sloutsky, 2020, in prep) is revealing that both four-year-old children and adults treat novel and familiar words that are empirically manipulated to directly co-occur in sentences as semantically related. In these studies, participants heard sentences in which novel and familiar words such as “foobly” and “apple” directly co-occurred in sentences such as “I saw a foobly apple”. Subsequently, both children and adults consistently used novel words to label pictures of both the familiar word with which they directly co-occurred (e.g., pictures of apples), and items that are visually and semantically similar (e.g. pictures of peaches).

Taken together, these lines of research provide evidence that sensitivity to direct co-occurrence between sounds and between objects is present beginning in infancy. Sensitivity to direct co-occurrence between words may emerge by at least 26–28 months of age. Moreover, recent findings suggest that this sensitivity contributes to semantic relations between concepts denoted by words by at least four years of age.

Sensitivity to Shared Co-Occurrence Regularities

In contrast with evidence for sensitivity to direct co-occurrence, sensitivity to shared co-occurrence regularities has received comparatively little study. However, findings from a handful of studies suggest that sensitivity to shared co-occurrence regularities may begin to emerge in in infancy, and come to play a key role in semantic development over the course of childhood and into adulthood.

Evidence that sensitivity to shared co-occurrence regularities begins to emerge early in development comes from a handful of studies with infants. In one study conducted by Cuevas, Rovee-Collier, and Learmonth (2006), 5.5–6 month old infants participated in sessions in which one hand puppet (Puppet A) separately directly co-occurred with: (1) A second hand puppet (Puppet B), and (2) A context in which a positive reinforcer was experienced. Thus, Puppet B and the positive reinforcer shared co-occurrence with Puppet A. Subsequently, infants’ behavior revealed that they perceived Puppet B and the positive reinforcer as related. Similarly, in a series of studies conducted by Yermolayeva and Rakison (2016), infants were first familiarized with objects in which two “Bodies” were each affixed with one of a pair of body “Parts” (e.g., Body 1 was always affixed with Part A or B, and Body 2 was always affixed with Part C or D). Thus, a given pair of body Parts shared each other’s co-occurrence with a given Body. Following familiarization, infants aged 7- and 11-months distinguished between objects consisting of a novel Body affixed with a pair of Parts that shared co-occurrence, versus the same novel Body affixed with Parts that did not share co-occurrence. These findings suggest that sensitivity to shared co-occurrence regularities emerge in infancy in some domains.

Sensitivity in infancy to shared co-occurrence between novel words has been investigated in studies of abilities that may support learning “lexical categories”, such as nouns and adjectives (Gerken, Wilson, & Lewis, 2005; Lany & Saffran, 2010, 2011). These studies provide evidence for sensitivity in infants and toddlers to shared co-occurrence between words, but only when shared co-occurrence is correlated with other similarities between words. Specifically, infants (ranging in age from 15 to 22 months) heard pairs of directly co-occurring novel words that were generated by combining words from four sets: Set-a, -b, -X, and -Y. Pairs were constructed such that set-a words directly co-occurred with set-X words, and set-b words with set-Y words. Thus, set-a words shared each other’s co-occurrence with set-X words, and set-b words shared each other’s co-occurrence with set-Y words. Across studies, participants showed evidence of sensitivity to these shared co-occurrence regularities only when additional cues differentiated between set-X and -Y words, such as that all set-X words were monosyllabic, whereas all set Y words were disyllabic. Moreover, some evidence suggests that this sensitivity to shared co-occurrence contributed to semantic relations between words. For example, following exposure to the word pairs, 22-month-old infants in Lany and Saffran (2011) heard subsets of the a-X and b-Y pairs, each with referents from one of two visually distinct semantic categories – i.e., animals and vehicles. If shared co-occurrence within set-a and set-b words contributed to semantic relations between them, infants could potentially generalize these word-referent mappings, and thus associate the remaining set-a words with members of one category (e.g., with animals), and set-b words with members of the other category (e.g., with vehicles). This contribution of shared co-occurrence to semantic relations transpired in infants who scored highly on tests of language proficiency (e.g., vocabulary size). Thus, infants and toddlers may possess some sensitivity to shared co-occurrence between words, but only when this regularity is correlated with additional cues. Moreover, this sensitivity may contribute to semantic relations between words in some toddlers.

Beyond infancy and toddlerhood, research in a handful of domains suggests that sensitivity to shared co-occurrence becomes increasingly robust over the course of development. For example, in studies conducted by Bauer and colleagues, participants were presented with pairs of stem facts that both linked new characteristics to the same familiar concept, such as “Dolphins live in pods” and “Dolphins communicate by clicking and squeaking”. Thus, the two characteristics such as “live in pods” and “communicate by clicking and squeaking” shared co-occurrence with the familiar concept. These studies have assessed the development of sensitivity to shared co-occurrence based on children’s ability to derive new facts by combining the characteristics that shared co-occurrence, such as “Pods communicate by clicking and squeaking”. These studies have found that sensitivity to shared co-occurrence is weak in young (four-year-old) children, and improves substantially with age (Bauer & Larkina, 2017; Bauer & San Souci, 2010). Similarly, research conducted by Schlichting and colleagues (Schlichting et al., 2017) has investigated the degree to which children (6 to 11 years), adolescents (12 to 17 years) and adults (18 to 30 years) who were instructed to memorize pairs of images subsequently link images that were never paired, but instead shared each other’s pairing with the same image. This research has revealed substantial developmental improvements in abilities to link images based on shared pairing over the course of childhood and adolescence.

Very little research has investigated the development of sensitivity to shared co-occurrence between words that may contribute to semantic relations between the concepts that words denote. However, emerging evidence suggests that, with age, exposure to regularities in which words share co-occurrence increasingly contributes to semantic relations between words. Specifically, in the studies introduced above by Savic et al. (2020); and Savic et al. (in prep), 4-year-old children and adults heard sentences in which familiar words both directly co-occurred and shared co-occurrence with novel words, such as “I saw a foobly apple” and “My friend Sally saw a foobly mipp”. As noted in the previous section, direct co-occurrence (e.g., between “foobly” and “apple”) contributed to semantic relations between words in both 4-year-old children and adults. In contrast, shared co-occurrence only contributed to semantic relations between words in adults. Specifically, only adults reliably used the novel words to label pictures of the familiar concept with which they shared co-occurrence (e.g., labeled apples and other fruits “mipp”). Taken with the evidence reviewed throughout this section, these findings suggest that sensitivity to direct co-occurrence that contributes to semantic relations is increasingly supplemented by sensitivity to shared co-occurrence with age.

Finally, by adulthood, people are sensitive to shared co-occurrence in multiple domains, including shared co-occurrence between words. For example, studies conducted by Schapiro and colleagues (Schapiro, Rogers, Cordova, Turk-Browne, & Botvinick, 2013) have revealed that, following incidental exposure to sequences of abstract images, images that shared each other’s direct co-occurrence with other images came to prompt similar patterns of neural activity (for similar evidence of sensitivity to shared co-occurrence between abstract images, see also Hall, Mitchell, Graham, & Lavis, 2003). Similar evidence for adult sensitivity to the regularity with which novel words share co-occurrence with other words in sentences comes from studies conducted by McNeill (1963, 1966). Moreover, evidence that this sensitivity contributes to semantic relations between words comes from a series of studies conducted by Jones and Love (2007). These studies revealed that adults rated pairs of familiar words such as “polar bear” and “seal” as more similar after reading separate sentences in which these words shared co-occurrence with averb such as “chase”, than when the words directly co-occurred with different verbs.

Together, these separate lines of research suggest that sensitivity to shared co-occurrence may be present to some degree starting in infancy. With age, this sensitivity becomes increasingly robust. Critically, a handful of studies suggest that the developing sensitivity to shared co-occurrence contributes to semantic relations between the concepts that words denote.

Summary

For sensitivity to co-occurrence regularities to contribute to the development of semantic organization, it is not sufficient for co-occurrence regularities to be present in the environment: developing learners must also be sensitive to these regularities. Moreover, this sensitivity to co-occurrence regularities must also contribute to semantic relations between concepts. The studies reviewed in this section provide substantial evidence for early-emerging sensitivity to direct co-occurrence that contributes to semantic relations between concepts. In contrast, the development of sensitivity to shared co-occurrence has received less empirical attention. However, emerging evidence suggests that sensitivity to shared co-occurrence that contributes to semantic relations may gradually supplement sensitivity to direct co-occurrence with development.

Accounting for the Trajectory of Semantic Development

Over the course of development, semantic organization both expands to incorporate new concepts and new relations between concepts (Bjorklund & Jacobs, 1985; Blaye, Bernard-Peyron, Paour, & Bonthoux, 2006; Coley, 2012; Crowe & Prescott, 2003; Howard & Howard, 1977; Nguyen, 2007; Storm, 1980; Tversky, 1985; Unger, Fisher, Nugent, Ventura, & MacLellan, 2016; Vales, Stevens, & Fisher, 2020; Walsh, Richardson, & Faulkner, 1993). The trajectory of semantic organization development has been extensively studied in prior developmental research. In this section, we discuss whether developmental changes in semantic organization are consistent with the possibility that they are driven by sensitivity to co-occurrence regularities.

To evaluate whether developmental changes in semantic organization may be driven by sensitivity to co-occurrence regularities, it is important to predict the developmental changes that would be likely to occur if semantic organization were shaped by sensitivity to co-occurrence regularities. Converging predictions about the trajectory of semantic organization development come from both simulations of development in distributional semantics models, and empirical evidence about the development of abilities to learn semantic relations from direct and shared co-occurrence. First, as described above, some recent efforts have been made to predict the development of semantic organization from distributional semantics models. These simulations suggest that Associative relations derived from direct co-occurrence should emerge early in development, and become gradually supplemented by Taxonomic relations derived from shared co-occurrence (Sloutsky et al., 2017). Similarly, as described above, empirical research with children and adults suggests that sensitivity to direct co-occurrence that contributes to semantic relations emerges early in development, and is supplemented by sensitivity to shared co-occurrence with age².

Overall, these lines of research predict that the development of semantic organization should broadly consist of the early emergence of Associative relations that are gradually supplemented (but not replaced) by Taxonomic relations. In what follows, we therefore evaluate the degree to which observations of the development of semantic organization are consistent with this prediction.

Early Emergence of Associative relations

The predicted early emergence of Associative relations is a robust empirical finding observed across studies spanning decades that have used a variety of methods for investigating the development of semantic organization. For example, numerous studies have assessed relations between concepts from toddler-age onward using match-to-sample (also referred to as “forced choice”) tasks in which participants are presented with a Target item (e.g., chicken), and must choose a Match item that is linked to the Target by a relation (e.g., egg, to assess Associative relations) from a set of options. These studies have revealed that even toddlers (from at least 26 months, Fenson et al., 1989; Nguyen, 2007) can reliably choose Match items that are Associatively related to Targets. Moreover, this behavior is not a transient phase of development, but instead persists into adulthood (Estes et al., 2011; Lin & Murphy, 2001; Ross & Murphy, 1999).

Converging evidence for the early emergence of Associative relations comes from research using a variety of paradigms. For example, from an early age, children: (1) Remember word pairs or lists composed of Associatively related words better than those composed of unrelated words (from at least age four, Bjorklund & Jacobs, 1985; Blewitt & Toppino, 1991), (2) Produce Associatively related words in response to word prompts in free association tasks (from at least age three, Nelson, 1977; Wojcik & Kandhadai, 2019), and (3) Infer that a new word, e.g., “dax” shares a meaning with a target familiar word, e.g., “animal” upon hearing the new word with other familiar words Associated with the target familiar word, e.g., “furry” and “zoo” (from at least age four, Sloutsky et al., 2017).

The proposal that semantic relations learned from direct co-occurrence regularities emerge early and remain robust in the course of development has been more explicitly tested and corroborated in a recent series of studies conducted by Unger et al. (2020a). Specifically, across three different paradigms, these studies revealed that concepts whose labels reliably directly co-occur in language are robustly related in the semantic knowledge of both young children (four years of age) and adults. In sum, extensive evidence supports the prediction that early-emerging sensitivity to direct co-occurrence regularities fosters the early emergence of Associative relations in semantic knowledge.

Gradual Emergence of Taxonomic Relations

As described above, the proposal that co-occurrence regularities contribute to semantic development predicts the gradual development of Taxonomic relations that may be formed from shared co-occurrence. However, in evaluating this prediction, it is critical to note that this perspective predicts a pattern of gradual emergence of Taxonomic relations. However, this perspective does not reject the possibility that some Taxonomic knowledge may emerge relatively early. For example, other sources of input, particularly visual similarity, may foster some relations between members of the same taxonomic category (Blaye et al., 2006; Eimas & Quinn, 1994; Fenson et al., 1988; Oakes, Coppage, & Dingel, 1997; Quinn & Johnson, 2000; Sloutsky, 2010; see the New Avenues for Future Research section below for a discussion of synergistic contributions of co-occurrence and perceptual input).

Several investigations into the emergence of Taxonomic knowledge reported evidence that Taxonomic relations began to emerge only in older children (e.g., age six and above), often following the earlier emergence of Associative or visual-similarity-based relations. For example, numerous studies using match-to-sample paradigms reported that older children but not younger children could reliably choose Taxonomic matches to a target item over Associative and/or visual similarity matches, (Lucariello, Kyratzis, & Nelson, 1992; Tversky, 1985; Walsh et al., 1993). Other studies that assessed taxonomic knowledge using sorting (Blaye et al., 2006), list recall (Bjorklund & Jacobs, 1985; Monnier & Bonthoux, 2011), word association (see Nelson, 1977 for a review), label extension (Baldwin, 1992; Imai, Gentner, & Uchida, 1994), and word learning (Sloutsky et al., 2017) tasks have revealed a similarly late onset for Taxonomic relations compared to Associative relations or visual-similarity-based relations. The age at which children began to exhibit behavior consistent with knowledge of Taxonomic relations varied across tasks and studies, but became evident in most studies at approximately 6 to 7 years.

Further evidence that Taxonomic relations develop more gradually than other relations comes from studies that have investigated the effects of task context on children’s choices of related items. These studies have provided evidence that young children’s ability to choose Taxonomic matches in match-to-sample tasks is less robust and more dependent on task context than choices based on other relations. First, a number of studies have reported that young children can make Taxonomic choices, but only when Taxonomic relations are highlighted by shared or similar-sounding labels (e.g., Gentner & Namy, 1999; Long, Lu, Zhang, Li, & Deák, 2012; Namy & Gentner, 2002; Sloutsky & Fisher, 2004, 2012). For example, four-year-old children in a study conducted by Gentner and Namy (1999) were presented with realistic depictions of a target, such as a red apple, a Taxonomic match, such as a yellow banana, and a visually similar object, such as a red balloon. Children only consistently chose the Taxonomic match when they were presented with two Targets from the same Taxonomic category that were given the same label, and asked to decide which match also shared the label (see also Namy & Gentner, 2002). Similarly, numerous studies have observed that young children only consistently identified a Taxonomic match as the same kind of thing as a target when both were labeled with the same or a similar-sounding word (Deák & Bauer, 1996; Long et al., 2012; Sloutsky & Fisher, 2004; 2012, for further discussion of the implications of these findings, see below). Further evidence for the task-dependence of Taxonomic relations comes from findings that children’s use of Taxonomic relations relied on being trained to do so (Smiley & Brown, 1979), or on seeing realistic depictions of items such as photographs and objects that may provide richer perceptual access to similarities between Taxonomically related items (Deák & Bauer, 1996; Long et al., 2012). These findings suggest that Taxonomic relations are comparatively weak early in development, such that they reliably influence children’s behavior only when they are highlighted by supportive task contexts. Similar evidence for the comparative weakness of Taxonomic relations early in development comes from a study conducted by Fisher (2011). In this study, 3 to 5-year-old children completed two phases of a matching task, such that they were instructed to choose taxonomic matches in one phase, and visual similarity matches in another. Children showed greater costs of switching from visual similarity to taxonomic matches than for switching in the opposite direction (Fisher, 2011). By five years of age, the costs of switching to visual similarity were negligible, while children still showed sizeable costs in switching to taxonomic matches. These findings support the notion that Taxonomic relations are comparatively weak early in development, such that children struggle to use Taxonomic relations to guide behavior when explicitly prompted to do so.

More recent studies have provided evidence that adds nuance to this developmental trajectory. Specifically, a more nuanced picture of the gradual emergence of taxonomic links is presented by a series of studies conducted by Unger et al. (2020a). This research acquired sensitive, implicit measures of semantic relations between pairs of concepts, such as measures of the time course of looking at a picture of one concept upon hearing a label for another. These measures convergently revealed that Taxonomic relations that were weak and present only in some four-year-old children became robust by adulthood. For example, adults looked substantially more at a picture of a bed upon hearing a Taxonomically related word such as “table” or an Associatively related word such as “pillow” than an unrelated word such as “train”. Children’s looking was also influenced by both Taxonomically and Associatively related words, but the effect of Taxonomically related words was substantially weaker than the effect of Associatively related words. Together with earlier research, these findings support the prediction that Taxonomic relations gradually come to supplement Associative relations over the course of development.

It is important to note that in contrast to this body of evidence, a number of studies have yielded conflicting evidence that Taxonomic relations contribute to semantic organization from an early age. For example, two-year-old children consistently preferred Taxonomic over Associative matches in a match-to-sample study conducted by Bauer and Mandler (1989). Similar early preferences for Taxonomic matches (in some cases only under specific experimental conditions) have been reported in other studies, such as those conducted by Deák and Bauer (1996), Gelman and Markman (1986), and Waxman and Namy (1997).

However, in studies that suggested that robust Taxonomic relations emerge early in development, many Taxonomically related items also shared additional commonalities. For example, some of these studies included Taxonomically related items that likely co-occur (e.g., chair and table, dog and cat; Bauer & Mandler, 1989; Markman & Hutchinson, 1984; Waxman & Namy, 1997) and/or are visually similar (e.g.. glass and cup, car and jeep, corn and banana; Bauer & Mandler, 1989; Markman & Hutchinson, 1984; Waxman & Gelman, 1986; Waxman & Namy, 1997). Similarly, as noted above, information that could support Taxonomic choices was also present in multiple studies in the form of identical verbal labels for targets and Taxonomic matches (Deák & Bauer, 1996; Gelman & Markman, 1986). These identical labels may have served to help children access otherwise weak Taxonomic knowledge. Alternatively, a number of findings suggest that labels act as salient perceptual features in young (e.g., 4 to 5 year-old) children, such that children could group together Taxonomically related items on the basis of perceptual similarity in spite of weak or absent knowledge of Taxonomic relations (e.g., Sloutsky & Fisher, 2004, 2012; Sloutsky, Lo, & Fisher, 2001). Even when targets and Taxonomic matches were referred to by non-identical but semantically similar labels, these labels were often ones that reliably directly co-occur in language (e.g., puppy and dog; Gelman & Markman, 1986). The use of directly co-occurring labels raises the possibility that Taxonomic match choices were driven or supported by direct co-occurrence. Indeed, subsequent studies (Fisher, 2010; Fisher et al., 2011) found that when Target and Taxonomic match items were referred to by semantically similar labels, four-year-old children only reliably chose Taxonomic matches when semantically similar labels reliably directly co-occurred in language input to children. In contrast, young children responded at chance when labels for Taxonomic matches and targets did not reliably co-occur (e.g., lamb and sheep) (see also Unger, Vales, & Fisher, 2020b, for further evidence that co-occurrence contributes relations between members of the same Taxonomic category in children).

The availability of perceptual similarity and/or direct co-occurrence between Taxonomically related items also characterizes most studies that have investigated semantic relations in infants (Arias-Trejo & Plunkett, 2009; Bergelson & Aslin, 2017; Styles & Plunkett, 2009; Willits, Wojcik, Seidenberg, & Saffran, 2013). One exception is a study of Associative and Taxonomic relations in 21 and 24-month-old infants conducted by Arias-Trejo and Plunkett (2013). However, although this study provided clear evidence for Associative relations in 24-month-olds, the evidence for Taxonomic relations was less clear-cut.

Overall, there is growing evidence to suggest that the appearance of robust Taxonomic relations early in development may be due to the availability of other, non-Taxonomic information, such as direct co-occurrence or perceptual similarity. Specifically, no prior study that reported presence of robust Taxonomic relations early in development (i.e., at or below age four) has simultaneously controlled for both perceptual similarity and direct co-occurrence as an additional source of information children could use during testing. Critically, in the absence of such additional information, findings from numerous studies reviewed in this section provide evidence that Taxonomic relations emerge gradually over the preschool and early school years.

Summary

The evidence reviewed in this section is largely consistent with a role for direct and shared co-occurrence regularities in semantic development that fosters Associative relations from an early age, and gradually supplements these relations with Taxonomic relations. This support in turn highlights the importance of investigating this potentially powerful driver of semantic development, as discussed in the following section.

New Avenues of Research

This synthetic review has cast a spotlight on co-occurrence regularities as potentially powerful drivers of semantic organization in development. However, this possibility has received little direct investigation to date. Therefore, we highlight new questions opened up by this possibility that may further illuminate how we form richly semantically organized bodies of knowledge in the course of development.

Explicitly Evaluating the Role of Co-Occurrence Regularities in Semantic Development

In the course of this synthetic review, we have highlighted how the developmental changes in semantic organization observed in many studies are consistent with a key role for co-occurrence regularities. However, with few exceptions (e.g., Unger et al., 2020a), these studies were not explicitly designed to investigate this possibility.

Explicitly investigating the role of co-occurrence regularities in semantic development will require research approaches that deviate from those typically used to study semantic development. Specifically, much of the research in this field has involved measuring developmental changes in semantic relations between concepts that researchers or other adults judge as related in some specified way. For example, research investigating the development of Associative and/or Taxonomic relations has often involved identifying Associatively or Taxonomically related concepts based on the researchers’ own intuition (e.g., Bauer & Mandler, 1989; Gelman & Markman, 1986; Nguyen, 2007; Tversky, 1985; Waxman & Namy, 1997), ratings provided by adult participants (e.g., Deák & Bauer, 1996; Fenson et al., 1989; Unger et al., 2016; Walsh et al., 1993), or measures of association strength from normative responses on free association tasks (e.g., Nation & Snowling, 1999; Smiley & Brown, 1979). Importantly, these approaches can tell us little about the sources of information, such as co-occurrence regularities, from which semantic relations between concepts are learned. This insight can instead be gained by investigating the development of semantic relations that can be formed from measurable information available in the environment. Specifically, investigating the contributions of direct and shared co-occurrence to semantic development could involve testing the degree to which direct and shared co-occurrence regularities (measured from, e.g., recordings of child language input, MacWhinney, 2000) predict the development of semantic relations (Unger et al., 2020a; Unger et al., 2020b).

Development of Sensitivity to Co-Occurrence that Contributes to Semantic Relations

For co-occurrence regularities to be important drivers of semantic development, sensitivity to co-occurrence regularities that contributes to semantic relations must be present during development. The present review brings together multiple lines of evidence that sensitivity to co-occurrence regularities emerges in development. However, much of this research investigated sensitivity to co-occurrence regularities in domains that lie outside the domain of semantic relations between concepts, such as sensitivity to co-occurrence between speech sounds. Moreover, none of these studies were designed to investigate how sensitivity to co-occurrence regularities that contributes to semantic relations emerges and changes over the course of development. Therefore, investigating this developmental trajectory represents a key target for understanding the role of co-occurrence regularities in semantic development. Further insight into whether co-occurrence regularities represent key drivers of semantic development could be gained by investigating whether the development of sensitivity to co-occurrence regularities predicts the development of semantic organization within individual children.

Synergies between Co-Occurrence and Sensorimotor Input

Throughout this review, we have emphasized how co-occurrence regularities may contribute to the organization of our semantic knowledge about the world. Much of the research we have reviewed has focused on contributions of co-occurrence between words in language. In contrast, in most cases, developing humans can also readily observe the world around them. Developing humans therefore can observe co-occurrences between words in language and objects in the world, while also acquiring a wealth of sensorimotor input, such as the colors, shapes, textures of objects, the sounds objects produce, the way objects move, and so on. The contributions of co-occurrence regularities between words highlighted by much of the research described in this review may therefore be insufficient to explain semantic development on their own.

As stated in the opening of this review, our perspective is that co-occurrence between words, co-occurrence between objects, and sensorimotor input together contribute to the development of semantic organization. Some insight into the combined contributions of exposure to language and observing the world comes from research in the field of distributional semantics. First, co-occurrences between words in language may correspond closely to co-occurrences of their object counterparts in the world (Sadeghi et al., 2015). Thus, co-occurrence regularities between words and between objects may provide redundant sources of input for semantic relations between concepts. Second, the contributions of co-occurrence regularities may also be in part redundant with the contributions of perceptual similarity. This possibility is supported by evidence from Riordan and Jones (2011), that concepts can be organized in similar ways on the basis of both co-occurrence regularities in language, and overlap between sensory features recorded in feature norms (see analyses presented in Louwerse, 2011 for similar evidence). In addition to redundancy, co-occurrence regularities and sensory input may also represent complementary sources of input (Andrews, Frank, & Vigliocco, 2014; Andrews, Vigliocco, & Vinson, 2009; Bruni, Boleda, Baroni, & Tran, 2012; Bruni, Tran, & Baroni, 2014; Johns & Jones, 2012). Specifically, observing the world conveys some information more directly and reliably than language: For example, it is easier to observe the similarity in shape, texture, and manner of motion between cats and lions than it is to acquire the corresponding information from language. Conversely, language can be informative about semantic relations between concepts that are not readily observed in the world, for example that birds lay eggs. Evidence supporting the possibility that language and observing the world provide complementary sources of input comes from research with models that form representations of concepts based on co-occurrence regularities in language that are enriched with sensory information (e.g., from normative human reports of visual features or computer vision algorithms) (Andrews et al., 2009; Bruni et al., 2012; Bruni et al., 2014; Johns & Jones, 2012; Steyvers, 2010). Specifically, this research has provided evidence that representations formed from both co-occurrence and sensory input better account for human semantic phenomena (e.g., semantic similarity ratings) than representations formed from either source alone.

Further advances in understanding semantic development will likely require investigating how multiple sources of input mutually reinforce and constrain each other over the course of development. One potentially illuminating approach to tackling this issue is to investigate whether and how infants and children integrate information they acquire from co-occurrence (e.g., between words in language) and sensorimotor input. Alternative investigations may capitalize on emerging efforts to record and annotate both language and visual input to infants and children (Smith et al., 2015). Although this research is in its early stages and represents a lengthy and effortful undertaking, it may eventually provide insight into the wealth of co-occurrence and sensorimotor experiences that combine to drive semantic development.

Mechanisms

As discussed in this synthetic review, the proposed roles for co-occurrence regularities are motivated in part by evidence that much of the relations between concepts in human semantic knowledge can be captured by distributional semantics models that form representations of concepts based on co-occurrence regularities. This evidence suggests that co-occurrence regularities are a source of input from which humans may learn semantic relations. However, many of these models do not make commitments about the nature of the psychological processes through which exposure to co-occurrence regularities fosters semantic relations in humans (e.g., Landauer & Dumais, 1997; Lund & Burgess, 1996). Thus, we currently know little about the mechanisms by which exposure to co-occurrence regularities fosters semantic relations in humans, and how well these mechanisms are captured by existing models.

There are a number of open questions about the way in which exposure to co-occurrence regularities fosters semantic development. One key question pertains to the role of prediction in learning. For example, in some distributional semantics models that do propose processes through which exposure to co-occurrence fosters semantic relations (e.g., Huebner & Willits, 2018; Mikolov et al., 2013), there is a key role for prediction and errors generated when predictions are violated. In contrast, prediction does not play any role in other models and accounts of how relations may be formed from co-occurrence regularities (e.g., Jones & Mewhort, 2007; Thiessen, 2017; Thiessen & Pavlik Jr, 2013). Thus, future research may investigate the importance of prediction and prediction error in the formation of semantic relations from co-occurrence regularities in humans. Beyond comparing the success with which semantic relations are captured by models that do versus do not use prediction (e.g., Baroni, Dinu, & Kruszewski, 2014; Mandera, Keuleers, & Brysbaert, 2015), future research might expose participants to input rich in co-occurrence regularities, and assess whether the formation of corresponding semantic relations is related to prediction measured during exposure.

Another key question pertains to how Associative and Taxonomic relations are distinguished from each other when they are learned from exposure to direct and shared co-occurrence regularities. Specifically, most distributional semantics models do not clearly distinguish between Associative relations that can be formed from direct co-occurrence, and Taxonomic relations that can be formed from shared co-occurrence (Jones & Mewhort, 2007; Landauer & Dumais, 1997; Lund & Burgess, 1996). In contrast, evidence from humans highlights distinctions between these relations. First, as described in the present review, sensitivity to direct co-occurrence regularities that can foster Associative relations develops earlier than sensitivity to shared co-occurrence regularities that can foster Taxonomic relations. Moreover, adults tend to treat Associative and Taxonomic relations as different in nature (e.g., Gentner & Brem, 1999). For example, adults typically see the way in which chickens are related to eggs as different from the way in which they are related to pigeons. One avenue for incorporating these distinctions in models that has recently been proposed is to posit two interconnected learning systems: One that forms Associative relations from direct co-occurrence, and another that takes the relations formed in the first system as input for forming Taxonomic relations from shared co-occurrence (Sloutsky et al., 2017).

Summary

As highlighted in this synthetic review, the possibility that co-occurrence regularities act as simple but powerful drivers of semantic development is supported by evidence from a variety of lines of research, including behavioral studies with children and adults, and computational modeling studies. Importantly, the reviewed evidence suggests that co-occurrence regularities can foster the emergence of both Associative and Taxonomic relational knowledge, and explain the different developmental trajectories for emergence of these semantic relations. However, few of the reviewed studies were conceived as direct investigations into this possibility. Therefore, the proposal that co-occurrence regularities drive semantic development opens several avenues for future research with the potential to illuminate how we build richly organized bodies of knowledge.

Figure 2.

Schematic summary of the reviewed evidence for the developmental trajectories of sensitivities to direct and shared co-occurrence, and knowledge of Associative and Taxonomic relations. (A) Sensitivity to direct co-occurrence is documented in a large body of research starting from early infancy, and improves over the course of infancy. (B) Knowledge of Associative relations between familiar concepts is relatively robust from toddlerhood onward. With development, children’s knowledge of Associative relations may expand to incorporate new concepts. (C) A small body of evidence suggests that sensitivity to shared co-occurrence may be present to some degree in infancy. This sensitivity undergoes substantial improvement with development through adolescence. (D) In toddlers and young children, knowledge of Taxonomic relations between even familiar concepts may be comparatively weak, such that it is only evident when highlighted by tasks or supported by other commonalities between concepts. Knowledge of Taxonomic relations becomes increasingly robust with development.

Highlights.

• Co-occurrence regularities have been overlooked as a driver of semantic development

• Co-occurrence in language and visual input contribute to semantic development

• Evidence from behavioral and modeling studies supports these conclusions