Symbol ungrounding: what the successes (and failures) of large language models reveal about human cognition

Abstract

Large language models can handle sophisticated natural language processing tasks. This raises the question of how their understanding of semantic meaning compares to that of human beings. Supporters of embodied cognition often point out that because these models are trained solely on text, their representations of semantic content are not grounded in sensorimotor experience. This paper contends that human cognition exhibits capabilities that fit with both the embodied and artificial intelligence approaches. Evidence suggests that semantic memory is partially grounded in sensorimotor systems and dependent on language-specific learning. From this perspective, large language models demonstrate the richness of language as a source of semantic information. They show how our experience with language might scaffold and extend our capacity to make sense of the world. In the context of an embodied mind, language provides access to a valuable form of ungrounded cognition.

This article is part of the theme issue ‘Minds in movement: embodied cognition in the age of artificial intelligence’.

1. Introduction

How does the natural language processing (NLP) of contemporary artificial intelligence (AI) systems compare to that of human beings? This has become a central theoretical question in the discussion surrounding the effectiveness of large language models (LLMs). Deciphering the extent to which systems employing these models understand, or fail to understand, semantic content has become something of a cottage industry (e.g. [1,2]). The fact that such systems are trained only on text raises the possibility that the knowledge that they acquire is insufficiently grounded in experience with the physical world. Notable failures of extant models suggest that this problem may be endemic to them [3].

Given that a lack of grounding has become a widely recognized theoretical challenge for LLMs, there is reason to look to another contemporary research programme, embodied cognition, for a possible solution to this problem. Embodied cognition emphasizes the degree to which human cognition depends on experiential systems associated with bodily action and perception. The stratagem of evaluating LLMs in relation to embodiment has led to the following two general responses, one optimistic and the other sceptical. The optimistic approach seeks to mimic the embodied nature of human cognition by adding models trained on perceptual data or incorporating epigenetic robotics. The sceptical approach contends that the disembodied character of LLMs excludes them from having access to a rich understanding of semantic meanings.

The purpose of this essay is to move away from this dichotomous conceptual framework. I argue that aspects of human cognition fit with both the embodied and AI approaches. More specifically, there are several reasons to think that human semantic memory is both partially grounded in sensorimotor systems and dependent on language-specific learning. From this perspective, LLMs amount to an exaggerated version of an important human skill. Despite their biological implausibility and massive training input, they demonstrate the richness of language as a source of semantic information about our world. At a minimum, they serve as a proof of concept with respect to how language might extend our cognitive reach [4,5]. In other words, they show how experience with language might scaffold and extend our capacity to make sense of the world. Although my focus will primarily be on the implications that LLMs have for our understanding of human cognition, this proposed division of cognitive labour has implications for the prospects of an embodied AI.

2. Surprising successes and disappointing failures

LLMs are neural networks containing up to hundreds of billions of parameters. They are typically transformer-based models [6] trained on massive sets of human-produced text. For example, even an early version of the LLM, GPT-3 (Generative Pre-trained Transformer 3), was initially trained on a corpus consisting of approximately 200 billion words—estimated to be equivalent to 20 000 years’ worth of human experience [7]. Fundamentally, what these models do is carry out a sequence prediction task. More specifically, they treat words or parts of words as individual tokens and attempt to predict the next token in a particular series. They are thus generative mathematical models of the statistical distribution of such tokens in the corpus that they have been trained on.

The capacity of such models to generate further text in response to a given chunk of text is often described as a kind of pattern completion [8]. Despite the seemingly basic nature of the task, systems employing LLMs exhibit what appears to be extraordinarily sophisticated performance on language understanding tasks. For instance, several recent models have been shown to exhibit human-like performance on language understanding metrics specifically designed to challenge NLP systems [9–11]. These metrics include comprehension, entailment, question–answer, reasoning, recognition and summarizing tasks.

Recent research suggests that LLMs can develop intriguing degrees of competence with respect to the formal aspects of language processing [7,12]. LLMs appear to glean information about subject–verb agreement [13,14], constituent labelling [15] and filler–gap dependencies [16]. Syntactic information can be recovered from the word embeddings of BERT (Bidirectional Encoder Representations from Transformers) [17]. Despite these successes, LLMs have also exhibited disturbing patterns of failure in relation to understanding aspects of semantic meaning. For example, some systems containing LLMs have been found to struggle with negation [18–20]. LLMs can also struggle with pragmatic inferences [21], common sense reasoning [22] and understanding the relationships between properties and affordances [23].

Reviewing the literature on the language processing of LLMs, Mahowald et al. [24] draw a distinction between formal and functional types of linguistic competence: formal linguistic competence involves knowledge of the rules and distributional patterns of a natural language, and functional linguistic competence involves understanding how language applies to real-world situations. They argue that LLMs exhibit a lot of promise with respect to capturing formal linguistic competence but struggle greatly when it comes to functional linguistic competence. They attribute this divergence to the fact that real-life language use requires non-linguistic cognitive skills.

3. Embodied AI

LLMs show great potential with a diverse collection of language-related processing and production tasks, but questions remain with respect to whether they can, even in principle, achieve human-like understanding [3]. Bender & Koller offer a blunt and categorical assessment ([25, p. 5185]; italics in the original): ‘We argue that the language-modeling task, because it only uses form as training data, cannot in principle lead to learning of meaning’. They argue that meaning involves a relation between linguistic symbols and something external to language. In particular, they identify meanings with the communicative intentions behind utterances. These intentions may involve conveying information, but they can also encompass actions and social activities. Because LLMs lack communicative intentions, they lack access to meanings. Shanahan [8, p. 5] offers a similarly sceptical take—albeit one that does not require a commitment to a specific conception of linguistic meaning: ‘A bare bones LLM doesn’t “really” know anything because all it does, at a fundamental level, is sequence prediction’. The idea is that although some of the sequences that it produces may fit the form of propositions, this does not mean that the models have any meaningful understanding of the states of affairs that these propositions capture.

The reasoning behind this scepticism echoes the symbol grounding problem [26], which arises with any computational account that characterizes cognition in terms of the relation of symbols to other symbols. This kind of functional organization leads to a symbolic merry-go-round in which cognitive processes involve going from one meaningless symbol to another. To help illustrate the nature of the problem, Harnad [26] asks us to imagine trying to learn Chinese as a new second language solely by means of a Chinese-to-Chinese dictionary. The obvious difficulty is that each word would be defined in terms of its connections to other words in the unknown language. LLMs carry out this sort of symbol-to-symbol processing on a grand scale.

Cognitive scientists have traditionally assumed that computational theories of cognition that are developed in a top–down manner will eventually connect with theories of peripheral experiential systems. In other words, the presumption has been that the cognitive system would ultimately be connected to the external world by means of links between amodal conceptual representations and those contained within functionally independent action and perception systems. Harnad [26] argues that this approach underestimates the extent of the symbol grounding problem and that a different approach is needed. He suggests that theories of cognition should incorporate intrinsically dedicated symbols that retain their connection to the world.

Embodied cognition explores this approach. It is a broad research programme that emphasizes the ways in which the neural systems associated with bodily action and perception contribute to cognition [27]. Theories of embodied cognition tend to focus on the contribution of the systems associated with processing the regularities of physical and social experiences. Within the context of semantic memory, evidence implicating affective and sensorimotor systems in cognition has led to the influential grounded cognition model [28], which holds that we think about objects and events by means of the same mechanisms that experience and interact with them. According to this model, conceptual processing relies on the reactivation of neural circuits contained within modality-specific regions of the cortex—that is, thinking involves simulations of experience.

There are good reasons to speculate that embodied cognition could help researchers develop better NLP systems. Some of the weaknesses of LLMs appear to be connected to their lack of access to suitably grounded representations. In keeping with this, some researchers have explored the possibility of embedding LLMs with other networks trained on perceptual data such as visual images [29,30]. Others have explored the possibility of embedding LLMs within systems that directly interact with the world, such as robots [31]. My aim in this paper is to explore a related speculation, namely, that the sort of language-based processing exhibited by LLMs might help us recognize and overcome the theoretical shortcomings associated with a strictly embodied approach to human cognition.

4. Symbol ungrounding

The symbol grounding problem has become part of the origin story of embodied cognition because a cognitive system that relies on the redeployment of representations that are grounded in sensorimotor experience seems well-positioned to overcome this problem [32,33]. Unfortunately, linking cognition to bodily experience creates a different problem. Although embodiment holds promise with respect to the symbol grounding, grounded cognitive systems face a corresponding symbol ungrounding problem: How does a system that relies on representations contained within action, perception and emotion systems acquire knowledge that requires going beyond particular experiences? Conceptual knowledge requires abstraction from perceptual and motor particulars [34].

The need for symbol ungrounding in the context of embodied cognition has received two prominent, yet relatively independent, treatments. The first originates from the theoretical framework developed by Terrence Deacon [35]. Rączaszek-Leonardi & Deacon [36] argue that the way researchers typically conceptualize the symbol grounding problem is flawed for two reasons. The first flaw is that the problem is typically formulated in a way that assumes that cognitive systems have prior access to abstract, language-like symbols. This seems implausible from a developmental perspective. The second flaw is that it fails to acknowledge the possibility that ‘non-symbolic structures also play a vital role in regulating the relationships among organisms and—crucially—the emergence of the symbols themselves’ [36, p. 232]’. They propose that symbols become endowed with meaning through a process that depends on simpler semiotic relationships—associated with icons and indices—that are linked to bodily dynamics and action in the world.

The shift to an embodied perspective creates a new theoretical challenge: How does symbolic meaning emerge out of complex interactive contexts? As Rączaszek-Leonardi & Deacon [36, p. 232] put it:

In such contexts it becomes evident that the real problem is not the grounding of symbols but rather explaining the mystery of how such an embedded, embodied and situated use of signs can ever (at least partially) become liberated from the immediate reliance on the online events, thus, how they become, at least partially, ungrounded.

The emergence of symbolic cognition out of the dynamical activity associated with perceiving and acting in specific contexts requires a process of symbol ungrounding. Explaining just how this happens is a fundamental theoretical challenge for any approach that emphasizes the importance of situated, embedded and embodied action. From this perspective, language development becomes a central test case for symbolic emergence. The question at hand for researchers is how meaningful symbols emerge from the grounded interactions of children and their linguistic and social niches. Rączaszek-Leonardi & Deacon [36] suggest that children capitalize on the relations between iconic and indexical signs. Given that this approach focuses on situated bodily action, it is not surprising that they emphasize the promise of the field of epigenetic robotics. At first glance, their approach would seem to have little in common with that of contemporary NLP. Nevertheless, LLMs provide a clear example of how language-based ungrounding might occur.

The second treatment of symbol ungrounding occurs in my own work. In an early paper [37], I argue that abstract concepts pose three variants of the symbol ungrounding problem for embodied cognition. The first of these is the problem of generalization. Many abstract concepts involve generalizing away from certain experiential particulars. For instance, superordinate concepts such as MAMMAL have a broader scope than basic-level concepts such as DOG and subordinate concepts such as PUG. The second is the problem of flexibility. Abstract concepts tend to be more context-dependent than concrete concepts [38,39]. The third is the problem of disembodiment. Certain abstract concepts, such as PRIME NUMBER, QUARK and TRUTH, appear to be fundamentally divorced from sensorimotor experience [40]. More recently, I have emphasized the universality of the problem of abstraction for theories of concepts [34].

Embodied cognition struggles to explain our capacity to employ concepts that are disconnected from affective, motor and perceptual experience. Experience with language may help our brains overcome the symbol ungrounding problem [41] by offering us access to an external symbol system that has different computational properties from non-linguistic grounded cognition [4]. Some of our knowledge may be captured by means of the associative relationships between linguistic representations and other linguistic representations [5,42]. In other words, cognition may be shaped to some significant degree by our experiences with words, sentences and conversations [43].

5. A rich source of information

LLMs demonstrate how knowledge pertaining to language use can provide access to semantic content that goes beyond sensorimotor experience. Language provides an additional—yet in many ways complementary—source of information about the world. I am going to defend this hypothesis by pointing to research in two domains. First, evidence suggests that language can provide help with the acquisition and use of concrete concepts, even perceptually based ones, when normally grounded experience is unavailable. Second, there are compelling reasons to think that language-based information provides an important leg up with respect to the acquisition and use of abstract concepts. This fits naturally with the idea that language provides a means of symbol ungrounding.

(a). Colour concepts

Bender & Keller [25, p, 5192] argue that LLMs lack an understanding of linguistic meaning because meaning ‘is based on a link between linguistic form and something that is not language’. What happens when individuals lack the appropriate grounded access to something external to language? Consider the case of congenitally blind individuals and colour concepts. Clearly, such individuals lack perceptual access to the referents of these terms. This example has a clear historical precedent. Hume [44, p. 15] famously asserts:

If it happen, from a defect of the organ, that a man is not susceptible to any species of sensation, we always find that he is as little susceptible of the correspondent ideas. A blind man can form no notion of colours; a deaf man of sounds.

Although this may initially seem to be a definitional or a priori claim, it is in fact a defeasible empirical claim. Indeed, the semantic understanding of colour terms by blind individuals is a well-studied area of psycholinguistic and cognitive neuroscience research. Congenitally and early blind individuals have been shown to have sophisticated semantic knowledge of colour similarity [45] and the colour of objects [46–48].

Gubelmann [49] argues that the ability of congenitally and early blind individuals to acquire rich colour concepts shows that grounded representations are not required for language understanding—even in the context of semantic domains that involve perceptual experience. In keeping with this, a research programme for defending the sophistication of the semantic knowledge acquired by NLP systems has emerged. Pavlick [50, p. 11] explains, ‘One way to show that grounding is not necessary for learning meaning is to show that conceptual representations learned by an (ungrounded) LLM are isomorphic to grounded representations of those same concepts’. Evidence suggests that it is possible to glean a substantial amount of information about colour from the statistical co-occurrence patterns of colour words [51]. LLMs appear to be able to acquire contextual representations that correspond to the human perception of colour [52–54].

None of this is to say that we do not typically acquire semantic knowledge through our direct experience of the world. It is just that the ability of ungrounded LLMs to learn about experiential phenomena such as colour suggests that language-based cognition can be an alternative route to such knowledge—particularly in the context of an otherwise grounded cognitive system. As Borghi et al. [55, p. 362] point out, the success of these models raises the possibility that ‘humans could also extract part of their knowledge and skills in a similar manner’.

(b). Abstract concepts

A diverse body of evidence suggests that language plays an important role in our ability to acquire and use abstract concepts [41,56]. Some of this evidence implicates the sort of distributional information leveraged by LLMs. For instance, hybrid models that statistically combined language-based distributional data and non-linguistic experiential data have been found to match up with observed human performance better than models that relied exclusively on one of these forms of data [57–59]. These results suggest that the ability to take advantage of the distributional information contained within natural language might benefit an otherwise grounded conceptual system.

Given the apparent importance of language to the encoding and processing of abstract concepts, we would expect LLMs to be particularly effective at handling abstract concepts. Unfortunately, this may not be the case. Liao, Chen and Du [60] examined the conceptual understanding capabilities of six different pre-trained LLMs. More specifically, they compared how these LLMs handled abstract and concrete concepts on a hypernym discovery task. They found that, across the board, the LLMs performed more poorly with abstract concepts than with concrete ones. This result is surprising when compared to the previous success of distributional semantic models.

Clearly, more research is needed to uncover just what is going on here. Several possible explanations come to mind. It could be that the task was not the right one for such a comparison. Or it could be a problem specific to LLMs. They have been shown to struggle with abstract reasoning [61]. In addition, abstract concepts tend to be more polysemous than concrete concepts [62], and polysemy has been shown to be a particular problem for LLMs [63]. Our ability to understand abstract concepts may also be more dependent on context-dependent pragmatic factors that are not captured well by LLMs [55].

6. From how-possibly to how-actually

The ability of LLMs to meet or exceed pre-established benchmarks suggests that they have achieved a moderate form of semantic competence. They thus provide a how-possibly explanation for the acquisition of some conceptual knowledge. They do this by simultaneously demonstrating the informational richness of the linguistic input and the power of deep learning architectures. This form of language-based learning fits well with certain aspects of human semantic competence, such as the ability of congenitally blind individuals to acquire rich colour concepts and our general capacity to acquire abstract concepts that go well beyond sensorimotor experience.

One might object, though, that there is an important disanalogy between the human and AI cases. ¹ Unlike LLMs, humans have direct access to grounded conceptual content. Consider the acquisition of colour concepts by the congenitally blind. It is not clear that these individuals learn these concepts in the same way that LLMs do—that is, through an analysis of statistical co-occurrence patterns in language. After all, these individuals would likely have access to grounded conceptual knowledge associated with other sensory modalities. Perhaps they can leverage this knowledge in the process of acquiring colour concepts. If this is correct, then any model that relies on distributional semantics is unlikely to help in the search for a how-actually explanation.

I maintain that there are several reasons to be interested in the sort of processing contained within LLMs. First, they provide more than just a possible explanation of how semantic competence is achieved. As outlined above, they also provide a promising explanation of how formal linguistic competence is acquired. If our brains use statistical co-occurrence information to acquire this sort of competence, it seems reasonable to conjecture that they might also use it to acquire some aspects of semantic competence. In other words, LLMs enjoy some promise with respect to explanatory power.

Second, LLMs have indirect access to grounded conceptual content—it is just that this content comes from the utterances that they are trained on. These utterances are produced by grounded cognitive agents. Criticisms of the purported status of LLMs as understanders of language often turn on this observation [7,8,25]. Recall, though, that I am proposing that LLMs support a how-possibly conjecture with respect to symbol ungrounding. As this terminology suggests, this process requires access to grounded symbols. The idea is that LLMs show how such ungrounding might occur. In keeping with this, Lupyan et al. [64, p. 938] claim that ‘co-occurrence statistics can act as a kind of echo of real-world linkages and causal relationships’. The suggestion on offer is that our brains take advantage of this echo.

Finally, even though LLMs are unrealistic models of our cognition for several reasons—including the size of their training sets, the nature of their learning algorithms and the number of parameters they involve—they might still contribute to a research programme aimed at providing a how-actually explanation of human cognition. Buckner [65] argues that deep learning networks fit with a moderate form of empiricism tied to faculty psychology. If this is correct, LLMs can be viewed as an initial step in this direction.

7. Conclusion

Rather than treat LLMs as fully realized consumers of semantic knowledge or embodied cognition as a full-throated counterexample to the plausibility of AI, I have argued that lessons can be learned by putting them into conversation with each other. Many of those interested in the promise of LLMs—and by extension, of systems in which they are embedded—have been so focused on ambitious assessments of their capacity to understand language that they have tended to dismiss problems arising from a lack of effective grounding of these systems. Supporters of embodied cognition, on the other hand, have been so focused on overcoming symbol grounding problems that they have failed to adequately recognize the need for symbol ungrounding.

Reframing the LLMs as an exaggeration of an important aspect of human cognition enables a kind of rapprochement between the two sides of the debate concerning the ability of LLMs to master linguistic meaning. Critics tend to argue that the outputs of systems employing them amount to little more than sophisticated collage [66] or pretending to think [24]. Supporters tend to argue that LLMs have the capacity to achieve a special sort of language-based semantic competence [49].

The suggestion explored in this essay is that our experience with language provides a means of symbol ungrounding for what is otherwise a largely embodied conceptual system. In other words, the abstractness of language-derived semantic representations is a feature rather than a bug. There are times in which pretending to think in the absence of a fully grounded mechanism is useful and advantageous. To some important degree, learning a language requires faking it until one makes it. The very features of LLMs that lead people to question them as successful models of knowledge make them useful in the context of the symbol ungrounding problem.

This approach has consequences for future research into how we think and how machines might think. With respect to cognitive science, LLMs can be viewed as biologically implausible models of the way that language-based experience might help to unground a fundamentally embodied conceptual system. With respect to AI, the need for grounding provides an impetus to explore larger systems that include action- and perception-oriented representations. More generally, symbol ungrounding appears to be an important design feature of cognition, whether human or artificial.

Ethics

This work did not require ethical approval from a human subject or animal welfare committee.

Data accessibility

This article has no additional data.

Declaration of AI use

We have not used AI-assisted technologies in creating this article.

Authors’ contributions

G.D.: conceptualization, writing—original draft, writing—review and editing.

Conflict of interests

I declare I have no competing interests.

Funding

No funding has been received for this article.