Lexico-semantics obscures lexical syntax

Abstract

A recently emerging generalization about language and the brain is that brain regions implicated in language that show syntax-related activations (e.g., increased activation for more complex sentence structures) also tend to show word-related activations, such as increased activation to reading real words (e.g. RAIN) relative to pseudowords (e.g. PHREZ) (Fedorenko et al., 2016). Fedorenko et al., (2020) generalize as follows: “…syntactic/combinatorial processing is not separable from lexico-semantic processing at the level of brain regions—or even voxel subsets—within the language network”. Based on this generalization, Fedorenko et al. have made the conclusion “…that a cognitive architecture whereby syntactic processing is not separable from the processing of individual word meanings is most likely”, arguing against “syntax-centric” views of language as promulgated by Chomsky and others. However, the notion of “lexico-semantics” articulated here obscures the fact that words are both syntactic and semantic entities. Because of this, any functional neuroimaging experiment that manipulates lexicality will almost assuredly tax both syntactic and semantic resources, and is therefore inadequate for isolating conceptual-semantic processing in the brain in exclusion to syntax. In addition, Fedorenko et al. do not satisfactorily account for robust lesion data that shows clear functional-anatomical dissociations within the language network. Finally, a “syntax-centric” view of language is perfectly compatible with Fedorenko et al.’s conclusions about syntax-selectivity in the brain because of the multiple potential mappings between linguistic theory and neurobiology beyond individual brain regions.

Words and Rules

“The way language works, then, is that each person’s brain contains a lexicon of words and the concepts they stand for (a mental dictionary) and a set of rules that combine the words to convey relationships among concepts (a mental grammar).”

(Pinker, 1995)

Pinker’s work had a major impact in popularizing the ideas of Chomsky and mainstream generative grammar (MGG). Like Chomsky, Pinker forcibly argued for the concept of an innate linguistic module of the human brain that allows us to learn language. However, his work included some simplifications that have become ensconced in cognitive science. A major one was “Words and Rules” (Pinker, 1999; Pinker & Ullman, 2002): the idea that language is fundamentally two diametrically-opposed systems, a database of form-meaning pairs (the lexicon) and a rule-based, combinatorial system (syntax), each rooted in distinct underlying brain systems (Figure 1, left). Pinker combined the traditional Saussurean notion of human language as fundamentally a system of arbitrary form-meaning pairs with the focus of generative grammar on combinatorial syntactic operations.

Figure 1.

LEFT: The “Words and Rules” theory of Pinker (1999), adapted from Pinker & Ullman, (2002). Separate cognitive systems are posited, each with distinct memory mechanisms, one containing a lexicon of form-meaning pairs (“words”) and one involved in combining lexical elements together to create complex structures and semantic relations (“rules”). RIGHT: The Y-model posited in mainstream generative grammar (e.g. Chomsky, 1965; 1980; 1995). A syntactic module which combines lexical elements into complex structures is then interpreted in two separate modules, one for phonetic form (how an expression is pronounced), and one for semantic form (the meaning of an utterance). In the Y-model, there is only an indirect and loose connection between the lexicon and meaning, whereas in the Words and Rules model the lexicon is a repository of form-meaning pairs.

While many have taken the “Words and Rules” theory of Pinker to accurately summarize the MGG approach, these approaches are actually fundamentally incompatible (see also Embick & Marantz, 2005). The first models of MGG didn’t even contain a lexicon (Chomsky, 1955, 1957). Later models understood words as syntactic objects; inputs to syntactic computation, only later acquiring phonological and semantic expression (the inverted Y-model; (Chomsky, 1965, 1981, 1995b) (Figure 1, right). Even further, (Chomsky, 1995a) proposed the theory of bare phrase structure, in which the labels of phrasal projections are not traditional syntactic categories like noun and verb, but are rather derived from the lexical items themselves. E.g., instead of verb phrases there are “eat phrases”, instead of noun phrases there are “cat phrases”, etc. Some researchers have pointed to compelling evidence that words do not bear a direct mapping to meaning as Saussure claimed (Pietroski, 2018; Preminger, 2021), referring to phenomenon such as polysemy, in which the meaning of the same lexical item, e.g. “book”, is determined by syntactic context (e.g., the book’s pages were torn – physical object, the book has challenged millions of readers to re-examine their views – abstract collection of words).

I do not claim that this approach is incontrovertibly correct. However, almost every modern (psycho-)linguistic approach acknowledges that words have syntax, a point not raised or addressed by Fedorenko et al. (2020). The most popular alternative approach to MGG, the Construction Grammar/usage-based approach (Goldberg, 2003; Jackendoff, 2002), while advocating for a clearly distinct approach to language, does not abolish the distinction between syntax and semantics, but rather articulates that words and constructions are pairs of syntactic form and meaning. In addition, popular psycholinguistic models of word production involve two stages: the first stage involves going from meaning to the lemma, which includes the syntactic representation of a concept, and in the second stage from the lemma to phonological form (Dell & O’Seaghdha, 1992; Kempen & Huijbers, 1983; Levelt, 1989; Levelt et al., 1999)¹. The idea that word retrieval involves access to syntactic information, even in the context of single word production, is mostly uncontroversial in this literature. Thus, the idea of a coherent lexico-semantic system entirely distinct from and diametrically opposed to syntax, is in many ways an aberration, yet it appears to have had substantial impact on cognitive neuroscience.²

Functional differentiation in the language network

Fedorenko et al.’s claims about the neurobiological implementation of language are unusually strong and are based on the problematic notion of “lexico-semantics” reviewed above. First, Fedorenko and colleagues (2020) argue against the idea of a purely syntactic system in the brain because several previous studies “found that any language-responsive brain region or electrode that shows sensitivity to syntactic structure… is at least as sensitive, and often more sensitive, to meanings of individual words.” This is simply the observation that regions implicated in syntax activate more to real words than pseudowords;^3,4 there is no evidence that these activations only reflect meaning. A syntactic system in the brain should activate more to real words relative to pseudowords, reflecting access to syntactic elements as reviewed above. Thus, the “lexico-semantic” activations reported in these experiments are ambiguous between lexical-syntactic and lexical-semantic processing.

Then, Fedorenko et al. (2020) performed a series of three additional functional magnetic resonance imaging (fMRI) experiments designed to separately target “lexico-semantic” and syntactic processing, following similar experimental designs in previous research (Dapretto & Bookheimer, 1999; Friederici, 2003; Kuperberg et al., 2000, 2003; Menenti et al., 2011; Noppeney & Price, 2004; Segaert et al., 2012). Here are example stimuli from the critical, putatively “lexico-semantic” conditions in Fedorenko et al. (2020). In these experiments, the initial sentence or clause is followed up by a second sentence or clause which contrasts with the initial one:

• Experiment 1: “Although his ears were damaged… the man could still cook” (meaning violation)

• Experiment 2: “The protestor quoted the leader -> The striker cited the chief” (different words)

• Experiment 3: “The scientist flattered David -> The scientist misled David” (non-synonym)

All three of these experiments conflate syntax and semantics within the “lexico-semantic” condition. In Experiment 1, the meaning violation is also a violation of the expected word, which is a syntactic as well as semantic element. In addition, it is quite possible that violations of meaning are accompanied by syntactic revision processes in order to attempt to re-interpret the sentence. In Experiment 2, different words are different syntactic elements as well as different meanings. In Experiment 3, the manipulation involves changing the final word, which is again both a syntactic and semantic element. It is therefore no surprise that brain areas thought to be potentially selective to syntax (such as the inferior frontal lobe and posterior temporal lobe as postulated by many authors, Bornkessel-Schlesewsky & Schlesewsky, 2013; Friederici, 2017; Hagoort, 2005; Matchin & Hickok, 2020; Tyler & Marslen-Wilson, 2008) show activations to both the “lexico-semantic” and syntactic conditions, because the “lexico-semantic” conditions always involve a “hidden” syntactic manipulation. That is, these experiments always manipulate words, which are intrinsically syntactic as well as semantic, which is a consensus position in linguistic theory as reviewed above.

Researchers using similar experimental conditions in brain imaging research have reported dissociations of syntactic and semantic processing (Dapretto & Bookheimer, 1999; Friederici, 2003; Kuperberg et al., 2000, 2003; Menenti et al., 2011; Noppeney & Price, 2004; Segaert et al., 2012), which seem discrepant with the results reported by Fedorenko et al. (2020). However, these previous authors reported whole-brain activation maps, whereas Fedorenko et al. do not. It is possible that the subtle spatial dissociations reported by previous authors would be replicated in the Fedorenko et al. experiments if whole-brain analyses had been reported.⁵ Regardless, it is more important that these previous authors operated under the same mistaken assumption as Fedorenko et al.: that “lexico-semantic” manipulations do not tax syntax. The fact that Fedorenko et al. do not (appear to) replicate the syntax-semantics dissociations reported by previous authors is more likely due to the fact that these original experimental designs were flawed to begin with.

Future functional neuroimaging studies investigating the syntax-semantics distinction should account for the dual semantic and syntactic nature of the lexicon by eschewing the conventional notion of “lexical-semantics” itself. Instead, researchers should develop more careful experiments that independently vary the richness of conceptual-semantic content and lexical-syntactic complexity, or separately model these components during sentence comprehension (see Hale et al., 2022; Pylkkänen, 2019, 2020 for reviews).

Syntax in the brain: multiple viable instantiations

Fedorenko et al. (2020) focus on the idea of a syntactic system in the brain which should not be activated by lexical manipulations, a prediction which does not follow from “syntax-centric” theories of MGG (e.g. Chomsky 1965; 1980; 1995). However, they do report a significant preference for the syntactic condition in the posterior temporal lobe for Experiment 2 when using a localizer more sensitive to syntax. It is not a coincidence that the posterior temporal lobe is strongly implicated in syntax by recent authors (Bornkessel-Schlesewsky & Schlesewsky, 2013; Matchin & Hickok, 2020; Pylkkänen, 2019). Recent lesion-symptom mapping literature supports a strong association between syntactic comprehension deficits and damage to posterior temporal-parietal areas (Matchin, Basilakos, et al., 2022; Matchin, den Ouden, et al., 2022; Pillay et al., 2017; Rogalsky et al., 2018), a similar pattern that is also emerging for paragrammatic speech production deficits (Matchin et al., 2020; Yagata et al., 2017). Residual functional activation in posterior temporal lobe after accounting for lesion effects appears to be uniquely associated with aphasia recovery (Wilson et al., 2022; Schneck, 2022). Given that lesion-symptom mapping provides much stronger causal inference than functional neuroimaging (Rorden & Karnath, 2004), such data need to be addressed together.

Finally, while I find the evidence for a hierarchical, abstract syntactic system in posterior temporal lobe to be highly compelling, a variety of multiple perspectives on this issue are possible. First, even if no brain region is selective for syntax, specific network configurations could be (Anderson, 2016; Buzsaki, 2006; Farahani et al., 2019; Schnitzler & Gross, 2005). Further, linguistic theories do not make predictions about how much cortical surface area would be needed to process syntax and semantics, or whether there must be large cortical areas dedicated to processing syntax at all (Embick & Poeppel, 2015; Poeppel & Embick, 2005). The ideas of Chomsky regarding the uniqueness and expressive power of syntax are perfectly compatible with a “slight rewiring of the brain” (Chomsky, 2005) of an evolutionarily recent hominin ancestor, augmenting a sea of brain mechanisms that resulted in the modern human language faculty (Berwick & Chomsky, 2016), regardless of whether there is clear evidence of a large swath of syntax-selective cortex.