Towards a New Neurobiology of Language

Abstract

Theoretical advances in language research and the availability of increasingly high-resolution experimental techniques in the cognitive neurosciences are profoundly changing how we investigate and conceive of the neural basis of speech and language processing. Recent work closely aligns language research with issues at the core of systems neuroscience, ranging from neurophysiological and neuroanatomic characterizations to questions about neural coding. Here we highlight, across different aspects of language processing (perception, production, sign language, meaning construction), new insights and approaches to the neurobiology of language, aiming to describe promising new areas of investigation in which the neurosciences intersect with linguistic research more closely than before. This paper summarizes in brief some of the issues that constitute the background for talks presented in a symposium at the Annual Meeting of the Society for Neuroscience. It is not a comprehensive review of any of the issues that are discussed in the symposium.

Introduction

Until ∼25 years ago, our knowledge of the brain basis of language processing, the mental faculty considered to be at the very core of human nature, derived largely from rather coarse measures (neurobiologically speaking): deficit-lesion correlations in stroke patients, electrophysiological data from electroencephalographic as well as occasional intracranial recordings (associated with surgical interventions), and relatively crude measures such as the Wada test for language lateralization. The “classical model” of brain and language, developed in the 19th century by the insightful neurologists Broca (1861), Wernicke (1874), Lichtheim (1885), and others, dominated discourse in the field for more than one hundred years. To this day the Wernicke-Lichtheim model of a left inferior frontal region and a posterior temporal brain area, connected by the arcuate fasciculus fiber bundle, is familiar to every student of the neurosciences, linguistics, and psychology. This iconic model, published in virtually every relevant textbook, had (and continues to have) enormous influence in discussions of the biological foundations of language (for a famous modern instantiation, see Geschwind, 1970, 1979, reproduced in Fig. 1A).

Figure 1.

A, The classical brain language model, ubiquitous but no longer viable. From Geschwind (1979). With permission of Scientific American. B, The dorsal and ventral stream model of speech sound processing. From Hickok and Poeppel (2007). With permission from Nature Publishing Group.

The big questions, with regard to the neurosciences, concerned “lateralization of function” and “hemispheric dominance;” with regard to the cognitive sciences and language research, they concerned apportioning different aspects of language function to the anterior versus posterior language regions, e.g., production versus comprehension, respectively, although the oldest versions (i.e., Wernicke's), were actually more nuanced than much subsequent work. Crucially (and problematically), the assessment of speech, language, and reading disorders in clinical contexts continues to be informed largely by the classical view and ideas and data that range from the 1860s to the 1970s.

Not surprisingly, but worth pointing out explicitly, the era of the classical model is over. The underlying conceptualization of the relation between brain and language is hopelessly underspecified, from the biological point of view as well as from the linguistic and psychological perspectives (Poeppel and Hickok, 2004). As useful as it has been as a heuristic guide to stimulate basic research and clinical diagnosis, it is now being radically adapted, and in parts discarded. That being said, its longevity is a testament to how deeply the classical model has penetrated both the scientific and popular imaginations; few scientific models in any discipline have such staying power.

Two major changes in the conceptualization of brain-language relations have occurred, both related to better “resolution.” Obviously, in the last 20 years, research in this area has been dominated by noninvasive functional brain imaging. The new techniques have increased the spatial and temporal resolutions with which we can investigate the neural foundations of language processing. Improved resolution of the experimental techniques has led to better quality neural data, generating a range of new insights and large-scale models (Hickok and Poeppel, 2004, 2007, see Fig. 1B; Hagoort, 2005; Ben Shalom and Poeppel, 2008; Kotz and Schwartze, 2010, Friederici, 2012; Price, 2012). However, it is fair to say that a different kind of resolution has played at least as critical a role, namely improved conceptual resolution. The increasingly tight connection of neuroscience research to linguistics, cognitive psychology, and computational modeling, forged over the last two decades, has placed the work on significantly richer theoretical footing, made many of the questions computationally explicit, and is beginning to yield hints of neurobiological mechanisms that form the basis of speech and language processing.

Regarding advances in methods, the remarkable improvements in recording techniques and data analyses have yielded new maps of the functional anatomy of language. Speaking spatially, local regions, processing streams, the two hemispheres, and distributed global networks are now implicated in language function in unanticipated ways. For example, the canonical language region, Broca's area, is now known, based on innovative cytoarchitectural and immunocytochemical data, to be composed of a number of subregions (on the order of 10, ignoring possible laminar specializations), plausibly implicating a much greater number of different functions than previously assumed (Amunts et al., 2010; Fig. 2), supporting both language and non-language processing. Moreover, there is emerging consensus that regions are organized into at least two (and likely more) processing streams, dorsal and ventral streams (Hickok and Poeppel, 2004; Saur et al., 2008), that underpin different processing subroutines, for example, mediating aspects of lexical recognition and lexical combination versus aspects of sensorimotor transformations for production, respectively (Hickok and Poeppel, 2007; Fig. 1B). Third, the classical issue of lateralization of function is now being investigated with respect to very specific physiological specializations, ranging from computational optimization for the encoding of sounds (Zatorre et al., 2002; Giraud et al., 2007) to the encoding of word meanings (Federmeier et al., 2008). Finally, network approaches to brain organization (Sporns, 2010) are beginning to inform how language processing is executed.

Figure 2.

The anatomic organization of Broca's region. From Amunts et al. (2010). With permission of the Public Library of Science.

Conceptual advances in linguistic and psychological research have defined what constitutes the “parts list” that comprises knowledge of language, its acquisition, and use. This description extends from important broader distinctions (e.g., phonology vs syntax) to subtle analyses of the types of representations and operations that underlie, say, meaning construction or lexical access. Early work on the neurobiology of language was largely disconnected from research on language and psychology per se and took language to be rather monolithic, with distinctions being made at the grain size of “perception” or “comprehension” versus “production.” (Imagine, for comparison, making generalizations about “vision” as an un-deconstructed area of inquiry.) In the subsequent phase (arguably the current wave of research) the experimental approaches focus on the decomposition of language in terms of subdomains. Typical studies attempt to relate brain structure and function to, say, phonology, syntax, or semantics. This represents significant progress, but the results are still of a correlational nature, with typical outcomes being of the type “area x supports syntax, area y supports semantics,” and so on. From a neurobiological perspective, the goal is to develop mechanistic (and ultimately explanatory) linking hypotheses that connect well defined linguistic primitives with equally well defined neurobiological mechanisms. It is this next phase of neurolinguistic research that is now beginning, developing a computational neurobiology of language, and there are grounds for optimism that genuine linking hypotheses between the neurosciences and the cognitive sciences are being crafted (Poeppel, 2012).

Is the classical model salvageable? Are the new approaches and insights merely expansions and adaptations to the traditional view? We submit that the model is incorrect along too many dimensions for new research to be considered mere updates. While these issues have been discussed in more detail, we point to just two serious shortcomings. First, due to its underspecification both biologically and linguistically, the classical model makes incorrect predictions about the constellation of deficit-lesion patterns (for discussion, see Dronkers et al., 2004; Hickok and Poeppel, 2004; Poeppel and Hickok, 2004). To exemplify, presenting with a Broca's area lesion does not necessarily coincide with Broca's aphasia, and analogously, presenting with Broca's aphasia does not mean having a Broca's area lesion. Current multiple stream models (that incorporate additional regions) fare better in characterizing the neuropsychological data. Second, the definition of and restriction to the few brain regions comprising the classical model dramatically underestimates the number and distribution of brain regions now known to play a critical role in language comprehension (Price, 2012) and production (Indefrey, 2011). Not just the role of the right (non-dominant) hemisphere has failed to be appreciated, but other left-lateralized extra-Sylvian regions such as the middle temporal gyrus or the anterior superior temporal lobe are now known to be essential (Narain et al., 2003; Turken and Dronkers, 2011), as is the involvement of subcortical areas (Kotz and Schwartze, 2010).

Many of the concepts (e.g., efference copy, oscillations, etc.) and methods (neuroanatomy, neurophysiology, neural coding) central to current research on cognitive neuroscience of language build on work in systems neuroscience. In light of the granularity of the linking hypotheses (Poeppel and Embick, 2005), connections to other areas of related research are more principled, allowing the investigation of mechanistic similarities and differences (say relating to work on birdsong and other animal vocalizations, on genetics, on neural circuits, on evolution, etc.). The level of analysis obviates the need for parochial discussions of what should or should not count as language and allows a sharper focus on underlying mechanisms. This review covers four areas of investigation in speech and language processing in which there has been considerable progress because of the closer connection between linguistics, psychology, and neuroscience: perception, production, sign language, meaning construction, providing a summary of research presented in a symposium at the Annual Meeting of the Society for Neuroscience.

Speech perception and cortical oscillations: emerging computational principles

The recognition of spoken language requires parsing more or less continuous input into discrete units that make contact with the stored information that underpins processing, informally, words. In addition to a parsing (or chunking) stage, there must be a decoding stage in which acoustic input is transformed into representations that underpin linguistic computation. Both functional anatomic and physiological studies have aimed to delineate the infrastructure of the language-ready brain. The functional anatomy of speech sound processing is comprised of a distributed cortical system that encompasses regions along at least two processing streams (Hickok and Poeppel, 2004, 2007). A ventral, temporal lobe pathway primarily mediates the mapping from sound input to meaning/words. A dorsal path incorporating parietal and frontal lobes enables the sensorimotor transformations that underlie mapping to output representations. New experiments aim to understand how the parsing and decoding processes are supported.

Speech (and other dynamically changing auditory signals, as well as naturalistic dynamic visual scenes) typically contain critical information required for successful decoding that is carried at multiple time scales (e.g., intonation-level information at the scale of 500–1000 ms, syllabic information closely correlated to the acoustic envelope of speech, ∼150–300 ms, and rapidly changing featural information, ∼20–80 ms). The different aspects of signals (slow and fast temporal modulation, frequency composition) must be processed for successful recognition. What kind of neuronal infrastructure forms the basis for the required multi-time resolution analyses? A series of neurophysiological experiments suggests that intrinsic neuronal oscillations at different, “privileged” frequencies (delta 1–3 Hz, theta 4–8 Hz, low gamma 30–50 Hz) may provide some of the underlying mechanisms (Ghitza, 2011; Giraud and Poeppel, 2012). In particular, to achieve parsing of a naturalistic input signal into manageable chunks, one macroscopic-level mechanism consists of the sliding and resetting of temporal windows, implemented as phase locking of low-frequency activity to the envelope of speech and resetting of intrinsic oscillations on privileged time scales (Luo and Poeppel, 2007; Luo et al., 2010). The successful resetting of neuronal activity provides time constants (or temporal integration windows) for parsing and decoding speech signals (Luo and Poeppel, 2012). These studies link very directly the infrastructure provided by well described neural oscillations to principled perceptual challenges in speech recognition. One emerging generalization is that acoustic signals must contain some type of “edge,” i.e., an acoustic discontinuity that the listener can use to chunk the signal at the appropriate temporal granularity. Although the “age of the edge” is over and done for vision, acoustic edges likely play an important (and slightly different) causal role in the successful perceptual analysis of complex auditory signals.

Computational neuroanatomy of speech production

Most research on speech production has been conducted from within two different traditions: a psycholinguistic tradition that seeks generalizations at the level of phonemes, morphemes, and phrasal-level units (Dell, 1986; Levelt et al., 1999), and a motor control/neural systems tradition that is more concerned with kinematic forces, movement trajectories, and feedback control (Guenther et al., 1998; Houde and Jordan, 1998). Despite their common goal, to understand how speech is produced, little interaction occurred between these areas of research. There is a standard position regarding the disconnect: the two approaches are focused on different levels of the speech production problem, with the psycholinguists working at a more abstract, perhaps even amodal, level of analysis, and the motor control/neuroscientists largely examining lower-level articulatory control processes. However, closer examination reveals a substantial convergence of ideas, suggesting that the chasm between traditions is more apparent than real and, more importantly, that both approaches have much to gain by paying attention to each other and working toward integration.

For example, psycholinguistic research has documented the existence of a hierarchically organized speech production system in which planning units ranging from articulatory features to words, intonational contours, and even phrases are used. Motor control approaches, on the other hand, have emphasized the role of efference signals from motor commands and internal forward models in motor learning and control (Wolpert et al., 1995; Kawato, 1999; Shadmehr and Krakauer, 2008). An integration of these (and other) notions from the two traditions has spawned a hierarchical feedback control model of speech production (Hickok, 2012).

The architecture of the model is derived from state feedback models of motor control but incorporates processing levels that have been identified in psycholinguistic research. This basic architecture includes a motor controller that generates forward sensory predictions. Communication between the sensory and motor systems is achieved by an auditory–motor translation system. The model includes two hierarchical levels of feedback control, each with its own internal and external sensory feedback loops. As in psycholinguistic models, the input to the model starts with the activation of a conceptual representation that in turn excites a corresponding word representation. The word level projects in parallel to sensory and motor sides of the highest, fully cortical level of feedback control, the auditory–Spt–BA44 loop (Spt stands for Sylvian fissure at the parieto-temporal boundary, BA44 is Brodmann area 44). This higher-level loop in turn projects, also in parallel, to the lower-level somatosensory–cerebellum–motor cortex loop. The model differs both from standard state feedback control and psycholinguistic models in two main respects. First, “phonological” processing is distributed over two hierarchically organized levels, implicating a higher-level cortical auditory–motor circuit and a lower-level somatosensory–motor circuit, which roughly map onto syllabic and phonemic levels of analysis, respectively. Second, a true efference copy signal is not a component of the model. Instead, the function served by an efference copy is integrated into the motor planning process. The anatomical correlates of these circuits (Fig. 3) have been derived from a range of functional imaging and neuropsychological studies, and some of the computational assumptions of the model have been demonstrated in computer simulations.

Figure 3.

Computational and functional anatomic analysis of speech production. From Hickok (2012). With permission from Nature Publishing Group.

Modality dependence and independence: the perspective of sign languages

The study of sign languages has provided a powerful tool for investigating the neurobiology of human language (for review, see Emmorey, 2002; MacSweeney et al., 2008; Emmorey and McCullough, 2009). Signed languages differ dramatically from spoken languages with respect to the linguistic articulators (the hands vs the vocal tract) and the perceptual system required for comprehension (vision vs audition). Despite these biological differences, research over the past 35 years has identified striking parallels between signed and spoken languages, including a level of form-based (phonological) structure, left-hemisphere lateralization for many language-processing subroutines, and similar neural substrates, e.g., a frontotemporal neural circuit for language production and comprehension (Hickok et al., 2001; Sandler and Lillo-Martin, 2006). These similarities provide a strong basis for comparison and serve to highlight universal properties of human language. In addition, differences between signed and spoken languages can be exploited to discover how specific vocal-aural or visual-manual properties impact the neurocognitive underpinnings of language. For example, the visual-manual modality allows for iconic expression of a wide range of basic conceptual structures, such as human actions, movements, locations, and shapes (Taub, 2001). In American Sign Language (ASL) and many other sign languages, the verbs HAMMER, DRINK, and STIR resemble the actions they denote. In addition, the location and movement of objects are expressed by the location and movement of the hands in signing space, rather than by a preposition or locative affix. These differences between spoken and signed languages arise from differences in the biology of linguistic expression. For spoken languages, neither sound nor movements of the tongue can easily be used to create motivated representations of human actions or of object locations/movements, whereas for signed languages, the movement and configuration of the hands are observable and can be readily used to create non-arbitrary, iconic linguistic representations of human actions and spatial information.

To exemplify some of the empirical work, H₂ ¹⁵O-PET studies have explored whether these biological differences have an impact on the neural substrates that support action verb production or spatial language. In Study 1, deaf native ASL signers and hearing non-signers generated action pantomimes or ASL verbs in response to pictures of tools and manipulable objects. The generation of ASL verbs engaged left inferior frontal cortex, but when non-signers produced pantomimes in response to the same objects, no frontal activation was observed (Fig. 4A). Both groups recruited left parietal cortex during pantomime production. Overall, these results indicate that the production of pantomimes versus ASL verbs (even those that resemble pantomime) engage partially segregated neural systems that support praxic versus linguistic functions (Emmorey et al., 2011). In Study 2, deaf ASL signers performed a picture description task in which they overtly named objects or produced spatial classifier constructions in which the hand shape indicates object type [e.g., long-thin object (Fig. 4C), vehicle] and the location/motion of the hand iconically and gradiently depicts the location/motion of a referent object (Emmorey and Herzig, 2003). In contrast to the expression of location and motion, the production of both lexical signs and object type morphemes engaged left inferior frontal cortex and left inferior temporal cortex, supporting the hypothesis that classifier hand shapes are categorical morphemes that are retrieved via left hemisphere language regions. In addition, lexical signs engaged the anterior temporal lobes to a greater extent than classifier constructions (Fig. 4D), reflecting the increased semantic processing required to name individual objects, compared to simply indicating the type of object. Both location and motion components of classifier constructions engaged bilateral superior parietal cortex (Fig. 4E), but not left frontotemporal cortices, suggesting that these components are not lexical morphemes. Parietal activation reflects the different biological basis of sign language. That is, to express spatial information, signers must transform visual-spatial representations into a body-centered reference frame and reach toward target locations within signing space, functions associated with superior parietal cortex. In sum, the biologically based iconicity found in signed language alters the neural substrate for spatial language but not for lexical verbs expressing handling actions.

Figure 4.

A, Generation of “pantomimic” signs by deaf signers engaged left IFG (inferior frontal gyrus), but production of similar pantomimes by hearing non-signers did not. D, Lexical “pantomimic” signs shown in B activate the anterior temporal lobes, in contrast to less specific classifier constructions shown in C. E, Location and motion classifier constructions activate bilateral parietal cortex, in contrast to lexical signs.

The observation that spoken and signed languages are subserved by (many of) the same neural structures has been one of the foundational findings of the cognitive neuroscience of language and has cemented the insight that knowledge of language reflects a cognitive system of a certain type, despite differences in modalities and input/output systems. The data suggest that there are areas that are selective for language tasks regardless of modality, and as such “universal.” That being said, there exists considerable evidence for experience-dependent plasticity in these (and related) circuits, as well. For example, both the demands imposed by the properties of particular languages (Paulesu et al., 2000) and the ability to read or not (Castro-Caldas et al., 1998) can shape the organization of these brain regions. Both stability (e.g., across modalities) and plasticity (across speaker-dependent experience) are hallmarks of the neurocognitive system.

Building meanings: the computations of the composing brain

Whether as speech, sign, text, or Braille, the essence of human language is its unbounded combinatory potential: Generative systems of syntax and semantics allow for the composition of an infinite range of expressions from a limited set of elementary building blocks. The construction of complex meaning is not simple string building; rather, the combinatory operations of language are complex and full of mysteries that merit systematic investigation. Why, for example, does every English speaker have the robust intuition that “piling the cushions high” results in a high pile and not high cushions, whereas “hammering the ring flat” gives you a flat ring instead of a flat hammer? Questions such as these are answered by research in theoretical syntax and semantics, a branch of linguistics that offers a cognitive model of the representations and computations that derive complex linguistic meanings. So far, brain research on language (at least semantics) has remained relatively disconnected from this rich body of work. As a result, our understanding of the neurobiology of combination of words and composition of meanings is still grossly generic: neuroscience research on syntax and semantics implicates a general network of “sentence processing regions” but the computational details of this system have not been uncovered.

Recent research using magnetoencephalographic recordings aims to bridge this gap: guided by results in theoretical syntax and semantics, these studies systematically varied the properties of composition to investigate the detailed computational roles and spatiotemporal dynamics of the various brain regions participating in the construction of complex meaning (Pylkkänen et al., 2011). The combinatory network implicated by this research comprises at least the left anterior temporal lobe (LATL), the ventromedial prefrontal cortex (vmPFC), and the angular gyrus (AG). Of these regions, the LATL operates early (∼200–300 ms) and appears specialized to the combination of predicates with other predicates to derive more complex predicates (as in red boat) (Bemis and Pylkkänen, 2011, 2012), but not predicates with their arguments (as in eats meat). The roles of the AG and especially the vmPFC are more general and later in time (∼400 ms). Effects in the vmPFC, in particular, generalize across different combinatory rule types and even into the processing of pictures, suggesting a multipurpose combinatory mechanism. However, this mechanism appears specifically semantic, as opposed to syntactic, since vmPFC effects are systematically elicited for expressions that involve covert, syntactically unexpressed components of meaning (Pylkkänen and McElree, 2007; Brennan and Pylkkänen, 2008, 2010). In sum, contrary to hypotheses that treat natural language composition as monolithic and localized to a single region (Hagoort, 2005), the picture emerging from these experiments suggests that composition is achieved by a network of regions which vary in their computational specificity and domain generality. By connecting the brain science of language to formal models of linguistic representation, the work decomposes the various computations that underlie the brain's multifaceted combinatory capacity.

Summary

Research on the biological foundations of language is at an exciting moment of paradigmatic change. The linking hypotheses are informed by computational analyses, both of the cognitive functions and of the neuronal circuitry. The four presentations in this symposium highlight the new directions that are recasting how we study the brain basis of language processing. Spanning four domains, speech perception, speech production, the combination of linguistic units to generate meaning, and processing sign language, the presentations outline both major empirical developments and show new methodologies. Concepts familiar from systems neuroscience now play a central role in the neurobiology of language, pointing to important synergistic opportunities across the neurosciences. In the new neurobiology of language, the field is moving from coarse characterizations of language and largely correlational insights to fine-grained cognitive analyses aiming to identify explanatory mechanisms.