Neurobiological roots of language in primate audition: common computational properties

Abstract

This paper presents a new perspective on an old question: how does the neurobiology of human language relate to brain systems in nonhuman primates? We argue that higher-order language combinatorics – including sentence and discourse processing – can be situated in a unified, cross-species dorsal-ventral streams architecture for higher auditory processing, and that the functions of the dorsal and ventral streams in higher-order language processing can be grounded in their respective computational properties in primate audition. This view challenges an assumption, common in the cognitive sciences, that a nonhuman primate model forms an inherently inadequate basis for modeling higher-level language functions.

How does the brain implement language?

Neurobiologically plausible models of human brain function are typically based on detailed animal models. However, while the applicability of this modeling strategy is widely accepted for domains such as vision or audition, its transferability to human language is considerably more controversial. The reason for this perspective – particularly at the level of sentences and above – relates to complex computational properties of human grammars and their purported specificity to our species [1,2].

With respect to neurobiological models of speech and language, these considerations have led to an interesting dualism. It is generally accepted that human speech and language processing is supported by a cortical dorsal-ventral-streams architecture that shares many anatomical characteristics with the extended auditory system of nonhuman primates (e.g. [3–8]). This architecture involves a division of labor between two cortical streams of information transfer from auditory cortex (AC) to prefrontal regions. As shown in more detail in Figure 1, the postero-dorsal stream connects AC to the posterior and dorsal part of inferior frontal cortex (IFC) (Brodmann area [BA] 44) via posterior superior temporal (pST) cortex, inferior parietal lobule (IPL), and premotor cortex (PMC); the antero-ventral stream, by contrast, traverses anterior superior temporal cortex (aST) to terminate in more anterior and ventral parts of the inferior frontal gyrus (BA 45). Importantly, most models in this domain have focused primarily on speech and word processing, rather than on the complex combinatorial properties of language claimed to be unique to humans. The few available dual–stream models of sentence processing, by contrast, typically assume that the neural circuitry of nonhuman primates is insufficient to support sentence comprehension because of a fundamental difference in its computational architecture that is not simply a matter of degree (e.g. [8]). They thus posit uniquely human additions to this circuitry in the dorsal stream, which are assumed to have evolved late from a phylogenetic perspective and to mature late from an ontogenetic perspective [9]. Hence, in spite of the broad consensus regarding the anatomical overlap between the primate auditory system and the cortical speech and language architecture, it is typically assumed that the nonhuman primate system is neither quantitatively nor qualitatively sufficient to support the computational needs of higher-order language (i.e., sentence and discourse) processing.

Figure 1. Dual streams supporting language processing in the human brain.

Multiple streams of information processing underlie the human ability for speech and language. Neuroanatomically, two streams, a (postero-) dorsal (red) and an (antero-)ventral (green) stream are particularly important, and show clear homologies to the nonhuman primate brain and specifically to its auditory cortical system with its extensions into parietal, premotor and prefrontal cortex. The dorsal stream, in particular, serves to connect sensory (auditory) and motor-related regions of the cerebral cortex and thus implements bidirectional perception-action loops. Cross-stream integration in the inferior frontal gyrus is depicted by the transition from red to green. Black dashed arrows denote feedback connections.

Figure adapted from [4].

Abbreviations: AC, auditory cortex; CS, central sulcus; IFC, inferior frontal cortex; IPL, inferior parietal lobule; PMC, premotor cortex. Numbers denote Brodmann areas.

In addition, recent research has even questioned the necessity of a neural architecture akin to that of the primate auditory system for the computational mechanisms underlying higher-order language. As nonhuman primates are generally considered to not be complex vocal learners, there has been an increased interest in alternative animal models, focusing on species that do show vocal learning abilities. In this context, songbirds have played a dominant role, based on the shared ability for complex sequence processing in avians and humans (e.g., [10,11]). Thus, by shifting the focus onto evolutionary convergence as opposed to common descent, birdsong models have further perpetuated the move away from a nonhuman primate model for the neurobiology of higher-order language [2,10] – the importance of such a model for basic aspects of speech, and possibly word-level processing notwithstanding. (For approaches advocating the comparison of multiple nonhuman animal models, see e.g. [12,13].)

Here, we argue that the tendency to abandon the nonhuman primate auditory system as a suitable animal model for the neurobiology of higher-order language may be premature. (For a similar, recent argument regarding the evolution of speech, see [14].) To the contrary, we suggest that, when the computational requirements for sentence and discourse processing are broken down into more basic mechanistic components, there is indeed quite compelling evidence to suggest that the computational architecture of the nonhuman primate dorsal and ventral auditory streams is qualitatively sufficient for performing the requisite computations. In other words, the basic computational building blocks necessary for language processing are already in place in the nonhuman primate, though the system lacks the necessary quantitative scale to support language. We also offer some suggestions as to why the primate auditory system may in fact even fulfill the stronger criterion of necessity, based on the notion of cross-stream integration and the role of prefrontal cortex.

Basic computational properties underlying higher-order language combinatorics

One of the hallmarks of higher-order language (sentences and discourse) in humans is its combinatorial flexibility, i.e., the ability to combine smaller units into larger units in order to express a wide range of meanings. Two basic combinatorial mechanisms are generally agreed upon in linguistic theory: (i) the combination of elements into sequences (i.e., combining elements A and B to form the sequence A-before-B); (ii) the combination of elements to form dependencies, independent of sequential order. To illustrate the difference between the two mechanisms, consider the phrase "the red boat" and its French counterpart "le bateau rouge" (literally: 'the boat red'). In both cases, red describes a property of the boat (i.e., there is a dependency between boat and red), but the sequential order in which the two words are expressed differs between the two languages. How strongly dependencies rely on particular sequential orders also differs across the languages of the world. While in English, sequential order is the primary cue for extracting dependencies (e.g., "The boy kissed the girl" can only mean that the boy was the kisser and the girl the person being kissed), most other languages show more flexibility in this regard (e.g., German, where "Den Jungen küsste das Mädchen", 'The boy.ACCUSATIVE kissed the girl.NOMINATIVE', means that the girl kissed the boy; with dependencies indicated here by the accusative and nominative case marking). As is apparent from these simple examples, sequencing and dependency formation are basic – and separable – properties of human language, and all theories of grammar assume these two computational mechanisms in one form or another (e.g., [15–18]).

We propose here that sequence-based (order-sensitive) and dependency-based (order-insensitive) combinatorics are supported by the dorsal and ventral cortical streams, respectively. Box 1 discusses this assumption in more detail in the context of neurobiologically motivated design principles. In the following sections, we review evidence for this claim from language studies and demonstrate that these basic computational functions are already in place in the dual auditory streams of nonhuman primates.

Box 1. Neurobiological design principles and the computational division of labor between the antero-ventral and postero-dorsal streams.

The assumption that the auditory cortical system of (nonhuman) primates (with its extensions into parietal, premotor and prefrontal cortices) provides a suitable animal model for the neurobiology of human language allows us to adopt several neurobiological design principles that have been tested empirically within the animal (primarily rhesus macaque) model. The two most important of these are:

• Hierarchical organization of the processing streams. Hierarchical organization implies increasing combination sensitivity to individual stimulus features with increasing distance from primary sensory cortices (here: from auditory cortex, AC). Single-unit studies in rhesus monkeys have demonstrated hierarchical organization of the ventral auditory stream [72,96], with neurons in primary auditory cortex A1 and in the middle lateral belt region of AC processing elementary auditory features (e.g., frequency-modulated (FM) sweeps or bandpass noise bursts) and neurons in more anterior portions of the lateral belt responding to more complex spectro-temporal patterns (e.g. species-specific vocalizations, monkey calls).

• Separable but internally unified computational properties for each stream. The traditional subdivision of dual (visual or auditory) streams into a "what" and a "where" (or "how") stream suggests a computational division of labor between streams with a common computational denominator within each stream [20,35,36]. We adopt this assumption and posit that the major dorsal-ventral distinction is functionally relevant irrespective of the possible presence of anatomical sub-pathways (e.g. multiple white-matter tracts within one stream).

The functional division of labor between the antero-ventral and postero-dorsal streams envisaged here is illustrated in Figure I. The antero-ventral stream (Figure Ia) recognizes successively more complex auditory objects. As described in the main text for linguistic dependency formation (i.e. the establishment of order-independent interpretive relations between elements), feature combination in more complex auditory objects is assumed to be commutative (i.e., order insensitive). Order insensitivity is a crucial property of conceptual representations, since these can be identified via varying sensory stimuli (auditory, visual, olfactory, etc.), and the combination of individual features to form a concept is not dependent on a particular order of features in the input (e.g. has-four-legs, has-a-tail, barks as features of the concept “dog” are not ordered with respect to one another). The postero-dorsal stream (Figure Ib), by contrast, performs predictive sequence processing via hierarchically organized internal models. Prediction at one hierarchical level is assumed to influence predictive processing at the next-lower level via feedback connections, eventually leading to a comparison of predicted sensory information with the actual sensory input. Error signals resulting from this comparison are transmitted upwards in the hierarchy via feedforward connections [46,52].

Antero-ventral stream: Computation of successively more complex auditory objects

In nonhuman primates, the ventral stream subserves the recognition of successively more complex auditory objects, ranging from elementary auditory features (e.g., frequency-modulated (FM) sweeps or bandpass noise bursts) to species-specific vocalizations (monkey calls) [4,19,20]. Auditory object formation is a form of categorization, in which spectro-temporal properties are grouped into perceptual [21] and, at higher hierarchical levels, conceptual units. It thus provides the computational basis for an elementary mapping from spectro-temporal patterns to concepts.

Language also involves the identification of auditory objects (e.g., phonemes, syllables, words, phrases), with evidence for a similar hierarchical organization of auditory object recognition along the antero-ventral stream [22]. This involves the mapping from spectro-temporal patterns to sensory-independent categories (e.g., [23] for syllables) and time-invariant semantic structures (concepts) [5–7]. Notably, time-invariance constitutes a crucial property of conceptual representations (see Box 1). Further evidence to implicate the ventral stream in conceptual processing in humans stems from the deficits shown by patients with Semantic Variant Primary Progressive Aphasia (semantic PPA; also known as semantic dementia, though the two terms are not completely synonymous [24]). This condition is characterized by atrophy of the anterior temporal lobes (e.g., [25,26]). Accordingly, many researchers view portions of the anterior temporal lobe as a hub of semantic processing, binding together perceptually-based semantic representations into coherent concepts (but see [27,28]). Converging evidence for this view stems from various experimental approaches, including neuroimaging [7,29], transcranial magnetic stimulation [30,31], and computational modeling [6].

Although the involvement of the ventral stream in mapping auditory input onto conceptual representations is widely accepted, it has been considerably less clear to date how this perspective on ventral stream function might be integrated with the results of a separate line of research, which suggests that the anterior temporal lobe also contributes to combinatorial processing [22,32,33], an assumption that features in several dual-stream models of speech and language [5,8].

We propose (see also [34]) that these two functions of the ventral stream – auditory-conceptual mappings and combinatorial processing – can be subsumed under a single mechanism, which is moreover motivated by independent assumptions regarding the recognition of auditory objects and hierarchical processing (see Box 1). Auditory objects can be rendered more complex (feature-rich) by combining an increasing number of attributes. An important characteristic of this combinatorial property is its commutativity, i.e., order is not important (AB = BA). In other words, adding information leads to an updating of existing auditory objects via the formation of dependencies rather than the formation of a sequence of auditory objects, as in the dorsal stream (see below). For example, the auditory object "boat" is modified to include an additional property (that it is red) via processing of the word "red", irrespective of whether "red" occurs before "boat" or after. Like auditory objects, dependencies are formed based on grouping cues (e.g., case marking, likelihood of co-occurrence).

Postero-dorsal stream: Sequential processing via an internal model

In keeping with its visual counterpart, the postero-dorsal auditory stream was originally characterized as a “where” [35] or "how" [36] stream [3,20]. Besides processing auditory space and motion, it subserves auditory-motor mappings [4,37,38], including the processing of sound sequences (e.g., [39–41]) and speech rhythm [42]. The computational machinery common to these functions can be characterized in terms of an internal (forward) model [19], which serves to predict upcoming sensory events within a sequence on the basis of the previous input [43–48].

For sentence comprehension, there is substantial evidence linking the dorsal stream to the processing of word order (i.e., sentence-level sequencing). A number of fMRI studies comparing sentences with object-before-subject and subject-before-object orders have reported activation in regions associated with the dorsal stream, including pST, IPL, and PMC regions, as well as the IFG [49–51]. Crucially, findings such as these do not implicate the dorsal stream as a syntax stream, since the activations in question also rely on non-syntactic (e.g., semantic) parameters ([50,51]).

As proposed for the primate model and for speech processing [4,19], the implementation of a forward model provides a unified functional explanation for the dorsal-stream sequence processing capability at different linguistic levels. To explain the interplay among the various, hierarchically organized levels of sequences (discourse is composed of sentences which are made up of words which comprise sounds), a hierarchically organized set of forward models is required, each yielding predictions for the next level down (see Box 1) [46,52]. Similar claims have been made in the domains of action control, which shows a similar hierarchical organization [53,54], and of vision [55].

Evidence for the computational division of labor between streams

Experimental evidence from a variety of different methods provides direct support for the computational division of labor between streams as proposed here. One study used fMRI to demonstrate a parametric increase in activation of both anterior (ventral stream) and posterior (dorsal stream) temporal lobes, the latter in the posterior superior temporal sulcus (pSTS), in response to linguistic constituents of increasing size [56]. However, the increase proved to be dependent on the use of real words in the anterior but not posterior temporal lobe. Posteriorly, the pSTS showed a similar parametric activation increase for sequences of real words and phonologically legal nonwords with grammatical features preserved (i.e., pseudowords). This attests to the importance of conceptual combinatorics for the ventral stream, while the dorsal stream responds to more abstract sequencing demands.

Neuroimaging studies with PPA patients confirm this assumption. Syntactic deficits in PPA patients – evidenced by a reduced ability to comprehend sentences not adhering to the typical (canonical) Agent-Action-Object sequence for English [57] – correlate with damage to dorsal, but not ventral white-matter tracts [58]. Conversely, semantic PPA patients with atrophy of the anterior temporal lobe show comparable dorsal stream fMRI activation to control subjects for non-canonical sentences [59]. This finding suggests that the known increase in activation of the anterior temporal lobe for sentences versus word lists (see [32] for a review) should be attributed to a different combinatorial mechanism than that engendering dorsal-stream activation for non-canonical sentences, i.e., building commutative (order-insensitive) dependencies rather than processing sequences.

Cross-stream integration and prefrontal cortex

In the model proposed here, prefrontal cortex is viewed as a controller of information flow along the two streams [60], subserving cross-stream integration and mediating top-down feedback from one stream to the other [61,62]. From this perspective, integration can be envisaged as a recoding of information originating in one stream into a signal for top-down modulation of the other (see also [4]). (Note, however, that in addition to cross-stream integration via feedback loops, there is some evidence for interactive cross-talk between streams [61] and for subcortical contributions – particularly striatal – to the neural language architecture [63]).

In contrast to several other current neurobiological models of language [8,64], our proposal does not imply a specific involvement of prefrontal cortex – and particularly the IFG – in language processing per se (e.g., in accomplishing unification or syntactic processing). Rather, it subscribes to the view that the prefrontal cortex performs domain-general regulatory functions that play a nonspecific but crucial role in language processing [65–67]. (For a review of recent evidence from the language domain, see [34]). Increased regulation (or cognitive control) becomes necessary when there is a need for top-down biases to mediate among conflicting representations or when the recoding of information for cross-stream interaction leads to an incompatibility with the current state of the stream on which the top-down influence is exerted. A prediction following from this assumption is that the prefrontal cortex should be capable of up- or down-regulating activity within a particular stream in accordance with current processing demands.

On sufficiency: Differences between nonhuman primates and humans

So far, the focus has been on commonalities between humans and nonhuman primates regarding the computational properties of the dorsal and ventral auditory streams. Nevertheless, nonhuman primates obviously do not have language – at least not in a comparable way to humans – and they do not appear to be able to acquire human language even when taught in gestural form [68]. How can a proposal positing only quantitative neurocomputational differences between humans and nonhuman primates – and, hence, a qualitative sufficiency for higher-order language processing of the computational mechanisms supported by the auditory system of nonhuman primates– account for such differences? Importantly, in keeping with the approach of examining the basic computational building blocks underlying higher-order language, we focus on the capacity of nonhuman primates for information processing rather than on their production abilities, which may depend on constraints of a fundamentally different nature (see [69]; for a similar argument, see [12]).

With regard to the ventral stream, the potential complexity (feature-richness) of auditory objects in humans is undoubtedly higher than in nonhuman primates. This may, in part, be due to the possibility of auditory object combination (or unification in linguistic theory), as outlined above. Nevertheless, whether the capacity for auditory object combination is unique to humans or whether nonhuman primates also show certain limited capacities in this regard remains to be investigated. Certainly, the ability to combine features at the cellular level (combination sensitivity) is well documented for early stages of auditory cortex in nonhuman primates as well as other animals [70,71], rendering neurons selective, for example, to monkey calls as opposed to more elemental features such as FM sweeps or band-passed noise bursts [72]. It is not clear to date whether this also applies to higher levels of processing.

However, it is the dorsal stream that has been primarily implicated in the unique computational capacities assumed to support human language [2,8,9,73]. In contrast to these approaches, we do not subscribe to the notion (see Box 2) that, beyond basic sequence processing, a more elaborate and qualitatively distinct computational mechanism (i.e., discrete infinity produced by recursion) is required for human language. We assume instead, as proposed in [34], that the ability to combine two elements A and B in an order-sensitive manner to yield the sequence AB forms the computational basis for the processing capacity of the postero-dorsal stream in human language. Indeed, nonhuman primates show at least a very basic capacity for this type of simple sequence processing, e.g., chimpanzees can learn to produce ordered 2-word sequences [68] or imitate sequentially structured actions [74]. Nonhuman primates are also proficient at transitive inference (TI), i.e., they can correctly infer from the sequential presentation of paired stimuli that, if A precedes B and B precedes C, A also precedes C. In TI tasks, rhesus monkeys show similar performance in terms of error rates and reaction times to 4- and 6-year-old children [75,76]. Findings such as these, as well as results on numerical reasoning [77], have been used to argue for a spatial representation of serial order information [78], a suggestion that resonates with the function of the dorsal stream in both spatial and sequence processing.

Box 2. Proposed computational differences between humans and nonhuman primates reconsidered.

Behavioral differences between humans and nonhuman primates have played a major role in shaping the debate regarding a suitable animal model for the neurobiology of language. Upon closer consideration, however, some of the key differences that have been posited in this regard may not be as substantial as typically assumed.

Ventral stream

With regard to auditory object perception and production, language-learning experiments with chimpanzees have led some researchers to conclude that “chimps do not really have ‘names for things’ at all” on the basis of the fact that they “will use the same label apple to refer to the action of eating apples, the location where apples are kept […]” [97]. This property (“transcategoriality”), however, is also true of a number of human languages, including Classical Chinese [98], Riau Indonesian [99] and other Austronesian languages. Possible auditory objects and their functions must thus be considered in the face of the full range of human languages (~ 7000).

Dorsal stream

Neuroscientists interested in the neurobiology of complex language structures often base their line of enquiry on one specific, assumed aspect of language competence: the capacity for discrete infinity [1]. We argue that this notion is based on a logical calculus that abstractly characterizes human grammars, rather than on empirically substantiated, biologically-motivated hypotheses (see also [11]). Humans themselves are actually quite severely limited in the number of center embeddings that they can process, cf. the difficulty arising in "If if the cat is in, then the dog cannot come in, then the cat and dog dislike each other" [100]. Note that this sentence is analogous to the AABB structure that is commonly used in experiments on the neural correlates of recursive (“human”) grammars (e.g. [73]). Once actual human performance is considered the benchmark, these can be processed by considerably simpler computational mechanisms than those posited as necessary in the majority of neuroscientific examinations of human vs. nonhuman sequence processing abilities [100]. Hence, if human abilities are measured in the same way as those of nonhuman primates or birds necessarily must be, i.e. by means of behavioral performance, then cross-species differences can be described almost entirely in quantitative rather than qualitative terms. For converging comparative evidence and the conclusion that syntactic structure as present in human language may in fact be more widespread throughout animal communication systems than phonological structure, see [101].

Furthermore, neuroanatomical tracer studies in monkeys provide evidence for direct connections from posterior-dorsal auditory regions to parietal and prefrontal regions [79–81], similar to those in humans [82], which can form the substrate for auditory-motor coupling required for these forms of neurocomputation. These existing structures clearly enable demonstrated audio-motor behavior in monkeys [83,84]. Thus, while monkeys may not possess the level of sophistication in this domain enabling them to have speech and complex vocal learning, they certainly have the machinery necessary to link perception and action. Such sensorimotor control loops, commonly referred to as internal models, provide the basis for sound sequences being matched to action sequences and we posit here that the basic mechanism of sequence processing, which we claim is subserved by the dorsal stream, relies on the computations afforded by an internal model (see Box 1). Crucially, our focus is on sequencing as a basic computational mechanism in higher-order language rather than in speech processing. In other words, we are not claiming that monkeys have either articulatory or acoustic phonetics, but that they have brain structures that could support – given other factors – the essential structures of higher-order language. While the lack of a human-like articulatory apparatus could lead to evolutionary structural differences in the brain mechanisms for phonetics, we are not aware of any evidence that such differences need necessarily impact higher order combinatorial processing.

We posit that the difference in sequence processing capacity between humans and nonhuman primates – and hence the considerably more complex structure of human language – can be traced back to the hierarchical system of forward models implemented by the human dorsal stream. By contrast, it appears likely that the complexity of the forward model implemented by the dorsal stream of nonhuman primates is hierarchically limited (e.g., by a smaller number of specialized cortical areas [85], smaller cortical volume [86], or smaller number of connecting axons), thus accounting for the limitations shown by nonhuman primates in the learning of hierarchically structured sequential action sequences (i.e., the reduced hierarchical “depth” shown in comparison to humans [87]).

Thus, it appears that the limitations of nonhuman primate vocal productions and their vocal learning abilities may not be key to understanding the possible neurobiological roots of higher-order language in the nonhuman primate brain. In further support of this assumption, a separate line of research has identified possible precursors to human speech and language in the rhythm of nonhuman primate oro-facial movements [14,88]. Recent studies in rhesus macaques suggest that lip smacking behavior may be particularly relevant in this regard, as it shows similar rhythmic properties to human speech and language, namely a rhythmical structure in the 3–8 Hz range [89,90]. It further shares a range of additional characteristics with speech, including similar developmental trajectories of rhythmical structure in production as well as perceptual tuning to the preferred frequency range in perception ([91], for a review). Finally, recent findings show that new-world monkeys (marmosets) engage in turn-taking during their vocal communicative behaviour and, in doing so, manifest multiple signs of cooperative coordination, including waiting before responding to prevent overlapping calls, call coupling and reciprocal entrainment of calls between monkeys [92]. This suggests that some of the fundamental prerequisites for discourse communication are already in place in nonhuman primates [14] and, in this particular case, even in a species that is on a different branch of the evolutionary tree to humans.

In summary, there is substantial evidence to support the assumption that nonhuman primates possess at least rudimentary versions of the fundamental computational mechanisms underlying human language: auditory object recognition and combination in the ventral stream (including a mapping from acoustic input to conceptual representations) and basic sequence processing in the dorsal stream. This suggests that a nonhuman primate (e.g., rhesus macaque) model may indeed constitute a suitable animal model for the neurobiology of higher-order human language processing.

On necessity: Interacting streams as the basis for all aspects of human language

Assuming that our argument for sufficiency as advanced in the previous section can be upheld, the rapidly increasing literature on parallels between birdsong and human language begs another, perhaps even more difficult question: Does the auditory dual-streams architecture of the nonhuman primate model constitute a necessary prerequisite for the emergence of language? Obviously, a definitive answer to this question will remain outstanding unless a species is discovered that has a human-like language system with a distinct neural architecture to support it. Nevertheless, we propose that the dual-streams architecture of the nonhuman primate auditory system does offer some unique advantages that render its necessity for language at least a possibility. Specifically, we assume that the role of prefrontal cortex in information integration between the dorsal and ventral streams may be of crucial importance in this regard.

We have suggested above that cross-stream interaction is crucial for language and assumed that it is mainly implemented as a transformation of information from one stream to produce a top-down influence on the other, with prefrontal cortex playing a crucial role in this transformation. In this view, the rich expressive power of language is grounded in a neural architecture that accomplishes the integration of auditory object recognition and combination with sequential prediction, and that the prefrontal cortex mediates this ventral/dorsal integration. (For a somewhat similar perspective, though we do not subscribe to the specific linguistic assumptions of this proposal, see [93]).

Thus, perhaps the most promising explanation for the striking differences between humans and nonhuman primates is not within a single stream, but in cross-stream interaction. Anatomical differences in prefrontal cortex between nonhuman primates and humans (e.g., [94]) can be expected to play a crucial role in resolving the question of why humans have language and nonhuman primates do not. While we assume that the relevant computational neurobiological functions of the dorsal and ventral streams are already present in nonhuman primates – at least to some degree – the complexity of interactive computation across the two streams may be what yields the vast communicative power of language. Selected predictions following from this approach and outstanding questions are outlined in Box 3 and Box 4, respectively.

Box 3. Selected predictions arising from the model put forward here.

• Languages with only a limited (or even no) capacity for recursion. The assumption that computational differences between audition in nonhuman primates and language in humans are quantitative rather than qualitative leads to the prediction that, though recursion is a property of most human languages, a small number of languages at the outer edge of the distribution may show only a limited capacity for recursion. This prediction indeed appears to be borne out, e.g. in the much discussed case of the Pirahã language spoken in the Amazon region of Brazil ([102]; but see e.g. [103]).

• Neurobiological differences between sequence-based and non-sequence-based language processing. The computational functions of the dorsal and ventral streams posited here predict that the default division of labor between streams may differ depending on language-specific characteristics. For example, as already noted in the main text, it is well established that some languages are more sequence-based than others (e.g. [104]). Accordingly, the model put forward here predicts that strongly sequence-based (e.g. English) versus weakly sequence-based languages (e.g. German, Turkish) may rely on a different organization of the underlying neurobiological processing architecture. Initial evidence from the electrophysiological domain suggests that this prediction is borne out [105,106].

• Extension to reading. We claim here that the neurobiological basis of language is rooted in primate higher audition and the computational division of labor between the two auditory processing streams. By contrast, there is no evidence for a biological basis of reading or writing in any other species. This leads to the following two predictions. Firstly, reading will likely rely on neural mechanisms (dorsal and ventral) that largely overlap with (or extend from) those described here for (auditory) language processing. (Note that all current neurobiological models of reading focus on the word level. Thus, perhaps surprisingly, this claim is largely untested for higher levels, such as sentence reading). Secondly, the differing input system obviously drawn upon during reading must necessarily have been exapted from another neural system (presumably within the visual ventral stream). This second prediction is compatible with the neuronal recycling hypothesis [107,108], which postulates the re-purposing of existing brain maps for culturally recent tasks.

Box 4. Outstanding questions.

• To what extent can the model proposed here be generalized to language production? We have emphasized comprehension in the present paper, partly in view of the obvious discrepancies between the vocal abilities of humans and nonhuman primates and the resulting insight that computational parallels may lie elsewhere. However, the perspective that there may be a close link between human speech processing and the perception and production of rhythmic orofacial movements in nonhuman primates [88,89,91,109] also opens up potentially new avenues of investigation for connections at the production level.

• What role do subcortical structures and cortico-subcortical loops play in the dual-streams architecture advocated here? It has been suggested that the phylogenetic development of cortico-striatal circuits may have played a crucial role in the evolution of spoken language [110]. From the perspective of the current model, this proposal provides a possible explanation for the gap between nonhuman primate vocal abilities and the claim that they possess the necessary computational building blocks for language processing. According to an alternative theory, however, subcortical structures - and particularly the basal ganglia - are not only crucial for speech, but also contribute critically to basic sequencing processes [63], and indeed, the basal ganglia have been shown to play an important role in sequence learning both in humans [111] and monkeys [112].

• How does the model proposed here relate to differences between different nonhuman primate species? In this paper, the nonhuman primate model under discussion has primarily been a rhesus macaque model, based on the fact that most of the critical neurobiological nonhuman primate data on which our model is based were obtained in studies with macaques. However, a more detailed future proposal would clearly need to take into account any differences between different nonhuman primate species, e.g. with respect to the size and cytoarchitectonic structure of prefrontal cortex, which we have emphasized here as being crucial for cross-stream integration. Such data should increasingly become available with the use of noninvasive neuroimaging techniques in nonhuman primates (see [12,113]).

Concluding remarks

The model proposed here is conservative in certain respects and radical in certain others. It is conservative in assuming a primate model as the basis for the neurobiology of language, since the neuroanatomy of nonhuman primates has the closest correspondence to that of humans. It is radical for the very same reason: In spite of the obvious and extensive anatomical homologies between humans and nonhuman primates, neurobiological models of higher-order language have sought other candidate animal models (or no such models) to rely on, with songbird models as the most prominent (e.g. [95]), based on the behavioral and assumed computational differences between humans and nonhuman primates in communication and sequence processing. We have suggested that, to the contrary, the basic computational biological prerequisites for human language, including sentence and discourse processing, are already present in nonhuman primates. Across species, the antero-ventral stream performs a mapping from complex spectro-temporal patterns to time-invariant conceptual representations; it subserves the identification of increasingly more complex auditory objects and hence performs dependency formation as a basic computational mechanism in language. The postero-dorsal stream, by contrast, performs sequence processing in accordance with the constraints posed by an internal model. This ability may be rooted in mechanisms for spatial processing and/or action understanding, which have been generalized to apply to other domains. While the functions of the two streams differ quantitatively across species, we have argued that there is currently no compelling evidence for qualitative leaps in either stream between humans and nonhuman primates. Rather, the fascinating communicative power of human language may be rooted in the ability to share information dynamically across streams. This proposition reconciles the substantial phylogenetic differences in prefrontal cortex, which go hand in hand with the ability for language, with the neurobiologically plausible assumption that prefrontal cortex serves to regulate brain activity and behavior rather than performing domain-specific computational functions.

Figure I.

A: Computational properties of the antero-ventral stream Simplified depiction of the computational properties of the antero-ventral stream, which detects auditory objects in a hierarchically organized manner. Thus, neuronal populations close to auditory cortex (AC) detect elementary auditory features, while neuronal populations further downstream from AC detect auditory objects defined via more complex feature combinations. This stream therefore performs a transformation from auditory feature detection to conceptual feature detection (i.e., conceptual schemata are viewed as more complex and abstract auditory objects are viewed as bundles of conceptual features). We hypothesize that, in language, the transition from auditory to conceptual processing may take place at the level of the morpheme as the smallest meaning-bearing unit. "Auditory" object detection is thus predicted to be independent of sensory modality from the morpheme level upwards.

B. Computational properties of the postero-dorsal stream Simplified depiction of the computational properties of the postero-dorsal stream, which performs sequence processing in accordance with a hierarchically organized set of internal models. In contrast to the antero-ventral stream, feature combinations forming successively more complex (i.e., longer and hierarchically deeper) sequences are order-sensitive (non-commutative). Predictions generated at one hierarchical level provide a feedback (top-down) influence on the next level down in the hierarchy, while error signals arising from the comparison between the predicted and actual sensory input are transmitted up the hierarchy via feedforward connections. The three hierarchical levels (word, sentence, discourse) shown here are based on the empirical finding of successively larger auditory integration windows comprising these levels in the dorsal auditory stream [114].

Figure adapted with permission from [54].

Highlights.

• -We present a unified dual-streams model for primate audition and human language
• -Dorsal/ventral stream computations in language are grounded in primate audition
• -Ventral stream: auditory object recognition and commutative combinatorics
• -Dorsal stream: hierarchical predictive coding of sequences from phonemes to discourse