Neural architecture of the vertebrate brain: implications for the interaction between emotion and cognition

Abstract

Cognition is considered a hallmark of the primate brain that requires a high degree of signal integration, such as achieved in the prefrontal cortex. Moreover, it is often assumed that cognitive capabilities imply “superior” computational mechanisms compared to those involved in emotion or motivation. In contrast to these ideas, we review data on the neural architecture across vertebrates that support the concept that association and integration are basic features of the vertebrate brain, which are needed to successfully adapt to a changing world. This property is not restricted to a few isolated brain centers, but rather resides in neuronal networks working collectively in a context-dependent manner. In different vertebrates, we identify shared large-scale connectional systems involving the midbrain, hypothalamus, thalamus, basal ganglia, and amygdala. The high degree of crosstalk and association between these systems at different levels supports the notion that cognition, emotion, and motivation cannot be separated – all of them involve a high degree of signal integration.

Signals associated with perception, cognition, action, motivation, and emotion are intermingled in complex ways. How do they interact and become integrated in the brain? In the primate brain, “association” areas not directly linked to the sensory periphery were shown to exhibit complex response properties in the 1970–80s. The primate prefrontal cortex, in particular, was described as the apex of such association where signals converge and are integrated (Fuster, 2015; Miller and Cohen, 2001). In the present paper, we review evidence that the integration of diverse classes of signals, far from being a novel property of the primate prefrontal cortex, the primate brain, or even the mammalian brain, is evolutionarily ancient. It is a hallmark of the vertebrate brain.

As reviewed here, vertebrates have a brain architecture that allows considerable signal communication and integration. We propose that this type of architecture confers a high degree of flexibility that allows animals to cope with the complex interactions in their habitats, involving predators, prey, potential mates, and so on. Survival, and adapting, may benefit from circuits that can link multifaceted signals, as the number of conditions related to the internal and external worlds of the animal are exceedingly high.

In recent decades, and at a pace that has recently accelerated, ethology has revealed that behaviors across vertebrates are considerably more flexible and sophisticated than previously acknowledged (Brown, 2015; Doody et al., 2013; Pritchard et al., 2016; Riehl and Frederickson, 2016; Seyfarth and Cheney, 2015; Shettleworth, 2010). We briefly review relevant new data to motivate the type of neural architecture that may support such behavioral capabilities. We review evidence that the vertebrate central nervous system contains several large-scale connectional systems, and that there is a high degree of interaction and association among them. The evidence presented has major implications for understanding how information flows in the brain, and reveals a strong potential for the interaction of emotion and motivation with perception, cognition, and action. Taken together, the study of evolutionarily divergent and convergent patterns in neural networks, in combination with animal behavior studies in different species, offers unique opportunities for understanding some of the basic requirements of nervous system organization that allow animals to achieve their complex behavioral capabilities.

Changing views on brain evolution and animal cognition

Current understanding of animals’ behavioral capabilities has changed substantially in recent decades, thanks to the contributions of both ethologists and comparative psychologists (Shettleworth, 2010). Field studies of a large number of animals have uncovered the existence of complex and flexible behaviors in phylogenetically distant taxa (in both vertebrates and invertebrates) (Mikhalevich et al., 2017). In addition, the potential mechanisms underlying animal behavior have been investigated in different species with new techniques and approaches (Shettleworth, 2010). Together, these studies have shown that behavioral flexibility and plasticity are widespread in the animal kingdom, and that similar behavioral capabilities are at times present in phylogenetically distant species (e.g., de Waal and Ferrari, 2010; Wascher and Bugnyar, 2013). The modern view is in stark contrast to ideas prevalent during the first half of the twentieth century, which considered non-mammalian vertebrates as shaped by a narrow repertory of stimulus-driven stereotypical behaviors, with at most a basic ability to establish simple new relationships between stimuli (Edinger and Rand, 1908). These ideas were based on anthropomorphically simplistic interpretations of behavior guided by the scientifically obsolete idea of scala naturae – the notion of a natural progression across the animal kingdom (Burghardt, 2013; Hodos and Campbell, 1969).

Such antiquated and misguided notions about animal behavior were reinforced by a model of brain evolution through the sequential addition of new parts, as proposed by Edinger (Edinger and Rand, 1908). According to him, the forebrain contains an “old encephalon”, found in all vertebrates, and a “new encephalon”, which exists in animals “higher” than cartilaginous fishes. The new encephalon is more elaborated in mammals and shows a progressive evolutionary enlargement culminating in the human neocortex. The addition of entirely new parts would allow the brain to carry out new associations, which would support new cognitive capabilities and complex behaviors. These ideas inspired MacLean’s triune brain concept (MacLean, 1990) which still enjoys popularity among some researchers and clinical practitioners. However, by 1990, when MacLean published his model, a large amount of information about the brain organization of different vertebrates had accumulated, resoundingly demonstrating that the model was fundamentally wrong (Butler, 2009; Butler and Hodos, 2005; for discussions of misunderstandings about brain evolution, see Hodos and Campbell, 1969; Murray et al., 2016; for criticisms of the triune-brain model, see Reiner, 1990; Shimizu, 2004).

Comparative neuroscience has shown that the basic organization of the brain is shared across vertebrates (see below), while documenting differences in relative size of brain structures and/or in connectivity between them, enhancing interest in studying both commonalities and differences in behavioral abilities across species. “Spatial cognition”, which considers how animals process information about position to navigate through the world, is one of the areas that has generated vigorous interest (Tommasi et al., 2012). Animals have to navigate their environment effectively for survival, considering both potential resources (such as food and mates) and threats (such as rivals and predators). Studies in different species of vertebrates (mammals, birds, reptiles, amphibians, and fishes) indicate related spatial abilities (involved in homing behavior, spatial navigation, and spatial learning), possibly encompassing potentially homologous (for definition of terms in bold font, see Glossary) brain regions (Roth II et al., 2019; reviewed in Salas et al., 2003; see also Vallortigara, 2018). At the same time, the relative size of the neural structures involved differs between species with different spatial behaviors (for example, food-storing versus non-storing birds Krebs, 1990), or that live in habitats with varying complexity (e.g., Safi and Dechmann, 2005). A second area of research focus has studied social behaviors, including discriminating among individuals and categorizing them as offspring, mate, rival, ally, or neighbor, as well as the ability to learn from others (Bradbury and Vehrencamp, 1998; Bshary et al., 2014; Seyfarth and Cheney, 2015; Wiley, 2013). In particular, some shared neurobiological mechanisms might be involved in socially-relevant behaviors in different vertebrates (Goodson and Thompson, 2010; Medina, 2019).

When similar behaviors are observed in only some species of phylogenetically distant taxa, the possibility of convergent evolution processes must be seriously entertained. In such cases, no common ancestral origin is implicated, and the features or traits in common evolved independently. This probably underlies the case for behaviors exhibited by a few species of primates and birds, like the ability to use tools or to manipulate and deceive competitors (for tool use in fishes, see Brown, 2012; Clayton et al., 2007; Emery and Clayton, 2004; Güntürkün and Bugnyar, 2016). Convergent evolution of behaviors can provide not only valuable information about the selection pressures that promoted them, but also about nervous system design features that support them (Mikhalevich et al., 2017).

A common brain morphoplan in vertebrates: Understanding commonalities and differences

The history of vertebrates spans over 500 million years. An important aspect of their evolution is that the evolutionary trajectory of mammals is separate from that of reptiles and birds (sauropsids). During the early Carboniferous, the amniotes evolved from the reptiliomorph amphibians. Shortly after, the basal amniotes split into the mammalian lineage (synapsid) and the sauropsid lineage (Evans, 2000). Therefore, no current reptile was ancestral to any mammal, or vice versa, and both lineages have been independently evolving for more than 300 million years (Carroll, 1988; Evans, 2000; Liem and Walker, 2001).

The brain of all vertebrates is organized according to a common “building plan”, also called Bauplan or morphoplan (Nieuwenhuys and Puelles, 2016; Puelles et al., 2016; Striedter, 2005), such that it shares the same basic subdivisions (reviewed in Nieuwenhuys et al., 1998). To unravel the common architecture, it is critical to identify and map the same – homologous – brain regions in different vertebrates. Homology refers to relationships between traits that are shared as a result of common ancestry (Weiss, 1994). For example, the human upper limb (arm) and the bird wing are considered homologous because they arose from a corresponding character in the tetrapod common ancestor through descent with modification. Note, however, that they differ greatly in terms of both detailed structure and function.

Only by having a good handle on homology can we properly evaluate the degree of conservation, divergence, and/or convergence throughout evolution in structures and their connections. Identifying homologous brain regions is also essential to understanding the evolution of highly integrative areas that underlie emotion and cognition, and that support flexible, sophisticated behaviors. To identify homologies across animals, it is essential to first understand the building plan (morphoplan) of the body or the organ in question (such as the brain), and to recognize the relative position of the fundamental building blocks or subdivisions within the “general plan” (Puelles and Medina, 2002; Striedter, 2005).

At present, there are competing views about the “general plan” of the vertebrate brain (reviewed by Puelles et al., 2013; Striedter, 2005). Controversy is particularly acute regarding the telencephalon, the largest subdivision of the forebrain in amniotes (see next section). This is perhaps not surprising given the great divergence of the telencephalon between mammals and their closest relatives, the sauropsids, which introduces considerable challenges in determining correspondences between brain regions and/or parts.

To attempt to identify common subdivisions in the brain of different vertebrates, investigators have employed a broad range of approaches. The identification of basic building blocks in the brain of different species, and/or its subdivisions, has considerably benefited from the boom of “chemical architecture” (or chemoarchitecture) (for a review, see Nieuwenhuys et al., 1998) and, more recently, “genetic architecture” (genoarchitecture for short) studies (Medina, 2007; Puelles et al., 2000; Puelles and Medina, 2002). One approach, which we believe is particularly informative, is to focus on early developmental stages (phylotypic stages), and to study expression patterns of highly conserved genes involved in the specification and/or other early aspects of the development of the fundamental building blocks (Medina, 2007; Nieuwenhuys and Puelles, 2016; Puelles and Medina, 2002).

Fiber tract studies have greatly contributed to our understanding of how the basic building blocks of the brain and/or its subdivisions are connected and have often been used to guide the search for similar pathways in different species (for example, for the basal ganglia, as reviewed by Reiner et al., 1998). Variability in the connectivity pattern is also relevant because it might underlie and help understand inter-species differences in cognitive capabilities and/or behavioral repertoires. Comparative studies of connectivity patterns in the context of the brain morphoplan can also help to identify cases of convergence, providing a unique opportunity to analyze basic requirements needed to achieve similar functions. Here, we briefly review current knowledge about basic brain subdivisions with the goal of subsequently analyzing findings about short- and large-scale axonal fiber tract systems in the context of these subdivisions.

Adult vertebrate brains can be subdivided into three major territories: forebrain (prosencephalon), midbrain (mesencephalon), and hindbrain (rhombencephalon) (Figure 1). Overall, the forebrain is the part that has changed the most in evolution, while the midbrain and hindbrain are relatively more conserved. However, this generalization does not take into account the high degree of inter-species variability in some hindbrain and midbrain structures, such as the cerebellum and the midbrain tectum (called the inferior and superior colliculi in mammals) (Nieuwenhuys et al., 1998). The cerebellum is quite large in some fishes, as well as in birds and mammals, but not in most reptiles or amphibians, which indicates independent variation in the structure’s size across lineages. The visual part of midbrain tectum is large in most fishes, amphibians, and reptiles, and is huge in birds (with variations between species; Charvet and Striedter, 2008). In mammals, the homologous superior colliculus is not so large in relative terms, but the auditory-related inferior colliculus shows greater development (for details on these structures across, readers can refer to the comprehensive treatise on the vertebrate brain by Nieuwenhuys et al., 1998).

Figure 1.

Variability in vertebrate brains. (A) Schematic brains and cross sections at the level of the telencephalon. Green: telencephalon; blue: optic tectum/superior colliculus; orange: cerebellum. In the slices, gray represents the pallium and salmon represents the striatum (subpallium). (B) Vertebrate radiation. (C) Major components of the vertebrate brain. For fishes, a teleostean brain is shown (Meek and Nieuwenhuys, 1998). Abbreviations: d, dorsal; v, ventral.

The forebrain has a caudal division that includes the thalamus, and a rostral division that includes the hypothalamus (ventrally) and telencephalon (dorsally). The thalamus and hypothalamus are often considered part of the diencephalon, an intermediate subdivision located between telencephalon and mesencephalon (reviewed by Butler and Hodos, 2005), but here we follow a different scheme based on the embryonic genoarchitecture of the brain (see Albuixech-Crespo et al., 2017; Puelles et al., 2016; Puelles and Rubenstein, 2015). The telencephalon is relatively small in fishes and amphibians, but is larger in reptiles and very prominent in birds and mammals. The telencephalon shows two basic divisions in all vertebrates: a dorsally located pallium and a ventral subpallium (Figure 2). The term “pallium” (Latin for cloak) is used by comparative neuroanatomists to refer to the dorsal telencephalon of vertebrates.

Figure 2.

Organization of the forebrain in tetrapods. Light blue: pallium. Beige: subpallium, striatal-related. Magenta: subpallium, pallidal-related.

Abbreviations: A, arcopallium; Ac, nucleus accumbens; ACo, anterior cortical amygdala area; ADVR, anterior dorsal ventricular ridge; BC, basal complex of the amygdala; BST, bed nucleus of the stria terminals; Ce, central amygdala; DLP, dorsolateral pallium; DP, dorsal pallium; H, hyperpallium; HF, hippocampal formation; Hy, hypothalamus; LP, lateral pallium; M, mesopallium; Me, medial amygdala; N, nidopallium; NCx, neocortex; MP, medial pallium OT, optic tract; PCo, posterior cortical amygdala area; PO, preoptic area; Pt, pretectum Se, septum; St, striatum; Stv, ventral striatum; Th, thalamus; VCP, ventrocaudal pallium; VP, ventral pallium.

Across vertebrates, the subpallium is relatively conserved and contains the striatum, pallidum (not to be confused with “pallium”), parts of the amygdala (subpallial amygdala) and the bed nucleus of the stria terminalis (BST), among others (Gonzalez et al., 2014; Medina et al., 2017; Moreno et al., 2009). In contrast, the pallium is highly divergent and the comparison of its subdivisions across vertebrates is subject to considerably controversy, as briefly explained in the following section (Belgard et al., 2013; Briscoe and Ragsdale, 2018, 2019; Butler et al., 2011; Desfilis et al., 2018; Box 1; Dugas-Ford et al., 2012; Jarvis et al., 2005; Karten Harvey, 2015; Karten, 1991; Karten, 1997; Karten, 2013; Puelles, 2001; Puelles et al., 2017; Reiner et al., 2004a; Tosches et al., 2018).

The increase in relative size of the forebrain in amniotes (reptiles, birds, and mammals) is largely associated with the increase in volume and complexity of the pallium, and with its connections with the thalamus (directly and/or by way of the subpallium and hypothalamus). Whereas we discuss below general traits observed in major vertebrate groups (reptiles, birds, and so on), it is important to keep in mind that there is important inter-species variability within each group (Charvet and Striedter, 2008).

Controversies concerning the pallium

For almost a century, large portions of the telencephalon of reptiles and birds, including a large region called the dorsal ventricular ridge (DVR), were considered homologous to the basal ganglia (in the subpallium) of mammals. However, it is now understood that the DVR is actually part of the pallium (Reiner et al., 2004b). The homologies of the DVR and other pallial sectors of reptiles and birds to pallial areas of mammals have been the subject of intense debate (Jarvis et al., 2005; Karten, 1997; Puelles et al., 2013; Puelles et al., 2017; Striedter, 2005). Here, we briefly consider some of the competing views.

The cell-type homology hypothesis

Tract-tracing studies in birds and mammals have demonstrated considerable similarity of major sensory conduction pathways traveling to the forebrain (Butler, 1994a, b). According to the cell-type homology hypothesis, the corresponding targets of these pathways in the pallium of mammals and sauropsids (i.e., mammalian neocortex, and sauropsid DVR and dorsal pallium, the latter including the Wults in birds) should be homologous (Karten, 1997). For instance, the sauropsid DVR includes cell groups that receive input from auditory and visual thalamic nuclei and that are proposed to be homologous to the thalamorecipient layer 4 cells of the temporal neocortex (see also Butler et al., 2011; Karten, 1991). The proposal also suggests the homology of association cells of the DVR (the nidopallium in birds) with neocortical layers 2–3 cells, and the homology of “motor” output cells of the caudal DVR (the arcopallium in birds) with layer 5 cells of the motor neocortex (Karten, 1997). The “motor” cell group located in the arcopallium of birds is particularly well studied in songbirds and is involved in vocal production (reviewed by Jarvis et al., 2005). The cell-homology proposal has recently received support from comparative studies using gene expression patterns in adult animals (see also Briscoe and Ragsdale, 2018; Dugas-Ford et al., 2012).

The developmental genoarchitecture-based homology hypothesis

This view is based on shared combinatorial expression patterns of highly conserved regulatory genes observed in the brain of different vertebrates at early embryonic stages (Puelles and Medina, 2002). When comparative genoarchitecture data are combined with key morphological landmarks, the embryonic pallium of vertebrates can be subdivided into four (Puelles et al., 2017) or six (Desfilis et al., 2018) compartments that are comparable across species. To unify the proposals of four versus six pallial divisions, here we refer to them as medial, dorsal, dorsolateral/lateral, and ventral/ventrocaudal pallial divisions (Figure 3).

Figure 3.

Brain divisions and neural networks in different vertebrates. In fishes, high integration occurs mainly at the midbrain tectum and hypothalamus, although the telencephalon (striatum and pallium) also contributes. In amphibians, the striatum (and other striatal-like areas like the central amygdala) increases its relevance as it gains inputs from the thalamus. In amniotes (reptiles, birds and mammals), the pallium and its associated networks through the subpallium and thalamus gain importance, and there is a gradual shift to the pallium as a major integration center and pallial networks become major drivers of brain function. Solid arrows represent conserved projections, while open arrows represent new projections.

According to the developmental-genoarchitecture proposal, the medial pallium gives rise to the hippocampal formation (Nieuwenhuys et al., 1998; Wullimann and Mueller, 2004); Medina (Medina et al., 2017). In mammals the dorsal pallium gives rise to the isocortex (also called neocortex, with six layers), the dorsolateral/lateral pallium produces the claustro-insular region (as well as the orbitofrontal cortex rostrally, and the perirhinal/lateral entorhinal cortex caudally), while the ventral/ventrocaudal pallium gives rise to the olfactory cortex and part of the amygdala, the so-called pallial amygdala (Desfilis et al., 2018; Medina et al., 2017; Moreno and González, 2006; Puelles et al., 2017).

Most of these pallial compartments greatly diverge during late development in different species, giving rise to areas of varying cytoarchitecture, chemoarchitecture, and connections. Accordingly, in the adult, they might contain novel cell groups that cannot be considered homologous to any of those found in the same compartment of other vertebrates (Medina, 2007; Medina et al., 2017; Puelles and Medina, 2002; Striedter, 2005; Striedter, 2016). Nevertheless, structures derived from a given compartment share some features (commonalities) in different vertebrates as a result of their common embryonic origin (Medina et al., 2017). For example, in mammals the dorsal pallium produces the isocortex (also called neocortex), while the ventral/ventrocaudal pallium gives rise to the pallial amygdala. In contrast, in birds and reptiles, the dorsal pallium produces the Wulst (also called hyperpallium, and an equivalent rudiment in reptiles), while the ventral/ventrocaudal pallium gives rise to most of the DVR (Desfilis et al., 2018; Medina and Abellán, 2009; Puelles, 2001; Puelles et al., 2000). The mammalian isocortex and the avian Wulst are highly dissimilar, although they share some commonalities, including visual and somatosensory inputs from the part of the thalamus called the lemnothalamus (Butler, 1994b; Medina and Reiner, 2000). The mammalian pallial amygdala (including the basolateral complex) and the avian/reptilian DVR (including the so-called nidopallium and arcopallium in birds) are also very different, but they share their sensory input from the part of the thalamus called the collothalamus (Butler, 1994a, b), and both show extensive reciprocal glutamatergic (excitatory) connections with other pallial areas (Medina and Abellán, 2009; Medina et al., 2017; Medina et al., 2011). This hypothesis, in particular the homology between pallial amygdala and DVR, has received support by data from single cell transcriptome, when focused on comparisons of glutamatergic cells (Tosches et al., 2018).

In this review, we follow the developmental genoarchitecture-based hypothesis because, in our view, it has great potential to explain the organization of the brain in different species, and because it partially agrees with other predictive models of the pallium that focus on the mammalian brain (Barbas, 2007; García-Cabezas et al., 2019); for a discussion of how brain models help understanding and predicting patterns of connections of different divisions, see Barbas (2007), García-Cabezas et al. (2019), and Medina et al. (2019). Because different subdivisions might process information in different ways, when discussing the anatomical pathways below, we believe that it is important to maintain the distinction between pallial subdivisions, rather than considering pallial projections in general (that is, the pallium as an undifferentiated unit).

Readers seeking further discussion on the pallium are encouraged to consult additional literature (for example, Briscoe, 2019; Briscoe and Ragsdale, 2018; Medina, 2019; Nieuwenhuys et al., 1998; Striedter, 2005). Whereas the ongoing debates are anticipated to continue for some time, we believe their precise resolution does not undermine our main thesis: Integration of diverse signals is evolutionarily ancient, and is probably a key property undergirding flexible behaviors in all vertebrate taxa.

Integration involving the midbrain tectum and hypothalamus

As developed throughout the paper, a common property of the vertebrate brain is that it combines signals from the internal and external worlds to support adaptive behaviors and responses. Two brain regions play key roles in such integration processes: the midbrain tectum and the hypothalamus (Figure 3). The tectum receives visual, somatosensory, and auditory inputs, and supports visually- and auditorily-guided spatial orientation and multisensory guidance of behavior (reviewed by ten Donkeelar, 1998a, b). In all vertebrates, the hypothalamus receives visual, gustatory, and olfactory signals from the external world (directly from the eye and/or by way of the thalamus, brainstem, and telencephalon), as well as internal-state signals, and participates in, among many others, homeostasis-related processes by engaging with both the autonomic and endocrine nervous systems (reviewed by Nieuwenhuys et al., 1998) (Figure 3). The hypothalamus is substantially connected with the telencephalon (with both the subpallium and the pallium) in a reciprocal manner. The tectum is reciprocally connected with the telencephalon, too, but to a lesser degree. In some vertebrates, such as birds and mammals, a tectum-telencephalon-tectum¹ loop involves the collothalamus (as a relay in the ascending pathway) and a ventral/ventrocaudal part of the pallium that contains the pallial amygdala (or the comparable region in birds), and appears to play a role in early social orienting responses (Mayer et al., 2017). In amphibians, a tecto-telencephalo-tectal loop also involves the collothalamus in the ascending pathway, but in the telencephalon it mainly involves the basal ganglia of the subpallium, although the latter receives input from the pallial amygdala (Marin et al., 1997a; Marin et al., 1998).

In fishes, the thalamus is small and shows little interaction with the telencephalon, although it receives retinal input and is reciprocally connected with the tectum and the hypothalamus (Figure 3). A major change during vertebrate evolution was the increase in connections between the thalamus and the telencephalon, which are substantial in amniotes (reptiles, birds, and mammals) (reviewed by Nieuwenhuys et al., 1998). The appearance and potentiation of reciprocal connections between the thalamus and the pallial component of the telencephalon, both of which gained relevance as integration hubs, is particularly noteworthy. These events were accompanied by an increase in complexity of both the thalamus and the pallium, via substantially divergent evolutionary trajectories in birds (and sauropsids, in general) and mammals (Figure 1).

In what follows, we review three noteworthy connectional systems that were amplified in amniotes: thalamo-cortical bidirectional connections, basal ganglia loops, and amygdala circuits. We then highlight the ample associations between these systems and discuss functional implications of the anatomical features explained here.

Thalamocortical systems

In mammals, the thalamus heavily innervates the cortex (Figure 3C) (Jones, 1985). The connections involve both “first-order” thalamic nuclei with specific sensorimotor signals and “higher-order” thalamic nuclei that may play a particularly prominent role in corticocortical communication (Sherman, 2009). Corticothalamic projections reciprocate the ascending projections of first-order nuclei and additional cortical projections target higher-order thalamic nuclei (Figure 3C). These projections provide robust feedback that dynamically sharpens and shapes sensory processing under different contexts and changing demands (Briggs and Usrey, 2008; Crandall et al., 2015). Together, thalamocortical loops are a prominent feature of the mammalian brain and are believed to play computational roles that are relevant for spatiotemporal coding and attention, among others (Buzsáki, 2006).

By and large, thalamocortical circuits have been conceptualized as independent, parallel streams, such as those in the visual and somatosensory systems. In this regard, the following considerations are relevant for understanding these circuits and their evolution:

1. Thalamocortical circuits involve several cell types and “channels”. The major corticothalamic channel, which is centered on cortical layer 6, involves intermediate actors such as the prethalamus (called in mammals the reticular nucleus), which receives collaterals of thalamocortical and corticothalamic axons that are organized topographically (Briggs and Usrey, 2008; Crandall et al., 2015; Desilets-Roy et al., 2002). The projection through the reticular nucleus is involved in lateral inhibition, and appears to be important in shaping sensory processing and attention (Briggs and Usrey, 2008; McAlonan et al., 2006).

2. Thalamocortical circuits are not entirely “closed”, that is to say, there exists crosstalk between adjacent, as well as distant loops. A key underpinning of such intercommunication is that single neurons of specific corticothalamic projections extend beyond the specific thalamic source of the thalamocortical projections (Bourassa and Deschênes, 1995). Thus, they contact both first- and high-order thalamic nuclei, with the latter being involved in providing driving-type inputs to different cortical areas (Deschênes et al., 1998; Sherman, 2017).

3. Some thalamocortical loops include pathways that course indirectly by way of other brain sectors. These may involve the pallium itself, the subpallium, and/or the hypothalamus, thereby intersecting with distinct, but interrelated functional networks (as outlined in the following sections; see Figures 3–4). For example, in mammals, there are pathways from the thalamus (more specifically, the collothalamus) to the pallial amygdala, which in turn projects to several other pallial sectors (for example, the hippocampal formation and the perirhinal/entorhinal areas) with projections to the thalamus, as well as the subpallium and hypothalamus (Linke et al., 2004; Pikkarainen and Pitkänen, 2001). These pathways provide additional opportunities for inter-circuit communication at pallial, subpallial, hypothalamic, and thalamic levels (see also Usrey and Sherman, 2019).

Figure 4.

Basal ganglia and amygdalar loops in vertebrates. (A) Mammals. (B) Birds. (C) Reptiles. (D) Amphibians (based on frogs). A color-code is used to indicate GABAergic/inhibitory (orange) versus glutamatergic/excitatory (blue) projections. There is a common organization to all, involving pallial /cortical-like) inputs to a striatal center that then projects to a pallidal center, which in turn projects to a thalamic nucleus projecting to the pallium. Ventral basal ganglia loops (through the accumbens) and subpallial amygdalar loops (through the central amygdala) are found across tetrapods, whereas dorsal basal ganglia loops are only seen in birds and mammals. Different pallial compartments providing differential input to the striatum are represented. In each loop, additional channels via the prethalamus are found across all vertebrates. As explained in the text, there is ample crosstalk and association between loops at different levels, including the pallium. Notably, some pallial divisions project through all loops.

Abbreviations (we present them separated by groups; common abbreviations are not repeated): Mammals: Ac, nucleus accumbens; ACC, anterior cingulate cortex; BST, bed nucleus of the stria terminalis; Ce, central amygdala; CPu, Caudate-Putamen; DLP, Dorsolateral pallium; DP, Dorsal pallium; GP, Globus Pallidus; Hipp, Hippocampal formation; Ins, Insular Cortex; LP, Lateral pallium; MD, Mediodorsal thalamic nucleus; MP, Medial pallium; OFC, Orbitofrontal Cortex; Pa, Pallidum; Pal Amyg, Pallial Amygdala; PTh, Prethalamus; PVT, Paraventricular thalamic nucleus; Rh, Perirhinal and Lateral Entorhinal Cortices; Rt, Reticular nucleus; SPF, Subparafascicular Nucleus; St, Striatum; PFC, Prefrontal Cortex; PTh, Prethalamus; Th, Thalamus; VA, Ventral anterior thalamic nucleus; VCP, Ventrocaudal pallium; VP, Ventral pallium. Birds: ArP, Arcopallium; DIVA, Dorsointermediate thalamic nucleus; DM, Dorsomedial thalamus; Md, Mesopallium, dorsal part; NC, Nidopallium, caudal part; SRT, Subrotundal thalamic nucleus; VIA, Ventrointermediate thalamic nucleus. Reptiles: DLA, Dorsolateral amygdala; DM, Dorsomedial thalamus; MPT, Medial posterior thalamic nucleus; PDVR, Posterior dorsal ventricular ridge. Amphibians (frogs): A, Anterior thalamic nucleus; LA, Lateral amygdala; RP, Rostral pallium. For the projections in non-mammals, we followed the following major sources: Basal ganglia: (Folgueira et al., 2004; Gonzalez et al., 1990; Guirado et al., 1999; Marin et al., 1997a; Marin et al., 1997b, 1998; Medina and Reiner, 1995; Medina and Smeets, 1991; Moreno et al., 2009; Reiner et al., 1998; Russchen and Jonker, 1988; Smeets and Medina, 1995). Amygdala (or Amygdala-like regions including caudal parts of the ventral/ventrocaudal pallium, such as the caudal nidopallium in birds, posterior dorsal ventricular ridge in reptiles, and lateral amygdala in amphibians, as identified by Moreno and González, (2006): (Kröner and Güntürkün, 1999; Lanuza et al., 1998; Martínez-García et al., 2002; Martínez-García et al., 2007; Medina et al., 2017; Moreno and González, 2006; Moreno et al., 2012) Other connections of the pallium in amphibians and reptiles: (Bruce and Butler, 1984a, b; Folgueira et al., 2004; Laberge et al., 2008; Roth et al., 2007)

Corticothalamic circuits are highlighted as a prominent feature of the mammalian brain. Are there thalamus-pallium-thalamus loops in non-mammals? If so, how are these organized? Although interconnectivity between cortical regions (pallium) and the thalamus is most substantially developed in mammals (Figure 3), thalamopallial loops are also found in birds and reptiles (reviewed by Butler, 2008; see also Nieuwenhuys et al., 1998). Some of these involve the dorsal pallium. In the pigeon, reciprocal connections are present between the visual part of the Wulst (in the dorsal pallium) and a retinorecipient thalamic nucleus often compared to the dorsal lateral geniculate nucleus of mammals (Medina and Reiner, 2000); in addition, a somatosensory part of the Wulst is reciprocally connected with a somatosensory thalamic nucleus (Medina and Reiner, 2000). These visual and somatosensory thalamic nuclei also project to pallial areas outside the Wulst (Kröner and Güntürkün, 1999), suggesting the existence of partially open loops. The situation may be similar at least in some reptiles, like turtles (reviewed by ten Donkeelar, 1998b).

In birds and reptiles (sauropsids), there are also indirect thalamopallial pathways involving other sectors of the pallium, including the medial pallium (the hippocampal formation) and the ventral/ventrocaudal pallium (the caudal nidopallium and the arcopallium of birds, and the equivalent areas of reptiles that are homologous to the pallial amygdala of mammals; Desfilis et al., 2018; Tosches et al., 2018). These projections mainly involve unimodal and multimodal sensory nuclei of the thalamus. For example, in sauropsids there are visual and auditory pathways from the thalamus (collothalamus) to pallial amygdala-like areas in the DVR² (Martínez-García et al., 2002; Martínez-García et al., 2007), which in turn project to several other pallial sectors (for example, the hippocampal formation and areas of the dorsal/dorsolateral pallium) with projections to the thalamus, as well as the hypothalamus and/or subpallium (Kröner and Güntürkün, 1999; Medina et al., 2017; Veenman et al., 1995). These indirect pathways coursing through the pallial amygdala are at least partially comparable to those seen in mammals. Moreover, additional indirect thalamopallial connections course through the hypothalamus and the subpallium (as explained in the following sections). The pallium in amphibians is more rudimentary and, although there are thalamopallial projections to the medial pallium and ventral pallium (Figure 3A–B), these animals lack direct reciprocal thalamopallial loops (Laberge et al., 2008; Moreno and González, 2006). Therefore, a bidirectional communication system between the cortex/pallium and the thalamus is a feature of the amniote brain, and is most developed in mammals and birds, and less so in reptiles.

Basal ganglia loops and large-scale connectional systems

Classically linked to movement control and disorders, the basal ganglia are now known to be involved in cognition, motivation, and emotion, and viewed as essential for higher level behavioral control, including learning and regulation of stimulus-driven behaviors, as well as action selection supporting goal-directed behaviors (DeLong and Wichmann, 2009; Nelson et al., 2004; Yin and Knowlton, 2006). Work focusing on mammals in the 1970–80s uncovered the cortical-subcortical connectional architecture of the basal ganglia (Alexander et al., 1986; Alheid and Heimer, 1988). Nearly the entire cortical sheet projects to the striatum, and whereas striatal territories receiving cortical input do not directly reciprocate their connections, pathways return to cortex via different parts of the thalamus after an additional step in the pallidum. Together, these studies have led to the important concept of cortico-basal ganglia-thalamo-cortical systems, or basal ganglia loops in short (Alexander et al., 1986; Haber, 2003). An important feature of mammalian basal ganglia loops is that they involve both dorsal (caudate-putamen) and ventral (nucleus accumbens) basal ganglia components (Figure 4A). Some cortical areas project to the dorsal striatum (for example, motor and somatosensory areas), while others project to the ventral striatum (in primates, for example, orbitofrontal, prefrontal, and anterior cingulate cortices). It is often assumed that dorsal basal ganglia loops are primarily involved in motor functions, while ventral basal ganglia loops are involved in motivation and reward (reviewed by Reiner et al., 1998). However, similar to the ventral basal ganglia, the dorsal basal ganglia also receive input from the prefrontal cortex and the pallial amygdala (Figure 4), and are involved in non-motor functions (DeLong and Wichmann, 2009; Nelson and Kreitzer, 2014).

How is basal ganglia connectivity organized in other vertebrates? Is the looped architecture observed, too? Birds exhibit basal ganglia loops via both dorsal and ventral basal ganglia sectors (Reiner et al., 1998) (Figure 3B). Most of the pallium projects to the striatum, and subsequently to the globus pallidus, returning to several areas of the pallium via the thalamus (Reiner et al., 1998) (Figure 4). Basal ganglia loops are important for song learning (Bolhuis and Gahr, 2006; Jarvis, 2004), which is reminiscent of the role of these loops in language learning in humans (Scharff and Haesler, 2005). This represents a case of functional convergence, as vocal learning is a derived feature present exclusively in a few distantly related groups of mammals (humans, bats, and some cetaceans) and birds (parrots, hummingbirds, and songbirds) (Jarvis, 2004). Moreover, the pallial sectors involved in these loops are not homologous. The convergent coevolution of behaviors and brain mechanisms in distant taxa may suggest that there may be a limited number of ways in which vertebrate nervous systems can be configured to compute the information needed to produce that behavior (Mikhalevich et al., 2017). Other vertebrates do not have loops involving the dorsal basal ganglia; they lack the projection from the dorsal pallidum (corresponding to the mammalian globus pallidus) to the thalamus. However, loops involving the ventral basal ganglia exist in reptiles (Figure 4C) and amphibians (Figure 4D) (Medina, 2009; Moreno et al., 2009; Reiner et al., 1998).

In mammals and other vertebrates, the standard view is that dorsal and ventral basal ganglia loops are anatomically and functionally segregated. However, there is ample evidence of associations between them, as well as for their interlinking with subpallial amygdala circuits (see next section). An important way in which the two loops are interlinked relies on substantial pathways between cortical/pallial areas from different compartments, which in turn project to different subpallial sectors (Figure 4). For example, in mammals, the pallial amygdala is reciprocally connected with the isocortex, the lateral entorhinal cortex, and the hippocampal cortex (Ghashghaei and Barbas, 2002; Pikkarainen and Pitkänen, 2001; Pikkarainen et al., 1999), and projects to the accumbens, the caudate-putamen complex, and the subpallial amygdala (reviewed by Medina et al., 2017). This is similar to the situation found in birds and reptiles, where the caudal nidopallium (and the equivalent posterior dorsal ventricular ridge in reptiles) is reciprocally connected with the dorsolateral pallium (which includes an entorhinal-like area as well as an orbitofrontal-like area; Desfilis et al., 2018; Medina, 2019; Medina et al., 2017), the Wulst (a dorsal pallial region homologous to part of the neocortex), and the hippocampal formation, and projects to the nucleus accumbens, dorsal striatum and subpallial amygdala (Medina, 2019; Medina et al., 2017) (Figure 4). Together, these pathways interlink dorsal and ventral basal ganglia loops in ways that are not usually considered (see also Averbeck et al., 2014; Haber, 2003). Because this interlinked arrangement is found not only in birds and mammals, but also in reptiles, this feature was likely present in the most recent amniote ancestor. Moreover, the ample crosstalk between loops suggests a major role of both the dorsal and ventral basal ganglia of reptiles in processing non-motor signals that are more closely aligned with contextual, emotional, and motivational signals, and less so with sensorimotor information.

As pointed out, amphibians contain a rudimentary pallium. Accordingly, basal ganglia circuits involve relatively limited pathways to or from the pallium. Nevertheless, the striatum receives inputs from the pallium and the thalamus, and can be subdivided into dorsal and ventral components (Marin et al., 1998; Reiner et al., 1998). Direct basal ganglia loops through the thalamus are only weak for the ventral part and do not exist for the dorsal basal ganglia (Moreno et al., 2009) (Figure 4D). However, related long-range circuits exist that may provide functionality reminiscent to that found in basal ganglia loops. In particular, both dorsal and ventral components of the basal ganglia project to the prethalamus, which loops back to the pallium via the thalamus. Indeed, circuits via the prethalamus exist in all tetrapods (Figure 4).

In fishes, basal ganglia loops involving the thalamus have not been observed (Folgueira et al., 2004). The basal ganglia are mainly connected with the hypothalamus, which processes multiple types of sensory information (Folgueira et al., 2004). A noteworthy feature not seen in other vertebrates is that the striatum projects directly to the pallium, placing the former as an important integration center able to modulate pallial as well as hypothalamic function. By way of the hypothalamus and prethalamus, it may also modulate the thalamus and the midbrain tectum. Thus, in fishes, palliostriatal loops are likely involved in the integration of a wide spectrum of signals.

Taken together, basal ganglia loops of the type observed in mammals are present in tetrapods. In birds, in particular, the architecture is very similar to that of mammals. Only birds and mammals have direct loops to the thalamus involving both the dorsal and ventral basal ganglia, while reptiles and amphibians have direct loops involving the ventral basal ganglia. In reptiles and amphibians, some aspects of the pathways through the dorsal basal ganglia are also present, but not the component involving a direct projection via the thalamus; thus, not a “proper” dorsal loop. In many vertebrates (including reptiles, amphibians, and fishes), there are basal ganglia projections to the hypothalamus and prethalamus, both of which in turn project to the thalamus, thus providing another way by which the basal ganglia can modulate thalamic function; these projections may correspond to the ancestral condition of vertebrate basal ganglia circuits (Medina, 2009).

Amygdala circuits and connections

The amygdala of mammals consists of a basolateral complex (including lateral, basal, and accessory basal nuclei), some cortical areas, and an extended amygdala (including the central nucleus, the medial nucleus, and the complex of the bed nucleus of the stria terminalis, BST) (Alheid and Heimer, 1988; Amaral et al., 1992). Current evidence suggests that the basolateral amygdala is mostly of pallial origin³ and the extended amygdala is mostly of subpallial origin⁴ (Martínez-García et al., 2007; Medina et al., 2017). In an important proposal, Alheid and Heimer (1988) described how the architecture of cortical-basal ganglia loops is more general and involves similar pathways through other regions of the forebrain, including the septum and the extended amygdala (Figure 3). Here, we consider the pathways involving the pallial amygdala and the central extended amygdala, where the central amygdala plays a role comparable to that of the striatum in basal ganglia loops (Medina et al., 2011; Waclaw et al., 2010). The central amygdala receives inputs from the four major pallial compartments, including that of the pallial amygdala (Phelps and LeDoux, 2005). In fact, the pallial amygdala interfaces with the extended amygdala much like the isocortex interfaces with standard basal ganglia loops (functionally, this also matches the integrative properties of the pallial amygdala which receives massive inputs from across the cortex). The central amygdala projects to the BST (mostly its lateral part), which can be considered a pallidum-like region in terms of the GABAergic molecular profile of its cells and their embryonic origin (Bupesh et al., 2011; reviewed by Medina et al., 2017; in terms of the GABAergic molecular profile of its cells and their embryonic origin; Nóbrega-Pereira et al., 2010). The BST subsequently projects to the hypothalamus, thalamus, and brainstem (Dong et al., 2001; Phelps and LeDoux, 2005). Among these, the projections to the thalamus are interesting, because this region in turn projects to several pallial/cortical targets (Kirouac, 2015). The overall arrangement thus establishes the pathway through the central extended amygdala and back to the pallium. However, while pathways through the basal ganglia and back to the pallium are semi-closed loops, those through the extended amygdala appear to be more open, involving several pallial sectors as targets, indicating more extensive influence on pallial function (as discussed below).

The pathways from the BST to the thalamus target the paraventricular thalamic nucleus (PVT) and other midline nuclei (Dong et al., 2001; Dong and Swanson, 2006) (Figure 4A, Figure 5). The PVT is particularly relevant from the point of view of interlinking the basal ganglia and the amygdalar connectional systems (Kirouac, 2015). This thalamic nucleus projects to both the central extended amygdala and the nucleus accumbens, and is reciprocally connected with pallial areas, such as the insular cortex, the prefrontal cortex (including orbitofrontal cortex), the hippocampal formation, and the basolateral complex of the amygdala (all of these pallial sectors are reciprocally interconnected and project to the central extended amygdala and nucleus accumbens) (reviewed by Kirouac, 2015). In addition, the PVT receives substantial inputs from the hypothalamus and the brainstem (reviewed by Kirouac, 2015). Together, the PVT is a key node for integration of emotional and reward-relevant information and for modulating behavior in a context-dependent manner. Indeed, recent research focused on this region has uncovered its critical contribution during both appetitive and aversive processes (Beas et al., 2018; Do-Monte et al., 2015; Penzo et al., 2015; Zhu et al., 2018). Finally, the BST projects to the ventral tegmental area (VTA), a dopaminergic center that provides a modulatory projection to both the ventral basal ganglia (nucleus accumbens) and the extended amygdala (including the central amygdala), thus providing another channel for crosstalk (Figure 6).

Figure 5.

Interlinking of basal ganglia and other connectional systems. Lateral schematic views of primate (A, A’) and avian (B, B’) brains illustrating the major sensory inputs to the amygdala (from the olfactory bulb and from the collothalamus), as well as the substantial connections of the amygdala with the pallium and the ventral basal ganglia system. Homologous subdivisions of the amygdala are represented using the same color code. For simplicity, only representative connections are shown. Abbreviations: A-A’ (Primates): BC, basal amygdalar complex (derives from ventral and ventrocaudal pallium); BST, bed nucleus of the stria terminalis; cc, corpus callosum; CeA, central amygdala; Cer, cerebellum; Fr, frontal lobe of NCx; Hyp, hypothalamus; MeA, medial amygdala (purple in scheme); NCx, neocortex; OA, anterior olfactory area; OB, olfactory bulb; Oc, occipital lobe of NCx; OFC, orbitofrontal cortex; Pa, pallidum; PAG, periaqueductal gray; Par, parietal lobe of NCx; PFC, prefrontal cortex; SC, superior colliculus; St, striatum; Te, temporal lobe of NCx; Th, thalamus; VTA, ventral tegmental area. B-B’ (Birds; common abbreviations are not repeated): A, arcopallium (ventrocaudal pallium); APH, parahippocampal area; Ce, central amygdala; DLP, dorsolateral pallium; H, hyperpallium (dorsal pallium); LAN, laminar pallial nucleus (rostral part of DLP comparable to the orbitofrontal cortex); M, mesopallium (lateral pallium); Me, medial amygdala; N, nidopallium (ventral pallium); NC, caudal nidopallium; TeO, optic tectum of the midbrain (comparable to the superior colliculus).

Figure 6.

Pallial amygdala circuits in mammals and birds. Similar principles are observed in both, such as input from the pallial amygdala (or the equivalent region in the ventral/ventrocaudal pallium of birds) to the subpallial extended amygdala, following a basal ganglia-type organization. The pallial amygdala is highly and reciprocally connected with multiple pallial areas from different compartments, including the hippocampal formation (Hipp), the dorsal pallium (neocortex and Wulst), areas of the lateral/dorsolateral pallium (in mammals including orbitofrontal, insular and perirhinal/lateral entorhinal cortices; in birds they include the mesopallium (M) and dorsolateral pallium (DLP), as well as other areas of the ventral pallium (in birds, the rostral and intermediate parts of the nidopallium (N). It is thus an important integration center. The pallial amygdala projects through different loops of the subpallium, being in a pivotal position to influence on emotion and motivation-driven behaviors but also on cognitive functions implying attention, learning and adaptation to a changing world. Moreover, given the high association between different loops and the high influence of the pallial amygdala on all, it is not longer possible to separate emotion and cognition as classically done, and understanding this is essential for understanding brain function (see text for discussion).

Abbreviations: Ac, Nucleus accumbens; ArP, Arcopallium; BST, Bed nucleus of the stria terminalis; Ce, Central amygdala; CPu, Caudate-putamen striatal complex; DLP, Dorsolateral pallium; GP, Globus pallidus; Hipp, Hippocampal formation; Ins, Insular Cortex; LHy, Lateral Hypothalamus; M, Mesopallium; N, Nidopallium; NC, Caudal part of the Nidopallium; NCx, Neocortex; Pall Amyg, Pallial amygdala; Rh, Perirhinal and lateral entorhinal cortices; SC/IC, Superior and inferior colliculi; St, Striatum; TeO/TS, Optic tectum and torus semircularis; Th, Thalamus; VTA, ventral tegmental area.

Comparative studies have sought to identify and characterize the organization of the two main amygdalar subcomponents (pallial and subpallial) across vertebrates (Martínez-García et al., 2007; Medina et al., 2017; Moreno and González, 2006). Given that fear- and other emotion-related responses are found in all animals and are essential for survival, one would expect that a key brain region regulating these responses, namely the amygdala, would be highly conserved and the task of identifying it across vertebrates would be relatively straightforward. Rather, this is far from being the case, and particularly problematic for the pallial amygdala. Genoarchitecture studies focusing on development have helped to identify the same embryonic pallial compartments in different amniotes (Desfilis et al., 2018; Medina, 2009; Medina et al., 2017; Puelles et al., 2000; Puelles et al., 2017). For the case of the hippocampal formation, which is produced by the medial pallium and shows a high degree of conservation during late development, there is broad consensus about its homology across species (Belgard et al., 2013; Medina et al., 2017; Tosches et al., 2018). However, this is not the case with the pallial amygdala, which originates from the ventral/ventrocaudal pallium (Medina et al., 2017). In birds and reptiles, the same embryonic compartment produces the dorsal ventricular ridge (DVR; with the avian nidopallium and arcopallium), a structure that is specific to the sauropsid brain. Their common embryonic origin has led to the proposal of a developmentally based homology between pallial amygdala and at least a caudal part of the DVR (caudal nidopallium and arcopallium in birds), perhaps extending to the rostral part of the DVR that includes sensory areas (see Box 1 for further discussion; see also discussion by Bruce and Neary, 1995; Medina et al., 2017). This proposal has recently received strong support from comparative data based on single-cell transcriptomics (Tosches et al., 2018) and is followed here, as mentioned above. However, we need to be cautious because homology of embryonic compartments does not necessarily imply homology of all the derived structures (that is, those in adults), especially in cases of considerable species divergence (Puelles and Medina, 2002). For other proposals on the homology of the nidopallium and the arcopallium, see Karten (1997; 2013), Güntürkün (2005), Butler et al. (2011), Dugas-Ford et al. (2012) and Herold et al. (2018).

The caudal nidopallium and arcopallium in birds, and the equivalent areas in the posterior DVR and the dorsolateral amygdala of reptiles, show patterns of connections that closely resemble those of the mammalian pallial amygdala (reviewed by Medina et al., 2017). They have ample glutamatergic reciprocal connections with many parts of the pallium, and project to a central amygdala-like area in the subpallium (Atoji et al., 2006; Desfilis et al., 2018; Kröner and Güntürkün, 1999; Lanuza et al., 1998; reviewed by Martínez-García et al., 2002; Martínez-García et al., 2007; Medina, 2019; Medina et al., 2017). They are also involved in functions similar to those of the pallial amygdala of mammals, including regulation of fear/stress responses (at least partially by way of projections to the BST; Nagarajan et al., 2014; Saint-Dizier et al., 2009), value-related processing (Dykes et al., 2018) and decision-making (Güntürkün, 2005; Güntürkün and Bugnyar, 2016). Accordingly, we refer to these sauropsid areas as the pallial amygdala-like region (Figure 4).

In sauropsids, the central amygdala projects directly and by way of the BST to areas of the hypothalamus, thalamus, and the brainstem comparable to those of mammals (Martínez-García et al., 2002; Martínez-García et al., 2007; Medina et al., 2017) (Figure 4B, C; Figure 6). As in mammals, the projections to the hypothalamus and brainstem likely function to engage the endocrine, autonomic, and motor systems, while those through the thalamus have the potential to exert influences across the pallium and the subpallium, including the ventral basal ganglia and the extended amygdala (for the connections see Atoji et al., 2006; Atoji and Wild, 2005, 2012; Kröner and Güntürkün, 1999; Veenman et al., 1997).

In birds, the caudal nidopallium (part of the proposed avian pallial amygdala-like region) receives multiple inputs from sensory association areas of the ventral pallium (and other parts of the nidopallium), dorsal pallium (Wulst), dorsolateral pallium (Desfilis et al., 2018; Medina, 2019; including orbitofrontal-like and entorhinal-like areas; Medina et al., 2017) and medial pallium (hippocampal formation), and projects back to these regions (Figure 6). Due to its extensive reciprocal connections with other pallial areas, the caudolateral nidopallium is frequently considered functionally analogous (that is, functionally similar but not homologous) to the prefrontal cortex of mammals (Güntürkün and Bugnyar, 2016). However, the pallial amygdala in mammals is highly associative and participates in multiple frontal and temporal circuits (Swanson and Petrovich, 1998; Whalen and Phelps, 2009). Thus, an alternative possibility that can be entertained is that the caudal nidopallium can be considered functionally similar to the pallial amygdala of mammals.

Importantly, the caudal nidopallium, the pallial amygdala, and the prefrontal cortex are all involved in extensive networks of palliopallial connections, and are high-level association areas important for cognition (Bicks et al., 2015; for the caudal nidopallium and prefrontal cortex, see Güntürkün and Bugnyar, 2016; for the pallial amygdala, see Pessoa, 2008; Whalen and Phelps, 2009). Moreover, these three highly integrative areas of the avian and mammalian pallium are engaged with subpallial areas involved in motivation (ventral basal ganglia) and emotion (extended amygdala), thus intertwining the networks important for cognition with those important for motivation and emotion. Based on its connections with multiple areas of the pallium (Atoji et al., 2006; Kröner and Güntürkün, 1999) and projections to both the ventral basal ganglia and the extended amygdala (Hanics et al., 2017), the avian arcopallium also plays an important role in interlinking contextual, motivational and emotional signals. These remarkable aspects of neural architecture are common to these integrative areas of the avian and mammalian pallium, independent of their status as homologous or not. Accordingly, these might represent minimal requirements needed for the complex computations involved in decision making (in social and non-social contexts), for which motivation and emotion play crucial roles (O’Connell and Hofmann, 2011).

In reptiles, as in birds, a comparable part of a pallial amygdala-like region (posterior DVR) also has a broad range of intratelencephalic connections with multiple sensory and association areas of the ventral and medial pallia, and is an important association center (Lanuza et al., 1998; Martínez-García et al., 2002; Martínez-García et al., 2007; Medina et al., 2017). In mammals, birds, and reptiles, the pallial amygdala projects not only to the subpallial extended amygdala, but also to the ventral and dorsal basal ganglia, septum, and hypothalamus. Because of the paucity of studies in reptiles, the existence of direct projections to comparable centers of the thalamus and brainstem is unclear, but communication can be achieved by way of the ventral basal ganglia and the extended amygdala (Lanuza et al., 1998; Smeets and Medina, 1995). Thus, across mammals, birds, and reptiles, the pallial amygdala gains access to similar functional systems, including thalamopallial circuits via basal forebrain regions (basal ganglia and central extended amygdala) (Figure 4).

To sum up, the connectional system of the pallial amygdala-like area of birds and reptiles spans multiple levels of the neuroaxis allowing it to be involved in multifaceted signaling (sensory, contextual, emotional, motivational). It exhibits exuberant connections with other pallial areas (as in mammals), and additional circuits via the basal ganglia, hypothalamus, and thalamus offer the potential for further signal communication (Figure 4), especially in birds where thalamic projections reach a broad spectrum of pallial areas. Thus, the pallial amygdala in all amniotes, but particularly more so in mammals and birds, is in a pivotal position for integrating multiple signals and having an influence on both emotion/motivation driven behaviors as well as general functions that include attention, learning, and adapting to a changing world (Figure 6).

A pallial amygdala-like area has been identified in the ventral pallium of amphibians and teleost fishes (Medina et al., 2017), where several aspects of the connectivity are reminiscent of the amniote organization, including connectivity with other pallial regions (Moreno and González, 2006). For example, in frogs, the pallial amygdala (called lateral amygdala) is reciprocally connected with several pallial areas including the rostral pallium (perhaps comparable to rostral aspects of the dorsolateral pallium of amniotes; Desfilis et al., 2018), the lateral pallium, and the olfactory bulb (Roth et al., 2007), and projects to the central amygdala and BST (Moreno and González, 2006; Moreno et al., 2012). Both parts of the central extended amygdala project also to the pallium, in addition to the hypothalamus and brainstem (Moreno and González, 2006). Thus, palliopallial and basal ganglia like (pallium-striatum-pallidum) connectivity systems are general principles of the pallial amygdala architecture in vertebrates, although palliopallial connections are less prominent in amphibians compared to amniotes (Figures 3, 4). These connectivity patterns again highlight the extensive crosstalk and association between systems frequently characterized as involving mostly segregated circuits related to motivation/reward, motor control, or emotion.

Evolution and the reorganization of large-scale circuits

In amniotes, both the thalamus and its pallial telencephalic targets are considerably expanded (Butler, 2008; Medina, 2009). Many thalamic projections that mostly reach the subpallium in anamniotes (for example, frogs) also send collaterals to the pallium in amniotes (Martínez-García et al., 2002; Martínez-García et al., 2007; Medina et al., 2017; Moreno and González, 2006). Thus, the overall processing of information can be viewed as shifting from the subpallium to the pallium. Consider the organization of the basal ganglia. As discussed, important components of its architecture are conserved across vertebrates. Yet, substantial differences are observed, too. In reptiles and amphibians, prominent pathways link the basal ganglia with the optic tectum (Marin et al., 1998; Medina and Reiner, 1995; Reiner et al., 1998), while a less extensive system interlinks the basal ganglia and the pallium, mostly via projections from the ventral basal ganglia to the thalamus, which in turn project to the hippocampal formation and pallial amygdala (Medina, 2009). In birds and mammals, basal ganglia-thalamo-pallial loops are expanded and include both ventral and new dorsal components. The considerable development and elaboration of connectional systems involving the cortex/pallium and basal ganglia can be viewed as reflecting, in part, the expansion of the thalamus and pallium in these vertebrate classes (see also Medina, 2009; Medina and Abellán, 2009). Taken together, considerable reorganization is observed across vertebrates. What are the potential advantages for tetrapods, and especially birds and mammals, of having pallium-basal ganglia-thalamus-pallium loops?

First, let us consider potential advantages of having direct pallium-thalamus-pallium loops. Some of their properties are correlated with an increase in pallial size. At least in some mammals, there is a remarkable increase in projections to the thalamus from the isocortex, so that the latter becomes a major driving force for thalamic function. It is unknown whether this is also the case for other pallial inputs, such as those from the pallial amygdala. In addition, it is unknown if the direct pallial inputs to the thalamus outnumber the thalamopallial projection in birds, too, although this may well be the case in avian species with a larger pallium (Charvet and Striedter, 2008). In mammals, there is only rudimentary understanding of the role of the thalamus in functions beyond sensory and motor processing. The contributions of this structure to attentional processes are the most clearly documented (Halassa and Kastner, 2017). In addition, the thalamus may be adept at performing flexible computations that adjust to changing behavioral and contextual demands (Rikhye et al., 2018). More generally, an emerging theme is that a central function of the thalamus involves coordinating the function of cortical networks (Pessoa and Adolphs, 2010), including shifting, coordinating, and sustaining task-related cortical computations (Halassa and Kastner, 2017). It is unknown whether this is so in birds or other sauropsids.

Returning to the basal ganglia, it has been suggested that the complexity of the mammalian cortical-basal ganglia system is a key factor in developing appropriate responses to environmental cues and for adapting behaviors accordingly (Haber, 2014). More generally, loops through the basal ganglia may provide behavioral advantages by supporting the flexible modulation of attention and decision making by motivationally and emotionally relevant variables, as well as the flexible handling and learning associated with both novel events and previously experienced situations.

Now, consider the pallial amygdala. In mammals, the pallial amygdala has substantial bidirectional connections with the dorsal pallium, notably frontal, parietal, and temporal cortices (Amaral et al., 1992), but also with other pallial sectors including the hippocampal formation and perirhinal/entorhinal cortices (Pikkarainen and Pitkänen, 2001; Pikkarainen et al., 1999). In birds, substantial connections are also present between the pallial amygdala and other pallial sectors, including dorsal pallium (Wulst), hippocampal formation, dorsolateral/lateral pallium, and other parts of the ventral pallium. In contrast, the pallial amygdala of reptiles exhibits few connections with the dorsal pallium, but shows prominent bidirectional connections with other pallial areas, including the hippocampal formation, entorhinal-like cortex in the dorsolateral pallium, and other parts of the ventral pallium (Martínez-García et al., 2007). The dorsal pallium in reptiles is relatively small (Desfilis et al., 2018), and the pallial amygdala of ancestral amniotes was possibly the major highly associative center involved in non-spatial complex behaviors (Medina et al., 2017). In all, the pallial amygdala provides an additional example of substantial reorganization of signal communication and integration across vertebrates that likely expanded associated behavioral repertoires.

Further functional implications for the architecture of emotion and cognition

Here, we further discuss several types of changes across vertebrates that have important implications for the functional architecture of emotion and cognition in vertebrates, and mammals in particular. Some of the changes were discussed above but are briefly highlighted again here.

First, connectional systems are considerably more exuberant and complex in amniotes, especially in birds and mammals, compared to anamniotes. Second, a shift of the “center of mass” of integration can be discerned. For example, integration involving the optic tectum, hypothalamus, and striatum occurs in fishes and amphibians, and extended integration across the forebrain, including the thalamus and pallium in sauropsids and mammals. Whereas in mammals the pallium became a major driving force for learning, attention, and decision making, birds and reptiles exhibit two prominent integration systems, one involving the optic tectum (Ewert, 1987a, b; Nieuwenhuys et al., 1998), another involving the telencephalic pallium (especially large in birds, as reviewed here). A third property pertains to the relative segregation of some circuits. For example, in amphibians the basal ganglia influence the optic tectum via robust pallidal pathways (directly and by way of the pretectum) (Marin et al., 1998). In birds and reptiles, pallidal outputs reach the optic tectum via the pretectum, while the direct pathway to the optic tectum is weaker or does not exist in mammals (Medina and Reiner, 1995; Medina and Smeets, 1991; Reiner et al., 1998). A fourth type of change indicates the potential for enhanced integration, which is illustrated by increased convergence of signals in some brain areas (such as the pallial amygdala in birds, reptiles, and mammals), as well as enhanced distribution of signals, which is illustrated by the broad reach of thalamic pathways to the telencephalon in birds and mammals. A fifth change involved the addition of new neuron subtypes in the basal ganglia and in the amygdala in amniotes, which are enrolled in local and/or long-distance connections (discussed by Medina et al., 2017; Medina et al., 2014; Vicario et al., 2015).

What are potential implications of the above properties for emotion and cognition? Vertebrate large-scale connectional systems interlink the brain at multiple levels across the neuroaxis. We propose that these circuits enable brain regions to be sensitive to a broad spectrum of sensory, motor, affective, and motivational signals. Specific regions do not work in isolation but are part of large-scale connectional systems that support a rich repertoire of functions (Pessoa, 2014). The architecture allows for more extensive communication and integration of affective/emotional and motivational signals, so that they can influence and be influenced by a large variety of signals. Insofar as cognition is tied to processes that support behaviors that are relatively insulated from immediate stimuli and responses, these properties are proposed to allow more complex behaviors to be expressed.

In terms of emotion and cognition, whereas the architecture of brain networks in vertebrates supports some segregation of information, the overall organization of large-scale connectional systems fosters a robust amount of integration. In amphibians, this is illustrated by the striatum and striatal-like central amygdala, which are sites of diverse signal convergence; in reptiles, birds, and mammals pallium-subpallium-thalamus circuits involving both the basal ganglia and the amygdala demonstrate the exuberance of information exchange. Thus, the purported segregation of emotion and cognition often emphasized in the literature (e.g., Panksepp, 1998) is not supported by vertebrate neuroanatomy. Furthermore, new data on the divisional organization of the pallium based on genoarchitecture, combined with previous data on its connections in reptiles and other vertebrates, suggest that some of the oldest pallial subdivisions (hippocampus-related medial pallium, entorhinal cortex-related dorsolateral pallium, and amygdala-related ventral pallium) were likely centers of integration and association (Desfilis et al., 2018). Finally, the present framework resonates with the Structural Model of cortical organization (García-Cabezas et al., 2019), which proposes that phylogenetically ancient cortices (including limbic cortices such as the agranular/dysgranular parts of the orbitofrontal cortex, the insular cortex, and the perirhinal cortex) are areas of high integration and plasticity.

The implications of large-scale connectional systems for the organization of emotion and cognition speak to the concept of “conserved” brain regions, circuits, and functions, too. Mechanisms of emotion and motivation, in particular, are frequently described as highly conserved. But how should conservation be interpreted? Returning to the basal ganglia, as noteworthy components of the architecture can be identified across vertebrates, it can be argued legitimately that the basal ganglia are deeply conserved (Grillner and Robertson, 2016). As reviewed above, however, important changes are also observed in the organization of both the basal ganglia and associated structures, including the addition of new cell types and connections. In all likelihood, such changes have important functional ramifications. Consider, for example, adding components to an existing circuit, such as the new looped pathways via the dorsal basal ganglia in birds and mammals (Figure 4), or the extended connectivity between the pallial amygdala and other telencephalic regions. Describing the change as simply “adding a function” to the previous circuit assumes that the system behaves linearly, such that circuit components add or subtract functionality in a straightforward manner. However, the new component probably alters the circuit’s computational profile in a more complex fashion, such as affording the combination of signals that were previously (relatively) segregated. Taken together, invoking the idea of conservation without considering such factors is problematic, and may considerably distort understanding of brain organization and function.

Conclusions

Knowledge of the general organization of vertebrate brains has expanded considerably in the past few decades, and accelerated considerably more recently. The vertebrate neuroanatomy exhibits several large-scale connectional systems (involving the cortex/pallium and thalamus, basal ganglia, and amygdala), with high crosstalk between them, which have important implications for how information flows and is integrated in the brain. These properties inform our understanding of the potential interaction and integration of emotion and motivation with perception, action, and cognition (Pessoa, 2013). In this context, understanding circuit commonalities, due to either “conservation” or “convergence” of structure and/or function, is equally important as unravelling how altered circuits may support extended functional repertoires.

The vertebrate brain allows a considerable amount of communication and integration of signals. This general architecture supports a degree of “computational flexibility” that enables animals to cope successfully with complex and ever-changing environments. Thus, one way to view the more elaborate architecture of birds and mammals, in particular, is in terms of the enhanced potential for combinatorial interactions that they afford, such that the ways different signals can influence each other is considerably expanded. This overall type of architecture may produce circuits with local specificity but relatively large-scale sensitivity, a type of “global within local” design, which likely contributes to more sophisticated and plastic behaviors. Integration is evolutionarily ancient – it is a hallmark of the vertebrate brain – and might be an important property affording flexible behaviors in all vertebrate taxa.

Highlights.

• Integration is a basic features of the vertebrate brain needed to adapt to a changing world

• This property is not restricted to few isolated brain centers, but resides in neuronal networks working together in a context-dependent manner

• In different vertebrates, we identify shared large-scale connectional systems

• There is a high degree of crosstalk and association between these systems at different levels, giving support to the notion that cognition cannot be separated from emotion and motivation

Glossary

Amniotes

Vertebrates with an amnion during the embryonic stage, including mammals, birds and reptiles

Bauplan

the generalized structural body plan that characterizes a group of organisms. In the context of the brain, it refers to a common plan of divisions along the rostrocaudal and dorsoventral axis that is observed across vertebrates

Caudolateral nidopallium

Caudolateral part of the avian ventral pallium. Given its extensive connectivity with pallial areas, it has been suggested to be functionally analogous (but not homologous) to the prefrontal cortex of mammals. An alternative possibility is that it is functionally similar to the pallial amygdala of mammals, as proposed here

Collothalamus

Thalamic nuclei that predominantly receive inputs from the roof of the midbrain, that is, from the tectum/colliculus. See lemnothalamus

Convergent evolution

The process by which the same solution (in terms of form and function) is independently reached (and not inherited from a common ancestor) in response to a similar biological problem. For example, circuits involved in song learning in songbirds and language learning in humans evolved independently, but share commonalities given convergent evolution

Dorsal ventricular ridge (DVR)

Large telencephalic region of reptiles and birds, located laterally and ventrally within the pallium (above the striatum). Its name relates to the fact that in some reptiles, particularly in turtles, it protrudes or bulges into the lateral ventricle. It exhibits a non-laminar organization

Genoarchitecture

Gene expression patterns with respect to the fundamental plan of divisions (Bauplan or morphoplan). In the developing brain, when regulatory genes that play fundamental roles in specification of brain divisions are highly conserved, their expression patterns help identify the same (homologous) divisions across species

Homology

The same character in different animals, independent of its form and function, inherited from a common ancestor. It applies to animal parts or organ (like the brain) sharing the same building plan. Having the same embryonic origin within the building plan is one of the main criteria for identifying homology

Lemnothalamus

Thalamic nuclei that predominantly receive direct projections not involving the midbrain roof (that is, not from the tectum/colliculus); these include retinal, somatosensory, and other afferents. See collothalamus

Morphoplan

see Bauplan

Neocortex (also called isocortex)

Six-layered cortex in the dorsal pallium of mammals

Neuroaxis

Main axis of the central nervous system (roughly vertical in bipedal humans); also spelled “neuraxis.”

Pallial amygdala

Component of the amygdala that is of pallial origin. In mammals, it includes the lateral, basal, and accessory basal nuclei

Pallium

Dorsal division of the telencephalon. It includes several subdivisions that are comparable across vertebrates. In mammals, its dorsal subdivision (dorsal pallium) is considerably expanded and gives rise to the cerebral cortex. The pallium also produces the hippocampal formation, the entorhinal cortex, the claustrum, and the pallial amygdala

Paraventricular thalamic nucleus

A midline nucleus of the thalamus that interlinks the basal ganglia and the amygdalar connectional systems

Prethalamus

Term used at times for the ventral thalamus. It includes the reticular thalamic nucleus of mammals and the equivalent nuclei in non-mammals

Sauropsids

Birds and reptiles

Subpallium

Ventral division of the telencephalon. It includes striatal and pallidal divisions in all vertebrates, which give rise to the striatum (caudate-putamen) and pallidum of the basal ganglia (which includes dorsal and ventral components; e.g., in mammals, the globus pallidus is part of the dorsal pallidum), and parts of the amygdala and septum

Subpallial amygdala

Component of the amygdala that is of subpallial origin. In mammals, it includes the central nucleus and parts of the bed nucleus of the stria terminalis

Taxa (singular, taxon)

A taxonomic unit that applies to organisms inferred to be phylogenetically related and that have characters in common

Tetrapods

Vertebrates with “four legs” that include amphibians, reptiles, birds, and mammals. Vertebrate groups that descend from the common tetrapod ancestor but do not have four legs are also referred to as tetrapods; for example, snakes (which are a subgroup of squamates) and whales (cetaceans)

Thalamus, dorsal (or simply thalamus)

The dorsal thalamus is the main gateway to the telencephalon. Two major subdivisions are found in all vertebrates: collothalamus and lemnothalamus (Butler, 1994a)

Thalamus, ventral (also known as prethalamus)

In the columnar model, the ventral thalamus lies ventral to the dorsal thalamus, but according to the prosomeric model it lies rostral to the thalamus (Puelles and Rubenstein, 2015). It is involved in some sensory processing and with motor control systems

Wulst

Dorsal pallium of birds, also known as hyperpallium