Evolution of the amniote pallium and the origins of mammalian neocortex

Abstract

Karten's neocortex hypothesis holds that many component cell populations of the sauropsid dorsal ventricular ridge (DVR) are homologous to particular cell populations in layers of auditory and visual tectofugal-recipient neocortex of mammals (i.e., temporal neocortex), as well as to some amygdaloid populations. The claustroamygdalar hypothesis, based on gene expression domains, proposes that mammalian homologues of DVR are found in the claustrum, endopiriform nuclei, and/or pallial amygdala. Because hypotheses of homology need to account for the totality of the evidence, the available data on multiple forebrain features of sauropsids and mammals are reviewed here. While some genetic data are compatible with the claustroamygdalar hypothesis, and developmental (epigenetic) data are indecisive, hodological, morphological, and topographical data favor the neocortex hypothesis and are inconsistent with the claustroamygdalar hypothesis. Detailed studies of gene signaling cascades that establish neuronal cell-type identity in DVR, tectofugal-recipient neocortex, and claustroamygdala will be needed to resolve this debate about the evolution of neocortex.

Introduction

Early comparative anatomists viewed the neocortex, the largest component of the pallium in mammals, as having a de novo evolutionary origin. In contrast, they considered most of the telencephalic regions in sauropsids (birds and reptiles, the latter term used in this paper to refer to nonavian diapsids—turtles, lizards and snakes, the tuatara, and crocodiles) to be an enlarged subpallial area, mostly consisting of basal ganglia (striatopallidum) due to the lack of cortex-like lamination. During the past several decades, a major revision in our thinking about the sauropsid telencephalon and its evolutionary relation to mammalian neocortex has occurred, based on the seminal discovery that a large portion of the sauropsid telencephalon is pallial in nature rather than subpallial. This discovery rested on the observation that the telencephalic areas of intense staining for acetylcholinesterase and dopamine, both subpallial striatal markers, were limited to a much more ventral territory than previously known.^1,2 This finding led to the promulgation of two hypotheses as to the nature of the pallial part of the sauropsid telencephalon.

The first of these hypotheses built on the earlier work of Källén (see Ref. 1 for references for these and other earlier contributions) and was based on Karten's additional discovery of ascending projections from auditory and visual thalamic nuclei to the pallial parts of the telencephalon in birds^1,3,4 and developed further in subsequent papers^5–16 and others discussed below. It now states that the avian “Wulst,” a dorsomedial pallial area, contains neuronal cell populations homologous to those in specific layers of mammalian neocortical areas that receive projections from dorsal thalamic nuclei that do not receive a substantial input from the mid-brain roof. It further states that most of the dorsal ventricular ridge (DVR), called the anterior DVR, which occupies most of the rest of the pallium, contains visual- and auditory-recipient neuronal cell populations homologous to those in specific layers of mammalian association (extrastriate) visual and auditory neocortical areas, respectively. It is referred to here as the neocortex hypothesis. For clarity, however, it should be noted that although this hypothesis compares most of the sauropsid DVR to neocortex, it does not exclude the possibility of some amygdalar homologues in the caudal part of the DVR. Zeier and Karten⁸ proposed homology of parts of the archistriatum (now called arcopallium¹⁷) to the mammalian amygdala, based on their efferent projections to the hypothalamus.

The second, competing hypothesis was proposed about 15 years ago by Bruce and Neary¹⁸ and stated that the entire sauropsid DVR is homologous to the mammalian basolateral amygdala, originating what is now referred to as the claustroamygdalar hypothesis. In its current form, this hypothesis proposes that the DVR is exclusively homologous to the thalamorecipient parts of the mammalian amygdala, that is, the basolateral amygdala,^18,19 endopiriform nucleus, and/or claustrum.^20–23 The claustroamygdalar hypothesis also posits that the dorsomedial part of the pallium, the Wulst of birds and the dorsal cortex of reptiles, is the only sauropsid homologue of all of mammalian neocortex.

For the dorsomedial pallium, homology of the Wulst with primary visual and somatosensory cortices of mammals is well established and noncontroversial.^15,24–26 Thus, some expansion of the medial part of the dorsal pallium likely had already occurred in the common stem amniote ancestor of mammals and sauropsids. The evolution of the sauropsid DVR is more ambiguous.

Both hypotheses agree on the definition of phylogenetic homology—that a homologous character in two taxa is one that has been derived from an antecedent in a common ancestor, in this case the ancestral stem amniotes—and on the criteria for recognizing homology of neural structures in extant animals, which include hodological, morphological, histochemical, topological, developmental (epigenetic), and genetic evidence.^27–32 Both hypotheses thus agree on the underlying assumption that at least some of the cell populations within the DVR of sauropsids and the particular cell populations in the mammalian pallium, to which they are compared, are phylogenetically homologous, but they differ as to which populations. Furthermore, both hypotheses also propose homologies between sauropsid and mammalian thalamic nuclei that project to the pallium but differ as to the correspondences between those that project to the DVR and mammalian pallium. For example, the neocortex hypothesis compares the avian auditory thalamic nucleus, nucleus ovoidalis, with the ventral division of the medial geniculate nucleus of mammals and the avian nucleus rotundus with the tecto-recipient divisions of the mammalian caudal pulvinar. The claustroamygdalar hypothesis, in contrast, compares both nuclei ovoidalis and rotundus to components of the mammalian posterior nuclear group and related nuclei that relay sensory inputs to the lateral nucleus of the amygdala.

Here, we review the evidence in support of the neocortex hypothesis in some detail, because it is supported by a much more extensive body of data than recent papers on the claustroamygdalar hypothesis address. Only Karten's major theoretical position will be considered here. Some other positions that might be considered intermediate^33,34 will not be discussed, as they are not directly germane to the points considered herein. Nevertheless, one of the present authors (Butler) still holds that the anterior DVR is homologous as a field to both the lateral (tectofugal recipient) parts of neocortex and the basolateral amygdala³⁴ (for this hypothesis, Butler and Molnár ³⁴ define field homology as the relationship of the set of derivatives of an embryonic field in one taxon to the set of derivatives of the homologous embryonic field in another taxon, consistent with the definitions of Smith³⁵ and Puelles and Medina³²). The arguments made in this paper support the neocortex hypothesis and the latter, more encompassing field hypothesis as well.

The neocortex hypothesis

Original tenets of the neocortex hypothesis

The neocortex hypothesis^1,7 included two major tenets: first, that the avian pallium (the Wulst and DVR) is dorsal to the region directly comparable to the mammalian basal ganglia, and, second, that it contains numerous cytologically distinct subdivisions that possess cell types and circuitry closely resembling those of mammalian neocortex. The first tenet was based on the use of acetylcholinesterase and dopamine localization to recognize that the extensive telencephalic regions, then called hyperstriatum and neostriatum, in birds are pallial rather than striatal.^1,2

The second tenet, that equivalent pallial cell populations and circuitry (equivalent meaning homologous as per Karten's usage) are present in birds and mammals, arose from Karten's exploration of ascending sensory systems in birds.¹ Recognizing their importance as key features for comparative analysis, Karten began to study these ascending pathways, beginning with the cranial nerves for each. In this manner, the ascending spinal projections (with William Mehler) and subsequently the ascending auditory^3,4,36,37 and visual^5,6,9 pathways to the telencephalon were mapped.

The thalamo-recipient populations within what was then called neostriatum (lower part of the DVR, now called nidopallium) and hyperstriatum (now called either the Wulst or the hyperpallium) were recognized as comparable to the thalamo-recipient neuron types of neocortical layer IV in auditory and visual neocortex. In birds, the auditory and visual tectofugal pathways via nuclei ovoidalis and rotundus, respectively, terminate in cell populations that are located within the neostriatum (nidopallium)—field L for the auditory pathway and ectostriatum (now called entopallium) for the visual pathway—and the visual thalamofugal pathway from the dorsal lateral geniculate nucleus terminates in a cell population located within the visual Wulst (Fig. 1). Hereafter, the new avian nomenclature^15,17 for these pallial regions will be used. Karten¹ suggested the thalamo-recipient sensory cell populations were specifically comparable to the neurons of layer IV of corresponding sensory neocortical areas, calling them “cortical-equivalent structures.” Furthermore, he speculated that there might be a dual origin of those structures in birds— the Wulst and the nidopallium—and their respective cortices in mammals—“striate and somatosensory cortex” (comparable to Wulst) and those temporal cortices that receive tectofugal projections (comparable to nidopallium).

Figure 1.

Photomicrographs of Nissl-stained hemisections through the telencephalon, rostral to caudal as A–C, with mirror-image line drawings. The visual thalamofugal projection from the dorsal lateral geniculate nucleus to the Wulst (hyperpallium) is shown in A (the comparable somatosensory pathway to the Wulst is rostral to the level shown here). The visual tectofugal pathway via nucleus rotundus to the entopallium is shown in A and B, and the auditory tectofugal pathway via nucleus ovoidalis to field L is shown in C. Abbreviations: CM, caudal mesopallium; E, entopallium; Ep, perientopallium; H, hippocampal formation; L1, field L1 pallii; L2, field L2 pallii; L3, field L3 pallii; M, mesopallium; N, nidopallium; Nc, nidopallium caudale; W, Wulst (hyperpallium). The subpallium is indicated by shading.

The neocortex hypothesis thus focuses on comparing the features of neuronal populations within a region rather than on comparing the regions globally as previous workers had done. This approach is based on the view that these neuronal phenotypes are robust traits that are the end product of gene signaling cascades, which can be used to recognize homologous populations when studied in species that allow inferences about the ancestral character states. This notion extends back as far as the earliest histo-logical comparisons of, for example, Purkinje cells, cerebellar granule cells, cranial nerve nuclei, retinal cells, and tectal neurons across vertebrates,^38–40 but it had never been systematically applied to thalamic or telencephalic neurons.

To account for the differing cytoarchitectures of avian and mammalian pallia, Karten¹ proposed a developmental model in mammals that involved tangential migration of the tectofugal-recipient cell populations to the pallial mantle during embryological development to form layer IV of the auditory and visual temporal cortices, versus a more limited local migration of the equivalent populations in birds to form the nidopallial DVR. The following sections review the currently available evidence germane to the neocortex hypothesis for the various recognized criteria for homology, in the order listed above.

Hodological support for the neocortex hypothesis

An important source of support for the neocortex hypothesis has been the identification, in diverse reptilian species, of pathways corresponding to the ascending thalamofugal visual pathway and the tectofugal visual, auditory, and visual-somatosensory systems of birds. These pathways include the thalamofugal visual pathway in turtles^41,42 and lizards;^43,44 the tectofugal visual pathway in turtles,^41,45 lizards,^46,47 snakes,⁴⁸ and crocodiles;⁴⁹ and the tectofugal auditory pathway in turtles,⁴⁵ lizards,^46,50 and crocodiles.⁵¹ Thus, a thalamofugal visual pathway, a tectofugal visual pathway, and a tectofugal auditory pathway appear to have been ancestral for amniotes. Moreover, birds have a thalamofugal, somatosensory pathway via the nucleus dorsalisintermediusventralisanterior(DIVA)tothe somatosensory part of the Wulst, which corresponds to the pathway to mammalian somatosensory cortex.^25,52

These various and separate ascending pathways highlight a crucial issue that Karten recognized but has not frequently commented upon—namely that, like sensory neocortex, the thalamorecipient visual and auditory sensory regions within sauropsid DVR are topographically discrete and unimodal due to their separate hodology. This feature is shared by the ascending sensory pathways to neocortex. In contrast, the sensory projections to the basolateral amygdala (lateral nucleus of the amygdala) do not constitute hodologically discrete, unimodal pathways.⁵³ This difference is in fact fundamental in distinguishing temporal neocortex from the lateral nucleus of the amygdala. The claustrum does not even receive thalamic input.⁵⁴

Studies on intratelencephalic circuitry and telencephalic efferent projections also are consistent with the neocortex hypothesis. For example, rotundo-recipient entopallial neurons project to the perientopallium (entopallial belt),^55,56 which in turn relays to the arcopallium, which then gives rise to descending projections.^11,13 The intratelencephalic projections from entopallium are hodologically comparable to the neocortical projections from layer IV to layers II/III, and the arcopallial population to the cortical efferent neurons of layers V/VI. More recent work has found that the majority of rotundo-recipient entopallial neurons are small, similar to layer IV granule cells in posterior temporal cortex that receive input from caudal pulvinar in squirrels (Fredes, Mpodozis, Marin, and Karten, unpublished observations). Furthermore, a radial columnar organization has been identified that involves both components of the perientopallium (entopallial belt) and part of the mesopallium.⁵⁷

Additional support is provided by the stunning degree of similarity of the morphology and microcircuitry of neurons of the auditory pallium in birds and mammals.¹⁶ In birds, the auditory pallium includes the caudal mesopallium (CM) and three parallel layers of field L (L1, L2a, and L3; Fig. 1). Tonotopic columns, identified by Scheich⁵⁸ using 2-deoxyglucose, are present across these structures, which are aligned in a laminar fashion. Interconnections of neurons between the lamina are strikingly similar to those between the layers of mammalian auditory cortex.¹⁶ The projection of the thalamic-recipient field L2a to the more superficially lying CM and field L1 resembles the neocortical layer IV projection to the supragranular layers, while the CM and field L1 projections back to field L2, as well as to field L3, resemble the supragranular to infragranular neocortical projections. Finally, field L1 pars externus projects upon the arcopallial neurons that project to the auditory thalamus and brain stem in a manner characteristic of layer V/VI neurons of the auditory cortex.^16,59 Some of these impressive similarities, such as the circuitry within field L and perhaps that withinthe entopallium as well, have not yet been demonstrated in reptiles. Detailed studies of intratelencephalic visual and auditory circuits in reptiles are therefore needed to provide evidence of homology. Nonetheless, the “minuteness of the resemblance and multiplicity of similarities”⁶⁰ suggest the possibility that they are at least an instance of parallelism, derived from the same genetic basis, itself inherited from a common amniote ancestor.³¹

One further point that is rarely mentioned is that no ascending pathways that would qualify as homologues of the mammalian visual and auditory tectofugal pathways to lateral neocortex have been identified to any part of the dorsomedial pallium in sauropsids—the Wulst in birds and the dorsal cortex in reptiles. By discounting the cell populations of the DVR as possible homologues of neocortical cell populations, the claustroamygdalar hypothesis fails to offer any explanation of the absence of these pathways in sauropsids or what it implies would be their unique evolutionary origin in mammals.

Overall, the ascending sensory pathways from thalamus to pallium, particularly their prominent hodological feature of unimodality, and the intratelencephalic circuitry both strongly favor the neocortex hypothesis over the claustroamygdalar hypothesis.

Morphological support for the neocortex hypothesis: homologous tectofugal cell populations in birds, reptiles, and mammals

Karten et al.^61–64 have identified highly similar tectofugal neurons possessing large dendritic arbors with distinctive spiny terminal dendritic arbors, called bottlebrush (BB) dendritic endings, in the optic tectum of birds and mammals. These neurons were first described in reptiles and birds by Ramón³⁸ and his brother Ramon y Cajal^39,40 on the basis of their unusual dendritic morphology. In pigeons⁶² and chicks,⁶³ two types of these BB neurons have been distinguished. In pigeons, the type I BB somas lie in the outer, and the type II BB neuron somas lie in the inner sublamina of tectal layer 13 (also called the stratum griseum centrale). The dendrites of the type I neurons ascend and arborize in retinorecipient layer 5b (Fig. 2), while those of the type II neurons arborize in layer 9, which is deep to the layers of direct retinal input. In chicks, the terminal dendritic arbors of the type II neurons lie mostly in layers 8 and 9, again deep to the direct retinal input, while those of type I have their terminal dendritic arbors in layer 5b. These two tectofugal neuron types are also distinguished by their projections to different subregions of nucleus rotundus, with the type I neurons projecting to its anterior and central divisions and the type II to its posterior and triangularis divisions.^61,62 Physiologically, type I neurons are characterized by “chattering,” whereas type II neurons demonstrate sustained responses to depolarization.⁶⁵

Figure 2.

Summary diagram of the type I bottlebrush tectothalamic neuron pathway in mammals (A, based on work in ground squirrels) and birds (B, based on work in pigeons and chickens) from Major et al.⁶⁴ The schematic shows the high similarity in neuronal morphology of these neuron types in mammals and birds. Additionally, the circuitry of this neuron type shows many similarities between birds and mammals, including (1) the inputs from W-type retinal ganglion cells (RGCs) and from cholinergic neurons of the parabigeminal nucleus in mammals and nucleus isthmi parvocellularis (Ipc) in birds; (2) an output to a thalamic cell group projecting to a visual processing region of the temporal telencephalic pallium (caudal/inferior pulvinar in mammals and nucleus rotundus in birds); and (3) an output to a GABAergic pretectal cell group(s) projecting to the thalamic target of the type bottlebrush neurons. Abbreviations: Ach, acetylcholine; GABA, gamma aminobutyric acid; Glut, glutamate; IPS, interstitio-pretecto-subpretectalis; NPA, anterior pretectal nucleus; NPP, posterior pretectal nucleus; NPT, nucleus pretectalis; PBG, parabigeminal nucleus; SP, nucleus subpretectalis; W RGCs, W-type retinal ganglion cells. Used with the kind permission of John Wiley and Sons.

Two types of neurons with the characteristic BB dendritic endings are likewise present in the ground squirrel,⁶⁴ which is a cone-dominated, strongly diurnal species. The somas of type I BB neurons (Fig. 2) lie in the deepest of the three sublayers of the stratum griseum superficiale (SGS₃), and have vertically arrayed dendrites with BB endings within the retinorecipient sublayer SGS₁. Neurons classified as type II lie at a slightly deeper level, some in SGS₃ and some within stratum opticum (SO). Their BB dendritic endings are correspondingly deep compared to those of the type I neurons, lying within upper SGS₂. Whether these neurons receive direct retinal input is unknown. The BB neurons of SGS₃ and SO project bilaterally to the caudal pulvinar in the gray squirrel,⁶⁶ ground squirrel,⁶⁷ degu,⁶⁸ hamster,⁶⁹ and tree shrew,⁷⁰ while in macaque monkeys, they project bilaterally to the caudal subdivision of the inferior pulvinar.^71,72

Neurons with more vertically oriented, non-BB dendritic arbors project ipsilaterally to rostrolateral pulvinar in nonprimates^{66,67,69,70,73–75} and to the lateral pulvinar in primates.^71,76,77 In the absence of information as to morphological distinctions between types, the non-BB tectothalamic neurons and the BB tectothalamic neurons were lumped together in earlier papers. The non-BB, ipsilateral pathway seen in ground squirrels appears to be the major tectothalamic pathway in rats and mice⁷⁸ and cats,⁷⁹ which terminates in the lateral posterior nucleus. Whether birds possess a homologue of this neuron type is uncertain. Hellmann et al.⁸⁰ described an additional layer 13, tectorotundal neuron type in pigeons that projects ipsilaterally to anterior rotundus and appears to be calbindinergic.⁸¹ Calbindinergic neurons have been identified in tectal layers SGS₃ and SO in rats, and found to project to the lateral posterior nucleus.^82,83 It may be that these non-BB tectothalamic neurons of mice, rats, and cats that project to the lateral posterior nucleus correspond to the calbindinergic neurons projecting to anterior rotundus in birds.

In ground squirrels, the thalamic target of the BB tectal neurons, the caudal pulvinar, projects to the granule cells in layer IV of the posterior temporal cortex, Tp⁶⁶ (and Fredes et al., unpublished observations). This projection is clearly similar to the projection from avian nucleus rotundus to the entopallial core.⁵⁷ Also similar to the avian situation are the tectopretectal projections in mammals that arise from BB neurons of the SGS₃ and SO.^67,84–86 In both birds and mammals, one pretectal cell group receiving BB input is GABAergic and projects to the same thalamic target as the BB neurons themselves.^87,88 These studies thus reveal multiple features of cellular similarity in the retinotectothalamopallial systems of birds and those of many mammals.⁶⁴ The existence of these highly similar neuron types across amniotes is compelling evidence for homology of their targets, the sauropsidian nucleus rotundus and mammalian caudal pulvinar divisions of the thalamus.

The morphological features that support homology of the BB neurons across amniotes, including their dendritic morphology and the bilaterality of their thalamic projections, thus support homology of their thalamic targets, nucleus rotundus in sauropsids and the caudal pulvinar in mammals, and, by extension, their telencephalic targets, the entopallium and extrastriate cortex, respectively, and thereby favor the neocortex hypothesis over the claustroamygdalar hypothesis.

Histochemical support for the neocortex hypothesis: homologous thalamic cell populations in birds, reptiles, and mammals

A major disparity between the neocortex hypothesis and the claustroamygdalar hypothesis involves the identification of the dorsal thalamic nuclei relaying the various ascending sensory pathways. The findings of Yamamoto and Reiner⁸¹ strongly support the original neocortex hypothesis interpretation for the auditory and tectofugal-visual thalamic nuclei. The auditory nucleus ovoidalis is rich in parvalbuminergic fibers and poor in limbic system-associated membrane protein (LAMP) or calbindin, a profile similar to that of the ventral division of the me-dial geniculate (MGv) but different from the posterior intralaminar nucleus (PIN), to which the claus-troamygdalar hypothesis compares it. Instead, PIN resembles the region below ovoidalis. For the visual system, nucleus rotundus possesses prominent LAMP and parvalbumin immunoreactivity, which also characterizes the mammalian pulvinar, particularly its tecto-recipient caudal part, whereas LAMP and parvalbumin immunoreactivity are present at a much lower level in the posterior nuclear complex. These findings indicate that the targets of ovoidalis and MGv are likely homologous populations, as are rotundus and the tecto-recipient part(s) of the lateral posterior/pulvinar nucleus.

These histochemical findings support homology of the sauropsid nucleus ovoidalis with the ventral division of the medial geniculate nucleus of mammals and of the sauropsid nucleus rotundus with the tecto-recipient caudal pulvinar and thereby favor the neocortex hypothesis over the claustroamygdalar hypothesis.

Topological support for the neocortex hypothesis

In the early 1990s, Butler^89–91 and Reiner⁹² independently suggested the DVR of reptiles and birds are comparable to the tectofugal-recipient regions of mammalian neocortex (isocortex) and the Wulst of birds and the dorsal cortex/pallial thickening of reptiles are comparable to the thalamofugal-recipient regions of neocortex. Butler⁹⁰ termed the thalamofugal-recipient pallium the lemnopallium (the prefix lemno- referring to sensory projections to the thalamus that, like a straight ribbon [Latin: lemniscus], predominantly bypass the midbrain roof) and the tectofugal-recipient pallium the collopallium (the prefix collo- referring to sensory projections that are relayed to the thalamus via the midbrain roof, called the colliculi in mammals). These two divisions of the nonlimbic pallium and the two corresponding divisions of the dorsal thalamus have been supported by subsequent findings.⁹³ Among amniotes, Butler⁹⁰ also illustrated the topologic similarity between extant sauropsids and mammals in the sequence of the hippocampal complex through the lemnopallial subicular/cingular/prefrontal/somatomotor (medial part of reptilian dorsal cortex–avian somatosensory Wulst) territory to V1 (more lateral part of dorsal cortex–pallial thickening/visual Wulst). This sequence is shown in Figure 3, and the reader is referred to Ref. 90 for a detailed discussion of it. She further showed that this sequence continues through the collopallial territories of extrastriate cortices (visual DVR) to insular multisensory areas (visual-somatosensory DVR region) and then to auditory cortex (auditory DVR; Fig. 3). The sequence then terminates with the neighboring three-layered piriform cortex. Reiner^92,94 also noted these topographic/topologic relationships. Based on and consistent with the neocortex hypothesis, Reiner^15,94 has used the temporal sulcus as a landmark (in mammalian species where present) to homologize the Wulst pallial region to neocortex medial to this sulcus, called superior neocortex, and the DVR to neocortex lateral to it, called temporal neocortex. In mammalian species where the sulcus is not prominent, the respective homologous cortical areas are nonetheless present.

Figure 3.

Topological comparison of the pallial regions of a stem amniote, a generalized extant mammal, and a generalized extant sauropsid, as postulated by Butler.⁹⁰ For this and subsequent figures, the medial pallium, i.e., hippocampal formation, is indicated by light shading, the lemnopallial regions in hatching from upper left to lower right, the collopallial regions in hatching from upper right to lower left, and the piriform cortex in darker shading. The same topological order holds across extant mammalian neocortex and the extant sauropsid dorsal cortex/Wulst and dorsal ventricular ridge sensory-recipient pallial regions. Abbreviations: A, auditory pallium; Ci, cingulate gyrus; Dl, dorsolateral pallium; Dml, lateral part of dorsomedial pallium; Dmm, medial part of dorsomedial pallium; F, frontal neocortex; I, insular cortex; P, piriform cortex; S, subiculum; SM, somatomotor cortex; V1, primary visual cortex; V2, secondary visual cortex. Where abbreviations for mammalian pallial terms, such as cingulate gyrus and insular cortex, are used in the sauropsid diagram, their respective homologues are indicated.

Because the topological order of the various uni-modal sensory areas in the DVR of sauropsids is the same as those in the lateral neocortical areas in mammals, these data favor the neocortex hypothesis over the claustroamygdalar hypothesis. The latter cannot account for the same unimodal, topological order in sauropsids and mammals, as the amygdala does not have any unimodal sensory areas.⁵³

Developmental data on neuronal generation in the pallium: the “proto-DVR” hypothesis and the subventricular zone

Based on a cladistic analysis, Butler⁹⁰ recognized that some expansion of the dorsolateral part of the pallial mantle likely occurred in the tetrapod to stem amniote transition (Fig. 4), accompanied by expansion of the collothalamic projections from their prior predominant projection to the striatum (in amphibians) to predominant projections to the pallium (in stem amniotes). These newly acquired projections to pallium maintained separate terminal zones for the different sensory system pathways as they evolved further in the mammalian and sauropsid lineages. Note that these collothalamic cell groups have maintained their projections to striatum in mammals, birds, and reptiles. In basic agreement, Reiner⁹⁴ also postulated a small formation in the dorsolateral part of the pallium in stem amniotes that he termed a “proto-DVR” (Fig. 4), which in sauropsids elaborated into a definitive DVR and in mammals into the lateral, collothalamic-recipient parts of neocortex. This view of the pallial condition in the common stem amniote is fully consistent with the neocortex hypothesis but leaves open the possibility of convergences and divergences in some DVR versus temporal neocortical neuron types and features.

Figure 4.

Comparison of Butler's,⁹⁰ on the left, and Reiner's,⁹⁴ on the right, postulated pallial components in the common stem amniote ancestor of the synapsid radiation leading to mammals and the diapsid radiation leading to sauropsids. Each figure illustrates a schematized transverse hemisection on the right side, with medial to the left. Reiner coined the term proto-DVR for the pallial region constituting V2 and A in sauropsids. As can be seen in comparing the two diagrams, Butler and Reiner agree that the initial pallial expansion of collothalamic-recipient regions was very small and constituted neither a full-blown DVR as seen in extant sauropsids nor a full-blown neocortex as seen in extant mammals. Abbreviations: A, primary auditory pallium; C, collopallium; H, hippocampal pallium; L, lemnopallium; P, piriform cortex; V1, primary visual pallium; V2, secondary visual pallium.

Based on the cell types present in turtle and other reptile dorsal cortex and DVR, Reiner^92,95 proposed that the neuron populations of the stem amniote common ancestor would have consisted of those present in layers V and VI of neocortex, but lacked those of layers II and III. In this context, it is important to note that turtles, once considered to be of uncertain phylogenetic origin, have now been found, based on molecular data, to be definitively within the reptile clade.^96,97 Reiner suggested that within the separate lines leading to birds from their stem reptile ancestors on the one hand and to mammals from their stem amniote ancestors on the other, neuronal populations characteristic of neo-cortical layers II and III may have arisen independently. The absence of a pallial subventricular zone in reptiles (from which layer II/III neurons derive), but its presence in birds and mammals, is consistent with these interpretations.^98,99 The similarity between mammals and birds in the layers II/III-like neuronal population would thus be an example of parallel evolution. Further studies of reptiles are needed to determine if the layer II–III populations are truly absent and whether the appearance of these neuron types in birds is an avian specialization. Even if this is the case, it does not argue against the homology of the basic collothalamic circuits themselves.

These data on the subventricular zone cannot be used to differentiate between the neocortex hypothesis and the claustroamygdalar hypothesis until more is known about the contribution of the subventricular zone to the basolateral amygdala. Nonetheless, they are consistent with Reiner's proto-DVR model and the neocortex hypothesis.

Developmental data on neuronal migration in the pallium

Karten's original comparative model of pallial development¹ included tangential migration of glutamatergic neurons contributing to the formation of layer IV of lateral neocortex. Subsequent evidence for the radial generation of glutamatergic cortical neurons¹⁰⁰ and the tangential migration of GABAergic neurons from subpallium to cortex¹⁰¹ appeared to be incompatible with this model, as no tangential migration of glutamatergic neurons was observed. Other work, however, as discussed by Karten,¹² has demonstrated some tangential migration of clonally related, glutamatergic neurons.^102–107 Additionally, the putative pallial region in stem amniotes that was the forerunner of DVR and its mammalian correspondent, the temporal neocortex, was likely to have been far simpler than the DVR or neocortex of extant amniotes.^90,92 Thus, differences in neuronal migration between DVR and temporal neocortex are to be expected as the result of some differences in their genetic control. Differences in neuronal migration do not, however, obviate homology, because structures such as the nucleus ovoidalis of birds and the ventral medial geniculate nucleus (MGv) of mammals are clearly homologous by multiple criteria yet differ substantially in their final adult position (with ovoidalis medial and MGv lateral).

The currently available developmental data cannot be used to differentiate between the neocortex hypothesis and the claustroamygdalar hypothesis. More information is needed on the migration patterns of all pallial neuronal populations and their genetic determinants.

Gene expression domains and the neocortex hypothesis

A low level of Emx1 expression in the lateroventral pallial mantle in amphibians, reptiles, birds, and mammals, and the expression of other pallial patterning genes in these same groups, led to the naming of that Emx1-poor area as the ventral pallium.^{21,22,108,109} The low Emx-1 in the ventral pallium has been interpreted as indicative of homology between the mammalian lateral amygdala (and/or other ventral pallial derivatives), the rostral part of the amphibian lateral claustroamygdalar nucleus (terminology of Marín et al.¹¹⁰), and the avian nidopallium (and thalamorecipient DVR in reptiles). The collothalamic nuclei projecting to visual and auditory DVR in reptiles and birds are, however, clearlynotthesameasthosethatprojecttothemammalian lateral amygdala,⁸¹ and the claustrum does not even receive thalamic input.⁵⁴ Likewise, with regardtotheexpressiondomainsofvariouspatterning genes, most are not specific to one pallial domain. For example, in the developing mouse brain,²² the genes Tbr-1 and Sema-5A both are expressed in a decreasing lateral-to-medial gradient in the pallium, and Cad-8 just the opposite, with relatively heavy expression of Tbr-1 and Sema-5A and low expression of Cad-8 in the more lateral pallium that gives rise to the collothalamic-recipient neocortical areas. This also appears true of the nidopallium of birds.^21,111 Thus, a homology of the collothalamic-recipient DVR in sauropsids with the collothalamic-recipient areas of neocortex cannot be excluded by the gene expression patterns across amniotes and is consistent with some expression data. For example, the sauropsid homologues of the collothalamic-recipient neocortical primordium with intense Tbr-1 and Sema-5A expression and low Cad-8 expression may become the collothalamic-recipient DVR in sauropsids, while the gene-expression domains of the lateral and ventral pallial primordia that give rise to the pallial components of the amygdala in mammals may give rise to caudal claustroamygdalar-like regions in birds.¹⁵

Finally, there is danger in relying on single gene expression patterns to make inferences about homology; for example, witness the robust trait of gender and its differing genetic determination in birds versus mammals. Despite some obvious differences, such as the absence of sry as the male determinant in birds, analysis of the gene signaling pathways underlying gender determination highlights the commonalities upon which the evolutionary divergence in the role of sry has occurred.¹¹² So too, reliance on a single trait such as low Emx-1 expression to define part of DVR as the homologue of parts of claustroamygdala is problematic. Emx-1 expression is diagnostic of cortical pyramidal neurons, and its absence in avian nidopallium can be interpreted to identify it as containing neocortical layer IV granule cell homologues. Moreover, other developmentally regulated genes rebut the view that DVR is homologous to parts of claustroamygdala. For example, Lmo4 is expressed at high levels by many cells in the mammalian claustroamygdala and should thus also be enriched in avian DVR by the claustroamygdala hypothesis.^23,113 Yet it is poor in avian DVR, except for a caudolateral pallial territory that, therefore, resembles claustroamygdala.²³ Thus, elucidation of DVR evolution will require identifying the gene signaling cascades that determine neuronal phenotypes in these two structures, and assessing their commonalities and divergences.

Although the gene expression data have been used to strongly argue in favor of the claustroamygdalar hypothesis, as noted here, they are not definitive for it and do not falsify the neocortex hypothesis.

Summary and conclusions

The neocortex hypothesis and the claustroamygdalar hypothesis currently dominate discussions as to the origins of mammalian neocortex and its relation to the evolution of the amniote pallium. While there is currently no consensus as to which is the more “accurate,” our review of the evidence suggests that the neocortex hypothesis is better supported by the comparative data. Developmental evidence of the parallel evolution of the subventricular zone in birds and mammals and of some of the neuronal migration patterns neither favors nor falsifies either hypothesis. The gene expression data have been claimed to strongly support the claustroamygdalar hypothesis, but these are not definitive and do not falsify the neocortex hypothesis. The rest of the evidence favors the neocortex hypothesis including: hodology of thalamopallial unimodal pathways and the intrapallial circuitry, morphological evidence for homology of the BB tectofugal system, histo-chemical evidence for thalamic nuclear homologies, topology of the sensory-recipient pallial areas, and developmental evidence of some tangential migratory behavior of pallial glutamatergic neurons in mammals (as postulated by Karten in his original 1969 paper). Furthermore, these data are inconsistent with the claustroamydgalar hypothesis. In assessing homology, it is important to remember that no one line of evidence can be taken as definitive. All data need to be accounted for. The neocortex hypothesis thus remains a highly plausible scenario of amniote pallial evolution.