Evolution of brain elaboration

Abstract

Large, complex brains have evolved independently in several lineages of protostomes and deuterostomes. Sensory centres in the brain increase in size and complexity in proportion to the importance of a particular sensory modality, yet often share circuit architecture because of constraints in processing sensory inputs. The selective pressures driving enlargement of higher, integrative brain centres has been more difficult to determine, and may differ across taxa. The capacity for flexible, innovative behaviours, including learning and memory and other cognitive abilities, is commonly observed in animals with large higher brain centres. Other factors, such as social grouping and interaction, appear to be important in a more limited range of taxa, while the importance of spatial learning may be a common feature in insects with large higher brain centres. Despite differences in the exact behaviours under selection, evolutionary increases in brain size tend to derive from common modifications in development and generate common architectural features, even when comparing widely divergent groups such as vertebrates and insects. These similarities may in part be influenced by the deep homology of the brains of all Bilateria, in which shared patterns of developmental gene expression give rise to positionally, and perhaps functionally, homologous domains. Other shared modifications of development appear to be the result of homoplasy, such as the repeated, independent expansion of neuroblast numbers through changes in genes regulating cell division. The common features of large brains in so many groups of animals suggest that given their common ancestry, a limited set of mechanisms exist for increasing structural and functional diversity, resulting in many instances of homoplasy in bilaterian nervous systems.

1. Introduction

The conserved expression patterns of homologous developmental genes have provided a foundation for the hypothesis that the ancestral bilaterian possessed a centralized nervous system. A largely conserved battery of transcription factors and signalling molecules pattern elements of the nervous system in Bilateria (reviewed by [1,2]), while a number of genes involved in neuron differentiation and function are also shared with the Cnidaria [3,4]. By this criterion, the use of a different array of developmental genes in the Ctenophora suggests that their nervous systems evolved independently from those of Cnidaria + Bilateria [5]. In the latter grouping, however, the molecular developmental evidence suggests that neurons arose in a common ancestor of Cnidaria + Bilateria, while mechanisms for patterning subcompartments along the anterior–posterior and dorsal–ventral axes were present in the ancestral bilaterian.

Overlying these conserved underpinnings of animal nervous systems, natural selection on body plans and behaviours has driven tremendous morphological and functional diversification of nervous systems. These selective pressures interact with constraints imposed by the inherited structural and functional framework of the nervous system, constraints owing to developmental processes, and constraints imposed by the processing roles that must be fulfilled by the nervous system [6,7]. When similar selective pressures and constraints are encountered by divergent lineages, convergent and parallel evolution of structures and circuits will often result. Convergence refers to the independent evolution of similar structures from different ancestral structures and genetic mechanisms, while parallelism occurs when similar structures evolve independently from shared ancestral and genetic mechanisms [8]. Convergence and parallelism can be difficult to differentiate, especially when the phylogeny of the species being compared is incomplete or unavailable. Thus, the remainder of this review will refer to convergence and parallelism collectively as ‘homoplasy’.

Homoplasy in sensory systems is often relatively straightforward to link to shared behaviours and environments across species. The olfactory system, as described below, provides an excellent example of architectural and functional homoplasy between distantly related animals, which is associated with lifestyles dependent on the detection and discrimination of odourants. By contrast, complex, integrative higher brain centres such as the mammalian cerebral cortex and the arthropod mushroom bodies are required for many behaviours, and the selective pressures that drive homoplasy in these structures can be much more difficult to ascertain.

Homoplasy is pervasive in animal nervous systems, and can only be detected through comparative studies. Such studies can also reveal the origins of nervous system features, providing insight into their adaptive functions. However, detailed functional studies of neural circuits are most easily carried out in a small number of genetic model species. A comprehensive understanding of nervous system function must incorporate both approaches.

2. Homoplasy in sensory nervous systems

Homoplasy is widespread in animal nervous systems. The concept of deep homology has allowed conserved patterns of gene expression in cellular fields during development to be interpreted as evidence of common ancestry of brain regions [9]. Yet, the evolution of complex neuroarchitectures such as the retinas and eyes of insects, vertebrates and cephalopods is still most plausibly attributed to homoplasy, even if some of the component cell types share patterns of gene expression that support deep homology [10]. Homoplasy of sensory neuroarchitectures may encompass stunning similarities in development, function and morphology. Homoplasy may also be evident when considering how sensory centres have adapted to changing selective pressures in different lineages.

First-order olfactory centres, such as the antennal lobe of insects and the olfactory bulb of vertebrates, provide excellent examples of brain regions with striking morphological and functional similarities that have almost certainly arisen through homoplasy. These olfactory centres are characterized by a glomerular organization, in which each glomerulus processes inputs from olfactory receptor neurons expressing just a single type of olfactory receptor protein. Local interneurons (mostly inhibitory) interconnect these glomeruli, and projection neurons convey the output of glomeruli to higher brain centres [7,11]. Glomeruli allow convergence of many olfactory receptor neurons expressing the same receoptor protein onto a small number of outputs, greatly increasing the signal-to-noise ratio and thus the sensitivity of the olfactory system [12]. An activated glomerulus also represents a specific component of each odourant molecule, owing to the homogeneity of inputs from similarly tuned olfactory receptor neurons [13,14]. The activity of each glomerulus is sharpened relative to that of other glomeruli through lateral inhibition by local interneurons, resulting in a spatial and temporal representation of the molecular features of each odourant [15,16] (reviewed in [17,18]). Perhaps, one or all of these functions make glomeruli particularly adaptive for the challenges of olfactory coding.

Further adaptations of the primary olfactory system also take a similar form across phyla: both insects and vertebrate species that rely heavily on olfaction have larger arrays of functional olfactory receptor genes in the genome [19–21] and larger olfactory bulbs/antennal lobes with more glomeruli [22–25]. Interestingly, these structures do not appear to be capable of adapting to a return to an aquatic lifestyle, as olfactory bulbs and antennal lobes are lost or greatly reduced in secondarily aquatic animals such as cetaceans or whirligig beetles [26,27].

The antennal lobe and olfactory bulb reveal their independent origins at the molecular level. Most strikingly, the olfactory receptor proteins that perform the critical role of binding odourants share little sequence homology between vertebrates and insects (reviewed in [28–30]). Signal transduction within olfactory receptor neurons also involves different intracellular pathways; in vertebrates, olfactory transduction uses canonical G protein-coupled receptor pathways, while in insects, olfactory receptor proteins dimerize with a coreceptor to form ion channels that are ligand- and/or cyclic nucleotide-gated [31,32].

Why the striking morphological homoplasy despite independently evolved odourant binding and signal transduction? The olfactory systems of animals have evolved under shared selection pressures and constraints. Selection for the need to detect and identify airborne odours is constrained by physical properties of the sensory stimulus, and probably by the organization of circuitry that can compute basic parameters of the stimulus (glomeruli).

3. Factors driving homoplasy in higher brain centres

Highly conserved developmental gene expression patterns appear to define the anterior-most segments of the bilaterian nervous system: the forebrain of vertebrates and protocerebrum of arthropods and annelids [2,9,33,34]. The debate over whether this developmental brain segment contains homologous cell populations or even basic circuitry across phyla is ongoing [9,35], although such ‘deep homology’ does not preclude homoplasy of later-arising features, such as higher brain centres contained within these anterior brain segments [8].

Higher brain centres within a lineage such as the vertebrates or the insects share a highly conserved structural and developmental groundplan, suggesting homology within that lineage [36,37]. On top of this lineage-specific groundplan, higher brain centres have clearly evolved additional features contributing to higher processing and complex behavioural output. Among the best-studied evolutionary trajectories in higher brain centres is the increase in neuron number and size of these centres relative to the rest of the brain, which has occurred several times independently in large taxonomic clades like the insects and mammals [38,39].

Changes in size and structural complexity of primary sensory centres are often easy to relate to the behaviour and ecological niche of the animal. For example, increased size and complexity of the olfactory bulb or antennal lobe is associated with increased numbers of olfactory receptor genes expressed in a greater number of olfactory receptor neurons occupying an expanded peripheral olfactory epithelium, all adaptations driven by an increased dependence on olfaction for survival. By contrast, it is very difficult to pinpoint causes of homoplasy in higher brain centres, as they typically receive inputs from multiple sensory stimuli and other brain regions, and participate in many complex behaviours. It is often unclear what behaviours of a particular animal required increased processing capabilities and thus drove elaboration of the higher brain centre necessary for those behaviours. Unlike sensory brain centres, homoplasy in gross morphologies of higher brain centres, such as significant increases or decreases in size relative to the rest of the brain, appears to have occurred as a result of different selective pressures on different groups of animals [38,39]. This difference reflects the fact that the definition of a complex behaviour may be relatively lineage-specific, and multiple such behaviours may have impacted higher brain centre evolution in each lineage.

In mammals, the social brain hypothesis was proposed to explain how large brains, and especially large cerebral cortices, had evolved in primates (reviewed in [40,41]). This hypothesis posits in particular that the social behaviour of anthropoid primates, including humans, was especially cognitively demanding owing to the formation of complex societies requiring interaction and communication between many individuals. However, attempts to extend the social brain hypothesis to other groups of mammals and birds primarily served to emphasize the different types of animal sociality, as correlations were not consistently observed between general social behaviour (as measured by social group size) and brain size in these other animals [42–45]. Outside of the anthropoid primates, social effects on large brain evolution were seen primarily in species that pair bonded. It was proposed that in anthropoid primates, pair bond-like associations are extended to members of the individual's larger, long-lasting social group, leading to a correlation between group size and brain size in this taxon only [43,46,47]; reviewed in [40]. Thus, all types of sociality are not equal in their propensity to drive the evolution of large brains, and by extension, their requirement for complex processing capabilities that would be provided by larger and more complex brains.

Non-social behaviours have also influenced the evolution of large brains in vertebrates, although these behaviours have been even more difficult to define than the nature of sociality. ‘General intelligence’ encompasses a number of behaviours that demonstrate the ability of the animal to engage in flexible, innovative behaviours when confronted with a problem. Innovative behaviours typically involve novel means of food acquisition and are exemplified by the multiple independent instances of British birds learning to open foil-covered milk bottles [48]. Across birds, the highest innovation rates are observed in lineages with the largest forebrains relative to body size, for example, the Corvoidea (crows and jays) [49]. The same association is also observed in primates [50], suggesting that benefits of being able to behave flexibly exert similar selection pressures on birds and primates, resulting in convergent evolution of large forebrains, including the cerebral cortex.

Dietary and habitat variability and complexity have also been associated with the evolution of large brains in vertebrates, sometimes together with social behaviours [42,46,51,52]. Taken together, the selective pressures that promote increased brain and higher brain centre size and complexity are many, interacting, and at this point almost entirely correlational. There is little direct evidence to link novel cognitive abilities to processing capabilities found only in larger brains.

Although hundreds of millions of years diverged from vertebrates, insects also possess integrative higher brain centres called mushroom bodies. Mushroom bodies are found in polychaete annelids, turbellarian platyhelminths, onychophorans, and many, but not all arthropods [53–56]. They are best studied in the insects, where they are likely to have arisen in a common ancestor prior to the divergence of the wingless Zygentoma (silverfish and firebrats) [37,57]. Insect mushroom bodies typically have a single, ovoid sensory input neuropil called the calyx that primarily receives input from olfactory centres, although important exceptions exist and will be discussed below.

Mushroom body size and architecture vary widely across this tremendously diverse and speciose class. Mushroom body size has increased relative to brain size in clades belonging to a handful of divergent lineages, suggesting that these are evolutionarily independent events [38]. There has been a tendency to ascribe the same selective pressures that have influenced the evolution of the vertebrate brain, and especially the primate cortex, to the insect mushroom bodies. The social Hymenoptera, exemplified by ants, bees and wasps but mainly composed of solitary species, may serve as a case study. Social hymenopterans, including the well-studied honeybee (Apis mellifera), possess greatly enlarged mushroom bodies that include duplicated and convoluted calyces that are subdivided functionally by afferent source [58,59] (figure 1). Further functional subdivisions are observed in the lobes, which both receive higher-level afferent input and provide output to other brain regions [62,63]. These elaborate mushroom bodies of social Hymenoptera have for decades been tacitly (and perhaps anthropomorphically) assumed to be adaptations for the sensory and cognitive complexities of eusocial behaviour (reviewed in [64,65]).

Figure 1.

Independent evolution of elaborate mushroom bodies in two lineages of insects, the scarabaeid Coleoptera (scarab beetles) and the Hymenoptera (ants, bees and wasps). Each panel shows a mushroom body in one hemisphere of the brain. Mushroom body intrinsic neurons (Kenyon cells, Kc) and sensory input regions called calyces (Ca) are labelled. Note in particular the expansion and subcomparmentalization of the calyces (asterisks) in the large mushroom bodies in (b) and (d). The ventral (bottom) subcompartment in both cases marks the location of novel visual inputs to the mushroom bodies that are not observed in species with smaller mushroom bodies (a,c). (a) The mushroom body of the feeding specialist scarab beetle Phanaeus vindex (Coleoptera: Scarabaeinae). (b) Mushroom body of the feeding generalist scarab beetle Cotinus mutabilis (Coleoptera: Cetoniinae). (c) Mushroom body of the phytophagous sawfly Dolerus sp. (Hymenoptera: Tenthredinidae). (d) Mushroom body of the parasitoid wasp Ophion sp. (Hymenoptera: Ichneumonidae). Scale bars, (a) 100 µm, (b) 50 µm, (c and d) 20 µm. a and b from [60]; c and d from [61]. (Online version in colour.)

However, studies in other insect species have long suggested that the proposed link between complex sociality and mushroom body size and complexity had at the very least a number of significant exceptions. Cockroaches (Dictyoptera) have mushroom bodies very similar in gross morphology to those of the social Hymenoptera. However, cockroaches are not social and form at most loosely organized aggregates [66,67]. Large mushroom bodies are also found in Odonata (dragonflies and damselflies) [37], Isoptera (termites) [68], butterflies of the genus Heliconius (Lepidoptera) [69], feeding generalist scarab beetles (Coleoptera) [70,71], and parasitoid and parasitic wasps (Hymenoptera), all of which are solitary [61,72–74] (figure 1). With the exception of termites, none of these species demonstrates true colony-based social behaviour.

In feeding generalist scarab beetles and social Hymenoptera, studies of afferent input to the mushroom body calyces revealed a striking feature that had been acquired independently in both groups, but was not found in most other insects: large tracts providing visual inputs directly from the optic lobes to novel subcompartments in the calyces [58,75]. As stated above, mushroom bodies of most insects are small with a single calyx, and the predominant sensory input to the mushroom body calyces is olfactory; this includes scarabaeoid beetles basal to scarab beetles and the basal, phytophagous Hymenoptera [61,71]. Interestingly, the latter study demonstrated that large mushroom bodies with visual inputs were not restricted to social hymenopteran species, but appeared widespread in solitary and parasitoid wasps [61]. Tracts from the optic lobes to the calyces are also been described in dragonflies [76], the cockroach Periplaneta americana [77,78], the butterfly Pieris rapae [79] and the whirligig beetle Deineutus sublineatus [27]. In most of these insects, the mushroom bodies are large with expanded, often duplicated calyces, much like the mushroom bodies of the social Hymenoptera. However, none of these species is social, and all occupy branches of divergent lineages of the insect phylogenetic tree, suggesting that they acquired large mushroom bodies with significant visual input independently [60,80].

A comprehensive comparative study in the Hymenoptera showed that large mushroom bodies are present in the earliest lineage of parasitoid wasps, the Orussidae [61]. Large mushroom bodies were found throughout the Euhymenoptera (all parasitoids + Aculeata, the latter containing the social ants, bees and wasps). Visual inputs to the calyces were also found throughout this group, and quantitative comparisons of mushroom body volume found no difference between parasitoid and social species. Thus, the acquisition of large, elaborate mushroom bodies with novel visual inputs predated the evolution of sociality in the Hymenoptera by approximately 90 Myr, suggesting that sociality arose in hymenopteran ancestors that already possessed these modifications to the mushroom bodies. Interestingly, the same appears to be true for the other major eusocial insect group, the termites (Isoptera). Current phylogenies place the Isoptera within the Dictyoptera [81], and all cockroaches investigated to date possess large, elaborate mushroom bodies [68].

In insects, the evolutionary acquisition of elaborate mushroom bodies is not driven by sociality. What novel or enhanced cognitive abilities are provided by elaborate mushroom bodies, and what selective pressures drove their evolution in multiple independent lineages? The well-studied landmark learning and navigation abilities of the social Hymenoptera, combined with the common acquisition of visual inputs across species, suggest that additional visual processing capabilities, perhaps spatial learning, may be performed by elaborate mushroom bodies. Environments in which this type of visual processing is beneficial, say where food sources are patchy but persistent, provide selective pressure for learning and remembering locations of food sources. In social insects, food sources often fit exactly this type of profile, and individuals must also navigate between food sources and a nest site [82].

The behaviour of insects that are not agriculturally beneficial, detrimental or particularly charismatic is often poorly understood. But spatial learning abilities have been demonstrated in a few non-social hymenopteran species. For example, Heliconius butterflies have enormous mushroom bodies (although visual input has not been studied) [69], return to common roost sites at night, and visit food patches in a predictable order (‘trap-lining’, a behaviour also observed in social Hymenoptera) [83].

Host location strategies that use spatial learning have been demonstrated in the parasitoid wasp Hyposoter horticola [84,85] and the kleptoparasitic wasp Dasymutilla coccineohirta [86]. Other insects with elaborate mushroom bodies receiving visual input, such as dragonflies and whirligig beetles, have preferences for patrolling from or aggregating in fixed locations that might require landmark learning [87,88]. There is incidental evidence that the generalist scarab beetle Popillia japonica learns food source locations and returns to them repeatedly [89]. Finally, outside of the insects in the Chelicerata, the wandering spider Cupiennius salei has mushroom bodies that primarily receive visual input from the optic lobes [90], and the orb-web spider Cyclosa octotuberculata employs spatial learning to monitor more attentively the parts of the web that have previously been successful in capturing prey [91].

Experimentally, only a single study has demonstrated participation of elaborate mushroom bodies in spatial learning tasks. In the cockroach P. americana, mushroom body lesions prevent individuals from learning to quickly navigate a heated maze using remote spatial cues [92]. Interestingly, mushroom body lesions do little to impede simple forms of spatial learning in the fruit fly Drosophila melanogaster [93,94], an insect that may rely predominantly on olfactory rather than visual navigation in its natural environment [93,95]. Although not yet demonstrated experimentally, it is possible that the evolution of elaborate mushroom bodies with direct visual input from the optic lobes facilitated spatial learning using distal landmarks that insects like honeybees are adept at. This also suggests that some aspect of mushroom body circuitry and function made this brain centre a target for repeated, independent evolution of novel computations involving visual inputs.

In vertebrates, the hippocampus performs spatial learning computations and is relatively larger in several lineages of animals that rely heavily on spatial learning [22,96,97]. Although more study will be required before selection for spatial learning ability can be definitively pinned on mushroom body elaboration, the finding that sociality is not a factor in driving mushroom body size and complexity underscores the importance of carefully considering the behavioural ecology of each animal being studied. For example, social behaviour in insects, especially eusociality, is very different from mammalian sociality (with the exception of the single eusocial species, the naked mole rat). Owing to the division of labour in eusocial colonies, individuals typically perform fewer behaviours than do individuals in closely related solitary species [98]. If the evolution of elaborate higher brain centres reflects an increase in behavioural and computational complexity, then insect eusociality would be expected to have the opposite effect on brain size; a recent study suggests that this is indeed the case [99]. So while the ability to perform more complex behaviours in some contexts appears generally to underlie the convergent evolution of large, complex higher brain centres, what constitutes a complex behaviour may differ a great deal in different groups of animals.

4. Homoplasy of morphology and development

Despite the differences in selective pressures driving the elaboration of higher brain centres, they share many gross structural features across taxa, suggesting common constraints in development and circuit organization. Many of these anatomical features have been described in previous reviews [60,75], and their functional roles may in some cases be inferred. For example, both arthropod mushroom body calyces and mammalian cerebral cortex become more convoluted (increased gyrification) as neuron number in these structures increases. Experiments in which cortical cell number is increased in mice demonstrate that gyrification occurs as cortical surface area increases [100,101]. Gyrification benefits the organism by allowing a larger cortex to fit within the skull, as well as facilitating the formation of local (small-world) connectivity that is more rapid and energy efficient than long-distance connections [102–104]. Parcellation, the subdivision of a brain region into functional and structural subdivisions, would also enable the formation of small world circuitry, and is observed in both mushroom bodies and cortex as they increase in size [58,75,105]. Interestingly, despite the precise patterns of gyrification observed in the cortex of a given species, gyri and sulci appear to result from mechanical forces that are especially pronounced as the cortical sheet increases in area relative to thickness [103,106–109]. Again, these aspects of cerebral cortex evolution highlight the interplay between constraints and adaptive benefits. However, although the insect mushroom bodies acquire similar modifications as they increase in size, it is not clear how they benefit processing in the mushroom bodies, nor whether the same developmental constraints that act on a cerebral cortex containing billions of neurons would be at play in an insect mushroom body, which at maximum has a few hundred thousand neurons.

During development, evolutionary increases in both mushroom body and cerebral cortical size are associated with an increased number of stem cells providing neurons for that structure [71,75]. Although this seems like a logical way to increase the number of neurons in a structure, it is not known whether similar mechanisms for regulating stem cell number and proliferation have been employed across distant taxa such as insects and vertebrates. In vertebrates, more progenitors can be produced through increased early divisions of progenitors and later production of a second population of intermediate progenitors that give rise to neurons [110–112]. Rapidly evolving genes in the human lineage that play a role in regulating cell division have been demonstrated to have a profound effect on cortex size when introduced into the developing mouse brain [113,114]. No candidate genes for the increased progenitor number observed in insects with large mushroom bodies have been identified. Those involved in cell division symmetry may be a good starting point, as mutations in these genes in the fruit fly D. melanogaster greatly increase the number of neural progenitors, including those that give rise to the intrinsic neurons of the mushroom bodies [115–118].

5. Conclusion

Molecular developmental evidence suggests that the nervous systems of most animals (Ctenophora excepted) arose from cellular and molecular building blocks present in a common ancestor [1–5]. In 600 Myr of bilaterian radiation, nervous systems have diversified in structure and function. In cases where similar selective pressures are coupled with shared constraints, homoplasy of neural circuitry occurs. However, identifying the particular selective pressures and constraints may be difficult for complex higher brain centres, which have increased in size many times independently, most probably under selective pressure for more complex processing capabilities. Broad, phylogenetically informed comparative studies paired with detailed knowledge of the ecology and behaviour of the species studied are crucial for the identification of aspects of an animal's behavioural ecology that have driven the evolution of functionally and structurally elaborate higher brain centres. Comparative studies also shed light onto the ancestral organization of neural circuits and the developmental and functional constraints that shaped subsequent diversification. Comparisons of nervous systems across broadly divergent animal groups provides deep insight into how neural circuits develop, function and change in evolutionary time, eventually giving rise to new structures and functions in some lineages.

Comparative demonstrations of behaviourally relevant functional capabilities of evolutionarily novel neural circuits are largely absent in the current literature. While the genetic dissection of the developmental, structural and functional features of neural circuits in model systems represents the modern juggernaut in neuroscience research, this wealth of experimental methods for exploring nervous system function is rarely applied in a comparative context (although see [113,114]). As such, model systems studies rarely consider the selective pressures under which nervous systems evolved and function, risking the assignment of functions to neural circuits that have little relationship to the natural behaviour of the animal, or may not be generalizable across species. Fusing both traditional neuroethological and newer model systems studies will allow a comprehensive understanding of the evolution of nervous systems to suit a myriad of functional goals.

Competing interests

I declare I have no competing interests.

Funding

I received no funding for this study.