Neural mechanisms underlying the evolvability of behaviour

Abstract

The complexity of nervous systems alters the evolvability of behaviour. Complex nervous systems are phylogenetically constrained; nevertheless particular species-specific behaviours have repeatedly evolved, suggesting a predisposition towards those behaviours. Independently evolved behaviours in animals that share a common neural architecture are generally produced by homologous neural structures, homologous neural pathways and even in the case of some invertebrates, homologous identified neurons. Such parallel evolution has been documented in the chromatic sensitivity of visual systems, motor behaviours and complex social behaviours such as pair-bonding. The appearance of homoplasious behaviours produced by homologous neural substrates suggests that there might be features of these nervous systems that favoured the repeated evolution of particular behaviours. Neuromodulation may be one such feature because it allows anatomically defined neural circuitry to be re-purposed. The developmental, genetic and physiological mechanisms that contribute to nervous system complexity may also bias the evolution of behaviour, thereby affecting the evolvability of species-specific behaviour.

1. Introduction

The field of evolutionary development (evo-devo) seeks to explain phylogenetic differences in the form or function of organisms in terms of developmental and genetic processes [1–3]. This has been particularly successful in clarifying the origins of species differences in morphology that can be directly observed. Applying the principles of evo-devo to behaviour is more complicated because behaviour is produced by the nervous system interacting with the body and the environment. Therefore, mechanistic explanations for phylogenetic differences in behaviour must explain how species differences in behaviour are created by nervous systems that are derived from a common ancestor, i.e. what developmental and genetic processes led to the neural mechanisms underlying behaviours seen in the various species?

The nervous systems in major animal phyla, such as vertebrates, arthropods, molluscs and annelids, contain a greater variety of cell types than any other organ in the body. Furthermore, these cells (neurons) form highly specific synaptic interconnections and exhibit temporally dynamic neural activity. Given the complex nature of the nervous system, one might wonder how it would be possible to evolve adaptive behaviour at all; any alteration of a complex system would be likely to produce deleterious results. In fact, large reorganizations or structural transformations have been rare, indicating that the nervous system and behaviour are to a large extent phylogenetically constrained.

Paradoxically, the structure and dynamics of complex nervous systems may facilitate the evolution of particular behaviours, which appear repeatedly in different species within a lineage. The mechanisms underlying the development of nervous system complexity include rules that enable novel structures to be incorporated. Furthermore, neural dynamics allow the generation of multiple activity patterns. Thus, precisely because it is complex, the nervous system exhibits features that allow for and even promote the evolution of certain behaviours. Here, I will argue that such features can be said to affect the evolvability of behaviour, where evolvability is defined as the capacity of a lineage to evolve [4].

It has been asserted that assessing evolvability is critical for a mechanistic understanding of evolutionary phenomena [5,6]. This has been discussed from a genetic point of view [4,7,8], but not often from a macroscopic view. There has been disagreement on whether evolvability itself is a trait that can be selected for because evolvability never benefits the fitness of the individual, it acts at the level of species selection [9–13]. In this article, it will be argued that evolvability of behaviour (both positive and negative) can arise directly from the development and physiology of nervous systems. Thus, regardless of whether it has been selected for, evolvability of behaviour can arise as a secondary consequence of having a complex nervous system.

2. Nervous systems are phylogenetically constrained

In many respects, the gross structures of nervous systems have been strongly conserved within phyla. This has allowed neuroscientists to extrapolate the functions of homologous brain regions across species within a taxon. For example, work on rodent hippocampus informs our understanding of primate hippocampus. Clearly, the shapes and relative sizes of neural structures vary [14,15]; yet, all Gnathostomata (Vertebrata) have forebrain, midbrain and hindbrain and the cerebellum always has Purkinje cells that project to deep cerebellar nuclei [16]. Still, there are differences in the details across taxa. For example, in mammals, the cerebellum has a highly conserved pattern of transverse stripes that subdivide transverse zones, which can be revealed by gene or protein-expression patterns. Microchiropteran bats as a group, however, have an altered expression pattern of these markers, which may have functional significance for echolocation [17].

Gross organizational characteristics of the brain map very well onto phylogenetic trees for mammals [18] as well as insects [19], suggesting that gross morphological characters exhibit phylogenetic constraints and do not account for species-level differences in behaviour. In insects, major areas and pathways are recognizable across members of a taxon such as Diptera [19]. These brain regions are also recognizable across major taxa such as between insects and Crustacea [20] and possibly even across phyla [21–23]. This is not to say that major changes in organization have not occurred; certainly, there are important differences in structure that correspond to functional divergence in the visual system and mushroom body of insects [24–26]. In contrast to gross morphology, there are species differences in microcircuitry that are not readily apparent in the overall connectivity in dipteran nervous systems [27,28]. The extent to which neural circuitry changed during evolution to produce species-specific behaviour in closely related species is still an open question.

It might seem that the complexity of the nervous system would be a constraint on the evolution of behaviour [29]; random changes to a complex system would more probably have a negative impact on system function than be adaptive. This would result in the retention of ancestral neural traits and pathways, thereby decreasing the evolvability of the nervous system [30]. However, the very developmental mechanisms that allow the nervous system to be so complex might also enable it to accept novel inputs. For example, much of the wiring of the vertebrate nervous system self-assembles using simple developmental rules. One such rule is that neurons that tend to be active at the same time will form synapses with each other. Such activity-dependent sorting rules enable the nervous system to develop in a coherent manner even in the presence of a novel set of inputs such as occurs either experimentally or evolutionarily [31–34]. Thus, although these developmental rules play a role in setting up the nervous system, they also might enable the evolution of novel sensory systems.

3. Evolvability of sensory reception

Behavioural responses to conspecifics, predators or food sources depend upon sensory transduction. Species differences in the range or qualities of sensory stimuli that can be transduced could account for species-specific behaviour. For example, in butterflies, the evolution of the ability to detect short-wavelength light is thought to have driven the evolution of blue wing coloration. Moreover, sexual dimorphisms in long-wavelength opsins may have evolved for conspecific recognition [35]. Similarly, it has been suggested that the evolution of trichromatic vision in primates was an adaptation that provided selective advantage for finding food sources [36–39].

There have been recurrent evolutionary gains and losses of chromatic sensitivity in vertebrates as well as in arthropods [40,41]. Changes in wavelength sensitivity have occurred several times through duplication of photopigment genes that allowed diversification of opsins and subsequent amino acid substitutions that shifted their absorbance spectra [42–44]. In insects, long-wavelength photopigments arose independently in various lepidoptera (figure 1a) and hymenoptera clades [45,46]. The same amino acid substitutions repeatedly occurred in the transition to long-wavelength opsin in both groups (figure 1b) [44]. Models of the protein's function show that sites of parallel amino acid substitutions are close to the chromophore [47]. Mutagenesis of this site in bovine opsin alters spectral sensitivity [48]. There may be a limited number of protein configurations that generate long-wavelength sensitivity. Selection may therefore have repeatedly arrived at one of the available configurations that imparts long-wavelength sensitivity. Such structural determinants can even be seen across phyla; independently evolved short-wavelength opsins in both butterflies and primates show similar substitutions in binding sites for 11-cis-retinal that produce a shift in absorbance towards blue [49]. These results provide an example of the twin faces of evolvability; inherent in the nature of the photopigment is the ability to shift the absorbance to long or short wavelengths, but only certain critical amino acid substitutions will suffice. Thus, the same substitutions are observed to have occurred repeatedly over the course of evolution.

Figure 1.

Parallel evolution of long-wavelength photopigments in butterflies and moths. (a) Dendrogram showing parallel evolution of opsins that absorb long-wavelength light. The numbers in parentheses after the species names represent the maximum absorbance wavelength (in nanometres). Both Apodemia mormo (LWRh2) and Papilio xuthus (PxRh3) are significantly red-shifted with respect to other pigments. The shift occurred independently as intermediate pigments show absorbance at shorter wavelengths. (b) Alignment of partial coding sequences for lepidopteran opsins. Identical amino acid substitutions occurred in positions 10, 23 and 29 of transmembrane domain no. 1 (TM1) for Apodemia mormo (LWRh2) and Papilio xuthus (PxRh3) (adapted from Frentiu et al. [44]).

4. Evolvability of sensory circuits

Evolution of additional opsins seems straightforward; however, for those opsins to have an effect on behaviour, the neural circuits would need to respond adaptively. For instance, for trichromatic vision to evolve from dichromatic vision, not only would there need to be an additional photopigment, it would also have to be segregated into different photoreceptors and those photoreceptors would need to form the proper synaptic connections. In the mammalian retina, this presumably would involve the formation of antagonistic centre-surround receptive field properties in retinal ganglion cells.

Experiments in mice suggest that developmental rules and plasticity are sufficient to allow neural circuits to form trichromatic circuits when presented with additional photopigments. Transgenic mice that expressed modified human photopigments showed a remarkable segregation of those opsin genes to different photoreceptors [50]. Subsequent work suggested that there is a developmental mechanism that causes mutually exclusive expression of opsin genes in cones [51]. This leads to chromatic sensitivity in retinal ganglion cells [52]. Adult mice with additional photopigments exhibit a novel ability to discriminate colour [53]. Thus, retinal circuits have mechanisms that seem to fortuitously allow them to process input from additional photopigments, thereby facilitating the evolution of trichromatic from dichromatic vision.

Having a single receptor gene expressed per sensory cell has been thought to be important for the circuits to encode differences in the responses to each of the receptor types. It was asserted that there may be developmental mechanisms present in many sensory systems across phyla, which allow just a single receptor type to be expressed in a primary sensory neuron [54]. Such a pattern could be produced by a process of negative feedback [55]. These mechanisms may be in use for certain olfactory systems as well as visual systems [56]. Although such mechanisms would play a role in normal development, they also provide a means for the evolution of sensory neural circuits that respond to different qualities of the stimuli simply through the process of gene duplication and subsequent sequence divergence. Such simple developmental rules allow a complex system to evolve in a coherent manner and retain its functionality.

Perhaps even more remarkable is the recent observation that the adult nervous system can adapt to the introduction of novel photopigments. In adult squirrel monkeys that were red–green-deficient dichromats, viral addition of another opsin allowed the monkeys to distinguish red and green [57]. In this case, the opsin was not introduced early in development and did not segregate to different photoreceptors. Instead, it was expressed unevenly, so that some photoreceptors expressed a single native opsin and some expressed both the native opsin and the exogenous opsin. Over the course of a few weeks, the monkeys were able to use the information provided by the exogenous opsin for behavioural tasks, revealing that even in the absence of developmental mechanisms, neural circuits exhibit flexibility to novel inputs. It was previously shown in colour-deficient humans that visual experience can modify the perception of colour in adults [58]. This suggests that there is ongoing plasticity that plays a role in the adjustment of neural circuits. Thus, neural circuit plasticity can facilitate the incorporation of novel changes to the transduction apparatus and thereby provide a mechanism for the evolution of novel sensory input.

5. Motor system evolution

Although sensory systems exhibit properties that clearly affect their evolvability, the question has arisen as to whether species differences in motor behaviours are caused by central circuitry at all. It has been argued that species differences in feeding behaviours in fish and amphibians arise not from differences in the outflow of the nervous system but from divergence in the organization of the musculo-skeletal system that the nervous system controls [59–62]. Others have noted though that quantification of the motor output is difficult in the absence of an understanding of the neural circuitry [63]. Nonetheless, clear examples of species differences in the motor patterns underlying feeding behaviour have been observed in reptiles [64], fish [65] and insects [66]. However, it is important to recognize the potential role that sensory feedback plays in shaping motor output [67], making it even more difficult to assess the extent to which the underlying motor output has changed.

Still, important species differences in motor behaviour are caused by differences in motor output. Locomotor behaviour can be species-specific and species differences in locomotion are not always caused by differences in external anatomy. For example, white-tailed deer gallop when alarmed, whereas mule deer stott, i.e. spring up into the air with all four legs leaving the ground simultaneously [68]. Furthermore, the behaviours are genetically determined; hybrids of these two species produce a somewhat intermediate behaviour when startled. Similarly, hybrids between two species of crickets produce calling songs that are distinct from each of the parental lines in the temporal pattern of syllables [69].

The challenge of determining the neural basis of species-specific motor behaviour is complicated by the fact that behaviour is caused by the dynamic interactions of many neurons, making it difficult to understand the neural basis of behaviour at the cellular level even within a single species. For example, although it has been studied intensively for more than a century [70,71], there is still no agreement on the cellular basis for spinal-generated locomotion [72–74]. Thus, in order to understand how motor behaviours evolved, one would need to study nervous systems with clear species differences that are accessible to analysis.

6. Identified homologous neurons aid in the study of motor systems

The difficulties in studying the neural basis of motor behaviour are lessened in some invertebrate nervous systems, in which neural circuits are composed of individually identifiable neurons [75–77]. This allows the activity and synaptic connectivity of particular neurons to be directly related to the behaviour of the animal. Furthermore, just as individual neurons can be uniquely recognized within a species based on a set of characteristics, homologous neurons can be recognized across species based on those same characteristics [77,78], allowing the neural function to be compared across species and related to neural properties and connectivity.

As with gross morphological features, there is substantial phylogenetic conservation of function of homologous neurons across species within a taxon. So many examples of conservation of function exist that, in the absence of evidence to the contrary, a basic assumption when studying related species is that homologous neurons play similar roles. For example, homologous neurons have been found to serve similar roles in the feeding circuitry of gastropod molluscs [79,80]. Similarly, in the well-studied stomatogastric ganglion of decapod crustaceans, the AB neuron is the rhythmic pacemaker for the pyloric central pattern generator (CPG) in the stomatogastric ganglion of lobsters, crabs and spiny lobsters [81].

Despite the strong phylogenetic conservation, there are examples where homologous neurons have changed function. One dramatic example is in the evolution of a novel means of swimming by sand crabs [82]. Over the course of evolution, muscles have been modified in size and orientation and the articulation of the exoskeleton has been altered to allow these animals to swim with their tails up, using their tail fans as flippers [83]. Along with the transformation in morphology, there is the evolutionary emergence of stretch receptors that do not exist in other crustacean lineages [84]. Phylogenetic analysis suggests that these novel primary sensory neurons are actually homologous to motor neurons in other species [85]. In other words, neurons have had their functions converted from motor to sensory, an extraordinary transformation.

7. Biophysical properties and neuromodulation in multi-functional circuits

Work on neural circuits composed of identified neurons has shown that the dynamics of neural circuitry is dependent upon the biophysical properties of the neurons and synapses within the circuits. Therefore, species-specific motor behaviour could, in principle, arise from small differences in the expression of ion channels or other proteins. In this way, the same anatomically defined circuit might exist in different species, but produce different patterns of neural activity. If small changes in biophysical properties underlie species-specific behaviour, then there might be an almost infinite flexibility to the possible behaviours that a neural circuit can produce. However, recent evidence, particularly from the stomatogastric nervous system in crustaceans, suggests that the output of neural circuits is actually impervious to small changes in properties. In fact, there is evidence to suggest that similar behavioural outputs can be produced by neuronal circuits composed of neurons with different ion channel compositions and different synaptic strengths [86–90]. The overall output of the network may be maintained through homeostatic mechanisms allowing different combinations of ion channels and synapses to achieve a similar set point of activity [91–93]. Thus, if species-specific motor behaviours were caused by differences in the biophysical properties of the neurons and synapses, it would probably involve a suite of biophysical differences rather than one or two small differences. This could be good news for studying the evolution of neural circuits because it means that species differences that underlie behaviour are likely to be substantial, rather than subtle. On the other hand, the intra-species variability may make electrophysiological properties poor candidates for characters used in phylogenetic analysis.

The biophysical properties of neurons and synapses within an animal are altered by neuromodulatory inputs to circuits [94–96]. Species-specific behaviour might arise from differences in the activity of these neuromodulatory inputs or in the responses to them [81,97,98]. In the stomatogastric nervous systems of crustaceans, there are species differences in the presence of neurotransmitters and in the effects of neuromodulatory substances [99–101]. In aplysiid molluscs, the effect of serotonin on sensory neuron excitability and synaptic strength varies in a phylogenetic manner and may underlie species differences in behavioural sensitization [102–104]. Species differences in neuromodulatory actions also underlie differences in the swimming behaviours of frog embryos [105,106]. Thus, species-specific behaviour could arise from differences in neuromodulation of cellular and synaptic properties, allowing anatomically defined circuitry to be re-specified for another pattern of activity. Therefore, one might expect to see similar circuitry in closely related animals producing different patterns of activity.

8. Similar behaviours have independently evolved using homologous neurons

The test of whether evolvability is affected by the structure of the nervous system is whether animals independently evolved analogous behaviour using the same structures. This has been examined at the level of individual neurons that produce the swimming behaviour of nudipleura molluscs [107,108]. Based on a phylogenetic analysis, it appears that homologous neurons have independently come to play similar roles in the swimming behaviours of two species: Tritonia diomedea and Pleurobranchaea californica. Both species swim by repeatedly flexing their bodies in the dorsal and ventral directions. In Tritonia, the dorsal swim interneuron (DSI) and interneuron C2 fire bursts of action potentials during the dorsal phase of this behaviour (figure 2a) and are part of the CPG circuit that underlies swimming [110,111] (figure 2b). Pleurobranchaea contains neurons called As1–3 and A1 that are homologous to the Tritonia DSI-A–C and C2 based on anatomical and physiological criteria [109,112]. The neural circuitry for Pleurobranchaea swimming resembles the swim CPG circuit in Tritonia (figure 2c). As1–3 and A1 are rhythmically active during the swim motor pattern in a manner that strongly resembles the neural activity in Tritonia (figure 2d). Thus, homologous neurons play similar roles in the production of swimming behaviour in these two species.

Figure 2.

Homologous identified neurons in sea slugs have divergent or similar roles in behaviour. (a) Tritonia diomedea swims by flexing its body in the dorsal and ventral directions as shown in the diagram to the left. Simultaneous intracellular microelectrode recordings from DSI and C2, two neurons in the central pattern generator (CPG) for the swimming behaviour, display rhythmic bursts of action potentials after a body wall nerve is electrically stimulated (nerve stim.). This comprises the swim motor pattern. (b) The swim CPG in Tritonia contains three neuronal types: DSI, C2 and VSI. There are three DSIs: DSI-A, DSI-B, DSI-C. They are being grouped together for simplicity. The triangles represent excitatory synapses, the circles represent inhibitory synapses and multicomponent synapses are presented by combinations of the two. (c) The swim CPG in Pleurobranchaea has many similarities to that in Tritonia. As1–3 are homologous to the DSIs in Tritonia. There is an As4 that is in the same cell cluster and is in the swim CPG, but is not homologous to the DSIs. Its homologue exists in Tritonia, but the function of this neuron has not been determined in Tritonia. The I_VS neuron has not been identified, but its synaptic actions, which can be inferred from recordings of the other neurons, are similar to those of the Tritonia VSI. A1 (which is homologous to C2 in Tritonia) is strongly electrically coupled to neuron A10 and so both are represented together. Homologues of A3 and A10 have not been identified in Tritonia. (d) Pleurobranchaea californica swims with dorsal–ventral body flexions. Intracellular recordings show that the As2,3 neurons and the A1 neuron both exhibit bursting behaviour during the swim motor pattern (adapted from [109], American Physiological Society, with permission). (e) Melibe leonina swims by flexing its body from side-to-side. Intracellular recordings from CeSP (which is homologous to the DSI in Tritonia) and swim interneuron 1 (Si1) show that the CeSP neuron is not rhythmically active during the swim motor pattern.

Tritonia and Pleurobranchaea belong to a clade called Nudipleura [113–115]. Most Nudipleura species do not swim as Tritonia and Pleurobranchaea do, i.e. using dorsal/ventral body flexions. There are many species that swim with side-to-side or lateral flexions and still others that do not exhibit any swimming behaviour. Plotting these traits on the phylogenetic tree of Nudipleura (figure 3) reveals that one or all three of these behaviours must have arisen independently several times. It is most probable that swimming was lost several times. If dorsal–ventral flexion swimming is ancestral, then lateral flexion swimming must have arisen independently at least three times. A more plausible hypothesis given the phylogenetic distribution of swimming behaviours is that Tritonia and Pleurobranchaea evolved the dorsal–ventral swim behaviour independently [108]. Thus, the underlying structure of the ancestral nervous system that contained the ancestral C2/A1 and DSI/As1–3 cells seems to have been predisposed to the evolution of the neural circuitry to produce dorsal–ventral flexions.

Figure 3.

Phylogeny of the Nudipleura based on both anatomical and molecular data [114,116–118]. The phylogenetic tree shows selected genera and their swimming behaviours. DV (green), dorsal–ventral flexion; L (blue), lateral flexion; N (red), non-swimming.

9. Species-specific behaviours are produced from nervous systems comprisING homologous neurons

Owing to phylogenetic constraints, animals that evolved divergent behaviours would nonetheless have homologous structures. Nudibranchs that do not swim like Tritonia or Pleurobranchaea also have homologues of the swim CPG neurons [119]. For example, Melibe leonina is more closely related to Tritonia than Pleurobranchaea, but swims by flexing its body from side to side instead of dorsally and ventrally [120,121] (figure 3). This behaviour differs in several other fundamental ways from the Tritonia swim, including the duration of the episodes and the stimuli that will elicit the response. Different sets of neurons control swimming in Melibe and Tritonia [108,122]. Nonetheless, Melibe contains a homologue of the Tritonia DSI called CeSP [119], which is not rhythmically active during side-to-side swimming [107] (figure 2e). Thus, these homologous neurons play very different roles in the generation of swimming behaviour.

Even though the DSI homologue is not part of the swim CPG in Melibe, it still affects the swimming behaviour in a way that sheds light on how these different behaviours may have evolved. In all of the Nudipleura, the DSI homologues use the neurotransmitter serotonin. In Tritonia, DSI uses serotonin to modulate the strength of synapses of other neurons in the swim circuit [123–126]. Serotonergic neuromodulation was found to be necessary for producing the swimming behaviour; serotonin receptor antagonists block the swim motor pattern [127]. Also, serotonin [127] or DSI stimulation [128] is sufficient to trigger swimming. In Melibe, CeSP is also serotonergic, but unlike in Tritonia, neither the DSI homologue nor serotonin is necessary for swimming [107]. However, serotonin can still modulate the swim motor programme in Melibe. Thus, the function of homologous serotonergic neurons differs in species with different behaviours; in the dorsal–ventral flexion swimmer, Tritonia, serotonergic neurons play an intrinsic neuromodulatory role, whereas in the lateral flexion swimmer, Melibe, they are extrinsic to the CPG, but modulate its activity.

Most nudibranch species do not swim at all; however, homologues of the DSI have been identified in 10 different genera, including three non-nudibranch opisthobranchs [109,119,129–131]. What are homologues of the so-called dorsal swim interneuron doing in species that do not swim? One answer to that the question is that the DSI in Tritonia is multi-functional; in addition to being part of the swim CPG, it also accelerates crawling when the animal is not swimming [132]. This function is mediated, in part, by specific synaptic connections to efferent neurons. The synaptic connections between the DSI homologues and the efferent neurons are also observed in each of the species where it was examined [119,133]. Furthermore, the DSI homologues also play a role in modulating feeding in Pleurobranchaea [133] and Aplysia [129]. Thus, homologous neurons can maintain one function while taking on additional functions. This suggests that the multi-functionality of neurons allows them to be re-purposed in different species without interfering with their other functions.

10. Parallel and convergent evolution of complex motor behaviours

Homoplasy, such as the independent evolution of swimming in Tritonia and Pleurobranchaea, provides an opportunity to test how neural circuits evolved to produce species-specific motor behaviour [134]. There are many examples of independent evolution of locomotor behaviour. For instance, anatomical and fossil evidence suggests that knuckle-walking in apes evolved independently more than once [135] as did the pacing gait in Camelids [136]. Stick insects re-evolved flight several times [137]. If two species independently evolved a particular behaviour, then the neural mechanisms for those behaviours can be examined to determine if the same mechanisms underlie the behaviour. If the mechanisms arise from non-homologous components, then this is said to be convergent evolution. However, if the behaviours arise from independent evolution using homologous neural structures, then this would be parallel evolution. The presence of parallel evolution is an indication of the extent to which the nervous system provides a pathway for evolution and thus affects evolvability.

Parallel evolution is common for animals that share homologous brain structures. For instance, different clades of monkeys independently evolved dexterity using expansion of similar brain areas [138]. Parallel evolution of brain areas in response to similar environmental pressures has occurred among different clades of shrews [139] and also between marsupials and placental mammals [140]. Reduction of the pretectal area occurred independently in two clades of eel, showing that secondary loss of structures can also be a route for evolutionary change [141].

Vocal learning by birds is another example where homologous brain regions have come to assume similar functions in independently evolved behaviour. Although many species of birds produce vocalizations, only songbirds, hummingbirds and parrots learn their vocalizations. Given the phylogenetic relationships of these birds, it is likely that vocal learning evolved independently at least twice and possibly three times [142–144]. Recent data suggest that the same anterior forebrain areas are used for learning the song in these species. These areas may be generally involved in motor learning. Therefore, the independent evolution of this sensory-motor behaviour may occur through the reuse of homologous brain areas, i.e. parallel evolution of neuronal circuits. Thus, the structure and organization of nervous systems may have provided unique avenues for evolution and biased the direction of evolutionary change, thus affecting the evolvability of behaviour.

The independent evolution of active electrolocation behaviour of weakly electric fish involves both convergent and parallel evolution. The African Mormyriformes and the South American Gymnotiformes independently evolved electrosensory systems consisting of sensory structures (tuberous and ampullary electroreceptors), a motor structure (the electric organ) and neural circuitry to process the information [145–153]. There are many similarities in the behaviours of fish in these two groups. Both groups have species with pulse-like electric organ discharges as well as species with wave-like discharges. Having wave-like discharges requires a jamming avoidance behaviour, which also evolved independently in the two clades [151]. Many of the similarities arose through parallel evolution. For example, both groups of fish have independently come to express the same sodium channel genes in their electric organs and these sodium channels have independently evolved similar changes in the gating region of the protein [152,154]. It has been said that the nervous system was ‘pre-adapted’ for electrolocation and for jamming avoidance responses [151]. This is another way to say that the nervous system affected the evolvability of these behaviours by providing a substrate on which selection could readily act.

However, the independent evolution of electrolocation behaviour also provides a clear example of convergent evolution, where non-homologous structures have come to have analogous properties. In particular, distinct areas of the brains of Mormyriformes and Gymnotiformes are responsible for some of the processing of the electrosensory signal; in the South American fish, timing and amplitude comparisons are made in the midbrain, whereas in the African fish, similar computations are carried out in a medullary structure [151,153]. Thus, both convergent and parallel evolutions have played roles in the independent evolution of electrolocation in South American and African electric fish. It would be of interest to determine if there are additional examples of similar computations being carried out in non-homologous brain regions.

11. Evolvability of neural circuits underlying social behaviour

Nervous system evolvability may play a significant role in shaping the evolution of social behaviour. A classic example of this comes from work on male parental care and pair-bonding, which has been correlated with the expression of arginine-vasopressin 1a (V1a) receptors in particular brain areas in voles [155–159]. Monogamous species such as the prairie vole (Microtus ochrogaster) exhibit high levels of V1a receptor expression in the ventral pallidum, an area that receives input from the nucleus accumbens. Non-monogamous species such as the meadow vole (Microtus pennsylvanicus) exhibit lower levels of expression in this area (figure 4a).

Figure 4.

(a) Comparison of vasopressin 1a (V1a) receptor distribution in the brains of six mammalian species. The species in the left column display monogamous behaviour and those in the right column are non-monogamous. The arrows in monogamous species point to the high level of expression in the ventral pallidum (VP). The boxes show the lack of staining in this region in non-monogamous species. Images provided by Larry Young. (b) A schematic of the reward circuitry, which is common to rodents. Dopamine (green) from the ventral tegmental area (VTA) is released in the prefrontal cortex (PFC) and the nucleus accumbens (NAcc). The NAcc also receives excitation from the periaqueductal grey (PAG) and nucleus tractus solitarus (NTS), which are activated during sex. The NAcc projects to the ventral pallidum (VP), which is the major output relay that helps reinforce motor behaviour. The medial amygdala (MeA), which gets input from the olfactory bulb (OB), projects fibres to the VP that contain vasopressin (magenta). Differences in the level of V1a receptor expression in VP can modulate the reinforcement of mate-related odours (based upon Young & Wang [159] and Young et al. [160]).

It was shown that this receptor distribution was not just correlative, but causal; increasing the expression of V1a receptors in the ventral pallidum of the meadow vole brain, using viral vector gene transfer, transformed the behaviour of the meadow vole on a partner preference task to be more like that of a prairie vole [161]. This suggests that the underlying neural circuitry was already in place to allow the pair-bonding behaviour to occur if the receptor is expressed in this brain area (figure 4b).

Pair-bonding is relatively rare in mammals, occurring in only 3–5% of species. Yet in several species that have independently evolved pair-bonding, the monogamous species exhibit a higher level of V1a receptor expression in the ventral pallidum than closely related non-monogamous species (figure 4a) [162]. This has been observed in other rodents such as the California mouse (Peromyscus californicus) as well as in a primate, the common marmoset (Callithrix jacchus) [162]. The association of V1a receptor expression in the ventral pallidum with monogamous behaviour is still another example of parallel evolution where homologous brain regions have independently undergone the same change to produce analogous behaviours.

The parallel evolution of V1a receptor expression in the ventral pallidum suggests a mechanism for the evolution of male affiliative behaviour that is inherent in the structure of the brain. The ventral pallidum is part of the reward circuitry [163]. Similar circuitry is found in most mammals (figure 4b). Expressing the appropriate receptors along this pathway can cause particular behaviours to be reinforced.

The mechanism for directing the expression of V1a receptors may not be consistent across species. In the vole species, the expression differences can be accounted for by a region upstream of the gene for the V1A receptor; prairie voles have long tandem repeats in this area, whereas meadow voles have shorter repeats [164–166]. Although this gene is polymorphic in primates, the length of the tandem repeat does not appear to correlate with social structure [162,167,168]. This demonstrates the importance of considering the effect of the gene relative to the nervous system; a similar genetic alteration does not necessarily lead to an equivalent phenotype in a different neural environment.

12. Summary

A mechanistic understanding of the evolution of behaviour must take into account the bias that neural structures impose on the evolvability of particular behaviours. The nervous system is quite complex. Phylogenetic and developmental constraints presumably prevent large differences in nervous system structure from arising in closely related species. In spite of this, clear species differences in behaviour exist. Rather than impeding the evolution of behaviour, the developmental, physiological and genetic processes that allow the nervous system to be so complex may also bias the evolution of behaviour towards particular outcomes. This results in independent evolution of behaviour through parallel changes in the nervous system. Parallel evolution suggests that certain nervous system properties are easily achieved and thus can be selected for repeatedly. The nervous system, therefore, plays a role in the evolvability of behaviour by constraining the potential behaviours that can evolve while facilitating the evolution of particular behaviours. The end result is that the structure and physiology of the nervous system helps direct the evolution of behaviour down certain paths that recur over evolutionary time.

The appearance of parallel changes to homologous structures is not the result of random chance or wild coincidence; it indicates a predisposition of the system towards that outcome. Modelling studies have shown that at the molecular level, the probability of parallel nucleotide substitutions under natural selection (i.e. homoplasious nucleotide substitutions that result in equivalent amino acid substitutions) is twice as high as neutral changes [169]. By analogy, at a macroscopic level, there might be certain developmental changes that have a higher probability of yielding functional outputs based on factors such as the constraints imposed by pre-existing neural pathways. This creates the paradox that although the complexity of the nervous system constrains evolution, it also may guide it.

For a complex nervous system to develop and function in a coherent manner, it must be regulated by developmental and homeostatic rules. Homeostatic rules compensate for changes in the environment or in the activity of the brain area. These very rules play a role in the ability of the nervous system to compensate for changes in the periphery of the body, such as the appearance of novel photopigments. Such developmental plasticity assists the evolution of species-specific sensory processing.

Motor networks are capable of producing flexible patterns of activity through the actions of neuromodulatory inputs. Evidence suggests that different species express different behaviours using the same circuit components (such as Tritonia and Melibe). The conservation of the circuitry might allow behaviours to re-appear in other species (such as Tritonia and Pleurobranchaea). Mechanisms that allow for a flexible motor output within a species might also contribute to phylogenetic flexibility.

Complex social behaviours, such as pair-bonding, could independently arise through the exploitation of basic reward circuitry that is conserved in all mammals. Once again, the change is not through gross alterations in connectivity, but rather in the expression pattern of G protein-coupled receptors (in this case V1a receptors). These receptors are likely to change the dynamics of activity in the reward circuitry and thereby change the behaviour. It points again to the importance of neuromodulation of neural circuits in shaping the evolution of behaviour.

The presence of parallel evolution of behaviour through recurrent changes to neural circuits suggests that the nervous system affects the evolvability of behaviour by facilitating certain changes or conversely, by limiting the range of possible functional states. Thus, the evolvability itself, while not necessarily being selected for, results as a natural consequence of having a complex nervous system.