Developmental constraints on behavioural flexibility

Abstract

We suggest that variation in mammalian behavioural flexibility not accounted for by current socioecological models may be explained in part by developmental constraints. From our own work, we provide examples of constraints affecting variation in behavioural flexibility, not only among individuals, but also among species and higher taxonomic units. We first implicate organizational maternal effects of androgens in shaping individual differences in aggressive behaviour emitted by female spotted hyaenas throughout the lifespan. We then compare carnivores and primates with respect to their locomotor and craniofacial adaptations. We inquire whether antagonistic selection pressures on the skull might impose differential functional constraints on evolvability of skulls and brains in these two orders, thus ultimately affecting behavioural flexibility in each group. We suggest that, even when carnivores and primates would theoretically benefit from the same adaptations with respect to behavioural flexibility, carnivores may nevertheless exhibit less behavioural flexibility than primates because of constraints imposed by past adaptations in the morphology of the limbs and skull. Phylogenetic analysis consistent with this idea suggests greater evolutionary lability in relative brain size within families of primates than carnivores. Thus, consideration of developmental constraints may help elucidate variation in mammalian behavioural flexibility.

1. Introduction

One of the most important and intriguing questions in contemporary biology asks how a balance is struck between change and stasis during the evolution of complex phenotypes. Although phenotypes must be mutable enough to respond to selection pressures that vary over evolutionary time, they must also be sufficiently stable to ensure that neither development nor function of the phenotype is disrupted [1]. Evolutionary stasis can be promoted by selection and also by various factors that limit evolvability [2–4], defined as the capacity of a given trait or trait complex to respond to natural selection [5,6]. Developmental constraints are mechanisms or processes that limit the evolutionary response of a trait or trait complex to external selection acting during a focal life stage [1]. Functional constraints, which comprise one common type of developmental constraints [7], are imposed when compromises must be made between competing demands of multiple functions because an individual is not ‘decomposable into independent and separately optimized parts’ [8, p. 591]. Thus, functional constraints represent trade-offs in optimization of form such that evolutionary enhancement of a beneficial trait may be linked by antagonistic pleiotropy to another trait that generates fitness costs [9]. Finally, phylogenetic constraints arise because evolutionary change in a particular lineage can occur only within the functional and genetic context of the ancestors of that lineage [10]. Phylogenetic constraints are expressed as tendencies for closely related species to resemble one another, so they represent patterns rather than processes [11].

Developmental constraints affect evolution largely by biasing or reducing the variation upon which selection can operate, so they provide directionality in evolution at a hierarchical level that is different from adaptation by natural selection [2,10]. Constraints, in effect, help shape the raw materials on which selection acts. It is now widely recognized that adaptations in ancestral forms can produce certain types of patterning in their descendents, and that this patterning may result in processes that actively constrain the pathways available to the descendents. This patterning can lead to specific new adaptive and selective regimes such that subsequent adaptive evolution in response to identical ecological conditions takes very different pathways in different taxonomic groups [10]. Furthermore, selection acts on phenotypes throughout ontogeny, and it is often impossible to separate the developmental generation of phenotypic variation from the action of selection, because developmental constraints themselves can result from selection operating earlier in the history of a lineage, or from selection pressures acting concurrently but in opposing directions [12,13].

Here, we inquire whether constraints might account for some of the variation in behavioural plasticity unexplained by current socioecological models, which are implicitly adaptationist. Behaviours are more reversible in their expression than other aspects of the phenotype, but they are not necessarily more developmentally plastic than morphological or life-history traits [13–16]. Variation in behaviour is mediated by variation in morphological features that include brain anatomy, brain case structure, connectivity among neurons, endocrine glands, hormone receptor distributions and limb structure [16]. The development of neuroendocrine, muscular and skeletal systems may thus limit the possible phenotypic variations expressed by individuals, thereby constraining behavioural flexibility, which we define as the ability to switch readily among alternative strategies in order to solve problems. We use ‘behavioural flexibility’ interchangeably with ‘behavioural plasticity’. It is widely agreed that behavioural flexibility is a good measure of cognitive ability [17,18], that larger relative brain size is correlated with superior cognitive abilities [19,20] and that mammalian fitness in novel environments varies with relative brain size [21]. Thus, enlarged brains appear to enhance the capacities required for individuals to modify their behaviour in potentially adaptive ways [22]. Here, we suggest that consideration of developmental constraints might help us account for variation not explained by current socioecological models of social behaviour and brain evolution.

Shortcomings have recently been noted [23–28] in current models that purport to explain, not only the structure of primate societies and primate social relationships, but also the remarkable behavioural plasticity of primates relative to that seen in other mammals. Current models feature adaptation to social and ecological features of the environment, but they very rarely consider constraint. For example, the socioecological model attempts to explain variation in primate social relationships from an evolutionary perspective, focusing on adaptation to ecological factors that include predation, food distribution and the intensity of within and between-group competition [29–31]. Similarly, the Machiavellian Intelligence or ‘social brain’ model suggests that behavioural flexibility in primates, mediated by enlarged neocortex and sophisticated cognitive abilities, has evolved in adaptive response to social complexity [32,33]. Although these models have a great deal of predictive power, considerable variation in the social behaviour and relative brain size of primates nevertheless remains unexplained. In addition, we are particularly struck by cases in which evolutionary convergence in behavioural plasticity fails to occur among species confronting remarkably similar selection pressures imposed by the physical and social environments, even when theory predicts that convergent traits should enhance fitness. This becomes starkly clear in comparative perspective, and we focus here specifically on mammalian carnivores in comparison with primates.

Similar to primates, many carnivores are large-bodied, large-brained, highly mobile mammals that occupy a vast array of habitat types, and frequently encounter novel environmental conditions during their long lifespans. Furthermore, some carnivores even live in social groups of the same size, structure and complexity as primate groups, and the social bonds among individuals in such groups appear no less important to fitness than they are in primates [34–40]. Despite these similarities, the average behavioural repertoire size (number of behaviours in the ethogram for a species) is much larger in primates than carnivores (mean for carnivores is 71 versus 107 for primates [41]), there is a striking grade shift between them in brain size and size of specific brain regions [42–44], and the mean encephalization quotient in primates is over twice that in carnivores [45]. Frontal cortex hyper-scales relative to the rest of neocortex and the rest of the brain in primates, but not in carnivores [46]. Furthermore, behavioural flexibility differs markedly between these two orders of mammals [24,47]. For example, Roberts et al. [48] attempted to induce both primates and dogs to adopt a novel behaviour that would reveal information about the location of a hidden reward. Both monkeys and apes quickly adopted a new behaviour that allowed them to obtain the critical information, but dogs never did, even after having been shown how to do so by human handlers.

These differences between carnivores and primates are usually ascribed to different selective regimes affecting the two groups. For example, proponents of the social brain hypothesis argue that these taxonomic differences in brain size are owing to unique cognitive demands imposed by primate sociality [43,49]. However, the social lives of certain carnivores are astonishingly similar to those of many primates [34], so these two groups should theoretically have been exposed to the same selection pressures promoting behavioural flexibility and concomitant increases in brain size. Nevertheless, relative brain size in the most gregarious carnivore species is considerably smaller, not only than that of primates living in the same types of societies, but even than that of many solitary carnivores [50,51]. Thus, mammalian carnivores appear not to respond to social selection pressures in the same ways, or to the same extent, that primates do. We recognize, of course, that many other contemporary selection pressures and aspects of general biology differ between carnivores and primates. However, we suggest that myriad taxonomic differences in behavioural flexibility and brain size may be explained in part by developmental constraints, many of which were imposed by selection on ancestral forms, and fitness surfaces shaped by current morphology. Just as morphologists invoke developmental constraints to explain gaping holes in the multi-dimensional space representing mammalian form [11], here we argue that constraints might similarly be able to explain considerable variation in the behavioural plasticity of extant mammals. With examples taken from our own work, we show that many disparate factors can constrain behavioural flexibility during development, and that constraints can potentially be invoked to explain both variation in behavioural plasticity among individuals within a population and variation among species or higher taxonomic units.

2. Constraints on behavioural plasticity among individuals

Until recently, the developmental processes involved in the generation of phenotypic plasticity were never considered as significant sources of micro- and macro-evolutionary change [13,14]. However, it is now widely recognized that epigenetic processes underlie much phenotypic plasticity, and that these processes can have important effects on adaptation and speciation [52]. Epigenetic inheritance occurs when phenotypic variations not stemming from variants in DNA base sequences are transmitted among generations; heritable variation in ecologically relevant traits can be generated through a suite of epigenetic mechanisms, including molecular processes affecting gene expression  [53–56]. Maternal effects represent one of the most widespread forms of epigenetic inheritance known in vertebrates [57,58]. In species with prolonged periods of juvenile dependency on the mother, there may be long-lasting effects of maternal traits on offspring phenotypes [59–62]. A sizeable recent literature has also revealed that maternal effects mediated by prenatal hormone exposure are important for non-genetic inheritance of phenotypic traits, and thus for shaping individual variation in behaviour, including social behaviour (reviewed by [63,64]).

In our own work, we have studied the effects of maternal hormones on offspring aggressive behaviour in the spotted hyaena (Crocuta crocuta). Aggressiveness represents a well-documented dimension of animal personality, one that has important fitness consequences [65–68]. Although consistent individual variation in aggressiveness has been observed among adults of many vertebrate species, little is known about the developmental processes that give rise to this variation. Consistency within individuals in the expression of aggressiveness can emerge as a result of limits to flexibility in the pathways along which behaviour develops. Individual differentiation in behavioural flexibility emerges as a function of underlying variability in the activation of a brain circuitry that includes the prefrontal cortex and its key neurochemical signalling (e.g. dopaminergic and serotonergic) pathways [69]. Variation among individuals in aggressiveness is linked in many species to variation in the serotonergic system and to variation in early exposure to androgens [70,71]. In earlier work [72], we found that both male and female spotted hyaena cubs born to mothers excreting high concentrations of faecal androgens during pregnancy exhibit higher rates of aggression than do cubs born to mothers with lower androgen concentrations (figure 1a). This relationship between maternal faecal androgen concentration and cub aggression was not directly affected by maternal rank, maternal aggressiveness or any other variable explored. Such hormone-mediated maternal effects had never previously been reported in mammals. Our results suggested an organizational mechanism for the development of female aggressiveness in spotted hyaenas, traits presumably mediated by effects of early hormone exposure on structure and function of the developing nervous system. Organizational effects of androgens are known to include modification of morphological features of the developing brain via influencing processes of synaptogenesis and cell death [74–76]. In our earlier work, we were able to report only on the behavioural phenotypes observed in offspring during their first few months of life. Even in studies of maternal hormone effects on oviparous animals, offspring phenotypes are usually only documented shortly after hatching, and in most species, little is known about the persistence of maternal effects beyond the juvenile period [77]. Here, using the same methods as those used by Dloniak et al. [72], we inquired whether hormone-mediated maternal effects on offspring aggressiveness persist into adulthood among female spotted hyaenas.

Figure 1.

Relationship between maternal faecal androgen (fA) concentrations during gestation and rates of aggression [73] emitted by the female offspring produced from those pregnancies when offspring were (a) two to four months old (n = 19) towards their peers at dens and (b) adults (older than 24 months) towards their adult peers (n = 14). Aggressive behaviour here occurred in the context of competition over access to solid food as well as in other contexts. We assessed multiple variables as possible covariates of aggression rates observed among these offspring as both cubs [72] and adults (maternal rank during gestation, R² partial =−0.31, p = 0.31, maternal rates of aggression during gestation, R² partial = 0.05, p = 0.87, and average offspring rank, R² partial =−0.34, p = 0.25), but none were significant.

Male spotted hyaenas usually disperse from their natal groups after puberty [78–81], making it extremely difficult to observe the behaviour during adulthood of sons exposed to known androgen concentrations in utero. However, although we have not yet been able to evaluate maternal effects of early androgen exposure on adult males, we have now been able to follow a number of females into adulthood for which we have hormone data from their mothers during their gestation. From the behaviour of these females, it is now clear that the maternal effects observed in cubs persist into adulthood [73,82]. Specifically, adult females whose mothers have higher androgen concentrations during gestation are considerably more aggressive during adulthood than are adult females exposed prenatally to lower androgen concentrations (figure 1b). Interestingly, although we assessed maternal rank during gestation, maternal aggressiveness during gestation and average offspring rank in adulthood as possible covariates in this analysis, none of these factors were correlated with aggression rates among offspring (statistical results are shown in the legend for figure 1). As Dloniak et al. [72] found earlier in regard to aggressiveness among cubs, only maternal androgen concentrations during pregnancy predicted offspring aggressiveness during adulthood. Thus, a developmental perspective suggests that prenatal hormone exposure constrains the flexibility of aggressive behaviour in individual spotted hyaenas throughout their lives by influencing, to some extent, their baseline aggressiveness. Epigenetic effects of hormones, such as those illustrated in figure 1, are not predicted by socioecological models of mammalian behaviour, which suggest that aggressiveness should be shaped adaptively by resource competition [31]. Among spotted hyaenas, resource competition is almost inevitably more intense for low- than high-ranking individuals, yet the relationship we observed between prenatal androgen exposure and aggressiveness occurs independently of social rank.

Epigenetic effects of prenatal androgen exposure are also known to shape the behaviour of juvenile and adult primates and rodents, although these hormone effects have mainly been investigated in the context of research on the endocrine mediation of sexually dimorphic behaviour [83–85]. Organizational effects of androgens or their metabolites involve alteration of the developing nervous system during specific periods of development, and permanent induction of specific behavioural traits such that males and females act differently, even in identical environments [86]. A developmental perspective can thus shed considerable light on the processes determining the range of phenotypes available to selection [16,87,88]. When developmental processes restrict this range, as with aggressiveness in female spotted hyaenas or sex differences in the behaviour of monkeys and rodents, the evolution of a phenotype is developmentally constrained [89,90].

3. Constraints on behavioural plasticity among species

In addition to effects of hormone exposure on the developing nervous system, other constraints on behavioural flexibility not predicted by socioecological models include functional constraints imposed by the development of other morphological traits. Here, we compare carnivores and primates with respect to their locomotor and craniofacial adaptations, and inquire whether ancestral adaptations may have subsequently constrained behavioural flexibility in one or both groups. We suggest that, even when exposed to identical selective regimes in the present, carnivores do not exhibit the same behavioural flexibility as that seen in primates owing to functional constraints imposed by past adaptations in the morphology of the limbs and skull. Functional constraints are limitations on trait evolution caused by functional interactions among suites of traits [7]. Primates and carnivores last shared a common ancestor over 90 million years ago [91], and selection acted on basal forms in each group to shape them into very different creatures, and thus to set them on divergent evolutionary trajectories. Whereas most primates are adapted to move through arboreal habitat and feed at middle or low positions in trophic cascades, most fissiped carnivores are adapted for terrestrial locomotion and for life at the very top of trophic cascades. We suggest here that selective forces acting on ancestral forms may have limited the extent to which their extant descendents can respond to contemporary selection. In addition, we argue that the opposing functional demands imposed by feeding and housing the brain have been traded off against one another very differently in carnivores and primates, and that these different trade-offs may have resulted in differential evolvability or evolutionary flexibility of both morphology and behaviour in these two orders of mammals.

(a). Locomotor constraints

Primates move via plantigrade locomotion, which means the podials and metatarsals are usually flattened against the substrate. By contrast, most carnivores move via digitigrade locomotion, which involves walking on the toes, with the heel and wrist permanently raised above the substrate. The limbs of digitigrade mammals are proportionally longer, and lighter at their distal ends, than those of plantigrade forms, with much less distal surface area in contact with the substrate, permitting digitigrade mammals to achieve greater speeds during terrestrial locomotion. Limb form affects not only a mammal's range of locomotor behaviour, but also its range of feeding and social behaviours [92], all of which differ strikingly between carnivores and primates. Here, we offer examples in which a single morphological trait favoured in ancestral carnivores to facilitate terrestrial locomotion, namely the presence of paws, has profoundly affected what is possible and what is not possible in the contemporary social interactions among their descendents.

First, because carnivores have paws instead of hands, the richness of their tactile interactions with their environments is quite limited relative to that of primates, and this in turn limits both the complexity of their social interactions and the broader embodiment of their intelligence. Hands permit primates to manipulate things in their environments with much greater precision than can carnivores. Computer scientists and robotic engineers have understood for decades that the embodiment of intelligent machines dramatically affects their ability to adapt and learn via feedback obtained during their interactions with the environment, mediated by sensors and activators [93–96]. Although anthropologists have considered implications of the transition from quadrupedalism to bipedalism for the evolution of intelligence [97], to date biologists and psychologists have paid little attention to the question of how morphological traits other than the nervous system might have affected the evolution of intelligence (but see [98]). We believe this deserves further scrutiny.

The general lack of manual dexterity exhibited by carnivores relative to primates also suggests that mutations that might otherwise generate less-stereotyped and more-flexible behaviour in carnivores may simply be invisible to selective forces in the environment because they cannot be embodied in the limbs. Thus mutations, for example, in nervous system structure or function that might affect fitness in primates via modified use of hands, cannot affect fitness in carnivores, so the fitness landscape for carnivore behaviour is effectively limited by limb morphology. Hands permit many novel forms of social interaction, not the least of which is the elaborate manual allogrooming behaviour typical of many primates. The most elaborate grooming one carnivore can give to another is a vigorous licking, and this rarely occurs outside of mother–infant pairs. Even the raccoon (Procyon lotor), which is the mammalian carnivore with the greatest manual dexterity [99], grooms with its mouth rather than its paws. The coherence of social relationships and social groups is maintained in non-human primates by frequent and prolonged bouts of manual allogrooming, and individuals often groom each other strategically to gain future benefits, including support during coalition formation [100]. Although licking can function to maintain sexual pair bonds and placate dominant animals in some carnivore societies [38,101], carnivores appear to use allogrooming as a facultative response to antagonism rather than as a pre-emptive strategy to avert it by establishing a network of associations, as evidently occurs among primates [100]. Thus, there appear to be emergent properties in primate social relationships that arise from the extreme manual dexterity characteristic of these animals; such properties cannot emerge in carnivores.

Finally, the lack of hands may impede the evolution of culture in carnivore societies. Here, we define ‘culture’ very simply as behaviour patterns observed in multiple group members that are transmitted among individuals and across generations via processes of social learning. Van Schaik [102,103] has proposed a ‘cultural intelligence’ hypothesis, which suggests that innovative solutions to social and ecological problems are effectively made heritable in many primates by social learning. The cultural intelligence hypothesis predicts that cognitive abilities should reflect exposure of offspring to tolerant role models for social learning. Tolerance by role models is likely to be more constrained in carnivores than primates owing to resource competition, but we suggest that paws may further limit exposure to such role models among carnivores. Like most primates, most carnivores produce highly altricial young. Whereas carnivores keep their young at dens or crèches, most primates can keep their offspring with them at all times because hands permit young to cling to their mother's fur. Hands thereby vastly increase opportunities for social learning throughout ontogeny because, from birth onward, young primates can observe or participate in their mothers' interactions with their physical and social environments. The early social environment may have an effect on the capacity for social learning, if not on the socially learned behaviours themselves, in several different taxonomic groups [104,105]. Although some non-primate mammals (e.g. cetaceans, elephants) manage cultural transmission without hands, their young are precocial and they are always in close proximity to their mothers and other conspecifics, communicating via an elaborate repertoire of signals, so they nevertheless have countless opportunities for social learning. The inability of young carnivores to travel constantly with their mothers may limit their opportunities for social learning, and consequently also for the evolution of culture and intelligence [102,103,106]. Selection operates when animals interact with their environments, so the broader the array of interactions permitted by limb structure, the more variable the extended phenotype [107] presented to the selective forces in its environment, and hence the greater the evolvability of the animal's behavioural flexibility. We hypothesize that the evolution of hands in ancestral primates greatly expanded the possibilities available to their descendents in regard to behavioural flexibility, but that such possibilities were never available to carnivores or many other groups of mammals.

(b). Craniofacial constraints

The skull is a non-technical term that refers to the brain case and the facial skeleton, which includes the snout and jaws [108]. The mammalian skull is composed of a surprisingly large number of bones, some of which fuse during development. The largest portion of the jaw-closing musculature originates on the sides or top of the brain case, passing ventrally to insert along the lower jaw. Attachment of muscle to bones is more effective on edges and ridges than on smooth surfaces for coping with the large stresses on the bones of the skull that may be imposed by certain styles of feeding [108]. Because carnivores generally need relatively massive jaw muscles for efficient capture and consumption of their prey, their skulls have many more edges and ridges than do those of primates, which are often relatively rounded and smooth. Furthermore, the sutures joining the bones comprising the cranial vault typically fuse relatively early during development in carnivores, and the sutures often close so tightly that it becomes impossible to determine visually where one bone ends and another begins.

The skull functions concurrently as both a protective housing for the brain and sense organs, and a platform for the feeding apparatus. Brain case expansion is required to house a larger brain, but that may be incompatible with elaboration of the feeding apparatus and protection of the brain case from stresses imposed by certain styles of feeding. The crania of ancestral carnivores were optimized to resist higher stresses than were those of ancestral primates. Within the temporal region of the skull, cross-sectional area places limits on the maximal force that can be generated by muscle [109], and expansion of brain volume impinges on available muscle area within the zygomatic arches. Therefore, within a skull of given length and width, greater brain size impinges on maximal bite force [110].

In the tradeoff between the dual functions of the skull, we suggest that the demands of feeding and prey capture in ancestral carnivores may have tipped the balance in favour of the feeding apparatus, and that the two conflicting functions of the skull trade off against one another more antagonistically in carnivores than in primates, even in extant forms. We further hypothesize that these functional constraints in carnivores have affected the evolvability of their skulls, and ultimately of their brains as well. If indeed the evolutionary lability of carnivore brains has been constrained relative to that in primates, then we should find evidence of more evolutionary change, indicated by greater variation in relative brain size, in primates than carnivores, and we should observe stronger phylogenetic autocorrelation in relative brain size in carnivores than primates. In addition, if functional constraints in the skull have affected brain evolution, then we should find tighter evolutionary associations between skull traits and brain traits in carnivores than primates. Our recent work revealed that, in regard to relative brain size and relative size of particular brain areas, the strongest autocorrelations among carnivores were observed within families [51]. Therefore, we focus here on comparisons among species within families, among families within suborders, and between the primate and carnivore orders.

(i). Material and methods

We collected data from three sources: skull measurements, body mass and endocranial volumes for Carnivora were drawn from Swanson et al. [51]; skull measures for Primates were drawn from Takahashi et al. [111] and body and brain mass data for Primates from Boddy et al. [45,112]. Endocranial volumes in carnivores were multiplied by 1.036 to render the estimates comparable with brain mass [113,114]. We constructed phylogenies for primates and carnivores using information from Bininda-Emonds et al. [115], using their estimated branch lengths for all analyses. Skull size was estimated as the first axis of a phylogenetically corrected principal component analysis [116] on zygomatic arch breadth and skull length (condylobasal length for carnivores and maximum length for primates). Relative brain mass was calculated by regressing log-transformed brain size on log-transformed body mass using phylogenetically corrected generalized least-squares models [117–119]. This method of correcting for phylogenetic relationships estimates a parameter ‘lambda’, for which a value of zero suggests no phylogenetic autocorrelation and unity suggests Brownian motion. We allowed lambda to take its maximum-likelihood value in each case. We then used residuals from the model as relative brain sizes. We performed our analyses in R v. 2.14.1 [120]. To test for phylogenetic autocorrelation at different taxonomic levels, we estimated Moran's I [121,122], both among species within families and among families within suborders, using the ‘ape’ package [123] in R.

(ii). Results

Figure 2 directly compares relative brain size among families within each order, and it also directly compares the Primates and Carnivora orders themselves. The variation observed in relative brain size among and within primate families greatly exceeds that seen among or within carnivore families, suggesting that evolutionary change in relative brain size has been more rapid within primate than carnivore families (figure 2). Our analysis of phylogenetic autocorrelation using Moran's I supports this hypothesis (see figure 3 and electronic supplementary material, tables S1 and S2), indicating that relative brain mass evolves more slowly in carnivores than in primates. Different carnivore families in the same suborder are uncorrelated, whereas primate families in the same suborder exhibit relative brain masses that are more similar to one another; the high correlation among primate families in the same suborder may be driven in part by a grade shift towards increased encephalization among the Haplorrhini [114]. Interestingly, within families, trends for overall size are reversed in the two orders, with both body mass and skull size evolving more rapidly among carnivores than primates. Among primate species in the same family, relative brain size exhibits lower autocorrelation than mass or skull size; the opposite trend holds in carnivores, suggesting that relative brain size evolution at the family level among primates is much less constrained relative to general body size evolution than it is among carnivores. Figure 3 also reveals that the autocorrelations for skull size and body mass in primates are virtually identical, whereas in carnivores the autocorrelation for skull size falls between those for mass and relative brain size. Taken together these results suggest that the rate of skull size evolution is closely tied to that of body mass, but not to that of relative brain size, in primates; by contrast skull size is linked evolutionarily to both body size and relative brain size in carnivores.

Figure 2.

Phylogeny for seven families in the order Carnivora and 10 families in the order Primates taken from Bininda-Emonds et al. [115]. Horizontal box plots display relative brain mass corrected for body mass using phylogenetic regression. Branch lengths on phylogenies are not shown to scale, but the relative brain mass values for the two orders are scaled to be comparable. Sample sizes, representing species within each family, are given in parentheses following each family name. Boxes indicate interquartile range, and whiskers spread to the furthest points outside the interquartile range, but within 1.5 times the interquartile range from the median.

Figure 3.

Correlograms showing values of Moran's I for morphological traits at two taxonomic levels in Carnivora and Primates. Hollow shapes denote values of Moran's I significantly different from zero. Filled points denote failure to demonstrate a statistically significant difference from zero. ‘Within family’ autocorrelation designates the autocorrelation among species within the same family, whereas ‘among family/within suborder’ designates the autocorrelation among families within suborders. Families and sample sizes used for this analysis correspond to those shown in figure 2. Suborders used for Carnivora included Feliformia and Caniformia. Suborders used for Primates included Haplorrhini and Strepsirrhini. Exact values for Moran's I and p-values are given in the electronic supplementary material, table S1 (primates) and table S2 (carnivores).

Our results indicate that skull and brain evolution are linked in carnivores, independently of changes in body mass, in ways that they are not in primates. This is consistent with the notion that the dual role of the skull as both brain case and feeding apparatus constrains brain evolution in carnivores far more strongly than in primates. As relative brain volume has been linked to diet in both orders [51,124], it is also likely that diet influences the relationship between skull and brain evolution. However, it is clear that skull and brain evolution appear more tightly linked evolutionarily in carnivores than in primates.

Observations consistent with our hypothesis have also been made by other researchers, including those studying integration within and among modular complexes of traits in the mammalian skull. Here, we refer to integration, not between brain and skull, but rather among parts of the skull itself. Cheverud [125,126] suggested several years ago that weak integration of the cranial vault in primates might allow for the great expansion of the brain during primate evolution. Subsequent workers found that the primate cranial vault and other skull modules are indeed significantly less integrated than the same modules in carnivores [127,128]. Greater integration limits variation and reduces evolvability in mammalian skulls [127]. Although cranial integration does not appear to be correlated with phylogeny in the primate groups in which this has been studied [129,130], this correlation is observed in carnivores [128]. In addition, encephalization in some carnivoran groups is related to measures of cranial shape [131]. Most importantly, the greater interdependence among skull modules in carnivores results in considerably less evolutionary flexibility in this group than in primates [6,132].

Encephalization patterns vary among mammalian groups during the course of evolution, and ancestral reconstructions have revealed evidence for both increases and decreases in brain size throughout evolutionary history [44,45,133]. In addition to an increase in relative brain mass, increased variance characterizes the most highly encephalized mammalian lineages, suggesting relaxed phylogenetic constraints on brain and body mass coevolution in these taxa; whereas primates show this greater variance in encephalization, mammalian carnivores do not [45]. All these findings are consistent with the notion that the evolutionary lability of brains varies among groups of mammals due, at least in part, to variation in the strength of antagonistic selection pressures acting on the skull to perform its multiple functions.

For adaptation to occur in complex systems, they must be evolvable, and evolvability critically depends on how genetic variation maps onto phenotypic variation, as this mapping determines the propensity of traits to vary [4]. Selection for functional integration among traits can lead to the evolution of genetic integration among these traits [125,134–136], and this further limits independent evolution of behavioural traits [16]. Evolvability is improved by limiting the interference between adaptation for different functions, largely because such interference is often mediated by pleiotropic effects among characters serving different functions [4]. Deconstraining properties minimize lethal damage caused by mutation, and reduce the number of mutational changes needed for phenotypic novelty [137]. We suggest that the relatively low levels of integration within and between modules in the primate skull may have effectively set the stage for selection to act on variation induced by mutations that ultimately shaped the uniquely large brains found in recent hominoids. As an example, we briefly consider patterns of skull suture closure during ontogeny.

There has been positive selection on the modern human variant of the RUNX2 gene [138], which affects fusion of the metopic suture (MS) in the primate skull. In the human foetus and young child, the confluence of MS with the coronal and sagittal sutures is the anterior fontanelle, or ‘soft spot’ in a baby's cranium. Fusion of MS occurs at 2–4 years of age in humans, whereas the fontanelle closes by 2 years of age [139]. Fusion of MS, which begins near the nose and progresses towards the fontanelle, occurs much later in humans than in great apes, and partially or unfused MS are common in humans but very rare in apes [140]. Skull sutures also fuse much earlier in carnivores than in primates, and the time elapsed between birth and sutural closure is proportional to brain size in these groups [141]. As the cranium forms around the developing brain, premature closure of MS causes atypical cranial development [142]. The delayed fusion of the MS in hominoids, especially humans, was evidently favoured by natural selection because it allowed rapid brain growth in late gestation and infancy as well as reorganization of frontal cortex [140]. Interestingly, selection shaping suture closure patterns apparently acted early in the histories of extant primate groups [143], and although some extant primates have skulls specialized for the generation of large bite forces (e.g. gorillas), their skulls represent a derived condition from ancestral forms that already possessed large brains and delayed suture closure [144], albeit not so far delayed as in humans. Thus, rather radical skull modification can clearly occur after brain expansion, but the order in which these changes occur may be critical. Perhaps brain expansion after certain types of skull modification becomes considerably less likely during the course of evolution. In any case, all of the observations noted above are consistent with the idea that reduced integration within the primate skull set the stage for generation of developmental patterns that are associated with the evolution of enormous brains in some modern forms.

4. Conclusions

Just as a developmental approach helps morphologists explain lacunae in the morphospace of animal form, a developmental approach may help behavioural ecologists to account for variations in social behaviour and cognitive abilities not explained by socioecological models. Our examples suggest that developmental constraints may have important effects on behavioural flexibility, and that these effects are reflected in variation among both individuals and species. Hormone-mediated maternal effects on behaviour have now been documented in a wide array of oviparous and viviparous animals [85,145–147]. The selective forces shaping basal primates and carnivores generated both the phylogenetic patterns and the functional constraints we observe in modern forms, and may cause them to be differentially refractory to the same selection pressures applied in the present, even when similar responsiveness to selection with respect to brain size and behavioural flexibility should theoretically enhance fitness similarly in both groups. Although we have focused here on mammalian carnivores, constraints probably play equally important roles in shaping behavioural flexibility and brain size in other taxonomic groups. For example, relative brain size in bats is correlated with relative wing area, suggesting that these traits co-evolved, and that brain size has been reduced in some forms to reduce weight, minimize energetic costs and improve aerodynamic performance during flight [133]. Although earlier workers recognized that small body size limits brain size [19,103], developmental constraints can also have important and widespread consequences in regard to the evolution of brains and behaviour. Whereas most scientists working on the evolution of behaviour have mainly been concerned with the consequences of variability within populations, it is also important to consider the origins of that variability. The idea that developmental constraints affect behavioural flexibility is not incompatible with the social brain hypothesis, the idea that ecological variables and unavoidable mortality affect brain evolution [22,103], or the cultural intelligence hypothesis [103]. Therefore, we believe it would be useful to integrate developmental constraints into the broader framework within which we strive to understand the evolution of brains and behavioural flexibility.