Skip to main content
Anatomy Research International logoLink to Anatomy Research International
. 2011 May 29;2011:236894. doi: 10.1155/2011/236894

Mosaic Evolution of Brainstem Motor Nuclei in Catarrhine Primates

Seth D Dobson 1,*, Chet C Sherwood 2
PMCID: PMC3335461  PMID: 22567289

Abstract

Facial motor nucleus volume coevolves with both social group size and primary visual cortex volume in catarrhine primates as part of a specialized neuroethological system for communication using facial expressions. Here, we examine whether facial nucleus volume also coevolves with functionally unrelated brainstem motor nuclei (trigeminal motor and hypoglossal) due to developmental constraints. Using phylogenetically informed multiple regression analyses of previously published brain component data, we demonstrate that facial nucleus volume is not correlated with the volume of other motor nuclei after controlling for medulla volume. Our results show that brainstem motor nuclei can evolve independently of other developmentally linked structures in association with specific behavioral ecological conditions. This finding provides additional support for the mosaic view of brain evolution.

1. Introduction

Two competing models of brain evolution have dominated the neuroscience literature over the past 15 years. The first posits that the interspecific scaling of vertebrate brain components is explained mostly by a conserved pattern of neurogenesis, such that structures that develop later tend to be relatively large [1, 2]. This is supported by the fact that later developing structures exhibit larger allometric exponents when scaled against overall brain size [1]. Supporters of the developmental correlation model argue that brain structure evolves due primarily to selection on overall brain size, as opposed to the specialization of particular areas for specific functions [2]. Thus, individual brain structures vary in size according to general scaling principles that constrain adaptive evolution, thereby limiting the impact of behavioral ecological conditions on brain structure.

The alternative model posits that natural selection can act to expand or contract the size of individual brain components, independent of overall brain size, without necessarily altering the size of functionally unrelated regions [3, 4]. Supporters of this mosaic evolution model argue that the coordinated evolution of individual brain regions is due to functional and/or structural connections [5, 6]. According to this model, developmental constraints can be overridden by selection to enlarge separate neural systems in response to specific behavioral ecological conditions. This idea is supported by comparative analyses of neural specialization in species as diverse as primates [7], birds [8], and fish [9]. The mosaic evolution model also posits a role for constraints, but supporters of this model tend to emphasize energetic trade-offs influencing overall brain size [10] rather than developmental correlations per se [11].

In a previous paper, we examined the coordinated evolution of brain regions involved in producing and processing facial expressions in anthropoid primates [12]. The results of our study revealed that social group size is positively correlated with the relative size of the facial motor nucleus, which sends motor neurons from the brainstem to the muscles of facial expression [13]. This pattern, which we observed in catarrhines but not platyrrhines, is consistent with the idea that facial communication is an important form of conflict management and bonding within catarrhine social groups [1416]. In addition, we found that facial nucleus volume is positively correlated with primary visual cortex volume, after controlling for the size of the rest of the brain [12]. These results bolster the mosaic view of brain evolution. However, they do not preclude the possibility of developmental correlations within the brainstem, that is, correlated size changes in functionally unrelated motor nuclei.

The purpose of the present study is to test the hypothesis that the size of the facial motor nucleus in catarrhines evolves in coordination with other brainstem motor nuclei due to developmental correlation. Developmental covariation among brainstem motor nuclei is to be expected since these nuclei show similar patterns of growth-factor receptor expression and coordinated modulation of neuronal proliferation and survival [17]. We will examine two comparative predictions of the developmental correlation model: (i) facial nucleus size is positively correlated with trigeminal motor nucleus size and hypoglossal nucleus size, after controlling for the size of the medulla and (ii) the relative sizes of the trigeminal motor nucleus and hypoglossal nucleus are positively correlated with social group size. The former prediction addresses the essence of the developmental correlation model: coordinated size changes due to a shared developmental basis. The latter prediction derives from the fact that facial nucleus size is correlated with group size [12]. To assess the specificity of this group-size effect, we examine the possibility that the other two brainstem orofacial motor nuclei are also correlated with group size. The results of our study contribute to debates regarding the relative importance of developmental constraints versus adaptive specializations in mammalian brain evolution.

2. Materials and Methods

Brain component volumes for 14 group-living, nonhuman catarrhine species were taken from previously published sources [13, 18, 19]. Group size data were taken from an unpublished dataset available on C. Nunn's website [20]  (http://www.people.fas.harvard.edu/~nunn/index.html). We examined trait correlations using multiple regression analyses. Two sets of analyses were carried out: (i) we examined the volume of the trigeminal motor nucleus and hypoglossal nucleus in relation to facial nucleus volume after controlling for medulla size [13] and (ii) we examined the relative volume of the trigeminal motor nucleus and hypoglossal nucleus in relation to group size. Autocorrelation, which can occur when the independent variable represents a large part of the dependent variable, is not a serious issue for our analyses because each nucleus comprises less than 0.5% of the volume of the total medulla [21]. All data were log-transformed (natural) prior to analysis.

Regression coefficients and standard errors were generated using a phylogenetic generalized least-squares (PGLSs) approach [22]. We used COMPARE 4.6b [23] to perform PGLSs multiple regressions based on a single consensus tree with chronometric branch lengths [24], which we downloaded from the 10kTrees website (version 2) (http://10ktrees.fas.harvard.edu/). To take into account phylogenetic uncertainty, we also ran each analysis on a block of trees (N = 1000) in which the position of any given node varies as a function of its Bayesian posterior probability [24]. The null hypothesis (slope = 0) was assessed using 95% confidence intervals for each regression coefficient [25]. In the case of the tree block analyses, we incorporated both sampling variance and variance due to phylogenetic uncertainty into the calculation of confidence intervals [26].

3. Results and Discussion

Figure 1 demonstrates the strong degree of covariation between brainstem orofacial motor nuclei in catarrhines prior to size correction. However, the multiple regression results in Table 1 indicate that this pattern of covariation disappears after controlling for medulla size, that is, neither trigeminal motor nucleus volume nor hypoglossal nucleus volume is a significant predictor of facial nucleus volume independent of medulla volume. Moreover, social group size is not positively correlated with either trigeminal motor nucleus volume or hypoglossal nucleus volume after size correction (Table 1). The results of the tree block analyses are identical with the consensus tree results because the degree of phylogenetic uncertainty in our sample is negligible. Thus, the hypothesis that catarrhine brainstem motor nuclei evolve in coordination with each other due to a shared developmental basis is not supported by our results. Instead, it appears that relative facial motor nucleus size evolves independently of the rest of the medulla and in association with social group size.

Figure 1.

Figure 1

Scatter-plot matrix depicting relationships among brainstem motor nuclei volumes in catarrhines before size correction. Data are natural log transformed. Variable abbreviations—medulla: medulla volume; Vmo: trigeminal motor nucleus volume; VII: facial motor nucleus volume; XII: hypoglossal nucleus volume. Species abbreviations—Ca: Cercopithecus ascanius; Cm: Cercopithecus mitis; Ep: Erythrocebus patas; Gg: Gorilla gorilla; Hl: Hylobates lar; La: Lophocebus albigena; Mm: Macaca mulatta; Mt: Miopithecus talapoin; Nl: Nasalis larvatus; Pp: Pan paniscus; Pt: Pan troglodytes; Pa: Papio anubis; Pb: Procolobus badius; Pn: Pygathrix nemaeus.

Table 1.

Multiple regression results using phylogenetic generalized least squares. Results are presented for a single consensus tree. The results of the tree block analyses are identical to the consensus tree results because the degree of phylogenetic uncertainty in our sample is negligible (see Materials and Methods for further details). CI: confidence interval (±2 standard errors); Vmo: trigeminal motor nucleus volume; VII: facial motor nucleus volume; XII: hypoglossal nucleus volume.

Dependent R 2 Independent Slope 95% CI
VII 91% Medulla 0.61 0.03, 1.19
Vmo −0.08 −0.78, 0.62
XII 0.53 −0.07, 1.13
Vmo 86% Medulla 0.89 0.67, 1.11
Group size 0.09 −0.05, 0.23
XII 86% Medulla 1.04 0.78, 1.30
Group size 0.08 −0.08, 0.24

Taken together, the results of our previous work [12] and the present study provide additional support for the mosaic model of brain evolution [3]. Proponents of this model assert that natural selection can target functionally interconnected neural systems resulting in structural changes that are relatively unconstrained by developmental processes [11]. We found that the catarrhine facial motor nucleus evolves independently of other brainstem orofacial motor nuclei in response to a specific behavioral ecological condition, group size, and in coordination with a functionally linked region, the primary visual cortex [12]. The latter result is particularly striking because it involves coevolution between two brain components that are not structurally interconnected by direct axonal pathways. Moreover, because the medulla and neocortex undergo neurogenesis at different times [1], developmental correlation is an unlikely explanation for this pattern of correlated evolution.

It appears that general trends in brain evolution observed at higher taxonomic levels can mask adaptive diversity at lower taxonomic levels. For example, the negative relationship between relative neocortex size and relative limbic structure size observed at the ordinal level in mammals does not apply within orders [27]. Similarly, it has been suggested that the mammalian central visual system exhibits a high degree of evolutionary conservatism [28]. However, numerous studies of the primary visual cortex in primates have demonstrated that species can deviate from allometric trends as a function of activity period, dietary preference, and/or sociality [7, 12, 2932]. Thus, it seems that much of the debate concerning the relative importance of adaptive specialization versus developmental constraint is driven by differences in the taxonomic sampling.

4. Conclusions

Previous research has shown that correlated evolution may occur within structurally interconnected neural systems [5, 6]. Our findings are unique in demonstrating that mosaic brain evolution can also involve coordinated changes in the volume of brain components that are not structurally linked by direct axonal pathways, but that participate in a common adaptive complex [12]. These results also provide further support for the idea that neural specializations in mammals are not restricted to executive brain functions. Brainstem structures can also undergo adaptive specialization in response to the motor and/or sensory demands of specific behavioral ecological conditions [3337].

Acknowledgments

The authors thank the following: Anne Burrows, Kathleen Muldoon, and Adam Sylvester for inviting them to contribute to this paper and two anonymous reviewers for helpful comments. The writing of this article was supported by a Junior Faculty Fellowship from Dartmouth College awarded to S. D. Dobson.

References

  • 1.Finlay BL, Darlington RB. Linked regularities in the development and evolution of mammalian brains. Science. 1995;268(5217):1578–1584. doi: 10.1126/science.7777856. [DOI] [PubMed] [Google Scholar]
  • 2.Finlay BL, Darlington RB, Nicastro N. Developmental structure in brain evolution. Behavioral and Brain Sciences. 2001;24(2):263–308. [PubMed] [Google Scholar]
  • 3.Barton RA, Harvey PH. Mosaic evolution of brain structure in mammals. Nature. 2000;405(6790):1055–1058. doi: 10.1038/35016580. [DOI] [PubMed] [Google Scholar]
  • 4.de Winter W, Oxnard CE. Evolutionary radiations and convergences in the structural organization of mammalian brains. Nature. 2001;409(6821):710–714. doi: 10.1038/35055547. [DOI] [PubMed] [Google Scholar]
  • 5.Whiting BA, Barton RA. The evolution of the cortico-cerebellar complex in primates: anatomical connections predict patterns of correlated evolution. Journal of Human Evolution. 2003;44(1):3–10. doi: 10.1016/s0047-2484(02)00162-8. [DOI] [PubMed] [Google Scholar]
  • 6.Barton RA, Aggleton JP, Grenyer R. Evolutionary coherence of the mammalian amygdala. Proceedings of the Royal Society of London Series B. 2003;270(1514):539–543. doi: 10.1098/rspb.2002.2276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Barton RA. Visual specialization and brain evolution in primates. Proceedings of the Royal Society of London Series B. 1998;265(1409):1933–1937. doi: 10.1098/rspb.1998.0523. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Iwaniuk AN, Hurd PL. The evolution of cerebrotypes in birds. Brain, Behavior and Evolution. 2005;65(4):215–230. doi: 10.1159/000084313. [DOI] [PubMed] [Google Scholar]
  • 9.Pollen AA, Dobberfuhl AP, Scace J, et al. Environmental complexity and social organization sculpt the brain in Lake Tanganyikan cichlid fish. Brain, Behavior and Evolution. 2007;70(1):21–39. doi: 10.1159/000101067. [DOI] [PubMed] [Google Scholar]
  • 10.Isler K, van Schaik CP. The expensive brain: a framework for explaining evolutionary changes in brain size. Journal of Human Evolution. 2009;57(4):392–400. doi: 10.1016/j.jhevol.2009.04.009. [DOI] [PubMed] [Google Scholar]
  • 11.Barton RA. Primate brain evolution: integrating comparative, neurophysiological, and ethological data. Evolutionary Anthropology. 2006;15(6):224–236. [Google Scholar]
  • 12.Dobson SD, Sherwood CC. Correlated evolution of brain regions involved in producing and processing facial expressions in anthropoid primates. Biology Letters. 2011;7(1):86–88. doi: 10.1098/rsbl.2010.0427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Sherwood CC, Hof PR, Holloway RL, et al. Evolution of the brainstem orofacial motor system in primates: a comparative study of trigeminal, facial, and hypoglossal nuclei. Journal of Human Evolution. 2005;48(1):45–84. doi: 10.1016/j.jhevol.2004.10.003. [DOI] [PubMed] [Google Scholar]
  • 14.Waller BM, Dunbar RIM. Differential behavioural effects of silent bared teeth display and relaxed open mouth display in chimpanzees (Pan troglodytes) Ethology. 2005;111(2):129–142. [Google Scholar]
  • 15.Flack JC, de Waal F. Context modulates signal meaning in primate communication. Proceedings of the National Academy of Sciences of the United States of America. 2007;104(5):1581–1586. doi: 10.1073/pnas.0603565104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Bout N, Thierry B. Peaceful meaning for the silent bared-teeth displays of mandrills. International Journal of Primatology. 2005;26(6):1215–1228. [Google Scholar]
  • 17.Sendtner M, Pei G, Beck M, Schweizer U, Wiese S. Developmental motoneuron cell death and neurotrophic factors. Cell and Tissue Research. 2000;301(1):71–84. doi: 10.1007/s004410000217. [DOI] [PubMed] [Google Scholar]
  • 18.de Sousa AA, Sherwood CC, Mohlberg H, et al. Hominoid visual brain structure volumes and the position of the lunate sulcus. Journal of Human Evolution. 2010;58(4):281–292. doi: 10.1016/j.jhevol.2009.11.011. [DOI] [PubMed] [Google Scholar]
  • 19.Frahm HD, Stephan H, Baron G. Comparison of brain structure volumes in insectivora and primates. V. Area striata (AS) Journal Fur Hirnforschung. 1984;25(5):537–557. [PubMed] [Google Scholar]
  • 20.Nunn CL, van Schaik CP. A comparative approach to reconstructing the socioecology of extinct primates. In: Plavcan JM, Jungers WL, Kay RF, van Schaik CP, editors. Reconstructing Behavior in the Fossil Record. New York, NY, USA: Kluwer Academic; 2001. pp. 159–188. [Google Scholar]
  • 21.Deacon TW. Fallacies of progression in theories of brain-size evolution. International Journal of Primatology. 1990;11(3):193–236. [Google Scholar]
  • 22.Martins EP, Hansen TF. Phylogenies and the comparative method: a general approach to incorporating phylogenetic information into the analysis of interspecific data. American Naturalist. 1997;149(4):646–667. [Google Scholar]
  • 23.Martins EP. COMPARE, version 4.6b. Computer programs for the statistical analysis of comparative data, (Bloomington, IN: Department of Biology, Indiana University), 2004, http://compare.bio.indiana.edu/
  • 24.Arnold C, Matthews LJ, Nunn CL. The 10kTrees website: a new online resource for primate phylogeny. Evolutionary Anthropology. 2010;19(3):114–118. [Google Scholar]
  • 25.Cumming G. Replication and p-intervals: p-values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science. 2008;3(4):286–300. doi: 10.1111/j.1745-6924.2008.00079.x. [DOI] [PubMed] [Google Scholar]
  • 26.Martins EP. Conducting phylogenetic comparative studies when the phylogeny is not known. Evolution. 1996;50(1):12–22. doi: 10.1111/j.1558-5646.1996.tb04468.x. [DOI] [PubMed] [Google Scholar]
  • 27.Reep RL, Finlay BL, Darlington RB. The limbic system in mammalian brain evolution. Brain, Behavior and Evolution. 2007;70(1):57–70. doi: 10.1159/000101491. [DOI] [PubMed] [Google Scholar]
  • 28.Kaskan PM, Franco ECS, Yamada ES, Silveira LCD, Darlington RB, Finlay BL. Peripheral variability and central constancy in mammalian visual system evolution. Proceedings of the Royal Society of London Series B. 2005;272(1558):91–100. doi: 10.1098/rspb.2004.2925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Joffe TH, Dunbar RIM. Visual and socio-cognitive information processing in primate brain evolution. Proceedings of the Royal Society of London Series B. 1997;264(1386):1303–1307. doi: 10.1098/rspb.1997.0180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Barton RA, Purvis A, Harvey PH. Evolutionary radiation of visual and olfactory brain system in primates, bats and insectivores. Philosophical Transactions of the Royal Society of London Series B. 1995;348(1326):381–392. doi: 10.1098/rstb.1995.0076. [DOI] [PubMed] [Google Scholar]
  • 31.Barton RA. Evolutionary specialization in mammalian cortical structure. Journal of Evolutionary Biology. 2007;20(4):1504–1511. doi: 10.1111/j.1420-9101.2007.01330.x. [DOI] [PubMed] [Google Scholar]
  • 32.Bush EC, Allman JM. Three-dimensional structure and evolution of primate primary visual cortex. Anatomical Record Part A. 2004;281A(1):1088–1094. doi: 10.1002/ar.a.20114. [DOI] [PubMed] [Google Scholar]
  • 33.Matano S. A volumetric comparison of the vestibular nuclei in primates. Folia Primatologica. 1986;47(4):189–203. doi: 10.1159/000156277. [DOI] [PubMed] [Google Scholar]
  • 34.Baron G, Frahm HD, Stephan H. Comparison of brain structure volumes in insectivora and primates. VIII. Vestibular complex. Journal Fur Hirnforschung. 1988;29(5):509–523. [PubMed] [Google Scholar]
  • 35.Baron G, Stephan H, Frahm HD. Comparison of brain structure volumes in insectivora and primates. IX. Trigeminal complex. Journal Fur Hirnforschung. 1990;31(2):193–200. [PubMed] [Google Scholar]
  • 36.Glendenning KK, Masterton RB. Comparative morphometry of mammalian central auditory systems: variation in nuclei and form of the ascending system. Brain, Behavior and Evolution. 1998;51(2):59–89. doi: 10.1159/000006530. [DOI] [PubMed] [Google Scholar]
  • 37.Hutcheon JM, Kirsch JAW, Garland T. A comparative analysis of brain size in relation to foraging ecology and phylogeny in the Chiroptera. Brain, Behavior and Evolution. 2002;60(3):165–180. doi: 10.1159/000065938. [DOI] [PubMed] [Google Scholar]

Articles from Anatomy Research International are provided here courtesy of Wiley

RESOURCES