Selection against admixture and gene regulatory divergence in a long-term primate field study

Abstract

Genetic admixture is central to primate evolution. Here, we combine 50 years of field observations of immigration and group demography with genomic data from ~9 generations of hybrid baboons to investigate the consequences of admixture in the wild. Despite no obvious fitness costs to hybrids, we find signatures of selection against admixture similar to those described for archaic hominins. These patterns are concentrated near genes where ancestry is strongly associated with gene expression. Our analyses also show that introgression is partially predictable across the genome. They demonstrate the value of integrating genomic and field data for revealing how “genomic signatures of selection” (e.g., reduced introgression in low recombination regions) manifest in nature, and underscore the importance of other primates as living models for human evolution.

One Sentence Summary:

Coupled genomic and field data indicate selection against baboon hybrids, despite no overt fitness costs in the wild.

The ancestors of modern humans intermixed with Neanderthals and other close, now-extinct lineages, leaving a genetic legacy that continues to shape human trait variation today (1-3). Even as these findings reshape our conception of human origins, they also bring us more closely in line with our primate relatives, where hybridization is observed in many species (4, 5). Studies of other living primates therefore provide context for understanding admixture dynamics in our own lineage. Field studies in hybrid zones, for instance, offer the opportunity to integrate demographic (e.g., reproductive success, immigration/emigration), phenotypic, and genomic data on early generation hybrids, which studies in humans suggest experienced the greatest fitness costs (6, 7).

Thus far, studies suggest that ancestry frequently predicts trait variation in primate hybrid zones, but admixture often does not result in overt fitness costs (8-11). However, field observations have not been combined with population and functional genomic analyses to investigate both the organismal and molecular consequences of admixture in primates. Here, we take such an approach to investigate whether selection against introgression (i.e., alleles introduced by gene flow from one distinct lineage to another) is compatible with apparently healthy hybrids, investigate the functional consequences of introgressed alleles, and follow the course of hybridization and natural selection across generations.

We focus on admixture between yellow baboons (Papio cynocephalus) and anubis baboons (P. anubis): large-bodied, terrestrial primates long used as a model for human biology and evolution (12). Although baboon taxonomy has undergone many revisions over time, six extant baboon species are currently recognized based on distinct phenotypic differences and a pattern of phylogenetic divergence supported by recent whole-genome sequencing data (12-14). This phylogeny establishes two major baboon lineages (the “northern” and “southern” clades) that separated ~1.4 million years ago, although the complex evolutionary history of baboons means that they may have experienced episodes of gene flow since that time (14-16). Anubis and yellow baboons belong to the northern and southern clades, respectively, and both phylogenetic and population genomic analyses confirm their divergence into distinct taxa (13, 14). Nevertheless, they interbreed to produce viable and fertile offspring where their ranges meet (Fig. 1A).

Figure 1: The structure of the baboon hybrid zone in Amboseli and the surrounding region.

(A) Geographic locations and local ancestry estimates for baboons in this study (black asterisk=Amboseli). For each population, each row corresponds to the first 20 Mb of chromosome 1 for one individual, organized vertically by global ancestry. For Amboseli, a subsample of 100 individuals is shown. Central map: ranges of yellow baboons and anubis baboons in Kenya and Tanzania; small map: ranges of all six African baboons (47), modified from a map by Kenneth Chiou (CC BY 3.0 license). Baboon drawings by Christopher Smith. (B) PCA of genotype data for high coverage genomes. Inset: distribution of “yellow-like” individuals along PC1. SNPRC yellow baboon founders resemble admixed Amboseli baboons. (C) IBDMix (26) results for three sets of yellow or majority yellow baboons. SNPRC yellow baboon founders and Amboseli baboons exhibit substantial identity-by-descent (IBD) with anubis baboons, while IBD estimates for Mikumi yellow and anubis baboons are low. The excess IBD in the SNPRC and Amboseli samples points to the contribution of gene flow beyond residual incomplete lineage sorting.

We concentrate on the region in and around the Amboseli basin of Kenya, where data from fifty years of continuous observation on a population near the center of the hybrid zone are available (17). Members of this majority-yellow baboon population include descendants of historical admixture prior to the start of monitoring in 1971, as well as descendants of a directly observed, recent wave of admixture beginning in 1982 (15, 18). In Amboseli, hybrids do not experience obvious fitness costs, and anubis ancestry may in fact confer benefits, including accelerated maturation, increased mating success, and higher rates of male-female affiliation (19-21). However, field and microsatellite data indicate that the hybrid zone is narrow (22), suggesting that natural selection may act to limit gene flow.

Structure of the baboon hybrid zone

To assess selection against introgression in hybrid anubis-yellow baboons, we used whole-genome resequencing data to evaluate ancestry patterns for animals sampled in and near the Amboseli hybrid zone (Fig. 1 and table S1). We generated resequencing data from 430 wild baboons from Amboseli and Mikumi National Park in Tanzania (17 high coverage [mean=22.51x]; 413 low coverage [mean=1.04x]), which we combined with published baboon genomes from Amboseli (n=22), Mikumi (n=5), the Maasai Mara National Reserve (n=7), and the Aberdares region of central Kenya (n=2) (14, 15, 23). In Amboseli, our sample included 442 baboons born between 1969 and 2016. Finally, we also included the genomes of 39 captive baboons from the Southwest National Primate Research Center (SNPRC; n=33, n=31 of whom were colony founders) and the Washington National Primate Research Center (WNPRC; n=6) (14, 15, 24).

We estimated global and local ancestry for each individual using a composite likelihood method suitable for low coverage data, LCLAE, which uses genotype likelihoods across genomic windows rather than requiring genotypes at specific variants (13, 15). These results confirm that admixture is minimal or absent in the anubis baboon founders of the SNPRC colony, anubis baboons from Maasai Mara, and yellow baboons from Mikumi (Fig. 1 and fig. S5, although we cannot exclude ancient bouts of admixture that affect all living baboons). In contrast, all baboons from Amboseli are admixed (Fig. 1A; mean=30-37% genome-wide anubis ancestry ± 10% s.d.), including many whose ancestry can be traced to anubis-like immigrants within the last seven generations. These results closely match F₄-ratio estimates (25) (<2% difference for nine high coverage Amboseli genomes), indicating that putative anubis ancestry in Amboseli reflects admixture, not incomplete lineage sorting (13).

We also detected a signal of ~17% mean anubis ancestry in the putatively yellow baboon founders of the SNPRC colony, who were previously thought to be unadmixed (Fig. 1A-C) (24). Identity-by-descent (IBD) analysis using IBDMix (26) confirms this pattern (Fig. 1C). As IBD between Mikumi yellow baboons and anubis baboons is <5%, these findings also implicate admixture rather than incomplete lineage sorting (Fig. 1C; (13)). Combined with evidence for yellow ancestry in a central Kenyan anubis baboon (13, 14), our results indicate that gene flow has been a common feature of baboon evolution in east Africa.

Selection against introgression in Amboseli

To investigate whether selection restricts gene flow between anubis and yellow baboons, we focused on the multigenerational data set from Amboseli. We replicated three analyses used to infer selection against Neanderthal or Denisovan introgression in humans (27-29). First, we tested for a relationship between anubis ancestry in Amboseli and yellow-anubis genetic divergence (based on unadmixed populations: (13)). Because the Amboseli population is largely of yellow baboon origin, if hybridization is deleterious, selection is expected to be less permissive of anubis alleles that are more diverged from their yellow counterparts. Indeed, anubis ancestry is systematically lower in regions of the genome with more fixed differences (Fig. 2A; Spearman’s rho=−0.119, p=8.05 x 10⁻³⁴). In Amboseli, anubis alleles are 6.7% less common in the most diverged percentile relative to the least diverged percentile of the baboon genome. These results are similar to the negative correlation between the density of fixed human-Neanderthal differences and introgressed Neanderthal ancestry in modern humans (27) (Fig. 2B and table S2).

Figure 2: Selection against introgression in the Amboseli baboons mirrors patterns described for archaic hominin admixture.

(A, C, E) Proportion of introgressed (anubis) ancestry in Amboseli in 250 kb windows (n=10,324 total windows) as a function of (A) fixed differences between yellow and anubis baboons (Spearman’s rho=−0.119, p=8.05 x 10⁻³⁴), (C) mean B statistic (rho=0.168, p=1.73 x 10⁻⁶⁶), and (E) mean recombination rate (rho=0.127, p=2.49 x 10⁻³⁸), divided into quintiles for visualization purposes only. Dashed grey lines show median anubis ancestry across all windows. (B, D, F) Predicted relationships between introgressed ancestry and all three measures are observed for both anubis ancestry in the Amboseli baboons (solid lines) and Neanderthal ancestry in modern human genomes (dashed lines) (27-29), consistent with selection against introgression. Panels show the relationship between introgressed ancestry and the rank-ordered (B) number of fixed differences, (D) mean B statistic, and (F) mean local recombination rate. Mean introgressed ancestry is centered on 0 and divided by the standard deviation for each species to facilitate comparison.

Second, we tested whether introgressed anubis ancestry is depleted in genomic regions that are likely to be affected by linked selection, as summarized by B statistic values calculated for the baboon genome (13, 30) (i.e., due to high gene density per recombination distance). Again paralleling the case of Neanderthal ancestry in modern humans (28), anubis ancestry is most common in regions that are predicted to be least affected by linked selection (Fig. 2C-D; Spearman’s rho=0.168, p=1.73 x 10⁻⁶⁶). Consequently, anubis ancestry per individual is reduced, on average, by 7.03% in protein-coding regions relative to random, size-matched regions of the genome (±4.20% s.d.; n=442 Amboseli baboons). Reductions in promoters and putative peripheral blood mononuclear cell enhancers were 5.56% (±4.10% s.d.) and 6.22% (±4.20%), respectively.

Third, we tested whether introgressed anubis ancestry is positively correlated with local recombination rate. This relationship is predicted if recombination influences the rate at which natural selection eliminates deleterious introgressed ancestry and uncouples deleterious from neutral introgressed variants. This prediction, documented across diverse taxa (29, 31, 32), is also observed in baboons (Fig. 2E; Spearman’s rho=0.127, p=1.48 x 10⁻³⁸), with a magnitude similar to that reported for Neanderthal and Denisovan gene flow into modern humans (Fig. 2F; Spearman’s rho=0.17 and 0.14 for Neanderthal and Denisovan ancestry, respectively (29)).

To investigate these patterns further, we took advantage of the dynamic history of admixture within the Amboseli population. At the beginning of monitoring in 1971, all Amboseli animals were considered to be yellow baboons (33). Phenotypically anubis and admixed animals immigrated into the population starting in 1982, and the proportion of hybrid animals increased over the following decades (18, 34). Whole-genome data recapitulate these patterns, documenting an increase of 11.8% anubis ancestry from 1971 (23.1-29.6%) to 2020 (34.9-41.4%) (Fig. 3A). However, animals with no known anubis ancestors during the 50-year field study are also clearly admixed (Fig. 3B). Additionally, while immigrant males are more anubis-like than the study population as a whole (Fig. 3A), one immigrant male was among the most yellow-like in our sample (78.8% yellow ancestry), indicating ongoing gene flow involving both parental taxa.

Figure 3: Recent and historic hybrid ancestry in Amboseli.

(A) Mean genome-wide anubis ancestry in the Amboseli population has increased since the 1970s. Numbers above the x-axis indicate the number of individuals used to calculate annual ancestry (black=all individuals, green=male immigrants). (B) Pedigree and ancestry estimates for example historical (left) and recent (right) hybrids. Pedigree individuals with resequencing data are colored based on ancestry. The two examples share a maternal grandmother and were born a few years apart (yellow and bright green asterisks in [A]). The father of the recent hybrid immigrated in 2004 (olive green asterisk in [A]). (C) Genome-wide anubis ancestry in Amboseli, with density plots overlaid for historical and recently admixed individuals. (D-F) The relationships between introgressed anubis ancestry and the rank-ordered (D) number of fixed anubis-yellow differences, (E) mean B statistic, and (F) mean local recombination rate. All relationships are stronger for historic hybrids than recent hybrids. Right-hand panels show anubis ancestry within each data set mean-centered to 0.

The Amboseli population today therefore contains individuals who descend from ancient, unobserved admixture events as well as those affected by recent hybridization, generating a bimodal distribution of genome-wide ancestry (Fig. 3C) (15). By integrating local ancestry calls, pedigree information, and field observations, we identified 188 “recently” admixed individuals whose ancestors include at least one anubis-like immigrant within the last 0-7 generations (mean=1.7 generations, although due to historical gene flow, these animals are not classical F1 or F2 hybrids). We also classified 214 baboons as “historically” admixed, as their genomes only contain anubis ancestry from before 1971. Forty baboons could not be assigned to either hybrid class (13). Based on a single-pulse model of admixture using DATES (35), historical admixture is dated to 283 ± 242 s.d. generations ago (n=7 high-coverage genomes), in contrast to 5 and 21 generations ago for two recent hybrids sequenced to high coverage.

Stratifying individuals in the data set by admixture history reveals that signatures of selection against introgression are driven by historical admixture (i.e., genomes sampled dozens to hundreds of generations post-contact). Historically admixed individuals are more depleted of anubis ancestry in highly diverged and low B value regions of the genome than recently admixed animals (Fig. 3D-E). Further, the relationship between anubis ancestry and recombination rate is exclusive to the historically admixed data set, even based on recombination rates measured on chromosome-level scales (Fig. 3F and table S3, (13)). The weaker signature of selection in recent hybrids likely reflects intermittent gene flow in the last few generations and stochastic inheritance processes. In contrast, sufficient generations have passed since historical admixture to break apart large introgressed haplotypes, allowing us to observe non-random patterns of ancestry across the genome. This result emphasizes the importance of complementing field observations with genomic data, which provide insight into selective processes that operate over timescales longer than even the longest-running field studies.

Selection against regulatory divergence

Analyses of human-Neanderthal admixture suggest a consistent pattern of selection against regulatory variants (36). If so, the introgressed regions that persist in modern humans have likely been purged of many alleles with large regulatory effects (37, 38). However, direct comparisons between the effect sizes of retained versus lost archaic alleles are difficult, as only a fraction of archaic hominin alleles (e.g., Neanderthal, Denisovan, or other ghost lineages) segregate in modern human genomes today (28, 39). Extant primate populations, where hybridization and selection are ongoing, provide an opportunity to test this hypothesis.

To test for selection against gene regulatory divergence in baboons, we paired genetic ancestry data with blood-derived RNA-sequencing data from 145 individuals (n=157 samples (40-42); table S1). This data set includes whole blood and white blood cells, which were analyzed separately while controlling for age, sex, and kinship (13). Among 10,192 analyzed genes, we identified no significant associations between genome-wide ancestry and gene expression levels (10% FDR). In contrast, local ancestry predicted gene expression levels for 20.1% (2,046) of tested genes in one or both data sets (Fig. 4A), with concordant additive effects between data sets (Pearson’s R=0.43, p<10⁻²⁰⁰), and little evidence for non-additivity (13).

Figure 4: Selection against gene regulatory divergence and prediction of local introgression levels.

(A) Local ancestry predicts gene expression in the Amboseli population, as depicted for an example gene (MRPL2). Inset: quantile-quantile plot comparing p-value distributions for observed local ancestry effects (y-axis) to a permutation-based null (x-axis). (B) Difference in introgressed anubis ancestry between genes with the smallest versus largest local ancestry effects on gene expression. Violin plots show the distribution of differences across individuals; boxplots show the median and inter-quartile range (all p<0.05; table S4). (C) Correlation between anubis introgression and recombination rate calculated for sets of genes with the largest (blue) versus smallest (magenta) local ancestry absolute effect sizes. Asterisks denote bootstrapped p-value < 0.05 (table S4); error bars show standard deviations. (D) Distribution of effect sizes for features that consistently predict the extent of anubis introgression in Amboseli baboon genomes (table S5). The number of single nucleotide variants (SNVs) is derived from unadmixed yellow and anubis baboon genomes.

If introgressed alleles that perturb gene regulation are a primary target of selection, we reasoned that selection should purge anubis ancestry near genes where ancestry strongly affects gene expression. In support of this prediction, the top 15% of genes with the largest local ancestry effects on gene expression harbor 1.5% less anubis ancestry, on average, than the bottom 15% of genes with the smallest local ancestry effects (Fig. 4B; paired t-test p=1.10 x 10⁻³⁶, n=442). This difference is exaggerated within historically admixed individuals (1.9% reduction, p=1.26 x 10⁻²⁷, n=214; table S4). Further, the correlation between anubis ancestry and local recombination rate is larger for genes with the largest local ancestry effects than those with the smallest (Fig. 4C; rho_diff=0.07 for the top and bottom 15% of genes, bootstrapped p-value=0.027; table S4). Combined with the depletion of introgressed sequence in regulatory elements, these results support the hypothesis that introgressed alleles that affect gene regulation are nonrandomly purged post-hybridization. They are therefore consistent with the idea that natural selection removed archaic variants with large regulatory effects from the genomes of modern humans (37).

Predicting the genomic landscape of introgression

Finally, we investigated our ability to predict the genomic locations most and least affected by introgression. We modeled mean anubis ancestry as a function of local recombination rate, SNP density in the reference yellow and anubis populations, yellow-anubis genetic divergence, gene and enhancer content, linked selection, and local ancestry-associated gene expression in blood. We iteratively trained an elastic net regression model on nonoverlapping 250 kb windows of the genome, representing 75% of the genome, and applied the model to a test set of windows in the remaining 25% (13). We found that our predicted values were consistently positively correlated with observed levels of anubis ancestry in the test sets (mean Pearson’s R=0.254 ± 0.016 s.d. vs. 0.014 ± 0.011 s.d. for models fit to permuted data), with frequent contributions from features capturing local recombination rate, linked selection, genetic variation, and sequence divergence (Fig. 4D and table S5). We consistently predicted anubis ancestry more accurately in historical hybrids than in recent hybrids (mean Pearson’s R=0.265 ± 0.017 s.d. vs. 0.177 ±0.018 s.d., bootstrapped p-value <10⁻³).

Our longitudinal data also indicate that increases in anubis ancestry across the fifty-year field study are non-randomly distributed throughout the genome. Controlling for the starting level of anubis ancestry in 1979, 100 kb windows characterized by lower F_ST and higher recombination rates experienced larger increases in anubis ancestry between 1979 and 2020, although both effect sizes are small (F_ST and recombination rate p-values <10⁻³, β=−2.965 x 10⁻⁴ and 1.020 x 10⁻⁴ respectively, n=25,797 windows; table S6). B statistic values did not predict temporal change in anubis ancestry independently of recombination rate.

Divergence and hybridization in primates

Our genomic analysis reveals evidence for selection against admixture that is remarkably consistent with results obtained for archaic introgression in humans. Our results also support a hypothesis that can only be indirectly tested in our own lineage: that natural selection has acted to eliminate introgressed alleles that strongly perturb gene regulation (37). These results contrast with the behavioral and life history evidence to date in Amboseli—one of the largest and longest-running primate field sites in the world—which indicates that hybrid baboons suffer no obvious fitness costs (19-21). Our results identify subtle selection against hybridization that may help explain the maintenance of primate taxonomic integrity in the face of frequent interspecific gene flow (4, 5). Ultimately, the outcome of this process will depend on the relative balance between this selection pressure, possible advantages to introgressed ancestry, migration rates, and demographic stochasticity—potentially explaining cases of nuclear swamping in baboons despite costs to hybridization (16).

The mode of selection against hybrids is unclear. Unlike in humans, hybridization load is unlikely to explain our results: yellow and anubis baboons harbor similar levels of genetic diversity compared to humans and Neanderthals (<50% difference in baboons compared to >3-fold between humans and Neanderthals (6, 14, 15, 27)). Both hybrid incompatibilities and ecological selection, however, could play a role. For example, some reports suggest that anubis and yellow baboons occupy distinct climatic niches (43). Previously described assortative mating by ancestry in the Amboseli baboons (20) may also limit introgression. Understanding the genetic and phenotypic mechanisms that influence interspecific gene flow, including the role of the X chromosome and adaptive introgression, remains an important goal for future work.

Combined, our findings illustrate the importance of contextualizing genomic data with phenotypic and demographic information to understand the evolutionary dynamics of admixture. Genomes harbor information about historical processes that stretch back many generations, and can capture subtle signatures of selection that may not be obvious in natural populations where demographic stochasticity is high, sample sizes are modest, and the specific phenotypes under selection may be unknown. Conversely, field data reveal the range of phenotypic and fitness outcomes that are compatible with genomic signatures of selection. Indeed, genomic evidence alone has led some researchers to posit that the costs of modern human-archaic hominin interbreeding must have been high, reflecting species at the brink of reproductive incompatibility (44, 45). Our results point to the limits of these inferences by indicating that qualitatively similar evidence for selection against introgression can be compatible with primate hybrids that thrive (19-21). This work therefore highlights the crucial role of other primates for understanding human evolution, especially for phenomena that are impossible to study in our lineage alone.