What do we mean when we talk about hybrid speciation?

In their recent article Nieto Feliner and colleagues discuss in detail our previous article “How common is homoploid hybrid speciation?” They conclude that our criteria for homoploid hybrid speciation are so stringent that they will cause researchers to overlook many of the important contributions of hybridization to evolution and speciation (Schumer et al. 2014; Nieto Feliner et al. 2017). In particular, the authors object to our proposal that for homoploid hybrid speciation to be invoked, there should be evidence that hybridization was important in generating reproductive isolation between hybrids and their parental species. We recognize that this view is one shared by many in the community (Seehausen 2013; Liu et al. 2014; Lavretsky et al. 2015; Nieto Feliner et al. 2017), yet it is still an area of avid discussion and disagreement among evolutionary biologists (Mavarez and Linares 2008; Hermansen et al. 2014; Capblancq et al. 2015; Payseur and Rieseberg 2016).

While Nieto Feliner et al. suggest that the criteria we propose for establishing homoploid hybrid speciation may limit our understanding of the importance of hybridization in evolution, we argue here that such an approach is in fact crucial for appreciating its importance.

As a field, our understanding of the extent of hybrid ancestry in the genome has far outpaced our progress in understanding its evolutionary impact. We view this as one of the major challenges in contemporary evolutionary biology. Whole-genome sequencing approaches in the past decade have revealed dozens of cases of hybridization that were previously unrecognized (Mallet 2005; Mallet 2008; Abbott et al. 2013), with anywhere from ~ 2% of the genome up to nearly 50% of the genome derived from admixture (e.g., Rieseberg et al. 2003a; Heliconius Genome 2012; Stukenbrock et al. 2012; Cui et al. 2013; Sankararaman et al. 2014; Trier et al. 2014; Vernot and Akey 2014; Sankararaman et al. 2016; Wall et al. 2016; Meier et al. 2017). These empirical observations have caused a massive shift in our views about the frequency and potential importance of hybridization in evolution. In contrast, we understand the ecological, evolutionary, or genetic factors shaping hybrid ancestry in the genome in only a handful of cases (e.g., Rieseberg et al. 2003a; Heliconius Genome 2012; Trier et al. 2014; Leducq et al. 2016).

Does empirical evidence of extensive hybrid ancestry in the genome mean that hybridization has played an important role in the diversification of that lineage? We focus on an example from our own work that we find particularly helpful in illustrating the complexity of this question. The genome of the swordtail fish Xiphophorus nezahualcoyotl is derived from admixture between a lineage ancestral to its sister species Xiphophorus montezumae and another species, Xiphophorus cortezi (we originally proposed this species as a possible case of hybrid speciation; Cui et al. 2013). We estimate that this hybridization resulted in mixture proportions of about 90% of the genome derived from the X. montezumae lineage and 10% from the X. cortezi lineage (Schumer et al. 2016). Strikingly, hybrid ancestry in X. nezahualcoyotl’s genome has apparently rapidly stabilized (i.e., fixed for one parental species ancestry or the other) and has done so uniformly across its range. This type of pattern is as or more extreme than genomic patterns that have been interpreted as evidence of hybrid speciation in other studies (e.g., Gompert et al. 2006; Buerkle and Rieseberg 2008; Stukenbrock et al. 2012). Importantly, X. nezahualcoyotl falls under the definition by Nieto Feliner et al. of a homoploid hybrid species—it is established, persistent, and morphologically and ecologically distinct from its parent species.

However, a closer examination of patterns of hybrid ancestry in X. nezahualcoyotl’s genome complicates this picture. We see evidence that hybrid ancestry from the X. cortezi lineage is systematically depleted from functionally important regions of the genome. Thus, with our current data, it is difficult to reject the alternative hypothesis that the extensive admixture we observe in X. nezahualcoyotl is the neutral remnant of a process of purging hybrid ancestry from the most functionally important regions of the genome.

Just because there is evidence for extensive selection against hybrid ancestry in the genome in cases like X. nezahualcoyotl does not mean other mechanisms are not at play—it is still possible that ecological, sexual, or genetic traits derived from hybridization have importantly shaped the evolution of this species and others—but this is a hypothesis that should be tested. Such cases highlight our view that as a field we have been making conclusions about the role of hybridization in speciation based on too little information. For example, using the definition proposed by Nieto Feliner et al., a case where an already ecologically and morphologically distinct species experiences neutral gene flow from a closely related species could satisfy the required evidence for hybrid speciation.

Several groups have implicitly used a more stringent definition than that proposed by Nieto Feliner and colleagues—specifically that the morphologically or ecologically distinct traits in the hybrid lineage are plausibly or demonstrably a result of hybridization between the two parental species. Though such observations can provide evidence that admixture has had phenotypic effects, it will also almost certainly be the case when large proportions of the genome are derived from hybridization. To us, the question remains of whether these hybridization-derived phenotypes promote isolation through mechanisms such as assortative mating, ecological, or genetic divergence or are simply byproducts of admixture.

One concern raised by Nieto Feliner et al. is that the criteria for homoploid hybrid speciation that we propose will lead to an under-appreciation of the importance of hybridization in generating new species because few species will be well-studied enough to satisfy these criteria. On the other hand, less-stringent definitions of hybrid speciation have led to a rapid increases in the number of proposed cases (see Fig. 2 from Schumer et al. 2014). Because so many species have hybrid ancestry in their genomes, and are also morphologically and ecologically distinct from the taxa from which this hybrid ancestry was derived (e.g., Cui et al. 2013; Li et al. 2016; Schumer et al. 2016; Malinsky et al. 2017; Meier et al. 2017; Tung and Barreiro 2017; Turissini and Matute 2017), the definition proposed by Nieto Feliner et al. could lead to the conclusion that a huge number of species originated by hybrid speciation instead of forms of speciation historically thought to be more common, particularly in animals (e.g., allopatric speciation followed by secondary gene flow; but see Grant 1971). Although possible—and it is important not to reject the possibility of hybrid speciation offhand—our argument is that we do not yet have enough information in most cases to link the presence of hybrid ancestry in the genome to the process of speciation by hybridization.

This highlights the crux of the problem from our perspective: despite a wealth of recent studies demonstrating the nature and extent of hybridization between species, many fewer ask about its function and the evolutionary processes shaping it. This is in part because we have just begun to ask these questions from a genomic perspective and much foundational work documenting the nature of the problem was needed. The challenge we face now is understanding the function of hybridization in the genome, how and in what scenarios hybrid ancestry experiences selection, and ultimately how it may contribute to speciation. On the one hand, hybridization may introduce genes that through recombination or epistasis generate phenotypic novelty that allows hybrids to thrive in new ecological niches (Gross and Rieseberg 2005; Rieseberg et al. 2003b), or provide sufficient “raw material” for the evolution of a novel hybrid lineage (Seehausen 2013). On the other, hybridization can unleash an avalanche of genetic conflict with deleterious effects (e.g., Masly and Presgraves 2007). Nor are these possibilities mutually exclusive, as exemplified by the work on human–Neanderthal admixture (Sankararaman et al. 2014; Vernot and Akey 2014; Juric et al. 2015; Racimo et al. 2015; Harris and Nielsen 2016). Teasing apart the impacts of selection against hybrid ancestry genome-wide and positive selection on individual hybrid traits is a complex challenge that will require new ideas, theory, and approaches, particularly because genome-wide signals of selection against hybridization may swamp out adaptive signals at fewer loci. Further, functional work addressing the evolutionary ecology of hybrid phenotypes can provide an important complement to this work (e.g., Rieseberg et al. 2003a). These approaches present substantial intellectual and technical challenges but will ultimately be necessary in moving as a field from understanding patterns of hybrid ancestry to understanding the evolutionary processes at play.

We think a notable parallel to the debate over homoploid hybrid speciation is the neutralist–selectionist debate in population genetics of the 1970s and 1980s. The neutral theory, which proposed that effectively all variation observed at the DNA level within and between species is either neutral or slightly deleterious, was an important model for understanding genetic variation (Kimura 1968; Kimura 1983), and provided clear, testable hypotheses for researchers searching for evidence of adaptation at the molecular level. Though it is now apparent that the neutral theory cannot explain many features of genetic variation, it provided a simple and powerful framework for understanding how evolutionary forces act on the genome and at the same time has allowed us to more clearly understand cases in which it fails (Kreitman 1996). We think a similar argument can be made about the importance of carefully evaluating alternative hypotheses in studies of hybrid speciation. For the neutralist–selectionist debate the development of new theory and methods in the last two decades (e.g., Hudson et al. 1987; Tajima 1989; McDonald and Kreitman 1991; Zhang et al. 1998), in combination with more data, has made it clear that neutral evolutionary processes fail to explain certain patterns in the data (Kreitman 1996; Fay and Wu 2001; Hahn 2008; Sella et al. 2009). Seriously considering the simple models of neutral or deleterious admixture when evaluating putative cases of hybrid speciation may similarly lead to new approaches that substantially change our views about the frequency of homoploid hybrid speciation. In other words, we acknowledge that our view that homoploid hybrid speciation is relatively rare (Schumer et al. 2014) could reflect a lack of the appropriate kind of evidence rather than the scarcity of the phenomenon. Distinguishing between these alternatives should be a key focus of research going forward.

One important issue raised by Nieto Feliner et al. is that their definition of homoploid hybrid speciation addresses the challenge faced by empiricists of understanding the evolution of genomes and species in the context of hybridization. Since hybridization is an important feature of the evolutionary history of these species, we agree that describing the nature of hybrid ancestry in a lineage is essential, but propose that it can be done without invoking a particular speciation mechanism in cases where direct evidence is lacking. Specifically, observing hybrid ancestry in the genome provides direct evidence that the species has an admixed genome (or even a genome of hybrid origin) but does not necessarily tell us about the evolutionary processes that gave rise to that species. Thus, although we fully agree with Nieto Feliner et al. that hybridization is an important force driving the evolution of genomes and species, we believe that understanding the evolutionary importance of hybridization requires acknowledging its ubiquity while carefully asking questions about its functional consequences, including whether hybridization frequently leads to reproductive isolation between hybrids and their parental species.

Compliance with ethical standards

Conflict of interests

The authors declare that they have no conflict of interest.