Reinforcement as an initiator of population divergence and speciation

Abstract

When hybridization results in reduced fitness, natural selection is expected to favor the evolution of traits that minimize the likelihood of hybridizing in the first place. This process, termed reinforcement (or, more generally, reproductive character displacement), thereby contributes to the evolution of enhanced reproductive isolation between hybridizing groups. By enhancing reproductive isolation in this way, reinforcement plays an important role in the final stages of speciation. However, reinforcement can also contribute to the early stages of speciation. Specifically, because selection to avoid hybridization occurs only in sympatric populations, the unfolding of reinforcement can lead to the evolution of traits in sympatric populations that reduce reproduction between conspecifics in sympatry versus those in allopatry. Thus, reinforcement between species can lead to reproductive isolation—and possibly speciation—between populations in sympatry versus those in allopatry or among different sympatric populations. Here, I describe how this process can occur, the conditions under which it is most likely to occur, and the empirical data needed to evaluate the hypothesis that reinforcement can initiate speciation.

Introduction

Interbreeding—hybridization—between genetically distinct populations or species often results in reduced fitness. In cases where hybridization carries fitness costs, selection is expected to favor the evolution of traits that either prevent interbreeding in the first place or, if mating occurs, prevent production of hybrid offspring (Dobzhansky 1937; Howard 1993; Servedio and Noor 2003; Coyne and Orr 2004). The evolution of traits that minimize hybrid formation in response to selection is termed reinforcement (Butlin 1987a; Servedio and Noor 2003; Coyne and Orr 2004; Pfennig and Pfennig 2012). Over time, reinforcement should reduce the incidence of hybridization, thereby enhancing the strength of reproductive isolation between species (Jones 1973; Pfennig 2003; but see Britch et al. 2001; Urbanelli et al. 2014 where such was not the case). Essentially, reinforcement acts to finalize the speciation process.

Reinforcement’s role, if any, in speciation has been historically controversial (Howard 1993; Butlin 1995; Servedio and Noor 2003; Coyne and Orr 2004). Nevertheless, empirical data suggest that reinforcement plays a key role in enhancing and strengthening existing species boundaries, thereby contributing to the final stages of speciation (reviewed in Howard 1993; e.g. Noor 1995; Saetre et al. 1997; Rundle and Schluter 1998; Pfennig 2003; Hoskin et al. 2005; Jaenike et al. 2006; Van der Niet et al. 2006; Matute 2010; Hopkins and Rausher 2011). Indeed, evidence in support of reinforcement has accumulated such that controversy surrounding the question of its mere existence has abated (Servedio and Noor 2003; Coyne and Orr 2004; Pfennig and Pfennig 2012; but see Barton 2013; Butlin and Ritchie 2013). Now, the field is turning to evaluate the conditions under which reinforcement occurs and the broader evolutionary and ecological consequences of reinforcement (Pfennig and Pfennig 2012; Abbott et al. 2013).

Here, I focus on 1 key ramification of reinforcement: namely, that the unfolding of reinforcement that enhances divergence between species can, in turn, generate divergence, and possibly even speciation, within species (Figure 1). This hypothesis was put forward by Howard (1993) in his now classic paper on reinforcement and has been discussed subsequently in other reviews and commentaries (Ortiz-Barrientos et al. 2009; Pfennig and Pfennig 2009, 2010, 2012; Hoskin and Higgie 2010, 2013; Nosil 2012; Abbott et al. 2013). In particular, selection to avoid hybridization only occurs in sympatric populations where 2 species (or incipient species) actually encounter each other (Figure 1). Consequently, in sympatric populations, reinforcement leads to the evolution of reproductive traits that minimize deleterious reproductive interactions with heterospecifics. Such traits include, for example, changes in levels of mate discrimination, recognition, or “choosiness” (e.g. Noor 1995; Hudson and Price 2014); mate preferences or sexual traits (see reviews by and references in Howard 1993; Servedio and Noor 2003; Coyne and Orr 2004; Pfennig and Pfennig 2012); and even postmating traits involved in gamete recognition or sperm precedence (reviewed in Howard 1999; e.g. Matute 2010). Because evolutionary changes in these reproductive traits occur only in sympatric populations, reinforcement generates divergent reproductive traits between sympatric and allopatric populations.

Figure 1.

Species often co-occur with other species with which they hybridize. In this region of sympatry (signified with gray and dots), selection will favor divergence in reproductive traits between the 2 species (species 1 and 2 in figure) if hybridization is costly, a process termed reinforcement. Because reinforcement occurs only in sympatric populations, traits in these populations become divergent from ancestral traits in allopatric populations. If these traits generate reproductive isolation, sympatric and allopatric populations can become new species (indicated as species 3 and 4) (modified from Pfennig and Pfennig 2012).

Critically, this reproductive trait divergence can reduce the likelihood of successful reproduction between individuals from sympatry and those from allopatry (McPeek and Gavrilets 2006; Pfennig and Ryan 2006; Lemmon 2009; Porretta and Urbanelli 2012; Pfennig and Rice 2014; Kozak et al. 2015). Note that reinforcement can also initiate reproductive isolation between different sympatric populations if reinforcement proceeds differently in each population (Hoskin et al. 2005), but for simplicity, I refer to divergence between sympatry and allopatry throughout.

The reduced likelihood of interbreeding between individuals from sympatry and those from allopatry essentially sets the stage for speciation between them: populations that are less likely to exchange genes will accumulate further differences (e.g. via adaptive evolution or genetic drift) that can ultimately result in a new species in sympatry derived from that in allopatry (Howard 1993; Pfennig and Pfennig 2009; Pfennig and Pfennig 2012). Indeed, in extreme cases where novel sympatric traits eliminate reproduction between sympatric individuals and allopatric individuals, speciation between these population types would be a direct result of reinforcement’s action in sympatric populations (sensu Hoskin et al. 2005; see also Pfennig and Ryan 2006).

The notion that populations can diverge, and even undergo speciation, owing to differences in their interactions with heterospecifics is not fundamentally different from other speciation models such as ecological speciation, speciation mediated by sexual selection, or even allopatric speciation. As highlighted below, the conditions that promote reproductive isolation between populations that do and do not undergo reinforcement (or that undergo reinforcement differently) are the same conditions that facilitate these alternative speciation scenarios. Thus, the hypothesis that reinforcement can initiate population divergence and speciation draws upon each of these models and, therefore, represents a particular route by which these more general models of speciation occur (especially allopatric speciation and speciation driven by sexual selection).

Nevertheless, the potential for reproductive isolation to arise among populations that differentially experience reinforcement is worth special consideration. Because reinforcement acts on reproductive traits, its action potentially increases the likelihood of generating reproductive isolation among populations relative to divergence in ecological traits. Moreover, reinforcement reflects adaptive evolution in response to selection favoring avoidance of deleterious reproductive interactions between species that possess similar reproductive traits. This divergence between species can generate parallel effects among populations that experienced reinforcement versus ancestral populations that still contain traits resembling those of heterospecifics. In other words, trait divergence that is capable of generating reproductive isolation between species could generate similar levels of reproductive isolation within species.

The goal of this review is to: describe the means by which reinforcement can initiate speciation between conspecific populations; discuss the conditions in which reinforcement is most likely to contribute to population divergence; and outline approaches to testing the hypothesis that reinforcement can drive reproductive isolation—and, possibly, speciation—between conspecific populations that do and do not risk hybridization.

Before doing so, it is important to address potential issues with terminology. I use the term “reinforcement” to refer to the process by which traits evolve as an adaptive response to deleterious hybridization between species. Reinforcement is, therefore, a specific form of the more general process of “reproductive character displacement” (Blair 1974; Pfennig and Pfennig 2012), which is trait evolution that arises as an adaptive response to deleterious reproductive interactions (including “sexual interference” (Gröning and Hochkirch 2008)) between species (Grant 1972; reviewed in Pfennig and Pfennig 2012, see also references therein). I use the term reinforcement below to be synonymous with reproductive character displacement, to which all the same principles below apply.

Additionally, different terms have been used to describe the downstream consequences of reinforcement: “speciation cascades” (Pfennig and Ryan 2006; Pfennig and Pfennig 2009, 2012), “cascade of speciation” (Lemmon 2009), “reinforcement cascades” (Ortiz-Barrientos et al. 2009; Nosil 2012; Nosil and Hohenlohe 2012), and “reproductive character displacement speciation” (Hoskin and Higgie 2010). This article does not seek to reconcile these terms. Instead, I directly describe the phenomena of interest.

Finally, reinforcement can contribute to population divergence under different biogeographic contexts that are rarely made explicit, and failure to do so can create confusion, especially when conspecific populations are characterized as “sympatric” or “allopatric.” In particular, reinforcement’s downstream consequences on population divergence are often characterized in terms of divergence between populations that co-occur with heterospecifics and, therefore, undergo reinforcement (usually referred to as “sympatric populations” or “sympatry”) versus populations that do not encounter heterospecifics (usually referred to as “allopatric populations” or “allopatry”). Such designations represent shorthand for populations that have and have not undergone reinforcement, respectively. However, reinforcement can generate divergence under complex biogeographic scenarios that potentially render such shorthand confusing. For example, 2 isolated populations that co-occur with a heterospecific could undergo reinforcement differently and so diverge (e.g. Hoskin et al. 2005); in this scenario, populations that are both characterized as “sympatric” would be allopatric to each other. Likewise, populations that are sympatric and allopatric with a heterospecific could themselves be parapatric to each other.

As is highlighted below, the distribution—and potential for gene exchange—of conspecific populations that differ in whether or how they undergo reinforcement is potentially critical to whether such populations actually diverge. Biogeographic context, therefore, contributes to reinforcement’s likelihood of initiating speciation among conspecific populations. However, fully articulating all of these possibilities for every point is overly cumbersome in a review such as this, and I, therefore, adopt the approach of previous authors. I use the terms “sympatric populations” or “populations in sympatry” to refer to populations of a focal species that co-occur with heterospecifics and, therefore, undergo reinforcement (or, more generally, reproductive character displacement). I use the terms “allopatric populations” or “populations in allopatry” to refer to populations of a focal species that do not co-occur with heterospecifics and have not undergone reinforcement (or, more generally, reproductive character displacement). The reader is asked to keep in mind throughout that the distribution of these populations of the focal species relative to each other can be complex and impact reinforcement’s downstream consequences, as described below.

How Can Reinforcement Initiate Speciation?

Reinforcement can generate reproductive isolation and thereby initiate speciation between conspecific populations in 2 nonmutually exclusive ways. First, reinforcement can directly generate reproductive isolation between sympatric and allopatric populations. Specifically, reinforcement can generate the evolution of reproductive traits in sympatry that render reproduction between sympatric and allopatric conspecifics less likely. For example, mate preferences that evolve in sympatric populations might cause sympatric females to reject not only heterospecific males as mates but also allopatric males (e.g. Hoskin et al. 2005; Jaenike et al. 2006; Lemmon 2009; Kozak et al. 2015; see also Noor 1999; Pfennig and Ryan 2006). Similarly, sympatric males might evolve sexual signals that distinguish them from heterospecifics, but that make them less attractive to allopatric females (Pfennig and Ryan 2006).

Second, reinforcement can indirectly generate reproductive isolation between sympatric and allopatric populations. In particular, crosses between individuals from sympatry and those from allopatry could produce “hybrid” offspring that have lower fitness in either population type. Consequently, natural selection would favor the evolution of reproductive traits that minimize matings between individuals from sympatric populations and those from allopatric populations (sensu Rundle and Nosil 2005; Nosil 2012). Essentially, the action of reinforcement could generate postmating incompatibilities in the offspring of crosses between individuals from sympatry and those from allopatry. For instance, male offspring derived from crosses between individuals from sympatry and those from allopatry could possess sexual signals that are unattractive to females in either parent population (sensu Svedin et al. 2008). Likewise, “hybrids” could possess inappropriate responses to spatiotemporal cues for reproduction that render them unlikely to locate mates (if reinforcement shifts the timing or location for mating) or hybrids could produce gametes that are less able to fertilize, or are less likely fertilized by, gametes from either population (if reinforcement contributes to evolutionary changes in gamete recognition or sperm precedence). Regardless of the nature of postmating incompatibilities, the production of such offspring would be selectively disfavored. Thus, reinforcement acting in sympatric populations creates the conditions that promote a further round of reinforcement between conspecific populations in sympatry and those in allopatry.

Reinforcement can contribute both directly and indirectly to reproductive isolation in the same system (i.e. the direct and indirect effects are not mutually exclusive). Nevertheless, distinguishing between these alternative routes to reinforcement-generated divergence is important because they could differ in the likelihood of occurrence and the conditions under which they will occur.

In the next section, I discuss a key argument against the hypothesis that reinforcement can initiate reproductive isolation and speciation between sympatric and allopatric populations. In doing so, I describe when reinforcement is likely to generate reproductive isolation between sympatric and allopatric populations (either directly or indirectly) and the conditions favorable to such an outcome.

When Might Reinforcement Initiate Speciation?

Reinforcement occurs between evolutionarily distinct groups (species or incipient species) that are already so diverged that hybridization between them generates offspring with zero fitness (when hybrids are inviable or sterile) or reduced fitness (when hybrids have lower viability or fertility). Consequently, reinforcement enhances reproductive isolation between groups that are already partially reproductively isolated by existing, postzygotic barriers to gene flow (Servedio and Noor 2003; Coyne and Orr 2004).

Such postzygotic barriers do not necessarily exist between conspecific populations in sympatry and those in allopatry. Presumably in the absence of postzygotic barriers, conspecific populations in sympatry and those in allopatry can freely exchange genes with the result that any accumulated trait differences should break down when interbreeding between them occurs. Thus, a major critique of the notion that reinforcement can initiate speciation between sympatric and allopatric populations is that any gene flow between these population types would tend to eliminate divergence between them (note that gene flow between allopatric and sympatric populations is a factor that could prevent reinforcement from occurring in the first place) (Barton 2013; Servedio et al. 2013).

Accordingly, for reinforcement to generate reproductive isolation between populations in sympatry and those in allopatry requires that: 1) conditions exist that limit gene exchange between them; and/or 2) the traits that evolve via reinforcement reduce gene flow sufficiently to maintain—and even enhance—divergence between them (Abbott et al. 2013).

That reinforcement occurs in sympatric populations implies that gene flow from allopatric populations into sympatric populations is not so strong as to prevent the evolution and persistence of distinct sympatric traits that minimize hybridization. Moreover, selection to avoid hybridization can be strong and thereby counteract the homogenizing effects of gene flow from allopatric populations into sympatric populations (for fuller discussion see Servedio and Noor 2003 and references therein). Given that allopatric populations are often assumed to experience weaker selection relative to sympatric populations, how might the reverse be true? In other words, what prevents traits that evolved via reinforcement from spreading into allopatric populations via gene flow, thereby homogenizing the population types? Addressing this issue is a critical step to ascertaining whether and how reinforcement can initiate downstream speciation between sympatric and allopatric populations. The discussion below highlights 4 factors that contribute to divergence, reproductive isolation, and ultimately, speciation between populations in sympatry and those in allopatry.

First, the geographical distribution of sympatric and allopatric populations might limit gene flow between them (Hoskin and Higgie 2010; Abbott et al. 2013). Specifically, distance or barriers that prevent dispersal between them might separate sympatric and allopatric populations. Likewise, sympatric and allopatric populations could occur along an ecotonal boundary such that migrants between the population types fail to succeed in the region to which they are not adapted. Interestingly, support to date for the hypothesis that reinforcement drives reproductive isolation between populations in sympatry and those in allopatry comes from systems with high population structure within population type (indicative of low gene flow) or are isolated by distance or barriers to dispersal (e.g. Hoskin et al. 2005; Jaenike et al. 2006; Lemmon 2009; Porretta and Urbanelli 2012; Pfennig and Rice 2014; Kozak et al. 2015).

In cases where sympatric and allopatric populations are geographically isolated, one could argue that reproductive isolation between them is caused by other factors (e.g. isolation by distance or local adaptation to other ecological variables, including resource competition with the heterospecific with which hybridization occurs) and not reinforcement per se (Barton 2013; Servedio et al. 2013; see also Pfennig and Pfennig 2012). This is an important consideration that must be addressed in evaluating reinforcement’s downstream consequences (see next section). Nevertheless, distance and barriers to dispersal appear to enhance the likelihood that reinforcement will generate reproductive isolation between populations in sympatry and those in allopatry.

In addition to geographic context as a limit to gene flow’s homogenizing effects, a second factor that contributes to reproductive isolation between populations in sympatry and those in allopatry is the outcome of reinforcement itself. Specifically reinforcement in sympatry can result in the reduction or elimination of gene flow between sympatric and allopatric populations. As described above, reinforcement can generate traits in sympatric populations (e.g. mate preferences) that directly reduce or eliminate the likelihood of reproduction between sympatric and allopatric individuals (e.g. Hoskin et al. 2005; Jaenike et al. 2006; Lemmon 2009; Porretta and Urbanelli 2012; Pfennig and Rice 2014; Comeault et al. 2015; Kozak et al. 2015). Consequently, even if individuals from 1 population type disperse into the alternative population type, they would fail to reproduce successfully. Presumably, these direct effects of reinforcement are more likely to maintain reproductive isolation between populations in sympatry and those in allopatry (because gene flow is curtailed by reduced reproduction) than when reinforcement’s effects are indirect (because gene flow occurs, but further reinforcing selection must act to eliminate it).

A criticism of the notion that reinforcement directly generates reproductive isolation between populations in sympatry and those in allopatry argues that such a situation is unlikely to ever occur. In particular, reproductive traits (e.g. female preferences or sexual signals) that arise in sympatric populations likely represent a subset of variants that already occur in allopatric populations, especially in the early stages of reinforcement. Because reinforcement generally acts on the ancestral variation that is present in allopatric populations (Pfennig and Pfennig 2012; see also Rice and Pfennig 2007; Barrett and Schluter 2008). Thus, sympatric reproductive traits brought about by reinforcement might not be so novel in allopatric populations that they would prevent reproduction by migrants from sympatric populations. Moreover, in sympatric populations, reinforcement might lead to enhanced mate discrimination or narrowing of recognition of conspecifics rather than the origin of novel mate preferences (Noor 1999; Hudson and Price 2014). Thus, sympatric individuals might reject allopatric individuals as mates, but the converse need not be true (Lemmon 2009; Kozak et al. 2015). Consequently, matings between the 2 population types could still occur, and even low mating rates could generate enough gene flow to allow sympatric traits to spread into allopatric populations and thereby homogenize the population types.

The above criticism rests in part on the assumption that sympatric mating traits are neutral, and so will not be selectively disfavored, in allopatric populations. Yet, the process of reinforcement in sympatric populations could generate selective barriers to gene flow between populations in sympatry and those in allopatry (Ortiz-Barrientos et al. 2009; Pfennig and Pfennig 2009, 2012; e.g. Pfennig and Pfennig 2005; Higgie and Blows 2008; Hopkins and Rausher 2014). These selective barriers to gene flow represent a third factor that enhances the likelihood that reinforcement’s effects will generate reproductive isolation and possibly speciation between sympatric and allopatric populations.

Selective barriers to gene flow arise when traits that evolve via reinforcement in sympatric populations are strongly disfavored in allopatric populations whereas traits from allopatric populations are strongly disfavored in sympatric populations. Allopatric reproductive traits are disfavored in sympatric populations because they increase the likelihood of hybridization (as indicated above, selection in sympatric populations is generally assumed to be strong and sufficiently high to counteract any effects of gene flow from allopatric populations). But how can the converse be true: that sympatric reproductive traits, which evolve via reinforcement, become strongly disfavored in allopatric populations?

For reproductive traits, phenotypes that evolve in sympatric populations are unlikely to be selectively neutral in allopatric populations (Pfennig 2000; Higgie and Blows 2007; Hopkins and Rausher 2014). Consider, for example, mating behaviors: Females are expected to evolve preferences that enhance their fitness, and males are expected to evolve traits that attract females by advertising their ability to enhance female fitness (Andersson 1994). When reinforcement acts in sympatric populations, it potentially disrupts existing patterns of sexual selection in which females prefer male signals indicative of a male’s ability to provide females and their offspring with fitness benefits (a process sometimes referred to as mate-quality recognition) (Figure 2; sensu Ryan and Rand 1993; Pfennig 1998). When sympatric females evolve preferences that better identify conspecifics (or sympatric males evolve distinct signals from those of heterospecifics), the resulting mating traits might be less suited for identifying high quality mates (or signaling quality) (Pfennig 1998; e.g. Pfennig 2000, 2008; Higgie and Blows 2007, 2008). Indeed, although sympatric females might enhance their fitness by identifying conspecific males with signals that are dissimilar from heterospecifics, these males might be of relatively low quality (Figure 2; Gerhardt 1994; Pfennig 2000; see also Gabor et al. 2010 for a trade-off involving male mate choice; but see Lemmon 2009 for a case where sympatric males evolved more costly traits).

Figure 2.

(A) If traits that evolve via reinforcement in sympatric populations are disfavored in allopatric populations, whereas traits from allopatric populations are disfavored in sympatric populations, selective barriers to gene flow can arise. (B) One such trade-off occurs when females use exaggerated traits to identify males that provide them or their offspring with fitness benefits in allopatric populations. If reinforcement generates preferences for less exaggerated traits (because such traits differ from those possessed by heterospecifics) in sympatric populations, these preferences will be selectively disfavored in allopatric populations (because females will fail to identify fitness-enhancing mates). By contrast, preferences from allopatric populations will be disfavored in sympatric populations because they enhance hybridization risk. Thus, selective trade-offs between population types can generate opposing patterns of selection on male signals while increasing the chances that females will reject mates of the opposite population type.

Similar selective trade-offs could arise involving postmating traits, such as sperm and egg compatibilities or components of seminal fluid that enhance sperm competitive ability. In sympatric populations, reinforcement can favor the evolution of sperm and egg recognition proteins that reduce hybrid zygote formation (Matute 2010). Reinforcement can also impact traits that foster conspecific sperm precedence (Howard 1999). These traits can, in turn, be selectively disfavored in allopatric populations if they result in reduced fertilization success or reduced sperm competitive ability in allopatric populations (Comeault et al. 2015).

Such traits represent a “best of a bad situation” (Pfennig and Pfennig 2005): They are favored in sympatric populations when the risk and costs of hybridization are worse than the fitness costs that are otherwise incurred with such traits. But when these sympatric traits are expressed in allopatric populations, they will be selectively disfavored relative to the ancestral traits prevailing in those populations (Pfennig and Pfennig 2005; Higgie and Blows 2007; Hopkins and Rausher 2014). Regardless of how selective trade-offs arise, the key point is that reproductive traits that arise in sympatric populations can be costly, and, therefore, selectively disfavored, in allopatric populations. These conditions will enhance the likelihood that reinforcement will foster reproductive isolation, and possibly speciation, between conspecific populations in sympatry and those in allopatry.

A final, fourth, factor that could determine whether reinforcement drives divergence, and ultimately speciation, between populations in sympatry and those in allopatry is the potential for reinforcement to generate effects on other aspects of the phenotype beyond those involved in reproduction (Konuma and Chiba 2007; Pfennig and Pfennig 2009, 2012). Changes in reproductive traits are potentially accompanied by changes in other traits that are not directly related to reproduction (Pfennig 2008). For example, shifts in male signaling could be accompanied by changes in morphology that affect resource use (e.g. bird beaks affect both mating song production and resource acquisition (Podos and Nowicki 2004)) or modifications to the physiology and energy storage mechanisms that mediate signal production (e.g. males might alter their investment in long-term versus short-term energy stores [sensu Tomkins et al. 2004]). Similarly, changes in the timing or location of reproduction to avoid heterospecifics might expose sympatric populations to new selective environments (e.g. novel temperature regimes, lighting conditions, or predation risk) that affect not only reproductive traits but also nonreproductive traits associated with them (Boughman 2007).

These extended effects enhance the potential for reinforcement to indirectly promote reproductive isolation between sympatric and allopatric populations (Pfennig and Pfennig 2012). The accumulation of trait differences across a suite of traits beyond those strictly involved in reproduction enhances that chances for genetic incompatibilities to arise that contribute to low fitness in any offspring that might be produced by sympatric–allopatric crosses. Moreover, divergence across traits involved in habitat selection, adaptation to local habitats, and resource use, enhances the potential for “extrinsic incompatibilities” in which sympatric–allopatric offspring are unable to succeed in either the ancestral allopatric niche or the novel, reinforcement-generated, sympatric niche. When either genetic or extrinsic incompatibilities arise between sympatric and allopatric populations, natural selection will favor reduced mating between them: i.e. reinforcement. Thus, the effects of reinforcement can generate the conditions that promote a subsequent round of reinforcement between sympatric and allopatric populations (Rundle and Nosil 2005; Nosil 2012).

In sum, whether reinforcement in sympatric populations concomitantly promotes population divergence, reproductive isolation, and possibly speciation, between populations in sympatry and those in allopatry depends on biogeographic context, the nature of sympatric traits that evolve via reinforcement, the fitness consequences of those traits in allopatric populations, and the effects of reinforcement on nonreproductive traits. Moreover, the very conditions (e.g. geographic separation of sympatric and allopatric populations) that foster reinforcement will potentially enhance reinforcement’s downstream effects on divergence and reproductive isolation between populations in sympatry and those in allopatry. As a final note, the above factors can also generate selection against dispersal behaviors that underlie gene flow (Yukilevich and True 2006), thereby strengthening reproductive isolation between sympatric and allopatric populations.

The above discussion focuses on pairwise interactions between sympatric and allopatric populations, implying that a single speciation event follows reinforcement acting in sympatric populations. However, species distributions (and concomitantly, co-occurrence with other species) can be patchy and a focal species can co-occur with different species in different parts of its range. Moreover, reinforcement between any 2 species in different sympatric populations need not unfold in the same way (i.e. reinforcement might not lead to parallel evolution) or to the same extent (Hoskin et al. 2005; Jaenike et al. 2006; Lemmon 2009). Consequently, sympatric populations can become reproductively isolated from allopatric populations and/or from each other (Hoskin et al. 2005), leading to a series of speciation events. That reinforcement (and, more generally, reproductive character displacement) can drive such diversification has both theoretical and empirical support (McPeek and Gavrilets 2006; Pfennig and Ryan 2006; Lemmon 2009). Thus, reinforcement’s downstream consequences might contribute to broader patterns of diversification (Ortiz-Barrientos et al. 2009; Pfennig and Pfennig 2009, 2010; Hoskin and Higgie 2010), including adaptive radiation (sensu Schluter 2000).

How Might the Hypothesis that Reinforcement Initiates Speciation be Tested?

Finding that populations in sympatry and those in allopatry are genetically distinct is consistent with the hypothesis that reinforcement has generated reproductive isolation between conspecific populations. Likewise, finding that sympatric and allopatric populations differ in reproductive traits is consistent with the hypothesis that reinforcement has initiated divergence between them. Nevertheless, both lines of evidence are insufficient to either: 1) ascertain whether reinforcement in sympatric populations has initiated reproductive isolation between sympatric and allopatric populations; or 2) rule out alternative hypotheses explaining divergence between populations in sympatry and those in allopatry.

Critically, populations in sympatry and those in allopatry can diverge when they are geographically isolated. These populations might accumulate genetic and phenotypic differences—including differences in reproductive traits that foster assortative mating by population type—because of genetic drift, mutation order processes, or selection favoring local adaptation to their particular habitat (Ritchie 2007; Price 2008; e.g. McKinnon et al. 2004; Ólafsdóttir et al. 2006; Etges et al. 2010). Sympatric and allopatric populations can also diverge owing to the evolution of traits in sympatric populations that minimize resource competition between heterospecifics (Pfennig and Pfennig 2009, 2012). This process of ecological character displacement can engender divergence, including reproductive isolation, between sympatric and allopatric populations (Pfennig and Rice 2007; Rice and Pfennig 2010). Moreover, ecological character displacement can affect reproductive traits (Pfennig and Pfennig 2009, 2012 and references therein), and such action must necessarily be ruled out in ascribing divergence between populations in sympatry and those in allopatry to the action of reinforcement (Rundle and Schluter 1998). Thus, in any of these circumstances, reinforcement (and, more generally, reproductive character displacement) does not drive divergence between sympatric and allopatric populations (Pfennig and Pfennig 2012; Barton 2013; Servedio et al. 2013; cf. Hoskin and Higgie 2010). Ruling out these possibilities is a necessary component to establishing reinforcement’s role in divergence between sympatric and allopatric populations.

Moreover, divergence between sympatric and allopatric populations in reproductive traits does not provide evidence that such traits serve as reproductive isolating barriers that reduce gene exchange between the population types. If individuals from sympatry and those from allopatry still reproduce successfully despite divergent reproductive traits, then reinforcement would not contribute to reproductive isolation between sympatric and allopatric populations.

Ascertaining the downstream consequences of reinforcement, therefore, requires, first and foremost, determining that reinforcement has indeed occurred in sympatric populations. Because sympatric and allopatric populations can differ for reasons that are not related to reproductive interactions with heterospecifics, establishing that sympatric traits diverged from allopatric traits because of reinforcement is necessary to evaluating reinforcement’s downstream consequences. The criteria for (and difficulties with) establishing whether reinforcement has occurred are discussed elsewhere (e.g. Butlin 1987b; Howard 1993; Butlin and Ritchie 1994; Noor 1999; Servedio and Noor 2003). Assuming that trait evolution via reinforcement in sympatric populations is established, it then becomes possible to ascertain whether the traits that evolved via reinforcement subsequently contribute to reproductive isolation between sympatric and allopatric populations.

A direct means of evaluating whether reinforcement contributes to reproductive isolation is to ascertain whether traits that evolve via reinforcement diminish the likelihood of reproduction or reproductive success between individuals in sympatry and those in allopatry (Hoskin et al. 2005; Lemmon 2009). In the case of mating behaviors, females might evolve mating preferences so that, when presented with males from allopatry, sympatric females reject such males as mates (Hoskin et al. 2005; Lemmon 2009; Pfennig and Rice 2014; Kozak et al. 2015). Likewise, if sympatric males evolve sexual signals that are distinct from heterospecifics, such signals might cause them to be rejected by allopatric females (but see Lemmon 2009; Kozak et al. 2015). This pattern would be expected if reinforcement directly generates reproductive isolation between sympatric and allopatric populations. In these cases, tests of mating success would come via behavioral experiments measuring female preferences for males of both population types or direct mating trials between sympatric and allopatric individuals to determine if mating behaviors reduce reproduction between individuals from sympatry and those from allopatry.

A necessary caveat to this approach is that the traits that are identified as playing a key role in reinforcement might not directly translate into isolating mechanisms between sympatric and allopatric populations. For example, a key line of evidence supporting the reinforcement hypothesis is that sympatric females discriminate against heterospecific males whereas allopatric females do not (e.g. Noor 1995). For these data, the focal behavior that appears to have evolved via reinforcement is discrimination of heterospecifics (sensu Ryan and Getz 2000). Yet, this behavior alone would be difficult to establish as a contributing factor to reproductive isolation, if any, between populations in sympatry and those in allopatry. The failure of allopatric females to distinguish between heterospecific and conspecific males does not mean they will mate indiscriminately with sympatric conspecifics. Instead, in this scenario, it becomes necessary to ascertain how the evolution of discrimination against heterospecifics impacts female mate preferences among conspecifics in sympatric populations relative to those preferences in allopatric populations (sensu Ryan and Getz 2000; Pfennig and Ryan 2006, 2007).

For example, Lemmon (2009) used chorus frogs (Pseudacris) to combine 2 approaches: female discrimination of conspecifics from heterospecifics and female preferences for conspecifics. She found that sympatric females discriminated conspecifics from heterospecifics, whereas allopatric females did not. Importantly, she also evaluated sympatric female preferences for conspecific males from sympatric populations versus those from allopatric populations. She found that sympatric females preferred their own male type (although allopatric females also tended to prefer the sympatric males) (Lemmon 2009). The combination of these different types of behavioral data, therefore, provided not only a test of reinforcement but also an evaluation of the potential for reinforcement to generate reproductive isolation between sympatric and allopatric populations (but see below).

Reinforcement can also lead to the evolution of sympatric postmating traits that contribute to isolation between populations in sympatry and those in allopatry (Comeault et al. 2015). Detecting whether these postmating traits contribute to reproductive isolation requires measuring the outcome of matings between sympatric individuals and allopatric individuals (Comeault et al. 2015).

Regardless of whether premating or postmating reproductive traits are the focus of study, evaluating whether reinforcement contributes to reproductive isolation between populations in sympatry and those in allopatry requires that crosses between population types be reciprocal (meaning that matings or behavioral assays involve males and females from allopatric and sympatric populations). Reciprocal tests are critical, because premating or postmating incompatibilities can occur in 1 direction only (Lemmon 2009; Comeault et al. 2015; Kozak et al. 2015). For example, sympatric females might reject allopatric males, whereas allopatric females might mate readily with sympatric males. Such would be the case if sympatric females evolved enhanced discrimination or more narrow windows of recognition (Noor 1999; Ryan and Getz 2000; Hudson and Price 2014). Likewise, sympatric sperm might be incompatible with eggs from allopatry whereas allopatric sperm might be compatible with both sympatric and allopatric eggs. When such directionality in the likelihood of reproduction occurs, it would be necessary to evaluate dispersal patterns to establish if they are bidirectional or one-way, and in the case of the latter, whether the direction of dispersal coincides with the direction of discrimination or incompatibility (and so lead to reproductive isolation) or not (and so contribute to gene flow).

Generally, finding that sympatric, but not allopatric, individuals favor members of their own population types (or that postmating incompatibilities are unidirectional) is problematic for the hypothesis that reinforcement can contribute to reproductive isolation and speciation between sympatric and allopatric populations. This is for the reasons described above: Gene flow mediated by the reproductive traits of allopatric individuals would tend to counteract divergence between sympatric and allopatric populations. Moreover, finding that sympatric individuals possess pre- or postmating traits that reduce reproduction with allopatric individuals might not be a downstream effect of reinforcement. Instead, such trait evolution might be a component of the reinforcement process per se that enabled local adaptation (i.e. reinforcement) to occur in the face of gene flow from allopatric populations. Such would be the case if selection in sympatric populations disfavors both mating with heterospecifics and mating with migrants from allopatric populations (sensu Yukilevich and True 2006). Consequently, reinforcement might result in one-way incompatibilities such that allopatric individuals would be reproductively unsuccessful in sympatric populations but sympatric individuals could reproduce in allopatric populations. In such a case, divergence between sympatric and allopatric populations to the point of speciation might be unlikely because of continued gene flow (albeit directional) between the populations. Teasing apart these different issues is necessary to determining whether reinforcement generates reproductive isolation and speciation between sympatric and allopatric population types.

A further means of testing the hypothesis that reinforcement contributes to reproductive isolation is to ascertain whether sympatric and allopatric populations are more genetically divergent than expected based on other factors (Abbott et al. 2013). If sympatric and allopatric populations are reproductively isolated, these population types should experience lower gene flow relative to the amount of gene flow that occurs within population type (Pfennig and Rice 2014). Such a pattern would rule out isolation by distance (though it would not rule out divergence driven by ecological factors [Rice and Pfennig 2010]). Note that the absence of any genetic differences, especially in neutral markers, would not exclude the hypothesis that reinforcement drives reproductive isolation between populations in sympatry and those in allopatry particularly if divergence is recent and differences have not had time to accumulate (Abbott et al. 2013).

Further evidence of reinforcement’s role in divergence between sympatric and allopatric populations can be gleaned from systems that show variation in selection against hybridization. In these systems, sympatric populations of a given species that experience stronger selection against hybridization show more pronounced trait divergence (and greater reproductive isolation) from heterospecifics—and ancestral allopatric populations—than those sympatric populations with weaker selection against hybridization (Waage 1975, 1979; Pfennig and Pfennig 2005). Evaluating whether trait divergence in pairwise contrasts between sympatric populations versus allopatric populations predicts reproductive isolation between them would test the efficacy of reinforced traits in reproductive isolation. Specifically, sympatric versus allopatric populations that exhibit greater trait divergence should show higher levels of reproductive isolation (e.g. as measured by reproductive success in sympatric × allopatric pairings or genetic differentiation) than sympatric versus allopatric populations that exhibit lower trait divergence.

Evidence of reinforcement’s downstream effects could also be obtained from phylogenetic and comparative analyses that evaluate whether diversification patterns are consistent with the possibility that reinforcement drives reproductive isolation between populations in sympatry and those in allopatry. For example, species that are sympatric with heterospecifics should be younger than, and derived from, species in allopatry. Moreover, diversification should be higher in taxonomic groups (or geographic regions) that experience greater opportunities for hybridization or other deleterious reproductive interactions between species. Such evidence would need to be considered in light of the alternative causes of reproductive isolation outlined above.

In sum, determining whether reinforcement generates divergence between sympatric and allopatric populations requires many of the same approaches used to evaluate reproductive isolation between any incipient species pair (Coyne and Orr 2004; Hendry 2009; Sobel et al. 2010; Nosil 2012). Nevertheless, establishing that reproductive isolation exists between sympatric and allopatric populations is insufficient; such patterns must be accompanied by evidence that the isolation is attributable to reinforcement acting in sympatric populations. Finding this evidence will be most straightforward when reinforcement in sympatric populations directly contributes to reproductive isolation between sympatric populations and allopatric populations. However, reciprocal tests of isolation are needed between populations in sympatry and those in allopatry (i.e. finding only that sympatric individuals do not reproduce with those from allopatry is insufficient to show reproductive isolation or speciation between sympatric and allopatric populations). Identifying reinforcement’s indirect impact on reproductive isolation between populations in sympatry and those in allopatry will be more challenging, especially when population divergence arises from downstream effects that could be confounded with direct selection from other environmental differences between the population types.

Final Caveats

Much of the above discussion regarding reinforcement’s effects on reproductive isolation rests on the assumptions that: population types remain stable (sympatric populations remain sympatric and allopatric populations remain allopatric) and the conditions observed in present studies reflect progress in the process of speciation (i.e. our contemporary studies predict future diversity).

Yet, species distributions are dynamic, not fixed, over both ecological and evolutionary time scales. Moreover, sympatric populations often occur at the edges of species ranges where populations are typically smaller and at higher risk of extinction. Thus, over time, selection to avoid hybridization, and any associated trait evolution, would be reversed if a previously sympatric population becomes allopatric because the species with which it co-occurs goes locally extinct. Consequently, the selective dynamics that generate—and maintain—divergence between sympatric and allopatric populations could wax and wane depending on the population and range dynamics of the interacting species.

Moreover, even populations that are sufficiently diverged to be considered possibly “good” species can collapse in response to changes in environment that are unrelated to the effects of reinforcement (sensu Seehausen et al. 1997; Hendry et al. 2006; Grant and Grant 2008; Behm et al. 2010). Thus, contemporary studies do not necessarily reflect the ultimate fate—in terms of whether or not speciation actually occurs—of diverged populations (sensu Hendry 2009; Nosil et al. 2009).

At some level, the possibility that conditions can change and thereby undermine our ability to predict the evolutionary future of populations applies to almost any evolutionary question that is answered with empirical data from natural systems. Nevertheless, directly addressing these criticisms could help us better understand whether and when reinforcement will have the downstream consequences described above (sensu Hendry 2009; Nosil et al. 2009). For example, comparative analyses could reveal whether species with more stable populations or range overlap are more likely to show reproductive isolation between populations in sympatry and those in allopatry when contrasted with those species that have more variable population and range dynamics. Moreover, although it will never be possible to know what the future holds, long-term studies and the use of museum collections (e.g. Rowe et al. 2011; Bi et al. 2013) can be used to test predictions about progress toward speciation (sensu Jones 1973; Britch et al. 2001; Pfennig 2003; Grant and Grant 2008; Urbanelli et al. 2014; see also Brodersen and Seehausen 2014 for a nice discussion of the value of long-term data and ecological monitoring). For example, long-term and historical data can reveal whether or not sympatric and allopatric populations are becoming increasingly genetically divergent in response to reproductive trait evolution between them.

Additionally, a better understanding of how environmental factors and their change break down population divergence would provide enhanced insight into what systems might be more or less likely to proceed to speciation (Grant and Grant 2008; Nosil et al. 2009). For example, changes in water clarity can obscure visual signaling in aquatic species and so contribute to random mating and the breakdown of population divergence (Seehausen et al. 1997). Species that rely on visual signals and experience (or are likely to experience) changes in water clarity, therefore, might be less likely to proceed to speciation than systems using less readily obscured cues or that occur in more stable habitat. Likewise, systems in which divergence occurs along a single trait axis might be less (or more) likely to proceed to speciation despite environmental perturbations than systems in which divergence occurs across a suite of traits (Nosil et al. 2009).

Sympatric and allopatric populations might never reach speciation’s end point of becoming “good species” (an already vague endpoint that depends in part on the problematic issue of defining what constitutes a “good species” [Coyne and Orr 2004]). When biodiversity represents more than a simple species count, this might not matter. If biodiversity reflects variation both within and between species (Wilson 1988), and if the goal is to determine how biodiversity arises and is maintained, then understanding reinforcement’s diversifying effects is in no way undercut if the ultimate product does not add to the species count.

Conclusions

When hybridization generates fitness costs, natural selection should favor the evolution of traits that minimize the chances of hybridization—a process that occurs where the 2 species co-occur and risk hybridization. Reinforcement acts only in sympatric populations, so it generates reproductive differences between conspecific populations in sympatry and those in allopatry. By diverging in reproductive traits, populations in sympatry and those in allopatry have a high likelihood of becoming reproductively isolated. The likelihood that reproductive isolation will arise in this way is enhanced by factors (e.g. geographic isolation) that reduce the potential for gene flow between population types—including the action of reinforcement itself.

Further work is needed to evaluate reinforcement’s impact on evolutionary diversification. To date, reinforcement has been perceived as a process that finalizes speciation between already diverged groups (indeed, many cases of reinforcement involve taxa that are already deemed separate species [Howard 1993; Servedio and Noor 2003]). Yet, by its very action, reinforcement generates divergence between conspecific populations in sympatry and those in allopatry. Moreover, the conditions that enable reinforcement to occur (reduced gene flow between allopatric and sympatric populations) might be the same factors that enable reinforcement to initiate the differences between sympatric and allopatric populations that set the stage for future speciation events. Thus, reinforcement might simultaneously finalize speciation while initiating subsequent bouts of speciation.