Do genetic loci that cause reproductive isolation in the lab inhibit gene flow in nature?

Abstract

The genetic dissection of reproductive barriers between diverging lineages provides enticing clues into the origin of species. One strategy uses linkage analysis in experimental crosses to identify genomic locations involved in phenotypes that mediate reproductive isolation. A second framework searches for genomic regions that show reduced rates of exchange across natural hybrid zones. It is often assumed that these approaches will point to the same loci, but this assumption is rarely tested. In this perspective, we discuss the factors that determine whether loci connected to postzygotic reproductive barriers in the laboratory are inferred to reduce gene flow in nature. We synthesize data on the genetics of postzygotic isolation in house mice, one of the most intensively studied systems in speciation genetics. In a rare empirical comparison, we measure the correspondence of loci tied to postzygotic barriers via genetic mapping in the laboratory and loci at which gene flow is inhibited across a natural hybrid zone. We find no evidence that the two sets of loci overlap beyond what is expected by chance. In light of these results, we recommend avenues for empirical and theoretical research to resolve the potential incongruence between the two predominant strategies for understanding the genetics of speciation.

The genetic basis of reproductive isolation

Viewing speciation through the lens of genetics

An influential definition of species posits that new species form by accumulating barriers to reproduction (Mayr, 1942). Within this framework, researchers seek to understand the genetics of reproductive isolation for two reasons. First, those barriers to gene exchange that are inherited are more likely to persist over time, leading to stable species. Second, by discovering the numbers, frequencies, genomic locations, phenotypic effects, and molecular mechanisms of mutations that generate reproductive isolation, we learn key ingredients in the origin of species.

Genetic mapping of isolation phenotypes in the lab

When reproductive barriers are evolving but incomplete, they can be genetically dissected in experimental crosses by finding DNA variants that co-segregate with relevant phenotypes, such as reductions in the fertility or viability of hybrids. The ability to standardize the environment in which offspring are raised makes this linkage mapping approach well-suited to identify genomic regions and genes connected to intrinsic postzygotic isolation. The genetic mapping of reproductive barriers in the laboratory was pioneered by Dobzhansky (1936) and has enjoyed a renaissance beginning in the 1980s (Coyne, 1992). Species that are easy to breed in the laboratory have seen the most progress, including species of monkeyflowers (Fishman et al., 2013; Zuellig & Sweigart, 2018a), Arabidopsis (Chae et al., 2014; Vaid & Laitinen, 2019), rice (Ouyang et al., 2010), fruit flies (Brideau et al., 2006; Phadnis et al., 2015; Presgraves & Meiklejohn, 2021; Presgraves et al., 2003), swordtails (Malitschek et al., 1995; Moran et al., 2024; Wittbrodt et al., 1989), and house mice (Forejt et al., 2021; Mihola et al., 2009; Turner et al., 2014). To date, many genomic regions and a handful of specific genes have been linked to phenotypes involved in postzygotic isolation. Although we focus this Perspective on postzygotic isolation, progress has also been made toward understanding the genetics of barriers that prevent the formation of hybrids (prezygotic isolation) (Coyne & Orr, 2004; Davis et al., 2021; Huang et al., 2023; Kay & Surget-Groba, 2022; Liang et al., 2023; Merrill et al., 2023; Moyle et al., 2014).

Several general messages have emerged from the genetic characterization of postzygotic isolation in the laboratory. Postzygotic barriers are common byproducts of divergence between populations at two or more epistatically interacting loci, nicknamed “Dobzhansky–Muller incompatibilities” (Coyne, 1992; Dobzhansky, 1936; Muller, 1942). The number of loci involved in individual incompatibilities ranges from two to several, as does the number of incompatibilities responsible for hybrid dysfunction (Coughlan & Matute, 2020; Fishman & Sweigart, 2018; Maheshwari & Barbash, 2011; Presgraves, 2010). The number of incompatibilities between two lineages appears to increase non-linearly with divergence time (Matute et al., 2010; Moyle & Nakazato, 2010; Wang et al., 2015), as predicted by theory (Orr, 1995). Genes tied to hybrid sterility or hybrid inviability perform a variety of functions in their native genetic backgrounds (Maheshwari & Barbash, 2011; Presgraves, 2010). Some genes show evidence of positive selection, and some genes display signs of genetic conflict (Johnson, 2010). In plants, chromosomal rearrangements, including reciprocal translocations, sometimes cause dysfunction in F₁ hybrids (Fishman & Sweigart, 2018). How these underdominant variants become common within lineages remains a mystery. Other patterns that characterize the genetics of postzygotic isolation include the following: the X chromosome (or Z chromosome) exerts a disproportionate effect (Coyne, 1992, 2018; Coyne & Orr, 1989; Masly & Presgraves, 2007; Presgraves, 2008); when one sex evolves hybrid dysfunction first, it is usually the heterogametic sex (Coyne, 1992, 2018; Coyne & Orr, 1989; Haldane, 1922; Laurie, 1997; Orr, 1997); and species pairs display genetic variation for isolation phenotypes (Cutter, 2012; Larson et al., 2018; Reed et al., 2008).

Measurement of gene flow in nature

A second strategy for unveiling the genetics of reproductive isolation is to measure the rate of gene exchange between diverging lineages in wild hybrid populations. Combinations of mutations that reduce fitness should be selected against in hybrids, thereby reducing gene flow at these sites in the genome. Due to linkage, neutral variants will be discarded too (Baird, 1995; Barton, 1979, 1983; Barton & Bengtsson, 1986; Bengtsson, 1985; Gavrilets, 1997), creating a local genomic signature around the genes involved in reproductive barriers (Harrison & Larson, 2014; Payseur, 2010; Szymura & Barton, 1986). Most advances toward deciphering the genetics of reproductive isolation in the wild emanate from geographic regions in which diverging populations come into secondary contact and hybridize, known as hybrid zones. By genotyping ancestry-informative variants in population samples from hybrid zones, researchers can search for genomic outliers among geographic clines in allele frequency (Payseur, 2010; Porter et al., 1997; Szymura & Barton, 1986), look for variants with genotype frequencies that deviate from the genomic distribution (“genomic clines”; Gompert & Buerkle, 2009, 2011), and/or locate genomic regions in which ancestry from the minor parent is depleted (Schumer et al., 2018).

Collectively, genomic analyses of hybrid zones point to several salient inferences about reproductive isolation in nature. Levels of gene flow between diverging lineages differ substantially along the genome (Payseur & Rieseberg, 2016; Taylor & Larson, 2019). Gene flow tends to be reduced in genomic regions with less recombination and higher densities of coding or conserved sequences (Schumer et al., 2018). Although population differentiation is often higher on the X/Z chromosome relative to the autosomes (Presgraves, 2018), whether the X/Z chromosome experiences lower gene flow depends on the species pair (Fraïsse & Sachdeva, 2021). Genomic patterns of gene flow are repeatable across hybrid zone transects in some pairs of nascent species but not in others (Langdon et al., 2022; Simon et al., 2021; Teeter et al., 2010). Repeatability could be shaped by selection against the same loci, by shared genome architecture, or both.

Comparing two approaches to dissecting the genetics of speciation

Conceptual and theoretical considerations

An implicit assumption underlying the genetic mapping of reproductive barriers in the laboratory and the detection of genomic regions with reduced gene flow in the wild is that the same loci will be implicated (Figure 1). Heterospecific combinations of alleles at loci responsible for reproductive isolation phenotypes should be deleterious. Theory predicts that selection against hybrids will remove variants involved in reproductive isolation when selection is stronger than recombination, creating a barrier to gene flow for linked neutral alleles (Baird, 1995; Barton, 1979, 1983; Barton & Bengtsson, 1986; Bengtsson, 1985; Gavrilets, 1997). Although the distribution of gene flow along the genome is difficult to predict because it depends on the genetic architecture of reproductive isolation (number of loci and their phenotypic effects), the landscape of recombination, and the rate of migration into the hybrid zone, variants located near barrier loci are usually expected to show narrower clines (Payseur, 2010).

Figure 1.

Expectations for overlap between loci identified by genetic mapping of barrier traits and loci that reduce gene flow in nature. (A) Strong overlap is expected if traits that we map in the laboratory experience strong selection in nature. (B) Weak overlap is expected if either method is underpowered to find most or all underlying loci. (C) No overlap is expected if the loci underlying barriers observed in the laboratory are distinct from those that impede gene flow. These categories are not mutually exclusive.

Despite the theoretical expectation that gene flow should be reduced at loci involved in isolation phenotypes, plausible scenarios exist that could produce other outcomes. First, the two approaches to discovering the genetics of reproductive barriers could fail to identify the same loci for methodological reasons. If genetic mapping and/or hybrid zone studies are underpowered to find loci with modest effects or suffer from high false-positive rates, concordance could be masked. Furthermore, laboratory studies are limited to a subset of the reproductive barriers potentially active in hybrid zones; genetic mapping is biased toward those isolation phenotypes that are strong and easy to measure. Laboratory genetic studies also tend to ignore ecologically mediated (extrinsic) isolation, which can reduce gene flow in ways that mimic intrinsic barriers (Kruuk et al., 1999). Finally, intraspecific polymorphism in reproductive isolation could lead to differences in barriers among mapping populations (Larson et al., 2018; Pardy et al., 2021) and/or variation in selection among replicate hybrid zones (Janousek et al., 2015; Langdon et al., 2022; Mandeville et al., 2017). Although significant overlap among loci may exist between the “right” combinations of laboratory populations and natural populations, this signal could be erased when multiple groups are combined.

Perhaps more interestingly, there are also biological reasons to expect the loci identified by the two approaches to be different. First, genetic mapping targets traits associated with reproductive isolation, whereas gene flow across hybrid zones points to selection. Reproductive barriers characterized in the lab need not reduce fitness in nature. Furthermore, the efficacy of selection can depend on demographic factors such as population density, which may be lower in hybrid zones (Buggs, 2007). Even when hybrid incompatibilities are targeted by selection in a hybrid zone, the resulting genomic signatures can be highly variable (McFarlane et al., 2023).

A second biological reason the two frameworks could point to distinct loci is that there are differences in the present versus historic forces acting in hybrid zones. Genetic mapping focuses on reproductive barriers that exist currently, whereas signatures of reduced gene flow across hybrid zones may reflect a long history of barriers. As demographic and ecological conditions change, the strength of selection and the relative importance of different barrier phenotypes may shift (Kulmuni et al., 2020), potentially dampening signatures of selection. In some cases, incompatible alleles mapped in crosses could be removed by selection in hybrid zones, challenging the ability of these incompatibilities to maintain species boundaries (Bank et al., 2012; Barton & Bengtsson, 1986; Lindtke & Buerkle, 2015; Virdee & Hewitt, 1994). In these cases, laboratory crosses might uncover incompatibilities between alleles that are present in allopatric populations but no longer exist in a hybrid zone. Alternatively, a hybrid zone may carry a signature of selection against older incompatibilities that no longer exist in any population and thus cannot be recovered by mapping.

Finally, genetic mapping targets early phases of hybridization (e.g., the F₂ generation), whereas the subjects of studies of gene flow may be highly admixed, leading to disparities in genomic composition. The severity and form of reproductive isolation may differ between stages of hybridization, especially when epistasis plays an important role. Such differences can be observed in the laboratory in cases where later-stage mapping populations are used (e.g., Sotola et al., 2023), and differences in the strength of reproductive isolation are known to occur in hybrid zones of varying ages (e.g., Liao et al., 2019).

Empirical comparisons

Whether loci implicated in reproductive isolation in the laboratory inhibit gene flow in nature is ultimately an empirical question. Some studies have measured natural gene flow at certain genomic regions linked to postzygotic isolation. In monkeyflowers, at each of the two loci involved in a lethal incompatibility identified in the laboratory, the most common allele from Mimulus nasutus is found mostly within compatible Mimulus guttatus variants, indicating selection against the incompatibility (Zuellig & Sweigart, 2018b). In natural populations formed by hybridization between swordtail species Xiphophorus birchmanni and Xiphophorus malinche, a genomic region with depleted ancestry from X. birchmanni displays transmission ratio distortion in F₂ crosses (Langdon et al., 2022; Moran et al., 2024). Although these studies reveal potential connections between postzygotic isolation in the laboratory and selection against hybrids in nature for certain loci, they leave open the broader question of whether the collection of loci identified by the two strategies is the same.

A case study: the relationship between loci connected to reproductive barriers in the laboratory and loci with reduced gene flow in house mice

To our knowledge, the concordance between loci with reduced gene flow in nature and postzygotic barrier loci mapped in the laboratory has yet to be examined on a genomic scale. Amalgamating datasets should reduce the effects of biases inherent in individual studies, populations, or barriers, yielding a more holistic picture of loci linked to reproductive isolation. Given the popularity and importance of the two strategies for identifying loci involved in reproductive barriers, the dearth of empirical comparisons between them constitutes a significant gap in our understanding of the genetics of speciation. Here, we compare loci tied to reproductive barriers in the laboratory to loci experiencing reduced gene flow in nature in house mice, one of the most intensively studied systems in the genetics of speciation.

House mice as a model system

The Western European house mouse, Mus musculus domesticus, and the Eastern European house mouse, Mus musculus musculus, exhibit partial reproductive isolation that has evolved since the two subspecies began to diverge 125–625 KYA (Boursot et al., 1993; Geraldes et al., 2008; Phifer-Rixey et al., 2020). Sterility or subfertility observed in hybrid males has received the most attention from a genetic perspective, with mapped loci from across the genome contributing to reproductive traits such as testis size; counts of spermatocytes, spermatids, and sperm; sperm shape; and sperm motility (Campbell & Nachman, 2014; Forejt & Iványi, 1974; Good et al., 2008; Larson et al., 2017; Schwahn et al., 2018; Storchová et al., 2004; White et al., 2011). There is evidence that disruptions in gene expression during spermatogenesis are connected to hybrid male sterility, particularly on the X chromosome (Bhattacharyya et al., 2014; Good et al., 2010; Hunnicutt et al., 2022; Kopania et al., 2022; Larson et al., 2017, Larson et al., 2022; Mack et al., 2016; Turner et al., 2014). Forejt and colleagues exploited intrasubspecific variation in sterility to identify the first-known hybrid sterility gene in vertebrates—Prdm9 (Forejt & Iványi, 1974; Forejt et al., 1991; Mihola et al., 2009; Trachtulec et al., 2005). Prdm9, a histone methyltransferase (Hayashi et al., 2005), forms one component of a complex incompatibility (Bhattacharyya et al., 2013, 2014; Forejt et al., 2021; Valiskova et al., 2022).

Other forms of reproductive isolation exist between M. m. domesticus and M. m. musculus. Hybrid females show signs of reduced fertility (Suzuki & Nachman, 2015), though this barrier has yet to be probed by genetic mapping. There is mixed evidence that hybrids suffer reduced viability in the form of developmental instability (Mikula et al., 2010), higher parasite load (Balard & Heitlinger, 2022), and transgressive microbiome phenotypes (Wang et al., 2015). There are also signs of prezygotic isolation between the subspecies (discussed later).

Mus musculus domesticus and M. m. musculus form a hybrid zone that stretches across Europe from Norway to Bulgaria (Boursot et al., 1993; Jones et al., 2010; Sage et al., 1993). Gene flow across the hybrid zone has been measured in multiple transects. Studies of geographic clines suggest the width of the hybrid zone reflects a balance between dispersal and selection against hybrids, especially in the center of the zone (Boursot et al., 1993; Dod et al., 1993; Fel-Clair et al., 1998; Moulia et al., 1993; Sage et al., 1993; Tucker et al., 1992; Vanlerberghe et al., 1986). Analyses of geographic clines and genomic clines reveal substantial variation among loci in the level of genetic exchange (Janoušek et al., 2012; Wang et al., 2011; Macholán et al., 2011; Payseur et al., 2004; Teeter et al., 2008) and discordant patterns across transects (Janousek et al., 2015; Teeter et al., 2010).

Compiling datasets characterizing the genetics of reproductive isolation between M. m. domesticus and M. m. musculus

Across the vast literature on reproductive isolation in house mice, we were able to identify 58 studies that implicated specific genomic locations in reproductive barriers. From these studies, we selected the subset with accessible datasets, excluded those that were redundant (e.g., keeping only the most recent of any series of studies that progressively narrowed genomic intervals) and removed those that focused on the Y chromosome (because it is usually treated as a single locus). For each study, all locations were converted from the original coordinates to the mm10 assembly of the mouse genome sequence, using LiftOver on the UCSC Genome Browser (Nassar et al., 2023). Studies or loci that could not be converted were excluded. Due to the highly variable nature of these loci, we decided to use the data as reported. As a result, some quantitative trait loci (QTL) are defined by 2-logarithm-of-odds (LOD) intervals and others by 1.5-LOD intervals, and loci surveyed in the hybrid zone are defined as targets of selection using thresholds unique to each study. This approach expands the range of studies we can include, though it prohibits us from conducting a formal meta-analysis.

Our final dataset draws on 33 studies (Table 1). It contains 3,200 unique intervals connected to reproductive isolation, mostly QTL, single nucleotide polymorphism (SNP) markers, and genes. Intervals from laboratory studies and intervals from hybrid zone studies both span the genome (Figure 2), providing plenty of opportunity for overlap. The “laboratory” intervals are primarily associated with phenotypes involved in hybrid sterility, but there are also genes related to hybrid inviability (in the form of metabolic dysfunction). Because the full set of intervals we compiled covers a large portion of the genome, it is difficult to randomize the locations of all intervals in the most expansive version of the dataset. For that reason, we compared various subsets of the dataset (described below). The dataset we used for our main comparison (highlighted in Table 1) contains 1,562 intervals from 24 studies. The full dataset is available on Dryad (DOI: 10.5061/dryad.m63xsj495).

Table 1.

Sources of data on the genetics of reproductive isolation between house mouse subspecies M. m. domesticus and M. m. musculus.

Study	Source	Mapping population	Genomic coverage	Locus type	Phenotype(s)
Included in the permutation tests
Balcova et al., 2016	Lab	PWDxC57BL6	Several X-linked markers	QTL	Recombination rate
Bímová et al., 2011	Nature	Czech transect	12 SNPs	Geographic clines	NA
Campbell et al., 2012	Lab	Good introgression lines	18 X microsats	QTL	Sperm count, sperm head morphology, testis weight
Campbell et al., 2013	Lab	Good introgression lines	7 X linked genes	Genes	Male sterility
Gompert & Buerkle, 2011	Nature	Bavarian and Saxon transects	41 markers	Genomic clines	NA
Good et al., 2008	Lab	Good introgression lines	18 X microsats	QTL	Sperm count, sperm head morphology, testis weight
Good et al., 2010	Lab	WSB/LEWESxPWK/CZECHII	39,000 transcripts	Genes	Male sterility
Hunnicutt et al., 2022	Lab	WSB/LEWESxPWK/CZECHII	RNAseq	Genes	Male sterility
Janoušek et al., 2012	Nature	Bavarian and Czech transects	1,316 SNPs	Epistatic regions	NA
Janousek et al., 2015	Nature	Bavarian, Saxon, and Czech transects	1,316 SNPs	Genomic clines	NA
Larson et al., 2017	Lab	WSB/LEWESxPWK/PWD	500 transcripts	Genes	X chromosome inactivation, male sterility
Larson et al., 2018	Lab	WSB/LEWESxPWK/CZECHII	Genome-wide	QTL	Sperm count, sperm motility, sperm head morphology, testis weight
Lustyk et al., 2019	Lab	PWDxC57BL6	1 locus	Locus	Male sterility
Macholán et al., 2011	Nature	Czech transect	24 loci	Genomic and geographic clines	NA
Mack et al., 2016	Lab	LEWESxPWK	Expression for 9851 gene	Genes	Male sterility
Mihola et al., 2009	Lab	PWDxC57BL6	1 locus	Gene	Male sterility
Morgan et al., 2020	Lab	WSBxPWD	RNAseq	Genes	Male sterility
Payseur et al., 2004	Nature	Bavarian transect	13 X loci	Geographic clines	NA
Schwahn et al., 2018	Lab	WBSxPWD	198 SNPs	Single and multiple QTL	Testis area, seminiferous tubules with apoptosis, round spermatids, multinucleated syncytia
Teeter et al., 2008	Nature	Bavarian transect	53 SNPs	Geographic clines	NA
Teeter et al., 2010	Nature	Bavarian and Saxon transects	41 SNPs	Geographic clines	NA
Turner & Harr, 2014	Nature	Laboratory-bred F1 from Bavarian transect parents	156,000 SNPs	GWAS	Testis weight
Turner et al., 2014	Lab	WSBxPWD	Transcripts of 20,000 genes, and 198 SNPs for QTL mapping	eQTL hotspot clusters and interaction loci	Male sterility
Valiskova et al., 2022	Lab	(PWDxCAST)xB6	11,000 SNP array	QTL	Testes weight, sperm count, asynapsis
White et al., 2011	Lab	WSBxPWD	331 SNP array	Single and multiple QTL	Sperm head density, sperm head morphology, testis weight, sperm tail morphology, seminiferous tubule area
Excluded from the permutation tests
Dzur‐Gejdosova et al., 2012	Lab	B6xPWDxB6 Backcross	100 markers	Single QTL	Sperm count, testis weight
Kass et al., 2014	Nature	Combination	1 locus	Gene	NA
Kopania et al., 2022	Lab	LEWESxPWK	RNAseq	Genes	Testis expression
Rottscheidt & Harr, 2007	Lab	STRAxSTUS	11,000 transcripts	Genes	Misexpression
Shorter et al., 2017	Lab	Collaborative cross	381,351 SNPs	Single QTL	Fertility, testis weight, seminal vesicle weight, hyperactivated sperm, broken sperm, epididymis and vas deferens weight, sperm head morphology
Wang et al., 2011	Nature	Bavarian and Czech transects	1,316 SNPs	Geographic clines	NA
Wang et al., 2015	Lab	WSBxPWD	234 SNPs	QTL, genes	Microbiome structure
Widmayer et al., 2020	Lab	PWKxB6|AJ|129S|DBA3	Whole genome sequencing	Regions of differentiation	NA

Figure 2.

Genomic locations of loci connected to reproductive isolation between Mus musculus musculus and Mus musculus domesticus. For each chromosome, segments used in the broadest permutation tests are indicated by darker bands, and additional segments from the full dataset are indicated by lighter bands. The top row (dashed lines) depicts loci with reduced gene flow across the hybrid zone and the bottom row (solid lines) shows barrier loci identified in the laboratory.

Evaluating overlap between loci linked to isolation phenotypes in the laboratory and loci showing reduced gene flow in nature

To evaluate the overlap between datasets, we used a permutation approach. We adopted the simple strategy of counting the number of overlaps between datasets, rather than attempting to estimate the amount of overlap. Following this methodology should reduce biases generated by the diverse criteria employed by different studies. Because intervals were often implicated in more than one study (or multiple times in the same study), we collapsed each dataset into one set of merged intervals (separately for “laboratory” and “nature”). This collapsing of the dataset also addresses the lack of independence between studies, which limits our ability to perform more detailed comparisons of individual studies. For each comparison, we randomly permuted the nature dataset 10,000 times and calculated a p-value as the proportion of permutations with the same number or a greater number of overlaps as observed in the data. We permuted only the nature dataset due to the presence of large intervals in the laboratory dataset. Permutation tests were completed using the R package GenomicRanges (Lawrence et al., 2013) and custom R code (available on Dryad, DOI: 10.5061/dryad.m63xsj495).

We conducted several additional analyses to examine the sensitivity of our results to biological and methodological factors. First, we performed separate permutation tests that included or excluded the X chromosome. Second, we conducted separate tests that treated each QTL interval as either the full reported LOD interval or as a 1 Mb interval including ± 500 kb surrounding the estimated QTL position. Third, we investigated the robustness of our results by repeating comparisons after removing datasets from individual papers or from groups of related papers. Finally, we performed comparisons that included loci derived from studies of wild hybrids that did not measure gene flow, such as a genome-wide association study for hybrid male sterility (Turner & Harr, 2014).

All permutation tests show the same pattern: the overlap between loci implicated in reproductive isolation in the laboratory and loci showing reduced gene flow in nature is no greater than expected by chance (Table 2; Figure 3). This pattern persists when we exclude the X chromosome, when we reduce the size of each QTL to a 1 Mb interval around the QTL peak, and when we do both. Moreover, removing data contributed by one study at a time produces no significant overlap in any comparison. Removing data from groups of studies that combined information from the laboratory and from nature or featured lower confidence when genomic positions were remapped also leads to no significant overlaps. There is no improvement in overlap when we add genomic intervals mapped using wild hybrids.

Table 2.

Results from permutation tests of the null hypothesis that loci connected to reproductive isolation in the laboratory and loci with reduced gene flow in nature overlap as much as expected by chance. p-values were derived from 10,000 permutations of each dataset.

Data subset		Treatment of QTL
Loci	Chromosomes	Full QTL intervals	1 Mb QTL intervals
Main dataset	All	0.4345	0.1094
	Autosomes only	0.6843	0. 1701
Including GWAS intervals	All	0.5597	0.1783
	Autosomes only	0.6741	0.1409
Removing papers with a dual lab/nature approach	All	0.4862	0.869
	Autosomes only	0.4245	0.2536
Removing papers with lower confidence position conversions	All	0.4366	0.1632
	Autosomes only	0.6874	0.1733

Figure 3.

For two subspecies of house mice, the number of overlaps between loci linked to reproductive barriers in the laboratory and loci showing reduced gene flow across a hybrid zone is no greater than expected by chance. The histogram shows the results of 10,000 permutations of full-length QTL intervals for all chromosomes (p = 0.4345). The vertical line indicates the number of overlaps observed in the data.

Our dataset contains several studies that report allele frequency clines at individual SNPs. The “significant” SNPs included in our analyses are likely to be linked to selected sites rather than be targets of selection themselves. We attempted to address this issue by creating 1 Mb intervals (±500 kb) around each SNP and using these intervals to count overlaps. With this approach, we once again observe no significant overlap (full QTL, p = 0.4586; 1 Mb QTL, p = 0.8513).

As a further quantitative test of the connection between loci associated with reproductive isolation in the laboratory and in nature, we conducted comparisons involving a single hybrid zone study. Wang et al. (2011) reported estimates of geographic cline width for 1,401 SNPs scattered across the genome in two transects of the hybrid zone located in Bavaria and the Czech Republic. We asked whether the subset of these SNPs that overlap with laboratory-discovered loci differ in cline width from the SNPs that do not overlap with laboratory-discovered loci, using a Wilcoxon rank sum test. This approach enabled comparisons free from heterogeneity among hybrid zone studies and allowed us to include a broader set of laboratory-derived loci (indicated in Table 1) since permutations were not necessary. Once again, we also conducted tests including vs. excluding the X chromosome, incorporating full QTL LOD intervals vs. 1 Mb windows around QTL positions, and removing one laboratory-derived dataset at a time.

In many comparisons, SNPs that overlap loci implicated in reproductive isolation in the laboratory show significantly narrower clines (i.e., less gene flow) than SNPs that do not overlap isolation loci (Table 3). However, interpretation of this result is complicated by the fact that clines on the X chromosome are significantly narrower than clines on the autosomes (Wilcoxon rank sum test: Bavarian transect, p < 2e-16; Czech transect, p < 2e-16). When considering only autosomal SNPs, the difference in cline width disappears (Table 3). These results strongly suggest that the reduced cline width of markers within loci connected to isolation in the laboratory reflects disparities between the X chromosome and the autosomes rather than a bona fide genome-wide phenomenon. In the Czech transect, this effect is less pronounced, and in some cases, loci that overlap display wider clines. Similar results are recovered when data from each individual study are removed, one dataset at a time (although a few such tests yield p < 0.05, this constitutes weak evidence for enriched overlap when accounting for multiple testing). This pattern is recapitulated when we use a smaller set of geographic clines (53 SNPs) from a third transect of the hybrid zone in Saxony (Teeter et al., 2010) instead of using SNPs from Wang et al. (2011), and when we use genomic clines also estimated from the smaller dataset (Gompert & Buerkle, 2011) (Supplementary Results). Comparing cline widths of SNPs that do or do not overlap with loci detected in genome-wide association studies of wild hybrids yields similar results (Table 3).

Table 3.

Results of tests comparing cline widths at SNPs that overlap with loci connected to reproductive isolation in the laboratory to cline widths that do not overlap. p-values were computed using Wilcoxon rank sum tests.

Data subset		Bavarian transect		Czech transect
Loci	Treatment of QTL intervals	All chromosomes	Autosomes only	All chromosomes	Autosomes only
Main subset	Full intervals	2.03E-05a	0.256595	0.005853a	0.973438
	1 Mb intervals	4.34E-10a	0.817968	3.9E-06a	0.243754
All lab loci	Full intervals	0.000339a	0.187057	0.241044	0.370589
	1 Mb intervals	4.58E-06a	0.836538	0.020352a	0.038508b
Only single QTL	Full intervals	8.73E-07a	0.062814	0.00552a	0.984957
	1 Mb intervals	0.001054a	0.825004	0.154961	0.138699
Only multiple QTL	Full intervals	0.111049	0.292399	0.53721	0.203668
	1 Mb intervals	0.051982	0.314247	0.176887	0.529121
Main subset, only genes	-	0.777824	0.126815	0.107514	0.00767b
All genes	-	0.800819	0.1553875	0.095743	0.00781b
GWAS intervals	-	2.95E-04a	0.398	2.24E-05a	0.674

^aSites overlapping QTL have significantly narrower cline widths.

^bSites overlapping QTL have significantly wider cline widths.

Understanding the disconnect between barrier loci mapped in the lab and those identified in nature in house mice

Our results suggest that the loci restricting gene flow between two subspecies of house mice and those controlling reproductive isolation phenotypes in experimental crosses between the subspecies are different. Both biological factors and characteristics of the studies we compiled likely contribute to the disparity we observe.

The old age of the hybrid zone (estimates range from 700 to 6,000 generations; Cucchi et al., 2005; Raufaste et al., 2005) provides one explanation. If migration of non-admixed mice has been limited following the formation of the hybrid zone, alleles involved in incompatibilities mapped in early generations of hybridization in the lab could have been removed from the zone long ago, leaving behind dampened signatures of selection. In one example, the only gene known to cause hybrid sterility in house mice, Prdm9, resides in a genomic location with mixed evidence for reduced gene flow across the hybrid zone (Wang et al., 2011). An essential component of Prdm9-mediated sterility is heterozygosity at a certain proportion of binding sites (Gregorova et al., 2018), which might lead to rapid breakdown of the underlying incompatibility in a hybrid population as ancestry fixes along the genome.

Another possibility is that isolation phenotypes mapped in the lab do not constitute strong barriers to gene flow in nature. In house mice, most of the loci (QTL and genes) that have been connected to reproductive isolation are tied to hybrid male sterility. This form of isolation is polymorphic within both M. m. domesticus and M. m. musculus (Britton-Davidian et al., 2005; Forejt, 1996; Good et al., 2008; Larson et al., 2018), which could weaken its effects on gene exchange. Perhaps reproductive barriers that have yet to be mapped (or be characterized) in house mice experience stronger selection in the hybrid zone.

Our analysis focused on postzygotic isolation, but there is evidence for prezygotic isolation in house mice. In a putative case of reinforcement, mice caught near hybrid populations in the wild prefer mates from the same subspecies, especially in M. m. musculus (Christophe & Baudoin, 1998; Ganem et al., 2008; Smadja & Ganem, 2002, 2005; Smadja et al., 2004); mice far away from a contact zone display no directional mate preference (Bímová et al., 2011; Smadja & Ganem, 2002, 2005; Smadja et al., 2015). Assortative mating appears to be mediated by volatile (Mucignat-Caretta et al., 2010) and non-volatile (Hurst et al., 2017) molecules in the urine as well as salivary androgen-binding proteins (Laukaitis et al., 1997). Nevertheless, adding to our dataset the small number of loci associated with prezygotic isolation in three studies does not impact our findings (Supplementary Results).

Heterogeneity among studies also could obscure a relationship between loci with restricted gene flow and loci tied to isolation traits in the lab. Within laboratory studies and within hybrid zone studies, we find significant overlaps among loci (Supplementary Results), a sign that the discordance we document is not purely generated by variation among investigations. Still, differences in experimental design are likely to dilute underlying signals.

One potential way to better unite studies of gene flow and reproductive isolation phenotypes is to conduct mapping in a natural hybrid population. A genome-wide association study (GWAS) involving offspring of hybrids sampled from the house mouse contact zone identified four genomic regions connected to testis weight and 17 regions connected to testis gene expression that overlap with hybrid sterility loci mapped in the laboratory (though most regions do not overlap; Turner & Harr, 2014). However, we see no evidence for enhanced overlap between this subset of loci and those loci showing reduced gene flow in the hybrid zone.

Guidance for future research on the genetics of speciation

Our findings should motivate deeper and broader examination of the two primary strategies for dissecting the genetics of species barriers. The field would benefit greatly from additional empirical comparisons that formally test overlap between loci identified by the two approaches. Progress in the genetic mapping of reproductive barriers and in the measurement of gene flow on a genome-wide scale has positioned researchers to conduct these comparisons across a variety of species. Analysis of species pairs that collectively vary in the form of reproductive isolation and in the age of hybrid zones should be particularly informative.

We focused on postzygotic isolation in this Perspective, but we might expect similar principles to apply to prezygotic barriers. Considering two divergent ecotypes of the monkeyflower Mimulus aurantiacus, loci linked to pollinator isolation by genetic mapping and loci showing narrow geographic clines in a contact zone do not overlap more than expected by chance (Stankowski et al., 2023). The authors provide several potential explanations for this discrepancy, including low mapping resolution and unmeasured forms of reproductive isolation. While the strength of pollinator isolation has received considerable attention in this system (Stankowski et al., 2023), partial male sterility also has been detected (Sobel & Streisfeld, 2015).

In addition to empirical comparisons, we need new theoretical work to further delimit the conditions under which loci implicated in reproductive isolation will impede gene flow in nature. Should we expect the concordance between loci found in the lab and in nature to be higher for younger hybrid zones, which feature genomic compositions closer to those created by experimental crosses? Should forms of reproductive isolation with simple genetic architectures (if such conditions exist) predict stronger or weaker correspondence among loci throughout the genome? What is the role of polymorphic reproductive isolation in generating this pattern? If the disparity we observed turns out to be common, will it mostly be driven by contrasts between phenotype-based mapping vs. inferences about gene flow or by differences between lab-based reproductive barriers vs. natural reproductive barriers? Could we use the presence or lack of overlap between datasets to reconstruct the forces that have impacted the history of hybridization?

Both genetic mapping of reproductive isolation phenotypes and the measurement of gene flow in nature have led to great leaps in our understanding of the process of speciation. This progress has inspired many researchers to call for studies that combine these approaches as a way to identify the “true” genetic basis of speciation. We support these endeavors. However, we encourage speciation researchers to recognize the interesting possibility that these two strategies will point to different regions of the genome for biological reasons, rather than purely methodological shortcomings. The presence or lack of overlap itself could be a revealing attribute, providing fresh insights into the forces that shape hybrid populations and the evolution of reproductive isolation. A more nuanced interpretation of emerging datasets could inspire an improved synthesis of the genetic factors responsible for the origin of species.

Supplementary material

Supplementary material is available online at Evolution.

Data availability

All data and code from this study is available on Dryad (DOI: 10.5061/dryad.m63xsj495).

Author contributions

M.E.F. and B.A.P. conceived and designed the study. M.E.F. compiled and analyzed the data. B.A.P. and M.E.F. wrote the manuscript.

Funding

This research was funded by the National Institutes of Health (NIH) (grants R35 GM139412 and R01 GM120051) and the National Science Foundation (NSF) (grant DEB 1353737) to B.A.P. M.E.F. was supported by an NSF Postdoctoral Research Fellowship in Biology (grant no. 2305853) and NIH (1R35GM150907) to Jenn M. Coughlan.

Conflict of interest: The authors declare no conflict of interest.