Sexual selection against natural hybrids may contribute to reinforcement in a house mouse hybrid zone

Abstract

Sexual selection may hinder gene flow across contact zones when hybrid recognition signals are discriminated against. We tested this hypothesis in a unimodal hybrid zone between Mus musculus musculus and Mus musculus domesticus where a pattern of reinforcement was described and lower hybrid fitness documented. We presented mice from the border of the hybrid zone with a choice between opposite sex urine from the same subspecies versus hybrids sampled in different locations across the zone. While no preference was evidenced in domesticus mice, musculus males discriminated in favour of musculus signals and against hybrid signals. Remarkably, the pattern of hybrid unattractiveness did not vary across the hybrid zone. Moreover, allopatric populations tested in the same conditions did not discriminate against hybrid signals, indicating character displacement for signal perception or preference. Finally, habituation–discrimination tests assessing similarities between signals pointed out that hybrid signals differed from the parental ones. Overall, our results suggest that perception of hybrids as unattractive has evolved in border populations of musculus after the secondary contact with domesticus. We discuss the mechanisms involved in hybrid unattractiveness, and the potential impact of asymmetric sexual selection on the hybrid zone dynamics and gene flow between the two subspecies.

1. Introduction

Hybridization in nature is mostly observed at the limits of species or population range. Its presence in the wild can either facilitate or impede speciation [1–3]. If hybridization is maladaptive, selection is expected to favour the evolution of assortative mate preference in the two hybridizing populations, a process called reinforcement [4–7]. This adaptation can be driven by the cost of hybridization between parental populations [8–10], but not only. Interactions with the hybrids could also be selected against. In particular, if hybrid signals are less attractive, sexual selection could reduce gene flow across the hybrid zone [11,12].

Mating signals evolve under sexual and ecological selection [13]. Their complexity depends on the number and characteristics of the genes involved in their determinism [14–16], as well as on environmental factors influencing their expression and their perception [13,17]. Hybrid genotypes could produce unrecognized or unattractive signals when new associations of alleles are brought together by hybridization [18], and sexual selection can occur against such signals (review in the electronic supplementary material, S1). Sexual selection is considered a powerful potential driver of speciation [19]. Nevertheless, it was recently argued that empirical demonstration of its importance is still insufficient, notably because most studies do not link the divergence in mating traits and preferences with their actual effect on gene flow and genetic divergence of populations [20]. Here, we investigate mating traits divergence between hybrid populations with different levels of genetic divergence from their parental populations. The rationale behind our study is that if individuals of parental populations seldom mate with hybrids, gene flow across the hybrid zone can be hindered and the established zone could act as a barrier between incipient species. By contrast, hybrids could form a bridge between parental genomes if fitness of crosses between neighbouring, genetically less divergent, populations allows step-by-step gene flow across the zone [21], but only if they are not discriminated against [22].

Our study model is a unimodal hybrid zone formed during a secondary contact between two subspecies of the house mouse Mus musculus musculus and Mus musculusdomesticus (thereafter musculus and domesticus). The history of divergence between these subspecies is characterized by a long initial period in allopatry (a few hundred thousand years ago [23,24]), which could account for the accumulation of numerous genetic incompatibilities resulting in selection against hybridization and underlying the formation of a tension zone crossing Europe [25–29].

Olfactory signals present in mouse urine were shown to play a central role in social and sexual communication and to be shaped both by sexual and natural selection [30–33]. Populations of both subspecies at the border of the hybrid zone display assortative mate preference for signals present in the urine, and reproductive character displacement (for both preferences and signals) was documented between border and allopatric populations of the two subspecies in male and female mice [16,34,35]. The cline of transition of preferences of wild hybrids from one parental odour type to the other was found to be steep, coincident with the centre of the genetic cline in a Czech transect [35], but shifted roughly 10 km into the musculus territory in a Danish transect [16].

This study addresses the signal component of potential hybrid mates, by assessing how such signals are perceived, mainly by the choosiest subspecies (musculus) [16,34,36].

The specific questions addressed here are listed below.

• (1) Are signals of hybrid mice discriminated against by the two parental subspecies? What is the generality of this pattern? Does it vary with the genetic characteristics (hybrid index) across the hybrid zone? To address this series of questions, we assessed: (i) preference of male and female mice of parental populations from the border of the hybrid zone during two-way choice tests involving hybrid versus parental opposite sex urine as the stimuli; (ii) we replicated this test changing the origin of parental urine; and (iii) we replicated the test, presenting against the parental urine hybrid urine from different locations across the hybrid zone. We predicted that if discrimination occurred and preference was linked to genetic similarities between the chooser and the stimuli, assortative preference would be more marked in the presence of distant as compared to neighbour hybrids.

• (2) Did sexual selection against hybrids evolve in the hybrid zone? We compared patterns of preference in populations of mice distant versus at the border of the hybrid zone (map in the electronic supplementary material, S2).

• (3) What may be the proximal reason for hybrid unattractiveness, i.e. how different are hybrid and parental odours? We investigated odour similarities between hybrids and the two parental subspecies and addressed whether they were distinctively different (i.e. transgressive), using habituation–discrimination/generalization tests [37].

2. Material and methods

(a). Biological material

(i). Mice

All mice involved in this study were either wild trapped in Jutland, Denmark, in commensal indoor habitats, in October 2010 and June 2011, or descendants of these, bred in the laboratory (see details in the electronic supplementary material, S3). We sampled mice in several farms at the northern and southern edges of the hybrid zone (‘border mice’), across the hybrid zone (‘hybrid mice’) and approximately 40 km north of the musculus border of the zone, where ‘pure’ musculus individuals putatively occur (‘close allopatric mice’, see the electronic supplementary material, S2). Hybrid mice used in this study were wild, and after keeping in laboratory conditions for several months, were used as urine donors for subsequent behavioural experiments (see below). Border and close allopatric mice were first- and second-generation progeny of wild-trapped mice, and provided test mice and urine stimuli.

(ii). Genetic markers and hybrid index estimation

We genotyped wild mice using loci with insertion/deletion polymorphisms of transposable elements, determined to have contrasted allele frequencies (alternatively fixed or nearly so) between the two subspecies in allopatry (marker design in the electronic supplementary material, S3). Eighteen of them were autosomal, two were X-linked and one Y-linked (methods as in ref. [38], described in the electronic supplementary material, S4). On this basis, population samples of this study could be characterized by a multilocus hybrid index (HI, defined throughout as the proportion of musculus ancestry) estimated by maximum likelihood [39] given the parental allele frequencies estimated in samples from populations more than 200 km away from the hybrid zone.

The genetic cline of the mouse hybrid zone is geographically well structured [25]. Hence, geographical location of a sample was considered as a good predictor of its average genetic composition. Genetic typing involved a relatively limited number of markers as it served to check that none of the studied populations or individuals was an outlier as compared to its geographical origin, which could happen as a result of accidental, recent long distance migration. The HI estimates of wild samples and their support limits are reported in figure 1 and the electronic supplementary material, S5.

Figure 1.

Trapping sites and their genetic characteristics plotted on a transect orthogonal to the genetic centre (thick line) of the Danish hybrid zone defined in [25]. The thick light grey line describes the general shape of the cline from [25]. Trapping sites are represented by white circles with reference to their HI estimated in this study (detailed in the electronic supplementary material, table S5). Rectangles show grouping of hybrid sites into categories; from left (light) to right (dark) across the hybrid zone, respectively: border domesticus, hybrids H7, H6, H5, H4, H3, H2, H1 and border musculus. Circle sizes are proportional to the sample size used to estimate the site HI. Dashed lines refer to an earlier study [16] testing wild hybrids preference for parental stimuli and showing an abrupt transition (behavioural centre) from preference for domesticus, left of the dashed lines, to preference for musculus, right of these lines.

(iii). Urinary stimuli

Urine donors were either first-generation laboratory-born (border or close allopatric populations) or wild hybrid mice maintained in the same standardized conditions for at least two months after trapping. Urine was collected at different times of the day and over several days to capture intra- and interday variations in urine composition, both upon handling of mice and pipetted from a cleaned surface, and stored at −20°C. All stimuli were pools of urine from three to four mice of the same sex and of different populations or farms to account for genetic and environmental interindividual variance within a given category. In the specific case of hybrids, categories were defined with regard to their geographical positions along the hybrid zone (figure 1), going from sites neighbouring the musculus border (H1) to those closer to the domesticus border (H7). We only trapped one male in the H5 category, so we combined its urine with those of males of the H6 category to match the minimum of three different urine donors in a pool. The hybrid categories were intended to capture potential variations with reference to the shift of the preference cline (figure 1).

(b). Behavioural tests

Mate preference was assessed from relative time spent by a mouse investigating two urinary stimuli deposited in two peripheral boxes connected to a Y maze during 5 min trials (two-way choice tests). As a mouse nose can detect odour differences in less than a second [33], even small differences in stimulus investigation time can be considered biologically relevant. Odour discrimination and ability to perceive differences between two odorant stimuli were addressed via habituation–discrimination (or generalization) tests. This experiment is based on observations that mice investigate novel stimuli more than familiar ones. Practically, a mouse is first presented with a single odour long enough to induce familiarity (habituation). Immediately after the same mouse is presented with two new stimuli, which will be investigated differently if one is more similar to the habituation stimulus than the other (review in [37], details in the electronic supplementary material, S3). Preference tests involved one sex presented with urine pools of the opposite sex, and habituation tests involved males presented with female urine. We used protocols described by Smadja & Ganem [40,41], with minor changes (see the electronic supplementary material, S3).

A total of 234 wild derived mice (musculus: 124 males, 81 females; domesticus: 20 males, nine females) were involved in two-way choice experiments, and 39 male musculus in the habituation/discrimination or habituation/generalization experiments (‘the noses’). Female mice were tested when sexually receptive. All tested mice were unrelated to urine donors, and pregnant females were excluded. Each different choice or discrimination test was conducted with 8–17 unrelated mice.

(c). Experimental designs

(i). Are signals of hybrid mice discriminated against by parental subspecies?

• Discrimination against hybrids. We first determined whether males and females of border populations displayed homogamous choice when presented with ‘musculus border’ versus ‘domesticus border’ stimuli. Mice discriminating against heterosubspecific stimuli, i.e. male and female musculus mice and male domesticus (see Results), were involved in choice tests between a border conspecific stimulus and a stimulus from the geographically closest hybrid category (H1 for musculus or H7 for domesticus, see figure 1). We expected assortative preference if the stimuli differed and the hybrid signal was discriminated against.

• Generality. To test the generality of the assortative mate preference assessed in ‘Discrimination against hybrids’, we performed a control test presenting musculus border mice with a hybrid (H1) and a conspecific stimulus from close allopatry rather than from border (see Discrimination against hybrids) populations. We predicted preference for the allopatric signal if discrimination against the hybrid stimulus was independent of the conspecific stimulus geographical origin.

• Variation. We determined whether the perception of hybrid stimuli varied with their genetic or geographical characteristics during choice tests where we diversified the origin of the hybrid stimulus presented against the parental stimulus (hybrid categories described in figure 1). Only the most discriminating subspecies, musculus, participated in these tests. If hybrid attractiveness varied with geographical proximity or genetic similarity, we expected higher discrimination against hybrids more distant from the noses. Alternatively, all hybrids could be perceived as equally unattractive (or attractive).

(ii). Did discrimination against hybrid signals evolve in the hybrid zone?

To determine the origin of discrimination against hybrid signals (evolution in the contact zone or early divergence in allopatry), we assessed preference of close allopatric musculus mice during choice tests between conspecific (musculus border) and H1 hybrid stimuli. We expected mice from close allopatry to discriminate less between the stimuli than border mice (test Discrimination against hybrids) if discrimination against hybrid signals evolved in the contact zone.

(iii). How similar are hybrid and parental odours?

The odours of the two extreme hybrid categories (H1 and H7) were each compared to the domesticus parent, via habituation–discrimination tests to ascertain that they differed from this subspecies. A second test assessed whether H7 was more similar to domesticus than H1, which is expected if odour determinism is additive and odour similarity parallels genetic similarity. In line with what precedes, we tested whether similarity between H1 and domesticus was higher than similarity between the two parental subspecies (see the electronic supplementary material, S3). If the hybrid stimulus shared similarities with domesticus, we expected it to be less investigated than the musculus stimulus, and if odour similarities with domesticus varied with genetic similarities, we expected H7 to be less investigated than H1.

(d). Statistical analyses

Preference and discrimination were assessed by pair comparisons of time spent sniffing or touching the proposed stimuli. Random choice or absence of discrimination was concluded when there was no significant difference between the times spent in contact with either stimulus.

To compare preference across tests, we used an index (R): time spent in contact with homosubspecific stimulus divided by total time in contact with both stimuli. We performed a mixed ANCOVA on this transformed variable (expR) with sex and hybrid category as fixed factors, population as a random factor and motivation (i.e. the sum of times spent in the right and left sides of the Y maze) as a covariable.

All statistical analyses were performed with R 2.15.0 software [42–44].

3. Results

(a). Are signals of hybrid mice discriminated against by parental subspecies?

• Discrimination against hybrids. Male and female musculus mice from populations at the border of the hybrid zone showed assortative preference when presented with musculus versus their most neighbour hybrid stimuli (H1, figure 2a,b; males: n = 17, median = 5.91 s, V = 130, p = 0.009; females: n = 10, median = 0.81s, V = 49, p = 0.027). By contrast, although border domesticus males discriminated in favour of their own subspecies during choice between musculus and domesticus stimuli (figure 2b; n = 10, median = 2.65 s, V = 8, p = 0.049), they did not show any directional choice when the alternative to their own subspecies signal was their most neighbour hybrid category (H7, figure 2b; n = 10, median = 2.07 s, V = 18, p = 0.375). Female domesticus did not show any directional choice when presented with musculus versus domesticus stimuli (figure 2a; n = 9, median = −0.92, V = 20, p = 0.8203).

• Generality. When border musculus mice were presented with a choice between a musculus stimulus from close allopatry (instead of border population in Discrimination against hybrids) versus a hybrid stimulus, the allopatric signal was investigated significantly more than the hybrid one (paired Wilcoxon test: n = 10, median = 10.54, V = 54, p = 0.004).

• Variation. Remarkably, preference varied neither with the hybrid category presented as an alternative to the musculus stimulus (table 1; maximal model: F₁₃₈ = 1.18, p = 0.319), nor interactively with the sexes (table 1). However, male musculus were choosier than the females (table 1; minimal model: F₁₆₆ = 9.85, p = 0.002), a pattern which did not vary with motivation (table 1; maximal model: F₁₃₈ = 0.62, p = 0.434). Considering the direction of preference for hybrids across the zone, a consistent trend was for hybrid stimuli to be less investigated than musculus ones (figure 3; males: n = 84, median = 6.19 s, V = 3335, p = 4.832 × 10⁻¹²; females: n = 61, median = 0.91 s, V = 1406, p = 9.529 × 10⁻⁴).

Figure 2.

Results of two-way choice tests assessing preference of musculus and domesticus mice from the border of the hybrid zone when one of the stimuli represents their geographically closest hybrid neighbours. The figure shows preferences of (a) females and (b) males (left, musculus; right, domesticus mice) when the homosubspecific stimulus is presented against a heterosubspecific (white) or a hybrid (grey) stimulus. Identity of stimuli is indicated above (homosubspecific) and below (heterosubspecific or hybrid) each plot. Preference is calculated as the difference in time spent with one or the other stimulus (positive when assortative). The horizontal dotted line represents random choice (preference = 0). Box-plots show preference median (thick line), first and third quartiles, and whiskers represent the entire data range. Sample sizes (n) and p-values of paired Wilcoxon rank tests (p) are given below each plot. Preference values were tested against random choice (*p < 0.05, **p < 0.01).

Table 1.

A summary of the results of a mixed ANCOVA testing the influence of sex, stimulus category, the nose population and motivation^a, on variation of preference (expR) displayed by border musculus mice, during choice tests between musculus versus a hybrid or a domesticus stimulus. (Bold p-values highlight significant effects of the variable.)

model		explanatory variable	fixed/random	n.d.f.	d.d.f.	F	p-value
maximal model	expR ∼	sex	F	1	138	9.23	0.003
AIC: 64.1		motivation	F	1	138	0.62	0.434
		stimulus categoryb,c	F	6	138	1.18	0.319
		sex × motivation	F	1	138	0.13	0.724
		sex × stimulus categoryb,c	F	6	138	0.9	0.499
		motivation × stimulus categoryb,c	F	6	138	1.26	0.279
		sex × motivation × stimulus categoryb,c	F	6	138	1.05	0.395
		population	R			0.823d	0.365
minimal model	expR ∼	sex	F	1	166	9.85	0.002
AIC: 39.5

^aMotivation is the total time spent in either the right or left part of the maze (by opposition to the neutral central part), a measure of the mice willingness to participate in the test.

^bHybrids plus domesticus.

^cH5 and H6 were grouped for this analysis, see text.

^dLikelihood ratio.

Figure 3.

Results of preference assessing attractiveness of urinary hybrid stimulus categories sampled across the hybrid zone. Preferences of musculus individuals from the border of the hybrid zone (a, female noses; b, male noses) are shown as the difference in time spent in contact with the homosubspecific stimulus (musculus border, indicated as ‘mus’. above the plot) versus a more distant hybrid stimulus (‘H1’, ‘H2’… specified below the plot). Preference data is as in figure 2. The plot colour code follows that described in figure 1 (from light to dark grey: from south to north of the hybrid zone). The estimated hybrid indexes (HI) of the hybrid stimuli presented against musculus stimulus, and sample size (n: number of noses involved in each test) are given below each plot. The plots on the right indicate preference results when data for all hybrid categories were considered as a block after mixed ANCOVA analysis revealed no significant difference between the stimulus categories; preference values were tested against random choice (Wilcoxon tests ***p < 0.001).

(b). Did discrimination against hybrid signals evolve in the hybrid zone?

Allopatric musculus mice did not show any directional preference when presented with a choice between musculus versus hybrid stimuli (table 2; males: n = 17, median = 0.74, V = 94, p = 0.431; females: n = 10, median = 0.70, V = 40, p = 0.232), while border mice presented with the same stimuli discriminated against the hybrid one (table 2; males: n = 17, median = 5.91s, V = 130, p = 0.009; females: n = 10, median = 0.81s, V = 49, p = 0.027).

Table 2.

Results of preference tests assessing whether musculus discrimination against hybrids evolved in allopatry or in the contact zone. (Preference is time spent in contact with musculus border stimulus minus with H1 neighbour hybrid stimulus (positive values means homogamous preferences). Sample size (n) are given alongside statistics of paired Wilcoxon tests. Bold p-values highlight significant deviance from the null hypothesis (H0: noses spent the same time in contact with the two stimuli).)

	border of hybrid zone					allopatry
	median	first/third quartiles	n	V	p	median	first/third quartiles	n	V	p
females	0.805	−0.0300/2.66	10	49	0.027	0.695	−0.220/1.32	10	40	0.232
males	5.910	−0.320/9.33	17	130	0.009	0.740	−3.89/5.10	17	94	0.431

(c). How similar are hybrid and parental odours?

Assortative preference displayed by border musculus mice (results above) indicated that hybrids and musculus signals differed. We hence compared hybrid and domesticus signals. The results of the habituation–discrimination experiment indicated that signals of hybrid categories neighbouring (H7) and most distant from domesticus (H1) were perceived as different from domesticus (respectively, figure 4a,b: n = 8, V = 35, p = 0.016, and n = 8, V = 36, p = 0.008). Moreover, despite its geographical proximity and genetic similarity, H7 was not perceived as more similar to domesticus than H1 (figure 4c: n = 13, V = 62, p = 0.273). Finally, unlike predicted if hybrid signals shared similarities with the two parental subspecies, when musculus and H1 stimuli were presented simultaneously after habituation to domesticus, H1 signals did not show more similarities with domesticus than did musculus signals (figure 4d: n = 10, V = 28, p = 1).

Figure 4.

(a,b) Results of the discrimination and (c,d) generalization tests assessing similarities between hybrid signals and parental odours. The noses were musculus border males for all experiments. The stimulus used for habituation was domesticus and the stimuli used during discrimination (or generalization) are indicated below each plot. Results are presented as the average ± standard error of time spent in contact with each of the test stimuli. The plot colour code refers to the geographical origins of the stimuli as in figures 1 and 3 (light for the domesticus side and dark for the musculus side of the zone). Results of paired comparisons: ‘n.s.’ p > 0.05, *p < 0.05, **p < 0.01.

4. Discussion

Selection against hybrids that carry less attractive signals could be an important mechanism hindering gene flow between parapatric or sympatric populations [12,19,45,46]. However, this process remains scarcely demonstrated in nature. We investigated this process in a house mouse hybrid zone where hybrid populations with highly recombined genomes are established between the parental ones. We hypothesized that sexual selection against hybrids could take part in the reinforcement process described between musculus and domesticus [40,41] by decreasing hybrid mating success and hence impeding step by step gene flow through the zone. Our results indicate that hybrids are perceived as less attractive than own subspecies stimuli only in musculus populations from the border of the hybrid zone, pointing out character displacement of preference between allopatric and border populations of this subspecies, and a potential for sexual selection against hybrids on the musculus side of the zone. Studies involving other models, e.g. flycatchers [47] and chorus frogs [48], evaluated that sexual selection could account, respectively, for three-quarters and four-fifths of the reduction in hybrid male fitness, suggesting that sexual selection against hybrids could be an important force in the evolution of reproductive isolation in contact zones. Here, asymmetric sexual selection against hybrids could take part in the reinforcement process already described in the zone on the basis of parental odour and preference traits [40,41], and limit admixture in border musculus populations.

Our results also indicate that the unattractiveness of hybrids did not vary with genetic similarities between hybrid and parental populations, suggesting that the hybrid signal may not significantly vary across the zone and/or that it could share an unattractive component sufficient to trigger a negative response (see below). This study also points out that asymmetry exists between the sexes, as choosiness was higher in males compared with females.

(a). Lower choosiness of females

The extent to which males and females contribute to behavioural isolation varies between taxa [49,50] and findings, such as reported here, that male mate choice plays an important role are not an exception [51–55]. In the house mouse, both males and females may be involved in mate choice [56–58].

The cost of heterospecific mating is expected to be similar in the two subspecies [59] but higher in females. First, because female physiological investment in each reproduction event is supposed to be higher than male investment [60], but see [52]. Second, in our biological system, sterility affects hybrid males more than hybrid females [26,28,61,62]. This should result in higher selection on females than males to discriminate against hybrid mates, which our results partially contradict. The patterns of lower female choosiness pointed out in our study could be explained if postmating–prezygotic mechanisms preventing the production of costly hybrid zygotes occur [63]. Indeed, in vitro laboratory investigations involving the two house mouse subspecies [64] suggest that sperm competition may favour homogamous fecundation; such a mechanism could possibly put less pressure on the evolution of female premating choosiness, explaining the patterns evidenced in our study.

Notwithstanding, we cannot completely exclude that lower preference of females in our study could be caused by the experimental design being less appropriate to assess female than male mate choice. Indeed, it may be that simultaneous presentation of two male stimuli to a female induces anxiety, while for males simultaneous presentation of two female stimuli may induce higher pressure to properly allocate their energy [52,65].

(b). Gaining insight into hybrid signal unattractiveness

Hybrid unattractiveness for border population mice did not vary significantly with the origin of hybrid stimuli, suggesting that different categories of hybrids were perceived as equally unattractive. The theory of odour–genes covariance proposes that odour similarity may parallel genetic similarity [37]. If this was true, we would have expected odour similarity between parental and hybrid stimuli to vary with the hybrid index HI, which did not seem to be the case because stimuli of the most different hybrids were discriminated against to the same extent and perceived as equally different from domesticus.

A study assessing preference of wild hybrids when presented with a choice between stimuli of musculus versus domesticus [16] revealed dominance of the preference for domesticus and a shift in favour of higher choosiness and preference for musculus 10 km north of the genetic centre of the Danish hybrid zone (figure 1), in a location where hybrid genomes were very similar to musculus. Our study concerns the same hybrid zone, hence our expectation that, if hybrid preference followed a self-matching scheme, hybrid odours would be dominantly domesticus with a shift in the hybrid signal nature roughly at the same location as the preference shift. Precisely, we expected H7, neighbouring the domesticus border of the hybrid zone, to be more similar to domesticus than H1, which does not seem to be the case. Further, if the hybrid signals were intermediate (i.e. carried similarities with both parents), we would have expected H1 to be perceived as more similar to domesticus than musculus. Our study indicates that all hybrid signals tested were rejected by musculus noses (during choice tests), but not because they had similarities with the domesticus signal, leading us to propose that hybrid signals may share an unattractive transgressive component, i.e. out of the range of variation of the two subspecies (see also [66]).

(c). Evolution of sexual selection against hybrids

The fact that, unlike border populations, mice from neighbour allopatry did not discriminate against hybrid stimuli suggests that hybridization may not produce signals that are not recognized by musculus or unattractive per se, and that the perception of hybrid signals as unattractive has evolved in the border of the hybrid zone. Discrimination against hybrid signals could be the consequence of changes in the olfactory receptors leading to non-recognition of hybrid signals in the contact zone, or, possibly, changes in the neuronal integration of the message transmitted by the receptor (discrimination leading to negative behavioural response). Evolution of hybrid perception by musculus receivers could be a pleiotropic consequence of narrowing the range of acceptable signals during evolution of assortative mating (i.e. preference for alike) in response to selection against maladaptive hybridization with domesticus. Alternatively, natural selection against possibly maladaptive backcross hybridization [26,59,61] could have directly influenced the pattern evidenced in this study. In both cases, if after selection in secondary contact musculus receivers are finely tuned to the extent that they are able to reject any signal different from their ‘own’ (i.e. self-matching), one may consider that strong signal divergence is not necessary to induce premating isolation in a hybrid zone, which might be sometimes the case [67]. As far as the mouse hybrid zone is concerned, our results suggest that both the musculus receiver component has diverged and the hybrid signals are different from the parental ones.

(d). Sexual selection against hybrids and the hybrid zone dynamics

Sexual selection against hybrids, as our study suggests, can contribute to strengthen reproductive isolation between diverging taxa [11,19,45]. Further, asymmetry in premating divergence between parental populations has been reported in many taxa [68–72] and can impact the dynamics of hybrid zones [73]. Here, asymmetric sexual selection against hybrids could have similar consequences. Particularly, given that domesticus, unlike musculus, tends not to discriminate against hybrids (this study) and that hybrids tend not to reject domesticus [16], we could expect backcrossing to occur more frequently on the domesticus side of the hybrid zone, facilitating some domesticus gene flow into the zone. Further, several studies suggest that domesticus is dominant over musculus and more aggressive (review in [74]), a behavioural trait that might be an advantage when dispersing [75]. In the absence of geographical or ecological constraints (e.g. absence of favourable habitats), aggressiveness might facilitate domesticus progression across the zone. Nevertheless, about 10 km north of the genetic centre of the zone, this progression could be hindered by lower attractiveness of domesticus mice to hybrids in this region [16], and strong assortative mating of musculus border population mice would further slow down domesticus advance. Under such a hypothesis, the advance of domesticus into musculus territory would meet a resistance linked to behavioural discrimination on the musculus side, described in this study. Such predictions should be further confronted to the results of other approaches bringing information on demographic parameters, dispersal characteristics, population growth rate etc. which are scarce for both subspecies in the context of the hybrid zone.

5. Conclusion

This study provides arguments in favour of a role for sexual selection against hybrids in shaping mate recognition patterns and limiting gene flow in this house mouse hybrid zone. Together with natural selection against hybrids, it could significantly contribute to reproductive isolation between the two European subspecies of the house mouse. New insights into the neurophysiology, chemistry and genetic bases of discrimination and signalling components will further extend our understanding of the evolutionary forces in action in this study model for speciation with gene flow.

This study was performed under the permission of the French authorities to G.G. (authorization no. C34-265).

Data accessibility

Hybrid Index estimations are uploaded as the electronic supplementary material, table S4. Raw behavioural data is available from the Dryad Digital Repository at http://doi.org/10.5061/dryad.qg2rk.

Funding statement

This work was financially supported by the French National Research Agency (ANR, No. 2010BLAN171401-AssortMate), ISEM 2013-167.