The Effect of Dominance on the Use of the Q_ST − F_ST Contrast to Detect Natural Selection on Quantitative Traits

THE comparison between Wright's among-population fixation index F_ST (a descriptor of the effect of the breeding structure on population diversity for neutral genes) and Spitze's quantitative index of population divergence Q_ST [defined as V_b/(V_b + 2V_w), where V_b and V_w are, respectively, the between-population and the additive within-population components of variance for the trait considered] is widely used to assess the relative importance of selection and drift as determinants of genetic differentiation for quantitative traits. For neutral genes with additive gene action between and within loci, Q_ST is the quantitative analog of F_ST. Otherwise, divergent or convergent selection is invoked, respectively, as a cause of the observed increase (Q_ST > F_ST) or decrease (Q_ST < F_ST) of Q_ST from its neutral expectation (Q_ST = F_ST). However, the validity of the Q_ST − F_ST contrast has been questioned, as neutral nonadditive gene action (dominance and/or epistasis) can mimic the additive expectations under selection (López-Fanjul et al. 2003). Recently, Goudet and Büchi (2006) have compared Q_ST and F_ST using computer simulation for dominant (nonepistatic) loci and have concluded that (1) on average, dominance decreases the value of Q_ST relative to F_ST (i.e., Q_ST − F_ST < 0); (2) the magnitude of the contrast Q_ST − F_ST increases with population differentiation (i.e., with increasing F_ST); and (3) dominance is unlikely to produce the result Q_ST − F_ST > 0. In this Letter to the Editor, we question the evidence that led to these claims.

It is well known that inbreeding can change the magnitude of the contribution of neutral dominant loci (with or without additional epistasis) to the between-population and the additive within-population components of the genetic variance of a quantitative trait, relative to their additive expectations (Robertson 1952; Willis and Orr 1993; López-Fanjul et al. 2002; Barton and Turelli 2004). Therefore, those changes can also affect the magnitude and sign of the contribution of such loci to Q_ST, depending on their effects and allele frequencies as well as on the pertinent F_ST value. Using the formulas given in López-Fanjul et al. (2003), we show in Figure 1 the contribution of a partially recessive diallelic locus (h = 0.25; see below) to Q_ST − F_ST, as a function of F_ST and the original frequency of the recessive allele (q). Generally, Q_ST − F_ST < 0 for q < 2/3 and Q_ST − F_ST > 0 otherwise, but |Q_ST − F_ST| absolute values for q < 2/3 were larger than those for q > 2/3. In addition, for all allele frequencies, |Q_ST − F_ST| values increased with F_ST until a maximum was reached when F_ST was close to 0.5 and then declined. Varying degrees of dominance h (0 ≤ h ≤ 1/2, where h = 0 or 1/2 for complete recessive or additive gene action, respectively) resulted in essentially the same pattern but, for each frequency and F_ST, the corresponding |Q_ST − F_ST| values increased as h departed from h = 1/2, and the frequencies at which Q_ST − F_ST < 0 went from q < 3/4 (h = 0) to q < 1/2 (h close to 1/2) (not shown). Of course, Q_ST = F_ST for h = 1/2, irrespective of allele frequencies. In Goudet and Büchi (2006) notation, h = (a − d)/2a, where 2a is the difference between the genotypic values of the two homozygotes at a diallelic locus and d is the value of the corresponding heterozygote.

Figure 1.—

The contribution of a partially recessive locus (h = 0.25) to Q_ST − F_ST plotted against F_ST, for different frequencies of the recessive allele (q).

Goudet and Büchi (2006) presented results averaged over uniformly distributed allele frequencies and concluded that Q_ST − F_ST < 0 for dominant loci. However, this is the obvious outcome of averaging Q_ST − F_ST values of a different sign, which are negative over a wider range of allele frequencies. As pointed out by Turelli and Barton (2006), this procedure can be misleading as it ignores that qualitatively different results are obtained for different allele frequencies and, therefore, the critical role of particular loci is lost when one averages over the whole possible range of original allele frequencies. Thus, the conclusion of Goudet and Büchi (2006) that dominance is unlikely to cause Q_ST − F_ST > 0 may not be valid for a plausible region of parameter space. Furthermore, the magnitude of Q_ST − F_ST contributed by single loci did not monotonically increase with population differentiation, as indicated by Goudet and Büchi (2006), but it did so only until a value of F_ST ≈ 0.5 was reached and then subsequently decreased to zero. Our formulas apply to the classical “island model” and give the expected contribution of single loci at equilibrium to the between-line and the additive within-line components of the genetic variance of a quantitative trait (or to Q_ST) after t consecutive bottlenecks of size N or, when bottleneck sizes are not constant from generation to generation, they can be alternatively expressed in terms of the inbreeding coefficient after t generations for the loci affecting the trait. These properties have also been incorrectly interpreted by Goudet and Büchi (2006; see their discussion section, p. 1344).

In the simulations by Goudet and Büchi (2006), the trait studied was encoded by 10 loci with 10 alleles/locus. For each locus, 10 additive allelic effects and 45 dominance deviations were drawn independently from a normal distribution N(0, 1), and genotypic values were obtained as the sum of the additive effects of the pertinent alleles plus the corresponding dominance deviation in the case of heterozygotes. Individual genotypic values were made equal to the sum of the genotypic contributions of the 10 loci considered. This procedure, however, has several drawbacks. First, it implies that the additive and dominance effects are independent, an unrealistic assumption as it is well known that loci with large homozygous effects tend to be recessive. Second, the additive and dominance effects were assigned arbitrarily, thereby resulting in complex dominance relationships among the 10 alleles at each locus (including overdominance and underdominance); i.e., a large region of the parameter space being explored is biologically unrealistic, as overdominance (or underdominance) has been empirically discarded as a common explanation of heterosis at the single-locus level. Third, and more important, the method implies averaging over a wide range of dominance deviations and allele frequencies; therefore, the crucial role of a particular type of gene action is again lost. For instance, after bottlenecks, recessive (nonepistatic) loci will contribute only to an excess of the within-population additive variance (or to Q_ST − F_ST < 0) if the recessive alleles segregate at low frequencies, but overdominant loci will do so only for equilibrium (intermediate) frequencies (López-Fanjul et al. 2002).

To investigate these points, we have simulated the simplest case of a single locus with two alleles, originally segregating at frequencies q and 1 − q, with additive and dominance effects sampled from a normal distribution as in Goudet and Büchi (2006). Starting from a large base population at Hardy–Weinberg equilibrium, lines were derived and subsequently maintained with effective size N = 10 during a number of generations; the pertinent values of Q_ST − F_ST plotted against F_ST are given in Figure 2 for a range of initial frequencies 0.1 ≤ q ≤ 0.9 and inbreeding coefficients 0 < F_ST < 1 (1500 replicates). Our results clearly show that Q_ST was always smaller than F_ST (Q_ST − F_ST < 0) for all allele frequencies (as found by Goudet and Büchi 2006); i.e., they are artifacts of averaging the Q_ST values obtained for the same allele frequency and F_ST over the distribution of dominance effects generated by sampling and are not representative of the general situation described by our analytical model.

Figure 2.—

Simulation results giving the contribution of a single diallelic locus to Q_ST − F_ST (additive and dominant effects drawn from a normal distribution) plotted against F_ST, for different allele frequencies.

In summary, the extension of single-locus models to multilocus systems can have different outcomes, as the contribution of specific dominant loci to the Q_ST − F_ST value for a quantitative trait depends, both in sign and magnitude, on the variance of the average effects of gene substitution, the covariance between these effects and the corresponding heterozygosities, and the allele frequencies at the loci involved (López-Fanjul et al. 2003). In other words, generalizations to quantitative traits can be made only if individual loci show the same type of gene action and segregate with similar frequencies.

Finally, it is generally accepted that alleles of large effect on a given trait tend to be recessive and deleterious and, therefore, natural selection holds their frequencies at low values. This is the case of Drosophila viability, where Wang et al. (1998) have shown that the observed changes in mean, additive genetic variance, and between-line variance following bottlenecks can be mainly attributed to lethals and partially recessive mutations of large deleterious effect, giving Q_ST < F_ST [Q_ST = 0.17 or 0.20 for F_ST = 0.25 or 0.5, respectively (López-Fanjul et al. 2003)]. However, this may not be the case with other quantitative traits where loci of relatively smaller effects and varying degrees of dominance can contribute differentially to a joint Q_ST − F_ST value, which may be of either sign or even statistically undistinguishable from zero.