Evolutionary theory for modifiers of epistasis using a general symmetric model

Abstract

Genetic interactions in fitness are studied by using modifier theory. The effects on fitness of two linked genes are perturbed by alleles at a third linked locus that controls the extent of epistasis in fitness between the first two. This epistasis is determined by a symmetric interaction matrix, and it is shown that a modifier allele that increases epistasis will invade when the linkage between the other two genes is sufficiently tight and these genes are in linkage disequilibrium. With linkage equilibrium among the major loci, increased or decreased epistasis may evolve depending on the allele frequencies at these loci.

In quantitative genetics, epistasis is viewed as a contribution to overall statistical variation in a phenotype by terms due to gene-by-gene interactions. A number of recent studies have focused on the evolution of this epistatic variance (1–3). One conclusion of these studies is that, for the most part, evolution should proceed in the direction of more modularity, that is, toward a decrease in the relative contribution of the epistatic component of variance (4, 5).

Epistatic variance in the context of quantitative genetics is a statistical average of contributions in a linear analysis-of-variance framework. However, in biological terms, there may be substantial variability in the amount of interaction between genes and the type of interaction. Phillips et al. (6) claim that evolutionary genetics “may have significantly underestimated the importance of gene interactions” by focusing on statistical averages of interaction effects. Wright (7) suggests that overall genotypic fitness is likely to involve interaction among a large fraction of genes in the genome. Further, the evolutionary dynamics of linked genes may be sensitive to the precise nature of the epistatic interactions among them. Indeed, variance in epistasis can generate different evolutionary outcomes (8).

Although there was a period in the 1990s during which the existence and importance of epistasis were somewhat neglected, there has been a shift in the last decade toward more detailed modeling, both at the evolutionary level and in the analysis of quantitative variation (1–3, 9, 10). There is an emerging consensus that intraspecific genetic interactions may be important in development and evolution (11, 12). Different approaches to modeling the evolution of epistasis have developed, depending on whether the approach originates in the statistics of quantitative inheritance (1) or in the formal population genetics of genetic modifiers of fitness (13).

A recent computational approach to the evolution of epistasis (14) is based on an artificial gene network model (15, 16) originally introduced to study the evolution of genetic robustness. The model in ref. 14 also includes mutation in the gene networks, and among the results of the simulations is the finding that recombination strongly affected the direction of the epistasis that evolved during selection for genetic robustness. The claim in ref. 14 is that negative epistasis among genes should be widespread in nature, because there are so many interacting gene networks and so much genetic variation in real organisms.

Much of the computational analysis of multiple gene systems has focused on haploids with deleterious mutations only (e.g., ref. 17). For such haploid systems, it is easy to write fitness as a function of the deleterious effect of each single mutation and a single epistatic parameter that describes how the multiple mutant fitness decays as a function of the number of mutants; the latter is the epistasis parameter. The analysis in ref. 17 suggested that in haploids with deleterious mutations and positive epistasis, evolution should reduce this epistasis during selection to reduce the overall genetic load.

Epistasis in diploids is somewhat harder to study, because each set of chromosomes must be evaluated in the backgrounds of all chromosomes. Thus, it is clear that some assumptions will be necessary just to pose the problem of the evolution of epistasis. The present study models explicit epistatic fitness interactions between a pair of loci in a diploid genetic system and follows the fate of alleles at a third locus, which modifies the fitness epistasis between the first two. To do this, we must address the kinds of evolutionary dynamics expected under epistatic selection, including the roles of linkage disequilibrium and recombination. To this end, we consider a “baseline” model of additive fitnesses that entails no additive interactions between the loci and frames the problem as the evolution of modifier alleles that cause departure from additivity (see, e.g., ref. 18). In an earlier study of this kind with the Lewontin–Kojima (19) model of fitness interactions, we found that an allele that increased epistatic interactions would increase in frequency when introduced near a stable state of linkage disequilibrium (13). In this work, we study fitness schemes that are not as symmetric as the Lewontin–Kojima viabilities, although they fall under a more general class of symmetric viability models (20, 21). We show that, with sufficiently tight linkage between the major loci, epistasis will tend to increase, but that with some general classes of symmetric viabilities, looser linkage will result in the success of alleles that decrease epistasis.

Model

The model assumes a pair of “major” loci with two alleles A, a at the first and B, b at the second, and a third “modifier” locus whose sole function is to determine properties of the fitness parameters at the major loci. At the modifier locus, there may be n possible alleles M₁, M₂, …, M_n. Thus, there are four possible gametes at the two major loci AB, Ab, aB, ab numbered 1, 2, 3, 4, respectively, and 4n three-locus gametes ABM_i, AbM_i, aBM_i, abM_i, for i = 1, 2, …, n.

The modifier locus determines the fitness parameters at the major loci with a family of 4 × 4 fitness matrices W^ij = [w_kl^ij] for k, l = 1, 2, 3, 4 and i, j = 1, 2, …, n. For example, w₁₁^ij is the fitness of the three-locus genotype ABM_i/ABM_j, and w₂₃^ij is the fitness of AbM_i/aBM_j, etc. The fitness parameters are symmetric; that is, w_kl^ij = w_kl^ji = w_lk^ij = w_lk^ji for all k, l = 1, 2, 3, 4 and i, j = 1, 2, …, n. In addition, there are “no position” effects in the fitness parameters, so that for all i, j = 1, 2, …, n, w₁₄^ij = w₂₃^ij.

Recombination occurs between the two major loci as well as between the modifier locus and the major loci. For simplicity (but with no effect on the results), take the first two loci to be the major loci with the modifier locus next to them, as shown in Fig. 1. Recombination can occur in the two genetic intervals I₁ and I₂, resulting in four mutually exclusive recombination events (0, 0), (0, 1), (1, 0), (1, 1), where (0, 0) denotes that no recombination occurred in both I₁ and I₂, (1, 0) indicates that recombination occurred in I₁ but not in I₂, etc. There are, therefore, four crossover probabilities c₀₀, c₀₁, c₁₀, c₁₁, such that c₀₀ + c₀₁ + c₁₀ + c₁₁ = 1.

Fig. 1.

Organization of three-locus model.

If r is the recombination rate between the two major loci and R between the modifier locus and the two major loci, then

The crossover probabilities can be represented as

where ζ measures the extent of interference in recombination. Following Haldane (22), ζ < 0 corresponds to the case of negative interference, ζ > 0 to positive interference, and ζ = 0 entails no interference, in which case the recombination events in I₁ and I₂ are independent.

Let the frequencies of the four gametes at the major loci, AB, Ab, aB, ab, in the “present” generation be x₁, x₂, x₃, x₄, respectively. The three-locus frequencies of the 4n possible gametes can be represented as

where 0 ≤ p_i, q_i, r_i, s_i ≤ 1, and Σ_i p_i = Σ_i q_i = Σ_i r_i = Σ_i s_i = 1. Here, we use notation suggested by Lessard (23), in that p_i, for example, is the frequency of ABM_i among AB gametes. The population state at the present generation is thus specified by the frequency vectors x, p, q, r, s, where

Let x′, p′, q′, r′, s′ be the frequency vectors for the population state at the “next” generation after random mating, recombination, Mendelian segregation, and viability selection have taken place. The transformation of frequencies from the present to the next generation is given in equations 2.12–2.15 in ref. 13. For ready reference, see supporting information (SI) Supporting Text 1.

The modifier locus determines the fitness parameters at the major loci through the fitness matrices W^ij = [w_kl^ij]. Specifically, w_kl^ij is the fitness parameter associated with the kl genotype at the major loci with the genotype M_iM_j at the modifier locus.

We assume that fitness modification is “linear,” namely

for all k, l = 1, 2, 3, 4, and i, j = 1, 2, …, n. Thus the “nonmodified” or the “basic” fitness matrix is W = [w_kl], and in the presence of the genotype M_iM_j at the modifier locus, the “modified” fitness parameters are w_kl + ε_ijδ_kl, where ε_ij represents the effect of M_iM_j on epistasis.

The matrices W = [w_kl], Δ = [δ_kl], and ε = [ε_ij] are symmetric matrices; W is a positive matrix; and because we assume no position effects in fitness, we also have w₁₄ = w₂₃ and δ₁₄ = δ₂₃.

Invasion by an Epistasis Modifier

In the simplest case, the population is initially at a stable equilibrium where the allele M₁ at the epistasis-modifying locus is fixed, and the epistasis parameter is ε₁₁. A new allele M₂ is introduced near this equilibrium, producing epistasis parameters ε₁₂ for M₁M₂ and ε₂₂ for M₂M₂. We seek conditions on ε₁₁ and ε₁₂ that will enable M₂ to increase when rare, that is, conditions for the original equilibrium with M₁M₁ to be externally unstable.

Suppose the initial equilibrium frequencies of ABM₁, AbM₁, aBM₁, and abM₁ are x₁*, x₂*, x₃*, x₄*, respectively, and write

Then the frequency vector x* = (x₁*, x₂*, x₃*, x₄*) satisfies

with k = 1, 2, 3, 4 (and “−” for k = 1, 4 and “+” for k = 2, 3).

The external stability of x* to invasion by M₂ is given by the linear approximation ℒ for the frequencies p₂, q₂, r₂, s₂ of ABM₂, AbM₂, aBM₂, and abM₂. This external stability is determined by the largest root in absolute value of the fourth-degree characteristic polynomial Q(z) associated with . Using ref. 13, we find that

where

l* = δ₁₄x*₁x*₄ − δ₂₃x*₂x*₃, d* = w*₁₄x*₁x*₄ − w*₂₃x*₂x*₃, δ* = Σ_l,k=1⁴ δ_lkx*_lx*_k, and

Because the characteristic polynomial Q(z) is a fourth-degree polynomial with a positive coefficient of z⁴, when Q(1) < 0, there is a positive eigenvalue larger than one, and the equilibrium x* is externally unstable; that is, the new epistasis modifying allele M₂ increases in frequency when it is rare. In view of Eq. 8, if det(M̃) < 0, then M₂ invades if its epistasis ε₁₂ is greater than the epistasis ε₁₁ produced by the resident modifier genotype M₁M₁.

In SI Supporting Text 2, we show that, if an arbitrary number of epistasis-controlling alleles M₁, M₂, …, M_n are present at a specific class of equilibria called viability analogous Hardy—Weinberg (VAHW) equilibria (24), then the condition for a new allele M_n+1 to increase when rare is completely analogous to Q(1) < 0 with an appropriate modification of Q(1) to account for multiple modifying alleles at the M locus.

We now address the conditions on the equilibrium x* and hence on the matrices W = [w_kl] and Δ = [δ_kl], that entail det(M̃) < 0.

Choosing W and Δ

We use the linear epistasis modification scheme given by Eq. 5. Because we are interested in the evolution of epistasis, it is natural to assume that W involves no epistasis; that is, W = W_add is an additive fitness matrix that can be represented as

The interaction matrix Δ determines how each modifier genotype M_iM_j with associated epistasis measure ε_ij affects the fitness of each genotype at the major locus. Following Bodmer and Felsenstein (20), there are four additive row epistasis measures associated with the scheme (Eq. 5) given by

for k = 1, 2, 3, 4 and i, j = 1, 2, …, n. Because W = [w_kl] is W_add, its four additive row epistasis measures are zero, and in our linear scheme, we have for each row k = 1, 2, 3, 4

A simple and natural assumption is that the interaction matrix Δ = [δ_kl] is such that for given i, j, all four row epistasis measures e_k^ij are the same in absolute value. This assumption gives rise to an interaction structure analogous to that in standard linear models for two-way analysis of variance. Liberman and Feldman (13) assume that e₁^ij = −e₂^ij = −e₃^ij = e₄^ij, and Δ is chosen to be

Another epistatic scheme takes e₁^ij = e₂^ij = e₃^ij = e₄^ij, which is the case for the two-parameter family of Δ matrices:

where

In both cases, results can be obtained concerning the external stability of VAHW equilibria x* ⊗ y* when the major locus equilibrium x* is either in linkage equilibrium, D* = x₁*x₄* − x₂*x₃* = 0, or in linkage disequilibrium, D* ≠ 0. Analysis of the case D* ≠ 0 requires the additional assumption that W_add of Eq. 11 is symmetric, namely α₁ = α₃ and β₁ = β₃. With this additional assumption, for each ε_ij, the fitness matrix W_add + ε_ijΔ, where Δ is either Δ₀ or Δ(s, t), takes the form of the two-locus symmetric viability matrix introduced by Bodmer and Felsenstein (20) and gives rise to a two-locus symmetric equilibrium x̂ = (x̂₁, x̂₂, x̂₃, x̂₄) with

where D̂ = x̂₁x̂₄ − x̂₂x̂₃ is a solution of a cubic equation. By assuming that x* is in fact a symmetric equilibrium, the external stability analysis becomes tractable.

This leads us to ask in the next section how general is the choice of Δ that will enable analytical treatment to determine whether the evolutionary increase of epistasis shown in ref. 13 is a more general phenomenon.

General Symmetric Modification Scheme

Consider the modified fitness scheme W = W_add + εΔ, where W_add is given by Eq. 11. We assume that the interaction matrix Δ is such that if W_add is symmetric (α₁ = α₃, β₁= β₃), then for all ε, W = W_add + εΔ is a symmetric viability matrix of the form (20)

Hence, the interaction matrix Δ takes the form

With this choice of Δ, the four row epistasis measures are

When W_add is symmetric (α₁ = α₃, β₁ = β₃), the modified fitness matrix W_add + εΔ is a Lewontin–Kojima symmetric fitness matrix if d = a, and a Wright–Kimura symmetric fitness matrix if d = 2(b + c) − (a + 2e).

Moreover, when W is of the form (Eq. 18), the two-locus model with recombination rate r, corresponding to the two major loci, gives rise to a symmetric equilibrium x* = (x₁*, x₂*, x₃*, x₄*) of the form (Eq. 17), where D* = x₁*x₄* − x₂*x₃*, the linkage disequilibrium, is a solution of the cubic equation (21)

with l = 2(β + γ) − (α + δ), m = δ − α.

Observe that when α₁ = α₃, β₁ = β₃ in W_add, W_add + εΔ is of the symmetric form (Eq. 18) with

Thus,

and

Because the entries of Δ can be either positive or negative, we will assume, without loss of generality, that ε > 0.

From the cubic equation G(D) = 0, when l > 0, G (±∞) = ±∞, and so when m < 0, G(D) = 0 has a positive root, and a negative root when m > 0. In fact, for any W of the form (Eq. 18), we have the following result when l ≥ 0.

Result 1.

(i) When m < 0 and l ≥ 0, the cubic equation has a unique positive solution 0 < D* < ¼ for all 0 < r ≤ ½, with D* = ¼ when r = 0. Moreover, D* = D*(r) is a decreasing function of r.

(ii) When m > 0 and l ≥ 0, the cubic equation has a unique negative solution − ¼ < D* < 0 for all 0 < r ≤ ½ with D* = −¼ for r = 0. D* = D*(r) is an increasing function of r.

Remark.

When m = 0 and l ≠ 0, we have the Lewontin–Kojima model, where the cubic equation has three roots, D* = 0 and $D^{*}=\pm \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$4$}\right.\sqrt{1-\raisebox{1ex}{$8r$}\!\left/ \!\raisebox{-1ex}{$l$}\right.}$. The positive (negative) root is then a decreasing (increasing) function of r.

The proof is in SI Supporting Text 3 for r > 0 and in SI Supporting Text 6 for r = 0. Thus, in the general symmetric modification scheme, when W_add is symmetric, and Δ takes the form (Eq. 19), a unique symmetric equilibrium x* = (x₁*, x₂*, x₃*, x₄*) exists at the two major loci with any fitness matrix of the form W = W_add + εΔ. This allows us to carry out the external stability analysis of the equilibrium x* of the three-locus system.

Evolution of Epistasis Near D* ≠ 0 Equilibria

Let the selection modification scheme take the form W = W_add + εΔ, where W is a symmetric viability matrix. Let x* be a symmetric equilibrium of the two major loci with

Suppose that ε₁₁ and ε₁₂ are the epistatic parameters for M₁M₁ and M₁M₂. We then have the following result, whose proof is given for a general VAHW equilibrium in SI Supporting Text 4.

Result 2.

Consider the symmetric equilibrium x* = (x₁*, x₂*, x₃*, x₄*), where M₁ is fixed with epistatic parameter ε₁₁, and x₁* = x₄* = ¼ + D*, x₂* = x₃* = ¼ − D*, and D* ≠ 0. Then there exists an r_crit, such that this equilibrium is externally unstable to the introduction of a new modifier allele M₂ with greater associated epistasis (ε₁₂ > ε₁₁) provided 0 ≤ r ≤ r_crit, and a > 0, a + e > 0 in the interaction matrix Δ.

Evolution of Epistasis near D* = 0 Equilibria

Let the modification scheme be W = W_add + εΔ where W_add is additive of the form (Eq. 12) (not necessarily symmetric), and the interaction matrix Δ takes the symmetric form (Eq. 19). If ε₁₁ = 0, then with overdominance at each of the two major loci (α₁, α₃ < α₂; β₁, β₃ < β₂) for r > 0, there is a stable x* equilibrium with D* = x₁*x₄* − x₂*x₃* = 0. If ε₁₁ > 0, then only when d = a in Δ, namely when m = 0, can the symmetric equilibrium x* have D* = 0, because D* = 0 is then one of the roots of the cubic equation G(D) = 0 of Eq. 21. From ref. 13, a new modifier allele M₂ with associated epistasis ε₁₂ > ε₁₁ will invade if δ* = Σ_k,l=1⁴ δ_klx*_kx*_l > 0 provided R, r > 0, and |ε₁₂ − ε₁₁| is small. We have the following result, whose proof is given in SI Supporting Text 5 for the more general VAHW case.

Result 3.

Let x* be an equilibrium with M₁ fixed and epistatic parameter ε₁₁, and linkage equilibrium, D* = x₁*x₄* − x₂*x₃* = 0. Under mutually exclusive conditions that are independent of the recombination rates, a new modifier allele M₂ with associated epistasis value ε₁₂ for M₁M₂ that is greater than ε₁₁ or less than ε₁₁ may invade, inducing the evolution of increased or decreased epistasis, respectively.

Discussion

The models examined here represent a considerable generalization of that discussed by Liberman and Feldman (13). The interaction matrix Δ in Eq. 19 allows the analyses in the previous papers as special numerical cases that give the Lewontin–Kojima model or the Wright–Kimura model. When an epistasis-modifying allele is introduced near a stable state of linkage disequilibrium, the allele will succeed if it increases epistasis, provided the linkage of the major loci is tight enough. In the “supersymmetric” Lewontin–Kojima case, it turns out there is no restriction on this linkage: epistasis always increases. But the proof of Result 2 given in SI Supporting Text 4 specifies the conditions on r that make the linkage between the major loci tight enough that epistasis will increase.

Of course, when r is larger, these conditions do not hold, and the magnitude of epistasis will decrease. Importantly, these results do not depend on the sign of the linkage disequilibrium (as long as it is nonzero) or on the sign of the epistasis. The recombination R between the modifier and major loci also plays no qualitative role in these results. Similar findings applied for the success of recombination-reducing alleles near stable linkage disequilibrium, the so-called reduction principle of Feldman and Liberman (24).

Introduction of epistasis modifiers near states where D* = 0 may be regarded as a more special problem, because the classes of symmetric fitness models that allow D* = 0 are restricted. The conditions contained in SI Supporting Text 5, the proof of Result 3, allow epistasis to either increase or decrease near D = 0, provided the major locus allele frequencies are not at 0.5 (see also ref. 13).

The maintenance of linkage disequilibrium in large populations requires some level of epistasis, and the effect of this epistasis in preserving associations between genes is stronger the closer the linkage is between the genes. It is not surprising, therefore, that tight linkage facilitates the strengthening of epistasis when there is linkage is disequilibrium. In finite populations, however, mutation and random genetic drift can generate linkage disequilibrium in the absence of selection. It remains to be seen whether new alleles that introduce epistatic selection into such a neutral situation will have a greater chance of succeeding than fitness modifiers with no epistatic effects. In the same way, if one gene is maintained polymorphic by selection, and a new allele at another locus appears at low frequency and starts to increase by virtue of its interactions with the first locus, we can ask whether epistasis between the two loci will increase during transient evolution. This is a different question from the one studied in this paper, which assumes an equilibrium with linkage disequilibrium, and has yet to be addressed with formal analysis (see, e.g., ref. 25, pp. 155–156.)

There is ample evidence that epistatic selection is involved in adaptive variation that is often attributed to single-locus effects (26). On the other hand, there is little evidence that this epistasis is itself under control of other genes and might therefore be subject to evolutionary change. The detection of such effects might be facilitated by careful examination of genetic networks in which some nodes are known to exhibit signatures of selection.

Templeton (26) stresses the importance of epistasis in both evolutionary and quantitative genetic contexts: “Epistasis appears to be a nearly universal component of the genetic architecture of most common traits.” He also points out that epistasis is itself subject to change: “…epistasis itself can appear and disappear if the interaction between two loci itself interacts with additional genetic or environmental factors….” Our model addresses this analytically through the use of an epistasis-modifying gene, which certainly interacts with the two major loci.

There is expected to be a relationship between conditions on the selection regime that would favor an increase of recombination and those that would produce negative epistasis. Simulation studies (14) have suggested that sex (i.e., recombination) would lead to the evolution of negative epistasis in a situation of mutation to deleterious alleles. These results depended, however, on the rate of mutation; infrequent mutation seemed to maintain a low level of positive epistasis.

Our models have attempted to separate main single locus effects from interaction effects due to the epistasis modifier. In the case of multiple deleterious mutations, numerical and experimental studies (14, 17, 27, 28) have shown that evolution of epistasis is correlated with the strength of the fitness loss from single mutations. In fact, deleterious mutations of small effect may evolve epistasis of different signs depending on resource availability in the environment (27). This raises the interesting question of whether the evolution of epistasis determines the nature of environmental canalization, or the evolution of robustness determines the direction of epistasis.

Although our investigation has focused on only three loci, the number of parameters involved is much larger than seen in most studies on deleterious mutations and epistasis. The latter, as mentioned above, invoke only two parameters: the fitness loss per mutation and a parameter that determines the overall functional form of the epistasis. We require the fitness structure of the full two-locus system; thus, it is surprising that the results depend only on the change in epistasis and the rate of recombination. Of course, as with studies of deleterious mutations, linkage disequilibrium must be present and maintained during the evolution of epistasis at a geometric rate.

Our final point concerns the distinction we have drawn between the modifier gene M and the major loci A/a and B/b. Gene M is not “neutral” in the sense that a recombination or mutation modifier might be regarded. It is, in fact, part of the selection system in the same sense that a dominance modifier is. As a result, changes in the genetic background, including new alleles at an epistasis modifier, may alter the way in which selection affects an allele because of that allele's epistatic interactions. Under some circumstances, as we have shown, this alteration of the genetic architecture of fitness may depend on the extent of linkage between the components, not just on the pattern of epistatic fitness interactions.

Glossary

Abbreviation

VAHW

viability analogous Hardy–Weinberg.