Genotypic Complexity of Fisher’s Geometric Model

In his celebrated model of adaptation, Fisher assumed a smooth phenotype fitness map with one optimum. This assumption is at odds with the rugged.....

Abstract

Fisher’s geometric model was originally introduced to argue that complex adaptations must occur in small steps because of pleiotropic constraints. When supplemented with the assumption of additivity of mutational effects on phenotypic traits, it provides a simple mechanism for the emergence of genotypic epistasis from the nonlinear mapping of phenotypes to fitness. Of particular interest is the occurrence of reciprocal sign epistasis, which is a necessary condition for multipeaked genotypic fitness landscapes. Here we compute the probability that a pair of randomly chosen mutations interacts sign epistatically, which is found to decrease with increasing phenotypic dimension n, and varies nonmonotonically with the distance from the phenotypic optimum. We then derive expressions for the mean number of fitness maxima in genotypic landscapes comprised of all combinations of L random mutations. This number increases exponentially with L, and the corresponding growth rate is used as a measure of the complexity of the landscape. The dependence of the complexity on the model parameters is found to be surprisingly rich, and three distinct phases characterized by different landscape structures are identified. Our analysis shows that the phenotypic dimension, which is often referred to as phenotypic complexity, does not generally correlate with the complexity of fitness landscapes and that even organisms with a single phenotypic trait can have complex landscapes. Our results further inform the interpretation of experiments where the parameters of Fisher’s model have been inferred from data, and help to elucidate which features of empirical fitness landscapes can be described by this model.

A fundamental question in the theory of evolutionary adaptation concerns the distribution of mutational effect sizes and the relative roles of mutations of small vs. large effects in the adaptive process (Orr 2005). In his seminal 1930 monograph, Ronald Fisher devised a simple geometric model of adaptation in which an organism is described by n phenotypic traits and mutations are random displacements in the trait space (Fisher 1930). Each trait has a unique optimal value and the combination of these values defines a single phenotypic fitness optimum that constitutes the target of adaptation. Because random mutations act pleiotropically on multiple traits, the probability that a given mutation brings the phenotype closer to the target decreases with increasing n. Fisher’s analysis showed that, for large n, the mutational step size in units of the distance to the optimum must be smaller than $1/\sqrt{n}$ for the mutation to be beneficial with an appreciable probability. He thus concluded that the evolution of complex adaptations involving a large number of traits must rely on mutations of small effect. This conclusion was subsequently qualified by the realization that small effect mutations are likely to be lost by genetic drift, and therefore mutations of intermediate size contribute most effectively to adaptation (Kimura 1983; Orr 1998, 2000).

During the past decade, Fisher’s geometric model (FGM) has become a standard reference point for theoretical and experimental work on fundamental aspects of evolutionary adaptation (Tenaillon 2014). In particular, it has been found that FGM provides a versatile and conceptually simple mechanism for the emergence of epistatic interactions between genetic mutations in their effect on fitness (Martin et al. 2007; Gros et al. 2009; Blanquart et al. 2014). For this purpose, two extensions of Fisher’s original formulation of the model have been suggested. First, phenotypes are assigned an explicit fitness value, which is usually taken to be a smooth function on the trait space with a single maximum at the optimal phenotype. Second, and more importantly, mutational effects on the phenotypes are assumed to be additive. As a consequence, any deviations from additivity that arise on the level of fitness are solely due to the nonlinear mapping from phenotype to fitness, or, in mathematical terms, due to the curvature of the fitness function. Because the curvature is largest around the phenotypic optimum, epistasis generally increases upon approaching the optimal phenotype and is weak far away from the optimum. Several recent studies have made use of the framework of FGM to interpret experimental results on pairwise epistastic interactions and to estimate the parameters of the model from data (Martin et al. 2007; Velenich and Gore 2013; Weinreich and Knies 2013; Perfeito et al. 2014; Schoustra et al. 2016).

A particularly important form of epistatic interaction is sign epistasis, where a given mutation is beneficial or deleterious depending on the genetic background (Weinreich et al. 2005). Two types of sign epistasis are distinguished depending on whether one of the mutations affects the effect sign of the other, but the reverse is not true [simple sign epistasis (SSE)]; or whether the interaction is reciprocal [reciprocal sign epistasis (RSE)]. For a pictorial representation of the two kinds of sign epistasis, see, for example, Poelwijk et al. (2007). Sign epistasis can arise in FGM either between large effect beneficial mutations that in combination overshoot the fitness optimum, or between mutations of small fitness effect that display antagonistic pleiotropy (Blanquart et al. 2014). The presence of sign epistasis is a defining feature of genotypic fitness landscapes that are complex, in the sense that not all mutational pathways are accessible through simple hill climbing, and multiple genotypic fitness peaks may exist (Weinreich et al. 2005; Franke et al. 2011; de Visser and Krug 2014). Specifically, RSE is a necessary condition for the existence of multiple fitness peaks (Poelwijk et al. 2011; Crona et al. 2013).

Following common practice, here a genotypic fitness landscape is understood to consist of the assignment of fitness values to all combinations of L haploid, biallelic loci that together constitute the L-dimensional genotype space. A peak in such a landscape is a genotype that has higher fitness than all its L neighbors that can be reached by a single point mutation (Kauffman and Levin 1987). Note that, in contrast to the continuous phenotypic space on which FGM is defined, the space of genotypes is discrete.

Blanquart et al. (2014) showed that an ensemble of L-dimensional genotypic landscapes can be constructed from FGM by combining subsets of L randomly chosen mutational displacements. Each sample of L mutations defines another realization of the landscape ensemble, and the exploratory simulations reported by Blanquart et al. (2014) indicate a large variability among the realized landscapes. Nevertheless, some general trends in the properties of the genotypic landscapes were identified. In particular, as expected on the basis of the considerations outlined above, the genotypic landscapes are essentially additive when the focal phenotype representing the unmutated wild type is far away from the optimum and become increasingly rugged as the optimal phenotype is approached.

In this article we present a detailed and largely analytic study of the properties of genotypic landscapes generated under FGM. The focus is on two types of measures of landscape complexity, that is, the fraction of sign-epistatic pairs of random mutations and the number of fitness maxima in the genotypic landscape. A central motivation for our investigation is to assess the potential of FGM and related phenotypic models to explain the properties of empirical genotypic fitness landscapes of the kind that have been recently reported in the literature (Szendro et al. 2013; Weinreich et al. 2013; de Visser and Krug 2014). The ability of nonlinear phenotype-fitness maps to explain epistatic interactions among multiple loci has been demonstrated for a virus (Rokyta et al. 2011) and for an antibiotic resistance enzyme (Schenk et al. 2013), but a comparative study of several different data sets using approximate Bayesian computation (ABC) has questioned the broader applicability of phenotype-based models (Blanquart and Bataillon 2016). It is thus important to develop a better understanding of the structure of genotypic landscapes generated by phenotypic models such as FGM.

In the next section we describe the mathematical setting and introduce the relevant model parameters: the phenotypic and genotypic dimensionalities n and L, the distance of the focal phenotype to the optimum, and the standard deviation (SD) of mutational displacements. As in previous studies of FGM, specific scaling relations among these parameters have to be imposed to arrive at meaningful results for large n and L. We then present analytic results for the probability of sign epistasis and the behavior of the number of fitness maxima for large L, both in the case of fixed phenotypic dimension n and for a situation where the joint limit $n,L\to \mathrm{\infty }$ is taken at constant ratio $\alpha =n/L.$

Similar to other probabilistic models of genotypic fitness landscapes (Kauffman and Levin 1987; Weinberger 1991; Evans and Steinsaltz 2002; Durrett and Limic 2003; Limic and Pemantle 2004; Neidhart et al. 2014), the number of maxima generally increases exponentially with L, and we use the exponential growth rate as a measure of genotypic complexity. We find that this quantity displays several phase transitions as a function of the parameters of FGM which separate parameter regimes characterized by qualitatively different landscape structures. Depending on the regime, the genotypic landscapes induced by FGM become more or less rugged with increasing phenotypic dimension. This indicates that the role of the number of phenotypic traits in shaping the fitness landscapes of FGM is much more subtle than has been previously appreciated, and that the sweeping designation of n as (phenotypic) “complexity” can be misleading. Further implications of our study for the theory of adaptation and the interpretation of empirical data will be elaborated in the Discussion.

Model

Basic properties of FGM

In FGM, the phenotype of an organism is modeled as a set of n real-valued traits and represented by a vector $\overrightarrow{y}=\left(y_1,y_2,\dots ,y_n\right)$ in the n-dimensional Cartesian space, $\overrightarrow{y}\in {ℝ}^n.$ The fitness $W\left(\overrightarrow{y}\right)$ is assumed to be a smooth, single-peaked function of the phenotype $\overrightarrow{y}.$ By choosing an appropriate coordinate system, the optimum phenotype, i.e., the combination of phenotypic traits with the highest fitness value, can be placed at the origin in ${\mathrm{ℝ}}^n.$ We also assume that the fitness $W\left(\overrightarrow{y}\right)$ depends on the distance to the optimum $\left|\overrightarrow{y}\right|$ but not on the direction of $\overrightarrow{y},$ which can be justified by arguments based on random matrix theory (Martin 2014). The uniqueness of the phenotypic optimum at the origin implies that $W\left(\overrightarrow{y}\right)$ is a decreasing function of $\left|\overrightarrow{y}\right|.$ The form of the fitness function will be specified below when needed. Most of the results presented in this article are, however, independent of the explicit shape of $W\left(\overrightarrow{y}\right),$ as they rely solely on the relative ordering of different genotypes with respect to their fitness.

When a mutation arises the phenotype of the mutant becomes $\overrightarrow{y}+\overrightarrow{\xi },$ where $\overrightarrow{y}$ is the parental phenotype and the mutational vector $\overrightarrow{\xi }$ corresponds to the change of traits due to the mutation. The key result derived by Fisher (1930) concerns the fraction $P_b$ of beneficial mutations arising from a wild-type phenotype located at distance d from the optimum. Assuming that mutational displacements have a fixed length $\left|\overrightarrow{\xi }\right|=r$ and random directions, he showed that for $n\gg 1$

$$P_b=\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_x^{\mathrm{\infty }}{e}^{-t^2/2}\hspace{0.17em}dt}=\frac{1}{2}\text{erfc}\left(x/\sqrt{2}\right),$$ (1)

where $\text{erfc}$ denotes the complementary error function and $x=r\sqrt{n}/\left(2d\right).$ Thus, for large n the mutational step size has to be much smaller than the distance to the optimum, $r\sim d/\sqrt{n}\ll d,$ for the mutation to have a chance of increasing fitness.

As has become customary in the field, we here assume that the mutational displacements are independent and identically distributed random variables drawn from an n-dimensional Gaussian distribution with zero mean. The covariance matrix can be taken to be of diagonal form ${\sigma }^{2}I,$ where I is the n-dimensional identity matrix and ${\sigma }^2$ is the variance of a single trait (Blanquart et al. 2014). In the limit $n\to \mathrm{\infty },$ the form of the distribution of the mutational displacements becomes irrelevant owing to the central limit theorem (CLT), and therefore Fisher’s result of Equation 1 also holds in the present setting of Gaussian mutational displacements of mean size $r=\sigma \sqrt{n}$ (Waxman and Welch 2005; Ram and Hadany 2015); an explicit derivation will be provided below. Because lengths in the phenotype space can be naturally measured in units of σ, the parameters d and σ should always appear as the ratio $d/\sigma ,$ as can be seen in Equation 1. Thus, without loss of generality, we can set $\sigma =1.$ In the following we denote the (scaled) wild-type phenotype by $\overrightarrow{Q},$ its distance to the optimum by

$$Q=\left|\overrightarrow{Q}\right|=\frac{d}{\sigma },$$ (2)

and draw the displacement vectors $\overrightarrow{\xi }$ from the n-dimensional Gaussian density $p\left(\overrightarrow{\xi }\right)$ with unit covariance matrix.

By normalizing phenotypic distances to the SD σ of the mutational effect on a single trait, we are adopting a particular pleiotropic scaling that has been referred to as the “Euclidean superposition model” (Hermisson and McGregor 2008; Wagner et al. 2008). An alternative choice which is closer to Fisher’s original formulation but appears to have less empirical support is the “total effect model,” wherein the total length r of the mutational displacements is taken to be independent of n. Since $r=\sigma \sqrt{n},$ this implies that the single trait effect size decreases with n as $\sigma \sim 1/\sqrt{n}.$ As a consequence, the parameter Q defined by Equation 2 becomes n dependent and increases as $\sqrt{n},$ provided d does not depend on n (Orr 2000). The results presented below will always be given in terms of ratios of the basic parameters of FGM, such that their translation to the total effect model is in principle straightforward. We will nevertheless explicitly point out instances where the two settings give rise to qualitatively different behaviors.

The genotypic fitness landscape induced by FGM

To study epistasis within FGM, Fisher’s original definition has to be supplemented with a rule for how the effects of multiple mutations are combined. Based on earlier work (Lande 1980) in quantitative genetics, Martin et al. (2007) introduced the assumption that mutations act additively on the level of the phenotype. Thus the phenotype arising from two mutations $\overrightarrow{{\xi }_1}$ and $\overrightarrow{{\xi }_2}$ applied to the wild-type $\overrightarrow{Q}$ is simply given by $\overrightarrow{Q}+\overrightarrow{{\xi }_1}+\overrightarrow{{\xi }_2}.$ This definition suffices to associate an L-dimensional genotypic fitness landscape to any set of L mutational displacements $\overrightarrow{{\xi }_1},\overrightarrow{{\xi }_2},\dots ,\overrightarrow{{\xi }_{\mathrm{L}}}$ (Blanquart et al. 2014). For this purpose the haploid genotype τ is represented by a binary sequence with length L, $\tau =\left({\tau }_1,{\tau }_2,\dots ,{\tau }_L\right)$ with ${\tau }_i=1$ $\left({\tau }_i=0\right)$ in the presence (absence) of the ith mutation. For the wild type ${\tau }_i=0$ for all i, and in general the phenotype vector associated with the genotype τ reads

$$\overrightarrow{z}\left(\tau \right)=\overrightarrow{Q}+{\displaystyle \sum _{i=1}^L{\tau }_i\overrightarrow{{\xi }_{\mathrm{i}}}}.$$ (3)

Two examples illustrating this genotype–phenotype map and the resulting genotypic fitness landscapes with $L=3$ and $n=2$ are shown in Figure 1.

Figure 1.

Examples of three-dimensional genotypic fitness landscapes induced by FGM with two phenotypic dimensions (L = 3 and n = 2). The panels show the projection of the discrete genotype space onto the phenotype plane, where the phenotypic optimum is represented by a black ●. In the left panel, the binary sequence notation for genotypes is indicated. The wild-type genotype 000, marked by a green ▴, is located at distance Q from the phenotypic optimum. The nodes represented by red ▪’s are local fitness maxima of the genotypic landscapes, as can be seen from the contour lines of constant fitness. In the right panel, the mutant phenotypes overshoot the optimum, whereas in the left panel they do not.

As can be seen from the figure, the projection of the discrete genotype space onto the continuous phenotype space can give rise to multiple genotypic fitness maxima, although the phenotypic landscape is single peaked. It is the assumption of a finite (and hence discrete) set of phenotypic mutation vectors that distinguishes our setting from much of the earlier work on FGM, where mutations are drawn from a continuum of alleles (Fisher 1930; Orr 1998, 2000, 2005) and the probability of further improvement (as given by Equation 1) vanishes only strictly at the phenotypic optimum. Remarkably, our analysis shows that the conventional setting is not simply recovered by taking the number of mutational vectors L to infinity; rather, the number of genotypic fitness maxima is found to increase exponentially with L.

Since fitness decreases monotonically with the distance to the optimum phenotype, a natural proxy for fitness is the negative squared magnitude of the phenotype vector

$$-{\left|\overrightarrow{z}\left(\tau \right)\right|}^2=-{\left|\overrightarrow{Q}\right|}^2-2{\displaystyle \sum _{i=1}^L\left(\overrightarrow{Q}\cdot \overrightarrow{{\xi }_{\mathrm{i}}}\right){\tau }_i}-{\displaystyle \sum _{i,j=1}^L\left(\overrightarrow{{\xi }_{\mathrm{i}}}\cdot \overrightarrow{{\xi }_{\mathrm{j}}}\right){\tau }_i{\tau }_j},$$ (4)

where $\overrightarrow{x}\cdot \overrightarrow{y}$ denotes the scalar product between two vectors $\overrightarrow{x}$ and $\overrightarrow{y}.$ This quantity is thus seen to consist of a part that is additive across loci with coefficients given by the scalar products $\overrightarrow{Q}\cdot \overrightarrow{{\xi }_{\mathrm{i}}},$ and a pairwise epistatic part with coefficients ${\overrightarrow{\xi }}_i\cdot {\overrightarrow{\xi }}_j.$

It is instructive to decompose Equation 4 into contributions from the mutational displacements parallel and perpendicular to $\overrightarrow{Q}.$ Writing ${\overrightarrow{\xi }}_i={\xi }_i^{\parallel }{Q}^{-1}\overrightarrow{Q}+{\overrightarrow{\xi }}_i^{\perp }$ with $\overrightarrow{Q}\cdot {\overrightarrow{\xi }}_i^{\perp }=0,$ Equation 4 can be recast into the form

$$-{\left|\overrightarrow{z}\left(\tau \right)\right|}^2=-{\left(Q+{\displaystyle \sum _{i=1}^L{\xi }_i^{\parallel }{\tau }_i}\right)}^2-{\displaystyle \sum _{i,j=1}^L\left({\overrightarrow{\xi }}_i^{\perp }\cdot {\overrightarrow{\xi }}_j^{\perp }\right){\tau }_i{\tau }_j}.$$ (5)

The first term on the right-hand side contains both additive and epistatic contributions associated with displacements along the $\overrightarrow{Q}$ direction. The second term is dominated by the diagonal contributions with $i=j$ and is of order $L\left(n-1\right)$ because ${\left|{\overrightarrow{\xi }}_i^{\perp }\right|}^2=n-1$ on average.

We now show how the first term on the right-hand side of Equation 5 can be made to vanish for a range of Q. For this purpose, consider the subset of phenotypic displacement vectors for which the component ${\xi }_i^{\parallel }$ in the direction of $\overrightarrow{Q}$ is negative. There are on average $L/2$ such mutations, and the expected value of each component is

$$2{\displaystyle {\int }_{-\mathrm{\infty }}^{0}dy}\hspace{0.17em}\frac{y}{\sqrt{2\pi }}{e}^{-y^2/2}=-\sqrt{\frac{2}{\pi }}\equiv -2q_0,$$ (6)

where the factor 2 in front of the integral arises from conditioning on ${\xi }_i^{\parallel }<0.$ Setting ${\tau }_i=1$ for s out of these $L/2$ vectors and ${\tau }_i=0$ for all other mutations, the sum inside the brackets in Equation 5 becomes approximately equal to $-2q_{0}s,$ which cancels the Q term for $s=Q/\left(2q_0\right).$ Since s can be at most $L/2$ in a typical realization, such genotypes can be constructed with a probability approaching unity provided $Q<q_{0}L.$

We will see below that the structure of the genotypic fitness landscapes induced by FGM depends crucially on whether or not the phenotypes of multiple mutants are able to closely approach the phenotypic optimum. Assuming that the contributions from the perpendicular displacements in Equation 5 can be neglected, which will be justified shortly, the simple argument given above shows that a close approach to the optimum is facile when $Q<q_{0}L,$ but becomes unlikely when $Q\gg q_{0}L.$ This observation hints at a possible transition between different types of landscape topographies at some value of Q which is proportional to L. The existence and nature of this transition is a central theme of this article.

Scaling limits

Since we are interested in describing complex organisms with large phenotypic and genotypic dimensions, appropriate scaling relations have to be imposed to arrive at meaningful asymptotic results. Three distinct scaling limits will be considered.

1. Fisher’s classic result (Equation 1) shows that the distance of the wild type from the phenotypic optimum has to be increased with increasing n to maintain a nonzero fraction of beneficial mutations for $n\to \mathrm{\infty }.$ In our notation Fisher’s parameter is

   $$x=\frac{n}{2Q}$$ (7)

   • and hence Fisher scaling implies taking $n,Q\to \mathrm{\infty }$ at fixed ratio $n/Q.$ We will extend Fisher’s analysis by computing the probability of sign epistasis between pairs of mutations for fixed x and large n, which amounts to characterizing the shape of genotypic fitness landscapes of size $L=2.$

2. We have argued above that the distance toward the phenotypic optimum that can be covered by typical multiple mutations is of order L, and hence the limit $L\to \mathrm{\infty }$ is naturally accompanied by a limit $Q\to \mathrm{\infty }$ at fixed ratio

   $$q=\frac{Q}{L}.$$ (8)

   • From a biological point of view, one expects that $L\gg n\gg 1,$ which motivates considering the limit $L,Q\to \mathrm{\infty }$ at constant phenotypic dimension n. Under this scaling, the first term on the right-hand side of Equation 5 is of order $L^2,$ whereas the contribution from the perpendicular displacements is only $\left(n-1\right)L.$ Thus in this regime the topography of the fitness landscape is determined mainly by the one-dimensional mutational displacements in the $\overrightarrow{Q}$ direction, which is reflected by the fact that the genotypic complexity is independent of n to leading order and coincides with its value for the case $n=1,$ in which the perpendicular contribution in Equation 5 does not exist (see Results).

3. By contrast, the perpendicular displacements play an important role when both the phenotypic and genotypic dimensions are taken to infinity at fixed ratio

   $$\alpha =\frac{n}{L}.$$ (9)

   • Combining this with the limit $Q\to \mathrm{\infty }$ at fixed $q=Q/L,$ both terms on the right-hand side of Equation 5 are of the same order $\sim L^2.$ Fisher’s parameter (Equation 7) is then also a constant given by $x=\alpha /\left(2q\right).$

Preliminary considerations about genotypic fitness maxima

To set the stage for the detailed investigation of the number of genotypic fitness maxima in Results, it is useful to develop some intuition for the behavior of this quantity based on the elementary properties of FGM that have been described so far. For this purpose we consider the probability $P_{\text{wt}}$ for the wild type to be a local fitness maximum, which is equal to the probability that all the L mutations are deleterious. Since mutations are statistically independent, we have

$$P_{\text{wt}}={\left(1-P_b\right)}^L=2^{-L}{\left[1+\text{erf}\left(x/\sqrt{2}\right)\right]}^L,$$ (10)

where $\text{erf}=1-\text{erfc}$ is the error function. Under the (highly questionable) assumption that this estimate can be applied to all $2^L$ genotypes in the landscape, we arrive at the expression

$${\mathcal{N}}_{\text{wt}}=2^L{P}_{\text{wt}}={\left[1+\text{erf}\left(x/\sqrt{2}\right)\right]}^L$$ (11)

for the expected number of genotypic fitness maxima.

Consider first the scaling limit 2, where $x=n/\left(2Q\right)=n/\left(2qL\right)\to 0.$ Expanding the error function for small arguments as $\text{erf}\left(y\right)\approx 2y/\sqrt{\pi }$ we obtain

$${\mathcal{N}}_{\text{wt}}\approx {\left(1+\frac{2x}{\sqrt{2\pi }}\right)}^L\to \text{exp}\left(\frac{q_{0}n}{q}\right)$$ (12)

for $L\to \mathrm{\infty },$ where $q_0=1/\sqrt{2\pi }$ was defined in Equation 6. We will show below that this expression correctly captures the asymptotic behavior for very large q but generally grossly underestimates the number of maxima. The reason for this is that for moderate values of q (in particular for $q<q_0),$ the relevant mutant phenotypes are much closer to the origin than the wild type, which entails a mechanism for generating a large number of fitness maxima that grows exponentially with L.

Such an exponential dependence on L is expected from Equation 11 in the scaling limit 3, where $x=\alpha /\left(2q\right)$ is a nonzero constant and the expression in the square brackets is >1. Although this general prediction is confirmed by the detailed analysis for this case, the behavior of the number of maxima predicted by Equation 11 will again turn out to be valid only when q is very large. In particular, whereas Equation 11 is an increasing function of α for any q, we will see below that the expected number of maxima actually decreases with increasing phenotypic dimension (hence increasing α) in a substantial range of q. In qualitative terms, this can be attributed to the effect of the perpendicular displacements in Equation 5, which grows with α and makes it increasingly more difficult for the mutant phenotypes to closely approach the origin.

The observation that the number of genotypic fitness maxima grows exponentially with L in most cases motivates us to make use of the corresponding growth rate as a measure of the ruggedness of the landscape. We therefore define the genotypic complexity $\Sigma *$ through the limiting relation

$$\Sigma *=\underset{L\to \mathrm{\infty }}{\lim }\frac{\ln \mathcal{N}}{L},$$ (13)

where $\mathcal{N}$ is the average number of genotypic fitness maxima and L is the sequence length. Since the total number of binary genotypes is $2^L,$ the complexity is bounded from above by $\ln 2.$ If any genotype had the same probability $P_{\max }$ of being a fitness maximum (which is in fact not the case for FGM), we could write $\mathcal{N}=2^L{P}_{\max }$ and hence $P_{\max }\sim \exp \left[-\left(\ln \hspace{0.17em}2-\Sigma *\right)L\right].$

Data availability

The authors state that all data necessary for confirming the conclusions presented in the article are represented fully within the article. All numerical calculations including simulations described in this work were implemented in Mathematica and C++. When counting the number of local genotypic maxima, we checked all genotypes and counted the exact number for a randomly realized landscape, then took an average. All relevant source codes are available upon request.

Results

Preliminary note

In the following sections our results on the structure of genotypic fitness landscapes induced by FGM are stated in precise mathematical terms and the key steps of their derivation are outlined, with some technical details relegated to the appendices. To facilitate the navigation through the inevitable mathematical formalism, we display the definitions of the most commonly used mathematical symbols in Table 1. Moreover, we provide numbered summaries at the end of each subsection which state the main results without resorting to mathematical expressions.

Table 1. List of mathematical symbols.

Symbols	Description
n	Number of phenotypic traits, also referred to as phenotypic dimension
L	Length of the binary genetic sequence, also referred to as genotypic dimension
α	Ratio of phenotypic to genotypic dimension $\left(\alpha =n/L\right)$
$\overrightarrow{Q},$ Q	n-dimensional vector $\overrightarrow{Q}$ representing the wild-type phenotype and its magnitude $Q=\left|\overrightarrow{Q}\right|$
$\overrightarrow{q},$ q	Wild-type phenotype vector in units of L $\left(\overrightarrow{q}=\overrightarrow{Q}/L\right)$ and its magnitude $\left(q=\left|\overrightarrow{q}\right|\right)$
τ	Genotype represented by a binary sequence of length L
${\tau }_i$	Binary number indicating absence (0) or presence (1) of a mutation at site i $\left(i=1,\dots ,L\right)$ of the genotype τ
$\overrightarrow{z}\left(\tau \right)$, $z\left(\tau \right)$	Phenotype vector corresponding to the genotype τ (Equation 3) and its magnitude $z=\left|\overrightarrow{z}\right|$
$\overrightarrow{\xi },$ ξ	Random phenotypic displacement vector representing a mutation and its magnitude $\xi =\left|\overrightarrow{\xi }\right|$
$p\left(\overrightarrow{\xi }\right)$	Probability density of the random vector $\overrightarrow{\xi }.$ In this article, the density is Gaussian with unit covariance matrix
x	Fisher’s scaling parameter; in our notation $x=n/\left(2Q\right)$
$\mathcal{N}$	Total number of fitness maxima in a genotypic fitness landscape averaged over all realizations of sets of $\overrightarrow{\xi }$’s
$\Sigma \text{*}$	Genotypic complexity defined as the ratio of $\ln \hspace{0.17em}\mathcal{N}$ to L for $L\to \mathrm{\infty }$; see Equation 13
$q_c$	Transition point of q that separates regimes I and II
$q_0$	Half of the average mutational displacement ${\xi }_i$ of a single trait conditioned on being positive $\left(q_0=1/\sqrt{2\pi }\right)$
ρ	Fraction of mutations that are present in a genotype τ, $\rho =L^{-1}{\displaystyle {\sum }_{i=1}^L{\tau }_i}$
$\rho *$	Mean value of ρ of a local maximum, also referred to as mean genotypic distance from the wild type
$z*$	Mean value of $z\left(\tau \right)$ of a local maximum, also referred to as mean phenotypic distance from the optimum

Sign epistasis

Random mutations:

We first study the local topography of the fitness landscape around the wild type, focusing on the epistasis between two random mutations with phenotypic displacements $\overrightarrow{\xi }$ and $\overrightarrow{\eta }.$ Since fitness is determined by the magnitude of a phenotypic vector, i.e., the distance of the phenotype from the origin, the epistatic effect of the two mutations can be understood by analyzing how the magnitudes of the four vectors $\overrightarrow{Q},$ $\overrightarrow{Q}+\overrightarrow{\xi },$ $\overrightarrow{Q}+\overrightarrow{\eta },$ and $\overrightarrow{Q}+\overrightarrow{\xi }+\overrightarrow{\eta }$ are ordered. To this end, we introduce the quantities

$$\begin{array}{c}{R}_1\equiv \frac{1}{n}\left({\left|\overrightarrow{\xi }+\overrightarrow{Q}\right|}^2-Q^2\right),\hspace{1em}{R}_2\equiv \frac{1}{n}\left({\left|\overrightarrow{\eta }+\overrightarrow{Q}\right|}^2-Q^2\right),\hspace{1em}\text{and}\\ R\equiv \frac{1}{n}\left({\left|\overrightarrow{\xi }+\overrightarrow{\eta }+\overrightarrow{Q}\right|}^2-Q^2\right),\end{array}$$ (14)

where division by n guarantees the existence of a finite limit for $n\to \mathrm{\infty }.$ The sign of these quantities determines whether a mutation is beneficial or deleterious. For example, if $R_1<0,$ the mutation $\overrightarrow{\xi }$ is beneficial; if $R>0,$ the two mutations combined together confer a deleterious effect; and so on. We will see later that $R_{1,2}$ and R are actually closely related to the selection coefficients of the respective mutations.

We proceed to express the different types of pairwise epistasis defined by Weinreich et al. (2005) and Poelwijk et al. (2007) in terms of conditions on the quantities defined in Equation 14. Without loss of generality we assume $R_1<R_2$ and consider first the case where both mutations are beneficial, $R_1<R_2<0.$ Then magnitude epistasis (ME), the absence of sign epistasis, applies when the fitness of the double mutant is higher than that of each of the single mutants, i.e., $R<R_1<R_2<0.$ Similarly, for two deleterious mutations the condition for ME reads $R>R_2>R_1>0.$ When one mutant is deleterious and the other beneficial, in the case of ME, the double mutant fitness has to be intermediate between the two single mutants, which implies that $R_1<R<R_2$ when $R_2>0>R_1.$

The condition for RSE reads $R>R_2>R_1$ when both single mutants are beneficial and $R<R_1<R_2$ when both are deleterious, and the remaining possibility $R_1<R<R_2$ corresponds to SSE between two mutations of the same sign. If the two single mutant effects are of different signs, RSE is impossible and SSE applies when $R<R_1<0<R_2$ or $R>R_2>0>R_1.$ Figure 2 depicts the different categories of epistasis as regions in the $\left(R_2,R\right)$ plane. Note that the corresponding picture for $R_1>R_2$ is obtained by exchanging $R_1\leftrightarrow R_2.$

Figure 2.

Domains in the $\left(R_2,R\right)$ plane contributing to different types of epistasis: ME, SSE, and RSE. The two panels illustrate the two cases: (A) $R_1>0$ and (B) $R_1<0.$ The red solid lines indicate $R=R_1+R_2.$ The labeling of the domains $D_1,\dots ,D_6$ is used in the derivation in Appendix B.

To find the probability of each epistasis, we require the joint probability density $\mathcal{P}\left(R_1,R_2,R\right).$ In Appendix A it is shown that

$$\mathcal{P}\left(R_1,R_2,R\right)=\frac{x^2{n}^{1/2}}{4\sqrt{2}{\pi }^{3/2}}{e}^{-\frac{1}{8}n{\left(-R+R_1+R_2\right)}^2-\frac{x^2}{2}\left({\left(R_1-1\right)}^2+{\left(R_2-1\right)}^2\right)}\times \left[1+O\left(\frac{1}{n}\right)\right],$$ (15)

which can be obtained rather easily by resorting to the CLT. The applicability of the CLT follows from the fact that $R_{1,2}$ and R are sums of a large number of independent terms for $n\to \mathrm{\infty }$ (Waxman and Welch 2005; Ram and Hadany 2015). According to the CLT, it is sufficient to determine the first and second cumulants of these quantities. Denoting averages by angular brackets, we find the mean $\langle R_i\rangle =1,$ the variance $\langle R_i^2\rangle -{\langle R_i\rangle }^2=1/x^2,$ and the covariance $\langle R_1{R}_2\rangle -\langle R_1\rangle \langle R_2\rangle =0$ $\left(i=1,2\right).$ Similarly, the corresponding quantities evaluated for $R-R_1-R_2$ are $\langle R-R_1-R_2\rangle =0,$ $\langle {\left(R-R_1-R_2\right)}^2\rangle -{\langle R-R_1-R_2\rangle }^2=4/n,$ and $\langle \left(R-R_1-R_2\right)R_i\rangle -\langle R-R_1-R_2\rangle \langle R_i\rangle =0$ $\left(i=1,2\right).$ With an appropriate normalization constant, this leads directly to Equation 15.

As a first application, we rederive Fisher’s Equation 1 by integrating $\mathcal{P}\left(R_1,R_2,R\right)$ over the region $R_1<0$ for all $R_2$ and R, which indeed yields

$$P_b={\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}d{R}_2}{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}dR\hspace{0.17em}\mathcal{P}\left(R_1,R_2,R\right)}=\frac{1}{2}\text{erfc}\left(\frac{x}{\sqrt{2}}\right).$$

An immediate conclusion from the form of $\mathcal{P}\left(R_1,R_2,R\right)$ is that it is unlikely to observe sign epistasis for large n, because $\mathcal{P}\left(R_1,R_2,R\right)$ becomes concentrated along the line $R=R_1+R_2$ as n increases. As can be seen in Figure 2, this line touches the region of SSE in one point for $R_1<0,$ whereas it maintains a finite distance to the region of RSE everywhere. This indicates that the probability of RSE decays more rapidly with increasing n than the probability of SSE. Moreover, one expects the latter probability to be proportional to the width of the region around the line $R=R_1+R_2,$ where the joint probability in Equation 15 has appreciable weight, which is of order $1/\sqrt{n}.$

To be more quantitative, we need to integrate $\mathcal{P}\left(R_1,R_2,R\right)$ over the domains in Figure 2 corresponding to the different categories of epistasis. In Appendix B, we obtain the asymptotic expressions

$$P_{\text{RSE}}=\frac{2x^2}{\pi n}{e}^{-x^2}+O\left(n^{-3/2}\right)$$ (16)

and

$$P_{\text{SSE}}=\frac{4x}{\pi \sqrt{n}}\hspace{0.17em}{e}^{-x^2/2}+O\left(n^{-1}\right)$$ (17)

for the probabilities of RSE $\left(P_{\text{RSE}}\right)$ and SSE $\left(P_{\text{SSE}}\right).$ Due to the nonlinearity of the phenotype-fitness map, FGM does not allow for strictly nonepistatic combination of fitness effects. The probability of ME, therefore, is given by $P_{\text{ME}}=1-P_{\text{RSE}}-P_{\text{SSE}}.$ Interestingly, the probability of sign epistasis varies nonmonotonically with x. To confirm our analytic results, we compare our results with simulations in Figure 3, which shows an excellent agreement.

Figure 3.

Comparison of analytic results for the probability of epistasis with simulations. Depicted are probabilities of SSE (P_SSE) and RSE (P_RSE) between two randomly chosen mutations among nearest neighbor genotypes of the wild type (A) as functions of n for fixed Fisher parameter $x=0.5$ and (B) as functions of x for fixed phenotypic dimension $n=640.$ For each parameter set, ${10}^4$ randomly generated landscapes were analyzed. The asymptotic expressions provide accurate approximations even for moderate $n>10.$ The nonmonotonic behavior with respect to x means that the probabilities are nonmonotonic functions of Q for fixed n and vice versa.

Similarly, we can calculate the probabilities of sign epistasis conditioned on both mutations being beneficial, which in our setting means $R_2<0.$ The conditioning requires normalization by the unconditional probability of two random mutations being beneficial, which is given by the square of $P_b$ in Equation 1. Hence

$$P_{\text{RSE}}^b=\frac{2\text{Pr}\left(D_1\right)}{P_b^2}\approx \frac{4x^2}{\pi n\text{erfc}{\left(x/\sqrt{2}\right)}^2}{e}^{-x^2}$$ (18)

and

$$P_{\text{SSE}}^b=\frac{2\text{Pr}\left(D_5\right)}{P_b^2}\approx \frac{4x}{\pi \sqrt{n}\text{erfc}\left(x/\sqrt{2}\right)}{e}^{-x^2/2},$$ (19)

where $\mathrm{Pr}\left(D_i\right)$ denotes the integral of the joint probability density over the domain $D_i$ in Figure 2 (see Appendix B).

As anticipated from the form of Equation 15, the fraction of sign-epistatic pairs of mutations decreases with increasing phenotypic dimension n, and this decay is faster for RSE $\left(\sim 1/n\right)$ than for SSE $\left(\sim 1/\sqrt{n}\right).$ At first glance this might seem to suggest that FGM has little potential for generating rugged genotypic fitness landscapes. However, as we will see below, the results obtained in this section apply only to the immediate neighborhood of the wild-type phenotype. They are modified qualitatively in the presence of a large number of mutations that are able to substantially displace the phenotype and allow it to approach the phenotypic optimum.

Mutations of fixed effect size:

As a slight variation to the previous setting, one may consider the fraction of sign epistasis conditioned on the two single mutations to have the same selection strength, as recently investigated by Schoustra et al. (2016). In our notation this implies that $R_1=R_2\equiv \stackrel{\sim }{R},$ and it is easy to see that sign epistasis is always reciprocal in this case. If the two mutations are beneficial, $\stackrel{\sim }{R}<0,$ and the condition for (reciprocal) sign epistasis is $R>\stackrel{\sim }{R}.$ The corresponding probability is

$${\stackrel{\sim }{P}}_{\text{RSE}}\left(\stackrel{\sim }{R}\right)=\frac{{\displaystyle {\int }_{\stackrel{\sim }{R}}^{\mathrm{\infty }}\mathcal{P}\left(\stackrel{\sim }{R},\stackrel{\sim }{R},R\right)dR}}{{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\mathcal{P}\left(\stackrel{\sim }{R},\stackrel{\sim }{R},R\right)dR}}=\frac{1}{2}\text{erfc}\left(-\frac{\sqrt{n}\stackrel{\sim }{R}}{2\sqrt{2}}\right).$$ (20)

Following the same procedure for deleterious mutations $\left(\stackrel{\sim }{R}>0\right)$ one finds that the probability is actually symmetric around $\stackrel{\sim }{R}=0$ and hence depends only on $\left|\stackrel{\sim }{R}\right|.$

To express ${\stackrel{\sim }{P}}_{\text{RSE}}$ in terms of the selection coefficient of the single mutations, we introduce a Gaussian phenotypic fitness function of the form

$$W\left(\overrightarrow{y}\right)=W_0\exp \left(-\lambda {\left|\overrightarrow{y}\right|}^2\right),$$ (21)

where $\lambda >0$ is a measure for the strength of selection. The selection coefficient of a mutation with phenotypic effect $\overrightarrow{\xi }$ is then given by

$$S=\ln \left[\frac{W\left(\overrightarrow{Q}+\overrightarrow{\xi }\right)}{W\left(\overrightarrow{Q}\right)}\right]=-\lambda \left({\left|\overrightarrow{Q}+\overrightarrow{\xi }\right|}^2-{\left|\overrightarrow{Q}\right|}^2\right)=-\lambda n\stackrel{\sim }{R}.$$ (22)

To fix the value of λ we note that the largest possible selection coefficient, which is achieved for mutations that reach the phenotypic optimum, is $S_0=\lambda Q^2,$ and hence $\stackrel{\sim }{R}$ is related to the selection coefficient through $\stackrel{\sim }{R}=-\left(Q^2/n\right)\left(S/S_0\right).$ With this substitution, the result in Equation 20 becomes

$${\stackrel{\sim }{P}}_{\mathrm{RSE}}\left(S\right)=\frac{1}{2}\mathrm{erfc}\left(\frac{n^{3/2}}{8\sqrt{2}{x}^2}\frac{\left|S\right|}{S_0}\right).$$ (23)

The probability of sign epistasis conditioned on selection strength takes on its maximal value ${\stackrel{\sim }{P}}_{\text{RSE}}=1/2$ in the neutral limit $S\to 0$ and decreases monotonically with $\left|S\right|.$ Similar to the results of Equations 16, 17, and 18 for unconstrained mutations, it also decreases with increasing phenotypic dimension n when S and x are kept fixed.

In a previous numerical study carried out at finite Q and n, it was found that ${\stackrel{\sim }{P}}_{\text{RSE}}$ varies nonmonotonically with S for the case of beneficial mutations, and displays a second peak at the maximum selection coefficient $S=S_0$ (Schoustra et al. 2016). The two peaks were argued to reflect the two distinct mechanisms giving rise to sign epistasis within FGM (Blanquart et al. 2014). Mutations of small effect correspond to phenotypic displacements that proceed almost perpendicularly to the direction of the phenotypic optimum, and sign epistasis is generated through antagonistic pleiotropy. On the other hand, for mutations of large effect, the dominant mechanism for sign epistasis is through overshooting of the phenotypic optimum. Because of the Fisher scaling implemented in this section with $Q,n\to \infty$ at fixed $x=n/\left(2Q\right),$ the second class of mutations cannot be captured by our approach and only the peak at small S remains. Figure 4A shows the full two-peak structure for a few representative values of n, and Figure 4B illustrates the convergence to the asymptotic expression Equation 23 for the left peak. Using the results of Schoustra et al. (2016), it can be shown that the right peak becomes a step function for $n\to \mathrm{\infty },$ displaying a discontinuous jump from ${\stackrel{\sim }{P}}_{\text{RSE}}=0$ to ${\stackrel{\sim }{P}}_{\text{RSE}}=1$ at $S/S_0=8/9=0.888\dots$

Figure 4.

Probability of RSE ${\stackrel{\sim }{P}}_{\text{RSE}}$ conditioned on the selection coefficients S of the two single mutations to be equal and positive: (A) for the full range of S on a linear scale and (B) for S/S₀ smaller than 0.2 on a semilogarithmic scale. Here, the fitness of a phenotype $\overrightarrow{y}$ is assumed to be $W\left(\overrightarrow{y}\right)=W_0\exp \left(-\lambda {\left|\overrightarrow{y}\right|}^2\right),$ where the parameter λ is related to the maximal beneficial selection coefficient $S_0$ through the relation $S_0=\lambda Q^2.$ Dashed lines depict the asymptotic expression Equation 23, and solid lines were obtained numerically using the Gaussian approximation for the distribution of epistasis developed by Schoustra et al. (2016).

Summary 1:

When the phenotypic dimension n is large and the Fisher parameter x is moderate, the probability of RSE decays as $1/n,$ while that of SSE decays as $1/\sqrt{n}.$ Although these probabilities decrease monotonically with n at fixed x, they have a nonmonotonic behavior as a function of x: For small x they increase with x and for large x they decrease with x (see Figure 3). Under the pleiotropic scaling adopted in this work, this implies that the probabilities are nonmonotonic functions of the wild-type distance Q at fixed n and vice versa. In contrast, under the total effect model, where both the wild-type distance Q and x scale as $\sqrt{n},$ the probabilities decrease monotonically and exponentially with n.

Genotypic complexity at a fixed phenotypic dimension

In this section, we are interested in the number of local maxima in the genotypic fitness landscape. We focus on the expected number of maxima, which we denote by $\mathcal{N},$ and analyze how this quantity behaves in the limit of large genotypic dimension, $L\to \mathrm{\infty },$ when the phenotypic dimension n is fixed. For the sake of clarity, the (unique) maximum of the phenotypic fitness landscape will be referred to as the phenotypic optimum throughout.

The number of local fitness maxima:

Since fitness decreases monotonically with the distance to the phenotypic optimum, a genotype τ is a local fitness maximum if the corresponding phenotype defined by Equation 3 satisfies

$$\left|\overrightarrow{z}\left(\tau \right)\right|<\left|\overrightarrow{z}\left(\tau \right)+\left(1-2{\tau }_i\right){\overrightarrow{\xi }}_i\right|$$ (24)

for all $1\le i\le L.$ The phenotype vector appearing on the right-hand side of this inequality arises from $\overrightarrow{z}\left(\tau \right),$ either by removing a mutation vector that is already part of the sum in Equation 3 $\left({\tau }_i=1\right)$ or by adding a mutation vector that was not previously present $\left({\tau }_i=0\right).$ The condition in Equation 24 is obviously always fulfilled if $\overrightarrow{z}\left(\tau \right)=0,$ that is, if the phenotype is optimal, and we will see that in general the probability for this condition to be satisfied is larger the more closely the phenotype approaches the origin. A graphical illustration of the condition in Equation 24 is shown in Figure 5.

Figure 5.

Illustration of the condition for a genotype to be a local fitness maximum. The circle encloses phenotypes that have higher fitness than the focal phenotype $\overrightarrow{z}\left(\tau \right).$ For τ to be a genotypic fitness maximum, both a phenotype with a further mutation (dash-dotted green arrow) and a phenotype without one of the mutations in τ (red segment and blue dotted arrows) should lie outside the circle.

The ability of a phenotype $\overrightarrow{z}\left(\tau \right)$ to approach the origin clearly depends on the number $s={\displaystyle {\sum }_{i=1}^L{\tau }_i}$ of mutant vectors it is composed of, and all phenotypes with the same number of mutations are statistically equivalent. The expected number of fitness maxima can therefore be decomposed as

$$\mathcal{N}={\displaystyle \sum _{s=0}^L\left(\begin{array}{c}L\\ s\end{array}\right){\mathrm{ℛ}}_s\left(L\right)},$$ (25)

where $\left(\begin{array}{c}L\\ s\end{array}\right)$ is the number of possible combinations of s out of L mutation vectors and ${\mathrm{ℛ}}_s\left(L\right)$ is the probability that a genotype with s mutations is a fitness maximum. The latter can be written as

$${\mathrm{ℛ}}_s\left(L\right)={\displaystyle {\int }_{n}d\overrightarrow{z}}\left[{\displaystyle \prod _{i=s+1}^L{\displaystyle {\int }_{\mathcal{D}\left(-\overrightarrow{z}\right)}d\overrightarrow{{\xi }_{\mathrm{i}}}p\left(\overrightarrow{{\xi }_{\mathrm{i}}}\right)}}\right]\times \left[{\displaystyle \prod _{i=1}^s{\displaystyle {\int }_{\mathcal{D}\left(\overrightarrow{z}\right)}d\overrightarrow{{\xi }_{\mathrm{i}}}p\left(\overrightarrow{{\xi }_{\mathrm{i}}}\right)}}\right]\delta \left(\overrightarrow{z}-\overrightarrow{Q}-{\displaystyle \sum _{i=1}^s\overrightarrow{{\xi }_{\mathrm{i}}}}\right),$$ (26)

with

$$\mathcal{D}\left(\overrightarrow{y}\right)\equiv \left\{\overrightarrow{\xi }\in {\mathrm{ℝ}}^n|\left|\overrightarrow{\xi }-\overrightarrow{y}\right|>\left|\overrightarrow{y}\right|\right\}.$$ (27)

Here and below, ${\int }_n$ stands for the integral over ${\mathbf{ℝ}}^n.$

Equation 26 can be understood as follows. First, the δ function $\delta \left(\overrightarrow{z}-\overrightarrow{Q}-{\displaystyle {\sum }_{i=1}^s\overrightarrow{{\xi }_{\mathrm{i}}}}\right)$ constrains $\overrightarrow{z}$ to be the phenotype of τ as defined in Equation 3. Next, the integration domains of the $\overrightarrow{{\xi }_{\mathrm{i}}}’\text{s}$ reflect the condition in Equation 24. Assuming, without loss of generality, that the L genetic loci are ordered such that ${\tau }_i=1$ for $i\le s$ and ${\tau }_i=0$ for $i>s,$ the maximum condition for $i\le s$ requires $\left|\overrightarrow{z}\right|<\left|\overrightarrow{z}-{\overrightarrow{\xi }}_i\right|,$ so the integration domain should be $\mathcal{D}\left(\overrightarrow{z}\right);$ whereas for $i>s$ the condition is $\left|\overrightarrow{z}\right|<\left|\overrightarrow{z}+\overrightarrow{{\xi }_{\mathrm{i}}}\right|,$ corresponding to the integration domain $\mathcal{D}\left(-\overrightarrow{z}\right).$ Using the integral representation of the δ function,

$$\delta \left(\overrightarrow{y}\right)=\frac{1}{{\left(2\pi \right)}^n}{\displaystyle {\int }_{n}d\overrightarrow{k}}\exp \left(i\overrightarrow{k}\cdot \overrightarrow{y}\right),$$ (28)

we can write

$${\mathrm{ℛ}}_s\left(L\right)={\displaystyle {\int }_n{\displaystyle {\int }_n\frac{d\overrightarrow{z}d\overrightarrow{k}}{{\left(2\pi \right)}^n}}}\exp \left[i\overrightarrow{k}\cdot \left(\overrightarrow{z}-\overrightarrow{Q}\right)\right]F{\left(\overrightarrow{k},\overrightarrow{z}\right)}^s\times F{\left(0,-\overrightarrow{z}\right)}^{L-s},$$ (29)

where

$$F\left(\overrightarrow{k},\overrightarrow{z}\right)\equiv {\displaystyle {\int }_{\mathcal{D}\left(\overrightarrow{z}\right)}d\overrightarrow{\xi }}p\left(\overrightarrow{\xi }\right)\exp \left(-i\overrightarrow{k}\cdot \overrightarrow{\xi }\right).$$ (30)

It was argued on qualitative grounds in Model that phenotypes that approach arbitrarily close to the origin are easily generated when the scaled wild-type distance q is small, but they become rare for large q. As a consequence, it turns out that the main contribution to the integral over $\overrightarrow{z}$ in Equation 29 comes from the region around the origin $\overrightarrow{z}=0$ for small q, but shifts to a distance $z\sim L$ along the $\overrightarrow{Q}$ direction for large q. To account for this possibility, it is necessary to divide the integral domain into two parts, $\left|\overrightarrow{z}\right|<z_0$ and $\left|\overrightarrow{z}\right|>z_0,$ where $z_0$ is an arbitrary nonzero number with $z_0/L\to 0$ as $L\to \mathrm{\infty }.$ Thus, we write ${\mathrm{ℛ}}_s\left(L\right)$ as

$${\mathrm{ℛ}}_s\left(L\right)={\mathrm{ℛ}}_s^{<}\left(L\right)+{\mathrm{ℛ}}_s^{>}\left(L\right),$$ (31)

where

$$\begin{array}{l}{\mathrm{ℛ}}_s^{<}\left(L\right)={\displaystyle {\int }_{\left|\overrightarrow{z}\right|<z_0}d\overrightarrow{z}}{\displaystyle {\int }_n\frac{d\overrightarrow{k}}{{\left(2\pi \right)}^n}}{e}^{i\overrightarrow{k}\cdot \left(\overrightarrow{z}-\overrightarrow{Q}\right)}F{\left(\overrightarrow{k},\overrightarrow{z}\right)}^{s}F{\left(0,-\overrightarrow{z}\right)}^{L-s},\\ {\mathrm{ℛ}}_s^{>}\left(L\right)={\displaystyle {\int }_{\left|\overrightarrow{z}\right|>z_0}d\overrightarrow{z}}{\displaystyle {\int }_n\frac{d\overrightarrow{k}}{{\left(2\pi \right)}^n}{e}^{i\overrightarrow{k}\cdot \left(\overrightarrow{z}-\overrightarrow{Q}\right)}F{\left(\overrightarrow{k},\overrightarrow{z}\right)}^{s}F{\left(0,-\overrightarrow{z}\right)}^{L-s}},\end{array}$$ (32)

and correspondingly define ${\mathcal{N}}_{<}$ and ${\mathcal{N}}_{>}$ as

$${\mathcal{N}}_{<}={\displaystyle \sum _s\left(\begin{array}{c}L\\ s\end{array}\right){\mathrm{ℛ}}_s^{<}\left(L\right)}\hspace{1em}\text{and}\hspace{1em}{\mathcal{N}}_{>}={\displaystyle \sum _s\left(\begin{array}{c}L\\ s\end{array}\right){\mathrm{ℛ}}_s^{>}\left(L\right)}.$$ (33)

The total number of local maxima is then $\mathcal{N}={\mathcal{N}}_{<}+{\mathcal{N}}_{>}.$

Regime I:

We first consider ${\mathrm{ℛ}}_s^{<}\left(L\right).$ Expanding $F\left(\overrightarrow{k},\overrightarrow{z}\right)$ around the origin $\overrightarrow{z}=0,$ we show in Appendix C that

$${\mathrm{ℛ}}_s^{<}\left(L\right)\approx \frac{s^{-n/2}\exp \left[-Q^2/\left(2s\right)\right]}{s\exp \left[-Q^2/\left(2s^2\right)\right]+L-s}.$$ (34)

For an interpretation of Equation 34 it is helpful to refer to Figure 5. Note first that the probability that $\overrightarrow{z}=\overrightarrow{Q}+{\displaystyle {\sum }_{i=1}^s\overrightarrow{{\xi }_{\mathrm{i}}}}$ lies in the ball $\left|\overrightarrow{z}\right|<\zeta$ with radius $\zeta \ll 1$ is

$$\mathrm{Prob}\left(\left|\overrightarrow{z}\right|<\zeta \right)\approx \frac{V_n}{{\left(2\pi \right)}^{n/2}}{s}^{-n/2}\exp \left[-Q^2/\left(2s\right)\right],$$ (35)

where $V_n\left(\zeta \right)\sim {\zeta }^n$ is the volume of the ball. We need to estimate how small ζ has to be for τ to be a local fitness maximum with an appreciable probability. Since the s random vectors contributing to $\overrightarrow{z}$ are statistically equivalent, it is plausible to assume that their average component parallel to $\overrightarrow{Q}$ is ${\xi }_i^{\parallel }\approx -Q/s.$ We further assume that the conditional probability density ${\stackrel{\sim }{p}}_s\left(\overrightarrow{\xi }\right)$ of these vectors, conditioned on their sum $\overrightarrow{z}$ reaching the ball around the origin, can be approximated by a Gaussian, which consequently has the form

$${\stackrel{\sim }{p}}_s\left(\overrightarrow{\xi }\right)\approx \frac{1}{{\left(2\pi \right)}^{n/2}}\exp \left(-\frac{1}{2}{\left|\overrightarrow{\xi }+\frac{\overrightarrow{Q}}{s}\right|}^2\right).$$ (36)

For $\overrightarrow{z}$ to be a phenotype vector of a local maximum, all these random vectors should lie in the region $\mathcal{D}\left(\overrightarrow{z}\right)$ and the remaining (unconstrained) $L-s$ vectors should lie in $\mathcal{D}\left(-\overrightarrow{z}\right).$ This event happens with probability Thus, we can estimate the typical value of ζ as the solution of

$$\frac{V_n\left(\zeta \right)}{{\left(2\pi \right)}^{n/2}}\approx {\left\{s\hspace{0.17em}\exp \left[-Q^2/\left(2s^2\right)\right]+L-s\right\}}^{-1},$$ (38)

which, combined with Equation 35, indeed gives Equation 34.

To find the asymptotic behavior of ${\mathcal{N}}_{<}$ for large L, we use Stirling’s formula in Equation 33 and approximate the summation over s by an integral over $\rho \equiv s/L.$ This yields

$${\mathcal{N}}_{<}\approx {\displaystyle {\int }_0^{1}d\rho }\frac{1}{L^{n/2}{\rho }^{n/2}}\frac{e^{L\Sigma \left(\rho \right)}}{\sqrt{2\pi L\rho \left(1-\rho \right)}}\frac{1}{1-\rho +\rho e^{-\frac{q^2}{2{\rho }^2}}},$$ (39)

where the exponent $\Sigma \left(\rho \right)$ is given by

$$\Sigma \left(\rho \right)\equiv -\rho \ln \rho -\left(1-\rho \right)\ln \left(1-\rho \right)-\frac{q^2}{2\rho }.$$ (40)

Under the condition $L\gg 1,$ the remaining integral with respect to ρ can be performed by expanding $\Sigma \left(\rho \right)$ to the second order around the saddle point $\rho *$ determined by the condition

$$0={\frac{\partial }{\partial \rho }\Sigma \left(\rho \right)|}_{\rho =\rho *}=\frac{q^2}{2{\left(\rho *\right)}^2}-\ln \frac{\rho *}{1-\rho *}.$$ (41)

Performing the resulting Gaussian integral with respect to ρ one finally obtains

$${\mathcal{N}}_{<}\approx \frac{1}{L^{1+n/2}}\sqrt{\frac{1}{1+\left(1-\rho *\right){\left(q/\rho *\right)}^2}}\frac{{\left(\rho *\right)}^{-n/2}{e}^{L\Sigma \left(\rho *\right)}}{1-\rho *+\rho *e^{-\frac{q^2}{2{\left(\rho *\right)}^2}}},$$ (42)

where $\rho *=\rho *\left(q\right)$ is the solution of Equation 41, which is the (scaled) mean number of mutations in a local maximum. We will call $\rho *$ the mean genotypic distance. This solution is not available in closed form, but it can be shown that $\rho *=\left(1/2\right)+\left(q^2/2\right)+O\left(q^4\right)$ and $\Sigma \left(\rho *\right)=\ln 2-q^2+O\left(q^4\right)$ for small q. Figure 6 compares Equation 42 with the mean number of local maxima obtained by numerical simulations for various q’s with $n=1,$ to show an excellent agreement even for $L=10.$

Figure 6.

Plots of mean number of local maxima $\mathcal{N}$ as a function of the genotypic dimension L for $q=0,$ 0.2, 0.4, and 0.6 with $n=1$ on a semilogarithmic scale. Data from numerical simulations are represented as dots, and the analytical prediction of Equation 42 is shown as solid lines. Each dot represents the average over ${10}^5$ realizations of landscapes. In this parameter regime, $\mathcal{N}$ grows exponentially with L and the growth rate (i.e., the slopes of the lines) decreases with increasing q.

$$\begin{array}{c}{\left[{\displaystyle {\int }_{\mathcal{D}\left(\overrightarrow{z}\right)}d\overrightarrow{\xi }{\stackrel{\sim }{p}}_s\left(\overrightarrow{\xi }\right)}\right]}^s{\left[{\displaystyle {\int }_{\mathcal{D}\left(-\overrightarrow{z}\right)}d\overrightarrow{\xi }p\left(\overrightarrow{\xi }\right)}\right]}^{L-s}\approx {\left\{1-\frac{V_n}{{\left(2\pi \right)}^{n/2}}\exp \left[-Q^2/\left(2s^2\right)\right]\right\}}^s{\left[1-\frac{V_n}{{\left(2\pi \right)}^{n/2}}\right]}^{L-s}\\ \approx \exp \left[-\frac{V_n}{{\left(2\pi \right)}^{n/2}}\left\{s\hspace{0.17em}\exp \left[-Q^2/\left(2s^2\right)\right]+L-s\right\}\right].\end{array}$$ (37)

It is obvious that $\Sigma \left(\rho \right)$ will eventually be negative as q increases for any value of ρ, and this must be true also for the maximum value $\Sigma \left(\rho *\right).$ Indeed, we found the threshold $q_c\approx 0.924\hspace{0.17em}809,$ above which $\Sigma \left(\rho *\right)$ is negative. This signals a phase transition in the landscape properties. Inspection of Equation 40 shows that the transition is driven by a competition between the abundance of genotypes with a certain number of mutations and their likelihood to bring the phenotype close to the optimum. The first two terms in the expression for $\Sigma \left(\rho \right)$ are the standard sequence entropy (see, for example, Schmitt and Herzel 1997) which is maximal at $\rho =1/2$ $\left(s=L/2\right),$ whereas the last term represents the statistical cost associated with “stretching” the phenotype toward to origin. With increasing q, the genotypes contributing to the formation of local maxima become increasingly atypical, in the sense that they contain more than the typical fraction $\rho =1/2$ of mutations, and $\rho *$ increases. For $q>q_c,$ the cost can no longer be compensated by the entropy term and $\Sigma \left(\rho *\right)$ becomes negative. In this regime ${\mathcal{N}}_{<}$ decreases exponentially with L, and therefore the total number of fitness maxima $\mathcal{N}$ which by construction cannot be <1, must be dominated by the second contribution ${\mathcal{N}}_{>}.$

Regime II:

We defer the detailed derivation of ${\mathcal{N}}_{>}$ to Appendix C and here only report the final result obtained in the limit $L\to \mathrm{\infty },$ which is independent of L and reads

$${\mathcal{N}}_{>}\approx {\left[\frac{q-q_0}{q}\exp \left(\frac{1}{q/q_0-1}\right)\right]}^{n-1}.$$ (43)

This expression is valid for $q>q_0=1/\sqrt{2\pi }\approx 0.399,$ but it dominates the contribution ${\mathcal{N}}_{<}$ for large L only when $q>q_c.$ Figure 7 indeed shows that Equation 43 approximates the mean number of local maxima for $q>q_c,$ that is, $\mathcal{N}$ converges to ${\mathcal{N}}_{>}$ for large L. This figure also shows, as is clear by Equation 43, that $\mathcal{N}$ is an increasing (decreasing) function of n (q) for a fixed value of q (n). The expected number of maxima is small in absolute terms in this regime, which can be attributed to the fact that the expression inside the parentheses in Equation 43 takes the value $1.214\dots$ at $q=q_c,$ and decreases rapidly toward unity for larger q.

Figure 7.

Comparison of simulation results (symbols) of the mean number of local maxima $\mathcal{N}$with analytic approximations (lines) for $q>q_c.$ Each symbol is the result of averaging over $2\times {10}^6$ realizations. (A) $\mathcal{N}$is shown to increase with n for fixed q. (B) $\mathcal{N}$is shown to decrease with q for fixed n. (C) Deviation of the analytic expression from the simulation results, defined as $1-\left({\mathcal{N}}_{\text{data}}/{\mathcal{N}}_{\text{theory}}\right),$ is depicted as a function of L on a double logarithmic scale. The phenotypic dimension for this panel is $n=4,$ where the largest deviations are observed in (A). The deviation decreases inversely with L as indicated by the black dashed line with slope $-1.$

To understand the appearance of $q_0,$ we refer to Model, where it was argued that $2q_{0}s$ is the maximal distance toward the origin, which can be covered by a phenotype made up of s typical mutation vectors. Correspondingly, the analysis in Appendix C shows that the main contribution to ${\mathrm{ℛ}}_s^{>}\left(L\right)$ comes from phenotypes located at a distance $z=2s\left(q-q_0\right)$ from the origin, i.e., at a distance $2s{q}_0$ from the wild type. The sum over s in Equation 33 is dominated by typical genotypes with $s=L/2,$ and therefore the main contribution to ${\mathcal{N}}_{>}$ comes from phenotypes at a distance $z=\left(q-q_0\right)L$ from the origin. The seeming divergence of ${\mathcal{N}}_{>}$ as $q\to q_0^{+}$ is an artifact of the approximation scheme, which assumes that the main contribution comes from the region where $z\sim O\left(L\right);$ clearly this assumption becomes invalid when $q\to q_0^{+}.$ We note that for very large q and large n, Equation 43 reduces to the expression ${\mathcal{N}}_{\text{wt}}$ obtained in Equation 11 on the basis of Fisher’s formula for the fraction of beneficial mutations from the wild-type phenotype.

Phase transition:

To sum up, the leading behavior of $\mathcal{N}$ is

$$\mathcal{N}=\{\begin{array}{cc}{\mathcal{N}}_{<},& q<q_c,\\ {\mathcal{N}}_{>},& q\ge q_c,\end{array}$$ (44)

with ${\mathcal{N}}_{<}$ and ${\mathcal{N}}_{>}$ given by Equation 42 and Equation 43, respectively. Since ${\mathcal{N}}_{<}$ decreases to zero with L in a power-law fashion at $q=q_c,$ the dominant contribution at this value is ${\mathcal{N}}_{>}.$ At $q=q_c,$ the mean genotypic distance $\rho *$ jumps discontinuously from $\rho *\left(q_c\right)\approx 0.7035$ to $\rho *=1/2;$ and the mean phenotypic distance $z*,$ which is defined as the averaged magnitude of phenotype vectors for local maxima, jumps from $z*\approx 0$ to $z*=\left(q_c-q_o\right)L.$ The genotypic complexity $\Sigma *$ defined in Equation 13 is given by

$$\Sigma *=\{\begin{array}{ll}\Sigma \left(\rho *\right),\hfill & q<q_c,\hfill \\ 0,\hfill & q\ge q_c,\hfill \end{array}$$ (45)

where $\rho *$ is the solution of Equation 41, and hence vanishes continuously at $q=q_c.$ These results are graphically represented in Figure 8. Recall that the value $\Sigma *=\ln 2$ attained at $q=0$ is the largest possible, because the total number of genotypes is $2^L=\exp \left(L\ln 2\right).$ Remarkably, these leading order results are independent of the phenotypic dimension. A dependence on n emerges at the subleading order, and it affects the number of fitness maxima in qualitatively different ways in the two phases. For $q<q_c,$ the preexponential factor in Equation 42 is a power law in L with exponent $1+n/2$ and hence decreases with increasing n; whereas the expression in Equation 43 describing the regime $q>q_c$ increases exponentially with n.

Figure 8.

Plot of the genotypic complexity $\Sigma \text{*}$ as a function of the scaled phenotypic wild-type distance q. Here the phenotypic dimension n is kept finite while taking the genotypic dimension L to infinity. The complexity vanishes at the phase transition point $q=q_c\approx 0.924809.$ Inset: Plot of the mean genotypic distance $\rho *$ of local maxima from the wild type as a function of q. Starting from $1/2,$ $\rho *$ increases with q for $q<q_c$ and remains at $1/2$ for $q>q_c.$

Interpretation:

The phase transition reflects a shift between two distinct mechanisms for generating genotypic complexity in FGM, which are analogous to the two origins of pairwise sign epistasis that were identified by Blanquart et al. (2014) and discussed above in Sign epistasis. In regime I $\left(q<q_c\right),$ the mutant phenotype closely approaches the origin and multiple fitness maxima are generated by overshooting the phenotypic optimum. By contrast, in regime II $\left(q>q_c\right),$ the phenotypic optimum cannot be reached and the genotypic complexity arises from the local curvature of the fitness isoclines. These two situations are exemplified by the two panels of Figure 1. For the sake of brevity, in the following discussion we will refer to the two mechanisms as mechanism I and mechanism II, respectively.

The approach to the origin in regime I is a largely one-dimensional phenomenon governed by the components of the mutation vector along the direction of the wild-type phenotype $\overrightarrow{Q},$ which explains why the leading order behavior of the genotypic complexity is independent of n. For $q<q_c,$ the n dependence of the preexponential factor in Equation 42 arises from the increasing difficulty of the random walk formed by the mutational vectors to locate the origin in high dimensions. By contrast, mechanism II operating for $q>q_c$ relies on the existence of the transverse dimensions, which is the reason why ${\mathcal{N}}_{>}$ in Equation 43 is an increasing function of n with ${\mathcal{N}}_{>}=1$ for $n=1.$

When $q_0<q<q_c,$ both mechanisms seem to be present simultaneously. As our analysis is restricted to the average number of local maxima, at this point we cannot decide whether both mechanisms appear in a single realization of the fitness landscape, or if one of them dominates for a given realization. To answer this question, we generated ${10}^4$ fitness landscapes randomly for given parameter sets and identified all local maxima for each landscape. We then determined the number of local maxima and averaged the phenotypic distance of the local maxima to the optimum for each realization. This mean distance will be denoted by $\stackrel{\sim }{z}$ and is itself a random variable; it should not be confused with the mean phenotypic distance $z*,$ which is calculated by taking an average over all fitness peaks in all realizations, giving the same weight to each peak. The results are depicted as a two-dimensional histogram in Figure 9A.

Figure 9.

Coexistence of the two mechanisms I and II for $q_0<q<q_c.$ (A) Two-dimensional histogram of the number of fitness maxima and the average phenotypic distance of the maxima to the optimum within a single realization. Here $L=15$ and $n=2$ are used and ${10}^4$ different landscapes are randomly generated for each value of q. Only a small number of realizations have a small average distance but these contribute an exceptionally large number of fitness peaks. (B) Two examples of genotype–phenotype maps selected from realizations with $q=0.5,$ $L=6,$ and $n=2.$ The wild-type phenotype is marked by a green ▴ and local fitness maxima by red ▪’s. When the phenotypes of the local fitness maxima are close to (far away from) the origin, the number of maxima is large (small), which corresponds to mechanism I (II).

The figure shows that the marginal distribution of $\stackrel{\sim }{z}$ displays a pronounced peak around $\stackrel{\sim }{z}/L\approx q-q_0,$ which corresponds to the behavior that is typical of mechanism I. For most realizations, $\stackrel{\sim }{z}/L$ deviates significantly from zero and only a small number of landscapes have local maxima near $z=0.$ However, these landscapes have many more maxima than typical landscapes and therefore dominantly contribute to the mean number of maxima $\mathcal{N}.$ This shows that within a single realization the two mechanisms are not operative together and only a single mechanism exists. Since most realizations exhibit mechanism II, whereas the mean number of local maxima grows exponentially as expected for mechanism I, we conclude that mechanism I occurs rarely but once it does, it generates a huge number of local maxima, which compensates the low probability of occurrence. We may thus say that both mechanisms coexist for $q_0<q<q_c$ and $q_0$ can be regarded as the threshold of coexistence. Two fitness landscape realizations generated for the same value of q located in the coexistence region that exemplify the two mechanisms are shown in Figure 9B.

Summary 2:

If the dimension n of phenotypic space is much smaller than the dimension L of genotypic space, there exists a threshold $q_c$ of the scaled wild-type distance q to the phenotypic optimum below which the mean number $\mathcal{N}$of local maxima in a genotypic fitness landscape increases exponentially with L, and above which it saturates to a finite value. The genotypic complexity $\text{Σ*},$ which is defined as the exponential growth rate of $\mathcal{N}$with L, is a decreasing function of q but does not depend on n. On the other hand, $\mathcal{N}$ decreases with n for $q<q_c$ yet increases with n for $q>q_c.$ Figure 8 depicts $\Sigma *$ and the mean genotypic distance $\rho *$ as functions of q. For $q_0<q<q_c,$ where $q_0=1/\sqrt{2\pi },$ $\mathcal{N}$ is dominated by a small fraction of landscape realizations that display an exceptionally large number of maxima. If the pleiotropic scaling is assumed to follow the total effects model, we need to specify how the unscaled wild-type distance d in Equation 2 depends on L. Assuming that $d=d_{0}L,$ where $d_0$ is independent of n (Orr 2000), the scaled wild-type distance $q=Q/L=d_0\sqrt{n}$ becomes an increasing function of n, and therefore the relation $q<q_c$ for regime I is never realized when n is sufficiently large.

Genotypic complexity in the joint limit

In the previous subsection, we have calculated the mean number of local fitness maxima $\mathcal{N}$at a fixed phenotypic dimension n, assuming that the genotypic dimension L is much larger than n $\left(L\gg n\right).$ However, in applications of FGM one often expects that both L and n are large and possibly of comparable magnitude. In this case, the results derived above can be unreliable for large n, as exemplified by the fact that the subleading correction to Equation 42 is of the order of $O\left(L^{-1/n}\right)$ (see Appendix C).

To obtain a reliable expression for $\mathcal{N}$that is valid when both n and L are large, we now consider the joint limit $n,L\to \mathrm{\infty }$ at fixed ratio $\alpha =n/L.$ This will allow us to find the leading behavior of the mean number of local maxima with a correction of order $O\left(1/L\right).$ Furthermore, we will clarify the role of the phenotypic dimension in the two phases described in the previous subsection, and we will uncover a third phase that appears at large α (see Figure 10).

Figure 10.

Phase diagrams in the parameter space $\left(q,\alpha \right).$ Here, $q=Q/L$ is the scaled distance of the wild-type phenotype from the origin and $\alpha =n/L$ is the ratio of phenotypic dimension to genotypic dimension. Dashed lines are phase boundaries at which the mean genotypic and phenotypic distances change discontinuously. (A) The phase boundary separating regimes I and II starts at $\left(q,\alpha \right)\simeq \left(0.925,0\right)$ and continues to exist until approximately $\alpha \simeq \mathrm{0.18.}$ (B) The phase boundary separating regimes II and III starts at $\left(q,\alpha \right)\simeq \left(0,2.38\right)$ and continues to exist until approximately $q\simeq \mathrm{0.62.}$

The number of local fitness maxima:

We relegate the detailed calculation to Appendix D and directly present our final expression for the mean number of local maxima,

$$\mathcal{N}=C\left(a*,b*,g*\right)e^{L{\Sigma }^{\text{red}}{}^{\left(a*,b*,g*\right)}}\left[1+O\left(\frac{1}{L}\right)\right],$$ (46)

where the function ${\Sigma }^{\text{red}}\left(a,b,g\right)$ in the exponent is given by

$$\begin{array}{l}{\Sigma }^{\text{red}}\left(a,b,g\right)=-\frac{\alpha }{2}\ln \left[\frac{\alpha \left(\alpha +g\right)}{2\left(ac\left(g\right)+b^2\right)}\right]+\frac{\alpha +2b+g}{2}-\ln 2\\ +\ln \left\{e^{-2c\left(g\right)}\left[\text{erf}\left(\frac{\alpha +2b}{\sqrt{2a}}\right)+1\right]+\mathrm{erf}\left(\frac{\alpha }{\sqrt{2a}}\right)+1\right\},\end{array}$$ (47)

with $c\left(g\right)=\left({\alpha }^2-g^2\right)/\left(16q^2\right).$ As before, the starred variables $a*,$ $b*,$ and $g*$ denote the solution of the extremum condition

$${\nabla {\Sigma }^{\text{red}}\left(a,b,g\right)|}_{\left(a,b,g\right)=\left(a*,b*,g*\right)}=\left(0,0,0\right),$$ (48)

where $\nabla$ is the gradient with respect to the three variables $\left(a,b,g\right).$ When several solutions of Equation 48 exist, the one giving the largest value of ${\Sigma }^{\text{red}}$ is chosen. The prefactor $C\left(a*,b*,g*\right),$ which is independent of L, can be determined from Equation D17 presented in Appendix D. Even though the variables $\left(a,b,g\right)$ lack a direct interpretation in terms of the original setting of FGM, we show in Appendix E that $a*$ is related to the mean phenotypic distance $z*$ by the equation $z*=L\sqrt{a*}/2.$

An immediate consequence of Equation 46 is that the number of local maxima increases exponentially in L for any value of q and α without algebraic corrections of the kind found in Equation 42. Obtaining closed-form solutions of Equation 48, which ultimately determine the functional dependence of the complexity $\Sigma *$ on α and q, seems to be a formidable task. Instead, we resort to numerical methods by sweeping through the most interesting intervals, $q\in \left(0,2\right)$ and $\alpha \in \left(0,3\right).$ Surprisingly, we find three independent branches of solutions that correspond to distinct phases. To acquire a qualitative understanding of these branches, it is instructive to first focus on the small α behavior, where one expects a smooth continuation to the results of Equations 42 and 43 as $\alpha \to 0.$

Small α behavior:

In contrast to the fixed n case where two separate analyses were carried out for the two regimes $q<q_c$ and $q>q_c,$ the present approach yields a single expression describing the genotypic complexity for arbitrary values of q and α. Consistently with the fixed n analysis, only two out of the three branches of solutions that were found in the numerical analysis exist for sufficiently small α, and they are separated by a phase transition as shown in the phase diagram in Figure 10A. By extrapolating the behavior of $\Sigma *$ toward $\alpha \to 0$ as shown in Figure 11, we are able to identify the correct counterparts for each of the two previously found regimes.

Figure 11.

Convergence of the complexity to the fixed n case for small α. (A) The solid lines depict numerical solutions of Equation 48 for values of α belonging to regime I. The convergence to Equation 42 (dashed line) is clearly seen as $\alpha \to 0.$ (B) The blue solid line depicts the numerical solution of Equation 48 for $\alpha =0.1$ belonging to regime II. Except for a slight deviation detectable when q is close to $q_0,$ Equation 49 (dashed line) remains a good approximation.

The extrapolation is straightforward in regime II, where the replacement $n\to \alpha L$ in Equation 43 yields an exponential dependence of $\mathcal{N}$on L with the growth rate

$${\Sigma }_{\text{approx}}^{*\left(\text{II}\right)}=\alpha \ln \left\{\frac{q-q_0}{q}\exp \left[{\left(q/q_0-1\right)}^{-1}\right]\right\}.$$ (49)

This crude approximation turns out to be remarkably accurate even at $\alpha =0.1,$ as illustrated in Figure 11B. By contrast, in regime I the naive replacement of n by $\alpha L$ in Equation 42 yields an expression that vanishes faster than exponential in L, as $\exp \left[-\left(\alpha /2\right)L\ln L\right].$ This reflects the fact that the mean phenotypic distance $z*$ moves away from the origin for any $\alpha >0$ and hence the complexity cannot be derived only by inspecting Equation 26 around $z=0$ (see Figure 12, A and D). At the same time, the mean genotypic distance $\rho *$ decreases with increasing α and eventually falls below the value $\rho *=1/2$ favored by the sequence entropy (Figure 12, B and E).

Figure 12.

Plots of scaled mean phenotypic distance $z*/L$ (left column), mean genotypic distance $\rho *$ (middle column), and genotypic complexity $\Sigma *$ (right column) against q for fixed α (top row) and against α for fixed q (bottom row). The curves in the top (bottom) panels are drawn along the arrows in the inset of (C) and (F). Top row (A, B, and C): When α is small, the landscape behaves similarly to the fixed n case which effectively corresponds to $\alpha =0.$ In this case $z*$ and $\rho *$ for large q are well approximated by $q-q_0$ and $1/2,$ respectively. As α increases beyond the transition line, the first-order transition visualized by the red dashed lines disappears and all quantities change smoothly with q. Bottom row (D, E, and F): As α increases for small q, another phase transition with discontinuities in $z*$ and $\rho *$ (blue dashed lines) signals the appearance of regime III. The genotypic maxima in regime III are located very close to the wild-type position, $z*/L\simeq q$ and $\rho *\simeq 0.$ This transition ceases to exist when q exceeds ∼0.62. Note that the dependence of $\text{Σ*}$ on α is nonmonotonic for $q=0.7$ (F).

Both trends can be attributed to the increasing role of the perpendicular mutational displacements that make up the second term on the right-hand side of Equation 5. Under the scaling of the joint limit, this term is of order $\rho L\left(n-1\right)\approx \rho \alpha L^2$ and hence comparable to the first term originating from the parallel displacements. The perpendicular displacements always increase the phenotypic distance to the origin, and they are present even when $q=0.$ The additional cost to reduce the perpendicular contribution results in a smaller value of $\Sigma *$ compared to the case of fixed n. Moreover, whereas the parallel contribution is minimized (for $q>q_0)$ by making ρ as large as possible, the reduction of the perpendicular displacements requires small ρ.

In the fixed n analysis, the number of fitness maxima was found to decrease (increase) with n in regime I (II) and this tendency is recovered from the joint-limit case when α is not too large (Figure 12C). Because of these opposing trends of $\Sigma *$ in the two regimes, the location of the phase transition separating them is expected to decrease with increasing α, as can be seen in Figure 10A. If one ignores the contribution from the perpendicular displacements, the phenotypic position of the fitness maxima is expected to jump from $z*=0$ to $z*=q_c-q_0$ at the transition, and thus the jump size should decrease as $q_c$ decreases. This observation suggests that the two branches should merge into one when $q_c$ reaches $q_0.$ With the additional contribution of perpendicular dimensions, we numerically found that this critical end point at which the phases I and II merge occurs even earlier, at $\alpha \simeq 0.18$ and $q\simeq 0.62>q_0$ (Figure 10). For α > 0.18, $\rho *$ does not show any discontinuity for any q as long as the parameters are in regime II.

Large α behavior and regime III:

To develop some intuition about the FGM fitness landscape in the regime where $\alpha =n/L\gg 1,$ we revisit the results obtained in Sign epistasis, where pairs of mutations were considered. Two conclusions can be drawn about the typical shape of these small genotypic landscapes (of size $L=2)$ in the limit $n\to \mathrm{\infty }.$ First, the probability that the wild type is a genotypic maximum tends to unity according to Equation 10. Second, the joint distribution given in Equation 15 enforces additivity of mutational effects for large n, and correspondingly the probability for sign epistasis vanishes. Thus for large n the two-dimensional genotypic landscape becomes smooth with a single maximum located at the wild type. Assuming that this picture holds more generally whenever the limit $n\to \mathrm{\infty }$ is taken at finite L, we expect the following asymptotic behaviors of the quantifiers of genotypic complexity for large α: (i) $\mathcal{N}\to 1,$ $\Sigma *\to 0$ (unique genotypic optimum); and (ii) $z*/L\to q,$ $\rho *\to 0$ (location of the maximum at the wild-type phenotype and genotype).

This expectation is largely borne out by the numerical results shown in the bottom panels of Figure 12. However, depending on the value of q, the approach to the limit of a smooth landscape can be either continuous (for large q) or display characteristic jumps as indicated by the blue dashed lines in Figure 12, D and E. These jumps as well as the discontinuity in the slope of $\Sigma *$ as a function of α in Figure 12F are hallmarks of the phase transition to the new regime III, which is represented by the dashed line in Figure 10B.

Fortunately, the solution of Equation 48 describing the new phase can be obtained analytically from Equation 47 or Equation F3 in Appendix F as a series expansion. The derivation presented in Appendix G yields

$$\begin{array}{l}a*=4q^2-\left[\frac{16\sqrt{\frac{2}{\pi }}{q}^3}{{\alpha }^2}+O\left(q^4/{\alpha }^3\right)\right]\mathit{ϵ}+O\left({\mathit{ϵ}}^2\right),\\ b*=-\alpha +\frac{\alpha \mathit{ϵ}}{\sqrt{2\pi }q}+O\left({\mathit{ϵ}}^2\right),\\ g*=\alpha +O\left({\mathit{ϵ}}^2\right),\hspace{1em}\text{and}\hspace{1em}\rho *=\frac{\sqrt{\frac{2}{\pi }}q\mathit{ϵ}}{\alpha }+O\left({\mathit{ϵ}}^2\right),\end{array}$$ (50)

where the expansion parameter $\mathit{ϵ}=e^{-{\alpha }^2/\left(8q^2\right)}$ decays rapidly with increasing $\alpha /q.$ The corresponding genotypic complexity can also be evaluated in a series expansion,

$${\Sigma }^{\left(\text{III}\right)}\left(a*,b*,c*\right)=\frac{\alpha {\mathit{ϵ}}^2}{4\pi q^2}+O\left({\mathit{ϵ}}^3\right),$$ (51)

which shows that $\Sigma *$ is positive but vanishingly small in this regime. We note that using Equation E5, the expression for $a*$ in Equation 50 amounts to

$$\frac{z*}{L}\simeq q-\frac{2\sqrt{\frac{2}{\pi }}{q}^2}{{\alpha }^2}\mathit{ϵ},$$ (52)

implying that the small number of local maxima that exist in this phase are located very close to the wild-type phenotype.

To first order in ε, the results for $\rho *$ and $z*$ in Equations 50 and 52 can be easily derived from the idea that mutational effects become approximately additive for large α, thus providing further support for this assumption. If mutational effects are strictly additive, the probability for a genotype containing s mutations to be a local fitness maximum is given by

$${\mathrm{ℛ}}_s^{\text{add}}=P_b^s{\left(1-P_b\right)}^{L-s},$$ (53)

where $P_b$ is the probability for a mutation to be beneficial. Equation 53 expresses the condition that reverting any one of the s mutations contained in the genotype as well as adding one of the unused $L-s$ mutations should lower the fitness. Using Fisher’s Equation 1, the probability for a beneficial mutation is $P_b\approx \sqrt{\left(2/\pi \right)}\left(q/\alpha \right)\mathit{ϵ}$ for large α. Thus to linear order in ε or $P_b,$ the expected number of mutations contributing to such a genotype is $L{P}_b=L\rho *=L\sqrt{\left(2/\pi \right)}\left(q/\alpha \right)\mathit{ϵ},$ which is consistent with Equation 50.

The phenotypic location of a local maximum deviates from $\overrightarrow{Q}$ in those rare instances where one of the mutations from the wild type is beneficial, which happens with probability $P_b.$ To estimate the corresponding shift in $z*\text{,}$ we refer to the results of subsection Sign epistasis, where it was shown that the squared phenotypic displacement $R_1$ defined in Equation 14 has a Gaussian distribution with mean 1 and variance $1/x^2=4q^2/{\alpha }^2$ for large n. Using this, it is straightforward to show that the expected value of $R_1$ conditioned on the mutation to be beneficial $\left(R_1<0\right)$ is ${\overline{R}}_1=-4q^2/{\alpha }^2$ to leading order. Multiplying this by the expected number of mutations $L{P}_b$ we obtain the relation

$$L\rho *{\overline{R}}_1\approx \frac{{\left(z*\right)}^2-Q^2}{n},$$ (54)

which yields the same leading behavior for $z*/L$ as in Equation 52.

As previously observed for the transition between regimes I and II, the phase boundary separating regimes II and III terminates at a point where the two solutions defining the regimes merge (Figure 10B). Beyond this point the jumps in $z*$ and $\rho *$ seen in Figure 12, D and E, disappear and all quantities approach smoothly to their asymptotic values. A surprising feature of the large α behavior that persists also for larger q is that the complexity becomes an increasing function of q when $\alpha >1.7$ (Figure 12F). In Figure 13 we verify this behavior using direct simulations of FGM. These simulations also show that the predictions based on Equation 47 are already remarkably accurate for moderate values of L and n.

Figure 13.

Semilogarithmic plots of the mean number of local maxima $\mathcal{N}$ vs. the genotypic dimension L for (A) $\alpha =0.2$ and (B) $\alpha =2.5$ and for various values of q. Each symbol represents the average over ${10}^5$ randomly generated landscapes, and lines depict the analytic approximation of Equation D17. The approximation is good even for moderate L.

Summary 3:

When the dimension n of the phenotypic trait space and the dimension L of the genotypic space are large and comparable, the genotypic complexity $\Sigma *$ is always nonzero and depends on the ratios $\alpha =n/L$ and $q=Q/L.$ There are three regimes where the behavior of the genotypic complexity and the mean genotypic distance $\rho *$ (the average number of mutations in a local maximum divided by L) are qualitatively different. In regime I, which is roughly characterized by small q and small α, there are many local maxima in the region located far away from the wild type but close to the phenotypic optimum, and the fitness landscape is quite rugged. In regime II, which is roughly characterized by large q and small α, there is an appreciable number of local maxima, though smaller than in regime I, and typically half of the L mutations contribute to the corresponding genotypes. In regime III, which is roughly characterized by large α, the genetic complexity is very small, though nonzero. Also $\rho *$ is close to zero, which means that the wild type has a high probability to be the global fitness maximum. An overview of the three regimes is found in Table 2.

Table 2. Characteristics of the three regimes in the joint limit.

Regime	Condition	$\Sigma *$	$\rho *$	Landscape
I	$q\ll 1,$ $\alpha \ll 1$	$>0$	$>1/2$	rugged
II	$q\gg 1,$ $\alpha \ll 1$	$\simeq 0$	$\approx 1/2$	intermediate
III	$\alpha \gg 1$	$\simeq 0$	$\simeq 0$	almost smooth

Discussion

FGM provides a simple yet generic scenario for the emergence of complex epistatic interactions from a nonlinear mapping of an additive, multidimensional phenotype onto fitness. Its role in the theory of adaptation may be aptly described as that of a “proof-of-concept model” (Servedio et al. 2014), and as such it is widely used in fundamental theoretical studies (Gros et al. 2009; Chevin et al. 2010; Blanquart et al. 2014; Martin 2014; Fraïsse et al. 2016; Moura de Sousa et al. 2016) as well as for the parameterization and interpretation of empirical data (Martin et al. 2007; Velenich and Gore 2013; Weinreich and Knies 2013; Bank et al. 2014; Perfeito et al. 2014; Blanquart and Bataillon 2016; Schoustra et al. 2016). Rather than tracing the mutational effects and their interactions to the underlying molecular basis, the model aims at identifying robust features of the adaptive process that can be expected to be shared by large classes of organisms.

To give an example of such a feature that is of central importance in the present context, it was pointed out by Blanquart et al. (2014) that pairwise sign epistasis is generated in FGM through two distinct mechanisms. In one case the mutational displacements overshoot the phenotypic optimum, whereas in the other case the displacements are directed approximately perpendicular to the direction of the optimum, and sign epistasis arises because the fitness isoclines are curved. The first mechanism is obviously also operative in a one-dimensional phenotype space, but in the second case (termed antagonistic pleiotropy by Blanquart et al. 2014) at least two phenotypic dimensions are required. Interestingly, both mechanisms have been invoked in empirical studies where a nonlinear phenotype-fitness map was used to model epistatic interactions between multiple mutations. In one study, Rokyta et al. (2011) explained the pairwise epistatic interactions between nine beneficial mutations in the single-stranded DNA bacteriophage ID11 by assuming that fitness is a single-peaked nonlinear function of a one-dimensional additive phenotype. In the second study the genotypic fitness landscapes based on all combinations of two groups of four antibiotic resistance mutations in the enzyme β-lactamase were parameterized by a nonlinear function mapping a two-dimensional phenotype to resistance (Schenk et al. 2013). The fitted function was in fact monotonic and did not possess a phenotypic optimum, which makes it clear that the epistatic interactions arose solely from antagonistic pleiotropy in this case.

In this work we have shown that the two mechanisms described by Blanquart et al. (2014) lead to distinct regimes or phases in the parameter space of FGM, where the genotypic fitness landscapes display qualitatively different properties (Figure 10A). When the phenotypic dimension n is much smaller than the genotypic dimension L, the two regimes are separated by a sharp phase transition where the average number and location of genotypic fitness maxima changes abruptly as the distance q of the wild-type phenotype from the optimum is varied. In regime I $\left(q<q_c\right),$ the phenotypic optimum is reachable at least by some combinations of mutational displacements. Overshooting of the optimum is therefore possible and sign epistasis is strong, leading to rugged genotypic landscapes with a large number of local fitness maxima that grows exponentially with L. By contrast, in regime II $\left(q>q_c\right),$ only antagonistic pleiotropy is operative and the number of fitness maxima is much smaller. More precisely, for finite n the number tends to a finite limit for $L\to \mathrm{\infty },$ but the limiting value is an exponentially growing function of n.

An important consequence of our results is that the dependence of the fitness landscape ruggedness on the phenotypic dimension n is remarkably complicated. For $n\ll L,$ landscapes become less rugged with increasing n in regime I $\left(q<q_c\right),$ but display increasing ruggedness in regime II $\left(q>q_c\right).$ When $n\gg L,$ the ruggedness decreases with n for all q and the landscapes become approximately additive (regime III). In particular, the probability of sign epistasis vanishes algebraically with n in this regime. Thus n cannot in general be regarded as a measure of “phenotypic complexity,” as a larger value of n does not imply that the corresponding fitness landscape is more complex.

This observation is relevant for the interpretation of experiments where the parameters of FGM are estimated from data. In recent work, FGM was used to analyze data on pairwise epistasis between beneficial mutations in the filamentous fungus Aspergillus nidulans growing in two different media (Schoustra et al. 2016). The estimates obtained for the phenotypic dimension and the distance of the wild-type phenotype from the optimum were n = 19.3 and Q = 6.89 in complete medium and n = 34.8 and Q = 9.81 in minimal medium, which, surprisingly, may seem to suggest a higher phenotypic complexity in the minimal medium. Using the results derived in this article, we can translate the estimated parameter values into the average number of maxima that a genotypic fitness landscape of a given dimension L would have. As can be seen in Figure 14, with respect to this measure the fitness landscape of the fungus growing in complete medium is actually more rugged. This is consistent with experiments using Escherichia coli, which found a greater heterogeneity of fitness trajectories in complete medium (Rozen et al. 2008), and indicates that the complete medium allows for a greater diversity of paths to adaptation than the minimal medium.

Figure 14.

The logarithm of the number of local fitness maxima divided by the number of loci L is shown as a function of L for FGM with the parameter values $n=19.3,\hspace{0.17em}Q=6.89$ and $n=34.8,Q=9.81$ obtained by Schoustra et al. (2016) for the fungus A. nidulans growing in complete (CM) and minimal medium (MM), respectively. For the evaluation of $\mathcal{N}$ Equation 47 was used.

We hope that the results presented here will promote the use of FGM as part of the toolbox of probabilistic models that are currently available for the analysis of empirical fitness landscapes (Hayashi et al. 2006; Szendro et al. 2013; de Visser and Krug 2014; Neidhart et al. 2014; Bank et al. 2016; Blanquart and Bataillon 2016). Compared to purely genotype-based models such as the NK and rough-Mount-Fuji (RMF) models, FGM is arguably more realistic in that it introduces an explicit phenotypic layer mediating between genotypes and fitness (Martin 2014). Somewhat similarly to the RMF model, the fitness landscapes of FGM are anisotropic and display a systematic change of properties as a function of the distance to the optimal phenotype (FGM) or the reference sequence (RMF), respectively (Neidhart et al. 2014). The idea that fitness landscape ruggedness increases systematically and possibly abruptly when approaching the optimum has been proposed previously in the context of in vitro evolution of proteins (Hayashi et al. 2006). If this is indeed a generic pattern, it may have broader implications. For example, de Visser et al. (2009) showed that the evolutionary benefits of recombination are severely limited by the presence of multiple peaks. If such peaks are rarely encountered far away from the optimum, the benefits of recombination would be most pronounced for particularly maladapted populations.

A recent investigation of 26 published empirical fitness landscapes using ABC concluded that FGM could account for the full structure of the landscapes only in a minority of cases (Blanquart and Bataillon 2016). One of the features of the empirical landscapes that prevented a close fit to FGM was the occurrence of sign epistasis far away from the phenotypic optimum. Our analysis confirms that this is an unlikely event in FGM, and precisely quantifies the corresponding probability through Equation 16 and Equation 17. Blanquart and Bataillon (2016) also found that the phenotypic dimension is particularly difficult to infer from realizations of genotypic fitness landscapes, which matches our observation that the structure of the landscape depends only weakly on n when $n\ll L.$ We expect that our results will help to further clarify which features of an empirical fitness landscape make it more or less amenable to a phenotypic description in terms of FGM or some generalization thereof.

We conclude by mentioning some open questions that should be addressed in future theoretical work on FGM. First, a significant limitation of our results lies in their restriction to the average number of local fitness maxima. The number of maxima induced by a given realization of mutational displacements is a random variable, and unless the distribution of this variable is well concentrated, the average value may not reflect the typical behavior. The large fluctuations between different realizations of fitness landscapes generated by FGM were noticed already by Blanquart et al. (2014) on the basis of small-scale simulations, and they clearly contribute to the difficulty of inferring the parameters of FGM from individual realizations that was reported by Blanquart and Bataillon (2016). In light of our analysis, this pronounced heterogeneity can be attributed to the existence of multiple phases in the model, and it is exemplified by the simulation results in Figure 9. To quantitatively characterize the fluctuations between different realizations, a better understanding of the distribution of the number of fitness maxima and its higher moments is required.

Second, the consequences of relaxing some of the assumptions underlying the formulation of FGM used in this work should be explored. The level of pleiotropy can be reduced by restricting the effects of mutational displacements to a subset of traits (Chevin et al. 2010; Moura de Sousa et al. 2016), and it would be interesting to see how this affects the ruggedness of the fitness landscape. However, the most critical and empirically poorly motivated assumption of FGM is clearly the absence of epistatic interactions on the level of phenotypes. It would therefore be important to understand how robust the results presented here are with respect to some level of phenotypic epistasis, which should ideally arise from a realistic model of phenotypic networks (Martin 2014).

Third, a natural extension of the present study is to consider multiallelic genetic sequences. An immediate generalization keeping the additivity of mutational effects on the level of phenotypes is to consider the following genotype–phenotype map:

$$\overrightarrow{z}\left(\tau \right)=\overrightarrow{Q}+{\displaystyle \sum _{i=1}^L{\displaystyle \sum _{k=1}^A{\tau }_{ik}{\overrightarrow{\xi }}_{ik}}},$$ (55)

where A is the size of the alphabet from which the sequence elements are drawn (e.g., $A=4$ for DNA or RNA and $A=20$ for proteins), ${\tau }_{ik}=1$ (0) if the allele at site i is (is not) k, and the ${\overrightarrow{\xi }}_{ik}$ are uncorrelated random vectors. Clearly, our results for pairwise epistasis remain the same for this generalized model because they only concern mutations at different sites. However, the condition for a local fitness maximum now involves mutations to different alleles at the same site, which may lead to a nontrivial dependence on A. On the basis of a recent study of evolutionary accessibility in multiallelic sequence spaces, one may expect the fitness landscapes to become less rugged with increasing A (Zagorski et al. 2016), but this conjecture would have to be corroborated by a detailed analysis.

Finally, whereas the present work focused on the structure of the fitness landscapes induced by FGM, it is of obvious importance to understand how the adaptive process actually proceeds on such a landscape (Orr 2005). A simple framework that allows us to address this question is provided by adaptive walks following Gillespie’s strong selection/weak mutation dynamics (Gillespie 1983, 1984; Orr 2002). In a pioneering study, Orr (1998) considered adaptive walks in FGM assuming that the number L of possible mutations is unlimited. In this setting, any population not located precisely at the phenotypic optimum has a nonzero probability of generating another beneficial mutation and the adaptive walk never stops; see Park and Krug (2008) for a related analysis of adaptation in the house-of-cards landscape. For finite but large L, an interesting question concerns the number of steps until the population finds a local fitness maximum when the adaptive dynamics is random (Kauffman and Levin 1987; Park et al. 2015; Park and Krug 2016) or greedy (Orr 2003; Park et al. 2016). This problem is currently under investigation.

Appendix A: Derivation of the Joint Probability Density $\mathcal{P}\left(R_1,R_2,R\right)$

For the purpose of this calculation it will turn out to be convenient to locate the wild-type phenotype on the diagonal of the trait space, i.e., to set $\overrightarrow{Q}=Q/\sqrt{n}\left(1,1,1,\dots ,\right).$ The probability density $\mathcal{P}\left(R_1,R_2,R\right)$ can then be formally defined as

$$\mathcal{P}\left(R_1,R_2,R\right)=n^3{\langle \delta \left\{n{R}_1-{\displaystyle \sum _{i=1}^n\left[{\left({\xi }_i+Q_i\right)}^2-Q_i^2\right]}\right\}\delta \left\{n{R}_2-{\displaystyle \sum _{i=1}^n\left[{\left({\eta }_i+Q_i\right)}^2-Q_i^2\right]}\right\}\delta \left\{nR-{\displaystyle \sum _{i=1}^n\left[{\left({\xi }_i+{\eta }_i+Q_i\right)}^2-Q_i^2\right]}\right\}\rangle }_{\overrightarrow{\xi },\overrightarrow{\eta }},$$ (A1)

where $Q_i=Q/\sqrt{n}$ and ${\langle \cdots \rangle }_{\overrightarrow{\xi },\overrightarrow{\eta }}$ stands for the average over the distribution of $\overrightarrow{\xi }$ and $\overrightarrow{\eta }.$ Using the integral representation of the δ function, we can write

$$\mathcal{P}\left(R_1,R_2,R\right)=\frac{n^3}{{\left(2\pi \right)}^3}{\displaystyle \int d\overrightarrow{k}}{e}^{i{k}_{1}n{R}_1+i{k}_{2}n{R}_2+iknR}{\displaystyle \prod _i{\langle e^{-i{k}_1{\left({\xi }_i+Q_i\right)}^2-i{k}_2{\left({\eta }_i+Q_i\right)}^2-ik{\left({\xi }_i+{\eta }_i+Q_i\right)}^2+i{Q}_i^2\left(k_1+k_2+k_3\right)}\rangle }_{\overrightarrow{\xi },\overrightarrow{\eta }}},$$ (A2)

where $d\overrightarrow{R}$ and $d\overrightarrow{k}$ stand for $d{R}_{1}d{R}_{2}dR$ and $d{k}_{1}d{k}_{2}dk,$ respectively, and we factorized the average by taking into account that the ${\xi }_i’\text{s}$ and ${\eta }_i’\text{s}$ are all independent and identically distributed. The average in Equation A2 is readily calculated as

$$\begin{array}{l}\frac{1}{2\pi }{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}d\xi }{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}d\eta }\exp \left[-\frac{{\eta }^2}{2}-\frac{{\xi }^2}{2}-i{k}_1{\xi }^2-i{k}_2{\eta }^2-ik{\left(\xi +\eta \right)}^2-2i{k}_1\frac{Q}{\sqrt{n}}\xi -2i{k}_2\frac{Q}{\sqrt{n}}\eta -2ik\frac{Q}{\sqrt{n}}\left(\xi +\eta \right)\right]\\ =\sqrt{\frac{1}{\left(1+2i{k}_1\right)\left(1+2i{k}_2\right)-4k\left(k_1+k_2-i\right)}}\exp \left[\frac{2Q^2}{n}\frac{2i\left(k+k_1\right)\left(k+k_2\right)\left(k_1+k_2-i\right)+{\left(k_1-k_2\right)}^2}{4k\left(k_1+k_2-i\right)+\left(2k_1-i\right)\left(2k_2-i\right)}\right],\end{array}$$ (A3)

which gives

$$\mathcal{P}\left(R_1,R_2,R\right)=\frac{n^3}{{\left(2\pi \right)}^3}{\displaystyle \int d\overrightarrow{k}}\frac{e^{i{k}_{1}n{R}_1+i{k}_{2}n{R}_2+iknR}}{{\left[\left(1+2i{k}_1\right)\left(1+2i{k}_2\right)-4k\left(k_1+k_2-i\right)\right]}^{n/2}}\exp \left[\frac{n^2}{2x^2}\frac{2i\left(k+k_1\right)\left(k+k_2\right)\left(k_1+k_2-i\right)+{\left(k_1-k_2\right)}^2}{4k\left(k_1+k_2-i\right)+\left(2k_1-i\right)\left(2k_2-i\right)}\right].$$ (A4)

In the limit $n\to \mathrm{\infty },$ the integral is dominated by contributions from the vicinity of the extremum of the exponent, which can be algebraically determined to be $k=k_1=k_2=0.$ By expanding the argument of the exponential function up to the second order around this point and performing the Gaussian integral, we obtain

$$\begin{array}{c}\mathcal{P}\left(R_1,R_2,R\right)\approx \frac{n^3}{{\left(2\pi \right)}^3}{\displaystyle \int d\overrightarrow{k}}\exp \{-\frac{n^2}{2x^2}\left[{\left(k_1+k_2+k\right)}^2-2k_1{k}_2\right]-n\left[4k^2+k_1^2+k_2^2-i{k}_1\left(R_1-1\right)-i{k}_2\left(R_2-1\right)+k\left(2i+2k_1+2k_2-iR\right)\right]\}\\ =\frac{\sqrt{n}{x}^2}{4\sqrt{2}{\pi }^{3/2}}\left[1+O\left(n^{-1}\right)\right]\exp \left\{-\frac{n}{8}\left(R-R_1-R_2\right)-\frac{x^2}{2}\left[{\left(R_1-1\right)}^2+{\left(R_2-1\right)}^2\right]\right\},\end{array}$$ (A5)

which is Equation 15.

Appendix B: Probability of Sign Epistasis

In this appendix, we present the mathematical details of the derivation of the probabilities $P_r$ and $P_s$ of observing RSE and SSE, respectively. As in the main text, let us assume $R_1<R_2.$ In calculating the probabilities, the integral over R takes one of three forms

$$\begin{array}{l}{\displaystyle {\int }_{-\mathrm{\infty }}^{R_1}\sqrt{\frac{n}{8\pi }}{e}^{-\frac{n}{8}{\left(-R+R_1+R_2\right)}^2}\hspace{0.17em}dR}=\frac{1}{2}\text{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right),\hspace{1em}{\displaystyle {\int }_{R_2}^{\mathrm{\infty }}\sqrt{\frac{n}{8\pi }}{e}^{-\frac{n}{8}{\left(-R+R_1+R_2\right)}^2}\hspace{0.17em}dR}=\frac{1}{2}\mathrm{erfc}\left(-\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right),\hspace{1em}\text{or}\\ {\displaystyle {\int }_{R_1}^{R_2}\sqrt{\frac{n}{8\pi }}{e}^{-\frac{n}{8}{\left(-R+R_1+R_2\right)}^2}\hspace{0.17em}dR}=\frac{1}{2}\left[\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)-\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)\right].\end{array}$$ (B1)

First, we consider RSE which corresponds to the two domains

$$D_1=\left\{\left(R_1,R_2,R\right)|R_1<R_2<R,R_2<0\right\}\hspace{1em}\text{and}\hspace{1em}{D}_2=\left\{\left(R_1,R_2,R\right)|R<R_1<R_2,R_1>0\right\},$$ (B2)

as illustrated in Figure 2. The probability of being in $D_1$ is

$$\begin{array}{l}\mathrm{Pr}\left(D_1\right)={\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_{R_1}^{0}d{R}_2}{\displaystyle {\int }_{R_2}^{\mathrm{\infty }}dR\mathcal{P}\left(R_1,R_2,R_3\right)}=\\ \frac{x^2}{2\pi }{\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_{R_1}^{0}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\mathrm{erfc}\left(-\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)\\ =\frac{x^2}{2\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_0^{R_1}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1+1\right)}^2-\frac{x^2}{2}{\left(R_2+1\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right),\end{array}$$ (B3)

where we have changed variables $R_i\mapsto -R_i.$ Since $\mathrm{erfc}\left(y\right)\sim e^{-y^2}/\left(y\sqrt{\pi }\right)$ for $y\gg 1,$ the above integral is dominated by the region $R_1\ll 1$ for large n. Thus, it is sufficient to approximate $\exp \left[-x^2\left({\left(R_1-1\right)}^2+{\left(R_2-1\right)}^2\right)/2\right]\approx e^{-x^2},$ which yields

$$\mathrm{Pr}\left(D_1\right)\approx \frac{x^2{e}^{-x^2}}{4\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_0^{R_1}d{R}_2}\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)\approx \frac{x^2{e}^{-x^2}}{4\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{R}_1\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)=\frac{2x^2{e}^{-x^2}}{n\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}dy}\hspace{0.17em}y\text{erfc}\left(y\right)=\frac{x^2}{2n\pi }{e}^{-x^2}.$$ (B4)

The probability of being in $D_2$ has the same leading behavior,

$$\begin{array}{c}\mathrm{Pr}\left(D_2\right)={\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2}{\displaystyle {\int }_0^{R_2}d{R}_1}{\displaystyle {\int }_{-\mathrm{\infty }}^{R_1}dR}\mathcal{P}\left(R_1,R_2,R_3\right)=\frac{x^2}{2\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2}{\displaystyle {\int }_0^{R_2}d{R}_1}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)\\ \approx \frac{x^2}{2\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_0^{R_1}d{R}_2}\hspace{0.17em}{e}^{-x^2}\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)=\frac{x^2}{2n\pi }{e}^{-x^2},\end{array}$$ (B5)

where we have exchanged the variables $R_1\leftrightarrow R_2.$ Due to the symmetrical roles of $R_1$ and $R_2,$ the total probability of RSE is

$$P_r\equiv 2{\displaystyle \sum _{i=1}^2\mathrm{Pr}\left(D_i\right)}\approx \frac{2x^2}{n\pi }{e}^{-x^2}.$$ (B6)

We can use a similar approximation scheme to calculate the probability of SSE. There are four domains contributing to SSE (see Figure 2):

$$\begin{array}{l}{D}_3=\left\{\left(R_1,R_2,R\right)|R<R_1<0<R_2\right\},\hspace{1em}{D}_4=\left\{\left(R_1,R_2,R\right)|R_1<0<R_2<R\right\},\\ D_5=\left\{\left(R_1,R_2,R\right)|R_1<R<R_2<0\right\},\hspace{1em}{D}_6=\left\{\left(R_1,R_2,R\right)|0<R_1<R<R_2\right\}.\end{array}$$ (B7)

As we will see, all integrals can be represented by the functions

$$\begin{array}{l}{G}_1\left(a,b\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_0^{R_1}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1+a\right)}^2-\frac{x^2}{2}{\left(R_2+b\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right),\\ \end{array}$$ (B8)

$$\begin{array}{c}{G}_2\left(a,b\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_0^{R_1}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1+a\right)}^2-\frac{x^2}{2}{\left(R_2+b\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)\\ =\frac{x}{4\sqrt{2\pi }}\mathrm{erfc}\left(\frac{bx}{\sqrt{2}}\right){\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}\exp \left[-\frac{x^2}{2}{\left(R_1+a\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)-G_1\left(b,a\right),\end{array}$$ (B9)

where $a,b=\pm 1$ and we have used that

$${\displaystyle {\int }_0^{\mathrm{\infty }}dy}{\displaystyle {\int }_y^{\mathrm{\infty }}dz}f\left(y,z\right)={\displaystyle {\int }_0^{\mathrm{\infty }}dy}{\displaystyle {\int }_0^{y}dz}f\left(z,y\right).$$ (B10)

To be specific, we write the probabilities of being in each domain as

$$\mathrm{Pr}\left(D_3\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)=G_1\left(1,-1\right)+G_2\left(-1,1\right),$$

$$\mathrm{Pr}\left(D_4\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\mathrm{erfc}\left(-\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)=G_1\left(-1,1\right)+G_2\left(1,-1\right),$$

$$\mathrm{Pr}\left(D_5\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_{-\mathrm{\infty }}^{0}d{R}_1}{\displaystyle {\int }_{R_1}^{0}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\left[\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)-\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)\right]=G_1\left(1,1\right)-G_2\left(1,1\right),$$

$$\mathrm{Pr}\left(D_6\right)=\frac{x^2}{4\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1}{\displaystyle {\int }_{R_1}^{\mathrm{\infty }}d{R}_2}\exp \left[-\frac{x^2}{2}{\left(R_1-1\right)}^2-\frac{x^2}{2}{\left(R_2-1\right)}^2\right]\left[\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)-\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)\right]=G_1\left(-1,-1\right)-G_2\left(-1,-1\right),$$

where we have changed negative integral domains into positive domains and made use of Equation B10. Using the approximation scheme explained above, we get

$$\begin{array}{c}{G}_1\left(a,b\right)=\frac{x}{4\sqrt{2\pi }}{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2{e}^{-x^2{\left(R_2+b\right)}^2/2}\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)\mathrm{erfc}\left[\frac{x\left(R_2+a\right)}{\sqrt{2}}\right]}\\ \approx \frac{x}{4\sqrt{2\pi }}\mathrm{erfc}\left(\frac{ax}{\sqrt{2}}\right)e^{-x^2/2}{\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_2\mathrm{erfc}\left(\frac{\sqrt{n}{R}_2}{2\sqrt{2}}\right)}=\frac{x}{2\sqrt{n}\pi }\mathrm{erfc}\left(\frac{ax}{\sqrt{2}}\right)e^{-x^2/2}+O\left(1/n\right).\end{array}$$ (B11)

Since

$$\frac{x}{4\sqrt{2\pi }}\mathrm{erfc}\left(\frac{bx}{\sqrt{2}}\right){\displaystyle {\int }_0^{\mathrm{\infty }}d{R}_1\exp \left[-\frac{x^2}{2}{\left(R_1+a\right)}^2\right]\mathrm{erfc}\left(\frac{\sqrt{n}{R}_1}{2\sqrt{2}}\right)}\approx \frac{x}{2\sqrt{n}\pi }\mathrm{erfc}\left(\frac{bx}{\sqrt{2}}\right)e^{-x^2/2}+O\left(1/n\right),$$ (B12)

we conclude that $G_2\left(a,b\right)=O\left(1/n\right).$ Using $\mathrm{erfc}\left(y\right)+\mathrm{erfc}\left(-y\right)=2,$ we finally obtain

$$P_s\equiv 2{\displaystyle \sum _{i=3}^6\mathrm{Pr}\left(D_i\right)}\approx \frac{4x}{\sqrt{n}\pi }{e}^{-x^2/2}.$$ (B13)

Appendix C: Large L Behavior of ${\mathbf{ℛ}}_s\left(L\right)$ for Fixed n

In this appendix, we calculate the asymptotic behavior of the probability ${\mathrm{ℛ}}_s\left(L\right)$ for a genotype with s mutations to be a local fitness maximum in the limit where L is large and the phenotype dimensions n is fixed. As explained in the main text, this probability has two contributions which arise from expanding the function $F\left(\overrightarrow{k},\overrightarrow{z}\right)$ defined in Equation 30 near $\left|\overrightarrow{z}\right|=0$ and $\left|\overrightarrow{z}\right|=z\text{*}\sim L,$ respectively (see Equation 31).

First, we consider the contribution from the region $\left|\overrightarrow{z}\right|\ll 1.$ In this case, we can approximate $F\left(\overrightarrow{k},\overrightarrow{z}\right)$ as

$$\begin{array}{l}F\left(\overrightarrow{k},\overrightarrow{z}\right)={\displaystyle {\int }_n{e}^{-i\overrightarrow{k}\cdot \overrightarrow{\xi }}p\left(\overrightarrow{\xi }\right)d\overrightarrow{\xi }}-{\displaystyle {\int }_{{\mathcal{D}}^c\left(\overrightarrow{z}\right)}{e}^{-i\overrightarrow{k}\cdot \overrightarrow{\xi }}p\left(\overrightarrow{\xi }\right)d\overrightarrow{\xi }}\approx e^{-k^2/2}-{\displaystyle {\int }_{{\mathcal{D}}^c\left(\overrightarrow{z}\right)}p\left(0\right)d\overrightarrow{\xi }=e^{-k^2/2}-A_n{z}^n}\approx \\ \exp \left[-\frac{k^2}{2}-A_n{z}^n{e}^{k^2/2}\right],\end{array}$$ (C1)

where $k=\left|\overrightarrow{k}\right|;$ $z=\left|\overrightarrow{z}\right|;$ ${\mathcal{D}}^c(\overrightarrow{z})=\left\{\overrightarrow{y}|\left|\overrightarrow{y}-\overrightarrow{z}\right|\le z\right\},$ which is the complement of $\mathcal{D}\left(\overrightarrow{z}\right);$ $A_n=p\left(0\right)S_{n-1}/n$ with $S_{n-1}=2{\pi }^{n/2}/\text{Γ}\left(n/2\right)$ being the surface area of the unit sphere in $\left(n-1\right)$ dimensions; and $p\left(0\right)={\left(2\pi \right)}^{-n/2}.$ Note that the error of the above approximation is $O\left(z^{n+1}\right).$ Thus, setting $\rho \equiv s/L$ we can approximate

$${\mathrm{ℛ}}_s^{<}\left(L\right)\approx {\displaystyle {\int }_n\frac{d\overrightarrow{z}d\overrightarrow{k}}{{\left(2\pi \right)}^n}\text{exp}\left[i\overrightarrow{k}\cdot \overrightarrow{z}+L{H}_1\left(\overrightarrow{k},\overrightarrow{z}\right)\right]},\hspace{1em}{H}_1\left(\overrightarrow{k},\overrightarrow{z}\right)\equiv -i\overrightarrow{k}\cdot \overrightarrow{q}-\rho \frac{k^2}{2}-\rho A_n{z}^n{e}^{k^2/2}-\left(1-\rho \right)A_n{z}^n,$$ (C2)

where $\overrightarrow{q}=\overrightarrow{Q}/L.$ Since L is large, we can employ the saddle-point approximation. One can easily see that the saddle point solving the equations ${\partial }_{k_j}{H}_1={\partial }_{z_k}{H}_1=0$ is at $\overrightarrow{z}=0$ and $k_j=-i{q}_j/\rho .$ Around the saddle point, we expand

$$H_1\approx -\frac{q^2}{2\rho }-\frac{\rho }{2}{\left(\overrightarrow{k}+i\overrightarrow{q}/\rho \right)}^2-A_n{z}^n\left[\rho e^{-q^2/\left(2{\rho }^2\right)}+\left(1-\rho \right)\right],$$ (C3)

which gives

$$\begin{array}{c}{\mathrm{ℛ}}_s^{<}\left(L\right)\approx \exp \left(-L\frac{q^2}{2\rho }\right){\displaystyle {\int }_{n}d\overrightarrow{z}}\exp \left\{-L{A}_n{z}^n\left[\rho e^{-q^2/\left(2{\rho }^2\right)}+\left(1-\rho \right)\right]\right\}{\displaystyle {\int }_n\frac{d\overrightarrow{k}}{{\left(2\pi \right)}^n}\exp \left[-\frac{L\rho }{2}{\left(\overrightarrow{k}+i\overrightarrow{q}/\rho \right)}^2\right]}\\ =\frac{\exp \left(-L\frac{q^2}{2\rho }\right)}{{\left(2\pi L\rho \right)}^{n/2}}{\displaystyle {\int }_0^{\mathrm{\infty }}{S}_{n-1}{z}^{n-1}dz}\exp \left\{-L{A}_n{z}^n\left[\rho e^{-q^2/\left(2{\rho }^2\right)}+\left(1-\rho \right)\right]\right\}=\frac{s^{-n/2}\exp \left[-Q^2/\left(2s\right)\right]}{s\exp \left[-Q^2/\left(2s^2\right)\right]+L-s}\left[1+O\left(L^{-1/n}\right)\right],\end{array}$$ (C4)

and the last step involves a change of variables $z\to t=S_{n-1}{z}^n/n.$ Since L appears in the integrand in the combination $L{z}^n,$ the error that arises from neglecting terms of $O\left(z^{n+1}\right)$ is $L^{-1/n}.$ The leading order of Equation C4 was reported in Equation 34.

Now we move on to the calculation of ${\mathrm{ℛ}}_s^{>}\left(L\right),$ where the dominant contribution to $F\left(\overrightarrow{k},\overrightarrow{z}\right)$ comes from a region where $z\sim O\left(L\right).$ Using ${\displaystyle \int d\overrightarrow{\xi }p\left(\overrightarrow{\xi }\right)\exp \left(-i\overrightarrow{k}\cdot \overrightarrow{\xi }\right)}=\exp \left(-k^2/2\right),$ we calculate the integral $I\equiv \exp \left(-k^2/2\right)-F\left(\overrightarrow{k},\overrightarrow{z}\right)$ as

$$\begin{array}{c}I=\frac{1}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{2z}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}{\displaystyle {\int }_{B\left(2z,{\xi }_n\right)}d{\overrightarrow{\xi }}_{\perp }{e}^{-i{\overrightarrow{k}}_{\perp }\cdot {\overrightarrow{\xi }}_{\perp }-{\overrightarrow{\xi }}_{\perp }^2/2}}\\ =\frac{1}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{2z}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}\left[{\displaystyle {\int }_{{\mathrm{ℝ}}^{n-1}}d{\overrightarrow{\xi }}_{\perp }{e}^{-i{\overrightarrow{k}}_{\perp }\cdot {\overrightarrow{\xi }}_{\perp }-{\overrightarrow{\xi }}_{\perp }^2/2}}-{\displaystyle {\int }_{B^c\left(2z,{\xi }_n\right)}d{\overrightarrow{\xi }}_{\perp }{e}^{-i{\overrightarrow{k}}_{\perp }\cdot {\overrightarrow{\xi }}_{\perp }-{\overrightarrow{\xi }}_{\perp }^2/2}}\right]\\ =\frac{e^{-{\overrightarrow{k}}_{\perp }^2/2}}{\sqrt{2\pi }}\left[{\displaystyle {\int }_0^{\mathrm{\infty }}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}-{\displaystyle {\int }_{2z}^{\mathrm{\infty }}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}\right]-\frac{1}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{2z}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}{\displaystyle {\int }_{B^c\left(2z,{\xi }_n\right)}d{\overrightarrow{\xi }}_{\perp }{e}^{-i{\overrightarrow{k}}_{\perp }\cdot {\overrightarrow{\xi }}_{\perp }-{\overrightarrow{\xi }}_{\perp }^2/2}}\\ =\frac{e^{-k^2/2}}{2}\left[1-\mathrm{erf}\left(\frac{i{k}_n}{\sqrt{2}}\right)\right]-\frac{e^{-{\overrightarrow{k}}_{\perp }^2/2}}{\sqrt{2\pi }}{\displaystyle {\int }_{2z}^{\mathrm{\infty }}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}-C_1\left(\overrightarrow{k},z\right)},\end{array}$$ (C5)

where we set $\overrightarrow{z}=z{\overrightarrow{e}}_n,$ ${\overrightarrow{\xi }}_{\perp }=\overrightarrow{\xi }-{\xi }_n{\overrightarrow{e}}_n,$ and ${\overrightarrow{k}}_{\perp }=\overrightarrow{k}-k_n{\overrightarrow{e}}_n$ with ${\overrightarrow{e}}_n=\left(0,\dots ,0,1\right),$ $B\left(2z,{\xi }_n\right)$ is an (n − 1)-dimensional ball with radius $\sqrt{{\xi }_n\left(2z-{\xi }_n\right)}$ whose center is located at the origin, $B^c$ is the relative complement of B with respect to ${\mathrm{ℝ}}^{n-1},$ and $\mathrm{erf}\left(z\right)=2{\displaystyle {\int }_0^z{e}^{-t^2}dt/\sqrt{\pi }}$ is the error function. The definition of $C_1$ is self-explanatory. Since

$$\left|{\displaystyle {\int }_{2z}^{\mathrm{\infty }}d{\xi }_n{e}^{-i{k}_n{\xi }_n-{\xi }_n^2/2}}\right|\le {\displaystyle {\int }_{2z}^{\mathrm{\infty }}d{\xi }_n{e}^{-{\xi }_n^2/2}}=2z{\displaystyle {\int }_1^{\mathrm{\infty }}dy{e}^{-2z^2{y}^2}}\approx \frac{e^{-2z^2}}{2z},$$ (C6)

where we used the Laplace method for the asymptotic expansion, the leading finite z correction is expected to come from $C_1$ for $n>1.$ Note that $C_1$ is identically zero for $n=1.$ Thus we get

$$F\left(\overrightarrow{k},\overrightarrow{z}\right)\approx \frac{1}{2}{e}^{-k^2/2}\left[1+\mathrm{erf}\left(\frac{i\overrightarrow{k}\cdot \overrightarrow{z}}{\sqrt{2}z}\right)\right]+C_1\left(\overrightarrow{k},z\right),$$ (C7)

where $k_n$ is written as a projection of $\overrightarrow{k}$ along the $\overrightarrow{z}$ direction, $k_n=\overrightarrow{k}\cdot \overrightarrow{z}/z.$ Since

$$\left|C_1\left(\overrightarrow{k},z\right)\right|\le \frac{1}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{2z}d{\xi }_n{e}^{-{\xi }_n^2/2}}{\displaystyle {\int }_{B^c\left(2z,{\xi }_n\right)}d{\overrightarrow{\xi }}_{\perp }{e}^{-{\overrightarrow{\xi }}_{\perp }^2/2}}=C_1\left(0,z\right),$$ (C8)

it is sufficient to find an approximate formula for $C_1\left(0,z\right)$ to determine the z dependence of $C_1\left(\overrightarrow{k},z\right).$ Using spherical coordinates in ${\mathrm{ℝ}}^{n-1},$ we get

$$\begin{array}{c}{C}_1\left(0,z\right)=\frac{S_{n-2}}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{2z}dy}{e}^{-y^2/2}{\displaystyle {\int }_{\sqrt{y\left(2z-y\right)}}^{\mathrm{\infty }}dx}{x}^{n-2}{e}^{-x^2/2}\\ =\frac{S_{n-2}}{{\left(2\pi \right)}^{n/2}}\left\{{\displaystyle {\int }_0^{2z}dy}{e}^{-y^2/2}{\displaystyle {\int }_z^{\mathrm{\infty }}dx}{x}^{n-2}{e}^{-x^2/2}+{\displaystyle {\int }_0^{z}dx}{x}^{n-2}{e}^{-x^2/2}\left[{\displaystyle {\int }_0^{M\mathrm{\_}\left(x,z\right)}dy}{e}^{-y^2/2}+{\displaystyle {\int }_{M_{+}\left(x,z\right)}^{z}dy}{e}^{-y^2/2}\right]\right\},\end{array}$$ (C9)

where $S_{n-2}=2{\pi }^{\left(n-1\right)/2}/\Gamma \left[\left(n-1\right)/2\right]$ is the surface area of the unit $\left(n-2\right)$ sphere. In the second term on the second line, the order of integration was reversed and the integration boundaries $M_{\pm }\left(x,z\right)=z\pm \sqrt{z^2-x^2}$ were introduced. Since the first integral $\left({\displaystyle {\int }_z^{\mathrm{\infty }}dz}\right)$ and the third integral $\left({\displaystyle {\int }_{M_{+}}^{z}dy}\right)$ decrease exponentially with z, the main contribution to $C_1\left(0,z\right)$ comes from the second integral. Thus,

$$\begin{array}{c}{C}_1\left(0,z\right)\approx \frac{S_{n-2}}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{z}dx}{x}^{n-2}{e}^{-x^2/2}{\displaystyle {\int }_0^{M_{-}\left(x,z\right)}dy}{e}^{-y^2/2}=\frac{S_{n-2}{z}^n}{{\left(2\pi \right)}^{n/2}}{\displaystyle {\int }_0^{1}dx}{x}^{n-2}{e}^{-z^2{x}^2/2}{\displaystyle {\int }_0^{M_{-}\left(x,1\right)}dy}{e}^{-z^2{y}^2/2}\\ =\frac{S_{n-2}{z}^{n-1}}{{\left(2\pi \right)}^{n/2}}\sqrt{\frac{\pi }{2}}{\displaystyle {\int }_0^{1}dx}{x}^{n-2}{e}^{-z^2{x}^2/2}\text{erf}\left[M_{-}\left(x,1\right)z/\sqrt{2}\right].\end{array}$$ (C10)

Since the last integral is dominated by the region $xz\le 1,$ we can approximate $M_{-}\left(x,1\right)z\approx x^{2}z/2\sim O\left(1/z\right)$ and $\text{erf}\left[M_{-}\left(x,1\right)z/\sqrt{2}\right]=x^{2}z/\sqrt{2\pi }.$ Finally, we get

$$C_1\left(0,z\right)\approx \frac{S_{n-2}{z}^n}{{\left(2\pi \right)}^{n+1/2}}\sqrt{\frac{\pi }{2}}{\displaystyle {\int }_0^{1}dx}{x}^n{e}^{-z^2{x}^2/2}\approx \frac{S_{n-2}{z}^n}{{\left(2\pi \right)}^{n+1/2}}\sqrt{\frac{\pi }{2}}{\displaystyle {\int }_0^{\mathrm{\infty }}dx}{x}^n{e}^{-z^2{x}^2/2}=\frac{n-1}{2\sqrt{2\pi }z},$$ (C11)

which also implies that $C_1\left(\overrightarrow{k},z\right)\sim O\left(z^{-1}\right).$

If we write

$$F\left(\overrightarrow{k},\overrightarrow{z}\right)=\frac{1}{2}{e}^{-k^2/2}\left[1+\text{erf}\left(\frac{i\overrightarrow{k}\cdot \overrightarrow{r}}{\sqrt{2}r}\right)\right]\left[1+\frac{1}{L}f\left(\overrightarrow{r},\overrightarrow{k}\right)+O\left(z^{-2}\right)\right],$$ (C12)

where $\overrightarrow{r}=\overrightarrow{z}/L$ and $r=z/L,$ then comparison with Equation C7 and Equation C11 shows that $f\left(\overrightarrow{r},0\right)=\left(n-1\right)/\left(\sqrt{2\pi }r\right).$ Inserting Equation C12 into Equation 32, it follows that

$${\mathrm{ℛ}}_s^{>}\left(L\right)\approx \frac{L^n}{2^L}{\displaystyle \int \frac{d\overrightarrow{r}d\overrightarrow{k}}{{\left(2\pi \right)}^n}}\exp \left[\rho f\left(\overrightarrow{r},\overrightarrow{k}\right)+\left(1-\rho \right)f\left(\overrightarrow{r},0\right)\right]\exp \left[L{H}_2\left(\overrightarrow{k},\overrightarrow{r}\right)\right],$$ (C13)

with

$$H_2\left(\overrightarrow{k},\overrightarrow{r}\right)=i\overrightarrow{k}\cdot \left(\overrightarrow{r}-\overrightarrow{q}\right)-\rho \frac{k^2}{2}+\rho \ln \left[1+\text{erf}\left(\frac{i\overrightarrow{k}\cdot \overrightarrow{r}}{\sqrt{2}r}\right)\right].$$ (C14)

Now we employ the steepest-descent method. For convenience, we set $\overrightarrow{q}=\left(q,0,\dots ,0\right).$ The saddle point satisfies the equations

$$\frac{\partial H_2}{\partial r_j}=i{k}_j+i\frac{\sqrt{2}\rho }{\sqrt{\pi }{r}^3}\exp \left[\frac{{\left(\overrightarrow{k}\cdot \overrightarrow{r}\right)}^2}{2r^2}\right]\left[k_j{r}^2-r_j\left(\overrightarrow{k}\cdot \overrightarrow{r}\right)\right]{\left[1+\text{erf}\left(\frac{i\overrightarrow{k}\cdot \overrightarrow{r}}{\sqrt{2}r}\right)\right]}^{-1}=0,$$ (C15)

$$\frac{\partial H_2}{\partial k_j}=i\left(r_j-q{\delta }_{j1}\right)-\rho k_j+i\frac{\sqrt{2}\rho }{\sqrt{\pi }r}\exp \left[\frac{{\left(\overrightarrow{k}\cdot \overrightarrow{r}\right)}^2}{2r^2}\right]r_j{\left[1+\text{erf}\left(\frac{i\overrightarrow{k}\cdot \overrightarrow{r}}{\sqrt{2}r}\right)\right]}^{-1}=0,$$ (C16)

with the solution $\overrightarrow{k}*=0$ and $\overrightarrow{r}*=\left(q-\rho \sqrt{2/\pi },0,\dots ,0\right).$ Note that there is no solution if $q<\rho \sqrt{2/\pi },$ so the valid range of ρ has the upper boundary ${\rho }_c\left(q\right)=\min \left(1,\sqrt{\pi /2}q\right).$ The matrix of second derivatives around the saddle point $\overrightarrow{k}*,\overrightarrow{r}*$ is

$${\frac{{\partial }^2{H}_2}{\partial r_l\partial r_j}|}_{*}=0,\hspace{1em}{\frac{{\partial }^2{H}_2}{\partial k_m\partial k_j}|}_{*}=-\rho {\delta }_{mj}\left(1-\frac{2}{\pi }{\delta }_{l1}\right),\hspace{1em}{\frac{{\partial }^2{H}_2}{\partial r_m\partial k_j}|}_{*}=i{\delta }_{jl}\left[1+\left(1-{\delta }_{m1}\right)\frac{\sqrt{2}\rho }{\sqrt{\pi }q-\sqrt{2}\rho }\right].$$ (C17)

Thus, we get

$${\mathrm{ℛ}}_s^{>}\left(L\right)\approx \frac{L^n}{2^L}{e}^{f\left(\overrightarrow{r}*,0\right)}{\displaystyle \int \frac{d\overrightarrow{y}d\overrightarrow{k}}{{\left(2\pi \right)}^n}}\exp \left[-L\frac{\rho }{\pi }\left(\pi -2\right)k_1^2-L\rho {\overrightarrow{k}}_{\perp }^2+iL{k}_1{y}_1+iL\frac{\sqrt{\pi }q}{\sqrt{\pi }q-\sqrt{2}\rho }{\overrightarrow{k}}_{\perp }\cdot {\overrightarrow{y}}_{\perp }\right],$$ (C18)

where $\overrightarrow{y}=\overrightarrow{r}-\overrightarrow{r}*,$ ${\overrightarrow{y}}_{\perp }=\left(0,y_2,\dots ,y_n\right),$ and ${\overrightarrow{k}}_{\perp }=\left(0,k_2,\dots ,k_n\right).$ If we perform the integration over $\overrightarrow{y}$ first, we obtain δ functions which make the integral over $\overrightarrow{k}$ trivial. Finally, we arrive at

$${\mathrm{ℛ}}_s^{>}\left(L\right)\approx 2^{-L}\theta \left({\rho }_c-\rho \right){\left[\frac{q-2\rho /\sqrt{2\pi }}{q}\exp \left(\frac{1}{\sqrt{2\pi }q-2\rho }\right)\right]}^{n-1},$$ (C19)

where $\theta \left(x\right)$ is the Heaviside step function defined by $\theta \left(x\ge 0\right)=1$ and $\theta \left(x<0\right)=0.$

To evaluate the corresponding contribution to the number of fitness maxima, ${\mathcal{N}}_{>},$ we replace the summation over s in Equation 33 by an integral over $\rho =s/L$ and use Stirling’s formula to approximate the binomial coefficients. This yields

$${\mathcal{N}}_{>}=\sqrt{L}{\displaystyle {\int }_0^{1}d\rho }\frac{\exp \left\{L\left[-\rho \ln \rho -\left(1-\rho \right)\ln \left(1-\rho \right)-\ln 2\right]\right\}}{\sqrt{2\pi \rho \left(1-\rho \right)}}{\left[\frac{q-2\rho /\sqrt{2\pi }}{q}\exp \left(\frac{1}{\sqrt{2\pi }q-2\rho }\right)\right]}^{n-1}\theta \left({\rho }_c-\rho \right).$$ (C20)

If ${\rho }_c<1/2$ or $q<{\sqrt{2\pi }}^{-1}=q_0,$ the integral is dominated around $\rho \approx {\rho }_c,$ which results in an exponential decrease with L. On the other hand, if ${\rho }_c>1/2,$ the integral is dominated around $\rho \approx 1/2,$ which gives

$${\mathcal{N}}_{>}\approx \sqrt{\frac{2L}{\pi }}{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}dx}{e}^{-2L{x}^2}{\left[\frac{q-1/\sqrt{2\pi }}{q}\exp \left(\frac{1}{\sqrt{2\pi }q-1}\right)\right]}^{n-1}={\left[\frac{q-1/\sqrt{2\pi }}{q}\exp \left(\frac{1}{\sqrt{2\pi }q-1}\right)\right]}^{n-1}$$ (C21)

as reported in Equation 43.

Appendix D: Derivation of Equation 47

In this appendix, we calculate the average number of fitness maxima $\mathcal{N}$ in the limit $n,L\to \mathrm{\infty }$ at fixed ratio $\alpha \equiv n/L.$ To this end, we write $I_{\tau },$ the probability for the genotype τ to be a local fitness maximum, using the Heaviside step function as

$$I_{\tau }={\displaystyle \int {\displaystyle \prod _{k=1}^L\left[d{\overrightarrow{\xi }}_k\hspace{0.17em}p\left({\overrightarrow{\xi }}_k\right)\theta \left\{\frac{1}{L}{\left[\overrightarrow{z}+\left(1-2{\tau }_k\right){\overrightarrow{\xi }}_k\right]}^2-\frac{1}{L}{\left|\overrightarrow{z}\right|}^2\right\}\right]}}\equiv {\displaystyle \int \mathcal{D}\xi }{\displaystyle \prod _k\theta \left({\epsilon }_k\right)},$$ (D1)

where $\overrightarrow{z}$ is determined by τ through Equation 3, ${\displaystyle \int \mathcal{D}\xi }\equiv {\displaystyle \int {\displaystyle \prod _{k}d{\overrightarrow{\xi }}_{k}p\left({\overrightarrow{\xi }}_k\right)}},$ and ${\epsilon }_k$ is defined as

$${\epsilon }_k=\frac{1}{L}{\left[\overrightarrow{z}+{\overrightarrow{\xi }}_k\left(1-2{\tau }_k\right)\right]}^2-\frac{1}{L}{\left|\overrightarrow{z}\right|}^2=\frac{1}{L}\left[2\overrightarrow{z}\cdot {\overrightarrow{\xi }}_k\left(1-2{\tau }_k\right)+{\left|\overrightarrow{\xi }\right|}^2\right]=\frac{1}{L}\left[2\left(\overrightarrow{Q}+{\displaystyle \sum _j{\overrightarrow{\xi }}_j{\tau }_j}\right)\cdot {\overrightarrow{\xi }}_k\left(1-2{\tau }_k\right)+{\left|{\overrightarrow{\xi }}_k\right|}^2\right].$$ (D2)

Note that the prefactor $1/L$ is introduced to make ${\epsilon }_k$ finite in the limit $L\to \mathrm{\infty }$ and we have used that ${\left(1-2{\tau }_k\right)}^2=1.$ Applying the identity (Tanaka and Edwards 1980; Bray and Moore 1980)

$$\theta \left({\epsilon }_k\right)={\displaystyle {\int }_0^{\mathrm{\infty }}d{\lambda }_k}\delta \left({\lambda }_k-{\epsilon }_k\right)={\displaystyle {\int }_0^{\mathrm{\infty }}d{\lambda }_k}{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{d{\phi }_k}{2\pi }}\exp \left[i{\phi }_k\left({\lambda }_k-{\mathrm{\xi }}_k\right)\right]$$ (D3)

to Equation D1, the expected number of local fitness maxima reads

$$\begin{array}{c}\mathcal{N}={\displaystyle \sum }_{\tau }{{\displaystyle \int }}^{\text{}}\mathcal{D}\xi \stackrel{L}{{\displaystyle \prod }_{k=1}}\left[{{{\displaystyle \int }}^{\text{}}}_0^{\mathrm{\infty }}d{\lambda }_k{{{\displaystyle \int }}^{\text{}}}_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{d{\phi }_k}{2\pi }{e}^{i{\phi }_k\left({\lambda }_k-{\epsilon }_k\right)}\right]={\displaystyle \sum _{\tau }{\displaystyle \int \mathcal{D}\xi \mathcal{D}\lambda \mathcal{D}\phi }}\exp \left\{{\displaystyle \sum _{k=1}^L\left[i{\phi }_k{\lambda }_k+\frac{i}{L}{\phi }_k\left(2{\overrightarrow{\xi }}_k\cdot {\displaystyle \sum _{j=1}^L{\overrightarrow{\xi }}_j{\tau }_j}+2{\overrightarrow{\xi }}_k\cdot \overrightarrow{Q}-{\left|{\overrightarrow{\xi }}_k\right|}^2\right)\right]}\right\}\\ ={\displaystyle \sum _{\tau }{\displaystyle \int \mathcal{D}\xi \mathcal{D}\lambda \mathcal{D}\phi }}\exp \left\{{\displaystyle \sum _{k=1}^L\left[i{\phi }_k{\lambda }_k+\frac{i}{L}{\phi }_k\left(2{\overrightarrow{\xi }}_k\cdot \overrightarrow{Q}-{\left|{\overrightarrow{\xi }}_k\right|}^2\right)\right]}\right\}\exp \left[\frac{i}{L}{\displaystyle \sum _{k=1}^L{\phi }_k{\overrightarrow{\xi }}_k}\cdot {\displaystyle \sum _{j=1}^L{\overrightarrow{\xi }}_j\left(2{\tau }_j\right)}\right],\end{array}$$ (D4)

where ${\displaystyle \int \mathcal{D}\lambda }\equiv {\displaystyle {\int }_0^{\mathrm{\infty }}{\displaystyle {\prod }_{k}d{\lambda }_k}},$ ${\displaystyle \int \mathcal{D}\phi }\equiv {\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\displaystyle {\prod }_{k}d{\phi }_k/2\pi }},$ and we made the change of variables $\left(2{\tau }_k-1\right){\overrightarrow{\xi }}_k\mapsto {\overrightarrow{\xi }}_k$ to arrive at the second equality. Using the identity

$$\exp \left(\frac{i}{L}\overrightarrow{X}\cdot \overrightarrow{Y}\right)=L^n{\displaystyle {\int }_{n}d\overrightarrow{\nu }}\delta \left(L\overrightarrow{\nu }-\overrightarrow{X}\right)\exp \left(i\overrightarrow{Y}\cdot \overrightarrow{\nu }\right)={\left(\frac{L}{2\pi }\right)}^n{\displaystyle {\int }_{n}d\overrightarrow{\mu }d\overrightarrow{\nu }}\exp \left[iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }-i\overrightarrow{X}\cdot \overrightarrow{\mu }+i\overrightarrow{Y}\cdot \overrightarrow{\nu }\right],$$ (D5)

which is valid for any n-dimensional real vectors $\overrightarrow{X}$ and $\overrightarrow{Y},$ we can write the last term of Equation D4 as

$$\exp \left[\frac{i}{L}{\displaystyle \sum _k{\phi }_k{\overrightarrow{\xi }}_k}\cdot {\displaystyle \sum _j{\overrightarrow{\xi }}_j\left(2{\tau }_j\right)}\right]={\left(\frac{L}{2\pi }\right)}^n{\displaystyle {\int }_{n}d\overrightarrow{\mu }d\overrightarrow{\nu }}\exp \left[iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }+i{\displaystyle \sum _k{\overrightarrow{\xi }}_k}\cdot \left(-{\phi }_k\overrightarrow{\mu }+2{\tau }_k\overrightarrow{\nu }\right)\right],$$ (D6)

which gives

$$\mathcal{N}={\displaystyle \sum _{\tau }{\displaystyle \int \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }}{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}{\displaystyle \prod _{k=1}^L{\displaystyle \prod _{s=1}^n{\displaystyle \int \frac{d{\xi }_{ks}}{\sqrt{2\pi }}}\exp \left\{i\left[\frac{1}{L}{\phi }_k\left(2{\xi }_{ks}{Q}_s-{\xi }_{ks}^2\right)+{\xi }_{ks}\left(-{\phi }_k{\mu }_s+2{\tau }_k{\nu }_s\right)\right]-\frac{{\xi }_{ks}^2}{2}\right\}}},$$ (D7)

where ${\displaystyle \int \mathcal{D}\mu \mathcal{D}\nu }\equiv {\left(\frac{L}{2\pi }\right)}^n{\displaystyle {\int }_{n}d\overrightarrow{\mu }d\overrightarrow{\nu }},$ $\phi \cdot \lambda \equiv {\displaystyle {\sum }_{k=1}^L{\phi }_k{\lambda }_k},$ and ${\xi }_{ks},$ $Q_s,$ ${\mu }_s,$ ${\nu }_s$ are the sth components of the vectors ${\overrightarrow{\xi }}_k,$ $\overrightarrow{Q},$ $\overrightarrow{\mu },$ and $\overrightarrow{\nu },$ respectively. Note that the integrals over the ${\xi }_{ks}’\text{s}$ become independent of each other. If we choose $Q_s=Q/\sqrt{n}=q\sqrt{L/\alpha }$ for all s and define $\chi \equiv 2Q_s/L,$ the integral over ${\xi }_{ks}$ becomes

$$\begin{array}{l}{\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{d{\xi }_{ks}}{\sqrt{2\pi }}}\exp \left\{-\frac{{\xi }_{ks}^2}{2}\left(1+2i{\phi }_k/L\right)+i{\xi }_{ks}\left[{\phi }_k\left(\chi -{\mu }_s\right)+2{\tau }_k{\nu }_s\right]\right\}=\frac{1}{\sqrt{1+2i{\phi }_k/L}}\exp \left\{-\frac{{\left[2{\nu }_s{\tau }_k+\left(\chi -{\mu }_s\right){\phi }_k\right]}^2}{2\left(1+2i{\phi }_k/L\right)}\right\}\\ =\frac{1}{\sqrt{1+2i{\phi }_k/L}}\exp \left[\frac{-{\left({\mu }_s-\chi \right)}^2{\phi }_k^2+4{\tau }_k\left({\mu }_s-\chi \right){\nu }_s{\phi }_k-4{\nu }_s^2{\tau }_k}{2\left(1+2i{\phi }_k/L\right)}\right],\end{array}$$ (D8)

which, in turn, gives

$$\mathcal{N}={\displaystyle \sum _{\tau }{\displaystyle \int \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }}{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}{\displaystyle \prod _{k=1}^L\frac{1}{{\left(1+2i{\phi }_k/L\right)}^{n/2}}\exp \left[\frac{-{\phi }_k^2{\displaystyle {\sum }_s{\left({\mu }_s-\chi \right)}^2}+4{\tau }_k{\phi }_k{\displaystyle {\sum }_s\left({\mu }_s-\chi \right){\nu }_s}-4{\tau }_k{\displaystyle {\sum }_s{\nu }_s^2}}{2\left(1+2i{\phi }_k/L\right)}\right]}.$$ (D9)

If we now insert the identity

$$\begin{array}{c}1={\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int dbdc}\delta \left(a-{\displaystyle \sum _s{\left({\mu }_s-\chi \right)}^2}\right)\delta \left(b-{\displaystyle \sum _s\left({\mu }_s-\chi \right){\nu }_s}\right)\delta \left(c-{\displaystyle \sum _s{\nu }_s^2}\right)\\ ={\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int dbdc}{\displaystyle \int \frac{dA}{2\pi /L}\frac{dB}{2\pi /L}\frac{dC}{2\pi /L}}\exp \left[iAL\left\{a-{\displaystyle \sum _s{\left({\mu }_s-\chi \right)}^2}\right\}+iBL\left\{b-{\displaystyle \sum _s\left({\mu }_s-\chi \right){\nu }_s}\right\}+iCL\left\{c-{\displaystyle \sum _s{\nu }_s^2}\right\}\right],\end{array}$$ (D10)

we can write

$$\begin{array}{c}\mathcal{N}={\displaystyle \sum _{\tau }{\displaystyle \int \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }}{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}{\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int \frac{dAdbdBdcdC}{{\left(2\pi /L\right)}^3}}{\displaystyle \prod _k{\left(1+2i{\phi }_k/L\right)}^{-L\alpha /2}}\exp \left[\frac{-a{\phi }_k^2+4b{\tau }_k{\phi }_k-4c{\tau }_k}{2\left(1+2i{\phi }_k/L\right)}\right]\\ \times \exp \left\{iAL\left[a-{\displaystyle \sum _s{\left({\mu }_s-\chi \right)}^2}\right]+iBL\left[b-{\displaystyle \sum _s\left({\mu }_s-\chi \right){\nu }_s}\right]+iCL\pi \left(c-{\displaystyle \sum _s{\nu }_s^2}\right)\right\},\end{array}$$ (D11)

where we have replaced n by $L\alpha .$ The integral domain of a is restricted to the positive real axis to ensure that the integral with respect to ${\phi }_k$ in Equation D9 continues to be well-defined after the substitution. Performing the integrals over ${\mu }_s$ and ${\nu }_s,$ we get

$$\frac{L}{2\pi }{\displaystyle \int d{\mu }_{s}d{\nu }_s}{e}^{iL{\mu }_s{\nu }_s-iLA{\left({\mu }_s-r\right)}^2-iLB\left({\mu }_s-r\right){\nu }_s-iLC{\nu }_s^2}=\exp \left[-\frac{1}{2}(\mathrm{Ln}\left\{\frac{{\left(B-1\right)}^2-4AC}{Ai}\right\}+\mathrm{Ln}\left(Ai\right))+\frac{i4q^{2}A/\alpha }{4AC-{\left(B-1\right)}^2}\right],$$ (D12)

where $\text{Ln}\left(x\right)$ is the principal value of the logarithm with argument in the interval $\left(-\pi ,\pi \right]$ and the branch cut lies on the negative real axis.

Subsequently, the remaining integral over ${\phi }_i$ and ${\lambda }_i$ can be readily evaluated as follows:

$$\frac{1}{2\pi }{\displaystyle {\int }_0^{\mathrm{\infty }}d{\lambda }_k}{\displaystyle \int d{\phi }_k{\left(1+2i{\phi }_k/L\right)}^{-L\alpha /2}}\exp \left[-\frac{a{\phi }_k^2+4b{\tau }_k{\phi }_k-4c{\tau }_k}{2\left(1+2i{\phi }_k/L\right)}+i{\phi }_k{\lambda }_k\right]=T\left(a,bi,c,{\tau }_k\right)+\frac{1}{L}U\left(a,bi,c,{\tau }_k\right)+O\left(1/L^2\right),$$ (D13)

where

$$\begin{array}{l}T\left(a,b,c,\tau \right)=\frac{1}{2}{e}^{-2c\tau }\left[\text{erf}\left(\frac{\alpha +2b\tau }{\sqrt{2a}}\right)+1\right],\\ \mathcal{U}\left(a,b,c,\tau \right)=-\frac{4ac\tau +a+2b\tau \left(\alpha +2b\tau \right)}{\sqrt{2\pi }{a}^{3/2}}\exp \left[-\frac{{\left(\alpha +2b\tau \right)}^2}{2a}-2c\tau \right].\end{array}$$ (D14)

After summing over the ${\tau }_k’\text{s},$ we arrive at the equation

$$\mathcal{N}={\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int \frac{dAdbdBdcdC}{{\left(2\pi /L\right)}^3}}\exp \left[\frac{\mathit{U}\left(a,bi,c,1\right)+\mathit{u}\left(a,bi,c,0\right)}{T\left(a,bi,c,1\right)+T\left(a,bi,c,0\right)}\right]\exp \left[L\Sigma \left(a,bi,c,Ai,B,Ci\right)\right],$$ (D15)

where

$$\Sigma \left(a,b,c,A,B,C\right)=aA+bB+cC-\frac{1}{2}\alpha \ln \left[4AC+{\left(B-1\right)}^2\right]-\frac{4A{q}^2}{4AC+{\left(B-1\right)}^2}+\ln \left[T\left(a,b,c,1\right)+T\left(a,b,c,0\right)\right].$$ (D16)

The remaining integrals are hard to evaluate analytically. Instead, we resort to the saddle-point method to obtain an asymptotic expansion of the integral. Since $\Sigma$ is the exponential growth factor of the number of local maxima which must be a real number, one expects that the saddle points of Equation D16 are formed for the real arguments of $\Sigma \text{.}$ This suggests that we should make the changes of variables $b\to b/i,$ $A\to A/i,$ and $C\to C/i.$ For large L, the integrals are then dominated by the saddle point $\left(a*,b*,c*,A*,B*,C*\right)$ of $\Sigma \left(a,b,c,A,B,C\right).$ If there is more than one saddle point, the one giving the largest value of $\Sigma \left(a,b,c,A,B,C\right)$ has to be chosen. Then, the leading behavior of the number of maxima can be expressed in terms of the saddle point as

$$\mathcal{N}=\frac{1}{\sqrt{\left|\det H(\Sigma )\right|}}\exp \left[\frac{U\left(a*,b*,c*,1\right)+\mathit{U}\left(a*,b*,c*,0\right)}{T\left(a*,b*,c*,1\right)+T\left(a*,b*,c*,0\right)}\right]\exp \left[L\Sigma \left(a*,b*,c*,A*,B*,C*\right)\right],$$ (D17)

where $H(\Sigma )$ is the Hessian matrix around the saddle point. The reader may have noticed that the two principal values of the logarithm defined in Equation D12 are replaced by a real-valued logarithm in Equation D16, which can be dangerous in general. However, it can be shown that this substitution is indeed correct by verifying that ${\left(B*-1\right)}^2+4A*C*$ is always positive for all saddle points of Equation D16, and thus the imaginary arguments always cancel each other out.

Now, let us evaluate the saddle-point conditions. The derivatives of $\Sigma$ with respect to A, B, and C are

$$\begin{array}{l}\frac{\partial \Sigma }{\partial A}=a-2\frac{\alpha C\left[4AC+{\left(B-1\right)}^2\right]+2q^2{\left(B-1\right)}^2}{{\left[4AC+{\left(B-1\right)}^2\right]}^2},\hspace{1em}\frac{\partial \Sigma }{\partial B}=b-\frac{\left(B-1\right)\left[\alpha {\left(B-1\right)}^2+A\left(4C\alpha -8q^2\right)\right]}{{\left[4AC+{\left(B-1\right)}^2\right]}^2},\\ \frac{\partial \Sigma }{\partial C}=c-\frac{2A\left[\alpha {\left(B-1\right)}^2+A\left(4C\alpha -8q^2\right)\right]}{{\left[4AC+{\left(B-1\right)}^2\right]}^2}.\end{array}$$ (D18)

By requiring that the above three equations are zero at the saddle point, we get

$$A=\frac{\alpha c}{2\left(ac+b^2\right)},\hspace{1em}B-1=\frac{\alpha b}{ac+b^2},\hspace{1em}C=\frac{1}{4}\left(\frac{2a\alpha }{ac+b^2}+\frac{-\alpha \pm \sqrt{{\alpha }^2-16c{q}^2}}{c}\right).$$ (D19)

The two solutions of C force us to perform a two-fold analysis for the remaining integrals since we cannot a priori determine which solution will yield the correct saddle point. Instead, we introduce another real number $g=\pm \sqrt{{\alpha }^2-16c{q}^2}$ which is allowed to take both signs. Then, by imposing the functional relation $c\left(g\right)=({\alpha }^2-g^2)/\left(16q^2\right),$ both solutions are covered by a single analysis. In this way, the saddle point is obtained in terms of g instead of c. Finally, substituting this solution into Equation D17 gives Equation 47.

Appendix E: Mean Phenotypic Distance $z*$ in the Joint Limit

In this appendix, we will associate the saddle-point value $a*$ of the variable a entering the complexity function Equation 47 with the mean phenotypic distance $z*.$ To this end, we first consider the probability density $P\left(\tau ,a\right)$ that a genotype τ whose phenotypic vector is of squared magnitude $L^{2}a/4$ is a local maximum. Formally, we can write

$$\begin{array}{c}P\left(\tau ,a\right)={\displaystyle \int D\xi }{\displaystyle \prod _k\theta \left({\epsilon }_k\right)\delta \left[a-\frac{4}{L^2}{\left(\overrightarrow{Q}+{\displaystyle \sum _k{\overrightarrow{\xi }}_k{\tau }_k}\right)}^2\right]}={\displaystyle \int \frac{L}{2\pi }dA}{\displaystyle \int D\xi }{\displaystyle \prod _k\theta \left({\epsilon }_k\right)}\exp \left[iLaA-i\frac{4A}{L}{\left(\overrightarrow{Q}+{\displaystyle \sum _k{\overrightarrow{\xi }}_k{\tau }_k}\right)}^2\right]\\ =\frac{L}{2\pi }{\displaystyle \int dA}{\displaystyle \int D\xi }{\displaystyle \prod _k\theta \left({\epsilon }_k\right)}{\displaystyle \int \mathcal{D}\psi }\exp \left[iLaA+i\frac{L}{16A}{\overrightarrow{\psi }}^2+i\overrightarrow{\psi }\cdot \left(\overrightarrow{Q}+{\displaystyle \sum _k{\overrightarrow{\xi }}_k{\tau }_k}\right)\right],\end{array}$$ (E1)

where we have used the identity ${\displaystyle {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}dx}\exp \left(ip{x}^2+iqx\right)=\sqrt{\pi /p}\hspace{0.17em}{e}^{-i\pi /4}\exp \left[-i{q}^2/\left(4p\right)\right]$ for $p>0,$ $\mathcal{D}\psi ={\displaystyle {\prod }_s\left(\sqrt{L}{e}^{i\pi /4}d{\psi }_s\right)/\left(4\sqrt{\pi A}\right)},$ and the notation is the same as in Appendix D. Following the same procedure in the previous appendix, we get

$$\begin{array}{c}P\left(\tau ,a\right)={\displaystyle \int \mathcal{D}\xi \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}\exp \left[{\displaystyle \sum _k\frac{i}{L}{\phi }_k\left(2{\overrightarrow{\xi }}_k\cdot \overrightarrow{Q}-{\left|{\overrightarrow{\xi }}_k\right|}^2\right)}+i{\displaystyle \sum _k{\overrightarrow{\xi }}_k\cdot \left(-{\phi }_k\overrightarrow{\mu }+2{\tau }_k\overrightarrow{\nu }\right)}\right]\\ \times \frac{L}{2\pi }{\displaystyle \int dA}{\displaystyle \int \mathcal{D}\psi }\exp \left[iLaA+i\frac{L}{16A}{\overrightarrow{\psi }}^2+i\overrightarrow{\psi }\cdot \left(\overrightarrow{Q}+{\displaystyle \sum _i{\overrightarrow{\xi }}_i{\tau }_i}\right)\right].\end{array}$$ (E2)

By shifting $\overrightarrow{\nu }\to \overrightarrow{\nu }-\overrightarrow{\psi }/2$ and integrating over $\overrightarrow{\psi },$ we have

$$\begin{array}{c}P\left(\tau ,a\right)={\displaystyle \int \mathcal{D}\xi \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}\exp \left[{\displaystyle \sum _k\frac{i}{L}{\phi }_k\left(2{\overrightarrow{\xi }}_k\cdot \overrightarrow{Q}-{\left|{\overrightarrow{\xi }}_k\right|}^2\right)}+i{\displaystyle \sum _k{\overrightarrow{\xi }}_k}\cdot \left(-{\phi }_k\overrightarrow{\mu }+2{\tau }_k\overrightarrow{\nu }\right)\right]\\ \times \frac{L}{2\pi }{\displaystyle \int dA}{\displaystyle \int \mathcal{D}\psi }\exp \left[iLaA+i\frac{L}{16A}{\overrightarrow{\psi }}^2+i\overrightarrow{\psi }\cdot \left(\overrightarrow{Q}-L\overrightarrow{\mu }/2\right)\right]\\ ={\displaystyle \int \mathcal{D}\xi \mathcal{D}\lambda \mathcal{D}\phi \mathcal{D}\mu \mathcal{D}\nu }{e}^{i\phi \cdot \lambda +iL\overrightarrow{\mu }\cdot \overrightarrow{\nu }}\exp \left[{\displaystyle \sum _k\frac{i}{L}{\phi }_k\left(2{\overrightarrow{\xi }}_k\cdot \overrightarrow{Q}-{\left|{\overrightarrow{\xi }}_k\right|}^2\right)}+i{\displaystyle \sum _k{\overrightarrow{\xi }}_k}\cdot \left(-{\phi }_k\overrightarrow{\mu }+2{\tau }_k\overrightarrow{\nu }\right)\right]\\ \times {\displaystyle \int \frac{dA}{2\pi /L}}\exp \left\{iLA\left[a-{\left(\frac{2\overrightarrow{Q}}{L}-\overrightarrow{\mu }\right)}^2\right]\right\}.\end{array}$$ (E3)

Since we have set $Q_s=Q/\sqrt{n}$ for $s=1,\dots ,n,$ the last integral becomes

$${\displaystyle \int \frac{dA}{2\pi /L}}\exp \left\{iLA\left[a-{\left(\frac{2\overrightarrow{Q}}{L}-\overrightarrow{\mu }\right)}^2\right]\right\}={\displaystyle \int \frac{dA}{2\pi /L}}\exp \left\{iLA\left[a-{\displaystyle \sum _s{\left({\mu }_s-\chi \right)}^2}\right]\right\}.$$ (E4)

Since $\mathcal{N}={\displaystyle {\sum }_{\tau }{\displaystyle {\int }_0^{\mathrm{\infty }}da}P\left(\tau ,a\right)},$ by applying the manipulations of Appendix D to Equation E3 we arrive at the same integral form as in Equation D10. Since ${\displaystyle {\sum }_{\tau }P\left(\tau ,a\right)}$ is the mean number of local maxima whose phenotypic vectors have squared magnitude a, we see that the saddle point $a*$ of Equation D16 determines the mean phenotypic distance $z*$ through

$$z*=L\frac{\sqrt{a*}}{2}.$$ (E5)

This shows in particular that $z*$ is linear in L.

Appendix F: Mean Genotypic Distance $\mathit{\rho }*$ in the Joint Limit

To have access to the information about the typical value of the genotypic (Hamming) distance of a local fitness maximum from the wild type, we rewrite Equation D15 as

$$\begin{array}{c}\mathcal{N}={\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int \frac{dAdbdBdcdC}{{\left(2\pi /L\right)}^3}}{\displaystyle \sum _{s=0}^L\left(\begin{array}{c}L\\ s\end{array}\right){\left[T\left(a,b,c,1\right)+\frac{U\left(a,b,c,1\right)}{L}\right]}^s{\left[T\left(a,b,c,0\right)+\frac{U\left(a,b,c,0\right)}{L}\right]}^{L-s}}\\ \approx {\displaystyle {\int }_0^{\mathrm{\infty }}da}{\displaystyle \int \frac{dAdbdBdcdC}{{\left(2\pi /L\right)}^3}}{\displaystyle {\int }_0^{1}d\rho }\frac{e^{L\Sigma \left(a,b,c,A,B,C,\rho \right)}}{\sqrt{2\pi L\rho \left(1-\rho \right)}}\exp \left\{\rho \left[1+\frac{U\left(a,b,c,1\right)}{T\left(a,b,c,1\right)}\right]+\left(1-\rho \right)\left[1+\frac{U\left(a,b,c,0\right)}{T\left(a,b,c,0\right)}\right]\right\},\end{array}$$ (F1)

where we have rearranged the summation ${\Sigma }_{\tau }$ as ${\displaystyle {\sum }_{s=0}^L\left(\begin{array}{c}L\\ s\end{array}\right)}.$ taking advantage of the inherent permutation symmetry, Stirling’s formula has been used to evaluate the binomial coefficients, ${\Sigma }_s$ is approximated as $L{\displaystyle {\int }_0^{1}d\rho }$ with $s=L\rho ,$ and

$$\begin{array}{c}\Sigma \left(a,b,c,A,B,C,\rho \right)\equiv aA+bB+cC-\frac{1}{2}\alpha \ln \left[4AC+{\left(B-1\right)}^2\right]-\frac{4A{q}^2}{4AC+{\left(B-1\right)}^2}\\ +\rho \ln T\left(a,b,c,1\right)+\left(1-\rho \right)\ln T\left(a,b,c,0\right)-\rho \ln \rho -\left(1-\rho \right)\ln \left(1-\rho \right).\end{array}$$ (F2)

The saddle-point equations for this expression involve seven variables including ρ. Since the saddle-point equations for A, B, and C are the same as Equation D18, we may again insert Equation D19 into Equation F2, which yields

$$\begin{array}{c}{\Sigma }^{\text{red}}\left(a,b,g,\rho \right)=-\ln 2+\frac{\alpha }{2}\left(1-\ln \frac{\alpha }{2}\right)-\frac{\alpha }{2}\ln \left[\frac{\alpha +g}{ac\left(g\right)+b^2}\right]+b+\frac{g}{2}-\rho \ln \rho -\left(1-\rho \right)\ln \left(1-\rho \right)\\ +\left(1-\rho \right)\ln \left(\text{erf}\left(\frac{\alpha }{\sqrt{2a}}\right)+1\right)+\rho \left\{\ln \left[\text{erf}\left(\frac{\alpha +2b}{\sqrt{2a}}\right)+1\right]-2c\left(g\right)\right\}.\end{array}$$ (F3)

Since

$$\frac{\partial \Sigma }{\partial \rho }=\ln \frac{T\left(a,b,c,1\right)}{T\left(a,b,c,0\right)}-\ln \rho +\ln \left(1-\rho \right),$$ (F4)

the saddle-point value of $\rho *$ is

$$\rho *=\frac{T\left(a*,b*,c*,1\right)}{T\left(a*,b*,c*,1\right)+T\left(a*,b*,c*,0\right)}={\left\{1+e^{2c*}\left[\frac{\text{erf}\left(\alpha /\sqrt{2a*}\right)+1}{\text{erf}\left[\left(\alpha +2b*\right)/\sqrt{2a*}\right]+1}\right]\right\}}^{-1}.$$ (F5)

By inserting $\rho *$ into the saddle-point equations for $a,b,c,$ one can easily see that the final equations are the same as those derived from Equation D16.

Appendix G: Derivation of Equation 50

The determination of the solution describing regime III relies on the intuition that as α becomes large, the fitness landscape is asymptotically linear with the wild type being the global fitness maximum, as demonstrated in Sign epistasis for $L=2.$ This suggests an ansatz where $a*$ is close to $4q^2,$ which corresponds to the wild-type phenotypic distance as shown in Equation E5. Given this clue, one can additionally find that

$$\frac{\partial }{\partial a}{\Sigma }^{\text{red}}\left(a,b,g\right)=0$$ (G1)

is solved by $a=4q^2,$ $b=-\alpha ,$ and $g=\alpha .$ Furthermore, if we evaluate the remaining saddle-point conditions around this point, we find that this solution fails to solve them by a slight margin,

$${\frac{\partial }{\partial b}{\Sigma }^{\text{red}}\left(a,b,g\right)|}_{a=4q^2,\hspace{0.17em}b=-\alpha ,\hspace{0.17em}g=\alpha }=\frac{e^{-\frac{{\alpha }^2}{8q^2}}}{\sqrt{2\pi }q}$$ (G2)

and

$${\frac{\partial }{\partial c}{\Sigma }^{\text{red}}\left(a,b,g\right)|}_{a=4q^2,\hspace{0.17em}b=-\alpha ,\hspace{0.17em}g=\alpha }=\frac{\alpha }{8q^2}\text{erfc}\left(\frac{\alpha }{2\sqrt{2}q}\right).$$ (G3)

Given the fact that $\text{erfc}\left(x\right)=e^{-x^2}\left[1/\left(\sqrt{\pi }x\right)+O\left(x^{-3}\right)\right],$ these nonvanishing terms are seen to be of the order of $\epsilon =e^{-{\alpha }^2/\left(8q^2\right)}.$ Hence, it is sufficient to consider an expansion around the zeroth-order solution of the form ${\Sigma }^{\text{red}}\left(4q^2+A_1\mathit{ϵ},-\alpha +A_2\mathit{ϵ},\alpha +A_3\mathit{ϵ}\right)$ to show that Equation 50 satisfies the saddle-point conditions of Equation 48. To this end, we first focus on the derivatives with respect to $A_1$ and $A_2,$

$$\begin{array}{l}\frac{1}{\mathit{ϵ}}\frac{\partial }{\partial A_1}{\Sigma }^{\text{red}}\left(4q^2+A_1\mathit{ϵ},-\alpha +A_2\mathit{ϵ},\alpha +A_3\mathit{ϵ}\right)=-\frac{A_3\mathit{ϵ}}{16q^2}+O\left({\mathit{ϵ}}^2\right),\\ \frac{1}{\mathit{ϵ}}\frac{\partial }{\partial A_2}{\Sigma }^{\text{red}}\left(4q^2+A_1\mathit{ϵ},-\alpha +A_2\mathit{ϵ},\alpha +A_3\mathit{ϵ}\right)=\left(\frac{1}{\sqrt{2\pi }q}-\frac{2A_2+A_3}{2\alpha }\right)\mathit{ϵ}+O\left({\mathit{ϵ}}^2\right).\end{array}$$ (G4)

The vanishing contributions in ε imply that the zeroth-order solution $\left(4q^2,-\alpha ,\alpha \right)$ satisfies the first two saddle-point conditions. Additionally, we find that the corrections of the order $O\left(\mathit{ϵ}\right)$ are $A_3=0$ and $A_2=\alpha /(\sqrt{2\pi }q).$ Since $A_3=0$ to leading order, the saddle-point equation with respect to g should be evaluated to order $O\left({\mathit{ϵ}}^2\right).$ This yields

$$\frac{1}{{\mathit{ϵ}}^2}\frac{\partial }{\partial B_3}{\Sigma }^{\text{red}}\left(4q^2+A_1\mathit{ϵ},-\alpha +A_2\mathit{ϵ},\alpha +B_3{\mathit{ϵ}}^2\right)=\left[-\frac{A_1}{16q^2}-\frac{\sqrt{\frac{2}{\pi }}q}{{\alpha }^2}+O\left(\frac{q^3}{{\alpha }^4}\right)\right]\mathit{ϵ}+O\left({\mathit{ϵ}}^2\right),$$ (G5)

and subsequently, $A_1$ is solved to be $A_1=\left[\left(16\sqrt{2/\pi }{q}^3/{\alpha }^2\right)+O\left(q^4/{\alpha }^3\right)\right]\mathit{ϵ}+O\left({\mathit{ϵ}}^2\right).$ Finally, by inserting the solutions $A_1,$ $A_2,$ and $A_3$ as well as the zeroth-order solutions into Equation F5, the solution for $\rho *$ is found to be

$$\rho *=\left[\frac{\sqrt{\frac{2}{\pi }}q\mathit{ϵ}}{\alpha }+O\left(\frac{q^3}{{\alpha }^3}\right)\right]+O\left({\mathit{ϵ}}^2\right).$$ (G6)