Unbiased inference of the fitness landscape ruggedness from imprecise fitness estimates

Abstract

Fitness landscapes map genotypes to their corresponding fitness under given environments and allow explaining and predicting evolutionary trajectories. Of particular interest is the landscape ruggedness or the unevenness of the landscape, because it impacts many aspects of evolution such as the likelihood that a population is trapped in a local fitness peak. Although the ruggedness has been inferred from a number of empirically mapped fitness landscapes, it is unclear to what extent this inference is affected by fitness estimation error, which is inevitable in the experimental determination of fitness landscapes. Here we address this question by simulating fitness landscapes under various theoretical models, with or without fitness estimation error. We find that all eight examined measures of landscape ruggedness are overestimated due to imprecise fitness quantification, but different measures are affected to different degrees. We devise a method to use replicate fitness measures to correct this bias and show that our method performs well under realistic conditions. We conclude that previously reported fitness landscape ruggedness is likely upward biased owing to the negligence of fitness estimation error and advise that future fitness landscape mapping should include at least three biological replicates to permit an unbiased inference of the ruggedness.

INTRODUCTION

A fitness landscape maps individual genotypes to their corresponding fitness in a given environment. If the genotypes considered contain N variable sites or loci considered, the genotype space has N dimensions. As a result, the fitness landscape has N+1 dimensions, with the first N dimensions representing the genotype space while the last dimension representing the fitness. However, due to the difficulty in envisioning and drawing high-dimensional figures, fitness landscapes are commonly depicted as three-dimensional figures, where the X–Y plane represents the genotype space while the Z-axis shows the fitness.

Fitness landscapes largely determine and hence allow explaining and predicting evolutionary trajectories. One of the key characteristics of a fitness landscape is its ruggedness, which is a measure of the complexity of the landscape due to epistasis (i.e., genetic interaction), or simply put, a quantification of the unevenness of the landscape. Compared with a population in a smooth fitness landscape, an adapting population in a rugged landscape tends to be trapped in a local fitness peak instead of reaching the global optimum. Consequently, in the rugged landscape, giving the starting genotype, it is more difficult to predict the population’s final genotype at the end of an adaptive walk, but given both the starting and final genotypes, it is easier to infer the path of the adaptive walk due to the existence of fewer paths in which no step leads to a reduction in fitness. Adaptive radiation is more likely to occur in rugged fitness landscapes than in smooth landscapes, because initially identical populations are more likely to end up at different local fitness peaks in the former than the latter landscapes (Martin & Wainwright, 2013; Pfaender et al., 2016). Therefore, mapping fitness landscapes and quantifying their ruggedness are important for understanding evolution.

Thanks in part to the rapidly advancing DNA sequencing technology, considerable fitness landscape data have been collected in the last decade. These data have allowed describing complete landscapes with typically a few variable sites (Aguilar-Rodriguez et al., 2017; Domingo et al., 2018; Franke et al., 2011; Poelwijk et al., 2019; Weinreich et al., 2006) or unevenly sampled incomplete landscapes with many variable sites (Firnberg et al., 2014; Li et al., 2016; Ogden et al., 2019; Sarkisyan et al., 2016), and have stimulated studies of landscape ruggedness. Four measures of landscape ruggedness are now commonly used. The first is the number of maxima (N_max), which is the number of fitness peaks in a landscape (Durrett & Limic, 2003; Limic & Pemantle, 2004; Weinberger, 1991). The second is the fraction of pairs of sites exhibiting reciprocal sign epistasis (F_rse) (Poelwijk et al., 2007; Pokusaeva et al., 2019). Sign epistasis is a type of epistasis where the fitness effect of a mutation on two different backgrounds has opposite signs. Reciprocal sign epistasis is an extreme version of sign epistasis where the effect of the mutation at site 1 shows opposite signs on the two backgrounds that vary only at site 2, and vice versa. The third measure is the roughness to slope ratio (r/s), which quantifies the extent to which the landscape cannot be described by a linear model where mutations additively determine fitness (Aita et al., 2001; Carneiro & Hartl, 2010; Lobkovsky et al., 2011). Specifically, one can fit genotype fitness as a linear function of the effects of mutations at individual sites; r is the standard deviation of the fitness residual in the linear regression and s is the average of the absolute values of the linear coefficients in the linear regression. The fourth is the fraction of pathways that are blocked (F_bp) (Franke et al., 2011; Lobkovsky et al., 2011; Poelwijk et al., 2007; Weinreich et al., 2006). Here, a pathway from genotype i to genotype j through single-mutation steps is considered blocked if the fitness is decreased in any of the steps. While the four measures quantify somewhat different aspects of landscape ruggedness, they all increase with landscape ruggedness. Additionally, epistasis of a specific order (E) calculated by Fourier decomposition (Szendro et al., 2013; Weinreich et al., 2018; Weinreich et al., 2013), local correlation of fitness effects (γ) (Ferretti et al., 2016), average length of adaptive walks (N_adapt), and probability of reaching the global optimum (P_adapt) (Bank et al., 2016) have also been used to quantify landscape ruggedness (see Materials and Methods).

Regardless of how precisely fitness is measured, estimation error is inevitable and could affect the inference of landscape ruggedness. For example, Li et al. measured the fitness values of over 65,000 genotypes of a yeast tRNA gene in six replicates under a high-temperature condition (Li et al., 2016). A particular five-step pathway between two compared genotypes is unblocked when the mean fitness of the six replicates is considered for each genotype (black bold line in Fig. 1). However, when the fitness estimates from the six replicates are separately considered, the pathway is blocked in each replicate (six colored lines in Fig. 1). Here, the imprecise fitness estimates from individual replicates have apparently led to an overestimation of the landscape ruggedness. Intuitively, when the random fitness estimation error is extremely large, the landscape will resemble that under the house-of-cards model and have the maximal ruggedness (Szendro et al., 2013), but how sensitive various measures of landscape ruggedness are and how they scale with fitness estimation errors are unknown. A survey of the literature shows that most previous studies of landscape ruggedness did not seriously consider the potential impact of fitness estimation error. Some studies did not have replicates or did not consider the effect of estimation error (Anderson et al., 2015; Jiang et al., 2013; Poelwijk et al., 2019). Some other studies treated any fitness difference that is not statistically significant or not exceeding a preset cutoff as zero (Aguilar-Rodriguez et al., 2017; Bank et al., 2016; Domingo et al., 2018; Pokusaeva et al., 2019; Weinreich et al., 2006), but it is unknown how such treatments affect the estimation of landscape ruggedness. A few authors resampled fitness values using the mean and standard deviation from experimental replicates, and then tested if the landscape based on the resampled fitness values resembles the original landscape in ruggedness (Franke et al., 2011; Weinreich et al., 2006). However, the theoretical basis and biological meaning of this test remain unclear, and as we will show in this study, passing this test does not mean that the ruggedness estimate is unbiased.

Fig. 1.

An example from Li et al.’s yeast tRNA fitness landscape where a particular pathway is predicted to be accessible based on the mean fitness from six replicates for each genotype but inaccessible (or blocked) based on every replicate in fitness estimation. The black line shows the trajectory based on the mean fitness (with error bars showing standard deviations), whereas the colored lines show trajectories based on fitness estimates from individual replicates.

In this work, we comprehensively analyze the impact of fitness estimation error on the inference of landscape ruggedness. Because the true fitness values in empirical fitness landscapes are unknown due to estimation error, we simulate biallelic fitness landscapes following three commonly used models: the NK model (Kauffman & Weinberger, 1989), Rough Mount Fuji (RMF) model (Neidhart et al., 2014), and polynomial model (Hansen & Wagner, 2001; Weinreich et al., 2013). We then artificially add estimation errors to the simulated fitness values and study how the inferred landscape ruggedness is influenced. We report that all landscape ruggedness measures are upward biased by fitness estimation error but are affected to different extents. We then devise a method to use the replicates in fitness estimation to correct this bias and show that this correction is effective when there are three or more replicates.

RESULTS

Fitness estimation error renders landscape ruggedness overestimated

We investigated the impact of fitness estimation error on the inference of ruggedness of fitness landscapes simulated under the NK, RMF, and polynomial models, respectively. While we use “site” and “nucleotide” to describe the genotype space, these terms can be replaced with “locus” and “allele”. Under each theoretical model, we considered three different genotype lengths (n = 5, 10, and 15 variable sites) and generated 1000 complete landscapes per n value. We varied model-specific parameters to diversify the ruggedness of the 1000 landscapes under each model (see Materials and Methods). In total, we generated 1000×3×3 = 9000 fitness landscapes. All fitness values in each landscape were linearly normalized to the range between 0 and 1. Fitness estimation error is assumed to follow a normal distribution with the mean equal to zero and standard deviation (sd) equal to 0.02, 0.04, 0.06, or 0.08. The error was randomly generated and added to each true fitness value simulated to yield the observed fitness; the observed fitness values were again linearly normalized to the range between 0 and 1. For each true landscape simulated, four observed landscapes with varying amounts of estimation error were generated. We then quantified the landscape ruggedness by estimating N_max, F_rse, r/s, F_bp, E, 1-γ, 1/N_adapt, and 1-P_adapt for each true landscape and its four observed landscapes (see Materials and Methods). Because all eight ruggedness measures show consistent results, we present only the analyses on N_max, F_rse, r/s, and F_bp in the main text, while the results on E, 1-γ, 1/N_adapt, and 1-P_adapt are included in supplementary materials.

Fig. 2 and Fig. S1 present the results for n = 10 sites, while Fig. S2 and Fig. S3 present the results for n = 5 and 15 sites, respectively. It is clear that, under all three theoretical landscape models and for all eight measures of landscape ruggedness, the estimated ruggedness from the observed landscapes exceed that from the corresponding true landscapes. In other words, fitness estimation error renders the landscape ruggedness ubiquitously overestimated. As expected, the ruggedness overestimation is greater when the fitness estimation error is larger. However, it is visually clear that the eight measures of ruggedness have unequal sensitivities to fitness estimation error (Figs. 2, S1, S2, S3): r/s, E, and 1/N_adapt appear to be less sensitive to fitness estimation error than the other five ruggedness measures. The eight ruggedness measures also behave differently in the range of ruggedness where they are particularly sensitive to fitness estimation error. For instance, under the NK or RMF model, all measures except r/s are more sensitive to fitness estimation error in smooth than in rugged landscapes. Under the polynomial model, this trend is seen for F_bp, E, and 1-P_adapt but not for r/s. Interestingly, although parameters are chosen to maximize the range of ruggedness among the landscapes simulated, landscapes generated under the polynomial model have a much narrower range of ruggedness than those generated under the NK and RMF models when N_max, F_rse, 1-γ, and 1/N_adapt are used to measure landscape ruggedness, making it difficult to assess the range of polynomial landscape ruggedness within which these measures are most sensitive to fitness estimation error.

Fig. 2.

Effects of fitness estimation error on the landscape ruggedness estimates of N_max, F_rse, r/s, and F_bp. The x-axis shows the true ruggedness values of various landscapes with 10 variable sites, while the y-axis shows the estimated ruggedness values in the presence of fitness estimation error quantified by the indicated standard deviation (sd). Each dot represents one observed landscape, and the red line shows the natural cubic spline of the data points. The black diagonal line shows the situation when the estimated ruggedness equals the true ruggedness.

Comparing Fig. 2 with Figs. S2 and S3, we found that all ruggedness measures become more sensitive to fitness estimation error as n increases, which may be explained as follows. Because genotypes with only one nucleotide difference (aka neighboring genotypes) tend to be similar in fitness due to shared additive effects and because there are more genotypes within the fitness range from 0 to 1 in larger landscapes, the relative fitness ranks of neighboring genotypes are more likely to be switched by the same amount of estimation error in large landscapes than in small landscapes.

Quantitative impact of fitness estimation error on inferred landscape ruggedness

To better visualize the quantitative impact of fitness estimation error on landscape ruggedness inference, we considered many levels of estimation error for a given landscape. Specifically, under each of the three theoretical models with n = 10 sites and for each ruggedness measure, we examined three groups of landscapes with different levels of true ruggedness (shown by different colors in each panel of Figs. 3 and S4; see Materials and Methods). Each group contains 10 landscapes with very similar true ruggedness (same colors in each panel of Figs. 3 and S4). For each landscape, we considered multiple levels of fitness estimation error (sd = 0.002, 0.005, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.12, 0.14, 0.16, 0.18, 0.2, 0.25, 0.3, 0.35, and 0.4) and estimated the corresponding ruggedness of the observed landscapes. As expected, for a given fitness landscape, the estimated ruggedness increases with the fitness estimation error (Figs. 3 and S4). Interestingly, the error-ruggedness relationship exhibits an S-shaped curve for all ruggedness measures (the trend for r/s can be seen more clearly when the r/s range is greater). That is, the ruggedness increases slowly with error when the error is very small or very large, but increases quickly when the error is intermediate. Below we offer an intuitive explanation of the S-shaped relationship. Very small fitness estimation errors are insufficient to alter the fitness-based rank order of neighboring genotypes. Because N_max, F_rse, F_bp, 1/N_adapt, and 1-P_adapt depend on the fitness-based rank order of neighboring genotypes instead of their actual fitness values, these measures of ruggedness are insensitive to very small fitness estimation error. When the fitness estimation error rises to a certain level, it becomes large enough to alter the rank order of neighboring genotypes so the error quickly raises the inferred ruggedness. When the fitness estimation error is very large, the inferred ruggedness approaches the maximum, so further increasing the error cannot effectively increase the inferred ruggedness. Nonetheless, note that although the ruggedness does not increase much with error when the error is very small or very large, the S-shaped curve is convex under the former condition but concave under the latter condition (Figs. 3 and S4). However, because computing E and 1-γ depends on the actual fitness values instead of fitness ranks, it is unclear why their error-ruggedness relationships are also S-shaped.

Fig. 3.

Error-ruggedness relationships for N_max, F_rse, r/s, and F_bp in fitness landscapes with 10 variable sites. The x-axis shows the fitness estimation error quantified by standard deviation (sd) whereas the y-axis shows the estimated landscape ruggedness. Each line represents a group of observed fitness landscapes (with different levels of fitness estimation errors) from the same original landscape. Lines of the same color are based on different original landscapes with similar ruggedness.

Inferring the true landscape ruggedness

Because fitness estimates are not error-free and because the error upward biases all measures of landscape ruggedness, it is important to devise a method to infer the ruggedness unbiasedly. The error-ruggedness relationship revealed in the preceding section helps design an extrapolation method. Specifically, multiple replicate experiments are often performed to measure the fitness of the genotypes concerned, and such replicate measurements can be used for our purpose. For example, if a study has six replicates in fitness measurement, we use the six individual replicates to represent six observed landscapes where the fitness estimation error has a mean of 0 and a standard deviation of σ. We further use the mean of every two replicates to represent an observed fitness landscape. Because now each fitness estimate is a mean of two measures, the fitness estimation error has a mean of 0 and a standard deviation reduced to $\frac{\sigma }{\sqrt{2}}$. There are $\left(\begin{array}{l}6\hfill \\ 2\hfill \end{array}\right)=15$ such observed landscapes. Similarly, we use the mean of every three replicates to represent a fitness landscape where the error has a mean of 0 and a standard deviation of $\frac{\sigma }{\sqrt{3}}$, and there are $\left(\begin{array}{l}6\hfill \\ 3\hfill \end{array}\right)=20$ such observed landscapes. In the end, we can draw an X-Y plot where the X-axis is the fitness estimation error, Y-axis is the inferred ruggedness, and each dot represents one observed landscape mentioned above. We further compute the mean ruggedness of all observed landscapes with the same amount of error measured by the standard deviation, which leads to a mean error-mean ruggedness curve with six dots in this example; the six dots have their X-axis values of 𝜎, $\frac{\sigma }{\sqrt{2}}$, $\frac{\sigma }{\sqrt{3}}$, $\frac{\sigma }{\sqrt{4}}$, $\frac{\sigma }{\sqrt{5}}$, and $\frac{\sigma }{\sqrt{6}}$, respectively. We can then fit an appropriate model to the six dots to extrapolate the true ruggedness when the X-axis value is zero.

We tried various sigmoid functions that characterize the expected S-shaped ruggedness-error curve, including Holling Type III (HT3) functional response model, logistic model, and three algebraic models. In addition, we tried linear, quadratic, and cubic models. The performance of each model was then evaluated for the eight ruggedness measures in landscapes simulated under three different models (NK, RMF, and polynomial), three different genotype lengths (n = 5, 10, and 15 variable sites), three different levels of true ruggedness (low, intermediate, and high), four different levels of fitness estimation error (sd = 0.0333, 0.0667, 0.1000, and 0.1333), and four different numbers of replication in fitness measurement (3, 4, 5, and 6).

To compare the performance of the extrapolation method with that of simply using averaged fitness from replicates and that of using one replicate in ruggedness estimation, we used two metrics. As depicted in Fig. 4A, α is the difference between the ruggedness of an observed replicate landscape and the ruggedness of the true landscape, averaged across all replicates; α measures the expected error in the inferred ruggedness when there is only one replicate. β is the difference between the ruggedness of the average observed landscape (i.e., fitness of each genotype is the average of all replicates) and the truth; β measures the error in the inferred ruggedness without using extrapolation when there are multiple replicates. δ is the difference between the ruggedness estimated using extrapolation and the truth, so it measures the error of ruggedness inferred by the extrapolation method. As is clear from Fig. 3, the extrapolation method may or may not succeed, depending on where the data points are located in the S-curve. We thus considered the extrapolation result valid only when the standard deviation of the ruggedness estimate based on the regression is no greater than one half of the ruggedness estimated using the mean fitness from replicates. If the standard deviation cannot be computed, for example when there are only three replicates to fit the HT3 model, the extrapolation results are considered valid if the left-most data point is located in the convex region of the fitted curve (i.e., the second derivative computed from the fitted curve is positive at the point) (Fig. 4A). In the following comparison, we only considered observed landscapes with valid extrapolation results, and the proportion of landscapes considered is referred to as the acceptance rate.

Fig. 4.

Assessing various extrapolation methods in inferring landscape ruggedness. (A) An example of a landscape with six replicates of fitness measurement, illustrating various inferences of the landscape ruggedness and their associated errors. The x-axis shows the fitness estimation error relative to that when there is only one replicate, whereas the y-axis shows the inferred ruggedness. Grey dots represent observed landscapes. For example, the right-most six dots represent the six originally observed landscapes based on the six individual replicates, whereas the second right-most group of 15 dots represent 15 observed landscapes, each being the average of two original replicates. The left-most dot represents the landscape in which the fitness of each genotype is the mean from the six original replicates. Red dots show mean ruggedness of each group of landscapes with the same x-axis value. The blue curve is the fitted Holling Type III functional response curve (HT3) and the orange curves show the 95% confidence interval of the regression. The red arrow at the y-axis shows the true landscape ruggedness. In the figure, α is the difference between the ruggedness of an observed replicate landscape and the ruggedness of the true landscape, averaged across all replicates, measuring the expected error of the inferred ruggedness when there is only one replicate. β is the difference between the ruggedness of the average observed landscape (i.e., fitness of each genotype is the average of all replicates) and the truth, measuring the error of the ruggedness inferred from all replicates without using the extrapolation methods. δ is the difference between the ruggedness estimated using the extrapolation method and the truth, measuring the error of ruggedness inferred by the extrapolation method. (B-E) Box plot showing the distributions of α, β, and δ of various extrapolation methods in predicting the true N_max (B), F_rse (C), r/s (D), and F_bp (E) in 10-site fitness landscapes. Box plots are not shown for data with three replicates for methods requiring at least four replicates. Only landscapes for which the extrapolation is considered valid (see main text) are used in the comparison. For each of eight ruggedness measures, the true landscapes and their fitness estimation replicates are sampled by varying all possible parameters: theoretical models (NK, RMF, or polynomial), ruggedness levels (low, intermediate, or high), fitness estimation error (sd = 0.0333, 0.0667, 0.1000, or 0.1333), and number of fitness estimation replicates (3, 4, 5, or 6). Additionally, three true landscapes are simulated under each parameter set and three observed datasets are simulated per true landscape. The orange bar in each box represents the median of the distribution, the lower and upper borders of the box respectively represent the first (Q1) and third (Q3) quartiles, and the lower and upper caps respectively represent the lowest datum above Q1−1.5×(Q3−Q1) and the highest datum below Q3+1.5×(Q3−Q1). The gray scale shows the acceptance rate (see main text).

In Fig. 4B–E and Fig. S5, for each ruggedness measure and number of replicates in fitness estimation (N = 3, 4, 5, or 6) under n = 10 variable sites, we presented the distributions of α, β, and δ from various fitting models. The results show that HT3 yields the smallest δ with the lowest variance and high acceptance rates in almost all scenarios. Hence, we will focus on the performance (δ) of HT3 in subsequent analyses. Detailed results on α, β, and δ from HT3 under n = 5, 10, and 15 sites are presented in Figs. S6, S7, and S8, respectively. The distributions of α show that the ruggedness inferred from one replicate of fitness estimation has a large error (mean |α| is large) and is highly biased (mean α is positive), especially when n is 10 or 15. The distributions of β show that the ruggedness inferred from multiple replicates has a reduced error (mean |β| is not as large as mean |α|) and is less biased (mean β is less positive). Finally, the distributions of δ show that the ruggedness inferred from multiple replicates via extrapolation has an even smaller error (mean |δ| is smaller than mean |β|) and is virtually unbiased (mean δ is close to 0). The advantage of extrapolation over the simple average in estimating ruggedness is bigger in larger landscapes. When n = 5, the performance of extrapolation is only slightly better than that of the simple average, and the acceptance rate is lower than 0.5 in most cases. This is not unexpected, because the ruggedness inference is less sensitive to fitness estimation error in small than in large landscapes. Importantly, for larger fitness landscapes (n = 10 or 15), the bias in ruggedness inference is reduced substantially by the extrapolation method, and the acceptance rate is moderately high (except for N_max) even when there are only three replicates. Having more replicates does not seem to reduce the bias further, but does reduce |δ| and increase the acceptance rate.

When stratifying the results by the three theoretical models, we found the performance of the extrapolation method to be good under each model (Fig. S9). When stratifying the results by the three levels of true landscape ruggedness, we again confirmed the good performance of the extrapolation method under all levels (Fig. S10). Finally, as expected, the superiority of the extrapolation method over the simple average method rises when the fitness estimation error increases (Fig. S11).

Inferring the ruggedness of empirical fitness landscapes

Because of the generally good performance of the extrapolation method, we applied it to three empirical fitness landscapes: an incomplete yeast tRNA fitness landscape of 69 variable sites with six experimental replicates in fitness estimation (Li et al., 2016), a nearly complete yeast tRNA fitness landscape of 10 variable sites with six experimental replicates (Domingo et al., 2018), and a nearly complete translational efficiency landscape of Shine-Dalgarno sequences of 9 variable sites with three experimental replicates (Kuo et al., 2020). We did not find publicly available data of complete empirical fitness landscapes with at least five variable sites and at least three replicates. Due to the incompleteness of Li et al.’s tRNA fitness landscape, we calculated F_bp by randomly sampling 300,000 four-step pathways consisting of genotypes with estimated fitness (see Materials and Methods). To compute E that requires a complete biallelic fitness landscape, we skipped Li et al.’s landscape and used the biallelic landscape subsets from Domingo et al.’s and Kuo et al.’s landscapes, with missing fitness estimates interpolated by filling 0s for Domingo et al.’s landscape (see Materials and Methods) and by linear predictions for Kuo et al.’s landscape, respectively. The other ruggedness indices were estimated by exhaustive sampling of all data.

As shown in Table 1, Fig. 5, and Fig. S12, all estimates of ruggedness by the extrapolation method are lower than the original estimates. Notably, for Domingo et al.’s landscape, N_max and F_rse inferred from across-replicate average fitness are 32% and 13% greater than those inferred by extrapolation, respectively, which might cause different interpretations of their evolutionary implications. Of the eight ruggedness measures, F_bp and 1-P_adapt are most difficult to extrapolate accurately because the data tend to be located in the concave segment of the S-shaped curve except for F_bp of Domingo et al.’s landscape (Table 1). This is consistent with the fact that these two measures are sensitive to measurement error (Fig. 3 and Fig. S4). In addition, N_max, F_rse, and F_bp estimation is unreliable in the translational efficiency landscape, because the observations are located in the concave region of the S-shaped curve (Fig. 5). For the same reason, the corresponding ruggedness measures based on the mean fitness across replicates are likely upward biased and unreliable.

Table 1.

Ruggedness of three empirical fitness landscapes analyzed. Extrapolated ruggedness is presented, followed in parentheses by that based on average fitness estimates.

Landscapes	Yeast tRNA fitness landscape (Li et al. 2016)	Yeast tRNA fitness landscape (Domingo et al. 2018)	E. coli Shine-Dalgarno sequence translational efficiency landscape (Kuo et al. 2020)
Number of variable sites	69	10	9
Number of available genotypes1	21,182	4,176	197,890
N max	2518 (2534)	53 (70)	NA2 (2404)
F rse	0.0793 (0.0820)	0.1550 (0.1748)	NA (0.2141)
r/s	3.246 (3.355)	1.964 (2.102)	3.965 (4.063)
F bp	NA (0.9157)	0.7187 (0.7444)	NA (0.7738)
E	?3	0.3362 (0.3699)	0.8040 (0.8066)
1-γ	0.5070 (0.5186)	0.6616 (0.7005)	0.5923 (0.6322)
1/N adapt	0.4644 (0.4680)	0.2885 (0.3007)	0.2030 (0.2329)
1-P adapt	NA (0.9887)	NA (0.9871)	NA (0.9996)

¹Genotypes with sufficient experimental replicates are considered.

²NA, ruggedness inference not applicable because the data are located in the concave segment of the S-shaped error-ruggedness relationship.

³?, ruggedness cannot be estimated due to the incompleteness of the landscape.

Fig. 5.

Inferring the ruggedness of Li et al.’s tRNA fitness landscape, Domingo et al.’s tRNA fitness landscape, and Kuo et al.’s translational efficiency landscape of Shine-Dalgarno sequences by extrapolation. Symbols have the same meanings as in Fig. 4A. Note that the inferences of F_bp in Li et al.’s landscape and N_max, F_rse, F_bp in Kuo et al.’s landscape are apparently unreliable here. For Kuo et al’s landscape, the 95% confidence interval of the regression cannot be estimated due to the lack of sufficient replicates.

DISCUSSION

By a comprehensive analysis of simulated fitness landscapes under three widely considered theoretical models, we found that fitness estimation error causes overestimation of all eight commonly used measures of landscape ruggedness. This problem is especially serious for N_max, F_rse, F_bp, 1–γ, and 1-P_adapt, while r/s, E, and 1/N_adapt are relatively insensitive. We also found that the ruggedness overestimation is more serious in larger landscapes (with 10 or more variable sites) and in smoother landscapes. Because most previous studies ignored fitness estimation error in quantifying landscape ruggedness, their ruggedness estimates are likely upward biased, which may have affected the relevant evolutionary conclusions. Some previous studies considered the impact of fitness estimation error by shuffling fitness estimates from replicates (Weinreich et al., 2006) or using the mean and standard deviation from experimental replicates (Franke et al., 2011). For example, Franke et al. considered the inferred ruggedness reliable when the inferred ruggedness of the landscape based on resampled fitness values is similar to that of the landscape based on the mean fitness values (Franke et al., 2011). However, a resampled landscape and the landscape based on mean fitness contain different levels of fitness estimation error. As the error-ruggedness curves of Fig. 3 and Fig. S4 showed, the inferred ruggedness increases slowly with the fitness estimation error in two regions, one where the inferred ruggedness is close to the true value and the other where the inferred ruggedness is far from the true value. Hence, the above procedure does not guarantee accurate inference of ruggedness. Some other studies treated any fitness difference that is not statistically significant or not exceeding a preset cutoff as zero (Aguilar-Rodriguez et al., 2017; Bank et al., 2016; Domingo et al., 2018; Pokusaeva et al., 2019; Weinreich et al., 2006). However, natural selection can detect a fitness differential of 1/N_e, far more sensitive than typical experiments. In other words, measured fitness differences that are not statistically significant may often be evolutionarily significant, making it difficult to use measurement errors to set a cutoff in evaluating evolutionary significance.

We devised an extrapolation method to infer virtually unbiased landscape ruggedness from replicates of fitness measures and showed that this method is useful as long as the landscape is not too small (with >5 variable sites). Importantly, we found that the extrapolation method does not require a large number of replicates; as few as three replicates can often (but not always) provide a substantial improvement in ruggedness inference. We thus recommend that future fitness landscape mapping should include at least three biological replicates to allow a relatively unbiased inference of the landscape ruggedness.

In our simulation, we assumed that the fitness estimation error is independent of the fitness estimate, which is more or less true in the limited empirical data examined (Fig. S13). It is, however, possible that the above assumption is violated in some datasets; in the future, it would be important to confirm our conclusions under this scenario. Our work focused on simulated landscapes with 5, 10, or 15 biallelic sites, so can only predict the properties of larger landscapes from the trend seen in these simulated landscapes. With the rapid rise of sequencing power, fitness landscapes bigger than the currently available landscapes will likely be mapped in the coming years. Hence, it will be valuable to validate our predictions in future studies.

MATERIALS AND METHODS

NK landscapes

In all theoretical fitness landscape models considered, each variable site of a genotype has the state of either 0 or 1. We focused on biallelic landscapes because most empirical fitness landscapes published thus far are biallelic, most theoretical studies analyze only biallelic landscapes, and biallelic landscapes are relatively easy to simulate. We considered three genotype lengths (5, 10, or 15 variable sites) and generated complete landscapes under each model. We constructed the NK fitness landscapes with the random neighborhood model (Hwang et al., 2018) using the code from https://github.com/Mac13kW/NK_model. The parameter N denotes the genotype length and the parameter K denotes the number of other sites with which each focal site interacts. The fitness contribution of the ith site of genotype x, F_i(x), is determined by the state of the site itself and the states of the K interacting sites, and is randomly drawn from a uniform distribution between 0 and 1. The overall fitness of a genotype is calculated by averaging the fitness contributions of all of its N sites:

$$F(x)=\frac{1}{N}\sum _i^N{F}_{\text{i}}(x)$$

We varied K from 1 to N-1 with a step size of 1 in our study to acquire landscapes of different ruggedness. The effect of K on the landscape ruggedness is shown in Fig. S14.

Rough Mount Fuji (RMF) landscapes

We constructed RMF landscapes following the original paper (Neidhart et al., 2014). Generally speaking, the fitness of a genotype linearly decreases with its genetic distance from the optimal genotype, with stochastic variation. To calculate the fitness of a genotype, we first determine the optimal genotype 𝑥^∗, which is chosen randomly and has a fitness of 1. The fitness of genotype x is then −cD(x, x^∗) + η_x, where D(x, x^∗) is the Hamming distance between the focal genotype x and the optimal genotype x^∗, c is a constant denoting the fixed distance effect and equals 1 in our study, and η_x is the random fitness deviation whose value is sampled from a normal distribution with mean equal to 0 and standard deviation equal to SD. We set SD at 10/k, where k is an integer that varied from 1 to 20. Hence, SD varied from 0.5 to 10, which allowed the generation of fitness landscapes of different ruggedness. The effect of SD on landscape ruggedness is shown in Fig. S14.

Polynomial landscapes

We constructed polynomial landscapes following the relevant literature (Hansen & Wagner, 2001). Given the presence of two-way and higher-order epistasis in empirical landscapes (Weinreich et al., 2018), we should ideally include additive and all orders of epistasis in the model. However, the prohibitively high computational cost allowed us to include only additive effects, pairwise interactions, and three-way interactions in the model as follows.

$$\begin{gathered} F(x)=\sum _i^n{\beta }_{\text{i}}{x}_{\text{i}}+\sum _{i<j}^n{\beta }_{\text{ij}}{x}_{\text{i}}{x}_{\text{j}}+\sum _{i<j<k}^n{\beta }_{\text{ijk}}{x}_{\text{i}}{x}_{\text{j}}{x}_{\text{k}} \\ {\beta }_{\text{i}}~N\left(0,{\sigma }_1^2\right) \\ {\beta }_{\text{ij}}~N\left(0,{\sigma }_2^2\right) \\ {\beta }_{\text{ijk}}~N\left(0,{\sigma }_3^2\right) \\ {\sigma }_1^2+{\sigma }_2^2+{\sigma }_3^2=1\text{and}{\sigma }_2^2={\sigma }_3^2 \end{gathered}$$

Here, F(x) is the fitness of genotype x and n is the number of variable sites. For any i, x_i is the state of the genotype x at site i, which can be either 0 or 1. β_i, β_ij, and β_ijk are coefficients for additive effects, pairwise interactions, and three-way interactions, respectively, and are sampled from normal distributions with zero means and variances of ${\sigma }_1^2$, ${\sigma }_2^2$, ${\sigma }_3^2$, respectively. We drew σ₁ from a set of 20 evenly spaced values between 0.05 and 0.95 to generate fitness landscapes with different ruggedness. The effect of σ₁ on landscape ruggedness is shown in Fig. S14.

Li et al.’s yeast tRNA fitness landscape

In the 69-site fitness landscape of a yeast tRNA gene that includes 65,537 genotypes with fitness measured in six replicates (Li et al., 2016), the genotypes are concentrated around the wild-type with 1 to 11 mutations. The fitness values in the original paper were Wrightian fitness and had a lower limit of 0.5, which represented no detectable growth of the genotype in the competition assay. Genotypes with measured fitness of 0.5 in ≥5 replicates were excluded because the rank order cannot be reliably determined; the filtered landscape included 21,182 genotypes. Because Wrightian fitness is multiplicative under no epistasis, we log-transformed the original fitness values to obtain additive fitness values so that they are more consistent with the assumptions in certain ruggedness measures such as r/s.

Domingo et al.’s yeast tRNA fitness landscape

The tRNA landscape has 10 variable sites that are known to vary among post-whole-genome-duplication yeast species, where 6 sites are biallelic and 4 sites are triallelic. The original paper used log-transformed Wrightian fitness, which we followed. The landscape includes 4176 genotypes with fitness measured in six replicates (Domingo et al., 2018). Given that genotypes with zero reads were discarded in the original paper, we filled all missing genotypes with fitness of 0 to obtain a complete fitness landscape to calculate E.

Kuo et al.’s translational efficiency landscape

The landscape has 9 variable sites each with four states, leading to a space of 4⁹ = 262,144 genotypes (Kuo et al., 2020). The translational efficiency of each genotype was measured in three replicates in each of three distinct mRNA contexts in Escherichia coli. In our study, the context with the highest genotype coverage across all three replicates (arti) was chosen, which has 197,890 genotypes after the exclusion of genotypes with missing replicate measures.

Simulating fitness estimation error

From the three empirical landscape data, we found that fitness estimation error is largely independent of the fitness estimate, with only a moderate reduction of error with the fitness estimate in Domingo et al.’s data (Fig. S13). Thus, we assume that the fitness estimation error follows the same distribution for all genotypes in a landscape when introducing fitness estimation errors to simulated landscapes.

Estimation of N_max

For a given landscape, we examined all genotypes to identify local fitness maxima. A genotype is considered a local fitness maximum if it is fitter than all of its neighboring genotypes, which differ from the focal genotype by one point mutation. N_max is the total number of local fitness maxima in the landscape. For the three empirical landscapes, we examined all genotypes with fitness estimates and treated neighboring genotypes without fitness estimates as having a fitness of 0.

Estimation of F_rse

For landscapes with n = 5 or 10 variable sites, we examined all possible genotype pairs that differ from each other at two sites. For each genotype pair, reciprocal sign epistasis is recorded if both genotypes are either fitter or less fit than the two intermediate genotypes between them. F_rse is the proportion of genotype pairs that exhibit reciprocal sign epistasis. For landscapes with n = 15, due to the extremely large genotype space, we randomly sampled 200,000 genotype pairs to estimate F_rse. For the three empirical landscapes, we examined all genotype pairs from genotypes with estimated fitness for F_rse estimation.

Estimation of r/s

For the simulated landscapes, we first fit the additive model $F(x)={\beta }_0+{\sum }_{i=1}^n{\beta }_{\text{i}}{x}_{\text{i}}$ to the fitness landscape, where x is a genotype, x_i is the state at the i^th site (0 or 1), F(x) is the fitness of x, and n is the number of variable sites. By fitting the above additive model, we acquired estimates of β_i. The roughness value r is the square root of the mean squared fitness residual from the above regression and s is the average of the absolute values of the linear coefficients β_i (i =1, 2, …, and n). That is, $r=\sqrt{\frac{{\sum }_x{\left(f_{\text{x}}-F(x)\right)}^2}{m}}$ and $S=\frac{{\sum }_{i=1}^n\left|{\beta }_{\text{i}}\right|}{n}$, where fx is the observed fitness of genotype x, F(x) is the model predicted fitness of x, and m is the number of genotypes with fitness estimates. Then, r/s is calculated by dividing r by s.

When a landscape has four instead of two states per variable site, we adjusted the above additive model to $F(x)={\beta }_0+{\sum }_{i=1}^n{\beta }_{\text{i}}^{\text{A}}{x}_{\text{i}}^{\text{A}}+{\beta }_{\text{i}}^{\text{T}}{x}_{\text{i}}^{\text{T}}+{\beta }_{\text{i}}^{\text{C}}{x}_{\text{i}}^{\text{C}}+{\beta }_{\text{i}}^{\text{G}}{x}_{\text{i}}^{\text{G}}$, $\text{r}=\sqrt{\frac{{\sum }_x{\left(f_{\text{x}}-F(x)\right)}^2}{m}}$, and $s=\frac{{\sum }_{i=1}^n\left|{\beta }_{\text{i}}^{\text{A}}\right|+\left|{\beta }_{\text{i}}^{\text{T}}\right|+\left|{\beta }_{\text{i}}^{\text{C}}\right|+\left|{\beta }_{\text{i}}^{\text{G}}\right|}{3n}$ where $x_{\text{i}}^{\text{A}}$ is a 0/1 binary state showing whether the genotype x has nucleotide A while $x_{\text{i}}^{\text{T}}$, $x_{\text{i}}^{\text{C}}$, and $x_{\text{i}}^{\text{G}}$ are simily defined.

Estimation of F_bp

For the simulated landscapes with n = 5, 10, and 15 variable sites, we respectively sampled all, 120,000, and 240,000 four-step pathways, and further subsampled qualified pathways where the starting genotype has fitness lower than the 20^th percentile in the fitness distribution and final genotype has fitness higher than the 80^th percentile. The qualified pathways are used to calculate the fraction of blocked pathways.

For Li et al.’s tRNA landscape, due to the large number of pathways and missing data, we randomly sampled 300,000 qualified pathways in which all genotypes have estimated fitness. For the other two landscapes, we investigated all possible qualified pathways.

Obviously, F_bp is sensitive to fitness estimation error because reversing the fitness rank for only one step blocks an otherwise open pathway. Some previous studies thus considered an evolutionary step impossible only when the fitness reduction associated with the step exceeds an arbitrary cutoff. We did not use this method because its performance depends on the cutoff relative to the size of the fitness estimation error so is difficult to evaluate objectively.

E estimation

E describes the relative magnitude of epistasis of all orders. We calculated the epistatic effect of each order by applying Fourier-Walsh transformation as described previously (Goldberg, 1989; Weinreich et al., 2018; Weinreich et al., 2013). Note that we excluded the zero-order “epistasis” (E₀), which represents the mean fitness of the landscape, from the calculation of E. First-order “epistasis” (E₁) is the additive fitness effect. Therefore, E equals $1-\frac{E_1}{{\sum }_{j=1}^n{E}_{\text{j}}}$, where Ej is the epistasis of the jth order.

Estimation of 1-γ

γ describes the correlation between the fitness effect of a mutation in one genetic background and that in another background differing from the first background at one site. We calculated γ by following a previous study (Ferretti et al., 2016) and regarded 1-γ as the ruggedness measure in our analysis so that it increases with ruggedness.

Estimation of 1/N_adapt

N_adapt denotes the expected number of steps before reaching local or global optimal in a greedy adaptive walk, where each adaptation step takes the mutation maximizing the fitness. By adapting an absorbing Markov chain model previously described (Bank et al., 2016) in the SSWM (strong selection weak mutation) regime, we solved N_adapt analytically from complete fitness landscapes. For empirical fitness landscapes with missing data, N_adapt is calculated by simulating greedy walks with all possible starting genotypes. We considered 1/N_adapt a ruggedness measure in our analysis because it increases with ruggedness.

Estimation of 1-P_adapt

P_adapt denotes the probability of reaching the global optimum in adaptive greedy walks. By adapting an absorbing Markov chain model previously described (Bank et al., 2016) in the SSWM regime, we solved P_adapt analytically from a complete fitness landscape. For empirical fitness landscapes with missing data, P_adapt is calculated by simulating greedy walks with all possible starting genotypes. We regarded 1-P_adapt as a ruggedness measure because it increases with ruggedness.

True ruggedness of simulated fitness landscapes

We chose simulated fitness landscapes with three levels of true ruggedness (low, intermediate, and high) to analyze the quantitative impact of fitness estimation error on inferred landscape ruggedness (Figs. 3, S4), and to evaluate the performance of the extrapolation method (Figs. 4B–E, S5–S11).

Because the distribution of true ruggedness can be very different depending on the theoretical fitness landscape model, ruggedness measure, number of variable sites, and other factors, the low, intermediate, and high levels of true ruggedness were chosen manually according to the specific ruggedness distribution. Roughly speaking, the low level of true ruggedness ranged from 0 to 30 percentile of the distribution, the intermediate level ranged from 40 to 60 percentile of the distribution, and the high level ranged from 70 to 100 percentile of the distribution. We required the true ruggedness of the fitness landscapes of the same group (i.e., same theoretical model, same number of variable sites, and same ruggedness level) to be as close to one another as possible.

Model selection

To determine the best model to extrapolate the landscape ruggedness under no fitness measurement error, we fitted various sigmoid (S-shaped) functions to the simulated data points, including Holling Type III (HT3) functional response model $(y=\frac{a{x}^2}{ab{x}^2+1}+c)$, logistic model $(y=\frac{a}{1+e^{-b(x-c)}}+d)$, and three algebraic models: Algebraic 1 $(y=\frac{a{x}^3}{ab{x}^3+1}+c)$, Algebraic 2 $(y=\frac{a(x-c)}{\sqrt{{(x-c)}^2+b}}+d)$, and Algebraic 3 $(y=\frac{a(x-c)}{|x-c|+b}+d)$, where a, b, c, d are parameters. In addition, we fitted linear, quadratic, and cubic models.

The performance of each model was evaluated for eight ruggedness measures, under three theoretical models (NK, RMF, and polynomial), three genotype lengths (n = 5, 10, and 15 variable sites), and three true ruggedness levels (low, intermediate, and high). We simulated three landscapes under each parameter combination, generating a total of 8×3×3×3×3 = 648 true landscapes. We then added four different levels of fitness estimation error (sd = 0.0333, 0.0667, 0.1000, and 0.1333) to each true landscape because these error levels are either typical or reachable in empirical data (Fig. S13), and considered 3, 4, 5, or 6 replicates in fitness measurement because most of the extrapolation methods require at least 3 replicates. For each set of parameters, we generated three datasets because fitness estimation error is stochastic. Thus, 4×4×3 = 48 extrapolations were performed per true landscape. Python package LMFIT was used to fit all models to simulated data. In particular, we used the Levenberg–Marquardt algorithm provided in the python package scipy.optimize.curve_fit for non-linear least squares fitting. Because the algorithm can be sensitive to the initial parameters, we set multiple initial parameters and chose the fitted model with the lowest error sum of squares.