Modelling heterogeneity among fitness functions using random regression

Introduction

The adaptive landscape or the mapping of variation in fitness as a function of phenotypic characters as envisioned by Simpson (1944), is a key concept in evolutionary biology (Arnold, Pfrender & Jones 2001). Also called the phenotypic selection surface, the idea derived from Sewall Wright’s definition of the adaptive topography in which fitness is a function of gene frequencies (Wright 1932). Simpson’s adaptive landscape concept led to Lande’s (Lande 1976; Lande 1979) framework for assessing phenotypic evolution of multivariate characters and multivariate selection (Lande & Arnold 1983), which models fitness as a quadratic function of trait values. The estimated coefficients of the Lande-Arnold model, (the vector of linear and the matrix of nonlinear selection gradients), occupy a central role in studies of phenotypic evolution as they are used to both describe the form of selection on traits, and directly link to equations of multivariate evolutionary change. Subsequent modeling of the selection surface includes the use of nonparametric techniques such as spline-fitting (Schluter 1988; Schluter & Nychka 1994), response surface methodology (Phillips & Arnold 1989), and most recently tensor decomposition (Calsbeek 2012).

The adaptive topography may be envisaged as heterogeneous with local fitness minima and maxima. Schluter (1988) and Schluter & Nychka (1994) advocated the use of nonparametric regression using penalized cubic splines to visualize and interpret selection surface curvature under conditions where data may not be best fit by a second-order quadratic fitness function (i.e., the Lande and Arnold model) and avoid misleading interpretations of the fitness function’s shape (Mitchell-Olds & Shaw 1987). The method of penalized cubic spline fitting has subsequently been widely adopted, and as demonstrated by Morrissey & Sakrejda (2013) numerical differentiation of spline-based models can be used to produce estimates of linear and nonlinear selection gradients. Thus spline-based approaches can qualitatively and quantitatively describe the covariance of fitness and phenotypes in a manner equivalent to the Lande–Arnold approach. Here we further develop the spline-based approach to fitness function analysis by introducing a methodology to allow researchers to test hypotheses that spline-modeled fitness functions vary among study units. The proposed framework could be adopted to test for heterogeneity among fitness functions modeled as second order quadratic using the standard Lande–Arnold approach, although we did not address this valid approach.

The structure of complex fitness landscapes suggests that differing environments or biological contexts and combinations of characters result in peaks and troughs in fitness. Intuitively, these local peaks and valleys may reflect multiple populations experiencing differing selection regimes or a single population’s fitness landscape that is variable across seasons or years. Selecting an appropriate method to test hypotheses of among-population variability in fitness functions and fitting the appropriate model to relate fitness and trait variation represent major challenges when comparing estimates of selection in natural populations (Calsbeek 2012; Kingsolver et al. 2012; Morrissey & Hadfield 2012; Haller & Hendry 2014; Morrissey 2014). Calsbeek (2012) recently introduced an application of tensor decomposition to allow comparison of changes in the strength and shape of multivariate selection across space or time using nonparametric penalized cubic spline models. A standard approach to evaluating the hypothesis of heterogeneity in selection however, has been to fit models that explicitly test for interaction between phenotypes and populations or years on fitness (e.g., (Svensson & Sinervo 2004; Egan, Hood & Ott 2011a). In this approach, significant population or year × trait interactions demonstrated by ANOVA are interpreted as statistical evidence of non-additivity in the relationship between traits and populations on fitness. For example, Svensson & Sinervo (2004) fitted interaction effects to estimate heterogeneity in linear and quadratic selection gradients and found that selection on egg mass in lizards was variable between years but only slightly variable among subpopulations. Similarly, Egan, Hood & Ott (2011a) applied tests of interactions in their study of plant-mediated selection on insect gall diameter (size) and found evidence for heterogeneity in linear and quadratic selection gradients across study units in each of two years.

There are two main limitations to the ANOVA-based approach described above. First, the model does not provide an estimate of the variability of records collected among the different groups (e.g., sites, populations, study units) and (or) across years, which in the ANCOVA formulation are assumed to be fixed. If the groups may be envisaged as effects having some common mean and variance across populations or years, for example, then the use of random effects models provides a way to maximize the information available from such an experimental design. Furthermore the number of degrees of freedom required to test the fixed effects of population, and population by trait, on fitness scales linearly with the number of populations. The second important limitation is that the model of the fitness function is strictly second-order quadratic, which can be problematic with respect to inference and (or) prediction if another statistical model provides a better fit to the data (Mitchell-Olds & Shaw 1987; Schluter 1988; Calsbeek 2012; Morrissey & Sakrejda 2013). These concerns favor the use of random regression models over the fixed effect parametric regression model (Henderson 1982; Laird & Ware 1982). In order to allow for complex fitness function estimation the hierarchical models can be extended flexibly by incorporating splines.

Random regression refers to regression models where the regression coefficients are subject-specific and are assumed to come from a (normal) distribution, the parameters of which (mean and variance) can be estimated from data using likelihood or Bayesian methods (Ruppert, Wand & Carroll 2003; Meyer 2005). The specification of the regression function may be parametric (e.g., polynomial) or semi-parametric (e.g., splines). Splines are essentially piecewise polynomial functions joined at knots, with knots usually positioned at the percentiles of a continuous independent variable (Ruppert, Wand & Carroll 2003). In the application of random regression models to the analysis of fitness functions estimated in each of multiple study units, the extent of inter-unit variability in the fitness function is quantified by the variance parameters, ${\sigma }_k^2$, associated with the regression coefficients. The treatment of the subject-specific spline regression coefficients as random induces shrinkage of estimate of those coefficients similar to the one induced with use of penalized regressions (Verbyla et al. 1999; Ruppert, Wand & Carroll 2003; Meyer 2005). Heterogeneity can be tested by likelihood ratio tests of full and reduced models without the variance component of interest (Stram & Lee 1994). Therefore, the random regression framework provides a direct test of the variance among the estimated coefficients that describe the relationship between fitness and trait values.

An additional advantage of the random regression framework employing spline fits to data is that if selection varies among study units, this implies that additional information is potentially available on the structure of the heterogeneity. What inferential procedures could then be applied to provide a more detailed analysis of how the fitness functions vary? For example, at what value (or range of values) within the phenotypic distribution is the relationship between phenotype and fitness most variable among study units? To address this question we propose and implement two additional inferential techniques. The first uses piecewise constant basis functions to yield a stepwise fitness function that is constant among phenotypes within predefined windows and steps up or down at pre-specified knots. When this approach is implemented in a random regression model, the variance components associated with each window can be directly interpreted as a measure of variability in the fitness function among study units for the specified region of the phenotypic distribution. The second technique is computation and comparison of local prediction error sums of squares (PSS) from fixed and random spline models fit to training data using 10–fold cross validation (Hastie, Tibshirani & Friedman 2009). By noting any changes in the magnitude of the local prediction error between fixed and random regression we again can isolate the regions of the phenotype distribution exhibiting significant among-study unit variability in the nonparametric fitness functions.

The main goal of the current study is to illustrate the application of random regression models and an associated analytical framework (Fig. 1) to the study of phenotypic selection to test directly the hypothesis that fitness functions vary among study units (populations). As an example, we analyze phenotypic selection on a single trait for which selection has been well characterized in multiple study units in each of two years (Egan, Hood & Ott 2011b; Ott & Egan 2014). The fitness function of interest is the relationship between the size of galls produced by a gall-forming insect and the probability of insect survival. We treat the individual populations of the gall former as the study units (see methods below) as justified by the findings of Egan & Ott (2007) and Egan, Hood & Ott (2011a). We address two objectives: (1) to determine whether viability selection on the trait “gall size” is variable among populations in each of the two years; and (2) to define where the heterogeneity among viability selection functions occurs with respect to the gall size (phenotype) distribution. We address the two objectives in three sections: (1) data visualization, in which fitness functions are defined under three model specifications—second-order polynomial, B-spline and stepwise, (2) inference of variance parameters under B-spline and stepwise model specifications, and (3) prediction, in which we assess the evidence that populations have variable fitness functions using 10-fold cross validation (Fig. 1). The analytical framework presented complements and extends the insights available on variation in selection among study units (treatments, populations, or years) available from the application of second-order polynomial models, (Lande & Arnold 1983), cubic spline fitting, (Schluter 1988), and selection gradient estimation from spline-based models (Morrissey & Sakrejda 2013).

Figure 1.

Analytical framework used to test for heterogeneity in fitness functions among study units. RBS = Random effects B-spline, FBS = Fixed effects B-spline, RSF= Random effects step function, FSF = Fixed effects Step function. ^aStatistically significant variance component estimates from RSF model indicate percentiles of the trait distribution that exhibit significant heterogeneity among study units. ^bLocal PSS described in detail in Methods, section “Model comparison”.

Methods

In the following we describe the fitting of the spline-based and stepwise models, hypothesis testing of model parameters and evaluation of model goodness of fit. We used a 6 DF quadratic (degree 2) B-spline transformation of gall size to model the nonparametric fitness curves. In a second formulation, we utilized a piecewise transformation of gall size according to evenly spaced percentiles of the gall size distribution in order to fit stepwise fitness functions. The stepwise fitness functions are complementary methods that allow us to assess the extent of among population variation in fitness according to exact bins of the trait distribution. Finally we used 10-fold cross validation to assess the relative predictive ability of the fixed and random regression models. A description of the study system and rationale for its use in testing the hypothesis of interpopulation variability in selection is provided in the supporting information.

Model fitting

We model survivorship (i.e., the probability that a gall wasp will emerge) from the j^th gall of the i^th population using the logit link; that is, $p\left(y_{ij}=1\right)=\frac{exp\left({\eta }_{ij}\right)}{1+exp\left({\eta }_{ij}\right)}$, where η_ij is a propensity score, which in our models is a function of gall size: η_ij = f(x_ij). Here, logit⁻¹(f(x_ij)) can be interpreted as a fitness function. A standard approach is to model this function parametrically. For instance in the Lande–Arnold selection model f(x_ij) is represented using a second-order polynomial on the trait defining fitness, in our case $f\left(x_{ij}\right)=b_0+b_1{x}_{ij}+b_2{x}_{ij}^2$. Inspection of our data using traditional modeling (Janzen & Stern 1998) and visualization (Schluter 1988) approaches suggested that the patterns relating survivorship to gall size might not be best described with polynomials (Supplemental Figure S1). Furthermore, Akaike Information Criterion (AIC, (Akaike 1973)), for both the fixed effects B-spline models were lower than that of the second-order polynomial (Table 1). Thus we focus on nonparametric approaches where the regression function is modeled as linear combinations of local basis functions of the form, 

Table 1.

Model selection summary for three fixed effect models describing the probability of gall wasp survival as a function of gall diameter (size) fit to the 2002 and 2008 data. For each model we report residual deviance (−2*LogLikelihood), number of parameters (K), Akaike Information Criterion (AIC) of the given model, the difference in AIC (ΔAIC) of each model relative to the best model (Bold), which in this case was the 6 df B-spline and the akaike weight (wt).

Model	Deviance	K	AIC	ΔAIC	wt
2002 N = 7181
Six df B-spline (fixed)	9211.7	7	9225.7	0	1.0
Second-order quadratic	9251.5	3	9257.5	31.8	0.0
Intercept only (Null model)	9562.6	1	9564.6	338.9	0.0
2008 N = 6787
Six df B-spline (fixed)	7853.5	7	7867.5	0	1.0
Second-order quadratic	8461.6	3	8467.6	600.1	0.0
Intercept only (Null model)	9314.5	1	9316.5	1449	0.0

$$\mathbf{Pooled},\mathbf{fixed}\mathbf{effects}\mathbf{model}:f\left(x_{ij}\right)={\sum }_{k=1}^K{\varphi }_k\left(x_{ij}\right){\beta }_k,$$ (1)

where the ϕ_k(x_ij)’s are local basis functions (either from a spline or piecewise constant) and the β_k’s are regression coefficients. We used a 6 DF quadratic (degree 2) B-spline transformation of gall size to model the nonparametric fitness curves. This specification resulted in 4 evenly spaced internal knots at the quartiles of the trait distribution and two knots at the minimum and maximum of the trait distribution. Our choice of spline model was based largely on practicality. Egan, Hood & Ott (2011a) clearly showed that nonlinear selection on gall size was prevalent in 2002 and 2008. Thus, we desired commonly used basis functions that could visually feature curvature in the selection surface. We also sought a spline model that was not overparameterized, simple and easily implemented in R (R Core DevelopmentTeam 2009) using the lme4 (Bates, Maechler & Bolker 2012) mixed model framework to facilitate hypothesis testing and model comparison. The B-spline bases have favorable numeric properties and are a common choice in model fitting (Ruppert, Wand & Carroll 2003). The number of knots affects the form of the fitted model. The 6 DF quadratic B-spline, based on visual inspection of the resulting fitness functions, was deemed a reasonable choice to model the curvature without worry of overparameterization. For example, the model contained 4 additional parameters over a second order polynomial model. Visual inspection of the fitted curves did not change appreciably when more knots were used (8 DF, 10 DF). Thus we opted for the more parsimonious formulation. We generated these basis functions using the bs() function with 6 DF and degree = 2 of the splines package in R 2.9.2 (R Core Development Team 2009). Finally, in modeling fitness functions with splines, the optimal number of knots, the polynomial degree and specific basis functions used should be considered on a case by case basis. 

We then let f(x_ij) be a step function. Thus, we first defined windows of gall size based on empirical quantiles {τ₁,…} and then set ϕ₁(x_ij) = {1 if x_ij ≤ τ₁ ; 0 otherwise} and ϕ₂(x_ij) = {1 if τ₁ < x_ij ≤ τ₂ ; 0 otherwise}. This gives a function that is constant within the intervals of gall size but potentially discontinuous at the cut-off values (see Fig. 2). We used seven windows, which corresponds to the exact dimension of the B-spline model including the intercept and the six basis functions.

Figure 2.

Predicted pooled population-level fitness functions depicting viability selection, that is, probability of emergence (survival) of the gall former as a function of gall size, as inferred from three model specifications in each of two years: 2002 (upper panel) and 2008 (lower panel): second-order polynomial (dashed), 6 DF quadratic B-spline (solid), and piecewise (gray). Quantiles (i.e., 0, 25, 50, 75, 100 percentiles) of the gall size distribution are indicated for each year by short segments on the horizontal axis. Variation in the relationship between phenotype and fitness was modeled by pooling data across all populations within each year. N = 7181 (2002), N = 6787 (2008).

In the models described above, the curve relating fitness to gall size is the same for all study units, hence equation (1) gives an overall fitness curve based on data pooled across all populations. This model can be extended to accommodate inter-study unit (i.e., among population) variability in the fitness functions by allowing the regression coefficients of the fitness function to vary among populations as follows:

$$\mathbf{Population}-\mathbf{specific},\mathbf{mixed}\mathbf{effects}\mathbf{model}:f\left(x_{ij}\right)={\sum }_{k=1}^K{\varphi }_k\left(x_{ij}\right)\left({\beta }_k+{\beta }_{ki}\right).$$ (2)

In this model, the regression coefficients associated with each basis function have two components: one that is common for all members of the pooled study units (among all populations in our application), β_k, and one that is study unit-specific (population-specific in our application), β_ki. We model the population-specific coefficients as random effects that are normally and independently distributed, that is, ${\beta }_{ki}~N\left(0,{\sigma }_k^2\right)$. The treatment of these regression coefficients as random induces shrinkage of estimates of effects, similar to the one induced when regression coefficients are estimated using penalized regression methods. In the mixed model formulation shrinkage is controlled by the ratio of variance components, ${\sigma }_e^2/{\sigma }_k^2$; this ratio plays a role similar to the regularization parameter (λ) of penalized regressions (Verbyla et al. 1999),(Meyer 2005). Thus the mixed effects model is a penalized model. The variance parameter, ${\sigma }_k^2$, reflects the magnitude of among-population variation around the pooled population regression coefficients. Combining two specifications for the basis functions (B-spline and step function) and two models of the fitness function (pooled population, or fixed effects, versus population-specific, or random effects), we fit four regression models: fixed effects B-spline (FBS), random effects B-spline (RBS), fixed effects step function (FSF), and random effects step function (RSF). We fitted these models using the glm and glmer functions of the base and lme4 package of R (Bates, Maechler & Bolker 2012).

The RBS and RSF models assume independent random effects. In the case of the RSF model the basis functions are mutually orthogonal; therefore assuming independent random effects seems reasonable. However, the basis function of the RBS model are not mutually orthogonal; therefore, in this case, one may argue that the random effects may not be independent. A full account of the co-dispersion parameters includes the 15 pairwise covariances; unfortunately the REML algorithm did not converge with such specification. Consequently, in addition to the specification based on RBS basis function with independent random effects, we also considered an orthogonal representation of the RBS basis functions. This was obtained from the singular-value decomposition of the RBS basis functions (Φ = {ϕ₁(.), …, ϕ₆(.)}, an n x 6 matrix, where ϕ_k(.) represents the kth basis function), that is Φ = UΣV^T where U and V are the left and right singular vectors of Φ, respectively, and Σ is a diagonal matrix containing the singular values of Φ. The factorization was obtained with the “svd” function in R. We then refit the model using the principal components of Φ, that is UΣ, instead of Φ. Since the principal components are mutually orthogonal we treated the random effects associated to the columns of UΣ as independent. We compared this orthogonal representation with the one based on Φ. The models’ AIC criterions were very similar, with the orthogonal basis slightly higher than RBS. Furthermore the correlation of predicted values for the two parameterizations was 0.99 (data not shown). Therefore we proceeded with the RBS model as described above (see expression 2) with independent random effects.

Hypothesis testing

For all four models the test for the significance of individual variance parameters is $\text{Ho}:{\sigma }_k^2=0\text{vs.}\text{Ha}:{\sigma }_k^2\ne 0$. These tests were implemented as likelihood ratio tests (Stram & Lee 1994; Bolker et al. 2009). Rejecting this hypothesis indicates significant heterogeneity in the fitness functions among populations. For the FSF and RSF models, the interpretation of tests is particularly informative as each of the (k = 1 − N) likelihood tests is a test for heterogeneity among populations in survivorship for a specific gall-size interval.

Model comparison

We compared the FBS vs. RBS and the FSF vs. RSF models using a measure of prediction accuracy, quantified using the sum of squares of prediction errors, PSS = Σ_iΣ_j(y_ij − ŷ_ij)², where ŷ_ij are predictions derived from a 10-fold cross validation (CV), (Hastie, Tibshirani & Friedman 2009). Ideally a selected model that is not overfit will generalize well on naive data, that is, classify the correct response: adult wasp emergence (survival). In contrast, an overfit model will have low bias in predicting an observation in the training data, but the same model may have high variance for predicting new observations (Hastie, Tibshirani & Friedman 2009). We estimated PSS for each model (FBS, RBS, FSF, and RSF) for the 2002 and 2008 data separately. We also estimated PSS within ranges of gall size to obtain a local assessment of prediction accuracy. Prediction sums of squares was computed at foci consisting of every 10^th observation of the sorted gall size dataset, beginning with the 100^th gall and ending with the N – 100^th gall. In order to obtain an accurate measure, PSS was computed from the observed and predicted emergence probabilities on sets of 200 galls—100 galls above and 100 below each focal gall of the sorted dataset. These local PSS calculations facilitate a comprehensive evaluation of the extent of heterogeneity in fitness functions among populations, beyond inference based solely on the magnitude of variance components for the basis functions. For example, if the likelihood ratio test resulted in statistically significant variance components for each and every basis function, but the local PSS estimates indicated negligible reduction in PSS from the fixed to random model for galls whose size was less than the median, then one would be less inclined to reject the null hypothesis of homogeneity among population fitness functions except for larger galls. We provide example R scripts for model fitting and cross validation in the supporting information.

Results

Model fitting and visualization

We fit three model specifications—second-order polynomial, 6 DF B-spline, and step-function to visualize the covariation between fitness and phenotype. Figure 2 depicts predicted survival probability of the gall former as a function of gall size for each model specification at the pooled–study unit level (all observations pooled across all populations within each year: N₂₀₀₂= 7181; N₂₀₀₈ = 6787). In both years fitness covaried with gall size, indicating that viability selection on gall wasps was a function of gall size. For all three fixed model specifications (FBS, FSF, and second-order polynomial), a predicted fitness maximum occurred around intermediate-sized galls. The stepwise fitness functions generated by binning observations in equal percentiles of the gall size distribution (FSF model) depicts the forced discrete movement in probability of survival as gall size varies across intervals for each year. In both 2002 and 2008, the greatest disparity in predicted survival between the second-order polynomial and the FBS model specifications occurred for galls above the 75^th percentile of the size distribution. Nevertheless, in 2002 the FBS and second-order quadratic models give remarkably similar predicted survival for galls through the 75^th percentile of the gall size distribution. In contrast, in 2008 the FBS (and the FSF) appear to differ markedly from the second-order polynomial model predicted survival probability across all percentiles of gall size.

Graphical comparison of the predicted pooled fitness functions with population-specific fitness functions provides a way to visualize the qualitative differences between the fixed (FBS) and random (RBS) regression models. Figures 3 and 4 depict the RBS model fits for each population based on the random regression analysis conducted within each year. The substantial dispersion of the population-level fitness functions around the pooled mean fitness function in each year illustrates the heterogeneity in viability selection functions among populations in both years. The magnitude of fitness function variability among populations appears to be greatest in 2002. Moreover, heterogeneity among viability selection functions in 2002 is highest for gall formers within galls whose size is greater than the pooled median (Fig. 3 lower panel). Similar patterns are evident in 2008 although the dispersion of viability selection functions about the pooled median is less pronounced (Fig. 4 lower panel).

Figure 3.

Predicted fitness functions, P(Emergence), plotted for each of N = 11 populations (top panel) in 2002, and overlain onto the pooled viability selection function (averaged across all populations, heavy line) (bottom panel). The individual population fitness functions were estimated as population-level deviations for each basis function of the 6 DF B-spline random regression model fit to the 2002 data. Each fitness function was mapped from a grid of 100 values of gall size taken on even intervals of (min, max) gall size in 2002. Tick marks on the x-axis indicate the 0, 25, 50, 75, and 100th percentiles of the gall size distribution in each population (upper panel) and for the pooled data (bottom panel).

Figure 4.

Predicted fitness functions, P(Emergence) plotted for each of N = 5 populations (top panel) in 2008, and overlain onto the pooled (heavy line) viability selection function (bottom panel). Population fitness functions were estimated as population-level deviations for each basis function of the 6 DF B-spline random regression model fit to the 2008 data. Each fitness function was mapped from a grid of 100 values of gall size taken on even intervals of (min, max) gall size in 2008. Tick marks on the x-axis indicate the 0, 25, 50, 75, and 100th percentiles of the gall size distribution in each population (upper panel) and for the pooled data (bottom panel).

Table 1 presents model selection results for a comparison of the 6 DF B-spline implemented as a fixed (FBS) effects model and the second-order fixed effects polynomial model in relation to a null model that included only an intercept term. In both years the best fit (minimum deviance) was obtained using the FBS model, but this model is also more complex (i.e., it has more parameters). However, in both years the FBS model had the smaller AIC; therefore, based on AIC one should choose this model specification relative to the second-order quadratic from the candidate set. In addition, the change in AIC between FBS and second order quadratic was greater in 2008 and 2002, which confirms the interpretation suggested by visual inspection of model fits.

Hypothesis testing of variance components

The heterogeneity among populations in the relationship between phenotype and fitness suggested by graphical comparison of the RBS and FBS models in each of the two years (Figs. 3 and 4) was confirmed by the likelihood ratio tests. One DF likelihood ratio tests for the 2002 data demonstrated that two of the six B-spline basis functions (Bf₍₁₎ and Bf₍₄₎) and the intercept had variance component estimates significantly different than zero (Table 2). Similarly, tests of each variance component for the 2008 data set revealed that B-spline basis functions 3, 4, and 5 were statistically significant (Table 2). Therefore, for both the 2002 and the 2008 RBS models, hypothesis testing demonstrates substantial dispersion of individual population fitness functions about the pooled-population fitness function. At least some of the variability in 2002 was due to heterogeneity in the intercept, indicating variation in mean survival probability among populations, which was not evident in 2008.

Table 2.

Parameter estimates for the intercept and the 6 basis functions (Bf_1-6) from the 6 df B-spline random regression model (eqn [1], Methods) fitted to the 2002 and 2008 data. Fixed effect (±SE) and random effects variance estimates, $\widehat{{\mathit{\sigma }}^{\mathbf{2}}}$, are reported for each model. P-values are based on logLikelihood ratio tests of the full model versus the same model without the variance parameter of interest. Significant P values are bolded. There were 11 populations in 2002 and 5 populations in 2008.

Parameter	Fixed effect	$\widehat{{\sigma }^2}$	P-value
2002
Intercept	4.59 (0.768)	0.398	3.78E-06
Bf (1)	4.30(1.03)	1.28	4.93E-04
Bf (2)	5.57 (0.708)	0.0	--
Bf (3)	5.51 (0.764)	0.0	--
Bf (4)	5.64 (0.770)	0.353	4.10E-03
Bf (5)	4.50 (0.826)	0.459	0.0642
Bf (6)	6.45 (1.07)	2.17	0.177
2008
Intercept	36.8(5.23)	0.0	--
Bf (1)	34.6 (5.56)	0.139	0.627
Bf (2)	38.3 (5.18)	0.0	--
Bf (3)	38.6 (5.27)	0.721	1.46E-06
Bf (4)	36.9 (5.23)	0.098	3.51E-03
Bf (5)	37.4(5.28)	1.17	1.23E-02
Bf (6)	36.0 (5.26)	0.299	0.828

The RSF model allowed us to test directly those regions within the phenotype distribution that exhibited significant among-population variation in fitness (Supplemental figures S2, S3 and Supplemental table ST1). As the basis functions in the RSF model consisted of indicator variables designating the presence or absence of galls in a specific size bin, predicted emergence has the form of a step function. To identify the specific regions of the phenotype distribution harboring significant among population variability in predicted survival of the gall former, we compared the log likelihoods of nested models with and without specific gall size bins (Supplemental table ST1). In the 2002 RSF model each of the seven variance components differed significantly from zero, indicating among population variability in predicted survival for all gall size bins. The largest magnitude variance components observed were for the smallest and the largest percentiles of the gall size distribution. Therefore across the pooled sample (Fig. 2) it was at the tails of the phenotypic distribution that survival probability was lowest and selection was most variable among populations. As well, there was a general trend in 2002 for the magnitudes of the variance components to increase as gall size increased from Bf₍₂₎ to Bf ₍₇₎. This trend is consistent with the results of the RBS model (Table 2), that showed increasing disparity among fixed and random effects models with increasing gall size.

In 2008, survival predicted from the RSF model varied across populations for all gall size bins except the 2^nd–3^rd (P > 0.05), (Supplemental table ST1). Also consistent with the 2002 results, the largest magnitude variance components observed were for the smallest and the largest gall size bins. However, the overall magnitude of the variance was lower in 2008 than in 2002 (Supplemental figures S2, S3), also similar to the result from the BSF models.

Model comparison

The global 10-fold cross-validation prediction error for the RBS model was lower than that of the FBS model in both study years (Fig. 5). The substantial improvement in model fit gained by estimating population-specific deviations from the pooled (average) fitness function was most evident in 2002, where the RBS prediction error was 16% lower than the FBS.

Figure 5.

Global 10-fold cross-validation prediction error from fixed effect B-spline and random regression B-spline models fit to the 2002 and 2008 data.

Local cross-validation prediction errors computed as a function of gall size for B-spline regression models confirm the result that population-level deviations from the pooled population mean fitness function in each year are both substantial and systematic (Fig. 6). In 2002, prediction error of the RBS model was generally lower than that of the FBS model except for the smallest galls. The deviation appears to be greatest for the galls in the upper 50% percentile of the gall size distribution. In 2008 the deviation is also greatest for galls greater than the median. However, for 2008 there is almost no difference between the fixed and random effects models for galls less than the median, and the overall magnitude of the deviation is far lower than in 2002. In this regard the results of the 10-fold cross validation are not completely consistent with the results of the variance components tests from the RBS and RSF models (Table 2 and S1).

Figure 6.

Local 10-fold cross-validation prediction error for 2002 and 2008 compared between fixed B-spline and random regression B-spline models. Sum of squares error was calculated at every 50^th gall of the sorted gall size distribution beginning with the 100^th observation and ending at the N 100^th observation. Prediction error calculations were based on N = 200 galls (100 upstream and downstream of each focal gall).

Discussion

Correctly identifying heterogeneity in selection among study units especially when coupled with explorations of the ecological basis for fitness variation provide the foundation for understanding how selection fueled by intraspecific genetic variation can generate a diverse array of ecological and evolutionary processes including local adaptation (Orr 2005; Barrett & Schluter 2008; Savolainen, Lascoux & Merilä 2013), incipient speciation (Schluter & Conte 2009), and community assembly (Hersch-Green, Turley & Johnson 2011; Farkas et al. 2013; Martin & Wainwright 2013). We employed a novel application of random regression analysis using B-splines to quantify and test the hypothesis of heterogeneity in fitness functions among populations (Figs. 3 and 4). We then inferred the regions within the distribution of phenotypic values that exhibited the greatest among-population fitness heterogeneity by 1) random regression analysis of fitness on discrete bins of the phenotype distribution (Supplemental figures S2 and S3) and 2) comparison of local 10-fold cross-validation estimates of PSS between the FBS and RBS models (Fig. 6). Using global 10-fold cross validation (Fig. 5) and likelihood ratio tests of individual variance components (Table 2) our framework for analysis demonstrated compelling evidence for significant among population variability in fitness functions within the example data of Egan, Hood & Ott (2011). A primary strength of the proposed analytic framework is that random regression models provide direct tests of the hypothesis that selection is variable across study units, which could represent multiple treatments, sites, populations, subpopulations, season, or years. Moreover, the random regression models considering variability in selection could be of any form. For example, while we used B-splines, any basis function or linear and (or) nonlinear fitness functions such as the Lande–Arnold second-order quadratic selection model could be assumed when one is confident that the model to be used adequately fits the data. For example, Engen (2012) considered the situation of fluctuating selection across age classes. In our analysis of the example data set, based on model selection, we found that the fitness functions for B. treatae were best considered as spline curves (Table 1). Thus B-spline transformation of the phenotype, gall size, was used to facilitate the fit of curves to data relating trait variation to fitness but with no pre-specified functional form.

The random regression B-spline model shares characteristics with the nonparametric cubic spline approach of Schluter (1988). However we have extended the spline-fitting approach by treating the coefficients of the B-splines as random deviates, which together define a potentially unique fitted curve in each population using information from all populations considered jointly. Applied to our example dataset, the random regression framework allowed us to reject the null hypothesis of no significant variance among host-plant-affiliated populations of the gall former in the form of the fitness functions relating probability of survival to gall size.

A second strength of our approach is the use of cross validation to estimate both local and global prediction error sums of squares in order to evaluate comprehensively the random regression models’ goodness of fit. We then used the cross-validation results in addition to model goodness-of-fit estimates (i.e., basis function parameter estimates) to further assess the evidence for fitness function heterogeneity. As the number of parameters in a model increases so too does the variance they potentially explain in the response, but at a price. Thus we wanted to avoid relying on a model that appears highly explanatory but at the expense of prediction (Hastie, Tibshirani & Friedman 2009). For example, likelihood ratio tests showed that selection is variable among populations in 2002 and 2008 for both RBS and RSF models. Therefore the hypothesis tests of variance components indicate a signal of extensive fitness function variability among-population particularly for the smallest and largest gall sizes. The variance components are statistically significant in both RBS and RSF model specifications, but do these models accurately predict fitness, as assessed by 10 fold cross validation?

The RBS models resulted in lower prediction error sums of squares compared with FBS models in both 2002 and 2008, indicating higher prediction accuracy, although the reduction in 2008 was small (Fig. 5). The 10-fold cross-validation results confirm the finding from variance component tests of extensive heterogeneity in selection among populations overall (Fig. 5) and pronounced heterogeneity in the tails of the trait distribution, but only in 2002 (Fig. 6). On this basis, we conclude that in 2002 population-level fitness functions are heterogeneous. This finding sets the stage for exploring the biological basis of differences in selection across the range of phenotypes and among populations. Here we have strong evidence within a species that selection can vary substantially among populations potentially indicating multiple adaptive peaks in the wild (Martin & Wainwright 2013).

Overall our results confirm the analysis by Egan, Hood & Ott (2011a) of the role of the host plant in generating variation in selection, with the exception that our model fit statistics based on prediction error from the random regression analysis were less optimistic regarding heterogeneity in fitness functions in 2008. We note that several significant variance components for the RBS and RSF models were observed in 2008, which suggests that there are indeed intervals of the gall size distribution where significant heterogeneity exists among populations in gall-former survival. Thus, population-level fitness functions as random deviates from the pooled population mean may most closely approximate the fitness functions’ true model. Nevertheless, overall prediction errors for the RBS and FBS models are nearly equal in 2008 (Fig. 5). Therefore we cannot rule out the possibility that there may be some bias present in 2008 due to over fitting, that is, the random regression model is fitting noise over signal. Furthermore, fitness function variability was tested using only 5 populations in 2008, which could have contributed to the smaller variance component estimates in 2008 and marginal predictive ability of the random regression model. Therefore a conservative interpretation of the results is that fitness function heterogeneity is evident in 2002, but that in 2008 fitness function heterogeneity appears much less pronounced and, if present, exists only for galls larger than the median.

Both the analysis of Egan, Hood & Ott (2011a), which involved estimation of selection gradients, tests of interactions between trait and fitness, and penalized cubic spline fitting and the current analysis reveal that B. treatae fitness functions are complex. However, application of the random regression B-spline approach to the gall size–fitness data demonstrates additional strengths of this approach and points out how different model specifications can lead to different conclusions. For example, testing hypotheses of heterogeneity using sites or years as covariates and coefficients of the interactions with untransformed traits as evidence (Table 2 in (Egan, Hood & Ott 2011a)) carries the clear assumption that the fitness function is second-order quadratic. However, our current analysis has shown that for both 2002 and 2008 the spline model is a better fit than the second-order polynomial. Therefore, while the estimates of linear and quadratic selection presented in (Egan, Hood & Ott 2011a) are reasonable given the model, the conclusions regarding host plant variation in selection, at least for 2008, do not appear robust to model specification. Thus appropriate models of the form of phenotypic selection appear essential to the conclusion regarding heterogeneity in fitness functions. Calsbeek (2012) also noted that failure to detect heterogeneity among years under the second-order polynomial model was probably due to its limitation in accurately portraying the true fitness surface.

The portfolio of tools for analyzing selection in natural populations has expanded greatly in recent years (Shaw et al. 2008; Shaw & Geyer 2010; Calsbeek 2012; Geyer et al. 2013; Morrissey & Sakrejda 2013; Morrissey 2014).The framework for analysis that we have outlined adds to this toolkit by providing a method to investigate heterogeneity in selection among populations and years for a single fitness component and one trait. We have presented a simple framework that can be deployed using conventional statistical software packages such as SAS and R. Future research could be directed toward random regression models with multiple traits and different fitness function model specifications. Calsbeek (2012) presented a tensor decomposition method that also quantifies heterogeneity among populations and years for multivariate fitness surfaces, each estimated with splines. Aster models (Geyer et al. 2013) can handle multiple fitness components and random covariates, therefore the same analyses presented here could in principle be made integrating across the life history of B. treatae. It is important to clarify that the main focus of the analysis we have presented has been to test for variation in the overall form of the fitness function among populations. We did not compute linear and nonlinear selection gradients based on the random regression models. However the conclusions we have drawn from the application of the random regression framework imply that linear and (or) nonlinear selection are variable among populations. We note that the method we have presented could be complexed with the approach presented by Morrissey & Sakrejda (2013) to estimate linear and nonlinear selection based on spline functions and thus simultaneously yield estimates of selection and tests of variability in selection among study units.

The random regression analysis using B-splines (or any basis function) has the potential to expand our knowledge of spatial or temporal heterogeneity in selection. Primary advantages of this approach are its accessibility, its ease of interpretation, and its ability to explicitly and efficiently test the hypothesis of heterogeneity of fitness functions among groups when studies of selection involve multiple populations, sites, or years. Importantly, comparison of fixed and random models can also be used to justify pooling of data across all group or levels having homogeneous fitness functions or guard against pooling when groups or levels exhibit heterogeneous fitness functions. When true heterogeneity in selection is not modeled and samples are pooled across populations to estimate selection, the resulting parameter estimates fail to accurately represent variation in selection among populations (Egan, Hood & Ott 2011a). Moreover, comparison of prediction error between fixed and random effects across the range of phenotypes can be used to distinguish that portion of the phenotype distribution harboring the majority of the variance in the fitness function, thereby facilitating hypothesis testing of the biological basis of fitness variation. The growing number of empirical studies of heterogeneity in selection in natural populations (Kingsolver et al. 2012) coupled with the increasing availability of public datasets call for continuing effort to advance methods that address a fundamental question in evolutionary biology: do complex fitness functions vary among populations?

Supplementary Material

Supp FigureS1-S3

Supplemental figure S1. Reproduced from Egan, Hood & Ott Evolution 65:3543-3557. Cubic spline (solid line) ± SE (dashed line) fit to survivorship as a function of gall diameter for 22 populations of Belonocnema treatae. The gall size–survival relationship was estimated for 17 populations in 2002 and 5 populations in 2008. Populations are arranged by sample size from top left to bottom right. The shaded histograms depict the observed distribution of gall diameters for each population. Note the change in the right y-axis scale for the frequency data to accommodate variable sample sizes. (Supplemental Fig 1.tif)

Supplemental figure S2. Predicted fitness, P(Emergence) as a piecewise function of gall size (mm) for each of N = 11 populations (top panel) from 2002, and overlain onto the pooled population-level fitness function (heavy line, bottom panel). Individual population fitness functions were estimated as population-level deviations at each of seven gall size bin basis functions using parameters of a random regression model (eqn (2), Methods) fit to the 2002 data. The piecewise function was mapped from a grid of 100 values of gall size taken on even intervals of (min, max) gall size in 2002. Tick marks on the x-axis indicate 0, 25, 50, 75, and 100th percentiles of the gall size distribution in each population (upper panel) and for the pooled data (bottom panel). (Supplemental Fig 2.tif)

Supplemental figure S3. Predicted fitness, P(Emergence) as a piecewise function of gall size (mm) for each of N = 5 populations (top panel) from 2008, and overlain onto the pooled population-level fitness function (heavy line, bottom panel). Individual population fitness functions were estimated as population-level deviations at each of seven gall size bin basis functions using parameters of a random regression model (eqn (2), Methods) fit to the 2008 data. The piecewise function was mapped from a grid of 100 values of gall size taken on even intervals of (min, max) gall size in 2008. Tick marks on the x-axis indicate 0, 25, 50, 75, and 100th percentiles of the gall size distribution in each population (upper panel) and for the pooled population data (bottom panel). (Supplemental Fig 3.tif)

Supp TableS1

Supplemental table ST1. Parameter estimates for the stepwise random regression models fit to the 2002 and 2008 data. (Supplemental table ST1.docx)

Summary.

1. Statistical approaches for testing hypotheses of heterogeneity in fitness functions are needed to accommodate studies of phenotypic selection with repeated sampling across study units, populations, or years. In this study we test directly for among population variation in complex fitness functions and demonstrate a new approach for locating the region of the trait distribution where variation in fitness and traits is greatest.

2. We modeled heterogeneity in fitness functions among populations by treating regression coefficients of fitness on traits as random variates. We applied random regression using two model specifications 1) spline-based curve, and 2) stepwise, to a two-year study of selection among 16 populations of the gall wasp, Belonocnema treatae. Log-likelihood ratio tests of variance components and 10-fold cross validation were used to assess the evidence that selection varied among populations.

3. Ten-fold cross-validation prediction error sums of squares (PSS) indicated that spline-based fitness functions were population specific and that the strength of evidence for heterogeneity in selection differed between years. Hypothesis testing of variance components from both models were consistent with the PSS results. Both the stepwise model and the local prediction error estimates of spline-based fitness functions identified the region(s) of the phenotype distribution harboring the greatest heterogeneity among populations.

4. The adopted framework advances our understanding of phenotypic selection in natural populations by extending the analysis of spline-based fitness functions to testing for heterogeneity among study units and isolating the regions of the phenotypic distribution where this variation is most pronounced.