Genomic-Enabled Prediction Based on Molecular Markers and Pedigree Using the Bayesian Linear Regression Package in R

Abstract

The availability of dense molecular markers has made possible the use of genomic selection in plant and animal breeding. However, models for genomic selection pose several computational and statistical challenges and require specialized computer programs, not always available to the end user and not implemented in standard statistical software yet. The R-package BLR (Bayesian Linear Regression) implements several statistical procedures (e.g., Bayesian Ridge Regression, Bayesian LASSO) in a unifi ed framework that allows including marker genotypes and pedigree data jointly. This article describes the classes of models implemented in the BLR package and illustrates their use through examples. Some challenges faced when applying genomic-enabled selection, such as model choice, evaluation of predictive ability through cross-validation, and choice of hyper-parameters, are also addressed.

Prediction of genetic values is a central problem in quantitative genetics. Accurate predictions of genetic values of genotypes whose phenotypes are yet to be observed (e.g., newly developed lines) are needed to attain rapid genetic progress and to reduce phenotyping costs (e.g., Bernardo and Yu, 2007). Over many decades, such predictions have been obtained using phenotypic and family data, the latter usually represented by a pedigree. However, pedigree-based models do not account for Mendelian segregation, a term that in an additive model and in the absence of inbreeding explains as much as one half of the genetic variance. This sets an upper limit on the accuracy of estimates of genetic values of individuals without progeny. Dense molecular markers (MM) are now available in the genome of humans and of many plant and animal species. Unlike pedigree data, MM allow tracing back Mendelian segregation events at many points along the genome. Potentially, this information can be used to improve the accuracy of estimates of genetic values of newly developed lines.

Following the ground-breaking contribution of Meuwissen et al. (2001), genomic selection (GS) has gained ground in plant and animal breeding (e.g., Bernardo and Yu, 2007; Hayes et al., 2009; VanRaden et al., 2009; de los Campos et al., 2009; Crossa et al., 2010). In practice, implementing GS involves analyzing large amounts of phenotypic and MM data and requires specialized computer programs. The main purpose of this article is to show how the R-package (R Development Core Team, 2009) BLR (Bayesian Linear Regression, de los Campos and Pérez, 2010) can be used to implement several models for GS. A first version of the algorithms and the R-code was presented in de los Campos et al. (2009). The package was developed further and its performance was significantly improved by the first two authors of this article; the package and data set are available from the R website (http://www.r-project.org; verified 29 July 2010). We provide a brief overview of parametric models for GS and describe the type of models implemented in BLR. We show two applications that illustrate the use of the package and several features of the models implemented in it.

Parametric Models for Genomic Selection

In parametric models for GS (e.g., Meuwissen et al., 2001), phenotypic outcomes, y_i (i = 1,…,n), are regressed on marker covariates x_ij (j = 1,…,p) using a linear model of the form $y_i={\sum }_{j=1}^p{x}_{ij}{\beta }_j+{\epsilon }_i$, where β_j is the regression of y_i on the jth marker covariate and ε_i is a model residual. In matrix notation, the regression model is expressed as y = Xβ + ε, where y = {y_i}, X = {x_ij}, β = {β_j} and ε = {ε_i}. The number of molecular markers (p) is usually larger than the number of observations (n) and, because of this, estimation of marker effects via multiple regression by ordinary least squares (OLS) is not feasible. Instead, penalized estimation methods such as ridge regression (RR, Hoerl and Kennard, 1970), or the Least Absolute Shrinkage and Selection Operator (LASSO; Tibshirani, 1996) or Bayesian methods such as those of Meuwissen et al. (2001) or the Bayesian LASSO (BL) of Park and Casella (2008) (e.g., Yi and Xu, 2008; de los Campos et al., 2009) can be used to estimate marker effects.

In RR, estimates of the effects of MM are obtained as the solution to the following optimization problem:

$$\widehat{\mathbf{\beta }}=\underset{\mathbf{\beta }}{\mathrm{argmin}}\left\{\sum _{i=1}^n{\left(y_i-{{\mathbf{x}}_i}^{\prime }\mathbf{\beta }\right)}^2+\tilde{\lambda }\sum _{j=1}^p{\beta }_j^2\right\}.$$

Here, $\tilde{\lambda }\ge 0$ is a regularization parameter controlling the trade-offs between goodness of fit measured by the residual sum of squares, ${\sum }_{i=1}^n{\left(y_i-{{\mathbf{x}}_i}^{\prime }\mathbf{\beta }\right)}^2$, and model complexity measured by the sum of squared marker effects (${\sum }_{j=1}^p{\beta }_j^2$). The first order conditions of the above optimization problem are satisfied by $\left[{\mathbf{X}}^{\prime }\mathbf{X}+\tilde{\lambda }\mathbf{I}\right]\widehat{\mathbf{\beta }}={\mathbf{X}}^{\prime }\mathbf{y}$; equivalently, $\widehat{\mathbf{\beta }}={\left[{\mathbf{X}}^{\prime }\mathbf{X}+\tilde{\lambda }\mathbf{I}\right]}^{-1}{\mathbf{X}}^{\prime }\mathbf{y}$. Relative to OLS, RR adds a constant, $\tilde{\lambda }$, to the diagonal of the matrix of coefficients; this makes the solution unique and shrinks estimates of marker effects towards zero, with the extent of shrinkage increasing as $\tilde{\lambda }$ increases (with $\tilde{\lambda }=0$ the solution to the above problem is the OLS estimate of marker effects).

From a Bayesian perspective, $\widehat{\mathbf{\beta }}$ can be viewed as the conditional posterior mode in a model with Gaussian likelihood and IID (independent and identically distributed) Gaussian marker effects, that is,

$$\{\begin{array}{cc}\text{Likelihood}:\hfill & p\left(\mathbf{y}\mid \mathbf{\beta },{\sigma }_{\epsilon }^2\right)=\prod _{i=1}^{n}N\left[y_i\mid \sum _{j=1}^p{x}_{ij}{\beta }_j,{\sigma }_{\epsilon }^2\right]\hfill \\ \text{Prior}:\hfill & p\left(\mathbf{\beta }\mid {\sigma }_{\beta }^2\right)=\prod _{i=1}^{n}N\left({\beta }_j\mid 0,{\sigma }_{\beta }^2\right)\hfill \end{array}\phantom{\}}$$ [1]

Above, ${\sigma }_{\beta }^2$ is the a priori variance of marker effects. The prior distribution of marker effects becomes increasingly informative and concentrated around zero as ${\sigma }_{\beta }^2$ decreases. The posterior mean and mode of β from [1] is equal to the ridge-regression estimate, with $\tilde{\lambda }={\sigma }_{\beta }^{-2}{\sigma }_{\epsilon }^2$, that is, $E\left(\mathbf{\beta }\mid y\right)=\widehat{\mathbf{\beta }}={\left[{\mathbf{X}}^{\prime }\mathbf{X}+{\sigma }_{\beta }^{-2}{\sigma }_{\epsilon }^2\mathbf{I}\right]}^{-1}{\mathbf{X}}^{\prime }\mathbf{y}$. Therefore, prediction of genetic values from [1] can be obtained as

$$\begin{array}{cc}\hfill E\left(\mathbf{X}\mathbf{\beta }\mid \mathbf{y},{\sigma }_{\epsilon }^2{\sigma }_{\beta }^2\right)& =\mathbf{X}E\left(\mathbf{\beta }\mid \mathbf{y},{\sigma }_{\epsilon }^2{\sigma }_{\beta }^2\right)\hfill \\ \hfill & =\mathbf{X}{\left[{\mathbf{X}}^{\prime }\mathbf{X}+{\sigma }_{\epsilon }^2{\sigma }_{\beta }^{-2}\mathbf{I}\right]}^{-1}{\mathbf{X}}^{\prime }\mathbf{y}\hfill \end{array}$$ [2a]

Alternatively, from [1] and using properties of the multivariate normal distribution, one has:

$$\begin{array}{cc}\hfill E\left(\mathbf{X}\mathbf{\beta }\mid \mathbf{y},{\sigma }_{\epsilon }^2,{\sigma }_{\beta }^2\right)=& \mathit{Cov}\left(\mathbf{X}\mathbf{\beta },{\mathbf{y}}^{\prime }\right)\mathrm{Var}{\left(\mathbf{y}\right)}^{-1}\mathbf{y}\hfill \\ \hfill =& {\mathbf{XX}}^{\prime }{\sigma }_{\beta }^2{\left[{\mathbf{XX}}^{\prime }{\sigma }_{\beta }^2+{\sigma }_{\epsilon }^2\mathbf{I}\right]}^{-1}\mathbf{y}\hfill \\ \hfill =& {\mathbf{XX}}^{\prime }{\left[{\mathbf{XX}}^{\prime }+{\sigma }_{\epsilon }^2{\sigma }_{\beta }^{-2}\mathbf{I}\right]}^{-1}\mathbf{y}\hfill \end{array}$$ [2b]

Formulae [2a] and [2b] are equivalent and correspond to the Best Linear Unbiased Predictor (BLUP) under the model defined by [1]. Note that [2a] and [2b] require solving systems of p and n equations, respectively. Therefore, with p >> n, expression [2b] is computationally more convenient. However, unlike [2a], expression [2b] does not yield estimates of marker effects.

In RR-BLUP, estimates of marker effects are penalized to the same extent, and this may not be appropriate if some markers are located in regions not associated with genetic variance whereas others are linked to QTLs (Goddard and Hayes, 2007). To overcome this limitation, methods performing variable selection and shrinkage (e.g., LASSO) or Bayesian methods using marker-specific shrinkage of effects, such as methods BayesA and BayesB of Meuwissen et al. (2001) or the BL of Park and Casella (2008), have been proposed.

The BLR package implements Bayesian regression with marker-specific or marker homogenous shrinkage of estimates effects. The package allows inclusion of covariates other than markers and regressions on a pedigree as well. In BLR, phenotypes are expressed as follows:

$$\mathbf{y}=\mathbf{1}\mu +{\mathbf{X}}_{\mathrm{F}}{\mathbf{\beta }}_{\mathrm{F}}+\mathbf{Zu}+{\mathbf{X}}_{\mathrm{R}}{\mathbf{\beta }}_{\mathrm{R}}+{\mathbf{X}}_{\mathrm{L}}{\mathbf{\beta }}_{\mathrm{L}}+\mathbf{\epsilon }$$ [3]

where y, the response, is a (n × 1) vector (missing values are allowed); μ is an intercept; ${\mathbf{X}}_{\mathrm{F}}={\left\{x_{Fij}\right\}}_{n\times p_F}$, $\mathbf{Z}={\left\{z_{ij}\right\}}_{n\times p_U}$, ${\mathbf{X}}_{\mathrm{R}}={\left\{x_{Rij}\right\}}_{n\times p_R}$, and ${\mathbf{X}}_{\mathrm{L}}={\left\{x_{Rij}\right\}}_{n\times p_L}$ are incidence matrices for the vectors of effects β_F, u, β_R, and β_L, whose dimensions are p_F, p_U, p_R, and p_L, respectively. These vectors of effects differ with respect to the prior distributions assigned, as discussed later on. Finally, ε is a vector of model residuals assumed to be distributed as $\mathbf{\epsilon }\sim N\left(0,\mathrm{Diag}\left\{\frac{{\sigma }_{\epsilon }^2}{w_i^2}\right\}\right)$, where ${\sigma }_{\epsilon }^2$ is an (unknown) variance parameter and the w_i’s are (known) weights that allow for heterogeneous-residual variances. From these assumptions, the conditional distribution of the data, given the location effects, the residual variance and the weights, is:

$$\begin{array}{cc}\hfill p& \left(\mathbf{y}\mid \mu ,{\mathbf{\beta }}_{\mathrm{F}},\mathbf{u},{\mathbf{\beta }}_{\mathrm{R}},{\mathbf{\beta }}_{\mathrm{L}}\right)\hfill \\ \hfill & =\prod _{i=1}^{n}N\left(\begin{array}{c}{y}_i\mid \mu +\sum _{j=1}^{p_{\mathrm{F}}}{x}_{\mathrm{F}\mathrm{i}\mathrm{j}}{\beta }_{\mathrm{F}\mathrm{j}}+\sum _{j=1}^{p_U}{z}_{ij}{\mathrm{u}}_j+\sum _{j=1}^{p_R}{x}_{\mathrm{R}\mathrm{i}\mathrm{j}}{\beta }_{\mathrm{R}\mathrm{j}}\hfill \\ +\sum _{j=1}^{p_{\mathrm{L}}}{x}_{\mathrm{L}\mathrm{i}\mathrm{j}}{\beta }_{\mathrm{L}\mathrm{j}},\frac{{\sigma }_{\epsilon }}{w_i^2}\hfill \end{array}\right)\hfill \end{array}$$ [4]

Any of the elements on the right-hand side of [3], except μ and ε, can be excluded in BLR; by default, the program runs an intercept model, i.e., y_i = μ + ε_i. If weights are not provided, all weights are set equal to one.

The Bayesian model is completed by assigning a prior to all model unknowns, $\mathbf{\theta }=\left\{\mu ,{\mathbf{\beta }}_{\mathrm{F}},\mathbf{u},{\sigma }_{\mathrm{u}}^2,{\mathbf{\beta }}_{\mathrm{R}},{\sigma }_{{\beta }_{\mathrm{R}}}^2,{\mathbf{\beta }}_{\mathrm{L}},{\tau }^2,\lambda ,{\sigma }_{\epsilon }^2\right\}$. The intercept, μ, and the vector of “fixed” effects, β_F, are assigned flat priors, that is, p(μ, β_F) ∝ constant. This treatment yields posterior means of these unknowns that are similar to those obtained with OLS, provided that μ and β_F are the only effects included in the model.

The vector u is modeled using the standard assumptions of the infinitesimal additive model (e.g., Fisher, 1918; Wright, 1921; Henderson, 1975), that is, $p\left(\mathbf{u}\mid {\sigma }_u^2\right)=N\left(\mathbf{0},\mathbf{A}{\sigma }_u^2\right)$, where A is a positive-definite matrix (usually a numerator relationship matrix computed from a pedigree) and ${\sigma }_{\mathrm{u}}^2$ is an unknown variance, whose prior is a scaled inverse-χ² density with degrees of freedom df_u and scale S_u, that is, ${\sigma }_{\mathrm{u}}^2\sim {\chi }^{-2}\left({\sigma }_{\mathrm{u}}^2\mid {\mathrm{df}}_{\mathrm{u}},S_{\mathrm{u}}\right)$; the hyper-parameters are user provided. In the parameterization used in BLR, $E\left({\sigma }_{\mathrm{u}}^2\mid {\mathrm{df}}_{\mathrm{u}},S_{\mathrm{u}}\right)=S_{\mathrm{u}}{\left({\mathrm{df}}_{\mathrm{u}}-2\right)}^{-1}$. The multivariate normal prior assigned to u induces shrinkage of estimates of effects u_j toward zero and borrowing of information between levels of the random effect, $\left\{u_j,u_{j^{\prime }}\right\}$.

The vector of regression coefficients β_R is assigned a Gaussian prior with variance common to all effects, that is, $p\left({\mathbf{\beta }}_{\mathrm{R}}\mid {\sigma }_{{\beta }_{\mathrm{R}}}^2\right)={\prod }_{j=1}^{p_R}N\left({\beta }_{\mathrm{R}j}\mid 0,{\sigma }_{{\beta }_{\mathrm{R}}}^2\right)$. This prior induces estimates that are the Bayesian counterpart of those obtained with Ridge Regression; we refer to this as “Bayesian Ridge Regression” (BRR). The variance parameter, ${\sigma }_{{\beta }_{\mathrm{R}}}^2$, is treated as unknown and is assigned a scaled inverse-χ² prior density, that is, $p\left({\sigma }_{{\beta }_{\mathrm{R}}}^2\right)={\chi }^{-2}\left({\sigma }_{{\beta }_{\mathrm{R}}}^2\mid {\mathrm{df}}_{{\beta }_{\mathrm{R}}},S_{{\beta }_{\mathrm{R}}}\right)$ with degrees of freedom, df_βR, and scale, S_βR, provided by the user.

The vector of regression coefficients β_L is treated as in the Bayesian LASSO of Park and Casella (2008); the conditional prior distribution of marker effects, $p({\mathbf{\beta }}_{\mathrm{L}}\mid {\tau }^2,{\sigma }_{\epsilon }^2)$, is Gaussian with marker-specific prior variances, that is, $p({\mathbf{\beta }}_{\mathrm{L}}\mid {\mathbf{\tau }}^2,{\sigma }_{\epsilon }^2)={\prod }_{j=1}^{p_L}N\left({\mathbf{\beta }}_{\mathrm{L}j}\mid 0,{\sigma }_{\epsilon }^2{\tau }_j^2\right)$. This prior induces marker-specific shrinkage of estimates of effects, whose extent depends on ${\tau }_{\mathrm{j}}^{-2}$. The variance parameters, ${\tau }_{\mathrm{j}}^2$, are assigned exponential IID priors, $p\left({\mathbf{\tau }}^2\mid \lambda \right)={\prod }_{j=1}^{p_L}\mathrm{Exp}\left({\tau }_j^2\mid {\lambda }^2\right)$. Finally, in the BLR package, the prior distribution of the regularization parameter λ, p(λ), can be:

1. a mass-point at some value (i.e., fixed λ),

2. p(λ² )~ Gamma(r,δ), as suggested by Park and Casella (2008), or

3. $p\left(\lambda \mid \max ,{\alpha }_1,{\alpha }_2\right)\propto \mathrm{Beta}\left(\phantom{\mid }\frac{\lambda }{\max }\mid {\alpha }_1,{\alpha }_2\right)$ (de los Campos et al., 2009).

With the above assumptions, the marginal prior of regression coefficients β_Lj, $p\left({\beta }_{\mathrm{L}j}\mid \lambda \right)=\int N\left({\beta }_{\mathrm{L}j}\mid 0,{\sigma }_{\epsilon }^2{\tau }_{\mathrm{j}}^2\right)\mathrm{Exp}\left({\tau }_{\mathrm{j}}^2\mid {\lambda }^2\right)\partial {\tau }_{\mathrm{j}}^2$, is Double-Exponential (DE). Figure 1 displays the Gaussian and Double-Exponential density functions of random variables with zero mean and unit variance. Relative to the Gaussian, the DE distribution places a higher density at zero and thicker tails, inducing stronger shrinkage of estimates for markers with relatively small effect and less shrinkage of estimates for markers with sizable effect.

Figure 1.

Gaussian and Double-Exponential densities, both for a random variable with zero mean and unit variance.

Finally, the residual variance is assigned a scaled inverse-χ² prior density with degrees of freedom, df_ε, and scale parameter, S_ε, provided by the user that is:

$$p\left({\sigma }_{\epsilon }^2\right)={\chi }^{-2}\left({\sigma }_{\epsilon }^2\mid {\mathrm{df}}_{\epsilon },S_{\epsilon }\right).$$

Collecting the aforementioned assumptions, the prior distribution in BLR is:

$$\begin{array}{cc}\hfill p\left(\mathbf{\theta }\right)\propto & N\left(\mathbf{u}\mid 0,\mathbf{A}{\sigma }_{\mathrm{u}}^2\right)\prod _{j=1}^{p_{\mathrm{R}}}N\left({\beta }_{\mathrm{R}j}\mid 0,{\sigma }_{{\beta }_{\mathrm{R}}}^2\right){\chi }^{-2}\left({\sigma }_{{\beta }_{\mathrm{R}}}^2\mid {\mathrm{df}}_{{\beta }_{\mathrm{R}}},S_{{\beta }_{\mathrm{R}}}\right)\hfill \\ \hfill & \times \left\{\prod _{j=1}^{p_{\mathrm{L}}}N\left({\beta }_{\mathrm{L}j}\mid 0,{\sigma }_{\epsilon }^2{\tau }_{\mathrm{j}}^2\right)\prod _{j=1}^{p_{\mathrm{L}}}\mathrm{Exp}\left({\tau }_{\mathrm{j}}^2\mid {\lambda }^2\right)\right\}p\left(\lambda \right)\hfill \\ \hfill & \times {\chi }^{-2}\left({\sigma }_{\epsilon }^2\mid {\mathrm{df}}_{\epsilon },S_{\epsilon }\right)\hfill \end{array}$$

The prior distribution is indexed by several hyper-parameters; in the Appendix, we provide guidelines for choosing these parameters on the basis of prior expectations about the proportion of phenotypic variance that can be attributed to each of the components on the righthand side of [3].

The posterior distribution of model unknowns is proportional to the product of the likelihood and the prior distribution, that is:

$$\begin{array}{cc}\hfill p(\mathbf{\theta }\mid \mathbf{y})\propto & \prod _{i=1}^{n}N\left(y_i\mid \mu +\sum _{j=1}^{p_{\mathrm{F}}}{x}_{\mathrm{F}ij}{\beta }_{\mathrm{F}\mathrm{j}}+\sum _{j=1}^{p_{\mathrm{U}}}{z}_{ij}{u}_{\mathrm{j}}+\sum _{j=1}^{p_{\mathrm{R}}}{x}_{\mathrm{R}ij}{\beta }_{\mathrm{R}j}+\sum _{j=1}^{p_{\mathrm{L}}}{x}_{\mathrm{L}ij}{\beta }_{\mathrm{L}\mathrm{j}},\frac{{\sigma }_{\epsilon }}{w_{\mathrm{i}}^2}\right)\hfill \\ \hfill & \times N\left(\mathbf{u}\mid 0,\mathbf{A}{\sigma }_{\mathrm{u}}^2\right)\hfill \\ \hfill & \times \prod _{j=1}^{p_{\mathrm{R}}}N\left({\beta }_{\mathrm{R}\mathrm{j}}\mid 0,{\sigma }_{{\beta }_{\mathrm{R}}}^2\right){\chi }^{-2}\left({\sigma }_{{\beta }_{\mathrm{R}}}^2\mid {\mathrm{df}}_{{\beta }_{\mathrm{R}}},S_{{\beta }_{\mathrm{R}}}\right)\hfill \\ \hfill & \times \prod _{j=1}^{p_{\mathrm{L}}}N\left({\beta }_{\mathrm{L}\mathrm{j}}\mid 0,{\sigma }_{\epsilon }^2{\tau }_{\mathrm{j}}^2\right)\prod _{j=1}^{p_{\mathrm{L}}}\mathrm{Exp}\left({\tau }_{\mathrm{j}}^2\mid {\lambda }^2\right)p\left(\lambda \right)\hfill \\ \hfill & \times {\chi }^{-2}\left({\sigma }_{\epsilon }^2\mid {\mathrm{df}}_{\epsilon },S_{\epsilon }\right)\hfill \end{array}$$ [5]

This posterior distribution does not have a closed form; however, a Gibbs sampler can be used to draw samples from it. The Gibbs sampler is as in de los Campos et al. (2009) but extended to accommodate “fixed” effects and BRR.

Using Bayesian Linear Regression

This section gives two examples that illustrate the use of the BLR package and describe features of the models. It is assumed that the reader is familiar with the R-language/environment. In Example 1, we study the impact of different shrinkage methods (BRR vs. BL) using simulated data. Example 2 illustrates how the package can be used to implement cross-validation (CV) using Bayesian methods. Cross-validation can be used for model comparison (e.g., to compare predictive ability of pedigree-based models versus marker-based models) or for selecting model parameters (e.g., λ in the BL). Both examples make use of a wheat data set made available with the package, whose main features are described next.

Wheat Data Set

These data contain a historical set of 599 wheat lines from CIMMYT’s Global Wheat Program that were genotyped for 1447 Diversity Array Technology markers (DArT, Triticarte Pty. Ltd, Canberra, Australia; http://www.triticarte.com.au; verified 26 July 2010) and evaluated for grain yield (GY) in four macroenvironments. The dataset becomes available in the R environment by running the following R-code:

library(BLR)
data(wheat)

Function library() loads the package, and data() loads datasets included in the package into the environment. The above code loads the following objects [type objects() in the R console to list them] into the environment: - Y, a matrix (599 × 4) containing the 2-yr average grain yield of each of these lines in each of the four environments (phenotypes were standardized to a unit variance within each environment); - A (599 × 599) is a numerator relationship matrix computed from a pedigree that traced back many generations. This relationship matrix was derived using the Browse application of the International Crop Information System (ICIS), as described in http://cropwiki.irri.org/icis/index.php/TDM_GMS_Browse; verified 26 July 2010 (McLaren, 2005); - X (599 × 1279) is a matrix with DArT genotypes; data are from pure lines and genotypes were coded as 0/1 denoting the absence/presence of the DArT. Markers with a minor allele frequency lower than 0.05 were removed, and missing genotypes were imputed with samples from the marginal distribution of marker genotypes, that is, x_ij ~ Bernoulli ( ${\widehat{p}}_{\mathrm{j}}$ ), where ${\widehat{p}}_{\mathrm{j}}$ is the estimated allele frequency computed from the nonmissing genotypes. The number of DArT MMs after edition was 1279. - sets (599 × 1) is a vector that assigns observations to 10 disjoint sets; the assignment was generated at random. This is used later to conduct a 10-fold CV.

Example 1: The Nature of Different Shrinkage Methods

As stated, BRR or BL use different priors for marker effects; this induces different types of shrinkage of estimates of such effects. The simulation presented in this section aims at illustrating these differences. Data were generated using marker genotypes from the wheat dataset (X) with marker effects and model residuals simulated as described below.

Data Simulation

Data were simulated under an additive model of the form, $y_i=\mu +{\sum }_{j=1}^{1279}{x}_{ij}{\beta }_{\mathrm{j}}+{\epsilon }_{ij}$, i = 1,…,599, where μ = 100 is an effect common to all individuals; {x_ij} are marker genotypes from a collection of wheat lines described previously;{β_j} are marker effects; and ε_i ~ N(ε_i|0,1) are IID standard normal residuals. We assumed that most markers (1267) had a relatively small effect and that only a few markers (12) had a sizable effect. Specifically, marker effects were sampled from the following mixture model:

$${\beta }_{\mathrm{j}}=\{\begin{array}{c}\text{Uniform}(-0.5,-0.8)\text{if}j\in \{3,235,467,699,931,1163\}\hfill \\ \text{Uniform}(0.5,0.8)\text{if}j\in \{119,351,583,815,1047,1279\}\hfill \\ N(0,k)\text{otherwise}.\hfill \end{array}\phantom{\}}$$

where k = 1279⁻¹. The R-code used to implement this simulation was:

set.seed(12345)
data(wheat)
p<-ncol(X)
nQTL<-12
n<-nrow(X)
b0<-rnorm(p,sd=1/sqrt(p))
isQTL<-seq(from=3,to=p,length=nQTL)
b0[isQTL]<-rep(c(−1,1),times=nQTL/2)*
           runif(min=.5,max=.8,n=12)
yHat0<-100+X%*%b0
e0<-rnorm(n,sd=1)
y<-yHat0+e0

Function set.seed() initializes the random number generator; X is the matrix with information on molecular markers. Functions rnorm() and runif() generate random draws from the uniform and normal distributions, respectively. Figure 2 shows realized marker effects obtained with the above R-code. In this example, the sample variance of phenotypes (y) was 1.78, and the ratio of the sample variance of genetic values relative to phenotypic variance was 0.43.

Figure 2.

Realized marker effects.

Choice of Hyper-Parameters

The Appendix provides guidelines on how to choose values of hyper-parameters. It is assumed that the user has prior beliefs about the proportion of phenotypic variance that can be attributed to each of the components of the regression. In the simulation, the variance of phenotypes was about 1.78, and the variance of model residuals was 1. In practice, one does not know the true proportion of phenotypic variance that can be assigned to the genetic signal and model residuals, unless a precise estimate of heritability is available. Suppose our prior belief is that 50% of the phenotypic variance can be attributed to the genetic signal. Using df_ε = 3 and ${\sigma }_{\epsilon }^2=0.9$ = 0.9 in formula [1A] of the appendix, we set S_ε = 4.5. Values of hyper-parameters of the prior distribution of ${\sigma }_{{\beta }_{\mathrm{R}}}^2$ and λ² can be chosen using formulae [3A] and [4A] in the Appendix. Using these formulae requires computing the sum of squares (over markers) of the average genotype, that is, ${\sum }_{j=1}^p{\stackrel{‒}{x}}_{\mathrm{j}}^2$, where ${\stackrel{‒}{x}}_{\mathrm{j}}^2=n^{-1}{\sum }_{i=1}^n{x}_{\mathrm{i}\mathrm{j}}$. In the wheat dataset, this quantity is approximately 504; using this and df_βR = 3, V_R = 0.9 in formula [3A], we set $S_{{\beta }_{\mathrm{R}}}^2=\frac{4.5}{504}\approx 0.009$. Finally, 504 using ${\sigma }_{\epsilon }^2{V}_{\mathrm{L}}=1$ in formula [4A] of the Appendix, we find $\widehat{\lambda }\approx 32$. Choosing λ² ~ G(λ²|r = 2×10⁻⁵,δ = 0.52) gives a prior density for λ that has high density and is relatively flat around $\widehat{\lambda }\approx 32$ (Fig. 3).

Figure 3.

Prior density of λ when λ² is assigned a gamma prior with rate equal to 2 × 10⁻⁵ and shape equal to 0.52. When λ² ~ G(r,δ), $p\left(\lambda \mid r,\delta \right)=G\left({\lambda }^2\mid r,\delta \right)\Vert \frac{\partial {\lambda }^2}{\partial \lambda }\Vert =G\left({\lambda }^2\mid r,\delta \right)2\lambda$; this is the density displayed above.

Fitting the Model

Using the aforementioned values of hyper-parameters, BL and BRR were fitted using the following R-code:

prior=list(
   varE=list(S=4.5,df=3),
   varBR=list(S=.009,df=3),
   lambda=list(type="random",
              value=30,shape=.52,rate=2e–5))

nIter<-60000
burnIn<-10000

fmR<-BLR(y=y,XR=X,nIter=nIter,burnIn=burnIn,
         thin=10,saveAt="R _ ",prior=prior)
dput(fmR,file="fmR.out")

fmL<-BLR(y=y,XL=X,nIter=nIter,
         burnIn=burnIn,thin=10,
	 saveAt="L _ ",prior=prior)
	 dput(fmL,file="fmL.out")

In the above code, y is the response vector, X is the matrix of genotypes, and nIter and burnIn define the number of iterations and burn-in period, respectively, used in the Gibbs sampler. The prior is provided as a list; type “help(BLR)” in the R console for more details. The BLR function returns a list with posterior means, posterior standard deviations, and Deviance Information Criterion (DIC, Spiegelhalter et al., 2002). Function dput() saves this list to the hard drive. The fitted model can then be retrieved using function dget(). In addition to returning posterior means and posterior standard deviations, as the Gibbs sampler runs, samples of the intercept, of the fixed effects, of the variance parameters and of λ are saved to the hard-drive using the thinning specified by the user (which is set to 10 by default).

Results of Example 1

We ran the processes in an Intel Xeon 5530 2.4 GHz Quad Core Processor (R was executed in a single thread) with 6 GB of RAM memory. With this dataset (599 subjects, 1279 markers) the process took about 1% of RAM memory. BRR took about 5.5 s for every 1000 iterations of the sampler; BL takes about twice as much time.

Figure 4 shows the estimated posterior density of the residual variance for each of the models; the prior density (up to a constant) is included as well. The posterior distributions moved away from the prior and were sharp. The estimated posterior means (standard deviation) of ${\sigma }_{\epsilon }^2$ were 1.00 (0.0883) and 0.930 (0.0814) for BRR and BL, respectively. These values are close to the true value of the parameter (one) and suggest that BL over-fitted the data slightly. The posterior standard deviation (SD) was 8% smaller in BL, and mixing of the residual variance was better in BRR. BL gives a slightly “less informative” posterior distribution and worse mixing for two reasons. First, because of use of marker-specific variances, the number of unknowns in BL is much larger than in BRR. Second, the BL has an extra level in which the regularization parameter (λ) indexing the prior distribution of the marker-specific variances is inferred from the data. In BRR, the counterparts of λ are the hyper-parameters indexing the prior assigned to ${\sigma }_{{\beta }_{\mathrm{R}}}^2$ ( df_βR and S_βR ) which are specified by the user.

Figure 4.

Prior and estimated posterior density of the residual variance, by model (BRR = Bayesian Ridge Regression; BL = Bayesian LASSO).

The posterior mean of λ in BL was 20.1, and a 95th highest posterior density confidence region was bounded by [15.7, 26.2]. These results also indicate that the posterior distribution of λ moved away from the prior (Fig. 3), indicating that Bayesian learning takes place.

Measures of goodness of fit and model complexity [pD = estimated effective number of parameters, Spiegelhalter et al. (2002), and DIC] are included in the fitted object. This can be assessed with the following code:

fmR$fit ## Bayesian Ridge Regression
fmL$fit ## Bayesian LASSO

Table 1 provides estimates of the log-likelihood evaluated at the posterior mean of model unknowns, $l\left(\stackrel{‒}{\theta }\right)$, the posterior mean of the log-likelihood, $\stackrel{‒}{l}(.)$, the estimated effective number of parameters, pD, and DIC for BRR and BL. The Bayesian LASSO fitted the data better, and had a higher estimated number of effective parameters; DIC, which balances goodness of fit and complexity, favored the BL.

Table 1.

Measures of goodness of fit, model complexity, and Deviance Information Criterion (DIC) by model.

	Bayesian Ridge Regression	Bayesian LASSO
$l\left(\stackrel{‒}{\theta }\right)$ †	−776.1	−748.7
$\stackrel{‒}{l}(.)$	−847.9	−825.8
pD	143.6	154.3
DIC	1839.5	1806.1

^†$l\left(\stackrel{‒}{\theta }\right)$ = log-likelihood evaluated at the estimated posterior mean of model unknowns; $\stackrel{‒}{l}(.)$ = estimated posterior mean of the log-likelihood; pD = estimated effective number of parameters.

The mean-squared error of estimates of marker effects, $\mathrm{MSE}\left(\mathbf{\beta },{\widehat{\mathbf{\beta }}}_{.}\right)=p^{-1}{\sum }_{j=1}^p{\left({\beta }_j-{\widehat{\beta }}_{.j}\right)}^2$, where (.) can be either L or R, were 0.0046 and 0.0039 for BRR and BL, respectively. The mean-squared error of estimates of genetic values, $\mathrm{MSE}\left(\mathbf{X}\mathbf{\beta },\mathbf{X}{\widehat{\mathbf{\beta }}}_{.}\right)=n^{-1}{\sum }_{j=1}^n{\left({\mathbf{x}}_{.i}\mathbf{\beta }-{\mathbf{x}}_{.i}^{\prime }{\widehat{\mathbf{\beta }}}_{.}\right)}^2$, were 0.3712 and 0.3116 for BRR and BL, respectively. Therefore, BL outperformed BRR in this simulation. The difference between methods is more pronounced at the level of marker effects (19% difference in MSE between methods for genetic values). Figure 5 shows scatter plots of the estimates of marker effects (left) and of genetic values (right) obtained with BL (horizontal axis) and BRR (vertical axis). The BRR shrinks estimates of markers with sizable effects to a larger extent than BL (see left panel in Fig. 5). On the other hand, the two models yielded similar predictions of genetic values (see right panel in Fig. 5). This occurs because with p >> n, one can arrive at similar predictions of genetic values either with a model where genetic values are highly dependent on a few markers with sizable effect (something that occurs in LASSO and, to a lesser extent, in BL) or with a model where a large number of markers make a small contribution to genetic values (something that occurs in RR-BLUP and BRR). Which model yields better prediction of genetic values will depend on the underlying architecture of the trait and on the available marker data.

Figure 5.

Estimates of marker effects (left panel) and of genetic values (right panel) obtained with the Bayesian Ridge Regression (BRR, vertical axis) versus those obtained with the Bayesian LASSO (BL, horizontal axis).

Example 2: Assessing Predictive Ability by Cross-Validation

Predicting genetic values of lines with yet-to-be observed phenotypes is a central problem in plant breeding programs. Such predictions can be used, for example, to decide which of the newly developed lines will be included in field trials or which of these lines will be parents for the next breeding cycle. Either of the models described above, BL or BRR, can be used to obtain these predictions. The rate of genetic progress will depend on how accurate such predictions are, i.e., on the ability of the model to predict future outcomes. Cross-validation (CV) methods can be used to assess predictive ability and for for tuning-up values of certain parameters. In this section, we illustrate how the regularization parameter of the BL, λ, can be chosen using CV methods, and compare the performance of this approach with that obtained with the fully Bayesian approach that consists of assigning a prior to λ. The comparison is made using the wheat dataset previously described.

Model

Here, the linear model is y = 1μ + X_Lβ_L + u + ε. We chose values of hyper-parameters using formulae presented in the Appendix. As in Example 1, it was assumed a priori that 50% of the phenotypic variance (which equals to one because phenotypes were standardized to a unit variance) could be attributed to genetic values. With this, and using df_ε = 3 in formula [1A] in the Appendix, we obtained S_ε = 2.5. Further, we assumed a priori that one half of the variance of genetic values can be accounted for by the regression on markers, X_Lβ_L, and that the regression on the pedigree, u, accounted for the other half. With this, and using df_u = 3 and $\stackrel{‒}{a}=1.98$ in [2A], we obtained S_u = 0.63. Finally, using ${\sum }_{j=1}^p{\left(n^{-1}{\sum }_{i=1}^n{x}_{\mathrm{L}\mathrm{i}\mathrm{j}}\right)}^2=504$ in formula [4A] in the Appendix, we obtained $\widehat{\lambda }=\sqrt{2{\sigma }_{\epsilon }^2{V}_{\mathrm{L}}^{-1}{\sum }_j^{p_{\mathrm{R}}}{\stackrel{‒}{x}}_{\mathrm{L}\mathrm{j}}^2}=\sqrt{2\frac{1}{2}\frac{4}{1}504}\approx 45$. Choosing λ² ~ G(λ²|r = 1×10⁻⁵,δ = 0.52) gives a prior for λ that has a maximum and is relatively flat in the neighborhood of 45.

The R-code in Fig. 6 illustrates how this CV was performed for the first trait. The vector sets assign lines to folds of the CV. The code involves two loops: the outer loop runs over folds of the CV; the inner loop fits models over a grid of values of λ. For every fold in the outer loop, the phenotypes of approximately 60 lines are declared as missing values; the fitted model yields a prediction of the performance of these lines.

Figure 6.

R-code used to fit models of example 2.

Results of Example 2

Figure 7 gives the estimated mean-squared error of predictive residuals (PMSE, vertical axis) versus values of the regularization parameter (λ), by environment. The vertical and horizontal dashed lines give the average (across 10 folds of the CV) estimated posterior mean of λ and the estimated PMSE obtained when a prior was assigned to λ (i.e., the fully-Bayesian LASSO). In all environments except E3, the curve relating PMSE and λ was U-shaped, with an optimum λ (minimum PMSE) near 20. However, the absolute value of the slope of the curve was higher for low values of λ, indicating that over-fitting, something that occurs with small values of λ, is more problematic. This may occur because the conditional expectation function in these models included two random components: the regression on markers and the regression on the pedigree. The presence of u in the linear model cannot prevent the over-fitting occurring when λ is “too small”. However, as values of λ increase (i.e., placing a higher penalty on the regression on markers), the contribution of the regression on the pedigree to the conditional expectation increases as well, preventing lack of fit. Environment 3 constitutes an extreme example of this; here the curve relating PMSE and λ looks L-shaped.

Figure 7.

Predictive mean squared error (PMS, vertical axis) versus values of the regularization parameter of the Bayesian LASSO (horizontal axis), by environment. The vertical and horizontal dashed lines give the average (across 10 folds of the cross-validation) estimated posterior mean of λ and the estimated PMSE obtained when a prior was assigned to λ.

The posterior modes of λ were always considerably smaller than the prior mode (45), indicating that Bayesian learning took place. In all environments, the fully Bayesian treatment yielded posterior means of λ that were in the neighborhood of the optimal values found when models were run over a grid of values of this unknown. Also, predictive ability of models with random λ was as good as the best obtained when CV was used for choosing λ. These results suggest that, at least for these traits and this population, the fully Bayesian treatment, which consists of inferring λ from the data, yields good results.

Conclusions

The BLR package allows fitting high-dimensional linear regression models including dense molecular markers, pedigree information, and covariates other than markers. The interface allows the user to choose models (e.g., BL versus BRR) and prior hyper-parameters easily. The algorithms implemented are relatively efficient and models with a modest number of markers (e.g., ~1000) can be fitted in a standard PC easily. The routines implemented in the package have also been used successfully in problems with larger numbers of molecular markers. For example, Weigel et al. (2009) used an earlier version of the package to fit models using data from the 50k Bovine Illumina Bead Chip, and our experience indicates that the software can be used with an even larger number of markers. Computational time is expected to increase linearly with the number of markers; the user also needs to be aware that marker information is loaded in the memory. Therefore, as the number of marker increases, so do the memory requirements.

In models for genomic selection, with p >> n, marker effects cannot be estimated uniquely from the likelihood. A unique solution can be obtained by using penalized estimation methods or, in a Bayesian framework, by assigning informative priors to marker effects. Because of lack of identification at the level of the likelihood, the choice of prior is expected to play a role. As illustrated in Example 1, different priors yield different estimates of marker effects. Although models per se cannot solve the intrinsic identification problem, one can use model comparison criteria, such as DIC, or the principle of parsimony, or CV methods, to choose among prior distributions of marker effects. Even if the choice of prior affects estimates of marker effects, it is still possible to obtain similar estimates of genetic values using different priors (see Example 1, or the simulation study presented in de los Campos et al., 2009). This occurs because, with p >> n, one can arrive at similar predictions either with a model where genetic values are highly dependent on some markers or with a model where all markers make a small contribution to genetic values.

Simulation studies (e.g., Habier et al., 2007) have suggested that models using marker-specific shrinkage of estimates of effects such as the BL or models Bayes A or Bayes B of Meuwissen et al. (2001) may capture linkage disequilibrium between markers and QTLs better than a BRR. Therefore, Habier et al. (2007) conclude, when estimates of effects are used to perform several rounds of selection and without retraining, those models should perform better than a BRR. However, as pointed out by Jannink et al. (2010), the superiority of these models over a BRR has not always been confirmed by real-data analysis.

Finally, estimates of marker effects obtained with BL or BRR could be used to assess the relative contribution of each region to genetic variability. However, one needs to be aware that the estimated marker effect reflects not only linkage between markers and genes affecting the trait but also the density of markers in the region. If a region containing a QTL has a high density of markers, the effect of the QTL is expected to be “distributed” across linked markers; conversely, if in the same region markers are sparse, estimated marker effects are expected to be larger in absolute value.

Abbreviations

BL

Bayesian LASSO

BLR

Bayesian Linear Regression

BLUP

Best Linear Unbiased Prediction

BRR

Bayesian Ridge Regression

CV

cross-validation

DE

Double-Exponential

DIC

Deviance Information Criterion

GY

grain yield

GS

Genomic selection

IID

independent and identically distributed

LASSO

Least Absolute Shrinkage and Selection Operator

MM

molecular markers

OLS

ordinary least squares

PMSE

mean-squared error of predictive residuals

RR

ridge regression

APPENDIX

The prior distribution of the BLR model is indexed by several hyper-parameters. We provide guidelines for choosing their values on the basis of beliefs as to what proportion of the variance of phenotypes can be attributed to each of the random components of the regression, that is, u_i, ${\sum }_{j=1}^{p_{\mathrm{R}}}{x}_{\mathrm{R}ij}{\beta }_{\mathrm{R}\mathrm{j}}$, ${\sum }_{j=1}^{p_{\mathrm{R}}}{x}_{\mathrm{L}\mathrm{i}\mathrm{j}}{\beta }_{\mathrm{L}\mathrm{j}}$ and ε_i. Other authors (e.g., Meuwissen et al., 2001) have discussed how to choose hyper-parameters in the context of models for genomic selection. However, the derivation used by those authors assumes that genotypes are random and marker effects are fixed quantities, while in fact the opposite is true in the Bayesian models that have been proposed (e.g., Gianola et al., 2009). Here, we derive formulae that are consistent with the standard treatment of marker and marker effects in Bayesian models for GS, where marker genotypes are observed quantities and marker effects are random unknowns. Unlike the formulae in Meuwissen et al. (2001), the ones presented here do not require making any assumption about the extent of linkage disequilibrium between markers.

Residual Variance

The prior distribution of the residual variance is indexed by two parameters, {S_ε, df_ε}. One can choose these parameters so that the prior mode of ${\sigma }_{\epsilon }^2$, $\frac{S_{\epsilon }}{{\mathrm{df}}_{\epsilon }+2}$ matches our prior beliefs about the variance of model residuals. In practice, df_ε can be chosen to be a small value, usually greater than two, to guarantee a finite prior expectation, e.g., df_ε = 3, then the prior scale can be:

$$S_{\epsilon }=V_{\epsilon }({\mathrm{df}}_{\epsilon }+2),$$ [1A]

where V_ε is chosen to reflect our expectation of the variance of model residuals. Formulas similar to that presented in this appendix can be derived using formulas for the prior expectation.

Variance of the Infinitesimal Effect

From the prior distribution, the variance of u_i is $a_{ii}{\sigma }_u^2$, where a_ii is the ith diagonal element of A, which in the absence of inbreeding equals one. If we let $\stackrel{‒}{a}$ be the average diagonal value of A and V_u be our prior expectation about $\stackrel{‒}{a}{\sigma }_u^2$, then we can choose df_u = 3 and set the prior scale to be:

$$S_u=\frac{V_u({\mathrm{df}}_{\epsilon }+2)}{\stackrel{‒}{a}}$$ [2A]

Prior Variance of Marker Effects

The contributions of X_R and X_L to phenotypes is ${\sum }_{j=1}^p{x}_{ij}{\beta }_{\mathrm{j}}$. Here, (.) stands for R or L, depending on whether BRR or BL is being used. In regressions for GS, marker genotypes are fixed and marker effects are random. Furthermore, at the level of the marginal distribution, marker effects are IID; therefore, $\mathrm{Var}\left({\sum }_{j=1}^{p_{.}}{x}_{.ij}{\beta }_{.j}\right)=V_{.}={\sum }_{j=1}^p{x}_{.ij}^2\mathrm{Var}\left({\beta }_{.j}\right)$. For the “average” genotype, this formula becomes $V_{.}=\left\{{\sum }_{j=1}^p{\stackrel{‒}{x}}_{.j}^2\right\}\mathrm{Var}\left({\beta }_{.j}\right)$ where V_. represent user’s beliefs about the variance that can be assigned to the regression on X., and ${\stackrel{‒}{x}}_{.\mathrm{j}}$ denotes the average value of the jth column of X.

In BRR, the prior distribution of marker effects is Gaussian and $\mathrm{Var}\left({\beta }_{\mathrm{R}\mathrm{j}}\right)={\sigma }_{\beta }^2$. As before, df_βR can be chosen to have a relatively small value, e.g., df_βR = 3 then the prior scale can be set to be

$$S_{{\beta }_{\mathrm{R}}}=\frac{V_{\mathrm{R}}\left({\mathrm{df}}_{{\beta }_{\mathrm{R}}}+2\right)}{\sum _j^{p_{\mathrm{R}}}{\stackrel{‒}{x}}_{\mathrm{R}\mathrm{j}}^2}$$ [3A]

where V_R is set to reflect our expectation of the variance of phenotypes that can be attributed to the regression on X_R.

In the BL, the marginal prior density of marker effects is Double-Exponential, and the prior variance of marker effects is $\mathrm{Var}\left({\beta }_{\mathrm{L}\mathrm{j}}\right)=2{\sigma }_{\epsilon }^2{\lambda }^{-2}$. Plugging this in

$$\mathrm{Var}\left(\sum _{j=1}^{p_{\mathrm{L}}}{x}_{\mathrm{L}i\mathrm{j}}{\beta }_{\mathrm{L}\mathrm{j}}\right)=V_{\mathrm{L}}=\left\{\sum _{j=1}^{p_{\mathrm{L}}}{\stackrel{‒}{x}}_{\mathrm{L}\mathrm{j}}^2\right\}\mathrm{Var}\left({\beta }_{\mathrm{L}\mathrm{j}}\right)$$

we obtain $V_{\mathrm{L}}=\left\{{\sum }_{j=1}^{p_{\mathrm{L}}}{\stackrel{‒}{x}}_{\mathrm{L}\mathrm{j}}^2\right\}2{\sigma }_{\epsilon }^2{\lambda }^{-2}$. Solving for λ, we get:

$$\widehat{\lambda }=\sqrt{2{\sigma }_{\epsilon }^2{V}_{\mathrm{L}}^{-1}\sum _j^{p_{\mathrm{L}}}{\stackrel{‒}{x}}_{\mathrm{L}\mathrm{j}}^2}$$ [4A]

where ${\sigma }_{\epsilon }^2{V}_{\mathrm{L}}^{-1}$ is a noise-to-signal variance ratio. With [4A] we can choose a target value for the regularization parameter. Then we can choose hyper-parameters so that the prior has a mode and is relatively flat, in the neighborhood of $\widehat{\lambda }$. Note that when λ² ~ G(r,δ), p(λ|r,δ) = G(λ²|r,δ)2λ.

The above formulas constitute guidelines for choosing values of hyper-parameters. In practice, if Bayesian learning takes place, one would expect the posterior distribution to move away from the prior. Furthermore, with small n, it is always useful to check the sensitivity of inferences with respect to the choice of hyper-parameters.