Genomic Prediction from Multiple-Trait Bayesian Regression Methods Using Mixture Priors

Abstract

Bayesian multiple-regression methods incorporating different mixture priors for marker effects are used widely in genomic prediction. Improvement in prediction accuracies from using those methods, such as BayesB, BayesC, and BayesCπ, have been shown in single-trait analyses with both simulated and real data. These methods have been extended to multi-trait analyses, but only under the restrictive assumption that a locus simultaneously affects all the traits or none of them. This assumption is not biologically meaningful, especially in multi-trait analyses involving many traits. In this paper, we develop and implement a more general multi-trait BayesC$\mathrm{\Pi }$ and BayesB methods allowing a broader range of mixture priors. Our methods allow a locus to affect any combination of traits, e.g., in a 5-trait analysis, the “restrictive” model only allows two situations, whereas ours allow all 32 situations. Further, we compare our methods to single-trait methods and the “restrictive” multi-trait formulation using real and simulated data. In the real data analysis, higher prediction accuracies were observed from both our new broad-based multi-trait methods and the “restrictive” formulation. The broad-based and restrictive multi-trait methods showed similar prediction accuracies. In the simulated data analysis, higher prediction accuracies to the “restrictive” method were observed from our general multi-trait methods for intermediate training population size. The software tool JWAS offers open-source routines to perform these analyses.

GENOMIC prediction was proposed by Meuwissen et al. (2001) to incorporate marker effects from whole-genome data into genetic evaluation. In genomic prediction, all the marker or haplotype effects are estimated simultaneously, and these estimates can then be used to predict breeding values of individuals not in the training population used to estimate the effects.

Bayesian multiple-regression methods incorporating mixture priors for marker effects are used widely in genomic prediction, including various extensions to the BayesB method of Meuwissen et al. (2001). BayesB accommodates models where the prior for each marker effect follows a mixture distribution with a point mass at zero with probability π and a univariate-t distribution with probability $1-\pi$ (Meuwissen et al. 2001; Gianola et al. 2009; Cheng et al. 2015b). Another model, BayesC, assumes a mixture with a point mass at zero with probability π and a univariate normal distribution with probability $1-\pi$ for all marker effects, and its extension known as BayesCπ further treats π as an unknown parameter with a uniform prior distribution (Habier et al. 2011).

Bayesian multiple-regression methods were first proposed for single-trait analyses but have been extended to some particular forms of multi-trait analyses (Calus and Veerkamp 2011; Jia and Jannink 2012). Those extensions have pertained to a particular, somewhat restrictive mixture model. The “restrictive” multi-trait BayesC$\mathrm{\Pi }$ presented by Jia and Jannink (2012) assumes any particular locus affects none of the traits or simultaneously affects all traits. This assumption of genetic architecture in that multi-trait BayesC$\mathrm{\Pi }$ model is violated if some loci have no effect on at least one of the traits while having an effect on the remaining traits.

In this paper, we propose a more general class of multi-trait BayesC$\Pi$ and BayesB methods, where each locus can have an effect on any combination of traits. For example, in a 5-trait analysis, the restricted model only allows two situations, whereas ours allows all 32 situations. The previous restrictive multi-trait models are special cases of this general class of models. Further, our model allows the use of a single-site Gibbs sampler that requires less computing effort than some alternative Markov chain Monte Carlo approaches, especially for analyses involving many traits. Methodologies for the new models are compared to single-trait methods and the previous multi-trait methods using real and simulated data.

Materials and Methods

Multi-trait marker effects model

For simplicity of our description, but without loss of generality, we will assume individuals have all traits measured with a general mean as the only fixed effect, and write the multi-trait model for individual i from n genotyped individuals as

$${\mathit{y}}_i=\mathit{\mu }+{\displaystyle \sum _{j=1}^p{m}_{ij}{\mathit{\alpha }}_j}+{\mathit{e}}_i,$$

where ${\mathit{y}}_i$ is a vector of phenotypes of t traits for individual i, $\mathit{\mu }$ is a vector of overall means for t traits, $m_{ij}$ is the genotype covariate at locus j for individual i (coded as 0,1, and 2), p is the number of genotyped loci, ${\mathit{\alpha }}_j$ is a vector of allele substitution effects or marker effects of t traits for locus j, and ${\mathit{e}}_i$ is a vector of random residuals of t traits for individual i. The fixed effects, or general mean in this case, are assigned flat priors. The residuals, ${\mathit{e}}_i,$ are a priori assumed to be independently and identically distributed multivariate normal vectors with null mean and covariance matrix $\mathit{R},$ which, in turn, is assumed to have an inverse Wishart prior distribution, $W_t^{-1}\left(S_e,{\nu }_e\right).$

We will show that, employing the concept of data augmentation, the vector of marker effects at a particular locus ${\mathit{\alpha }}_j$ can be written as ${\mathit{\alpha }}_j={\mathit{D}}_j{\mathit{\beta }}_j,$ where ${\mathit{D}}_j$ is a diagonal matrix whose kth diagonal entry is an indicator variable indicating whether the marker effect of locus j for trait k is zero or nonzero, and ${\mathit{\beta }}_j$ follows a multivariate normal distribution in multi-trait BayesC$\mathrm{\Pi }$ or a multivariate t distribution in multi-trait BayesB.

Multi-trait BayesC$\mathrm{\Pi }$ model

Priors for marker effects:

The prior for ${\alpha }_{jk},$ the allele substitution or marker effect of trait k for locus j, is a mixture with a point mass at zero and a univariate normal distribution conditional on ${\sigma }_k^2:$

$${\alpha }_{jk}|{\pi }_k,{\sigma }_k^2\{\begin{array}{ll}\sim N\left(0,\hspace{0.17em}{\sigma }_k^2\right)\hfill & probability\left(1-{\pi }_k\right)\hfill \\ 0\hfill & probability\hspace{0.17em}{\pi }_k\hfill \end{array}$$

and the covariance between effects for traits k and $k^{\prime }$ at the same locus, i.e., ${\alpha }_{jk}$ and ${\alpha }_{j{k}^{\prime }}$ is

$$cov\left({\alpha }_{jk},{\alpha }_{j{k}^{\prime }}|{\sigma }_{k{k}^{\prime }}\right)=\{\begin{array}{ll}{\sigma }_{k{k}^{\prime }}\hfill & if\hspace{0.17em}both\hspace{0.17em}{\alpha }_{jk}\ne 0\hspace{0.17em}and\hspace{0.17em}{\alpha }_{j{k}^{\prime }}\ne 0\hfill \\ 0\hfill & otherwise\hfill \end{array}.$$

The vector of marker effects at a particular locus ${\mathit{\alpha }}_j$ is written as ${\mathit{\alpha }}_j={\mathit{D}}_j{\mathit{\beta }}_j,$ where ${\mathit{D}}_j$ is a diagonal matrix with elements $diag\left({\mathit{D}}_j\right)={\mathit{\delta }}_j=\left({\mathit{\delta }}_{j1},{\mathit{\delta }}_{j2},{\mathit{\delta }}_{j3}\dots {\mathit{\delta }}_{jt}\right),$ where ${\delta }_{jk}$ is an indicator variable indicating whether the marker effect of locus j for trait k is zero or nonzero, and the vector ${\mathit{\beta }}_j$ follows a multivariate normal distribution with null mean and covariance matrix $\mathit{G}=\left[\begin{array}{ccc}{\sigma }_1^2& \cdots & {\sigma }_{1t}\\ ⋮& \ddots & ⋮\\ {\sigma }_{1t}& \cdots & {\sigma }_t^2\end{array}\right]$ The covariance matrix $\mathit{G}$ is a priori assumed to follow an inverse Wishart distribution, $W_t^{-1}\left(S_{\beta },{\nu }_{\beta }\right).$ Thus, the multi-trait BayesC$\mathrm{\Pi }$ model with data augmentation is written as

$${\mathit{y}}_i=\mathit{\mu }+{\displaystyle \sum _{j=1}^p{m}_{ij}{\mathit{D}}_j{\mathit{\beta }}_j}+{\mathit{e}}_i.$$ (1)

In the most general case, any marker effect might be zero for any possible combination of t traits resulting in $2^t$ possible combinations of ${\mathit{\delta }}_j.$ For example, in a t=2 trait model, there are $2^t=4$ combinations for ${\mathit{\delta }}_j:$ $\left(0,\hspace{0.17em}0\right),$ $\left(0,\hspace{0.17em}1\right),$ $\left(1,\hspace{0.17em}0\right),$ $\left(1,\hspace{0.17em}1\right).$ In the restrictive special case of this model described by Jia and Jannink (2012), only two combinations, i.e., $\left(0,\hspace{0.17em}0\right)$ and $\left(1,\hspace{0.17em}1\right),$ have nonzero probability. Suppose, in general, we use numerical labels “1,” “2,”$\dots ,$ “l” for the $2^t$ possible outcomes for ${\delta }_j,$ then the prior for ${\mathit{\delta }}_j$ is a categorical distribution

$$p\left({\mathit{\delta }}_j=“i”\right)={\mathrm{\Pi }}_{1}I\left({\mathit{\delta }}_j=“1”\right)+{\Pi }_{2}I\left({\mathit{\delta }}_j=“2”\right)+\dots +{\mathrm{\Pi }}_{l}I\left({\mathit{\delta }}_j=“l”\right),$$

where ${\displaystyle {\sum }_{i=1}^l{\mathrm{\Pi }}_i}=1$ with ${\mathrm{\Pi }}_i$ being the prior probability that the vector ${\mathit{\delta }}_j$ corresponds to the vector labeled $“i”.$ A Dirichlet distribution with all parameters equal to one, i.e., a uniform distribution, can be used for a prior for $\mathbf{\Pi }=\left({\mathrm{\Pi }}_1,{\mathrm{\Pi }}_2,\dots ,{\mathrm{\Pi }}_l\right).$

As shown below, we consider two Gibbs samplers to draw samples for all the parameters in this model. Gibbs sampler I is a single-site sampler, where only one of the t indicator labels is sampled at a time. Thus, in a 2-trait model, for example, this sampler cannot move from $\left(0,\hspace{0.17em}0\right)$ to $\left(1,\hspace{0.17em}1\right)$ in a single step without stepping through $\left(1,\hspace{0.17em}0\right)$ or $\left(0,\hspace{0.17em}1\right)$ for ${\mathit{\delta }}_j.$ Therefore, Gibbs sampler I cannot be used for the restrictive model which excludes $\left(1,\hspace{0.17em}0\right)$ and $\left(0,\hspace{0.17em}1\right)$ from the state space for ${\mathit{\delta }}_j.$ Gibbs sampler II, however, samples all elements of ${\mathit{\delta }}_j$ jointly, and can move from $\left(0,\hspace{0.17em}0\right)$ to $\left(1,\hspace{0.17em}1\right)$ in a single step. However, Gibbs sampler II is computationally more intensive because it requires drawing samples from a multivariate normal distribution of order t, the number of traits.

Gibbs sampler I for multi-trait BayesC$\mathrm{\Pi }$:

Suppose the prior for ${\mathit{\delta }}_j$ is a categorical distribution for which the support is the set of $2^t$ outcomes of ${\mathit{\delta }}_j.$ For convenience, from now on let “1” denote trait k and “2” the other $t-1$ traits. In our sampling scheme, ${\beta }_{j1}$ and ${\mathit{\delta }}_{j1}$ are sampled from their joint full conditional distributions, which can be written as the product of the full conditional distribution of ${\beta }_{j1}$ given ${\mathit{\delta }}_{j1}$ and the marginal full conditional distribution of ${\mathit{\delta }}_{j1}.$ Let $\mathit{\theta }$ denote all other parameters except ${\mathit{\delta }}_{j1}$ and ${\beta }_{j1},$ then our sampling scheme can be written as

$$f\left({\beta }_{j1},{\mathit{\delta }}_{j1}|\mathit{\theta },\mathit{y}\right)=f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)f\left({\delta }_{j1}|\mathit{\theta },\mathit{y}\right).$$

The full conditional distributions of ${\beta }_{j1},$ ${\mathit{\delta }}_{j1},$ $\mathbf{\Pi },$ $\mathit{G}$ and $\mathit{R}$ for Gibbs sampler I, whose derivations are in the Appendix, are given below. The full conditional distributions of ${\beta }_{j1}$ is

$$p\left({\beta }_{j1}|{\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)=\{\begin{array}{cc}N\left({\widehat{\beta }}_{j1}^0,{\left(G^{11}\right)}^{-1}\right)& when\hspace{0.17em}{\mathit{\delta }}_{j1}=0\\ N\left({\widehat{\beta }}_{j1}^1,{\left(C_{j,11}^1\right)}^{-1}\right)& when\hspace{0.17em}{\mathit{\delta }}_{j1}=1\end{array},$$

with

$${\widehat{\beta }}_{j1}^0=-{\left(G^{11}\right)}^{-1}{G}^{12}{\mathit{\beta }}_{j2},$$

$${\widehat{\beta }}_{j1}^1={\left(C_{j,11}^1\right)}^{-1}\left(r_{j1}-C_{j,12}^1{\mathit{\beta }}_{j2}\right),$$

$$C_{j,11}^1=G^{11}+R^{11}{\displaystyle \sum _{i=1}^n{m}_{ij}^2}$$

$$C_{j,12}^1={\mathit{G}}^{12}+{\mathit{R}}^{12}{\mathit{D}}_{j2}{\displaystyle \sum _{i=1}^n{m}_{ij}^2},$$

$$r_{j1}=\left({\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{R}^{11}\\ {\mathit{R}}^{21}\end{array}\right],$$

where ${\mathit{w}}_i={\mathit{y}}_i-\mathit{\mu }-{\displaystyle {\sum }_{j^{\prime }\ne j}{m}_{i{j}^{\prime }}{\mathit{D}}_{j^{\prime }}{\mathit{\beta }}_{j^{\prime }}},$ $G^{11}$ and $G^{12}$ are the partitions of $G^{-1}$ corresponding to trait k and covariances between trait k and other traits, respectively. $R^{11}$ and $R^{12}$ are the partitions of $R^{-1}$ corresponding to trait k and covariances between trait k and other traits, respectively.

The marginal full conditional probability that ${\delta }_{j1}=1$ is

$$f\left({\mathit{\delta }}_{j1}=1|\mathit{\theta },\mathit{y}\right)={\left\{1+{\left(\frac{Pr\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{Pr\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}H\right)}^{-1}\right\}}^{-1},$$

where $H=$

$$exp\left\{-\frac{1}{2}\left(log{C}_{j,11}^1-{\widehat{{\beta }_{j1}^1}}^2{C}_{j,11}^1\right)-\left(-\frac{1}{2}\left(log{G}^{11}-{\widehat{{\beta }_{j1}^0}}^2{G}^{11}\right)\right)\right\}.$$

The full conditional distribution for $\mathbf{\Pi }$ can be written as

$$f\left(\mathbf{\Pi }|\mathit{\beta },\mathit{D},\mathit{G},\mathit{R},\mathit{y}\right)\propto Dirichlet\left(n_1+1,\hspace{0.17em}{n}_2+1,\dots \right),$$

where $n_i$ is the number of loci or markers for which ${\mathit{\delta }}_j=“i”.$

The full conditional distributions for $\mathit{R},$ the covariance matrix for residuals, is an inverse Wishart distribution, $W_t^{-1}\left(S_e+\mathit{e}\prime \mathit{e},{\nu }_e+n\right),$ where $\mathit{e}$ is the $n\times t$ matrix for residuals whose ith row is ${\mathit{e}}_i^{\prime }.$ The full conditional distribution for $\mathit{G},$ the covariance matrix for ${\beta }_j,$ is an inverse Wishart distribution, $W_t^{-1}\left(S_{\beta }+\mathit{\beta }\prime \mathit{\beta },{\nu }_{\beta }+p\right),$ where $\mathit{\beta }$ is the $p\times t$ matrix whose ith row is ${\mathit{\beta }}_i^{\prime }.$

Gibbs sampler II for multi-trait BayesC$\mathrm{\Pi }$:

The Gibbs sampler above, where only one of the t indicator labels is sampled at a time, cannot be used for the restrictive model assuming any particular locus affects all traits or none of them. Further, if some particular ${\mathrm{\Pi }}_i$ are near zero, the chain might exhibit mixing problems. Another, more general, but computationally intensive, Gibbs sampler that samples all elements of ${\mathit{\delta }}_j$ jointly and may exhibit improved mixing is proposed below.

The full conditional distributions of ${\mathit{\beta }}_j,$ ${\mathit{\delta }}_j,$ $\mathbf{\Pi },$ $G,$and $R$ for Gibbs sampler II, whose derivations are in the Appendix, are given below.

Let $\mathit{\theta }$ denote all other parameters except ${\mathit{\beta }}_j$ and ${\mathit{\delta }}_j,$ then our sampling scheme can be written as

$$f\left({\mathit{\beta }}_j,{\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right).$$

The full conditional distribution of ${\mathit{\beta }}_j$ is

$$f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)\propto N\left({\mathit{C}}_j^{-1}{\mathit{r}}_j,{\mathit{C}}_j^{-1}\right),$$

where ${\mathit{C}}_j={\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2+{\mathit{G}}^{-1}}$ and ${\mathit{r}}_j^{\prime }=\left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right){\mathit{R}}^{-1}{\mathit{D}}_j.$

The marginal full conditional probability of ${\mathit{\delta }}_j=“i”$ is

$$f\left({\mathit{\delta }}_j=“i”|\mathit{\theta },\mathit{y}\right)=\frac{f\left(\mathit{y}|{\mathit{\delta }}_j=“i”,\mathit{\theta }\right)f\left({\mathit{\delta }}_j=“i”|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{“\mathit{i}”\in \left\{“1”,“2”,\dots ,“\mathit{l}”\right\}}f\left(\mathit{y}|{\mathit{\delta }}_j=“i”,\mathit{\theta }\right)f\left({\mathit{\delta }}_j=“i”|\mathbf{\Pi }\right)}},$$

where

$$f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)={\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}exp\left\{\frac{1}{2}{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j\right\}.$$

This Gibbs sampler can accommodate the “restrictive” multi-trait BayesC$\mathbf{\Pi }$ that was proposed by Jia and Jannink (2012), which only allows ${\mathit{\delta }}_j$ to be a vector of all ones or a vector of all zeros.

Multi-trait BayesB model

The multi-trait BayesC$\mathrm{\Pi }$ model proposed above can be modified to accommodate the general multi-trait BayesB model. Model Equation (1) can also be used for the multi-trait BayesB method. The differences in multi-trait BayesB method is that the prior for ${\mathit{\beta }}_j$ is a multivariate t distribution, rather than a multivariate normal distribution. This is equivalent to assuming ${\mathit{\beta }}_j$ has a multivariate normal distribution with null mean and locus-specific covariance matrix ${\mathit{G}}_j,$ which is assigned an inverse Wishart prior, $W_t^{-1}\left(S_{\beta },{\nu }_{\beta }\right).$

The derivations of the full conditional distributions of parameters of interest for Gibbs samplers are shown in the Appendix. In the multi-trait BayesB model, the full conditional distributions for all parameters except ${\mathit{G}}_j$ are similar to the multi-trait BayesC$\mathrm{\Pi }$ model. The full conditional distribution for ${\mathit{G}}_j,$ the covariance matrix for ${\beta }_j,$ is an inverse Wishart distribution, $W_t^{-1}\left(S_{\beta }+{\mathit{\beta }}_j{\mathit{\beta }}_j^{\prime },{\nu }_{\beta }+1\right).$

Data analyses

Real data:

Published genotypic and deregressed breeding values based on phenotypic data for Loblolly Pine (Pinus taeda L.) were used (Resende et al. 2012; Daetwyler et al. 2013). Two disease traits, namely presence or absence of rust (Rust_bin) and gall volume (Rust_gall_vol) were analyzed. These are the two traits used in Jia and Jannink (2012). The reported heritabilities were 0.21 for Rust_bin and 0.12 for Rust_gall_vol. Loci with missing genotypes were imputed as the mean of the observed genotype covariates at that locus but loci with a missing rate $>$50% were excluded. After these quality control edits, 4828 SNPs on 807 individuals with deregressed phenotypes and genotypes on both traits remained.

Prediction accuracy was calculated as the correlation between the vector of deregressed phenotypes and the vector of estimated breeding values. Cross-validation using 10 folds formed the basis for comparing these methods. Paired t-tests were used for tests of significance of difference in prediction accuracies between two methods, where prediction accuracies for two different methods from each validation fold were considered as paired samples. The general multi-trait BayesC$\mathrm{\Pi }$ model (MT-BayesC$\mathrm{\Pi }$-G) was compared to a similar model where the prior for ${\mathit{\alpha }}_{\mathit{j}}$ is a multivariate normal rather than a mixture of multivariate normals (MT-BayesC0), the more restricted multi-trait BayesC$\mathrm{\Pi }$ proposed by Jia and Jannink (2012) (MT-BayesC$\mathrm{\Pi }$-R) and the usual single trait formulations of the mixture models (ST-BayesC0, ST-BayesCπ). Since BayesC0 is equivalent to random regression best linear unbiased prediction (RR-BLUP), ST-BayesC0 and MT-BayesC0 are denoted as ST-RR-BLUP and MT-RR-BLUP below. The prior for the residual covariance matrix $R$ in all multi-trait methods was an inverse Wishart distribution, $W^{-1}=\left(\left[\begin{array}{cc}0.003& 0\\ 0& 0.003\end{array}\right],6\right),$ for which the mean of $R$ is $\left[\begin{array}{cc}0.001& 0\\ 0& 0.001\end{array}\right],$ the SD of diagonal elements are $1.4\times {10}^{-3},$ and the SD of off-diagonal elements are 0. This same prior was used for the marker effects covariance matrix $\mathit{G}.$ The priors for the residual variance and marker effects variance in single-trait analyses were scaled inverted chi-squared distributions with scale parameter $S^2=0.0005$ and degrees of freedom $\nu =4,$ for which the mean of the prior was also 0.001. In the data analyses, multi-trait BayesB methods provided similar results as multi-trait BayesC$\mathrm{\Pi }$ methods. Thus, only results from BayesC$\mathrm{\Pi }$ analyses were presented below to demonstrate the superiority of our multi-trait methods.

Simulated data:

Simulated data described below were used to quantify the superiority of the general multi-trait Bayesian methods. Two scenarios were simulated. In scenario 1, as a known ideal condition, the simulated genome consisted of 100 loci on each of two chromosomes that were in Hardy-Weinberg and linkage equilibria. All these loci were considered as QTL or causative variants and used in the analyses. The QTL on the first chromosome had effects only on trait 1 and those on the second chromosome only on trait 2. The effects of these QTL were simulated from a standard normal distribution and then were equally scaled to provide unit genetic variance for each trait in the simulated population of 8000 unrelated individuals. The phenotypes for these traits were obtained by adding independent residuals to the genetic values. Two situations were simulated: (1) heritabilities for both traits were 0.5; (2) heritability for trait 1 was 0.2 and for trait 2 was 0.8. The XSim package was used in the simulation (Supplemental Material, File S1) (Cheng et al. 2015a).

In scenario 2, both markers and QTL were simulated. The simulated genome consisted of 100 evenly spaced loci on each of three chromosomes of length 10 cM. Ten loci were randomly selected on each chromosome as QTL. Allele states were sampled from a Bernoulli distribution with frequency 0.5 in the base population. Starting from a base population of 500 males and 500 females, random mating was simulated for 500 generations to generate linkage disequilibrium. Random mating was continued for five more generations to increase the population size to 4000 males and 4000 females, which were used in the analyses. The effects of QTL on the first two chromosomes were simulated following the same strategy in scenario 1, i.e., the QTL on the first chromosome had effects only on trait 1, and those on the second chromosome only on trait 2. All QTL on the third chromosome had effects on both traits. The effects of these QTL on the third chromosome were simulated from a standard bivariate normal distribution with correlation 0.5. The phenotypes for these traits were obtained by adding independent residuals to the genetic values. In total, 8000 individuals were simulated with heritability 0.2 for trait 1 and 0.8 for trait 2.

The same validation approaches were used for these two simulation scenarios. A total of 500 individuals were used for testing, and for each training population of size N, 100 replicates of the training population were sampled from the remaining individuals. The values considered for N were 50, 100, 200, 400, 1000, 2000, 4000, or 7000. The true genetic and residual variances were used to compute the scale parameters for the priors of the variance components. The general multi-trait BayesC$\mathrm{\Pi }$ model (MT-BayesC$\mathrm{\Pi }$-G) was compared to the more restricted multi-trait BayesC$\mathrm{\Pi }$ (MT-BayesC$\mathrm{\Pi }$-R) using this dataset.

All analyses were performed using JWAS (Cheng et al. 2018), an open-source, publicly available package for single-trait and multi-trait whole-genome analyses written in the freely available Julia language.

Data availability

The genotypic and phenotypic data used in the real data analysis are publicly available (Resende et al. 2012). The scripts used to generate the simulated data are provided as supplementary information. The authors state that all data necessary for confirming the conclusions presented in the article are represented fully within the article.

Results

Real data

The prediction accuracies from all methods for Rust_bin and Rust_gall_vol are in Figure 1. The prediction accuracies from all single-trait analyses using JWAS are similar to those in Resende et al. (2012).

Figure 1.

Comparison of single-trait and multi-trait methods for Rust_bin and Rust_gall_vol traits. ST, MT-G and MT-R indicate single-trait, our general multi-trait and restricted multi-trait analyses, respectively. * indicates a statistically significant (P < 0.01) difference between methods.

The predictions of Rust_bin exhibited no significant difference in accuracy between multi-trait and single-trait analyses within each method (ST-RR-BLUP vs. MT-RR-BLUP; ST-BayesCπ vs. MT-BayesC$\mathrm{\Pi }$-R; ST-BayesCπ vs. MT-BayesC$\mathrm{\Pi }$-G).

In contrast, prediction accuracies for the lower heritability Rust_gall_vol with MT-BayesC$\mathrm{\Pi }$-G were significantly higher than those from ST-BayesCπ. MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R showed similar prediction accuracies. The posterior means of $\mathrm{\Pi }$ for both methods are in Table 1. When RR-BLUP was used for the analysis, however, the advantage of the multiple-trait analysis (MT-RR-BLUP) over the single-trait analysis (ST-RR-BLUP) for Rust_gall_vol was not observed.

Table 1. Estimation of π for alternative multi-trait BayesC$\mathrm{\Pi }$ methods.

	Different categories of δ
	$\left(0,0\right)$	$\left(1,1\right)$	$\left(0,1\right)$	$\left(1,0\right)$
MT-BayesC$\mathrm{\Pi }$-G	0.966	0.029	0.002	0.003
MT-BayesC$\mathrm{\Pi }$-R	0.971	0.029	NAa	NAa

Posterior mean of $\mathrm{\Pi }$ were given for different categories of δ. Different categories of δ are denoted as $\left(k_1,k_2\right),$ where $k_1=0$ if a marker has a null effect on Rust_bin, otherwise $k_1=1,$ and similarly for $k_2$ representing sampled effects for Rust_gall_vol.

^aCombinations that do not exist in the restricted model.

Simulated data

The prediction accuracies from MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R methods were compared for varying size (N) of training populations under two simulation scenarios. In simulation scenario 1, Figure 2 shows the prediction accuracies where heritabilities for both traits were 0.5. Figure 3 shows the prediction accuracies where heritabilities for trait 1 and trait 2 were 0.2 and 0.8, respectively. When $\mathrm{N}=50,$ both methods had similar prediction accuracy. For both traits, as N increased, initially, MT-BayesC$\mathrm{\Pi }$-G became superior to MT-BayesC$\mathrm{\Pi }$-R, but, as expected, the accuracies of these methods asymptotically converged (Karaman et al. 2016). In most cases, the differences in accuracies for both traits were small. However, in Figure 3, the differences in accuracies for trait 1, for which the heritability was 0.2, were substantial for intermediate values of N. Figure 4 shows the prediction accuracies for simulation scenario 2. The pattern observed is similar to Figure 3 under simulation scenario 1. MT-BayesC$\mathrm{\Pi }$-G was superior to MT-BayesC$\mathrm{\Pi }$-R for intermediate training population, but as N increased, the accuracies of these methods asymptotically converged (Karaman et al. 2016).

Figure 2.

Comparison of multi-trait BayesC$\mathrm{\Pi }$ methods for situation 1 under simulation scenario 1.

Figure 3.

Comparison of multi-trait BayesC$\mathrm{\Pi }$ methods for situation 2 under simulation scenario 1.

Figure 4.

Comparison of multi-trait BayesC$\Pi$ methods under simulation scenario 2.

Discussion

Real data

Significant differences between multi-trait and single-trait analyses were only observed for Rust_gall_vol within BayesCπ methods (MT-BayesC$\mathrm{\Pi }$-G vs. ST-BayesCπ; MT-BayesC$\mathrm{\Pi }$-R vs. ST-BayesCπ). MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R outperformed ST-BayesCπ for Rust_gall_vol, and the accuracy gain was $26\%$ (from 0.287 to 0.364). The lower-heritability trait Rust_gall_vol benefited from information on the other correlated trait Rust_bin. Thus higher prediction accuracy from MT- BayesC$\mathrm{\Pi }$-G were observed in trait Rust_gall_vol but not for the high heritability Rust_bin. Results in Jia and Jannink (2012) showed no difference between MT-BayesC$\mathrm{\Pi }$ and ST-BayesCπ because a reduced marker panel (500 markers) was used.

The fact that RR-BLUP showed no improvement in multi-trait analyses suggested that benefits from MT-BayesC$\mathrm{\Pi }$-G may be due to the estimation of the hyper-parameter $\mathrm{\Pi }.$ In the MT-BayesC$\Pi$-G, the mean of the posterior probability that a marker has a null effect on Rust_gall_vol was ∼0.97, calculated as the summation of posterior mean of $\mathrm{\Pi }$ for categories $\left(0,0\right)$ and $\left(1,0\right).$ The posterior mean of π, the probability that a marker has a null effect, in ST-BayesCπ for Rust_gall_vol was 0.74, different from the equivalent value, 0.97, in MT-BayesC$\mathrm{\Pi }$-G shown above. Thus a ST-BayesC analysis with $\pi =0.97$ was undertaken. Prediction accuracy from this ST-BayesCπ analysis with $\pi =0.97$ was 0.361, which was similar to the accuracy from MT-BayesC$\mathrm{\Pi }$-G. This shows that including an additional correlated trait, especially one with high heritability, will bring in more data into the analysis, helping variable selection in a low-heritability trait to become more effective and result in improved prediction accuracy.

The difference between MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R is that MT-BayesC$\mathrm{\Pi }$-R assumes a locus has an effect on all traits or none of them. This assumption regarding genetic architecture is likely to be seldom true. MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R, however, showed similar prediction accuracies. This can be explained by the estimation of $\mathrm{\Pi }$ in MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R in Table 1. The posterior probability means for $\left(0,1\right)$ and $\left(1,0\right)$ were almost zero in MT-BayesC$\mathrm{\Pi }$-G and for $\left(0,0\right)$ and $\left(1,1\right)$ are similar in MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R, suggesting that the assumption of genetic architecture whereby the same loci affect both traits as explicit in MT-BayesC$\mathrm{\Pi }$-R may be valid for these two disease traits. Note that the lack of difference between the methods may also result from the limited size of the training population.

Simulated data

In scenario 1, we simulated bivariate data where each QTL had an effect on only one or the other of the traits. In MT-BayesC$\mathrm{\Pi }$-R, if a locus has an effect on one of the traits, that locus is included in the model for all traits. So, in the simulated data, MT-BayesC$\mathrm{\Pi }$-R would need to include all loci in the model for both traits. Thus for the trait that had heritability 0.2, the contribution of noise to the prediction from loci on chromosome 2, which had no effect on this trait, is large relative to the real signal from QTL on chromosome 1. In contrast, the general variable selection in MT-BayesC$\mathrm{\Pi }$-G allows loci on chromosome 2, which have no effect on trait 1, to be excluded from the model for trait 1. Thus when sufficient data were available for variable selection to exclude loci on chromosome 2 for trait 1, MT-BayesC$\mathrm{\Pi }$-G showed a substantial advantage over MT-BayesC$\mathrm{\Pi }$-R. On the other hand, for the trait with heritability 0.8, the contribution of noise to the prediction from the loci on chromosome 1, which had no effect on this trait, is small relative to the signal from loci on chromosome 2. Thus MT-BayesC$\mathrm{\Pi }$-G and MT-BayesC$\mathrm{\Pi }$-R had similar accuracies. As the training population size increased, the contribution of noise to the prediction of a trait from loci which had no effect on this trait, vanished even when the heritability was low. This was observed for both traits as apparent in Figure 2 and Figure 3. Since only bivariate data with different heritabilities showed substantial differences in prediction accuracies, traits with different heritabilities were simulated in scenario 2. In scenario 2, both markers and QTL were simulated. As expected, MT-BayesC$\mathrm{\Pi }$-G showed higher prediction accuracy to MT-BayesC$\mathrm{\Pi }$-R for intermediate training population, but as N increased, the accuracies of these methods asymptotically converged (Karaman et al. 2016).

Further, in both real and simulated analyses, MT-BayesC$\mathrm{\Pi }$-G gave equal or higher prediction accuracy than MT-BayesC$\mathrm{\Pi }$-R. In addition, MT-BayesC$\mathrm{\Pi }$-R requires drawing samples from a multivariate normal distribution of order t, whereas Gibbs sampler I, which can be used for MT-BayesC$\mathrm{\Pi }$-G, requires sampling from a univariate normal. Thus, in addition to MT-BayesC$\mathrm{\Pi }$-G giving equal or better performance than MT-BayesC$\mathrm{\Pi }$-R, MT-BayesC$\mathrm{\Pi }$-G can also be computationally more efficient.

Priors

In practice, genetic variances from previous conventional analyses are usually used to construct priors for marker effect variances. For single trait analyses, under some assumptions, it can be shown that the marker effect variance ${\sigma }_{\alpha }^2$ can be obtained as

$${\sigma }_{\alpha }^2=\frac{{\sigma }_g^2}{\left(1-\pi \right){\displaystyle \sum 2p_j\left(1-p_j\right)}},$$ (2)

where ${\sigma }_g^2$ is the genetic variance, $p_j$ is the allele frequency for locus j, and π is the probability that a marker has a null effect (Habier et al. 2007; Gianola et al. 2009; Fernando and Garrick 2013). Following a similar strategy, the marker effect covariance matrix $G$ in a two-trait analysis can be obtained as

$$G=\frac{1}{{\displaystyle \sum 2p_j\left(1-p_j\right)}}\left[\begin{array}{cc}\frac{Q_{11}}{p\left(\mathit{\delta }=\left(1,1\right)\right)+p\left(\mathit{\delta }=\left(1,0\right)\right)}& \frac{Q_{12}}{p\left(\mathit{\delta }=\left(1,1\right)\right)}\\ \frac{Q_{21}}{p\left(\mathit{\delta }=\left(1,1\right)\right)}& \frac{Q_{22}}{p\left(\mathit{\delta }=\left(1,1\right)\right)+p\left(\mathit{\delta }=\left(0,1\right)\right)}\end{array}\right],$$ (3)

where $Q=\left[\begin{array}{cc}{Q}_{11}& Q_{12}\\ Q_{21}& Q_{22}\end{array}\right]$ is the genetic covariance matrix and $p\left(\mathit{\delta }=\left(0,1\right)\right),$ $p\left(\mathit{\delta }=\left(1,0\right)\right),$ and $p\left(\mathit{\delta }=\left(1,1\right)\right)$ are the probabilities a marker has null effects on the first trait but not the second trait, on the second trait but not the first trait, or on neither trait. Thus the probability that a marker has an effect on the first trait can be obtained as $p\left(\mathit{\delta }=\left(1,1\right)\right)+p\left(\mathit{\delta }=\left(1,0\right)\right),$ which is the denominator of the upper left element in (3). This strategy relating marker effect covariance matrix to genetic covariance matrix can be readily extended to >2 traits. Note that positive definite matrix $Q$ may result in negative definite matrix $G$ using (3), especially when the prior for the probability a marker has null effects is far from the real value. In that case, the diagonal elements of $G,$ which are the marker effect variances for different traits, can be obtained using (2), where π may be estimated from previous single-trait analyses, and the off-diagonal elements of $G$ may be set to zero to guarantee positive definiteness of $G.$

Multi-trait variable selection:

In regard to a single trait, a locus either has an effect, or it does not. Hence, the scalar parameter π (and its complement $1-\pi$) completely defines this circumstance. In a multi-trait setting, it is conceivable that loci that influence one trait, may or may not influence other traits. In that circumstance, a vector $\mathrm{\Pi }$ is required to define the genetic architecture. The number of parameters that constitute the vector $\mathrm{\Pi }$ is $2^t,$ which grows rapidly with the number of traits. In most cases, the researcher will have little or no knowledge of the likely extent of pleiotropy of loci that influence two traits, other than knowing or having an estimate of the genetic covariance. There are two simple ways to reduce this complexity in priors.

First, one can assume, as did Jia and Jannink (2012), that, in the context of variable selection, a locus should be selected for all of the traits or selected for none of the traits, reducing the required probabilities to being analogous to the single trait π and $\left(1-\pi \right).$ This approach has the advantage of simplicity, but the disadvantage that many effects might need to be estimated for loci that have no effect on a trait, and this may erode the accuracy of prediction. This should not be a problem for asymptotically large datasets, as in that case the fitted locus effects should converge to zero for those traits not influenced by that locus.

A second simple way to accommodate the multiple trait circumstance is to assume the $2^t$ parameters can be derived from t trait-specific parameters. However, when the probability that a single trait locus has an effect is small for each of two or more traits, the pair-wise probability that a locus affects all the traits will be the product of those small probabilities, making it very difficult for loci to enter the model for all traits simultaneously.

The better way to solve this problem is to use a hyper-parameter $\mathrm{\Pi }$ that completely defines the alternative models that are required to capture all the alternative forms of genetic architecture. We have shown here how this can be done, with two alternative Gibbs sampling strategies. One involves single-site sampling for one locus and trait at a time. The other samples all the alternative combinations of effects for one locus considering all traits simultaneously. We have shown that both are practical with real data and can result in improved accuracies of prediction in certain circumstances in terms of genetic architecture and size of dataset.

Conclusions

Many researchers are interested in genome-wide association studies and finding causal genes and variants. For those researchers, pleiotropy is of considerable interest, and they would want to know which loci affect which traits, from a purely biological perspective. Practitioners are often interested in “breaking” the genetic correlation, by selecting parents to give a favorable selection response in respect to multiple trait consequences. In either of these circumstances, with intermediate- rather than asymptotically large datasets, we believe the methods described here and available in the open-source, freely-available JWAS package offer real promise.

Supplementary Material

Supplemental material is available online at www.genetics.org/lookup/suppl/doi:10.1534/genetics.118.300650/-/DC1.

Appendix

Gibbs Sampler Algorithm for Multi-Trait BayesC$\mathrm{\Pi }$-G

Single-site Gibbs sampler for multi-trait BayesC$\mathrm{\Pi }$-G

The full conditional distribution of ${\beta }_{j1}$ can be written as

$$\begin{array}{l}f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},\mathit{G},\mathit{R},\mathit{y}\right)\propto f\left(\mathit{y}|\mathit{\mu },\mathit{\beta },\mathit{D},\mathit{G},\mathit{R}\right)f\left({\beta }_{j1},{\mathit{\beta }}_{j2}|\mathit{G}\right)\\ \propto exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }}{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}^{-1}{\mathit{\beta }}_j\right),\end{array}$$

where ${\mathit{w}}_i={\mathit{y}}_i-{\mathit{\mu }}_i-{\displaystyle {\sum }_{j^{\prime }\ne j}{m}_{i{j}^{\prime }}{\mathit{D}}_{j^{\prime }}{\mathit{\beta }}_{j^{\prime }}}.$ Further, by dropping factors that do not involve ${\beta }_{j1},$

$$\begin{array}{c}f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},\mathit{G},\mathit{R},\mathit{y}\right)\propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2+{\mathit{G}}^{-1}}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[\left[\begin{array}{cc}{\beta }_{j1}& {\mathit{\beta }}_{j2}^{\prime }\end{array}\right]\left[\begin{array}{cc}{C}_{j,11}& {\mathit{C}}_{j,12}\\ {\mathit{C}}_{j,21}& {\mathit{C}}_{j,22}\end{array}\right]\left[\begin{array}{c}{\beta }_{j1}\\ {\mathit{\beta }}_{j2}\end{array}\right]-2\left[\begin{array}{cc}{r}_{j1}& {\mathit{r}}_{j2}^{\prime }\end{array}\right]\left[\begin{array}{c}{\beta }_{j1}\\ {\mathit{\beta }}_{j2}\end{array}\right]\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left(C_{j,11}{\beta }_{j1}^2+\left(2{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}-2r_{j1}\right){\beta }_{j1}\right)\right\}\\ \propto exp\left\{-\frac{C_{j,11}}{2}{\left({\beta }_{j1}+\left({\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}-r_{j1}\right)C_{j,11}^{-1}\right)}^2\right\}\\ \propto N\left(C_{j,11}^{-1}\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right),C_{j,11}^{-1}\right)\\ \propto N\left({\beta }_{{}_{j1}}^{^},C_{j,11}^{-1}\right)\end{array}$$

where ${\mathit{C}}_j={\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2+{\mathit{G}}^{-1}}$ and ${\mathit{r}}_j^{\prime }=\left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right){\mathit{R}}^{-1}{\mathit{D}}_j.$

Note that when ${\mathit{\delta }}_{j1}=0,$

$$\begin{array}{c}{C}_j=\left[\begin{array}{cc}{C}_{j,11}^0& {\mathit{C}}_{j,12}^0\\ {\mathit{C}}_{j,21}^0& {\mathit{C}}_{j,22}^0\end{array}\right]\\ =\left[\begin{array}{cc}{G}^{11}& {\mathit{G}}^{12}\\ {\mathit{G}}^{21}& {\mathit{G}}^{22}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{22}{\mathit{D}}_{j2}{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}\end{array}\right]\\ r_j^{\prime }=\left[\begin{array}{cc}{r}_{j1}^0& {\mathit{r}}_{j2}^{0^{\prime }}\end{array}\right]\\ =\left[\begin{array}{cc}0& \left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{\mathit{R}}^{12}\\ {\mathit{R}}^{22}\end{array}\right]{\mathit{D}}_{j2}\end{array}\right]\end{array}$$

When ${\mathit{\delta }}_{j1}=1,$

$$\begin{array}{c}{C}_j=\left[\begin{array}{cc}{C}_{j,11}^1& {\mathit{C}}_{j,12}^1\\ {\mathit{C}}_{j,21}^1& {\mathit{C}}_{j,22}^1\end{array}\right]\\ =\left[\begin{array}{cc}{G}^{11}+R^{11}{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}& {\mathit{G}}^{12}+{\mathit{R}}^{12}{\mathit{D}}_{j2}{{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}}_{ij}^2\\ {\mathit{G}}^{21}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{21}{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}& {\mathit{G}}^{22}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{22}{\mathit{D}}_{j2}{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}\end{array}\right]\\ r_j^{\prime }=\left[\begin{array}{cc}{r}_{j1}^1& {\mathit{r}}_{j2}^{1^{\prime }}\end{array}\right]\\ =\left[\begin{array}{cc}\left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{R}^{11}\\ {\mathit{R}}^{21}\end{array}\right]& \left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{\mathit{R}}^{12}\\ {\mathit{R}}^{22}\end{array}\right]{\mathit{D}}_{j2}\end{array}\right]\end{array}$$

Thus when ${\mathit{\delta }}_{j1}=0,$ the full conditional distribution of ${\beta }_{j1}$ is

$$f\left({\beta }_{j1}|{\mathit{\delta }}_{j1}=0,{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},\mathit{G},\mathit{R},\mathit{y}\right)\propto N\left(\widehat{{\beta }_{j1}^0},{\left(C_{j,11}^0\right)}^{-1}\right)=N\left(-{\left(G^{11}\right)}^{-1}{G}^{12}{\mathit{\beta }}_{j2},{\left(G^{11}\right)}^{-1}\right).$$

When ${\delta }_{j1}=1,$ the full conditional distribution of ${\beta }_{j1}$ becomes

$$f\left({\beta }_{j1}|{\mathit{\delta }}_{j1}=1,{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},\mathit{G},\mathit{R},\mathit{y}\right)\propto N\left(\widehat{{\beta }_{j1}^1},{\left(C_{j,11}^1\right)}^{-1}\right)=N\left({\left(C_{j,11}^1\right)}^{-1}\left(r_{j1}-C_{j,12}^1{\mathit{\beta }}_{j2}\right),{\left(C_{j,11}^1\right)}^{-1}\right).$$

The marginal full conditional distribution of ${\mathit{\delta }}_{j1}$ can be written as

$$\begin{array}{c}f\left({\mathit{\delta }}_{j1}=1|\mathit{\theta },\mathit{y}\right)=\frac{f\left({\mathit{\delta }}_{j1}=1,\mathit{\theta },\mathit{y}\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_{j1}\in \left(0,1\right)}f\left({\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)}}\\ =\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{{{\displaystyle \sum }}_{{\mathit{\delta }}_{j1}\in \left(0,1\right)}f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}.\\ ={\left\{1+\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}\right\}}^{-1}\end{array}$$

The factor $f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)$ can be written as

$$\begin{array}{c}f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)\propto {\displaystyle \int f\left(\mathit{y}|\mathit{\mu },{\beta }_{j1},{\mathit{\beta }}_{-j1},\mathit{D},\mathit{G},\mathit{R}\right)f\left({\beta }_{j1},{\mathit{\beta }}_{j2}|\mathit{G}\right)d{\beta }_{j1}}\\ \propto {\displaystyle \int exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]}exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}^{-1}{\mathit{\beta }}_j\right)d{\beta }_{j1}\\ \propto exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}}-{\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right)}^2{C}_{j,11}^{-1}\right)\right\}\\ \times {\displaystyle \int exp\left[-\frac{1}{2}{\left({\beta }_{j1}-\widehat{{\beta }_{j1}}\right)}^2{C}_{j,11}\right]d{\beta }_{j1}}\\ \propto {\left(C_{j,11}\right)}^{-\frac{1}{2}}exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}}-{\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right)}^2{C}_{j,11}^{-1}\right)\right\}\\ \propto {\left(C_{j,11}\right)}^{-\frac{1}{2}}exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}-{\widehat{{\beta }_{j1}}}^2{C}_{j,11}\right)\right\}\end{array}$$

Note that ${\displaystyle {\sum }_i{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i,{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2},{\mathit{\beta }}_{\mathit{j}2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}}$ are same when ${\mathit{\delta }}_{j1}=0$ or 1. Thus the ratio $f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)/f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)$ becomes

$$\begin{array}{l}H={\left(C_{j,11}^1\right)}^{-\frac{1}{2}}{\left(G^{11}\right)}^{\frac{1}{2}}exp\left(-\frac{1}{2}\left({\widehat{{\beta }_{j1}^0}}^2{G}^{11}-{\widehat{{\beta }_{j1}^1}}^2{C}_{j,11}^1\right)\right)\\ =exp\left\{-\frac{1}{2}\left(log{C}_{j,11}^1-{\widehat{{\beta }_{j1}^1}}^2{C}_{j,11}^1\right)-\left(-\frac{1}{2}\left(log{G}^{11}-{\widehat{{\beta }_{j1}^0}}^2{G}^{11}\right)\right)\right\}\end{array}$$

Thus the conditional probability of ${\mathit{\delta }}_{j1}=1$ is

$${\left\{1+\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}\right\}}^{-1}={\left\{1+{\left(\frac{{\mathbf{\Pi }}_{j0}}{{\mathbf{\Pi }}_{j1}}H\right)}^{-1}\right\}}^{-1},$$

where ${\mathrm{\Pi }}_{j0}=Pr\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)$ and ${\mathrm{\Pi }}_{j1}=Pr\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right).$

The full conditional distribution for $\mathbf{\Pi }$ can be written as

$$\begin{array}{c}f\left(\mathrm{\Pi }|\mathit{\beta },\mathit{D},\mathit{G},\mathit{R},\mathit{y}\right)\propto f\left(\mathit{\delta }|\mathbf{\Pi }\right)f\left(\mathbf{\Pi }\right)\\ \propto {\mathrm{\Pi }}_1^{n_1}{\mathrm{\Pi }}_2^{n_2}\dots {\mathrm{\Pi }}_l^{n_l}\\ \propto Dirichlet\left(n_1+1,n_2+1,\dots \right),\end{array}$$

where $n_i$ is the number of markers with ${\mathit{\delta }}_j=“i”.$

Joint Gibbs sampler for multi-trait BayesC$\Pi$-G

Let $\mathit{\theta }$ denote all other parameters except ${\mathit{\beta }}_j$ and ${\mathit{\delta }}_j,$ then our sampling scheme can be written as

$$f\left({\mathit{\beta }}_j,{\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)$$

The marginal full conditional distribution of ${\mathit{\delta }}_j$ can be written as

$$\begin{array}{c}f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=\frac{f\left({\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left({\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)}}\\ =\frac{f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}}.\end{array}$$

Denote ${\mathit{w}}_i={\mathit{y}}_i-{\mathit{\mu }}_i-{\displaystyle {\sum }_{j^{\prime }\ne j}{m}_{i{j}^{\prime }}{\mathit{D}}_{j^{\prime }}{\mathit{\beta }}_{j^{\prime }}},$ then

$$\begin{array}{c}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)\propto {\displaystyle \int f\left(\mathit{y}|\mathit{\beta },\mathit{D},\mathit{R}\right)f\left({\mathit{\beta }}_j|\mathit{G}\right)d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]}exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}^{-1}{\mathit{\beta }}_j\right)d{\mathit{\beta }}_j\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2}+{\mathit{G}}^{-1}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}\right]\right\}d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}\right]\right\}}d{\mathit{\beta }}_j\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j}\right]\right\}}d{\mathit{\beta }}_j\\ \propto exp\left\{-\frac{1}{2}\left[{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j}\right]\right\}\\ \times {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}{\displaystyle \int {\left|{\mathit{C}}_j^{-1}\right|}^{-\frac{1}{2}}}exp\left[-\frac{1}{2}\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)\right]d{\mathit{\beta }}_j\\ \propto {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}exp\left\{-\frac{1}{2}\left[{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j\right]\right\},\end{array}$$

where ${\mathit{C}}_j={\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}+{\mathit{G}}^{-1}$ and ${\mathit{r}}_j^{\prime }=\left({{{\displaystyle \sum }}^{}}_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}\right){\mathit{R}}^{-1}{\mathit{D}}_j.$

Note that ${\sum }_i^{}{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i$ is same for different ${\mathit{\delta }}_j.$ Thus the marginal full conditional distribution of ${\mathit{\delta }}_j$ can be written as

$$f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=\frac{f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}},$$

where

$$f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)\propto {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}exp\left\{\frac{1}{2}{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j\right\}.$$

The full conditional distribution of ${\mathit{\beta }}_j$ is

$$\begin{array}{c}f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)\propto exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}^{-1}{\mathit{\beta }}_j\right),\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2}+{\mathit{G}}^{-1}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)\right\}\\ \propto N\left({\mathit{C}}_j^{-1}{\mathit{r}}_j,{\mathit{C}}_j^{-1}\right)\end{array}$$

Gibbs Sampler Algorithm for Multi-Trait BayesB

Single-site Gibbs sampler for multi-trait BayesB

For convenience, from now on let “1” denote trait k and “2” the other traits. Thus, ${\mathit{\beta }}_j$ can be denoted as $\left[\begin{array}{c}{\beta }_{j1}\\ {\mathit{\beta }}_{j2}\end{array}\right]$ and ${\mathit{D}}_j$ can be denoted as $\left[\begin{array}{cc}{\mathit{\delta }}_{j1}& 0\\ 0& {\mathit{D}}_{j2}\end{array}\right].$ The Gibbs sampler for ${\beta }_{jk}$ and ${\mathit{\delta }}_{jk}$ is derived as below. In our sampling scheme, ${\beta }_{j1}$ and ${\mathit{\delta }}_{j1}$ are sampled from their joint full conditional distributions, which can be written as the product of the full conditional distribution of ${\beta }_{j1}$ given ${\mathit{\delta }}_{j1}$ and the marginal full conditional distribution of ${\mathit{\delta }}_j.$ Let $\mathit{\theta }$ denote all other parameters except ${\mathit{\delta }}_{j1}$ and ${\beta }_{j1},$ then our sampling scheme can be written as

$$f\left({\beta }_{j1},{\mathit{\delta }}_{j1}|\mathit{\theta },\mathit{y}\right)=f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)f\left({\mathit{\delta }}_{j1}|\mathit{\theta },\mathit{y}\right).$$

The full conditional distribution of ${\beta }_j$ can be written as

$$\begin{array}{c}f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},{\mathit{G}}_j,{\mathit{G}}_{-j},\mathit{R},\mathit{y}\right)\propto f\left(\mathit{y}|\mathit{\mu },\mathit{\beta },\mathit{D},{\mathit{G}}_j,{\mathit{G}}_{-j},\mathit{R}\right)f\left({\beta }_{j1},{\mathit{\beta }}_{j2}|{\mathit{G}}_j\right)\\ \propto exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}_j^{-1}{\mathit{\beta }}_j\right),\end{array}$$

where ${\mathit{w}}_i={\mathit{y}}_i-{\mathit{\mu }}_i-{\displaystyle {\sum }_{j^{\prime }\ne j}{m}_{i{j}^{\prime }}{\mathit{D}}_{j^{\prime }}{\mathit{\beta }}_{j^{\prime }}}.$ Further, by dropping factors that do not involve ${\beta }_{j1},$

$$\begin{array}{c}f\left({\beta }_{j1}|{\mathit{\delta }}_{j1},{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},G_j,{\mathit{G}}_{-j},\mathit{R},\mathit{y}\right)\propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2}+{\mathit{G}}_j^{-1}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[\left[\begin{array}{cc}{\beta }_{j1}& {\mathit{\beta }}_{j2}^{\prime }\end{array}\right]\left[\begin{array}{cc}{C}_{j,11}& {\mathit{C}}_{j,12}\\ {\mathit{C}}_{j,21}& {\mathit{C}}_{j,22}\end{array}\right]\left[\begin{array}{c}{\beta }_{j1}\\ {\mathit{\beta }}_{j2}\end{array}\right]-2\left[\begin{array}{cc}{r}_{j1}& {\mathit{r}}_{j2}^{\prime }\end{array}\right]\left[\begin{array}{c}{\beta }_{j1}\\ {\mathit{\beta }}_{j2}\end{array}\right]\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left(C_{j,11}{\beta }_{j1}^2+\left(2{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}-2r_{j1}\right){\beta }_{j1}\right)\right\}\\ \propto exp\left\{-\frac{C_{j,11}}{2}{\left({\beta }_{j1}+\left({\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}-r_{j1}\right)C_{j,11}^{-1}\right)}^2\right\}\\ \propto N\left(C_{j,11}^{-1}\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right),C_{j,11}^{-1}\right)\\ \propto N\left(\widehat{{\beta }_{j1}},C_{j,11}^{-1}\right)\end{array}$$

where ${\mathit{C}}_j={\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2+{\mathit{G}}_j^{-1}}$ and ${\mathit{r}}_j^{\prime }=\left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right){\mathit{R}}^{-1}{\mathit{D}}_j.$

Note that, when ${\mathit{\delta }}_{j1}=0,$

$$\begin{array}{c}{C}_j=\left[\begin{array}{cc}{G}_j^{11}& {\mathit{G}}_j^{12}\\ {\mathit{G}}_j^{21}& {\mathit{G}}_j^{22}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{22}{\mathit{D}}_{j2}{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2}\end{array}\right]\\ r_j^{\prime }=\left[\begin{array}{cc}0& \left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{\mathit{R}}^{12}\\ {\mathit{R}}^{22}\end{array}\right]{\mathit{D}}_{j2}\end{array}\right]\end{array}$$

When ${\mathit{\delta }}_{j1}=1,$

$$\begin{array}{c}{C}_j=\left[\begin{array}{cc}{C}_{j,11}^1& {\mathit{C}}_{j,12}^1\\ {\mathit{C}}_{j,21}^1& {\mathit{C}}_{j,22}^1\end{array}\right]\\ =\left[\begin{array}{cc}{G}_j^{11}+R^{11}{\sum }_{i=1}^n{m}_{ij}^2& {\mathit{G}}_j^{12}+{\mathit{R}}^{12}{\mathit{D}}_{j2}{\sum }_{i=1}^n{m}_{ij}^2\\ {\mathit{G}}_j^{21}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{21}{\sum }_{i=1}^n{m}_{ij}^2& {\mathit{G}}_j^{22}+{\mathit{D}}_{j2}^{\prime }{\mathit{R}}^{22}{\mathit{D}}_{j2}{\sum }_{i=1}^n{m}_{ij}^2\end{array}\right]\\ r_j^{\prime }=\left[\begin{array}{cc}{r}_{j1}^1& {\mathit{r}}_{j2}^{1\prime }\end{array}\right]\\ =\left[\begin{array}{cc}\left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{R}^{11}\\ {\mathit{R}}^{21}\end{array}\right]& \left({\displaystyle {\sum }_{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}}\right)\left[\begin{array}{c}{\mathit{R}}^{12}\\ {\mathit{R}}^{22}\end{array}\right]{\mathit{D}}_{j2}\end{array}\right]\end{array}$$

Thus, when ${\delta }_{j1}=0,$ the full conditional distribution of ${\beta }_{j1}$ is

$$f\left({\beta }_{j1}|{\mathit{\delta }}_{j1}=0,{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},G_j,{\mathit{G}}_{-j},\mathit{R},\mathit{y}\right)\propto N\left(-{\left(G_j^{11}\right)}^{-1}{G}_j^{12}{\mathit{\beta }}_{j2},{\left(G_j^{11}\right)}^{-1}\right).$$

When ${\delta }_{j1}=1,$ the full conditional distribution of ${\beta }_{j1}$ becomes

$$f(\left({\beta }_{j1}|{\mathit{\delta }}_{j1}=1,{\mathit{\beta }}_{-j1},{\mathit{D}}_{-j1},G_j,{\mathit{G}}_{-j},\mathit{R},\mathit{y}\right)\propto N\left(C_{j,11}^{1-1}\left(r_{j1}-C_{j,12}^1{\mathit{\beta }}_{j2}\right),{\mathrm{C}}_{j,11}^{1-1}\right).$$

The marginal full conditional distribution of ${\mathit{\delta }}_{j1}$ can be written as

$$\begin{array}{c}f\left({\mathit{\delta }}_{j1}=1|\mathit{\theta },\mathit{y}\right)=\frac{f\left({\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_{j1}\in \left(0,1\right)}f\left({\mathit{\delta }}_{j1},\mathit{\theta },\mathit{y}\right)}}\\ =\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_{j1}\in \left(0,1\right)}f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}}.\\ ={\left\{1+\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)}\right\}}^{-1}\end{array}$$

The factor $f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)$ can be written as

$$\begin{array}{c}f\left(\mathit{y}|{\mathit{\delta }}_{j1},\mathit{\theta }\right)\propto {\displaystyle \int f\left(\mathit{y}|\mathit{\mu },{\beta }_{j1},{\mathit{\beta }}_{-j1},\mathit{D},\mathit{G},\mathit{R}\right)f\left({\beta }_{j1},{\mathit{\beta }}_{j2}|{\mathit{G}}_j\right)d{\beta }_{j1}}\\ \propto {\displaystyle \int exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{R}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}_j^{-1}{\mathit{\beta }}_j\right)d{\beta }_{j1}}\\ \propto exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{R}^{-1}{\mathit{w}}_i-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}-{\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right)}^2{C}_{j,11}^{-1}}\right)\right\}\\ \times {\displaystyle \int exp\left[-\frac{1}{2}{\left({\beta }_{j1}-{\beta }_{{}_{j1}}^{^}\right)}^2{C}_{j,11}\right]d{\beta }_{j1}}\\ \propto {\left(C_{j,11}\right)}^{-\frac{1}{2}}exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{R}^{-1}{\mathit{w}}_i-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}-{\left(r_{j1}-{\mathit{C}}_{j,12}{\mathit{\beta }}_{j2}\right)}^2{C}_{j,11}^{-1}}\right)\right\}\\ \propto {\left(C_{j,11}\right)}^{-\frac{1}{2}}exp\left\{-\frac{1}{2}\left({\displaystyle \sum _i{\mathit{w}}_i^{\prime }{R}^{-1}{\mathit{w}}_i-2{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2}+{\mathit{\beta }}_{j2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}-{\widehat{{\beta }_{j1}}}^2{C}_{j,11}}\right)\right\}.\end{array}$$

Note that ${\displaystyle {\sum }_i{\mathit{w}}_i^{\prime }{R}^{-1}{\mathit{w}}_i,{\mathit{r}}_{j2}^{\prime }{\mathit{\beta }}_{j2},{\mathit{\beta }}_{\mathit{j}2}^{\prime }{\mathit{C}}_{j,22}{\mathit{\beta }}_{j2}}$ are same when ${\mathit{\delta }}_{j1}=0$ or 1. Thus the ratio $f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)/f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)$ becomes

$$\begin{array}{c}H={\left(C_{j,11}^1\right)}^{-\frac{1}{2}}{\left(G_j^{11}\right)}^{\frac{1}{2}}exp\left(-\frac{1}{2}\left({\widehat{{\beta }_{j1}^0}}^2{G}_j^{11}-\widehat{{\beta }_{j1}^1}{C}_{j,11}^1\right)\right)\\ =exp\left\{-\frac{1}{2}\left(log{C}_{j,11}^1-{\widehat{{\beta }_{j1}^1}}^2{C}_{j,11}^1\right)-\left(-\frac{1}{2}\left(log{G}_j^{11}-{\widehat{{\beta }_{j1}^0}}^2{G}_j^{11}\right)\right)\right\}\end{array}$$

Thus, the conditional probability of ${\mathit{\delta }}_{j1}=1$ is

$${\mathbf{\left\{}1+\frac{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=0,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|{\mathrm{\Pi }}_1,{\mathrm{\Pi }}_{\mathrm{2...}}\right)}{f\left(\mathit{y}|{\mathit{\delta }}_{j1}=1,\mathit{\theta }\right)f\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|{\mathrm{\Pi }}_1,{\mathrm{\Pi }}_{\mathrm{2...}}\right)}\mathbf{\right\}}}^{-1}={\left\{1+{\left(\frac{{\mathbf{\Pi }}_{j0}}{{\mathbf{\Pi }}_{j1}}H\right)}^{-1}\right\}}^{-1},$$

where ${\mathbf{\Pi }}_{j0}=Pr\left({\mathit{\delta }}_{j1}=0,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right)$ and ${\mathbf{\Pi }}_{j1}=Pr\left({\mathit{\delta }}_{j1}=1,{\mathit{\delta }}_{j2}|\mathbf{\Pi }\right).$

Joint Gibbs sampler for multi-trait BayesB

Let $\mathit{\theta }$ denote all other parameters except ${\mathit{\beta }}_j$ and ${\mathit{\delta }}_j,$ then our sampling scheme can be written as

$$f\left({\mathit{\beta }}_j,{\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)$$

The marginal full conditional distribution of ${\mathit{\delta }}_j$ can be written as

$$f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=\frac{f\left({\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left({\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)}}=\frac{f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}}.$$

Denote ${\mathit{w}}_i={\mathit{y}}_i-{\mathit{\mu }}_i-{\displaystyle {\sum }_{j^{\prime }\ne j}{m}_{i{j}^{\prime }}{\mathit{D}}_{j^{\prime }}{\mathit{\beta }}_{j^{\prime }}},$ then

$$\begin{array}{c}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)\propto {\displaystyle \int f\left(\mathit{y}|\mathit{\beta },\mathit{D},\mathit{R}\right)f\left({\mathit{\beta }}_j|{\mathit{G}}_j\right)d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left[-\frac{1}{2}\stackrel{n}{{\displaystyle \sum }_{i=1}}{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}_j^{-1}{\mathit{\beta }}_j\right)d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2+{\mathit{G}}_j^{-1}}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}\right]\right\}d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}\right]\right\}d{\mathit{\beta }}_j}\\ \propto {\displaystyle \int exp\left\{-\frac{1}{2}\left[\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)+{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j\right]\right\}d{\mathit{\beta }}_j}\\ \propto exp\left\{-\frac{1}{2}\left[{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j}\right]\right\}\\ \times {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}{\displaystyle \int {\left|{\mathit{C}}_j^{-1}\right|}^{-\frac{1}{2}}exp\left[-\frac{1}{2}\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)\right]d{\mathit{\beta }}_j}\\ \propto {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}exp\left\{-\frac{1}{2}\left[{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j}\right]\right\},\end{array}$$

where ${\mathit{C}}_j={\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle {\sum }_{i=1}^n{m}_{ij}^2+{\mathit{G}}_j^{-1}}$ and ${\mathit{r}}_j^{\prime }=\left({\displaystyle {\sum }_{i=1}^n{w}_i^{\prime }{m}_{ij}}\right){\mathit{R}}^{-1}{\mathit{D}}_j.$

Note that ${\displaystyle {\sum }_i{\mathit{w}}_i^{\prime }{\mathit{R}}^{-1}{\mathit{w}}_i}$ is same for different ${\mathit{\delta }}_j.$ Thus the marginal full conditional distribution of ${\mathit{\delta }}_j$ can be written as

$$f\left({\mathit{\delta }}_j|\mathit{\theta },\mathit{y}\right)=\frac{f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}{{\displaystyle {\sum }_{{\mathit{\delta }}_j}f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)f\left({\mathit{\delta }}_j|\mathbf{\Pi }\right)}},$$

where

$$f\left(\mathit{y}|{\mathit{\delta }}_j,\mathit{\theta }\right)\propto {\left|{\mathit{C}}_j^{-1}\right|}^{\frac{1}{2}}exp\left\{\frac{1}{2}{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}{\mathit{r}}_j\right\}.$$

The full conditional distribution of ${\mathit{\beta }}_j$ is

$$\begin{array}{c}f\left({\mathit{\beta }}_j|{\mathit{\delta }}_j,\mathit{\theta },\mathit{y}\right)\propto exp\left[-\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}^{\prime }{\mathit{R}}^{-1}\left({\mathit{w}}_i-m_{ij}{\mathit{D}}_j{\mathit{\beta }}_j\right)}\right]exp\left(-\frac{1}{2}{\mathit{\beta }}_j^{\prime }{\mathit{G}}_j^{-1}{\mathit{\beta }}_j\right),\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }\left({\mathit{D}}_j^{\prime }{\mathit{R}}^{-1}{\mathit{D}}_j{\displaystyle \sum _{i=1}^n{m}_{ij}^2+{\mathit{G}}_j^{-1}}\right){\mathit{\beta }}_j-2{\displaystyle \sum _{i=1}^n{\mathit{w}}_i^{\prime }{m}_{ij}{\mathit{R}}^{-1}{\mathit{D}}_j{\mathit{\beta }}_j}\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left[{\mathit{\beta }}_j^{\prime }{\mathit{C}}_j{\mathit{\beta }}_j-2{\mathit{r}}_j^{\prime }{\mathit{\beta }}_j\right]\right\}\\ \propto exp\left\{-\frac{1}{2}\left({\mathit{\beta }}_j^{\prime }-{\mathit{r}}_j^{\prime }{\mathit{C}}_j^{-1}\right){\mathit{C}}_j\left({\mathit{\beta }}_j-{\mathit{C}}_j^{-1}{\mathit{r}}_j\right)\right\}\\ \propto N\left({\mathit{C}}_j^{-1}{\mathit{r}}_j,{\mathit{C}}_j^{-1}\right)\end{array}$$