Comments on ‘Bayesian hierarchical error model for analysis of gene expression data’

Cho and Lee (2004) proposed a Bayesian hierarchical error model (HEM) to account for heterogeneous error variability in oligo-nucleotide microarray experiments. They estimated the parameters of their model using Markov Chain Monte Carlo (MCMC) and proposed an F-like summary statistic to identify differentially expressed genes under multiple conditions. Their HEM is one of the emerging Bayesian hierarchical modeling tools that have been developed for the analysis of multiple-level data structures and variation in microarray gene expression data (Broet et al., 2002; Tadesse and Ibrahim, 2004; Cho and Lee, 2004). In this letter, we first discuss the significance of the HEM developed by Cho and Lee. Then, we re-derive the fully conditional distributions for gene and conditional effects, since we think that these two fully conditional distributions were not presented properly in their paper. Finally, we expand the HEM to deal with biological or/and experimental correlations in gene expression data. A FORTRAN 90 program was developed to implement our extended method and it is available from the authors upon request.

Significance of HEM

Many aspects of the HEM presented in the article of Cho and Lee (2004) are significant. The following are the two aspects that we believe are important in the analysis of microarray gene expression data. First, error variability of gene expression data is decomposed into two components, biological and experimental. The former reflects variation owing to biological sample heterogeneity that affects gene expression, while the latter largely comes from variation in sample preparation and sample analysis. Estimating these two sources of errors separately will not only improve estimation of model parameters but also facilitate a better understanding of patterns of over all gene expression.

Second, and more importantly, the use of a Bayesian HEM is an effective way to represent complex relationships that have many parameters, because it allows the parameters to be modeled separately at several levels and provides a link (S) between the data (D) and the unobserved parameters (θ) of interest. In multilevel modeling, the top layer is the data, p(D|S, θ), which is modeled by an appropriate likelihood function. The data are assumed to be generated by some process that depends on certain unknown parameters. Modeling the link or process is the middle layer, where the art of statistical modeling takes place. On the bottom layer are the prior distributions that represent a priori beliefs about these parameters. Following the Bayesian inference, unknown parameters are inferred from the joint distribution of the parameters given the data

$$p(\theta ,S\mid D)\propto p(D\mid S,\theta )P(S\mid \theta )P(\theta )$$ (1)

Thus, Bayesian hierarchical modeling based on multivariate normal distributions effectively handles complex relationships that are governed by many parameters, and provides a tool for evaluating correlated gene expression data. For example, the model of Cho and Lee (2004) can be extended to allow for biological or/and experimental correlations. These will be discussed in more detail in the third section of this letter. Statistically, the estimation of these variables separately is biased when they are correlated (Johnston, 1972; Gianola and Sorensen, 2004). From a practical standpoint, ignoring correlations in gene expression data would result in high variability of statistical estimators and cause the reproducibility to deteriorate (Qiu et al., 2005).

Fully conditional distributions of gene (g_i) and condition (c_j) effects

We claim that Cho and Lee (2004) made a few mistakes when they derived the fully conditional distributions for the gene (g_i) and condition (c_j) effects. Let y be a vector containing all the data (y_ijkls), x a vector containing all the link variables (x_ijks) that correspond to expression intensities free from experimental errors, g a vector containing all the gene effects (g_is), c a vector containing all the condition effects (c_js), r a vector containing all the interaction effects (r_ijs) of genes and conditions, and Δ a variance–covariance matrix with diagonal elements or variance ( ${\sigma }_{b_{ij}}^2\text{s}$) and zero off-diagonal elements or covariance. Following their model specification and prior distributions, the posterior of unknown parameters, given the data (y), is

$$\begin{array}{l}p({\theta }_L,{\sigma }_e^2,x\mid y)\propto p(y\mid x,{\sigma }_e^2)p(x\mid {\theta }_L)p({\theta }_L)p({\sigma }_e^2)\propto p(y\mid x,{\sigma }_e^2)p(x\mid \mu ,g,c,r,\mathrm{\Delta })p(\mu )p(g)p(c)p(r)p(\mathrm{\Delta })p({\sigma }_e^2)\\ \propto {({\sigma }_e^{-2})}^{\left(\left({\sum }_i{\sum }_j{\sum }_k{n}_{\mathit{ijk}}\right)/2\right)}exp\left(-\frac{({\sigma }_e^{-2}){\sum }_i{\sum }_j{\sum }_k{\sum }_l{(y_{\mathit{ijk}l}-x_{\mathit{ijk}})}^2}{2}\right)\times \left(\prod _i\prod _j{({\sigma }_{b_{ij}}^{-2})}^{m_{ij}/2}\right)\\ \times exp\left(-\frac{{\sum }_i{\sum }_j{\sum }_k({\sigma }_{b_{ij}}^{-2}{(x_{\mathit{ijk}}-\mu -g_i-c_j-r_{ij})}^2)}{2}\right)\\ \times exp\left(-\frac{{\sum }_i{g}_i^2}{2{\sigma }_g^2}\right)\times exp\left(-\frac{{\sum }_j{c}_j^2}{2{\sigma }_c^2}\right)\times exp\left(-\frac{{\sum }_i{\sum }_j{r}_{ij}^2}{2{\sigma }_r^2}\right)\times \left(\prod _i\prod _j({({\sigma }_{b_{ij}}^{-2})}^{{\alpha }_b-1}exp(-{\beta }_b{\sigma }_{b_{ij}}^{-2}))\right)\times \left({({\sigma }_e^{-2})}^{{\alpha }_e-1}exp(-{\beta }_e{\sigma }_e^{-2})\right),\end{array}$$ (2)

where θ_L = {μ, g, c, r, Δ} is a collection of unknown parameters determining the link x, with μ, m_ij, n_ijk, ${\sigma }_g^2,{\sigma }_c^2,{\sigma }_r^2$ and ${\sigma }_e^2$ being defined the same as μ, m_i,j, n_i,j,k, ${\sigma }_g^2,{\sigma }_c^2,{\sigma }_e^2$ and ${\sigma }_r^2$ respectively, in the paper of Cho and Lee (2004). To ensure that our results are comparable with those of Cho and Lee (2004), we made the same assumptions regarding the prior distribution. That is, we also applied a flat prior to the grand mean μ such that p(μ) cancels out in (2) as a constant. [See Cho and Lee (2004) for a complete discussion of the implications of using flat priors.] To derive the fully conditional distribution for an unknown parameter, we simply pick from (2) the component or components that are related to the parameter of interest. First, consider the grand mean. Since

$$\begin{array}{l}p(\mu \mid ·)\propto exp\left(-\frac{{\sum }_i{\sum }_j{\sum }_k{\sigma }_{b_{ij}}^{-2}{(\mu -(x_{\mathit{ijk}}-g_i-c_j-r_{ij}))}^2}{2}\right)\\ \propto exp\left(-\frac{{\sum }_i{\sum }_j(m_{ij}{\sigma }_{b_{ij}}^{-2}){\mu }^2-2\mu {\sum }_i{\sum }_j{\sum }_k({\sigma }_{b_{ij}}^{-2}(x_{\mathit{ijk}}-g_i-c_j-r_{ij}))}{2}\right),\end{array}$$

the fully conditional distribution of the overall mean, given the data and all other parameters, is

$$\begin{array}{l}\mu \mid ·\sim N\left(\frac{{\sum }_i{\sum }_j{\sum }_k{\sigma }_{b_{ij}}^{-2}(x_{\mathit{ijk}}-g_i-c_j-r_{ij})}{{\sum }_i{\sum }_j(m_{ij}{\sigma }_{b_{ij}}^{-2})},\frac{1}{{\sum }_i{\sum }_j(m_{ij}{\sigma }_{b_{ij}}^{-2})}\right)\\ \sim N\left(\sum _i\sum _j\frac{{\sum }_k(x_{\mathit{ijk}}-g_i-c_j-r_{ij})}{{\mathrm{\Lambda }}_{··}{\sigma }_{b_{ij}}^2},{\mathrm{\Lambda }}_{··}^{-1}\right),\end{array}$$ (3)

where ${\mathrm{\Lambda }}_{··}={\sum }_i{\sum }_j(m_{ij})/({\sigma }_{b_{ij}}^2)$. In (3) and throughout this letter, we use ‘|·’ to mean ‘conditional on the data and all unknown parameters except the one currently under consideration’. For the effect that corresponds to the i-th gene, we pick components in (2) that are related to g_i, which leads to

$$\begin{array}{l}p(g_i\mid ·)\propto exp\left(-\frac{{\sum }_j{\sum }_k({\sigma }_{b_{ij}}^{-2}{(g_i-(x_{\mathit{ijk}}-\mu -c_j-r_{ij}))}^2)}{2}\right)\times exp\left(-\frac{g_i^2}{2{\sigma }_g^2}\right)\\ \propto exp\left(-\frac{({\sum }_j(m_{ij}{\sigma }_{b_{ij}}^{-2})+{\sigma }_g^{-2})g_i^2-2g_i{\sum }_j{\sum }_k({\sigma }_{b_{ij}}^{-2}(x_{\mathit{ijk}}-\mu -c_j-r_{ij}))}{2}\right)\end{array}$$

Thus, the fully conditional distribution of g_i given the data and all other parameters is

$$\begin{array}{l}{g}_i\mid ·\sim N\left(\frac{{\sum }_j{\sum }_k{\sigma }_{ij}^{-2}(x_{\mathit{ijk}}-\mu -c_j-r_{ij})}{{\sigma }_g^{-2}+{\sum }_j{m}_{ij}{\sigma }_{b_{ij}}^{-2}},\frac{1}{{\sigma }_g^{-2}+{\sum }_j{m}_{ij}{\sigma }_{b_{ij}}^{-2}}\right)\\ \sim N\left({\sum }_j\frac{{\sum }_k(x_{\mathit{ijk}}-\mu -c_j-r_{ij})}{({\sigma }_g^{-2}+{\mathrm{\Lambda }}_{i·}){\sigma }_{b_{ij}}^{-2}},{({\sigma }_g^{-2}+{\mathrm{\Lambda }}_{i·})}^{-1}\right),\end{array}$$ (4)

where ${\mathrm{\Lambda }}_{i·}={\sum }_j(m_{ij})/({\sigma }_{b_{ij}}^2)$. Similarly, since

$$\begin{array}{l}p(c_j\mid ·)\propto exp\left(-\frac{{\sum }_i{\sum }_k({\sigma }_{b_{ij}}^{-2}{(c_j-(x_{\mathit{ijk}}-\mu -g_i-r_{ij}))}^2)}{2}\right)\times exp\left(-\frac{c_i^2}{2{\sigma }_c^2}\right)\\ \propto exp\left(-\frac{({\sum }_i(m_{ij}{\sigma }_{b_{ij}}^{-2})+{\sigma }_c^{-2})c_j^2-2c_j{\sum }_i{\sum }_k({\sigma }_{b_{ij}}^{-2}(x_{\mathit{ijk}}-\mu -g_i-r_{ij}))}{2}\right)\end{array}$$

the fully conditional distribution of c_j, the effect that corresponds to the j-th condition, is

$$\begin{array}{l}{c}_j\mid ·\sim N\left(\frac{{\sum }_i{\sum }_k({\sigma }_{b_{ij}}^2(x_{\mathit{ijk}}-\mu -g_i-r_{ij}))}{{\sigma }_c^{-2}+{\sum }_i(m_{ij}{\sigma }_{ij}^{-2})},\frac{1}{{\sigma }_c^{-2}+{\sum }_i(m_{ij}{\sigma }_{b_{ij}}^{-2})}\right)\\ \sim N\left({\sum }_i\frac{{\sum }_k(x_{\mathit{ijk}}-\mu -g_i-r_{ij})}{({\sigma }_c^{-2}+{\mathrm{\Lambda }}_{·j}){\sigma }_{b_{ij}}^{-2}},{({\sigma }_c^{-2}+{\mathrm{\Lambda }}_{·j})}^{-1}\right)\end{array}$$ (5)

where ${\mathrm{\Lambda }}_{·j}={\sum }_i(m_{ij}/{\sigma }_{b_{ij}}^2)$. Comparing (3), (4) and (5) with the first three equations in Appendix 1 of the paper of Cho and Lee (2004), we found that they applied a common quantity ${\mathrm{\Lambda }}_{··}={\sum }_i{\sum }_j(m_{ij}/{\sigma }_{b_{ij}}^2)$ for the three fully conditional distributions and this introduced systematic biases to the posterior estimates of model parameters. In our simulation study, we found this flaw to be dramatically downward-biased posterior estimations of both condition effects and gene effects, and it consequently affected posterior estimation of interaction effects and biological variances. Figure 1 illustrates how it affected posterior sampling of a randomly chosen gene effect toward zero. In contrast, using the corrected fully conditional distribution as shown in (4), we were able to obtain a posterior estimate for this gene effect that was close to its true value (i.e. 0.7923). This situation was representative of all gene and condition effects. However, we observed no significant influence of this flaw on the posterior estimation of μ and ${\sigma }_e^{-2}$. In other words, the posterior estimates of μ and ${\sigma }_e^{-2}$ were very close to their true values using either the algorithm of Cho and Lee (2004) or the corrected one that we propose. This is not very surprising if one considers the restrictions Σ_ig_i =0 and Σ_jc_j =0 in our data simulation. When these restrictions hold, bias in the posterior estimation of g_i and c_j did not significantly affect the posterior sampling of μ and ${\sigma }_e^{-2}$.

Fig. 1.

Comparison of Markov chain sampling of a gene effect using (a) previous algorithm by Cho and Lee (2004) and (b) corrected algorithm by Wu, Forney and Joyce. The simulated gene effect was 0.7923. The y-axis represents posterior estimates of the gene effect whereas the x-axis represents saved (i.e. every one-tenth) Markov chain sampling steps.

In their paper, the conditional distribution x_ijk |· was derived with replicates, as shown below:

$$x_{\mathit{ijk}}\mid ·\sim N\left(\frac{{\sigma }_{b_{ij}}^2{\sum }_l{y}_{\mathit{ijk}l}/n_{\mathit{ijk}}+({\sigma }_e^2/n_{\mathit{ijk}})(\mu +g_i+c_j+r_{ij})}{{\sigma }_{b_{ij}}^2+{\sigma }_e^2/n_{\mathit{ijk}}},{\left(\frac{n_{\mathit{ijk}}}{{\sigma }_e^2}+\frac{1}{{\sigma }_{b_{ij}}^2}\right)}^{-1}\right).$$ (6)

We think it was improper for them to reduce (6) to

$$x_{\mathit{ijk}}\mid ·\sim N(\mu +g_i+c_j+r_{ij},{\sigma }_{b_{ij}}^2),$$

when there is no experimental replication. By our reasoning, no replication is equivalent to putting n_ijk = 1, not n_ijk = 0 as stated in their paper, and ${\sigma }_e^2=0$, implying that

$$x_{\mathit{ijk}}=\left(\frac{1}{n_{\mathit{ijk}}\equiv 1}\right)\sum _{l\equiv 1}{y}_{\mathit{ijk}}=y_{\mathit{ijk}}.$$

Modeling biological/experimental correlations

Cho and Lee (2004) assumed independence for both biological and experimental errors, but these assumptions may be relaxed in the HEM. In this section, we expand their HEM to model biological or/and experimental correlation in gene expression data. First, consider correlations among experimental replicates, which may arise when experimental replicates are exposed to some common environmental factors yet collected at different times or locations. In these cases, the gene expression data may show temporal or spatial variation. In bacterial biofilms, for example, microorganisms undergo significant phenotypic changes during the transition from planktonic to biofilm forms, which reflects altered underlying gene expression and regulation as they adapt to form highly organized and structured sessile communities (Schembri et al., 2003; Watnick and Kolter, 2000) with enhanced resistance to antimicrobial treatments and host immune responses. Furthermore, the development of a biofilm may be tuned differently as a response to a nutrient gradient from the bulk media to the inside of the biofilm. Thus, gene expression at different locations in a biofilm may show spatial variation similar to geographic data in such a way that ‘everything is related to everything else, but near things are more related than distant things’ (Tobler, 1970). Biofilms are the dominant mode of growth for bacteria in the environment, and are of great concern in medical, environmental and industry settings. Therefore, considering the spatial correlation in a model would help better understanding the formation of biofims as well as their altered biological activities.

Consider a balanced experimental design with observations on i = 1, …, G genes under j = 1, …, C conditions with k = 1, …, m biological samples each measured in l = 1, 2, …, n chips. Suppose that the n chips represent n experimental replicates (e.g. layers of a biofilm). Let R_n be a n × n spatial variance–covariance matrix such that the residuals e ~ N(0, I_m^* ⊗ R_n), where I_m^* is an m^*× m^* identity matrix (m^* = G × C × m) and ⊗ indicates the Kronecker product operation. Thus, in the posterior formulation as shown in (2), $p(y\mid x,{\sigma }_e^2)$ is then replaced with p(y | x, R_n) and the latter is given as

$$p(y\mid x,R_n)\propto {\mid I_{m\ast }\otimes R_n\mid }^{-1/2}exp({(y-x)}^{\prime }{(I_{m\ast }\otimes R_n)}^{-1}(y-x)).$$

If we assume that the prior distribution of R_n to be an inverted Wishart distribution with scale matrix V_e with a degree of freedom parameter κ_e [denoted as Wishart⁻¹ (κ_e, V_e)], it can be shown that posterior samples of R_n are sampled from the following:

$$R_n\mid ·\sim {\text{Wishart}}^{-1}(m^{\ast }+{\kappa }_e,S_e+V_e),$$ (7)

where S_e is a n × n matrix with the (l, l) diagonal element being Σ_i Σ_j Σ_k(y_ijkl − x_ijk)² and the (l, l′) off-diagonal element being Σ_i Σ_j Σ_k(y_ijkl − x_ijk)(y_ijkl′ − x_ijk). We assumed inverted Wishart distributions instead of inverted multivariate gamma distributions as the prior distributions for the variance–covariance matrix for the sake of convenience in programming. Next, each element in x is sampled from the following fully conditional distribution

$$x_{\mathit{ijk}}\mid ·\sim N\left(\frac{{\sum }_l({\gamma }^{ll}{y}_{\mathit{ijk}l}+(\frac{1}{2}){\sum }_{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }}(y_{\mathit{ijk}l}+y_{{\mathit{ijkl}}^{\prime }}))+{\sigma }_{b_{ij}}^{-2}{\mu }^{\ast }}{{\sigma }_{b_{ij}}^{-2}+{\sum }_l({\gamma }^{ll}+{\sum }_{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }})}{\left({\sigma }_{b_{ij}}^{-2}+\sum _l\left(r^{ll}+\sum _{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }}\right)\right)}^{-1}\right),$$ (8)

where μ^* = μ + g_i + c_j + r_ij, γ^ll′ is the (l, l′) element of matrix $R_n^{-1}$, tr represents the trace matrix operation. Under the assumption of common experimental error variance (i.e. ${\gamma }^{ll}={\sigma }_e^{-2}$) with zero covariance (i.e. γ^ll′ = 0), (8) is reduced to (6).

To model correlated biological errors, we similarly define a m × m covariance matrix Δ_m such that b_ijk ~ N(0, I_c^* ⊗ Δ_m where I_c^* is an C^* × C^* identity matrix (C^* = G × C). Biological covariance matters in analyzing gene expression data when samples are collected from a single experimental system. Then, p(x | μ, g, c, rΔ) in (2) is replaced with the following

$$p(x\mid \mu ,g,c,r,\mathrm{\Delta })\propto {\mid I_{C\ast }\otimes {\mathrm{\Delta }}_m\mid }^{-1/2}exp({(x-\mu -g-c-r)}^{\prime }{(I_{C\ast }\otimes {\mathrm{\Delta }}_m)}^{-1}(x-\mu -g-c-r))$$

Again, we assume the prior distribution for Δ_m to be an inverted Wishart distribution with scale matrix V_b and degree of freedom parameter κ_b, leading to the following fully conditional distribution of Δ_m.

$${\mathrm{\Delta }}_m\mid ·\sim {\text{Wishart}}^{-1}(C^{\ast }+{\kappa }_b,S_b+V_b)$$ (9)

where S_b is a m × m matrix with the diagonal element Σ_iΣ_j(x_ijk − μ − g_i − c_j − r_ij)² and the off-diagonal element Σ_iΣ_j(x_ijk − μ − g_i − c_j − r_ij)(x_ijk′ − μ − g_i − c_j − r_ij). The fully conditional distributions for elements in g, c and r are derived as below:

$$\mu \mid ·\sim N\left({\sum }_i{\sum }_j{\sum }_k\frac{{\delta }^{kk}{x}_{\mathit{ijk}(\mu )}^{\ast }+(\frac{1}{2}){\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}(x_{\mathit{ijk}(\mu )}^{\ast }+x_{{\mathit{ijk}}^{\prime }(\mu )}^{\ast })}{{\mathrm{\Lambda }}_{··}^{\ast }},{({\mathrm{\Lambda }}_{··}^{\ast })}^{-1}\right),$$ (10)

where δ^kk′ is the (k,k′) element in the inverse matrix ${\mathrm{\Delta }}_m^{-1},x_{\mathit{ijk}(\mu )}^{\ast }=x_{\mathit{ijk}}-g_i-c_j-r_{ij}$ and

$$\begin{array}{l}{\mathrm{\Lambda }}_{··}^{\ast }={\sum }_i{\sum }_j{\sum }_k\left({\delta }^{kk}+{\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}\right).\\ g_i\mid ·\sim N\left({\sum }_j{\sum }_k\frac{{\delta }^{kk}{x}_{\mathit{ijk}(g)}^{\ast }+(\frac{1}{2}){\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}(x_{\mathit{ijk}(g)}^{\ast }+x_{{\mathit{ijk}}^{\prime }(g)}^{\ast })}{{\sigma }_g^{-2}+{\mathrm{\Lambda }}_i^{\ast }},{({\sigma }_g^{-2}+{\mathrm{\Lambda }}_{i·}^{\ast })}^{-1}\right),\end{array}$$ (11)

where

$$\begin{array}{l}{x}_{\mathit{ijk}(g)}^{\ast }=x_{\mathit{ijk}}-\mu -c_j-r_{ij}\text{and}{\mathrm{\Lambda }}_{i·}^{\ast }={\sum }_j{\sum }_k\left({\delta }^{kk}+{\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}\right)\\ c_i\mid ·\sim N\left(\sum _i\sum _k\frac{{\delta }^{kk}{x}_{\mathit{ijk}(c)}^{\ast }+(\frac{1}{2}){\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}(x_{\mathit{ijk}(c)}^{\ast }+x_{{\mathit{ijk}}^{\prime }(c)}^{\ast })}{{\sigma }_c^{-2}+{\mathrm{\Lambda }}_{·j}^{\ast }},{({\sigma }_c^{-2}+{\mathrm{\Lambda }}_{·j}^{\ast })}^{-1}\right),\end{array}$$ (12)

where $x_{\mathit{ijk}(c)}^{\ast }=x_{\mathit{ijk}}-\mu -g_j-r_{ij}$ and ${\mathrm{\Lambda }}_{·j}^{\ast }={\sum }_j{\sum }_k\left({\delta }^{kk}+{\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}\right)$.

$$r_{ij}\mid ·\sim N\left(\sum _k\frac{{\delta }^{kk}{x}_{\mathit{ijk}(r)}^{\ast }+(\frac{1}{2}){\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}(x_{\mathit{ijk}(r)}^{\ast }+x_{{\mathit{ijk}}^{\prime }(r)}^{\ast })}{{\sigma }_r^{-2}+{\mathrm{\Lambda }}_r^{\ast }},{({\sigma }_r^{-2}+{\mathrm{\Lambda }}_r^{\ast })}^{-1}\right),$$ (13)

where $x_{\mathit{ijk}(c)}^{\ast }=x_{\mathit{ijk}}-\mu -g_i-c_j$ and ${\mathrm{\Lambda }}_r^{\ast }={\sum }_k({\delta }^{kk}+{\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }})$. Under the assumption of a non-zero biological covariance, (10), (11) and (12) are reduced to (3), (4) and (5), respectively, and (13) reduced to the fully conditional distribution of r_ij by Cho and Lee (2004). When both biological and experimental errors are correlated, (8) becomes

$$x_{\mathit{ijk}}\mid ·\sim N\left(\frac{{\sum }_l{\gamma }^{ll}{y}_{\mathit{ijk}}+(\frac{1}{2}){\sum }_{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }}(y_{\mathit{ijk}l}+y_{\mathit{ijk}{l}^{\prime }}))+({\delta }^{kk}{\mu }^{\ast }-{\sum }_{k^{\prime }\ne k}{\delta }^{{kk}^{\prime }}(x_{{\mathit{ijk}}^{\prime }}-{\mu }^{\ast }))}{{\delta }^{kk}+{\sum }_l({\gamma }^{ll}+{\sum }_{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }})},{({\delta }^{kk}+{\sum }_l({\gamma }^{ll}+{\sum }_{l^{\prime }\ne l}{\gamma }^{{ll}^{\prime }}))}^{-1}\right)$$ (14)

To show how our expanded model works, we simulated the expression data of 1000 genes, which were measured using 5 biological samples with 3 experimental replicates of each and 10 biological samples each with 6 experimental replicates of each. Briefly, the grand mean was given as a constant. Gene effects were generated from $N(0,{\sigma }_g^2)$, condition effects from $N(0,{\sigma }_c^2)$, and interaction effects from $N(0,{\sigma }_r^2)$, where ${\sigma }_g^2=0.5,{\sigma }_c^2=0.5$ and ${\sigma }_r^2=1.0$. Biological and experimental errors were randomly generated either from b ~ N(0‚I_C^* ⊗ Δ_m) and $e\sim N(0,I_{n\ast }\times {\sigma }_e^2)$, or from $b\sim N(0,I_{m\ast }\times {\sigma }_{b_{ij}}^2)$ and b ~ N(0‚I_m^* ⊗ R_n), where n^* = G×C×m×n, ${\sigma }_e^2=1.25$. We ran 10 independent Markov chains, each consisting of 100 000 updates for each dataset and 1000 burn-in updates. However, to reduce correlation between successive updates, we used only 1 out of every 10 posterior samples generated for a total of 10 000 saved updates per dataset. Model performance was evaluated based on decomposition of mean squared errors (MSE) of each parameter (say g_i) such that

$$\text{MSE}=var({\widehat{g}}_i)+{(\text{bias}({\widehat{g}}_i))}^2$$ (15)

where

$$var({\widehat{g}}_i)=\frac{1}{T}\times \sum _{t=1}^T{({\widehat{g}}_i^t-{\overline{\widehat{g}}}_i)}^2$$

represented sampling variation and

$${(\text{bias}({\widehat{g}}_i))}^2={\left(\frac{1}{T}\times \sum _{t=1}^T{\widehat{g}}_i^t-g_i\right)}^2$$

represented squared bias, and ${\widehat{g}}_i^t$ was an estimate of g_i at saved iteration t=1, 2, …, T.

Our extended model facilitates estimating patterns of biological or/and experimental (co)variance or correlations (Fig. 2) whereas such information could not be inferred using the previous model (Cho and Lee, 2004). As mentioned before, biological or experimental correlations could be of considerable interest and necessary to help us better understand the nature of correlated gene expression. For example, suppose the pattern of experimental (co)variance in Figure 2b represents the environmental correlations in a biofilm. We might conclude that nutrient differences result in higher correlations between the inner and bottom layers than between the top and the other two layers.

Fig. 2.

3D patterns of estimated (co)variance components among (a) five biological sample and (b) three experimental replicates.

Further, we investigated how different model specifications about biological/experimental covariance (i.e. zero versus non-zero) would affect the posterior estimates of parameters of interest. Ignoring experimental covariance in the model did not lead to significant differences in parameter estimation (data not shown). However biological covariance components are important for estimating gene and interaction effects. If the biological covariance components are ignored then gene effects and interaction effects will be estimated with large biases. However, grand mean, condition effects and the common residual variance appear to be more robust, and gave similar results under either model specification about biological (co)variance. As shown in Table 1, considering non-zero biological (co)variance in the model considerably decreased estimation biases of gene and interaction effects. Squared biases, and consequently MSE, of these parameters were smaller in the analysis considering biological covariance components than by ignoring them. Given the variation that occurs during the collection of gene expression data, posterior estimation of gene and interaction effects was better, in terms of MSE, sampling variation and squared biases, when more samples were used (Table 1). This result would challenge the present practice where a very limited number (say three) of samples are used in most gene expression DNA array experiments.

Table 1.

Mean squared error (MSE), sampling variance (SV) and squared bias (SB) in posterior estimation of model effects under the assumption of zero [COV(b) = 0] versus non-zero [COV(b) ≠ 0] biological covariance^a,^b

	COV(b) = 0			COV(b) ≠ 0
	MSE	SV	SB	MSE	SV	SB
B05R03
μ	0.0005	0.0004	0.0001	0.0004	0.0004	<0.0001
gi	0.4976	0.2471	0.2505	0.3750	0.2318	0.1432
cj	0.0024	0.0010	0.0014	0.0016	0.0009	0.0007
rij	0.8770	0.4321	0.4449	0.6667	0.3983	0.2684
V(b)	—	—	—	0.1455	0.0196	0.1259
${\sigma }_e^2$	0.0001	<0.0001	0.0001	0.0001	<0.0001	0.0001
B10R06
μ	0.0005	0.0004	0.0001	0.0004	0.0004	<0.0001
gi	0.4438	0.2276	0.2162	0.2971	0.1909	0.1062
cj	0.0012	0.0008	0.0004	0.0012	0.0008	0.0004
rij	0.6446	0.3500	0.2946	0.4214	0.2381	0.1833
V(b)	—	—	—	0.0924	0.0058	0.0866
${\sigma }_e^2$	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001

^aμ = grand mean; g_i = effect of gene i; c_j = effect of condition j; r_ij = interaction between gene i and condition j; V(b)= variance–covariance matrix of biological errors; ${\sigma }_e^2$ = variance of residual errors.

^bB05R03 = five biological samples each with three experimental replicates; B10R06 = 10 biological samples each with 6 experimental replicates.

Finally, we show how model specifications about biological covariance would affect the estimation of gene expression difference, (μ + g_i + c_j + r_ij) − (μ + g_i + c_j′ + r_ij′) = (c_j−c_j′) + (r_ij − r_ij′) between various conditions (Fig. 3), which is of importance in identifying differential gene expression. Clearly, the estimated gene expression differences were in better agreement with true differences when considering non-zero biological covariance as compared to ignoring them in the model. In the former case, the R-square value for the linear regression of the estimated and true expression differences was higher and the regression slope is closer to 1.0. The low slope in Figure 3b suggests an obvious systematic bias, which resulted from ignoring non-zero biological covariance. On average, the higher the biological correlations were, the greater the difference. In 10 independent simulations starting with different datasets, we found that 6.5–38.7% of the top 100 differentially expressed genes could vary when one assumes zero or non-biological covariance, respectively.

Fig. 3.

Regression of estimated to true gene expression difference obtained by assuming (a) zero versus (b) non-zero biological covariance components.

In conclusion, we extended the model of Cho and Lee (2004) to allow the analysis of gene expression data with biological/environmental correlations, which not only facilitate estimating biological/expression correlation patterns, but it also improved estimation of model parameters and consequently identifying differentially expressed genes in DNA array experiments. A FORTRAN 90 program was developed to implement our extended method and it is freely downloadable at http://www.ibest.uidaho.edu/tools/banal_array/