Bayesian Gene Selection Based on Pathway Information and Network-Constrained Regularization

Abstract

High-throughput data make it possible to study expression levels of thousands of genes simultaneously under a particular condition. However, only few of the genes are discriminatively expressed. How to identify these biomarkers precisely is significant for disease diagnosis, prognosis, and therapy. Many studies utilized pathway information to identify the biomarkers. However, most of these studies only incorporate the group information while the pathway structural information is ignored. In this paper, we proposed a Bayesian gene selection with a network-constrained regularization method, which can incorporate the pathway structural information as priors to perform gene selection. All the priors are conjugated; thus, the parameters can be estimated effectively through Gibbs sampling. We present the application of our method on 6 microarray datasets, comparing with Bayesian Lasso, Bayesian Elastic Net, and Bayesian Fused Lasso. The results show that our method performs better than other Bayesian methods and pathway structural information can improve the result.

1. Introduction

Identifying disease-associated genes, which can be treated as diagnostic biomarkers, can bring a significant effect on disease diagnosis, prognosis, and treatments [1, 2]. With the development of high-throughput technologies in recent years, gene expression profiling has provided a useful way to find biomarkers. Researchers can identify the genes which are differentially expressed between two groups of samples. These genes are regarded as disease-associated genes. However, gene expression data usually contains a large number of genes and a relatively small sample size [3, 4]. And many of the genes are also redundant or irrelevant to the prediction [5, 6]. Furthermore, there are also noises in the experiment procedures which will influence the gene expression values. Therefore, identifying the biomarkers from gene expression data is challenging.

During the last decades, a number of gene selection methods have been developed to tackle this problem. Feature selection and feature extraction are two major methods (we treat gene and feature equally in this paper). On the one hand, the aim of feature selection is to select relevant features and do not change the form of the features. On the other hand, feature extraction will extract the feature from the original data and may alter the form of the features. Here, we focus on the feature selection methods since the results of such methods could be interpreted easily. Feature selection methods can be generally organized into three categories: filter, wrapper, and embedded methods. Both the wrapper and embedded methods are classifier-dependent methods; thus, they are always time consuming and easy to overfitting. However, the filter methods are usually based on statistic approaches [7] such as mRMR [5], PLSRFE [8], lasso [9], and elastic net [10], which are relatively efficient in terms of computation and can derive a score of each of the genes which represents the significance of the gene. Therefore, we focus on the filter methods in this paper.

Although these methods are successful in many applications, they usually obtain suboptimal solutions. Therefore, the prediction accuracies are not satisfied and the disease-associated genes selected from different methods have few overlaps [11]. This is partly due to the fact that the discriminatory power of many biomarkers is similar. Furthermore, some genes which have low discriminatory powers play important roles in cellular functions. Their combinations are highly discriminative while they are usually ignored [12, 13].

Recently, with a large amount of biological information accumulated, there is an increased interest in gene selection with incorporating information on pathways, which can partially compensate for the lack of reliable expression data [14]. Pathways depict a series of chemical interactions in living cells; genes that interact with one another usually mean that they function together concertedly. Therefore, these genes should be highly correlated and have dependence structures. However, many studies only utilize the information that pathways cluster genes into the natural group; the pathway structural information is neglected. Li and Li have overcome this disadvantage by incorporating pathway structure information through a Laplacian matrix of a global graph [15, 16] and combined with lasso penalty to perform network-constrained penalty which can select subgroups of correlated features in the network. This penalty is based on the assumption that genes belonging to the same pathway have similar functions and therefore smoothed regression coefficients. And this penalty has been successfully applied in many studies [17–19].

The Bayesian approach has three major advantages over Bayesian selection methods [20]. Firstly, hyperparameters can be estimated automatically through fulfilling stochastic draws; thus, 10-fold cross-validation for estimating penalized parameters is not required. Secondly, the Bayesian framework can utilize the pathway information naturally by integrating it in the model as prior knowledge. Finally, the Bayesian estimation with the posterior distributions can provide credible intervals for the regression coefficients, which is a great advantage over frequentist methods.

In this paper, we work with a Bayesian framework to perform gene selection through network-constrained regularization. Similar to the Bayesian Lasso [21], Bayesian Elastic Net [22], and Bayesian Fused Lasso [23], we use shrinkage priors to perform regularization. We show that all the conditional posteriors of the proposed model are available in closed form and proper. Thus, parameter estimation can be performed through Gibbs sampling easily. The pathway information is obtained from the Kyoto Encyclopedia of Genes and Genomes (KEGG) [24], which is the most popular pathway public database, especially pathways associated with several types of cancer could be obtained in the model. Furthermore, following Held and Holmes [25], we extend the regression model to binary regression which can perform binary classification through an auxiliary variable. This method is assessed by applying it to several microarray datasets.

2. Method

2.1. The Bayesian Network-Constrained Model for Gene Selection

Considering an N × P matrix X, where P is the number of genes and N is the number of the samples, with a response vector y = (y₁, ⋯,y_n)^T, we normalize the values of each feature as the tradition in variable selection; thus, the mean and standard deviation of each feature are 0 and 1. We assume the likelihood function of the continuous response is Gaussian function:

$$\begin{array}{c}Y\hspace{0.17em}\mid \hspace{0.17em}X,\beta ,{\sigma }^2\sim N_n\left(X\beta ,{\sigma }^2{I}_n\right),\end{array}$$ (1)

which can be also expressed as

$$\begin{array}{c}y=X\beta +\epsilon ,\epsilon \sim N_n\left(0,{\sigma }^2{I}_n\right).\end{array}$$ (2)

Following Li and Li's work [16], we incorporate the pathway information through its normalized Laplacian matrix. Consider an undirected graph G = (V, E, W). In this graph, genes are represented by a set of nodes V, and the interactions between genes are represented by a set of edges E = {u ~ v}, and W is the weights of the edges, where w(u, v) represents the weight of edge e = (u ~ v) which indicates the uncertainty of the edge between the vertices u and v. The degree of each vertex is defined as d_v = ∑_u∼vw(u, v). Then, the normalized Laplacian matrix L for graph G with the uth and vth elements can be defined by

$$\begin{array}{c}L\left(u,v\right)=\left\{\begin{array}{c}1-\frac{w\left(u,v\right)}{d_u},\hspace{1em}\text{if\hspace{0.17em}}u=v\text{\hspace{0.17em}and\hspace{0.17em}}{d}_u\ne 0,\hfill \\ -\frac{w\left(u,v\right)}{\sqrt{d_u{d}_v}},\hspace{1em}\text{if\hspace{0.17em}}u\text{\hspace{0.17em}and\hspace{0.17em}}v\text{\hspace{0.17em}are\hspace{0.17em}adjacent},\hfill \\ 0,\text{otherwise}.\hfill \end{array}\right.\end{array}$$ (3)

Here, we let w(u, v) = 1 if there exists an interaction between gene u and v, and w(u, v) = 0, otherwise.

To form the network-constrained regularization, we assign the prior distribution for β as follows:

$$\begin{array}{c}\beta \sim N_p\left(0,\frac{{\sigma }^2}{r}{\mathrm{\Lambda }}^{-1}\right),\end{array}$$ (4)

where Λ is taking the form:

$$\begin{array}{c}\mathrm{\Lambda }=\mathrm{diag}\left({\tau }_1^{-1},{\tau }_2^{-1},\cdots ,{\tau }_p^{-1}\right)+L=\left\{\begin{array}{cccc}1+{\tau }_1^{-1}& L\left(1,2\right)& \cdots & L\left(1,p\right)\\ L\left(2,1\right)& 1+{\tau }_2^{-1}& \cdots & L\left(2,p\right)\\ ⋮& ⋮& \ddots & ⋮\\ L\left(p,1\right)& L\left(p,2\right)& \cdots & 1+{\tau }_p^{-1}\end{array}\right\}.\end{array}$$ (5)

Note that Λ only contains hyperparameter τ.

To eliminate the |Λ|^1/2 in the prior distribution of β, we assign the prior distribution for τ as follows:

$$\begin{array}{c}p\left({\tau }^2\hspace{0.17em}\mid \hspace{0.17em}\lambda \right)=C_{\tau }{\left|\mathrm{\Lambda }\right|}^{-1/2}\prod _{j=1}^p\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }_j^2\right),\end{array}$$ (6)

where C_τ is the normalizing constant.

The prior distribution defined in (6) is proper, due to the following analysis:

Let A = Λ − I_n, and A is a symmetric and positive semidefinite matrix.

Let D_A = diag(a₁, ⋯, a_p), where a₁, ⋯, a_p are eigenvalues of A and 0 ≤ a₁ ≤ ⋯≤a_p.

Since A is the symmetric and positive semidefinite, there exists an orthonormal matrix Q. Hence, the eigendecomposition of matrix A can be written as A = QD_AQ^T.

Because of Λ = A + I_n = QD_AQ^T + QQ^T = Q(D_A + I_n)Q^T, so ∣Λ | = ∏_i=1ⁿ(a_i + 1) ≥ 1.

Then,

$$\begin{array}{c}{\int }_0^{\infty }{C}_{\tau }{\left|\mathrm{\Lambda }\right|}^{-1/2}\prod _{j=1}^p\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }_j^2\right)d{\tau }^2\le C_{\tau }{\int }_0^{\infty }\prod _{j=1}^p\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }_j^2\right)d{\tau }^2<\infty ,\end{array}$$ (7)

where the integrand is kernels of the gamma density that indicates the integral is finite. Therefore, the prior distribution is proper.

Since

$$\begin{array}{c}{\beta }^T\mathrm{\Lambda }\beta ={\beta }^T{D}^{-1}\beta +\sum _{u\sim v}{\left(\frac{{\beta }_u}{\sqrt{d_u}}-\frac{{\beta }_v}{\sqrt{d_v}}\right)}^2\ge 0,\hspace{1em}D=\mathrm{diag}\left({\tau }_1^2,{\tau }_2^2,\cdots ,{\tau }_p^2\right),\end{array}$$ (8)

Λ is positive semidefinite.

The joint posterior distribution can be written as

$$\begin{array}{c}p\left(\beta ,\lambda ,{\sigma }^2,{\tau }^2,r\hspace{0.17em}\mid \hspace{0.17em}X,Y\right)\propto {\left({\sigma }^2\right)}^{-n/2}\exp \left(-\frac{{‖Y-X\beta ‖}^2}{2{\sigma }^2}\right){\left({\sigma }^2\right)}^{-p/2}{r}^{p/2}{\left|\mathrm{\Lambda }\right|}^{1/2}\exp \left(-\frac{r\beta D^{-1}\beta +r{\beta }^{T}L\beta }{2{\sigma }^2}\right){\left|\mathrm{\Lambda }\right|}^{-1/2}\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right)p\left(r\right)p\left({\sigma }^2\right)p\left(\lambda \right).\end{array}$$ (9)

Integrating out τ², we have

$$\begin{array}{c}p\left(\beta ,\lambda ,{\sigma }^2,r\hspace{0.17em}\mid \hspace{0.17em}X,Y\right)=\int p\left(\beta ,\lambda ,{\sigma }^2,{\tau }^2,r\hspace{0.17em}\mid \hspace{0.17em}X,Y\right)p\left({\tau }^2\right)d{\tau }^2\propto {\int }_0^{\infty }{\left({\sigma }^2\right)}^{-n/3}\exp \left(-\frac{{‖Y-X\beta ‖}^2}{2{\sigma }^2}\right){\left({\sigma }^2\right)}^{-p/2}{r}^{p/2}{\left|\mathrm{\Lambda }\right|}^{1/2}\exp \left(-\frac{r\beta D^{-1}\beta +r{\beta }^{T}L\beta }{2{\sigma }^2}\right){\left|\mathrm{\Lambda }\right|}^{-2/2}\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right)p\left(r\right)p\left({\sigma }^2\right)p\left(\lambda \right)d{\tau }^2\propto {\int }_0^{\infty }\exp \left(-\frac{{‖Y-X\beta ‖}^2+r{\beta }^{T}L\beta }{2{\sigma }^2}\right)\exp \left(-\frac{r\beta D^{-1}\beta }{2{\sigma }^2}\right)\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right)d{\tau }^2.\end{array}$$ (10)

Applying the fact as follows to the above equation:

$$\begin{array}{c}\frac{a}{2}\exp \left(-a\hspace{0.17em}\mid \hspace{0.17em}z\hspace{0.17em}\mid \hspace{0.17em}\right)={\int }_0^{\infty }\frac{1}{\sqrt{2\pi s}}\exp \left(-\frac{z^2}{2s}\right)\frac{a^2}{2}\exp \left(-\frac{a^{2}s}{2}\right)ds,a>0,\end{array}$$ (11)

we have

$$\begin{array}{c}p\left(\beta ,\lambda ,{\sigma }^2,r\hspace{0.17em}\mid \hspace{0.17em}X,Y\right)\propto {\int }_0^{\infty }\exp \left(-\frac{{‖Y-X\beta ‖}^2+r{\beta }^{T}L\beta }{2{\sigma }^2}\right)\exp \left(-\frac{r\beta D^{-1}\beta }{2{\sigma }^2}\right)\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right)d{\tau }^2=\exp \left(-\frac{{‖Y-X\beta ‖}^2+r\lambda \hspace{0.17em}\mid \hspace{0.17em}\beta \hspace{0.17em}\mid \hspace{0.17em}+r{\beta }^{T}L\beta }{2{\sigma }^2}\right).\end{array}$$ (12)

Thus, maximizing the posterior distribution is equivalent to minimizing the following equation:

$$\begin{array}{c}L\left(r,\lambda ,\beta \right)={\left(y-X\beta \right)}^T\left(y-X\beta \right)+r\lambda {\left|\beta \right|}_1+{\lambda \beta }^{T}L\beta ,\end{array}$$ (13)

which has the same regularization term as the method proposed in [19].

We assign the prior distribution for σ² as follows:

$$\begin{array}{c}{\sigma }^2\sim \text{Inverse\hspace{0.17em}Gamma}\left(a,b\right).\end{array}$$ (14)

And we assign the following prior for the hyperparameters r and λ:

$$\begin{array}{c}\begin{array}{c}r\sim \text{Gamma}\left(c,d\right),\\ \lambda \sim \text{Gamma}\left(e,f\right).\end{array}\end{array}$$ (15)

Then, the hierarchical Bayesian model is

$$\begin{array}{c}\begin{array}{c}Y\hspace{0.17em}\mid \hspace{0.17em}X,\beta ,{\sigma }^2\sim N_n\left(X\beta ,{\sigma }^2{I}_n\right),\\ \beta \hspace{0.17em}\mid \hspace{0.17em}{\sigma }^2,{\tau }^2,r\sim N_p\left(0,\frac{{\sigma }^2}{r}{\mathrm{\Lambda }}^{-1}\right),\\ {\tau }^2\hspace{0.17em}\mid \hspace{0.17em}\lambda \sim {\left|\mathrm{\Lambda }\right|}^{-1/2}\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right),\\ {\sigma }^2\sim \text{Inverse\hspace{0.17em}Gamma}\left(a,b\right),\\ r\sim \text{Gamma}\left(c,d\right),\\ \lambda \sim \text{Gamma}\left(e,f\right).\end{array}\end{array}$$ (16)

2.2. Gibbs Sampling Method

The likelihood is

$$\begin{array}{c}p\left(y\hspace{0.17em}\mid \hspace{0.17em}X,\beta ,{\sigma }^2\right)\propto {\left({\sigma }^2\right)}^{-n/2}\exp \left(-\frac{{\left(Y-X\beta \right)}^T\left(Y-X\beta \right)}{2{\sigma }^2}\right).\end{array}$$ (17)

According to the above hierarchical model and the likelihood, the joint posterior distribution on data is

$$\begin{array}{c}p\left(\beta ,{\sigma }^2,{\tau }^2,{\lambda }^2,r\hspace{0.17em}\mid \hspace{0.17em}Y,X\right)\propto {\left({\sigma }^2\right)}^{-n/2}\exp \left(-\frac{{\left(y-X\beta \right)}^T\left(y-X\beta \right)}{2{\sigma }^2}\right){\left({\sigma }^2\right)}^{-p/2}{r}^{p/2}{\left|\mathrm{\Lambda }\right|}^{1/2}\exp \left(-\frac{r{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}\right){\left|\mathrm{\Lambda }\right|}^{-1/2}\prod _{j=1}^p\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }_j^2\right){\left({\sigma }^2\right)}^{-a}\exp \left(-\frac{b}{{\sigma }^2}\right)r^{-c}\exp \left(-dr\right){\left({\lambda }^2\right)}^{-e}\exp \left(-f{\lambda }^2\right).\end{array}$$ (18)

Due to the fact that all the prior distributions are conjugated, the full conditional posterior distributions for the parameters have closed forms.

$$\begin{array}{c}p\left(\beta ,\hspace{0.17em}\mid \hspace{0.17em}{\sigma }^2,{\tau }^2,r,Y,X\right)\propto \exp \left(-\frac{{\left(Y-X\beta \right)}^T\left(Y-X\beta \right)}{2{\sigma }^2}\right)\exp \left(-\frac{r\beta \mathrm{\Lambda }\beta }{2{\sigma }^2}\right)\propto \exp \left(-\frac{\left(X^{\prime }X+r\mathrm{\Lambda }\right){\beta }^2-2YX\beta }{2{\sigma }^2}\right).\end{array}$$ (19)

Let μ = (X′X + rΛ)⁻¹X′Y, Σ = σ²(X′X + rΛ)⁻¹ , we have

$$\begin{array}{c}\beta \hspace{0.17em}\mid \hspace{0.17em}{\sigma }^2,{\tau }^2,r,X,Y\sim N_p\left(\mu ,\mathrm{\Sigma }\right),\end{array}$$ (20)

$$\begin{array}{c}p\left({\sigma }^2\hspace{0.17em}\mid \hspace{0.17em}\beta ,{\tau }^2,r,Y,X\right)\propto {\left({\sigma }^2\right)}^{-n/2}\exp \left(-\frac{{\left(y-X\beta \right)}^T\left(y-X\beta \right)}{2{\sigma }^2}\right){\left({\sigma }^2\right)}^{-p/2}\exp \left(-\frac{r{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}\right){\left({\sigma }^2\right)}^{-a}\exp \left(-\frac{b}{{\sigma }^2}\right)\propto {\left({\sigma }^2\right)}^{-n+p/2-a}\exp \left(\left(-\frac{{\left(y-X\beta \right)}^T\left(y-X\beta \right)+r{\beta }^T\mathrm{\Lambda }\beta }{2}+b\right)\frac{1}{{\sigma }^2}\right),\end{array}$$ (21)

$$\begin{array}{c}{\sigma }^2\hspace{0.17em}\mid \hspace{0.17em}\beta ,{\tau }^2,r,Y,X\sim \text{Inverse\hspace{0.17em}Gamma}\left(\frac{n+p}{2}+a,\frac{{\left(y-X\beta \right)}^T\left(y-X\beta \right)+r{\beta }^T\mathrm{\Lambda }\beta }{2}+b\right),\end{array}$$ (22)

$$\begin{array}{c}p\left({\tau }^2\hspace{0.17em}\mid \hspace{0.17em}\beta ,{\sigma }^2,{\lambda }^2,r\right)\propto \exp \left(-\frac{r{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}\right)\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right).\end{array}$$ (23)

This implies that τ² follows a generalized inverse Gaussian distribution:

$$\begin{array}{c}{\tau }_j^2\hspace{0.17em}\mid \hspace{0.17em}\beta ,r,{\sigma }^2,{\lambda }^2\sim \text{GIG}\left(\frac{1}{2},{\lambda }^2,\frac{r{\beta }_j^2}{{\sigma }^2}\right),\hspace{1em}j=1,2,\cdots ,p,\end{array}$$ (24)

$$\begin{array}{c}p\left(r\hspace{0.17em}\mid \hspace{0.17em}\beta ,{\sigma }^2,{\tau }^2\right)\propto r^{p/2}\exp \left(-\frac{r{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}\right)r^c\exp \left(-dr\right)\propto r^{{}^{p/2}+c}\exp \left(-\left(\frac{{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}+d\right)r\right),\end{array}$$ (25)

$$\begin{array}{c}r\hspace{0.17em}\mid \hspace{0.17em}{\sigma }^2,\beta ,{\tau }^2\sim \text{Gamma}\left(\frac{p}{2}+c,\frac{{\beta }^T\mathrm{\Lambda }\beta }{2{\sigma }^2}+b\right),\end{array}$$ (26)

$$\begin{array}{c}p\left({\lambda }^2\hspace{0.17em}\mid \hspace{0.17em}{\tau }^2\right)\propto \prod _{j=1}^p\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }_j^2\right){\left({\lambda }^2\right)}^e\exp \left(-f{\lambda }^2\right)\propto {\left({\lambda }^2\right)}^{p+e}\exp \left(-\left(\frac{1}{2}\sum _{j=1}^p{\tau }_j^2+f\right){\lambda }^2\right),\end{array}$$ (27)

$$\begin{array}{c}{\lambda }^2\hspace{0.17em}\mid \hspace{0.17em}{\tau }^2\sim \text{Gamma}\left(p+e,\frac{1}{2}\sum _{j=1}^p{\tau }_j^2+f\right).\end{array}$$ (28)

The Gibbs sampling scheme iterates as follows:

1. Update β by sampling from (20)

2. Update σ² by sampling form (22)

3. Update τ² by sampling from (24)

4. Update r by sampling from (26)

5. Update λ by sampling from (28)

2.3. The Binary Response Case

Binary data such as absence or presence or different types of a disease are often used as response variables in gene selection problems. To perform binary classification, we use probit regression using auxiliary variables. Then, the model can be represented as follows:

$$\begin{array}{c}P\left(y_i=1\right)=X_i\beta ,\end{array}$$ (29)

where X_i is the ith sample and P(y_i = 1) is the probability of y_i = 1. Here, latent variables Z = (z₁, z₂, ⋯, z_n) are defined as

$$\begin{array}{c}{z}_i=X_i\beta +\epsilon ,\epsilon \sim N_n\left(0,{\sigma }^2{I}_n\right).\end{array}$$ (30)

Then, the full conditional posterior distribution for each z_i is truncated normal:

$$\begin{array}{c}{z}_i\hspace{0.17em}\mid \hspace{0.17em}\beta ,X_i,y_i\sim \left\{\begin{array}{c}N\left(X_i\beta ,{\sigma }^2\right)I\left(z_i>0\right),\hspace{1em}{y}_i=1,\hfill \\ N\left(X_i\beta ,{\sigma }^2\right)I\left(z_i\le 0\right),\hspace{1em}\text{otherwise}.\hfill \end{array}\right.\end{array}$$ (31)

And Z follows a multivariate truncated normal distribution:

$$\begin{array}{c}p\left(Z\hspace{0.17em}\mid \hspace{0.17em}\beta ,{\sigma }^2,X,Y\right)\propto N_n\left(X\beta ,{\sigma }^2{I}_n\right)\prod _{i=1}^{n}I\left(A_i\right),\end{array}$$ (32)

$$\begin{array}{c}{A}_i=\left\{\begin{array}{c}\left\{Z_i\hspace{0.17em}\mid \hspace{0.17em}{Z}_i>0\right\},\left(Y_i=1\right),\hfill \\ \left\{Z_i\hspace{0.17em}\mid \hspace{0.17em}{Z}_i\le 0\right\},\left(Y_i=0\right).\hfill \end{array}\right.\end{array}$$ (33)

Sampling from this distribution directly is difficult. We use the method proposed in [26] to sample this latent variable.

Then, the hierarchical Bayesian model is

$$\begin{array}{c}\begin{array}{c}Z\hspace{0.17em}\mid \hspace{0.17em}X,Y,\beta ,{\sigma }^2\sim N_n\left(X\beta ,{\sigma }^2{I}_n\right)\prod _{i=1}^{n}I\left(A_i\right),\\ \beta \hspace{0.17em}\mid \hspace{0.17em}{\sigma }^2,{\tau }^2,r\sim N_p\left(0,\frac{{\sigma }^2}{r}{\mathrm{\Lambda }}^{-1}\right),\\ {\tau }^2\hspace{0.17em}\mid \hspace{0.17em}\lambda \sim {\left|\mathrm{\Lambda }\right|}^{-1/2}\frac{{\lambda }^2}{2}\exp \left(-\frac{{\lambda }^2}{2}{\tau }^2\right),\\ {\sigma }^2\sim \text{Inverse\hspace{0.17em}Gamma}\left(a,b\right),\\ r\sim \text{Gamma}\left(c,d\right),\\ \lambda \sim \text{Gamma}\left(e,f\right).\end{array}\end{array}$$ (34)

To derive the Gibbs sampling scheme, we only need to replace Y with Z in the Gibbs sampling scheme defined in Section 2.2. And the latent variables Z are sampled from (32).

3. Results

3.1. Datasets and Preprocessing

To demonstrate the effectiveness of our methods, a regression microarray dataset and 5 real-life binary classification microarray datasets were tested in this paper, which are described as follows. The pathway information was obtained from the KEGG database.

A breast cancer dataset was used to predict the survival time of patients [27]. We used gene expression profiles of 76 patients. Each patient was measured with 24481 probes. 3592 genes were found in the KEGG database from this dataset. We used the logarithm of survival times of patients as the response variable in this dataset.

The other 5 binary classification microarray datasets are shown in Table 1. No. genes mean the genes we found both existing in the microarray dataset and KEGG pathway database.

Table 1.

Binary classification microarray datasets used.

Dataset name	No. genes	Samples	P/N	References
Leukemia	1883	72	47/25	[28]
DLBCL	2427	77	58/24	[29]
Prostate	3238	102	50/52	[30]
GSE412	3234	108	60/48	[31]
GSE4922	4476	204	70/134	[26]

Lastly, the gene expression values were normalized; thus, its mean and standard deviation are 0 and 1.

3.2. Parameter Settings

In the procedure of Bayesian network-constrained regularization, we recommend small values for a, b, c, d, e, f in (16) and we set these values to 0.01 in our experiments. The Gibbs sampling iteration was conducted 6000 times, and we chose the second half of the samples to estimate the regression parameters. The posterior estimates of all parameters were obtained through the posterior averages of the chains. For the classification problem, the classifiers were built by a support vector machine (SVM). In this paper, we used the radial basic function as the kernel function in SVM. And the regularization parameter and the kernel width parameter were optimized by a grid search approach. We used Libsvm [32] to model the SVM.

3.3. Results and Analysis

In this section, we will describe the results on 6 microarray gene expression datasets (Table 1) to evaluate the performance of the proposed method. Our method was compared with the other three Bayesian regularized regression methods, including Bayesian Lasso, Bayesian Elastic Net, and Bayesian Fused Lasso. A comprehensive review of these methods can be found in [23]. When L = I, which means we know nothing about the pathway structure, the Bayesian network-constrained regularization is equivalent to Bayesian Elastic Net. And when L = O, our method is equivalent to Bayesian Lasso. These three methods can also be extended to perform binary classification through an auxiliary variable. We also used Gibbs sampling to perform parameter estimation. Previous review [23] also shows that these three Bayesian methods' performances are similar to and in some cases better than the frequentist methods. Prediction mean square error was used to evaluate the performance on regression problem. Meanwhile, ACC and AUC were used as the evaluation criteria for binary classification problem. According to previous studies, the number of important genes is probably about 50 [28]; thus, we selected the top 50 genes based on the absolute value of their regression coefficient for the binary classification problem.

Figure 1 shows the performance of all the four methods on the regression microarray dataset. And the classification performances on the five binary classification microarray datasets are summarized in Table 2. In the binary classification datasets, the first three datasets are usually treated as easy classification datasets, while the other two datasets are relatively hard to classify. From Figure 1, we can see that the PMSE of our method is lower than other Bayesian methods. Table 2 also shows that on the four easy binary classification datasets, our method achieves the highest ACC and AUC. In the other two hard classification datasets, our method achieves the highest ACC and AUC on GSE412. Although the AUC of Bayesian Elastic Net is higher than our method on GSE4922, our method achieves the highest ACC. In general, Bayesian network-constrained regularization shows better prediction and classification ability than other three Bayesian methods, which is similar to the results implied by [15]. Since our method can be transferred to Bayesian Lasso or Bayesian Elastic Net when the normalized Laplacian matrix L = O or L = I, the results also show that pathway information indeed contributes to the accuracy of the gene selection.

Figure 1.

PMSE performance on regression microarray dataset.

Table 2.

Comparison of results of 4 Bayesian methods.

Dataset	Methods	AUC	ACC
Leukemia	BEN	0.9955	0.9600
	BFused	1	0.9733
	BLasso	1	0.9447
	BNR	1	0.9733
DLBCL	BEN	0.9674	0.9223
	BFused	0.9674	0.9223
	BLasso	0.9485	0.9223
	BNR	0.9958	0.9482
Prostate	BEN	0.9784	0.9414
	BFused	0.9655	0.9314
	BLasso	0.9784	0.9419
	BNR	0.9900	0.9510
GSE412	BEN	0.9428	0.8498
	BFused	0.9046	0.8619
	BLasso	0.9541	0.8792
	BNR	0.9637	0.9074
GSE4922	BEN	0.6274	0.6666
	BFused	0.6028	0.6523
	BLasso	0.6132	0.6860
	BNR	0.6132	0.6860

Consistent with previous studies [33, 34], all the Bayesian regularization regression methods could classify Leukemia, DLBCL, Prostate, and GSE412 dataset accurately. However, the performances of all the methods were poor on GSE 4922 dataset. Therefore, we demonstrate the effectiveness of our method by selecting the top 18 genes which make the prediction accuracy to achieve the highest value and most of those genes are associated with breast cancer (Table 3).

Table 3.

Description of top 18 genes of GSE4922.

Gene symbol	Description	Reference
SYCP3∗	Synaptonemal complex protein 3	[35]
CDKN2A∗	Cyclin dependent kinase inhibitor 2A	[36]
PLB1∗ CTNNBIP1∗	Phospholipase B1 Catenin beta-interacting protein 1	[37] [38]
GBE1∗	1,4-Alpha-glucan-branching enzyme 1	[39]
SMURF1∗	SMAD-specific E3 ubiquitin protein ligase 1	[40]
NR1H4	Nuclear receptor subfamily 1 group H member 4	/
PDE11A	Phosphodiesterase 11A	/
UGT1A1∗	UDP glucuronosyltransferase family 1 member A1	[41]
FGF19∗	Fibroblast growth factor 19	[42]
OR51B4∗	Olfactory receptor family 51 subfamily B member 4	[43]
RAB7A∗	RAB7A, member RAS oncogene family	[44]
SDHD∗	Succinate dehydrogenase complex subunit D	[45]
IFNA8	Interferon alpha 8	/
VANGL2∗	VANGL planar cell polarity protein 2	[46]
UMPS	Uridine monophosphate synthetase	/
CASP3	Caspase 3	[47]
SUFU	SUFU negative regulator of hedgehog signaling	[48]

^∗The gene was reported as an oncogene in previous literatures.

4. Conclusion

In this paper, we propose a Bayesian approach to perform gene selection, which can incorporate the pathway information as prior biological knowledge through network-constrained regularization to improve the accuracy of gene selection. All the prior distributions we propose are strictly conjugated; thus, all the conditional posteriors of the model are available in closed form. An auxiliary variable is also introduced to extend the regression model to perform binary classification. An efficient Gibbs sampling method is used to estimate regression coefficients and tune parameters simultaneously, which can perform feature filter feasible for high dimensional microarray datasets. The performance of the proposed method is demonstrated by applying it to a regression microarray dataset and five binary classification microarray datasets. The results show that compared with Bayesian Lasso, Bayesian Elastic Net, and Bayesian Fused Lasso, our method performs better both in prediction and classification. And the pathway information indeed improves the accuracy of gene selection.

Data Availability

The breast cancer dataset could be obtained from the R package breast cancer NKI. Leukemia, DLBCL, and Prostate datasets are available on the website http://portals.broadinstitute.org/cgi-bin/cancer/. GSE412 and GSE4922 datasets are available in the GEO of NCBI under accession GSE412 and GSE4922.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.