Multiple Testing of Submatrices of a Precision Matrix with Applications to Identification of Between Pathway Interactions

Abstract

Making accurate inference for gene regulatory networks, including inferring about pathway by pathway interactions, is an important and difficult task. Motivated by such genomic applications, we consider multiple testing for conditional dependence between subgroups of variables. Under a Gaussian graphical model framework, the problem is translated into simultaneous testing for a collection of submatrices of a high-dimensional precision matrix with each submatrix summarizing the dependence structure between two subgroups of variables.

A novel multiple testing procedure is proposed and both theoretical and numerical properties of the procedure are investigated. Asymptotic null distribution of the test statistic for an individual hypothesis is established and the proposed multiple testing procedure is shown to asymptotically control the false discovery rate (FDR) and false discovery proportion (FDP) at the pre-specified level under regularity conditions. Simulations show that the procedure works well in controlling the FDR and has good power in detecting the true interactions. The procedure is applied to a breast cancer gene expression study to identify between pathway interactions.

1 Introduction

Simultaneous inference for the interactions among a large number of variables is an important problem in statistics with a wide range of applications. Many statistical methods have been proposed to infer about pairwise interactions (Ritchie et al., 2001; Chatterjee et al., 2006; Kooperberg and Ruczinski, 2005; Kooperberg and LeBlanc, 2008; Fan and Lv, 2008; Cai and Zhang, 2014; Cai and Liu, 2015, e.g). Most of the existing methods focus on marginal assessments of pairwise interactions without conditioning on the other variables. Such marginal methods may result in false identification of interactions due to the discrepancy between conditional and unconditional effects. When prior knowledge is available to group the variables of interest, it is often of interest to make simultaneous inference for the interactions at the group level. For example, functionally related genes are often grouped into pathways and inferring about between pathway interactions is important as they represent a majority of the genetic interactions (Kelley and Ideker, 2005).

Motivated by applications in genomics, in this paper we propose methods to efficiently identify between group interactions while accounting for the joint effects from all other variables of interest. Under a Gaussian graphical model framework, we translate the problem of detecting between group interactions into the statistical problem of simultaneous testing of a collection of submatrices of a high-dimensional precision matrix. We first discuss the motivating problem of detecting between pathway interactions before presenting the framework for large-scale multiple testing of submatrices of a high-dimensional precision matrix.

1.1 Detection of Between Pathway Interactions

It is well known that genes interact functionally in networks to orchestrate cellular processes. Biological interactions of genes are often inferred based on co-expression networks since coexpressed genes tend to be functionally related or controlled by the same transcriptional regulatory elements (Weirauch, 2011). Throughout, we use the term gene-gene interaction to refer to their biological interaction, quantified by conditional co-expression (given all other genes), rather than statistical interaction unless specified otherwise. Accurately identifying important gene-gene interactions is a difficult task due to the high dimensionality of the feature space spanned by gene pairs. Particularly in genome-wide studies where the sample sizes are typically small compared to the number of interactions of interest, gene level analyses often produce results that are difficult to interpret or replicate.

One approach to improve the interpretability and reproducibility is to incorporate prior biological knowledge such as gene structure or protein-protein interaction network information to group functionally related genes into pathways and perform analysis at the pathway level. Throughout, we use the term pathway to refer generically a gene group under study, whether or not the group is indeed representing a metabolic or signaling pathway. A large number of knowledge bases have become available to assemble biologically meaningful gene groups (Xenarios et al., 2002; Rual et al., 2005; Matthews et al., 2009; Craven and Kumlien, 1999; Khatri et al., 2012). The knowledge bases provide prior information on biological processes, components, or structures in which individual genes and proteins are involved in. Analyzing high-throughput molecular measurements at the functional level is very appealing due to its potential in reducing the complexity of the problem and improving the power (Subramanian et al., 2005; Glazko and Emmert-Streib, 2009).

Detecting pathway level interactions is also biologically relevant because in order to produce appropriate physiological responses to both internal and external factors, pathways often need to function in a coordinated fashion due to the complex nature of biological systems. In addition, there is accumulating evidence that complex traits are often influenced by multiple groups of functional related genes through their dynamic interaction and coregulation (Jia et al., 2011). Therefore, the knowledge of pathway crosstalk network is helpful for inferring the function of complex biological systems (Li et al., 2008). A wide range of between pathway crosstalk have been identified as critical for understanding many diseases including breast cancer, lung cancer, ovarian cancer, major depression disorder, and Alzheimer (Osborne et al., 2005; Shou et al., 2004; Jia et al., 2011; Liu et al., 2010; Pan, 2012; Puri et al., 2008).

In addition to applications to the identification of between genetic pathway interactions, the proposed procedures are also useful for other settings. Examples include interactions between biological markers when markers are measured at different time points with multiple measurements of each marker representing one group; and interactions between different brain regions when functional MRI measurements are taken over the entire brain with groups indexed by brain regions. We next describe our proposed framework for detecting between pathway interactions based on testing for submatrices of a high-dimensional precision matrix.

1.2 Multiple Testing of Submatrices of A Precision Matrix

Under a Gaussian graphical model framework, we formulate the problem of identifying between group interactions that account for joint effects from all genes of interest as the statistical problem of simultaneous testing of submatrices of a high-dimensional precision matrix. Let {X₁, · · ·, X_n} be a random sample consisting of n independent copies of a p dimensional Gaussian random vector X ~ N_p(μ,Σ). The precision matrix, which is the inverse of Σ, is denoted by Ω = (ω_i,j). It is well-known that the precision matrix is closely connected to the corresponding Gaussian graph G = (V,E), which represents the conditional independence between components of X = (X₁, …, X_p)^⊤. Here V is the vertex set consisting of the p components X₁, …, X_p and E is the edge set consisting of ordered pairs (i, j), where (i, j) ∈ E if there is an edge between X_i and X_j, indicating that X_i and X_j are conditionally dependent given {X_k, k ≠ i, j}. It is a well-known fact that the conditional independence between X_i and X_j given all other variables is equivalent to ω_i,j = 0. See, e.g., Lauritzen (1996).

Let 𝒥₁, …, 𝒥_M ⊂ {1, …, p} be a collection of prespecified non-overlapping sets which index group memberships (e.g. pathway membership), we wish to test simultaneously the hypotheses of the conditional independence between any two gene groups given all remaining genes in the collection with proper control of the false discovery rate (FDR) and false discovery proportion (FDP) asymptotically. It follows from the above discussion that this multiple testing problem can be equivalently formulated as testing the hypotheses on the submatrices of the precision matrix Ω,

$$H_{0,m,h}:{\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}=0\text{versus}{H}_{1,m,h}:{\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}\ne 0,1\le m<h\le M,$$ (1)

while controlling the FDR and FDP asymptotically. Hereafter, all results related to the FDR and FDP are studied in the asymptotic regime and we use FDR and FDP as simplifications for the expressions of asymptotic FDR and asymptotic FDP.

Simultaneous testing of between group interactions with FDR control is technically challenging, both in constructing a suitable test statistic and establishing its null distribution for testing the interactions between any two given groups and in developing a multiple testing procedure that accounts for the multiplicity and dependency with FDR control. To the best of our knowledge, there are no currently available methods with theoretical guarantees to infer about interactions between pre-specified gene groups that adjust for effects from a large number of other genes. Furthermore, no existing methods allow the testing for such group level interactions while properly controlling a desired FDR. Liu (2013) proposed a multiple testing procedure with the FDR control for the partial correlations under a Gaussian graphical model. Xia et al. (2015) considered the problem of identifying gene-by-gene interactions associated with a binary trait under a two-sample framework and proposed a procedure for testing the differential network by simultaneously testing entry-wise hypotheses with FDR control. These methods, which can identify the locations of individual gene-by-gene interactions, are however unable to detect the presence interactions between pairs of gene groups while controlling the FDR at the group level.

In this paper, we propose a novel multiple testing procedure for between group interactions that controls the FDR and FDP asymptotically at any pre-specified level 0 < α < 1. The simultaneous testing procedure is developed in two steps. In the first step, we construct a test statistic for testing the conditional independence of a given pair of variable groups 𝒥_m and 𝒥_h, H_0,m,h: Ω_𝒥m×𝒥h = 0, with m ≠ h. The test statistic is based on the Frobenius norm of a standardized submatrix estimate with unknown correlation structure. The estimation of this dependency structure is technically challenging, because correlations among the estimates of the entries of Ω_𝒥m×𝒥h not only depend on the entries within the submatrix, but also largely depend on the entries outside of it. To incorporate this dependency structure, we estimate the eigenvalues of the correlation matrix of the entry estimates of a given submatrix Ω_𝒥m×𝒥h through a Kronecker product by estimating the eigenvalues of two partial correlation submatrices R_𝒥m×𝒥m and R_𝒥h×𝒥h of R = D^−1/2ΩD^−1/2, where D is the diagonal matrix of Ω. It is shown that the test statistic has asymptotically the same limiting null distribution as a mixture of ${\chi }_1^2$ with the estimated correlation structure.

In the second step, we construct a simultaneous testing procedure based on these test statistics. A major difficulty here is that the correlation structures of the entry estimates vary across different submatrices. Consequently the limiting null distributions of the test statistics for different submatrices are different. We introduce a normal quantile transformation for each test statistic, and the transformed test statistics are shown to have asymptotically the same distribution as the absolute value of a standard normal random variable under the null. Based on them, we develop a multiple testing procedure to account for the multiplicity in testing a large number of hypotheses so that the overall FDR and FDP are controlled.

Both the theoretical and numerical properties of the proposed procedure are investigated. The theoretical results show that, under regularity conditions, the proposed procedure asymptotically controls both the overall FDR and FDP at the pre-specified level. As a comparison, it is discussed in Section 4.3 that a direct application of the well-known B-H procedure (Benjamini and Hochberg, 1995) to the individual test statistics is not able to control the FDP when the number of true alternatives is fixed. Simulation studies are carried out to examine the numerical performance of the multiple testing procedure in various settings. The results show that the procedure performs well numerically in terms of both the size and power of the test. We also consider a simulation setting that is similar to the breast cancer gene expression data analyzed in this paper by mimicking the true sizes of the gene groups in the breast cancer study. The result shows that the FDR is well controlled and this new group level based method significantly outperforms the alternative procedures.

Finally, we apply the proposed procedure to assess the between pathway interactions in a breast cancer gene expression study. Many of the identified interactions are consistent with those reported in the literature.

1.3 Structure of the Paper

The rest of the paper is organized as follows. We give a detailed construction of the statistic for testing a specific submatrix of a precision matrix in Section 2. The limiting null distribution of the test statistic and the theoretical properties of the testing procedure are obtained in Section 3. A multiple testing procedure for simultaneously assessing a collection of submatrices is proposed and its theoretical properties are established in Section 4. Simulation results demonstrating the performance of the proposed methods in finite sample are given in Section 5. In Section 6, we apply the new multiple testing procedure to a breast cancer gene expression study to identify between pathway interactions. A discussion on possible extensions is given in Section 7. All proofs are contained in the supplement Xia et al. (2016).

2 Testing A Given Submatrix

We consider in this section testing a given submatrix of the precision matrix Ω,

$$H_0:{\mathbf{\Omega }}_{\mathcal{I}\times \mathcal{J}}=0\text{versus}{H}_1:{\mathbf{\Omega }}_{\mathcal{I}\times \mathcal{J}}\ne 0,$$ (2)

under the framework of Section 1.2, where ℐ and 𝒥 index two non-overlapping gene groups. A rejection of H₀ means that at least one pair of variables from ℐ and 𝒥 are not conditionally independent from each other given all other variables. As the group information is considered as prior knowledge, performing analysis at the group level is more appealing than the entrywise procedure as discussed in Section 1. We shall construct a test statistic for H₀, corresponding to no interactions between gene groups ℐ and 𝒥 conditional on all other genes. Related works on testing for independence and conditional independence between random vectors can be found in, e.g., Gieser and Randles (1997); Um and Randles (2001); Beran et al. (2007); Su and White (2007, 2008); and Huang et al. (2010).

2.1 Notation and Definitions

Denote A ⊗ B the Kronecker product of matrix A and B. For a vector β = (β₁, …, β_p)^⊤ ∈ ℝ_p, define the ℓ_q norm by ${\mid \mathit{\beta }\mid }_q={({\sum }_{i=1}^p{\mid {\beta }_i\mid }^q)}^{1/q}$ for 1 ≤ q ≤ ∞. For any vector μ with dimension p × 1, let μ_−i denote the (p − 1) × 1 vector by removing the i^th entry from μ. For a symmetric matrix A, let λ_max(A) and λ_min(A) denote the largest and smallest eigenvalues of A. For any p × q matrix A, A_i,−j denotes the i^th row of A with its j^th entry removed and A_−i,j denotes the j^th column of A with its i^th entry removed. A_−i,−j denotes the (p−1)×(q −1) submatrix of A with its i^th row and j^th column removed. A_r×c denotes the submatrix of A corresponding to the row vector r and column vector c. For a n×p data matrix U = (U₁, …, U_n)^⊤, denote an n × (p − 1) matrix ${\mathit{U}}_{·,-i}={({\mathit{U}}_{1,-i}^{\top },\dots ,{\mathit{U}}_{n,-i}^{\top })}^{\top }$. Let ${\overline{\mathit{U}}}_{·,-i}=1/n{\sum }_{k=1}^n{\mathit{U}}_{k,-i}$ with dimension 1 × (p − 1), U_(i) = (U_1,i, …, U_n,i)^⊤ with dimension n × 1, Ū_(i) = (Ū_i, …, Ū_i)^⊤ with dimension n × 1, where ${\overline{U}}_i=1/n{\sum }_{k=1}^n{U}_{k,i}$, and ${\overline{\mathit{U}}}_{(·,-i)}={({\overline{\mathit{U}}}_{·,-i}^{\top },\dots ,{\overline{\mathit{U}}}_{·,-i}^{\top })}^{\top }$ with dimension n × (p − 1). For a matrix Ω = (ω_i,j)_p×p, the matrix 1-norm is defined by ${\Vert \mathbf{\Omega }\Vert }_{L_1}={max}_{1\le j\le p}{\sum }_{i=1}^p\mid {\omega }_{i,j}\mid$ and the matrix element wise infinity norm is defined to be ||Ω||_∞ = max_1≤i,j≤p |ω_i,j |. For a set ℋ, denote |ℋ| the cardinality of ℋ. For two sequences of real numbers {a_n} and {b_n}, write a_n = O(b_n) if there exists a constant C such that |a_n| ≤ C|b_n| holds for all n, write a_n = o(b_n) if lim_n→∞ a_n/b_n = 0, and write a_n ≍ b_n if lim_n→∞ a_n/b_n = 1.

2.2 Testing Procedure

We shall first define a standardized estimate W_i,j for each individual entry of the precision matrix, which is the one-sample version of the estimates proposed in Xia et al. (2015), then propose a novel test statistic S_ℐ×𝒥 based on the sum of all possible $W_{i,j}^2$, for (i, j) ∈ ℐ ×𝒥.

It is well known that in the Gaussian setting, the precision matrix can be described in terms of the regression models, see, e.g., Section 2.5 in Anderson (2003). Specifically, we may write

$$X_{k,i}={\alpha }_i+{\mathit{X}}_{k,-i}{\mathit{\beta }}_i+{\epsilon }_{k,i},1\le k\le n,$$ (3)

where ${\epsilon }_{k,i}~N(0,{\sigma }_{i,i}-{\mathbf{\sum }}_{i,-i}{\mathbf{\sum }}_{-i,-i}^{-1}{\mathbf{\sum }}_{-i,i})$ is independent of X_k,−i, and ${\alpha }_i={\mu }_i-{\mathbf{\sum }}_{i,-i}{\mathbf{\sum }}_{-i,-i}^{-1}{\mathit{\mu }}_{-i}$. The regression coefficient vector β_i and the error terms ε_k,i satisfy

$${\mathit{\beta }}_i=-{\omega }_{i,i}^{-1}{\mathbf{\Omega }}_{-i,i}\text{and}{r}_{i,j}\equiv \text{Cov}({\epsilon }_{k,i},{\epsilon }_{k,j})={\omega }_{i,j}/({\omega }_{i,i}{\omega }_{j,j}).$$

As in Xia et al. (2015), we first develop an estimator of ω_i,j and then base the test on its bias corrected standardization. We begin by constructing estimators of r_i,j.

Let β̂_i = (β̂_1,i, · · ·, β̂_p−1,i)^⊤ be estimators of β_i satisfying max

$$\underset{1\le i\le p}{max}{\mid {\widehat{\mathit{\beta }}}_i-{\mathit{\beta }}_i\mid }_1=o_P\{{(logp)}^{-1}\},$$ (4)

$$\underset{1\le i\le p}{max}{\mid {\widehat{\mathit{\beta }}}_i-{\mathit{\beta }}_i\mid }_2=o_P\left\{{(\mathit{n}logp)}^{-1/4}\right\}.$$ (5)

Such estimators can be obtained easily via the standard methods such as the Lasso and Dantzig Selector, see, e.g., Xia et al. (2015) Section 2.3. Specifically, if we use the Lasso estimator (see (18) in Section 5), then equations (4) and (5) can be satisfied under the condition (C1) in Section 3 and the sparsity condition max_1≤i≤p |β_i|₀ = o{n^1/2/(log p)^3/2}.

Define the fitted residuals by

$${\widehat{\epsilon }}_{k,i}=X_{k,i}-{\overline{X}}_i-({\mathit{X}}_{k,-i}-{\overline{\mathit{X}}}_{-i}){\widehat{\mathit{\beta }}}_i,$$

where ${\overline{X}}_i=\frac{1}{n}{\sum }_{k=1}^n{X}_{k,i},{\overline{\mathit{X}}}_{-i}=\frac{1}{n}{\sum }_{k=1}^n{\mathit{X}}_{k,-i}$. A natural estimator of r_i,j is the sample covariance between the residuals

$${\stackrel{\sim }{r}}_{i,j}=\frac{1}{n}\sum _{k=1}^n{\widehat{\epsilon }}_{k,i}{\widehat{\epsilon }}_{k,j}.$$ (6)

However, when i ≠ j, r̃_i,j tends to be biased due to the correlation induced by the estimated parameters. Xia et al. (2015) proposed a bias corrected estimator of r_i,j as

$${\widehat{r}}_{i,j}=-({\stackrel{\sim }{r}}_{i,j}+{\stackrel{\sim }{r}}_{i,i}{\widehat{\beta }}_{i,j}+{\stackrel{\sim }{r}}_{j,j}{\widehat{\beta }}_{j-1,i}),\text{for}1\le i<j\le p.$$

For i = j, we let r̃_i,i = r̃_i,i, which is a nearly unbiased estimator of r_i,i. For 1 ≤ i < j ≤ p, a natural estimator of ω_i,j can then be defined by

$$T_{i,j}={\widehat{r}}_{i,j}/({\widehat{r}}_{i,i}·{\widehat{r}}_{j,j}).$$

Since {T_i,j, 1 ≤ i < j ≤ p} are heteroscedastic and can possibly have a wide range of variability, we shall first standardize T_i,j. To estimate its variance, note that

$${\theta }_{i,j}\equiv \text{Var}({\epsilon }_{k,i}{\epsilon }_{k,j}/(r_{i,i}{r}_{j,j}))/n=(1+{\rho }_{i,j}^2)/(n{r}_{i,i}{r}_{j,j}),$$

where ${\rho }_{i,j}^2={\beta }_{i,j}^2{r}_{i,i}/r_{j,j}$. Then θ_i,j can be estimated by ${\widehat{\theta }}_{i,j}=(1+{\widehat{\beta }}_{i,j}^2{\widehat{r}}_{i,i}/{\widehat{r}}_{j,j})/(n{\widehat{r}}_{i,i}{\widehat{r}}_{j,j})$.

Define the standardized statistics

$$W_{i,j}=T_{i,j}/{({\widehat{\theta }}_{i,j})}^{1/2},\text{for}1\le i<j\le p.$$ (7)

Finally, we propose the following test statistic for testing a given submatrix Ω_ℐ×𝒥,

$$S_{\mathcal{I}\times \mathcal{J}}=\sum _{(i,j)\in \mathcal{I}\times \mathcal{J}}{W}_{i,j}^2.$$ (8)

We detail in Section 3 statistical properties of the proposed test statistic.

3 Theories on Testing A Given Submatrix

In this section, we investigate the theoretical properties including the limiting null distribution and the asymptotic power. We first show that the null distribution of S_ℐ×𝒥 converges to the distribution of a mixture of ${\chi }_1^2$ variables as (n, p) → ∞ and then demonstrate that the test based on S_ℐ×𝒥 is powerful under a large collection of alternatives.

3.1 Asymptotic Null Distribution

Before studying the null distribution of S_ℐ×𝒥, we first introduce the following condition on the eigenvalues of Ω, which is a common assumption in the high-dimensional setting (Cai et al., 2013; Xia et al., 2015; Liu, 2013).

(C1) Assume that log p = o(n^1/5), and for some constant C⁰ > 0, $C_0^{-1}\le {\lambda }_{min}(\mathbf{\Omega })\le {\lambda }_{max}(\mathbf{\Omega })\le C_0$. Suppose |𝒥_m| does not depend on n and p for 1 ≤ m ≤ M.

Let D be the diagonal of Ω and let (η_i,j) =: R = D^−1/2ΩD^−1/2. Under H₀, for (i₁, j₁), (i₂, j₂) ∈ ℐ × 𝒥, the covariance between the standardized statistics W_i1,j1 and W_i2,j2, as defined in (7), is approximately equal to η_i1,i2η_j1,j2, and thus can be estimated by T̃_i1,i2 T̃_j1,j2, where T̃:= (T̃_i,j)_p×p with ${\stackrel{\sim }{T}}_{i,j}={\widehat{r}}_{i,j}/\sqrt{{\widehat{r}}_{i,i}{\widehat{r}}_{j,j}}$. Thus, we shall estimate the covariance matrix of {W_i,j, (i, j) ∈ ℐ×𝒥} by the Kronecker product of T̃_ℐ×ℐ and T̃_𝒥×𝒥. Let ${\widehat{\mathbf{\Lambda }}}_{\mathcal{I}}={({\widehat{\lambda }}_1^{\mathcal{I}},\dots ,{\widehat{\lambda }}_{\mid \mathcal{I}\mid }^{\mathcal{I}})}^{\top }$ and ${\widehat{\mathbf{\Lambda }}}_{\mathcal{J}}={\{{\widehat{\lambda }}_1^{\mathcal{J}},\dots ,{\widehat{\lambda }}_{\mid \mathcal{J}\mid }^{\mathcal{J}}\}}^{\top }$ be the eigenvalues of T̃_ℐ×ℐ and T̃_𝒥×𝒥 respectively. We then estimate the eigenvalues of the covariance matrix of {W_i,j, (i, j) ∈ ℐ ×𝒥} by ${\widehat{\mathbf{\Lambda }}}^{\mathcal{I}\times \mathcal{J}}={({\widehat{\lambda }}_1^{\mathcal{I}\times \mathcal{J}},\dots ,{\widehat{\lambda }}_K^{\mathcal{I}\times \mathcal{J}})}^{\top }$ which is the vectorized Λ̃_ℐ ⊗ Λ̃ _𝒥, where K = |ℐ||𝒥|. The following theorem states the asymptotic null distributions for S_ℐ×𝒥.

Theorem 1

Suppose that (C1), (4) and (5) hold. Then under H₀: Ω_ℐ×𝒥 = 0, for any given t ∈ ℝ, we have

$$\frac{P(S_{\mathcal{I}\times \mathcal{J}}\le t)}{P({\sum }_{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2\le t)}\to 1,$$ (9)

as (n, p)→∞, where (Z₁, …, Z_K) ~ N(0, I_K×K).

Remark 1

The difficulty of Theorem 1 comes from the fact that, though Ω_ℐ×𝒥 = 0 under the null, the entries {ε_k,iε_k,j, (i, j) ∈ ℐ×𝒥} can still be highly dependent with each other and their correlations depend on the entries outside of submatrix Ω_ℐ×𝒥. Thus, the distribution of S_ℐ×𝒥 cannot be simply estimated by the chi-square distribution. Actually, if we use the chi-square approximation in the following FDR control procedure in Section 4, the choice of threshold level of each statistic will be too conservative and as the result the FDR cannot be controlled at the pre-specified level α, i.e., the FDR will be much larger than α.

It has been shown in the above theorem that S_ℐ×𝒥 has different asymptotic distribution for different submatrix Ω_ℐ×𝒥. Thus, we introduce the normal quantile transformation of S_ℐ×𝒥 as follows

$$N_{\mathcal{I}\times \mathcal{J}}={\mathrm{\Phi }}^{-1}\left\{1-\mathrm{P}(\sum _{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2\ge S_{\mathcal{I}\times \mathcal{J}})/2\right\},$$

where Φ(t) = P(N(0, 1) ≤ t) is standard normal cumulative distribution function (cdf) and S_ℐ×𝒥 is the observed value. Thus, we have $\mathrm{P}(\mid N(0,1)\mid \ge N_{\mathcal{I}\times \mathcal{J}})=\mathrm{P}({\sum }_{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2\ge S_{\mathcal{I}\times \mathcal{J}})$. Since asymptotically S_ℐ×𝒥 and ${\sum }_{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2$ have the same distribution as studied in Theorem 1, thus N_ℐ×𝒥 asymptotically has the same distribution as the absolute value of a standard normal random variable. We then define the test ${\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}$ by

$${\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}=I\left\{N_{\mathcal{I}\times \mathcal{J}}\ge {\mathrm{\Phi }}^{-1}(1-\alpha )\right\}.$$ (10)

The hypothesis H₀: Ω_ℐ×𝒥 = 0 is rejected whenever ${\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}=1$.

Remark 2

The eigenvalues { ${\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}$, l = 1, …, K} are calculated based on T̃_ℐ×ℐ and T̃_𝒥×𝒥 as described earlier. Given the values of { ${\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}$, l = 1, …, K}, the distribution of the mixture of ${\chi }_1^2$ variables ${\sum }_{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2$ can be approximated by a non-central chi-squared distribution with the parameters determined by the first four cumulants of the quadratic form, see, e.g., Liu et al. (2009). We will use this approximation in our numerical studies.

3.2 Asymptotic Power

We now turn to analyze the power of the test ${\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}$ given in (10). For a given pair of index sets ℐ and 𝒥, we shall first define the following class of precision matrices

$${\mathcal{W}}_{\mathcal{I}\times \mathcal{J}}(\alpha ,\beta )=\left\{\mathbf{\Omega }:\sum _{(i,j)\in \mathcal{I}\times \mathcal{J}}\frac{{\omega }_{i,j}^2}{{\theta }_{i,j}}\ge (2+\delta )({\mathrm{\Psi }}_{1-\alpha }^2+{\mathrm{\Psi }}_{1-\beta }^2)\right\},$$ (11)

for any δ > 0, where Ψ_1−α is the 1 − α quantile of ${\sum }_{l=1}^K{\widehat{\lambda }}_l^{\mathcal{I}\times \mathcal{J}}{Z}_l^2$ as defined in Theorem 1.

The next theorem shows that the test ${\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}$ is able to asymptotically distinguish the null parameter set in which Ω_ℐ×𝒥 = 0 from 𝒲_ℐ×𝒥 (α, β) for arbitrarily small constant δ > 0, with β → 0.

Theorem 2

Suppose that (C1), (4) and (5) hold. Then we have, for any constant δ > 0,

$$\underset{\mathbf{\Omega }\in {\mathcal{W}}_{\mathcal{I}\times \mathcal{J}}(\alpha ,\beta )}{inf}P\left({\mathrm{\Phi }}_{\alpha }^{\mathcal{I}\times \mathcal{J}}=1\right)\ge 1-\beta ,asn,p\to \infty .$$ (12)

Since θ_i,j is of order 1/n, Theorem 2 shows that the proposed test rejects the null hypothesis H₀: Ω_ℐ×𝒥 = 0 with high probability for a large class of precision matrices satisfying the condition that there exists one entry of the submatrix Ω_ℐ×𝒥 having a magnitude larger than C/n^1/2 for $C={\{2(2+\delta )C_0^2({\mathrm{\Psi }}_{1-\alpha }^2+{\mathrm{\Psi }}_{1-\beta }^2)\}}^{1/2}$, where C₀ is given in Condition (C1).

4 Multiple Testing of Submatrices with FDR Control

In practice, there are typically many pathways under investigation and it is often of significant interest to identify which pairs of the pathways interact with each other. A natural approach to investigate interactions among the M pathways, indexed by {𝒥_m,m = 1, …, M}, is to carry out simultaneous testing of

$$H_{0,m,h}:{\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}=0\text{versus}{H}_{1,m,h}:{\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}\ne 0,\text{for}1\le m<h\le M,$$ (13)

where 𝒥₁, …, 𝒥_M ⊂ {1, …, p} is a collection of pre-specified non-overlapping index sets. In this section, we introduce a multiple testing procedure with FDR and FDP control for testing a collection of ℳ= M(M − 1)/2 hypotheses, and we shall assume that ℳ is large. Let L_m denote the cardinality of 𝒥_m assumed to be independent of n or p for 1 ≤ m ≤ M. Let ℋ = {(m, h): 1 ≤ m < h ≤ M}, ℋ₀ = {(m, h): Ω_𝒥m×𝒥h = 0, 1 ≤ m < h ≤ M} be the set of true nulls and ℋ₁ = ℋ\ℋ₀ be the set of true alternatives. We shall assume that |ℋ₁| is relatively small compared to |ℋ|, and this assumption arises frequently in many contemporary applications.

4.1 Multiple Testing Procedure

Recall that the standardization of T_i,j is defined by W_i,j = T_i,j/(θ̂_i,j)^1/2 as in (7), and the test statistic S_𝒥m×𝒥h is defined based on W_i,j as in (8). It has been shown in Theorem 1 that S_𝒥m×𝒥h has different asymptotic null distribution for different submatrix Ω_𝒥m×𝒥h. Thus, as discussed in Section 3.1, the normal quantile transformation of S_𝒥m×𝒥h is defined by

$$N_{{\mathcal{J}}_m\times {\mathcal{J}}_h}={\mathrm{\Phi }}^{-1}\left\{1-\mathrm{P}(\sum _{l=1}^{L_m{L}_h}{\widehat{\lambda }}_l^{{\mathcal{J}}_m\times {\mathcal{J}}_h}{Z}_l^2\ge S_{{\mathcal{J}}_m\times {\mathcal{J}}_h})/2\right\},$$

and N_𝒥m×𝒥h approximately has the same distribution as the absolute value of a standard normal random variable under the null H_0,m,h. Let t be the threshold level such that H_0,m,h is rejected if N_𝒥m×𝒥h ≥ t. For any given t, denote the total number of false positives by

$$R_0(t)=\sum _{(m,h)\in {\mathcal{H}}_0}I\{N_{{\mathcal{J}}_m\times {\mathcal{J}}_h}\ge t\},$$ (14)

and the total number of rejections by

$$R(t)=\sum _{(m,h)\in \mathcal{H}}I\{N_{{\mathcal{J}}_m\times {\mathcal{J}}_h}\ge t\}.$$ (15)

The false discovery proportion (FDP) and false discovery rate (FDR) are defined as

$$\text{FDP}(t)=\frac{R_0(t)}{R(t)\vee 1}\text{and}\text{FDR}(t)=\mathrm{E}[\text{FDP}(t)].$$

An ideal choice of t is

$$t_0=inf\left\{0\le t\le \sqrt{2log\mathcal{M}}:\frac{R_0(t)}{R(t)\vee 1}\le \alpha \right\},$$

which would reject as many true positives as possible while controlling the FDR at the prespecified level α. However, the total number of false positives, R₀(t), is unknown as the set ℋ₀ is unknown. We propose to estimate R₀(t) by 2(1 − Φ(t))|ℋ₀| and simply estimate |ℋ₀| by ℳ because the number of true alternatives is relatively small. This leads to the following multiple testing procedure with FDR control.

1. Calculate test statistics N_𝒥m×𝒥h.

2. For given 0 ≤ α ≤ 1, calculate

   $$\widehat{t}=inf\left\{0\le t\le \sqrt{2log\mathcal{M}-2loglog\mathcal{M}}:\frac{2\mathcal{M}(1-\mathrm{\Phi }(t))}{R(t)\vee 1}\le \alpha \right\}.$$ (16)

   If (16) does not exist, then set $\widehat{t}=\sqrt{2log\mathcal{M}}$.

3. For (m, h) ∈ ℋ, reject H_0,m,h if N_𝒥m×𝒥h ≥ t̂.

4.2 Theoretical Properties

We now investigate the theoretical properties of the multiple testing procedure given above. For any 1 ≤ m ≤ M, define

$${\mathrm{\Xi }}_m(\gamma )=\left\{h:1\le h\le M,h\ne m,\exists i\in {\mathcal{J}}_m,j\in {\mathcal{J}}_h\text{s.t.}\mid {\omega }_{i,j}\mid \ge {(log\mathcal{M})}^{-2-\gamma }\right\}.$$

The following theorem shows that, under regularity conditions, the above multiple testing procedure controls the FDR and FDP at the pre-specified level α asymptotically.

Theorem 3

Assume that ℳ₀ =: |ℋ₀| ≍ ℳ, and (4) and (5) hold. Suppose there exists some γ > 0 such that max_1≤m≤M |Ξ_m(γ)| = o(M^τ ) for any τ > 0. Then under (C1) with p ≤ cn^r for some c > 0 and r > 0, we have

$${\overline{lim}}_{(n,\mathcal{M})\to \infty }\text{FDR}(\widehat{t})\le \alpha ,$$

and for any ε > 0,

$$\underset{(n,\mathcal{M})\to \infty }{lim}P(\text{FDP}(\widehat{t})\le \alpha +\epsilon )=1.$$

Remark 3

The technical condition on |Ξ_m(γ)| is to ensure that most of the submatrices are not highly correlated with each other. In the special case when max_1≤m≤M |Ξ_m(γ)| = 0, then all subgroups are weakly correlated with each other, i.e., |ω_i,j| ≤ (log ℳ)^−2−γ for all i ∈ 𝒥_m, j ∈ 𝒥_h with m ≠ h. Under this setting, it is shown in the supplement Xia et al. (2016) that the proposed multiple testing procedure performs asymptotically the same as the case when all submatrices are independent with each other. We do not need this strong condition, and the weaker condition max_1≤m≤M |Ξ_m(γ)| = o(M^τ ) for any τ > 0 assumed in the theorem allows the number of highly correlated submatrices growing with M.

When t̂ is not attained in the range [0, $\sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$] as described in equation (16), we shall threshold it at $\sqrt{2log\mathcal{M}}$. We state in the following corollary a condition to ensure the existence of t̂ in the range, and as a result, the FDR and FDP will converge to the pre-specified level α.

Corollary 1

Let

$${\mathcal{S}}_{\rho }=\left\{(m-h)\in \mathcal{H}:\exists (i,j)\in {\mathcal{J}}_m\times {\mathcal{J}}_h\text{such}\text{that}\mid {\omega }_{i,j}\mid /{({\theta }_{i,j})}^{1/2}\ge {(log\mathcal{M})}^{\frac{1}{2}+\rho }\right\}.$$

Suppose for some ρ > 0 and some δ > 0, $\mid {\mathcal{S}}_{\rho }\mid \ge (\frac{1}{\sqrt{\pi }\alpha }+\delta )\sqrt{log\mathcal{M}}$. Assume that ℳ₀ =: |ℋ₀| ≍ ℳ, and (4) and (5) hold. Suppose there exists some γ > 0 such that max_1≤m≤M |Ξ_m(γ)| = o(M^τ ) for any τ > 0. Then, under (C1) with p ≤ cn^r for some c > 0 and r > 0, we have

$$\underset{(n,\mathcal{M})\to \infty }{lim}\text{FDR}(\widehat{t})=\alpha ,\mathit{and}\text{FDP}(\widehat{t})/\alpha \to 1$$

in probability, as (n,ℳ)→∞.

Remark 4

The condition $\mid {\mathcal{S}}_{\rho }\mid \ge (\frac{1}{\sqrt{\pi }\alpha }+\delta )\sqrt{log\mathcal{M}}$ in Corollary 1 is mild, since there are ℳ hypotheses in total and this condition only requires a few submatrices having one entry with magnitude exceeding (log ℳ)^1/2+ρ/n^1/2 for some constant ρ > 0.

4.3 Differences with the B-H Procedure

In this section we first discuss the difference between our method and the Benjamini-Hochberg (B-H) procedure and then explain why in the multiple testing procedure it is critical to restrict t on the range $0\le t\le \sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$ in equation (16) and to threshold N_𝒥m×𝒥h at $\sqrt{2log\mathcal{M}}$ when t̂ is not attained in the range.

Once the test statistic N_𝒥m×𝒥h for a given submatrix is developed, a natural approach to construct a procedure for simultaneously testing a collection of submatrices is to apply the well-known B-H procedure to the p-values p_m,h = 2(1 − Φ(N_𝒥m×𝒥h)), 1 ≤ m < h ≤ M, computed from the transformed statistics N_𝒥m×𝒥h. Applying the B-H procedure to these p values is equivalent to rejecting the null hypotheses H_0,m,h whenever N_𝒥m×𝒥h ≥ t̂_BH, where

$${\widehat{t}}_{BH}=inf\left\{t\ge 0:\frac{2\mathcal{M}(1-\mathrm{\Phi }(t))}{R(t)\vee 1}\le \alpha \right\}.$$ (17)

Note that, the difference between our procedure and the B-H procedure is on the ranges of t in equations (16) and (17).

We first emphasize here that the restriction on the range $0\le t\le \sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$ in our proposed procedure as defined in (16) is critical. When $t\ge \sqrt{2log\mathcal{M}-loglog\mathcal{M}}$, 2 ℳ (1−Φ(t)) → 0 is not even a consistent estimate of R₀(t) because |R₀(t)/{2 ℳ (1−Φ(t))}− 1| ↛ 0 in probability as (n, ℳ)→∞. However, direct application of the B-H procedure to the p-values amounts to using 2 ℳ (1 − Φ(t)) as an estimate of R₀(t) for all t ≥ 0, and as a result it may not able to control the FDP with positive probability. For example, when the number of true alternatives is fixed, it is shown in Proposition 2.1 in Liu and Shao (2014) that the B-H procedure cannot control the FDP with positive probability. Thus, in order to control FDP, it is crucial to restrict t on the range $0\le t\le \sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$.

When t is not attained in the range, it is also critical to threshold N_𝒥m×𝒥h at $\sqrt{2log\mathcal{M}}$ instead of $\sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$. When t does not exist in the range, thresholding N_𝒥m×𝒥h at $\sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$ will cause too many false rejections, and consequently the FDR cannot be controlled asymptotically at level α. If the threshold level is increased to $\sqrt{2log\mathcal{M}}$, the probability of false rejections can then be perfectly controlled asymptotically as shown in equation (3) of the supplement Xia et al. (2016).

To summarize, in order to control FDR and FDP, it is crucial to restrict t on the range $0\le t\le \sqrt{2log\mathcal{M}-2loglog\mathcal{M}}$ in equation (16), and when it is not attained in the range, to threshold N_𝒥m×𝒥h at $\sqrt{2log\mathcal{M}}$.

5 Simulation Studies

We now turn to analyze the numerical performance of the proposed multiple testing procedure through simulation studies. We first investigate the size and power of the proposed method by considering three matrix models with a random selection of the size of submatrices. We then mimic the sizes of the pathways of the breast cancer dataset analyzed in Section 6 and study the numerical performance of the proposed multiple testing procedure in a setting that is similar to the real data application. Our method, which tests for the conditional dependence structure at a group level, is then compared with the entrywise testing method and the B-H procedure. We also compare the new method with the Bonferroni correction procedure and report the results in the supplement.

5.1 Simulation for Different Constructions of Submatrices

Our analysis is divided into two parts: the performance of the new test statistics for testing a given submatrix and the performance of the proposed multiple testing procedure. We first describe the construction of the submatrices. For a given precision matrix Ω, we randomly divide the upper triangular matrix of Ω into ℳ submatrices, where ℳ = ⌊p/s⌋ (⌊p/s⌋−1)/2 and s = 2 and 4. Thus the length of the index sets can range from 1 to (p − ⌊p/s⌋ + 1). This is equivalent to grouping the genes into ⌊p/s⌋ pathways and considering all possible conditional dependence structure between different pathways of different sizes.

The data {X₁, …, X_n} are generated from multivariate normal distribution with zero-mean and precision matrix Ω. Three choices of Ω are considered:

• Model 1: ${\mathbf{\Omega }}^{\ast (1)}=({\omega }_{i,j}^{\ast (1)})$ where ${\omega }_{i,i}^{\ast (1)}=1,{\omega }_{i,i+1}^{\ast (1)}={\omega }_{i+1,i}^{\ast (1)}=0.5,{\omega }_{i,i+2}^{\ast (1)}={\omega }_{i+2,i}^{\ast (1)}=0.5$. For each of the submatrices as we constructed above, if it contains one of those entries, we make the first row of the submatrices equal to 0.5. Let ${\omega }_{i,j}^{\ast (1)}=0$ otherwise. Ω⁽¹⁾ = D^1/2(Ω^*(1) + δI)/(1 + δ)D^1/2 with δ = |λ_min(Ω^*(1))| + 0.05.

• Model 2: ${\mathbf{\Omega }}^{\ast (2)}=({\omega }_{i,j}^{\ast (2)})$ where ${\omega }_{i,j}^{\ast (2)}={\omega }_{j,i}^{\ast (2)}=0.3$ for i = 10(k − 1) + 1 and 10(k − 1) + 2 ≤ j ≤ 10(k − 1) + 10, 1 ≤ k ≤ p/10. ${\omega }_{i,j}^{\ast (2)}=0$ otherwise. For each of the submatrices as we constructed above, if it contains less than three of those entries, we make the submatrices equal to 0. Let the first row of the submatrices which are closest to the diagonal equal to 0.3. Ω⁽²⁾ = D^1/2(Ω^*(2) + δI)/(1 + δ)D^1/2 with δ = |λ_min(Ω^*(2))| + 0.05.

• Model 3: ${\mathbf{\Omega }}^{\ast (3)}=({\omega }_{i,j}^{(3)})$. For each of the two submatrices closest to the diagonal, as we constructed above, pick a random row and make the entries equal to 0.3. Let ${\omega }_{j,i}^{\ast (3)}={\omega }_{i,j}^{\ast (3)}$. Ω⁽³⁾ = D^1/2(Ω^*(3) + δI)/(1 + δ)D^1/2 with δ = |λ_min(Ω^*(3))| + 0.05.

where D = (D_i,j) is a diagonal matrix with D_i,i = Unif(1, 3) for i = 1, …, p.

For each generated dataset, we use the Lasso to estimate the regression coefficients β_i:

$${\widehat{\mathit{\beta }}}_i={\mathit{D}}_i^{-\frac{1}{2}}\underset{\mathit{u}}{argmin}\left\{\frac{1}{2n}{\left|({\mathit{X}}_{-i}-{\overline{\mathit{X}}}_{-i}){\mathit{D}}_i^{-1/2}\mathit{u}-({\mathit{X}}_{(i)}-{\overline{\mathit{X}}}_{(i)})\right|}_2^2+{\lambda }_{n,i}{\mid \mathit{u}\mid }_1\right\},$$ (18)

where D_i = diag(Σ̂_−i,−i), and ${\lambda }_{n,i}=\kappa \sqrt{{\widehat{\sigma }}_{i,i}logp/n}$.

Performance for testing a given submatrix

We start by comparing our test based on the test statistic S_ℐ×𝒥 with the entrywise testing of a given submatrix where the null hypothesis H₀: Ω_ℐ×𝒥 = 0 is rejected whenever max_{(i,j)∈ℐ×𝒥} |W_i,j| ≥ Φ⁻¹(1 − α/K). As our target is the FDR control of the multiple comparisons, we focus on the power comparisons of these two methods for a range of significance levels from 0 to α = 0.1/ℳ. For illustration, we compare the performance of these two tests by testing against a randomly selected nonzero submatrix closest to the diagonal for Model 1 with s = 4. For each method, the sample size is taken to be n = 200, while the dimension p varies over the values 100, 200, 500 and 1000. For simplicity of the comparison, the tuning parameters λ_n,i in (18) is selected to be ${\lambda }_{n,i}=\sqrt{{\widehat{\sigma }}_{i,i}logp/n}$ for both methods. The power curves, illustrated in Figure 1, are estimated from 100 replications. We can see from the figure that the power of the new group method is much higher than the entrywise method, and the advantage becomes much clearer when the dimension of Ω grows.

Figure 1.

Power comparisons of the group method (red, solid) and entrywise method (blue, dash) for testing a given nonzero submatrix. 100 replications.

Comparison of the multiple testing procedures

We now compare the proposed group level FDR control procedure (Group) with three other methods: entrywise multiple testing method (Entrywise), B-H procedure (B-H) and Bonferroni correction procedure (Bonferroni).

For the new method, as described in Section 4, we select the tuning parameters λ_n,i in (18) adaptively by the data with the principle of making Σ_(m,h)∈ℋ0 /(N_𝒥m×𝒥h ≥ t) and (2 − 2Φ(t))|ℋ₀| as close as possible. The algorithm is similar as Xia et al. (2015) and is summarized as follows.

1. Let ${\lambda }_{n,i}=b/20\sqrt{{\widehat{\mathbf{\sum }}}_{i,i}logp/n}$ for b = 1, · · ·, 40. For each b, calculate ${\widehat{\mathit{\beta }}}_i^{(b)}$, i = 1, · · ·, p. Based on the estimation of regression coefficients, construct the corresponding standardized transformed statistics $N_{{\mathcal{J}}_m\times {\mathcal{J}}_h}^{(b)}$ for each b.

2. Choose b̂ as the minimizer of

   $$\sum _{d=1}^{10}{\left(\frac{{\sum }_{(m,h)\in \mathcal{H}}I(N_{{\mathcal{J}}_m\times {\mathcal{J}}_h}^{(b)}\ge {\mathrm{\Phi }}^{-1}(1-d(1-\mathrm{\Phi }(\sqrt{log\mathcal{M}}))/10))}{d(1-\mathrm{\Phi }(\sqrt{log\mathcal{M}}))/10·2\mathcal{M}}-1\right)}^2.$$

The tuning parameters λ_n,i are then chosen to be

$${\lambda }_{n,i}=\widehat{b}/20\sqrt{{\widehat{\mathbf{\sum }}}_{i,i}logp/n}.$$

We examine the power of the new method based on the average powers for 100 replications,

$$\frac{1}{100}\sum _{r=1}^{100}\frac{{\sum }_{(m,h)\in {\mathcal{H}}_1}I\{N_{{\mathcal{J}}_m\times {\mathcal{J}}_{h,r}}\ge \widehat{t}\}}{\mid {\mathcal{H}}_1\mid },$$ (19)

where r denotes the r-th replication.

For the entrywise multiple testing method, we select the tuning parameters λ_n,i adaptively using the principle as described in Section 5 in Xia et al. (2015). We applied the multiple testing procedure as developed in Section 4 of Xia et al. (2015) by restricting t on the range [0, $\sqrt{4logp-2loglog\mathit{p}}$] and threshold |W_i,j | at $\sqrt{4logp}$ if t̂ is not attained in the range. We then examine the empirical FDR by

$$\frac{1}{100}\sum _{r=1}^{100}\frac{{\sum }_{(m,h)\in {\mathcal{H}}_0}I\{{max}_{(i,j)\in {\mathcal{J}}_m\times {\mathcal{J}}_h}\mid W_{i,j}\mid \ge \widehat{t}\}}{{\sum }_{(m,h)}I\{{max}_{(i,j)\in {\mathcal{J}}_m\times {\mathcal{J}}_h}\mid W_{i,j}\mid \ge \widehat{t}\}},$$

and the empirical power by

$$\frac{1}{100}\sum _{r=1}^{100}\frac{{\sum }_{(m,h)\in {\mathcal{H}}_1}I\{{max}_{(i,j)\in {\mathcal{J}}_m\times {\mathcal{J}}_h}\mid W_{i,j}\mid \ge \widehat{t}\}}{\mid {\mathcal{H}}_1\mid }.$$

We apply the Bonferroni correction procedure to the new test statistics and calculate its power based on (19), with t̂ obtained by setting α_B = α/ℳ. The power of the B-H procedure applied to max_{(i,j)∈ℐ×𝒥} |W_i,j | are calculated by (19) with no restriction on the range of t̂.

We apply all procedures to these three models with s = 2 and 4. For each method, the sample size is taken to be n = 200, while the dimension p varies over the values 100, 200, 500 and 1000. The FDR level is set at α = 0.1 and α = 0.01 respectively, and the empirical FDRs and powers, summarized in Tables 1 and 2, are estimated from 100 replications. The standard errors of the estimated powers are much smaller than the powers themselves and are thus not reported.

Table 1.

Empirical FDRs (standard errors) (%) with n = 200, α = 0.1 and 0.01 respectively, 100 replications.

p	method	α = 10%						α = 1%
	s	2			4			2			4
	Models	1	2	3	1	2	3	1	2	3	1	2	3
		Empirical FDR (SE) (in %)
100	Group	8.9 (3.4)	9.5 (4.3)	9.0 (2.9)	4.6 (3.3)	8.6 (6.0)	9.7 (5.6)	0.8 (1.2)	1.0 (1.0)	0.6 (0.8)	0.2 (0.5)	0.8 (1.0)	0.4 (1.1)
	Entrywise	24.5 (4.6)	15.7 (5.0)	12.7 (3.7)	36.2 (7.6)	24.1 (8.3)	16.3 (6.1)	2.7 (2.3)	1.3 (1.8)	1.0 (1.2)	4.0 (4.1)	1.8 (3.3)	1.3 (5.5)
	B-H	10.0 (3.6)	12.7 (5.5)	9.8 (4.2)	8.3 (5.0)	12.4 (7.4)	11.7 (6.8)	0.8 (1.3)	1.2 (1.9)	0.8 (1.1)	0.4 (1.5)	1.5 (5.6)	1.1 (3.5)
200	Group	8.8 (2.5)	9.4 (3.5)	8.7 (2.5)	5.8 (3.3)	8.5 (4.5)	8.7 (3.9)	0.6 (0.5)	0.9 (0.5)	0.8 (0.8)	0.3 (0.4)	0.8 (0.2)	0.8 (0.7)
	Entrywise	23.5 (3.5)	14.9 (3.7)	13.6 (3.0)	33.1 (7.0)	23.0 (5.5)	15.1 (5.8)	2.1 (1.4)	1.0 (1.2)	0.9 (1.0)	3.1 (2.8)	1.3 (2.3)	0.7 (3.1)
	B-H	9.8 (2.9)	11.7 (3.8)	9.2 (3.1)	8.8 (4.2)	11.9 (4.6)	12.1 (5.5)	0.8 (0.9)	0.9 (1.9)	0.8 (1.1)	0.8 (1.5)	1.4 (2.5)	1.0 (4.2)
500	Group	7.9 (1.5)	9.9 (2.3)	8.1 (1.5)	5.4 (1.8)	8.7 (2.9)	9.3 (2.7)	0.8 (0.4)	0.9 (0.7)	0.7 (0.5)	0.5 (0.6)	0.7 (0.8)	0.9 (0.8)
	Entrywise	19.4 (2.2)	15.0 (2.3)	12.5 (2.1)	24.6 (4.9)	20.6 (4.1)	16.2 (5.3)	1.6 (1.0)	0.9 (0.9)	0.8 (0.8)	1.6 (1.4)	1.1 (1.6)	1.1 (2.1)
	B-H	8.3 (2.0)	11.3 (2.6)	9.2 (2.1)	8.7 (3.4)	11.9 (4.2)	11.6 (4.9)	0.8 (0.6)	1.1 (1.1)	0.9 (1.1)	0.7 (1.0)	1.2 (2.3)	1.4 (3.5)
1000	Group	7.9 (1.2)	9.8 (1.8)	8.7 (1.4)	6.0 (1.7)	9.0 (2.0)	10.0 (2.1)	0.7 (0.3)	0.9 (0.4)	0.7 (0.4)	0.4 (0.4)	0.8 (0.5)	1.0 (0.7)
	Entrywise	16.7 (2.0)	14.5 (1.8)	13.1 (2.1)	20.9 (3.5)	20.0 (2.9)	16.5 (5.4)	1.3 (0.7)	1.3 (0.9)	1.1 (1.0)	1.7 (1.7)	1.2 (1.6)	0.8 (2.0)
	B-H	7.8 (1.3)	11.8 (2.3)	10.1 (1.9)	9.5 (2.7)	12.0 (2.8)	12.7 (6.2)	0.6 (0.5)	1.1 (1.0)	0.8 (1.1)	0.9 (1.2)	1.3 (2.3)	0.8 (2.1)

Table 2.

Empirical powers (%) with n = 200, α = 0.1 and 0.01 respectively, 100 replications.

p	method	α = 10%						α = 1%
	s	2			4			2			4
	Models	1	2	3	1	2	3	1	2	3	1	2	3
100	Group	93.8	88.5	84.6	95.2	84.4	73.5	87.8	74.2	69.4	92.3	68.7	54.4
	Entrywise	92.8	90.3	85.2	92.8	85.1	67.3	83.4	71.9	66.8	85.9	56.3	28.5
	B-H	92.8	88.3	84.6	92.9	81.1	67.2	82.3	66.3	64.5	84.0	50.9	31.2
200	Group	90.4	83.3	72.5	92.6	77.9	58.8	82.6	66.8	56.1	87.9	62.7	42.6
	Entrywise	87.6	84.8	71.7	87.3	76.0	45.5	93.6	60.8	47.6	73.6	40.8	15.0
	B-H	87.6	81.4	70.9	86.9	71.3	43.3	72.2	54.6	46.7	71.1	33.8	14.1
500	Group	84.3	72.3	56.6	85.9	67.4	41.4	75.1	55.4	40.7	76.0	53.4	27.2
	Entrywise	78.7	70.8	50.8	72.5	60.0	24.2	60.0	43.3	25.6	48.3	25.9	5.0
	B-H	77.8	66.0	50.4	70.0	52.5	20.3	57.5	36.3	23.8	43.3	19.3	3.0
1000	Group	80.1	63.3	46.4	80.7	59.9	31.9	69.9	57.2	31.8	69.2	46.8	21.1
	Entrywise	71.2	59.2	37.1	60.7	48.5	14.5	49.3	31.5	15.5	33.5	18.5	3.1
	B-H	69.8	53.1	35.4	56.6	39.9	9.6	45.8	23.9	12.7	27.3	10.6	1.2

The average numbers of conditionally dependent (“true interaction”) and conditionally independent (“no interaction”) pairs of subgroups with 100 replications are summarized in Table 3. It can be seen that the number of “true interactions” is relatively small compared to the total number of pairs of subgroups in all cases, as we assumed in Section 4.1.

Table 3.

Average numbers of true interactions and no interactions based on 100 replications.

s	2			4			2			4
Models	1	2	3	1	2	3	1	2	3	1	2	3
p	True interactions						No interactions
100	72	54	96	29	28	46	1153	1171	1129	271	272	254
200	146	110	196	60	58	96	4803	4839	4754	1165	1167	1129
500	373	279	496	154	147	246	30752	30846	30629	7596	7603	7504
1000	748	560	996	311	297	496	124002	124190	123754	30814	30828	30629

The results in Table 1 show that the empirical FDRs of the new group level method are well maintained under the target FDR level and are reasonably close to α for almost all settings. The standard errors of the FDP are small in most cases, especially when the dimension grows. They are slightly larger in the cases when α = 0.01, mainly due to the fact that the estimation error of the standard deviation of FDP is of the order 1/l^1/2 with l = 100. As a comparison, the empirical FDRs of the entrywise method have serious distortion in most of the scenarios, especially when s = 4, in which case the empirical FDRs can be even larger than 4α. The empirical FDRs of the B-H procedure are well under control in most cases. However, its standard errors are much larger than the standard errors of the proposed method in many cases, which coincides with the discussion in Section 4.3. The numerical results also show that the Bonferroni correction procedure is much more conservative than the other two methods, and the detailed analysis is summarized in the supplement Xia et al. (2016).

Table 2 shows that the empirical powers of our proposed method for all these models are very high under various constructions of submatrices. In particular, it outperforms the entrywise testing method and the B-H procedure. Especially when the dimension is high, the powers of the new method are much higher than the other methods under all scenarios. Furthermore, the power gain of the new group level testing procedure over the entrywise testing method is significant when the dimension is high. Especially for model 3 when s = 4, the empirical powers of the new procedure are more than twice the entrywise testing method. This is because the advantage of the group level testing becomes more significant when the signals spread across various submatrices as in Model 3. We can see from the table that the empirical power of the new method gets smaller when the dimension p grows. This is because of the fact that we keep the magnitude of ω_i,j invariant for various range of dimensions.

5.2 Simulation by Mimicking the Sizes of Gene Groups

We now consider a simulation setting that is similar to the breast cancer data application given in Section 6. The submatrices of the precision matrix Ω is constructed by mimicking the sizes of the 249 gene groups used in the breast cancer application, with parameter values p = 1624, n = 295 and ℳ = 30876. The sizes of the gene groups range from 1 to 110, and the corresponding sizes of the off-diagonal submatrices range from 1×1 to 97×110. For the diagonal submatrices ${\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_m}^{\ast }:=({\omega }_{m,i,j}^{\ast })$ with sizes L_m×L_m, m = 1, …, 249, which describe the conditional dependency within the pathways, we let ${\omega }_{m,i,i}^{\ast }=1,{\omega }_{m,i,i+1}^{\ast }={\omega }_{m,i+1,i}^{\ast }=0.8$ if L_m ≥ 2, ${\omega }_{m,i,i+2}^{\ast }={\omega }_{m,i+2,i}^{\ast }=0.6$ if L_m ≥ 3, and ${\omega }_{m,j,i}^{\ast }={\omega }_{m,i,j}^{\ast }$. For each of the non-diagonal submatrices ${\mathbf{\Omega }}_{{\mathcal{J}}_M\times {\mathcal{J}}_{m+1}}^{\ast }$ and ${\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_{m+2}}^{\ast }$, we randomly pick one row and let min{10, |𝒥_m+1|} and min{10, |𝒥_m+2|} random entries of ${\omega }_{i,j}^{\ast }$ in the rows equal to 0.5 respectively. We then construct the precision matrix as Ω = D^1/2(Ω^*+δI)/(1+δ)D^1/2, with δ = λ_min(Ω^*)+0.05. The FDR level is set at α = 0.1 and α = 0.01 respectively.

By mimicking the gene group sizes, we apply the proposed method in Section 4.1, the entrywise testing procedure, the B-H procedure and the Bonferroni correction procedure as described in Section 5.1. The empirical FDR and power results are summarized in Table 4, and the performance of the Bonferroni method is reported in the supplement. The empirical FDR of the new method is equal to 0.062 when α = 0.1 and is equal to 0.006 when α = 0.01, and thus both are close to the corresponding pre-specified level. Similarly as in Section 5.1, the B-H procedure has larger standard errors than the new procedure, while the entrywise multiple testing procedure has serious FDR distortion. For the empirical powers, it is shown in Table 4 that, the new testing procedure is more powerful than all the other methods.

Table 4.

Empirical FDRs (SEs) and powers (%) by mimicking the real data with α = 0.1 and α = 0.01 respectively, based on 100 replications.

p	method	α = 10%		α = 1%
		FDR (SE) (in %)	Power (in %)	FDR (SE) (in %)	Power (in %)
1624	Group	6.2 (1.4)	47.2	0.5 (0.3)	33.6
	Entrywise	26.0 (2.7)	39.9	2.5 (1.5)	26.0
	B-H	11.1 (2.0)	44.9	1.1 (1.0)	19.8

6 Analysis of Breast Cancer Gene Expression Data

In this section, we apply the multiple testing procedure developed in Section 4 to identify between pathway interactions based on a breast cancer gene expression study as described in van’t Veer et al. (2002), to further illustrate the merit of the procedure.

This study consists of 295 subjects with primary breast carcinomas whose gene-expression levels (in log scale) are measured at cancer diagnosis. For illustration, we consider M = 70 breast cancer related pathways, including several major signaling pathways, assembled based on existing literature (Osborne et al., 2005; Pan, 2012, e.g.). These pathways consist of p = 1624 unique genes, from the molecular signature database. Examples include the MAPK signaling, WNT signaling, TGF-β signaling, calcium signaling, cell communication, p53 signaling and breast cancer estrogen signaling pathways. Note that many of the pathways have overlapping genes while our method requires group indices to be non-overlapping since two groups with shared genes are obviously dependent of each other. To remove the influence of such trivial dependence, we further partitioned the 70 pathways into 249 non-overlapping gene subgroups whose sizes range from 1 to 110 with an average of 6.5. The algorithm used for such partitioning aims to identify the smallest number of non-overlapping subgroups that can cover all the genes under consideration. The partitioning algorithm begins with creating an M × p index matrix, 𝕀 = [I₁, …, I_p]. For m = 1, …, M and q = 1, …, p, the (m, q)th element of 𝕀 is set to 1 if the qth gene belongs to the mth pathway, and 0 otherwise. Then the subgroups are indexed by the unique values of {I₁, …, I_p}.

Applying our proposed methods with target false discovery rate of 0.01, we identified 494 between subgroup interactions out of the 30876 possible subgroup pairs. These between subgroup interactions can be mapped to 311 unique between pathway interactions and 18 within pathway interactions. The top pathways with highest numbers of interactions with other pathways include MAPK signaling, calcium signaling, gycan structures biosynthesis, WNT signaling, cell communication, TGF-β signaling and breast cancer estrogen pathways. The MAPK signaling pathway has interactions with 92 gene subgroups which corresponds to 31 pathways including TGF-β, MTOR, P53, WNT, and ERBB signaling pathways. The WNT signaling pathway interacts with 25 other pathways including TGF-β, MTOR, MPAK and breast cancer estrogen signaling. The TGF-β signaling pathway interacts with 21 other pathways including MAPK, p53, WNT and calcium signaling.

Many of these interactions have been previously documented. For example, experimental data suggest that inhibition of mTORC1 leads to MAPK pathway activation (Carracedo et al., 2008). The interaction between TGF-β and WNT pathways has been known for a long time and is probably the most extensively studied. At the organism level, TGF-β interacts with many other pathways at every stage of life from birth to death. During embryonic development, the complex but delicate interactions between the TGF-β, WNT, MAPK, and other pathways are important for a range of processes including body patterning, stem cell maintenance and cell fate determination (Guo and Wang, 2008). Kouzmenko et al. (2004) showed the first direct evidence of interaction between WNT and estrogen signaling pathways via functional interaction between β-catenin and ERα.

To examine whether these 70 breast cancer pathways are enriched with interactions, we randomly selected 50 sets of 70 pathways of similar sizes as the breast cancer pathways from the C2 pathway gene sets curated from various online databases (available from the Broad Institute). For each of the 70 randomly selected pathways, we performed the same analysis as the breast cancer pathways by first partitioning them into non-overlapping subgroups and then applied our method to identify significant between subgroup interactions. To determine whether the 70 breast cancer pathways are enriched with between subgroup interactions relative to these randomly selected pathways, we calculate the proportion of between subgroup interactions deemed as significant at the FDR level of 0.01. Across the 50 randomly selected pathways, the average proportion of significant pairs was 0.011 with standard deviation 0.002. The proportion of significant pairs we identified in the breast cancer data is 0.016, which is 2.5 standard deviations higher than the mean of proportions from those 50 random sets. The results suggest that the selected 70 pathways are indeed enriched with “interaction” pairs.

7 Discussions

We proposed in this paper a multiple testing procedure under the Gaussian graphical models for detecting between group interactions. The proposed method can potentially be extended in several directions. We discuss in this section two of these possible extensions.

7.1 Extension to Gaussian Copula Graphical Models

In the present paper, the problem of identifying the conditional between group interactions is translated to the problem of multiple testing of submatrices of a high-dimensional precision matrix Ω under the Gaussian graphical model framework. The main reason for the success of this approach is that the conditional independence between two non-overlapping groups of variables is equivalent to the corresponding submatrix of Ω being 0. This approach can be extended to more general settings of the semiparametric Gaussian copula graphical models where the population distribution is non-Gaussian, see Liu et al. (2012) and Xue and Zou (2012). The semiparametric Gaussian copula model assumes that the variables follow a joint normal distribution after a set of unknown marginal monotonic transformations. It would be interesting to develop a multiple testing procedure and investigate its properties under the semiparametric Gaussian copula graphical models. Detailed analysis is involved and is an interesting topic for future research.

7.2 The Two-Sample Case

We have focused on the one-sample case in this paper. It is also of interest to study the two-sample case where the goal is to discover the changes in the conditional dependence between pathway interactions under two different disease settings. In the one-sample case studied in this paper, Ω_𝒥m×𝒥h = 0 under H_0,m,h. Thus the null is simple but the technical details of deriving the limiting distribution of a given submatrix is still very involved because the correlation structure of {W_i,j, (i, j) ∈ 𝒥_m × 𝒥_h} largely depends on the entries outside of the submatrix of interest. In the two-sample case, we wish to test the hypotheses $H_{0,m,h}:{\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}^{(1)}={\mathbf{\Omega }}_{{\mathcal{J}}_m\times {\mathcal{J}}_h}^{(2)}$. Under the null hypothesis H_0,m,h, each submatrix is not necessary a zero matrix. So the null is composite, consequently the dependence structures of the suitable test statistics depend on the entries both inside and outside of the submatrices of direct interest. The two-sample case is technically even more challenging and we leave it as future work.

Table 5: Empirical FDRs (standard errors) (%) with n = 200, α = 0.1 and 0.01 respectively, 100 replications.

Table 6: Empirical powers (%) with n = 200, α = 0.1 and 0.01 respectively, 100 replications.

Table 7: Empirical FDRs (SEs) and powers (%) by mimicking the real data with α = 0.1 and α = 0.01 respectively, based on 100 replications.