Use of random integration to test equality of high dimensional covariance matrices

Abstract

Testing the equality of two covariance matrices is a fundamental problem in statistics, and especially challenging when the data are high-dimensional. Through a novel use of random integration, we can test the equality of high-dimensional covariance matrices without assuming parametric distributions for the two underlying populations, even if the dimension is much larger than the sample size. The asymptotic properties of our test for arbitrary number of covariates and sample size are studied in depth under a general multivariate model. The finite-sample performance of our test is evaluated through numerical studies. The empirical results demonstrate that our test is highly competitive with existing tests in a wide range of settings. In particular, our proposed test is distinctly powerful under different settings when there exist a few large or many small diagonal disturbances between the two covariance matrices.

1. Introduction

Testing the equality of two covariance matrices arises from many important problems including both the classic experimental designs and the analysis of high throughout omic data. For example, gene expression data have often been explored to classify disease types. Igolkina et al. (2018) pointed out that variance in gene expression is an important characteristic of schizophrenia. Roberts et al. (2018) showed that many genes differ in the variances of their gene expressions between disease states. Comparison between two covariance matrices is therefore essential to analyze gene expression microarray data of two different groups. This comparison becomes difficult because the number of the samples is usually much smaller than the number of genes. In the literature, many methods have been proposed to test the equality of two covariance matrices, but perform poorly in practice, especially when there are a few large or many small diagonal disturbances between the two covariance matrices.

Let X and Y be the p-dimensional random variables with covariance matrices Σ₁ and Σ₂, respectively. Given independent samples ${\mathcal{X}}_m=\left\{X_1,\cdots ,X_m\right\}$ from X and ${\mathcal{Y}}_n=\left\{Y_1,\cdots ,Y_n\right\}$ from Y, we want to test:

$$H_0:{\Sigma }_1={\Sigma }_{2}vs{H}_1:{\Sigma }_1\ne {\Sigma }_2.$$ (1.1)

In the classic low-dimensional setting, Anderson, T. W. (2003) proposed a likelihood ratio test (LRT) statistic and showed that the proposed LRT statistic is asymptotically χ²-distribution with degrees of freedom p(p + 1)/2 under the multivariate normality assumption and H₀ when p is fixed.

In recent applications from gene expression (Pan et al., 2018), to neuroimaging (Le et al., 2001), and to risk management (Bollerslev, 2019), the dimension p can be much larger than the sample size n, e.g., “large p small n” or “large p large n.” In this setting, the sample covariance matrix does not converge to the population counterpart (Bai et al, 2009), and the aforementioned classical methods for the low dimensional case either are not applicable or perform poorly. As well summarized by Cai (2017), this problem is so important and difficult that it has attracted a great deal of attention. We briefly review some of the methods below.

The modified LRT based methods have been considered by Bai et al (2009), Jiang et al. (2013), and Jiang and Qi (2015). Because Σ₁ = Σ₂ is equivalent to the Frobenius norm tr(Σ₁ − Σ₂)² = 0, the Frobenius norm-based tests have also been proposed (Schott, 2007; Srivastava and Yanagihara, 2010). However, The modified LRT based and Frobenius norm based methods assume multivariate normality. Li and Chen (2012) removed the normality assumption by using a linear combination of three one-sample U statistics. The test proposed by Li and Chen (2012) can be applied without assuming parametric distributions for the two populations and is very powerful when there are many small differences between two population covariance matrices, but this test was found not powerful against sparse alternative (Yang and Pan, 2017) or when the two covariance matrices differ slightly only in the diagonal (Wu and Li, 2015). Using random projection, Wu and Li (2015) constructed a test statistic to improve the power when there are many small diagonal disturbances between the two covariance matrices, but again assuming the normality. He et al. (2021) introduced the adaptive testing to combine the finite-order U-statistics that includes the variants of Frobenius norm based statistics.

Assuming sparsity for Σ₁ − Σ₂ under the alternative hypothesis, Cai et al. (2013) introduced an extreme statistic, which is robust with respect to the population distributions, and is very powerful when there are only several large disturbances between two population covariance matrices, and Chang et al. (2017) proposed a computationally fast procedure. Zhu (2017) constructed a sparse-Leading-Eigenvalue-Driven (sLED) test to deal with the scenario in which the signals are both sparse and weak, and proved that sLED can achieve full power asymptotically when the sparse signal is strong enough. However, these tests are either not powerful when there are many small disturbances between Σ₁ and Σ₂ or the theoretical properties required an explicit relationship between n and p or too complicated for practical use.

Above mentioned tests often targeted at some specific situations. To accommodate various situations, some weighted combination tests are proposed. For example, Yang and Pan (2017) proposed a weighted test statistic based on random matrix theories. Zheng et al. (2020) proposed a power-enhancement high-dimensional test. These tests can handle both sparsity and non-sparsity structures. However, these tests depend on the proper choice of weights, which is a very challenging task. Furthermore, these procedures also required an explicit relationship between n and p. Yu et al. (2020) proposed a scale-invariant power enhancement test based on Fisher’s method, but again assuming the normality.

In short, although many methods exist, they are limited by different reasons. Through a novel use of random integration, we propose a method to test equality of two high dimensional covariance matrices. Specifically, the our proposed test possesses the following three merits at the same time:

1. Our test does not require the distributional assumption.

2. Our test works well for the paradigm of “large p”, even when there exist a few large or many small diagonal disturbances between the two covariance matrices.

3. The asymptotic theory is established under a general multivariate model with certain moment conditions, but without requiring an explicit relationship between p and n.

The rest of the paper is organized as follows. In Section 2, we introduce our test statistic via the random integration technique, establish the asymptotic properties. In Section 3, simulation studies are conducted to evaluate the finite sample performance of the proposed test. In Section 4, a real data set is analyzed to compare the proposed test with some existing methods. We conclude with some remarks in Section 5 and all technical proofs are presented in the Supplementary Material.

2. Methodology and main results

To introduce a proper statistic to test (1.1), especially when there are many small diagonal disturbances between the two covariance matrices, it would be helpful if we can strengthen the information of diagonal disturbances for Σ₁ −Σ₂. Unlike the random matrix projection method, which needs the normality assumption to strengthen the information on a line, we develop the random integration technique to strengthen the information through integrations. To this end, we first denote X^c = X − μ₁ and Y^c = Y − μ₂, where μ₁ = EX and μ₂ = EY. Then, Σ₁ = EX^cX^c⊤ and Σ₂ = EY^cY^c⊤. Note the following equivalences

$$\begin{gathered} {\Sigma }_1={\Sigma }_2\iff E{X}^c{X}^{c\top }=E{Y}^c{Y}^{c\top } \\ \iff {\alpha }^{\top }E{X}^c{X}^{c\top }\alpha ={\alpha }^{\top }E{Y}^c{Y}^{c\top }\alpha ,\text{ for any }\alpha \in R^p \\ \iff E\left[{\alpha }^{\top }\left(X^c{X}^{c\top }-Y^c{Y}^{c\top }\right)\alpha \right]=0,\text{ for any }\alpha \in R^p. \end{gathered}$$

Thus, testing whether Σ₁ and Σ₂ amounts to testing whether

$${\text{RI}}_w(X,Y)\triangleq \int E^2\left[{\alpha }^{\top }\left(X^c{X}^{c\top }-Y^c{Y}^{c\top }\right)\alpha \right]w(\alpha )d\alpha =0,$$ (2.1)

where w(α) is a positive weight function. A critical observation is that RI_w(X, Y) may be evaluated easily for certain properly chosen w. The following Theorem 1 enables us to derive an explicit form for (2.1) and obtain our difference measure of two covariances.

Theorem 1. If w(α) is a p-dimensional standard normal density function, then

$$RI(X,Y)\triangleq R{I}_w(X,Y)={\left[tr\left({\Sigma }_1-{\Sigma }_2\right)\right]}^2+2tr{\left({\Sigma }_1-{\Sigma }_2\right)}^2,$$ (2.2)

and RI(X, Y) ≥ 0 with the equality holds if and only if Σ₁ = Σ₂.

Remark 1. The p-dimensional standard normal random vector can be expressed as the independent product between a uniformly distributed random vector on the unit sphere $S_1^{p-1}$ and the radial random variable. We choose the standard normal random density function to treat every unit vector equally in the integration. Moreover, the multivariate normal distribution is a special case of a multivariate stable distribution, and multivariate stable distributions have been used as the weight functions by Chen et al. (2019). As future work, it may be worthwhile to consider other weight functions such as uniform distribution (Zhu et al., 2017; Kim et al., 2020) and Bernoulli distribution (Qiu et al., 2021).

Note that the second term on the right hand size of (2.2) is the Frobenius norm of the difference between the two covariance matrix. the test can be designed to be powerful when there are many small disturbances between Σ₁ and Σ₂. Meanwhile, unlike Li and Chen (2012), the same test can be powerful when the two covariance matrices only differ by a small amount in the diagonal, due to the first term on the right hand size of (2.2). It represents the square of the difference between the diagonal elements of the two covariance matrix. If nonzero signals of the difference between the diagonal elements of Σ₁ and Σ₂ are weakly dense with almost the same sign, RI(X, Y) should be higher powerful than a different test statistic in which [tr(Σ₁ − Σ₂)]² is replaced by ${\sum }_{k=1}^p{\left({\Sigma }_{1,kk}-{\Sigma }_{2,kk}\right)}^2$. In addition, the estimation of RI(X, Y) does not need the consistent estimates of the covariance matrices since RI(X, Y) is based on the trace of matrices. In the following, we will obtain an unbiased estimator of RI(X, Y) which is our desired test statistic to test (1.1).

Denote

$$A_m^1=\frac{1}{m(m-1)}\sum _{i\ne j}\left(X_i^{\top }{X}_i\right)\left(X_j^{\top }{X}_j\right)-\frac{2}{m(m-1)(m-2)}\sum _{i,j,k}^{\star }{X}_i^{\top }{X}_j{X}_k^{\top }{X}_k+\frac{1}{m(m-1)(m-2)(m-3)}\sum _{i,j,k,l}^{\star }{X}_i^{\top }{X}_j{X}_k^{\top }{X}_l,$$

$$B_n^1=\frac{1}{n(n-1)}\sum _{i\ne j}\left(Y_i^{\top }{Y}_i\right)\left(Y_j^{\top }{Y}_j\right)-\frac{2}{n(n-1)(n-2)}\sum _{i,j,k}^{\star }{Y}_i^{\top }{Y}_j{Y}_k^{\top }{Y}_k+\frac{1}{n(n-1)(n-2)(n-3)}\sum _{i,j,k,l}^{\star }{Y}_i^{\top }{Y}_j{Y}_k^{\top }{Y}_l,$$

$$C_{m,n}^1=\frac{1}{mn}\sum _{i=1}^m\sum _{j=1}^n\left(X_i^{\top }{X}_i\right)\left(Y_j^{\top }{Y}_j\right)-\frac{1}{nm(m-1)}\sum _{i,k}^{\star }\sum _{j=1}^n{Y}_j^{\top }{Y}_j{X}_i^{\top }{X}_k-\frac{1}{mn(n-1)}\sum _{i,k}^{\star }\sum _{j=1}^m{X}_j^{\top }{X}_j{Y}_i^{\top }{Y}_k+\frac{1}{m(m-1)n(n-1)}\sum _{i,k}^{\star }\sum _{j,l}^{\star }{X}_i^{\top }{X}_k{Y}_j^{\top }{Y}_l,$$

$$A_m^2=\frac{1}{m(m-1)}\sum _{i\ne j}{\left(X_i^{\top }{X}_j\right)}^2-\frac{2}{m(m-1)(m-2)}\sum _{i,j,k}^{\star }{X}_i^{\top }{X}_j{X}_j^{\top }{X}_k+\frac{1}{m(m-1)(m-2)(m-3)}\sum _{i,j,k,l}^{\star }{X}_i^{\top }{X}_j{X}_k^{\top }{X}_l,$$

$$B_n^2=\frac{1}{m(m-1)}\sum _{i\ne j}{\left(Y_i^{\top }{Y}_j\right)}^2-\frac{2}{m(m-1)(m-2)}\sum _{i,j,k}^{\star }{Y}_i^{\top }{Y}_j{Y}_j^{\top }{Y}_k+\frac{1}{m(m-1)(m-2)(m-3)}\sum _{i,j,k,l}^{\star }{Y}_i^{\top }{Y}_j{Y}_k^{\top }{Y}_l,$$

$$C_{m,n}^2=\frac{1}{mn}\sum _{i=1}^m\sum _{j=1}^n{\left(X_i^{\top }{Y}_j\right)}^2-\frac{1}{nm(m-1)}\sum _{i,k}^{\star }\sum _{j=1}^n{X}_i^{\top }{Y}_j{Y}_j^{\top }{X}_k-\frac{1}{mn(n-1)}\sum _{i,k}^{\star }\sum _{j=1}^m{Y}_i^{\top }{X}_j{X}_j^{\top }{Y}_k+\frac{1}{m(m-1)n(n-1)}\sum _{i,k}^{\star }\sum _{j,l}^{\star }{X}_i^{\top }{Y}_j{X}_k^{\top }{Y}_l.$$

where ∑^⋆ denotes summation over mutually distinct indices. Then the proposed sample test statistic is

$${\text{RI}}_{m,n}=A_m^1-2C_{m,n}^1+B_n^1+2\left(A_m^2-2C_{m,n}^2+B_n^2\right),$$ (2.3)

which is an unbiased estimator of RI(X, Y).

Remark 2. The computation cost is at the order of max{pn⁴, pm⁴} if computed RI_m,n directly. In fact, the computation burden comes from the last two sums in $A_m^1$, $B_n^1$, $A_m^2$, $B_n^2$ and the last three in $C_{m,n}^1$, $C_{m,n}^2$. Since RI_m,n is invariant under the location shift, we can assume without loss of generality that μ₁ = μ₂ = 0. Under this assumption, the last two sums in $A_m^1$, $B_n^1$, $A_m^2$, $B_n^2$ and the last three in $C_{m,n}^1$, $C_{m,n}^2$ are all of smaller order than the first. This indicates that we can first transform data X_i to $X_i-\overline{X}$ and Y_j to $Y_j-\overline{Y}$, and then compute only the first term in $A_m^1$, $B_n^1$, $A_m^2$, $B_n^2$, $C_{m,n}^1$, $C_{m,n}^2$. These will reduces the computation cost to the order of max{pn², pm²} without affecting the asymptotic properties of our proposed test.

2.1. Asymptotic properties

To establish the limiting distribution of RI_m,n, we assume the following three conditions:

E1. There exist a p × m₁ matrix Γ₁, a p × m₂ matrix Γ₂, m₁-dimensional random vectors ${\left\{Z_{1j}\right\}}_{j=1}^m$, and m₂-dimensional random vectors ${\left\{Z_{2j}\right\}}_{j=1}^n$, such that X_j = μ₁+Γ₁Z_1j for j = 1, ⋯ , m, and Y_j = μ₂+Γ₂Z_2j for j = 1, ⋯ , n. And Γ_i, i = 1, 2, and $Z_{ij}={\left(Z_{ik1},\cdots ,Z_{ij{m}_i}\right)}^{\top }$ for i = 1, j = 1, ⋯ , m and i = 2, j = 1, ⋯ , n satisfy:

1. ${\Gamma }_1{\Gamma }_1^{\top }={\Sigma }_1$ and ${\Gamma }_2{\Gamma }_2^{\top }={\Sigma }_2$ with min{m₁, m₂} ≥ p.

2. ${\left\{Z_{1j}\right\}}_{j=1}^m$ and ${\left\{Z_{2j}\right\}}_{j=1}^n$ are independent and identically distributed, respectively, with EZ_1j = 0, $\text{Var}\left(Z_{1j}\right)=I_{m_1}$, and EZ_2j = 0, $\text{Var}\left(Z_{2j}\right)=I_{m_2}$, where $I_{m_i}$, is the m_i × m_i identity matrix.

3. sup_i,k E|Z_{i jk}|⁸ < ∞ and $E{Z}_{ijk}^4=3+{\Delta }_i$ for some constant Δ_i. Also,

   $$E\left(Z_{ij{l}_1}^{{\varsigma }_1}\cdots Z_{ij{l}_q}^{{\varsigma }_q}\right)=E\left(Z_{ij{l}_1}^{{\varsigma }_1}\right)\cdots E\left(Z_{ij{l}_q}^{{\varsigma }_q}\right)$$ (2.4)

   for any positive integers q and ς_l such that ${\sum }_{l=1}^q{\varsigma }_l\le 8$, and l₁, l₂, ⋯ , l_q are distinct indices.

E2. As min{m, n} → ∞, p → ∞, and for any i, j, k, l ∈ {1, 2}, tr(Σ_iΣ_j) → ∞ and

$$tr\left\{\left({\Sigma }_i{\Sigma }_j\right)\left({\Sigma }_k{\Sigma }_l\right)\right\}=o\left\{tr\left({\Sigma }_i{\Sigma }_j\right)tr\left({\Sigma }_k{\Sigma }_l\right)\right\}.$$ (2.5)

E3. As min{m, n} → ∞, m/(m + n) → τ ∈ (0, 1).

Remark 3. Condition E1 gives rise to a general multivariate model for high-dimensional data analysis, which includes commonly used distributions such as the multivariate normal distribution (Chen et al., 2010b; Srivastava and Yanagihara, 2010; Li and Chen, 2012). According to Chen and Qin (2010a), min{m₁, m₂} ≥ p means that the rank and eigenvalues of Σ₁ or Σ₂ are not affected by the transformation. According to Chen and Qin (2010a); He and Chen (2018), (2.4) can be viewed as a pseudo-independent condition of Z_ij. namely, a relaxed independence relation that allows some margin over probabilities (Kim and Lesser, 2008). Obviously, if Z_ij has independent components, then (2.4) is true.

In Condition E2, we do not require a direct relationship between p and n. We know cases where E2 does not imply a relation between p and n. For example, if all the eigenvalues of Σ_i are bounded away from zero and infinity, E2 holds. Meanwhile, some of the commonly encountered covariance structures satisfy Condition E2 (Chen et al., 2010b).

Condition E3 is a standard regularity assumption in two-sample problems, which guarantees that m and n go to infinity proportionally.

Theorem 2. Under Conditions E1–E3, as min{m, n} → ∞, we have

$$\frac{R{I}_{m,n}-RI(X,Y)}{{\sigma }_{m,n}}\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,1),$$

where ${\sigma }_{m,n}^2$ is defined in (A.2) in Supplementary Material.

Under H₀, we can obtain RI(X, Y) = 0, and

$${\sigma }_{0,m,n}^2=24{\left(\frac{1}{m}+\frac{1}{n}\right)}^{2}t{r}^2\left({\Sigma }^2\right).$$

Therefore, we obtain the following corollary.

Corollary 1. Under Conditions E1–E3 and H₀ : Σ₁ = Σ₂ = Σ, as min{m, n} → ∞, we have

$$R{I}_{m,n}/{\sigma }_{0,m,n}\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,1).$$

To formulate a test procedure, we need to estimate σ_0,m,n. Since $E{A}_m^2=tr\left({\Sigma }_1^2\right)$, $E{B}_n^2=tr\left({\Sigma }_2^2\right)$, the following is a consistent estimate ${\widehat{\sigma }}_{0,m,n}$ of σ_0,m,n under H₀:

$${\widehat{\sigma }}_{0,m,n}=2\sqrt{6}\left(\frac{1}{m}{A}_m^2+\frac{1}{n}{B}_n^2\right).$$

Furthermore, the following theorem assures that ${\widehat{\sigma }}_{0,m,n}$ is ratio-consistent to σ_0,m,n.

Theorem 3. Under Conditions E1–E3 and H₀ : Σ₁ = Σ₂ = Σ, as min{m, n} → ∞, we have

$$R{I}_{m,n}/{\widehat{\sigma }}_{0,m,n}\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,1).$$ (2.6)

As we shall derive in the Supplementary Material,

$$\frac{A_m^2}{tr\left({\Sigma }_1^2\right)}\stackrel{P}{\to }1,\frac{B_n^2}{tr\left({\Sigma }_2^2\right)}\stackrel{P}{\to }1,\text{\hspace{1em} and \hspace{1em}}\frac{{\widehat{\sigma }}_{0,m,n}}{{\sigma }_{0,m,n}}\stackrel{P}{\to }1.$$

Theorem 3 follows from Corollary 1 and the Slutsky’s theorem. Therefore, the proposed test with a nominal θ level of significance rejects H₀ if ${\text{RI}}_{m,n}\ge {\widehat{\sigma }}_{0,m,n}{z}_{\theta }$, where z_θ is the upper-θ quantile of $\mathcal{N}(0,1)$. The approximation result in the Supplementary Material indicates that the standard normal distribution can adequately substitute for the null distribution of ${\text{RI}}_{m,n}/{\widehat{\sigma }}_{0,m,n}$.

Next, we study the power of our proposed test. Let $g_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)=P\left({\text{RI}}_{m,n}\ge {\widehat{\sigma }}_{0,m,n}{z}_{\theta }\mid H_1\right)$ be the power of the proposed test under H₁ : Σ₁ ≠ Σ₂. Let SNR_m,n(Σ₁, Σ₂) = RI(X, Y)/σ_m,n and ${\gamma }_{m,n}=\left[\frac{tr\left({\Sigma }_1^2\right)}{m}+\frac{tr\left({\Sigma }_2^2\right)}{n}\right]/\text{RI}(X,Y)$, then we obtain the following Theorem 4.

Theorem 4. Under Conditions E1–E3 and H₁ : Σ₁ ≠ Σ₂, we have

$$\underset{m,n\to \infty }{\text{lim}}{g}_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)\ge \underset{m,n\to \infty }{\text{lim}}\Phi \left(-\sqrt{2}{z}_{\theta }+{\mathit{\text{SNR}}}_{m,n}\left({\Sigma }_1,{\Sigma }_2\right)\right),$$

where Φ(·) is the cumulative standard normal distribution function.

Theorem 4 indicates that the power of our proposed test is bounded from below. Meanwhile, the power is mainly determined by SNR_m,n(Σ₁, Σ₂). From (A.2) in Supplementary Material, we have

$${\sigma }_{m,n}^2\le 24{\left[\frac{tr\left({\Sigma }_1^2\right)}{m}+\frac{tr\left({\Sigma }_2^2\right)}{n}\right]}^2+20\text{\hspace{0.17em}max}\left\{2+{\Delta }_1,2+{\Delta }_2\right\}\left[\frac{tr\left({\Sigma }_1^2\right)}{m}+\frac{tr\left({\Sigma }_2^2\right)}{n}\right]\text{RI}(X,Y),$$

that is,

$${\text{SNR}}_{m,n}\left({\Sigma }_1,{\Sigma }_2\right)\ge {\left[24{\gamma }_{m,n}^2+20\text{\hspace{0.17em}max}\left\{2+{\Delta }_1,2+{\Delta }_2\right\}{\gamma }_{m,n}\right]}^{-1/2}.$$

Therefore, when γ_m,n → 0 as min{m, n} → ∞, we have SNR_m,n(Σ₁, Σ₂) → ∞. Thus, we have

$$\underset{m,n\to \infty }{\text{lim}}{g}_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)=1.$$

Especially, we consider the following three cases.

Case I: Let Σ₁ = rI_p + AR(0.1) and Σ₂ = AR(0.1), where AR(ρ) = (a_ij)_p×p is a covariance matrix with a_ij = ρ^|i−j| for i, j = 1, ⋯ , p. Then, we have the following Corollary 2.

Corollary 2. Under Conditions E1–E3, and ${\text{Lim}}_{m,n,p\to \infty }\sqrt{p}(m+n)r^2=c$, where 0 < c < ∞, then we have

$$\underset{m,n\to \infty }{\text{lim}}{g}_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)=1.$$

Corollary 2 shows that our proposed test is very powerful when there are many small diagonal disturbances between the two covariance matrices. In addition, under the conditions in Corollary 2, the signal-to-noise ratio of the method proposed by Li and Chen (2012) diminishes to 0. Therefore, the power of the test proposed by Li and Chen (2012) has a low bounded from below.

Case II: Let Σ₁ = I_p, Σ₂ = I_p + H(ϖ₀, ϖ₁, p₀), where H(ϖ₀, ϖ₁, p₀) = (h_ij)_p×p with h_ij = 0 except h_ii = ϖ₀, i = 1, ⋯ , p₀ and h_i,i+1 = h_i+1,i = ϖ₁, i = 1, ⋯ , p₀−1. Then, we have the following Corollary 3.

Corollary 3. Under Conditions E1–E3, $\frac{p}{(m+n)p_0^2{\varpi }_0^2}=o\left(1\right)$ and np₀ϖ₀ → ∞, then we have

$$\underset{m,n\to \infty }{\text{lim}}{g}_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)=1.$$

According to Cai et al. (2013), we take ${\varpi }_0=O(\sqrt{p/(m+n)})$ and p₀ = p^1/4. Corollary 3 shows that the proposed test is powerful. Meanwhile, the method proposed by Cai et al. (2013) is also powerful under this case.

Case III: Let Σ₁ = I_p, Σ₂ = I_p + M, where M is a p × p matrix with M_ii = 0 and M_ij = ω₁ for i ≠ j. Then, we have the following Corollary 4.

Corollary 4. Under Conditions E1–E3, $(m+n)p{\omega }_1^2\to \infty$ as m, n, p → ∞, then we have

$$\underset{m,n\to \infty }{\text{lim}}{g}_{m,n}\left({\Sigma }_1,{\Sigma }_2;\theta \right)=1.$$

Corollary 4 indicates that our proposed test is also very powerful under some conditions when the diagonals are the same and there are many small non-diagonal disturbances between the two covariance matrices.

3. Simulation Studies

We carry out numerical simulations to investigate finite-sample performance of our proposed method. We consider our proposed method (RI) with the method proposed by Li and Chen (2012) (LC), the method proposed by Cai et al. (2013) (Cai), the method proposed by Zhu (2017) (sLED), the method proposed by Chang et al. (2017) (Ψ_B,α), T2 method proposed by Zheng et al. (2020), and a scale-invariant power enhancement test based on Fisher’s method (Yu et al., 2020) (F_m,n). We set the nominal level of significance at 0.05. We choose the sample sizes as m = n = 60,100, and m = 200, n = 60, and the dimension as p = 300, 500, 800, 1000, 1200, 1500. All empirical sizes and powers are calculated from 1000 replications. For Σ₁ and Σ₂, we consider the following six scenarios:

Scenario 1: ${\Sigma }_1^{(1)}=I_p$, the identity matrix, ${\Sigma }_2^{(1)}=I_p+H(0.04,0.2,\lfloor 0.3p\rfloor )$, where H(ϖ₀, ϖ₁, k) = (h_ij)_p×p with h_ij = 0 except h_ii = ϖ₀, i = 1, ⋯ , k and h_i,i+1 = h_i+1,i = ϖ₁, i = 1, ⋯ , k − 1;

Scenario 2: ${\Sigma }_1^{(2)}=I_p$, ${\Sigma }_2^{(2)}=I_p+H(0.04,0.2,p)$;

Scenario 3: ${\Sigma }_1^{(3)}=I_p$, ${\Sigma }_2^{(3)}=I_p+{\Sigma }_{*}^{(3)}$, where ${\Sigma }_{*}^{(3)}=\left({\sigma }_{ij,*}^{(3)}\right)$ is a p × p matrix with ${\sigma }_{ii,*}^{(3)}=0$ for i = 1, ⋯ , p and ${\sigma }_{ij,*}^{(3)}=1/\sqrt{p}$ for i ≠ j.

Scenario 4: ${\Sigma }_1^{(4)}=I_p$, ${\Sigma }_2^{(4)}=I_p+H(4,0.05,\lfloor 0.02p\rfloor )$;

Scenario 5: ${\Sigma }_1^{(5)}={\Sigma }_{*}^{(5)}+{\delta }_0{I}_p$, ${\Sigma }_2^{(5)}={\Sigma }_{*}^{(5)}+{\delta }_0{I}_p+U$, where ${\Sigma }_{*}^{(5)}=\left({\sigma }_{ij,*}^{(5)}\right)=D^{1/2}C{D}^{1/2}$, D = diag(d₁, ⋯ , d_p) and d₁, ⋯ , d_p ~^i.i.d. Unif(0.5, 2.5), C = (c_ij) with c_ii = 1, c_ij = 0.5 for 5(k − 1) + 1 ≤ i ≠ j ≤ 5k, where k = 1, ⋯ , [p/5] and c_ij = 0 otherwise. U is a p × p symmetric matrix with four nonzero entries from $\text{Unif}(0,4)\times {\text{max}}_{1\le j\le p}{\sigma }_{jj,*}^{(5)}$ randomly located in the upper triangle, and another four located in the lower triangle by symmetry. ${\delta }_0=\left|\text{min}\left\{{\lambda }_{\text{min}}\left({\Sigma }_{*}^{(5)}+U\right),{\lambda }_{\text{min}}\left({\Sigma }_{*}^{(5)}\right)\right\}\right|+0.05$, where λ_min(A) denotes the minimum eigenvalue of a symmetric matrix A.

Scenario 6: ${\Sigma }_1^{(6)}=0.2\times I_p+AR(0.1)$, ${\Sigma }_2^{(6)}=AR(0.1)$, where AR(ρ) = (a_ij)_p×p is a covariance matrix with a_ii = 1 and a_ij = ρ^{|i− j|} for i ≠ j;

Scenario 7: ${\Sigma }_1^{(7)}=0.1\times I_p+AR(0.2)$, ${\Sigma }_2^{(7)}=AR(0.24)$.

Scenario 8: ${\Sigma }_1^{(8)}={\Sigma }_{*}^{(8)}+{\lambda }_0{I}_p$, ${\Sigma }_2^{(8)}={\Sigma }_{*}^{(8)}+Q+{\lambda }_0{I}_p$, where ${\Sigma }_{*}^{(8)}={\left({\sigma }_{ij,*}^{(8)}\right)}_{1\le i,j\le p}$ with i.i.d ${\sigma }_{ii,*}^{(8)}~\text{Unif}(1,2)$ and ${\sigma }_{ij,*}^{(8)}=\left\{{(\left|i-j\right|+1)}^{2H}+{(\left|i-j\right|-1)}^{2H}-2{(\left|i-j\right|)}^{2H}\right\}/2$ with H = 0.85 for i ≠ j. A perturbation matrix Q has ⌊0.05p⌋ random non-zero elements in diagonal and in non-diagonal, respectively. ⌊0.05p⌋/2 non-zero elements are randomly allocated in the upper triangle part of Q and the others are in its lower triangle part by symmetry. The magnitudes of non-zero elements are randomly generated from Unif(τ/2, 3τ/2) with $\tau =8\text{\hspace{0.17em}max}\left\{{\text{max}}_{1\le i\le p}{\sigma }_{ii,*}^{(8)},{(\text{log\hspace{0.17em}}p)}^{1/2}\right\}$, and ${\lambda }_0=\left|\text{min}\left\{{\lambda }_{\text{min}}\left({\sum }_{*}^{(8)}+Q\right),{\lambda }_{\text{min}}\left({\sum }_{*}^{(8)}\right)\right\}\right|+0.05$.

Finally, the data are generated by $X_i={\Sigma }_1^{1/2}{Z}_i$ for i = 1, ⋯ , m and $Y_l={\Sigma }_2^{1/2}{Z}_{m+l}$ for l = 1, ⋯ , n, where {Z_i : i = 1, ⋯ , m + n} are independent p-dimensional random variables with i.i.d. coordinates Z_ij, j = 1, ⋯ , p. We consider the following four distributions for Z_ij:

1. The standard normal $\mathcal{N}(0,1)$;

2. The t-distribution with degrees of freedom 15, i.e., t(15);

3. The centralized Gamma distribution with a = 16, b = 0.25, i.e., Γ(16,0.25)−4;

4. The discrete distribution has five possible values −2, −1, 0, 3/2, 4 with probabilities 1/12, 4/25, 13/24, 16/75, 1/600, i.e.,

   $$Z_{ij}~\frac{1}{12}{\delta }_{-2}+\frac{4}{25}{\delta }_{-1}+\frac{13}{24}{\delta }_0+\frac{16}{75}{\delta }_{3/2}+\frac{1}{600}{\delta }_4.$$

   This distribution is used in Yang and Pan (2017), and they showed that the first four moments of Z_ij are the same as those of $\mathcal{N}(0,1)$.

There are reasonably small disturbances between Σ₁ and Σ₂, and these two covariance matrices are reasonably sparse in Scenario 1. This is similar to the case considered by Yang and Pan (2017). In Scenario 2, there are many small disturbances between Σ₁ and Σ₂. This case was also considered by Yang and Pan (2017) and Li and Chen (2012). In Scenario 3, the two matrices differ only in off-diagonal entries with many weakly dense signals. Scenario 4 deals with the sparse situation, with ⌊0.02p⌋ features exerting larger signals. Scenario 5 is for the extremely sparse situation, with 8 nonzero signals in the non-diagonal entries. Scenarios 5 was considered by Cai et al. (2013). In Scenario 6, the two covariance matrices only differ in the diagonal. The two covariance matrices are entirely different by a larger amount in Scenario 7. Scenarios 6 and 7 were also considered by Wu and Li (2015). Scenarios 8 was studied by Chang et al. (2017) except that a perturbation matrix Q adds some non-zero elements in diagonal.

In all scenarios, we first calculate the empirical p-values when Σ₁ = Σ₂ = Σ⁽ⁱ⁾ for i = 1, ⋯ , 8. The corresponding results are given in Tables 1–15 of the Supplementary Material. It can be seen from Tables 1–15 that the estimated p-values of the proposed RI method and the other six methods are controlled fairly well around 0.05 for all cases except the method proposed by Cai et al. (2013) for the discrete distribution.

Since the empirical powers for m = n = 100 and m = 200,n = 60 are very similar to those for m = n = 60, we only present the empirical powers with ${\Sigma }_1={\Sigma }_1^{(i)}$ and ${\Sigma }_2={\Sigma }_2^{(i)}$, i = 1, ⋯ , 8 for m = n = 60 in Figures 1–4. The empirical powers for m = n = 100 and m = 200, n = 60 are included in the Supplementary Material.

Figure 1:

Empirical powers for Scenarios 1–6 with Z_ij follows $\mathcal{N}(0,1)$ and m = n = 60.

Figure 4:

Empirical powers for Scenarios 1–6 with Z_ij follows the discrete distribution $\frac{1}{12}{\delta }_{-2}+\frac{4}{25}{\delta }_{-1}+\frac{13}{24}{\delta }_0+\frac{16}{75}{\delta }_{3/2}+\frac{1}{600}{\delta }_4$ and m = n = 60.

For Scenarios 1–2, we have the following findings:

1. Our proposed RI test is considerably more powerful than the other methods. LC is the second most powerful, suggesting its ability in detecting the covariance differences with many small disturbances, as also observed by Yang and Pan (2017). T2 is the third most powerful since it is a weighted statistic. The Cai, sLED and Ψ_B,α methods have poor power, below 0.20 in almost all cases.

2. The powers of the RI, LC, T2, and F_m,n methods increase from Scenario 1 to Scenario 2, which is expected because we increase the small disturbances in the differences between the two covariance matrices. So, RI, LC, T2, and F_m,n gain power as the amount of deviations increases, even if these deviations are not large. This is expected because they are defined from the Frobenius norm of the difference in the two covariance matrices. However, the other three methods are little affected as we increase the differences.

3. It is not surprising but reassuring that the power of our proposed method increases as the sample sizes n, m increases.

In Scenario 3, the two covariance matrices have many weakly dease signals in the non-diagonal entries. The power of RI, LC, sLED, T2, and F_m,n is close to 1, but the power of Cai test and Ψ_B,α is lower. The results are verified by Corollary 4.

Scenario 4 contains strong and sparse signals in the main diagonal of the difference between the two covariance matrices. All seven methods perform quite well, especially when the sample size m or n is large. Meanwhile, the power of our proposed RI method is close to 1.

Scenario 5 contains the extremely sparse, strong signals in the non-diagonal differences between the two covariance matrices. The power of Cai test, T2, Ψ_B,α, and F_m,n is high, but the power of RI, LC and sLED are lower.

When the two covariance matrices differ in the main diagonal with sparsely weak signals in Scenario 6, our proposed RI method still has the perfect power in all cases. The powers of the other six methods are poor. The results are verified by Corollary 2.

In Scenario 7, the two covariance matrices are entirely different to a large degree. The power of the RI method is close to 1, whereas the other six methods are much less powerful and not competitive with RI.

For Scenario 8, the two covariance matrices have long-range dependence according to Chang et al. (2017). The power of the RI method is more higher than other six methods for large p.

In conclusion, the RI method has considerably higher and more stable power than the six existing methods in a wide range of settings. Not only can it deal with the cases with many small deviations in difference of the covariance matrices, but also handle the cases with sparsely strong or sparsely weak signals. Thus it is applicable in broader applications in testing the difference between covariance matrices.

4. Real data Analysis

In this section, we apply the proposed method to analyze a gene expression dataset on breast cancer from a study reported by Schemidt et al. (2008). We downloaded the dataset from http://bioconductor.org/packages/release/data/experiment/html/breastCancerMAINZ.html.

The dataset contains gene expression patterns from 200 tumors of patients who were not treated by systemic therapy after surgery, and consists of 22,283 features. There were three groups of different tumor grades: 29 well differentiated tumors (group 1), 136 moderately differentiated tumors (group 2), and 35 poor/undifferentiated tumors (group 3). This dataset has been analyzed in the literature under the assumption that the two covariance matrices are equal; e.g., Teschendorff and Caldas (2008) and Haibe-Kains et al. (2012). The equality of the two covariance matrices is a very important assumption for the validity of the reported findings. We test whether this assumption is valid.

Following Gentleman et al. (2011), for quality control and computational burden considerations, we first select the features for which more than 50% of their intensities were greater than 5 and their coefficients of variation (CV) fall inside of the range (0.22, 1.0). The intensities are gene expressions as measured by Affymetrix hgu133a technology, and the coefficient of variation is the standard deviation divided by the absolute value of the mean. We also screen the features by predetermined cutoffs, which remove low-quality features while retaining high-quality features as previously done in this dataset (Sherafatian, 2018; Chong et al., 2018; Schiffman et al., 2008). After these selection steps, 1193 features remain in our analysis. Let Σ₁, Σ₂ and Σ₃ be the covariance matrices of these 1193 features with groups 1, 2 and 3, respectively. We standardized each of the 1193 features so that its mean is zero. Then, we apply the LC, Cai, sLED, T2, Ψ_B,α, F_m,n and RI methods to test separately (a) $H_0^{(1,2)}:{\Sigma }_1={\Sigma }_2$, and (b) $H_0^{(2,3)}:{\Sigma }_2={\Sigma }_3$. The p-values for all these methods are reported in Table 1. From Table 1, the RI method rejects both $H_0^{(1,2)}$ and $H_0^{(2,3)}$ at the significant level of 0.05, but the LC, sLED, T2, and F_m,n methods reject $H_0^{(2,3)}$ only.

Table 1:

The corresponding p-values of the LC, Cai, sLED, T2, Ψ_B,α, F_m,n, and RI methods for a gene expression dataset

Method	LC	Cai	sLED	T2	ψB,α	F m,n	RI
$H_0^{\left(1,2\right)}$	0.105	0.104	0.330	0.555	0.056	0.061	1.110×10−16
$H_0^{\left(2,3\right)}$	3.050×10−6	0.093	0.000	8.157×10−9	0.052	4.577×10−6	3.030×10−9

To visualize the comparison among the different methods, we plot the heat maps of $\left({\widehat{\Sigma }}_1-{\widehat{\Sigma }}_2\right)$ and $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$ for the top 100 features with the largest absolute values of the two-sample t statistics, respectively, where ${\widehat{\Sigma }}_1$, ${\widehat{\Sigma }}_2$, ${\widehat{\Sigma }}_3$ are the sample covariance matrices based on the selected 100 features. The corresponding results are shown in Figure 5. It is observed from Figure 5 that $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$ has stronger signals than $\left({\widehat{\Sigma }}_1-{\widehat{\Sigma }}_2\right)$. Meanwhile, many moderate disturbances are present in $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$. The maximum absolute value in the elements of the estimator of $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$ for the 1193 features is 8.972, which is larger than the maximum absolute value in $\left({\widehat{\Sigma }}_1-{\widehat{\Sigma }}_2\right)$, i.e., 4.648. However, the diagonal in $\left({\widehat{\Sigma }}_1-{\widehat{\Sigma }}_2\right)$ has much stronger signals than those in $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$.

Figure 5:

(a) the heat map of $\left({\widehat{\Sigma }}_1-{\widehat{\Sigma }}_2\right)$ for the selected 100 features; (b) the heat map of $\left({\widehat{\Sigma }}_2-{\widehat{\Sigma }}_3\right)$ for the selected 100 features.

Both Equation (2.2) and the simulation results of Scenarios 4–5 reveal that that the our proposed method is more powerful than the other three methods when there are more signals in the diagonal for the difference of two covariance matrices. This explains why our proposed method rejects $H_0^{(1,2)}$ whereas the others did not. Consequently, our method is only one that is able to detect a more subtle but very important difference in this commonly analyzed dataset.

5. Discussion

Conducting inference for high-dimensional covariance matrix is highly challenging. In this paper, we make a novel use of a random integration technique to develop a two-covariance matrix test statistic. This test can be performed without estimating the covariance matrices, which is known to be extremely difficult in high dimension data. We investigate both theoretical properties and numerical performance of our method, and find that it is not only competitive but also often much more powerful than the existing methods in both simulation studies and a real data analysis when there are many small diagonal disturbances between the two covariance matrices.

There are several issues that warrant further investigation. First, a general multivariate model is applied to obtain asymptotic results. Although it is a common assumption in the literature, it would be useful to investigate the asymptotic properties of our proposed method in weaker conditions, for example, assumption 2.1 in Han and Wu (2020). Second, we consider a two-sample test for high-dimensional covariance matrices. As in Zheng et al. (2020), it would be interesting to extend our method for testing the equality of more than two covariance matrices. Third, we will extend the proposed method to test for high dimensional correlation matrices, which is a more difficult task than testing the covariance matrix (Zheng et al., 2019). Finally, we use the standard multivariate normal density function as the weight function during the construction of our test statistic. This is a common practice, but it is worthy investigating other choices that may perform better under various settings.

Figure 2:

Empirical powers for Scenarios 1–6 with Z_ij follows t(15) and m = n = 60.

Figure 3:

Empirical powers for Scenarios 1–6 with Z_ij follows Γ(16, 0.25) – 4 and m = n = 60.