Projection Test for Mean Vector in High Dimensions

Abstract

This paper studies the projection test for high-dimensional mean vectors via optimal projection. The idea of projection test is to project high-dimensional data onto a space of low dimension such that traditional methods can be applied. We first propose a new estimation for the optimal projection direction by solving a constrained and regularized quadratic programming. Then two tests are constructed using the estimated optimal projection direction. The first one is based on a data-splitting procedure, which achieves an exact $t$-test under normality assumption. To mitigate the power loss due to data-splitting, we further propose an online framework, which iteratively updates the estimation of projection direction when new observations arrive. We show that this online-style projection test asymptotically converges to the standard normal distribution. Various simulation studies as well as a real data example show that the proposed online-style projection test retains the type I error rate well and is more powerful than other existing tests.

1. Introduction

Testing whether a population mean equals a known vector or the equality of means from two populations is a fundamental problem in high-dimensional statistics with wide applications. Such tests are commonly encountered in genome-wide association studies. For instance, Chen and Qin (2010) performed a test to identify sets of genes which are responsible for certain types of tumors in a genetics research. Xu et al. (2016) applied various tests to a bipolar disorder dataset collected by Wellcome Trust Case Control Consortium (2007) in which one would like to test whether there is any association between a disease and a large number of genetic variants. In these applications, the dimension of collected data $p$ is often much larger than the sample size $n$. Suppose that $\left\{{\mathbf{x}}_1,\cdots ,{\mathbf{x}}_n\right\}$ is a random sample from a $p$-dimensional population with mean $\mathit{\mu }$ and positive definite covariance matrix $\mathbf{\Sigma }$. Of interest is to test the following hypothesis

$$H_0:\mathit{\mu }={\mathit{\mu }}_0\hspace{1em}\text{versus}\hspace{1em}{H}_1:\mathit{\mu }\ne {\mathit{\mu }}_0,$$ (1)

for some known vector ${\mathit{\mu }}_0$. This is typically referred to as the one-sample hypothesis testing problem in multivariate analysis and has been extensively studied when $p$ is fixed. When $p\ge n$, the well-known Hotelling’s $T^2$ test (Hotelling 1931) is not directly applicable as the sample covariance matrix is not invertible. Even in the case $n>p$, the Hotelling’s $T^2$ test may suffer from low power against the alternative if $p/n\to c\in \left(\mathrm{0,1}\right)$ for some $c$ that is close to 1 (Bai and Saranadasa 1996).

More recently, various tests have been proposed for high-dimensional mean vectors, which can be roughly classified into three types (Huang et al. 2022). The first type is known as quadratic-form tests, which can be viewed as modified Hotelling’s $T^2$ tests. Quadratic-form tests substitute the singular sample covariance matrix by an invertible diagonal matrix, leading to a quadratic form of the sample mean. For example, Bai and Saranadasa (1996) substituted the sample covariance matrix by identity matrix I and showed the resulting statistic has asymptotic normal null distribution if $p/n\to c>0$. Quadratic-form tests are known to be more powerful against dense alternatives. However, these quadratic-form tests tend to ignore the dependence among the $p$ dimensions and would lose testing power when correlations among the variables are high. The second type is known as extreme-type tests. An extreme-type test usually makes a decision based on the maximum value of a sequence of candidate test statistics. For example, to test a high-dimensional mean vector, one can easily construct a marginal test for each dimension and then use the maximum value of the $p$ marginal test statistics as the final test statistic (Cai et al. 2014, Chang et al. 2017). Another example is the sum-of-powers test which is in the form of sum of $p$ marginal statistics with power index leading to the most extreme p-value (Xu et al. 2016). Though extreme-type tests are typically more powerful against sparse alternatives (Cai et al. 2014), they generally require stringent conditions in order to derive the limiting null distribution and are likely to suffer from size distortions due to slow convergence rate. The third type is projection test. The idea of projection test is to project the high-dimensional observations onto a space of low dimension and then traditional methods such as Hotelling’s $T^2$ can be directly applied. A primary task of projection test is to find a good projection matrix (or vector) such that the resulting test achieves high power. A random projection test was proposed in Lopes et al. (2011) where entries in the projection matrix are randomly drawn from the standard normal distribution. Instead of random projection, data-driven approaches are also studied to construct the projection matrix. Lauter (1996) proposed a test that projects data onto a 1-dimensional space by some weight vector which depends on data only through ${\mathbf{X}}^{\top }\mathbf{X}$. Recently, Huang (2015) studied the optimal projection such that the resulting projection test has the highest power among all possible projection matrices. It has been shown that we should project data onto a 1-dimensional space to achieve the highest power and the optimal projection matrix is of the form ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$. Li and Li (2021) further extended the idea of optimal projection matrix for hypothesis testing in high-dimensional linear models.

Optimal direction based projection tests have not been systematically studied yet and there are a few open questions remaining to be addressed. The first question is how to obtain a good estimation of the optimal projection with statistical guarantees. A naive ridge-type estimator is proposed in Huang (2015), but little is known about its theoretical properties. The power analysis of their test relies on the assumption that the ridge-type estimator is consistent, which is not necessarily true in the high-dimensional setting. The second question is how to construct a powerful test based on the estimated optimal projection. Notice that the optimal projection ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is typically unknown and needs to be estimated from observed data. Huang (2015) proposed a data-splitting strategy which utilizes half of the observations to estimate the optimal projection and uses the remaining half to test the hypothesis. The major advantage of the data-splitting strategy is the estimated projection direction is independent of the remaining data and an exact test can be easily constructed. However, there is substantial power loss due to data-splitting as only half of the data is used to perform the test. This paper aims to fill these gaps by constructing an online-style projection test via optimal projection. Firstly, we propose a new estimation of the optimal projection direction by solving a constrained and regularized quadratic programming problem. Under the assumption that the optimal projection is sparse, we can show that any stationary point of the quadratic programming problem is a consistent estimator. In other words, we do not need to find the global solution to the possibly nonconvex optimization problem as any stationary point enjoys desirable statistical properties. Secondly, we propose an online framework to boost testing power, which originates from online learning algorithms for streaming data. The main idea is to recursively update the estimation when new observations arrive. In particular, we first obtain an initial estimator of the optimal projection via a subset of data. Then we recursively project the incoming observations onto a 1-dimensional space and then update the estimation by including the newly arrived observations. We repeat this procedure until the arrival of the last observation and then perform a test based on the projected sample.

The rest of this paper is organized as follows. In Section 2, we propose a new estimation of the optimal projection via constrained and regularized quadratic programming and derive non-asymptotic $L_1$ and $L_2$ error bounds. Such error bounds hold for all stationary points. In Section 3, we propose the online-style projection test and study its asymptotic limiting distributions. In Section 4, numerical studies are conducted to examine the finite sample performances of different tests with application to a real data example. We conclude this paper with some discussion in Section 5. All technical proofs and additional simulation results are given in the supplemental material.

2. Optimal Direction Based Projection Tests

2.1. Preliminary

Without loss of generality, we assume ${\mathit{\mu }}_0=0$ and the one-sample problem (1) becomes

$$H_0:\mathit{\mu }=\mathbf{0}\hspace{1em}\text{versus}\hspace{1em}{H}_1:\mathit{\mu }\ne \mathbf{0}.$$ (2)

Given a random sample ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$ with mean $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }$, let $\stackrel{-}{\mathbf{x}}$ and $\widehat{\mathbf{\Sigma }}$ be the sample mean and the sample covariance matrix, respectively,

$$\overline{x}=\frac{1}{n}\sum _{i=1}^n{x}_i,\hspace{1em}\widehat{\Sigma }=\frac{1}{n-1}\sum _{i=1}^n\left(x_i-\overline{x}\right){\left(x_i-\overline{x}\right)}^{\top }.$$ (3)

When $n>p$, the Hotelling’s $T^2$ for problem (2) is $T^2=n{\stackrel{-}{\mathbf{x}}}^{\top }{\widehat{\mathbf{\Sigma }}}^{-1}\stackrel{-}{\mathbf{x}}$. If ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$ are independent and identically distributed (i.i.d.) normal random variables, then $(n-p)/\left\{\right(n-1\left)p\right\}{T}^2~F_{p,n-p}$ under $H_0$, where $F_{p,n-p}$ denotes $F$ distribution with degrees of freedom $p$ and $n-p$. The Hotelling’s $T^2$ requires the sample covariance matrix $\widehat{\mathbf{\Sigma }}$ to be invertible and cannot be directly applied when $n\le p$. Despite the singularity of $\widehat{\mathbf{\Sigma }}$, it has been observed that the power of the Hotelling’s $T^2$ test can be adversely affected even when $p<n$ if $\widehat{\mathbf{\Sigma }}$ is nearly singular (Bai and Saranadasa 1996).

Projection tests are proposed for high-dimensional data. The basic idea of projection tests is to reduce dimension by projecting the high-dimensional vector ${\mathbf{x}}_i$ onto a space of lower dimension and then traditional methods such as Hotelling’s $T^2$ can be carried out. Let $\mathbf{P}$ be a $p\times q$ projection matrix with $q\ll n$. We project the $p$-dimensional vector ${\mathbf{x}}_i$ to a $q$-dimensional space by left-multiplying ${\mathbf{P}}^{\top }$, i.e., ${\mathbf{y}}_i={\mathbf{P}}^{\top }{\mathbf{x}}_i,i=1,\dots ,n$. Conditional on the projection matrix $\mathbf{P}$, the projected sample ${\mathbf{y}}_1,\dots ,{\mathbf{y}}_n\in {\mathbb{R}}^q$ are independent and identically distributed with mean ${\mathbf{P}}^{\top }\mathit{\mu }$ and covariance matrix ${\mathbf{P}}^{\top }\mathbf{\Sigma }\mathbf{P}$. Then the original problem (2) can be reformulated as

$$H_0^{\prime }:{\mathbf{P}}^{\top }\mathit{\mu }=\mathbf{0}\hspace{1em}\text{versus}\hspace{1em}{H}_1^{\prime }:{\mathbf{P}}^{\top }\mathbf{\mu }\ne \mathbf{0}.$$ (4)

Note that the hypothesis in (4) is not necessarily equivalent to that in (2). If $H_0^{\prime }$ is rejected, then $H_0$ is rejected as well but not vice versa. Given the projection matrix $\mathbf{P}$, we construct Hotelling’s $T^2$ test, denoted by $T_{\mathbf{P}}^2$, based on the projected sample $\left\{{\mathbf{P}}^{\top }{\mathbf{x}}_1,\dots ,{\mathbf{P}}^{\top }{\mathbf{x}}_n\right\}$,

$$T_{\mathbf{P}}^2=n{\stackrel{-}{\mathbf{x}}}^{\top }\mathbf{P}{\left({\mathbf{P}}^{\top }\widehat{\mathbf{\Sigma }}\mathbf{P}\right)}^{-1}{\mathbf{P}}^{\top }\stackrel{-}{\mathbf{x}}.$$

A key question is how to choose the dimension $q$ and the projection matrix $\mathbf{P}$. Huang (2015) proved that under normality assumption, the optimal projection direction is proportional to ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ in the sense that the power of $T_{\mathbf{P}}^2$ is maximized. Without normality assumption, the direction ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is asymptotically optimal provided that the sample mean follows asymptotically normal. Throughout this paper, it is assumed that ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$ are i.i.d. sub-Gaussian, and hence ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is asymptotically the optimal direction.

Define $\mathit{\theta }={\mathbf{\Sigma }}^{-1}\mathit{\mu }$. With the choice of $\mathbf{P}=\mathit{\theta }$, the hypothesis becomes

$$H_0^{\prime \prime }:{\mathit{\theta }}^{\top }\mathit{\mu }=\mathbf{0}\hspace{1em}\text{versus}\hspace{1em}{H}_1^{\prime \prime }:{\mathit{\theta }}^{\top }\mathit{\mu }\ne \mathbf{0},$$ (5)

which is equivalent to the original hypothesis (2) as ${\mathit{\theta }}^{\top }\mathit{\mu }=0$ implies $\mathit{\mu }=0$. This optimal projection direction depends on unknown parameters and needs to be estimated from data. In order to control the type I error, Huang (2015) proposed an exact $t$-test via data-splitting. The entire dataset is randomly partitioned into two disjoint sets: ${𝒟}_1=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{n_1}\right\}$ and ${𝒟}_2=\left\{{\mathbf{x}}_{n_1+1},\dots ,{\mathbf{x}}_n\right\}$. The first dataset ${𝒟}_1$ is used to estimate $\mathit{\theta }$ and the second dataset ${𝒟}_2$ is used to construct the test statistic. To estimate $\mathit{\theta }$, a ridge-type estimator $\widehat{\mathit{\theta }}={\left({\widehat{\mathbf{\Sigma }}}_1+\lambda {\mathbf{D}}_1\right)}^{-1}{\stackrel{-}{\mathbf{x}}}_1$ is used in Huang (2015), where ${\stackrel{-}{\mathbf{x}}}_1$ and ${\widehat{\mathbf{\Sigma }}}_1$ are the sample mean and the sample covariance estimated from ${𝒟}_1$ an ${\mathbf{D}}_1=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({\widehat{\mathbf{\Sigma }}}_1\right)$. Define $y_i={\widehat{\mathit{\theta }}}^{\top }{\mathbf{x}}_i,i=n_1+1,\dots$, $n$, then the test statistic is constructed as $T_{\widehat{\mathit{\theta }}}^2=\sqrt{n_2}\bar{y}/s_y$, where $\bar{y}$ and $s_y$ are the sample mean and standard deviation based on $\left\{y_{n_1+1},\dots ,y_n\right\}$. The advantage of data-splitting is the constructed test $T_{\widehat{\mathit{\theta }}}^2$ is an exact test under normality assumption. The power analysis of $T_{\widehat{\mathit{\theta }}}^2$ relies on the consistency of ridge-type estimator $\widehat{\mathit{\theta }}$, which is no longer true in the high-dimensional regime. In addition, this data-splitting approach suffers from power loss since only a subset of data is used to perform the test. In this paper, we propose a different approach to estimate the optimal projection direction such that the resulting estimator is consistent. Furthermore, we propose an online framework to perform the test such that more observations are utilized to boost the power.

2.2. Estimation of optimal projection direction

We first introduce some notations. For a vector $\mathbf{v}={\left(v_j\right)}_{j=1}^p\in {\mathbb{R}}^p$, let $\parallel \mathbf{v}{\parallel }_k$ be its $L_k$ norm, $k=1$,2. Its $L_0$ norm $\parallel \mathbf{v}{\parallel }_0$ is the number of nonzero entries in $\mathbf{v}$ and $L_{\infty }$ norm is $\parallel \mathbf{v}{\parallel }_{\infty }=\mathrm{m}\mathrm{a}\mathrm{x}\left|v_j\right|$. For a matrix $\mathbf{M}={\left(m_{ij}\right)}_{i,j=1}^p\in {\mathbb{R}}^{p\times p}$, its entrywise $L_{\infty }$ norm is $\parallel \mathbf{M}{\parallel }_{\mathrm{m}\mathrm{a}\mathrm{x}}=\mathrm{m}\mathrm{a}\mathrm{x}\left|m_{ij}\right|$. For a set $𝒟$,$\left|𝒟\right|$ denotes its cardinality. We use $\lfloor a\rfloor$ to denote the largest integer that is smaller or equal to $a$, and use $\mathbf{X}={\left({\mathbf{x}}_1,\dots ,{\mathbf{x}}_p\right)}^{\top }$ to denote the data matrix.

In practice, the optimal projection direction ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is typically unknown and needs to learned from samples. A simple plug-in estimator for ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is replacing $\mathbf{\Sigma }$ and $\mathit{\mu }$ by their sample counterparts $\widehat{\mathbf{\Sigma }}$ and $\stackrel{-}{\mathbf{x}}$. The first challenge is when $p>n$, the sample covariance matrix is no longer invertible. This challenge can be overcomed by imposing additional sparsity structure on $\mathbf{\Sigma }$ or ${\mathbf{\Sigma }}^{-1}$ and apply thresholding or penalized likelihood (Fan et al. 2016). The second challenge is even if we achieve a consistent estimator ${\widehat{\mathbf{\Sigma }}}^{-1}$, the consistency of ${\widehat{\mathbf{\Sigma }}}^{-1}$ does not imply the consistency of ${\widehat{\mathbf{\Sigma }}}^{-1}\stackrel{-}{\mathbf{x}}$ (Chen et al. 2016). These challenges motivate us to seek a direct estimation of the vector ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$. Let $\mathit{\theta }={\mathbf{\Sigma }}^{-1}\mathit{\mu }$ denote the optimal projection direction. Notice that any vector that is proportional to $\mathit{\theta }$ is also optimal as the projection test statistic $T_{\mathit{\theta }}^2$ is scale invariant. Based on this observation, we consider the following constrained quadratic optimization problem

$$\underset{\mathit{\beta }}{\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }\mathbf{\Sigma }\mathit{\beta }\text{ subject to }{\mathit{\mu }}^{\top }\mathit{\beta }=1.$$ (6)

The solution to (6) is ${\mathit{\beta }}^{\star }={\mathbf{\Sigma }}^{-1}\mathit{\mu }/{\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$, which can be viewed as a scaled version of the optimal projection direction. Note that the formulation (6) rules out the null hypothesis $\mathit{\mu }=0$ as the solution is not well-defined for $\mathit{\mu }=0$. We want to point out that the optimal projection direction, which maximizes the power of the test $T_{\mathbf{P}}^2$, is derived when alternative hypothesis is true. Under null hypothesis, any choice of projection would work perfectly fine in the sense that type I error rate can be well controlled, see more discussions in Section 3. Hence excluding $\mathit{\mu }=0$ does not bring in any inconvenience.

In high-dimensional regimes, it is well known that consistent estimators cannot be achieved unless additional structural assumptions are imposed. A common assumption is the optimal projection direction ${\mathit{\beta }}^{\star }={\mathbf{\Sigma }}^{-1}\mathit{\mu }/{\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is sparse, i.e., a large portion of elements in ${\mathit{\beta }}^{\star }$ are 0. To encourage a sparse estimator of ${\mathit{\beta }}^{\star }$, we consider the following constrained and regularized quadratic programming,

$$\underset{\mathit{\beta }}{\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }\widehat{\mathbf{\Sigma }}\mathit{\beta }+P_{\lambda }\left(\mathit{\beta }\right)\text{ subject to }{\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1,$$ (7)

where $P_{\lambda }\left(\mathit{\beta }\right)={\sum }_{j=1}^p{P}_{\lambda }\left({\beta }_j\right)$ is some penalty function satisfying the conditions in Supplement S.2.1. Such conditions on $P_{\lambda }\left(t\right)$ are relatively mild and are satisfied for a wide variety of popular penalties (Fan et al. 2020).

Before we proceed to solve the optimization problem (7), we provide a different prospective on estimation of the optimal projection direction without knowing its explicit form ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$. Suppose we want to project data onto a 1-dimensional space. Let $\mathit{\beta }\in {\mathbb{R}}^p$ be the projection vector and $y_i={\mathit{\beta }}^{\top }{\mathbf{x}}_i,i=1,\dots ,n$ be projected sample. Then $\sqrt{n}\bar{y}/s_y$ is the $t$-statistic based on the projected sample, where $\bar{y}$ and $s_y^2$ are the sample mean and sample variance of $y_i$ ’s,

$$\frac{\sqrt{n}\overline{y}}{s_y}=\frac{\sqrt{n}{\beta }^{\top }\overline{x}}{\sqrt{\frac{1}{n-1}{\sum }_{i=1}^n{\beta }^{\top }\left(x_i-\overline{x}\right){\left(x_i-\overline{x}\right)}^{\top }\beta }}=\frac{\sqrt{n}{\beta }^{\top }\overline{x}}{\sqrt{{\beta }^{\top }\widehat{\Sigma }\mathit{\beta }}}.$$

The goal is to find $\mathit{\beta }$ that maximizes the absolute value of the $t$-statistic $\left|\sqrt{n}\bar{y}/s_y\right|$ (i.e., maximize power). Since $\sqrt{n}\bar{y}/s_y$ is an odd function w.r.t. $\mathit{\beta }$, maximizing $\left|\sqrt{n}\bar{y}/s_y\right|$ is the same as maximizing $\sqrt{n}\bar{y}/s_y$,${\mathrm{m}\mathrm{a}\mathrm{x}}_{\mathit{\beta }}\left\{{\mathit{\beta }}^{\top }\stackrel{-}{\mathbf{x}}/\sqrt{{\mathit{\beta }}^{\top }\widehat{\mathbf{\Sigma }}\mathit{\beta }}\right\}$, which is equivalent to

$$\underset{\mathit{\beta }}{\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }\widehat{\mathbf{\Sigma }}\mathit{\beta }\text{ subject to }{\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1,$$ (8)

Therefore, formulation (7) can be viewed as the regularized version of (8), which intends to maximize the test statistic and the penalty term $P_{\lambda }$ is added to avoid overfitting from a machine learning prospective. Another perspective that provides insights on the optimal projection direction is Fisher’s discriminant analysis. Consider a two-sample problem, and of interest is to test whether the two population means are the same. That is, $H_0:{\mathit{\mu }}_1={\mathit{\mu }}_2$. We can treat the hypothesis testing as a classification problem under Fisher’s discriminant analysis framework, which is to find the best linear combination of features that maximizes the ratio of variance between the two populations to the variance within the populations (Fisher 1936). That is to find $\mathit{\beta }$ to maximize the ratio

$$\frac{{\mathrm{V}\mathrm{a}\mathrm{r}}_{\text{between }}}{{\mathrm{V}\mathrm{a}\mathrm{r}}_{\text{within }}}=\frac{{\left({\mathit{\beta }}^{\top }\left({\mathit{\mu }}_2-{\mathit{\mu }}_1\right)\right)}^2}{{\mathit{\beta }}^{\top }\mathbf{\Sigma }\mathit{\beta }}.$$

This ratio measures the signal-to-noise ratio for the class labelling. It can be shown that the maximum ratio occurs when $\mathit{\beta }\propto {\mathbf{\Sigma }}^{-1}\left({\mathit{\mu }}_1-{\mathit{\mu }}_2\right)$, which is exactly the optimal projection direction for the two-sample problem (Huang 2015).

2.3. Non-asymptotic error bounds

In this section, we establish the non-asymptotic error bounds for all stationary points. The nonconvexity of $P_{\lambda }$ brings extra difficulty in solving the constrained and regularized quadratic form (7). Direct computation of the global solution is quite challenging in high-dimensional settings. For instance, Liu et al. (2016) developed a mixed integer linear programming to find a provably global solution, which is quite computationally expensive. Instead of searching for the global solution, many algorithms are developed to efficiently find local solutions with satisfactory theoretical guarantees (Zou and Li 2008, Wang et al. 2013, Loh and Wainwright 2015). We will show that any stationary points of (7) enjoy satisfactory statistical properties and introduce an ADMM with local linear approximation algorithm to obtain a stationary point. $\widehat{\mathit{\beta }}$ is a stationary point of optimization problem (7) if $\widehat{\mathit{\beta }}$ satisfies the following first order condition,

$$⟨\widehat{\mathbf{\Sigma }}\widehat{\mathit{\beta }}+\nabla P_{\lambda }\left(\widehat{\mathit{\beta }}\right),\mathit{\beta }-\widehat{\mathit{\beta }}⟩\ge 0\text{ for all feasible }{\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1,$$ (9)

where $\nabla P_{\lambda }(\cdot )$ denotes the subgradient of $P_{\lambda }$ and $⟨\cdot ,\cdot ⟩$ denotes the inner product of two vectors. The first order condition (9) is a necessary condition for $\widehat{\mathit{\beta }}$ to be a local minimum. In other words, the set of $\widehat{\mathit{\beta }}$ satisfying condition (9) includes all local minimizers as well as the global minimizer.

We further assume the sample covariance matrix $\widehat{\mathbf{\Sigma }}$ satisfies the following restricted strong convexity (RSC) condition,

$${\mathbf{\Delta }}^{\top }\widehat{\mathbf{\Sigma }}\mathbf{\Delta }\ge \nu \parallel \Delta {\parallel }_2^2-\tau \sqrt{\frac{\text{log}p}{n}}\parallel \mathbf{\Delta }{\parallel }_1\text{ for }\mathbf{\Delta }\in {\mathbb{R}}^p,\parallel \mathbf{\Delta }{\parallel }_1\ge 1,$$ (10)

where $\nu >0$ is a strictly positive constant and $\tau \ge 0$ is a nonnegative constant. Though we only impose the RSC condition on the set $\parallel \mathbf{\Delta }{\parallel }_1\ge 1$, it is easy to verify that the RSC condition actually holds for all $\mathbf{\Delta }\in {\mathbb{R}}^p$ due to the quadratic form, see Lemma S.3 in Supplement S.1. In a classical setting where $n>p$, the sample covariance matrix $\widehat{\mathbf{\Sigma }}$ is positive definite and hence one can set $\tau =0$ and $\nu ={\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}\left(\widehat{\mathbf{\Sigma }}\right)$, the minimal eigenvalue of $\widehat{\mathbf{\Sigma }}$. In the high-dimensional regime where $p>n$, the sample covariance matrix $\widehat{\mathbf{\Sigma }}$ is typically semi-positive definite and hence ${\mathbf{\Delta }}^{\top }\widehat{\mathbf{\Sigma }}\mathbf{\Delta }\ge 0$ for any $\mathbf{\Delta }\in {\mathbb{R}}^p$. Then the RSC condition (10) holds trivially for $\left\{\mathbf{\Delta }:\parallel \mathbf{\Delta }{\parallel }_1/\parallel \mathbf{\Delta }{\parallel }_2^2>a\right\}$, where $a=\frac{\alpha }{\tau }\sqrt{\frac{n}{\mathrm{l}\mathrm{o}\mathrm{g}p}}$. As a result, we only require the RSC condition to hold in the set $\left\{\mathbf{\Delta }:\parallel \mathbf{\Delta }{\parallel }_1/\parallel \mathbf{\Delta }{\parallel }_2^2\le a,\parallel \mathbf{\Delta }{\parallel }_1\ge 1\right\}$. Such RSC-type condition is widely adopted to develop the non-asymptotic error bounds in high-dimensional $M$-estimation and is satisfied with high probability if ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$ are i.i.d. sub-Gaussian random variables. See, for example, Loh and Wainwright (2015).

Recall that ${\mathit{\beta }}^{\star }={\mathbf{\Sigma }}^{-1}\mathit{\mu }/{\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ is the scaled optimal projection direction. It is challenging to directly derive the error bounds between $\widehat{\mathit{\beta }}$ and ${\mathit{\beta }}^{\star }$ as ${\mathit{\beta }}^{\star }$ is not necessarily feasible, i.e., ${\mathit{\beta }}^{\star }$ may not satisfy the linear equality constraint ${\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1$. To this end, we introduce the scaled feasible optimal projection direction $\tilde{\mathit{\beta }}={\mathbf{\Sigma }}^{-1}\mathit{\mu }/{\stackrel{-}{\mathbf{x}}}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$, which is the intersection of set of optimal projection direction $\{a\mathit{\theta }:a\in \mathbb{R}\}$ and feasible set $\left\{\mathit{\beta }:{\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1\right\}$. Note that $\tilde{\mathit{\beta }}$ is of the same direction as the optimal projection ${\mathbf{\Sigma }}^{-1}\mathit{\mu }$ but also satisfies the linear constraint ${\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1$. Figure 1 illustrates the relationship between the scaled optimal projection direction ${\mathit{\beta }}^{\star }$, the scaled feasible optimal projection direction $\tilde{\mathit{\beta }}$ and a stationary point $\widehat{\mathit{\beta }}$. We first derive error bounds between $\widehat{\mathit{\beta }}$ and $\tilde{\mathit{\beta }}$ and then error bounds between $\widehat{\mathit{\beta }}$ and ${\mathit{\beta }}^{\star }$ follow the triangle inequality

$${\parallel \widehat{\mathit{\beta }}-{\mathit{\beta }}^{\star }\parallel }_k\le {\parallel \widehat{\mathit{\beta }}-\tilde{\mathit{\beta }}\parallel }_k+{\parallel \tilde{\mathit{\beta }}-{\mathit{\beta }}^{\star }\parallel }_k,k=\mathrm{1,2}.$$ (11)

Figure 1:

Illustration of the scaled optimal projection direction ${\mathit{\beta }}^{\star }$, the scaled feasible optimal projection direction $\tilde{\mathit{\beta }}$ and a stationary point $\widehat{\mathit{\beta }}$.$\tilde{\mathit{\beta }}$ is the intersection of the optimal projection direction and feasible set. Both $\tilde{\mathit{\beta }}$ and $\widehat{\mathit{\beta }}$ lie in the feasible set while $\tilde{\mathit{\beta }}$ and ${\mathit{\beta }}^{\star }$ are optimal projection directions.

We impose the following conditions,

• (C1) ${\mathrm{x}}_1,\dots ,{\mathrm{x}}_n$ are independent and identically distributed sub-Gaussian vectors.

• (C2) The sample covariance matrix $\widehat{\mathbf{\Sigma }}$ satisfies the RSC condition (10) with $3\mu \le 4\nu$.

• (C3) There exists $C_1$,$C_2>0$ such that ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }\ge C_1$,$\parallel \mathit{\theta }{\parallel }_1\le C_2$.

The following theorem states the $L_1$ and $L_2$ error bounds for $\widehat{\mathit{\beta }}-\tilde{\mathit{\beta }}$ and $\widehat{\mathit{\beta }}-{\mathit{\beta }}^{\star }$.

Theorem 1. Suppose conditions (C1)–(C3) hold. Let $\widehat{\mathit{\beta }}$ be any stationary point of the program (7) with $\lambda =M\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}$ for some large constant $M$. If the sample size $n$ satisfies

$$n\ge {\left(\frac{M{C}_2}{\left(1-\delta \right)C_1}\right)}^2\text{log}p,$$ (12)

for some $0<\delta <1$. Then with probability at least $1-c{p}^{-1}$ for some constant $c$, we have

1. $\parallel \widehat{\mathit{\beta }}-\tilde{\mathit{\beta }}{\parallel }_1=O\left(s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}\right)$ and $\parallel \widehat{\mathit{\beta }}-\tilde{\mathit{\beta }}{\parallel }_2=O\left(\sqrt{s\mathrm{l}\mathrm{o}\mathrm{g}p/n}\right)$.

2. ${∥\widehat{\mathit{\beta }}-{\mathit{\beta }}^{\star }∥}_1=O\left(s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}\right)$ and ${∥\widehat{\mathit{\beta }}-{\mathit{\beta }}^{\star }∥}_2=O\left(\sqrt{s\mathrm{l}\mathrm{o}\mathrm{g}p/n}\right)$.

where $s={∥{\mathit{\beta }}^{\star }∥}_0$ is the number of nonzero elements in ${\mathit{\beta }}^{\star }$.

Remark. The error bounds in Theorem 1 hold for all stationary points satisfying condition (9). In other words, any local estimator is guaranteed with desirable statistical accuracy. Condition (12) describes the relationship between sample size $n$ and dimension $p$ and is satisfied if $\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}=o\left(1\right)$. If $\sqrt{s\mathrm{l}\mathrm{o}\mathrm{g}p/n}\to 0$ as $n,p\to \infty$, then we know $\widehat{\mathit{\beta }}$ is a consistent estimator of ${\mathit{\beta }}^{\star }$. In other words, in order to obtain a consistent estimator, we require the optimal projection direction to be sparse. Chernozhukov et al. (2017) established a central limit theorem for high-dimensional data with Gaussian and bootstrap approximations, which can be applied to a one-sample hypothesis testing problem and requires $\mathrm{l}\mathrm{o}\mathrm{g}p=o\left(n^{1/7}\right)$. In contrast, we impose a weaker condition on the dimension $\mathrm{l}\mathrm{o}\mathrm{g}p=o\left(n/s^2\right)$ on the proposed projection test. Conditions (C1)–(C2) are commonly adopted in high-dimensional statistics. Condition (C3) is posited to ensure the estimation error of covariance matrix is not enlarged by $\mathit{\theta }$ and passed along to the estimation of ${\mathit{\beta }}^{\star }$. Note that $\parallel \widehat{\mathbf{\Sigma }}\mathit{\theta }-\mathit{\mu }{\parallel }_{\infty }=\parallel \widehat{\mathbf{\Sigma }}\mathit{\theta }-\mathbf{\Sigma }\mathit{\theta }{\parallel }_{\infty }\le \parallel \widehat{\mathbf{\Sigma }}-\mathbf{\Sigma }{\parallel }_{\mathrm{m}\mathrm{a}\mathrm{x}}\cdot \parallel \mathit{\theta }{\parallel }_1$. A diverging $\parallel \mathit{\theta }{\parallel }_1$ would amplify the estimation error of $\widehat{\mathbf{\Sigma }}$.

Geometry of $\widehat{\mathit{\beta }}$.

Notice that the projection test is scale invariant with respect to $\widehat{\mathit{\beta }}$, i.e., $\widehat{\mathit{\beta }}$ and $a\widehat{\mathit{\beta }}$ have exact the same testing performance for any $a>0$. To eliminate scale effect, we also measure the closeness between $\widehat{\mathit{\beta }}$ and ${\mathit{\beta }}^{\star }$ using cosine similarity. Formally, the cosine similarity between two vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as

$$\text{cos}⟨\mathbf{u},\mathbf{v}⟩=\frac{{\mathbf{u}}^{\top }\mathbf{v}}{\parallel \mathbf{u}{\parallel }_2\parallel \mathbf{v}{\parallel }_2}.$$

If $\mathbf{u}$ and $\mathbf{v}$ are of the same direction ($\mathbf{u}=a\mathbf{v}$ for some $a>0$), then cosine similarity equals 1. In general, if the cosine similarity is closer to 1, then the directions of the two vectors are closer to each other. We that the cosine similarity between ${\mathit{\beta }}^{\star }$ and $\widehat{\mathit{\beta }}$ converges to 1.

Corollary 1. Suppose conditions (C1)–(C3) hold. Let $\widehat{\mathit{\beta }}$ be a stationary point of the program (7) with $\lambda =M\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}$ for some large constant $M$. Then we have

$$\mathrm{c}\mathrm{o}\mathrm{s}⟨\widehat{\mathit{\beta }},\tilde{\mathit{\beta }}⟩=\mathrm{c}\mathrm{o}\mathrm{s}⟨\widehat{\mathit{\beta }},{\mathit{\beta }}^{\star }⟩\to 1.$$

2.4. ADMM with local linear approximation

The error bounds in Theorem 1 indicate that any stationary point has satisfactory statistical accuracy. Instead of finding the global solution to the possibly nonconvex optimization problem (7), we only need to develop an algorithm which can find a stationary point that satisfies the first order condition (9). We propose to solve the constrained and regularized quadratic form using the local linear approximation (LLA) technique. The LLA algorithm was firstly proposed in Zou and Li (2008) to deal with nonconvex penalty in generalized linear models. The idea of LLA is to approximate the nonconvex penalty by its first order expansion, which is a convex function. It turns out that we only need to solve a convex problem iteratively until certain convergence criterion is reached. A similar idea is proposed in Wang et al. (2013) which approximates the nonconvex penalty by its tight upper bound. Here we extend the LLA algorithm to solve the constrained and regularized quadratic programming (7).

Suppose ${\beta }_{j0}$ is close to ${\beta }_j$, ignoring constants that do not involve ${\beta }_j$, the penalty function can be approximated by

$$P_{\lambda }\left(\left|{\beta }_j\right|\right)\approx P_{\lambda }^{\prime }\left(\left|{\beta }_{j0}\right|\right)\left(\left|{\beta }_j\right|-\left|{\beta }_{j0}\right|\right).$$

Given the $k$-th iteration ${\widehat{\mathit{\beta }}}^{\left(k\right)}={\left({\widehat{\beta }}_1^{\left(k\right)},\dots ,{\widehat{\beta }}_p^{\left(k\right)}\right)}^{\top }$, the optimization problem (7) can be updated as follows

$${\widehat{\beta }}^{\left(k+1\right)}=\underset{{\overline{x}}^{\top }\beta =1}{\text{arg min}}\frac{1}{2}{\beta }^{\top }\widehat{\Sigma }\mathit{\beta }+\sum _{j=1}^p{\omega }_j^{\left(k\right)}\left|{\beta }_j\right|$$ (13)

where ${\omega }_j^{\left(k\right)}=P_{\lambda }^{\prime }\left(\left|{\beta }_j^{\left(k\right)}\right|\right)$. We iteratively solve (13) until ${\widehat{\mathit{\beta }}}^{\left(k\right)}$ converges. The LLA algorithm is an instance of Majorization-Minimization (MM) algorithms and is guaranteed to converge to a stationary point (Zou and Li 2008). With the local linear approximation, we know there is a unique solution to the constrained and regularized quadratic form and this unique solution is a stationary point. Hence the theoretical results developed in Section 2.3 hold for the estimator given by the LLA algorithm. The theoretical properties of LLA estimators are systematically studied in Fan et al. (2014) in high dimensions and the LLA algorithm can find the oracle estimator with high probability after one iteration. To solve the convex optimization problem after local linear approximation (13), we adopt the alternating direction method of multipliers (ADMM) algorithm which naturally handles the linear equality constraint ${\stackrel{-}{\mathbf{x}}}^{\top }\mathit{\beta }=1$ (Boyd et al. 2011, Fang et al. 2015). The penalisation parameter $\lambda$ is chosen via high-dimensional BIC (Wang et al. 2013). The details of ADMM algorithm and choice of $\lambda$ are summarized in Supplement S.4 and S.5, respectively.

3. Data Splitting and Power Enhancement

Ideally, one would like to use the full sample to estimate the optimal projection direction so the estimated projection is the most accurate. Then we project the full sample to a 1-dimensional space and perform a test based on the projected sample. However, the limiting distribution of the test statistic is challenging to derive as the estimated projection direction and the data to be projected are dependent. In this section, we introduce a new projection test via an online framework. Before we present the full methodology of the online-style projection test, we first proceed with a data-splitting procedure.

3.1. A data-splitting procedure

Data-splitting technique has been widely used in high-dimensional statistical inference, where one can use a subset of data to learn the underlying model and use the remaining data to conduct statistical inferences. Given a dataset $𝒟=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_n\right\}$, we partition the full dataset into two disjoint subsets ${𝒟}_1=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{n_1}\right\}$ and ${𝒟}_2=\left\{{\mathbf{x}}_{n_1+1},\dots ,{\mathbf{x}}_n\right\}$. We use the first subset ${𝒟}_1$ to estimate the optimal projection direction ${\mathit{\beta }}^{\star }$ and use the second dataset ${𝒟}_2$ to perform the projection test. More specifically, let ${\stackrel{-}{\mathbf{x}}}_1$ and ${\widehat{\mathbf{\Sigma }}}_1$ be the sample mean and the sample covariance matrix estimated from ${𝒟}_1$. The projection direction can be estimated by solving the following constrained and penalized quadratic form introduced in Section 2.2,

$$\widehat{\mathit{\beta }}=\underset{{\stackrel{-}{\mathbf{x}}}_1^{\top }\mathit{\beta }=1}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }{\widehat{\mathbf{\Sigma }}}_1\mathit{\beta }+P_{\lambda }\left(\mathit{\beta }\right).$$ (14)

Then we project data in ${𝒟}_2$ onto a 1-dimensional space by taking inner product of $\widehat{\mathit{\beta }}$ and ${\mathbf{x}}_i$, i.e., $y_i={\mathbf{x}}_i^{\top }\widehat{\mathit{\beta }}$ for $i=n_1+1,\dots ,n$. Note that the estimated projection direction $\widehat{\mathit{\beta }}$ is independent of ${𝒟}_2$, which is the key benefit of the data-splitting procedure. Conditional on $\widehat{\mathit{\beta }}$,$y_{n_1+1},\dots ,y_n$ are independent and identically distributed with mean ${\widehat{\mathit{\beta }}}^{\top }\mathit{\mu }$ and variance ${\widehat{\mathit{\beta }}}^{\top }\mathbf{\Sigma }\widehat{\mathit{\beta }}$. Now we reduce the dimensionality from $p$ to 1, and we can simply apply the one-sample $t$-test to the projected data $\left\{y_{n_1+1},\dots ,y_n\right\}$. Let

$$\overline{y}=\frac{1}{n_2}\sum _{i=n_1+1}^n{y}_i,s_y^2=\frac{1}{n_2-1}\sum _{i=n_1+1}^n{\left(y_i-\overline{y}\right)}^2,$$

where $n_2=n-n_1$. Clearly, the test statistic is $T_y=\sqrt{n_2}\bar{y}/s_y$ asymptotically follows $N\left(\mathrm{0,1}\right)$ under $H_0$ and we reject null hypothesis whenever $\left|T_y\right|>z_{\alpha /2}$, where $z_{\alpha /2}$ is the upper $\alpha /2$ quantile of the standard normal distribution. If ${\mathbf{x}}_i$ ’s are normally distributed, then $T_y$ follows an exact $t$ distribution with degree of freedom $n_2-1$. We refer to this test as data-splitting projection test (DSPT) and summarize it in Algorithm 1. Thanks to the data-splitting procedure, the estimated $\widehat{\mathit{\beta }}$ is independent of the remaining subset ${𝒟}_2$. As a result, we are able to achieve an exact $t$-test and hence control the type I error rate. However, the performance of the data-splitting procedure may not be satisfactory as data in ${𝒟}_1$ is not utilized when conducting the $t$-test, leading to loss of power. Next, we derive the asymptotic power function for the DSPT defined as ${\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\mu }\right)=\mathrm{P}\mathrm{r}\left(\left|T_y\right|>z_{\alpha /2}\right)$.

Theorem 2. Suppose conditions (C1)–(C3) hold. Let $\widehat{\mathit{\beta }}$ be a stationary point of program (14) with $\lambda =M\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n_1}$ for some large constant $M$. If $\sqrt{s\mathrm{l}\mathrm{o}\mathrm{g}p/n_1}=o\left(1\right)$, then we have

$${\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\mu }\right)-\Phi \left(-z_{\alpha /2}+\sqrt{n_2\zeta }\right)\to 0\mathit{\text{ as }}{n}_1,n_2\to \infty ,$$

where $\zeta ={\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ and $\Phi (\cdot )$ is the cdf of the standard normal distribution.

The quantity $\zeta ={\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ can be interpreted as the signal strength in alternative hypothesis. As long as $n_2\zeta \to \infty$, the DSPT has asymptotic power approaching 1. If the full sample of size $n$ is used to perform the test, one would expect the power function to be $\Phi \left(-z_{\alpha /2}+\sqrt{n\zeta }\right)$. Theorem 2 clearly indicates power loss of the DSPT due to the data-splitting procedure - only observations in ${𝒟}_2$ are used to test the hypothesis. Though the DSPT is not optimal in terms of testing power, it is still more powerful than the quadratic-form tests which ignores the dependence among variables, as long as $\sqrt{n_2/n}>0.5$ (Huang 2015). In practice, it is recommended to set $n_1=\lfloor \tau n\rfloor$ with $\tau \in (0.4,0.6)$ based on numerical studies. Huang (2015) proposed a ridge-type estimator $(\widehat{\mathbf{\Sigma }}+\lambda \mathbf{I}{)}^{-1}\stackrel{-}{\mathbf{X}}$ to estimate the projection direction, which is not guaranteed to be consistent in high dimensions and its theoretical properties require $n$ and $p$ of the same order. Its power analysis is relied on the consistency of the ridge-type estimator. We propose to estimate the optimal projection via a constrained and regularized quadratic program. The resulting estimator is consistent provided that $\sqrt{s\mathrm{l}\mathrm{o}\mathrm{g}p/n}=o\left(1\right)$. We further derive the asymptotic power function ${\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\mu }\right)$ based on this condition.

3.2. An online-style data splitting

The data-splitting procedure introduced in the previous section achieves an exact $t$-test but suffers from power loss. In this subsection, we propose an online framework for projection test to enhance the testing power while retaining the type I error rate. Imagine that observations arrive one by one in chronological order, and we repeat a projection-estimation procedure whenever a new observation arrives. More specifically, we obtain an estimator of the optimal projection based on current observations, and when a new observation arrives we first project the new observation and then update the estimation by including the new observation. Suppose we observe ${𝒟}_t=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_t\right\}$ at time $t$ and we first use the $t$ observations to estimate the optimal projection direction via the penalized and constrained quadratic form introduced in Section 2.2,

$${\widehat{\mathit{\beta }}}_t=\underset{{\stackrel{-}{\mathbf{x}}}_t^{\top }\mathit{\beta }=1}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }{\widehat{\mathbf{\Sigma }}}_t\mathit{\beta }+P_{\lambda }\left(\mathit{\beta }\right),$$ (15)

where ${\stackrel{-}{\mathbf{x}}}_t$ and ${\widehat{\mathbf{\Sigma }}}_t$ are sample mean and sample covariance estimated from ${𝒟}_t$. When a new observation ${\mathbf{x}}_{t+1}$ arrives, we take the following Projection-Estimation steps:

• Projection: project the new observation using the current estimator ${\widehat{\mathit{\beta }}}_t$, i.e., $y_{t+1}={\mathbf{x}}_{t+1}^{\top }{\widehat{\mathit{\beta }}}_t$.

• Estimation: update the estimation of projection direction by incorporating the information of the new observation ${\mathbf{x}}_{t+1}$. That is,

  $${\widehat{\mathit{\beta }}}_{t+1}=\underset{{\stackrel{-}{\mathbf{x}}}_{t+1}^{\top }\mathit{\beta }=1}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }{\widehat{\mathbf{\Sigma }}}_{t+1}\mathit{\beta }+P_{\lambda }\left(\mathit{\beta }\right),$$

  where ${\stackrel{-}{\mathbf{x}}}_{t+1}$ and ${\widehat{\mathbf{\Sigma }}}_{t+1}$ are sample mean and sample covariance estimated from ${𝒟}_{t+1}=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{t+1}\right\}$. Given an integer $k_n\ll n$, we obtain an initial estimator ${\widehat{\mathit{\beta }}}_{k_n}$ based on the first $k_n$ observations in ${𝒟}_{k_n}$. Then we repeat the Projection-Estimation steps until the last observation arrives. We require $k_n=o\left(n\right)$ and $k_n\ge 2$ since we need at least two observations to estimate the covariance matrix. As a result, we obtain a sequence of projected sample $\left\{y_{k_n+1},\dots ,y_n\right\}$ of sample size $n-k_n+1$ and based on which we can carry out the one-sample $t$-test. Let

  $$T_y=\frac{\sqrt{n-k_n}\bar{y}}{s_y},$$ (16)

  where $\bar{y}$ and $s_y^2$ are the sample mean and the sample variance of $\left\{y_{k_n+1},\dots ,y_n\right\}$. Later in this section we will show the test statistic $T_y$ converges to $N\left(\mathrm{0,1}\right)$ under $H_0$. As a result, we can use $z_{\alpha /2}$ as the critical value and reject $H_0$ whenever $\left|T_y\right|>z_{\alpha /2}$. We refer to this test as Online-style Projection Test One by one (OPT-O) as we update ${\widehat{\mathit{\beta }}}_t$ whenever a new observation arrives. The details of the OPT-O are also summarized in Algorithm 2.

Under the one-by-one framework of OPT-O, one needs to solve a constrained and penalized quadratic programming, which can be nonconvex, whenever a new observation arrives. As a result, the OPT-O test can be computationally expensive especially when $n$ is large. To reduce the computational cost, we also propose a mini-batch version the OPT-O test. That is, we update the estimated projection direction only when a batch of observations of size $b$ arrive. Suppose we obtain an estimator ${\widehat{\mathit{\beta }}}_t$ at time $t$ based on ${𝒟}_t=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_t\right\}$. When the next $b$ observations ${\mathbf{x}}_{t+1},\dots ,{\mathbf{x}}_{t+b}$ arrive, we first project the $b$ observations to a 1-dimensional space by multiplying ${\widehat{\mathit{\beta }}}_t$, i.e., $y_{t+1}={\mathbf{x}}_{t+1}^{\top }{\widehat{\mathit{\beta }}}_t,\dots$,$y_{t+b}={\mathbf{x}}_{t+b}^{\top }{\widehat{\mathit{\beta }}}_t$. Then we update the estimation of projection direction by including the additional $b$ observations,

$${\widehat{\mathit{\beta }}}_{t+b}=\underset{{\stackrel{-}{\mathbf{x}}}_{t+b}^{\top }\mathit{\beta }=1}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\mathit{\beta }}^{\top }{\widehat{\mathbf{\Sigma }}}_{t+b}\mathit{\beta }+P_{\lambda }\left(\mathit{\beta }\right),$$ (17)

where ${\stackrel{-}{\mathbf{x}}}_{t+b}$ and ${\widehat{\mathbf{\Sigma }}}_{t+b}$ are sample mean and sample covariance matrix computed from the first $t+b$ observations ${𝒟}_{t+b}=\left\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{t+b}\right\}$. We repeat this procedure until the last batch arrives. Note that the size of the last batch can be small than $b$. Similar to the OPT-O, we reject $H_0$ if $\left|T_y\right|>z_{\alpha /2}$ where the test statistic $T_y$ is defined in (16). The details of the mini-batch version are summarized in Algorithm 3 and the corresponding test is referred to as Online-style Projection Test - mini Batch (OPT-B). Note that the data-splitting projection test can be regarded as a special case of the OPT-B where there is only one single batch. In practice, we recommend the OPT-O test (i.e., $b=1$) to maximize the power. If computing power is a concern, one should use a small value of $b$ to ensure high power provided that the computational cost is affordable. Section S.8 studies how the batch size $b$ affects the power of projection tests. Another way to reduce computational time is through warm starting. For both OPT-O and OPT-B, one can leverage the solution ${\widehat{\mathit{\beta }}}_t$ from the previous step as the initial solution in the next iteration to obtain ${\widehat{\mathit{\beta }}}_{t+1}$ or ${\widehat{\mathit{\beta }}}_{t+b}$. Such a warm start helps shorten time to find the optimal solution.

Algorithms 2 and 3 introduce a general online framework to construct projection test. Instead of using the constrained and penalized quadratic programming to estimate the projection direction, any appropriate estimator can be applied. For instance, one can use the ridge-type estimator proposed in Huang (2015).

3.3. Asymptotic distribution of online-style projection test

In this subsection, we establish the limiting distributions for the proposed online-style projection test. Under the online framework, we obtain a sequence of projected sample $y_{k_n+1},\dots$,$y_n$ one by one or via mini-batch. Then we need to make an inference regarding $H_0$ based on the projected sample. One challenge is that the projected sample $y_{k_n+1},\dots$,$y_n$ are no longer independent from each other. A key observation is that we can construct a sequence of martingale differences from the projected sample. Let ${𝓏}_t$ be the centralized version of $y_t$, i.e., ${𝓏}_t={\left({\mathbf{x}}_t-\mathit{\mu }\right)}^{\top }{\widehat{\mathit{\beta }}}_{t-1}$ for $t=k_n+1,\dots ,n$ and ${𝓏}_t$ has mean 0. The following lemma states that the sequence of ${𝓏}_t$ ’s is a martingale difference and the proof can be found in Section S.3.

Lemma 1. The sequence ${𝓏}_{k_n+1},\dots ,{𝓏}_n$ is a martingale difference with respect to the $\sigma$-field generated by $\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots \right\}$ under both the one-by-one and the mini-batch framework.

Lemma 1 also implies that $\left\{y_{k_n+1},\dots ,y_n\right\}$ is a martingale difference under $H_0$. With this observation, we are ready to establish the central limit theorem for the test statistic. To this end, let $V_{t+1}^2$ be the conditional variance of $z_{t+1}$ given ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_t$,

$$V_{t+1}^2=\mathrm{E}\left({𝓏}_{t+1}^2\mid {\mathbf{x}}_1,\dots ,{\mathbf{x}}_t\right)={\widehat{\mathit{\beta }}}_t^{\top }\mathbf{\Sigma }{\widehat{\mathit{\beta }}}_t,$$

and let $\overline{𝓏}$ and $s_{𝓏}^2$ be the sample mean and sample variance of ${𝓏}_{k_n+1},\dots ,{𝓏}_n$,

$$\overline{𝓏}=\frac{1}{n-k_n}\sum _{t=k_n+1}^n{𝓏}_t\hspace{1em}\text{and}\hspace{1em}{s}_{𝓏}^2=\frac{1}{n-k_n-1}\sum _{t=k_n+1}^n{𝓏}_t^2.$$

The following theorem establishes the asymptotic normality for $T_{𝓏}$.

Theorem 3. (Normality) Assume $k_n=o\left(n\right)$ and there exists $a_{n,p}>0$ that depends on $n,p$ such that ${\left(n-k_n\right)}^{-1}{a}_{n,p}{\sum }_{t=k_n+1}^n{V}_t^2\stackrel{p}{\to }{V}^2$, where $V^2$ is some almost surely finite random variable satisfying $\mathrm{P}\mathrm{r}\left(V^2>0\right)=1$, then we have

$$T_{𝓏}=\sqrt{n-k_n}\overline{𝓏}/s_{𝓏}\stackrel{d}{\to }N\left(\mathrm{0,1}\right).$$

In particular, under $H_0$,$T_y=\sqrt{n-k_n}\bar{y}/s_y\stackrel{d}{\to }N\left(\mathrm{0,1}\right)$.

We also empirically verify the asymptotic normality of proposed online-style tests under both $H_0$ and $H_1$ in Section S.6.

Remark. In order to establish the central limit theorem for the proposed online-style test statistic, we require the sample average of conditional variance $V_k^2$ to be nondegenerate. Note that for each $t$, the estimated projection direction satisfies the linear constraint ${\stackrel{-}{\mathbf{x}}}_t^{\top }{\widehat{\mathit{\beta }}}_t=1$ and hence ${∥{\stackrel{-}{\mathbf{x}}}_t∥}_{\infty }{∥{\widehat{\mathit{\beta }}}_t∥}_1\ge 1$. Since ${∥{\stackrel{-}{\mathbf{x}}}_t∥}_{\infty }\le \sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/t}$ with high probability, we know ${∥{\widehat{\mathit{\beta }}}_t∥}_1\ge \sqrt{t/\mathrm{l}\mathrm{o}\mathrm{g}p}$ which diverges in the high-dimensional settings, so is the conditional variance $V_t^2$. With a proper choice of $a_{n,p}$, we want the sample average of conditional variance (up to the factor $a_{n,p}$) to converge to some nonzero random variable. In general, one can set $a_{n,p}$ to be of the order $O\left(\left(n-k_n\right){\left({\sum }_{t=k_n+1}^n{V}_t^2\right)}^{-1}\right)$. In practice, we do not need to derive the explicit form of $a_{n,p}$ as the test statistic is scale invariant and $a_{n,p}$ will be cancelled out. Under the alternative hypothesis, we can show that the sample average of conditional variance $V_t^2$ converges to ${\zeta }^{-1}$ where $\zeta ={\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ and hence one can set $a_{n,p}=O\left(\zeta \right)$. Thanks to the linear constraint ${\stackrel{-}{\mathbf{x}}}_t^{\top }\mathit{\beta }=1$, the resulting estimator ${\widehat{\mathit{\beta }}}_t$ is bounded away from $0$ and the conditional variance $V_t^2$ is not degenerate. Without the linear constraint, we cannot guarantee that ${\widehat{\mathit{\beta }}}_t$ is bounded away from $0$ and hence the central limit theorem may not hold.

Here is an example to illustrate the convergence of $V_t^2$ with the following conditions imposed on the first moment and second moment of ${\widehat{\mathit{\beta }}}_t$. Assume there exists $\mathit{\beta },\underset{\_}{\mathbf{\Sigma }}$, and sequences $r_n$ such that $\mathrm{E}\left({\mathbf{a}}^{\top }{\widehat{\mathit{\beta }}}_t\right)-{\mathbf{a}}^{\top }\underset{\_}{\mathit{\beta }}\to 0$ and $r_n^2\mathrm{V}\mathrm{a}\mathrm{r}\left({\mathbf{a}}^{\top }{\widehat{\mathit{\beta }}}_t\right)-{\mathbf{a}}^{\top }\underset{\_}{\mathbf{\Sigma }}\mathbf{a}\to 0$ for all $\parallel \mathbf{a}{\parallel }_2=1$. Therefore, $\mathrm{E}\left({\widehat{\mathit{\beta }}}_t\right)-\underset{\_}{\mathit{\beta }}\to 0$ and $r_n^2\mathrm{V}\mathrm{a}\mathrm{r}\left({\widehat{\mathit{\beta }}}_t\right)-\underset{\_}{\mathbf{\Sigma }}\to 0$. Note that

$$\frac{1}{n-k_n}\sum _{t=k_{n+1}}^n{V}_t^2=\frac{1}{n-k_n}\sum _{t=k_{n+1}}^n{\widehat{\mathit{\beta }}}_{t-1}^{\top }\mathbf{\Sigma }{\widehat{\mathit{\beta }}}_{t-1}=\text{trace}\left(\mathbf{\Sigma }\frac{1}{n-k_n}\sum _{t=k_{n+1}}^n{\widehat{\mathit{\beta }}}_{t-1}{\widehat{\mathit{\beta }}}_{t-1}^{\top }\right).$$

Further assume $\frac{1}{n-k_n}{\sum }_{t=k_{n+1}}^n{\widehat{\mathit{\beta }}}_{t-1}{\widehat{\mathit{\beta }}}_{t-1}^{\top }$ converges to its expectation $\mathrm{E}\left({\widehat{\mathit{\beta }}}_t{\widehat{\mathit{\beta }}}_t^{\top }\right)$, where

$$\mathrm{E}\left({\widehat{\mathit{\beta }}}_t{\widehat{\mathit{\beta }}}_t^{\top }\right)-r_n^{-2}\underset{\_}{\mathbf{\Sigma }}+{\underset{\_}{\mathit{\beta }}\underset{\_}{\mathit{\beta }}}^{\top }=\mathrm{V}\mathrm{a}\mathrm{r}\left({\widehat{\mathit{\beta }}}_t\right)+\mathrm{E}\left({\widehat{\mathit{\beta }}}_t\right)\mathrm{E}{\left({\widehat{\mathit{\beta }}}_t\right)}^{\top }-r_n^{-2}\underset{\_}{\mathbf{\Sigma }}+{\underset{\_}{\mathit{\beta }}\underset{\_}{\mathit{\beta }}}^{\top }\to 0.$$

As a result, the sample average of conditional variance can be approximated by

$$\frac{1}{n-k_n}\sum _{t=k_{n+1}}^n{V}_t^2=r_n^{-2}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{e}\left(\mathbf{\Sigma }\underset{\_}{\mathbf{\Sigma }}\right)+{\underset{\_}{\mathit{\beta }}}^{\top }\mathbf{\Sigma }\underset{\_}{\mathit{\beta }.}$$

Under $H_0$, we have $\underset{\_}{\mathit{\beta }}=0$ and hence one can take $a_{n,p}=r_n^2$. Under $H_1$, we have $r_n=O\left(1\right)$ and hence one can take $a_{n,p}=1$.

Theorem 3 establishes the asymptotic normality of the test statistic $T_y$ under both null and alternative hypotheses assuming the sample average of conditional variances (up to some factor $\left.a_{n,p}\right)$ converges. This theorem justifies the usage of ${𝓏}_{\alpha /2}$ as the critical value and we reject the null hypothesis if and only if $\left|T_y\right|\ge {𝓏}_{\alpha /2}$. A bootstrap version of projection test is discussed in Supplement S.7 where the critical value is chosen based on bootstrap samples. Under conditions (C1)–(C3), we can show $V_t^2-{\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }\to 0$ as $t\to \infty$, based on which we can derive the asymptotic power function ${\beta }_{\text{opto} }^{\alpha }\left(\mathit{\mu }\right)=\mathrm{P}\mathrm{r}\left(\left|T_y\right|>{𝓏}_{\alpha /2}\right)$ for the OPT-O test. The following theorem establishes the asymptotic power function for the OPT-O test.

Theorem 4. Suppose conditions (C1)–(C3) hold. Let ${\widehat{\mathit{\beta }}}_t$ be a stationary point of program (15) with $\lambda =Ms\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/t}$. Assume that $s\mathrm{l}\mathrm{o}\mathrm{g}p/\sqrt{n}=o\left(1\right)$ and $k_n=o\left(n\right)$, we have

1. (Normality under alternative) $T_{𝓏}=\sqrt{n-k_n}\overline{𝓏}/s_{𝓏}\stackrel{d}{\to }N\left(\mathrm{0,1}\right)$.

2. (Power function) ${\beta }_{\mathrm{o}\mathrm{p}\mathrm{t}\mathrm{o}}^{\alpha }\left(\mathit{\mu }\right)-\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{\left(n-k_n\right)\zeta }\right)\to 0$ as $n\to \infty$.

Theorem 4 establishes the normality of the proposed OPT-O test under the alternative, and based on which we derive the asymptotic power function. The asymptotic power function ${\beta }_{\text{opto} }^{\alpha }\left(\mathit{\mu }\right)$ in Theorem 4 holds with any choice of $k_n=o\left(n\right)$. In practice, one may set $k_n$ be a fixed constant or $k_n=n^{\tau }$ for some $\tau \in \left(\mathrm{0,1}\right)$, see Section 3.4 for a numerical study on how to select $k_n$. For the mini-batch version test OPT-B, it is easy to verify that $\left\{{𝓏}_1,{𝓏}_2,\dots \right\}$ is also a martingale difference and thus the asymptotic normality and power function in Theorem 4 also hold for the OPT-B.

Remark The power function ${\beta }_{\text{opto} }^{\alpha }\left(\mathit{\mu }\right)$ indicates that ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$ can be regarded as the signal in the alternative. The larger ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$, the easier to reject the null. If $n{\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }\to \infty$, then we have asymptotic power 1 to reject the null. For the sum-of-square-type of tests, which ignore the dependence among the variables, have asymptotic power

$$\left.{\beta }_{\mathrm{S}\mathrm{S}}^{\alpha }\left(\mathit{\mu }\right)=\Phi \left(-{𝓏}_{\alpha }+n\parallel \mathit{\mu }{\parallel }_2^2/\sqrt{2\mathrm{t}\mathrm{r}\left({\mathbf{\Sigma }}^2\right.}\right)\right),$$

under the assumption that ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }=o\left(n\mathrm{t}\mathrm{r}\left({\mathbf{\Sigma }}^2\right)\right)$. Since $k_n=o\left(n\right)$, we have ${\beta }_{\text{opto} }^{\alpha }\left(\mathit{\mu }\right)>{\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\mu }\right)$. The OPT-O test improves the performance of the DSPT in the following two ways: (1) the OPT-O test keeps including new observations for the estimation of ${\mathit{\beta }}^{\star }$ and hence the estimator of ${\mathit{\beta }}^{\star }$ gets more and more accurate as more observations arrive. (2) Fewer observations are discarded for the OPT-O test compared with the DSPT when performing the test. For the OPT-O, only the first $k_n=o\left(n\right)$ observations are discarded while $\lfloor \tau n\rfloor =O\left(n\right)$ observations are discarded for the DSPT. Let us further examine the technical conditions for the asymptotic power. For the DSPT test, we require $s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n_1}=o\left(1\right)$ to control the error bounds of the estimated projection direction. In order to have high power, the sample size used for the test $n_2=n-n_1$ cannot be too small. In particular, if $\sqrt{n_2/n}>0.5$, then it is more powerful than the quadratic-form tests such as the CQ test. As a result, it is assumed that $n_1$ and $n_2$ are the same order of $n$ (i.e., $\left.n_1=O\left(n\right),n_2=O\left(n\right)\right)$ and $s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}=o\left(1\right)$ (hence $s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n_1}=o\left(1\right)$). The same condition on dimension $s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}=o\left(1\right)$ is needed for the OPT-O, but it allows the sample size discarded for the test is small, i.e., $k_n=o\left(n\right)$. As a result, more observations can be used to perform the test and has higher asymptotic power than the DSPT. To summarize, under the same condition $s\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}=o\left(1\right)$, the OPT-O is more powerful than the DSPT. We can easily translate the asymptotic power function with local alternative $\mathit{\mu }=\mathit{\delta }/\sqrt{n}$ for some fixed vector $\mathit{\delta }$. Given that $n_1=\lfloor \tau n\rfloor$ and $k_n=o\left(n\right)$, the local asymptotic power functions for the DSPT and the OPT-O are

$${\beta }_{\mathrm{o}\mathrm{p}\mathrm{t}\mathrm{o}}^{\alpha }\left(\mathit{\delta }\right)=\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{\left(1-o\left(1\right)\right)\eta }\right),$$

$${\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\delta }\right)=\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{\tau \eta }\right),$$

where $\eta ={\mathit{\delta }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\delta }$ and $0<\tau <1$. Clearly, for local alternatives, we still have ${\beta }_{\text{opto} }^{\alpha }\left(\mathit{\delta }\right)>{\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\delta }\right)$. To evaluate the efficiency of the proposed OPT-O and DSPT, we introduce the oracle projection test which utilizes the full sample with known optimal projection direction. Hence the asymptotic power for the oracle projection test is

$${\beta }_{\text{oracle}}^{\alpha }\left(\mathit{\mu }\right)-\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{n\zeta }\right)\to 0\mathit{\text{as}}n\to \infty .$$

Here we present an asymptotic power comparison between the proposed OPT-O, DSPT and the oracle projection test with $n=\mathrm{40,160}$,$n_1=n/2$,$k_n=⌊n^{0.6}⌋$ and different signal strength $\zeta ={\mathit{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\mathit{\mu }$. Figure 2 indicates that there is certain power loss due to data-splitting compared with the oracle test. The online-style test improves the power of the DSPT. Overall we have ${\beta }_{\text{oracle} }^{\alpha }\left(\mathit{\mu }\right)\ge {\beta }_{\text{opto} }^{\alpha }\left(\mathit{\mu }\right)\ge {\beta }_{\mathrm{d}\mathrm{s}}^{\alpha }\left(\mathit{\mu }\right)$.

Figure 2:

Asymptotic power functions of the OPT-O, the DSPT and the oracle projection test. The oracle power function is based on the full sample with known optimal direction.

3.4. Choice of ${\mathit{k}}_{\mathit{n}}$

The proposed online-style projection test involves choosing the parameter $k_n$ such that we use the first $k_n$ observations to obtain an initial estimator of the optimal projection direction. According to Theorem 4, any choice of $k_n=o\left(n\right)$ would result in testing power of order $\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{\left(n-k_n\right)\zeta }\right)$. Clearly, $k_n$ should not be too large since the first $k_n$ observations are not utilized when performing the online-style projection test. A large $k_n$ may lead to a significant loss in power. To maximize $\Phi \left(-{𝓏}_{\alpha /2}+\sqrt{\left(n-k_n\right)\zeta }\right)$ with respect to $k_n$, one may set $k_n=2$ (since we need at least two observations to estimate the covariance matrix). However, a small $k_n$ results in an inaccurate initial estimator and consequently affect the power. In what follows we conduct numerical studies to investigate how $k_n$ affect the testing power. Assume $k_n=n^{\tau }$ (so $\left.k_n=o\left(n\right)\right)$ with $\tau =(\mathrm{0.2,0.25},\dots ,0.95)$. We set $(n,p,c)=\left(\mathrm{100,1600,0.25}\right)$ and $\left(\mathrm{40,1600,0.5}\right)$ for both autocorrelation and compound symmetry covariance structures with $\rho \in \left\{\mathrm{0.25,0.50,0.75,0.95}\right\}$. Figure 3 depicts the testing power against the choice of $\tau$, where the upper two panels are for the autocorrelation structure and the lower two panels are for the compound symmetry structure. Figure 3 shows that the power of the test is almost flat when $\tau$ is small, then increases gradually and drops quickly as $\tau$ further increases. In other words, the power of the online-style projection test is not very sensitive to the choice of $k_n$ and gives very similar power when $\tau$ is relatively small $(\tau \le 0.8)$. When $k_n$ is large, we perform the online-style projection test only based on a dataset of a relatively small sample size and lower power is expected. According to Figure 3, we suggest choosing $\tau \in \left[\mathrm{0.4,0.8}\right]$ in practice and we use $\tau =0.6$ in the rest of this paper.

Figure 3:

Power of the online-style projection test against the choice of $\tau$. We set $(n,p,c)=\left(\mathrm{100,1600,0.25}\right)$ and $\left(\mathrm{40,1600,0.5}\right)$ for both autocorrelation and compound symmetry covariance matrix structure with $\rho \in \left\{\mathrm{0.25,0.50,0.75,0.95}\right\}$. The upper two panels are the power for autocorrelation structure and the lower two panels are the power for compound symmetry structure.

4. Numerical Results

In this section, we conduct numerical experiments to examine the finite sample performance of different tests for one-sample mean vector problem in high dimensions, including projection tests as well as quadratic-form tests. In particular, we consider the following optimal projection based projection tests:

• OPT-O: Online-style Projection Test One-by-one version according to Algorithm 2.

• OPT-B: Online-style Projection Test mini-Batch version according to Algorithm 3.

• OPT-R: Online-style Projection Test One-by-one version where the projection direction is estimated by the ridge-type estimator in Huang (2015).

• DSPT: Data-splitting Projection Test according to Algorithm 1.

• RDSPT: Data-splitting Projection Test where the projection direction is estimated by the ridge-type estimator in Huang (2015).

We use the SCAD penalty (Fan and Li 2001) to estimate the optimal projection direction. The batch size for the OPT-B test is set to be 10. We also include two other projection tests, which are the Lauter test (Lauter 1996) and the random projection test (RPT) proposed in Lopes et al. (2011). The quadratic-form tests include the BS test (Bai and Saranadasa 1996), the CQ test (Chen and Qin 2010) and the SD test (Srivastava and Du 2008).

4.1. Simulation studies

We generate random samples of size $n$ from multivariate normal distribution $N(c\mathit{\mu },\mathbf{\Sigma })$, multivariate $t$-distribution with degrees of freedom 6, and multivariate ${\chi }^2$-distribution with degree of freedom 1 where $\mathit{\mu }={\left(1_s^{\top },0_{p-s}^{\top }\right)}^{\top }$ and $s=10$. We set $c=\mathrm{0,0.5}$ and 1 to examine Type I error rate and compare power of different tests, where $c=0$ corresponds to the null hypothesis and $c=0.5$ or 1 corresponds to the alternative hypothesis. For $\rho \in \left(\mathrm{0,1}\right)$, we consider the following two covariance structures: (1) compound symmetry with ${\mathbf{\Sigma }}_1=(1-\rho )\mathbf{I}+\rho 11^{\top }$ and (2) autocorrelation with ${\mathbf{\Sigma }}_2={\left({\rho }^{|i-j|}\right)}_{i,j}$, where $1$ is a vector with all its entries being 1 and $\mathbf{I}$ is the identity matrix. We consider $\rho =\mathrm{0.25,0.5,0.75}$ and 0.95 to study the impact of correlation on testing power. We set sample size $n=\mathrm{40,160}$ and dimension $p=\mathrm{400,1600}$. For online-style projection tests (i.e., OPT-O, OPT-B and OPT-R), we set $k_n=⌊n^{\tau }⌋$ with $\tau =0.6$. For data splitting projection tests (i.e., DSPT and RDSPT), we set $n_1=\lfloor n\tau \rfloor$ with $\tau =0.5$. To this end, we replace the sample covariance matrix $\widehat{\mathbf{\Sigma }}$ by ${\widehat{\mathbf{\Sigma }}}_{\varphi }=\widehat{\mathbf{\Sigma }}+\varphi \mathbf{I}$ with a small positive number $\varphi =\sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}$ and all the theoretical results still hold when $\varphi \le \sqrt{\mathrm{l}\mathrm{o}\mathrm{g}p/n}$. Such a perturbation does not noticeably affect the computational accuracy of the estimator but leads to a faster convergence according to our numerical studies. We set the type I error rate $\alpha =0.05$. All simulation results are based on 10,000 replications. Top two panels of Table 1 report type I error rate and power for all tests with $n=40$ and $p=1600$ under compound symmetry structure and autocorrelation structure respectively. To save space, all other simulation results can be found in Tables S.2 – S.6 of Supplement S.9.

Table 1:

Size and power comparison for $n=40$ and $p=1600$ (values are in percentage).

	c = 0				c = 0.5				c = 1
ρ	0.25	0.5	0.75	0.95	0.25	0.5	0.75	0.95	0.25	0.5	0.75	0.95
$N\left(c\mathit{\mu },\sum \right)\text{with}\sum \text{=}{\sum }_1$
OPT-O	5.70	5.83	5.07	5.11	90.35	99.70	100.0	100.0	100.0	100.0	100.0	100.0
OPT-B	6.11	5.74	5.68	5.41	77.18	95.28	99.80	99.97	100.0	100.0	100.0	100.0
OPT-R	5.20	5.03	5.27	5.05	15.82	28.90	82.01	100.0	97.97	99.79	99.99	100.0
DSPT	5.22	4.99	5.21	5.08	50.43	79.97	98.51	99.98	99.92	99.94	99.99	100.0
RDSPT	5.01	4.71	5.06	4.94	14.62	23.71	54.68	98.14	71.49	81.98	95.74	100.0
BS	6.08	6.20	6.22	6.22	7.28	6.72	6.58	6.50	11.18	8.46	7.70	7.34
CQ	6.02	6.14	6.14	6.12	7.30	6.62	6.54	6.46	11.18	8.46	7.58	7.32
SD	2.29	0.56	0.11	0.01	2.50	0.58	0.12	0.01	4.06	0.72	0.14	0.01
Lauter	5.16	5.18	5.14	5.16	5.04	5.06	5.08	5.06	5.12	5.08	5.02	5.02
RPT	5.12	5.24	5.04	4.90	6.88	8.00	11.78	51.76	14.56	20.94	42.22	98.38
$N\left(c\mathit{\mu },\sum \right)\text{with}\sum \text{=}{\sum }_2$
OPT-O	5.87	5.99	5.76	5.88	74.33	59.22	40.02	25.12	100.0	100.0	99.98	99.84
OPT-B	5.48	5.90	6.06	5.98	63.21	49.59	32.92	20.84	100.0	100.0	99.87	98.33
OPT-R	6.14	5.45	5.96	5.47	32.44	24.99	15.73	8.37	99.89	98.48	84.55	40.02
DSPT	5.25	5.19	5.09	5.12	38.03	30.96	22.88	16.49	100.0	99.94	98.81	91.04
RDSPT	4.61	4.95	5.30	4.92	17.85	14.57	9.55	6.10	94.90	84.59	58.09	22.43
BS	5.18	4.96	5.24	4.46	38.02	29.42	17.74	8.02	100.0	99.84	91.44	33.82
CQ	5.16	5.06	5.24	4.44	38.08	29.46	17.72	8.06	100.0	99.84	91.48	33.92
SD	11.73	8.22	4.08	1.65	64.17	45.15	20.19	3.45	100.0	99.82	91.48	20.90
Lauter	4.90	4.66	5.20	5.10	8.70	6.42	5.94	5.14	14.64	9.48	6.98	5.36
RPT	4.86	4.98	5.16	4.90	6.34	6.30	6.60	6.86	11.88	12.16	11.52	13.36
$t_6\left(c\mathit{\mu },\sum \right)\text{with}\sum \text{=}{\sum }_1$
OPT-O	5.81	5.39	5.18	4.89	75.58	95.94	99.90	100.0	100.0	100.0	100.0	100.0
OPT-B	6.21	5.32	5.11	4.79	61.58	87.35	98.92	100.0	100.0	99.98	99.99	100.0
OPT-R	4.93	4.79	4.75	5.74	12.71	22.99	67.67	99.95	89.86	96.65	99.74	100.0
DSPT	5.28	5.09	4.83	4.62	40.44	69.29	95.49	99.97	99.82	99.83	99.99	99.99
RDSPT	4.88	5.06	5.02	4.74	11.40	18.51	46.12	96.50	61.99	74.62	92.62	99.98
BS	4.92	5.75	5.93	5.98	5.45	6.03	6.09	6.12	7.48	7.13	6.82	6.65
CQ	6.16	6.19	6.18	6.13	6.75	6.44	6.31	6.28	9.42	7.57	7.12	6.89
SD	1.19	0.43	0.05	0.02	1.34	0.43	0.05	0.02	1.89	0.53	0.05	0.02
Lauter	4.95	4.97	4.93	4.96	4.98	4.95	4.97	4.96	4.97	4.93	4.92	4.91
RPT	4.43	4.61	4.26	4.15	5.71	7.08	9.87	44.45	12.53	18.11	35.10	96.15
$t_6\left(c\mathit{\mu },\sum \right)\text{with}\sum \text{=}{\sum }_2$
OPT-O	5.93	5.86	5.91	6.03	58.34	45.07	30.92	17.82	99.99	99.96	99.36	96.20
OPT-B	5.46	5.82	6.02	6.24	48.12	36.98	26.30	16.15	99.99	99.90	98.59	91.16
OPT-R	5.48	5.96	6.20	6.19	23.81	18.51	12.64	7.43	97.26	91.16	70.57	30.22
DSPT	4.89	4.61	4.85	4.77	28.90	24.90	17.90	12.35	99.89	99.19	94.29	78.77
RDSPT	5.24	4.58	5.08	5.37	13.82	11.03	8.69	5.40	83.00	69.65	44.60	17.05
BS	0.00	0.00	0.01	0.78	0.00	0.01	0.06	1.36	3.05	3.60	5.24	6.85
CQ	5.14	5.17	5.09	5.25	21.73	17.47	11.39	7.08	97.03	90.64	66.82	20.99
SD	0.00	0.00	0.00	0.06	0.00	0.00	0.01	0.18	1.31	1.82	2.21	1.66
Lauter	5.07	5.12	4.76	5.09	7.72	6.65	5.08	5.27	12.20	9.06	6.38	5.44
RPT	3.89	4.63	4.00	4.34	5.47	5.81	5.14	5.78	10.12	10.39	10.17	11.02

We first examine the type I error rate. Among all these tests, the DSPT, the RDSPT, the Lauter test and the RPT are exact tests (under normality assumption) and their sizes are exactly $\alpha =0.05$. The online-style projection tests have asymptotic normal distribution under $H_0$ and retain the type I error very well. Similar to online-style projection tests, the BS test and the CQ test also control the size reasonably well. The SD test is very sensitive to the correlation level and its size is often deviated from the expected 0.05.

Next we compare the power of these tests. Table 1 suggests that the power of these tests strongly relies on the covariance structure, the correlation $\rho$ as well as the signal strength $c$. In summary, the proposed OPT-O test is the most powerful test in all settings. The one-by-one online-style projection (OPT-O) is slightly more powerful than the corresponding mini-batch version (OPT-B) and improves the performance of corresponding data splitting projection test (DSPT) a lot. This is not surprising since the OPT-O test keeps updating the estimation of projection direction whenever a new observation arrives and generally has a more accurate estimation than its mini-batch version. The mini-batch projection test slightly sacrifices the accuracy but reduces the computational cost. The DSPT is less powerful as it throws away much more information compared with online-style projection tests. The test based on constrained and regularized quadratic programming (DSPT) is more powerful than the one based on the ridge-type estimator (RDSPT) as the true optimal projection direction is (approximately) sparse. When the covariance structure is compound symmetry, the power of optimal projection based tests improve as $\rho$ increases as larger $\rho$ leads to stronger signal ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}_1^{-1}\mathit{\mu }$ in the alternative. As the value of $c$ jumps from 0.5 to 1, the power of all tests increases dramatically. As the dimension $p$ goes from 400 to 1600, there is a downward trend for the performance of these tests. However, even in the most challenging setting $(n,p,c)=\left(\mathrm{40,1600,0.5}\right)$, the proposed OPT-O still has very satisfactory performance with power greater than 90%. The quadratic-form tests tend to become less powerful as $\rho$ increases. This is because quadratic-form tests ignore the correlation among the variables and therefore their overall performances are not satisfactory when correlation is strong. When the covariance structure is autocorrelation, unlike the compound symmetry setting, the power of all the tests drops as $\rho$ increases since larger $\rho$ would lead to weaker signal ${\mathit{\mu }}^{\top }{\mathbf{\Sigma }}_2^{-1}\mathit{\mu }$ in the alternative. We notice that some of the quadratic-form tests have better power than the data splitting projection tests when $\rho$ is small. This is because ${\Sigma }_2^{-1}$ is a 3-sparse matrix, which is almost an identity matrix, and ignoring dependence among variables does not noticeably diminish the power. In fact, the power of quadratic-form tests decreases dramatically as $\rho$ increases and become less powerful than the DSPT when $\rho =0.95$, not to mention the OPT-O test. As shown in the bottom two panels of Table 1 and Tables S.4, S.5 and S.6, the overall pattern of size and power for multivariate $t_6$-distribution and ${\chi }_1^2$-distribution is very similar to that under normality assumption, indicating the proposed projection test is not very sensitive to the sub-Gaussianity assumption and performs well for asymmetric data. For non-sub-Gaussian data, we observe that online-style projection test still controls the type I error rate well and retains high power. The proposed OPT-O test is still the most powerful one among those which successfully retain the type I error rate. For other tests, the BS test and the CQ test have slight size distortion under ${\chi }_1^2$ while the SD test and the Lauter test completely fail to control the type I error under ${\chi }_1^2$.

4.2. Real data example

In this section, we apply the proposed online-style projection test as well as other tests to a real dataset of high resolution micro-computed tomography. This dataset consists of 58 mice’s skull bone density from three different genotypes (“T0A0”, “T0A1”, “T1A1”) in a genetic mutation study. For each mouse, bone density is measured at density levels from 130 – 249 for 16 different areas of its skull. See Percival et al. (2014) for a detailed description of protocols. In this empirical study, we would like to know if there is a difference in the bone density patterns of two different areas in mice’s skull. To emphasize the high-dimensionality nature of this dataset, we only use a subset of the dataset. We select the mice of the genotype “T0A1” and there are 29 observations available in the dataset, i.e., sample size $n=29$. The two areas of the skull “Mandible” and “Nasal” are selected. We use all density levels from 130 – 249 for our analysis, hence dimension $p=120$. We first take the difference of the bone density of the two selected areas at the corresponding density level for each subject since the two bones come from the same mouse. Then we normalize the bone density in the sense that $\frac{1}{29}{\sum }_{i=1}^{29}{X}_{ij}^2=1$ for all $1\le j\le 120$.

We first apply SOPT-O, SPT-DS and other existing tests to perform the one-sample test and compute the p-values. The p-values are reported in the first column in Table 2. All p-values are very close to 0 ($\ll 0.05$), implying that the bone densities of the two areas are significantly different. In order to compare the power of different tests, we also compute the p-values of different tests as we weaken the signal strength in alternative. To be more specific, let $\stackrel{-}{\mathbf{x}}$ be the sample mean and ${\mathbf{r}}_i={\mathbf{x}}_i-\stackrel{-}{\mathbf{x}}$ be the residual for the $i$th subject. Then we can construct a new observation ${\mathbf{z}}_i=\delta \stackrel{-}{\mathbf{x}}+{\mathbf{r}}_i$ for the $i$th subject, where $\delta \in \left(\mathrm{0,1}\right)$. By the construction, a smaller $\delta$ leads to a weaker signal and makes the test more challenging. Table 2 also reports the p-values of different tests with $\delta =1.0,0.8,0.6,0.4,0.3,0.2$. As expected, p-values of all tests increase as $\delta$ decreases. When $\delta =0.8$ or 0.6, all tests perform well and reject the null hypothesis at level 0.05. When $\delta =0.4$, the Lauter’s test starts to fail to reject the null hypothesis. When $\delta$ is further decreased to 0.3, all the three projection based tests SOPT-O, SPT-DS and RPT-DS are able to reject the null hypothesis while all other tests except the LJW2011 test fail to reject null hypothesis. When $\delta =0.2$, only SOPT-O and SPT-DS reject the null hypothesis, which suggests that our proposed sparse projection tests can still perform very well even when the signal is weak. Among those tests that fail to reject the null at $\delta =0.2$, RPT-DS is the most powerful one as it has the smallest p-value.

Table 2:

P-values of different tests for bone density dataset.

δ	1.0	0.8	0.6	0.4	0.3	0.2
OPT-O	0	0	4.6 × 10−13	4.0 × 10−10	1.6 × 10−4	0.0325
DSPT	7.9 × 10−10	6.7 × 10−9	3.3 × 10−7	9.9 × 10−6	3.4 × 10−4	0.0418
RDSPT	2.4 × 10−8	5.1 × 10−7	1.9 × 10−5	0.0014	0.0140	0.1462
BS	0	0	0	1.2 × 10−4	0.0763	0.7684
CQ	0	0	0	1.6 × 10−4	0.0810	0.7717
SD	0	0	2.0 × 10−9	1.6 × 10−2	0.2494	0.7995
Lauter	1.1 × 10−10	3.1 × 10−8	1.6 × 10−3	0.1265	0.2625	0.4574
RPT	3.8 × 10−9	4.2 × 10−8	4.5 × 10−6	8.5 × 10−4	9.6 × 10−3	0.2031

We plot the heatmap of absolute values of paired sample correlations of all bone density levels in Figure 4. The heatmap clearly shows that some bone density levels are highly correlated. This explains why the proposed projection tests are more powerful than the D1958 test, the BS1996 test and the SD2008w test as these tests do not take the dependence among variables into account.

Figure 4:

Heatmap of absolute values of paired sample correlations of bone densities. The yellow grids indicate that some bone density levels are highly correlated. The green curve in the right panel is the histogram of absolute values of paired sample correlation.

5. Discussion

This paper studies the projection test for mean vectors in high dimensions. Existing projection tests either fail to utilize the optimal projection or lack the theoretical justification of the estimated projection direction. To maximize the power of projection tests, a critical task is to obtain a good estimation of the optimal projection with statistical guarantees. We propose a constrained and regularized quadratic programming to obtain a consistent estimator under sparsity assumption. We further propose an online-style framework for projection tests to enhance the testing power. This is a general testing framework in the sense that any proper estimator (e.g., Ridge-type estimator) of the optimal projection can be applied and can be easily extended to the two-sample problem. Under this framework, the first $k_n$ observations are used to obtain an initial estimator and are not utilized to perform the test. In future work, it would be interesting to investigate how we can reuse the $k_n$ observations that are discarded to further enhance the power. A challenge is whether we can still control the type I error with guarantees by reusing the first $k_n$ observations. Another interesting question is how we can construct a project test when the sparsity assumption does not hold. When the optimal projection directions is not sparse, the ridge-type estimator in Huang (2015) seems a good candidate and it is worth investigating its theoretical properties in future work.