The Sparse Laplacian Shrinkage Estimator for High-Dimensional Regression

Abstract

We propose a new penalized method for variable selection and estimation that explicitly incorporates the correlation patterns among predictors. This method is based on a combination of the minimax concave penalty and Laplacian quadratic associated with a graph as the penalty function. We call it the sparse Laplacian shrinkage (SLS) method. The SLS uses the minimax concave penalty for encouraging sparsity and Laplacian quadratic penalty for promoting smoothness among coefficients associated with the correlated predictors. The SLS has a generalized grouping property with respect to the graph represented by the Laplacian quadratic. We show that the SLS possesses an oracle property in the sense that it is selection consistent and equal to the oracle Laplacian shrinkage estimator with high probability. This result holds in sparse, high-dimensional settings with p ≫ n under reasonable conditions. We derive a coordinate descent algorithm for computing the SLS estimates. Simulation studies are conducted to evaluate the performance of the SLS method and a real data example is used to illustrate its application.

1 Introduction

There has been much work on penalized methods for variable selection and estimation in high-dimensional regression models. Several important methods have been proposed. Examples include estimators based on the bridge penalty (Frank and Friedman 1993), the ℓ₁ penalty or the least absolute shrinkage and selection operator (LASSO, Tibshirani 1996; Chen, Donoho and Saunders 1998), the smoothly clipped absolute deviation (SCAD) penalty (Fan 1997; Fan and Li 2001), and the minimum concave penalty (MCP, Zhang 2010). These methods are able to do estimation and automatic variable selection simultaneously and provide a computationally feasible way for variable selection in high-dimensional settings. Much progress has been made in understanding the theoretical properties of these methods. Efficient algorithms have also been developed for implementing these methods.

A common feature of the methods mentioned above is the independence between the penalty and the correlation among predictors. This can lead to unsatisfactory selection results, especially in p≫ n settings. For example, as pointed out by Zou and Hastie (2005), the LASSO tends to only select one variable among a group of highly correlated variables; and its prediction performance may not be as good as the ridge regression if there exists high correlation among predictors. To overcome these limitations, Zou and Hastie (2005) proposed the elastic net (Enet) method, which uses a combination of the ℓ₁ and ℓ₂ penalties. Selection properties of the Enet and adaptive Enet have also been studied by Jia and Yu (2009) and Zou and Zhang (2009). Bondell and Reich (2008) proposed the OSCAR (octagonal shrinkage and clustering algorithm for regression) approach, which uses a combination of the ℓ₁ norm and a pairwise ℓ_∞ norm for the coefficients. Huang et al. (2010) proposed the Mnet method, which uses a combination of the MCP and ℓ₂ penalties. The Mnet estimator is equal to the oracle ridge estimator with high probability under certain conditions. These methods are effective in dealing with certain types of collinearity among predictors and has the useful grouping property of selecting and dropping highly correlated predictors together. Still, these combination penalties do not use any specific information on the correlation pattern among the predictors.

Li and Li (2008) proposed a network-constrained regularization procedure for variable selection and estimation in linear regression models, where the predictors are genomic data measured on genetic networks. Li and Li (2010) considered the general problem of regression analysis when predictors are measured on an undirected graph, which is assumed to be known a priori. They called their method a graph-constrained estimation procedure, or GRACE. The GRACE penalty is a combination of the ℓ₁ penalty and a penalty that is the Laplacian quadratic associated with the graph. Because the GRACE uses the ℓ₁ penalty for selection and sparsity, it has the same drawbacks as the Enet discussed above. In addition, the full knowledge of the graphical structure for the predictors is usually not available, especially in high-dimensional problems. Daye and Jeng (2009) proposed the weighted fusion method, which also uses a combination of the ℓ₁ penalty and a quadratic form that can incorporate information among correlated variables for estimation and variable selection. Tutz and Ulbricht (2009) studied a form of correlation based penalty, which can be considered a special case of the general quadratic penalty. But this approach does not do variable selection. The authors proposed a blockwise boosting procedure in combination with the correlation based penalty for variable selection. Hebiri and van de Geer (2010) studied the theoretical properties of the smoothed-Lasso and other ℓ₁ + ℓ₂-penalized methods in p ≫ n models. Pan, Xie and Shen (2009) studied a grouped penalty based on the L_γ-norm for γ > 1 that smoothes the regression coefficients over a network. In particular, when γ = 2 and after appropriate rescaling of the regression coefficients, this group L_γ penalty simplifies to the group Lasso (Yuan and Lin 2005) with the nodes in the network as groups. This method is capable of group selection, but it does not do individual variable selection. Also, because the group L_γ penalty is convex for γ > 1, it does not lead to consistent variable selection, even at the group level.

We propose a new penalized method for variable selection and estimation in sparse, high-dimensional settings that takes into account certain correlation patterns among predictors. We consider a combination of the MCP and Laplacian quadratic as the penalty function. We call the proposed approach the sparse Laplacian shrinkage (SLS) method. The SLS uses the MCP to promote sparsity and Laplacian quadratic penalty to encourage smoothness among coefficients associated with the correlated predictors. An important advantage of the MCP over the ℓ₁ penalty is that it leads to estimators that are nearly unbiased and achieve selection consistency under weaker conditions (Zhang, 2010).

The contributions of this paper are as follows.

• First, unlike the existing methods that use an ℓ₁ penalty for selection and a ridge penalty or a general ℓ₂ penalty for dealing with correlated predictors, we use the MCP to achieve nearly unbiased selection and proposed a concrete class of quadratics, the Laplacians, for incorporating correlation patterns among predictors in a local fashion. In particular, we suggest to employ the approaches for network analysis for specifying the Laplacians. This provides an implementable strategy for incorporating correlation structures in high-dimensional data analysis.

• Second, we prove that the SLS estimator is sign consistent and equal to the oracle Laplacian shrinkage estimator under reasonable conditions. This result holds for a large class of Laplacian quadratics. An important aspect of this result is that it allows the number of predictors to be larger than the sample size. In contrast, the works of Daye and Jeng (2009) and Tutz and Ulbricht (2009) do not contain such results in p ≫ n models. The selection consistency result of Hebiri and van de Geer (2010) requires certain strong assumptions on the magnitude of the smallest regression coefficient (their Assumption C) and on the correlation between important and unimportant predictors (their Assumption D), in addition to a variant of the restricted eigenvalue condition (their Assumption B). In comparison, our assumption involving the magnitude of the regression coefficients is weaker and we use a sparse Riese condition instead of imposing restriction on the correlations among predictors. In addition, our selection results are stronger in that the SLS estimator is not only sign consistent, but also equal to the oracle Laplacian shrinkage estimator with high probability. In general, similar results are not available with the use of the ℓ₁ penalty.

• Third, we show that the SLS method is potentially capable of incorporating correlation structure in the analysis without incurring extra bias. The Enet and the more general ℓ₁ + ℓ₂ methods in general introduces extra bias due to the quadratic penalty, in addition to the bias resulting from the ℓ₁ penalty. To the best of our knowledge, this point has not been discussed in the existing literature. We also demonstrate that the SLS has certain local smoothing property with respect to the graphical structure of the predictors.

• Fourth, unlike in the GRACE method, the SLS does not assume that the graphical structure for the predictors is known a priori. The SLS uses the existing data to construct the graph Laplacian or to augment partial knowledge of the graph structure.

• Fifth, our simulation studies demonstrate that the SLS method outperforms the ℓ₁ penalty plus a quadratic penalty approach as studied in Daye and Jeng (2009) and Hebri and van de Geer (2010). In our simulation examples, the SLS in general has smaller empirical false discovery rates with comparable false negative rates. It also has smaller prediction errors.

This paper is organized as follows. In Section 2 we define the SLS estimator. In Section 3 we discuss ways to construct graph Laplacian, or equivalently, its corresponding adjacency matrix. In Section 4 we study the selection properties of the SLS estimators. In Section 5 we investigate the properties of Laplacian shrinkage. In Section 6 we describe a coordinate descent algorithm for computing the SLS estimators, present simulation results and an application of the SLS method to a microarray gene expression dataset. Discussions of the proposed method and results are given in Section 7. Proofs for the oracle properties of the SLS and other technical details are provided in the Appendix.

2 The sparse Laplacian shrinkage estimator

Consider the linear regression model

$$\mathit{y}=\sum _{j=1}^p{\mathit{x}}_j{\beta }_j+\mathit{\epsilon }$$ (2.1)

with n observations and p potential predictors, where y = (y₁, …, y_n)′ is the vector of n response variables, x_j = (x_1j, …, x_nj)′ is the jth predictor, β_j is the jth regression coefficient and ε = (ε₁, …, ε_n)′ is the vector of random errors. Let X = (x₁, …, x_p) be the n × p design matrix. Throughout, we assume that the response and predictors are centered and the predictors are standardized so that ${\sum }_{i=1}^n{x}_{ij}^2=n$, j = 1, …, p. For λ = (λ₁, λ₂) with λ₁ ≥ 0 and λ₂ ≥ 0, we propose the penalized least squares criterion

$$M(\mathit{b};\lambda ,\gamma )=\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma )+\frac{1}{2}{\lambda }_2\sum _{1\le j<k\le p}\mid a_{jk}\mid {(b_j-s_{jk}{b}_k)}^2,$$ (2.2)

where || · || denotes the ℓ₂ norm, ρ is the MCP with penalty parameter λ₁ and regularization parameter γ, |a_jk| mesaures the strength of the connection between x_j and x_k, and s_jk = sgn(a_jk) is the sign of a_jk, with sgn(t) = −1, 0, or 1 respectively for t < 0, = 0, or > 0. The two penalty terms in (2.2) play different roles. The first term promotes sparsity in the estimated model. The second term encourages smoothness of the estimated coefficients of the connected predictors. We can associate the quadratic form in this term with the Laplacian for a suitably defined undirected weighted graph for the predictors. See the description below. For any given (λ, γ), the SLS estimator is

$$\widehat{\mathit{\beta }}(\lambda ,\gamma )=\underset{\mathit{b}}{\text{argmin}}M(\mathit{b};\lambda ,\gamma ).$$ (2.3)

The SLS uses the MCP, defined as

$$\rho (t;{\lambda }_1,\gamma )={\lambda }_1{\int }_0^{\mid t\mid }{(1-x/(\gamma {\lambda }_1))}_{+}dx,$$ (2.4)

where for any a ∈ R, a₊ is the nonnegative part of a, i.e., a₊ = a1_{a≥0}. The MCP can be easily understood by considering its derivative,

$$\stackrel{.}{\rho }(t;{\lambda }_1,\gamma )={\lambda }_1{(1-\mid t\mid /(\gamma {\lambda }_1))}_{+}sgn(t).$$ (2.5)

We observe that the MCP begins by applying the same level of penalization as the ℓ₁ penalty, but continuously reduces that level to 0 for |t| > γλ. The regularization parameter γ controls the degree of concavity. Larger values of γ make ρ less concave. By sliding the value of γ from 1 to ∞, the MCP provides a continuum of penalties with the hard-threshold penalty as γ → 1+ and the convex ℓ₁ penalty at γ = ∞. Detailed discussion of MCP can be found in Zhang (2010).

The SLS also allows the use of different penalties than the MCP for ρ, including the SCAD (Fan 1997; Fan and Li 2001) and other quadratic splines. Because the MCP minimizes the maximum concavity measure and has the simplest form among nearly unbiased penalties in this family, we choose it as the default penalty for the SLS. Further discussion of the MCP and its comparison with the LASSO and SCAD can be found in Zhang (2010) and Mazumder, Friedman and Hastie (2009).

We express the nonnegative quadratic form in the second penalty term in (2.2) using a positive semi-definite matrix L, which satisfies

$${\mathit{b}}^{\prime }L\mathit{b}=\sum _{1\le j<k\le p}\mid a_{jk}\mid {(b_j-s_{jk}{b}_k)}^2,\forall \mathit{b}\in {\text{IR}}^p.$$

For simplicity, we confine our discussion to the symmetric case where a_kj = a_jk, 1 ≤ j < k ≤ p. Since the diagonal elements a_jj do not appear in the quadratic form, we can define them any way we like for convenience. Let A = (a_jk, 1 ≤ j, k ≤ p) and D = diag(d₁, …, d_p), where $d_j={\sum }_{k=1}^p\mid a_{jk}\mid$. We have Σ_1≤j<k≤p |a_jk|(b_j − s_jkb_k)² = b′(D − A)b. Therefore, L = D − A. This matrix is associated with a labeled weighted graph = (V, ) with vertex set V = {1, …, p} and edge set = {(j, k): (j, k) ∈ V × V}. Here the |a_jk| is the weight of edge (j, k) and d_j is the degree of vertex j. The d_j is also called the connectivity of vertex j. The matrix L is called the Laplacian of and A its signed adjacency matrix (Chung 1997). The edge (j, k) is labeled with the “+” or “−” sign, but its weight |a_jk| is always nonnegative. We use a labeled graph to accommodate the case where two predictors can have a nonzero adjacent coefficient but are negatively correlated. Note that the usual adjacency matrix can be considered a special case of signed adjacency matrix when all a_jk ≥ 0. For simplicity, we will use the term adjacency matrix below.

We usually require that the adjacency matrix to be sparse in the sense that many of its entries are zero or nearly zero. With a sparse adjacency matrix, the main characteristic of the shrinkage induced by the Laplacian penalty is that it occurs locally for the coefficients associated with the predictors connected in the graph. Intuitively, this can be seen by writing

$${\lambda }_2\sum _{1\le j<k\le p}\mid a_{jk}\mid {(b_j-s_{jk}{b}_k)}^2=\frac{1}{2}{\lambda }_2\sum _{(j,k):a_{jk}\ne 0}\mid a_{jk}\mid {(b_j-s_{jk}{b}_k)}^2.$$

Thus for λ₂ > 0, the Laplacian penalty shrinks b_j − s_jkb_k towards zero for a_jk ≠ 0. This can also be considered as a type of local smoothing on the graph G associated with the adjacency matrix A. In comparison, the shrinkage induced by the ridge penalty used in the Enet is global in that it shrinks all the coefficients towards zero, regardless of the correlation structure among the predictors. We will discuss the Laplacian shrinkage in more detail in Section 5.

Using the matrix notation, the SLS criterion (2.2) can be written as

$$M(\mathit{b};\lambda ,\gamma )=\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma )+\frac{1}{2}{\lambda }_2{\mathit{b}}^{\prime }(D-A)\mathit{b}.$$ (2.6)

Here the Laplacian is not normalized, meaning that the weight d_j is not standardized to 1. In problems where predictors should be treated without preference with respect to connectivity, we can first normalized the Laplacian L^* = I_p − A^* with A^* = D^−1/2AD^−1/2 and use the criterion

$$M^{\ast }(\mathit{b};\lambda ,\gamma )=\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma )+\frac{1}{2}{\lambda }_2{\mathit{b}}^{\prime }({\mathit{I}}_p-A^{\ast })\mathit{b}.$$

Technically, a normalized Laplacian L^* can be considered a special case of a general L. We only consider the SLS estimator based on the criterion (2.6) when studying its properties. In network analysis of gene expression data, genes with large connectivity also tend to have important biological functions (Zhang and Horvath 2005). Therefore, it is prudent to provide more protection for such genes in the selection process.

3 Construction of adjacency matrix

In this section, we describe several simple forms of adjacency measures proposed by Zhang and Horvath (2005), which have have been successfully used in network analysis of gene expression data. The adjacency measure is often defined based on the notion of dissimilarity or similarity.

1. A basic and widely used dissimilarity measure is the Euclidean distance. Based on this distance, we can define adjacency coefficient as $a_{jk}=\phi (||{\mathit{x}}_j-{\mathit{x}}_k||/\sqrt{n})$, where φ: [0, ∞) ↦ [0, ∞). A simple adjacency function is the threshold function φ(x) = 1{x ≤ 2r}. Then

   $$a_{jk}=\{\begin{array}{ll}1\hfill & \text{if}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|/\sqrt{n}\le 2r\hfill \\ 0\hfill & \text{if}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|/\sqrt{n}>2r.\hfill \end{array}$$ (3.1)

   It is convenient to express a_jk in terms of the Pearson’s correlation coefficient r_jk between x_j and x_k, where $r_{jk}={\mathit{x}}_j^{\prime }{\mathit{x}}_k/(|\left|{\mathit{x}}_j\right|\left|\right|\left|{\mathit{x}}_k\right||)$. For predictors that are standardized with ||x_j||² = n, 1 ≤ j ≤ p, we have ||x_j − x_k||²/n = 2 − 2r_jk. Thus in terms of correlation coefficients, we can write a_jk = 1{r_jk > r}. We determine the value of r based on the Fisher transformation z_jk = 0.5 log((1 + r_jk)/(1 − r_jk)). If the correlation between x_j and x_k is zero, $\sqrt{n-3}{z}_{jk}$ is approximately distributed as N(0, 1). We can use this to determine a threshold c for $\sqrt{n-3}{z}_{jk}$. The corresponding threshold for r_jk is $r=(exp(2c/\sqrt{n-3})-1)/(exp(2c/\sqrt{n-3})+1)$.

   We note that here we use the Fisher transformation to change the scale of the correlation coefficients from [−1, 1] to the normal scale for determining the threshold value r, so that the adjacency matrix is relatively sparse. We are not trying to test the significance of correlation coefficients.

2. The adjacency coefficient in (3.1) is defined based on a dissimilarity measure. Adjacency coefficient can also be defined based on similarity measures. An often used similarity measure is Pearson’s correlation coefficient r_jk. Other correlation measures such as Spearman’s correlation can also be used. Let

   $$s_{jk}=sgn(r_{jk})\text{and}{a}_{jk}=s_{jk}1\{\mid r_{jk}\mid >r\}.$$

   Here r can be determined using the Fisher transformation as above.

3. With the power adjacency function considered in Zhang and Horvath (2005),

   $$a_{jk}=max{(0,r_{jk})}^{\alpha }\text{and}{s}_{jk}=1.$$

   Here α > 0 and can be determined by, for example, the scale-free topology criterion.

4. A variation of the above power adjacency function is

   $$a_{jk}=\mid r_{jk}{\mid }^{\alpha }\text{and}{s}_{jk}=\mathit{sgn}(r_{jk}).$$

For the adjacency matrices given above, (i) and (ii) use dichotomized measures, whereas (iii) and (iv) use continuous measures. Under (i) and (iii), two covariates are either positively or not connected/correlated. In contrast, under (ii) and (iv), two covariates are allowed to be negatively connected/correlated.

There are many other ways for constructing an adjacency matrix. For example, a popular adjacency measure in cluster analysis is a_jk = exp(− ||x_j − x_k||²/nτ²) for τ > 0. The resulting adjacency matrix A = [a_jk] is the Gram matrix associated with the Gaussian kernel. For discrete covariates, the Pearson correlation coefficient can still be used as a measure of correlation or association between two discrete predictors or between a discrete predictor and a continuous one. For example, for single nucleotide polymorphism data, Pearson’s correlation coefficient is often used as a measure of linkage disequilibrium (i.e. association) between two markers. Other measures, such as odds ratio or measure of association based on contingency table can also be used for r_jk.

We note that how to construct the adjacency matrix is problem specific. Different applications may require different adjacency matrices. Since construction of adjacency matrix is not the focus of the present paper, we will only consider the use of the four adjacency matrices described above in our numerical studies in Section 6.

4 Oracle properties

In this section, we study the theoretical properties of the SLS estimator. Let the true value of the regression coefficient be ${\mathit{\beta }}^o={({\beta }_1^o,\dots ,{\beta }_p^0)}^{\prime }$. Denote $\mathcal{O}=\{j:{\beta }_j^o\ne 0\}$, which is the set of indices of nonzero coefficients. Let d^o = | | be the cardinality of . Define

$${\widehat{\mathit{\beta }}}^o({\lambda }_2)=\underset{\mathit{b}}{\text{argmin}}\{\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\frac{1}{2}{\lambda }_2{\mathit{b}}^{\prime }L\mathit{b},b_j=0,j\notin \mathcal{O}\}.$$ (4.1)

This is the oracle Laplacian shrinkage estimator on the set . Theorems 1 and 2 below provide sufficient conditions under which P(sgn(β̂) ≠ sgn(β^o) or β̂ ≠ β̂^o) → 0. Thus under those conditions, the SLS estimator is sign consistent and equal to β̂^o with high probability.

We need the following notation in stating our results. Let Σ = n⁻¹X′X. For any A ∪ B ⊆ {1, …, p}, vectors v, the design matrix X and V = (v_ij)_p×p, define

$${\mathit{v}}_B={(v_j,j\in B)}^{\prime },X_B=({\mathit{x}}_j,j\in B),V_{A,B}={(v_{ij},i\in A,j\in B)}_{\mid A\mid \times \mid B\mid },V_B=V_{B,B}.$$

For example, ${\mathrm{\sum }}_B=X_B^{\prime }{X}_B/n$ and (λ₂) = + λ₂ . Let |B| denotes the cardinality of B. Let c_min(λ₂) be the smallest eigenvalue of Σ + λ₂L. We use the following constants to bound the bias of the Laplacian:

$$C_1={\left|\right|{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}_{\infty },C_2={\left|\right|\{{\mathrm{\sum }}_{{\mathcal{O}}^c,\mathcal{O}}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}-L_{{\mathcal{O}}^c,\mathcal{O}}\}{\mathit{\beta }}_{\mathcal{O}}^0\left|\right|}_{\infty }.$$ (4.2)

We make the following sub-Gausian assumption on the error terms in (2.1).

Condition (A): For a certain constant ε ∈ (0, 1/3),

$$\underset{\left|\right|\mathit{u}\left|\right|=1}{sup}P\{{\mathit{u}}^{\prime }\mathit{\epsilon }>\sigma t\}\le e^{-t^2/2},0<t\le \sqrt{2log(p/\epsilon )}.$$

4.1 Convex penalized loss

We first consider the case where Σ(λ₂) = Σ + λ₂L is positive definite. Since (4.1) is the minimizer of the Laplacian restricted to the support , it can be explicitly written as

$${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o={({\mathrm{\sum }}_{\mathcal{O}}+{\lambda }_2{L}_{\mathcal{O}})}^{-1}{X}_{\mathcal{O}}^{\prime }\mathit{y}/n,{\widehat{\mathit{\beta }}}_{{\mathcal{O}}^c}^o=0,$$ (4.3)

provided that (λ₂) is invertible. Its expectation β* = E β̂^o, considered as a target of the SLS estimator, must satisfy

$${\mathit{\beta }}_{\mathcal{O}}^{\ast }={({\mathrm{\sum }}_{\mathcal{O}}+{\lambda }_2{L}_{\mathcal{O}})}^{-1}{\mathrm{\sum }}_{\mathcal{O}}{\mathit{\beta }}^o,{\mathit{\beta }}_{{\mathcal{O}}^c}^{\ast }=0,$$ (4.4)

Condition (B): (i) c_min(λ₂) > 1/γ with ρ(t; λ₁, γ) in (2.2). (ii) The penalty levels satisfy

$${\lambda }_1\ge {\lambda }_2{C}_2+\sigma \sqrt{2log((p-d^o)/\epsilon )}\underset{j\le p}{max}\left|\right|{\mathit{x}}_j\left|\right|/n$$

with C₂ in (4.2). (iii) With {v_j, j ∈ } being the diagonal elements of ${\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}\{{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)\}$,

$$\underset{j\in \mathcal{O}}{min}\{\mid {\beta }_j^{\ast }\mid {(n/v_j)}^{1/2}\}\ge \sigma \sqrt{2log(d^o/\epsilon )}.$$

Define ${\beta }_{\ast }=min\{\mid {\beta }_j^o\mid ,j\in \mathcal{O}\}$. If is an empty set, that is, when all the regression coefficients are zero, we set β_* = ∞.

Theorem 1

Suppose Conditions (A) and (B) hold. Then,

$$\text{P}\left(\{j:{\widehat{\beta }}_j\ne 0\}\ne \mathcal{O}or\widehat{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^o\right)\le 3\epsilon .$$ (4.5)

If ${\beta }_{\ast }\ge {\lambda }_2{C}_1+{max}_j\sqrt{(2v_j/n)log(d^o/\epsilon )}$ instead of Condition (B) (iii), then

$$\text{P}\left(\mathit{sgn}(\widehat{\mathit{\beta }})\ne \mathit{sgn}({\mathit{\beta }}^o)or\widehat{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^o\right)\le 3\epsilon .$$ (4.6)

Here note that p, d^o, γ and c_min(λ₂) are all allowed to depend on n.

The probability bound on the selection error in Theorem 1 is nonasymptotic. If the conditions of Theorem 1 hold with ε → 0, then (4.5) implies selection consistency of the SLS estimator and (4.6) implies sign consistency. The conditions are mild. Condition (A) concerns the tail probabilities of the error distribution and is satisfied if the errors are normally distributed. Condition (B) (i) ensures that the SLS criterion is strictly convex so that the solution is unique. The oracle estimator β̂^o is biased due to the Laplacian shrinkage. Condition (B) (ii) requires a penalty level λ₁ to prevent this bias and noise to cause false selection of variables in Condition (B) (iii) requires that the nonzero coefficients not be too small in order for the SLS estimator to be able to distinguish nonzero from zero coefficients.

In Theorem 1, we only require c_min(λ₂) > 0, or equivalently, Σ + λ₂L to be positive definite. The matrix Σ can be singular. This can be seen as follows. The adjacency matrix partitions the graph into disconnected cliques V_g, 1 ≤ g ≤ J for some J ≥ 1. Let node j_g be a (representative) member of V_g. A node k belongs to the same clique V_g iff (if and only if) a_jgk1a_k1k2 · · · a_kmk ≠ 0 through a certain chain j_g → k₁ → k₂ → ··· → k_m → k. Define x̄_g = Σ_k∈Vg a_jgk1a_k1k2 ··· a_kmkx_k/|V_g|, where |V_g| is the cardinality of V_g. The matrix Σ + λ₂L is positive definite iff b′Σb = b′Lb = 0 implies b = 0. Since b′Lb = 0 implies Σ_k∈Vg b_kx_k = b_jg |V_g|x̄_g, Σ+λ₂L is positive definite iff the vectors x̄_g are linearly independent. This does not require n ≥ p. In other words, Theorem 1 is applicable to p > n problems as long as the vectors x̄_g are linearly independent.

4.2 The nonconvex case

When Σ(λ₂) = Σ + λ₂L is singular, Theorem 1 is not applicable. In this case, further conditions are required for the oracle property to hold. The key condition needed is the sparse Reisz condition, or SRC (Zhang and Huang 2008), in (4.9) below. It restricts the spectrum of diagonal subblocks of Σ(λ₂) up to a certain dimension.

Let X̃ = X̃(λ₂) be a matrix satisfying X̃′X̃/n = Σ(λ₂) = X′X/n + λ₂L and ỹ = ỹ (λ₂) be a vector satisfying X̃′ỹ = X′y. Define

$$\stackrel{\sim }{M}(\mathit{b};\lambda ,\gamma )=\frac{1}{2n}{\left|\right|\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma ).$$ (4.7)

Since M(b; λ, γ) − M̃ (b; λ, γ) = (||y||² − ||ỹ||²)/(2n), the two penalized loss functions have the same set of local minimizers. For the penalized loss (4.7) with the data (X̃, ỹ), let

$$\widehat{\mathit{\beta }}(\lambda )=\mathit{\delta }(\stackrel{\sim }{X}({\lambda }_2),\stackrel{\sim }{\mathit{y}}({\lambda }_2),{\lambda }_1),$$ (4.8)

where the map δ(X, y, λ₁) ∈ IR^p defines the MC+ estimator (Zhang, 2010) with data (X, y) and penalty level λ₁. It was shown in Zhang (2010) that δ (X, y, λ₁) depends on (X, y) only through X′y/n and X′X/n, so that different choices of X̃ and ỹ are allowed. One way is to pick ỹ = (y′, 0)′ and X̃ = diag(X, (nλ₂L)^1/2). Another way is to pick X̃′X̃/n = Σ(λ₂) and ỹ = (X̃′)^†X′y of smaller dimensions, where (X̃′)^† is the Moore-Penrose inverse of X̃′.

Condition (C)

1. For an integer d^* and spectrum bounds 0 < c_*(λ₂) ≤ c^*(λ₂) < ∞,

   $$0<c_{\ast }({\lambda }_2)\le {\mathit{u}}_B^{\prime }{\mathrm{\sum }}_B({\lambda }_2){\mathit{u}}_B\le c^{\ast }({\lambda }_2)<\infty ,\forall B\mathit{with}\mid B\cup \mathcal{O}\mid \le d^{\ast },\left|\right|{\mathit{u}}_B\left|\right|=1,$$ (4.9)

   with d^* ≥ d^o(K_* + 1), $\gamma \ge c_{\ast }^{-1}({\lambda }_2)\sqrt{4+c_{\ast }({\lambda }_2)/c^{\ast }({\lambda }_2)}$ in (2.2), and K_* = c^*(λ₂)/c_*(λ₂) − 1/2.

2. With $C_2={\left|\right|\{{\mathrm{\sum }}_{B,\mathcal{O}}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}-L_{B,\mathcal{O}}\}{\beta }_{\mathcal{O}}^o\left|\right|}_{\infty }$,

   $$max\{1,\sqrt{c_{\ast }({\lambda }_2)K_{\ast }/(K_{\ast }+1)}\}{\lambda }_1\ge {\lambda }_2{C}_2+\sigma \sqrt{2log(p/\epsilon )}\underset{j\le p}{max}\left|\right|{\mathit{x}}_j\left|\right|/n.$$
3. With {v_j, j ∈ } being the diagonal elements of ${\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}\{{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)\}$,

   $$\underset{j\in \mathcal{O}}{min}\{\mid {\beta }_j^{\ast }\mid -\gamma (2\sqrt{c^{\ast }({\lambda }_2)}{\lambda }_1)\}{(n/v_j)}^{1/2}\ge \sigma \sqrt{2log(d^o/\epsilon )}.$$

Theorem 2

1. Suppose Conditions (A) and (C) hold. Let β̂(λ) be as in (4.8). Then,

   $$\text{P}\left(\{j:{\widehat{\beta }}_j\ne 0\}\ne \mathcal{O}or\widehat{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^o\right)\le 3\epsilon .$$ (4.10)
   If ${\beta }_{\ast }\ge {\lambda }_2{C}_1+\gamma (2\sqrt{c^{\ast }({\lambda }_2)}{\lambda }_1)+{max}_j\sqrt{(2v_j/n)log(d^o/\epsilon )}$ instead of Condition (C) (iii), then

   $$\text{P}\left(\mathit{sgn}(\widehat{\mathit{\beta }})\ne \mathit{sgn}({\mathit{\beta }}^o)or\widehat{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^o\right)\le 3\epsilon .$$ (4.11)

   Here note that p, γ, d^o, d^*, K_*, ε, c_*(λ₂) and c^*(λ₂) are all allowed to depend on n, including the case c_*(λ₂) → 0 as long as the conditions hold as stated.

2. The statements in (i) also hold for all local minimizers β̂ of (2.6) or (4.7) satisfying #{j ∉ : β̂_j ≠ 0} + d^o ≤ d^*.

If the conditions of Theorem 2 hold with ε → 0, then (4.10) implies selection consistency of the SLS estimator and (4.11) implies sign consistency.

Condition (C), designed to handle the noncovexity of the penalized loss, is a weaker version of Condition (B) in the sense of allowing singular Σ(λ₂). The SRC (4.9), depending on X or X̃ only through the regularized Gram matrix X̃′X̃/n = Σ(λ₂) = Σ + λ₂L, ensures that the model is identifiable in a lower d^*-dimensional space. When p > n, the smallest singular value of X is always zero. However, the requirement c_*(λ₂) > 0 only concerns d^*×d^* diagonal submatrices of Σ(λ₂), not the Gram matrix Σ of the design matrix X. We can have p ≫ n but still require d^*/d^o ≥ K_*+1 as in (4.9). Since p, d⁰, γ, d^*, K_*, c_*(λ₂) and c^*(λ₂) can depend on n, we allow the case c_*(λ₂) → 0 as long as Conditions (A) and (C) hold as stated. Thus, we allow p ≫ n but require that the model is sparse, in the sense that the number of nonzero coefficients d^o is smaller than d^*/(1 + K_*). For example, if c_*(λ₂) ≍ O(n^−α) for a small α > 0 and c^*(λ₂) ≍ O(1), then we require γ ≍ O(n^3α/2) or greater, K^* ≍ O(n^α) and d^*/d^o ≍ O(n^α) or greater. So all these quantities can depend on n, as long as the other requirements are met in Condition (C).

By examining the Conditions C(ii) and C(iii), for standardized predictors with $\left|\right|{\mathit{x}}_j\left|\right|=\sqrt{n}$, we can have log(p/ε) = o(n) or p = ε exp(o(n)) as long as condition C(ii) is satisfied. As in Zhang (2010), under a somewhat stronger version of Condition C, Theorem 2 can be extended to quadratic spline concave penalties satisfying $\rho (t;{\lambda }_1,\gamma )={\lambda }_1^2\rho (t/\lambda ;\gamma )$ with a penalty function satisfying (∂/∂t)ρ(t; γ) = 1 at t = 0+ and 0 for t > γ.

Also, comparing our results with the selection consistency results of Hebiri and Geer (2010) on the smoothed ℓ₁+ℓ₂-penalized methods, our conditions tend to be weaker. Notably, Hebiri and Geer (2010) require an condition on the Gram matrix which assumes that the correlations between the truly relevant variables and those which are not are small. No such assumption is required for our selection consistency results. In addition, our selection results are stronger in the sense that the SLS estimator is not only sign consistent, but also equal to the oracle Laplacian shrinkage estimator with high probability. In general, similar results are not available with the use of the ℓ₁ penalty for sparsity.

Theorem 2 shows that the SLS estimator automatically adapts to the sparseness of the p-dimensional model and the denseness of a true submodel. From a sparse p-model, it correctly selects the true underlying model . This underlying model is a dense model in the sense that all its coefficients are nonzero. In this dense model, the SLS estimator behaves like the oracle Laplacian shrinkage estimator in (4.1). As in the convex penalized loss setting, here the results do not require a correct specification of a population correlation structure of the predictors.

4.3 Unbiased Laplacian and variance reduction

There are two natural questions concerning the SLS. First, what are the benefits from introducing the Laplacian penalty? Second, what kind of Laplacian L constitutes a reasonable choice? Since the SLS estimator is equal to the oracle Laplacian estimator with high probability by Theorem 1 or 2, these questions can be answered by examining the oracle Laplacian shrinkage estimator (4.1), whose nonzero part is

$${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)={\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)X_{\mathcal{O}}^{\prime }\mathit{y}/n.$$

Without the Laplacian, i.e., when λ₂ = 0, it becomes the least squares (LS) estimator,

$${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o(0)={\mathrm{\sum }}_{\mathcal{O}}^{-1}{X}_{\mathcal{O}}^{\prime }\mathit{y}/n.$$

If some of the predictors in {x_j, j ∈ } are highly correlated or | | ≥ n, the LS estimator ${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o(0)$ is not stable or unique. In comparison, as discussed below Theorem 1, (λ₂) = + λ₂ can be a full rank matrix under a reasonable condition, even if the predictors in {x_j, j ∈ } are highly correlated or | | ≥ n.

For the second question, we examine the bias of ${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)$. Since the bias of the target vector (4.4) is ${\mathit{\beta }}_{\mathcal{O}}^o-{\mathit{\beta }}_{\mathcal{O}}^{\ast }({\lambda }_2)={\lambda }_2{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o,{\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)$ is unbiased iff $L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o=0$. Therefore, in terms of bias reduction, a Laplacian L is most appropriate if the condition $L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o=0$ is satisfied. We shall say that a Laplacian L is unbiased if $L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o=0$. It follows from the discussion at the end of Subsection 4.1 that $L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o=0$ if ${\beta }_k^o={\beta }_{j_g}^o{a}_{j_g{k}_1}{a}_{k_1{k}_2}\cdots a_{k_{m}k}$, where j_g is a representative member of the clique V_g ∩ and {k₁, …, k_m, k} ⊆ V_g ∩ .

With an unbiased Laplacian, the mean square error of ${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)$ is

$$E{\left|\right|{\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)-{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}^2=\frac{{\sigma }^2}{n}\text{trace}({\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)).$$

The mean square error of (0) is

$$E{\left|\right|{\widehat{\mathit{\beta }}}_{\mathcal{O}}^o(0)-{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}^2=\frac{{\sigma }^2}{n}\text{trace}({\mathrm{\sum }}_{\mathcal{O}}^{-1}).$$

We always have $E{\left|\right|{\widehat{\mathit{\beta }}}_{\mathcal{O}}^o({\lambda }_2)-{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}^2<E{\left|\right|{\widehat{\mathit{\beta }}}_{\mathcal{O}}^o(0)-{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}^2\text{for}{\lambda }_2>0$. Therefore, an unbiased Laplacian reduces variance without incurring any bias on the estimator.

5 Laplacian shrinkage

The results in Section 4 show that the SLS estimator is equal to the oracle Laplacian shrinkage estimator with probability tending to one under certain conditions. In addition, an unbiased Laplacian reduces variance but does not increase bias. Therefore, to study the shrinkage effect of the Laplacian penalty on β̂, we can consider the oracle estimator ${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o$. To simplify the notation and without causing confusion, in this section, we study some other basic properties of the Laplacian shrinkage and compare it with the ridge shrinkage. The Laplacian shrinkage estimator is defined as

$$\stackrel{\sim }{\mathit{\beta }}({\lambda }_2)=\underset{\mathit{b}}{\text{argmin}}\left\{G(\mathit{b};{\lambda }_2)\equiv \frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\frac{1}{2}{\lambda }_2{\mathit{b}}^{\prime }L\mathit{b},\mathit{b}\in {ℝ}^q\right\}.$$ (5.1)

The following proposition shows that the Laplacian penalty shrinks a coefficient towards the center of all the coefficients connected to it.

Proposition 1

Let r̃ = y − Xβ̃.

1. $${\lambda }_2\underset{1\le j\le q}{max}{d}_j\mid {\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}/d_j\mid \le \left|\right|\stackrel{\sim }{\mathit{r}}\left|\right|\le \left|\right|\mathit{y}\left|\right|.$$
2. $${\lambda }_2\mid d_j{\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}-(d_k{\stackrel{\sim }{\beta }}_k-{\mathit{a}}_k^{\prime }\stackrel{\sim }{\mathit{\beta }})\mid \le \frac{1}{n}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|\left|\right|\mathit{y}\left|\right|.$$

Note that ${\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}/d_j={\sum }_{k=1}^q{a}_{jk}{\stackrel{\sim }{\beta }}_k/d_j={\sum }_{k=1}^{q}sgn(a_{jk})\mid a_{jk}\mid {\stackrel{\sim }{\beta }}_k/d_j)$ is a signed weighted average of the β̃_k’s connected to β̃_j, since d_j = Σ_k |a_jk|. Part (i) of Proposition 1 provides an upper bound on the difference between β̃_j and the center of all the coefficients connected to it. When ||r̃||/(λ₂d_j) → 0, this difference converges to zero. For standardized d_j = 1, part (ii) implies that the difference between the centered β̃_j and β̃_k converges to zero when ||x_j − x_k|| ||y||/(λ₂n) → 0.

When there are certain local structures in the adjacency matrix A, shrinkage occurs at the local level. As an example, we consider the adjacency matrix based on partition of the 16 predictors into 2r-balls defined in (3.1). Correspondingly, the index set {1, …, q} is divided into disjoint neighborhoods/cliques V₁, …, V_J. We consider the normalized Laplacian L = I_q − A, where I_q is a q × q identity matrix and A = diag(A₁, …,A_J) with $A_g=v_g^{-1}{\mathbf{1}}_g^{\prime }\mathbf{1}$. Here v_g = |V_g|, 1 ≤ g ≤ J. Let b_g = (b_j, j ∈ V_g)′. We can write the objective function as

$$G(\mathit{b};{\lambda }_2)=\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\frac{1}{2}{\lambda }_2\sum _{g=1}^J{\mathit{b}}_g^{\prime }({\mathit{I}}_g-v_g^{-1}{\mathbf{1}}_g^{\prime }{\mathbf{1}}_g){\mathit{b}}_g.$$ (5.2)

For the Laplacian shrinkage estimator based on this criterion, we have the following grouping properties.

Proposition 2

1. For any j, k ∈ V_g, 1 ≤ g ≤ J,

   $${\lambda }_2\mid {\stackrel{\sim }{\beta }}_j-{\stackrel{\sim }{\beta }}_k\mid \le \frac{1}{n}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|·\left|\right|\mathit{y}\left|\right|,j,k\in V_g.$$
2. Let β̄_g be the average of the estimates in V_g. For any j ∈ V_g and k ∈ V_h, g ≠ h,

   $${\lambda }_2\mid {\stackrel{\sim }{\beta }}_j-{\overline{\beta }}_g-({\stackrel{\sim }{\beta }}_k-{\overline{\beta }}_h)\mid \le \frac{1}{n}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|·\left|\right|\mathit{y}\left|\right|,j\in V_g,k\in V_h.$$

This proposition characterizes the smoothing effect and grouping property of the Laplacian penalty in (5.2). Consider the case ||y||²/n = O(1). Part (i) implies that, for j and k in the same neighborhood and λ₂ > 0, the difference β̃_j − β̃_k → 0 if ||x_j − x_k||/(λ₂n^1/2) → 0. Part (ii) implies that, for j and k in different neighborhoods and λ₂ > 0, the difference between the centered β̃_j and β̃_k converges to zero if ||x_j − x_k||/(λ₂n^1/2) → 0.

We now compare the Laplacian shrinkage and ridge shrinkage. The discussion at the end of Section 4 about the requirement for the unbiasedness of Laplacian can be put in a wider context when a general positive definite or semidefinite matrix Q is used in the place of L. This wider context includes the Laplacian shrinkage and ridge shrinkage as special cases. Specifically, let

$${\widehat{\mathit{\beta }}}_Q(\lambda ,\gamma )=\underset{\mathit{b}}{\text{argmin}}\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma )+\frac{1}{2}{\lambda }_2{\mathit{b}}^{\prime }Q\mathit{b}.$$

For Q = I_p, β̂_Q becomes the Mnet estimator (Huang et al. 2010). With some modifications on the conditions in Theorem 1 or Theorem 2, it can be shown that β̂_Q is equal to the oracle estimator defined as

$${\widehat{\mathit{\beta }}}_Q^o({\lambda }_2)=\underset{\mathit{b}}{\text{argmin}}\left\{\frac{1}{2n}{\left|\right|\mathit{y}-X\mathit{b}\left|\right|}^2+\frac{1}{2}{\mathit{b}}^{\prime }Q\mathit{b},b_j=0,j\notin \mathcal{O}\right\}.$$

Then in a way similar to the discussion in Section 4, β̂_Q is nearly unbiased iff $Q_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o=0$. Therefore, for $\left|\right|{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|\ne 0$, Q_O must be a rank deficient matrix, which in turn implies that Q must be rank deficient. Note that any Laplacian L is rank deficient. This rank deficiency requirement excludes the ridge penalty with Q = I_p. For the ridge penalty to yield an unbiased estimator, it must hold that ||β^o|| = 0 in the underlying model.

We now give a simple example that illustrates the basic characteristics of Laplacian shrinkage and its differences from ridge shrinkage.

Example 5.1

Consider a linear regression model with two predictors satisfying ||x_j||² = n, j = 1, 2. The Laplacian shrinkage and ridge estimators are defined as

$$({\widehat{b}}_{L1}({\lambda }_2),{\widehat{b}}_{L2}({\lambda }_2))=\underset{b_1,b_2}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{(y_i-x_{i1}{b}_1-x_{i2}{b}_2)}^2+\frac{1}{2}{\lambda }_2{(b_1-b_2)}^2,$$

and

$$({\widehat{b}}_{R1}({\lambda }_2),{\widehat{b}}_{R2}({\lambda }_2))=\underset{b_1,b_2}{\text{argmin}}\frac{1}{2n}\sum _{i=1}^n{(y_i-x_{i1}{b}_1-x_{i2}{b}_2)}^2+\frac{1}{2}{\lambda }_2(b_1^2+b_2^2).$$

Denote r₁ = cor(x₁, y), r₂ = cor(x₂, y) and r₁₂ = cor(x₁, x₂). The Laplacian shrinkage estimates are

$${\widehat{b}}_{L1}({\lambda }_2)=\frac{(1+{\lambda }_2)r_1-(r_{12}-{\lambda }_2)r_2}{{(1+{\lambda }_2)}^2-{(r_{12}-{\lambda }_2)}^2},{\widehat{b}}_{L2}({\lambda }_2)=\frac{(1+{\lambda }_2)r_2-(r_{12}-{\lambda }_2)r_1}{{(1+{\lambda }_2)}^2-{(r_{12}-{\lambda }_2)}^2}.$$

Let

$${\widehat{b}}_{\mathit{ols}1}=\frac{r_1-r_{12}{r}_2}{1-r_{12}^2},{\widehat{b}}_{\mathit{ols}2}=\frac{r_2-r_{12}{r}_1}{1-r_{12}^2},{\widehat{b}}_L(\infty )=\frac{r_1+r_2}{2(1+r_{12})},$$

where (b̂_ols1, b̂_ols2) is the ordinary least squares (OLS) estimator for the bivariate regression, b̂_L(∞) is the OLS estimator that assumes the two coefficients are equal, that is, it minimizes ${\sum }_{i=1}^n{(y_i-(x_{i1}+x_{i2})b)}^2$. Let w_L = (2λ₂)/(1−r₁₂+2λ₂). After some simple algebra, we have

$${\widehat{b}}_{L1}({\lambda }_2)=(1-w_L){\widehat{b}}_{\mathit{ols}1}+w_L{\widehat{b}}_L(\infty )\text{and}{\widehat{b}}_{L2}({\lambda }_2)=(1-w_L){\widehat{b}}_{\mathit{ols}2}+w_L{\widehat{b}}_L(\infty ).$$

Thus for any fixed λ₂, b̂_L(λ₂) is a weighted average of b̂_ols and b̂_L(∞) with the weights depending on λ₂. When λ₂ → ∞, b̂_L1 → b̂_L(∞) and b̂_L2 → b̂_L(∞). Therefore, the Laplacian penalty shrinks the OLS estimates towards a common value, which is the OLS estimate assuming equal regression coefficients.

Now consider the ridge regression estimator. We have

$${\widehat{b}}_{R1}({\lambda }_2)=\frac{(1+{\lambda }_2)r_1-r_{12}{r}_2}{{(1+{\lambda }_2)}^2-r_{12}^2}\text{and}{\widehat{b}}_{R2}({\lambda }_2)=\frac{(1+{\lambda }_2)r_2-r_{12}{r}_1}{{(1+{\lambda }_2)}^2-r_{12}^2}.$$

The ridge estimator converges to zero as λ₂ → ∞. For it to converge to a nontrivial solution, we need to rescale it by a factor of 1+λ₂. Let $w_R=\lambda /(1+\lambda -r_{12}^2)$. Let b̂_u1 = r₁ and b̂_u2 = r₂. Because $n^{-1}{\sum }_{i=1}^n{x}_{i1}^2=1$ and $n^{-1}{\sum }_{i=1}^n{x}_{i2}^2=1$, r₁ and r₂ are also the OLS estimators of univariate regressions of y on x₁ and y on x₂, respectively. We can write

$$\begin{array}{l}(1+{\lambda }_2){\widehat{b}}_{R1}({\lambda }_2)=c_{{\lambda }_2}(1-w_R){\widehat{b}}_{\mathit{ols}1}+c_{\lambda }{w}_R{\widehat{b}}_{u1},\\ (1+{\lambda }_2){\widehat{b}}_{R2}({\lambda }_2)=c_{{\lambda }_2}(1-w_R){\widehat{b}}_{\mathit{ols}2}+c_{\lambda }{w}_R{\widehat{b}}_{u2},\end{array}$$

where $c_{{\lambda }_2}=\{{(1+{\lambda }_2)}^2-(1+\lambda )r_{12}^2\}/\{{(1+{\lambda }_2)}^2-r_{12}^2\}$. Note that c_λ2 ≈ 1. Thus (1+λ₂) b̂_R is a weighted average of the OLS and the univariate regression estimators. The ridge penalty shrinks the (rescaled) ridge estimates towards individual univariate regression estimates.

6 Simulation studies

We use a coordinate descent algorithm to compute the SLS estimate. This algorithm optimizes a target function with respect to a single parameter at a time and iteratively cycles through all parameters until convergence. This algorithm was originally proposed for criterions with convex penalties such as LASSO (Fu 1998; Genkin et al. 2004; Friedman et al. 2007; Wu and Lange 2007). It has been proposed to calculate the MCP estimates (Breheny and Huang 2011). Detailed steps of this algorithm for computing the SLS estimates can be found in the technical report accompanying this paper (Huang et al. 2010).

In simulation studies, we consider the following ways of defining the adjacency measure. (N.1) a_jk = I(r_jk > r) and s_jk = 1. Here the cutoff r is computed as 3.09 using the approach described in Section 3 with a p-value of 10⁻³; (N.2) a_jk = I(|r_jk| > r) and s_jk = sgn(r_jk). Here the cutoff r is computed as 3.29 using the approach described in Section 3 with a p-value of 10⁻³; (N.3) a_jk = max(0, r_jk)^α and s_jk = 1. We set α = 6, which satisfies the scale-free topology criteria (Zhang and Horvath 2005); (N.4) $a_{jk}=r_{jk}^{\alpha }$ and s_jk = sgn(r_jk). We set α = 6.

The penalty levels λ₁ and λ₂ are selected using V-fold cross validation. In our numerical study, we set V = 5. To reduce computational cost, we search over the discrete grid of 2^{…−1, −0.5,0,0.5…}. For comparison, we also consider the MCP estimate and the approach proposed in Daye and Jeng (2009; referred to as D-J hereafter). Both the SLS and MCP involve the regularization parameter γ. For MCP, Zhang (2010) suggested using $\gamma =2/(1-{max}_{j\ne k}\mid x_j^{\prime }{x}_k\mid /n)$ for standardized covariates. The average γ value of this choice is 2.69 in his simulation studies. The simulation studies in Breheny and Huang (2009) suggest that γ = 3 is a reasonable choice. We have experimented with different γ values and reached the same conclusion. Therefore, we set γ = 3.

We set n = 100 and p = 500. Among the 500 covariates, there are 100 clusters, each with size 5. We consider two different correlation structures. (I) Covariates in different clusters are independent, whereas covariates i and j within the same cluster have correlation coefficients ρ^|i−j|; and (II) Covariates i and j have correlation coefficients ρ^|i−j|. Under structure I, zero and nonzero effects are independent, whereas under structure II, they are correlated. Covariates have marginal normal distributions with mean zero and variance one. We consider different levels of correlation with ρ = 0.1, 0.5, 0.9. Among the 500 covariates, the first 25 (5 clusters) have nonzero regression coefficients. We consider the following scenarios for nonzero coefficients: (a) all the nonzero coefficients are equal to 0.5; and (b) the nonzero coefficients are randomly generated from the uniform distribution on [0.25, 0.75]. In (a), the Laplacian matrices satisfy the unbiasedness property Lβ^o = 0 discussed in Section 4. We have experienced with other levels of nonzero regression coefficients and reached similar conclusions.

We examine the accuracy of identifying nonzero covariate effects and the prediction performance. For this purpose, for each simulated dataset, we simulate an independent testing dataset with sample size 100. We conduct cross validation (for tuning parameter selection) and estimation using the training set only. We then make prediction for subjects in the testing set and compute the PMSE (prediction mean squared error).

We simulate 500 replicates and present the summary statistics in Table 1. We can see that the MCP performs satisfactorily when the correlation is small. However, when the correlation is high, it may miss a considerable number of true positives and have large prediction errors. The D-J approach, which can also accommodate the correlation structure, is able to identify all the true positives. However, it also identifies a large number of false positives, causing by the over-selection of the Lasso penalty. The proposed SLS approach outperforms the MCP and D-J methods in the sense that it has smaller empirical false discovery rates with comparable false negative rates. It also has significantly smaller prediction errors.

Table 1.

Simulation study: median based on 500 replicates. In each cell, the three numbers are positive findings, true positives, and PMSE ×100, respectively.

Coefficient	ρ	MCP	D-J				SLS
			N.1	N.2	N.3	N.4	N.1	N.2	N.3	N.4
		Correlation structure I
0.5	0.1	27 25 41.33	61 25 125.34	53 25 46.64	55 25 60.14	59 25 51.24	27 25 40.53	27 25 39.84	26 25 41.74	27 25 39.34
	0.5	28 25 54.10	51 25 66.38	67 25 66.84	72 25 56.22	63 25 53.43	27 25 37.71	28 25 39.18	28 25 33.87	27 25 36.00
	0.9	22 15 137.52	66 25 55.51	55 25 56.94	61 25 49.22	74 25 51.41	29 25 48.89	28 25 49.96	29 25 45.16	27 25 41.49
U[.25, .75]	0.1	37 25 52.24	72 25 54.28	61 25 88.00	59 25 70.00	78 25 60.51	33 25 51.80	36 25 52.19	30 25 53.03	30 25 52.22
	0.5	29 24 65.12	66 25 78.76	54 25 72.34	63 25 63.55	57 25 66.33	28 25 42.24	28 25 43.96	27 24 54.72	28 24 58.77
	0.9	17 13 152.42	67 25 63.43	62 25 57.30	50 25 53.88	74 25 57.98	29 25 47.73	29 25 49.14	27 25 48.49	28 25 50.83
		Correlation structure II
0.5	0.1	26 25 38.22	62 25 121.69	58 25 117.10	63 25 127.34	72 25 122.34	27 25 40.33	27 25 40.65	27 25 41.49	27 25 37.40
	0.5	29 25 53.01	52 25 55.99	49 25 62.04	66 25 62.70	65 25 64.41	27 25 36.97	28 25 39.47	28 25 38.53	27 25 39.53
	0.9	15 13 140.69	48 25 55.75	34 25 56.71	32 25 60.27	38 25 59.78	29 25 66.79	29 25 60.52	29 25 57.91	30 25 60.19
U[.25, .75]	0.1	37 25 54.31	77 25 60.02	72 25 66.14	74 25 78.32	66 25 74.50	29 25 50.05	32 25 51.34	37 25 50.74	29 25 49.47
	0.5	27 24 57.66	74 25 61.71	66 25 67.54	75 25 62.01	74 25 66.91	28 25 44.92	28 25 46.65	28 25 41.35	28 25 41.17
	0.9	14 13 136.49	33 25 61.50	35 25 55.08	34 25 54.54	38 25 60.67	29 25 56.87	29 25 57.03	30 25 53.28	30 25 56.79

6.1 Application to a microarray study

In the study reported in Scheetz et al. (2006), F1 animals were intercrossed and 120 twelve-week-old male offspring were selected for tissue harvesting from the eyes and microarray analysis using the Affymetric GeneChip Rat Genome 230 2.0 Array. The intensity values were normalized using the RMA (robust multi-chip averaging, Bolstad 2003, Irizzary 2003) method to obtain summary expression values for each probe set. Gene expression levels were analyzed on a logarithmic scale. For the probe sets on the array, we first excluded those that were not expressed in the eye or that lacked sufficient variation. The definition of expressed was based on the empirical distribution of RMA normalized values. For a probe set to be considered expressed, the maximum expression value observed for that probe among the 120 F2 rats was required to be greater than the 25th percentile of the entire set of RMA expression values. For a probe to be considered “sufficiently variable,” it had to exhibit at least 2-fold variation in expression level among the 120 F2 animals.

We are interested in finding the genes whose expression are most variable and correlated with that of gene TRIM32. This gene was recently found to cause Bardet-Biedl syndrome (Chiang et al. 2006), which is a genetically heterogeneous disease of multiple organ systems including the retina. One approach to find the genes related to TRIM32 is to use regression analysis. Since it is expected that the number of genes associated with gene TRIM32 is small and since we are mainly interested in genes whose expression values across samples are most variable, we conduct the following initial screening. We compute the variances of gene expressions and select the top 1000. We then standardize gene expressions to have zero mean and unit variance.

We analyze data using the MCP, D-J, and proposed approach. In cross validation, we set V = 5. The numbers of genes identified are MCP: 23, D-J: 31 (N.1), 41 (N.2), 34 (N.3), 30 (N.4), SLS: 25 (N.1), 26 (N.2), 16 (N.3) and 17 (N.4), respectively. More detailed results are available from the authors. Different approaches and different ways of defining the adjacency measure lead to the identification of different genes. As expected, the SLS identifies shorter lists of genes than the D-J, which may lead to more parsimonious models and more focused hypothesis for confirmation. As the proposed approach pays special attention to the correlation among genes, we also compute the median of the absolute values of correlations among the identified genes, which are MCP: 0.171, D-J: 0.201 (N.1), 0.207 (N.2), 0.215 (N.3), 0.206 (N.4), SLS: 0.247 (N.1), 0.208 (N.2), 0.228 (N.3), 0.212 (N.4). The D-J and SLS, which incorporate correlation in the penalty, identify genes that are more strongly correlated than the MCP. The SLS identified genes have slightly higher correlations than those identified by D-J.

Unlike in simulation study, we are not able to evaluate true and false positives. This limitation is shared by most existing studies. We use the following V-fold (V=5) cross validation based approach to evaluate prediction. (a) Randomly split data into V-subsets with equal sizes; (b) Remove one subset from data; (c) Conduct cross validation and estimation using the rest V − 1 subsets; (d) Make prediction for the one removed subset; (e) Repeat Steps (b)–(d) over all subsets and compute the prediction error. The sums of squared prediction errors are MCP: 1.876; D-J: 1.951 (N.1), 1.694 (N.2), 1.534 (N.3) and 1.528 (N.4); SLS: 1.842 (N.1), 1.687 (N.2), 1.378 (N.3) and 1.441 (N.4), respectively. The SLS has smaller cross validated prediction errors, which may indirectly suggest better selection properties.

7 Discussion

In this article, we propose the SLS method for variable selection and estimation in high-dimensional data analysis. The most important feature of the SLS is that it explicitly incorporates the graph/network structure in predictors into the variable selection procedure through the Laplacian quadratic. It provides a systematic framework for connecting penalized methods for consistent variable selection and those for network and correlation analysis. As can be seen from the methodological development, the application of the SLS variable selection is relatively independent of the graph/network construction. Thus, although graph/network construction is of significant importance, it is not the focus of this study and not thoroughly pursued.

An important feature of the SLS method is that it incorporates the correlation patterns of the predictors into variable selection through the Laplacian quadratic. We have considered two simple approaches for determining the Laplacian based on dissimilarity and similarity measures. Our simulation studies demonstrate that incorporating correlation patterns improves selection results and prediction performance. Our theoretical results on the selection properties of the SLS are applicable to a general class of Laplacians and do not require the underlying graph for the predictors to be correctly specified.

We provide sufficient conditions under which the SLS estimator possesses an oracle property, meaning that it is sign consistent and equal to the oracle Laplacian shrinkage estimator with high probability. We also study the grouping properties of the SLS estimator. Our results show that the SLS is adaptive to the sparseness of the original p-dimensional model with p ≫ n and the denseness of the underlying d^o-dimensional model, where d^o < n is the number of nonzero coefficients. The asymptotic rates of the penalty parameters are derived. However, as in many recent studies, it is not clear whether the penalty parameters selected using cross validation or other procedures can match the asymptotic rate. This is an important and challenging problem that requires further investigation, but is beyond the scope of the current paper. Our numerical study shows a satisfactory finite-sample performance of the SLS. Particularly, we note that the cross validation selected tuning parameters seem sufficient for our simulated data. We are only able to experiment with four different adjacency measures. It is not our intention to draw conclusions on different ways of defining adjacency. More adjacency measures are hence not explored.

We have focused on the linear regression model in this article. However, the SLS method can be applied to general linear regression models. Specifically, for general linear models, the SLS criterion can be formulated as

$$\frac{1}{n}\sum _{i=1}^n\ell (y_i,b_0+\sum _j{x}_{ij}{b}_j)+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma )+\frac{1}{2}{\lambda }_2\sum _{1\le j<k\le p}\mid a_{jk}\mid {(b_j-s_{jk}{b}_k)}^2,$$

where ℓ is a given loss function. For instance, for generalized linear models such as logistic regression, we can take ℓ to be the negative log-likelihood function. For Cox regression, we can use the negative partial likelihood as the loss function. Computationally, for loss functions other than least squares, the coordinate descent algorithm can be applied iteratively to quadratic approximations to the loss function. However, further work is needed to study theoretical properties of the SLS estimators for general linear models.

There is a large literature on the analysis of network data and much work has also been done on estimating sparse covariance matrices in high-dimensional settings. See for example, Zhang and Horvath (2005), Chung and Lu (2006), Meinshausen and Bühlmann (2006), Yuan and Lin (2007), Friedman, Hastie and Tibshirani (2008), Fan, Feng and Wu (2009), among others. It would be useful to study ways to incorporate these methods and results into the proposed SLS approach. In some problems such as genomic data analysis, partial external information may also be available on the graphical structure of some genes used as predictors in the model. It would be interesting to consider approaches for combining external information on the graphical structure with existing data in constructing the Laplacian quadratic penalty.

8 Appendix

In the appendix, we give proofs of Theorems 1 and 2 and Propositions 1 and 2.

Proof of Theorem 1

Since c_min(λ₂) > 1/γ, the criterion (2.2) is strictly convex and its minimizer is unique. Let $\stackrel{\sim }{X}=\stackrel{\sim }{X}({\lambda }_2)=\sqrt{n}{(\mathrm{\sum }+{\lambda }_{2}L)}^{1/2}$,ỹ = ỹ(λ₂) = X̃⁻¹X′y and

$$\stackrel{\sim }{M}(\mathit{b};\lambda ,\gamma )={(2n)}^{-1}{\left|\right|\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}\mathit{b}\left|\right|}^2+\sum _{j=1}^p\rho (\mid b_j\mid ;{\lambda }_1,\gamma ).$$

Since X̃′(X̃/n, ỹ) = (Σ+λ₂L, X′y), M(b; λ, γ) − M̃(b; λ, γ) = (||y||² − ||ỹ ||²)/(2n) does not depend on b. Thus, β̂ is the minimizer of M̃ (b; λ, γ).

Since $\mid {\widehat{\beta }}_j^o\mid \ge \gamma {\lambda }_1$ gives ${\rho }^{\prime }(\mid {\widehat{\beta }}_j^o\mid ;{\lambda }_1)=0$, the KKT conditions hold for M̃ (b; λ, γ) at β̂(λ) = β̂^o(λ) in the intersection of the events

$${\mathrm{\Omega }}_1=\{{\left|\right|{\stackrel{\sim }{X}}_{{\mathcal{O}}^c}^{\prime }(\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}{\widehat{\mathit{\beta }}}^o)/n\left|\right|}_{\infty }\le {\lambda }_1\},{\mathrm{\Omega }}_2=\{\underset{j\in \mathcal{O}}{min}sgn({\beta }_j^{\ast }){\widehat{\beta }}_j^o\ge \gamma {\lambda }_1\}.$$ (8.1)

Let ε̃^* = ỹ − X̃β^* = ε̃ + Eε̃^* with ε̃ = ỹ − Eỹ. Since X̃′ỹ = X′y and both β^o and β^* are supported in ,

$$\begin{array}{l}{\stackrel{\sim }{X}}_B^{\prime }{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast }/n=X_B^{\prime }X{\mathit{\beta }}^o/n-{\stackrel{\sim }{X}}_B^{\prime }\stackrel{\sim }{X}{\mathit{\beta }}^{\ast }/n\\ ={\mathrm{\sum }}_{B,\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o-{\mathrm{\sum }}_{B,\mathcal{O}}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o\\ ={\lambda }_2\{{\mathrm{\sum }}_{B,\mathcal{O}}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}-L_{B,\mathcal{O}}\}{\mathit{\beta }}_{\mathcal{O}}^o,\end{array}$$ (8.2)

which describes the effect of the bias of β̂^o on the gradient in the linear model ỹ = X̃β^*+ε̃^*. Since ${\stackrel{\sim }{X}}_{\mathcal{O}}^{\prime }{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast }/n=0$, we have || X̃′Eε̃^*/n||_∞ = λ₂C₂.

Since X̃′ε̃ = X̃′ỹ − EX̃′ỹ = X′y − EX′y = X′ε, (8.2) gives

$${\mathrm{\Omega }}_1\subseteq \{{\left|\right|X_{{\mathcal{O}}^c}^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }<{\lambda }_1-{\lambda }_2{C}_2\}.$$ (8.3)

Since ${\widehat{\mathit{\beta }}}_{\mathcal{O}}^o={\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)X_{\mathcal{O}}^{\prime }\mathit{y}/n$ can be written as ${\mathit{\beta }}_{\mathcal{O}}^{\ast }+{({(v_j/n)}^{1/2}{\mathit{u}}_j^{\prime }\mathit{\epsilon },j\in \mathcal{O})}^{\prime }$, where ||u_j|| = 1 and {v_j, j ∈ } are the diagonal elements of ${\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}\{{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)\}$. Thus,

$${\mathrm{\Omega }}_2^c\subseteq {\cup }_{j\in \mathcal{O}}\left\{sgn({\beta }_j^{\ast }){\mathit{u}}_j^{\prime }\mathit{\epsilon }\ge {(n/v_j)}^{1/2}\mid {\widehat{\beta }}_j^{\ast }\mid \ge \sigma \sqrt{2log(\mid \mathcal{O}\mid /\epsilon )}\right\}.$$ (8.4)

Since ${\lambda }_1\ge {\lambda }_2{C}_2+\sigma \sqrt{2log(p/\epsilon )}{max}_{j\le p}\left|\right|{\mathit{x}}_j\left|\right|/n$, the sub-Gausian condition (A) yields

$$\begin{array}{l}1-P\{{\mathrm{\Omega }}_1\cap {\mathrm{\Omega }}_2\}\le P\{{\left|\right|X_{{\mathcal{O}}^c}^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }>\sigma \sqrt{2log((p-\mid \mathcal{O}\mid )/\epsilon )}\underset{j\le p}{max}|\left|{\mathit{x}}_j\right||/n\}+\sum _{j\in \mathcal{O}}P\{sgn({\beta }_j^{\ast }){\mathit{u}}_j^{\prime }\mathit{\epsilon }\ge \sigma \sqrt{2log(\mid \mathcal{O}\mid /\epsilon )}\}\\ \le 2\mid {\mathcal{O}}^c\mid \epsilon /(p-\mid \mathcal{O}\mid )+\mid \mathcal{O}\mid \epsilon /\mid \mathcal{O}\mid =3\epsilon .\end{array}$$

The proof of (4.5) is complete, since ${\widehat{\beta }}_j^o\ne 0$ for all j ∈ in Ω₂.

For the proof of (4.6), we have ${\left|\right|{\mathit{\beta }}_{\mathcal{O}}^{\ast }-{\mathit{\beta }}_{\mathcal{O}}^o\left|\right|}_{\infty }={\lambda }_2{C}_1$ due to

$${\mathit{\beta }}_{\mathcal{O}}^{\ast }-{\mathit{\beta }}_{\mathcal{O}}^o={\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2){\mathrm{\sum }}_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o-{\mathit{\beta }}_{\mathcal{O}}^o=-{\lambda }_2{\mathrm{\sum }}_{\mathcal{O}}^{-1}({\lambda }_2)L_{\mathcal{O}}{\mathit{\beta }}_{\mathcal{O}}^o.$$ (8.5)

It follows that the condition on β_* implies Conditon (B) (iii) with $sgn({\mathit{\beta }}_{\mathcal{O}}^{\ast })=sgn({\mathit{\beta }}_{\mathcal{O}}^o)=sgn({\widehat{\mathit{\beta }}}_{\mathcal{O}}^o)$ in Ω₂.

Proof of Theorem 2

For m ≥ 1 and vectors u in the range of X̃, define

$$\stackrel{\sim }{\zeta }(\mathit{v};m,\mathcal{O},{\lambda }_2)=max\left\{\frac{{\left|\right|({\stackrel{\sim }{P}}_B-{\stackrel{\sim }{P}}_{\mathcal{O}})\mathit{v}\left|\right|}_2}{{(mn)}^{1/2}}:\mathcal{O}\subseteq B\subseteq \{1,\dots p\},\mid B\mid =m+\mid \mathcal{O}\mid \right\},$$ (8.6)

where ${\stackrel{\sim }{P}}_B={\stackrel{\sim }{X}}_B{({\stackrel{\sim }{X}}_B^{\prime }{\stackrel{\sim }{X}}_B)}^{-1}{\stackrel{\sim }{X}}_B^{\prime }$. Here ζ̃ depends on λ₂ through P̃. Since β̂(λ) is the MC+ estimator based on data (X̃, ỹ) at penalty level λ₁ and (4.9) holds for Σ(λ₂) = X̃′X̃/n, the proof of Theorem 5 in Zhang (2010) gives β̂(λ) = β̂^o (λ) in the event $\mathrm{\Omega }={\cap }_{j=1}^3{\mathrm{\Omega }}_j$, where ${\mathrm{\Omega }}_1=\left\{{\left|\right|{\stackrel{\sim }{X}}_{{\mathcal{O}}^c}^{\prime }(\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}{\widehat{\mathit{\beta }}}^o)/n\left|\right|}_{\infty }\le {\lambda }_1\right\}$ is as in (8.1) and

$${\mathrm{\Omega }}_2=\left\{\underset{j\in \mathcal{O}}{min}sgn({\beta }_j^{\ast }){\widehat{\beta }}_j^o>\gamma (2\sqrt{c^{\ast }}{\lambda }_1)\right\},{\mathrm{\Omega }}_3=\left\{\zeta (\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}{\mathit{\beta }}^{\ast };d^{\ast }-\mid \mathcal{O}\mid ,\mathcal{O},{\lambda }_2)\le {\lambda }_1\right\}.$$

Note that (λ_1,ε, λ_2,ε, λ_3,ε, α) in Zhang (2010) is identified with (λ₁, $2\sqrt{c^{\ast }}{\lambda }_1$, λ₁, 1/2) here.

Let ε̃^* = ỹ − X̃β^* = ε̃ + Eε̃^* with ε̃ = ỹ − Eỹ. Since X̃′ỹ = X′y, (8.2) still holds with ||X̃′Eε̃^*/n||_∞ = λ₂C₂. Since X̃′ε̃ = X′y − EX′y = X′ε, (8.2) still gives (8.3). A slight modification of the argument for (8.4) yields

$${\mathrm{\Omega }}_2^c\subseteq {\cup }_{j\in \mathcal{O}}\left\{sgn({\beta }_j^{\ast }){\mathit{u}}_j^{\prime }\mathit{\epsilon }\ge {(n/v_j)}^{1/2}(\mid {\beta }_j^{\ast }\mid -\gamma (2\sqrt{c^{\ast }}{\lambda }_1))\ge \sigma \sqrt{2log(\mid \mathcal{O}\mid /\epsilon )}\right\}.$$ (8.7)

For |B| ≤ d^*, we have $\left|\right|{\stackrel{\sim }{P}}_B{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast }\left|\right|/\sqrt{n}=\left|\right|{\mathrm{\sum }}_B^{-1/2}({\lambda }_2){\stackrel{\sim }{X}}_B^{\prime }{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast }\left|\right|/n\le {\left|\right|{\stackrel{\sim }{X}}_B^{\prime }{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast }/n\left|\right|}_{\infty }\sqrt{\mid B\mid /c_{\ast }({\lambda }_2)}$ and $\left|\right|{\stackrel{\sim }{P}}_B\stackrel{\sim }{\mathit{\epsilon }}\left|\right|/\sqrt{n}=\left|\right|{\mathrm{\sum }}_B^{-1/2}({\lambda }_2){\stackrel{\sim }{X}}_B^{\prime }\stackrel{\sim }{\mathit{\epsilon }}\left|\right|/n\le {\left|\right|{\stackrel{\sim }{X}}_B^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }\sqrt{\mid B\mid /c_{\ast }({\lambda }_2)}$. Thus, by (8.6)

$$\zeta (\stackrel{\sim }{\mathit{y}}-\stackrel{\sim }{X}{\mathit{\beta }}^{\ast };d^{\ast }-\mid \mathcal{O}\mid ,\mathcal{O},{\lambda }_2)=\zeta (\stackrel{\sim }{\mathit{\epsilon }}+{E\stackrel{\sim }{\mathit{\epsilon }}}^{\ast };d^{\ast }-\mid \mathcal{O}\mid ,\mathcal{O},{\lambda }_2)\le \frac{({\left|\right|X^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }+{\lambda }_2{C}_2)\sqrt{d^{\ast }}}{\sqrt{(d^{\ast }-\mid \mathcal{O}\mid )c_{\ast }({\lambda }_2)}}.$$

Since | | ≤ d^*/(K_* + 1), this gives

$${\mathrm{\Omega }}_3\subseteq \{{\left|\right|X^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }<\sqrt{c_{\ast }({\lambda }_2)K_{\ast }/(K_{\ast }+1)}{\lambda }_1-{\lambda }_2{C}_2\}.$$ (8.8)

Since $max\{1,\sqrt{c_{\ast }({\lambda }_2)K_{\ast }/(K_{\ast }+1)}\}{\lambda }_1\ge {\lambda }_2{C}_2+\sigma \sqrt{2log(p/\epsilon )}{max}_{j\le p}\left|\right|{\mathit{x}}_j\left|\right|/n$, (8.3), (8.7), (8.8) and Condition (A) imply

$$\begin{array}{l}1-P\{{\mathrm{\Omega }}_1\cap {\mathrm{\Omega }}_3\}+P\{{\mathrm{\Omega }}_2^c\}\le P\{{\left|\right|X^{\prime }\mathit{\epsilon }/n\left|\right|}_{\infty }>\sigma \sqrt{2log(p/\epsilon )}\underset{j\le p}{max}|\left|{\mathit{x}}_j\right||/n\}+\sum _{j\in \mathcal{O}}P\{sgn({\beta }_j^{\ast }){\mathit{u}}_j^{\prime }\mathit{\epsilon }\ge \sigma \sqrt{2log(\mid \mathcal{O}\mid /\epsilon )}\}\\ \le 2p(\epsilon /p)+{\mid \mathcal{O}\mid }_{\epsilon }/\mid \mathcal{O}\mid =3\epsilon .\end{array}$$

The proof of (4.10) is complete, since ${\widehat{\beta }}_j^o\ne 0$ for all j ∈ in Ω₂. We omit the proof of (4.11) since it is identical to that of (4.6).

Proof of Proposition 1

The β̃ satisfies

$$-\frac{1}{n}{\mathit{x}}_j^{\prime }(\mathit{y}-X\stackrel{\sim }{\mathit{\beta }})+{\lambda }_2(d_j{\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }})=0,1\le j\le q.$$ (8.9)

Therefore, by Cauchy-Schwarz and using ||x_j||² = n, we have

$${\lambda }_2\underset{1\le j\le q}{max}\mid d_j{\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}\mid \le \frac{1}{n}\underset{1\le j\le q}{max}\mid {\mathit{x}}_j^{\prime }(\mathit{y}-X\stackrel{\sim }{\mathit{\beta }})\mid \le \frac{1}{\sqrt{n}}\left|\right|\stackrel{\sim }{\mathit{r}}\left|\right|.$$

Now because G(β̃; λ₂) ≤ G(0; λ₂), we have ||r̃|| ≤ ||y||. This proves part (i).

For part (ii), note that we have

$${\lambda }_2(d_j{\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}-(d_k{\stackrel{\sim }{\beta }}_k-{\mathit{a}}_k^{\prime }\stackrel{\sim }{\mathit{\beta }}))=\frac{1}{n}{({\mathit{x}}_j-{\mathit{x}}_k)}^{\prime }\stackrel{\sim }{\mathit{r}}.$$

Thus

$${\lambda }_2\mid d_j{\stackrel{\sim }{\beta }}_j-{\mathit{a}}_j^{\prime }\stackrel{\sim }{\mathit{\beta }}-(d_k{\stackrel{\sim }{\beta }}_k-{\mathit{a}}_k^{\prime }\stackrel{\sim }{\mathit{\beta }})\mid \le \frac{1}{n}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|\left|\right|\stackrel{\sim }{\mathit{r}}\left|\right|.$$

Part (ii) follows.

Proof of Proposition 2

The β̃ must satisfy

$$-\frac{1}{n}{\mathit{x}}_j^{\prime }(\mathit{y}-X\stackrel{\sim }{\mathit{\beta }})+{\lambda }_2({\stackrel{\sim }{\beta }}_j-v_g^{-1}{\mathbf{1}}_g^{\prime }{\stackrel{\sim }{\mathit{\beta }}}_g)=0,j\in V_g,1\le g\le J.$$ (8.10)

Taking the difference between the jth and kth equations in (8.10) for j, k ∈ V_g, we get

$${\lambda }_2({\stackrel{\sim }{\beta }}_j-{\stackrel{\sim }{\beta }}_k)=\frac{1}{n}{({\mathit{x}}_j-{\mathit{x}}_k)}^{\prime }(\mathit{y}-X\stackrel{\sim }{\mathit{\beta }}),j,k\in V_g.$$

Therefore,

$${\lambda }_2\mid {\stackrel{\sim }{\beta }}_j-{\stackrel{\sim }{\beta }}_k\mid \le \frac{1}{n}\left|\right|{\mathit{x}}_j-{\mathit{x}}_k\left|\right|·\left|\right|\mathit{y}-X\stackrel{\sim }{\mathit{\beta }}\left|\right|,j,k\in V_g.$$

Part (i) follows from this inequality.

Define ${\overline{\beta }}_g=v_g^{-1}{\mathbf{1}}_g^{\prime }{\stackrel{\sim }{\mathit{\beta }}}_g$. This is the average of the elements in β̃_g. For any j ∈ V_g and k ∈ V_h, g ≠ h, we have

$${\lambda }_2({\stackrel{\sim }{\beta }}_j-{\stackrel{\sim }{\beta }}_g-({\stackrel{\sim }{\beta }}_k-{\overline{\beta }}_h))=\frac{1}{n}{({\mathit{x}}_j-{\mathit{x}}_k)}^{\prime }(\mathit{y}-X\stackrel{\sim }{\mathit{\beta }}),j\in V_g,k\in V_h.$$

Thus part (ii) follows. This completes the proof of Proposition 2.