Simultaneous Multiple Response Regression and Inverse Covariance Matrix Estimation via Penalized Gaussian Maximum Likelihood

Abstract

Multivariate regression is a common statistical tool for practical problems. Many multivariate regression techniques are designed for univariate response cases. For problems with multiple response variables available, one common approach is to apply the univariate response regression technique separately on each response variable. Although it is simple and popular, the univariate response approach ignores the joint information among response variables. In this paper, we propose three new methods for utilizing joint information among response variables. All methods are in a penalized likelihood framework with weighted L₁ regularization. The proposed methods provide sparse estimators of conditional inverse co-variance matrix of response vector given explanatory variables as well as sparse estimators of regression parameters. Our first approach is to estimate the regression coefficients with plug-in estimated inverse covariance matrices, and our second approach is to estimate the inverse covariance matrix with plug-in estimated regression parameters. Our third approach is to estimate both simultaneously. Asymptotic properties of these methods are explored. Our numerical examples demonstrate that the proposed methods perform competitively in terms of prediction, variable selection, as well as inverse covariance matrix estimation.

1. Introduction

Parameter estimation and variable selection are two important goals in linear regression analysis. In traditional statistical procedures, these two objectives are often achieved separately. For example, parameter estimation can be done by the least squares regression method and variable selection can be achieved by certain subset selection techniques. However, with a large number of predictors available in practice, these methods may not be feasible. When the dimension gets large, the least squares method may have an overfitting problem which reduces predictive accuracy. When the dimension is larger than the sample size, the least squares regression solution cannot even be calculated directly. In terms of variable selection, the all subset selection method can be unstable because the procedure is not continuous [3], and it can be computationally infeasible when the dimension is large. To solve these problems, a large number of methods have been proposed based on the regularization framework. Some well-known methods include the least absolute shrinkage and selection operator (LASSO) proposed by Tibshirani [18], the nonnegative garrote proposed by Breiman [2], and the smoothly clipped absolute deviation (SCAD) proposed by Fan and Li [6]. These regularized methods can help to avoid overfitting. More importantly, these techniques can perform parameter estimation and variable selection simultaneously.

With multiple response variables available, the standard approach to model them is to regress each response variable separately on the same set of explanatory variables. All marginal univariate regression procedures including the above methods can be applied to each response. However, this approach may not be optimal since they do not utilize the information among response variables. To solve this multi-response regression problem, Breiman and Friedman [4] proposed a method, called the curd and whey that uses the relationship among response variables to improve predictive accuracy. They showed that their method can outperform separate univariate regression approaches when there are correlations among the response variables. However, their method did not address the topic of variable selection. Recently, Yuan et al. [21] proposed a method based on dimension reduction. Their idea is to obtain dimension reduction by encouraging sparsity among singular values of the parameter matrix. However, their approach focuses on dimension reduction rather than variable selection. Thus, it does not give a subset of explanatory variables for each response. Variable selection can be a very important issue when the number of explanatory variables is large or when explanatory variables are highly correlated. To relate with variable selection, Turlach et al. [19] proposed a penalized method using the max-L₁ penalty to select a common subset of explanatory variables for multiple response regression. Their method aims to select a subset which can be used as predictors for all response variables. However, this assumption may be too strong when each response has different sets of explanatory variables.

Recently, Rothman et al. [16] proposed a penalized log-likelihood approach with the multivariate Gaussian assumption. In this paper, we further extend their method and propose three approaches to tackle the multiple response regression problem via utilizing the joint information among multiple response variables. To handle the problem, we need to estimate two parameter matrices, the regression parameter matrix B and the conditional inverse covariance matrix of response variables C = Σ⁻¹. The first two approaches are plug-in methods, i.e., plugging in an estimator of one parameter matrix to solve the other one. The third approach tries to jointly estimate both parameter matrices. In particular, the first proposed method maximizes a sparse penalized log-likelihood using a previously estimated inverse covariance matrix Ĉ. Similarly, the second proposed method maximizes a sparse penalized log-likelihood using a previously estimated regression parameter matrix B̂. The last proposed method simultaneously estimates regression parameters and the inverse covariance matrix by maximizing a doubly penalized joint likelihood function. These methods involve two penalty terms: the weighted L₁ penalty on the inverse covariance matrix C and the weighted L₁ penalty on the regression parameter matrix B. Note that the joint approach is more general than that of Rothman et al. [16], which used unweighted L₁ penalty terms. Our framework allows flexible weights on the penalty terms and it is more general. To handle the computational difficulty of high dimensional problems, we recommend some prescreening procedure to eliminate noise variables before further estimation.

In the following sections, we describe the new proposed methods in more details with theoretical justification and numerical examples. In Section 2, we introduce our proposed methodology. Section 3 explores the corresponding theoretical properties. Section 4 develops coordinate descent computational algorithms to obtain solutions for proposed methods. A prescreening step is suggested for the joint method to speed up the computation. Section 5 provides some brief results of our numerical examples. We conclude the paper with some discussion in Section 6. The proofs of the theorems are provided in Appendix.

2. Methodology

Consider the regression problem of p covariates and m response variables. Suppose the data contain n observations. Let y_i = (y_i1, …, y_im)^T; i = 1, …, n, be m-dimensional responses and Y = [y₁, …, y_n]^T be the n × m response matrix. Let x_i = (x_i1, …, x_ip)^T; i = 1, …, n, be p-dimensional predictors and X = [x₁, …, x_n]^T be the n × p design matrix. For simplicity of notations, let y^k = (y_1k, …, y_nk)^T be the k-th response vector (k = 1, …, m) and x^j = (x_1j, …, x_nj)^T be the j-th predictor (j = 1, …, p). Consider the following model,

$$\mathbf{Y}=\mathbf{XB}+\mathbf{e},\text{with}\mathbf{e}={[{\epsilon }_1,\dots ,{\epsilon }_n]}^T,$$

where B = {β_jk}; j = 1, …, p, k = 1, …, m, is an unknown p × m parameter matrix. The errors ε_i = (ε_i1, …, ε_im)^T; i = 1, …, n, are i.i.d. m-dimensional random vectors following a multivariate normal distribution N(0, Σ) with the nonsingular covariance matrix Σ.

Our goal is to estimate B so that we can use X to predict Y. A simple way to estimate B is to build m single response models separately and the least squares solution is denoted by B̂_S = (X^T X)⁻¹X^T Y, provided that X^T X is nonsingular. However, this approach ignores information on Σ. When Σ is diagonal, this separate modeling approach can work well. However, when Σ is not diagonal, we sometimes have strong correlations among the response variables. The separate modeling approach does not make use of the joint information among the response variables. To produce a better estimator, we consider to incorporate Σ in the estimation procedure of B. Denote Σ⁻¹ by C. If we assume that Σ is known, the log-likelihood for B conditional on X is

$$-\frac{1}{2}\text{tr}\{(\mathbf{Y}-\mathbf{XB})\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T\},$$ (1)

up to a constant not depending on B. Interestingly, although the maximum likelihood function involves Σ, the corresponding maximum likelihood estimate turns out to be identical to the least squares estimate using the separate maximum likelihood method. This implies that the maximizer of (1) does not take any advantage from the known information on Σ. However, when we impose penalties on the likelihood, the joint method can bring some advantage in estimation.

In this paper, we propose to build multivariate regression models through joint shrinkage. The goal is to utilize the joint information among the m response variables to improve estimation and prediction. Since Σ is involved in the joint estimation and it is often unknown, we consider three different approaches: two plug-in methods and the doubly penalized approach. The plug-in approach in Section 2.1 uses some estimator Ĉ for C to plug in the penalized likelihood function and then estimate B jointly. The plug-in approach in Section 2.2 estimates C after plugging in a reasonable estimator of B. The doubly penalized approach in Section 2.3 estimates C and B simultaneously via regularizing the estimation of both C and B.

For discussion, we first assume that Σ is known. To regress Y on X, we can model them separately, such as applying the LASSO for m different responses. Alternatively, we can use joint shrinkage estimation for the m response variables simultaneously. To demonstrate the difference between separate shrinkage and joint shrinkage, we consider a simple toy example for illustration. Suppose that m = 2, p = 1, and X^T X = 1. Let ${\widehat{\mathbf{B}}}_S=({\widehat{\beta }}_{11}^S,{\widehat{\beta }}_{12}^S)$ be the least squares solution and assume that both ${\widehat{\beta }}_{11}^S$ and ${\widehat{\beta }}_{12}^S$ are positive and $\mathbf{\sum }=\left(\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right)$. With the penalty parameter λ, the separate LASSO solution is given by

$$\begin{array}{l}{\widehat{\beta }}_{1m}^{\mathit{LASSO}}=\underset{{\beta }_{1m}}{\text{argmin}}\{{({\mathbf{y}}^m-\mathbf{X}{\beta }_{1m})}^T({\mathbf{y}}^m-\mathbf{X}{\beta }_{1m})+\lambda \mid {\beta }_{1m}\mid \}\\ ={[{\widehat{\beta }}_{1m}^S-\frac{\lambda }{2}]}_{+};m=1,2,\end{array}$$ (2)

where [u]₊ = u if u ≥ 0 and [u]₊ = 0 if u < 0. In the joint shrinkage estimation, however, the solution is given by

$$\underset{\mathbf{B}}{\text{argmin}}[\text{tr}\{(\mathbf{Y}-\mathbf{XB})\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T\}+\lambda \mid {\beta }_{11}\mid +\lambda \mid {\beta }_{12}\mid ].$$ (3)

We can show that (3) is equivalent to

$$\underset{\mathbf{B}}{\text{argmin}}\left[(\mathbf{B}-{\widehat{\mathbf{B}}}_S)\mathbf{C}{(\mathbf{B}-{\widehat{\mathbf{B}}}_S)}^T+\lambda \mid {\beta }_{11}\mid +\lambda \mid {\beta }_{12}\mid \right]$$ (4)

and the solution of (4) is given by

$${\widehat{\beta }}_{1m}={[{\widehat{\beta }}_{1m}^S-\frac{\lambda }{2}(1+\rho )]}_{+};m=1,2.$$ (5)

Compared with the separate LASSO solution (2), the solution (5) obtains more shrinkage if ρ is positive, while negative ρ results in less shrinkage. Figure 1 provides some insight on the reason why the amount of shrinkage changes with ρ for the joint method. Solid curves in Figure 1 are contour curves of (B – B̂_S)C(B – B̂_S)^T as the quadratic function of B and dashed lines correspond to the penalty function. When ρ is positive, the quadratic function increases along the 45° line to the horizontal axis slower than the case when ρ is zero. Note that the solution of the joint method with ρ = 0 is identical to the separate LASSO solution. Thus, the solution of (4) can be closer to the origin with more shrinkage than the solution with ρ = 0. On the other hand, the quadratic function with negative increases faster along the 45° line to the horizontal axis. Thus, the solution of (4) tends to be closer to the least squares solution than the solution with ρ = 0. Therefore, the joint method can help us to produce more accurate estimators via utilizing the joint information through C.

Figure 1.

Contour plots for the toy example to illustrate the change of shrinkage with ρ for the joint method.

We propose three approaches, including two plug-in methods and one joint method. In Sections 2.1 and 2.2, we introduce two different plug-in penalized likelihood methods, one is for multiple response regression and the other one is for inverse covariance estimation. In the plug-in method for multiple response regression, we estimate C prior to the step of regression and then use the estimator of C to produce a better estimator of B. In the plug-in method for inverse covariance estimation, we estimate B first and then estimate C with the estimator B̂ available. In Section 2.3, we estimate B and C together via double penalization. Section 2.4 provides some guidance on three proposed methods and model selection.

2.1. Plug-in Joint Weighted LASSO Estimator

To ensure that estimation of B includes the information on Σ, we propose a joint penalized likelihood method, namely the plug-in joint weighted LASSO (PWL) estimator. In particular, the corresponding penalized likelihood function is as follows

$$\text{tr}\{(\mathbf{Y}-\mathbf{XB})\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T\}+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid .$$ (6)

Here λ₁ is a tuning parameter and w_jk ≥ 0; j = 1, …, p, k = 1, …, m, are prespecified weights for the L₁-penalty of β_jk. If C is an m × m diagonal matrix with diagonal entries ( ${\sigma }_1^2,\dots ,{\sigma }_m^2$), then y¹, …, y^m are mutually independent. In that case, the minimizer of (6) is equivalent to the weighted LASSO solution obtained by applying the weighted LASSO separately to each response vector y^k with the penalty parameter ${\lambda }_1/{\sigma }_k^2(k=1,\dots ,m)$. However, if C is not diagonal, the minimizer of (6) can be different from the separate penalized likelihood method which handles each response vector y^k separately. Our numerical examples indicate that the joint method can be more accurate when the response variables are highly correlated.

In practice, C is often not available. Thus, we need to estimate it. To estimate C, we assume that ${\mathbf{z}}_i={({\mathbf{y}}_i^T,{\mathbf{x}}_i^T)}^T$ is an (m+p)-dimensional random vector following a multivariate normal distribution N(μ, Σ_y,x), where ${\mathbf{\sum }}_{\mathbf{y},\mathbf{x}}=\left(\begin{array}{cc}{\mathrm{\sum }}_{y,y}& {\mathrm{\sum }}_{y,x}\\ {\mathrm{\sum }}_{x,y}& {\mathrm{\sum }}_{x,x}\end{array}\right)$. Because Σ is the covariance matrix of y_i conditioned on x_i, it can be expressed by $\mathbf{\sum }={\mathrm{\sum }}_{y,y}-{\mathrm{\sum }}_{y,x}{\mathrm{\sum }}_{x,x}^{-1}{\mathrm{\sum }}_{x,y}$. Therefore, we can estimate Σ by first estimating Σ_y,x. To estimate Σ_y,x, we adapt the Graphical LASSO (GLASSO) method proposed by Friedman et al. [7]. The GLASSO method considers the problem of estimating the inverse covariance matrix in the context of sparse Gaussian graphical models [12]. This technique was also considered by Yuan and Lin [23], Banerjee et al. [1] and Rothman et al. [15].

The GLASSO estimator, ${\widehat{\mathbf{\sum }}}_{\mathbf{y},\mathbf{x}}^{-1}$, is given as the minimizer of the following penalized likelihood function

$$-logdet({{\mathbf{\sum }}_{\mathbf{y},\mathbf{x}}}^{-1})+\frac{1}{n}\sum _{i=1}^n{({\mathbf{z}}_i-\overline{\mathbf{z}})}^T{{\mathbf{\sum }}_{\mathbf{y},\mathbf{x}}}^{-1}({\mathbf{z}}_i-\overline{\mathbf{z}})+{\lambda }_0\left|\right|{{\mathbf{\sum }}_{\mathbf{y},\mathbf{x}}}^{-1}\left|\right|.$$ (7)

Here z̄ is the sample mean, || Σ_y,x⁻¹|| is the sum of the absolute values of the off-diagonal elements of Σ_y,x⁻¹, and λ₀ is a tuning parameter.

The PWL method is a two-step procedure. With the estimate Σ̂ available, the PWL method solves the following problem

$$\underset{\mathbf{B}}{\text{argmin}}\left[\text{tr}\left\{(\mathbf{Y}-\mathbf{XB})\widehat{\mathbf{C}}{(\mathbf{Y}-\mathbf{XB})}^T\right\}+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid \right],$$ (8)

where ${\widehat{\mathbf{\sum }}}_{\mathbf{y},\mathbf{x}}=\left(\begin{array}{cc}{\widehat{\mathrm{\sum }}}_{y,y}& {\widehat{\mathrm{\sum }}}_{y,x}\\ {\widehat{\mathrm{\sum }}}_{x,y}& {\widehat{\mathrm{\sum }}}_{x,x}\end{array}\right),\widehat{\mathbf{\sum }}={\widehat{\mathrm{\sum }}}_{y,y}-{\widehat{\mathrm{\sum }}}_{y,x}{\widehat{\mathrm{\sum }}}_{x,x}^{-1}{\widehat{\mathrm{\sum }}}_{x,y}$ and Ĉ = Σ̂⁻¹.

2.2. Plug-in Weighted Graphical LASSO Estimator

In Section 2.1, we propose a plug-in method, PWL, which estimates C first and then estimates B given Ĉ. In this section, we propose another plug-in method to estimate C. In particular, we first estimate B by using univariate regression techniques. With the estimator B̂ available, we propose a penalized likelihood method, the plug-in weighted graphical LASSO (PWGL) estimator, by solving

$$\underset{\mathbf{C}}{\text{argmin}}\left[-\mathit{n}logdet(\mathbf{C})+\text{tr}\left\{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})\mathbf{C}{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})}^T\right\}+{\lambda }_2\sum _{s\ne t}{v}_{st}\mid c_{st}\mid \right],$$ (9)

where C = {c_st}; s = 1, …, m, t = 1, …, m. Here λ₂ is a tuning parameter and v_st ≥ 0; s = 1, …, m, t = 1, …, m, are prespecified weights for the L₁ penalty of c_st.

2.3. Doubly Penalized Maximum Likelihood Estimator

In Sections 2.1 and 2.2, we propose two plug-in methods. PWL estimates C first and then estimates B given Ĉ while PWGL estimates B first and then estimates C given B̂. In this section, we propose to estimate (B, C) simultaneously. Since y_i|x_i ~ N(B^Tx_i, Σ), the log-likelihood of (B, C) conditional on X is

$$\frac{n}{2}logdet(\mathbf{C})-\frac{1}{2}\text{tr}\{(\mathbf{Y}-\mathbf{XB})\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T\}.$$ (10)

It can be shown that the maximum likelihood estimator of B is also given by (X^T X)⁻¹X^TY. Interestingly, the resulting estimator of B is the same as the ordinary least square estimator, which can be obtained without using the information on the relationship among the response vectors y¹, …, y^m. To incorporate the information among different response variables in estimation of B, we propose a joint penalized method, the doubly penalized maximum likelihood (DML) estimator, by solving

$$\underset{\mathbf{B},\mathbf{C}}{\text{argmin}}\left[-\mathit{n}logdet(\mathbf{C})+\text{tr}\{(\mathbf{Y}-\mathbf{XB})\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T\}+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid +{\lambda }_2\sum _{s\ne t}{v}_{st}\mid c_{st}\mid \right]$$ (11)

Note that the objective function in (11) is not convex with respect to (B, C). The corresponding optimization can be unstable sometimes when p ≥ n. This is because the first term in (11) can dominate the other terms if some diagonal elements of tr (Y – XB)^T (Y – XB)} are zeros, which may occur when p ≥ n. This can be shown by taking a diagonal matrix C and increasing the values of its diagonal elements corresponding to the zero diagonal entries in tr (Y − XB)^T (Y − XB). As a result, the numerical solution of C in (11) can have some large diagonal entries. In practice, the solution of C with very large diagonal entries is not desirable as it leads to very small residual variances of the corresponding response variables. We recommend to first use the plug-in method in Section 2.1 or separate modeling methods to screen the variables and reduce the dimensions. Then one can apply the joint method on the reduced set of variables. As shown in our simulation examples, the joint method can often outperform the plug-in methods when p is moderate compared to n.

2.4. Model Selection

Two plug-in methods are preferable if one of B and C is of main interest and the other is already well estimated. Another advantage of two plug-in methods is that they have lower computational cost than the joint method. On the other hand, the joint method does not require good estimate of B or C. Even though the joint method is computationally more intensive, it often performs better than two plug-in methods in the sense that it optimizes the log-likelihood of (B, C) jointly.

The tuning parameters λ₁ and λ₂ in (8), (9) and (11) control the sparsity of the resulting estimators of (B, C). They can be selected either using validation sets or through K-fold cross-validation. The K-fold cross-validation method randomly splits the dataset into K segments of equal sizes. For the k-th fold, we denote the estimated regression parameter matrix and the estimated inverse covariance matrix using all data excluding those in the k-th segment and the tuning parameters λ₁ and λ₂ by ( ${\widehat{\mathbf{B}}}_{{\lambda }_1}^{(-k)},{\widehat{\mathbf{C}}}_{{\lambda }_2}^{(-k)}$). We also denote the data in the k-th segment as (Y^(k), X^(k)). Specifically, for the PWL method, we select the optimal tuning parameter λ̂₁ which minimizes the prediction error as follows,

$$\text{CV}({\lambda }_1)=\sum _{k=1}^K{\left|\right|{\mathbf{Y}}^{(k)}-{\mathbf{X}}^{(k)}{\widehat{\mathbf{B}}}_{{\lambda }_1}^{(-k)}\left|\right|}_F^2,$$ (12)

where ${\left|\right|·\left|\right|}_F^2$ is the Frobenius norm of a matrix. For the PWGL method, we select the optimal tuning parameter λ̂₂ which minimizes the predictive negative log-likelihood as follows,

$$\text{CV}({\lambda }_2)=\sum _{k=1}^K\left[-n_{k}logdet({\widehat{\mathbf{C}}}_{{\lambda }_2}^{(-k)})+\text{tr}\left\{({\mathbf{Y}}^{(k)}-{\mathbf{X}}^{(k)}\widehat{\mathbf{B}}){\widehat{\mathbf{C}}}_{{\lambda }_2}^{(-k)}{({\mathbf{Y}}^{(k)}-{\mathbf{X}}^{(k)}\widehat{\mathbf{B}})}^T\right\}\right],$$ (13)

where n_k is the sample size of the k-th segment. For the DML method, we first select the optimal λ̂₁ by using (12) with a prespecified λ₂ and select λ̂₂ by using (13) with the selected optimal λ̂₁. It helps to avoid a two dimensional grid search of (λ₁, λ₂). We have found in simulations that the selected optimal λ̂₁s are almost identical for a wide range of prespecified λ₂.

In the use of validation sets, we split the dataset into two parts, the training set and the validation set. With a pair of (λ₁, λ₂), we first estimate (B, C) using the training set. The prediction error and the predictive negative log-likelihood of the resulting estimator are obtained using the validation set as (Y^(k), X^(k)) in (12) and (13). The validation set is not used to construct the final estimator with the selected (λ̂₁, λ̂₂), while the K -fold cross-validation uses all data for the final estimator with (λ̂₁, λ̂₂).

3. Asymptotic Properties

To investigate a sparse regression technique, it is necessary to investigate its asymptotic behaviors. Fan and Li [6] pointed out that a good variable selection procedure should have oracle properties. Asymptotically with probability tending to 1, a procedure with oracle properties can identify the true underlying subset of predictor variables. The resulting estimator of the procedure also asymptotically performs as well as if the true underlying subset were known in advance. In this section, we study the asymptotic behavior of our three proposed methods. In particular, we show that with a proper choice of (λ₁, λ₂), all three methods enjoy the oracle properties.

For the asymptotic analysis, we use the set-up of Fan and Li [6], Yuan and Lin [23] and Zou [25]. The technical derivation uses the results in Knight and Fu [10]. Let ${\mathbf{B}}^{\ast }=({\beta }_{jk}^{\ast })$; j = 1, …, p, k = 1, …, m, be the true regression parameter matrix and ${\mathbf{C}}^{\ast }=(c_{st}^{\ast })$; s = 1, …, m, t = 1, …, m, be the true inverse covariance matrix. Let $\mathcal{A}=\{(j,k):{\beta }_{jk}^{\ast }\ne 0\}$ and $\mathcal{C}=\{(s,t):c_{st}^{\ast }\ne 0\}$. Then we assume the following conditions for our theoretical results:

• (A1) $\frac{1}{n}{\mathbf{X}}^T\mathbf{X}\to A$ where A is a positive definite matrix.
• (A2)The cardinality of , | | = q₁ > 0.
• (A3)There exists β̃_jk which is a $\sqrt{n}$-consistent estimator of ${\beta }_{jk}^{\ast }$; j = 1, …, p, k = 1, …, m.
• (A4)The cardinality of , | | = q₂ > 0.
• (A5)There exists c̃_st which is a $\sqrt{n}$-consistent estimator of $c_{st}^{\ast }$; s = 1, …, m, t = 1, …, m.

Note that conditions (A3) and (A5) are generally satisfied by maximum likelihood estimators or L₂ regularized maximum likelihood estimators with proper choices of penalty parameters. For example, the least square estimator of B can be used as the β̃_jks and the inverse of residual sample covariance matrix can be used as c̃_sts. For the theoretical analysis, we define w_jk and v_st as $w_{jk}=\frac{1}{{\mid {\widehat{\beta }}_{jk}\mid }^{\gamma }}$; j = 1, …, p, k = 1, …, m, γ > 0, and $v_{st}=\frac{1}{\mid {\stackrel{\sim }{c}}_{st}\mid }$; s = 1, …, m, t = 1, …, m, respectively.

In Sections 3.1 and 3.2, we show the plug-in estimators enjoy the oracle properties. Section 3.3 develops the asymptotic theory that reveals the oracle properties of the DML solution.

3.1. Oracle properties of the PWL solution

In this section, we first show that with the known C*, the minimizer of (6) is consistent in variable selection and has the asymptotic normality. Then we show that with a consistent estimator of C*, the PWL estimator also enjoys the same properties.

Define the true regression parameter vector as ${\mathit{\beta }}^{\ast }={({\beta }_{11}^{\ast },\dots ,{\beta }_{p1}^{\ast },\dots ,{\beta }_{1m}^{\ast },\dots ,{\beta }_{pm}^{\ast })}^T$. Let ${\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)}$ be the estimator of β* obtained by minimizing (6) with the penalty parameter λ_1,n. Let ${\mathit{\beta }}_{\mathcal{A}}^{\ast }$ be the q₁-dimensional true parameter vector which consists of nonzero components in β*. Let ${\widehat{{\mathit{\beta }}^{\mathbf{1}}}}_{\mathcal{A}}^{(n)}$ be the corresponding estimators of ${\mathit{\beta }}_{\mathcal{A}}^{\ast }$. Let D = (C* ⊗ A) be the q × q matrix obtained by removing the (j + (k −1)m)-th row and column of C* ⊗ A for (j, k) ∉ . Then the following lemma shows the oracle properties of the penalized likelihood estimator ${\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)}$ with the known C*, as the minimizer of (6) defined previously.

Lemma 1

(Oracle properties of the minimizer of (6), ${\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)}$, with the known C*) Suppose that ${\lambda }_{1,n}{n}^{-\frac{1}{2}}\to 0$ and ${\lambda }_{1,n}{n}^{\frac{\gamma -1}{2}}\to \infty$ as n → ∞. Under the conditions (A1)–(A3), we have the following results:

1. (Selection consistency) ${lim}_{n}P({\widehat{{\beta }^1}}_{jk}^{(n)}=0)=1$ if ${\beta }_{jk}^{\ast }=0$;

2. (Asymptotic normality) $\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{1}}}}_{\mathcal{A}}^{(n)}-{\mathit{\beta }}_{\mathcal{A}}^{\ast }){\to }_d\mathbf{N}(0,D^{-1})$.

Lemma 1 tells us that the penalized maximum likelihood estimator with the known C* satisfies the oracle properties. Since C* is typically unknown in practice, one often uses an estimator for C*. With slight modification of Lemma 1, we can show that the PWL solution also enjoys the oracle properties. Denote the PWL estimator of β* with the penalty parameter λ_1,n as ${\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)}$. Let ${\widehat{{\mathit{\beta }}^{\mathbf{2}}}}_{\mathcal{A}}^{(n)}$ be the corresponding estimator of ${\mathit{\beta }}_{\mathcal{A}}^{\ast }$.

Theorem 1

(Oracle properties of the PWL solution) In addition to the assumptions in Lemma 1, suppose that Ĉ is a consistent estimator of C*. Under the conditions (A1)–(A3), we have the following results:

1. (Selection consistency) ${lim}_{n}P({\widehat{{\beta }^2}}_{jk}^{(n)}=0)=1if{\beta }_{jk}^{\ast }=0$ if ${\beta }_{jk}^{\ast }=0$;

2. (Asymptotic normality) $\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{2}}}}_{\mathcal{A}}^{(n)}-{\mathit{\beta }}_{\mathcal{A}}^{\ast }){\to }_d\mathbf{N}(0,D^{-1})$.

Theorem 1 states that with a consistent estimator of C*, variable selection in the PWL is consistent and the resulting estimator still enjoys the asymptotic normality.

3.2. Oracle properties of the PWGL solution

In this section, we show the oracle properties of the PWGL solution. To this end, we first show the oracle properties of the solution of

$$\underset{\mathbf{C}}{\text{argmin}}\left[-\mathit{n}logdet(\mathbf{C})+\text{tr}\{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })\mathbf{C}{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T\}+{\lambda }_2\sum _{j\ne k}{v}_{jk}\mid c_{jk}\mid \right],$$ (14)

with the known B*. Then we show that with a consistent estimator of B*, the PWGL estimator still enjoys the same properties.

Denote by Ĉ⁽¹⁾ the minimizer of (14) with the known B*. Let ${\widehat{\mathbf{C}}}_0^{(1)}$ be the matrix obtained from Ĉ⁽¹⁾ by replacing ${\widehat{c}}_{jk}^{(1)}$ with 0 if $c_{jk}^{\ast }=0$. Then the following lemma shows the oracle properties of Ĉ⁽¹⁾.

Lemma 2

(Oracle properties of the minimizer of (14), Ĉ⁽¹⁾, with known B*) Suppose that ${\lambda }_{2,n}{n}^{-\frac{1}{2}}\to 0$ and λ_2,n → ∞ as n → ∞. Under the conditions (A1), (A4) and (A5), we have the following results:

1. (Selection consistency) ${lim}_{n}P({\widehat{c}}_{jk}^{(1)}=0)=1$ if $c_{jk}^{\ast }=0$;

2. (Asymptotic distribution) $\sqrt{n}({\widehat{\mathbf{C}}}_0^1-{\mathbf{C}}^{\ast }){\to }_{d}argmin\mathit{V}(U)$,

   where V (U) = tr(UΣUΣ) + tr(UW) and W is an m × m random symmetric matrix such that vec(W) ~ N(0, Λ) in which cov(w_ij, w_kl) = cov(ε_1iε_1j, ε_1kε_1l). The minimum is taken over all symmetric matrices U satisfying u_jk = 0 if $c_{jk}^{\ast }=0$.

In Lemma 2, we show that the penalized maximum likelihood estimator with the known B* satisfies the oracle properties. Since B* is typically unknown in practice, one often applies an univariate regression technique to obtain an estimator for B*. With slight modification of Lemma 2, we can show that the PWGL solution also enjoys the oracle properties. Denote the PWGL estimator of C* with the penalty parameter λ_2,n as Ĉ⁽²⁾. Let ${\widehat{\mathbf{C}}}_0^{(2)}$ be the matrix obtained from Ĉ⁽²⁾ by replacing ${\widehat{c}}_{jk}^{(2)}$ with 0 if $c_{jk}^{\ast }=0$. Then the following theorem shows the oracle properties of the PWGL estimator.

Theorem 2

(Oracle properties of the PWGL solution) In addition to the assumptions in Lemma 2, suppose that B̂ is a consistent estimator of B*. Under the above conditions, we have the following results:

1. (Selection consistency) ${lim}_{n}P({\widehat{c}}_{jk}^{(2)}=0)=1$ if $c_{jk}^{\ast }=0$;

2. (Asymptotic distribution) $\sqrt{n}({\widehat{\mathbf{C}}}_0^2-{\mathbf{C}}^{\ast }){\to }_{d}argmin\mathit{V}(U)$,

   where V(U) = tr(UΣUΣ) + tr(UW) and W is an m × m random symmetric matrix such that vec(W) ~ N(0, Λ) in which cov(w_ij, w_kl) = cov(ε_1iε_1j, ε_1kε_1l). The minimum is taken over all symmetric matrices U satisfying u_jk = 0 if $c_{jk}^{\ast }=0$.

Theorem 2 states that with a consistent estimator of B*, the PWGL solution satisfies the oracle properties.

3.3. Oracle properties of the DML solution

In Sections 3.1 and 3.2, we establish the oracle properties of plug-in estimators. In this section, we explore oracle properties of the DML solution in which (B̂, Ĉ) are obtained together. First, we show that with a proper choice of (λ₁, λ₂), there exists a $\sqrt{n}$-consistent local minimizer of (11). Then we show that this local minimizer enjoys the oracle properties as a solution of the DML estimator.

The following lemma shows the existence of a local minimizer of (11) which is $\sqrt{n}$-consistent.

Lemma 3

Suppose that ${\lambda }_{1,n}{n}^{-\frac{1}{2}}\to 0$ and ${\lambda }_{2,n}{n}^{-\frac{1}{2}}\to 0$. Under the conditions (A1)–(A5), there exists a local minimizer of (11) such that

$$\Vert {(\mathit{vec}{(\widehat{\mathbf{B}})}^T,\mathit{vec}{(\widehat{\mathbf{C}})}^T)}^T-{(\mathit{vec}{({\mathbf{B}}^{\ast })}^T,\mathit{vec}{({\mathbf{C}}^{\ast })}^T)}^T\Vert =O_p(1/\sqrt{n}).$$

From Lemma 3, it is clear that there exists a $\sqrt{n}$-consistent doubly penalized maximum likelihood estimator. As the DML estimator of (B*, C*), denote by (B̂⁽ⁿ⁾, Ĉ) the $\sqrt{n}$-consistent local solution of (11) with the penalty parameter (λ_1,n, λ_2,n). Let β̂⁽ⁿ⁾ = vec(B̂⁽ⁿ⁾) and let ${\widehat{\beta }}_{\mathcal{A}}^{(n)}$ be the corresponding estimator of ${\mathit{\beta }}_{\mathcal{A}}^{\ast }$. Let Ĉ₀ be the matrix obtained from Ĉ by replacing ĉ_jk with 0 if $c_{jk}^{\ast }=0$. We now show that with a proper choice of (λ₁, λ₂), the DML estimator as this local minimizer enjoys the oracle properties in the following theorem.

Theorem 3

(Oracle properties of the DML solution) Suppose that ${\lambda }_{1,n}{n}^{-\frac{1}{2}}\to 0$ and ${\lambda }_{1,n}{n}^{-\frac{\gamma -1}{2}}\to \infty$. In addition to that, suppose that ${\lambda }_{2,n}{n}^{-\frac{1}{2}}\to 0$ and *lamda;_2,n → ∞. Under the conditions (A1)–(A5), we have the following results:

1. ${lim}_{n}P({\widehat{\beta }}_{jk}^{(n)}=0)=1$ if ${\beta }_{jk}^{\ast }=0$;

2. $\sqrt{n}({\widehat{\beta }}_{\mathcal{A}}^{(n)}-{\mathit{\beta }}_{\mathcal{A}}^{\ast }){\to }_d\mathbf{N}(0,D^{-1})$;

3. lim_n P(ĉ_jk = 0) = 1 if $c_{jk}^{\ast }=0$;

4. $\sqrt{n}({\widehat{\mathbf{C}}}_0-{\mathbf{C}}^{\ast }){\to }_{d}argmin\mathit{V}(U)$,

   where V (U) = tr(UΣUΣ) + tr(UW) and W is a m × m random symmetric matrix such that vec(W) ~ N(0, Λ) in which cov(w_ij, w_kl) = cov(ε_1iε_1j, ε_1kε_1l). The minimum is taken over all symmetric matrices U satisfying u_jk = 0 if $c_{jk}^{\ast }=0$.

4. Computational Algorithm

In this section, we describe computational algorithms to solve problems (8), (9), and (11). In particular, we apply the GLASSO algorithm for (9). To solve the problems (8) and (11), we apply the coordinate-descent algorithm as described in Peng et al. [14], which can be viewed as a modification of the shooting algorithm [8]. The basic idea of the coordinate-descent algorithm is to optimize each parameter at one time while holding the other parameters fixed at the current solution. The corresponding optimization at each step can be very simple to solve.

We now describe the coordinate-descent algorithm for the PWL method in details. Denote Ĉ by (ĉ_ij)_m×m. Then (8) is equivalent to minimizing

$$\sum _{i=1}^n\sum _{k,l=1}^m{\widehat{c}}_{kl}(y_{ik}-\sum _{j=1}^p{\beta }_{jk}{x}_{ij})(y_{il}-\sum _{j=1}^p{\beta }_{jl}{x}_{ij})+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid .$$ (15)

Consider (15) as a function of β_jk with other coefficients fixed. Then the minimizer of (15) is equivalent to

$$\underset{{\beta }_{jk}}{\text{argmin}}\left[\left\{\sum _{i=1}^n\left({\widehat{c}}_{kk}{(y_{ik}-\sum _{j^{\prime }\ne j}{\beta }_{j^{\prime }k}{x}_{{ij}^{\prime }}-{\beta }_{jk}{x}_{ij})}^2+2\sum _{k^{\prime }\ne k}{\widehat{c}}_{{kk}^{\prime }}(y_{{ik}^{\prime }}-\sum _j{\beta }_{{ik}^{\prime }}{x}_{ij})(y_{ik}-\sum _{j^{\prime }\ne j}{\beta }_{j^{\prime }k}{x}_{{ij}^{\prime }}-{\beta }_{jk}{x}_{ij})\right)\right\}+{\lambda }_1{w}_{jk}\mid {\beta }_{jk}\mid \right].$$

This problem is essentially a one-dimensional LASSO optimization which has a closed form solution. Therefore, the algorithm can be summarized as follows:

Algorithm 1: the Coordinate-Descent Algorithm for the PWL Method

• Step 1(Initial value). Set the separate LASSO solution ${\beta }_{jk}^{(\mathit{old})}$; j = 1, …, p, k = 1, …, m, as the initial value for B.
• Step 2(Updating rule). For j = 1, …, p and k = 1, …, m,
  ${\beta }_{qr}^{(\mathit{new})}={\beta }_{qr}^{(\mathit{old})}$,if q ≠ j and r ≠ k,

  $${\beta }_{jk}^{(\mathit{new})}=\text{sign}\left(\frac{{\sum }_{l=1}^m{\widehat{c}}_{lk}{(e_l^{(\mathit{old})})}^T{\mathbf{x}}^{\mathbf{j}}}{{\widehat{c}}_{kk}{\mathbf{x}}^{{\mathbf{j}}^T}{\mathbf{x}}^{\mathbf{j}}}+{\beta }_{jk}^{(\mathit{old})}\right){\left(\left|\frac{{\sum }_{l=1}^m{\widehat{c}}_{lk}{(e_l^{(\mathit{old})})}^T{\mathbf{x}}^{\mathbf{j}}}{{\widehat{c}}_{kk}{\mathbf{x}}^{{\mathbf{j}}^T}{\mathbf{x}}^{\mathbf{j}}}+{\beta }_{jk}^{(\mathit{old})}\right|-\frac{{\lambda }_1{w}_{jk}}{2{\widehat{c}}_{kk}{\mathbf{x}}^{{\mathbf{j}}^T}{\mathbf{x}}^{\mathbf{j}}}\right)}^{+},$$

  where $e_l^{(\mathit{old})}={\mathbf{y}}^l-\mathbf{X}{\mathit{\beta }}^{l(\mathit{old})}$ and ${\mathit{\beta }}^{l(\mathit{old})}=({\beta }_{1l}^{(\mathit{old})},\dots ,{\beta }_{pl}^{(\mathit{old})})$.

• Step 3(Iteration). Repeat Step 2 until convergence. Our stopping rule is that the change of the objective function in (8) is less than δ = 0.1.

To be computationally more efficient, we combine the above algorithm with the active shooting algorithm proposed by Peng et al. [14]. The basic idea of the active shooting algorithm is to update the coefficients within the active set until convergence instead of iterating all coefficients at each step. The active set is defined as the set of currently nonzero coefficients and it is typically small. Once the coefficients in the active set converge, then we continue to update other coefficients. This step can speed up the algorithm significantly if the final solution is very sparse.

Next we describe the problem (9) in the GLASSO framework. Since (9) is equivalent to minimizing

$$-logdet(\mathbf{C})+\text{tr}\left\{\frac{1}{n}{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})}^T(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})\mathbf{C}\right\}+\frac{{\lambda }_2}{n}\sum _{j\ne k}{v}_{jk}\mid c_{jk}\mid ,$$ (16)

we can apply the GLASSO algorithm [7] to solve (9) by substituting the sample covariance matrix with $\frac{1}{n}{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})}^T(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})$. Therefore, the algorithm for (9) proceeds as follows:

Algorithm 2: the GLASSO Algorithm for the PWGL Method

• Step 1(Estimator of B) Set the separate LASSO solution as the estimator, B̂, of B.
• Step 2(Estimator of C) Given B̂, apply the GLASSO algorithm to solve (16).

Next, we combine Algorithm 1 and the GLASSO algorithm to solve problem (11) for the doubly penalized method DML in Section 2.3. The algorithm can be summarized as follows:

Algorithm 3: the Coordinate-Descent Algorithm for the DML Method

• Step 1(Initial values of B and C). Set the separate LASSO solution ${\beta }_{jk}^{(\mathit{old})}$; j = 1, …, p, k = 1, …, m, as the initial value for B and the solution of (9), C^(old), as the initial value of C.
• Step 2
  (B updating rule). For a given C^(old), update B^(old) → B^(new) with

  $${\mathbf{B}}^{(\mathit{new})}=\underset{\mathbf{B}}{\text{argmin}}\left[\text{tr}\{(\mathbf{Y}-\mathbf{XB}){\mathbf{C}}^{(\mathit{old})}{(\mathbf{Y}-\mathbf{XB})}^T\}+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid \right].$$

  This step can be solved using the Algorithm 1.

• Step 3
  (C updating rule). For a given B^(new), update C^(old) → C^(new) by

  $${\mathbf{C}}^{(\mathit{new})}=\underset{\mathbf{C}}{\text{argmin}}\left[\text{tr}\left\{\frac{1}{n}{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{(\mathit{new})})}^T(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{(\mathit{new})})\mathbf{C}\right\}-logdet(\mathbf{C})+\frac{{\lambda }_2}{n}\sum _{s\ne t}{v}_{st}\mid c_{st}\mid \right].$$

  This can be solved using the GLASSO algorithm.

• Step 4(Iteration). Repeat Steps 2 and 3 until convergence. Our stopping rule is that the change of the objective function in (11) is less than δ = 0.1.

Based on our experiment, the coordinate-descent algorithm works very efficiently. Since the DML method involves estimation of both B and C, the computation can be intensive when the dimension is high. We consider a prescreening step to speed up the computation. In particular, we adapt the group lasso method considered by Yuan and Lin [22] and Meier et al. [11]. The basic idea of the group lasso method is to employ group penalty in the regression problem so that model selection can be achieved in terms of group selection. In our multiple response variable regression problem, (β_j1, …, β_jm); j = 1, …, p, can be considered as p groups. Therefore, for the prescreening step, the group lasso estimator, B̂^group of B, is given as the minimizer of the following penalized function

$$\sum _{i=1}^n\sum _{k=1}^m{(y_{ik}-\sum _{j=1}^p{\beta }_{jk}{x}_{ij})}^2+\lambda \sum _{j=1}^p\sqrt{{\beta }_{j1}^2+\dots +{\beta }_{jm}^2},$$

where λ is a tuning parameter. We screen out a variable if the corresponding coefficients are estimated as zeros for all response variables. In other words, we remove the variable x^j from our model if ${\widehat{\beta }}_{j1}^{\mathit{group}}=\dots ={\widehat{\beta }}_{jm}^{\mathit{group}}=0$. This prescreening step can not only speed up the computation, but also improve the prediction performance as shown in our examples.

5. Numerical Examples

In this section, our proposed methods are compared with several existing methods. The first existing method we compare is the curds and whey (CW) method proposed by Breiman and Friedman [4]. The other two methods are the separate ridge regression (RR) and the separate LASSO. In particular, we apply the RR and the LASSO to each response variable separately. Separate LASSO solutions are constructed by a modification of the LARS algorithm proposed by Efron et al. [5]. The main idea of this modified LARS algorithm was also considered by Osborne et al. [13]. In this paper, some brief summaries of the results are provided. All details about numerical examples can be found in the online supplement materials.

In simulated examples, all methods are compared in two ways, B estimation and C estimation. Performance of B estimation is compared in terms of prediction and variable selection. For the comparison of C estimation, we use the entropy criterion [9, 24] which measures the difference of two matrices. In terms of prediction, overall, the proposed DML method works the best. The PWL method also works reasonably well in all cases, although it is not as accurate as the DML estimator. In the example where the true inverse covariance matrix is not sparse, LASSO gives the worst prediction performance while the other methods show similar performance. This implies that joint approaches outperform separate approaches with the joint information. DML outperforms LASSO and PWL in terms of identification of zero coefficients. We also notice that the ratios of correctly identified zeros for PWL and DML increase as the sample size increases. This supports the selection consistency shown in Section 3. When the dimension of predictor variables is low, the DML estimator gives the best performance in C estimation. When the dimension of predictor variables is close to the sample size, PWGL outperforms DML. Since the DML method simultaneously estimates both B and C, with a small sample size, the C estimation may not be as good.

We apply our methodology to a Glioblastoma multiforme (GBM) cancer data set studied by the Cancer Genome Atlas (TCGA) Research Network [17]. As noted in [20], GBM is the most common primary form of brain tumor in adults. In terms of prediction, our method, DML, performs best even though the difference between DML and the separate LASSO is not statistically significant in view of the standard errors. In terms of the number of included genes in models, PWL and DML construct sparser models than the separate LASSO. One possible explanation is that there may be some strong positive correlations among microRNAs which are response variables. As we have discussed in the toy example of Section 2, with strong positive correlations among response variables, joint methods tend to obtain more shrinkage than the separate LASSO. To explore this further, correlations among microRNAs are examined. Some strong postive correlations among the microRNAs are detected while negative correlations are not strong. Interestingly, with much fewer number of gene expressions than the separate LASSO, PWL and DML perform competitively in terms of prediction accuracy.

6. Discussion

In this paper, we have proposed three methods for utilizing joint information among response variables in a penalized likelihood framework with weighted L₁ regularization. Our theoretical investigation shows that our proposed estimators enjoy oracle properties. Simulated examples and an application to the GBM cancer data set demonstrate that our proposed methods perform competitively.

Our current study assumes Gaussian distribution of the response vector. One future research direction is to extend the proposed method with other distributional assumptions. Although we mainly focus on the weighted L₁ penalty, our methods can be directly extended for other penalty functions as well. It will be interesting to compare the performance of various choices of penalty in this context.

Appendix A. Proof of Lemma 1

Appendix A.1. Asymptotic normality

Let Ỹ = ((y¹)^T, …, (y^m)^T)^T be the nm-dimensional response vector and ε̃ be the corresponding nm-dimensional error vector which consists of ε_ik; i = 1, …, n, k = 1, …, m. Let β̃ = (β₁₁, …, β_p1, …, β_1m, …, β_pm)^T be the pm-dimensional vector and X̃ = I_m ⊗ X. Then the minimizer of (6) is equivalent to

$$\underset{\stackrel{\sim }{\mathit{\beta }}}{\text{argmin}}\left[{(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}\stackrel{\sim }{\mathit{\beta }})}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}\stackrel{\sim }{\mathit{\beta }})+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid \right].$$

Let $\stackrel{\sim }{\mathit{\beta }}={\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}$ and

$$V_n(\mathit{u})={(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}({\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}))}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}({\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}))+{\lambda }_{1,n}\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\mid .$$

Let û⁽ⁿ⁾ = argmin_u V_n(u) and then ${\widehat{\mathbf{u}}}^{(n)}=\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)}-{\mathit{\beta }}^{\ast })$. Note that û⁽ⁿ⁾ = argmin_u V_n(u) = argmin_u{V_n(u) – V_n(0)} and

$$V_n(\mathbf{u})-V_n(\mathbf{0})=\frac{1}{n}{\mathbf{u}}^T{\stackrel{\sim }{\mathbf{X}}}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}-\frac{2}{\sqrt{n}}{\stackrel{\sim }{\mathit{\epsilon }}}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}+{\lambda }_{1,n}\sum _{j,k}{w}_{jk}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid ).$$ (A.1)

We know that $\frac{1}{n}{\mathbf{u}}^T{\stackrel{\sim }{\mathbf{X}}}^T(\mathbf{C}\otimes I_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}={\mathbf{u}}^T(\mathbf{C}\otimes \frac{1}{n}{\mathbf{X}}^T\mathbf{X})\mathbf{u}\to {\mathbf{u}}^T(\mathbf{C}\otimes A)\mathbf{u}$. For the second term of the right hand side of (A.1), note that ε̃ ~ N(0, Σ ⊗ I_n). Thus, $\frac{1}{\sqrt{n}}{\stackrel{\sim }{\mathit{\epsilon }}}^T(\mathbf{C}\otimes I_n)\stackrel{\sim }{\mathbf{X}}{\to }_d\mathbf{Z}$ where Z ~ N(0, C ⊗ A) as $\frac{1}{n}{\stackrel{\sim }{\mathbf{X}}}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\mathbf{\sum }\otimes {\mathbf{I}}_n)(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}=\frac{1}{n}{\stackrel{\sim }{\mathbf{X}}}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\to \mathbf{C}\otimes A$. Now we consider the last term of the right hand side of (A.1):

• If ${\beta }_{jk}^{\ast }=0$, then ${\lambda }_{1,n}{w}_{jk}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid )=\frac{{\lambda }_{1,n}}{\sqrt{n}}{w}_{jk}\mid u_{jk}\mid ={\lambda }_{1,n}{n}^{\frac{\gamma -1}{2}}\frac{\mid u_{jk}\mid }{{(\sqrt{n}\mid {\stackrel{\sim }{\beta }}_{jk}\mid )}^{\gamma }}\to \infty$ as $\sqrt{n}{\stackrel{\sim }{\beta }}_{jk}=O_p(1)$.

• if ${\beta }_{jk}^{\ast }\ne 0$, then ${\lambda }_{1,n}{w}_{jk}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid )=\frac{{\lambda }_{1,n}}{\sqrt{n}}{w}_{jk}\sqrt{n}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid )$. Note that $\frac{{\lambda }_{1,n}}{\sqrt{n}}\to 0,w_{jk}{\to }_p\frac{1}{{\mid {\beta }_{jk}^{\ast }\mid }^{\gamma }}$ and $\sqrt{n}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid )\to u_{jk}\text{sign}({\beta }_{jk}^{\ast })$. By the Slutsky’s theorem, ${\lambda }_{1,n}{w}_{jk}(\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|-\mid {\beta }_{jk}^{\ast }\mid ){\to }_{p}0$.

By combining above statements and using the Slutsky’s theorem again, we obtain the following:

$$V_n(\mathbf{u})-V_n(\mathbf{0}){\to }_{d}V(\mathbf{u})=\{\begin{array}{lll}{\mathbf{u}}_{\mathcal{A}}^{T}D{\mathbf{u}}_{\mathcal{A}}-2{\mathbf{u}}_{\mathcal{A}}^T{\mathbf{Z}}_{\mathcal{A}}\hfill & \text{if}\hfill & u_{jk}=0\text{for}\text{all}(j,k)\notin \mathcal{A},\hfill \\ \infty \hfill & \text{if}\hfill & \text{otherwise},\hfill \end{array}$$

where consists of u_jk for (j, k) ∈ and ~ N(0, D).

Let û = argmin_u V(u). Then we have

$$\{\begin{array}{l}{\widehat{\mathbf{u}}}_{\mathcal{A}}=D^{-1}{\mathbf{Z}}_{\mathcal{A}},\hfill \\ {\widehat{u}}_{jk}=0\forall (j,k)\notin \mathcal{A}.\hfill \end{array}$$

Note that V_n(u) – V_n(0) is convex and so argmin_u(V_n(u) – V_n(0)) →_d argmin_u V(u). Since ~ N(0, D), thus ${\widehat{\mathbf{u}}}_{\mathcal{A}}^{(n)}{\to }_d\mathbf{N}(0,D^{-1})$. Finally, we have that ${\widehat{\mathbf{u}}}_{\mathcal{A}}^{(n)}=\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{1}}}}_{\mathcal{A}}^{(n)}-{\mathit{\beta }}_{\mathcal{A}}^{\ast }){\to }_d{D}^{-1}{\mathbf{Z}}_{\mathcal{A}}$ as n → ∞.

Appendix A.2. Selection consistency

We need to show that ∀(j, k) ∉ , $P({\widehat{{\beta }^1}}_{jk}^{(n)}\ne 0)\to 0$. For fixed (j, k) ∉ , let $(j,k)\in {\mathcal{A}}_n^1$. Then $\mid {\widehat{{\mathit{\beta }}^1}}_{jk}^{(n)}\mid \ne 0$ and so we have that $2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^1}}_{jk}^{(n)})$ by the KKT conditions, where x̃_jk is (j + (k − 1))-th row of X̃. Therefore, $P({\widehat{{\beta }^{\text{1}}}}_{jk}^{(n)}\ne 0)\le P(2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^1}}_{jk}^{(n)}))$. Note that

$$\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})}{\sqrt{n}}=\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\sqrt{n}({\mathit{\beta }}^{\ast }-{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})}{n}+\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathit{\epsilon }}}{\sqrt{n}}.$$

From the asymptotic normality part, we know that $\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\sqrt{n}({\mathit{\beta }}^{\ast }-{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})}{n}$ converges in distribution to some normal random vector. We also have that $\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathit{\epsilon }}}{n}{\to }_d\mathbf{N}(0,{(\mathbf{C}\otimes A)}_{jk,jk})$, where (C ⊗ A)_jk,jk is the (j + (k − 1))-th diagonal element of C ⊗ A. As $\frac{{\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^1}}_{jk}^{(n)})}{\sqrt{n}}={\lambda }_{1,n}{n}^{\frac{\gamma -1}{2}}\frac{\text{sign}({\widehat{{\beta }^1}}_{jk}^{(n)})}{(\sqrt{n}\mid {\stackrel{\sim }{\beta }}_{jk}\mid )\gamma }\to \pm \infty$ with $\sqrt{n}{\stackrel{\sim }{\beta }}_{jk}=O_p(1)$, we have $P(2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\mathbf{C}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{1}}}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^1}}_{jk}^{(n)}))\to 0$. Therefore, $P({\widehat{{\beta }^1}}_{jk}^{(n)}\ne 0)\to 0$as n → ∞.

Appendix B. Proof of Theorem 1

The proof is similar to that of Lemma 1 except we replace C by Ĉ.

Appendix B.1. Asymptotic normality

Note that (8) is equivalent to

$$\underset{\stackrel{\sim }{\mathit{\beta }}}{\text{argmin}}\left[{(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}\stackrel{\sim }{\mathit{\beta }})}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}\stackrel{\sim }{\mathit{\beta }})+{\lambda }_1\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid \right].$$

Let $\stackrel{\sim }{\mathit{\beta }}={\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}$ and

$$V_n^{\ast }(\mathbf{u})={(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}({\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}))}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}({\mathit{\beta }}^{\ast }+\frac{\mathit{u}}{\sqrt{n}}))+{\lambda }_{1,n}\sum _{j,k}{w}_{jk}\left|{\beta }_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\right|.$$

Let ${\widehat{\mathbf{u}}}^n={\text{argmin}}_{\mathbf{u}}{V}_n^{\ast }(\mathbf{u})$ and then ${\widehat{\mathbf{u}}}^{(n)}=\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)}-{\mathit{\beta }}^{\ast })$. We can show that

$$V_n^{\ast }(\mathbf{u})-V_n^{\ast }(0)=V_n(\mathbf{u})-V_n(\mathbf{0})+\frac{1}{n}{\mathbf{u}}^T{\stackrel{\sim }{\mathbf{X}}}^T((\widehat{\mathbf{C}}-\mathbf{C})\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}-\frac{2}{\sqrt{n}}{\stackrel{\sim }{\mathit{\epsilon }}}^T((\widehat{\mathbf{C}}-\mathbf{C})\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u},$$

where V_n(u) is defined in the proof of Lemma 1. As Ĉ is a consistent estimator of C, $\frac{1}{n}{\mathbf{u}}^T{\stackrel{\sim }{\mathbf{X}}}^T((\widehat{\mathbf{C}}-\mathbf{C})\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}{\to }_{p}0$ and $\frac{2}{\sqrt{n}}{\stackrel{\sim }{\mathit{\epsilon }}}^T((\widehat{\mathbf{C}}-\mathbf{C})\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\mathbf{u}{\to }_{d}0$. From the proof of Lemma 1, we also know that V_n(u) – V_n(0) →_d V(u). By combining the above statements and using the Slutsky’s theorem, we have that $V_n^{\ast }(\mathbf{u})-V_n^{\ast }(0){\to }_{d}V(\mathbf{u})$. By using the same arguments as in the proof of Lemma 1, finally we have that ${\widehat{\mathbf{u}}}_{\mathcal{A}}^{(n)}=\sqrt{n}({\widehat{{\mathit{\beta }}^{\mathbf{2}}}}_{\mathcal{A}}^{(n)}-{\mathit{\beta }}_{\mathcal{A}}^{\ast }){\to }_d{D}^{-1}{\mathbf{Z}}_{\mathcal{A}}$ as n → ∞.

Appendix B.2. Selection consistency

Now it suffices to show that ∀(j, k) ∉ , $P({\widehat{{\beta }^2}}_{jk}^{(n)}\ne 0)\to 0$ as n → ∞. For fixed (j, k) ∉ , let $(j,k)\in {\mathcal{A}}_n^2$. Then $\mid {\widehat{{\beta }^2}}_{jk}^{(n)}\mid \ne 0$ and so we have that $2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^2}}_{jk}^{(n)})$ by the KKT conditions. Therefore, $P({\widehat{{\beta }^2}}_{jk}^{(n)}\ne 0)\le P(2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^2}}_{jk}^{(n)}))$. Note that

$$\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)})}{\sqrt{n}}=\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\sqrt{n}({\mathit{\beta }}^{\ast }-{\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)})}{n}+\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathit{\epsilon }}}{\sqrt{n}}.$$

From the asymptotic normality part and the fact that Ĉ is consistent, we know that $\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathbf{X}}\sqrt{n}({\mathit{\beta }}^{\ast }-{\widehat{{\mathit{\beta }}^{\mathbf{2}}}}^{(n)})}{n}$ converges in distribution to some normal random vector. We also have that $\frac{2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)\stackrel{\sim }{\mathit{\epsilon }}}{\sqrt{n}}{\to }_d\mathbf{N}(0,{(\mathbf{C}\otimes A)}_{jk,jk})$. As $\frac{{\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^2}}_{jk}^{(n)})}{\sqrt{n}}={\lambda }_{1,n}{n}^{\frac{\gamma -1}{2}}\frac{\text{sign}({\widehat{{\beta }^2}}_{jk}^{(n)})}{{(\sqrt{n}\mid {\widehat{\beta }}_{jk}\mid )}^{\gamma }}\to \pm \infty$, we have $P(2{\stackrel{\sim }{\mathbf{x}}}_{jk}^T(\widehat{\mathbf{C}}\otimes {\mathbf{I}}_n)(\stackrel{\sim }{\mathbf{Y}}-\stackrel{\sim }{\mathbf{X}}{\widehat{{\mathit{\beta }}^2}}^{(n)})={\lambda }_{1,n}{w}_{jk}\text{sign}({\widehat{{\beta }^2}}_{jk}^{(n)}))\to 0$. Therefore, $P({\widehat{{\beta }^2}}_{jk}^{(n)}\ne 0)\to 0$ as n → ∞.

Appendix C. Proof of Lemma 2

Let $R=\frac{1}{n}{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })$. With given B*, define Q(C) as

$$Q(\mathbf{C})=-\mathit{n}logdet(\mathbf{C})+n\text{tr}(\mathbf{C}R)+{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}\mid c_{jk}\mid .$$ (C.1)

Appendix C.1. Selection consistency

Using the definition of Q(C) in (C.1), define V_n(U) as

$$\begin{array}{l}{V}_n(U)=Q({\mathbf{C}}^{\ast }+\frac{U}{\sqrt{n}})-Q({\mathbf{C}}^{\ast })\\ =-\mathit{n}logdet(({\mathbf{C}}^{\ast }+\frac{U}{\sqrt{n}}){{\mathbf{C}}^{\ast }}^{-1})+n\text{tr}(\frac{UR}{\sqrt{n}})+{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}(\mid c_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\mid -\mid c_{jk}^{\ast }\mid ).\end{array}$$

Using a similar argument as in the proof of Theorem 1 in Yuan and Lin (2007), it can be shown that

$$V_n(U)=\text{tr}(U\mathbf{\sum }U\mathbf{\sum })+\text{tr}[U\sqrt{n}(R-\mathbf{\sum })]+{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}(\mid c_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\mid -\mid c_{jk}^{\ast }\mid )+o(1).$$

Note that as $v_{st}=\frac{1}{\mid {\stackrel{\sim }{c}}_{st}\mid },{\lambda }_{2,n}{n}^{-\frac{1}{2}}\to 0$, and ${\stackrel{\sim }{c}}_{jk}{\to }_p{c}_{jk}^{\ast }$, we have

$$\begin{array}{l}{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}(\mid c_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\mid -\mid c_{jk}^{\ast }\mid )={\lambda }_{2,n}\sum _{c_{jk}^{\ast }=0}\frac{\mid u_{jk}\mid }{\sqrt{n}\mid {\stackrel{\sim }{c}}_{jk}\mid }+\frac{{\lambda }_{2,n}}{\sqrt{n}}\sum _{c_{jk}^{\ast }\ne 0}(\frac{\mid u_{jk}\mid }{\mid {\stackrel{\sim }{c}}_{jk}\mid }\text{sign}(c_{jk}^{\ast })+o(1))\\ ={\lambda }_{2,n}\sum _{c_{jk}^{\ast }=0}\frac{\mid u_{jk}\mid }{\sqrt{n}\mid {\stackrel{\sim }{c}}_{jk}\mid }+o_p(1).\end{array}$$

On the other hand, $\sqrt{n}(R-\mathbf{\sum }){\to }_d\mathbf{N}(0,\mathrm{\Lambda })$ by the central limit theorem as $R=\frac{1}{n}{\sum }_i^n{\epsilon }_i{\epsilon }_i^T$. Therefore, V_n(U) can be written as

$$V_n(U)=\text{tr}(U\mathbf{\sum }U\mathbf{\sum })+\text{tr}({UW}_n)+{\lambda }_{2,n}\sum _{c_{jk}^{\ast }=0}\frac{\mid u_{jk}\mid }{\sqrt{n}\mid {\stackrel{\sim }{c}}_{jk}\mid }+o_p(1),$$

where W_n →_d N(0, Λ). Denote by Û the minimizer of V_n(U). Note that λ_2,n → ∞ and $\sqrt{n}\mid {\stackrel{\sim }{c}}_{jk}\mid =O_p(1)$. Therefore, if $c_{jk}^{\ast }=0$, P (û_jk = 0) → 1 as n → ∞. This completes the proof of the variable selection consistency.

Appendix C.2. Asymptotic distribution

Suppose U satisfies that u_jk = 0 if $c_{jk}^{\ast }=0$. Then, V_n(U) can be written as

$$V_n(U)=\text{tr}(U\mathbf{\sum }U\mathbf{\sum })+\text{tr}[U\sqrt{n}(R-\mathbf{\sum })]+o_p(1).$$

By using the Slutsky’s theorem, we have that

$$V_n(U){\to }_{d}V(U)=\text{tr}(U\mathbf{\sum }U\mathbf{\sum })+\text{tr}(UW)\text{where}\text{vec}(W)\sim \mathbf{N}(0,\mathrm{\Lambda }).$$

Since V_n(U) and V(U) are both convex and V(U) has a unique minimum, argmin V_n(U) →_d argmin V(U). From the fact that $\text{argmin}{V}_n(U)=\text{argmin}Q({\mathbf{C}}^{\ast }+\frac{U}{\sqrt{n}})=\sqrt{n}({\widehat{\mathbf{C}}}_0^1-{\mathbf{C}}^{\ast }),\text{argmin}{V}_n(U)=\sqrt{n}({\widehat{\mathbf{C}}}_0^1-{\mathbf{C}}^{\ast }){\to }_d\text{argmin}V(U)$. This completes the proof of the asymptotic distribution.

Appendix D. Proof of Theorem 2

With a $\sqrt{n}$-consistent estimator B̂ of B, let $\widehat{R}=\frac{1}{n}{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})}^T(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})$. Define Q(C) as

$$Q(\mathbf{C})=-\mathit{n}logdet(\mathbf{C})+n\text{tr}(\mathbf{C}\widehat{R})+{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}\mid c_{jk}\mid .$$ (D.1)

By using the above definition, define V_n(U) as

$$\begin{array}{l}{V}_n(U)=Q({\mathbf{C}}^{\ast }+\frac{U}{\sqrt{n}})-Q({\mathbf{C}}^{\ast })\\ =-\mathit{n}logdet(({\mathbf{C}}^{\ast }+\frac{U}{\sqrt{n}}){\mathbf{C}}^{\ast -1})+n\text{tr}(\frac{U\widehat{R}}{\sqrt{n}})+{\lambda }_{2,n}\sum _{j\ne k}{v}_{jk}(\mid c_{jk}^{\ast }+\frac{u_{jk}}{\sqrt{n}}\mid -\mid c_{jk}^{\ast }\mid ).\end{array}$$

Note that

$$n\text{tr}(\frac{U\widehat{R}}{\sqrt{n}})=n\text{tr}(\frac{U(\widehat{R}-R)}{\sqrt{n}})+n\text{tr}(\frac{UR}{\sqrt{n}}).$$

Therefore, by the proof of Lemma 2 and the Slutsky’s theorem, it suffices to show that

$$n\text{tr}(\frac{U(\widehat{R}-R)}{\sqrt{n}})=o_p(1).$$ (D.2)

The left-hand side of (D.2) can be written as

$$\begin{array}{l}n\text{tr}(\frac{U(\widehat{R}-R)}{\sqrt{n}})=\text{tr}(\frac{U}{\sqrt{n}}{(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}})}^T(\mathbf{Y}-\mathbf{X}\widehat{\mathbf{B}}))-\text{tr}(\frac{U}{\sqrt{n}}{(\mathbf{Y}-\mathbf{XB})}^T(\mathbf{Y}-\mathbf{XB}))\\ =\text{tr}(U\sqrt{n}{(\widehat{\mathbf{B}}-\mathbf{B})}^T\frac{{\mathbf{X}}^T\mathbf{X}}{n}(\widehat{\mathbf{B}}-\mathbf{B}))-2\text{tr}(U\frac{{(\mathbf{Y}-\mathbf{XB})}^T\mathbf{X}}{\sqrt{n}}(\widehat{\mathbf{B}}-\mathbf{B})),\end{array}$$

where we add and subtract XB in the first term. Since $\sqrt{n}(\widehat{\mathbf{B}}-\mathbf{B})=O_p(1),\frac{{(\mathbf{Y}-\mathbf{XB})}^T\mathbf{X}}{\sqrt{n}}=O_p(1)$ , (B̂-B = o_p(1) and $\frac{1}{n}{\mathbf{X}}^T\mathbf{X}\to A$, (D.2) holds.

Appendix E. Proof of Lemma 3

Define Q(B, C) for the jointly penalized likelihood as

$$Q(\mathbf{B},\mathbf{C})=-\mathit{n}logdet(\mathbf{C})+\text{tr}\{\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T(\mathbf{Y}-\mathbf{XB})\}+{\lambda }_{1,n}\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid +{\lambda }_{2,n}\sum _{s\ne t}{v}_{st}\mid c_{St}\mid .$$ (E.1)

To show the results, we use the similar idea of the proof of Theorem 1 in Fan and Li [6]. It suffices to show that for any given δ > 0, there exists a large constant D such that

$$P\{\underset{\left|\right|U\left|\right|=D}{sup}Q({\mathbf{B}}^{\ast }+\frac{U_1}{\sqrt{n}},{\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}})>Q({\mathbf{B}}^{\ast },{\mathbf{C}}^{\ast })\}\ge 1-\delta ,$$ (E.2)

where U = (vec(U₁)^T, vec(U₂)^T)^T. Using the definition of Q(B, C) in (E.1), define V_n(U) as

$$V_n(U)=Q({\mathbf{B}}^{\ast }+\frac{U_1}{\sqrt{n}},{\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}})-Q({\mathbf{B}}^{\ast },{\mathbf{C}}^{\ast }).$$

Since $\mid {\beta }_{jk}^{\ast }+\frac{u_{1jk}}{\sqrt{n}}\mid -\mid {\beta }_{jk}^{\ast }\mid =\mid \frac{u_{1jk}}{\sqrt{n}}\mid$ for ${\beta }_{jk}^{\ast }=0$ and $\mid c_{st}^{\ast }+\frac{u_{2st}}{\sqrt{n}}\mid -\mid c_{st}^{\ast }\mid =\mid \frac{u_{2st}}{\sqrt{n}}\mid$ for $c_{st}^{\ast }=0$,

$$\begin{array}{l}{V}_n(U)\ge -\mathit{n}logdet(({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){\mathbf{C}}^{\ast -1})+\text{tr}\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){(\mathbf{Y}-\mathbf{X}({\mathbf{B}}^{\ast }+\frac{U_1}{\sqrt{n}}))}^T(\mathbf{Y}-\mathbf{X}({\mathbf{B}}^{\ast }+\frac{U_1}{\sqrt{n}}))\}\\ -\text{tr}\{{\mathbf{C}}^{\ast }{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })\}+{\lambda }_{1,n}\sum _{{\beta }_{kj}\ne 0}{w}_{jk}(\mid {\beta }_{jk}^{\ast }+\frac{u_{1jk}}{\sqrt{n}}\mid -\mid {\beta }_{jk}^{\ast }\mid )\\ +{\lambda }_{2,n}\sum _{c_{st}\ne 0}{v}_{st}(\mid c_{st}^{\ast }+\frac{u_{2st}}{\sqrt{n}}\mid -\mid c_{st}^{\ast }\mid )\\ =-\mathit{n}logdet(({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){\mathbf{C}}^{\ast -1})+\text{tr}\{\frac{U_2}{\sqrt{n}}{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })\}\\ +\text{tr}\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){(\frac{{\mathbf{X}U}_1}{\sqrt{n}})}^T(\frac{{\mathbf{X}U}_1}{\sqrt{n}})\}-2\text{tr}\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\frac{{\mathbf{X}U}_1}{\sqrt{n}})\}\\ +{\lambda }_{1,n}\sum _{{\beta }_{kj}\ne 0}{w}_{jk}(\mid {\beta }_{jk}^{\ast }+\frac{u_{1jk}}{\sqrt{n}}\mid -\mid {\beta }_{jk}^{\ast }\mid )+{\lambda }_{2,n}\sum _{c_{st}\ne 0}{v}_{st}(\mid c_{st}^{\ast }+\frac{u_{2st}}{\sqrt{n}}\mid -\mid c_{st}^{\ast }\mid ).\end{array}$$ (E.3)

For the first term and the second term on the right-hand side of (E.3), it has been shown in Lemma 2 that

$$-\mathit{n}logdet(({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){\mathbf{C}}^{\ast -1})+\text{tr}\{\frac{U_2}{\sqrt{n}}{(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })\}=\text{tr}(U_2\mathbf{\sum }{U}_2\mathbf{\sum })+\text{tr}(U_2{W}_n).$$

Let U₁ = vec(U₁). For the third term on the right-hand side of (E.3), as $\frac{1}{n}{\mathbf{X}}^T\mathbf{X}\to A$, note that

$$\text{tr}\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){(\frac{{\mathbf{X}U}_1}{\sqrt{n}})}^T(\frac{{\mathbf{X}U}_1}{\sqrt{n}})\}={\stackrel{\sim }{U}}_1^T\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}})\otimes (\frac{{\mathbf{X}}^T\mathbf{X}}{n})\}{\stackrel{\sim }{U}}_1={\stackrel{\sim }{U}}_1^T({\mathbf{C}}^{\ast }\otimes A){\stackrel{\sim }{U}}_1+o(1).$$

For the fourth term on the right-hand side of (E.3), we have

$$\text{tr}\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}}){(\mathbf{Y}-\mathbf{X}{\mathbf{B}}^{\ast })}^T(\frac{{\mathbf{X}U}_1}{\sqrt{n}})\}={\stackrel{\sim }{U}}_1^T{(\frac{\stackrel{\sim }{\mathbf{X}}}{\sqrt{n}})}^T\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}})\otimes {\mathbf{I}}_n\}\stackrel{\sim }{\mathit{\epsilon }}.$$

Note that ${(\frac{\stackrel{\sim }{\mathbf{X}}}{\sqrt{n}})}^T\{({\mathbf{C}}^{\ast }+\frac{U_2}{\sqrt{n}})\otimes {\mathbf{I}}_n\}\stackrel{\sim }{\mathit{\epsilon }}{\to }_{d}Z$ where Z has multivariate normal distribution of dimension n × m. By combining above statements, we have

$$V_n(U)\ge \text{tr}(U_2\mathbf{\sum }{U}_2\mathbf{\sum })+\text{tr}(U_2{W}_n)+{\stackrel{\sim }{U}}_1^T({\mathbf{C}}^{\ast }\otimes A){\stackrel{\sim }{U}}_1+{\stackrel{\sim }{U}}_1^T{Z}_n+o_p(1)+{\lambda }_{1,n}\sum _{{\beta }_{kj}\ne 0}{w}_{jk}(\mid {\beta }_{jk}^{\ast }+\frac{u_{1jk}}{\sqrt{n}}\mid -\mid {\beta }_{jk}^{\ast }\mid )+{\lambda }_{2,n}\sum _{c_{st}\ne 0}{v}_{st}(\mid c_{st}^{\ast }+\frac{u_{2st}}{\sqrt{n}}\mid -\mid c_{st}^{\ast }\mid ).$$

As ${\lambda }_{1,n}{n}^{-\frac{1}{2}}\to 0$ and ${\lambda }_{2,n}{n}^{-\frac{1}{2}}\to 0$, we have

$$\begin{array}{r}{\lambda }_{1,n}\sum _{{\beta }_{jk}^{\ast }\ne 0}{w}_{jk}(\mid {\beta }_{jk}^{\ast }+\frac{u_{1jk}}{\sqrt{n}}\mid -\mid {\beta }_{jk}^{\ast }\mid )=\frac{{\lambda }_{1,n}}{\sqrt{n}}\sum _{{\beta }_{jk}^{\ast }\ne 0}(\frac{\mid u_{1jk}\mid }{{\mid {\stackrel{\sim }{\beta }}_{jk}\mid }^{\gamma }}\text{sign}({\beta }_{jk}^{\ast })+o(1))=o_p(1),\\ {\lambda }_{2,n}\sum _{c_{st}\ne 0}{v}_{st}(\mid c_{st}^{\ast }+\frac{u_{2st}}{\sqrt{n}}\mid -\mid c_{st}^{\ast }\mid )=\frac{{\lambda }_{2,n}}{\sqrt{n}}\sum _{c_{st}^{\ast }\ne 0}(\frac{\mid u_{2st}\mid }{\mid {\stackrel{\sim }{c}}_{st}\mid }\text{sign}(c_{st}^{\ast })+o(1))=o_p(1).\end{array}$$

Therefore,

$$V_n(U)\ge \text{tr}(U_2\mathbf{\sum }{U}_2\mathbf{\sum })+\text{tr}(U_2{W}_n)+{\stackrel{\sim }{U}}_1^T({\mathbf{C}}^{\ast }\otimes A){\stackrel{\sim }{U}}_1+{\stackrel{\sim }{U}}_1^T{Z}_n+o_p(1).$$ (E.4)

By choosing a sufficiently large D, V_n(U) > 0 uniformly on {U : ||U || = D} with the probability greater than 1-δ as C* and A are positive-definite, W_n = O_p(1), and Z_n = O_p(1). Therefore, (E.2) holds. This completes the proof of this lemma.

Appendix F. Proof of Theorem 3

As defined in Lemma 3, define Q(B, C) for the jointly penalized likelihood as

$$Q(\mathbf{B},\mathbf{C})=-\mathit{n}logdet(\mathbf{C})+\text{tr}\{\mathbf{C}{(\mathbf{Y}-\mathbf{XB})}^T(\mathbf{Y}-\mathbf{XB})\}+{\lambda }_{1,n}\sum _{j,k}{w}_{jk}\mid {\beta }_{jk}\mid +{\lambda }_{2,n}\sum _{s\ne t}{v}_{st}\mid c_{st}\mid .$$

Note that (B̂⁽ⁿ⁾, Ĉ) is a $\sqrt{n}$-consistent local minimizer of Q(B, C). As B̂⁽ⁿ⁾ = argmin_B Q(B, Ĉ) and Ĉ is $\sqrt{n}$-consistent, the oracle properties of B̂⁽ⁿ⁾ hold by Theorem 1. Similarly, since Ĉ = argmin_C Q(B̂⁽ⁿ⁾, C) and B̂⁽ⁿ⁾ is $\sqrt{n}$-consistent, the oracle properties of Ĉ hold by Theorem 2. These complete the proof of this theorem.