GLOBALLY ADAPTIVE QUANTILE REGRESSION WITH ULTRA-HIGH DIMENSIONAL DATA

Abstract

Quantile regression has become a valuable tool to analyze heterogeneous covaraite-response associations that are often encountered in practice. The development of quantile regression methodology for high dimensional covariates primarily focuses on examination of model sparsity at a single or multiple quantile levels, which are typically prespecified ad hoc by the users. The resulting models may be sensitive to the specific choices of the quantile levels, leading to difficulties in interpretation and erosion of confidence in the results. In this article, we propose a new penalization framework for quantile regression in the high dimensional setting. We employ adaptive L₁ penalties, and more importantly, propose a uniform selector of the tuning parameter for a set of quantile levels to avoid some of the potential problems with model selection at individual quantile levels. Our proposed approach achieves consistent shrinkage of regression quantile estimates across a continuous range of quantiles levels, enhancing the flexibility and robustness of the existing penalized quantile regression methods. Our theoretical results include the oracle rate of uniform convergence and weak convergence of the parameter estimators. We also use numerical studies to confirm our theoretical findings and illustrate the practical utility of our proposal.

1. Introduction

We consider the problem of analyzing high and ultra-high dimensional data, which have become widely available in a large variety of scientific fields, such as biomedical imaging, signal processing, machine learning, and finance. In ultra-high dimensional data sets, the number of candidate covariates p is allowed to increase at an exponential rate of the number of observations n, but only a relatively small number s of them have real impact on the response variable. In such a situation, it becomes very useful but challenging to identify relevant variables and measure their influences.

Effort has been made to address this unprecedent challenge in the context of linear regression (Meinshausen and Buhlmann, 2006; Zhang and Huang, 2008; Huang et al., 2008; Kim et al., 2008; Lv and Fan, 2009; Fan and Lv, 2011, among others). Quantile regression (Koenker and Bassett (1978)) has emerged as a flexible tool to models the effects of covariates on the conditional quantiles, and it permits investigation of heterogeneity across quantiles. For example, meteorologists typically focus on the extreme temperatures in climate studies. Gaussian model based procedures would be inadequate for addressing scientific questions of this kind, and quantile models have a natural role to play. Most of current literature on quantile regression for high dimensional data inquire into covariate effects at a single or multiple prespecified quantile levels, to which we shall refer as locally concerned quantile regression. A number of authors, for example, Knight and Fu (2000), Li et al. (2007), Zou and Yuan (2008), Wu and Liu (2009), Rocha et al. (2009), considered the locally concerned quantile regression using penalization to achieve sparsity. Several authors, such as Wang et al. (2012), Zheng et al. (2013) and Fan et al. (2014), investigated cases with ultra-high dimensional covariates.

There are subtle and yet important issues with the practical use of locally concerned quantile regression. For example, when interests lie in identifying variables that impact the upper quantiles, would one just consider a single τ-th quantile at τ = 0.9, or several quantile levels? There is usually no clear scientific support for choosing one τ over another nearby value. With a limited sample size, there is variability in the set of selected variables as τ changes, even if just slightly. Such variability is clearly undesirable for interpretation. More importantly, some important variables are likely to be missed, simply due to chance, if we perform variable selection at any given τ.

To address the limitations of locally concerned quantile regression, we propose an alternative model selection strategy, called globally concerned quantile regression, that examines regression quantiles over a set of quantile levels, denoted by Δ ⊂ (0, 1). Typically, Δ is selected as an interval of quantile levels that well captures part of the conditional distributions. For example, Δ may be chosen as [0.4, 0.6] if we would like to identify variables that impact the center of the conditional distributions, or [0.75, 0.9] if we are interested in the upper tails. If we are interested in identifying variables that have impact on any quantile of the conditional distributions, we may choose Δ = [0.1, 0.9]. If Δ is a singleton set or a finite set, globally concerned quantile regression reduces to locally concerned quantile regression. Therefore, we can take the view that globally concern quantile regression extends locally concerned quantile regression by allowing for contemporaneous evaluation of the covariate effects at a continuum of quantile levels. This additional flexibility offered by globally concerned quantile regression can enhance high-dimensional sparse modeling. Specifically, a globally concerned quantile regression approach can take advantage of all useful information across quantiles to improve the stability of variable selection. Even if an active variable is missed by locally concerned quantile regression at the targeted quantile level, its trail may still be captured within the neighborhood of the quantile level.

A naive pointwise approach for globally concerned quantile regression in the ultra-high dimensional setting would perform an existing locally concerned penalization method separately at each τ ∈ Δ (including tuning parameter selection), and then take the union of active variable sets identified at each τ. Such an approach tends to result in considerably overfitted models, as clearly demonstrated by simulation studies in Section 4. The model overfitting phenomenon is analogous to the inflated type-I error in multiple comparisons. Similar to the strategy of controlling the familywise error rates by adjusting critical values of individual tests in multiple comparison problems, we consider appropriate selection of the tuning parameter in the penalization.

Belloni and Chernozhukov (2011) investigated the L₁-penalized quantile regression (L₁-QR) for the ultra-high dimensional data setting and established some useful results that hold uniformly across Δ. They established a near oracle consistency rate $\sqrt{slogp/n}$ for p > n, and showed that the model identified by L₁-QR contains the true model as a submodel. Nevertheless, there remain some important open questions. First, in the ultrahigh dimensional setting, log p could be o(n^b) for some b > 0. Thus the convergence rate $\sqrt{slogp/n}$ is less satisfactory. Notice that AR-Lasso proposed by Fan et al. (2014) enjoys a $\sqrt{slogn/n}$ convergence rate at a given quantile level. We ask if we can achieve the same convergence rate for the penalized regression quantiles uniformly in Δ. Secondly, as commented by Fan et al. (2014) and Wang et al. (2012), L₁-QR typically does not possess the desired model selection oracle property. Can this deficiency be corrected by adopting an adaptively weighted L₁-penalty instead of the L₁-penalty under globally concerned quantile regression? Thirdly, L₁-QR requires an assumption that the active covariate effects do not cross 0 as τ varies in Δ. We hope that this restriction can be removed.

Motivated by the precursor work by Peng et al. (2013) on variable selection under globally concerned quantile regression with a fixed covariate dimension, we study the penalization strategy of adaptively weighting the L₁-penalty in the ultra-high dimensional setting, in combination with a GIC-type uniform selector of the tuning parameter. The theoretical development for the high dimensional case cannot rely on the traditional empirical process arguments used in Peng et al. (2013); instead we modify a chaining idea from Talagrand (2005) to circumvent the difficulty with high dimensions. We are able to show that with probability tending to one, the proposed estimator can successfully identify the set of relevant covariates, including those having effects on some or all quantile levels in Δ. The model selection oracle property can be established, implicating the same convergence rate for the proposed estimator and the oracle estimator. We employ the empirical process techniques to derive the convergence rate of the oracle estimator and thus that of the proposed estimator. We demonstrate that the adaptively weighted penalties can reduce the bias induced by the L₁ penalties. Compared to L₁-QR, which may select a model of size as large as O(n/log p), the proposed method can achieve consistent model selection, and then an improved estimation convergence rate, which we show can be $\sqrt{slogn/n}$ uniformly in τ ∈ Δ. When Δ is a singleton set and our estimator becomes one for penalized locally concerned quantile regression, the convergence rate can be further improved to the oracle rate, $O_p(\sqrt{s/n})$. Our theoretical investigations do not preclude the cases where the covariates effects equal zero at some quantile.

In addition, we show that any linear combination of the proposed estimator converges weakly to a Gaussian process as a function of τ ∈ Δ. Such weak convergence results have been lacking in the high dimensional setting, possibly because the increasing dimensionality makes the classical approaches to weak convergence inapplicable. With a mild constraint on the model size, we show, for the first time, that the proposed GIC-type uniform tuning parameter selector can ensure consistent model selection in the ultra-high dimensional setting.

The rest of the article is organized as follows. In Section 2, we introduce a globally concerned quantile regression framework and propose an adaptively weighted L₁-penalization procedure. In Section 3, we investigate the theoretical properties of the proposed estimator. We conduct Monte Carlo study to evaluate its finite sample performance in Section 4. In Section 5, we demonstrate the proposed method with a real data example. Further discussions are contained in Section 6. We defer all technical proofs to Section 7.

2. Adaptively weighted L₁-penalized quantile regression

2.1. A globally concerned quantile regression framework

We consider a globally concerned quantile regression model, which takes the form,

$$Q_Y(\tau \mid \mathbf{X})={\mathbf{X}}^T{\mathit{\beta }}_0(\tau ),\text{for}\tau \in \mathrm{\Delta },$$ (2.1)

where Q_Y(τ|X) := inf{y : Pr(Y ≤ y|X) ≥ τ} denotes the τth conditional quantile of a response variable Y given X, X := (1, Z^T)^T is a p × 1 vector of covariates, ${\mathit{\beta }}_0(\tau ):={({\alpha }_0(\tau ),{\mathit{\beta }}_0^{\ast }{(\tau )}^T)}^T$ is a p × 1 vector of unknown coefficient functions of τ, Δ ⊂ (0, 1) is a prespecified set of interest. Here, Δ can take a general form as the union of multiple disjoint intervals. In contrast, a locally concerned quantile regression model can be expressed as (2.1) but with Δ being a singleton set or a countable set. The coefficients, α₀(τ) and ${\mathit{\beta }}_0^{\ast }(\tau )$, represent intercept and covariates effects on the τ-th conditional quantile of Y given X respectively, which are allowed to vary over τ ∈ Δ. It is worth pointing out that the globally concerned quantile regression model (2.1) bears a meaningful distinction from a location shift linear model only when the covariates are confined to a compact set (Koenker, 2005). By this consideration, a bounded covariate space is assumed throughout the presentation of theoretical results in Section 3. Nevertheless, our technical proofs in Section 5 adopt a weaker assumption on covariates, which allows the support of covariates to expand with n.

We consider the ultra-high dimensional setting, where log p = o(n^b) for some b > 0. The set of relevant/active covariates under the globally concerned quantile regression model (2.1) is defined as

$$S_{\mathrm{\Delta }}:=\text{support}({\mathit{\beta }}_0(\tau ),\tau \in \mathrm{\Delta })=\left\{j\in \{2,\cdots ,p\}:\exists \tau \in \mathrm{\Delta },\mid {\beta }_0^{(j)}(\tau )\mid >0\right\}.$$

The implication from this definition is that we intend to identify all variables that impact the selected segment of the conditional distribution of the response (reflected by the choice of Δ). A relevant variable may influence all or some of the quantiles of interest. Let u^(j) denote the jth component of the vector u. To ensure the model identifiability of (2.1), we impose a global sparsity condition, which assumes that the number of relevant covariates is small relatively to the sample size, that is, |S_Δ| = s = o(n), where |·| denotes the cardinality. It is easy to see that this global sparsity condition implies the local sparsity condition for a τ-th quantile regression model, |S_τ| = o(n), where

$$S_{\tau }:=\text{support}({\mathit{\beta }}_0(\tau ))=\left\{j\in \{2,\cdots ,p\}:\mid {\beta }_0^{(j)}(\tau )\mid >0\right\}.$$

It is important to note that the stronger global sparsity condition is indispensable for the globally concerned quantile regression. Suppose we only assume the local sparsity for each τ ∈ Δ. Although |S_τ| = o(n) for all τ ∈ Δ, |S_Δ| = |⋃_τ∈Δ S_τ| could still be greater than n when β̰₀(τ) is allowed to change with τ and so is S_τ. On the other hand, Belloni and Chernozhukov (2011) studied the globally concerned L₁-QR under the local sparsity condition, |S_τ| < s ≤ n/log(n ∨ p) for all τ ∈ Δ. This condition, when coupled with their constraint that the active covariate effects do not cross 0, implies |S_Δ| = o(n), the global sparsity condition assumed in this paper.

Without loss of generality, we assume that S_Δ = {2, ···, s}, i.e. the first s variables have non-vanishing effects on the conditional distribution of interest. We use $S_{\mathrm{\Delta }}^c:=\{s+1,\cdots ,p\}$ to denote the collection of all irrelevant variables. We allow the number of covariates p = p_n and the true model size s = s_n to increase with the sample size n. To ease the presentation, we often omit the subscript n, when it is clear from the context.

2.2. Adaptively weighted L₁-penalized quantile regression

Given a random sample consisting of n independent and identically distributed observations, denoted by {(Y_i, Z_i), i = 1, ···, n}, we use the adaptively weighted L₁-penalized quantile regression estimator β̂_λn(τ), which is the minimizer of the following objective function:

$$Q_n(\mathit{\beta };\tau )=\sum _{i=1}^n{\rho }_{\tau }(Y_i-X_i^T\mathit{\beta })+{\lambda }_n\sum _{j=2}^p{\omega }_j(\tau )\mid {\beta }^{(j)}\mid ,\text{for}\tau \in \mathrm{\Delta },$$ (2.2)

where ${\mathbf{X}}_i:={(1,{\mathbf{Z}}_i^T)}^T$, ρ_τ(t) := t(τ − 1{t ≤ 0}) is the τ-th quantile loss function, λ_n is a tuning parameter controlling the overall model complexity over Δ, and ω_j(τ), j = 2, ···, p, are nonnegative weight functions assigned to β^(j)(τ)’s. Consequently, the estimated global model,

$${\widehat{S}}_{\mathrm{\Delta }}:=\left\{j\in \{2,\cdots ,p\}:\exists \tau \in \mathrm{\Delta },\mid {\widehat{\beta }}^{(j)}(\tau )\mid >0\right\},$$

is a collection of all active variables identified by β̂(τ) over Δ. The general requirements about w_j(τ) and λ_n are expounded in theoretical studies presented in Section 3. Note that, adopting adaptive weights distinguishes the proposed estimation method from L₁-QR in which the penalties assigned to covariates are non-adaptive.

The proposed globally concerned quantile regression framework allows for more flexibility in selecting the form of ω_j(τ), as compared to locally concerned quantile regression. For example, we can choose ω_j(τ) to be one of the following functions:

$$\begin{array}{ll}(w1){\omega }_j(\tau )=1/\mid {\stackrel{\sim }{\beta }}^{(j)}(\tau )\mid ,\hfill & (w2){\omega }_j(\tau )=1/({sup}_{\tau \in \mathrm{\Delta }}\mid {\stackrel{\sim }{\beta }}^{(j)}(\tau )\mid ),\hfill \\ (w3){\omega }_j(\tau )=1/{\int }_{\mathrm{\Delta }}\mid {\stackrel{\sim }{\beta }}^{(j)}(\tau )\mid d\tau ,\hfill & \hfill \end{array}$$

where β̃(τ) is any uniformly consistent initial estimate of β₀(τ). The design of these adaptive weight functions essentially follows the same idea behind the classical adaptive-Lasso (Zou, 2006) to assign different penalty levels according to the conjectured variable importance. The choice (w1) corresponds to a traditional choice of adaptive weights as seen in adaptive-Lasso (Zou, 2006), while the other two examples reflect a global use of the information on β₀(τ), because ω_j(τ) is the same for all τ ∈ Δ. By using (w2) or (w3), we take the view that the variable importance is respectively captured by the maximum signal strength or the cumulative signal strength over τ ∈ Δ, corresponding to the L^∞ and L¹ norm of coefficient functions. Clearly other function norms may also be used to define adaptive weights. Compared to (w2) and (w3), the adaptive weight (w1) is less tailored to the objective of identifying the set of globally relevant variables; it changes with τ and reflects the local relevance rather than the global importance of variables. As elaborated in Section 3, the proposed adaptively weighted L₁-penalized quantile regression can achieve the model selection oracle property with all the weights listed above. However, (w1) is generally subject to stronger signal conditions compared to (w2) and (w3). Our numerical studies suggest that adopting (w2) or (w3) may lead to more favorable finite sample performance as compared to using the traditional adaptive weight, (w1).

For the initial estimator β̃(τ), we recommend using the estimator from L₁-QR, which is a minimizer of (2.2) by setting ω_j(τ) = 1, j = 2, ···, p. Therefore, the proposed adaptively weighted L₁-penalized quantile regression consists of two stages in the implementation. In the first stage, we employ the L₁-QR to generate β̃(τ) and then construct the adaptively weight function ω_j(τ), which are used in the second stage to produce the function estimator, β̂_λn(·).

Another important component of the proposed penalized estimation is to adopt a common tuning parameter λ_n for τ ∈ Δ. This is key to achieving the oracle model selection property for globally concerned quantile regression. If we tune λ_n(τ) for local sparsity, it would result in inconsistent selection for the set of globally irrelevant covariates, S_Δ. This phenomenon shares a similar spirit to the issue of inflated type I error in the multiple comparison setting, and is clearly demonstrated via some simulations reported in Section 4.

To select λ_n, we propose a uniform selector of tuning parameter by using a GIC criterion adapted to the globally concerned quantile regression model (2.1):

$$\mathit{GIC}(\lambda ):={\int }_{\mathrm{\Delta }}log{\widehat{\sigma }}_{\lambda }(\tau )d\tau +\mid {\widehat{S}}_{\lambda }\mid {\phi }_n$$ (2.3)

where ${\widehat{S}}_{\lambda }:=\{j\in \{2,\cdots ,p\}:{sup}_{\tau \in \mathrm{\Delta }}\mid {\widehat{\beta }}_{\lambda }^{(j)}(\tau )\mid \ne 0\},{\widehat{\sigma }}_{\lambda }(\tau ):=n^{-1}{\sum }_{i=1}^n{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_{\lambda }(\tau ))$, and φ_n is a sequence converging to 0 with n. We show in Theorem 3.4 that the proposed GIC criterion would ensure the oracle model selection property when a reasonable upper bound for model size can be imposed to eliminate clearly oversized models in model selection. The simulation studies in Section 4 demonstrate satisfactory behaviors of the proposed tuning parameter selection.

2.3. Computation Algorithm for β̂_λn(τ)

Here we assume that the adaptive weight w_j(τ) is a function or functional of the initial estimator β̃_λn(·).

To begin, we review some important results about standard quantile regression to help understand the proposed computation algorithm. As elaborated in Koenker and Bassett (1978); Bassett and Koenker (1982); Portnoy (1991), the sample regression quantile function is piecewise constant on τ ∈ (0, 1), due to the parametric programming nature of the optimization problem. By Corollary 3.1 of Portnoy (1991), the size of breakpoint set for a regression quantile function is roughly O(mlogm), where m is the number of data points involved in the quantile regression minimization problem. Koenker and D’Orey (1987) provided a detailed account of how to compute a regression quantile function for all τ ∈ (0, 1) via parametric linear programming including the procedure to determine all the breakpoints. The implementation of Koenker and D’Orey (1987)’s algorithm has been made available in rq() function in R package quantreg.

Below we provide a detailed description of the proposed computation algorithm for β̂_λn(·). Our basic strategy for determining the full function path of β̂_λn(·) is to formulate β̂_λn(τ) as regression quantile of some augmented datasets. By the general property of regression quantile functions discussed above, we can conclude that β̂_λn(τ) is a piecewise constant function changing at only a finite number of breakpoints. Our algorithm delineates how the breakpoint set of β̂_λn(·) can be obtained from using rq() function after appropriate data manipulations. We also provide sample code in Section 4 of the supplemental article [Zheng et al. (2015)].

• Step 1: Follow Belloni and Chernozhukov (2011) to obtain the tuning parameter λ̃_n for computing the L₁-QR estimator β̃(τ).

• Step 2: Fit the rq() function to the augmented dataset, { ( $y_i^{\ast },{\mathbf{x}}_i^{\ast }$), 1 ≤ i ≤ n+2p − 2}, where $(y_i^{\ast },{\mathbf{x}}_i^{\ast })=(y_i,{\mathbf{x}}_i)$ for 1 ≤ i ≤ n, $(y_{n+j}^{\ast },{\mathbf{x}}_{n+j}^{\ast })=(0,{\lambda }_n{\mathbf{e}}_j)$, and $(y_{n+p+j}^{\ast },{\mathbf{x}}_{n+p+j}^{\ast })=(0,-{\stackrel{\sim }{\lambda }}_n{\mathbf{e}}_j)$for 1 ≤ j ≤ p − 1. Here e_j is a p-dimensional vector with the jth component equal to 1 and all the others equal to 0. The results would include the set of all breakpoints of β̃(·), denoted by R₁, and also the value of β̃(τ) at each breakpoint. The function estimator β̃(·) is a piecewise constant function that jumps only at breakpoints in R₁.

• Step 3: Compute the adaptive weights ω_j(τ) (j = 2, … p) based on β̃(·). Let

  $$\{I_k(R_1)=[{\tau }_k(R_1),{\tau }_{k+1}(R_1)),k=1,\dots ,\mid R_1\mid \}$$

  be the set of τ-intervals on each of which the weights, ω₂(τ), …, ω_p(τ), are all constant. Here |R₁| denotes the size of the breakpoint set R₁.

• Step 4: Calculate GIC(λ) on a sequence of tuning parameter candidates. More specifically, given any fixed tuning parameter λ,

  • Step 4.1: Set k = 1.

  • Step 4.2: Fit the rq() function to the augmented dataset: { $y_i^{†},{\mathbf{x}}_i^{†}$, 1 ≤ i ≤ n + 2p − 2}, where $(y_i^{†},{\mathbf{x}}_i^{†})=(y_i,{\mathbf{x}}_i)$ for 1 ≤ i ≤ n, $(y_{n+j}^{†},{\mathbf{x}}_{n+j}^{†})=(0,\lambda {\omega }_j(\tau ){\mathbf{e}}_j)$, and $(y_{n+p+j}^{†},{\mathbf{x}}_{n+p+j}^{†})=(0,-\lambda {\omega }_j(\tau ){\mathbf{e}}_j)$ for 1 ≤ j ≤ p − 1. Denote the resulting breakpoint set by R_2,k(R₁). For each breakpoint τ ∈ R_2,k(R₁) ∩ I_k(R₁), let β̂_λ(τ) be the corresponding rq() coefficient estimate.

  • Step 4.3: Increase k by 1 and go back to Step 5.2 unless k > |R₁|.

  • Step 4.4: Calculate GIC(λ) based on β̂_λ(·), which is now fully determined with the breakpoint set given by ${\cup }_{k=1}^{\mid R_1\mid }{R}_{2,k}(R_1)\cap I_k(R_1)$.

• Step 5: Find λ̂_n that minimizes GIC(λ).

• Step 6: Obtain β̂_λn(·) over τ ∈ Δ corresponding to λ = λ̂_n.

It is worth pointing out that Belloni and Chernozhukov (2011) proposed to choose their tuning parameter, λ̃_n, based on simulations of a pivot quantity, and thus Step 1 can be carried out without iterating with other steps. The construction of the augmented dataset in Step 2 is motivated by the fact that the definition of ( $y_{n+k}^{\ast },x_{n+k}^{\ast }$) and ( $y_{n+p+k-1}^{\ast },x_{n+p+k-1}^{\ast }$) makes ${\rho }_{\tau }(y_{n+k}^{\ast }-{\mathbf{x}}_{n+k}^{\ast T}\mathit{\beta })+{\rho }_{\tau }(y_{n+p+k-1}^{\ast }-{\mathbf{x}}_{n+p+k-1}^{\ast T}\mathit{\beta })={\stackrel{\sim }{\lambda }}_n\mid {\beta }^{(k+1)}\mid (k=1,\dots ,p-1)$. Consequently,

$$\sum _{i=1}^{n+2p-2}{\rho }_{\tau }(y_i^{\ast }-x_i^{\ast T}\mathit{\beta })=\sum _{i=1}^n{\rho }_{\tau }(y_i-x_i^T\mathit{\beta })+{\stackrel{\sim }{\lambda }}_n\sum _{j=2}^p\mid {\beta }^{(j)}\mid .$$

This finding indicates that β̃(τ) can be equivalently formulated as the τth regression quantile of the augmented dataset, {( $y_i^{\ast },{\mathbf{x}}_i^{\ast }$), 1 ≤ i ≤ n + 2p − 2}. Thus β̃(τ) is piecewise constant over τ and so is w_j(τ). It allows us to use existing software, such as rq(), to solve the L₁-QR as a standard quantile regression problem and to obtain the full function path of the initial estimator, β̃(·). The same idea is applied in Step 5.2, with additional attention paid to incorporating adaptive weights, which may change with τ. More specifically, given that w_j(τ) is piecewise constant over τ, we can separately handle β̂_λn (·) in each τ-interval where adaptive weights are constant over τ. With data manipulations similar to those for β̃(τ), we can formulate β̂_λn(τ) as the τth regression quantile of an augmented dataset, as shown in Step 5.2. Therefore, the breakpoint set of β̂_λn (·) in the given τ-interval can be obtained from using rq() function. The union of all such breakpoint sets then gives the breakpoint set of β̂_λn(·) (see Step 5.4). In the special case where the weight function is chosen as (w2) or (w3), |R₁| would be 1 and the loop between Steps 5.2 and 5.3 would not be needed.

Note that, the above algorithm enables us to determine β̂_λn(·) for all τ ∈ Δ. However, the computation burden could be heavy in ultra-high dimensional settings, since the size of breakpoint set is roughly O((n + 2p) log(n + 2p)) (Portnoy, 1991). To alleviate the computational burden, one can approximate β̂_λn(·) by a cadlag step function which jumps at a sufficiently fine pre-specified grid in Δ. Specifically, let denote a τ-grid in Δ, inf{Δ} = τ₀ < τ₁ < … < τ_M(n) = sup{Δ}. Define the size of by || || = max{τ_k − τ_k−1; k = 1, …, M(n)}. We propose to approximate β̂_λn(·) by ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^{{\mathcal{S}}_n}(·)$, where for τ ∈ Δ, ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^{{\mathcal{S}}_n}(\tau )={\sum }_{k=1}^{M(n)}{\widehat{\mathit{\beta }}}_{{\lambda }_n}({\tau }_k)I({\tau }_{k-1}<\tau \le {\tau }_k)$. With the smoothness of β₀(·) assumed in condition (C3) in Section 3, we can show that ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^{{\mathcal{S}}_n}(\tau )$ has the same uniform convergence rate and asymptotic distribution as β̂_λn(τ) when (ns)^1/2 || || = o(1). The proof is provided in Section 3 of the supplemental article [Zheng et al. (2015)].

In our simulation studies, we chose the grid of size 5/2n. The estimator based on grid approximation performs well with realistic sample sizes. In our real data analysis, β̂_λn(τ)’s obtained from the exact and approximate calculations select the same set of variables.

Clearly, adopting ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^{{\mathcal{S}}_n}(\tau )$ is computationally appealing because it only requires calculating β̂_λn(·) at M(n) grid points. Given that the only technical constraint for M(n) is lim_n→∞ M (n)/(ns)1/2 = ∞, M (n) can be chosen to be much smaller than the total number of exact breakpoints. For example, under our simulation set-up (II) with n = 200 and p = 400, the average number of breakpoints (with (w2) adopted) from 50 simulations is 4130, while the number of grid points that we used in simulations is 2n/5 = 80. Such a dramatic difference between the number of exact breakpoints and M(n) indicates a huge saving in computation from adopting a grid based approximation instead of the exact calculation of β̂_λn(τ) when n and p are large.

The interpretation of the final model after variable selection may be enhanced by imposing a reasonable semiparametric structure to the coefficient functions in β₀(·). For example, constant effects may be assumed for a subset of selected variables according to preliminary scientific information or exploratory analysis. Taking into account such additional information in the post-variable-selection estimation can lead to efficiency gains in addition to simplified interpretations. Such model polishing may follow similar lines of existing work, such as Qian and Peng (2010) and Yang and He (2012), but will not be further exploited in the paper.

3. Theoretical Properties

In this section, we investigate the theoretical properties of the adaptively weighted L₁-penalized quantile regression. To ease the presentation, we decompose X_i, i = 1, ···, n, into ${({\mathbf{X}}_{ia}^T,{\mathbf{X}}_{ib}^T)}^T$, where X_ia, X_ib correspond to the relevant and irrelevant covariates, respectively. Similarly, we write β as ${({\mathit{\beta }}_a^T,{\mathit{\beta }}_b^T)}^T$. In particular, we have the true coefficients ${\mathit{\beta }}_0(\tau )={({\mathit{\beta }}_{0a}^T(\tau ),\mathbf{0})}^T$.

3.1. Preliminaries

We impose the following regularity conditions to facilitate our technical derivations.

• (C1)
  [Condition on the conditional density] Let f_τ (·|x) denote the probability density function of $Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )$ given X_i = x. For each x at the support of X, sup_τ∈Δf_τ (·|x) < f̄ and inf_τ∈Δ f_τ (0|x) ≥ f for some constants f̄, f > 0. Moreover, there exists a constant A₀ > 0, such that for all u,

  $$\underset{\tau \in \mathrm{\Delta },\mathbf{x}}{sup}\mid f_{\tau }(u\mid \mathbf{x})-f_{\tau }(0\mid \mathbf{x})\mid \le A_0\mid u\mid .$$
• (C2)
  [Conditions on covariates] The covariates are normalized such that $E\left[{(X_i^{(j)})}^2\right]=1$, for j = 2, ···, p, and ${\widehat{\sigma }}_j=\sqrt{{\sum }_{i=1}^n{(X_i^{(j)})}^2/n}$ obeys

  $$\underset{n\to \infty }{lim}Pr\left(\underset{2\le j\le p}{max}\mid {\widehat{\sigma }}_j-1\mid \le 1/2\right)=1.$$
• (C3)
  [Condition on the true regression coefficients β₀(τ)] There exists a positive constant L, such that

  $$\Vert {\mathit{\beta }}_{0a}({\tau }_1)-{\mathit{\beta }}_{0a}({\tau }_2)\Vert \le L\sqrt{s}\mid {\tau }_1-{\tau }_2\mid ,\text{for}\text{all}{\tau }_1,{\tau }_2\in \mathrm{\Delta },$$

  where ||·|| denotes the the l₂-norm, and s is the size of S_Δ.

• (C4)
  [Conditions on the identifiability of the true model] The eigenvalues of $E[{\mathbf{X}}_{ia}{\mathbf{X}}_{ia}^T]$ is bounded from below and above by some constants λ_min and λ_max, respectively. Moreover,

  $$q:=\underset{\mathit{\delta }\in R_s,\mathit{\delta }\ne 0}{inf}\frac{E{\left[{({\mathbf{X}}_i^T\delta )}^2\right]}^{3/2}}{E\left[{\left|{\mathbf{X}}_i^T\delta \right|}^3\right]}>0,$$

  where R_s = {δ ∈ ℝ^p: δ_b = 0}.

Conditions (C1)–(C4) basically follow assumptions imposed in Belloni and Chernozhukov (2011), since we will adopt L₁-QR to construct the consistent initial estimates. Those conditions are also needed to derive the convergence rate of our proposed estimator. Condition (C1) only imposes mild conditions on the conditional density of the response given covariates, and does not assume homoscedasticity. Condition (C2) requires the covariates to be well-behaved with finite second moments. Condition (C3) implies that the true regression coefficient function is Lipschitz continuous with respect to τ. The eigenvalue conditions from (C4) are widely assumed in literature of high dimensional models. They are critical to the model identifiability. The restricted nonlinear impact (RNI) coefficient q controls the quality of minoration of the quantile loss function by a quadratic function over the true global model.

3.2. Main Results

We start with a hypothetical scenario assuming that all relevant covariates are known in advance, and establish the asymptotic properties of the oracle regularized estimator obtained from it. Then we will show that our proposed estimator can enjoy the same properties despite the lack of true model information. The oracle regularized estimator, denoted by ${\widehat{\mathit{\beta }}}^o(\tau )={({({\widehat{\mathit{\beta }}}_a^o(\tau ))}^T,\mathbf{0})}^T$ is the minimizer of (2.2) over R_s. The following theorem shows that the oracle regularized estimator is uniformly consistent over Δ, with the described convergence rate.

Theorem 3.1

Under the regularity conditions (C1)–(C4), if ${sup}_{\tau \in \mathrm{\Delta },j\in S_{\tau }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn})$, then the oracle regularized estimator satisfies

$$\underset{\tau \in \mathrm{\Delta }}{sup}\Vert {\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )\Vert =O_p\left(\sqrt{slogn/n}\right).$$

Theorem 3.1 shows that if λ_n and ω_j(τ)’s are appropriately chosen, the bias introduced by the penalty terms can be well controlled. The uniform upper bound for $\Vert {\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )\Vert$ indicates the uniform consistency of the oracle regularized estimator with the convergence rate $\sqrt{slogn/n}$, which is within a logarithmic factor of the optimal rate $\sqrt{s/n}$ shown by He and Shao (2000). If we only consider a finite number of τ’s, we obtain the following corollary,

Corollary 3.1

Suppose the regularity conditions (C1)–(C4) hold. Given a fixed τ₀, if ${max}_{j\in S_{{\tau }_0}}{\lambda }_n{\omega }_j({\tau }_0)=O_p(\sqrt{n})$, then the oracle regularized estimator satisfies

$$\Vert {\widehat{\mathit{\beta }}}_{{\lambda }_n}^o({\tau }_0)-{\mathit{\beta }}_0({\tau }_0)\Vert =O_p\left(\sqrt{s/n}\right).$$

According to Corollary 3.1, the locally concerned oracle regularized estimator can achieve the optimal convergence rate. This suggests that the additional logarithmic factor in Theorem 3.1 is needed to guarantee the uniform convergence.

Next, we establish the weak convergence for the penalized oracle estimator, ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$. We define two p × p matrices,

$$H_{\tau }:=\left(\begin{array}{cc}E[{\mathbf{X}}_{ia}{\mathbf{X}}_{ia}^T{f}_{\tau }(0\mid {\mathbf{X}}_{ia})]& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right)\text{and}{H}_{\tau }^{-1}:=\left(\begin{array}{cc}{[E[{\mathbf{X}}_{ia}{\mathbf{X}}_{ia}^T{f}_{\tau }(0\mid {\mathbf{X}}_{ia})]]}^{-1}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right).$$

Theorem 3.2

Suppose the regularity conditions (C1)–(C4) hold, s = o (n^1/3), and ${sup}_{j\in S_{\tau },\tau \in \mathrm{\Delta }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn})$. Given any α ∈ ℝ^p such that ||α|| = 1, we have

1. if $\sqrt{n/(slogn)}{inf}_{2\le j\le s,\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid \to \infty$, then we have

   $$2n^{1/2}{\mathit{\alpha }}^T{H}_{\tau }\left({\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )+\frac{{\lambda }_n}{2n}{H}_{\tau }^{-1}{[\mathit{\omega }(\tau )\circ \mathit{sgn}({\mathit{\beta }}_0(\tau ))]}^T\right)$$

   converges weakly to a mean zero Gaussian process with covariance $\mathrm{\sum }(\tau ,{\tau }^{\prime }):=(\tau \wedge {\tau }^{\prime }-\tau {\tau }^{\prime })E[{\mathit{\alpha }}_a^T{\mathbf{X}}_{ia}{\mathbf{X}}_{ia}^T{\mathit{\alpha }}_a]$. Here ∘ denotes the Hadamard product, ^ is the minimum operator, and ω(τ) = (ω₁(τ), ···, ω_p(τ))^T.

2. if sup_τ∈Δ n^−1/2λ_n||ω_a(τ)|| = o_p(1), then

   $$2n^{1/2}{\mathit{\alpha }}^T{H}_{\tau }\left({\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )\right)$$

   converges weakly to a mean zero Gaussian process with covariance Σ(τ, τ′).

Theorem 3.2 states two weak convergence results under two different conditions. Part (a) requires that ${inf}_{2\le j\le s,\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid$, the minimum signal strength of all relevant covariates is beyond the level $O(\sqrt{slogn/n})$. This would exclude the cases where some active variables have coefficient functions crossing 0 or partially equal to 0. Part (b) assumes sup_τ∈Δ n^−1/2λ_n||ω_a(τ)|| = o_p(1). If we choose adaptive weight (w2), with β̃(τ) obtained from Belloni and Chernozhukov (2011), then we have

$$\underset{\tau \in \mathrm{\Delta }}{sup}{\lambda }_n\Vert {\mathit{\omega }}_a(\tau )\Vert \le {\lambda }_n\sqrt{s}{\left[\underset{2\le j\le s}{min}\underset{\tau \in \mathrm{\Delta }}{sup}\left(\mid {\beta }_0^{(j)}(\tau )\mid +O_p\left(\sqrt{slogp/n}\right)\right)\right]}^{-1}.$$

Thus, the condition in (b) is satisfied if $\sqrt{n}/({\lambda }_n\sqrt{s}){min}_{2\le j\le s}{sup}_{\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid \to \infty$ and $\sqrt{slogp/n}=o({min}_{2\le j\le s}{sup}_{\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid )$. These two conditions require ${sup}_{\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid$, j = 2, ···, s the maximum impact of each relevant variable is beyond the level $O(\sqrt{slogp/n}\vee {\lambda }_n\sqrt{s}/\sqrt{n})$. Yet $\sqrt{slogn/n}=o(\sqrt{slogp/n}\vee {\lambda }_n\sqrt{s}/\sqrt{n})$ is typical in the high dimensional cases, and the varying covariate effects are allowed to cross zero or partially equal zero in our framework. Using similar arguments, with adaptive weight (w3) used, condition (b) is satisfied if ${inf}_{2\le j\le s}{\int }_{\mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid d\tau$ is beyond the level $O(\sqrt{slogp/n}\vee {\lambda }_n\sqrt{s}/\sqrt{n})$. Thus, condition (b) is neither weaker nor stronger than condition (a). If we choose (w1) as the adaptive weight, then condition (b) can be satisfied by the signal assumption that ${inf}_{2\le j\le s,\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid$ is beyond the level $O(\sqrt{slogp/n}\vee {\lambda }_n\sqrt{s}/\sqrt{n})$. Therefore, Theorem 3.2 entails weaker signal conditions from using global adaptive weights, (w2) and (w3), than that from adopting the local adaptive weight, (w1).

For the oracle regularized estimator to have asymptotic normality, the true model size is allowed to be o(n^1/3), the fastest model growth rate required in Welsh (1989) and He and Shao (2000) for the unpenalized quantile regression estimator to have the asymptotic normality.

If we can show the the oracle regularized estimator ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$ is also the minimizer of (2.2) over ℝ^p with probability tending to 1, then our proposed estimator enjoys the same properties established in Theorem 3.1 and Theorem 3.2. The following theorem shows that with probability approaching 1, ${\widehat{\mathit{\beta }}}_{{\lambda }_n}(\tau )={\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$.

Theorem 3.3

Suppose the regularity conditions (C1)–(C4) and ${sup}_{j\in S_{\tau },\tau \in \mathrm{\Delta }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn})$ hold. Furthermore, we assume following conditions holds,

$$\underset{j>s,\mathit{\delta }\in R_s}{sup}\frac{E[{(X_i^{(j)}{\mathbf{X}}_i^T\mathit{\delta })}^2]}{{\Vert \mathit{\delta }\Vert }^2}=o\left(\frac{logp}{s{log}^{2}n}\right),$$

and n/(s² log p) → ∞. If ${\lambda }_n/(\sqrt{s}logp)\to \infty$ and ${({inf}_{j>s,\tau \in \mathrm{\Delta }}{\omega }_j(\tau ))}^{-1}\sqrt{n}/\sqrt{slogp}=O_p(1)$, then we have

$$Pr\left(\underset{\tau \in \mathrm{\Delta }}{sup}\mid {\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\widehat{\mathit{\beta }}}_{{\lambda }_n}(\tau )\mid =\mathbf{0}\right)\to 1.$$

Theorem 3.3 not only establishes the model selection oracle property of β̂_λn(τ), but also indicates β̂_λn (τ) enjoys the weak convergence under mild conditions stated in Theorem 3.2. Moreover, Theorem 3.3, coupled with Theorem 3.1, indicates that the bias of the proposed estimator can be bounded by $o(\sqrt{slogn/n})$, the same rate achieved by Fan et al. (2014) for AR-Lasso at a single quantile level. In high-dimensional settings, this bound can be smaller than $o(\sqrt{slogp/n})$, a bias bound established by Belloni and Chernozhukov (2011), which is applicable to the globally concerned quantile regression method with nonadaptive L₁ penalty.

The condition ${sup}_{j>s,\mathit{\delta }\in R_s}E[{(X_i^{(j)}{\mathbf{X}}_i^T\mathit{\delta })}^2]/{\Vert \mathit{\delta }\Vert }^2=o(logp/(s{log}^{2}n))$ essentially asks for weak correlations between relevant covariates and irrelevant covariates so that relevant and irrelevant covariates can be well separated. It can be seen that the model growth rate condition, n/(s² log p) → ∞, is satisfied by the conditions for Theorem 3.2. The conditions, ${sup}_{j\in S_{\tau },\tau \in \mathrm{\Delta }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn})$ and ${\lambda }_n/(\sqrt{s}logp)\to \infty$, indicate different signal strength requirements under different choices of adaptive weights. With adaptive weight (w1), the signal condition,

$$\underset{n\to \infty }{lim}\underset{2\le j\le s,\tau \in \mathrm{\Delta }}{inf}\mid {\beta }_0^{(j)}(\tau )\mid \frac{\sqrt{n}}{\sqrt{s}log(p)}=\infty ,$$ (3.1)

is necessary for one to apply Theorem 3.3. When (w2) or (w3) is used, the signal condition can be relaxed to ${lim}_{n\to \infty }{inf}_{2\le j\le s}{sup}_{\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid \frac{\sqrt{n}}{\sqrt{s}log(p)}=\infty$, or ${lim}_{n\to \infty }{inf}_{2\le j\le s}{\int }_{\tau \in \mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid d\tau \frac{\sqrt{n}}{\sqrt{s}log(p)}=\infty$ respectively. These weaker signal conditions can well accommodate scenarios with covariate effects crossing zero at some τ ∈ Δ.

In the presence of heavy-tailed distributions, the signal condition (3.1), coupled with the estimation error bounded established in Theorem 3.1 for the oracle estimator, implies uniform sign consistency across τ ∈ Δ, namely ${lim}_{n\to \infty }Pr({sup}_{\tau \in \mathrm{\Delta },2\le j\le s}I\{sgn({\widehat{\beta }}_{{\lambda }_n}^{(j)}(\tau ))\ne sgn({\beta }_0^{(j)}(\tau ))\}=1)=0$. Consider the special case with finite s and p = O(n^1/2), which Fan et al. (2014) (see Proposition 1) used to investigate the suboptimality of Lasso in the case of heavy-tailed error distributions. Our signal requirement for globally concerned quantile regression achieving uniform sign consistency can be weaker than what is required by least squares-based Lasso. This finding provides further evidence for the suboptimality of Lasso in the presence of heavy-tailed distributions.

3.3. GIC procedure for λ_n

The performance of the proposed adaptively weighted L₁-penalized quantile regression hinges on the choice of tuning parameter λ_n. To achieve the model selection oracle property, the tuning parameter λ_n and the adaptive weights ω_j(τ)’s are required to satisfy the following conditions: ${sup}_{j\in S_{\tau },\tau \in \mathrm{\Delta }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn}),{\lambda }_n/(\sqrt{s}logp)\to \infty$, and ${({inf}_{j>s,\tau \in \mathrm{\Delta }}{\omega }_j(\tau ))}^{-1}\sqrt{n}/\sqrt{slogp}=O_p(1)$. However, the theoretically optimal λ_n is not practically achievable, because it depends on the unknown true model size s. Although many existing criteria including AIC and cross-validation could be potentially employed to select the tuning parameter, Wang et al. (2007) and Zhang et al. (2010) showed that the tuning parameter selected by AIC and cross-validation may fail to consistently identify the true model. Wang et al. (2009) considered the tuning parameter selection with a modified BIC in the setting of high dimensional linear regression with p < n, and Wang and Zhu (2011) extended the modified BIC to the situation where p could be larger than n. They demonstrated the modified BIC can identify the true model consistently for high dimensional linear regression. The modified BIC can be considered as the generalized information criterion (GIC) according to from Nishii (1984). Motivated by those results, we select the practically optimal λ̂_n by minimizing GIC.

Fan and Tang (2013) recently considered the GIC tuning parameter selection in high dimensional penalized generalized linear regression. In addition to the challenging aspects commented by Fan and Tang (2013), our problem has one more difficult aspect due to varying τ ∈ Δ. As a result, we introduce ∫_Δlog σ̂_λ(τ)dτ to measure the cumulative model fitting of varying effects over Δ.

We define over-fitted model as OF := {S : S ⫌ S_Δ}, and under-fitted model as UF := {S : S ∉ OF, S ≠ S_Δ}. To study the properties of the proposed GIC in (3.2), we first introduce an axillary GIC of model S, which is defined as

$$\text{GIC}(S)={\int }_{\tau \in \mathrm{\Delta }}log({\widehat{\sigma }}_S(\tau ))d\tau +\mid S\mid {\phi }_n,$$ (3.2)

where ${\widehat{\sigma }}_S(\tau )=n^{-1}{\sum }_{i=1}^n{\rho }_{\tau }\left(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_S(\tau )\right)$ measure the fitness of model S at τ-th quantile, and β̂_S(τ) is obtained by the unpenalized quantile regression. We also define $\sigma (\tau )=n^{-1}{\sum }_{i=1}^n{\rho }_{\tau }\left(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )\right)$, which serves as a baseline of model fitness.

In addition, we set a model size upper bound, denoted by C_m(n), with s < C_m(n) < p. This means, we limit model search to submodels with size no more than C_m(n). Such an assumption has been widely assumed in the studies of tuning parameter selection for high dimensional models [see e.g. Chen and Chen (2008), Wang and Zhu (2011), Fan and Tang (2013)]. On one hand, it reduces the number of candidate models from 2^p to 2^C_m(n) and greatly alleviates the computation. On the other hand, it is needed so that the fitness of the model can be correctly measured.

To establish the asymptotic properties of the GIC tuning parameter selector, we assume the following condition (C4+), which is an enhanced version of (C4):

• (C4+)
  1. $${\mathrm{\Lambda }}_{min}:=\underset{\mathit{\delta }\in {ℝ}^p,{\Vert \mathit{\delta }\Vert }_0\le C_m(n),\mathit{\delta }\ne 0}{inf}\frac{{\mathit{\delta }}^{T}E[{\mathbf{X}}_i{\mathbf{X}}_i^T]\mathit{\delta }}{{\Vert \mathit{\delta }\Vert }^2}>0,$$
  2. $${\mathrm{\Lambda }}_{max}:=\underset{\mathit{\delta }\in {ℝ}^p,{\Vert \mathit{\delta }\Vert }_0\le C_m(n),\mathit{\delta }\ne 0}{sup}\frac{{\mathit{\delta }}^{T}E[{\mathbf{X}}_i{\mathbf{X}}_i^T]\mathit{\delta }}{{\Vert \mathit{\delta }\Vert }^2}<\infty ,$$
  3. $$q^{\prime }:=\underset{\mathit{\delta }\in {ℝ}^p,{\Vert \mathit{\delta }\Vert }_0\le C_m(n),\mathit{\delta }\ne 0}{inf}\frac{E{\left[{({\mathbf{X}}_i^T\delta )}^2\right]}^{\frac{3}{2}}}{E\left[{\left|{\mathbf{X}}_i^T\delta \right|}^3\right]}>0,$$

  where ||·||₀ denotes the L₀ norm.

Recall φ_n is defined in (2.3) as a sequence converging to 0. Let ξ_n denote ${min}_{1\le j\le s}{\int }_{\mathrm{\Delta }}\mid {\beta }_0^{(j)}(\tau )\mid d\tau$, which measures the minimal overall effect of intercept and relevant variables upon the conditional distribution. We characterize the behavior the GIC for any model S with size no more than C_m(n) by the following theorem:

Theorem 3.4

Suppose conditions (C1), (C2), (C3) and (C4+) hold. If log p/n = o (φ_n), ${\phi }_n=o\left({\xi }_n^{5/2}\right)$, and $C_m(n)logp/n=o\left({\xi }_n^3\right)$, we have

$$Pr\left(\underset{S\ne S_{\mathrm{\Delta }},\mid S\mid \le C_m(n)}{inf}\mathit{GIC}(S)>\mathit{GIC}(S_{\mathrm{\Delta }})\right)\to 1.$$

Theorem 3.4 indicates that the true model S_Δ has the smallest GIC value among the models with size no more than C_m(n). Let λ̂_n denote the minimizer of (2.3). The following corollary shows that the proposed GIC tuning parameter selector can consistently identify the true model with probability approaching 1,

Corollary 3.2

Under the same conditions as in Theorem 3.4, if ${sup}_{\tau \in \mathrm{\Delta },j\in S_{\tau }}{\omega }_j(\tau )=O_p(\sqrt{n}/(\sqrt{s}logp))$, and ${({inf}_{j>s,\tau \in \mathrm{\Delta }}{\omega }_j(\tau ))}^{-1}\sqrt{n}/\sqrt{slogp}=O_p(1)$, then we have

$$Pr\left({\widehat{S}}_{{\widehat{\lambda }}_n}=S_{\mathrm{\Delta }}\right)\to 1,$$

where Ŝ_λ is defined in (2.3).

4. Simulation studies

We conduct simulation studies to evaluate the finite sample performance of the proposed method. We consider the adaptively weighted L₁-penalized quantile regression with three weight functions, (w1), (w2) and (w3), denoted by AW₁, AW₂ and AW₃ respectively. We compare them with several other methods: (i) the adaptive-LASSO quantile regression at a single predetermined quantile level τ, denoted by SS(τ); (ii) adaptive-LASSO with least squares objective function, denoted by LS; (iii) the L₁-QR over Δ, which is the initial estimator (Step 1) of our proposed algorithm; (iv) the pointwise approach by collecting estimates from SS(τ) over τ ∈ Δ denoted by PS1; and (v) the onestep estimate from PS1 by unpenalized quantile regression using variables selected by PS1 over τ ∈ Δ, which is denoted by PS2. Note that, PS1 and PS2 have the same variable selection results but differ in their coefficient estimates. Except for L₁-QR, the tuning parameter λ_n for the other methods is selected by a GIC criterion with φ_n = log(log n) log p/n. The candidate values for λ_n include the n/4 equally-spaced grid points between 0 and n/10. We use the sample size n = 200, and generate Y based on model (2.1) with p = 400 covariates. We consider four setups, denoted by (I), (II), (III) and (IV).

Setup (I)

The intercept coefficient function is $\sqrt{2}{\mathrm{\Phi }}^{-1}(\tau )$, where Φ⁻¹ is the quantile function of the standard normal distribution. The covariates Z_i are generated from the multivariate normal distribution N_p(0, Σ) with Σ = (σ_jk)_p×p and σ_jk = 0.5^|j−k|. The coefficients for Z⁽¹⁾, Z⁽²⁾, Z⁽⁵⁾, Z⁽⁸⁾, Z⁽¹²⁾ and Z⁽¹⁶⁾ are set to be 2, 1.5, 3, 1, 0.9, and 1 constantly over τ, and the other coefficients are zero. It is clear that setup (I) is a standard linear regression model with normal random errors.

Setup (II)& (III)

The intercept coefficient function is set to be 0, and only Z⁽¹⁾, Z⁽²⁾ and Z⁽⁸⁾ have nonzero coefficients. We plot the coefficient functions for Z⁽¹⁾, Z⁽²⁾ and Z⁽⁸⁾ in Figure 1(a) and (b), respectively for these two setups. We first generate Z̃_i from N_p(0, Σ) where Σ is the same as that used in Setup (I), and then set Z_i = Φ(Z̃_i) (componentwise operation), where Φ(·) is the standard normal distribution function. Setup (II) and (III) are designed to assess the performance of the proposed estimator in varying covariate effects models.

Fig. 1.

The coefficient functions in simulation set-ups (II) & (III)

In these setups, we set Δ = [0.1, 0.9] and choose τ = 0.25, 0.50, and 0.75 for SS(τ). We compare the performance of the different methods described above in terms of the following criteria.

• NCN : mean number of correctly identified variables (with nonzero coefficient functions)

• NIN : mean number of incorrectly selected variables

• PUF : percentage of under-fitted models

• PCF : percentage of correctly fitted models

• POF : percentage of over-fitted models

• RAEE_Δ : median of relative absolute estimation errors with respect to the oracle unpenalized quantile estimator β̂^o(τ) over Δ (under the correctly specified sparse model). It is defined as

$${\mathit{RAEE}}_{\mathrm{\Delta }}(\mathit{\beta })=\frac{{\int }_{\mathrm{\Delta }}{\Vert \mathit{\beta }(\tau )-{\mathit{\beta }}_0(\tau )\Vert }_{1}d\tau }{{\int }_{\mathrm{\Delta }}{\Vert {\widehat{\mathit{\beta }}}^o(\tau )-{\mathit{\beta }}_0(\tau )\Vert }_{1}d\tau }$$

The first five criteria are used to assess the model selection performance. We would like to have NCN to be close to the true number of signals (ie. 6 in setup (I) and 3 in setups (II) & (III)). The ideal PCF is 100%, but the other measures (NIN, PUF and POF) should be as close to 0 as possible. The measure RAEE_Δ is aimed to evaluate the global estimation accuracy for τ ∈ Δ. We hope this measure to be as small as possible. For SS(τ) and LS, we calculate RAEE_Δ by extrapolating the coefficient function estimate as the constant valued function over τ ∈ Δ.

In Table 1, we present simulation results for setup (I)–(III) based on 400 simulations. Under setup (I), where the true model is a standard linear model with normal random errors, we find that AW₁, AW₂, AW₃, SS(0.25), SS(0.50), SS(0.75) and LS all have very similar performance in model selection and estimation accuracy. Since the oracle estimator adopted here is based on the quantile regression and the random errors follow a normal distribution, it is reasonable to observe that LS has the best performance in estimation accuracy. Nevertheless, this advantage over the other approaches is only moderate. We also observe that applying the naive strategy of merging variable selection results from all SS(τ), as in PS1 and PS2, tends to produce an overfitted model (2.1) with POF=94.0%. The large RAEE_Δ of PS2 suggests that such an overfitted model tends to have considerably deteriorated estimation accuracy.

Table 1.

Simulation results for setup (I)–(III)

set-up	Method	NCN	NIN	PUF(%)	PCF(%)	POF(%)	RAEEΔ
(I)	AW1	6.00	.09	.0	90.7	9.3	1.54
	AW2	6.00	.07	.0	93.3	6.7	1.39
	AW3	6.00	.06	.0	93.7	6.3	1.36
	SS(0.25)	5.99	.16	1.3	84.5	14.2	1.22
	SS(0.50)	6.00	.07	.0	93.8	6.2	1.06
	SS(0.75)	5.99	.16	.8	85.7	13.5	1.19
	LS	6.00	.28	.0	79.9	20.1	.77
	L1-QR	6.00	109.90	.0	.0	100.0	3.89
	PS1	6.00	2.71	.0	6.0	94.0	1.28
	PS2	6.00	2.71	.0	6.0	94.0	13.45
(II)	AW1	2.84	.04	14.8	80.5	4.7	3.62
	AW2	2.91	.05	7.7	87.8	4.5	1.83
	AW3	2.87	.04	11.5	84.8	4.5	1.92
	SS(0.25)	2.42	.10	39.9	55.7	4.4	4.48
	SS(0.50)	1.99	.00	100.0	.0	.0	3.07
	SS(0.75)	2.51	.05	38.8	59.5	1.7	4.26
	LS	1.95	.18	100.0	.0	.0	3.85
	L1-QR	3.00	26.30	.0	.0	100.0	4.90
	PS1	3.00	1.77	.5	21.7	78.3	2.58
	PS2	3.00	1.77	.5	21.7	78.3	12.04
(III)	AW1	2.83	.19	16.0	70.3	13.7	2.05
	AW2	2.85	.21	14.2	71.0	14.8	1.34
	AW3	2.80	.09	19.5	73.8	6.7	1.38
	SS(0.25)	1.79	.12	100.0	.0	.0	3.64
	SS(0.50)	1.01	.01	100.0	.0	.0	3.29
	SS(0.75)	1.76	.08	100.0	.0	.0	2.70
	LS	2.10	.18	99.5	0.2	.3	3.44
	L1-QR	2.98	26.78	1.8	.0	98.2	3.67
	PS1	2.94	1.65	5.5	24.2	70.3	1.48
	PS2	2.94	1.65	5.5	24.2	70.3	8.78

In setups (II) and (III), we observe that LS has very poor performance in identifying the true model. For example, in setup (II), although Z⁽⁸⁾ has a strong effect on the response at all quantile levels except for τ = 0.5, the symmetric traits make its mean effect on Y neutralized. Thus, LS frequently fails to select Z⁽⁸⁾, yielding NCN= 1.95 and PUF= 100%. Yet, if one simply applies the locally concerned penalized median regression, Z⁽⁸⁾ is characterized as an inactive variable as well and its effects on other segments of the conditional distribution of Y given Z are overlooked. The unsatisfactory results of SS(0.50) supports the use of our globally concerned quantile regression, which enables a more thorough assessment of covariates impacting across various quantiles. Compared to SS(0.25) and SS(0.75), where Z⁽⁸⁾ is an active variable, the proposed method for globally concerned quantile regression seems to considerably enhance the power for selecting the correct set of relevant variables. This is clearly indicated by the PCF improvement ranging between 20% and 30%. Like in setup (I), PS1 and PS2, the naive “global” methods, suffer from the overfitting problem with POF= 78.3%. The superiority of the proposed method is more evident in setup (III), where Z⁽¹⁾ and Z⁽²⁾ have partial effects on two disjoint quantile ranges. In this case, the PCFs of AW₁, AW₂ and AW₃ are still over 70%, which is far above the PCFs achieved by the other methods.

By comparing RAEE_Δ in Table 1, we observe that the proposed globally adaptive estimators outperform the other methods also in estimation accuracy for non-i.i.d error models. In addition, Table 1 suggests that adopting weight functions, (w2) and (w3), which are designed to capture the global signal strength, generally yields better finite-sample performance, compared to the traditional weight (w1). We also note that AW₂ and AW₃ have similar or even smaller RAEE_Δ as compared to PS1, an approach that penalizes for local sparsity separately for each quantile index τ. This shows that the proposed globally adaptive method does not sacrifice estimation accuracy while avoiding the overfitting issue associated with the naive pointwise approach. This observation is in line with our theoretical results in Theorem 3.1 and Corollary 3.1.

We also examine the local estimation accuracy of different methods. Specifically, we compute the L₁ loss of each estimator of β₀(0.5) and then calculate its ratio to that obtained from SS(0.50). The results in Table 2 suggest that our globally concerned penalization method may lose a small proportion of local estimation efficiency compared to the locally concerned SS(0.5), a reasonable price to pay for achieving better global model selection and estimation results. On the other hand, PS2 has much larger local estimation errors compared to the other methods. This may be viewed as a natural consequence of the poor global sparsity control by the naive pointwise approach.

Table 2.

Local estimation error ratio over SS(0.50) at τ = 0.5 for setup (I)–(III)

set-up	AW1	AW2	AW3	PS1	PS2	LS
(I)	1.16	1.13	1.14	1.00	11.01	0.75
(II)	1.19	1.61	1.63	1.00	18.52	2.35
(III)	1.01	1.17	1.18	1.00	6.21	1.33

Setup (IV)

We intend to assess the utility of the proposed method in selecting active variables at tail quantiles. We generate data from a linear regression model, where the covariates Z_i are generated from the multivariate normal distribution N_p(0, Σ), the same covariance matrix used in setup (I). The effects of Z⁽¹⁾, Z⁽²⁾, Z⁽⁵⁾, Z⁽¹²⁾, Z⁽¹⁶⁾ and Z⁽²⁵⁾ are set to be 1.5, 1.25, 2, 4/3, 2 and 3 constantly over τ, and all other covariates have constantly zero coefficients across τ. For each simulated dataset, to identify active variables at quantile level τ = 0.05, we implement SS(0.05) as well as AW₁, AW₂, AW₃, L₁-QR, PS1, PS2 with Δ = [0.03, 0.07]. The results are reported in Table 3 based on 400 simulated data sets. By comparing SS(0.05) with SS(0.25), SS(0.50), and SS(0.75) in setup (I), we observe a larger variability in model selection for quantile regression at the tail quantile with τ = 0.05, reflected by a relatively smaller PCF=51.0%. On the other hand, the proposed methods, AW₁, AW₂ and AW₃, still maintain satisfactory performance in model selection with PCFs above 80%. This demonstrates that, by accumulating information in the neighborhood around a tail quantile, the proposed method may boost the power of correct model selection at tail quantiles. We also conduct simulations with Δ chosen as [0.025, 0.075]. Note that such a choice of Δ may represent the same interest in the response distribution reflected by Δ = [0.03, 0.07]. Thus, it would be desirable to observe that the variable selection results between these two choices of Δ are in close proximity. The results in Table A.1 (see the supplemental article [Zheng et al. (2015)]) are very similar to those in Table 3. The results in Table A.2 further demonstrate a good agreement in the selected variables between the two choices of Δ, [0.03, 0.07] and [0.025, 0.075]. These suggests that the proposed estimators are quite robust to reasonable variations in the choice of Δ.

Table 3.

Simulation results over Δ = [0.03, 0.07] for setup (IV)

set-up	Method	NCN	NIN	PUF	PCF	POF	RAEEΔ
(IV)	AW1	5.83	.01	11.8	87.2	1.0	2.15
	AW2	5.87	.03	11.8	86.2	2.0	1.63
	AW3	5.80	.03	17.8	80.5	1.8	1.62
	SS(0.05)	5.99	.85	1.0	51.0	48.0	1.28
	L1-QR	6.00	35.53	.0	.0	100.0	3.85
	PS1	6.00	3.18	.2	13.8	87.0	1.29
	PS2	6.00	3.18	.2	13.8	87.0	10.55

5. A real data example

We now illustrate the application of the proposed method by analyzing a microarray data set reported by Scheetz et al. (2006). This real dataset contains expression values of 31,042 probe sets on 120 12-week-old male offsprings of rats. As in Huang et al. (2008), Kim et al. (2008) and Wang et al. (2012), we are interested in identifying genes whose expressions are predictive for gene TRIM32, which is associated with a genetically heterogeneous disease of multiple organ systems. The probe corresponding to gene TRIM32 is 1389163 at. Our approach to finding relevant genes is to apply high dimensional quantile regression analyses upon the remaining probe sets. In our analysis, we further narrow down the problem to selecting variables that impact ordinary expression levels of probe 1389163 at, which may be roughly captured by the middle part of the response distribution. Therefore, we formulate the target of our analysis as , considering two reasonable choices of Δ, (0.2, 0.8) and (0.25, 0.75).

We use the data set available in R package flare, which have been processed to exclude probes that are not expressed or lack variation. There remain 200 probe sets serving as covariates. We choose AW₂ as the representative of the proposed globally concerned method. In addition, we also implement the adaptive-LASSO method for locally concerned quantile regression, SS(τ), at τ = 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, and the pointwise approaches, PS1 and PS2, as described in Section 4.

To evaluate each method, we compute its prediction error as follow. First, we randomly split the 120 rats into a training set and a test set, each consisting of 60 subjects. We apply the method to the training data set and obtain the estimator of β₀(τ), denoted by β̂^train(τ). Next, we calculate the prediction error over the test set as

$$PE(\mathrm{\Delta })=\frac{{\sum }_{i=1}^{n}1\{\text{Subject}i\text{in}\text{test}\text{set}\}{\int }_{\mathrm{\Delta }}{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}^{\mathit{train}}(\tau ))d\tau }{{\sum }_{i=1}^{n}1\{\text{Subject}i\text{in}\text{test}\text{set}\}}.$$

For SS(τ), we calculate PE(Δ) by treating the coefficient estimate as a constant valued function over τ ∈ Δ.

Table 4 lists the probe sets selected by each method using all 120 observations. In addition, we present the averages of PE(Δ) along with the corresponding standard deviations (within parentheses). All calculations are based on 400 replications of random splitting into training and test sets.

Table 4.

Probe sets identified by various methods

Δ	Method	Probes	PE(Δ)
			Δ = (0.2, 0.8)	Δ = (0.25, 0.75)
(0.2, 0.8)	AW2	“24565” “25141” “25367” “29045”	.146 (.016)	—
(0.25, 0.75)	AW2	“24565” “25141” “25367” “29045”	—	.124 (.011)
(0.2, 0.8)	PS1	17 probes	.151 (.016)	—
(0.25, 0.75)	PS1	13 probes	—	.129 (.012)
(0.2, 0.8)	PS2	17 probes	.143 (.013)	—
(0.25, 0.75)	PS2	13 probes	—	.124 (.011)
0.25	SS	“6222”, “15863”, “22140”, “25141”, “25439”, “29045”	.191 (.024)	.151 (.018)
0.30	SS	“25141”, “29045”	.181 (.023)	.146 (.017)
0.35	SS	“14949”, “24245”, “25141”, “29045”	.174 (.022)	.141 (.016)
0.40	SS	“25141”, “29045”	.170 (.020)	.138 (.015)
0.45	SS	“25141”	.167 (.020)	.135 (.015)
0.50	SS	“25141”	.165 (.020)	.135 (.015)
0.55	SS	“25141”	.164 (.020)	.136 (.015)
0.60	SS	“21092”, “24565”, “25141”	.167 (.021)	.137 (.017)
0.65	SS	“24565”, “25141”	.169 (.023)	.140 (.018)
0.70	SS	“11711”, “24565”, “25141”	.176 (.025)	.146 (.019)
0.75	SS	“24565”, “25141”, “25367”	.186 (.028)	.155 (.023)

From Table 4, it is clear that the set of probes selected by the locally concerned quantile method considerably varies with τ. Only probe “25141” is selected by all SS(τ)’s considered. The probe “24565”, which was consistently selected by SS(τ) with τ = 0.60, 0.65, 0.70, 0.75, does not show up in the estimated active variable set for lower τ’s. These may suggest a heterogeneous relationship across different segments of the response distribution, but the variations are partly due to the sensitivity of the variable selection methods as τ varies. In fact, there is a quite large variability in the selected probes at low quantile indices. For example, SS(0.25) selected 6 probes, but SS(0.30) selected only 2 probes. It would be difficult to interpret such variable selection results at two close quantile indices.

To identify , PS1 and PS2, which take the union of selected variables from locally concerned SS(τ), select a total number of 13 probes for Δ = (0.25, 0.75) and 17 probes for Δ = (0.2, 0.8). In contrast, the proposed globally concerned quantile regression AW₂ only selects 4 probes but renders very competitive prediction errors. It has the smallest PE(Δ) when Δ = (0.25, 0.75) and the second smallest PE(Δ) when Δ = (0.2, 0.8). The results show that the globally concerned quantile regression approach can mitigate the over-fitting problem of the alternative methods based on local quantile regression approach. Furthermore, AW₂ selects the same set of probes with the two slightly different choices of Δ. This suggests the robustness of our method to small changes of Δ, which is a desirable feature from the perspective of model selection.

6. Remarks

We aim to provide a rigorous statistical approach to identifying variables that are predictive to some or all quantiles at the percentile level in a predetermined set Δ. Our work does not directly address how Δ should be chosen because the choice of Δ should align with the scientific problem at hand. In practice, Δ can be chosen as a large interval, say [0.1, 0.9] to reflect an interest in normal outcomes, or a smaller interval, say [0.75, 0.9] to reflect our interest in the upper tail of the response distribution. Even in the cases where one is interested in a single quantile level τ, our work suggests that applying the proposed method with Δ chosen as a small interval containing τ can lead to more stable variable selection results when the sample size n is limited (relative to p).

In applications, it is likely that we have more than one reasonable choices of Δ. For example, either (0.7, 0.9) or (0.75, 0.9) can be a reasonable choice for Δ when we are interested in identifying variables that impact the upper tails of the response variable. This reinforces our belief that the variable selection results should be quite insensitive to small changes in the choice of Δ. Our simulation studies and data example suggest that our approach is robust to small variations of Δ. These empirical results further endorse the practical value of the proposed method.

7. Technical proofs

We present the proofs of our main results in this section. We start with introducing some notations. Given a random sample Z₁, ···, Z_n, we adopt the following empirical process notations. Let ${\mathbb{G}}_n(f)={\mathbb{G}}_n(f(Z_i)):=n^{-1/2}{\sum }_{i=1}^n(f(Z_i)-E[f(Z_i)])$ and ${\mathbb{E}}_{n}f={\mathbb{E}}_{n}f(Z_i):={\sum }_{i=1}^{n}f(Z_i)/n$. Let ψ_τ (u) = τ − 1{u < 0} denote the score function of ρ_τ(u) and $M_n(\tau ,\mathit{\delta }):=n^{-1/2}{\sum }_{i=1}^n{\mathbf{X}}_i{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )-{\mathbf{X}}_i^T\mathit{\delta })$. We define (t) = {β ∈ ℝ^p : β_b = 0, inf_{τ ∈ Δ} ||β − β₀(τ)|| ≤ τ} and R_s(B) = {δ ∈ R_s : ||δ|| ≤ (s log² n/n)^1/2B} for some B > 0. We used τ^* to denote min{Δ ∪ 1 − Δ} and ∂A to denote the boundary of a set A.

As mentioned in Section 2.1, it is more meaningful to investigate the globally concerned quantile regression model (2.1) when the support of X is compact. However, our theoretical work can be established in more general conditions, where the magnitude of covariates is allowed to increase in a suitable rate with n. In particular, the following condition is assumed for the proof:

• (C5)
  [Conditions on the largest covaraites] We assume

  $$Pr\left(\underset{1\le i\le n,2\le j\le s}{max}\mid X_i^{(j)}\mid \le M_{1n}\right)\to 1\text{and}Pr\left(\underset{1\le i\le n,s+1\le j\le p}{max}\mid X_i^{(j)}\mid \le M_{2n}\right)\to 1,$$

  for some M_1n = o(n^1/2/(s log n)^3/2) and $M_{2n}\le \sqrt{n/(s^{2}logp)}$.

We denote the event that max_2≤j≤p |σ̂_j − 1| ≤ 1/2, the event that ${max}_{1\le i\le n,2\le j\le s}\mid X_i^{(j)}\mid <M_{1n}$, and the event that ${max}_{1\le i\le n,s<j\le p}\mid X_i^{(j)}\mid M_{2n}$ by Ω₀, Ω₁, and Ω₂, respectively. According to conditions (C2) and (C5), we have ${\gamma }_{0n}:=Pr({\mathrm{\Omega }}_0^c)\to 0,{\gamma }_{1n}:=Pr({\mathrm{\Omega }}_1^c)\to 0$, and ${\gamma }_{2n}:=Pr({\mathrm{\Omega }}_2^c)\to 0$.

Next, we present the technical lemmas used in the proofs of our theorems and corollaries. The proofs of lemmas and Corollaries 1–2 are relegated to Section 2 of the supplemental article [Zheng et al. (2015)].

Lemma 7.1

Under conditions (C1)-(C4), if $t<3q\underset{\_}{f}{\lambda }_{min}/(4A_0{\lambda }_{max}^{3/2})$, then for any δ ∈ R_s; ||δ|| ≤ t, we have

$$E[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\ge \frac{\underset{\_}{f}{\lambda }_{min}}{4}{t}^2$$ (7.1)

Lemma 7.2

Under conditions (C2)-(C4), with probability at least 1 − 16/n³ − γ₀, we have

$$\begin{array}{l}\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s,\Vert \mathit{\delta }\Vert \le t}{sup}\mid {\mathbb{G}}_n[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\mid \\ \le 16\sqrt{slogn}t+120\sqrt{s}t\sqrt{logn+logL/2-logt/2}\end{array}$$ (7.2)

Lemma 7.3

Under conditions (C4), given a fixed τ₀, with probability at least 1 − 8/a, we have

$$\underset{\mathit{\delta }\in R_s,\Vert \mathit{\delta }\Vert \le t}{sup}\frac{1}{\sqrt{n}}\mid {\mathbb{G}}_n[{\rho }_{{\tau }_0}(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{{\tau }_0}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\mid \le 12a\sqrt{{\lambda }_{max}s}t$$

Lemma 7.4

Under conditions (C1)-(C5), give any α ∈ R_s, such that ||α|| = 1, if s = o (n^1/3), then we have

$$\underset{\mathit{\delta }\in R_s(B)}{sup}\underset{\tau \in \mathrm{\Delta }}{sup}\mid {\mathit{\alpha }}^T\{M_{n,\tau }(\mathit{\delta })-E[M_{n,\tau }(\mathit{\delta })]-M_{n,\tau }(\mathbf{0})+E[M_{n,\tau }(\mathbf{0})]\}\mid =o_p(1).$$

Lemma 7.5

Under conditions (C1),(C3) and (C4) given an α ∈ R_s such that ||α|| = 1, α^TM_n(τ, 0) converges weakly to a mean zero Gaussian process with variance $\tau (1-\tau )E[{\mathit{\alpha }}_a^T{\mathbf{X}}_{ia}{\mathbf{X}}_{ia}^T{\mathit{\alpha }}_a]$.

Lemma 7.6

Under condition (C1), if $E[\mid Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0({\tau }_0)\mid ]$ exists for some τ₀ ∈ Δ, then we have

$$\begin{array}{l}{\tau }^{\ast }[E[\mid Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0({\tau }_0)\mid ]-{(2\underset{\_}{f})}^{-1}]\le \underset{\tau \in \mathrm{\Delta }}{inf}\sigma (\tau )\\ \le \underset{\tau \in \mathrm{\Delta }}{sup}\sigma (\tau )\le E[\mid Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0({\tau }_0)\mid ]+{(2\underset{\_}{f})}^{-1},\end{array}$$

almost surely.

Lemma 7.7

Under conditions (C1), (C2), (C3), and (C4+), for some k ≤ C_m(n) = o(n/log p), with probability at least 1 − 16 exp(−(A² − 2) log p) − γ₀, we have

$$\underset{S_{\mathrm{\Delta }}\subseteq S,\mid S\mid =k}{sup}\underset{\tau \in \mathrm{\Delta }}{sup}\Vert {\widehat{\mathit{\beta }}}_S(\tau )-{\mathit{\beta }}_0(\tau )\Vert \le 16(2+\sqrt{192})A\sqrt{klogp/n}/(\underset{\_}{f}{\mathrm{\Lambda }}_{min}),$$

for some constant A > 2.

Lemma 7.8

Under the same conditions as in Theorem 3.4, for some constant A > 2, we have

$$Pr\left(\underset{S\in OF,\mid S\mid \le C_m(n)}{inf}\mathit{GIC}(S)>\mathit{GIC}(S_{\mathrm{\Delta }})\right)\ge 1-16exp(-(A^2-3)logp)-12/p-16/n^3-{\gamma }_{0n}.$$

Lemma 7.9

Under the same conditions as in Theorem 3.4, for some constant A > 2, we have

$$Pr\left(\underset{S\in UF,\mid S\mid \le C_m(n)}{inf}\mathit{GIC}(S)>\mathit{GIC}(S_{\mathrm{\Delta }})\right)\ge 1-16exp(-(A^2-3)logp)-16/n^3-{\gamma }_0.$$

Lemma 7.10

Under the same conditions as in Theorem 3.4, for some constant A > 2, we have

$$\underset{S\in OF,\mid S\mid =k\tau \in \mathrm{\Delta }}{inf}{\widehat{\sigma }}_S(\tau )-{\widehat{\sigma }}_{S_{\mathrm{\Delta }}}(\tau )\ge -225{(\underset{\_}{f}{\mathrm{\Lambda }}_{min})}^{-1}(k-s)logp,$$

with probability at least 1 − 16 exp(−(A² − 2) log p) − 12/p − γ_0n.

Lemma 7.11

Under the same conditions as in Theorem 3.3, with probability at least 1 − 4 exp(C₃s log p/2 + 4s log n) − γ₂, for some constant C₃ > 1, we have

$$\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}{n}^{1/2}{\mathit{\alpha }}_j\{M_n(\tau ,\mathit{\delta })-E[M_n(\tau ,\mathit{\delta })]-M_n(\tau ,\mathbf{0})+E[M_n(\tau ,\mathbf{0})]\}\le 4C_3\sqrt{nlogp}$$

In the following, we provide the proofs for the theoretical results in Section 3.2.

Proof of Theorem 3.1

We consider Q_n(β; τ) − Q_n(β₀(τ); τ) over {β ∈ ℝ^p : inf_τ∈Δ ||β − β₀(τ)|| ≤ t}

$$\begin{array}{l}{Q}_n(\mathit{\beta };\tau )-Q_n({\mathit{\beta }}_0(\tau );\tau )=\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathbf{x}}_i^T\mathit{\beta })-\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathbf{x}}_i^T{\mathit{\beta }}_0(\tau ))+{\lambda }_n\sum _{j=2}^p{\omega }_j(\tau )\left(\mid {\beta }^{(j)}\mid -\mid {\beta }_0^{(j)}(\tau )\mid \right)\\ =nE[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T\mathit{\beta })-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]+\sqrt{n}{\mathbb{G}}_n[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T\mathit{\beta })-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]+{\lambda }_n\sum _{j=2}^p{\omega }_j(\tau )\left(\mid {\beta }^{(j)}\mid -\mid {\beta }_0^{(j)}(\tau )\mid \right)\\ :=I_1+I_2+I_3.\end{array}$$

According to Lemma 7.1, if $t<3q\underset{\_}{f}{\lambda }_{min}/(4A_0{\lambda }_{max}^{3/2})$, then we obtain

$$I_1\ge n\underset{\_}{f}{\lambda }_{min}{t}^2/4.$$ (7.3)

By Lemma 7.2, with probability at least 1 − 16/n³ − γ_0n,

$$\underset{\tau \in \mathrm{\Delta }}{sup}\mid I_2\mid \le \sqrt{n}\left\{16\sqrt{slogn}t+12\sqrt{s}t\sqrt{logn+logL/2-logt/2}\right\}.$$ (7.4)

For I₃, since ${sup}_{j\in S_{\tau },\tau \in \mathrm{\Delta }}{\lambda }_n{\omega }_j(\tau )=O_p(\sqrt{nlogn})$, we have

$$\begin{array}{l}{I}_3=\sum _{j\in S_{\tau }}{\lambda }_n{\omega }_j(\tau )\left(\mid {\beta }^{(j)}\mid -\mid {\beta }_0^{(j)}(\tau )\mid \right)+\sum _{j\notin S_{\tau }}{\lambda }_n{\omega }_j(\tau )\left(\mid {\beta }^{(j)}\mid -0\right)\\ \ge \sum _{j\in S_{\tau }}{\lambda }_n{\omega }_j(\tau )\left(\mid {\beta }^{(j)}\mid -\mid {\beta }_0^{(j)}(\tau )\mid \right)\ge -\underset{j\in S_{\tau }}{max}{\lambda }_n{\omega }_j(\tau )\sum _{j\in S_{\tau }}\left(\mid {\beta }^{(j)}-{\beta }_0^{(j)}(\tau )\mid \right)\\ \ge -\underset{j\in S_{\tau }}{max}{\lambda }_n{\omega }_j(\tau )\sqrt{s}t=-O_p(\sqrt{nslogn}t).\end{array}$$ (7.5)

(7.3), (7.4), and (7.5) together imply that with probability at least 1 − 16/n³ − γ_0n,

$$\begin{array}{l}\underset{\mathit{\beta }\in \partial {\mathcal{B}}_s(t),\tau \in \mathrm{\Delta }}{inf}{Q}_n(\mathit{\beta };\tau )-Q_n({\mathit{\beta }}_0(\tau );\tau )\ge n\frac{\underset{\_}{f}{\lambda }_{min}}{4}{t}^2-O_p(\sqrt{nslogn}t)\\ -\sqrt{n}\left(16\sqrt{slogn}t+120\sqrt{s}t\sqrt{logn+logL/2-logt/2}\right)\\ \ge \frac{\underset{\_}{f}{\lambda }_{min}}{4}{C}_1^{2}slogn-O_p(slogn)C_1\ge 0,\end{array}$$

where $t=C_1\sqrt{slogn/n}$, with a sufficiently large C₁.

Since Q_n(β; τ) is a convex function with respect to β, then ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$, the minimizer of (2.2) over R_s, must locate within { $\mathit{\beta }\in {ℝ}^p:\Vert \mathit{\beta }-{\mathit{\beta }}_0(\tau )\Vert \le C_1\sqrt{slogn/n}$}. This completes our proof of Theorem 3.1.

Proof of Theorem 3.2

According to Theorem 3.1, we know that

$$Pr\left(\Vert {\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )\Vert >B\sqrt{s{log}^{2}n/n}\right)\to 0,$$

for some B > 0. Therefore, minimizing the objective function Q_n(β; τ) from (2.2) over R_s is equivalent to minimizing n^−1/2[Q_n(β₀(τ) + δ; τ) − Q_n(β₀(τ); τ)] over R_s(B) with probability approaching 1.

Given any δ ∈ R_s(B), we have

$$\begin{array}{l}{n}^{-1/2}[Q_n({\mathit{\beta }}_0(\tau )+\mathit{\delta };\tau )-Q_n({\mathit{\beta }}_0(\tau );\tau )]\\ ={\mathbb{G}}_n[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\\ +n^{1/2}E[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\\ +n^{-1/2}{\lambda }_n\sum _{j\le s}{\omega }_j(\tau )\left(\left|{\beta }_0^{(j)}(\tau )+{\delta }^{(j)}\right|-\left|{\beta }_0^{(j)}(\tau )\right|\right)\\ ={\mathbb{G}}_n[Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )\{{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))-{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\}]\\ -{\mathit{\delta }}^T[M_n(\tau ,\mathit{\delta })-E[M_n(\tau ,\mathit{\delta })]-M_n(\tau ,\mathbf{0})+E[M_n(\tau ,\mathbf{0})]]\\ -{\mathit{\delta }}^T[M_n(\tau ,\mathbf{0})-E[M_n(\tau ,\mathbf{0})]]\\ +n^{1/2}E[{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))-{\rho }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))]\\ +n^{-1/2}{\lambda }_n\sum _{j\le s}{\omega }_j(\tau )\left(\left|{\beta }_0^{(j)}(\tau )+{\delta }^{(j)}\right|-\left|{\beta }_0^{(j)}(\tau )\right|\right)\\ :={II}_1+{II}_2+{II}_3+{II}_4+{II}_5\end{array}$$

We consider II₂ first. By Lemma 7.4, we have uniformly for δ ∈ R_s(B) and τ ∈ Δ,

$$\mid {II}_2\mid =o_p(\Vert {\mathit{\delta }}_a\Vert ).$$ (7.6)

Next we deal with II₄. From the proof of Lemma 7.1, it is easy to observe uniformly for δ ∈ R_s(B) and τ ∈ Δ,

$$\left|{II}_4-n^{1/2}{\mathit{\delta }}^T{H}_{\tau }\mathit{\delta }\right|\le n^{1/2}\frac{\underset{\_}{f}{\lambda }_{max}^{3/2}}{4q}{\Vert {\mathit{\delta }}_a\Vert }^3=O\left(\frac{s{log}^{2}n}{\sqrt{n}}\Vert {\mathit{\delta }}_a\Vert \right).$$ (7.7)

Now we deal with II₁. Note that,

$$\begin{array}{l}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))\{{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))-{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\}\\ =\left\{\begin{array}{ccc}{Y}_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )& \text{if}& {\mathbf{X}}_i^T\mathit{\delta }<Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\\ -[Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )]& \text{if}& 0<Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<{\mathbf{X}}_i^T\mathit{\delta }\\ 0& \text{otherwise}.& \end{array}\right\}\end{array}$$

then we obtain that

$$\frac{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )}{{\mathbf{X}}_i^T\mathit{\delta }}\{{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau ))-{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\}$$

is a bounded function. Applying similar arguments used in Lemma 7.4, we have uniformly for δ ∈ R_s(B) and τ ∈ Δ

$$\mid {II}_1\mid =o_p(\Vert {\mathit{\delta }}_a\Vert ).$$ (7.8)

For II₃, since E[M_n(τ, 0)] = 0, we have

$${II}_3=-{\mathit{\delta }}^T{M}_n(\tau ,\mathbf{0}).$$ (7.9)

Now we consider II₅. On one hand, if $\sqrt{n/(slogn)}{inf}_{\tau \in \mathrm{\Delta },j\le s}\mid {\beta }_0^{(j)}\mid \to \infty$, then

$$n^{-1/2}{\lambda }_n\sum _{j=2}^s{\omega }_j(\tau )\left(\left|{\beta }_0^{(j)}(\tau )+{\delta }^{(j)}|-|{\beta }_0^{(j)}(\tau )\right|\right)=n^{-1/2}{\lambda }_n{[\mathit{\omega }(\tau )\circ sgn({\mathit{\beta }}_0(\tau ))]}^T\mathit{\delta },$$ (7.10)

Combining (7.6), (7.7), (7.8), (7.9) and (7.10) together, we have

$$n^{-1/2}[Q_n({\mathit{\beta }}_0(\tau )+\mathit{\delta };\tau )-Q_n({\mathit{\beta }}_0(\tau );\tau )]=n^{1/2}{\mathit{\delta }}^T{H}_{\tau }\mathit{\delta }-{\mathit{\delta }}^T{M}_n(\tau ,\mathbf{0})+n^{-1/2}{\lambda }_n{[\mathit{\omega }(\tau )\circ sgn({\mathit{\beta }}_0(\tau ))]}^T\mathit{\delta }+o_p(\Vert {\mathit{\delta }}_a\Vert ).$$

Let δ̂ be the minimizer of L(δ; τ) := n^1/2δ^TH_τδ−δ^TM_n,τ(0)+n^−1/2λ_n[ω(τ)∘sgn(β₀(τ))]^Tδ+o_p (||δ_a||) over R_s(B). Then we have

$$2n^{1/2}{H}_{\tau }\left(\widehat{\mathit{\delta }}+\frac{{\lambda }_n}{2n}{H}_{\tau }^{-1}[\mathit{\omega }(\tau )\circ sgn({\mathit{\beta }}_0(\tau ))]\right)=M_n(\tau ,\mathbf{0})$$

Given any α ∈ ℝ^p, ||α|| = 1, applying Lemma 7.5 yields

$$2n^{1/2}{\mathit{\alpha }}^T{H}_{\tau }\left({\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )+\frac{{\lambda }_n}{2n}{H}_{\tau }^{-1}[\mathit{\omega }(\tau )\circ sgn({\mathit{\beta }}_0(\tau ))]\right)$$

converges weakly to a mean zero Gaussian process with covariance Σ(τ, τ′).

On other hand, if n^−1/2λ_n||ω_a|| = o_p(1), then

$${II}_5\le n^{-1/2}{\lambda }_n\Vert \mathit{\omega }(\tau )\Vert \Vert \mathit{\delta }\Vert =o_p(\Vert {\mathit{\delta }}_a\Vert ).$$ (7.11)

Consequently,

$$n^{-1/2}[Q_n({\mathit{\beta }}_0(\tau )+\mathit{\delta };\tau )-Q_n({\mathit{\beta }}_0(\tau );\tau )]=n^{1/2}{\mathit{\delta }}^T{H}_{\tau }\mathit{\delta }-{\mathit{\delta }}^T{M}_{n,\tau }(0)+o_p(\Vert {\mathit{\delta }}_a\Vert ).$$

With similar arguments, we can show that $2n^{1/2}{\mathit{\alpha }}^T{H}_{\tau }\left({\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )-{\mathit{\beta }}_0(\tau )\right)$ converges weakly to a mean zero Gaussian process with covariance Σ(τ, τ′). This completes the proof of Theorem 3.2.

Proof of Theorem 3.3

According to KKT conditions, to show that ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$ is a global minimizer of Q_n(β; τ) over ℝ^p, we only need to check the following condition

$$\underset{\tau \in \mathrm{\Delta }}{sup}\left\{\left|\sum _{i=1}^n{X}_i^{(j)}{\psi }_{\tau }\left(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}^o(\tau )\right)\right|-{\lambda }_n{\omega }_j(\tau )\right\}<0,\text{for}j>s.$$ (7.12)

By Theorem 3.1, we know that Pr (sup_τ∈Δ ||β̂^o(τ) − β₀(τ)|| ∈ R_s(B)) → 1. Therefore, if we can we can show that

$$\underset{s<j\le p,\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\left\{\left|\sum _{i=1}^n{X}_i^{(j)}{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\right|-{\lambda }_n{\omega }_j(\tau )\right\}<0,$$ (7.13)

then (7.12) follows immediately, and the probability that ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$ is also the global minimizer of (2.2) over ℝ^p is approaching 1.

We first state some results which will be used in the following proof. Under conditions stated in the theorem, ∀δ ∈ R_s(B), we have

$$\begin{array}{l}\mid E[X_i^{(j)}(1\{Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta })\le 0\}-1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )\le 0\})]\mid \le \overline{f}E\left[\left|X_i^{(j)}{\mathbf{X}}_i^T\delta \right|\right]\le \overline{f}{\left(E\left[{\left(X_i^{(j)}{\mathbf{X}}_i^T\mathit{\delta }\right)}^2\right]\right)}^{1/2}=o\left({\left(\frac{logp}{s{log}^{2}n}\right)}^{1/2}\Vert \mathit{\delta }\Vert \right)\\ =o\left(\sqrt{logp/n}\right),\end{array}$$ (7.14)

where the first inequality follows from the condition expectation on X_i and condition (C1), and the second inequality follows from Cauchy-Schwartz inequality. Moreover, we have

$$\begin{array}{l}\text{Var}\left[X_i^{(j)}(1\{Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta })\le 0\}-1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )\le 0\})\right]\le E\left[{\left(X_i^{(j)}\right)}^2{(1\{Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta })\le 0\}-1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )\le 0\})}^2\right]\\ \le \overline{f}E\left[{\left(X_i^{(j)}\right)}^2\mid {\mathbf{X}}_i^T\mathit{\delta }\mid \right]\le \overline{f}{\left(E\left[{\left(X_i^{(j)}\right)}^2\right]\right)}^{1/2}{\left(E\left[{\left(X_i^{(j)}{\mathbf{X}}_i^T\mathit{\delta }\right)}^2\right]\right)}^{1/2}\\ =o\left(\sqrt{logp/n}\right).\end{array}$$ (7.15)

In the following, we restrict our attention on Ω₀. Let α_j denote the vector where the jth component is 1 and all other components are 0. Then ${\sum }_{i=1}^n{X}_i^{(j)}{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))=n^{1/2}{\mathit{\alpha }}_j{M}_n(\tau ,\mathit{\delta })$.

$$\begin{array}{l}{n}^{1/2}{\mathit{\alpha }}_j{M}_n(\tau ,\mathit{\delta })=n^{1/2}{\mathit{\alpha }}_j\{M_n(\tau ,\mathit{\delta })-E[M_n(\tau ,\mathit{\delta })]-M_n(\tau ,\mathbf{0})+E[M_n(\tau ,\mathbf{0})]\}+n^{1/2}{\mathit{\alpha }}_j\{E[M_n(\tau ,\mathit{\delta })]-E[M_n(\tau ,\mathbf{0})]\}+n^{1/2}{\mathit{\alpha }}_j{M}_{n,\tau }(\mathbf{0})\\ :={\mathit{III}}_1+{\mathit{III}}_2+{\mathit{III}}_3\end{array}$$

First, we evaluate III₂. By (7.14), we have

$$\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\mid {\mathit{III}}_2\mid \le \sqrt{nlogp}.$$ (7.16)

Next, we consider III₃.

$$n^{1/2}{\mathit{\alpha }}_j{M}_n(\tau ,\mathbf{0})=\sum _{i=1}^n{X}_i^{(j)}(\tau -1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\}).$$

Applying Lemma 2.3.7 in Van Der Vaart and Wellner (2000) yields

$$\begin{array}{l}Pr\left(n^{-1/2}\underset{\tau \in \mathrm{\Delta }}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}(\tau -1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\})\right|>M\right)\\ \le \frac{2Pr\left(n^{-1/2}{sup}_{\tau \in \mathrm{\Delta }}\left|{\sum }_{i=1}^n{X}_i^{(j)}{V}_i(\tau -1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\})\right|>M/4\right)}{1-1/M^2},\text{for}M>1.\end{array}$$

We have

$$\begin{array}{l}Pr\left(\tau \left|\sum _{i=1}^n{X}_i^{(j)}{V}_i\right|>\sqrt{6nlogp}\right)\le Pr\left(\left|\sum _{i=1}^n{X}_i^{(j)}{V}_i\right|>\sqrt{6nlogp}\right)\le Pr\left(\left|\sum _{i=1}^n{X}_i^{(j)}{V}_i\right|>\sqrt{6nlogp}\mid {\mathrm{\Omega }}_0\right)+Pr({\mathrm{\Omega }}_0^c)\\ \le 2exp\left(-\frac{1}{2}\frac{6nlogp}{{\sum }_{i=1}^n{\left(X_i^{(j)}\right)}^2}\mid {\mathrm{\Omega }}_0\right)+{\gamma }_{0n}\le \frac{2}{p^2}+{\gamma }_{0n},\end{array}$$ (7.17)

where the first two inequalities are elementary, the third inequality follows from Lemma 1.5 in Ledoux and Talagrand (1991), and the last inequality follows from condition (C2).

Conditional on (X_i, Y_i), i = 1, ···, n, we can partition Δ with a grid {τ₀, τ₁, τ₂, ···, τ_n} such that for each 1 ≤ i ≤ n, there is only one observation (X_(i), Y_(i)) satisfies ${\mathbf{X}}_{(i)}^T{\mathit{\beta }}_0({\tau }_{i-1})<Y_{(i)}\le {\mathbf{X}}_{(i)}^T{\mathit{\beta }}_0({\tau }_i)$. Here, τ₀ = 0. Then we have

$$n^{-1/2}\underset{\tau \in \mathrm{\Delta }}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{V}_{i}1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\}\right|=n^{-1/2}\underset{\tau \in ({\tau }_{i-1},{\tau }_i),1\le i\le n}{sup}\left|\sum _{i=1}^n{X}_{(i)}^{(j)}{V}_{(i)}1\{Y_{(i)}-{\mathbf{X}}_{(i)}^T{\mathit{\beta }}_0(\tau )<0\}\right|$$

As τ move from τ₀ to τ_n, we observe

$$\underset{\tau \in ({\tau }_{i-1},{\tau }_i),1\le i\le n}{sup}\left|\sum _{i=1}^n{X}_{(i)}^{(j)}{V}_{(i)}1\{Y_{(i)}-{\mathbf{X}}_{(i)}^T{\mathit{\beta }}_0(\tau )<0\}\right|=\underset{1\le k\le n}{max}\mid S_k\mid ,$$

where $S_k={\sum }_{i=1}^k{X}_{(i)}^{(j)}{V}_{(i)}$. Since V_(i)’s are independent and $X_{(i)}^{(j)}{V}_{(i)}1\{Y_{(i)}-{\mathbf{X}}_{(i)}^T{\mathit{\beta }}_0(\tau )<0\}$’s are symmetric about 0, According to Levy’s inequality, we have P(max_1≤k≤n |S_k| ≥ u) ≤ 2P(|S_n| ≥ u). Therefore, we have

$$\begin{array}{l}Pr\left(n^{-1/2}\underset{\tau \in \mathrm{\Delta }}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{V}_{i}1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\}\right|>M/4\right)=\int Pr\left(n^{-1/2}\underset{\tau \in \mathrm{\Delta }}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{V}_{i}1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\}\right|>M/4\mid {\mathbf{X}}_i,Y_i\right)dP\\ =\int Pr\left(\underset{1\le k\le n}{max}\mid S_k\mid >\sqrt{n}M/4\mid {\mathbf{X}}_i,Y_i\right)dP\le \int 2Pr(\mid S_n\mid >\sqrt{n}M/4\mid {\mathbf{X}}_i,Y_i)dP\\ =2Pr(\mid S_n\mid >\sqrt{n}M/4).\end{array}$$

Choose $M=\sqrt{96logp}$, we obtain

$$Pr\left(\underset{\tau \in \mathrm{\Delta }}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{V}_{i}1\{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_0(\tau )<0\}\right|>\sqrt{96nlogp}\right)\le \frac{4}{p^2}+{\gamma }_{0n},$$

where the inequality follows the arguments in (7.17). The above inequality and (7.17) together imply

$$Pr\left(\underset{\tau \in \mathrm{\Delta }}{sup}{\mathit{III}}_3>20\sqrt{6nlogp}\right)\le \frac{12}{p^2}+{\gamma }_{0n}.$$ (7.18)

Now we consider III₁. By Lemma 7.11, we can show

$$Pr\left(\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\mid {\mathit{III}}_1\mid >32C_3\sqrt{nlogp}\mid {\mathrm{\Omega }}_2\right)\le 8exp\left[-\frac{1}{2}{C}_{3}slogp+4slogn\right].$$ (7.19)

let $\stackrel{\sim }{K}=160\sqrt{nlogp}$. Then (7.16), (7.18), and (7.19) together yield

$$\begin{array}{l}Pr\left(\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\right|>\stackrel{\sim }{K}\right)\\ \le Pr\left(\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\right|>\stackrel{\sim }{K}|{\mathrm{\Omega }}_0\cap {\mathrm{\Omega }}_2\right)+{\gamma }_{0n}+{\gamma }_{2n}\\ \le \frac{20}{p^2}+{\gamma }_{0n}+{\gamma }_{2n}\end{array}$$

which implies

$$Pr\left(\underset{j>s}{max}\underset{\tau \in \mathrm{\Delta },\mathit{\delta }\in R_s(B)}{sup}\left|\sum _{i=1}^n{X}_i^{(j)}{\psi }_{\tau }(Y_i-{\mathbf{X}}_i^T({\mathit{\beta }}_0(\tau )+\mathit{\delta }))\right|>\stackrel{\sim }{K}\right)\le \frac{20}{p}+{\gamma }_{0n}+{\gamma }_{2n}.$$ (7.20)

${({inf}_{j>s,\tau \in \mathrm{\Delta }}{\omega }_j(\tau ))}^{-1}\sqrt{n}/\sqrt{slogp}=O_p(1)$, coupled with ${\lambda }_n/(\sqrt{s}logp)\to \infty$implies that, with probability approaching 1, λ_nω_j(τ) > K̃ for all j > s. This, (7.20), and condition (C5) together infer that (7.13) holds with probability tending to 1, and so does (7.12). Therefore, ${\widehat{\mathit{\beta }}}_{{\lambda }_n}^o(\tau )$ is a global minimizer of (2.2) with probability approaching 1. This completes the proof of Theorem 3.3.

Proof of Theorem 3.4

Theorem 3.4 is a direct implication of Lemmas 7.8 and 7.9.