Shrinkage Estimation of Varying Covariate Effects Based On Quantile Regression

Abstract

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l₁-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.

1 Introduction

In regression analysis, there has been growing awareness that the association under study may be prone to considerable heterogeneity. Inhomogeneous associations, embodied by varying covariate effects, are often scientifically plausible and entail important practical implications. There have been numerous reports in literature, for example, Kaslow et al (1987), Dickson et al (1989), Thorogood et al (1990), Verweij and Van Houwelingen (1995), Carey et al (1995), and Jensen et al (1997). As a more recent example, an analysis presented in Peng and Huang (2008) suggested that the severity of restless leg syndrome (RLS) symptoms may have prognostic power only in dialysis patients with high risk of mortality but not in long-term survivors. Such a varying association pattern sheds useful insight on dialysis prognosis but is precluded by traditional linear regression with the constraint of constant coefficients.

Quantile regression Koenker and Bassett (1978), by its modeling strategy, offers an excellent platform to accommodate and identify inhomogeneous associations. For example, a linear quantile regression model may take the form,

$$Q_Y(\tau \mid \mathit{Z})={\mathit{Z}}^{\mathrm{T}}{\mathit{\beta }}_0(\tau ),\tau \in ({\tau }_L,{\tau }_U),$$ (1)

where Y is a response variable, Z ≡ (1, Z̰^T)^T is a p × 1 vector of covariates, β₀(τ) ≡ (α₀(τ), β̰₀(τ)^T)^T is a p × 1 vector of unknown coefficients, Q_Y (τ∣Z) ≡ inf {y : pr(Y ≤ y∣Z) ≥ τ} denotes the τ–th conditional quantile of Y given Z, and (τ_L, τ_U) specifies the quantile level range of interest with 0 < τ_L < τ_U < 1. Here α₀(τ) and β̰₀(τ) represent intercept and covariate effects on the τ–th quantile of Y respectively, which are allowed to change over τ . As commented by Portnoy (2003), nonconstant coefficients in β̰₀(τ) across τ reflect any sources of heterogeneity affecting Y but not captured by the covariates in Z̰. Throughout this paper, we shall refer model (1) to as a “global” linear quantile regression model, reflecting the fact the linear quantile assumption is imposed over a τ-continuum, in contrast to a “local” model that only asserts quantile linearity for a single or a finite number of specified τ’s. It is worth emphasizing that model (1) is a strict extension of the traditional linear model, Y = Z^T β + ε, with or without distributional assumptions on the error term ε.

In practice, a large number of covariates are often collected at initial study stage. An appropriate variable selection procedure is thus crucial for building a model that excels in both interpretation and prediction accuracy. The consideration of varying covariate effects poses some conceptual subtlety for variable selection. Take model (1) for instance. Some covariates may exhibit a strong effect throughout the whole τ-range, namely a full effect, while some may impact part of but not all quantiles of interest and thus demonstrate a partial effect. In the dialysis example, RLS shows strong effects on lower quantiles of survival time but its effects seem to vanish for upper quantiles. It is scientifically important to retain such type of covariates in the final model because they can contribute a better understanding of the underlying response-covariate association and facilitate more profound predictions. This need calls for a new perspective for variable selection, which is to identify covariates with full effects as well as those with partial effects. Numerical studies presented in Section 3 indicate that existing variable selection methods that assume constant effects are incapable of fulfilling this need.

In this work, we adopt the varying coefficient quantile regression model (1) as the vehicle of tackling the variable selection problem that concerns potentially inhomogeneous associations. Penalization methods, as a special case of general shrinkage methods, have been actively studied for “local” quantile regression models. For example, Koenker (2005) gave a summary of the use of LASSO penalty (Tibshirani, 1996) and other bridge penalties (Frank and Friedman, 1993) in the context of quantile regression. Li and Zhu (2005) developed the solution path of the L₁ penalty quantile regression. Zou and Yuan (2008) proposed a regression method, called composite quantile regression, which was shown to have some advantage in estimation efficiency over least squares methods. Wu and Liu (2009) presented a detailed study on the adaptation of the SCAD (Fan and Li, 2001) and adaptive-LASSO (Zou, 2006) penalties to quantile regression. Rocha et al (2009) extended the work of Knight and Fu (2000) to more general l₁-penalized M-estimation settings and provided elegant theoretical results that can be applied to penalized quantile regression with fixed τ. However, all these approaches only concern covariate effects at a single or multiple specified quantile levels. Therefore, it is easy to perceive that they can miss important variables at a great chance when the quantile level targeted by the “local” quantile regression model is not in the τ-range where some covariates have influences. For example, a covariate that only influences the response from its 20th to 40th quantiles can be missed by the variable selection based on a median regression. An intuitive remedy for this problem may be to fit penalized “local” quantile regression at all possible τ’s of interest and remove variables that have shrunken coefficients at all τ’s considered. However, we shall show that this approach, which we call pointwise penalization approach hereafter, does not produce consistent variable selection.

Shrinkage estimation has been studied for other varyingcoefficient models. For example, Wang and Xia (2009) considered a linear model with coefficients formulated as functions of a univariate index variable. More recently, Kai et al (2011) extended the idea of composite quantile regression to estimate a similar type of varying coefficient model which contains both nonparametric coefficients and parametric coefficients. Both of these models permit extra data dynamics in contrast to a standard linear model by allowing coefficients to vary smoothly over the group stratified by the index variable. Of note, they require estimation based on kernel smoothing. As a result, the corresponding penalized estimators involve two levels of regularization; one for kernel bandwidth and one for penalization parameter. This may incur complexity in computation and theory.

The quantile regression model (1) considered in this work offers an alternative for accommodating varying associations. It offers a straightforward interpretation of the coefficient functions in β₀(τ), and has gained increased popularity recently, for example, in survival analysis (Portnoy, 2003; Neocleous et al, 2006; Peng and Huang, 2008; Portnoy and Lin, 2010; Huang, 2010). The fact that the estimation of model (1) does not require smoothing is expected to alleviate the computational complexity of the shrinkage estimation of β₀(·). The technical challenges now largely come from the need to deal with nonparametric coefficient functions and the non-differentiability of the loss functions tied to the estimation of β₀(τ).

Belloni and Chernozhukov (2011) studied a l₁-penalized estimator for the “global” quantile regression model in high-dimensional sparse case, allowing for τ-varying coefficients. They established the elegant near-oracle convergence rate (uniformly in τ) and characterized conditions under which the selected model contains the true model as the submodel with finite sample data. One critical condition required for their model selection properties is that the absolute value of any non-zero coefficient function must be bounded below by a positive separation bound uniformly in τ. While the separation bound can go to zero as n increases, it is not clear how their finite sample result on model selection can be extended to imply consistent model selection in the large sample sense without precluding the presence of covariates with partial effects (i.e. their coefficients are zero for some but not all τ’s), an important practical scenario like in the dialysis example discussed earlier.

In this paper, we develop a shrinkage estimation procedure based on the quantile regression model (1), which provides a useful variable selection tool that can flexibly accommodate inhomogeneous associations. Specifically, we follow the idea of adaptive LASSO (Zou, 2006) and propose a new penalization strategy to capture covariate effects’ “global” departure from null. We show that such a strategy can lead to a shrinkage estimator of β₀(·) that enjoys the following oracle properties (Fan and Li, 2001): (i) the probability of correctly identifying all zero and nonzero coefficient functions approaches 1, which means consistent variable selection; (ii) the proposed estimator converges weakly to a Gaussian process in the same manner as if the set of irrelevant covariates was known, which entails efficient estimation. Note that, this is a stronger result than the oracle properties attained under a “local” quantile regression model as it asserts the uniform behavior of the coefficient functions. We do not require the full separation of non-zero coefficient functions from zero across τ and thus the new method is capable of detecting covariates that have effects on a partial range of quantile indices. Furthermore, we present an adaptation of the new shrinkage approach for randomly right censored data. This extension greatly enhances the applicability of this work in biomedical study settings, where censoring often arises. Findings from our numerical studies provide evidences for the utility of the proposed methods as well as the importance of accounting for the presence of varying covariate effects in variable selection. In this paper, we only concern the situation where p is fixed. The investigation of the case with p ≫ n will be reported in separate work.

2 The Proposed Methods

2.1 Uniform adaptive-LASSO estimation

Suppose the observed data include n independent and identically distributed replicates of (Y, Z), denoted by ${\left\{(Y_i,{\mathit{Z}}_i)\right\}}_{i=1}^n$. We use u^(j) to denote the jth component of the vector u, and adopt a shorthand notation Δ for the quantile level range of interest, (τ_L, τ_U). Without loss of generality, we assume that ${\sup }_{\tau \in \Delta }|{\beta }_0^{(j)}(\tau )|\ne 0$ for 2 ≤ j ≤ s and ${\sup }_{\tau \in \Delta }|{\beta }_0^{(j)}(\tau )|=0$ 0 for s+1 ≤ j ≤ p. Based on model (1), this means the first s − 1 covariates in Z̰ have some effects on Y while the rest do not.

It is worth noting that, here we adopt a global sparsity assumption, $|\{2\le j\le p:{\sup }_{\tau \in \Delta }|{\beta }_0^{(j)}(\tau )|>0\}|=s-1<\infty$. This represents a distinction of this work from penalized local quantile regression, which imposes sparsity only on β₀(τ) with a given τ (i.e. local sparsity). It is clear that the global sparsity would imply the local sparsity but not vice versa.

We begin with an unpenalized estimator of β₀(τ) (Koenker and Bassett, 1978), defined as β̃_n (τ) = arg min_βW_n(β; τ), where

$$W_n(\mathit{\beta };\tau )\equiv \sum _{i=1}^n\rho \left(Y_i-{\mathit{Z}}_i^{\mathrm{T}}\mathit{\beta };\tau \right),$$

and ρ(u; τ) = u{τ − I(u < 0)} is the so-called check function. Despite lack of a close form, β̃_n(τ) can be efficiently computed, even in the presence of a large number of covariates, by various algorithms, for example, Koenker and d’Orey (1987), Lustig et al (1994), and Madsen and Nielsen (1993).

To proceed the shrinkage estimation of β₀(τ), we adopt the adaptive LASSO idea from Zou (2006). In a general form, we present the penalized estimator as

$${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}(\tau )=\arg \underset{\mathit{\beta }}{\min }{W}_{n,{\mathrm{\lambda }}_n}(\mathit{\beta };\tau ),$$

where

$$W_{n,\mathrm{\lambda }}(\mathit{\beta };\tau )=W_n(\mathit{\beta };\tau )+\mathrm{\lambda }\sum _{j=2}^p{w}_{n,j}(\tau )·|{\beta }^{(j)}|,$$

and w_n,j(τ) stands for the weight assigned to |β^(j)|. Note that when w_n,j(τ) takes the form of $|{\stackrel{\sim }{\beta }}_n^{(j)}(\tau ){|}^{-\gamma }$ with γ > 0, the above estimator becomes the adaptive-LASSO estimator studied by Wu and Liu (2009) for a “local” linear quantile regression model assumed for a fixed τ . The naive pointwise penalization approach discussed in Section 1 is to compute this type of estimator with γ = 1 separately for each τ ∈ Δ to form a function estimator of β₀(·). As shown by our simulation studies, this approach may yield problematic variable selection results.

The proposed shrinkage estimator is ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ with $w_{n,j}(\tau )=w_{n,j}^{\mathrm{US}}$, where

$$w_{n,j}^{\mathrm{US}}=1/(\underset{\tau \in \Delta }{\sup }|{\stackrel{\sim }{\beta }}_n^{(j)}(\tau )|).$$

The key idea is about the new penalization strategy, by which we assign a high penalty to covariates, effects of which consistently exhibit small departure from zero for all τ, but not to those that have small effects at some τ’s but have large effects on the other τ’s. Using the uniform adaptive LASSO penalty $w_{n,j}^{\mathrm{US}}$ in W_{n, λ}(β; τ) is expected to effectively differentiate zero coefficient function of τ versus those which are nonzero for some τ ∈ Δ. As elucidated in Section 2.2, uniform shrinkage behaviors across τ can be achieved by the proposed uniform adaptive LASSO estimator, ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$.

To compute the proposed penalized estimator, ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$, one can adopt the existing routines for quantile regression, which have been well framed under linear programming (Koenker, 2005; Li and Zhu, 2005; Wu and Liu, 2009). More specifically, one may use the rq() function in R package quantreg (Koenker, 2011) to obtain β̃_n(·), the unpenalized estimator of β₀(τ) for τ ∈ (0, 1). Given that β̃_n(·) is piecewise constant, applying the rq() function would yield a finite number of function values of β̃_n(τ) along with the corresponding τ–jumping points, say 0 ≡ τ₀ < τ₁ < … < τ_R < τ_R+1 = 1. Based on these results, one can readily evaluate ${\sup }_{\tau \in \Delta }|{\stackrel{\sim }{\beta }}_n^{(j)}(\tau )|$ and thus $w_{n,j}^{\mathrm{US}}$. Next, for any given τ ∈ (0, 1), one may compute ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ by using the rq.fit.lasso() function in R package quantreg with the penalty parameter specified as ${\mathrm{\lambda }}_n·w_{n,j}^{\mathrm{US}}$.

It is worth pointing out that in theory, ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(·)$ obtained from a finite size dataset is also piecewise constant. However, it is not straightforward to locate the τ–points where ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(·)$ jumps. To circumvent this complication, one can approximate ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(·)$ by a cadlag step function which jumps at a sufficiently fine pre-specified grid in Δ. For example, in our numerical studies, we chose the grid points as (τ_j + τ_j+1)/2, j = 0, …, R. Our theoretical studies suggest that adopting a grid of size O(n^−1/2) would lead to only asymptotically negligible deviations from the exact ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(·)$.

2.2 Asymptotic results

To study the asymptotic properties of ${\widehat{\mathit{\beta }}}_{n,{\lambda }_n}^{\mathrm{US}}(·)$, we require the following regularity conditions:

• C1(i) Z is bounded; that is, sup_i ∥Z_i∥ < ∞; (ii) E(Z Z^T) > 0; (iii) β₀ (τ) is Lipschitz in τ for τ ∈ Δ.
• C2(i) inf_{τ ∈ Δ, z} f_τ (0∣z) > 0; (ii) For some A₀ > 0 and δ₁ > 0, sup_{τ ∈ Δ, z} |f_τ (x∣z) − f_τ (0∣z)| ≤ A₀ |x| for |x| < δ₁. Here f_τ (x∣z) denotes the density of Y − Z^T β₀(τ) given Z = z.

Both C1 and C2 impose rather mild data assumptions. Here, we assume bounded Z to simplify asymptotic arguments. This condition can be relaxed to E(Z²) < ∞, similar to the theoretical studies on accelerated failure time (AFT) model (Ying, 1993). The positive definite covariate matrix assumption stated in C1(ii) is critical for the identifiability of β₀(τ), and is common in regression analysis. Condition C2 requires the smoothness and positiveness of f_τ (x∣z) in a neighborhood of x = 0, which is often met in practice. Here and hereafter, we let ℓ₂n be shorthand for log log n, and ∥ · ∥ denote the Euclidean norm.

Theorem 1

Assume that (n^1/2ℓ₂n)⁻¹ λ_n = O(1). Under regularity conditions C1-C2, we have ${\sup }_{\tau \in \Delta }\Vert {\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )-{\mathit{\beta }}_0(\tau )\Vert =O_p(n^{-1/2}{\ell }_{2}n)$.

The uniform upper bound for, $\Vert {\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )-{\mathit{\beta }}_0(\tau )\Vert$, obtained in Theorem 1, implies the uniform consistency of β̂_n,λn (τ). This result also helps establish the oracle property of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ stated in the following theorem.

Theorem 2

Assume that lim_n→∞ (ℓ₂n)⁻² λ_n = ∞ and lim_n→∞ n^−1/2 λ_n = 0. Under regularity conditions C1-C2, we have

1. (selection consistency) ${\lim }_{n\to \infty }\mathit{pr}(\{2\le j\le p:{\sup }_{\tau \in \Delta }|{\widehat{\beta }}_{n,{\lambda }_n}^{\mathrm{US}(j)}(\tau )|=0\}=\{s+1,\dots ,p\})=1$;

2. (weak convergence) $n^{1/2}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1:s)}(\tau )-{\mathit{\beta }}_0^{(1:s)}(\tau )\}$ converges weakly to a mean zero Gaussian process with a covariance matrix Σ_oracle(τ, τ′). Here, the superscript ^(1:s) denotes the first s components of a vector, Σ_oracle (τ, τ′) is the same as the limit covariance matrix of n^1/2{β̂_oracle(τ)−β₀(τ)}, where β̂_oracle (τ) is the unpenalized estimator of ${\mathit{\beta }}_0^{(1:s)}(\tau )$ obtained as the minimizer of W_n(β; τ) with ${\beta }_0^{(j)}(\tau )(s+1\le j\le p)$ known to be zero functions.

The oracle property established in Theorem 2 ensures the uniform shrinkage behaviors of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ over τ ∈ Δ, which is needed in the context that allows for varying covariate effects. To prove both theorems, we utilize empirical process theory to examine the asymptotic behavior of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}$ as a process of τ. Delicate probabilistic arguments are also used to handle the non-differentiability of W_n,λn(β; τ). Proofs are sketched in Appendices A and B, with a more detailed version presented in Supplementary Materials (available upon request).

Theorem 2 indicates the asymptotic order of λ_n which can yield consistent selection and efficient estimation. With a real dataset with finite sample size, we propose a BIC-like criterion to select a good penalization parameter, λ_n. Specifically, we define the BIC criterion as a sum of two terms:

$$\mathrm{BIC}(\mathrm{\lambda })={\mathrm{\int }}_{\Delta }\log {\widehat{\sigma }}_{\mathrm{\lambda }}(\tau )\mathit{d\tau }+\frac{\log n}{n}|S_{\mathrm{\lambda }}|,$$

where $S_{\mathrm{\lambda }}=\{2\le j\le p:{\sup }_{\tau \in \Delta }|{\widehat{\beta }}_{n,\mathrm{\lambda }}^{\mathrm{US}(j)}(\tau )|\ne 0\}$, |S_λ| denotes the size of S_λ, and

$${\widehat{\sigma }}_{\mathrm{\lambda }}(\tau )=n^{-1}\sum _{i=1}^n\rho (Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\widehat{\mathit{\beta }}}_{n,\mathrm{\lambda }}^{\mathrm{US}}(\tau );\tau ).$$

Note that the first term in BIC(λ) quantifies the goodness of fit and the second term measures the model complexity. The use of log σ̂_λ in BIC was motivated Koenker (2005)’s model selection criteria for quantile regression (see page 135). A similar form of BIC was also used by other authors, for example, Wang and Leng (2007). We select the penalization parameter as λ̂_n = arg min_λ BIC(λ). Simulations reported in Section 3.1 demonstrate satisfactory performance of λ̂_n. We have also tested other methods such as AIC and generalized cross validation (GCV) for selecting λ_n. The simulation results presented in Supplementary Materials (available upon request) show that, compared to BIC based procedure, the AIC and GCV based procedures have similar performance in terms of the model error but appear to select the true model at substantially lower rates. It would be important to further investigate the justification for our proposal of selecting the penalty parameter based on BIC.

2.3 Inference

The oracle property stated in Theorem 2 facilitates the statistical inference for β₀(τ) based on ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$. In this subsection, we outline the procedures for variance estimation and construction of confidence intervals.

Let C_n = {1} ∪ S_λn and q_n = |C_n|. For a vector u and an index set Φ, we let u^(Φ) denote a subvector of u with Φ indicating the components of u included in u^(Φ). In the rest of this subsection, we use β̂_n(τ) as shorthand notation for ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$. Define ${\mathit{r}}_n(\tau )={\left(w_{n,1}(\tau )\mathrm{sgn}\{{\widehat{\mathit{\beta }}}^{(1)}(\tau )\},\dots ,w_{n,p}(\tau )\mathrm{sgn}\{{\widehat{\mathit{\beta }}}^{(p)}(\tau )\}\right)}^{\mathrm{T}}$, ${\widehat{\iota }}_j(\tau )=-{\mathit{Z}}_j^{({\mathit{C}}_n)}[\tau -I\{Y_j-{\mathit{Z}}_j^{\mathrm{T}}\widehat{\mathit{\beta }}(\tau )<0\}-{\mathrm{\lambda }}_n{\mathit{r}}_n^{({\mathit{C}}_n)}(\tau )]$, and ${\mathit{\Omega }}_n(\tau ,\tau )=n^{-1}{\sum }_{j=1}^n{\widehat{\iota }}_j{(\tau )}^{\otimes 2}.$

We propose the following steps to obtain variance estimates for regression quantiles that do not shrinkage for all τ ∈ Δ:

1. Find a symmetric and nonsingular q_n × q_n matrix E_n(τ) ≡ {e_n,1 (τ), …, e_n,qn (τ)} such that Ω_n(τ, τ) = {E_n(τ)}².

2. Calculate the q_n × q_n matrix,

   $${\mathit{D}}_n(\tau )=\left(\widehat{{\mathit{b}}_{e,1}}(\tau )-{\widehat{\mathit{\beta }}}^{({\mathit{C}}_n)}(\tau ),\dots ,\widehat{{\mathit{b}}_{e,q_n}}(\tau )-{\widehat{\mathit{\beta }}}^{({\mathit{C}}_n)}(\tau )\right),$$

   where

   $$\begin{array}{l}\widehat{{\mathit{b}}_{e,k}}(\tau )=\arg \underset{\mathit{b}\in R^{q_n}}{\min }\{\sum _{i=1}^n\rho \left(Y_i-{\mathit{Z}}_i^{{({\mathit{C}}_n)}^{\mathrm{T}}}\mathit{b};\tau \right)+\\ {\mathrm{\lambda }}_n\sum _{j=2}^{q_n}{w}_{n,}{\mathit{C}}_n(j)·|b^{(j)}|-n^{1/2}{\mathit{b}}^{\mathrm{T}}{\mathit{e}}_{n,k}(\tau )\}\end{array}$$

   with C_n(j) standing for the jth element in set C_n.

3. Estimate the variance matrix of $n^{1/2}\{{\widehat{\mathit{\beta }}}_n^{({\mathit{C}}_n)}(\tau )-{\mathit{\beta }}_0^{({\mathit{C}}_n)}(\tau )\}$ by n{D_n(τ)}^⊗2, where u^⊗2 = uu^T.

The technique used for variance estimation is an adaptation of Huang (2002) and Peng and Fine (2009)’s methods. Based on the proof of Theorem 2 (ii), specifically equation (17) in the Appendix B, and the uniform consistency of β̂(τ) from Theorem 1, we can readily apply the lines of Peng and Fine (2009) to show that n{D_n (τ)}^⊗2 is a consistent variance estimate for β̂^(1:s) (τ) when C_n = {1, …, s}. This, coupled with Theorem 2 (i), implies that for any ε > 0,

$$\begin{array}{l}\underset{n\to \infty }{\lim }\mathrm{pr}({\mathit{C}}_n=\{1,\dots ,s\}\mathrm{and}\\ \Vert n{\{{\mathit{D}}_n(\tau )\}}^{\otimes 2}-{\mathit{\sum }}_{\mathrm{oracle}}(\tau ,\tau )\Vert \le \epsilon )\\ \ge \underset{n\to \infty }{\lim }\{1-\mathrm{pr}({\mathit{C}}_n\ne \{1,\dots ,s\})-\mathrm{pr}(C_n=\{1,\dots ,s\}\\ \mathrm{and}\Vert n{\{{\mathit{D}}_n(\tau )\}}^{\otimes 2}-{\mathit{\sum }}_{\mathrm{oracle}}(\tau ,\tau )\Vert >\epsilon )\}=1.\end{array}$$

The above arguments provide a justification for the proposed variance estimation. Likewise, we propose to construct a 100(1 − α)% confidence interval for ${\mathit{\beta }}_0^{(j)}(\tau )$ as (β̂^(j) (τ) − z_α/2 σ̂_j (τ), β̂^(j) (τ) + z_α/2 σ̂_j (τ)) if j ∈ C_n and {0} otherwise, where ${\widehat{\sigma }}_j^2$ is the jth diagonal element of {D_n (τ)}^⊗2 and z_α is the 100(1 − α)th percentile of N(0, 1). Denote such a defined confidence interval as I_n. For 1 ≤ j ≤ s, given that pr(j ∉ C_n) = 0 and $n^{1/2}\{{\widehat{\beta }}^{(j)}(\tau )-{\beta }_0^{(j)}(\tau )\}$ is asymptotically normal with nσ̂_j (τ)² being a consistent variance estimate, we have

$$\begin{array}{l}\underset{n\to \infty }{\lim }\mathrm{pr}({\beta }_0^{(j)}(\tau )\in I_n)\\ =\underset{n\to \infty }{\lim }\mathrm{pr}(|{\beta }_0^{(j)}(\tau )-{\widehat{\beta }}^{(j)}(\tau )|<z_{\alpha /2}{\widehat{\sigma }}_j(\tau )\mathrm{and}j\in {\mathit{C}}_n)\\ =\underset{n\to \infty }{\lim }\mathrm{pr}(|{\beta }_0^{(j)}(\tau )-{\widehat{\beta }}^{(j)}(\tau )|<z_{\alpha /2}{\widehat{\sigma }}_j(\tau )|j\in {\mathit{C}}_n)=1-\alpha .\end{array}$$

This implies that the asymptotic confidence level of I_n is (1 − α).

2.4 An extension to randomly censored data

Censoring often occurs in biomedical study settings. In this subsection, we propose an extension of the shrinkage method proposed in Section 2.1 to accommodate scenarios with random censoring. Specifically, let C denote the censoring variable to the response Y. We assume C is independent of (Y, Z). Note, this independent assumption is stronger than the standard random censoring assumption (i.e. C is independent of Y given Z). Define X = Y ∧ C and δ = I(Y ≤ C), where ∧ is the minimum operator. Observables include n independent and identically distributed replicates of (X, δ, Z), denoted by ${\left\{\left(X_i,{\delta }_i,{\mathit{Z}}_i\right)\right\}}_{i=1}^n$.

For unpenalized estimation of model (1) in the presence of random censoring by C, we consider an adaptation of Ma and Yin (2010)’s estimator, which can be obtained as

$${\stackrel{\sim }{\mathit{\beta }}}_{\mathrm{C},n}(\tau )=\arg \underset{\mathit{\beta }}{\min }{V}_n(\mathit{\beta };\tau ),$$

where

$$V_n(\mathit{\beta };\tau )=\sum _{i=1}^n\frac{{\delta }_i}{\widehat{G}(X_i)}\rho (X_i-{\mathit{Z}}_i^{\mathrm{T}}\mathit{\beta };\tau ).$$

Here Ĝ(·) is the Kaplan-Meier estimator of G(x) = pr(C ≥ x). One can use the available software, for example, the R package quantreg, to compute β̃_C,n(τ) as weighted regression quantiles.

Using the same reasoning as that in Section 2.1, we propose the shrinkage estimator of β₀(τ) as β̂_C,n,λC,n (τ) = arg min_β V_n,λC,n (β; τ), where

$$V_{n,\mathrm{\lambda }}(\mathit{\beta };\tau )=V_n(\mathit{\beta };\tau )+\mathrm{\lambda }\sum _{j=2}^p{w}_{\mathrm{C},n,j}(\tau )|{\beta }^{(j)}|$$

with $w_{\mathrm{C},n,j}(\tau )={({\sup }_{\tau \in \Delta }|{\stackrel{\sim }{\beta }}_{\mathrm{C},n}^{(j)}(\tau )|)}^{-1}$.

By the analogy between V_n(β; τ) and a standard weighted quantile regression problem, the implementation of β̂_C,n,λC,n(τ) is expected to be very similar to our proposal for the uncensored shrinkage estimator ${\mathit{\beta }}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$. So is the asymptotic studies for β̂_C,n,λC,n(τ). With some additional regularity condition on censoring, for example,

• C3There exists ν > 0 such that pr(C = ν) > 0 and pr(C > ν) = 0;

we would have the uniform consistency of Ĝ(t) for t < ν, and thus can obtain the oracle property of β̂_C,n,λC,n(τ) as in the uncensored case. Note that condition C3 is a rather mild assumption and is often satisfied in practice in the presence of administrative censoring. As pointed out by Peng and Fine (2009), artificial truncation may be imposed to ensure this condition always met. The detailed results are stated in the theorem below with the proof omitted.

Theorem 3

Assume that lim_n→∞(ℓ₂n)⁻²λ_C,n = ∞, and lim_n→∞ n^−1/2 λ_C,n = 0. Under regularity conditions C1-C2 and some mild assumption on censoring as those in Ma and Yin (2010), we have

1. (selection consistency) ${\lim }_{n\to \infty }\mathit{pr}(\{2\le j\le p:{\sup }_{\tau \in \Delta }|{\widehat{\beta }}_{\mathrm{C},n,{\mathrm{\lambda }}_{\mathrm{C},n}}^{(j)}(\tau )|=0\}=\{s+1,\dots ,p\})=1$;

2. (weak convergence) $n^{1/2}\{{\widehat{\mathit{\beta }}}_{\mathrm{C},n,{\mathrm{\lambda }}_{\mathrm{C},n}}^{(1:s)}(\tau )-{\mathit{\beta }}_0^{(1:s)}(\tau )\}$ converges weakly to a mean zero Gaussian process with a covariance matrix Σ_C,oracle(τ, τ′). Here, Σ_C,oracle(τ, τ′) is the same as the limit covariance matrix of $n^{1/2}\{{\widehat{\mathit{\beta }}}_{\mathrm{C},\mathrm{oracle}}(\tau )-{\mathit{\beta }}_0^{(1:s)}(\tau )\}$ where β̂_C,oracle (τ) is the unpenalized estimator of ${\mathit{\beta }}_0^{(1:s)}(\tau )$ obtained as the minimizer of V_n(β; τ) with ${\beta }_0^{(j)}(\tau )(s+1\le j\le p)$ known to be zero functions.

Similarly, we propose to select λ_n based on a BIC-like criterion:

$$\stackrel{\sim }{\mathrm{BIC}}(\mathrm{\lambda })={\int }_{\Delta }\log {\widehat{\sigma }}_{\mathrm{C},\mathrm{\lambda }}(\tau )\mathit{d\tau }+\frac{\log n}{n}|S_{\mathrm{\lambda }}|,$$

where

$${\widehat{\sigma }}_{\mathrm{C},\mathrm{\lambda }}(\tau )=\frac{{\sum }_{i=1}^n{\delta }_i\rho (X_i-{\mathit{Z}}_i^{\mathrm{T}}{\widehat{\mathit{\beta }}}_{\mathrm{C},n,\mathrm{\lambda }}(\tau );\tau )/\widehat{G}(X_i)}{{\sum }_{i=1}^n{\delta }_i/\widehat{G}(X_i)}$$

Note that σ̂_C,λ, like σ̂_λ, serves as an approximation to $n^{-1}{\sum }_{i=1}^n\rho \left(Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0\left(\tau \right);\tau \right)$ with censoring to Y_i properly handled. For a given dataset, we choose λ_C,n as ${\widehat{\mathrm{\lambda }}}_{\mathrm{C},n}=\arg {\min }_{\mathrm{\lambda }}\stackrel{\sim }{\mathrm{BIC}}(\mathrm{\lambda })$. Inference, such as variance estimation and confidence intervals, may mimic the procedures proposed for the uncensored case.

3 Numerical Studies

3.1 Monte-Carlo simulations

Extensive simulations were conducted to evaluate the finite-sample performance of the proposed shrinkage estimators in both uncensored and censored cases.

We first considered situations where there is no censoring to the response Y. We generated Y based on model (1) with eight covariates, Z⁽¹⁾,…,Z⁽⁸⁾. Only Z⁽¹⁾, Z⁽²⁾ and Z⁽⁵⁾ have non-zero effects, and the coefficients for the other covariates are zero functions of τ. We considered three set-ups, denoted by (I), (II), and (III). For each of these set-ups, we plot the non-zero coefficients for Z⁽¹⁾, Z⁽²⁾, and Z⁽⁵⁾ in Figure 1 in bold lines, dashed lines and dotted lines respectively. The intercept coefficient function in set-up (I) is $\sqrt{2}$ times the quantile function of the standard normal distribution, and that in set-up (II) or (III) is set as a zero function in τ. It is easy to see from the specification of β₀(τ) that a standard linear model with normal random errors holds in set-up (I) but not in set-ups (II) and (III). In set-up (I), Z⁽¹⁾,…,Z⁽⁸⁾ were generated as standard normal variates with the correlation between Z⁽ⁱ⁾ and Z^(j) equal to 0.5^|i−j| (1 ≤ i, j ≤ 8), and were truncated to be between −3 and 3. In set-ups (II) and (III), Z⁽ⁱ⁾ = Φ(ξ_i) (i = 1,…, 8), where ξ₁,…, ξ₈ are standard normal variates with the correlation between ξ_i and ξ_j equal to 0.5^|i−j| (1 ≤ i, j ≤ 8). Additional simulations were also conducted for cases with a larger number of covariates (i.e. p = 40) and are reported in Supplementary Materials (available upon request).

Fig. 1.

True coefficient functions in simulation set-ups.

We assessed the proposed shrinkage estimation, denoted by US, and compare it with several other methods: (i) the naive pointwise penalization method described in Section 1 and 2.1, denoted by PS; (ii) the adaptive-LASSO estimator for “local” quantile regression at a single predetermined quantile level τ, denoted by SS(τ) with τ = 0.25, 0.50 or 0.75; (iii) adaptive-LASSO estimator for the least-square estimation of linear regression of Y on Z, denoted by LS. In the LS and SS(τ) approaches, it is implicitly assumed that the importance of a covariate can be adequately captured by its effect on the conditional mean or a conditional quantile at a known τ. We set τ_L = 0.1 and τ_U = 0.9. For each approach under comparison, the tuning parameter λ_n is selected as the minimizer of the corresponding BIC criterion among the 25(n/100) equally-spaced grid points between 0 and 12(n/100).

In Table 1, we present results based on 400 simulation replications for the proposed method and all the other methods described above. Specifically, we report mean number of correctly identified zero effects (NC), mean number of incorrectly identified zero effects (NIC), percentage of under-fitted models (PUF), percentage of correctly fitted models (PCF), and percentage of over-fitted models (POF). As a further clarification, NC counts the number of unselected covariates, which have zero coefficient functions (i.e. zero for all τ’s). In set-ups (I)–(III), we hope NC to be close to 5 as much as possible. We also evaluated the estimation accuracy of different estimators by examining their relative estimation errors (REE) with respect to the unpenalized estimator β̃(τ) or the oracle estimator β̂_oracle(τ). The REE of β(·) is defined as

$$\mathit{REE}(\mathit{\beta })=100\times \frac{{\mathit{\int }}_{\Delta }{\sum }_{i=1}^n{\sum }_{j=1}^p|{\beta }^{(j)}(\tau )-{\beta }_0^{(j)}(\tau )|\mathit{d\tau }}{{\mathit{\int }}_{\Delta }{\sum }_{i=1}^n{\sum }_{j=1}^p|{\overline{\beta }}^{(j)}(\tau )-{\beta }_0^{(j)}(\tau )|\mathit{d\tau }},$$

where β̄(τ) is either β̃(τ) or β̂_oracle(τ). The estimate from the SS(τ) or LS approach was taken as the constant value of estimated β₀ (τ) over τ. The median of REE with respect to the unpenalized estimator (MREE_F) and the median of REE with respect to the oracle estimator (MREE_O) are presented in Table 1.

Table 1.

Simulation results for uncensored data

Set-up	Method	n	NC	NIC	PUF (%)	PCF (%)	POF (%)	MREEF (%)	MREEO (%)
(I)	US	200	4.95	.00	.0	95.0	5.0	36.5	110.7
		400	5.00	.00	.0	99.5	.5	35.4	104.8
	PS	200	4.19	.00	.0	46.5	53.5	36.2	106.7
		400	4.49	.00	.0	59.8	40.2	34.6	103.4
	SS(0.25)	200	4.92	.00	.0	93.2	6.8	34.3	107.0
		400	4.96	.00	.0	95.8	4.2	32.1	99.1
	SS(0.50)	200	4.95	.00	.0	95.5	4.5	31.2	94.4
		400	4.97	.00	.0	97.5	2.5	29.4	91.9
	SS(0.75)	200	4.91	.00	.0	91.8	8.2	33.2	104.3
		400	4.97	.00	.0	97.8	2.2	31.7	100.7
	LS	200	4.90	.00	.0	90.8	9.2	24.3	73.8
		400	4.95	.00	.0	95.2	4.8	24.3	74.4
(II)	US	200	4.93	.00	.2	92.5	7.2	50.8	160.3
		400	4.99	.00	.0	99.2	.8	42.6	122.8
	PS	200	4.32	.00	.0	52.5	47.5	48.2	147.7
		400	4.76	.00	.0	78.8	21.2	44.7	129.4
	SS(0.25)	200	4.93	.19	13.8	80.5	5.8	115.4	364.9
		400	4.98	.00	.0	98.2	1.8	166.5	500.0
	SS(0.50)	200	4.99	.98	98.0	2.0	.0	84.7	267.6
		400	5.00	1.00	99.5	.5	.0	127.6	371.1
	SS(0.75)	200	4.96	.03	3.0	93.5	3.5	111.5	356.4
		400	5.00	.00	.2	99.8	.0	166.6	490.4
	LS	200	4.82	0.96	95.2	2.5	2.2	101.9	318.5
		400	4.93	0.98	98.2	1.8	0.0	142.1	423.2
(III)	US	200	4.94	0.01	1.0	93.8	5.2	33.5	105.1
		400	4.99	.00	.0	99.0	1.0	32.3	96.2
	PS	200	4.42	.00	.0	58.2	41.8	28.1	89.5
		400	4.81	.00	.0	83.2	16.8	27.1	82.5
	SS(0.25)	200	4.95	1.00	99.2	.8	.0	94.5	296.1
		400	4.98	.99	99.0	.8	.2	141.2	429.6
	SS(0.50)	200	4.98	1.94	100.0	.0	.0	78.5	242.8
		400	4.99	1.94	99.8	0.2	.0	113.9	341.5
	SS(0.75)	200	4.96	0.98	98.0	1.8	0.2	95.2	297.3
		400	4.98	1.00	99.8	.2	.0	143.9	428.3
	LS	200	4.78	1.05	63.2	24.5	12.2	90.1	277.0
		400	4.79	.36	24.0	60.5	15.5	133.1	394.6

From Table 1, we see that in set-up (I) where a linear model with normal random error holds, US, SS(0.25), SS(0.50), SS(0.75) and LS have quite similar performance in terms of model selection. The PCF’s are all above 90%. Among them, LS shows the best estimation accuracy. Nevertheless, this advantage over the other approaches is only moderate. Note that the oracle estimator adopted here is defined based on the quantile regression model (1). Hence it is reasonable to observe MREE_O below 0.8 for LS. Our finding on the superior estimation accuracy of LS estimator is consistent with the theory in Zou and Yuan (2008). It is important to note that the PS approach, though demonstrating good estimation accuracy as the other methods, shows a rather high tendency to produce an overfitted model, with POF=53.5% and 40.2% for n = 200 and 400 respectively. This strongly suggests that the intuitive PS approach may fail to generate consistent model selection.

In set-ups (II) and (III) where a standard linear model does not hold, LS has poor performance in identifying covariates that have effects on the response with PCF less than 65% when n = 400. The inconsistent shrinkage behavior also leads to inflated estimation errors. For example, MREE_F for the LS estimator even exceeds 1 when n = 400. As in set-up (I), PS remains to have the overfitting problem in these configurations. When covariates with partial effects are present, the penalized “local” quantile regression method SS(τ) can be troublesome. For example, in set-up (II), Z⁽¹⁾ impacts Y at all quantile levels except for τ = 0.5. When one simply applies the penalized median regression, Z⁽¹⁾ is identified as an unimportant covariate and its effects on other quantiles of Y are overlooked. This, reflected by NIC=1 and a low PCF of 2%, indicates a need to conduct a more thorough assessment of covariates’ impact across various quantiles. This message is more clear in set-up (III) where Z⁽¹⁾ and Z⁽²⁾ have partial effects on two disjoint quantile ranges. Finally, we note that in either set-up (II) or (III), US demonstrates satisfactory performance; PCF is above 99% when n = 400 and MREE_O decreases to approach 1 as n increases.

We also considered situations where Y is subject to random censoring. In the new set-ups labeled as (IV), (V), (VI), Y and Z were generated in the same way as in set-ups (I), (II), (III) respectively, except that the intercept coefficient function in set-up (IV) is set as the quantile function of the extreme value distribution. As a result, the relationship between Y and Z in set-up (IV) satisfies a linear model with extreme value error distribution and also a proportional hazards model. The censoring variable, C, was generate as Unif(−3, 14), Unif(−2, 19), and Unif(−2, 16) respectively in set-ups (IV), (V), and (VI) to produce 20% censoring. We set τ_L = 0.1 and τ_U = 0.8.

The simulation results for the censored cases are presented in Table 2 in the same manner as that for Table 1. The proposed method for censored data is now denoted by US_C, while the methods that we compared US_C with include PS_C, SS_C(0.25), SS_C(0.50), and SS_C(0.75), which are modified PS, SS(0.25), SS(0.5), and SS(0.75) that adjust for censoring. As a counter-part of LS in the censoring case, we considered adaptive LASSO for accelerated failure time (AFT) model, which was implemented by the method of least squares approximation (Wang and Leng, 2007) based on the log-rank estimator (Tsiatis, 1990; Wei and Gail, 1983). This approach is denoted by AAFT. In addition, we considered adaptive-LASSO estimator for the Cox proportional hazards model (Zhang and Lu, 2007), denoted by PH. From Table 2, we see that the empirical results support the oracle properties of β̂_C,n,λC,n (·). Though PCF’s may be lower than those observed in the uncensored case, which is not surprising given the information loss due to censoring, there is still a clear trend of PCF approaching 100 as n increases. The pointwise penalization approach remains to demonstrate a tendency of producing an overfitted model. Findings on the penalized local quantile regression approach are the same as those from Table 1. AAFT and PH perform well in set-up (IV) where data satisfy a linear model and a proportional hazards model. However, when the constant effect assumption is violated as in set-ups (V) and (VI), these methods can either overfit or underfit the model. By comparing PCF, MREE_F and MREE_O between US_C and AAFT, we see that assuming covariate effects constant when they are indeed varying can lead to not only misleading variable selection results but also substantially inflated estimation errors which entail very biased prediction of quantile profile of Y under model (1).

Table 2.

Simulation results for censored data

Set-up	Method	n	NC	NIC	PUF (%)	PCF (%)	POF (%)	MREEF (%)	MREEO (%)
(IV)	USC	200	4.77	.00	.0	83.0	17.0	38.3	113.2
		400	4.82	.00	.0	93.0	7.0	35.7	107.6
	PSC	200	2.62	.00	.0	9.5	90.5	42.6	121.9
		400	2.45	.00	.0	8.2	91.8	42.0	118.1
	SSC(0.25)	200	4.63	.00	.0	73.8	26.2	46.8	132.9
		400	4.62	.00	.0	76.5	23.5	42.8	127.4
	SSC(0.50)	200	4.76	.00	.0	83.2	16.8	31.7	97.0
		400	4.72	.00	.0	81.0	19.0	33.4	99.8
	SSC(0.75)	200	4.66	.00	.0	76.0	24.0	32.1	94.6
		400	4.7	.00	.0	80.8	19.2	31.7	93.5
	AAFT	200	4.75	.00	.0	77.8	22.2	26.4	78.1
		400	4.84	.00	.0	85.0	15.0	24.5	72.4
	PH	200	4.85	.00	.0	86.2	13.8	37.9	112.2
		400	4.91	.00	.0	91.0	9.0	33.6	99.4
(V)	USC	200	4.88	.00	.2	89.5	10.2	54.1	169.5
		400	4.97	.00	.0	97.5	2.5	44.1	128.5
	PSC	200	3.96	.00	.0	38.2	61.8	48.9	151.1
		400	4.48	.00	.0	59.0	41.0	46.1	136.9
	SSC(0.25)	200	4.93	.31	21.5	73.5	5.0	93.1	302.7
		400	4.98	.02	1.8	96.0	2.2	124.5	379.3
	SSC(0.50)	200	4.97	.95	94.5	5.2	.2	70.6	226.3
		400	5.00	.97	96.8	3.2	.0	98.9	300.2
	SSC(0.75)	200	4.78	.06	6.5	75.0	18.5	112.2	360.0
		400	4.94	.00	.2	94.0	5.8	158.1	477.5
	AAFT	200	4.78	0.56	55.5	33.8	10.8	100.9	320.4
		400	4.80	0.38	38.0	46.5	15.5	143.1	438.4
	PH	200	4.63	.26	25.8	51.5	22.8	–	–
		400	4.75	.02	2.5	75.8	21.8	–	–
(VI)	USC	200	4.86	.11	11.2	78.0	10.8	34.9	111.5
		400	4.98	.02	2.0	95.8	2.2	30.9	92.6
	PSC	200	3.64	.00	.2	24.5	75.2	32.3	101.5
		400	4.05	.00	.0	35.2	64.8	30.7	93.8
	SSC(0.25)	200	4.96	1.05	98.5	1.5	.0	68.6	221.1
		400	4.99	1.00	100.0	.0	.0	103.4	309.2
	SSC(0.50)	200	4.91	1.8	98.5	1.2	.2	63.5	200.9
		400	4.96	1.83	99.0	1.0	.0	93.6	277.3
	SSC(0.75)	200	4.69	.93	92.0	4.2	3.8	92.6	294.6
		400	4.86	.95	95.2	3.2	1.5	136.2	414.2
	AAFT	200	4.60	1.03	89.2	4.2	6.5	81.9	261.6
		400	4.75	0.92	87.0	7.8	5.2	120.0	349.5
	PH	200	4.82	.99	89.8	6.8	3.5	–	–
		400	4.87	.80	79.8	15.5	4.8	–	–

3.2 Applications to real datasets

Example 1

We considered the Air Pollution to Mortality Data (McDonald and Schwing, 1973), which are publicly available in the R package SMPracticals. This dataset contains information about a measure of total mortality (y) and 15 independent variables on 60 US metropolitan areas in 1959-1961. The 15 variables are precipitation (x₁), January temperature (x₂), July temperature(x₃), the percentage of age 60 or older(x₄), population per household(x₅), education(x₆), the percentage of soundhouse (x₇), population per square mile (x₈), the percentage of non-white (x₉), the percentage of white collar (x₁₀), the percentage of family income under 3000 dollars (x₁₁), HC potential (x₁₂), NOx potential (x₁₃), SO2 potential (x₁₄), and relative humidity (x₁₅). Of scientific interest is to predict y given x₁, …, x₁₅.

We applied the methods, including US, PS, SS(0.25), SS(0.5), SS(0.75) and LS, to the air pollution dataset. To evaluate different methods, we computed prediction errors (PE) as follows. First, we splitted the data randomly into a training set comprising 30 observations and a test set containing the rest 30 observations. We applied the method under evaluation to the training dataset and obtained a shrinkage estimator of β₀(τ), denoted by β̂^train(τ). Then the prediction error was calculated as

$$\mathit{PE}=\frac{{\sum }_{i=1}^{n}I(i\mathrm{in\; Test\; Set}){\mathit{\int }}_{\Delta }\rho (Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\widehat{\mathit{\beta }}}^{\mathrm{train}}(\tau );\tau )d\tau }{{\sum }_{i=1}^{n}I(i\mathrm{in\; Test\; Set})}.$$

In our analysis, all variables were standardized to have mean zero and unit variance. We set Δ = [0.1, 0.9]. In Table 3, we present the selected variables and average PE based on 10,000 replications of random splitting of training and test sets (APE).

Table 3.

Example 1: Analysis of the air pollution to mortality data

Method	Selected variables	APE
US	x1, x9, x10, x14	0.215
PS	x1, x2, x3, x5, x6, x7, x8, x9, x10, x12, x13, x14, x15	0.345
LS	x1, x2, x6, x9, x13	0.277
SS(0.25)	x1, x3, x9, x10, x13, x14	0.243
SS(0.5)	x1, x8, x9, x10, x12, x14	0.239
SS(0.75)	x6, x8, x9, x12, x14	0.255

From Table 3, we first note that PS may retain too much variables; a total of 13 variables are included in the final model. This result is consistent with the PS’s overfitting behavior observed in the simulation studies. The proposed method, US, may yield the most parsimonious model with only 4 variables, which is associated with the smallest APE. To better understand the discrepancy in selected variables among different approaches, we plot in Figure 2 the coefficient functions from fitting model (1) with all covariates along with the corresponding pointwise 95% confidence intervals. We also depict the least square coefficient estimates and the corresponding 95% confidence intervals by horizonal lines. It is observed from the regression quantile curves that the effects of x₁₀ and x₁₄ demonstrate apparent non-constant patterns with different signs at lower quantiles and upper quantiles, while x₁ and x₉ exhibit fairly consistent significant effects in the whole τ-range of interest. This finding may help explain why LS selected x₁ and x₉ but not x₁₀ and x₁₄. The variable selection results from penalized local quantile regression are closer to that from US compared to PS and LS. The APE’s from SS(0.25), S(0.50), S(0.75) are all slightly larger than the APE from US.

Fig. 2.

Air pollution example: Regression quantiles (bold dotted lines) and the corresponding pointwise 95% confidence intervals (shaded areas), and least square estimates (horizontal solid lines) and the corresponding pointwise 95% confidence intervals (horizonal dashed lines).

Example 2

We also illustrated the proposed method for randomly censored data to the dialysis example discussed in §1. A more detailed study background can be found in Kutner et al (2002) and Peng and Huang (2008). The dataset contains the information on the vital status of 191 incident dialysis patients up to December of 2005, and 14 covariates including severity of restless leg syndrome (x₁), patient’s age at study admission (x₂), patient’s BMI at study admission (x₃), baseline dialysis modality (x₄), baseline hematocrit (x₅), baseline mental health scale score (x₆), baseline ferritin concentration (x₇), baseline serum albumin (x₈), primary diagnosis of diabetes (x₉), indicator for cardiovascular comorbidity (x₁₀), indicator for high education (x₁₁), gender (x₁₂), indicator for fish eater (x₁₃), and race (x₁₄). One goal of this study is to investigate the risk factors for dialysis mortality, assessed by time from study admission to death, subject to censoring due to renal transplantation or end of the study. The censoring rate in this dataset is 35%. In our analysis, the response variable y is taken as the log transformed survival time and Δ is set as [0.1, 0.9]. We excluded ten subjects with missing covariates. The calculation of prediction error is slightly modified for the censored case. Specifically, the training set consists of 91 randomly selected observations, and

$$\mathit{PE}=\frac{{\sum }_{i=1}^{n}I(i\mathrm{in\; Test\; Set}){\mathit{\int }}_{\Delta }{\delta }_i\rho (X_i-{\mathit{Z}}_i^{\mathrm{T}}{\widehat{\mathit{\beta }}}^{\mathrm{train}}(\tau );\tau )\mathit{d\tau }/\widehat{G}(X_i)}{{\sum }_{i=1}^n{\delta }_{i}I(i\mathrm{in\; Test\; Set})/\widehat{G}(X_i)}$$

Here the Kaplan-Meier estimator, Ĝ(·), is computed based on the test dataset.

In Table 4, we summarize the results from the methods, US, PS, SS(0.25), SS(0.50), SS(0.75), AAFT, and PH. In Figure 3, we plot the regression quantiles and the estimates of AFT model coefficients when all covariates are included. It is seen from Table 4 that, as in example 1, PS tends to select more variables than the other methods, while US renders the smallest APE. With a particular interest in the impact of severity of restless leg syndrome on dialysis survival, we note that x₁ was selected by US and SS(0.25) but not SS(0.50), SS(0.75), AAFT and PH. This result may be explained by the regression quantile curve for x₁ in Figure 3. That is, the effect of x₁ on the τ-th quantile of y may have a changing trend with τ; the coefficient magnitude decreases and approaches 0 around τ = 0.6. As a result, the penalization method based on local quantile regression at a large τ would fail to identify restless leg syndrome as an important risk factor for predicting the mortality of dialysis patients. Fitting models that only allow for constant effects, such as the Cox proportional hazards model and the AFT model, also appears to attenuate the influence of restless leg syndrome on dialysis survival, thereby excluding x₁ in the final model. This example, again, demonstrates the importance of appropriately accounting for varying covariate effects in identifying important variables and building parsimonious predictive models.

Table 4.

Example 2: Analysis of the dialysis data

Method	Selected variables	APE
US	x1, x2, x5, x14	0.236
PS	x1, x2, x3, x4, x5, x11, x14	0.244
SS(0.25)	x1, x14	0.244
SS(0.50)	x2, x14	0.242
SS(0.75)	x2, x11, x14	0.243
AAFT	x2, x4, x10, x13, x14	0.252
PH	x2, x4, x8, x10, x11, x13, x14	0.258

Fig. 3.

Dialysis example: Regression quantiles (dotted lines) and the corresponding pointwise 95% confidence intervals (shaded areas), and penalized AFT estimates (horizontal solid lines) and the corresponding pointwise 95% confidence intervals (horizonal dashed lines).

4 Remarks

Meaningful heterogeneous associations between covariates and response are naturally embedded in many real datasets but are precluded by common regression models with constant coefficients. Some of empirical work presented in the paper shows that making inference based on a constant coefficient model in the presence of varying covariate effects can lead to misleading variable selection results and also deteriorated prediction accuracy.

In this work, we adopt a global quantile regression model as the platform to address potential inhomogeneity in the association of interest. This choice of model facilitates the development of the proposed variable selection method by avoiding smoothing and enabling stable and efficient computation.

Our empirical work also strongly suggests that fitting penalized “local” quantile regression pointwisely would often lead to an overfitted model. The intuition for this phenomenon shares similar flavor with the multiple comparison problem. To be more clear, consider the problem of testing the null hypothesis, $H_0:{\beta }_0^{(j)}(\tau )=0$ for all τ ∈ Δ. The estimation of ${\beta }_0^{(j)}(·)$ based on the pointwise penalization approach is analogous to the testing procedure that tests $H_0(\tau ):{\beta }_0^{(j)}(\tau )=0$ for each τ ∈ Δ and rejects H₀ when H₀(τ) is rejected at any τ ∈ Δ. In this view, the chance that the pointwise estimator of a true zero coefficient function fails to shrink uniformly in τ would be analogous to the type-I error in the above naive test procedure, which can be significantly inflated particularly when Δ contains an infinite number of τ.

For randomly censored data, the proposed shrinkage method that assumes the independence between the censoring variable C and Z can be generalized to allow for covariate-dependent censoring. For example, one may replace the Ĝ(X_i) in V_n(β; τ) by some adhoc estimator of G(t∣Z) ≡ pr(C > t∣Z) obtained via reasonable parametric or semiparametric modeling of G(·∣Z).

Appendix A: Proof of Theorem 1

Define B_n,τ (C) = {β : β = β₀(τ)+(n^−1/2ℓ₂n)u,∥u∥ ≤ C}, and let ∂B_n,τ (C) denote the boundary set of B_n,τ (C). Since W_n,λn(β; τ) is convex in β for all τ ∈ Δ, it is sufficient to show that for any ε > 0, there exists C₀ > 0 and N₀ > 0 such that for n ≥ N₀,

$$\mathrm{pr}(\underset{\tau \in \Delta }{\inf }[\underset{\mathit{\beta }\in \partial B_{n,\tau }(C_0)}{\inf }{W}_{n,{\mathrm{\lambda }}_n}(\mathit{\beta };\tau )-W_{n,{\mathrm{\lambda }}_n}\{{\mathit{\beta }}_0(\tau );\tau \}]>0)\ge 1-\epsilon .$$ (2)

To show equation (2), first note that

$$\begin{array}{l}{n}^{-1}{W}_{n,{\mathrm{\lambda }}_n}(\mathit{\beta };\tau )-n^{-1}{W}_{n,{\mathrm{\lambda }}_n}({\mathit{\beta }}_0(\tau );\tau )\\ =n^{-1}\sum _{i=1}^n[\rho (Y_i-{\mathit{Z}}_i^{\mathrm{T}}\mathit{\beta };\tau )-\rho \{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau );\tau \}]\\ +n^{-1}{\mathrm{\lambda }}_n\sum _{j=2}^p\{|{\beta }^{(j)}|w_{n,j}(\tau )-|{\beta }_0^{(j)}(\tau )|w_{n,j}(\tau )\}\\ \equiv I+\mathit{II}\end{array}$$

Write $e_i(\tau )=Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau )$ and u(τ ; β) = β − β₀(τ), and let D and D_i respectively denote operators such that D(U) = U − E(U) and D_i(U) = U − E(U∣Z_i) for a random variable U. We can further decomposed the term I as I = III + IV, where

$$\begin{array}{lll}\mathit{III}& =& E[I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\{-e_i(\tau )+{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}]\\ & & +E[I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<e_i(\tau )<0\}\{e_i(\tau )-{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}]\mathrm{and}\\ & & n·(\mathit{IV})\\ & & =\sum _{i=1}^{n}D\left(I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\frac{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })-e_i(\tau )}{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}{\mathit{Z}}_i^{\mathrm{T}}\right)\mathit{u}(\tau ;\mathit{\beta })\\ & & +\sum _{i=1}^{n}D\left(I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<e_i(\tau )<0\}\frac{e_i(\tau )-{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}{-{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}(-{\mathit{Z}}_i^{\mathrm{T}})\right)\mathit{u}(\tau ;\mathit{\beta })\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })>0\}{D}_i(I\{e_i(\tau )>{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\})\{-\tau {\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })>0\}{D}_i(I\{e_i(\tau )<0\})\{(1-\tau ){\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })>0\}{D}_i(I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\})\{-\tau {\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<0\}{D}_i(I\{e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\})\{(1-\tau ){\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<0\}{D}_i(I\{e_i(\tau )>0\})\{-\tau {\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & +\sum _{i=1}^{n}I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<0\}{D}_i(I\{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })<e_i(\tau )<0\})\{(1-\tau ){\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\\ & & \equiv \sum _{j=1}^{8}n·({\mathit{IV}}_j)\end{array}$$

For the term IV₁, we can show that the function class, $F=\left\{I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\frac{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })-e_i(\tau )}{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}{\mathit{Z}}_i,\tau \in \Delta ,\mathit{\beta }\in \mathit{A}(C_0)\right\}$, is a Donsker class (page 81 in Van der Vaart and Wellner (2000)), where A(C₀) = {b ∈ R^p : inf_τ∈Δ∥b − β₀(τ)∥ ≤ C₀}. Given the uniform boundedness of the functional class F and since A(C₀) covers B_n,τ (C₀) for all τ ∈ Δ, we can apply the functional law of the iterated logarithm (LIL) (Goodman et al, 1981) to $n^{-1}{\sum }_{i=1}^{n}D\left(I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\frac{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })-e_i(\tau )}{{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}{\mathit{Z}}_i^{\mathrm{T}}\right)$, and get

$$\underset{\tau \in \Delta ,\mathit{\beta }\in \partial B_{n,\tau }(C_0)}{\sup }\left|{\mathit{IV}}_1\right|\le O_p(n^{-1/2}{\ell }_{2}n)(C_0{n}^{-1/2}{\ell }_{2}n).$$

Similarly, we can show that, for j = 2, …, 8,

$$\underset{\tau \in \mathit{\Delta },\mathit{\beta }\in \partial B_{n,\tau }(C_0)}{\sup }\Vert {\mathit{IV}}_j\Vert \le O_p(n^{-1/2}{\ell }_{2}n)(C_0{n}^{-1/2}{\ell }_{2}n).$$

Therefore, we have

$$\underset{\tau \in \Delta ,\mathit{\beta }\in \partial B_{n,\tau }(C_0)}{\sup }\left|\mathit{IV}\right|\le O_p(n^{-1/2}{\ell }_{2}n)(C_0{n}^{-1/2}{\ell }_{2}n).$$ (3)

For the term III, note that, under condition C2 (i), inf_{τ∈Δ, z} f_τ (0∣z) > 0. By the definition of B_n,τ (C₀), there exists N₁ > 0 such that for n ≥ N₁,

$$\underset{\mathit{\beta }\in \partial B_{n,\tau }(C_0),\mathit{z}}{\sup }|{\mathit{z}}^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })|\le {\delta }_1\wedge \underset{\tau \in \Delta ,\mathit{z}}{\inf }\{f_{\tau }(0|\mathit{z})/\left(2A_0\right)\}.$$

This, coupled with condition C2 (ii), implies that for n > N₁, f_τ(x∣z) > inf_{τ∈Δ, z} f_τ (0∣z)/2 for any x ∈ (0, z^T u(τ ; β)) with β ∈ ∂B_n,τ (C₀). Let δ₂ ≡ inf_{τ, z}f_τ (0∣z)/2 and δ₃ ≡ eig min E (ZZ^T), where eig min(·) denotes the minimum eigenvalue of a matrix. Let E_Z(·) denote the expectation with regard to Z. We get, for any τ ∈ Δ and β ∈ ∂B_n,τ (C₀),

$$\begin{array}{lll}& & E[I\{0<e_i(\tau )<{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\{-e_i(\tau )+{\mathit{Z}}_i^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}]\\ & & =E_{\mathit{Z}}\left[{\mathit{\int }}_0^{{\mathit{Z}}^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}\{-x+{\mathit{Z}}^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}{f}_{\tau }(x\mid \mathit{Z})\right],\\ & & \ge {\delta }_2{E}_{\mathit{Z}}\left[{\mathit{\int }}_0^{{\mathit{Z}}^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })}\{-x+{\mathit{Z}}^{\mathrm{T}}\mathit{u}(\tau ;\mathit{\beta })\}\mathit{dx}\right]\\ & & ={\delta }_2{E}_{\mathit{Z}}\left\{\frac{1}{2}\mathit{u}{(\tau ;\mathit{\beta })}^{\mathrm{T}}(\mathit{Z}{\mathit{Z}}^{\mathrm{T}})\mathit{u}(\tau ;\mathit{\beta })\right\}\\ & & \ge \frac{1}{2}{\delta }_2{\delta }_3{C}_0^2{(n^{-1/2}{\ell }_{2}n)}^2>0.\end{array}$$

Since a similar result can be shown for the second term in III, it follows that

$$\underset{\mathit{\beta }\in \partial B_{n,\tau }(C_0),\tau \in \Delta }{\inf }\mathit{III}\ge C_0^2{O}_p({({\ell }_{2}n)}^2/n).$$ (4)

For the term II, it is easy to see that

$$\mathit{II}\ge -\frac{{\mathrm{\lambda }}_n}{n}\sum _{j=2}^s|{\beta }^{(j)}-{\beta }_0^{(j)}(\tau )|w_{n,j}(\tau ).$$

By the uniform consistency of β̃_n (τ), for 2 ≤ j ≤ s, ${\sup }_{\tau \in \Delta }|w_{n,j}(\tau )|={({\sup }_{\tau \in \Delta }|{\stackrel{\sim }{\beta }}_n^{(j)}(\tau )|)}^{-1}=O_p(1)$. Therefore, with (n^1/2 ℓ₂n)⁻¹ λ_n = O(1),

$$\underset{\mathit{\beta }\in \partial B_{n,\tau }(C_0),\tau \in \Delta }{\inf }\mathit{II}\ge -C_0·O_p({({\ell }_{2}n)}^2/n).$$ (5)

Based on equations (3), (4), and (5), it follows that

$$\underset{\tau \in \Delta ,\mathit{\beta }\in \partial B_{n,\tau }(C_0)}{\inf }[n^{-1}{W}_{n,{\mathrm{\lambda }}_n}(\mathit{\beta };\tau )-n^{-1}{W}_{n,{\mathrm{\lambda }}_n}\{{\mathit{\beta }}_0(\tau );\tau \}]\ge C_0^2·O_p({({\ell }_{2}n)}^2/n)-C_0·O_p({({\ell }_{2}n)}^2/n).$$

Therefore, (2) holds if we choose C₀ large enough. This completes the proof of Theorem 1.

Appendix B: Proof of Theorem 2

Define sgn(x) = I(x > 0) − I(x < 0), $U_{n,j}(\mathit{\beta };\tau )=\frac{\partial W_{n,{\mathrm{\lambda }}_n}(\mathit{\beta };\tau )}{\partial {\beta }^{(j)}}$, and let U_n(β; τ) = (U_n,1 (β, τ), …, U_n,p (β, τ))^T. Define ${\mu }_j(\mathit{\beta })=E[-Z_i^{(j)}\{\tau -I(Y_i-{\mathit{Z}}_i^{\mathrm{T}}\mathit{\beta }<0)\}]$, μ (β) = (μ₁ (β), …, μ_p (β))^T, and A (β) = ∂μ (β)/∂β ≡ (A₁(β)^T, …, A_p(β)^T)^T.

First, by Theorem 1, for j = 2, …, s, we have

$$\underset{n\to \infty }{\lim }\mathrm{pr}(\underset{\tau \in \Delta }{\sup }|{\widehat{\beta }}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )|=0)\le \underset{n\to \infty }{\lim }\mathrm{pr}(\underset{\tau \in \Delta }{\sup }|{\widehat{\beta }}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )-{\beta }_0^{(j)}(\tau )|\ge \underset{\tau \in \Delta }{\sup }|{\beta }_0^{(j)}(\tau )|)=0.$$ (6)

Next, we note that given the uniform consistency of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ implied by Theorem 1, it can be shown by following the lines of Lemma 1 in Peng and Huang (2008) that

$$\begin{array}{l}\underset{\tau \in \Delta }{\sup }|-n^{-1/2}\sum _{i=1}^n{Z}_i^{(j)}[\tau -I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )<0\}]\\ +n^{-1/2}\sum _{i=1}^n{Z}_i^{(j)}[\tau -I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau )<0\}]\end{array}$$ (7)

$$-n^{1/2}[{\mu }_j\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )\}-{\mu }_j\{{\mathit{\beta }}_0(\tau )\}]|=o_p(1),j=1,\dots ,s.$$ (8)

This implies, for j = s + 1, …, p,

$$\begin{array}{l}\underset{\tau \in \Delta }{\sup }|E_{n,j}(\tau )|\\ \equiv \underset{\tau \in \Delta }{\sup }\mid n^{-1/2}{U}_{n,j}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau );\tau \}-n^{-1/2}{U}_{n,j}\{{\mathit{\beta }}_0(\tau );\tau \}\\ -n^{1/2}[{\mu }_j\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )\}-{\mu }_j\{{\mathit{\beta }}_0(\tau )\}]\\ -n^{1/2}{\mathrm{\lambda }}_n{w}_{n,j}(\tau )\mathrm{sgn}\{{\widehat{\beta }}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )\}\mid =o_p(1)\end{array}$$ (9)

By the definition of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$,

$$\underset{\tau \in \Delta }{\sup }|n^{-1/2}{U}_{n,j}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau );\tau \}|=o_p(1).$$ (10)

Applying Functional LIL to n⁻¹ U_n,j{β₀(τ); τ} gives

$$\underset{\tau \in \Delta }{\sup }|n^{-1/2}{U}_{n,j}\{{\mathit{\beta }}_0(\tau );\tau \}|=O_p({\ell }_{2}n).$$ (11)

An application of Taylor expansion to ${\mu }_j\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )\}-{\mu }_j\{{\mathit{\beta }}_0(\tau )\}$, coupled with Theorem 1 and the uniform boundedness of A_j (β), shows that

$$\underset{\tau \in \Delta }{\sup }|n^{1/2}[{\mu }_j\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )\}-{\mu }_j\{{\mathit{\beta }}_0(\tau )\}]|=O_p({\ell }_{2}n).$$ (12)

In addition, the proof of Theorem 1 can be used to justify that ${\sup }_{\tau \in \Delta }|{\stackrel{\sim }{\beta }}_n^{(j)}(\tau )|=O_p(n^{-1/2}{\ell }_{2}n)$ and thus

$$\underset{n\to \infty }{\lim }\left(\frac{1}{{\ell }_{2}n}\right)\left(n^{-1/2}{\mathrm{\lambda }}_n{w}_{n,j}(\tau )\right)=\infty .$$ (13)

By (10), (11), (12), and (13), with a fixed M > 0, lim_n→∞ pr(sup_τ∈Δ |E_n,j (τ)| ≤ M, ${\sup }_{\tau \in \Delta }|{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )|\ne 0)=0$. This, coupled with (9), implies that, for j = s + 1, …, p,

$$\begin{array}{l}\underset{n\to \infty }{\lim }\mathrm{pr}(\underset{\tau \in \Delta }{\sup }|{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )|\ne 0)\\ \le \underset{n\to \infty }{\lim }\{\mathrm{pr}(\underset{\tau \in \Delta }{\sup }|E_{n,j}(\tau )|\le M,\underset{\tau \in \Delta }{\sup }|{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(j)}(\tau )|\ne 0)\end{array}$$ (14)

$$+\mathrm{pr}(E_{n,j}(\tau )\mid >M)\}=0.$$ (15)

The proof of Theorem 2 (i) is completed based on (6) and (14).

Let $\stackrel{a}{=}$ denote asymptotic equivalence in the sense that the difference converges to zero in probability uniformly in τ ∈ Δ. Define ${\overline{\mathit{\beta }}}_n(\tau )=\left(\begin{array}{c}{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1:s)}(\tau )\\ 0\end{array}\right)$. By the result in Theorem 2 (i), we have

$$n^{-1/2}{\mathit{U}}_n({\overline{\mathit{\beta }}}_n;\tau )\stackrel{a}{=}{n}^{-1/2}{\mathit{U}}_n({\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}};\tau )$$

and hence $n^{-1/2}{\mathit{U}}_n({\overline{\mathit{\beta }}}_n;\tau )\stackrel{a}{=}0$. Using the result in (7) and applying Taylor expansion to μ_j {β̄_n (τ)} − μ_j {β₀ (τ)}, we get

$$\begin{array}{l}0\stackrel{a}{=}{n}^{-1/2}{\mathit{U}}_n({\overline{\mathit{\beta }}}_n;\tau )\\ \stackrel{a}{=}-n^{-1/2}\sum _{i=1}^n{\mathit{Z}}_i[\tau -I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau )<0\}]\\ +[\mathit{A}\{\stackrel{⌣}{\mathit{\beta }}(\tau )\}]n^{1/2}\{{\overline{\mathit{\beta }}}_n(\tau )-{\mathit{\beta }}_0(\tau )\}+n^{-1/2}{\mathrm{\lambda }}_n{\mathit{b}}_n(\tau ),\end{array}$$ (16)

where β̆(τ) is on the line segment between β₀ (τ) and β̄(τ), and ${\mathit{b}}_n(\tau )={(w_{n,1}(\tau )\mathrm{sgn}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1)}(\tau )\},\dots w_{n,s}(\tau )\mathrm{sgn}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(s)}(\tau )\},0\dots 0)}^{\mathrm{T}}$.

Since lim_n→∞ n^−1/2 λ_n = 0, sup_{τ ∈ Δ} |w_n,j (τ)| = O_p(1) for j = 1, …, s, and $A\{\stackrel{⌣}{\mathit{\beta }}(\tau )\}\stackrel{a}{=}A\{{\mathit{\beta }}_0(\tau )\}$ given the uniform convergence of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}}(\tau )$ to β₀ (τ), it follows from (16) that

$$\begin{array}{l}{n}^{1/2}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1:s)}(\tau )-{\mathit{\beta }}_0^{(1:s)}(\tau )\}\\ \stackrel{a}{=}{\mathit{A}}_{11}\{{\mathit{\beta }}_0(\tau )\}{n}^{-1/2}\sum _{i=1}^n{\mathit{Z}}_i^{(1:s)}I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau )<0\},\end{array}$$ (17)

where A₁₁ (·) stands for the submatrix of A (·) formed by the first s rows and columns. An application of the Donsker theorem based on (17) thus shows that $n^{1/2}\{{\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1:s)}(\tau )-{\mathit{\beta }}_0^{(1:s)}(\tau )\}$ converges weakly to a mean zero Gaussian process with the covariance matrix,

$$\begin{array}{r}\sum ({\tau }^{\prime },\tau )=E[{\mathit{A}}_{11}\{{\mathit{\beta }}_0({\tau }^{\prime })\}{\mathit{Z}}_i^{(1:s)}{\mathit{Z}}_i^{{(1:s)}^{\mathrm{T}}}{\mathit{A}}_{11}^{\mathrm{T}}\{{\mathit{\beta }}_0(\tau )\}\\ I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0({\tau }^{\prime })<0\}I\{Y_i-{\mathit{Z}}_i^{\mathrm{T}}{\mathit{\beta }}_0(\tau )<0\}].\end{array}$$

Using similar steps, we can show that (17) still holds when β̂_oracle (τ) is in place of ${\widehat{\mathit{\beta }}}_{n,{\mathrm{\lambda }}_n}^{\mathrm{US}(1:s)}(\tau )$. This completes the proof of Theorem 2.