Penalized Bregman divergence for large-dimensional regression and classification

Summary

Regularization methods are characterized by loss functions measuring data fits and penalty terms constraining model parameters. The commonly used quadratic loss is not suitable for classification with binary responses, whereas the loglikelihood function is not readily applicable to models where the exact distribution of observations is unknown or not fully specified. We introduce the penalized Bregman divergence by replacing the negative loglikelihood in the conventional penalized likelihood with Bregman divergence, which encompasses many commonly used loss functions in the regression analysis, classification procedures and machine learning literature. We investigate new statistical properties of the resulting class of estimators with the number p_n of parameters either diverging with the sample size n or even nearly comparable with n, and develop statistical inference tools. It is shown that the resulting penalized estimator, combined with appropriate penalties, achieves the same oracle property as the penalized likelihood estimator, but asymptotically does not rely on the complete specification of the underlying distribution. Furthermore, the choice of loss function in the penalized classifiers has an asymptotically relatively negligible impact on classification performance. We illustrate the proposed method for quasilikelihood regression and binary classification with simulation evaluation and real-data application.

1. Introduction

Regularization is used to obtain well-behaved solutions to overparameterized estimation problems, and is particularly appealing in high dimensions. The topic is reviewed by Bickel & Li (2006). Regularization estimates a vector parameter of interest β ∈ 𝕉^p_n by minimizing the criterion function,

$${\ell }_n(\beta )=L_n(\beta )+P_{{\lambda }_n}(\beta )\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}({\lambda }_n>0),$$

consisting of a data fit functional L_n, which measures how well β fits the observed set of data; a penalty functional P_λn, which assesses the physical plausibility of β; and a regularization parameter λ_n, which regulates the penalty. Depending on the nature of the output variable, the term L_n quantifies the error of an estimator by different error measures. For example, the quadratic loss function has nice analytical properties and is usually used in regression analysis. However, it is not always adequate in classification problems, where the misclassification loss, deviance loss, hinge loss for the support vector machine (Vapnik, 1996) and exponential loss for boosting (Hastie et al., 2001) are more realistic and commonly used in classification procedures.

Currently, most research on regularization methods is devoted to variants of penalty methods in conjunction with linear models and likelihood-based models in regression analysis. For linear model estimation with a fixed number p of parameters, Tibshirani (1996) introduced the L₁-penalty for the proposed lasso method, where the quadratic loss is in use. Theoretical properties related to the lasso have been intensively studied; see Knight & Fu (2000), Meinshausen & Buhlmann (2006) and Zhao & Yu (2006). Zou (2006) mentioned that the lasso is in general not variable selection consistent, but the adaptive lasso via combining appropriately weighted L₁-penalties is consistent. Huang et al. (2008) extended the results in Zou (2006) to high-dimensional linear models. Using the smoothly clipped absolute deviation penalty, Fan & Li (2001) showed that the penalized likelihood estimator achieved the oracle property: the resulting estimator is asymptotically as efficient as the oracle estimator. In their treatment, the number p_n of model parameters is fixed at p, and the loss function equals the negative loglikelihood. Fan & Peng (2004) extended the result to p_n diverging with n at a certain rate.

On the loss side, the literature on penalization methods includes much less discussion of either the role of the loss function in regularization for models other than linear or likelihood-based models, or the impact of different loss functions on classification performance. The least angle regression algorithm (Efron et al., 2004) for L₁-penalization was developed for linear models using the quadratic loss. Rosset & Zhu (2007) studied the piecewise linear regularized solution paths for differentiable and piecewise quadratic loss functions with L₁ penalty. It remains desirable to explore whether penalization methods using other types of loss functions can potentially benefit from the efficient least-angle regression algorithm. Moreover, theoretical results on the penalized likelihood are not readily translated into results for approaches, such as quasilikelihood (Wedderburn, 1974; McCullagh, 1983; Strimmer, 2003), where the distribution of the observations is unknown or not fully specified. Accordingly, a discussion of statistical inference for penalized estimation using a wider range of loss functions is needed.

In this study, we broaden the scope of penalization by incorporating loss functions belonging to the Bregman divergence class which unifies many commonly used loss functions. In particular, the quasilikelihood function and all loss functions mentioned previously in classification fall into this class. We introduce the penalized Bregman divergence by replacing the quadratic loss or the negative loglikelihood in penalized least-squares or penalized likelihood with Bregman divergence, and call the resulting estimator a penalized Bregman divergence estimator. Nonetheless, the Bregman divergence in general does not fulfill assumptions specifically imposed on the likelihood function associated with penalized likelihood.

We investigate new statistical properties of large-dimensional penalized Bregman divergence estimators, with dimensions dealt with separately in two cases:

$$\text{Case}\hspace{0.17em}\text{I}:\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{p}_n\hspace{0.17em}\text{is}\hspace{0.17em}\text{diverging}\hspace{0.17em}\text{with}\hspace{0.17em}n;$$ (1)

$$\text{Case}\hspace{0.17em}\hspace{0.17em}\text{II}:\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{p}_n\hspace{0.17em}\text{is}\hspace{0.17em}\text{nearly}\hspace{0.17em}\text{comparable}\hspace{0.17em}\text{with}\hspace{0.17em}n.$$ (2)

Zhang & Zhang (2010) give an application of the penalization method developed in this paper to estimating the hemodynamic response function for brain fMRI data where p_n is as large as n. The current paper shows that the penalized Bregman divergence estimator, combined with appropriate penalties, achieves the same oracle property as the penalized likelihood estimator, but the asymptotic distribution does not rely on the complete specification of the underlying distribution. From the classification viewpoint, our study elucidates the applicability and consistency of various classifiers induced by penalized Bregman divergence estimators. Technical details of this paper are in the online Supplementary Material.

2. The penalized Bregman divergence estimator

2.1. Bregman divergence

We give a brief overview of Bregman divergence. For a given concave function q with derivative q′, Bregman (1967) introduced a device for constructing a bivariate function,

$$Q(\nu ,\hspace{0.17em}\mu )=-q(\nu )+q(\mu )+(\nu -\mu )q^{\prime }(\mu ).$$ (3)

Figure 1 displays Q and the corresponding q. It is readily seen that the concavity of q ensures the nonnegativity of Q. Moreover, for a strictly concave q, Q(ν, μ) = 0 is equivalent to ν = μ. However, since Q(ν, μ) is not generally symmetric in ν and μ, Q is not a metric or distance in the strict sense. Hence, we call Q the Bregman divergence and call q the generating function of Q. See Efron (1986), Lafferty et al. (1997), Lafferty (1999), Kivinen & Warmuth (1999), Grünwald & Dawid (2004), Altun & Smola (2006) and references therein.

Fig. 1.

Illustration of Q(ν, μ) as defined in (3). The concave curve is q; the two dashed lines indicate locations of ν and μ; the solid straight line is q(μ) + (ν – μ)q^′(μ); the length of the vertical line with arrows at each end is Q(ν, μ).

The Bregman divergence is suitable for a broad array of error measures Q. For example, q(μ) = aμ – μ² with some constant a yields the quadratic loss Q(Y, μ) = (Y – μ)². For a binary response variable Y, q(μ) = min{μ, (1 – μ)} gives the misclassification loss Q(Y, μ) = I {Y ≠ I (μ > 1/2)}, where I (·) denotes the indicator function; q(μ) = −{μ log(μ) + (1 –μ) log(1 – μ)} gives the Bernoulli deviance loss Q(Y, μ) = −{Y log(μ) + (1 – Y) log(1 – μ)}; q(μ) = 2 min{μ, (1 – μ)} results in the hinge loss; and q(μ) = 2{μ(1 – μ)}^1/2 yields the exponential loss Q(Y, μ) = exp[–(Y – 0.5) log{μ/(1 – μ)}].

Conversely, for a given Q, Zhang et al. (2009) provided necessary and sufficient conditions for Q being a Bregman divergence, and in that case derived an explicit formula for q. Applying this inverse approach from Q to q, they illustrated that the quasilikelihood function, the Kullback–Leibler divergence or the deviance loss for the exponential family of probability functions, and many margin-based loss functions (Shen et al., 2003) are Bregman divergences. To our knowledge, there is little theoretical work in the literature on thoroughly examining the penalized Bregman divergence, via methods of regularization, for large-dimensional model building, variable selection and classification problems.

2.2. The model and penalized Bregman divergence estimator

Let (X, Y) denote a random realization from some underlying population, where X = (X₁, …, X_pn)^T is the input vector and Y is the output variable. The dimension p_n follows the assumption in (1) or (2). We assume the parametric model,

$$m(x)=E(Y|X=x)=F^{-1}(b_{0;0}+x^T{\beta }_0),$$ (4)

where F is a known link function, b_0;0 ∈ 𝕉¹ and β₀ = (β_1;0, …, β_pn;0)^T ∈ 𝕉^p_n are the unknown true parameters. Throughout the paper, it is assumed that some entries in β₀ are exactly zero. Write ${\beta }_0={\{{\beta }_0^{(\text{I})\text{T}},\hspace{0.17em}{\beta }_0^{(\text{II})\text{T}}\}}^{\text{T}}$, where ${\beta }_0^{(\text{I})}$ collects all nonzero coefficients, and ${\beta }_0^{(\text{II})}=0$.

Our goal is to estimate the true parameters via penalization. Let {(X₁, Y₁), …, (X_n, Y_n)} be a sample of independent random pairs from (X, Y), where X_i = (X_i1, …, X_ipn)^T. The penalized Bregman divergence estimator (b̂₀, β̂) is defined as the minimizer of the criterion function,

$${\ell }_n(b_0,\hspace{0.17em}\beta )=\frac{1}{n}\sum _{i=1}^{n}Q\{Y_i,\hspace{0.17em}{F}^{-1}(b_0+X_i^T\beta )\}+\sum _{j=1}^{p_n}{P}_{{\lambda }_n}(|{\beta }_j|),$$ (5)

where β = (β₁, …, β_pn)^T, the loss function Q(·, ·) is a Bregman divergence, and P_λn (·) represents a nonnegative penalty function indexed by a tuning constant λ_n > 0. Set β̃ = (b₀, β^T)^T, and correspondingly ${\tilde{X}}_i={(1,\hspace{0.17em}{X}_i^{\text{T}})}^{\text{T}}$. Then (5) can be written as

$${\ell }_n(\tilde{\beta })=\frac{1}{n}\sum _{i=1}^{n}Q\{Y_i,\hspace{0.17em}{F}^{-1}({\tilde{X}}_i^{\text{T}}\tilde{\beta })\}+\sum _{j=1}^{p_n}{P}_{{\lambda }_n}(|{\beta }_j|).$$ (6)

The penalized Bregman divergence estimator is β̃_E = (b̂₀, β̂₁, …, β̂_pn)^T = arg min_β̃ ℓ_n(β̃).

Regarding the uniqueness of β̃_E, assume that the quantities

$$q_j(y;\theta )=({\partial }^j/\partial {\theta }^j)Q\{y,\hspace{0.17em}{F}^{-1}(\theta )\}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}(j=0,\hspace{0.17em}1,\hspace{0.17em}\dots ),$$ (7)

exist finitely up to any order required. Provided that for all θ ∈ 𝕉 and all y in the range of Y,

$$q_2(y;\theta )>0,$$ (8)

it follows that $L_n(\tilde{\beta })=n^{-1}{\sum }_{i=1}^{n}Q\{Y_i,\hspace{0.17em}{F}^{-1}({\tilde{X}}_i^T\tilde{\beta })\}$ in (6) is convex in β̃. In that case, if convex penalties are used in (6), then ℓ_n(β̃) is necessarily convex in β̃, and hence the local minimizer β̃_E is the unique global penalized Bregman divergence estimator. For nonconvex penalties, however, the local minimizer may not be globally unique.

3. Penalized Bregman divergence with nonconvex penalties: p_n ≪ n

3.1. Consistency

We start by introducing some notation. Let s_n denote the number of nonzero coordinates of β₀, and set ${\tilde{\beta }}_0={(b_{0;0},\hspace{0.17em}{\beta }_0^{\text{T}})}^{\text{T}}$. Define

$$a_n=\underset{j=1,\dots ,s_n}{\text{max}}|{P^{\prime }}_{{\lambda }_n}(|{\beta }_{j;0}|)|,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{b}_n=\underset{j=1,\dots ,s_n}{\text{max}}|{P^{″}}_{{\lambda }_n}(|{\beta }_{j;0}|)|,$$

where $P_{\lambda }^{(j)}(|\beta |)$ is shorthand for (d^j/dx^j)P_λ(x)|_x=|β|, j = 1, 2. Unless otherwise stated, ‖·‖ denotes the L₂-norm. Theorem 1 guarantees the existence of a consistent local minimizer for (6), and states that the local penalized Bregman divergence estimator β̃_E is (n/p_n)^1/2-consistent.

Theorem 1 (Existence and consistency). Assume Condition A in the Appendix, a_n = O(1/n^1/2) and b_n = o(1). If $p_n^4/n\to 0$, (p_n/n)^1/2/λ_n → 0 and min_j=1,...,sn |β_j;0|/λ_n → ∞ as n → ∞, then there exists a local minimizer β̃_E of (6) such that ‖β̃_E – β̃₀‖ = O_P {(p_n/n)^1/2}.

3.2. Oracle property

Following Theorem 1, the oracle property of the local minimizer is given in Theorem 2 below. Before stating it, we need some notation. Write X = (X^(I)_T, X^(II)_T)^T, X̃^(I) = (1, X^(I)_T)^T, and β̃^(I) = (b₀, β^(I)_T)^T. For the penalty term, let

$$\begin{array}{l}{d}_n={\{0,\hspace{0.17em}{P^{\prime }}_{{\lambda }_n}(|{\beta }_{1;0}|)\text{sign}({\beta }_{1;0}),\dots ,{P^{\prime }}_{{\lambda }_n}(|{\beta }_{s_n;0}|)\text{sign}({\beta }_{s_n;0})\}}^{\text{T}},\\ {\Sigma }_n=\text{diag}\hspace{0.17em}\{0,\hspace{0.17em}{P^{″}}_{{\lambda }_n}(|{\beta }_{1;0}|),\dots ,\hspace{0.17em}{P^{″}}_{{\lambda }_n}(|{\beta }_{s_n;0}|)\}.\end{array}$$

For the q function, define F_n = q⁽²⁾{m(X)}/[F⁽¹⁾{m(X)}]²X̃^(I)X̃^(I)_T and

$${\mathrm{\Omega }}_n=E\hspace{0.17em}[\text{var}(Y|X)q^{(2)}\{m(X)\}{F}_n],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{H}_n=-E(F_n).$$

Theorem 2 (Oracle property). Assume Condition B in the Appendix.

1. If $p_n^2/n=O(1)$, (p_n/n^1/2)/λ_n → 0 and lim inf_n→∞ lim inf_θ→0+ P′_λn (θ)/λ_n > 0 as n → ∞, then any (n/ p_n)^1/2-consistent local minimizer ${\tilde{\beta }}_{\text{E}}={({\tilde{\beta }}_{\text{E}}^{(\text{I})\text{T}},\hspace{0.17em}{\widehat{\beta }}^{(\text{II})\text{T}})}^{\text{T}}$ satisfies pr(β̂^(II) = 0) → 1.

2. Moreover, if a_n = O(1/n^1/2), $p_n^5/n\to 0$ and min_j=1,...,sn|β_j;0|/λ_n → ∞, then for any fixed integer k and any k × (s_n + 1) matrix A_n such that A_n $A_n^{\text{T}}\to G$ with G being a k × k nonnegative-definite symmetric matrix, $n^{1/2}{A}_n{\mathrm{\Omega }}_n^{-1/2}\{(H_n+{\mathrm{\Sigma }}_n)({\tilde{\beta }}_{\text{E}}^{(\text{I})}+{\tilde{\beta }}_0^{(\text{I})})+d_n\}\to N(0,\hspace{0.17em}G)$ in distribution.

Theorem 2 has some useful consequences: First, the p_n-dimensional penalized Bregman divergence estimator, combined with appropriate penalties, achieves the same oracle property as the penalized likelihood estimator of Fan & Peng (2004): the estimators of the zero parameters take exactly zero values with probability tending to 1, and the estimators of the nonzero parameters are asymptotically normal with the same means and variances as if the zero coefficients were known in advance. Second, the asymptotic distribution of the penalized Bregman divergence estimator relies on the underlying distribution of Y | X through E(Y | X) and var(Y | X), but does not require a complete specification of the underlying distribution. Third, the asymptotic distribution depends on the choice of the Q-loss only through the second derivative of its generating q function. This enables us to evaluate the impact of loss functions on the penalized Bregman divergence estimators and to derive an optimal loss function in certain situations.

According to Theorem 2, the asymptotic covariance matrix of ${\tilde{\beta }}_{\text{E}}^{(\text{I})}$ is V_n = (H_n + Σ_n)⁻¹ Ω_n(H_n + Σ_n)⁻¹. In practice, V_n is unknown and needs to be estimated. Typically, the sandwich formula can be exploited to form an estimator of V_n by

$${\widehat{V}}_n={({\widehat{H}}_n+{\widehat{\mathrm{\Sigma }}}_n)}^{-1}{\widehat{\mathrm{\Omega }}}_n{({\widehat{H}}_n+{\widehat{\mathrm{\Sigma }}}_n)}^{-1},$$ (9)

where ${\widehat{\mathrm{\Omega }}}_n=n^{-1}{\sum }_{i=1}^n{q}_1^2(Y_i;{\tilde{X}}_i^{{(\text{I})}_{\text{T}}}{\tilde{\beta }}_{\text{E}}^{(\text{I})}){\tilde{X}}_i^{(\text{I})}{\tilde{X}}_i^{{(\text{I})}_{\text{T}}}$, ${\widehat{H}}_n=n^{-1}{\sum }_{i=1}^n{q}_2(Y_i;{\tilde{X}}_i^{{(\text{I})}_{\text{T}}}{\tilde{\beta }}_{\text{E}}^{(\text{I})}){\tilde{X}}_i^{(\text{I})}{\tilde{X}}_i^{{(\text{I})}_{\text{T}}}$ and Σ̂_n = diag{0, P″_λn (|β̂₁|), …, P″_λn (|β̂_sn|)}.

Proposition 1 below demonstrates that for any (n/ p_n)^1/2-consistent estimator ${\tilde{\beta }}_{\text{E}}^{(\text{I})}$ of ${\tilde{\beta }}_0^{(\text{I})}$, V̂_n is a consistent estimator for the covariance matrix V_n, in the sense that $A_n({\widehat{V}}_n-V_n)A_n^{\text{T}}\to 0$ in probability for any k × (s_n + 1) matrix A_n satisfying $A_n{A}_n^{\text{T}}\to G$, where k is any fixed integer.

Proposition 1 (Covariance matrix estimation). Assume Condition B in the Appendix, and b_n = o(1). If $p_n^4/n\to 0$, (p_n/n)^1/2/λ_n → 0 and min_j=1,...,sn |β_j;0|/λ_n → ∞ as n → ∞, then for any $\Vert {\tilde{\beta }}_{\text{E}}^{(\text{I})}-{\tilde{\beta }}_0^{(\text{I})}\Vert =O_P\{{(p_n/n)}^{1/2}\}$, we have that $A_n({\widehat{V}}_n-V_n)A_n^{\text{T}}\to 0$ in probability for any k × (s_n + 1) matrix A_n satisfying $A_n{A}_n^{\text{T}}\to G$, where G is a k × k matrix.

Is there an optimal choice of q such that the corresponding V_n matrix achieves its lower bound? We have that $V_n=H_n^{-1}{\mathrm{\Omega }}_n{H}_n^{-1}$ in two special cases. One is Σ_n = 0 for large n and large min_j=1,...,sn |β_j;0|, which results from the smoothly clipped absolute deviation and hard thresholding penalties; another one is Σ_n = 0 for all n, which results from the weighted L₁-penalties in Theorem 6 below. In these cases, it can be shown via matrix algebra that the optimal q satisfies the generalized Bartlett identity in (11) below. On the other hand, for an arbitrary Σ_n ≠ 0, the complication rises; the optimal q is generally not available in closed-form.

3.3. Hypothesis testing

We consider hypothesis testing about ${\tilde{\beta }}_0^{(\text{I})}$ formulated as

$$H_0:A_n{\tilde{\beta }}_0^{(\text{I})}=0\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{versus}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{H}_1:A_n{\tilde{\beta }}_0^{(\text{I})}\ne 0,$$ (10)

where A_n is a given k × (s_n + 1) matrix such that $A_n{A}_n^{\text{T}}=G$ with G being a k × k positive-definite matrix. This form of linear hypothesis allows one to test simultaneously whether a subset of variables used are statistically significant by taking some specific form of the matrix A_n; for example, A_n = [I_k, 0_{k, sn + 1–k}] yields $A_n{A}_n^{\text{T}}=I_k$.

We propose a generalized Wald-type test statistic of the form

$$W_n=n{\left(A_n{\tilde{\beta }}_{\text{E}}^{(\text{I})}\right)}^{\text{T}}{\left(A_n{\widehat{H}}_n^{-1}{\widehat{\mathrm{\Omega }}}_n{\widehat{H}}_n^{-1}{A}_n^{\text{T}}\right)}^{-1}\left(A_n{\tilde{\beta }}_{\text{E}}^{(\text{I})}\right),$$

where Ω̂_n and Ĥ_n are as defined in (9). This test is asymptotically distribution-free, as Theorem 3 justifies that, under the null, W_n would for large n be distributed as ${\chi }_k^2$.

Theorem 3 (Wald-type test under H₀). Assume Condition C in the Appendix, and let a_n = o{1/(ns_n)^1/2} and $b_n=o(1/p_n^{1/2})$. If $p_n^5/n\to 0$, (p_n/n)^1/2/λ_n → 0 and min_j=1,...,sn|β _j;0|/λ_n → ∞ as n → ∞, then under H₀ in (10), $W_n\to {\chi }_k^2$ in distribution.

Remark 1. To appreciate the discriminating power of W_n in assessing the significance, the asymptotic power can be analyzed. It can be shown that under H₁ in (10) where ‖A_nβ̃₀‖ is independent of n, W_n → +∞ in probability at the rate n. Hence W_n has power function tending to 1 against fixed alternatives. Besides, W_n has a nontrivial local power detecting contiguous alternatives approaching the null at the rate n^−1/2. We omit the lengthy details.

In the context of penalized likelihood estimator β̃_E, Fan & Peng (2004) showed that the likelihood-ratio-type test statistic

$${\mathrm{\Lambda }}_n=2n\left\{\underset{\tilde{\beta }\in {ℝ}^{p_n+1}:\hspace{0.17em}{A}_n{\tilde{\beta }}^{(\text{I})}=0}{\text{min}}{\ell }_n(\tilde{\beta })-{\ell }_n({\tilde{\beta }}_E)\right\}$$

follows an asymptotic χ² distribution under the null hypothesis. Theorem 4 below explores the extent to which this result can feasibly be extended to Λ_n constructed from the broad class of penalized Bregman divergence estimators.

Theorem 4 (Likelihood-ratio-type test under H₀). Assume (8) and Condition D in the Appendix, a_n = o{1/(ns_n)^1/2} and $b_n=o(1/p_n^{1/2})$. If $p_n^5/n\to 0$, (p_n/n)^1/2/λ_n → 0 and min_{j=1,..., sn} |β_j;0|/λ_n → ∞ as n → ∞, then under H₀ in (10), provided that q satisfies the generalized Bartlett identity,

$$q^{(2)}\{m(\cdot )\}=-\frac{c}{\text{var}\left(Y|X=\cdot \right)},$$ (11)

for a constant c > 0, we have that ${\mathrm{\Lambda }}_n/c\to {\chi }_k^2$ in distribution.

Curiously, the result in Theorem 4 indicates that in general, condition (11) on q restricts the application domain of the test statistic Λ_n. For instance, in the case of binary responses, the Bernoulli deviance loss satisfies (11), but the quadratic loss and exponential loss violate (11). This limitation reflects that the likelihood-ratio-type test statistic Λ_n may not be straightforwardly valid for the penalized Bregman divergence estimators.

Remark 2. For a Bregman divergence Q, condition (11) with c = 1 is equivalent to the equality E[∂²Q{Y, m(·)}/∂m(·)² | X = ·] = E([∂ Q{Y, m(·)}/ ∂m(·)]² | X = ·), which includes the Bartlett identity (Bartlett, 1953) as a special case, when Q is the negative loglikelihood. Thus, we call (11) the generalized Bartlett identity. It is also seen that the quadratic loss satisfies (11) for homoscedastic regression models even without knowing the error distribution.

4. Penalized Bregman divergence with convex penalties: p_n ≈ n

4.1. Consistency, oracle property and hypothesis testing

For the nonconvex penalties discussed in § 3, the condition $p_n^4/n\to 0$ or $p_n^5/n\to 0$ can be relaxed to $p_n^3/n\to 0$ in the particular situation where the Bregman divergence is a quadratic loss and the link is an identity link. It remains unclear whether p_n can be relaxed in other cases.

This section aims to improve the rate of consistency of the penalized Bregman divergence estimators and to relax conditions on p_n using certain convex penalties, the weighted L₁-penalties, under which the penalized Bregman divergence estimator β̃_E = (b̂₀, β̂^T)^T is defined to minimize the criterion function,

$${\ell }_n(\tilde{\beta })=\frac{1}{n}\sum _{i=1}^{n}Q\{Y_i,F^{-1}({\tilde{X}}_i^{\text{T}}\tilde{\beta })\}+{\lambda }_n\sum _{j=1}^{p_n}{w}_j|{\beta }_j|,$$ (12)

with w₁, …, w_pn representing nonnegative weights. Define

$$w_{\text{max}}^{(\text{I})}=\underset{j=1,\dots ,s_n}{\text{max}}{w}_j,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{w}_{\text{min}}^{(\text{I})}=\underset{s_n+1⩽j⩽p_n}{\text{min}}\hspace{0.17em}{w}_j.$$

Lemma 1 obtains the existence of a (n/p_n)^1/2-consistent local minimizer of (12). This rate is identical to that in Theorem 1 but, unlike Theorem 1, Lemma 1 includes the L₁-penalty. Other results parallel to those in § 3 can similarly be obtained.

Lemma 1 (Existence and consistency). Assume Conditions A1–A7 in the Appendix and $w_{\text{max}}^{(\text{I})}=O_P\{1/({\lambda }_n{n}^{1/2})\}$. If $p_n^4/n\to 0$ as n → ∞, then there exists a local minimizer β̃_E of (12) such that ||β̃_E – β̃₀|| = O_P {(p_n/n)^1/2}.

Lemma 1 imposes a condition on the weights of nonzero coefficients alone, but ignores the weights on zero coefficients. Theorem 5 below reflects that incorporating appropriate weights to the zero coefficients can improve the rate of consistency from (p_n/n)^1/2 to (s_n/n)^1/2.

Theorem 5 (Existence and consistency). Assume Conditions A1–A7 in the Appendix, $w_{\text{max}}^{(\text{I})}=O_P\{1/({\lambda }_n{n}^{1/2})\}$ and there exists a constant M ∈ (0, ∞) such that ${\text{lim}}_{n\to \infty }\text{pr}(w_{\text{min}}^{(\text{II})}{\lambda }_n>M)=1$. If $s_n^4/n\to 0$ and s_n(p_n – s_n) = o(n), then there exists a local minimizer β̃_E of (12) such that ||β̃_E – β̃₀|| = O_P {(s_n/n)^1/2}.

More importantly, conditions on the dimension p_n are much relaxed. For example, Theorem 5 allows p_n = o(n^{(3+δ)/(4+δ)}) for any δ > 0, provided s_n = O(n^1/(4+δ)), whereas Theorem 1 requires p_n = o(n^1/4) for any s_n ⩽ p_n. This implies that p_n can indeed be relaxed to the case (2) of being nearly comparable with n. On the other hand, the proof of Theorem 5 relies on the flexibility of the weights {w_j}, as seen in an $I_{2,1}^{(\text{II})}$ term. Thus, directly carrying the proof of Theorem 5 through to either the nonconvex penalties in Theorem 1 or the L₁-penalty is not feasible.

Theorem 6 gives an oracle property for the (n/s_n)^1/2-consistent local minimizer.

Theorem 6 (Oracle property). Assume Conditions A1, A2, B3, A4, B5, A6–A7 in the Appendix.

1. If $s_n^2/n=O(1)$ and $w_{\text{min}}^{(\text{II})}{\lambda }_n{n}^{1/2}/{\left(s_n{p}_n\right)}^{1/2}\to \infty$ in probability as n → ∞, then any (n/s_n)^1/2-consistent local minimizer ${\tilde{\beta }}_{\text{E}}={({\tilde{\beta }}_{\text{E}}^{{(\text{I})}_{\text{T}}},{\widehat{\beta }}^{{(\text{II})}_{\text{T}}})}^{\text{T}}$ satisfies pr(β̂^(II) = 0) → 1.

2. Moreover, if $w_{\text{max}}^{(\text{I})}=O_P\{1/({\lambda }_n{n}^{1/2})\}$, $s_n^5/n\to 0$ and min_j=1,...,sn |β_j;0|/(s_n/n)^1/2 → ∞, then for any fixed integer k and any k × (s_n + 1) matrix A_n such that $A_n{A}_n^{\text{T}}\to G$ with G being a k×k nonnegative-definite symmetric matrix, $n^{1/2}{A}_n{\mathrm{\Omega }}_n^{-1/2}\{H_n({\tilde{\beta }}_{\text{E}}^{(\text{I})}-{\tilde{\beta }}_0^{(\text{I})})+{\lambda }_n{\text{W}}_n\text{sign}({\tilde{\beta }}_0^{(\text{I})})\}\to N(0,G)$ in distribution, where W_n = diag(0, w₁, …, w_sn) and $\text{sign}\{{\tilde{\beta }}_0^{(\text{I})}\}={\{\text{sign}(b_{0;0}),\dots ,\text{sign}({\beta }_{s_n;0})\}}^{\text{T}}$.

For testing hypotheses of the form (10), the generalized Wald-type test statistic W_n proposed in § 3.3 continues to be applicable. Theorem 7 derives the asymptotic distribution of W_n.

Theorem 7 (Wald-type test under H₀). Assume Conditions A1, A2, B3, C4, B5, A6–A7 in the Appendix, and that $w_{\text{max}}^{(\text{I})}=o_P[1/\{{\lambda }_n{(n{s}_n)}^{1/2}\}]$. If $s_n^5/n\to 0$ and min_{j=1,..., sn} |β_j;0|/(s_n/n)^1/2 → ∞ as n → ∞, then under H₀ in (10), $W_n\to {\chi }_k^2$ in distribution.

4.2. Weight selection

We propose a penalized componentwise regression method for selecting weights by

$${\widehat{w}}_j={|{\widehat{\beta }}_j^{\text{PCR}}|}^{-1}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left(j=1,\dots ,p_n\right),$$ (13)

based on some initial estimator, ${\widehat{\beta }}^{\text{PCR}}={({\widehat{\beta }}_1^{\text{PCR}},\dots ,{\widehat{\beta }}_{p_n}^{\text{PCR}})}^{\text{T}}$, minimizing

$${\ell }_n^{\text{PCR}}(\beta )=\frac{1}{n}\sum _{i=1}^n\sum _{j=1}^{p_n}Q\{Y_i,F^{-1}(X_{ij}{\beta }_j)\}+{\kappa }_n\sum _{j=1}^{p_n}|{\beta }_j|,$$ (14)

with some sequence κ_n > 0. Theorem 8 indicates that under assumptions on the correlation between the predictor variables and the response variable, the weights selected by the penalized componentwise regression satisfy the conditions in Theorem 5.

Theorem 8 (Penalized componentwise regression for weights: p_n ≈n). Assume Conditions A1, A2, B3, A4, A6, A7 and E. Assume that in Condition E, 𝒜_n = λ_nn^1/2, 𝒜_n/κ_n → ∞ and 𝒝_n/κ_n = O(1) for κ_n in (14). Suppose λ_nn^1/2 = O(1), λ_n = o(κ_n) and $\text{log}(p_n)=o(n{\kappa }_n^2)$. Assume that E(X) = 0 in model (4). Then there exist local minimizers ${\widehat{\beta }}_j^{\text{PCR}}$ (j = 1, …, p_n), of (14) such that the weights ŵ_j (j = 1, …, p_n), defined in (13) satisfy that ${\widehat{w}}_{\text{max}}^{(\text{I})}=O_P\{1/{\lambda }_n\surd n\}$ and ${\widehat{w}}_{\text{min}}^{(\text{II})}{\lambda }_n\to \infty$ in probability in Theorem 5, where ${\widehat{w}}_{\text{max}}^{(\text{I})}={\text{max}}_{j=1,\dots ,s_n}{\widehat{w}}_j$ and ${\widehat{w}}_{\text{min}}^{(\text{II})}={\text{min}}_{s_n+1⩽j⩽p_n}{\widehat{w}}_j$.

5. Consistency of the penalized Bregman divergence classifier

This section deals with the binary response variable Y, which takes values 0 and 1. In this case, the mean regression function m(x) in (4) becomes the class label probability, pr(Y = 1 | X = x). From the penalized Bregman divergence estimator (b̂₀, β̂^T)^T proposed in either § 3 or § 4, we can construct the penalized Bregman divergence classifier, ϕ̂ (x) = I {m̂(x) > 1/2}, for a future input variable x, where m̂(x) = F⁻¹(b̂₀ + x^Tβ̂).

In the classification literature, the misclassification loss of a classification rule ϕ at a sample point (x, y) is l{y, ϕ(x)} = I {y ≠ϕ(x)}. The risk of ϕ is the expected misclassification loss, R(ϕ) = E[l{Y, ϕ(X)}] = pr{ϕ(X) ≠ Y}. The optimal Bayes rule, which minimizes the risk with respect to ϕ, is ϕ_B(x) = I {m(x) > 1/2}. For a test sample (X^o, Y^o), which is an independent and identically distributed copy of samples in the training set 𝒯_n = {(X_i, Y_i), i = 1, …, n}, the optimal Bayes risk is then R(ϕ_B) = pr{ϕ_B(X^o) ≠Y^o}. Meanwhile, the conditional risk of the penalized Bregman divergence classification rule ϕ̂ is R(ϕ̂) = pr{ϕ̂(X^o) ≠ Y^o | 𝒯_n}. For ϕ̂ induced by the penalized Bregman divergence regression estimation using a range of loss functions combined with either the smoothly clipped absolute deviation, L₁ or weighted L₁-penalties, Theorem 9 verifies the classification consistency attained by ϕ̂.

Theorem 9 (Consistency of the penalized Bregman divergence classifier). Assume Conditions A1 and A4 in the Appendix. Suppose that ||β̃_E – β̃ ₀|| = O_P (r_n). If $r_n{p}_n^{1/2}=o(1)$, then the classification rule ϕ̂ constructed from β̃_E is consistent in the sense that E { R(ϕ̂)} – R(ϕ_B) → 0 as n → ∞.

6. Simulation study

6.1. Set-up

For illustrative purposes, four procedures for penalized estimators are compared: (I) the smoothly clipped absolute deviation penalty, with an accompanying parameter a = 3.7, combined with the local linear approximation; (II) the L₁ penalty; (III) the weighted L₁-penalties with weights selected by (13); and (IV) the oracle estimator using the set of significant variables. Throughout the numerical work in the paper, methods (I)–(III) utilize the least angle regression algorithm, F is the log link for count data and the logit link for binary response variables.

6.2. Penalized quasilikelihood for overdispersed count data

A quasilikelihood function Q relaxes the distributional assumption on a random variable Y via the specification ∂Q(Y, μ)/∂μ = (Y – μ)/ V(μ), where var(Y | X = x) = V {m(x)} for a known continuous function V (·) > 0. Zhang et al. (2009) verified that the quasilikelihood function belongs to the Bregman divergence and derived the generating q function,

$$q(\mu )={\int }_{-\infty }^{\mu }\frac{s-\mu }{V(s)}ds.$$ (15)

We generate overdispersed Poisson counts Y_i satisfying var(Y_i | X_i = x_i) = 2m(x_i). In the predictor X_i = (X_i1, …, X_ipn)^T, p_n = n/8, n/2 and n – 10, and X_i1 = i/n – 0.5. For j = 2, …, p_n, X_ij = Φ(Z_ij) – 0.5, where Φ is the standard normal distribution function, and (Z_i2, …, Z_ipn)^T ∼ N{0, $\rho 1_{{\rho }_n-1}{1}_{p_n-1}^{\text{T}}+\left(1-\rho \right){\text{I}}_{{\rho }_n-1}$}, with 1_d a d × 1 vector of ones and I_d a d × d identity matrix. Thus (X_i2, …, X_ipn) are correlated Un(0, 1) if ρ ≠ 0. The link function is log{m(x)} = b_0;0 + x^Tβ₀, where b_0;0 = 5 and β₀ = (2, 2, 0, 0, …, 0)^T.

First, to examine the effect of penalized regression estimates on model fitting, we generate 200 training sets of size n. For each training set, the model error is calculated by ${\sum }_{l=1}^L{\{\widehat{m}(x_l)-m(x_l)\}}^2/L$, at a randomly generated sequence ${\{x_l\}}_{l=1}^{L=5000}$, and the relative model error is the ratio of model error using penalized estimators and that using nonpenalized estimators. The tuning constants λ_n for the training set in each simulation for methods (I)–(II) are selected separately by minimizing the quasilikelihood on a test set of size equal to that of the training set; λ_n and κ_n for method (III) are searched on a surface of grid points. The mean relative model error can be obtained from those 200 training sets. Table 1 summarizes the penalized quasilikelihood estimates of parameters by means of (15). It is clearly seen that if the true model coefficients are sparse, the penalized estimators reduce the function estimation error compared with the nonpenalized estimators.

Table 1.

Simulation results from penalized quasilikelihood estimates, with dependent predictors. n = 200, ρ = 0.2

Loss	pn	Method	Regression	Variable selection
			MRME	CZ (sd)	IZ (sd)
Quasilikelihood	n/8	SCAD	0.2428	17.74 (5.46)	0 (0)
		L1	0.3503	14.21 (4.91)	0 (0)
		Weighted L1	0.1077	21.32 (2.48)	0 (0)
		Oracle	0.0861	23	–
Quasilikelihood	n/2	SCAD	0.0409	91.73 (12.56)	0 (0)
		L1	0.0712	88.00 (14.94)	0 (0)
		Weighted L1	0.0161	94.84 (5.89)	0 (0)
		Oracle	0.0105	98	–
Quasilikelihood	n – 10	SCAD	0.0010	184.37 (8.52)	0 (0)
		L1	0.0019	181.13 (13.87)	0 (0)
		Weighted L1	0.0004	185.25 (4.97)	0 (0)
		Oracle	0.0002	188	–

SCAD, smoothly clipped absolute deviation; MRME, mean of relative model errors obtained from the training sets; CZ, average number of coefficients that are correctly estimated to be zero when the true coefficients are zero; IZ, average number of coefficients that are incorrectly estimated to be zero when the true coefficients are nonzero; sd, standard deviation.

Second, to study the utility of penalized estimators in revealing the effects in variable selection under quasilikelihood, Table 1 gives the average number of coefficients that are correctly estimated to be zero when the true coefficients are zero, and the average number of coefficients that are incorrectly estimated to be zero when the true coefficients are nonzero. The standard deviations of the corresponding estimations across 200 training sets are given in brackets. Overall, the penalized estimators help yield a sparse solution and build a sparse model. These results lend support to the theoretical results in § 3 and § 4.

In summary, the smoothly clipped absolute deviation and weighted L₁ penalties outperform the L₁ penalty in terms of regression estimation and variable selection. As expected, the oracle estimator, which is practically infeasible, performs better than the three penalized estimators.

6.3. Penalized Bregman divergence for binary classification

We generate data with two-classes from the model,

$$X={\left(X_1,\dots ,X_{p_n}\right)}^{\text{T}}~N(0,\mathrm{\Sigma }),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}Y|X=x~\text{Ber}\{m(x)\},$$

where p_n = n/8, n/2, n – 10, $\mathrm{\Sigma }=\rho 1_{{\rho }_n}{1}_{p_n}^{\text{T}}+\left(1-\rho \right){\text{I}}_{{\rho }_n}$ and logit{m(x)} = b_0;0 + x^T β₀ with b_0;0 = 3 and β₀ = (1.5, 2, −2, −2.5, 0, 0, …, 0)^T. Table 2 summarizes the penalized estimates of parameters. The results reinforce the conclusion drawn in § 6.2.

Table 2.

Simulation results from penalized Bregman divergence estimates for binary classification, with dependent predictors. n = 200, ρ = 0.2

Loss	pn	Method	Regression	Variable selection		Classification
			MRME	CZ (sd)	IZ (sd)	MAMR
Deviance	n/8	SCAD	0.2504	18.86 (4.37)	0.01 (0.10)	0.1153
		L1	0.3774	11.31 (5.48)	0.00 (0.00)	0.1218
		Weighted L1	0.2409	18.11 (2.26)	0.01 (0.10)	0.1160
		Oracle	0.1164	21	0	0.1042
Exponential	n/8	SCAD	0.2566	18.92 (4.13)	0.00 (0.00)	0.1162
		L1	0.3356	12.28 (5.54)	0.00 (0.00)	0.1232
		Weighted L1	0.2176	19.07 (1.66)	0.01 (0.10)	0.1175
		Oracle	0.1276	21	0	0.1042
Deviance	n/2	SCAD	0.0612	94.74 (2.32)	0.03 (0.17)	0.1166
		L1	0.1148	76.39 (12.97)	0.00 (0.00)	0.1313
		Weighted L1	0.0782	89.00 (6.38)	0.04 (0.19)	0.1235
		Oracle	0.0240	96	0	0.1043
Exponential	n/2	SCAD	0.0915	94.37 (2.91)	0.05 (0.21)	0.1209
		L1	0.1141	76.05 (11.99)	0.00 (0.00)	0.1315
		Weighted L1	0.0723	90.60 (4.70)	0.04 (0.19)	0.1222
		Oracle	0.0310	96	0	0.1043
Deviance	n – 10	SCAD	0.0230	185.09 (1.53)	0.02 (0.14)	0.1136
		L1	0.0847	158.19 (17.26)	0.00 (0.00)	0.1401
		Weighted L1	0.0539	176.51 (8.17)	0.03 (0.17)	0.1273
		Oracle	0.0121	186	0	0.1044
Exponential	n – 10	SCAD	0.0360	184.62 (2.20)	0.01 (0.10)	0.1170
		L1	0.0746	161.15 (14.73)	0.00 (0.00)	0.1386
		Weighted L1	0.0489	178.70 (5.91)	0.04 (0.19)	0.1271
		Oracle	0.0150	186	0	0.1044

MAMR, mean of the average misclassification rates calculated from training sets.

Moreover, to investigate the performance of penalized classifiers, we evaluate the average misclassification rate for 10 independent test sets of size 10 000. Table 2 reports the mean of the average misclassification rates calculated from 100 training sets. Evidently, all penalized classifiers perform as well as the optimal Bayes classifier. This agrees with results of Theorem 9 on the asymptotic classification consistency. Furthermore, the choice of loss functions in the penalized classifiers has an asymptotically relatively negligible impact on classification performance.

7. Real data

The Arrhythmia dataset (Güvenir et al., 1997) consists of 452 patient records in the diagnosis of cardiac arrhythmia. Each record contains 279 clinical measurements, from electrocardiography signals and other information such as sex, age and weight, along with the decision of an expert cardiologist. In the data, class 01 refers to normal electrocardiography, class 02–class 15 each refers to a particular type of arrhythmia, and class 16 refers to the unclassified remainder.

We intend to predict whether a patient can be categorized as having normal electrocardiography or not. After deleting missing values and class 16, the remaining 430 patients with 257 attributes are used in the classification. To evaluate the performance of the penalized estimates of model parameters in $\text{logit}\{\text{pr}\left(Y=1|X_1,\dots ,X_{257}\right)\}=b_0+{\sum }_{j=1}^{257}{\beta }_j{X}_j$, we randomly split the data into a training set and a test set in the ratio 2:1. For each training set, the tuning constant is selected by minimizing a 3-fold crossvalidated estimate of the misclassification rate; λ_n and κ_n for the penalized componentwise regression are found on a grid of points. We calculate the mean of the misclassification rates and the average number of selected variables over 100 random splittings. It is seen from Table 3 that the penalized classifier using the deviance loss and that using the exponential loss have similar values of misclassification rates. In contrast, the nonpenalized classifiers select all attributes, yielding much higher misclassification rates.

Table 3.

Arrhythmia data: mean misclassification rate and the average number of selected variables

Loss	Method	MMR	# Selected variables
Deviance	Nonpenalized	0.4265	257.00
	SCAD	0.2550	16.13
	L1	0.2358	45.46
	Weighted L1	0.2340	26.44
Exponential	Nonpenalized	0.4323	257.00
	SCAD	0.2666	15.83
	L1	0.2397	43.79
	Weighted L1	0.2366	18.77

MMR, mean of the misclassification rates.

Appendix

For a matrix M, its eigenvalues, minimum eigenvalue, maximum eigenvalue and trace are labelled by λ_j (M), λ_min(M), λ_max(M) and tr(M), respectively. Let ‖M‖ = sup_‖x‖=1 ‖Mx‖ = {λ_max(M^TM)}^1/2 be the matrix L₂ norm; let ‖M‖_F = {tr(M^TM)}^1/2 be the Frobenius norm. See Golub & Van Loan (1996) for details. Throughout the proof, C is used as a generic finite constant.

We first impose some regularity conditions, which are not the weakest possible.

Condition A consists of the following.

• A1. Assume ${\text{sup}}_{n⩾1}{\Vert {\tilde{\beta }}_0^{(\text{I})}\Vert }_1<\infty$ and ‖X‖_∞ is bounded;

• A2. the matrix E(X̃X̃^T) exists and is nonsingular;

• A3. assume E(Y²) < ∞;

• A4. there is a large enough open subset of 𝕉^{p_n +1}, which contains the true parameter point β̃₀, such that F⁻¹(X̃^Tβ̃) is bounded for all β̃ in the subset;

• A5. the eigenvalues of the matrix −E(q⁽²⁾{m(X)}/[F⁽¹⁾{m(X)}]²X̃X̃^T) are uniformly bounded away from 0;

• A6. the function q⁽⁴⁾(·) is continuous, and q⁽²⁾(·) < 0;

• A7. the function F(·) is a bijection, F⁽³⁾(·) is continuous and F⁽¹⁾(·) ≠0; and finally

• A8. assume P_λn (0) = 0. There are constants C and D such that when θ₁ > Cλ_n and θ₂ > Cλ_n, |P″_λn (θ₁) − P″_λn (θ₂)| ⩽ D|θ₁ − θ₂|.

Condition B: These are identical to Condition A except that A3 and A5 are replaced by B3 and B5:

• B3. there exists a constant C ∈ (0, ∞) such that E{|Y − m(X)|^j} ⩽ j!C^j for all j ⩾ 3. Also, inf_{n ⩾ 1, 1 ⩽ j ⩽ pn} $E\{\text{var}\left(Y|X\right)X_j^2\}>0$; and

• B5. assume λ_j (Ω_n) and λ_j (H_n) are uniformly bounded away from 0; $\Vert H_n^{-1}{\mathrm{\Omega }}_n\Vert$ is bounded away from ∞.

Condition C: These are identical to Condition B except that B4 is replaced by:

• C4. there is an open subset of 𝕉^{p_n +1} which contains the true parameter point β̃₀, such that F⁻¹(X̃^Tβ̃) is bounded for all β̃ in the subset. Moreover, the subset contains the origin.

Condition D: This is identical to Condition C except that C5 is replaced by:

• D5. assume λ_j (H_n) are uniformly bounded away from 0; $\Vert H_n^{-1/2}{\mathrm{\Omega }}_n^{1/2}\Vert$ is bounded away from ∞.

Condition E is as follows.

E1. Assume min_{j=1,..., sn} |E(X_jY)| ≽ 𝒜_n and max_{sn+1 ⩽ j ⩽ pn} |E(X_j Y)| = o(𝒝_n) for some positive sequences 𝒜_n and 𝒝_n, where s_n ≽ t_n, for two nonnegative sequences s_n and t_n, denotes that there exists a constant c > 0 such that s_n ⩾ c t_n for all n ⩾ 1.

Proof of Theorem 1. Let r_n = (p_n/n)^1/2 and ũ = (u₀, u₁, …, u_pn)^T ∈ 𝕉^p_n+1. Similar to Fan & Peng (2004), it suffices to show that for any given ∊ > 0, there is a large constant C_∊ such that, for large n,

$$\text{pr}\left\{\underset{\Vert \tilde{u}\Vert =C_{∊}}{\text{inf}}{\ell }_n\left({\tilde{\beta }}_0+r_n\tilde{u}\right)>{\ell }_n({\tilde{\beta }}_0)\right\}⩾1-∊.$$ (A1)

Define β̃_L = β̃₀ + r_nũ. To show (A1), consider

$$\begin{array}{lll}{D}_n(\tilde{u})\hfill & =\hfill & \frac{1}{n}\sum _{i=1}^n\left[Q\{Y_i,F^{-1}\left({\tilde{X}}_i^{\text{T}}{\tilde{\beta }}_L\right)\}-Q\{Y_i,F^{-1}\left({\tilde{X}}_i^{\text{T}}{\tilde{\beta }}_0\right)\}\right]\hfill \\ \hfill & \hfill & +\sum _{j=1}^{p_n}\{P_{{\lambda }_n}\left(|{\beta }_{j;0}+r_n{u}_j|\right)-P_{{\lambda }_n}\left(|{\beta }_{j;0}|\right)\}\equiv I_1+I_2.\hfill \end{array}$$ (A2)

First, we consider I₁. For μ = F⁻¹(θ), obtain q_j (y; θ) (j = 1, 2, 3), from (7). By Taylor’s expansion,

$$I_1=I_{1,1}+I_{1,2}+I_{1,3},$$ (A3)

where $I_{1,1}=r_n/n{\sum }_{i=1}^n{q}_1\left(Y_i;{\tilde{X}}_i^{\text{T}}{\tilde{\beta }}_0\right){\tilde{X}}_i^{\text{T}}\tilde{u}$, $I_{1,2}=r_n^2/(2n){\sum }_{i=1}^n{q}_2\left(Y_i;{\tilde{X}}_i^{\text{T}}{\tilde{\beta }}_0\right){\left({\tilde{X}}_i^{\text{T}}\tilde{u}\right)}^2$ and $I_{1,3}=r_n^3/(6n){\sum }_{i=1}^n{q}_3\left(Y_i;{\tilde{X}}_i^{\text{T}}{\tilde{\beta }}^{*}\right){\left({\tilde{X}}_i^{\text{T}}\tilde{u}\right)}^3$ for β̃^* located between β̃₀ and β̃₀ + r_nũ. Hence |I_1,1| ⩽ O_P {r_n(p_n/n)^1/2}‖ũ‖ and $I_{1,2}=-\left(r_n^2/2\right){\tilde{u}}^{\text{T}}E\left(q^{(2)}\{m(X)\}/{\left[F^{(1)}\{m(X)\}\right]}^2\tilde{X}{\tilde{X}}^{\text{T}}\right)\tilde{u}+O_P\left(r_n^2{p}_n/n^{1/2}\right){\Vert \tilde{u}\Vert }^2$. Conditions A1 and A4 give $|I_{1,3}|⩽O_P\left(r_n^3{p}_n^{3/2}\right){\Vert \tilde{u}\Vert }^3$.

Next, we consider I₂. By Taylor’s expansion, $I_2⩾r_n{\sum }_{j=1}^{s_n}{P}_{{\lambda }_n}^{\text{'}}\left(|{\beta }_{j;0}|\right)\text{sign}\left({\beta }_{j;0}\right)u_j+\left(r_n^2/2\right){\sum }_{j=1}^{s_n}{P}_{{\lambda }_n}^{(2)}\left(|{\beta }_j^{*}|\right)u_j^2\equiv I_{2,1}+I_{2,2}$, for ${\beta }_j^{*}$ between β_j;0 and β_j;0 + r_nu_j. Thus, |I_2,1| ⩽ r_na_n‖u^(I)‖₁ and $|I_{2,2}|⩽r_n^2{b}_n{\Vert u^{(\text{I})}\Vert }^2+D{r}_n^3{\Vert u^{(\text{I})}\Vert }^3$, where u^(I) = (u₁, …, u_sn)^T. Since $p_n^4/n\to 0$, we can choose some large C_∊ such that I_1,1, I_1,3, I_2,1 and I_2,2 are all dominated by I_1,2, which is positive by Condition A5. This implies (A1).

Proof of Lemma 1. Analogous to the proof of Theorem 1, it suffices to show (A1). Note that (A2) continues to hold with $I_2\equiv {\lambda }_n{\sum }_{j=1}^{p_n}{w}_j\left(|{\beta }_{j;0}+r_n{u}_j|-|{\beta }_{j;0}|\right)$ and I₁ is unchanged. Clearly, $I_2⩾-{\lambda }_n{r}_n{\sum }_{j=1}^{s_n}{w}_j|u_j|\equiv I_{2,1}$, in which $|I_{2,1}|⩽{\lambda }_n{r}_n{w}_{\text{max}}^{(\text{I})}{\Vert u^{(\text{I})}\Vert }_1$. The rest of the proof resembles that of Theorem 1 and is omitted.

Proof of Theorem 5. Write ũ = {ũ^(I)T, u^(II)T}^T, where ũ^(I) = (u₀, u₁, …, u_sn)^T and u^(II) = (u_sn+1, …, u_pn)^T. Following the proof of Lemma 1, it suffices to show (A1) for r_n = (s_n/n)^1/2.

For I_1,1 in (A3), $I_{1,1}=I_{1,1}^{(\text{I})}+I_{1,1}^{(\text{II})}$ according to ũ^(I) and u^(II). It follows that $|I_{1,1}^{(\text{I})}|⩽r_n{O}_P\{{\left(s_n/n\right)}^{1/2}\}{\Vert {\tilde{u}}^{(\text{I})}\Vert }_2$ and $|I_{1,1}^{(\text{I})}|⩽r_n{O}_P\left(1/n^{1/2}\right){\Vert u^{(\text{II})}\Vert }_1$.

For I_1,2 in (A3), similar to the proof of Theorem 1, I_1,2 = I_1,2,1 + I_1,2,2. Define d_i = q⁽²⁾{m(X_i)}/[F⁽¹⁾{m(X_i)}]². This yields

$$I_{1,2,1}⩾-\frac{r_n^2}{2n}\sum _{i=1}^n{d}_i{\left(X_i^{{(\text{I})}_{\text{T}}}{\tilde{u}}^{(\text{I})}\right)}^2-\frac{r_n^2}{n}\sum _{i=1}^n{d}_i\left(X_i^{{(\text{I})}_{\text{T}}}{\tilde{u}}^{(\text{I})}\right)\left(X_i^{{(\text{II})}_{\text{T}}}{u}^{(\text{II})}\right)=I_{1,2,1}^{(\text{I})}-I_{1,2,1}^{(\text{cross})}.$$

Then there exists a constant C > 0 such that $I_{1,2,1}^{(\text{I})}⩾C{r}_n^2\{1+o_P(1)\}{\Vert {\tilde{u}}^{(\text{I})}\Vert }_2^2$ and $|I_{1,2,1}^{(\text{cross})}|⩽O_P\left(r_n^2{s}_n^{1/2}\right){\Vert {\tilde{u}}^{(\text{I})}\Vert }_2\cdot {\Vert u^{(\text{II})}\Vert }_1$. For I_1,2,2, partitioning ũ into ũ^(I) and u^(II) gives

$$I_{1,2,2}\equiv I_{1,2,2}^{(\text{I})}+I_{1,2,2}^{(\text{cross})}+I_{1,2,2}^{(\text{II})},$$

where $|I_{1,2,2}^{(\text{I})}|⩽r_n^2{O}_P\left(s_n/n^{1/2}\right){\Vert {\tilde{u}}^{(\text{I})}\Vert }_2^2$, $|I_{1,2,2}^{(\text{cross})}|⩽r_n^2{O}_P\{{\left(s_n/n\right)}^{1/2}\}{\Vert {\tilde{u}}^{(\text{I})}\Vert }_2{\Vert u^{(\text{II})}\Vert }_1$ and $|I_{1,2,2}^{(\text{II})}|⩽r_n^2{O}_P\left(n^{-1/2}\right){\Vert u^{(\text{II})}\Vert }_1^2$.

For I_1,3 in (A3), since s_np_n = o(n), ‖β̃^*‖₁ is bounded and thus $|I_{1,3}|⩽O_P\left(r_n^3\right){\Vert {\tilde{u}}^{(\text{I})}\Vert }_1^3+O_P\left(r_n^3\right){\Vert u^{(\text{II})}\Vert }_1^3\equiv I_{1,3}^{(\text{I})}+I_{1,3}^{(\text{II})}$, where $|I_{1,3}^{(\text{I})}|⩽O_P\left(r_n^3{s_n}^{3/2}\right){\Vert {\tilde{u}}^{(\text{I})}\Vert }_2^3$ and $|I_{1,3}^{(\text{II})}|⩽O_P\left(r_n^3\right){\Vert u^{(\text{II})}\Vert }_1^3$.

For I₂ in (A2), $I_2⩾I_{2,1}^{(\text{I})}+I_{2,1}^{(\text{II})}$, where $I_{2,1}^{(\text{I})}=-{\lambda }_n{r}_n{\sum }_{j=1}^{s_n}{w}_j|u_j|$ and $I_{2,1}^{(\text{II})}={\lambda }_n{r}_n{\sum }_{j=s_n+1}^{p_n}{w}_j|u_j|$. Hence $|I_{2,1}^{(\text{I})}|⩽{\lambda }_n{r}_n{w}_{\text{max}}^{(\text{I})}{s}_n^{1/2}{\Vert u^{(\text{I})}\Vert }_2$ and $I_{2,1}^{(\text{II})}⩾{\lambda }_n{r}_n{w}_{\text{min}}^{(\text{II})}{\Vert u^{(\text{II})}\Vert }_1$.

It can be shown that either $I_{1,2,1}^{(\text{I})}$ or $I_{2,1}^{(\text{II})}$ dominates all other terms in groups, ${𝒢}_1=\left(I_{1,2,2}^{(\text{I})},I_{1,3}^{(\text{I})}\right)$, ${𝒢}_2=\left(I_{1,1}^{(\text{II})},I_{1,2,2}^{(\text{II})},I_{1,3}^{(\text{II})},I_{1,2,1}^{(\text{cross})},I_{1,2,2}^{(\text{cross})}\right)$ and ${𝒢}_3=\left(I_{1,1}^{(\text{I})},I_{2,1}^{(\text{I})}\right)$. Namely, $I_{1,2,1}^{(\text{I})}$ dominates 𝒢₁, and $I_{2,1}^{(\text{II})}$ dominates 𝒢₂. For 𝒢₃, if ‖u^(II)‖₁ ⩽ C_∊/2, then 𝒢₃ is dominated by $I_{1,2,1}^{(\text{I})}$, which is positive; if ‖u^(II)‖₁ > C_∊ /2, then 𝒢₃ is dominated by $I_{2,1}^{(\text{II})}$, which is positive.

Proof of Theorem 8. Minimizing (14) is equivalent to minimizing ${\ell }_{n,j}^{\text{PCR}}(\alpha )=n^{-1}{\sum }_{i=1}^{n}Q\{Y_i,F^{-1}\left(X_{ij}\alpha \right)\}+{\kappa }_n|\alpha |$, for j = 1, …, p_n. The proof may be separated into two parts.

Part 1. To show ${\widehat{w}}_{\text{max}}^{(\text{I})}=O_P\{1/\left({\lambda }_n{n}^{1/2}\right)\}$, it suffices to show that for 𝒜_n = λ_nn^1/2, there exist local minimizers ${\widehat{\beta }}_j^{\text{PCR}}$ of ${\ell }_{n,j}^{\text{PCR}}(\alpha )$ such that ${\text{lim}}_{\delta \to 0}{\text{inf}}_{n⩾1}\text{pr}\left({\text{min}}_{1⩽j⩽s_n}|{\widehat{\beta }}_j^{\text{PCR}}|>{𝒜}_n\delta \right)=1$. It suffices to prove that for j = 1, …, s_n there exist some b_j with |b_j | = 2δ such that

$$\underset{\delta \to 0}{\text{lim}}\underset{n⩾1}{\text{inf}}\text{pr}\left[\underset{1⩽j⩽s_n}{\text{min}}\left\{\underset{|\alpha |⩽\delta }{\text{inf}}{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n\alpha \right)-{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n{b}_j\right)\right\}>0\right]=1,$$ (A4)

and there exists some large enough C_n > 0 such that

$$\underset{\delta \to 0}{\text{lim}}\underset{n⩾1}{\text{inf}}\text{pr}\left[\underset{1⩽j⩽s_n}{\text{min}}\left\{\underset{|\alpha |⩾C_n}{\text{inf}}{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n\alpha \right)-{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n{b}_j\right)\right\}>0\right]=1.$$ (A5)

Note that (A5) holds, since for every n ⩾ 1, when |α| → ∞, ${\text{min}}_{1⩽j⩽s_n}\left\{{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n\alpha \right)-{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n{b}_j\right)\right\}⩾{\kappa }_n{𝒜}_n|\alpha |-{\text{max}}_{j=1,\dots ,s_n}{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n{b}_j\right)\to \infty$ in probability. To prove (A4), note that |𝒜_nα| ⩽ 𝒜_nδ = O(1)δ → 0 as δ ↓ 0. By Taylor’s expansion,

$$\begin{array}{l}\underset{j=1,\dots ,s_n}{\text{min}}\left\{\underset{|\alpha |⩽\delta }{\text{inf}}{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n\alpha \right)-{\ell }_{n,j}^{\text{PCR}}\left({𝒜}_n{b}_j\right)\right\}⩾{𝒜}_n\underset{j=1,\dots ,s_n}{\text{min}}\underset{|\alpha |⩽\delta }{\text{inf}}\left\{\left(\alpha -b_j\right)\frac{1}{n}\sum _{i=1}^n{q}_1\left(Y_i;0\right)X_{ij}\right\}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\frac{{𝒜}_n^2}{2}\underset{j=1,\dots ,s_n}{\text{min}}\underset{|\alpha |⩽\delta }{\text{inf}}\left\{{\alpha }^2\frac{1}{n}\sum _{i=1}^n{q}_2\left(Y_i;X_{ij}{𝒜}_n{\alpha }_j^{*}\right)X_{ij}^2-b_j^2\frac{1}{n}\sum _{i=1}^n{q}_2\left(Y_i;X_{ij}{𝒜}_n{b}_j^{*}\right)X_{ij}^2\right\}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{𝒜}_n\underset{1⩽j⩽s_n}{\text{min}}\underset{|\alpha |⩽\delta }{\text{inf}}\left\{{\kappa }_n\left(|\alpha |-|b_j|\right)\right\}\\ \equiv I_1+I_2+I_3,\end{array}$$

with ${\alpha }_j^{*}$ between 0 and α and $b_j^{*}$ between 0 and b_j. Let μ₀ = F⁻¹(0) and C₀ = q″(μ₀)/ F′(μ₀) ≠ 0. Then

$$\begin{array}{lll}{I}_1\hfill & =\hfill & {𝒜}_n\underset{j=1,\dots ,s_n}{\text{min}}\underset{|\alpha |⩽\delta }{\text{inf}}\{C_0(\alpha -b_j)E(Y{X}_j)\}+{𝒜}_n\underset{j=1,\dots ,s_n}{\text{min}}\underset{|\alpha |⩽\delta }{\text{inf}}\left[C_0(\alpha -b_j)\frac{1}{n}\sum _{i=1}^n\{Y_i{X}_{ij}-E(Y{X}_j)\}\right]\hfill \\ \hfill & \hfill & -{𝒜}_n\underset{1⩽j⩽s_n}{\text{max}}\underset{|\alpha |⩽\delta }{\text{sup}}\left\{C_0{\mu }_0(\alpha -b_j)\frac{1}{n}\sum _{i=1}^n{X}_{ij}\right\}\hfill \\ \hfill & \equiv \hfill & I_{1,1}+I_{1,2}+I_{1,3}.\hfill \end{array}$$

We see that |I_1,3| ⩽ O_P [𝒜_n{log(s_n)/ n}^1/2]δ, by Bernstein’s inequality (Lemma 2.2.11 in van der Vaart & Wellner 1996). Again |I_1,2| = O_P [𝒜_n{log(s_n)/ n}^1/2]δ by an argument similar to that of Theorem 2. Choosing b_j = −2δsign{C₀ E(Y X_j)}, which satisfies |b_j| =2δ, gives $I_{1,1}⩾|C_0{|}_c{𝒜}_n^2\delta$. For I₂ and I₃, we observe that $|I_2|⩽|O_P({𝒜}_n^2){\delta }^2$ and |I₃| = O(𝒜_nκ_n)δ.By the assumptions, we can choose a small enough δ > 0 such that with probability tending to 1, I_1,2, I_1,3, I₂ and I₃ are dominated by I_1,1, which is positive. Thus (A4) is proved.

Part 2. To verify that ${\widehat{w}}_{\text{min}}^{(\text{II})}{\lambda }_n\to \infty$ in probability, it suffices to prove that for any ∊ > 0, there exist local minimizers ${\widehat{\beta }}_j^{\text{PCR}}$ of ${\ell }_{n,j}^{\text{PCR}}(\alpha )$ such that lim_n→∞ pr(max_{sn+1 ⩽ j ⩽ pn}| ${\widehat{\beta }}_j^{\text{PCR}}$| ⩽ λ_n∊) = 1. Similar to the proof of Theorem 1, we will prove that for any ∊ > 0,

$$\underset{n\to \infty }{\text{lim}}\text{pr}\left[\underset{j=s_n+1,\dots ,p_n}{\text{min}}\left\{\underset{|\alpha |=∊}{\text{inf}}{\ell }_{n,j}^{\text{PCR}}({\lambda }_n\alpha )-{\ell }_{n,j}^{\text{PCR}}(0)\right\}>0\right]=1.$$ (A6)

For j = 1, …, s_n, by Taylor’s expansion,

$$\begin{array}{l}\underset{j=s_n+1,\dots ,p_n}{\text{min}}\left\{\underset{|\alpha |=∊}{\text{inf}}{\ell }_{n,j}^{\text{PCR}}({\lambda }_n\alpha )-{\ell }_{n,j}^{\text{PCR}}(0)\right\}⩾{\lambda }_n\underset{j=s_n+1,\dots ,p_n}{\text{min}}\underset{|\alpha |=∊}{\text{inf}}\left\{\alpha \frac{1}{n}\sum _{i=1}^n{q}_1(Y_i;0)X_{ij}\right\}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\frac{{\lambda }_n^2}{2}\underset{j=s_n+1,\dots ,p_n}{\text{min}}\underset{|\alpha |=∊}{\text{inf}}\left\{{\alpha }^2\frac{1}{n}\sum _{i=1}^n{q}_2(Y_i;X_{ij}{\lambda }_n{\alpha }_j^{*})X_{ij}^2\right\}+{\lambda }_n\underset{|\alpha |=∊}{\text{inf}}({\kappa }_n|\alpha |)\hfill \\ \equiv I_1+I_2+I_3,\hfill \end{array}$$

where ${\alpha }_j^{*}$ is between 0 and α. Similar to the proof in Part 1, |I₁| ⩽ O_P [λ_n{log(p_n – s_n +1)/n}^1/2]∊ + o(λ_n𝒝_n)∊. Note that |I₂| ⩽ O_P ( ${\lambda }_n^2$)∊² and I₃ = λ_nκ_n∊. By assumptions, with probability tending to 1, I₁ and I₂ are dominated by I₃ > 0. So (A6) is proved.

Proof of Theorem 9. We first need to show Lemma A1.

Lemma A1. Suppose that (X^o, Y^o) follows the distribution of (X, Y) and is independent of the training set 𝒯_n. If Q satisfies (3), then E[Q{Y^o, m̂ (X^o)}] = E[Q{Y^o, m(X^o)}] + E[Q{m(X^o), m̂ (X^o)}].

Proof. Let q be the generating function of Q. We deduce from Corollary 3, p. 223 of Chow & Teicher (1988) that E{q(Y^o) | 𝒯_n, X^o} = E{q(Y^o) | X^o} and E[Y^oq′{m̂ (X^o)} | 𝒯_n, X^o] = E(Y ^o | X^o)q′ {m̂ (X^o)} = m(X^o)q′{m̂ (X^o)}.

We now show Theorem 9. Setting Q in Lemma A1 to be the misclassification loss gives

$$\begin{array}{lll}1/2[E\{R(\widehat{\varphi })\}-R{(\varphi )}_{\text{B}})]\hfill & ⩽\hfill & E[|m(X^o)-0.5|I\{m(X^o)⩽0.5,\widehat{m}(X^o)>0.5\}]\hfill \\ \hfill & \hfill & +E[|m(X^o)-0.5|I\{m(X^o)>0.5,\widehat{m}(X^o)⩽0.5\}]\hfill \\ \hfill & \equiv \hfill & I_1+I_2.\hfill \end{array}$$

For any ∊ > 0, I₁ ⩽ pr{|m̂ (X^o) – m(X^o)| > ∊} + ∊ and I₂ ⩽ ∊ + pr{|m̂ (X^o) – m(X^o)| ⩾ ∊}. The proof completes by showing I₁ → 0 and I₂ → 0.

Supplementary material

Supplementary material is available at Biometrika online.