Variable Selection in Measurement Error Models

Summary

Measurement error data or errors-in-variable data are often collected in many studies. Natural criterion functions are often unavailable for general functional measurement error models due to the lack of information on the distribution of the unobservable covariates. Typically, the parameter estimation is via solving estimating equations. In addition, the construction of such estimating equations routinely requires solving integral equations, hence the computation is often much more intensive compared with ordinary regression models. Because of these difficulties, traditional best subset variable selection procedures are not applicable, and in the measurement error model context, variable selection remains an unsolved issue. In this paper, we develop a framework for variable selection in measurement error models via penalized estimating equations. We first propose a class of selection procedures for general parametric measurement error models and for general semiparametric measurement error models, and study the asymptotic properties of the proposed procedures. Then, under certain regularity conditions and with a properly chosen regularization parameter, we demonstrate that the proposed procedure performs as well as an oracle procedure. We assess the finite sample performance via Monte Carlo simulation studies and illustrate the proposed methodology through the empirical analysis of a familiar data set.

1. Introduction

In the regression analysis, some covariates often can only be measured imprecisely or indirectly, thus resulting in the measurement error models, also known as errors-in-variable models in the literature. Various statistical procedures have been developed for statistical inference in measurement error models (Carroll, Ruppert, Stefanski and Crainiceanu, 2006). The study on linear measurement error models dates back to Bickel and Ritov (1987), where an efficient estimator is provided. Stefanski and Carroll (1987) constructed consistent estimators for generalized linear measurement error models. Recently, Tsiatis and Ma (2004) extended the model framework to an arbitrary parametric regression setting. Liang, Härdle and Carroll (1999) proposed partially linear measurement error models. Ma and Carroll (2006) studied generalized partially linear measurement error models. Further active research development has been established recently in the nonparametric measurement error area, see, for example, Delaigle and Hall (2007), and Delaigle and Meister (2007). The goal of this paper is to develop a class of variable selection procedures for general measurement error models. We would emphasize here that the scope of the paper is not limited to generalized linear models.

This study was motivated by examining the effects of systolic blood pressure (SBP), a covariate with error, and effects of three other covariates, respectively serum cholesterol, age and smoking status on the probability of the occurrence of heart disease. In our initial analysis, we include interactions between SBP and covariates, and interactions among covariates and quadratic terms of covariates to reduce modeling bias. It is found in our preliminary analysis that some interactions and quadratic terms are not significant and should be excluded to achieve a parsimonious model. To select significant variables in further analysis, we realized that the traditional AIC and BIC criteria are not well-defined for the model we consider in Section 4.4. Recently, a class of variable selection procedures for partially linear measurement error models via using penalized least squares and penalized quantile regression were proposed in Liang and Li (2009). However, their procedures are not applicable to cases beyond partially linear models, such as partially linear logistic regression models, and therefore, the procedures in Liang and Li (2009) cannot be applied for the model in Section 4.4 either. In fact, variable selection for general parametric or semiparametric measurement error models is challenging. One major difficulty is the lack of a likelihood function in these models, due to the difficulty in obtaining the distribution of the error-prone covariates. For example, using Y to denote the response variable, X to denote the unobservable covariate, and W to denote an observed surrogate of X. The likelihood of a single observation (w, y) is then ∫ p_{Y |X}(y|x, β)p_W|X(w|x)p_X(x)dx. In order to calculate this likelihood, one will need to estimate p_X, yielding a deconvolution problem which is known to have very slow rate (Carroll and Hall, 1988, Fan, 1991) and is typically avoided in parametric measurement error models. Although a reasonable criterion function can be used in place of the likelihood, the difficulty persists in that, except for very special models such as in linear or partially linear cases, even a sensible criterion function is unavailable. In other models such as the ones arise in survival analysis, the lack of a likelihood function also causes a problem. To perform variable selection in these models, rather complicated methods have been proposed where for each potential model, one needs to fit the model, derive the asymptotic properties of the estimator, then form some artificial criterion function based on the asymptotic properties of the estimators and finally add penalty to perform the procedure. The procedure is complicated and unnatural. This motivates us to develop some simple variable selection procedures for measurement error models when a reasonable criterion function is unavailable. Although there exists a few variable selection procedures for linear or partially linear measurement error models (Liang and Li, 2009), to the best of our knowledge, variable selection for general parametric or semiparametric measurement error models has never been systematically studied in the literature. This paper intends to fill this gap by developing a class of variable selection procedures for both parametric and semiparametric measurement error models. In addition, the method proposed here is applicable to the more general situation where likelihood or a natural criterion is not available, and the estimation is performed through solving a set of estimating equations.

The variable selection procedures we propose is indeed a penalized estimating equation method, which can be applied for both parametric and semiparametric measurement error models. In addition, the penalized estimating equation method is applicable to any set of consistent estimating equations. Note that here, the measurement error model we consider is completely general and not limited to generalized linear models. Variable selection and feature selection are very active research topics in the current literature. Candès and Tao (2007) and Fan and Lv (2008) have studied variable selection for linear models when the sample size is much smaller than the dimension of regression parameter space. Their results are inspiring, but only valid for linear models with very strong assumptions on design matrix or the distribution of covariates. Thus, we in this paper follow Fan and Peng (2004), and consider the statistical setting in which the number of regression coefficients diverges to infinity at a certain rate as the sample size tends to infinity. We systematically study the asymptotic properties of the proposed estimator. It is worth pointing out that theoretic results in this paper provide explicit results on the asymptotic properties when the dimension of regression coefficients increases as the sample size increases. This advances the results in current literature, where estimation and inference are studied only for fixed finite dimensional parameter for measurement error models. In our asymptotic analysis, we show that with a proper choice of the regularization parameters and the penalty function, our estimator possesses the oracle property, which roughly means that the estimate is as good as when the true model is known (Fan and Li, 2001). We also demonstrate that the oracle property holds in a simpler form for the more familiar setting where the true number of regression coefficients is fixed.

In addition, we address issues of practical implementation of the proposed methodology. It is desirable to have an automatic, data-driven method to select the regularization parameters. To this end, we propose GCV-type and BIC-type tuning parameter selectors for the proposed penalized estimating equation method. Monte Carlo simulation studies are conducted to assess finite sample performance in terms of model complexity and model error. From our simulation studies, both tuning parameter selectors result in sparse models, while the BIC-type tuning parameter selector outperforms the GCV-type tuning parameter selectors.

The rest of the paper is organized as follows. In Section 2, we propose a new class of variable selection procedures for parametric measurement error models and study asymptotic properties of the proposed procedures. We develop a new variable selection procedure for semiparametric measurement error models in Section 3. Implementation issues and numerical examples are presented in Section 4, where we describe data driven automatic tuning parameter selection methods (Section 4.1), define the concept of approximate model error to evaluate the selected model (Section 4.2), carry out a simulation study to assess the finite sample performance of the proposed procedures (Section 4.3), and illustrate our method in an example (Section 4.4). Technical details are collected in the Appendix.

2. Parametric measurement error models

A general parametric measurement error model has two parts, written as

$$p_{Y\mid X,Z}(Y\mid X,Z,\beta )\text{and}{p}_{W\mid X,Z}(W\mid X,Z,\xi ).$$ (2.1)

Here, the main model is p_{Y |X,Z}(Y|X, Z, β), which denotes the conditional probability density function (pdf) of the response variable Y on the covariates measured with error X and the covariates measured without error Z. Note that here, the conditional distribution of Y on the covariates is completely general hence it includes many familiar regression families. For example, linear model with normal error Y = Xβ + e, logistic model Pr(Y = 1|X) = exp(β₀ + Xβ₁)/{1+exp(β₀ + Xβ₁)} or Poisson model p_{Y |X}(Y |X) = exp{−(β₀ + X^Tβ₁)}(β₀ + X^Tβ₁)^Y/Y ! are all special forms of the model we consider. The error model is denoted p_W|X,Z(W |X, Z, ξ), where W is an observable surrogate of X. Parameter β is a d-dimensional regression coefficient, ξ is a finite dimensional parameter, and our main interest is in selecting the relevant subset of covariates in X and Z and estimating the subsequent parameters contained in β. Typically, ξ is a nuisance parameter, and its estimation usually requires multiple observations or instrumental variables. As routinely done in the literature, we assume that the model is identifiable. Furthermore, for simplicity, we assume in the main context of this paper that the error model p_W|X,Z(W |X, Z) is completely known and hence ξ is suppressed. The extension to the unknown ξ case is rather straightforward and is discussed in Section 5. The observed data is of the form {(W_i, Z_i, Y_i), i = 1, …, n}.

Denote $S_{\beta }^{\ast }$ as the purported score function. That is,

$$S_{\beta }^{\ast }(W,Z,Y)=\frac{\partial log\int p_{W\mid X,Z}(W\mid X,Z)p_{Y\mid X,Z}(Y\mid X,Z)p_{X\mid Z}^{\ast }(X\mid Z)d\mu (X)}{\partial \beta },$$

where $p_{X\mid Z}^{\ast }(X\mid Z)$ is a conditional pdf that one posits, which can be either equal or not equal the true pdf p_X|Z(X|Z). Let function a(X, Z) satisfy

$$E[E^{\ast }\{a(X,Z)\mid W,Z,Y\}\mid X,Z]=E\{S_{\beta }^{\ast }(W,Z,Y)\mid X,Z\},$$

where E^* indicates that the expectation is calculated using the posited $p_{X\mid Z}^{\ast }(X\mid Z)$. Note that here and in the sequel, a model $p_{X\mid Z}^{\ast }(X\mid Z)$ has to be proposed in order to actually construct the estimator. Define

$$S_{\mathit{eff}}^{\ast }(W,Z,Y)=S_{\beta }^{\ast }(W,Z,Y)-{\text{E}}^{\ast }\{a(X,Z)\mid W,Z,Y\}.$$

To select significant variables and estimate the corresponding parameters simultaneously, we propose the penalized estimating equations for model (2.1) as

$$\sum _{i=1}^n{S}_{\mathit{eff}}^{\ast }(W_i,Z_i,Y_i,\beta )-n{\stackrel{.}{p}}_{\lambda }(\beta )=0,$$ (2.2)

where ${\stackrel{.}{p}}_{\lambda }(\beta )={\{p_{\lambda }^{\prime }({\beta }_1),\cdots ,p_{\lambda }^{\prime }({\beta }_d)\}}^T$ and $p_{\lambda }^{\prime }(·)$ is the first order derivative of a penalty function pλ (·). Solving for β̂ from (2.2) gives the estimate of β. In practice, we may allow different coefficients to have penalty functions with different regularization parameters. For example, we may want to keep some variables in the model without penalizing their coefficients. For ease of presentation, we assume that the penalty functions and the regularization parameters are the same for all coefficients in this paper.

The penalties in the classic variable selection criteria, such as AIC and BIC, cannot be applied to the penalized estimating equations. Following the study on the choice of the penalty functions in Fan and Li (2001), we use the SCAD penalty, whose first order derivative is defined as

$$p_{\lambda }^{\prime }(\gamma )=\lambda \left\{I(\mid \gamma \mid \le \lambda )+\frac{{(a\lambda -\mid \gamma \mid )}_{+}}{(a-1)\lambda }I(\mid \gamma \mid >\lambda )\right\}\text{sign}(\gamma )$$ (2.3)

for any scalar γ, where sign(·) is the sign function, i.e., sign(γ) = −1, 0, and 1 when γ < 0, = 0 and > 0 respectively. Here, a > 2 is a constant, and a choice of a = 3.7 is appropriate from a Bayesian point of view. A property of (2.2) is that with a proper choice of penalty functions, such as the SCAD penalty, the resulting estimate contains some exact zero coefficients. This is equivalent to excluding the corresponding variables from the final selected model, thus variable selection is achieved at the same time as parameter estimation.

Concerns about model bias often prompt us to build models that contain many variables, especially when the sample size becomes large. A reasonable way to capture such tendency is to consider the situation where the dimension of the parameter β increases along with the sample size n. We therefore study the asymptotic properties of the penalized estimating equation estimator under the setting in which both the dimension of the true non-zero components of β and the total length of β tend to infinity as n goes to infinity. Denote β₀ = (β₁₀, ···, β_d0)^T as the true value of β. Let

$$a_n=max\{\mid p_{{\lambda }_n}^{\prime }(\mid {\beta }_{j0}\mid )\mid :{\beta }_{j0}\ne 0\},\text{and}{b}_n=max\{\mid p_{{\lambda }_n}^{″}(\mid {\beta }_{j0}\mid )\mid :{\beta }_{j0}\ne 0\},$$ (2.4)

where we write λ as λ_n to emphasize its dependence on the sample size n.

Theorem 1

Suppose that Condition (P1) in the Appendix holds. Under regularity conditions (A1)–(A3) in the Appendix, and if $d_n^4{n}^{-1}\to 0$, λ_n → 0 when n → ∞, then with probability tending to one, there exists a root of (2.2), denoted β̂, such that $\left|\right|\widehat{\beta }-{\beta }_0\left|\right|=O_p\{\sqrt{d_n}(n^{-1/2}+a_n)\}$, where we write d as d_n to emphasize its dependence on the sample size n.

The proof of Theorem 1 is given in the Appendix. Theorem 1 demonstrates that the convergence rate depends on the penalty function and the regularization parameter λ_n through a_n. From Theorem 1, it requires $a_n=O(1/\sqrt{n})$ to achieve root n/d_n convergence rate. For the L₁ penalty, a_n = λ_n. Thus, the root n/d_n convergence rate requires that ${\lambda }_n=O(1/\sqrt{n})$. While for the SCAD penalty, a_n = 0 as λ_n → 0. Thus, the resulting estimate with the SCAD penalty is root n/d_n consistent.

To present the oracle property of the resulting estimate, we first introduce some notation. Without loss of generality, we assume ${\beta }_0={({\beta }_{I0}^T,{\beta }_{II0}^T)}^T$, and in the true model, any element in β_I0 is not equal to 0 while β_II0 ≡ 0. Denote the dimension of β_I as d₁. Furthermore, denote

$$b={\{p_{{\lambda }_n}^{\prime }({\beta }_{10}),\cdots ,p_{{\lambda }_n}^{\prime }({\beta }_{d_{1}0})\}}^T\text{and}\mathrm{\sum }=\text{diag}\{p_{{\lambda }_n}^{″}({\beta }_{10}),\cdots ,p_{{\lambda }_n}^{″}({\beta }_{d_{1}0})\},$$ (2.5)

and the first d₁ components of $S_{\mathit{eff}}^{\ast }(W,Z,Y,\beta )$ as $S_{\mathit{eff},I}^{\ast }(\beta )$. In the following theorem, we use the same formulation as that in Cai, Fan, Li and Zhou (2005).

Theorem 2

Suppose that Condition (P1) holds. Under regularity conditions (A1)–(A3), assume λ_n → 0 and $d_n^5/n\to 0$ when n → ∞. If

$$\underset{n\to \infty }{liminf}\underset{\gamma \to 0+}{liminf}\sqrt{n/d_n}{p}_{{\lambda }_n}^{\prime }(\gamma )\to \infty ,$$ (2.6)

then with probability tending to one, any root n consistent solution ${\widehat{\beta }}_n={({\widehat{\beta }}_I^T,{\widehat{\beta }}_{II}^T)}^T$ of (2.2) must satisfy that

1. β̂_II = 0,

2. for any d₁ × 1 vector v, s.t. v^Tv = 1,

$$\begin{array}{l}\sqrt{n}{v}^T{[E\{S_{\mathit{eff},I}^{\ast }({\beta }_{I0})S_{\mathit{eff},I}^{\ast T}({\beta }_{I0})\}]}^{-1/2}\left\{E\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}-\mathrm{\sum }\right\}\\ \left[{\widehat{\beta }}_I-{\beta }_{I0}-{\left\{E\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}-\mathrm{\sum }\right\}}^{-1}b\right]\stackrel{\text{D}}{\to }N(0,1).\end{array}$$

where the notation $\stackrel{\text{D}}{\to }$ stands for convergence in distribution.

The proof of Theorem 2 is given in the Appendix. For some penalty functions, including the SCAD penalty, b and Σ are zero when λ_n is sufficiently small, hence the results in Theorem 2 imply that the proposed procedure has the celebrated oracle property: i.e., β̂_II = 0, and for any d₁ × 1 vector v, s.t. v^Tv = 1,

$$\sqrt{n}{v}^T{[E\{S_{\mathit{eff},I}^{\ast }({\beta }_{I0})S_{\mathit{eff},I}^{\ast T}({\beta }_{I0})\}]}^{-1/2}E\left\{\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}\right\}({\widehat{\beta }}_I-{\beta }_{I0})\stackrel{\text{D}}{\to }N(0,1).$$ (2.7)

Theorems 1 and 2 imply that for fixed and finite d, ||β̂ − β₀|| = O_p(n^−1/2+a_n) and with probability tending to one, any root n convergence solution $\widehat{\beta }={({\widehat{\beta }}_I^T,{\widehat{\beta }}_{II}^T)}^T$ of (2.2) must satisfy that β̂_II = 0 and

$$\begin{array}{l}\sqrt{n}\left[{\widehat{\beta }}_I-{\beta }_{I0}-{\left\{E\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}-\mathrm{\sum }\right\}}^{-1}b\right]\stackrel{\text{D}}{\to }\\ N\left[0,{\left\{E\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}-\mathrm{\sum }\right\}}^{-1}E\{S_{\mathit{eff},I}^{\ast }({\beta }_{I0})S_{\mathit{eff},I}^{\ast T}({\beta }_{I0})\}{\left\{E\frac{\partial S_{\mathit{eff},I}^{\ast }({\beta }_{I0})}{\partial {\beta }_I^T}-\mathrm{\sum }\right\}}^{-T}\right],\end{array}$$

where the notation M^−T denotes (M⁻¹)^T for a matrix M. These same results are still valid under much weaker conditions. See an elaborated version of this paper, Ma and Li (2007), for details.

For SCAD penalty and for fixed and finite d, (2.7) becomes

$$\sqrt{n}({\widehat{\beta }}_I-{\beta }_{I0})\to N\{0,E{(\partial S_{\mathit{eff},I}^{\ast }/\partial {\beta }_I^T)}^{-1}E(S_{\mathit{eff},I}^{\ast }{S}_{\mathit{eff},I}^{\ast T})E{(S_{\mathit{eff},I}^{\ast }/\partial {\beta }_I^T)}^{-T}\}$$

in distribution. In other words, with probability tending to 1, the penalized estimator performs the same as the locally efficient estimator under the correct model.

3. Semiparametric measurement error models

To motivate the problems considered in this section, we start with some commonly-used semiparametric regression models for which the proposed procedure in this section can be directly applied. Consider first the error-free regression cases, and let Y be the response, Z and S be covariates. Throughout this paper, we consider univariate Z only. Consider the partially linear model defined as follows:

$$Y=\theta (Z)+S^T\beta +\epsilon .$$ (3.1)

The partially linear model keeps the flexibility of nonparametric model for the baseline function, while maintaining the explanatory power of parametric models. Therefore, it has received a lot of attention in the literature. See, for example, Härdle, Liang and Gao (2000) and references therein. Various extensions of the partially linear model have been proposed in the literature. Li and Nie (2007, 2008) proposed the partially nonlinear models

$$Y=\theta (Z)+f(S;\beta )+\epsilon ,$$ (3.2)

where f(S; β) is a specific, known function, which may be nonlinear in β. See Li and Nie (2007, 2008) for some interesting examples. Li and Liang (2008) and Lam and Fan (2008) studied the generalized varying-coefficient partially linear model

$$g\{E(Y\mid Z,S)\}=S_1^T\beta +S_2^T\theta (Z),$$ (3.3)

where g(·) is a link function, and (S₁, S₂, Z) are covariates. Model (3.3) includes most commonly-used semiparametric models, such as the partially linear models (3.1), the generalized partially linear models (Severini and Staniswalis, 1994), and semi-varying coefficient partially linear models (Fan and Huang, 2005).

In the presence of covariates measured with error, one may extend the aforementioned semiparametric regression models for measurement error data. As in the last section, let X be the covariate vector measured with error. Among these semiparametric models with error, the partially linear measurement error model

$$Y=\theta (Z)+X^T{\beta }_1+S^T{\beta }_2+\epsilon$$ (3.4)

has been studied in Liang, Härdle and Carroll (1999). Liang and Li (2009) proposed a class of variable selection for model (3.4) using penalized least squares and penalized quantile regression. Our procedure in this section, however, is directly applicable for both the generalized varying-coefficient partially linear measurement error model

$$g\{E(Y\mid X,Z,S)\}=X^T{\beta }_1+S_1^T{\beta }_2+S_2^T\theta (Z),$$ (3.5)

where $S={(S_1^T,S_2^T)}^T$, and the partially nonlinear measurement error model

$$Y=\theta (Z)+f(X,S;\beta )+\epsilon ,$$ (3.6)

when an error distribution is assumed. It is worth noting that model (3.6) includes the following model as a special case

$$Y=X^T{\beta }_1+S^T{\beta }_2+{(XZ)}^T{\beta }_3+\theta (Z)+\epsilon ,$$ (3.7)

where (XZ) consists of all interaction terms between X and Z, but model (3.4) does not. Thus, the variable selection procedures proposed in Liang and Li (2009) are not directly applicable for model (3.7).

In summary, in this section, we consider a general semiparametric error model that include models (3.4), (3.5) and (3.6) as its special cases. Specifically, the semiparametric measurement error model we consider here also has two parts:

$$p_{Y\mid X,Z,S}\{Y\mid X,Z,S,\beta ,\theta (Z)\}\text{and}{p}_{W\mid X,Z,S}(W\mid X,Z,S).$$ (3.8)

The major difference from its parametric counterpart is that the main model contains unknown functions θ (Z). It is easy to check that models (3.4), (3.5) and (3.6) are special cases of model (3.8). Note that a simpler version of this model is considered in Ma and Carroll (2006), where the dimension of β is assumed to be fixed, and the dimension of θ is assumed to be one.

Throughout this paper, we assume that the model is identifiable. We propose the penalized estimating equation for the semiparametric model:

$$\sum _{i=1}^n\mathcal{L}(W_i,Z_i,S_i,Y_i,\beta ,{\widehat{\theta }}_i)-n{\stackrel{.}{p}}_{\lambda }(\beta )=0,$$ (3.9)

where ṗ_λ (β) has the same form as in (2.3). However, the computation of ℒ is more involved as we now describe. Denote the dimension of θ (Z) as m, a fixed and finite integer. If we replace θ (Z) with a single unknown m-dimensional parameter α, and append α to β, we obtain from (3.8) a parametric measurement error model with parameters (β^T, α^T)^T. For this parametric model, we can compute its corresponding $S_{\mathit{eff}}^{\ast }$ as done in the last section. Specifically, we will have $S_{\mathit{eff}}^{\ast }(W,Z,S,Y)=S_{\beta ,\alpha }^{\ast }(W,Z,S,Y)-E^{\ast }\{a(X,Z,S)\mid W,Z,S,Y\}$, where a(X, Z, S) satisfies $E[E^{\ast }\{a(X,Z,S)\mid W,Z,S,Y\}\mid X,Z,S]=E\{S_{\beta ,\alpha }^{\ast }(W,Z,S,Y)\mid X,Z,S\}$. Note that $S_{\mathit{eff}}^{\ast }$ has the same dimension as the dimension of β plus m. We write the last m components of $S_{\mathit{eff}}^{\ast }$ as Ψ (X, Z, S, Y, β, α) and the rest as ℒ (X, Z, S, Y, β, α). We now solve for θ̂_i, i = 1, …, n, from

$$\begin{array}{c}\sum _{i=1}^n{K}_h(z_i-z_1)\mathrm{\Psi }(w_i,z_i,s_i,y_i;\beta ,{\theta }_1)=0\\ ⋮\\ \sum _{i=1}^n{K}_h(z_i-z_n)\mathrm{\Psi }(w_i,z_i,s_i,y_i;\beta ,{\theta }_n)=0,\end{array}$$ (3.10)

where K_h(z) = h⁻¹ K(z/h), K is a smooth symmetric kernel function with compact support that satisfies ∫ K(t)t²dt = 1, and h is a bandwidth. Note that θ₁, …, θ_n are all m-dimensional parameters. Inserting the θ̂_i’s into ℒ in (3.9), we obtain a complete description of the estimator. Note that θ̂_i depends on β, so a more precise notation for θ̂_i is θ̂_i(β). Solving equation (3.9) yields a penalized estimating equation estimate. Theorem 3 below gives its convergence rate.

Theorem 3

Suppose that condition (P1) holds. Under regularity conditions (B1)–(B4) in the Appendix, and if $d_n^4{n}^{-1}\to 0$, λ_n → 0 when n → ∞, then with probability tending to one, there exists a root of (3.9), denoted β̂, such that $\left|\right|\widehat{\beta }-{\beta }_0\left|\right|=O_p\{\sqrt{d_n}(n^{-1/2}+a_n)\}$.

The proof of Theorem 3 is given in the Appendix. Theorem 3 indicates that to achieve root n/d_n convergence rate (or root n convergence rate for finite and fixed d), λ_n and the penalty function must be chosen such that a_n = O_p(n^−1/2).

Let ℒ_I be the first d₁ components of ℒ; ℒ_IβI the partial derivative of ℒ_I with respect to β_I; ℒ_Iθ the partial derivative of ℒ_I with respect to θ; Ψ _θ the partial derivative of Ψ with respect to θ and Ψ_βI the partial derivative of Ψ with respect to β_I. Also define Ω(Z) = E (Ψ_θ|Z), (Z) = E(ℒ_Iθ|Z)Ω⁻¹ (Z) and θ_βI (Z) = −Ω⁻¹(Z)E(Ψ_βI|Z). Further defining

$$\begin{array}{l}A=E[{\mathcal{L}}_{I{\beta }_I}\{W,Z,S,Y,{\beta }_0,{\theta }_0(Z)\}+{\mathcal{L}}_{I\theta }\{W,Z,S,Y,{\beta }_0,{\theta }_0(Z)\}{\theta }_{{\beta }_I}(Z,{\beta }_0)],\\ B=cov[{\mathcal{L}}_I\{W,Z,S,Y,{\beta }_0,{\theta }_0(Z)\}-{\mathcal{U}}_{\mathcal{ℐ}}(Z)\mathrm{\Psi }\{W,Z,S,Y,{\beta }_0,{\theta }_0(Z)\}],\end{array}$$

we obtain the following results.

Theorem 4

Suppose that condition (P1) holds. Under regularity conditions (B1)–(B4), if λ_n → 0, $d_n^5{n}^{-1}\to 0$, and (2.6) holds, then with probability tending to one, any root n consistent estimator ${\widehat{\beta }}_n={({\widehat{\beta }}_I^T,{\widehat{\beta }}_{II}^T)}^T$ obtained in (3.9) must satisfy that

1. β̂_II = 0,

2. for any d₁ × 1 vector v such that v^Tv = 1,

$$\sqrt{n/d_n}{v}^T{B}^{-1/2}(A-\mathrm{\sum })\left\{{\widehat{\beta }}_I-{\beta }_{I0}-{(A-\mathrm{\sum })}^{-1}b\right\}\stackrel{\text{D}}{\to }N(0,1).$$

The proof of Theorem 4 is given in the Appendix. Theorem 4 implies that for fixed and finite d, the convergence rate of the resulting estimate is n^−1/2 + a_n. It also implies that any root n consistent solution $\widehat{\beta }={({\widehat{\beta }}_I^T,{\widehat{\beta }}_{II}^T)}^T$ of (3.9) must satisfy β̂_II = 0, and β̂_I has the following asymptotic normality,

$$\sqrt{n}\left\{{\widehat{\beta }}_I-{\beta }_{I0}-{(A-\mathrm{\sum })}^{-1}b\right\}\stackrel{\text{D}}{\to }N\{0,{(A-\mathrm{\sum })}^{-1}B{(A-\mathrm{\sum })}^{-T}\}.$$

See the earlier version of this work, Ma and Li (2007) for details.

4. Numerical studies and application

In this section, we provide implementation details such as tuning parameter selection and model error approximation. Issues related to the numerical procedure to solve (2.2) and (3.9), the choice of kernel and bandwidth in the semiparametric model, and the treatment of multiple roots have been addressed in Ma and Carroll (2006) hence is not further discussed here. We assess the finite sample performance of the proposed procedure by Monte Carlo simulation, and illustrate the proposed methodology by an empirical analysis of the Framingham heart study data. In our simulation, we concentrate on the performance of the proposed procedure for a quadratic logistic measurement error model and a partially linear logistic measurement error model in terms of model complexity and model error.

4.1 Tuning parameter selection

An MM algorithm (Hunter and Li, 2005) and a local linear approximation (LLA) algorithm (Zou and Li, 2008) have been proposed for penalized likelihood with nonconcave penalty. However, both the MM algorithm and the LLA algorithm are difficult to be implemented for the measurement error models we consider. Thus, we employ the local quadratic approximation (LQA) algorithm (Fan and Li, 2001) to solve the penalized estimating equations. Specifically, in implementing the Newton-Raphson algorithm to solve the penalized estimating equations, we locally approximate the first order derivative of the penalty function by a linear function, following the idea of local quadratic approximation (LQA) algorithm. Specifically, suppose that at the kth step of the iteration, we obtain the value β^(k). Then, for ${\beta }_j^{(k)}$ not very close to zero,

$$p_{\lambda }^{\prime }({\beta }_j)=p_{\lambda }^{\prime }(\mid {\beta }_j\mid )\text{sign}({\beta }_j)\approx \frac{p_{\lambda }^{\prime }(\mid {\beta }_j^{(k)}\mid )}{\mid {\beta }_j^{(k)}\mid }{\beta }_j,$$

otherwise, we set ${\beta }_j^{(k+1)}=0$, and exclude the corresponding covariate from the model. This approximation is updated in every step of the Newton-Raphson algorithm iteration. The LQA was proposed by Fan and Li (2001). In practice, we set the initial value of β to be the unpenalized estimating equation estimate. It can be shown that when the algorithm converges, the solution will satisfy the penalized estimating equations. Following Theorem 2 and 4, we can further approximate the estimation variance of the resulting estimator. That is

$$\widehat{cov}(\widehat{\beta })=\frac{1}{n}{(E-{\mathrm{\sum }}_{\lambda })}^{-1}F{(E-{\mathrm{\sum }}_{\lambda })}^{-T},$$

where Σ_λ is a diagonal matrix with elements equal to $p_{\lambda }^{\prime }(\mid {\widehat{\beta }}_j\mid )/\mid {\widehat{\beta }}_j\mid$ for nonvanishing β̂_j, a linear approximation of Σ defined in (2.5). We use E to denote the sample approximation of $E\partial S_{\mathit{eff},I}^{\ast }(W,Z,Y,{\beta }_I)/\partial {\beta }_I$ evaluated at β̂ for the parametric model (2.1) and the sample approximation of the matrix A evaluated at β̂ for the semiparametric model (3.8). Similarly, we use F to denote the sample approximation of $cov(S_{\mathit{eff},I}^{\ast })$ evaluated at β̂ for the parametric model and the sample approximation of the matrix B evaluated at β̂ for the semiparametric model, respectively. The consistency of the proposed sandwich formula can be shown by using similar techniques in Fan and Peng (2004). The accuracy of this sandwich formula will be tested in our simulation studies.

It is desirable to have automatic, data-driven methods to select the tuning parameter λ. Here we will consider two tuning parameter selectors, the GCV and BIC tuning parameter selectors. To define the GCV statistic and the BIC statistic, we need to define the degrees of freedom and goodness of fit measure for the final selected model. Similar to the nonconcave penalized likelihood approach, we may define effective number of parameters or degrees of freedom to be

$$d{f}_{\lambda }=\text{trace}\{I{(I+{\mathrm{\sum }}_{\lambda })}^{-1}\},$$

where I stands for the Fisher Information matrix. For the logistic regression models employed in this section, a natural approximation of I, ignoring the measurement error effect, is V^TQV, where V represents the covariates included in the model, and Q is a diagonal matrix with the ith element equal to μ̂_λ,i(1 − μ̂_λ,i). Here, μ̂_λ,i = P (Y_i = 1|V_i).

In the logistic regression model context of this section, we may employ its deviance as a goodness of fit measure. Specifically, let μ_i be the conditional expectation of Y_i given its covariates for i = 1, ···, n. The deviance of a model fit μ̂_λ = (μ̂_λ,1, ···, μ̂_λ,n) is defined to be

$$D({\widehat{\mu }}_{\lambda })=2\sum _{i=1}^n[Y_{i}log(Y_i/{\widehat{\mu }}_{\lambda ,i})+(1-Y_i)log\{(1-Y_i)/(1-{\widehat{\mu }}_{\lambda ,i})\}].$$

Define the GCV statistic to be

$$\mathit{GCV}(\lambda )=\frac{D({\widehat{\mu }}_{\lambda })}{n{(1-d{f}_{\lambda }/n)}^2},$$

and the BIC statistic to be

$$\mathit{BIC}(\lambda )=D({\widehat{\mu }}_{\lambda })+2log(n)d{f}_{\lambda }.$$

The GCV and the BIC tuning parameter selectors select λ by minimizing GCV (λ) and BIC(λ) respectively. Note that the BIC tuning parameter selector is distinguished from the traditional BIC variable selection criterion, which is not well defined for estimating equation methods. Wang, Li and Tsai (2007) provided a study on the asymptotic behavior for the GCV and BIC tuning parameter selectors for the nonconcave penalized least squares variable selection procedures in linear and partially linear regression models. It is needed to further study the asymptotic property of the proposed tuning parameter selection, but it is out of the score of this paper.

4.2 Model error

Model error is an effective way of evaluating model adequacy versus model complexity. To implement the concept of model error in evaluating our procedure, we first simplify its definition for logistic partially linear measurement error model. Denote μ(S, X, Z) = E(Y |S, X, Z), and define the model error for a model μ̂(S, X, Z) as

$$ME(\widehat{\mu })=E{\{\widehat{\mu }(S^{+},X^{+},Z^{+})-\mu (S^{+},X^{+},Z^{+})\}}^2,$$

where the expectation is taken over the new observation S⁺, X⁺ and Z⁺. Let g(·) be the logit link. For the logistic partially linear model, the mean function has the form μ(S, X, Z) = g⁻¹{θ (Z)+β^TV}, where V = (S^T, X^T)^T. If θ̂(·) and β̂ are consistent estimators for θ(·) and β, respectively, then by a Taylor expansion, the model error can be approximated by

$$\begin{array}{l}ME(\widehat{\mu })\approx E({\stackrel{.}{g}}^{-1}{\{\theta (Z^{+})+{\beta }^T{V}^{+}\}}^2[{\{\widehat{\theta }(Z^{+})-\theta (Z^{+})\}}^2\\ +{({\widehat{\beta }}^T{V}^{+}-{\beta }^T{V}^{+})}^2+2\{\widehat{\theta }(Z^{+})-\theta (Z^{+})\}({\widehat{\beta }}^T{V}^{+}-{\beta }^T{V}^{+})]).\end{array}$$

The first component is the inherent model error due to θ̂(·), the second one is due to the lack of fit of β̂, and the third one is the cross-product between the first two components. Thus, to assess the performance of the proposed variable selection procedure, we define the approximate model error (AME) for β̂ to be

$$\mathit{AME}(\widehat{\beta })=E\left[{\stackrel{.}{g}}^{-1}{\{\theta (Z^{+})+{\beta }^T{V}^{+}\}}^2{({\widehat{\beta }}^T{V}^{+}-{\beta }^T{V}^{+})}^2\right].$$

Furthermore, the AME of β̂ can be written as

$$\mathit{AME}(\widehat{\beta })={(\widehat{\beta }-\beta )}^{T}E[{\stackrel{.}{g}}^{-1}{\{\theta (Z^{+})+{\beta }^T{V}^{+}\}}^2{V}^{+}{V}^{+T}](\widehat{\beta }-\beta )\stackrel{\wedge}{=}{(\widehat{\beta }-\beta )}^T{C}_X(\widehat{\beta }-\beta ).$$ (4.1)

In our simulation, the matrix C_X is estimated by 1,000,000 Monte Carlo simulations. For measurement error data, we observe W instead of X. We also consider an alternative approximate model error

$${\mathit{AME}}_W(\widehat{\beta })={(\widehat{\beta }-\beta )}^T{C}_W(\widehat{\beta }-\beta ),$$ (4.2)

where C_W is obtained by replacing X with W in the definition of C_X. The AME(β̂) and AME_W (β̂) are defined for the parametric model case by setting θ (·) = 0. Note that although we defined the model error in the context with a logistic link function, it is certainly not restricted to such case. The general approach for calculating AME is to approximate the probability density function evaluated at the estimated parameters around the true parameter value, and to extract the linear term of the parameter of interest. AME_W is calculated by replacing X with W.

4.3 Simulation examples

To demonstrate the performance of our method in both parametric and semiparametric measurement error models, we conduct two simulation studies. In our simulation, we will examine only the performance of the penalized estimating equation method with the SCAD penalty.

Example 1

In this example, we generate data from a logistic model where the covariate measured with error enters the model through a quadratic function, and the covariates measured without error enter linearly. The measurement error follows a normal additive pattern. Specifically,

$$\begin{array}{l}\text{logit}\{p(Y=1\mid X,Z)\}={\beta }_0+{\beta }_{1}X+{\beta }_2{X}^2+{\beta }_3{Z}_1+{\beta }_4{Z}_2+{\beta }_5{Z}_3+{\beta }_6{Z}_4\\ +{\beta }_7{Z}_5+{\beta }_8{Z}_6+{\beta }_9{Z}_7,\\ \text{and}W=X+U,\end{array}$$

where β = (0, 1.5, 2, 0, 3, 0, 1.5, 0, 0, 0)^T, the covariate X is generated from a normal distribution with mean 0 and variance 1, (Z₁, …, Z₆)^T is generated from a normal distribution with mean 0 and covariance between Z_i and Z_i is 0.5^|i−j|. The last component of the Z covariates, Z₇, is a binary variable taking value 0 or 1 with equal probability. U is normally distributed with mean 0 and standard deviation 0.1. In our simulation, the sample size is taken to be either n = 500 or n = 1000.

For the selected model, the model complexity is summarized in terms of the number of zero coefficients, and the model error is summarized in terms of relative approximation model error (RAME), defined to be the ratio of model error of the selected model to that of the full model. In Table 1, the RAME column corresponds to the sample median and median absolute deviation (MAD) divided by a factor of 0.6745 of the RAME values over 1000 simulations. Similarly, the RAME_W column corresponds to those of the RAME_W values over 1000 simulations. From Table 1, it can be seen that the values of RAME and RAME_W are very close. The average count of zero coefficients is also reported in Table 1, where the column labeled “C” presents the average count restricted only to the true zero coefficients, while the column labeled “E” displays the average count of the coefficients erroneously set to 0.

Table 1.

MRMEs and Model Complexity for Example 1

	n	RAME	RAMEW	# of Zero Coefficients
		Median (MAD)	Median (MAD)	C	E
GCV	500	0.694(0.231)	0.698(0.228)	4.574	0.006
BIC	500	0.396(0.188)	0.396(0.187)	5.857	0.074
GCV	1000	0.766(0.187)	0.770(0.185)	4.456	0
BIC	1000	0.390(0.157)	0.401(0.158)	5.758	0.010

We next verify the consistency of the estimators and test the accuracy of the proposed standard error formula. Table 2 displays the bias and sample standard deviation (SD) of the estimates for two nonzero coefficients, (β₁, β₂), over 1000 simulations, and the sample average and the sample standard deviations of the 1000 standard errors obtained by using the sandwich formula. The row labeled ‘EE’ corresponds to the unpenalized estimating equation estimator. We omit here the results for other nonzero coefficients and the results under sample size n = 500. Interested readers can find them in an earlier version of this work, Ma and Li (2007). Overall, the estimators are consistent and the sandwich formula works well.

Table 2.

Bias and Standard Errors for Example 1 (n = 1000)

	β̂1		β̂2
	bias (SD)	SE (Std(SE))	bias (SD)	SE (Std(SE))
EE	0.072 (0.273)	0.268 (0.062)	0.124 (0.332)	0.321(0.088)
GCV	0.029 (0.254)	0.250 (0.048)	0.009 (0.258)	0.253(0.057)
BIC	0.024 (0.290)	0.249 (0.054)	0.052 (0.255)	0.244(0.052)

Example 2

In this example, we illustrate the performance of the method for a semiparametric measurement error model. Simulation data are generated from

$$\begin{array}{l}\text{logit}(Y)={\beta }_{1}X+{\beta }_2{S}_1+\cdots +{\beta }_{10}{S}_9+\theta (Z)\\ W=X+U\end{array}$$

where β, X and W are the same as in the previous simulation. We generate S’s in a similar fashion as the Z’s in Example 1. That is, (S₁, …, S₈) is generated from a normal distribution with mean zero and covariance between S_i and S_j is 0.5^|i−j|. S₉ is a binary variable with equal probability to be zero or one. The random variable Z is generated from a uniform distribution in [−π/2, π/2]. The true function θ (Z) = 0.5 cos(Z). The parameter takes values β = (1.5, 2, 0, 0, 3, 0, 1.5, 0, 0, 0)^T.

The simulation results are summarized in Table 3, with notation similar to that of Table 1. From Table 3, we can see that the penalized estimating equation estimators can significantly reduce model complexity. Overall, the BIC tuning parameter selectors perform better, while GCV is too conservative. We have further tested the consistency and the accuracy of the standard error formula derived from the sandwich formula. The result is summarized in Table 4, with notation similar to that of Table 2. We note the consistency of the estimator and that the standard error formula performs very well. More simulation results are summarized in the earlier version of the work, Ma and Li (2007).

Table 3.

MRMEs and Model Complexity for Example 2

Method	n	RMEX	RMEW	# of Zero Coefficients
		Median (MAD)	Median (MAD)	C	E
GCV	500	0.878(0.161)	0.880(0.158)	4.060	0
BIC	500	0.381(0.158)	0.387(0.155)	5.713	0
GCV	1000	0.868(0.164)	0.873(0.160)	4.061	0
BIC	1000	0.386(0.162)	0.392(0.161)	5.694	0

Table 4.

Bias and Standard Errors for Example 2 (n = 1000)

Method	β̂1		β̂2
	Bias (SD)	SE (Std(SE))	Bias (SD)	SE (Std(SE))
EE	0.039(0.170)	0.166(0.018)	0.057(0.194)	0.190(0.018)
GCV	0.047(0.174)	0.172(0.020)	0.069(0.196)	0.191(0.021)
BIC	0.031(0.169)	0.170(0.019)	0.044(0.179)	0.185(0.019)

4.4 An application

The Framingham heart study data set (Kannel et al., 1986) is a well known data set where it is generally accepted that measurement error exists on the long term systolic blood pressure (SBP). In addition to SBP, other measurements include age, smoking status and serum cholesterol. In the literature, there has been speculation that a second order term involving age might be needed to analyze the dependence of heart disease occurrence. In addition, it is unclear if the interaction between the various covariates plays a role in influencing the heart disease rate. The data set includes 1,615 observations.

With the method developed here, it is possible to perform a variable selection to address these issues. Following the literature, we adopt the measurement error model of log(MSBP − 50) = log(SBP − 50) + U, where U is a mean zero normal random variable with variance ${\sigma }_u^2=0.0126$, MSBP is the measured SBP. We denote the standardized log(MSBP − 50) as W. The standardization using the same parameters on log(SBP − 50) is denoted X. The standardized serum cholesterol and age are denoted by Z₁, Z₂ respectively, and Z₃ denotes the binary variable smoking status. Using Y to denote the occurrence of heart disease, the saturated model which includes all the interaction terms and also the square of age term is of the form

$$\begin{array}{l}\text{logit}\{p(Y=1\mid X,Z^{\prime }s)\}={\beta }_{1}X+{\beta }_{2}X{Z}_1+{\beta }_{3}X{Z}_2+{\beta }_{4}X{Z}_3+{\beta }_5+{\beta }_6{Z}_1+{\beta }_7{Z}_2\\ +{\beta }_8{Z}_3+{\beta }_9{Z}_2^2+{\beta }_{10}{Z}_1{Z}_2+{\beta }_{11}{Z}_1{Z}_3+{\beta }_{12}{Z}_2{Z}_3,\\ W=X+U.\end{array}$$

We used both GCV and BIC tuning parameter selectors to choose λ. We present the tuning parameters and the corresponding GCV and BIC scores in Figure 1. The final chosen λ is 0.073 and 0.172 by the GCV and BIC selectors, respectively. The selected model is depicted in Table 5. The GCV criterion selects the covariates X, XZ₁, 1, Z₁, Z₂, Z₃, $Z_2^2$, Z₂Z₃ into the model, while the BIC criterion selects the covariates X, 1, Z₁, Z₂ into the model. We report the selection and estimation results in Table 5, as well as the semiparametric estimation results without variable selection.

Figure 1.

Tuning parameters and their corresponding BIC and GCV scores for the Framingham data. The scores are normalized to the range [0, 1].

Table 5.

Results for the Framingham data set.

	EE	GCV	BIC
	β̂ (SE)	β̂ (SE)	β̂ (SE)
X	0.643(0.248)	0.416(0.093)	0.179(0.039)
XZ1	−0.167(0.097)	−0.072(0.041)	0 (NA)
XZ2	−0.059(0.111)	0 (NA)	0 (NA)
XZ3	−0.214(0.249)	0 (NA)	0 (NA)
Intercept	−3.415(0.428)	−3.255(0.356)	−2.555(0.092)
Z1	0.516(0.212)	0.332(0.085)	0.124(0.033)
Z2	1.048(0.341)	1.044(0.329)	0.398(0.067)
Z3	1.060(0.443)	0.907(0.373)	0 (NA)
$Z_2^2$	−0.253(0.125)	−0.262(0.121)	0 (NA)
Z1Z2	−0.072(0.103)	0 (NA)	0 (NA)
Z1Z3	−0.161(0.225)	0 (NA)	0 (NA)
Z2Z3	−0.442(0.336)	−0.473(0.326)	0 (NA)

As seen, the terms X, 1, Z₁, Z₂ are selected by both criteria, while Z₃, $Z_2^2$ and some of the interaction terms are selected only by GCV. The BIC criterion is very aggressive and it results in a very simple final model while the GCV criterion is much more conservative, hence the resulting model is more complex. This agrees with the simulation results obtained. Since both criteria have included the covariate X, the measurement error feature and its treatment in the Framingham data is unavoidable.

5. Discussion

In this paper, we have proposed a new class of variable selection procedures in the framework of measurement error models. The procedure is proposed in a completely general functional measurement error model setting, and is suitable for both parametric and semiparametric models that contain unspecified smooth functions of an observable covariate. We have assumed the error model p_W|X,Z(W|X, Z) to be completely known for ease of presentation. In the situation when the error model contains an unknown parameter ξ, the identifiability of the problem requires additional information such as multiple measurements or instruments. Such information should be incorporated to estimate ξ. Specifically, in the variable selection context, we can simply append the estimating equation with these additional estimating equations obtained from the corresponding score functions with respect to ξ, and append the penalty function $p_{\lambda }^{\prime }$ with zeros. Because the augmented estimating equations still have the same convergence property as illustrated in Ma and Carroll (2006), the same asymptotic convergence rates and oracle properties hold as in the known ξ case, without any efficiency loss. When the error model p_W|X,Z(W|X, Z) is completely unspecified, a nonparametric estimation of the measurement error distribution has to be carried out first, then the result can be plugged into the proposed variable selection and estimation procedure. In this case, the asymptotic convergence rate of the parameters and the oracle property remain the same, but the asymptotic variance will increase; The details on incorporating the estimation of unknown error and demonstrating its subsequent convergence property in the estimation framework is the focus of Hall and Ma (2007).

We also would like to point out that in the special case of generalized linear models and normal additive error with possible heteroscedasticity, the procedure of solving linear integral equations can be spared and the estimating equations are simplified significantly (Ma and Tsiatis, 2006). In such situations, the computation complexity of the proposed procedure will be reduced to about the same level as for variable selection in regressions without errors in the variables.

As pointed out by the referee, it is of great interest to perform variable selection for high dimensional data. In this paper, we allow the number of covariates to grow to infinity at a o_p(n^−1/5) rate as the sample size n increases. However, the proposed procedures and the used algorithm in this paper may not directly applied to the large p, small n problems. Variable selection for the large p, small n setting is a very active research topic. It is very challenging to extend the existing variable selection procedures for large p, small n problems to measurement error data. Further researches are needed on this topic, but this is out of scope of this paper.

Appendix

Global assumption (P1) on the penalty function

(P1) Let $c_n=max\{\mid p_{\lambda }^{″}(\mid {\beta }_{j0}\mid )\mid :{\beta }_{j0}\ne 0\}$. Assume that λ_n → 0, a_n = O(n^−1/2) and c_n → 0 n → ∞. In addition, there exist constants C and D such that when γ₁, γ₂ > Cλ, $\mid p_{\lambda }^{″}({\gamma }_1)-p_{\lambda }^{″}({\gamma }_2)\mid \le D\mid {\gamma }_1-{\gamma }_2\mid$.

It is easy to verify that both the L₁ and the SCAD penalties satisfy this condition.

The regularity conditions for Theorems 1 and 2

• (A1) The expectation of the first derivative of $S_{\mathit{eff}}^{\ast }$ with respect to β exists at β₀ and its left eigenvalues are bounded away from zero and infinity uniformly for all n. For any entry S_jk in $\partial S_{\mathit{eff}}^{\ast }({\beta }_0)/\partial {\beta }^T,E(S_{jk}^2)<C_1<\infty$.

• (A2) The eigenvalues of the matrix $E(S_{\mathit{eff},I}^{\ast }{S}_{\mathit{eff},I}^{\ast T})$, satisfy 0 < C₂ < λ_min < ··· < λ_max < C₃ < ∞ for all n. For any entries, S_k, S_j in $S_{\mathit{eff}}^{\ast }({\beta }_0),E(S_k^2{S}_j^2)<C_4<\infty$.

• (A3) The second derivatives of $S_{\mathit{eff}}^{\ast }$ with respect to β exist and the entries are uniformly bounded by a function M(W_i, Z_i, Y_i) in a large enough neighborhood of β₀. In addition, E(M²) < C₅ < ∞ for all n, d.

Conditions (A1) – (A3) are mild regularity conditions. They guarantee that the solution of the following estimating equation

$$\sum _{i=1}^n{S}_{\mathit{eff}}^{\ast }(W_i,Z_i,Y_i,\beta )=0$$

is root n/d_n convergent, and possesses asymptotic normality.

Proof of Theorem 1

Condition (A1) allows us to define

$$J={\left\{E\left(\frac{\partial S_{\mathit{eff}}^{\ast }}{\partial {\beta }^T}{\mid }_{{\beta }_0}\right)\right\}}^{-1},{\phi }_{\mathit{eff}}^{\ast }+{JS}_{\mathit{eff}}^{\ast }\text{and}{q}_{{\lambda }_n}^{\prime }(\beta )=J{p}_{{\lambda }_n}^{\prime }(\beta ).$$

Let α_n = n^−1/2 + a_n, and ${\phi }_{\mathit{eff},i}^{\ast }(\beta )={\phi }_{\mathit{eff}}^{\ast }(W_i,Z_i,Y_i,\beta )$. It suffices to show that

$$n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }(\beta )-n^{1/2}{q}_{{\lambda }_n}^{\prime }(\beta )=0$$ (A1)

has a solution β̂ that satisfies $\left|\right|\widehat{\beta }-{\beta }_0\left|\right|=O_p(\sqrt{d_n}{\alpha }_n)$. This will be shown using the Brouwer fixed point theorem. Using the Taylor expansion, we have

$$\begin{array}{l}{n}^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }(\beta )-n^{1/2}{q}_{{\lambda }_n}^{\prime }(\beta )\\ =n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }({\beta }_0)-n^{1/2}{q}_{{\lambda }_n}^{\prime }({\beta }_0)+n^{-1/2}\sum _{i=1}^n\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }({\beta }^{\ast })}{\partial {\beta }^T}(\beta -{\beta }_0)\\ -n^{1/2}\frac{\partial q_{{\lambda }_n}^{\prime }({\beta }_0)}{\partial {\beta }^T}(\beta -{\beta }_0)\{1+o_p(1)\}\end{array}$$

where β^* is in between β and β₀. It can be shown by Conditions (A1), (A2) and (A3) and definition of ${\phi }_{\mathit{eff}}^{\ast }(·)$ that

$${(\beta -{\beta }_0)}^T\left\{\frac{1}{n}\sum _{i=1}^n\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }({\beta }^{\ast })}{\partial {\beta }^T}\right\}(\beta -{\beta }_0)={\left|\right|\beta -{\beta }_0\left|\right|}^2\{1+o_P(1)\}$$

We next check the key condition for the Brouwer fixed point theorem. For any β such that $\left|\right|\beta -{\beta }_0\left|\right|=C\sqrt{d_n}{\alpha }_n$ for some constant C, it follows by Condition (P1) that

$$\begin{array}{c}{(\beta -{\beta }_0)}^T\left\{\frac{1}{\sqrt{n}}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }(\beta )-n^{1/2}{q}_{{\lambda }_n}^{\prime }(\beta )\right\}\\ ={(\beta -{\beta }_0)}^T\left\{\frac{1}{\sqrt{n}}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }({\beta }_0)-\sqrt{n}{q}_{{\lambda }_n}^{\prime }({\beta }_0)\right\}+\sqrt{n}{\left|\right|\beta -{\beta }_0\left|\right|}^2\{1+o_P(1)\}.\end{array}$$

Using the Cauchy-Schwartz inequality, it can be shown that the first term in the above equation is of order $\left|\right|\beta -{\beta }_0\left|\right|O_p(\sqrt{d_n+d_{n}n{a}_n^2})=O_p(C{n}^{1/2}{d}_n{\alpha }_n^2)$. Note that $\sqrt{n}{\left|\right|\beta -{\beta }_0\left|\right|}^2=C^2{n}^{1/2}{d}_n{\alpha }_n^2$. Thus the second term in the above equation dominates the first term with probability 1 − ε for any ε > 0 as long as C is large enough. Thus, for any ε > 0, and for large enough C, the probability for the above display to be larger than zero is at least 1 −ε. From the Brouwer fixed-point theorem, we know that with probability at least 1 − ε, there exists at least one solution for (A1) in the region $\left|\right|\beta -{\beta }_0\left|\right|\le C\sqrt{d_n}{\alpha }_n$.

Lemma on sparsity for Theorem 2

lemma 1

If the conditions in Theorem 2 hold, then for any given β that satisfies $\left|\right|\beta -{\beta }_0\left|\right|=O_p(\sqrt{d_n/n})$, with probability tending to 1, any solution ${({\beta }_I^T,{\beta }_{II}^T)}^T$ of (2.2) satisfies that β_II = 0.

Proof

Denote the kth element in ${\sum }_{i=1}^n{S}_{\mathit{eff}}^{\ast }(W_i,Z_i,Y_i,\beta )$ as L_nk(β), k = d₁ +1, …, d_n. We next show that the order of L_nk(β) is $O_p(\sqrt{n{d}_n})$

$$\begin{array}{l}{L}_{nk}(\beta )=L_{nk}({\beta }_0)+\sum _{j=1}^{d_n}\frac{\partial L_{nk}({\beta }_0)}{\partial {\beta }_j}({\beta }_j-{\beta }_{j0})\\ +2^{-1}\sum _{l=1}^{d_n}\sum _{j=1}^{d_n}\frac{{\partial }^2{L}_{nk}({\beta }^{\ast })}{\partial {\beta }_l\partial {\beta }_j}({\beta }_l-{\beta }_{l0})({\beta }_j-{\beta }_{j0}),\end{array}$$ (A2)

where β^* is in between β and β₀. Because of condition (A2), the first term of (A2) is of order $O_p(n^{1/2})=o_p(\sqrt{n{d}_n})$. The second term in (A2) can be further written as

$$\sum _{j=1}^{d_n}\left\{\frac{\partial L_{nk}({\beta }_0)}{\partial {\beta }_j}-E\frac{\partial L_{nk}({\beta }_0)}{\partial {\beta }_j}\right\}({\beta }_j-{\beta }_{j0})+\sum _{j=1}^{d_n}E\frac{\partial L_{nk}({\beta }_0)}{\partial {\beta }_j}({\beta }_j-{\beta }_{j0}).$$ (A3)

Using the Cauchy-Schwartz inequality and condition (A1), it can be shown by straightforward calculation that the first term in (A3) is of order $O_p(\sqrt{d_n/n})=o_p(\sqrt{n{d}_n})$. Using the Cauchy-Schwartz inequality again, the second term in (A3) is controlled by

$$n{\left\{\sum _{j=1}^{d_n}{\left(E\frac{\partial S_{\mathit{eff},k}}{\partial {\beta }^T}\right)}^2\right\}}^{1/2}\left|\right|\beta -{\beta }_0\left|\right|\le n{\lambda }_{max}{\left(E\frac{\partial S_{\mathit{eff},k}}{\partial {\beta }^T}\right)}^2\left|\right|\beta -{\beta }_0\left|\right|=O_p(\sqrt{n{d}_n}).$$

Thus, the second term of (A2) is of order $O_p(\sqrt{n{d}_n})$. As for the third term of (A2), we can have a similar decomposition to that of (A3). Using the Cauchy-Schwartz inequality in matrix form and condition (A3), it can be shown that the third term of (A2) is of order $O_p(d_n^2)+O_p(n^{-1/2}{d}_n^2)=o_p(\sqrt{n{d}_n})$ as $d_n^5/n\to 0$. Thus, L_nk(β) is of order $O_p(\sqrt{n{d}_n})$. Hence we have

$$L_{nk}(\beta )-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)=-\sqrt{n}\{\sqrt{n/d_n}{p}_{{\lambda }_n}^{\prime }(\mid {\beta }_k\mid )\text{sign}({\beta }_k)+O_p(1)\}.$$

Using condition (2.6), the sign of $L_{nk}(\beta )-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)$ is decided by sign(β_k) completely when n is large enough. From the continuity of $L_{nk}(\beta )-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)$, we obtain that it is zero at β_k = 0.

Proof of Theorem 2

From Theorem 1 and condition (P1), there is a root n/d_n consistent estimator β̂. From Lemma 1, $\widehat{\beta }={({\widehat{\beta }}_I^T,0^T)}^T$, so (i) is shown. Denote the first d₁ equations in ${\sum }_{i=1}^n{S}_{\mathit{eff}}^{\ast }\{W_i,Z_i,Y_i,{({\beta }_I^T,0^T)}^T\}$ as L_n(β_I). Now consider solving the first d₁ equations in (2.2) for β_I, while β_II = 0. we have

$$\begin{array}{l}0=L_n({\widehat{\beta }}_I)-n{p}_{{\lambda }_n,I}^{\prime }({\widehat{\beta }}_I)\\ =L_n({\beta }_{I0})+\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}({\widehat{\beta }}_I-{\beta }_{I0})-n{b}_n-n{p}_{{\lambda }_n}^{″}({\beta }_I^{\ast })({\widehat{\beta }}_I-{\beta }_{I0}),\end{array}$$

where ${\beta }_I^{\ast }$ is between β_I0 and β̂_I. It follows by condition (P1) that

$$\begin{array}{l}{\left|\right|n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-p_{{\lambda }_n,I}^{″}({\beta }_I^{\ast })-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}+p_{{\lambda }_n,I}^{″}({\beta }_{I0})\left|\right|}^2\\ \le 2{\left|\right|n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}\left|\right|}^2+O_p(n^{-1}{d}_n)\end{array}$$

Furthermore, for any fixed ε > 0, it follows by conditions (A1), (A3) and the Chebyshev inequality that

$$\begin{array}{l}{P}_r\left\{\left|\right|n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}\left|\right|\ge \epsilon {d_n}^{-1}\right\}\\ \le \frac{{d_n}^2}{n^2{\epsilon }^2}E{\left|\right|\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-nE\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}\left|\right|}^2=O({d_n}^2{n}^{-2}{d}_1^{2}n)=o(1).\end{array}$$

since d₁ ≤ d_n. Thus, we have

$$\left|\right|n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}\left|\right|=o_p({d_n}^{-1}).$$

Therefore,

$${\left|\right|n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-p_{{\lambda }_n,I}^{″}({\beta }^{\ast })-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}+p_{{\lambda }_n,I}^{″}({\beta }_{I0})\left|\right|}^2=o_p({d_n}^{-2}),$$

and subsequently,

$$\begin{array}{l}\left|\right|\left\{n^{-1}\frac{\partial L_n({\beta }_{I0}^{\ast })}{\partial {\beta }_I^T}-p_{{\lambda }_n,I}^{″}({\beta }^{\ast })-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}+p_{{\lambda }_n,I}^{″}({\beta }_{I0})\right\}({\widehat{\beta }}_I-{\beta }_{I0})\left|\right|\\ \le o_p(d_n^{-1})O_p(n^{-1/2}{d}_n^{1/2})=o_p(n^{-1/2}).\end{array}$$

We thus obtain that

$$\left\{-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}+{\mathrm{\sum }}_n\right\}({\widehat{\beta }}_I-{\beta }_{I0})+b_n=n^{-1}{L}_n({\beta }_{I0})+o_p(n^{-1/2}).$$

Denote $I^{\ast }=E\{S_{n,\mathit{eff},I}^{\ast }({\beta }_{I0})S_{n,\mathit{eff},I}^{\ast T}({\beta }_{I0})\}$. Using condition (A2), it follows that

$$n^{1/2}{v}^T{I}^{\ast -1/2}\left[\left\{-E\frac{\partial L_n({\beta }_{I0})}{\partial {\beta }_I^T}+{\mathrm{\sum }}_n\right\}({\widehat{\beta }}_I-{\beta }_{I0})+b_n\right]=n^{-1/2}{v}^T{I}^{\ast -1/2}{L}_n({\beta }_{I0})+o_p(1).$$

Let Y_i = n^−1/2v^TI^*−1/2 S_n,eff,I(W_i, Z_i, Y_i, β_I0). It follows that for any ε > 0,

$$\sum _{i=1}^{n}E{\left|\right|Y_i\left|\right|}^2\mathbf{1}(|\left|Y_i\right||>\epsilon )=nE{\left|\right|Y_1\left|\right|}^2\mathbf{1}(|\left|Y_1\right||>\epsilon )\le n{(E{\left|\right|Y_1\left|\right|}^4)}^{1/2}{\{P_r(\left|\right|Y_1\left|\right|>\epsilon )\}}^{1/2}.$$

Using Chebyshev’s inequality, we have

$$P_r(|\left|Y_1\right||>\epsilon )\le \frac{E{\left|\right|Y_1\left|\right|}^2}{{\epsilon }^2}=\frac{E{\left|\right|v{I}^{\ast -1/2}{S}_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})\left|\right|}^2}{n{\epsilon }^2}=\frac{v^{T}v}{n{\epsilon }^2}=O(n^{-1}).$$

Note that E(||Y₁||⁴) = n⁻²E||v^TI^*−1/2S_eff,I(W₁, Z₁, Y₁, β_I0)||⁴. Note that the rank of vv^T is one, and hence λ_max(vv^T) equals the trace of vv^T. So, λ_max(vv^T) = 1 as v^Tv = 1. Thus, it follows that

$$\begin{array}{l}E({\left|\right|Y_1\left|\right|}^4)=n^{-2}E{\{S_{\mathit{eff},I}{(W_1,Z_1,Y_1,{\beta }_{I0})}^T{I}^{\ast -1/2}v{v}^T{I}^{\ast -1/2}{S}_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})\}}^2\\ \le n^{-2}{\lambda }_{max}^2(I^{\ast -1})E{\{S_{\mathit{eff},I}{(W_1,Z_1,Y_1,{\beta }_{I0})}^T{S}_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})\}}^2\\ =n^{-2}{\lambda }_{max}^2(I^{\ast -1})E{\left|\right|S_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})\left|\right|}^4=O(d_1^2{n}^{-2}),\end{array}$$

due to condition (A2). Hence,

$$\sum _{i=1}^{n}E{\left|\right|Y_i\left|\right|}^2\mathbf{1}(|\left|Y_i\right||>\epsilon )=O(n{d}_1{n}^{-1}{n}^{-1/2})=o(1).$$

On the other hand,

$$\begin{array}{c}\sum _{i=1}^{n}cov(Y_i)=ncov\{n^{-1/2}v{I}^{\ast -1/2}{S}_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})\}\\ =v{I}^{\ast -1/2}E\{S_{\mathit{eff},I}(W_1,Z_1,Y_1,{\beta }_{I0})S_{\mathit{eff},I}{(W_1,Z_1,Y_1,{\beta }_{I0})}^T\}{I}^{\ast -1/2}{v}^T=1.\end{array}$$

Following Lindeberg-Feller central limit theorem, the results in (ii) now follow.

Regularity conditions for Theorem 3 and 4

The notation C_i below is generic and is allowed to be different from that in Conditions (A1)–(A3).

• (B1) The first derivatives of ℒ with respect to β and θ exist and are denoted as ℒ_β and ℒ_θ respectively. The first derivative of θ with respect to β exists and is denoted as θ_β. Thus E(ℒ_β + ℒ_θθ_β) exists and its left eigenvalues are bounded away from zero and infinity uniformly for all n at β₀ and the true function θ₀(Z). For any entry S_jk of the matrix d(ℒ_β + ℒ_θθ_β)/dβ, $E(S_{jk}^2)<C_1<\infty$.

• (B2) The eigenvalues of the matrix E{ℒ_I − (Z)Ψ}{ℒ_I − (Z)Ψ}^T satisfy 0 < C₂ < λ_min < ··· < λ_max < C₃ < ∞ for all n; for any entries S_k, S_j in (ℒ_β + ℒ_θθ_β), $E(S_k^2{S}_j^2)<C_4<\infty$.

• (B3) The second derivatives of ℒ with respect to β and θ exist, the second derivatives of θ with respect to β exists, and the entries are uniformly bounded by a function M(W_i, Z_i, S_i, Y_i) in a neighborhood of β₀, θ₀. In addition, E(M²) < C₅ < ∞ for all n, d.

• (B4) The random variable Z has compact support and its density f_Z(z) is positive on that support. The bandwidth h satisfies nh⁴ → 0 and nh² → ∞. θ(z) has bounded second derivative.

Proof of Theorem 3

Denote

$$J={[E\{({\mathcal{L}}_{\beta }+{\mathcal{L}}_{\theta }{\theta }_{\beta }){\mid }_{{\beta }_0,{\theta }_0}\}]}^{-1},{\phi }_{\mathit{eff}}^{\ast }(\beta ,\theta )=J\mathcal{L}(\beta ,\theta )\text{and}{q}_{{\lambda }_n}^{\prime }(\beta )=J{p}_{{\lambda }_n}^{\prime }(\beta ).$$

Let α_n = n^−1/2 + a_n, and ${\phi }_{\mathit{eff},i}^{\ast }(\beta ,\widehat{\theta })={\phi }_{\mathit{eff}}^{\ast }\{W_i,Z_i,S_i,Y_i,\beta ,\widehat{\theta }(\beta )\}$. It will be shown that

$$n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }(\beta ,\widehat{\theta })-n^{1/2}{q}_{{\lambda }_n}^{\prime }(\beta )=0$$ (A4)

has a solution β̂ that satisfies $\left|\right|\widehat{\beta }-{\beta }_0\left|\right|=O_p(d_n^{1/2}{\alpha }_n)$.

Due to the usual local estimating equation expansion, we have

$$\begin{array}{l}\widehat{\theta }(z,{\beta }_0)-{\theta }_0(z)\\ =(h^2/2){\theta }_0^{″}(z)-n^{-1}\sum _{j=1}^n{K}_h(Z_j-z){\mathrm{\Omega }}^{-1}(z){\mathrm{\Psi }}_j({\beta }_0,{\theta }_0)/f_Z(z)+o_p(n^{-1/2}),\end{array}$$ (A5)

which implies that θ̂(z, β₀) − θ₀(z) = O_p(h² + n^−1/2h^−1/2). For any β such that $\left|\right|\beta -{\beta }_0\left|\right|=C\sqrt{d_n}{\alpha }_n$ for some constant C, we obtain the expansion

$$\begin{array}{l}{n}^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }\{\beta ,\widehat{\theta }(\beta )\}\\ =n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0,\widehat{\theta }({\beta }_0)\}\\ +n^{-1/2}\sum _{i=1}^n\left[\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0+\widehat{\theta }({\beta }_0)\}}{\partial {\beta }^T}+\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0+\widehat{\theta }({\beta }_0)\}}{\partial {\theta }^T}\frac{\partial \widehat{\theta }}{\partial {\beta }^T}\right](\beta -{\beta }_0)\\ +\frac{1}{2\sqrt{n}}\sum _{i=1}^n{(\widehat{\beta }-{\beta }_0)}^T\frac{d\left[{\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0+\widehat{\theta }({\beta }_0)\}+{\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0+\widehat{\theta }({\beta }_0)\}\frac{d\widehat{\theta }}{d\beta }\right]}{d{\beta }^T}{\mid }_{{\beta }^{\ast }}(\beta -{\beta }_0),\end{array}$$

where β^* is in between β and β₀. Because of condition (B3), each component of the last term is uniformly of order O_p(n^1/2||β − β₀||²). The second term can be written as n^1/2{1 + o_p(1)}(β −β₀) under conditions (B1), (B3) and (B4). The first term can be further expanded as

$$\begin{array}{l}{n}^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0,\widehat{\theta }({\beta }_0)\}\\ =n^{-1/2}\sum _{i=1}^n\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0,\widehat{\theta }({\beta }_0)\}}{\partial {\theta }^T}\{\widehat{\theta }({\beta }_0)-{\theta }_0\}+O_p(n^{1/2})\{\widehat{\theta }({\beta }_0)-{\theta }_0\}{\{\widehat{\theta }({\beta }_0)-{\theta }_0\}}^T\\ =n^{-1/2}\sum _{i=1}^n\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }\{{\beta }_0,\widehat{\theta }({\beta }_0)\}}{\partial {\theta }^T}\{\widehat{\theta }({\beta }_0)-{\theta }_0\}+o_p(1)\end{array}$$

under conditions (B3) and (B4). Summarizing the above results, making use of (A5), we obtain

$$\begin{array}{l}{n}^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }\{\beta ,\widehat{\theta }(\beta )\}\\ =n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }({\beta }_0,{\theta }_0)+n^{1/2}(\beta -{\beta }_0)\\ -n^{-3/2}\sum _{j,i=1}^n\frac{\partial {\phi }_{\mathit{eff},i}^{\ast }({\beta }_0,{\theta }_0)}{\partial {\theta }^T}{K}_h(Z_j-Z_i)\frac{\mathrm{\Omega }(Z_i){\mathrm{\Psi }}_j({\beta }_0,{\theta }_0)}{f_Z(Z_i)}+o_p(1)\\ =n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }({\beta }_0,{\theta }_0)+n^{1/2}(\beta -{\beta }_0)-n^{-1/2}\sum _{i=1}^{n}J\mathcal{U}(Z_i){\mathrm{\Psi }}_j({\beta }_0,{\theta }_0)+o_p(1)\end{array}$$

under condition (B4). Similar to the situation in Theorem 1, under condition (P1), we further obtain

$$\begin{array}{l}{(\beta -{\beta }_0)}^T\left\{n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }(\beta ,\widehat{\theta })-n^{1/2}{q}_{{\lambda }_n}^{\prime }(\beta )\right\}\\ ={(\beta -{\beta }_0)}^T\left\{n^{-1/2}\sum _{i=1}^n{\phi }_{\mathit{eff},i}^{\ast }({\beta }_0,{\theta }_0)-n^{1/2}{q}_{{\lambda }_n}^{\prime }({\beta }_0)-n^{-1/2}\sum _{i=1}^{n}J\mathcal{U}(Z_i){\mathrm{\Psi }}_i({\beta }_0,{\theta }_0)\right\}\\ +n^{1/2}{\left|\right|\beta -{\beta }_0\left|\right|}^2+o_p\{n^{1/2}{\left|\right|\beta -{\beta }_0\left|\right|}^2\}.\end{array}$$ (A6)

The first term in the above display is of order $O_p(C{n}^{1/2}{d}_n{\alpha }_n^2)$, the second term equals $C^2{n}^{1/2}{d}_n{\alpha }_n^2$, which dominates the first term as long as C is large enough. The last term is dominated by the first two terms. Thus, for any ε > 0, as long as C is large enough, the probability for the above display to be larger than zero is at least 1 − ε. From Brouwer fixed-point theorem, we know that with probability at least 1 − ε, there exists at least one solution for (A4) in the region $\left|\right|\beta -{\beta }_0\left|\right|\le C{d}_n^{1/2}{\alpha }_n$.

Lemma for Theorem 4

lemma 2

If conditions in Theorem 4 hold, then for any given β that satisfies $\left|\right|\beta -{\beta }_0\left|\right|=O_p(\sqrt{d/n})$, with probability tending to 1, any solution ${({\beta }_I^T,{\beta }_{II}^T)}^T$ of (2.2) satisfies that β_II = 0.

Proof

Denote the kth equation in ${\sum }_{i=1}^n{\mathcal{L}}_i\{\beta ,\widehat{\theta }(\beta )\}$ as L_nk(β, θ̂), and that in ${\sum }_{i=1}^n\mathcal{U}(Z_i){\mathrm{\Psi }}_i({\beta }_0,{\theta }_0)$ as G_nk(β₀, θ₀), k = d₁ + 1, …, d_n, then the expansion in Theorem 3 leads to

$$\begin{array}{l}{L}_{nk}(\beta ,\widehat{\theta })-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)\\ =L_{nk}({\beta }_0,{\theta }_0)-G_{nk}({\beta }_0,{\theta }_0)+n\sum _{j=1}^d{(J^{-1})}_{kj}({\beta }_j-{\beta }_{j0})-n{p}_{{\lambda }_n}^{\prime }(\mid {\beta }_k\mid )\text{sign}({\beta }_k)+o_p(\sqrt{n{d}_n}).\end{array}$$

Similar to the derivation in Lemma 1, the first three terms of the above display are all of order $O_p(\sqrt{n{d}_n})$, hence we have

$$L_{nk}(\beta ,\widehat{\theta })-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)=-\sqrt{n{d}_n}\{\sqrt{n/d_n}{p}_{{\lambda }_n}^{\prime }(\mid {\beta }_k\mid )\text{sign}({\beta }_k)+O_p(1)\}.$$

Because of (2.6), the sign of $L_{nk}(\beta )-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)$ is decided by sign(β_k) completely. From the continuity of $L_{nk}(\beta )-n{p}_{{\lambda }_n}^{\prime }({\beta }_k)$, we obtain that it is zero at β_k = 0 with probability larger than any 1 − ε.

Proof of Theorem 4

(i) immediately follows by Lemma 1. Denote the first d₁ equations in ${\sum }_{i=1}^n{\mathcal{L}}_i\{{({\beta }_I^T,0^T)}^T,\widehat{\theta }\}$ as L_n{β_I, θ̂(β_I)}, and that in ${\sum }_{i=1}^n{\mathrm{\Psi }}_i({\beta }_0,{\theta }_0)\mathcal{U}(Z_i)$ as G_n(β_I0, θ₀). Note that the d₁ × d₁ upper left block of J⁻¹ is the matrix A defined in Theorem 4. Using the Taylor expansion for the penalized estimating function at $\beta ={({\beta }_I^T,0)}^T$, the first d₁ equations yield

$$\begin{array}{l}0=L_n\{{\widehat{\beta }}_I,\widehat{\theta }({\widehat{\beta }}_I)\}-n{p}_{{\lambda }_n,I}^{\prime }({\widehat{\beta }}_I)\\ =L_n({\beta }_{I0},{\theta }_0)-G_n({\beta }_{I0},{\theta }_0)+nA({\widehat{\beta }}_I-{\beta }_{I0})-n{b}_n-n\{{\mathrm{\sum }}_n+o_p(1)\}({\widehat{\beta }}_I-{\beta }_{I0})+o_p(d_n^{1/2}{n}^{1/2})\\ =L_n({\beta }_{I0},{\theta }_0)-G_n({\beta }_{I0},{\theta }_0)+n(A-{\mathrm{\sum }}_n)\left[{\widehat{\beta }}_I-{\beta }_{I0}-{(A-{\mathrm{\sum }}_n)}^{-1}{b}_n\right]+o_p(d_n^{1/2}{n}^{1/2}).\end{array}$$

Using condition (B2), we have

$$\begin{array}{l}{n}^{1/2}{v}^T{B}^{-1/2}\left\{(-A+{\mathrm{\sum }}_n)({\widehat{\beta }}_I-{\beta }_{I0})+b_n\right\}\\ =n^{-1/2}{v}^T{B}^{-1/2}\{L_n({\beta }_{I0},{\theta }_0)-G_n({\beta }_{I0},{\theta }_0)\}+o_p(v^T{B}^{-1/2})\\ =n^{-1/2}{v}^T{B}^{-1/2}\{L_n({\beta }_{I0},{\theta }_0)-G_n({\beta }_{I0},{\theta }_0)\}+o_p(1).\end{array}$$

Let Y_i = n^−1/2v^TB^−1/2 {ℒ_nIi(β_I0, θ₀) − (Z_i) Ψ_i(β_I0, θ₀)}, i = 1, …, n. It follows that for any ε > 0,

$$\sum _{i=1}^{n}E{\left|\right|Y_i\left|\right|}^2\mathbf{1}(|\left|Y_i\right||>\epsilon )=nE{\left|\right|Y_1\left|\right|}^2\mathbf{1}(|\left|Y_1\right||>\epsilon )\le n{(E{\left|\right|Y_1\left|\right|}^4)}^{1/2}{\{P_r(\left|\right|Y_1\left|\right|>\epsilon )\}}^{1/2}.$$

Using the Chebyshev inequality, we have P_r(||Y₁|| > ε) = O(n⁻¹), and E(||Y₁||⁴) is bounded by

$$n^{-2}{\lambda }_{max}^2(B^{-1})E{[{\{{\mathcal{L}}_{nI1}({\beta }_{I0},{\theta }_0)-{\mathcal{U}}_{nI}(Z_1){\mathrm{\Psi }}_1({\beta }_{I0},{\theta }_0)\}}^T\{{\mathcal{L}}_{nI1}({\beta }_{I0},{\theta }_0)-{\mathcal{U}}_{nI}(Z_1){\mathrm{\Psi }}_1({\beta }_{I0},{\theta }_0)\}]}^2$$

which equals $n^{-2}{\lambda }_{max}^2(B^{-1})E{\left|\right|\{{\mathcal{L}}_{nI1}({\beta }_{I0},{\theta }_0)-{\mathcal{U}}_{nI}(Z_1){\mathrm{\Psi }}_1({\beta }_{I0},{\theta }_0)\}\left|\right|}^4=O(d_2^2{n}^{-2})$ by condition (B2). Hence,

$$\sum _{i=1}^{n}E{\left|\right|Y_i\left|\right|}^2\mathbf{1}(|\left|Y_i\right||>\epsilon )=O(n{d}_n{n}^{-1}{n}^{-1/2})=o(1).$$

On the other hand,

$$\sum _{i=1}^{n}cov(Y_i)=ncov[n^{-1/2}{v}^T{B}^{-1/2}\{{\mathcal{L}}_{nI1}({\beta }_{I0},{\theta }_0)-{\mathcal{U}}_{nI}(Z_1){\mathrm{\Psi }}_1({\beta }_{I0},{\theta }_0)\}]=1.$$

(ii) follows by Lindeberg-Feller central limit theorem.