Penalized Quadratic Inference Function-Based Variable Selection for Generalized Partially Linear Varying Coefficient Models with Longitudinal Data

Abstract

Semiparametric generalized varying coefficient partially linear models with longitudinal data arise in contemporary biology, medicine, and life science. In this paper, we consider a variable selection procedure based on the combination of the basis function approximations and quadratic inference functions with SCAD penalty. The proposed procedure simultaneously selects significant variables in the parametric components and the nonparametric components. With appropriate selection of the tuning parameters, we establish the consistency, sparsity, and asymptotic normality of the resulting estimators. The finite sample performance of the proposed methods is evaluated through extensive simulation studies and a real data analysis.

1. Introduction

Identifying the significant variables is of great significance in all regression analysis. In practice, a number of variables are available for an initial analysis, but many of them may not be significant and should be excluded from the final model in order to increase the accuracy of prediction. Various procedures and criteria, such as stepwise selection and subset selection with Akaike information criterion (AIC), Mallows Cp, and Bayesian information criterion (BIC), have been developed. Nevertheless, these selection methods suffer from expensive computational costs. Many shrinkage methods have been developed for the purpose of computational efficiency, e.g., the nonnegative garrote [1], the LASSO [2], the bridge regression [3], the SCAD [4], and the one-step sparse estimator [5]. Among those, the SCAD possesses the virtues of continuity, unbiasedness, and sparsity. There are a number of works on the SCAD estimation methods in various regression models, e.g., [6–9]. Zhao and Xue [8] proposed a variable selection method to select significant variables in the parametric components and the nonparametric components simultaneously for the varying coefficient partially linear models (VCPLMs).

On the other hand, longitudinal data occurs frequently in biology, medicine, and life science, in which it is often necessary to make repeated measurements of subjects over time. The responses from different subjects are independent, but the responses from the same subject are very likely to be correlated. This feature is called “within-cluster correlation”. Qu et al. [10] proposed a method of quadratic inference functions (QIFs) to treat the longitudinal data. The QIF can efficiently take the within-cluster correlation into account and is more efficient than the generalized estimating equation (GEE) [11] approach when the working correlation is misspecified. The QIF approach has been applied to many models, including varying coefficient models (VCM) [12, 13], partially linear models (PLM) [14], varying coefficient partially linear models (VCPLMs) [15], and generalized partially linear models (GPLM) [16]. Wang et al. [13] proposed a group SCAD procedure for variable selection of VCM with longitudinal data. More recently, Tian et al. [15] proposed a QIF-based SCAD penalty for the variable selection for VCPLM with longitudinal data.

As introduced in Li and Liang [17], the generalized partially linear varying coefficient model (GPLVCM) possesses the great flexibility of a nonparametric regression model and provides the explanatory power of a generalized linear regression model, which arises naturally due to categorical covariates. Many models are the special case of GPLVCM, e.g., VCM, VCPLM, PLM, and GLM. Li and Liang [17] studied variable selection for GPLVCM, where the parametric components are identified via the SCAD but the nonparametric components are selected via a generalized likelihood ratio test instead of shrinkage. In this paper, we extend the QIF-based group SCAD variable selection procedure to GPLVCM with longitudinal data, and the B-spline methods are adopted to approximate the nonparametric component in the model. With suitable chosen tuning parameters, the proposed variable selection procedure is consistent, and the estimators of regression coefficients have oracle property, i.e., the estimators of the nonparametric components achieve the optimal convergence rate, and the estimators of the parametric components have the same asymptotic distribution as that based on the correct submodel.

The rest of this paper is organized as follows. In Section 2, we propose a variable selection procedure for the GPLVCM with longitudinal data. Asymptotic properties of the resulting estimators and an iteration algorithm are presented in Section 3. In Section 4, we carry out simulation studies to assess the finite sample performance of the method. A real data analysis is given in Section 5 to illustrate the proposed methodology. The details of proofs are provided in the appendix.

2. Methodology

2.1. GPLVCM with Longitudinal Data

In this article, we consider a longitudinal study with n subjects and m_i observations over time for the ith subject (i = 1, ⋯, n) for a total of N = ∑_i=1ⁿ m_i observations. Each observation consists of a response variable Y_ij and the predicator variables (X_ij, Z_ij, U(ij)), where X_ij ∈ R^p, Z_ij ∈ R^q and U_ij is a scalar. We assume that the observations from different subjects are independent, but those within the same subject are dependent. The generalized varying coefficient partially linear model (GPLVM) with longitudinal data takes the form

$$\begin{array}{c}{\mu }_{ij}=E\left(Y_{ij}\hspace{0.17em}\mid \hspace{0.17em}{X}_{ij},Z_{ij},U_{ij}\right)=h\left(X_{ij}^T\beta +Z_{ij}^T\alpha \left(U_{ij}\right)\right),\end{array}$$ (1)

where μ_ij is the expectation of Y_ij when X_ij, Z_ij, and U_ij are given, β = (β₁,⋯,β_p)^T is an unknown p × 1 regression coefficient vector, h(·) is a known smooth link function, and α(u) = (α₁(u), α₂(u),⋯,α_q(u))^T is a q × 1 unknown monotonic smooth function vector. Without loss of generality, we assume U ~ U[0, 1].

We approximate α(·) by B-spline basis functions B(u) = (B₁(u),⋯,B_L(u))^T with the order of M, where L = K + M + 1 and K is the number of interior knots, i.e.,

$$\begin{array}{c}{\alpha }_k\left(u\right)\approx {\alpha }_k^{\ast }\left(u\right)=B{\left(u\right)}^T{\gamma }_k,\hspace{1em}k=1,\cdots ,q,\end{array}$$ (2)

where γ_k = (γ_k1,⋯,γ_kL)^T is a L × 1 vector of unknown regression coefficients. Accordingly, μ_ij is approximated by

$$\begin{array}{c}{\mu }_{ij}=E\left(Y_{ij}\hspace{0.17em}\mid \hspace{0.17em}{X}_{ij},Z_{ij},U_{ij}\right)=h\left(X_{ij}^T\beta +Z_{ij}^T·I_q\otimes B{\left(U_{ij}\right)}^T\gamma \right),\end{array}$$ (3)

where γ = (γ₁^T, ⋯,γ_q^T)^T and “⊗” is the Kronecker product. We use the B-spline basis functions because they are numerically stable and have bounded support [18]. The spline approach also treats a nonparametric function as a linear function with the basis functions as pseudodesign variables, and thus, any computational algorithm for the generalized linear models can be used for the GPLVCMs.

To incorporate the within-cluster correlation, we apply the QIFs to estimate β and γ, respectively. Denote θ = (β^T, γ^T)^T, we define the extended score g_N(θ) as follows:

$$\begin{array}{c}{g}_N\left(\theta \right)=\frac{1}{n}\sum _{i=1}^n{g}_i\left(\theta \right)=\frac{1}{n}\sum _{i=1}^n\left(\begin{array}{c}{\dot{\mu }}_i^T{A}_i^{-\frac{1}{2}}{M}_1{A}_i^{-\frac{1}{2}}\left(Y_i-{\mu }_i\right)\\ ⋮\\ {\dot{\mu }}_i^T{A}_i^{-\frac{1}{2}}{M}_s{A}_i^{-\frac{1}{2}}\left(Y_i-{\mu }_i\right)\\ \end{array}\right),\end{array}$$ (4)

where ${\dot{\mu }}_i=\partial {\mu }_i/\partial \theta$, A_i = diag(Var(Y_i1), ⋯, Var(Y_im)) is the marginal variance matrix of subject Y_i, and M₁, ⋯, M_s are the base matrices to represent the inverse of the working correlation matrix R in GEE approach. Following Qu et al. [10], we define the quadratic inference functions to be

$$\begin{array}{c}{\mathcal{Q}}_n\left(\theta \right)=n{g}_N^T\left(\theta \right){\Omega }_n{\left(\theta \right)}^{-1}{g}_N\left(\theta \right),\end{array}$$ (5)

where Ω_n(θ) = (1/n)∑_i=1ⁿ g_i(θ)g_i(θ)^T. Note that Ω_n depends on θ. The QIF estimate $\tilde{\theta }$ is then given by

$$\begin{array}{c}\tilde{\theta }=\text{argmi}{\text{n}}_{\theta }{\mathcal{Q}}_n\left(\theta \right).\end{array}$$ (6)

2.2. Penalized QIF

In real data analysis, the true regression model is always unknown. An overfitted model lowers the efficiency of estimation while an underfitted one leads to a biased estimator. A popular approach to identify the relevant predictors while estimating the nonzero parameters and functions in model (1) simultaneously is to exert some kind of “penalty” on the original objective function. Here, we choose the smoothly clipped absolute deviation (SCAD) penalty because it has several advantages such as unbiasedness, sparsity, and continuity. The SCAD-penalized quadratic inference function (PQIF) is defined as follows:

$$\begin{array}{c}{\mathcal{Q}}_n^p\left(\theta \right)={\mathcal{Q}}_n\left(\theta \right)+n\sum _{l=1}^p{p}_{{\lambda }_1}\left(\left|{\beta }_l\right|\right)+n\sum _{k=1}^q{p}_{{\lambda }_2}\left({‖{\gamma }_k‖}_H\right),\end{array}$$ (7)

where ‖γ_k‖_H = (γ_k^THγ_k)^1/2, H = (h_ij)_L×L, h_ij = ∫₀¹ B_i(u)B_j^T(u)du and p_λ is the SCAD penalty function, where the derivative is defined as

$$\begin{array}{c}p{\prime }_l\left(\omega \right)=\lambda \left\{I\left(\omega ⩽l\right)+\frac{{\left(a\lambda -\omega \right)}_{+}}{\left(a-1\right)\lambda }I\left\{\omega >\lambda \right\}\right\},\end{array}$$ (8)

where a > 2, ω > 0, p_λ(0) = 0; here, we choose a = 3.7 as in [4].

Note that

$$\begin{array}{c}{‖{\gamma }_k‖}_H={\left({\int }_0^1{\gamma }_k^{T}B\left(u\right)B^T\left(U\right){\gamma }_{k}du\right)}^{1/2}={\left({\int }_0^1{\left[{\alpha }^{\ast }\left(u\right)\right]}^{2}du\right)}^{1/2}.\end{array}$$ (9)

This group-wised penalization ensures that the spline coefficient vector of the same nonparametric component is treated as an entire group in model selection.

Denote $\widehat{\theta }$ to be the penalized estimator obtained by minimizing the penalized objective function of (7). Then, $\widehat{\beta }={\left({\stackrel{\wedge }{\theta }}_1,\cdots ,{\stackrel{\wedge }{\theta }}_p\right)}^T$ is the estimator of the parameter β and the estimator of the nonparametric function α(u) is calculated by $\widehat{\alpha }\left(u\right)=B{\left(u\right)}^T\widehat{\gamma }$, where $\widehat{\gamma }={\left({\stackrel{\wedge }{\gamma }}_1^T,\cdots ,{\stackrel{\wedge }{\gamma }}_q^T\right)}^T={\left({\stackrel{\wedge }{\theta }}_{p+1},\cdots ,{\stackrel{\wedge }{\theta }}_{p+L},{\stackrel{\wedge }{\theta }}_{p+L+1},\cdots ,{\stackrel{\wedge }{\theta }}_{p+qL}\right)}^T$.

3. Asymptotic Properties

3.1. Oracle Property

We next establish the asymptotic properties of the resulting penalized QIF estimators. We first introduce some notations. Let β₀ and α₀(·) denote the true values of β(·) and α(·). In addition, γ₀ is the spline coefficient vector from the spline approximation to α₀(·). Without loss of generality, we assume that β_0l ≠ 0, l = 1, ⋯, p₁ and β_0l = 0, l = p₁ + 1, ⋯, p, i.e., only the first p₁ component of β₀ is nonzero. Similarly, we assume that α_0k(·) ≠ 0, k = 1, ⋯, q₁ and α_0k(·) = 0, k = q₁ + 1, ⋯, q, i.e., only the first q₁ component of α₀(·) is nonzero. For convenience and simplicity, let C denote a positive constant that may have different values at each appearance throughout this paper and ||A|| denote the modulus of the largest singular value of matrix or vector A. Before the proof of our main theorems, we list some regularity conditions used in this paper.

Assumption 1 (A1). —

The spline regression parameter γ is identifiable, that is, γ₀ is the spline coefficient vector from the spline approximation to α₀(·). In addition, there is a unique θ₀ = (β₀, γ₀) ∈ S satisfying E{g_N(θ₀)} = 0, where S is the parameter space.

Assumption 2 (A2). —

The weight matrix Ω_n = (1/n)∑_i=1ⁿ g_i(θ)g_i^T(θ) converges almost surely to a constant matrix Ω₀, where Ω₀ is invertible.

Assumption 3 (A3). —

The covariate matrices X_i and Z_i, i = 1, ⋯, n, satisfy sup_iE‖X_i‖⁴ < ∞ and sup_iE‖Z_i‖⁴ < ∞.

Assumption 4 (A4). —

The error ε_i = Y_i − μ_i satisfies E(ε_iε_i^T) = V_i, sup_i‖V_i‖ < ∞, and there exists a positive constant δ such that sup_iE‖ε_i‖^2+δ < ∞.

Assumption 5 (A5). —

All marginal variances A_i ≥ 0 and sup_i‖A_i‖ < ∞.

Assumption 6 (A6). —

{m_i} is a bounded sequence of positive integers.

Assumption 7 (A7). —

α _i(u), i = 1, 2, ⋯, q is rth continuous differentiable on (0, 1), where r ≥ 2.

Assumption 8 (A8). —

The inner knots {c_i, i = 1, ⋯, K} satisfy

$$\begin{array}{c}\underset{1\le i\le K}{\max }\left|h_{i+1}-h_i\right|=o\left(K^{-1}\right),\\ \frac{\max h_i}{\min h_i}\le C_0,\end{array}$$ (10)

where h_i = c_i − c_i−1.

Assumption 9 (A9). —

The link function h(·) is 2th continuous differentiable and E{h^2+δ} < ∞ for some δ > 2.

Assumption 10 (A10). —

a _n = O(n^−1/2); b_n⟶0 as n⟶∞, where

$$\begin{array}{c}{a}_n=\underset{k,l}{\max }\left\{\left|p{\prime }_{{\lambda }_1}\left(\left|{\beta }_{0l}\right|\right)\right|,\left|p{\prime }_{{\lambda }_2}\left({‖{\gamma }_{0k}‖}_H\right)\right|\text{:}{\beta }_{ol}\ne 0,{\gamma }_{0k}\ne 0\right\},\\ b_n=\underset{k,l}{\max }\left\{\left|p\prime {\prime }_{{\lambda }_1}\left(\left|{\beta }_{0l}\right|\right)\right|,\left|p\prime {\prime }_{{\lambda }_2}\left({‖{\gamma }_{0k}‖}_H\right)\right|\text{:}{\beta }_{ol}\ne 0,{\gamma }_{0k}\ne 0\right\}.\end{array}$$ (11)

Theorem 1 indicates that the estimator of nonparametric components achieve the optimal convergence rate.

Theorem 1 . —

Assume that Assumptions (A.1)–(A.10) hold and the number of knots K = O(N^{1/(2r + 1)}), then

$$\begin{array}{c}‖{\widehat{\alpha }}_k\left(·\right)-{\alpha }_{0k}\left(·\right)‖=O_p\left(n^{-r/\left(2r+1\right)}\right),\hspace{1em}k=1,\cdots ,q.\end{array}$$ (12)

Furthermore, under suitable condition, Theorem 1 shows that the penalized QIF estimator has the sparsity property.

Theorem 2 . —

Assume that the conditions in Theorem 1 hold and ${\lambda }_{\max }⟶0,\sqrt{n}{\lambda }_{\min }⟶\infty$ as n⟶∞, with probability approaching 1,

$$\begin{array}{c}{\widehat{\beta }}_l=0,\hspace{1em}l=p_1+1,\cdots ,p,\\ {\widehat{\alpha }}_k\left(·\right)\equiv 0,\hspace{1em}k=q_1+1,\cdots ,q,\end{array}$$ (13)

where λ_max = max{λ₁, λ₂}, λ_min = min{λ₁, λ₂}.

Theorems 1 and 2 indicate that with the tune parameter λ being suitably chosen, the proposed selection method possesses model selection consistency. Next, we establish the asymptotic property for the estimator of the nonzero parametric components. Let β^∗ = (β₁, ⋯,β_p1)^T, α^∗(·) = (α₁^∗(·), ⋯,α_q1^∗(·))^T and let β₀^∗ and α₀^∗(·) denote their true value, respectively. In addition, let γ^∗ = (γ₁^T, ⋯,γ_q1^T)^T and γ₀^∗ = (γ₀₁^T, ⋯,γ_0q1^T)^T denote the spline coefficient vector of α^∗(·) and α₀^∗(·), respectively, and let X_i^∗ and Z_i^∗, i = 1, ⋯, n denote their correspondent covariate. Let ${\tilde{X}}_i=H^{\prime }\left({\eta }_i\right)X_i^{\ast },\tilde{X}=\left({\tilde{X}}_1^T,\cdots ,{\tilde{X}}_n^T\right),{\tilde{W}}_i=H^{\prime }\left({\eta }_i\right)W_i^{\ast },\tilde{W}=\left({\tilde{W}}_1^T,\cdots ,{\tilde{W}}_n^T\right)$, and

$$\begin{array}{c}\mathrm{\Gamma }=E\left\{{\tilde{X}}^T\tau \tilde{X}-E\left[{\tilde{X}}^T\tau \tilde{W}\left|u\right.\right]E{\left[{\tilde{W}}^T\tau \tilde{W}\left|u\right.\right]}^{-1}{\tilde{W}}^T\tau \tilde{X}\right\},\\ \mathrm{\Delta }=E{\left\{\left[\tau -E\left[{\tilde{X}}^T\tau \tilde{W}\left|u\right.\right]E{\left[{\tilde{W}}^T\tau \tilde{W}\left|u\right.\right]}^{-1}E\left[{\tilde{W}}^T\tau \left|u\right.\right]\right]\epsilon \right\}}^{\otimes 2},\end{array}$$ (14)

where Δ^⊗2 = ΔΔ^T, τ = (τ_ij)_n×n is a n × n block matrix with its (i, j) block taking the form

$$\begin{array}{c}{\tau }_{ij}=\sum _{k=1}^s\sum _{l=1}^s{A}_i^{-1/2}{M}_k{A}_i^{-1/2}{H}^{\prime }\left({\eta }_i\right)P_i^{\ast }{\Omega }_{lk}^{-1}{P}_j^{\ast T}{H}^{\prime }\left({\eta }_j\right)A_j^{-1/2}{M}_l{A}_j^{-1/2}.\end{array}$$ (15)

Theorem 3 states that ${\stackrel{\wedge }{\beta }}^{\ast }$ is asymptotically normally distributed.

Theorem 3 . —

Suppose that Assumptions (A.1)–(A.9) hold and the number of knots K = O(N^{1/(2r + 1)}), then

$$\begin{array}{c}\sqrt{n}\left({\stackrel{\wedge }{\beta }}^{\ast }-{\beta }_0^{\ast }\right)\stackrel{L}{⟶}N\left(0,\mathrm{\Sigma }\right),\end{array}$$ (16)

where Σ = (ΓΔ⁻¹Γ)⁻¹ and $\stackrel{L}{⟶}$ represents the convergence in distribution.

3.2. Selection of Tuning Parameters

Theorems 1–3 imply that the proposed variable selection procedure possessed the oracle property. However, this attractive feature relies on the choice of tuning parameters λ_i. The popular criteria to choose λ_i include cross-validation, generalized cross-validation, AIC, and BIC. Wang et al. [19] suggested using BIC for the SCAD estimator in linear models and partially linear models and proved its model selection consistency property, i.e., the optimal parameter chosen by BIC can identify the true model with probability tending to one. Tian proved that for partially linear models. Hence, we adopt BIC to choose the optimal {λ₁, λ₂}. Following [19–21], we simplify the tuning parameters as

$$\begin{array}{c}{\lambda }_1=\frac{{\lambda }_0}{{‖{\tilde{\gamma }}_k^{\left(0\right)}‖}_H},\\ {\lambda }_2=\frac{{\lambda }_0}{\left|{\tilde{\beta }}_k^{\left(0\right)}\right|},\end{array}$$ (17)

where ${\tilde{\beta }}_k^{\left(0\right)}$ and ${\tilde{\gamma }}_k^{\left(0\right)}$ are the unpenalized QIF estimates. Consequently, the original two-dimensional problem becomes a univariate problem about λ₀, which can be selected according to the following BIC-type criterion:

$$\begin{array}{c}\text{BI}{\text{C}}_{\lambda }={\mathcal{Q}}_n^p\left({\widehat{\theta }}_{\lambda }\right)+d{f}_{\lambda }\times \log \left(n\right),\end{array}$$ (18)

where ${\widehat{\theta }}_{\lambda }=\left({\widehat{\beta }}_{\lambda },{\widehat{\gamma }}_{1\lambda }^T,\cdots ,{\widehat{\gamma }}_{q\lambda }^T\right)$ is the regression coefficient estimated by minimizing the penalized QIF in (2.8) for a given λ and df_λ is the number of nonzero coefficients of ${\widehat{\beta }}_{\lambda }$ and ${‖{\widehat{\gamma }}_{1\lambda }‖}_H,\cdots ,{‖{\widehat{\gamma }}_{q\lambda }‖}_H$. Thus, the tuning parameter λ is obtained by

$$\begin{array}{c}\widehat{\lambda }=\arg \underset{\lambda }{\min }\text{BI}{\text{C}}_{\lambda }.\end{array}$$ (19)

From Theorem 4 of Tian et al. [15], the BIC tuning parameter selector enables us to select the true model consistently.

3.3. An Algorithm Using Local Quadratic Approximation

Based on Fan and Li's local quadratic approximating approach [4], we propose an iterative algorithm to minimize the PQIF (7). Similar with Tian et al. [15], we choose the unpenalized QIF estimator $\tilde{\mathbf{\theta }}$ as the initial estimator. Let θ^k = (β₁^k, ⋯,β_p^k, γ₁^kT, ⋯,γ_q^kT)^T be the value of θ at the kth iteration. If β_l^k (or γ_l^k) is close to 0 (or 0), i.e., |β_l^k| ⩽ ϵ (or ‖γ_l^k‖_H ⩽ ϵ) with some small threshold value ϵ, then we set β_l^k = 0 (or γ_l^k = 0). We consider ϵ = 10⁻⁶ in our simulations.

Suppose β_l^k+1 = 0, for l = p_k + 1, ⋯, p, and γ_l^k+1 = 0, for l = q_k + 1, ⋯, q, and β^k+1 = (β₁^k+1, ⋯,β_pk^k+1, β_pk+1^k+1, ⋯,β_p^k+1)^T = ((β_N^k+1)^T, (β_Z^k+1)^T)^T, where β_N^k+1 = (β₁^k+1, ⋯,β_pk^k+1)^T are the nonzero parametric components and β_Z^k+1 = (β_pk+1^k+1, ⋯,β_p^k+1)^T = 0. Similarly, let γ^k+1 = ((γ₁^k+1)^T, ⋯,(γ_qk^k+1)^T, (γ_qk+1^k+1)^T, ⋯,(γ_q^k+1)^T)^T = ((γ_N^k+1)^T, (γ_Z^k+1)^T)^T, where γ_N^k+1 = ((γ₁^k+1)^T, ⋯,(γ_qk^k+1)^T)^T and γ_Z^k+1 = ((γ_qk+1^k+1)^T, ⋯,(γ_q^k+1)^T)^T correspond to q_k zero functions and q − q_K zero functions, respectively. Let θ = (β_N^T, β_Z^T, γ_N^T, γ_Z^T)^T denote a vector which has the same length and same partition with θ^k+1.

For the parametric term, if |β_l^k| > ϵ, the penalty function at β_l ≈ β_l^k is approximated by

$$\begin{array}{c}{p}_{\lambda }\left({\beta }_l\right)\approx p_{\lambda }\left({\beta }_l^k\right)+\frac{1}{2}\frac{p_{\lambda }^{\prime }\left(\left|{\beta }_l^k\right|\right)}{\left|{\beta }_l^k\right|}\left({\beta }_l^2-{\left({\beta }_l^k\right)}^2\right).\end{array}$$ (20)

Similarly, to the nonparametric component, if ‖γ_l^k‖_H > ϵ, the penalty function at γ_l ≈ γ_l^k is approximated by

$$\begin{array}{c}{p}_{\lambda }\left({‖{\mathbf{\gamma }}_l‖}_H\right)\approx p_{\lambda }\left({‖{\mathbf{\gamma }}_l^k‖}_H\right)+\frac{1}{2}\frac{p_{\lambda }^{\prime }\left({‖{\mathbf{\gamma }}_l^k‖}_H\right)}{{‖{\mathbf{\gamma }}_l^k‖}_H}\left({{‖{\mathbf{\gamma }}_l‖}_H}^2-{{‖{\mathbf{\gamma }}_l^k‖}_H}^2\right)=p_{\lambda }\left({‖{\mathbf{\gamma }}_l^k‖}_H\right)+\frac{1}{2}\frac{p{\prime }_{\lambda }\left({‖{\mathbf{\gamma }}_l^k‖}_H\right)}{{‖{\mathbf{\gamma }}_l^k‖}_H}\left({\mathit{\beta }}_l^{T}H{\mathit{\beta }}_l-{\mathit{\beta }}_l^{kT}H{\mathit{\beta }}_l^k\right),\end{array}$$ (21)

where p′_λ is the first-order derivative of the penalty function p_λ. This leads to the local approximation of the PQIF 𝒬_n^p(θ) by a quadratic function:

$$\begin{array}{c}{\mathcal{Q}}_n\left({\mathbf{\theta }}^k\right)+{\dot{\mathcal{Q}}}_n{\left({\mathbf{\theta }}^k\right)}^T\left({\mathbf{\omega }}_{11}-{\mathbf{\omega }}_{11}^k\right)+\frac{1}{2}{\left({\mathbf{\omega }}_{11}-{\mathbf{\omega }}_{11}^k\right)}^T{\ddot{\mathcal{Q}}}_n\left({\mathbf{\theta }}^k\right)\left({\mathbf{\omega }}_{11}-{\mathbf{\omega }}_{11}^k\right)+\frac{n}{2}{\mathbf{\omega }}_{11}^t\mathrm{\Lambda }\left({\theta }^k\right){\mathbf{\omega }}_{11},\end{array}$$ (22)

where ${\dot{\mathcal{𝒬}}}_n\left({\mathbf{\theta }}^k\right)=\partial {\mathcal{𝒬}}_n\left({\mathbf{\theta }}^k\right)/\partial {\mathbf{\omega }}_{11}$, ${\ddot{\mathcal{𝒬}}}_n\left({\mathbf{\theta }}^k\right)={\partial }^2{\mathcal{𝒬}}_n\left({\mathbf{\theta }}^k\right)/\partial {\mathbf{\omega }}_{11}\partial {\mathbf{\omega }}_{11}^T$, with ω₁₁ = (β_N^T, γ_Z^T)^T, and

$$\begin{array}{c}\mathrm{\Lambda }\left({\mathbf{\theta }}^k\right)=\mathrm{diag}\left\{\frac{p{\prime }_{{\lambda }_2}\left(\left|{\beta }_1^k\right|\right)}{\left|{\beta }_1^k\right|},\cdots ,\frac{p{\prime }_{{\lambda }_2}\left({\beta }_{p_k}^k\right)}{\left|{\beta }_{p_k}^k\right|},\frac{p{\prime }_{{\lambda }_1}\left({‖{\mathbf{\gamma }}_1^k‖}_H\right)}{{‖{\mathbf{\gamma }}_1^k‖}_H}H,\cdots ,\frac{p{\prime }_{{\lambda }_1}\left({‖{\mathbf{\gamma }}_{q_k}^k‖}_H\right)}{{‖{\mathbf{\gamma }}_{q_k}^k‖}_H}H\right\}.\end{array}$$ (23)

Minimizing the quadratic function (22), we obtain ω₁₁^k+1. The Newton-Raphson method then iterates the following process to convergence:

$$\begin{array}{c}{\mathbf{\omega }}_{11}^{k+1}={\mathbf{\omega }}_{11}^k-{\left\{{\ddot{\mathcal{Q}}}_n\left({\mathbf{\omega }}_{11}^k\right)+n\mathrm{\Lambda }\left({\mathbf{\omega }}_{11}^k\right)\right\}}^{-1}\left\{{\dot{\mathcal{Q}}}_n\left({\mathbf{\omega }}_{11}^k\right)+n\mathrm{\Lambda }\left({\mathbf{\omega }}_{11}^k\right){\mathbf{\omega }}_{11}^k\right\}.\end{array}$$ (24)

4. Simulation Studies

4.1. Assessing Rule

In this section, we conduct a simulation study to assess the finite sample performance of the proposed procedures. Following [17], the performance of estimator $\widehat{\beta }$ will be assessed by the generalized mean square error (GMSE), which is defined as

$$\begin{array}{c}\text{GMSE}=\frac{1}{n}\sum _{i=1}^n\left(\widehat{\beta }-\beta \right)X_i^{\ast }{X_i^{\ast }}^T\left(\widehat{\beta }-\beta \right).\end{array}$$ (25)

The performance of estimator $\widehat{\alpha }\left(·\right)$ will be assessed by the square root of average square errors (RASE)

$$\begin{array}{c}\text{RASE}={\left\{\frac{1}{M}\sum _{v=1}^M\sum _{k=1}^q{\left[{\stackrel{\wedge }{\alpha }}_k\left(u_v\right)-{\alpha }_k\left(u_v\right)\right]}^2\right\}}^{1/2},\end{array}$$ (26)

where u_v, v = 1, ⋯, M are the grid points where the function $\widehat{\alpha }\left(u\right)$ is evaluated. In our simulation, M = 300 is used.

To assess the performance of the variable selection, we use “C” to denote the average number of zero regression coefficients that are correctly estimated as zero and use “IC” to denote the average number of nonzero regression coefficients that are erroneously set to zero. The more closer the value of “C” to the number of true zero coefficient in the model and the more closer the value of “IC” to zero, the better the performance of the variable selection procedure is.

In our simulations, we use the sample quantiles of U_ij as knots and take the number of internal knots to be 3, that is, O(N^1/5). This particular choice is consistent with the asymptotic theory in Section 3 and performs well in the simulations. For each simulated dataset, the proposed estimation procedures for finding out penalized QIF estimators with SCAD and LASSO penalty functions are considered. The tuning parameters λ₁, λ₂ for the penalty functions are chosen by BIC from 50 equispaced grid points in [−15, 5]. For each of these methods, the average of zero coefficients over the 500 simulated datasets is reported.

4.2. Study 1 (Partial Penalty)

Consider a Bernoulli response

$$\begin{array}{c}\text{logit}\left\{Y_{ij}\right\}=X_{ij}^T\beta +\alpha \left(U_{ij}\right),\end{array}$$ (27)

where β = (2,1.5,0.7, 0₁₇^T)^T, m = 6, X_ij ~ N(0, I₂₀), α(U_ij) = 0.4cos((π/2)U_ij), and U_ij are drawn independently from U[0, 1]. Response variable Y_ij with compound symmetry correlation structure (CS) is generated according to Oman [22]. In our simulation study, we consider ρ = 0.25 and 0.75, representing weak and strong correlations, respectively. In some situations, we prefer not to shrink some certain components in the variable selection procedure when some kind of prior information is available. Partial penalty arises naturally for such case. In this example, we only exert penalty on the parametric component, i.e., coefficient β. In this situation, the PQIF (7) becomes

$$\begin{array}{c}{\mathcal{Q}}_n^p\left(\theta \right)={\mathcal{Q}}_n\left(\theta \right)+n\sum _{l=1}^p{p}_{{\lambda }_1}\left({\beta }_l\right).\end{array}$$ (28)

The variable selection result is reported in Tables 1 and 2.

Table 1.

Variable selection for the parametric components under different methods.

	Method	n = 150			n = 200			n = 300
		GMSE	C	IC	GMSE	C	IC	GMSE	C	IC
ρ = 0.75	SCAD	0.0011	15.83	0	0.0006	16.246	0	0.0005	16.746	0
	LASSO	0.0006	14.81	0	0.0005	15.346	0	0.0004	15.574	0
ρ = 0.25	SCAD	0.0011	15.75	0	0.0006	16.70	0	0.0004	16.846	0
	LASSO	0.0007	14.82	0	0.0006	14.96	0	0.0005	15.35	0

Table 2.

RASE of $\widehat{\alpha }\left(u\right)$ under different methods.

	Method	n = 150	n = 200	n = 300
ρ = 0.75	SCAD	0.1920	0.2051	0.1054
	LASSO	0.0999	0.0840	0.1064
ρ = 0.25	SCAD	0.2449	0.2460	0.0694
	LASSO	0.1399	0.1205	0.1033

Tables 1 and 2 show that the performance of the proposed variable selection approach improves as n increases, e.g., the number of correctly recognized zero coefficient increases to the number of true zero coefficient in the model and the GMSE of $\widehat{\beta }$ decreases as n increases. In addition, the RASE of $\widehat{\alpha }\left(u\right)$ also decreases as n increases, which means the estimated curve of $\widehat{\alpha }\left(u\right)$ fits better to the true line of α(u) when the sample size increases. Moreover, the SCAD penalty method outperforms the LASSO penalty ones in the sense of correct variable selection rate, which significantly reduces the model uncertainty and complexity.

4.3. Study 2 (Fixed-Dimensional Setup)

In this example, we generate data from the following model:

$$\begin{array}{c}\text{logit}\left\{Y_{ij}=1\hspace{0.17em}\mid \hspace{0.17em}{X}_{ij},U_{ij}\right\}=X_{ij}^T\beta +Z_{ij}^T\alpha \left(U_{ij}\right),\end{array}$$ (29)

where β = (2,1.5,0.7, 0₇^T) and α(u) = (α₁(u), α₂(u), 0₅^T)^T with α₁(u) = 0.8cos((π/2)u), α₂(u) = 1.5 + u², X_ij and Z_ij(j = 1, ⋯, 6) come from a multivariate normal distribution with mean zero, marginal variance 1 and correlation coefficient 0.5, and u ~ U(0, 1). Response variable Y_ij with compound symmetry correlation structure (CS) is generated by the same method as study 1 and we also consider ρ = 0.25 and 0.75, representing weak and strong correlations, respectively. We generated 500 datasets for each pair of (N, ρ). The results are also reported in Tables 3 and 4.

Table 3.

Variable selection for the parametric components under different methods.

	Method	n = 150			n = 200			n = 300
		GMSE	C	IC	GMSE	C	IC	GMSE	C	IC
ρ = 0.75	SCAD	0.0048	6.76	0	0.0036	6.846	0	0.0030	6.864	0
	LASSO	0.0039	4.694	0	0.0033	4.766	0	0.0028	5.074	0
ρ = 0.25	SCAD	0.0047	6.76	0	0.0035	6.718	0	0.0028	6.846	0
	LASSO	0.0038	4.814	0	0.0035	4.98	0	0.0029	5.048	0

Table 4.

Variable selection for the nonparametric components under different methods.

	Method	n = 150			n = 200			n = 300
		GMSE	C	IC	GMSE	C	IC	GMSE	C	IC
ρ = 0.75	SCAD	0.1696	4.35	0	0.1221	4.66	0	0.0812	4.83	0
	LASSO	0.1932	4.38	0	0.1540	4.36	0	0.1235	4.57	0
ρ = 0.25	SCAD	0.1636	4.42	0	0.1076	4.72	0	0.0344	4.85	0
	LASSO	0.1982	4.40	0	0.1160	4.68	0	0.0398	4.76	0

Table 3 reports the variable selection for the parametric components; it shows that the performances become better and better as n increases, e.g., the number of correctly recognized zero coefficients, which is denoted as values in the column labeled “C,” becomes more and more closer to the true number of zero regression coefficients in the model. At the same time, the GMSE decreases steadily as n increases. Table 4 shows that, for the nonparametric components, the performances of the proposed variable selection method are similar to those of the method for the parametric components. As n increases, the RASE of the estimated nonparametric function also becomes smaller and smaller. This reflects that the estimate curves fit better to the corresponding true line as the sample size increases. Moreover, the SCAD penalty method outperforms the LASSO penalty ones in the sense of correct variable selection rate, which significantly reduces the model uncertainty and complexity.

To study the influence of misspecified correlation structure to the proposed approach, we perform variable selection when the working correlation structure is specified to be CS and first-order autoregressive (AR-1), respectively. The result is listed in Table 5. It is known that the QIF estimator is insensitive to misspecification in correlation structure. Table 5 shows that the proposed variable selection procedure gives similar results even when the correlation structure is misspecified. This indicates that our method is robust.

Table 5.

Variable selection when the true R is CS when n = 300.

Working R	Method	β			α(·)
		GMSE	C	IC	RASE	C	IC
ρ = 0.75
CS	SCAD	0.0030	6.864	0	0.0812	4.83	0
	LASSO	0.0028	5.074	0	0.1235	4.57	0
AR-1	SCAD	0.0033	6.856	0	0.0935	4.82	0
	LASSO	0.0034	4.924	0	0.1230	4.57	0
ρ = 0.25
CS	SCAD	0.0028	6.846	0	0.0344	4.85	0
	LASSO	0.0029	5.048	0	0.0398	4.76	0
AR-1	SCAD	0.0030	6.846	0	0.0354	4.86	0
	LASSO	0.0031	5.048	0	0.0411	4.75	0

4.4. Study 3 (High-Dimensional Setup)

In this example, we discuss how the proposed variable selection procedure can be applied to the “large n, diverging p/q” setup for longitudinal models. We consider the high-dimensional setup of study 2. In this simulation, we take n = 300, m = 6, p = 20 = O(N^1/4), q = 10 = O(N^1/4). The true coefficient vector is β = (2,1.5,0.7, 0₁₇^T)^T, α(u) = (α₁(u), α₂(u), 0₁₀^T)^T, where α₁(u) and α₂(u) are defined in study 2. The other settings are the same with study 2. The results are reported in Table 6. It is easy to see that the proposed variable selection procedure is able to correctly identify the true model and works well in the “large n, diverging p/q” setup.

Table 6.

Variable selection under high-dimensional setup.

	Method	β			α(·)
		GMSE	C	IC	RASE	C	IC
ρ = 0.75	SCAD	0.0036	16.664	0	0.1148	9.656	0
	LASSO	0.0033	15.574	0	0.1239	9.546	0
ρ = 0.25	SCAD	0.0034	16.846	0	0.1047	9.875	0
	LASSO	0.0039	15.35	0	0.1138	9.802	0

5. Application to Infectious Disease Data

We apply the proposed method to analyze an infectious disease data (indon.data), which has been well analyzed by many authors, such as [16, 23–27]. In this study, a total of 275 preschool children were examined every three months for 18 months. The response is the presence of respiratory infection (1 = yes, 0 = no). The primary interest is in studying the relationship between the risk of respiratory infection and vitamin A deficiency (1 = yes, 0 = no).

In our study, we consider the following GPLVCM model

$$\begin{array}{c}\text{logit}\left\{{\mu }_{ij}\hspace{0.17em}\mid \hspace{0.17em}{X}_{ij},t_{ij}\right\}=\sum _{k=1}^6{\beta }_i{x}_{ij}+{\alpha }_0\left(t_{ij}\right)+z_{ij}{\alpha }_1\left(t_{ij}\right),\end{array}$$ (30)

where t is age, X₁ is vitamin A deficiency, X₂, X₃ are the seasonal cosine and seasonal sine variables, respectively, which indicate the season when those examinations took place, X₄ is gender (1 = female, 0 = male), X₅ is height, X₆ is stunting status (1 = yes, 0 = no), and Z₁ = X₅² is the square of height. The with-cluster correlation structure is assumed to be exchangeable, i.e., compound symmetric. This structure is also used in [16, 26, 27].

We apply the proposed QIF-based group SCAD variable selection procedure to the above model and recognize five nonzero coefficients and one nonzero function α₀(t), where β₁ = 0.842, β₂ = −0.685, β₃ = −0.309, β₄ = −0.554, and β₆ = 0.966. The results are generally consistent with those previous studies, but our results show that the height has no significant impact on the infectious rate and can be removed from the model. Figure 1 reports the curve of baseline age function α₀(t) estimated by QIF-based group SCAD that is estimated by QIF and that is estimated by QIF-based SCAD partial penalty to β in [16], where the GPLM without the varying coefficient term is used. Figure 1 implies that the probability of having respiratory infection increases at the very early stage, then decreases steadily, and declines dramatically when the age is over 5.5 years old. This also coincides with previous results [16, 26, 27].

Figure 1.

The estimated function on age for the infectious disease data.

6. Conclusion and Discussion

We proposed a QIF-based group SCAD variable selection procedure for the generalized partially linear varying coefficient models with longitudinal data. This procedure can select significant variables in the parametric components and nonparametric components simultaneously. Under mild conditions, the estimators of regression coefficients have oracle property. Simulation studies indicate that the proposed procedure is very effective in selecting significant variables and estimating the regression coefficients.

In this paper, we assume that the dimensions of the covariates X and Z are fixed. Study 3 in simulations shows that the proposed approach still have desired results when the dimensions p and q go to infinity as n⟶∞. However, when in ultrahigh-dimensional case, the proposed variable selection procedure may not work well anymore. As a future research topic, it is interesting to consider the variable selection for the generalized partially linear varying coefficient models with ultrahigh-dimensional covariates.

Appendix

A. Proofs of the Main Results

For convenience and simplicity, let C denote a positive constant that may have different values at each appearance throughout this paper and ‖A‖ denote the modulus of the largest singular value of matrix or vector A.

Let η_ij = X_ij^Tβ + Z_ij^T · I_q ⊗ B(U_ij)^Tγ, then μ_ij = h(η_ij). Let η_i = (η_i1, ⋯,η_im)^T, μ_i = (μ_i1, ⋯,μ_im)^T, and θ = (β^T, γ^T)^T, Y_i = (Y_i1, ⋯,Y_im)^T, X_i = (X_i1, ⋯,X_im)^T.

Similarly, let W_ij = B(U_ij) ⊗ I_q · Z_ij, P_ij = (X_ij^T, W_ij)^T, and W_i = (W_i1, ⋯,W_im))^T, P_i = (P_i1, ⋯,P_im)^T = (X_i, W(U_i)); then, η_ij = P_ij^Tθ, η_i = P_iθ, and ∂η_ij/∂θ = P_ij, ∂η_i/∂θ = P_i^T.

Let h′(t) = dh(t)/dt, then ∂μ_ij/∂θ = h′(η_ij)P_ij. Let

$$\begin{array}{c}{H}^{\prime }\left({\eta }_i\right)\triangleq \left(\begin{array}{ccc}{h}^{\prime }\left({\eta }_{i1}\right)& & \\ & \ddots & \\ & & h^{\prime }\left({\eta }_{im}\right)\\ & & \end{array}\right),H^{\prime \prime }\left({\eta }_i\right)\triangleq \left(\begin{array}{ccc}{h}^{\prime \prime }\left({\eta }_{i1}\right)& & \\ & \ddots & \\ & & h^{\prime \prime }\left({\eta }_{i,}\right)\\ & & \end{array}\right).\end{array}$$ (A.1)

Then,

$$\begin{array}{c}{\dot{\mu }}_i=\left(\begin{array}{ccc}\frac{\partial {\mu }_{i1}}{\partial {\beta }_1}& \cdots & \frac{\partial {\mu }_{i1}}{\partial {\gamma }_{qL}}\\ ⋮& \cdots & ⋮\\ \frac{\partial {\mu }_{im}}{\partial {\beta }_1}& \cdots & \frac{\partial {\mu }_{im}}{\partial {\gamma }_{qL}}\\ & & \end{array}\right)=\left(\begin{array}{c}{\left(\frac{\partial {\mu }_{i1}}{\partial \theta }\right)}^T\\ ⋮\\ {\left(\frac{\partial {\mu }_{im}}{\partial \theta }\right)}^T\\ \end{array}\right)=\left(\begin{array}{c}{P}_{i1}^T{h}^{\prime }\left({\eta }_{i1}\right)\\ ⋮\\ P_{im}^T{h}^{\prime }\left({\eta }_{im}\right)\\ \end{array}\right)=H^{\prime }\left({\eta }_i\right)P_i.\end{array}$$ (A.2)

Proof of Theorem 1. —

Let δ = n^−1/2, β = β₀ + δD₁, γ = γ₀ + δD₂, and D = (D₁^T, D₂^T)^T. We first show that for any given ε > 0, there exists a large constant C such that

$$\begin{array}{c}P\left\{\underset{‖D‖=C}{\inf }{\mathcal{Q}}_n^P\left(\beta ,\gamma \right)>{\mathcal{Q}}_n^P\left({\beta }_0,{\gamma }_0\right)\right\}\ge 1-\epsilon .\end{array}$$ (A.3)

Note that β_0l = 0, for all l = P₁ + 1, ⋯, p, and γ_0k = 0, for all k = q₁, ⋯, q, together with Assumption (A1) and p_λ(0) = 0, we have

$$\begin{array}{c}{\mathcal{Q}}_n^p\left(\theta \right)-{\mathcal{Q}}_n^p\left({\theta }_0\right)⩾\left[{\mathcal{Q}}_n\left(\theta \right)-{\mathcal{Q}}_n\left({\theta }_0\right)\right]+n\sum _{l=1}^{p_1}\left[p_{{\lambda }_2}\left(\left|{\beta }_l\right|\right)-p_{{\lambda }_2}\left(\left|{\beta }_{0l}\right|\right)\right]+n\sum _{k=1}^{q_1}\left[p_{{\lambda }_1}\left({‖{\gamma }_k‖}_H\right)-p_{{\lambda }_1}\left({‖{\gamma }_{0k}‖}_H\right)\right]\triangleq I_1+I_2+I_3.\end{array}$$ (A.4)

By Taylor expansion and Assumption (A4), we have

$$\begin{array}{c}{I}_2=n\sum _{l=1}^{p_1}\left[\delta p{\prime }_{{\lambda }_2}\left(\left|{\beta }_{0l}\right|\right)\mathrm{sgn}\left({\beta }_{0l}\right)\left|D_{1l}\right|+\delta p\prime {\prime }_{{\lambda }_2}\left(\left|{\beta }_{0l}\right|\right)\mathrm{sgn}\left({\beta }_{0l}\right){\left|D_{1l}\right|}^2\left\{1+o\left(1\right)\right\}\right]⩽\sqrt{p_1}{a}_n‖D‖O\left(n^{1/2}\right)+b_n{‖D‖}^{2}O\left(1\right)=\sqrt{p_1}‖D‖O\left(n^{-1/2}\right)+{‖D‖}^{2}o\left(1\right).\end{array}$$ (A.5)

Invoking the proof of Theorem 2 in Zhang and Xue [16],

$$\begin{array}{c}{I}_1={\mathcal{Q}}_n\left(\theta \right)-{\mathcal{Q}}_n\left({\theta }_0\right)=D^T{\dot{g}}_N^T\left({\theta }_0\right){\Omega }_n^{-1}\left({\theta }_0\right){\dot{g}}_N\left({\theta }_0\right)D+{‖D‖}^2{o}_p\left(1\right)+‖D‖O_p\left(1\right).\end{array}$$ (A.6)

By choosing a sufficient large C, I₁ dominates I₂. Similarly, I₁ dominates I₃ for a sufficient large C. Thus (A.3) holds, i.e., with probability at least 1 − ε, there exists a local minimizer $\widehat{\theta }$ that satisfies $‖\widehat{\theta }-{\theta }_0‖=O_p\left(\delta \right)$. Therefore, $‖\widehat{\gamma }-{\gamma }_0‖=O_p\left(n^{-1/2}\right)$ and $‖\widehat{\beta }-{\beta }_0‖=O_p\left(n^{-1/2}\right)$. Let R_k(u) = α_k(u) − B(U)^Tγ_k and γ_ok denote the spline coefficient vector from the spline approximation to α_k(·). From Assumptions (A7) and (A8) and Theorem 12.7 in [18], we get that ‖R_k(u)‖ = O(K^−r). Therefore,

$$\begin{array}{c}{‖{\stackrel{\wedge }{\alpha }}_k\left(u\right)-{\alpha }_{0k}\left(u\right)‖}^2={\int }_0^1{\left\{{\stackrel{\wedge }{\alpha }}_k\left(u\right)-{\alpha }_{0k}\left(u\right)\right\}}^{2}du={\int }_0^1{\left\{B{\left(u\right)}^T{\stackrel{\wedge }{\gamma }}_k-B{\left(u\right)}^T{\gamma }_{0k}+R_k\left(u\right)\right\}}^{2}du\le 2{\int }_0^1{\left\{B{\left(u\right)}^T{\stackrel{\wedge }{\gamma }}_k-B{\left(u\right)}^T{\gamma }_{0k}\right\}}^{2}du+2{\int }_0^1{R}_k{\left(u\right)}^{2}du=2{\left({\stackrel{\wedge }{\gamma }}_k-{\gamma }_{0k}\right)}^T{\int }_0^{1}B\left(u\right)B^T\left(u\right)du\left({\widehat{\gamma }}_k-{\gamma }_{0k}\right)+2{\int }_0^1{R}_k{\left(u\right)}^{2}du=O_p\left(n^{-2r/\left(2r+1\right)}\right).\end{array}$$ (A.7)

Thus, we complete the proof of Theorem 1.

Proof of Theorem 2. —

According to Theorem 2, in order to prove the first part of Theorem 2, we need only to prove that, for any γ satisfying ‖γ − γ₀‖ = O_p(n^−1/2) and for any β_l satisfying ‖β_l − β_0l‖ = O_p(n^−1/2), l = 1, ⋯, p₁, there exists a certain ϵ = Cn^−1/2 that satisfies, as n⟶∞, with probability tending to 1:

$$\begin{array}{c}\frac{\partial {\mathcal{Q}}_n^p\left(\beta ,\gamma \right)}{\partial {\beta }_l}>0,\text{for}\hspace{1em}0<{\beta }_l<ϵ,l=p_1+1,\cdots ,p,\end{array}$$ (A.8)

$$\begin{array}{c}\frac{\partial {\mathcal{Q}}_n^p\left(\beta ,\gamma \right)}{\partial {\beta }_l}<0,\text{for}\hspace{1em}-ϵ<{\beta }_l<0,l=p_1+1,\cdots ,p.\end{array}$$ (A.9)

These imply that the PQIF 𝒬_n^p(β, γ) reaches its minimum at β_l = 0, l = p₁ + 1, ⋯, p.

Following Lemmas 3 and 4 of [16], we have

$$\begin{array}{c}\frac{\partial {\mathcal{Q}}_n^p\left(\beta ,\gamma \right)}{\partial {\beta }_l}=\frac{\partial g_n^T\left(\beta ,\gamma \right)}{\partial {\beta }_l}{\Omega }_n^{-1}\left(\beta ,\gamma \right)g_n\left(\beta ,\gamma \right)+O_p\left(1\right)+np{\prime }_{{\lambda }_2\left(\hspace{0.17em}\mid \hspace{0.17em}{\beta }_l\hspace{0.17em}\mid \hspace{0.17em}\right)}\mathrm{sgn}\left({\beta }_l\right)=-2\sum _{i=1}^n{\left(\begin{array}{c}{\dot{\mu }}_i^T{A}_i^{-1/2}{M}_1{A}_i^{-1/2}\frac{\partial {\mu }_i}{\partial {\beta }_l}\\ ⋮\\ {\dot{\mu }}_i^T{A}_i^{-1/2}{M}_s{A}_i^{-1/2}\frac{\partial {\mu }_i}{\partial {\beta }_l}\end{array}\right)}^T{\Omega }_n^{-1}\left(\beta ,\gamma \right)g_n\left(\beta ,\gamma \right)+np{\prime }_{{\lambda }_2\left(\hspace{0.17em}\mid \hspace{0.17em}{\beta }_l\hspace{0.17em}\mid \hspace{0.17em}\right)}\mathrm{sgn}\left({\beta }_l\right)+O_p\left(1\right)=n^{1/2}\left[n^{1/2}{\lambda }_2\left\{{\lambda }_2^{-1}p{\prime }_{{\lambda }_2}\left(\hspace{0.17em}\mid \hspace{0.17em}{\beta }_l\hspace{0.17em}\mid \hspace{0.17em}\right)\mathrm{sgn}\left({\beta }_l\right)\right\}+O_p\left(1\right)\right].\end{array}$$ (A.10)

According to (8), the expression of the derivative of SCAD-penalized function, it is easy to see that lim_n→∞liminf_βl→0λ₂⁻¹p′_λ2(∣β_l∣) = 1. Together with Assumption (A10), λ₂n^1/2 > λ_minn^1/2⟶∞, it is clear that the sign of (A.10) is decided by that of β_l. This implies (A.8) and (A.9) hold. Thus, we complete the proof of the first part.

Similarly, we can prove that with probability tending to 1, ${\widehat{\gamma }}_k=0,k=q_1+1,\cdots ,q$. Note that ‖B(u)‖ = O(1) and ${\widehat{\alpha }}_k\left(u\right)=B^T\left(u\right){\widehat{\gamma }}_k$; the second part of Theorem 2 is proved. Thus, we complete the proof of Theorem 2.

Proof of Theorem 3. —

Let θ^∗ = (β^∗T, γ^∗T)^T and let P_i^∗ = (X_i^∗T, W_i^∗T)^T, i = 1, ⋯, n denote the covariates corresponding to θ^∗. Denote ${\dot{\mathcal{𝒬}}}_{1n}\left(\beta ,\gamma \right)$ and ${\dot{\mathcal{𝒬}}}_{2n}\left(\beta ,\gamma \right)$ to be the first derivatives of the PQIF 𝒬_n^p with respect to β and γ, respectively, i.e.,

$$\begin{array}{c}{\dot{\mathcal{Q}}}_{1n}\left(\beta ,\gamma \right)=\frac{\partial {\mathcal{Q}}_n^p\left(\beta ,\gamma \right)}{\partial \beta },\\ {\dot{\mathcal{Q}}}_{2n}\left(\beta ,\gamma \right)=\frac{\partial {\mathcal{Q}}_n^p\left(\beta ,\gamma \right)}{\partial \gamma }.\end{array}$$ (A.11)

By Theorems 1 and 2, ${\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T$ and ${\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T$ satisfies that

$$\begin{array}{c}{\dot{\mathcal{Q}}}_{1n}\left({\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T\right)={\mathbf{0}}^T,\\ {\dot{\mathcal{Q}}}_{2n}\left({\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T\right)={\mathbf{0}}^T.\end{array}$$ (A.12)

By the Taylor expansion, we have

$$\begin{array}{c}{\mathcal{Q}}_{1n}{\left|{}_{\left({\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T\right)}={\mathcal{Q}}_{1n}\right|}_{\left(\left({\beta }_0^{\ast T},{\mathbf{0}}^T\right){\left({\beta }_0^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\gamma }_0^{\ast T},{\mathbf{0}}^T\right)}^T\right)}+\frac{\partial {\mathcal{Q}}_{1n}}{\partial \beta }{\left|{}_{\theta =\tilde{\theta }}\left\{{\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T-{\left({\beta }_0^{\ast T},{\mathbf{0}}^T\right)}^T\right\}+\frac{\partial {\mathcal{Q}}_{1n}}{\partial \gamma }\right|}_{\theta =\tilde{\theta }}\left\{{\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T-{\left({\gamma }_0^{\ast T},{\mathbf{0}}^T\right)}^T\right\}+\sum _{i=1}^{p_1}np{\prime }_{{\lambda }_2}\left({\widehat{\beta }}_l\right)\mathrm{sgn}\left({\widehat{\beta }}_l\right),\end{array}$$ (A.13)

where $\tilde{\theta }$ is between ((β₀^∗T, 0^T)^T, (γ₀^∗T, 0^T)^T) and $\left({\left({\stackrel{\wedge }{\beta }}^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\stackrel{\wedge }{\gamma }}^{\ast T},{\mathbf{0}}^T\right)}^T\right)$. Apply the Taylor expansion to $p{\prime }_{{\lambda }_2}\left(\mid {\widehat{\beta }}_l\mid \right)$, we obtain

$$\begin{array}{c}p{\prime }_{{\lambda }_2}\left(\left|{\widehat{\beta }}_l\right|\right)=p{\prime }_{{\lambda }_2}\left(\left|{\beta }_{0l}\right|\right)+\left\{p\prime {\prime }_{{\lambda }_{2l}}\left(\left|{\beta }_{0l}\right|\right.+o_p\left(1\right)\right\}\left({\widehat{\beta }}_l-{\beta }_{0l}\right).\end{array}$$ (A.14)

By Assumption (A10), p′′_λ2(∣β_0l∣) = o_p(1). Note that p′_λ2l(∣β_0l∣) = 0 as λ_max⟶0; therefore, by Lemma 4 of [16] and through some calculation, we have

$$\begin{array}{c}{\left.\frac{1}{n}{\mathcal{Q}}_{1n}\right|}_{\left({\left({\beta }_0^{\ast T},{\mathbf{0}}^T\right)}^T,{\left({\gamma }_0^{\ast T},{\mathbf{0}}^T\right)}^T\right)}=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n\sum _{k=1}^s\sum _{l=1}^s\left\{X_i^{\ast T}{H}^{\prime }\left({\eta }_i\right)A_i^{-1/2}{M}_k{A}_i^{-1/2}{H}^{\prime }\left({\eta }_i\right)P_i^{\ast }·{\Omega }_{kl}^{-1}{P}_j^{\ast T}{H}^{\prime }\left({\eta }_j\right)A_j^{-1/2}{M}_l{A}_j^{-1/2}\left(Y_j-{\mu }_{0j}\right)\right\}+o_p\left(n^{-1/2}\right)=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n{X}_i^{\ast T}{H}^{\prime }\left({\eta }_i\right){\tau }_{ij}\left(Y_j-{\mu }_{0j}\right)+o_p\left(n^{-1/2}\right)=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n{\tilde{X}}_i^T{\tau }_{ij}\left(\tilde{R}\left(U_j\right)+{\epsilon }_j\right)+o_p\left(n^{-1/2}\right),\end{array}$$ (A.15)

where ${\tilde{X}}_i=H^{\prime }\left({\eta }_i\right)X_i^{\ast },\tilde{R}\left(U_i\right)=H^{\prime }\left({\eta }_i\right)R\left(U_i\right),{\Omega }_{kl}^{-1}$ is the (l, k) block of Ω⁻¹ and

$$\begin{array}{c}{\tau }_{ij}=\sum _{k=1}^s\sum _{l=1}^s{A}_i^{-1/2}{M}_k{A}_i^{-1/2}{H}^{\prime }\left({\eta }_i\right)P_i^{\ast }{\Omega }_{kl}^{-1}{P}_j^{\ast T}{H}^{\prime }\left({\eta }_j\right)A_j^{-1/2}{M}_l{A}_j^{-1/2}.\end{array}$$ (A.16)

Similarly, we have

$$\begin{array}{c}\begin{array}{cc}{\left.\frac{1}{n}\frac{\partial {\mathcal{Q}}_{1n}}{\partial \beta }\right|}_{\theta =\tilde{\theta }}& =-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n{\tilde{X}}_i^T{\tau }_{ij}{\tilde{X}}_j+o_p\left(n^{-1/2}\right),\\ & \\ & \end{array}\\ {\left.\frac{1}{n}\frac{\partial {\mathcal{Q}}_{1n}}{\partial \gamma }\right|}_{\theta =\tilde{\theta }}=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n{\tilde{X}}_i^T{\tau }_{ij}\tilde{W}\left(U_j\right)+o_p\left(n^{-1/2}\right),\end{array}$$ (A.17)

where $\tilde{W}\left(U_j\right)=H^{\prime }\left({\eta }_j\right)W^{\ast }\left(U_j\right),W^{\ast }\left(U_j\right)={\left(W_{j1}^{\ast },\cdots ,W_{jm}^{\ast }\right)}^T,W_{ij}^{\ast }=B\left(U_{ij}\right)\otimes I_q·Z_{ij}^{\ast }$. Hence,

$$\begin{array}{c}{\left.\frac{1}{n}{\mathcal{Q}}_{1n}\right|}_{\left({\left({\beta }_0^{\ast T},{\mathbf{0}}^{T}0\right)}^T,{\left({\gamma }_0^{\ast T},{\mathbf{0}}^{T}0\right)}^T\right)}=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n{\tilde{X}}_i^T{\tau }_{ij}\left\{{\tilde{X}}_j\left({\beta }_0^{\ast }-{\stackrel{\wedge }{\beta }}^{\ast }\right)+\tilde{W}\left(U_j\right)·\left({\gamma }_0^{\ast }-{\stackrel{\wedge }{\gamma }}^{\ast }\right)+\tilde{R}\left(U_j\right)+{\epsilon }_j\right\}+o_p\left({\stackrel{\wedge }{\beta }}^{\ast }-{\beta }_0^{\ast }\right),\\ {\left.\frac{1}{n}{\mathcal{Q}}_{2n}\right|}_{\left({\left({\beta }_0^{\ast T},{\mathbf{0}}^{T}0\right)}^T,{\left({\gamma }_0^{\ast T},{\mathbf{0}}^{T}0\right)}^T\right)}=-\frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n\tilde{W}{\left(U_i\right)}^T{\tau }_{ij}\left\{{\tilde{X}}_j\left({\beta }_0^{\ast }-{\stackrel{\wedge }{\beta }}^{\ast }\right)+\tilde{W}\left(U_j\right)·\left({\gamma }_0^{\ast }-{\stackrel{\wedge }{\gamma }}^{\ast }\right)+\tilde{R}\left(U_j\right)+{\epsilon }_j\right\}+o_p\left({\stackrel{\wedge }{\gamma }}^{\ast }-{\gamma }_0^{\ast }\right).\end{array}$$ (A.18)

Following the proof of Theorem 2 in [16], we prove (16). Thus, we complete the proof of Theorem 3.

Data Availability

The data can be downloaded from https://content.sph.harvard.edu/xlin/dat/indon.dat.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Supplementary Materials

Supplementary Materials

The R code presented in Word format for the real data analysis is included in the supplementary file.