A Regularized Estimation Approach for Case-cohort Periodic Follow-up Studies with an Application to HIV Vaccine Trials

Abstract

This paper discusses regression analysis of the failure time data arising from case-cohort periodic follow-up studies, and one feature of such data, which makes their analysis much more difficult, is that they are usually interval-censored rather than right-censored. Although some methods have been developed for general failure time data, it does not seem to exist an established procedure for the situation considered here. To address the problem, we present a semiparametric regularized procedure and develop a simple algorithm for the implementation of the proposed method. In addition, unlike some existing procedures for similar situations, the proposed procedure is shown to have the oracle property, and an extensive simulation is conducted and suggests that the presented approach seems to work well for practical situations. The method is applied to a HIV vaccine trial that motivated this study.

1. Introduction

The case-cohort design was proposed by Prentice (1986) and is commonly used as a means of cost reduction in collecting or measuring expensive covariates in large cohort studies. Among others, one area where the design is often used is epidemiological cohort studies, in which the outcomes of interest are often times to failure events such as AIDS, cancer, heart disease and HIV infection. Under the design, complete covariate information is obtained from only a random sub-cohort of the study subjects plus all of the cases or the subjects who experience the event. In addition to the incomplete nature on covariate information, another feature of the failure time data arising from case-cohort periodic follow-up studies is that the observations are usually interval-censored rather than right-censored (Sun 2006). By interval-censored data, we mean that the failure time of interest is known or observed only to belong to an interval instead of being observed exactly. It is easy to see that among others, most medical follow-up studies such as clinical trials will yield such data and interval-censored data include right-censored data as a special case. One real study that motivated this investigation is the HVTN 505 Trial to assess the efficacy of a DNA prime-recombinant adenovirus type 5 boost (DNA/rAd5) vaccine to prevent human immunodeficiency virus type 1 (HIV-1) infection as well as to explore or identify various biomarkers that are significantly related to the HIV-1 infection. More details about the trial are given below. In the following, we will discuss regression analysis of such failure time data with focus on variable or covariate selection.

The identification of relevant predictors or variable selection is a commonly asked question in statistical analysis and many methods have been developed for it, especially under the context of linear regression, such as forward selection, backward selection and best subset selection. Among them, the regularized or penalized estimation procedure, which optimizes an objective function with a penalty function, has recently become increasingly popular and in particular, many penalty functions have been proposed. For example, one of the early work was given by Tibshirani (1996), who proposed the least absolute shrinkage and selection operator (LASSO) penalty for linear regression models, and Fan and Li (2001) developed the smoothly-clipped absolute deviation (SCAD) penalty. Also Zou (2006) generalized the LASSO penalty to the adaptive LASSO (ALASSO) penalty, and Lv and Fan (2009) and Dicker et al. (2013) proposed the smooth integration of counting and absolute deviation (SICA) penalty and the seamless-L₀ (SELO) penalty, respectively.

Many authors have investigated the variable or covariate selection problem under the failure time analysis context too (Cai et al. 2009; Fan and Li 2002; Huang and Ma 2010; Martinussen and Scheike 2009). For example, Tibshirani (1997), Fan and Li (2002) and Zhang and Lu (2007) discussed the generalizations of the LASSO, SCAD and ALASSO penalty-based procedures, respectively, to the failure time data arising from the proportional hazards (PH) model, the most commonly used regression models for failure time data. More recently, Shi et al. (2014) extended the SICA penalty-based procedure to the same situation. In addition, there exist some methods in the literature for the failure time data arising from other regression models such as the additive hazards model (Martinussen and Scheike 2009; Lin and Lv 2013) and the accelerated failure time model (Cai et al. 2009; Huang and Ma 2010). However, most of the existing methods for failure time data only apply to right-censored data except the two parametric procedures given by Scolas et al. (2016) and Wu and Cook (2015), who considered interval-censored data arising from the PH model. In particular, the latter assumed that the baseline hazard function is a piecewise constant function and it is known that one drawback of this is that the piecewise constant function is neither continuous nor differentiable. More importantly, there is no theoretical justification available for both procedures.

There exists a great deal of literature on the analysis of failure time data from the case-cohort design but almost all is on right-censored data except Zhou et al. (2017b), who discussed regression analysis of interval-censored data under the PH model. In particular, Ni et al. (2016) and Ni and Cai (2017) discussed the variable selection problem but also with right-censored data. Although the number of covariates is often very large in these situations, there does not seem to exist an established procedure that is specifically developed for variable or covariate selection in the presence of interval censoring. Note that one main difference between right-censored and interval-censored data is that the latter have much more complex structures. One way for seeing this is to note the fact that for inference about the PH model based on right-censored data, a partial likelihood function can be derived and is commonly used. In particular, since it is free of the baseline hazard function, it is also used as the parametric objective function for all of the existing penalized variable selection procedures. In contrast, for the latter, no similar partial likelihood function or parametric objective function is available any more for the development of a penalized procedure. Nevertheless, in the following, we will overcome this difficulty by using the sieve approach and develop a sieve penalized estimation procedure with the use of Bernstein polynomials.

To present the proposed variable selection approach, we will first in Section 2 introduce some notation and assumptions that will be used throughout the paper and then describe the model and the basic ideas behind the method. In particular, a sieve space based on Bernstein polynomials is defined that naturally results in a parametric objective function for the variable selection procedure to be developed. The proposed sieve penalized estimation approach will be given in Section 3 and for the implementation of the procedure, a coordinate descent algorithm will be described, which is faster and can be easily implemented than the EM algorithm given in Wu and Cook (2015). In addition, the asymptotic properties of the proposed procedure including the oracle property are established. The method applies to interval-censored data both in general and from case-cohort studies. Section 4 presents some results obtained from an extensive simulation study conducted for the assessment of the proposed method, and they suggest that the method seems to work well for practical situations. In Section 5, the method is applied to the motivating HIV vaccine trial described above and Section 6 contains some discussion and remarks. As will be seen, the proposed method is also valid for interval-censored data arising from regular failure time studies.

2. Notation, Assumptions and Review

Consider a cohort study that consists of n independent subjects and let T_i and X_i denote the failure time of interest and a p-dimensional vector of covariates associated with subject i, i = 1, … , n. To describe the covariate effect, we will assume that given X_i, T_i follows the PH model given by

$$\lambda (t;{\mathit{X}}_i)={\lambda }_0(t)\text{exp}\{{\beta }^{\prime }{\mathit{X}}_i\}$$ (1)

where λ₀(t) denotes an unknown baseline hazard function and β is a vector of regression parameters. In the following, it will be supposed that the main goal is about inference on β with the focus on covariate selection or the identification of important covariates such as various biomarkers in the HVTN 505 Trial described above.

For each failure time T_i, we will assume that only an interval-censored observation is available and given by (L_i, R_i, Δ_1i, Δ_2i) with L_i ≤ R_i, where L_i and R_i denote two random observation times, Δ_1i = I(T_i ≤ L_i) and Δ_2i = I(L_i < T_i ≤ R_i). It is apparent that Δ_1i = 1 and Δ_1i = Δ_2i = 0 mean that one has the left- and right-censored observation on T_i, respectively. Under the case-cohort design, as described above, the covariates X_i’s are available only for the subjects from a sub-cohort and for those who have experienced the failure event of interest, meaning Δ_1i = 1 or Δ_2i = 1. Define δ_i = 1 if the covariate X_i is available or observed and 0 otherwise, i = 1, … , n. Then the observed data have the form

$$\{{\mathbf{O}}_i=(L_i,R_i,{\Delta }_{1i},{\Delta }_{2i},{\delta }_i,{\delta }_i{\mathit{X}}_i):i=1,\dots ,n\}.$$

If δ_i = 1 for all i, we have regular interval-censored data.

For estimation of the regression parameter β, as mentioned above, a partial likelihood function is available when one has right-censored data but for the case of interval-censored data, one usually has to deal with or estimate both β and the baseline cumulative hazard function ${\Lambda }_0(t)={\int }_0^t{\lambda }_0(s)ds$ together, which is clearly much more complicated. For this, one approach is to approximate Λ₀ by some parametric functions such as Bernstein polynomials (Ma et al. 2015; Zhou et al. 2017a). More specifically, let $\Theta =\{\theta =(\beta ,{\Lambda }_0)\in \mathcal{B}\otimes \mathcal{M}\}$ denote the parameter space of θ, where $\mathcal{B}=\{\beta \in {\mathcal{R}}^p,\Vert \beta \Vert \le M\}$ with M being a positive constant and $\mathcal{M}$ is the collection of all bounded and continuous nondecreasing, nonnegative functions over the interval [c, u]. Here c and u are usually taken to be min(L_i) and max(R_i), respectively. Also define the sieve space ${\Theta }_n=\{{\theta }_n=(\beta ,{\Lambda }_{0n})\in \mathcal{B}\otimes {\mathcal{M}}_n\}$, where

$${\mathcal{M}}_n=\left\{{\Lambda }_{0n}(t)=\sum _{k=0}^m{\varphi }_k{B}_k(t,m,c,u):\sum _{0\le k\le m}\mid {\varphi }_k\mid \le M_n,0\le {\varphi }_0\le {\varphi }_1\le \dots \le {\varphi }_m\right\}$$

with B_k(t, m, c, u) denoting the Bernstein basis polynomial defined as

$$B_k(t,m,c,u)=\left(\begin{array}{c}\hfill m\hfill \\ \hfill k\hfill \end{array}\right){\left(\frac{t-c}{u-c}\right)}^k{\left(1-\frac{t-c}{u-c}\right)}^{m-k},k=0,1,\dots ,m.$$

In the above, we define 0⁰ = 1 as usual, M_n is a positive constant, and m = o(n^v), denoting the degree of Bernstein polynomials, for some v ∈ (0, 1). More discussion about m will be given below.

Instead of the Bernstein polynomials, of course, one can employ other functions such as piecewise constant functions as used in Wu and Cook (2015). It is worth to note that although the two approximations may look similar, they are actually quite different. One major difference is that unlike the piecewise constant function, the Bernstein polynomial approximation is a continuous one and has some nice properties. More specifically, Bernstein polynomials have the optimal shape preserving property among all approximation polynomials (Zhou et al., 2017a) and can naturally model the non-negativity and monotonicity of Λ₀ with some simple restrictions that can be easily removed through reparameterization. In addition, they are easier to work with since they do not require the specification of interior knots. In consequence, as will be seen below, the resulting various functions have relatively simpler forms and also the resulting method can be easily implemented. In particular, a much faster and simpler algorithm can be developed for the implementation of the method than the EM algorithm developed in Wu and Cook (2015). Also it allows the establishment of the asymptotic properties of the resulting method including the oracle property. In addition, the method developed below can provide a better and more natural way than that in Wu and Cook (2015) for the estimation of a survival function, which is often of interest in medical studies among others.

For the selection of the sub-cohort, we will assume the independent Bernoulli sampling with the selection probability ρ ∈ (0, 1). It follows that we have P(δ_i = 1) = Δ_1i + Δ_2i + (1 − Δ_1i − Δ_2i)ρ. Then under the independent censoring assumption, for estimation, Zhou et al. (2017b) suggested to maximize the inverse probability weighted log-likelihood function

$$l_n(\theta )\equiv \sum _{i=1}^n{\pi }_i\log L_i(\beta ,{\Lambda }_0)$$ (2)

over the sieve space Θ_n. Here π_i = δ_i/{Δ_1i + Δ_2i + (1 − Δ_1i − Δ_2i)ρ},

$$\begin{gathered} L_i(\beta ,{\Lambda }_{0n})={\left[\text{exp}\{-{\Lambda }_{0n}(R_i)e^{{\beta }^{\prime }{\mathit{X}}_i}\}\right]}^{(1-{\Delta }_{1i}-{\Delta }_{2i})}\times {\left[1-\text{exp}\{-{\Lambda }_{0n}(L_i)e^{{\beta }^{\prime }{\mathit{X}}_i}\}\right]}^{{\Delta }_{1i}} \\ \times {\left[\text{exp}\{-{\Lambda }_{0n}(L_i)e^{{\beta }^{\prime }{\mathit{X}}_i}\}-\text{exp}\{-{\Lambda }_{0n}(R_i)e^{{\beta }^{\prime }{\mathit{X}}_i}\}\right]}^{{\Delta }_{2i}} \end{gathered}$$ (3)

and α = (α₀, … , α_m)′ with ϕ₀ = e^α₀ and ${\varphi }_k={\sum }_{i=0}^k{e}^{{\alpha }_i}$, 1 ≤ k ≤ m, the reparameterization of the parameters ϕ_j’s to remove the constraint 0 ≤ ϕ₀ ≤ ϕ₁ ≤ ⋯ ≤ ϕ_m. To use l_n(θ), by following Zhou et al. (2017b), we will assume that ρ is known. Note that for the case of regular studies or interval-censored data where all covariates are observed, we have δ_i = π_i = 1 for all i and l_n(θ) reduces to ${\sum }_{i=1}^n\log L_i(\beta ,{\Lambda }_0)$ log L_i (β, Λ₀), the log likelihood function based on the observed interval-censored data.

3. Sieve Penalized Variable Selection Procedure

Now we discuss the variable or covariate selection as well as estimation of the parameters β and Λ₀. For this, we propose to estimate β and Λ₀ by the sieve penalized inverse probability weighted estimator (SPIPWE) ${\widehat{\theta }}_n=({\widehat{\beta }}_n,{\widehat{\Lambda }}_{0n})$ defined as the values of θ that maximize

$$Q_n(\theta )\equiv Q_n(\beta ,{\Lambda }_0)=l_n(\beta ,{\Lambda }_0)-n\sum _{j=1}^p{p}_{\lambda }({\beta }_j)$$ (4)

over Θ_n, where p_λ denotes a penalty function that depends on a tuning parameter λ > 0.

To determine the estimator defined above, in the following, we will employ several commonly used penalty functions. In particular, we will consider the LASSO and ALASSO penalty functions with the latter defined as p_λ(β_j) = λ w_j ∣β_j∣, where w_j is a weight for β_j. By following the suggestion of Zou (2006), in the following, we will set the weights as $w_j=1∕\mid {\tilde{\beta }}_j\mid$, where ${\tilde{\beta }}_j$ is the estimator of β_j given by maximizing l_n(β, Λ_0n). By setting w_j = 1 for all j, the function p_λ(β_j) above will reduce to the LASSO penalty function. Another penalty function to be investigated here is the SCAD penalty function that has the form

$$p_{\lambda }({\beta }_j)=\{\begin{array}{cc}\lambda \mid {\beta }_j\mid ,\hfill & \mid {\beta }_j\mid \le \lambda ,\hfill \\ -\frac{{\beta }_j^2-2a\lambda \mid {\beta }_j\mid +{\lambda }^2}{[2(a-1)]},\hfill & \lambda <\mid {\beta }_j\mid \le a\lambda ,\hfill \\ (a+1){\lambda }^2∕2,\hfill & \mid {\beta }_j\mid \ge a\lambda ,\hfill \end{array}\phantom{\}}$$ (5)

where the constant a is set to be 3.7 to follow the suggestion of Fan and Li (2001). In addition, we will study the SICA and SELO penalty functions. The former is defined as

$$p_{\lambda }({\beta }_j)=\lambda \frac{({\gamma }_1+1)\mid {\beta }_j\mid }{\mid {\beta }_j\mid +{\gamma }_1}$$ (6)

with γ₁ > 0 being a shape parameter, while the latter has the form

$$p_{\lambda }({\beta }_j)=\frac{\lambda }{\log (2)}\log \left(\frac{\mid {\beta }_j\mid }{\mid {\beta }_j\mid +{\gamma }_2}+1\right)$$ (7)

with γ₂ > 0 being another tuning parameter besides λ. In the numerical studies below, we will set γ₁ = γ₂ = 0.01 (Dicker et al. 2013).

To maximize Q_n(θ) over Θ_n or obtain the SPIPWE (${\widehat{\beta }}_n$, ${\widehat{\Lambda }}_{0n}$), we propose an iterative procedure that estimates β and α alternately. In particular, we will use the Nelder-Mead simplex algorithm to update the estimator of α given the current estimator of β and then update the estimator of β by employing the coordinate descent algorithm while fixing α (Fu 1998; Fan and Lv 2011; Lin and Lv 2013). The specific steps can be described as follows.

Step 1. Choose the initial values for both β and α.

Step 2. Given the current estimate of β, update the estimate of α by using the Nelder-Mead simplex algorithm.

Step 3. Given the current estimate of α, update the estimate of β by using the coordinate descent algorithm. In particular, update each element of β by maximizing Q_n(β, Λ_0n) while holding the other elements of β fixed.

Step 4. Repeat steps 2 and 3 until convergence.

The main idea behind the coordinate descent algorithm described above is to conduct the univariate maximization for each element of β repeatedly, and for each univariate maximization, one can use the golden-section search algorithm (Kiefer 1953). To check the convergence, a common way, which is used below, is to compare the summation of the absolute differences between the current and updated estimates of each component of both β and α. Note that the algorithms similar to that described above have been developed and shown to have good convergence properties in the literature (Fu, 1998; Fan and Lv, 2011; Lin and Lv, 2013). The theorem given below will show that the proposed estimator exists and the algorithm will converge. Also in the numerical studies reported below, the algorithm seems to work well and we did not have any convergence issues with the criterion. On the covariate selection, at the convergence, we will set the estimates of the components of β whose absolute values are less than a pre-specified threshold to be zero. For the numerical studies reported below, we used the threshold of 10⁻⁶ by following Wang et al. (2007) and 0 as the initial values for both β and α by following Fan and Lv (2011) and Lin and Lv (2013). Also for the numerical study below, we implement the Nelder-Mead simplex algorithm by using the R function optim and employ the R function optimize for the implementation of the golden-section search algorithm.

To establish the asymptotic properties of the proposed estimators ${\widehat{\beta }}_n$ and ${\widehat{\Lambda }}_{0n}$, we will focus on the ALASSO penalty function and some comments on this will be given below. Let θ₀ = (β₀, Λ₀₀) denote the true value of θ = (β, Λ₀) and assume that ${\beta }_0=({\beta }_{10}^{\prime },0^{\prime }{)}^{\prime }$, where β₁₀ is the s-dimensional nonzero elements with s < p. For a vector v, let ∥v∥ denote its Euclidean norm and define the supremum norm ∥f∥_∞ = sup_t ∣f(t)∣ for a function f. Also define $\Vert \Lambda {\Vert }_2^2=\int \{[\Lambda (l){]}^2+[\Lambda (r){]}^2\}dF(l,r)$, where F(l, r) denotes the joint distribution function of the L_i’s and R_i’s. Furthermore write λ_n = λ for sample size n and define the distance between θ₁ = (β₁, Λ₁) ∈ Θ and θ₂ = (β₂, Λ₂) ∈ Θ as

$$d({\theta }_1,{\theta }_2)=\{\Vert {\beta }_1-{\beta }_2{\Vert }^2+\Vert {\Lambda }_1-{\Lambda }_2{\Vert }_2^2{\}}^{1∕2}.$$

The asymptotic properties of ${\widehat{\theta }}_n=({\widehat{\beta }}_n,{\widehat{\Lambda }}_{0n})$ described below is with respect to n → ∞. Theorem 1. Assume that Conditions (I) - (IV) given in the Appendix hold and $\sqrt{n}{\lambda }_n=O_p(1)$. Then with probability tending to one, there exists a local maximizer ${\widehat{\theta }}_n$ of Q_n(θ) over Θ_n with $p_{{\lambda }_n}({\beta }_j)={\lambda }_n\mid {\beta }_j\mid ∕\mid {\tilde{\beta }}_j\mid$ such that

1. $d({\widehat{\theta }}_n,{\theta }_0){\to }_{a.s.}0$ and

2. $d({\widehat{\theta }}_n,{\theta }_0)=O_p(n^{-(1-\nu )∕2}+n^{-r\nu ∕2})$,

where ν ∈ (0, 1) such that m = o(n^ν) and r is defined in Condition (IV).

Theorem 2. Assume that Conditions (I) - (IV) given in the Appendix hold with r > 2 and ν > 1/(2r). Also assume that $\sqrt{n}{\lambda }_n\to 0$ and nλ_n → ∞. Then with probability tending to one, the maximizer ${\widehat{\beta }}_n=({\widehat{\beta }}_{1n}^{\prime },{\widehat{\beta }}_{2n}^{\prime }{)}^{\prime }$ in Theorem 1 satisfies

1. (Sparsity) ${\widehat{\beta }}_{2n}=0$ and

2. (Asymptotic normality) $\sqrt{n}({\widehat{\beta }}_{1n}-{\beta }_{10})\to N(0,\Sigma )$ in distribution, where Σ is given in the Appendix.

The proof for the results given above will be sketched in the Appendix. The theorems above tell us that the proposed estimator ${\widehat{\theta }}_n$ is consistent and the covariate selection procedure has the oracle property (Fan and Li 2001). Also it will be seen below that ${\widehat{\beta }}_{1n}$ is efficient. Note that based on Theorem 1, the choice of ν = 1/(1 + r) yields the optimal rate of convergence n^r/2(1+r), which equals n^1/3 when r = 2 and improves as r increases. Also note that the main idea behind the proof is similar to that for the similar results for other situations (Fan and Li, 2001; Zhang and Lu, 2007) but there exist some new challenges. One is that unlike the case of right-censored data, the baseline function cannot be cancelled and thus has to be dealt with regression parameters together. Another is that although the penalized log-likelihood function is still concave, the traditional Taylor expansion-based approach such as that used in Zhang and Lu (2007) cannot be used here, and thus instead we will adopt the empirical process theory to prove the strong consistency of the proposed estimator and derived the consistence rate.

To implement the sieve penalized estimation procedure described above, it is apparent that we need to choose the tuning parameter λ as well as the degree m of Bernstein polynomials. For this, we propose to use the C-fold cross-validation. Specifically, let C be an integer and suppose that the observed data can be divided into C non-overlapping parts with approximately the same size. Also let $l_n^c(\beta ,\alpha )$ denote the inverse probability weighted log-likelihood function based on the c^th part of the whole data set and ${\widehat{\beta }}^{-c}$ and ${\widehat{\alpha }}^{-c}$ the proposed SPIPWE of β and α, respectively, obtained based on the whole data without the c^th part. For given λ and m, the cross-validation statistic can be defined as

$$CV(\lambda ,m)=\sum _{c=1}^C{l}_n^c({\widehat{\beta }}^{-c},{\widehat{\alpha }}^{-c}),$$ (8)

and one can choose the values of λ and m that maximize CV (λ, m). In practice, one may perform grid search over possible ranges of λ and m and furthermore, may fix m to be the closest integer to n^0.25 to focus on the selection of λ. For the inference about β₁₀, one needs to estimate Σ in Theorem 2 and for this, one way is to employ the observed Fisher information matrix. An alternative, which is much simpler and will be used in the application below, is to apply the bootstrap procedure and more comments on this are given below.

4. A Simulation Study

An extensive simulation study was conducted to assess the performance of the sieve penalized estimation procedure proposed in the previous sections. In the study, we considered the situation where there exist p = 100 covariates with the first and last five elements of β being set to be 0.5 and the other elements being 0. Furthermore the covariate vector X_i ~ MVN_p(0, Ψ) are i.i.d., where the (j, k) element of Ψ is Ψ_j,k = 0.5^∣j−k∣. Given the true failure time, by following Zhou et al. (2017b), the interval-censored data {L_i, R_i, Δ_1i, Δ_2i : i = 1, … , n} were generated as follows. First we generated the total number of examination time points for each subject from the zero-truncated Poisson distribution with mean μ, and given the number of examination time points, the examination times were generated from the uniform distribution over (0, τ) with τ = 1. Then for subject i, if the failure had already occurred before the first examination time, we defined R_i = τ and L_i to be the first examination time and (Δ_1i, Δ_2i) = (1, 0); if the failure had not yet occurred at the last examination time, we defined R_i to be the last examination time and L_i = 0 and (Δ_1i, Δ_2i) = (0, 0). Otherwise, we set (Δ_1i, Δ_2i) = (0, 1) and defined L_i and R_i to be the largest examination time point before the true T_i and the smallest examination time point after T_i, respectively.

In the study, we considered the interval-censored data arising from both case-cohort studies and general situation where all covariates are observed. For the former, we generated the failure times T_i’s from the exponential distribution with the hazard function λ(t∣X_i; β) = η e^X_iβ. By setting η = 0.075 or 0.008, we have the right-censored percentage being approximately 80% and 95%, respectively. To generate the sub-cohort, by following Zhou et al. (2017b), we employed the independent Bernoulli sampling with the selection probability ρ = 0.2. Table 1 presents the covariate selection results given by the proposed approach based on 100 replications and with n = 1400 or 2000, and μ = 10. In the table, the MMSE and SD represent the median and the standard deviation of the MSE, respectively, given by $(\widehat{\beta }-{\beta }_0{)}^T{\Sigma }_{\mathit{X}}(\widehat{\beta }-{\beta }_0)$ among 100 data sets, where Σ_X denotes the covariance matrix of the covariates. The quantity TP denotes the averaged number of the correctly selected covariates whose true coefficients are not 0, and FP the averaged number of incorrectly selected covariates whose true coefficients are 0. Here we considered all five penalty functions LASSO, ALASSO, SCAD, SICA and SELO described above, set m to be the closest integer to n^0.25, and used the 5-fold cross-validation for the selection of λ based on the grid search. One can see that the proposed variable selection procedure with all five penalty functions seems to provide good and consistent performance, especially for the case with 80% right-censored rate. Also as expected, the performance got better as the full cohort size n increased.

Table 1:

Results on the covariate selection based on case-cohort interval-censored data

Penalty	MMSE (SD)	TP(10)	FP(90)	MMSE (SD)	TP(10)	FP(90)
	n = 1400
	80% right-censored			95% right-censored
LASSO	0.385 (0.094)	10	13.11	0.620 (0.199)	9.71	17.14
ALASSO	0.408 (0.134)	9.89	1.11	0.806 (0.607)	8.58	2.06
SCAD	0.247 (0.158)	9.48	1.36	2.819 (2.771)	6.48	9.62
SICA	0.154 (0.125)	9.69	0.95	1.116 (0.508)	6.62	1.6
SELO	0.152 (0.133)	9.55	0.61	1.144 (0.404)	6.47	1.33
Oracle	0.093 (0.053)	10	0	0.344 (0.263)	10	0
	n = 2000
	80% right-censored			95% right-censored
LASSO	0.331 (0.084)	10	10.49	0.455 (0.140)	9.88	13.11
ALASSO	0.289 (0.096)	9.99	0.48	0.441 (0.332)	9.66	2.45
SCAD	0.114 (0.085)	9.88	0.99	1.586 (0.814)	7.19	7.21
SICA	0.081 (0.074)	9.86	0.31	0.769 (0.311)	7.57	1.69
SELO	0.082 (0.079)	9.83	0.27	0.761 (0.327)	7.8	1.95
Oracle	0.062 (0.039)	10	0	0.226 (0.167)	10	0

Simulation results with μ = 10, 100 continuous covariates. The selection probability for the sub-cohort is ρ = 0.2. MMSE (SD): median (standard deviation) of the MSEs of 100 replications. TP: averaged true positive selection rate. FP: averaged false positive selection rate.

Table 2 gives the covariate selection results obtained by the use of the proposed method for the case of general interval-censored data with n = 500, p = 100, μ = 10 or 20 and 100 replications. Here the covariates, the true failure times and the censoring intervals were generated in the same way as above except that for the true failure time, we took η = log(20), which gave the percentage of right-censored observations being about 30%. In the table, as above, we considered the same quantities MMSE, SD, TP and FP and the same five penalty functions LASSO, ALASSO, SCAD, SICA and SELO, set m = 5, the closest integer to n^0.25, and used the 5-fold cross-validation for the selection of λ based on the grid search. In addition, for comparison, we also implemented the method proposed by Wu and Cook (2015), referred to as WC method in the table, with the use of the LASSO and ALASSO penalty functions, the four piecewise constant function and the program provided in the paper. One can see from Table 2 that the proposed method seems to perform well no matter which penalty function was used, especially in terms of the quantity TP, the measure of true positive selection. Also in terms of TP, the proposed and WC methods gave similar performance, but the proposed method gave much smaller FP. More comments on the comparison are given below.

Table 2:

Results on the covariate selection based on general interval-censored data with continuous covariates

Penalty	Method	MMSE (SD)	TP	FP
μ = 10
LASSO	WC	0.2679 (0.1783)	10	18.24
	Proposed	0.3994 (0.1821)	10	11.29
ALASSO	WC	0.0549 (0.0576)	10	0.12
	Proposed	0.1322 (0.0861)	10	0
SCAD	Proposed	0.0689 (0.1132)	9.93	0.58
SICA	Proposed	0.0555 (0.0583)	9.94	0.13
SELO	Proposed	0.0602 (0.0697)	9.94	0.06
μ = 20
LASSO	WC	0.3118 (0.1059)	10	12.32
	Proposed	0.3969 (0.1549)	10	9.72
ALASSO	WC	0.0501 (0.0408)	10	0.09
	Proposed	0.2234 (0.0953)	10	0.01
SCAD	Proposed	0.0646 (0.1553)	9.93	0.35
SICA	Proposed	0.0522 (0.0573)	9.98	0.03
SELO	Proposed	0.0516 (0.0545)	9.98	0.04

Simulation results with n = 500, p = 100 and continuous covariates. MMSE (SD): median (standard deviation) of the MSEs of 100 replications. TP: averaged true positive selection rate. FP: averaged false positive selection rate.

Note that in the set-ups above, all covariates were continuous and it is apparent that in practice, we may have both continuous and discrete covariates. To investigate such situations, we considered the case with all set-up being the same as with Table 2 except that only the first fifty covariates were generated as above and the other fifty covariates were assumed to take values 0 and 1 with E(X_i) = 0.2 and the correlation 0.5^{∣j₁−j₂∣} among them, 51 ≤ j₁ < j₂ ≤ 100. Here the different types of covariates were supposed to be independent and the obtained results are presented in Table 3. One can see that overall they gave similar conclusions and again suggest that the proposed method performed well and consistently with respect to all penalty functions. In addition, we investigated different values for m to assess the robustness of the proposed method with respect to m and the results suggested that it seems to be robust.

Table 3:

Results on the covariate selection based on general interval-censored data with both continuous and discrete covariates

Penalty	Method	MMSE (SD)	TP	FP
μ = 10
LASSO	WC	0.6026 (0.2873)	9.88	13.31
	Proposed	1.2907 (0.3721)	9.32	10.73
ALASSO	WC	0.4163 (0.4055)	8.12	0.48
	Proposed	0.3647 (0.2353)	9.93	0.03
SCAD	Proposed	0.4638 (0.2615)	8.07	1.28
SICA	Proposed	0.4087 (0.2628)	8.73	0.55
SELO	Proposed	0.4088 (0.2571)	8.65	0.59
μ = 20
LASSO	WC	0.5818 (0.2307)	9.92	10.04
	Proposed	1.2136 (0.3322)	9.45	9.73
ALASSO	WC	0.3315 (0.4237)	8.69	0.43
	Proposed	0.2851 (0.1933)	9.96	0.03
SCAD	Proposed	0.4432 (0.3052)	7.92	1.61
SICA	Proposed	0.3624 (0.2309)	8.74	0.43
SELO	Proposed	0.3664 (0.2494)	8.74	0.54

Simulation results with n = 500, p = 100 and mixed covariates. MMSE (SD): median (standard deviation) of the MSEs of 100 replications. TP: averaged true positive selection rate. FP: averaged false positive selection rate.

On the comparison between the proposed method and the WC method, as seen from Table 2, the former always gave much smaller FP than the latter. With respect to MMSE and TP, the two actually gave similar performance as sometimes one yielded smaller values but in other situations the other did. In addition, as expected, the proposed method was much faster than the WC method. For example, on average, it took 620 seconds for the WC method to finish one replication for the results given in Table 2, while the proposed method only needed 96 seconds. In the study, we also considered some other set-ups including different values for n, p and ρ and all results indicated that the proposed method performed well and either better than or similarly to the WC method on the covariate selection. Furthermore on the comparison, following a reviewer’s suggestion, we obtained the average of the estimates of the cumulative baseline hazard function given by the two methods for the situation considered in Table 2 and present them in Figure 1. It is apparent that the proposed method gave a more natural and reasonable estimate than that of Wu and Cook (2015).

Figure 1:

The estimates of the baseline cumulative hazard function.

To assess the performance of the bootstrap procedure, for the results given in Table 1 with n = 1400 and the right censoring rate being 80%, we calculated the averages of the estimated standard errors given by the bootstrap procedure (ESE) and the sample standard errors (SSE) and present them in Table 4 for the first and last five estimated regression parameters. They suggest that the bootstrap procedure seems to perform well for all penalty functions except that it could underestimated the standard error for the case with the ALASSO function.

Table 4:

Simulation results on the assessment of the bootstrap procedure

		β1	β2	β3	β4	β5	β96	β97	β98	β99	β100
LASSO	ESE	0.0847	0.0939	0.0974	0.0966	0.0805	0.0813	0.0959	0.0953	0.0924	0.0815
	SSE	0.0989	0.1087	0.1160	0.1152	0.0916	0.0992	0.1014	0.0892	0.1074	0.1038
ALASSO	ESE	0.0693	0.0824	0.0882	0.0887	0.0645	0.0662	0.0885	0.0887	0.0864	0.0681
	SSE	0.1390	0.1383	0.1560	0.1311	0.1312	0.1320	0.1257	0.1146	0.1344	0.1501
SCAD	ESE	0.1812	0.1938	0.1955	0.2000	0.1580	0.1737	0.1947	0.1929	0.1970	0.1658
	SSE	0.1208	0.1492	0.1878	0.1999	0.1845	0.1209	0.1634	0.1450	0.1788	0.2220
SELO	ESE	0.1632	0.1824	0.1850	0.1939	0.1423	0.1555	0.1862	0.1829	0.1891	0.1501
	SEE	0.1467	0.1798	0.1936	0.1956	0.1470	0.1233	0.1603	0.1457	0.2019	0.2105
SICA	ESE	0.1623	0.1824	0.1864	0.1918	0.1475	0.1573	0.1864	0.1829	0.1853	0.1504
	SSE	0.1384	0.1748	0.1740	0.1741	0.1489	0.1177	0.1434	0.1309	0.1678	0.1756

5. Analysis of the HVTN 505 Trial for HIV-1 Infection

The HVTN 505 Trial is a randomized, multiple-sites clinical trial of men or transgender women who had sex with men for assessing the efficacy of the DNA/rAd5 vaccine for HIV-1 infection (Fong et al. 2018; Hammer et al. 2013; Janes et al. 2017). In the study, each subject was randomly assigned to receive either the DNA/rAd5 vaccine or placebo. It is well-known that HIV-1 infection is deadly as it causes AIDS for which there is no cure and thus it is important and essential to develop a safe and effective vaccine for the prevention of the infection. The original primary analysis (Hammer et al. 2013) and the correlates analyses (Janes et al. 2017, Fong et al. 2018) studied the vaccine effect on or marker associations with the time to HIV-1 infection diagnosis using a Cox model via right-censored failure time methods. However, studying the alternative questions defined in terms of the time to true HIV-1 infection is arguably more important, which the previous analyses only studied in an approximate fashion. In the following, we directly address the alternative questions based on the observed interval-censored data.

The original trial consists of 2504 subjects randomly assigned to either the DNA/rAd5 vaccine (1253) or placebo (1251) group and for each subject, the information on four demographic covariates, age, race, BMI and behavioural risk, was collected. In addition, for all 25 HIV infection cases and other 125 subjects in the vaccine group, a number of T cell response biomarkers and antibody response biomarkers were measured. For the analysis below, by following Fong et al. (2018) and Janes et al. (2017), we will focus on these subjects for the determination of the important or relevant covariates or biomarkers for HIV-1 infection conditional on the four demographic covariates. That is, the covariates age, race, BMI and behavioural risk will always be included in the model.

First we will consider all available biomarkers including 38 T cell response biomarkers and 6 primary and 27 secondary antibody response biomarkers. In addition, by following the suggestion of Fong et al. (2018), we include the 21 pairwise interactions between the T cell response biomarker Env CD8+ polyfunctionality score, referred to as Env CD8 Score in the table below, and 6 primary antibody response biomarkers, which gives 96 covariates in total. Table 5 presents the covariates selected by the proposed estimation procedure with the use of the same penalty functions discussed in the simulation study. Here we considered 100 candidate values for the grid search of λ and the results given in the table correspond to m = 3 which is the closest integer to n^0.25 and λ was chosen based on the 5-fold cross-validation. It includes the estimated covariate effects and standard errors (SE) given by the bootstrap procedure based on 100 bootstrap samples. One can see that among all of the biomarkers considered, three T cell response biomarkers, including Env CD8 Score, and eight antibody response biomarkers were identified that may be correlated to the HIV-1 infection. In addition, three pairwise interactions were selected but none of the interactions had significant effects on the HIV-1 infection time.

Table 5:

Analysis results of the HVTN 505 Trial (with interactions)

Covariate	LASSO	ALASSO	SCAD	SICA	SELO
age	−0.192(0.362)	−0.224(0.384)	−0.155(0.336)	−0.229(0.377)	−0.227(0.369)
race	−0.648(0.645)	−0.628(0.622)	−0.650(0.598)	−0.718(0.534)	−0.730(0.542)
BMI	−0.103(0.338)	−0.089(0.358)	−0.116(0.339)	−0.169(0.334)	−0.167(0.340)
behavioural risk	1.171(0.756)	1.246(0.718)	1.116(0.724)	1.125(0.795)	1.126(0.846)
ANY.VRC.ENV.logpctpos	0.031(0.145)	-	-	-	-
CMV.logpctpos	0.105(0.215)	0.284(0.331)	-	-	-
Env CD8 Score	−1.031(0.387)	−0.981(0.315)	−0.965(0.327)	−0.925(0.399)	−0.925(0.403)
IgG.AEA244V1V2Tags293F	−0.094(0.168)	-	-	-	-
IgA.BioV3B	0.084(0.122)	0.119(0.114)	-	-	-
IgG.Cconenv03140CF.avi	−0.256(0.203)	−0.406(0.283)	-	−0.433(0.058)	−0.440(0.080)
IgA.ConSgp140CFI	0.123(0.138)	0.258(0.230)	-	-	-
IgG.BioC4.427B	-	0.044(0.281)	-	-	-
IgG.V2	0.375(0.303)	0.282(0.346)	0.285(0.322)	-	-
IgG.V3	0.076(0.371)	-	0.156(0.384)	-	-
IgG.env	0.029(0.370)	0.166(0.359)	−0.200(0.355)	-	-
Env CD8 Score × IgG.V2	0.289(0.259)	0.288(0.240)	0.262(0.227)	-	-
Env CD8 Score × IgG.env	0.457(0.330)	0.376(0.391)	0.390(0.307)	-	-
IgG.V2 × IgG.V3	−0.185(0.165)	-	−0.046(0.105)	-	-

age: scaled age; race: indicator of Caucasian; BMI: scaled body mass index; behavioural risk: baseline behavioral risk score; variables ending with logpctpos: Month 7 log of scaled net percentage of cells positive for IL2/IFN-gamma cytokine expression for various HIV-1 antigens with antigen names in the front; Env CD8 Score: Month 7 scaled CD8+ polyfunctionality score for the ANY VRC ENV antigen; variables starting with IgG/IgA: Month 7 IgG or IgA type antibody binding level measured by the binding antibody multiplex assay.

Based on the results above, we reanalyzed the data by considering only all 75 individual biomarkers without the interactions and Table 6 gives the covariates selected by the proposed estimation procedure again with the use of the same penalty functions discussed in the simulation study. It can be seen that only one antibody response biomarker IgG.Cconenv03140CF.avi was selected and seems to be significantly related to the HIV-1 infection risk. Among the T cell response biomarkers, although six were selected, only two of them, including Env CD8 Score, seem to have significant effects and furthermore, the other biomarker, ANY.VRC.ENV.logpctpos, was only selected by the LASSO procedure and is highly correlated with the biomarker Env CD8 Score. Due to this and also the correlation between Env CD8 Score and other selected T cell response biomarkers, one can see that the estimated Env CD8 Score effects by different procedures have large variation. In addition, the results suggested that the behavioural risk also had significant effect on the HIV-1 infection time among the demographic covariates. The results above on the individual biomarkers are similar to those given by Fong et al. (2018) and Janes et al. (2017), which carried out the covariate selection based on either the univariate analysis or simplified multivariate analysis and suggested that Env CD8 Score is a significant biomarker. Specifically, in the analysis performed by Janes et al.(2017), they performed multiple testing in assessing exploratory immune response variables. They calculated p–values and considered p < .20 to be statistically significant. While Fong et al.(2018) applied the forward stepwise regression to build a logistic regression model with the significance of each covariate tested using the p–value. However, unlike above, they also suggested that Env CD8 Score may have significant pairwise interactions with two other antibody response biomarkers.

Table 6:

Analysis results of the HVTN 505 Trial (without interactions)

Covariate	LASSO	ALASSO	SCAD	SICA	SELO
age	−0.258(0.295)	−0.254(0.300)	−0.261(0.289)	−0.227(0.380)	−0.225(0.376)
race	−0.502(0.487)	−0.563(0.509)	−0.776(0.451)	−0.728(0.565)	−0.738(0.574)
BMI	−0.101(0.265)	−0.153(0.275)	−0.059(0.263)	−0.167(0.337)	−0.165(0.346)
behavioural risk	1.019(0.603)	1.111(0.619)	1.333(0.590)	1.121(0.781)	1.120(0.832)
ANY.VRC.ENV.logpctpos	−0.225(0.114)	-	-	-	-
CMV.logpctpos	0.063(0.131)	0.122(0.176)	-	-	-
VRC.ENV.A.logpctpos	−0.067(0.177)	-	-	-	-
VRC.ENV.B.logpctpos	−0.091(0.189)	-	−0.064(0.166)	-	-
VRC.ENV.C.logpctpos	−0.249(0.198)	−0.254(0.182)	−0.256(0.187)	-	-
Env CD8 Score	−0.042(0.159)	−0.625(0.284)	−0.214(0.147)	−0.921(0.413)	−0.921(0.444)
IgG.Cconenv03140CF.avi	−0.266(0.219)	−0.233(0.190)	−0.261(0.207)	−0.434(0.089)	−0.440(0.104)

age: scaled age; race: indicator of Caucasian; BMI: scaled body mass index; behavioural risk: baseline behavioral risk score; variables ending with logpctpos: Month 7 log of scaled net percentage of cells positive for IL2/IFN-gamma cytokine expression for various HIV-1 antigens with antigen names in the front; Env CD8 Score: Month 7 scaled CD8+ polyfunctionality score for the ANY VRC ENV antigen; IgG.Cconenv03140CF.avi: Month 7 IgG antibody binding level to the HIV-1 antigen Cconenv03140CF.avi measured by the binding antibody multiplex assay.

6. Discussion and Conclusion Remarks

This paper discussed the covariate selection problem when one faces interval-censored failure time data arising from case-cohort studies and for it, a sieve penalized estimation procedure was developed. In the method, Bernstein polynomials were employed to approximate the unknown cumulative baseline hazard function and the method allows the use of various commonly used penalty functions. In terms of the penalty function, it is apparent that one question of practical interest is that for a given data set, which one should be used. For the proposed method, the simulation study seems to suggest that SICA and SELO share the similar performance and ALASSO, SICA and SELO may be better than LASSO and SCAD. On the other hand, it is well-known that it does not seem to exist a general guideline on this in the literature, and a common approach is to apply multiple penalty functions and select the variables in the sense of average. For the implementation of the proposed method, an iterative algorithm was presented with the use of the Nelder-Mead simplex algorithm and the coordinate descent algorithm. In addition, the asymptotic properties of the procedure were established and the simulation study indicated that it works well for practical situations.

As mentioned above, the proposed variable selection procedure applies to interval-censored data arising from both case-cohort studies and general failure time studies. For the latter, Wu and Cook (2015) developed a similar method by using the piecewise constant functions to approximation the unknown baseline hazard function. Although their method and the proposed one may look similar, they are actually quite different. One key difference is that compared to the piecewise constant functions, Bernstein polynomials have much better properties such as being continuous and differentiable and can provide much more natural approximation to the cumulative baseline hazard function. In consequence, this allows the development of a simpler, faster and more stable algorithm for the implementation of the proposed method than the EM algorithm given in Wu and Cook (2015). More specifically, for general interval-censored data, the proposed method directly maximizes the penalized log-likelihood function, while Wu and Cook’s method needs to calculate the conditional expectation of the penalized log-likelihood function and maximize the penalized conditional expectation iteratively. Furthermore, the asymptotic properties of the proposed method can be established and in comparison, no theoretical justification was available for the method given in Wu and Cook (2015).

There exist several directions for future research. One is that the focus above has been on the situation where the number of covariates is smaller than the sample size and it is apparent that it would be interesting to generalize the method to high dimensional situations where the number of covariates is larger than the sample size. Note that although the latter is important, the situation discussed here is also quite important as it occurs quite often in, for example, medical follow-up studies such as clinical trials, and there did not exist an established procedure with theoretical justification available. For the generalization to the high dimensional case, although the idea and the procedure discussed above are applicable and valid, one issue that may be difficult is the establishment of some theoretical justification.

Another direction for further research is that in the previous sections, we have focused on the data from independent subjects. In practice, however, sometimes one may face the data from correlated subjects such as clustered interval-censored data and it is straightforward to generalize the idea discussed above to this latter situation. For example, consider a regular failure time study that consists of n clusters with J_i subjects in the ith cluster. Furthermore assume that the subjects within the same cluster may be correlated with each other but the subjects between different clusters are independent. Let T_ij and X_ij denote the failure time of interest and the vector of covariates associated with the jth subject in the ith cluster, respectively, and suppose that only an interval (L_ij, R_ij] is observed for T_ij such that T_ij ∈ (L_ij, R_ij]. For inference, a simple generalization of the Cox model above is to assume that the cumulative hazard function of T_ij has the form

$$\Lambda (t;{\mathbf{X}}_{ij};{\mu }_i)={\mu }_i{\Lambda }_0(t)\text{exp}\{{\beta }^{\prime }{\mathbf{X}}_{ij}\}$$ (9)

given the latent variables μ_i’s. Under the assumptions above, the resulting full likelihood function can be written as the following expectation

$$L_{\mu }(\beta ,{\Lambda }_0)=E_{\mu }\left[\prod _{i=1}^n\prod _{j=1}^{J_i}\left\{\text{exp}\left(-{\mu }_i{\Lambda }_0(L_{ij})e^{{\beta }^{\prime }\mathbf{X}}\right)-\text{exp}\left(-{\mu }_i{\Lambda }_0(R_{ij})e^{{\beta }^{\prime }\mathit{X}}\right)\right\}\right]$$ (10)

with respect to the distribution of the μ_i’s. Then a covariate selection procedure can be developed as above with replacing the likelihood function given in (1) by the likelihood function above.

Appendix: Asymptotic Properties of ${\widehat{\theta }}_n$

In this Appendix, we will sketch the proof of the asymptotic properties of the proposed estimator ${\widehat{\theta }}_n$ given in the theorems above with the focus on the ALASSO penalty function. First, we will describe the regularity conditions that will be needed.

Condition (I). (i) $\mathcal{B}$ is a compact subset of ${\mathcal{R}}^p$ and β₀ is an interior point of $\mathcal{B}$. (ii) E(XX′) is non-singular with X being bounded. That is, there exists x₀ > 0 such that P(∥X∥ ≤ x₀) = 1.

Condition (II). (i) There exists a positive number ς such that P(R − L ≥ ς) = 1. (ii) The union of the supports of L and R is contained in an interval [c, u], where 0 < c < u < ∞.

Condition (III). The conditional density g(l, r∣x) of (L, R) given X has bounded partial derivatives with respect to (l, r), and the bounds of these partial derivatives do not depend on (l, r, x).

Condition (IV). The function Λ₀(·) is continuously differentiable up to order r in [c, u] and satisfies ${\tilde{\xi }}^{-1}<{\Lambda }_0(c)<{\Lambda }_0(u)<\tilde{\xi }$ for some positive constant $\tilde{\xi }$.

Next we apply the empirical process theory to prove Theorems 1-2 and in the following, For the proof, define P f = ∫ f(y)dP(y), the expectation of f(Y) taken under the distribution P, and $P_{n}f=n^{-1}{\sum }_{i=1}^{n}f(Y_i)$, the empirical process indexed by f(Y). Also let C denote a positive constant which may differ from place to place in the proofs.

Proof of Theorem 1.

In this section, we apply the empirical process theory to prove the strong consistency of ${\widehat{\theta }}_n$. For the proof, let $Q(\theta )=l(\theta )-{\sum }_{j=1}^p{p}_{\lambda }({\beta }_j)$ and M(θ) = −Q(θ), where l(θ) is the log-likelihood function based on a single observation.

Define K_ϵ = {θ : d(θ, θ₀) ≥ ϵ, θ ∈ Θ_n} for ϵ > 0 and

$${\zeta }_{1n}=\underset{\theta \in {\Theta }_n}{\text{sup}}\mid P_{n}M(\theta )-PM(\theta )\mid ,{\zeta }_{2n}=P_{n}M({\theta }_0)-PM({\theta }_0).$$

Consider the class of functions ${\mathcal{L}}_n=\{Q(\theta ):\theta \in {\Theta }_n\}$ and based on Lemmas 1-2 in the Supplementary materials of Zhou et al.(2017a), we have the following facts:

1. The covering number of ${\mathcal{L}}_n$ satisfies $N(ϵ,{\mathcal{L}}_n,L_1(P_n))\le C{M}_n^{m+1}{ϵ}^{-[p+m+1]}$;

2. ζ_1n → 0 a.s.

Then one can show that

$$\underset{K_{ϵ}}{\text{inf}}PM(\theta )=\underset{K_{ϵ}}{\text{inf}}\{PM(\theta )-P_{n}M(\theta )+P_{n}M(\theta )\}\le {\zeta }_{1n}+\underset{K_{ϵ}}{\text{inf}}{P}_{n}M(\theta )$$ (11)

If ${\widehat{\theta }}_n\in K_{ϵ}$, then

$$\underset{K_{ϵ}}{\text{inf}}{P}_{n}M({\widehat{\theta }}_n)\le P_{n}M({\theta }_0)={\zeta }_{2n}+PM({\theta }_0)$$ (12)

Denote ${\delta }_{ϵ}=\underset{K_{ϵ}}{\text{inf}}PM(\theta )-PM({\theta }_0)$, then from (12), we can prove δ_ϵ > 0 via proof by contradiction. It follows from (11) and (12) that

$$\underset{K_{ϵ}}{\text{inf}}PM(\theta )\le {\zeta }_{1n}+{\zeta }_{2n}+PM({\theta }_0)={\zeta }_n+PM({\theta }_0)$$

with ζ_n = ζ_1n + ζ_2n and ζ_n ≥ δ_ϵ, which yields $\{{\widehat{\theta }}_n\in K_{ϵ}\}\subseteq \{{\zeta }_n\ge {\delta }_{ϵ}\}$. Based on the fact (ii) above and the strong law of large numbers, we have both ζ_1n → 0 and ζ_2n → 0 almost surely. Thus,

$${\cup }_{k=1}^{\infty }{\cap }_{n=k}^{\infty }\{{\widehat{\theta }}_n\in K_{ϵ}\}\subseteq {\cup }_{k=1}^{\infty }{\cap }_{n=k}^{\infty }\{{\zeta }_n\ge {\delta }_{ϵ}\},$$

which implies that $d({\widehat{\theta }}_n,{\theta }_0)\to 0$ almost surely.

Next we discuss the convergence rate of ${\widehat{\Lambda }}_{0n}$. Since Q_n(θ) only penalizes the regression coefficients β and p_λ(·) does not involve Λ, then for any ζ, we define the class

$${\mathcal{F}}_{\zeta }=\{l_n({\theta }_{0n})-l_n(\theta ):\theta \in {\Theta }_n,d(\theta ,{\theta }_{0n})<\zeta \}$$

with ${\theta }_{0n}=({\beta }_0^{\prime },{\Lambda }_{0n}{)}^{\prime }$. Following the calculation of Shen and Wong (1994) (page 597), we can establish that the entropy with bracketing

$$\log N_{[]}(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2)\le CN\log (\zeta ∕\epsilon ),$$

where N = m + 1, $N_{[]}(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2)$ is the bracketing number of the class ${\mathcal{F}}_{\zeta }$. Moreover, some algebraic calculations lead to $\Vert l_n({\theta }_{0n})-l_n(\theta ){\Vert }_2^2\le C{\zeta }^2$ for any $l_n({\theta }_{0n})-l_n(\theta )\in {\mathcal{F}}_{\zeta }$. Under Conditions (I), (II) and (IV), we have that ${\mathcal{F}}_{\zeta }$ is uniformly bounded. Therefore it follows from Lemma 3.4.2 of van der Vaart and Wellner (1996) that

$$E_P\Vert n^{1∕2}(P_n-P){\Vert }_{{\mathcal{F}}_{\zeta }}\le C{J}_{\zeta }(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2)\left\{1+\frac{J_{\zeta }(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2)}{{\zeta }^2{n}^{1∕2}}\right\},$$ (13)

where $J_{\zeta }(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2)={\int }_0^{\zeta }\{1+\log N_{[]}(\epsilon ,{\mathcal{F}}_{\zeta },\Vert \cdot {\Vert }_2){\}}^{1∕2}d\epsilon \le C{N}^{1∕2}\zeta$.

Note that the right-hand side of (13) gives ϕ_n(ζ) = C(N^1/2ζ + N/n^1/2). Also it is easy to see that ϕ_n(ζ)/ζ decreases in ζ and

$$r_n^2{\varphi }_n(1∕r_n)=r_n{N}^{1∕2}+r_n^{2}N∕n^{1∕2}<2C{n}^{1∕2},$$

where r_n = N^−1/2_n1/2 = O(n^(1−ν)/2) with 0 < ν < 0.5. Hence $n^{(1-\nu )∕2}d({\widehat{\theta }}_n,{\theta }_{0n})=O_p(1)$ by Theorem 3.4.1 of van der Vaart and Wellner (1996). Also note that by the Theorem 1.6.2 of Lorentz (1986), if m = o(n^ν), there exist Bernstein polynomials Λ_0n such that

$$\Vert {\Lambda }_{0n}-{\Lambda }_{00}{\Vert }_{\infty }=O(n^{-\frac{r\nu }{2}}).$$ (14)

Thus,

$$\Vert {\widehat{\Lambda }}_{0n}-{\Lambda }_{00}{\Vert }_2=O_p(n^{-(1-\nu )∕2}+n^{-r\nu ∕2}).$$

This completes the proof.

Proof of Theorem 2.

(i) First we will show that ${\widehat{\beta }}_{2n}=0$ and for this, it is sufficient to show that, for any θ = (β, Λ₀) satisfying d(θ, θ₀) = O_p(n^{−(1−ν)/2} + n^−rν/2), we have

$$Q_n({\beta }_1,0,{\Lambda }_0)=\underset{\Vert {\beta }_2\Vert \le C{n}^{-1∕2}}{\max }{Q}_n({\beta }_1,{\beta }_2,{\Lambda }_0).$$

For each β in a neighbourhood of β₀, by Taylor expansion, we have that

$$l_n(\beta ,{\Lambda }_0)=l_n({\beta }_0,{\Lambda }_0)+O_p(\sqrt{n}\Vert \beta -{\beta }_0\Vert )-\frac{n}{2}(\beta -{\beta }_0{)}^{\prime }(I_{1,2}({\beta }_0,{\Lambda }_{00})+o_p(1))(\beta -{\beta }_0),$$

where I_1,2(β₀, Λ₀₀) is the leading p × p submatrix of Fisher information matrix. Then for j = s + 1, … , p, one can easily show that

$$\begin{gathered} \frac{\partial Q_n(\beta ,{\Lambda }_0)}{\partial {\beta }_j}=\frac{\partial l_n(\beta ,{\Lambda }_0)}{\partial {\beta }_j}-n{\lambda }_n\frac{\text{sgn}({\beta }_j)}{\mid {\tilde{\beta }}_j\mid } \\ =O_p(\sqrt{n})-(n{\lambda }_n)\sqrt{n}\frac{\text{sgn}({\beta }_j)}{\mid \sqrt{n}{\tilde{\beta }}_j\mid }. \end{gathered}$$

Note that $\sqrt{n}({\tilde{\beta }}_j-0)=O_p(1)$. It follows that

$$\frac{\partial Q_n(\beta ,{\Lambda }_0)}{\partial {\beta }_j}=\sqrt{n}\left\{O_p(1)-n{\lambda }_n\frac{\text{sgn}({\beta }_j)}{\mid O_p(1)\mid }\right\}.$$

Since nλ_n → ∞, it is easily seen that for ${\beta }_j\in (-C∕\sqrt{n},C∕\sqrt{n})$ and j = s + 1, … , p, ∂Q_n(β, Λ₀)/∂β_j and β_j have different signs when n is large enough. Thus, when Q_n(β, Λ₀) achieves its maximum, β_j = 0, that is, ${\widehat{\beta }}_{2n}=0$.

(ii) Note that $\sqrt{n}({\widehat{\beta }}_{1n}-{\beta }_{10})=\sqrt{n}({\widehat{\beta }}_{1n}-{\tilde{\beta }}_1)+\sqrt{n}({\tilde{\beta }}_1-{\beta }_{10})$, where ${\tilde{\beta }}_1$ is the vector composed of the first s elements of the sieve MLE, $\tilde{\beta }$. To prove the asymptotic normality of ${\widehat{\beta }}_{1n}$, it suffices to show the following two facts:

1. $\sqrt{n}({\widehat{\beta }}_{1n}-{\tilde{\beta }}_1)=o_p(1)$; and

2. $\sqrt{n}({\tilde{\beta }}_1-{\beta }_{10})\to N(0,\Sigma )$.

To prove (a), considering ${\widehat{\eta }}_n=({\widehat{\beta }}_{1n}^{\prime },0,{\widehat{\Lambda }}_{0n}{)}^{\prime }$ being a $\sqrt{n}$-consistent maximizer of $Q_n({\beta }_1,0,{\widehat{\Lambda }}_{0n})$ and using the Taylor expansion, we have

$$0=\frac{\partial Q_n({\widehat{\eta }}_n)}{\sqrt{n}\partial {\beta }_1}={\phantom{\mid }\frac{\partial l_n(\eta )}{\sqrt{n}\partial {\beta }_1}\mid }_{\eta =({\tilde{\beta }}^{\prime },{\widehat{\Lambda }}_{0n}{)}^{\prime }}+\frac{{\partial }^2{l}_n({\eta }^{\ast })}{n\partial {\beta }_1\partial {\beta }_1^{\prime }}\sqrt{n}({\widehat{\beta }}_{1n}-{\tilde{\beta }}_1)-\sqrt{n}{\lambda }_n{\left(\frac{\text{sgn}({\beta }_{10})}{{\tilde{\beta }}_1},\dots ,\frac{\text{sgn}({\beta }_{s0})}{{\tilde{\beta }}_s}\right)}^{\prime },$$

where η* lies between ${\widehat{\eta }}_n$ and $({\tilde{\beta }}^{\prime },{\widehat{\Lambda }}_{0n}{)}^{\prime }$. Since $({\tilde{\beta }}^{\prime },{\widehat{\Lambda }}_{0n}{)}^{\prime }$ maximizes l_n(β, Λ₀) and $\sqrt{n}{\lambda }_n\to 0$, we have that

$${\phantom{\mid }\frac{\partial l_n(\eta )}{\partial {\beta }_1}\mid }_{\eta =({\tilde{\beta }}^{\prime },{\widehat{\Lambda }}_{0n}{)}^{\prime }}=0,-\frac{{\partial }^2{l}_n({\eta }^{\ast })}{n\partial {\beta }_1\partial {\beta }_1^{\prime }}\sqrt{n}({\widehat{\beta }}_{1n}-{\tilde{\beta }}_1)=o_p(1).$$

Also because η* → η₀, we have $-\frac{{\partial }^2{l}_n({\eta }^{\ast })}{n\partial {\beta }_1\partial {\beta }_1^{\prime }}\to I_1({\eta }_0)$ in probability, where I₁(η₀) is the leading s × s submatrix of the matrix I(η₀). Therefore, $\sqrt{n}({\widehat{\beta }}_{1n}-{\tilde{\beta }}_1)=o_p(1)$.

To prove fact (b), denote V as the linear span of Θ − θ₀, where θ₀ is the true value of parameter θ. Let l(θ; X) be the weighted log-likelihood for a sample of size one and τ_n = n^{−(1−ν)/2} + n^−rv/2. For any θ ∈ Θ with ∥θ − θ₀∥ = O(τ_n), define the first order directional derivative of l(θ, X) at the direction v ∈ V as

$$\stackrel{.}{l}(\theta ,X)[v]=\frac{dl(\theta +tv,X)}{dt}{\mid }_{t=0},$$

and the second order directional derivative as

$$\begin{gathered} \ddot{l}(\theta ,X)[v,\tilde{v}]=\frac{d\stackrel{.}{l}((\theta +\tilde{t}\tilde{v},X))}{d\tilde{t}}{\mid }_{\tilde{t}=0} \\ =\frac{d^{2}l(\theta +tv+\tilde{t}\tilde{v},X)}{d\tilde{t}dt}{\mid }_{\tilde{t}=t=0}. \end{gathered}$$

Also define the Fisher inner product on the space V as

$$<v,\tilde{v}>=P\{\stackrel{.}{l}(\theta ,X)[v]\stackrel{.}{l}(\theta ,X)[\tilde{v}]\}$$

and the Fisher norm for v ∈ V as ∥v∥^1/2 = < v, v >. Let $\overline{V}$ be the closed linear span of V under the Fisher norm. Then ($\overline{V}$, ∥ · ∥) is a Hilbert space.

Furthermore, define the smooth functional of θ as ψ(θ) = b′β, where b is any vector of p dimension with ∥b∥ ≤ 1. For any v ∈ V, we denote

$$\stackrel{.}{\psi }({\theta }_0)[v]=\frac{d\psi ({\theta }_0+tv)}{dt}{\mid }_{t=0}.$$

Note that $\psi (\theta )-\psi ({\theta }_0)=\stackrel{.}{\psi }({\theta }_0)[\theta -{\theta }_0]$. It follows from the Riesz representation theorem that there exists $v^{\ast }\in \overline{V}$ such that $\stackrel{.}{\psi }({\theta }_0)[v]=<v^{\ast },v>$ for all $v\in \overline{V}$ and $\Vert v^{\ast }{\Vert }^2=\Vert \stackrel{.}{\psi }({\theta }_0)\Vert$. Thus it follows from the Cramér-Wold device and the formula $b^{\prime }(\tilde{\beta }-{\beta }_0)=\psi (\tilde{\theta })-\psi ({\theta }_0)={\stackrel{.}{\psi }}^{\prime }({\theta }_0)[\tilde{\theta }-{\theta }_0]=<\tilde{\theta }-{\theta }_0,v^{\ast }>$, we only need to show that

$$n^{1∕2}<\tilde{\theta }-{\theta }_0,v^{\ast }>\to N(0,b^{\prime }\Omega b)\text{in distribution},$$ (15)

where $\tilde{\theta }$ is the sieve inverse probability weighted estimator of θ. In fact, (15) can be proved using the similar arguments of Theorem 1 of Shen (1997). For each component β_j, j = 1, … , p, we denote by $b_j^{\ast }$ the solution to inf_bj E{l_β · e_j − l_bj [b_j]}², where l_β is the partial derivative of the log-likelihood about β and e_j is a p-dimensional vector of zeros except the j-th element equal to 1. Then l_bj [b_j] is the directional derivative with respect to Λ₀ and can be calculated as directional derivative defined at the beginning of the proof of Theorem 2. Now let $b^{\ast }=(b_1^{\ast },\dots ,b_p^{\ast })$. By the calculations of Chen et al. (2006), we have

$$\Vert v^{\ast }{\Vert }^2=\Vert \stackrel{.}{\psi }({\theta }_0)\Vert =\underset{v\in \overline{V}:\Vert v\Vert >0}{\text{sup}}\frac{\mid \stackrel{.}{\psi }({\theta }_0)[v]\mid }{\Vert v\Vert }=b^{\prime }\Omega \mathit{b},$$

and $\sqrt{n}(\tilde{\beta }-{\beta }_0)\to N(0,\Omega )$, where $\Omega =[E(S_{\beta }{S}_{\beta }^{\prime }){]}^{-1}=[I({\beta }_0){]}^{-1}$, S_β = l_β − l_b* [b*] and I(β₀) is the efficient Fisher information for β. Hence the semiparametric efficiency can be established by applying the result of Theorem 4 in Shen (1997).

Combining (a) and (b), we have $\sqrt{n}({\widehat{\beta }}_{1n}-{\beta }_{10})=\sqrt{n}({\tilde{\beta }}_1-{\beta }_{10})+o_p(1)\to N(0,\Sigma )$ in distribution, where Σ is the leading s × s submatrix of [I(β₀)]⁻¹. This completes the proof.