Feature screening in ultrahigh-dimensional varying-coefficient Cox model

Abstract

The varying-coefficient Cox model is flexible and useful for modeling the dynamic changes of regression coefficients in survival analysis. In this paper, we study feature screening for varying-coefficient Cox models in ultrahigh-dimensional covariates. The proposed screening procedure is based on the joint partial likelihood of all predictors, thus different from marginal screening procedures available in the literature. In order to carry out the new procedure, we propose an effective algorithm and establish its ascent property. We further prove that the proposed procedure possesses the sure screening property. That is, with probability tending to 1, the selected variable set includes the actual active predictors. We conducted simulations to evaluate the finite-sample performance of the proposed procedure and compared it with marginal screening procedures. A genomic data set is used for illustration purposes.

1. Introduction

Feature screening can effectively reduce ultrahigh dimensionality and therefore has attracted considerable attention in the recent literature. Fan and Lv [12] proposed a marginal screening procedure for ultrahigh-dimensional Gaussian linear model, and further showed that marginal screening procedures may possess a sure screening property under certain conditions. Feature screening procedures for varying-coefficient models (VCM) with ultrahigh-dimensional covariates have been proposed in the literature. Liu et al. [20] developed a sure independence screening (SIS) procedure for ultrahigh-dimensional VCM by taking conditional Pearson correlation coefficients as a marginal utility for ranking the importance of predictors. Fan et al. [13] proposed an SIS procedure for ultrahigh-dimensional VCM by extending B-spline techniques in Fan et al. [10] for additive models. Xia et al. [26] further extended the SIS procedure proposed in [13] to generalized varying-coefficient models (GVCM). Cheng et al. [5] proposed a forward variable selection procedure for ultrahigh-dimensional VCM based on techniques related to B-splines regression and grouped variable selection. Song et al. [22] extended the procedure in [13] to longitudinal data without taking into account within-subject correlation, while Chu et al. [6] proposed an SIS procedure for longitudinal data based on a weighted residual sum of squares to use within-subjection correlation to improve accuracy of feature screening. Kong et al. [17] proposed a new screening method that leaves a variable in the active set if it has, jointly with some other variables, a high canonical correlation with the response.

Survival analysis has been widely used in medical science, economics, finance, and social science, among others. In many studies, survival data have primary outcomes or responses that are subject to censoring. The Cox model [7, 8] is the most commonly used regression model for survival data, and the partial likelihood method has become a standard approach to parameter estimation and statistical inference. Recently, variable selection and parameter estimation in Cox regression models have been considered by various authors (see, e.g., [4, 9, 14, 18, 19, 30]). Huang et al. [15] studied the penalized partial likelihood with the ℓ₁-penalty for the Cox model with high-dimensional covariates. Yan and Huang [29] proposed the adaptive group Lasso in a Cox regression model with time-varying coefficients. However, they have not considered varying-coefficient models.

In this paper, we propose a new feature screening procedure for ultrahigh-dimensional varying-coefficient Cox models. It is distinguished from SIS procedures [11, 32] in that the proposed procedure is based on the joint partial likelihood of potentially important features, rather than the marginal partial likelihood of individual features. Xu and Chen [27] proposed a joint screening procedure and showed its advantage over SIS procedures in the context of generalized linear models. Yang et al. [28] extended the procedures in [27] to the Cox models. This work further extends the joint screening strategy and develops a feature screening procedure for varying-coefficient Cox models, which are natural extensions of Cox models and can be useful to explore nonlinear interaction effects between a primary covariate and other covariates.

The asymptotic properties of the proposed procedure are studied systematically. It is technically challenging to establish its sure screening property. The techniques used in [28] and other works related to SIS procedures cannot be applied for the present setting. We first develop Hoeffding’s inequality for a sequence of martingale differences and then establish a concentration inequality for the score function of a partial likelihood. Based on the concentration inequality, we prove the screening property for our proposed sure joint screening procedure. We also conduct simulation studies to assess the finite-sample performance of the proposed procedure and compare its performance with existing sure screening procedures for ultrahigh-dimensional survival data. The proposed methodology is demonstrated through an empirical analysis of a genomic data set.

The rest of this paper is organized as follows. In Section 2, we propose a new feature screening procedure for the varying-coefficient Cox model, develop an algorithm to carry it out, and demonstrate the ascent property of the proposed algorithm. We study the sampling property of the proposed procedure and establish its sure screening property. In Section 3, we present numerical comparisons and an empirical analysis of a real data set. Discussion is in Section 4. Technical proofs are in the Appendix.

2. New feature screening procedure for varying-coefficient Cox model

Let T be the survival time and x and U be p-dimensional covariate vector and univariate covariate, respectively. Throughout this paper, we consider the varying-coefficient Cox proportional hazard model given by

$$h(t|\mathbf{x},U)=h_0(t)+\exp \{{\mathbf{x}}^{\top }\alpha (U)\},$$ (1)

where h₀(t) is an unspecified baseline hazard function and α(U) = (α₁(U), …, α_p(U))^T consists of the unknown nonparametric coefficient functions. It is assumed that the support of U is finite and denoted by [a, b]. In survival data analysis, survival times are subject to a censoring time C. Denote the observed time by Z = min(T, C) and the event indicator by δ = 1(T ≤ C). It is assumed throughout this paper that the censoring mechanism is noninformative. That is, given x and U, T and C are conditionally independent.

Suppose that (x₁, U₁, Z₁, δ₁), …, (x_n, U_n, Z_n, δ_n) is a random sample from model (1). Let $t_1^0<\cdots <t_N^0$ be the ordered observed failure times. Let ( j) be the label for the subject failing at time $t_j^0$, so that the covariates associated with the N failures are x₍₁₎, …, x_(N) and U₍₁₎, …, U_(N). Denote the risk set right before time $t_j^0$ by $R_j=\{i:Z_i\le t_j^0\}$. The partial likelihood function [8] of the random sample is

$${\ell }_p\{\alpha (U)\}=\sum _{j=1}^N\left[{\mathbf{x}}_{(j)}^{\top }\alpha (U_{(j)})-\ln \left[\sum _{i\in R_j}\exp \{{\mathbf{x}}_i^{\top }\alpha (U_i)\}\right]\right].$$ (2)

To estimate the nonparametric regression, we use a B-spline basis. Let S_n be the space of polynomial splines of degree ℓ ≥ 1 and $\{{\psi }_{j1},\hspace{0.17em}\dots \hspace{0.17em},{\psi }_{j{d}_{nj}}\}$ denote a normalized B-spline basis with ${\Vert {\psi }_{jk}\Vert }_{\infty }\le 1$ and d_n j = O(n^1/5), where ${\Vert \hspace{0.17em}\cdot \hspace{0.17em}\Vert }_{\infty }$ is the supremum norm. For any j ∈ {1, …, p} and ${\alpha }_{nj}\in {\mathcal{S}}_n$, we have

$${\alpha }_{nj}(U)=\sum _{k=1}^{d_{nj}}{\beta }_{jk}{\psi }_{jk}(U)={\mathbf{\beta }}_j^{\top }{\mathbf{\psi }}_j(U)$$ (3)

for some coefficients ${\beta }_{j1},\hspace{0.17em}\dots \hspace{0.17em},{\beta }_{j{d}_{nj}}$. Here we allow d_{n j} to increase with n and differ for different j because different coefficient functions may have different smoothness. Under some conditions, the nonparametric coefficient functions ${\alpha }_1(U),\cdots ,{\alpha }_p(U)$ can be well approximated by functions in ${\mathcal{S}}_n$.

Denote $\mathbf{\beta }={({\mathbf{\beta }}_1^{\top },\hspace{0.17em}\dots \hspace{0.17em},{\mathbf{\beta }}_p^{\top })}^{\top }$ and ${\mathbf{z}}_i{(x_{i1}{\mathbf{\psi }}_1{(U_i)}^{\top },\hspace{0.17em}\dots \hspace{0.17em},x_{ip}{\mathbf{\psi }}_p{(U_i)}^{\top })}^{\top }$, and define z_(j) similarly to x_(j). Substituting (3) into (2), the maximum partial likelihood estimate of (2) is to maximize

$${\ell }_p(\mathbf{\beta })\widehat{=}\sum _{j=1}^N\left[z_{(j)}^{\top }\mathbf{\beta }-\ln \left\{\sum _{i\in R_j}\exp (z_i^{\top }\mathbf{\beta })\right\}\right]$$ (4)

with respect to β. We next propose a feature screening procedure based on (4).

2.1. A new feature screening procedure

Denote ${\Vert {\alpha }_j(\cdot )\Vert }_2={\{\text{E}{\alpha }_j^2(U)\}}^{1/2}$, the L₂-norm of α _j(·). For ease of presentation, denote s as an arbitrary subset of {1, …, p}, x_s = {x _j : j ∈ s} and α_s(U) = {α _j(U) : j ∈ s}. For a set s, τ(s) stands for the cardinality of s. Suppose the effect of x is sparse and the true value of α(U) is α∗(U), where β∗ is the corresponding coefficients of α∗(U). Denote $s^{\ast }=\{j:{\Vert {\alpha }_j(\cdot )\Vert }_2>0\}$. By sparsity, we mean that τ(s∗) is much less than p. The goal of feature screening is to identify a subset s such that s∗ ⊂ s with overwhelming probability and τ(s) is also much less than p. According to (4), we propose screening features for the varying-coefficient Cox model by the constrained partial likelihood

$${\widehat{\mathbf{\beta }}}_m=\underset{\mathbf{\beta }}{\arg \max }{\ell }_p(\mathbf{\beta })\hspace{1em}\text{subject}\hspace{0.17em}\text{to}\hspace{1em}\tau (\{j:{\Vert {\mathbf{\beta }}_j\Vert }_2>0\})\le m$$ (5)

for a pre-specified m, which is assumed to be greater than the number of nonzero elements of β*.

For high-dimensional problems, it becomes almost impossible to solve the constrained maximization problem (5) directly. Alternatively, we consider a proxy of the partial likelihood function. It follows by the Taylor expansion for the partial likelihood function ℓ_p(γ) at β lying within a neighborhood of γ that

$${\ell }_p(\mathbf{\gamma })\approx {\ell }_p(\mathbf{\beta })+{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }{\ell }_p^{\prime }(\mathbf{\beta })+{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }{\ell }_p^{″}(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta })/2,$$

where ${\ell }_p^{\prime }(\mathbf{\beta })=\partial {\ell }_p(\mathbf{\gamma })/\partial \mathbf{\gamma }{|}_{\mathbf{\gamma }=\mathbf{\beta }}$ and ${\ell }_p^{″}(\mathbf{\beta })={\partial }^2{\ell }_p(\mathbf{\gamma })/\partial \mathbf{\gamma }\partial {\mathbf{\gamma }}^{\top }{|}_{\mathbf{\gamma }=\mathbf{\beta }}$. Denote $P_t=d_{n1}+\hspace{0.17em}\cdots \hspace{0.17em}+d_{np}$. If ${\ell }_p^{″}(\mathbf{\beta })$ is invertible, the computational complexity of calculating the inverse of ${\ell }_p^{″}(\mathbf{\beta })$ is $O(P_t^3)$. For large P_t, small n problems (i.e., P_t ≫ n), ${\ell }_p^{″}(\mathbf{\beta })$ becomes not invertible. Low computational cost is always desirable for feature screening. To deal with singularity of the Hessian matrix and save computational cost, we propose using the approximation

$$h(\mathbf{\gamma }|\mathbf{\beta })={\ell }_p(\mathbf{\beta })+{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }{\ell }_p^{\prime }(\mathbf{\beta })-u{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }W(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta })/2$$ (6)

for ${\ell }_p^{″}(\mathbf{\gamma })$, where u is a scaling constant to be specified and W(β) = diag{W₁(β), …, W_p(β)}, a block diagonal matrix with W_j(β) being a d_nj × d_nj matrix. Here (6) is the minimization of the original objective function, $h(\mathbf{\gamma }|\mathbf{\beta })\le {\ell }_p(\mathbf{\beta })$, for all γ under some conditions. Due to the properties of the majorization and minorization algorithm, using (6) we can obtain the same estimates as the original objective function. The two functions themselves, however, are not numerically equal. Here we allow W(β) to depend on β. This implies that we approximate ${\ell }_p^{″}(\mathbf{\beta })$ by −uW(β). Throughout this paper, we will use $W_j(\mathbf{\beta })=-{\partial }^2{\ell }_p(\mathbf{\beta })/\partial {\mathbf{\beta }}_j\partial {\mathbf{\beta }}_j^{\top }$.

It can be seen that $h(\mathbf{\beta }|\mathbf{\beta })={\ell }_p(\mathbf{\beta })$, and, under some conditions, $h(\mathbf{\gamma }|\mathbf{\beta })\le {\ell }_p(\mathbf{\beta })$ for all γ. This ensures the ascent property. See Theorem 1 below for more details. Since W(β) is a block diagonal matrix, h(γ|β) is an additive function of γ _j for any given β. The additivity enables us to have a closed form solution for the maximization problem

$$\underset{\mathbf{\gamma }}{\max }h(\mathbf{\gamma }|\mathbf{\beta })\hspace{1em}\text{subject}\hspace{0.17em}\text{to}\hspace{1em}\tau \left(\{j:{\Vert {\mathbf{\gamma }}_j\Vert }_2>0\}\right)\le m$$ (7)

for given β and m. Define ${\tilde{\mathbf{\gamma }}}_j={\mathbf{\beta }}_j+u^{-1}{W}_j^{-1}({\mathbf{\beta }}_j)\partial {\ell }_p(\mathbf{\beta })/\partial {\mathbf{\beta }}_j$ for j ∈ {1, …, p}, and $\tilde{\mathbf{\gamma }}={\left({\tilde{\mathbf{\gamma }}}_1^{\top },\hspace{0.17em}\dots \hspace{0.17em},{\tilde{\mathbf{\gamma }}}_p^{\top }\right)}^{\top }=\mathbf{\beta }+u^{-1}{W}^{-1}(\mathbf{\beta }){\ell }_p^{\prime }(\mathbf{\beta })$ is the maximizer of h(γ|β). Denote $g_j={\tilde{\mathbf{\gamma }}}_j^{\top }{W}_j({\mathbf{\beta }}_j){\tilde{\mathbf{\gamma }}}_j$ for j ∈ {1, …, p}, and sort g_j so that $g_{(1)}\ge \hspace{0.17em}\cdots \hspace{0.17em}\ge g_{(p)}$. The solution of the maximization problem (7) is the hard-thresholding rule defined below:

$${\widehat{\mathbf{\gamma }}}_j={\widehat{\mathbf{\gamma }}}_{j}1\{g_j>g_{(m+1)}\}.$$

This enables us to effectively screen features by using the following algorithm.

$$\begin{array}{l}\text{Feature}\hspace{0.17em}\text{Screening}\hspace{0.17em}\text{Algorithm}\hspace{0.17em}\text{of}\hspace{0.17em}\text{Varying}\hspace{0.17em}\text{Coefficient}\hspace{0.17em}\text{Cox}\text{'}\text{s}\hspace{0.17em}\text{Models}\\ \overline{\begin{array}{l}\mathbf{\text{Step}}\hspace{0.17em}\mathbf{1}.\hspace{0.17em}\text{Set}\hspace{0.17em}\text{the}\hspace{0.17em}\text{initial}\hspace{0.17em}\text{value}\hspace{0.17em}{\mathbf{\beta }}_1^{(0)}=\cdots ={\mathbf{\beta }}_p^{(0)}=\mathbf{0}.\\ \mathbf{\text{Step}}\hspace{0.17em}\mathbf{2}.\hspace{0.17em}\text{For}\hspace{0.17em}t\in \{0,1,\dots \},\hspace{0.17em}\text{iteratively}\hspace{0.17em}\text{conduct}\hspace{0.17em}\text{Step}\hspace{0.17em}\text{2a}\hspace{0.17em}\text{and}\hspace{0.17em}\text{Step}\hspace{0.17em}\text{2b}\hspace{0.17em}\text{below}\\ \hspace{1em}\hspace{1em}\hspace{1em}\text{until}\hspace{0.17em}\text{the}\hspace{0.17em}\text{algorithm}\hspace{0.17em}\text{converges}:\\ \hspace{1em}\hspace{1em}\hspace{1em}\mathbf{\text{Step}}\hspace{0.17em}\mathbf{2}\mathbf{a}.\hspace{0.17em}\text{Calculate}\hspace{0.17em}{\tilde{\mathbf{\gamma }}}_j^{(t)}={\mathbf{\beta }}_j^{(t)}+u_t^{-1}{W}_j^{-1}({\mathbf{\beta }}_j)\partial \ell ({\mathbf{\beta }}^{(t)})/\partial {\mathbf{\beta }}_j,\hspace{0.17em}\text{and}\hspace{0.17em}{g}_j^{(t)}={\{{\tilde{\mathbf{\gamma }}}_j^{(t)}\}}^{\top }{W}_j({\mathbf{\beta }}^{(t)}){\tilde{\mathbf{\gamma }}}_j^{(t)}.\end{array}}\\ \hspace{1em}\hspace{1em}\hspace{1em}\text{Let}\hspace{0.17em}{g}_{(1)}^{(t)}\ge \cdots \ge g_{(p)}^{(t)},\hspace{0.17em}\text{the}\hspace{0.17em}\text{order}\hspace{0.17em}\text{statistics}\hspace{0.17em}of\hspace{0.17em}{g}_j^{(t)}\text{s}.\hspace{0.17em}\text{Set}\hspace{0.17em}{S}_t=\{j:g_j^{(t)}\ge g_{(m+1)}^{(t)}\},\\ \hspace{1em}\hspace{1em}\hspace{1em}\text{the}\hspace{0.17em}\text{nonzero}\hspace{0.17em}\text{index}\hspace{0.17em}\text{set}.\\ \hspace{1em}\hspace{1em}\hspace{1em}\mathbf{\text{Step}}\hspace{0.17em}\mathbf{2}\mathbf{b}.\hspace{0.17em}\text{Update}\hspace{0.17em}\mathbf{\beta }\hspace{0.17em}\text{by}\hspace{0.17em}{\mathbf{\beta }}^{(t+1)}={({\mathbf{\beta }}_1^{(t+1)},\dots ,{\mathbf{\beta }}_p^{(t+1)})}^{\top }\text{as}\hspace{0.17em}\text{follows}.\hspace{0.17em}\text{If}\hspace{0.17em}j\notin S_t,\text{set}\hspace{0.17em}{\mathbf{\beta }}_j^{(t+1)}=\mathbf{0},\\ \underset{\_}{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{otherwise,}\hspace{0.17em}\text{set}\hspace{0.17em}\{{\mathbf{\beta }}_j^{(t+1)}:j\in S_t\}\hspace{0.17em}\text{be}\hspace{0.17em}\text{the}\hspace{0.17em}\text{partial}\hspace{0.17em}\text{likelihood}\hspace{0.17em}\text{estimate}\hspace{0.17em}\text{of}\hspace{0.17em}\text{the}\hspace{0.17em}\text{submodel}\hspace{0.17em}{S}_t.}\end{array}$$

Theorem 1. Suppose that Conditions (D1)–(D4) in the Appendix hold. Let β^(t) be the sequence defined in Step 2b in the above algorithm. Denote

$${\rho }^{(t)}=\underset{\mathbf{\beta }}{\sup }\left[{\lambda }_{\max }\{W^{-1/2}({\mathbf{\beta }}^{(t)})\{-{\ell }_p^{″}(\mathbf{\beta })\}{W}^{-1/2}({\mathbf{\beta }}^{(t)})\}\right],$$

where λ_max(A) stands for the maximal eigenvalue of a matrix A. If u_t ≥ ρ^(t), then ${\ell }_p({\mathbf{\beta }}^{(t+1)})\ge {\ell }_p({\mathbf{\beta }}^{(t)})$, where β^(t+1) is defined in Step 2b in the above algorithm.

Theorem 1 claims the ascent property of the proposed algorithm if u_t is appropriately chosen. That is, the proposed algorithm may improve the current estimate within the feasible region (i.e., $\tau \left(\{j:{\Vert {\alpha }_j(U)\Vert }_2>0\}\right)\le m$), and the resulting estimate in the current step may serve as a refinement of the last step. This theorem provides us with some insights into how to choose u_t in practical implementation.

2.2. Sure screening property

For a subset s of {1, …, p} with size τ(s), recall the notation x_s = {x_j : j ∈ s} and associated coefficients α_s(U) = {α _j(U) : j ∈ s} corresponding to β_s = {β _j : j ∈ s} with ${\mathbf{\beta }}_j={({\beta }_{j1},\hspace{0.17em}\dots \hspace{0.17em}{\beta }_{j{d}_{n_j}})}^{\top }$. We denote the true model by $s^{\ast }=\{j:\text{E}{\alpha }_j^2(U)>0,1\le j\le p\}$ with τ(s∗) = q. The objective of feature screening is to obtain a subset ŝ such that s∗ ⊂ ŝ with very high probability.

We now provide some theoretical justifications for the proposed screening procedure for the ultrahigh-dimensional varying-coefficient Cox model. The sure screening property [12] is referred to as

$$\underset{n\to \infty }{\lim }\mathrm{Pr}(s^{\ast }\subset \widehat{s})=1.$$ (8)

To establish this sure screening property for the proposed screening procedure, we introduce some additional notation as follows. For any model s, let ${\ell }^{\prime }({\mathbf{\beta }}_s)=\partial \ell ({\mathbf{\beta }}_s)/\partial {\mathbf{\beta }}_s$ and ${\ell }^{″}({\mathbf{\beta }}_s)={\partial }^2\ell ({\mathbf{\beta }}_s)/\partial {\mathbf{\beta }}_s\partial {\mathbf{\beta }}_s^{\top }$ be the score function and the Hessian matrix of ℓ as a function of β_s, respectively. Assume that a screening procedure retains m out of p features such that τ(s∗) = q < m. So, we define

$$S_{+}^m=\{s:s^{\ast }\subset s,{\Vert s\Vert }_0\le m\}\hspace{1em}\text{and}\hspace{1em}{S}_{-}^m=\{s:s^{\ast }\not\subset s,{\Vert s\Vert }_0\le m\}$$

as the collections of the over-fitted models and the under-fitted models, respectively. We investigate the asymptotic properties of ${\widehat{\mathbf{\beta }}}_m$ under the scenario where p, q, m and β* are allowed to depend on the sample size n. We impose the following conditions, some of which are purely technical and merely serve to facilitate theoretical understanding of the proposed feature screening procedure. For ease of presentation and without loss of generality, it is assumed that $d_{n1}=\cdots =d_{np}\widehat{=}{d}_n$.

(C1) The support of U is bounded on [a, b].

(C2) The functions α₁(U), …, α_p(U) belong to a class of functions $\mathcal{F}$, whose rth derivative ${\alpha }_j^{(r)}$ exists and is Lipschitz of order η,

$$\mathcal{F}=\{{\alpha }_j:\left|{\alpha }_j^{(r)}(s)-{\alpha }_j^{(r)}(t)\right|\le K{\left|s-t\right|}^{\eta }\text{for}\hspace{0.17em}s,t\in [a,b]\},$$

for some positive constant K, where r is a nonnegative integer and η ∈ (0, 1] such that ν = r + η > 0.5.

(C3) There exist w₁, w₂ > 0 and some non-negative constants τ₁, τ₂ such that τ₁ + τ₂ < ½ and

$$\underset{j\in s^{\ast }}{\min }{\Vert {\alpha }_j(U)\Vert }_2\ge w_1{n}^{-{\tau }_1}\hspace{1em}\text{and}\hspace{1em}q<m\le w_2{n}^{-{\tau }_2}.$$

(C4) ln p = O(n^κ) for some 0 ≤ κ < 1 − 2(τ₁ + τ₂).

(C5) There exist constants C₁, C₂ > 0, δ > 0, such that for sufficiently large n,

$$C_1{d}_n^{-1}\le {\lambda }_{\min }\{-n^{-1}{\ell }_p^{″}({\mathbf{\beta }}_s)\}\le {\lambda }_{\max }\{-n^{-1}{\ell }_p^{″}({\mathbf{\beta }}_s)\}\le C_2{d}_n^{-1},$$

for ${\mathbf{\beta }}_s\in \{\mathbf{\beta }:{\Vert {\mathbf{\beta }}_s-{\mathbf{\beta }}_s^{\ast }\Vert }_2\le \delta \}$ and $s\in S_{+}^{2m}$, where λ_min and λ_max denote the smallest and largest eigenvalues of a matrix, respectively.

Under Conditions (C1)–(C2), the following two properties of B-splines are valid.

(a) de Boor [3]: For k ∈ {1, …, d_n}, ψ _jk (U) ≥ 0 and ${\psi }_{j1}(U)+\cdots +{\psi }_{j{d}_n}(U)=1$, U ∈ [a, b]. In addition, there exist positive constants C₃ and C₄ such that $C_3{d}_n^{-1}\le \text{E}{\psi }_{jk}^2(U)\le C_4{d}_n^{-1}$.

(b) Stone [23, 24]: If {α₁, …, α_p} is a set of functions in $\mathcal{F}$ described in Condition (C2), there exists a positive constant C₅ that does not depend on α _j(U); then the uniform approximation error satisfies $\rho ={\sup }_{U\in [a,b]}{\Vert {\alpha }_j(U)-{\alpha }_{nj}(U)\Vert }_2\le C_5{d}_n^{-v}$ for all j ∈ {1, …, p}, as d_n → ∞.

Conditions (C1)–(C2) ensure properties (a) and (b), which are required for the B-spline approximation and establishing the sure screening properties. Note that ${\Vert {\alpha }_{nj}(U)\Vert }_2^2={\mathbf{\beta }}_j^{\top }\text{E{}{\psi }_j(U){\psi }_j{(U)}^{\top }\}{\mathbf{\beta }}_j$. Based on properties (a) and (b) and Condition (C3), we can derive that

$$\underset{j\in s^{\ast }}{\min }{\Vert {\mathbf{\beta }}_j\Vert }_2\ge w_1{d}_n{n}^{-{\tau }_1}.$$

Condition (C3) states a few requirements for establishing the sure screening property of the proposed procedure. The first one is the sparsity of α∗(U), which makes the sure screening possible with $\tau (\widehat{s})=m>q$. Also, it requires that the minimal component in α∗(U) does not degenerate too quckly, so that the signal is detectable in the asymptotic sequence. Meanwhile, together with (C4), it confines an appropriate order of m that guarantees the identifiability of s∗ over s for τ(s) ≤ m. Condition (C5) assumes that p diverges from n at up to an exponential rate; it implies that the number of covariates can be substantially larger than the sample size.

We establish the sure screening property of the quasi-likelihood estimation in the following theorem.

Theorem 2. Suppose that Conditions (C1)–(C5) and Conditions (D1)–(D7) in the Appendix hold. Let ŝ be the model obtained by Eq. (5) of size m. We have $\mathrm{Pr}(s^{\ast }\subset \widehat{s})\to 1$ as n → ∞.

The proof is given in the Appendix. The sure screening property is an appealing property of a screening procedure because it ensures that the true active predictors are retained in the model selected by the screening procedure. To be distinguished from the SIS procedure, the proposed procedure is referred to as a sure joint screening (SJS) procedure.

3. Numerical studies

In this section, we assess the finite-sample performance of the proposed procedure, compare it with existing procedures via simulation, and illustrate the proposed procedure by an empirical analysis of a genomic data set.

3.1. Simulation studies

The main purpose of our simulation studies is to assess the performance of the proposed procedure by comparing it with the SIS [11] and the SJS [28] procedures for the Cox model. The model sizes selected by the three methods are set to be the same for comparison. We vary the dimension of predictors p, sample size n and sample correlation ρ to examine their impact on the performance of the proposed procedure. We use the success rate of active predictors being selected and computing time as our criteria to compare the performance of screening procedures.

In our simulation, the predictors x are generated from a p-dimensional normal distribution with mean zero and covariance matrix $\Sigma =({\sigma }_{ij})$. Two commonly used covariance structures are used in our simulation:

(S1) Σ is compound symmetric, (i.e., σ_ij = ρ for i ≠ j and equal 1 for i = j). We choose ρ ∈ {0.25, 0.5, 0.75}.

(S2) Σ has autoregressive structure with AR(1), (i.e., σ_ij = ρ^{|i− j|}). We choose ρ ∈ {0.25, 0.5, 0.75}.

We generate the survival time from the Cox model with h₀(t) = 1 and the censoring time from a uniform distribution $\mathcal{U}[0,\hspace{0.17em}10]$. Three different coefficient function settings α(u)s are considered:

(a1): ${\alpha }_1^{(1)}(u)=1+2\sin (2\pi u)$, ${\alpha }_2^{(1)}(u)=1-2\cos (2\pi u)$, ${\alpha }_3^{(1)}(u)=0.5+2u^2$;

(a2): ${\alpha }_1^{(2)}(u)=5\sin (2\pi u)$, ${\alpha }_2^{(2)}(u)=5\cos (2\pi u)$, ${\alpha }_3^{(2)}(u)=2.5+5u^2$;

(a3): ${\alpha }_1^{(3)}(u)=e^{0.5u}$, ${\alpha }_2^{(3)}(u)=2(u^3+1.5{(u-0.5)}^2)$, ${\alpha }_3^{(3)}(u)=2u$.

We consider n ∈ {200, 400}, and p ∈ {2000, 5000}. For the feature screening model size, we follow Liu et al. [20] and set m = ⌊n^0.8/ ln(n^0.8)⌋, where ⌊a⌋ denotes the integer part of a. For each combination of different inputs, we conduct 1000 repetitions.

To illustrate the performance of a statistical procedure in survival data analysis, we want the censoring rates to lie within a reasonable range. Table 1 depicts the censoring rates for the 18 combinations of covariance structure, sample correlation ρ and the values of α(u). The censoring rates range from 22% to 37%, which is reasonable for simulation studies.

Table 1:

Censoring rates.

	ρ = 0.25			ρ = 0.5			ρ = 0.75
Σ	(a1)	(a2)	(a3)	(a1)	(a2)	(a3)	(a1)	(a2)	(a3)
S1	.276	.367	.223	.277	.356	.260	.277	.340	.248
S2	.275	.365	.265	.279	.358	.283	.278	.347	.245

We compare the performance of feature screening procedures using the following two criteria: P_s, the probability that an individual active predictor is selected, and P_a, the probability that all active predictors are selected. It is expected that the performance of the proposed varying-coefficient SJS (VSJS) procedure depends on the following factors: the structure of the covariance matrix, the values of α(u), the dimension of all candidate features p, the sample correlation ρ and the sample size n.

Tables 2–3 report P_s and P_a of VSJS, SIS and SJS for the active predictors under (S1). Overall, VSJS outperforms both SIS and SJS for all the three sets of α(u) in terms of Ps and Pa. For (a1), VSJS achieves a high success rate in detecting signals of ${\alpha }_1^{(1)}$ and ${\alpha }_2^{(1)}$, while SIS and SJS fail from time to time.

Table 2:

Comparison between VSJS, SIS and SJS with Σ = (1 − ρ)I + ρ11^T (n = 200).

	VSJS					SIS					SJS
	Ps			Pa	Time	Ps			Pa	Time	Ps			Pa	Time
α(U)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)
	n = 200, p = 2000 and ρ = .25
α(1)	.989	1	1	.989	74.5	.796	.747	.990	.580	9.5	.499	.419	.936	.190	3.6
α(2)	.999	.998	.999	.996	67.7	.016	.002	1	0	8.3	.018	.037	.999	.002	2.4
α(3)	1	.810	.993	.803	82.2	1	.771	.992	.763	6.0	1	.785	.996	.781	2.8
	n = 200, p = 2000 and ρ = .5
α(1)	.970	.976	.915	.868	68.9	.621	.557	.968	.325	9.2	.392	.311	.863	.092	2.9
α(2)	.922	.922	.990	.848	66.8	.006	.003	1	0	7.8	.020	.052	.997	0	2.5
α(3)	.998	.617	.938	.581	74.8	.999	.611	.932	.573	5.3	1	.574	.932	.542	3.2
	n = 200, p = 2000 and ρ = .75
α(1)	.628	.670	.682	.259	62.4	.357	.316	.879	.093	9.4	.247	.211	.701	.031	3.0
α(2)	.485	.535	.738	.204	67.3	.005	.001	1	0	6.8	.018	.059	.935	0	3.4
α(3)	.910	.361	.686	.247	62.5	.987	.341	.736	.250	5.3	.958	.286	.644	.181	3.4
	n = 200, p = 5000 and ρ = .25
α(1)	1	1	.993	.993	464.0	.721	.649	.983	.456	15.4	.391	.326	.865	.097	32.9
α(2)	.996	.994	1	.990	416.3	.004	.004	1	0	18.1	.007	.016	.994	0	17.6
α(3)	1	.708	.984	.694	451.5	1	.684	.974	.667	15.2	1	.627	.980	.615	16.8
	n = 200, p = 5000 and ρ = .5
α(1)	.925	.930	.845	.725	412.7	.496	.430	.954	.199	22.9	.281	.224	.779	.040	16.8
α(2)	.856	.876	.976	.740	423.7	.005	.002	1	0	16.1	.007	.030	.968	0	18.9
α(3)	.992	.508	.884	.446	390.4	.999	.455	.866	.38	15.2	.998	.435	.878	.383	24.0
	n = 200, p = 5000 and ρ = .75
α(1)	.510	.501	.504	.121	398.1	.261	.218	.803	.042	15.3	.135	.140	.541	.010	20.3
α(2)	.372	.399	.625	.093	396.6	.002	0	.999	0	14.9	.006	.022	.867	0	22.2
α(3)	.892	.276	.597	.158	369.5	.977	.258	.624	.159	13.3	.909	.164	.493	.075	24.7

Table 3:

Comparison between VSJS, SIS and SJS with Σ = (1 − ρ)I + ρ11^T (n = 400).

	VSJS					SIS					SJS
	Ps			Pa	Time	Ps			Pa	Time	Ps			Pa	Time
α(U)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)
	n = 400, p = 2000 and ρ = .25
α(1)	1	1	1	1	217.7	1	.960	1	.960	8.8	.859	.805	.999	.686	5.8
α(2)	1	1	1	1	205.9	.020	.001	1	0	7.9	.010	.076	1	0	5.6
α(3)	1	1	1	1	215.3	1	.974	1	.974	8.3	1	.997	1	.997	4.9
	n = 400, p = 2000 and ρ = .5
α(1)	1	1	1	1	190.2	.900	.871	.999	.779	8.5	.736	.607	.998	.437	4.6
α(2)	1	1	1	1	184.3	.010	.001	1	0	8.5	.023	.133	1	.002	6.3
α(3)	1	.988	1	.988	199.5	1	.918	.997	.916	8.2	1	.944	1	.944	5.1
	n = 400, p = 2000 and ρ = .75
α(1)	.984	.991	.976	.955	169.0	.655	.566	.997	.349	8.6	.474	.356	.955	.155	6.3
α(2)	.998	.995	1	.994	162.2	.001	0	1	0	9.5	.035	.193	.999	.004	6.6
α(3)	1	.733	.982	.719	162.8	1	.676	.968	.657	8.2	1	.576	.938	.540	6.1
	n = 400, p = 5000 and ρ = .25
α(1)	1	1	1	1	1202	.963	.957	1	.920	21.6	.963	.957	1	.920	21.6
α(2)	1	1	1	1	1164	.006	.001	1	0	20.6	.004	.038	1	.001	31.1
α(3)	1	1	1	1	1180	1	.960	1	.960	18.2	1	.993	1	.993	36.5
	n = 400, p = 5000 and ρ = .5
α(1)	1	1	1	1	1086	.849	.798	.999	.669	21.0	.849	.798	.999	.669	21.1
α(2)	1	1	1	1	1101	.001	0	1	0	22.2	.011	.071	1	.002	32.1
α(3)	1	.975	1	.975	1071	1	.840	.998	.838	19.6	1	.872	1	.872	40.3
	n = 400, p = 5000 and ρ = .75
α(1)	1	1	.980	.980	929.0	.562	.426	.994	.224	21.0	.336	.267	.933	.073	35.9
α(2)	.994	.992	.997	.988	936.7	.001	0	1	0	20.8	.016	.109	1	.001	35.3
α(3)	.995	.621	.926	.586	909.6	1	.580	.935	.535	18.3	.999	.446	.900	.401	46.1

We next consider the performance of VSJS under (a2). For the zero-centered ${\alpha }_1^{(2)}$ and ${\alpha }_2^{(2)}$, VSJS successfully detects their variation signal and achieves high success rates. As a comparison, SIS and SJS fail to identify ${\alpha }_1^{(2)}$ and ${\alpha }_2^{(2)}$ as active predictors completely in (a2). In general, VSJS still performs better to some extent in (a3), though SIS slightly outperforms VSJS in a few cases.

Tables 2–3 clearly show how performance is affected by sample correlation ρ, predictor dimension p, and sample size n. When ρ increases, n decreases, or p increases, the three methods perform worse under (S1). Compared to SIS and SJS, the performance of VSJS is more resistant to these changes. Also, Tables 2–3 suggest that VSJS is more computationally inefficient than SIS and SJS.

Tables 4–5 report P_s and P_a of VSJS, SIS, and SJS for the active predictors under (S2). Overall, VSJS still outperforms SIS and SJS. It is worth noting that the three methods have much better performance under (S2) than previous cases under (S1), especially when the correlation ρ is larger. In (a1), VSJS and SIS both perform perfectly and slightly better than SJS. When we consider (a2), SIS and SJS perform better under (S2) and successfully identify ${\alpha }_1^{(2)}$ and ${\alpha }_2^{(2)}$ from time to time. However, VSJS again outperforms them in (a2). For (a3), the three methods achieve almost 100% success rate for selecting active predictors. SJS misses some active predictors in a few cases.

Table 4:

Comparison between VSJS, SIS and SJS with Σ = (ρ^{|i− j|}) (n = 200).

	VSJS					SIS					SJS
	Ps			Pa	Time	Ps			Pa	Time	Ps			Pa	Time
α(U)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)
	n = 200, p = 2000 and ρ = .25
α(1)	1	1	1	1	76.9	1	1	.988	.988	5.2	.856	.809	.997	.684	2.6
α(2)	1	1	1	1	70.8	.042	.116	1	.008	5.9	.047	.027	1	0	2.4
α(3)	1	1	1	1	86.6	1	1	1	1	7.1	1	.981	1	.981	3.0
	n = 200, p = 2000 and ρ = .5
α(1)	1	1	1	1	73.1	1	1	.999	.999	8.2	.889	.792	.990	.690	2.5
α(2)	1	1	1	1	67.6	.166	.611	1	.145	5.8	.052	.065	1	.011	2.4
α(3)	1	1	1	1	82.8	1	1	1	1	7.7	1	.977	1	.977	3.1
	n = 200, p = 2000 and ρ = .75
α(1)	1	1	1	1	75.5	1	1	1	1	5.2	.877	.768	.990	.642	3.0
α(2)	1	1	1	1	68.6	.722	.968	1	.720	5.8	.125	.417	.997	.076	2.6
α(3)	1	.997	1	.997	79.4	1	1	1	1	8.4	1	.926	.991	.917	3.1
	n = 200, p = 5000 and ρ = .25
α(1)	1	1	1	1	456.4	.968	.997	1	.965	15.4	.785	.734	.989	.559	16.1
α(2)	1	1	1	1	463.8	.016	.067	1	.004	14.6	.016	.022	.999	0	14.9
α(3)	1	1	.998	.998	477.1	1	.999	1	.999	16.2	1	.967	1	.967	20.1
	n = 200, p = 5000 and ρ = .5
α(1)	1	1	1	1	451.1	1	1	1	1	13.1	.799	.730	.979	.543	13.2
α(2)	1	1	1	1	439.9	.121	.501	1	.103	14.3	.030	.025	1	.003	16.0
α(3)	1	1	1	1	475.4	1	1	1	1	15.8	1	.966	.997	.963	20.3
	n = 200, p = 5000 and ρ = .75
α(1)	1	1	1	1	448.2	1	1	1	1	15.4	.844	.685	.987	.538	19.0
α(2)	1	1	1	1	427.3	.627	.938	1	.626	14.8	.062	.327	1	.040	15.9
α(3)	1	.996	1	.996	453.9	1	1	1	1	14.4	1	.916	.980	.896	23.3

Table 5:

Comparison between VSJS, SIS and SJS with Σ = (ρ^{|i− j|}) (n = 400).

	VSJS					SIS					SJS
	Ps			Pa	Time	Ps			Pa	Time	Ps			Pa	Time
α(U)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)	X1	X2	X3	all	(s)
	n = 400, p = 2000 and ρ = .25
α(1)	1	1	1	1	229.6	1	1	1	1	8.6	.991	.979	1	.970	6.3
α(2)	1	1	1	1	223.3	.083	.251	1	.036	8.5	.047	.040	1	.001	5.2
α(3)	1	1	1	1	240.1	1	1	1	1	11.9	1	1	1	1	7.0
	n = 400, p = 2000 and ρ = .5
α(1)	1	1	1	1	225.9	1	1	1	1	7.5	.992	.959	1	.951	5.3
α(2)	1	1	1	1	226.1	.387	.922	1	.382	8.8	.070	.263	1	.031	5.2
α(3)	1	1	1	1	236.8	1	1	1	1	8.5	1	1	1	1	7.3
	n = 400, p = 2000 and ρ = .75
α(1)	1	1	1	1	217.9	1	1	1	1	8.9	.979	.907	1	.886	6.4
α(2)	1	1	1	1	218.4	.969	1	1	.969	9.1	.139	.598	1	.080	5.8
α(3)	1	.999	1	.999	227.8	1	1	1	1	11.9	1	.997	1	.997	7.6
	n = 400, p = 5000 and ρ = .25
α(1)	1	1	1	1	1264	1	1	1	1	20.6	.988	.962	1	.952	29.5
α(2)	1	1	1	1	1265	.054	.183	1	.018	18.7	.029	.032	1	0	28.8
α(3)	1	1	1	1	1215	1	1	1	1	20.8	1	1	1	1	33.8
	n = 400, p = 5000 and ρ = .5
α(1)	1	1	1	1	1274	1	1	1	1	20.5	.976	.924	1	.900	32.5
α(2)	1	1	1	1	1256	.318	.884	1	.312	19.9	.038	.162	1	.017	29.1
α(3)	1	1	1	1	1194	1	1	1	1	20.6	1	.999	1	.999	35.6
	n = 400, p = 5000 and ρ = .75
α(1)	1	1	1	1	1202	1	1	1	1	20.7	.969	.902	1	.871	36.9
α(2)	1	1	1	1	1225	.954	1	1	.954	21.9	.085	.548	1	.051	29.9
α(3)	1	1	1	1	1139	1	1	1	1	29.5	1	.995	1	.995	34.6

We can conclude from Tables 4 and 5 that SIS and SJS tend to perform better when ρ increases, n increases, or p decreases. For VSJS, it performs perfectly in all three settings under (S1). Similarly, Tables 4 and 5 show that VSJS is more computationally intensive than SIS and SJS.

3.2. Real data analysis

We analyze The Cancer Genome Atlas (TCGA; http://cancergenome.nih.gov/) data on liver hepatocellular carcinoma to illustrate the proposed procedure. Liver hepatocellular carcinoma is the most common form of liver cancer and the third cancer death cause worldwide. Zhang and Sun [31] studied 17,255 patients in the Surveillance, Epidemiology, and End Results Program (SEER; https://seer.cancer.gov/) cancer registry and suggested that age is a prognostic factor for liver cancer. Therefore, we consider age as the univariate covariate for coefficient functions, allowing the effects of gene expression on survival time to vary with age. After removing five subjects whose survival time is zero, we obtain 354 subjects with gene expressions (IlluminaHiSeq RNA-seq v2 platform), age at diagnosis, and survival months. We apply a log 2 transformation to gene expressions and analyze 14,683 genes that have more than 90% nonzero observations.

For VSJS, we use a linear combination of five B-spline basis functions to approximate the varying-coefficient functions. As a result, VSJS retains 23 = ⌊354^0.8/ ln(354^0.8)⌋ genes and the partial likelihood function value for the corresponding model is −544.9. With the same number of genes retained, the resulting partial likelihood function values for SIS and SJS are −589.2 and −588.4, respectively. Simultaneous modeling of the 23 retained genes shows a clear advantage of VSJS in terms of higher partial likelihood value.

To better understand the screening result of VSJS, we apply the backward selection procedure to those 23 genes and obtain a more parsimonious model. Specifically, each backward elimination step removes a gene with the smallest likelihood ratio test statistic until all the genes are significant at level 0.05. Table 6 provides the final list of 11 genes after applying the backward elimination, and Figure 1 depicts their varying coefficients.

Table 6:

Genes selected by backward elimination.

Gene Name	ANLN	CEP55	DYNC1LI1	GTPBP4
LRT Stat	15.869	14.137	18.171	22.658
p-value	0.00723	0.0148	0.00274	< 001
Gene Name	SLC2A1	KIF2C	KIF20A	KPNA2
LRT Stat	18.465	26.261	15.839	14.511
p-value	0.00241	< 0.001	0.00731	0.0127
Gene Name	LIMS2	TRIP13	UCK2
LRT Stat	23.093	17.517	14.671
p-value	< 0.001	0.00361	0.0119

Figure 1:

Estimated coefficient functions and the pointwise conference intervals of selected genes. The red line represents the average level of the varying-coefficient functions.

Our literature search reveals that those 11 genes are all associated with cancer risk and some genes. For example, GTPBP4 [21] and SLC2A2 [16] are promising prognostic factors for hepatocellular carcinoma. To test whether those 11 genes have varying coefficients versus constant coefficients, a test of ${\mathcal{H}}_0:{\alpha }_j(u)={\alpha }_j$ for some constant α _j versus ${\mathcal{H}}_1:{\alpha }_j(u)\equiv {\alpha }_j$ can be conducted for each j in the selected gene set. The test result is shown in Table 7, and all the genes except DYNC1LI1 have significant varying-coefficient functions of age at the 5% level of significance. There is no evidence of their time-varying effects in the current medical literature, but our study may suggest some evidence for potential granular investigation on those genes.

Table 7:

LRT statistics and p-values for the varying coefficients of the final selected genes

Gene Name	ANLN	CEP55	DYNC1LI1	GTPBP4
LRT Stat	15.058	10.495	8.268	19.036
p-value	0.00458	0.0328	0.0822	0.000773
Gene Name	SLC2A1	KIF2C	KIF20A	KPNA2
LRT Stat	17.473	24.253	15.183	14.238
p-value	0.00156	0.000071	0.00433	0.00657
Gene Name	LIMS2	TRIP13	UCK2
LRT Stat	23.097	16.191	13.803
p-value	0.000121	0.00277	0.00795

4. Discussion

We have proposed an SJS procedure for the varying-coefficient Cox model with ultrahigh-dimensional covariates based on partial likelihood. The proposed SJS is distinguished from the existing SIS procedure in that the proposed procedure is based on the joint likelihood of potential candidate features. We also proposed an effective algorithm to carry out the feature screening procedures and show that the proposed algorithm possesses an ascent property. We studied the sampling property of SJS and established the sure screening property for SJS.

Theorem 1 ensures the ascent property of the proposed algorithm under certain conditions, but it does not imply that the proposed algorithm converges to the global optimizer. If the proposed algorithm converges to a global maximizer of (5), then Theorem 2 shows that such a solution enjoys the sure screen property.

Appendix

We use the following notation to present the regularity conditions for the partial likelihood and the Cox model. Most notations are adapted from Andersen and Gill [1], in which counting processes were introduced for the Cox model and the consistency and asymptotic normality of the partial likelihood estimate were established. Denote ${\overline{N}}_i(t)=1(T_i\le t,T_i\le C_i)$ and R_i(t) = {T_i ≥ t, C_i ≥ t}. Assume that there are no two component processes N_i(t) jumping at the same time. For simplicity, we work on the finite interval [0, τ].

In Cox’s model, properties of stochastic processes, such as being a local martingale or a predictable process, are relative to a right-continuous nondecreasing family $\{{\mathcal{F}}_t:t\in [0,\tau ]\}$ of sub σ-algebras on a sample space $(\mathrm{\Omega },\mathcal{F},\mathcal{P})$; ${\mathcal{F}}_t$ represents everything that happens up to time t. Throughout this section, we define ${\mathrm{\Lambda }}_0(t)={\int }_0^t{h}_0(u)du$

By stating that ${\overline{N}}_i(t)$ has intensity process $h_i(t)\widehat{=}h(t|{\mathbf{x}}_i,U_i)$, we mean that the processes M_i(t) defined, for each i ∈ {1, …, n}, by

$$M_i(t)={\overline{N}}_i(t)-{\int }_0^{\top }{h}_i(u)du,$$

are local martingales on the time interval [0, τ]. For k ∈ {0, 1, 2}, define

$${\mathbf{S}}^{(k)}(\beta ,t)=\frac{1}{n}\sum _{i=1}^n{R}_i(t)\exp ({\mathbf{z}}_i^{\top }\beta ){\mathbf{z}}_i^{\otimes k},\hspace{1em}{\mathbf{s}}^{(k)}(\beta ,t)=\text{E}\{{\mathbf{S}}^{(k)}(\beta ,t)\}$$

and

$$\text{E}(\beta ,t)={\mathbf{S}}^{(1)}(\beta ,t)/{\mathbf{S}}^{(0)}(\beta ,t),\hspace{1em}\text{V}(\beta ,t)={\mathbf{S}}^{(2)}(\beta ,t)/{\mathbf{S}}^{(0)}(\beta ,t)-\text{E}{(\beta ,t)}^{\otimes 2},$$

where ${\mathbf{z}}_i^{\otimes 0}=1$, ${\mathbf{z}}_i^{\otimes 1}={\mathbf{z}}_i$ and ${\mathbf{z}}_i^{\otimes 2}={\mathbf{z}}_i{\mathbf{z}}_i^{\top }$. Note that S⁽⁰⁾(β, t) is a scalar, S⁽¹⁾(β, t) and E(β, t) are p-vector, and S⁽²⁾(β, t) and V(β, t) are p × p matrices. Define

$$Q_j=\sum _{i=1}^n{\int }_0^{t_j}\left\{{\mathbf{z}}_i-\sum _{i\in R_j}{\mathbf{z}}_i\exp ({\mathbf{z}}_i^{\top }\beta )/\sum _{i\in R_j}\exp ({\mathbf{z}}_i^{\top }\beta )\right\}d{M}_i.$$

Here, $\text{E}(Q_j|{\mathcal{F}}_{j-1})=Q_{j-1}$, i.e., $\text{E}(Q_j-Q_{j-1}|{\mathcal{F}}_{j-1})=0$. Let b_j = Q_j − Q_j−1, then b₁, b₂, … is a sequence of bounded martingale differences on $(\mathrm{\Omega },\mathcal{F},P)$. That is, b_j is bounded almost surely and $\text{E}(b_j|{\mathcal{F}}_{j-1})=0$ as for j ∈ {1, 2, …}.

(D1) Finite interval: ${\mathrm{\Lambda }}_0(\tau )={\int }_0^{\tau }{h}_0(t)dt<\infty$.

(D2) Asymptotic stability: There exists a neighborhood $\mathcal{B}$ of β* and scalar, vector and matrix functions s⁽⁰⁾,s⁽¹⁾ and s⁽²⁾ defined on $\mathcal{B}\times [0,\tau ]$ such that for k ∈ {0, 1, 2},

$$\underset{t\in [0,\tau ],\beta \in \mathcal{B}}{\sup }\Vert {\mathbf{S}}^{(k)}(\beta ,t)-{\mathbf{s}}^{(k)}(\beta ,t)\Vert \stackrel{p}{\to }0.$$

(D3) Lindeberg condition: There exists δ > 0 such that

$$n^{-1/2}\underset{i,t}{\sup }\left|{\mathbf{z}}_i\right|R_i(t)\mathbf{1}\{{\beta }_0^{\top }{\mathbf{z}}_i>-\delta \left|{\mathbf{z}}_i\right|\}\stackrel{p}{\to }0,$$

(D4) Asymptotic regularity conditions: Let $\mathcal{B}$, s⁽⁰⁾, s⁽¹⁾ and s⁽²⁾ be as in Condition (D2) and define e = s⁽¹⁾/s⁽⁰⁾ and v = s⁽²⁾/s⁽⁰⁾ − e^⊗2. For all $\beta \in \mathcal{B}$, t ∈ [0, τ],

$${\mathbf{s}}^{(1)}(\beta ,t)=\partial {\mathbf{s}}^{(0)}(\beta ,t)/\partial \beta ,\hspace{1em}{\mathbf{s}}^{(2)}(\beta ,t)={\partial }^2{\mathbf{s}}^{(0)}(\beta ,t)/\partial {\beta }^2,$$

s⁽⁰⁾(·, t), s⁽¹⁾(·, t) and s⁽²⁾(·, t) are continuous functions of $\beta \in \mathcal{B}$, uniformly in t ∈ [0, τ], s⁽⁰⁾, s⁽¹⁾ and s⁽²⁾ are bounded on $\mathcal{B}\times [0,\tau ]$; s⁽⁰⁾ is bounded away from zero on $\mathcal{B}\times [0,\tau ]$, and the matrix

$$\mathbf{S}={\int }_0^{\tau }v({\beta }_0,t){\mathbf{s}}^{(0)}({\beta }_0,t)h_0(t)dt$$

is positive definite.

(D5) The function S⁽⁰⁾(β*, t) and s⁽⁰⁾(β*, t) are bounded away from 0 on [0, τ].

(D6) There exist constants C₁, C₂ > 0, such that max_ij |z_ij| < C₁ and ${\max }_i\left|{\mathbf{z}}_i^{\top }{\beta }^{\ast }\right|<C_2$.

(D7) b₁, b₂, … is a sequence of martingale differences and there exist nonnegative constants c₁, …, c_N such that for every real number t and all j ∈ {1, …, N}, $\text{E}\{\exp (t{b}_j)|{\mathcal{F}}_{j-1}\}\le \exp (c_j^2{t}^2/2)$ almost surely. For each j ∈ {1, …, N}, the minimum of those c_j is denoted by η(b_j) and |b_j| ≤ K_j as and E(b_j1, b_j2, …, b_jk ) = 0 for b_j1 < … < b_jk.

Note that the partial derivative conditions on s⁽⁰⁾, s⁽¹⁾ and s⁽²⁾ are satisfied by S⁽⁰⁾, S⁽¹⁾ and S⁽²⁾; furthermore, S is automatically positive semidefinite. Moreover, the interval [0, τ] in the conditions may everywhere be replaced by the set {t : h₀(t) > 0}.

Conditions (D1)–(D5) are standard requirements for the proportional hazards model [1], which are weaker than those required by Bradic et al. [4], and S^(k)(β₀, t) converges uniformly to s^(k)(β₀, t). Condition (D6) is a routine one, which is needed to apply the concentration inequality for general empirical processes. For example, the bounded covariate assumption is used by Huang et al. [15] for discussing the Lasso estimator of proportional hazards models. Condition (D7) is needed for the asymptotic behavior of the score function ${\ell }_p^{\prime }(\mathbf{\beta })$ of partial likelihood because the score function cannot be represented as a sum of independent random vectors, but it can be represented as sum of a sequence of martingale differences.

Proof of Theorem 1. Applying the Taylor expansion to ℓ_p(γ) at γ = β, one finds

$${\ell }_p(\mathbf{\gamma })={\ell }_p(\mathbf{\beta })+{\ell }_p^{\prime }(\mathbf{\gamma })(\mathbf{\gamma }-\mathbf{\beta })+{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }{\ell }_p^{″}(\tilde{\mathbf{\beta }})(\mathbf{\gamma }-\mathbf{\beta })/2,$$

where $\tilde{\mathbf{\beta }}$ lies between γ and β.

$$\begin{gathered} {(\mathbf{\gamma }-\mathbf{\beta })}^{\top }\{-{\ell }_p^{″}(\tilde{\mathbf{\beta }})\}(\mathbf{\gamma }-\mathbf{\beta })={(\mathbf{\gamma }-\mathbf{\beta })}^{\top }{W}^{1/2}(\mathbf{\beta })W^{-1/2}(\mathbf{\beta })\{-{\ell }_p^{″}(\tilde{\mathbf{\beta }})\}{W}^{-/2}(\mathbf{\beta })W^{1/2}(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta }) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le {\lambda }_{\max }[W^{-1/2}(\mathbf{\beta })\{-{\ell }_p^{″}(\tilde{\mathbf{\beta }})W^{-1/2}(\mathbf{\beta })]{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }W(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta }), \end{gathered}$$

where W(β) is a block diagonal matrix with W_j(β) being a $d_{n_j}\times d_{n_j}$ matrix. Given that −ℓ″(β) is non-negative definite, ${\lambda }_{\max }[W^{-1/2}(\mathbf{\beta })\{-{\ell }_p^{″}(\tilde{\mathbf{\beta }})\}{W}^{-1/2}(\mathbf{\beta })]\ge 0$. Thus, if $u>{\lambda }_{\max }[W^{-1/2}(\mathbf{\beta })\{-{\ell }_p^{″}(\tilde{\mathbf{\beta }})\}{W}^{-1/2}(\mathbf{\beta })]\ge 0$, then

$${\ell }_p(\mathbf{\gamma })\ge {\ell }_p(\mathbf{\beta })+{\ell }_p^{\prime }(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta })-u{(\mathbf{\gamma }-\mathbf{\beta })}^{\top }W(\mathbf{\beta })(\mathbf{\gamma }-\mathbf{\beta })/2=h(\mathbf{\gamma }|\mathbf{\beta }).$$

Thus it follows that ${\ell }_p(\mathbf{\gamma })\ge h(\mathbf{\gamma }|\mathbf{\beta })$ and ${\ell }_p(\mathbf{\beta })=h(\mathbf{\beta }|\mathbf{\beta })$ by the definition of h(γ|β). The solution of $\partial h(\mathbf{\gamma }|\mathbf{\beta })/\partial \mathbf{\gamma }=0$ is $\mathbf{\gamma }=\mathbf{\beta }+u^{-1}W(\mathbf{\beta }\hspace{0.17em}){\ell }^{\prime }(\mathbf{\beta })$. Hence, under the conditions of Theorem 1, it follows that

$${\ell }_p({\mathbf{\beta }}_j^{\ast (t+1)})\ge h({\mathbf{\beta }}^{\ast (t+1)}|{\mathbf{\beta }}^{(t)})\ge h({\mathbf{\beta }}^{(t)}|{\mathbf{\beta }}^{(t)})=\ell ({\mathbf{\beta }}^{(t)}).$$

The second inequality is due to the fact that

$$\tau [\{j:{\Vert {\mathbf{\beta }}_j^{\ast (t+1)}\Vert }_2>0\}]=\tau [\{j:{\Vert {\mathbf{\beta }}_j^{(t)}\Vert }_2>0\}]=m$$

and ${\mathbf{\beta }}^{\ast (t+1)}=\arg {\min }_{\mathbf{\gamma }}h(\mathbf{\gamma }|{\mathbf{\beta }}^{(t)})$ subject to $\tau [\{j:{\Vert {\mathbf{\gamma }}_j\Vert }_2>0\}]\le m$. By definition of β^(t+1), ${\ell }_p({\mathbf{\beta }}^{(t+1)})\ge {\ell }_p({\mathbf{\beta }}^{\ast (t+1)})$ and $\tau [\{j:{\Vert {\mathbf{\beta }}_j^{(t+1)}\Vert }_2>0\}]=m$. This proves Theorem 1. □

Proof of Theorem 2. For a given model s, a subset of {1, …, p}, let ${\widehat{\mathbf{\alpha }}}_s(U)$ be the partial likelihood estimate of α_s(U) based on the spline approximation. The theorem is implied if $\mathrm{Pr}(\widehat{s}\in S_{+}^m)\to 1$. Thus, it suffices to show that

$$\underset{n\to \infty }{\lim }\mathrm{Pr}\left[\underset{s\in S_{-}^m}{\max }{\ell }_p\{{\widehat{\mathbf{\alpha }}}_s(U)\}\ge \underset{s\in S_{+}^m}{\min }{\ell }_p\{{\widehat{\mathbf{\alpha }}}_s(U)\}\right]=0,$$ (A.1)

For each j ∈ {1, …, p}, we approximate the coefficient function α _j(U) by

$${\alpha }_{nj}(U)=\sum _{k=1}^{d_n}{\beta }_{jk}{\psi }_{jk}(U)={\mathbf{\beta }}_j^{\top }{\mathbf{\psi }}_j(U).$$ (A.2)

where ${\psi }_{j1}(U),\hspace{0.17em}\dots \hspace{0.17em},{\psi }_{j{d}_n}(U)$ are basis functions and d_n is the number of basis functions, which is allowed to increase with the sample size n. For α_{n j}(U), define the approximation error for each j ∈ {1, …, p}, by

$${\rho }_j(U)={\alpha }_j(U)-{\alpha }_{nj}(U)={\alpha }_j(U)-{\mathbf{\beta }}_j^{\top }{\mathbf{\psi }}_j(U).$$

Let $\{{\alpha }_j(U),{\mathcal{S}}_j\}={\inf }_{{\alpha }_{nj}(U)\in {\mathcal{S}}_j}{\sup }_{U\in [a,b]}{\Vert {\rho }_j(U)\Vert }_2$, and take $\rho ={\max }_{1\le j\le p}\text{dist}\{{\alpha }_j(U),{\mathcal{S}}_j\}$. Let α_n(U) = (α_n1(U), …, α_np(U))^T and α(U) = (α₁(U), …, α_p(U))^T. For any s,

$${\mathbf{\alpha }}_s(U)={\left(\begin{array}{ccc}{\mathbf{\psi }}_1(U)& \hspace{0.17em}& \hspace{0.17em}\\ \hspace{0.17em}& \ddots & \hspace{0.17em}\\ \hspace{0.17em}& \hspace{0.17em}& {\mathbf{\psi }}_s(U)\end{array}\right)}_{s\times s{d}_n}{\left(\begin{array}{c}{\mathbf{\beta }}_1\\ ⋮\\ {\mathbf{\beta }}_s\end{array}\right)}_{s{d}_n\times 1}+\left(\begin{array}{c}{\rho }_1(U)\\ ⋮\\ {\rho }_s(U)\end{array}\right)\widehat{=}{\mathrm{\Psi }}_s(U){\mathbf{\beta }}_s+{\rho }_s(U),$$

where Ψ_s(U) = diag{ψ₁(U), …, ψ_s(U)} with ${\mathbf{\psi }}_j(U)=({\psi }_{j1}(U),\hspace{0.17em}\dots \hspace{0.17em},{\psi }_{j{d}_n}(U))$, and ${\mathbf{\beta }}_j={({\beta }_{j1},\hspace{0.17em}\dots \hspace{0.17em},{\beta }_{j{d}_n})}^{\top }$ for all j ∈ {1, …, s}. For any $s\in S_{-}^m$, define $s\prime =s\cup s^{\ast }\in S_{+}^{2m}$. So, we have

$$\begin{gathered} {\ell }_p\{{\mathbf{\alpha }}_{s\prime }(U)\}-{\ell }_p\{{\mathbf{\alpha }}_{s\prime }^{\ast }(U)\}={\ell }_p\{{\mathrm{\Psi }}_{s\prime }(U){\mathbf{\beta }}_{s\prime }+{\rho }_{s\prime }(U)\}-{\ell }_p\{{\mathrm{\Psi }}_{s\prime }(U){\mathbf{\beta }}_{s\prime }^{\ast }+{\rho }_{s\prime }^{\ast }(U)\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\ell }_p\{{\mathrm{\Psi }}_{s\prime }(U){\mathbf{\beta }}_{s\prime }\}+{\ell }_p^{\prime }\{{\mathrm{\Psi }}_{s\prime }(U){\tilde{\mathbf{\beta }}}_{s\prime }\}{\rho }_{s\prime }(U)-{\ell }_p\{{\mathrm{\Psi }}_{s\prime }(U){\mathbf{\beta }}_{s\prime }^{\ast }\}-{\ell }_p^{\prime }\{{\mathrm{\Psi }}_{s\prime }(U){\tilde{\mathbf{\beta }}}_{s\prime }^{\ast }\}{\rho }_{s\prime }^{\ast }(U), \end{gathered}$$

where ${\tilde{\mathbf{\beta }}}_{s\prime }$ and ${\tilde{\mathbf{\beta }}}_{s\prime }^{\ast }$ are two immediate values. Denote

$${\mathrm{\Delta }}_1=\{{\ell }_p({\mathbf{\beta }}_{s\prime })-{\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })\},\hspace{1em}{\mathrm{\Delta }}_2={\ell }_p^{\prime }({\tilde{\mathbf{\beta }}}_{s\prime }){\rho }_{s^{\prime }}(U),\hspace{1em}{\mathrm{\Delta }}_3={\ell }_p^{\prime }({\tilde{\mathbf{\beta }}}_{s\prime }^{\ast }){\rho }_{s\prime }^{\ast }(U).$$

Thus, we have ${\ell }_p\{{\mathbf{\alpha }}_{s\prime }(U)\}-{\ell }_p\{{\mathbf{\alpha }}_{s\prime }^{\ast }(U)\}={\mathrm{\Delta }}_1+{\mathrm{\Delta }}_2+{\mathrm{\Delta }}_3$. For ∆₂, by the Cauchy–Schwarz inequality, we have

$$\text{E}\left|{\mathrm{\Delta }}_2\right|=\text{E}\left|{\ell }_p^{\prime }({\tilde{\mathbf{\beta }}}_{s\prime }){\rho }_{s\prime }(U)\right|\le \sqrt{\text{E}{\Vert {\ell }_p^{\prime }({\tilde{\mathbf{\beta }}}_{s\prime })\Vert }^2}\sqrt{\text{E}{\Vert {\rho }_{s\prime }(U)\Vert }^2}.$$

By Condition (C5) and Corollary 1 in [25], we obtain ∆₂ = o_p(1). Similarly to ∆₂, we can also conclude that ∆₃ = o_p(1).

Next, we consider the term ∆₁. For any $s\in S_{-}^m$, define $s^{\prime }=s\cup s^{\ast }\in S_{+}^{2m}$. Under Condition (C3), we consider ${\mathbf{\beta }}_{s\prime }$ close to ${\mathbf{\beta }}_{s\prime }^{\ast }$ such that $\Vert {\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast }\Vert =w_1{d}_n{n}^{-{\tau }_1}$ for some w₁, τ₁ > 0. Clearly, when n is sufficiently large, β_s′ falls into a small neighborhood of ${\mathbf{\beta }}_{s\prime }^{\ast }$, so that Condition (C5) becomes applicable. Thus, it follows from Condition (C5) and the Cauchy–Schwarz inequality that

$$\begin{gathered} {\ell }_p({\mathbf{\beta }}_{s\prime })-{\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })={({\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast })}^{\top }{\ell }_p^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })+(1/2){({\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast })}^{\top }{\ell }_p^{″}({\tilde{\mathbf{\beta }}}_{s\prime })({\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast }) \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\le {({\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast })}^{\top }{\ell }_p^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })-(C_1{d}_n^{-1}/2)n{\Vert {\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast }\Vert }_2^2 \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\le w_1{d}_n{n}^{-{\tau }_1}{\Vert {\ell }_p^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\Vert }_2-(C_1{d}_n/2)w_1^2{n}^{1-2{\tau }_1}, \end{gathered}$$ (A.3)

where ${\tilde{\mathbf{\beta }}}_{s\prime }$ is an intermediate value between β_s′ and ${\mathbf{\beta }}_{s\prime }^{\ast }$. Thus, we have

$$\begin{gathered} \mathrm{Pr}\{{\ell }_p({\mathbf{\beta }}_{s\prime })-{\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })\ge 0\}\le \mathrm{Pr}\{{\Vert {\ell }_p^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\Vert }_2\ge (C_1{w}_1/2)n^{1-{\tau }_1}\}=\mathrm{Pr}\left[\sum _{j\in s\prime }{\{{\ell }_p^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\}}^2\ge {(C_1{w}_1/2)}^2{n}^{2-2{\tau }_1}\right] \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le \sum _{j\in s\prime }\mathrm{Pr}[{\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\}}^2\ge {(2m)}^{-1}{(C_1{w}_1/2)}^2{n}^{2-2{\tau }_1}]. \end{gathered}$$

Also, by (C3), we have $m\le w_2{n}^{{\tau }_2}$, and also the following probability inequality

$$\begin{gathered} \mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\ge {(2m)}^{-1/2}(C_1{w}_1/2)n^{1-{\tau }_1}\}\le \mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\ge {(2w_2{n}^{{\tau }_2})}^{-1/2}(C_1{w}_1/2)n^{1-{\tau }_1}\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\le c{n}^{1-{\tau }_1-0.5{\tau }_2}\}=\mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\le nc{n}^{1-{\tau }_1-0.5{\tau }_2}\}, \end{gathered}$$ (A.4)

where $c=C_1{w}_1/(2\sqrt{2w_2})$ denotes some generic positive constant. Recall (2), by differentiation and rearrangement of terms, it can be shown as in [1] that the gradient of ℓ_p(β) is

$${\ell }_p^{\prime }(\mathbf{\beta })\equiv \partial {\ell }_p(\mathbf{\beta })/\partial \mathbf{\beta }=\frac{1}{n}\sum _{i=1}^n{\int }_0^{\infty }\{z_i-{\overline{z}}_n(\mathbf{\beta },t)\}d{\overline{N}}_i(t),$$ (A.5)

where ${\overline{\mathbf{z}}}_n(\mathbf{\beta },t)={\sum }_{i\in R}{\mathbf{z}}_i\exp ({\mathbf{z}}_i^{\top }\mathbf{\beta })/{\sum }_{i\in R}\exp ({\mathbf{z}}_i^{\top }\mathbf{\beta })$. As a result, the partial score function ${\ell }_p^{\prime }(\mathbf{\beta })$ no longer has a martingale structure, and the large deviation results for continuous time martingale in [4] and [15] are not directly applicable. The martingle process associated with ${\overline{N}}_i(t)$ is given by

$$M_i(t)={\overline{N}}_i(t){\int }_0^{\top }{R}_i(u)\exp (z^{\top }{\mathbf{\beta }}^{\ast })d{\mathrm{\Lambda }}_0(u).$$

For each j ∈ {1, …, N}, let t_j be the time of the jth jump of the process ${\sum }_{i=1}^n{\int }_0^{\infty }{R}_i(t)d{\overline{N}}_i(t)$ and set t₀ = 0. Then, t _j are stopping times. For j ∈ {0, …, N}, further define

$$Q_j=\sum _{i=1}^n{\int }_0^{t_j}{b}_i(u)d{\overline{N}}_i(u)=\sum _{i=1}^n{\int }_0^{t_j}{b}_i(u)d{M}_i(u).$$ (A.6)

where $b_i(u)=z_i-{\overline{z}}_n(\mathbf{\beta },u)$ for all i ∈ {1, …, n} are predictable, provided that no two component processes jump at the same time, (D6 holds), and |b_i(u)| ≤ 1.

Since M_i(u) are martingales and b_i(u) are predictable, {Q₀, Q₁, …} is a martingale with the difference |Q_j − Q_j−1| ≤ max_u,i |b_i(u)| ≤ 1. Recall definition of N in Section 2, we define $C_0^{2}n\le N$, where C₀ is a constant. So, by the martingale version of Hoeffding’s inequality [2] and under Condition (D7), we have

$$\mathrm{Pr}(\left|Q_N\right|>n{C}_{0}x)\le 2\exp \{-n^2{C}_0^2{x}^2/(2N)\}\le 2\exp (-n{x}^2/2).$$

By (A.6), $Q_N=n{\ell }_p^{\prime }(\mathbf{\beta })$ if and only if ${\sum }_{i=1}^n{\int }_0^{\infty }{R}_i(t)d{\overline{N}}_i(t)\le N$ Thus, the left-hand side of (3.15) in Lemma 3.3 of [15] is no greater than Pr(|Q_N| > nC₀ x) ≤ 2 exp(−nx²/2). Now (A.4) can be rewritten as follows:

$$\mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\ge nc{n}^{-{\tau }_1-0.5{\tau }_2}\}\le \exp \{-0.5n{n}^{-2{\tau }_1-{\tau }_2}\}=\exp \{-0.5n^{1-2{\tau }_1-{\tau }_2}\}.$$ (A.7)

By the same arguments, we have

$$\mathrm{Pr}\{{\ell }_j^{\prime }({\mathbf{\beta }}_{s\prime }^{\ast })\le -m^{-1/2}(C_1{w}_1/2)n^{1-{\tau }_1}\}\le \exp \{-0.5n^{1-2{\tau }_1-{\tau }_2}\}.$$ (A.8)

Inequalities (A.7) and (A.8) imply that,

$$\mathrm{Pr}\{{\ell }_p({\mathbf{\beta }}_{s\prime })\ge {\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })\}\le 4m\exp \{-0.5n^{1-2{\tau }_1-{\tau }_2}\}.$$

Consequently, by Bonferroni’s inequality and under conditions (C3)–(C4), we have

$$\begin{gathered} \mathrm{Pr}\{\underset{s\in S_{-}^m}{\max }{\ell }_p({\mathbf{\beta }}_{s\prime })\ge {\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })\}\le \sum _{s\in S_{-}^m}\mathrm{Pr}\{{\ell }_p({\mathbf{\beta }}_{s\prime }^{\ast })\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le 4m{p}^m\exp \{-0.5n^{1-2{\tau }_1-{\tau }_1}\}=4\exp \{\log m+m\log p-0.5n^{1-2{\tau }_1-{\tau }_1}\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le 4\exp \{\log w_2+{\tau }_2\log n+w_2{n}^{{\tau }_2}\tilde{c}{n}^{\kappa }-{0.5}^{1-2{\tau }_1-{\tau }_2}\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=4w_2\exp \{{\tau }_2\log n+w_2\tilde{c}{n}^{{\tau }_2+\kappa }-0.5n^{1-2{\tau }_1-{\tau }_2}\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=a_1\exp \{{\tau }_2\log n+a_2{n}^{{\tau }_2+\kappa }-0.5n^{1-2{\tau }_1-{\tau }_2}\}=o(1), \end{gathered}$$ (A.9)

as n → ∞ for some generic positive constants a₁ = 4w₂ and $a_2=w_2\tilde{c}$. By Condition (C5), ℓ_p(β_s′ ) is concave in β_s′ and (A.9) holds for any β_s′ such that $\Vert {\mathbf{\beta }}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast }\Vert =w_1{d}_n{n}^{-{\tau }_1}$.

For any $s\in S_{-}^{mm}$, let ${\stackrel{⌣}{\mathbf{\beta }}}_{s\prime }$ be ${\widehat{\mathbf{\beta }}}_s$ augmented with zeros corresponding to the elements in s′/s∗, i.e., s′ = {s ∪ (s∗/s′)} ∪ (s′/s∗). By Condition (C3),

$${\Vert {\stackrel{⌣}{\mathbf{\beta }}}_{s\prime }-{\mathbf{\beta }}_{s\prime }^{\ast }\Vert }_2={\Vert {\stackrel{⌣}{\mathbf{\beta }}}_{s^{\ast }\cup \hspace{0.17em}(s\prime /s^{\ast })}-{\mathbf{\beta }}_{s^{\ast }\cup \hspace{0.17em}(s\prime /s^{\ast })}^{\ast }\Vert }_2={\Vert {\stackrel{⌣}{\mathbf{\beta }}}_{s^{\ast }\cup \hspace{0.17em}(s\prime /s^{\ast })}-{\mathbf{\beta }}_{s^{\ast }}^{\ast }\Vert }_2\ge {\Vert {\mathbf{\beta }}_{s^{\ast }\cup \hspace{0.17em}(s\prime /s^{\ast })}^{\ast }-{\mathbf{\beta }}_{s^{\ast }}^{\ast }\Vert }_2\ge {\Vert {\mathbf{\beta }}_{s\prime /s^{\ast }}^{\ast }\Vert }_2=w_1{d}_n{n}^{-{\tau }_1}.$$

Consequently,

$$\mathrm{Pr}\left\{\underset{s\in S_{-}^m}{\max }{\ell }_p({\widehat{\beta }}_s)\ge \underset{s\in S_{+}^m}{\min }{\ell }_p({\widehat{\beta }}_s)\right\}\le \mathrm{Pr}\left\{\underset{s\in S_{-}^m}{\max }{\ell }_p({\stackrel{⌣}{\beta }}_{s\prime })\ge {\ell }_p({\beta }_{s\prime }^{\ast })\right\}=o(1).$$

So, we have shown that

$$\underset{n\to \infty }{\lim }\mathrm{Pr}\left[\underset{s\in S_{-}^m}{\max }\ell \{{\widehat{\mathbf{\alpha }}}_s(U)\}\ge \underset{s\in S_{+}^m}{\min }\ell \{{\widehat{\mathbf{\alpha }}}_s(U)\}\right]=0.$$

Therefore, the theorem is proved.