Forward regression for Cox models with high-dimensional covariates

Abstract

Forward regression, a classical variable screening method, has been widely used for model building when the number of covariates is relatively low. However, forward regression is seldom used in high-dimensional settings because of the cumbersome computation and unknown theoretical properties. Some recent works have shown that forward regression, coupled with an extended Bayesian information criterion (EBIC)-based stopping rule, can consistently identify all relevant predictors in high-dimensional linear regression settings. However, the results are based on the sum of residual squares from linear models and it is unclear whether forward regression can be applied to more general regression settings, such as Cox proportional hazards models. We introduce a forward variable selection procedure for Cox models. It selects important variables sequentially according to the increment of partial likelihood, with an EBIC stopping rule. To our knowledge, this is the first study that investigates the partial likelihood-based forward regression in high-dimensional survival settings and establishes selection consistency results. We show that, if the dimension of the true model is finite, forward regression can discover all relevant predictors within a finite number of steps and their order of entry is determined by the size of the increment in partial likelihood. As partial likelihood is not a regular density-based likelihood, we develop some new theoretical results on partial likelihood and use these results to establish the desired sure screening properties. The practical utility of the proposed method is examined via extensive simulations and analysis of a subset of the Boston Lung Cancer Survival Cohort study, a hospital-based study for identifying biomarkers related to lung cancer patients’ survival.

1. Introduction

New biotechnologies have generated a vast amount of high-throughput data. In the Boston Lung Cancer Survival Cohort study, a hospital-based study for lung cancer patients, identifying high-throughput predictors such as molecular profiles that are associated with patients’ survival is a major research goal for understanding disease progression processes and designing more effective gene therapies. When the number p of covariates is less than the sample size n, the Cox proportional hazards model has been routinely used for modeling survival data in many practical settings. When p > n, penalized partial likelihood methods have been proposed by various authors [29, 41] and the oracle properties and statistical error bounds of estimation have been established [18]. However, when p >> n, these methods often fail because of serious challenges in “computational expediency, statistical accuracy, and algorithmic stability” [8]. Recently, Bradic et al. [3] established the oracle properties of the regularized partial likelihood estimates under a high-dimensional setting. However, the results require the estimates to be unique and global optimizers, which is, in general, difficult to verify, especially when the dimension of covariates is exceedingly high.

Forward regression has been widely used for model selection, but it has often been criticized for not achieving selection consistency as it fails to account for multiple comparisons in the model building process. Recently, some authors, e.g., [5, 13, 19, 23, 34, 39] have revamped forward regression in the context of linear regression or varying-coefficient linear models. The advantages can be summarized as follows. First, these authors have shown that, with some proper stopping criteria, forward regression can achieve screening consistency even in high-dimensional settings. Second, the variables are sequentially selected into the final model with the entry order determined by the size of the likelihood increment, which might reflect the relative importance of each selected variable. Third, the implementation is simple as no cross-validation for tuning parameters is needed. Finally, the method only needs assumptions on the original model and does not require restrictive faithfulness assumptions, in which the marginal models reflect the original model. However, to our knowledge, the aforementioned forward regression approaches are either based on the sum of residual squares from linear models [5, 34] or Lasso estimation [23]. It is unclear whether forward regression can be applied to more general regression settings, such as the Cox proportional hazards models.

In contrast, there has been active research in developing high-dimensional screening tools for survival data. These works include principled sure screening [37], feature aberration at survival times screening [12] and conditional screening [16], quantile adaptive sure independence screening [14], a censored rank independence screening procedure [26], and integrated powered density screening [15]; see [17] for an extensive review. However, the screening methods require a threshold to dictate how many variables to retain, for which unfortunately there are no clear rules. Zhao and Li [37] did tie the threshold with false discoveries, but it still needs to prefix the number of false positives that users are willing to tolerate. Recently, Li et al. [21] designed a model-free measure, namely the survival impact index, that sensibly captures the overall influence of a covariate on the survival outcome and can help guide selecting important variables. However, even this method, like the other screening methods, does not directly lead to a final model, for which extra modeling steps have to be implemented.

We introduce a new forward variable selection procedure for survival data based on partial likelihood. It selects important variables sequentially according to the increment of partial likelihood, with a stopping rule based on EBIC. We show that if the dimension of the true model is finite, within a finite number of steps forward regression can discover all relevant predictors, with the entry order determined by the size of the likelihood increment.

Our work is novel in the following aspects. It likely registers as the first attempt to thoroughly investigate the forward regression in high-dimensional survival settings, methodologically, theoretically and numerically. Technically this paper is also novel. First, our work represents technical advances and a broadened scope compared to the existing forward regression [5, 23, 34]. This may be the first work that investigates the partial likelihood-based forward regression in survival models with high-dimensional predictors, and establishes rigorous selection consistency results when the extended Bayesian information criterion (EBIC) [4] is used. It improves the partial likelihood-based variable selection developed by Volinsky and Raftery [33] and Xu et al. [35] for survival data in low dimensional settings. Second, as partial likelihood is not a regular density-based likelihood, it fails to satisfy the requirements for theories of forward regression. We revisit partial likelihood and develop some new inequalities, based on which we establish the desired sure screening properties. The derived theoretical framework and techniques will facilitate the extension of the procedure to other general likelihood-based settings, such as generalized linear regression models. Finally, we note that forward selection starts with an empty model or some important variables identified a priori, and then sequentially recruits variables given important variables identified in the previous steps. This may resemble the conditional screening approach [16], which incorporates prior knowledge into variable screening. However, our method is valid even in the absence of such information.

The rest of the paper is organized as follows. In Section 2, we introduce the proposed forward regression procedure. In Section 3, we rigorously establish forward regression’s screening consistency and asymptotic normality under some regularity conditions. We carry out simulation studies to assess the performance of the proposed method in Section 4, and apply the method in Section 5 to analyze a subset of the Boston Lung Cancer Survival Cohort study, our motivating study for identifying biomarkers related to lung cancer patients’ survival. We conclude the paper with a natural extension of the proposal in Section 6. Technical proofs and all of the lemmas are presented in the Appendix.

2. Partial likelihood-based forward regression

Suppose we have n independent subjects with p covariates, where p >> n. For subject i, denote by X_ij the jth covariate for subject i, write X_i = (X_i1, … , X_ip)^⊺, and let T_i and C_i be the underlying survival and censoring times. We, however, only observe Y_i = min(T_i, C_i), and the event indicator δ_i = 1(T_i ≤ C_i), where 1 is the indicator function. We assume random censoring such that C_i and T_i are independent given X_i. We assume that (Y₁, δ₁, X₁), … , (Y_n, δ_n, X_n) are mutually independently and identically distributed (iid). In particular, we assume that (Y₁, T₁, X_1j), … , (Y_n, T_n, X_nj), are mutually independent copies of (Y, T, X_j), the random variables that underlie the observed survival time, true survival time and covariates.

To link T_i to X_i, for each i ∈ {1, … , n}, we consider the Cox proportional hazards model, viz.

$$\lambda (t\mid {\mathbf{X}}_i)={\lambda }_0\left(t\right)\exp \left({\beta }_0^{⊺}{\mathbf{X}}_i\right),$$ (1)

where λ₀ is the unspecified baseline hazard function and β₀ = (β₀₁, … , β_0p)^⊺ is the vector of regression coefficients. Without loss of generality, we assume that E(X₁) = ⋯ = E(X_p) = 0. Denote the true model as $\mathcal{M}(j:B_{0j}\ne 0)$. The overarching goal of variable screening is to estimate $\mathcal{M}$ and we let its estimate be $\widehat{\mathcal{M}}$.

We introduce more notation. For an index set S ⊂ {1, … , p} and a p-dimensional vector A, we use A_S = {A_j : j ∈ S} to denote the subvector of A corresponding to S. For example, X_iS denotes the collection of covariates for the ith individual corresponding to S. We use |S| to denote the cardinality of S and let S^C denote the complement of S.

Now we elaborate on the idea of forward regression under model (1). Initialize S₀ = Ø. We can also start with a set of given variables according to some prior knowledge, which is in the same spirit as conditional screening [16] but is followed by a sequential selection process. Specifically, we sequentially select the sets of covariates in such a way that

$$S_0\subset \cdots \subset S_k,$$

where S_k ⊂ {1, … , p} is the index set of the selected covariates upon completion of the kth step, with k ≥ 0. Then at the (k + 1)th step, we need to choose a new candidate variable not in S_k and then decide whether we should stop at the kth step or we should include the new candidate in our selection and proceed to the next step. We emphasize that our selection criterion is based on the partial likelihood. The framework is much broader than that of the one based on the reduction in sum of squared residuals proposed in [5, 34] and can be extended to more general regression settings.

Now, given S_k, we consider estimation of the extended Cox model by adding a new variable index to S_k. Specifically, for every $j\in S_k^{\mathrm{C}}$, we denote by S_k,j = S_k ∪ {j}, and fit a Cox model on the variables indexed by S_k,j. We then compute the increment of log partial likelihood for each $j\in S_k^{\mathrm{C}}$, viz.

$${\ell }_{S_{k,j}}\left({\widehat{\beta }}_{S_{k,j}}\right)-{\ell }_{S_k}\left({\widehat{\beta }}_{S_k}\right).$$

Here, for a covariate set S, ℓ_S(β_S) is the log partial likelihood function given X_S, viz.

$${\ell }_S\left({\beta }_S\right)=\underset{i=1}{\overset{n}{\Sigma }}{\int }_0^{\tau }\left[{\beta }_S^{⊺}{\mathbf{X}}_{iS}-\ln \left\{\underset{\ell =1}{\overset{n}{\Sigma }}{\stackrel{‒}{Y}}_{\ell }\left(t\right)\exp \left({\beta }_S^{⊺}{\mathbf{X}}_{\ell S}\right)\right\}\right]d{N}_i\left(t\right),$$ (2)

and ${\widehat{\beta }}_S$ maximizes (2), where N_i(t) = 1(Y_i ≤ t, δ_i = 1) is the counting process, ${\stackrel{‒}{Y}}_i\left(t\right)=1(Y_i\ge t)$ is the at-risk process, and τ > 0 is the study duration such that Pr(Y ≥ τ) > 0. Then, the candidate index is chosen as

$$j^{\ast }=\underset{j\notin S_k}{\overset{\arg \max }{}}{\ell }_{S_{k,j}}\left({\widehat{\beta }}_{S_{k,j}}\right)-{\ell }_{S_k}\left({\widehat{\beta }}_{S_k}\right).$$

Upon completion of the (k + 1)th step, update S_k+1 = S_k ∪ {j*}.

We are now in a position to decide whether to stop at the kth step or to include variable j* in our selection and proceed to the next step. In the survival setting, the effective sample size is the number of uncensored events, in which case Volinsky and Raftery [33] showed that replacement of the sample size with the number of uncensored events in the penalty term of EBIC gives a better approximation to the Bayes factor. Therefore, we propose the following as the modified EBIC criterion for ultrahigh-dimensional survival data:

$$\mathrm{EBIC}\left(S_{k+1}\right)=-2{\ell }_{S_{k+1}}\left({\widehat{\beta }}_{S_{k+1}}\right)+\mid S_{k+1}(\ln d+2\eta \ln p)=-2{\ell }_{S_{k+1}}\left({\widehat{\beta }}_{S_{k+1}}\right)+(k+1)(\ln d+2\eta \ln p),$$

where d = δ₁ + ⋯ + δ_n is the number of events and η is some positive constant.

We stop if EBIC(S_k+1) > EBIC(S_k) and declare $\widehat{\mathcal{M}}=S_k$; otherwise, we proceed to the next step.

3. Theoretical properties

We first introduce more notation. Let →_p denote convergence in probability. Given a random sample Z₁, … , Z_n let ${\mathbb{E}}_n\left\{f\right(Z_i\left)\right\}={\Sigma }_{i=1}^{n}f\left(Z_i\right)∕n$ and ${\mathbb{G}}_n\left\{f\right(Z_i\left)\right\}={\Sigma }_{i=1}^n\left[f\right(Z_i-E\left\{f\right(Z_i\left)\right\}]∕\sqrt{n}$. For a column vector v, let v^⊗0 = 1, v^⊗1 = v, and v^⊗2 = vv^⊺. We denote the ℓ_q norm of v by ||v||_q for q ≥ 1, and, in particular, denote its ℓ₂ norm by ||v||. For any symmetric matrix A, let λ_min(A) and λ_max(A) represent the smallest and largest eigenvalues. Given an index set S and an index j ∈ S, we use S \ j to denote the set {r : r ∈ S, r ≠ j}.

Given an index set S ⊂ {1, … , p}, for k ∈ {0, 1, 2}, define

$$R_S^{\left(k\right)}({\beta }_S,t)={\mathbb{E}}_n\left\{{\stackrel{‒}{Y}}_i{\mathbf{X}}_{iS}^{\otimes k}\exp \right({\beta }_S^{⊺}{\mathbf{X}}_{iS}\left)\right\},r_S^{\left(k\right)}({\beta }_S,t)=\mathrm{E}\left\{R_S^{\left(k\right)}\right({\beta }_S,t\left)\right\},$$

$$V_S^{\left(k\right)}\left(t\right)={\mathbb{E}}_n\left\{{\stackrel{‒}{Y}}_i\right(t\left){\mathbf{X}}_{iS}^{\otimes k}{\lambda }_0\right(t\left)\exp \right({\beta }_0^{⊺}{\mathbf{X}}_i\left)\right\},v_S^{\left(k\right)}\left(t\right)=E\left\{V_S^{\left(k\right)}\right(t\left)\right\}.$$

In addition, we use ${\beta }_S^{\ast }$ to denote the least false value, which is the unique root of

$${\int }_0^{\tau }\left\{v_S^{\left(1\right)}\left(t\right)-\frac{r_S^{\left(1\right)}({\beta }_S,t)}{r_S^{\left(0\right)}({\beta }_S,t)}{v}_S^{\left(0\right)}\left(t\right)\right\}dt=0.$$

We use $F_T(t;\mathbf{X}\mathcal{M})$, $f_T(t;{\mathbf{X}}_{\mathcal{M}})$ and $S_T(t;{\mathbf{X}}_{\mathcal{M}})$ to denote the conditional cumulative distribution function (cdf), probability density function (pdf), and survival function of T given the true model ${\mathbf{X}}_{\mathcal{M}}$, respectively. Likewise, the conditional cdf, pdf, and survival function of C given ${\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}$ are denoted by $F_C(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})$, $fC(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})$, and $S_C(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})$, respectively, where ${\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}$ is the collection of covariates that are related to the censoring time C. Let ${\mathcal{M}}_O=\mathcal{M}\cup {\mathcal{M}}_{\mathcal{A}}$.

3.1. Regularity conditions

We posit the regularity conditions, followed by explanations. The assumptions are, in general, mild, well justified, and follow the same lines as suggested by the existing literature.

(A) The study has a finite duration τ such that ω = Pr(Y ≥ τ) > 0.

(B) The X_j are time-independent and bounded by a constant K > 1 with E(X_j) = 0 and $E\left(X_j^2\right)=1$ for all j ∈ {1, … , p}.

(C) There exist two positive constants 0 < κ_min < κ_max < ∞, such that

$${\kappa }_{\min }<{\lambda }_{\min }\left\{\mathrm{E}\right({\mathbf{X}}_S^{{\otimes }^2}\left)\right\}\le {\lambda }_{\max }\left\{\mathrm{E}\right({\mathbf{X}}_S^{\otimes 2}\left)\right\}<{\kappa }_{\max },$$

uniformly in S ⊂ {1, … , p} satisfying |S| ≤ ρ for some $\rho >\mid \mathcal{M}\mid$.

(D) ${\sup }_{\mid S\mid \le \rho }\mid \mid {\beta }_S^{\ast }\mid {\mid }_1\le L$ for some constant L.

(E) For some constant α ∈ (0, 1/2),

$$\underset{j\in \mathcal{M}}{\overset{\inf }{}}\mid {\int }_0^{\tau }\mathrm{E}\left\{X_j{f}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\left)\right\}dt\mid \ge K{n}^{-\alpha }.$$

(F) There exists ζ > 0 such that, for all t > 0,

$$\begin{gathered} {\kappa }_{\min }\le \underset{{\parallel {\beta }_S-{\beta }_S^{\ast }\parallel }_{\infty \le \zeta ,\mid S\mid \le \rho }}{\inf }{\lambda }_{\min }\left[{\int }_0^{\tau }\left[\frac{r_S^{\left(2\right)}({\beta }_S,t)}{r_S^{\left(0\right)}({\beta }_S,t)}-\frac{\left\{r_S^{\left(1\right)}\right({\beta }_S,t){\}}^{\otimes 2}}{\left\{r_S^{\left(0\right)}\right({\beta }_S,t){\}}^2}\right]v_S^{\left(0\right)}\left(t\right)dt\right] \\ \le \underset{{\parallel {\beta }_S-{\beta }_S^{\ast }\parallel }_{\infty \le \zeta ,\mid S\mid \le \rho }}{\sup }{\lambda }_{\max }\left\{{\int }_0^{\tau }\frac{r_S^{\left(2\right)}({\beta }_S,t)}{r_S^{\left(0\right)}({\beta }_S,t)}{v}_S^{\left(0\right)}\left(t\right)dt\right\}\le {\kappa }_{\max }. \end{gathered}$$

(G) $\mathrm{E}\left\{X_j{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/j}}\}$ and $\mathrm{E}\left\{X_j{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/j}}\}$ have the same sign across t, for any $j\in \mathcal{M}$.

Condition (A) is standard in survival models with censored data; see, e.g., [20]. Conditions (B) and (C) are commonly assumed in the literature for variable selection and screening; see, e.g., [5, 23, 34, 38]. The boundedness of X is adopted to simplify our theoretical development and can be relaxed to the Cramér condition as in [3]. Condition (D) replaces the Lipschitz assumption in [30] and has a similar flavor to the conditions in [37], and [20]. Condition (E) is introduced in [37], which is an adapted version of the conditions in [7] to survival data. Condition (F) is analogous to Condition 2 considered in [3] for regularized Cox models. The condition essentially requires that the concavity of the log partial likelihood is well bounded in a neighborhood of ${\beta }_S^{\ast }$. We invoke Condition (G) in order to analyze the least false value ${\beta }_S^{\ast }$. The condition often holds in practice; for example, Lemma A in the Appendix shows that it is satisfied if $\mathcal{M}\cap {\mathcal{M}}_{\mathcal{A}}=\varnothing$.

Since the log partial likelihood function in (2) is the sum of non-iid random variables, we consider its asymptotically equivalent version, which can be expressed as the sum of iid terms, viz.

$${\tilde{\ell }}_S\left({\beta }_S\right)=\underset{i=1}{\overset{n}{\Sigma }}{\int }_0^{\tau }\{{\beta }_S^{⊺}{\mathbf{X}}_{iS}-\ln r_S^{\left(0\right)}({\beta }_S,t\left)\right\}d{N}_i\left(t\right).$$ (3)

According to Kong and Nan [20], the log partial likelihood function (2) can then be viewed as a “working” model of (3), and the corresponding loss function becomes

$$\gamma S({\beta }_S;{\mathbf{X}}_i,Y_i,{\delta }_i)=-{\int }_0^{\tau }\{{\beta }_S^{⊺}{\mathbf{X}}_{iS}-\ln r_S^{\left(0\right)}({\beta }_S,t\left)\right\}d{N}_i\left(t\right)$$

with the expected loss Γ_S(β_S) = E{γ_S (β_S; X_i, Y_i, δ_i).

To validate the replacement of the log partial likelihood (3), a commonly assumed condition in the literature is that there exists a neighborhood $\mathcal{B}$ of ${\beta }_0$ such that for k ∈ {0, 1, 2},

$$\underset{t\in [0,\tau ],\beta \in \mathcal{B}}{\sup }\parallel R_{\mathcal{M}}^{\left(k\right)}(\beta ,t)-r_{\mathcal{M}}^{\left(k\right)}\left(\beta ,t\right)\parallel {\to }_{p}0.$$

See, e.g., [3, 37]. Under Conditions (B) and (D), Lemma C shows that the above condition holds uniformly for all S ⊂ {1, … , p} satisfying |S| ≤ ρ.

We note that the proportional hazards assumption is made only on the true (and sparse) model. At each step of the screening procedure, we treat the misspecified Cox proportional hazards model as a working model following [10, 22]. Our theoretical derivations depend on the least false value, which helps us characterize the asymptotic behavior of our estimator at each step even without the proportional hazards assumption. Specifically, similar to [10, 22]. the proposed estimator will converge to the least false value at each step under the working model, and when the second derivative of the log partial likelihood is bounded in a neighborhood of the least false value, adding an active variable will increase the partial-likelihood, even if a mis-specified model is under consideration.

3.2. Main results

Theorem 1. Under Conditions (A)–(G), if ρ⁴ ln(p/n) → 0, then with probability at least 1 – 8 exp (−3ρ ln p), there exists α ∈ (0, 1/2) such that

$$\underset{S:\mid S\mid <\rho ,\mathcal{M}\not\subset S}{\min }\underset{j\in S^{\mathrm{C}}}{\max }\left\{{\ell }_{S\cup \left\{j\right\}}\right({\widehat{\beta }}_{S\cup \left\{j\right\}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}\ge c_1{n}^{1-2\alpha }-c_2\sqrt{n{\rho }^2\ln \left(p\right)}.$$

Theorem 1 shows that if $\mathcal{M}\not\subset S_k$ and |S_k| < ρ, then the increment of the log-likelihood at the (k + 1)th step is at least $c_1{n}^{1-2\alpha }-c_2\sqrt{n{\rho }^2\ln \left(p\right)}$. Since the maximum increment is bounded by |ℓ_S0(0)|, we naturally obtain an upper bound on the number of steps for the forward selection, which is stated in the following corollary.

Corollary 1. Suppose the same conditions as in Theorem 1 hold. If $\sqrt{{\rho }^2\ln (p∕n)}=o\left(n^{-2\alpha }\right)$ and

$$M=2\mathrm{E}\left[{\int }_0^{\tau }\ln \left\{n{r}_{S_0}^{\left(0\right)}\right(0,t\left)\right\}d{N}_i\left(t\right)\right]n^{2\alpha }<\rho$$

for some α ∈ (0, 1/2), then $\mathcal{M}\subset S_k$, for some k ≤ M, with probability of at least 1 – 11 exp(−3ρ ln p).

Corollary 1 establishes the screening consistency of the forward selection procedure. However, the upper bound M is not sharp, as it is calculated based on the lower bound of the increment of the log-likelihood. The following corollary establishes an upper bound of the number of steps by evaluating how likely a signal variable will be selected at each step.

Corollary 2. Under the same conditions as in Corollary 1, if the ${\mathbf{X}}_{{\mathcal{M}}^{\mathrm{C}}}$ are independent of ${\mathbf{X}}_{\mathcal{M}}$, then $\mathcal{M}\subset S_{c_3\mid \mathcal{M}\mid }$, for some c₃ ∈ (1, ∞), with probability at least 1 – 8 exp(−2ρ ln p).

The condition that the ${\mathbf{X}}_{{\mathcal{M}}^{\mathrm{C}}}$ are independent of ${\mathbf{X}}_{\mathcal{M}}$ stems from the assumption imposed in [37], and is similar to the partial orthogonality assumption introduced in [9]. It ensures that selecting a noise variable would bring less of an increment of the log-log-likelihoodlikelihood compared to choosing a signal variable. Thus, it is much more likely for our procedure to select a signal variable at each step.

The following result follows from Corollary 2. We expect the proposed forward procedure to stop early with $\mathcal{M}\subset S_k$ for some k.

Corollary 3. Under the same conditions as in Corollary 2, if $\mathcal{M}\not\subset S_{k-1}$ and $\mathcal{M}\subset S_k$, then with probability going to 1,

(i) (screening consistency) the procedure stops at the kth step and $\mathcal{M}\subset \widehat{\mathcal{M}}=S_k$;

(ii) (false discovery rate control) $\mid \widehat{\mathcal{M}}\cap {\mathcal{M}}^{\mathrm{C}}\mid ∕\mid \widehat{\mathcal{M}}\mid \le c_3-1$.

By Corollary 3, our proposed forward selection procedure will stop at a final step, denoted by $\widehat{k}$, which is at most $c_3\mid \mathcal{M}\mid$. The final model $\widehat{\mathcal{M}}$ not only achieves screening consistency, but also has well-controlled false discoveries.

4. Numerical studies

Simulations were conducted to compare the performance of the proposed forward regression (FR) with two partial likelihood based screening methods, the principled sure independence screening (PSIS) by Zhao and Li [37], and the conditional screening (CS) by Hong et al. [16]. The size of the models selected by the PSIS and CS was initially set to be ⌊n/ ln n⌋ as suggested in [7]. To further reduce false positives, we applied Lasso [28], SCAD [6], and MCP [36] penalties to further reduce the sizes of models selected by each method. In the tables, we used screening method + penalty to denote the corresponding procedure. Although FR could start from a null model, we tried different initial sets for FR, including active or inactive variables. In particular, we chose X₁ and X₁₀ to represent the active and inactive initial sets, respectively. When computing the model size for both FR and CS, we included the given initial set.

In this paper, we considered η as a fixed constant, which is analogous to the constant “a” parameter in the SCAD penalty function [6]. This distinguishes this from the other screening approaches that typically require data-driven thresholding tuning parameters and may incur more of a computational burden for finding them. To further justify the use of a fixed η, we considered various values of η between 0 and 1, the theoretical range of η in EBIC [4]. The BIC is a special case of EBIC when η = 0. We explored using BIC as the stopping criterion, but it incurred too many false positives compared to EBIC. This may cause overfitting of the Cox proportional hazards model with unreliably estimated regression coefficients and spuriously detected associations [32]. Thus, we elected not to use BIC.

We next considered three different values of η, 0.5, 1 – ln d/(3 ln p), and 1; see Tables 1–3. Essentially, a larger η gives more penalty to a complicated model, which may incur more false negatives, while a smaller η less penalizes the complexity of models and may lead to more false positives. Based on Tables 1–3, it seems that under all of the scenarios considered, the choice of η = 1 – ln d/(3 ln p) strikes a good balance between false negatives and false positives.

Table 1:

Comparisons of methods under mild censoring.

Example	Method	(n, p) = (200,1000)			(n, p) = (400,1000)
		PIT	TP	FP	PIT	TP	FP
1 (P0 = 4)	FR	3.65 (0.52)	3.61 (0.49)	0.05 (0.21)	3.99 (0.18)	3.98 (0.14)	0.01 (0.12)
	FR+Lasso	3.65 (0.52)	3.02 (0.15)	0.63 (0.48)	3.99 (0.18)	3.01 (0.12)	0.98 (0.14)
	FR+MCP	3.65 (0.52)	3.02 (0.15)	0.63 (0.48)	3.99 (0.18)	3.01 (0.12)	0.98 (0.14)
	FR+SCAD	3.65 (0.52)	3.02 (0.15)	0.63 (0.48)	3.99 (0.18)	3.01 (0.12)	0.98 (0.14)
	FRx1	3.65 (0.52)	3.61 (0.49)	0.05 (0.21)	3.99 (0.18)	3.98 (0.14)	0.01 (0.12)
	FRx10	4.64 (0.52)	3.60 (0.49)	1.04 (0.20)	4.99 (0.19)	3.98 (0.15)	1.01 (0.12)
	FR(η1)	5.04 (1.41)	3.89 (0.32)	1.16 (1.38)	4.53 (0.85)	4.00 (0.00)	0.53 (0.85)
	FR(η2)	3.95 (0.62)	3.77 (0.42)	0.18 (0.45)	4.08 (0.33)	3.99 (0.09)	0.09 (0.31)
	PSIS	38.00 (0.00)	3.03 (0.55)	34.97 (0.55)	67.00 (0.00)	3.39 (0.49)	63.61 (0.49)
	PSIS+Lasso	4.44 (2.71)	2.43 (0.93)	2.01 (2.12)	6.82 (2.70)	3.13 (0.48)	3.69 (2.64)
	PSIS+MCP	4.05 (2.14)	2.47 (0.85)	1.58 (1.67)	4.81 (1.60)	3.08 (0.47)	1.74 (1.55)
	PSIS+SCAD	4.22 (2.55)	2.36 (0.89)	1.86 (1.98)	5.33 (1.96)	3.06 (0.53)	2.27 (1.85)
	CS	38.00 (0.00)	3.08 (0.48)	34.92 (0.48)	67.00 (0.00)	3.23 (0.42)	63.77 (0.42)
	CS+Lasso	4.63 (2.49)	2.51 (0.77)	2.12 (2.09)	6.50 (2.45)	2.95 (0.43)	3.54 (2.35)
	CS+MCP	4.00 (1.86)	2.44 (0.76)	1.55 (1.47)	4.78 (1.40)	2.92 (0.37)	1.86 (1.33)
	CS+SCAD	4.30 (2.23)	2.37 (0.75)	1.93 (1.81)	5.39 (1.82)	2.90 (0.44)	2.48 (1.67)
2 (p0 = 6)	FR	0.91 (0.30)	5.89 (0.40)	1.34 (1.40)	1.00 (0.00)	6.00 (0.00)	1.01 (1.20)
	FR	6.16 (0.47)	6.00 (0.00)	0.16 (0.47)	6.02 (0.15)	6.00 (0.00)	0.02 (0.15)
	FR+Lasso	6.16 (0.47)	6.00 (0.00)	0.16 (0.47)	6.02 (0.15)	6.00 (0.00)	0.02 (0.15)
	FR+MCP	6.16 (0.47)	6.00 (0.00)	0.16 (0.47)	6.02 (0.15)	6.00 (0.00)	0.02 (0.15)
	FR+SCAD	6.08 (0.35)	5.91 (0.32)	0.16 (0.47)	6.02 (0.15)	6.00 (0.00)	0.02 (0.15)
	FRx1	6.08 (0.33)	6.00 (0.00)	0.08 (0.33)	6.02 (0.15)	6.00 (0.00)	0.02 (0.15)
	FRx10	7.13 (0.42)	6.00 (0.00)	1.13 (0.42)	7.01 (0.13)	6.00 (0.00)	1.01 (0.13)
	FR(η1)	7.51 (1.78)	6.00 (0.00)	1.51 (1.78)	6.78 (1.16)	6.00 (0.00)	0.78 (1.16)
	FR(η2)	6.34 (0.72)	6.00 (0.00)	0.34 (0.72)	6.11 (0.40)	6.00 (0.00)	0.11 (0.40)
	PSIS	38.00 (0.00)	3.42 (0.93)	34.58 (0.93)	67.00 (0.00)	4.83 (0.53)	62.17 (0.53)
	PSIS+Lasso	8.54 (4.97)	3.71 (0.97)	4.84 (4.66)	8.74 (5.33)	4.82 (0.54)	3.92 (5.21)
	PSIS+MCP	5.75 (2.39)	3.34 (1.01)	2.42 (1.97)	9.21 (3.05)	4.80 (0.61)	4.41 (2.77)
	PSIS+SCAD	6.73 (3.29)	3.51 (0.95)	3.21 (3.08)	8.80 (4.80)	4.77 (0.63)	4.03 (4.56)
	CS	38.00 (0.00)	4.45 (0.83)	33.55 (0.83)	67.00 (0.00)	5.66 (0.53)	61.34 (0.53)
	CS+Lasso	8.68 (4.49)	4.55 (0.93)	4.13 (4.07)	11.60 (4.50)	5.67 (0.53)	5.93 (4.22)
	CS+MCP	6.41 (2.38)	4.27 (1.09)	2.14 (1.97)	7.44 (2.20)	5.64 (0.63)	1.80 (2.46)
	CS+SCAD	6.68 (2.92)	4.34 (0.98)	2.34 (2.69)	6.83 (2.54)	5.63 (0.63)	1.20 (2.71)
3 (P0 = 6)	FR	5.24 (0.95)	5.02 (1.03)	0.22 (0.49)	5.75 (0.45)	5.74 (0.44)	0.01 (0.12)
	FR+Lasso	5.24 (0.95)	5.19 (0.87)	0.05 (0.25)	5.75 (0.45)	5.74 (0.44)	0.01 (0.09)
	FR+MCP	5.24 (0.95)	5.19 (0.87)	0.05 (0.25)	5.75 (0.45)	5.74 (0.44)	0.01 (0.09)
	FR+SCAD	5.21 (0.94)	5.16 (0.87)	0.05 (0.25)	5.75 (0.45)	5.74 (0.44)	0.01 (0.09)
	FRx1	5.25 (0.85)	5.10 (0.84)	0.16 (0.40)	5.75 (0.45)	5.74 (0.44)	0.01 (0.12)
	FRx10	6.05 (1.28)	4.87 (1.28)	1.17 (0.41)	6.75 (0.45)	5.73 (0.44)	1.01 (0.12)
	FR(η1)	6.84 (1.87)	5.38 (0.93)	1.45 (1.70)	6.43 (0.87)	5.93 (0.26)	0.50 (0.84)
	FR(η2)	5.60 (1.09)	5.18 (1.00)	0.41 (0.73)	5.93 (0.48)	5.85 (0.36)	0.07 (0.33)
	PSIS	38.00 (0.00)	2.57 (0.72)	35.43 (0.72)	67.00 (0.00)	3.35 (0.58)	63.65 (0.58)
	PSIS+Lasso	3.43 (2.69)	2.06 (1.05)	1.37 (1.96)	6.18 (2.77)	3.22 (0.56)	2.96 (2.59)
	PSIS+MCP	3.04 (1.64)	2.21 (0.90)	0.83 (1.16)	3.51 (0.95)	3.11 (0.49)	0.40 (0.79)
	PSIS+SCAD	3.64 (2.24)	2.21 (0.99)	1.43 (1.61)	3.91 (1.18)	3.13 (0.49)	0.78 (1.04)
	CS	38.00 (0.00)	3.14 (0.68)	34.86 (0.68)	67.00 (0.00)	4.17 (0.62)	62.83 (0.62)
	CS+Lasso	4.74 (3.02)	2.67 (1.18)	2.07 (2.21)	7.97 (3.03)	4.05 (0.65)	3.93 (2.77)
	CS+MCP	3.76 (1.67)	2.84 (0.80)	0.93 (1.39)	4.31 (0.97)	3.94 (0.58)	0.38 (0.79)
	CS+SCAD	4.62 (2.16)	2.86 (0.95)	1.76 (1.69)	4.66 (1.08)	3.95 (0.58)	0.71 (0.98)

NOTE: FR, forward regression; PSIS, the principled sure independence screening; CS, the conditional screening with the given conditioning variable; PIT, estimated probability of including all true predictors in the selected predictors; TP, average number of true positives; FP, average number of false positives; p₀ denotes the number of true signals; numbers in the parentheses are standard deviations. We used η₁ = .5 and η₂ = 1 – ln d/(3 ln p). When it is not noted, η = 1 was used. FR_S0 denotes that FR was performed with the initial set of S₀. We considered two initial sets x₁ and x₁₀.

Table 3:

Comparisons of methods under covariate-dependent censoring

Example	Method	(n, p) = (200, 1000)			(n, p) = (400, 1000)
		PIT	TP	FP	PIT	TP	FP
1* (P0 = 4)	FR	3.58 (0.51)	3.55 (0.50)	0.03 (0.17)	3.99 (0.15)	3.98 (0.13)	0.00 (0.06)
	FR+Lasso	3.58 (0.51)	3.01 (0.09)	0.57 (0.50)	3.99 (0.15)	3.00 (0.06)	0.98 (0.13)
	FR+MCP	3.58 (0.51)	3.01 (0.09)	0.57 (0.50)	3.99 (0.15)	3.00 (0.06)	0.98 (0.13)
	FR+SCAD	3.58 (0.51)	3.01 (0.09)	0.57 (0.50)	3.99 (0.15)	3.00 (0.06)	0.98 (0.13)
	FRx1	3.58 (0.51)	3.55 (0.50)	0.03 (0.17)	3.99 (0.15)	3.98 (0.13)	0.00 (0.06)
	FRx10	4.58 (0.51)	3.56 (0.50)	1.03 (0.17)	4.98 (0.14)	3.98 (0.13)	1.00 (0.04)
	FR(η1)	5.05 (1.56)	3.84 (0.37)	1.21 (1.53)	4.56 (0.85)	4.00 (0.00)	0.56 (0.85)
	FR(η2)	3.89 (0.64)	3.70 (0.46)	0.19 (0.48)	4.08 (0.32)	4.00 (0.06)	0.09 (0.31)
	PSIS	38.00 (0.00)	2.96 (0.51)	35.04 (0.51)	67.00 (0.00)	3.41 (0.50)	63.59 (0.50)
	PSIS+Lasso	4.39 (2.82)	2.29 (0.96)	2.11 (2.24)	6.58 (2.63)	3.06 (0.59)	3.53 (2.53)
	PSIS+MCP	4.00 (2.15)	2.37 (0.90)	1.63 (1.70)	4.81 (1.60)	3.08 (0.54)	1.73 (1.55)
	PSIS+SCAD	4.30 (2.54)	2.27 (0.91)	2.03 (2.01)	5.34 (1.91)	3.04 (0.59)	2.30 (1.83)
	CS	38.00 (0.00)	3.00 (0.49)	35.00 (0.49)	67.00 (0.00)	3.25 (0.43)	63.75 (0.43)
	CS+Lasso	4.50 (2.67)	2.36 (0.77)	2.14 (2.28)	6.50 (2.44)	2.92 (0.41)	3.58 (2.33)
	CS+MCP	4.01 (2.00)	2.38 (0.75)	1.63 (1.65)	4.72 (1.45)	2.91 (0.35)	1.81 (1.38)
	CS+SCAD	4.27 (2.39)	2.29 (0.76)	1.98 (1.96)	5.45 (1.87)	2.89 (0.39)	2.57 (1.77)
2* (P0 = 6)	FR	6.13 (0.39)	6.00 (0.04)	0.13 (0.39)	6.01 (0.09)	6.00 (0.00)	0.01 (0.09)
	FR+Lasso	6.12 (0.38)	5.99 (0.09)	0.13 (0.39)	6.01 (0.09)	6.00 (0.00)	0.01 (0.09)
	FR+MCP	6.12 (0.38)	5.99 (0.09)	0.13 (0.39)	6.01 (0.09)	6.00 (0.00)	0.01 (0.09)
	FR+SCAD	6.04 (0.25)	5.91 (0.30)	0.13 (0.39)	6.01 (0.09)	6.00 (0.00)	0.01 (0.09)
	FRx1	6.06 (0.26)	6.00 (0.04)	0.06 (0.25)	6.01 (0.09)	6.00 (0.00)	0.01 (0.09)
	FRx10	7.09 (0.32)	6.00 (0.04)	1.09 (0.31)	7.01 (0.11)	6.00 (0.00)	1.01 (0.11)
	FR(η1)	7.55 (1.72)	6.00 (0.00)	1.55 (1.72)	6.68 (1.05)	6.00 (0.00)	0.68 (1.05)
	FR(η2)	6.32 (0.64)	6.00 (0.00)	0.32 (0.64)	6.11 (0.39)	6.00 (0.00)	0.11 (0.39)
	PSIS	38.00 (0.00)	3.44 (0.92)	34.56 (0.92)	67.00 (0.00)	4.80 (0.59)	62.20 (0.59)
	PSIS+Lasso	8.55 (5.16)	3.63 (1.00)	4.92 (4.81)	9.14 (5.81)	4.79 (0.58)	4.34 (5.65)
	PSIS+MCP	5.50 (2.47)	3.25 (1.04)	2.24 (2.05)	9.01 (2.91)	4.76 (0.67)	4.25 (2.63)
	PSIS+SCAD	6.27 (3.19)	3.42 (0.99)	2.86 (2.98)	8.67 (4.77)	4.73 (0.68)	3.95 (4.50)
	CS	38.00 (0.00)	4.40 (0.89)	33.60 (0.89)	67.00 (0.00)	5.66 (0.53)	61.34 (0.53)
	CS+Lasso	8.74 (4.59)	4.47 (0.97)	4.27 (4.22)	11.42 (4.57)	5.66 (0.55)	5.76 (4.32)
	CS+MCP	6.19 (2.39)	4.12 (1.15)	2.06 (1.95)	7.30 (2.11)	5.62 (0.62)	1.67 (2.37)
	CS+SCAD	6.54 (2.98)	4.22 (1.08)	2.33 (2.76)	6.91 (2.81)	5.61 (0.66)	1.30 (2.98)
3* (P0 = 6)	FR	5.11 (1.02)	4.87 (1.13)	0.24 (0.51)	5.72 (0.47)	5.70 (0.46)	0.02 (0.13)
	FR+Lasso	5.11 (1.02)	5.06 (0.96)	0.04 (0.22)	5.72 (0.47)	5.71 (0.45)	0.01 (0.10)
	FR+MCP	5.11 (1.02)	5.06 (0.96)	0.04 (0.22)	5.72 (0.47)	5.71 (0.45)	0.01 (0.10)
	FR+SCAD	5.08 (1.01)	5.03 (0.94)	0.04 (0.22)	5.72 (0.47)	5.71 (0.45)	0.01 (0.10)
	FRx1	5.11 (0.98)	4.92 (1.00)	0.19 (0.46)	5.72 (0.47)	5.70 (0.46)	0.02 (0.13)
	FRx10	5.74 (1.49)	4.59 (1.50)	1.15 (0.38)	6.72 (0.49)	5.70 (0.46)	1.03 (0.17)
	FR(η1)	7.11 (1.96)	5.38 (0.87)	1.73 (1.84)	6.60 (1.00)	5.91 (0.28)	0.69 (0.96)
	FR(η2)	5.51 (1.09)	5.07 (1.02)	0.44 (0.73)	5.97 (0.54)	5.83 (0.37)	0.14 (0.39)
	PSIS	38.00 (0.00)	2.55 (0.72)	35.45 (0.72)	67.00 (0.00)	3.34 (0.63)	63.66 (0.63)
	PSIS+Lasso	3.08 (2.58)	1.85 (1.10)	1.23 (1.79)	5.93 (2.76)	3.19 (0.70)	2.74 (2.46)
	PSIS+MCP	3.10 (1.84)	2.10 (0.96)	1.00 (1.37)	3.48 (1.06)	3.05 (0.57)	0.43 (0.87)
	PSIS+SCAD	3.38 (2.40)	2.02 (1.11)	1.36 (1.67)	4.12 (1.48)	3.11 (0.58)	1.01 (1.37)
	CS	38.00 (0.00)	3.12 (0.70)	34.88 (0.70)	67.00 (0.00)	4.10 (0.68)	62.90 (0.68)
	CS+Lasso	4.47 (3.28)	2.53 (1.21)	1.94 (2.42)	7.88 (3.17)	3.96 (0.69)	3.92 (2.89)
	CS+MCP	3.97 (1.99)	2.77 (0.89)	1.20 (1.62)	4.35 (1.07)	3.86 (0.62)	0.50 (0.87)
	CS+SCAD	4.60 (2.57)	2.72 (1.05)	1.87 (1.97)	4.83 (1.28)	3.87 (0.63)	0.96 (1.13)

We considered p = 1000 and two sample sizes, viz. n ∈ {200, 400}. The survival time was generated from a Cox model λ(t|X) = λ₀(t)exp(β^⊺X) with a Weibull baseline hazard. Specifically, λ₀(t) = αγt^γ−1, with α = 1 and γ = 1.5. We considered various models for X and different parameter configurations for β in the following four examples. The censoring time was independently generated from a uniform distribution, $\mathcal{U}[0,c)$. We varied c for each example in order to yield mild (around 25%) and heavy censoring proportions (around 50%). For each configuration, a total of 500 simulated datasets were generated.

Example 1. We chose β = (1, 0.5, −1, 0, 1, 0_p–5) and generated X from a multivariate normal distribution where the mean was 0, the variance 1, and cor(X_j, X_j’) = 0.5^|j–j’.

Example 2. We set $\beta =(1_5^{⊺},-2.5,0_{p-6}^{⊺})$ and generated X from a multivariate normal distribution with mean 0, variance 1, and cor(X_j, X_j’) = 0.5. In this case, since cov(T, X₆) = 0, X₆ has a lower marginal utility than all of the noise variables with cov(T, X_j) = 1.25 for $j\in {\mathcal{M}}^{\mathrm{C}}$.

Example 3. We let $\beta =(1,-1,1,-1,1,-\nu +{\nu }^2-{\nu }^3+{\nu }^4-{\nu }^5,0_{p-6}^{⊺})$ with ν = 0.5. We generated X from a multivariate normal distribution with mean 0, variance 1, and cor(X_j, X_j’) = 0.5^|j–j’|. In this case, since cov(T, X₆) = 0, X₆ is an active but hidden variable. Furthermore, the signals of the active variables are weak due to signal cancellation.

Tables 1–2 report the average of the estimated probabilities of including the true models (PIT), the average numbers of true positive (TP) and false positive (FP), and their standard deviations in parenthesis, under mild and heavy censoring. We use p₀ to denote the number of true signals. We have observed the competing performance of the proposed method as detailed below.

Table 2:

Comparisons of methods under heavy censoring.

Example	Method	(n, p) = (200, 1000)			(n, p) = (400, 1000)
		PIT	TP	FP	PIT	TP	FP
1 (P0 = 4)	FR	3.33 (0.54)	3.29 (0.49)	0.04 (0.22)	3.86 (0.39)	3.85 (0.36)	0.01 (0.13)
	FR+Lasso	3.33 (0.54)	3.00 (0.16)	0.33 (0.48)	3.86 (0.39)	3.01 (0.10)	0.85 (0.36)
	FR+MCP	3.33 (0.54)	3.00 (0.16)	0.33 (0.48)	3.86 (0.39)	3.01 (0.10)	0.85 (0.36)
	FR+SCAD	3.33 (0.54)	3.00 (0.16)	0.33 (0.48)	3.86 (0.39)	3.01 (0.10)	0.85 (0.36)
	FRx1	3.33 (0.54)	3.29 (0.49)	0.04 (0.22)	3.86 (0.39)	3.85 (0.36)	0.01 (0.13)
	FRx10	4.35 (0.55)	3.29 (0.49)	1.05 (0.26)	4.85 (0.39)	3.84 (0.37)	1.01 (0.13)
	FR(η1)	5.24 (1.98)	3.62 (0.49)	1.62 (1.88)	4.71 (1.04)	3.98 (0.15)	0.74 (1.02)
	FR(η2)	3.64 (0.72)	3.45 (0.51)	0.19 (0.49)	4.02 (0.42)	3.93 (0.26)	0.09 (0.33)
	PSIS	38.00 (0.00)	2.93 (0.59)	35.07 (0.59)	67.00 (0.00)	3.37 (0.49)	63.63 (0.49)
	PSIS+Lasso	3.58 (2.57)	2.02 (0.92)	1.55 (1.98)	5.92 (2.46)	2.91 (0.58)	3.01 (2.26)
	PSIS+MCP	3.56 (2.23)	2.13 (0.87)	1.43 (1.74)	4.70 (1.60)	2.92 (0.53)	1.79 (1.49)
	PSIS+SCAD	3.66 (2.48)	2.01 (0.86)	1.65 (1.96)	5.22 (2.17)	2.85 (0.64)	2.37 (1.92)
	CS	38.00 (0.00)	2.90 (0.55)	35.10 (0.55)	67.00 (0.00)	3.27 (0.45)	63.73 (0.45)
	CS+Lasso	3.60 (2.29)	2.09 (0.83)	1.52 (1.81)	5.69 (2.51)	2.80 (0.56)	2.89 (2.30)
	CS+MCP	3.36 (1.91)	2.07 (0.79)	1.28 (1.50)	4.70 (1.66)	2.83 (0.47)	1.87 (1.51)
	CS+SCAD	3.67 (2.19)	2.04 (0.78)	1.63 (1.74)	5.21 (2.29)	2.74 (0.56)	2.48 (2.03)
2 (P0 = 6)	FR	6.22 (0.65)	5.85 (0.61)	0.36 (0.74)	6.03 (0.18)	6.00 (0.00)	0.03 (0.18)
	FR+Lasso	6.21 (0.65)	5.95 (0.23)	0.26 (0.60)	6.03 (0.18)	6.00 (0.00)	0.03 (0.18)
	FR+MCP	6.21 (0.65)	5.95 (0.23)	0.26 (0.60)	6.03 (0.18)	6.00 (0.00)	0.03 (0.18)
	FR+SCAD	6.07 (0.5)	5.81 (0.45)	0.26 (0.60)	6.02 (0.17)	6.00 (0.06)	0.03 (0.18)
	FRx1	6.10 (0.45)	5.94 (0.33)	0.16 (0.46)	6.02 (0.17)	6.00 (0.00)	0.02 (0.17)
	FRx10	7.14 (0.58)	5.88 (0.49)	1.25 (0.60)	7.03 (0.17)	6.00 (0.00)	1.03 (0.17)
	FR(η1)	8.36 (2.55)	5.97 (0.27)	2.39 (2.56)	6.98 (1.31)	6.00 (0.00)	0.98 (1.31)
	FR(η2)	6.5 (0.90)	5.93 (0.44)	0.57 (0.95)	6.14 (0.43)	6.00 (0.00)	0.14 (0.43)
	PSIS	38.00 (0.00)	3.13 (0.94)	34.87 (0.94)	67.00 (0.00)	4.61 (0.67)	62.39 (0.67)
	PSIS+Lasso	8.17 (4.84)	3.38 (1.05)	4.80 (4.46)	8.98 (5.70)	4.62 (0.70)	4.36 (5.48)
	PSIS+MCP	5.34 (2.38)	2.97 (1.09)	2.37 (1.95)	7.82 (2.80)	4.47 (0.82)	3.35 (2.41)
	PSIS+SCAD	6.52 (3.31)	3.17 (1.06)	3.36 (3.02)	7.45 (4.02)	4.48 (0.77)	2.97 (3.73)
	CS	38.00 (0.00)	4.04 (0.87)	33.96 (0.87)	67.00 (0.00)	5.42 (0.64)	61.58 (0.64)
	CS+Lasso	8.41 (4.48)	4.13 (0.99)	4.29 (4.07)	10.64 (4.83)	5.41 (0.66)	5.23 (4.59)
	CS+MCP	5.81 (2.35)	3.68 (1.13)	2.12 (1.84)	7.53 (2.29)	5.33 (0.78)	2.20 (2.44)
	CS+SCAD	6.75 (3.19)	3.88 (1.07)	2.87 (2.81)	7.13 (3.07)	5.32 (0.79)	1.81 (3.17)
3 (P0 = 6)	FR	4.45 (1.51)	4.08 (1.64)	0.37 (0.61)	5.49 (0.54)	5.46 (0.50)	0.03 (0.21)
	FR+Lasso	4.45 (1.51)	4.40 (1.44)	0.05 (0.24)	5.49 (0.54)	5.48 (0.50)	0.01 (0.16)
	FR+MCP	4.45 (1.51)	4.40 (1.44)	0.05 (0.24)	5.49 (0.54)	5.48 (0.50)	0.01 (0.16)
	FR+SCAD	4.44 (1.50)	4.39 (1.43)	0.05 (0.24)	5.49 (0.54)	5.48 (0.50)	0.01 (0.16)
	FRx1	4.57 (1.39)	4.30 (1.44)	0.26 (0.50)	5.49 (0.54)	5.46 (0.50)	0.03 (0.21)
	FRx10	4.97 (1.91)	3.72 (1.88)	1.24 (0.48)	6.48 (0.53)	5.45 (0.50)	1.03 (0.21)
	FR(η1)	6.99 (2.59)	4.72 (1.40)	2.27 (2.29)	6.55 (1.18)	5.76 (0.43)	0.80 (1.09)
	FR(η2)	4.93 (1.62)	4.35 (1.56)	0.58 (0.84)	5.74 (0.63)	5.60 (0.49)	0.14 (0.42)
	PSIS	38.00 (0.00)	2.32 (0.76)	35.68 (0.76)	67.00 (0.00)	3.20 (0.61)	63.8 (0.61)
	PSIS+Lasso	2.17 (2.01)	1.46 (1.01)	0.71 (1.28)	5.24 (2.50)	2.94 (0.73)	2.30 (2.17)
	PSIS+MCP	2.66 (1.86)	1.81 (0.98)	0.86 (1.25)	3.44 (1.16)	2.88 (0.54)	0.56 (1.02)
	PSIS+SCAD	2.78 (2.35)	1.66 (1.08)	1.11 (1.55)	4.23 (1.53)	2.92 (0.59)	1.31 (1.37)
	CS	38.00 (0.00)	2.87 (0.71)	35.13 (0.71)	67.00 (0.00)	3.88 (0.69)	63.12 (0.69)
	CS+Lasso	3.34 (2.81)	2.03 (1.17)	1.31 (1.92)	7.16 (3.05)	3.71 (0.75)	3.45 (2.71)
	CS+MCP	3.45 (1.88)	2.44 (0.98)	1.00 (1.34)	4.19 (1.15)	3.62 (0.64)	0.57 (0.92)
	CS+SCAD	4.13 (2.60)	2.42 (1.14)	1.71 (1.82)	4.80 (1.36)	3.66 (0.66)	1.14 (1.24)

First, Example 1 was designed in such a way that all of the true signals have nonzero marginal correlations with the outcome and are detectable by marginal screening methods. In particular, the dependence among the active variables in Example 1 can further strengthen the marginal correlations between them and the outcome. Even under these situations, FR was found to perform better than the marginal screening methods, including the conditional screening method, with larger PIT, TP and smaller FP. When the sample size decreases to 200 or with more censored events, FR’s performance was still decent with, for example, a PIT around 0.8. In contrast, the performances of PSIS and CS deteriorated quickly with smaller sample sizes or with more censoring.

Second, Examples 2 and 3 were designed so that X₆, though active, has a 0 marginal correlation with the outcome and, therefore, is not detectable by marginal screening methods. As a result, PSIS and CS failed in this challenging situation. In contrast, FR remained competent by being able to detect the hidden variable. It even outperformed the conditional screening that used the prior information.

Third, even using penalties such as Lasso, SCAD, and MCP to further reduce false positives, the screening methods still incurred many false positives. The final models selected by the different penalties vary a bit. However, compared to FR, all of these penalties still perform similarly. In contrast, with the EBIC-based stopping rule, the proposed FR caused much fewer false positives without the help of Lasso.

Finally, the simulation results hint that the performance of FR is quite robust toward the choice of the initial set. Even if the “wrong” set was employed as the initial set, the TP is almost the same as the one that starts from the null set.

We further explored the robustness of the method to the violation of the independence censoring assumption. We considered Examples 1*–3*, which have the same setup as Examples 1–3, except that the underlying survival times and censoring times have a common latent variable b which was generated from the standard Gaussian distribution and the censoring times C_i were covariate-dependent. That is,

$$\lambda (t\mid \mathbf{X})=\exp (b+{\beta }^{⊺}\mathbf{X})\text{and}{\lambda }_C(t\mid \mathbf{X})=c\exp (b+{\alpha }^{⊺}\mathbf{X}),$$

where α = (0.75, 0.75, 0_p−;2) and c was chosen to censor approximately 25% of the observations. The results are documented in Table 3.

We found that, with dependent censoring, the performance of all of the methods deteriorated a bit across the board. However, our proposed method still outperformed all the other methods, hinting at the usability of the proposal under dependent censoring. More work is warranted.

5. Analysis of the Boston Lung Cancer Survival Cohort (BLCSC) study

Recent studies demonstrate that aberrant methylation may be the most common mechanism of inactivating cancer-related genes in lung cancer. It occurs in the smoking-damaged bronchial epithelium from cancer-free individuals, can be reversed in vitro by demethylating agents, and may be a useful biomarker for lung cancer risk assessment [40]. It is thus of substantial interest to identify the methylations that play an important role in the pathogenesis of lung cancer, which affects patients’ overall survival.

The motivating data represented a subset of the Boston Lung Cancer Survival Cohort (BLCSC) and included 124 samples, each with 442,613 methylations. The median follow up time of the subjects was 6.2 years and during the follow up, 84 deaths were observed. Each methylation resides within a certain gene. Prior literature has suggested that the following genes are associated with the development of lung cancer: ROS1, RET, PIK3CA, NRAS, BRAF, ALK, AKT 1, VGLL2, MET, KRAS, EGFR, KDM4, ST3GAL3, and CDH13. We used the array annotations from the Bioconductor package FDb.InfiniumMethylation.hg19 (version 2.2.0) to identify methylations that lie within these genes. A total of 589 methylations were identified. The other available environmental exposure and demographic variables in the data included lifetime tobacco exposure (SMOK), computed by multiplying the number of packs of cigarettes smoked per day by the number of years the person has smoked until the beginning of the study; AGE, the age at diagnosis in years; and SEX (1 = male; 0 = female).

Our analytical goal was to explore what methylations and their interactions with demographic and environmental exposure variables might be related to patients’ overall survival. Thus, the outcome was the time to death, while the dependent variables included the aforementioned variables, consisting of demographic information, environmental exposures, and methylations and their interactions, for a total of 2,359 variables. We applied FR to the dataset and identified cg04187088×SEX and cg14363146×SMOK.

To check the model adequacy for the final model obtained by FR, we plotted the Cox-Snell residuals based on the final model that includes two predictors, cg04187088×SEX and cg14363146×SMOK. Figure 1 shows that the final model fits the data reasonably well.

Figure 1:

Cox–Snell residual plot.

Furthermore, we conducted the score test for the scaled Schoenfeld residuals to test the proportional hazards assumption on each included predictor in the final model. We obtained p-values of 0.542 and 0.670 for cg04187088 × SEX and cg14363146 × SMOK, respectively. It appears that the proportional hazards assumption is not rejected for either of them.

We also applied competing methods to the BLCSC dataset, including the PSIS, CS, and SII. For each competing method, we selected the top ⌊n/ ln(n)⌋ = 25 genes, and compared them with the genes selected by FR.

In terms of computing time, PSIS, CS, SII, and FR took 2.47, 2.52, 1473.65, and 20.89 seconds, respectively. Due to the sequential nature of the proposed procedure, FR is understandably more computationally intensive than the marginal screening approaches such as PSIS and CS. However, FR appears to run faster than SII, a nonparametric approach.

Table 4 lists the overlapping genes across the four methods. It appears that two genes selected by FR did not overlap with any genes selected by PSIS, CS and SII.

Table 4:

The number of overlapped genes chosen by PSIS, CS, SII and FR

	PSIS	CS	SII	FR
PSIS	25	1	3	0
CS	1	25	0	0
SII	3	0	25	0
FR	0	0	0	2

In contrast, using the SMOK, AGE, and SEX as the initial set, FR further selected cg11704212, in addition to these identified interactions. Our Pubmed review did not detect any previous literature that discusses these two SNPs. This may indicate the ability of the proposed FR to identify some novel SNPs, which may not have been detected by the existing methods. We expect that future studies are warranted to confirm and study the functionality of these detected biomarkers.

To elucidate the identified effects, we further conducted Kaplan–Meier analysis. We first dichotomized methylations by using the median values and used “+” or “−” if a subject’s methylation is higher or lower than its median, respectively. Figure 2 compared survival curves across various subgroups. Figure 2(a) clearly indicated that female patients with low cg04187088 had a higher survival probability than the other comparison groups, while Figure 2(b) revealed that heavy smokers with high cg14363146 had a much lower survival probability than the other comparison groups. Figure 2(c) also showed that patients with high cg11704212 had a larger survival probability than those with low cg11704212.

Figure 2:

Kaplan–Meier plots to illustrate the main and interaction effects identified by the proposed method.

6. Concluding remarks

This article proposes forward regression with partial likelihood for high-dimensional survival data, and has obtained computationally and theoretically useful results. We envision the established theoretical framework will facilitate the extension of the procedure to other general settings, such as generalized linear models.

To further improve the computational efficiency, we can consider a natural extension of the proposed forward selection in the spirit of boosting [11]. That is, at each step, we use the selected variables and the obtained coefficient estimates to construct an offset, and search for a variable that will maximize the partial likelihood with such an offset. The advantage is that we need to maximize the partial likelihood with respect to only one covariate at each step, which may enhance computational efficiency. Specifically, we denote the log partial likelihood with an offset term O and covariate j in the model by

$${\ell }_{O,j}\left(\beta \right)=\underset{i=1}{\overset{n}{\Sigma }}{\int }_0^{\tau }\left[O_i+\beta X_{ij}-\ln \left\{\underset{\ell =1}{\overset{n}{\Sigma }}{\stackrel{‒}{Y}}_{\ell }\left(t\right)\exp (O_{\ell }+{\beta }_j{X}_{\ell j})\right\}\right]d{N}_i\left(t\right),$$

Here O_i refers to the offset term O evaluated at the ith subject. We let O^(k) be the offset term evaluated at kth step and S_k be the set of indices of covariates selected up to the kth step.

For FR, we initialize S₀ = Ø and set O⁽⁰⁾ = 0. For j ∈ {1, … , p}, compute

$${\widehat{\beta }}_j^{\left(1\right)}=\underset{\beta }{\overset{\arg \max }{}}{\ell }_{O^{\left(0\right)},j}\left(\beta \right)$$

Then j₁ = arg max_j∈{1…,p}ℓ_O(0),_j ${\left(\widehat{\beta }\right)}_j^{\left(1\right)}$ Now set $O^{\left(1\right)}={\widehat{\beta }}_{j1}^{\left(1\right)}{X}_{j1}$ and S₁ = {j₁}. Given O^(k) and S_k, compute

$${\widehat{\beta }}_j^{(k+1)}=\underset{\beta }{\overset{\arg \max }{}}{\ell }_{O^{\left(k\right)},j}\left(\beta \right)$$

for $j\in S_k^{\mathrm{C}}$. Then $j_{k+1}={\arg \max }_{j\in S_k^{\mathrm{C}}}{\ell }_{O^{\left(k\right)},j}\left({\widehat{\beta }}_j^{(k+1)}\right)$. Now set $O^{(k+1)}=O^{\left(k\right)}+{\widehat{\beta }}_{jk+1}^{(k+1)}{X}_{jk+1}$ and S_k+1 = Sk ∪ {j_k+1}. We note this proposal does not require re-estimation of the coefficients of the covariates selected in the previous steps, which expedites computation. We will explore this further.

We employed a modified EBIC to select the final models. Although it worked well in our simulations, it tends to be conservative in real data analysis and recruits too few variables, whereas BIC recruits too many variables. It would be interesting to investigate the optimal η in the EBIC penalty term to strike a balance between EBIC and BIC.

Appendix

A. Preliminary lemmas

We present some preliminary lemmas in this section. Given an index set S ⊂ {1,…,p}, we use S_+r to denote S ∪ {r} for some r ∈ S^C.

Lemma A. Condition (G) is satisfied if $\mathcal{M}\cap {\mathcal{M}}_{\mathcal{A}}=\varnothing$.

Without loss of generality, we assume that X_r is the last element of X_S+r, with ${\beta }_{\ast }^r$ being the corresponding least false coefficient under the model S_+r.

Lemma B. Given S ⊂ {1,…,p} satisfying |S| < ρ and r ∈ S^C, the following statements hold under Conditions (B) and (G).

(i) If $r\in {\mathcal{M}}^{\mathrm{C}}$ and the ${\mathbf{X}}_{{\mathcal{M}}^{\mathrm{C}}}$ are independent of ${\mathbf{X}}_{\mathcal{M}}$, then ${\beta }_{S_{+r}}^{\ast }=({{\beta }_S^{\ast }}^{⊺},0{)}^{⊺}$, i.e., the least false coefficient for X_r under the model S_+r is 0.

If $r\in \mathcal{M}$, then ${\beta }_{S_{+r}}^{\ast }\ne ({{\beta }_S^{\ast }}^{⊺},0{)}^{⊺}$.

If $r\in \mathcal{M}$ and Conditions (E) and (F) are satisfied, then $\parallel {\beta }_{S_{+r}}^{\ast }-({{\beta }_S^{\ast }}^{⊺},0{)}^{⊺}\parallel \ge K{k}_{\max }^{-1}{n}^{-\alpha }$.

Lemma B quantifies the difference between ${\beta }_{S_{+r}}^{\ast }$ and ${({{\beta }_S^{\ast }}^{⊺},0)}^{⊺}$ when a noise variable or a signal variable is selected into the model.

Lemma C. Under Conditions (B) and (D), if ρ³ ln(p/n) → 0, then for each S ⊂ {1,…, p} satisfying |S| ≤ ρ, we can find a neighborhood ${\mathcal{B}}_S^0\left(c\right)$ of ${\beta }_S^{\ast }$, for some constant c, such that, for k ∈ {0,1,2},

$$\underset{\mid S\mid \le \rho ,t\in [0,\tau ],{\beta }_S\in {\mathcal{B}}_S^0\left(c\right)}{\sup }\parallel R_S^{\left(k\right)}({\beta }_S,t)-r_S^{\left(k\right)}({\beta }_S,t)\parallel {\to }_{p}0.$$

Define

$$Z_S\left({\beta }_S\right)=\mid \left[{\mathbb{E}}_n\right\{{\gamma }_S({\beta }_S;{\mathbf{X}}_i,Y_i,{\delta }_i)\}-{\Gamma }_S({\beta }_S\left)\right]-\left[{\mathbb{E}}_n\right\{{\gamma }_S({\beta }_S^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i)\}-{\Gamma }_S({\beta }_S^{\ast }\left)\right]\mid .$$

Lemma D. Under the same conditions as in Lemma C,

$$\mathrm{Pr}\left\{\underset{\mid S\mid {\le }_{\rho },{\beta }_S\in {\mathcal{B}}_S^0\left(c\right)}{\sup }{Z}_S\left({\beta }_S\right)\ge 2\frac{c}{K}{a}_n+3\frac{c\sqrt{\rho }}{K\sqrt{S}}{a}_n+6c\tau \sqrt{\rho \ln (p∕n)}+2K\sqrt{\rho }{\zeta }_n\right\}\le 3\exp (-6\rho \ln p),$$

where

$$a_n=\sqrt{2K^2\ln \left(2p\right)∕n}+K\ln \left(2p\right)∕n\text{and}{\zeta }_n=c∕\left(K{n}^2\right).$$

Define $D_S\left({\beta }_S\right)=\mid \left\{{\ell }_S\right({\beta }_S)-{\tilde{\ell }}_S({\beta }_S\left)\right\}-\left\{{\ell }_S\right({\beta }_S^{\ast })-{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}\mid ∕n$

Lemma E. Under Conditions (A), (B), and (D), we have

$$\mathrm{Pr}\left\{\underset{\mid S\mid {\le }_{\rho },{\beta }_S\in {\mathcal{B}}_S^0\left(c\right)}{\sup }{D}_S\left({\beta }_S\right)\ge A_{10}\sqrt{{\rho }^2\ln (p∕n)}\right\}\le 2\exp (-3\rho \ln p),$$

for some constant A₁₀ that does not depend on n.

Lemma F. Under Conditions (B), (D), and (F), given an index set S satisfying |S| ≤ ρ, then for any ${\beta }_S\in {\mathcal{B}}_S^0\left(c\right)$,

$${\kappa }_{\min }{\parallel {\beta }_S-{\beta }_S^{\ast }\parallel }^2∕2\le {\Gamma }_S\left({\beta }_S\right)-{\Gamma }_S\left({\beta }_S^{\ast }\right)\le {\kappa }_{\max }{\parallel {\beta }_S-{\beta }_S^{\ast }\parallel }^2∕2.$$

Lemma G. Under Conditions (A)–(F), if ρ⁴ ln(p/n) → 0, there exist two constants A₁₁ and A₁₂ that do not depend on n such that

$$\mathrm{Pr}\left[\underset{\mid S\mid \le \rho }{\overset{\sup }{}}\parallel {\widehat{\beta }}_S-{\beta }_S^{\ast }\parallel \le A_{11}\left\{{\rho }^2\ln \right(p∕n){\}}^{1∕4}\right]\ge 1-5\exp (-3\rho \ln p)$$ (i)

and

$$\mathrm{Pr}\left\{\underset{\mid S\mid \le \rho }{\overset{\sup }{}}{n}^{-1}\mid {\ell }_S\left({\widehat{\beta }}_S\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\mid \le A_{12}\sqrt{{\rho }^2\ln (p∕n)}\right\}\ge 1-5\exp (-3\rho \ln p)$$ (ii)

Lemma H. Under Conditions (B) and (D), there exists some constant A₁₄, which does not depend on n, such that

$$\mathrm{Pr}\left[\underset{\mid S\mid <\rho ,r\in S^{\mathrm{C}}}{\overset{\sup }{}}\mid n^{-1}\left\{{\ell }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\mid \ge A_{14}\sqrt{\rho \ln (p∕n)}\right]\le 3\exp (-3\rho \ln \rho ).$$

B. Proofs of main theoretical results in Section 3.2

In the following, we provide the proofs for the theoretical results in Section 3.2.

Proof of Theorem 1. We prove the theorem for a generic index set S satisfying S ⊂ {1,…, p}, $\mathcal{M}\not\subset S$ and |S| < ρ. The change of log-likelihood by adding a variable X_r, where r ∈ S^C, can be decomposed as

$$\begin{gathered} n^{-1}\left\{{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}=n^{-1}\left[{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S)-\{{\ell }_{S_{+r}}\left({\beta }_{S_{+r}}^{\ast }\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\left\}\right] \\ +\left[n^{-1}\right\{{\ell }_{S_{+r}}\left({\beta }_{S_{+r}}^{\ast }\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\}-\{-{\Gamma }_{S_{+r}}\left({\beta }_{S_{+r}}^{\ast }\right)+{\Gamma }_S\left({\beta }_S^{\ast }\right)\left\}\right]-\left\{{\Gamma }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}. \end{gathered}$$

We restrict our attention ${\Omega }_5^{\mathcal{C}}\cap {\Omega }_6^{\mathcal{C}}$, where

$${\Omega }_5=\left\{\underset{\mid S\mid \le \rho }{\overset{\sup }{}}{n}^{-1}\mid {\ell }_S\left({\widehat{\beta }}_S\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\mid A_{12}\sqrt{{\rho }^2\ln (p∕n)}\right\}$$

and

$${\Omega }_6=\left\{\underset{\mid S\mid <\rho ,r\in S^{\mathrm{C}}}{\overset{\sup }{}}\mid n^{-1}\left\{{\ell }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\mid \ge A_{14}\sqrt{\rho \ln (p∕n)}\right\}.$$

According to Lemmas G and H, ${\Omega }_5^{\mathrm{C}}\cap {\Omega }_6^{\mathrm{C}}$ holds with probability at least 1 – 8 exp(−3ρ ln p).

If $r\in \mathcal{M}$, then by Lemma B (iii), $\mid \mid {\beta }_{S+r}^{\ast }-({\beta }_S^{\ast ⊺},0{)}^{⊺}\mid \mid \ge K{\kappa }_{\max }^{-1}{n}^{-\alpha }$. For any β_S+r such that $\mid \mid {\beta }_{S_{+r}}-{\beta }_{S_{+r}}^{\ast }\mid \mid =K{\kappa }_{\max }^{-1}{n}^{-\alpha }$, noting that ${\beta }_{S_{+r}}^{\ast }$ is the solution to (4) under model S_+r, we apply Taylor’s expansion to obtain

$$\begin{gathered} {\Gamma }_{S_{+r}}\left({\beta }_{S_{+r}}\right)-{\Gamma }_{S_{+r}}\left({\beta }_{S_{+r}}^{\ast }\right)=\frac{1}{2}({\beta }_{S_{+r}}^{\ast }-{\beta }_{S_{+r}}{)}^{⊺}\times \left[{\int }_0^{⊺}\frac{r_{S_{+r}}^{\left(2\right)}({\tilde{\beta }}_{S_{+r}},u)}{r_{S_{+r}}^{\left(0\right)}({\tilde{\beta }}_{S_{+r}},u)}-\frac{\left\{r_{S_{+r}}^{\left(1\right)}\right({\tilde{\beta }}_{S_{+r}},u){\}}^{\otimes 2}}{\left\{S^{\left(0\right)}\right({\tilde{\beta }}_{S_{+r}},u){\}}^2}{v}^{\left(0\right)}\left(u\right)du\right]({\beta }_{S_{+r}}^{\ast }-{\beta }_{S_{+r}}) \\ \ge K^2{\kappa }_{\min }{\kappa }_{\max }^{-2}{n}^{-2\alpha }\equiv c_1{n}^{-2\alpha }, \end{gathered}$$

where ${\tilde{\beta }}_{S_{+r}}$ is between β_S+r and ${\beta }_{S_{+r}}^{\ast }$, the last inequality follows from Condition (F) and $c_1=K^2{\kappa }_{\min }{k}_{\max }^{-2}$. By the convexity of Γ_S+r (β, we have ${\Gamma }_{S_{+r}}\left\{\right({{\beta }_S^{\ast }}^{⊺},0{)}^{⊺}\}-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })\ge c_1{n}^{-2\alpha })$ Thus,

$$\begin{gathered} n^{-1}\left\{{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}\ge -\{n^{-1}\mid {\ell }_{S_{+r}}({\widehat{\beta }}_{S_{+r}})-{\ell }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })\mid +n^{-1}\mid {\ell }_S({\widehat{\beta }}_S)-{\ell }_S({\beta }_S^{\ast })\mid \} \\ -\mid n^{-1}\left\{{\ell }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\mid +\left\{{\Gamma }_S\right({\beta }_S^{\ast })-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast }\left)\right\} \\ \ge -2A_{12}\sqrt{{\rho }^2\ln (p∕n)}-A_{14}\sqrt{\rho \ln (p∕n)}+c_1{n}^{-2\alpha }. \end{gathered}$$

Consequently,

$$\underset{\mid S\mid <\rho ,r\in S^{\mathrm{C}}}{\overset{\sup }{}}{n}^{-1}\left\{{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}\ge c_1{n}^{-2\alpha }-2A_{12}\sqrt{{\rho }^2\ln (p∕n)}-A_{14}\sqrt{\rho \ln (p∕n)}\ge c_1{n}^{-2\alpha }-c_2\sqrt{{\rho }^2\ln (p∕n)},$$

for some constant c2 that does not depend on n. Then, we obtain

$$\underset{r\in S^{\mathrm{C}}}{\overset{\max }{}}\left\{{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}\ge c_1{n}^{1-2\alpha }-c_2\sqrt{n{\rho }^2\ln p}.$$

Withdrawing the restriction on ${\Omega }_5^{\mathrm{C}}\cap {\Omega }_6^{\mathrm{C}}$ completes the proof of Theorem 1. □

Proof of Corollary 1. As shown in Theorem 1, for any S such that $\mid S\mid <\rho ,\mathcal{M}\not\subset S$, with probability at least 1 – 8 exp(−3ρ ln p),

$$\underset{r\in S^{\mathrm{C}}}{\overset{\max }{}}{\ell }_{S_{+r}}\left({\widehat{\beta }}_{S_{+r}}\right)-{\ell }_S\left({\widehat{\beta }}_S\right)\ge c_1{n}^{1-2\alpha }-c_2\sqrt{n{\rho }^2\ln \left(p\right)}.$$

If $\sqrt{n{\rho }^2\ln p}=o\left(n^{1-2\alpha }\right)$, then ln d + 2η ln p = o(n^1–2α), and consequently,

$$\begin{gathered} \mathrm{EBIC}\left(S_{+r}\right)-\mathrm{EBIC}\left(S\right)=-2{\ell }_{S_{+r}}\left({\widehat{\beta }}_{S_{+r}}\right)+(\mid S\mid +1)(\ln d+2\eta \ln p)-\{-2{\ell }_S({\widehat{\beta }}_S)+\mid S\mid (\ln d+2\eta \ln p\} \\ \le c_2\sqrt{n{\rho }^2\ln p}-c_1{n}^{1-2\alpha }+(\ln d+2\eta \ln p)<0. \end{gathered}$$

Therefore, our forward selection does not stop when $\mathcal{M}\not\subset S$ and |S| < ρ with probability at least 1 – 8 exp(−3ρ ln p). Noting that S₀ = Ø, if $\mathcal{M}\not\subset S_k$, then

$$\begin{gathered} \underset{i=1}{\overset{n}{\Sigma }}\left[{\int }_0^{\tau }\ln \left\{n{R}_{S_0}^{\left(0\right)}\right(0,t\left)\right\}d{N}_i\left(t\right)\right]-0\ge \left\{{\ell }_{S_1}\right({\widehat{\beta }}_{S_1})-{\ell }_{S_0}({\widehat{\beta }}_{S_0}\left)\right\}+\left\{{\ell }_{S_2}\right({\widehat{\beta }}_{S_2})-{\ell }_{S_1}({\widehat{\beta }}_{S_1}\left)\right\}+\cdots +\left\{{\ell }_{S_k}\right({\widehat{\beta }}_{S_k})-{\ell }_{S_{k-1}}({\widehat{\beta }}_{S_{k-1}}\left)\right\}\ge \\ k{c}_1{n}^{1-2\alpha }∕2, \end{gathered}$$

when n is sufficiently large such that 2c₂ $\sqrt{{\rho }^2\ln (p∕n)}\le c_1{n}^{-2\alpha }∕2$.

As shown in the proof of Lemma H,

$${\mathbb{E}}_n\left\{{\int }_0^{\tau }\ln R_{S_0}^{\left(0\right)}(0,t)d{N}_i\left(t\right)\right\}\to \mathrm{E}\left\{{\int }_0^{\tau }\ln r_{S_0}^{\left(0\right)}(0,t)d{N}_i\left(t\right)\right\},$$

with probability at least 1 – 3 exp(−3ρ ln p). If $\mathcal{M}\not\subset S_M$, then

$$\frac{2}{c_1{n}^{-2\alpha }}\mathrm{E}\left[{\int }_0^{\tau }\ln \left\{n{r}_{S_0}^{\left(0\right)}\right(0,t\left)\right\}d{N}_i\left(t\right)\right]>M,$$

which contradicts the definition of M. Hence, we have some k such that $\mathcal{M}\subset S_k$. This completes the proof of Corollary 1. □

Proof of Corollary 2. By Lemma B (i), If $r\in \mathcal{M}\mathrm{C}$ and ${\mathbf{X}}_{{\mathcal{M}}^{\mathrm{C}}}$ are independent of ${\mathbf{X}}_{\mathcal{M}}$, then ${\beta }_{S_{+r}}^{\ast }=({\beta }_S^{\ast ⊺},0{)}^{⊺}$. Thus, under ${\Omega }_5^{\mathrm{C}}\cap {\Omega }_6^{\mathrm{C}}$,

$$n^{-1}\left\{{\ell }_{S_{+r}}\right({\widehat{\beta }}_{S_{+r}})-{\ell }_S({\widehat{\beta }}_S\left)\right\}\le n^{-1}\mid {\ell }_{S_{+r}}\left({\widehat{\beta }}_{S_{+r}}\right)-{\ell }_{S_{+r}}\left({\beta }_{S_{+r}}^{\ast }\right)\mid +n^{-1}\mid {\ell }_S\left({\widehat{\beta }}_S\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\mid \le 2A_{12}\sqrt{{\rho }^2\ln (p∕n)}.$$

If $\sqrt{n{\rho }^2\ln p}=o\left(n^{1-2\alpha }\right)$, we have for any S such that |S| < ρ and $\mathcal{M}\not\subset S$, when n is sufficiently large.

$$\underset{r\in S^{\mathrm{C}}}{\overset{\arg \min }{}}\mathrm{EBIC}\left(S_{+r}\right)-\mathrm{EBIC}\left(S\right)\in \mathcal{M}.$$

Withdrawing the restriction on ${\Omega }_5^{\mathrm{C}}\cap {\Omega }_6^{\mathrm{C}}$, we obtain that, at each step, the probability of selecting a noise variable is at most 8 exp(−3ρ ln p). Since $\mathcal{M}\not\subset S_k$ implies that at more than $k-\mid \mathcal{M}\mid$ steps, a noise variable is selected, then for $k=c_3\mid \mathcal{M}\mid$,

$$\begin{gathered} \mathrm{Pr}(\mathcal{M}\not\subset S_k)\le \underset{j=k-\mid \mathcal{M}\mid }{\overset{k}{\Sigma }}\left(\begin{array}{c}\hfill k\hfill \\ \hfill j\hfill \end{array}\right)\left\{8\exp \right(-3\rho \ln p){\}}^j\le \mid \mathcal{M}\mid k^{\mid \mathcal{M}\mid }\{8\exp (-3\rho \ln p){\}}^{k-\mid \mathcal{M}\mid } \\ \le 8\exp (-3\rho \ln p+\ln \mid \mathcal{M}\mid +\mid \mathcal{M}\mid \ln k)8\exp (-2\rho \ln p). \end{gathered}$$

Therefore, $\mathcal{M}\subset S_{c_3\mid \mathcal{M}\mid }$ with probability at least 1 – 8 exp(−2ρ ln p). This completes the proof of Corollary 2. □

Proof of Corollary 3. By Corollary 2, we know that $\mathcal{M}\subset S_k$, for some $k\le c_3\mid \mathcal{M}\mid$. Thus, both $S_k\subset S_{k+1}\in {\mathcal{A}}_0$, where

$${\mathcal{A}}_0=\{S:\mathcal{M}\subset S,\mid S\mid \le c_3\mid \mathcal{M}\mid \}.$$

(i): It can be shown that EBIC(S_k+1) < EBIC(S_k) if and only if $2{\ell }_{S_{k+1}}\left({\widehat{\beta }}_{S_{k+1}}\right)-2{\ell }_S\left({\widehat{\beta }}_{S_k}\right)<\ln d+2\eta \ln p$. Following the same arguments used to show Eq. (14) in [24], we can show that with probability tending to 1,

$$2{\ell }_{S_{k+1}}\left({\widehat{\beta }}_{S_{k+1}}\right)-2{\ell }_S\left({\widehat{\beta }}_{S_k}\right)<\ln d+2\eta \ln p,$$

for all η > 0. Therefore, with probability tending to 1, the procedure stops at the kth step and $\mathcal{M}\subset \widehat{\mathcal{M}}=S_k$.

(ii): From Corollary 2 and Part (i), we have $\mid \widehat{\mathcal{M}}\cap {\mathcal{M}}^{\mathrm{C}}\mid =\mid \widehat{\mathcal{M}}\mid -\mid \mathcal{M}\mid$ as well as $\mid \widehat{\mathcal{M}}\mid \ge \mid \mathcal{M}\mid$ with probability going to 1. Hence, the stated result follows. □

Proof of Lemma A. We first show that $r\in \mathcal{M}$, $\mathrm{E}\left\{X_r{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {{\mathbf{X}}_{\mathcal{M}}}_{o/r}\}$ has the same sign across t > 0. Let $S_0\left(t\right)=\exp \{-{\int }_0^t{\lambda }_0(u\left)du\right\}$.

$$\begin{gathered} \mathrm{E}\left\{X_r{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/r}}\}=\mathrm{E}\left\{X_r{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}})\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\} \\ =\mathrm{E}\left\{fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\}=\mathrm{E}\left\{X_r{S}_0\right(t{)}^{\exp \left({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\} \\ =\mathrm{E}\{fC(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\}=\mathrm{E} \\ =\{X_r{S}_0(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)\exp \left({\beta }_{0r}{X}_r\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\}\mathrm{E}\left\{fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\} \\ =\frac{1}{{\beta }_{0r}}\mathrm{E}\left[{\beta }_r{X}_r\left\{S_0\right(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}{\}}^{\exp \left({\beta }_{0r}{X}_r\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\right]\mathrm{E}\left\{fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{\mathcal{M}\mathcal{A}}\} \\ =-\frac{1}{{\beta }_{0r}}\mathrm{E}\left[-U\left\{S_0\right(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}{\}}^{\exp \left(U\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\phantom{\mid }\right]\mathrm{E}\left\{fC\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\}, \end{gathered}$$

where U =β_0rX_r. Noting that S₀(t) ≤ 1 and $\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)>0$, we have $S_0(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}\le 1$. Therefore, given ${\mathbf{X}}_{\mathcal{M}\backslash r}$, $-\left\{S_0\right(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}{\}}^{\exp \left(U\right)}$ is monotone increasing with respect to U.

Then by [25],

$$\mathrm{E}\left[-U\left\{S_0\right(t{)}^{\exp \left({\beta }_{0\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}{\}}^{\exp \left(U\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\right]\ge 0,$$

for all t > 0, and hence

$$\mathrm{E}\left\{X_r{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)\right\}\mathrm{E}\left\{f_C\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\}=-\frac{1}{{\beta }_{0r}}\mathrm{E}\left[-U\left\{S_0\right(t{)}^{\exp \left({\beta }_{\mathcal{M}\backslash r}^{⊺}{\mathbf{X}}_{\mathcal{M}\backslash r}\right)}{\}}^{\exp \left(U\right)}\mid {\mathbf{X}}_{\mathcal{M}\backslash r}\right]\mathrm{E}\left\{f_C\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\}$$

has the same sign as −1/β_0r, for all t > 0.

Next, by the same argument used above, $\mathrm{E}\left\{X_r{S}_T\right(t;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(t;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/r}}\}$ has the same sign as −1/β_0r across t > 0. This completes the proof of Lemma A. □

Proof of Lemma B. We first note that ${\beta }_S^{\ast }$ is the root of (4). Let ${\mathbf{U}}_S\left({\beta }_S\right)={\int }_0^{\tau }\left\{v_S^{\left(1\right)}\right(t)-r_S^{\left(1\right)}({\beta }_S,t\left)v_S^{\left(0\right)}\right(t)∕r_S^{\left(0\right)}({\beta }_S,t\left)\right\}dt$.

$$\begin{gathered} {\mathbf{U}}_S\left({\beta }_S^{\ast }\right)={\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_S{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\} \\ -\frac{\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_S\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S\left)\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du=0. \end{gathered}$$ (A.1)

Part (i): Let $S_{\mathcal{M}}=S\cap \mathcal{M}$ and $S_{{\mathcal{M}}^{\mathrm{C}}}=S\cap {\mathcal{M}}^{\mathrm{C}}$. By the condition that the ${\mathbf{X}}_{{\mathcal{M}}^{\mathrm{C}}}$ is independent of ${\mathbf{X}}_{\mathcal{M}}$, the ${\mathbf{X}}_{S_{\mathcal{M}\mathrm{C}}}$ is independent of ${\mathbf{X}}_{S_{\mathcal{M}}}$. Thus, by Condition (B),

$$\begin{gathered} {\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_{S_{{\mathcal{M}}^{\mathrm{C}}}}{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\} \\ -\frac{\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_{S_{{\mathcal{M}}^{\mathrm{C}}}}\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du \\ ={\int }_0^{\tau }\mathrm{E}\left({\mathbf{X}}_{S_{{\mathcal{M}}^{\mathrm{C}}}}\right)\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\} \\ -\frac{\mathrm{E}\left({\mathbf{X}}_{S_{{\mathcal{M}}^{\mathrm{C}}}}\right)\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du=0, \end{gathered}$$

where the last equality follows from Condition (B). Combining the result and (A.1), it can be shown that

$${\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_S{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}-\frac{\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_S\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_{S_{\mathcal{M}}}^{\ast ⊺}{\mathbf{X}}_{S_{\mathcal{M}}}\left)\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du=0.$$

Therefore, ${\beta }_{\ast }^S=({\beta }_{S_{\mathcal{M}}}^{\ast ⊺},0^{⊺}{)}^{⊺}$.

If $r\in {\mathcal{M}}^{\mathrm{C}}$, then by the same arguments, we can show that ${\beta }_{S_{+r}}^{\ast }=({\beta }_S^{\ast ⊺},0{)}^{⊺}$.

Part (ii): Suppose ${\beta }_{S_{+r}}^{\ast }=({\beta }_S^{\ast ⊺},0{)}^{⊺}$. By the martingale property, ${\beta }_{S_{+r}}^{\ast }$ is also the solution to the following equation,

$${\int }_0^{\infty }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du={\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\left)t\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du.$$

Furthermore, it is straightforward to see that

$$\begin{gathered} {\int }_0^{\infty }E\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\text{exp}\right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}})du \\ ={\int }_0^{\infty }E\{X_r{S}_T(u;{\mathbf{X}}_{\mathcal{M}}){\lambda }_0\left(u\right)\text{exp}\left({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\right)S_C(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\}du \\ ={\int }_0^{\infty }E\{X_r{f}_T(u;\mathbf{X}\mathcal{M})S_C(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\}du=E\left\{X_r{\int }_0^{\infty }{F}_T(u;{\mathbf{X}}_{\mathcal{M}})f_C(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})du\right\} \\ =-{\int }_0^{\infty }E\{X_r{S}_T(u;{\mathbf{X}}_{\mathcal{M}})f_C(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\}du=-{\int }_0^{\infty }E[E\left\{X_r{S}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)f_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {{\mathbf{X}}_{\mathcal{M}}}_{o/r}\}]du, \end{gathered}$$

which has the opposite sign as $\mathrm{E}\left\{X_r{S}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)f_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {{\mathbf{X}}_{\mathcal{M}}}_{o/r}\}$ Besides,

$$\begin{gathered} \mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_{S}0\}=\mathrm{E}\{X_r{S}_T(u;{\mathbf{X}}_{\mathcal{M}})S_C(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\exp \left({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\right)\} \\ =\mathrm{E}[\mathrm{E}\left\{X_r{S}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\left)\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S)\mid {\mathbf{X}}_{{\mathcal{M}}_{o/r}}\}] \\ =\mathrm{E}[\mathrm{E}\left\{X_r{S}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/r}}\}\exp \left({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\right)], \end{gathered}$$

which has the same sign as $\mathrm{E}\left\{X_r{S}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)f_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}})\mid {\mathbf{X}}_{{\mathcal{M}}_{o/r}}\}$ by Condition (G) and the fact that $\left({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\right)>0$.

Thus, ${\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du$ and

$${\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\left)\right\}}{\mathrm{E}\left\{1\right(Y\ge u\left)\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\left)\right\}}\mathrm{E}\left\{1\right(Y\ge u\left){\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du$$

have the opposite signs. Contradiction! Therefore, ${\beta }_{S_{+r}}^{\ast }\ne ({\beta }_S^{\ast ⊺},0{)}^{⊺}$.

Part(iii): Without loss of generality, we assume that X_r is the last element of X_S+r. Let e_r be a vector of length |S| + 1 with the rth element 1 and all other elements 0. By definition, $v_S^{\left(0\right)}\left(u\right)=v_{S_{+r}}^{\left(0\right)}\left(u\right)$. Then by the Mean Value Theorem,

$$\begin{gathered} {\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du \\ -{\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\left)\right\}}{r_S^{\left(0\right)}({\beta }_S^{\ast },u)}{v}_S^{\left(0\right)}\left(u\right)du \\ -{\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du+{\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_{S_{+r}}^{\ast ⊺}{\mathbf{X}}_{S_{+r}}\left)\right\}}{r_{S_{+r}}^{\left(0\right)}({\beta }_{S_r}^{\ast },u)}{v}_{S_{+r}}^{\left(0\right)}\left(u\right)du \\ ={\int }_0^{\tau }\left[\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\mathbf{X}}_{S_{+r}}^{⊺}\exp \right({\tilde{\beta }}_{S_{+r}}^{⊺}{\mathbf{X}}_{S_{+r}}\left)\right\}{r}_{S_{+r}}^{\left(0\right)}({\beta }_{S_{+r}}^{\ast },u)}{\left\{r_{S_{+r}}^{\left(0\right)}\right({\beta }_{S_{+r}}^{\ast },u){\}}^2} \\ -\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_{S_{+r}}^{\ast ⊺}{\mathbf{X}}_{S_{+r}}\left)\right\}\mathrm{E}\left\{1\right(Y\ge u\left){\mathbf{X}}_{S_{+r}}^{⊺}\exp \right({\beta }_{S_{+r}}^{\ast ⊺}{\mathbf{X}}_{S_{+r}}\left)\right\}}{\left\{r_{S_{+r}}^{\left(0\right)}\right({\beta }_{S_{+r}}^{\ast },u){\}}^2}\right] \\ \times \{{\beta }_{S_{+r}}^{\ast }-({\beta }_S^{\ast ⊺},0{)}^{⊺}\}\times \mathrm{E}\{1(Y\ge u){\lambda }_0\left(u\right)\exp \left({\beta }_{\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\right)\}du, \end{gathered}$$

which reduces to

$${\int }_0^{\tau }{\mathbf{e}}_r^{⊺}\left[\frac{r_{S_{+r}}^{\left(2\right)}({\tilde{\beta }}_{S_{+r}},u)}{r_{S_{+r}}^{\left(0\right)}({\tilde{\beta }}_{S_{+r}},u)}-\frac{r_{S_{+r}}^{\left(1\right)}({\tilde{\beta }}_{S_{+r}},u){\}}^{\otimes 2}}{\left\{r_{S_{+r}}^{\left(0\right)}\right({\tilde{\beta }}_{S_{+r}},u){\}}^2}\right]({\beta }_{S_{+r}}^{\ast }-{\beta }_S^{\ast })v_{S_{+r}}^{\left(0\right)}\left(u\right)du,$$

where ${\tilde{\beta }}_{S_{+r}}$ is between ${\beta }_{S_{+r}}^{\ast }$ and $({\beta }_S^{\ast ⊺},0{)}^{⊺}$. Thus,

$$\begin{gathered} \mid {\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du\mid \\ \le \mid {\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{\mathcal{M}}^{\ast ⊺}{\mathbf{X}}_{\mathcal{M}}\left)\right\}du-{\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast ⊺}{\mathbf{X}}_S\}}{r_S^{\left(0\right)}({\beta }_S^{\ast },u)}{v}_S^{\left(0\right)}\left(u\right)du\mid \\ \le {\kappa }_{\max }\mid \mid {\mathbf{e}}_r\mid \mid \mid \mid {\beta }_{S_{+r}}^{\ast }-({\beta }_S^{\ast ⊺},0{)}^{⊺}\parallel , \end{gathered}$$

where the first inequality follows from the proof of part (ii) that ${\int }_0^{\tau }\mathrm{E}\left\{1\right(Y\ge u\left)X_r{\lambda }_0\right(u\left)\exp \right({\beta }_{0\mathcal{M}}^{\tau }{\mathbf{X}}_{\mathcal{M}}\left)\right\}du$ and

$${\int }_0^{\tau }\frac{\mathrm{E}\left\{1\right(Y\ge u\left)X_r\exp \right({\beta }_S^{\ast \tau }{\mathbf{X}}_S\left)\right\}}{r_S^{\left(0\right)}({\beta }_S^{\ast },u)}{v}_S^{\left(0\right)}\left(u\right)du$$

have the opposite signs, and the second inequality follows from Condition (F). Then, by Condition (E)

$$\mid \mid {\beta }_{S_{+r}}^{\ast }-({\beta }_S^{\ast ⊺},0{)}^{⊺}\mid \mid \ge \mid {\int }_0^{\tau }\mathrm{E}\left\{X_r{f}_T\right(u;{\mathbf{X}}_{\mathcal{M}}\left)S_C\right(u;{\mathbf{X}}_{{\mathcal{M}}_{\mathcal{A}}}\left)\right\}du\mid \ge K{\kappa }_{\max }^{-1}{n}^{-\alpha }.$$

This completes the proof of Lemma B.

Proof of Lemma C. We only prove k = 1, as k = 2, 3 can be proved similarly. Given an index set S of size |S| = s ≤ ρ, let ${\mathcal{B}}_S^0\left(c\right)=\{{\beta }_s:\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid \mid \le c∕(K\sqrt{S}\left)\right\}$. By Conditions (B) and (D), it can be shown that for any π ∈ $\pi \in {\mathbb{R}}^s$ satisfying ||π|| = 1,

$$0\le \mid \mid {\pi }^{⊺}\mathbf{X}\mid \mid \le \sqrt{s}K,\exp \left({\beta }_S^{⊺}{\mathbf{X}}_S\right)\le \exp \left\{\right({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}{\mathbf{X}}_S\left\}\exp \right({\beta }_S^{\ast }{\mathbf{X}}_S)\le \exp (c+KL).$$

Define $h({\beta }_S,\pi ,t)=\left\{\sqrt{S}K\exp \right(c+KL\left){\}}^{-1}\stackrel{‒}{Y}\right(t\left){\pi }^{⊺}{\mathbf{X}}_S\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S)$. Then by (B) and (C), h(β_S, π, u) is bounded between −1 and 1 uniformly over ${\mathcal{B}}_S^0\left(c\right),\mid \mid \pi \mid \mid =1$ and u ∈ [0, τ]. Define the function class

$${\mathcal{H}}_S=\left\{h\right({\beta }_S,\pi ,u):{\beta }_S\in {\mathcal{B}}_S^0(c),\mid \mid \pi \mid \mid =1,u\in [0,t\left]\right\}.$$

Following the arguments used for Lemma 11 in [1] and Lemma 2.6.17 in [31], we can show that there exists some universal constant A₁ such that the class of functions ${\mathcal{H}}_S$ has a VC index bounded by A₁ s; for the definitions of VC index, see p. 85 in [31]. By Theorem 2.6.7 in [31], for any probability measure Q, there exists some universal constant A₂, such that the covering number sup_Q $N[∊\mid \mid {\mathcal{H}}_S\mid {\mid }_{Q,2},{\mathcal{H}}_S,L_2(Q\left)\right]$ is bounded by (A₂/ϵ)^2A1s for any ϵ > 0; for the definition of covering numbers, see p. 83 in [31].

Thus, by Theorem 1.1 in [27], there exists some constant A₃ that depends on A₂ only, such that for all ϵ > 0,

$$\mathrm{Pr}\left[\underset{{\beta }_S\in {\mathcal{B}}_S^0\left(c\right),\mid \mid \pi \mid \mid =1,u\in [0,\tau ]}{\overset{\sup }{}}\mid \sqrt{n}{\mathbb{G}}_n\left\{h\right({\beta }_S,\pi ,u\left)\right\}\mid \ge \sqrt{n}∊\right]\le \frac{A_3}{∊}{\left(\frac{A_3{∊}^2}{A_{1^S}}\right)}^{A_{1^S}}\exp (-2{∊}^2).$$

By choosing $∊=A_4\sqrt{\rho \ln \left(p\right)}$ for some universal constant A₄, we obtain

$$\mathrm{Pr}\left[\underset{{\beta }_S\in {\mathcal{B}}_S^0\left(c\right),\mid \mid \pi \mid \mid =1,u\in [0,\tau ]}{\overset{\sup }{}}\mid {\mathbb{G}}_n\left\{h\right({\beta }_S,\pi ,u\left)\right\}\mid \ge A_4\sqrt{\rho \ln p}\right]\le \exp (-5\rho \ln p).$$

Consequently,

$$\mathrm{Pr}\left[\underset{\mid S\mid =s,{\beta }_S\in {\mathcal{B}}_S^0\left(c\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(1\right)}({\beta }_S,u)-r_S^{\left(1\right)}({\beta }_S,u)\mid \mid \ge A_4\sqrt{s}K\exp (c+KL)\sqrt{\rho \ln (p∕n)}\right]\le {\left(\frac{ep}{S}\right)}^s\exp (-5\rho \ln p),$$

where the inequality follows from the combinatoric inequality $\left(\begin{array}{c}\hfill p\hfill \\ \hfill s\hfill \end{array}\right)\le (ep∕s{)}^S$.

Let A₅ = A₄K exp(c + KL). Then,

$$\begin{gathered} \mathrm{Pr}\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(c\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(1\right)}({\beta }_S,u)-r_S^{\left(1\right)}({\beta }_S,u)\mid \mid \ge A_5\sqrt{{\rho }^2\ln (p∕n)}\right\} \\ \le \underset{s=1}{\overset{\rho }{\Sigma }}{\left(\frac{ep}{s}\right)}^s\exp (-5\rho \ln p)\le \exp (-3\rho \ln p), \end{gathered}$$

when n is sufficiently large. Thus, if ρ² ln(p/n) → 0,

$$\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(c\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(1\right)}({\beta }_S,u)-r_S^{\left(1\right)}({\beta }_S,u)\mid \mid {\to }_{p}0.$$

Similarly, we can show that

$$\mathrm{Pr}\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(c\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(0\right)}({\beta }_S,u)-r_S^{\left(0\right)}({\beta }_S,u)\mid \mid \ge A_6\sqrt{\rho \ln (p∕n)}\right\}\le \exp (-3\rho \ln p)$$

and

$$\mathrm{Pr}\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(c\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(2\right)}({\beta }_S,u)-r_S^{\left(2\right)}({\beta }_S,u)\mid \mid \ge A_7\sqrt{{\rho }^3\ln (p∕n)}\right\}\le \exp (-3\rho \ln p)$$

for some constants A₆ and A₇ that do not depend on n. This completes the proof of Lemma C.

Proof of Lemma D. Given an index set S such that |S| < ρ, it is easy to see that Z_s (β_S) ≤ I + II, where

$$I=n^{-1∕2}\mid {\mathbb{G}}_n\left[{\int }_0^{\tau }\{{\beta }_S^{⊺}{\mathbf{X}}_{iS}-{\beta }_S^{\ast ⊺}{\mathbf{X}}_{iS}\}d{N}_i\left(t\right)\right]\mid ,II=n^{-1∕2}\mid {\mathbb{G}}_n\left[{\int }_0^{\tau }\left\{\ln r_s^{\left(0\right)}\right({\beta }_S,t)-\ln r_S^{\left(0\right)}({\beta }_S^{\ast },t\left)\right\}d{N}_i\left(t\right)\right]\mid .$$

For term I, by Conditions (B) and (C), it can be shown that $\mid {\beta }_S^{⊺}{\mathbf{X}}_{iS}-{\beta }_S^{\ast ⊺}{\mathbf{X}}_{iS}\mid \le c∕\left(K\sqrt{S}\right)K\sqrt{S}\le c$, and $\mathrm{var}\left\{\right({\beta }_S^{\tau }{\mathbf{X}}_{iS}-{\beta }_S^{\ast T}{\mathbf{X}}_{iS}\left)N_i\right(\tau \left)\right\}\le {\kappa }_{\max }{c}^2∕\left(K^{2}s\right)$, for any ${\beta }_S\in {\mathcal{B}}_S^0\left(c\right)$. Let ϵ₁,…,ϵ_n be a Rademacher sequence. Then we have

$$\begin{gathered} \mathrm{E}\left(\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(c\right)}{\overset{\sup }{}}I\right)\le 2\mathrm{E}\left[\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(c\right)}{\overset{\sup }{}}\mid {\mathbb{E}}_n\left\{{∊}_i\right({\beta }_S^{⊺}{\mathbf{X}}_{iS}-{\beta }_S^{\ast ⊺}{\mathbf{X}}_{iS}\left)N_t\right(\tau \left)\right\}\mid \right] \\ \le 2\mathrm{E}\left[\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(c\right)}{\overset{\sup }{}}\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid {\mid }_1\underset{1\le j\le p}{\overset{\max }{}}\mid {\mathbb{E}}_n\left\{{∊}_i{X}_{ij}{N}_i\right(\tau \left)\right\}\mid \right]\le 2\frac{c}{K\sqrt{S}}\sqrt{S}\mathrm{E}\left[\underset{1\le j\le p}{\overset{\max }{}}\mid {\mathbb{E}}_n\left\{{∊}_i{X}_{ij}{N}_i\right(\tau \left)\right\}\mid \right]\le 2a_{n}c∕K, \end{gathered}$$

where the first inequality follows from Lemma 2.3.1 in‘[31], the second inequality is trivial, and the third inequality follows from Condition (B) and Lemma A.1 in [30]. Applying Bousquet’s concentration theorem [2] yields that, for any r > 0,

$$\mathrm{Pr}\left[\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(c\right)}{\overset{\sup }{}}I\le 2c{a}_n∕K+r{a}_n\sqrt{2\{c^2∕(K^{2}s)+2c^2{a}_n∕K\}}+2r^2{a}_n^{2}c∕3\right]\le \exp (-n{r}^2{a}_n^2).$$

Choose $r=2\sqrt{\rho }$. As $\rho \sqrt{\ln (p∕n)}\to 0$, we obtain that, when n is sufficiently large,

$$\mathrm{Pr}\left(\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(c\right)}{\overset{\sup }{}}I\ge 2\frac{c}{K}{a}_n+3\frac{c\sqrt{\rho }}{K\sqrt{s}}{a}_n\right)\le \exp \left[-4n\rho {\left\{\sqrt{2K^2\ln \left(2p\right)∕n}+K\ln \left(2p\right)∕n\right\}}^2\right]\le \exp (-8K^2\rho \ln p).$$ (A.2)

For term II, let ${\mathcal{R}}_s\left(c\right)$ denote a ball with dimensionality s and radius $c∕\left(K\sqrt{s}\right)$. Then ${\mathcal{B}}_s^0\left(c\right)={\mathcal{R}}_s\left(c\right)+{\beta }_s^{\ast }$. Let ${\mathcal{C}}_s=\left\{\mathcal{C}\right({\xi }_{\ell }\left)\right\}$ be a collection of cubes that cover the ball R_s(c), where $\mathcal{C}\left({\xi }_{\ell }\right)$ is a cube containing ξ_ΐ with sides of length $c∕\left(K\sqrt{s}{n}^2\right)$. Then $\mid \mathcal{C}{\mid }_s\le (4n^2{)}^2$ and $\mid \mid {\xi }_{\ell }\mid \mid \le C∕\left(K\sqrt{s}\right)$. For any $\xi \in \mathcal{C}\left({\xi }_{\ell }\right),\mid \mid \xi -{\xi }_{\ell }\mid \mid \le c∕\left(K{n}^2\right)\equiv {\zeta }_n$, ||ξ−ξ_ℓ|| ≤ c/(Kn²) ≡ ζ_n Let $T_S\left(\xi \right)={\mathbb{E}}_n{\int }_0^{\tau }\ln r_s^{\left(0\right)}({\beta }_S^{\ast }+\mathit{xi},t)d{N}_i\left(t\right)$. By the Mean Value Theorem,

$$\mid {\int }_0^{\tau }\ln r_s^{\left(0\right)}({\beta }_s^{\ast }+{\xi }_{\ell },t)d{N}_i\left(t\right)-{\int }_0^{\tau }\ln r_s^{\left(0\right)}({\beta }_s^{\ast },t)d{N}_i\left(t\right)\mid =\mid {\int }_0^{\tau }\frac{1}{r_s^{\left(0\right)}({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell },t)}\mathrm{E}\left[\stackrel{‒}{Y}\right(t\left)\exp \right\{({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell }{)}^{⊺}{\mathbf{X}}_s\}{\xi }_{\ell }^{⊺}{\mathbf{X}}_s\left]d{N}_i\right(t)\mid \le \mid {\int }_0^{\tau }\frac{1}{r_s^{\left(0\right)}({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell },t)}\mathrm{E}\left[\stackrel{‒}{Y}\right(t\left)\exp \right\{({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell }{)}^{⊺}{\mathbf{X}}_s\}]\mid \mid {\xi }_{\ell }\mid {\mid }_1\mid \mid \mathbf{X}\mid {{\mid }_{\infty }}_{d{N}_i}(t)\mid \le \mid {\int }_0^{\tau }\frac{1}{r_s^{\left(0\right)}({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell },t)}{r}_s^{\left(0\right)}({\beta }_s^{\ast }+{\tilde{\xi }}_{\ell },t)d{N}_i\left(t\right)\mid \mid \mid {\mathit{xi}}_{\ell }\mid {\mid }_1\mid \mid \mathbf{X}\mid {\mid }_{\infty }\le \tau \mathcal{C}∕\left(K\sqrt{s}\right)\sqrt{s}K=\mathcal{C}\tau ,$$

where ${\tilde{\xi }}_{\ell }$ is between ξ_ι and 0. Applying Bernstein’s inequality yields that for any r > 0,

$$\mathrm{Pr}[n\mid T_s({\xi }_{\ell })-T_s(0)-\mathrm{E}\{T_s\left({\xi }_{\ell }\right)-T_s\left(0\right)\}\mid \ge r]\ge 2\exp \left(-\frac{1}{2}\frac{r^2}{n{c}^2{\tau }^2+2\mathcal{c}\tau r∕3}\right),$$

and consequently, by choosing $r=6\mathcal{c}\tau \sqrt{n\rho \ln p}$,

$$\begin{gathered} \mathrm{Pr}\left[\underset{1\le \ell \le (4n^2{)}^s}{\overset{\max }{}}\mid T_s\left({\xi }_{\ell }\right)-T_s\left(0\right)-\mathrm{E}\left\{T_s\right({\xi }_{\ell })-T_s(0\left)\right\}\mid >6\mathcal{c}\tau \sqrt{n\rho \ln p}∕n\right] \\ \le 2(4n^2{)}^s\exp \left(-\frac{1}{2}\frac{36{\mathcal{c}}^2{\tau }^{2}n\rho \ln p}{n{\mathcal{c}}^2{\tau }^2+12{\mathcal{c}}^2{\tau }^2\sqrt{n\rho \ln p}}\right) \\ \le 2(4n^2{)}^s\exp (-12\rho \ln p)\le 2\exp (-12\rho \ln p+s\ln 4+2s\ln n))\le 2\exp (-10\rho \ln p), \end{gathered}$$ (A.3)

where n is sufficiently large.

Given any $\xi \in \mathcal{C}\left({\xi }_{\ell }\right)$, we can similarly show that

$$\mid {\int }_0^{\tau }\left\{\ln r_s^{\left(0\right)}\right({\beta }_s^{\ast }+\xi ,t)-\ln r_s^{\left(0\right)}({\beta }_s^{\ast }+{\xi }_{\ell },t\left)\right\}d{N}_i\left(t\right)\mid \le K\sqrt{s}{\zeta }_n.$$

Therefore,

$$\mid T_s\left(\xi \right)-T_s\left({\xi }_{\ell }\right)-\mathrm{E}\left\{T_s\right(\xi )-T_s({\xi }_{\ell }\left)\right\}\mid \le 2K\sqrt{s}{\zeta }_n\le 2K\sqrt{\rho }{\zeta }_n.$$ (A.4)

Combining (A.3) and (A.4) implies that

$$\mathrm{Pr}\left(\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(\mathcal{c}\right)}{\overset{\sup }{}}II\ge 6\mathcal{c}\tau \sqrt{\rho \ln (p∕n)}+2K\sqrt{\rho }{\zeta }_n\right)\le 2\exp (-10\rho \ln p).$$ (A.5)

By (A.2) and (A.5), we have

$$\mathrm{Pr}\left(\underset{{\beta }_s\in {\mathcal{B}}_s^0\left(\mathcal{c}\right)}{\overset{\sup }{}}{Z}_s\left({\beta }_s\right)\ge 2\frac{\mathcal{c}}{K}{a}_n+3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}{a}_n+6\mathcal{c}\tau \sqrt{\rho \ln (p∕n)}+2K\sqrt{\rho }{\zeta }_n\right)\le \exp (-8K^2\rho \ln p)+2\exp (-10\rho \ln p).$$

By the combinatoric inequality $\left(\begin{array}{c}\hfill p\hfill \\ \hfill s\hfill \end{array}\right)\le (ep∕s{)}^s$, we obtain that

$$\begin{gathered} \mathrm{Pr}\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(\mathcal{c}\right)}{\overset{\sup }{}}{Z}_s\left({\beta }_s\right)\ge 2\frac{\mathcal{c}}{K}{a}_n+3\frac{c\sqrt{\rho }}{K\sqrt{s}}{a}_n+6\mathcal{c}\tau \sqrt{\rho \ln (p∕n)}+2K\sqrt{\rho }{\zeta }_n\right\} \\ \le \underset{s=1}{\overset{\rho }{\Sigma }}(ep∕s{)}^s\{\exp (-8K^2\rho \ln p)+2\exp (-10\rho \ln p)\}\le 3\exp (-6\rho \ln p). \end{gathered}$$

This completes the proof of Lemma D. □

Proof of Lemma E. We first define the following events:

$${\Omega }_1=\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(\mathcal{c}\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid R_s^{\left(0\right)}({\beta }_s,u)-r_s^{\left(0\right)}({\beta }_s,u)\mid \ge A_6\sqrt{\rho \ln (p∕n)}\right\},$$

$${\Omega }_2=\left\{\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^1\left(\mathcal{c}\right),u\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_s^{\left(1\right)}({\beta }_s,u)-r_s^{\left(1\right)}({\beta }_s,u)\mid \mid \ge A_5\sqrt{{\rho }^2\ln (p∕n)}\right\}.$$

By Lemma C, we obtain that Pr (Ω₁) ≤ exp(−3ρ ln p) and Pr (Ω₂) ≤ exp(−3ρ ln p). In the rest of the proof, we restrict our attention to ${\Omega }_1^{\mathrm{C}}\cap {\Omega }_2^{\mathrm{C}}$. By the arguments in [20], we can show that

$$\begin{gathered} D_S\left({\beta }_S\right)\le {\mathbb{E}}_n\mid {\int }_0^{\tau }\left[\ln \left\{\frac{R_S^{\left(0\right)}({\beta }_S,t)}{r_S^{\left(0\right)}({\beta }_S,t)}\right\}-\ln \left\{\frac{R_S^{\left(0\right)}({\beta }_S^{\ast },t)}{r_S^{\left(0\right)}({\beta }_S^{\ast },t)}\right\}\right]d{N}_i\left(t\right)\mid \le \underset{0\le t\le \tau }{\overset{\sup }{}}\mid \ln \left\{\frac{R_S^{\left(0\right)}({\beta }_S,t)}{r_S^{\left(0\right)}({\beta }_S,t)}\right\}-\ln \left\{\frac{R_S^{\left(0\right)}({\beta }_S^{\ast },t)}{r_S^{\left(0\right)}({\beta }_S^{\ast },t)}\right\}\mid \\ \le \underset{0\le t\le \tau }{\overset{\sup }{}}\mid \frac{r_S^{\left(0\right)}({\tilde{\beta }}_S,t)}{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)}({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\left[\frac{R_S^{\left(1\right)}({\tilde{\beta }}_S,t)}{r_S^{\left(0\right)}({\tilde{\beta }}_S,t)}-\frac{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)r_S^{\left(1\right)}({\tilde{\beta }}_S,t)}{{\left\{r_S^{\left(0\right)}({\tilde{\beta }}_S,t)\right\}}^2}\right]\mid \\ =\underset{0\le t\le \tau }{\overset{\sup }{}}\mid ({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\frac{R_S^{\left(1\right)}({\tilde{\beta }}_S,t)-r_S^{\left(1\right)}({\tilde{\beta }}_S,t)}{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)}\mid +\underset{0\le t\le \tau }{\overset{\sup }{}}\mid \{{\beta }_S-{\beta }_S^{\ast }{)}^{⊺}{r}_S^{\left(1\right)}({\tilde{\beta }}_S,t)\left\{\frac{1}{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)}-\frac{1}{r_S^{\left(0\right)}({\tilde{\beta }}_S,t)}\right\}\mid \\ \equiv I+II. \end{gathered}$$

We first consider I. By Condition (B),

$$\begin{gathered} r_S^{\left(0\right)}({\beta }_S,t)=\mathrm{E}\left\{1\right(Y\ge t\left)\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S\left)\right\}\ge \mathrm{E}\left\{1\right(Y\ge t\left)\exp \right(-\mid \mid {\beta }_S\mid {\mid }_1\mid \mid {\mathbf{X}}_S\mid {\mid }_{\infty }\left)\right\} \\ \ge \mathrm{E}\left[1\right(Y\ge t\left)\exp \right\{-(\mathcal{c}+KL)\left\}\right]\ge \omega \exp \{-(\mathcal{c}+KL\left)\right\}, \end{gathered}$$

for any ${\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)$. We obtain that

$$\underset{\mid S\mid \le \rho ,{\beta }_s\in {\mathcal{B}}_s^0\left(\mathcal{c}\right),t\in [0,\tau ]}{\overset{\inf }{}}{r}_S^{\left(0\right)}({\beta }_S,t)\ge \omega \exp \{-(\mathcal{c}+KL\left)\right\}.$$

Then,

$$\begin{gathered} I\le \underset{0\le t\le \tau }{\overset{\sup }{}}\mid \frac{1}{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)}\mid \mid ({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\{R_S^{\left(1\right)}({\tilde{\beta }}_S,t)-r_S^{\left(1\right)}({\tilde{\beta }}_S,t)\}\mid \\ \le \frac{\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid \mid }{{\inf }_{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right),t\in [0,\tau ]}{r}_S^{\left(0\right)}({\beta }_S,t)-A_6\sqrt{\rho \ln (p∕n)}}\underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right),t\in [0,\tau ]}{\overset{\sup }{}}\mid \mid R_S^{\left(1\right)}({\tilde{\beta }}_S,t)-r_S^{\left(1\right)}({\tilde{\beta }}_S,t)\mid \mid \\ \le \frac{1}{\omega \exp \{-(\mathcal{c}+KL\left)\right\}-A_6\sqrt{\rho \ln (p∕n)}}\times \frac{\mathcal{c}}{K\sqrt{s}}\times A_5\sqrt{{\rho }^2\ln (p∕n)}\le A_8\sqrt{{\rho }^2\ln (p∕n)}, \end{gathered}$$

for some constant A₈ that does not depend on n.

We now bound II. By Condition (B),

$$\mathrm{E}\left\{1\right(Y\ge t\left)X_j\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S\left)\right\}\le K\mathrm{E}\left\{\exp \right({\beta }_S^{⊺}{\mathbf{X}}_S\left)\right\}\le K\exp (\mid \mid {\mathbf{X}}_S\mid {\mid }_{\infty }\mid \mid {\beta }_S\mid {\mid }_1)\le K\exp (\mathcal{c}+KL).$$

Thus,

$$\begin{gathered} II\le \underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right),t\in [0,\tau ]}{\overset{\sup }{}}\mid \mid r_S^{\left(1\right)}({\beta }_S,t)\mid \mid \underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right),t\in [0,\tau ]}{}\mid \frac{1}{R_S^{\left(0\right)}({\tilde{\beta }}_S,t)r_S^{\left(0\right)}({\tilde{\beta }}_S,t)}\mid \\ \times \underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right),t\in [0,\tau ]}{\overset{\sup }{}}\mid R_S^{\left(0\right)}({\beta }_S,t)-r_S^{\left(0\right)}({\beta }_S,t)\mid \mid \mid {\beta }_S-{\beta }_S^{\ast }t\mid \mid \\ \le \frac{K\sqrt{\rho }\exp (\mathcal{c}+KL)}{\omega \exp \{-(\mathcal{c}+KL\left)\right\}\left[\omega \exp \{-(\mathcal{c}+KL\left)\right\}-A_6\sqrt{\rho \ln (p∕n)}\right]}\times \frac{A_6\sqrt{\rho \ln (p∕n)}\times \mathcal{c}}{K\sqrt{s}}\le A_9\sqrt{{\rho }^2\ln (p∕n)}, \end{gathered}$$

for some constant A₉ that is free of n.

Withdrawing the restriction to ${\Omega }_1^{\mathrm{C}}\cap {\Omega }_2^{\mathrm{C}}$, the above results indicate that

$$\mathrm{Pr}\left\{\underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)}{\overset{\sup }{}}{D}_S\left({\beta }_S\right)\ge A_{10}\sqrt{{\rho }^2\ln (p∕n)}\right\}\le 2\exp (-3\rho \ln p),$$

for some constant A₁₀. This completes the proof of Lemma E. □

Proof of Lemma F: Given any index set S satisfying $\mid S\mid \le \rho$ and ${\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)$, by Taylor’s expansion,

$$\begin{gathered} {\Gamma }_S\left({\beta }_S\right)=\mathrm{E}\left[-{\int }_0^{\tau }\{{\beta }_S^{⊺}{\mathbf{X}}_S-\ln r_S^{\left(0\right)}({\beta }_S,t\left)\right\}dN\left(t\right)\right] \\ =\mathrm{E}\left[-{\int }_0^{\tau }\{{\beta }_S^{\ast ⊺}{\mathbf{X}}_S-\ln r_S^{\left(0\right)}({\beta }_S^{\ast },t\left)\right\}dN\left(t\right)\right]+\mathrm{E}\left[-{\int }_0^{\tau }\{{\mathbf{X}}_S^{⊺}-r_S^{\left(1\right)}({\beta }_S^{\ast },t)∕r_S^{\left(0\right)}({\beta }_S^{\ast },t\left)\right\}dN\left(t\right)\right]({\beta }_S-{\beta }_S^{\ast }) \\ +\frac{1}{2}({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\mathrm{E}\left[{\int }_0^{\tau }\left[r_S^{\left(2\right)}\right({\tilde{\beta }}_S,t)∕r_S^{\left(0\right)}({\tilde{\beta }}_S,t)-\{r_S^{\left(1\right)}({\tilde{\beta }}_S,t){\}}^{\otimes 2}∕\left\{r_S^{\left(0\right)}\right({\tilde{\beta }}_S,t\left){\}}^2\right]dN\left(t\right)\right]({\beta }_S-{\beta }_S^{\ast }), \end{gathered}$$

where ${\tilde{\beta }}_S$ is between β_S and ${\beta }_S^{\ast }$. Noting that

$$\mathrm{E}\left[-{\int }_0^{\tau }\{{\mathbf{X}}_S^{⊺}-r_S^{\left(1\right)}({\beta }_S^{\ast },t)∕r_S^{\left(0\right)}({\beta }_S^{\ast },t\left)\right\}dN\left(t\right)\right]=0,$$

by Condition (F), we have ${\Gamma }_S\left({\beta }_S\right)-{\Gamma }_S\left({\beta }_S^{\ast }\right)\ge {\kappa }_{\min }\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid {\mid }^2∕2$. Similarly,

$$\begin{gathered} {\Gamma }_S\left({\beta }_S\right)-{\Gamma }_S\left({\beta }_S^{\ast }\right)=\frac{1}{2}({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\mathrm{E}\left[{\int }_0^{\tau }\left[r_S^{\left(2\right)}\right({\tilde{\beta }}_S,t)∕r_S^{\left(0\right)}({\tilde{\beta }}_S,t)-\{r_S^{\left(1\right)}({\tilde{\beta }}_S,t){\}}^{\otimes 2}∕\left\{r_S^{\left(0\right)}\right({\tilde{\beta }}_S,t\left){\}}^2\right]dN\left(t\right)\right]({\beta }_S-{\beta }_S^{\ast }) \\ \le \frac{1}{2}\left[{\int }_0^{\tau }({\beta }_S-{\beta }_S^{\ast }{)}^{⊺}\{r_S^{\left(2\right)}({\tilde{\beta }}_S,t)∕r_S^{\left(0\right)}(\tilde{\beta },t)\left\}\right({\beta }_S-{\beta }_S^{\ast }\left)dN\right(t)\right]\le \frac{1}{2}{\kappa }_{\max }\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid {\mid }^2, \end{gathered}$$

where the last inequality follows from Condition (F). This completes the proof of Lemma F. □

Proof of Lemma G. Let

$${\Omega }_3=\left\{\underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)}{\overset{\sup }{}}{Z}_S\left({\beta }_S\right)\ge 2\frac{\mathcal{c}}{K}{a}_n+3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}{a}_n+6\mathcal{c}\tau \sqrt{\rho \ln (p∕n)}+2K\sqrt{\rho }{\zeta }_n\right\},{\Omega }_4=\left\{\underset{\mid S\mid \le \rho ,{\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)}{\overset{\sup }{}}{D}_S\left({\beta }_S\right)\le A_{10}\sqrt{{\rho }^2\ln (p∕n)}\right\}.$$

We consider ${\Omega }_3^{\mathrm{C}}\cap {\Omega }_4^{\mathrm{C}}$, which holds with probability at least 1 – 5 exp(−3ρ ln p), by Lemmas D and E. In the rest of the proof, we restrict our attention to ${\Omega }_3^{\mathrm{C}}\cap {\Omega }_4^{\mathrm{C}}$.

Given an index set S such that |S| ≤ ρ, for any β_S with $\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid \mid =A_{11}\left\{{\rho }^2\ln \right(p∕n){\}}^{1∕4}$ for some constant A₁₁ defined later, we have ${\beta }_S\in {\mathcal{B}}_S^0\left(\mathcal{c}\right)$ when n is sufficiently large such that $A_{11}\left\{{\rho }^2\ln \right(p∕n){\}}^{1∕4}\le \mathcal{c}∕(K\sqrt{\rho })$, since ρ⁴ ln(p/n) → 0.

Therefore, ${\beta }_S\in \partial {\mathcal{B}}_S^0\left(\mathcal{c}\right)$, where $\partial {\mathcal{B}}_S^0\left(\mathcal{c}\right)$ denotes the boundary of ${\mathcal{B}}_S^0\left(\mathcal{c}\right)$.

Noting that $\tilde{\ell }\left({\beta }_S\right)∕n=-{\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_S;{\mathbf{X}}_i,Y_i,{\delta }_i\left)\right\}$, we observe that

$$n^{-1}\{-{\ell }_S({\beta }_S)+{\ell }_S({\beta }_S^{\ast }\left)\right\}=n^{-1}\{-{\ell }_S({\beta }_S)+{\ell }_S({\beta }_S^{\ast }\left)\right\}-n^{-1}\{-{\tilde{\ell }}_S({\beta }_S)+{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}+n^{-1}\{-{\tilde{\ell }}_S({\beta }_S)+{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}-\left\{{\Gamma }_S\right({\beta }_S)-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}+\left\{{\Gamma }_S\right({\beta }_S)-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\ge \left\{{\Gamma }_S\right({\beta }_S)-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}-n^{-1}\mid \left\{{\ell }_S\right({\beta }_S)-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\left\{{\tilde{\ell }}_S\right({\beta }_S)-{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}\mid -\mid n^{-1}\{-{\tilde{\ell }}_S({\beta }_S)+{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}-\left\{{\Gamma }_S\right({\beta }_S)-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\mid =\left\{{\Gamma }_S\right({\beta }_S)-{\Gamma }_S({\beta }_S^{\ast }\left)\right\}-\mid D_S\left({\beta }_S\right)\mid -\mid Z_S\left({\beta }_S\right)\mid$$

and that the right-hand term can be bounded from below by

$$\frac{1}{2}{\kappa }_{\min }\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid {\mid }^2-2\frac{\mathcal{c}}{K}{a}_n-3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}{a}_n-6\mathcal{c}\tau \sqrt{\rho \ln (p∕n)}-2K\sqrt{\rho }{\zeta }_n-A_{10}\sqrt{{\rho }^2\ln (p∕n)}\ge \frac{1}{2}{\kappa }_{\min }{A}_{11}^2\frac{\sqrt{{\rho }^2\ln p}}{\sqrt{n}}-2\frac{\mathcal{c}}{K}\frac{\sqrt{2K^2\ln \left(2p\right)}}{\sqrt{n}}-2\frac{\mathcal{c}}{K}K\frac{\ln \left(2p\right)}{n}-3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}\frac{\sqrt{2K^2\ln \left(2p\right)}}{\sqrt{n}}-3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}K\frac{\ln \left(2p\right)}{n}-6\mathcal{c}\tau \frac{\sqrt{\rho \ln p}}{\sqrt{n}}-2K\sqrt{\rho }\frac{\mathcal{c}}{K{n}^2}-A_{10}\frac{\sqrt{{\rho }^2\ln p}}{\sqrt{n}}>0,$$

with some constant A₁₁ satisfying ${\kappa }_{\min }{A}_{11}^2>A_{10}$. Therefore,

$$\underset{\mid \mid {\beta }_S-{\beta }_S^{\ast }\mid \mid =A_{11}\left({\rho }^2\ln \right(p∕n){)}^{1∕4}}{\overset{\inf }{}}{n}^{-1}\{-{\ell }_S({\beta }_S)+{\ell }_S({\beta }_S^{\ast }\left)\right\}>0.$$

By the concavity of ${\ell }_S,\mid \mid {\widehat{\beta }}_S-{\beta }_S^{\ast }\mid \mid \le A_{11}\left\{{\rho }^2\ln \right(p∕n){\}}^{1∕4}$. Withdrawing the restriction to ${\Omega }_3^{\mathrm{C}}\cap {\Omega }_4^{\mathrm{C}}$, we have

$$\mathrm{Pr}\left[\underset{\mid S\mid \le \rho }{\overset{\sup }{}}\mid \mid {\widehat{\beta }}_S-{\beta }_S^{\ast }\mid \mid \le A_{11}\left\{{\rho }^2\ln \right(p∕n){\}}^{1∕4}\right]\ge 1-5\exp (-3\rho \ln p).$$

Furthermore,

$$\mid {\ell }_S\left({\widehat{\beta }}_S\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\mid \le \frac{1}{2}{\kappa }_{\max }\frac{\sqrt{{\rho }^2\ln p}}{\sqrt{n}}+2\frac{\mathcal{c}}{K}\frac{\sqrt{2K^2\ln \left(2p\right)}}{\sqrt{n}}+2\frac{\mathcal{c}}{K}K\frac{\ln \left(2p\right)}{n}+3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{S}}\frac{\sqrt{2K^2\ln \left(2p\right)}}{\sqrt{n}}+3\frac{\mathcal{c}\sqrt{\rho }}{K\sqrt{s}}K\frac{\ln \left(2p\right)}{n}+6\mathcal{c}\tau \frac{\sqrt{\rho \ln p}}{\sqrt{n}}+2K\sqrt{\rho }\frac{\mathcal{c}}{K{n}^2}+A_{10}\frac{\sqrt{{\rho }^2\ln p}}{\sqrt{n}}\le A_{12}\frac{\sqrt{{\rho }^2\ln p}}{\sqrt{n}},$$

for some constant A₁₂. Similarly, we obtain that

$$\mathrm{Pr}\left\{\underset{\mid S\mid \le \rho }{\overset{\sup }{}}\mid {\ell }_S\left({\widehat{\beta }}_S\right)-{\ell }_S\left({\beta }_S^{\ast }\right)\mid \le A_{12}\sqrt{{\rho }^2\ln (p∕n)}\right\}\ge 1-5\exp (-3\rho \ln p).$$

This complete the proof of Lemma G. □ Proof of Lemma H. Given any S ⊂ (1,…,p} and r such that |S| < ρ, r ∈ S^C,

$$n^{-1}\left\{{\ell }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}=I+II,$$

where

$$I=n^{-1}\left\{{\ell }_{S_{+r}}\right\}\left({\beta }_{S_{+r}}^{\ast }\right)-{\ell }_S\left({\beta }_S^{\ast }\right\}-n^{-1}\left\{{\tilde{\ell }}_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\},II=n^{-1}\left\{{\tilde{\ell }}_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\tilde{\ell }}_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}.$$

Noting that $n^{-1}{\tilde{\ell }}_S\left({\beta }_S\right)=-{\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_S;{\mathbf{X}}_i,Y_i,{\delta }_i\left)\right\}$, it is easy to check that

$$\mid I\mid \le \mid {\mathbb{E}}_n\left[{\int }_0^{\tau }\left\{\ln R_S^{\left(0\right)}\right({\beta }_S^{\ast },u)-\ln r_S^{\left(0\right)}({\beta }_S^{\ast },u\left)\right\}d{N}_i\left(u\right)\right]\mid +\mid {\mathbb{E}}_n\left[{\int }_0^{\tau }\left\{\ln R_S^{\left(0\right)}\right({\beta }_{S_{+r}}^{\ast },u)-\ln r_S^{\left(0\right)}({\beta }_{S_{+r}}^{\ast },u\left)\right\}d{N}_i\left(u\right)\right]\mid \equiv I_1+I_2,\mid II\mid =\mid \left[{\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_{S_{+r}}^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i\left)\right\}-{\Gamma }_S\left({\beta }_{S_{+r}}^{\ast }\right)\right]-\left[{\mathbb{E}}_n\right\{{\gamma }_S({\beta }_S^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i)\}-{\Gamma }_S({\beta }_S^{\ast }\left)\right]\mid \le \mid {\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_{S_{+r}}^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i\left)\right\}-{\Gamma }_S\left({\beta }_{S_{+r}}^{\ast }\right)\mid +\mid {\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_S^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i\left)\right\}-{\Gamma }_S\left({\beta }_S^{\ast }\right)\mid \equiv I{I}_1+I{I}_2.$$

We restrict our attention to ${\Omega }_1^{\mathrm{C}}$, where Ω₁ is defined in Lemma E. We consider I₁ first. We have

$$\mid I_1\mid =\mid {\mathbb{E}}_n\left[{\int }_0^{\tau }\left\{\ln R_S^{\left(0\right)}\right({\beta }_S^{\ast },u)-\ln r_S^{\left(0\right)}({\beta }_S^{\ast },u\left)\right\}d{N}_i\left(u\right)\right]\mid \le \underset{0\le t\le \tau }{\overset{\sup }{}}\mid \ln R_S^{\left(0\right)}({\beta }_S^{\ast },t)∕r_S^{\left(0\right)}({\beta }_S^{\ast },t)\mid =\underset{0\le t\le \tau }{\overset{\sup }{}}\mid \left\{R_S^{\left(0\right)}\right({\beta }_S^{\ast },t)-r_S^{\left(0\right)}({\beta }_S^{\ast },t\left)\right\}∕\tilde{r}\left(t\right)\mid \le \underset{0\le t\le \tau }{\overset{\sup }{}}\mid \frac{A_6\sqrt{\rho \ln (p∕n)}}{r_S^{\left(0\right)}({\beta }_S^{\ast },t)-A_6\sqrt{\rho \ln (p∕n)}}\mid \le \frac{A_6\sqrt{\rho \ln (p∕n)}}{\omega \exp \{-(\mathcal{c}+KL\left)\right\}-A_6\sqrt{\rho \ln (p∕n)}}\le A_{13}\sqrt{\rho \ln (p∕n)},$$

where $\tilde{r}\left(t\right)$ is between $R_S^{\left(0\right)}({\beta }_S^{\ast },t)$ and $r_S^{\left(0\right)}({\beta }_S^{\ast },t)$. By the same argument, $I_2\le A_{13}\sqrt{\rho \ln (p∕n)}$ as well. Therefore, $\mid I\mid \le 2A_{13}\sqrt{\rho \ln (p∕n)}$. By Conditions (B) and (D), $\mid {\gamma }_S({\beta }_S^{\ast };{\mathbf{X}}_i,Y_i,{\delta }_i)\mid \le \mid {\beta }_S^{\ast ⊺}{\mathbf{X}}_i\mid +\ln r_S^{\left(0\right)}({\beta }_S^{\ast },t)\le 2KL$. Applying Bernstein’s inequality, we get

$$\mathrm{Pr}\left[\mid {\mathbb{E}}_n\left\{{\gamma }_S\right({\beta }_S^{\ast };{\mathbf{X}}_{i}i,Y_i,{\delta }_i\left)\right\}-{\Gamma }_S\left({\beta }_S^{\ast }\right)\mid >6KL\sqrt{\rho \ln (p∕n)}\right]\le 2\exp \left(-\frac{1}{2}\frac{36K^2{L}^{2}n\rho \ln p}{4n{K}^2{L}^2+12K^2{L}^2\sqrt{\rho n\ln p}∕3}\right)\le 2\exp (-4\rho \ln p).$$

Thus,

$$\mathrm{Pr}\left\{\underset{\mid S\mid <\rho ,r\in S^{\mathrm{C}}}{\overset{\sup }{}}\mid II\mid >12KL\sqrt{\rho \ln (p∕n)}\right)\le \underset{\mid S\mid \le \rho }{\Sigma }\underset{r=1}{\overset{p}{\Sigma }}2\exp (-4\rho \ln p)\le p\underset{s=1}{\overset{\rho -1}{\Sigma }}(ep∕s{)}^{s}2\exp (-4\rho \ln p)\le 2\exp (-3\rho \ln p).$$

Withdrawing the restriction to ${\Omega }_1^{\mathrm{C}}$, we obtain that

$$\mathrm{Pr}\left[\underset{\mid S\mid <\rho ,r\in S^{\mathrm{C}}}{\overset{\sup }{}}\mid n^{-1}\left\{{\ell }_{S_{+r}}\right({\beta }_{S_{+r}}^{\ast })-{\ell }_S({\beta }_S^{\ast }\left)\right\}-\{-{\Gamma }_{S_{+r}}({\beta }_{S_{+r}}^{\ast })+{\Gamma }_S({\beta }_S^{\ast }\left)\right\}\mid \ge 2A_{13}\sqrt{\rho \ln (p∕n)}+12KL\sqrt{\rho \ln (p∕n)}\right]\le 2\exp (-3\rho \ln p)+\exp (-3\rho \ln p)\le 3\exp (-3\rho \ln p).$$

Let A₁₄ = 2A₁₃ + 12KL and we complete the proof of Lemma H. □