Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates

Abstract

The Fine-Gray proportional sub-distribution hazards (PSH) model is among the most popular regression model for competing risks time-to-event data. This article develops a fast safe feature elimination method, named PSH-SAFE, for fitting the penalized Fine-Gray PSH model with a Lasso (or adaptive Lasso) penalty. Our PSH-SAFE procedure is straightforward to implement, fast, and scales well to ultrahigh dimensional data. We also show that as a feature screening procedure, PSH-SAFE is safe in a sense that the eliminated features are guaranteed to be inactive features in the original Lasso (or adaptive Lasso) estimator for the penalized PSH model. We evaluate the performance of the PSH-SAFE procedure in terms of computational efficiency, screening efficiency and safety, run-time, and prediction accuracy on multiple simulated datasets and a real bladder cancer data. Our empirical results show that the PSH-SAFE procedure possesses desirable screening efficiency and safety properties and can offer substantially improved computational efficiency as well as similar or better prediction performance in comparison to their baseline competitors.

1 |. INTRODUCTION

Lasso penalization¹ is among the most widely used methodology for high dimensional problems. However, large-scale and high dimensional data can pose substantial computational challenges to solving a Lasso-type penalization problem because its computational cost is in the order of np².² To improve computational efficiency, a safe feature elimination (SAFE) algorithm has been recently developed for large scale Lasso-type problems.³ Briefly speaking, SAFE serves as a screening procedure to remove features that are guaranteed to be inactive with zero coefficients in the original Lasso solution. Hence preceding Lasso with SAFE screening can effectively reduce the data dimension and the computational complexity. Due to its superb numerical performance and desirable theoretical properties, SAFE screening has been widely applied to classification and regression problems.^4–7

The purpose of this article is to develop SAFE algorithms for the Lasso or adaptive Lasso penalized Fine-Gray proportional subdistribution hazards (PSH) model⁸ for competing risks survival data with ultrahigh dimensional covariates. Competing risks data arise commonly in many applications when individuals may fail from multiple causes and the occurrence of one failure event precludes the others from happening.^8–11 The Fine-Gray PSH model directly models the impact of covariates on the marginal probability of failure for a specific cause, namely, the cumulative incidence function (CIF) or subdistribution, and has been commonly used for competing risks data. However, despite of the rich literature on high dimensional methods for the PSH model,^12–18 to the best of our knowledge, no SAFE procedure has been developed for the PSH model.

In this article, we derive a fast safe feature elimination method, named PSH-SAFE, for the Fine-Gray PSH model combined with the Lasso and adaptive Lasso penalties, respectively. The detailed PSH-SAFE rules are described later in (9) and (11). Our PSH-SAFE procedure is straightforward to implement, fast, and scales well to ultrahigh dimensional data. We also show rigorously that as a feature screening procedure, PSH-SAFE is safe in a sense that the eliminated features are guaranteed to be inactive features in the original Lasso-PSH (or adaptive Lasso-PSH) estimator. We conduct extensive simulations to evaluate the performance of the PSH-SAFE procedure in terms of computational efficiency, screening efficiency and safety, and prediction accuracy in multiple scenarios. Our empirical results demonstrate that the PSH-SAFE procedure possesses desirable screening efficiency and safety properties and can offer substantially improved computational efficiency as well as similar or better prediction performance in comparison to their baseline competitors.

The rest of this article is organized as follows. In Section 2.1, we review the PSH model⁸ and the pseudo-partial likelihood estimation method. In Section 2.2, we derive the SAFE screening rules for both the Lasso and adaptive Lasso PSH models, with theoretical guarantees. In Section 3, the performance of the proposed method is demonstrated using extensive simulations and a publicly available high-dimensional bladder cancer dataset. Concluding remarks are provided in Section 4.

2 |. SAFE SCREENING FOR PENALIZED PROPORTIONAL SUBDISTRIBUTION HAZARDS MODELS

2.1 |. Preliminaries

Without loss of generality, we assume that there are two causes of failure for the competing risks outcome, where cause 1 is the event of interest and cause 2 is a competing risk. A competing risks data consists of n iid observations $\left\{\left(y_i,{\delta }_i,{\delta }_i{\epsilon }_i,X_i\right),i=1,\dots ,n\right\}$, where y_i = min(T_i, C_i) is the observed time, T_i, C_i and ε_i ∈ {1, 2} denote the failure time, the censoring time, and the failure type, δ_i = I(T_i ≤ C_i) is the censoring indicator, and X_i is a vector of p covariates for subject i. Denote the feature matrix by $X=\left[x_1,x_2,\dots ,x_p\right]\in {ℝ}^{n\times p}$. Hence, $X_i^T$ denotes the ith row of X and $x_i\in {ℝ}^p$ be the jth column of X.

A fundamental quantity in competing risks problems is the CIF for each competing event. The CIF of event type i is

$$F_i\left(t|X\right)=P\left(T\le t,\epsilon =i|X\right)={\int }_0^t{H}_i\left(t|X\right)S\left(u|X\right)du,$$

where H_i(t|X) is the cause-specific hazard (CSH) and S(t|X) is the event-free survival. With Fine-Gray’s PSH model, one can directly estimate the impact of the covariates on the hazard of the CIF without estimating the individual CSH for the different failure type. The PSH model based on the subdistribution hazard for cause 1 is defined as

$$\begin{gathered} h_1\left(t|X\right)=d{F}_1\left(t|X\right)/\left\{1-F_1\left(t|X\right)\right\} \\ =\underset{\Delta t\to 0}{\lim }\frac{1}{\Delta t}Pr\left\{t\le T\le t+\Delta t,\epsilon =1|T\ge t\cup \left(T\le t\cap \epsilon \ne 1\right)|X\right\}. \end{gathered}$$ (1)

The subdistribution hazard of cause 1 is assumed to follow a proportional hazard model, $h_1\left(t|X\right)=h_{10}\left(t\right)\exp \left({\beta }^{T}X\right)$, where h₁₀(t) is an unspecified baseline subdistribution hazard function, and β is a p-dimensional vector of regression coefficients.

For right censored competing risk data, the pseudo log-partial likelihood function of the PSH model is defined as¹⁹

$$l\left(\beta \right)=\sum _{i=1}^n{\int }_0^{\infty }\left[{\beta }^T{X}_i-\log \left\{\sum _j{\omega }_j\left(u\right)Y_j\left(u\right)\exp \left({\beta }^T{X}_j\right)\right\}\right]\times {\omega }_i\left(u\right)d{N}_i\left(u\right),$$ (2)

where $N_i\left(t\right)=I\left(T_i\le t,{\epsilon }_i=1\right),Y_i\left(t\right)=1-N_i\left(t-1\right)$, and ω_i(t) is a time dependent weight developed based on inverse probability of censoring weighting (IPCW) technique, allowing for dependence between censoring times and covariates. For an individual i at time t, the IPCW weight is defined as ${\omega }_i\left(t\right)=I\left(C_i\ge T_i\wedge t\right)\widehat{G}\left(T_i\wedge t\right)$. Here G(t) = Pr(C ≥ t) is the survival function of the censoring time C and $\widehat{G}\left(t\right)$ is the Kaplan-Meier estimate for G(t). At a given time t, if the individual is right censored or failed due to an event of interest, ω_i(t)Y_i(t) = 0; if failed due to competing risks, then ω_i(t)Y_i(t) is between 0 and 1 and decreasing over time; otherwise, ω_i(t)Y_i(t) = 1.

The pseudo log-partial likelihood can also be written as²⁰

$$l\left(\beta \right)=\sum _{i=1}^{n}I\left({\delta }_i{\epsilon }_i=1\right)\left[{\beta }^T{X}_i-\log \left\{\sum _{j\in R_i}{\omega }_j\left(t_i\right)\exp \left({\beta }^T{X}_j\right)\right\}\right],$$ (3)

where the risk set $R_i=\left\{j:\left(T_j\ge T_i\right)\cup \left(\left(T_j\le T_i\right)\cap \left({\delta }_j=1\right)\cap \left({\epsilon }_i\ne 1\right)\right)\right\}$, including subjects still at risk and those who have already failed from competing cause prior to time t.

Recently, penalization methodology is extended to the PSH model by proposing a generalized objective function and rigorously established the asymptotic properties of the proposed penalized estimators:²¹

$$Q\left(\beta \right)=l\left(\beta \right)-\sum _{j=1}^p{p}_{\lambda }\left(\left|{\beta }_j\right|\right),$$ (4)

where l(β) is defined in Equation (3), p_λ(|β_j|) is the penalty function, and λ is a tuning parameter that controls the complexity of selected models.

As can see from the literature,^17,19,21,22 Lasso-type penalties are the most widely used ones among all regularized models. However, when the dimension of the feature space and/or the number of samples are extremely large, solving common optimization algorithms for the lasso-type problem remains challenging.

In this research, using SAFE feature screening rules, we can quickly remove a significant number of features without solving the L₁ optimization problems and these discarded features are guaranteed not to appear in an optimal solution. Consequently, the computational burden associated with computationally intensive optimization problems can be substantially reduced.

2.2 |. SAFE rules for Lasso-PSH and adaptive Lasso-PSH models

In this subsection, we will derive SAFE rules for PSH model with Lasso and adaptive Lasso penalties:

• Lasso penalty: p_λ(|β_j|) = λ|β_j|.

• Adaptive Lasso: p_λ(|β_j|) = λw_j|β_j|, where w_j is a data-adaptive weight assigned to each regression parameter. Generally speaking, a default of $w_j=1/\left|{\tilde{\beta }}_j\right|$ can yield the oracle properties, and the penalized estimator $\tilde{\beta }={\left[{\tilde{\beta }}_1,{\tilde{\beta }}_2,\dots ,{\tilde{\beta }}_p\right]}^T$ is the maximizer of the log partial likelihood l(β) in Equation (3).

First, we derive the SAFE screening rule for the Lasso-PSH model. The primal optimal problem for the Lasso PSH model is as follows:

$$\underset{\beta }{\max }l\left(\beta \right)-\lambda {\Vert \beta \Vert }_1=\underset{\beta }{\max }\left\{\sum _{i=1}^{n}I\left({\delta }_i{\epsilon }_i=1\right)\left[{\beta }^T{X}_i-\log \left\{\sum _{j\in R_i}{\omega }_j\left(t_i\right)\exp \left({\beta }^T{X}_j\right)\right\}\right]-\lambda {\Vert \beta \Vert }_1\right\}.$$ (5)

Let β* be the optimum of the primal problem. To get the dual form of the optimization problem (5), we introduce the following notations. An event time of interest matrix can be defined as an indicator matrix I := (I_ij) ∈ {0, 1}^f×n where f is the number of unique observed events of interest failure times $\left({\sum }_{i=1}^{n}I\left({\delta }_i{\epsilon }_i=1\right)\right)$ and I_ij = 1 if j ∈ R_i. Assuming $Z:=\left(z_{ij}\right)=1{\beta }^T{X}^T\in {ℝ}^{f\times n},1={\left(1,\dots ,1\right)}^T\in {ℝ}^f$.

Then the optimization problem (5) for PSH model can be written as

$$\begin{gathered} \underset{\mathit{\beta },Z}{\max }\left\{\sum _{\left\{i|{\delta }_i{\epsilon }_i=1\right\}}\left[{\mathit{x}}_{\mathit{i}}\mathit{\beta }-\log \left\{\sum _{j\in R_i}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right\}\right]-\lambda {\Vert \beta \Vert }_1\right\} \\ =\underset{\mathit{\beta },Z}{\max }\left\{c^T\mathit{\beta }-\sum _{i=1}^f\log \left\{\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right\}-\lambda {\Vert \mathit{\beta }\Vert }_1\right\}, \end{gathered}$$ (6)

where $c={\sum }_{\left\{i|{\delta }_i{\epsilon }_i=1\right\}}{x}_{\mathit{i}}\in R^p$.

For (6), introducing a dual variable U := (u_ij) ∈ R^f×n, the dual form can be written as

$$\underset{U}{\min }\underset{\mathit{\beta },Z}{\max }{c}^T\mathit{\beta }-\sum _{i=1}^f\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right)-\lambda {\Vert \mathit{\beta }\Vert }_1+tr\left(U\left(Z^T-X\mathit{\beta }{1}^T\right)\right),$$ (7)

where tr(A) refers to the trace of the matrix A.

Following the derivation detailed in Appendix A, the dual problem given in Equation (7) can be rewritten as⁴:

$$\begin{gathered} \underset{U}{\min }\sum _{i=1}^f\sum _{j=1}^n{u}_{ij}\left(\log u_{ij}-\log {\omega }_j\left(t_i\right)\right), \\ \text{s.t.}{\Vert X^{T}U1-c\Vert }_{\infty }\le \lambda ,U^{T}1=1,U\ge 0,U\circ \left(1-I\right)=0, \end{gathered}$$ (8)

where ∘ denotes the element-wise multiplication.

Theorem 1 below gives the SAFE rule for the Lasso PSH model. The proof of this theorem is available in Appendix B.

Theorem 1 (SAFE rule for Lasso-PSH). Consider the optimization problem Lasso-PSH in (5). Denote by x_k the kth feature (column) of the matrix X. We can obtain the index set for all inactive (excluded) features:

$$\zeta =\left\{k|\lambda >\max \left(c_k-\sum _{i=1}^f\underset{j:I_{ij}=1}{\min }{x}_{jk},\sum _{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}-c_k\right)\right\},$$ (9)

where $c={\sum }_{\left\{i|{\delta }_i{\epsilon }_i=1\right\}}{x}_i\in {ℝ}^p$, f is the number of unique events of interest failure times and $I_{ij}=I_{i\in R_j},R_j=\left\{k:\left(T_k\ge T_j\right)\cup \left(\left(T_k\le T_j\right)\cap \left({\delta }_k=1\right)\cap \left({\epsilon }_k\ne 1\right)\right)\right\}$.

According to the above SAFE screening rule (9), for every index k ∈ ζ, the kth entry of β* (the optimum of the primal problem) is zero, that is, (β*)_k = 0, and feature x_k can be safely eliminated from X, a priori to solving the optimization problem (6).

With the adaptive Lasso penalty, we can obtain the SAFE rule for the adaptive Lasso PSH model as stated in following Theorem 2.

Theorem 2 (SAFE rule for adaptive Lasso-PSH). Similar to the Lasso PSH model, the primal optimal problem for the adaptive Lasso PSH model can be written as:

$$\begin{gathered} \underset{\mathit{\beta }}{\max }l(\mathit{\beta })-\lambda \sum _{j=1}^p\left|{\beta }_j\right|/\left|{\tilde{\beta }}_j\right| \\ =\underset{\mathit{\beta }}{\max }\left\{\sum _{i=1}^{n}I\left({\delta }_i{\epsilon }_i=1\right)\left[{\mathit{\beta }}^T{x}_i-\log \left\{\sum _{j\in R_i}{\omega }_j\left(t_i\right)\exp \left({\mathit{\beta }}^T{x}_j\right)\right\}\right]-\lambda \sum _{j=1}^p\left|{\beta }_j\right|/\left|{\tilde{\beta }}_j\right|\right\}. \end{gathered}$$ (10)

Following the procedure in Theorem 1, we can obtain the index set for all inactive (excluded) features:

$$\zeta =\left\{k\mid \frac{\lambda }{\left|{\tilde{\beta }}_k\right|}>\max (c_k-\sum _{i=1}^f\underset{j:I_{ij}=1}{\min }{x}_{jk},\sum _{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}-c_k)\right\}.$$ (11)

Based on the SAFE screening rule in (11), for every index k ∈ ζ, the kth entry of β* is zero, that is, (β*)_k = 0, and feature x_k can be safely eliminated from X, a priori to solving the optimization problem (10).

In penalized Lasso-type problems, the tuning parameter λ plays an important role. Typically, the optimal value of λ is chosen via cross-validation, Akaike information criterion (AIC), Bayesian information criterion (BIC), generally cross-validation (GCV), and/or other criteria. Since optimization problems over a sequence of tuning parameter values are involved, such procedures are usually time consuming.

With the help of the proposed screening method, the computational burden on solving Lasso or adaptive Lasso PSH models can be greatly alleviated. Hence, for a given λ, some inactive features of (5) or (10) can be identified and discarded. In other words, for a specified λ, only a partial data matrix is involved to solve the optimization problem in (5) or (10), and consequently the algorithm efficiency is substantially improved.

3 |. NUMERICAL STUDIES

Since the Lasso penalty leads to biased estimates for true nonzero coefficients²³ and tends to select too many noninformative variables²⁴ while the adaptive Lasso ensures the existence of global optimizers, produces less biased estimators and reduces the number of false positives, here we only provide empirical results of the adaptive Lasso penalty.

In the following, we will systematically evaluate the screening and predictive performance of the proposed PSH-SAFE method with adaptive Lasso-type penalty (shorten as “PSH_SAFE_aLasso”) on simulation and real-world datasets.

3.1 |. Evaluation criteria

To evaluate the performance of our proposed algorithm, the following evaluation criteria are specified before presenting the experimental results.

3.1.1 |. Efficiency of screening

To measure the efficiency of SAFE screening rules, we choose two popular criteria, that is, the rejection ratio^25,26 and screen ratio:^27,28

$$\text{Rejection ratio}=\frac{\text{Number of eliminated features by screening}}{\text{Number of inactive features in original Lasso solution}{\beta }^{\ast }},$$

$$\text{Screen ratio}=\frac{\text{Number of retained features}}{\text{Original feature dimension}(p)}.$$

Here, inactive features are those features whose coefficients will be set to zero in the original Lasso optimization problem. The rejection ratio less than or equal to 1 implies the screening is SAFE (coefficients of discarded features are guaranteed to be zero in the targeted optimal solution), and a lower screening ratio means feature dimensionality is dramatically decreased.

In this article, we adopt the approach in a previous study²⁷ to build the solution path: initialize λ_max to a sufficiently large value, which force all β to a zero vector, and then gradually decrease λ in each iteration. Hence, λ_max is obtained by setting all ${\widehat{\beta }}_j$ to 0. As for λ_min, if n ≥ p, we set λ_min = 0.001λ_max, else we set λ_min = 0.05λ_max. In our experiments, we search m different λ values in total. For the kth step, λ_k = λ_max(λ_min/λ_max)^k/m.

3.1.2 |. Prediction performance

After variable screening and/or variable selection procedures, it is usually necessary to evaluate the predictive performance power of the model in question. In this study, a concordance index (C-index)²⁹ is adopted to evaluate the accuracy of survival models in competing risks. The C-index metric is calculated by comparing a risk score $\tilde{M}\left(X\right)$ at time t with the survival time of each subject. Here, a higher value of $\tilde{M}\left(X\right)$ implies a higher risk of the event of interest. For two subjects X_i, X_j, the concordance value can be obtained by

$$C_1(t):=P\left(M\left(t,X_i\right)>M\left(t,X_j\right)\mid {\epsilon }_i=1\text{and}{T}_i\le t\text{and}\left(T_i<T_j\text{or}{\epsilon }_j=2\right)\right)\text{.}$$ (12)

In this study, the above risk scores are based on estimates of the cumulative incidence function obtained by the Fine-Gray’s model. Here, the reported C-index values are evaluated over the generated 100 λs on the test sets for each of the three methods (PSH_Lasso, PSH_aLasso, PSH_SAFE_aLasso).

3.2 |. Simulated study

We first investigate the screening performance on data with different censoring rates, dimensions and different correlations between covariates. Then, we explore the computational efficiency under different combinations of dimensions and samples.

3.2.1 |. Simulation settings

Similar to Reference 20, we consider the following two settings: (n, p) = (100, 500) and (n, p) = (200, 800). And covariates X = (x₁ … ,x_p) are marginally standard normal with pairwise correlations corr(x_i, x_j) = ρ^|i−j|. In the experiments, we set ρ = 0.6, 0.9 to reflect moderate and high correlated cases among the covariates. Censoring times are generated from a uniform distribution U(0, c₀), where c₀ is chosen to obtain the low (about 25%), moderate (about 50%) an high (about 70%) censoring rates. Here, we consider two events, one primary event and one competing event. In the following, denote the primary event by 1 and the competing event by 2. Also denote the regression parameter of cause 1 by β₁ = (β₁₁, β₁₂, … ,β_1p)^T and set β₁ = (0.5, 0.5, −0.5, 0.5, 0, … , 0)^T; and for cause 2, β₂ = −β₁. The CIF of cause 1 is:

$$F_i(t\mid X)=P(T\le t,\epsilon =1\mid X)=1-{[1-pr\{1-\exp (-t)\}]}^{\exp \left({\mathit{\beta }}_1^{T}X\right)},$$

which is a unit exponential mixture with mass 1 – pr at ∞ when X = 0. The value of pr is set to 0.3. The CIF for cause 2 is obtained by taking P(ε = 2|X) = 1 − P(ε = 1|X) and then using an exponential distribution with rate $\exp \left({\beta }_2^{T}X\right)$ for the conditional CIF, P(T ≤ t, |ε = 2, X).

Hence, altogether 12 simulated scenarios are investigated. Simulated datasets with moderate correlations (Case 1: ρ = 0.6) and high correlations (Case 2: ρ = 0.9) are summarized in Tables 1 and 2, respectively.

TABLE 1.

Simulated datasets with moderated correlations (Case 1: ρ = 0.6)

Dataset	#instance	#feature	Censoring rate	Event 1 proportion
S1	100	500	31%	35%
S2	100	500	62%	19%
S3	100	500	80%	7%
S4	200	800	25%	44%
S5	200	800	54%	23%
S6	200	800	79%	9.5%

TABLE 2.

Simulated datasets with high correlations (Case 2: ρ = 0.9)

Dataset	#instance	#feature	Censoring rate	Event 1 proportion
S7	100	500	21%	39%
S8	100	500	49%	22%
S9	100	500	70%	13%
S10	200	800	27.5%	34%
S11	200	800	56%	15.5%
S12	200	800	74.5%	8%

3.2.2 |. Comparison results on simulated data

First, we present the simulated results for Case 1 (ρ = 0.6) when moderated correlations between covariates are present. The screening related results, namely, screen ratio and rejection ratio with different λs are shown in Figures 1 and 2, respectively. While the predictive results in terms of C-index with different λs is presented in Figure 3.

FIGURE 1.

Screen ratio for Case 1 given 100 λs parameters. (A) S1, (B) S2, (C) S3, (D) S4, (E) S5, and (F) S6

FIGURE 2.

Rejection ratio for Case 1 given 100 λs parameters. (A) S1, (B) S2, (C) S3, (D) S4, (E) S5, and (F) S6

FIGURE 3.

C-index boxplots for Case 1 over different λs. (A) S1, (B) S2, (C) S3, (D) S4, (E) S5, and (F) S6

From Figure 1, one can observe that the screen ratio decreases very rapidly with the increase of λ/λ_max, indicating that it is very effective in reducing the dimensionality of data. According to Figure 2, we can see that under all these six simulated scenarios, the rejection ratio is always less than or equal to 1, which satisfies the SAFE property, that is, the rejection ratio will not be greater than one.³

According to Figure 3, our algorithm (PSH_SAFE_aLasso, the rightmost one of each subfigure) outperforms the original PSH_aLasso in all these cases and gain better results than PSH_Lasso in most cases (moderate censoring rate with/without a larger sample size). We also notice that, when highly censored data (S3 and S6) are present, all compared models produces unsatisfactory or incorrect predictions as most C-index predictions are below 0.5. This not something unexpected since highly censoring rates (80% and 79%) imply only a small portion of data (7% and 9.5%) have events of interest. With such few data, parameter estimation for PSH may not be reliable and these inaccurate parameters will lead to low predictive performance.

Next, we present the simulated results for Case 2 (ρ = 0.9) when high correlations between covariates are present. This is often the case when Omics data are involved. The screening related results, namely, screen ratio and rejection ratio with different λs are shown in Figure 4 and Figure 5, respectively. And the predictive results in terms of C-index with different λs is presented in Figure 6.

FIGURE 4.

Screen ratio for Case 2 given 100 λs parameters. (A) S7, (B) S8, (C) S9, (D) S10, (E) S11, and (F) S12

FIGURE 5.

Rejection ratio for Case 2 given 100 λs parameters. (A) S7, (B) S8, (C) S9, (D) S10, (E) S11, and (F) S12

FIGURE 6.

C-index boxplots for Case 2 over different λs. (A) S7, (B) S8, (C) S9, (D) S10, (E) S11, and (F) S12

According to Figures 4 and 5, the results on screen ratio and rejection ratio is very similar to the results in Case 1. These results again demonstrate that the proposed method is effective in screening efficiency and safe in eliminating inactive features.

From Figure 6, one can find that, in terms of predictive capability, the proposed method (PSH_SAFE_aLasso, the rightmost one in each subfigure) again show similar performance: PSH_SAFE_aLasso beats PSH_aLasso in almost all cases and outperforms the lasso method in most scenarios, except the heavily censoring case S9 where all three methods achieve comparable results.

From both Figures 5 and 6, the proposed algorithm generally performs the best in terms of C-index and the superiority stands out when moderate censoring and/or high correlated covariates are present. However, with a relative small sample size (n = 100, 200), if highly censored rates are encountered, only a small proportion (about 10% in four scenarios) event of interest data will be used for the screening procedure and estimating the weight $\tilde{\mathit{\beta }}$. Consequently, the noise or instability incurred during both procedures make PSH_SAFE_aLasso may not work as expected.

In the above experiments, the proportion of the primary event (interest of event 1) varies from 8% to 44% and all the rejection and screen ratios results from 12 scenarios suggest that this does not have any effect on the screening efficiency or the SAFE property of the algorithm. As to the effect on predictive capability, we find that the proposed method (PSH_SAFE_aLasso) works best when the primary event data occupy a large proportion (20%) (S4, S5, S7, S8, S10). But, PSH_SAFE_aLasso also beats the other two models on some low proportion cases (S3, S11). Therefore, we may conclude that the proposed method is not too much sensitive to the proportion of primary event.

3.2.3 |. Results on computation efficiency

In survival data with competing risks, we may come across different kinds of big data such as large sample sized and/or ultra-high dimensional data. The computation efficiency of different kinds of method under such settings is our primary concern. To validate the effectiveness of the proposed method in reducing computational burdens, we evaluate the running times of our algorithm and other competitive algorithms under different settings.

In this experiment, simulation settings are almost the same with the above comparison study in the case of ρ = 0.9 and moderate censoring. However, we vary the values of n and p to denote the high dimensional case, ultra-high dimensional case, large sample sized case and the case of both large sample size and ultra-high dimensional data. Here, cases C1, C2, and C3 have the same dimensionality (p = 500) but with different sample sizes (n = 100, 200, 2000). Cases C2, C4, and C5 have the fixed sample size (n = 200) but different dimensionality (p = 500, 800, 5000). The most challenging case is C6, where one can find both a large sample size (n = 2000) and a high dimensionality (p = 5000). In all these cases, λs are chosen to screening out about half of the dimensionality and all models are trained 100 times. Detailed information for these simulated datasets can be found in Table 3.

TABLE 3.

Simulated cases for computational efficiency comparison

Dataset	#instance	#feature	Censoring rate	Event 1 proportion
C1	100	500	58%	21%
C2	200	500	62%	19%
C3	2000	500	49%	25%
C4	200	800	49%	19%
C5	200	5000	51%	24%
C6	2000	5000	51%	25%

Figure 7 shows the running times of all three compared algorithms over 100 runs. Table 4 gives the average running times of PSH_SAFE_aLasso (with screening) and PSH_aLasso algorithms (without screening). We also provide the speed-ups of the proposed PSH_SAFE_aLasso method in all simulated cases.

FIGURE 7.

Runtime comparison for simulation datasets. (A) C1, (B) C2, (C) C3, (D) C4, (E) C5, and (F) C6

TABLE 4.

Average runtime comparison for the PSH-aLasso with and without SAFE screening rule over 100 runs

Dataset	With screening	Without screening	Speed up
C1 (n = 100, p = 500)	131.29 ms	7947.51 ms	60.53
C2 (n = 200, p = 500)	2.59 s	8.66 s	3.35
C3 (n = 2000, p = 500)	14.19 s	19.73 s	1.39
C4 (n = 200, p = 800)	2.69 s	19.47 s	7.25
C5 (n = 200, p = 5000)	94.06 s	324.56 s	3.45
C6 (n = 2000, p = 5000)	702.00 s	1307.87 s	1.86

As can be seen from Figure 7, in terms of time efficiency, the proposed algorithm always takes the lead in all six simulated scenarios. PSH_aLasso takes the second while PSH_Lasso always takes the most amount of running times. We also observe that there is a sharp difference in running times between the proposed PSH_SAFE_aLasso and methods without safe screening for small sample-sized high dimensional and ultra-high dimensional data. However, with the increase of sample size, PSH_SAFE_aLasso begins to deteriorate but is still faster than PSH_aLasso and much faster than PSH_Lasso.

From Table 4, one may find that adaptive lasso method with safe screening generally outperforms its no-screening counterpart by a noticeable margin in terms of computational efficiency. With safe screening, the speedups can be several times or even several dozens of times in our simulations.

3.3 |. Real application

In this part, we use a publicly available bladder cancer dataset to perform an empirical analysis of the proposed method.³⁰ This dataset is further preprocessed by eliminating all columns have the same values or data with missing values. The resulting dataset includes 329 samples, each corresponding to 1381 publicly available preprocessed custom platform microarray features.

In this dataset, the response of interest is the time to progression or death from bladder cancer. And death from other or unknown causes is the competing event. For the former event, 57 patients were observed while for the latter, 49 were observed. The remaining 223 patients are censored samples.

3.3.1 |. Results on screening efficiency and safety

The screening efficiency and safety of the proposed screening procedure is shown in Figure 8. According to Figure 8(a), the screen ratio decreases fast with the increase of dimensionality, indicating that the method can dramatically decrease the feature dimensionality. From Figure 8(b), we know that the higher the dimension, the higher the rejection ratio. The rejection ratio ≤ 1, which means the PSH_SAFE_aLasso can successfully identify a majority of the inactive features and they only eliminate features that are guaranteed to be absent after solving the optimization problem.

FIGURE 8.

Empirical analysis of screening efficiency and safety: Screen ratio given 100 λs parameters. (A) Screen ratio and (B) rejection ratio

3.3.2 |. Results on prediction accuracy

In the experiments, results obtained are based on the 5×2 cross-validation procedure.³¹ In 5 × 2 folds cross-validation, the dataset is randomly divide into two equal-sized blocks. The model is trained on the first block and evaluated on the second block and vice versa. This process is repeated five times. Here, following the reviewers’ suggestions, instead of comparing all the predictive power over the 100 generated λs values, we compare the predictive power of all three models by choosing their best parameter λs, respectively. Here, the best tuning parameter is chosen via cross-validation.

From Figure 9, we can clearly see that in terms of C-index, adaptive methods (with screening or not) give almost identical results and both methods outperform the competing (PSH_Lasso) method. This again demonstrates the effectiveness of the proposed method.

FIGURE 9.

Prediction performance comparison: C-index on the real datasets with best chosen λs

3.3.3 |. Results on time efficiency

In simulated study, we have seen that with a fixed λ value (screening out about half of the dimensionality), the proposed method achieves the best performance in terms of time efficiency in all simulated cases. Here, we want to explore the running times at different values of tuning parameter λs. Here, we select four representative quantiles (ie, the upper 5th, 25th, 75th, and 95th) from the generated 100 λs and compare the runtimes of all these three methods with these four λs. Again, for each λ, the experiment is run a hundred times. Figure 10 shows the runtimes of all three compared models on the bladder cancer dataset.

FIGURE 10.

Runtime on the real dataset with four specific λs over 100 runs. (A) λ₅, (B) λ₂₅, (C) λ₇₅, and (D) λ₉₅

From Figure 10, we can clearly see that the PSH_SAFE_aLasso takes the least time compared with PSH_aLasso and PSH_Lasso. In order to show the advantages of our algorithm’s fast speed more clearly, we display average running time values in the form of a Table 5.

TABLE 5.

Average runtime comparison for the PSH-aLasso with and without SAFE screening rule over 100 runs

λ	With screening	Without screening	Speed up
λ 5	82.68 ms	98886.73 ms	1196.02
λ 25	2.56 s	98.74 s	38.57
λ 75	82.4 s	100.45 s	1.31
λ 95	93.94 s	106.10 s	1.13

It is seen from Table 5 that a larger λ value (such as λ₅ or λ₂₅) is associated with a larger speed-up. This is reasonable, since with a large λ value, a large proportion of inactive features will be removed prior to training the adaptive Lasso PSH model. Hence, the model is trained on a much reduced data matrix, which consequently leads to greater savings in the computational time.

4 |. DISCUSSION

We have proposed fast SAFE screening algorithms for the PSH model for competing risks data with ultrahigh dimensional covariates. Our empirical studies demonstrate that our algorithm is able to efficiently and safely eliminate some features whose corresponding coefficients of optimization problem are guaranteed to be zero. Moreover, it can significantly reduce the running time for large high dimensional competing risks data while maintaining competitive prediction performance. In particular, the computational superiority stands out for large penalty parameter values.

We point out that the proposed PSH-SAFE screening strategy works with any PSH Lasso solvers. Inspired by the fact that the computational complexity for the log-pseudo likelihood and its derivatives for the PSH model can be reduced from O(n²) to O(n),¹⁷ our team is currently working on a more efficient PSH-SAFE implementation to handle competing risks survival data with both ultrahigh dimensionality and massive sample size.

An R package “SFEcmprsk” has been developed for the proposed screen procedure and the code is available at https://github.com/whcsu/safecomp.

APPENDIX A. DERIVATION OF THE DUAL FORM (8)

In this appendix, we provide the detailed derivation of the dual form (8) of Lasso PSH.

For the dual form,

$$\underset{U}{\min }\underset{\mathit{\beta },Z}{\max }{c}^T\mathit{\beta }-\sum _{i=1}^f\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right)-\lambda \parallel \mathit{\beta }{\parallel }_1+tr\left(U\left(Z^T-X\mathit{\beta }{1}^T\right)\right)$$

using U = [u₁, u₂, …, u_f], Z = [z₁, z₂, …, z_f], where u_i, z_i ∈ Rⁿ, the above formula can be rewritten as⁴

$$P^{\ast }=\underset{U}{\min }\sum _{i=1}^f\underset{z_i}{\max }{\mathit{u}}_{\mathit{i}}^T{\mathit{z}}_{\mathit{i}}-\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right)+\underset{\mathit{\beta }}{\max }{\mathit{c}}^T\mathit{\beta }-\lambda \parallel \mathit{\beta }{\parallel }_1-tr\left(1^{T}UX\mathit{\beta }\right).$$ (A1)

In order to get P*, first consider the situation containing only β

$$\begin{gathered} f(\beta )=\underset{\mathit{\beta }}{\max }{\mathit{c}}^T\mathit{\beta }-\lambda \parallel \mathit{\beta }{\parallel }_1-tr\left(1^{T}UX\mathit{\beta }\right) \\ =\underset{\mathit{\beta }}{\max }{\mathit{\beta }}^T\left(\mathit{c}-X^T{U}^{T}1\right)-\lambda \parallel \mathit{\beta }{\parallel }_1. \end{gathered}$$ (A2)

For (A2), f(β) is convex but not smooth. Next, we need to consider its subgradient

$$\frac{\partial f(\mathit{\beta })}{\partial \mathit{\beta }}=\mathit{c}-X^T{U}^{T}1-\mathit{\lambda }v=0.$$ (A3)

In which ν is the subgradient of ‖β‖₁, its definition by element-wise is: ∀k, k = 1, 2, …, p

$$v_k=\{\begin{array}{ll}+1,& {\beta }_k>0,\\ -1,& {\beta }_k<0,\\ \left[-1,+1\right],& {\beta }_k=0.\end{array}$$

Plugging into Equation (A2), we can obtain that if ‖X^TU1 − c‖_∞ ≤ λ, then f*(β) = 0. So

$$P^{\ast }=\underset{U}{\min }\sum _{i=1}^f\underset{z_i}{\max }{\mathit{u}}_{\mathit{i}}^T{\mathit{z}}_{\mathit{i}}-\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right):{\Vert {\mathit{X}}^{T}U1-c\Vert }_{\infty }\le \lambda .$$ (A4)

For every i, consider every optimization problem

$$\underset{z_i}{\max }{\mathit{u}}_{\mathit{i}}^T{\mathit{z}}_{\mathit{i}}-\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right):{\Vert {\mathit{X}}^{T}U1-c\Vert }_{\infty }\le \lambda .$$ (A5)

The solution is

$$=\{\begin{array}{ll}{\sum }_{j=1}^n{u}_{ij}\left(\log u_{ij}-\log {\omega }_j\left(t_i\right)\right),& u_i\ge 0,{\mathbf{1}}^T{u}_i=1,\forall j:u_{ij}\left(1-I_{ij}\right)=0,\\ +\infty ,& \text{otherwise.}\end{array}$$

So,

$$\begin{gathered} P^{\ast }=\underset{U}{\min }\sum _{i=1}^f\sum _{j=1}^n{u}_{ij}\left(\log u_{ij}-\log {\omega }_j\left(t_i\right)\right), \\ \text{s.t.}{\Vert {\mathit{X}}^{T}U1-c\Vert }_{\infty }\le \lambda ,U^{T}1=1,U\ge 0,U\circ (1-I)=0, \end{gathered}$$ (A6)

where ∘ is the multiplication between the element-wise.

APPENDIX B. PROOF OF THEOREM 1

After solving the dual problem of optimization, we can obtain the SAFE rule. In this appendix, we present the proof of Theorem 1.

We restate the optimization problem for PSH model here:

$$\underset{\mathit{\beta },Z}{\max }\left\{c^T\mathit{\beta }-\sum _{i=1}^f\log \left\{\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right\}-\lambda \parallel \mathit{\beta }{\parallel }_1\right\},$$

where $\mathit{c}={\sum }_{\left\{i\mid {\delta }_i{\epsilon }_i=1\right\}}{\mathit{x}}_{\mathit{i}}\in R^p$. For each feature k = 1, 2, …, p, based on the convex optimization theory, if the following holds, then ${\beta }_k={\beta }_k^{\ast }$, this is that β_k is at optimum.

$$\lambda >\underset{U}{\max }\left|{\mathit{x}}_{\mathit{k}}^T{U}^{T}1-c_k\right|:U1=1,U\ge 0,U\circ (1-I)=0.$$ (B1)

At the same time, from the dual form equation

$$\underset{U}{\min }\underset{\mathit{\beta },Z}{\max }{\mathit{c}}^T\mathit{\beta }-\sum _{i=1}^f\log \left(\sum _{j=1}^n{I}_{ij}{\omega }_j\left(t_i\right)\exp \left(z_{ij}\right)\right)-\lambda \parallel \mathit{\beta }{\parallel }_1+tr\left(U\left(Z^T-X\mathit{\beta }{1}^T\right)\right),$$

we have if ${\beta }_k^{\ast }=0$ , the following must be true at optimum ${\beta }_k^{\ast }$.

$$-\lambda \left|{\beta }_k^{\ast }\right|+\left({\mathit{x}}_{\mathit{k}}^{T}U1-c_k\right){\beta }_k^{\ast }<0.$$ (B2)

If all U in the feasible set satisfies Equation (B2), then the feature can be safely eliminated. To obtain the feature screening rule, the maximization problem (B1) must be solved. First, consider the below expression for each x_k.

$$S_{+}\left({\mathit{x}}_{\mathit{k}}\right)=\underset{U}{\max }{\mathit{x}}_{\mathit{k}}^{T}U1:U^{T}1=1,U\ge 0,U\circ (1-I)=0.$$

Based on duality, we can get

$$\begin{gathered} S_{+}\left(x_k\right)=\underset{Z}{\min }\underset{U\ge 0,U^T\mathbf{1}=\mathbf{1}}{\max }{\mathit{x}}_{\mathit{k}}^{T}U1+tr{Z}^T\left(\left(I-11^T\right)\circ U\right) \\ =\underset{Z}{\min }\underset{U\ge 0,U^T\mathbf{1}=\mathbf{1}}{\max }trU\left(\left(I-11^T\right)\circ Z+1{\mathit{x}}_{\mathit{k}}^T\right) \\ =\underset{Z}{\min }\sum _{i=1}^f\underset{1\le j\le n}{\max }\left(\left(I_{ij}-1\right)z_{ij}+x_{jk}\right) \\ =\sum _{i=1}^f\underset{z}{\min }\underset{1\le j\le n}{\max }\left(x_{jk}+\left(I_{ij}-1\right)z_{ij}\right) \\ =\sum _{i=1}^f\underset{z}{\min }\max \left(\underset{j:I_{ij}=1}{\max }{x}_{jk},\underset{j:I_{ij}=0}{\max }{x}_{jk}-z_{ij}\right) \\ \ge \sum _{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}. \end{gathered}$$ (B3)

It can also be shown that $S_{+}\left({\mathit{x}}_{\mathit{k}}\right)\le {\sum }_{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}.$ by choosing z_ij = 0 for I_ij = 0, $z_{ij}=\underset{h,I_{hj}=0}{\max }{x}_{hj}-\underset{h,I_{hj}=1}{\max }{x}_{hj}$ otherwise. As a result, the expression can be written as

$$S_{+}\left({\mathit{x}}_{\mathit{k}}\right)=\sum _{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}$$ (B4)

Similarly, we can get

$$\begin{gathered} S_{-}\left({\mathit{x}}_{\mathit{k}}\right)=\underset{U}{\min }{\mathit{x}}_{\mathit{k}}^T{U}^{T}1:U1=1,U\ge 0,U\circ (1-I)=0 \\ =-S_{+}\left(-{\mathit{x}}_{\mathit{k}}\right) \\ =\sum _{i=1}^f\underset{j:I_{ij}=1}{\min }{x}_{jk}. \end{gathered}$$ (B5)

Based on these two expressions, Equation (B1) can be written as

$$\lambda >\max \left(\left|S_{+}\left({\mathit{x}}_{\mathit{k}}\right)-c_k\right|,\left|S_{-}\left({\mathit{x}}_{\mathit{k}}\right)-c_k\right|\right),$$ (B6)

$$\lambda >\max \left(c_k-\sum _{i=1}^f\underset{j:I_{ij}=1}{\min }{x}_{jk},\sum _{i=1}^f\underset{j:I_{ij}=1}{\max }{x}_{jk}-c_k\right).$$ (B7)

If the kth feature satisfies the SAFE condition (B7), then β_k = 0 at optimum.

DATA AVAILABILITY STATEMENT

The bladder cancer dataset is available at https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE5479.