A Modified Adaptive Lasso for Identifying Interactions in the Cox Model with the Heredity Constraint

Abstract

In many biomedical studies, identifying effects of covariate interactions on survival is a major goal. Important examples are treatment-subgroup interactions in clinical trials, and gene-gene or gene-environment interactions in genomic studies. A common problem when implementing a variable selection algorithm in such settings is the requirement that the model must satisfy the strong heredity constraint, wherein an interaction may be included in the model only if the interaction’s component variables are included as main effects. We propose a modified Lasso method for the Cox regression model that adaptively selects important single covariates and pairwise interactions while enforcing the strong heredity constraint. The proposed method is based on a modified log partial likelihood including two adaptively weighted penalties, one for main effects and one for interactions. A two-dimensional tuning parameter for the penalties is determined by generalized cross-validation. Asymptotic properties are established, including consistency and rate of convergence, and it is shown that the proposed selection procedure has oracle properties, given proper choice of regularization parameters. Simulations illustrate that the proposed method performs reliably across a range of different scenarios.

1. Introduction

The Cox proportional hazards regression model (Cox, 1972, 1975) is the most widely used statistical model for evaluating relationships between an event time, T, and baseline covariates, X = (X₁, ⋯, X_p). The Cox model is characterized by the hazard function

$$h(t\mid X,\beta )=h_0\left(t\right)\exp \left\{g\right(X,\beta \left)\right\},t>0,$$ (1)

for a subject with covariates X, where h₀(t) is an unspecified baseline hazard and the linear component $g(X,\beta )={\Sigma }_{j=1}^p{\beta }_j{X}_j$ is characterized by a vector β = (β₁, ⋯, β_p)^T of unknown regression coefficients. In many applications, this simple form of g(X, β) does not adequately describe the relationship between T and X due to interactions between elements of X. For example, an anti-cancer agent tailored to attack a certain biological target typically has effects hypothesized to differ between the subgroups of patients who do and do not have the target, identified by a binary “biomarker” covariate. In a randomised trial of the targeted agent versus standard therapy, such a differential effect is characterized as a treatment-biomarker interaction which, if found to be sufficiently large, may lead to regulatory approval of the agent for patients who are biomarker positive. Identifying such treatment-biomarker interactions thus is a key step in developing personalized treatments. The presence of other covariates that may or may not be associated with T, and that also may interact with treatment, complicates identification and estimation of biomarker effects. Fitting the Cox model with interactions is challenging since, even with a moderate number of covariates, the number of interaction terms may be large, and not all covariates and interactions may have meaningful effects on h(t|X, β).

There is a large literature on variable selection methods for survival models. A family of penalized partial likelihood methods have been proposed for the Cox proportional hazard model, including the Lasso (Tibshirani et al., 1997) and the smoothly clipped absolute deviation method (Fan and Li, 2002). By shrinking some regression coefficients to zero, these methods simultaneously select important variables and estimate the regression model parameters. Zhang and Lu (2007) proposed an adaptive Lasso estimator for variable selection in the Cox model, with an adaptively weighted L₁ penalty on the regression coefficients that has a convex form. They showed that this method enjoys the oracle properties, global optima exist, and it can be implemented efficiently using standard numerical algorithms (Boyd and Vandenberghe, 2004).

All the above variable selection methods for the Cox model treat the candidate variables equally. However, when interactions are included, there is a natural hierarchical ordering among the variables in the model (Chipman, 1996; Joseph, 2006; Yuan et al., 2007). This motivates the strong heredity requirement that an interaction can be included in the model only if the interaction’s component variables are included as main effects (Hamada and Wu, 1992), since models that violate this property are difficult to interpret. For linear and generalized linear regression models, Yuan et al. (2009) proposed non-negative garrote methods that naturally incorporate a general hierarchical structure among predictors. Along this line, Choi et al. (2010) extended the Lasso to identify interaction terms while obeying the strong heredity constraint, which is achieved by reparameterizing the coefficients of the interaction terms. Bien et al. (2013) investigated a Lasso for hierarchical interactions, and Radchenko and James (2010) considered a more general case with nonlinear interactions. To our knowledge, none of these papers studied the setting with time-to-event data, and none of the variable selection methods for the Cox model noted above satisfy the strong heredity constraint when interactions are included. This paper aims at filling this gap.

We propose a modified Lasso procedure for the Cox model to adaptively select covariates and interactions while automatically enforcing the strong heredity constraint. The main challenges compared to linear/generalized linear regression are that the Cox model is semiparametric and involves right-censored data. We carry out estimation and variable selection by optimizing a modified log partial likelihood that includes two adaptively weighted penalty terms, one for main effects and one for reparameterized interactions, with each penalty multiplied by a tuning parameter. The main reason that we chose the reparameterization approach is that this method can handle all p main effects simultaneously in one iteration, which has tremendous numerical advantage, especially for survival analysis when no closed form can be found. The two-dimensional tuning parameter is determined by generalized cross-validation. The proposed method is computationally convenient, has good convergence properties, and implementation is straightforward. We establish asymptotic properties, including selection consistency, rate of convergence, and the oracle property (Fan and Li, 2001; Fan and Peng, 2004) that it performs as well as if the correct underlying model were known in advance. These theoretical properties and algorithms have not been studied previously for variable selection for Cox models with interaction terms subject to the strong hierarchy constraint.

2. Adaptive Lasso with Strong Heredity Constraint Using Penalized Partial Like-lihood

Let T_i denote the failure time, C_i the censoring time, and X_i = (X_i1, ⋯, X_ip) the covariate vector of the i^th subject, for i = 1, ⋯, n, with T̃_i = min(T_i, C_i) and δ_i = I(T_i ≤ C_i). Suppose that (X₁, T₁, C₁), ⋯, (X_n, T_n, C_n) are independent and identically distributed. We further assume non-informative censoring, T_i ⫫ C_i | X_i, i.e., T_i is conditionally independent of C_i given X_i. Although our methods can be generalized to handle higher order interactions, for the ease of exposition we consider the Cox model having linear component with all possible two-way interactions. That is, in model (1)

$$g(X,\beta ,\alpha )=\sum _{j=1}^p{\beta }_j{X}_j+\sum _{1\le j<j^{\prime }\le p}{\alpha }_{j,j^{\prime }}{X}_j{X}_{j^{\prime }},$$ (2)

where α = (α_1,2, ⋯, α_p−1,p)^T. Our goals are to provide a method that determines which terms in g(X, β, α) have important effects on the hazard, and develop a corresponding computational algorithm and parameter estimators having desirable properties. The existing variable selection methods for the Cox model do not guarantee the strong heredity constraint, as they treat all elements of (β, α) equally and do not distinguish between elements of β and α.

We first re-parameterize the coefficients for the interaction terms in (2) as α_j,j′ = γ_j,j′β_jβ_j′, so that the linear term becomes

$$g(X,\theta )=\sum _{j=1}^p{\beta }_j{X}_j+\sum _{1\le j<j^{\prime }\le p}{\gamma }_{j,j^{\prime }}{\beta }_j{\beta }_{j^{\prime }}{X}_j{X}_{j^{\prime }}$$ (3)

and the parameter vector is θ = (β, γ) = (β₁, ⋯ β_p, γ_1,2, ⋯, γ_p−1,p)^T. With this reparameterization, the coefficient for an interaction term X_jX_j′ must be 0 if either of its two main effects X_j or X_j′ has coefficient 0. Conversely, if γ_j,j′β_jβ_j′ ≠ 0, this implies that both β_j ≠= 0 and β_j′ ≠= 0, which guarantees the strong heredity constraint. With the reparameterization (3), the log partial likelihood is

$$l_n\left(\theta \right)=\sum _{i=1}^n{\delta }_i\left(g(X_i,\theta )-\log \left[\sum _{r=1}^{n}I({\tilde{T}}_r\ge {\tilde{T}}_i)\exp \left\{g\right(X_r,\theta \left)\right\}\right]\right),$$ (4)

where I(A) denotes the indicator of the event A. For the variable selection problem at hand, we will minimize the adaptive penalized negative log partial likelihood

$$Q_n(\theta ,{\lambda }_{\beta },{\lambda }_{\gamma })=-l_n\left(\theta \right)+n{\lambda }_{\beta }\sum _{j=1}^p{w}_j^{\beta }\mid {\beta }_j\mid +n{\lambda }_{\gamma }\sum _{1\le j<j^{\prime }\le p}{w}_{j,j^{\prime }}^{\gamma }\mid {\gamma }_{j,j^{\prime }}\mid ,$$ (5)

where $\left\{w_j^{\beta }\right\}$ and $\left\{w_{j,j^{\prime }}^{\gamma }\right\}$ are prespecified weights, and λ_β, λ_γ are tuning parameters. Following Breiman (1995) and Zou (2006), we compute the weights in (5) using

$$w_j^{\beta }=\frac{\log \left(n\right)}{n}\mid \frac{1}{{\tilde{\beta }}_j}\mid ,w_{j,j^{\prime }}^{\gamma }=\frac{\log \left(n\right)}{n}\mid \frac{{\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}}{{\tilde{\alpha }}_{j,j^{\prime }}}\mid ,$$

where ${\tilde{\beta }}_j$’s and ${\tilde{\alpha }}_{j,j^{\prime }}$’s are the estimates from a usual, unpenalized fitted Cox model with linear component (2). The multiplier log(n)/n is included to satisfy the convergence rate and the asymptotic properties introduced in Section 3, below. The objective function becomes

$$Q_n(\theta ,{\lambda }_{\beta },{\lambda }_{\gamma })=-l_n\left(\theta \right)+\log \left(n\right){\lambda }_{\beta }\sum _{j=1}^p\frac{\mid {\beta }_j\mid }{\mid {\tilde{\beta }}_j\mid }+\log \left(n\right){\lambda }_{\gamma }\sum _{1\le j<j^{\prime }\le p}\mid {\gamma }_{j,j^{\prime }}\mid \mid \frac{{\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}}{{\tilde{\alpha }}_{j,j^{\prime }}}\mid .$$ (6)

The two tuning parameters in (6) control the coefficient estimates at different levels. The first tuning parameter λ_β controls main effect estimates. If β_j is shrunk to zero, all terms involving X_j, including β_jX_j and the interactions γ_j,j′β_jβ_j′ X_jX_j′, for any j′, are removed from the model. The second tuning parameter λ_γ controls the estimates only at the interaction effect level. Even if both β_j ≠ 0 and β_j′ ≠ 0, it is possible that γ_j,j′ = 0 if X_j and X_j′ do not interact. The penalty term controlled by λ_γ thus provides the flexibility of selecting only main effects of X_j and X_j′ but not their interaction.

The weights act on the objective function, Q_n(θ, λ_β, λ_γ), as follows. If the initial estimate ${\tilde{\beta }}_j$ is close to 0 then $w_j^{\beta }$ will be large and hence, as can be seen from (5), the coefficient β_j of X_j will be heavily penalized. Similarly, if ${\tilde{\alpha }}_{j,j^{\prime }}$ is small relative to ${\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}$ then $w_{j,j^{\prime }}^{\gamma }$ will be large and the coefficient ${\gamma }_{j,j^{\prime }}$ of the interaction term X_jX_j′ will be heavily penalized.

3. Theoretical Properties of the Estimator

In this section, we study the asymptotic properties of our proposed variable selection procedure and the corresponding estimator. As n → ∞, our estimator possesses the oracle property under certain regularity conditions, that is, it performs as well as if the true model were known in advance (Fan and Li, 2001). The regularity conditions that we need throughout the development are given in Appendix 1 in the web supplementary materials.

Let ${\theta }_0={({\beta }_0^T,{\gamma }_0^T)}^T$ denote the true parameter vector, where

$${\gamma }_{0j,j^{\prime }}=\{\begin{array}{cc}\hfill & {\alpha }_{0j,j^{\prime }}∕{\beta }_{0j}{\beta }_{0j^{\prime }}\text{if}{\beta }_{0j}\ne 0\text{and}{\beta }_{0j^{\prime }}\ne 0\hfill \\ \hfill & 0\text{otherwise}\hfill \end{array}\phantom{\}}$$ (7)

This guarantees that the true model obeys the strong heredity constraint, that is, α_0j,j′ = γ_0j,j′ = 0 if either β_0j = 0 or β_0j′ = 0.

Define the covariate-specific tuning parameters ${\lambda }_{j,n}^{\beta }={\lambda }_{\beta }{w}_j^{\beta }=n^{-1}\log \left(n\right){\lambda }_{\beta }∕\mid {\tilde{\beta }}_j\mid$ for j = 1, ⋯, p and the interaction-specific tuning parameters ${\lambda }_{j,j^{\prime },n}^{\gamma }={\lambda }_{\gamma }{w}_{j,j^{\prime }}^{\gamma }=n^{-1}\log \left(n\right){\lambda }_{\gamma }∕\mid {\tilde{\gamma }}_{j,j^{\prime }}\mid$ for 1 ≤ j < j′ ≤ p, where ${\tilde{\gamma }}_{j,j^{\prime }}={\tilde{\alpha }}_{j,j^{\prime }}∕{\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}$. Our proposed estimator is

$$\widehat{\theta }={\mathrm{argmin}}_{\theta }\left\{-l_n\left(\theta \right)+n\sum _{j=1}^p{\lambda }_{j,n}^{\beta }\mid {\beta }_j\mid +n\sum _{1\le j<j^{\prime }\le p}{\lambda }_{j,j^{\prime },n}^{\gamma }\mid {\gamma }_{j,j^{\prime }}\mid \right\}.$$ (8)

Note that the criterion in (8) is the same as Q_n(θ, λ_β, λ_γ) in (6) with different notation.

Without loss of generality, we denote ${\beta }_0={({\beta }_{a0}^T,{\beta }_{b0}^T)}^T$, where β_a0 consists of all nonzero components and β_b0 consists of the remaining zero components of β₀. Similarly, for the true coefficient of interactions, write ${\gamma }_0=({\gamma }_{a0}^T,{\gamma }_{b0}^T,{\gamma }_{c0}^T)T$, where γ_a0 contains all nonzero components of γ₀, γ_b0 contains the zero components of γ₀ whose corresponding main effects both are nonzero, and γ_c0 contains the remaining zero components of γ₀ whose corresponding components of main effects have at least one zero. Correspondingly, we denote the maximizer of (8) as ${\widehat{\theta }}_n={({\widehat{\beta }}_{an}^T,{\widehat{\beta }}_{bn}^T,{\widehat{\gamma }}_{an}^T,{\widehat{\gamma }}_{bn}^T,{\widehat{\gamma }}_{cn}^T)}^T$, the covariate-specific tuning parameters as $\{{\left({\lambda }_{an}^{\beta }\right)}^T,{\left({\lambda }_{bn}^{\beta }\right)}^T\}$, and the interaction-specific tuning parameters as $\{{\left({\lambda }_{an}^{\gamma }\right)}^T,{\left({\lambda }_{bn}^{\gamma }\right)}^T,{\left({\lambda }_{cn}^{\gamma }\right)}^T\}$.

Let

$${\xi }_n=\max \{{\left({\lambda }_{an}^{\beta }\right)}^T,{\left({\lambda }_{an}^{\gamma }\right)}^T\},{\zeta }_n=\min \left\{{\left({\lambda }_{bn}^{\beta }\right)}^T,{\left({\lambda }_{bm}^{\gamma }\right)}^T\right\}.$$

Then ξ_n is the maximum among both covariate-specific and interaction-specific tuning parameters that correspond to nonzero coefficients in the model. But for ζ_n, we only consider those specific tuning parameters corresponding to zero main effects and zero interaction terms when coefficients are in γ_b0. That is, we consider all zero cases for the main effects, but for the interactions, we only consider those zero terms whose corresponding main effects are nonzeros. We will refer to such terms in the definition of ζ_n as non-trivial zero terms.

For the proposed variable selection procedure to perform properly, as n → ∞ the covariate-specific and interaction-specific penalties for the terms whose true coefficients are nonzeros should converge to 0, and the penalties for those non-trivial zero terms should be large enough so that the estimates shrink to 0. In fact, if the tuning parameters in (8) that correspond to β_a0 and γ_a0 converge to 0, and those corresponding to β_b0 and γ_b0 are sufficiently large, then our proposed estimating procedure will have the so-called oracle property (Fan and Li, 2001). This can be guaranteed when $\sqrt{n}$–consistent estimates of θ₀ are used in the definition of ${\lambda }_{j,n}^{\beta }$ and ${\lambda }_{j,j^{\prime },n}^{\gamma }$, where one can easily show that $\sqrt{n}{\xi }_n$ → 0 and $\sqrt{n}{\zeta }_n$ → ∞.

Denote the initial estimates as $\tilde{\beta }=({\tilde{\beta }}_1,\cdots ,{\tilde{\beta }}_p)$, $\tilde{\gamma }=({\tilde{\gamma }}_{1,2},\cdots ,{\tilde{\gamma }}_{p-1,p})$, and we summarize the above results in Theorem 1. A sketch of the proof of Theorem 1 is given in Appendix 2.

Theorem 1. When $\sqrt{n}(\tilde{\beta }-{\beta }_0)=O_p\left(1\right)$ and $\sqrt{n}(\tilde{\gamma }-{\gamma }_0)=O_p\left(1\right)$ in (8) to calculate ${\lambda }_{j,n}^{\beta }$ and ${\lambda }_{j,j^{\prime },n}^{\gamma }$, $\sqrt{n}{\xi }_n\to \infty$ and $\sqrt{n}{\zeta }_n\to \infty$ as n → ∞. Under the regularity conditions (1)-(3) in Appendix 1, there exists a local minimizer ${\widehat{\theta }}_n$ of Q_n(θ) such that

1. (Sparsity) $P({\widehat{\beta }}_{bn}=0)\to 1$, $P({\widehat{\gamma }}_{bn}=0)\to 1$, and $P({\widehat{\gamma }}_{cn}=0)\to 1$ as n → ∞.

2. (Asymptotic normality)

   $$\sqrt{n}\left\{\left(\begin{array}{c}\hfill {\widehat{\beta }}_{an}\hfill \\ \hfill {\widehat{\gamma }}_{an}\hfill \end{array}\right)-\left(\begin{array}{c}\hfill {\beta }_{a0}\hfill \\ \hfill {\gamma }_{a0}\hfill \end{array}\right)\right\}\stackrel{d}{\to }N\{0,{\mathcal{I}}_a^{-1}({\beta }_{a0},{\gamma }_{a0}\left)\right\},$$

where ${\mathcal{I}}_a({\beta }_{a0},{\gamma }_{a0})$ is the Fisher information matrix evaluated at β_a0 and γ_a0 assuming that β_b0 = 0, γ_b0 = 0, and γ_c0 = 0 is known in advance.

Part (i) of Theorem 1 presents the sparsity property and shows that our proposed method can consistently remove the zero-effect terms with probability tending to 1. This implies that, with a sufficiently large sample, in practice our method can select the underlying true model with high probability. In part (ii) of Theorem 1, we establish that the estimates of nonzero elements of θ₀ are $\sqrt{n}$–consistent and asymptotically normal. The asymptotic distribution is the same as what would be obtained if it were known in advance which elements of θ₀ are 0 and which are not 0, the so-called oracle property.

Theorem 2, below, establishes the asymptotic behavior of the proposed method when p is very large, especially when p → ∞ as n → ∞. This is an important case in many biomedical studies, with the advance of new high throughput technologies. A proof of Theorem 2 is given in the Appendix 3, provided in the web supplementary materials.

When the number of predictors may increase with the sample size n, we denote p as p_n, which allows the possibility that p_n → ∞ as n → ∞. Including all main effects and pairwise interactions, the total number of parameters is q_n = (p_n + 1)p_n/2. Similarly, when appropriate we add a subscript n to other notation, and we let d_n denote the number of non-zero coefficients in the underlying true model. Then we have

Theorem 2. Under the regularity conditions (4)-(6) in Appendix 1, if p_n = o(n^1/5) and $\sqrt{nqn}{\xi }_n\to 0$, $\sqrt{n∕qn}{\zeta }_n\to \infty$ as n → ∞, then there exists a local minimizer ${\widehat{\theta }}_n$ of Q_n(θ) such that

1. (Sparsity) $P({\widehat{\beta }}_{bn}=0)\to 1$, $P({\widehat{\gamma }}_{bn}=0)\to 1$, and $P({\widehat{\gamma }}_{cn}=0)\to 1$ as n → ∞.

2. (Asymptotic Normality)

   $$\sqrt{n}{\Omega }_n{\mathcal{I}}_{an}^{1∕2}({\beta }_{a0},{\gamma }_{a0})\left\{\left(\begin{array}{c}\hfill {\widehat{\beta }}_{an}\hfill \\ \hfill {\widehat{\gamma }}_{an}\hfill \end{array}\right)-\left(\begin{array}{c}\hfill {\beta }_{a0}\hfill \\ \hfill {\gamma }_{a0}\hfill \end{array}\right)\right\}\stackrel{d}{\to }N\{0,\Sigma \},$$

where Ω_n is any arbitrary d × d_n matrix with a finite d such that ${\Omega }_n{\Omega }_n^T\to \Sigma$, Σ is a d × d semipositive definite symmetric matrix, and ${\mathcal{I}}_{an}({\beta }_{a0},{\gamma }_{a0})$ is the d_n × d_n Fisher information matrix evaluated at (β_a0, γ_a0) assuming that β_b0 = 0, γ_b0 = 0, and γ_c0 = 0 is known in advance.

The reason we consider an arbitrary linear combination ${\Omega }_n{({\widehat{\beta }}_{an}^T,{\widehat{\gamma }}_{an}^T)}^T$ in Theorem 2, instead of ${({\widehat{\beta }}_{an}^T,{\widehat{\gamma }}_{an}^T)}^T$ as in Theorem 1, is because the latter has dimension d_n → ∞ as n → ∞ in the current setup, while the former has finite dimension d.

4. Computational Algorithm

Expanding l_n(θ) in the objective function (6), Q_n(θ, λ_β, λ_γ) becomes

$$-\sum _{i=1}^n{\delta }_i\left[\left(\sum _{j=1}^p{\beta }_k{X}_{ij}+\sum _{1\le j<j^{\prime }\le p}{\gamma }_{j,j^{\prime }}{\beta }_j{\beta }_{j^{\prime }}{X}_{ij}{X}_{i{j}^{\prime }}\right)-\log \left\{\sum _{r=1}^{n}I({\tilde{T}}_r\ge {\tilde{T}}_i)\times \exp \left(\sum _{j=1}^p{\beta }_j{X}_{rj}+\sum _{1\le j<j^{\prime }\le p}{\gamma }_{j,j^{\prime }}{\beta }_j{\beta }_{j^{\prime }}{X}_{rj}{X}_{r{j}^{\prime }}\right)\right\}\right]+\log \left(n\right){\lambda }_{\beta }\sum _{j=1}^p\frac{\mid {\beta }_j\mid }{\mid {\tilde{\beta }}_j\mid }+log\left(n\right){\lambda }_{\gamma }\sum _{1\le j<j^{\prime }\le p}\mid {\gamma }_{j{j}^{\prime }}\mid \mid \frac{{\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}}{{\tilde{\alpha }}_{j{j}^{\prime }}}\mid$$

Our estimator minimizes Q_n(θ, λ_β, λ_γ), that is, ${\widehat{\theta }}_n={\arg \min }_{\theta }{Q}_n(\theta ,{\lambda }_{\beta },{\lambda }_{\gamma })$. To carry out the optimization, we apply a modified version of the iteratively reweighted least squares algorithm (Green, 1984) with weighted L₁ penalties. Denoting the gradient vector of the partial likelihood by $\stackrel{.}{l}\left(\theta \right)=-\partial l_n\left(\theta \right)∕\partial \theta$ and the Hessian matrix by $\ddot{l}\left(\theta \right)=-{\partial }^2{l}_n\left(\theta \right)∕\partial \theta \partial {\theta }^T$, and using the Cholesky decomposition $\ddot{l}\left(\theta \right)=M{M}^T$, where M is an invertible lower triangular matrix, we define the pseudo response vector $Y={\left(M\right)}^{-1}\left\{\ddot{l}\right(\theta )\theta -\stackrel{.}{l}(\theta \left)\right\}$. By the usual second-order Taylor expansion, −l_n(θ) in (6) can be approximated by the quadratic form (Y − M^Tθ)^T (Y − M^Tθ)/2, and at each penalized iteratively reweighted least squares iteration we minimize

$$\frac{1}{2}{(Y-M^T\theta )}^T(Y-M^T\theta )+\log \left(n\right){\lambda }_{\beta }\sum _{j=1}^p\frac{\mid {\beta }_j\mid }{\mid {\tilde{\beta }}_j\mid }+log\left(n\right){\lambda }_{\gamma }\sum _{1\le j<j^{\prime }\le p}\mid {\gamma }_{j,j^{\prime }}\mid \mid \frac{{\tilde{\beta }}_j{\tilde{\beta }}_{j^{\prime }}}{{\tilde{\alpha }}_{j,j^{\prime }}}\mid .$$ (9)

Because the main effect coefficients β and the interaction coefficients γ are controlled at different levels, in each step we also iterate between these two sets, first fixing β to estimate γ, then fixing γ to estimate β, and iterating until convergence. This algorithm is guaranteed to converge, since the objective function decreases at each step. When β is fixed, the optimization in γ becomes a Lasso problem, hence one can use either the LARS/Lasso algorithm (Efron et al., 2004) or quadratic programming to solve for γ efficiently. When γ is fixed, we solve for β₁, ⋯, β_p sequentially. For each j = 1, ⋯, p, we fixγ and β_[−j] = (β₁, ⋯, β_j−1, β_j+1, ⋯, β_p), and the optimization becomes a simple Lasso problem with only one parameter, β_j. This is similar to the shooting algorithm (Fu, 1998; Zhang and Lu, 2007; Friedman et al., 2007). The tuning parameters are selected by minimizing the generalized cross validation statistic,

$$C_{pseudo-GCV}=\frac{l\left(\widehat{\theta }\right)}{{(1-df∕n)}^2},$$

over a reasonable range of λ_β and λ_γ, where df is the number of nonzero parameters in the fitted model.

For a fixed λ_β and λ_γ, the optimization algorithm proceeds as follows:

Step 1. Center and normalize each term X_j, X_jX_j′, j < j′, and j, j′ = 1, …, p.

Step 2. Start with plausible initial values ${\widehat{\beta }}_j^{\left(0\right)}$ and ${\widehat{\gamma }}_{j,j^{\prime }}^{\left(0\right)}$, j < j′, and j, j′ = 1, …, p, such as the conventional Cox regression parameter estimates. Set m=1.

Step 3. Compute Y and M based on the current value ${\widehat{\theta }}^{(m-1)}$. Denote

$$\widehat{l}\left(\theta \right)=-\frac{1}{2}{(Y-M^T\theta )}^T(Y-M^T\theta ).$$

Step 4. To update $\widehat{\gamma }$, let ${\widehat{\gamma }}^{\left(m\right)}=\arg {\min }_{\gamma }{\Sigma }_{i=1}^n\{-\tilde{l}({\widehat{\beta }}^{(m-1)},\gamma )+n{\lambda }_{\gamma }{\Sigma }_{j<j^{\prime }}{w}_{j,j^{\prime }}^{\gamma }\mid {\gamma }_{j,j^{\prime }}\mid \}$.

Step 5. To update $\widehat{\beta }$ for each j = 1, …, p in sequence, let

$${\widehat{\beta }}_j^{\left(m\right)}=\arg \underset{{\beta }_j}{\min }\sum _{i=1}^n\{-\tilde{l}({\widehat{\beta }}_{[-j]}^{(m-1)},{\beta }_j,{\widehat{\gamma }}^{\left(m\right)})+{\lambda }_{\beta }{w}_j^{\beta }\mid {\beta }_j\mid \}.$$

Step 6. If the relative difference between $Q_n\left({\widehat{\theta }}^{(m-1)}\right)$ and $Q_n\left({\widehat{\theta }}^{\left(m\right)}\right)$,

$${\Delta }^{\left(m\right)}=\frac{\mid Q_n\left({\widehat{\theta }}^{(m-1)}\right)-Q_n\left({\widehat{\theta }}^{\left(m\right)}\right)\mid }{\mid Q_n\left({\widehat{\theta }}^{(m-1)}\right)\mid },$$

is small enough, then stop. Otherwise, increment m to m+1 and return to Step 3.

Since γ_j,j′ = α_j,j′ /β_jβ_j′, in Step 3 the minimization is actually over {α_j,j′} with $\beta ={\widehat{\beta }}^{(m-1)}$. This algorithm gives exact zeros for some coefficients and guarantees that the coefficients of the corresponding interactions are set to 0 whenever the corresponding is shrunk to 0. Based on our empirical experience, the algorithm converges quickly.

5. Numerical Results

In this section, we report results of a simulation study comparing our proposed method with the Lasso and adaptive Lasso, two popular variable selection methods for the Cox model, neither of which guarantees the strong heredity constraint. We follow the simulation setup in Zhang and Lu (2007), but also include two-way interaction terms as candidates for the variable selection. We consider sample size n = 200, and assume that there are p = 5 covariates of interest in each simulated dataset, denoted by X₁, X₂, ⋯, X₅. Thus, the number of all possible two-way interaction terms is p×(p−1)/2 = 10, and there are a total of 15 candidate terms. We assume each covariate follows a standard normal distribution, and consider two scenarios: (i) the covariates are independent, and (ii) the covariates have pairwise correlations Corr(X_j, X_j′) = ρ^|j−j′|, with ρ = 0.5. We generate the censoring times from a Uniform distribution having support (0, τ), with τ chosen to obtain a specified censoring rate of either 25% or 40%.

Then suppose failure times are generated from a Cox model with constant baseline hazard λ₀ = 0.1, where the coefficients of X₂, X₃, and the interaction term X₂ X₃ are non-zero, and the other 12 coefficients are zero. We consider the following two models:

Model 1: ${\beta }_2^0$ = −0.8, ${\beta }_3^0$ = −0.8, and ${\alpha }_{2,3}^0$ = −0.8, corresponding to large effects.

Model 2: ${\beta }_2^0$ = −0.3, ${\beta }_3^0$ = −0.3, and ${\alpha }_{2,3}^0$ = −0.3, corresponding to small effects.

We simulate each case 100 times, and apply all three methods to each simulated data set. For all methods, the tuning parameters are selected using the generalized cross validation criterion C_pseudo−GCV .

Table 1 summarizes the percentage of fully correct variable selection among 100 replications for each method under each scenario. For all cases, the proposed method correctly selects the true model more frequently than the regular Lasso and adaptive Lasso. Specifically, for all scenarios under Model 2, where important variables and interactions have small effects, the Lasso and adaptive Lasso perform about the same, with the latter slightly better, while our proposed method performs much better than these two methods. In Model 1, where important variables and interactions have large effects, the advantage of our proposed method is still substantial when the censoring percentage is high, 40%. When censoring is moderate, 25%, the adaptive Lasso becomes competitive, but still is worse than the proposed method. The performance of the Lasso is always the worst among the three methods.

Table 1.

Percentages of correctly selected models among 100 replications.

			Method
	Censoring Percentage	ρ for Correlation	Lasso	Adaptive Lasso	Proposed Method
Model 1	25%	0	11%	70%	86%
		0.5	6%	66%	79%
	40%	0	3%	58%	80%
		0.5	3%	54%	82%
Model 2	25%	0	43%	49%	74%
		0.5	26%	31%	63%
	40%	0	28%	32%	74%
		0.5	22%	33%	62%

Table 2 summarizes the individual frequencies of being selected into the model for each main effect and interaction term. We only present the results here for 25% censoring under both models when the covariates are independent (ρ = 0) due to space limitations, but similar results are observed for the 40% censored case, and correlated covariates case. As shown in Table 2, the proposed variable selection procedure always chooses important variable and interaction terms much more often than the other two methods. Under Model 1 for large effects, the adaptive Lasso and the proposed method in general select the unimportant terms less often than the Lasso, while for Model 2 with small effects, the proposed method actually selects some of the unimportant terms more often. However, as illustrated in Table 1, the proposed method simultaneously selects all the important terms and removes the unimportant ones more frequently.

Table 2.

Term-specific percentages of being selected for each main effect and interaction. (25% censoring and ρ = 0 for independent covariates)

	Method	x 1	x 2	x 3	x 4	x 5	x 1 x 2	x 1 x 3	x l x 4
Model 1	Lasso	22%	100%	100%	18%	25%	15%	12%	15%
	adaptive Lasso	6%	100%	100%	3%	7%	2%	4%	1%
	proposed method	5%	100%	100%	4%	6%	5%	5%	0%
Model 2	Lasso	6%	91%	91%	6%	5%	2%	6%	3%
	adaptive Lasso	2%	87%	85%	4%	4%	1%	2%	1%
	proposed method	10%	98%	99%	11%	8%	10%	10%	1%

	Method	x 1 x 5	x 2 x 3	x 2 x 4	x 2 x 5	x 3 x 4	x 3 x 5	x 4 x 5
Model 1	Lasso	15%	100%	13%	13%	11%	13%	15%
	adaptive Lasso	2%	100%	3%	2%	2%	2%	0%
	proposed method	1%	100%	4%	6%	4%	6%	0%
Model 2	Lasso	3%	89%	4%	3%	9%	5%	2%
	adaptive Lasso	2%	82%	1%	2%	1%	0%	0%
	proposed method	1%	97%	10%	8%	11%	8%	2%

A simple, ad hoc alternative approach often employed in practice for variable selection involving interaction terms is to run either the Lasso or the adaptive Lasso, and then, depending on which terms were selected, manually add back any main terms that were not selected but that were components of any interaction term that was selected. This ensures the strong heredity constraint. As seen in our simulation, this practice does not help much in terms of how often the true model is selected correctly, while the false positive error rate is greatly increased. For example, as one can see in Table 2 for model 2, since the proposed method selected the true interaction term x₂x₃ more often than the other two methods, using the above ad hoc approach following the Lasso or the adaptive Lasso still cannot beat the proposed method. For model 1 with large effects in Table 2, the frequency of x₂, x₃, and x₂x₃ being correctly selected already achieves 100%, so this ad hoc approach does not help at all.

To measure the prediction accuracy, we average the mean squared error $C_{MSE}={(\widehat{\theta }-\theta )}^T\Gamma (\widehat{\theta }-\theta )$ over 100 replications by following Tibshirani et al. (1997) and Zhang and Lu (2007), where is the population variance-covariance matrix of the covariates. Standard errors are given in parentheses. For all scenarios, the proposed method has the smallest mean squared error (Table 3), and thus outperform the other two competitors in terms of prediction accuracy.

Table 3.

Mean squared error, with standard errors in parentheses.

	Censoring Percentage	ρ for Correlation	Method
			Lasso	Adaptive Lasso	Proposed Method
Model 1	25%	0	0.213 (0.124)	0.072 (0.068)	0.063 (0.065)
		0.5	0.219 (0.136)	0.071 (0.068)	0.069 (0.065)
	40%	0	0.337 (0.209)	0.168 (0.144)	0.117 (0.219)
		0.5	0.351 (0.215)	0.162 (0.172)	0.118 (0.216)
Model 2	25%	0	0.118 (0.053)	0.105 (0.055)	0.042 (0.041)
		0.5	0.131 (0.059)	0.130 (0.059)	0.054 (0.048)
	40%	0	0.117 (0.053)	0.122 (0.058)	0.061 (0.062)
		0.5	0.137 (0.066)	0.142 (0.068)	0.082 (0.078)

6. Discussion

We have extended the adaptive Lasso method to accommodate the Cox proportional hazard model including interaction terms while ensuring that the strong heredity constraint is satisfied in the selected model. Hamada and Wu (1992) consider other constraints, such as weak heredity where only one of the two main terms is required to be included when an interaction term is considered. Although this situation is not of our main interest, our methods can easily be modified to handle it by employing a different reparameterization.

Similarly, as discussed in Zhang and Lu (2007), the adaptive choice of weights may become problematic when some elements of θ are not estimable. This may occur, for example, when strong collinearity exists among covariates, or when the number of covariates p is much larger than the sample size n in high-dimensional data. In such settings, one cannot obtain the initial estimates to determine the adaptive weights. Alternatively, robust estimation of θ, such as ridge regression, may be considered.

Our work was motivated by the desire to identify key treatment-biomarker interactions for developing personalized treatments, where the number of candidate biomarkers is usually fixed or may grow slowly with n. Even with a moderate number of candidate biomarkers, however, our methodology can have an important impact on physician’s actual behavior, if clinically meaningful treatment-biomarker effects are identified. The condition p = o(n^1/5) that we used may be relaxed though. There have been recent theoretical developments on high-dimensional survival models such as those in Bradic et al. (2011) and Lin and Lv (2013), but their underlying theory cannot be applied directly to our setting, which uses a reparameterization approach for selecting important interactions with heredity constraint. In order to relax our condition p = o(n^1/5), modification of the above theory to our setting would be required, and this wouldentail a substantial amount of work. While this is beyond the scope of the current paper, it certainly would be an interesting future research project.