Sparse Estimation of Cox Proportional Hazards Models via Approximated Information Criteria

Summary

We propose a new sparse estimation method for Cox (1972) proportional hazards models by optimizing an approximated information criterion. The main idea involves approximation of the ℓ₀ norm with a continuous or smooth unit dent function. The proposed method bridges the best subset selection and regularisation by borrowing strength from both. It mimics the best subset selection using a penalised likelihood approach yet with no need of a tuning parameter. We further reformulate the problem with a reparameterisation step so that it reduces to one unconstrained nonconvex yet smooth programming problem, which can be solved efficiently as in computing the maximum partial likelihood estimator (MPLE). Furthermore, the reparameterisation tactic yields an additional advantage in terms of circumventing post-selection inference. The oracle property of the proposed method is established. Both simulated experiments and empirical examples are provided for assessment and illustration.

1. Introduction

Consider the usual setup for censored survival data. Let ( $T_i^{\prime },C_i^{\prime }$) denote the failure and censoring times for the i^th individual for i = 1, …, n. The observed failure time is $T_i=min(T_i^{\prime },C_i^{\prime })$ with failure status indicated by ${\delta }_i=I\{T_i^{\prime }\le C_i^{\prime }\}$. Let z_i ∈ ℝ^p denote the p-dimensional covariate vector associated with subject i. Without loss of generality (WLOG), we assume that all the covariates have been standardised. For identifiability concern in the ensuing modelling and inference, we assume that $T_i^{\prime }$ and $C_i^{\prime }$ are independent given z_i, i.e., $T_i^{\prime }\perp C_i^{\prime }|{\mathbf{z}}_i$. Thus the observed data consist of {(T_i, δ_i, z_i) : i = 1, …, n}. The Cox (1972) proportional hazards (PH) model formulates the hazard function of $T_i^{\prime }$ given z_i as

$$h(t|{\mathbf{z}}_i)=h_0(t)exp({\mathit{\beta }}^T{\mathbf{z}}_i)$$ (1)

where β = (β_j) ∈ ℝ^p is the unknown regression parameter vector. Estimation of model (1) is based on the partial likelihood (Cox, 1975). Throughout the article, we shall restrict our discussion to the traditional finite dimension scenario, i.e., p is fixed and p < n, while possible high-dimensional extensions will be discussed later. Assuming no or few ties in the observed failure times, the partial log-likelihood function for β is given by

$$l(\mathit{\beta })=\sum _{i=1}^n{\delta }_i[{\mathbf{z}}_i^T\mathit{\beta }-log\sum _{i^{\prime }=1}^n\{I(T_{i^{\prime }}\ge T_i)exp({\mathbf{z}}_{i^{\prime }}^T\mathit{\beta })\}].$$

Concerning variable selection, the true β is often sparse in the sense that some of its components are zeros. By ‘sparse estimation’, we refer to methods and procedures that allow for identification of zero components in β and estimation of its nonzero components simultaneously.

There are two major types of variable selection techniques for survival models. Both can be generally formulated as a penalised partial likelihood form:

$$\underset{\mathit{\beta }}{min}-2l(\mathit{\beta })+\lambda \cdot \text{pen}(\mathit{\beta }),$$ (2)

where pen(β) is a penalty function and the penalty parameter λ is either fixed a priori or treated as a tuning parameter. The first type is the best subset selection (BSS) methods, where a model selection criterion such as AIC (Akaike, 1974) or BIC (Schwarz, 1978) is employed to compare models of all choices. BSS solves

$$\underset{\mathit{\beta }}{min}-2l(\mathit{\beta })+{\lambda }_0{\Vert \mathit{\beta }\Vert }_0,$$ (3)

where the ℓ₀ norm ${\Vert \mathit{\beta }\Vert }_0=\text{card}(\mathit{\beta })={\sum }_{j=1}^{p}I\{{\beta }_j\ne 0\}$ measures the model complexity and the penalty parameter λ₀ is fixed as λ₀ = ln(n₀) for BIC, with n₀ being the total number of uncensored failures (Vollinsky and Raftery, 2000). If AIC is used, then λ₀ = 2. Due to the discrete nature of the ℓ₀ norm, solving (3) is NP-hard and its optimisation is proceeded in two steps: fit every model with the maximum partial likelihood method and then compare the fitted models according to an information criterion. Although faster algorithms such as branch-and-bound (Furnival and Wilson, 1974) and iterative hard thresholding (Blumensath and Davies, 2009) and heuristic surrogates such as stepwise procedures are available for this combinatorial optimisation problem, the best subset selection is infeasible for moderately large p.

The second type is regularisation, as exemplified by Least Absolute Shrinkage and Selection Operator (LASSO; Tibshirani, 1997), adaptive LASSO (ALASSO; Zhang and Lu, 2007), and Smoothly Clipped Absolute Deviation penalty (SCAD; Fan and Li, 2002). LASSO replaces the ℓ₀ norm with the ℓ₁ norm for convex relaxation, i.e., $\text{pen}(\mathit{\beta })={\Vert \mathit{\beta }\Vert }_1={\sum }_{j=1}^p|{\beta }_j|$, so that the problem becomes

$$\underset{\mathit{\beta }}{min}-2l(\mathit{\beta })+\lambda {\Vert \mathit{\beta }\Vert }_1.$$ (4)

The significance of LASSO is that it reformulates sparse estimation into a continuous convex programming problem. Nevertheless, the performance of LASSO is unsatisfactory in either variable selection or parameter estimation. To improve, ALASSO applies a weighted ℓ₁ norm and SCAD employs a nonconvex penalty, both enjoying the oracle property, i.e., consistency in selecting variables and efficiency in estimating the nonzero coefficients, under certain conditions given that λ can be appropriately chosen.

With the regularisation approach, the penalty function does not correspond well to the model complexity ‖β‖₀. As a result, the value of the penalty parameter λ is no longer trackable by referring to AIC or BIC. Therefore, optimisation of (4) has to resort to two steps as well: first solve (4) for every fixed λ > 0 to obtain a regularisation path {β̃(λ) : λ > 0}, and then select the best λ via an information criterion along the path. While two fast algorithms, homotopy (Osborne, Presnell, and Turlach, 2000) and coordinate descent (Fu, 1998), have been proposed for ℓ₁ regularisation, the two-step procedure can be time-consuming when solving (4) with a fixed λ entails iterative procedures, which is the case in Cox PH models. Compared to BSS, the ℓ₁ regularisation procedure amounts to seeking minimum AIC or BIC only along the regularisation path, which is a much reduced search space since β̃(λ) is nothing but a one-dimensional curve indexed by λ in the original search space ℝ^p. Therefore, it is reasonable to deduce that the regularisation-based estimators may not perform as well as the estimator obtained with BSS, if AIC or BIC is used as the performance criterion.

Besides the concerns about performance and computational efficiency, another major challenge that both methods face is post-selection inference. Statistical inference is routinely done based on the nonzero coefficient estimates in the regularisation or these selected variables in BSS. In such a practice, it has been taken for granted that model selection has no or little effect on the subsequent inferences, a myth recently debunked by Leeb and Pötscher (2005) who discussed an impossibility result for some post-selection estimation. The problem can be manifested by the fact that no standard error results are available for zero coefficient estimates in regularisation approaches. One is referred to Berk et al. (2013) and Lockhart et al. (2014) for further discussions and recent developments on this issue.

In this article, we put forward a new method of conducting sparse estimation for Cox PH models that helps address the aforementioned deficiencies. The main idea is to approximate the information criteria so that it yields a continuous or smooth objective function for easier optimisation. For simplicity, we abbreviate the proposed method as MIC for ‘minimum information criterion’. MIC extends the best subset selection to scenarios with large p. At the same time, MIC can be regarded as a regularisation method, yet free of tuning. In order to circumvent the post-selection inference, we also propose a technical maneuver to obtain a valid statistical testing for parameters with zero estimates. The remainder of the paper is organised as follows. Section 2 presents the proposed method in detail, as well as its asymptotic properties. Section 3 addresses the post selection inference problem. Section 4 contains numerical results based on both simulated experiments and a real data example. Section 5 ends the article with a short discussion.

2. Minimizing the Approximated Information Criterion

We seek a new sparse estimation method that can borrow strength from both BSS and regularisation and bridge them. We start with BSS by approximating the discrete ℓ₀ norm and make further improvement by capitalizing on knowledge of regularisation.

2.1 Approximation of ℓ₀ Norm

While the idea of optimisation plays a critical role in both BSS and regularisation, the primary motivation of our approach comes from approximation. The discrete nature of ℓ₀ norm in (3) poses the main obstacle for BSS, which motivates us to seek a continuous or smooth approximation to it with a continuous surrogate function. This essentially involves approximation of I(β ≠ 0). For this purpose, we introduce the concept of unit dent functions. We call a continuous function w : ℝ → [0, 1] a unit dent function at 0 if it satisfies: (i) w(·) is an even function such that w(β) = w(−β); (ii) w(0) = 0 and lim_|β|→∞ w(β) = 1; and (iii) w(β) is increasing on ℝ₊. Denote by 𝒟₀ the space of all unit dent functions at 0. It can be seen that 𝒟₀ is closed under operations such as composition and product. Clearly, any unit dent function in 𝒟₀ can be viewed as a continuous approximation of I(β ≠ 0).

Among many others, one natural choice in 𝒟₀ is the hyperbolic tangent function given by

$$tanh(a{|\beta |}^r)=\frac{exp(2a{|\beta |}^r)-1}{exp(2a{|\beta |}^r)+1},$$

where a > 0 is a scale parameter that controls the sharpness of the approximation and r ∈ ℕ has typical values of 1 and 2. Figure 1 plots the tanh function with r = 1 in (a) and r = 2 in (b), for different choices of a = 1, 2, …, 200. It can be seen that a relatively large a is needed in order to provide a good approximation. It is also interesting to note that the curve with r = 2 is smooth while the curve with r = 1 has a cusp at β = 0.

Figure 1.

Hyperbolic tangent penalty functions tanh(a|γ|^r) that approximate the indicator function I(γ ≠ 0): (a) r = 1 and (b) r = 2. The value of a ranges from 1 to 200.

From the perspective of regularisation, Fan and Li (2001) spelled out three desired properties for the penalty function: unbiasedness, sparsity, and continuity. It can be seen that both unbiasedness and continuity are easily satisfied by both choices. To enforce sparsity, the choice of r = 1 is favorable as opposed to r = 2 at the first sight. However, setting r = 1 leads to a non-smooth optimisation problem and the resultant estimates suffer from the same post-selection inference. These concerns motivate us to restrict our attention to r = 2 in order to ensure smoothness. We then devise a different way of enforcing sparsity and circumventing post-selection inference.

2.2 Reparameterisation

With r = 2, solving ${min}_{\mathit{\beta }}\{-2l(\mathit{\beta })+{\lambda }_0{\sum }_{j=1}^{p}tanh(a{\beta }_j^2)\}$ facilitates a surrogate BSS method as in (3). This is a smooth optimisation problem; however, it does not provide sparse estimates. To remedy, we shall reparameterise the problem by introducing γ = (γ_j) ∈ ℝ^p, which relates to β as follows. Define $w_j=w({\gamma }_j)=tanh(a{\gamma }_j^2)$ for j = 1, …, p and matrix W = diag(w_j). Then set β_j = γ_jw_j for each j. That is, we reparameterise β in terms of γ such that β = Wγ. Now we consider the following optimisation problem

$$\underset{\mathit{\gamma }}{min}-2l(\mathbf{W}\mathit{\gamma })+{\lambda }_0\text{tr}(\mathbf{W}),$$ (5)

where $\text{tr}(\mathbf{W})={\sum }_{j=1}^p{w}_j$ is the trace of matrix W. It turns out that this simple reparameterisation step not only helps enforce sparsity while keeping the optimisation problem smooth but also addresses the inference issue with zero estimates.

One original motivation of the above reparameterisation step came from nonnegative garotte (NG; Breiman, 1995). NG is formulated as a sign-constrained regularisation problem based on the decomposition β = sgn(β) · |β|. Assuming that the signs of β can be correctly specified by another consistent estimator, say, the MPLE β̂, it remains to estimate |β|. Reparameterising β = diag(β̂)γ for γ = (γ_j) with γ_j ≥ 0, NG first estimates γ by solving

$$\underset{\mathit{\gamma }}{min}-2l(\mathit{\beta })\mathrm{s}.\mathrm{t}.\sum _{j=1}^p{\gamma }_j\le \tau \text{and}{\gamma }_j\ge 0$$

with τ being a tuning parameter, and then obtains the estimated regression coefficients β̂_j = γ̂_jβ̂_j for j = 1, …, p. One fundamental problem with NG is that if any sign of the initial estimator β̂ is wrongly specified, which occurs often with real data owing to multicollinearity or other complexities, then it becomes hopeless for NG to make correction. Comparatively, MIC is based on a different decomposition β = β·I{β ≠ 0}. Setting γ = β and approximating I{γ ≠ 0} by w(γ) lead to the reparameterisation β = γw(γ), which does not depend on an initial estimate.

With the reparameterization, the regression coefficient vector remains β. However, the decision vector in (5) becomes γ. This helps keep the optimisation problem smooth. We first obtain the estimate of γ, γ̃, and hence W̃ = diag{w(γ̃_j)}, then we compute the estimate of β as β̃ = W̃γ̃. The function w(γ) is smooth in γ with the first two derivatives given by ẇ = dw/dγ = 2aγ(1 – w²) and ẅ = 2a(1 – w²)(1 – 4aγ²w). To see why (5) leads to sparse estimation of β, it is helpful to examine the penalty w(γ) = tanh(aγ²) as a function of β. First of all, there is one-to-one correspondence between β and γ, as shown in Figure 2(a). As a function of β, w(γ) is a unit dent function that can be used to approximate its ℓ₀ norm. Applying the chain rule and differentiation of the inverse function yields

Figure 2.

Illustration of the reparameterisation step: (a) plot of β vs. γ and (b) plot of w(γ) as a function of β, where w(γ) = tanh(aγ²) and β = γw(γ) for a = 1, 2,…, 200.

$$\frac{dw(\gamma )}{d\beta }=\frac{dw(\gamma )}{d\gamma }·{\left(\frac{d\beta }{d\gamma }\right)}^{-1}=\frac{\dot{w}}{w+\gamma \dot{w}}.$$

Similar arguments can be applied to obtain its higher order derivatives. A closer look reveals that w(γ) is a smooth function in β everywhere except at β = 0 where dβ/dγ = 0. Figure 2(b) plots w(γ) versus β, showing that w(γ) possesses all the properties of the desired penalty function for sparse estimation of β.

2.3 Asymptotic Results

In this section, the asymptotic properties of the MIC estimator β̃ are studied. Owing to the use of the counting processes and martingale theories, all the arguments hold for time dependent covariates z = z(t). WLOG, we work on the time interval t ∈ [0, 1]. Our notations follow those similar to Anderson and Gill (1982), Fan and Li (2002), and Zhang and Lu (2007). We consider the MIC estimator β̃ as the solution of min_β Q_n (β̃) with the objective function

$$Q_n(\mathit{\beta })=-\frac{2}{n}·l(\mathit{\beta })+\frac{ln(n_0)}{n}·\sum _{j=1}^p{\rho }_n({\beta }_j),$$ (6)

where the penalty function ρ_n(β_j) is defined through the reparametrisation β_j = γ_jw(γ_j) and ρ_n(β_j) = w(γ_j) = tanh(a_n γ_j²). Furthermore, we assume that a_n = O_p(n).

Let β₀ denote the true sparse parameter vector and partition it as ${\mathit{\beta }}_0={({\mathit{\beta }}_{0(1)}^T,{\mathit{\beta }}_{0(2)}^T)}^T$, where β₀₍₁₎ ∈ ℝ^p consists of all q nonzero components and β₀₍₂₎ consists of all the (p – q) zero components. Let Y_i(t) = I {T_i ≥ t} be the at-risk process. Define

$$S^{(k)}(\mathit{\beta },t)=\frac{1}{n}\sum _{i=1}^n{Y}_i(t)exp\{{\mathit{\beta }}^T{\mathbf{z}}_i(t)\}{\mathbf{z}}_i^{\otimes k}$$ (7)

for k = 0, 1, and 2, where the outer product notation ⊗ is operated as follows: a^⊗0 = 1, a^⊗1 = a, and a^⊗2 = aa^T for any vector a. Let s^(k)(β, t) = E [Y(t) exp {z(t)^Tβ} z(t)^⊗k] be the expected value of S^(k)(β, t). Then the expected Fisher information matrix associated with the true model is

$$\mathbf{I}({\mathit{\beta }}_0)={\int }_0^1[s^{(2)}({\mathit{\beta }}_0,t)-\frac{{\{s^{(1)}({\mathit{\beta }}_0,t)\}}^{\otimes 2}}{s^{(0)}({\mathit{\beta }}_0,t)}]h_0(t)dt.$$

The following theorem shows that, under regularity conditions, there exists a local minimiser β̃ of Q_n(β) that is √n-consistent and this √n-consistent β̃ enjoys the ‘oracle’ property.

Theorem 1: Assume that {( $T_i^{\prime },C_i^{\prime },{\mathbf{z}}_i$) : i = 1, …, n} are n i.i.d. copies of (T′, C′, z), $T_i^{\prime }╨C_i^{\prime }|{\mathbf{z}}_i$ for each i, and n₀ = O_p(n). Under the regularity conditions (A)–(D) in Anderson and Gill (1982) or Fan and Li (2002), we have

1. (√n-Consistency) there exists a local minimiser β̃ of Q_n(β) such that ‖β̃ – β₀‖ = O_p(n^−1/2).

2. (Sparsity and Asymptotic Normality) Partition the √n-consistent local estimator in (i) as $\stackrel{\sim }{\mathit{\beta }}={({\stackrel{\sim }{\mathit{\beta }}}_{(1)}^T,{\stackrel{\sim }{\mathit{\beta }}}_{(2)}^T)}^T$ in a similar manner to β₀. With probability tending to 1, β̃ must satisfy that β̃₍₂₎ = 0 and

   $$\sqrt{n}({\stackrel{\sim }{\mathit{\beta }}}_{(1)}-{\mathit{\beta }}_{0(1)})\to N\{\mathbf{0},{\mathbf{I}}_{11}^{-1}({\mathit{\beta }}_0)\}$$

   as n → ∞, where I₁₁(β₀) is the leading q × q submatrix of I(β₀).

Theorem 1 is analogous to Theorems 1 & 2 in Zhang and Lu (2007). Its proof, deferred to the Supplementary Materials, follows Fan and Li (2002) in principle. Nevertheless, since there is no further flexibility offered by adjusting the tuning parameter as in SCAD or ALASSO, properties of the hyperbolic tangent penalty also play a critical role in the proof.

Theorem 1 offers a way of computing the standard errors (SE) for nonzero components β̃₍₁₎ in β̃. Note that I₁₁, the leading q × q submatrix of I, is exactly the same as the Fisher information matrix associated with the reduced model obtained by eliminating terms associated with zero components β̃₍₂₎. An alternative sandwich SE formula for β̃₍₁₎ is also available following arguments similar to Fan and Li (2002), for which we shall not pursue further. However, the SE formulas in both approaches are only available for nonzero MIC estimates. Thus these practices belong to post-selection inference and should be used with caution.

3. Inference on β via γ

Post-selection inference is inherent for the best subset selection and regularization due to their two-step estimation process. In MIC, variable selection and parameter estimation are completed in one single optimization step. This offers us a unique opportunity to circumvent this fundamental problem. We achieve this with the aid of reparameterisation.

The transformation β = γw(γ) facilitates an important convenience: inference on β can be made via γ. This is because the mapping between β and γ is a bijection and β = 0 if and only if γ = 0. Therefore, testing H₀ : β_j = 0 is equivalent to testing H₀ : γ_j = 0. For a zero estimate β̃_j, we cannot compute its standard error. But the objective function remains smooth in γ. The statistical properties of γ̃ are readily available following standard M-estimation arguments. In particular, its asymptotic normality is given in the following theorem.

Theorem 2: Let γ₀ be the reparameterised parameter vector associated with β₀. Under the regularity conditions (A)–(D) in Anderson and Gill (1982), we have

$$\sqrt{n}[\mathbf{D}({\mathit{\gamma }}_0)(\stackrel{\sim }{\mathit{\gamma }}-{\mathit{\gamma }}_0)+{\mathbf{b}}_n]\stackrel{d}{\to }\mathrm{N}\{\mathbf{0},{\mathbf{I}}^{-1}({\mathit{\gamma }}_0)\}.$$ (8)

where

$$\mathbf{D}({\mathit{\gamma }}_0)=\mathit{\text{diag}}(w_j+{\gamma }_j{\dot{w}}_j){|}_{\mathit{\gamma }={\mathit{\gamma }}_0}=\mathit{\text{diag}}(D_{jj})$$ (9)

and the asymptotic bias

$${\mathbf{b}}_n={\{-\ddot{L}({\mathit{\beta }}_0)\}}^{-1}\frac{ln(n_0)}{2}{\left(\frac{{\dot{w}}_j}{w_j+{\stackrel{\sim }{\gamma }}_j{\dot{w}}_j}\right)}_{j=1}^p=(b_{nj})$$ (10)

satisfy (i) lim_n→∞ D_jj = I{γ_0j = 0} and (ii) b_n = o_p(1).

The proof of Theorem 2 is given in the Supplementary Materials. Several comments are in order. Accordingly, an asymptotic (1 – α) × 100% confidence interval for D_jj γ_0j can be simply given as

$$({\stackrel{\sim }{D}}_{jj}{\stackrel{\sim }{\gamma }}_j+{\stackrel{\sim }{b}}_{nj})\pm z_{i-\alpha /2}\sqrt{{({\mathbf{I}}^{-1}(\stackrel{\sim }{\mathit{\gamma }})/n)}_{jj}}.$$ (11)

where D̃_jj is an estimate of D_jj by replacing γ_0j with γ̃_j and similarly for b̃_nj and I⁻¹(γ̃). Empirically, we replace the expected Fisher information matrix I(γ̃) with the observed Fisher information matrix I_n(γ̃) given by

$${\mathbf{I}}_n(\stackrel{\sim }{\mathit{\gamma }})=-{\nabla }^{2}l(\stackrel{\sim }{\mathit{\gamma }})=\sum _{i=1}^n{\delta }_i\{\frac{S^{(2)}(\stackrel{\sim }{\mathit{\gamma }};T_i)}{S^{(0)}(\stackrel{\sim }{\mathit{\gamma }};T_i)}-{(\frac{S^{(1)}(\stackrel{\sim }{\mathit{\gamma }};T_i)}{S^{(0)}(\stackrel{\sim }{\mathit{\gamma }};T_i)}}^{\otimes 2}\},$$

where functions S^(k)(γ̃; T_i) are defined earlier in (7). Note that it is computationally advantageous to use ${\mathbf{I}}_n^{-1}(\stackrel{\sim }{\mathit{\gamma }})$ rather than ${\mathbf{I}}_n^{-1}(\stackrel{\sim }{\mathit{\beta }})$, although both γ̃ and β̃ are consistent for β₀. Working with β̃ entails handling very small or large numbers numerically.

Further simplification of (11) is available by ignoring both D̃_jj and b̃_nj. This is because D_jj ≥ 0 is bounded and equals 0 only when γ_0j = 0. Asymptotically, lim_n→∞ D_jj = I{γ_0j ≠ 0}. Moreover, the bias term b_nj is o_p(1) with exponential convergence for estimates of the nonzero components in γ₀ and O_p{ln(n₀)/√n} for estimates of its zero components. Thus an asymptotic (1 – α) × 100% confidence interval for γ_0j can be simply given as

$${\stackrel{\sim }{\gamma }}_j\pm z_{1-\alpha /2}\sqrt{{({\mathbf{I}}_n^{-1}(\stackrel{\sim }{\mathit{\gamma }})/n)}_{jj}}.$$ (12)

Significance testing on γ₀ can be done in a similar manner.

4. Numerical Studies

In this section, we first discuss numerical and optimization issues in implementing MIC, then present simulation studies that are designed to assess the performance of MIC and compare it to other available methods. We also explore the standard error formula for nonzero components in β̃ and inference on β via γ. Finally, a real data example illustration is provided via analysis of the PBC data. Additional numerical results are presented in the Supplementary Materials.

4.1 Implementation Issues

The asymptotic results in Section 2.3 entail a_n = O_p(n). In all the reported numerical results throughout the article, we have set a_n = n₀, i.e., number of observed deaths in the data, because $n_0/n\stackrel{p}{\to }\text{Pr}\{C^{\prime }\ge T^{\prime }\}$ by WLLN. In summary, MIC have the following simple form

$$\underset{\mathit{\gamma }}{min}-2l(\mathit{\beta })+ln(n_0)\sum _{j=1}^p{w}_j.$$ (13)

where $w_j=tanh(n_0{\gamma }_j^2)$ and β = (β_j) = (γ_jw_j). We would like to emphasize that a is not a tuning parameter as important as the penalty parameter λ. In fact, the MIC estimate stays rather invariant with the choice of a, as demonstrated with additional numerical results presented in the Supplementary Materials. Comparatively, a small change in λ can dramatically change the final estimate and hence fine tuning is necessary in other regularization methods.

The MIC formulation of (13) leads to a smooth programming problem. Nevertheless, the unit dent function is non-convex in nature. This implies that (13) may have multiple local optima. In our implementation, we tried to make efficient use of readily available optimisation routines. We have found that simulated annealing (SA; Belisle, 1992) followed by a BFGS quais-Newton algorithm (see, e.g., Nocedal and Wright, 1999), both implemented in R (R Development Core Team, 2015) function optim(), is quite efficient and effective in computing MIC estimators. Simulated annealing is a global optimisation technique that helps seek the global optimum. Succeeding SA with the BFGS method makes sure that the final estimate converges to a critical point.

4.2 Simulation Results

For the convenience of comparison, we have simulated data from the same models as in Zhang and Lu (2007). A total of p = 9 covariates (Z₁, …, Z₉)^T are generated from a multivariate normal distribution MVN_p (0, Σ), where the covariance matrix Σ is given by

$$\sum =({\sum }_{j{j}^{\prime }})\text{with element}{\sum }_{j{j}^{\prime }}={0.5}^{|j-j^{\prime }|}\text{for}j,j^{\prime }=1,\dots ,9.$$ (14)

Two models, A and B, were considered with true regression coefficients

$$\text{Model A}:\mathit{\beta }={(-0.7,-0.7,0,0,0,-0.7,0,0,0)}^T$$

$$\text{Model B}:\mathit{\beta }={(-0.4,-0.3,0,0,0,-0.2,0,0,0)}^T,$$

corresponding to larger and smaller effects, respectively. Two censoring rates, 25% and 40%, and three sample sizes n = 100, 200, and 300 are experimented. For each simulated data set, seven methods were applied: the oracle model, the full model, the best subset selection; MIC; LASSO with minimum GCV selection of λ; ALASSO; and SCAD with BIC selection. All the computations were conducted in R (R Development Core Team, 2015). We documented how each method was implemented in the Supplementary Materials.

For performance measures, we reported the mean weighted squared error (MSE) (γ̂ – γ)^TΣ(γ̂–γ) with Σ given by (14), the averaged model size (i.e., number of nonzero parameter estimates), and percentage of correct selection. Table 1 presents the results based on 100 simulation runs. It can be seen that MIC performs similarly to BSS. However, BSS (even backward deletion) becomes infeasible for moderately large p. More elaboration on this point will be made in the comparison of computing time (see Section B.1 in the Supplementary Materials). Compared to the regularisation methods, MIC performs better in terms of all measures in Model A (the case with stronger signals). With weaker signals (Model B), all methods perform poorly when n = 100. As sample size increases, their performances all improve. MIC compares favorably to others in terms of the correct selection rate, but less favorably to LASSO or ALSSO in terms of MSE. This can be explained by the fact that MIC is aimed to achieve minimum BIC via approximation and BIC works best with relatively large samples and stronger signals (see, e.g., McQuarrie and Tsai, 1998).

Table 1.

Comparison of selection methods. Data are generated from Models A & B with 100 simulation runs. Three criteria are used: the mean squared error (MSE); the average model size (Size); and the percentage of correct selections (Correct%).

		Model A						Model B
		Censoring Rate =25%			Censoring Rate =40%			Censoring Rate =25%			Censoring Rate =40%
n	Method	MSE	Size	Correct%	MSE	Size	Correct%	MSE	Size	Correct%	MSE	Size	Correct%
100	oracle	0.0672	3.00	100	0.0947	3.00	100	0.0542	3.00	100	0.0832	3.00	100
	full	0.2247	9.00	0	0.2948	9.00	0	0.1830	9.00	0	0.2389	9.00	0
	stepwise	0.1198	3.31	72	0.1690	3.32	69	0.1440	2.08	9	0.1851	1.95	4
	MIC	0.1108	3.43	72	0.1543	3.35	69	0.1469	2.06	8	0.1835	1.95	8
	LASSO	0.1456	5.42	10	0.1989	5.51	8	0.1000	4.20	6	0.1424	3.99	3
	ALASSO	0.1178	4.25	35	0.1632	3.97	45	0.1105	3.47	8	0.1572	3.42	4
	SCAD	0.1427	3.44	63	0.1644	3.28	58	0.1233	3.20	11	0.1455	2.70	15
200	oracle	0.0341	3.00	100	0.0459	3.00	100	0.0239	3.00	100	0.0294	3.00	100
	full	0.0863	9.00	0	0.1146	9.00	0	0.0744	9.00	0	0.0980	9.00	0
	stepwise	0.0412	3.18	84	0.0582	3.22	81	0.0644	2.49	32	0.0886	2.42	21
	MIC	0.0428	3.21	80	0.0741	3.49	67	0.0662	2.75	32	0.0812	2.60	24
	LASSO	0.0627	5.45	8	0.1011	5.76	6	0.0529	5.11	7	0.0671	4.93	10
	ALASSO	0.0462	3.97	49	0.0783	4.19	43	0.0554	4.14	15	0.0690	3.77	16
	SCAD	0.0589	4.10	53	0.0626	3.81	62	0.0637	4.16	15	0.0814	3.75	13
300	oracle	0.0208	3.00	100	0.0267	3.00	100	0.0153	3.00	100	0.0208	3.00	100
	full	0.0562	9.00	0	0.0666	9.00	0	0.0469	9.00	0	0.0644	9.00	0
	stepwise	0.0267	3.17	85	0.0331	3.16	86	0.0355	2.79	55	0.0529	2.70	39
	MIC	0.0279	3.26	78	0.0343	3.22	81	0.0316	2.93	61	0.0495	2.67	39
	LASSO	0.0471	5.60	6	0.0635	5.68	7	0.0288	5.18	9	0.0419	5.20	6
	ALASSO	0.0298	3.69	57	0.0445	4.03	51	0.0305	4.27	27	0.0428	4.05	22
	SCAD	0.0325	3.78	61	0.0402	3.88	59	0.0333	3.87	36	0.0479	3.89	22

To investigate the post-selection SE formula for non-zero estimates, we compare the actual standard deviation (ASD) of estimated β̃_j with the mean SE estimates over 500 simulation. The results are presented in Table 2, together with the coverage probability. It can be seen that the mean SE values are close to the ASD values in most scenario, except in the case of weak signal (Model B) with small sample size (n = 100), where the asymptotic SE is smaller than the ASD to a substantial amount. This is the scenario where MIC does poorly in selecting variables due to the use of BIC. The SE formula performs reasonably well in all scenarios in terms of empirical coverage probabilities, which are all close to the nominal confidence level 95%.

Table 2.

Standard errors for nonzero MIC estimates. The actual sample standard deviation (ASD) of each β̃_j, the mean of the asymptotic standard errors (SE–Mean), and the coverage probability (CP) of the 95% confidence intervals are obtained from 500 simulation runs.

			Censoring 25%			Censoring 40%
	n		ASD	SE–Mean	CP	ASD	SE–Mean	CP
Model A	100	β̃1	0.174	0.164	93.19%	0.214	0.183	91.55%
		β̃2	0.195	0.168	92.15%	0.224	0.190	92.94%
		β̃6	0.185	0.156	92.38%	0.223	0.177	92.14%
	300	β̃1	0.091	0.089	94.60%	0.102	0.099	94.20%
		β̃2	0.094	0.091	95.20%	0.113	0.102	92.40%
		β̃6	0.095	0.084	92.60%	0.114	0.095	91.20%
Model B	100	β̃1	0.187	0.146	91.79%	0.228	0.163	91.18%
		β̃2	0.214	0.150	91.30%	0.232	0.167	90.34%
		β̃6	0.202	0.139	84.76%	0.209	0.157	88.57%
	300	β̃1	0.086	0.083	94.80%	0.096	0.092	95.00%
		β̃2	0.098	0.083	93.48%	0.109	0.094	96.07%
		β̃6	0.099	0.075	94.17%	0.113	0.084	93.29%

To assess the γ-based inference procedure as proposed in Section 3, we recorded the 95% confidence intervals for each individual γ_j and the p-values associated with the Wald test of H₀ : γ_j = 0. Since MIC is really fast, we have increased the number of simulation runs to 500. Table 3 presents the coverage probability (CP) of the 95% confidence interval, as well as the empirical power (EP) for testing on nonzero coefficients at the significance level α = 0.05. It can be seen that the coverage probabilities of the confidence intervals are around the nominal level of 95% in all scenarios. The proposed significance testing procedure also performs reliably in terms of empirical powers, although its performance deteriorates with smaller sample sizes and weak signals as expected.

Table 3.

Inference on β via reparameterised γ in MIC. The coverage probability (CP) of the 95% confidence interval for each individual parameter and the empirical power (EP) at level α = 0.05 are based on 500 simulation runs.

		Model A				Model B
Censoring		n = 100		n = 300		n = 100		n = 300
Rate		CP	EP	CP	EP	CP	EP	CP	EP
25%	γ1	93.6%	99.8%	95.2%	100.0%	94.2%	78.6%	95.6%	100.0%
	γ2	96.0%	98.8%	95.4%	100.0%	95.6%	41.6%	97.4%	90.8%
	γ3	95.4%		96.8%		93.4%		94.0%
	γ4	96.4%		97.0%		96.4%		96.2%
	γ5	96.2%		98.0%		95.4%		95.4%
	γ6	96.4%	99.0%	96.4%	100.0%	97.6%	18.2%	97.2%	62.4%
	γ7	94.6%		96.2%		96.4%		93.2%
	γ8	95.2%		97.0%		95.8%		95.0%
	γ9	91.6%		97.0%		95.0%		95.0%
40%	γ1	94.8%	95.8%	92.8%	100.0%	93.6%	73.8%	95.2%	99.0%
	γ2	94.0%	95.8%	96.6%	100.0%	95.2%	40.0%	97.8%	87.0%
	γ3	95.4%		96.8%		96.0%		96.8%
	γ4	95.8%		96.6%		93.2%		96.8%
	γ5	96.2%		95.8%		94.4%		95.2%
	γ6	96.4%	95.6%	96.6%	100.0%	96.8%	16.8%	99.2%	51.4%
	γ7	96.8%		95.4%		95.2%		93.2%
	γ8	97.0%		97.0%		96.8%		97.0%
	γ9	96.8%		96.6%		95.0%		96.2%

4.3 Data Example: A Revisit to PBC Data

To a real data example illustration, we pay a revisit to the primary biliary cirrhosis (PBC) data set that is well-known in the survival analysis literature. Another illustration is provided in the Supplementary Materials (Section B.3) where we applied the proposed methods to a gene expression data set. A description of the PBC study, which has been omitted here, can be found in Dickson et al. (1989). This data set has been analysed by many authors including both Tibshirani (1997) and Zhang and Lu (2007). Note that the results presented in Tibshirani (1997; Table I on p.390) are based on the standardised predictors while the estimated coefficients in Zhang and Lu (2007; Table 5 on p.699) have been transformed back to the original scales. The results from MIC and several other methods, as presented in Table 4, were based on standardised data.

Table 4.

Analysis of PBC data. The p-value in MIC is computed via the reparameterised γ̃ as discussed in Section 3.

	Full Model		MIC			Stepwise
	γ̂	SE	β̃	SE	P-Value*	γ̂	SE	LASSO	ALASSO	SCAD
trt	−0.062	0.108	0.000		1.0000
age	0.304	0.123	0.331	0.107	0.0067	0.330	0.107	✓	✓	✓
sex	−0.120	0.103	0.000		1.0000			✓
ascites	0.022	0.098	0.000		1.0000			✓
hepato	0.013	0.126	0.000		1.0000
spiders	0.046	0.111	0.000		1.0000			✓
edema	0.273	0.107	0.222	0.094	0.0368	0.222	0.094	✓	✓	✓
bili	0.368	0.117	0.391	0.089	0.0006	0.392	0.089	✓	✓	✓
chol	0.116	0.104	0.000		1.0000					✓
albumin	−0.300	0.125	−0.290	0.110	0.0201	−0.291	0.110	✓	✓	✓
copper	0.220	0.103	0.252	0.087	0.0165	0.252	0.087	✓	✓	✓
alk.phos	0.002	0.084	0.000		1.0000
ast	0.231	0.111	0.248	0.103	0.0276	0.248	0.102	✓	✓	✓
trig	−0.064	0.087	0.000		1.0000
platelet	0.084	0.110	0.000		1.0000
protime	0.234	0.107	0.229	0.102	0.0283	0.229	0.102	✓	✓
stage	0.388	0.150	0.369	0.124	0.0124	0.370	0.124	✓	✓	✓

MIC selects eight variables, which are the same as those selected by the stepwise selection and ALASSO. The SCAD model has eight variables too, yet with slightly different choices. The LASSO model is much larger, having 11 selected variables. The final MIC model fit is nearly identical to the one resulted from the stepwise selection, indicating again that MIC mimics the best subset selection method well. The individual parameter testings based on reparameterised γ in MIC, free of post-selection inference, also support the selected variables.

5. Discussion

MIC offers a new perspective for conducting sparse estimation by approximating a model selection criterion. Su (2015) first experimented a preliminary version of this method in linear regression for the variable selection purpose only, while the current research comprehensively examines sparse estimation within the context of Cox PH models. The main advantages of MIC are summarized as follows. First of all, MIC is free of tuning owing to its special formulation. As a result, MIC is computationally more efficient than many other competitors. MIC only entails the same level of computational complexity as what one would encounter in computing the MPLE. Secondly, BIC is optimal not only for its selection consistency but also because it is derived as an approximation to the posterior distribution of candidate models. The latter property renders the penalty parameter in BIC, i.e., λ₀ = ln(n₀), unique in some sense. This is why BIC is often used as an ultimate yardstick in many variable selection procedures. MIC mimics BSS but extends well to large p scenarios. MIC is also advantageous to regularization methods as it seeks to optimize an approximated BIC without reducing the search space. Even if the fitting algorithm does not guarantee to identify the true global optimum, the final MIC result should correspond to a competitive model with a relatively small BIC. Thirdly, the reparameterisation step not only yields computational advantages but also facilitates a leeway to circumvent the fundamental post-selection inference problem.

While all our discussions in this paper have been restricted to fixed finite dimensions, the general approximation idea of MIC may be extended to scenarios with diverging number of parameters (i.e., p → ∞ yet p/n → 0) or ultra-high dimensions with p ≫ n, where various extended, modified, or generalized information criteria are available as pioneered by Chen and Chen (2008) and others. As a part of our ongoing research efforts, we are investigating how to obtain a modified information criteria for high-dimensional Cox PH models so that MIC can be readily applied. For future research, MIC can also be possibly applied to other survival models (e.g., accelerated failure time models and frailty models) and various sparsity structures (e.g., grouped or fused LASSO).