BAYESIAN VARIABLE SELECTION FOR SURVIVAL DATA USING INVERSE MOMENT PRIORS

Abstract

Efficient variable selection in high dimensional cancer genomic studies is critical for discovering genes associated with specific cancer types and for predicting response to treatment. Censored survival data is prevalent in such studies. In this article we introduce a Bayesian variable selection procedure that uses a mixture prior composed of a point mass at zero and an inverse moment prior in conjunction with the partial likelihood defined by the Cox proportional hazard model. The procedure is implemented in the R package BVSNLP, which supports parallel computing and uses a stochastic search method to explore the model space. Bayesian model averaging is used for prediction. The proposed algorithm provides better performance than other variable selection procedures in simulation studies, and appears to provide more consistent variable selection when applied to actual genomic datasets.

1. Introduction.

Recent developments in sequencing technology have made it easier to collect massive genomic datasets that can be used to study cancer and other diseases. Given such data, there is great interest in linking genomic data to patient outcomes, and in many cases such outcomes are censored survival times.

Survival times for patients generally represent either the time to death or disease progression, the time to study termination, or the time until the subject is lost to follow up. In the latter cases, the subject’s survival time is censored. The relation between survival times and covariates is modeled through the conditional hazard function, which is the limiting probability of death in the interval (t, t+Δt) as Δt becomes small, given patient covariates. More precisely, the hazard function h for patient i may be defined as

$$h\left(t|{\mathbf{\text{x}}}_i\right)=\underset{\Delta t\to 0}{\text{lim}}\frac{1}{\Delta t}P\left(t\le T\le t+\Delta t|T\ge t,{\text{x}}_i\right),$$ (1.1)

where x_i is a p vector of covariates thought to influence survival. We denote by X the n × p design matrix obtained by stacking n patient covariate vectors. Proportional hazard models take the form

$$h\left(t|x_i\right)=h_0(t)\mathbf{\Phi }\left(x_i\right),$$ (1.2)

with an identifiability constraint of Φ(0) = 1. In this formula, h₀(t) denotes the baseline hazard function. The Cox proportional hazards model (Cox, 1972) is defined by taking $\mathbf{\Phi }\left(x_i\right)=\text{exp}\left\{x_i^T\beta \right\}$, leading to

$$h\left(t|x_i\right)=h_0(t)e^{x_i^T\beta }.$$ (1.3)

Here, β is a p × 1 vector of coefficients.

An important feature of the proportional hazards model is that it yields a partial likelihood function that is independent of the baseline hazard function, h₀. For complete survival analyses, however, the baseline hazard function is necessary for predicting survival times and can be estimated non-parametrically. Further details regarding the Cox proportional hazard model may be found in Cox and Oakes (1984), Kalbfleisch and Prentice (1980) or Cox (1972).

Gene expression datasets usually contain measurements on thousands of genes collected for only hundreds of subjects. Biologically it seems plausible that only a relatively small number of these genes contribute significantly to survival. This implies that most of the elements in the vector β are small or are close to zero. The challenge is to find covariates with non-zero coefficients or, equivalently, those genes that contribute the most in determining the survival outcome.

Many common penalized likelihood methods originally introduced for linear regression have been extended to survival data. These methods include LASSO (Tibshirani et al., 1997), in which an L₁ penalty is imposed on regression coefficients. Zhang and Lu (2007) utilized adaptive LASSO methodology for time to event data, while Antoniadis, Fryzlewicz and Letué (2010) adopted the Dantzig selector for survival outcomes. The extension of non-convex penalized likelihood approaches, in particular SCAD, to the Cox proportional hazard model is discussed in Fan and Li (2002). The Iterative Sure Independence Screening (ISIS) approach introduced by Fan and Lv (2008) is also extended for ultrahigh dimensional survival data in Fan et al. (2010), where it is used on Cox proportional hazard models and the SCAD penalty is employed for variable selection.

Some Bayesian approaches have also been proposed. Faraggi and Simon (1998) proposed a method based on approximating the posterior distribution of the parameters in the proportional hazard model by defining a Gaussian prior on regression coefficients. A loss function was then imposed to select a parsimonious model. A semi-parametric Bayesian approach was utilized by Ibrahim, Chen and MacEachern (1999), who employed a discrete gamma process for the baseline hazard function and a multivariate Gaussian prior for the coefficient vector. Sha, Tadesse and Vannucci (2006) considered Accelerated Failure Time (AFT) models along with data augmentation to impute failure times. A mixture prior proposed by George and McCulloch (1997) was used to impose sparsity. In more recent work, Held, Gravestock and Sabanés Bové (2016) proposed the use of a g-prior model for the coefficient vector and employed test-based Bayes factors (Johnson, 2005) to the Cox proportional hazard models. However, this method is intended for use only when the number of covariates is less than the number of observations; that is, when p < n.

To our knowledge, all previous Bayesian procedures for variable selection in survival data have used local priors on model coefficients. In this article, we propose a Bayesian method based on a mixture prior comprised of a point mass at zero and a nonlocal prior on the regression coefficients. To handle the computational burden of implementing the resulting procedure we employ a stochastic search method, S5 (Shin, Bhattacharya and Johnson, 2018), which we implement in an R package BVSNLP. We also discuss a general procedure for setting the tuning parameter of the nonlocal prior.

This article is structured as follows. In Section 2 we introduce notation, and discuss the modeling of the problem in a Bayesian framework. Section 3 discusses the proposed method, with details of parameter selection, model search, and assessment of the accuracy of the proposed variable selection procedure. Sections 4 and 5 provide simulation and real data analyses with various predictive performance measures to demonstrate how the proposed method compares to several other competing methods. Section 6 concludes with discussion.

2. Problem Modeling.

2.1. Preliminaries.

Let T_i denote the survival time and C_i denote the censoring time for individual i. Each element in the observed vector of survival times, y, is defined as y_i = min{T_i, C_i}. The status for each individual is defined as δ_i = I(T_i ≤ C_i). The status vector is represented by δ = (δ₁, δ₂,…,0 δ_n)^T. We assume that the censoring mechanism is at random, meaning that C_i and T_i are conditionally independent given x_i, where $x_i\in {ℝ}^p$ are the covariates for individual i and comprise the i^th row of X. The observed data is of the form {(y_i, δ_i, x_i); i = 1, 2,…,n}.

Model k is defined as k = {k₁,…, k_j} where (1 ≤ k₁ < · · · < k_j ≤ p) and it is assumed that ${\beta }_{k_1}\ne 0,\dots ,{\beta }_{k_j}\ne 0$ and all other elements of β are 0. The design matrix corresponding to model k is denoted by X_k, and the regression vector by ${\beta }_k={\left({\beta }_{k_1},{\beta }_{k_2},\dots ,{\beta }_{k_j}\right)}^T$.

Let $\mathcal{R}(t)=\left\{i:y_i\le t\right\}$ represent the risk set at time t, the set of all individuals who are still present in the study at time t and are neither dead nor censored. We assume throughout this article that the failure times are distinct. In other words, only one individual fails at a specific failure time. With this assumption and letting ${\xi }_{ki}=\text{exp}\left\{X_{k_i}^T{\beta }_k\right\}$, the partial likelihood (Cox, 1972) for β_k in model k can be written as

$$L_p\left({\beta }_k\right)=\prod _{i=1}^n{\left[\frac{{\xi }_{k_i}}{\sum _{j\in R\left(y_i\right)}{\xi }_{k_j}}\right]}^{{\delta }_i}.$$ (2.1)

Our method uses this partial likelihood as the sampling distribution in our Bayesian model selection procedure. We acknowledge that there is some information loss in (2.1) with respect to β_k. For instance, Basu (Ghosh, 1988) argues that partial likelihoods cannot usually be interpreted as sampling distributions. On the other hand, Berger et al. (1999) encourage the use of partial likelihoods when the nuisance parameters are marginalized out.

Sorting the observed unique survival times in ascending order and consequently re-ordering the status vector δ as well as the design matrix X with respect to the ordered y, the sampling distribution of y for model k can be written as

$$\pi \left(y|{\beta }_k\right)=\prod _{i=1}^n{\left[\frac{e^{X_{k_i}^T{\beta }_k}}{\sum _{j=i}^n{e}^{X_{k_j}^T{\beta }_k}}\right]}^{{\delta }_i}.$$ (2.2)

A Bayesian hierarchical model can be defined in which π(y |β_k) in (2.2) represents the sampling distribution, π_k(β_k) is the prior of model coefficients β_k and p(k) is the prior for model k. Using Bayes rule, the posterior probability for model j is written as

$$p(j|y)=\frac{p(j)m_j(y)}{{\sum }_{k\in \mathcal{J}}p(k)m_k(y)},$$ (2.3)

where $\mathcal{J}$ is the set of all possible models and the marginal probability of the data under model k is defined by

$$m_k(y)=\int \pi \left(y|{\beta }_k\right){\pi }_k\left({\beta }_k\right)d{\beta }_k.$$ (2.4)

The prior density for β_k and the prior on the model space impact the overall performance of the selection procedure and the amount of sparsity imposed on candidate models. Note that the sampling distribution in (2.2) is continuous in β_k, and in Section 2.3 we define an inverse moment prior (Johnson and Rossell, 2010) on each of the coefficients in model k.

2.2. Prior on Model Space.

Let γ_k = {γ₁, · · · , γ_p} denote a binary vector indicating which covariates are included in model k. Suppose the size of model k is k. That is, there are k nonzero indices in γk. The nonzero indices of γk represent the indices of the nonzero elements in the coefficient vector, β, which a priori are modeled as independent Bernoulli random variables with success probability P (γ_i = 1) = θ for every 1 ≤ i ≤ p. As discussed in Scott et al. (2010), no fixed value for θ adjusts for multiplicity. As a result, it is necessary to define a prior on θ, say π(θ). The resulting marginal probability for model k in a fully Bayesian approach may then be written as

$$p(k)\propto \int {\theta }^k{(1-\theta )}^{p-k}\pi (\theta )d\theta .$$ (2.5)

A common choice for π(θ) is the beta distribution, θ ~ Beta(a, b), where in the special case of a = b = 1, π(θ) is a uniform distribution. The marginal probability for model k derived from (2.5) is then equal to

$$p(k)=\frac{B(a+k,b+p-k)}{B(a,b)},$$ (2.6)

where B(·) is the Beta function. A priori, the model size, k, thus follows a Beta-binomial distribution. By choosing b = p − a, the mean and variance of the selected model size k is

$$\mathbb{E}(k)=a,\text{\hspace{1em}Var}(k)=\frac{2p^{2}a(p-a)}{p^3}\approx 2a.$$ (2.7)

The approximation in the variance formula follow from a large p and a fairly small a under the sparsity assumption on the true model size. To incorporate the belief that the optimal predictive models are sparse, we recommend setting a = 1 and b = p − a. The resulting prior assigns comparatively small prior probabilities to models that contain many covariates.

2.3. Product Inverse MOMent (piMOM) Prior.

We impose nonlocal prior densities on the non-zero coefficients, β_k. Specifically, we assume the prior densities on the non-zero coefficients in model k take the form of a product of independent iMOM priors, or piMOM densities (Johnson and Rossell, 2012), expressible as

$$\pi \left({\beta }_k|\tau ,r\right)=\frac{{\tau }^{rk/2}}{\Gamma {(r/2)}^k}\prod _{i=1}^k{|{\beta }_i|}^{-(r+1)}\text{exp}\left(-\frac{\tau }{{\beta }_i^2}\right),r,\tau >0.$$ (2.8)

The hyperparameter τ represents a scale parameter that determines the dispersion of the prior around 0, while r determines the tail behavior of the density. These priors have two symmetric modes with Cauchy-like tails when r = 1, and assign negligible probability to a region around zero. In comparison to local priors, this characteristic of nonlocal priors potentially leads to smaller false positive rates in selection procedures by discouraging the selection of variables with small coefficients. On the other hand, piMOM priors possess Cauchy-like tails, which introduce comparatively small penalties on large coefficients. Unlike many penalized likelihood methods, large values of regression coefficients are thus not heavily penalized by these priors. As a result, they do not necessarily impose significant penalties on non-sparse models provided that the estimated coefficients in those models are not small. For these reasons, piMOM priors work well as a default choice of priors on non-negligible coefficients in variable selection problems.

Consistency properties of piMOM priors for linear models were studied in Shin, Bhattacharya and Johnson (2018). In that setting it was shown that piMOM priors are consistent for $p=O\left(e^{n^{\alpha }}\right)$, 0 < α < 1. By comparison, this property does not hold for pMOM priors. An example of an iMOM prior is depicted in Figure 4 for r = 1 and τ = 0.5. The source of inconsistency for pMOM priors in p » n settings stems from the fact that their densities go to zero only at an inverse polynomial rate in a neighborhood of the origin.

Fig 4:

iMOM and MOM prior with r = 1 and τ = 0.5.

3. Methods.

3.1. Selection of Hyperparameters.

We use the procedure described in Nikooienejad, Wang and Johnson (2016) to select hyperparameter values for the piMOM prior. In that method, the null distribution of the maximum likelihood estimator for β_k (i.e., all components of β_k are 0), obtained from randomly selected design matrices X_k, is compared to the prior density on β_k for various values of (r, τ). Fixing r = 1 to achieve Cauchy-like tails, a value of τ is chosen so that the overlap between the two densities is less than a specified threshold, $1/\sqrt{p}$, and is denoted by τ₁. It can be shown that the maximum of the iMOM prior occurs at $\pm \sqrt{\tau }$. We also allow users to input a prior parameter α that controls where the modes in the prior occur. This can be useful in constraining the prior density when covariates are highly correlated (resulting in an over-dispersed prior when the sampling distribution of the null MLE under the null model becomes overly broad). We then set the value of τ according to

$$\tau =\text{min}\left({\tau }_1,{\alpha }^2\right).$$ (3.1)

To implement the procedure for computing τ₁ for survival models, we generate response vectors under the null model using the procedure described by Bender, Augustin and Blettner (2005). Survival times are sampled from a standard exponential model.

Let t^s and c^s be the vector of sampled survival times and censoring times, respectively. The sampled survival time and status for each observation is then computed as

$$y_i^s=\text{min}\left\{t_i^s,c_i^s\right\}\text{\hspace{1em}and\hspace{1em}}{\delta }_i^s=I\left(t_i^s\le c_i^s\right),$$ (3.2)

which comprise y^s and δ^s under the null model. Using the pair (y^s, δ^s), the MLE from Cox model is computed. It should be noted that the asymptotic distribution of the MLE for the Cox model under the null hypothesis is $\widehat{\beta }~\mathcal{N}(0,I(\widehat{\beta }))$, where I(β) is the information matrix of the partial likelihood function. Thus, it is appropriate to approximate the pooled estimated coefficients in that algorithm with a normal density function. When the sample size gets large, the variance of the MLE decreases and causes the overlap to become small and consequently small values of τ are selected.

In general, we find that r = 1 and τ = 0.25 are good default values if one chooses not to run the hyperparameter selection algorithm. When r = 1, the peaks of the iMOM prior occur at $-\sqrt{\tau }$ and $\sqrt{\tau }$. By equating $\sqrt{\tau }$ to the absolute value of the expected effect size for a given application, insight can be gained on what value of τ is appropriate. Further details regarding this algorithm can be found in Nikooienejad, Wang and Johnson (2016).

3.2. Computing Posterior Probability of Models.

Computing the posterior probability for each model requires the marginal probability of observed survival times under each model as shown in (2.3), (2.4). The marginal probability is approximated by using the Laplace approximation, where the regression coefficients in β_k are integrated out. This leads to

$$m_k\left(y_n\right)=\pi \left(y_n|{\widehat{\beta }}_k\right)\pi \left({\widehat{\beta }}_k\right){(2\pi )}^{k/2}{|G_{{\widehat{\beta }}_k}|}^{-1/2}.$$ (3.3)

Here, ${\widehat{\beta }}_k$ is the maximum a posteriori (MAP) estimate of β_k, $G_{{\widehat{\beta }}_k}$ is the Hessian of the negative of the log posterior function,

$$g\left({\beta }_k\right)=-\text{log}\left(\pi \left(y|{\beta }_k\right)\right)-\text{log}\left(\pi \left({\beta }_k\right)\right),$$ (3.4)

computed at ${\widehat{\beta }}_k$ and k is the size of model k. Finding the MAP of β_k is equivalent to finding the minimum of g(β_k).

The details of computing the gradient and Hessian matrix of g(β_k) are discussed in Section A of the Appendix. The gradient and Hessian matrix, described by equations (A.3) to (A.7), are used to find the MAP, and to compute the Laplace approximation of the marginal probability of y.

We use the limited memory version of the Broyden-Fletcher-Goldfarb-Shanno optimization algorithm (L-BFGS) (Liu and Nocedal, 1989) to find the MAP. The initial value for the algorithm is ${\widehat{\beta }}_k$ the MLE for the Cox proportional hazard model.

Having all the components of formula (2.3), it is possible to define a MCMC framework to sample from the posterior distribution on the model space. A birth-death scheme, similar to that used in Nikooienejad, Wang and Johnson (2016), could be used for this purpose. However, for computational reasons we use another stochastic algorithm to search the model space; this algorithm is described in the next section.

The highest posterior probability model (HPPM) is defined as the model having the highest posterior probability among all visited models. In practice, many models may be assigned probabilities that are close to the probability achieved by the HPPM. For this reason and for predictive purposes, it is useful to obtain the Median Probability Model (MPM) (Barbieri et al., 2004), which is the model containing covariates that have posterior inclusion probabilities of at least 0.5. According to Barbieri et al. (2004), the posterior inclusion probability for covariate i is defined as

$$p_i=\sum _{k:{\gamma }_{k_i}=1}p\left(M_k|y\right).$$ (3.5)

That is, the sum of posterior probabilities of all models that have covariate i as one of their variables. In this expression, γ_ki is a binary value determining the inclusion of the i^th covariate in model k.

3.2.1. Stochastic Search Algorithm.

To increase the efficiency of exploring the model space, we use the S5 algorithm. S5 was proposed by Shin, Bhattacharya and Johnson (2018) for variable selection in linear regression problems, and we adapt it here for survival models. It is a stochastic search method that screens covariates at each step. The algorithm is scalable and its computational complexity is only linearly dependent on p (Shin, Bhattacharya and Johnson, 2018).

Screening is the essential part of the S5 algorithm. In linear regression, screening is based on the correlation between excluded covariates and the residuals of the regression using the current model (Fan and Lv, 2008). The concept of screening covariates for survival response data is proposed in Fan et al. (2010), and is defined based on the marginal utility for each covariate.

To illustrate the screening technique, suppose that the current model is k. Let k^c denote the complement of set k containing columns of the design matrix that are not in the current model, k. The conditional utility of covariate m ∈ k^c represents the amount of information covariate m contributes to the survival outcome, given model k, and is defined as

$$u_{m|k}=\underset{\begin{array}{c}{\beta }_m\\ m\in k^c\end{array}}{\text{max}}{\delta }^T\left[\left({\beta }_m{X}_{(m)}+X_k{\beta }_k\right)-\text{log}\left\{\sum _{j=i}^n\text{exp}\left({\beta }_m{x}_{jm}+X_{k_j}{\beta }_k\right)\right\}\right].$$ (3.6)

By comparing u_m | k to (A.1), it follows heuristically that the conditional utility is the maximum likelihood for covariate m after accounting for the information provided by model k. Finding u_m | k is a univariate optimization procedure that can be computed rapidly.

With this background, the S5 algorithm for survival data works as follows. At each step, the d covariates with highest conditional utility are candidates to be added to the current model k and comprise the addition set, Γ⁺. The deletion set, Γ⁻ contains the current model, except that one variable is removed. From the current model, k, we consider moves to each of its neighbors in Γ⁺ and Γ⁻ with a probability proportional to the marginal probabilities of these neighboring models.

To avoid local maxima, the model probabilities used in S5 are raised to the power of 1/t_l, where t_l is the l^th temperature in an annealing schedule in which “temperatures” decrease. To increase the number of visited models, a specified number of iterations are performed at each temperature. At the end of the procedure, the model with the highest posterior probability of visited models is identified as the HPPM.

In our version of the S5 algorithm, we used 10 equally spaced temperatures varying from 3 to 1 and 30 iterations within each temperature. Section D of the Appendix provides some discussion on how these values are chosen for this application. To increase the number of visited models, we parallelized the S5 procedure so that it could be distributed to multiple CPUs. Each CPU executes the S5 algorithm independently with a different starting model. All visited models are pooled together at the end and the HPPM and MPM are determined. Using posterior probabilities of the visited models, the posterior inclusion probability for each covariate can be computed using (3.5). In our simulations, we used 120 CPUs to explore the model space for design matrices with O(10⁴) covariates.

3.3. Predictive Accuracy Assessment.

In addition to looking at the selected genes and their pathways to determine their biological relevance in analyzing the real data sets, we used the time dependent AUC, obtained from time dependent ROC curves as introduced by Heagerty, Lumley and Pepe (2000) for survival times to summarize and compare the predictive performance of the various algorithms. This measure has a relatively straightforward interpretation, and unlike other summary measures such as the c-index (Harrell Jr et al., 1982), can be computed without requiring specific conditions or additional assumptions to hold (Blanche, Kattan and Gerds, 2018). However, predictive performance measures including the c-index, Integrated Brier Score (IBS)(Gerds and Schumacher, 2006), and prediction error curves are investigated and reported in Sections 4 and 5 for both simulation and real data sets.

There are different methods to estimate time dependent sensitivity and specificity. In our algorithm, we adapted a method proposed by Uno et al. (2007), henceforth called Uno’s method. In that method, after splitting data into training and test sets, sensitivity is estimated by and specificity is estimated by

$${\widehat{\text{SE}}}_k(t,c)=\frac{{\sum }_{i=1}^n{\delta }_{i}I\left(X_{k_i}^T{\widehat{\beta }}_k>c,T_i\le t\right)/\widehat{G}\left(T_i\right)}{{\sum }_{i=1}^n{\delta }_{i}I\left(T_i\le t\right)/\widehat{G}\left(T_i\right)},$$ (3.7)

and specificity is estimated by

$${\widehat{\text{SP}}}_k(t,c)=\frac{{\sum }_{i=1}^{n}I\left(X_{k_i}^T{\widehat{\beta }}_k\le c,T_i>t\right)}{{\sum }_{i=1}^{n}I\left(T_i>t\right)}.$$ (3.8)

These values are estimated for the test set. Therefore, in the equations above, n is the number of observations in the test set, δ_i is the status of observation i and T_i is the observed time for that observation in the test set. The variable c is the discrimination threshold that is varied to obtain the ROC curve. The function $\widehat{G}$ is the Kaplan-Meier estimate of the survival function obtained from the training set. For each observation i in the test set with observed time T_i, $\widehat{G}\left(T_i\right)$ is computed by a basic interpolation procedure. That is,

$$\widehat{G}\left(T_i\right)=\widehat{G}\left(T_{tr}^{*}\right),\text{\hspace{1em}where }{T}_{tr}^{*}=\underset{T\in T_{tr}}{\text{argmin}}\left|T-T_i\right|.$$ (3.9)

Here, T_tr is the set of all observed survival times in the training set. In (3.7) and (3.8), $\widehat{\beta }$ represents the estimated coefficient under a specific model.

3.3.1. Bayesian Model Averaging (BMA).

BMA can be used to improve the predictive accuracy by accounting for the uncertainty in selected models. From (3.7) and (3.8) the final sensitivity and specificity using BMA may be defined as

$${\widehat{\text{SE}}}_{\text{BMA}}(t,c)=\sum _{j=1}^{\mathcal{N}}{\widehat{\text{SE}}}_{k^j}(t,c)p\left({\mathcal{M}}_{k^j}|y_n\right),$$ (3.10)

and

$${\widehat{\text{SP}}}_{\text{BMA}}(t,c)=\sum _{j=1}^{\mathcal{N}}{\widehat{\text{SP}}}_{k^j}(t,c)p\left({\mathcal{M}}_{k^j}|y_n\right),$$ (3.11)

where, $p\left({\mathcal{M}}_{k^i}|y_n\right)$ is the posterior probability of model ${\mathcal{M}}_{k^i}$ The value of $\mathcal{N}$ depends on what type of BMA is used. We use Occam’s window, which means only models that have posterior probability of at least $w\times p\left({\mathcal{M}}_{\mathit{\text{HPPM}}}|y_n\right)$ are used in model averaging. We set w = 0.01 for our applications.

In the proposed method, individual survival curves are estimated using the highest posterior probability model. Section B provides the details of this procedure. Similar approaches were also adopted by Held, Gravestock and Sabanés Bové (2016) in estimating the survival curve for each individual in a study.

4. Simulation Results.

To investigate the performance of the proposed model selection procedure, we applied our method to simulated datasets. To design different simulation cases we followed the guidance of Morris, White and Crowther (2019) as a basis for our simulation study protocol. In particular, the simulation design was based on the ADEMP structure (Aims, Data generating mechanism, Estimands, Methods, and Performance measures) discussed in that article. We refer to each of those elements as we explain different parts of the simulation design in the following.

Regarding ‘Methods’, we compared the performance of our algorithm to ISIS-SCAD (Fan et al., 2010) and GLMNET (Friedman, Hastie and Tibshirani, 2010), two of the most highly used algorithms for high dimensional variable selection for survival data. We used the published R packages of those two methods to run the simulations. We also performed a comparison with a case when pMOM priors are used as the prior for nonzero coefficients instead of piMOM priors.

The ‘Aim’ of the simulation study is to compare the performance of our method with the other two methods with respect to the correlation structure between covariates in the design matrix. More specifically, we reported three different simulation settings that consider different combinations of correlation structure, true model size, and the magnitude of true coefficients. This is the basis of our ‘Data generating mechanism’. The correlation structure used in those settings are similar to the simulations reported in Fan et al. (2010).

For Case 1, X1,…,X_p are multivariate Gaussian random variables with mean 0 and marginal variance of 1. The correlation structure is corr(X_i, X₅) = 0 for all i ≠ 4, 5, $\text{corr}\left(X_5,X_4\right)=1/\sqrt{2}$, and corr(X_i, X_j) = 0.5 for i, j ∈ {1,…, p} \ {4, 5}. The size of the true model is 5 with non-zero regression coefficients β₁ = 0.8, β₂ = −0.8, β₃ = 0.8, β₄ = −2.2, β5 = 0.66 and β_i = 0 for i > 5. The number of observations and covariates are n = 400 and p = 1000. The censoring rate for this case is approximately 27:6%. The survival and censoring times are both sampled from an exponential distribution. The rate parameter for the distribution of censoring times is set to 0:1.

For Case 2, X₁,…, X_p are multivariate Gaussian random variables with mean 0 and marginal variance of 1. The correlation structure between variables is corr(X_i, X_j) = 0.5; i ≠ j. The size of the true model is 6 with nonzero regression coefficients β₁ = −0.5140, β₂ = 1.2799, β₃ = −2.5307, β₄ = 0.7164, β₅ = 1.3020, β₆ = −0.8833, and β_i = 0 for i > 6. The number of observations and covariates are n = 400 and p = 1000. In this case, the survival times are sampled from a Weibull distribution with rate parameter λ = 0.1 and shape parameter k = 15. The censoring times are sampled uniformly from [0, 8], and the resulting censoring rate for this case is approximately 14.8%.

For Case 3, the design matrix and correlation structure between variables is the same as Case 2, where corr(X_i, X_j) = 0.5, i ≠ j. The size of the true model is 20 with non-zero regression coefficients β₁ = −1.6702, β₂ = −1.7122, β₃ = 0.2313, β₄ = −1.3798, β₅ = 0.9305, β₆ = −1.3835, β₇ = −1.8575, β₈ = 1.0676, β₉ = 1.2013, β₁₀ = 0.5116, β₁₁ = 0.9871, β₁₂ = −0.7680, β₁₃ = 0.5159, β₁₄ = −0.2649, β₁₅ = −1.7482, β₁₆ = −1.6041, β₁₇ = 1.4234, β₁₈ = −1.2884, β₁₉ = −1.1299, β₂₀ = 1.2569, and β_i = 0 for i > 20.The number of observations and covariates are n = 400 and p = 1000. The censoring rate for this case is approximately 34.1%. The survival and censoring times are both sampled from an exponential distribution. The rate parameter for the distribution of censoring times is set to 0.1.

Each simulation case is then repeated 50 times, n_iter = 50, and each time with different random seed numbers in order to generate different datasets.

The primary targets of our simulation study or the ‘Estimands’, according to Morris, White and Crowther (2019), are identifying the true model as well as estimating the vector of coefficients of the true model. Accordingly, we reported four different quantities as ‘Performance measures’ for those estimands. The first two quantities are the mean l₁ norm of the error in estimating vector of coefficients, and the mean squared error (MSE). The mean l₁ norm is computed as $\frac{1}{n_{\mathit{\text{iter}}}}{\sum }_{i=1}^p\left|{\widehat{\beta }}_i-{\beta }_i\right|$, and the MSE is computed as $\frac{1}{n_{\mathit{\text{iter}}}}{\sum }_{i=1}^p{\left({\widehat{\beta }}_i-{\beta }_i\right)}^2$. The third quantity is the mean model size of the selected models and is denoted by MMS. MTP and MFP denote mean false positive and mean true positive values for each algorithm. Formal definitions of MFP, MTP are provided in Section C in the Appendix.

Table 1 compares the performance of our method, BVSNLP, the default variant type of ISIS-SCAD and GLMNET algorithms. The λ parameter in GLMNET wass picked by cross validation.

Table 1.

Comparison between BVSNLP, ISIS-SCAD and GLMNET for simulation Cases 1, 2, and 3 with n = 400 and p = 1000.

	BVSNLP	ISIS-SCAD	GLMNET
Case 1:
MSE	0.185	0.367	2.314
Mean l1 norm	0.534	0.749	5.719
MMS	5.16	5.68	67.06
MTP	4.98	4.94	4
MFP	0.18	0.74	63.06
Case 2:
MSE	0.058	0.268	0.983
Mean l1 norm	0.464	0.557	3.224
MMS	6	6.22	44.22
MTP	6	6	6
MFP	0	0.22	38.22
Case 3:
MSE	0.302	1.520	2.627
Mean l1 norm	1.915	6.268	13.416
MMS	18.38	16	117.64
MTP	18.34	15.96	19.52
MFP	0.04	0.04	98.12

Table 2 compares the Monte Carlo standard errors (Morris, White and Crowther, 2019) of the MSEs for all three different methods.

Table 2.

Monte Carlo Standard Errors for the MSE of the coefficient vector for all three methods.

	Case 1	Case 2	Case 3
BVSNLP	0.086	0.015	0.021
ISIS-SCAD	0.063	0.014	0.053
GLMNET	0.080	0.025	0.037

In the S5 algorithm, 30 iterations are used within each temperature. The parameter d was chosen as 2⎡log(p)⎤. As described in Section 3.2.1, d represents the number of candidate covariates that are added to the current model to make the addition set, Γ⁺ Each S5 algorithm was run in parallel on 120 CPUs for both simulation cases. The beta-binomial prior was imposed on the model space with a = 1, b = p − a. The hyperparameters of the piMOM prior were selected using the algorithm discussed in Section 3.1 with α = 0.4 for Case 1 and α = 0.8 for Cases 2 and 3, imposed as the prior mode. Finally, the average run-time of the BVSNLP algorithm for the entire simulation is summarized in Table 3.

Table 3.

Average BVSNLP run time over 50 iterations in each simulation case using 120 CPUs.

	Case 1	Case 2	Case 3
Run time (seconds)	21.5	19.0	23.7

As demonstrated in Table 1, our method performs better than the other two methods according to all selected metrics, regardless of the size of the true model. The di erence between BVSNLP and ISIS-SCAD is best illustrated as the size of the true model increases. GLMNET has significantly higher mean false positive rates than the other two methods.

Figures 1, 2, and 3 compare the average IBS over 50 iterations between the methods discussed above. IBS is computed using the R package pec (Mogensen, Ishwaran and Gerds, 2012) based on a five-fold cross validation. A benchmark model based on Kaplan-Meier estimate, which includes no covariates, is also added to the figures as a reference for the comparison. The average c-index measures for all the methods are also reported in Table 4. The c-index measures are computed based on the method discussed in van Houwelingen and Putter (2011), using the dynpred package in R. Because a new dataset was created at each iteration, it was not possible to get the average prediction errors, due to the fact that the times points where prediction errors change were different for different data sets.

Fig 1:

Average IBS for all methods in simulation case 1.

Fig 2:

Average IBS for all method in simulation case 2.

Fig 3:

Average IBS for all method in simulation case 3.

Table 4.

Average c-index measures over 50 iterations in each simulation case.

	Case 1	Case 2	Case 3
BVSNLP	0.823	0.872	0.939
ISIS-SCAD	0.828	0.872	0.925
GLMNET	0.845	0.911	0.971

As shown in the IBS plots, all three methods perform better than the reference. BVSNLP and ISIS-SCAD have a very similar performance. For Case 3, where the true model has 20 covariates, BVSNLP outperforms the other two methods, whereas in the Case 2, GLMNET has the best performance. The c-index is similar for all methods, and seems to provide a smaller penalty for model size. This feature of the c-index is discussed further in Section 6.

4.1. Comparison With pMOM Prior.

Another nonlocal prior that might be considered as a potential candidate for the prior densities on the non-zero coefficients in model k is the product of independent MOM priors, or the pMOM densities (Johnson and Rossell, 2012), specified by

$$\pi \left({\beta }_k|\tau ,r\right)={(2\pi )}^{-k/2}{\tau }^{(-rk-k/2)}\text{exp}\left(-\frac{{\beta }_k^{\prime }{\beta }_k}{2\tau }\right)\prod _{i=1}^k{\beta }_i^{2r},r\in \mathbb{N},\tau >0.$$ (4.1)

The hyperparameter τ has the same role as in piMOM densities in (2.8) and r is the order of the density. An example of a MOM prior for r = 1 and τ = 0.5 is depicted in Figure 4.

Following the discussion of nonlocal priors in Section 2.3, we again note that for r = 1 and a fixed τ, piMOM densities assign negligible probability to a wider region around zero than do pMOM densities. More specifically, pMOM densities decrease to zero at only an inverse polynomial rate while piMOM densities decrease at a rate that is order exp(−τ/β²), which is much faster. Consequently, smaller false positive rates are expected for procedures based on piMOM priors that those based on pMOM priors. On the other hand, pMOM densities have tails that converge to zero at an exponential rate, while piMOM densities have heavier, Cauchy-like tails. Moreover, the consistency property of piMOM priors discussed previously for linear models does not hold for pMOM priors. For these reasons, piMOM-based procedures are more effective for variable selection in p » n settings.

To better demonstrate the practical importance of these properties, we performed 20 simulation studies where the number of observations and covariates were n = 200 and p = 10, 000. The true model had size 6 with true coefficients equal to (0.5, 0.85, 1.00, 1.50, 1.85, 2.5). The sign of coefficients were chosen randomly with probability 0.5 in each simulation. The columns of the design matrix were multivariate Gaussian random variables with mean 0 and marginal variance of 1. The correlation between every two variables was 0.5. In each of the simulations, we fixed r = 1 and assigned τ = 15 different values in the interval [0.01, 10]. The survival times were simulated from an exponential distribution with mean 10.

The proposed variable selection algorithm was implemented on the simulation data using both pMOM and piMOM priors. Table 5 summarizes the outcome of the selection procedure using different hyperparameter values for both priors. The numbers are averaged over 20 simulations. In that table, MTPR is the mean true positive rate, MFPR is the mean false positive rate and TMP is the proportion of times that the true model was found without any false positives.

Table 5.

Comparison of variable selection outcomes between pMOM and piMOM for different values of hyperparameter,τ, over 20 simulations.

Hyperparameter τ	MTPR(%)		MFPR(%)		TMP
	piMOM	pMOM	piMOM	pMOM	piMOM	pMOM
0.01	100	22.50	0	0.15	1	0
0.2	100	21.67	0	0.12	0.9	0
0.4	99.17	21.67	0.01	0.12	0.9	0
0.6	99.17	20.83	0.01	0.12	0.9	0
0.8	97.50	20.83	0.01	0.12	0.85	0
1.0	95	20.83	0	0.12	0.70	0
1.25	95	20.83	0	0.12	0.70	0
1.6	94.16	20	0	0.12	0.65	0
2.0	93.33	20	0	0.12	0.60	0
2.3	93.33	19.17	0	0.12	0.60	0
3.8	90.83	18.33	0	0.12	0.45	0
5.4	87.50	18.33	0	0.12	0.35	0
6.9	85	18.33	0	0.12	0.30	0
8.4	81.67	18.33	0	0.12	0.20	0
10.0	81.67	18.33	0	0.12	0.20	0

As shown in Table 5, the pMOM model never finds the true model for any of the τ values. Moreover, the average true positive rate for pMOM is always 5 times less than that for piMOM, and the average false positive rate for pMOM is higher than it is for piMOM. This suggests variable selection based on piMOM priors in ultrahigh dimensional settings is likely to perform better than variable selection based on pMOM priors.

5. Application to Real Data.

We applied our method to selected genes associated with patient survival times for two common cancer types using datasets from The Cancer Genome Atlas (TCGA) projects: kidney renal clear cell carcinoma (KIRC) (Cancer Genome Atlas Research Network, 2013) and kidney renal papillary cell carcinoma (KIRP) (Cancer Genome Atlas Research Network, 2016). We compared the performance of our algorithm to ISIS-SCAD (Fan et al., 2010), GLMNET (Friedman, Hastie and Tibshirani, 2010) and Stability Selection (Meinshausen and Bühlmann, 2010). Stability Selection is combined with a high dimensional selection algorithm such as GLMNET and selects the most stable features for a given level of Type I error.

We included patient’s ‘Age’, ‘Gender’ and a clinical stage variable, ‘Stage’, in the design matrix. On the advice of a clinician, the ‘Stage’ variable was developed by combining the histological stage, pathological stage and clinical stage, into one variable that is a summary of how advanced each subject’s cancer was when the tissue sample was taken. ‘Stage’, like ‘Gender’, is a categorical variable but with 3 levels, where ‘Stage i’ represents the i^th class of that variable; ‘Stage 3’ represents the most advanced stage.

To remove stromal contaminations from the gene expression data, the DeMixT algorithm (Wang et al., 2017), was performed on the design matrix and the tumor-specific expression data were used in the analyses for all algorithms.

The predictive performance was measured by a time-dependent AUC, as discussed in Section 3.3, based on a five-fold cross-validation. The observations in each fold were randomly chosen under a constraint which balanced censoring rate between folds. The AUC values were computed for the test set using the model that was obtained by performing variable selection on the training set. The selected covariates for each cancer type were also compared. For our method, we report the covariates associated with the highest posterior probability model. The hyperparameter τ of the piMOM prior was selected using the algorithm in Section 3.1 with α = 0.1 as the maximum of the piMOM prior. This is our choice of α for real datasets. (In simulation data analyses, α was chosen to be 0.8 since the magnitude of non-zero coefficients in the dataset designed by Fan and Li (2002) were unrealistically large for unit variance covariates). The results for each cancer type are discussed in separate sections below. Note that GLMNET has a random output when the hyperparameter is selected by cross-validation. As a result, based on the recommendation of the inventors of that algorithm, we ran it 100 times for each fold and took the average of results as the outcome for that fold.

We treated categorical variables ‘Stage’ and ‘Gender’, as well as the continuous variable ‘Age’ as fixed covariates in our model. However, available ISIS-SCAD and Stability Selection software packages are not able to fix preselected covariates to include in all models. For this reason, dummy variables associated to ‘Stage’ and ‘Gender’ were manually added to the design matrix and were subject to the selection procedure for those procedures.

To run the Stability Selection method, we used the c060 R package (Sill et al., 2014) and the recommended values for function arguments.

5.1. Kidney Renal Clear Cell Carcinoma (KIRC).

The KIRC dataset (Cancer Genome Atlas Research Network, 2013) contains 490 observations with 13,267 covariates, after removing covariates with missing expressions and observations with missing survival times. The censoring rate for this dataset is 66.94%. Table 6 shows the covariates selected by each method. As mentioned previously, GLMNET produces random outputs at each run and therefore for this table, only the output for one of the runs are indicated; other runs produced a similar number of selected covariates.

Table 6.

Selected genes and covariates for KIRC across different variable selection algorithms

BVSNLP	Age	Gender	Stage
	SUDS3	AR
ISIS-SCAD	Stage 3	AR	Age
	HEBP1	ATP2C1	GADD45A
	MTERF2	ADGRL3	GPSM1
	SERPINI1	SP6	ZNF815P
	INAFM2
GLMNET	Stage	AR	Age
	HEBP1	SEC61A2	TRMT6
	PCBP4	FAHD2A	MCM8
	E2F5	SLC5A6	NARF
	RAB28	DONSON	GPSM1
	HACD1	MARS	FASN
	TRAIP	RPL17P50	SLC26A6
	GPR162	INAFM2	ACACA
Stability Selection	Stage 3	AR	Age
	INAFM2

In addition to the categorical covariate ‘Stage’, BVSNLP selects ‘AR’ and ‘SUDS3’ in the HPPM as the most significant covariates in the design matrix. The posterior inclusion probabilities for ‘AR’ and ‘SUDS3’ are 0.80 and 0.08, respectively. The ‘Age’, ‘Gender, and ‘Stage’ were fixed in all models and thus were selected with probability 1. The MAP estimates for the coefficients of ‘Age’, ‘Gender Male’, ‘Stage 2’, ‘Stage 3’, ‘AR’ and ‘SUDS3’ were 0.33, −0.11, 0.45, 1.61, −0.60 and 0.36, respectively. These coefficients indicate that patients with the most advanced stages of cancer had the poorest survival rates, and that a patient with tumor sample characterized as advanced has a hazard rate that was exp(1.61) ≈ 5 times higher than a patient with tumor sample characterized as localized, when all other covariates were the same. These coefficients also show that the hazard rate in females is 1.12 times that in males, and age has an unfavorable impact on the hazard rate, as expected. Moreover, the negative sign for the ‘AR’ gene indicates it has a favorable impact on survival for KIRC. ‘AR’, the Androgen Receptor gene, functions as a steroid-hormone activated transcription factor. It has been well documented that ‘AR’ promotes the progression of renal cell carcinoma (RCC) through hypoxia-inducible factors HIF-2α and vascular endothelial growth factor regulation (Fenner, 2016). The favorable impact of the ‘AR’ gene was also studied by Hata et al. (2017) in bladder cancer. ‘SUDS3’ is a regulatory protein that is part of the SIN3A corepressor complex component that potentially has a role in tumor suppressor pathways through regulation of apoptosis. There was previous evidence of the down-regulation of the SIN3A gene in tumorigenesis of lung cancer (Suzuki et al., 2008).

It is noteworthy that the algorithm selected the same highest posterior probability model for different values of the hyperparameter τ in the range [0.01, 0.9], where there were no constraints on the modes of the piMOM prior. This shows the robustness of the proposed variable selection algorithm to the choice of hyperparameter τ for a range of plausible values.

For this particular run of GLMNET, a much larger model was selected with 24 variables including two of the variables reported by BVSNLP. ISIS-SCAD selected 13 covariates, which included the 4 covariates that were selected by the Stability Selection method. ‘AR’ and the last level of ‘Stage’ are the common covariates among all methods.

The time dependent AUC plot for all four methods, obtained by performing a five-fold cross validation, is depicted in Figure 5.

Fig 5:

Average AUC of different variable selection methods based on a five fold cross validation for KIRC dataset.

As illustrated in Figure 5, BVSNLP has slightly better predictive accuracy than GLMNET and Stability Selection. However, it achieves this accuracy with a much sparser model. We investigated the covariates that were selected by each of those algorithms in all five folds and found that BVSNLP, in addition to those fixed covariates, selects only 10 unique genes in total, where ‘AR’ is selected in three of the 5 folds.

GLMNET selected 160 different covariates across all 5 folds. Only five out of 24 selected covariates in Table 6 were selected in all five training datasets in cross validation. Those include ‘Age’, ‘Stage’ and ‘AR’. GLMNET was run 100 times for each fold.

ISIS-SCAD selected 45 different covariates and only ‘Stage 3’ was selected in all training datasets in cross validation. The Stability Selection method selected sparser models compared to ISIS-SCAD and GLMNET by selecting 13 different covariates. It picked and only ‘Age’ and ‘Stage 3’ in all five folds.

Figures 6 and 7 compares IBS and prediction error curves, respectively, between different methods for the KIRC dataset. These two measures are computed based on a five-fold cross validation. Computation of IBS and prediction error were done using the R package pec (Mogensen, Ishwaran and Gerds, 2012). A benchmark model based on the Kaplan-Meier estimate, which includes no covariates, was also added to the figures as a reference for the comparison. The c-index measures are also reported in Table 7. The c-index was computed as it was in Section 4 using the dynpred package in R.

Fig 6:

IBS comparison between all methods for the KIRC dataset.

Fig 7:

Comparison of prediction errors between all methods for the KIRC dataset.

Table 7.

Average c-index measure of different methods for the KIRC dataset.

	BVSNLP	GLMNET	ISIS-SCAD	Stability Selection
c-index measure	0.804	0.816	0.846	0.797

GLMNET has almost the same IBS curve as the reference Kaplan Meier curve. BVSNLP outperforms ISIS-SCAD and Stability selection has the best IBS performance among all. For prediction error curves, BVSNLP is second to ISIS-SCAD, and GLMNET and Stability Selection have almost the same performance. A different behavior can be seen for c-index measures where GLMNET and ISIS-SCAD have higher c-indices than BVSNLP.

The average run-time for different methods in each fold of the cross validation is summarized in Table 8. BVSNLP is run on 120 CPUs, Stability Selection is run on 4 CPUs, while GLMNET and ISIS-SCAD are run on a single CPU.

Table 8.

Average run time for different methods in each fold of the cross validation for KIRC data set.

	BVSNLP	GLMNET	ISIS-SCAD	Stability Selection
Run time (minutes)	6.4	180	5.0	1.3

In our previous study of binary outcomes using the same dataset (Nikooienejad, Wang and Johnson (2016)), we performed hierarchical clustering on the de-convolved tumor-specific expression matrix and identified two clusters of patient samples. We saw these two groups of patients present significantly different survival outcomes and therefore assigned good vs. bad survival to the groups. The dichotomization was done solely based on the clustering results of de-convolved gene expression levels. Survival times and censoring did not play any role in that process. However, there was a loss of information in dichotomizing a survival dataset and analyzing it with logistic regression. Now, with BVSNLP, we are able to use the original time to event with censoring information. To further compare the biological implications between the two analyses, we looked for known expression regulation networks between the gene sets found in the binary analysis, SAV1 and NUMBL, and the new genes found in this analysis, AR and SUDS3, using Pathway Studio® (Nikitin et al., 2003; Elsevier, 2018). We found that the well-studied cancer genes TGFB1, BCL2, PPARG, NEDD4, and CTNNB1, and a regulatory microRNA, MIR21, constitute the shortest paths between SAV1 and AR. Similarly, we found cancer genes CDKN1A, WNT3A, two genes that determine cell fate (SOX17 (connected with CTNNB1) and NANOG), and PAX6 that regulates transcription, to constitute the shortest paths between NUMBL and SUDS3. These are depicted in Figure 8. These findings suggest a high biological consistency between our two analyses, using BVSNLP to select features for binary and survival outcomes.

Fig 8:

Expression regulation networks connecting the old and new gene sets. a) This diagram shows all genes that are in the shortest pathways through the expression regulation between SAV1 and AR. b) This diagram shows all genes in the shortest pathways through expression regulation between NUMBL and SUDS3.

In summary, the binary model using SAV1 and NUMBL to predict overall survival of patients with kidney cancer is not as effective as the model using AR and SUDS3, as shown in Figure 9. Thus, although the findings of Nikooienejad, Wang and Johnson (2016) were all biologically justified, some limitations were associated with those findings due to the information loss incurred by clustering and dichotomizing the data, and the BVS-NLP model provides better insight on the genes associated with this cancer type.

Fig 9:

Comparison between BVSNLP model selection using survival and dichotomized versions of the KIRC dataset.

5.2. Kidney Renal Papillary Cell Carcinoma (KIRP).

The KIRP dataset (Cancer Genome Atlas Research Network, 2016) contains 244 samples with 13,335 covariates (after necessary data cleaning) and has a fairly high censoring rate of 85.7%. The covariates selected by each method are summarized in Table 9.

Table 9.

Selected covariates for KIRP across different variable selection algorithms

BVSNLP	Age	Gender	Stage
	CDK1
ISIS-SCAD	CDK1	COL6A1	C19orf33
GLMNET	No covariates were slected
Stability Selection	Stage 3	MTC02P12	RPL39P3

In addition to the fixed covariates ‘Age’, ‘Gender’, and ‘Stage’, BVSNLP selects the ‘CDK1’ gene in the HPPM as the most significant covariate in the design matrix. The posterior inclusion probability for ‘CDK1’ was 0.12. The MAP estimates for the coefficients of ‘Age, ‘Gender Male’, ‘Stage 2’, ‘Stage 3’ and ‘CDK1’ were 0.12, −0.10, 0.11, 0.79 and 1.13, respectively. This shows that a unit increase in ‘CDK1’ (Cyclin dependent kinase 1) gene expression increases the hazard rate by a factor of 3, for given values of the other covariates. CDK1 is a cell cycle regulator and has been reported previously as a prognostic marker gene for various cancer types. Many experimental studies have been performed to further understand the molecular mechanism behind the complex functions of CDK1 (Malumbres and Barbacid, 2009). This is the first time, however, that CDK1 has been reported as a prognostic marker gene in human data for papillary renal cell carcinoma. As expected, patients at the most advanced stage cancer have a hazard rate that is 2.2 times higher than patients at a localized stage of cancer, given the values of all other covariates. As in the case of KIRC patients, age and male gender have unfavorable and favorable impacts on the hazard rate, respectively.

Surprisingly, GLMNET does not select any covariates and ISIS-SCAD selects covariates that do not intersect BVSNLP. Stability Selection picked 3 covariates, with only ‘Stage 3’ in common with BVSNLP. As in the previous dataset, we tested BVSNLP for different choices of τ in the interval [0.01, 0.9] and the same model was selected for all values within this range. The total run-time of BVSNLP for this dataset was around 5 minutes using 120 CPUs.

Figure 12 shows the predictive accuracy for the proposed method based on a five-fold cross-validation. The outcomes for GLMNET, ISIS-SCAD and Stability Selection are not displayed in the plot because those methods did not converge or failed to produce results for at least one of the five folds in the cross-validation experiment.

Fig 12:

Average AUC of BVSNLP based on a five fold cross validation for the KIRP dataset.

The small AUC values in this plot for t < 1 warrant comment. Because there were few events soon after entry of tissue samples into the TCGA database, the AUC for early timepoints falls close to the 50% benchmark reflecting no predictive value.

Figures 10 and 11 respectively depict IBS and prediction error curves of the BVSNLP method, based on a five-fold cross validation for the KIRP dataset and compares it to the reference curve obtained by the Kaplan Meier method.

Fig 10:

IBS of BVSNLP for the KIRP dataset.

Fig 11:

Prediction error of BVSNLP for the KIRP dataset.

The c-index measure for the BVSNLP method is 0.876. The average run-time for BVSNLP in each fold of the cross validation was 3.6 minutes on 120 CPUs.

6. Discussion.

In this article a Bayesian variable selection method, BVSNLP, was proposed for selecting variables in high and ultrahigh dimensional datasets with survival time as outcomes. BVSNLP uses an inverse moment nonlocal prior density on non-zero regression coefficients. Analyses of simulated and real data suggest that BVSNLP performs comparably or better than other existing methods for variable selection for survival data. Moreover, the real data results indicated that the proposed algorithm is robust to the choice of the hyperparameter τ in the piMOM prior for values of τ in the range [0.01, 0.9].

Various outputs are provided by the algorithm. These include the HPPM, MPM and the posterior inclusion probability for each covariate in the model. For real datasets, Bayesian model averaging is used to incorporate uncertainty in selected models when computing time dependent AUC plots using Uno’s method (Uno et al., 2007). Finally, an R package named BVSNLP has been implemented to make the algorithm freely available and adaptable to interested researchers. The package can be run in parallel fashion where hundreds of CPUs can be exploited in order to increase the number of visited models in the search for highest posterior probability model. The BVSNLP package is available in the R repository, CRAN, at https://CRAN.R-project.org/package=BVSNLP. The user manual for the package is also available from this site.

Two real cancer genomic datasets from the TCGA website were considered in this article. Compared to other methods, BVSNLP found sparser models with biologically relevant genes. The proposed method showed a reliable predictive accuracy as measured by AUC using substantially fewer variables.

We have based our assessments on time dependent AUC and biological interpretation of the results, but other measures, like IBS, prediction error and the concordance index (also know as the c-index or Harrell’s c-index) are also reported. Difficulties associated with such measures are identified in Blanche, Kattan and Gerds (2018). In particular, the authors of the article demonstrate that the concordance index can favor misspecified models over the correctly specified model because it is based on the order of event times rather than the event status at the prediction horizon. This may explain the slightly higher c-index values for GLMNET in both simulation and real data sets. The time dependent AUC does not suffer from this deficiency. Of course, different evaluation criteria can be expected to result in different rankings of models, and criteria that emphasize prediction error over low false positive rates can be expected to favor larger models. Similarly, criteria that place a higher premium on eliminating false positives will tend to select smaller models.

APPENDIX A: CALCULATING THE GRADIENT AND HESSIAN OF g(β_K)

Let l(y; β_k) = log(π(y | β_k)) and lπ (β_k) = log(π(β_k)). For a n × p matrix A, let A_(i) denote the n × 1 vector corresponding to the i^th column of A and A_j denote the 1 × p vector corresponding to the i^th row of A. Also let ${\overline{A}}_i={\left(A_{i:n,.}\right)}^T$, where A_i:n,. is the sub-matrix of A from row i to the last row where all columns are included. This makes the dimension of ${\overline{A}}_i$ equal to p × (n − i + 1). Similarly, for a vector α of size n, let ${\overline{\alpha }}_i$ denote the sub-vector of α components i, i + 1,…, n, a vector of size (n − i + 1).

Let ${\psi }_{k_i}=\sum _{j=i}^n{e}^{X_{k_j}{\beta }_k}$ and ${\psi }_k={\left({\psi }_{k_1},\dots ,{\psi }_{k_n}\right)}^T$. Also let η denote the n × 1 column vector exp{X_kβ_k}. The logarithm of π(y | β_k) in (2.2) can then be expressed as

$$l\left(y;{\beta }_k\right)={\delta }^T\left(X_k{\beta }_k-\text{log}\left({\psi }_k\right)\right).$$ (A.1)

For each n × k design matrix X_k and β_k vector, define a new k × n matrix ${\mathbb{X}}_k$, with i^th column

$${\mathbb{X}}_{k(i)}=\frac{\left({\overline{X}}_{ki}\right)\left({\overline{\eta }}_i\right)}{{\psi }_{k_i}}.$$ (A.2)

Here, ${\overline{X}}_{ki}$ and ${\overline{\eta }}_i$ are obtained from matrix X_k and vector η, respectively, using the notation described in the beginning of this section.

The negative gradient of l(y; β_k) can then be written as

$$-\frac{\partial l\left(y;{\beta }_k\right)}{\partial {\beta }_k}=\left[{\mathbb{X}}_k-X_k^T\right]\delta .$$ (A.3)

To compute the Hessian matrix, let ${\mathbb{X}}_{k_{ij}}$ be the (i, j) element of ${\mathbb{X}}_k$. The k × k identity matrix is denoted by I_k and D(α) denotes a diagonal matrix with the elements of the vector α on its diagonal. Finally, let ζ^j = X_k(j) denote the j^th column of X_k.

Row j of the k × k Hessian matrix of −l(y; β_k) is defined as

$$-\frac{{\partial }^{2}l\left({\beta }_k\right)}{\partial {\beta }_{k_j}\partial {{\beta }_k}^T}={\delta }_{1\times n}^T{\Omega }_{n\times k}^j.$$ (A.4)

The ${\Omega }_{n\times k}^j$ matrix itself is constructed row by row, with row i equal to

$${\Omega }_i^j={\left[{\overline{X}}_{ki}\frac{D\left({\overline{\zeta }}_i^j\right)}{{\psi }_{k_i}}{\overline{\eta }}_i-{\mathbb{X}}_{kji}{\mathbb{X}}_{k(i)}\right]}^T.$$ (A.5)

Computing the Hessian can be implemented with a computational complexity of O(n).

The gradient and Hessian of the logarithm of the piMOM prior is more straightforward, and is given by

$$-\frac{\partial l_{\pi }\left({\beta }_k\right)}{\partial {\beta }_{ki}}=\frac{r+1}{{\beta }_{ki}}-\frac{2\tau }{{\beta }_k{}_i^3}.$$ (A.6)

The Hessian of −l_πˇ(β_k) is a diagonal matrix, D(α), where

$${\alpha }_i=\frac{6\tau }{{\beta }_k{}_i^4}-\frac{r+1}{{\beta }_k{}_i^2}.$$ (A.7)

APPENDIX B: ESTIMATING INDIVIDUAL SURVIVAL CURVES

In the Cox proportional hazard model the survival function for individual i under model ${\mathcal{M}}_k$ is defined as

$$S_i\left(t;{\mathcal{M}}_k\right)=\text{exp}{\left\{-H_0\left(t;{\mathcal{M}}_k\right)\right\}}^{\text{exp}\left\{X_{k_i}^T{\beta }_k\right\}}$$ (B.1)

where $H_0\left(t;{\mathcal{M}}_k\right)={\int }_0^t{h}_0\left(\tau ;{\mathcal{M}}_k\right)d\tau$ is the cumulative baseline hazard function, which can be estimated by

$${\widehat{H}}_0\left(t;{\mathcal{M}}_k\right)=\sum _{i:t_i\le t}\frac{{\delta }_i}{\sum _{j=i}^n\text{exp}\left\{X_{k_j}^T{\beta }_k\right\}}.$$ (B.2)

This is known as the Breslow estimator of $H_0\left(t;{\mathcal{M}}_k\right)$ (the observed times are sorted as in (2.2)). At this point three approaches can be exploited to estimate the survival curve for individual i. The first approach is to compute the HPPM survival curve by replacing ${\mathcal{M}}_k$ with ${\mathcal{M}}_{\mathit{\text{HPPM}}}$, and use the MAP estimate of β under the HPPM, ${\widehat{\beta }}_{\mathit{\text{HPPM}}}$, in (B.1) and (B.2). That is,

$${\widehat{S}}_i(t)=\text{exp}{\left\{-{\widehat{H}}_0\left(t;{\mathcal{M}}_{\mathit{\text{HPPM}}}\right)\right\}}^{\text{exp}\left\{X_{HPP{M}_i}^T{\widehat{\beta }}_{\mathit{\text{HPPM}}}\right\}}.$$ (B.3)

The second approach is computationally more intensive but takes into account the uncertainty of the posterior samples of the model space. In this approach, samples from the posterior distribution of the survival function are generated by replacing k in (B.1) with every posterior sample of the model space. The estimated survival curve is then obtained by taking the average of the posterior samples. That is,

$${\widehat{S}}_i(t)=\frac{1}{\mathcal{K}}\sum _{j=1}^{\mathcal{K}}\text{exp}{\left\{-{\widehat{H}}_0\left(t;{\mathcal{M}}_{k_j}\right)\right\}}^{\text{exp}\left\{X_{k_{j_i}}^T{\widehat{\beta }}_{k_j}\right\}},$$ (B.4)

where $\mathcal{K}$ is the number of posterior samples.

The third approach is to use Bayesian model averaging. As discussed in the previous section, we use Occam’s window where only the models with posterior probability of at least $0.01\times p\left({\mathcal{M}}_{\mathit{\text{HPPM}}}|y_n\right)$ are used in model averaging. Suppose $\mathcal{N}$ models fall in Occam’s window. Then

$${\widehat{S}}_i(t)=\sum _{j=1}^{\mathcal{N}}\text{exp}{\left\{-{\widehat{H}}_0\left(t;{\mathcal{M}}_{k_j}\right)\right\}}^{\text{exp}\left\{X_{k_{j_i}}^T{\widehat{\beta }}_{k_j}\right\}}p\left({\mathcal{M}}_{k_j}|y_n\right).$$ (B.5)

APPENDIX C: DEFINITIONS OF MTP, MFP AND P

Let S_i be the set of all covariates selected as the final model by the method at iteration i. Also let k be the set of covariates in the true model.

Define

$$\begin{gathered} {(TP)}_i=\left|S_i\cap k\right|,\text{ and} \\ {(FP)}_i=\left|S_i\backslash k\right|, \end{gathered}$$ (C.1)

where |A| denotes cardinality of set A, A \ B denotes set minus operation.

Following definitions above, MTP and MFP are obtained as follows:

$$\text{MTP}=\frac{1}{m}\sum _{i=1}^m{(TP)}_i;\text{\hspace{1em}MFP}=\frac{1}{m}\sum _{i=1}^m{(FP)}_i,$$ (C.2)

where m is the total number of iterations.

APPENDIX D: DISCUSSION ON THE PARAMETERS OF THE S5 ALGORITHM

It should be noted that the final model is the one with the highest posterior probability out of all visited models, collectively obtained from 120 S5 procedures with different starting models. Thus, the main objective in our algorithm is to increase the number of visited models. This is the first attempt towards reducing the sensitivity of finding the highest posterior probability model (HPPM) to the parameters of the S5 algorithm.

There are two important parameters in the S5 algorithm. The temperature vector for the annealing schedule, and the number of iterations at each temperature. We use 10 equally spaced temperature values decreasing from 3 to 1, where at temperature t_l, the posterior probability is raised to the power of 1/t_l. Values of t_<1 increase the posterior probability to unreasonably large values, making it susceptible to being trapped in local extremes, this reducing the number of visited models. Therefore, 1.0 is the lowest chosen temperature for the annealing schedule. Values higher than 3, on the other hand, were found empirically to not improve the performance of the algorithm because it then visited too high a proportion of models with comparatively low posterior probability.

The other parameter, the number of iterations at each temperature, can be chosen by the user in the R package. Theoretically, the higher number of iterations, the more models that will be visited. For the analyses in this paper, the number of iterations was chosen to be 30. This choice was based on a sensitivity analysis performed on simulation data for different numbers of iteration values ranging from 20 to 50, where we investigated the identification of the HPPM and number of visited models. The details of this experiment follow.

We defined a simulation batch as 50 different datasets that were generated with the same settings as Case 3 of the simulations discussed in Section 4, but with true model size of 6 and coefficients equal to β₁ = −1.5140, β₂ = 1.2799, β₃ = −1.5307, β₄ = 1.5164, β₅ = −1.3020, β₆ = 1.5833, and β_i = 0 for i > 6. A run of the BVSNLP was run on each dataset to find the simulation truth. For each simulation batch, in addition to the average number of visited models, the proportion of times (out of 50) that the algorithm selected the true model, without any false positives or false negatives, was also recorded. The niter parameter of the S5 algorithm was varied for each simulation batch, ranging from 10 to 50 in increments of 5.

The outcome of the sensitivity analysis for this parameter of the S5 algorithm is summarized in Figure 13 for the average number of visited models, and Table 10 for the proportion of times, P, the true model was found with no false positives or negatives.

These results suggest the S5 algorithm’s performance in finding the true model was not significantly impacted by the parameter niter, and that the number of visited unique models just changed 1.73% from an average of 18,674.78 unique models in 10 iterations per temperature to 18,997.64 unique models in 50 iterations per temperature in the S5 algorithm. This experiment suggests that the BVSNLP algorithm is relatively insensitive to the parameters of the S5 stochastic search algorithm, at least within the range of values considered in this study.

Fig 13:

Number of visited unique models by BVSNLP for different iterations in S5 algorithm, for simulation Case 3.

Table 10.

Proportion of times the true model is found for different iterations in S5 algorithm, for simulation Case 3.

niter	10	15	20	25	30	35	40	45	50
P	100%	100%	100%	100%	100%	100%	100%	100%	100%