BAYESIAN EMPIRICAL LIKELIHOOD FOR ULTRA-HIGH DIMENSIONAL VARIABLE SELECTION WITH APPLICATION TO DRUG SENSITIVITY

Abstract

Predicting drug sensitivity and selecting informative biomarkers are fundamental challenges in precision oncology. These tasks are complicated by the ultra-high dimensional nature of omics data, where the number of covariates $p$ far exceeds the sample size $n$. While variable selection methods are widely used, traditional high-dimensional regression techniques often rely on the assumption of Gaussian noise, which is frequently violated in real-world omics datasets. To overcome these limitations, we propose a Bayesian empirical likelihood approach (BEL_HD) for variable selection and prediction in a semiparametric framework. BEL_HD avoids strict distributional assumptions by utilizing estimating equations, and imposes joint regularization on both regression coefficients and the Lagrange multipliers to promote sparsity. We also introduce an efficient active set MCMC algorithm that enables scalable inference in ultra-high dimensional spaces. Through simulations and real-data applications, including a case study on leukemia drug sensitivity, BEL_HD demonstrates superior predictive performance and selects biologically interpretable features. Our method combines the flexibility of empirical likelihood with the inferential benefits of Bayesian modeling, offering a robust and extensible tool for high-dimensional biomedical research.

1. Introduction.

1.1. Statistical Challenges in Precision Oncology.

Cancer treatment is undergoing a paradigm shift toward precision medicine, where therapeutic decisions are increasingly guided by the molecular characteristics of an individual’s tumor. Advances in biotechnology have enabled researchers to collect large-scale omics data, such as transcriptomics, epigenomics, and proteomics, at an unprecedented resolution (Chawla et al., 2022). These datasets reveal complex biological processes such as gene regulation, pathway activation, and tumor microenvironmental interactions, offering a rich foundation for constructing predictive models to guide clinical decision-making.

Despite their transformative potential, omics data pose significant statistical and computational challenges. Most notably, these datasets are ultra-high dimensional, where the number of features $p$ greatly exceeds the number of samples $n$. For example, transcriptomic experiments may measure over 20,000 genes per sample, yet drug sensitivity measurements are typically remain limited to fewer than 100 cell lines within a given context. Such imbalance undermines model identifiability, amplifies overfitting risk, and hampers interpretation. Moreover, omics data often deviate from Gaussianity, exhibiting skewness, heavy tails, or multimodality (Pérez-Rodríguez et al., 2018; Church, T. and Mar, 2019), further complicating model development.

1.2. Existing Methods and Limitations.

Variable selection has become a standard approach to address the above mentioned issues, enabling the identification of informative biomarkers while improving interpretability and prediction (Warner, 2016; Cheng et al., 2016). Frequentist solutions rely on regularization-based techniques such as LASSO (Tibshirani, 1996), SCAD (Fan and Li, 2001), adaptive LASSO (Zou, 2006), and MCP (Zhang, 2010). For ultra-high dimensional problems, methods like Sure Independence Screening (SIS; Fan and Lv, 2008) and PICASSO (Zhao et al., 2018) employ screening and active set strategies to reduce the effective dimensionality (Friedman et al., 2007; Mazumder, Friedman and Hastie, 2011). Bayesian models achieve sparsity through shrinkage priors (e.g., spike-and-slab, Laplace) while naturally quantifying uncertainty (George and McCulloch, 1993; Ishwaran and Rao, 2005; Wang and George, 2007; Scott and Berger, 2010; Johnson and Rossell, 2012; Shin, Bhattacharya and Johnson, 2018).

However, most of these approaches assume Gaussian noise, which can bias inference in practice. Although transformations (e.g., log, Box-Cox) may reduce skewness, they do not always restore normality or maintain interpretability. Even after transformation, many features in omics datasets—such as lowly expressed genes in single-cell experiments—retain non-Gaussian properties.

1.3. Empirical Likelihood and Related Work.

Empirical likelihood (EL) offers a semi-parametric alternative by constructing likelihoods from estimating equations, thereby circumventing strict distributional assumptions. It was first proposed for univariate mean estimation (Owen, 1988), and then extended to multivariate mean estimation and linear regression (Owen et al., 1990; Owen, 1991). Since its introduction, it has been successfully applied to many different areas, such as censored data (Murphy and van der Vaart, 1997; Pan and Zhou, 1999; Li and Wang, 2003), truncated data (Li, 1995), time series data (Kitamura, 1997; Chuang and Chan, 2002; Nordman, 2009), and biased sampling problems (Qin, 1993, 1999).

Recently, penalized empirical likelihood approaches such as those by Tang and Leng (2010) and Leng and Tang (2012) have extended EL to variable selection. More recent Bayesian extensions incorporate priors to link penalization with posterior inference (Bedoui and Lazar, 2020; Cheng and Zhao, 2020; Moon and Bedoui, 2023).

However, these methods generally apply only when $p<n$, as convexity conditions may fail in high dimensions (Tsao, 2004; Chen, Peng and Qin, 2009). For instance, we encountered numerical issues when applying Moon and Bedoui (2023)’s method to ultra-high dimensional data. The only few attempts to extend EL to the $p\ge n$ case include modified EL for means (Lahiri and Mukhopadhyay, 2012) and doubly penalized EL for estimating equations (Chang et al., 2018), which adds penalties on both regression coefficients and Lagrange multipliers, thus enabling variable selection in the $p>n$ case.

1.4. Our Contribution.

To address these challenges, we propose BEL_HD, a Bayesian empirical likelihood method for ultra-high dimensional data. Our method introduces shrinkage priors on both regression coefficients and Lagrange multipliers, implementing a doubly penalized Bayesian EL. To address computational scalability, we adopt an active set strategy within our MCMC sampler, updating only a subset of relevant parameters in each iteration. This approach enables BEL_HD to efficiently handle non-Gaussian data, maintain interpretability, and enforce sparsity. In addition, its Bayesian framework naturally offers adaptive learning of penalization strength without extensively tuning penalty parameters, direct uncertainty quantification through posterior distributions, and flexibility in incorporating prior knowledge, compared to its frequentist counterpart. Through extensive simulations and applications, including a leukemia drug sensitivity case study that motivated our methodological development, we demonstrate that BEL_HD consistently improves both predictive accuracy and variable selection performance compared to existing methods.

The remainder of the paper is organized as follows. Section 2 introduces the empirical likelihood framework and our proposed Bayesian model. Section 3 presents simulation studies. Sections 4 and 5 showcase applications to benchmark datasets and a case study on leukemia drug sensitivity. Section 6 concludes with discussion and future directions.

2. Method.

2.1. Preliminaries.

2.1.1. Empirical likelihood and penalized empirical likelihood.

Let $\mathbf{y}$ be the outcome vector of length $n$, centered to have mean $0;\mathbf{X}$ be the $n\times p$ covariate matrix, with each column representing a covariate and standardized to have mean 0 and standard deviation 1; $\mathit{e}$ is the $n\times 1$ vector of independent and identically distributed (iid) errors with mean 0 and unknown variance ${\sigma }^2.\mathbf{X}$ and $\mathbf{y}$ are assumed to follow a linear model with the distribution of $\mathit{\text{e}}$ unspecified:

$$\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathit{e};\text{E}\left(\mathit{e}\right)=\mathbf{0};\text{Cov}\left(\mathit{e}\right)={\sigma }^2\mathbf{I}.$$ (1)

Empirical likelihood (Owen, 1988; Owen et al., 1990) estimates the vector of regression coefficients $\mathit{\beta }$ by finding the maximizer of the following empirical likelihood function:

$$L(\mathit{\beta })=\text{sup}\left\{\prod _{i=1}^n{w}_i:w_i\ge 0,\sum _{i=1}^n{w}_i=1,\sum _{i=1}^n{w}_i{\mathbf{g}}_i(\mathit{\beta })=\mathbf{0}\right\},$$ (2)

where $w_i$ is the weight for the $i$th observation and ${\mathbf{g}}_i\left(\mathit{\beta }\right)=\mathbf{0}$ represents $r$ unbiased estimating equations for $\mathit{\beta }$ based on the $i$th observation. In the linear regression setting, ${\mathbf{g}}_i\left(\mathit{\beta }\right)={\mathbf{x}}_i\left(y_i-\right.\left.{\mathbf{x}}_i^T\mathit{\beta }\right)$. Here $y_i$ is the $i$th element of $\mathbf{y}$ and ${\mathbf{x}}_{\mathbf{i}}$ is a vector being the transpose of the $i$th row of $\mathbf{X}$. Using the Lagrange multiplier method, the constrained optimization problem is maximized when $w_i\left(\mathit{\beta }\right)=n^{-1}{\left[1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right]}^{-1}$, where $\mathit{\lambda }$ satisfies the equation:

$$\sum _{i=1}^n\frac{{\mathbf{g}}_i\left(\mathit{\beta }\right)}{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)}=\mathbf{0}.$$ (3)

Note that finding the $\mathit{\beta }$ that maximizes (2) is equivalent to solving

$$\stackrel{\mathit{ˆ}}{\mathit{\beta }}=\underset{\mathit{\beta }}{\text{argmin}}\underset{\mathit{\lambda }}{\text{max}}l(\mathit{\beta },\mathit{\lambda }),$$

where $l(\mathit{\beta },\mathit{\lambda })=\sum _{i=1}^n\text{log}\left[1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right]$ arises from the Lagrange dual formulation of the empirical likelihood maximization problem (Owen, 2001; Chang et al., 2018). At the solution, the implied weights $w_i\left(\mathit{\beta }\right)$ define a data-adaptive probability distribution over the observations, and substituting these weights into the empirical likelihood shows that, up to an additive constant, $-l(\mathit{\beta },\mathit{\lambda })$ coincides with the empirical log-likelihood evaluated at $\beta$. Consequently, minimizing $l(\mathit{\beta },\mathit{\lambda })$ with respect to $\mathit{\beta }$ is equivalent to maximizing the empirical likelihood.

Variable selection based on penalized empirical likelihood (Tang and Leng, 2010; Leng and Tang, 2012) is to find ${\stackrel{\mathit{ˆ}}{\mathit{\beta }}}_P$ that minimizes the combination of the negative log empirical likelihood and the extra penalty term

$$l_P(\mathit{\beta })=\sum _{i=1}^n\text{log}\left\{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right\}+n\sum _{j=1}^{p}P\left(\left|{\beta }_j\right|\right),$$

where $P$ is a penalty function of $\mathit{\beta }$ that encourages sparsity and $\mathit{\lambda }$ satisfies the equation (3).

2.1.2. Bayesian empirical likelihood with a LASSO-type prior.

As noted by Tibshirani (1996) and Park and Casella (2008), the LASSO estimates can be interpreted as the posterior mode when the regression parameters ${\beta }_k$’s have independent and identical Laplace priors, which can be viewed as a mixture of normal distributions (Andrews and Mallows, 1974). Analogous with the original Bayesian LASSO, Cheng and Zhao (2020) proposed a Bayesian approach for variable selection using empirical likelihood with the following hierarchical structure for the prior distributions:

$$\pi \left(\mathit{\beta }\mid {\tau }_1^2,\dots ,{\tau }_p^2\right)~𝒩\left(\mathbf{0},{\mathbf{D}}_{\tau }\right),\text{and}\pi \left({\tau }_1^2,\dots ,{\tau }_p^2\mid {\gamma }^2\right)~\prod _{k=1}^p\frac{{\gamma }^2}{2}\text{exp}\left(-{\gamma }^2{\tau }_k^2/2\right),$$

where ${\mathbf{D}}_{\tau }=\text{diag}\left({\tau }_1^2,\dots ,{\tau }_p^2\right),{\tau }_1^2,\dots ,{\tau }_p^2$ are the latent variance parameters for $\mathit{\beta }$ and $\gamma$ is a hyper-parameter. Consequentially, the posterior (empirical) likelihood is given by

$$\pi \left(\mathit{\beta },{\mathbf{D}}_{\tau }\mid \mathbf{X},\mathbf{y}\right)\propto \prod _{i=1}^n\frac{1}{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)}\prod _{k=1}^p\left[{\left(2\pi {\tau }_k^2\right)}^{-1/2}\text{exp}\left(-\frac{{\beta }_k^2}{2{\tau }_k^2}\right)\cdot \frac{{\gamma }^2}{2}\text{exp}\left(-{\gamma }^2{\tau }_k^2/2\right)\right],$$

where $\mathit{\lambda }$ solves Equation (3).

2.1.3. The doubly penalized empirical likelihood.

As mentioned in the introduction, conventional EL based methods can be problematic when the dimension $p$ diverges faster than the sample size $n$. Thus, it is challenging to employ EL in the ultra-high dimensional $(p\gg n)$ situation (Tang and Leng, 2010; Leng and Tang, 2012; Chang et al., 2018). Under the estimating equation setting, Chang et al. (2018) suggested to add a second penalty term on the Lagrange multipliers in addition to the existing penalty term on regression parameters, thus enforcing the sparsity of $\mathit{\lambda }$, i.e., the selection of estimating equations besides the selection of covariates. The sparse parameter estimates based on estimating equations can be obtained by

$$\stackrel{\mathit{ˆ}}{\mathit{\beta }}=\underset{\mathit{\beta }}{\text{argmin}}\underset{\lambda }{\text{max}}\left[\sum _{i=1}^n\text{log}\left\{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right\}+n\sum _{k=1}^p{P}_{1,\tau }\left(\left|{\beta }_k\right|\right)-n\sum _{j=1}^r{P}_{2,\nu }\left(\left|{\lambda }_j\right|\right)\right],$$ (4)

where $\mathit{\beta }={\left({\beta }_1,\dots ,{\beta }_p\right)}^T,\mathit{\lambda }={\left({\lambda }_1,\dots ,{\lambda }_r\right)}^T,g_{ij}\left(\mathit{\beta }\right)$ is the $j$th estimating equation evaluated using the $i$th observation, $r$ is the total number of estimating equations, and $P_{1,\tau }(\cdot )$ and $P_{2,\nu }(\cdot )$ are two penalty functions with tuning parameters $\tau$ and $\nu$. Chang et al. (2018) showed that under the new double-penalized framework, the estimation is sparse and consistent when the number of parameters is allowed to grow exponentially with the sample size. Note that, for linear regression, the number of estimating equations is the same as the number of regression parameters (i.e., $r=p$).

2.2. The proposed Bayesian EL method.

Motivated by the works described above, We propose a Bayesian empirical likelihood (BEL) based method for variable selection in the ultra-high dimensional setting, labeled BEL_HD.

Given the empirical likelihood $L\left(\mathit{\beta }\right)\propto \prod _{i=1}^n{\left[1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right]}^{-1}$, we first apply Laplace priors on $\mathit{\beta }$ to encourage sparsity (which can be expressed as a mixture of Gaussians):

$$\pi \left(\mathit{\beta }\mid {\tau }_1^2,\dots ,{\tau }_p^2\right)~𝒩\left(\mathbf{0},{\mathbf{D}}_{\tau }\right),\text{and}\pi \left({\tau }_1^2,\dots ,{\tau }_p^2\mid {\gamma }_1^2\right)~\prod _{k=1}^p\text{Exponential}\left({\tau }_k^2\left|\frac{{\gamma }_1^2}{2}\right.\right)$$

where ${\mathbf{D}}_{\tau }=\text{diag}\left({\tau }_1^2,\dots ,{\tau }_p^2\right)$ are the latent variance parameters for $\mathit{\beta }$ and ${\gamma }_1$ is a global hyper-parameter introduced by the exponential priors for ${\tau }_k^2$’s. Therefore, the full conditional posterior for $\mathit{\beta }$ can be written as follows:

$$\pi (\mathit{\beta }\mid \cdot )\propto \prod _{i=1}^n\frac{1}{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)}\prod _{j=1}^p{\left(2\pi {\tau }_j^2\right)}^{-1/2}\text{exp}\left(-\frac{{\beta }_j^2}{2{\tau }_j^2}\right),$$

where $\mathit{\lambda }$ is the maximizer of the following objective function

$$\stackrel{\mathit{ˆ}}{\mathit{\lambda }}=\underset{\mathit{\lambda }}{\text{argmax}}\left\{\prod _{i=1}^n\left\{1+{\mathit{\lambda }}^T{\mathbf{g}}_i\left(\mathit{\beta }\right)\right\}\prod _{j=1}^p{\left(2\pi {\nu }_j^2\right)}^{-1/2}\text{exp}\left(-\frac{{\lambda }_j^2}{2{\nu }_j^2}\right)\right\}.$$ (5)

Consistent with the max-min structure in Equation (4), the first term in the objective for $\stackrel{\mathit{ˆ}}{\mathit{\lambda }}$ involves the product rather than its reciprocal to align with the required dual maximization. Further, a penalty term is imposed on $\mathit{\lambda }$ to achieve more shrinkage on the final estimates, where ${\nu }_1^2,\dots ,{\nu }_p^2$ are the shrinkage parameters for ${\lambda }_1,\dots ,{\lambda }_p$. Like ${\tau }_k^2$, we specify independent exponential priors for ${\nu }_k^2$:

$$\pi \left({\nu }_1^2,\dots {\nu }_p^2\mid {\gamma }_2^2\right)~\prod _{k=1}^p\text{Exponential}\left({\nu }_k^2\left|\frac{{\gamma }_2^2}{2}\right.\right),$$

where ${\gamma }_2$ is a global hyper-parameter introduced by the exponential priors for ${\nu }_k^2$’s.

It is easy to see the full conditional posteriors of ${\tau }_k^{-2}$’s follow inverse-Gaussian distributions,

$${\tau }_k^{-2}\mid \cdot ~\text{Inverse Gaussian}\left(\sqrt{\frac{{\gamma }_1^2}{{\beta }_k^2}},{\gamma }_1^2\right).$$

We further apply a conjugate prior $\text{Gamma}(\zeta =0.1,\delta =0.1)$ on the hyper-parameters ${\gamma }_1^2$ and ${\gamma }_2^2$. Therefore, the full conditional posterior of ${\gamma }_1^2$ follows a Gamma distribution

$${\gamma }_1^2\mid \cdot ~\text{Gamma}\left(p+\zeta ,\sum _{k=1}^p\frac{{\tau }_k^2}{2}+\delta \right).$$

Similarly, the full conditional posteriors of ${\nu }_k^{-2}$ and ${\gamma }_2^2$ are given by

$${\nu }_k^{-2}\mid \cdot ~\text{Inverse Gaussian}\left(\sqrt{\frac{{\gamma }_2^2}{{\lambda }_k^2}},{\gamma }_2^2\right),$$

$${\gamma }_2^2\mid \cdot ~\text{Gamma}\left(p+\zeta ,\sum _{k=1}^p\frac{{\nu }_k^2}{2}+\delta \right).$$

Posterior sampling can be achieved by iteratively sampling from the full conditionals listed above via a Gibbs sampler.

In general, the sparsity patterns of $\mathit{\beta }$ and $\mathit{\lambda }$ can differ. In the linear regression setting, although each covariate is associated with a corresponding estimating equation, the coefficient ${\beta }_j$ for that covariate appears in other estimating equations through the residuals. As a result, variable selection is directly characterized by the sparsity in $\mathit{\beta }$, while the sparsity in $\mathit{\lambda }$ plays an indirect but essential role. Following Chang et al. (2018), the penalty on $\mathit{\lambda }$ ensures the existence of the EL estimator in high dimensions by regularizing the space where the convex hull condition may be violated. In our Bayesian setup, this is achieved through shrinkage priors on both parameters, which stabilizes the semi-parametric likelihood and allows the sparsity of $\mathit{\lambda }$ to reinforce the selection of relevant covariates.

2.3. Algorithm.

Bayesian computation for ultra-high dimensional problems can be slow if we were to update coefficients of all variables in each iteration. To accelerate the computational speed, we adopt the active set idea introduced for path-wise coordinate optimization (Friedman et al., 2007; Mazumder, Friedman and Hastie, 2011; Zhao et al., 2018). The path-wise coordinate optimization algorithm proposed by Zhao et al. (2018) consists of three loops: the outer loop to update regularization parameters, the middle loop to update the active set, and the inner loop to update the estimates of regression coefficients for variables in the active set. Here, an active set includes all candidates of variables with non-zero coefficient.

We simplify this algorithm and adjust it for our Bayesian framework. A diagram for the flow of our algorithm is shown in Figure 1. The proposed algorithm consists of two loops: the outer loop updates (global) hyper-parameters ${\gamma }_1$ and ${\gamma }_2$, and the active set $𝒜$; the inner loop updates covariate-level parameters including regression the coefficients ${\beta }_j$, the Lagrange multipliers ${\lambda }_j$, and ${\tau }_j^2$ and ${\nu }_j^2$ for $j\in 𝒜$. The details of two loops are provided in Sections S1 and S2 of the Supplementary Material (Xu et al., 2025). The active set is updated in the outer loop. At each iteration $k$, it is a union of two disjoint subsets of the set of all $p$ covariates. The first subset includes all variables with $\left|{\beta }_j^{\left(k\right)}\right|>s$, given regression coefficient estimates cannot be exactly zero under fully Bayesian inference with continuous Laplace priors. In practice, one could put a non-informative prior on $s$, such as one that takes small positive values in the interval (0, 0.1), or select by cross-validation. In our simulation study, we simply set it to be 0.01 for all the scenarios. We also tried 0.05 and 0.1 in the simulation study and found the results were not sensitive to the choice of $s$, as shown in Section S3 of the Supplementary Material (Xu et al., 2025). The second subset, denoted by $𝒯$, consists of variables with small $\left|{\beta }_j^{\left(k\right)}\right|$ but large $\left|{\nabla }_{j}l\left({\mathit{\beta }}^{\left(k\right)}\right)\right|$ (i.e., small estimates but large gradients). Let $𝒢$ be the set of variables with $\left|{\nabla }_{j}l\left({\mathit{\beta }}^{\left(k\right)}\right)\right|\ge t$ and $\left|{\beta }_j^{\left(k\right)}\right|\le s$. We rank the variables in $𝒢$ by their gradients and select the top $r\%$ of the variables in $𝒢$ to enter $𝒯$. We suggest to set the threshold $t$ to the maximum value of $\left|{\nabla }_{j}l\left({\mathit{\beta }}^{\left(k\right)}\right)\right|$ in the active set of the previous iteration $(k-1)$ multiplied by a constant. When $p$ is extremely large compared to $n,𝒯$ can be directly set to an empty set, to accelerate the computation speed and achieve a sparse solution.

Fig 1.

The flow of our proposed algorithm. It consists of two loops. The inner loop updates $\mathit{\beta }$ and other covariate-level latent variables in the active set. The outer loop updates global hyper-parameters and the active set based on the inner loop results.

In the inner loop, we update $1/{\mathit{\tau }}^2$ and $1/{\mathit{\nu }}^2$ by sampling from inverse-Gaussian distributions. Following Cheng and Zhao (2020), we update ${\mathit{\beta }}_{𝒜}$ using the Metropolis-Hastings (MH) algorithm with a multivariate normal distribution as the proposal distribution. The corresponding ${\mathit{\lambda }}_{𝒜}$ is updated via optimization in (5) rather than sampling; as dual variables, $\mathit{\lambda }$ are functionals of $\mathit{\beta }$ that define the likelihood support, and optimization reduces computational complexity while enhancing numerical stability in ultra-high dimensions.

3. Simulation.

We conducted simulations to illustrate the performance of the proposed method in parameter estimation and variable selection, and to compare it with six existing approaches, including LASSO (Tibshirani, 1996), SCAD (Fan and Li, 2001), SIS (Fan and Lv, 2008), PICASSO (Zhao et al., 2018), BayesS5 (Shin, Bhattacharya and Johnson, 2018), and the doubly penalized EL method (EL_DP) (Chang et al., 2018). The first five methods were implemented through R packages “lars”(version 1.3; function “cv.lars”), “ncvreg”(version 3.14.1; function “cv.ncvreg”), “SIS”(version 0.8–8; function “SIS”), “PICASSO”(version 1.3.1; function “picasso”), and “BayesS5”(version 1.41; function “S5”), whereas EL_DP was implemented using a demo code on the authors’ website. As discussed in Section 1, there are many variable selection methods, which were not originally proposed for ultra-high dimensional problems. Among them, we chose LASSO and SCAD as comparison benchmarks due to their popularity and/or overall better performance. The rest four methods were proposed for ultra-high dimensional problems and showed better performance than other alternatives in their numerical studies. For our proposed BEL_HD, we ran 300 iterations for the outer loop and 100 iterations for the inner loop, where the first 150 and 20 iterations were used as burn-in samples, respectively. The numbers of iterations for the outer and inner loops were chosen because our preliminary experiments showed them to be sufficient for achieving general convergence. This was verified using Geweke’s diagnostic (Geweke et al., 1991), which confirmed that the means of the early and late segments of the chain (e.g., the first 30% and last 50% iterations after burn-in) reached a stable stationary distribution. We used the estimates obtained from PICASSO (with the default ${ℒ}_1$ regularizer) as our initial values. To start from a relatively liberal point, we tuned the smallest regularization parameter ratio (lambda.min.ratio) to be 0.01 in PICASSO, which would lead to more false positives but also include as many true positives as possible (i.e., high recall but low precision). We set $s=0.01$ and $t=3\text{max}\left(\left|{\nabla }_{j\in {𝒜}^{(k-1)}}l\left({\mathit{\beta }}^{\left(k\right)}\right)\right|\right)$¹ as our thresholds, and $r=10$ to select the top 10% of variables to enter the active set in each outer loop, whose gradients have absolute values larger than $t$. Note that the gradient threshold $t$ varies from iteration to iteration in the outer loop. These values are also recommended for general use when applying the method. For “SIS”, we set “greedy=TRUE”, which ran the greedy modification of the permutation-based ISIS and achieved overall better performance. For other methods, we used the default setting.

We designed seven simulation scenarios to explore the impacts of potentially important factors, including the number of the nonzero elements reflecting the sparsity of the selection problem, the mean and standard deviation (SD) of nonzero regression parameters reflecting the average (nonzero) effect size and their variability, respectively, the ratio $p/n$, covariate dependence, and types of error distributions. For each scenario, 100 independent replicates were simulated. The average mean squared error (MSE) is used to assess the performance of all methods in parameter estimation. Precision, recall, and F score (defined as $F_1=2\times (\text{precision}\times \text{recall})/(\text{precision}+\text{recall}))$ are used to assess the accuracies in variable selection.

3.1. Settings.

We simulated data from $\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathit{ϵ}$ with $\mathbf{X}$ generated from $𝒩(0,𝚺)$, and $\mathit{ϵ}$ generated from one of the following distributions independently and identically: normal, t, lognormal or a mixture of these distributions. The first $q$ elements of $\mathit{\beta }$ were independently sampled from $𝒩\left({\mu }_{\beta },{\sigma }_{\beta }^2\right),{\mu }_{\beta }\ne 0$, and the remaining components of $\mathit{\beta }$ were all set to zero. We centered our simulation around one basic setting, and for each scenario described below, we varied one design parameter only while fixing all the others. For the basic setting, $\mathit{ϵ}$ was generated from a mixture of two altered $t_3$ distributions (shifted and rescaled to have mean= 3 or −3 with SD=1), each with probability 0.5; the first $q=20$ elements of $\mathit{\beta }$ were randomly sampled from $𝒩\left(5,3^2\right)$; we set $n=100,p=800,{\Sigma }_{kk}=1$ and ${\Sigma }_{kl}=0.5$ for any $l\ne k$. The following seven scenarios were considered:

• Scenario I: different numbers of the non-zero elements. The first $q$ elements of $\mathit{\beta }$ were randomly sampled from $𝒩\left(5,3^2\right)$, where $q\in \left\{\mathrm{10,15,20,25,30}\right\}$.

• Scenario II: different mean sizes of the non-zero elements of $\mathit{\beta }$. The first 20 elements of $\mathit{\beta }$ were randomly sampled from $𝒩\left({\mu }_{\beta },3^2\right)$, where ${\mu }_{\beta }\in \{\mathrm{1,3},\mathrm{5,7},9\}$.

• Scenario III: different levels of variation for the non-zero elements of $\mathit{\beta }$. The first 20 elements of $\mathit{\beta }$ were randomly sampled from $𝒩\left(5,{\sigma }_{\beta }^2\right)$, where ${\sigma }_{\beta }\in \{\mathrm{1,2},\mathrm{3,4},5\}$.

• Scenario IV: different signal-to-noise ratios. We fixed the distribution of $\mathit{\beta }$ at $𝒩\left(5,3^2\right)$, but varied the scale of random errors by generating $\mathit{ϵ}$ from a mixture of $t_3$ distributions rescaled to have standard deviation $\sigma \in \{\mathrm{1,2},\mathrm{3,4},5\}$. Thus, the larger $\sigma$ is, the lower the signal-to-noise ratio is.

• Scenario V: different $p/n$ ratios. We fixed $n$ to be 100, with $p$ increased exponentially: 200, 400, 800, 1600, or 3200.

• Scenario VI: different levels of correlations of the covariates. We generated $\mathbf{\text{X}}$ from $𝒩(0,\Sigma )$, where ${\Sigma }_{kk}=1$ and ${\Sigma }_{kl}=\mathrm{0,0.1,0.3,0.5}$, or 0.7 for any $l\ne k$.

• Scenario VII: different types of error distributions. Besides the mixture of $t$ distributions, we consider $3t_3$ (a heavy tailed distribution); altered lognormal (a skewed distribution), i.e. lognormal(0, 1) shifted and rescaled to have mean 0 and SD 3); $N\left(0,3^2\right)$; a mixture of two (altered) log normal distributions: one with mean 3 and SD 1 and the other with mean −3 and SD 1, each with probability 0.5; and a mixture of normal distributions $𝒩\left(\mathrm{3,1}\right)$ and $𝒩(-\mathrm{3,1})$, each with probability 0.5.

3.2. Results for parameter estimation.

We report the average MSE (in the natural log scale) of each method in Figure 2 for Scenarios I – VI and in Table 1 for Scenario VII. The average MSE is defined as $\sum _{j=1}^p{\left({\hat{\beta }}_j-{\beta }_j\right)}^2/p$, averaged over all the replicates in each scenario, with ${\hat{\beta }}_j$ being the estimated ${\beta }_j$, the $j$ th element of $\mathit{\beta }$.

Fig 2.

The average MSE (in the natural log scale) of all methods for Simulation Scenarios I–VI.

Table 1.

The average MSE of all methods for Scenario VII.

Distribution	LASSO	SCAD	SIS	PICASSO	BayesS5	EL_DP	BEL_HD
t	0.24	0.75	0.38	0.09	1.37	0.09	0.07
N	0.24	0.64	0.41	0.09	1.42	0.09	0.08
LN	0.24	0.67	0.44	0.09	1.41	0.09	0.08
Mix-t	0.25	0.66	0.42	0.1	1.38	0.09	0.08
Mix-N	0.24	0.74	0.39	0.09	1.37	0.09	0.08
Mix-LN	0.26	0.71	0.37	0.1	1.42	0.1	0.09

First of all, we find from Figure 2 that the proposed BEL_HD produced the lowest (average) MSE for most cases. In the few other cases when the variance of $\mathit{ϵ}$ is large or when the correlation among covariates is large, BEL_HD still produced competitive results that are very close to the best.

From Panel I, II and IV of Figure 2, we observe that increasing $q$ (the number of nonzero elements) or ${\mu }_{\beta }$ (the mean size of nonzero elements) or $\sigma$ (the error variance) increases the MSE in general for all methods. Note that selecting a zero element as an active one would not affect the MSE much because their estimates are usually quite close to zero; however, incorrectly shrinking a nonzero element (with larger ${\beta }_j$) to zero would yield a larger MSE. Thus, increasing $q$ or ${\mu }_{\beta }$ would increase the MSE. It is also evident why the MSE has an increasing trend over $\sigma$: the signal to noise ratio (SNR) gets smaller as $\sigma$ increases, rendering parameter estimation more difficult. We further included five additional settings with lower SNR values and reported the results and expected SNR values for all configurations in Section S4 of the Supplementary Material (Xu et al., 2025). The overall conclusions remain consistent across these settings. Panel III of Figure 2 further shows that increasing ${\sigma }_{\beta }$ (the variability of the nonzero elements) would decrease the MSE. In this experiment, as ${\sigma }_{\beta }$ gets large from 1 to 5, the nonzero elements have a larger chance to fall in a neighbor of zero (as the mean ${\mu }_{\beta }$ is fixed at 5), in which the MSE would not be as large as in the case when ${\sigma }_{\beta }$ is small so that most nonzero elements stay away from zero. Overall, changing $q$ or ${\mu }_{\beta }$ has a (much) larger impact on the MSE than changing the scale parameters $\sigma$ or ${\sigma }_{\beta }$.

Panel V of Figure 2 shows that the effect of the $p/n$ ratio on the MSE can be very different for different methods. As the $p/n$ ratio increases exponentially, BayesS5 and SCAD show a decreasing pattern; SIS and LASSO show a concave pattern; the proposed BEL_HD shows a somewhat convex pattern; PICASSO and EL_DP have similar performance and show a quite flat pattern, except for the highest value 5 of the log2 ratio, where EL_DP suddenly became the worst, caused by its computational instability. From Panel VI, we find that for all methods except SCAD, increasing covariate dependence typically increases the MSE when the correlation is not close to zero.

Table 1 shows that the effect of type of error distribution on the MSE is perhaps smallest, compared to the other six factors considered. Among all, BayesS5, SCAD, and SIS are more sensitive to the change of the distribution type, and they have largest MSE values as in other scenarios. The proposed BEL_HD shows better performance regardless of the distribution type. It is also interesting to observe that EL_DP and PICASSO show similar MSE results in many settings across the different scenarios.

3.3. Results for variable selection.

To evaluate the performance of variable selection, we present the recall, precision, and F score of each method for Scenarios I – VII in Figures 3 (I–IV) and 4 (V–VII). Overall, the proposed BEL_HD produced competitive recall/precision, leading to the best F score in nearly all the scenarios. The results indicate that BEL_HD achieved a better balance between recall and precision than its competitors. Like BEL_HD, BayesS5 has the highest (or second highest) precision. However, BayesS5 has the lowest recall, because it tends to select fewer variables, leading to low true positives and low false positives. By contrast, PICASSO has the highest recall but the lowest precision, because it tends to select more variables, leading to high true positives and high false positives. As mentioned at the beginning of Section 3, to use estimates from PICASSO as our liberal starting point, we set the tuning parameter lambda.min.ratio for PICASSO to 0.01. In fact, doing so has led to very small MSEs for PICASSO, as reported in Section 3.2, because high false positives would not hurt the MSE as much as high false negatives. If we set lambda.min.ratio to the default value 0.05, PICASSO would have much worse performance in parameter estimation with slight performance improvement in variable selection. Even with lambda.min.ratio=0.5, PICASSO still has quite many false positives (this is also why we selected PICASSO to generate initial points) and so it can not beat the proposed BEL_HD in variable selection.

Fig 3.

Average recall, precision, and F scores for Simulation Scenarios I–IV.

Fig 4.

Recall, precision, and F scores of all methods in Simulation Scenarios V-VII.

It is interesting to observe that the trends for variable selection performance do not always match with those of the estimation performance. In particular, when the scale of non-zero elements of $\mathit{\beta }$ increases, the estimation accuracy increases while the variable selection performance decreases.

3.4. Computing time.

The computing time in seconds on the natural log scale of all methods for different $p/n$ ratios in Scenario 5 is shown in Figure 5. Our proposed method and the other Bayesian approach BayesS5 are slower than the other non-Bayesian methods except for EL_DP. While enjoying the merits of both EL and Bayesian approaches, BEL_HD inherits their disadvantage in computing speed, too. Still, with the aid of the active set concept, BEL_HD is faster than EL_DP, which is also based on EL.

Fig 5.

The computational time on the natural log scale of all methods under different $p/n$ ratios.

4. Benchmarking using real data.

We further illustrate the performance of our proposed BEL_HD and compare it with five existing methods including LASSO, SCAD, SIS, PICASSO, and BayesS5, using two benchmark datasets. The first is the U.S. crime dataset (Ehrlich, 1973), a widely used benchmark in machine learning, which contains 15 explanatory variables and the response variable (crime rate) for 47 U.S. states. To assess variable selection in ultra-high dimensional settings, we augment this dataset with additional noise predictors. The second is a gene expression dataset (Lan et al., 2006), consisting of expression levels for 45,037 genes measured across 60 mice, which was previously used to benchmark BayesS5 in Shin, Bhattacharya and Johnson (2018). This dataset represents a challenging high-dimensional biological scenario, typical of modern transcriptomic studies. EL_DP was not included in the comparison, because it was overly time consuming in both studies. For instance, it took more than 24 hours to finish one replicate in our second study. LASSO was excluded in the second as well, because the extremely high dimensionality caused some computational issue when using the R package “lars”. For PICASSO, we used the default setting with lambda.min.ratio = 0.05 for better performance here.

Given the ground truth is unknown for real data, mean squared prediction error (MSPE) is used to assess the performance of all methods in terms of predictive accuracy. Each data set was randomly split into test and training sets. Due to the limited sample size, we only chose 5 samples as the test set $S_{\mathit{\text{test}}}$ and the rest as the training set. We define $\text{MSPE}=\sum _{i\in S_{\mathit{\text{test}}}}{\left(y_i-\widehat{y_i}\right)}^2\left|/\left|S_{\mathit{\text{test}}}\right|\right.$ with $\widehat{y_i}={\mathbf{x}}_i^T\stackrel{\mathit{ˆ}}{\mathit{\beta }}$ and $\stackrel{\mathit{ˆ}}{\mathit{\beta }}$ computed using the corresponding training set, averaged over 100 replicates. As in our simulation, for BEL_HD, we still used estimates of PICASSO (lambda.min.ratio = 0.01) as the initial values. For the first problem, we ran 300 iterations for the outer loop and 100 iterations for the inner loop, where the first 150 and 20 iterations were used as burn-in samples. For the second problem involving an extremely large $p$, we use the same iteration and burn-in numbers for the inner loop but run 1000 iterations for the outer loop and used 500 as burn-in samples. These values serve as routine defaults; users may scale the number of outer iterations upward or downward proportional to the size of $p$. In order to speed up the convergence and also reduce the computational burden, we chose the threshold $s$ in BEL_HD by 10 fold cross-validation using the training set from the first replicate. For the US crime data, we used the same threshold $t$ as in our simulation, i.e. $t=3\text{max}\left(\left|{\nabla }_{j\in {𝒜}^{(k-1)}}l\left({\mathit{\beta }}^{\left(k\right)}\right)\right|\right)$; for the gene expression data, we directly set $𝒯=\varnothing$ as $p$ is extremely large (over 40, 000) and $n<100$ and the sparsity may be extreme. We used Geweke’s diagnostic to detect the convergence of an MCMC chain.

4.1. US crime data example.

US crime data was available in the Uniform Crime Reports of the FBI and first studied by Ehrlich (1973) to investigate the effect of punishment regimes on crime rates. The errors in the data were corrected by Vandaele et al. (1976). The data consists of crime rates for 47 states in US, and 15 variables. We treated the crime rate on the log scale as the response variable. To construct high-dimensional settings, we added noise (irrelevant) variables using the following two scenarios:

1. Noise variables were generated from $N(\mathbf{0},𝚺)$, where ${\Sigma }_{kk}=1$ and ${\Sigma }_{kl}=0.1$ for any $l\ne k$. We consider three different $p/n$ ratios: 2, 4, and 8. Therefore, 79, 173, and 361 noise variables were added.

2. The $p/n$ ratio was fixed to be 4. For each original variable, we generated one correlated confounding variable from the standard normal distribution. The rest of the noise variables were still generated as in the first setting. Three different correlation values were considered for the confounding variables: 0.3, 0.5, and 0.7.

The data set was randomly split into 5 test samples and 42 training samples, and the average MSPE of 30 replicates is shown in Tables 2 and 3 for Scenarios 1 and 2, respectively, in which the smallest average MSPE in each setting is highlighted in bold. We also list how many original (the first number in the parenthesis) and noise variables (the second number in the parenthesis) selected by each method. Although the truth is unknown, a good method should select fewer noise variables at least. We can evaluate the false positive rate of each method by checking how many noise variables were selected. For most methods, increasing the $p/n$ ratio renders estimation and variable selection more challenging. They tend to select fewer original variables and more noise variables as the ratio increases, yielding high false positive rates and larger MSPEs. Similarly, adding the correlated confounding variables tends to worsen the performance as well.

Table 2.

A benchmarking study using US crime data: average MSPE of each method under settings with different $p/n$ ratios. The average numbers of original and noise variables selected by each method are listed in the parenthesis (original/noise). The smallest average MSPE is highlighted in bold for each setting.

$\mathit{p}/\mathit{n}$	2	4	8
LASSO	0.12 (1.04/1.13)	0.13 (0.91/0.84)	0.13 (0.86/1.07)
SCAD	0.11 (2.55/5.29)	0.11 (2.15/6.45)	0.12 (1.78/8.68)
SIS	0.16 (0.02/0)	0.16 (0.01/0)	0.15 (0.02/0)
PICASSO	0.15 (5.53/25.9)	0.15 (3.83/31.00)	0.14 (2.15/34.17)
BayesS5	0.11 (1.66/0.35)	0.11 (1.62/0.38)	0.13 (1.41/0.59)
BEL_HD	0.10 (0.96/0.04)	0.10 (0.95/0.05)	0.11 (0.92/0.09)

Table 3.

A benchmarking study using US crime data: average MSPE of each method under settings where confounding variables with different correlations with the original variables were generated. The average numbers of original and noise variables selected by each method are listed in the parenthesis (original/noise).The smallest average MSPE is highlighted in bold for each setting.

Correlation	0.3	0.5	0.7
LASSO	0.13 (0.87/0.73)	0.14 (0.94/1.08)	0.15 (0.85/1.29)
SCAD	0.12 (2.11/7.99)	0.12 (2.06/6.31)	0.13 (1.89/7.82)
SIS	0.17 (0/0)	0.18 (0.03/0)	0.19 (0.02/0)
PICASSO	0.15 (3.44/31.25)	0.16 (3.54/31.25)	0.16 (3.16/31.16)
BayesS5	0.12 (1.50/0.52)	0.12 (1.52/0.50)	0.12 (1.38/0.63)
BEL_HD	0.10 (0.96/0.07)	0.11 (0.94/0.07)	0.11 (0.91/0.08)

We can see that our method gives the smallest MSPE and significantly fewer false positives. Although SIS always gives no false positive, it tends not to select any variable, yielding higher MSPE. PICASSO tends to select many more variables, which causes high false positives as expected. The other three, SCAD, LASSO and BayesS5 have MSPEs quite close to BEL_HD but all with higher false positives.

4.2. Gene expression data example.

The gene expression data used here was collected by Lan et al. (2006), which studied an F2 segregating for obesity and diabetes to expose key components of gene regulatory networks. An F2 represents an allelic block permutation. They found that some lipid metabolism genes showed strong correlations with stearoyl-CoA desaturase 1 (SCD1) expression.

From each liver of selected 60 mice, 45,037 expression measurements were obtained using the Affymetrix MOE430 microarrays (MOE430A and MOE430B). The data was normalized by Robust Multi-array Average (RMA) method and is available at the NCBI website with accession number GSE3330. Shin, Bhattacharya and Johnson (2018) illustrated the performance of BayesS5 using the first set of data (MOE430A, $p\approx \mathrm{23,000}$), while here we used the whole data set (MOE430A and MOE430B, $p\approx \mathrm{45,000}$). The data was randomly split into 5 test samples and 55 training samples, and the expression of gene SCD1 was used as the response variable. The average MSPE based on 100 replicates are reported in Table 4. We can find that BEL_HD produced the smallest MSPE among all the competing methods.

Table 4.

Analysis of gene expression data: average MSPE of each method.

Methods	SCAD	SIS	PICASSO	BayesS5	BEL_HD
MSPE	0.182	0.137	0.106	0.242	0.097

5. Application to drug sensitivity.

This study, focused on understanding drug sensitivity in leukemia, directly motivates the methodological development of BEL_HD. Cancer cells often exhibit elevated baseline replication stress, primarily due to oncogenic activation and rapid proliferation (da Costa et al., 2023). This phenomenon is particularly pronounced in leukemias, especially those characterized by TP53 deficiency, RAS mutations, or other highly proliferative molecular subtypes, which rely heavily on S/G2 cell-cycle checkpoints and DNA damage response pathways for cell survival (Sarmento et al., 2015; Smith et al., 2020; Chen et al., 2025). Therapeutic strategies that exacerbate replication stress while concurrently impairing these critical checkpoint defenses can induce catastrophic DNA damage, leading to leukemia cell death (Zhang et al., 2016; De Mel et al., 2024). This approach capitalizes on a key vulnerability of leukemia cells—their limited ability to manage uncontrolled replication stress when essential repair and checkpoint mechanisms are disrupted.

To investigate this vulnerability, we used a panel of six drugs known to target critical pathways of the replication stress response, including Eloxatin, Adavosertib, Alpelisib, Taselisib, CS-1421, and Prexasertib (Faivre et al., 2003; Parmar et al., 2019; He et al., 2021; Zhang et al., 2024). For example, Adavosertib (AZD1775), a WEE1 kinase inhibitor, abrogates the G₂/M checkpoint and forces cells with unresolved DNA lesions into mitosis, precipitating mitotic catastrophe in cells that lack a functional p53-mediated arrest (Rajeshkumar et al., 2011). CS-1421, an ERK pathway inhibitor to attenuate RAS/MAPK-driven proliferation, can reduce the tolerance for replication stress (Meyer et al., 2009). By correlating the sensitivity of these drugs in leukemia with corresponding gene expression profiles, we aim to identify patterns of vulnerability and informative biomarkers. Such findings can inform new therapeutic strategies, enabling the matching of specific pathway inhibitors to the molecular features of a patient’s leukemia.

Drug sensitivity is commonly measured by IC₅₀, the concentration at which a drug achieves half-maximal inhibition of the related biological process. In this study, gene expression profiles of 19,098 genes and IC₅₀ values for the six drugs across 34 leukemia cell lines were obtained from DepMap (Tsherniak et al., 2017). Raw count data of gene expression were normalized, batch-corrected and log₂ transformed, resulting in continuous values; and IC₅₀ served as the response variable.

As demonstrated in our preliminary analysis using residuals from fitting SCAD, the normality assumption is consistently violated. Based on the Q-Q plots (the left panel of Figure 6, the residuals for all six drugs show moderate to severe deviation from the theoretical quantiles of a normal distribution, particularly in the tails of TASELISIB, Prexasertib, and CS.1421, where the points form curves rather than following the straight diagonal line. This non-normality is further supported by the residual density plots (the right panel of Figure 6, which reveal various deviations from a Gaussian density curve: for instance, the residuals for ALPELISIB exhibit a sharp, leptokurtic peak; CS.1421 and Prexasertib show obvious left skewness; and TASELISIB displays multiple bumps or modes.These heterogeneous distributional characteristics preclude a uniform transformation across drugs, complicating both modeling and interpretation, and thus highlight the need for a more flexible statistical framework.

Fig 6.

Application to drug sensitivity: Q-Q plots (Left) and density curves (Right) of the residuals from fitting the leukemia data of each drug with SCAD.

Our objective is to assess the performance of BEL_HD in predicting IC₅₀ values and selecting informative biomarkers as well as quantifying selection uncertainty. For each drug, the dataset was randomly partitioned into 27 training samples and 7 test samples. This split was repeated 50 times to generate replicates. For each replicate, we ran 500 iterations and used 300 as burn-in samples for the outer loop; and the other settings were the same as the US crime data example. We similarly excluded LASSO and EL_DP due to the persistent computational issues. The MSPE across 50 replicates was computed for each drug (Table 5). BEL_HD achieved the lowest MSPE for three of the six drugs and demonstrated competitive performance for the remaining drugs.

Table 5.

Application to drug sensitivity: average MSPE of each method for six drugs targeting leukemia.

Drug	SCAD	SIS	PICASSO	BayesS5	BEL_HD
Eloxatin	0.022	0.012	0.046	0.012	0.011
Adavosertib	0.010	0.010	0.010	0.010	0.009
Alpelisib	0.002	0.002	0.004	0.002	0.003
Taselisib	0.015	0.013	0.031	0.013	0.029
CS-1421	0.013	0.014	0.015	0.014	0.012
Prexasertib	0.008	0.007	0.011	0.007	0.011

To assess the relevance of selected biomarkers. we performed gene set enrichment analysis (GSEA). Each drug was matched to pathways representing its primary mechanism of action. In total, 186 MSigDB pathways related to the DNA-damage response, WEE1, PI3K, ERK and CHK1 signaling were considered. For each method and drug combination, we aggregated genes that were selected at least once across the 50 replicates as input to GSEA. SIS and BayesS5 were excluded from this analysis because their conservative selection behavior in noisy data often led to no gene selection, explaining the identical MSPE values reported for both in Table 5. We then counted, for each combination of drug and method, how many pathways were significantly enriched (Figure 7).

Fig 7.

Application to drug sensitivity: gene set enrichment analysis results for each drug–method combination. Top: Benjamini–Hochberg adjusted p-values plotted against pathway size for the 6 drugs examined. Bottom: Number of significantly enriched pathways $(q<0.05)$ for every drug-method combination.

Figure 7 shows the adjusted p-values of pathway enrichment vs. pathway size (i.e., number of genes within a pathway) in the top panel and the number of significantly enriched pathways $(q<0.05)$ per drug–method combination in the bottom panel. For the three drugs where BEL_HD achieved the best MSPE (Eloxatin, Adavosertib, and CS-1421), the genes selected by BEL_HD were enriched in a greater number of pathways relevant to each drug’s mechanism. For example, EGF overexpression to the RAS-ERK signaling pathway was enriched for CS-1421, a well-known ERK1/2 inhibitor. Since EGF is a canonical upstream activator of the RAS–RAF–MEK–ERK cascade (Wee and Wang, 2017; Unni et al., 2018), the enrichment of this gene set implies that cell lines with high baseline EGF-driven ERK signaling are more sensitive to ERK1/2 blockade, thereby providing a clear mechanistic link. Conversely, for drugs where BEL_HD exhibited lower predictive performance, all methods yielded a limited number of, or no, enriched pathways, suggesting much weaker mechanistic signals underlying the data.

To further assess selection uncertainty, we calculated the posterior inclusion probability (PIP) for each gene feature, a common measure used in Bayesian variable selection (James and James, 2010; Shin, Bhattacharya and Johnson, 2018). We then averaged these PIP values across 50 replicates for each of the six drugs. The top five genes for each drug are displayed in Table 6.

Table 6.

Application to drug sensitivity: average posterior inclusion probability (PIP) of the top five genes for each of six leukemia-targeting drugs.

Eloxatin	PIP	Adavosertib	PIP	Alpelisib	PIP
APOBEC3H	0.700	GNG11	0.762	CYP17A1	0.304
H4C11	0.479	FCHO2	0.476	RAPGEF4	0.230
MAPK10	0.424	LONRF1	0.423	PDIA3	0.228
FAM25G	0.408	HMGA2	0.402	DNAJC15	0.219
DNAH14	0.385	OR6A2	0.333	CFAP157	0.205
Taselisib	PIP	CS-1421	PIP	Prexasertib	PIP
LDLRAD2	0.551	RBM4	0.339	PAK5	0.423
MGAT3	0.251	GORASP2	0.336	TPK1	0.234
C17orf78	0.226	KRAS	0.319	GTDC1	0.214
ZNF727	0.203	TMEM121B	0.297	PLPPR1	0.202
MAP3K7CL	0.194	TBX3	0.245	UBE2E2	0.180

Among the six drugs, Eloxatin and Adavosertib stand out with single, highly dominant predictors—APOBEC3H (PIP = 0.700) and GNG11 (PIP = 0.762), which are strong candidates for true biomarkers. APOBEC3H encodes a DNA-editing enzyme (Conticello, 2008), aligning with Eloxatin’s platinum-induced DNA-damage mechanism, while GNG11, a G-protein subunit involved in cell-cycle signaling (Hossain et al., 2006), plausibly interacts with Adavosertib’s WEE1-checkpoint inhibition. By contrast, Alpelisib shows a much lower top PIP (0.304 for CYP17A1), reflecting a more diffuse or uncertain genetic signature. Importantly, PIPs decline sharply after the top gene for nearly every drug, demonstrating the model’s capacity to concentrate posterior probability on the most credible predictors while naturally down-weighting less certain variables—a desirable property that curbs false positives and highlights targets worthy of further validation.

Overall, this case study strongly suggests that BEL_HD not only improves predictive accuracy but also identifies biologically interpretable genes and pathways aligned with known drug mechanisms, offering insight into leukemia cell vulnerabilities.

6. Conclusions and Discussion.

Ultra-high dimensional data are ubiquitous nowadays, posing major challenges to traditional statistical methods primarily designed for relatively low-dimensional problems. Predicting IC₅₀ for precision oncology and selecting interpretable biomarkers represent just one critical application area within this broader field.

In recent years, several new parametric-based variable selection methods have been developed for ultra-high dimensional problems. On the contrary, there are few methods developed in a semi-parametric field. In this paper, we have developed a new Bayesian semi-parametric method, labeled BEL_HD, which requires the assumption of a linear structure for the mean response only but no distributional assumptions. Inspired by Chang et al. (2018), we have applied Laplace priors on both the regression coefficients and the Lagrange multipliers in a Bayesian setup to boost the sparsity of the final estimates. Furthermore, we have adapted the path-wise coordinate optimization algorithm (Friedman et al., 2007; Mazumder, Friedman and Hastie, 2011; Zhao et al., 2018), which was proposed and applied in traditional likelihood approaches, to cope with the features of a Bayesian framework. The adjusted algorithm has been then fused seamlessly into the doubly penalized empirical likelihood Bayesian framework, which can screen potentially important variables and reduce the computational demand for ultra-high dimensional problems.

For practitioners implementing BEL_HD, we offer a few key practical considerations. While our method is robust to various data characteristics, careful selection of initial values and tuning parameters can further optimize performance. We recommend utilizing estimates from established methods, such as PICASSO, for initialization. Detailed guidance on setting thresholds for active set updates, including how to determine appropriate values for $s$ (for identifying non-zero coefficients) and $t$ (for gradient-based active set expansion) to balance sparsity and completeness, and managing other computational considerations, are thoroughly discussed and empirically validated in our simulation studies (Section 3). Users are also encouraged to employ standard MCMC convergence diagnostics, like Geweke’s diagnostic, to ensure the reliability of posterior inference for their specific datasets. An R package implementing our method is freely available from GitHub: https://github.com/canx2021/BEL_HD.

Numerical evidences from seven simulation studies, two benchmark data examples and our motivating application in drug sensitivity show that our proposed method BEL_HD has better performance in parameter estimation and variable selection as compared to existing methods. It shows steadily good performance against varying scale and location of nonzero regression coefficients, different levels of sparsity, signal-to-noise ratios, $n/p$ ratios, and correlations of covariates. It is also robust to the violation of the normality assumption which is often assumed in ordinary regression models. In particular, our case study on leukemia drug sensitivity highlights not only the model’s competitive predictive performance but also its ability to recover biologically meaningful gene-drug associations and signaling pathways. The superior performance of BEL_HD can be attributed to three aspects. First, the proposed method inherits the merits of empirical likelihood. Second, the idea of double penalties leads to a sparser solution as compared to traditional “single-penalty” approaches. Last but not the least, the adjusted path-wise coordinate optimization algorithm serves well as the screening process in the variable selection. It is also worth mentioning that the improved performance of BEL_HD comes at the cost of increased computing time.

Nevertheless, several limitations remain. First, while our method supports general estimating equations, the current implementation is focused on linear regression models. Extending BEL_HD to generalized linear models and other structured outcomes (e.g., survival or longitudinal data) is an important direction. Second, the active set strategy improves computational efficiency, but further optimization (e.g., variational approximation) could facilitate even larger-scale applications. Finally, although BEL_HD provides posterior summaries, its theoretical properties such as posterior consistency and uncertainty quantification under model misspecification warrant further investigation. Future work will explore these extensions and also consider integrating multi-omics data, incorporating structured sparsity (e.g., group or network-based priors), and adapting BEL_HD for causal inference in ultra-high dimensional settings.

Supplementary Material

Supplementary Material: Algorithms and Analysis Results

In this Supplementary Material, we provided the detailed pseudo algorithm of the outer and inner loop of the proposed Bayesian computation algorithm. We also provided the results of the sensitivity analysis for the threshold $s$, as well as the results from five additional simulation settings with lower signal-to-noise ratios.