Low‐Rank Variational Correction Estimation for Multi‐Source Heterogeneous Quantile Linear Regression Models

ABSTRACT

High‐dimensional data arising in genomics, econometrics, and clinical medicine often exhibit substantial heterogeneity across multiple sources. While existing methods address multi‐source heterogeneity, they do not adequately accommodate the combined challenges of high dimensionality and between‐source heterogeneity. To address this gap, we propose a scalable Bayesian framework for multi‐source heterogeneous quantile regression with spike‐and‐slab priors for simultaneous parameter estimation and feature selection. To overcome computational challenges, we combine mean‐field variational inference with Laplace approximation and introduce a novel low‐rank variational correction strategy that substantially improves approximation accuracy and adaptability in high‐dimensional heterogeneous settings. This low‐rank correction effectively captures the underlying dependence structure, leading to more robust and efficient inference. For model assessment and diagnostic analysis, we further develop a Bayesian score test coupled with local influence analysis. Extensive simulation studies and an analysis of The Cancer Genome Atlas (TCGA) data from four cancer cohorts (ESCA, PAAD, PCPG, and READ) demonstrate the computational efficiency, scalability, and practical utility of the proposed method in high‐dimensional heterogeneous applications. The proposed low‐rank variational correction algorithms are implemented in the R package LRQVB, which is publicly available on CRAN.

1. Introduction

High‐dimensional data, in which the number of variables may reach thousands or more and often exceeds the sample size, are now routinely collected in many scientific fields, including genomics, economics, and clinical medicine. The analysis of such data poses substantial challenges to classical statistical methods, which often rely on assumptions such as homogeneity and normality that are rarely satisfied in practice [1].

In this paper, we consider statistical inference for high‐dimensional multi‐source heterogeneous data, a setting characterized by two pervasive features of modern biomedical studies. High dimensionality arises when the number of covariates is large relative to the sample size, while multi‐source heterogeneity refers to distributional or structural differences across data sources, such that the underlying model parameters or data‐generating mechanisms may vary between sources. As an illustrative example, we examine the distribution of patients' age at diagnosis across four cancer cohorts from The Cancer Genome Atlas (TCGA): ESCA, PAAD, PCPG, and READ. As shown in Figure 1, the four cohorts exhibit distinct distributional patterns. Specifically, ESCA, PAAD, and READ are concentrated primarily among older patients, whereas PCPG tends to involve relatively younger patients and displays a broader age distribution. These differences provide an intuitive illustration of heterogeneity across multiple biomedical data sources. In our framework, each cancer cohort is treated as a distinct source, while the associated high‐dimensional molecular measurements further highlight the complexity of modern multi‐source biomedical data. Such challenges have motivated increasing interest in developing statistical methods for high‐dimensional heterogeneous models.

FIGURE 1.

Histogram of age at onset for four types of cancer.

Quantile regression [2] has emerged as a flexible alternative to traditional ordinary least squares (OLS) regression, particularly useful when the normality assumption is violated. Unlike OLS, which models only the conditional mean of the response variable, quantile regression estimates conditional quantiles, allowing it to better capture heterogeneous effects across different parts of the distribution [3]. In addition, due to its robustness against heteroskedasticity, outliers, and anomalous recordings in the response variable [4], quantile regression has been widely applied in various fields, such as economics [5, 6, 7, 8], survival analysis [9, 10, 11, 12], microarray study [13, 14, 15], among others.

Bayesian quantile methods have recently received much attention for the problems of variable selection and parameter estimation that play an important role in the model building process in the analysis of sparse and high‐dimensional data, as they not only improve the accuracy and efficiency of parameter estimation [16, 17, 18, 19, 20, 21, 22], but also reliably identify important predictors while removing potential modeling bias. In addition, Bayesian methods effectively address the challenges of highly non‐convex optimization problems through incorporating diverse priors on model parameters; see, for example, the normal mixture prior [23], the spike‐and‐slab prior [24, 25], the horseshoe prior [26], the shrinking and diffusing prior [27], to name just a few.

Markov chain Monte Carlo (MCMC) methods have traditionally been central to Bayesian inference in quantile regression but often face efficiency challenges in high‐dimensional settings. Variational inference (VI) [28, 29] offers a scalable alternative by minimizing the Kullback‐Leibler (KL) divergence between a tractable variational distribution and the true posterior, thereby transforming inference into an optimization problem. VI has been applied in various settings, including proportional hazards models [30], logistic accelerated failure time models [31], and partially linear mean shift models [32]. Notably [33], introduced variational Bayesian methods for quantile regression using the $\text{horseshoe}+$ prior, while [34] proposed a variational Bayesian approach with a spike‐and‐slab lasso penalty for high‐dimensional quantile regression. More recently [35], developed the spike‐and‐slab quantile LASSO through a fully Bayesian spike‐and‐slab framework based on the Expectation‐Maximization (EM) algorithm. However, variational Bayes often struggles to fully capture the complex structures and heterogeneity inherent in high‐dimensional data, leading to reduced model expressiveness and less accurate inference outcomes. To address this limitation, we refer to [36], who proposed a low‐rank variational Bayes correction under the spike‐and‐slab prior. This correction leverages a Laplace approximation method [37, 38] based on second‐order expansions.

With rapid advances in data acquisition technologies, substantial progress has been made in the analysis of high‐dimensional heterogeneous data. The complexity of modern data structures, together with high dimensionality and diverse data types, poses major challenges for statistical modeling and inference [39]. To address these challenges, a range of methodological developments have been proposed. For example, Bayesian integrative clustering has been used for high‐dimensional multi‐modal molecular data, whereas representation learning methods combining variational inference and neural networks have been developed to extract low‐dimensional representations from complex datasets [40]. Variational autoencoders have also been successfully applied to cancer data integration by embedding heterogeneous sources into a unified latent space [41]. In high‐dimensional regression, adaptive penalization within a variational Bayes framework has been introduced for parameter inference from observed data [42]. More recently, heterogeneity‐aware clustered distributed learning methods have been developed for multi‐source data analysis [43].

However, existing research has largely addressed high dimensionality and heterogeneity as separate challenges [44, 45, 46], leaving an important gap in joint modeling and inference for high‐dimensional heterogeneous data, including but not limited to multi‐source settings. Motivated by this gap, we propose a novel variational Bayes framework for high‐dimensional heterogeneous data arising from multiple sources or subpopulations. The proposed method aims to identify shared predictors across heterogeneous groups while detecting noteworthy individuals that deviate from the common structure. More specifically, our goal is not only to model heterogeneity effectively, but also to characterize individual‐level deviations within complex high‐dimensional data.

To achieve this, we employ variational Bayesian local influence analysis [47, 48] to assess model sensitivity to individual observations and model specifications, thereby providing insights into model dependence. This approach has been widely applied in various statistical models [49, 50, 51, 52]; however, to the best of our knowledge, no existing work has jointly addressed high‐dimensionality and multi‐source heterogeneity within this framework. Specifically, we develop a low‐rank variational correction method in which spike‐and‐slab priors are imposed on sparse homogeneous coefficients. Variational inference based on a mean‐field approximation combined with a Laplace approximation is used to obtain parameter estimates. After variable selection, a variational Bayesian correction is further applied to the posterior mean in a reduced‐dimensional space. Based on the corrected estimates, we conduct Bayesian local influence analysis and develop a corresponding score test. Finally, we perform simulation studies to evaluate the finite‐sample performance of the proposed methods and demonstrate their applicability to multi‐cohort TCGA cancer data, including ESCA, PAAD, PCPG, and READ. The proposed low‐rank variational correction method offers two main advantages: (i) it adjusts standard variational estimates to better accommodate heterogeneous data structures; and (ii) it retains computational efficiency comparable to Laplace variational approximation, ensuring scalability with respect to both model complexity and sample size.

The remainder of this paper is organized as follows In Section 2, we detail the models, prior, and variational Bayes methods for multi‐source heterogeneous data. Section 3 proposes the variational Bayes low‐rank correction method. Section 4 presents the local influence measure and score test for perturbations. In Section 5, we conduct simulation studies to investigate the finite sample performance of the proposed methods. Section 6 applies these methods to analyze the four cancer datasets. A brief summary is given in Section 7. Finally, technical details and additional results regarding the simulation studies and real‐data application are given in the Supporting Information.

2. Models and Materials

2.1. Models

Let $({\mathbf{Y}}_k,{\mathbf{X}}_k,{\mathbf{Z}}_k)$ represent the sample observation matrices for the $k$th data source, where $k=1,\dots ,K$. The components are defined as

$$\begin{array}{ll}\hfill {\mathbf{Y}}_k& ={(Y_{1,k},\dots ,Y_{n_k,k})}^{\top },{\mathbf{X}}_k={({\mathbf{X}}_{1,k},\dots ,{\mathbf{X}}_{n_k,k})}^{\top },\\ \hfill & \text{and}{\mathbf{Z}}_k={({\mathbf{Z}}_{1,k},\dots ,{\mathbf{Z}}_{n_k,k})}^{\top },\end{array}$$

with ${\mathbf{X}}_{i,k}={(X_{i1,k},\dots ,X_{ip,k})}^{\top }$ and ${\mathbf{Z}}_{i,k}={(Z_{i1,k},\dots ,Z_{iq,k})}^{\top }$. Assume the data sources are mutually independent. The model for multi‐source heterogeneous data can be uniformly described as follows

$${\mathbf{Y}}_k=f\left({\mathbf{X}}_k\right)+g_k\left({\mathbf{Z}}_k\right)+{\mathit{\epsilon }}_k,k=1,\cdots ,K,$$ (1)

where $f({\mathbf{X}}_k)$ represents the homogeneous part, which captures the relationship between the response variable ${\mathbf{Y}}_k$ and the homogeneous variable ${\mathbf{X}}_k$; the second component $g_k({\mathbf{Z}}_k)$, represents the heterogeneous part, which captures the relationship between the response variable ${\mathbf{Y}}_k$ and the heterogeneous variable ${\mathbf{Z}}_k$. In this paper, we focus on the linear quantile regression model case for $f(·)$ and $g(·)$ in model (1), which can be expressed as

$$\begin{array}{ll}\hfill Q_{\tau }(Y_{i,k}|{\mathbf{X}}_{i,k},{\mathbf{Z}}_{i,k})& ={\mathbf{X}}_{i,k}^{\top }{\theta }_{\tau }+{\mathbf{Z}}_{i,k}^{\top }{\beta }_{k,\tau },\\ \hfill & i=1,\cdots ,n_k,k=1,\cdots ,K,\end{array}$$ (2)

where $\tau$ is the quantile level $(0<\tau <1)$, ${\mathit{\theta }}_{\tau }$ and ${\mathit{\beta }}_{\mathit{k}\mathit{,}\mathit{\tau }}$ are the unknown homogeneous and heterogeneous coefficient vectors of dimensions $p\times 1$ and $q\times 1,$ respectively. This formulation defines the quantile regression model without requiring any parametric distributional assumptions on the error term. Estimation is typically carried out by minimizing the check loss function [2]

$$\mathrm{\Omega }({\mathit{\theta }}_{\tau },{\mathit{\beta }}_{1,\tau },\cdots ,{\mathit{\beta }}_{K,\tau })=\sum _{k=1}^K\sum _{i=1}^{n_k}{\rho }_{\tau }(y_{i,k}-{\mathbf{X}}_{i,k}^{\top }{\mathit{\theta }}_{\tau }-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_{k,\tau }),$$ (3)

where ${\rho }_{\tau }(u)=u\{\tau -I(u<0)\}$ is the check function, $I(·)$ is an indicator function, and $n_k$ is the sample size of the $k$th data source. It has been shown by [53] that minimizing (3) is equivalent to maximizing the likelihood function associated with (2) under the asymmetric Laplace distribution (ALD) error for ${\epsilon }_{i,k}$, following model

$${\mathbf{Y}}_k={\mathbf{X}}_k^{\top }{\theta }_{\tau }+{\mathbf{Z}}_k^{\top }{\beta }_{k,\tau }+{\epsilon }_{i,k},k=1,\cdots ,K,$$

This framework differs from many existing integrative analysis approaches. Rather than clustering or subgrouping data sources into latent groups, or aggregating separate learners across sources, the formulation in (2) targets a common homogeneous effect ${\mathit{\theta }}_{\tau }$ shared across all sources, while simultaneously allowing source‐specific heterogeneous effects ${\mathit{\beta }}_{k,\tau }$ at the parameter level. The common design $({\mathbf{X}}_k,{\mathbf{Z}}_k)$ ensures that the coefficients are directly comparable across sources and enables joint estimation of ${\mathit{\theta }}_{\tau }$ with interpretable deviations ${\mathit{\beta }}_{k,\tau }$, in contrast to methods that focus primarily on discovering clusters of studies or constructing prediction ensembles [54, 55].

The probability density function is given by

$$p_{\tau }({\epsilon }_{i,k})=\tau (1-\tau )/\sigma \left[e^{(1-\tau ){\epsilon }_{i,k}/\sigma }{I}_{{ℝ}_{-}}({\epsilon }_{i,k})+e^{-\tau {\epsilon }_{i,k}/\sigma }{I}_{{ℝ}_{+}}({\epsilon }_{i,k})\right],$$

where $\tau$ determines the skewness of distribution, $\sigma$ is a scale parameter, ${ℝ}_{+}=(0,\infty )$, and ${ℝ}_{-}=(-\infty ,0]$. In particular [56], demonstrated that by utilizing a location‐scale mixture of normal and exponential distributions for the asymmetric Laplace distribution, the multi‐source heterogeneous linear quantile regression model can be obtained by modeling the error process as

$${\epsilon }_{i,k}=\sigma {\kappa }_1{\xi }_{i,k}+\sigma {\kappa }_2\sqrt{{\xi }_{i,k}}u,$$

where $u\sim N(0,1),{\xi }_{i,k}\sim \mathrm{Exp}(1)$, which are independent from each other, and ${\kappa }_1=\frac{1-2\tau }{\tau (1-\tau )}$, ${\kappa }_2^2=\frac{2}{\tau (1-\tau )}$ are deterministic quantile specific parameters. By taking ${\xi }_{i,k}\sim \mathrm{Exp}(1/\sigma )$, we see that $y_{i,k}$ is normally distributed, $y_{i,k}|{\xi }_{i,k}\sim N({\mathbf{X}}_{i,k}^{\top }{\mathit{\theta }}_{\tau }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_{k,\tau }+{\kappa }_1{\xi }_{i,k},{\kappa }_2^2\sigma {\xi }_{i,k})$. Then, the multi‐source heterogeneous linear quantile regression model (2) can be written as the following hierarchical model

$$\left\{\begin{array}{ll}& y_{i,k}|{\xi }_{i,k}\sim N({\mathbf{X}}_{i,k}^{\top }{\mathit{\theta }}_{\tau }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_{k,\tau }+{\kappa }_1{\xi }_{i,k},{\kappa }_2^2\sigma {\xi }_{i,k})\\ & {\xi }_{i,k}|\sigma \sim \mathrm{Exp}(1/\sigma )\end{array}\right..$$ (4)

Note that we will omit the subscript $\tau$ for notational simplicity from this point onward.

2.2. The Spike‐and‐Slab Prior and Variational Approximations

We propose a variational Bayesian framework that allows for simultaneous variable selection of both homogeneous coefficients $\mathit{\theta }$ and heterogeneous coefficients ${\mathit{\beta }}_k$. While the methodology accommodates this full selection scheme, the present paper focuses on the scenario where variable selection is applied exclusively to the homogeneous coefficients. This focused approach enables a clearer exposition of the core variational Bayesian algorithm, highlighting the key methodological innovations and computational mechanisms without introducing the additional notational and algorithmic complexity associated with joint selection.

We adopt a unified spike‐and‐slab prior for both the homogeneous coefficients $\mathit{\theta }$ and the heterogeneous coefficients ${\mathit{\beta }}_k$ as follows. Let $𝒦=\{0,1,\dots ,K\}$ index the coefficient blocks, where $k=0$ corresponds to the homogeneous block and $k\ge 1$ to the heterogeneous blocks. Set $q_0=p$ and define the unified coefficients

$$c_{kj}=\left\{\begin{array}{ll}{\theta }_j,& k=0,j=1,\dots ,p,\\ {\beta }_{kj},& k=1,\dots ,K,j=1,\dots ,q_k.\end{array}\right.$$

The hierarchical prior $\pi (\mathit{\theta },\mathit{\beta },z,w)$ is specified as

$$\begin{array}{ll}\hfill & \left\{\begin{array}{l}{\pi }_k\stackrel{\text{iid}}{\sim }\text{Beta}(a_k,b_k),\\ {\gamma }_{kj}|{\pi }_k={\pi }_k^{{\gamma }_{kj}}{(1-{\pi }_k)}^{1-{\gamma }_{kj}},\\ c_{kj}|{\gamma }_{kj}={\gamma }_{kj}N(0,{\tau }_1^2)+(1-{\gamma }_{kj})N(0,{\tau }_0^2),\end{array}\right.k\in 𝒦,j=1,\dots ,q_k.\end{array}$$

Here, ${\pi }_k$ is the latent inclusion probability and ${\gamma }_{kj}$ is the binary selection indicator. The Beta–Bernoulli hierarchy, governed by $\text{Beta}(a_k,b_k)$ and $\text{Bernoulli}({\pi }_k)$, induces adaptive sparsity across coefficients. Specifically, $({\pi }_0,{\gamma }_{0j},{\theta }_j)$ correspond to the homogeneous coefficients when $k=0$, whereas $({\pi }_k,{\gamma }_{kj},{\beta }_{kj})$ correspond to the heterogeneous coefficients ${\mathit{\beta }}_k$ for $k\ge 1$.

When variable selection targets only the homogeneous coefficients $\mathit{\theta }$, we treat the heterogeneous coefficients ${\mathit{\beta }}_k$ as dense and assign the Gaussian prior

$${\mathit{\beta }}_k\stackrel{\text{iid}}{\sim }N({\mathbf{0}}_q,{\mathbf{\sum }}_{0,k}),k=1,\cdots ,K,$$

where ${\mathbf{0}}_q$ and ${\mathbf{\sum }}_{0,k}$ represent the prior mean and covariance of the heterogeneity coefficient of the $k$th data source. While we present empirical results for both frameworks in the numerical experiments, the methodological development in the subsequent sections focuses on the local selection paradigm to maintain clarity and to better highlight the core variational Bayesian algorithm.

Finally, we specify an inverse‐gamma prior for $\sigma$ in (4), namely $\sigma \sim \text{IG}(r_0/2,s_0/2)$, where $r_0$ and $s_0$ are prespecified hyperparameters and $\text{IG}(a,b)$ denotes the inverse‐gamma distribution with shape parameter $a$ and scale parameter $b$.

Let $\mathbf{Y}={\{{\mathbf{Y}}_k\}}_{k=1}^K,\mathbf{X}={\{{\mathbf{X}}_k\}}_{k=1}^K,\mathbf{Z}={\{{\mathbf{Z}}_k\}}_{k=1}^K$ denote the responses and the corresponding design matrices from the $K$ sources. Our model specifies the conditional distribution of the responses given the predictors [57, 58]. Based on the prior specifications above, the posterior distribution of $\mathbf{\Theta }=(\mathit{\theta },\mathit{\gamma },{\pi }_0,$ ${\{{\mathit{\beta }}_k\}}_{k=1}^K,{\{{\xi }_{i,k}\}}_{i=1,k=1}^{n_k,K},\sigma )$ given the observed data $(\mathbf{Y},\mathbf{X},\mathbf{Z})$ is given by

$$\begin{array}{ll}p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})& \propto L(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})p(\mathit{\theta }|\mathit{\gamma })p(\mathit{\gamma }|{\pi }_0)p({\pi }_0)p(\mathit{\beta })p(\mathit{\xi }|\sigma )p(\sigma )\\ \hfill & \propto \prod _{k=1}^K\prod _{i=1}^{n_k}\left\{N(Y_{i,k}|{\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+{\kappa }_1{\xi }_{i,k},{\kappa }_2^2\sigma {\xi }_{i,k})\right\}\\ \hfill & \times \prod _{j=1}^p\left\{{[N({\theta }_j|0,{\tau }_1^2)]}^{{\gamma }_j}{[N({\theta }_j|0,{\tau }_0^2)]}^{(1-{\gamma }_j)}\right.\\ \hfill & \left.\text{Bernoulli}({\gamma }_j|{\pi }_0)\right\}\text{Beta}({\pi }_0|a_0,b_0)\\ \hfill & \times \prod _{k=1}^K\left\{N({\mathit{\beta }}_k|{\mathbf{0}}_q,{\mathbf{\sum }}_{0,k})\right\}\times \prod _{k=1}^K\prod _i^{n_k}\left\{\mathrm{Exp}({\xi }_{i,k}|1/\sigma )\right\}\text{IG}\left(\sigma |\frac{r_0}{2},\frac{s_0}{2}\right)\\ \hfill & \propto \prod _{k=1}^K\prod _{i=1}^{n_k}{({\kappa }_2^2\sigma {\xi }_{i,k})}^{-\frac{1}{2}}\exp \left(-\frac{{(Y_{i,k}-{\mathbf{X}}_{i,k}^{\top }\mathit{\theta }-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k-{\kappa }_1{\xi }_{i,k})}^2}{2{\kappa }_2^2\sigma {\xi }_{i,k}}\right)\\ \hfill & \times \prod _{j=1}^p{\left[\frac{1}{{\tau }_1}\exp \left(-\frac{{\theta }_j^2}{2{\tau }_1^2}\right)\right]}^{{\gamma }_j}{\left[\frac{1}{{\tau }_0}\exp \left(-\frac{{\theta }_j^2}{2{\tau }_0^2}\right)\right]}^{1-{\gamma }_j}\\ \hfill & \times {\pi }_0^{{\gamma }_j}{(1-{\pi }_0)}^{1-{\gamma }_j}{\pi }_0^{a_0-1}{(1-{\pi }_0)}^{b_0-1}\\ \hfill & \times \prod _{k=1}^K\exp \left(-\frac{1}{2}{\mathit{\beta }}_k^{\top }{\mathbf{\sum }}_{0,k}^{-1}{\mathit{\beta }}_k\right)\\ \hfill & \times \prod _{k=1}^K\prod _{i=1}^{n_k}\frac{1}{\sigma }\exp \left(-\frac{{\xi }_{i,k}}{\sigma }\right){\sigma }^{-\frac{r_0}{2}-1}\exp \left(-\frac{s_0}{2\sigma }\right).\end{array}$$

This posterior distribution is analytically intractable. Therefore, MCMC methods may in principle be used to draw posterior samples of the unknown parameters. However, these methods become computationally prohibitive as the dimensionality of the model increases, motivating the development of scalable variational inference approaches.

To address this computational issue in high‐dimensional settings, we consider the Variational Bayes (VB) method, a computationally efficient approach to approximate Bayesian inference. VB approximates the posterior distribution

$$p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})\propto p(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})p(\mathbf{\Theta })$$

by introducing a tractable variational density $q(\mathbf{\Theta })\in Q$, where $Q$ denotes a chosen variational family, and seeks the optimal approximation by minimizing the Kullback‐Leibler (KL) divergence given by

$$q^{\ast }(\mathbf{\Theta })=\underset{q\in Q}{\text{argmin}}\text{KL}(q(\mathbf{\Theta })\Vert p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})),$$

where the KL divergence can be expanded as

$$\begin{array}{cc}\hfill \text{KL}(q(\mathbf{\Theta })\Vert p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z}))& =\int \log \left\{\frac{q(\mathbf{\Theta })}{p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})}\right\}q(\mathbf{\Theta })d\mathbf{\Theta }\hfill \\ \hfill & =\int \log \left\{\frac{q(\mathbf{\Theta })p(\mathbf{Y}|\mathbf{X},\mathbf{Z})}{p(\mathbf{\Theta },\mathbf{Y}|\mathbf{X},\mathbf{Z})}\right\}q(\mathbf{\Theta })d\mathbf{\Theta }\hfill \\ \hfill & =E_{q(\mathbf{\Theta })}\{\log q(\mathbf{\Theta })\}-E_{q(\mathbf{\Theta })}\{\log p(\mathbf{\Theta },\mathbf{Y}|\mathbf{X},\mathbf{Z})\}\hfill \\ \hfill & +\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z})\ge 0,\hfill \end{array}$$

where $E_{q(\mathbf{\Theta })}$ denotes expectation with respect to $q(\mathbf{\Theta })$. Define the evidence lower bound (ELBO) as

$$\text{ELBO}(q)=E_{q(\mathbf{\Theta })}\{\log p(\mathbf{\Theta },\mathbf{Y}|\mathbf{X},\mathbf{Z})\}-E_{q(\mathbf{\Theta })}\{\log q(\mathbf{\Theta })\}.$$

Then minimizing the KL divergence is equivalent to maximizing the ELBO

$$q^{\ast }(\mathbf{\Theta })=\underset{q\in Q}{\text{argmin}}\text{KL}(q(\mathbf{\Theta })\Vert p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z}))=\underset{q\in Q}{\text{argmax}}\text{ELBO}(q).$$ (5)

The complexity of the variational family $Q$ directly influences the difficulty of the associated optimization problem in (5). To simplify the optimization, we restrict $Q$ to the mean‐field variational family, denoted by $Q_{\text{MF}}$, under which the components of $\mathbf{\Theta }$ are assumed to be mutually independent in the variational distribution. Specifically, we adopt a fully factorized approximation of the form

$$\begin{array}{cc}\hfill q(\mathbf{\Theta })& =q(\mathit{\theta })q(\mathit{\gamma })q({\pi }_0)\left[\prod _{k=1}^{K}q({\mathit{\beta }}_k)\right]q(\sigma )\left[\prod _{k=1}^K\prod _{i=1}^{n_k}q({\xi }_{i,k})\right]\hfill \\ \hfill & =\prod _{j=1}^J{q}_j({\mathbf{\Theta }}_j),q\in Q_{\text{MF}}.\hfill \end{array}$$

Accordingly, the variational approximation is obtained by solving

$$q^{\ast }(\mathbf{\Theta })=\underset{q\in Q_{\text{MF}}}{\text{argmax}}\text{ELBO}(q).$$ (6)

3. Low‐Rank Variational Correction

Following the coordinate ascent variational inference (CAVI) strategy [59], we update each variational factor while holding all others fixed. In particular, the optimal variational density for the $j$th block, denoted by $q_j^{\ast }({\mathbf{\Theta }}_j)$, is obtained by maximizing the ELBO with respect to $q_j({\mathbf{\Theta }}_j)$. Let

$${\mathbf{\Theta }}_{-j}=\{{\mathbf{\Theta }}_{\ell }:\ell \ne j,\ell =1,\dots ,J\},$$

and define

$$q_{-j}({\mathbf{\Theta }}_{-j})=\prod _{\ell \ne j}{q}_{\ell }({\mathbf{\Theta }}_{\ell }).$$

Then the optimal variational factor satisfies

$$q_j^{\ast }({\mathbf{\Theta }}_j)\propto \exp \left[E_{-j}\{\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z})\}\right],$$ (7)

where $E_{-j}(·)$ denotes expectation with respect to $q_{-j}({\mathbf{\Theta }}_{-j})$. Equation (7) shows that each variational factor is updated by taking the exponential of the expected complete‐data log joint density, leading to closed‐form updates whenever the resulting distribution belongs to a known exponential family.

The optimal density $q^{\ast }({\pi }_0)$ is given by

$$q^{\ast }({\pi }_0)\sim \text{Beta}(a,b),$$ (8)

where $a={\sum }_{j=1}^p{\mu }_{{\gamma }_j}^{\ast }+a_0$, $b=p-{\sum }_{j=1}^p{\mu }_{{\gamma }_j}^{\ast }+b_0$. Then we have ${\mu }_{\log ({\pi }_0)}^{\ast }={\mathrm{E}}_{q^{\ast }}(\log ({\pi }_0))=\psi (a)-\psi (a+b)$ and ${\mu }_{\log (1-{\pi }_0)}^{\ast }={\mathrm{E}}_{q^{\ast }}(\log (1-{\pi }_0))=\psi (b)-\psi (a+b)$. The mean estimate ${\pi }_0^{\ast }$ of ${\pi }_0$ for the variational density $q^{\ast }({\pi }_0)$ is ${\mu }_{{\pi }_0}^{\ast }={\mathrm{E}}_{q^{\ast }}({\pi }_0)=\frac{a}{a+b}$.

The optimal density $q^{\ast }({\mathit{\beta }}_k)$ for $1\le k\le K$ is given by

$$q^{\ast }({\mathit{\beta }}_k)\sim N({\mathit{\mu }}_k,{\mathbf{\sum }}_k),$$ (9)

where ${\mathbf{\sum }}_k={\left(\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}{\sum }_{i=1}^{n_k}{\mu }_{1/{\xi }_{i,k}}^{\ast }{\mathbf{Z}}_{i,k}^{\top }{\mathbf{Z}}_{i,k}+{\mathbf{\sum }}_{0,k}\right)}^{-1}$ and ${\mathit{\mu }}_k={\mathbf{\sum }}_k(\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}{\sum }_{i=1}^{n_i}{\mathbf{Z}}_{i,k}({\mu }_{1/{\xi }_{i,k}}^{\ast }$ $(Y_{i,k}-$ ${\mathbf{X}}_{i,k}^{\top }{\mathit{\mu }}_{\mathit{\theta }}^{\ast })-{\kappa }_1))$. The mean estimate ${\mathit{\beta }}_k^{\ast }$ of ${\mathit{\beta }}_k$ for the variational density $q^{\ast }({\mathit{\beta }}_k)$ is ${\mathit{\mu }}_{{\mathit{\beta }}_k}^{\ast }={\mathrm{E}}_{q^{\ast }}({\mathit{\beta }}_k)={\mathit{\mu }}_k$.

The optimal density $q^{\ast }({\xi }_{i,k})$ for $1\le k\le K,1\le i\le n_k$ is given by

$$q^{\ast }({\xi }_{i,k})\sim \text{GIG}\left(\frac{1}{2},{\chi }_{i,k},{\psi }_{i,k}\right),$$ (10)

where $\text{GIG}(\lambda ,{\chi }_{i,k},{\psi }_{i,k})$ represents the generalized inverse Gaussian distribution with shape parameter $\lambda$ and scale parameter ${\chi }_{i,k},{\psi }_{i,k}$, and its density function is $p(x)=\frac{{({\chi }_{i,k}/{\psi }_{i,k})}^{\lambda /2}}{2K_{\lambda }(\sqrt{{\chi }_{i,k}{\psi }_{i,k}})}{x}^{\lambda -1}{e}^{-\frac{1}{2}({\chi }_{i,k}x+{\psi }_{i,k}/x)}I(x\ge 0)$, with $K_{\lambda }(·)$ being the modified Bessel function of the second kind and ${\chi }_{i,k}=\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}[{(Y_{i,k}-{\mathbf{X}}_{i,k}^{\top }{\mathit{\mu }}_{\mathit{\theta }}^{\ast }-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\mu }}_{{\mathit{\beta }}_k}^{\ast })}^2+{\mathbf{X}}_{i,k}^{\top }\mathbf{\sum }{\mathbf{X}}_{i,k}$ $+{\mathbf{Z}}_{i,k}^{\top }{\mathbf{\sum }}_k{\mathbf{Z}}_{i,k}]$, ${\psi }_{i,k}={\mu }_{1/\sigma }^{\ast }\left(2+\frac{{\kappa }_1^2}{{\kappa }_2^2}\right)$. We have ${\mu }_{{\xi }_{i,k}}^{\ast }={\mathrm{E}}_{q^{\ast }}({\xi }_{i,k})=\frac{{\sqrt{\psi }}_{i,k}{K}_{-3/2}(\sqrt{{\chi }_{i,k}{\psi }_{i,k}})}{{\sqrt{\chi }}_{i,k}{K}_{1/2}(\sqrt{{\chi }_{i,k}{\psi }_{i,k}})}$ and ${\mu }_{1/{\xi }_{i,k}}^{\ast }={\mathrm{E}}_{q^{\ast }}(1/{\xi }_{i,k})=-\frac{1}{{\chi }_{i,k}}$ $+\frac{{\sqrt{\chi }}_{i,k}{K}_{-3/2}(\sqrt{{\chi }_{i,k}{\psi }_{i,k}})}{{\sqrt{\psi }}_{i,k}{K}_{1/2}(\sqrt{{\chi }_{i,k}{\psi }_{i,k}})}$. The mean estimate ${\xi }_{i,k}^{\ast }$ of ${\xi }_{i,k}$ for the variational density $q^{\ast }({\xi }_{i,k})$ is ${\mu }_{{\xi }_{i,k}}^{\ast }$.

The optimal density $q^{\ast }(\sigma )$ is given by

$$q^{\ast }(\sigma )\sim \text{IG}(\frac{r}{2},\frac{s}{2}),$$ (11)

where $\text{IG}(r/2,s/2)$ represents the inverse gamma distribution with shape parameter $r/2$ and scale parameter $s/2$. Specifically, $r=r_0+3{\sum }_{k=1}^K{n}_k$ and $s=\frac{1}{{\kappa }_2^2}[{\sum }_{k=1}^K{\sum }_{i=1}^{n_k}{\kappa }_1^2{\mu }_{{\xi }_{i,k}}^{\ast }-2{\kappa }_1(Y_{i,k}-{\mathbf{X}}_{i,k}^{\top }{\mathit{\mu }}_{\mathit{\theta }}^{\ast }-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\mu }}_{{\mathit{\beta }}_k}^{\ast })+{\mu }_{1/{\xi }_{i,k}}^{\ast }({(Y_{i,k}-{\mathbf{X}}_{i,k}^{\top }{\mathit{\mu }}_{\mathit{\theta }}^{\ast }-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\mu }}_{{\mathit{\beta }}_k}^{\ast })}^2+{\mathbf{Z}}_{i,k}^{\top }{\mathbf{\sum }}_k\mathbf{Z}+{\mathbf{X}}_{i,k}^{\top }\mathbf{\sum }{\mathbf{X}}_{i,k})]$ $+2{\sum }_{k=1}^K{\sum }_{i=1}^{n_k}{\mu }_{1/{\xi }_{i,k}}^{\ast }+s_0$. Then we have ${\mu }_{\sigma }^{\ast }={\mathrm{E}}_{q^{\ast }}(\sigma )=s/r$ and ${\mu }_{1/\sigma }^{\ast }={\mathrm{E}}_{q^{\ast }}(\sigma )=r/s$. The mean estimate ${\sigma }^{\ast }$ of $\sigma$ for the variational density $q^{\ast }(\sigma )$ is ${\mu }_{\sigma }^{\ast }$.

It is worth mentioning that the approximate posterior distribution of $\mathit{\theta }$ is complicated by the above methods. In this paper, we employ the idea of Laplace approximation to update the variational distribution $q^{\ast }(\mathit{\theta })$. Based on [60], the approximate Laplace distribution of $q^{\ast }(\mathit{\theta })$ is $N({\mathit{\theta }}_0,{\mathbf{H}}^{-1}{|}_{\mathit{\theta }={\mathit{\theta }}_0})$, where $\mathbf{H}$ is negative Hessian matrix of $\log p(\mathbf{\Theta },\mathbf{Y}|\mathbf{X},\mathbf{Z})$, and the covariance of $\mathit{\theta }$ is ${\mathbf{H}}^{-1}$, evaluated in $\mathit{\theta }={\mathit{\theta }}_0$. To find the mode, we solve for ${\mathit{\theta }}_0$ from the following equation given by

$$\mathbf{H}{|}_{\mathit{\theta }={\mathit{\theta }}_0}{\mathit{\theta }}_0=\nabla {|}_{\mathit{\theta }={\mathit{\theta }}_0}+\mathbf{H}{|}_{\mathit{\theta }={\mathit{\theta }}_0}{\mathit{\theta }}_0,$$

where $\nabla {|}_{\mathit{\theta }={\mathit{\theta }}_0}$ is the gradient of $\log p(\mathbf{\Theta },\mathbf{Y}|\mathbf{X},\mathbf{Z})$. Specifically, calculate

$$\begin{array}{cc}\hfill & \nabla =-\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}\sum _{k=1}^K\sum _{i=1}^{n_k}\left(-{\mu }_{1/{\xi }_{i,k}}^{\ast }{\mathbf{X}}_{i,k}^{\top }(Y_{i,k}-{\mathbf{X}}_{i,k}^{\top }{\mathit{\theta }}_0-{\mathbf{Z}}_{i,k}^{\top }{\mathit{\mu }}_{{\mathit{\beta }}_k}^{\ast })-{\kappa }_1{\mathbf{X}}_{i,k}^{\top }\right)\hfill \\ \hfill & -{\mathit{\theta }}_0\circ \left({\mathit{\mu }}_{\mathit{\gamma }}^{\ast }{\tau }_1^{-2}+(1-{\mathit{\mu }}_{\mathit{\gamma }}^{\ast }){\tau }_0^{-2}\right),\hfill \\ \hfill & \mathbf{H}=\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}\sum _{k=1}^K\sum _{i=1}^{n_k}{\mu }_{1/{\xi }_{i,k}}^{\ast }{\mathbf{X}}_{i,k}^{\top }{\mathbf{X}}_{i,k}+\text{diag}\left({\mathit{\mu }}_{\mathit{\gamma }}^{\ast }{\tau }_1^{-2}+(1-{\mathit{\mu }}_{\mathit{\gamma }}^{\ast }){\tau }_0^{-2}\right).\hfill \end{array}$$

where $\circ$ is the Hadama product, which represents the product of the corresponding elements of the two vectors, diag($x$) represents that each element of the vector $x$ is used as the diagonal matrix of diagonal elements. We can iterate

$${\mathit{\theta }}_0^{\ast }={\mathbf{H}}^{-1}(\nabla +\mathbf{H}{\mathit{\theta }}_0).$$ (12)

As $\mathbf{H}$ is a $p\times p$ matrix, it becomes difficult to compute its inverse, when the dimension $p$ is high. Therefore, when $p>N={\sum }_{k=1}^K{n}_k$, we can use the Woodbury matrix identity to compute the inverse of $\mathbf{H}$

$${(\mathbf{A}+\mathbf{U}\mathbf{C}\mathbf{V})}^{-1}={\mathbf{A}}^{-1}-{\mathbf{A}}^{-1}\mathbf{U}{\left({\mathbf{C}}^{-1}+\mathbf{V}{\mathbf{A}}^{-1}\mathbf{U}\right)}^{-1}\mathbf{V}{\mathbf{A}}^{-1},$$

where diagonal matrix $\mathbf{A}$ is of size $p\times p$, $\mathbf{U}$ is $p\times N$, $\mathbf{C}$ is $N\times N$, and $\mathbf{V}$ is $N\times p$. In our model $\mathbf{H}$ can be expressed as $\mathbf{A}=\text{diag}\left({\mathit{\mu }}_{\mathit{\gamma }}^{\ast }{\tau }_1^{-2}+(1-{\mathit{\mu }}_{\mathit{\gamma }}^{\ast }){\tau }_0^{-2}\right),\mathbf{U}={\mathbf{X}}^{\top },\mathbf{V}=\frac{{\mu }_{1/\sigma }^{\ast }}{{\kappa }_2^2}{\mathbf{X}}^{\top }\text{diag}({\mathit{\mu }}_{1/\xi }^{\ast }),\mathbf{C}={\mathbf{I}}_N$, where $\mathbf{X}={({\mathbf{X}}_{1,1},\dots ,{\mathbf{X}}_{n_1,1},\dots ,{\mathbf{X}}_{1,K},\dots ,{\mathbf{X}}_{n_K,K})}^{\top }$ and ${\mathit{\mu }}_{1/\xi }^{\ast }={({\mu }_{1/{\xi }_{1,1}}^{\ast },\dots ,{\mu }_{1/{\xi }_{n_1,1}}^{\ast },\dots ,{\mu }_{1/{\xi }_{1,K}}^{\ast },\dots ,{\mu }_{1/{\xi }_{n_K,K}}^{\ast })}^{\top }$.

The estimate ${\mathit{\theta }}_0^{\ast }$ obtained through the Laplace method serves only as an initial approximation; we also need to incorporate the selection coefficient $\mathit{\gamma }$ for effective selection of variables. To derive the optimal density $q^{\ast }(\mathit{\gamma })$, we denote

$$\begin{array}{cc}\hfill {\zeta }_j& ={\mu }_{\log ({\pi }_0)}^{\ast }-{\mu }_{\log (1-{\pi }_0)}^{\ast }+\frac{1}{2}(-\log ({\tau }_1^2)+\log ({\tau }_0^2))\hfill \\ \hfill & +\frac{{\theta }_{0,j}^{\ast 2}+{\sum }_{jj}}{2}\left(-\frac{1}{{\tau }_1^2}+\frac{1}{{\tau }_0^2}\right),\hfill \end{array}$$ (13)

where ${\sum }_{jj}$ is the $j$th diagonal element of $\mathbf{\sum }$. Thus we have $q^{\ast }({\gamma }_j)\sim \text{Bernoulli}\left(\frac{\exp ({\zeta }_j)}{\exp ({\zeta }_j)+1}\right)$. The mean estimate ${\gamma }_j^{\ast }$ of ${\gamma }_j$ for the variational density $q^{\ast }({\gamma }_j)$ is ${\mu }_{{\gamma }_j}^{\ast }=\left(\frac{\exp ({\zeta }_j)}{\exp ({\zeta }_j)+1}\right)$ and ${\mathit{\mu }}_{\mathit{\gamma }}^{\ast }={({\mu }_{{\gamma }_1}^{\ast },\dots ,{\mu }_{{\gamma }_p}^{\ast })}^{\top }$. Then we obtain that the mean estimate ${\mathit{\theta }}^{\ast }$ of $\mathit{\theta }$ is ${\mathit{\mu }}_{\mathit{\theta }}^{\ast }={\mathit{\mu }}_{\mathit{\gamma }}^{\ast }\circ {\mathit{\theta }}_0^{\ast }$.

In summary, we obtain explicit forms of the updates required to obtain the optimal parameters of $q^{\ast }({\mathit{\theta }}_0)$, $q^{\ast }(\mathit{\gamma })$, $q^{\ast }({\pi }_0)$, $q^{\ast }({\mathit{\beta }}_k)$, $q^{\ast }(\sigma )$ and $q^{\ast }({\xi }_{i,k})$ for $k=1,\dots ,K$ and $i=1,\dots ,n_k$, which can be summarized in Algorithm 1 with its derivation deferred to Supporting Information A. We clarify that we use the mean of the approximate posterior as the point estimate for each parameter. Because our method yields the full approximate posterior distribution, it readily enables uncertainty quantification (e.g., construction of credible intervals).

ALGORITHM 1. An iterative scheme for obtaining the parameters in the optimal densities $q^{\ast }({\mathit{\theta }}_0)$, $q^{\ast }(\mathit{\gamma })$, $q^{\ast }({\pi }_0)$, $q^{\ast }({\mathit{\beta }}_k)$, $q^{\ast }(\sigma )$ and $q^{\ast }({\xi }_{i,k})$ for $k=1,\dots ,K$ and $i=1,\dots ,n_k$ under the multi‐source heterogeneous quantile linear regression models.

Having established the explicit update forms, we next examine the theoretical properties of the proposed variational algorithm. In particular, we aim to ensure that the optimization procedure is stable and that the evidence lower bound (ELBO) converges under mild regularity conditions. To this end, we introduce the following assumptions, which impose smoothness and boundedness conditions sufficient to guarantee monotonic improvement of the ELBO.

To establish convergence of the ELBO sequence under the conditional model, we impose the following regularity conditions. Let $𝒜$ denote the index set of nonconjugate continuous blocks updated via Laplace approximation.

(A1) For each $j\in 𝒜$, define $h_j({\mathbf{\Theta }}_j)=E_{q_{-j}}[\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z})].$ Assume that the function $h_j({\mathbf{\Theta }}_j)$ is twice continuously differentiable in ${\mathbf{\Theta }}_j$. Moreover, there exist constants $0<m_j\le M_j<\infty$ such that

$$m_{j}I⪯-{\nabla }_{{\mathbf{\Theta }}_j}^2{h}_j({\mathbf{\Theta }}_j)⪯M_{j}I\text{for all}{\mathbf{\Theta }}_j.$$

(A2) The posterior distribution $p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})$ is proper, that is, $\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z})<\infty .$ Consequently,

$$\text{ELBO}(q)\le \log p(\mathbf{Y}|\mathbf{X},\mathbf{Z})$$

for all admissible variational distributions $q$.

Assumption (A1) guarantees sufficient smoothness and uniform curvature bounds, enabling construction of a global quadratic lower bound for each nonconjugate block. This yields a strictly concave Gaussian surrogate objective and ensures monotone ascent of the ELBO under each Laplace update [61, 62]. Assumption (A2) ensures that the ELBO sequence is bounded above. Therefore, convergence of the ELBO values follows from monotonicity together with the standard variational inequality

$$\text{ELBO}(q)\le \log p(\mathbf{Y}|\mathbf{X},\mathbf{Z}),$$

as established in [28, 63, 64].

Theorem 1

Under Assumptions (A1)–(A2), consider the variational algorithm that updates the parameter blocks sequentially, using exact mean‐field (CAVI) updates for conjugate blocks and Laplace updates for nonconjugate blocks. Then each block update does not decrease the ELBO. Consequently, the sequence ${\{\text{ELBO}(q^{(t)})\}}_{t\ge 0}$ is nondecreasing and bounded above by $\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z}).$

Therefore, there exists a finite constant

$L^{\ast }<\infty$

such that

$$\text{ELBO}(q^{(t)})\to L^{\ast }\text{as}t\to \infty .$$

This theorem provides a basic convergence guarantee for the proposed hybrid CAVI–Laplace algorithm. Specifically, each block update does not decrease the ELBO, so the objective value evolves monotonically throughout the iterative procedure. Combined with Assumption (A2), which guarantees that the ELBO is uniformly bounded above by the finite log marginal likelihood $\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z})$, the ELBO sequence is therefore bounded above. Together with its monotonic non‐decreasing property, this implies that the ELBO sequence converges to a finite limit. Therefore, under Assumptions (A1)–(A2), Algorithm 1 defines a stable variational optimization procedure, including in the presence of nonconjugate blocks handled by Laplace updates.

However, when dealing with heterogeneous data from multiple sources, the model must accommodate varying effect structures across datasets, which increases its complexity by introducing additional parameters or latent heterogeneity [65]. This complicates the posterior landscape and makes it challenging to achieve accurate inference using a simple Laplace approximation. Our goal is to refine the mean of the Gaussian approximation to obtain a more accurate estimate. Building on the work of [36], we propose a low‐rank variational Bayesian correction method under spike‐and‐slab priors. Of particular note from Equation (12) is that ${\mathit{\theta }}_0^{\ast }$ is implicitly corrected by explicitly correcting the estimated gradient

$${\mathit{\theta }}_1^{\ast }={\mathit{\theta }}_0^{\ast }+{\mathbf{H}}^{-1}\mathit{\lambda }={\mathit{\theta }}_0^{\ast }+\mathbf{\sum }\mathit{\lambda },$$ (14)

where $\mathit{\lambda }$ is the $p\times 1$ dimensional correction parameter. The current central issue is the estimation of $\mathit{\lambda }$. Instead of using the ELBO, we return to the fundamental ideas of variational Bayes [66] and the recent optimization perspective on Bayesian rules [67]. Using the prior information about the unknown parameter $p(\mathit{\theta })$ and the conditional likelihood function of the data $p(\mathbf{Y}|\mathit{\theta },\mathbf{X},\mathbf{Z})$, we need to compute

$$\begin{array}{cc}\hfill \widehat{\mathit{\lambda }}& =\arg \underset{\mathit{\lambda }}{\min }\left[{\mathrm{E}}_{\mathit{\theta }\sim N\left({\mathit{\theta }}_0^{\ast }+\mathbf{\sum }\mathit{\lambda },\mathbf{\sum }\right)}[-\log p(\mathbf{Y}|\mathit{\theta },\mathbf{X},\mathbf{Z})]\right.\hfill \\ \hfill & \left.+\text{KL}\left(\varphi \left(\mathit{\theta }|{\mathit{\theta }}_0^{\ast }+\mathbf{\sum }\mathit{\lambda },\mathbf{\sum }\right)\Vert p(\mathit{\theta })\right)\right],\hfill \end{array}$$ (15)

where $\varphi$ is the probability density function of multivariate normal distribution. This $p$‐dimensional optimization problem becomes computationally expensive as $p$ increases. Given the high‐dimensional nature of the problem, our focus is on estimating non‐sparse variables. Therefore, we only make corrections in the direction of non‐sparse variables (i.e., those for which ${\gamma }_j>0.5$). Let the set of indices be denoted as $I=\{j:{\mu }_{{\gamma }_j}^{\ast }>0.5\}$. We then extract the relevant columns of $\mathbf{\sum }$ and denote this submatrix by ${\mathbf{\sum }}_I$. Additionally, ${\mathit{\lambda }}_I$ represents the $s\times 1$ dimensional correction parameter, where $s=\Vert {\mathit{\theta }}_0^{\ast }{\Vert }_0$.

In Equation (15), we substitute ${\mathbf{\sum }}_I$ and ${\mathit{\lambda }}_I$ for the corresponding terms, resulting in a low‐rank corrected estimate of ${\mathit{\theta }}_1^{\ast }$, expressed as ${\mathit{\theta }}_1^{\ast }={\mathit{\theta }}_0^{\ast }+{\mathbf{\sum }}_I{\widehat{\mathit{\lambda }}}_I$. Then we can obtain that the mean estimate ${\mathit{\theta }}^{\ast }$ of $\mathit{\theta }$ is ${\mathit{\mu }}_{{\mathit{\theta }}_{LR}}^{\ast }={\mathit{\mu }}_{\mathit{\gamma }}^{\ast }\circ {\mathit{\theta }}_1^{\ast }$. In summary, we derive low‐rank correction forms of the $\mathit{\theta }$ and outline the procedures in Algorithm 2. The algorithm for simultaneously selecting variables and performing low‐rank correction for both types of variables is shown in the Supplementary C. The LRQVB R package, which implements the proposed approach, is publicly available on CRAN and can also be downloaded from https://cran.r‐project.org/web/packages/LRQVB/index.html. LRQVB supports variable selection on homogeneous coefficients only or on both homogeneous and heterogeneous coefficients, and returns both variational and low‐rank corrected estimates.

ALGORITHM 2. Low‐rank variational correction.

4. Bayesian Local Influence Analysis

4.1. Bayesian Perturbation Model and Manifold

Following [47], we consider a class of perturbation models that simultaneously perturb the prior distribution, the observed data, and the sampling mechanism. Under our conditional modeling framework, the perturbed joint density is given by

$$p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega })=p(\mathbf{\Theta }|{\mathit{\omega }}_p)\prod _{k=1}^K\prod _{i=1}^{n_k}p(Y_{i,k}|\mathbf{\Theta },{\mathbf{X}}_{i,k},{\mathbf{Z}}_{i,k},{\mathit{\omega }}_d,{\mathit{\omega }}_s),$$

which satisfies the normalization condition $\int p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega })d\mathbf{Y}d\mathbf{\Theta }=1.$ Here, ${\mathit{\omega }}_p\in {ℝ}^{m_p}$, ${\mathit{\omega }}_d\in {ℝ}^{m_d}$, and ${\mathit{\omega }}_s\in {ℝ}^{m_s}$ denote perturbations to the prior, data, and sampling components, respectively, with total dimension $m=m_p+m_d+m_s$. We assume that ${\mathit{\omega }}^0=({\mathit{\omega }}_p^0,{\mathit{\omega }}_d^0,{\mathit{\omega }}_s^0)\in {ℝ}^m$ corresponds to the unperturbed model.

Under suitable regularity conditions and following the geometric framework of [47], the model class $ℳ=\{p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega }):\mathit{\omega }\in {ℝ}^m\}$ can be viewed as an $m$‐dimensional manifold indexed by $\mathit{\omega }$. The tangent space $T_{{\mathit{\omega }}^0}$ of $ℳ$ at ${\mathit{\omega }}^0$ is spanned by the $m$ score functions ${\partial }_{{\omega }_k}\ell ({\mathit{\omega }}^0)={\left.\frac{\partial \ell (\mathit{\omega })}{\partial {\omega }_k}\right|}_{\mathit{\omega }={\mathit{\omega }}^0},k=1,\dots ,m,$ where $\ell (\mathit{\omega })=\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega }).$ Moreover, the Fisher information metric is defined by $g_{jk}(\mathit{\omega })=E_{\mathit{\omega }}\left[{\partial }_{{\omega }_j}\ell (\mathit{\omega }){\partial }_{{\omega }_k}\ell (\mathit{\omega })\right]=E_{\mathit{\omega }}\left[-{\partial }_{{\omega }_j{\omega }_k}^2\ell (\mathit{\omega })\right],j,k=1,\dots ,m,$ where $E_{\mathit{\omega }}(·)$ denotes expectation under the perturbed conditional joint density $p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega }),$ and ${\partial }_{{\omega }_j{\omega }_k}^2=\frac{{\partial }^2}{\partial {\omega }_j\partial {\omega }_k}.$ Let $\mathbf{G}(\mathit{\omega })=\left(g_{jk}(\mathit{\omega })\right)$ denote the Fisher information matrix. The diagonal element $g_{jj}({\mathit{\omega }}^0)$ quantifies the local sensitivity of the model to perturbation ${\omega }_j$, while the normalized quantity ${\rho }_{jk}=\frac{g_{jk}(\mathit{\omega })}{\sqrt{g_{jj}(\mathit{\omega })g_{kk}(\mathit{\omega })}}$ measures the local association between perturbation components ${\omega }_j$ and ${\omega }_k$.

If $G({\mathit{\omega }}^0)$ is a diagonal matrix, then all components of $\mathit{\omega }$ are orthogonal to each other, and this perturbation scheme is known as an appropriate perturbation. On the other hand, when $G({\mathit{\omega }}^0)$ is not a diagonal matrix, we can always choose a new perturbation vector $\tilde{\mathit{\omega }}={\mathit{\omega }}^0+\mathit{G}{({\mathit{\omega }}^0)}^{1/2}(\mathit{\omega }-{\mathit{\omega }}^0)$ such that $G(\tilde{\mathit{\omega }})$ evaluated at ${\mathit{\omega }}^0$ becomes $c{I}_m$, where $c$ is a positive scalar.

We now introduce the following perturbation schemes on the multi‐source heterogeneous quantile linear regression models.

Example 1

Consider the perturbations of a class of sampling distribution given by

$$\begin{array}{cc}\hfill p(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z},\mathit{\omega })& =p(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})\hfill \\ \hfill & \exp \{\sum _{k=1}^K\sum _{i=1}^{n_k}\left[{\omega }_{i,k}{\rho }_{i,k}(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})\right]-\frac{1}{2}{\omega }_{i,k}^2{\rho }_{i,k}{(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})}^2\hfill \\ \hfill & \hspace{0.17em}-𝒞(𝒞(\mathbf{\Theta },\mathbf{X},\mathbf{Z},\mathit{\omega }))\},\hfill \end{array}$$ (16)

where $𝒞(\mathbf{\Theta },\mathbf{X},\mathbf{Z},\mathit{\omega })$ is the normalization constant, and ${\rho }_{i,k}(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})$ is an arbitrary zero‐mean scalar function. In this case, ${\mathit{\omega }}^0=(0,0,\cdots ,0)$ indicates no perturbation. The tangent space $T_{\mathit{\omega }}$ of $ℳ$ is spanned by ${\partial }_{{\omega }_{i,k}}\ell (\mathit{\omega }){|}_{\mathit{\omega }={\mathit{\omega }}^0}={\rho }_{i,k}(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})-\frac{\partial 𝒞(\mathbf{\Theta },\mathbf{X},\mathbf{Z},\mathit{\omega })}{\partial {\omega }_{i,k}}{|}_{\mathit{\omega }={\mathit{\omega }}^0}.$. It is easily shown that

$$G({\mathit{\omega }}^0)=\text{diag}(g_{11},\dots ,g_{tt})+E_{\mathit{\omega }}\{{\partial }^2𝒞(𝒞(\mathbf{\Theta },\mathbf{X},\mathbf{Z},\mathit{\omega }))/\partial \mathit{\omega }\partial {\mathit{\omega }}^{\top }\},$$

where $g_{i,k}=E_{{\mathit{\omega }}^0}\{{\rho }_{i,k}{(\mathbf{Y}|\mathbf{\Theta },\mathbf{X},\mathbf{Z})}^2\},,i=1,\cdots ,m$ and $m={\sum }_{k=1}^K{n}_k$.

4.2. Bayesian Local Influence Measures

Consider a finite‐dimensional manifold $ℳ$ and a $m\times 1$ objective function $f(\mathit{\omega }):ℳ\to {ℝ}^m$, such as the Bayes factor, $\varphi$ divergence, and posterior mean distance. Let $\mathit{\omega }(t)$ be a measurement on $ℳ$ with $\mathit{\omega }(0)={\mathit{\omega }}^0$ and ${\partial }_t\mathit{\omega }(t){|}_{t=0}=h\in {ℝ}^m$. By using Taylor expansion, we have $\mathit{f}(\mathit{\omega }(t))=\mathit{f}(\mathit{\omega }(0))+{\dot{\mathit{f}}}_h(0)t+O(t^2)$, where ${\dot{\mathit{f}}}_h(0)={\nabla }_f^{\top }h={\partial }_{\mathit{\omega }}^{\top }f({\mathit{\omega }}^0)h$.

First, consider the case with ${\nabla }_f\ne 0$. In this scenario, the first‐order influence (FI) measure in the direction $\mathit{h}\in {ℝ}^m$ is defined as

$$F{I}_{\mathit{f},\mathit{h}}=F{I}_{\mathit{f}({\omega }^0),\mathit{h}}=\frac{{\mathit{h}}^{\top }{\nabla }_f^{\top }{\mathit{W}}_f{\nabla }_f\mathit{h}}{{\mathit{h}}^{\top }\mathit{G}\mathit{h}},$$

where $\mathit{G}\mathit{=}\mathit{G}({\mathit{\omega }}^{\mathit{0}})$ and ${\mathit{W}}_f$ is some user‐specified positive semi‐definite matrix. In particular, for an appropriate perturbation $\tilde{\mathit{\omega }},F{I}_{\mathit{f},\mathit{h}}$ can be rewritten as

$$F{I}_{f(\tilde{\mathit{\omega }}),h}{|}_{\tilde{\mathit{\omega }}={\mathit{\omega }}^0}=\frac{{\mathit{h}}^{\top }{\mathit{G}}^{-1/2}{\nabla }_f{\mathit{W}}_f{\nabla }_f^{\top }{\mathit{G}}^{-1/2}\mathit{h}}{{\mathit{h}}^{\top }\mathit{h}}.$$

Let $\mathit{Q}={\mathit{G}}^{-\frac{1}{2}}{\nabla }_f{\mathit{W}}_f{\nabla }_f^{\top }{\mathit{G}}^{-\frac{1}{2}},\mathit{B}=\frac{\mathit{Q}}{tr(\mathit{Q})}$. We then have $FI{C}_{f({\tilde{\omega }}^0),\mathbf{h}}=\frac{{\mathbf{h}}^{\top }\mathbf{Qh}}{tr(\mathbf{Q})}$. Similar to [49], we can use $M{(0)}_j=FI{C}_{\mathit{f}({\tilde{\mathit{\omega }}}^0),{\mathit{e}}_j}=b_{jj}$ for $j=1,\cdots ,m$ to evaluate the impact of different minor perturbations. Here, $e_j$ represents a basic perturbation vector with the $j$th element being 1 and 0 elsewhere, and $b_{jj}$ is the $j$th diagonal element of the matrix $B$. A benchmark for assessment can be $\bar{M}(0)+2SM(0)$, where $\bar{M}(0)$ and $SM(0)$ are the mean and standard error of $\{M{(0)}_j,j=1,\cdots ,m\}$, respectively.

Example 2

(The Bayes factor) We take $f(\mathit{\omega })$ to be the Bayes factor defined by $BF(\mathit{\omega })=\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z},\mathit{\omega })-\log p(\mathbf{Y}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0),$ where $p(\mathbf{Y}|\mathbf{X},\mathbf{Z},\mathit{\omega })=\int p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega })d\mathbf{\Theta }$. $BF(\mathit{\omega })$ is a continuous mapping from $ℳ$ to $ℝ$ from the Bayesian perturbation model. If we denote ${\mathit{W}}_f=\mathit{I}$, then ${\nabla }_{BF}=E\left[{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)|\mathbf{Y},\mathbf{X},\mathbf{Z}\right]$, where the expectation $E(.)$ is obtained for the density $p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)$. Thus, ${\nabla }_{BF}$ can be approximated by

$${\nabla }_{BF}\approx \frac{1}{S}\sum _{s=1}^S{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},{\mathbf{\Theta }}^{(s)}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0),$$ (17)

where the observed values $\{{\mathbf{\Theta }}^{(s)}:s=1,\cdots ,S\}$ are generated from the approximate distribution $q(\mathbf{\Theta })$.

When ${\nabla }_f=0$, Taylor expansion yields $f(\mathit{\omega }(t))=f(\mathit{\omega }(0))+\frac{1}{2}\ddot{f_h}(0)t^2+O(t^3),$ where $\ddot{f_h}(0)={\mathit{h}}^{\top }{\mathit{H}}_f\mathit{h}$, ${\mathit{H}}_f={\partial }_{\mathit{\omega }}^{2}f({\mathit{\omega }}^0)$. The second‐order effect measurement (SI) on the direction $\mathit{h}\in {ℝ}^m$ is defined as

$$S{I}_{f,\mathit{h}}=\frac{{\mathit{h}}^{\top }{\mathit{H}}_f\mathit{h}}{{\mathit{h}}^{\top }\mathit{G}\mathit{h}}.$$

For the appropriate perturbation $\tilde{\mathit{\omega }}$, $S{I}_{f,\mathit{h}}$ degenerates into the following expression

$$S{I}_{f(\tilde{\mathit{\omega }}),\mathit{h}}{|}_{\tilde{\mathit{\omega }}={\mathit{\omega }}^0}=\frac{{\mathit{h}}^{\top }{\mathit{G}}^{-1/2}{\mathit{H}}_f{\mathit{G}}^{-1/2}\mathit{h}}{{\mathit{h}}^{\top }\mathit{h}}\text{and}SI{C}_{f(\widetilde{{\mathit{\omega }}^0}),\mathit{h}}=\frac{{\mathit{h}}^{\top }{\mathit{Q}}_s\mathit{h}}{\text{tr}({\mathit{Q}}_s)},$$

where $\mathit{Q}={\mathit{G}}^{-1/2}{\mathit{H}}_f{\mathit{G}}^{-1/2}$.

Similar to the first‐order influence measure, we only need to consider the diagonal elements of $\mathit{B}=\frac{\mathit{Q}}{\text{tr}(\mathit{Q})}$, which can also be used to identify strong influence points, unrobust priors, and inappropriate sample distributions.

Example 3

($\varphi$‐divergence) We take $f(\mathit{\omega })$ to be the $\varphi$‐divergence between the posterior distributions before and after the introduction of perturbation $\mathit{\omega }$, defined by

$$D_{\varphi }(\mathit{\omega })=\int \varphi (R(\mathbf{\Theta }|\mathit{\omega }))p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})d\mathbf{\Theta },$$

where $R(\mathbf{\Theta }|\mathit{\omega })=\frac{p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z},\mathit{\omega })}{p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z})},$ and $\varphi (·)$ is a convex function satisfying $\varphi (1)=0$. Then ${\nabla }_{\varphi }=\mathbf{0},$ and the Hessian matrix at the unperturbed point ${\mathit{\omega }}^0$ is given by where $a^{\otimes 2}=a{a}^{\top }$, and the expectation is taken with respect to $p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z}).$ Similarly, ${\mathit{H}}_{\varphi }$ can be approximated by

$$\begin{array}{cc}\hfill & \ddot{\varphi }(1)\left[\frac{1}{S}\sum _{s=1}^S{\left\{{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},{\mathbf{\Theta }}^{(s)}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)\right\}}^{\otimes 2}\right.\hfill \\ \hfill & \left.-{\left(\frac{1}{S}\sum _{s=1}^S{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},{\mathbf{\Theta }}^{(s)}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)\right)}^{\otimes 2}\right],\hfill \end{array}$$ (18)

where $\{{\mathbf{\Theta }}^{(s)}:s=1,\dots ,S\}$ are sampled from the variational approximation $q(\mathbf{\Theta })$.

Example 4

(Posterior mean distance) After introducing the perturbation $\mathit{\omega }$, define the posterior mean of a function $h(\mathbf{\Theta })$ by ${\mathit{M}}_h(\mathit{\omega })=\int h(\mathbf{\Theta })p(\mathbf{\Theta }|\mathbf{Y},\mathbf{X},\mathbf{Z},\mathit{\omega })d\mathbf{\Theta }.$ To assess the effect of $\mathit{\omega }$ on the posterior mean of $h(\mathbf{\Theta })$, we consider the following Cook posterior mean distance:

$$C{M}_h(\mathit{\omega })={\left({\mathit{M}}_h(\mathit{\omega })-{\mathit{M}}_h({\mathit{\omega }}^0)\right)}^{\top }{\mathit{G}}_h\left({\mathit{M}}_h(\mathit{\omega })-{\mathit{M}}_h({\mathit{\omega }}^0)\right),$$

where ${\mathit{G}}_h={(\text{Cov}(h(\mathbf{\Theta })|\mathbf{Y},\mathbf{X},\mathbf{Z}))}^{-1}.$ Let $f(\mathit{\omega })=C{M}_h(\mathit{\omega })$. Then $f({\mathit{\omega }}^0)=0,\ddot{f}({\mathit{\omega }}^0)={\dot{\mathit{M}}}_h^{\top }{\mathit{G}}_h{\dot{\mathit{M}}}_h,$ where ${\dot{\mathit{M}}}_h=\text{Cov}(h(\mathbf{\Theta }),{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega })|\mathbf{Y},\mathbf{X},\mathbf{Z}){|}_{\mathit{\omega }={\mathit{\omega }}^0}.$ This quantity can be approximated by

$$\begin{array}{cc}\hfill {\dot{\mathit{M}}}_h& \approx \frac{1}{S}\sum _{s=1}^S\left[h({\mathbf{\Theta }}^{(s)}){\partial }_{\mathit{\omega }}\log p(\mathbf{Y},{\mathbf{\Theta }}^{(s)}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)\right]\hfill \\ \hfill & -\left(\frac{1}{S}\sum _{s=1}^{S}h({\mathbf{\Theta }}^{(s)})\right)\left(\frac{1}{S}\sum _{s=1}^S{\partial }_{\mathit{\omega }}\log p(\mathbf{Y},{\mathbf{\Theta }}^{(s)}|\mathbf{X},\mathbf{Z},{\mathit{\omega }}^0)\right),\hfill \end{array}$$ (19)

where $\{{\mathbf{\Theta }}^{(s)}:s=1,\dots ,S\}$ are sampled from the variational approximation $q(\mathbf{\Theta })$.

4.3. Score Test for Perturbations

To test the $j$th perturbation ${\omega }_j$, we consider the hypothesis testing $H_0:{\omega }_j={\omega }_j^0\leftrightarrow H_1:{\omega }_j\ne {\omega }_j^0$. Let $\ell (\mathit{\omega })=\log p(\mathbf{Y},\mathbf{\Theta }|\mathbf{X},\mathbf{Z},\mathit{\omega })$ denote the log‐likelihood function, and ${\widehat{\mathit{\omega }}}_{-j}=\text{argmax}\ell (\mathit{\omega })$ represent the maximum likelihood estimate of $\omega$ under the null hypothesis $H_0$.

The score statistic can be expressed as

$${\text{score}}_j={\nabla }_{\omega }\ell {(\mathit{\omega })}^{\top }I{(\mathit{\omega })}^{-1}{\nabla }_{\omega }\ell (\mathit{\omega }){|}_{\mathit{\omega }={\widehat{\mathit{\omega }}}_{-j}}\stackrel{H_0}{\to }{\chi }_1^2,$$

where $I(\mathit{\omega })=-\mathrm{E}({\nabla }_{\omega }^2\ell (\mathit{\omega }))=G(\mathit{\omega })$. By performing the transformation $\tilde{\mathit{\omega }}={\mathit{\omega }}^0+G{({\mathit{\omega }}^0)}^{\frac{1}{2}}(\mathit{\omega }-{\mathit{\omega }}^0)$, we have $G(\tilde{\mathit{\omega }})={\mathit{I}}_n$ at ${\mathit{\omega }}^0$. In this case, the hypothesis testing becomes $H_0:{\tilde{\omega }}_j={\omega }_j^0\leftrightarrow H_1:{\tilde{\omega }}_j\ne {\omega }_j^0,$ Under $H_0$, the $j$‐th diagonal component of $G(\tilde{\mathit{\omega }})$ is 1. Since ${\widehat{\mathit{\omega }}}_{-j}=\text{argmax}\ell (\mathit{\omega })$ is the maximum likelihood estimate, all components of ${\nabla }_{\omega }\ell (\mathit{\omega })$ are 0 except for the $j$‐th component. Therefore, the score statistic can be expressed as

$${\text{score}}_j={({\nabla }_{\widetilde{{\mathit{\omega }}_j}}\ell (\tilde{\mathit{\omega }}))}^2{|}_{\tilde{\mathit{\omega }}={\widehat{\mathit{\omega }}}_{-j}},$$ (20)

where $j=1,\cdots ,m$.

5. Numerical Studies

In this section, we conduct simulation studies to evaluate the finite‐sample performance of the proposed method under different scenarios. The model (2) used in these studies are outlined below

$$Y_{i,k}={\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+{\mathit{\epsilon }}_{i,k},i=1,\cdots ,n_k,k=1,\cdots ,K.$$ (21)

For the purposes of the simulation study presented below, we simplify our analysis by assuming that $n_k$ and $q_k$ are identical across all data sources. To enhance clarity in our discussion, we will denote these parameters simply as $n$ and $q$. We consider the following simulation settings. All experiments are conducted in R. We implement the proposed Low‐rank Variational Correction algorithms in a new R package LRQVB, which is available on CRAN.

Simulation 1. The homogeneous variables ${\mathbf{X}}_{i,k}$ are drawn from multivariate normal distributions $N_p(0,{\mathbf{\sum }}_p)$, where the covariance matrix ${\mathbf{\sum }}_p$ is defined as ${\sum }_{p,ij}=0.5^{|i-j|}$. The heterogeneous variables ${\mathbf{Z}}_{i,k}$ are sampled from multivariate normal distributions $N_q(0,{\mathbf{\sum }}_q)$, with ${\mathbf{\sum }}_q$ representing the covariance structure for the heterogeneous data. The homogeneous parameters are given by

$$\mathit{\theta }={\left((-1,2,1,-2,1),{\mathbf{0}}_{p-5}\right)}^{\top },$$

where ${\mathbf{0}}_{p-5}$ denotes a zero vector of length $p-5$. The heterogeneous parameters ${\mathit{\beta }}_k$ are defined as

$${\mathit{\beta }}_k=\log (k){\mathbf{1}}_q,$$

where $k=1,\dots ,K$, and ${\mathbf{1}}_q$ is a vector of ones of length $q$. The remaining parameters are set as follows $n=100$, $q\in \{5,100\}$ and $K=5$.

The results presented below are based on dimension $p\in \{200,500,1000,2000\}$ and in both homogeneous and heterogeneous noise models. The model under homogeneous noise is specified in Equation (21), where the error terms ${\mathit{\epsilon }}_{i,k}$ are assumed to be i.i.d. across all sources $k$ with a common variance. We consider two types of heterogeneous noise models. The first heterogeneous noise model has the same form across all data sources

$$Y_{i,k}={\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+{\mathit{\epsilon }}_{i,k}(1+X_{i2,k}),i=1,\cdots ,n_k,k=1,\cdots ,K,$$ (22)

and we let $X_{i1,k}=X_{i1,k}^2+0.5$ for $1\le k\le K$. The second heterogeneous model takes a different form within each data source

$$Y_{i,k}={\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+{\mathit{\epsilon }}_{i,k}(1+X_{ip,k}),i=1,\cdots ,n_k,k=1,\cdots ,K,$$ (23)

and we let $X_{ik,k}=X_{ik,k}^2+0.5$ for $1\le k\le K$. Meanwhile, five probability distributions are considered for random error ${\epsilon }_{i,k}$ in each model

• error1: Gaussian $N(0,1)$,

• error2: Skewed bimodal $0.75N(-0.43,1)+0.25N(1.07,\frac{1}{3^2})$,

• error3: Trimodal $0.45N(-1.2,0.6^2)+0.45N(1.2,0.6^2)+0.1N(0,0.25^2)$,

• error4: Laplace $\text{Laplace}(1)$,

• error5: Student‐$t$ $t(2)$.

For each setting, comparisons were conducted under three quantile levels (30%, 50%, and 70%). These settings encompass a range of scenarios, including variations in variable dimensions, model characteristics, and probability distributions, enabling a comprehensive assessment of the robustness and effectiveness of our approach. Each setting is designed to replicate realistic data conditions, ensuring that our findings are both relevant and applicable to real‐world applications.

The results presented below are based on 200 replications, performed on an Intel Xeon Platinum 9242@2.3 GHz with 96 cores and 384 GB of RAM. To implement the variational Bayes approach outlined (where VB_SSL represents the variational estimation and LR_SSL represents the low‐rank correction), we specified the following hyperparameters: ${\mathbf{\sum }}_{0,k}={\mathit{I}}_q,{\tau }_1^2=1,{\tau }_0^2=1/n,r_0=4,s_0=1,a_0=1,b_0=1$.

Table 1 presents simulation results for estimating the multi‐source heterogeneous quantile linear model under the homogeneous noise setting with five different error distributions, using the parameter settings $\tau =0.5$ and $p=200,q=5$. The performance metrics include true positives rate (TPR), false positives rate (FPR), ${\ell }_2$‐error, ${\ell }_1$‐error, computation time in minutes (Time/mins), and mean squared error (MSE). For comparison, we consider several competing methods: the variational Bayes approach with a $\text{Horseshoe}+$ prior (denoted VB_HS), the expectation‐maximization (EM) algorithm [35], quantile LASSO, and quantile adaptive LASSO (using the R package $\mathtt{RQPEN}$). The results show that key variables are successfully selected and the estimated regression coefficients closely match the true values across all methods. Notably, our proposed variational Bayesian low‐rank correction method achieves the highest estimation accuracy, particularly when the data deviate from normality, and requires the shortest execution time among the Bayesian approaches, highlighting its efficiency in modeling high‐dimensional, multi‐source heterogeneous quantile data.

TABLE 1.

Comparison of the six methods under homogeneous noise model, with $\tau =0.5$ and $p=200$, $q=5$.

Quantile	Error	Model	TPR	FPR	${\ell }_1$‐loss	${\ell }_2$‐loss	Time/min
0.5	error1	VB_SSL	1.000	0.000	0.260	0.021	0.029
		LR_SSL	1.000	0.000	0.217	0.015	0.033
		VB_HS	1.000	0.036	0.861	0.080	0.344
		EM	1.000	0.021	0.583	0.051	0.176
		LASSO	1.000	0.245	2.195	0.209	0.019
		ALASSO	1.000	0.025	0.585	0.058	0.148
	error2	VB_SSL	1.000	0.000	0.202	0.013	0.030
		LR_SSL	1.000	0.000	0.167	0.009	0.035
		VB_HS	1.000	0.037	0.656	0.045	0.345
		EM	1.000	0.014	0.357	0.023	0.176
		LASSO	1.000	0.258	1.962	0.158	0.019
		ALASSO	1.000	0.022	0.463	0.039	0.151
	error3	VB_SSL	1.000	0.000	0.065	0.001	0.030
		LR_SSL	1.000	0.000	0.053	0.001	0.035
		VB_HS	1.000	0.029	0.174	0.004	0.344
		EM	1.000	0.001	0.051	0.001	0.175
		LASSO	1.000	0.333	1.576	0.081	0.019
		ALASSO	1.000	0.022	0.280	0.014	0.159
	error4	VB_SSL	1.000	0.000	0.297	0.028	0.027
		LR_SSL	1.000	0.000	0.237	0.017	0.031
		VB_HS	1.000	0.017	0.542	0.049	0.341
		EM	1.000	0.023	0.696	0.069	0.173
		LASSO	1.000	0.230	2.274	0.242	0.019
		ALASSO	1.000	0.027	0.691	0.075	0.144
	error5	VB_SSL	0.996	0.000	0.352	0.055	0.025
		LR_SSL	0.996	0.000	0.295	0.044	0.029
		VB_HS	1.000	0.009	0.478	0.050	0.355
		EM	0.998	0.028	1.000	0.138	0.172
		LASSO	1.000	0.256	2.717	0.311	0.019
		ALASSO	1.000	0.030	0.830	0.103	0.140

Table 2 reports additional simulation results under the first heterogeneous noise model, using the same parameter settings, $\tau =0.5$, and $p=200,q=5$ and five error distributions. Consistent with the homogeneous noise case, our method performs robustly across all scenarios. Due to space constraints, only partial results for the two noise models are shown in Tables 1 and 2; the full set of results is provided in the Supporting Information: Section (Tables D1–D30), which support the same conclusions.

TABLE 2.

Comparison of the six methods under the first heterogeneous noise model, with $\tau =0.5$ and $p=200$, $q=5$.

Quantile	Error	Model	TPR	FPR	${\ell }_1$‐loss	${\ell }_2$‐loss	Time/min
0.5	error1	VB_SSL	1.000	0.000	0.236	0.019	0.030
		LR_SSL	1.000	0.000	0.190	0.012	0.034
		VB_HS	1.000	0.009	0.296	0.020	0.326
		EM	1.000	0.017	0.501	0.043	0.169
		LASSO	1.000	0.242	2.191	0.213	0.018
		ALASSO	1.000	0.027	0.598	0.059	0.153
	error2	VB_SSL	1.000	0.000	0.209	0.015	0.026
		LR_SSL	1.000	0.000	0.172	0.010	0.030
		VB_HS	1.000	0.009	0.248	0.014	0.322
		EM	1.000	0.011	0.333	0.023	0.168
		LASSO	1.000	0.275	2.101	0.183	0.018
		ALASSO	1.000	0.024	0.513	0.047	0.154
	error3	VB_SSL	1.000	0.000	0.061	0.001	0.026
		LR_SSL	1.000	0.000	0.046	0.001	0.030
		VB_HS	1.000	0.006	0.062	0.001	0.322
		EM	1.000	0.000	0.044	0.001	0.167
		LASSO	1.000	0.309	1.546	0.083	0.018
		ALASSO	1.000	0.021	0.270	0.013	0.162
	error4	VB_SSL	1.000	0.000	0.250	0.022	0.027
		LR_SSL	1.000	0.000	0.200	0.014	0.030
		VB_HS	1.000	0.003	0.211	0.013	0.324
		EM	1.000	0.021	0.647	0.064	0.168
		LASSO	1.000	0.237	2.273	0.236	0.018
		ALASSO	1.000	0.028	0.655	0.069	0.149
	error5	VB_SSL	0.995	0.000	0.308	0.053	0.026
		LR_SSL	0.995	0.000	0.260	0.044	0.030
		VB_HS	1.000	0.001	0.320	0.046	0.324
		EM	0.980	0.020	0.917	0.240	0.169
		LASSO	1.000	0.244	2.544	0.282	0.018
		ALASSO	1.000	0.035	0.834	0.094	0.143

The simulation results presented in Tables 3 and 4 are based on the same parameter settings ($\tau =0.5$, $p=200$, and $q=100$), with the first table reporting the performance for homogeneous variables and the second for heterogeneous variables. In the high‐dimensional setting, both tables highlight the challenge of simultaneously performing variable selection for both homogeneous and heterogeneous variables. VB_SSL and LR_SSL demonstrate strong performance across both types of variables, achieving high TPR and low FPR while maintaining efficient computation times. These methods are particularly effective in high‐dimensional settings, where the complexity of selecting relevant variables increases.

TABLE 3.

Homogeneous variables comparison of the six methods under homogeneous noise model, with $\tau =0.5$ and $p=200$, $q=100$.

Quantile	Error	Model	TPR	FPR	${\ell }_1$‐loss	${\ell }_2$‐loss	Time/min
0.5	error1	VB_SSL	0.990	0.000	0.301	0.070	0.050
		LR_SSL	0.990	0.000	0.270	0.064	0.103
		VB_HS	1.000	0.046	1.048	0.101	0.492
		EM	1.000	0.013	1.467	0.417	0.780
		LASSO	1.000	0.173	1.391	0.137	0.066
		ALASSO	1.000	0.012	0.438	0.042	0.393
	error2	VB_SSL	1.000	0.000	0.199	0.012	0.053
		LR_SSL	1.000	0.000	0.175	0.009	0.111
		VB_HS	1.000	0.043	0.759	0.056	0.473
		EM	1.000	0.017	1.554	0.423	0.777
		LASSO	1.000	0.217	1.296	0.098	0.063
		ALASSO	1.000	0.011	0.354	0.029	0.401
	error3	VB_SSL	1.000	0.000	0.072	0.002	0.061
		LR_SSL	1.000	0.000	0.057	0.001	0.124
		VB_HS	1.000	0.019	0.145	0.003	0.500
		EM	1.000	0.015	1.394	0.358	0.777
		LASSO	1.000	0.227	0.824	0.037	0.059
		ALASSO	1.000	0.012	0.195	0.008	0.485
	error4	VB_SSL	0.989	0.000	0.353	0.084	0.058
		LR_SSL	0.989	0.000	0.304	0.066	0.109
		VB_HS	1.000	0.021	0.663	0.067	0.510
		EM	1.000	0.016	1.558	0.446	0.773
		LASSO	1.000	0.194	1.697	0.182	0.067
		ALASSO	1.000	0.013	0.481	0.050	0.388
	error5	VB_SSL	0.984	0.000	0.438	0.122	0.065
		LR_SSL	0.984	0.000	0.282	0.081	0.121
		VB_HS	1.000	0.011	0.533	0.057	0.512
		EM	0.990	0.016	2.023	0.721	0.763
		LASSO	1.000	0.203	1.897	0.209	0.067
		ALASSO	1.000	0.016	0.571	0.064	0.379

TABLE 4.

Heterogeneous variables comparison of the six methods under homogeneous noise model, with $\tau =0.5$ and $p=200$, $q=100$.

Quantile	Error	Model	TPR	FPR	${\ell }_1$‐loss	${\ell }_2$‐loss	Time/min
0.5	error1	VB_SSL	0.989	0.000	0.557	0.117	0.050
		LR_SSL	0.989	0.000	0.494	0.095	0.103
		VB_HS	1.000	0.027	1.361	0.273	0.492
		EM	0.923	0.234	26.733	21.355	0.780
		LASSO	0.996	0.246	2.596	0.458	0.066
		ALASSO	0.000	0.000	6.579	9.410	0.393
	error2	VB_SSL	0.994	0.000	0.433	0.070	0.053
		LR_SSL	0.994	0.000	0.365	0.053	0.111
		VB_HS	1.000	0.028	1.033	0.152	0.473
		EM	0.924	0.236	25.972	20.117	0.777
		LASSO	1.000	0.258	2.047	0.269	0.063
		ALASSO	0.000	0.000	6.579	9.410	0.401
	error3	VB_SSL	1.000	0.000	0.183	0.011	0.061
		LR_SSL	1.000	0.000	0.118	0.004	0.124
		VB_HS	1.000	0.018	0.264	0.011	0.500
		EM	0.939	0.234	24.382	17.694	0.777
		LASSO	1.000	0.256	0.827	0.043	0.059
		ALASSO	0.000	0.000	6.579	9.410	0.485
	error4	VB_SSL	0.986	0.000	0.687	0.171	0.058
		LR_SSL	0.986	0.000	0.609	0.140	0.109
		VB_HS	1.000	0.011	0.980	0.203	0.510
		EM	0.915	0.239	29.397	25.037	0.773
		LASSO	1.000	0.255	2.940	0.565	0.067
		ALASSO	0.000	0.000	6.579	9.410	0.388
	error5	VB_SSL	0.973	0.001	0.915	0.309	0.065
		LR_SSL	0.973	0.001	0.833	0.265	0.121
		VB_HS	0.988	0.005	0.954	0.269	0.512
		EM	0.894	0.237	35.563	40.295	0.763
		LASSO	0.999	0.283	3.611	0.789	0.067
		ALASSO	0.000	0.000	6.579	9.410	0.379

For heterogeneous variables, the methods face additional challenges due to the increased model complexity. Despite this, VB_SSL and LR_SSL continue to perform well, exhibiting only slightly higher variability compared to homogeneous variables. In contrast, VB_HS shows more false positives and errors, especially in the presence of heterogeneity, and LASSO and ALASSO struggle with variable selection accuracy under these conditions. The complete set of results is provided in the Supporting Information: Section (Tables D31–D48), and the conclusions remain consistent with those observed here. Overall, these results emphasize the difficulty of high‐dimensional variable selection, where both homogeneous and heterogeneous variables must be selected simultaneously. The variational Bayesian methods, particularly VB_SSL and LR_SSL, provide reliable and efficient solutions for this task.

Figure 2 presents the ${\ell }_2$‐loss of the estimated parameters across varying dimensions $p=(100,200,500,1000,2000)$, data source sizes $K=(10,20,30,40,50,100)$, and sample sizes $n=(100,200,300,400,500,1000)$ when the error type is error1 and $\tau =0.5$ under the homogeneous noise model. For these experiments, the default parameter values are fixed at $p=500$, $K=10$, and $n=100$. The results indicate that incorporating low‐rank correction (LR_SSL) significantly reduces the ${\ell }_2$‐loss in all settings compared to VB_SSL, especially when the dimensionality $p$ is large or the number of mixture components $K$ is small. This demonstrates that the low‐rank correction effectively stabilizes the posterior approximation and enhances estimation accuracy. Figure 3 reports the corresponding computational time under the same experimental conditions. The runtime of LR_SSL is slightly higher than that of VB_SSL as $p$, $K$, or $n$ increase, reflecting the additional cost associated with the low‐rank adjustment. However, the growth in computational time remains approximately linear, and the additional overhead is modest relative to the improvements in estimation accuracy. Overall, these results confirm that the proposed low‐rank correction strikes a favorable balance between accuracy and efficiency. For completeness, additional results under different error types and quantile levels are provided in Supporting Information D (see Figures D1–D18), with the findings remaining qualitatively similar.

FIGURE 2.

${\ell }_2$ error comparison with and without low‐rank correction for varying $p$, $K$, and $n$.

FIGURE 3.

Time comparison with and without low‐rank correction for varying $p$, $K$, and $n$.

Table 5 presents the coverage probabilities of six methods under the homogeneous noise setting with $n=100$, $K=5$, $p=200$, and $q=5$. Both the proposed VB_SSL and LR_SSL methods demonstrate high and stable coverage probabilities across all error measures and quantile levels, with values typically ranging from 0.935 to 0.965. In contrast, the competing methods exhibit notable shortcomings: VB_HS and EM show significant variability across different settings, LASSO consistently underestimates coverage, and ALASSO produces excessively wide intervals with inflated coverage. All frequentist confidence intervals were constructed using 100 bootstrap replications [68]. Collectively, these results highlight the superior reliability of VB_SSL and LR_SSL for uncertainty quantification in this context. Additional results under other noise model are reported in Supporting Information D (see Tables D49 and D50).

TABLE 5.

A coverage probability comparison of the six methods under the homogeneous noise model, where $n=100$, $K=5$, $p=200$, and $q=5$.

Quantile	Error	VB_SSL	LR_SSL	VB_HS	EM	LASSO	ALASSO
0.3	error1	0.947	0.949	0.935	0.920	0.944	0.988
	error2	0.943	0.947	0.941	0.932	0.942	0.988
	error3	0.943	0.945	0.972	0.957	0.935	0.989
	error4	0.937	0.941	0.929	0.931	0.922	0.981
	error5	0.935	0.940	0.926	0.933	0.924	0.980
0.5	error1	0.965	0.958	0.959	0.942	0.942	0.987
	error2	0.953	0.955	0.957	0.943	0.936	0.988
	error3	0.937	0.941	0.980	0.965	0.932	0.988
	error4	0.939	0.943	0.964	0.961	0.938	0.981
	error5	0.936	0.942	0.965	0.971	0.922	0.988
0.7	error1	0.952	0.954	0.931	0.916	0.947	0.986
	error2	0.945	0.948	0.936	0.948	0.948	0.986
	error3	0.936	0.938	0.969	0.954	0.941	0.987
	error4	0.939	0.944	0.961	0.959	0.939	0.988
	error5	0.938	0.943	0.967	0.962	0.936	0.987

Simulation 2. To illustrate the Bayesian local influence measures introduced above for detecting impactful observations and incorrectly specified models, we consider the heterogeneous model (23) with errors following a Trimodal distribution. For clarity, we focus on $K=2$ and $n=100$, resulting in a total of 200 data points. In the accompanying image, data points 1 to 100 correspond to the 100 data entries from the first data source, while data points 101 to 200 represent the 100 data entries from the second data source. The remaining parameters are set as follows: $\mathit{\theta }=\left((-1,2,1,-2,1),{\mathbf{0}}_{p-5}\right)$, ${\mathit{\beta }}_k=\log (k){\mathbf{1}}_q$, $p=500$, and $q=5$.

This simulation is divided into 4 cases to measure the performance of the proposed method in detecting disturbances, based on perturbation schemes (16). We perform cases at quantiles $\tau =0.3,0.5,0.7$ and consider a Bayesian local influence analysis using the Bayes factor (17), $\varphi -$divergence (18), posterior mean distance (19), and score test (20), respectively. Specifically, the significance level $\alpha =0.05$ of the score test.

• Case1: Change the data $y_{25,1}=y_{25,1}+5,y_{75,1}=y_{75,1}+5,y_{50,2}=y_{50,2}+5$ to obtain the corresponding perturbation referring to [49].

• Case2: Change the data $y_{25,1}=y_{25,1}+5,y_{75,1}=y_{75,1}-5,y_{50,2}=y_{50,2}+5$ to test the identification of positive and negative outliers by different quantiles.

• Case3: Change the model $Y_{i,k}={\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+0.2+{\mathit{\epsilon }}_{i,k}(1+X_{ip,k})$ as change the data $y_{25,1}=y_{25,1}+5,y_{75,1}=y_{75,1}+5,y_{50,2}=y_{50,2}+5$.

• Case4: Change only the data and model of the first data source based on Case3.

Due to space constraints, we are only presenting the results of Case1 for the low‐rank variational correction in Figure 4. On one hand, the low‐rank correction yields more accurate identification results, making the distinction between outliers and other points more pronounced compared to the only variational approach, particularly at the quantile level $\tau =0.3$. On the other hand, the score test we proposed is significantly more effective than the three commonly used objective functions for identifying perturbations. The remaining table results are included in Supporting Information D, in Figures D1–D7, and conclusions are similar (Table 6).

FIGURE 4.

The simulation results under Case1 based on LR_SSL, where a, b, and c represent the influence measures of three different objective functions and score statistic values at quantile $\tau =0.3,0.5$, and 0.7, respectively.

TABLE 6.

Results of common cancer‐related gene selection in four TCGA datasets, where $\tau =0.5$.

Gene	VB_SSL	LR_SSL	VB_HS	EM	LASSO	ALASSO	Studies
CREBBP	$✓$	$✓$	$✓$				$✓$
BCLAF1	$✓$	$✓$		$✓$	$✓$	$✓$	$✓$
ZCCHC8	$✓$	$✓$	$✓$	$✓$		$✓$
ARID1B	$✓$	$✓$	$✓$	$✓$			$✓$
THRAP3	$✓$	$✓$		$✓$	$✓$		$✓$
CCAR1	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$
U2AF2	$✓$	$✓$					$✓$
RHOA	$✓$	$✓$	$✓$			$✓$	$✓$
MARK2	$✓$	$✓$		$✓$	$✓$		$✓$
XPO1	$✓$	$✓$					$✓$
CNOT3	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$
POU2F2				$✓$	$✓$	$✓$	$✓$
FCRL4				$✓$		$✓$
CDH1				$✓$	$✓$	$✓$	$✓$
ATP2B3				$✓$	$✓$
MECOM					$✓$	$✓$
SAMHD1					$✓$	$✓$

6. A Real Application

In this section, we apply the proposed method to real cancer data from The Cancer Genome Atlas (TCGA), which is publicly accessible at https://cancergenome.nih.gov/. TCGA is a collaborative initiative between the National Cancer Institute and the National Human Genome Research Institute to provide high‐quality molecular profiling data across a broad range of cancer types. Specifically, we analyze four cancer types: esophageal carcinoma (TCGA‐ESCA), pancreatic adenocarcinoma (TCGA‐PAAD), pheochromocytoma and paraganglioma (TCGA‐PCPG), and rectum adenocarcinoma (TCGA‐READ), with sample sizes of 197, 179, 186, and 171, respectively. Although gene expression profiles vary substantially across cancer types because of differences in tissue origin, etiological exposures, and molecular pathways [69, 70], recurrent driver genes and core oncogenic programs are often shared across malignancies [71, 72]. Evidence from PCAWG and IntOGen further suggests that common oncogenic alterations recur across multiple cancer types despite substantial biological heterogeneity [71, 72]. Motivated by these findings, our objective is to identify genes whose expression is consistently associated with age at first diagnosis across these four cancer types, particularly those involved in shared oncogenic pathways.

Let $\tilde{Y}$ denote age at first diagnosis. In TCGA, this corresponds to the variable “time to first diagnosis,” which records the patient's age (in years) at the initial histologically confirmed diagnosis of a primary tumor. In our analysis, we apply a logarithmic transformation to $\tilde{Y}$ and define the response variable as $Y=\log (\tilde{Y})$. Gene expression levels are measured using unstranded transcripts per million (TPM). The original TCGA expression matrices contain approximately 60 660 genes. Because many of these genes are unlikely to be directly involved in tumorigenesis and may introduce substantial noise in high‐dimensional modeling, we restrict our analysis to a curated panel of 633 cancer driver genes obtained from the IntOGen resource [71]. Their TPM values are used as shared high‐dimensional covariates across all cancer types. Additionally, primary site, AJCC pathologic stage, tumor grade, tumor classification, and gender are included as low‐dimensional heterogeneous covariates to account for clinical heterogeneity. Following established integrative modeling frameworks [35, 73], these clinical variables are treated as unpenalized covariates, while penalization is applied only to the genomic features. To evaluate whether this heterogeneous specification is empirically justified, we also considered a simpler baseline model in which these five clinical variables were treated as homogeneous effects. The predictive comparison, reported in Supporting Information: Appendix E, shows that the heterogeneous model outperforms the homogeneous alternative. Accordingly, the high‐dimensional multi‐source heterogeneous quantile linear model is formulated as

$$Y_{i,k}={\mathbf{X}}_{i,k}^{\top }\mathit{\theta }+{\mathbf{Z}}_{i,k}^{\top }{\mathit{\beta }}_k+{\mathit{\epsilon }}_{i,k},k=1,\cdots ,4,i=1,\cdots ,n_k,$$

where $n_1=197,n_2=179,n_3=186,n_4=171$. This modeling framework is particularly well suited to the distributional characteristics of the data. As shown in Figure 1, the age‐at‐diagnosis variable exhibits noticeable skewness and heavy tails, violating the normality and homoscedasticity assumptions typically required for classical linear regression. Under such conditions, mean‐based estimation may yield biased or inefficient results. In contrast, quantile regression does not rely on a specific error distribution and directly estimates conditional quantiles, making it more robust to skewness, heavy tails, and heteroscedasticity [4]. Moreover, quantile regression provides additional insight into how gene effects vary across different regions of the conditional age‐at‐diagnosis distribution. For example, certain genes may exert stronger effects among early‐onset or late‐onset patients, patterns that conventional linear regression would fail to detect. For these reasons, we adopt a quantile regression framework rather than classical linear regression, as it better accommodates the observed distributional features and yields richer insight into age‐related genetic effects.

We applied VB_SSL and LR_SSL, together with four competing methods, to the real dataset and evaluated predictive performance at three quantile levels: $\tau =0.3,0.5,$ and $0.7$. The boxplots of squared residuals obtained from five‐fold cross‐validation are presented in Figure 5. Overall, both proposed methods achieve smaller squared residuals and narrower interquartile ranges than the competing approaches across all quantile levels, indicating superior predictive accuracy and greater stability. In particular, LR_SSL and VB_SSL consistently attain the lowest median prediction errors, with their advantage especially pronounced at $\tau =0.5$. By contrast, HS, EM, Lasso, and Adaptive Lasso generally produce larger residuals and greater variability, particularly at the lower and upper quantiles. These results demonstrate that the proposed methods provide more robust predictive performance than existing alternatives in the real‐data analysis.

FIGURE 5.

Squared residuals comparison of six methods under real data.

Among the genes prioritized by our method, substantial prior evidence supports their relevance to cancer development and progression. CREBBP is a recurrent tumor suppressor implicated in both hematologic malignancies and solid tumors [74]. BCLAF1 has context‐dependent oncogenic and tumor‐suppressive roles across cancer types, with evidence linking it to tumor growth, survival, and immune evasion [75]. ARID1B, a component of the SWI/SNF chromatin‐remodeling complex, has also been implicated in carcinogenesis, and combined loss of ARID1A and ARID1B has been shown to accelerate colorectal tumor development [76]. THRAP3 has been linked to cancer cell survival through its role in regulating R‐loop resolution and has also been implicated in acute myeloid leukemia [77, 78]. CCAR1 contributes to tumor cell growth and survival and has been reported to promote tumorigenesis in gastric cancer [79]. Cancer‐associated mutations in U2AF2 can alter RNA interactions, disrupt splicing, and induce gene‐expression changes consistent with neoplastic transformation [80].

Several additional genes identified by our method are similarly supported by prior studies. RHOA is a well‐recognized cancer gene with context‐dependent oncogenic and tumor‐suppressive functions, including important roles in digestive tract cancers [81]. MARK2 has been associated with oncogenic signaling and YAP/TAZ dependency in human cancers [82]. XPO1 is frequently overexpressed or mutated across multiple cancer types and is widely regarded as both an oncogenic driver and a therapeutic target [83]. CNOT3 has been linked to an aggressive colorectal cancer subtype and has recently been identified as a post‐transcriptional vulnerability in acute myeloid leukemia [84, 85]. Finally, although the supporting evidence remains comparatively limited, ZCCHC8 has also been implicated in cancer through oncogenic ZCCHC8–ROS1 fusion events reported in multiple tumor types [86, 87]. Collectively, these findings suggest that the genes selected by our method are not only statistically important but also biologically meaningful, with strong relevance to both established and emerging mechanisms of tumorigenesis.

To identify patients with particularly strong influence on model estimation, we compute Bayesian influence measures and focus on the perturbation of individual observations at the median quantile level, $\tau =0.5$. Due to space limitations, we present only the sample detection results for the TCGA‐PAAD dataset based on the LR_SSL method in the main text; the corresponding results for the other three cancer datasets are provided in Supporting Information E (Figures E1–E3). As shown in Figure 6, the proposed score‐based local influence method identifies patients 49, 67, and 94 as influential observations in the TCGA‐PAAD dataset. Compared with other local influence measures, the score‐based approach produces a more stable and parsimonious set of detected influential observations. Accordingly, patients 49, 67, and 94 are highlighted as the primary influential cases, warranting further clinical and biological investigation.

FIGURE 6.

Important patient identification in TCGA‐PAAD, where $\tau =0.5$.

Consistent with standard practice in Bayesian local influence analysis, influential observations are not automatically excluded; rather, they are prioritized for targeted follow‐up, since even small perturbations may induce substantial changes in the posterior distribution [48, 49, 50, 51]. From a clinical perspective, the influential patients identified in Figure 6 exhibit markedly atypical age‐at‐diagnosis patterns. Specifically, while the average age at diagnosis in the TCGA‐PAAD cohort is 65 years, patients 49, 67, and 94 were diagnosed at ages 20, 36, and 40 years, respectively. These substantially early‐onset cases deviate markedly from the typical disease pattern of the cohort and may reflect distinct biological mechanisms or unusual genetic architectures. As such, they represent particularly informative candidates for deeper clinical and genomic investigation.

7. Conclusion

In this paper, we proposed a fast and scalable variational Bayes framework with low‐rank correction for high‐dimensional multi‐source heterogeneous quantile linear models. By combining mean‐field variational inference with Laplace approximations, we developed an efficient Bayesian estimation procedure under spike‐and‐slab priors and introduced a low‐rank correction strategy to improve estimation accuracy. To implement the proposed approach, we designed a coordinate‐ascent algorithm and systematically evaluated its performance through extensive simulation studies and comparisons with several widely used competing methods.

Our empirical evaluation focused on variable selection accuracy, estimation precision, and computational efficiency. Both simulation experiments and real‐data analyses demonstrated that the proposed variational Bayes methods, particularly the low‐rank corrected version, consistently outperform competing approaches. The low‐rank correction was especially beneficial in heterogeneous settings, substantially improving estimation accuracy while preserving computational efficiency. These findings highlight the proposed framework as a practical, robust, and scalable alternative for high‐dimensional multi‐source modeling across a broad range of applications.

Several important directions remain for future research. First, extending the proposed framework to more complex sparse heterogeneous models, such as those involving nonlinear heterogeneous effects or more intricate multi‐source dependence structures, may further enhance its applicability to modern biomedical data; integrating flexible machine learning tools, such as neural networks, may be especially promising in this context. Second, developing user‐friendly software implementations would improve the accessibility and broader practical impact of the proposed methods. Finally, establishing stronger theoretical guarantees for the variational Bayes estimators, including consistency and asymptotic normality, would provide deeper insight into their statistical properties and further strengthen the methodological foundation of this work.

Author Contributions

Huiqiong Li and Lu Luo: contributed equally to this work. Huiqiong Li: conceptualized the problem and performed the computations. Lu Luo: wrote the R code, developed the methodology. Min Wang: proved the claim and planned the experiments and the applications. Niansheng Tang: explained the results and revised the manuscript. All authors shared in the writing of the article.

Funding

The research was partially supported by a grant from the Natural Science Foundation of China [Grant Number 12261102] and the grants from Yunnan Fundamental Research Project China [202301AS070044].

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Data S1: Supporting Information.

Data Availability Statement

The data that support the findings of this study are openly available in The Cancer Genome Atlas (TCGA) at https://cancergenome.nih.gov.