An AI-powered Bayesian Generative Modeling Approach for Causal Inference in Observational Studies

Abstract

Causal inference in observational studies with high-dimensional covariates presents significant challenges. We introduce CausalBGM, an AI-powered Bayesian generative modeling approach that captures the causal relationship among covariates, treatment, and outcome. The core innovation is to estimate the individual treatment effect (ITE) by learning the individual-specific distribution of a low-dimensional latent feature set (e.g., latent confounders) that drives changes in both treatment and outcome. This individualized posterior representation yields estimates of the individual treatment effect (ITE) together with well-calibrated posterior intervals while mitigating confounding effect. CausalBGM is fitted through an iterative algorithm to update the model parameters and the latent features until convergence. This framework leverages the power of AI to capture complex dependencies among variables while adhering to the Bayesian principles. Extensive experiments demonstrate that CausalBGM achieves superior or competitive performance compared to state-of-the-art methods, particularly in scenarios with high-dimensional covariates and large-scale datasets. By addressing key limitations of existing methods, CausalBGM emerges as a robust and promising framework for advancing causal inference in a wide range of modern applications. The code for CausalBGM is available at https://github.com/liuq-lab/bayesgm. The document for CausalBGM is available at https://bayesgm.readthedocs.io. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

1. Introduction

One central goal for causal inference in observational studies is to estimate the causal effect of one variable (e.g., treatment) on another (e.g., outcome) while accounting for covariates that represent all other measured variables (Pearl 2009; Imbens and Rubin 2015; Ding 2024). Covariates are often high-dimensional for modern applications in genomics, economics, and healthcare (Prosperi et al. 2020; Davey Smith et al. 2020), which makes the covariate adjustment difficult due to the “curse of dimensionality” (D’Amour et al. 2021). Additionally, large sample sizes, as is often the case in those scenarios, can further complicate the process by making traditional methods computationally intensive and slow to converge, highlighting the need for developing scalable and effective causal inference methods.

To handle the issue of high-dimensional covariates, several dimension reduction methods have been proposed to alleviate the difficulty. For example, one of the most popular approaches is to do adjustment or matching based on the propensity score (Rubin 1974; Rosenbaum and Rubin 1983; Hirano and Imbens 2004), which is a one-dimensional feature (e.g., a scalar), denoting the probability of receiving a particular treatment given observed covariates. These methods require fitting a propensity score model first, which is typically done by fitting a logistic regression or a machine learning model (Lee, Lessler, and Stuart 2010). Another type of dimension reduction method is sufficient dimension reduction (SDR) (Li 1991, 1992), which projects covariates into a lower-dimensional space, assuming conditional independence of treatment and outcome given the projected features (Luo, Zhu, and Ghosh 2017; Ghosh, Ma, and De Luna 2021). However, SDR-based causal inference methods often restrict dimension reduction to be linear transformations and apply separate projections for each treatment value, limiting its applicability in settings with continuous treatments or complex dependencies. The latent factor approach has also been used as surrogate confounders to adjust for biases in causal effect estimation caused by unobserved confounders (Yuan and Qu 2024).

Recently, the rapid development of AI-powered causal inference approaches have shown promising results for causal effect estimation (Berrevoets et al. 2023; Lagemann et al. 2023). These AI-based approaches typically leverage deep learning techniques and demonstrate superior power in modeling complex dependency and estimation accuracy when the sample size is large. In particular, the Causal Encoding Generative Modeling approach, CausalEGM (Liu, Chen, and Wong 2024), developed by our group, combines auto-encoding and generative modeling to enable nonlinear, structured dimension reduction in causal inference. CausalEGM stands at the intersection of AI and causal inference and has been shown to provide superior performance for developing deep learning-based estimates for the structural equation modeling that describes the causal relations among variables.

Despite its strong empirical performance, there are two key limitations of the CausalEGM architecture from a Bayesian perspective. First, the joint use of an encoder and a generative decoder introduces a structural loop (dotted arrow in Figure 1(B)). Such circularity violates the acyclicity assumption that is fundamental to Bayesian networks and causal diagrams. Without carefully ensuring a proper directed acyclic graph (DAG) structure, the learned model may struggle to reflect genuine causal relationships. Second, similar to existing AI-based methods that primarily focus on point estimate, CausalEGM relies on deterministic functions to establish the mapping between observed data and latent features. Deterministic mappings can limit the model’s ability to capture and quantify uncertainty, thereby undermining the statistical rigor of the approach and making it challenging to draw reliable causal conclusions in many applications where uncertainty plays a critical role. Probabilistic modeling, in contrast, provides well-defined uncertainty quantification and more robust inference, ensuring that the predictive distributions of the causal effect estimates reflect the true underlying uncertainty in the causal mechanism.

Figure 1.

Illustration of CausalBGM framework. (A) The typical causal diagram in the observational study where the treatment, outcome, and covariates are observed variables. (B) An overview of CausalBGM model where variables are in rectangles and functions are in circles with incoming arrows indicating inputs to the function and outgoing arrows indicating outputs. $G,H$, and $F$ represent generative models for covariates, treatment, and outcome variables, respectively. $E$ represents the encoding function that creates circularity and is used for initialization purpose only. $E$ is removed in CausalBGM during the model training.

To address the above issues, we introduce CausalBGM, an AI-powered Bayesian Generative Modeling (BGM) framework for estimating causal effects in the presence of high-dimensional covariates. Compared to CausalEGM, the new CausalBGM removes the encoder function entirely and employs a fully Bayesian procedure to infer latent features (Figure 1(B) without the dotted arrow). By eliminating the encoder-decoder loop, CausalBGM guarantees a clear DAG structure that is consistent with statistical modeling principles. Both the latent variables and model parameters are drawn from probabilistic distributions rather than being deterministically encoded, allowing for the incorporation of prior information and the generation of posterior distributions that more accurately represent uncertainty. By leveraging this fully Bayesian methodology, CausalBGM achieves substantial improvements, providing a principled alignment with Bayesian causal inference (Li, Ding, and Mealli 2023). The model eliminates problematic cycles, adopts Bayesian inference, and ultimately provides a more robust and interpretable framework for estimating causal effects in complex, high-dimensional data settings. We highlight several key innovations of CausalBGM as follows.

First, traditional iterative sampling methods (e.g., Gibbs sampling) typically require evaluating conditional distributions that depend on the full dataset at each iteration, which is computationally intensive and often impractical for large-scale datasets. In contrast, CausalBGM introduces a novel iterative algorithm that computes the likelihood only on the current sample or a mini-batch of samples in each iteration step, significantly enhancing scalability. Sampling low-dimensional latent features for each individual is fully decoupled, enabling efficient parallelization and further improving computational efficiency.

Second, as shown in Section 3.7, iterative sampling of latent features and model parameters can often exhibit suboptimal convergence and performance. To address this, we propose to initiate the updates using estimates from the generative functions obtained by the CausalEGM method, which has strong empirical performance and proven theoretical properties. This strategy ensures a strong starting point for the model, facilitating more stable and accurate iterative updates. Experimental results consistently demonstrate that the EGM initialization significantly enhances predictive accuracy and stability across diverse datasets, underscoring its critical role in achieving superior performance.

Third, instead of directly updating model parameters as deterministic values as standard practice in AI, CausalBGM treats them as random variables and iteratively updates their posterior distributions to account for model uncertainty or variation. Besides, while many existing AI-driven causal inference methods, including CausalEGM, focus solely on modeling the mean function, CausalBGM simultaneously models both the mean and variance functions of observed variables. By incorporating variance modeling, CausalBGM captures a more comprehensive representation of data variability, allowing for the construction of well-calibrated posterior intervals for causal effect estimates.

These innovations uniquely position CausalBGM as a scalable, statistically rigorous, and interpretable framework, bridging the gap between AI and Bayesian causal inference. By addressing key limitations of existing methods, CausalBGM achieves superior performance across a wide range of scenarios, offering a versatile and robust solution for tackling complex causal inference challenges in modern applications.

2. Methods

2.1. Problem Setup

Our goal is to estimate the causal effect of one variable $X$ on another variable $Y$ given the presence of the variable $V$ in an observational study based on iid observations of $\left\{\left(X_i,Y_i,V_i\right)\mid i=\right.1,\dots ,N\}.X$ denotes the treatment or exposure variable and $Y$ denotes the outcome or response variable. $V\in {\mathbb{R}}^p$ represents the covariates in a $p$-dimensional space. $Y\in 𝒴$ is typically real-valued where the support $𝒴$ is a bounded interval in $\mathbb{R}.X\in 𝒳$ can be either discrete or continuous where the support $𝒳$ is either a finite set or a bounded interval in $\mathbb{R}$.

In order to investigate how the potential outcome will respond to the change of treatment, our primary interest is in estimating the population average of this outcome function, also known as the average dose–response function (ADRF), defined by:

$$\mu (x)=\mathbb{E}[Y(x)].$$ (1)

Since we only observe the potential outcomes indexed by the treatment variable (e.g., factual outcome), the random variable $Y(x)$ is not directly observable due to the counterfactual outcomes, and the expectation $\mu (x)$ cannot generally be directly identified from the joint distribution of the observed data $(X,Y,V)$. Therefore, additional assumptions are required to ensure the identifiability of $\mu (x)$.

We first assume $X,Y$, and $V$ are generated by a latent variable $Z\in {\mathbb{R}}^q$ where $q\ll p$. We denote $Z_0$ as a subset of the latent variable $Z$, which affects both treatment and outcome. Next, we introduce a modified version of the “unconfoundedness” assumption with respect to the latent confounding variable $Z_0$.

Assumption 1 (Unconfoundedness). Given the low-dimensional latent confounding variable $Z_0$, the potential outcome $Y(x)$ is independent of treatment variable $X$,

$$X⫫Y(x)\mid Z_0.$$ (2)

Under the traditional “unconfoundedness” assumption, one typically conditions on the high-dimensional covariates $V$. However, our Assumption 1 makes this requirement less restrictive by showing that it is sufficient to condition on a low-dimensional feature set representing the covariates. Once $Z_0$ is given, there should be no unobserved confounding variables that induce correlated changes between the treatment and the outcome.

Based on Assumption 1, it follows that the ADRF can be identified through the following equation,

$$\mu \left(x\right)=\int \mathbb{E}\left[Y\mid X=x,Z_0=z_0\right]p_{Z_0}\left(z_0\right)d{z}_0.$$ (3)

The identification proof is given in Appendix A. Equation (3) transforms the causal inference problem into the problem of learning a latent confounding variable $Z_0$ given the observational data. In the following section, we will outline an AI-powered Bayesian framework in order to learn $Z_0$ and estimate the $\mu (x)$ in (3).

2.2. Causal Generative Modeling

Our model is described in Figure 1, where $X,Y,V$ represent observed variables and $Z=\left(Z_0,Z_1,Z_2,Z_3\right)$ denotes the low-dimensional latent variable that needs to be learned. The whole latent space is partitioned into four parts that play different roles in the following generative models of $X,Y$, and $V$.

$$\{\begin{array}{l}Z\sim {\pi }_Z\left(Z\right),\hfill \\ {\theta }_X\sim {\pi }_{{\theta }_X}\left({\theta }_X\right),{\theta }_Y\sim {\pi }_{{\theta }_Y}\left({\theta }_Y\right),{\theta }_V\sim {\pi }_{{\theta }_V}\left({\theta }_V\right),\hfill \\ V\sim P\left(V\mid Z;{\theta }_V\right),\hfill \\ X\sim P\left(X\mid Z_0,Z_2;{\theta }_X\right),\hfill \\ Y\sim P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right),\hfill \end{array}$$ (4)

where $Z_0$ denotes the latent confounding variable that affects both treatment and outcome, $Z_1$ represents the latent features that affect only the outcome, $Z_2$ relates to the latent features that affect only the treatment, and $Z_3$ comprises the remaining latent features that affect neither treatment nor outcome. By partitioning the latent features $Z$ into four different independent components, CausalBGM is able to isolate the underlying dependencies of covariates on treatment and outcome in the low-dimensional latent space. Through the above partition, we aim to identify a minimal covariate feature set (e.g., $Z_0$) that affects both treatment and outcome. ${\theta }_X,{\theta }_Y$, and ${\theta }_V$ are the parameters of the three generative models of treatment, outcome, and covariates, respectively. All the prior distributions are set to be standard multivariate normal distributions.

The three generative models can be flexibly parameterized by any parametric family, such as the exponential family (see Appendix B). By default, we model the conditional distribution as normal distributions for continuous variables and logistic regression for discrete variables. In typical causal inference settings, the generative processes are defined as follows:

• Covariate Modeling. The covariate variable $V$ is modeled as a multivariate normal distribution as follows,

  $$P\left(V\mid Z;{\theta }_V\right)=𝒩\left({\mu }_v(Z),{\Sigma }_v(Z)\right),$$ (5)

  where both mean and covariance matrix are learnable functions of latent variable $Z$ parameterized by ${\theta }_V$. To simplify, the covariance matrix ${\Sigma }_v(Z)$ is represented as ${\sigma }_v^2(Z)I_p$ where $I_p$ is the $p$-dimensional identity matrix and ${\sigma }_v^2(Z)$ is a learnable variance function.
• Treatment Modeling. For continuous treatments, the treatment variable $X$ is modeled as

  $$P\left(X\mid Z_0,Z_2;{\theta }_X\right)=𝒩\left({\mu }_x\left(Z_0,Z_2\right),{\sigma }_x^2\left(Z_0,Z_2\right)\right),$$ (6)

  where both mean ${\mu }_x\left(Z_0,Z_2\right)$ and variance ${\sigma }_x^2\left(Z_0,Z_2\right)$ are learnable functions of $Z_0$ and $Z_2$ parameterized by ${\theta }_X$.
  For binary treatments, $X$ is modeled using a generalized logistic regression:

  $$P\left(X=1\mid Z_0,Z_2;{\theta }_X\right)=1/\left(1+e^{-\xi }\right),$$ (7)

  where $\xi ~𝒩\left({\mu }_x\left(Z_0,Z_2\right),{\sigma }_x^2\left(Z_0,Z_2\right)\right)$, and the resulting probability is equivalent to the propensity score.
• Outcome Modeling. The outcome variable $Y$ is modeled as a normal distribution:

  $$P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right)=𝒩\left({\mu }_y\left(X,Z_0,Z_1\right),{\sigma }_y^2\left(X,Z_0,Z_1\right)\right),$$ (8)

  where both mean ${\mu }_y\left(X,Z_0,Z_1\right)$ and variance ${\sigma }_y^2\left(X,Z_0,Z_1\right)$ are learnable functions of $X,Z_0$, and $Z_1$, parameterized by ${\theta }_Y$.

Note that the learnable functions $\left({\mu }_x,{\sigma }_x^2\right),\left({\mu }_y,{\sigma }_y^2\right)$, and $({\mu }_v,{\sigma }_v^2)$ are represented by three Bayesian neural networks (Jospin et al. 2022), parameterized by ${\theta }_X,{\theta }_Y$, and ${\theta }_V$, respectively. In the next section, we will illustrate how we learn the distribution of model parameters ${\theta }_X,{\theta }_Y,{\theta }_V$ in order to account for the model uncertainty or variation.

2.3. Iterative Updating Algorithm

We adopted a stochastic iterative algorithm (Liu and Wong 2026) to update the posterior distribution of model parameters and the posterior distribution of latent variable $Z$ until convergence. According to Bayes’ theorem, the joint posterior distribution of the latent features and model parameters is represented as

$$\begin{gathered} P\left(Z,{\theta }_X,{\theta }_Y,{\theta }_V\mid X,Y,V\right) \\ =P\left({\theta }_X,{\theta }_Y,{\theta }_V\mid X,Y,V\right)P\left(Z\mid X,Y,V,{\theta }_X,{\theta }_Y,{\theta }_V\right). \end{gathered}$$ (9)

Since the true joint posterior is intractable, we approximate the problem by designing an iterative algorithm. Specifically, we iteratively (a) update the posterior distribution of latent variable $Z$ from $P\left(Z\mid X,Y,V,{\theta }_X,{\theta }_Y,{\theta }_V\right)$. (b) update the posterior distribution of model parameters $({\theta }_X,{\theta }_Y,{\theta }_V)$ from $P\left({\theta }_X,{\theta }_Y,{\theta }_V\mid X,Y,V,Z\right)$.

To estimate the posterior distribution of the latent variable $Z$ in step 1), we denote the log-posterior of the latent variable $Z$ as

$$\begin{gathered} \text{log}P\left(Z\mid X,Y,V,{\theta }_X,{\theta }_Y,{\theta }_V\right) \\ =\text{log}{\pi }_Z\left(Z\right)+\text{log}P\left(X,Y,V\mid Z,{\theta }_X,{\theta }_Y,{\theta }_V\right)+C, \\ =\text{log}{\pi }_Z\left(Z\right)+\text{log}P\left(V\mid Z,{\theta }_X,{\theta }_Y,{\theta }_V\right)+\text{log}P\left(X,Y\mid Z,{\theta }_X,{\theta }_Y,{\theta }_V\right)+C, \\ =\text{log}{\pi }_Z\left(Z\right)+\text{log}P\left(V\mid Z;{\theta }_V\right)+\text{log}P\left(X\mid Z_0,Z_2;{\theta }_X\right)+\text{log}P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right)+C, \end{gathered}$$ (10)

where $C=\text{log}{\pi }_{{\theta }_X}\left({\theta }_X\right)+\text{log}{\pi }_{{\theta }_Y}\left({\theta }_Y\right)+\text{log}{\pi }_{{\theta }_V}\left({\theta }_V\right)-\text{log}P\left(X,Y,V,{\theta }_X,{\theta }_Y,{\theta }_V\right)$ is irrelevant to $Z$. The second equality in (10) is obtained by the conditional independence in Assumption 1. The log-likelihood of the three generative models are denoted as

$$\left\{\begin{array}{l}\mathit{\text{log}}P\left(V\mid Z;{\theta }_V\right)=-\frac{p}{2}\mathit{\text{log}}\left({\sigma }_v^2(Z)\right)-\frac{1}{2{\sigma }_v^2(Z)}{‖V-{\mu }_v(Z)‖}_2^2+C_1,\\ \\ \mathit{\text{log}}P\left(X\mid Z_0,Z_2;{\theta }_X\right)=-\frac{1}{2}\mathit{\text{log}}\left({\sigma }_x^2\left(Z_0,Z_2\right)\right)-\frac{1}{2{\sigma }_x^2\left(Z_0,Z_2\right)}{\left(X-{\mu }_x\left(Z_0,Z_2\right)\right)}^2+C_2,\\ \\ \mathit{\text{log}}P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right)=-\frac{1}{2}\mathit{\text{log}}\left({\sigma }_y^2\left(X,Z_0,Z_1\right)\right)-\frac{1}{2{\sigma }_y^2\left(X,Z_0,Z_1\right)}{\left(Y-{\mu }_y\left(X,Z_0,Z_1\right)\right)}^2+C_3,\\ \end{array}\right.$$ (11)

where $C_1,C_2$, and $C_3$ are constants.

To update the posterior $P\left({\theta }_X,{\theta }_Y,{\theta }_V\mid X,Y,V,Z\right)$ over all model parameters ${\theta }_X,{\theta }_Y$, and ${\theta }_V$ from three generative models in step (b), we further decompose the joint posterior for the model parameters based on conditional independence, which is denoted as

$$\left\{\begin{array}{l}\mathit{\text{log}}P\left({\theta }_X\mid X,Y,V,Z\right)=\mathit{\text{log}}{\pi }_{{\theta }_X}\left({\theta }_X\right)+\mathit{\text{log}}P\left(X\mid Z_0,Z_2;{\theta }_X\right)+C_4,\\ \\ \mathit{\text{log}}P\left({\theta }_Y\mid X,Y,V,Z\right)=\mathit{\text{log}}{\pi }_{{\theta }_Y}\left({\theta }_Y\right)+\mathit{\text{log}}P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right)+C_5,\\ \\ \mathit{\text{log}}P\left({\theta }_V\mid X,Y,V,Z\right)=\mathit{\text{log}}{\pi }_{{\theta }_V}\left({\theta }_V\right)+\mathit{\text{log}}P\left(V\mid Z;{\theta }_V\right)+C_6,\\ \end{array}\right.$$ (12)

where $C_4$ is irrelevant with ${\theta }_X,C_5$ is irrelevant with ${\theta }_Y$, and $C_6$ is irrelevant with ${\theta }_V$. Since the posterior distribution of parameters in each generative model is intractable, we employ three Bayesian neural networks, which use variational inference (VI) to approximate each term in (12). Specifically, we introduce three variational distributions $q_{{\varphi }_X}\left({\theta }_X\right),q_{{\varphi }_Y}\left({\theta }_Y\right)$, and $q_{{\varphi }_V}\left({\theta }_V\right)$ to approximate the true posteriors in (12), respectively. The variational distributions are chosen to be normal distributions as $q_{{\varphi }_X}\left({\theta }_X\right)~𝒩\left({\theta }_X\mid {\mu }_{{\varphi }_X},{\sigma }_{{\varphi }_X}^2\right),q_{{\varphi }_Y}\left({\theta }_Y\right)~𝒩\left({\theta }_Y\mid {\mu }_{{\varphi }_Y},{\sigma }_{{\varphi }_Y}^2\right)$, and $q_{{\varphi }_V}\left({\theta }_V\right)~𝒩\left({\theta }_V\mid {\mu }_{{\varphi }_V},{\sigma }_{{\varphi }_V}^2{I}_p\right)$. Note that ${\varphi }_X=\left({\mu }_{{\varphi }_X},{\sigma }_{{\varphi }_X}^2\right),{\varphi }_Y=\left({\mu }_{{\varphi }_Y},{\sigma }_{{\varphi }_Y}^2\right)$, and ${\varphi }_V=\left({\mu }_{{\varphi }_V},{\sigma }_{{\varphi }_V}^2\right)$ are learnable parameters for the variational distributions (variational parameters). The evidence lower bound (ELBO) for each posterior is defined as

$$\left\{\begin{array}{l}ℒ\left({\varphi }_X\right)={\mathbb{E}}_{q_{{\varphi }_X}\left({\theta }_X\right)}\left[\mathit{\text{log}}P\left(X\mid Z_0,Z_2;{\theta }_X\right)\right]-KL\left(q_{{\varphi }_X}\left({\theta }_X\right)\Vert {\pi }_{{\theta }_X}\left({\theta }_X\right)\right),\\ \\ ℒ\left({\varphi }_Y\right)={\mathbb{E}}_{q_{{\varphi }_Y}\left({\theta }_Y\right)}\left[\mathit{\text{log}}P\left(Y\mid X,Z_0,Z_1;{\theta }_Y\right)\right]-KL\left(q_{{\varphi }_Y}\left({\theta }_Y\right)\Vert {\pi }_{{\theta }_Y}\left({\theta }_Y\right)\right),\\ \\ ℒ\left({\varphi }_V\right)={\mathbb{E}}_{q_{{\varphi }_V}\left({\theta }_V\right)}\left[\mathit{\text{log}}P\left(V\mid Z;{\theta }_V\right)\right]-KL\left(q_{{\varphi }_V}\left({\theta }_V\right)\Vert {\pi }_{{\theta }_V}\left({\theta }_V\right)\right),\\ \end{array}\right.$$ (13)

where the first term in the ELBO denotes the expected log-likelihood under the variational distribution and the second term denotes Kullback-Leibler divergence between the variational posterior and the prior distribution over model parameters. To facilitate the computation of gradient w.r.t the variational parameters, we use the reparameterization trick, which is represented as

$$\left\{\begin{array}{l}{\hat{\theta }}_X={\mu }_{{\varphi }_X}+{\sigma }_{{\varphi }_X}\odot {ϵ}_X,\\ {\hat{\theta }}_Y={\mu }_{{\varphi }_Y}+{\sigma }_{{\varphi }_Y}\odot {ϵ}_Y,\\ {\hat{\theta }}_V={\mu }_{{\varphi }_V}+{\sigma }_{{\varphi }_V}\odot {ϵ}_V,\end{array}\right.$$ (14)

where ${ϵ}_X~𝒩\left(0,I_{d_X}\right),{ϵ}_Y~𝒩\left(0,I_{d_Y}\right)$, and ${ϵ}_V~𝒩\left(0,I_{d_V}\right).\odot$ is the element-wise product. $d_X,d_Y$, and $d_V$ denote the number of parameters in each generative model. Using variational inference in BNNs for each mini-batch may lead to high-variance gradient estimates. We adopt the Flipout technique (Wen et al. 2018) in the implementation of the reparameterization trick to reduce this variance by decorrelating the model parameters perturbations across different training examples in the same mini-batch. Briefly, instead of using a single shared random draw of model parameters for the entire mini-batch, Flipout constructs pseudo-independent perturbations for each example independently within a mini-batch, which decorrelates the gradients, reduces the variance, and stabilizes the training process.

Note that we update the posterior distribution of model parameters for treatment model, covariate model, and outcome model sequentially. For each generative model, we first update the variational parameters to maximize the ELBO in (13) and then sample model parameters from (14). Given the sampled model parameters, the regular forward pass through the network (e.g., each layer contains matrix multiplication followed by nonlinear activation function) is computed to get the mean and variance parameters in (5)–(8).

Each iteration only requires the current sample or a random mini-batch of observed samples. During the iteration algorithm, we first take a derivative of (10) w.r.t the latent variable $Z$ and employ a stochastic gradient ascent to update the latent variable $Z$ for each individual given the current model parameters. Then, we take a derivative of each ELBO term in (13) w.r.t the variational parameters $({\varphi }_X,{\varphi }_Y,\text{or}{\varphi }_V)$ in the three generative models sequentially and employ a stochastic gradient ascent to update the variational parameters given the current latent variables to maximize the ELBO. During test stage, we only need to infer the posterior distribution of latent variable $Z$ given the test data. To achieve this, we first sample model parameters $({\theta }_X,{\theta }_Y,\text{and}{\theta }_V)$ from the variational distribution parameterized by ${\varphi }_X,{\varphi }_Y$, and ${\varphi }_V$ through (14). Then we use Markov chain Monte Carlo (MCMC) method (Liu 2001) to sample from the posterior distribution of latent variable in (10) for each individual. We choose the standard Metropolis–Hastings algorithm (Robert and Casella 2004) by default. Note that this individual-level sampling process is fully decoupled, enabling parallelization and improving computational efficiency. The causal effect and the corresponding posterior interval with user-specified significance level can then be estimated based on the MCMC samples of latent variable and the learned generative models.

In the binary treatment setting, the individual treatment effect (ITE) for the $i$th unit is estimated as

$${\hat{\Delta }}_i=\frac{1}{S}\sum _{s=1}^S\left({\hat{y}}_{i,s}^{(1)}-{\hat{y}}_{i,s}^{(0)}\right),$$ (15)

where ${\hat{y}}_{i,s}^{(1)}~𝒩\left({\mu }_y\left(X=1,Z_0=z_{0,i}^s,Z_1=z_{1,i}^s\right),{\sigma }_y^2(X=1,Z_0=z_{0,i}^s,Z_1=z_{1,i}^s)\right)$ and ${\hat{y}}_{i,s}^{(0)}~𝒩\left({\mu }_y\left(X=0,Z_0=\right.\right.\left.\left.z_{0,i}^s,Z_1=z_{1,i}^s\right),{\sigma }_y^2\left(X=0,Z_0=z_{0,i}^s,Z_1=z_{1,i}^s\right)\right)$. Note that ${\left\{z_i^s=\left(z_{0,i}^s,z_{1,i}^s,z_{2,i}^s,z_{3,i}^s\right)\right\}}_{s=1}^S$ denotes the MCMC samples of the latent variable $Z$ from the $i$th unit. Equation (15) represents an unbiased estimation of ITE using the MCMC samples. The posterior interval for ITE can then be constructed. Given a desired significance level $\alpha$ (e.g., $\alpha =0.05$), we calculate the quantile to represent the lower and upper posterior interval bounds that meet the desired significance level as

$$\left\{\begin{array}{l}{\hat{L}}_{{\Delta }_i}={\text{Quantile}}_{\alpha /2}\left({\left\{\left({\hat{y}}_{i,s}^{(1)}-{\hat{y}}_{i,s}^{(0)}\right)\right\}}_{s=1}^S\right),\\ {\hat{U}}_{{\Delta }_i}={\text{Quantile}}_{1-\alpha /2}\left({\left\{\left({\hat{y}}_{i,s}^{(1)}-{\hat{y}}_{i,s}^{(0)}\right)\right\}}_{s=1}^S\right),\end{array}\right.$$ (16)

where ${\text{Quantile}}_{\alpha /2}(\cdot )$ is the quantile function of the sampling distribution that cuts off the lower $\alpha /2$ tail of the distribution.

In the continuous treatment setting, the ADRF is estimated by

$$\hat{\mu }\left(x\right)=\frac{1}{S\times N}\sum _{s=1}^S\sum _{i=1}^N{\hat{y}}_{i,s}\left(x\right),$$ (17)

where ${\hat{y}}_{i,s}\left(x\right)\sim 𝒩\left({\mu }_y\left(X=x,Z_0=z_{0,i}^s,Z_1=z_{1,i}^s\right),{\sigma }_y^2(X=x,Z_0=z_{0,i}^s,Z_1=z_{1,i}^s)\right)$. Equation (17) represents an unbiased estimation of ADRF using all the MCMC samples. Similarly, the lower and upper posterior interval bounds of $\mu (\hat{x})$ that satisfy a desired significance level $\alpha$ are estimated by

$$\{\begin{array}{l}{\hat{L}}_{\mu }\left(x\right)={\text{Quantile}}_{\alpha /2}\left({\left\{\frac{1}{N}\sum _{i=1}^N{\hat{y}}_{i,s}\left(x\right)\right\}}_{s=1}^S\right),\hfill \\ {\hat{U}}_{\mu }\left(x\right)={\text{Quantile}}_{1-\alpha /2}\left({\left\{\frac{1}{N}\sum _{i=1}^N{\hat{y}}_{i,s}\left(x\right)\right\}}_{s=1}^S\right).\hfill \end{array}$$ (18)

2.4. Choice of Latent Dimension

The latent space is partitioned into four independent parts that play different roles in the three generative models for treatment, covariates, and outcome variables. The previous work CausalEGM has demonstrated the robustness of such partition with respect to variations in the dimensionality of latent features. Here, we provide an intuitive strategy based on sufficient dimension reduction (SDR) to help determine the dimensionality of latent features. SDR aims to identify a $k$-dimensional subspace of the $p$-dimensional predictors $(k\ll p)$ that captures all the information about a scalar response. Here, we use sliced inverse regression (SIR) (Li 1991) that employs the covariance structure of the conditional expectations of predictors given response. We compute eigenvalues of the estimated covariance matrix from SIR and retain components by inspecting eigenvalue decay and cumulative variance. Considering that linear methods such as SIR may underestimate $k$ as they fail to capture nonlinear dependencies effectively. Here, we use a conservative strategy by using SIR $\mathbb{E}[V\mid X]$ to estimate $dim\left(Z_2\right)$ and using SIR $\mathbb{E}[V\mid Y]$ to estimate $dim\left(Z_1\right)$. A similar eigenvalue analysis for the covariance matrix of $V$ is conducted to estimate the total dimension of the latent space $dim(Z)$. The dimension of latent confounder $dim\left(Z_0\right)$ is a model hyperparameter.

2.5. Model Initialization

The parameters of neural networks (e.g., weights and biases) are typically initialized by a uniform or normal distribution. However, as shown by our experiments (Table 3), the model performance can be further improved in most cases through our designed innovative strategy for model parameters initialization, inspired from the encoding generative modeling (EGM) (Liu, Chen, and Wong 2024), compared to the traditional neural network initialization. An additional encoder function $E$, represented by a Bayesian neural network, is added to CausalBGM to directly map the covariate $V$ to the latent variable (dotted line in Figure 1). Specifically, we desire that the distribution of $Z=E(V)$ should match a pre-specified distribution, which is set to be a standard normal distribution (e.g., prior of $Z$). The distribution match is achieved by adversarial training (Goodfellow et al. 2014; Liu, Xu, Jiang, and Wong 2021). By the encoding process, the high-dimensional covariates with unknown distribution are mapped to a low-dimensional latent space with a desired distribution. Since the generative models in CausalBGM include both the mean and variance functions. We add additional $L_2$ regularization of all the variance terms to ensure reasonably small variance in each generative model during the initialization process.

Table 3.

Effect of initialization strategy on the performance of CausalBGM.

Dataset	Method	RMSE	MAPE
Imbens et al.	CausalBGM*	0.095 ± 0.009	0.025 ± 0.006
	CausalBGM	0.028 ± 0.007	0.013 ± 0.003
Sun et al.	CausalBGM*	0.171 ± 0.080	0.054 ± 0.012
	CausalBGM	0.037 ± 0.009	0.013 ± 0.005
Lee et al.	CausalBGM*	1.221 ± 0.128	0.362 ± 0.017
	CausalBGM	0.080 ± 0.030	0.072 ± 0.035
Twins	CausalBGM*	0.062 ± 0.018	0.067 ± 0.024
	CausalBGM	0.031 ± 0.007	0.077 ± 0.009

NOTE: CausalBGM adopts EGM initialization strategy by default. CausalBGM* represents CausalBGM without EGM initialization and directly adopts the Xavier uniform initializer. Each method was run 10 times and the standard deviations are shown. Bold values denote the better result for each dataset and metric.

2.6. Model Hyperparameters

CausalBGM contains three generative models, which are represented by three Bayesian neural networks (BNNs), respectively. The BNN for covariate $V$ contains five hidden Bayesian layers and each layer has 64 hidden nodes. The output of BNN for covariate $V$ is $(p+1)$-dimensional where the first $p$ dimensions denote the mean ${\mu }_{\nu }$ and the last dimension denotes variance ${\sigma }_v^2$. The BNNs for treatment $X$ and outcome $Y$ each contain three hidden Bayesian layers with 64, 32, and 8 hidden nodes. The outputs of BNNs of treatment $X$ and outcome $Y$ are two-dimensional, representing the mean and variance, respectively. The leaky-ReLu function ($\mathit{\text{LeakyReLU}}(x)=\text{max}(0.2x,x)$) is used as the nonlinear activation function in each hidden layer. The Softplus nonlinear activation function ($\mathit{\text{Softplus}}(x)=log\left(1+e^x\right)$) is applied to the last dimension of the BNN output to ensure positivity of variance. Adam optimizers (Kingma 2014) with learning rate 0.0001 are used to update latent variable and model parameters. The CausalBGM model is trained in a mini-batch manner with batch size 32. The default training epochs of CausalBGM with random initialization strategy is 500. If EGM initialization strategy is used, we initialize model parameters of CausalBGM by conducting EGM for 30,000 mini-batches by default. After model initialization, the encoder $E$ as a “shortcut” to learn the latent variable is removed during the follow-up CausalBGM training with an iterative approach for up to 100 epochs. In the random walk Metropolis algorithm, we set the proposal distribution to be a normal distribution centered at the current sample with covariance matrix $I_q$. The Markov chain samples from the first 5000 iterations are discarded so that the effect of initial values is minimized (burn-in stage). Then we run the Markov chains in parallel for all samples until 3000 MCMC latent samples are collected for each sample.

3. Results

We conducted a range of experiments to evaluate the performance of CausalBGM against various state-of-the-art methods across both continuous and binary treatment scenarios. In the continuous treatment setting, our focus was on assessing how well CausalBGM could learn the average dose–response function (ADRF) that describes the change of outcome variable in response to the treatment or exposure variable. In the binary treatment setting, we aimed to verify CausalBGM’s ability to estimate both the population-level average treatment effect (ATE) and the individual-level treatment effect (ITE).

3.1. Datasets

For the continuous treatment setting, we examined four public datasets used in previous studies (Hirano and Imbens 2004; Sun et al. 2015; Colangelo and Lee 2020), comprising three simulated datasets and one semi-synthetic dataset. Each of the simulation datasets has 20,000 samples and 200 covariates. We focus on the ADRF estimate in a bounded interval. The semi-synthetic data were derived from a sample of 71,345 twin births, where weight served as the continuous treatment variable and the risk of death is treated as the outcome variable, which is simulated from a risk model. Each individual has 50 covariates. In general, the simulation risk model suggests that a higher weight of an infant leads to a lower death rate.

In the binary treatment setting, we employed datasets from the 2018 Atlantic Causal Inference Conference (ACIC) competition, which were constructed from linked birth and infant death records (LBIDD) with 117 measured covariates. These semi-synthetic datasets have treatments and outcomes simulated from diverse data-generating processes. We chose nine datasets that used the most complex generation processes (e.g., the highest degree of generation function) with sample sizes spanning from 1000 to 50,000 observations. Complete details on all datasets can be found in Appendix C.

3.2. Model Evaluation

In the continuous treatment setting, the goal is to evaluate whether the ADRF under a bounded interval is accurately estimated. To quantitatively measure the difference between the true ADRF curve and the estimated ADRF curve, two metrics, including root mean squared error (RMSE) and mean absolute percentage error (MAPE), are used for evaluation purposes denoted as

$$\left\{\begin{array}{l}\text{RMSE}=\sqrt{\frac{1}{K}\sum _{k=1}^K{\left(\mu \left(x_k\right)-\hat{\mu }\left(x_k\right)\right)}^2},\\ \text{MAPE}=\frac{1}{K}\sum _{k=1}^K\left|\frac{\mu \left(x_k\right)-\hat{\mu }\left(x_k\right)}{\mu \left(x_k\right)}\right|,\end{array}\right.$$ (19)

where $K$ represents the number of different treatment values equally distributed in the bounded interval.

In the binary treatment setting, we aim to evaluate whether individual treatment effect (ITE) can be accurately estimated.

We adopt two evaluation metrics, including absolute error of average treatment effect $({ϵ}_{ATE})$ and mean squared error of precision in estimation of heterogeneous effect $({ϵ}_{\mathit{\text{PEHE}}})$, which are denoted as

$$\left\{\begin{array}{l}{ϵ}_{ATE}=\left|\frac{1}{N}\sum _{i=1}^N{\hat{\Delta }}_i-\frac{1}{N}\sum _{i=1}^N{\Delta }_i\right|,\\ {ϵ}_{\mathit{\text{PEHE}}}=\frac{1}{N}\sum _{i=1}^N{\left({\hat{\Delta }}_i-{\Delta }_i\right)}^2,\end{array}\right.$$ (20)

where $N$ is the sample size, ${\hat{\Delta }}_i$ and ${\Delta }_i$ denote the estimated and true individual treatment effect (ITE) for the $i$th unit, respectively.

3.3. Baseline Methods

For the continuous treatment setting, we considered four well-established baseline methods: ordinary least squares (OLS), the regression prediction estimator (REG) (Imai and Van Dyk 2004; Schafer and Galagate 2015; Galagate 2016), double debiased machine learning estimators (DML) (Colangelo and Lee 2020), and CausalEGM (Liu, Chen, and Wong 2024). Note that different machine learning methods can be used in the DML method. For the binary treatment setting, we compared CausalBGM against seven leading methods for estimating treatment effect, including two variants of CFR (Shalit, Johansson, and Sontag 2017), Dragonnet (Shi, Blei, and Veitch 2019), CEVAE (Louizos et al. 2017), GANITE (Yoon, Jordon, and Van Der Schaar 2018), CausalForest (Wager and Athey 2018), and CausalEGM (Liu, Chen, and Wong 2024). Additional details about these competing methods are provided in Appendix D.

3.4. Continuous Treatment Experiments

We conducted comprehensive experiments to evaluate the performance of CausalBGM against a suite of state-of-the-art baseline methods, including the previous CausalEGM framework under continuous treatment settings. The treatment variable $X$ is defined over a bounded interval in $\mathbb{R}$. We simulated three datasets from the previous studies with a sample size of 20,000 and 200 covariates. We used the same latent dimensions as those tested for CausalEGM to ensure a fair comparison in all datasets. Specifically, for four distinct data-generating processes, the latent dimensions of $(Z_0,Z_1,Z_2,Z_3)$ were set to (1,1,1,7), (2,2,2,4), (5,5,5,5), and (1,1,1,7), respectively.

Under these settings, CausalBGM demonstrated superior performance compared to all competing methods, including CausalEGM, REG, and double debiased machine learning estimators using lasso and neural networks, achieving consistently higher accuracy. In comparison to CausalEGM, which already showed significant gains over traditional approaches, CausalBGM further improved the accuracy of the ADRF estimate and reduced both bias and variance by a large margin. As illustrated in Figure 2, the REG continued to produce larger estimation errors and exhibited limited flexibility while the double debiased machine learning estimators displayed undesirable spikes and fluctuations in their dose–response curves. CausalBGM, by contrast, yielded smoother and more stable dose–response estimates, capturing the underlying causal structure more faithfully and with smaller variance.

Figure 2.

The performance of CausalBGM and baseline methods (Reg, DML with Lasso or neural network, and CausalEGM) under continuous treatment settings across three benchmark datasets. (A) Sun et al. dataset. (B) Hirano and Imbens dataset. (C) Colangelo and Lee dataset. The red curves represent the ground truth, while the blue curves indicate the estimated average dose-response of different methods with 95% confidence intervals based on 10 independent runs.

Specifically, all methods closely follow the ground truth with linear relationship of Sun et al. dataset, but CausalBGM exhibits the most stable and precise estimations (Figure 2(A)). In the other two datasets with a nonlinear relationship, CausalBGM consistently provides more accurate estimations, particularly at the boundaries of the treatment intervals, where other competing methods display substantial deviations (Figure 2(B)–(C)). These results highlight the robustness and accuracy of CausalBGM in capturing complex dose–response relationships, especially in challenging scenarios with nonlinear effects.

We further use quantitative metrics to evaluate the performance of different methods across the above three simulation datasets and a semi-synthetic dataset. As shown in Table 1, CausalBGM demonstrates consistently superior performance in estimating average dose–response functions, achieving the state-of-the-art RMSE and MAPE in all cases. For example, CausalBGM reduced the RMSE by half, from 0.074 to 0.037, compared to CausalEGM in Sun et al. dataset. CausalBGM achieved a nearly threefold improvement in the metric MAPE, reducing the value from 0.035 to 0.013 within the same dataset. These improvements underscore the effectiveness of CausalBGM and ensure more robust and reliable causal inference across a range of complex, high-dimensional continuous treatment settings.

Table 1.

Results for the continuous treatment setting.

Dataset	Method	RMSE	MAPE
	OLS	0.680 ± 0.0	0.367 ± 0.0
	REG	0.525 ± 0.0	0.214 ± 0.0
	DML(Lasso)	0.090 ± 0.0	0.037 ± 0.0
Imbens et al.	DML(NN)	0.133 ± 0.022	0.052 ± 0.011
	CausalEGM	0.041 ± 0.014	0.019 ± 0.006
	CausalBGM	0.028 ± 0.007	0.013 ± 0.003
	OLS	0.140 ± 0.0	0.041 ± 0.0
	REG	0.117 ± 0.0	0.039 ± 0.0
	DML(Lasso)	0.163 ± 0.0	0.050 ± 0.0
Sun et al.	DML(NN)	0.097 ± 0.019	0.035 ± 0.006
	CausalEGM	0.074 ± 0.040	0.035 ± 0.017
	CausalBGM	0.037 ± 0.009	0.013 ± 0.005
.	OLS	1.3 ± 0.0	1.2 ± 0.0
	REG	1.5 ± 0.0	0.565 ± 0.0
	DML(Lasso)	0.487 ± 0.0	0.168 ± 0.0
Lee et al	DML(NN)	1.3 ± 0.581	0.494 ± 0.181
	CausalEGM	0.125 ± 0.040	0.119 ± 0.080
	CausalBGM	0.080 ± 0.030	0.072 ± 0.035
	OLS	0.109 ± 0.0	0.260 ± 0.0
	REG	11 ± 0.0	64 ± 0.0
	DML(Lasso)	0.075 ± 0.0	0.165 ± 0.0
Twins	DML(NN)	0.059 ± 0.002	0.158 ± 0.006
	CausalEGM	0.034 ± 0.020	0.090 ± 0.053
	CausalBGM	0.031 ± 0.007	0.077 ± 0.009

NOTE: Each method was run 10 times, and the standard deviation is shown. The best performance is highlighted in bold.

3.5. Binary Treatment Experiments

Most causal inference methods target binary treatment settings, which are prevalent in many real-world applications and the treatment variable only takes binary value from {0, 1}. In such a setting, we evaluated CausalBGM alongside several state-of-the-art methods, including TARNET, CFRNET, CEVAE, GANITE, Dragonnet, CausalForest, and CausalEGM across datasets of varying sizes from the ACIC 2018 benchmark. The dimension of latent space is set to be (3, 6, 3, 6), which is the same as CausalEGM. Two evaluation metrics, including ${ϵ}_{ATE}$ (error in average treatment effect estimation) and ${ϵ}_{\mathit{\text{PEHE}}}$ (error in precision for estimating heterogeneous effects) were used for evaluation. As illustrated in Table 2, CausalBGM demonstrated competitive performance in ATE estimation, achieving the best performance in 3 out of 9 datasets. For example, CausalBGM achieves the lowest error of 0.0061 in the first dataset with sample size $1k$, far surpassing the second best method CausalEGM by 37.1%. However, CausalEGM remained the leading method for ATE estimation on large datasets, such as those with sample sizes of 50k, indicating its robustness in handling extensive data. In contrast, CausalBGM demonstrated superior performance in estimating ${ϵ}_{\mathit{\text{PEHE}}}$ that considers the individual treatment effects (ITEs). CausalBGM demonstrated superior performance by achieving the best results in 8 out of 9 datasets. The improvements were particularly substantial in specific datasets. For instance, CausalBGM reduced the error by 2.4-fold in the first dataset with sample size $1k$ compared with the second best method. It achieved a 2.3-fold improvement in the last dataset with sample size $50k$.

Table 2.

Binary treatment experiments with CausalBGM and competing methods on the ACIC 2018 datasets with varying sample size.

Metric	Dataset	TARNET	CFRNET	CEVAE	GANITE	Dragonnet	CausalForest	CausalEGM	CausalBGM
${ϵ}_{\mathit{\text{ATE}}}$	Datasets-1k	0.022 ± 0.015	0.018 ± 0.015	0.035 ± 0.021	0.27 ± 0.08	0.010 ± 0.004	0.021 ± 0.001	0.0097 ± 0.0075	0.0061 ± 0.0041
		0.038 ± 0.029	0.041 ± 0.027	0.12 ± 0.10	2.0 ± 0.3	0.012 ± 0.007	0.017 ± 0.003	0.032 ± 0.020	0.029 ± 0.028
		0.10 ± 0.06	0.095 ± 0.079	0.38 ± 0.27	2.0 ± 1.4	0.16 ± 0.10	0.23 ± 0.02	0.26 ± 0.07	0.13 ± 0.05
	Datasets-10k	6.4 ± 3.5	12 ± 7	204 ± 58	2.7 ± 1.2	124 ± 11	2.5 ± 1.1	1.3 ± 0.6	1.22 ± 0.80
		0.056 ± 0.001	0.056 ± 0.001	0.070 ± 0.031	1.2 ± 0.2	0.0097 ± 0.069	0.0057 ± 0.0004	0.0043 ± 0.0025	0.0038 ± 0.0029
		0.034 ± 0.023	0.060 ± 0.002	0.018 ± 0.011	0.12 ± 0.09	0.078 ± 0.057	0.013 ± 0.003	0.039 ± 0.016	0.019 ± 0.018
	Datasets-50k	0.038 ± 0.021	0.085 ± 0.105	0.59 ± 0.31	1.4 ± 0.5	0.89 ± 0.53	0.024 ± 0.003	0.020 ± 0.013	0.045 ± 0.015
		0.044 ± 0.003	0.045 ± 0.004	0.66 ± 0.59	2.3 ± 0.2	0.027 ± 0.028	0.010 ± 0.001	0.0098 ± 0.0089	0.010 ± 0.003
		0.30 ± 0.01	0.30 ± 0.01	0.64 ± 0.45	1.9 ± 0.3	0.16 ± 0.08	0.12 ± 0.01	0.0016 ± 0.0010	0.012 ± 0.009
${ϵ}_{\mathit{\text{PEHE}}}$	Datasets-1k	0.11 ± 0.02	0.00069 ± 0.00075	0.012 ± 0.005	0.14 ± 0.04	0.038 ± 0.003	0.00080 ± 0.00005	0.0069 ± 0.0016	0.00029 ± 0.00020
		0.35 ± 0.03	0.29 ± 0.04	0.27 ± 0.04	4.34 ± 1.24	0.34 ± 0.01	0.27 ± 0.01	0.25 ± 0.01	0.18 ± 0.01
		0.31 ± 0.14	0.28 ± 0.23	7.6 ± 5.3	12 ± 6	1.7 ± 0.4	0.075 ± 0.006	0.20 ± 0.03	0.12 ± 0.03
	Datasets-10k	433 ± 106	662 ± 288	46200 ± 15500	78.7 ± 26.8	22200 ± 4130	483.72 ± 31.68	7.2 ± 2.6	6.23 ± 1.92
		0.024 ± 0.005	0.022 ± 0.006	0.091 ± 0.019	2.08 ± 0.45	0.042 ± 0.003	0.015 ± 0.001	0.014 ± 0.001	0.0120 ± 0.0001
		0.012 ± 0.005	0.0040 ± 0.0028	0.0034 ± 0.0013	0.14 ± 0.08	0.036 ± 0.015	0.0016 ± 0.0008	0.0028 ± 0.0013	0.0010 ± 0.0014
	Datasets-50k	0.88 ± 0.04	0.90 ± 0.08	1.1 ± 0.5	3.4 ± 1.4	1.84 ± 0.83	0.65 ± 0.01	0.55 ± 0.01	0.54 ± 0.01
		0.031 ± 0.006	0.030 ± 0.011	0.84 ± 0.76	5.454 ± 0.65	0.039 ± 0.007	0.020 ± 0.002	0.022 ± 0.001	0.019 ± 0.001
		0.22 ± 0.07	0.27 ± 0.05	0.67 ± 0.61	3.8 ± 1.1	0.14 ± 0.06	0.022 ± 0.001	0.0054 ± 0.0013	0.0024 ± 0.0011

NOTE: Each method was run 10 times, and the standard deviations are shown. The best performance is marked in bold.

Overall, these results demonstrate the robustness, scalability, and precision of CausalBGM, particularly excelling in individual treatment effect estimation. Its substantial improvements over strong baselines underscore its potential as a state-of-the-art approach for causal inference tasks.

Furthermore, we evaluate whether the CausalBGM framework can learn a more effective low-dimensional representation compared to CausalEGM and sufficient dimension reduction (SDR). It is important to note that all SDR-based methods for causal inference rely on linear SDR, which is inherently restrictive and may fail to capture nonlinear relationships in complex datasets. To assess this, we conducted a comprehensive comparison of CausalBGM with SDRcausal under experimental settings that either satisfied or violated the SDR assumption. SDRcausal implements several variants proposed in the original study (Ghosh, Ma, and de Luna 2021), and for fairness, we always report the best-performing result. CausalBGM demonstrated significant improvements over SDRcausal in both experimental settings, with substantial advantages in nonlinear datasets where the linear SDR approach was unable to model the underlying complexity effectively (see Appendix E). These results highlight the capability of CausalBGM to overcome the limitations of linear assumptions and better capture intricate relationships in high-dimensional data.

3.6. Posterior Interval

Unlike most of the existing methods that only focus on point estimation, CausalBGM adopts the Bayesian inference principle, thus, enabling uncertainty quantification and providing posterior interval of the causal effect estimates. More importantly, since the latent features are inferred for each subject, CausalBGM is able to offer individual treatment effect estimate with a posterior interval. To assess the utility of the posterior interval, we evaluate it based on its coverage probability or empirical coverage, which involves checking how often the true causal effect (e.g., average dose–response) lies within the predicted interval from a frequentist perspective.

We used the Imbens et al. dataset as a case study to evaluate the empirical coverage rate of posterior intervals estimated by CausalBGM. Specifically, 100 independent datasets were generated using different random seeds, and CausalBGM was applied to each dataset to estimate the average dose–response function $\mu (x)$. For a given treatment value $x$, the empirical coverage rate was defined as the proportion of times a posterior interval successfully contains the true value at a specific significance level $\alpha$. By varying the significance level $\alpha$, we generated a calibration curve of the empirical coverage rate. Interestingly, the empirical coverage rate was more accurate at treatment values $x=\mathrm{1.5,2}$ compared to other treatment values (Figure 3(A)). This discrepancy across different treatment values can be attributed to (1) the treatment value distribution as shown in the marginal density plot of $x$ (Figure 3B). (2) The property (e.g., slope) of the truth average dose–response curve (Figure 2(B)). To further investigate, we took $x=2$, the best-performing case, as a focused study. By setting the significance level $\alpha$ to 0.01, 0.05, 0.1, the average length of the posterior interval decreased from 0.126 to 0.096 and 0.080, respectively (Figure 3(C)). Additionally, we visualized the 100 posterior intervals of the average dose–response at treatment value $x=2$. As expected, smaller significance level $\alpha$ corresponded to a higher empirical coverage (Figure 3(D)–(F)). For example, at $\alpha =0.01$, only one out of 100 intervals failed to cover the truth average dose–response value, highlighting the robustness of CausalBGM in providing accurate and well-calibrated posterior intervals.

Figure 3.

Posterior interval analysis of CausalBGM using Imbens et al. dataset. (A) The calibration of empirical coverage rate at different treatment values $(x=0.2,0.5,1,1.5,2,2.5)$. (B) The marginal density plot of treatment value $x$. Vertical dotted lines with different colors represent different treatment values $(x=0.2,0.5,1,\mathrm{1.5,2},2.5$) (C) The distribution of interval lengths at different significant levels $\alpha =0.01,0.05,0.1$. (D)–(F) The coverage indicator plots of CausalBGM at different significant levels $\alpha =0.01,0.05,0.1$ where the horizontal line indicates the truth average dose-response value at $x=2$, the “covered” intervals are marked in green, and “missed” intervals are marked in red.

3.7. Effect of Initialization

The EGM initialization strategy plays an important role in ensuring the superior performance of CausalBGM. To evaluate the contribution of the EGM initialization strategy, we conducted a series of experiments comparing it with the traditional Xavier uniform initializer (Glorot and Bengio 2010) across three simulation datasets and one semi-synthetic dataset under the continuous treatment setting. The results, summarized in Table 3, demonstrate that EGM initialization significantly enhances the performance of CausalBGM in terms of both RMSE and MAPE. Quantitatively, EGM initialization consistently reduced RMSE across all datasets and improved MAPE in three out of four datasets. For instance, in the Lee et al. dataset, EGM initialization achieved remarkable reductions in RMSE and MAPE by 93.4% and 80.1%, respectively. Similarly, in the Imbens et al. and Sun et al. datasets, EGM initialization substantially improved performance, with RMSE reductions of 70.5% and 78.4%, respectively. Even in the Twins dataset, where the impact on MAPE was marginal, EGM initialization still achieved a noticeable RMSE improvement of 50.0%.

These findings underscore the critical importance of proper initialization strategies in enhancing the predictive accuracy and stability of CausalBGM. By initializing the model parameters using the EGM strategy, CausalBGM effectively improved the prediction performance. Given the consistent improvements across multiple datasets, we adopt EGM initialization as the default strategy for the CausalBGM framework.

3.8. Scalability

Scalability has become a critical requirement in causal inference, particularly for modern applications involving increasingly large and complex datasets. To evaluate the scalability of CausalBGM, we conducted comprehensive experiments examining its ability to handle datasets with (i) increasing the number of covariates and (ii) increasing the sample size (see Appendix F). We first fixed number of covariates and increased the sample size using Imbens et al dataset. Only OLS, CausalEGM and CausalBGM can scale to million-sample dataset (e.g., $N={10}^6$) while other approaches failed due to memory or time limitations. We then fixed the sample size and increased the number of covariates from $p=50$ to $p=\mathrm{10,000}$. In this setting, several competing methods (e.g., DML) failed to run as $p$ increased while CausalBGM demonstrated consistently strong performance across different range of covariate dimensions. We further reported the computational time of CausalBGM by varying sample size over orders of magnitude from $N={10}^3$ to 10⁶ (see Appendix G).

4. Discussion

In this article, we introduced CausalBGM, an AI-powered Bayesian generative modeling framework for causal inference, particularly excelling in observational studies with high-dimensional covariates and large-scale datasets. By combining deep generative modeling with Bayesian principles and causal reasoning, CausalBGM provides a flexible and robust approach for covariate adjustment, individualized effect estimation, and principled uncertainty quantification.

One notable contribution of CausalBGM is the ability to estimate individual-specific posterior distributions of latent features (interpretable as latent confounders), enabling posterior intervals for individual treatment effects (ITEs) in addition to point estimates. This individualized posterior representation offers a practically useful and statistically grounded way to quantify uncertainty at the individual level, which remains underdeveloped in existing causal inference methods. Additionally, CausalBGM is designed to be scalable since the iterative updating algorithm only requires a mini-batch of samples at each step. Such scalability, combined with its robust statistical foundations, makes CausalBGM a practical and powerful tool for addressing the demands of modern applications in genomics, healthcare, and social sciences.

Despite these strengths, several limitations and open questions remain, and we explicitly address key methodological claims to improve transparency, reproducibility, and interpretability. First, the latent variables are typically unidentifiable for latent variable modeling (LVM) since different latent representations can induce the same marginal distribution over the observed variables. We discuss this issue by leveraging recent advances in nonlinear independent component analysis (ICA) theory (see Appendix H). Specifically, identifiability can be achieved under mild conditions by introducing an auxiliary variable and conditioning the latent prior on it. Building on this idea, we developed an identifiable variant, iCausalBGM and evaluated the empirical performance compared to CausalBGM.

Second, using variational inference (VI) may lead to an underestimation of the posterior uncertainty (Murphy 2012), we implement a “full MCMC” version of CausalBGM where Metropolis–Hastings is used for sampling individual latent variables and Hamiltonian Monte Carlo is used for sampling model parameters. The empirical performance is benchmarked against the original CausalBGM (see Appendix I).

Third, CausalBGM relies on a latent-variable version of unconfoundedness, which requires relevant confounding information can be captured by observed covariates. When important covariates are missing, the impact depends on their causal role. We evaluate the performance impact on CausalBGM under different “missing-covariate” scenarios to demonstrate the robustness of our method (see Appendix J).

Looking forward, several directions could further strengthen the framework. On the theory side, while CausalBGM demonstrates strong empirical performance with its Bayesian foundation, further theoretical work is needed to rigorously characterize the convergence properties of CausalBGM under the proposed iterative algorithm. On the methodological side, the sensitivity of CausalBGM to parameter initialization remains poorly understood, which could limit its adaptability in scenarios where the EGM framework is less effective. Investigating the underlying causes of this sensitivity and exploring alternative initialization strategies or adaptive learning mechanisms could further enhance the robustness and versatility of the framework.

In conclusion, CausalBGM provides a new perspective on developing Bayesian causal inference methods by harnessing the power of AI. Its flexibility, scalability, and strong empirical performance make it a valuable tool for a wide range of applications in modern data science.

Supplementary Material

Supplementary materials for this article are available online. Please go to www.tandfonline.com/r/JASA.

Supplementary Materials include source code, technical proofs, details of datasets and baseline methods, additional experiments and analysis.