A Bayesian approach to the g-formula via iterative conditional regression

Abstract

In longitudinal observational studies with time-varying confounders, the generalized computation algorithm formula (g-formula) is a principled tool to estimate the average causal effect of a treatment regimen. However, the standard non-iterative g-formula implementation requires specifying both the conditional distribution of the outcomes and the joint distribution of all time-varying covariates. This process can be cumbersome to implement and is prone to model misspecification bias. As an alternative, the iterative conditional expectation (ICE) g-formula estimator solely depends on a series of nested outcome regressions and avoids the need for specifying the full distribution of all time-varying covariates. This simplicity lends itself to the natural integration of flexible machine learning techniques to develop more robust average causal effect estimators with time-varying treatments. In this work, we introduce a Bayesian approach that includes parametric regressions and Bayesian Additive Regression Trees to flexibly model a series of outcome surfaces. We fit the ICE g-formula and develop a sampling algorithm to obtain samples from the posterior distribution of the final causal effect estimator. We illustrate the performance characteristics of the Bayesian ICE estimator and the associated variations via simulation studies and applications to two real world data examples.

1 |. INTRODUCTION

In longitudinal observational studies with time-varying treatments and confounders, traditional regression-based adjustment can often lead to biased causal effect estimates. This is because the confounders that are measured over time act both as intermediate outcomes of previous treatment assignments and also as confounders affecting future treatment assignments and the final outcome, thereby direct adjustment of post-treatment covariates opens up the back-door path between the treatment assignment and the final outcome.¹ Under a common set of identifying assumptions, ¹ g-methods have been developed as useful tools to estimate the causal effects of longitudinal treatments by properly adjusting for both the baseline and time-varying confounding; these include the generalized computation algorithm formula (g-formula),^2,3,4 marginal structural models^5,6 and structural nested models.^7,8 Each approach has its own merits and limitations, and a comparison between their implementation and application can be found in Daniel et al.⁹ Among the g-methods, marginal structural models and g-formulas are relatively more commonly used (likely due to their intuitive simplicity and availability in software implementation), with the former depending on models for propensity scores (via inverse probability of treatment weighting) and the latter depending on models for the outcome as well as the history of time-varying covariates. From exploiting information within a set of outcome models, the g-formula approach can lead to more efficient causal effect estimates compared to inverse probability of treatment weighting, and may be attractive as a result.

Operationally, the implementation of the standard non-iterative g-formula requires one to model the conditional outcome distribution given the history of time-varying covariates and treatments, and additionally to model all time-varying covariates given their history, which can be pursued in either a frequentist ⁹ or Bayesian paradigm.¹⁰ The Bayesian implementation of the g-formula can be appealing for several reasons. It allows the incorporation of informative priors, which helps stabilize estimates especially when the sample size is small, and it provides automatic uncertainty quantification for any well-defined function of parameters. ¹¹ For example, Keil et al. ¹⁰ discussed an implementation of the Bayesian g-formula which exploits the posterior predictive distributions to derive the point and interval estimates, and demonstrated that this approach maintains the desirable frequentist properties even when the sample size is limited or data are sparse. Chen et al. ¹² developed a general class of Bayesian g-formulas for discrete-time survival data that depend on the estimated propensity and censoring scores and integrated the Bayesian Additive Regression Trees (BART) priors to alleviate concerns on fully parametric model misspecification. With discrete time-varying confounders, Josefsson and Daniels¹³ considered a Bayesian g-formula with BART priors to estimate the time-varying survivor average causal effects when the primary quantitative outcome is truncated by death.

Broadly, there are two available implementations of the g-formula: the noniterative conditional expectation (NICE) and the iterative conditional expectation (ICE). The aforementioned implementations of Bayesian g-formula have exclusively focused on NICE,¹⁰ which requires modeling the joint density of all time-varying confounders, and the conditional outcome distribution given the treatment and confounder history. ¹⁴ This is often a non-trivial task especially as the number of time-varying confounders increases; the complexity primarily arises from the need to account for dependencies across multiple covariates within each time period and between different time periods. Shinohara et al. ¹⁵ and Achy-Brou et al. ¹⁶ have enhanced the practicality of the NICE estimator for handling multidimensional time-varying confounders by incorporating the longitudinal propensity score as a lower-dimensional summary of all confounders, reducing the complexity of the estimation process; Chen et al. ¹⁷ advanced this implementation for causal survival analysis with time-dependent confounding. In addition to modeling time-varying covariates, NICE also requires outcome modeling. Zhou et al. ¹⁸ introduced a multiple imputation method for implementing the NICE estimator that uses penalized splines and longitudinal propensity scores to mitigate model misspecification bias and improve robustness, but still requires modeling both the time-varying confounders and the outcome. Typically, the risk of model misspecification increases as the number of models increases. The most severe model misspecification can lead to the g-null paradox, which occurs when the union of the assumed parametric models becomes inconsistent with the causal null. ^19,20 As a result, these approaches may incorrectly reject the causal null a priori in sufficiently large samples, even when the null holds. These challenges motivated us to seek an implementation of the g-formula without the need to model any time-varying confounders.

In this article, our objective is to address two aforementioned limitations in existing Bayesian g-formula implementations—the daunting task of modeling time-varying covariates and the risk of g-null paradox due to potential model misspecification. To tackle the first challenge, we focus on an ICE implementation^21,22,23 of the g-formula to alleviate the need to model multidimensional confounders in each time period. The ICE representation only requires a series of nested outcome models and avoids the need for specifying the joint distributions of all time-varying covariates, thus reducing the burden in modeling different types and possibly many time-varying covariates. ^14,23 To address the second challenge, we leverage Bayesian nonparametric priors to represent each regression component in the ICE formulation. The parametric g-formula presupposes that the relationship between the outcome of interest and both the exposures and possible confounders should be correctly specified, and the ICE estimator may not converge in probability to the true parameter value if any component of the series of the outcome is incorrectly modeled.²⁴ For this reason, we integrate the nonparametric prior, BART, to flexibly characterize a series of outcome models to enable the estimation of the causal effect of a specified treatment regimen. In brief, BART is a nonparametric Bayesian sum-of-trees model which uses dimensionally adaptive random basis elements.^25,26 There is a burgeoning literature in adapting BART for flexible causal inference with randomized trials and observational data. For example, BART has been used to optimize dynamic treatment regimes by fitting sequential regression models, imputing counterfactual outcomes, and addressing uncertainties in decision rules. ^27,28 Several previous studies have also provided empirical evidence demonstrating that BART can be particularly powerful with a time-fixed treatment, not only in terms of estimating average causal effects^29,30,31 but also in identifying conditional average causal effects and quantifying treatment effect heterogeneity. ^{17,32,33,34,35,36} Additionally, BART has proven effective for estimating causal effects in the presence of intermediate variables. ^13,37,38 When compared to other Bayesian nonparametric methods, such as Dirichlet process priors, ³⁹ Gaussian process priors, ⁴⁰ and Gamma process priors, ⁴¹ BART may enjoy the advantage of computational efficiency and minimal tuning requirements. However, little is known about the performance of BART in studying time-varying treatments as a flexible modeling strategy to alleviate model misspecification concerns.

The remainder of this article is organized as follows. In Section 2, we provide a review of parametric g-formula using ICE and the identifying assumptions with time-varying treatment and confounding. In Section 3, we describe a Bayesian ICE estimator and propose a sampling algorithm to obtain samples from the posterior distribution of the average causal effect estimator in the context of linear regression. We show that the requisite sampling procedure must accurately propagate the uncertainty in each stage of regression, and introduce a multistage sampling algorithm for inference. To alleviate concerns on parametric model misspecification, we employ in Section 4 a nonparametric prior specification of each mean function using BART, ²⁷ and further incorporate the estimated longitudinal propensity scores as additional covariates when modeling the responses. In Section 5, we conduct simulations to examine the finite-sample performance of the Bayesian ICE approach and draw comparisons to competing methods under different data-generating processes. Two illustrative data examples are presented in Section 6. Section 7 concludes.

2 |. PARAMETRIC G-FORMULA WITH ITERATIVE CONDITIONAL EXPECTATION: A REVIEW

2.1 |. Notation and estimand

We consider a longitudinal observational study with treatment assignment at multiple time points which can depend on the history of time-varying covariates and past treatments. Assume that we have $n$ independent and identically distributed copies of sequences of observables $\left({\mathit{L}}_1,A_1,\dots ,{\mathit{L}}_K,A_K,Y\right)$ obtained from $n$ individuals, where $K$ denotes the number of time periods. The variable $Y$ is the final outcome of interest observed at the end of the $K$-th time interval. For $k=1,\dots ,K,A_k\in \{\mathrm{0,\; 1}\}$ is the binary treatment indicator given at the $k$-th time interval and ${\mathit{L}}_k$ is a vector of covariates measured at the start of the $k$-th time interval just prior to the treatment assignment $A_k$. A directed acyclic graph illustrating the causal relationships between the full sequence of observables is shown in Figure 1.

Figure 1.

Directed acyclic graph (DAG) representing the causal relationships between all observables in a typical longitudinal observational study.

Let ${\bar{A}}_k\equiv \left(A_1,\dots ,A_k\right)$ be the treatment history up to the $k$-th interval, ${\overline{\mathit{L}}}_k\equiv \left({\mathit{L}}_1,\dots ,{\mathit{L}}_k\right)$ be the history of confounders up to the $k$-th interval, and ${\bar{O}}_k\equiv \left({\mathit{L}}_1,A_1,\cdots ,{\mathit{L}}_k\right)$ be the combined history of all past information. Under the potential outcomes framework, we can write $Y\left({\bar{a}}_K\right)$ as the potential outcome at the end of the study when treatment regimen ${\bar{a}}_K$ is followed. For example, consider a scenario with six time intervals. An intervention regimen might be represented as ${\bar{a}}_K^{(\mathrm{*})}=(\mathrm{1,1},\mathrm{1,1},\mathrm{1,1})$, which denotes the assignment of a continuously active intervention throughout the entire six-month period. An alternative regimen can be represented as ${\bar{a}}_K^{(\mathrm{*}\mathrm{*})}=(\mathrm{0,0},\mathrm{0,1},\mathrm{1,1})$, indicating the control condition for the initial three-month period, followed by an active intervention for the subsequent three months. We pursue a super-population framework, ⁴² and define the population average treatment effect estimand (abbreviated as average treatment effect), comparing two intervention regimens ${\bar{a}}_K^{(\mathrm{*})}$ with ${\bar{a}}_K^{(\mathrm{*}\mathrm{*})}$, as $E\left[Y\left({\bar{a}}_K^{(\mathrm{*})}\right)-Y\left({\bar{a}}_K^{(\mathrm{*}\mathrm{*})}\right)\right]$; here, the expectation is defined over a super-population of which the observed sample is representative. For clarity and ease of reference, a glossary of key notation in this work is provided in Table 1.

Table 1.

A glossary of notation.

Notation	Explanation
Introduced in Section 2
n	Number of individuals in the study or sample size.
K	Number of time periods.
Y	Final outcome of interest observed at the end of the K-th time interval.
Ak	Binary treatment indicator at time interval k.
L k	Vector of covariates measured at the start of the k-th time interval prior to the treatment assignment Ak.
${\overline{A}}_k$	Treatment history up to time interval $k,(A_1,\dots ,A_k)$.
${\overline{\mathit{L}}}_k$	Covariate history up to time interval $k,({\mathit{L}}_1,\dots ,{\mathit{L}}_k)$.
${\overline{O}}_k$	Combined history of all past information, $({\mathit{L}}_1,A_1,\dots ,{\mathit{L}}_k)$, up to time interval k.
$Y({\overline{a}}_k)$	Potential outcome under treatment regimen ${\overline{a}}_K$
$Y({\overline{A}}_k,{\underset{\_}{a}}_{k+1})$	Potential outcome had an individual received the observed treatment regimen ${\overline{A}}_k$ during the initial k time intervals, followed by treatment assignment ${\underset{\_}{a}}_{k+1}$ from the k + 1 time interval onward.
$f_K({\overline{a}}_K,{\overline{\mathit{l}}}_K)$	Expectation of the observed outcome at the last observation time point K given the entire history.
$f_k({\overline{a}}_K,{\overline{\mathit{l}}}_k)$	Expectation of the nested (pseudo) outcome at time k + 1 conditional on the treatment history ${\overline{a}}_k$ and the covariate history ${\overline{\mathit{l}}}_k$ up to the k-th interval.
${\mathit{\delta }}_k$	Vector of regression coefficients parameterizing the conditional expectation function fk(·).
${\widehat{\mathit{\delta }}}_k$	Estimator for ${\mathit{\delta }}_k$.
${\widehat{\mathit{Y}}}_k\left({\overline{A}}_{k-1},{\underset{\_}{a}}_k\right)$	Predicted values of the outcome from a fitted regression model at time k, that is, ${\widehat{\mathit{Y}}}_k\left({\overline{A}}_{k-1},{\underset{\_}{a}}_k\right)=f_k\left({\overline{A}}_{k-1},{\underset{\_}{a}}_k,{\overline{\mathit{L}}}_k;{\widehat{\mathit{\delta }}}_k\right)$.
$𝒜$	Set of treatment regimens of interest.
Introduced in Section 3
${\sigma }_k^2$	Variance of the error term in a postulated regression model at time k.
$p_{{\mathit{\delta }}_k}({\mathit{\delta }}_k)$	Generic notation for prior distribution on ${\mathit{\delta }}_k$.
$p_{{\sigma }_k^2}({\sigma }_k^2)$	Generic notation for prior distribution on ${\sigma }_k^2$.
$p_{k,n}(\cdot |\cdot )$	Generic notation for full conditional distribution of parameters at time k
${\tilde{y}}^{a_2}$	A generic argument representing a draw from the time-2 posterior predictive distribution given treatment a2.
${\tilde{y}}^{(a_1,a_2)}$	A generic argument representing a draw from the time-1 posterior predictive distribution given treatment regimen (a1, a2).
D K	The full observed data up until the last time interval K.
D k	Observed history up to time k and the pseudo-outcomes generated by fitted outcome model at time k + 1, $k\in \{1,\dots ,K-1\}$.
${\tau }_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}$	Causal estimand: population average treatment effect, defined in equation (1).
${\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}$	Estimated average treatment effect comparing regimens (a1, a2) and $(a_1^{\prime },a_2^{\prime })$, defined in equation (10).
Introduced in Section 4
x	Vector of q generic covariates, $x=(x_1,\dots ,x_q)$.
m	Number of regression trees in BART.
$T_j$	The j-th binary tree structure in BART.
$b_j$	Number of terminal nodes in tree Tj.
${\mathit{M}}_j$	Set of parameter values associated with each of the terminal nodes in the j-th tree, ${\mathit{M}}_j=\left\{{\mu }_{j1},\dots ,{\mu }_{j{b}_j}\right\}$.
${\mu }_{jl}$	l-th terminal node value in the j-th tree.
$g(x;T_j,{\mathit{M}}_j)$	Function assigning a terminal node value to x according to the j-th binary tree.
$p(T_j)$	Prior distribution on tree structure Tj.
$p({\mu }_{il}|T_j)$	Prior distribution on terminal node parameters ${\mu }_{jl}$.
$p(\sigma )$	Prior distribution on error standard deviation $\sigma$.
${\pi }_k$	Longitudinal propensity score at time k.

2.2 |. Iterative conditional expectation

Given that we can observe only one potential outcome corresponding to the observed treatment sequence, the potential outcomes under the treatment regimens of interest cannot be fully observed for each individual. To identify the estimand, several standard assumptions are employed. Define $f_K\left({\bar{a}}_K,{\bar{\mathit{l}}}_K\right)\equiv E\left(Y\mid {\bar{A}}_K={\bar{a}}_K,{\overline{\mathit{L}}}_K={\bar{\mathit{l}}}_K\right)$ as the mean outcome at the last observation time point $K$ given the entire history, and for $k\in \{K-1,\dots ,1\}$, we define

$$f_k\left({\overline{a}}_K,{\overline{\mathit{l}}}_k\right)\equiv E\left(f_{k+1}\left({\overline{a}}_K,{\mathit{l}}_{k+1}\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right).$$

Let $f_0\left({\bar{a}}_K\right)\equiv E\left(f_1\left({\bar{a}}_K,{\mathit{l}}_1\right)\right)$. Here, $f_k\left({\bar{a}}_K,{\overline{\mathit{l}}}_k\right)$ represents the expected value of the nested outcome at time $k+1$ conditional on the treatment history ${\bar{a}}_k$ and the covariate history ${\overline{\mathit{l}}}_k$ up to the $k$-th interval; it captures the dependence of future outcomes on past treatments and covariates in an iterative fashion. Importantly, the subscript $k$ of $f_k$ acknowledges that the conditional mean function is time-specific, and should match the subscript of time-varying confounder argument ${\overline{\mathit{l}}}_k$ (but not necessarily the subscript of the treatment assignment argument ${\bar{a}}_K$). Under the identifying assumptions of (1) consistency: $Y\left({\bar{a}}_K\right)=Y$ if ${\bar{A}}_K={\bar{a}}_K,\overline{\mathit{L}}\left({\bar{a}}_k\right)={\overline{\mathit{L}}}_k$ if ${\bar{A}}_k={\bar{a}}_k$, (2) no unmeasured confounding: for all ${\bar{a}}_K$ and $k,Y\left({\bar{a}}_K\right)\perp A_k\mid {\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k,{\bar{A}}_{k-1}={\bar{a}}_{k-1}$, and (3) positivity: for all ${\bar{a}}_k$ and $k$, if $P\left({\bar{A}}_{k-1}={\bar{a}}_{k-1},{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right)>0$, then $P\left(A_k=a_k\mid {\bar{A}}_{k-1}={\bar{a}}_{k-1},{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right)>0$, it has been shown that $E\left(Y\left({\bar{a}}_K\right)\right)$ is point identified, i.e., the expectation of the counterfactual outcomes can be written in terms of the observed data. ^3,24 That is, there exists a nested relationship such that $f_k\left({\bar{a}}_K,{\overline{\mathit{l}}}_k\right)=E\left(f_{k+1}\left({\bar{a}}_K,{\overline{\mathit{l}}}_{k+1}\right)\mid {\bar{A}}_k={\bar{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right)=E\left(Y\left({\bar{a}}_K\right)\mid {\bar{A}}_k={\bar{a}}_k,{\bar{L}}_k={\bar{\mathit{l}}}_k\right)$, for $k\in \{1,\dots ,K-1\}$ and $f_0\left({\bar{a}}_K\right)=E\left(Y\left({\bar{a}}_K\right)\right)$. The original proof of this result is given in Robins et al., ² and we reproduce the proof in Appendix 1 for ease of reference. This approach has been referred to as the iterative conditional expectation, or ICE.

To operationalize the parametric g-formula via ICE, we first posit a series of parametric models such that $f_k\left({\bar{A}}_K,{\overline{\mathit{L}}}_k\right)=f_k\left({\bar{A}}_K,{\overline{\mathit{L}}}_k;{\mathit{\delta }}_k\right),1\le k\le K$, where ${\mathit{\delta }}_k$ is a finite-dimensional regression coefficient vector. For example, with a continuous final outcome (such as blood pressure measured at the end of the longitudinal study), a natural choice may be linear regression, to estimate the expected outcome based on a possible treatment assignment sequence and the history of confounders up to a certain time interval $k$. We introduce $Y\left({\bar{A}}_k,{\underset{\_}{a}}_{k+1}\right)$ as the potential outcome of an individual if had that individual received the observed treatment regimen ${\bar{A}}_k$ during the initial $k$ time intervals, followed by treatment assignment ${\underset{\_}{a}}_{k+1}$ from the $k+1$ time interval onward until the end of study. ⁴³

For point estimation, one can compute ${\widehat{\mathit{\delta }}}_K$ which solves the estimating equation (i.e., the score equation derived from the log likelihood function) defined by the outcome model $f_K$ given all histories with parameters ${\mathit{\delta }}_K$, $P_n\left[\frac{\partial }{\partial {\mathit{\delta }}_K}{f}_K\left({\bar{A}}_K,{\bar{L}}_K;{\mathit{\delta }}_K\right)\left(Y-f_K\left({\bar{A}}_K,{\bar{L}}_K;{\mathit{\delta }}_K\right)\right)\right]=0$, where $P_n(V)\equiv \frac{1}{n}\sum _{i=1}^n{V}_i$ is the empirical expectation. For $k\in \{K-1,K-2,\dots ,1\}$, with the assumed model at time $K$ and the estimated parameter value ${\widehat{\mathit{\delta }}}_K$, we can generate the pseudo-outcomes at time $K$ for each individual under all possible regimens ${\widehat{Y}}_K\left({\bar{A}}_{K-1},{\underset{\_}{a}}_K\right)\equiv f_K\left({\bar{A}}_{K-1},{\underset{\_}{a}}_K,{\bar{L}}_K;{\widehat{\mathit{\delta }}}_K\right)$, where ${\underset{\_}{a}}_K\in \{\mathrm{0,1}\}$. These obtained pseudo-outcomes will be used in the next regression at time $K-1$ to estimate the parameters for the outcome model, $f_{K-1}$, by solving the corresponding estimating equations. Then, pseudo-outcomes at time $K-1$ can be further imputed based on ${\widehat{\mathit{\delta }}}_{K-1}$. Therefore, in a sequential manner, for $k\in \{K-1,K-2,\dots ,1\}$, we can iteratively compute ${\widehat{\mathit{\delta }}}_k$ by solving the score equation with pseudo-outcomes obtained from the previous regression model. Within each iteration, we use ${\widehat{\mathit{\delta }}}_k$ to impute the pseudo-outcomes for each person considering the actual treatment path observed until time $k,{\bar{A}}_k$, and all future treatment regimen of interest up until the end of the study, ${\underset{\_}{a}}_{k+1}$. For example, consider a linear regression defined at time $k$, then ${\widehat{Y}}_{k+1}\left({\bar{A}}_k,{\underset{\_}{a}}_{k+1}\right)=f_k\left({\bar{A}}_k,{\underset{\_}{a}}_{k+1},{\overline{\mathit{L}}}_k;{\mathit{\delta }}_k\right)+{ϵ}_k$, where $f_k$ is a linear function of treatment history and covariates, ${ϵ}_k$ is the mean-zero error term, and ${\widehat{Y}}_{k+1}$ are predicted values of the outcome from a fitted regression model at time $k+1$. The estimated parameters ${\widehat{\mathit{\delta }}}_k$ are obtained by regressing pseudo-outcomes at $k+1$ on the treatment and confounder history up to time k (${\bar{A}}_k$ and ${\overline{\mathit{L}}}_k$) and the all possible future treatment regimens (${\underset{\_}{a}}_{k+1}\in \{\mathrm{0,1}{\}}^{\otimes K-k}$), based on ordinary least squares. In general, the pseudo-outcomes imputed at time $k$ are used to estimate the model parameters at time $k-1$. This sequential regression is repeated from the end of the study, $k=K$, to $k=1$. Once we have ${\widehat{\mathit{\delta }}}_1$, we can obtain the predicted potential outcomes for each individual under all treatment regimen of interest, ${\widehat{Y}}_1\left({\underset{\_}{a}}_1\right)={\widehat{Y}}_1\left({\bar{a}}_K\right)=f_1\left({\underset{\_}{a}}_1,{\overline{\mathit{L}}}_1;{\widehat{\mathit{\delta }}}_1\right)$. Finally, the ICE estimator of $E\left(Y\left({\bar{a}}_K\right)\right)$ is obtained as the empirical average of ${\widehat{Y}}_1\left({\bar{a}}_K\right)$ across all individuals. Under the series of assumed models, $f_k,k\in \{1\dots K\}$, the parametric ICE estimator is consistent and asymptotically normally distributed. ²⁴

It is worth noting that, in a study focusing on comparing only a subset of potential treatment regimens, rather than all possible $2^K$ regimens, computational complexity can be reduced by imputing potential outcomes solely for the regimens of interest. The generic procedure for carrying out imputation without limiting the set of regimens is illustrated in Table 2, where the potential outcomes for all $2^K$ possible regimens are imputed through sequential regression. In contrast, Table 3 demonstrates a computationally simpler process when restrictions are applied to the set of regimens $𝒜$ directly relevant to the set of estimands of interest. Consider the simple example where we wish to examine the average treatment effect by comparing the never-treat regimen (0, 0, 0) with the always-treat regimen (1, 1, 1). Without restrictions, the ICE estimator would impute the potential outcomes of $(A_1,A_2,1)$ and $(A_1,A_2,0)$ at $k=3$; impute the potential outcomes of $\left(A_1,\mathrm{1,1}\right),\left(A_1,\mathrm{0,1}\right),(A_1,\mathrm{1,0})$, and $(A_1,\mathrm{0,0})$ at $k=2$; and impute the potential outcomes of all 8 possible treatment regimens at $k=1$. However, with restrictions on the set of regimens, $𝒜=\{(\mathrm{0,0},0),(\mathrm{1,1},1)\}$, the imputation procedure would still impute the potential outcomes of $\left(A_1,A_2,1\right)$ and $(A_1,A_2,0)$ at $k=3$, but only impute the potential outcomes of $(A_1,\mathrm{1,1})$ and $(A_1,\mathrm{0,0})$ at $k=2$, and only impute the potential outcomes of (0, 0, 0) and (1, 1, 1) at $k=1$. This imputation procedure is most useful when the number of time intervals is large, in which case addressing all $2^K$ potential outcomes becomes computationally prohibitive. Only potential outcomes corresponding to regimens in $𝒜$ are considered in creating the pseudo-outcomes at each time interval. The cardinality of $𝒜$ is usually much smaller than $2^K$. In what follows, for the sake of generality, we do not restrict the space of regimens and employ unrestricted imputation to demonstrate our method in the more general case. The modification of the general procedure with a smaller $𝒜$ is straightforward.

Table 2.

Imputation of potential outcomes at three time intervals without restricting the set of regimens. “k” represents the step of imputation. “Observed Regimen” refers to the actual treatment sequence assigned to an individual. “✓” indicates the available imputed potential outcomes at each step. “?” marks the missing potential outcomes at each step.

Time Interval	Observed Regimen			Imputed Potential Outcomes
	A 1	A 2	A 3	$\widehat{Y}$(0, 0, 0)	$\widehat{Y}$(0, 0, 1)	$\widehat{Y}$(0, 1, 0)	$\widehat{Y}$(0, 1, 1)	$\widehat{Y}$(1, 0, 0)	$\widehat{Y}$(1, 0, 1)	$\widehat{Y}$(1, 1, 0)	$\widehat{Y}$(1, 1, 1)
k = 3	0	0	0	✓	✓	?	?	?	?	?	?
	0	0	1	✓	✓	?	?	?	?	?	?
	0	1	0	?	?	✓	✓	?	?	?	?
	0	1	1	?	?	✓	✓	?	?	?	?
	1	0	0	?	?	?	?	✓	✓	?	?
	1	0	1	?	?	?	?	✓	✓	?	?
	1	1	0	?	?	?	?	?	?	✓	✓
	1	1	1	?	?	?	?	?	?	✓	✓
k = 2	0	0	0	✓	✓	✓	✓	?	?	?	?
	0	0	1	✓	✓	✓	✓	?	?	?	?
	0	1	0	✓	✓	✓	✓	?	?	?	?
	0	1	1	✓	✓	✓	✓	?	?	?	?
	1	0	0	?	?	?	?	✓	✓	✓	✓
	1	0	1	?	?	?	?	✓	✓	✓	✓
	1	1	0	?	?	?	?	✓	✓	✓	✓
	1	1	1	?	?	?	?	✓	✓	✓	✓
k = 1	0	0	0	✓	✓	✓	✓	✓	✓	✓	✓
	0	0	1	✓	✓	✓	✓	✓	✓	✓	✓
	0	1	0	✓	✓	✓	✓	✓	✓	✓	✓
	0	1	1	✓	✓	✓	✓	✓	✓	✓	✓
	1	0	0	✓	✓	✓	✓	✓	✓	✓	✓
	1	0	1	✓	✓	✓	✓	✓	✓	✓	✓
	1	1	0	✓	✓	✓	✓	✓	✓	✓	✓
	1	1	1	✓	✓	✓	✓	✓	✓	✓	✓

Table 3.

Imputation of potential outcomes at three time intervals with restrictions on the set of regimens. $𝒜=\{(\mathrm{0,\; 0,\; 0}),(\mathrm{1,\; 1,\; 1})\}$ is the set of treatment regimens of interest. “k” represents the step of imputation. “Observed Regimen” refers to the actual treatment sequence assigned to an individual. “✓” indicates the available imputed potential outcomes at each step. “?” marks the missing potential outcomes at each step.

Time Interval	Observed Regimen			Imputed Potential Outcomes
	A 1	A 2	A 3	$\widehat{Y}$(0, 0, 0)	$\widehat{Y}$(0, 0, 1)	$\widehat{Y}$(0, 1, 0)	$\widehat{Y}$(0, 1, 1)	$\widehat{Y}$(1, 0, 0)	$\widehat{Y}$(1, 0, 1)	$\widehat{Y}$(1, 1, 0)	$\widehat{Y}$(1, 1, 1)
k = 3	0	0	0	✓	✓	?	?	?	?	?	?
	0	0	1	✓	✓	?	?	?	?	?	?
	0	1	0	?	?	✓	✓	?	?	?	?
	0	1	1	?	?	✓	✓	?	?	?	?
	1	0	0	?	?	?	?	✓	✓	?	?
	1	0	1	?	?	?	?	✓	✓	?	?
	1	1	0	?	?	?	?	?	?	✓	✓
	1	1	1	?	?	?	?	?	?	✓	✓
k = 2	0	0	0	✓	?	?	✓	?	?	?	?
	0	0	1	✓	?	?	✓	?	?	?	?
	0	1	0	✓	?	?	✓	?	?	?	?
	0	1	1	✓	?	?	✓	?	?	?	?
	1	0	0	?	?	?	?	✓	?	?	✓
	1	0	1	?	?	?	?	✓	?	?	✓
	1	1	0	?	?	?	?	✓	?	?	✓
	1	1	1	?	?	?	?	✓	?	?	✓
k = 1	0	0	0	✓	?	?	?	?	?	?	✓
	0	0	1	✓	?	?	?	?	?	?	✓
	0	1	0	✓	?	?	?	?	?	?	✓
	0	1	1	✓	?	?	?	?	?	?	✓
	1	0	0	✓	?	?	?	?	?	?	✓
	1	0	1	✓	?	?	?	?	?	?	✓
	1	1	0	✓	?	?	?	?	?	?	✓
	1	1	1	✓	?	?	?	?	?	?	✓

3 |. A BAYESIAN PARAMETRIC G-FORMULA VIA ITERATIVE CONDITIONAL EXPECTATION

In the rest of the article, we consider the Bayesian paradigm for inference; further rationales for pursuing Bayesian inference in our setting are provided in Section 1. Under this paradigm, the posterior predictive distribution of the potential outcomes can be derived by integrating over the posterior distributions of the unknown parameters for causal inference. To clearly demonstrate the algorithm for better intuition, our discussion is presently centered on a Bayesian ICE framework with two time intervals and a continuous outcome $Y$. This approach is readily generalizable to accommodate multiple time periods and categorical outcomes. To elucidate the essential components of the sampling step in the Bayesian ICE approach, we consider linear regression. We opt for a series of nested linear regressions to also ensure compatibility between the conditional and marginal models in terms of the time-varying covariates and therefore reducing the concern about model compatibility. After detailing the sampling algorithm, we will relax these modeling assumptions by incorporating Bayesian nonparametric priors with more flexible mean function specification to enhance the robustness in Section 4.

This estimation algorithm uses the Bayesian ICE approach to numerically approximate the posterior predictive distribution of the potential outcomes, as well as to derive quantities from a comparison of potential outcomes under two treatment regimens.¹⁰ With two time intervals, the average treatment effect that compares the regimen $\left(a_1,a_2\right)$ with regimen $\left(a_1^{\prime },a_2^{\prime }\right)$ is defined as

$${\tau }_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}=E\left[Y_i\left(a_1,a_2\right)-Y_i\left(a_1^{\prime },a_2^{\prime }\right)\right],\text{for}\left(a_1,a_2\right)\ne \left(a_1^{\prime },a_2^{\prime }\right)\in {\{0,1\}}^2.$$ (1)

To estimate this quantity, we start by first specifying the time-2 outcome model, which depends on the individual’s history up to time $k=2$. Assuming an outcome regression model for the potential outcomes, we have

$$Y(A_1,A_2)=f_2\left({\overline{O}}_2,A_2;{\mathit{\delta }}_2\right)+{ϵ}_2,$$ (2)

where ${\bar{O}}_2=\left({\mathit{L}}_1,A_1,{\mathit{L}}_2\right)$ is the individual’s history at time $k=2,{\mathit{\delta }}_2$ is an unknown parameter vector for time-2 outcome model, and ${ϵ}_2$ is the error term. In a linear regression setting, a common specification of the mean function is $f_2\left({\bar{O}}_2,A_2;{\mathit{\delta }}_2\right)={\delta }_{\mathrm{2,0}}+{\delta }_{\mathrm{2,1}}{A}_1+{\delta }_{\mathrm{2,2}}{A}_2+{\mathit{\delta }}_{\mathrm{2,3}}^{\prime }{\mathit{L}}_1+{\mathit{\delta }}_{\mathrm{2,4}}^{\prime }{\mathit{L}}_2,{ϵ}_2\sim 𝒩\left(0,{\sigma }_2^2\right)$, and ${\sigma }_2^2$ is the variance of the error term. The outcome is then modeled as

$$Y(A_1,A_2)\sim {\varphi }_2\left(y|{\overline{O}}_2,A_2;{\mathit{\delta }}_2,{\sigma }_2^2\right)=𝒩\left(f_2\left({\overline{O}}_2,A_2;{\mathit{\delta }}_2\right),{\sigma }_2^2\right).$$ (3)

We then proceed by specifying the prior distributions of ${\mathit{\delta }}_2$ and ${\sigma }_2^2$, denoted as $p_{{\mathit{\delta }}_2}\left({\mathit{\delta }}_2\right)$ and $p_{{\sigma }_2^2}\left({\sigma }_2^2\right)$. In the absence of additional external information, we specify conjugate, weakly-informative priors, facilitating straightforward analytical calculations of the posterior distributions and sampling procedures. Specifically, we use a weakly informative multivariate normal prior (e.g., mean 0 and a diagonal variance matrix with diagonal entries equal to 100) for the regression coefficients and a weakly informative inverse-gamma prior (e.g., the shape and rate parameters both set to 0.001) for the error variance, which are standard choices in Bayesian regression modeling.⁴⁴ Leveraging the observed data at time $k=2,{\mathit{D}}_2={\left\{\left({\mathit{L}}_{1,i},A_{1,i},{\mathit{L}}_{2,i},A_{2,i},Y_i\right)\right\}}_{i=1}^n$, we draw from the joint posterior distribution of ${\mathit{\delta }}_2$ and ${\sigma }_2^2$ via Gibbs sampling. Samples from the posterior distribution of time-2 model parameters are obtained by iterative draws from the full conditionals:

$$p_{2,n}({\mathit{\delta }}_2|{\mathit{D}}_2,{\sigma }_2^2)\propto p_{{\mathit{\delta }}_2}({\mathit{\delta }}_2)\prod _{i=1}^n{\varphi }_2\left(y_i|{\overline{O}}_2,A_2;{\mathit{\delta }}_2,{\sigma }_2^2\right),$$

$$p_{2,n}({\sigma }_2^2|{\mathit{D}}_2,{\mathit{\delta }}_2)\propto p_{{\sigma }_2^2}({\sigma }_2^2)\prod _{i=1}^n{\varphi }_2\left(y_i|{\overline{O}}_2,A_2;{\mathit{\delta }}_2,{\sigma }_2^2\right),$$

which are themselves multivariate normal and inverse-gamma, respectively. For each draw of (${\widehat{\mathit{\delta }}}_2,{\widehat{\sigma }}_2^2$) after burn-in, we then generate the pseudo-outcomes for each unit under observed treatment history $A_1$ and all possible treatment assignments at $k=2$. The time-2 posterior predictive distribution of the potential outcome under time-2 treatment $a_2$ is defined as

$$p({\tilde{y}}^{a_2}|{\mathit{D}}_2)\equiv E_{{\mathit{\delta }}_2,{\sigma }_2^2|{\mathit{D}}_2}\left[p({\tilde{y}}^{a_2}|{\mathit{\delta }}_2,{\sigma }_2^2)\right],$$ (4)

where the expectation is defined based on the joint posterior, $p\left({\mathit{\delta }}_2,{\sigma }_2^2\mid {\mathit{D}}_2\right)$, obtained through Gibb sampling, ${\tilde{y}}^{a_2}$ is a generic argument representing a draw from the time-2 posterior predictive distribution after setting the treatment at $k=2$ as $A_2=a_2$. Each unit $i$ has two pseudo-outcomes ${\widehat{Y}}_{i,2}\left(A_{1,i},1\right)$ and ${\widehat{Y}}_{i,2}\left(A_{1,i},0\right)$, respectively, at time 2; hence ${\widehat{\mathit{Y}}}_2={\left\{{\widehat{Y}}_{i,2}\left(A_{1,i},0\right),{\widehat{Y}}_{i,2}\left(A_{1,i},1\right)\right\}}_{i=1}^n$ is a vector with $2n$ entries, reflecting two predictions per unit for all $n$ units.

Next, for each generated ${\widehat{\mathit{Y}}}_2$, we move to the previous time point $k=1$. The time-1 outcome model is defined similarly,

$$Y(A_1,a_2)=f_1({\mathit{L}}_1,A_1,a_2;{\mathit{\delta }}_1)+{ϵ}_1,$$ (5)

$$Y(A_1,a_2)\sim {\varphi }_1(y|{\mathit{L}}_1,A_1,a_2;{\mathit{\delta }}_1,{\sigma }_1^2)=𝒩(f_1({\mathit{L}}_1,A_1,a_2;{\mathit{\delta }}_1),{\sigma }_1^2),$$ (6)

where $a_2\in \{\mathrm{0,1}\},{\mathit{\delta }}_1$ is the unknown parameter vector, ${\sigma }_1^2$ is the error variance, and ${ϵ}_1\sim 𝒩\left(0,{\sigma }_1^2\right)$. In linear regression setting, we may specify $f_1\left({\mathit{L}}_1,A_1,a_2;{\mathit{\delta }}_1\right)={\delta }_{\mathrm{1,0}}+{\delta }_{\mathrm{1,1}}{A}_1+{\delta }_{\mathrm{1,2}}{a}_2+{\mathit{\delta }}_{\mathrm{1,3}}^{\prime }{\mathit{L}}_1$. Prior distributions for ${\mathit{\delta }}_1$ and ${\sigma }_1^2$ are specified as $p_{{\delta }_1}\left({\mathit{\delta }}_1\right)$ and $p_{{\sigma }_1^2}\left({\sigma }_1^2\right)$, respectively. With similar conjugate and weakly informative priors as in the time-2 model, the time-1 model posterior parameter distribution is calculated using the observed history at the first time interval and the pseudo-outcomes, denoted as ${\mathit{D}}_1={\left\{\left({\mathit{L}}_{1,i},A_{1,i},0,{\widehat{Y}}_{i,2}\left(A_{1,i},0\right)\right),\left({\mathit{L}}_{1,i},A_{1,i},1,{\widehat{Y}}_{i,2}\left(A_{1,i},1\right)\right)\right\}}_{i=1}^n$. Then, conditional on ${\mathit{D}}_1$, the time-1 samples from the posterior distribution are obtained by iterative draws from the full conditionals:

$$p_{1,n}({\mathit{\delta }}_1|{\mathit{D}}_1,{\sigma }_1^2)\propto p_{{\mathit{\delta }}_1}({\mathit{\delta }}_1)\prod _{i=1}^n\prod _{a=0}^1{\varphi }_1(y_i|{\mathit{L}}_1,A_1,a_2=a;{\mathit{\delta }}_1,{\sigma }_1^2),$$ (7)

$$p_{1,n}({\sigma }_1^2|{\mathit{D}}_1,{\mathit{\delta }}_1)\propto p_{{\sigma }_1^2}({\sigma }_1^2)\prod _{i=1}^n\prod _{a=0}^1{\varphi }_1(y_i|{\mathit{L}}_1,A_1,a_2=a;{\mathit{\delta }}_1,{\sigma }_1^2),$$ (8)

which are again themselves multivariate normal and inverse-gamma, respectively, due to conjugacy.

For inference, we consider the idea of nested multiple imputation, which is an effective approach to mix the draws from multiple posterior distribution.⁴⁵ In other words, for each draw of (${\widehat{\mathit{\delta }}}_2,{\widehat{\sigma }}_2^2$), multiple draws of (${\widehat{\mathit{\delta }}}_1,{\widehat{\sigma }}_1^2$) are retained to predict the time-1 pseudo-outcomes. The time-1 posterior predictive distribution of the potential outcome under the specified treatment regimen ($a_1,a_2$) is defined as

$$p({\tilde{y}}^{(a_1,a_2)}|{\mathit{D}}_1)\equiv E_{{\mathit{\delta }}_1,{\sigma }_1^2|{\mathit{D}}_1}\left[p({\tilde{y}}^{(a_1,a_2)}|{\mathit{\delta }}_1,{\sigma }_1^2)\right],$$ (9)

where $p\left({\mathit{\delta }}_1,{\sigma }_1^2\mid {\mathit{D}}_1\right)$ can be obtained through time-1 joint posterior distribution approximated through Gibbs sampling, and ${\tilde{y}}^{\left(a_1,a_2\right)}$ is a generic argument representing a draw from the time-1 posterior predictive distribution given treatment regimen $\left(a_1,a_2\right)$. In practice, for each draw of $\left({\widehat{\mathit{\delta }}}_1,{\widehat{\sigma }}_1^2\right)$ preserved, we generate the time-1 pseudo-outcomes for each unit from ${\varphi }_1\left(y\mid {\mathit{L}}_1,a_1,a_2;{\widehat{\mathit{\delta }}}_1,{\widehat{\sigma }}_1^2\right)$, where $a_1,a_2\in \{\mathrm{0,1}\}$. Then, each unit $i$ has four predicted outcomes ${\widehat{Y}}_{i,1}(\mathrm{1,1}),{\widehat{Y}}_{i,1}(\mathrm{1,0}),{\widehat{Y}}_{i,1}(\mathrm{0,1})$, and ${\widehat{Y}}_{i,1}(\mathrm{0,\; 0})$, respectively. ${\widehat{\mathit{Y}}}_1={\left\{{\widehat{Y}}_{i,1}(\mathrm{0,\; 0}),{\widehat{Y}}_{i,1}(\mathrm{0,\; 1}),{\widehat{Y}}_{i,1}(\mathrm{1,\; 0}),{\widehat{Y}}_{i,1}(\mathrm{1,\; 1})\right\}}_{i=1}^n$ is the vector with $4n$ entries, reflecting four predictions per unit for all $n$ units. Thus, we obtain every participant’s time-1 pseudo-outcomes under all possible treatment assignment sequences ($a_1,a_2$) of interest in the study.

To enhance the efficiency of Markov chain Monte Carlo (MCMC) methods, thinning is an important technique to reduce the autocorrelation between successive samples, ensuring that the draws are as independent as possible and represenstative of the posterior distribution. By selectively retaining only a limited number of draws from the MCMC chain, thinning mitigates the effects of serial correlation, leading to a more robust and effective approximation of the posterior. For time 2, consider the total number of draws to be $S_2$, with $B_2$ burn-in and a thinning rate of $t_2$, resulting in $C_2=\frac{S_2-B_2}{t_2}$ final samples retained at time 2. The pseudo-outcome vector corresponding to each retained draw at time 2 is denoted as ${\widehat{\mathit{Y}}}_2^{(1)},\dots ,{\widehat{\mathit{Y}}}_2^{\left(C_2\right)}$. Similarly, for each retained sample at time 2, consider the number of draws at time 1 to be $S_1$, with $B_1$ burn-in and a thinning rate of $t_1$, resulting in $C_1=\frac{S_1-B_1}{t_1}$ final samples retained at time 1. Then, for a fixed draw, $c_2$, from time 2, the time 1 pseudo-outcome vector corresponding to each retained draw at time 1 is denoted as ${\widehat{\mathit{Y}}}_1^{\left(c_2,1\right)},\dots ,{\widehat{\mathit{Y}}}_1^{\left(c_2,C_1\right)}$. The total number of draws for the two time intervals is $C=C_1\times C_2.{\widehat{Y}}_{i,1}^{\left(c_2,c_1\right)}\left(a_1,a_2\right)$ and ${\widehat{Y}}_{i,1}^{\left(c_2,c_1\right)}\left(a_1^{\prime },a_2^{\prime }\right)$ are the imputed potential outcomes of individual $i$ for the treatment regimens $\left(a_1,a_2\right)$ and $\left(a_1^{\prime },a_2^{\prime }\right)$ respectively, based on the $c_2$-th retained posterior draw from time 2, and the according $c_1$-th retained draw from time 1. The point estimator for ${\tau }_{\left(a_1,a_2\right),\left(a_1^{\prime },a_2^{\prime }\right)}$ is estimated by calculating the difference of the posterior means of the imputed average potential outcomes,

$$\begin{gathered} {\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}=\frac{1}{C_1\times C_2}\sum _{c_2=1}^{C_2}\sum _{c_1=1}^{C_1}{\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}^{(c_2,c_1)} \\ =\frac{1}{C_1\times C_2}\sum _{c_2=1}^{C_2}\sum _{c_1=1}^{C_1}\left\{\frac{1}{n}\sum _{i=1}^n\left({\widehat{Y}}_{i,1}^{(c_2,c_1)}\left(a_1,a_2\right)-{\widehat{Y}}_{i,1}^{(c_2,c_1)}\left(a_1^{\prime },a_2^{\prime }\right)\right)\right\}. \end{gathered}$$ (10)

The 95% quantile-based credible interval is determined by computing the 2.5-th and 97.5-th percentiles of the posterior distribution. The implementation of this algorithm with two time intervals is outlined in Algorithm 1.

It is important to note that the estimand comparing regimen $\left(a_1,a_2\right)$ with $\left(a_1^{\prime },a_2^{\prime }\right),{\tau }_{\left(a_1,a_2\right),\left(a_1^{\prime },a_2^{\prime }\right)}$, defined in Equation (1), is a population average treatment effect. By construction, this estimand is a function of the distribution of the potential outcomes in the target population, integrated over the population distribution of baseline confounders. In principle, one needs to model the distribution of the baseline confounders ${\mathit{L}}_1$ in order to target such a quantity. However, in practice, accurately modeling multidimensional baseline confounders can be cumbersome and prone to bias under parametric model misspecification. As an alternative, we integrate over the empirical distribution of the baseline confounders to implement our estimator in (10). In the terminology of Li et al.,⁴² our target estimand should be regarded as a mixed average treatment effect, which approximates the population average treatment effect by replacing the theoretical distribution over ${\mathit{L}}_1$ in the target population with its empirical distribution. This approach is the focus of most Bayesian causal inference methods,^29,46 and is adopted both for convenience and to avoid additional model misspecification bias.

4 |. INTEGRATING BAYESIAN ADDITIVE REGRESSION TREES

4.1 |. Model formulation

A challenge with Bayesian ICE approach under linear regression working models is the risk of parametric model misspecification. The parametric g-formula assumes that the relationship between the outcome of interest, exposures, and potential confounders is correctly specified. However, if any component of the outcome model is incorrectly specified, the ICE estimator may fail to converge to the true parameter value in probability. To alleviate bias from potential model misspecification, we consider BART,²⁵ a Bayesian nonparametric prior, as a more flexible way to model the series of outcome surfaces for the estimation of average treatment effect comparing two prespecified treatment regimen. BART has emerged as a popular tool with a wide range of applications, particularly in estimating the average and heterogeneous causal effects.^{29,47,48,34,35} Tan and Roy²⁶ provide a comparative tutorial, demonstrating the superiority of BART over parametric regressions when data are generated from a complicated outcome surface. Although our primary focus is on continuous outcomes, the methodological flexibility of BART allows its application across various types of outcomes.^49,50,51

Algorithm 1.

A Bayesian ICE sampling algorithm with K = 2 time intervals

1: Input: Observed data: Dn. Prior distributions for model parameters at time 1 and 2.

2: Define time-2 outcome model based on the observed individual’s history at time k = 2.

$Y(A_1,A_2)=f_2\left({\overline{O}}_2,A_2;{\mathit{\delta }}_2\right)+{ϵ}_2,{ϵ}_2\sim 𝒩(0,{\sigma }_2^2)$.

3: Initialize time-2 model parameters and prior distributions, $p_{{\delta }_2}\left({\mathit{\delta }}_2\right)$ and $p_{{\sigma }_2^2}\left({\sigma }_2^2\right)$.

4: Obtain samples from the time-2 posterior distribution using Gibbs sampling.

5: Consider the total number of draws to be S2, with B2 burn-in and a thinning rate of t2.

6: The number of retained samples at time 2 is $C_2=\frac{S_2-B_2}{t_2}$.

7: for $c_2\in \{1,\dots ,C_2\}$ do

8:   $({\mathit{\delta }}_2^{(B_2+t_2{c}_2)},{\sigma }_2^{2(B_2+t_2{c}_2)})$ denotes the c2-th retained posterior draw of time-2 model.

9:   Generate the time-2 pseudo-outcomes, ${\widehat{\mathit{Y}}}_2^{(c_2)}={\left\{{\widehat{Y}}_{i,2}^{(c_2)}\left(A_{1,i},0\right),{\widehat{Y}}_{i,2}^{(c_2)}\left(A_{1,i},1\right)\right\}}_{i=1}^n$.

10:   Use ${\widehat{\mathit{Y}}}_2^{(c_2)}$ as the outcome for time-1 model.

11:   Specify time-1 outcome model,

$Y\left(A_1,a_2\right)=f_1\left({\mathit{L}}_1,A_1,a_2;{\mathit{\delta }}_1\right)+{ϵ}_1,{ϵ}_1\sim 𝒩\left(0,{\sigma }_1^2\right)$.

12:   Specify prior distributions of time-1 model parameters, $p_{{\mathit{\delta }}_1}\left({\mathit{\delta }}_1\right)$ and $p_{{\sigma }_1^2}\left({\sigma }_1^2\right)$.

13:   Obtain samples from the time-1 parameter posterior distribution.

14:   Consider the number of draws at time 1 to be S1, with B1 burn-in and a thinning rate of t1.

15:   The number of retained samples at time 1 is $C_1=\frac{S_1-B_1}{t_1}$.

16:   for $c_1\in \{1,\dots ,C_1\}$ do

17:     Based on ${\widehat{\mathit{Y}}}_2^{(c_2)},({\mathit{\delta }}_1^{(c_2,B_1+t_1{c}_1)},{\sigma }_1^{2(c_2,B_1+t_1{c}_1)})$ denotes the c1-th retained draw at time 1.

18:     Generate the time-1 pseudo-outcomes,

${\widehat{\mathit{Y}}}_1^{(c_2,c_1)}={\left\{{\widehat{Y}}_{i,1}^{(c_2,c_1)}(0,0),{\widehat{Y}}_{i,1}^{(c_2,c_1)}(0,1),{\widehat{Y}}_{i,1}^{(c_2,c_1)}(1,0),{\widehat{Y}}_{i,1}^{(c_2,c_1)}(1,1)\right\}}_{i=1}^n$.

19:     Estimate the posterior sample of causal contrast comparing regimens (a1, a2) and $(a_1^{\prime },a_2^{\prime })$,

${\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}^{(c_2,c_1)}=\frac{1}{n}\sum _{i=1}^n\left({\widehat{Y}}_{i,1}^{(c_2,c_1)}\left(a_1,a_2\right)-{\widehat{Y}}_{i,1}^{(c_2,c_1)}\left(a_1^{\prime },a_2^{\prime }\right)\right)$.

20:   end for

21: end for

22: The point estimate of average treatment effect comparing regimen (a1, a2) with $(a_1^{\prime },a_2^{\prime })$ is

${\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}=\frac{1}{C_1\times C_2}\sum _{c_2=1}^{C_2}\sum _{c_1=1}^{C_1}{\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}^{(c_2,c_1)}$.

23: Compute the 95% credible interval of estimated average treatment effect by taking the 2.5-th and 97.5-th percentile of $\left({\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}^{(1,1)},\dots ,{\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}^{(C_2,C_1)}\right)$.

24: Output: The estimated treatment effect, ${\widehat{\tau }}_{(a_1,a_2),(a_1^{\prime },a_2^{\prime })}$, and its credible interval.

The BART is composed of two main parts: a sum-of-trees model and a regularization prior on the model parameters.²⁵ Suppose that the final outcome $\mathit{Y}$ is related to $q$ generic covariates, represented as $x=\left(x_1,\dots ,x_q\right)$, and the BART is given by

$$Y=\sum _{j=1}^{m}g\left(x;T_j,M_j\right)+ϵ,$$ (11)

where $ϵ\sim N\left(0,{\sigma }^2\right)$. It approximates the mean function using a sum of $m$ regression trees denoted as $\sum _{j=1}^{m}g\left(x;T_j,M_j\right)$, where $T_j$ is the $j$-th binary tree structure and $M_j=\left\{{\mu }_{j1},\dots ,{\mu }_{j{b}_j}\right\}$ denotes a set of parameter values associated with each of the $b_j$ terminal nodes in tree $T_j$.²⁵ An increase in $m$ enhances the model flexibility, but the choice of $m$ can affect the computational demand. Previous work indicates that, starting with $m=1$, the predictive performance of BART improves dramatically until at some point it levels off and then begins to very slowly degrade for even larger values of $m$.²⁵ A specification of $m=50$ trees is often deemed sufficient according to Bleich et al. ⁵² Compared to the linear mean model specification, a major advantage of BART is its flexibility in capturing the potentially complex functional form of covariates in the mean function.²⁶ The function $g\left(x;T_j,M_j\right)$ assigns a terminal node value ${\mu }_{jl}\in M_j$ to $x$ according to binary tree $T_j$. The mean function, $E(Y\mid x)$, is the sum of all the terminal node values assigned to $x$ by each of the binary regression tree. When the number of trees $m$ exceeds one, each ${\mu }_{jl}$ contributes only partially to $E(Y\mid x)$. Additionally, ${\mu }_{jl}$ represents a main effect if $g\left(x;T_j,M_j\right)$ depends on a single component of $x$, and an interaction effect if it depends on multiple components. ²⁵ Thus, the sum-of-trees model by construction automatically addresses both main effects and interaction effects, enhancing the predictive performance.

To achieve regularization, BART assigns priors for paramaters $\left(T_1,M_1\right),\dots ,\left(T_m,M_m\right)$, and the variance ${\sigma }^2$. The regularization priors can prevent any single tree from having an overwhelming influence on the final model prediction. The priors are designed such that the tree components $\left\{\left(T_1,M_1\right),\dots ,\left(T_m,M_m\right)\right\}$ and ${\sigma }^2$ are a priori independent and that the tree components $\left(T_1,M_1\right),\dots ,\left(T_m,M_m\right)$ are independent of each other. The specification only requires for priors $p\left(T_j\right),p\left({\mu }_{jl}\mid T_j\right)$ and $p(\sigma )$. For the tree structure prior $p\left(T_j\right)$, the default specifications by Chipman, George, and McCulloch⁵³ (CGM98) are used.²⁵ The terminal node parameter priors $p\left({\mu }_{jl}\mid T_j\right)$ are the conjugate normal distribution, and the prior on ${\sigma }^2$ is an inverse Chi-squared distribution where the hyperparameters can be chosen by cross-validation. With these priors, posterior inference for a single BART model proceeds through an iterative Bayesian backfitting MCMC algorithm.²⁵

4.2 |. Outcome models for two time intervals

To flexibly model a series of outcome surfaces under the ICE representation using the BART model, the time-2 outcome model is given by

$$Y(A_1,A_2)=f_2({\mathit{L}}_1,A_1,{\mathit{L}}_2,A_2)+{ϵ}_2=\sum _{j=1}^m{g}_2\left({\mathit{L}}_1,A_1,{\mathit{L}}_2,A_2;T_{2j},M_{2j}\right)+{ϵ}_2,$$

where ${ϵ}_2\sim 𝒩\left(0,{\sigma }_2^2\right)$. Similarly, the time-1 outcome model is given by a separate BART specification based on the pseudo-outcomes

$$Y\left(A_1,a_2\right)=f_1({\mathit{L}}_1,A_1,a_2)+{ϵ}_1=\sum _{j=1}^m{g}_1\left({\mathit{L}}_1,A_1,a_2;T_{1j},M_{1j}\right)+{ϵ}_1,$$

where ${ϵ}_1\sim 𝒩\left(0,{\sigma }_1^2\right)$. Instead of generating posterior samples from the posterior distributions of the linear model parameters in linear regression (or other parametric regression), we now draw posterior samples of $\left(f_2,{\sigma }_2\right)$ at time 2, and $\left(f_1,{\sigma }_1\right)$ at time 1. The remaining steps of the sampling algorithm referred to in Algorithm 1 are unchanged from Section 3. Figure 2 presents a schematic illustration of the sampling process for $K=2$ time intervals using BART to model outcome surfaces under the ICE representation (BART-ICE), with prespecified numbers of MCMC draws for each BART fitting, burn-in, and thinning rate at both time points.

Figure 2.

Schematic of a BART-ICE sampling procedure for $K=2$ time intervals. ${\mathit{Y}}^{\mathit{obs}}$ denotes the observed outcome vector. ${\mathit{L}}_k$ denotes a set of confounders measured at the beginning of the $k$-th interval. $A_k$ denotes a binary treatment in the $k$-th time interval. The notation $\mathit{Y}\sim (\cdot )$ represents a model where $\mathit{Y}$ is the outcome and the terms inside the parentheses denote predictor variables, with lowercase terms indicating fixed values. At time 2, let the total number of draws be $S_2$, with $B_2$ draws discarded as burn-in and a thinning rate of $t_2$, resulting in $C_2=\frac{S_2-B_2}{t_2}$ retained samples. $\left(f_2^{\left(B_2+t_2{c}_2\right)},{\sigma }_2^{\left(B_2+t_2{c}_2\right)}\right)$ denotes the $c_2$-th retained draw from time-2 BART posterior, which generates the pseudo-outcomes ${\widehat{\mathit{Y}}}_2^{\left(c_2\right)}$ at time 2. For each retained sample at time 2, let the number of draws at time 1 be $S_1$, with $B_1$ burn-in and a thinning rate of $t_1$, resulting in $C_1=\frac{S_1-B_1}{t_1}$ retained samples. Then, for a fixed draw $c_2$ from time 2, $\left(f_1^{\left(c_2,B_1+t_1{c}_1\right)},{\sigma }_1^{\left(c_2,B_1+t_1{c}_1\right)}\right)$ denotes the $c_1$-th retained draw from time-1 BART posterior. ${\widehat{\mathit{Y}}}_1^{\left(c_2,c_1\right)}$ denotes the time-1 pseudo-outcomes generated by $\left(f_1^{\left(c_2,B_1+t_1{c}_1\right)},{\sigma }_1^{\left(c_2,B_1+t_1{c}_1\right)}\right).{\widehat{\tau }}^{\left(c_2,c_1\right)}$ is the estimated average treatment effect calculated from ${\widehat{\mathit{Y}}}_1^{\left(c_2,c_1\right)}$. The total number of draws retained for the two time intervals is $C=C_1\times C_2$.

4.3 |. Incorporation of longitudinal propensity scores

With a time-fixed treatment and a single time interval, Dorie et al.³⁵ and Hahn et al.³⁴ considered the inclusion of an estimated propensity score as an additional covariate in BART to enhance its robustness and efficiency for causal inference. With a time-to-event outcome, Hu et al.³³ also demonstrated that incorporating an estimated propensity score into the set of covariates in an accelerated failure time BART model can further enhance the efficiency in estimating conditional average treatment effect. With time-varying treatments and under a NICE formulation, Achy-Brou et al.¹⁶ and Chen et al.¹² developed the g-formula that includes an estimated longitudinal propensity score as a low-dimensional covariate that is sufficient to control for all time-varying confounding, for non-censored and censored outcomes, respectively. Following Achy-Brou et al.,¹⁶ the assumption of unconfoundedness given covariates implies unconfoundedness given longitudinal propensity scores. This suggests that the treatment assignment can be considered ignorable given the propensity scores. Consequently, adjusting for longitudinal propensity scores is sufficient to address time-varying confounding. More generally, as pointed out in Zigler et al.,⁵⁴ a Bayesian regression estimator that adjusts for the estimated propensity score along with all confounders in the propensity score model is akin to a doubly robust estimator, which consistently estimate the causal effect when either the propensity score model or the model for additional confounder adjustment is correct. Similarly, Li et al.⁴² mentioned that such a specification implicitly conducts regression within each propensity score strata to address residual confounding and is considered as a Bayesian analog of a doubly robust estimator (even though it does not strictly possess the frequentist double robustness property). These prior discussions have motivated us to adopt a similar approach under the ICE formulation, using the estimated longitudinal propensity scores as additional covariates. Following Hahn et al.,³⁴ one may not necessarily need to explicitly address the uncertainty of the estimated propensity score in a BART model, as Bayesian inference on treatment effects is derived directly from the conditional distribution of the outcome given treatments and time-varying covariates, with the estimated longitudinal propensity scores being functions of these covariates. Furthermore, the estimated longitudinal propensity scores are included in the model along with the full covariate vector, and therefore this approach should be at least as capable of estimating a complex response surface as the corresponding model without propensity scores.

In our setting, the longitudinal propensity score is defined as the conditional probability of receiving a treatment given pretreatment covariates at each time interval.⁵⁵ In the two time interval scenario, the longitudinal propensity scores is given by ${\pi }_k=\mathrm{P}\mathrm{r}\left(A_k=1\mid {\bar{A}}_{k-1},{\overline{\mathit{L}}}_k\right),k=\mathrm{1,2}$. Assuming ${\hat{\pi }}_1$ and ${\hat{\pi }}_2$ are estimates of longitudinal propensity scores, ${\pi }_1$ and ${\pi }_2$, we consider a variant of the Bayesian ICE estimator that includes $\left\{{\mathit{L}}_1,{\pi }_1,A_1,{\mathit{L}}_2,{\pi }_2,A_2\right\}$ in the time-2 model formulation, and includes $\left\{{\mathit{L}}_1,{\pi }_1,A_1,a_2\right\}$ in the time-1 model formulation. To mitigate potential model misspecification issues, we obtain the longitudinal propensity score estimates, ${\widehat{\pi }}_1$ and ${\widehat{\pi }}_2$, nonparametrically using the Super Learner.³³ We employ a Super Learner due to its flexibility to draw tools from multiple libraries, and highly accurate performance in binary classification problems.⁵⁶ These propensity score estimates are then treated as fixed covariates throughout all time intervals.

5 |. SIMULATION STUDIES

5.1 |. Simulation design

To compare the empirical performance of the BART-ICE estimators and the estimators derived from modeling outcome surfaces by fitting parametric linear regression (BLR-ICE) as well as other frequentist ICE alternatives, we carry out a series of simulation studies, focusing on estimating the average treatment effect in the context of longitudinal observational studies. Throughout we consider a total sample size of $n=500$ and $K=2$ time intervals, comparing regimen (1, 1) with (0, 0).

In the first time interval, we simulate $p_1=3$ covariates, where $X_{11}$ and $X_{21}$ are drawn from the standard normal distribution $𝒩(\mathrm{0,1})$, and $X_3$ is drawn from a folded normal distribution, i.e., $X_3=|Z|,Z\sim 𝒩(\mathrm{0,1})$. $X_{11}$ and $X_{21}$ are time-varying covariates while $X_3$ is a baseline covariate. The treatment assignment in the first time interval is generated from $A_1\mid X_{11},X_{21},X_3\sim \mathrm{Bernoulli}\left({\pi }_1\right)$. The true propensity score in the first time interval follows

$${\pi }_1=\text{Pr}\left(A_1=1|X_{11},X_{21},X_3\right)=\frac{\text{exp}\left\{{\mathit{\gamma }}_1{\mathit{L}}_1-\frac{1}{2}\right\}}{1+\text{exp}\left\{{\mathit{\gamma }}_1{\mathit{L}}_1-\frac{1}{2}\right\}},$$

where ${\mathit{\gamma }}_1=(1,-\mathrm{0.5,\; 0.25})$ and ${\mathit{L}}_1={\left(X_{11},X_{21},X_3\right)}^{\prime }$. In the second time interval, we simulate $p_2=2$ covariates. $X_{12}=Z_{12}{U}_2$ and $X_{22}=Z_{22}{U}_2$, where $Z_{12}$ and $Z_{22}$ are drawn from the standard normal distribution, and $U_2=2+\left(2A_1-1\right)/3$. The treatment assignment at the second time interval is generated from $\mathrm{Bernoulli}\left({\pi }_2\right)$. The true propensity score follows

$${\pi }_2=\text{Pr}\left(A_2=1|X_{11},X_{21},X_3,A_1,X_{12},X_{22}\right)=\frac{\text{exp}\left\{-A_1+{\mathit{\gamma }}_2{\mathit{L}}_2+\frac{1}{4}\right\}}{1+\text{exp}\left\{-A_1+{\mathit{\gamma }}_2{\mathit{L}}_2+\frac{1}{4}\right\}},$$

where ${\mathit{\gamma }}_2=(1,-0.5)$ and ${\mathit{L}}_2={\left(X_{12},X_{22}\right)}^{\prime }$. We then consider four data-generating processes with different complexities:

1. $Y=250-10A_1-10A_2+27.4X_{11}+13.7X_{21}+13.7X_3+27.4X_{12}+13.7X_{22}+ϵ$, where $ϵ\sim 𝒩(\mathrm{0,25})$.

2. $Y=-1-\frac{3}{1+\mathrm{e}\mathrm{x}\mathrm{p}\left(-6\left(X_{11}+X_{21}+X_3+X_{12}+X_{22}\right)\right)}+2X_{12}{X}_{22}+1.8\left(X_{11}+X_{21}+X_3\right)A_1-\frac{2}{1+\mathrm{e}\mathrm{x}\mathrm{p}\left(-4\left(X_{12}+X_{22}\right)\right)}{A}_2+ϵ$, where $ϵ\sim 𝒩(\mathrm{0,\; 0.64})$.

3. $Y=0.5+0.1A_1+0.7\mathrm{s}\mathrm{i}\mathrm{n}\left(A_1\right)+0.4\mathrm{s}\mathrm{i}\mathrm{n}\left(A_2\right)+0.2X_{11}+0.3X_{21}^2+0.7\mathrm{s}\mathrm{i}\mathrm{n}\left(X_3\right)+0.2X_{11}{X}_{21}+0.9\sqrt{\left|X_{11}{X}_3\right|}+0.8\mathrm{l}\mathrm{o}\mathrm{g}\left(\left|X_{12}{X}_{22}{X}_3+A_2\right|\right)+ϵ$, where $ϵ\sim 𝒩(\mathrm{0,6.25})$.

4. $Y=\left\{\begin{array}{ll}-10-5\mathrm{s}\mathrm{i}\mathrm{n}\left(\pi X_{11}{X}_3\right)+\frac{2}{1+e^{-X_{22}}}+ϵ& \text{if}{A}_1=1\text{and}{A}_2=1,\\ -6+3X_{11}^2-5\mathrm{s}\mathrm{i}\mathrm{n}\left(X_{21}\right)+\frac{4}{1+e^{-X_3}}+2X_{12}-3X_{22}+ϵ& \text{if}{A}_1=1\text{and}{A}_2=0,\\ -4+X_{11}^2-2\mathrm{s}\mathrm{i}\mathrm{n}\left(X_3\right)+\frac{2}{1+e^{-X_{22}}}+ϵ& \text{if}{A}_1=0\text{and}{A}_2=1,\\ 7.5-\frac{1}{1+e^{-X_{21}}}+\mathrm{s}\mathrm{i}\mathrm{n}\left(X_3\right)-X_{12}^2+2X_{12}-X_{22}^2+ϵ& \text{otherwise}.\end{array}\right.$

   Among all treatment conditions, $ϵ\sim 𝒩(\mathrm{0,225})$. In this final scenario, we intentionally set a large error variance to ensure that the true effect size is comparable to the standard deviation of the error. This helps to evaluate the robustness of the method under conditions where the signal-to-noise ratio is low, which is often encountered in real-world data.

Across all four data-generating processes, $\left\{X_{11},X_{21},X_3,X_{12},X_{22}\right\}$ are confounders influencing both the treatment assignments as well as the final outcome. In scenario (1), the true outcome model does not include any non-linear terms or transformations of the confounders or treatment assignments. In contrast, scenarios (2), (3), and (4) incorporate increasingly complex relationships, integrating both linear and non-linear confounding effects. Specifically, scenario (2) contains interactions between treatments and confounders, as well as among the confounders themselves. Scenario (3) includes the trigonometric functions with periodic relationships, and interaction terms involving a logarithm of an absolute value. In scenario (4), we introduce a piecewise function with four different equations depending on the values of treatment assignments. Each piecewise function includes both linear and non-linear terms, depending on the combinations of treatment assignments at the two time points. For each data-generating process, we evaluate and compare the performance of the following Bayesian methods (1–4) and frequentist methods (5–10), each within the ICE framework to ensure comparability.

1. (BLR-ICE) Introduced in Section 3, this approach models the mean outcome function at each time point by fitting a Bayesian version of linear regression based on the ICE representation.

2. (BLR-PS-ICE) The BLR-ICE estimator with the estimated longitudinal propensity scores included as additional covariates. We use the R package SuperLearner to estimate the longitudinal propensity scores, based on an ensemble library including generalized additive models, generalized linear models with and without regularization. The same approach is used to estimate longitudinal propensity scores for all subsequent comparators, whenever applicable.

3. (BART-ICE) This approach uses BART as the working outcome model for $f_k$ under the ICE representation at each time point. We used the bart function from the R package BayesTree with the default settings to implement BART-ICE.

4. (BART-PS-ICE) The BART-ICE estimator with the estimated longitudinal propensity scores included as additional covariates.

5. (BSpline-ICE) Under the general ICE formulation in Section 2 this approach uses the B-splines for outcome modeling by transforming continuous covariates into a basis matrix representing a family of piecewise polynomial functions, defined by specified interior knots and polynomial degree.⁵⁷ The transformation of continuous variables is implemented using the bs function from the splines package. For the simulation, the default settings of the spline basis, such as the degrees of freedom, interior knots, and the degree of the piecewise polynomial, are applied. For this and all subsequent approaches, variance and interval estimation proceed through nonparametric bootstrap with 100 replicates.

6. (BSpline-PS-ICE) The BSpline-ICE estimator with the estimated longitudinal propensity scores included as additional covariates.

7. (RF-ICE) This approach models $f_k$ nonparametrically by random forest, which is an ensemble learning method that constructs multiple decision trees during training and combines their predictions to improve regression accuracy and reduce overfitting. ⁵⁸ RF-ICE estimator fits a random forest at each time point using the randomForest function from the randomForest package⁵⁹. We set number of trees to be ntree = 500.

8. (RF-PS-ICE) The RF-ICE estimator with the estimated longitudinal propensity scores included as additional covariates.

9. (KR-ICE) This approach models $f_k$ nonparametrically by kernel regression, which estimates the relationship between covariates and outcome by weighting nearby observations based on a kernel function. We use the npreg function from the np package⁶⁰ to implement kernel regression, which allows continuous, unordered, and ordered factor data types, employing kernel smoothing with automated bandwidth selection. The neighborhood bandwidth specification is performed using function npregbw following the method of Racine and Li. ^61,62 Gaussian kernels were used for continuous covariates, while Aitchison-Aitken kernels were applied for categorical covariates.

10. (KR-PS-ICE) The KR-ICE estimator with the estimated longitudinal propensity scores included as additional covariates.

For computational considerations, each simulation data-generating process is independently replicated $H=100$ times, and all 10 estimators are fit to each one of the $H$ simulated datasets. The performance metrics are provided in Section 5.3.

5.2 |. Implementation details of the Bayesian methods

For Bayesian methods, BART and BLR, at time 2, we generate $S_2=4000$ total MCMC samples, with $B_2=3000$ as burn-in and a thinning rate of $t_2=10$. For each retained sample at time 2, the number of MCMC draws at time 1 was $S_1=4000$, with $B_1=3000$ discarded as burn-in and a thinning rate of $t_1=100$. Thus, we retained every 10-th time-2 posterior sample after burn-in, and accordingly, every 100-th time-1 sample from each chain after the burn-in period. The total number of draws for the two time intervals is $C=C_2\times C_1=\frac{S_2-B_2}{t_2}\times \frac{S_1-B_1}{t_1}=\frac{4000-3000}{10}\times \frac{4000-3000}{100}=1000$. By examining the traceplots of the MCMC samples, we verify the convergence of the algorithm, ensuring that the chains mix well and are representative of the target posterior distribution. Choosing a smaller $C_2=\frac{S_2-B_2}{t_2}$ ensures that we can propagate the tree even with larger data samples, thereby reducing the computational cost of sampling. Selecting an appropriate $C_1=\frac{S_1-B_1}{t_1}$ allows us to sufficiently explore the posterior distribution space through multiple imputation.⁴⁵ When computational resources permit, a larger $C_2$ can be considered to further improve the accuracy of the sampling process. With limited computational resources, thinning enables us to run a sufficiently long chain and select representative posterior samples. Compared to running a short chain and retaining all posterior samples, this approach is more effective in approximating the posterior distribution. All simulations were carried out using R version 4.3.1. ⁶³ To facilitate the application of these methods to estimate the average treatment effect, we provide code to replicate our simulation studies in the GitHub page https://github.com/Ruyi-Liu/Bayesian-ICE.

5.3 |. Performance metrics

We evaluate the overall accuracy in estimating the average treatment effect for each method using relative bias. For Bayesian methods, there are $C=1000$ different ${\widehat{\mathit{Y}}}_1$ generated for each simulated dataset $h$, and (${\widehat{\tau }}_h^{(1)},\cdots ,{\widehat{\tau }}_h^{(C)}$) is the sequence of estimated treatment effect corresponding to each ${\widehat{\mathit{Y}}}_1,h\in \{1,\dots ,H\}$. The point estimator for the average treatment effect of dataset $h$ based on the posterior mean is given by ${\widehat{\tau }}_h=\frac{{\widehat{\tau }}_h^{(1)}+\cdots +{\widehat{\tau }}_h^{(C)}}{C}$. For frequentist methods, we also use ${\widehat{\tau }}_h$ to generically denote the point estimator for the average treatment effect using dataset $h$. Then, the relative bias is defined as

$$\text{Relative Bias}=\frac{\frac{1}{H}{\sum }_{h=1}^H{\widehat{\tau }}_h-\tau }{\tau },$$

where the true causal effect, $\tau ={\tau }_{\left(a_1,a_2\right),\left(a_1^{\prime },a_2^{\prime }\right)}$, for each scenario is calculated from a sufficiently large sample—a simulated population with 10, 000, 000 participants. ⁶⁴ The Monte Carlo Standard Deviation (MCSD) is then calculated for each method to measure the spread of the estimated values across different simulation runs. For each method, MCSD is computed as

$$\text{MCSD}=\sqrt{\frac{1}{H-1}\sum _{h=1}^H{\left({\widehat{\tau }}_h-\left(\frac{1}{H}\sum _{h=1}^H{\widehat{\tau }}_h\right)\right)}^2}.$$

Additionally, we also calculate the root mean squared error (RMSE) for each method,

$$\text{RMSE}=\sqrt{\frac{{\sum }_{h=1}^H{\left({\widehat{\tau }}_h-\tau \right)}^2}{H}},$$

which is an overall measure accounting for the bias and variability of each estimator. Finally, the frequentist coverage of the 95% credible interval (for Bayesian methods) or confidence interval (for frequentist methods) is calculated as follows. For each simulated dataset h, the 95% credible interval in Bayesian methods is determined by the 2.5-th and 97.5-th percentiles from the sequence of MCMC draws, $\left({\widehat{\tau }}_h^{(1)},\dots ,{\widehat{\tau }}_h^{(C)}\right)$. For frequentist methods, the 95% confidence interval is computed as $\left({\widehat{\tau }}_h-1.96{\widehat{\sigma }}_h,{\widehat{\tau }}_h+1.96{\widehat{\sigma }}_h\right)$, where ${\widehat{\sigma }}_h$ is the estimated standard error of ${\widehat{\tau }}_h$ through nonparametric bootstrap with 100 replicates. Coverage is then calculated as the proportion of all $H$ datasets for which the interval estimator includes the true treatment effect, $\tau$.

5.4 |. Simulation results

We focus on estimating the average treatment effect comparing treatment regimens (1, 1) and (0, 0), using the ten methods introduced in Section 5.1 and under four data-generating processes of varying complexity. Figure 3 and Figure 4 summarize, for a sample size of $n=500$, the relative bias and the frequentist coverage of the estimated average treatment effect using all methods, respectively. For brevity in the main text, results on MCSD and RMSE are presented in Appendix 2 and 3.

Figure 3.

Box plots of relative deviation from the truth $\left(\left({\widehat{\tau }}_h-\tau \right)/\tau \right)$ for estimating average treatment effect comparing regimen (1, 1) with (0, 0) using each of ten methods across four data-generating processes. The methods are categorized into two groups: bayesian and frequentist. Each box represents the distribution of relative bias for a given method, with the horizontal dashed line indicating zero bias. The four data-generating processes are labeled as Scenario 1, Scenario 2, Scenario 3, and Scenario 4, with facet panels displaying the results separately. Bayesian methods include BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE, while frequentist methods include BSpline-ICE, BSpline-PS-ICE, RF-ICE, RF-PS-ICE, KR-ICE, and KR-PS-ICE.

Figure 4.

Frequentist coverage of the $95\mathrm{\%}$ confidence or credible interval across all simulated datasets for ten methods under four data-generating processes. The dashed horizontal line represents the nominal coverage level. Bar heights indicate the proportion of simulations in which the true parameter was contained within the estimated confidence or credible intervals. Bayesian methods include BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE, while frequentist methods include BSpline-ICE, BSpline-PS-ICE, RF-ICE, RF-PS-ICE, KR-ICE, and KR-PS-ICE.

Overall, the simulation results indicate that among the Bayesian methods, BART-ICE and BART-PS-ICE consistently yield lower relative bias across data-generating processes (2)–(4), compared to their fully parametric BLR-based counterparts. The most obvious relative bias reduction is in scenario (4), where the outcome model is the most complicated. This demonstrates the advantage of BART in adapting to increasing complexity, without the need for explicit specification of the functional forms. Across all scenarios under the non-linear data-generating processes, the BART-based methods also consistently yield a lower RMSE and MCSD than BLR-based methods, which indicates that BART-ICE and BART-PS-ICE provide more precise estimates with less variability compared to their fully parametric counterparts. When data are generating from a simpler linear model (1), BLR-based methods perform well, as expected, but BART-based methods still maintain similar bias and nominal frequentist coverage, with some inflation of MCSD and RMSE.

Turning our focus to the frequentist nonparametric methods, we find that BSpline-ICE and BSpline-PS-ICE perform well under scenario (1) and (2), but have less favorable performance under more complex data-generating processes (3) and (4) with slightly higher variability and notable under-coverage. Furthermore, RF-ICE and KR-ICE show poor performance across all data-generating processes, reflected by large relative bias, low coverage, and high RMSE. The relative bias of RF-based and KRbased methods also appears to be highly sensitive to the data-generating process. Notably, these methods perform particularly poorly under simpler data-generating processes, and the inclusion of longitudinal propensity scores can even introduce additional distortions in bias and coverage for these estimators, especially in scenarios (3)–(4). Despite having small MCSD in some scenarios, RF-based and KR-based methods often fail to provide unbiased inference and can show inflated RMSE. Among the frequentist methods, the BSpline-based approaches demonstrate the best overall performance. However, despite their relative advantage within the frequentist methods we have compared, BSpline-based estimators are still dominated by BART-based estimators regarding efficiency and frequentist coverage. Overall, our simulation results demonstrate the superior performance of BART in terms of accuracy, efficiency, and robustness, particularly under complex data-generating processes. Moreover, under data-generating processes with increased complexity, including the estimated longitudinal propensity scores as additional covariates appears to have improved the accuracy for BART-based methods, as indicated by the smaller relative bias and closer to nominal coverage, although at a slight cost of efficiency; this observation is consistent with prior findings in Hu et al. ³³ for BART survival estimators targeting conditional average treatment effects under a time-fixed treatment setting. Finally, while integrating longitudinal propensity scores may enhance the performance of BART-based methods, its benefits does not seem to carry to any of the frequentist methods across all data-generating processes.

6 |. ILLUSTRATIVE DATA EXAMPLES

6.1 |. Application to the Yale New Haven Health System Electronic Health Record data

We first demonstrate our methods to analyze data from the Yale New Haven Health System (YNHHS) Electronic Health Record. We examine the effect of prescribed antihypertensives on blood pressure among patients who developed severe inpatient hypertension. ⁶⁵ The data includes inpatients not initially admitted for hypertension but who later developed severe inpatient hypertension, systolic blood pressure (SBP) $>180\mathrm{m}\mathrm{m}\mathrm{H}\mathrm{g}$ or diastolic blood pressure (DBP) $>110\mathrm{m}\mathrm{m}\mathrm{H}\mathrm{g}$. ⁶⁵ We applied all four Bayesian ICE methods to analyze the YNHHS data from this cohort study. The cut of the data follows that in Cheng et al.⁴³ The study spanned from 2016 to 2020 and involved adult patients across five hospitals in New Haven, CT. The primary outcome is the blood pressure response within six hours after the development of severe hypertension. We aim to estimate the treatment effect of antihypertensive drug at (0, 2), [2, 4), and [4, 6) hours following the development of severe hypertension (HTN) on patients’ SBP at 6 hours following the development of severe HTN. The initial time interval (0 to 2 hours), the intermediate interval (2 to 4 hours), and the final interval (4 to 6 hours), are denoted as $k=\mathrm{1,2},3$ respectively. During each time interval, the administration of antihypertensive drug treatment, represented by $A_k$, is encoded as 1 if treatment was administered and 0 otherwise. The baseline covariates, denoted as ${\mathit{L}}_1$, include a range of demographic and clinical factors such as age, gender, race, body mass index, existing comorbidities with a focus on cardiovascular and hypertension history, and initial systolic and diastolic blood pressure (SBP and DBP) measurements at the development of severe HTN. Furthermore, the time-varying covariates, represented by ${\mathit{L}}_2$ and ${\mathit{L}}_3$, capture the most recent SBP and DBP measurements prior to 2 and 4 hours respectively. The primary outcome of interest is the SBP at 6 hours after developing severe HTN. Following Cheng et al.⁴³, our analysis includes 5, 925 individuals, specifically selected for having complete data across all relevant variables.

At time 3, we generated $S_3=4000$ total MCMC samples, with $B_3=3000$ as burn-in and a thinning rate of $t_3=10$. At time 2, we generated $S_2=4000$ total MCMC samples, with $B_2=3000$ as burn-in and a thinning rate of $t_2=100$. For each retained sample at the previous time point, the number of MCMC draws at time 1 was $S_1=4000$, with $B_1=3000$ discarded as burn-in and a thinning rate of $t_1=200$. Following Zhou et al., ⁴⁵ a time-1 thinning rate of $t_1=200$ is chosen to be adequate, as 5 samples are preserved for each retained chain. Through traceplots of the MCMC sample, we confirm the algorithm’s convergence, ensuring that the chains are well-mixed and accurately represent the posterior distribution. If computational resources are sufficient, increasing $C_3$ and $C_2$ can be explored to improve the sampling process. Applying four methods, the results are summarized in Table 4. Using BART-PS-ICE estimator and five cores for parallel computation, each analysis including all three estimands required approximately 104.31 hours to complete due to the non-trivial sample size. Using five cores and BART-ICE estimator, each analysis required approximately 103.62 hours to complete. Notice that the computation time is based on imputing all potential outcomes across all time points, similar to the scheme in Table 2, if the interest lies in one estimand only, using the restricted imputation scheme in Table 3 could further reduce the computational time.

Table 4.

Analysis of the Yale New Haven Health System (YNHHS) Electronic Health Record data: estimation results for average treatment effects comparing regimen (1, 1, 1) with (0, 0, 0), (0, 1, 1) with (0, 0, 0), and (0, 0, 1) with (0, 0, 0), using BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE. “ATE” denotes average treatment effect comparing two regimens.

ATE	Method	Estimated ATE	95% Credible Interval
${\tau }_{(\mathrm{1,1,1}),(\mathrm{0,0,0})}$	BLR-ICE	−4.377	(−6.589, −2.349)
	BLR-PS-ICE	−2.752	(−5.398, −0.252)
	BART-ICE	−3.458	(−6.229, −0.774)
	BART-PS-ICE	−3.463	(−6.231, −0.737)
${\tau }_{(\mathrm{0,1,1}),(\mathrm{0,0,0})}$	BLR-ICE	−4.932	(−6.869, −3.333)
	BLR-PS-ICE	−3.638	(−5.981, −1.573)
	BART-ICE	−3.901	(−6.365, −1.728)
	BART-PS-ICE	−3.497	(−5.816, −1.208)
${\tau }_{(\mathrm{0,0,1}),(\mathrm{0,0,0})}$	BLR-ICE	−3.444	(−4.860, −2.004)
	BLR-PS-ICE	−2.819	(−4.109, −1.083)
	BART-ICE	−3.010	(−4.625, −1.485)
	BART-PS-ICE	−2.760	(−4.468, −0.674)

We clarify that the estimands of scientific interest pertain to the sustained effects of antihypertensive exposure over time, as the drug remains active, while discontinuous treatment sequences are of less clinical interest. There are three different estimand of interest. The estimand ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ denotes the treatment effect for receiving antihypertensive drug treatment at all three time intervals, (0, 2), [2, 4), and [4, 6) hours following the development of severe HTN, as compared to not receiving any antihypertensive treatment during these time intervals. The estimand ${\tau }_{(\mathrm{0,1},1),(\mathrm{0,0},0)}$ represents the treatment effect for cases where treatment was not administered in the (0,2) hour period, but was received during the [2, 4) and [4, 6) hours following the development of severe HTN, as compared to not receiving any antihypertensive treatment during these time intervals. The estimand ${\tau }_{(\mathrm{0,0},1),(\mathrm{0,0},0)}$ represents the treatment effect for cases where antihypertensive drug treatment was not administered in the initial and intermediate time intervals but did in the final time interval, [4, 6) hours following the development of severe HTN, as compared to not receiving any antihypertensive treatment during all three time periods. The point estimates of ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ are negative across all four methods, indicating that the use of antihypertensive treatment at all three time intervals can reduce SBP among patients who developed severe inpatient hypertension. The estimated values of ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ range from −2.752 to −4.377. All blood pressure reductions are expressed in mmHg. The point estimates of ${\tau }_{(\mathrm{0,1},1),(\mathrm{0,0},0)}$ are all negative, ranging from −3.497 to −4.932, which indicates effective SBP reduction when treatment is initiated at 2 hours after the development of severe HTN. The estimated values for ${\tau }_{(\mathrm{0,0},1),(\mathrm{0,0},0)}$ are consistently negative, ranging from −2.760 to −3.444, suggesting that even a late intervention can effectively reduce SBP. The 95% credible intervals for the estimates of ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)},{\tau }_{(\mathrm{0,1},1),(\mathrm{0,0},0)}$, and ${\tau }_{(\mathrm{0,0},1),(\mathrm{0,0},0)}$ exclude the null. Therefore, the antihypertensive treatment activated at different time points after the development of severe HTN can significantly reduce the SBP.

The Bayesian ICE methods employed provide a range of estimated treatment effects and the corresponding credible intervals, confirming the effectiveness of the antihypertensive drug treatment. The negative estimated average treatment effect values across different time-varying treatment regimens suggest that antihypertensive drug treatment carried out at each time period is beneficial in reducing patients’ SBP at 6 hour following the development of severe HTN. From the results, we also observe that the lengths of the 95% credible intervals are relatively similar across all four methods. The BART-based approaches (with and without incorporating estimated longitudinal propensity scores) show more consistent results compared to the BLR-based approach in terms of point estimates and 95% credible intervals. The BLR-ICE method without incorporating propensity scores differs most notably from the other three methods, likely due to misspecification of the working model by linear regression.

6.2 |. Application to National Heart Lung and Blood Institute Growth and Health Study data

As an additional illustration, we apply our Bayesian ICE methods to analyze the data from the National Heart Lung and Blood Institute Growth and Health Study. We focus on understanding how disparities in diet, physical activity, and socioeconomic factors may contribute to obesity in pre-adolescent girls of different ethnic backgrounds. The research included African-American and Caucasian girls, aged 9 and 10, during 1987 and 1988, recruited from Richmond, CA, and Cincinnati, OH, as well as from families enrolled in a health maintenance organization in Washington, D.C.²⁴ The study conducted follow-up assessments annually over 9 years. This analysis utilizes the initial three periods data of the study, investigating how dietary habits impact Body Mass Index (BMI) by the fourth period. We quantify dietary intake by daily percentage of energy from saturated fats reported in the questionnaire of each time period. Following previous studies^24,66,67, if this percentage is under 10 at time period $k$, we assign a value of 1 to the diet variable $A_k,k=\mathrm{1,2},3$; otherwise, it is set to 0. The outcome variable $Y$ is the logarithm of BMI measured at the fourth time period. The baseline covariates, denoted as ${\mathit{L}}_1$, include race (1 for Caucasian, 0 for African-American), household income, and logarithm of a physical activity score recorded at the first time period. Furthermore, at the second time period, the time-varying covariates, represented by ${\mathit{L}}_2$, is logarithm of BMI recorded at $k=2$. At the third time period, ${\mathit{L}}_3$ include logarithm of BMI and logarithm of physical activity score recorded at $k=3$.

We use all four Bayesian methods, BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE, to analyze the longitudinal effect of dietary intake (daily percentage of energy from saturated fats) on BMI of pre-adolescent girls from various ethnic backgrounds. The results are summarized in Table 5. Given the large dataset, we generated 4000 total MCMC samples with 3000 as burn-in at each time point. Specifically, at time 3, the thinning rate was $t_3=10$; at time 2, it was $t_2=100$; and at time 1, it was $t_1=200$. This ensures that 5 samples are preserved for each retained chain at time 1. Using BART-PS-ICE estimator and one core for computation, each analysis including all three estimands required approximately 85.05 hours to complete. Using BART-ICE estimator, each analysis required approximately 90.41 hours to complete. Again, the computation time is based on imputing all potential outcomes across all time points, similar to the scheme in Table 2.

Table 5.

Analysis of the National Heart Lung and Blood Institute Growth and Health Study data: estimation results for average treatment effects comparing regimen (1, 1, 1) with (0, 0, 0), (0, 1, 1) with (0, 0, 0), and (0, 0, 1) with (0, 0, 0) using BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE. “ATE” denotes average treatment effect comparing two regimens.

ATE	Method	Estimated ATE	95% Credible Interval
${\tau }_{(\mathrm{1,1,1}),(\mathrm{0,0,0})}$	BLR-ICE	−0.027	(−0.054, 0.000)
	BLR-PS-ICE	0.038	(−0.789, 0.944)
	BART-ICE	−0.037	(−0.072, −0.002)
	BART-PS-ICE	−0.054	(−0.273, 0.165)
${\tau }_{(\mathrm{0,1,1}),(\mathrm{0,0,0})}$	BLR-ICE	−0.005	(−0.027, 0.018)
	BLR-PS-ICE	0.048	(−0.035, 0.125)
	BART-ICE	−0.009	(−0.029, 0.012)
	BART-PS-ICE	−0.047	(−0.206, 0.103)
${\tau }_{(\mathrm{0,0,1}),(\mathrm{0,0,0})}$	BLR-ICE	0.005	(−0.010, 0.021)
	BLR-PS-ICE	−0.040	(−0.088, 0.005)
	BART-ICE	0.001	(−0.011, 0.016)
	BART-PS-ICE	−0.053	(−0.147, 0.018)

There are three different estimands of interest. The estimand ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ represents the effect of having daily percentage of energy from saturated fats consistently lower than 10% across all three time intervals, as compared to having daily percentage of energy from saturated fats consistently greater than 10% across all time intervals. The estimand ${\tau }_{(\mathrm{0,1},1),(\mathrm{0,0},0)}$ represents the effect of reducing daily energy intake from saturated fats to below 10% starting from the second time interval. The estimand ${\tau }_{(\mathrm{0,0},1),(\mathrm{0,0},0)}$ represents the effect of reducing daily energy intake from saturated fats to below 10% in the final time interval. The point estimate of ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ ranges from −0.054 to 0.038. Focusing on BART-PS-ICE method, the point estimate for ${\tau }_{(\mathrm{1,1},1),(\mathrm{0,0},0)}$ stands at −0.054. This indicates that maintaining a daily intake of energy from saturated fats below 10% across all three time periods can effectively lead to a reduction in BMI; the estimates and findings for ${\tau }_{(\mathrm{0,1},1),(\mathrm{0,0},0)}$ and ${\tau }_{(\mathrm{0,0},1),(\mathrm{0,0},0)}$ are generally similar, but all credible intervals based on BART-PS-ICE include zero. The results overall suggest that dietary interventions that reduce saturated fat intake may have effects in reducing the BMI, even when implemented in later time points.

The variability in estimates and their associated credible intervals given by different methods can be attributed to methodological differences between the models, for example, their capability of handling non-linearity. In terms of the point estimate, the four methods provided roughly similar results, all of which are close to zero. For all estimands of interest in this data example, we find that incorporating estimated longitudinal propensity scores further widens the 95% credible interval. Based on our simulation findings, it is possible that the outcome surface is approximately linear. In this case, adding propensity scores as an additional covariate may inflate variance, similar to the simulation scenario (1).

7 |. DISCUSSION

To estimate the average causal effect of a treatment regimen in longitudinal observational studies, the ICE estimator can be an attractive version of the g-formula because it avoids the need for specifying the full distribution of all time-varying confounders, as in the standard g-formula implementation.¹⁰ This work offers a version of the ICE estimator under a Bayesian framework and further integrates nonparametric priors—BART—to flexibly model the series of outcome surfaces and improves the robustness. We presented the sampling algorithm to obtain samples from the posterior distribution of the Bayesian ICE estimator under nested regressions. Compared with BLR-based ICE estimator and competing frequentist nonparametric regression-based estimators, we demonstrated via simulations that BART-based ICE estimator can provide stronger protection against model misspecification, and often achieve closer to nominal frequentist coverage, under different data-generating processes.

We have considered two versions of the BART-ICE estimator—one including only the original time-varying confounders and one additionally including the nonparametrically estimated longitudinal propensity scores as additional confounders. The idea of adding longitudinal propensity scores in the regression-based causal effect estimators is not uncommon, especially in the setting of a time-fixed treatment. Specifically, Zigler et al. ⁵⁴ and Li et al. ⁴² have explained that a Bayesian regression estimator with the original confounders augmented by the estimated propensity score implicitly conducts outcome regression with strata defined by the propensity score, and can be viewed as the Bayesian analogue of a doubly robust estimator (even though this approach does not strictly possess the double robustness asymptotic property in the frequentist sense). Previous studies have exploited the additional value of adjusting for propensity scores in the regression setting. For example, Hahn et al. ³⁴ developed the Bayesian causal forest that integrates the estimated propensity scores as part of the model specification with a time-fixed treatment, to address regularization-induced confounding. Hu et al. ³³ demonstrated the benefit of integrating the estimated propensity scores for estimating heterogeneous treatment effects with a time-fixed treatment and censored survival outcomes, and found that adding a nonparametrically estimated propensity score as an additional fixed covariate in the accelerated failure time model formulation can further improve the frequentist coverage. With a time-varying treatment, Chen et al. ¹² demonstrated a slight efficiency advantage of including the estimated longitudinal propensity scores in the BART-based NICE estimator for causal survival analysis in discrete time. These prior findings across various contexts have motivated us to explore the use of nonparametrically estimated longitudinal propensity scores within the BART-ICE framework. Our results suggest that under more complex data-generating processes, augmenting BART-ICE with estimated longitudinal propensity scores can further reduce bias and improve frequentist coverage. However, in simpler data-generating processes, this augmentation may slightly inflate variance, likely due to over-specification of the outcome model. Thus, the benefits of incorporating estimated longitudinal propensity scores in BART-ICE appear to depend on the complexity of the data-generating process. In principle, traditional model checking tools may provide insight into assessing model fit to the data when evaluating model and prior distribution choices. Methods such as Deviance Information Criterion (DIC) or Widely Applicable Information Criterion (WAIC) could be useful in selecting the best model. However, the development of specific model checking methods requires future research, particularly given the complexity introduced by the imputation of pseudo-outcomes at each stage using nested regression.

In our reflection, although Bayesian methods have merits for causal inference, ⁴² they are relatively less studied in the context of time-varying treatments. Keil et al. demonstrated that the NICE approach for g-formula is amenable to a Bayesian approach and illustrated the frequentist properties of the Bayesian NICE estimator, which improves the accuracy of causal effect estimates in small samples or when data are sparse. ¹⁰ Different from their work, our work focuses on ICE estimator and develops a Bayesian sampling algorithm to obtain samples from its posterior distribution. Compared to Bayesian NICE, the ICE approach requires fewer model assumptions, as it does not require specifying models for the joint distribution of observed variables. This characteristic offers some convenience and potential protection against model misspecification. However, it is important to recognize that the backward iterative nature of the ICE method can also result in an increase in computational time, especially for longitudinal studies with a large number of periods; so there is an implicit trade-off when choosing between different gformula methods and this decision should depend on the study context, model complexity, and computational resources. Our primary goal is to provide a Bayesian version of the g-formula that obviates the need to specify the potentially complex time-varying covariates and offer an additional option when researchers are interested in implementing flexible g-computation with time-varying treatments. In our experiences, for longitudinal observational studies with a moderate number of time-varying covariates but a limited number of time periods, Bayesian ICE would be recommended, thereby minimizing the modeling efforts for possibly a handful of covariates and improving the model robustness to confounding distribution specifications. Conversely, in studies with a large number of periods but only a few time-varying covariates, the Bayesian NICE estimator could have a computational advantage, with some thoughtful efforts in specifying the joint likelihood for the entire time-varying confounders. Finally, as we demonstrated via simulations, it would be beneficial to consider more flexible specification of the component models—such as BART—in either version of the g-formula estimator to further improve the operating characteristics of the final average causal effect estimator.

Our work, like other prior studies on longitudinal causal inference, has primarily relied on the key identification assumptions of no unmeasured confounding (sequential ignorability) and positivity. For observational studies with either a time-fixed or time-varying treatment, the no unmeasured confounding assumption may not always hold in practice, and in such cases, the BART-ICE methods (despite its flexibility in representing outcome surfaces) are not guaranteed to produce valid estimates for the average causal effects; also see, for example, Section 4.6 in Hu et al.³³ for an empirical demonstration of bias that cannot be avoided by flexible BART causal estimators even in the presence of unmeasured baseline confounding. When unmeasured confounding is suspected in longitudinal observational studies, a useful approach is to incorporate sensitivity functions. In this regard, Cheng et al.⁴³ extended the confounding function approach originally proposed by Brumback et al.⁶⁸ to a generalized version of the ICE estimator in the context of estimating quantile causal effects. Their version of the sensitivity function can be adapted to our setting, and enable the development of a BART-based sensitivity analysis framework, following the example in Hu et al.³¹ for multi-valued time-fixed treatments. A comprehensive development of this approach would be a valuable direction for future work. Second, the positivity assumption could also be challenging to satisfy in longitudinal observational studies, particularly when the number of time intervals is large and the sample size is limited. While various strategies have been proposed to address positivity violations in the context of time-fixed treatments,^69,70 their extensions to time-varying treatments remain less developed. In a Bayesian setting, Papadogeorgou and Li⁷¹ highlighted that in regions of poor covariate overlap, BART may overstate the certainty of causal effect estimates, and recommended Gaussian process priors as a potential remedy. Zhu et al.⁴⁰ further explored this approach and systematically investigated the use of Gaussian processes for addressing positivity violations with a time-fixed treatment. Integrating Gaussian process into the g-formula estimator under the time-varying treatment setting, particularly under both NICE and ICE formulations, represents an important avenue for future research that improves the quality of time-varying causal effect estimation under positivity violation. Third, our current Bayesian ICE implementation adheres to the theoretical underpinnings of nested imputation.⁴⁵ However, we acknowledge that this approach can be computationally intensive due to the need to fit multiple Bayesian models per each draw from the previous time point. An anonymous reviewer suggested an alternative approach, which aggregates all draws from time 2 and fits a single model at time 1, thereby reducing the computational time. Assessing the theoretical justification and comparative performance of this approach against our current method, as well as developing more computationally efficient implementations of the Bayesian ICE method more generally, would be valuable directions for future research.

APPENDIX 1. THE ITERATIVE EXPECTATION OF THE COUNTERFACTUAL OUTCOMES

By definition, $f_k\left({\bar{a}}_K,{\bar{\mathit{l}}}_k\right)\equiv E\left(f_{k+1}\left({\bar{a}}_K,{\bar{\mathit{l}}}_{k+1}\right)\mid {\bar{A}}_k={\bar{a}}_k,{\bar{\mathit{L}}}_k={\bar{\mathit{l}}}_k\right)$, for $k\in \{1,\dots ,K-1\}$. For $k=K$, we have the following by consistency assumption

$$f_K\left({\overline{a}}_K,{\overline{\mathit{l}}}_K\right)=E\left(Y|{\overline{A}}_K={\overline{a}}_K,{\overline{\mathit{L}}}_K={\overline{\mathit{l}}}_K\right)=E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_K={\overline{a}}_K,{\overline{\mathit{L}}}_K={\overline{\mathit{l}}}_K\right).$$

For $k=K-1$,

$$\begin{gathered} f_{K-1}\left({\overline{a}}_K,{\overline{\mathit{l}}}_{K-1}\right)=E\left(f_K\left({\overline{a}}_K,{\overline{\mathit{l}}}_K\right)|{\overline{A}}_{K-1}={\overline{a}}_{K-1},{\overline{\mathit{L}}}_{K-1}={\overline{\mathit{l}}}_{K-1}\right) \\ =E\left(E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_K={\overline{a}}_K,{\overline{\mathit{L}}}_K={\overline{\mathit{l}}}_K\right)|{\overline{A}}_{K-1}={\overline{a}}_{K-1},{\overline{\mathit{L}}}_{K-1}={\overline{\mathit{l}}}_{K-1}\right) \\ =E\left(E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_{K-1}={\overline{a}}_{K-1},{\overline{\mathit{L}}}_K={\overline{\mathit{l}}}_K\right)|{\overline{A}}_{K-1}={\overline{a}}_{K-1},{\overline{\mathit{L}}}_{K-1}={\overline{\mathit{l}}}_{K-1}\right) \\ (\text{By no unmeasured confounding}.) \\ =E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_{K-1}={\overline{a}}_{K-1},{\overline{\mathit{L}}}_{K-1}={\overline{\mathit{l}}}_{K-1}\right) \\ (\text{By law of iterated expectation}.) \end{gathered}$$

Assuming $f_{k+1}\left({\bar{a}}_K,{\bar{\mathit{l}}}_{k+1}\right)=E\left(Y\left({\bar{a}}_K\right)\mid {\bar{A}}_{k+1}={\bar{a}}_{k+1},{\bar{\mathit{L}}}_{k+1}={\bar{\mathit{l}}}_{k+1}\right)$ holds for $k=1,\dots ,K-1$. Then,

$$\begin{gathered} f_k\left({\overline{a}}_K,{\overline{\mathit{l}}}_k\right)=E\left(f_{k+1}\left({\overline{a}}_K,{\overline{\mathit{l}}}_{k+1}\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right) \\ =E\left(E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_{k+1}={\overline{a}}_{k+1},{\overline{\mathit{L}}}_{k+1}={\overline{\mathit{l}}}_{k+1}\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right) \\ =E\left(E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_{k+1}={\overline{\mathit{l}}}_{k+1}\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right) \\ =E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right) \end{gathered}$$

Therefore, by induction, we can show that, for $k=1,\dots ,K$,

$$f_k\left({\overline{a}}_K,{\overline{\mathit{l}}}_k\right)=E\left(Y\left({\overline{a}}_K\right)|{\overline{A}}_k={\overline{a}}_k,{\overline{\mathit{L}}}_k={\overline{\mathit{l}}}_k\right).$$

Thus, $f_0\left({\bar{a}}_K\right)=E\left(f_1\left({\bar{a}}_K,{\mathit{l}}_1\right)\right)=E\left(E\left(Y\left({\bar{a}}_K\right)\mid {\bar{A}}_1={\bar{a}}_1,{\overline{\mathit{L}}}_1={\overline{\mathit{l}}}_1\right)\right)=E\left(Y\left({\bar{a}}_K\right)\right)$.

APPENDIX 2. MONTE CARLO STANDARD DEVIATION ACROSS DIFFERENT ESTIMATION METHODS AND SCENARIOS

Figure.

Monte Carlo Standard Deviation (MCSD) across all simulated datasets for ten methods under four data-generating processes. Each panel represents a different data-generating process. Bar heights indicate MCSD values, measuring the variability of parameter estimates across repeated simulations. Bayesian methods include BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE, while frequentist methods include BSpline-ICE, BSpline-PS-ICE, RF-ICE, RF-PS-ICE, KR-ICE, and KR-PS-ICE.

APPENDIX 3. ROOT MEAN SQUARED ERROR ACROSS DIFFERENT ESTIMATION METHODS AND SCENARIOS

Figure.

Root Mean Squared Error (RMSE) across all simulated datasets for ten methods under four data-generating processes. Each panel represents a different data-generating process, with bayesian methods and frequentist methods shown for comparison. Bar heights indicate RMSE values, measuring the average squared deviation from the true average treatment effect. Bayesian methods include BLR-ICE, BLR-PS-ICE, BART-ICE, and BART-PS-ICE, while frequentist methods include BSpline-ICE, BSpline-PS-ICE, RF-ICE, RF-PS-ICE, KR-ICE, and KR-PS-ICE.

DATA AVAILABILITY STATEMENT

The data used in Section 6.1 were obtained with the assistance and supervision of the Clinical and Translational Accelerator (CTRA), Department of Medicine, Yale School of Medicine and the Joint Data Analytics Team. This research data are not publicly available due to privacy or ethical restrictions. The data used in Section 6.2 were prepared using the Research Materials (NHLBI Growth and Health Study) obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC) and do not necessarily reflect the opinions or views of the NHLBI Growth and Health Study or the NHLBI. The NHLBI Growth and Health Study data is available from the NHLBI BioLINCC website and can be requested at https://biolincc.nhlbi.nih.gov/studies/nghs/.