Bayesian nonparametric trees for principal causal effects

ABSTRACT

Principal stratification analysis evaluates how causal effects of a treatment on a primary outcome vary across strata of units defined by their treatment effect on some intermediate quantity. This endeavor is substantially challenged when the intermediate variable is continuously scaled and there are infinitely many basic principal strata. We employ a Bayesian nonparametric approach to flexibly evaluate treatment effects across flexibly modeled principal strata. The approach uses Bayesian Causal Forests (BCF) to simultaneously specify 2 Bayesian Additive Regression Tree models; one for the principal stratum membership and one for the outcome, conditional on principal strata. We show how the capability of BCF for capturing treatment effect heterogeneity is particularly relevant for assessing how treatment effects vary across the surface defined by continuously scaled principal strata, in addition to other benefits relating to targeted selection and regularization-induced confounding. The capabilities of the proposed approach are illustrated with a simulation study, and the methodology is deployed to investigate how causal effects of power plant emissions control technologies on ambient particulate pollution vary as a function of the technologies’ impact on sulfur dioxide emissions.

1. INTRODUCTION

Many scientific questions relate to how causal effects of a treatment on a primary outcome relate to the treatment’s effect on some intermediate quantity. Complications to causal inference in such settings are well recognized, and have motivated a variety of approaches. In particular, principal stratification (Frangakis and Rubin, 2002) formulates how causal effects of a treatment on an outcome vary as a function of the treatment’s effect on the intermediate. This has proven useful in, among other problems, evaluating surrogate endpoints (Gilbert and Hudgens, 2008), treatment noncompliance in randomized studies (Frumento et al., 2012), and measuring varying local response to a national regulation (Zigler et al., 2018).

Treatment effect heterogeneity is at the center of principal stratification analyses, as the key goal is evaluating whether causal treatment effects on an outcome vary across “principal strata” defined by the treatment effect on the intermediate. Owing to the structure of potential outcomes, which are only observed for each unit under that unit’s observed treatment status, units’ membership in principal strata is unknown, with allocation of units to these strata a central analysis challenge. When the intermediate takes on continuous values, there are infinitely many principal strata, and the problem is complex. Early work in Jin and Rubin (2008) offered a Bayesian parametric approach for continuous intermediate variables, which was extended in Schwartz et al. (2011) to consider a nonparametric Dirichlet process mixture (DPM) for the intermediate outcomes to flexibly model the principal strata. Kim et al. (2019) further extended the flexibility of a DPM to model both the distributions of potential intermediate and outcome variables, using a Gaussian copula model to combine all marginal distributions into a coherent joint distribution. This approach facilitated the estimation of the potentially complex joint distribution of intermediate and outcome variables but, owing to the assumption of normality required in using the Gaussian copula model, the marginal distributions of potential intermediate and outcome variables remained linearly connected. Thus, even with extensions to leverage Bayesian nonparametric principal stratification analysis, potentially important relationships among continuous intermediates and continuous primary outcomes remain.

We continue the development of flexible Bayesian nonparametric principal stratification analysis with continuous intermediate and outcome variables to relax the parametric restrictions that remain in existing work in Jin and Rubin (2008), Kim et al. (2019), and Schwartz et al. (2011). Specifically, we offer an approach that utilizes Bayesian Additive Regression Trees (BART, Chipman et al., 2010) to flexibly model both intermediate and outcome variables. Methods based on BART have been extensively studied in the causal inference community since Hill (2011), owing to their promising predictive performance and ability to accommodate complex interactions (Hill et al., 2020; Tan and Roy, 2019). Recent work has applied BART in the context of principal stratification. Ren et al. (2021) use BART to estimate membership in each of the 4 principal strata defined by a binary intermediate outcome. Schnell et al. (2022) applied BART to a marginal outcome model to estimate causal effects conditional on the intermediate variable in the treatment arm. Beyond the general goal of incorporating model flexibility and providing a unified estimating procedure for continuous intermediate and outcome variables, we adopt an approach based on BART for reasons specific to the goal of principal stratification. Since the goal of principal stratification is to evaluate how causal effects of the treatment on the outcome vary across principal strata, we utilize the BART-derived approach of Bayesian Causal Forests (BCF, Hahn et al., 2020) for its targeted ability to capture flexible forms of treatment effect heterogeneity, in addition to its ability to adjust for complex confounding structures. We propose a joint model that specifies separate BCF models for continuous intermediate values and primary outcomes, designed specifically to (1) characterize complex principal strata structures and (2) use the capacity of BCF to capture how treatment effects vary across these strata.

The proposed methodology is illustrated with an investigation of emission-reduction strategies installed at coal-fired power plants in the United States. Such strategies have played a significant role in the marked reduction of air pollution and corresponding health burden associated with coal power generation in the United States (Henneman et al., 2023). The specific treatment of interest is whether a coal-fired electricity generating unit (EGU) has a flue-gas desulfurization scrubber installed to reduce emissions of sulfur dioxide (SO) a major byproduct of coal combustion. Interest lies in the extent to which the causal effect of such scrubbers on ambient fine particulate pollution (PM, outcome) varies as a function of the scrubber’s impact on SO emissions (intermediate). The relationship between scrubbers and SO is well understood and represents a clear link between the treatment and the intermediate. SO is also a known precursor to the formation of ambient PM, although this relationship is dictated by complex physical and chemical relationships that can be difficult to characterize with data. Thus, we consider a case where known relationships present a priori expectations about the relationships between the treatment and the intermediate and between the intermediate and the outcome, but where there is considerable uncertainty in how the complexity of the latter relationship can be used to understand varying effectiveness of the treatment on the primary outcome. A more complete understanding of the effectiveness of established control strategies, particularly as overall levels of this type of pollution have drastically reduced in the United States, is essential and represents a needed advance over similar investigations conducted during earlier time periods (Kim et al., 2019, 2020).

2. MODEL AND METHODS

2.1. Notation and causal estimands

Let denote the exposure, intermediate, and outcome variables of interest, and be a set of confounders for observation , all conceived as fixed pre-treatment features. Throughout the paper, we assume the binary exposure. With the stable unit treatment value assumption (SUTVA, Rubin, 1980), let and denote the potential values of the intermediate and outcome variables that would be observed under the exposure for subjects , respectively. The exposure () in the motivating air quality study is the presence of a flue-gas desulfurization scrubber (henceforth, “scrubber”), measured at each coal-fired power plant in 2014; the intermediate variable () is the annual average SO emissions from the power plant during 2014; and the outcome of interest is the annual ambient average PM concentration within a 50 km radius of the power plant (). and are the potential SO emissions and surrounding PM values based on the state of scrubber installation () at the th power plant.

Principal stratification partitions the population based on how affects the intermediate variable, encoded as membership in a principal stratum defined as . This principal stratum membership is a latent variable that is unaffected by exposure and can be considered the baseline characteristic of the th subject, encoding features that dictate how the primary treatment impacts the intermediate variable. In the power plant case, denotes a combination of the effectiveness of scrubbers for reducing SO as well as other plant characteristics dictating variability in this effect that may also relate to the formation of ambient PM such as patterns of operation, responses to regional and seasonal regulatory programs, or atmospheric and weather conditions surrounding the plant. Our goal is to examine causal effects of on across principal strata defined by different values of , known as principal causal effects (PCEs, Frangakis and Rubin, 2002).

PCEs can be described as a surface of across values of , which has been termed the “causal effect predictiveness” surface (Gilbert and Hudgens, 2008). Summarizing this surface can be challenging with continuously scaled , and deriving summary estimands will depend on context. Expected dissociative effects have been proposed and used to summarize average causal effects of on among principal strata for which has little or no impact on , representing something similar to a “direct effect” of on when remains unaffected. Expected associative effects can summarize average effects of on across strata, where . While the entire CEP surface can be considered the target estimand, we extend the work in Kim et al. (2020) to summarize the surface with multiple average PCEs based on predetermined threshold vectors, , and define the expected PCEs as follows:

where different values of define PCEs among strata with varying impacts of on via a specified function . For example, assesses heterogeneity through the difference between 2 intermediate values, while evaluates changes in the PCE based on the ratio of these values. Although various functions could be applied, our simulation studies focus on the difference (Scenarios I and II) and the ratio (Online Appendix I) to capture effect heterogeneity. For instance, setting and using would correspond to an expected dissociative effect among strata of power plants where scrubbers have minimal or no impact on SO emissions. If we consider the sample average estimand as the target estimand, the expression for is as follows:

where and is an indicator function.

Note that the fundamental problem of causal inference applies to both and . Thus, the principal stratum membership is unobserved and quantities such as are not identified based only on observed data. Unobserved values will determine the value of and must be estimated along with the unobserved values of .

2.2. Identifying assumptions

To estimate the PCEs, we first formulate an assignment mechanism, the distribution of conditional on the potential (intermediate) outcomes and observed covariates where and for . Under the assumption that the exposure assignment for each unit is independent of information from other units, the exposure assignment we consider for unit has the following form: . Note that this assumption implies that treatment decisions are independent even among power plants owned by the same company or operating in the same region. The ignorability assumption (Rosenbaum and Rubin, 1983) states that with a set of covariates , the exposure assignment mechanism can be reduced as follows:

Assumption 1

Strongly ignorable treatment assignment (unconfoundedness)

for all . Equivalently, it can be restated as , where for .

Here, covariate vector includes all common causes (confounders) of and relationships. Ignorability implies that, unlike observed , principal stratum membership is not affected by the exposure, and average comparisons between and within levels of can be interpreted as (principal) causal effects. The importance of covariates in satisfying this assumption demands careful consideration in the power plant analysis, where contains power plant characteristics that impact scrubber installation decisions and SO  emissions, and population and meteorological variables related to formation of ambient pollution.

Our target estimands are sample average PCEs for the sample units. Unlike corresponding population average estimands, the parameters governing associations between and and and cannot be ignored (Ding and Li, 2018; Jin and Rubin, 2008; Kim et al., 2019; Li et al., 2023; Schwartz et al., 2011). In the following section, we apply the BCF approach, incorporating specific conditional independence assumptions to define 2 joint conditional distributions for and , which are sufficient for estimating the PCEs.

Assumption 2

Conditional independence

This assumption implies that when all relevant variables (confounders and additional predictors) influencing the potential (intermediate) outcomes are known, the observed (intermediate) outcome is independent of its corresponding counterfactual outcome. In our application, this implies that modeled relationships between these characteristics and SO emissions are independent of what SO emissions would have been with the opposite treatment. Additionally, with the inclusion of zip code-level and environmental variables, this further implies that the PM level (primary outcome) in each zip code region is conditionally independent of its counterfactual PM level. We evaluated sensitivity to violations of this assumption through a simulation study that introduced correlations between the potential outcomes, confirming that the conditional independence assumption did not hugely affect the results when all relevant covariates were considered (Online Appendix M).

Under these assumptions, we further factorize the joint conditional distributions of potential (intermediate) outcomes as and , where represents the parameter vector governing both the intermediate and outcome models. Here, is omitted from both sides of the equations based on the unconfoundedness assumption. Under SUTVA and Assumptions 1 and 2, is identified for each , which permits identification of . Then, is identified for each with missing data , which can be imputed. Finally, with Assumption 2, the joint conditional distribution of is identified.

2.3. BCF overview

Among many causal inference methods based on BART, Hahn et al. (2020) proposed the BCF model to accurately measure heterogeneous effects with the goal of minimizing confounding. Ignoring for the moment the presence of the intermediate variable, , the BCF method specifies the response surface as

(1)

where independent BART priors are specified on the functions and . Here, is the propensity score, which is the exposure assignment mechanism under Assumption 1.

A key point of the BCF specification is the role of the function, also known as the prognostic function, which is equivalent to the expected outcomes of the unexposed (ie, ). If the predicted value of the outcome in the absence of exposure relates to the assignment of exposure, the prognostic function will exhibit some dependence on the propensity score . This phenomenon is referred to as targeted selection (Hahn et al., 2020), which can present regularization-induced confounding (RIC). The inclusion of the estimated propensity score within the function is designed specifically to mitigate the threat of RIC, and is a key feature of the BCF model in (1). In the power plant setting, we expect targeted selection to arise because scrubbers are more likely to be installed in power plants emitting a significant amount of pollutants (SO) and in areas in danger of violating ambient air quality standards.

The other main component of BCF is its inclusion of the function , which is the feature of the model specifically targeted to a characterization of treatment effect heterogeneity. Specifically, a flexibly modeled would represent a complex function of covariates that modify the relationship between and the expected outcome.

2.4. Bayesian principal causal forest model specification

Our proposed BCF models of and (hereafter, Bayesian principal causal forest; BPCF) can be expressed using the following form of BCF:

where and . This essentially corresponds to specifying 2 BCF models; one for the intermediate, , and another for the outcome, , conditional on both potential intermediate values. Any unobserved intermediate variable is treated as missing data and imputed using a data augmentation algorithm during estimation. These 2 BCF models correspond to the 2 joint distributions described as consequences of Assumption 2 in Section 2.2: , and . Here, independent BART priors are specified for the functions , , , and , and the propensity score can be estimated separately and replaced with for each th subject, where is a subvector of the global parameter . Alternatively, the discussion section outlines a method for jointly fitting the observed data model and estimating the propensity score function. Key points to highlight include (1) the flexibility of the outcome and intermediate models in mitigating the threat of RIC while effectively capturing the causal effects of on both and ; (2) the reliance of the outcome model on potential intermediate variables and rather than an observed intermediate variable ; and (3) the importance of the function in the outcome model, which allows the treatment effect of on to be modified by a complex function of principal stratum membership and covariates . The simulation study compares against an approach that excludes from the specification of . In either case, estimated PCEs will marginalize over values of within principal strata defined by . The settings for the BART prior specified for each and function are detailed in Online Appendix A.

3. ESTIMATION

3.1. Posterior computation

The detailed Markov chain Monte Carlo (MCMC) approach for the posterior computation process is illustrated in the pseudo-code (Algorithm 1) in the Supplementary Materials. As shown in Algorithm 1, a total of 4 BART priors (2 for model and 2 for model) were updated using “Bayesian backfitting” (Hastie and Tibshirani, 2000) as a Metropolis-within-Gibbs sampler in which each tree is fit sequentially through the unexplained responses. The number of trees included in each BART prior is set differently, as suggested by Hahn et al. (2020). The BART priors assigned to the prognostic functions and that adjust for confounders consist of a large number of trees (eg, 200 trees), whereas the BART priors assigned to and representing the heterogeneous effects consist of a small number of trees that regularize toward the homogeneous effects (eg, 50 trees). The Supplementary Materials (Appendix H) present the results of a simulation study specifying different numbers of trees for the and functions. This also showed that specifying a relatively smaller number of trees for the function led to better performance.

To summarize the entire procedure, for the th MCMC iteration, the BART priors assigned to the function (prognostic function) and the function in the model are sequentially updated. Thereafter, for each , is generated, implying the value of . Similarly, the BART priors corresponding to the model are updated. The 2 potential intermediates, and (forming the principal stratum membership variable, ), along with , serve as independent variables for splitting the tree during the growth of the function. After updating all of the BART parameters, the th iteration is completed by updating the variance parameters with the Gibbs sampler. It is worth noting that the sampling for potential intermediate variables incorporated during the MCMC takes into account the densities for both the and models. Thus, when sampling , it is done conditionally based on the structure of the model, which includes the pair of potential intermediate variables. The code for posterior computation is available at the GitHub repository BPCF (2024).

3.2. Causal effect estimation

With posterior samples for each parameter and unobserved potential outcome, the posterior means of the PCEs for a specific principal stratum can be estimated as follows:

where and are the th MCMC samples of potential intermediate and outcome variables, respectively.

4. SIMULATION

We examine the model performance based on simulated datasets with observations. We focus here on 1 simulation designed to illustrate essential features, with a detailed description of additional simulations in the Supplementary Materials: (Scenario I) 7 confounders () are independently generated from for . , , and are confounded through (). In this scenario, confounding is strong and effect heterogeneity exists through interactions between in the model and between in the model. We generate simulated data under each scenario. The specific formulas used to generate the data are provided in the Supplementary Materials (Appendix C). The MCMC chain runs for 10 000 iterations, and the first 5000 iterations are discarded as burn-in.

The number of trees in our proposed model was set to 150 for the prognostic function ( and ) of the intermediate/outcome models and 50 for the modifier function ( and ) in the intermediate/outcome models. The results of specifying different numbers of trees for the prognostic/modifier functions are shown in Online Appendix Table S2. The results were similar as long as the number of trees for and was not too large (eg, 100). A node at depth splits with the probability for the prognostic function but with the probability for the modifier function to shrink more toward homogeneous effects. To compare our model performance to other methods, we consider 3 competing models: (1) 4 separate BART models for each of the 4 potential outcomes , which is denoted by “BART”; (2) DPM model from Schwartz et al. (2011), which is denoted by “DP”; and (3) the proposed BPCF without covariates in the modifier function of the outcome model (ie, the modifier function only contains ), which is denoted by “BPCF.” The specific forms of the BART and DP models are included in the Online Appendix.

4.1. Simulation I

In this simulation study, target PCEs were defined based on 5 different intervals. These intervals were determined using quintiles of observed intermediate values obtained through the data-generating process. Specifically, the first interval was (2.00, 2.26), the second interval was (2.26, 2.53), the third interval was (2.53, 2.84), the fourth interval was (2.84, 3.29), and the last interval was (3.29, 5.09). The results are shown in Table 1. Relative bias and MSE for the average PCEs were compared, and the proposed BPCF performed relatively better across all PCEs. In particular, it demonstrated very high accuracy in estimating the causal effect for intermediate variables , which aids estimation of the PCEs.

TABLE 1.

Simulation results (relative bias [rBias] and mean square error [MSE]) under Scenario I.

Category	True Effect	BPCF		BPCF		BART		DP
		rBias	MSE	rBias	MSE	rBias	MSE	rBias	MSE
E[M(1) −M(0)]	2.79	0.004	0.001	0.001	0.001	0.009	0.001	0.016	0.001
E[Y(1) − Y(0)| interval 1]	–4.54	0.15	0.53	0.39	3.54	0.30	1.94	0.59	10.33
E[Y(1) − Y(0)| interval 2]	–5.71	0.02	0.09	0.18	1.36	0.09	0.29	0.23	3.44
E[Y(1) − Y(0)| interval 3]	–7.18	–0.03	0.12	0.03	0.31	–0.05	0.19	−0.03	0.74
E[Y(1) − Y(0)| interval 4]	–9.28	–0.06	0.38	–0.10	1.19	–0.14	1.89	−0.25	5.59
E[Y(1) − Y(0)| interval 5]	–14.11	–0.10	2.42	–0.27	15.41	–0.22	10.72	−0.48	47.79

A causal effect predictiveness surface was created to help interpret the results of each model more precisely. Figure 1 depicts the predictiveness surfaces created using 4 different methods, including true surface plots. The plane has 2 axes representing and variables, and the -axis represents the corresponding value. The dots on the plane represent and sample values from one MCMC iteration. Based on this, it is reasonable to interpret that the BART and BPCF methods form more accurate principal strata than the DP method. In particular, it can be observed that the DP method failed to accurately estimate . This is because the data-generating model for has a complex nonlinear structure that includes the interaction of and . In contrast, the DP method (based on Schwartz et al., 2011) is designed to vary only the intercept parameters of and . Extensions of Schwartz et al. (2011) to extend the DP model to additional model parameters might improve performance in a scenario where interactions between and dominate the data-generating model for . On the other hand, the BPCF and BART methods relatively accurately estimated the values of ; however, the response surface for the BPCF method specifically showed deviations from the true values. This can be interpreted as the tree structure shrinking toward a more homogeneous effect when the modifier function only includes and not all covariates.

FIGURE 1.

[Simulation I] Average surface plots for all combinations of from 4 different models: (A) Truth, (B) BPCF, (C) BPCF, (D) BART, and (E) DP. The points in plane are averages of 1 set of MCMC samples. As a response surface, the corresponding value of the effect on the outcome is plotted.

In the Supplementary Materials, we evaluated a variety of simulation scenarios: (Simulation II) a model resembling the data-generating process from the targeted selection example in Hahn et al. (2020); (Simulation III) a scenario where ; (Simulation IV) 2 scenarios involving unmeasured confounders; (Simulation V) a scenario without heterogeneity in the PCE; (Simulation VI) a scenario reflecting the structure of real data; and (Simulation VII) a scenario where Assumption 2 is violated.

5. EVALUATION OF POWER PLANT SCRUBBER IMPACTS ON EMISSIONS AND AMBIENT AIR QUALITY

Reducing SO emissions from power plants has been a major goal of the Clean Air Act, with clear benefits to health and welfare coming from the resulting improvements to ambient air quality (Chestnut and Mills, 2005; Henneman et al., 2023). Installation of scrubbers at coal powered plants is but one strategy used to reduce SO emissions, and while scrubber impacts on SO are well understood, there is limited empirical evidence of their effectiveness for reducing ambient PM. A recent study in Kim et al. (2019) used data from 2005 to estimate substantial associative effects of scrubbers on ambient PM around plants where scrubbers reduced SO. Here, we deploy the BPCF approach to extend this analysis to a more recent time frame when levels of SO and PM are lower, consider a more robust set of possible confounding variables, and focus in particular on targeted selection and heterogeneity of scrubber effectiveness across principal strata.

5.1. Data

We collected data on 385 coal fuel power plants located in the eastern United States and operating during 2014, incorporating information from the EPA Air Markets Program Data and the Energy Information Administration. The data encompassed plant characteristics, installed emissions control technologies (if applicable), and emissions of various pollutants, including SO. A power plant was categorized as treated with a scrubber if over half of its component EGUs implemented a control technology for SO (in practice, plants typically control SO from all or none of their EGUs, so scrubber installation is effectively binary, see Online Appendix Figure S.5). We only considered power plants with nonzero heat input and with complete data on important confounding factors. For linking with other data sources, we enumerated all zip codes having centroids within 50 km of the power plant location to designate the surrounding area.

The ambient PM surrounding each power plant was calculated as the average of zip code-level predictions from Wei et al. (2022) within 50 km. Meteorologic conditions that can affect the operation of power plants and the formation of ambient PM were characterized at the zip code level by averaging annual relative humidity, temperature, and precipitation at the zip code centroids within 50 km and with 100 km self-supervised learned latent representations of nonlocal meteorology at surrounding zip areas constructed in Tec et al. (2023) and extracted from the Weather2Vec Embedding Generator (2023). Demographic information for the area surrounding a power plant was obtained from the 2010 US Census. In summary, each of the 161 power plants used for analysis is associated with a designation of scrubber installation, a measure of annual SO emissions, the annual average concentration of PM in a 50 km surrounding area, and a host of confounding features selected to satisfy the ignorability assumption, including power plant characteristics, details of power plant operation, complex measures of local and nonlocal meteorology, and population demographics.

Table 2 shows the summary statistics for each variable in the 2 treatment groups. Propensity score estimates for inclusion in the prognostic functions and were obtained using logistic regression with all aforementioned confounders. Before presenting the analytical results, MCMC convergence was assessed with plots of the MCMC log-marginal likelihood (Online Appendix Figure S2). An assessment of residual spatial correlation of the PM outcome (Moran’s I  ) indicates that the inclusion of these covariates reduces the threat of residual spatial autocorrelation.

TABLE 2.

Summary statistics for covariates, intermediate, and outcomes available for the analysis.

	Have SO control (n = 74)		Have no SO control (n = 87)
	Mean	SD	Mean	SD
Air Quality and Weather Data
Ambient PM ()	9.15	(1.70)	8.89	(1.64)
Relative humidity (%)	74.32	(3.88)	73.95	(5.00))
Temperature at 2 m (°C)	11.86	(3.49)	11.62	(4.10)
Total precipitation (kg/m2)	2.73	(0.52)	2.70	(0.49)
Power Plant Data
Total SO emission (tons)	713.95	(818.80)	1063.70	(1353.82)
Total NO emission (tons; past year)	542.47	(400.97)	365.96	(365.69)
Total CO emission (1000 tons; past year)	704.16	(424.70)	334.89	(381.71)
Average heat input (10 000 MMBtu)	8139.89	(4825.14)	4446.26	(3870.44)
Phase of participation in the ARP (Phase I or II)	0.62	(0.48)	0.75	(0.44)
Total operating time (hours # units)	19329.67	(9843.08)	18464.60	(10605.52)
Sulfur content in coal (lb/MMBtu)	2.09	(1.23)	0.89	(0.96)
Number of NO controls (# units)	4.97	(2.93)	4.09	(2.45)
Number of units	2.85	(1.49)	3.31	(1.71)
Gross load (1000 MWh)	8455.37	(5042.04)	4603.13	(4111.47)
Capacity (%)	58.22	(11.55)	47.48	(16.68)
Census Data
Population 2010	8604.69	(6612.86)	10077.72	(7377.57)

All variables represent annual averages, with Census variables relying on annual data starting from the year 2010. Note that there are also 10 additional learned latent representations of neighboring weather information included in the analysis.

5.2. Preliminary analysis

Unadjusted comparisons indicate that power plants with scrubbers had (1) lower SO emissions but (2) higher PM within 50 km, highlighting the threat of confounding. Subsequent estimates are reported as posterior means along with 95% credible intervals, derived from the BPCF analysis. The estimated causal effect of SO control strategies on SO emissions was (−702.55, −378.54) tons, indicating a reduction in SO emissions, as the credible interval does not include zero. Similarly, the estimated causal effect of control strategies on ambient PM concentrations was (−0.83, −0.37), showing a reduction in PM concentrations within a 50 km radius, as the credible interval does not include zero. These results coincide with the expectation that introducing SO reduction technology in power plants would consistently result in reduced SO emissions and subsequently lower ambient PM levels. The principal stratification approach will more thoroughly examine the influence of power plants’ varying response in SO emissions on the downstream impacts on ambient PM.

Notably, Figure 2 indicates that, in both the intermediate and outcome models, there is a positive relationship between propensity score estimates and prognostic function estimates. Particularly for the outcome variable (PM), an increasing relationship between and indicates that targeted selection bias may exist. This serves as one key motivation for the BCF-based approach.

FIGURE 2.

[Power Plant Analysis] For the estimated prognostic values , the estimated propensity scores are plotted. The right plot for PM  concentrations () demonstrates a moderate monotone relationship between the propensity score and the prognostic function.

5.3. Principal causal effects

Figure 3 presents the estimated average CEP surface. The points on the axis represent the posterior predictive mean values for . The slope of the response surface generally decreases in regions where values are lower than values. This indicates that the reduction in SO emissions corresponds to a decrease in PM concentrations, with steeper impacts in areas of the largest SO reductions.

FIGURE 3.

Average surface plot of the causal effect on PM for different values of by tons. Posterior predictive values of are plotted on the plane. The corresponding value of the causal effect of SO control techniques installation on PM, , is plotted on the z-axis.

To present numerical summaries, average PCEs were constructed based on the estimated variability of effects on SO emissions. The posterior predictive mean values for each th were estimated, and multiplying the SD of these values by 0.2 and 0.5 resulted in values of 320 and 798, respectively. Using this information, 6 principal strata were constructed as follows: Strata 1, 2, and 3 are defined as having   −798, [−798, −320), and [−320, 0), respectively, all representing units where SO decreases due to scrubber installation. Strata 4, 5, and 6 are defined as having   [0, 320), [320, 798), and 798, respectively, all representing units where scrubbers actually increase SO emissions.

Figure 4 displays posterior mean PCE estimates. The center point of each circle represents the posterior mean of the PCE, and the vertical line represents the 95% credible interval. The size of each circle represents the average number of power plants estimated to lie within each respective stratum, and the numbers below indicate the posterior mean, posterior SD, and posterior average number of plants. The -axis shows the causal effect on SO emissions within each interval. The first 3 principal strata contain most of the power plants and all have PCEs with credible intervals lying completely below 0. This suggests that the implementation of SO control techniques led to a substantial reduction in SO emissions (with most plants showing reductions greater than 798 tons) and consequently lowered PM concentrations, with similar effects on PM across plants with varying degrees of scrubber-induced reductions in SO. Principal strata 4, 5, and 6 indicating scrubber-induced increases in SO emissions are estimated to contain markedly fewer power plants. The PCE for stratum 4 is estimated to have a similar magnitude to that of strata 1-3, indicating a reduction in ambient PM. Strata 5 and 6 have PCE estimates indicating a causal increase in PM concentration due to the installation of SO control, although these estimates are derived from a comparatively small number of plants and entail wide uncertainty. Thus, it remains inconclusive whether, in the small number of plants where installing a scrubber seems to increase SO emissions by more than 667 tons, scrubbers causally affect ambient PM.

FIGURE 4.

[Power Plant Analysis] Posterior mean estimates of principal causal effects for 6 different strata. These 6 strata are defined by intervals 1, 2, 3, 4, 5, and 6. The circle size is proportional to the number of plants in each stratum. The listed numbers represent the posterior mean of the effects, the SD, and the posterior mean of the number of units estimated in the corresponding principal stratum, respectively.

6. DISCUSSION

This paper presented a novel PCE estimation method that can investigate the role of intermediate variables in estimating causal effects. To estimate the PCE, which varies with changes in principal stratum membership, intermediate and outcome variable models were created using the BCF model, which is known to accurately estimate the heterogeneous effect. This resulted in a more detailed characterization of principal strata, which were incorporated into the modifier function in the outcome model as an effect modifier. The proposed method provides protection against the evident threat of targeted selection and RIC. The air quality study in Section 5 demonstrated the phenomenon of targeted selection in relation to the effect of SO emission reduction technology on PM concentrations, proving the efficacy of the proposed method.

In this study, a priori estimated propensity scores were utilized as fixed covariates, which means that the uncertainty surrounding the propensity score estimation was not considered in the final analysis. As claimed by Hahn et al. (2020), incorporating propensity score estimates is merely a means to control regularization by transforming covariates. Therefore, a priori estimation is not a major concern. Jointly modeling the propensity score model with the intermediate and outcome models is not a straightforward task, primarily due to the feedback between these models (Lunn et al., 2009; Zigler, 2016). A small number of power plants showed increased SO emissions relative to the absence of SO control techniques. This cuts against expectations given what is known about scrubbers’ effectiveness for reducing SO, suggesting the possibility of unmeasured confounders influencing the relationship between exposure and intermediate variables.

There are multiple directions for future research on this topic. If there are multiple intermediate variables, the intermediate model can be constructed using a multivariate response, and a multivariate principal stratum can be designed accordingly. In this case, the principal strata data entered the outcome model’s modifier function as multivariate data. The methods presented here could be integrated with similar BART-based approaches to prioritize confounders from a high-dimensional feature set (Kim et al., 2023). A limitation of principal stratification is its inability to identify the indirect effects of scrubbers mediated through emissions. These effects could be estimated through formal mediation analysis, which requires additional assumptions about hypothetical interventions that reduce emissions independently from the installation of scrubbers. As explored in Kim et al. (2019), it may be possible to develop common identifying assumptions and modeling frameworks to integrate the proposed method with (heterogeneous) mediation analysis, enabling a more comprehensive analysis. Finally, the outcome variable, ambient PM concentrations, exhibits spatial correlation. The inclusion of covariate information has mitigated the threat of residual spatial correlation in this analysis, but incorporation of spatial random effects could prove advantageous in other contexts.

Supplementary Material

ujaf024_Supplemental_Files

Web Appendix A-M referenced in Sections 2, 3, 4 and 5, along with R code files for the simulation studies and data analysis, are available with this paper at the Biometrics website on Oxford Academic.

FUNDING

The National Research Foundation of Korea (NRF) grant funded by the Korean government (NRF-2020R1F1A1A01048168, NRF-2022R1F1A1062904, RS-2024-00407300) and the US National Institutes of Health (R01ES034803 and R01ES035131).

CONFLICT OF INTEREST

None declared.

DATA AVAILABILITY

The data that support the findings of this paper are openly available in the GitHub repository BPCF (https://github.com/lit777/BPCF).