Invited Commentary: The Promise and Pitfalls of Causal Inference With Multivariate Environmental Exposures

Abstract

The accompanying article by Keil et al. (Am J Epidemiol. 2021;190(12):2647–2657) deploys Bayesian g-computation to investigate the causal effect of 6 airborne metal exposures linked to power-plant emissions on birth weight. In so doing, it articulates the potential value of framing the analysis of environmental mixtures as an explicit contrast between exposure distributions that might arise in response to a well-defined intervention—here, the decommissioning of coal plants. Framing the mixture analysis as that of an approximate “target trial” is an important approach that deserves incorporation into the already rich literature on the analysis of environmental mixtures. However, its deployment in the power plant example highlights challenges that can arise when the target trial is at odds with the exposure distribution observed in the data, a discordance that seems particularly difficult in studies of environmental mixtures. Bayesian methodology such as model averaging and informative priors can help, but they are ultimately limited for overcoming this salient challenge.

Editor ’s note: The opinions expressed in this article are those of the author and do not necessarily reflect the views of the American Journal of Epidemiology.

A key element throughout the history of causal inference is the foregrounding of framing observational studies as approximations of (hypothesized) randomized trials (1, 2). The work by Keil et al. (3) in this issue of the Journal deploys this perspective to evaluate the extent to which a reduction in 6 metal exposures linked to coal-fired power plant emissions might causally affect birth weights in Milwaukee, Wisconsin. A chief contribution of this work is the authors’ articulation of how explicit causal inference methods can direct research questions away from some of the thorny statistical issues related to analysis of complex mixtures and focus investigations around the evaluation of causal estimands that correspond to consequences of specific, well-defined actions. I congratulate Keil et al. for pursuing this important continuation of the recent uptick in use of causal methods in environmental epidemiology (4–11) and enthusiastically endorse these efforts to advance an already rich body of literature on the analysis of complex environmental mixtures.

FRAMING CAUSAL QUESTIONS WITH MIXTURES

Using the infrastructure of potential outcomes, Keil et al. (3) frame their analysis as an explicit contrast between exposure distributions associated with a specific action of the type that might occur in a (hypothetical) designed experiment. Framing observational studies as approximate experiments often involves stretching notions of what interventions could hypothetically occur. When dealing in the realm of the hypothetical, it is important to remember that simply declaring that an action could have hypothetically occurred does not guarantee that 1) investigating its impact has public health relevance or 2) the observed data contain the relevant information with which to reliably characterize it. The example in the paper by Keil et al. (3) stands on much firmer ground with regard to point 1 than it does to point 2, highlighting important friction that can arise in the practice of environmental epidemiology, especially when dealing with complex mixtures.

Hypothetical, simulated, and infeasible interventions

The nominal focus of Keil et al. is “a hypothetical intervention to reduce exposure to 6 airborne metals by decommissioning 3 coal-fired power plants in Milwaukee County prior to 2010” (3, p. 2648) and its causal effect (vs. no action) on birth weight. In contrast to similarly framed analyses of power plants (12–16), an important feature of the analysis of Keil et al. is that the decommissioning of coal plants never actually occurred. Absent actual decommissioning of plants, Keil et al. assume that hypothetical decommissioning would lead to a uniform proportional reduction in each of the 6 metals relative to their observed values across the study area. This reduction dictates the underlying trial the methods are designed to approximate: The “control arm” of the trial is the observed exposure mixture, and the “intervention arm” is the uniform proportional reduction in levels of all 6 metals. Confounders are those variables that are related to metals and birth weight; positivity relates to the likelihood of a census tract’s exhibiting a particular level of the 6 metals; and consistency assumes that hypothetical exposure levels are unrelated to the manner in which they were realized. The relationship to coal plants bears nominal relevance to the framing of the hypothetical target trial but is not, strictly speaking, related to the methodological underpinnings of the causal inference methodology. One purpose of the framing around coal plants is that it isolates one of infinitely many exposure contrasts that could be used to generate a hypothetical trial of these 6 exposures; the simultaneous uniform proportional reduction is the exposure contrast of interest, and not, say, a shift in each individual exposure dimension (while holding the others fixed) or a change in the relative mixture proportions of each component, both of which would correspond to different underlying hypothetical trials. Whether this uniform proportional decrease actually corresponds to what would happen upon decommissioning of coal plants is considered in Keil et al.’s paper (3), but detailed assessment of this correspondence was apparently beyond the scope of the investigation.

Defining versus designing the hypothetical trial

When considering causal estimation in the hypothetical target trial, it is important to suspend focus on power plants and evaluate the approximation of the target trial on the basis of the distribution of the 6 metal exposures. Here, the authors clearly identify that approximating their target trial would require extrapolation beyond the range of observed data. Keil et al.’s Web Figure 1 shows that the levels of these 6 metals implied by the uniform proportional reduction would fall almost entirely below the range of values observed in the study area. Thus, the hypothetical trial entails an “intervention arm” for which there is no representation in the observed data; estimates will necessarily extrapolate to these levels of exposure, and validity is almost entirely dependent upon the adequacy of the statistical model used to perform that extrapolation. This is dangerous territory: Results will be highly dependent on model specification (a cartoonish but possibly instructive example appears in Gutman and Rubin (17)). An alternatively defined target trial bounded by the percentiles of the observed exposure distribution (in a secondary analysis) is easier to approximate, but its relationship with the decommissioning of power plants is not clear. Hence the inherent friction: The readily interpretable framing relative to power plant closures is undercut by the realities of the observed data, and a framing that more closely hews to the observed data sacrifices the intervention-focused interpretation. I suspect that this friction arises more often in studies of environmental mixtures than in those of univariate exposures, since specifying hypothetical levels of multiple exposures may not cohere to the inherent correlations and interdependencies of the natural exposure mixture, thus increasing the likelihood of inconsistencies with the observed data.

BAYESIAN CAUSAL INFERENCE: MODEL AVERAGING AND REGULARIZATION

For estimation, Keil et al. provide a combination of g-computation with the mechanics of Bayesian model averaging (BMA) and informative prior distributions (3). The potential benefit of incorporating model uncertainty into g-computation gives, in a general sense, a clear answer to the common question of “Why Bayesian?” when it comes to established causal inference techniques such as g-computation that have been employed outside the Bayesian paradigm for decades. The set of correlated exposures, possible confounders, and higher-order interactions contained within the regression model underlying g-computation prime the analysis to exhibit unstable and erratic estimates. Coupling this challenge with that of extrapolating beyond the observed range of the data motivates averaging over many possible models instead of pinning hope on only one. While Keil et al. point toward some advantages of their approach (3), I see some clear future directions for more closely tailoring these methods to causal inference investigations. More importantly, these methods are not capable of resolving the friction at the heart of the analysis, and they cannot overcome poor “design” of the hypothetical target trial.

Bayesian model averaging

BMA simultaneously considers a model class of all 2⁸³ possible regression specifications using the 83 quantities enumerated by Keil et al. (3), effectively letting the data “choose” which terms are relevant for predicting birth weight, without forcing the investigator to include an unnecessarily high-dimensional set of regression parameters that risk exacerbating the difficulties created by multiple highly correlated exposures. The final posterior distribution summarizing inference for causal effects weighs each possible model in the model class according to its support from the data (although, as a matter of implementation, I am wondering how a space consisting of 2⁸³ distinct models could have been effectively explored with 36,000 random walk Markov chain Monte Carlo iterations).

As Keil et al. state in their Web Appendix 1, “under standard causal identification conditions and correct model assumptions,” BMA can form the basis of birth weight predictions under specified exposure levels. But does every model in the class of 2⁸³ models satisfy the requisite assumptions about, for example, confounding? Even when models satisfying the identification conditions are implicitly assumed to lie within the model class, BMA provides no guarantee that the posterior distribution will concentrate around these models. This issue has motivated a steady stream of research on “Bayesian confounding adjustment,” including work explicitly targeting model averaging for multiple exposures (18–24). The full implications of this line of work in the analysis of Keil et al. are not clear, although the different causal estimate obtained when the confounder main effects are forced into the model (sensitivity analysis G) is suggestive.

What’s more, I find it noteworthy that the primary analysis assigns substantial posterior weight to models that omit some (or even all!) of the 6 metal exposure main effects. As sensitivity analysis H shows, forcing the inclusion of all exposure main effects yields an effect estimate roughly twice the magnitude of that in the primary analysis! A careful consideration of why this might occur seems important, as it may relate to (among other issues) confounding differences across exposures or correlation among the exposures, both of which would illustrate the persistence of standard concerns around regression analysis of complex mixtures. In any case, how to interpret averages over models with different exposure variables (and wildly different causal estimates) is unclear, especially in relation to the hypothetical target trial and its relationship to coal plant closures.

Informative prior distributions and regularization

While it is not required for BMA, Keil et al. (3) additionally incorporate strong informative prior information on the underlying regression coefficients, as most completely described in Keil et al.’s Web Table 1. The “hierarchical specification” takes on a peculiar form, assuming that the regression coefficients associated with the 14 main-effect terms come from the same underlying normal distribution, effectively shrinking the parameter estimates toward each other. The unknown mean of this normal distribution (μ₁) is given a highly informative prior centered at 0 with variance 1, implying shrinkage of all of these effects toward 0 (a separate but analogous distribution is specified for the remaining 69 higher-order terms). Why one would specify that the main effect for, say, maternal age should be shrunk toward that of, say, chromium is never discussed. The nonhierarchical versions of the model shrink the regression parameters toward 0 without shrinking them toward each other.

Bayesian regularization—essentially shrinking regression coefficients toward 0 with informative priors—has benefits in mixture problems when regression estimates are prone to “blow up” due to sparsity and correlation among exposure terms. This is the presumed motivation for the informative priors in Keil et al.’s paper (3), although the rationale for adopting a strategy that falls outside the more mainstream literature on regularization (25–28) is not clear. Judging from the sensitivity analyses, the implications of the prior specification in the birth weight analysis are at least as consequential as the account of model uncertainty.

Model-averaged extrapolation

Despite possible improvements over “standard” g-computation, there is no reason to expect the Bayesian methodologies advanced by Keil et al. (3) to overcome the challenges inherent to the “design” of the hypothetical target trial. The simulation study is underequipped to support the claim that the Bayesian approach provides assurance that precision improvements do not come at the expense of large bias, as it evaluates settings where the simple data-generating model lies either in or close to the analysis model class. The possibilities for model misspecification in the birth weight analysis surely extend beyond slightly different variations of the linear model structure.

SUMMARY

Keil et al. (3) offer a useful installment of explicit causal inference methodology to advance to the study of complex environmental mixtures. In so doing, they highlight a key friction between the benefits of framing studies of mixtures with clearly defined actions (such as coal plant decommissioning) and the possible limitations of observed data for reliably estimating causal effects of those actions. While the adoption of Bayesian g-computation presents some potential improvements over more standard methods, these advances cannot resolve the rift between the nominal link to power plants and the necessary extrapolation beyond the range of the observed data. The dizzying sensitivity analyses highlight substantial sensitivity to the model specification, making it difficult to discern which of the many disparate results should be regarded as “primary” in any sense. Nonetheless, it is worth considering how the methods outlined here would fare in settings with fewer data limitations, particularly those that would permit more refined characterization of power plant exposures with distance-based proxies or methods specifically designed to characterize pollution from point sources, both of which have been employed in epidemiologic (and causal inference) investigations of power plants (12, 13, 15, 29, 30).