Is the Smog Lifting? Causal Inference in Environmental Epidemiology

We thank the Editor for this opportunity to comment on the accompanying paper by Pearce, Vandenbroucke, and Lawlor¹ [PVL]. In their commentary, PVL discuss methods that they argue have been and should be useful for making causal inferences in environmental epidemiology. They also cite part of an ongoing debate in the literature (their references 6-12) that criticizes some causal-inference methods in epidemiology. In this commentary, we first point out many areas of agreement with PVL. We then comment on several issues raised by their commentary. One issue concerns omission of important references, and four issues concern claims that were poorly supported or potentially misleading. We also note how, with consideration of a wider scope of methods and more nuanced interpretation of recommendations regarding emulation of RCTs, many newer causal inference methods have been and should continue to be quite useful in environmental epidemiology and, in fact, have been used to justify many of the methods PVL suggested.

Six main points of this paper, at least at heart, are clear, documented, and convincing. In simple terms, these points are: 1) causal inference for certain questions in environmental epidemiology, such as studying the health effects of climate change, is difficult; 2) traditional methods have worked quite well for certain problems in environmental epidemiology; 3) additionally, at least five specific other methods (“extensions”) should be helpful in environmental epidemiology; 4) many methods can be useful in environmental epidemiology; 5) related to 4, the subgroup of methods as defined by PVL (‘RCT-mimicking set of “causal inference” methods’) doesn’t include everything that might be useful; and, 6) triangulation methods are one such additional method. The first and second points are supported by examples; the third point is also supported by examples for each method noted. Interestingly, all of the “extensions” have benefited by considerations from the causal inference tool box, as we elaborate below. The fourth and fifth points are perhaps self-evident, as we know of no subgroup of methods that covers everything or claims to. The last point is essentially an extension of the well-accepted Bradford Hill considerations, with additional considerations about the directions of potential bias. We agree with these six points.

The first issue we highlight concerns the omission of important references that are part of an ongoing debate about the merits of certain aspects of modern causal inference concepts, methods, and tools. While we welcome the largely muted level of criticism in this commentary, PVL cite only one side (their references 6-12) of this debate, omitting many responses to and commentaries on these criticisms^2–9. To varying degrees, these responses and commentaries have clarified positions, provided strong rebuttals of certain criticisms, explained misunderstandings, pointed out a strawman fallacy, and offered conciliatory remarks.

Our second issue concerns PVL’s unqualified claim that ‘the term “causal inference” is being used to denote a specific set of newly developed methods …’, characterized as the ‘…RCT-mimicking set of “causal inference” methods, in contrast to the broader field of causal inference of which it is a part’. This claim is unsupported and potentially misleading because, in the context of the ongoing debate, it could be taken to suggest that those who have contributed to modern methods of causal inference and who are also involved in the ongoing debate used the term “causal inference” in this way. On the contrary, most, likely all, who have contributed to modern methods of causal inference and who are also involved in the ongoing debate about the merits of certain causal inference methods, do not, in general, use the term “causal inference” to refer only to the restricted, narrow subgroup of methods described by RVL as “RCT-mimicking”^a. In other words, the definition put forth by PVL is consistent with neither the language used by these key contributors to modern methods, nor with the entirety of methods that those contributors have developed, used, and cited in their work.

Our third issue concerns PVL’s claim that: “This [modern causal inference movement, as defined by PVL] proposes that observational studies should mimic key aspects of randomized trials, since this allows them to be rooted in counterfactual reasoning, which is said to formalize the natural way that humans think about causality”. This claim is also potentially misleading because, in view of the ongoing debate^2–9 and without qualification, the claim might be read as implying that all observational studies should mimic key aspects of randomized trials. Worse yet, it might be read as implying that it represented the position of those involved in the debate. These interpretations would be incorrect. We agree that Hernán strongly advocates attempting to emulate randomized experiments as a device to aid study design, a position justified in part because such emulation can help sharpen effect definitions as counterfactual contrasts of better-defined, possibly hypothetical interventions, thereby helping to reduce the vagueness in causal questions^{4, 10, 11}. But with a broader reading of Hernán’s work and that of other causal-inference contributors, three important additional observations emerge. First, these causal-inference authors are making a conditional claim: if such emulation is possible, then many advantages will likely accrue. For example, as just noted the definition of the causal effect of interest may be clarified; additionally, an intervention of potential public health utility might be better identified and evaluated, and certain properties that can contribute to valid causal inference including exchangeability, consistency and positivity might be better defined and more evaluable^{2, 10}. See also Daniel et al. and van der Laan.^{9, 12} Second, many causal-inference authors note that in some contexts (e.g., system-wide interventions), it may not be possible to conceptualize an RCT that can be usefully emulated by an observational study as part of the causal inference process^{2, 6, 13, 14}. Third, these authors note that if one cannot emulate an RCT, then other methods are available and may apply, including those mentioned by PVL as useful in environmental epidemiology, such as instrumental variables, Mendelian randomization, and regression-discontinuity and difference-in-differences designs^{2, 15–18}.

As an alternative to the restrictive definition of causal-inference methods used by PVL, we suggest, in broad agreement with their overall message, that casual-inference methods correspond to what the phrase suggests and should include any, useful, valid method, many of which are already being used in environmental epidemiology. The list would include the methods listed by PVL plus others, some not mentioned or emphasized, such as G-computation, G-estimation, marginal structural models^19–21, probabilistic causal models²², structural equation models, combining information from multiple sources in a structural equation framework²³, negative control outcomes and exposures²⁴ with specific applications to air pollution and environmental epidemiology^25–27, directed acyclic graphs (DAGs), simulation modeling, and more. We specifically note the important contribution of causal graphs²⁸, and in particular DAGs²⁹. They were used in the development of methods to detect confounding with examples from environmental epidemiology in mind^25–27. More widely, they are commonly used to describe causal relationships and might be viewed as providing a language for doing so efficiently. As with good notation and language, DAGs can aid in the thought process³⁰. In addition to the noted observational approaches, simulations, projection studies and agent-based modeling^{14, 31, 32} can be useful in environmental epidemiology, although more work remains in delineating and understanding conditions for valid effect estimation^{33, 34}. In research on climate change and health, the projection of estimated effects under varying emissions and socioeconomic scenarios has become more common.^35–37 These types of studies are useful for projecting the health burden attributable to climate change but results are, of course, dependent upon the initial estimate of effects.

Our fourth issue concerns PVL’s statement that: ‘We are not arguing that “causal inference” methods that mimic randomized controlled trials are not useful; for example, they can improve individual studies with individual-level exposures that can be seen as interventions.” This claim is also potentially misleading because it could be read as implying or suggesting that studies that emulate randomized, controlled trials must involve randomization of individual-level exposures. In fact, the emulation applies more widely – one need only consider group-randomized trials and appreciate that neither randomization nor the potential outcome framework precludes group-level exposure. This wider appreciation is particularly relevant for environmental epidemiology since, as PVL note, environmental exposures often “… affect individuals across entire communities”.

Interestingly, many of the causal inference methods (e.g., “extensions of traditional approaches”) specifically mentioned by PVL have benefited from and have been justified by some of the modern causal inference concepts, methods, and tools. For example, Balke and Pearl³⁸ used counterfactual models to derive identifiable limits for the magnitude of effect based on analyses of instrumental variables. Although the difference-in-differences design precedes the formulation of the more recent causal-inference concepts, the major identifying assumption of difference in differences – parallel trends – appears now to be generally understood in econometrics as based on a counterfactual approach (see, e.g. Abadie³⁹ or Lechner⁴⁰). Moreover, the regression-discontinuity design can be based on the assumption that potential outcomes have a continuous distribution at the threshold⁴¹. Indeed, Imbens and Woodridge, important contributors to the econometrics literature where instrumental-variable analysis, difference in differences, and regression discontinuity have long been used, stated that “… the Rubin potential outcomes framework is now the dominant framework”⁴².

In summary, we agree with much of PVL’s basic message as contained in at least six simplified main points. However, they omitted entirely the responses to and commentaries on the debate they cited. They also set up a restricted definition of “causal inference”, claiming without qualification that it refers to a subset of RCT-mimicking set of causal inference methods and proceeded to make additional, unqualified characterizations about the subset they defined. Further, they omitted discussion of how RCT-emulation can apply to group-level exposures although they note the importance of this in environmental epidemiology. These omissions, characterizations, and failure to qualify or note the conditional nature of potential advantages if a randomized trial can be emulated are, perhaps, unintentional. Regardless of intentionality, these distractions are unfortunate because the broad message – environmental epidemiology will benefit from use of a variety of causal inference methods – is one with which few would disagree including VanderWeele, Hernán, Tchetgen Tchetgen, and Robins, who pointed out these benefits more widely⁵.

Biographies

Dr. Flanders is a professor of Epidemiology at Rollins School of Public Health. He teaches epidemiologic methodology, and does research in several areas, including epidemiologic methodology, environmental epidemiology, and cancer epidemiology.

Mr. Garber is a PhD student in Epidemiology at Rollins School of Public Health. He is interested in the effects of the environment on physical activity and injury and in epidemiologic methods.