In Pursuit of Evidence in Air Pollution Epidemiology: The Role of Causally Driven Data Science

The impact of air pollution on health gained widespread attention in 1952 when pollution from coal burning in London, with smoke so thick that drivers needed headlights at all times of day, was linked to a dramatic rise in the number of deaths.^1,2 Over the past decades, evidence of adverse population health effects has accumulated from many thousands of epidemiologic studies, suggesting this is a critically important public health problem. While at the individual level the relative effects of exposure tend to be small, and potentially large confounding biases are difficult to rule out, air pollution is ubiquitous and the exposed population enormous. Thus, these adverse effects have important policy implications and intense scrutiny of these epidemiologic findings is inevitable.^3,4 Recently, the emergence onto the statistics landscape of modern causal inference methods, as well as machine learning and other novel data science techniques, has generated excitement and a sense of promise that estimation of the causal effects of air pollution exposures on health outcomes could be more definitively described.^5–7 (By causal effects we mean the effects that would be seen under experimental changes of exposures. By causal inference we mean the process of inferring causal effects from data.) We share the excitement of others that the discipline of causal inference has the potential to advance air pollution policy and allow the integration of modern statistical tools into air pollution epidemiology, but we also caution against unrealistic expectations by highlighting important difficulties ahead. Our goal is to provide a glimpse of the opportunities afforded by the use of causal inference and data science methods, raise awareness about some of the outstanding challenges, and inspire others to join in on the efforts to overcome them. In our note, we first discuss broad concepts that are relevant across applications, and then focus on issues germane to air pollution epidemiology.

There is a need for deliberate causal inference in policy-relevant research

When the goal of a scientific study is to inform policy, describing associations between exposures and outcomes generally does not suffice: an assessment of causal effects is needed.⁸ Unless a particular study design very clearly allows such an assessment, as in an appropriately designed and conducted randomized trial, epidemiologists and biostatisticians typically word their findings carefully to avoid implying causation.⁹ Nevertheless, since the premise of these analyses is to bring to light potentially causal relationships, despite cautious wording, investigators and end users both may implicitly infer causation. For this reason, we believe it is important for analysts to employ methodology that allows causal inference under the most realistic set of conditions possible, and to ensure clear communication of these conditions to facilitate the interpretation of the results.

Conventional model-based approaches may be inadequate for inferring causal effects

In many disciplines, conventional regression models (e.g., the very many varieties of the linear model) are considered a cornerstone of traditional statistical practice for inference about health effects. Indeed, the great majority of air pollution epidemiology studies estimate health effects as relative risks or hazard ratios obtained using regression models. In the conventional model-based approach, it is customary to begin by selecting a particular regression model, often on the basis of the type of outcome under study (e.g., logistic regression for binary outcomes, proportional hazards model for survival outcomes). How notions of causality may be ascribed is then considered as a second step, often restricted to consideration of which potential confounders ought to be included in the model. In this conventional approach, investigators often focus on parameters most conveniently reported on the basis of the model chosen (e.g., odds ratio in logistic regression, hazard ratio in proportional hazards model).

Unfortunately, this common and traditional use of conventional regression models for causal inference has at least two important drawbacks. First and foremost, whether the estimation procedure used yields valid inferences on a scientifically meaningful estimand (i.e., the quantity being estimated) -- let alone one with a causal interpretation -- generally hinges on the assumption that the specified regression model accurately reflects the relationship modeled.^10–12 That the true underlying mechanism follows the simple form of conventional regression models is often a strong assumption without much prior empirical support. Additionally, well-intentioned efforts to reduce model misspecification through an iteration of model revisions guided by diagnostics (e.g., residual plots) can compromise the interpretation and validity of inferences from these models.^13–15 Accounting for informal model selection in model-based analyses is difficult and remains an unsettled topic of methodologic research.^16–20 Second, even in the ideal scenario in which all relevant confounders are accounted for and the regression model postulated a priori holds true, model-based regression coefficients corresponding to the exposure of interest may still not refer to the causal contrast desired to address the scientific question at hand. This may occur, for example, because the regression coefficients quantify a causal effect on a different scale than desired (e.g., odds ratio versus relative risk) or do not provide the population-level summary desired (e.g., conditional versus marginal interpretation). The limitations of conventional model-based causal inference are exacerbated when modeling is performed on scales less amenable to causal comparisons (e.g., the hazard ratio), or when the exposure of interest occurs over a longer period of time.^21–25

How do modern causal inference methods differ?

Modern causal inference methods help circumvent the limitations of conventional model-based approaches. First, as we discuss below, these methods facilitate the use of more flexible learning approaches. As such, they eliminate the overreliance of conventional model-based approaches on strong model assumptions, thereby increasing the reliability of the resulting scientific findings. Second, in modern causal inference, the causal estimand is rigorously defined before the inferential method is determined. Because it is deliberately chosen rather than inherited from the model choice that the outcome data type might suggest, the estimand is more likely to be directly relevant to address the scientific question. These points are discussed further below.

The choice of causal estimand plays a fundamental role as the starting point of all statistical considerations in modern causal inference methods. Counterfactual (or potential) outcomes, which are used to refer to hypothetical versions of the outcome under different exposure profiles of interest, play a key role in this first step. Causal estimands are generally expressed as summaries of the distribution of counterfactuals.²⁶ For example, in a simple binary exposure setting, the average causal effect is the mean value of the difference between the counterfactual outcomes corresponding to each of the two exposure profiles (e.g., exposed at high vs. low levels). These causal estimands cannot be directly estimated because only a single counterfactual can be observed for each study participant -- for example, for an individual having been exposed at a high exposure level, only the counterfactual outcome corresponding to high exposure is observed. To identify causal estimands from the observed data, there must exist an appropriate “bridge” (i.e., mapping) from the counterfactual world, where causal estimands are defined, to the observed world, in which data are collected. This bridge must allow the causal estimand to be expressed as a statistical estimand, that is, as a summary of the distribution of the observed data (see Figure). The existence of a bridge hinges on meeting the causal conditions, many of which are untestable. For the average causal effect, typical causal conditions used for identification include: consistency (i.e., the intervention defining exposures is unambiguous), the stable unit treatment value assumption (i.e., no interference), positivity (i.e., each participant could have been observed in the counterfactual exposure group), and (sequential) ignorability (i.e., exchangeability, or no unmeasured confounding).²⁶ When such a bridge exists, the causal estimand is said to be identified.²⁶ For example, if the average causal effect (say of a high vs. low binary exposure) is the causal estimand of interest, the destination of one such bridge is the G-computation formula. The latter is given by the population average of the difference in strata-specific mean outcomes among individuals with high versus low exposure, where strata correspond to subpopulations defined by confounder levels.²⁶ We invite readers to consult the textbook of Hernán and Robins for an in-depth study of foundational ideas in causal inference.²⁶ For a comprehensive discussion of causal inference in air pollution epidemiology, we point readers to the recent review by Bind.⁷

Figure.

Framework for Causally Guided Data Science

Once identification of the causal estimand is established, the resulting statistical parameter can be estimated using a variety of strategies. At this point, the problem is purely statistical: the fact that the statistical parameter corresponds to the causal estimand under certain causal conditions is irrelevant to how estimation and inference should then proceed. Simplifying statistical conditions, such as parametric forms (e.g., linear mean regression model), can be imposed on the observed data distribution to facilitate estimation and inference, but they are not strictly needed. This is where flexible learning techniques, such as machine learning, can be deployed to perform statistical inference less prone to bias due to model misspecification. Prior knowledge, including characteristics of the study design, informs all the conditions and can justify imposing more stringent statistical conditions that can be leveraged to generate more precise inference. Of course, whether or not a causal interpretation may be ascribed to the inferences drawn depends on the validity of the causal conditions.

In summary, as we highlight in the figure, a desirable feature of the causal inference framework is that it separates conditions as being either causal or statistical, with the latter generally being imposed out of convenience rather than necessity. In contrast, in the conventional model-based approach, causal and statistical conditions are often entangled. As such, without additional work, it is often difficult to determine whether identification is in fact largely driven by strong statistical modeling conditions (e.g., assumption of linearity of the mean outcome beyond the range of observed exposures²⁷) -- this reduces transparency and rigor in the conduct of causal inference. Explicitly disentangling causal and statistical conditions is also important because very different strategies exist to relax each set of conditions.

Causally guided data science has the potential to advance policy-relevant decision-making

Because causal estimands, such as the average causal effect, have clearer scientific interpretability than statistical estimands based on (plausibly misspecified) conventional models (e.g., coefficients in regression models), they are better suited to inform air pollution policy-making.²⁸ These causal estimands can quantify the health consequences of hypothetical (possibly regulatory-driven) changes in exposure, and can predict the effects of future interventions or policies.⁵ Modern causal inference methods therefore widen the scope and increase the granularity of policy-relevant scientific questions that can be addressed from the available data.⁵ Additionally, reliable inference in air pollution epidemiology is needed, particularly in view of the wide-ranging implications of air pollution policies. Results from analyses that build upon unrealistic conditions may be misleading.

Causal conditions allow identification of the causal parameter without generally implying constraints on the observed data distribution. As discussed above, the use of conventional statistical models, such as conventional parametric models, often impose unnecessary conditions for the validity of inferences, regardless of whether the causal conditions are met. This is where we believe developments in data science can have the greatest impact. Statisticians and other data scientists have developed many flexible learning algorithms to reliably and robustly uncover structure from data. Their use, however, does not imply that more conventional learning methods (e.g., linear regression) must be discarded. Certain ensemble learning methods (e.g., the Super Learner, an example of model stacking) are able to combine algorithms in any user-specified collection of learning strategies in an automatic, data-driven, and optimal fashion.^29,30 This eliminates the need to bet on any particular learning tool and also largely negates the relevance of model diagnostics. To perform optimal inference for the statistical parameter of interest that identifies the causal estimand, the output of any flexible learning method must generally be corrected to achieve an optimal bias-variance trade-off for the desired estimand. There are several possible strategies for doing so, some of which have been used for decades (e.g., estimating equations methodology), and others that are more recent innovations (e.g., targeted minimum loss-based estimation).^31–33 What emerges is thus a two-step process, wherein relevant features of the observed data distribution are estimated flexibly and then corrected to ensure valid inference for the actual target of inference. The study of the tools and relevant issues to consider in this process has been the focus of targeted learning, a rapidly expanding area of methodologic research.^32,34–39 These tools allow practitioners to employ flexible learning strategies, thus facilitating robust analyses based upon as few statistical conditions as possible. They also enable reliable uncertainty assessment, including confidence intervals and p-values, a critical ingredient in the interpretation of results for the sake of policy-making.

When causal conditions fail to hold (e.g., there is interference or unmeasured confounding), a bridge may not exist, and full identification of the causal estimand is then not possible. In some cases, partial identification, wherein bounds on the true causal estimand are identified, holds.⁴⁰ Full identification can be recovered by modifying the definition of the exposure or the target population. An alternative strategy is to find an instrument -- a cause of exposure that otherwise has no bearing on the outcome -- in order to help circumvent failure to account for all important confounders, including the ones that are unmeasured.^41,42 Examples include quasi-experiments such as the coal ban in Dublin and the traffic plan implemented during the Beijing Olympics.^43,44 Regardless, even when causal conditions do not strictly hold, causally-motivated estimands still have a transparent, model-agnostic interpretation that is often more sensible than that of conventional model-based estimands. We believe this enables more informed policy-relevant decisionmaking about air pollution.

Causal inference in air pollution epidemiology is inherently challenging

Investigating the causal effect of exposure to air pollution on health outcomes is an intrinsically challenging endeavor.^45,46 We highlight only a few of these challenges below.

First and foremost, obvious ethical considerations often preclude the conduct of randomized trials, a design that allows easy derivation of causal inferences. Observational studies face confounding biases that may be difficult to fully account for. Furthermore, these biases can be in either direction and have unknown magnitude. This challenge is magnified because for individuals the relative increase in risk from air pollution exposures is typically small.⁴⁷ Of course, this is not to say that these adverse effects should be overlooked -- ubiquitous air pollution exposures can still affect individuals, and, more importantly, accumulate over large populations, resulting in a substantial burden for communities and health systems.⁴⁸ Uncontrolled confounding results in violation of ignorability, a key causal condition, and is difficult to avoid in air pollution epidemiology.

Second, when the health effects arise from long-term exposures, defining causal effects of interest can be challenging because of the many possible definitions of counterfactual outcomes. Defining the counterfactual outcomes requires selection of the relevant interventions on exposures. What exposure duration should be considered? Should daily exposure be fixed at a particular level across time, or allowed to vary within a window? Being more or less strict in defining patterns of exposure leads to a trade-off between the interpretability of causal effects and the feasibility of the statistical problem. Oversimplification of the complex exposure process -- for example, by ignoring the time-varying nature of exposures -- can potentially invalidate causal inferences, as some authors have recently argued.⁴⁹ Marginal structural models, a set of tools for performing parsimonious causal regression, and causal estimands based on stochastic interventions seem particularly promising to tackle these challenges.^25,50–55

Third, even when using causal inference methods, disentangling the causal effects of exposure to a particular pollutant may be difficult if it tends to co-occur with other pollutants.^56–63 This may be especially true if these pollutants emanate from the same source. Since the co-occurring exposures possibly confound the relationship between the pollutant and health outcome of interest, and must therefore be accounted for, positivity violations are likely. Learning accurately how the outcome differs based jointly on the various exposures and confounders is then difficult, since the health effects are at best only weakly identified.

Fourth, accurately measuring exposure to air pollutants is itself a challenging task.^47,64 While technical advances have made it possible to outfit individuals with personal sensors capable of measuring exposure histories, implementing this in large-scale studies is prohibitively costly. Instead, by combining data obtained from regulatory monitoring stations, geographic information systems, and satellite imaging, researchers have developed historical exposure models used to predict pollutant levels in space and time, particularly at unmonitored locations.^65–69 However sophisticated these models may be, their outputs remain estimates. Accounting for their inherent (spatially varying) uncertainty and biases largely remains an unresolved problem in air pollution epidemiology, although recent advances have been made.^70–75 Additionally, even ignoring the imperfect mapping of historical pollutant levels, individual location tracking would be needed to obtain accurate exposure histories. Currently, most cohort studies use outdoor pollutant levels at domicile location as a proxy to reconstruct exposure history, ignoring the fact that most exposures may occur in an entirely different locale. It is important to understand the repercussions of these issues on causal inference and to identify possible remedies.⁷⁵

Fifth, causal methods are emerging and their development remains the focus of active research. Considerable advances in methodological research and software tools are likely needed before many practitioners will be ready to adopt them as standard practice. Challenges include: determining appropriate counterfactuals; spelling out relevant causal estimands; determining identification formulas (i.e., relating these causal estimands to summaries of the data-generating distribution -- the bridge); and devising inferential procedures that leverage flexible, data science tools (e.g., machine learning). These tasks require substantial training in relatively recent technical areas, such as causal inference, targeted learning, and machine learning, which many academic programs do not yet cover in depth. The unfamiliarity of these modern tools to many scientists may also be a perceived barrier to publication, possibly driving researchers to persist in the use of more conventional methods.

Responsible policy-making should be informed but not paralyzed by causal inference

We view the greater adoption of cutting-edge data science tools and causal inference principles into mainstream air pollution epidemiology as an important step forward.^5,8,76–84 As Goldman and Dominici state, “well-validated methods for causal inference can play a useful role: this is because they include a more transparent disclosure of all the assumptions that are needed to properly adjust for confounding compared with regression modeling and therefore can infer causality in analyses of observational data.”⁸⁵ Nevertheless, we cannot expect the emergence of a methodologic silver bullet since many of the challenges of drawing valid causal inferences about air pollution health effects stem from inherent features of the observational nature of the available data. In other words, no statistical approach is likely to overcome these challenges entirely. Rather, cutting-edge methods must be combined with the use of innovative study designs or complementary data sources tailored to the particular difficulties encountered in air pollution epidemiology.

Some scientists have argued that, because epidemiologic studies provide measures of association and may not accurately predict the benefits of reducing air pollution, they should not be used, thereby dismissing a large body of evidence painstakingly gathered over decades.^86,87 Yet, the Clean Air Act mandates that, even despite the presence of uncertainty, air quality regulations be set providing for an adequate margin of safety.^88,89 In our view, causal inference methods should not be used as another opportunity to weaponize science against itself. Policymakers cannot wait for the data, study designs, and analytic tools that will ensure unarguable causal inferences: stalling until perfect evidence arises is irresponsible and does not protect public health.

About the authors

MARCO CARONE is Assistant Professor of Biostatistics at the University of Washington School of Public Health, and an Affiliate Investigator in the Vaccine and Infectious Disease Division at the Fred Hutchinson Cancer Research Center. His research lies at the intersection of nonparametric and semiparametric statistics, causal inference, survival analysis, and statistical epidemiology. He focuses on developing efficient and robust methods for performing statistical inference with machine learning tools. He is the Norman Breslow Endowed Faculty Fellow.

FRANCESCA DOMINICI, PhD is Professor of Biostatistics at the Harvard T.H. Chan School of Public Health and Co-Director of the Data Science Initiative at Harvard University. Her research focuses on developing and advancing methods for the analysis of large, heterogeneous data sets to identify and understand the health impacts of environmental threats and inform policy. She is a member of the National Academy of Medicine.

LIANNE SHEPPARD, PhD is Professor of Biostatistics, and Environmental and Occupational Health Sciences at the University of Washington School of Public Health. Her research interests focus on statistical methods for understanding the health effects of environmental and occupational exposures; they include study design, measurement error, exposure modeling and estimation, and estimation of environmental exposure effects with application to a wide range of health outcomes. She is a fellow of the American Statistical Association and served on the chartered Clean Air Scientific Advisory Committee.