Causal inference in cumulative risk assessment: The roles of directed acyclic graphs

Abstract

Cumulative risk assessments (CRAs) address exposures to multiple chemical and nonchemical stressors and often focus on characterization of health risks in vulnerable populations. Evaluating complex exposure-response relationships in CRAs requires the use of formal and rigorous methods for causal inference. Directed acyclic graphs (DAGs) are graphical causal models used to organize and communicate knowledge about the underlying causal structure that generates observable data. Using existing graphical theories for causal inference with DAGs, risk analysts can identify confounders and effect measure modifiers to determine if the available data are both internally valid to obtain unbiased risk estimates and are generalizable to populations of interest. Conditional independencies implied by the structure of a DAG can be used to test assumptions used in a CRA against empirical data in a selected study and can contribute to the evidence evaluations related to specific causal pathways. This can facilitate quantitative use of these data, as well as help identify key research gaps, prioritize data collection activities, and evaluate risk management alternatives. DAGs also enable risk analysts to be explicit about sources of uncertainty and to determine whether a causal effect can be estimated from available data. Using a conceptual model and DAG for a hypothetical community located near a concentrated animal feeding operation (CAFO), we illustrate the advantages of using DAGs for evaluating causality in CRAs. DAGs also can be used in conjunction with weight of evidence (WOE) methodology to improve causal analysis for CRA, which could lead to more effective interventions to reduce population health risks.

1. Introduction

Cumulative risk assessments (CRAs) examine, characterize, and quantify the combined risks to human health or the environment from exposures to multiple stressors, potentially including chemical, physical, biological, and social stressors (Lokke, 2010; Meek et al., 2011; NRC, 2009; U.S. EPA, 2003). Development of methods to conduct CRAs is responsive to the multifactorial nature of many human diseases and concerns that evaluation of multiple chemical and nonchemical stressors, including complex chemical mixtures, and vulnerabilities on a community relevant scale are incompletely addressed by traditional environmental risk assessment approaches (Callahan and Sexton, 2007; NEJAC, 2004; NRC, 2009; Sexton, 2012).

Deriving accurate and unbiased estimates of causal effects is critical to CRAs and other risk analyses. Experimental studies (e.g., randomized controlled trials) are considered the most reliable types of studies for causal inference and allow the researcher to interpret the effects of intervention in the treatment group as the true causal effect of the intervention. However, for quantifying risk to multiple stressors, data from observational epidemiological studies of environmentally relevant exposures are often more abundant. When observational epidemiological evidence is available, quantitative use of these data including extrapolation to other populations can be complicated by a variety of issues including variation in population characteristics and unmeasured alternative causal pathways caused by confounding or other sources of bias. Among modern causal inference methods, such as the Neyman-Rubin Model (i.e., Potential Outcomes Framework; Holland, 1986) and Pearl’s (2009) Structural Causal Model, Pearl’s approaches for graphical annotation of causal relationships, that unify graphs, counterfactuals, and the potential outcome framework, appear well-suited to CRA because they can depict complex relationships and serve as visual guides to address these sources of uncertainty.

When conducting CRAs, developing well-constructed graphical causal models is critical for evaluating causality and considering risk management opportunities (e.g., intervention points in the pathways from sources of stressors, to exposures, to health outcome/s). Directed acyclic graphs (DAGs), a type of graphical causal model, are comprised of nodes connected by unidirectional arrows containing no paths that form a cycle (Glymour, 2006; Greenland et al., 1999). Other types of graphical causal models, including conceptual or theoretical models, are DAGs if they consist of clearly directed causal paths without cycles.

DAGs provide opportunities for notating causal assumptions about statistical associations and determining whether relationships are causal. The available evidence conveyed in a DAG may vary from qualitative to quantitative; the arrows drawn between the nodes may represent expert opinions, statistical associations derived from epidemiological or toxicological studies, or a priori knowledge of cause-effect relationships including mode of action (MOA) processes. In conjunction with weight of evidence (WOE) approaches (Rhomberg et al., 2010; Weed, 2005), DAGs can be used in CRAs to enhance causal analysis by identifying sources of bias, enumerating the testable implications of the causal diagram, and identifying if causal effects and potential impacts of an intervention are estimable from available data (Pearl, 2009). Formal methods for evaluating causal hypotheses are recognized as important to support risk-based decisions (Linder et al., 2010). These include graphical causal modeling and conditional independence testing (described in this article), Granger causality tests, panel data analysis, and intervention analysis, among others (Cox, 2013).

This paper explores potential uses of DAGs for causal inference within CRAs and provides a starting point for their practical application by risk analysts. We describe existing usage of causal models in CRA, and then describe how to integrate DAGs as causal models throughout a CRA. To highlight the types of reasoning and information needed to develop DAGs for CRAs, we initially constructed a conceptual model and then developed a DAG for a hypothetical community near a concentrated animal feeding operation (CAFO) using associations identified from the published literature. While this article focuses on the uses of DAGs in CRAs, we realize that other analyses may utilize DAGs for other purposes and have different considerations (e.g., DAGs supporting epidemiological studies). Causal modeling and its applications in social science, psychology, economics, and epidemiology has been described elsewhere (Glymour, 2001; Greenland et al., 1999; Morgan and Winship, 2014; Pearl, 2009; Spirtes et al., 2001).

2. Existing usage of graphical causal models in CRA

Graphical causal models are graphs that depict background knowledge, previously established theories, hypotheses about causal structures, and mechanisms that may be altered through an external change or intervention. These graphs may be accompanied by a set of mathematical equations derived from empirical data (Pearl, 2009; Russo, 2009). This section describes their current and potential usage in the context of CRA.

The U.S. Environmental Protection Agency (USEPA) (U.S. EPA, 2003) describes the following three phases of CRAs: 1) planning, scoping, and problem formulation, 2) analysis and 3) risk characterization. As shown in Fig. 1, the development of conceptual models can be critical in the planning, scoping, and problem formulation phase.

Fig. 1.

Framework detailing three main phases of CRAs. Adapted from U.S. EPA (2003).

Conceptual models depict relationships among stressors and outcomes, usually as box and arrow diagrams, but can take alternate forms (Linder and Sexton, 2011; Suter, 1999; U.S. EPA, 2003). Conceptual models are useful in CRAs because they graphically depict known, predicted, and assumed causal relationships among stressors and responses for various receptors (U.S. EPA, 2003, 2007). The conceptual model in Fig. 2 depicts a hypothetical community located near a CAFO, a source of disparate stressors in the community that potentially cause a number of adverse ecological and human health effects (Flora et al., 2002; Herriges et al., 2005; Howard et al., 2010a; Howard et al., 2010b; Merchant et al., 2002; PCIFAP, 2008; U.S. EPA, 2004; Wing et al., 2008a, 2008b; Wing et al., 2013). The conceptual model incorporates these stressors along with ecological and human health effects at the community and individual scales, illustrating this CRA’s potential scope and a possible causal ordering for the components of community-driven, effects-based assessments (Menzie et al., 2007; NRC, 2009). In our hypothetical example, the initiating factor for the CRA is an increased prevalence of chronic bronchitis, the primary health outcome of interest, in the population residing near the CAFO (Kirkhorn and Garry, 2000; Melbostad et al., 1997; Radon et al., 2007).

Fig. 2.

Conceptual model for a CRA in a hypothetical community near a CAFO. This example could be used in an effects-based assessment with an increased prevalence in the community of one health outcome, Chronic Bronchitis, as the initiating factor for the assessment. Abbreviations: PMx = particulate matter 2.5, 10 μm; NH₃ = ammonia; H₂S = hydrogen sulfide; VOCs = volatile organic compounds; NOx = nitrogen oxides; N&P = nitrogen and phosphorus CAFO = concentrated animal feeding operation.

Theoretical frameworks, defined as formal structures ordering hypothesized causal factors and pathways, also can aid in the estimation of cumulative health risks (Linder and Sexton, 2011) by reinforcing relationships depicted in conceptual models. In the absence of a theoretical model, data availability could limit the scope of a CRA and exclude other important relationships (Linder and Sexton, 2011). We relied on theoretical frameworks (CPHHD, 2007; Linder and Sexton, 2011; Morello-Frosch and Shenassa, 2006; Schulz et al., 2005; Gee and Payne-Sturges, 2004; Alexeeff et al., 2010) to guide the pathway structure in the conceptual model.

The CRA literature describing conceptual models (Linder and Sexton, 2011; Menzie et al., 2007; NRC, 2009; U.S. EPA, 2003) does not specifically address alternative causal factors (i.e., confounders) or how to depict such variables in graphical models. Methods developed by Pearl (2009) for identifying confounders and other sources of bias using DAGs can be useful for CRAs. Using DAGs improves the characterization and communication of complicated relationships encompassed in the conceptual model by providing graphical depictions of causal structures and assumptions at a finer scale. Causal modeling with DAGs allows all common causes (also called confounders) of the exposure and disease to be identified and further described as either measured or unmeasured. A risk analyst can use this information to determine what variables should or should not be addressed as confounders in order to estimate causal effects for hazard identification. This type of information cannot be deduced from typical conceptual models. See Table 1 for a summary of the uses of DAGs for causal inference in CRAs.

Table 1.

Overview of the use of DAGs for causal inference in CRA.

Purpose	To improve the conduct of CRAs by using DAGs to inform planning and scoping efforts, enhance WOE evaluation in hazard identification, select appropriate studies for derivation of dose-response functions, and address uncertainty from a rigorous evaluation of specific sources of bias and variability common in observational studies.
Important concepts	Observed statistical associations (e.g., from effect estimates reported in empirical studies) can be a combination of the causal relationship and other spurious factors. Remove the spurious portion of the association (e.g., confounding) to estimate causal effects. If causal effects can be estimated, then this enhances the basis for the quantitative analysis comparing potential interventions (e.g., efficacy, equity, etc.).
Main questions	Answers
What is a DAG?	A graphical causal model comprised of nodes connected by single-headed arrows containing no closed loops.
What is the role of the DAG and how does it aid causal inference?	Identify potential sources of bias and determine if causal effects are estimable from observed data. Encode assumptions about the causal ordering of statistically dependent variables. Translate causal assumptions into testable statistical associations. Assumptions about causal relationships are unambiguous in a DAG. Cause and effect are clearly defined.
How does the use of DAGs in causal inference advance the field of CRA?	A rigorous and uniform method for causal inference will support communication of the causal theory used to evaluate cumulative risks.
	Planning, scoping, and problem formulation	Concisely diagram the scope of the problem including relevant sources, stressors, pathways, receptors and health endpoints. Evaluate opportunities for risk management input (e.g., intervention points in the pathways from sources of stressors, to exposures to health outcome/s). Facilitate initial consideration of alternative interventions and the planning of targeted data collection activities.
	Analysis	For hazard identification, DAG’s testable implications can add to WOE evaluations related to causal pathways and increase transparency of evidence evaluation. An iterative approach to revising a DAG should reflect the risk hypothesis being addressed and integrate the most robust available evidence available to support it. Aid in selection of relevant studies for dose-response estimation by evaluating confounding in studies, determining if study data are applicable to the population of interest, and identifying if causal effects are estimable for total, direct, indirect, intervention effects, and effect modification. Detail factors that influence exposure, improving identification of confounders, and delineate aspects such as time-varying exposures and biomarkers.
	Risk characterization	Scoping decisions and assumptions about the boundaries of the assessment are explicit in the DAG for communicating assessment approach. Assumptions regarding causal ordering and most relevant variables are explicitly identified in DAG for communication and for developing insights into conducting analyses of uncertainty and variability.
What are the challenges of using DAGs for causal inference?	DAGs cannot convey the magnitude of risk, distribution of variables, or functional forms of relationships. Purely inductive approaches to causal inference with DAGs are controversial. Combining with traditional deductive approaches using background knowledge, theoretical frameworks, conceptual models, or other inference methods is recommended.

Graphical methods for causal inference are particularly relevant in analyses examining epidemiological, sociological, or economic data as elements of a CRA, considering existing work in those disciplines (e.g., Greenland et al., 1999; Morgan and Winship, 2014). Epidemiologic research, for example, can characterize the impacts of socioeconomic factors and other nonchemical stressors along with chemical stressors in human populations and will often be a prominent source of information when undertaking a CRA (Levy, 2008; NRC, 2009). Although epidemiological studies provide useful information in the species of interest for human health CRAs, toxicological studies, ecological studies, mechanistic studies, and other lines of evidence are also important sources of information. These sources can include both qualitative and quantitative data, and both types of information are valuable for elucidating pathways in a DAG to ensure accurate characterization of risk and effective interventions in CRAs.

Fig. 3 depicts a DAG that might be developed to analyze the same hypothetical community near the CAFO. The different pathways and variables demonstrate that, while the conceptual model might be useful initially in the CRA, a DAG may be more suitable further into the process, after more data are evaluated or additional scoping decisions have been made. Here, scoping decisions led the analyst to focus on odor emissions from the CAFO and the pathways through which odor exposure could cause chronic bronchitis, considering socioeconomic factors and other causal linkages.

Fig. 3.

Example DAG for a CRA in a hypothetical community near a CAFO with one health outcome of interest, Chronic Bronchitis. The DAG was generated using DAGitty.net (Textor et al., 2011), which allows variables to be labeled as exposure, outcome, unobserved (in gray), adjusted (not shown), other (not shown), or confounders (not shown). Model code in Appendix A. Abbreviations: NH3 = ammonia; H2S = hydrogen sulfide.

3. Key elements of DAGs

The remainder of this paper focuses on important causal concepts and implications of pathway structure within the context of a CRA. Appendix B summarizes DAG terminology and related definitions. Applications of graph theory are not discussed further but are described by Pearl (2009), Greenland et al. (1999), Glymour (2006), Morgan and Winship (2014), and Elwert (2013).

In brief, three essential structures exist in a DAG. The causal pathway, X → A → Y, the backdoor pathway, X ← A → Y, and the blocked pathway, X → A ← Y. Guidance for analytical decision-making are provided for each pathway below:

• In a causal pathway X → A → Y, do not adjust for intermediate A. This creates overcontrol bias by blocking the flow of effects between X and Y.

• In the backdoor pathway X ← A → Y, adjust for common cause A which is creating a false association between X and Y. Failure to do so causes confounding bias. In Fig. 3, the pink biasing pathways are backdoor pathways.

• In the blocked pathway X → A ← Y, do not adjust for common effect A, or a descendant of A. Adjusting for A in this pathway induces a false association between X and Y, also called collider bias. Adjusting for a collider would create a backdoor or biasing pathway in a DAG.

The term confounder will be used in the remainder of the paper and is defined as a variable that is both a common cause of the outcome and associated with the exposure of interest. Graphically, a confounder will appear as a common cause of the both the exposure and the outcome, for example, in the backdoor pathway X ← A → Y variable A is a confounder (Hernán and Robins, 2014). Confounders are typically “controlled” or “adjusted” using various study design features and analytic methods, including randomization, restriction, standardization, statistical adjustment, stratification, propensity scores, inverse probability weighting, or a combination of these as with doubly robust estimation. There is an extensive literature on causal estimation methods for different analytical needs (e.g., Pearl, 2012; Vanderweele and Vansteelandt, 2014; Robins et al., 2000; Imbens, 2000; Breen et al., 2013; Ye et al., 2012; Funk et al., 2011; Glymour, 2001).

3.1. Overview of DAGs and causality in CRA

The key to causal inference using Pearl (2009) graphical methods is the identification of causal effects. Because observed associations can be a combination of causal relationships and spurious associations, the identifiability of the causal effect depends on the appropriate treatment of confounding and any other source of bias from the observed relationship (Elwert, 2013). The ability to identify causal effects means that the distribution of post-intervention effects is identifiable from the distribution of available data (Pearl, 2010). Essentially, if bias can be effectively controlled in the available data, then a formal causal determination can be made based on the available evidence and quantitative modeling can be conducted (see Section 6 for more details on estimation in the analysis phase of CRA). A DAG, used to represent the causal structure of observed data, cannot by itself provide quantitative predictions. Rather, it increases confidence that a causal relationship exists, that the chosen estimation model can be used to accurately estimate a causal effect, and that a proposed risk management plan will have the intended effect. Predicting the effects of an intervention is an exercise in causal inference – calculating what we want to know (i.e. the counterfactual) from what is known (i.e. the factual) (Greenland et al., 1999; Rubin, 1974; Holland, 1986). Evaluating intervention options based on our hypothetical scenario is equivalent to asking a counterfactual question such as, “what will be the effect on bronchitis incidence in the community after reducing exposure to odors from the CAFO to a certain level”? The answer to this risk management question is a causal effect because the act of intervening is itself essential in establishing causation (see discussion in Introduction on experimental studies).

DAGs are unique because they allow for both the translation of statistics into causal language, and causation into statistical language. An association alone cannot describe causation, this is inferred by the investigator and is transcribed onto a DAG. If, for instance, an association is observed between X and Y, then a DAG could indicate that X causes Y, that Y causes X, or that X and Y have causes in common. If no statistical association is found between X and Y in a study, a representative DAG would not connect variables X and Y by arrows. Likewise, the ability to infer statistical relations from the causal connections drawn in a DAG is important because they are the testable implications of the causal model (Pearl, 2009). If one can read from the structure of a DAG that X and Y are conditionally dependent given Z (X → A ← Y), this means that the data must show that X and Y are associated when adjusting for A (this test intentionally creates collider bias to evaluate the structure of a causal relationship—an inductive method for causal hypothesis checking, that being X and Y are common causes of A).

According to graphical methods for inference, the causal effect of an exposure is estimable from observed data if biasing pathways are identified and blocked in a DAG. Pearl (2009), Elwert (2013), and Greenland et al. (1999) describe steps for detecting confounders in a DAG and identifying sets of variables, called adjustment sets, that are minimally sufficient to control bias so that the causal effect may be identified. Free DAG-building software is publicly available that lists testable implications and minimally sufficient adjustment sets including DAGitty used here (Textor et al., 2011). DAGitty uses the directional separation or “d-separation” criterion developed by Pearl (2009) to determine if the path between exposure and outcome variables are causally connected, and what (if anything) needs to be adjusted to block any pathways that contain confounders. The DAG in Fig. 3, for instance, has 14 testable implications, and two adjustment sets (or two variables in this case, each variable is the complete set) that are sufficient for estimating the total unbiased effect of odor exposure on chronic bronchitis: Land Use/Zoning; Education Level. The adjustment sets indicate what variables can be adjusted to block biasing pathways, not what would be most effective among the identified sets. If only one set is identified, that is the only set that will block biasing pathways. Additional data or scientific judgments may be needed to choose appropriate adjustment sets.

Statistical or graphical tests may be necessary to determine what variable would be most effective at reducing bias for the different adjustment sets (Farrar et al., 2014; Pearl and Paz, 2014). For example, the adjustment sets Land Use/Zoning and Education Level may both help adjust for SES-related indicators. Although both indicators may conceptually capture SES, data available for measuring one variable may provide more accurate information. In this case, the variable with the least amount of measurement error should be selected. Alternatively, both variables may be adjusted as long as it does not result in overadjustment bias (Schisterman et al., 2009). See Appendix C for guidance on using DAGitty to check for overadjustment when using more than one adjustment set. The ability to block biasing pathways via statistical adjustment of confounders is also contingent on the quality of data on confounding variables and the functional form for which they are being considered in a statistical model (See Chaix et al., 2010; Hernan et al., 2002 for further discussion of confounding evaluation and residual confounding).

Theoretically, causal mechanisms are defined by nature and causal models will correspond to the true underlying causal structure to the extent that relationships can be accurately observed and approximated (Pearl, 2009). Therefore, asserting that a DAG is a causal model is asserting that all variables, measured and unmeasured, have been depicted including all common causes that are significant to a cause-effect relationship (Elwert, 2013; Pearl, 2009).

4. Building DAGs for CRA

4.1. Treatment of bi-directional arrows

A common difficulty in developing DAGs arises with reciprocal effects, typically indicated using a bidirectional arrow in conceptual models. A solution is to extend the cycle over time using single-headed arrows, in order to depict a more realistic representation of a cyclic relationship. The conceptual model in Fig. 2 contains two bidirectional arrows—the arrow between “dose” and “biological pathway” reflects the complexity of the dose-response relationship and the other joins economic effects at the community and individual level. While the economic effects are separated into community and individual headings, no successful intervention on only one contextual level could be achieved without understanding their causal ordering. The relationship Tourism ↔ Property Value from the conceptual model implies that declining tourism causes decreased property value and vice versa. To remove the double-headed arrow, Tourism and Property Value might be examined annually and the graph modified to reflect the relationship extended over time (Fig. 4). This interpretation allows cause to precede effect; declines in Tourism decrease Property Values the following year and decreased Property Values subsequently affects Tourism. As noted in other publications (Pearl and Robins, 1995; Robins, 1997; Robins et al., 2000), this same approach can be extended to time-varying exposures to eliminate feedback loops.

Fig. 4.

DAG for extending variables over time to eliminate bidirectional arrows. Adapted from Greenland et al. (1999). Figure generated using DAGitty.

One of the objectives of CRA is to better reflect real-world exposures to multiple chemical and non-chemical stressors; this requires careful consideration of many complexities including the potential for temporal ordering of exposures, interactions, and feedback loops. Feedback loops are usually “short hand” for more complicated relationships and may include processes that amplify (positive feedback) or correct (negative feedback) a problem (Sterman, 2006). As illustrated in the example above, some feedback loops can be explained in a DAG as a linear relationship of two or more variables measured at different points in time; other feedback relationships may be explained through more detailed causal modeling (e.g., Mulatu and Schooler, 2002; Fig. 4). Elucidating feedback in a DAG can clarify that, for instance, an intervention early in life may not have the same effect as an intervention later in life. Marginal structural models are available for estimating causal effects for time-varying exposure scenarios where the exposure(s) of interest might also be considered a confounder or causal intermediate (Robins et al., 2000). A number of statistical packages (e.g., Mplus) are also available for estimation of reciprocal effects, but if DAGs are used to support inference for reciprocal effects, any bidirectional arrows would need to be removed and the pathways revised to allow for identification (as in Fig. 4). System dynamics models may also be effective at describing and addressing the complete nature of feedback in complex systems (Galea et al., 2010; Homer and Hirsch, 2006; Sterman, 2006). Resolving feedback loops is a challenge and is an area of concern when developing DAGs for causal inferences in CRA, particularly as the resolution of such loops influences model predictions. Regardless of the approaches used, detailed mapping and ordering of these important causal processes is critical to understanding assumptions about the underlying causal structure and specific mechanisms involved in a CRA.

4.2. Hierarchical DAGs and creating DAG modules

Suter (1999) describes the construction of a series of hierarchical conceptual models (not to be confused with hierarchical statistical models or multilevel analysis) to depict complex relationships in risk assessments. The method can be adopted to make complicated exposure-response relationships in CRA more tractable by aggregating processes into higher level nodes. This can help stakeholders, assessors, and decision makers understand the “big picture” without becoming mired in mechanistic detail. For example, a successful clinical intervention to reduce chronic bronchitis in the community in Fig. 2 would require additional specification of the causal pathways between conceptual variables “dose” and “biological pathway,” but an intervention aimed at reducing the CAFO-associated exposures linked to chronic bronchitis might ignore this complexity.

A modular approach for combining conceptual models described by Suter (1999) can also be applied to CRAs by creating separate DAGs for important relationships at any point in the assessment for clarification, analysis, or communication of causal assumptions. The separate DAG modules can be assembled into a combined model or set of hierarchical models, expanding or contracting pathways based on the complexity necessary for the assessment. For example, these separate DAG modules may:

• Be created for evaluation of confounding in epidemiological studies needed for the analysis (see conceptual framework by Levy (2008)).

• Be used to determine if study data are generalizable to the population of interest (Bareinboim and Pearl, 2013a; Bareinboim and Pearl, 2013b).

• Depict biological mechanisms for clinical interventions.

• Detail sources, release mechanisms, and exposure pathways to identify effective methods for reducing exposures (e.g., reducing the population’s inhalation of CAFO odors as in Fig. 3).

• Depict characteristics and vulnerabilities of the population (e.g., demographic structure, lifestyle factors, etc.).

• Include MOA, toxicological interactions in chemical mixtures, interactions between chemical and nonchemical stressors and other pertinent data derived from a toxicity assessment.

Building DAG modules depicting the MOA for individual chemicals may be informative when evaluating exposure to chemical mixtures. The method may be useful for grouping chemicals with similar MOAs, for example, if it can be determined that the MOAs of the chemicals in a mixture converge on a specific biological target or endpoint. The adverse outcome pathway (AOP) framework is a promising source of additional toxicodynamic information useful for CRAs. AOPs use DAGs to depict existing knowledge regarding the linkage between a direct molecular initiating event and an adverse outcome (Ankley et al., 2010; Garcia-Reyero, 2015).

Other techniques may be used to display additional information in a graphical model. For instance, the sign of a relationship may be indicated by a plus or minus added next to the arrow between two variables, and be used to determine the expected impact of an intervention (Vanderweele and Robins, 2010). Annotating the WOE for a relationship with different arrow colors, thickness, or designs (e.g. dashed, dotted) may also be a useful augmentation for a DAG (Joffe and Mindell, 2006).

4.3. Treatment of latent variables in CRA

Some variables important to a causal explanation may not be directly measureable. In Fig. 3, SES is marked as a latent variable, because it is a complex multi-dimensional construct that cannot be directly observed. Instead, Education Level provides a surrogate measure of SES, and SES is depicted in the DAG as being an unobserved cause of an individual’s educational attainment. In some circumstances, the causal mechanism explaining the observed association may be more complex, including biological mechanisms causing the disease, other social constructs affecting educational attainment and in the broader sense, the circumstances placing an individual near a CAFO (environmental justice considerations that a CRA could evaluate). If the causal mechanism for the statistical association is known a priori, for instance, explanatory (marked unobserved/latent in a DAG) variables could be added to the DAG to aid interpretation of the relationship for decision making or intervention purposes.

Similarly in Fig. 3, Lung Inflammation is not directly observable, but Exhaled Nitric Oxide might be used as a biomarker of this outcome (Delen et al., 2000; Malerba and Montuschi, 2012; Ryan et al., 2007). Because lung inflammation could be caused by odor exposure or smoking, among other factors, evaluating exhaled nitric oxide data could help determine the potential risk of developing chronic bronchitis in both smokers and nonsmokers who have not yet developed the disease, and in characterizing exposure in those who spend time outdoors or live or work near the CAFO. The presence of the biomarker in this pathway is another example where data across a variety of disciplines can help inform CRAs and lead to a better understanding of underlying causal structures and mechanisms.

While the surrogate Education Level serves as a marker for an unobserved variable (SES), the biomarker Exhaled Nitric Oxide serves as an indicator of a disease process (lung inflammation to chronic bronchitis). Both of these represent latent variables and are depicted in the DAG in the same manner as all other variables, following the causal ordering determined appropriate for the pathway and at a level of detail needed for the assessment. In these two examples (and in many other situations), the appropriate level of complexity needed in the DAG may depend on scoping decisions and/or intervention options.

Generally, variables important to a causal explanation but not included in the analysis should be marked unobserved. As shown in Fig. 3, the variable CAFO is marked as an unobserved variable because it is not included in the analysis although it is an observed explanatory factor (marking a variable unobserved assures the program will not include CAFO in the adjustment set output).

5. Developing the DAG in the planning, scoping, and problem formulation phase

We address the following three general areas where DAGs can be used to support Planning, Scoping and Problem Formulation: 1) DAGs can be used to concisely diagram the scope of the problem including relevant sources, stressors, pathways, receptors and health endpoints; 2) DAGs can be used to evaluate opportunities for risk management input into this phase of a CRA (e.g., intervention points in the pathways from sources, to stressors, to health outcome/s); and 3) DAGs can facilitate the planning of targeted data collection activities.

In the initial stages of a CRA, conceptual models can provide an overview of the various components contributing to an adverse health outcome. For example, Fig. 2 displays multiple relationships identified from previous studies of the effects of CAFOs on human and ecological health that may be generalizable to the community under study, along with linkages from theoretical frameworks (Section 2). DAGs can begin to detail relationships selected from the conceptual model based on scoping decisions in the planning, scoping, and problem formulation phase. In this phase, scoping decisions made using decision analysis or feasible risk management options for the intended level of the CRA (i.e., community, state, federal) (Barzyk et al., 2015) will often reduce the number of variables considered in a conceptual model that will be included in the CRA making it more tractable. Other factors, such as legal considerations or regulatory purview, may also reduce the scope of variables considered for regulatory intervention. DAGs are useful for evaluating causal structures regardless of the type and scope of a risk assessment.

After a DAG is drawn depicting the causal ordering of selected variables, the conceptual model or relevant theoretical frameworks should be consulted to determine if important common cause variables (confounders) are in the model or would need to be included to obtain an unbiased effect estimate. Scoping should be done so that variables that allow for unbiased risk estimation are not excluded from the assessment even if important causal factors may be of less interest for risk management alternatives.

The final output of the Planning, Scoping, and Problem Formulation phase of a CRA is the risk analysis plan (Fig. 1). This plan describes how hypothesized pathways or “risk hypotheses” presented in the conceptual model will be examined in the analysis phase (U.S. EPA, 2003). The risk hypothesis for the CAFO community example is that air emissions from the CAFO are causing an increased incidence of chronic bronchitis in the surrounding community. The use of DAGs in this phase can facilitate the prioritization of targeted data collection activities (e.g., literature-based data analyses or collection and analysis of samples from the community) depending on what variables identified in the DAG need to be observed in order to estimate the causal effect. Testable implications derived from the DAG can also be employed to analyze the validity of the causal assumptions resulting from the planning scoping and problem formulation process. For instance, according to the structure of the DAG in Fig. 3, Education Level should be conditionally independent of Perception of Health Risk given Odors, meaning if the community were not exposed to Odors, Education Level would not be associated with a heightened Perception of Health Risk in observed data. If an association remained following statistical adjustment in the underlying data, it would mean that there is an additional causal factor or pathway that was unaccounted for, or residual confounding was present (e.g., due to measurement error). Therefore, some aspects of translating the problem to a DAG may be iterative. Testable implications not only aid in confirming that causal assumptions are reflected in data, but also helps refine the causal explanation for the problem. Causal hypotheses can begin with expert opinions and then be re-evaluated based on data – reasoning about causal relationships based solely on data can lead to incorrect conclusions, just as data availability may lead to the exclusion of relevant relationships. If available data are insufficient (e.g., subject to considerable measurement error) for adjustment, or if key studies did not properly account for a confounder, it should be identified as an uncertainty in the risk characterization phase of a CRA.

The two adjustment sets for Fig. 3 (Land Use/Zoning; Education Level) discussed in Section 3.1 highlight another use of DAGs for the analysis plan—the ability to identify multiple variables that are sufficient to block biasing pathways. This capability allows investigators to identify what variables should be measured to obtain unbiased effect estimates, to choose what variables to measure based on the cost or impracticality of measurement, to aid in planning of data collection activities, and as part of the development of a data analysis plan (Elwert, 2013; Pearl, 2010).

6. DAGs in the analysis phase

The analysis phase of CRA integrates the traditional aspects of the single-chemical risk assessment, namely hazard identification, dose-response assessment and exposure assessment. In this section, causal inference with DAGs will be described under these three traditionally separate phases of risk assessment for clarity. We address the following general areas where DAGs can be used to support the Analysis Phase: 1) Testable implications derived from DAGs can add to WOE for causal pathways, and in conjunction with WOE methods can increase transparency of evidence evaluation for hazard identification; 2) DAGs can aid in selection of relevant studies for dose-response estimation by comparing study characteristics to population of interest, and identify if causal effects are estimable for total, direct, indirect, intervention effects, and effect modification; and 3) DAGs can detail factors that influence exposure, improving identification of confounders, and delineate aspects such as time-varying exposures and biomarkers.

6.1. DAGs and hazard identification

In CRAs, the process of determining whether exposures to stressors can cause an increase in the incidence of specific health outcome entails the examination of multiple lines of scientific evidence. WOE approaches are commonly used to determine if statistical associations can be considered causal. The use of graphical techniques for causal inference such as DAGs can be used in conjunction with WOE methods. WOE evaluations can be used to either confirm or revise the pathways drawn in a DAG. An iterative approach to DAG development will reflect the most robust available evidence that will enable the analyst to answer causal (i.e., counterfactual) questions about the risk hypothesis.

CRAs will draw on many different sources and disciplines for information to evaluate exposure- or dose-response relationships for hazard identification, including previous knowledge and expert opinion. The use of DAGs to enhance the process of evidence selection and interpretation can increase transparency of WOE evaluations that utilize expert judgment. For WOE narratives, as described in USEPA’s Guidelines for Carcinogen Risk Assessment (U.S. EPA, 2005), DAGs could provide an easily understandable visual reference as part of a causal determination.

Using graphical techniques for testing assumptions used in CRAs (i.e., testable implications related to conditional independencies) also can provide additional insights when conducting a WOE assessment. For instance, a testable implication that is confirmed by empirical data in a selected study is an important data characteristic that could add to the WOE evaluations related to a specific causal pathway. Described in Section 4.2, creating separate DAGs may be helpful for evaluating confounding in studies, and determining if study data are applicable to the target population. Graphical methods for removing bias and testing assumptions combined with WOE evaluation provides support for making causal determinations for hazard identification and for selecting studies for dose-response estimation.

6.2. DAGs and dose-response assessment

In CRAs, the likelihood and severity of adverse health outcomes based on stressor exposure levels in a target population is developed during the dose-response assessment in the analysis phase. When multiple potential dose-response data sets are available for analysis, risk analysts will need to identify and justify the data that are the most relevant to derive a dose-response estimate or a reference value for the assessment scenario. Once key studies are identified and determined to be internally valid, external validity (i.e. generalizability) would be considered in relation to the target population of the CRA for dose-response characterization. The variables identified in the DAG could be compared to the variables that are controlled for across various studies that examine outcomes associated with the same primary exposure(s) (Bareinboim and Pearl, 2013a; Bareinboim and Pearl, 2013b). Additional assumptions and considerations, however, may be needed if the relationships between certain covariates, including confounders, have different conditional dependencies and/or distributions than the available empirical data. Such explicit comparisons could facilitate the choice of the most accurate and relevant studies to derive the dose-response estimate. DAGs can further aid in determining if relationships between intermediate variables in indirect pathways of interest are causally identified for estimation of direct and indirect effects. Selecting studies that performed analysis of direct and indirect effects can ensure consideration of multiple stressors and important causal mechanisms for dose-response estimates. Appendix C provides further explanation of how analysis of direct and indirect effects can inform casual inferences for dose-response assessment in CRAs (Pearl, 2014; Richiardi et al., 2013).

6.2.1. Effect modification and interactions

A key characteristic of CRAs is addressing both the variability in dose-response relationships in susceptible populations, and the effects of interactions among multiple stressors. Just as feedback loops or bidirectional arrows are simplifications for complex or incompletely understood relationships, effect (measure) modification is a statistical concept that can also indicate the presence of an underlying biological mechanism. Effect modification occurs when there is a difference in the magnitude of an association between an exposure and an outcome within levels of a third variable; this could include factors that alter an individual’s response to exposure including co-occurring exposures, preexisting health conditions, presence of a genetic polymorphism, or other life-stage related factors (EPA, 2016).

In CRAs, causal identification of interactions and effect modification is important for capturing the complexities of exposure to multiple stressors as well as the accurate characterization of risk. If the biological process underlying effect modification is known or can be hypothesized, it can be transcribed in a DAG (e.g. AOPs) and will aid causal identification. Even if the underlying mechanism cannot be fully elucidated, procedures for identification of effect modification and interactions are available (Vanderweele, 2009). These methods can be applied to analyze a DAG for CRA in DAGitty using steps described in Appendix C.

For example in Fig. 3, effect modification by smoking would not be causally identified. Smoking is a collider and introduces bias when adjusted (Greenland, 2003; Greenland and Pearl, 2011). However, the causal effect of an interaction between Smoking and Odors on Chronic Bronchitis is identified, but only if Chronic Stress is measured. In a DAG, an interaction is treated as an additional exposure that can be intervened upon (Vanderweele, 2009). In the context of CRA, this makes sense if thinking about multiple stressor interactions, particularly if risk management options included plans to reduce exposures to multiple stressors. The total effect of both Odors and Smoking can then be estimated conditional on either Education Level or Land Use/Zoning, and Chronic Stress (or a surrogate measure of Chronic Stress). If no resources were available to measure Chronic Stress, then the causal effect of the interaction would not be identified. This situation demonstrates the value of properly mapping and defining variables early in the CRA, and determining if important relationships are causally identified so that data collection and analysis plan can include risk management considerations.

6.3. DAGs and exposure assessment

In the analysis phase of a CRA, the magnitude, frequency, and duration of exposures to multiple (and potentially disparate) stressors are analyzed. Creating a DAG that depicts factors that influence exposure can improve identification of possible confounders (see modular approach in Section 4.2). Such variables could include environmental features unique to the location, seasonal changes, weather patterns, or social issues specific to the community that may cause variability in exposure. Other aspects of exposure that are important for causal identification may include temporality issues (e.g., time-varying exposures), and other environmental fate and transport considerations (e.g., stress- or bioaccumulation or degradation over time).

DAGs may help to better delineate specific exposure routes identified in a conceptual model that lead to an estimated dose, and, as in Fig. 3, depict where a biomarker of exposure (as well as markers of susceptibility or effect) may lie in a causal pathway. Odors are comprised of a complex chemical mixture and is represented by a single node in Fig. 3; if the mixture components were of interest, exposure pathways could be mapped individually for each chemical in a DAG (see Fig. C.1 in Appendix C for example). Further evaluation of an exposure may also help identify other sensitive populations that may not have been considered previously, for instance, exposure in different age groups or occupational settings (CAFO workers, nearby farmers). While beyond the scope of this paper, these issues are addressed elsewhere (U.S. EPA, 1992; Paustenbach, 2000).

7. DAGs in the risk characterization phase

A risk characterization in a CRA includes an integrative analysis that describes the technical aspects of the predicted risks and their uncertainties. The integrative analysis includes discussion of the quality and relevance of the collective information and identifies information gaps, as well as important uncertainties at the interfaces between different process steps and the appropriateness of the different levels of analysis across the steps of the risk assessment. Importantly, it also discusses variability in risk, for example if dose-response differences were detected or would be expected across vulnerable populations. Uncertainty factors are prominent features of many risk assessments, and can be used, for example, to address human variability if the key studies used for quantitative purposes did not allow for definitive risk estimates across different sub-groups. As noted in the Section 6.2.1, if effect measure modification is present in the target population of a CRA and can be causally identified in a DAG for a relevant study, this may reduce the need for use of uncertainty factors related to human variability.

Identifying all important assumptions in a CRA is essential for a thorough uncertainty analysis (Burns et al., 2014; U.S. EPA, 2003). DAGs make assumptions about the scope of the assessment explicit through the factors included in the model. Although DAGs by themselves cannot express quantitative uncertainty about the functional forms of relationships (e.g., dose response), application of Bayesian networks to DAGs constructed for CRAs potentially can contribute to uncertainty analysis, because the uncertainty related to how a variable was observed or measured is clearly represented by its probability distribution, and uncertainty related to the accuracy of expert opinions can also be expressed (Fenton and Neil, 2013).

Some CRAs, initiated to provide quantitative estimates of risk, may not be completed or may be limited to qualitative/semi-quantitative assessments due to missing data or different knowledge gaps. If the amount of missing information is not preclusive, data-poor variables marked in a DAG as unobserved would be easily communicated as uncertainties and areas of needed research. Likewise, in the absence of data on the impact of a confounder, knowledge that an unmeasured confounder may exist is helpful in explicitly identifying the potential sources of uncertainty that may preclude the use of certain epidemiological studies for dose-response quantitation or may impact the overall characterization of risk and uncertainty based on the CRA estimates. Other sources of data, including the expected proportion of the population exposed to a confounder and the underlying strength of associations between the confounder and exposure and the confounder and outcome, can help determine the potential magnitude and direction of confounding bias in key studies where these data are lacking. This information could be used to decide whether to pursue quantitative risk characterization. Alternatively, if confounding identified in studies used for the CRA cannot be controlled or the impact of the potential bias cannot be estimated, these studies could be excluded from the analysis. Using DAGs to infer causal effects for exposure and reference doses can improve confidence in hazard indices or other quantitative risk characterization methods.

8. Challenges of using DAGs in CRA

WOE methods are themselves broadly used approaches for causal inference and have been extensively modified for particular applications (e.g., Weed, 2005). Graphical causal modeling approaches for inference in CRA may also require similar refinements depending on their use. Numerous additional techniques for inferring causation can be found in the literature. For example, Norton et al. (2015) describe causal assessment in the ecological context, an important component to consider for CRAs addressing risks to both human and ecological health. Krieger and Smith (2016) advise cautious use of DAGs for causal inference, and encourage integration of evidence from a variety of disciplines, or ‘triangulation’, and also suggest an inductive approach to causal inference called inference to the best explanation. They also provide examples where flawed logic and incomplete explanations, represented in DAGs, led to flawed conclusions (Krieger and Smith, 2016). Emphasis should, therefore, be placed on accurate characterization of the underlying relationships depicted in DAGs so these can be used with other causal inference methods tailored to the application of interest that best addresses health risks.

The use of DAGs as an inductive approach to discover causal structures directly from conditional independence properties derived from data are also controversial (Dawid, 2009). Dawid (2009) distrusts “Pearlian” DAG’s abilities to coincidently represent both conditional independence and causal properties of a system. The author proposes to augment DAGs with an intervention node to distinguish between DAGs expressing causal relationships and DAGs describing conditional independencies. Dawid (2009) also notes that dependent relationships derived from observational data cannot be called causal without strong assumptions, and the assumption that all measured and unmeasured common causes are included in the model cannot be tested. Although not necessarily a challenge in using DAGs, DAGs cannot convey the magnitude of risks, nor can they convey the distributions of variables, or functional forms of relationships (Elwert, 2013).

Nearly all scientific disciplines that evaluate causality must make strong assumptions, but this should not discourage risk analysts from using DAGs as a tool for expressing assumptions about structure of causal relationships. Concerns expressed by Dawid (2009) and others demonstrate the importance of carefully considering and communicating both causal and statistical assumptions (e.g., see Russo (2009)) used to support inferences in CRAs. Even if a relationship is not completely supported by all WOE criteria, or all assumptions are not completely met, it may still be informative in the hazard identification step of a CRA and can still suggest a causal hypothesis for future study.

9. Conclusions

CRA is an important approach for evaluating health outcomes from exposure to multiple environmental stressors. Causal inference is a crucial step in developing accurate and scientifically defensible CRAs and ensuring successful evaluations of intervention options through CRAs. DAGs provide the logical structure necessary for making effective and reliable policy recommendations to address CRA challenges. Utilizing theoretical frameworks, conceptual models, and DAGs at the appropriate level of resolution can provide insight into the best opportunities for interventions. DAGs can support the evaluation of alternate interventions to address confounding factors, interactive effects, and multiple causal pathways (see Table 1).

Like some elements of a CRA, DAGs may be iterative and evolve with the assessment so that a final DAG would mirror the best scientific evidence available to describe the risks to the population. Although the true causal mechanism(s) of an exposure-response relationship might not be known and identification of causal effects is contingent on the validity of the causal DAG, it is important to the advancement of CRA to transparently depict all assumptions. Causal models built by risk assessors for a similar problem may differ, but this should not be viewed as a deficiency. Rather, it makes clear where assumptions or opinions are playing a role in construction of causal relationships. Similar causal models can be compared, and different causal pathways may be identified and used as a basis for further research. This transparency will help lead to identification of gaps in scientific knowledge, better study design and improved causal theory for risk assessment applications. Further development and application of DAGs for causal inference in CRA is needed.

Abbreviations:

CRA

cumulative risk assessment

DAG

directed acyclic graphs

CAFO

concentrated animal feeding operation

SES

socioeconomic status

MOA

mode of action

AOP

adverse outcome pathway

WOE

weight of evidence

NRC

National Research Council

USEPA

U.S. Environmental Protection Agency