Formalizing the Role of Agent-Based Modeling in Causal Inference and Epidemiology

Abstract

Calls for the adoption of complex systems approaches, including agent-based modeling, in the field of epidemiology have largely centered on the potential for such methods to examine complex disease etiologies, which are characterized by feedback behavior, interference, threshold dynamics, and multiple interacting causal effects. However, considerable theoretical and practical issues impede the capacity of agent-based methods to examine and evaluate causal effects and thus illuminate new areas for intervention. We build on this work by describing how agent-based models can be used to simulate counterfactual outcomes in the presence of complexity. We show that these models are of particular utility when the hypothesized causal mechanisms exhibit a high degree of interdependence between multiple causal effects and when interference (i.e., one person's exposure affects the outcome of others) is present and of intrinsic scientific interest. Although not without challenges, agent-based modeling (and complex systems methods broadly) represent a promising novel approach to identify and evaluate complex causal effects, and they are thus well suited to complement other modern epidemiologic methods of etiologic inquiry.

Editor's note: Invited commentaries on this article appear on pages 100and 103, and the authors’ response appears on page 106.

Although those in the field of epidemiology have long recognized that the production and perpetuation of disease occurs within complex population systems (1), much epidemiologic inquiry has sought to reduce such systems to a series of isolated and independent associational effects, from which biological, behavioral, or social causal processes are then inferred. The development of causal heuristics, from the Bradford Hill criteria (2), to the sufficient-component “causal pie” model (3), to the potential outcomes framework of disease causation (4), have all played central roles in the refinement of causal inference and the specific circumstances under which causation can be appropriately ascribed.

The approach shared by these frameworks is that a population health system can be compartmentalized, modeled, and analyzed as a series of cause-effect relationships, in which a particular condition (i.e., exposure) leads to a specific disease state (i.e., outcome). Causal frameworks, such as the sufficient-cause model, seek to examine particular causal elements within a population system (5), with the goal of identifying specific foci for public health intervention.

The paradigm of “compartmental” causation is deeply engrained in the public health structures, interventions, and clinical care models of the 20th and early 21st centuries (6). Notwithstanding the innumerable successes in population health facilitated by disease-causation models that parse and identify individual causal effect(s), many contemporary public health problems have proved difficult to solve despite, in some cases, decades of intervention. Examples include the rise and persistence of obesity (7) and the increasing costs and ineffective management of chronically ill patients (8). Although the reasons for these failures are many, it is increasingly clear that these problems are characterized by complex, multifactorial processes that are highly resistant to interventions that address only one or a few causal effects. Therefore, multiple, highly interdependent causal pathways, which are fundamental characteristics of complex systems (9), are integral to the challenges that public health now faces. To this end, the application of complexity theory to understand how disease is generated and reproduced within populations systems has begun to gain traction and currency (10).

Although first used within the context of infectious diseases (11), agent-based models (ABMs), which constitute one class of complex systems methods, have been utilized in other areas of population health, including chronic disease research and social epidemiology (12, 13). Their methodological congruency with macro- and eco-social frameworks that position health as a production of intersecting and interacting biological, social, and environmental factors (14) has also contributed to the increased use of these methods in public health (15). Examples of their successful implementation appear in diverse fields, such as substance-use epidemiology (16, 17), human immunodeficiency virus prevention (18), and the study of neighborhood-level effects on physical activity and diet (19–21).

CAUSAL INFERENCE IN THE FACE OF COMPLEXITY

Disease causes can be said to exhibit dynamic complexity if some or all of the following characteristics are observed: structural nonlinear relationships (e.g., phase transitions) between causes and outcomes; adaptivity, in that individual and population behavior can evolve based on past history; feedback loops, such that causal effects are magnified (i.e., positive feedback) or dampened (i.e., negative feedback) as disease processes progress; contextual effects, such that health outcomes are shaped by specific social, economic, and political contexts; and finally, a high degree of sensitivity to initial conditions (22, 23). Many causal inference frameworks in epidemiology assume that these features are absent (e.g., unidirectionality of causal effects, noninterference, etc.).

We argue, as have others (19, 24), that agent-based modeling holds promise to elucidate complex causal processes in epidemiology. However, considerable theoretical and practical issues impede the capacity of these methods to improve our understanding of disease etiology and illuminate new areas for intervention. First, the conditions under which agent-based simulations produce valid and meaningful estimates of average causal effects are poorly understood. Second, like in all types of epidemiologic modeling, the strength of causal inference relies on underlying assumptions, yet there exists little if any consensus on what these fundamental assumptions may be. Third, unified recommendations for testing these assumptions (and how failing to meet these assumptions might weaken causal inference in the presence of complexity) are poorly developed.

Although several authors have argued that agent-based modeling permits novel analyses of complex disease mechanisms (15, 25), to our knowledge, there exists no framework to evaluate the capacity of ABMs to provide valid estimates of average causal effects. Thus, drawing on the counterfactual theory of causation (26–29), the objectives of the present article are to: 1) formalize the adoption of counterfactual thinking as a foundational aspect of complex systems theory as applied in epidemiology and 2) introduce notation and terminology that we hope will be used to promote and articulate a more rigorous formalism in the estimation of causal effects in the presence of complexity. Finally, we document how agent-based modeling can address 2 of the key methodological challenges in causal inference: understanding the action(s) of multiple interdependent causal effects and examining causal processes that operate through interference. Although other complex systems methods, such as system dynamics and network analysis, have been used in different fields of public health (30), we focus our discussion on agent-based modeling, as the approach has become one of the most commonly used complex systems methods in epidemiology specifically (19).

SIMULATING COUNTERFACTUALS

ABMs simulate the behaviors and interactions of autonomous agents from which social structures and population-level outcomes emerge (31). In most epidemiologic applications of agent-based modeling, agents represent persons and are encoded with heterogeneous characteristics that can include endogenous phenomena (i.e., biological functions), spatial positioning, and various exposure and disease states (19). The multidimensional nature of ABMs, which can include time and space (as well as other high-dimensional covariates), can give rise to dynamic network effects (i.e., the formation and reconstitution of relationships between agents generate network structures that change over time). In epidemiologic applications, agent networks frequently represent contact patterns, spatial clustering, and other social processes of interest (e.g., the diffusion of health information).

Methodologically, ABMs function as an in silico laboratory in which the researcher inputs agent characteristics, specifies initial conditions, applies rules for agent-agent interactions, and programs static or transitory exposure and disease states. By running the simulation (i.e., a model “run”) many times and observing outcomes under different input parameters (collectively referred to as a “treatment”), the investigator can compare outcomes obtained from any number hypothetical scenarios. Given that all agents in the population receive the treatment under various scenarios specified by the researcher, the results obtained from an ABM are directly analogous to potential outcomes as defined in modern epidemiologic literature (32). Unlike observational studies in which only 1 treatment condition is observed, the outputs of an ABM can be obtained repeatedly and thus have direct interpretation as counterfactual outcomes (Figure 1). We define a specific condition or scenario of interest for which outputs are evaluated as an “agent-based counterfactual” treatment.

Figure 1.

Epidemiologic methods to estimate average causal effects. The black and white dashed areas indicate the subgroup of the population with disease; the dark gray areas indicate exposed/treated subjects; and the dotted gray area indicates that the exposure is present in the population before manipulation. In an observational study, average causal effects are estimable under the assumptions of exchangeability, consistency, positivity, and correct model specification. In a simulation study, causal effects are estimable under the assumptions of ergodicity and correct model specification.

NOTATION AND TERMINOLOGY

We define a population consisting of N > 1 agents. For each agent i = 1, … , N, a set of m = 1, … , M internal traits is described, such that the agent population at a time step t = 1, … , T can be represented by an N-by-M matrix S^t,

$${\mathbf{S}}^t=\left[\begin{array}{cccc}{s}_{1,1}^t& s_{1,2}^t& \dots & s_{1,M}^t\\ s_{2,1}^t& s_{2,2}^t& \dots & s_{2,M}^t\\ ⋮& ⋮& \ddots & ⋮\\ s_{N,1}^t& s_{N,1}^t& \dots & s_{N,M}^t\end{array}\right],$$

where each row describes the values for the set of agent i's M internal traits at time t. Traits are defined generally and can be continuous (real valued), nominal (categorical), or dichotomous and can represent sociodemographic characteristics, genetic traits, exposures, propensity to engage in some health behavior, social influence, etc.

At each time step t (t = 1, … ,T), each agent i interacts with a subset of the population {1, … , i − 1, i + 1, … , N}, described by an agent-agent interaction matrix K^t. That is, each element $k_{i,j}^t$ of K^t indicates whether agent i interacts with agent j during time step t, where i and j = 1, … , N. Note that K^t is not necessarily symmetric; for example, agent i may know j but not vice versa or information (e.g., disease transmission) may be unidirectional and “flow” only one way. The matrix can be dichotomous, where each element $k_{i,j}^t\in \{0,1\}$ defines an unweighted agent network, or take on other values, in which case the matrix represents a weighted network structure. We note that the special circumstance ${\mathbf{K}}^t={\mathit{I}}_N$ (i.e., K takes the form of an identity matrix) is analogous to the assumption of noninterference (i.e., one agent's exposure does not affect the outcome of others). In addition to the interaction network, agents can also be placed in different environments, represented by a matrix E^t, where each agent is located within one of p = 1, … , P possible environmental states at time t:

$${\mathbf{E}}^t=\left[\begin{array}{cccc}{e}_{1,1}^t& e_{1,2}^t& \dots & e_{1,P}^t\\ e_{2,1}^t& e_{2,2}^t& \dots & e_{2,P}^t\\ ⋮& ⋮& \ddots & ⋮\\ e_{N,1}^t& e_{N,1}^t& \dots & e_{N,P}^t\end{array}\right].$$

We note that the effect of spatial position is captured both by the agent-agent interaction matrix (i.e., agents in close proximity can influence each other) and by the environmental state matrix, which encompasses the effect of environmental factors that are independent of the location of other agents.

An ABM is initialized by populating the baseline agent trait matrix S⁰, environmental states (i.e., setting the matrix E⁰), and the interaction matrix K⁰ with values drawn from a priori–defined probability distributions and parameter functions. Random-number generators are used to define the model's initial conditions and distribute the traits, interactions, and environmental state values randomly in the agent population, conditional on the input parameter functions and probability distributions. A model run is defined as one execution of the model with a specific set of initial conditions (i.e., a random “seed”).

The simulation proceeds by defining a set of rules Z. Rules govern how agents update their internal states (agent rules), interact with other agents (agent-agent interaction rules), and move between or interact with environmental states (agent-environment rules). If we assume that the rules are stable and independent of agent behavior and interactions, the evolution of the model can be described by the functions

$$\begin{array}{rl}{k}_{i,j}^t& =g({\mathbf{S}}^{t-1},{\mathbf{K}}^{t-1},{\mathbf{E}}^{t-1},{\xi }_{i,j,t})\\ s_{i,m}^t& =f({\mathbf{S}}^{t-1},{\mathbf{K}}^t,{\mathbf{E}}^{t-1},{ϵ}_{i,m,t})\\ e_{i,p}^t& =h({\mathbf{S}}^{t-1},{\mathbf{K}}^t,{\mathbf{E}}^{t-1},{\zeta }_{i,p,t})\end{array}$$

for all agents i, j = 1, … , N, internal traits m = 1, … , M, environmental states p = 1, … , P, and time steps t = 1, … , T. The functions f(), g(), and h(), which include random error terms (ϵ, ξ, ζ) that are independent and identically distributed, map the space of all states at time t − 1 to the space at time t. Thus, the rules Z are coded in the model by defining functions f(), g(), and h() and by specifying the distribution of the random error terms. In other words, at each time step, the microsimulation updates the agent trait matrix, interaction matrix, and environmental state matrix based on previous values and on rules set by the modeler. For a given run r (r = 1, … ,R), after an arbitrary number of time steps T, the outcome(s) of interest can be evaluated in the agent population. The simulation can produce multiple outputs contemporaneously and can include epidemiologic outcomes of interest such disease incidence, prevalence, mortality, etc. The vector of O outcomes for run r at time T is denoted as ${\mathit{Y}}_r^T({\mathbf{S}}^{\mathrm{T}},{\mathbf{K}}^{\mathrm{T}},{\mathbf{E}}^{\mathrm{T}})=(y_{r,1}^T{y}_{r,2}^T\cdots y_{r,O}^T)$, which as shown is a function of the elements within matrices S^T, K^T, and E^T.

ESTIMATING CAUSAL EFFECTS IN THE PRESENCE OF COMPLEXITY

We now use counterfactual notation to define an average causal effect obtained from an ABM. However, instead of considering 1 counterfactual treatment condition or exposure level(s), we define an agent-based counterfactual treatment as a uniquely specified set of internal traits S, interaction matrices K, or environments E. In other words, agent-based modeling permits the examination of ensembles of counterfactual policy and programmatic scenarios, which may represent different populations (agent states), social interactions, environments, or combinations thereof. Specifically, we wish to compare the disease outcomes at time T of the agent population subjected to counterfactual scenario A (denoted as $[\mathit{Z},{\mathit{S}}_A^T,{\mathit{K}}_A^T,E_A^T]$) versus counterfactual scenario B (denoted as $[\mathit{Z},{\mathit{S}}_B^T,{\mathit{K}}_B^T,E_B^T]$).

A Monte Carlo simulation is used to obtain outcomes from runs r = 1, … , R at time T for counterfactual scenarios A and B, respectively. The expectation of outcomes for runs r = 1, … , R are obtained for both counterfactual scenarios A and B. Specifically, by executing the model R times, we define ${\hat{\mu }}_{o,A}^{T,R}$ and ${\hat{\mu }}_{o,B}^{T,R}$ for scenarios A and B, respectively, and outcome o (o = 1, … , O) as

$${\hat{\mu }}_{o,A}^{T,R}=\frac{{\sum }_{r=1}^R{y}_{r,o,A}^T}{R}$$

$${\hat{\mu }}_{o,B}^{T,R}=\frac{{\sum }_{r=1}^R{y}_{r,o,B}^T}{R},$$

where ${\hat{\mu }}_{o,A}^{T,R}$ and ${\hat{\mu }}_{o,B}^{T,R}$ represent point estimators for the expected value of the outcome of interest derived from R runs under hypothetical scenarios A and B, respectively, at time step T.

Given that the entire agent population is subjected to counterfactual scenarios A and B, exchangeability is guaranteed (i.e., the outcome observed in a set of simulations under scenario A is the same as the outcome that would have been observed in the set of simulations under scenario B if scenario A had been applied instead). Therefore, there exists a non-null average causal effect of scenario A compared with scenario B for an outcome o if

$${\hat{\mu }}_{o,A}^{T,R}\ne {\hat{\mu }}_{o,B}^{T,R}$$

for a sufficiently large agent population N and number of executions R at time step T. We now turn to our discussion to 2 key challenges in causal inference that we argue can be addressed using agent-based modeling.

Interdependence of causal effects

Many statistical methods in epidemiology, including regression-based models with “main-effects only” forms, assume that the actions of multiple causes are linear and independent. In part because of a historical lack of adequate statistical methodology to examine multiple causal mechanisms acting contemporaneously, epidemiologic inquiries are frequently limited to the estimation of one causal effect. If more than one cause is of interest, conventional epidemiologic study designs may require prohibitively large sample sizes to detect higher-order terms, and regression-based analyses typically make strict and frequently untestable assumptions regarding the functional form of the causal interaction (i.e., joint exposure) (33). Marginal structural models have recently been proposed as a plausible method to account for more complicated multicausal relationships (34), including the case in which the causes are interdependent (i.e., one cause affects both the second factor of interest and the outcome).

We argue that agent-based modeling offers an alternative and complementary approach to elucidate complex causal interdependencies that are of interest in epidemiology. Specifically, the forms of the relationships among causes (which are broadly defined here and can include agent traits as well as environments) are operationalized by the rules Z. The rule set consisting of functions f(), g(), and h() can include nonlinear components, including feedback loops, such that linear independence need not be assumed. By altering the rule set Z and running the simulation under different assumed causal relationships and processes, the effect(s) of interdependent (i.e., joint) exposures can be explored and interrogated.

By incorporating hypothesized actions of unmeasured confounders into the rule sets, simulation methods can also be used to confirm or repudiate previously observed causal mechanisms. For example, an ABM that simulates friendship formation has been used to examine the causal effect of social influence on adolescent obesity (35). In observational studies, this relationship has been confounded by homophily (the tendency of persons to select friends who are similar to themselves) and shared environmental influences (36). The study found evidence for social influence on adolescent body mass index and obesity-related behaviors, even after accounting for both shared environmental factors and multiple sources of homophily in friendship selection (35). Finally, constructing the ABM and defining causal processes may expose tacitly held assumptions regarding the form of the causal relationships (or lack thereof) in the analysis of observational study data and thus may illuminate novel mechanisms for empiric investigation.

Interference

A second important assumption of many counterfactual frameworks used to estimate causal effects is that of noninterference. Interference, otherwise known as “spillover effects,” is said to be present when the exposure or treatment assignment of an individual is influenced by the outcome(s) of others in the population. This assumption has long been characterized and is encompassed by the stable unit treatment value assumption (37). The assumption of noninterference is continually violated in the context of infectious disease epidemiology, in which an individual's risk of infection is dependent on other the disease statuses of others (38), and in studies of social processes and neighborhood-level effects (29, 39). Ignoring interference has been found to result in misleading or incorrect inference, given that the difference of the expected value of the outcome distributions derived from a treatment condition and an unexposed condition estimates not an average causal effect of treatment but the difference between the average effect among those exposed/treated and the spillover effect on the unexposed/untreated (39). Thus, an exposure or treatment can be inferred to be beneficial even when it is universally harmful. Moreover, spillover effects are often of intrinsic interest (e.g., in the case of herd immunity) and cannot be analyzed with methods that do not account for interference.

To address these issues, several methods have been developed. For example, one can estimate causal effects that are conditional on contact with an exposed individual, such that the potential outcome is a 2-stage counterfactual statement dependent on both coming into contact with an exposed person (due to interference) and treatment assignment (38). Recent work has also focused on developing inverse probability weighting estimators to obtain causal estimands of interest in the presence of interference (40).

We demonstrate here that agent-based modeling offers an alternative and appropriate method to examine spillover effects and to test how sensitive analyses are to the presence or absence of interference. In fact, given that an ABM explicitly accounts for interactions between heterogeneous units, elucidating the effects of interference should be a primary purpose of the application of agent-based modeling in epidemiology. Although a complete discussion is beyond the scope of this article, we note that the magnitude and influence of spillover effects can be examined by running simulations that assume different interaction matrices ${\mathit{K}}_A^T$ and ${\mathit{K}}_B^T$. For example, the magnitude of bias arising from an assumption of noninterference can be estimated by comparing the expected value of outcomes from scenario A, defined by $[\mathit{Z},{\mathit{S}}_A^T,{\mathit{K}}_A^T,{\mathit{E}}_A^T]$, to that from scenario B, defined by $[\mathit{Z},{\mathit{S}}_A^T,{\mathit{I}}_N,E_A^T]$. Specifically, the bias of an outcome brought about by ignoring interference at time T for a given interaction matrix ${\mathit{K}}_A^T$ is estimated by

$${\text{BIAS}}^T={\hat{\mu }}_{o,A}^{T,R}-{\hat{\mu }}_{o,B}^{T,R}.$$

Note that if no interference is present (i.e., the outcome is independent of the structure of the agent-agent interactions), the 2 terms will be equivalent and the bias arising from interference will be equal to 0.

DEFINING ASSUMPTIONS TO ESTIMATE AGENT-BASED CAUSAL EFFECTS

ABMs, like all statistical models, represent simplified abstractions of the reality and context in which disease is produced. In a manner analogous to traditional modes of epidemiology inquiry for causal inference, the assumption of correct model specification in an agent-based modeling approach implies that all relevant causal mechanisms (e.g., confounding) must be identified and incorporated into the model and that these mechanisms resemble those that operate in the real world (41). Demonstrating that the model reproduces empirically observed data (through calibration procedures) is one such method for testing this assumption, but we note there currently exists no consensus on best practice statistical or ad hoc methods to evaluate the validity of competing models.

Although several standard causal inference assumptions are not relevant in agent-based modeling (e.g., exchangeability is assured by design), a regularity assumption, referred to as ergodicity (i.e., that the means of the outcome across runs in well-defined), needs to be met. As described in the Web Appendix (available at http://aje.oxfordjournals.org/), counterfactual contrasts and corresponding causal effects are not well defined in models that exhibit nonergodic properties. In chaotic systems, nonergodic behavior can be observed even when differences in initial conditions across runs are small (41). It is likely that some population health systems and the corresponding ABMs constructed to analyze them will exhibit nonergodic, chaotic behavior and thus may violate the ergodicity assumption. In other disciplines, including economics, several authors have questioned the utility of agent-based modeling and counterfactual frameworks broadly, given that modern economic systems exhibit highly dynamic, nonergodic behavior that evolves over time (41, 42). We argue that agent-based modeling should not be seen as a panacea for the analysis of all complex systems in population health, particularly those systems for which phase changes, emergent properties highly sensitive to initial conditions, and other nonergodic behaviors are likely to be present. To this end, researchers utilizing agent-based modeling should conduct tests of ergodicity using recently published recommendations (43).

In addition to the important caveat that under high sensitivity to initial conditions, the assumption of ergodicity is violated, the nature of complex and dynamical systems is such that ABMs can be highly sensitive to model parameterization and inputs. Although a comprehensive discussion of ABM validation procedures is beyond the scope of this paper and is the topic of other previously published reports (41), we note that detailed sensitivity analyses are essential to establish model validity. In addition to examining how results depend on sensitivity to initial conditions and across-run variability (arising from stochastic elements in the model), current recommendations include investigating model robustness against changes in model rules (41). To advance the adoption of agent-based modeling approaches in epidemiology, we recommend the development of discipline-specific methodological standards and a minimally acceptable set of protocols for ABM construction, analysis, and validation, similar to those in other fields of science such as sociology (44).

DISCUSSION AND FUTURE RESEARCH DIRECTIONS

The intent of the present article was to illuminate under what circumstances agent-based modeling might be an appropriate method to examine causal effects in epidemiologic research. We have argued that agent-based modeling is of particular utility when interference and other stochastic person-to-person processes dominate the behavior of the system and thus influence exposure-disease relationships in critical ways. Additionally, ABMs can be used to examine the effect of multiple exposures that interact in nonlinear and dynamic ways to affect population-level health outcomes.

We do not wish to suggest that agent-based modeling is the only method (or even a superior method) by which dynamically complex processes can be explored in epidemiology. Rather, agent-based modeling is one of many recently developed approaches that seek to account for complex phenomena in population health. These include causal diagrams and marginal structural models (27), as well as novel approaches that assess interference using potential outcomes frameworks (40). Future research is required to determine under what conditions ABMs provide similar or novel insights into the causes of disease compared with traditional epidemiologic approaches and other modern methods.

Published studies in which ABMs have been used to capture complex disease processes (with results that provide practical insights into improved public health strategies) do exist. For example, an ABM was used to evaluate various mitigation strategies for pandemic influenza in the United States (45). The results of this model demonstrated that travel restrictions are unlikely to decrease the total number of ill persons within highly mobile populations but that the rapid production and distribution of vaccines, even if poorly matched to the circulating strain, could slow transmission and limit the number of ill persons to less than 10% of the total population (45). However, currently missing from the literature are comparative studies in which investigators interrogate an epidemiologic question with different types of causal inference models, including those that are agent-based. These may fruitfully be the focus of future work. For example, the “spread” of obesity in social networks has been examined using network analyses and standard regression-based approaches (46). Constructing and calibrating an ABM with these same data would permit direct comparisons of the assumptions made by each method and would also reveal specific situations in which the agent-based modeling approach may provide novel and important public health insights to curb obesity. Moreover, conducting comparative investigations will aid in the determination of the circumstances in which ABMs are likely to produce reliable results and valid causal inference.

CONCLUSIONS

In the present article, we sought to provide an inceptive formalism to the adoption of counterfactual thinking in the determination of disease causes as they operate within complex population systems. We have focused our discussion on 2 commonly made assumptions in counterfactual causal inference—independence of causal effects and noninterference—because agent-based modeling represents a novel and particularly apt way to tackle these challenges in modern epidemiology. As the role of these methods within an epidemiologist's expanding toolbox remains to be fully elucidated, their continued adoption in the field is and should be predicated upon their potential contribution to causal inference.