Agent-based models of inflammation in translational systems biology: A decade later

Abstract

Agent-based modeling is a rule-based, discrete-event, and spatially explicit computational modeling method that employs computational objects that instantiate the rules and interactions among the individual components (“agents”) of system. Agent-based modeling is well suited to translating into a computational model the knowledge generated from basic science research, particularly with respect to translating across scales the mechanisms of cellular behavior into aggregated cell population dynamics manifesting at the tissue and organ level. This capacity has made agent-based modeling an integral method in translational systems biology (TSB), an approach that uses multiscale dynamic computational modeling to explicitly represent disease processes in a clinically relevant fashion. The initial work in the early 2000s using agent-based models (ABMs) in TSB focused on examining acute inflammation and its intersection with wound healing; the decade since has seen vast growth in both the application of agent-based modeling to a wide array of disease processes as well as methodological advancements in the use and analysis of ABM. This report presents an update on an earlier review of ABMs in TSB and presents examples of exciting progress in the modeling of various organs and diseases that involve inflammation. This review also describes developments that integrate the use of ABMs with cutting edge technologies such as high-performance computing, machine learning, and artificial intelligence, with a view toward the future integration of these methodologies.

1 |. INTRODUCTION

This article is an update of a review from 2009 of a then-relatively novel method for computational modeling, namely agent-based modeling (An, Mi, Dutta-Moscato, & Vodovotz, 2009). In the decade that has passed, agent-based modeling has moved from the cutting edge to the mainstream, now constituting a well-accepted method for modeling biomedical pathophysiology, primarily at the cell-as-agent level. At the time of that initial review, we had also proposed the concept of translational systems biology (TSB; Vodovotz, Csete, Bartels, Chang, & An, 2008), an engineering-based approach (An, Faeder, & Vodovotz, 2008) centered on dynamic, mechanistic computational models with direct clinical applications (Figure 1). During the early years of TSB, the primary focus was on modeling the acute inflammatory response, especially in the context of sepsis, traumatic injury, and wound healing. Importantly, TSB is agnostic with regard to the modeling framework, be it equation-based or agent-based (An et al., 2008; Vodovotz et al., 2008). In the intervening decade, we have been grateful to see a large increase in both breadth and depth of applications consistent with tenets of TSB, that is, studies that are directed at representing specific, clinically identifiable disease states rather than a focus on elucidating a specific cellular or molecular mechanism (summarized in Table 1).

FIGURE 1.

Agent-based models (ABMs) of inflammation: past, present, and future. Initial (First-Gen) ABMs of inflammation served as tools for hypothesis instantiation, dynamic knowledge representation, and proof-of-concept tools for core translational systems biology applications such as in silico clinical trials and patient-specific models. Due to methodological and conceptual advances, next-generation (Next-Gen) agent-based modeling of inflammation is poised to advance further in the coming decade

TABLE 1.

Key properties of agent-based models (ABM) of inflammation discussed in this review

References	Modeled Entity	Model Scale	Model Type	Topology
(An, 2004)	Sepsis (virtual organism)	Whole-organism, population	ABM	2-d, On Lattice
(C. Cockrell & An, 2017)	Sepsis (virtual organism)	Whole-organism, population	ABM	2-d Grid, On Lattice
(An, Wandling, & Christley, 2013)	Lung (pulmonary inflammation)	Organ	ABM	2-d, On Lattice
(Fallahi-Sichani, Schaller, Kirschner, Kunkel, & Linderman, 2010)	Tuberculosis granuloma in lung	Organ	ABM	2-d and 3-d, On Lattice
(Cilfone, Kirschner, & Linderman, 2015; Marino, El-Kebir, & Kirschner, 2011)	Tuberculosis granuloma in lung	Organ	Hybrid	2-d and 2-d, On Lattice
(Brown et al., 2011)	Pulmonary inflammation and fibrosis, COPD	Organ	ABM	2-d, On Lattice
(Pothen, Poynter, & Bates, 2013, 2015)	Asthma	Tissue/Organ	ABM	2-d, On Lattice
(Song, Guo, Deng, & Li, 2012)	Asthma	Tissue	ABM	2-d, On Lattice
Warsinske et al., 2016	Pulmonary Fibrosis	In Vitro Tissue	ABM	3-d, On Lattice
(Ceresa, Olivares, Noailly, & Gonzalez Ballester, 2018; Wellman, Mondonedo, Davis, Bates, & Suki, 2018)	Pulmonary inflammation and fibrosis, IPF	Organ	Hybrid	2-d, On Lattice, Finite Element
(Dutta-Moscato et al., 2014)	Hepatic infection, inflammation, and fibrosis	Organ	ABM	2-d Off Lattice
(Goyal & Murray, 2016; Murray & Goyal, 2015)	Hepatic Infection	Organ	ABM	3-d, On Lattice
(Shi, Chapes, Ben-Arieh, & Wu, 2016)	Hepatic Infection	Organ	ABM	2-d, On Lattice
(Alam et al., 2015; Carbo et al., 2013; Mei et al., 2015)	Gastrointestinal inflammation	Organ	ABM	Abstract Organ Compartments
(C. Cockrell, Christley, & An, 2014)	Gastrointestinal inflammation	Organ	ABM	3-d, On Lattice
(R. C. Cockrell, Christley, Chang, & An, 2015)	Gastrointestinal inflammation	Organ	ABM / HPC	3-d, On Lattice
(Kim, Christley, Alverdy, Liu, & An, 2012)	Gastrointestinal inflammation and microbiome	Organ / ecology	ABM	2-d, On Lattice
(Peer & An, 2014)	Gastrointestinal infection and microbiome	Organ / ecology	ABM	2-d, On Lattice
(Seal, Alverdy, Zaborina, & An, 2011)	Gastrointestinal infection and microbiome	Organ / ecology	ABM	2-d, On Lattice
(Heberling, Dhurjati, & Sasser, 2013; Weston, Fogal, Cook, & Dhurjati, 2015)	Gastrointestinal infection and microbiome	Organ / ecology	ABM	2-d, On Lattice
(Shashkova et al., 2016)	Microbiome	Organ	ABM	Abstract Organ Compartments
(Gopalakrishnan, Kim, & An, 2013)	Soft tissue injury, inflammation, and wound healing	Tissue	ABM	2-d, On Lattice
(Li et al., 2008; Li, Vodovotz, Hebda, & Verdolini, 2010; Li et al., 2011)	Soft tissue injury, inflammation, and wound healing	Tissue	ABM	2-d, On Lattice
(Ziraldo et al., 2015)	Soft tissue injury, inflammation, and wound healing	Tissue	ABM	2-d, Off Lattice
(Martin, Blemker, Peirce-Cottler, & Physiology, 2015; K. S. Martin et al., 2017; Kyle S Martin et al., 2017)	Soft tissue injury, inflammation, and wound healing	Tissue	ABM	3-d, Off Lattice
(Solovyev, Mi, Tzen, Brienza, & Vodovotz, 2013)	Soft tissue injury, inflammation, and wound healing	Tissue	Hybrid	2-d, Off Lattice
(Seekhao, Shung, JaJa, Mongeau, & Li-Jessen, 2018)	Soft tissue injury, inflammation, and wound healing	Tissue	ABM / HPC	3-d, Off Lattice
(Bailey, Lawrence, Shang, Katz, & Peirce, 2009; Bailey, Thorne, & Peirce, 2007; Hashambhoy, Chappell, Peirce, Bautch, & Mac, 2011; Kleinstreuer et al., 2013; Peirce, 2008)	Angiogenesis and vasculogenesis	Tissue	ABM	2-d, On Lattice
(Alfonso et al., 2016)	Inflammation and oncogenesis	Tissue	ABM	2-d, Off Lattice
(An & Kulkarni, 2015)	Inflammation and oncogenesis	Tissue	ABM	2-d, On Lattice
(Gong et al., 2017)	Cancer-immune interactions, cancer immunotherapy	Tissue	Hybrid	3-d, On Lattice

The remaining article reports on developments in agent-based modeling of inflammation since 2009. We note that we have not incorporated immune system modeling per se, though this is an area that is related to inflammation. The initial section of this review lists many of the methodological and technological advances that have enhanced greatly the ability to create agent-based models (ABMs), perform simulation experiments, and extract knowledge from agent-based modeling (Figure 1). The next section provides an overview of ABMs concerning inflammation developed over the past decade, a fair number of which would not have been possible without the aforementioned computational advancements. Of note, many of these projects involve utilizing the ability of ABMs to readily represent spatial relationships and patterns, as key aspect of how biological systems are characterized (Table 1). This property allows an additional level of calibration and validation from histological or imaging data and enhances the utility of ABMs. We summarize key features of these ABMs in Table 1. The final section discusses the integration of ABMs with cutting edge developments in artificial intelligence as a pathway toward one of the primary goals of TSB: an engineering approach to the design of new and novel therapeutic interventions for disease.

2 |. METHODOLOGICAL AND TECHNOLOGICAL DEVELOPMENTS

One of the persistent limitations of agent-based modeling is the fact that they are computationally expensive (as compared to more traditional differential equation-based models). As such, the run time needed to execute ABMs limits the degree of fidelity (i.e., increased level of detail of number and type of components, and disease phenotypes represented) of the models, as well as the scale of the simulation experiments that utilize them (Table 1). The community has evolved in several directions in order to meet some of these challenges (Figure 1). The much-recognized increase in pure computing capacity (as noted by Moore’s Law) has vastly increased the resources available to individual researchers and groups; nearly all agent-based modeling at the academic level now utilizes local computing clusters. At a larger scale, advances in high-performance computing (HPC) and the advent of cloud-computing platforms can provide computing environments for previously unheard-of scale of simulation experiments. Specific agent-based modeling environments that have been developed to facilitate the implementation of ABMs on HPC systems include RepastHPC (https://repast.github.io/repast_hpc.html) and Biocellion (https://biocellion.com/), while the Extreme Model Exploration With Swift/T (EMEWS) framework provides a simulation-experiment environment that semi-automates various types of model evaluation/investigation workflows for use on HPC systems (Ozik, Collier, Wozniak, & Spagnuolo, 2016). In addition to technical advances, there is increased interest support for biological applications on HPC resources by United States Federal agencies such as the Department of Energy’s National Energy Research Scientific Computing Center and the National Science Foundation-supported Extreme Science and Engineering Discovery Environment (XSEDE) program.

Another approach that has been used to both increase computational efficiency and improved model fidelity is the increasing use of “hybrid” models that combine the discrete elements of an ABM with functions better represented by differential equations. This is a natural convergence between ABMs and equation-based models, as more detailed rules for ABMs are expressible as mathematical functions, while increased compartmentalization and spatial discretization increase the ability of equation-based models to capture the heterogeneity and structural features of biological systems (see Ceresa et al., 2018; Cilfone et al., 2015; Dutta-Moscato et al., 2014; Marino et al., 2011; Marino & Kirschner, 2016; Mayorga, Cebrian, Verma, Hoops, & Bassaganya-Riera, 2018; Solovyev et al., 2013; Wellman et al., 2018).

In addition to the ability to facilitate the development and use of increasingly sophisticated ABMs, there have also been methodological improvements for both improving ABMs as well as analyzing the output of simulation experiments utilizing them (Figure 1). These developments include work on uncertainty quantification in ABMs (Marino, Hogue, Ray, & Kirschner, 2008), sensitivity analysis in ABMs (Alam et al., 2015), methods for increasing the computational efficiency of ABMs via “tuneable resolution” (Kirschner, Hunt, Marino, Fallahi-Sichani, & Linderman, 2014), the use of Bayesian statistical model checking for parameter estimation in ABMs (Hussain et al., 2015), the use of optimization algorithms in conjunction with ABMs (Cicchese, Pienaar, Kirschner, & Linderman, 2017; R. C. Cockrell & An, 2018), the use of HPC (C. Cockrell & An, 2017; R. C. Cockrell & An, 2018; R. C. Cockrell et al., 2015; Petersen et al., 2019; Seekhao et al., 2018), strategies for data-driven model validation (Renardy et al., 2019), and the incorporation of model-based dynamic control discovery (R. C. Cockrell & An, 2018; Petersen et al., 2019). These are exciting developments that have, without a doubt, increased the range of biomedical problems and applications to which ABMs could be applied.

3 |. SYSTEMIC INFLAMMATION: RECONSIDERING TRAUMA AND SEPSIS

The earliest examples of agent-based modeling of inflammation examined system-wide disordered inflammation arising from either trauma or sepsis. The paucity of effective mediator-based therapies for these diseases keeps these older models relevant, perhaps even more so than when they were originally developed. Specifically, the Innate Immune Response Agent-based Model (IIRABM) from 2004 (An, 2004) predicted a series of behaviors associated with sepsis that have since been recognized in the subsequent years, specifically the temporal concurrence of pro- and anti-inflammatory cytokine responses (as opposed to sequential pro- and compensatory responses) (Osuchowski, Welch, Siddiqui, & Remick, 2006; Tamayo et al., 2011) and the importance of the immunoparalyzed recovery phase of sepsis, particularly with respect to its prolonged duration (Boomer et al., 2011; Ferguson, Galley, & Webster, 1999; Hotchkiss, Monneret, & Payen, 2013a, 2013b). The robustness of this model has allowed it to be used as a reference model in a more recent series of investigations into more fundamental dynamic mathematical properties of the behavior of sepsis (C. Cockrell & An, 2017). Specifically, this ABM has been used to recast the “problem” of heterogeneity in sepsis populations as a function of parameter space, with numerous implications including: the complex nature of the distribution of sepsis patients (i.e., non-Gaussian) when examined from the perspective of mediator or omic profiles, identifying the futility of the search for biomarker panels to predict sepsis outcome, and the boundary conditions for putative control strategies for sepsis (C. Cockrell & An, 2017). The finding from this work suggests that clinical data sets will perpetually undersample the possible biomarker configurations seen in sepsis, and that trajectory analysis is vital for rationally predicting and eventually treating the disease.

4 |. ABMs OF INFLAMMATION AND INDIVIDUAL ORGAN SYSTEMS AND DISEASE PROCESSES

Despite its systemic nature, sepsis and trauma both invariably are decomposable to dysfunction manifesting in individual organ systems (hence the terms “multiple organ dysfunction syndrome” and “multiple organ failure”). Also, many inflammatory diseases are directed at and originate from individual organ systems, with specific characteristics and features related to the organ system, tissue, or pathophysiological processes. The development of specific organ ABMs is the largest growth area in the use of ABMs in the past decade, reflecting a key strength of agent-based modeling with regard to reproducing specific tissue architectures. The following sections below discuss ABMs developed in the past decade with a specific translational orientation to various organs, tissues, and disease processes.

4.1 |. Pulmonary inflammation

Pulmonary inflammation can arise from either sterile or infectious processes. As with other examples of pathophysiological manifestations of inflammation, a heterogeneous set of perturbations can lead to different disease phenotypes, all operating through a similar set of pathways and cellular actors. An abstract ABM of pulmonary inflammation, the acute pulmonary injury ABM was developed to dynamically represent the molecular, cellular and organ-level dynamics of acute pulmonary inflammation and provide a unifying basis for the response to multiple types of acute lung injury, namely direct trauma (pulmonary contusion), bacterial infection (primary pneumonia), and systemic inflammation (acute lung injury/acute respiratory distress syndrome or acute lung injury/acute respiratory distress syndrome) (An et al., 2013).

Other ABMs of lung inflammation were more focused on reproducing a specific disease and associated pathophysiology. The Kirschner group has utilized agent-based modeling extensively to gain novel insights into granulomatous inflammation, with a significant volume of work studying pulmonary tuberculosis (Cilfone et al., 2015; Cilfone, Perry, Kirschner, & Linderman, 2013; Fallahi-Sichani et al., 2010; Gong et al., 2013; Gong, Linderman, & Kirschner, 2014; Linderman et al., 2010; Marino et al., 2007, 2008, 2011; Marino & Kirschner, 2016; Marino, Myers, Flynn, & Kirschner, 2010; Pienaar et al., 2015, 2017; Riggs et al., 2008; Segovia-Juarez, Ganguli, & Kirschner, 2004; Warsinske, Pienaar, Linderman, Mattila, & Kirschner, 2017; Wong et al., 2018; Ziraldo, Gong, et al., 2015). These studies present high-resolution models with extensive characterization of the spatial patterning seen in host–microbe interactions and have been used to investigate the optimization of various combinations of anti-tuberculosis drugs (Cicchese et al., 2017) (see Section 4.6). Brown et al. (2011) generated an ABM of lung inflammation and fibrosis, which was applied to gain insights into lung pathobiology in the setting of tobacco smoking and subsequent chronic obstructive pulmonary disease (COPD). The authors validated key patterns and features from their ABM experimentally in lung histology sections from mice exposed to tobacco smoke. More recently, Wellman et al. (2018) and Ceresa et al. (2018) extended this work through the use of hybrid modeling approaches. Ceresa et al. (2018) generated a hybrid finite-element model/ABM of COPD and tested key features against publicly available databases. Wellman et al. (2018) used a hybrid, physics-based ABM to carry out a study in the setting of idiopathic pulmonary fibrosis, a complex disease in which inflammation and fibroblast dysfunction have been implicated. The authors hypothesized that, since the pulmonary pleura is stiffer than the parenchyma, then local stiffness of the underlying extracellular matrix can influence fibroblast activation, collagen deposition, and tissue stiffness in a positive feedback loop. They tested this hypothesis with their model and found that a key histological pattern emerged only when mechanical rupture was simulated in the setting of highly strained normal tissue surrounded by stiff fibrotic tissue. Modeling studies by Warsinske et al. (2016) suggested that to treat pulmonary fibrosis, therapies might need to be targeted at both fibroblasts and epithelial cells. Song et al. (2012) generated an ABM to study the Th17 inflammatory pathway in asthma and its interaction with adaptive immune pathways. Pothen et al. (2013, 2015) used agent-based modeling to examine allergic inflammation in the lung, including in silico knock-out experiments to study key aspects of their hypothesis regarding the interplay between onset and resolution of inflammation in this context.

4.2 |. Hepatic inflammation

Similar to pulmonary inflammation, hepatic inflammation can arise from both infectious and noninfectious origin, leading to the development of multiple, special-purpose ABMs of inflammation. Moyo et al. (2018) combined transcriptomic analyses with an ABM to suggest mechanisms by which chemokine production may act to limit infection in the liver via granuloma formation. Additional work in hepatic infection and inflammation involved modeling of the viral dynamics of hepatitis. Goyal and Murray (2016) and Murray and Goyal (2015) used an ABM to examine hepatitis B virus infection, and specifically how various T cell subsets affect how viruses from one cell infect another. More recently, Shi et al. (2016) developed an ABM of hepatic inflammation associated with Salmonella infection and subsequent sepsis. Based on a large number of published studies, this ABM describes interactions among a multitude of cells and inflammatory mediators and, importantly, incorporates over 200 experimental data sets and/or data estimates from those studies. Their simulations pointed to a variety of cells and inflammatory mediators or their ratios as being predictive of hepatic inflammation, as well as simulating various therapies.

Multiple ABMs of sterile hepatic inflammation have also been developed. Dutta-Moscato et al. (2014) created an ABM of hepatic injury, inflammation, and fibrosis that incorporates physical forces as well as biological interactions. This model reproduced the histological patterns of fibrosis observed experimentally in rats undergoing chemically induced liver injury as well as leading to predictions of greater tissue stiffness that could, at some point, be compared to noninvasive tissue elastography. The model was used for in silico clinical trials of agents targeting transforming growth factor-β1.

4.3 |. Gastrointestinal inflammation

The intersection between inflammation and the histological architecture of the gut makes it particularly suitable for agent-based modeling. The mutability of the gut mucosal histology in the face of inflammation was the specific target of the Spatially Explicit General-purpose Model of Enteric Tissue (SEGMEnT), which is an ABM that represented a dynamically modulated mucosal crypt-villus architecture based on the differentiation and apoptotic characteristics of the gut enterocyte population (C. Cockrell et al., 2014). As depicted in its name, SEGMEnT was designed to be able to reproduce a series of experimental and clinical conditions: various morphogen knock-out experiments, gut-ischemia reperfusion, local tissue wounding and colonic metaplasia due to chronic long-standing inflammation. Recognizing that certain disease processes spanned anatomic scales of the intestinal track, SEGMEnT was extended to an HPC version that allowed the simulation of 3 m of colon, accounting for 100 billion cellular agents (R. C. Cockrell et al., 2015). The Bassaganya-Riera group has also utilized agent-based modeling to study intestinal inflammation in chronic inflammatory diseases, with a focus on integrating innate and adaptive immune mechanisms (Alam et al., 2015; Carbo et al., 2013; Mei et al., 2015). These studies have led to a greater understanding of the immune response to Helicobacter pylori infection, including the role of Th1, Th2, and Th17 cells.

One of the most dramatic developments in biomedical research in the past decade is the recognition of the importance of the microbiome in human health and disease. In particular, the gut microbiota, consisting of more cellular elements than the entire population of human eukaryotic cells, has been a central focus of this recent research. Unfortunately, the conversion of genomic and transcriptomic analysis of the gut microbiome from static representations to dynamic models remains a significant challenge and an area of active research (Christley, Cockrell, & An, 2015). Given its background in ecological modeling, agent-based modeling has been used to characterize the intersection between microbial behavior and the host environment. While promising, most of these models are only able to incorporate at most a few microbial species, thereby missing a considerable amount of the complexity of the actual system. However, even with this limitation, certain general principles are able to be elucidated by these in silico studies. One such example is the work of Shashkova et al. (2016), which looked at the ecological relationships between a small subset of bacterial species, and the changes involved associated with different system perturbations, such as metabolic manipulation and the administration of antibiotics.

Several ABMs have been developed to study the response of the gut to specific microbes/pathogens. Clostridium difficile is a significant nosocomial infection arising from antibiotic use for other sites of infection; it is due to a disturbance of baseline commensal gut microbiota that allows the emergence of a pathogenic Gram-positive bacillus, C. difficile, which can produce a secretory diarrhea and severe colitis. An ABM of C. difficile dynamics (the C. difficile infection ABM or CDIABM) demonstrated the importance of bile salt metabolism by the commensal microbiota as a means of suppressing colonized C. difficile spores, as well as examining the relative efficacy of various types of therapies, including fecal microbial transplant, for C. difficile colitis (Peer & An, 2014).

Several groups utilized a similar ABM approach to examine the “leaky gut” hypothesis in the context of the potential to develop autism, suggesting that Clostridia growth rate could be a key determinant of risk of autism development (Heberling et al., 2013; Weston et al., 2015). Lin et al. (2018) modified this work by developing an ABM of gut microbial populations in the ileum; this model was used to demonstrate the response of resident bacterial populations to the administration of different types of probiotic mixtures.

Another example of microbe–gut interactions is seen in the examination of gut-derived sepsis. Among the plethora of pathophysiological models for the perpetuation of sepsis, the gut-derived sepsis model proposes that a critically ill individual’s gut becomes a “hostile milieu” that leads to the activation of microbial virulence through direct sensing of otherwise benign resident bacteria. The bacteria Pseudomonas aeruginosa is particularly invoked as such an organism. An ABM of gut-derived sepsis arising from P. aeruginosa was developed that linked a series of pathophysiological insults consistent with critical illness to the activation of P. aeruginosa virulence and subsequent induction of gut, then systemic inflammation (Seal et al., 2011).

Another disease that involves the intersection between the gut microbiota and intestinal inflammation is necrotizing enterocolitis (NEC). NEC is a significant source of morbidity and mortality in neonates, and while its specific pathophysiology is unclear, it is known to involve a triad of prematurity, enteral feeding and a microbial component. An attempt to unify these factors into a minimally sufficient pathophysiological hypothesis was the subject of an ABM looking at immature oxidative stress capacity in neonatal enterocytes (Kim et al., 2012).

4.4 |. Soft tissue and wound healing

Martin et al. (2017) reported on an ABM of muscle injury and subsequent inflammation, with a focus on macrophages and satellite stem cells. Their simulations suggested that delivery of a particular growth factor would help drive tissue healing after muscle injury. Gopalakrishnan et al. (2013) reported on an ABM that examined how the processes of muscle injury, inflammation and healing intersected with potential bacterial contamination to lead to surgical site infections. In particular, this work demonstrated how the extent of tissue trauma present in a wound (and specifically muscle) affected the activation of microbial virulence, and presented a transition map between successful healing, nonhealing and a wound infection based on a combination of tissue damage and bacterial virulence potential.

The Blemker Lab has generated extensive work on the dynamics of muscle atrophy and repair. Within this group, Virgilio, Martin, Peirce, and Blemker (2018) reported on an ABM that integrates muscle cells, fibroblasts, satellite stem cells, and inflammatory cells in the context of Duchenne muscular dystrophy. These authors used this model to study contraction-induced injury, chronic inflammation, fibrosis, and impaired regenerative capacity. Using this ABM, the authors suggest that the extent of injured muscle depends on satellite stem cell activity in a complex, emergent fashion. In turn, multiple cell types affected the levels of these stem cells, though no single parameter influenced stem cell counts. Based on their simulations, the authors conclude that interventions might be targeted to the microenvironment rather than to the satellite stem cells themselves. Another project involved the examination of use/disuse on the development of muscle atrophy (Martin et al., 2015). They also used an ABM of muscle laceration and repair to simulate the effect of administering macrophage colony stimulating factor in conjunction with in vivo experiments (Martin et al., 2017).

The earliest patient-specific ABM of virtual human tissue injury was generated by Li et al. in 2008, in a simulation of vocal fold injury induced by loud phonation (Li et al., 2008). These authors calibrated their model against inflammatory mediator data obtained from laryngeal lavages during early time points following phonation injury, then simulated several behavioral therapeutic strategies and validated their model against data obtained following those strategies in individual patients (Li et al., 2008). This work was later extended to simulating surgical injury and validating the model against data obtained in rats (Li et al., 2010, 2011). Recently, the lead author (Li-Jessen) and her group extended this work further to modeling three-dimensional vocal folds via the use of HPC to track a large number of cells and molecules, and used to simulate surgical injury (Seekhao et al., 2018).

Solovyev et al. (2013), Ziraldo, Gong, et al. (2015) and Ziraldo, Solovyev, et al. (2015) utilized ABMs to gain insights into the process of pressure ulcer formation in patients with spinal cord injuries. This included a hybrid equation/ABM that was calibrated with blood flow data from individual patients to yield patient-specific predispositions to form ulcers (Solovyev et al., 2013), as well as the generation of tissue-realistic pressure ulcer models that were calibrated and validated against clinical images from spinal cord injury patients (Ziraldo, Gong, et al., 2015; Ziraldo, Solovyev, et al., 2015) that was used for in silico clinical trials of corticosteroids as well as agents targeting damage-associated molecular pattern molecules such as HMGB1.

Inherent to the healing process is the ability to generate new blood vessels in areas of damaged tissue. Following on from earlier work from the Peirce-Cottler group that used agent-based modeling to study angiogenesis (Bailey et al., 2007, 2009; Hashambhoy et al., 2011; Peirce, 2008), Kleinstreuer et al. (2013) simulated vasculogenesis using an ABM that encompasses growth factors, chemokines, and platelet factors, and also simulated anti-angiogenic strategies.

4.5 |. ABMs of inflammation and oncogenesis

The study of cancer is one of the largest growth areas in the development and use of ABMs (see Metzcar, Wang, Heiland, & Macklin, 2019 for a recent review). It is beyond the scope of this review to address this very wide field; instead, we focus on those projects that explicitly examine the intersection between inflammation and oncogenesis. Alfonso et al. (2016) generated an ABM to study how persistent inflammation in breast tissue might lead to a microenvironment in which breast cancer might develop, using data from breast tissue of women undergoing breast reduction surgery. While they did not test specifically novel hypotheses, their model may serve to drive novel hypotheses in the context of breast cancer.

A more general investigation of the link between inflammation and oncogenesis was performed with an ABM that utilized an inflammation module derived from the original Innate Immune Response ABM (An, 2004), with an oncogenesis module that represented the functional effects of accumulating mutations on a simulated genome (An & Kulkarni, 2015). This model provided two key insights that have subsequently become recognized in the cancer field: (a) that the evolutionary dynamics of selection undergoes a frames shift from the multicellular organism to the tumor, which behaves as a unicellular colony subject to game theory and (b) that there is a continuum of accumulating genetic damage, the vast amount of which represent neutral mutations, but with punctuated dynamics as vital cellular functions are degraded.

More recently, Gong et al. (2017) examined cancer–immune system interactions as well as immunotherapy using a hybrid ABM/equation-based framework. In this study, the authors reproduced key dynamics of interactions between cancer cells and cytotoxic T cells, as well as simulating the impact of immune checkpoint inhibitors.

5 |. MEDICAL TREATMENT AS CONTROL: THE ROLE OF ABMs

One of the tenets of TSB is that the modeling projects have direct representation of clinical scenarios or situations (An et al., 2008; An & Vodovotz, 2014; Vodovotz et al., 2008). Inherent to the goal of TSB is that computational modeling should facilitate the development of future therapies. This is consistent with the overall goal of medicine, where the ability to diagnose and prognose are waystations on the path to intervention. Cast in this fashion, we note that the ultimate goal of medicine can be cast as a dynamic control problem, with the task of health care through pharmacological intervention is to control a disease trajectory back to a state of health. There had been prior studies combining model-based dynamic control with equation-based models of sepsis (Day, Rubin, & Clermont, 2010); however, this work relied on highly simplified mathematical models that could be treated with classical optimal control approaches. Alternatively, models that have a high-degree of component fidelity, such as ABMs, are too complex for the application of classical control theory methods; rather, they should be thought of as intermediate, proxy systems that can connect the full complexity of the biological world to the more abstract mathematical representations that can be subjected to formal analysis (An et al., 2017). This concept of using ABMs as proxy systems can be extended to their use as tools to define the scope of various control problems. The Kirschner group has pioneered the use of ABMs to guide the optimization of existing therapies, to a great degree presaging the concept of personalized, precision therapeutics. These studies also demonstrated the efficacy of using ABM-compression methods to greatly increase the efficiency of the optimization process across the set of possible combinations of current anti-tuberculosis drugs (Cicchese et al., 2017). While the optimization of existing therapies is a significant advance toward the great rational design of multimodal therapies, there are other domains of inflammatory diseases where there are no extant molecular-based interventions. In particular, the field of sepsis remains a desert with respect to available therapeutic modalities that can actually alter the pathophysiological processes of the disease process, despite being one of the initial disease applications of agent-based modeling. Therefore, the application of advanced control discovery methods to ABM proxy models has the potential to meet a vital need in a field that has not demonstrated meaningful translational progress over the past 20 years.

The Innate Immune Response Agent-based Model (IIRABM) demonstrated (in 2004) the ability of ABMs to serve as means of dynamic knowledge representation (An, 2010) to test putative therapies thought to be potentially effective. The result in the case of sepsis was invariably that of predicting inefficacy, results that, while they may be useful in demonstrating what therapeutic approach would not work, did not necessarily suggest a set of treatments that would work. This placed sepsis in a very different problem space compared to either cancer or cardiovascular disease with the advent of precision/personalized medicine (Buchman et al., 2016). In cancer and cardiovascular disease, “personalizing” a treatment regimen meant finding combinations of existing, effective therapeutic agents that would best serve a particular patient; essentially, this represented an optimization problem across a known solution space. Sepsis faced a considerably more difficult situation, as there was no accepted biologic agent shown to be effective in a clinical trial (Angus, 2011).

In fact, up to 2017 it was unclear what would be required to control sepsis through the manipulation of inflammatory mediators. At that time two studies, using the concept of the ABM as a proxy system, were undertaken that used powerful control search methods that utilized the significant increase in readily available computing power provided by advances HPC. The first of these simulation experiments utilized a form of evolutionary computation, genetic algorithms to determine if the IIRABM (see above), as a proxy system, could be controlled effectively (R. C. Cockrell & An, 2018). The simulation experiment utilized a combination of clinically realistic constraints (e.g., a sensor and dosing schedule spaced at 12-hr intervals) with a speculative component in which each inflammatory cytokine in the IIRABM could be manipulated up or down a series of specified amounts. The rationale for this approach was that these simulations could help direct research to potentially fruitful targets for future drug development. These simulation experiments did demonstrate that sepsis (as represented by the IIRABM) could be controlled. The converged-upon therapeutic regimen was effective in reducing mortality in a virtual cohort from 80 to 10%. However, this reduction in mortality required multiple targets of interventions, the combinations of which varied at nearly every dosing interval. Furthermore, the scalability of the arrived-upon strategy did generalize to some degree, but there were sufficient ineffective conditions that the static strategy did not appear to be the solution. Rather, an adaptive strategy appeared to be needed, and this led to the second series of simulation experiments that applied deep reinforcement learning (DRL) to the IIRABM (Petersen et al., 2019).

DRL is a type of machine learning that is associated with training artificial intelligence systems (Lillicrap et al., 2016). A technical review of DRL is beyond the scope of this review, but here we present a brief nontechnical introduction. “Reinforcement Learning (RL)” is a class of methods within machine learning for finding near-optimal solutions of a high-dimensional control problem that may not be analytically tractable (i.e., suitable to classical optimal control theory). The process of RL involves training a computational agent that sits outside a model of a system and interacts with it by changing the values of a directed set of components; these actions are applied throughout the course of the simulation in an attempt to guide the system toward a desired final state. RL is a suitable candidate for controlling ABMs, which do not have analytic expressions for their global state dynamics and generally cannot be approached via classical control method. The goal of RL is to find the best action to take for each possible state during the simulation; it learns an adaptive policy for maximizing a reward function (e.g., a patient’s health). The “Deep” component of DRL refers to the nature of the “controlling agent.” For less complex tasks this agent takes the form of what is essentially a large lookup table, but for highly complex, nonlinear systems, such as the IIRABM, the “controlling agent” takes the form of a deep convolutional neural network. Furthermore, the DRL used in studying control of the IIRABM is of a specific type termed “model-based DRL.” This is the approach that has led to the recent advances in game-playing artificial intelligence agents (AIs) from Google’s Deep Mind project, AlphaGo and AlphaZero, function by learning on simulated game played between two AIs (Silver et al., 2016, 2018). The same approach was applied to the IIRABM, treating it as a “game” to be won; no prior application of DRL had been applied to a simulation in the biomedical arena. Notably, this is in distinct contrast to a recent publication on the use of reinforcement learning to train an artificial intelligence agent to optimize the current set of interventions for sepsis (Komorowski, Celi, Badawi, Gordon, & Faisal, 2018). This article is limited by existing data sets of sepsis patients (and, as such, existing therapeutic modalities) and is unable to guide the development of future interventions, that is, new drugs and/or drug combinations. Conversely, the DRL work with the IIRABM mimics the Deep Mind projects insomuch it is simulation based and the trained artificial intelligence (AI) allows for the discovery of novel control strategies (i.e., new drugs, drug combinations, and the timing of thereof) based on the much broader control/action space. For the initial proof-of-concept study, the boundaries and action space of the training were set into the speculative region in order to determine if a robust control policy was even possible: the DRL AI had complete knowledge of the state of the Innate Immune Response ABM as every time step and had the ability to manipulate any of the included mediators within a range of up or down. The results of this work were quite promising; the DRL AI was able to construct a robust policy of managing sepsis that could heal the vast number of initial parameter conditions corresponding to a clinical population. Though promising, this is certainly very preliminary work; future developments involve applying more clinically relevant constraints on the action space for the DRL AI, as well as developing an iterative workflow that would use the results of the DRL to enhance, improve, and refine the IIRABM.

6 |. CONCLUSION: PATHS FORWARD

We have attempted to summarize succinctly a decade’s worth of studies on agent-based modeling of inflammation with a translational focus. Clearly, agent-based modeling in the biomedical arena has come very far. Agent-based modeling was, for many years, a niche modeling approach utilized initially in the fields of social sciences and economics. Perhaps for this reason, as well as the “bottom-up,” quasi-mathematical, and stochastic and thus computationally intensive nature of ABMs that hindered model analysis and parameter estimation, the adoption of agent-based modeling in the biomedical arena lagged initially in comparison to equation-based modeling. When we summarized the field a decade ago, the number of groups utilizing ABMs in their work was small, with a relatively narrow breadth of approach (An et al., 2009). The past decade has seen a rapid rise in the use of agent-based modeling in the biomedical space, as well as efforts focused on increasing the utility of ABMs through innovative approaches to parameter estimation, sensitivity analysis, hybrid modeling, computational efficiency, and integration of dynamic control theory (Figure 1). While we only summarize those studies focused on modeling inflammation in the present article, it is clear that these methodological improvements have increased the mainstream use of agent-based modeling alongside traditional, equation-based modeling methods.

Over the coming decade, we anticipate multiple paths forward for the field of agent-based modeling and its application to the study of inflammation-related diseases (Figure 1). We foresee a continuation of the trend of more investigators adopting agent-based modeling as software packages become both more user-friendly and more powerful. We also anticipate investigators coalescing around key concepts regarding inflammation. We anticipate a greater use of HPC and translation to the cloud, thereby increasing the resolution, detail and size of ABMs developed. We also expect the integration of model-based dynamic control strategies with ABMs in an HPC environment. Challenges remain developing pathways for improvement such that ABMs will begin to achieve an industrial-grade level of robustness, making them suitable for diagnostic applications such as digital pathology or diagnostic imaging combined with tissue-realistic ABMs (Dutta-Moscato et al., 2014; Ziraldo, Gong, et al., 2015; Ziraldo, Solovyev, et al., 2015), as well computationally augmented surgery (Kassab et al., 2016).

To drive these developments, we suggest that funding bodies, industry, and regulatory bodies need to devote resources that will support agent-based modeling alongside more traditional modeling approaches. In a similar vein, there is a need for continuing dialog with traditional modeling communities in both academia and industry. We hope that, when we survey the field 10 years from now, we will be able to describe a robust, clinically applicable ecosystem of ABMs and the major insights derived therefrom.