Table 1:

Overview of modeling tools to potentially combine in an integrated system dynamics model

Modeling tool	Strengths	Limitations if used by itself	Polio-related policy questions addressed by use of the given tool by itself
Systems thinking	* Identify and communicate dynamic complexity * Maintain system perspective	* Qualitative	* What does the global polio surveillance system look like, what are the key delays and outputs (Kalkowska et al. 2015b)?
Deterministic DEB modeling	* Account for aggregate-level feedbacks, accumulation, and delays * Ability to focus on and systematically identify critical dynamic complexity * Highly extendable with complimentary approaches	* Steady-state errors * Stocks never reach 0 * Potentially unrealistic distributions implied by delay processes or aging chains * Only captures average- behavior (i.e., ignores randomness in transitions, uncertainty about averages for stocks, and variability around these averages)	* What can we learn for the future from dynamic modeling of past polio experiences (Duintjer Tebbens et al. 2005; Duintjer Tebbens et al. 2013c)? * What are the trade-offs between speed and coverage of outbreak response immunization (Thompson et al. 2006a)? * What are key polio under-vaccinated subpopulations in the United States, what are the trends, and what are the implications for the national polio vaccine stockpile (Thompson et al. 2012)? * What are the impacts of various immunization options on poliovirus transmission and population immunity to poliovirus transmission in various settings of interest (Thompson et al. 2012; Kalkowska et al. 2014b; Kalkowska et al. 2014a; Duintjer Tebbens et al. 2015c; Kalkowska et al. 2015a; Thompson and Duintjer Tebbens 2015a; Thompson et al. 2015b; Duintjer Tebbens and Thompson 2017b; Thompson and Duintjer Tebbens 2017b)? * How does expanding the target age groups of immunization campaigns affect polio incidence and population immunity to transmission (Duintjer Tebbens et al. 2014)? * What are the best strategies to manage various risks associated with the implementation of OPV cessation (Duintjer Tebbens and Thompson 2014; Thompson and Duintjer Tebbens 2014a; Duintjer Tebbens et al. 2016b; Duintjer Tebbens et al. 2016a; Duintjer Tebbens et al. 2016c)?
Stochastic DEB modeling	* Account for feedbacks, accumulation, delays, and stochastic variability in transitions between stocks	* Computationally more intensive and less tractable than deterministic DEB models * Still subject to steady-state errors and unrealistic distributions implied by delay processes	* How do different decision rules perform to prioritize resources for multiple eradicable diseases (Duintjer Tebbens and Thompson 2009)? * What is the probability of undetected poliovirus circulation as a function of time since the last detection (Kalkowska et al. 2012; Kalkowska et al. 2015b)?
DES modeling	* Accounts for feedback, accumulation, delays, and stochastic variability in transitions between stocks based on individual-level variability	* Computationally more intensive and less tractable than DEB models * Less easy to identify feedback structure	* What is the expected prevalence of iVDPV excretors after OPV cessation and how does this depend on iVDPV surveillance and polio antiviral drug properties (Duintjer Tebbens et al. 2015b)?
Agent-based modeling	* Accounts for feedbacks, accumulation, delays, stochastic variability in transitions between stocks, and individual-level variability, interactions, and decisions	* Computationally highly intensive * Systems perspective sometimes not explicit * Depends on adequate detailed-level data * Not easily amenable to uncertainty and sensitivity analysis	* How do assumptions about contact networks affect modeled poliovirus outbreaks (Rahmandad et al. 2011)? * How might a wild poliovirus introduced into an under-vaccinated North American Amish community propagate geographically (Kisjes et al. 2014)?
Decision analysis	* Structures complex decision space * Deals with conditional probabilities and/or choices	* Traditionally does not account for dynamic complexity	* What are the key poliovirus risk management options for countries and the world (Sangrujee et al. 2003; Thompson and Duintjer Tebbens 2012; Thompson et al. 2013a)? * What are possible frameworks to assess the value of information obtained through poliovirus surveillance (de Gourville et al. 2006) ?
Economic modeling	* Characterizes economic inputs * Supports evaluation of trade-offs in the context of limited resources * Facilitates evaluation of behavioral responses to economic incentives * Facilitates optimization of resources and economic outcomes	* Ignores population-level or other dynamics * Ignores uncertainty and variability	* What are the costs of post-eradication policies (Duintjer Tebbens et al. 2006b)? * What are the costs of the Global Polio Laboratory Network (de Gourville et al. 2006)? * What are the true costs of different IPV formulations and delivery options (Thompson and Duintjer Tebbens 2014b)? * What are the national incentives to stop OPV and what does this mean for global coordination (Thompson and Duintjer Tebbens 2008a)?
Probabilistic risk analysis, including expert judgment	* Quantifies impact of random events with rigorous methods based on probability and statistics * Accounts for dependence between random variables * Quantifies uncertainty and variability using available data and/or expert judgment * Identifies knowledge gaps and research priorities	* Requires significant effort * Depends on quality of available evidence/data and/or expert knowledge	* What are the risks of poliovirus reintroductions in the post-eradication era (Duintjer Tebbens et al. 2006a)? * What is the evidence-base to assess poliovirus immunity, transmission, and evolution, and what are the key limitations of the existing studies (Duintjer Tebbens et al. 2013a; Duintjer Tebbens et al. 2013d)? * What are the consensus and uncertainty related to estimates to characterize poliovirus immunity and transmission and what future studies may fill key knowledge gaps (Duintjer Tebbens et al. 2013b)?
(Probabilistic) uncertainty and sensitivity analysis	* Helps understand and communicate impact of uncertainty * Accounts for non-linearity and interaction and/or dependence between uncertain inputs (depending on method) * Helps prioritize research to fill knowledge gaps	Always used in combination with other modeling tools	* What are the key uncertainties affecting the net benefits of long-term poliovirus risk management options (used in combination with integrated models) (Duintjer Tebbens et al. 2008a; Duintjer Tebbens and Thompson 2016b)? * What are the key uncertainties affecting the net benefits of policies to manage long-term iVDPV risks (used in combination with DES and integrated models) (Duintjer Tebbens and Thompson 2017a)?