Simulation modeling to enhance population health intervention research for chronic disease prevention

Abstract

Population Health Intervention Research (PHIR) is an expanding field that explores the health effects of population-level interventions conducted within and outside of the health sector. Simulation modeling—the use of mathematical models to predict health outcomes in populations given a set of specified inputs—is a useful, yet underutilized tool for PHIR. It can be employed at several phases of the research process: (1) planning and designing PHIR studies; (2) implementation; and (3) knowledge translation of findings across settings and populations. Using the example of community-wide, built environment interventions for the prevention of type 2 diabetes, we demonstrate how simulation models can be a powerful technique for chronic disease prevention research within PHIR. With increasingly available data on chronic disease risk factors and outcomes, the use of simulation modeling in PHIR for chronic disease prevention is anticipated to grow. There is a continued need to ensure models are appropriately validated and researchers should be cautious in their interpretation of model outputs given the uncertainties that are inherent with simulation modeling approaches. However, given the complexity of disease pathways and methodological challenges of PHIR studies, simulation models can be a valuable tool for researchers studying population interventions that hold the potential to improve health and reduce health inequities.

What is population health intervention research?

Population Health Intervention Research (PHIR) is the study of health effects of interventions at the population-level. PHIR methods vary widely: from observational studies of the health effects of “natural experiments” to cluster randomized controlled trials, where interventions are purposefully implemented differentially between populations (Hawe and Potvin 2009). PHIR can greatly inform chronic disease prevention efforts by informing and enacting interventions that have potential for a large and distributive impact in the population. There are, however, many challenges to conducting PHIR: from the practical issues of implementing and evaluating complex interventions across an entire population to the ethical constraints of the study of interventions affecting an entire community.

What is simulation modeling?

Simulation modeling approaches are used to describe complex systems and are employed extensively in the fields of economics, environmental sciences, engineering, and increasingly in health care (Stout et al. 2009). The use of modeling to describe the relationship—whether causal or not—between exposures, covariates, and outcomes in a study population is known as statistical modeling. It is an essential analytical tool to study the complex web of relationships in chronic disease prevention research. However, the focus of this paper is simulation modeling—the use of mathematical and statistical models to predict health outcomes in populations given a set of specified inputs (Kopec et al. 2010). Simulation models differ from statistical modeling by focussing on the dynamic relationships between different agents and the effects of their environment, incorporating changing states, feedback loops, and interdependent non-linear connections (Population Health 2015). These models often seek to predict what will occur in the future, in contrast to statistical modeling which describes, often linearly, the observed relationships between exposures and outcomes. Simulation models vary in complexity from single-variable assessments of risk using attributable-fraction techniques (e.g., assessing the simple what-if scenario of eliminating a single risk factor) to advanced micro-simulation techniques where individual units are moved through multiple transition states using a set of probabilities. There are many examples of simulation models. Predictive risk algorithms for cardiovascular disease are simulation models that aid clinicians’ decisions on the use of statins for dyslipidemia (Genest et al. 2009). At a population level, simulation models inform decisions on prevention and control activities for chronic diseases including cancer (Stout et al. 2009) and coronary heart disease (Unal et al. 2006).

Simulation modeling in chronic disease prevention PHIR

Simulation modeling is a tool that can be helpful in addressing the challenges of PHIR. Despite increasing calls for the integration of simulation modeling into PHIR and chronic disease prevention research, it remains underutilized in the field (Galea et al. 2010). Using the example of community-wide interventions of the built environment for the prevention of type 2 diabetes, we will demonstrate how simulation modeling can be used at different stages of the PHIR cycle (Fig. 1).

Fig. 1.

Uses of simulation modeling in different phases of Population Health Intervention Research (PHIR)

Planning for PHIR studies

PHIR has the challenge of comparing interventions with different effectiveness that target populations with considerable variation in size and baseline risk profile. Take, for example, two interventions in an urban centre for chronic disease prevention: (1) an active transportation plan to develop a network of cycling routes, (2) the establishment of neighbourhood community gardens. The active transportation policy may impact a larger proportion of the population. However, depending on factors such as socio-demographic characteristics, culture, and population density, the latter intervention may have a substantial population effect due to a higher baseline risk of outcomes in the targeted population.

Baseline risk is estimated by considering the risk factors for developing the outcome of interest—for type 2 diabetes incidence, this should include age, sex, and body mass index (BMI) at a minimum. By combining these risk factors into a model that simulates the risk of diabetes, followed by validation in a population of known incidence, one can create predictive risk algorithms to quantify baseline risk. An example is the Diabetes Population Risk Tool (DPoRT) which was developed for PHIR studies and includes the additional risk predictors of: ethnicity, immigrant status, education, hypertension, and alcohol consumption (Rosella et al. 2011). For each individual processed in DPoRT, pre-intervention risk factor profiles can generate a baseline risk for the outcome. The pre-intervention population-level health burden can then be computed by summing the risk of all individuals or by multiplying the target population size by the average baseline risk (Eq. 1).

$$\mathit{\text{Population health burden}}=\mathit{\text{target population size}}x\mathit{\text{average baseline risk}}$$ 1

Estimating population health burden can be useful in study planning to identify and prioritize target populations and risk factors. Whenever possible, baseline risk should be characterized for important subpopulations to identify which subpopulations bear the greatest outcome risk. This process can reduce sample size requirements and/or increase power by maximizing the intervention effect size. In our example, the researcher may be able to identify a geographic area that has a much higher diabetes burden to select the locations for neighbourhood gardens. Assessing baseline risk in planning a study can also identify when high-risk strategies are inferior to interventions targeting a larger segment of the population. This is the case when baseline risk is not concentrated in high-risk groups, but rather diffused throughout the population (Rose 2001).

Study implementation

Population health benefit (effectiveness) of an intervention during implementation can be estimated through simulation models by extending Eq. 1 above to include an estimate of the relative benefit of the intervention and population coverage (Eq. 2).

$$\begin{array}{l}\mathit{\text{Population health benefit}}\left(\mathit{\text{effectiveness}}\right)=\\ \mathit{\text{target population size}}\\ x\mathit{\text{average baseline risk}}\\ x\mathit{\text{relative benefit of the intervention}}\left(\mathit{\text{or policy}}\right)\\ x\mathit{\text{intervention coverage}}\left(\mathit{\text{or adherence}}\right)\end{array}$$ 2

Aside from prospective observation of outcomes, relative benefit of the intervention reducing outcomes is commonly taken from a published study or a meta-analysis. An alternate method uses the reduction in risk factor exposure due to the intervention (usually from literature) and simulates a baseline risk reduction in the population of interest. In our example, estimates of reduction in diabetes risk from increases in physical activity due to a built environment intervention can be obtained from a meta-analysis (Jeon et al. 2007).

A researcher can directly measure intervention coverage and adherence, or estimates can be taken from similar past interventions in comparable communities. The method can be replicated rapidly for varying estimates of intervention effect or population coverage, reflecting the uncertainty in the estimates and outputting a range of possible values.

Simulation models employed with the use of Eq. 2 are particularly useful for estimating the effects of interventions that cannot be measured directly for reasons of practicality, cost, ethics, or acceptability. In our example of increasing neighbourhood walkability, the researcher may seek to use a cluster randomized controlled trial study design, with randomization at the neighbourhood level. However, there are barriers to implementing this design. First, it is challenging and resource intensive to enroll sufficient participants and conduct the appropriate follow-up measures. Second, it may be unethical or unacceptable to change features of the community design in a random fashion that does not allow for appropriate community consultation and participatory decision-making.

Another circumstance where direct measurement of an intervention effect may be impractical—necessitating a modeling approach—occurs when the outcomes of interest happen far in the future. In our example, observing the reduction of diabetes in young adults would require monitoring over decades. To estimate effects in these circumstances, a micro-simulation modeling method could be used to simulate life-course trajectories of individuals within a population. An example of this type of modeling method is the Canadian POpulation HEalth Model (POHEM) which allows alteration of physical activity levels at different stages of simulated life courses (Hennessy et al. 2015). Micro-simulation models are an example of more complex models that may simulate the multiple components of Eqs. 1 and 2 (e.g., the multiple factors that affect baseline risk and adherence). In such models, the agents (i.e., individuals) being simulated may also have varying probabilities of experiencing exposures, mediators, and outcomes, to reflect uncertainty.

Finally, simulation modeling can be used to signal that there is a problem or unanticipated occurrence with study implementation. Initial trial results may show deviations from what is expected based on a modeled estimate. This could be due to incomplete penetration of the intervention into the population, differences between the anticipated and observed baseline risk of the target population, or a variation in the relative benefit of the intervention. Identifying discrepancies early in the implementation phase may lead to alterations in the study design to ensure adequate study power and maximize the benefit of the intervention. As simulation models increase in complexity, there are both increasing opportunities and challenges to modeling both the failure and success factors in the implementation of interventions.

Translation of PHIR study findings

Impactful PHIR includes diffusion research: evidence-based translation of findings to real-world situations and scaling of interventions across settings and within populations (Hawe and Potvin 2009). Modeling proves useful for this purpose. After completing a PHIR study demonstrating the efficacy of community gardens, using a simulation model and Eq. 2, the effect size from the study could be applied to a neighbouring city to provide estimates of potential effect. The researcher would need an estimate of baseline risk and the expected intervention coverage/adherence in the new population.

The application of PHIR to inform action is particularly important because actors outside of the healthcare sector (e.g., housing and education) frequently make decisions that affect population health. Simulation modeling—whether it be conducted in conjunction, separately, or after the fact—can illustrate the health effects to decision-makers in these circumstances.

In our example, a policy to encourage active transportation via cycling might be considered with the primary goal of reducing traffic congestion. To inexpensively make the case for health impact, the researcher could use a modeled estimate of baseline risk for diabetes, published evidence of the effects of cycling on risk reduction of diabetes and combine it with information from the municipality on the number of individuals who took up cycling after the policy shift.

Their utility in decision-making has made modeling techniques valuable in Health Impact Assessments—typically conducted prior to the introduction of a policy or project that resides primarily outside of the health sector (Weinstein et al. 2001). However, as we have demonstrated, simulation modeling is not limited to pre-implementation phase of a policy; it can equally be used for evaluation of interventions that have occurred in the past or at any other phase of the policy cycle. Simulation modeling also engages knowledge users, as their “real-world” decision-making scenarios are often the impetus for model development. Collected data from knowledge users can be combined with study data and used in a cycle to both inform and refine model inputs.

Future directions and challenges

The use of modeling in PHIR can be expected to grow to coincide with the increase in accessible data on the exposures, covariates, mediators, and outcomes that form the complex web of causation. In addition to data collected primarily for research and evaluation purposes, many countries routinely collect a wealth of data from health administrative databases, health surveys, disease registries, vital statistics, and other sources (Rosella et al. 2011). Data from sectors outside of health (e.g., education and social services) are also becoming linked to health datasets. Dataset linkages will enable the development of further complex modeling techniques such as systems dynamic models and agent-based models (Homer and Hirsch 2006).

One of the primary challenges of simulation modeling is to ensure that models are properly validated and accurately reflect the “real-world” system they are intended to represent (Rosella et al. 2011). This includes choosing the data sources that best represent the studied populations. It is imperative that we recognize the risk of relying on poorly validated models that lead to false conclusions. In determining the transferability of PHIR results to different populations and settings, differences in the distribution of exposures and covariates need to be carefully considered. It will also be important that transparent and consistent methods are developed, so that users are fully aware of the limitations and do not extrapolate beyond a model’s intended purpose (Bennett and Manuel 2012). This includes making conclusions that accurately reflect model complexity, which is still restrained by our limited understanding of disease pathways. Model validation, along with calibration, and the handling of unknown model parameters are the subject of intensive research.

In addition to validity, a related challenge lies in handling of uncertainty surrounding model outputs. Sensitivity analysis, whereby the parameters in the model are varied to determine a range of possible outcomes, is standard practice in simulation studies in PHIR. Inevitably, however, there will be some factors that cannot be reasonably predicted or even measured and thus cannot be accurately inputted into a model, such as the acceptability of an intervention or unmeasurable biologic differences. Nevertheless, a well-calibrated and validated model may still provide the best method of predicting an outcome. This is especially true when the alternatives are “mental models” based simply on human perceptions or clinical judgements that are prone to a multitude of psychological biases (Sterman 2006).

Despite these challenges, the value of simulation modeling in PHIR is apparent. It is important to emphasize that simulation modeling does not discount the continued need for conventional methods in PHIR. On the contrary, the collection of both quantitative and qualitative primary data that richly describe the relationships between intervention and outcome in the study population is helpful for model input. Still, the complexity of disease pathways and the challenges in employing interventions at the population level means that simulation models may become essential in the toolbox of the population health intervention researcher who seeks to improve the health of the population and reduce health inequalities.

Funding

This commentary was unfunded. PT was the first author and receives support from the Department of Medicine at the University of Ottawa, Bruyère Research Institute, and the Ottawa Hospital Research Institute. Funders did not have any influence in any aspect of this manuscript.

Compliance with ethical standards

Conflict of interests

The authors declare that they have no conflict of interests.