Pop-GUIDE: Population modeling Guidance, Use, Interpretation, and Development for Ecological Risk Assessment

Abstract

The assimilation of population models into Ecological Risk Assessment (ERA) has been hindered by their range of complexity, uncertainty, resource investment, and data availability. Likewise, ensuring that the models address risk assessment objectives has been challenging. Recent research efforts have begun to tackle these challenges by creating an integrated Modeling Framework and Decision Guide to aid the development of population models with respect to ERA objectives and data availability. In the Framework, the trade-offs associated with the generality, realism, and precision of an assessment are used to guide the development of a population model commensurate with the protection goal. The Decision Guide provides risk assessors with a stepwise process to assist them in developing a conceptual model that is appropriate for the assessment objective and available data. We have merged the Decision Guide and Modeling Framework into a comprehensive approach (Pop-GUIDE, Population modeling Guidance, Use, Interpretation, and Development for Ecological risk assessment) for the development of population models for ERA that is applicable across regulatory statutes and assessment objectives. In Phase 1 of Pop-GUIDE, assessors are guided through the trade-offs of ERA generality, realism, and precision, which are translated into model objectives. In Phase 2, available data are assimilated and characterized as general, realistic, and/or precise. Phase 3 provides a series of dichotomous questions to guide development of a conceptual model that matches the complexity and uncertainty appropriate for the assessment that is in concordance with the available data. This phase guides model developers and users to ensure consistency and transparency of the modeling process. We introduce Pop-GUIDE as the most comprehensive guidance for population model development provided to date. It is the first of such documents that includes an evaluation of uncertainty as a function of the tolerance of the assessment in which its applied. We demonstrate Pop-GUIDE through case studies using fish as an example taxon and the US Federal Insecticide Fungicide and Rodenticide Act and Endangered Species Act as example regulatory statutes.

Introduction

Population models can be used to estimate long-term impacts to a species from exposure to stressors, translate lethal and sublethal impacts on organisms into changes at the population level, and simulate a number of exposure and management scenarios that would be logistically challenging to assess in field studies. Population models have long been recognized as potentially valuable tools for ecological risk assessments (ERAs), with recommendations for their use included in early ERA guidance (e.g., Pastorok et al. 2003) and continued advocacy from more recent workgroups (Forbes et al. 2009; Galic et al. 2010; Schmolke, Thorbek, DeAngelis et al. 2010; Forbes et al. 2016). The ecological foundation of population models has been thoroughly documented (Barnthouse et al. 2008), and mathematical and simulation approaches are available and widely accepted (Caswell 2001; Grimm and Railsback 2005). Their value and application in ERA have been demonstrated through numerous case studies (e.g., Spromberg and Meador 2006; Schmolke et al. 2017a; Schmolke, Brain et al. 2018; Schmolke, Roy et al. 2018; Thursby et al. 2018; Forbes et al. 2019). Despite the established scientific basis and promotion of their use for regulatory assessments (NRC 2013), population models continue to be underutilized in ERA due to challenges associated with model complexity, uncertainty, endpoint utility for risk assessment objectives, added investment, data availability, and general acceptance of the models themselves (Forbes et al. 2016; Raimondo et al. 2018).

Recent work has advanced guidance for population model development and application to address the challenges that impede their use and to move toward integrating them into standard ERA practices (Schmolke et al. 2017b; Raimondo et al. 2018). Schmolke et al. (2017b) provided a series of decision steps (herein, “decision guide”), as demonstrated with an endangered plant exposed to a pesticide, that guide model developers to a conceptual model. The decision guide consists of 4 phases: 1) identification of model objectives, 2) compilation of available data, 3) decision steps for the development of the conceptual model, and 4) summary of the minimal conceptual model and its uncertainties. Raimondo et al. (2018) presented a framework (herein, “modeling framework”) in which the trade-offs of generality (i.e., representative of a wide range of species, environments, and/or exposure scenarios), realism (i.e., captures the necessary biological, environmental, and/or chemical features to reflect real world processes), and precision (i.e., having narrow confidence bands) of an ERA are used to guide appropriate decisions in model development. In Raimondo et al. (2018), generality, realism, and precision are relative and considered on a continuum demonstrated by case studies of various taxa under several regulatory statutes. These 2 approaches address the challenges of model building and use from different, yet complementary perspectives. Additionally, the European Food Safety Authority (EFSA) has published a Scientific Opinion on Good Modelling Practice (EFSA 2014), which outlines the tasks for systematic and transparent development of ecological models used for ERAs.

Here, we bring the decision guide and modeling framework together with the recommendations of EFSA (2014) as a comprehensive approach for Population model Guidance, Use, Interpretation, and Development for Ecological risk assessment (Pop-GUIDE). Although Pop-GUIDE is initially presented in the North American risk assessment context, we believe this approach could be applied within other regulatory contexts, for example, in the European Union (EU). Pop-GUIDE is intended to serve as a comprehensive tool to facilitate development and implementation of population models in ERA. It broadens the systematic process of conceptual model development introduced by Schmolke et al. (2017b) to be applicable across taxa and serves as documentation of the conceptual model development process. The conceptual model that results from decisions and assumptions is recorded and summarized as part of the Pop-GUIDE process, which is designed to be easily shared with and understood by stakeholders who are not modelers. The conceptual model facilitates subsequent model implementation and evaluation. In addition, it sets models developed for and applied to ERA firmly within the requirements, objectives, and complexities of its protection goals. The process of model development is rendered transparent, and decisions and assumptions taken during development are made explicit. The trade-offs of generality, realism, and precision are considered with respect to resources (e.g., time), the objectives of the ERA, and data availability to ensure the resulting uncertainties of the final model are consistent with the requirements of the ERA. Modelers can use this process to facilitate model development and communication, whereas model users (e.g., risk assessors) can use Pop-GUIDE to assess how best to use a model in a given ERA context and improve existing ERAs by identifying ways to make them more relevant. Here, we present Pop-GUIDE, a taxa-independent model development approach, by discussing different phases, adapted and expanded from Schmolke et al. (2017b), which include 1) model objectives, 2) data compilation, 3) decision steps, 4) conceptual model, and 5) model implementation and evaluation.

We demonstrate Pop-GUIDE with 2 case studies that use fish (delta smelt Hypomesus transpacificus and fathead minnow Pimephales promelas) exposed to chlorpyrifos. Chlorpyrifos is a broad-spectrum organophosphate insecticide regulated under the US Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA, 1980, 7 U.S.C. §136 et seq). It is widely applied for both agricultural and nonagricultural uses throughout the United States and is frequently detected in urban, suburban, and agricultural streams (Gilliom et al. 2006). In fish, chlorpyrifos has been found to cause long-term behavioral impairments, altered stress response, and impaired learning (Levin et al. 2004; Sledge et al. 2011). Due to its insecticidal properties, it has also been associated with mortality of nontarget invertebrates (Kuivila and Moon 2004; Anderson et al. 2014). The delta smelt is a slender-bodied fish endemic to the upper Sacramento–San Joaquin Estuary of California listed as endangered under the US Endangered Species Act (ESA, 1973, 16 U.S.C. §1531 et seq). It inhabits the freshwater–saltwater mixing zone of the estuary and migrates upstream to fresh water during spawning season (March to May). Delta smelt populations are declining as a result of drought and limited freshwater flows, with record low numbers of individuals counted in recent surveys (Moyle et al. 2016). Chlorpyrifos has been frequently detected in the upper Sacramento–San Joaquin Delta during the spring and summer when vulnerable larval and juvenile delta smelt are present. The fathead minnow is a common freshwater fish species that inhabits temperate regions throughout most of North America and is regularly used as a surrogate species to represent sensitivity of freshwater fish to a diversity of environmental contaminants (Ankley and Villeneuve 2006). The delta smelt case study is an example of a species-specific application of Pop-GUIDE for a listed species assessment, whereas the fathead minnow case study is framed in the context of a national level risk assessment. Through these case studies, we highlight the unique and transparent approaches of Pop-GUIDE as a valuable tool for advancing ERA.

POP-GUIDE FRAMEWORK

Phase 1: Model objectives

Model objectives should be determined in the context of the objective of the specific ERA in which it is to be applied, and the complexity of the ERA will guide the complexity of the model. An ERA is conducted following standards according to a regulatory statute (USEPA 1998; Biddinger et al. 2008; Hommen et al. 2010), which may provide the context for the necessary degree of realism and precision in the model. However, it is important to note that the regulatory statute alone does not determine the trade-offs of generality, realism, and precision for an ERA because the objective of an ERA may vary within a statute or across assessment tiers within a statute. For example, in the 3-step process for pesticide ERAs conducted for listed species, the objective of step 1 is to determine if an action may affect a species, whereas the objective of step 2 is to determine if the action is likely to adversely affect (LAA) the species, and the objective of step 3 is to determine if the action will cause jeopardy to the species (NRC 2013). In this example, step 1 is intended to be less precise than the other steps with low investment in realism and serves only to identify the listed species that will move on to step 2. The step 2 assessment is required to have a higher level of realism than step 1 with respect to processes and factors that could influence an LAA determination, and relative precision should increase in concordance with realism. Given that step 3 is the determination of whether or not the action can be taken, it is required to have a high level of realism and precision on which to make a jeopardy determination.

Although the objectives of the model may be narrower than the ERA, that is, the model may be developed to address a specific set of questions within the ERA, models should generally be no more complex than what is required for the ERA trade-offs. Thus, the first step in identifying the model objective is to identify the trade-offs of generality, realism, and precision associated with the ERA objective (Figure 1A; Raimondo et al. 2018). The language we use for discussing trade-offs is based on that of Levins (1966; general, realistic, precise); however, although this language is intuitive, its current application was proposed specifically for ERA (Raimondo et al. 2018), and we emphasize that the trade-offs as discussed herein for models should be considered only within their particular ERA context. The terms “generality,” “realism,” and “precision” are by their nature, relative; they are most informative when considered on a continuum and compared to some alternative.

Figure 1.

Process flow of Pop-GUIDE that involves defining model objective (Raimondo et al. 2018) (A), data compilation (B), decision steps to guide model complexity (C), and development of conceptual model and model implementation and evaluation (D).

Initial guidance for associating the ERA objective with the appropriate levels of realism and precision was primarily determined by taxonomic specificity, spatial considerations, and temporal considerations (Raimondo et al. 2018). Pop-GUIDE refines that guidance using a decision tree to determine the location of the ERA within the framework trade-off space as a starting point (Figures 1A and 2), and then provides subsequent questions to translate the ERA trade-offs into model trade-offs. The decision tree first distinguishes an ERA based on the inclusion or exclusion of listed species as 1 function of taxonomic specificity. Subsequent questions within the decision tree that pertain to spatial and taxonomic considerations determine if the assessment is at the national, regional, site-specific, or habitat-specific scale. These questions direct the user to one of 5 trade-off categories: general, realistic, general–realistic, general–precise, realistic–precise. As shown in Figure 1A, these trade-off categories reflect their approximate location within the trade-off space. The “general” category would include ERA objectives that do not target specific locations, habitats, or species. The specificity of these attributes increases with increasing realism. “Precision” refers to the level of confidence in the data available to inform model processes. Given that precision is either targeted or sacrificed based on the objective, and general ERAs may require some level of realism in exposure scenarios for specific habitats, hybrid categories (general–realistic, general–precise, realistic–precise) are possible based on ERA objectives (Figure 2).

Figure 2.

Decision tree to identify the category of trade-offs among generality, realism, and precision of an ERA based on its objectives. ERA = ecological risk assessment.

Once the category of ERA trade-offs is identified, it is necessary to translate the ERA objective and trade-offs into those for model building. The questions presented in Box 1 aim to facilitate the translation of the ERA trade-offs to those of the model to be used in its context. They should be initially addressed in this section and revisited as needed after working through the more specific questions in Phase 2 (data compilation) and Phase 3 (decision steps).

Box 1. PHASE 1 QUESTIONS TO DERIVE MODEL OBJECTIVES.

1. Ideally, how will the population model be used in the ERA, for example, as a direct assessment tool in species- or location-specific ERA (e.g., endangered species, Superfund) or as part of a weight of evidence for broader ecological protections? If the model will be used as a direct assessment tool, its trade-offs should match that of the ERA category.

2. What assessment endpoints are most relevant to the ERA objective and the intended model use (e.g., population growth, abundance, quasi-extinction probability)?

3. Are there temporal considerations that are important to the realism of the ERA, for example, seasonal chemical application or persistence in the environment?

4. What uncertainties are acceptable for the ERA?

5. What are the project resources (timeline, budget, etc.)?

The overall objective of a model is to meet the intended use for the ERA (question 1) by providing relevant endpoints (questions 2 and 3) within the constraints of the acceptable ERA uncertainty (question 4) and available resources (question 5). The specific objective of a model is guided by the answers to these questions. The case studies using delta smelt and fathead minnow demonstrate this process in the following 2 sections.

Delta Smelt Case Study

For the delta smelt case study, the ERA objective is to determine if chlorpyrifos will adversely affect delta smelt through labeled applications in the California Central Valley. Using the flow chart in Figure 2, the first question of the decision tree is answered “Yes” due to the listing status of the delta smelt. The second branch of the decision tree is also answered “Yes” because the species has a limited distribution in the upper Sacramento–San Joaquin watersheds. Based on these questions, the objective requires an assessment that is realistic and precise. To translate this ERA category into model objectives, we use the questions presented in Box 1. It should be noted that this case study is not an official risk assessment and is presented only as a hypothetical example, so the following answers to the questions in Box 1 are provided for demonstration purposes only.

1. Ideally, how will the population model be used in the ERA, for example, as a direct assessment tool in species- or location-specific ERA (e.g., endangered species, Superfund) or as weight of evidence with additional data for broader ecological protections?

   In this example, the model would ideally be used as a direct assessment tool. Thus, precision and realism should be targeted for the model.

2. What assessment endpoints are most relevant to the ERA objective and the intended model use (e.g., population growth, abundance, quasi-extinction probability)?

   Given the small population size of the delta smelt, abundance is the primary model endpoint that will be used. Secondary endpoints include population growth rate and probability of extinction. Considering the low number of delta smelt recorded in surveys, model predictions should be as precise as possible.

3. Are there temporal considerations that are important to the ERA, for example, seasonal application or persistence in the environment?

   Because the smelt relies on fresh water for spawning, seasonal fluctuations in freshwater flow and pesticide concentrations are important to consider. Because precise predictions of future rainfall and snowmelt that drive freshwater fluctuations are unlikely to be available for the model, the temporal considerations important to this species’ life history require a realistic representation of seasonal changes and should contain a temporal resolution adequate to describe these changes.

4. From the ERA category and intended use, what uncertainties are acceptable?

   The ERA category is realistic–precise. Given that the assessment is for a listed species, which requires a high level of certainty in the accuracy of the predicted risk, precision should be the primary targeted trade-off of the model. As such, the uncertainty of the model should be represented by quantitative confidence bounds derived from empirical functions within the model.

5. What are the project resources (timeline, budget, etc.)?

   This case study requires that the model is developed with existing data, with no time or budget available to collect additional data.

Based on the answers to these questions, the model objective is to provide reliable estimates of potential impacts of realistic chlorpyrifos exposure scenarios on delta smelt abundance using available data.

Fathead minnow case study

For the purposes of the fathead minnow case study, the objective of the assessment is to evaluate the risk to fish from chlorpyrifos exposure at the national scale, such that the assessment ensures that chlorpyrifos will not pose any unreasonable risks to wildlife and the environment. Using Figure 2, the first question of the decision tree is answered “No” because the objective is at the national scale and aims to protect wildlife and the environment in general. The second branch of the decision tree is also answered “No” because the application of chlorpyrifos is not restricted geographically within the United States. Finally, the third branch of the decision tree is answered “No” because the pesticide application may potentially affect diverse habitats. Based on these questions, the objective results in an assessment that needs to be general.

To translate this ERA category into model objectives, we use the questions presented in Box 1. It should be noted that this case study is not an official risk assessment and is presented only as a hypothetical example, so the following answers to the questions in Box 1 are provided for demonstration purposes only.

1. Ideally, how will the population model be used in the ERA, for example, as a direct assessment tool in species- or location-specific ERA (e.g., endangered species, Superfund) or as weight of evidence with additional data for broader ecological protections?

   In this example, the model would be developed for fathead minnow to leverage available toxicological studies to represent potential impacts to fish. The assessed impacts to fish would be combined with potential impacts to other taxonomic groups to determine if chlorpyrifos applications posed risks to the environment. As such, the model would contribute to the weight of evidence in evaluating ecological risks of chlorpyrifos application.

2. What assessment endpoints are most relevant to the ERA objective and the intended model use (e.g., population growth, abundance, quasi-extinction probability)?

   At the scale of ecosystems, which contain an inherent level of resiliency beyond that of individual organisms, the model endpoint that would be indicative of an ecological impact is a decline in long-term population abundance (or negative population growth rate).

3. Are there temporal considerations that are important to the ERA, for example, seasonal application or persistence in the environment?

   Because timing of application relative to fish reproduction could significantly influence population growth rate, the interaction of these 2 factors should be considered.

4. From the ERA category and intended use, what uncertainties are acceptable?

   The ERA category is general and intended for broad ecological protection. The ERA assumes a high level of uncertainty at the level of individual species, while being conservative enough to protect diverse communities. As such, the model does not require a high level of precision for specific populations of the fathead minnow and should be able to be applied across multiple scenarios.

5. What are the project resources (timeline, budget, etc.)?

   This case study requires that the model is developed with existing data, with no time or budget available to collect additional data.

Based on the answers to these questions, the model objective is to translate impacts of survival, reproduction, and growth into potential effects on fish population growth rate and/or abundance from exposure to chlorpyrifos throughout its use area using available data.

Phase 2: Data compilation

Schmolke et al. (2017b) identified data that should be compiled for a comprehensive assessment of species and exposure–effects information, as were important for their case study of an herbaceous plant exposed to pesticides. Data collection targets information pertaining to the focal species, the environment, and chemical impacts (Figure 1B). Pop-GUIDE provides tables that group characteristics as organism-level characteristics (Table 1), population-level and spatial characteristics (Table 2), external factors and habitat characteristics (Table 3), and exposure and effects characteristics (Table 4). Pop-GUIDE identifies characteristics that are relevant for all taxa and provides a description of what types of data may be categorized as general, realistic, and/or precise for each characteristic. Not every characteristic listed within these tables may apply to each species and risk scenario; rather, the tables represent a comprehensive list of factors known to influence populations and could influence risk for some cases (e.g., Table 3). The intention of Phase 2 is to consistently survey, collect, and evaluate available data relevant to population models in the risk assessments. Later phases will determine how the data are used in model development. However, if Phase 1 determines that a more general model is appropriate for an ERA, investment into extensive data-gathering exercises to search for precise data sets could be minimized.

Table 1.

Life history characteristics that should be targeted to inform a population model for ERA^a

Characteristic	General	Realistic	Precise
Life span	Average life span	Life span range and/or variance	Population-specific estimates of life span
Reproductive/breeding season	Range of dates during which eggs, offspring, or seeds are produced	Range of dates based on local and annual environmental conditions (e.g., temperature, precipitation)	Field-based observations of dates of reproductive events for multiple years
Reproductive frequency	Average number of reproductive opportunities per year	Simulated variance in number of reproductive opportunities over multiple years	Observed reproductive events and measured variance over multiple years
Reproductive output/clutch size	Average number of eggs, offspring, or seeds per reproductive event	Relationship of eggs, offspring, or seeds per reproductive event based on female size or age	Measured variance in number of eggs, offspring, or seeds per reproductive event as measured in field
Onset of maturation	Average age, size at first reproduction, or duration of immature stages	Age or size at first reproduction; observed immature duration in focal regions	First reproduction as a function of age, size, and/or sex of individuals from field populations
Hatching (eggs)/germination (seeds) rate	Average estimate of eggs hatched or seeds germinated	Range of hatching or germination rate based on site-specific distinctions (e.g., water quality)	Estimate based on relationship with local environmental quality parameters
Immature transition rate (including metamorphosis)b	Average number of immatures to transition to next stage	Transition rate range in similar habitat	Estimates of transition based on local conditions and species
Sex ratio	Assumed even (0.5 male)	Estimate based on expected shift in operational sex ratio	Based on observed sex ratio
Recruitment rate	Average number of juveniles to become adults in the population	Range of recruitment based on site-specific distinctions	Field-based estimates (sex-specific) of recruitment
Survival rate	Average annual or stage or size specific survival rate	Observation or site-specific range in annual or stage or size specific survival	Field-based (sex-specific) estimates of annual or stage or size specific survival
Growth ratec	Average growth rate	Range in growth rate	Field-based estimates of growth
Seedling emergence; emergence after hibernation; emergence of new generation	Average date of emergence	Estimate based on environmental factors, e.g., temperature	Field-based estimates of emergence based on multi-year observations
Dormancy duration (inactive life stages, e.g., hibernation, soil seed bank)	NA	Distribution of dormancy based on environmental conditions, e.g., temperature, moisture	Field-based estimates of dormancy based on studies over multi-years or variable conditions

ERA = ecological risk assessment; NA = not available or applicable.

^aExamples of the types of information that could be considered to provide general, realistic, or precise information are described in each column.

^bIncludes discrete “growth” rates for transition of immature life stages depicted in stage-based models (Caswell 2001).

^cIncludes continuous growth rate for used for size-based functions.

Table 2.

Population and spatial characteristics that should be targeted to inform a population model for ERA^a

Characteristic	General	Realistic	Precise
Density dependence	NA or ceiling type	Stage-specific relationship of vital rate and density	Field-based relationship of density and demographic endpoint
Population size	NA or relative abundance	Abundance estimates based on available habitat and estimated vital rates	Recent field-based estimates and confidence limits of abundance for multiple years
Spatial metapopulation structure	NA	Connectivity functions for individuals in different subpopulations	Connectivity based on measured distances of species movement
Movement	NA	Movement estimates based on assumptions of habitat suitability	Movement estimates based on species-specific measurements
Habitat features	NA	Estimate of resource capacity based on available habitat	Field-based estimate of home range or breeding habitat availability
Geographical range	Regional occurrence of taxa of concern	Mapped habitat characteristics	Occurrence determined by field-based surveys
Habitat classification/suitability	Presence or absence of suitable habitat	Habitat suitability for reproduction and offspring development used to determine likelihood of breeding or stage-specific survival	Field-based habitat quality linked to fecundity/stage-specific survival based on needs for offspring production and development

ERA = ecological risk assessment; NA = not available or applicable.

^aExamples of the types of information that could be considered to provide general, realistic, or precise information are described in each column.

Table 3.

External factors that should be targeted to inform a population model for ERA^a

Characteristic	General	Realistic	Precise
Predation/herbivory	NA	Inclusion of predatory and/or herbivory influences where important	Predator–prey dynamics demonstrated to have significant influence on the species of concern
Competition	NA	Identification of interspecific competition where important	Quantitative dynamic relationship demonstrated to have significant influence on the species of concern
Environmental conditions	NA or random	Understanding of vital rate response to conditions (i.e., hydroperiod, seasonal forcing)	Field-based estimates of effects of reproduction/survival as function of environmental conditions
Stressors: pathogens	NA	Known impacts and spread of pathogen where important	Observed impacts on species of concern
Stressors: abiotic, other	NA	Known impacts of abiotic stressors or extreme events	Observed impact on species of concern
Existing management	NA	Known management plans that influence territories/habitat availability	Observed impacts of management effects included in parameter estimates
Indirect effects (obligatory relationships)	NA	Other known influences of resources on growth, survival, and/or reproduction	Measured reduced resource effects on growth, survival, and/or reproduction

ERA = ecological risk assessment; NA = not available or applicable.

^aExamples of the types of information that could be considered to provide general, realistic, or precise information are described in each column.

Table 4.

Exposure and effects characteristics that should be targeted to inform a population model for ERA^a

Characteristic	General	Realistic	Precise
Chemical exposure	Estimated EEC	Modeled fate and daily EECs based on environmental conditions	Modeled and measured concurrence of spatial and/or temporal pattern of exposure
Temporal exposure pattern	NA	Representative values of seasonal effects	High resolution (e.g., daily) modeling of chemical concentration and effect
Exposure pattern within and across habitat	NA	Estimate of proportion of population impacted by exposure based on spatial overlap. Estimated or hypothesized concentration distribution	Spatial distribution of chemical concentration and/or gradient
Representation of toxic effects	Threshold of effect	Modeled or hypothesized dose–response function	Measured dose–response functions. Estimation of effect via, e.g., TKTD models
Effects by life stage or size	Simulated impact based on assumptions or general trends (i.e., adverse outcome pathways)	Effect based on observations in representative species	Measured effects in focal species included in parameter estimates
Effects depending on exposure route (e.g., dietary, chemical in water)	NA	Modeled or simulated internal dose value, effects based on categorical data (e.g., diet, exposure route)	Effects linked to body burden measurements

EEC = expected exposure concentration; ERA = ecological risk assessment; NA = not available or applicable; TKTD = toxicokinetic–toxicodynamic.

^aExamples of the types of information that could be considered to provide general, realistic, or precise information are described in each column.

Consistent with Raimondo et al. (2018), Pop-GUIDE considers characteristics to exist on a spectrum from general to realistic with a degree of precision that reflects the degree of variability and/or confidence in the data. Characteristics that provide general information include limited taxonomic specificity (e.g., trait-based data) with minimal to no spatial and/or temporal resolution. “Realism” refers to the degree to which a characteristic incorporates key processes, structures, and mechanisms, and increases as processes become more reflective of the modeled species, location, and exposure scenarios. “Increased precision” refers to higher level of confidence that data are describing real world dynamics and can translate to narrow confidence intervals. Realism may be represented in a simplistic, binary fashion, such as incorporating a condition for reproduction to be turned on or off based on a threshold temperature or start of a season (e.g., excluding reproduction during winter months). Alternatively, realism can be introduced as a complex function of migration based on combined interspecific and environmental factors. Finally, theory can be applied to represent mechanistic processes and/or to substitute missing data to increase realism. For instance, physiology can be represented based on dynamic energy budget theory (Kooijman 2000), whereas feeding patterns can be based on foraging theory (Stephens and Krebs 1986). Thus, realistic characteristics can have low precision, and characteristics with high precision may not fully represent details of realistic scenarios. For instance, data from experiments conducted in controlled laboratory settings may provide high precision for a measured endpoint (e.g., survival), but may not be a realistic description of the process under field conditions that contain a number of important drivers not included in laboratory estimates (e.g., predation). These classifications of realism and precision may be correlated in some cases, but they are not always so. For the field-based characteristics listed in the precise columns of Tables 1 to 4, it is important to consider variability in the data, including where the data were collected relative to the location of interest.

The classification of characteristics discussed in Phase 2 is limited to general, realistic, and precise categories and does not include the hybrid categories (general–realistic, general–precise, realistic–precise) that are used to describe ERAs (Figures 1A and 2). The concept of hybrid categories is based on holistic ERA or models that consider the entire parameter space and needs of the ERA. At the resolution of model data and information, hybrid categories are not likely to exist for most characteristics, and replication may exist across categories, for example, because precise estimates may include realistic ones. Taxonomic surrogacy should also be documented in this phase because surrogate species are most often a function of available data rather than their representativeness of other species. In many cases, data may not be available for the focal species, and the adequacy of the surrogate in representing life history of the focal species will be important in understanding the level of generality and precision of a particular characteristic (Banks et al. 2014, 2019). Categorizing data available for each characteristic into general, realistic, or precise categories in this phase is performed to facilitate the evaluation of trade-offs as they pertain to the conceptual model.

In cases for which data are lacking, trait-based information can be used to reduce model uncertainty. Linking biological and ecological traits to the processes responsible for variation in exposure, sensitivity, or demography could inform initial estimation of population-level ERA (Rubach et al. 2011). Phylogenetic differences, as demonstrated in aquatic insects, can capture meaningful ecophysiological traits that explain variance in toxicant susceptibility (Buchwalter et al. 2008). In the context of Pop-GUIDE, traits may be included in Phase 2 to increase realism where they have been identified to influence a population’s susceptibility to chemical exposure (Awkerman et al. 2020). For data-limited taxa, such as the majority of listed species, trait-based approaches and comparative analyses provide a systematic approach for cross-species comparisons (Rueda-Cediel et al. 2019) and for identifying potential representative species for modeling. Traits could help fill data gaps for characteristics listed in Tables 1 and 3.

Delta smelt case study

A literature review was performed to access and evaluate the data that are currently available to develop a population model for the delta smelt (Supporting Information A Table SI-A 1). For each characteristic, the available data were categorized as general, realistic, or precise based on the definitions listed in Tables 1 to 4, and a summary of the data was placed in the corresponding box. Taxonomic specificity of the data and a reference were included for proper data documentation. Although extensive data are often not available for listed species, the delta smelt has been the subject of numerous recent studies as well as having been monitored by the Interagency Ecological Program for the San Francisco Estuary (IEP 2016). As such, field-derived data for organism-level, population, and spatial characteristics are available for the delta smelt, providing realistic and precise estimates for these characteristics. External factors, such as predation, competition, biotic and abiotic stressors, are qualitatively known but lack high-confidence empirical relationships or values, so were classified as general or realistic. Exposure patterns can be estimated using realistic application scenarios; however, effects of exposure are measured in only a few laboratory studies on surrogate species, which would be considered general in this context (Table 4, Supporting Information A Table SI-A 1).

Fathead minnow case study

The fathead minnow has been widely studied for almost a century, and abundant data are available to inform models for this species (Supporting Information A Table SI-A 2). As with the delta smelt case study, data from the literature were reviewed against the definitions provided in Tables 1 to 4 and assigned as either general, realistic, or precise. If a data characteristic was both realistic and precise, it was included in the precise column. For organism-level characteristics, reproducible data collected from both lab and field studies were available, as well as generalized, trait-based information. Because the fathead minnow is not a species of conservation concern and enjoys a widespread distribution, population and spatial characteristics are primarily general. External factors contribute a realistic understanding of ecological relationships but are not backed by validated empirical relationships, so these characteristics were categorized as realistic. Lastly, exposure and effects characteristics include general and realistic information that lack external validation, such that precision of these characteristics in field settings is unknown.

Phase 3: Decision Steps

Phase 3 of Pop-GUIDE facilitates model development by working through a series of decision steps: 1) life history representation, 2) organism-level processes (growth and development, maturation and reproduction), 3) population and spatial factors (population status, density dependence, movement and behavior, habitat characteristics), 4) external factors (diet, interspecific interactions), and 5) exposure and effects (toxicity data, temporal resolution). Each step consists of multiple decisions that evaluate both available data and the importance of various attributes to the model objective. Pop-GUIDE maintains the original primary decision steps of Schmolke et al. (2017b) but modifies the decisions within each step to be applicable across all taxa and to be consistent with data collected from Phase 2. Phase 3 should be considered an iterative process, with the conceptual model as the result of the final iteration of the decision steps (Figure 1C). The following paragraphs describe the decision steps provided in Supporting Information B, which can be used directly for ERA model development and documentation.

The first decision step identifies an initial life history representation. This is an important step in model development that summarizes the critical demographic processes to be included in the model. Although life history representations may be very similar to classical life cycle graphics, they differ in that not all life cycle components and processes are necessarily included in a model. The life cycle processes that are captured in a population model are dependent on available data. For example, if a species’ life cycle includes eggs, several larval stages, juveniles, and adults, but demographic and toxicological data are not available for any of the immature stages, a modeler may opt to represent only immature and mature individuals in the life history representation and associated model. Phase 2 of Pop-GUIDE informs what life history data are available for the modeled species (Table 1). Examples of life history representations for 6 broad taxonomic groups (fish, amphibians, mammals, birds, invertebrates, and plants) are provided in Supporting Information B as a starting point.

The second decision step is to evaluate how organism-level processes can be included in the model. Primarily, the processes described in this step address growth, development, maturation, and reproduction. The questions asked in this step focus on the type of data available (compiled in Table 1) that can be used for refined functions to describe these processes. The answers to the questions asked within this step identify where functions can be used to increase model realism (e.g., relationship of offspring number and female size) and/or precision (e.g., measured variance in number of offspring as measured in the field).

The third decision step is to identify population and spatial factors that need to be considered during model development. This step addresses important factors for population dynamics, such as population status, density dependence, movement, behavior, and habitat characteristics. Many of the questions within this step are directed toward the landscape level and involve the data or information available to refine spatial resolution of the model (e.g., important migration and/or dispersal factors). Although data may be available to address questions for some categories (e.g., population status, habitat characteristics), other categories within this step might increase model realism but with limited potential to robustly increase precision (e.g., density dependence, behavior). Examples of the types of data collected in Phase 2 that will assist with these decision steps are presented in Tables 2 and 4.

The fourth decision step considers external factors that could be incorporated in the model. These include abiotic factors and interspecific interactions associated with diet and other critical relationships. Diet may be a critically important factor to include in models, for example, where the prey of a focal taxon is expected to be susceptible to chemical impacts (e.g., acetylcholinesterase inhibitors on the prey of insectivores). Similarly, other interspecies relationships should be considered where impacts are expected to influence the modeled species’ sustainability with respect to the assessment objectives. These may include predation, competition, pathogens, pollination, or other mutualistic relationships. Abiotic factors and environmental conditions (e.g., drought) may also have significant interactions with chemical fate and effects and may be necessary to include for improved realism. Existing management plans should be considered if they are expected to influence the species’ likelihood of experiencing chemical impacts (e.g., species range partially located within a protected conservation area). Inclusion of external factors can increase model realism; however, it is important to note that data quantifying community interactions are typically scarce and validated empirical relationships will often be lacking. Many characteristics identified in this step target adding realism, so they may not be applicable to general models. Data collected in Phase 2 that will assist with these decision steps are presented in Table 3.

The fifth decision step asks questions pertaining to routes of exposure, potential effects, and temporal resolution. The questions identify the type of toxicity data available (e.g., acute, chronic), endpoints affected, and if those endpoints can translate into demographic endpoints. Also included in this step are questions that will help guide the optimal temporal resolution of the model. Data collected in Phase 2 that will assist with these decision steps are presented in Table 4.

Delta smelt case study

The Phase 3 decision steps for the delta smelt case study are provided in Supporting Information C. In step 1 of Phase 3 for delta smelt, the life history representation for fish with iteroparous spawning was selected because survival rates are available for egg, larva, juvenile, and adult stages (Figure 3). The decision to use this life history representation was supported by the available data for each life stage. As previously noted, delta smelt is a listed species for which data are relatively abundant, allowing for growth, maturation, and fecundity functions to be potentially included in the model (step 2). In step 3, the critically small population size of the delta smelt suggests both environmental stochasticity and demographic variation should be accounted for, and the questions on density-dependent relationships indicate the importance of representing survival as a function of density. Questions in step 3 also determine that migration and/or dispersal should be considered while explicitly representing spatial distributions of habitat and exposure. An acetylcholinesterase-inhibiting compound, chlorpyrifos has the potential to reduce delta smelt prey. The data collected in Phase 2 provide evidence that the smelt–prey relationship is a significant driver of smelt populations, and thus the consideration of diet was highlighted as an important factor to include in step 4. Step 4 questions also highlighted the delta smelt’s dependence on environmental conditions (i.e., freshwater flow, drought) as important factors to include, as well as any relevant management scenarios in place as part of the species recovery plan. The exposure and effects questions in step 5 determine that the aqueous route of exposure is the primary consideration for toxicological impacts on the delta smelt. Data from studies conducted with surrogate species show that chlorpyrifos is potentially lethal to fish. These lethal effects could be represented in the model through either toxicokinetic–toxicodynamic (TKTD) models or dose–response functions. Sublethal impacts on fish are also reported for chlorpyrifos, which can also be incorporated as dose–response functions within the model. The final decision step (5.4) determined, after evaluating all prior decision steps, that the temporal resolution of the model should be less than 1 y and reflective of the temporal scale of seasonal and environmental changes important to represent key processes.

Figure 3.

Life history representation used for both delta smelt and fathead minnow case study. Life history of both fish is represented by a fish with iteroparous spawning, where survival rates are available for egg, larva, juvenile, and adult stages.

Fathead minnow case study

The Phase 3 decision steps for the fathead minnow case study are provided in Supporting Information D. As with the delta smelt, the life history of the fathead minnow is represented by fish with iteroparous spawning, where survival rates are available for egg, larva, juvenile, and adult stages (Figure 3). Step 2 determined that data are available to include functions for growth, maturation, and fecundity in the model. Within step 3, demographic variation and density-dependent growth and fecundity were identified for possible model inclusion, but no additions to the model for movement, behavior, or habitat features were needed. Although fathead minnow prey could be impacted by chlorpyrifos exposure, there are no data to support how the fish might be impacted by alterations in prey abundance, so categorical or qualitative impacts to diet could be included in the model, as determined in step 4. The only other potential addition to the model identified in step 4 is inclusion of seasonal drivers in population dynamics, such as overwinter survival in young of year fish. In step 5, exposure and effects decisions for the fathead minnow were similar to those for the delta smelt, given that both case studies relied on the same toxicity test information. Similarly, the time step for the fathead minnow model was also determined to be less than 1 y to include the temporal scale of seasonal and environmental changes that are important to key population processes.

Phase 4: Conceptual Model

A conceptual model, as described here, provides a high-level, graphical, and textual summary of the components and functions within a model and their linkages. The questions of Phase 3 identify what functions could or should be included to develop a conceptual model representative of a population exposed to a chemical stressor. The development of the conceptual model begins with the initial life history representation determined in Phase 3 and includes survival, growth, and reproduction, which are parameterized using data collected in Phase 2 (Table 1). The life history and ecological attributes identified in Phase 3 that should be included in the model will help identify the trade-off between generality and realism based on the data available. A general model would result where surrogate species data or theoretical principles largely inform demographic processes or where decisions are made to exclude processes that are not well supported by available data. Realistic models would result from species- or location-specific data and decisions to include processes that reflect important scenarios even if they are assumption or theory based. A cursory evaluation of precision can be made at this stage based on the type of available data for different aspects of the model, but precision of the entire model within its designed application cannot thoroughly be evaluated until the model is implemented and uncertainty and sensitivity analyses are performed (see Phase 5). In practice, the data compilation performed in Phase 2 will often result in a mix of general, realistic, and precise information for model development. Determining whether the resulting model is realistic enough or precise enough for the risk assessment and whether it aligns with the ERA objective can be informed by data compiled in Phase 2 and model evaluation in Phase 5. However, this determination will inevitably involve the professional judgment of key stakeholders.

It should be noted that the decision to increase realism by including functions that are not based on robust empirical data may reduce precision (Raimondo et al. 2018). Rather than increasing model complexity solely based on the availability of data, functions should be added to the base model with these trade-offs in mind. Although the decision steps identify potentially important factors to consider, the model developer should consider the needs of the ERA objective before determining which relationships to include in the conceptual model. Conceptual models for the case studies are presented both graphically (Figure 4) and with narrative descriptions that combine the results of Phases 2 and 3 into a summary of processes that will be included in the model. It should be noted that mathematical formalisms that may have been identified in the preceding phases may not be explicitly represented in the conceptual model figure (e.g., density dependence functions).

Figure 4.

Conceptual model for assessing the risks of chlorpyrifos exposure to delta smelt (A) and fathead minnow (B). The entirety of the modeling space is represented by the large grey box. Chemical exposure (orange box) and effects (green box) are overlapping compartments within the model. The white boxes are model components, and the arrows depict their connections. Red arrows depict adverse pathways of chlorpyrifos, and black arrows represent all other connections. The extent of overlap for various compartments is conceptual and not intended to be a scaled representation of proportional overlap of layers.

Delta smelt case study

The conceptual model for the delta smelt is represented Figure 4A. It contains the entirety of the modeling domain represented by the large grey box. Within the modeling space, there is an overlapping subcomponent for chemical exposure (orange box) and chemical effects (green box). The white boxes are model components and the arrows depict their connections. Red arrows depict adverse pathways of chlorpyrifos, and black arrows represent all other connections. The components represented by spatially explicit landscape, habitat management, variation in habitat suitability, and environmental stochasticity are independent of exposure and effects. Organism distribution and migration are driven by habitat suitability and life history and are largely independent of exposure. However, there may be some interactions between chemical exposure and organism distribution and/or migration, hence this model component partially overlaps with the chemical exposure space of the conceptual model. Model output is depicted by the blue circle. Extent of overlap for various compartments is conceptual and not intended to be a scaled representation of proportional overlap of layers.

Within the chemical exposure space, chemical and toxicological properties are used to inform the environmental concentration via exposure models, which are not discussed within the scope of Pop-GUIDE. However, toxicological properties, such as the acetylcholinesterase-inhibiting mode of action of chlorpyrifos, can be used to indicate a direct impact on both delta smelt and their prey. Because both distribution and density dependence have been demonstrated to be influenced by prey availability, potential impacts of chemical exposure on prey abundance are included. The box for organism-level effects on fish represents effects measured in laboratory studies of surrogate species and may also include potential impacts that result from reduced prey (e.g., reduced growth). The level of these effects on the population is also driven by environmental concentration.

Chemical exposure and effects inform the dynamics depicted by the life history representation, which includes all 4 life stages (egg, larva, juvenile, adult). As the focus of an ongoing, active management strategy and recovery plan, stage-based field data collected for more than 15 y provide demographic rates with known variance for the delta smelt (USFWS 1996). Where field-based information was not available for some characteristics, lab-based estimates for the delta-smelt provided realistic estimates (e.g., size–egg relationship). Based on available data and the answers to the decision steps on life history characteristics (Phase 3, step 2), the life history representation can include a simple, continuous growth function and maturation and fecundity based on body size. Indices of population abundance for each life stage can be used to incorporate realistic variation around the demographic rates and population growth rate. Given that the population is very small, additional environmental stochasticity should be included to account for extrinsic influences on the population as well as noted trends in density dependence. Because seasonally varying demographic rates and life history attributes are important to realistically capture delta smelt dynamics, they are also included.

Fathead minnow case study

The conceptual model for the fathead minnow is represented in Figure 4B. Within the chemical exposure space, chemical and toxicological properties are used to inform the environmental concentration via exposure models, and toxicological properties inform a direct impact on various life stages of fathead minnow. The level of these effects on the population is also driven by environmental concentration. Chemical exposure and effects inform the dynamics depicted by the life history representation. Given that overwinter survival is important to most fish in temperate regions, and seasonality is important to appropriately align exposure scenarios with demographic parameters for pesticide risk assessment, incorporation of these elements is also included. Although the data availability for the fathead minnow offered the potential to develop a more realistic and precise model, Phase 3 guided us toward a more general model for a broader application that met the ERA objectives and matched the ERA trade-off space.

Phase 5: Model Implementation and Evaluation

The first 4 phases of Pop-GUIDE identify the data that are available and processes that should be included in the model, resulting in a conceptual model that links the components together. Mathematical formalism is not considered in the decision steps because this may vary based on available data, decisions made in Phase 3, and preference of the model developers. However, the model implementation and evaluation described in Phase 5 assumes the modeler extends the conceptual model to a fully parameterized computational tool following best modeling practices. Further guidance on model type and the implementation of different key features in population models of relevance for ERA is provided by Accolla et al. (2020). Following model parameterization (direct or via calibration), model analysis should be performed to evaluate the behavior and performance of the model to provide information on its uncertainty and how it can be used for regulatory purposes (EFSA 2014). As the modeler moves through Pop-GUIDE, any functions or theoretical concepts that are deemed relevant during the decision steps should be documented because this will allow more straightforward model implementation and evaluation (Grimm et al. 2014).

Model implementation requires a full model description to transparently convey the model to risk managers and stakeholders. Pop-GUIDE provides a crucial part of this documentation organized into the distinct phases. The materials created through the execution of the phases (e.g., Phase 2 tables, Phase 3 decision steps) can provide primary or supplemental documentation of the model building process. Because the conceptual model does not include the comprehensive description of model mathematics, a complete description of model formalisms needs to be generated with the model implementation. Full model description formats, such as the Overview, Design, Details (ODD; Grimm et al. 2006) and TRAnsparent and Comprehensive model Evaluation (TRACE; Schmolke, Thorbek, DeAngelis et al. 2010; Grimm et al. 2014) can be adapted to incorporate the phases of Pop-GUIDE. Model descriptions, such as ODD, should allow the reimplementation of the model by third parties.

To be consistent with the original framework described by Raimondo et al. (2018) and Pop-GUIDE, the trade-offs of the final model should be identified and compared to that of the assessment. The model trade-offs are determined through a holistic evaluation of the type of data (Phase 2) and functions (Phase 3) included, and the sensitivity of the model to the various data or assumption-driven functions. For example, if a model excludes functions that increase realism and a sensitivity analysis demonstrates that the model is largely driven by field-based parameters with high certainty, the model would be classified as general–precise. If the trade-off category of the model matches that of the assessment as determined in Phase 1, the model may be applied as a direct assessment tool. If data and/or resources are not available to develop a model that falls in the same category as the ERA objective, then the model may be used as part of the weight of evidence, identifying differences in the trade-offs as they apply in the ERA uncertainty analysis.

The model evaluation hinges on implementation in appropriate software (i.e., coded) and includes data evaluation, sensitivity analysis, uncertainty analysis, robustness analysis, and model validation (Rykiel 1996; Railsback and Grimm 2011; Augusiak et al. 2014; EFSA 2014). These analyses can help identify parameters that have a high influence on model outputs but may be highly uncertain. In Pop-GUIDE, data evaluation is conducted in Phase 2; however, this transgresses into Phase 5 to determine the quality of the functions that were used to design the model (general trends, empirical knowledge, etc.). Sensitivity analysis investigates how influenced the model outputs are by the values of model parameters and structure, and quantitatively identifies the magnitude of each parameter’s contribution to model output. It is also a part of a robustness analysis, which aims to determine the influence of more structural model elements on model outputs (Grimm and Berger 2016). Uncertainty analysis describes and evaluates the factors that affect the uncertainty of model outputs. Sources of uncertainty are those linked to model structure, which is a simplification of the real system being modeled, and/or parameter estimation as a result of measurement errors, biological variability, or extrapolation among species or environments (EFSA 2014).

In this phase, the model is evaluated to determine if the functions and data that are most influential to the output are consistent with trade-offs of generality, realism, and precision of the ERA. Model evaluation may prompt modelers to reformulate part of the conceptual model, reevaluate data, or even gather additional data when possible. For example, if a model robustness analysis (Grimm and Berger 2016) were to demonstrate that arbitrarily defined functions used to increase realism heavily influence model outcome and uncertainty but are not needed to match a lower level of realism required by the ERA, the model developer may opt to remove those functions and reduce model complexity while ensuring an accurate representation of species characteristics and population dynamics. Finally, to estimate how well the model represents populations of species of interest, validation of the model should be conducted if independent empirical data are available for comparison to model outputs (Rykiel 1996; Augusiak et al. 2014; EFSA 2018; Schmolke et al. 2020). However, full model validation is often not feasible because relevant empirical data sets are not available. In some cases, validation can be approached in a more qualitative way, often termed “pattern-oriented modeling” (Grimm and Railsback 2012), which addresses how well the model represents multiple patterns in empirical data rather than the quantitative comparison to empirical data corresponding to the risk assessment endpoints.

Discussion

Pop-GUIDE integrates and advances guidance for developing and documenting population models for ERA and provides a transferrable, consistent, and transparent approach to evaluate model applicability and uncertainty for the purpose of meeting a risk assessment objective. The value of a model can be evaluated only in the context in which it was developed, and by developing models within the trade-off space of generality, realism, and precision of the ERA, model uncertainties can be consistent with those tolerated by the assessment. Pop-GUIDE preserves all the concepts presented in Schmolke et al. (2017b) and Raimondo et al. (2018) and combines them into a comprehensive approach that is consistent with the ERA paradigm from problem formulation through risk characterization. Although Pop-GUIDE focuses specifically on population models developed for ERA, the foundational principle that the trade-off space of the application objective should guide model development can be applied more broadly to other model types and applications.

Complexity and uncertainty are central to both ERAs and model building (Brooks and Tobias 1996; Schmolke, Thorbek, Chapman et al. 2010; Accolla et al. 2020) and Pop-GUIDE is developed to align these attributes so that model outputs are of a sufficient quality to serve as the basis for a decision (USEPA 1998). For example, ERAs at the screening or national level may contain low spatiotemporal resolution, be general to species or location, and can thus be relatively less complex than those used beyond the screening level. In these types of assessments, models are likely to be used to forecast possible toxicant effects at broad scales or to test different management strategies, and therefore, as an example, site-specific uncertainty is more acceptable. This is demonstrated in our case study for the fathead minnow. Conversely, some assessments will require a higher level of realism and precision, and models will be required to include more spatial context and resolution, as was demonstrated by our delta smelt case study. Differences in model complexity for the 2 case study species are demonstrated in the conceptual models, which show several model components included for the realistic–precise delta smelt model that are not included in the general fathead minnow model (Figure 4). For some species, metapopulation dynamics may be a significant driver of a population and require additional spatial context to achieve the level of realism and precision required by a model or an ERA. The phases of Pop-GUIDE will identify where such levels of complexity are necessary to meet these ERA requirements and approach model development as a function of available data and requirements of the assessment.

When evaluating model and data characteristics in terms of generality, realism, and precision, it is most meaningful to consider these terms in a specific context. For example, a model may be defined as more general with respect to its spatial extent because it is not parameterized for a particular geographic location, or a model may be considered to be more realistic (or precise) because natural mortality rates are used as opposed to having survival rates based solely on laboratory measurements (Levins 1993). Similarly, use of the term “precise” also requires context, and its use may be controversial or difficult to determine in some applications. Before model implementation, discussions among stakeholders could clarify which characteristic should be more realistic or more precise for the particular ERA objective, where possible. If the conceptual model does not meet the level of realism and/or precision that is required of the ERA objective, a determination will need to be made on its appropriate application in the assessment, as initially identified in Phase 1. In the delta smelt case study, the ERA objective targeted a realistic–precise model to be used as a direct assessment tool, which was plausible with the data available in this example. If data were not available to develop a model that was realistic and precise enough, the risk assessor and stakeholders would need to determine if the model supports other data for in a weight of evidence approach. If a model lacks the complexity to be implemented as a direct assessment tool, it may be used to guide mitigation and “what if” scenarios, recovery plans, or relative risk assessments. It should also be noted that in some cases a risk assessor may determine throughout the phases of Pop-GUIDE that data and information are too limited to develop a model that could serve the assessment in a defensible way with available resources. For example, Phase 1 may determine that resources are not available for model development, or Phase 2 may determine data are not available for even the most simplistic model. This process provides a more specific guidance process that can be tailored to individual assessments while being consistent with previously published recommendations for best modeling practices (Pastorok et al. 2002; Barnthouse et al. 2008; Wentsel et al. 2008; Schmolke, Thorbek, DeAngelis et al. 2010; Schuwirth et al. 2019).

Pop-GUIDE was developed to be applied following the problem formulation phase of a chemical-specific risk assessment using the ERA objective as its starting point. In this way, Pop-GUIDE puts the ERA objective at the beginning of the model development process, which is a shift from past practices that tended toward developing a model with available data and fitting it into an assessment post hoc. Within Phase 1, a risk assessor is asked to identify how a model would ideally be used in the assessment, while recognizing that its final application may change based on an evaluation of the data and model (Phase 5). Because the trade-off categories represent a relative space that relies on the ERA for context, Pop-GUIDE is not intended to guide model development without a predefined assessment objective. Although there is value in developing flexible models independent of an ERA that can be included in a risk assessor’s toolbox, Pop-GUIDE will aid in customizing and evaluating such a model prior to consideration for a particular assessment. The applicability domain of Pop-GUIDE as presented here is for chemical-specific risk assessment and does not currently address potential confounding issues that may be present for assessments involving mixtures or multiple stressors (e.g., invasive species, habitat loss). However, Pop-GUIDE could be modified for chemical mixtures by adding relevant information on multiple chemical components.

When applying Pop-GUIDE, professional judgment will be required to evaluate data for related or surrogate species where data are lacking for the focal species. Toxicity effects data are typically limited to a small number of standard test species, and life history characteristics will be limited for most species. In such a case, a model developer will need to determine if surrogate species or trait-based data are sufficient substitutes for missing data, or if the development of a targeted realistic and/or precise model is not possible with available data. The delta smelt case study demonstrated a listed species for which abundant data were available, relying on surrogate species only to represent toxicity effects. Even such limited use of surrogate data introduces uncertainty into the model that should be evaluated. Banks et al. (2014) demonstrated through modeling of 4 different parasitoid wasp species that any 1 species could not predict how the other species would respond to pesticide exposure, noting uncertainty in extrapolating even across closely related species. When possible, a separate evaluation of surrogate species and their ability to represent the focal species will strengthen uncertainty analyses associated with Pop-GUIDE (e.g., Banks et al. 2019).

Transparent documentation of the entire modeling cycle is essential if models are to be more broadly accepted as useful tools for decision making. Systematic model description protocols, such as the ODD (Grimm et al. 2006) and TRACE (Schmolke, Thorbek, DeAngelis et al. 2010; Grimm et al. 2014) formed a basis for documenting steps listed in EFSA’s good modeling practice opinion (EFSA 2014), but their focus is on model documentation and description and less so on model development, especially for environmental decision making and ERA. Thorough documentation of the decision process within Pop-GUIDE makes assumptions explicit, particularly for processes that are not represented in the model due to lack of data or other considerations. Although the descriptions of a model and its evaluation are essential, they are often difficult to assess by experts and stakeholders who were not involved in the modeling process and who may not be modelers themselves. The Pop-GUIDE process describes the model with the decisions and assumptions taken throughout model development with the focus on the biological processes. Stakeholders and other experts can participate in and comment on this process on a par with the modelers. This transparency is essential to achieve buy-in from stakeholders prior to full model implementation and testing.

Risk managers are charged with interpretation of models and ERAs to make sound management decisions. In this role, they need to ensure that their decisions reflect a strong scientific basis of the models used and that they understand both qualitative and quantitative uncertainties of the models and assessments. Pop-GUIDE provides a logical, consistent, and transparent process for population model development and documentation that can be applied to all taxa. It is the most comprehensive guidance for population model development provided to date and is the first of such documents that includes an evaluation of uncertainty as a function of the tolerance of the assessment in which it is applied. Pop-GUIDE will be valuable for model developers to ensure the robustness and reproducibility of their models, and for model users (e.g., risk assessors and policy makers) to aid understanding of population models and build confidence in their use for ERA and decision support. The principles underlying trade-off uncertainties that define the Pop-GUDE framework can be applied to other types of models used in ERA, as well as for population models developed for applications beyond risk assessments.