Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2020 Oct 8;17(1):131–146. doi: 10.1002/ieam.4331

Parameterization Framework and Quantification Approach for Integrated Risk and Resilience Assessments

Mariana Goodall Cains 1,✉, Diane Henshel 1
PMCID: PMC7821186  PMID: 32841472

ABSTRACT

A growing challenge for risk, vulnerability, and resilience assessment is the ability to understand, characterize, and model the complexities of our joint socioecological systems, often delineated with differing natural (e.g., watershed) and imposed (e.g., political) boundaries at the landscape scale. To effectively manage such systems in the increasingly dynamic, adaptive context of environmental change, we need to understand not just food web interactions of contaminants or the flooding impacts of sea level rise and storm surges, but rather the interplay between social and ecological components within the inherent and induced feedforward and feedback system mechanisms. Risk assessment, in its traditional implementation, is a simplification of a complex problem to understand the basic cause‐and‐effect relationships within a system. This approach allows risk assessors to distill a complex issue into a manageable model that quantifies, or semiquantifies, the effects of an adverse stressor. Alternatively, an integrated risk and resilience assessment moves toward a solution‐based assessment with the incorporation of adaptive management practices as 1 of 4 parts of system resilience (i.e., prepare, absorb, recover, and adapt), and directly considers the complexities of the systems being modeled. We present the Multilevel Risk and Resilience Assessment Parameterization framework for the systematic parameterization and deconstruction of management objectives and goals into assessment metrics and quantifiable risk measurement metrics and complementary resilience measurement metrics. As a proof‐of‐concept, the presented framework is paired with the Bayesian Network–Relative Risk Model for a human‐focused subset of a larger risk and resilience assessment of climate change impacts within the Charleston Harbor Watershed of South Carolina. This new parameterization framework goes beyond traditional simplification and embraces the complexity of the system as a whole, which is necessary for a more representative analysis of an open, dynamic complex system. Integr Environ Assess Manag 2021;17:131–146. © 2020 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC)

Keywords: Bayesian networks, Complex systems, Indicators, Metrics, Risk assessment

KEY POINTS

  • Integrated risk and resilience assessment is solution‐oriented with the incorporation of adaptive management practices as part of system resilience (i.e., prepare, absorb, recover, and adapt) and directly considers the complexities and uncertainties of the complex system(s) being modeled.

  • The Multilevel Risk and Resilience Assessment Parameterization framework is a systematic approach to operationalize conceptual models into risk and resilience metrics and their respective interactions for quantification during the analysis phase of an integrated risk and resilience assessment.

  • The Bayesian Network‐Relative Risk Model can incorporate resilience parameters as control factors that modulate how risk is carried through the model from sources of stress to endpoints of value.

INTRODUCTION

Watersheds, airsheds, and other landscape‐scale ecosystems—especially those including humans—are complex systems. These systems, depending on spatial and temporal scale, contain multiple interacting ecosystems, human activities, ecosystem services, and feedback mechanisms between humans and ecosystems, that is, socioecological systems. Additionally, these socioecological systems are governed by various levels of policies and regulations that affect humans and their uses of, and interactions within, a given region. When assessing at landscape or system scale, the impact of active processes on metrics must be taken into account through data collection, modeling, or both, and models need to integrate the interactions and influences between metrics (Schröder and Seppelt 2006). The key to modeling complex and dynamic systems, as indicated by John Boyd's conceptualized Destruction and Creation, is first parameterization and second metric integration (Boyd 1987). The process of parameterization deconstructs the system into its representative components to identify system critical assessment and measurement metrics and interactions. The subsequent integration of the metrics and models of their interactions produces a single, iterative, dynamic model.

Little formal guidance exists to help the practitioner with the parameterization and integration of multiple metrics of dissimilar sources that enable a quantitative integration of multidomain, multistressor, and multiendpoint risk assessment metrics (Suter 2007). Although cumulative and integrated risk assessment models recognize and allow for multimetric integration, the frameworks provide a way to conceptualize, rather than calculate, cumulative or multistressor risk (USEPA 2003). Efforts to advance multistressor, cumulative, and holistic assessments are exemplified by the Ecosystem Based Management‐Driver, Pressure, State, Ecosystem service, and Response (EBM‐DPSER) conceptual model (Kelble et al. 2013); the Chesapeake Bay oyster restoration study (Richkus and Menzie 2013); Common International Classification of Ecosystem Services framework (ECC 2017); and the Bayesian Network–Relative Risk Model (Ayre and Landis 2012; Landis et al. 2017). The Bayesian network analytical approach paired with the Relative Risk Model conceptualization enables the quantitative integration of dissimilar measurement metrics and data for multistressor, multiendpoint risk assessments, which is necessary for the assessment of socioecological systems.

Purpose and motivation

We present a framework for the systematic parameterization and deconstruction of management objectives and goals into assessment metrics and quantifiable risk measurement metrics and complementary resilience measurement metrics. In contribution to the field of integrated (i.e., holistic) socioecological risk assessments, the presented multilevel risk and resilience parameterization framework facilitates operationalization within the problem formulation phase for complex systems. Operationalization is the “process of defining the measurements of a phenomenon that is not directly measureable, though its existence is inferred by other phenomena” (Wikipedia 2020).

Risk is the probability of an effect (i.e., consequence) on an endpoint resulting from exposure to a stressor (i.e., hazard; NRC 2009). Risk assessment, in its traditional implementation, is a simplification of a complex problem to understand the basic cause‐and‐effect relationships within a system. Such an approach allows risk assessors to distill a complex issue into a manageable model that quantifies, or semiquantifies, the effect of an adverse stressor. Resilience is the ability of a system to plan and prepare for the stressor, absorb the perturbations induced by the stressor, recover from damage created by the stressor, and adapt to prevent future severe effects from the same stressor (NRC 2012). Risk and resilience are clearly related concepts but are not solely inverse measures of each other (Cains and Henshel 2019). In a simplified framing of the difference, risk assessment characterizes the potential impacts of stressors on the system; resilience assessment characterizes the response and plasticity of the system to stressors that reflect the homeostatic mechanisms of the system (Carpenter et al. 2001).

An integrated risk and resilience assessment moves toward a solution‐based assessment with the incorporation of adaptive management practices as 1 of 4 parts of system resilience (i.e., prepare, absorb, recover, and adapt), and directly considers the complexities of the system(s) being modeled. The presented risk and resilience parameterization framework paired with Bayesian network analysis embraces the complexity of the system as a whole, which is necessary for a more representative analysis of an open and dynamic complex system.

Paper structure

In this paper, we first discuss and compare the problem formulation stage of select assessment frameworks relevant to the integrated assessment of risk and resilience of socioecological systems. We then examine the role of parameterization within risk assessment problem formulation and review resilience assessment scoping. Next, we introduce the expanded Multilevel Risk and Resilience Assessment Parameterization (MulRRAP) framework for the systematic deconstruction of assessment metrics into quantifiable measurement metrics as a critical process within problem formulation. The utility of the presented parameterization framework is exemplified through a case study focused on the Charleston Harbor Watershed region of South Carolina, USA. The case study uses a Bayesian network analysis approach to quantify the risk of regional climate change impacts within the context of managing for increased resilience to climate change (e.g., advanced warning systems for flood events, reinforced sea walls). Last, the parameterization framework is placed within the current body of environmental risk assessment research and complex system modeling.

THE ROLE OF PROBLEM FORMULATION

Across multiple disciplines, the problem formulation step is noted as the most critical step and foundation of the risk management and risk assessment process (Dai et al. 2002; Suter et al. 2003; Zavadskas et al. 2010; Oltramari et al. 2015). Although the specific vocabulary and components of the problem formulation step have evolved with each domain ideation and application of the risk management and risk assessment process, problem formulation establishes the scope, context, and criteria specific to the assessed management questions and goals through conceptual models and analysis plan development (USEPA 1989, 1998, 2003; Suter et al. 2003; ISO 2018).

Risk analysis, especially the assessment and management of risk, has been practiced for millennia (Bernstein 1996; Aven 2016). In 1983, the National Research Council (NRC) established the foundation of environmental and human health risk assessment as the construction of a single stressor risk characterization through the process of hazard identification, release and exposure assessments, and a dose–response curve (i.e., an exposure–response relationship; NRC 1983). The hazard identification lays out the beginnings of problem formulation. In 1989, the USEPA promulgated the Risk Assessment Guidance for Superfund (RAGS), which adapted the NRC framework into a formalized process for human health risk assessment of hazardous substances. The RAGS framework renames and repurposes the hazard identification phase to data collection and data evaluation, which includes guidance on the development of a site‐specific conceptual model and analysis plan (USEPA 1989). The NRC and RAGS frameworks were developed for the assessment of one risk agent (e.g., chemical, stressor) at a time and the qualitative aggregation of assessment results at the end.

The problem formulation step, formally named such in the USEPA Guidelines for Ecological Risk Assessment (ERA), (1) identifies assessment endpoints (i.e., quantifiable attribute of a specified entity) relevant to the risk management goals; (2) builds conceptual models illustrating the relationships between stressors and assessment endpoints; and (3) develops an analysis plan appropriate for the selected assessment endpoints and available data (USEPA 1998). The USEPA ERA guidelines presented the first risk assessment framework that acknowledged and attempted to address the system‐level effects within the risk assessment process and provided a foundational framework for the contemporary development of the Framework for the Integration of Health and Ecological Risk Assessment (Suter et al. 2003), USEPA Framework for Cumulative Risk Assessment (USEPA 2003), and USEPA Framework for Human Health Risk Assessment to Inform Decision Making (USEPA 2014a).

Cumulative and integrated risk assessments examine and integrate, within a system, either multiple stressors from a single domain or multiple similar stressors from multiple domains. A simple risk assessment only examines one domain (e.g., human health, environment) of risk within a system, which forces the result of that risk assessment to reside within the predetermined domain of risk. A cumulative risk assessment is the “analysis, characterization, and possible quantification of the combined risks to health or the environment from multiple agents or stressors” (USEPA 2003), whereas an integrated risk assessment “combines the processes of risk estimation for humans, biota, and natural resources in one assessment” (Suter et al. 2003). The problem formulation phase of a cumulative risk assessment and an integrated risk assessment share the same structure and output (i.e., assessment endpoints, conceptual model, and analysis plan) but differ in the scope used to inform the problem formulation phase. The integrated risk assessment scope explicitly integrates human, ecological, and natural resource endpoints, whereas the cumulative risk assessment assesses human health and/or the environment (i.e., “ecological targets”; Suter et al. 2003; USEPA 2003).

The existing traditional risk assessment guidance is not sufficient for complex systems as it was developed for simplified systems analysis (e.g., baseline of single stressor to single endpoint). Integrated multistressor risk assessments move beyond system simplification and embrace the complexity of the system as a whole, which more accurately reflects the complexity of interacting stressors on interacting nodes of a system.

Problem (re)formulation for complex systems

The assessment of complex systems at a regional scale can be overwhelming because of (1) a lack of problem formulation guidance relevant to complex systems; and (2) the layered complexity of socioecological systems with internal and external interactions and processes that cannot always be directly measured or modeled (e.g., Bristol Bay, Alaska ecological risk assessment; USEPA 2014b). Conceptual models are developed during the problem formulation phase of risk assessment (Figure 1A and B) to identify relevant parameters and relationships within the system for a given space and time. Conceptual models serve as the foundation for the quantitative model built during the analysis phase of problem formulation. Figure 1 is an example of the systematic development from a generic regional conceptual model (Figure 1C) to a parameterized conceptual model (Figure 1D) with an elaboration of the relationships between those parameters leading to a specific endpoint (e.g., ‘Human Health’; Figure 1E).

Figure 1.

Figure 1

Open in a new tab

The generic framework for risk assessment (A) and the Problem Formulation structure expanded to include multilevel risk and resilience assessment parameterization framework and parameterized conceptual model (B). The resilience‐integrated Relative Risk Model modified to include control factors (C) is used to outline a site‐specific multistressor, multiendpoint conceptual model. (D) Integrated risk and resilience conceptual model of climate change impacts in Charleston Harbor Watershed, South Carolina, USA, from which an endpoint‐specific conceptual model is built (E) Conceptual model of risk to human health caused by climate change‐compounded coastal flooding and parameterized (Figure 4) to serve as the structural foundation of the Bayesian network in Figure 5. MulRRAPP = Multilevel Risk and Resilience Assessment Parameterization.

Conceptual models organize and visualize the connections between the sources of stress in the system, the location and habitats affected by the stress, the effects of the stress on the system, and the subsequent impacts on assessment endpoints (van den Brink et al. 2016). Conceptual models also enable integration of the models characterizing the interactions between system nodes (i.e., entities), thus visualizing dynamic system states. The specific structure and content of the conceptual model are dependent on the assessment management objectives and the system in question. The components of the conceptual model can be generalized into sources, stressors, exposure, effect, and impact on endpoints of interest (USEPA 1989, 1998, 2003, 2014a; Landis and Wiegers 1997). Definitions of each term can be found in Table 1.

Table 1.

Risk assessment terminology resulting from the integration and modification of terminology used by numerous environmental risk assessment effortsa

Source Any system input or activity that produces stressor(s) in the system of interest.
Stressor Any entity or process (e.g., physical, chemical, or biological) that causes or can cause an effect, either positive or negative, to, on, or in an entity.
Entity An organism, resource, or service of the natural or built system that has the potential to be affected by a stressor. It is the attributes of an entity that provide value to a system.
Habitat The type(s) of environment and/or location in which the entity is found.
Exposure When there is an interaction between an entity with a stressor within a defined space and time, which results in an effect to, on, or in an entity.
Effect A change in the state or dynamics of or in the entity resulting from exposure to a stressor.
Response The effect of exposure to a stressor to, on, or in an entity.
Assessment endpoint or impact An entity or characteristic of the natural or built system that is of value to society, the local community, and the ecology of the system.
Measurement endpoint or metric An effect or response that is measurable in or for an entity and, ideally, causally links the effects of a stressor to an assessment endpoint.
Open in a new tab
a

For example, human health, ecological, cumulative, integrative, and regional (Landis and Wiegers 1997; USEPA 1998, 2003, 2014a, 2014b, 2019; Suter et al. 2003; Suter 2007; NRC 2009).

The analysis plan is the last step of the problem formulation phase of risk assessment and is specific to the risk management question being assessed. Where the conceptual model visualizes the flow of information and subsequent risk propagation within a system, the analysis plan identifies the measures used to quantify the sources, stressors, exposure, effect, and impact on the endpoints of interest (USEPA 1998, 2003, 2014a). Although research and formal documentation on the development of conceptual models and selection of assessment metrics do exist, there is currently no framework that explicitly guides the parameterization of measurement metrics and their interrelationships. The expanded multilevel risk and resilience assessment parameterization framework (Figure 2) introduced in this paper outlines a systematic approach to operationalize conceptual models into risk and resilience metrics (Figure 3), and their respective interactions (Figure 4), for quantification during the analysis phase of risk assessment (Figure 5).

Figure 2.

Figure 2

Open in a new tab

Structure of the complementary multilevel risk and resilience assessment parameterization (MulRRAP) Framework (C) within the problem formulation step (B) of the generic risk assessment framework (A). The bracketed numbers in the upper right corner of the MulRRAP components signify the parameterization sequence further detailed in Figure 3 and Tables 2 and 3. The dotted arrows flowing from the “Resilience Goals → Resilience Indicators” denote where resilience parameters can influence risk propagation. The dash‐dotted boundaries encompass the risk and resilience parameters subject to data quality and uncertainty assessment.

Figure 3.

Figure 3

Open in a new tab

The problem formulation structure expanded to include the multilevel risk and resilience assessment parameterization (MulRRAP) framework and parameterized conceptual model (A). Hierarchical taxonomy of the MulRRAP framework to identify risk assessment metric and measurement metrics (B) and resilience goals and indicators (the 5 black dotted rounded rectangles within C) represent the control factors for the management objective and goal of “Determine and minimize risk to human health due to climate change compounded flood events.” The bracketed numbers in the upper left corner of the MulRRAP components signify the parameterization sequence outlined in Figure 2 and detailed in Figure 4. The gray dash‐dotted boundaries encompassed the risk and resilience parameters whose data quality and uncertainty must be quantified. AD = assessment domain; AS = assessment subdomain; AS = assessment endpoint; AM = assessment metric; DQU = data quality & uncertainty; Ef = effects; EMM = exposure measurement; Ex = exposure; MM = measurement metric; PS = primary stressor; SMM = stressor measurement metric; SS = secondary stressor; TS = tertiary stressor.

Figure 4.

Figure 4

Open in a new tab

(A) The problem formulation structure expanded to include the multilevel risk and resilience assessment parameterization (MulRRAP) framework and parameterized conceptual model. (B) Overlay of the MulRRAP framework (gray‐outlined and numbered boxes) and relative risk model (rectangle, arrow, and rounded rectangle) produces the parameterized conceptual model (gray‐filled shapes) which provides the structural foundation of the Bayesian network in Figure 5.

Figure 5.

Figure 5

Open in a new tab

(A) The problem formulation structure expanded to include the multilevel risk and resilience assessment parameterization (MulRRAP) framework and parameterized conceptual model. (B) Example parameterized Bayesian network for risk to human health from climate change‐compounded coastal flooding. Network overlaid with the MulRRAP framework categories. The black dash‐bordered nodes denote model control factors, that is, resilience indicators.

MULTILEVEL RISK AND RESILIENCE ASSESSMENT PARAMETERIZATION FRAMEWORK

Resilience frameworks have been developed across multiple disciplines in an effort to establish a method for deconstructing and quantifying the resilience of a system through the use of resilience indicators and scenario analysis (Frazier et al. 2014; Cutter 2016; Quinlan et al. 2016). Although there are no regulatory‐based standards for conducting a socioecological resilience assessment, research organizations have published practitioner‐focused guides and workbooks for assessing resilience in socioecological systems (Resilience Alliance 2010; Mahmoudi et al. 2018; IEMA 2020). Analogous to the problem formulation phase within risk assessment, the initial phase of the socioecological resilience assessment framework addresses the question “resilience of what to what?” and determines if general or shock‐specific resilience is being assessed (Carpenter et al. 2001). This phase describes the system of analysis (e.g., spatial boundaries, stakeholders) with a conceptual model and then identifies external system shocks (i.e., stressors; e.g., climate change) and management alternatives to alleviate the impact of system shocks (Walker et al. 2002; Resilience Alliance 2010; IEMA 2020).

Just as with risk, resilience cannot be captured in a single metric; resilience is an emergent property of the system as a whole. Measuring resilience for a quantitative resilience assessment requires the identification of proxy variables to serve as resilience indicators (Quinlan et al. 2016). The identification of resilience indicators is a relatively recent endeavor within the fields of geography (Cutter et al. 2010), policy analysis (Hinkel 2011), and environmental science (Nelson et al. 2007). The identification of risk measurement metrics and resilience indicators must be specific to the modeled complex system and the sources of risk represented in that system.

The presented MulRRAP framework (Figure 2) explicitly includes the parameterization of resilience indicators that influence how vulnerability (i.e., sensitivity to a hazard) and ultimately risk (i.e., probability of effect from stressor exposure) propagate through the modeled system throughout the 4 phases of resilience. The 4 phases of resilience are (1) preparation and planning for system stress; (2) absorption of stress; (3) recovery from impact; and (4) adaptation to new and changed system state to inform the next preparation and planning phase (NRC 2012). The inclusion of resilience indicators allows for the parameterization of a management practice to be quantified within the modeled risk (e.g., for the 4th phase of resilience—hard and soft adaptation practices; Sovacool 2011; NRC 2012).

The MulRRAP framework is an expansion of the multilevel risk assessment parameterization framework, originally developed to operationalize cybersecurity risk management via the parameterization of risk metrics for cyber, social, and sociotechnical systems of varying complexity (Oltramari et al. 2015; Henshel et al. 2016).

Structure of MulRRAP framework

The MulRRAP framework facilitates the systematic evaluation of the conceptual model parameters and the identification of representative and quantifiable risk measurement metrics and correlative resilience measurement metrics relevant to selected assessment endpoints and system stressors (see Figure 2). At the highest level, the MulRRAP framework consists of the parameterization of (1) risk assessment objectives and risk management goals; (2) risk measurement metrics; and (3) complementary resilience indicators (e.g., management actions, societal characteristics) for a given phase of resilience.

The hierarchical structure of the MulRRAP framework begins with the assessment domains (AD), which frame the assessment metrics and represent the management goal domains of the system and the high‐level organization of disciplines that have traditionally conducted singular risk assessments (Figure 2). For each managed domain the MulRRAP framework follows the subsequent levels of hierarchical deconstruction for risk measurement metrics: assessment subdomain (AS, as needed), assessment endpoint (AE), assessment metric (AM), measurement metric (MM), and data quality & uncertainty (DQU), as illustrated in Figure 2 and detailed in Table 2. The interconnections between the AD are represented in the parameters of the subsequent levels and interacting measurement metrics.

Table 2.

Organizational levels and taxonomy (with examples) within the multilevel risk and resilience parameterization framework to identify assessment metric and measurement metric

Organizational level Purpose of level Example
Assessment objective What is the objective of the risk assessment? Determine risk to human health from climate change‐compounded flood events
Management goal What is the management goal of the system being assessed? Minimize human health risk from climate change‐compounded flood events
Assessment domain (AD) What domains within the management goal are being assessed? Human health and well‐being
Environment
Society
Assessment subdomain (AS) What subdomains, if any, of that domain are being assessed? Acute adverse health impact
Physical stressor
Geography
Assessment endpoint (AE) What domain attribute(s) is/are being evaluated? Mortality
Flooding
Population distribution
Assessment metric (AM) How is the assessment endpoint quantified? Death caused by flood events
Measurement metric (MM) What is the actual metric and unit used to quantify the assessment metric? Flood mortality rates, population living within flood‐prone areas, extent and depth of flooding
Data quality & uncertainty (DU) Do the metric data exist? How reliable is the measurement method? How robust is the dataset?
Open in a new tab

Note: Figure 3B is a parallel visualization of this taxonomy.

Resilience measurement metrics are identified by following the causal pathway and connecting the primary system stressor to a previously parameterized assessment endpoint. The primary stressor is identified from the assessment objective based on the management goal and can be iteratively redefined when given new information. From the assessment endpoint, from which the assessment metrics are derived, the MulRRAP framework logically links the following levels: primary stressor (PS), secondary stressor (SS), stressor measurement metric (SMM), exposure (Ex), exposure measurement metric (EMM), effect (Ef), resilience goal (RG), resilience indicator (RI), and data quality & uncertainty (DQU), as illustrated in Figure 2 and detailed in Table 3.

Table 3.

Organizational levels and taxonomy (with examples) within the multilevel risk and resilience parameterization framework to develop the conceptual model and identify an absorption phase resilience indicator

Organizational level Purpose of level Example
Management Goal What is the management goal of the system being assessed? Minimize human health risk from climate change‐compounded flood events
Primary Stressor What is the source of the stress in the system relevant to the management goal? Global climate change
Secondary Stressor What is/are the consequence(s) of the primary stressor? ACUTE Hurricane intensity Sea level rise CHRONIC
Tertiary stressor What is/are the consequence(s) of the secondary stressor? (optional) Storm surges, extreme precipitation N/A
Stressor measurement metric How is the stressor quantified? Extent and depth of flooding
Exposure How do the assessment endpoints (i.e., entities) interact with the stressor? Floodplains, riparian zones, and low‐lying coastal regions
Exposure Measurement Metric How is/are exposure(s) of the assessment endpoints to the stressor quantified? Flood‐prone areas populated with endpoints of value (e.g., residences, businesses, vulnerable populations, ecosystem services)
Effect How are the assessment endpoints effected by stressor exposure? Ability to evacuate flooded areas; drowning mortality
Resilience goal (i.e., assessment metric) What system or endpoint characteristics enhance planning/preparing, absorption, recovery, and/or adaptation?
  • −
    System governance: governing rules and policies of the system (i.e., soft adaptation)
  • −
    Infrastructure: physical characteristics and components of the system (i.e., hard adaptation)
  • −
    Communication and process: information communication and process implementation (i.e., semisoft adaptation)
  • −
    Behavior: personal adaptive behaviors of groups and individuals (i.e., soft adaptation)
Resilience indicator (i.e., measurement metric) What control factors reduce exposure to, and the effect of, the stressor for the assessment endpoint?
  • −
    Systemic actions for greenhouse gas emission reductions
  • −
    Infrastructure changes for flood protection: wetlands, flood walls, directed drainage system, check valves
  • −
    Information communication and process implementations: alternative transportation routing, updated and sea level rise‐inclusive floodplain maps
  • −
    Personal adaptive behavior: access to vehicle, flood education and preparedness
Data quality & uncertainty (DQU) Do the metric data exist? How reliable is the measurement method? How robust is the dataset?
Open in a new tab

Note: Figure 3C is a parallel visualization of this taxonomy.

For integrated risk and resilience, the managed domains are human health and well‐being, environment, and societal context (Figure 3). The human health and well‐being domain captures adverse health impacts and quality of life. The environment domain captures biological, chemical, and physical entities and processes within the natural environment in addition to ecosystem services. The societal context domain captures the demographic, geographic, economic, policy, and regulatory aspects of the system that influence the human health and well‐being domain.

Figure 3B and Table 2 in the following case study illustrate the deconstruction (i.e., parameterization) of a risk assessment objective (e.g., risk to human health from flood events) and risk management goal (e.g., minimize risk to human health) into relevant and representative assessment and measurement metrics. Figure 3C and Table 3 illustrate how resilience indicators, such as management options, are parameterized to complement the assessment endpoint identified in Table 2.

PROOF‐OF‐CONCEPT: CASE STUDY OF REGIONAL CLIMATE CHANGE IMPACTS IN CHARLESTON HARBOR WATERSHED, SOUTH CAROLINA, USA

The greater Charleston Harbor region of South Carolina is highly susceptible to the current and projected impacts of climate change because of its low‐lying geography, a disparate socioeconomic spectrum, and invaluable ecosystem services (Carter et al. 2018; Del Giudice and Wei 2018; Moore 2018). This case study highlights the application of the MulRRAP and Bayesian network–relative risk model (BN‐RRM) as a means of quantifying climate change risk and resilience beyond the vulnerability assessment approach (Cutter et al. 2003; Chakraborty et al. 2005; Flanagan et al. 2011). The presented approach and application of an integrated risk and resilience assessment are used to assess the risk posed to a defined system by climate change and the ability of that system to absorb, recover from, adapt to, and plan for current and future climate change stressors.

Bayesian network–relative risk model

The BN‐RRM assesses the relative risk of assessment endpoints through causal pathways visualized with conceptual models and quantified with Bayesian network analysis (Ayre and Landis 2012; Landis et al. 2017). The RRM was developed to use spatial data to causally determine risk posed by multiple stressors to multiple endpoints at the regional or landscape scale (Landis and Wiegers 1997; Wiegers et al. 1998). The RRM divides the region of interest into spatially distinct subregions to geographically organize causal relationships and data. The RRM then uses a conceptual model to identify the spatially explicit links between system stressors, impacts, and endpoints (Figure 1C). Exposure–response (i.e., dose–response) data are discretized into a ranking scheme as a means of combining data of dissimilar units. The BN‐RRM uses Bayesian network analysis to probabilistically quantify risk and dynamic interactions, rather than hazard quotients and risk quotients typically used in risk assessments. There are numerous examples of the RRM and BN‐RRM being successfully used to calculate the risk of multiple stressors to multiple ecological endpoints (Herring et al. 2015; Harris et al. 2017; Johns et al. 2017; Landis et al. 2017). Figure 1 illustrates the rendering of a general RRM conceptual model (Figure 1C) into a region‐specific conceptual model (Figure 1D) and then to an endpoint‐specific conceptual model (Figure 1E). Once parameterized (Figures 3 and 4), the endpoint‐specific conceptual model serves as the structural foundation for the Bayesian network (Figure 5).

Bayesian networks (BN) are acyclic graphical models that quantitatively visualize direct and indirect relationships via conditional probability (Woodberry et al. 2004), such as those linking climate change stressors and impacts on endpoints of value. Moe et al. (this issue) and Kaikkonen et al. (this issue) provide detailed descriptions of BNs and their properties. Additionally, Landis (this issue) details the history of the RRM and integration of BNs into the RRM and the resulting BN‐RRM methodology, which produces a risk score (in relative risk units) for each subregion of the system.

Application of BN‐RRM and MulRRAP

As a proof‐of‐concept, the MulRRAP framework and the BN‐RRM were used sequentially to parameterize and quantify risk to human health resulting from climate change‐compounded flooding in 3 representative regions of the Charleston Harbor Watershed using historical and projected climate data, sea level rise projections, and hurricane storm surge model output (NOAA 2017, 2018, 2019). The 3 representative regions are Region 1 with a coastline in a high‐risk flood zone, for example, the historic and heavily developed peninsula; Region 2 with coastline in a medium‐risk flood zone, for example, residential development on James Island; and Region 3 with no coastline and a low‐risk flood zone, for example, the industrial area around Cooper River (Figure S1).

Parameterization

The MulRRAP framework breaks down the primary stressor of climate change into secondary (e.g., sea level rise, extreme precipitation) and tertiary (e.g., nuisance flooding, habitat alteration) stressors, a process exemplified in Table 2. Tertiary climate change stressors are consequences of the secondary stressors. For this case study, the secondary stressors are regional consequences of global climate change. The multistressor model, illustrated in Figure 1E, is an adaptation of the BN‐RRM and incorporates resilience parameters as control factors that modulate how risk is carried through the model (e.g., physical flood protection). These control factors modify the degree of exposure to stressors and intensity of effects. The control factors represent resilience characteristics (i.e., plan and/or prepare, absorb, recover, and adapt), management practices, and/or personal actions that, when implemented, reduce risk to the assessed endpoints by reducing exposure to a stressor or the impact of an effect or both. From left to right in Figure 4, the 1st control factor represents the system governance needed to reduce the source of stress, the 2nd represents local government‐level hard adaptations (i.e., physical entities, e.g., infrastructure) that control the degree of exposure experience by a location or habitat, the 3rd represents individual‐level hard or soft adaptions (i.e., behaviors) that control the degree of impact of effects, and the 4th and last control factor represents the forward‐thinking hard and soft adaptations needed at the local level to further reduce risk in the future.

The risk measurement metrics of Charleston residents living within floodplains, the extent and depth of flooding, and the mortality rates for drowning in flood waters were identified using the MulRRAP framework for the evaluation of the assessment endpoint “death due to flood events” (see Table 2 and Figure 3B). The resilience goals (i.e., assessment metrics) related to climate change‐compounded flooding include CO2 emissions regulations, storm surge protection (e.g., oyster reefs as a measurement metric), physical flood protection (e.g., wetlands, flood walls, dam maintenance, directed drainage system as measurement metrics), flood preparedness, and updated floodplain maps that incorporate projected sea level rise (see Table 2 and Figure 3C). Data quality and uncertainty for each measurement metric were evaluated for existence, reliability, and robustness. For example, flood drowning mortality rates were not available for site‐specific flooding events, so rates from the primary literature were used instead (Ashley and Ashley 2008; Doocy et al. 2013). The distributions within the respective conditional probability tables were widened to incorporate the estimated uncertainty because the mortality rates were not site specific (see Netica files [.neta] in the Supplemental Data).

The regional governments of the Charleston Harbor Watershed cannot directly affect international governance on the causes of climate change. For this proof‐of‐concept, only the 2nd and 3rd control factors (e.g., exposure and impact control) are integrated into the parameterized conceptual model as resilience indicators because the assessment is quantifying the hard and soft adaptations at the local level. The identified resilience indicators (i.e., measurement metrics) are incorporated into the conceptual model as a means to reduce exposure to stressors (physical flood and storm surge protections) and the intensity of stressor effects (flood preparedness and access to a personal vehicle). The primary stressor organization level corresponds to the “Source” category of the RRM conceptual model, secondary and tertiary stressor to “Stressor,” exposure to “Habitat/Location,” and effect to “Effect” (Figures 4 and 1, respectively).

Analysis

Figure 5 is a Bayesian network representation from Netica of the endpoint‐specific conceptual model (Figure 1E) parameterized using the MulRRAP framework (Norsys Software Corporation 2020). These Bayesian network parameters (i.e., nodes) and relationships (i.e., edges) create a causal pathway between secondary climate change stressors and selected human health risk endpoints. Control factors are the dash‐bordered nodes (Figure 5) in ‘Exposure and Effect’ and represent adaptive management practices as resilience indicators (e.g., percent of coastline with physical storm surge protection; percent of population with personal vehicle for evacuation).

The case study input node risk parameter states are discretized into the following relative risk states and corresponding relative numeric values: zero (0), low (2), medium (4), and high (6). The higher the numerical risk state the more adverse the state; so, within each risk node, risk increases when read top (low) to bottom (high). Resilience parameter states are “no,” that is, lack of resilience indicator and unfiltered propagation of risk, and “yes,” that is, presence of resilience indicator and reduction of propagated risk. The current representation of resilience‐only models is the ability of the system to “absorb” risk (i.e., the 2nd resilience phase in response to adverse effects). The model currently does not quantitatively reflect the other attributes of resilience (i.e., plan and/or prepare, recover, and adapt), but these are captured in the fourth control factor in the conceptual model (Figure 1). The conditional probability tables (Tables S1 to S24) were populated by establishing the probability of the extreme states. When all the input nodes were zero or high risk, the intermediary or child node would be 100% zero or 100% high risk, respectively. The remaining cells of the conditional probability table were conservatively interpolated using observational and projected data (Marcot et al. 2006). The output node of the analysis produces a continuous endpoint relative risk score between 0 and 6. Detailed explanations of the parameters, discretization schemes, data sources, and Netica files (.neta) are available in the Supplemental Data.

Results

The expected value for risk to human health was calculated using projected and estimated input data producing the following relative risk scores: Region 1 (e.g., peninsula), 2.42; Region 2 (e.g., coast), 3.54; and Region 3 (e.g., no coastline), 1.56 (Table 4). The higher the relative risk score, the more at‐risk the region. When all resilience parameters were set to 100% “no,” that is, low resilience, the risk to all 3 regions increased compared to the expected values: Region 1, 4.63; Region 2, 4.55; and Region 3, 1.86. Region 1 was most sensitive to the decreased resilience with a risk increase of 91%. Conversely, when all resilience parameters were set to 100% “yes,” that is, high resilience, the risk to all 3 regions decreased compared to the expected values: Region 1, 1.70; Region 2, 1.87; and Region 3, 1.20. Region 2, with more populated floodplains than the other regions, was most sensitive to the increased resilience with a risk decrease of 47%. Region 3, with fewer floodplains, was the least influenced by resilience scenarios, varying approximately 20% in either direction.

Table 4.

Relative risk output table for changes in scenario risk scores caused by varying levels of resilience as affected by control factors

Scenario Risk region

1

(e.g., peninsula)

2

(e.g., coast)

3

(e.g., no coastline)
Expected values Relative risk score 2.42 3.54 1.56
Low resilience Relative risk score 4.63 4.55 1.86
Difference 2.21 1.01 0.30
91% 29% 19%
High resilience Relative risk score 1.70 1.87 1.20
Difference −0.70 −1.67 −0.36
−29% −47% −23%
Open in a new tab

Note: The “Relative Risk Score” is the output of the Bayesian network analysis within the Relative Risk Model. The “Difference” is the percent change of the “Relative Risk Score” when modeled with “Low Resilience” and “High Resilience” compared to the “Expected Values” state. Bolded numbers indicate largest magnitude of change. The higher the “Relative Risk Score,” the more at‐risk the region. A negative difference between relative risk scores results from increased resilience.

The current configuration of the Bayesian network embodies dynamic changes (e.g., “what if” scenarios) to risk and resilience through the automatic recalculation of relative risk whenever evidence (e.g., “Hurricane Category” at 100% medium risk) is set for a scenario occurring during a specific time point. The generation of a “what if” scenario allows managers to identify worst‐ and best‐case scenarios that can inform current and future management decisions.

DISCUSSION

This paper presents the MulRRAP framework applied to a subsystem of a complex socioecological system (e.g., Charleston Harbor Watershed) using the Bayesian Network–Relative Risk Model (BN‐RRM) as the integrating quantitative method (Ayre and Landis 2012; Landis et al. 2017). The MulRRAP approach unfolds the problem formulation step, allowing for systematic analysis of a complex system within the risk and resilience management framework. The application of the MulRRAP framework facilitates the hierarchical organization and characterization of system components within a complex system across ecological, spatial, and temporal scales, thus enabling the identification of system entities and relationships from which potential measurement metrics and indicators are derived from management‐focused assessment goals (Angeler et al. 2016).

The MulRRAP framework is a logical extension of early biomarker frameworks (Burger 2006), parallels the Adverse Outcome Pathway (AOP) framework (Ankley et al. 2010), systematizes the parameterization process for both risk and resilience, and integrates a quantitative approach to resilience assessment. However, models resulting from any assessment framework, even the MulRRAP framework, are abstractions of complex systems and their properties (e.g., socioecological resilience indicators). Socioecological systems are open systems despite the spatial and temporal boundaries defined during the assessment process. Therefore, the parameterization and assessment of any open system will always be incomplete. Two of the strengths of Bayesian network analysis are the ability to incorporate uncertainty (e.g., probability distribution) and to account for a lack of data, typically addressed by using a uniform distribution. Additionally, assessment parameter values can be readily updated with new data for rapid recalculation.

The MulRRAP framework also explicitly identifies where data quality and data uncertainty should be quantified within the parameterization process and further characterizes data uncertainty before quantification. Parameterization is critical to reach the point of measurement and quantification. Ideally, data for each parameterized metric and indicator should (1) exist; (2) have reliable measurement methods; and (3) have robust datasets. Although the existence of data is an obvious criterion (that is not always met), an underappreciated aspect of data uncertainty, especially in the social component of socioecological systems, is the logistical difficulty in obtaining data theoretically suitable as a metric or indicator. Therefore, 1 key metric for data uncertainty must include an attribute for whether the data exists, and not just a qualitative or quantitative estimate of the size of the uncertainty relative to the values used (i.e., data robustness) or the measurement methodology (i.e., data reliability).

The overall approach we have proposed enables the quantification of a complex metasystem, such as a socioecological system, to be systematically analyzed using the equivalent of a systems engineering approach. The systems engineering perspective evaluates all likely aspects of a system, which are then integrated to create a system that is greater than the sum of its components (Holstein and Bode 2015). For example, increased flooding events and adverse health impacts caused by water contamination may well be synergistic. Thus, the socioecological watershed system should be examined as a whole when characterizing and modeling such a relationship. Additionally, the impact of sea level rise on storm surge flooding (e.g., extent, intensity, and duration) can be predicted using hydrodynamic models (Castrucci and Tahvildari 2018). However, the storm surge models produced by the National Hurricane Center do not account for sea level rise, thus adding calculation uncertainty by modeling the predictions for each phenomenon separately. Extrapolating from modeled predictions is only a start to the analysis. Future modeling scenarios must include synergistic impacts across critical infrastructure. For example, although the direct impacts of Hurricane Sandy affected the eastern coastal states, indirect infrastructure impacts reached as far west as Ohio and Michigan (NERC 2014).

A benefit of the BN‐RRM is that inherent node and internode mechanisms and interactions can be integrated in the same model, even when internode interaction mechanisms require very different assumptions to be built into the node dynamics models. The current discretization scheme used within the BN‐RRM approach simplifies a continuous probability distribution into a 4‐bin approach (i.e., zero, low, medium, high). However, the scheme can be modified for tipping point analysis, such as “points of no return” when too many critical infrastructure systems fail and the dependent socioecological system follows (Costanza 1999; Fortna n.d.).

As we strive to assess the true complexity of complex systems, it becomes more important to build into our models the interactions and dynamics of the system parameters (i.e., nodes). The MulRRAP process makes complex and dynamic systems more approachable and feeds easily into Bayesian network analysis. Characterization of node interactions is understudied but critical (Kelble et al. 2013; Landis et al. 2013), especially when evaluating multistressor phenomena such as those associated with climate change. Resilience is a dynamic, multiphase process (e.g., preparing and hardening, resisting and absorbing stressors, recovery from induced damages, and adaptation for future protections) that influences the node states and interactions while amplifying the complexity of analysis (Holling 2001; Linkov and Trump 2019). Future dynamic complex risk and resilience analysis must also include the ability to quantify the influence of both feedforward (as per Bayesian network analysis) and feedback processes inherent to large‐scale, highly complex, evolving metasystems, such as the socioecological‐physical systems as found in any human‐populated watershed.

Disclaimer

The authors declare no conflict of interest. This research did not receive any specific grant from funding agencies in the public, commercial, or not‐for‐profit sectors.

SUPPLEMENTAL DATA

The Supplemental Data files contain: 1) supplemental image of the case study site, Charleston Harbor Watershed, SC; 2) the discretization scheme for the Bayesian network; 3) the conditional probability tables for Region 3 of the case study site; and 4) Bayesian network analysis files (Netica; .neta).

Supporting information

This article contain online‐only Supplemental Data.

Supporting information.

Click here for additional data file. (2.5MB, docx)

Supporting information.

Click here for additional data file. (3.5KB, neta)

Supporting information.

Click here for additional data file. (3.5KB, neta)

Supporting information.

Click here for additional data file. (3.4KB, neta)

Acknowledgment

We thank the reviewers and editors for their valuable and attentive feedback. Responding to their comments improved the paper substantially. The authors thank the special series guest editors for their prereview feedback.

This article is part of the special series “Applications of Bayesian Networks for Environmental Risk Assessment and Management” and was generated from a session on the use of Bayesian networks (BNs) in environmental modeling and assessment in 1 of 3 recent conferences: SETAC North America 2018 (Sacramento, CA, USA), SETAC Europe 2019 (Helsinki, Finland), and European Geosciences Union 2019 (Vienna, Austria). The 3 sessions aimed at showing the state‐of‐the art and new directions in the use of BN models in environmental assessment, focusing on ecotoxicology and water quality modeling. This series aims at reflecting the broad applicability of BN methodology in environmental assessment across a range of ecosystem types and scales, and discusses the relevance for environmental management.

Data Availability Statement

All data sources are public government dataset available via the links provided in the References and Supplemental Data. Bayesian network analysis files are available by contacting corresponding author Mariana Goodall Cains (cainsmg@gmail.com).

REFERENCES

  1. Angeler DG, Allen CR, Garmestani AS, Gunderson LH, Linkov I. 2016. Panarchy use in environmental science for risk and resilience planning. Environ Syst Decis 36(3):225–228. [Google Scholar]
  2. Ankley GT, Bennett RS, Erickson RJ, Hoff DJ, Hornung MW, Johnson RD, Mount DR, Nichols JW, Russom CL, Schmieder PK et al. 2010. Adverse outcome pathways: A conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem 29(3):730–741. [DOI] [PubMed] [Google Scholar]
  3. Ashley ST, Ashley WS. 2008. Flood fatalities in the United States. J Appl Meteorol Clim 47(3):805–818. [Google Scholar]
  4. Aven T. 2016. Risk assessment and risk management: Review of recent advances on their foundation. Eur J Oper Res 253(1):1–3. [Google Scholar]
  5. Ayre KK, Landis WG. 2012. A Bayesian approach to landscape ecological risk assessment applied to the Upper Grande Ronde watershed, Oregon. Hum Ecol Risk Assess 18(5):946–970. [Google Scholar]
  6. Bernstein PL. 1996. Against the gods: The remarkable story of risk. New York (NY): Wiley; p 1269–1275. [Google Scholar]
  7. Boyd JR. 1987. A discourse on winning and losing. In: Hammond GT, editor. 2018. Maxwell Air Force Base, AL: Air University Library Document No. M‐U 43947. [accessed 2020 Apr 30]. https://www.airuniversity.af.edu/Portals/10/AUPress/Books/B_0151_Boyd_Discourse_Winning_Losing.PDF
  8. Burger J. 2006. Bioindicators: Types, development, and use in ecological assessment and research. Environ Bioindic 1:22–39. [Google Scholar]
  9. Cains M, Henshel D. 2019. Community as an equal partner for region‐based climate change vulnerability, risk, and resilience assessments. Curr Opin Env Sust 39:24–30. [Google Scholar]
  10. Carpenter S, Walker B, Anderies JM, Abel N. 2001. From metaphor to measurement: Resilience of what to what? Ecosystems 4(8):765–781. [Google Scholar]
  11. Carter L, Terando A, Dow K, Hiers K, Kunkel KE, Lascurain A, Marcy D, Osland M, Schramm P. 2018. Southeast In: Reidmiller DR, Avery CW, Easterling DR, Kunkel KE, Lewis KLM, Maycock TK, Stewart BC, editors. Impacts, risks, and adaptation in the United States: Fourth national climate assessment, Vol II. Washington (DC): US Global Change Research Program; p 743–808. 10.7930/NCA4.2018.CH19 [DOI] [Google Scholar]
  12. Castrucci L, Tahvildari N. 2018. Modeling the impacts of sea level rise on storm surge inundation in flood‐prone urban areas of Hampton Roads, Virginia. Mar Technol Soc J 52(2):92–105. [Google Scholar]
  13. Chakraborty J, Tobin GA, Montz BE. 2005. Population evacuation: Assessing spatial variability in geophysical risk and social vulnerability to natural hazards. Nat Hazards Rev 6(1):23–33. [Google Scholar]
  14. Costanza R. 1999. Four visions of the century ahead: Will it be Star Trek, Ecotopia, Big Government or Mad Max? Futurist 33(2):23. [Google Scholar]
  15. Cutter SL. 2016. The landscape of disaster resilience indicators in the USA. Nat Hazards 80(2):741–758. [Google Scholar]
  16. Cutter SL, Boruff BJ, Shirley WL. 2003. Social vulnerability to environmental hazards. Soc Sci Q 84(2):242–261. [Google Scholar]
  17. Cutter SL, Burton CG, Emrich CT. 2010. Disaster resilience indicators for benchmarking baseline conditions. J Homel Secur Emerg Manag 7(1):1–24. [Google Scholar]
  18. Dai FC, Lee CF, Ngai YY. 2002. Landslide risk assessment and management: An overview. Eng Geol 64(1):65–87. [Google Scholar]
  19. Del Giudice V, Wei L. 2018. These are the US cities with the fastest‐growing wealth gaps. 2018 April 19; Bloomberg. [accessed 2019 Dec 28]. https://bloomberg.com
  20. Doocy S, Daniels A, Murray S, Kirsch TD. 2013. The human impact of floods: A historical review of events 1980–2009 and systematic literature review. PLoS Curr Disast 5 10.1371/currents.dis.f4deb457904936b07c09daa98ee8171a [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. [ECC] European Centre for Nature Conservation . 2017. Operationalisation of natural capital and ecosystem services. Tilburg (NL). [Google Scholar]
  22. Flanagan BE, Gregory EW, Hallisey EJ, Heitgerd JL, Lewis B. 2011. A social vulnerability index for disaster management. J Homel Secur Emerg Manag 8(1):1–22. [Google Scholar]
  23. Fortna . n.d. Tipping point analysis: A strategic tool in uncertain times. [accessed 2020 Feb 6]. http://www.fortna.com/wp-content/uploads/Article_Tipping-Point-Analysis_FINAL_EN.pdf
  24. Frazier TG, Thompson CM, Dezzani RJ. 2014. A framework for the development of the SERV model: A spatially explicit resilience‐vulnerability model. Appl Geogr 51:158–172. [Google Scholar]
  25. Harris MJ, Stinson J, Landis WG. 2017. A Bayesian approach to integrated ecological and human health risk assessment for the South River, Virginia mercury‐contaminated site. Risk Anal 37(7):1341–1357. [DOI] [PubMed] [Google Scholar]
  26. Henshel D, Alexeev A, Cains M, Rowe J, Cam H, Hoffman B, Neamtiu I. 2016. Modeling cybersecurity risks: Proof of concept of a holistic approach for integrated risk quantification. 2016 IEEE HST. p 1–5.
  27. Herring CE, Stinson J, Landis WG. 2015. Evaluating nonindigenous species management in a Bayesian networks derived relative risk framework for Padilla Bay, WA, USA. Integr Environ Assess Manag 11(4):640–652. [DOI] [PubMed] [Google Scholar]
  28. Hinkel J. 2011. Indicators of vulnerability and adaptive capacity: Towards a clarification of the science–policy interface. Glob Environ Change 21(1):198–208. [Google Scholar]
  29. Holling CS. 2001. Understanding the complexity of economic, ecological, and social systems. Ecosystems 4(5):390–405. [Google Scholar]
  30. Holstein WK, Bode HW. 2015. Systems engineering. Encyclopaedia Britannica. [accessed 2020 Feb 7]. https://www.britannica.com/topic/systems-engineering
  31. [IEMA] Institute of Environmental Management and Assessment . 2020. Environmental impact assessment guide to climate change resilience & adaptation. [accessed 2020 Aug 15]. https://www.iema.net/assets/newbuild/Policy%202020/IEMA%20EIA%20Climate%20Change%20Resilience%20June%202020.pdf
  32. [ISO] International Organization for Standardization . 2018. Risk management—Guidelines: 6. Process. ISO 31000:2018(en). [accessed 2019 Dec 19]. https://www.iso.org/obp/ui/#iso:std:iso:31000:ed-2:v1:en
  33. Johns AF, Graham SE, Harris MJ, Markiewicz AJ, Stinson JM, Landis WG. 2017. Using the Bayesian network relative risk model risk assessment process to evaluate management alternatives for the South River and Upper Shenandoah River, Virginia. Integr Environ Assess Manag 13(1):100–114. [DOI] [PubMed] [Google Scholar]
  34. Kaikkonen L, Parviainen T, Rahikainen M, Uusitalo L, Lehikoinen A. 2021. Bayesian networks in environmental risk assessment: A review. Integr Environ Assess Manag 17(1):62–78. 10.1002/ieam.4332 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kelble CR, Loomis DK, Lovelace S, Nuttle WK, Ortner PB, Fletcher P, Cook GS, Lorenz JJ, Boyer JN. 2013. The EBM‐DPSER conceptual model: Integrating ecosystem services into the DPSIR framework. PLoS One 8(8):e70766 10.1371/journal.pone.0070766 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Landis WG. 2021. The origin, development, application, lessons learned, and future regarding the Bayesian network relative risk model for ecological risk assessment. Integr Environ Assess Manag 17(1):79–94. 10.1002/ieam.4351 [DOI] [PubMed] [Google Scholar]
  37. Landis WG, Ayre KK, Johns AF, Summers HM, Stinson J, Harris MJ, Herring CE, Markiewicz AJ. 2017. The multiple stressor ecological risk assessment for the mercury‐contaminated South River and upper Shenandoah River using the Bayesian network–relative risk model. Integr Environ Assess Manag 13(1):85–99. [DOI] [PubMed] [Google Scholar]
  38. Landis WG, Durda JL, Brooks ML, Chapman PM, Menzie CA, Stahl RG Jr, Stauber JL. 2013. Ecological risk assessment in the context of global climate change. Environ Toxicol Chem 32(1):79–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Landis WG, Wiegers J. 1997. Design considerations and a suggested approach for regional and comparative ecological risk assessment. Hum Ecol Risk Assess 3:287–297. [Google Scholar]
  40. Linkov I, Trump BD. 2019. The science and practice of resilience. Springer International. 10.1007/978-3-030-04565-4 [DOI]
  41. Mahmoudi H, Sayahnia R, Esmaeilzadeh H, Azadi H. 2018. Integrating resilience assessment in environmental impact assessment. Integr Environ Assess Manag 14(5):567–570. [DOI] [PubMed] [Google Scholar]
  42. Marcot BG, Steventon JD, Sutherland GD, McCann RK. 2006. Guidelines for developing and updating Bayesian belief networks applied to ecological modeling and conservation. Can J For Res 36(12):3063–3074. [Google Scholar]
  43. Moe SJ, Wolf R, Xie L, Landis WG, Kotamäki N, Tollefsen KE. 2021. Quantification of an adverse outcome pathway network by Bayesian regression and Bayesian network modeling. Integr Environ Assess Manag 17(1):147–164. 10.1002/ieam.4348 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Moore T. 2018. Charleston's income inequality gap is growing nearly as fast as Seattle's. The Post and Courier 2018. April 28. [accessed 2019 Dec 28]. http://www.postandcourier.com
  45. Nelson DR, Adger WN, Brown K. 2007. Adaptation to environmental change: Contributions of a resilience framework. Annu Rev Environ Resour 32(1):395–419. [Google Scholar]
  46. [NERC] North American Electric Reliability Corporation . 2014. Hurricane Sandy event analysis report. Atlanta (GA). [accessed 2020 Feb 28]. https://www.nerc.com/pa/rrm/ea/Oct2012HurricanSandyEvntAnlyssRprtDL/Hurricane_Sandy_EAR_20140312_Final.pdf
  47. [NOAA] National Oceanic and Atmospheric Administration . 2017. South Carolina. State climate summaries. National Centers for Environmental Information. [accessed 2018 Jan 15]. https://statesummaries.ncics.org/sc
  48. [NOAA] National Oceanic and Atmospheric Administration . 2018. Charleston, SC. Local climate data and plots. [accessed 2018 Jan 15]. https://www.weather.gov/chs/climate
  49. [NOAA] National Oceanic and Atmospheric Administration . 2019. Sea level rise data download: South Carolina. [accessed 2019 Dec 19]. https://coast.noaa.gov/slrdata/
  50. Norsys Software Corporation . 2020. Netica. Vancouver (BC). Copyright 2020. [accessed 2020 Jul 31]. https://norsys.com/netica.html
  51. [NRC] National Research Council . 1983. Risk assessment in the federal government: Managing the process. Washington (DC): National Academies Press; http://www.nap.edu/catalog/366.html [PubMed] [Google Scholar]
  52. [NRC] National Research Council . 2009. Science and decisions: Advancing risk assessment. Washington (DC): National Academies Press. 10.17226/12209 [DOI] [PubMed]
  53. [NRC] National Research Council . 2012. Disaster resilience: A national imperative. Washington (DC): National Academies Press. 10.17226/13457 [DOI]
  54. Oltramari A, Henshel DS, Cains M, Hoffman B. 2015. Towards a human factors ontology for cyber security. STIDS 2015:26–33. [Google Scholar]
  55. Quinlan AE, Berbés‐Blázquez M, Haider LJ, Peterson GD. 2016. Measuring and assessing resilience: Broadening understanding through multiple disciplinary perspectives. J Appl Ecol 53(3):677–687. https://www.resalliance.org/files/ResilienceAssessmentV2_2.pdf [Google Scholar]
  56. Resilience Alliance . 2010. Assessing resilience in social‐ecological systems: Workbook for practitioners. Version 2.0. [accessed 2020 Aug 15]. https://www.resalliance.org/files/ResilienceAssessmentV2_2.pdf
  57. Richkus WA, Menzie CA. 2013. Application of an ecological risk assessment for evaluation of alternatives considered for restoration of oysters in Chesapeake Bay: Background and approach. Hum Ecol Risk Assess 19(5):1172–1186. [Google Scholar]
  58. Schröder B, Seppelt R. 2006. Analysis of pattern–process interactions based on landscape models—Overview, general concepts, and methodological issues. Ecol Model 199(4):505–516. [Google Scholar]
  59. Sovacool BK. 2011. Hard and soft paths for climate change adaptation. Clim Policy 11(4):1177–1183. [Google Scholar]
  60. Suter GW II. 2007. Ecological risk assessment. Boca Raton (FL): CRC Press; 680 p. 10.1201/9781420012569 [DOI] [Google Scholar]
  61. Suter GW II, Vermeire T, Munns WR, Sekizawa J. 2003. Framework for the integration of health and ecological risk assessment. Hum Ecol Risk Assess 9(1):281–301. [Google Scholar]
  62. [USEPA] United States Environmental Protection Agency . 1989. Risk assessment guidance for Superfund Part A. Washington (DC). EPA/540/1‐89/002.
  63. [USEPA] United States Environmental Protection Agency . 1998. Guidelines for ecological risk assessment. Washington (DC). EPA/630/R‐95/002F.
  64. [USEPA] United States Environmental Protection Agency . 2003. Framework for cumulative risk assessment. Washington (DC). EPA/630/P‐02/001F.
  65. [USEPA] United States Environmental Protection Agency . 2014a. Framework for human health risk assessment to inform decision making. Washington (DC). EPA/100/R‐14/001.
  66. [USEPA] United States Environmental Protection Agency . 2014b. An assessment of potential mining impacts on salmon ecosystems of Bristol Bay, Alaska. Seattle (WA). EPA 910‐R‐14‐001A.
  67. [USEPA] United States Environmental Protection Agency . 2019. Guidelines for human exposure assessment. Washington (DC): Risk Assessment Forum. EPA/100/B‐19/001.
  68. van den Brink P, Bo Choung C, Landis W, Mayer‐Pinto M, Pettigrove V, Scanes P, Smith R, Stauber J. 2016. New approaches to the ecological risk assessment of multiple stressors. Mar Freshw Res 67:429–439. [Google Scholar]
  69. Walker B, Carpenter S, Anderies J, Abel N, Cumming G, Janssen M, Lebel L, Norberg J, Peterson GD, Pritchard R. 2002. Resilience management in social‐ecological systems: A working hypothesis for a participatory approach. Conserv Ecol 6(1):14 http://www.consecol.org/vol6/iss1/art14 [Google Scholar]
  70. Wiegers JK, Feder HM, Mortensen LS, Shaw DG, Wilson VJ, Landis WG. 1998. A regional multiple‐stressor rank‐based ecological risk assessment for the fjord of Port Valdez, Alaska. Hum Ecol Risk Assess 4:1125–1173. [Google Scholar]
  71. Wikipedia . 2020. Operationalization. [accessed 2020 Aug 15]. https://en.wikipedia.org/w/index.php?title=Operationalization&oldid=960327058
  72. Woodberry O, Nicholson AE, Korb KB, Pollino CA. 2004. Parameterising Bayesian networks In: Webb GI, Yu X, editors. AI 2004: Advances in artificial intelligence. Lecture Notes in Computer Science, Vol 3339 Berlin (DE): Springer; p 1101–1107. 10.1007/978-3-540-30549-1_108 [DOI] [Google Scholar]
  73. Zavadskas EK, Zenonas T, Tamošaitiene J. 2010. Risk assessment of construction projects. J Civ Eng Manag 16(1):33–46. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

This article contain online‐only Supplemental Data.

Supporting information.

Click here for additional data file. (2.5MB, docx)

Supporting information.

Click here for additional data file. (3.5KB, neta)

Supporting information.

Click here for additional data file. (3.5KB, neta)

Supporting information.

Click here for additional data file. (3.4KB, neta)

Data Availability Statement

All data sources are public government dataset available via the links provided in the References and Supplemental Data. Bayesian network analysis files are available by contacting corresponding author Mariana Goodall Cains (cainsmg@gmail.com).

Articles from Integrated Environmental Assessment and Management are provided here courtesy of Wiley

RESOURCES