Skip to main content
EPA Author Manuscripts logoLink to EPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 27.
Published in final edited form as: Risk Anal. 2017 Nov 23;38(6):1183–1201. doi: 10.1111/risa.12933

Characterizing Risk for Cumulative Risk Assessments

Margaret M MacDonell 1, Richard C Hertzberg 2, Glenn E Rice 3, J Michael Wright 3, Linda K Teuschler 4
PMCID: PMC8315329  NIHMSID: NIHMS1504000  PMID: 29168988

Abstract

In assessing environmental health risks, the risk characterization step synthesizes information gathered in evaluating exposures to stressors together with dose-response relationships, characteristics of the exposed population, and external environmental conditions. This manuscript summarizes key steps of a cumulative risk assessment (CRA) followed by a discussion of considerations for characterizing cumulative risks. Cumulative risk characterizations differ considerably from single chemical- or single source-based risk characterization. CRAs typically focus on a specific population instead of a pollutant or pollutant source and should include an evaluation of all relevant sources contributing to the exposures in the population and other factors that influence dose-response relationships. Second, CRAs may include influential environmental and population-specific conditions, involving multiple chemical and nonchemical stressors. Third, a CRA could examine multiple health effects, reflecting joint toxicity and the potential for toxicological interactions. Fourth, the complexities often necessitate simplifying methods, including judgment-based and semiquantitative indices that collapse disparate data into numerical scores. Fifth, because of the higher dimensionality and potentially large number of interactions, information needed to quantify risk is typically incomplete, necessitating an uncertainty analysis. Three approaches that could be used for characterizing risks in a CRA are presented: the multiroute hazard index, stressor grouping by exposure and toxicity, and indices for screening multiple factors and conditions. Other key roles of the risk characterization in CRAs are also described, mainly the translational aspect of including a characterization summary for lay readers (in addition to the technical analysis), and placing the results in the context of the likely risk-based decisions.

Keywords: Risk Characterization, Cumulative Risk Assessment, Multiple Stressors

1. INTRODUCTION

Environmental health risk assessments have usually been comprised of four sequential and often independent parts in accordance with the National Research Council (NRC) paradigm: hazard identification, exposure assessment, dose-response assessment, and risk characterization.(1) Cumulative risk assessment (CRA) is the analysis, characterization, and possible quantification of the combined risks from multiple stressors via multiple exposure routes (the ways individuals physically contact the chemicals, e.g., oral, inhalation, or dermal exposures) in a defined community or specific population.(2) While some CRAs evaluate the risks posed by a single type of stressor through multiple exposure routes, such as specific pesticide classes (e.g., organophosphates, N-methyl carbamates) or dioxin-like compounds, other CRAs consider the hazards or risks associated with different types of stressors (e.g., co-exposures to hazardous chemicals and a psychosocial stress such as high rates of crime near homes or workplaces). CRAs can be conducted at a community scale or a regional scale, extending to national and international scales. Some CRAs are conducted to discriminate among risk management alternatives. We contrast CRAs with single chemical- or single source-based assessments termed “conventional” assessments in this manuscript. A CRA has a different structure than the earlier conventional chemical risk assessments, consisting of three steps that encompass the original four steps noted above: (1) planning, scoping, and problem formulation; (2) analysis; and (3) risk characterization.(2) During the analysis phase, the evaluations generally follow the NRC paradigm, except that the hazard identification and the exposure and dose-response assessments are evaluated together in an interdependent manner rather than separately. This approach increases the relevance of the assembled information and reduces the extrapolations required to apply that information to the risk scenario(s). A tiered approach to the analysis that sequentially considers additional chemicals and relevant non-chemical stressors also has been advocated to enhance analytic efficiency.(3)

The scope of the environmental risk scenario also differs between CRAs and conventional risk assessments. For the latter, most focus on exposures to a single agent or a group of related agents over a fixed time period, with emphasis on a single “critical” effect, (i.e., the toxicity occurring at the lowest dose). In comparison, CRA scenarios are typically broader, involving multiple risks associated with multiple stressors, where all health effects of concern are considered together and where interactions of exposure and/or toxicity are estimated for specific effects.

“Stressors” often are defined to include any physical, chemical, or biological agent capable of inducing an adverse response.(2) For example, whereas conventional assessments might present health risks estimated for chronic exposures to a single chemical in drinking water, CRAs might evaluate risks from chronic exposure to pathogens and chemicals in drinking water and food, as well as intermittent exposure to physical stressors such as heat, utilizing (when available) information about interactions that can affect exposure and/or toxicity. Table I provides examples of nonchemical stressors and associated health effects. Nonchemical stressors also can be more broadly defined to include a condition or influential factor, the lack of an essential entity (such as shelter), or a genetic predisposition.(2) Examples include: behavioral or lifestyle factors (e.g., poverty, diet, smoking behavior, and physical activity level), intrinsic biological factors (e.g., race/ethnicity, sex, and genetics), psychological stress, socioeconomic status (e.g., educational attainment and household income), or community characteristics (e.g., exposure to violence or other crimes). These can influence an individual’s exposures (i.e., exposure modifying factors) and modify the individual’s responses following stressor exposures (i.e., effect measure modifiers) affecting the magnitudes of the risks being evaluated. Some CRAs also could consider factors that lessen the impact of the environmental stressors of concern (e.g., sufficient nutrition can reduce effective oral absorption of some metals, thus reducing the effective exposure);(4, 5) these factors are sometimes termed buffers.

Table I.

Examples of Nonchemical Stressors and Associated Health Outcomes

Stressor Types Example Stressors Example Health Outcomes

Physical: Thermal Cold temperature Hypothermia, lethargy-confusion, frostbite
Heat Hyperthermia, heat stroke, heat exhaustion, respiratory disorders
Acoustic Vibration Tissue injury, Raynaud’s syndrome
Sound Hearing loss (synergistic with solvents)
Noise Poor sleep, stress response, allostatic load
Radiation Ionizing radiation Cancer; developmental effects

Biological: Pollen, mold, pet or pest allergen Respiratory irritation, asthma, stress response, allostatic load
Bacteria, viruses, parasites Infectious and contagious diseases, weakened immune system (note this can also result from other stressors, such as psychological stress, chemical and physical exposures)

Psychosocial: Under- and unemployment; home violence, instability; racism/discrimination Poor sleep, stress response, allostatic load
Limited access to or use of healthy food resources Poor nutrition, obesity, cardiovascular disease, diabetes, developmental effects
Limited access to or use of health care resources Stress response, allostatic load, disease progression, developmental effects
Crime, including violent crime Stress response, allostatic load

While the risk characterization step of conventional assessments synthesizes the analyses of environmental conditions and estimated risks for the population(s) of interest, in a CRA the translational role of risk characterization is emphasized. Historically, risk characterization has been described as “estimating the incidence of a health effect”(1) or evaluating the consequences of exceeding a “safe exposure level” for the most critical effect,(6,7) including the discussion of uncertainties in these estimates. In addition to these elements, cumulative risk characterizations can also address the context of the decision the CRA is informing. For example, NRC(7) (see also Sexton(8)) suggests that CRAs be oriented to evaluate risk management options (notably interventions) as a way of reducing analytical complexities. For environmental health risks, the decision context might include the following questions:

  1. What are the magnitudes of the risks?

  2. What stressors are causing specific health outcomes and are other factors contributing to these outcomes by altering a population’s vulnerability? (See Kasperson(9) and U.S. EPA.(2))

  3. What is the likely distribution of health benefits (i.e., anticipated reductions in adverse health outcomes) associated with alternative risk management interventions across a population?

In addition, the challenging risk communication aspect of a risk characterization has been emphasized in earlier guidance, which notes “... risk characterization is not just about science. It makes clear that science doesn’t tell us certain things and that science policy choices must be made.”(10) This paper addresses risk characterization in CRAs, highlighting differences between conventional risk assessments and a CRA, and describing three approaches that can address certain challenges associated with characterizing risks in CRAs: the multiroute HI, grouping by exposure or toxic outcome, and the use of indices for screening assessments.

2. CUMULATIVE RISK CHARACTERIZATION

Risk characterization is often described as the final “synthesis” step in a risk assessment. It provides an overview of the assessment process and summarizes the main results and their uncertainties. One distinguishing feature of CRAs when compared with conventional assessments is the emphasis on stakeholder involvement throughout the CRA process. This section reviews that involvement and presents the structure of a CRA characterization.

2.1. Stakeholder Involvement in the CRA Process Prior to Risk Characterization

Stakeholder involvement is infused throughout the CRA.(11,12) Stakeholders could include anyone potentially affected by the risk management decision. Taken broadly, this set could include community residents, managers and other members of a federal, state, or local government, academic institutions, public health groups, religious groups, recreators such as fishers and hunters, workers, and businesses and industries, among others.

In the planning, scoping, and problem formulation step, stakeholders including decision makers help define the assessment’s goals and context and also help establish the type, rationale, and scope of the assessment. The assessment type can range from qualitative (e.g., prioritizing which waste sites to assess first), to more quantitative (e.g., a “semiquantitative” screening assessment that asks whether site risks are acceptable using conservative exposure assumptions), to a quantitative analysis that develops cumulative risk estimates. The type of assessment and risk management decisions the CRA is intended to inform also will guide both the types of stressors included and the level of detail required for the data acquisition and analysis. Stakeholders can help to both provide and evaluate exposure information as well as providing insights into how best to collect data on local populations (such as through citizen science activities). They also can identify population characteristics that could render certain populations more vulnerable to stressor combinations than the general population, or specific populations that are more impacted. For CRAs that evaluate different risk management options, stakeholders could also include technical experts familiar with the potential alternative interventions under consideration; such experts could characterize relevant aspects of the interventions including the feasibility, efficacy, and reliability, as well as the size and other characteristics of the population potentially affected by the intervention.

2.2. Structure of a Cumulative Risk Characterization

Summarizing the CRA goals and analyses can be complex and multifaceted. While the variety of CRA goals and settings will lead to many different assessments, the preparation of a cumulative risk characterization can be informed by a set of questions that consider the unique demands of this integrating portion of the CRA. Table II presents questions that can help prepare the planning and analysis steps of the CRA. The same questions can be used later to evaluate the completed CRA. These questions are organized by three topic areas: population and health effects; sources and exposures; and toxicities, groupings, and extrapolations.

Table II.

Questions to Guide Risk Characterization for a CRA

A Population(s) and Health Effect(s) of Interest

1 Are the important cause(s) of the health effect(s) identified?
2 Are susceptibilities or vulnerabilities of the population to exposures or effects described?
3 Do their locations or exposure patterns/activities change over the time frame assessed?
4 Do risk estimates for the population align with community concerns identified during planning and scoping?
5 If the CRA were initiated by a health effect, is it adequately addressed in the risk characterization?
6 Is the health effect(s) of concern in fact elevated in the community?
7 Are health effects observed in the community biologically plausible and consistent with toxicology or epidemiology data collected in other communities or populations?
8 How much detail and accuracy is lost in combining across health effects?
9 Do causal analyses adequately address epidemiological concepts of causation or those from evidence-based toxicology?
10 If the causal agents cannot be identified or if there is too much uncertainty, are the research gaps and specific types of studies needed adequately described?

B Multiple Sources and Exposure Levels

1 Are potential contributing sources identified? Has source tracking (forensics) or apportionment been delineated?
2 Have the nature and amount of stressors released by various sources been adequately characterized?
3 Are there other potential sources of these stressors in the community?
4 Has the environmental fate/behavior of the stressors been adequately evaluated over the time frame of interest?
5 Have changes in the composition of the mix of stressors or exposure pathways over the time frame of interest been addressed?
6 Have the relevant exposure pathways been identified?
7 If environmental or biomonitoring data initiated the CRA, are levels in the community elevated?
8 Are the spatial and temporal scales of the analysis consistent across each analytical component?
9 Do exposure levels and units from the dose-response data match estimated exposure ranges? Are the levels of understanding, accuracy, and detail for exposure and toxicity metrics sufficiently consistent?
10 Could alternative mitigation options change the mixture composition or total exposure level/dose?

C Toxicities, Groupings, and Extrapolations

1 If an index stressor or surrogate marker has been used to characterize exposure, how does that contribute to uncertainty in the assessment?
2 Can toxicological interactions be described at least in terms of direction (i.e., would they be expected to increase or decrease the risk in specific populations)?
3 Can the toxicological interaction magnitude be estimated for the most important stressor-pathway combinations?
4 How many toxicological interactions cannot be quantified, and how does this influence uncertainty of the CRA?
5 If no single stressor, exposure, or health effect is the predominant factor contributing to the overall risk, is the array of possible combinations of factors adequately presented?
6 Could stressors, exposure pathways, and effects be grouped differently? Does the grouping approach used avoid double counting?
7 Considering the information lost, what impact does reducing all measures to the lowest common level for grouping and composite analysis have on the results?
8 How much extrapolation is required, and how dependent is the extrapolation on default values?
9 Is the strength of support for the assumptions and default parameters used reflected in the CRA?
10 How do the composite estimates (e.g., index chemical equivalent dose, dose-additive predicted response, and EJ index) compare with a summary where each factor, chemical exposure, or response is presented separately?

The risk characterization step in a CRA is expected to include two parts to facilitate communication with two different audiences (Table III). The first part is an integrative analysis that synthesizes the different pieces of information characterizing the predicted risks. The second part, a nontechnical summary for lay audiences, discusses the results clearly, describes how the results can inform risk management decisions, and considers how the study limitations and uncertainties affect the use of the results.(10)

Table III.

Elements of a Cumulative Risk Characterization

Characterization Element Special Emphases for CRAs
Assessment goals Clearly state the CRA goals, including the initiating factor/s (or impetus) for the assessment.
Identify anticipated uses of the CRA (e.g., guiding consideration of alternate risk management actions).
Type of CRA Identify whether the CRA is qualitative, semi-quantitative, or quantitative. (When the type of CRA is determined prior to identifying specific goals and risk management decisions, this element may be combined with the preceding one.)
Ensure that the level of analysis matches the CRA type (e.g., screening vs. full risk) and is consistent throughout.
Exposures to combined stressors Identify the population and geographic areas of interest.
Describe the environmental conditions and exposure scenarios, considering the stressors, pathways, population characteristics, and additional factors that could affect exposures (i.e., exposure modifying factors) and health outcomes (i.e., effect measure modifiers including susceptibilities, vulnerabilities, buffers). Develop the population profile.
Describe the models used to assess exposures, accounting for the time frames of interest (including any critical exposure windows), and considering interactions that can affect fate and transport.
Joint toxicity and health effects from combined stressors Describe the data and models used to assess joint toxicity and effects from the relevant stressors considered (including effect measure modifiers), and provide context for the baseline health status of the population of interest.
Provide estimates of the cumulative risk predicted from combined stressors across relevant exposure routes and durations, accounting for population-level characteristics.
Uncertainty characterization, sensitivity analysis Examine the influences of study assumptions and limitations on the CRA results, such as the use of alternative toxicity values or different approaches for quantifying exposure levels for nonchemical stressors. Identify the strength of evidence for interactions (including toxicological) and for the relevance of those interactions to the population(s) of concern.
Describe (either qualitatively or quantitatively) the quality of the data used to assess the relationships between the stressors (and their combinations) and the predicted health effects, including whether risks are suspected to vary among different populations. In addition to performing quantitative sensitivity analyses, this uncertainty characterization could discuss the influence of chemical or nonchemical stressors that may be difficult to quantify or that cannot be integrated into the main analysis due to limited data, but that should still be included as part of a qualitative judgment of potential hazard.

2.2.1. Integrative Analysis

The integrative analysis describes the technical aspects associated with the predicted risks and their uncertainties. It evaluates the quality and relevance of the collective information and identifies information gaps, important uncertainties at the interfaces between different process steps, and the appropriateness of the different levels of analysis across the steps of the risk assessment. It also can highlight the important uncertainties within each CRA step from those that have already been identified.

The U.S. EPA CRA Framework(2) and Concepts document,(13) are among several reports that outline the set of steps commonly found in CRAs. (In focusing on risk characterization, this paper extends from the brief discussion of the integration step in the Concepts document.) As described in these documents, many CRAs focus on specific populations or communities and initially evaluate the population’s characteristics. Such evaluations are anticipated to be iterative because of stakeholder participation in the planning phase, with a refined population characterization that has the potential to identify several vulnerable populations. With such additional information, the exposure assessment and dose-response assessment could produce distinct results for each population. This adds to the complexity of a cumulative risk characterization and further highlights the value of simplifying methods. Some simplifications address joint toxicity and borrow from previous methods for chemical mixture risk, e.g., grouping chemicals by exposure media and toxicity (Table IV), while other simplifications involve categories of nonchemical stressors and population characteristics; these are further discussed with examples in Section 3.

Table IV.

Example Organization of Four Exposure Groups into Exposure-Toxicity Groups

Target Organs/Systems
to Guide
Toxicity Groupings
Exposure Groupingsa
1. Same Media,
Same Time
2. Same Media, Different Time 3. Different Media, Same Time 4. Different Media,
Different Time

1. Brain Group 1,1 Group 2,1 Group 3,1 Group 4,1
2. Liver Group 1,2 Group 2,2 Group 3,2 Group 4,2
n. (Organ/System) Group 1,n Group 2,n Group 3,n Group 4,n
a

The first column group is a common scenario for joint toxicity: same media, same time.

Joint toxicity.

The concept of joint toxicity used here is the consequence of multiple exposures affecting the same outcome, which is distinguished from the composite of separate evaluations of multiple unrelated exposures or toxic effects. The NRC Phthalates Report(14) proposed that the term “common adverse outcomes” be the basis for such assessments, noting that multiple toxicity pathways “can lead to a common outcome” (or related group of outcomes) and observing that “a focus on only a specific [toxicity] pathway can lead to too narrow an approach in conducting a cumulative risk assessment” for chemical and non-chemical stressors. An evaluation of the potential for joint toxicity begins with spatial overlap, i.e., identifying populations that are exposed to multiple stressors. The joint toxicity evaluation then usually requires temporal overlap of either the exposures or effects (the latter is illustrated by persistent effects after the initial exposure ends). If there is temporal overlap, the potential for joint toxicity should be evaluated. If these temporal overlaps are considered implausible, then joint exposure and joint toxicity might not need to be quantified, although such judgments should be explained. If joint toxicity is not estimated, those multiple exposures or toxicities should still be evaluated individually and the overall impacts on health should be described.

Joint toxicity or risk for chemical mixtures usually begins with identification of a common toxic effect for application of component-based risk assessments.(15) The two main types of formulas included in the risk characterization phase of a CRA used for chemical mixtures are those based on dose addition (for toxicologically similar chemicals, such as those exhibiting the same modes of action) and those using response addition (for toxicologically independent chemicals, such as those exhibiting different modes of action). The most common formulas for similar chemicals are: (1) the hazard index (HI), used most often to evaluate the potential for noncancer effects from mixture exposures at contaminated sites (Equation 1) and (2) the total margin of exposure (MOE) for gauging the hazard associated with exposures to some pesticide mixtures.(16,17) These are risk characterization formulas because they combine exposure estimates with dose-response estimates. For example, the HI formula is:

HI=i=1nEiRfVi (1)

where, for the ith chemical, Ei is the estimated exposure level for the population being assessed and RfVi is the risk-based reference value with the same units as the exposure value. For example, when E represents estimated daily oral intake, the RfV can be the U.S. EPA’s reference dose (RfD) where E and RfD are in the same units of mg/kg-day.(18) An HI of 1 or lower indicates that noncancer effects are unlikely to be of concern. An HI greater than 1 leads to a more detailed evaluation beyond this screening step, which can involve calculating (segregating) the HI for each target organ or system affected by the chemicals assessed (15, 16, 19).

Information on joint toxicity of chemical mixtures often exists only for two-chemical mixtures.(2022) The Agency for Toxic Substances and Disease Registry (ATSDR) and U.S. EPA have published binary weight-of-evidence (BINWOE) approaches to describe toxicological interactions between two chemicals for use in screening-level mixture assessments.(15,20) Both approaches evaluate the strength of the interaction evidence and its relevance to human health. The U.S. EPA (2000) approach(15) uses four ordered categories of evidence based on information quality and the extent of extrapolation from the key studies to interactions in humans: (1) clear relevance to humans with minimal extrapolation, (2) likely relevance to humans extrapolated from animal models, (3) plausible interaction direction but weak supporting evidence, and (4) inadequate interaction evidence or good evidence of no interaction. In contrast, the ATSDR (2004) approach(20) involves three detailed sets of categories: mechanistic understanding, toxicological significance, and scenario modifiers, with each containing ordered subcategories. The scenario modifiers set accounts for the use of surrogate information, e.g., in vitro instead of in vivo data.

A distinctive feature of those BINWOE schemes for toxicological interactions is they include specific categorical notations and a numerical score for each category. For example, ATSDR’s scheme characterizes the interaction evidence as I-A (with a score of 1*1=1) when unambiguous mechanistic data and direct evidence of toxicological significance exist, and as II-C (with a score of 0.71*0.32=0.23) when mechanistic data only exist on interactions of related compounds and when toxicological significance of the interaction is unclear. Each interaction BINWOE category and its score reflect a judgment of the extent of extrapolation required when using the available interaction information to modify the health risk estimate. This conveys a qualitative evaluation of the available evidence and suggests a level of confidence for the specific interactive effects addressed in a risk characterization.

While the ATSDR application is mainly qualitative, the U.S. EPA approach to addressing interactions goes further in that the BINWOE scores change the magnitude of the HI; in the U.S. EPA approach, the BINWOE score is positive for greater than dose-additive interaction and negative for less than dose-additive interaction. When the evidence indicates greater than dose-additive toxicity in the mixture, the EPA’s interaction-based HI (Equation 2) is increased over the usual HI.(15)

HIINT=j=1nHQj(kjnfjkMjkBjkgjk) (2)

In Equation 2, each chemical’s HQ is modified by the pairwise evidence for interactions, i.e., the quantity in parentheses (see (19) for full explanation). At this time, data are insufficient for most chemical combinations to quantify the magnitude of even binary interactions (M in Equation 2), especially at typical environmental doses;(23) for this reason, a default magnitude value of 5 is used by U.S. EPA.(15) While substantial uncertainties exist and should be discussed in the risk characterization, a BINWOE approach can communicate what is known about chemical interactions, including information provided to an interested community such as those living near a hazardous waste site.(5) One improved approach over what either agency has adopted is to use the U.S. EPA interaction-based HI formula along with the more detailed ATSDR BINWOE scheme. That would quantitatively change the HI by using the scores in the multilevel BINWOE scheme of ATSDR, e.g., setting B=−0.23 in Equation 2 when the BINWOE category is II-C and the interaction is less than dose-additive. That cross-agency approach provides a stronger interaction-based HI than what either Agency’s approach would give alone.

Information on joint toxicity from chemical and nonchemical stressors is generally lacking or sparse. One suggested contribution is a weight-of-evidence (WOE) approach similar to the toxicological interaction BINWOE approach for chemicals. In particular, the WOE approach could focus on the degree of causal evidence linking a nonchemical stressor with a health endpoint or for identifying effect measure modification. Such evidence can be based on epidemiological concepts of causation(24,25) or those from evidence-based toxicology(26) but structured in a few general categories. One example characterizes the impact of environmental disasters on psychological stress with resultant impact on reproductive success.(27) The approach involves two categories: relevance and depth of understanding, where each is subdivided into three subcategories.(27)

Relevance:
  • A = direct, adequate causal evidence in humans for the scenario being assessed

  • B = adequate causal evidence in humans for a similar scenario (some stressor or the levels of the stressors differ but not substantively), although alternative noncausal explanations cannot be ruled out; adequate causal evidence in nonhuman mammals

  • C = suggestive evidence in nonhuman mammals with some concerns over relevance to humans

Depth of understanding:
  • 1 = strong empirical or mechanistic/mode of action evidence (e.g., accurate model, key toxicological events are known)

  • 2 = moderate quantitative or mode of action evidence (e.g., empirical support but variation is high or some key events are unknown)

  • 3 = limited or no biological underpinnings of the link to the effects of concern

To illustrate, consider the published example linking the extent of maternal exposure to hurricane destruction with the risk of fetal distress.(27) The evidence for hurricanes leading to psychological stress and fetal distress could be category A1 because of direct human evidence, including “biomarkers”, following hurricane strikes.(28,29) In comparison, the evidence for a different source of psychological stress (e.g., community violence, neighborhood traffic noise) causing fetal distress could be category B1, where “B” is because much of the human information on maternal stress is qualitative and could be for different adverse environmental conditions, and much of the quantitative data are from experimental animals.(30) The subcategory for understanding could be “1” because maternal stress is linked to specific biomarkers such as increased fetal cortisol, which is linked to fetal distress.(29)

Quantitative causal information for modeling health risk to exposures from both multiple chemicals and nonchemical stressors is rarely available. For this reason, a quantitative approach to estimating cumulative health risk usually cannot be performed. However, a modified HI may offer a partial solution. A recent example develops a cumulative hazard index to reflect the joint contributions of noise and volatile organics exposure to hearing loss.(31) Because it was generated as an extension of the chemical-based HI approach (see Equation 1), the first step involved setting reference values for exposure to noise that would correspond to the RfD used to assess oral chemical exposures. While this cumulative HI is only a simple risk characterization index of the potential for specific exposures to cause noncancer health effects, its application here to the combination of noise and volatile organics is supported by data and knowledge of the underlying biological processes that describe how both stressors are causally related to hearing loss.(32,33) Future applications of a cumulative HI would be strengthened if additional information was available on interactions. For example, studies concerning other stressors have observed an antagonistic effect of zinc on lead toxicity,(4) while others have reported the combined impacts of psychosocial stress and exposure to hazardous chemicals on asthma(34) and on blood pressure.(35)

The risk characterization step should identify key stressors and present estimates of cumulative risk. If the overall impact of multiple effects cannot be estimated, it is useful to present an array with separate risk estimates for each effect. The risk estimates presented could be quantitative if sufficient scientific data exist, but they will often be semiquantitative or qualitative and the conclusions could also be qualitative. Some general phrases for qualitative-based conclusions include: “Exposures are considered below levels of toxicological concern” or “Possible interactions involving poor nutrition and heavy metals in children suggest that certain vulnerable populations such as infants and young children with poor nutrition status could potentially experience adverse health effects.” To enhance the usefulness to subsequent decision making, all risk estimates should be presented specific to the effects of most concern to each identified population, along with the elements most influential for each risk estimate (e.g., for chemicals: the exposure pathway and route, chemical group, and most important toxicological effect in the sensitive populations). Route is used here to define the physical interface through which human exposure to the chemical occurs, e.g., inhalation, oral (ingestion), or dermal contact. Exposure pathway is used here to describe the path a contaminant takes from its source through the environmental media (e.g., air, surface water, soil) to a point of contact with an individual, and the route through which that individual is exposed.

Uncertainty analysis.

The final part of the integrative analysis step involves evaluating the overall quality of the various analyses. Because uncertainties are to have been described in each part of a CRA, this evaluation should summarize the main sources of uncertainty, including measurement and methodological limitations, information loss from stressor groupings, and uncertainties due to reliance on a variety of extrapolations, such as using single-stressor toxicity assessments in dose-additive formulas when interactions cannot be quantified, or using structure-activity or in vitro studies for estimating some chemicals’ relative potency factors (RPFs) because their dose-response information is inadequate.(36) As with any uncertainty characterization, it is important to discuss the main assumptions used, including any mathematical models and default parameter values applied. Other possible sources of uncertainty include missing data, exposure assessment limitations, and use of uncertainty factors1. When the CRA involves many different types of stressors and/or effects, various approaches can be used to combine these disparate lines of evidence, which often include both animal and epidemiological data.(4244) The approaches can vary widely and may include meta-analytical techniques, empirically based probability distribution methods for data-rich cases to Bayesian methods, or expert judgments. As an example of the latter approach, U.S. EPA’s multicriteria integrated resource assessment tool was applied to assist regulatory decision making on air quality using both stakeholder and expert panel involvement.(45)

Because of the variety of approaches likely to have been implemented, it is important to identify the methods used and describe any critical elements affecting the resulting synthesis. For quantitative assessments, if sensitivity analyses were conducted to identify and evaluate the assumptions most influential on the risk estimate, the implications of those analyses should be described, especially regarding the value of new information and alternative assumptions. Also important is consideration of those elements not included in the risk model that could influence the ultimate risk estimate. At this final stage of a CRA, recommendations are useful for identifying ways the assessment could be improved, such as collecting and analyzing new data, using different analysis methods or modeling approaches, replacing default uncertainty factors with stressor-specific values, or even changing the scope of the assessment.

Simplifications and assumptions are especially important in a risk characterization because they are often the basis of report summaries used in risk communication, and so understanding their uncertainties is important. While an uncertainty discussion is also included in the Risk Characterization Summary, it is in the Integrated Analysis section where details are presented. For the simplifying approaches such as stressor grouping by exposure and/or toxicity (see Section 3.2), or the use of composite numerical indices that incorporate both environmental and population covariates (see Section 3.3), the uncertainty analysis should discuss why certain stressors or covariates were included (e.g., each one’s WOE regarding the effects of concern, both for relevance and depth of understanding of causality). It is also important to present the possible impact on the assessment of those stressors that were excluded. The Technical Documentation for the U.S. EPA’s EJSCREEN tool has excellent examples of presenting uncertainties and limitations when an index is used in a cumulative risk characterization.(46)

2.2.2. Nontechnical Risk Characterization Summary

The nontechnical summary of the risk characterization is critical for communicating the results to some stakeholders. This summary should include the main findings regarding potential health effects (likelihood and severity), identification of populations likely affected, the stressors of primary concern, and the key uncertainties. These results are interpreted in the context of the assessment goals established during the planning, scoping, and problem formulation step, including an examination of how relevant the available information was to the problem being addressed. For example, if the initial concern were a geographical area with a high incidence of a particular disease, then evaluating the relationships between specific exposures in that local population to that disease would be relevant to the risk assessment objectives and, ultimately, to the plausibility and stakeholder acceptance of proposed risk management decisions based on such analyses. Finally, U.S. EPA’s Risk Characterization Handbook(10) identifies four principles as key to effectively communicating the risk characterization: transparency, clarity, consistency, and reasonableness. While these apply to the technical summary as well, they are particularly germane to the nontechnical summary.

2.3. Evaluating the Success of the CRA

The evaluation of success of a CRA involves an objective examination of how well the results address the assessment goals, including any feedback from the risk assessors, stakeholders, and decision makers involved in the CRA. The level of the analysis should match that recommended in the planning, scoping, and problem formulation step, or an explanation should be provided if it does not. The evaluation of a cumulative risk characterization can be guided by a checklist designed to consider the often-complicated demands of this integrating portion of the CRA. Table II presents the example questions suggested in Section 2.2 to assist developing the risk characterization. These same questions could also be used to evaluate how successful the CRA was in addressing the scope and goals, including the quality of the data and methods used. These questions can be applied to all of the stressors considered (Table I).

The analysis of uncertainties is pivotal in determining whether the goals of the CRA have been met. Many uncertainties are of only minor concern for a qualitative or semiquantitative (e.g., screening-level) assessments, but they can be key determinants of the relevance and accuracy of a quantitative cumulative risk assessment. If the results do not sufficiently address the goals, iteration through one or more of the previous steps may be warranted, including reconsideration of the planning, scoping, and problem formulation step. For example, if childhood asthma is a concern in an urban community that is not well explained by chemical exposures alone, the CRA also might evaluate the potential interaction between smog and nonchemical stressors such as psychological stress from living in a high-crime area,(34) and the uncertainty analysis might focus on the relevance (e.g., how extrapolatable other study results are to the current population of interest), quality and uncertainties associated with that reported interaction.

3. EXAMPLE RISK CHARACTERIZATION APPROACHES FOR CRAs

This section presents three approaches that offer ways to address certain challenges associated with characterizing risks for CRAs. In addition to the increased complexity of CRAs, including more exposures and health effects to consider, these measures are often disparate in nature, which as noted above complicates the generalizability of study results when applied to a different exposure or target population. Targeted small-scale epidemiology studies have been recommended as a preferred approach for collecting data on the health effects associated with specific exposures to multiple stressors and buffers.(47, 48) (The term buffer is used here to represent agents and activities that benefit health or increase resilience, such as good nutrition, physical activity, and healthy weight.) However, the cost of such studies is often an obstacle, which means that few studies have been undertaken that directly address CRA complexities. With the more direct exposure measures and combinations of stressors and confounding factors that are examined in epidemiological studies, it is more important for statistical modeling challenges to be considered (e.g., Type II errors from insufficient statistical power to detect associations or statistical interactions, multi-collinearity amongst mixtures).(49, 50) Principal components regression and meta-regression analyses may offer solutions to some of these challenges.(51, 52) As with risk characterization of chemical mixtures, another helpful strategy is to use information on individual stressors and pairwise interactions to quantify risk estimates for the different stressor combinations.

3.1. Hazard Index for Multiroute Exposures

The HI (see Equation 1) is a decision tool that helps characterize the potential for noncancer effects from exposures to chemical mixtures.(15,16) This concept is illustrated in the following hypothetical scenarios evaluated for hazardous waste sites, which predict future exposures to multiple contaminants originating from the site to help inform cleanup decisions. For example, consider an abandoned site with several waste pits, from which chemicals could leach to underlying groundwater that flows offsite over time. In this hypothetical exposure scenario, for a nearby community, this groundwater source is used for drinking water. The estimated daily oral intake (E) of each chemical in that groundwater is then compared to its oral RfD or other acceptable (safe) intake level (see Equation 1). This ratio is called the hazard quotient (HQ), and, equivalent to Equation 1, for an exposure involving n chemicals the HQs are summed to estimate the HI for this drinking water exposure pathway.

Many environmental exposures involve more than one exposure route or pathway, such as inhalation of contaminated urban air and ingestion of contaminated food or water. An HI that considers exposures to multiple chemicals by multiple exposure routes or pathways can be termed a multiroute or multipathway hazard index, MHI (sometimes called a cumulative hazard index).(13, 16) For ease of notation in this manuscript, “route” will usually be used, even though “pathway” can be equally applicable. This distinction can be important. If the concern is exposure from contaminated tap water, then three exposure routes could be considered: ingestion (drinking the water), dermal (bathing), and inhalation (bathing and use of indoor steam-producing appliances such as dishwashers and washing machines) for volatile compounds. The tap water exposure pathway assessment then involves three route-specific assessments that are combined into the MHI.

The modification of the HI approach then begins with a route-specific HQ for a single chemical:

HQjk=EjkRfVjk (3)

where

HQjk = hazard quotient for the jth chemical, kth exposure pathway

Ejk = exposure for jth chemical, kth exposure pathway

RfVj = the health risk reference value for jth chemical, kth exposure route

Route-specific RfVs used by EPA include the RfD for oral exposures and the reference concentration (RfC) for inhalation exposures. This MHI can be determined two ways, each generating the same numerical estimate. In the first case, where the risk concern focuses on individual contaminants, a multiroute HQ (MHQ) is calculated for each chemical across all m routes (see Equation 4), then the MHI is derived from the sum of all n chemical-specific MHQs (see Equation 5).

MHQj=k=1mHQjk (4)
MHI=j=1nMHQj (5)

In the second case, if the risk concern focuses on affected environmental media (e.g., contaminated soil, contaminated groundwater), the HIs are calculated by route (indexed by k in the following formula) and then the m route-specific hazard indices are summed to obtain the MHI:

MHI=k=1mHIk=k=1m(j=1nHQjk) (6)

Consider the results from these two alternatives. In the first calculation sequence, if one or a small set of chemicals accounts for most of the MHI (based on their MHQs in Equation 4), then risk management options might involve controlling the sources of those chemicals (e.g., the waste pits that represent the source of those chemicals in the waste site example). In the second approach (inner sum in Equation 6), the initial calculation allows exposure routes (e.g., oral ingestion) or exposure pathways (e.g., oral and inhalation exposure from tap water use of drinking and showering) to be compared. If one pathway-specific HI (e.g., the HI for tap water ingestion and steam inhalation) dominates the MHI, then risk management options could focus on that medium and potential exposure options (e.g., consider an alternate water supply). These alternative calculations help identify the importance of various chemicals across affected media and the exposure routes and pathways by which people might be exposed, to help inform the associated risk management decision.

A multiroute HI calculation for a screening-level CRA can be illustrated for a hypothetical community whose water is supplied by a nearby river, from which they also eat fish. If the water were contaminated by surface discharges from a nearby waste site containing the volatile and semivolatile compounds shown in Table V, a simplified CRA might consider three exposures: ingesting fish, ingesting drinking water, and inhaling volatile compounds released from the water while showering or bathing. The assessment then involves applying standard toxicity values to calculate chemical/media- and route-specific HIs, which can be used to identify key chemicals and exposures for potential emphasis by risk management options.

Table V.

Estimated Hazard Quotients (HQs) and Hazard Indexes (HIs) for Hypothetical Residential Exposures to Chemicals from a Surface Water Source

graphic file with name nihms-1504000-t0001.jpg
a

BDCM = bromodichloromethane, DCA = dichloroacetic acid, MeHg = methyl mercury.

b

The toxicity reference values are for chronic exposures from U.S. EPA’s Integrated Risk Information System,18 except for the BDCM inhalation value; that value is derived from subchronic exposure data, which increases the uncertainty underlying an inhalation HQ estimate for chronic exposures. Note that where available, current IRIS values should be used in regulatory applications. Dose additivity is assumed for this screening HI estimate. The calculated HQ and HI values are presented to two significant figures.

c

A dash ( – ) indicates the value is not calculated (because a standard reference concentration has not been established for DCA).

For the example shown in Table V, the oral route includes contaminated fish and tap water. The combined oral HI reflects the four chemicals and two media in that set. The sum of the multiroute HIs (4.1 + 0.44) is 4.5 (bottom right); the primary route of concern is oral exposure (drinking water and ingesting fish), with a combined HI of 4.1 (3.5 + 0.57); the primary contributor is dichloroacetic acid. If the example had not included this chemical (or if its concentration were much lower), the MHI would still exceed 1 because of the other three chemicals. That MHI would then be segregated by organ/system to determine if the exposures still collectively contributed to an organ-/system-specific HI above 1.

If this hypothetical community received its household water from groundwater rather than the river, then the exposure assessment would not include drinking tap water supplied by the river or inhaling volatiles released to indoor air from that water. In this case, the river water would only be associated with an HI below 1 from fish ingestion, so no further assessment of the river water would be warranted beyond this screening HI. However, if the groundwater were also contaminated with similar compounds, and that water served as the source of the residential water supply, the CRA would then assess drinking water ingestion and inhalation of volatile compounds released from that water while showering or bathing. (Note that for simplicity, dermal exposures were not included in these example calculations.)

An MHI evaluation that includes risk estimates for multiple effects can lead to an increased focus on potentially susceptible population groups. For example, if certain contaminants in air were linked to an increased incidence of asthma attacks in children and young adults, and if certain contaminants in both air and water were linked to adverse reproductive effects, then the inhalation pathway would be of primary concern to asthmatics, while both inhalation and ingestion could be of concern to women of childbearing age. Young asthmatic women of childbearing age might then be of increased concern because of their vulnerability to both effects.

When one toxicological effect is common across multiple exposure routes, then the exposure analysis can shift to emphasize internal tissue dose, as this is one approach for integrating chemical exposures across multiple exposure routes.(13) In that case, multiroute exposures would be of primary concern for systemic effects rather than portal-of-entry effects (an example of the latter is effects on the nasal passage from inhaling a chemical; those would not be relevant to oral exposure of that chemical). Systemic effects reflect integrated toxicokinetic and toxicodynamic processes occurring beyond that entry point. Note that exposure route-specific health effects are not necessarily toxicologically independent even if restricted to portal-of-entry effects. In general, when multiple effects are indicated, interactions between effects can occur, particularly for high-dose, acute-duration exposures. For example, one effect could influence an exposure or uptake of a stressor (e.g., another chemical) by a different exposure route either by altering physiological processes (e.g., a food allergy restricting airways) or by affecting personal behavior (e.g., noxious odors from airborne contaminants resulting in mouth breathing and subsequent swallowing of the contaminants).

Multiroute and multieffect assessments are improved when information exists connecting these stressors and toxicities to specific populations. Information is often lacking on multiple stressors due to insufficient data on joint exposures among sensitive populations or on joint toxicity. This can result in conservative approaches such as assuming the exposures and toxicities apply to the entire population. This multiple stressor nature of a CRA can also lead to other uncertainties such as variable data quality (e.g., different measurement error structures across different exposure estimates) and variable data availability across the exposure routes of concern. The MHI application is usually strengthened if epidemiological studies, for example, are available that examine exposures by all routes for the stressors of concern or that include multiroute dose estimates (e.g., biomonitoring data that integrate internal doses across exposure pathways and routes).

When this MHI approach is used for screening-level assessments, all endpoints and their related stressors can be included in the MHI with its value interpreted the same way as the standard HI (i.e., at or below 1 indicates an acceptable condition, above 1 indicates further evaluation is warranted). When sufficient interaction information exists, the HQs (e.g., fourth column of Table V) can be adjusted using an appropriate construct such as EPA’s interaction HI (Equation 2). Greater than additive interaction evidence does exist for methyl mercury with Aroclor 1254/1260 but the evidence is fairly weak, which results in a low score that would not substantially change the HQs or overall MHI value.(53) Whenever considerable uncertainty exists for one chemical or exposure pathway that dominates the MHI, obtaining additional information can substantially improve the utility of the results for guiding an appropriate risk management decision. In the example of Table V, bromodichloromethane is the only chemical contributing to the inhalation HI, with its HQ = 0.44. While that value seems low enough not to be of concern, it is based on a subchronic inhalation reference value because no chronic value is available.(54) Often subchronic reference values are several-fold higher than their chronic counterparts, so a true chronic inhalation HQ for bromodichloromethane might be high enough that the HQ would exceed 1, thus obtaining chronic data might be a valuable improvement.

3.2. Grouping Stressors by Exposure and Toxicological Effect

U.S. EPA (13) proposed that grouping diverse stressors prior to assessing risks can simplify some complexities associated with CRAs. When assessing chemical mixtures, this approach initially entails developing CRA exposure groups based on the affected media and timing of specific exposure combinations, including duration and intermittency of exposures/effects.(55) CRA toxicity groups are then developed using toxicological and epidemiological information to evaluate whether the chemicals in question share a common toxic mode of action, causing the same primary or secondary health effects or influencing those effects through toxic interactions. The intersection of the CRA exposure and toxicity groups leads to the development of integrated CRA groups (hypothetical example provided in Table V). The chemicals in an integrated CRA group plausibly have exposures that overlap sufficiently in terms of pharmacokinetics or pharmacodynamics (e.g., persistent tissue damage) and exhibit interactions or cause the same health outcome(s).

This simple grouping concept, focusing on the toxicity and/or exposure pathway, is appealing because it achieves a simpler scenario, making the CRA more tractable and subsequent results easier to communicate. Grouping also provides opportunities to focus on the particular combination of stressors likely to affect a population. Grouping by overlapping exposure might also suggest possible interactions, e.g., chemical and physical processes affecting fate and transport of pollutants within or across environmental media. Grouping by common toxicological effects or effect precursors has been used successfully with chemical mixtures, such as the dose-additive RPF approach that has been applied to mixtures of toxicologically similar pesticides (e.g., U.S. EPA(17)) and to mixtures of dioxin-like chemicals(56). Those RPF dose addition approaches achieve another simplification, by expressing the multichemical exposures for the similarity group in terms of the equivalent exposure of an index chemical. As with all simplifications, caution is needed when implementing such an approach, and it is important to articulate the underlying logic and any data supporting such judgments about grouping. For example, inappropriately excluding a relevant stressor or factor, such as a large population fraction with concurrent occupational exposures, could lead to an underestimate of risk. Generally, uncertainties associated with such groupings need to be identified and the implications of incorrectly including or excluding a stressor on the predicted cumulative risks need to be described.

Stressors grouped because their exposures overlap in time and location could affect the same populations (i.e., such overlaps can be experienced by populations exposed to the same contaminated medium, e.g., drinking water). For any health effects in common, the risk characterization should address the multiple exposures (e.g., using the HI or MHI). When the stressors in the group are each associated with a single but different health effect, the risk characterization should address the potential for occurrence of multiple effects in the same set of people.

3.3. Use of Indices for Screening by Multiple Factors and Conditions

The multidimensionality of a CRA often could lead analysts to communicate predicted risks using different kinds of summaries that clearly and efficiently summarize and communicate results for the risk manager, the stakeholders, and subsequently for the general public. Indices can reduce dimensionality of a CRA, especially important when a single index can represent several risks derived from disparate measures of exposure and/or effect. High dimensional data and risk estimates can be difficult to understand. The HI for chemical mixtures (described previously) is among the simpler and better-known indices used in cumulative risk characterization, reducing the multiple exposure levels and toxicity-based values to a single numerical value to inform decisions on safety versus further investigation.

Indices can usefully represent risks associated with stressors and buffers that are difficult to quantify. Some indices may also be better suited to differentiate the impact of related stressors that may be occurring at different contextual levels. Several stressors and various population-level characteristics influence environmental health risks but are not easily incorporated into a quantitative CRA. Frequently, some stressors and factors are not relevant for regulatory intervention or are not amenable to measurement. Population-level characteristics in particular are often only identifiable based on either indirect or surrogate measures or simple categorical classifications. The former include demographic characteristics and markers of socioeconomic status (e.g., income level and educational attainment), while the latter include race (often a surrogate for socioeconomic stressors) and community-level factors (e.g., exposure to violence and access to medical care). Some environmental justice (EJ) investigations have used spatial statistics to investigate correlations between single health endpoints and various factors, such as estimating cancer risks from air toxics influenced by population-level characteristics including socioeconomics and race,(57) and determining retrospective descriptions of the joint influence on childhood asthma of air pollution and exposure to violence.(34)

Screening-level CRA methods based on semiquantitative approaches have fewer data requirements than quantitative risk estimation methods and thus offer more promise for wider use in combining information on chemical and nonchemical stressors along with population characteristics. Recent assessments have used indices in regulatory and risk-related contexts, such as to identify communities at higher health risk.(46,58) Environmental index formulas have a long history in science and regulation, originally applied to air and water pollution at the state and national levels.(59,60) Many of the properties listed for an ideal water quality index are also valid with current approaches to cumulative risk characterization: ease of use, inclusion of widely available variables, and the balance between oversimplification and technical complexity.(60) Some CRA approaches develop one group of indices for environmental conditions and another group for individual or community characteristics, and then combine these into an overall index to help identify the most vulnerable populations and/or most impacted environments.(61) For example, one version, California’s CalEnviroScreen, develops multiple environmental indices and multiple community or demographic indices and then calculates the overall score as the product of a “pollution burden” score and a “population characteristics” score. The first score reflects seven indicators of chemical exposures and five indicators of environmental effects, while the second score reflects three indicators of sensitive populations and four indicators that represent socioeconomic factors.(58,62) In contrast, the U.S. EPA’s EJSCREEN approach develops separate indices for each of 12 environmental indicators, with each index representing the product of the index for the selected environmental indicator and the demographic index and population count for the selected geographic area.(46)

The Cumulative Environmental Vulnerability Assessment (CEVA) is a different approach that has been applied in California.(63,64) The environmental and population indicators are developed separately in this method. The Cumulative Environmental Hazards Index (CEHI) reflects three groups of information on measured or potential chemical exposure. The Social Vulnerability Index (SVI) reflects six factors; five consider demographics and socioeconomics, and one is a community factor (close proximity to an in-patient health care facility). As part of the CEVA, the CEHI and SVI scores are then ranked and the population areas with joint scores of medium-high, high-medium, or high-high are considered to identify environmental justice areas, (i.e., populations in the San Joaquin Valley that are potentially at higher environmental health risk than the state average).

As with the chemical HI, these cumulative indices are not risk estimates (as projected incidence or severity of toxic effects) but are indicators of a level of concern related to the potential for higher exposure and vulnerability in specific communities and populations. Most of the documentation for the indices described do acknowledge their limitations, but some of those are embedded in other discussions or not clearly identified. For example, the technical manual for the EJSCREEN tool(46) notes that the environmental data are mainly on air pollutants or accidental releases, so concurrent exposures from other media (e.g., drinking water) or sources (e.g., drift from nearby agricultural spraying of pesticides) are not reflected. To help address that limitation, the EJSCREEN website includes additional information source linkages for those missing exposures. One deficiency common to several index approaches, which is identified in the EJSCREEN documentation, is the missing characterization of the existing health condition of the population.

The technical details supporting specific environmental health indices vary and can be confusing. Some descriptions of such indices detail the steps from raw data to subindex calculations to the final composite index. For most indices, what is missing is an indication of the scientific support for including those stressors, (i.e., the evidence linking those stressors to the health endpoints of concern). For example, the U.S. EPA’s EJSCREEN tool combines environmental with demographic information into a single overall index. The EJSCREEN Technical Documentation(46) has sections on “Rationale for Inclusion” for several environmental indicators, but no counterpart sections for any of the demographic indicators. More common is summaries of correlations or links between demographics or other population characteristics and certain health endpoints of concern. Rarely is there quantitative evidence of the extent of the combination effect of environmental exposure and demographics on health. The EJSCREEN documentation has one good example, but of a qualitative description: “Blood lead’s association with cardiovascular outcomes appears to be stronger among Mexican Americans and non-Hispanic blacks than non-Hispanic whites.”

In general, a major uncertainty with cumulative risk indices is the lack of a “common currency.” This commonality could include some standard descriptors that can apply to the body’s responses to chemical and nonchemical stressor exposures, and standard terms for describing the quality of evidence of the interaction between population characteristics and the stressors regarding vulnerability to toxic effects. While this is a significant research need, it is possible to envision the development of compilations of responses to combinations of specific population factors and chemical exposures forming a database for cumulative risk assessors. Possibly one could include in that database a structured WOE approach, like the one proposed by Rider,(27) to characterize the degree of causal evidence linking a nonchemical stressor and chemical stressor with a health outcome or for identifying effect measure modification. With consistent concepts and terminology, such a structured approach would also be useful for risk communication, such as explanations of why certain stressors and characteristics were included or excluded.

Although some of the uncertainties are not well understood for the aforementioned indices and the approaches to assigning scores, the results do identify general patterns and the more extreme situations, providing support for priority setting and, in some cases, specific program investment decisions. For example, CalEPA used the EnviroScreen results to identify disadvantaged communities that were then specifically targeted to receive investments of proceeds from the State’s cap-and-trade program.(65) These investments were aimed at improving public health, quality of life, and economic opportunity in the communities identified as most burdened, while reducing pollutants associated with climate change.(66) Recent investments include programs for waste diversion (recycling), agricultural land preservation, and urban forests in these targeted communities.(67)

4. DISCUSSION AND CONCLUSIONS

The risk characterization step for a CRA is more complicated than for conventional risk assessments. One obvious reason is related to the inherent complexity of considering a higher number of stressors and buffers that may be included. For example, a relatively simple EJ study of the influences on cardiovascular mortality included only two air pollutants (total suspended particulates and sulfur dioxide, SO2), one comparative index (a deprivation index based on income, unemployment rate, and high school completion rate), and proximity to traffic.(68) Despite this narrow scope, the analysis included three univariate models, two bivariate models (deprivation and traffic, deprivation and pollution), and the trivariate model, along with further analyses of effect measure modification by presence of chronic obstructive pulmonary disease. It is easy to understand the efforts made to develop simple formulas and decision aids.

During the Planning, Scoping, and Problem Formulation phase, a CRA might seem intractable because of the need to synthesize very different types of information regarding stressors. Some of the stressor exposure and toxicity data might rely on scientific judgments instead of empirical data. Further, there might be many uncertainties when integrating the health effects associated with different types of stressors. Gee et al.(69) observed that a key obstacle to conducting CRAs is often the scarcity of essential data, such as quantitative descriptors of interactions that have been qualitatively shown to be important. In the problem formulation phase of the CRA, missing or desirable information can be identified, such as evidence for interactions based on toxicokinetics or dynamics, and exposure information.(70) Often that realization about important data gaps occurs during the analysis phase of the CRA. The risk characterization then must derive conclusions and recommendations that are based on incomplete information, and characterizing the potential importance of such information can be challenging.

The procedures described for assessing cumulative risk include some elements that enhance the likelihood of success and relevance of the risk assessment. The first is the involvement of key participants, including stakeholders, risk managers, and risk experts, throughout the process. Stakeholder involvement early in the CRA process, in the Planning, Scoping, and Problem Formulation phase, facilitates the integral risk communication steps later in the process. Input from community groups or specific populations who might be perceived as being at risk also can help focus the scope of the CRA especially for site-specific concerns or targeted combinations of stressors, thus helping assure stakeholders that their concerns will be investigated. Consensus on the evaluation process to be applied for a given CRA and on the main information sources to be used can substantially improve the likelihood of completion and success of the CRA as well as acceptance and utility of the results. Transparency regarding the methods and related decisions also contributes to a more collaborative sense of ownership of the results and the ensuing risk management decisions. Further, the option to revisit the Planning, Scoping, and Problem Formulation phase during development of the CRA itself reinforces the iterative and cooperative nature of the process, as well as the intent to improve the CRAs and the decisions that follow whenever feasible.

Some specific conclusions can be drawn about the cumulative risk characterization step. The basic step has two parts: an integrative analysis that contains the risk estimates and can be highly technical, and a risk characterization summary that focuses on recommendations and uncertainties. For CRAs, this represents an expansion of the basic risk characterization step for conventional assessments, beginning with outputs from the previous steps, such as the population profile and chemical groups considered. The risk characterization context includes multiple stressors that might be a combination of chemical, biological, and physical agents to which people could be exposed via multiple routes, and other stressors (such as social and economic) that could further contribute to various health effects. Different interactions among stressors and interactions with other environmental conditions and population-specific characteristics are also assessed. Because CRA results are expected to be used by risk managers to make a practical decision, the transparency of the presentation of the results, including the uncertainty analysis, is particularly important, as is the scientific “soundness” of the planning and scoping process, data sources, analytical techniques, and logic used to make various technical decisions. It is important to keep in mind that CRAs may extend well beyond the identification of individual pollutants and their effects to also include personal and community conditions that can affect overall vulnerability. It is not surprising then that several CRAs have involved EJ concerns, and that one of the important conclusions can be the identification of population groups that might be particularly vulnerable to the exposures and health effects of interest. Finally, the cumulative risk characterization might suggest alternatives for the risk manager that cross the legal or political boundaries of health agencies, (e.g., to go beyond chemical cleanup and consider improvements to health care and many other types of collaborative interventions).(71)

In summary, the characterization of cumulative risk is more complex than for a conventional risk characterization in many ways, often because of missing data or a lack of understanding of the various stressor or stressor-factor combinations and their interactions. Important differences from a conventional risk characterization include:

  • Results and recommendations for decision makers could be multifaceted; (e.g., it might be difficult to identify a single stressor or exposure pathway or critical effect that is the primary contributor to the risk estimate).

  • Results could be based on groupings of stressors, exposure pathways, and effects, where such groupings are at least partially based on subjective judgments. More detailed analyses could then be conducted for those stressor groups and other factors considered important to health outcomes that are amenable to risk management options (e.g., to be reduced if stressors or enhanced if buffers).

  • The uncertainty analysis might be predominantly qualitative (including the overall uncertainty characterization) because of (1) the simplifying procedures used; (2) the default assumptions applied to address missing information, including interactions and multiple effects; and (3) exposure measures and factor descriptions that often are qualitative, uncertain, or incompletely capture the range of the exposure or factor occurrence in the population. Quantitative uncertainty analyses may also be included and would be of value to decision makers, but may be limited in scope due to data gaps or limited to specific aspects of the CRA.

The risk characterization step of a CRA can aid in identifying important missing information and research needs as well as explaining their impacts on the interpretation of the results. For example, in addition to identifying populations that might be more highly exposed (and possibly at greater risk), the risk characterization can identify important exposures that are not quantified and discuss the consequent limitations of the estimates provided.

In addition to an overarching uncertainty analysis and the integrated exposure and dose-response analyses, the risk characterization phase also contains an overview and critical evaluation of the CRA process. Because the risk characterization contains both a technically sophisticated synthesis and a nontechnical summary, it is developed to be accessible to all interested parties. By grouping stressors according to exposure and toxicity characteristics, the conceptual problem can be simplified to help make the analysis more feasible, with potential for improving the risk communication and focusing the risk management options. Much work remains in describing various nonchemical stressor exposures and in understanding the potential for interactions including joint toxicity of chemical and nonchemical stressors. The risk characterization approaches highlighted here are intended to provide a starting point for encouraging the preparation of CRAs, especially as further information becomes available from measurements of complex environmental conditions and associated human exposures and effects.

Footnotes

1

Uncertainty factors have been used by U.S. EPA and ATSDR to account for several extrapolations (or scaling) or other uncertainties in the data, such as interspecies scaling and accounting for sensitive population subgroups.(3741) For example, the EPA defines uncertainty factors as “one of several, generally 10-fold, default factors used in operationally deriving the RfD and RfC from experimental data. The factors are intended to account for (1) variation in susceptibility among the members of the human population (i.e., inter-individual or intraspecies variability); (2) uncertainty in extrapolating animal data to humans (i.e., interspecies uncertainty); (3) uncertainty in extrapolating from data obtained in a study with less-than-lifetime exposure (i.e., extrapolating from subchronic to chronic exposure); (4) uncertainty in extrapolating from a LOAEL rather than from a NOAEL; and (5) uncertainty associated with extrapolation when the database is incomplete.”

REFERENCES

  • 1.NRC. Risk assessment in the federal government: Managing the process. Washington, DC: National Academy Press, 1983. [PubMed] [Google Scholar]
  • 2.U.S. EPA. Framework for cumulative risk assessment. Report No.: EPA/600/P-02/001F. Washington, DC: Office of Research and Development, National Center for Environmental Assessment, 2003. [Google Scholar]
  • 3.Moretto A, Bachman A, Boobis A et al. A framework for cumulative risk assessment in the 21st century. Critical Reviews in Toxicology, 2017; 47(2):85–97. [DOI] [PubMed] [Google Scholar]
  • 4.ATSDR. Interaction profile for: Lead, manganese, zinc, and copper. Atlanta, GA: Agency for Toxic Substances and Disease Registry, U.S. Department of Health and Human Services, Public Health Service, 2004. [PubMed] [Google Scholar]
  • 5.Pohl HR, Mumtaz MM, Scinicariello F et al. Binary weight-of-evidence evaluations of chemical interactions−−15 years of experience. Regulatory Toxicology and Pharmacology, 2009; 54(3):264–271. [DOI] [PubMed] [Google Scholar]
  • 6.Barnes DG, Dourson ML. Reference dose (RfD): Description and use in health risk assessments. Regulatory Toxicology and Pharmacology, 1988; 8(4):471–486. [DOI] [PubMed] [Google Scholar]
  • 7.NRC. Science and decisions: Advancing risk assessment. Washington, DC: The National Academies Press, 2009. [PubMed] [Google Scholar]
  • 8.Sexton K Cumulative health risk assessment: Finding new ideas and escaping from the old ones. Human and Ecological Risk Assessment, 2015; 21(4):934–951. [Google Scholar]
  • 9.Kasperson J, Kasperson R, Turner B. Regions at risk: Comparisons of threatened environments. NY: United Nations University Press, 1995. [Google Scholar]
  • 10.U.S. EPA. Science Policy Council handbook: Risk characterization. Report No.: EPA 100-B-00–002. Washington, DC: Science Policy Council, 2000. [Google Scholar]
  • 11.DeFur PL, Evans GW, Cohen Hubal EA et al. Vulnerability as a function of individual and group resources in cumulative risk assessment. Environmental Health Perspectives, 2007; 115(5):817–824. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Makri A, Stilianakis NI. Vulnerability to air pollution health effects. International Journal of Hygiene and Environmental Health, 2008; 211(3–4):326–336. [DOI] [PubMed] [Google Scholar]
  • 13.U.S. EPA. Concepts, methods, and data sources for cumulative health risk assessment of multiple chemicals, exposures and effects: A resource document. Report No.: EPA/600/R-06/013F. Cincinnati, OH: Office of Research and Development, National Center for Environmental Assessment, 2007. [Google Scholar]
  • 14.NRC. Phthalates and cumulative risk assessment: The task ahead. Washington, DC: National Academies Press, 2008. [PubMed] [Google Scholar]
  • 15.U.S. EPA. Supplementary guidance for conducting health risk assessment of chemical mixtures. Report No.: EPA/630/R-00/002. Washington, DC: Risk Assessment Forum, 2000. [Google Scholar]
  • 16.U.S. EPA. Risk assessment guidance for Superfund. Vol. 1. Human health evaluation manual (Part A). Report No.: EPA/540/1–89/002. Washington, DC: Office of Solid Waste and Emergency Response, 1989. [Google Scholar]
  • 17.U.S. EPA. Guidance on cumulative risk assessment of pesticide chemicals that have a common mechanism of toxicity. Washington, DC: Office of Pesticide Programs, 2002. [Google Scholar]
  • 18.U.S. EPA. Integrated Risk Information System (IRIS), 2017. [PubMed] [Google Scholar]
  • 19.Hertzberg RC, Rice GE, Teuschler LK. Methods for health risk assessment of combustion mixtures. In: Roberts S; Teaf C; Bean J, Editors, Hazardous Waste Incineration: Evaluating the Human Health and Environmental Risks. Boca Raton: CRC Press, 1999. (pp. 105–148). [Google Scholar]
  • 20.ATSDR. Guidance Manual for the Assessment of Joint Toxic Action of Chemical Mixtures. Atlanta: Agency for Toxic Substances and Disease Registry, U.S. Department of Health and Human Services, Public Health Service, 2004. [Google Scholar]
  • 21.MacDonell MM, Haroun LA, Teuschler LK et al. Cumulative risk assessment toolbox: Methods and approaches for the practitioner. Journal of Toxicology, 2013; Article ID 310904 (36 pp.). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Mumtaz M, Hertzberg RC. The status of interaction data in risk assessment of chemical mixtures. In: J Saxena, Editor, Hazard Assessment of Chemicals. Vol. 8. Washington, DC: Hemisphere, 1993. (pp. 47–49). [Google Scholar]
  • 23.Boobis A, Budinsky R, Collie S et al. Critical analysis of literature on low-dose synergy for use in screening chemical mixtures for risk assessment. Critical Reviews in Toxicology, 2011; 41(5):369–383. [DOI] [PubMed] [Google Scholar]
  • 24.Hill AB. The environment and disease: Association or causation? Proceedings of the Royal Society of Medicine, 1965; 58(5):295–300. [PMC free article] [PubMed] [Google Scholar]
  • 25.Susser M What is a cause and how do we know one? A grammar for pragmatic epidemiology. American Journal of Epidemiology, 1991; 133(7):635–648. [DOI] [PubMed] [Google Scholar]
  • 26.Guzelian PS, Victoroff MS, Halmes NC et al. Evidence-based toxicology: A comprehensive framework for causation. Human and Experimental Toxicology, 2005; 24(4):161–201. [DOI] [PubMed] [Google Scholar]
  • 27.Rider CV, Dourson ML, Hertzberg RC et al. Incorporating nonchemical stressors into cumulative risk assessments. Toxicological Sciences, 2012; 127(1):10–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Zahran S, Breunig IM, Link BG et al. Maternal exposure to hurricane destruction and fetal mortality. Journal of Epidemiology and Community Health, 2014; 68(8):760–766. [DOI] [PubMed] [Google Scholar]
  • 29.Zahran S, Snodgrass JG, Peek L et al. Maternal hurricane exposure and fetal distress risk. Risk Analysis, 2010; 30(10):1590–1601. [DOI] [PubMed] [Google Scholar]
  • 30.McEwen BS, Tucker P. Critical biological pathways for chronic psychosocial stress and research opportunities to advance the consideration of stress in chemical risk assessment. American Journal of Public Health, 2011; 101(S1):S131–S139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Evans AM, Rice GE, Wright JM et al. Exploratory cumulative risk assessment (cra) approaches using secondary data. Human and Ecological Risk Assessment, 2014; 20(3):704–723. [Google Scholar]
  • 32.Morata TC. Interaction between noise and asphyxiants: A concern for toxicology and occupational health. Toxicological Sciences, 2002; 66(1):1–3. [DOI] [PubMed] [Google Scholar]
  • 33.Morata TC. Chemical exposure as a risk factor for hearing loss. Journal of Occupational and Environmental Medicine, 2003; 45(7):676–682. [DOI] [PubMed] [Google Scholar]
  • 34.Clougherty JE, Levy JI, Kubzansky LD et al. Synergistic effects of traffic-related air pollution and exposure to violence on urban asthma etiology. Environmental Health Perspectives, 2007; 115(8):1140–1146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Peters JL, Kubzansky L, McNeely E et al. Stress as a potential modifier of the impact of lead levels on blood pressure: The normative aging study. Environmental Health Perspectives, 2007; 115(8):1154–1159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Hertzberg RC. Extrapolation, risk-analytic. In: Wiley StatsRef: Statistics reference online: John Wiley and Sons, Ltd., 2016. (pp. 1–8). [Google Scholar]
  • 37.Abernathy CO, Donohue JM, Cicmanec J et al. Some comments on the selection of human intraspecies uncertainty factors. Human and Ecological Risk Assessment, 2004; 10(1):29–37. [Google Scholar]
  • 38.Dourson ML, Parker AL. Past and future use of default assumptions and uncertainty factors: Default assumptions, misunderstandings, and new concepts. Human and Ecological Risk Assessment, 2007; 13(1):82–87. [Google Scholar]
  • 39.Meek B, Renwick A, Sonich-Mullin C. Practical application of kinetic data in risk assessment--an IPCS initiative. Toxicology Letters, 2003; 138(1–2):151–160. [DOI] [PubMed] [Google Scholar]
  • 40.U.S. EPA. Characterization of data uncertainty and variability in IRIS assessments pre-pilot vs pilot/post-pilot: Draft. Washington, DC: National Center for Environmental Assessment, 2000. [Google Scholar]
  • 41.U.S. EPA. Consideration of the fqpa safety factor and other uncertainty factors in cumulative risk assessment of chemicals sharing a common mechanism of toxicity. Washington, DC: Office of Pesticide Programs, 2002. [Google Scholar]
  • 42.Brown KG, Strickland JA. Utilizing data from multiple studies (meta-analysis) to determine effective dose–duration levels. Example: Rats and mice exposed to hydrogen sulfide. Regulatory Toxicology and Pharmacology, 2003; 37(2):305–317. [DOI] [PubMed] [Google Scholar]
  • 43.Carroll RJ, Simpson D, Zhou H. Stratified ordinal regression: A tool for combining information from disparate toxicological studies. Report No.: 26. Research Triangle Park, NC: National Institute of Statistical Sciences, 1994. [Google Scholar]
  • 44.Reiss R, Gaylor D. Use of benchmark dose and meta-analysis to determine the most sensitive endpoint for risk assessment for dimethoate. Regulatory Toxicology and Pharmacology, 2005; 43(1):55–65. [DOI] [PubMed] [Google Scholar]
  • 45.Stahl C, Cimorelli A. A demonstration of the necessity and feasibility of using a clumsy decision analytic approach on wicked environmental problems. Integrated Environmental Assessment and Management, 2013; 9(1):17–30. [DOI] [PubMed] [Google Scholar]
  • 46.U.S. EPA. Ejscreen technical documentation. Washington, DC: Office of Policy, 2015. [Google Scholar]
  • 47.IOM. Toward environmental justice: Research, education, and health policy needs. Institute of Medicine, Washington, DC: The National Academy Press, 1999. [PubMed] [Google Scholar]
  • 48.Levy JI. Is epidemiology the key to cumulative risk assessment? Risk Analysis, 2008; 28(6):1507–1513. [DOI] [PubMed] [Google Scholar]
  • 49.Braun JM, Gennings C, Hauser R et al. What can epidemiological studies tell us about the impact of chemical mixtures on human health? Environmental Health Perspectives, 2016; 124(1):A6–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Taylor KW, Joubert BR, Braun JM et al. Statistical approaches for assessing health effects of environmental chemical mixtures in epidemiology: Lessons from an innovative workshop. Environmental Health Perspectives, 2016; 124(12):A227–229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Govarts E, Remy S, Bruckers L et al. Combined effects of prenatal exposures to environmental chemicals on birth weight. International Journal of Environmental Research and Public Health, 2016; 13(5):E495 (19 pp.). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Wang M, Kuchiba A, Ogino S. A meta-regression method for studying etiological heterogeneity across disease subtypes classified by multiple biomarkers. American Journal of Epidemiology, 2015; 182(3):263–270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.ATSDR. Interaction profile for: Persistent chemicals found in fish (chlorinated dibenzo-p-dioxins, hexachlorobenzene, p,p’-DDE, methylmercury, and polychlorinated biphenyls). Atlanta, GA: Agency for Toxic Substances and Disease Registry, U.S. Department of Health and Human Services, Public Health Service, 2004. [PubMed] [Google Scholar]
  • 54.U.S. EPA. Provisional peer reviewed toxicity values for bromodichloromethane (CASRN 75–27-4). Cincinnati, OH: Superfund Health Risk Technical Support Center, National Center for Environmental Assessment, 2009. [Google Scholar]
  • 55.Rice G, MacDonell M, Hertzberg RC et al. An approach for assessing human exposures to chemical mixtures in the environment. Toxicology and Applied Pharmacology, 2008; 233(1):126–136. [DOI] [PubMed] [Google Scholar]
  • 56.van den Berg M, Birnbaum L, Denison M et al. The 2005 world health organization re-evaluation of human and mammalian toxic equivalency factors for dioxins and dioxin-like compounds. Toxicological Sciences, 2006; 93(2):223–241. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Apelberg BJ, Buckley TJ, White RH. Socioeconomic and racial disparities in cancer risk from air toxics in maryland. Environmental Health Perspectives, 2005; 113(6):693–699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.CalEPA. California Communities Environmental Health Screening Tool, Version 3.0 (CalEnviroScreen 3.0): Guidance and screening tool. Sacramento, CA: Office of Environmental Health Hazard Assessment, California Environmental Protection Agency, 2017. [Google Scholar]
  • 59.Ott WR. Environmental Indices: Theory and Practice. Ann Arbor, MI: Ann Arbor Science, 1978. [Google Scholar]
  • 60.U.S. EPA. Water quality indices: A survey of indices used in the United States. Report No.: EPA/600/4–78 005. Washington, DC: Monitoring Technology Division, Office of Monitoring and Technical Support, 1978. [Google Scholar]
  • 61.Su JG, Morello-Frosch R, Jesdale BM et al. An index for assessing demographic inequalities in cumulative environmental hazards with application to Los Angeles, California. Environmental Science and Technology, 2009; 43(20):7626–7634. [DOI] [PubMed] [Google Scholar]
  • 62.Cushing L, Faust J, August LM et al. Racial/ethnic disparities in cumulative environmental health impacts in California: Evidence from a statewide environmental justice screening tool (calenviroscreen 1.1). American Journal of Public Health, 2015; 105(11):2341–2348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Huang G, London JK. Cumulative environmental vulnerability and environmental justice in California’s San Joaquin Valley. International Journal of Environmental Research and Public Health, 2012; 9(5):1593–1608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.London JK, Huang GH, Zagofsky T. Land of risk/ land of opportunity: Cumulative environmental vulnerability in California’s San Joaquin Valley. Davis, CA: UC Davis Center for Regional Change, 2011. [Google Scholar]
  • 65.CalEPA. CalEPA identifies communities targeted for cap-and-ttrade investments (October 31). Sacramento, CA: Office of Environmental Health Hazard Assessment, California Environmental Protection Agency, 2014. [Google Scholar]
  • 66.CalEPA. Greenhouse gas-reduction investments to benefit disadvantaged communities. Sacramento, CA: California Environmental Protection Agency, 2017. [Google Scholar]
  • 67.CalARB. Greenhouse gas fund reduction programs – appropriations as of September 2015. Sacramento, CA: California Air Resources Board; (February 4). [Google Scholar]
  • 68.Finkelstein MM, Jerrett M, Sears MR. Environmental inequality and circulatory disease mortality gradients. Journal of Epidemiology and Community Health (1979-), 2005; 59(6):481–487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Gee GC, Payne-Sturges DC. Environmental health disparities: A framework integrating psychosocial and environmental concepts. Environmental Health Perspectives, 2004; 112(17):1645–1653. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Solomon KR, Wilks MF, Bachman A et al. Problem formulation for risk assessment of combined exposures to chemicals and other stressors in humans. Critical Reviews in Toxicology, 2016; 46(10):835–844. [DOI] [PubMed] [Google Scholar]
  • 71.Nweke OC, Lee C. Achieving environmental justice: Perspectives on the path forward through collective action to eliminate health disparities. American Journal of Public Health, 2011; 101(S1):S6–S8. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES