Estimating the health benefits of environmental regulations

One Sentence Abstract:

Economic benefits analysis for environmental policy can be improved by incorporating current scientific approaches for evaluating health risks.

Public policies addressing environmental contaminants can improve the health of the population (1) and assessing these benefits is important both for decision making and for informing the public about how policy affects their welfare. Benefits analysis can be relatively straightforward when sufficient data are available on dose-response relationships, changes in exposure expected from a proposed policy, and other key inputs. For example, the U.S. Environmental Protection Agency (EPA) regularly produces estimates of the health benefits of regulations that reduce fine particles and ozone in air, sometimes amounting to billions of dollars annually (2).

However, estimating the health benefits from reducing exposure to other toxic environmental contaminants is frequently not done for health effects other than cancer, and thus the benefits of preventing exposure to chemicals linked to adverse health outcomes such as birth defects, neurodevelopmental effects and cardiovascular disease are typically not quantified. This shortcoming can be due to more limited data on health effects from exposure to these contaminants – a widely recognized issue (3) – but also can be due to analytic choices about how to use the available data.

Since 1981, executive orders have required benefit-cost analysis (BCA) for all economically significant federal regulations, and the results play an important role in decisions about regulating toxic environmental contaminants. Well-executed BCAs provide vital information on the range of possible outcomes and associated uncertainties, and should strive to be rigorous, objective, and complete. Recent amendments to the Toxic Substances Control Act, which require EPA to evaluate more environmental contaminants than it has in the past, underscore the timely importance of quantifying the full range of health benefits related to policy options.

After presenting an overview of benefits analysis, we focus on two problems with the treatment of non-cancer health outcomes in many benefits analyses.¹ First, health effects with less certain evidence or without a clear summary statement of the strength of the evidence are usually excluded from benefits analysis, even though it is highly likely that there is some positive value to reduction of those risks. Second, analysts frequently do not estimate dose-response relationships that provide changes in the probability of developing a specific health outcome from changes in exposure, and in such cases benefits remain unquantified. We argue that economic theory and scientific advances in the risk assessment literature provide an initial way forward.

Need to assess benefits comprehensively

Grounded in welfare economic theory, BCA compares, in dollar terms, the positive outcomes of a policy change (benefits) with detrimental outcomes (costs).² EPA has advanced many widely-used BCA methods and has well-established peer-reviewed guidelines for economic analysis (4). Both costs and benefits are difficult to estimate ex ante. Cost estimation requires choices regarding models and frameworks, predictions about private sector responses, and some costs – such as changes in consumer product quality – cannot be measured well (4,5). However, our focus here is on improving benefits estimation and in particular the omission of significant categories of benefits.

The goal of benefits analysis is to estimate society’s total willingness to pay (WTP), i.e., the total value to society of the positive effects of a policy, including health improvements. WTP is defined as the greatest amount that individuals would be willing to give up in income to obtain these improvements and not be any worse off.

WTP for health improvements encompasses the value of avoided treatment costs, lost productivity, and the value of avoided pain, suffering, and discomfort (4). WTP for particular health outcomes may be estimated from market transactions or through survey techniques. The value for reduced mortality risk, i.e., the “value of statistical life,” has been estimated using both methods. Estimating WTP for reductions in risks of non-fatal health effects is challenging, in part because value is a function of frequency, duration, severity, and other aspects of a given health effect. When WTP estimates are lacking BCAs may use more limited “cost of illness” estimates that reflect only direct medical costs and reduced productivity from missed work, and generally underestimate WTP. But values based on WTP are preferred; they are more comprehensive, represent the preferences of affected individuals, and are consistent with economic theory (4).

Whatever valuation approach is used, benefits analysis begins with selecting health effects with suitable evidence and dose-response information to estimate risks at varying levels of environmental contaminant exposure, and then quantifying population changes in health effects due to exposure reduction (Figure 1). Thus, BCAs rely on risk assessment documents that evaluate and synthesize the health effects literature (usually laboratory animal toxicology studies and/or human observational epidemiology studies) for a particular environmental contaminant.

Fig. 1. Analytical process for benefits analysis involving risk assessment and valuation.

Economic valuation draws upon outputs from key steps in risk assessment which are used to determine 1) which health effects are included in benefits analysis; and 2) the changes in risks of health effects associated with different levels of exposure. The first stage in the figure is characterized as “changes in emissions,” but applies to any changes in concentrations of environmental contaminants that leads to changes in human exposures (e.g. reduced use of a chemical in consumer products).

This reliance often leads benefits analyses to omit health improvements and/or risk reductions valued by those affected by a policy as described further below. More complete and informative BCAs require updating current risk assessment and benefits analysis practices to use the most current science and align better with economic theory.

Including health effects with less certain evidence in benefits analysis

EPA risk assessments for cancer and criteria air pollutants summarize strength of evidence regarding a health effect with standard terms ranging from “unknown” or “insufficient evidence” to “known” or “causal.” A high degree of confidence in the association between exposure and a health outcome is indicated by a qualitative descriptor such as “known,” “causal,” or “likely.” Selection of one of these descriptors is usually based on high quality epidemiologic studies (e.g. “known”) and/or on high quality animal studies (e.g. “likely”). EPA typically includes health outcomes designated as “known”, “causal” and “likely” in BCA. Moderate or lesser confidence in the available evidence is indicated with a descriptor of “suggestive,” applied for example, when there is evidence of an association between exposure and health outcome but human studies are few or lacking, and/or there are concerns regarding human or animal study quality (e.g., high risk of bias, indirect or imprecise evidence), and/or findings that are inexplicably inconsistent across studies.

When EPA judges the evidence for a health outcome to be “suggestive,” or – as is typical for noncancer health effects– there is no summary descriptor from an authoritative review of the evidence, EPA generally excludes the potential health risk from its primary quantitative benefits analysis (2). For example, Clewell and Crump (2005) observed that exclusion of cardiovascular effects (due to uncertainty of the effect at exposure levels relevant in the U.S.) from an EPA BCA of arsenic in drinking water may have substantially affected the results of the analysis, and that benefits related to the excluded cardiovascular effects may have been greater than the included cancer-related benefits (6). This practice implicitly assumes that exposed populations have zero WTP for reduced exposure when there is some evidence of an adverse health effect, but that evidence is not unambiguous. This assumption violates economic principles and is contradicted by findings (7).

Moreover, theory and evidence indicate WTP for reducing risks is higher for more severe health effects (e.g., cancer, cardiovascular disease, chronic health effects in children). As a result, the benefits of reduced exposure to a chemical with a “suggestive” relationship to serious heath endpoints may be higher than the benefits of reduced exposure to a chemical with a deemed “known” relationship to less serious health endpoints. It may be therefore misleading to take account of the latter but not of the former.

Including these less certain effects requires changes in both risk assessment and economics. An important first step is to provide greater clarity on the strength of evidence of each health effect (with, for example, a summary descriptor), even if summarizing health effects evidence with terms like “known” or “probable” can be complex and uncertain (8,9). Benefits analysis can include health effects that have any particular level of evidence, from those that might be considered “known as well as they can be known” to those with lesser degrees of certainty. In most cases the best quantitative weight for these latter effects is not zero.³ Benefits analysis could also accommodate qualitative statements of uncertainty if studies examined how WTP is affected by this information. These changes require additional analytic and policy choices.

Need for dose-response relationships to quantify potential health outcomes

Epidemiological studies of criteria air pollutants typically provide dose-response relationships applicable across a wide range of human exposures. These dose-response functions allow for quantifying and monetizing the benefits of reducing exposures at every level of exposure.⁴

However, other EPA risk assessments typically don’t quantify risks for health outcomes other than cancer, such as cardiovascular, respiratory, neurological or developmental effects. Instead, EPA practice is to estimate a Reference Dose (RfD) for these effects that assumes a threshold below which exposures have no quantifiable risk. Specifically, the RfD is a level “likely to be without appreciable risk of deleterious effects” in an exposed population. “Likely” and “appreciable” are not defined, so the RfD is not associated with any quantitative risk target and provides no direct measure of the risk of adverse health outcomes for any level of exposure either above, below or at the RfD. Without a dose-response relationship it is impossible to include these risk reductions in a BCA and, as a result, the related health benefits are implicitly valued at zero. A landmark 2009 report by the National Academy of Sciences (NAS) describes this issue well, observing that “many noncancer health end points are generally given little weight in benefit-cost analyses…in part because of the nature of the resulting qualitative risk characterization.”(10)

The assumptions embedded in the RfD were developed in the 1950s-1980s and do not reflect the current scientific understanding of the influences of environmental contaminant exposures on health. This limitation is recognized in the 2009 NAS report, which drew upon numerous prior publications in noting that the current RfD approach provides limited information for decision-making and “does not make the best possible use of available scientific evidence.” (10–12)

The NAS further observed that the default assumption of a population threshold built into the RfD is questionable for most environmental contaminants. Instead, no population threshold is expected, even for most instances of dose-response relationships that may have thresholds for each individual, due to multiple sources of variability in the population including differences in both intrinsic (e.g., life stage, reproductive status, age, gender, genetic traits) and acquired (e.g., pre-existing disease, geography, socioeconomic, cultural, workplace, and exposure to other environmental contaminants) factors (6,10–13) (Figure 2). The NAS recommended using approaches that do not assume a threshold for population dose-response assessment, unless the science specifically supports assuming a population threshold for the environmental contaminant in question.

Fig. 2. Individual dose-response relationships (left) and population dose-response relationship (right).

Individual dose-response relationships exhibit varying individual thresholds. The variability across individuals results from differences in genetics, lifestage, background disease processes and susceptibility, and exposure to other environmental contaminants. The population dose-response relationship then results from the proportion of the population whose individual thresholds are exceeded at any dose.

Source: Committee on Improving Risk Analysis Approaches Used by the US EPA, Science and Decisions: Advancing Risk Assessment (Figure 5-4). Reprinted with permission from the National Academies Press, Copyright 2009, National Academy of Sciences.

Implementing the NAS recommendations for dose-response assessment to assess differences in risk at varying levels of exposure would allow for quantifying and monetizing health risk reductions for additional health effects that are not currently monetized, and in doing so provide quantified estimates of benefits for a range of potential policy choices. Two main approaches to implementing the NAS recommendations are available. First, similar to existing practice for criteria air pollutants, regression models can be used to estimate a dose-response function when health effects studies are sufficient, as in the case of inorganic arsenic (14). Second, probabilistic models may be used to extrapolate from experimental or epidemiological data, accounting for uncertainties in animal-to-human differences, human population variability, and limitations in the database as appropriate; the World Health Organization recently issued guidance for this approach (15).⁵ In either approach, a single “bright line” akin to the RfD may still be specified by identifying a target level of risk and estimating the dose associated with that risk.

Steps forward

BCA is needlessly constrained by analytic practices that are scientifically outdated and are inconsistent with economic theory, and these limitations can result in the exclusion of important health effects from the estimated benefits of reducing exposure to toxic environmental contaminants. The current practice may have a substantial, undesirable effect on related public policy decisions.

Initial steps for more comprehensive assessments of benefits of regulating toxic environmental contaminants are: (1) to include health effects with less certain evidence and to have a clear summary statement for those that currently do not to include in benefits analysis; and (2) estimating dose-response relationships to quantify health risk reductions that provide changes in the probability of developing a specific health outcome from changes in exposure. Together, these would help ensure reductions in exposure and risk are valued according to how those affected value them, and decision makers would be provided with more complete estimates of policy outcomes. Greater collaboration between risk assessors and economists will be important for developing necessary methods, tools, and approaches.

Government regulatory decisions influence the health and welfare of the population, and BCA remains important for informing these decisions. Joint advancement of methods in both risk assessment and economics can improve