The low but uncertain measured benefits of US water quality policy

Abstract

US investment to decrease pollution in rivers, lakes, and other surface waters has exceeded $1.9 trillion since 1960, and has also exceeded the cost of most other US environmental initiatives. These investments come both from the 1972 Clean Water Act and the largely voluntary efforts to control pollution from agriculture and urban runoff. This paper reviews the methods and conclusions of about 20 recent evaluations of these policies. Surprisingly, most analyses estimate that these policies’ benefits are much smaller than their costs; the benefit–cost ratio from the median study is 0.37. However, existing evidence is limited and undercounts many types of benefits. We conclude that it is unclear whether many of these regulations truly fail a benefit–cost test or whether existing evidence understates their net benefits; we also describe specific questions that when answered would help eliminate this uncertainty.

Investments to decrease pollution in rivers, lakes, and other surface waters have constituted one of the largest environmental expenditures in US history. Since 1960, US public and private actors have spent over $1.9 trillion ($2014) to abate surface water pollution. This comes to over $140 per person per year, or over $35 billion total per year (Fig. 1). These totals exceed total public and private spending to abate air pollution (1), and they exclude investments to purify drinking water. At peak spending in 1977, these investments represented 0.7% of the US gross domestic product (GDP).*

Fig. 1.

Spending on US water pollution control efforts, 1960–2014: CWA programs, industrial abatement, and USDA conservation spending. This figure displays annual spending on water pollution control efforts from 1960 to 2014. Estimates shown are in billions of dollars, deflated to 2014 using the Engineering News Record 2014 Construction Price Index. Estimates of municipal spending (dark blue) include state, federal, and local capital and operations and maintenance costs associated with CWA federal grants and federal and state-sponsored Clean Water State Revolving Funds. These municipal values exclude investments that occurred independently of the CWA, which were relatively more important before 1968. Industrial abatement costs (medium blue) are derived from annual Pollution Abatement Costs and Expenditure Survey from the US Census. Nonpoint source expenditures (light blue) include funds provided under section 319 of the CWA and technical and financial expenditures related to soil and water conservation programs sponsored by USDA. Total expenditures as a fraction of deflated GDP are shown as a dashed red line. See SI Appendix for more details.

These investments have large costs but could have larger benefits. In the early 20th century, water-related mortality like cholera and typhoid killed tens of thousands of people every year. At the same time, regular fires occurred on many US rivers. These problems largely ceased by the late 20th century, plausibly due in part to water quality regulation. More broadly, water quality may be important for outdoor recreation, industrial production, agriculture, housing, commercial fishing, and health. The benefits of early investments in water quality are generally believed to exceed their costs (2). Actual cost–benefit analyses (CBAs) were rarely done for regulations before the 1970s, however, and were still rudimentary during the 1980s (3, 4).

Regulations promulgated since 2000 have been subject to detailed CBAs. Most of these analyses have the surprising finding that these regulations’ benefits are much smaller than their costs (i.e., they have negative net benefits). Table 1 summarizes 20 such CBAs. The mean analysis found that a regulation’s benefits are one-half of its estimated costs; the median analysis found a benefit–cost ratio of 0.37.^† Only 2 of these 20 analyses estimate benefits that clearly exceed costs. One is for a regulation with zero estimated costs, and the other is part of a controversy surrounding the costs and benefits of the recent Waters of the United States (WOTUS) rule.^‡ We believe this fact, that most government and academic benefit–cost analyses find negative net benefits from surface water quality, is not generally known.^§

Table 1.

CBAs of water quality programs

Regulation	Study time frame	Benefit-to-cost ratio	Benefits, per year	Costs, per year
CWA
Freeman (6)	1985	0.19–1.23	$13.6B to $65.9B	$53.7B to $71.6B
Carson and Mitchell (7)	1990s	0.61–1.25	$98.1B	$78.3B to $160.2B
Lyon and Farrow (8)	1990s	0.25–1.16	$10.9B to $22.0B	$18.9B to $43.7B
US EPA (21, 61)	1990s	0.79–0.88	$18.9B	$21.5B to $24.0B
Keiser and Shapiro (1)	1962–2001	0.24	$3.9B	$16.3B
WOTUS
Obama Administration	2015	1.10–2.41	$0.3B to $0.6B	$0.2B to $0.5B
Trump Administration	2017	0.11–0.30	$0.03B to $0.07B	$0.2B to $0.5B
CRP
Hansen (47)	2000s	0.76–0.87	$2.1B	$2.4B to $2.7B
Effluent Guidelines
Centralized Waste Treatment	2000	0.07–0.23	$4M to $14M	$60M
Landfills	2000	0.00	<$0.1M	$13M
Transportation Equipment Cleaning	2000	0.11–0.33	$3M to $9M	$27M
Waste Combustors	2000	0.15–0.5	$0.3M to $1M	$2M
Coal Mining	2002	>1	$22M to $24M	$0M
Iron and Steel Manufacturing	2002	0.11–0.58	$2M to $11M	$19M
Concentrated Animal Feeding Operations	2003	0.61–1.06	$320M to $557M	$526M
Metal Products and Machinery	2003	0.09	$2M	$22M
Concentrated Aquatic Animal Production	2004	0.05	$0.1M	$2M
Meat and Poultry Products	2004	0.05	$4M	$86M
Construction and Development	2009	0.39	$429M	$1,108M
Steam Electric	2015	0.94–1.18	$464M to $582M	$493M

The study time frame describes the time period for which benefits and costs were estimated for the CWA and CRP estimates. For the WOTUS, the initial rule was released in 2015 during the Obama Administration. The Trump Administration calculations reflect a 2017 proposed recodification of existing rules. For effluent guidelines, the time period indicates the year that the rule was released. Freeman (6) estimates benefits in 1985 of removing conventional water pollutants. Corresponding costs are estimates of annual control costs in 1985 based on the Council of Environmental Quality’s estimate of water pollution control costs in 1978. Carson and Mitchell (7) estimate the benefits and costs of moving water quality from a national baseline of nonboatable to swimmable water. Cost estimates range from Department of Commerce’s estimates of $78.3B in 1988 expenditures to projected expenditures of $160.2B in the year 2000. Lyon and Farrow (8) estimate benefits of a one-step ($10.9B) or two-step ($22B) improvement in the water quality ladder. Cost estimates reflect various control options considered by the authors. US EPA (21) estimates in-place annual benefits due to the CWA in the mid-1990s. Corresponding cost estimates from US EPA (61) are incremental costs of controlling water pollution due to the CWA for 1994 ($21.5B) and 1997 ($24B). Keiser and Shapiro (1) estimate the benefits and costs of the CWA’s municipal grants program. The $16.3B in costs per year reflect total of $650B in costs spread over 40 y. Benefits reflect the increase in housing values due to the grants program (0.24*$16.3B, where 0.24 comes from the last column of table 6 in Keiser and Shapiro). Costs and benefits for WOTUS taken from the 2015 rule and 2017 proposed rule published in the Federal Register. Reported benefit-to-cost ratios for WOTUS reflect two individual scenarios considered by the rule, while cost and benefit ranges reflect the lower and upper bounds of these scenarios. Costs and benefits for effluent rules taken from finalized rules published in the Federal Register. All dollars have been deflated to 2014 dollars using the Engineering News Record 2014 Construction Price Index. Abbreviations: B, billion; M, million.

This situation is unusual, since the US government generally implements policies with positive ex ante estimated net benefits. A recent review summarized CBAs of the 112 major federal rules implemented over the period 2002–2012 across the entire US government (10). Summed over all rules and years, the ratio of estimated benefits to estimated costs ranged from 3.5 to 12.3 (lower versus upper bound). Surprisingly, a large majority of these total benefits and costs of major federal regulations came from Environmental Protection Agency (EPA) regulations, even though the EPA was only one of many departments studied. The total benefit–cost ratio for EPA regulations alone ranged from 3.7 to 17.5. Essentially each department had positive net benefits overall for its regulations. (The only exception is the lower-bound estimate for Homeland Security, since its two major regulations have a lower-bound benefits estimate of 0.) Six of the 112 regulations in this Office of Management and Budget (OMB) review focused on surface waters, however, and these regulations had total estimated benefits of $23 to $33 million and total estimated costs of $434 to $579 million, implying an unfavorable benefit–cost ratio of 0.05–0.06. Apart from these surface water regulations, only a few of the reviewed regulations covering the entire federal government had negative estimated net benefits. The OMB review did cover two regulations of drinking water, which had more favorable benefit–cost ratios of 5.3–14.8.

This provocative fact—that most analyses, including those from EPA, estimate negative net benefits from surface water quality regulations—leads to a critical question: Do the costs of current US water quality regulations actually exceed their benefits, or do existing analyses substantially understate true benefits or overstate true costs? We conclude that available evidence is insufficient to answer this question, although it is clear that current analyses exclude potentially important benefits.

The rest of the paper proceeds as follows. Cost–Benefit Analysis introduces readers to CBA. Methods for Measuring Costs and Benefits of Water Quality Policy describes two sets of tools that researchers use to conduct these analyses for water quality policy—integrated assessment models (IAMs) and econometric approaches. Are Current CBAs Biased? discusses the potential biases in current studies of water quality regulations. Discussion suggests direction for future research.^¶

Cost–Benefit Analysis

For water quality or any other policy, economists use CBA to assess whether a policy increases the total value of resources available to all members of society, accounting for market goods and services (e.g., labor and firm outputs) and nonmarket goods and services (e.g., changes in water quality). In the absence of regulation, since the parties that create an externality like pollution do not bear its full costs, private decisions through markets do not necessarily maximize aggregate well-being. CBA can help determine the level of pollution that maximizes social welfare. Even though some laws like the Clean Air Act explicitly forbid comparisons of costs to benefits, evidence strongly suggests that regulators compare costs and benefits in implementing policy (14).

Each US president since Ronald Reagan has issued or upheld variations on prior executive orders requiring the use of CBA to evaluate proposed federal regulation.^# For example, President Obama issued Executive Order (EO) 13563 stating that a regulation should be proposed or adopted only upon a reasoned determination that its benefits justify its costs, although this order recognizes that some benefits and costs are difficult to quantify. Supported by these executive orders, the EPA has undertaken thousands of economic analyses since 1982, including cost–benefit and cost effectiveness analyses. Of ∼4,500 regulatory analyses listed in an EPA database, about 1,300 involve water quality (Fig. 2).^||

Fig. 2.

Number of US EPA economic analyses by year. This figure displays the number of regulatory economic analyses performed each year by the EPA, including CBAs, cost effectiveness analyses, and other types of economic analysis. Total analyses are represented by the dashed gray line. Number of water pollution analyses are displayed as the bottom portion of the bar chart (solid blue). Number of air pollution analyses are displayed as the top portion of the bar chart (red dots). See SI Appendix for more details.

Why use CBA at all? Some argue that policymakers would not implement a regulation unless its benefits exceed its costs, so the mere fact that these policies were implemented shows they have positive net benefits. A few ideas, however, show the importance of CBA even after policymakers and the public choose to implement a policy. Policymakers implement laws and regulations before all of their benefits and costs are known, so their ex ante beliefs may differ greatly from a policy’s ex post effects. Elected officials also have many objectives including winning votes and campaign fundraising, so may support policies that improve chances of reelection but do not increase social welfare. Furthermore, since many voters choose representatives based on a single issue, voting outcomes may not accurately reflect benefits and costs to all parties. Indeed, the reasoning behind EO 12291 (requiring a regulatory impact analysis of all economically significant regulations) and its successors was precisely a belief in the 1970s and 1980s that many regulations did not pass a cost–benefit test (4).

Moreover, even a perfect CBA may provide an imperfect guide to policy, since social decisions may depend on many features that CBA does not capture (15). One important issue that CBA ignores is equity within and across generations, although careful CBAs at least seek to identify which social groups receive the benefits and pay the costs (16).

Methods for Measuring Costs and Benefits of Water Quality Policy

The preceding section explains what CBA seeks to accomplish, but not how researchers implement it. The methods of CBA for water quality are important to explain since it is the details of prevailing methods that lead in part to our conclusions that existing CBAs may understate true net benefits, but that bias in existing estimates is not inevitable. Researchers use two general categories of tools to measure the costs and benefits of water quality policy—IAMs and econometric methods. We explain each in turn.

IAMs.

IAMs combine quantitative descriptions of economic and ecological processes to study environmental problems. Full IAMs combine four components—emissions, pollution transport, environmental and human outcomes, and valuation (SI Appendix, Fig. S1). Each component could have a stand-alone model, and some IAMs merely consist of links among four existing stand-alone models. The first component may include equations describing firm production and emissions decisions given prices, market structure, and relevant policies. The second has hydrologic equations that track the transport of these pollutants through a riverine network. The third then quantifies how these transported pollutants affect nonpollution outcomes like fish populations or endangered species habitats. The fourth places a dollar value on these environmental outcomes for a CBA.

IAMs differ in how they model pollution transport, and pollution transport models developed in tandem with EPA regulations (17). Before 1995, most regulatory assessments lacked pollution transport models. CBAs in this period often reflected local case studies. Assessments in the late 1990s, including Clean Water Act (CWA) effluent limits for several industries, began accounting for dilution of water pollutants and dispersion. More sophisticated models of pollution transport appeared in the early 2000s.

IAMs also differ in how they link changes in ambient water pollution to social welfare. Some IAMs incorporate estimates of marginal willingness to pay for changes in specific physical water pollutants. Others link willingness to pay numbers to changes in index measures of water quality like whether water meets a safety standard for fishing. Still others link to nonpollution outcomes like fish populations (refs. 17–19 provide reviews). These willingness-to-pay estimates may come from revealed or stated preference studies. (Revealed preference studies combine data on behavioral outcomes like recreational choices, health, or human values with data on water quality to estimate benefits in a statistical model. Stated preference studies use surveys of what individuals state they are willing to pay for water quality improvements.)

Finally, IAMs can help assess cost effectiveness or compare costs and benefits. For example, Rabotyagov et al. (20) use an IAM to identify the most cost-effective regions for conservation actions to shrink the low-oxygen (hypoxic) zone in the Gulf of Mexico. If they had incorporated estimates of the benefits of reducing the hypoxic zone, they could have compared costs and benefits of different policies.

IAMs have broad influence in part due to their flexibility. Researchers can use an IAM to analyze many potential policies, and hence regulatory analyses typically use IAMs. For example, many EPA analyses have used the National Water Pollution Control Assessment Model (NWPCAM) to evaluate the potential benefits of effluent regulations, as well as to conduct an ex post analysis of the CWA (21). EPA is developing a replacement for NWPCAM, called the Hydrologic and Water Quality System (HAWQS). [See the US EPA’s website for more details: https://www.epa.gov/waterdata/hawqs-hydrologic-and-water-quality-system (accessed February 28, 2018).] HAWQS has similar structure to NWPCAM but uses more sophisticated description of pollution transport and economic valuation. Likewise, many analyses of the US Department of Agriculture (USDA) rely on the Conservation Effects Assessment Project (CEAP), which uses an IAM to assess the cost effectiveness of nonpoint source controls like conservation programs and agricultural best management practices (22, 23).

IAMs also have important limitations. Each constituent piece of an IAM involves strong modeling assumptions. Typical IAMs rely on dozens of underlying parameter estimates, each from a separate study. The aggregated nature of an IAM may obscure this uncertainty. While any model only approximates reality, combining multiple models from disparate fields may worsen this approximation. Most IAMs do not produce confidence intervals, and the few that do generally reflect only sampling variability for a few parameters, rather than model uncertainty. Pindyck (24) reviews similar concerns for IAMs used to analyze greenhouse gases.

Statistical and Econometric Approaches.

The second general approach employs statistical or econometric models. These studies typically use regressions to assess how past policies have affected pollution emissions, ambient water quality, or human uses and values.

Existing analyses often study emissions from municipal treatment plants, industrial facilities, or other point sources. These studies may use theories of economic behavior to generate hypotheses of how a policy might affect industrial emissions, and then test these hypotheses empirically (e.g., refs. 25–27). While many such studies estimate how a policy or action influences emissions, most stop short of estimating costs or benefits of the policies they study. Although researchers have used field-level plots to test how land management practices affect agricultural and urban runoff (e.g., refs. 28 and 29), we are unaware of similar econometric analyses for nonpoint sources.

In addition to studying how policies affect ambient pollution emissions, these papers may also study how policies affect ambient surface water quality. For example, scholars have used a few long-term monitoring stations to estimate how the Conservation Reserve Program (CRP) and hydraulic fracturing affect water quality (30, 31). Others investigate how decentralized environmental regulation affects transboundary pollution (32, 33). Keiser and Shapiro (1) examine how CWA grants the federal government gave to cities to improve wastewater treatment affected US surface water quality. Through these grants, they estimate it cost approximately $0.5 million per year to increase dissolved oxygen saturation in a river-mile by 10%. This extends water quality analysis to obtain a cost effectiveness analysis.

A less common approach examines the effects of water quality policy directly on human outcomes. This approach is useful for a few reasons. It can directly compare the costs and benefits of a policy rather than tracing its effects through effluent, then ambient water quality, then human use, then valuation. Correspondingly, it limits or at least makes more transparent the modeling and statistical uncertainty challenges that IAMs can create. This approach also avoids the external validity challenges of transferring results of benefit estimates from other studies, which often requires strong assumptions about preferences for water quality across space and time. Keiser and Shapiro (1), one example of this approach, estimate the effects of CWA grants on local housing values. Many other studies examine the effects of water quality (although not a specific policy) on recreation and home values through travel cost and hedonic studies (34–36). Many IAMs use results of these studies to calibrate their valuation functions.

Are Current CBAs Biased?

The preceding section explains methods economists use to analyze water quality policies but says little about their accuracy. Can we trust the common finding that benefits of many water quality regulations are less than the costs? Or have study limitations led to biased estimates of costs or benefits, making such a conclusion unwarranted? If this conclusion is unwarranted, what are the most important sources of bias and what steps can help resolve these uncertainties? One could ask these questions of many types of CBAs. Since estimated net benefits of water quality regulation are so much less positive than estimated net benefits of many other types of regulation, an important question is whether these issues are relatively more important for water pollution. This section discusses several types of bias, and for many discusses the extent to which the bias is inevitable or addressable.

Some challenges with existing water quality CBAs are general. One problem is that many CBAs are undertaken before the regulation takes effect. This is precisely when we know the least about a policy and often requires relying on simulation models of the economy and the environment (37). Additionally, due to data limits or inadequate resources, ex ante CBAs can be incomplete and fail to account for some benefits or costs (3).

Another is data limitations that have limited the ability of researchers to create ex post analyses of policies’ realized costs and benefits (38–40). Harrington (41), for example, writes:

Thirty years (1972–2002) is certainly enough time to observe the effects of the Clean Water Act…. Unfortunately, these changes are very difficult to document systematically because the relevant data, when collected at all, are scattered in EPA regional offices, state DEQs [Departments of Environmental Quality], and POTWs [Publicly Owned Treatment Works].

Keiser and Shapiro (1) have complied 50 million water pollution readings from 240,000 monitoring sites over the period 1962–2001, which may help limit this constraint for future research. However, the range of available data on many aspects of surface water pollution is still poor, especially relative to data on other environmental goods. For ambient pollution, federal agencies use standardized methods to measure air pollution and weather hourly, daily, or weekly at thousands of US locations; measuring of water pollution is far less common, standardized, or centralized. For emissions, the largest air pollution sources have continuous emissions monitoring systems to record hourly emissions; water pollution emissions should be reported quarterly in a discharge monitoring report, but those emissions are self-reported and systematically suffer from nonreporting. For outcomes, data on health for studying air pollution are widely available at precise levels (e.g., individual birth certificate records); data on water-based recreation are far more limited. A large, high-frequency, national panel survey of recreation including residents’ precise home locations, destination locations, and choice attributes would help.

Important Limitations of Current Methodologies.

Additional weaknesses in existing CBAs are specific to IAMs and econometric methods.

IAMs.

One challenge is that many IAMs rely on results from studies that are not at the methodological frontier. For example, many water pollution analyses, especially for landmark regulations in the 1970s and 1980s, used rudimentary scientific models to project movement of pollutants through the environment. Scientific knowledge on how pollutants travel has advanced substantially, but these analyses have not been revisited.

Another example is the use of limited methods for valuation. The US EPA has used NWPCAM in prior CBAs, including its own retrospective assessment of the CWA. To go from pollution transport to human and environmental outcomes and values, this model uses individual analyses or metaanalyses of stated preference surveys that measure the economic value of water quality (7, 42). Although these studies adhered to best practices at the time, they do not necessary reflect current best practices such as consequentiality or incentive compatible mechanisms, both of which increase estimates’ validity. The primary reliance on stated as opposed to revealed preference is also a topic of discussion (43, 44). These studies also focus on recreational uses of water and may therefore omit important components of the value of water quality improvements.

Furthermore, many IAMs transfer cost and benefit estimates from very different settings than the regulation of interest (45). This is particularly problematic for water pollution regulations since the benefits of a water quality regulation vary with demographics, preferences, water flow, river networks, etc. The recent controversy over the WOTUS CBA (5) demonstrates how the CBA outcome can depend on decisions made in the transfer of values. The Trump Administration’s revision of this CBA excluded studies of wetland values from before the year 2000. Given the few recent studies, it assigned zero benefits to water quality improvements in wetlands. In general, the sign of the bias from transferring benefits is unclear.

Additionally, current IAMs also suffer from incomplete and uncertain links between water quality and changes in economic use and value (18, 19). For example, Keeler et al. (18) note uncertainties in the links between nutrient loading and commercial fishing, and the causes and consequences of harmful algal blooms. Furthermore, Keeler et al. note the difficulty in measuring nonuse values, such as “the intrinsic value of intact food webs or the cultural values associated with the existence of species or habitats.” If these links are completely missing for certain categories of benefits (or costs), these flaws in current IAMs will bias benefit (or costs) estimates downward. However, if the links are modeled, but uncertain, this uncertainty does not necessarily bias estimates in one clear direction. We believe missing links are likely the most important aspect of current IAMs that bias benefit estimates downward and discuss several important categories in Missing Categories of Benefits below.

Statistical and econometric approaches.

While IAMs have these weaknesses, econometric papers face their own challenges. Many econometric studies must establish whether the policy of interest actually caused a change in outcomes like water quality, or was merely associated with some unobserved variable like industrial activity or population growth that itself caused the change in outcomes. Randomized controlled trials would help solve this problem but are rare for water pollution for ethical and logistical reasons. Recent empirical work instead seeks to mimic the internal validity of a randomized controlled trial by exploiting variation in the location or timing of a policy’s activities. This approach is sometimes called a “quasi-experiment” or “natural experiment.”

Many current estimates of the effects of water quality policies come from cross-sectional ordinary least-squares research designs. One example is a recent assessment of how the CRP affected nitrogen and phosphorus emissions from agriculture (30). Sprague and Gronberg (30) find a perversely signed result—more area in conservation increases emissions of these pollutants. The authors provide several possible explanations, including unobserved variables such as the higher baseline level of agricultural activity in areas with higher conservation practices, which may lead to a spurious correlation between conservation efforts and pollution emissions. This kind of concern is common—areas that policymakers target for regulation may be more polluted, more densely populated, more politically connected, or differ in other ways that are hard to measure. For these reasons, comparisons of regulated versus unregulated areas risk inaccurately measuring the effects of regulation.

Recent research has begun using econometric approaches that help address these concerns. For example, Keiser (46) suggests that measurement error in pollution data may have biased prior statistical estimates of the impacts of water quality on recreation toward zero. An improved research design that corrects for this and other potential sources of bias suggests that the benefits of the CRP are potentially twice as large as its costs. This finding contrasts with Hansen’s (47) benefit-to-cost ratio of the CRP that falls below 1 (Table 1). In a different setting, however, Keiser and Shapiro (1) study the CWA’s large municipal grants program. The authors find benefits as measured with changes in the housing market that are substantially smaller than costs. In interpreting these findings, it is noteworthy that the CRP addresses a lightly controlled pollution source (agriculture), while the CWA requires fairly uniform and stringent upgrades in municipal wastewater treatment.

A second challenge is that econometric and statistical approaches may exclude general equilibrium changes. For example, many travel cost analyses examine the value individuals place on water quality at a particular site or a small group of sites. Many current empirical methods are well suited to recover benefit estimates of small changes in water quality at a particular site, but face more difficult challenges in recovering benefit estimates that arise from a policy that causes large and widespread changes to water quality and other economic conditions such as wages (48).

A third challenge is that econometric and statistical approaches may suffer bias if consumers have incomplete information about benefits of a water quality change. For example, hedonic analyses assume that housing values reflect implicit values that households place on a bundle of goods. This bundle includes both structural aspects of the property (i.e., number of bedrooms, square footage) as well as characteristics of the location (i.e., surface and drinking water quality, air quality, school quality, crime rates). If households are uninformed of the quality of nearby surface water or their drinking water, housing values will not properly reflect these values.

The use of more credible research designs has led to more robust estimates of the benefits of controlling air pollution (49, 50). Accounting for general equilibrium changes have yielded even higher estimates (51), and the impact of air and climate pollution on averting behavior have also proven important (52, 53). Studies that implement more precise research designs, however, have not always found large benefits of other environmental programs (54). These advances in research on air and climate pollution have not become common in research on surface water pollution.

Missing Categories of Benefits.

In addition to mismeasuring categories of benefits they cover, existing CBAs also exclude some important categories of benefits altogether. Health effects of surface water pollution via drinking water are one potentially important channel not in most CBAs. In EPA analyses of CWA regulations, health accounts for little or none of the total benefits, which reflects the EPA’s general practice of assigning zero benefits from especially uncertain channels (55). In EPA and academic analyses of air pollution regulation, by contrast, health can account for more than 95% of all benefits (56–58). Of course, most people breathe air without having it pass through a filter, while most people drink water that has passed through a drinking water treatment plant, so air pollution may create greater health damages than surface water pollution creates. However, CBAs typically assume that drinking water treatment is stringent enough to remove all pollution in surface waters to nonharmful levels before it enters drinking water systems.

Another potentially important excluded category is existence values. These values reflect the willingness to pay for clean water and aquatic ecosystems due to their pure existence and divorced from any specific uses. Their exclusion could significantly bias benefit estimates downward. For example, a stated preference survey of the damages from the Exxon Valdez spill including existence values yielded benefit estimates over a 1,000 times larger than a corresponding revealed preference survey (43, 44).

A third potentially important excluded category is nonstandard pollutants. Many current CBAs focus on conventional pollutants and define common water quality indicators such as dissolved oxygen, sediments, and nutrients, and mention the exclusion of benefits from reducing toxic and nonconventional pollutants (e.g., refs. 21 and 47). CBAs typically account for the costs of reducing these pollutants so should account for the benefits of doing so as well. Some evidence suggests that households value reducing toxic pollutants in water and air (59, 60). Given that little economic research studies these pollutants, the magnitude of their social costs is unknown.

Finally, many CBAs exclude certain types of resources. For example, several analyses of the CWA exclude benefits to coastal areas and also exclude interactions of surface water and groundwater. None of the CWA analyses in Table 1, for example, count benefits from groundwater, which are potentially important since groundwater contributes over a third of all water for public supply [source: US Geological Survey; https://water.usgs.gov/edu/wateruse-diagrams.html (accessed July 3, 2018)]. Impacts of surface water regulations on coastal areas may also be important since nearly 40% of the US population lives in coastal counties [source: National Oceanic and Atmospheric Administration; https://oceanservice.noaa.gov/facts/population.html (accessed July 3, 2018)]. Lyon and Farrow’s (8) analysis of the CWA assigns ∼11% of total benefits to saltwater recreational and commercial fishing, but many analyses exclude this category altogether (e.g., refs. 7 and 21). [In Lyon and Farrow (8), freshwater recreational benefits account for 72% of total benefits. Diversionary benefits (e.g., decreased drinking water treatment costs) account for 17% of total benefits.]

Costs.

We have emphasized mismeasurement of benefits, but the estimated costs of water pollution regulation may also be inaccurate, for at least three reasons.

One challenge involves measuring abatement costs for water pollution (61). The sign of this bias is unclear. The Bureau of Economic Analysis (62) has emphasized challenges including that many pollution abatement technologies generate valuable by-products; managers cannot easily distinguish which capital goods, materials, or workers are used for abatement versus production; managers cannot always easily identify which business decisions are environmental and how they affect production; and managers cannot easily distinguish expenditures on pollution abatement from expenditures for industrial safety and related purposes. In addition, it is difficult to estimate the effect of a new policy or regulation on the kind of innovation that decreases abatement costs. Requirements to meet new standards or profitability from trading also provide incentives for firms to innovate and find ways to lower costs of abatement.

These challenges appear for all environmental goods, but are arguably worse for water pollution. Several major air and climate pollution regulations use cap and trade markets, including the Acid Rain Program for sulfur dioxide, the Nitrogen Oxides (NOx) Budget Trading Program and the Regional Clean Air Incentives Market for NOx, the Regional Greenhouse Gas Initiative for carbon dioxide, and others. These markets make it easier to observe the marginal cost of abating pollution, since under common assumptions, the marginal abatement cost equals the market price of pollution allowances. Recovering the total abatement cost in these markets is more challenging. Because water pollution regulation does not generally use market-based instruments like taxes or emissions markets, however, it does not even reveal marginal abatement costs.

A second challenge in measuring costs involves accurately accounting for market power and the costs that regulation can create for consumers. This challenge leads existing estimates to understate the true economic costs of pollution. Standard estimates of the costs of abating water pollution involve engineering estimates of the cost to build and sell an abatement technology, or accounting estimates of the expenditure for a new abatement process. These accounting measures exclude market power and associated penalties to consumers. Many of the industries that emit substantial amounts of water pollution also have considerable market power, including iron, steel, cement, and electricity. The concentration in these industries implies that they decrease production to increase prices and exercise market power, and therefore that they produce less than they would if they accounted for the penalty of market power to consumers. However, the water pollution externalities in these industries imply that they produce more than they would if they accounted for the penalty of production to the environment. Pollution regulation can address the externality but also augment the exercise of market power by increasing production costs, and thereby create additional penalties to consumers.

The idea that concentration in output markets can increase the costs of pollution regulation has a long history (63). Recent empirical research has quantified its relevance for cement (64, 65) and a few individual industries like electricity, but does not have general estimates of its empirical relevance for many industries.

Another reason why accounting measures may understate the economic costs of pollution regulations involves the interactions of regulation with the tax system (66–69). Taxes create a wedge between the marginal value to a person of working and the marginal cost to a firm of hiring a worker. Environmental taxes and regulations increase the cost of producing goods and services, which increases their price to consumers. Hence such taxes effectively decrease real wages, which equal nominal wages divided by the price of final goods and services. By decreasing real wages, such regulations act like taxes on labor supply and so can further increase the deadweight loss due to income, sales, and payroll taxes. One can make similar arguments for capital and other factors of production.

Some estimates find that the magnitude of these tax interaction effects for climate change and energy taxes is 25–35% of the magnitude of direct abatement costs, although one should be cautious to generalize these numbers to other settings (70, 71).

In addition to these challenges in measuring costs, it is important to highlight that current water quality policies are not cost-effective (2). Water pollution regulation relies largely on “command and control” policies like effluent and technology standards. By contrast, air pollution regulation relies much more on market-based instruments like cap-and-trade systems. Such systems generally equate the marginal cost of abating pollution across sources, and hence cost less to achieve a given level of abatement. Moreover, while most sources of air pollution face stringent regulation, nonpoint sources of water pollution face little or no binding regulation. Failure to regulate a large polluting sector increases the cost to achieve a given level of abatement. If US policy moves toward more cost-effective solutions, abatement costs could correspondingly decrease (8, 20).

Discussion

Expenditures to clean up rivers, lakes, and other surface waters have exceeded the cost of investments to clean up air pollution and also have exceeded the costs of most other US environmental initiatives. Research has found that many of these expenditures have decreased water pollution and has suggested ways to make these investments more effective.

A majority of analyses, however, find that these investments’ benefits are less than their costs. This includes studies by the EPA, private consultants, and academics; using revealed or stated preference methods; applying IAMs or econometric methods; and from papers covering an over 20-y period. This is not the case for most environmental goods, such as air and climate pollution. Are the benefits of these investments truly less than their costs, or are available estimates of costs and benefits biased?

We conclude that available estimates of the costs and benefits of water pollution control programs are incomplete and do not conclusively determine the net benefits of surface water quality. At the same time, we argue that this uncertainty is not inevitable, and that targeted research in areas we outline could be sufficient to resolve the uncertainty. Existing estimates of the benefits of surface water quality may be biased downward due to the exclusion of missing services like health impacts, missing pollutants like toxics, and missing resources like impacts on coastal areas and surface–groundwater interactions. Research that estimates true economic costs of controlling pollution will also help gain a more precise measure of abatement costs, although the sign of the bias in current estimates is unclear.

Economic research over the last two decades has made great strides in understanding the impacts of air pollution and climate on a wide range of outcomes, including health, housing, and labor productivity. Similar research could provide important guidance on US surface water quality regulation.