Measuring the social benefits of water quality improvements to support regulatory objectives: Progress and future directions

The U.S. Environmental Protection Agency (EPA) has undertaken dozens of regulatory impact analyses (RIAs) and benefit–cost studies related to the objectives of the Clean Water Act (CWA). Originating with President Reagan’s Executive Order 12291 in 1981, and supported through subsequent Executive Orders, the EPA is required to undertake these analyses for all major rules, with the objective of maximizing net benefits and avoiding regulations where the costs to the society do not justify the benefits.^* Because the environmental research field of nonmarket valuation was nascent when the early Executive Orders were initiated, leadership at the EPA committed funds and expertise to support original research by economists, usually in collaboration with scientists from other disciplines. One goal of these efforts was to provide estimates of monetary value for water quality improvements to inform regulatory analysis.

Estimating economic benefits in this setting requires the use of nonmarket valuation techniques, as most water quality and related aquatic ecosystem services are not commodities that are bought and sold directly in markets. Sources of benefits tied to the direct use of water bodies, such as fishing in a river or living near a lake, can be inferred from the expenditures people make to use the resource. But, benefits can accrue that are unrelated to use. For example, people may value water quality because it is beneficial to other species, to preserve the resource for future generations, or out of a sense of stewardship for the environment. As these nonuse values cannot be inferred from observed behavior in markets or related settings, economists have devised survey methods to measure the public’s willingness to pay (WTP) for improved environmental quality. These stated preference methods conceptually capture both use and nonuse values. Directed by the Executive Orders to quantify all sources of value possible, the EPA has played an important role in supporting research that measures the benefits of water quality improvements.

Notable early studies provided foundational methods and empirical estimates to support federal regulatory design for water resources at EPA and other federal agencies (e.g., refs. 1–3). Despite the methodologies and findings generated by these studies and a related body of economics research, the funding and knowledge base for water quality valuation has not kept up with new and evolving needs of the Agency (4–6). This has resulted in EPA economic analyses with categories of water quality benefits that are either left unquantified (e.g., nonuse values) or estimated using arguably obsolete methods and data (7, 8).

To fill these critical knowledge gaps, the EPA issued a request for applications (RFA) in 2015 to provide financial support for interdisciplinary nonmarket valuation studies.^† The basis of this RFA was to improve estimates of benefits across multiple types of water resources including, “…the Nation’s inland fresh water small streams, lakes and rivers, estuaries, coastal waters, and the Great Lakes.” The Agency identified three research areas that could prove fruitful for future benefit estimates. The first of these is to understand how individuals perceive and value changes in water quality. For instance, how do individuals value the attainment of a water quality standard versus a comparable change in water quality that does not cross a defined threshold? The second pertains to the spatial scale and scope of water quality benefits and sought interdisciplinary research that could strengthen the link between hydrological and ecological models and end points that affect human welfare. Third, this RFA provided support for methodological advancements, such as new ways to communicate changes in environmental quality within stated-preference questionnaires, to improve the estimation of both use and nonuse sources of value.

In response to the RFA, EPA awarded $4.8 million in funding to six multi-institutional teams.^‡ To increase the quality and collective impact of the studies and enhance their ability to inform local and national decisions on water quality, the EPA encouraged sharing of methodological advances across the teams and promoted joint efforts to address research challenges. This resulted in a tightly integrated body of valuation research. The set of studies, together, will support the next generation of water quality valuation methods at EPA. The articles in this PNAS symposium provide key results from studies funded under the RFA by academic researchers and closely related original research by EPA analysts and scientists.

Historical Perspective on EPA-Funded Valuation Research

Although there are multiple ways for the agency to support economic research, the 2015 RFA was issued under the EPA’s most extensive grant program, the Science to Achieve Results (STAR) program. Established in 1995, the STAR program funds individual research projects in six primary research areas: Safer chemicals, air, ecosystem, health, sustainability, and water. Each year, the program receives 2,000 to 2,500 grant proposals and to date EPA has awarded more than 7,600 research grants under a competitive solicitation process. Project abstracts and annual progress reports are posted on the program’s website^§, along with descriptions and links to presentations and journal articles on the research.

Water research grants support inquiry on a broad range of water-related problems. Since its inception, the STAR program has issued over $350 million in grants on drinking water, water quality, water treatment, and other water-related issues. Grants for water-related economic valuation are a subset of this total and come from RFAs for broad methodological valuation work. Since 1996, the bulk of water-related economic research has come from four RFA titles^¶: Decision-Making and Valuation for Environmental Policy (1996 to 2001), Valuation for Environmental Policy (1995; 2003 to 2005), Methodological Advances in Benefit Transfer Methods (2006), and Water Quality Benefits (2015). Of the $18.6 million in grants distributed under these RFAs, 34 grants and $10.5 million were for water research, although not all were directly related to surface water. These grants produced a total of 293 publications and presentations, which included 66 journal articles. In addition, eight other grants under these RFAs featured methodological research that applies to water issues. To highlight progress and results associated with this grant-funded research, EPA has held over a dozen public workshops for grant recipients to present their results. Several have been on the topic of nonmarket valuation, including some on ecosystem valuation for which water quality is often a central focus. In particular, the workshops for the 2015 grant awardees facilitated the cooperation evident from this symposium.

The use of economic research in EPA regulatory analysis has evolved over time. Quantitative analysis in the 1980s and 1990s consisted of case studies, along with simple benefit transfer approaches that use the results from one completed study and adapt them to another policy application.^# Benefit transfers are commonplace in regulatory analysis, as primary data collection for nonmarket valuation is often a years-long effort that does not fit within the time and resource constraints faced by regulatory agencies. Results from Mitchell and Carson (2) were used to estimate recreation and nonuse benefits in the analysis of the 1987 Organic Chemicals, Plastics, and Synthetic Fibers Effluent Guidelines based on a unit transfer approach. The same study later supplied estimates for three subsequent regulations through 2004 using a benefit function transfer. When regulations were expected to remove consumption advisories from contaminated fish or similar toxic contamination, Lyke (9) was the primary source for benefit transfer beginning with the Great Lakes Water Quality Guidance of 1995.

Since 2009, the EPA’s valuation of surface water improvements has shifted away from unit and function transfers that rely on a single study to meta-analyses that allow multiple studies and context-specific factors to estimate the values used in benefits analysis. EPA’s meta-analytic approach was first developed in the study by Johnston et al. (11) with two updated versions used in the 2009 Construction and Development Effluent Guidelines and Standards and the 2015 Steam Electric Power Generating Effluent Limitations Guidelines and Standards. The approach has been refined over time, with Johnston et al. relying on 34 stated preference studies, the 2009 analysis utilizing 45 studies, and the 2015 analysis using 51 studies. As one indication of impact, four journal articles included in the meta-analyses were funded by EPA grants, three were developed under a cooperative agreement with the EPA, and one was created as a white paper with partial EPA funding.^||

Of course, the influence of EPA-funded projects goes beyond accumulating data points to use in benefit transfers. EPA has supported dozens of studies to improve benefit transfer practices and sponsored several workshops for authors to present their results. The first was held in 1992 and was jointly sponsored with National Oceanic and Atmospheric Administration and US Department of Agriculture (12). The papers presented at this workshop were instrumental in identifying the most pressing issues in benefit transfer and establishing protocols to improve the reliability of results given the state of the literature. In 2005, EPA’s National Center for Environmental Economics and Environment Canada sponsored a second workshop on benefit transfer (13). Several studies presented at this workshop would influence how EPA estimates benefits of environmental quality including functional transfers and incorporating prior beliefs in a Bayesian framework. Most recently, EPA commissioned five papers to advance the conceptual and empirical methods used in benefit transfer. The authors presented their findings at a workshop in 2016 and published the proceedings in a special issue of Environmental and Resource Economics (14).

EPA’s support for methods development in nonmarket valuation and benefit transfer has been integral to advances in these areas of research (see Fig. 1 for more examples). Next, we turn to contemporary work on the frontiers of nonmarket valuation, and stated preference research in particular, that will further improve the Agency’s capability to estimate economic benefits of water quality improvements.

Fig. 1.

Examples of EPA-supported research that led to methodological advances in nonmarket valuation and improvements in regulatory impact analysis performed by EPA.

Research Progress and Priorities

The literature on stated preference valuation is large, with separate areas of research focused on methodological development, best practices, assessing validity and reliability, and applications (e.g., refs. 15 and 16). Work specific to water quality is likewise substantial. Rather than providing a comprehensive review, in this section, we discuss three areas of research that were emphasized in the 2015 STAR Grant RFA and provide context for the papers featured in this symposium. We specifically examine commodity definition, which includes how to link water quality improvements to economic benefits. Next, we explore the spatial aspects that are inherent in estimating water quality benefits. Finally, we note three related areas of methodological development that are emphasized in the EPA-funded projects.

Commodity Definition: Services, Metrics, Magnitude.

Stated preference applications require a careful definition of the environmental goods and services to be valued to ensure a consistent interpretation across respondents and allow quantitative assessments by analysts. Although not unique to stated preference, the literature offers no standard approach for defining the commodity for valuation (17). As a result, water quality applications have used a variety of approaches to commodity definition (18). We define and discuss three overlapping elements that together determine the commodity of interest.

The first element is identifying the type of services provided by water quality. Water quality can affect human well-being through direct use mechanisms such as outdoor recreation opportunities. Many studies estimate the economic value of water quality based on its contribution to activities such as fishing, swimming, hiking, picnicking, wildlife photography, and boating as well as indirectly through habitat quality, wildlife populations, esthetics, cultural values, and mental health benefits of natural spaces. Surface water quality can also affect human well-being through less tangible mechanisms, such as when a person holds nonuse values for the ecological conditions in waterbodies they do not interact with directly (19).

The second element is the metric or metrics used to define and communicate water quality. The challenge is to accurately translate a change in waterbody condition into services respondents can understand and that might affect their well-being. Conceptually, the choice of metric and the specific way it is described to survey respondents should be related to the services to be valued in the study. For example, if outdoor recreation is the focus, an approach is needed that communicates relationships between surface water quality and recreation activities (e.g., presence of harmful algal blooms for swimming or consumption advisories for fishing). Similarly, for combinations of services, a metric that conveys varying levels of ecological conditions may be appropriate. Conditional on these needs, researchers have investigated a range of approaches to communicate water quality changes within stated preference surveys.

Previous reviews of water quality metrics vary in their categorization, but typically identify qualitative descriptions, biophysical measures, and simple or complex indices (5, 15, 20). The simplest approach is to describe different water quality outcomes using ordinal descriptors such as “poor,” “fair,” “good,” and “excellent.” While these terms are relatively easy for respondents to understand, this approach requires people to assign specific meaning to the general labels. As a result, valuation requires assumptions about how respondents performed that mapping between specific environmental attributes and qualitative descriptions. To avoid this, some studies have used biophysical measures such as water clarity, dissolved oxygen levels, or nutrient or chlorophyll a concentration. For water clarity, changes in measures such as Secchi depth or turbidity may directly affect individuals’ well-being through impacts on recreational swimming (21). In most cases, however, biophysical measures are proxies for the actual end points that affect valuations (17). For example, anglers value fish populations that are supported by high oxygen levels, rather than the oxygen levels per se. As a result, when biophysical measures are used as quality metrics, survey respondents must both understand the technical aspects of the measures and mentally link them to the actual water quality outcomes that determine their economic value (20).

Connecting chemical and physical measures to specific outcomes can help illustrate how water quality supports different services. For example, states establish designated uses for surface water and the levels of water quality measures that indicate when the use is supported. They also issue fish consumption advisories and beach closures when thresholds for criteria pollutants are exceeded, and these categorizations can serve as valuation end points. Siikamäki and Larson (22) provide an example using water bodies in California that do not meet standards for their designated use as the metric for valuing water quality. Such water body categorizations can be readily understood by respondents, but using them to estimate values for improvements that do not cross a threshold requires strong assumptions about people’s preferences for water quality.

Alternatively, researchers have developed indices that aggregate information on water quality levels into unidimensional “ladders,” which are communicated to respondents via graphics and linked to services (Fig. 2). The best known of these is the “RFF water quality ladder” for recreation services (23) which is a transformation of the commonly used National Sanitation Foundation’s Water Quality Index (WQI) (24). Via this ladder, the researchers defined a hierarchy of recreation uses that are supported by higher levels of water quality. Despite its frequent use by practitioners and effectiveness for recreation services, the original ladder does not explicitly connect to other ecosystem services—many of which may be poorly represented by a use-focused hierarchy.

Fig. 2.

Graphical representations of water quality used on stated preference surveys (25–27).

Recent research has adapted the ladder concept in two promising ways. First, researchers have sought indices that provide a broader characterization of conditions and are grounded in ecological concepts (17, 18). A key feature has been to research how different levels of ecological integrity correspond to observable features of waterbodies, which are then communicated to lay audiences as ordinal levels (e.g., refs. 25, 26, 28). This has allowed analysts to consider value-generating mechanisms that are tied to holistic representations of environmental outcomes, with which nonuse values can be more credibly estimated. Second, researchers have codeveloped their indices with auxiliary tools that allow direct links between the water quality levels described in stated preference surveys and measurable outcomes in actual waterbodies (29, 30).

The final element in commodity definition relates to the scale of change. Stated preference surveys are framed around specific water resources such as a river system, lakes in a region, or all surface water in an administrative unit or geographical area. Respondents typically consider policy alternatives that move reference waterbodies from baseline quality to a new quality. The scale dimension defines how many units are affected by the policy. Conveying the quantity of surface water that would be improved is a challenge when those waters may include small streams, rivers, lakes, reservoirs, and estuaries. This element plays an important role in how people value changes in water quality (19) and relates closely to the spatial issues that we consider next.

Spatial Issues.

Geographical and spatial dimensions are important considerations in the design, implementation, and interpretation of stated preference surveys (31). Spatial dimensions play a critical role in defining both the scale and scope of the water quality and policy outcomes being valued and for identifying the population that may benefit from the proposed changes. Indeed, the role of spatial dimensions in environmental preferences was initially motivated by the problem of defining the “extent of the market” when aggregating estimated values estimated at the individual or household level.

Economic theory and empirical evidence provide support for the idea that spatial dimensions such as distance from the valued resource, the quantity of the commodity affected within varying distances, and proximity to available substitute and complementary resources may influence the value of the resource in particular study contexts (19, 32). The evidence supporting distance-based variation in preferences is robust for use values, while the evidence is mixed for nonuse values (33). For instance, with increasing distance, it is generally more costly for people to visit a particular resource, there may be a wider variety of nearby alternatives, or people may have less knowledge of or feel less responsible for more distant resources.

While the most common method for characterizing spatial heterogeneity is distance-based, there is increasing evidence that preferences for environmental goods may occur in patchy, spatial clusters (34). These spatial patterns may arise from people with similar preferences moving to the same location or from the influence of the surrounding environment in shaping preferences for environmental goods (35).

Finally, it is important to consider spatial dimensions when the results are used for benefit transfer. The extent to which spatial patterns estimated in one study context are valid for other sites or regions remains a topic to be explored. If value estimates vary across space, this potentially compromises reliability and the transportability of the values to another location.

Other Methodological Considerations.

Issues of commodity definition and space relate to methodological issues that have received recent attention in stated preference research. In the study of water quality, the notion of an ecological production function—the relationship between ecological indicators and the quantity or quality of value-generating services—is important because it allows an analyst to map changes in the physical environment to changes in economic benefits. However, translating this theoretical idea into practice has proven challenging (5). Von Haefen et al. (30) make progress on this front by estimating three ecological production functions, based on expert elicitations, that predict changes in ecosystem condition, human health risk, and murky water days. They then use stated preference data to estimate values associated with predicted changes. There remains much to learn about how such applications can be generalized to value a broader set of end points over larger geographic areas.

Recent research has focused on integrating ecological, economic, and hydrological modeling to enable systems-based estimation of how specific pollution reduction policies affect the distribution of ambient water quality changes in space, how this translates to changes in ecosystem services, and ultimately the economic costs or benefits from these changes. These integrated assessment models (IAMs) require coordinated progress across complementary disciplines to succeed. Corona et al. (4) describe the development of the HAWQS-BenSPLASH IAM that combines a water quantity and quality modeling system with a valuation module that estimates household WTP for improvements to waterbodies within a specified radius. Additional research on how hydrological, ecological, and economic models can be integrated to inform environmental policy will help to advance such efforts and improve regulatory analysis.

Persistent Challenges and Emerging Issues

While the decades of nonmarket valuation research and the ongoing efforts discussed above have advanced EPA’s ability to estimate the benefits of improving water quality, challenges remain that require new research. Among them are persistent challenges that arise from the diversity and breadth of aquatic resources under the EPA’s jurisdiction. Regulatory analysis occasionally requires valuing changes to resources for which there are no studies suitable for benefit transfer. For example, there is anecdotal evidence that, even after adjusting for physical characteristics, people value the environmental quality of iconic resources differently than natural assets that do not possess the same name recognition and historical significance (e.g., refs. 36–38). Empirical studies of the apparent “iconic premium”, and case studies demonstrating how to incorporate such a valuation differential into regulatory analysis, would aid the EPA in fully and accurately capturing the social benefits of environmental regulation.

Regulations passed under the CWA can affect large regions of the United States but have relatively small impacts on water quality. For example, the 2015 Steam Electric Power Generating Effluent Limitation Guidelines were expected to improve water quality in nearly 20,000 miles of rivers and streams, with changes for about 99% of these miles being less than one point on the 100-point WQI (39). Nearly all the peer-reviewed studies valuing surface water quality that are suitable for benefit transfer examine values associated with large changes in water quality over comparatively small geographic areas. Transferring benefits from peer-reviewed studies to policy cases involves implicit tradeoffs between the size of the improvement and the amount of water affected. Given the tendency for valuation studies to focus on small areas, there is little or no empirical basis to estimate that relationship. Studies that examine smaller changes in quality over larger geographic areas would improve the reliability of benefit transfers by providing values derived from more policy-relevant scenarios.

An extension of the ongoing efforts to analyze spatial dimensions of stated preference valuation is research on the extent of the market for nonuse values. There is general agreement among environmental economists that nonuse values should be included in policy analysis (40) (41) and, indeed, both Office of Management and Budget (42) and EPA (43) have issued guidance regarding how they should be treated when estimating benefits. While there is theoretical and empirical support for distance decay of use values, evidence of distance effects on total value is ambiguous, which raises the question of which households should be assumed to value a resource they do not plan to visit when estimating total benefits. Vossler et al. (25) provide some relevant evidence with respect to water quality, and in particular find that total value for an improvement in a particular area depends on the fraction of the area that occurs within one’s state of residence. They also find that on average people are willing to pay substantial amounts for policies that would only impact water quality hundreds of miles away, but do not identify any clear boundaries beyond which values approach zero. Additional studies are needed to better understand the extent of the market and how water quality improvements vary with distance from the affected resource.

In addition to these persistent challenges, there are emerging issues that EPA will face in coming years that require new research to address. There is an implicit assumption in most regulatory analyses that baseline environmental conditions are static. Changing weather patterns brought on by climate change amplify the biases created by this assumption. Water temperature and flow have significant impacts on ecological condition in rivers, lakes, and streams. As EPA acts to reduce pollution in aquatic ecosystems, shifting baselines and synergistic effects of climate conditions and pollutant levels may impact the outcomes of regulatory analyses substantially. Research on the effects of a changing climate on aquatic resources and the interactions with traditional pollutants will help EPA prepare for a shifting regulatory landscape.

A complicating feature that climate impacts share with some other environmental threats is irreversibility. Because of atmospheric and climate dynamics, climate change is likely to continue for a millennium even after a cessation of greenhouse gas emissions (44). Habitat loss, species extinction, and the release of persistent chemicals (e.g., polyfluoroalkyl substances) into our waterways will impact social welfare and human health for generations. Drawing upon a large literature that documents the need to incorporate uncertainty and irreversibility in benefit assessment (45, 46), Sunstein (47) reiterates that an “irreversibility premium” may be appropriate in benefit–cost analysis when that which is permanently lost (or at least for relevant timescales) is unique or has few substitutes. In some cases, such as species extinction, irreversibility is inherent to the scenario and WTP estimates will conceptually include a premium that people hold for such losses. In others, particularly when benefit transfer is used, the permanence of the loss is ambiguous, and estimates may not capture the full magnitude of societal losses. As EPA addresses more irreversible impacts, additional research on when and how an irreversibility premium should be applied to benefit–cost analysis will be valuable to the agency.

The surging availability of “big data” presents challenges and opportunities for new nonmarket valuation applications. Big data is characterized by high-volume and high-frequency observations that provide a spatial and temporal resolution that is unachievable with other sources. The sources of big data, however, were not created for social science research and thus require complementary data and careful filtering to generate useful results. Remotely sensed data, from satellite and aircraft, can be used to infer several water quality parameters such as clarity, chlorophyll a, total suspended sediments, and temperature (48). Two recently published studies use variation in near-lake property values to derive welfare estimates from changes in water clarity (49) and the frequency of harmful algae blooms (50) detected using remotely sensed data.

To be useful for nonmarket valuation, information on the physical characteristics of our environment must be complemented with behavioral and demographic data. Mobile phone networks and social media are two data sources that can provide the behavioral link, and both have been used in recreation demand studies. Kubo et al. (51) utilize mobile phone network data to value coastal tourism, but do not include water quality parameters in their analysis. Keeler et al. (52) use geotagged photos posted by lake visitors to the Flickr photo-sharing website to value water clarity using a recreational demand approach. While these studies are instrumental in demonstrating the application of big data to nonmarket valuation, the greatest benefit to EPA would be a flexible tool capable of utilizing the continually updated information these sources provide and capturing the dynamic nature of our physical environment and people’s preferences.

New Advances and Next Steps

The set of studies presented in this symposium represent valuable advances in the state-of-the-art of benefit estimation for water quality improvements. These improvements will allow EPA and other federal agencies to perform regulatory analysis with improved accuracy, leading to a superior understanding of the economic efficiency tradeoffs inherent to regulatory decisions. Previous RIAs have struggled with monetizing the full breadth of services from changes in water quality, the spatial extent of those benefits, and predictive modeling of ecosystem service provision under baseline and policy scenarios.

The studies contained in this symposium make important strides in filling those gaps. Vossler et al. (25) provide insight into the extent of the market for water quality improvements for rivers and streams while developing a new, more ecologically inclusive approach to communicate changes in water quality and integrally related ecological integrity. Johnston et al. (53) use interactive mapping of current and predicted water quality indicators in a stated preference survey to further explore spatial aspects of households’ WTP for improvements in the suitability of water for human uses and to support aquatic life. Using a split sample study design, Lupi et al. (54) find that aggregating indicators for water contact recreation and fish abundance into a single metric leads to lower value estimates than when the indicators are presented to respondents separately. Similarly, Hill et al. (55) develop methods to estimate use values and existence values in a separable way to allow future exploration of how each varies with respect to distance from the improved resource. Von Haefen et al. (30) examine the impact of “urban stream syndrome” in an ecological production function framework to value improvements in ecosystem condition, human health risk, and water clarity.

While these advances will improve economists’ ability to value water quality changes, and will have positive spillovers to other applications, taking full advantage of the remarkable breadth and scale of new data sources available to us requires yet more work. With constrained research budgets limiting original data collection, remotely sensed data, social media information, and cell phone data can provide high-quality and current information to inform policy choices. Another recent development that several papers in this symposium leverage is Internet-based surveys, which have the potential to lower costs and shorten data collection times. Questions surrounding sample representativeness and data quality remain, however, and answering them will require more research to assess and improve the reliability of electronic surveys. Finally, some environmental data presented in these papers can only be collected through in-situ measurement and observation (e.g., EPA’s National Aquatic Resources Survey). Constraints on the public resources allocated to these collection efforts have created long lags and gaps in their coverage. However, similar data are often collected through citizen science projects and environmental assessments that precede water withdrawal and development decisions. Identifying methods and datasets that can supplement national data collection efforts (e.g., via web scraping) could provide more timely measurements with improved spatial coverage.