The Environmental Benefits Mapping and Analysis Program – Community Edition (BenMAP–CE): A tool to estimate the health and economic benefits of reducing air pollution

Abstract

A number of software tools exist to estimate the health and economic impacts associated with air quality changes. Over the past 15 years, the U.S. Environmental Protection Agency and its partners invested substantial time and resources in developing the Environmental Benefits Mapping and Analysis Program – Community Edition (BenMAP–CE). BenMAP–CE is a publicly available, PC-based open source software program that can be configured to conduct health impact assessments to inform air quality policies anywhere in the world. The developers coded the platform in C# and made the source code available in GitHub, with the goal of building a collaborative relationship with programmers with expertise in other environmental modeling programs. The team recently improved the BenMAP–CE user experience and incorporated new features, while also building a cadre of analysts and BenMAP–CE training instructors in Latin America and Southeast Asia.

1. Introduction

Risk assessors, policy analysts and policy makers have long relied upon decision-support tools to assess the human health impacts of air pollution (Fann et al., 2012; Pascal et al., 2013; Guttikunda and Khaliquzzaman, 2014; Viana et al., 2015; U.S. EPA, 2009; Boldo et al., 2014). While these tools vary in complexity, sophistication, and installed base (i.e., number of users), they share a core attribute: each software program draws upon evidence reported in the air pollution epidemiology literature to calculate estimated cases of air quality-related adverse health impacts (Anenberg et al., 2016). As compared to ad-hoc solutions such as spreadsheets or statistical programs like SAS or R, these programs can be more time-efficient, transparent and reliable. As such, these programs are generally designed for a multi-disciplinary audience, feature a graphical user interface (GUI), and include some (or, in certain cases, all) of the data needed to quantify the estimated number, and often the economic value, of air pollution-related deaths and illnesses (Anenberg et al., 2016).

Over the past decade, the number of these types of tools has proliferated–due in part to the growing body of epidemiologic evidence that provides the quantitative parameters of the air pollution – health effect concentration-response relationship, as well as the increased interest among decision makers to inform public health policies by conveying the potential estimated benefits of improved air quality (Samet, 2009; Burnett et al., 2014; HEI, 2003). While carefully evaluating the features and design of each tool is beyond the scope of this manuscript, it is worth noting that these programs exist along a spectrum of complexity. For example, programs like the World Health Organization’s AirQ and Aphekom (Improving Knowledge and Communication þfor Decision Making on Air Pollution and Health in Europe) are intended to be accessible to a broad class of users and make it quite easy to answer a defined set of policy questions related to city-level impacts (Pascal et al., 2013; Goudarzi et al., 2012).

By contrast, the program that is the focus of this manuscript–the Environmental Benefits Mapping and Analysis Program–Community Edition (BenMAP–CE) is a PC-based and open-source software platform designed for flexibility to perform a broad array of analyses at the local, regional, national and global scale. Below we describe the history of the BenMAP–CE software, its capabilities, and demonstrate its use through a case study.

2. Background

The approach for calculating the estimated benefits of improving air quality is well established, highly structured, and draws upon information from a number of disparate datasets, which has allowed for the development of decision-support tools that inform air quality policy decisions (NRC, 1983; EPA COUNCIL, 2010). Tools that quantify the human health impacts of air quality generally rely on four key pieces of information: (1) air quality data, (2) population data, (3) baseline rates of death or disease, and (4) a risk estimate (generally the coefficient from a statistical model that measures the response of a health effect for a one-unit change in an air pollutant concentration (e.g., per μg/m³), which we refer to as a beta [β] coefficient) from an air pollution epidemiologic study that quantitatively characterizes the relationship between air pollution exposure and health effects. The formula for calculating an air pollution-related health impact is referred to as a health impact function (HIF). The functional form of the HIF is based on the statistical approach used in the epidemiologic study from which the beta coefficient was obtained (most often a log-linear statistical model), resulting in a HIF most commonly defined as (Eq. (1)):

$$\mathrm{\Delta }Y=(1-e^{-\beta \ast \Delta AQ})\ast Yo\ast \mathit{Pop}$$ (1)

Where ΔY = the estimated health impact attributed to air pollution, β = the beta coefficient from an epidemiologic study, ΔAQ = defined change in air quality, Yo = baseline rate (i.e., incidence) for the health effect of interest, Pop = population exposed to air pollution. Users may calculate this function once at a national or regional scale, or may instead calculate it across multiple locations (like U.S. counties) and then sum the results. The economic value of air pollution-related cases of death and disease are quantified using either willingness-to-pay (WTP) or cost-of-illness (COI) estimates corresponding to each health outcome. These dollar unit values are multiplied by the estimated count of adverse health outcomes to yield a total economic value of the change in air quality.

The U.S. Environmental Protection Agency (EPA) began developing, applying, and deploying tools to support its risk and benefits analyses in the mid-1990’s, when it first quantified the benefits of air quality policies resulting from the recently enacted Clean Air Act Amendments of 1990 (U.S. EPA, 1999). The tool the Agency initially used to quantify air pollution-related health impacts and economic benefits was called the Criteria Air Pollutant Modeling System (CAPMS) (Abt, 2000). The CAPMS tool featured a GUI and a static array of population data, baseline rates of death and disease, and beta coefficients preloaded into a database. Additionally, it was often challenging to load air quality data into CAPMS. Due in part to these limitations, the Agency transitioned to the Environmental Benefits Mapping and Analysis Program (BenMAP) in 2003 (Davidson et al., 2007).

In contrast to CAPMS, BenMAP enabled users to add and remove data, including air quality, population data, baseline rates of death and disease, and HIFs, from the tool, which it stored in the Microsoft SQL Server Data Engine (MSDE). The BenMAP tool was originally written in Delphi and included a basic Geographic Information System (GIS) that was used both to perform calculations involving data stored at varying spatial scales and display geospatial results. BenMAP also allowed users to report an audit trail, detailing the user’s analytical choices and data inputs; this feature was critical for analyses supporting environmental policies, for which transparency and reproducibility were particularly important. Between 2003 and 2012 the Agency updated the tool regularly to include new air pollution data, additional beta coefficients from recently published epidemiologic studies, and economic value estimates.

Researchers applied the initial version of the program extensively to quantify the burden of air pollution and the economic value associated with improving air quality (Viana et al., 2015; Boldo et al., 2014; Kheirbek et al., 2014; Hubbell et al., 2005; Berman et al., 2012). Likewise, the Agency used the tool when predicting the health benefits associated with attaining more stringent National Ambient Air Quality Standards (NAAQS) for the criteria air pollutants (e.g., particulate matter [PM], ozone [O₃], nitrogen dioxide [NO₂], and sulfur dioxide [SO₂]) (U.S. EPA, 2012; U.S. EPA, 2015a), as well as important regulations that reduced emissions of the precursors to fine particulate matter (PM_2.5) and ozone including NO_x, VOCs, and SO₂ (U.S. EPA, 2011a; U.S. EPA, 2011b). However, both the CAPMS and BenMAP tools were limited by the fact that their source code was proprietary. This feature made the program more challenging and more resource intensive to develop, as it required the Agency to contract with a single environmental consulting firm to maintain and improve the software. Moreover, the proprietary code inhibited EPA from building a user community around the tool.

Beginning in 2012, the Agency began building an entirely new version of BenMAP into an open source framework that shared none of the code with the old version, but had the same functionality and produced the same results as the original version. The Agency aimed to achieve multiple goals. First, the source code would be freely available to the user community. Second, the new version of the program would serve two primary user communities: computer programmers and researchers/policy analysts. Additionally, the open source aspect of the updated tool would foster transparency both throughout the research community as well as with the broader public in terms of Agency analyses that used the tool. This new version of the tool was publicly available in March 2015 and relabeled the Environmental Benefits Mapping and Analysis Program – Community Edition (BenMAP–CE) to convey these new objectives.

In addition to conducting analyses using the full capabilities of BenMAP–CE, there is a module within the tool that relies on the underlying data from the Global Burden of Disease (GBD) study (Cohen et al., 2017). The module, referred to as the GBD Rollback tool since it allows the user to “rollback” or reduce air pollution concentrations using different algorithms, is mostly used by international users to convey the potential public health benefit of improving air quality. This is because the health and air quality data needed to conduct a full scale BenMAP – CE analysis is often difficult to obtain in many countries. Using air quality information obtained from the GBD study, the GBD Rollback tool allows users to easily estimate the number of premature mortalities avoided for any country or region of the world using a predefined non-linear HIF that accounts for the full range of PM_2.5 concentrations observed throughout the world, and can in turn be used to inform the development of a more comprehensive analysis using policy-specific air quality modeling data. Similar to the original version of the tool, BenMAP–CE has been used broadly by the research community and the Agency, and with the modifications to the tool that have occurred over time, other entities as well, both domestic and international. The Agency has supported rollout of BenMAP–CE with webinar presentations, multiday training programs, and an online user forum which meets quarterly to share information about updates to the software and user experiences in applying BenMAP–CE.

3. Software development and data requirements

By way of developing BenMAP–CE into an open source software platform the Agency has progressed towards its long-term goal of making the program sustainable. The following sections outline the features of the tool most relevant to the two audiences critical to the long-term viability of the software: developers (i.e., computer programmers) and end-users (i.e., researchers/policy analysts). We begin first with the programming community.

3.1. Computer programmer

BenMAP–CE was programmed using C# in a .NET 4 framework. It uses Firebird as the relational database and DotSpatial for spatial analysis and mapping. DotSpatial has been previously used in other applications (Sci et al., 2014; Zhao and Liu, 2018). Fig. 1 provides an overview of BenMAP–CE’s system architecture.

Fig. 1.

Overview of the system structure of BenMAP–CE.

The software incorporates numerous open source Dynamic Link Libraries (DLLs), a subset of which is summarized below (Table 1). The development team ensured that each of the data libraries conformed to one or more of the following open source licenses: GNU Library General Public License; Microsoft Public License; The MIT License; Initial Developer’s Public License; Apache License.

Table 1.

List of the Dynamic Link Libraries used in BenMAP–CE.

Software Components	Description
Firebird SQL Server v2.5.3.26780	Firebird relational database
Firebird ADO.NET Data Provider 4.1.0.0	Provides access to Firebird database
DotSpatial v2.0	Geographic information system
GDAL 1.11.1	Raster geospatial data translator
protobuf-net 2.0.0.622	Data serialization
Meta.Numerics 2.1.0	Math and statistics
Troschuetz.Random 1.4.0.0	Random number generator
Oxyplot 1.0.0.0	Graphs
ObjectListView v2.5.0.0	Drag-and-drop tables
Open XML SDK 2.5	Open XML package manipulation
Excel Data Reader 2.1	Excel file reader
LumenWorks.Framework.IO CSV Reader 3.8.0	CSV file reader
SharpZipLib 0.85.5	Compression library
RestSharp 104.4.0.0	REST and HTTP API
Newtonsoft.Json 3.5.0.0	JSON framework
DataWorker 1.0.06	Client-side data management

There are three Firebird databases packaged with, and used by, BenMAP–CE. One that supports the program’s core functionality and two that support embedded tools:

• BENMAP50.FDB – This is the primary database supporting the core functionality of BenMAP–CE. It stores information about setups, grid definitions (i.e., the spatial domain over which various data inputs are available), pollutants, monitor datasets, population demographics, HIFs, incidence/prevalence rates, valuation functions, income growth, and inflation estimates. The database contains preloaded datasets for the United States, China, and a case study for Detroit, Michigan. User added data sets are also stored here.

• BENMAP50_GBD.FDB – This database supports BenMAP–CE’s GBD Rollback tool. For each country, the tool contains gridded PM_2.5 concentrations (i.e., particles with a nominal mean aerodynamic diameter less than or equal to 2.5 μm), gridded 2015 population estimates, and country-level baseline death rates for use by the GBD Rollback tool. Future versions of the tool will also quantify ozone-related impacts.

• POPSIMDB.FDB – This database supports BenMAP–CE’s population simulation (PopSim) tool. It contains nationwide multi-year U.S. population data, population growth (birth, migration) rates, and baseline rates of death and disease used by the PopSim tool.

BenMAP–CE includes a User Feedback tool, a form which allows users to provide information about software bugs or other recommendations he or she would like to see addressed. The tool collects contact information, limited system attributes, and optional audit trails and uploads the information to the Agency’s online issue tracker. The Agency uses this information to investigate and prioritize software improvements. BenMAP–CE also has tools that connect to a cloud-based data repository and allow users to import or export selected datasets. These tools are intended to facilitate data sharing among the BenMAP–CE user community. The software development team also welcomes user community-contributed code. After testing newly committed code to ensure that it is free of bugs, the team will merge it into the development branch. Although there is no direct incentive for users to share new code, building a connected user community through the currently available listserv (https://forum.benmap.org) helps to foster an atmosphere of shared interests.

In the process of conducting an analysis using BenMAP–CE the program both requires and generates a series of program specific files (Table 2). Fig. 2 provides a flowchart illustrating the data inputs, internal calculations, and result outputs for conducting a health impact analysis using BenMAP–CE.

Table 2.

Files required and generated during the process of estimating air pollution-related health and economic impacts in BenMAP–CE.

File extension	Name	Purpose
.aqgx	Air quality grid	Gridded air quality data
.cfgx	Configuration	Health impact functions and other options
.cfgrx	Configuration results	Health impact estimates
.apvx	Aggregation, pooling and valuation	Specifies geographic level to aggregate results, pool estimates and value results
.apvrx	Aggregation, pooling and valuation results	Aggregated, pooled and valued results
.shp	Shapefile	Defines the geographic area of each grid
.csv & .xlsx	Comma-separated and excel file	Results
.projx	Project file	Saved project data and references to configuration files
.bdbx	Exported database file	Archive BenMAP–CE data in binary format

Fig. 2.

Flowchart and structure of BenMAP–CE.

3.2. Researcher/policy analyst

Analysts using the BenMAP–CE tool to estimate the number and economic value of air pollution health impacts will specify seven key pieces of information: (1) the air pollution change; (2) the population exposed to the change in air pollution; (3) the baseline rate of death and disease among the exposed population; (4) a beta coefficient from an epidemiologic study; (5) the functional form of the health impact function; (6) an analysis year; and (7) an economic value function (Table 3).

Table 3.

Key parameters for conducting a health impact analysis in BenMAP–CE.

	Air quality	Population	Baseline Rate of Death and Disease	β Coefficient	Economic values
Input parameter					$
Default parameters	Year 2000–2013 PM2.5 and ozone monitoring data for the contiguous U.S. Voronoi Neighborhood Average interpolates to user-specified grid Grids defined for common air pollution models	U.S. population stratified by sex/age/race/ethnicity Population projected from 2000 to 2050 in 1-year increments Aggregated from census blocks to 12 km grid cells	Cause-specific county-level death rates projected from 2000 to 2060 in 5-year increments Hospital and emergency department visits for 2013 at county and state-level	Over 100 PM2.5 and ozone health impact functions (β) for mortality, hospital admission, emergency department, exacerbated asthma, acute respiratory symptoms, school/work loss days	# of cost-of-illness (COI) and willingness-to-pay (WTP) studies for each health endpoint quantified by health impact functions
Updating parameters	Import .csv or .xlsx file specifying air quality modeling or monitoring data. As needed, import shapefile boundary defining location of air quality impacts	Import .csv or .xlsx file specifying sex/age/race/ethnicity stratified population counts assigned to grid	Import .csv or .xlsx file specifying age/race/ethnicity stratified baseline incidence rate assigned to each grid cell	Import .csv or .xlsx file specifying health impact function(s), including health endpoint, functional form, effect coefficient, applicable age/sex/race/ethnicity	Import .csv or .xlsx file specifying COI or WTP function(s), including health endpoint, unit value

Many of the parameters detailed in Table 3 are stored in the “setup” database that contains a variety of preloaded datasets and to which users can add their own data. Fig. 3 illustrates the process of defining the air quality surfaces and estimating health impacts. After those two steps are complete, analysts can aggregate, pool, and value the estimated health impact results. We describe each stage of the analysis below, referencing a case study as context. In the case study detailed within this manuscript, the goal is to estimate the number and economic value of the deaths and illnesses that would have been avoided in 2013 had all monitors met an annual PM_2.5 standard of 12 μg/m³. For this case study, if the reader would like to run this analysis, BenMAP–CE contains all of the necessary data, thus he/she will not need to load any new data.

Fig. 3.

Overview of the process of conducting a health impact analysis in BenMAP–CE.

3.2.1. Initial steps of preparing an analysis

As an initial step, users first consider the analytical question they wish to answer and then determine whether the program contains the necessary data. BenMAP–CE organizes datasets in a “setup” and comes preloaded with data allowing for analyses to be conducted in the U.S., as well as specifically for Detroit, Michigan, and China. Users performing a U.S. or China-based analysis may find that the program already contains the data necessary to perform their assessment; if not, they can load their own data following a series of steps we describe further below.

3.2.1.1. Defining the grid

BenMAP–CE uses a grid structure to perform its calculations. Grids may be both regularly shaped (like an air quality modeling domain, which is often broken into squares of uniform dimensions) or irregularly shaped (like a political boundary). Each dataset–including the air quality, population, and baseline rates of death and disease–is assigned to a grid, with the likely possibility that each data point will be at a different spatial resolution. As highlighted in Fig. 2, air quality data can be assigned to defined grid cells, often 12 km by 12 km, while population data can be at the zip code level, and the baseline rate of death or disease at the county level. However, the overall health impact results are generally calculated for the spatial resolution of the air quality data used in the analysis (i.e., grid cell level) and then can be aggregated up to a larger spatial resolution (e.g., county, nation, etc.). As a result of these disparate spatial resolutions, the GIS embedded within BenMAP–CE performs an area-weighted calculation when assigning data from one spatial scale (e.g. a 12 km by 12 km air quality model grid cell) to another (e.g. a U.S. county). BenMAP–CE comes preloaded with six different grid definitions for air quality data at various spatial domains. After selecting the pollutant of interest for an analysis, in this case PM_2.5, the user is required to select a grid definition, which for the case study is a 12 km by 12 km national domain of the continental U.S. (referred to as CMAQ 12 km – Nation – Clipped within BenMAP–CE).

3.2.2. Specifying the air quality data

In the first stage of a BenMAP–CE analysis, the user creates a “baseline” and “control” air quality grid in the .aqgx file format (Table 1). The baseline air quality grid generally reflects a “business-as-usual” or “as is” case, while the “control” air quality grid reflects the change in air quality after the policy intervention. The BenMAP–CE tool estimates counts of air pollution health effects that may result from the delta, or change, in air quality at the grid level. The program calculates the delta as the difference between the baseline and control air quality data, rendering a map of the baseline, control and delta air quality using DotSpatial GIS (Fig. 4). For the maps, users can select among a variety of approaches to plot the quantitative air quality values including Jenks natural breaks, quantiles, and user-specified values.

Fig. 4.

The change in PM_2.5 concentrations that would occur nationally due to reducing PM_2.5 concentrations in the year 2013 to meet an annual PM_2.5 standard of 12 μg/m³.

BenMAP–CE is preloaded with Federal Reference Method and Federal Equivalent Method monitored air quality data for PM_2.5 and ozone across the Continental United States for the years 2000–2013 (U.S. EPA, 2016). The program includes an interpolation algorithm called Voronoi Neighborhood Averaging, which is an inverse distance weighted algorithm (Gold et al., 1997), which can be used to assign monitored air quality to a grid. When the user is designing an analysis, regardless of whether they are using the preloaded monitored air quality data or importing their own monitored or modeled air quality data, he or she needs to specify the goal of the analysis. Specifically, the user should consider whether the goal of the analysis is to examine retrospective changes, future changes, or local/regional changes in air quality, which then dictates the types of air quality data needed (i.e., monitored or modeled) as well as the geographic scale.

BenMAP–CE can import air quality data from a variety of sources as .csv or .xlsx files including user provided monitored data and modeled air quality data, such as from photochemical transport models (e.g., Community Multiscale Air Quality [CMAQ] model). If importing monitored data, BenMAP–CE requires the location of each monitor be specified by latitude and longitude, while modeled data are indexed to column and row values. Regardless of the air quality data source used for the analysis, the air quality data must be at the same time scale (or averaging time) as that used in the epidemiologic study that produced the beta coefficient that will be used to calculate the health impact. That is, if the epidemiologic study used 24-hour average PM_2.5 concentrations or 8-hour maximum O₃ concentrations, the PM_2.5 and O₃ concentrations from monitors or model predictions must also be 24-hour average or maximum 8-hour average values.

As detailed previously, for the case study, the analysis is focusing on PM_2.5. Therefore, monitored PM_2.5 data was selected. To generate the “baseline” and “control” air quality grids we ran the “monitor rollback” feature of the program. This procedure entails creating a baseline air quality surface using historical PM_2.5 monitoring data (in this case, the year 2013), and then “rolling back” (or, adjusting) these monitoring data such that the ambient concentrations are no higher than an annual PM_2.5 standard of 12 μg/m³. There are several methods available in BenMAP–CE for adjusting air quality to meet standards, including proportional rollback and peak shaving. Which method to use can depend on the pollutant and type of policy scenario or standard examined (e.g., annual or daily). Fig. 4 depicts the ΔAQ for the change in PM_2.5 that would occur across the U.S. due to the scenario being examined in the case study.

3.2.3. Health effects: selecting endpoints, baseline rates of death and disease, and population counts

In stage two of a BenMAP–CE analysis, users quantify the count of adverse health impacts (either incurred or avoided) resulting from the air quality change estimated in stage one of the analysis. To accomplish this goal, the user has to first define the population data to be used. The program is preloaded with population counts stratified by age, sex, race and ethnicity to the year 2050 (Table 3). Projections of population are based on location specific growth rates provided by Woods and Poole (Woods and Poole Economics Inc, 2016).

An important step in quantifying the health impacts is the process of determining which HIFs to include in an analysis. BenMAP–CE uses beta coefficients, which represent the associations between air pollution exposure and a health outcome reported in epidemiologic studies, to construct HIFs and subsequently estimate the counts of air pollution-related deaths and illnesses. It is important to note that a vast body of literature describes the empirical basis for air pollution-related health effects, which is discussed in detail in a number of assessments (U.S. EPA, 2009; U.S. EPA, 2013) As a result, before applying evidence from epidemiologic studies in BenMAP–CE, we encourage users to consult the experimental evidence (i.e., controlled human exposure and animal toxicological studies) to ensure that the effects observed in the epidemiologic studies are consistent across a number of studies, coherent with experimental evidence, and biologically plausible (U.S. EPA, 2015b).¹ EPA analyses using BenMAP–CE, therefore, require that conclusions from previous scientific assessments support the derivation of HIFs for health outcomes where the collective body of evidence has provided reasonable confidence that a causal relationship exists. A list of health outcomes where HIFs have been derived and are currently included in BenMAP–CE can be found in Supplemental File 1.

Currently, the HIFs specific to the U.S. included in BenMAP–CE are for PM_2.5 and O₃ (Table 3). Additional functions specific to China are also preloaded into the tool and span a range of particulate matter size fractions. The incorporation of HIFs for different countries highlights the capability of BenMAP–CE to be tailored to specific geographic domains depending on data availability of input parameters, which will be detailed throughout the rest of this section. Users can also enter additional HIFs through the “Modify Datasets” feature of BenMAP–CE. BenMAP–CE allows for great flexibility in specifying HIFs with a variety of functional forms and parameters.

In the case study, we selected a historical population year of 2013, which corresponds to the date the monitored PM_2.5 data was collected. We then proceeded through the process of selecting the HIFs that will be used to estimate PM_2.5-related mortality, cardiovascular hospital admissions, cases of exacerbated asthma and lost work days. Once the HIFs are selected, a configuration file (.cfgx, Table 1) can be saved, which allows us or other users to conduct future analyses using the same suite of HIFs (See Supplemental File 2 for the audit trails for the case study).

3.2.4. Aggregating, pooling, and valuing

In stage three of a BenMAP–CE analysis, the user can aggregate the results by applying a meta-analytic technique to pool results from multiple HIFs, and then estimate the economic value of the adverse health impacts. Because the program estimates health impacts at the air quality grid cell level, users can aggregate these counts to a coarser spatial scale to be interpretable. Grid cell level results can be useful to understand geographic patterns in results using the built-in GIS or by exporting the results for use in other GIS packages.

Although the counts of air pollution-related deaths and illnesses based on HIFs from individual studies are informative, meta-analytic approaches allow for additional characterization of variability and uncertainty, and can increase the confidence in results. Therefore, in BenMAP–CE meta-analytic approaches can be applied to combine the estimated impacts from individual HIFs, which is particularly important if a number of HIFs are examined for the same health endpoint (e.g., all cardiovascular hospital admissions). A meta-analytic approach can account for the heterogeneity between the estimates generated for individual HIFs and can also result in a more stable estimate that is more representative of the entire body of epidemiologic literature for the health endpoint being examined (Fann et al., 2016). The pooling options available within BenMAP–CE consist of addition, subtraction, user-assigned weights, random–effects method, and fixed effects method (U.S. EPA, 2017). It is worth noting that the pooling process is not trivial and often requires consultation with a statistician to solidify decisions. Fig. 5 depicts the pooling window for the case study where results are being pooled across the HIFs selected to estimate the number of reduced cardiovascular hospital admissions.

Fig. 5.

Pooling window for applying meta-analytic methods to estimate the reduced number of cardiovascular hospital admissions in the year 2013 to meet an annual PM_2.5 standard of 12 μg/m³.

Once the user pools the estimates generated from the individual HIFs, BenMAP–CE can then value the impacts. Fig. 6 depicts the window for assigning dollar values to the health impacts estimated in the analysis.

Fig. 6.

Pooling window for applying valuation methods to estimate the economic benefits of reducing the number of cardiovascular hospital admissions in the year 2013 to meet an annual PM_2.5 standard of 12 μg/m³.

Economists value health impacts using several techniques. The most comprehensive valuation estimates reflect the amount of money society is willing to pay to reduce the risk of an adverse outcome by some amount. These WTP measures account for harder-to-measure components including the value of reduced pain and suffering. COI measures, by contrast, reflect the value of directly incurred medical costs (such as a hospital stay) and lost productivity, but omit the value of pain and suffering. Very often the Agency does not have the capability or resources to conduct original valuation research so the concepts of benefits transfer are applied to find similar policy contexts that allow for dollar values to be applied to changes in the incidence or prevalence of health endpoints (U.S. EPA, 2012). With BenMAP–CE, COI and WTP estimates are used to value morbidity endpoints. For mortality, the values are based on the value of a statistical life (VSL) or how much society is willing to pay in aggregate to reduce its risk of death to avoid one additional death across the population. When reaching the valuation step of an analysis, the user can apply various WTP and COI estimates to morbidity endpoints for different years of U.S. dollar values, a similar exercise can also occur when applying VSLs to mortality effects; this procedure allows users to account for changes in purchasing power and income over time, which are each factors that affect the size of the economic unit values.

3.2.5. Results of the case study

In this case study using BenMAP–CE, we estimate between about 6200 and 13,800 PM_2.5-related premature deaths would be avoided by meeting an annual PM_2.5 standard of 12 μg/m³, which nationally equates to an approximate 0.5 μg/m³ reduction in population-weighted average PM_2.5 concentrations. We quantify about 480 fewer cardiovascular hospital admissions, 1.1 million cases of exacerbated asthma, and approximately 600,000 fewer days of work lost. The economic value of reductions in mortality ranges from about $50 to $120B (2015$), while reductions in PM_2.5-related morbidity are much smaller ranging from $0.02 to $0.11B (2015$) (Table 4). A more detailed description of each individual step of this case study can be found in the audit trails (see Supplemental File 2).

Table 4.

Overall health impacts and economic benefits associated with meeting an annual PM_2.5 standard of 12 μg/m³ in 2013.

Health Endpoint	Health Events Avoided (95% Confidence Interval)	Value (billions of 2015$) (95% Confidence Interval)
PM2.5-related mortality (Estimated using coefficient from Lepeule et al. (2012))	13,800 (7000–20,300)	$120 ($10–$330)
PM2.5-related mortality (Estimated using coefficient from Krewski et al. (2009))	6200 (4200–8200)	$50 ($5–$140)
PM2.5-related cardiovascular hospitalizations	480 (220–1100)	$0.02 ($0.009–$0.04)
PM2.5-related asthma exacerbations	1,135,900 (−9700–3,758,300)	$0.07 (−$0.000.6–$0.3)
PM2.5-related work loss days	665,000 (564,600–764,400)	$0.11 ($0.09 - $0.13)

3.2.6. Global Burden of Disease (GBD) rollback tool

BenMAP–CE also contains a reduced-form tool embedded within it that applies outdoor air pollution data from the GBD study to quantify the number of PM_2.5-related deaths in any country in the world using the exposure estimates detailed in Brauer et al. (2012), which consisted of modeled global PM_2.5 concentrations predicted for the year 2015 at 0.1 0.1 or approximately 11 km by 11 km grid cells. For the same spatial domain, worldwide population data was obtained for the year 2015 from census data from the United Nations, Socioeconomic Data and Applications Center (SEDAC) Gridded Population of the World (GQW) v4 while country-specific mortality rates were obtained from the Institute of Health Metrics and Evaluation (IHME). Lastly, the tool relies on a HIF for premature mortality that is based on an integrated exposure response function that was developed to better account for air pollution exposures across the entire global range, from the lowest levels in North America to the higher levels (>100 μg/m³), which are often experienced in less developed and developing countries (Burnett et al., 2014).

When accessing the GBD Rollback tool the user can first define the scale of the analysis, i.e., focus on a few countries of the world or entire regions. Once the user has defined the country or regions for the analysis, he or she can then define the type of rollback analysis to conduct, hence the name GBD Rollback tool. Options include: (1) percentage rollback; (2) incremental rollback; or (3) rollback to standard, where PM_2.5 concentrations are reduced to meet a number of U.S. and international air quality standards. The GBD rollback tool will estimate the number of premature mortalities that could be avoided due to the chosen rollback scenario along with PM_2.5 concentration information for each country and/or region. These results are exported in the form of an Excel file. Overall the GBD rollback tool acts as a means to easily demonstrate both the capability of BenMAP–CE and the potential public health impact of improving air quality, particularly in countries that do not have the underlying data available to conduct a full-scale analysis in BenMAP–CE.

4. Conclusions

BenMAP–CE has become a widely used tool for the purposes of quantifying the population health impacts attributed to changes in air quality in academia as well as various levels of government both in the U.S. and internationally as reflected by its extensive use in peer-reviewed publications since its inception. The recent modifications to BenMAP–CE that occurred during the process of transferring to the open source platform have improved the user experience and in combination with the U.S. EPA’s expanded training programs, has facilitated the expansion of the use of the tool worldwide. Additionally, the incorporation of a reduced form version of the tool, the GBD Rollback tool, has further contributed to the expansion of BenMAP–CE by easily demonstrating the potential public health implications of improving air quality in regions of the world that do not have the resources or underlying data needed to assess the potential public health impact. It is through exercises such as the case study, and the GBD Rollback tool that the U.S. EPA is able to demonstrate to broader audiences that improving air quality can have substantial health and economic benefits for a country or even broad region of the world.

Although BenMAP–CE can be a powerful tool; it is also important to reflect on its limitations and the expertise required to conduct an analysis. Specifically, BenMAP–CE cannot be used to conduct source specific analyses without inputs from other modeling programs, such as CAMx, CMAQ or some other chemistry, fate, and transport model. In its current form, the air quality data contained within BenMAP–CE represents overall ambient concentrations of PM_2.5 and O₃, not source-specific contributions. Additionally, it is important to stress the importance of having the proper expertise in the process of both designing and conducting analyses using BenMAP – CE, or at a minimum consulting individuals with expertise in epidemiology, modeling, and even air quality management. This is never more evident than in the process of incorporating new HIFs into BenMAP – CE, particularly for health effects where the larger body of scientific evidence is rather limited or emerging.

The Agency’s investment in improvements in BenMAP–CE, including the open source aspect of the tool, described in this manuscript, coupled with providing increased numbers of domestic and international trainings and workshops have expanded the use of the tool. These trainings are geared towards groups of individuals or governments with specific research or policy questions as well as individuals that have no prior knowledge of the tool. Additionally, the Agency established quarterly webinars to provide users with real-world examples of how BenMAP–CE is applied along with information on ongoing improvements to the tool or the underlying methodology that governs how the tool functions. More recently, the Agency also established a listserv (https://forum.benmap.org) that allows users to pose questions or troubleshoot issues with both Agency experts in BenMAP–CE and the broader user community. The combination of all of these improvements and outreach efforts allows for the continuous evolution and improvement of BenMAP–CE.

Abbreviations

BenMAP–CE

Environmental Benefits Mapping and Analysis Program – Community Edition

CAPMS

Criteria Air Pollutant Modeling System

CIESIN

Center for International Earth Science Information Network

COI

cost-of-illness

DLLs

dynamic-link libraries

GBD

Global Burden of Disease

GIS

geographic information system

GUI

graphical user interface

IHME

Institute of Health Metrics and Evaluation

MSDE

Microsoft SQL Server Data Engine

NAAQS

National Ambient Air Quality Standards

NO₂

nitrogen dioxide

O₃

ozone

Pb

lead

PM

particulate matter

PM_2.5

particles with a nominal mean aerodynamic diameter less than or equal to 2.5 μm

SO₂

sulfur dioxide

VSL

value of a statistical life

WTP

willingness-to-pay

Appendix A. Supplementary data

Supplementary data related to this article can be found at https://doi.org/10.1016/j.envsoft.2018.02.009.