Particulate Matter Matters

Abstract

Quasi-experimental evidence is needed on the relations between human health and airborne particulate matter.

April 22nd is the 45th Earth Day, which marks the birth of the modern environmental movement that helped lead to the creation of the U.S. Environmental Protection Agency, the Clean Air Act Amendments, and the Clean Water Act. The result has been substantial improvements in environmental quality in the United States. Today, developing countries are contending with levels of pollution that are even higher than those in the United States before the first Earth Day. And in a period of considerable economic difficulty, the United States is trying to strike the right balance between the benefits and costs of further reductions in pollution.

Under federal law, the U.S. Office of Management and Budget (OMB) must report to Congress annually on the benefits and costs of major federal regulations. It is remarkable but true that from 2003 through 2012, reductions of emissions of just one category of pollutant—particulate matter (PM)—have accounted for about one-third to one-half of the total monetized benefits of all significant federal regulations and, by some estimates, more than that (1). With the estimated benefits of PM reductions playing such a central role in regulatory policy, it is critical to ensure that the estimated health benefits are based on the best available evidence. If the estimates are biased upward (downward), then the regulations may be too stringent (lenient).

In the last 40 years, the evidence that has led to revisions of the U.S. National Ambient Air Quality Standards has come mainly from observational studies aimed at estimating an exposure-response relation (2). But associational approaches to inferring causal relations can be highly sensitive to the statistical model and covariates used to adjust for confounding. Indeed, the U.S. government itself has drawn attention to the “uncertainty in the reduction of premature deaths associated with reduction in particulate matter” (3).

There is a growing consensus in economics, political science, statistics, and other fields that the associational or regression approach to inferring causal relations—on the basis of adjustment with observable confounders—is unreliable in many settings (4–6). We discuss how quasi-experimental (QE) techniques provide an opportunity to improve understanding of the relation between human health and regulation of air pollution from particulates.

Beijing shrouded in smog.

Limits of Observational Studies

Randomized control trials would be the best way to measure the health benefits of PM reductions (4), but for obvious reasons, true experiments are generally not feasible. One exception is chamber studies of controlled exposure, but such studies rely on healthy subjects and focus only on end points of limited value.

An observational study of the health effects of particulates boils down to a comparison of health outcomes across space and/ or time among places with differing levels of air pollution. For example, an influential study compared the health outcomes of individuals who lived in six cities with varying levels of air pollution (2). For such studies, one challenge is that the people who live in the more polluted places frequently have differing initial levels of health (e.g., due to differences in smoking rates, diet, or socioeconomic status) from the levels of people who live in the less polluted places. Another challenge is that there may be locational determinants of health (e.g., hospital quality or water pollution) that differ across the places and are correlated with air pollution levels. Further, people may choose to live in locations on the basis of their (likely unobserved) susceptibility to pollution and other related health problems, and/or they may spend greater resources on self-protection in polluted locations in ways that are not measured in available data sets.

Statistical methods, based mostly on regression approaches, aim to “adjust” for observed confounders, by including the available measures of behavioral, socioeconomic, and locational differences as covariates in the regression model. Since many determinants of health are unobserved, these methods that rely on adjustment for observed confounders can lead to biased estimates of the relation between health and particulates.

In 2010, the American Heart Association conducted a review of the available observational studies exploring the relation between fine particulate matter [diameter <2.5 μm (PM_2.5)] exposure and mortality and cardiovascular morbidity (7). The authors concluded: “It is the opinion of the writing group that the overall evidence is consistent with a causal relationship between PM_2.5 exposure and cardiovascular morbidity and mortality.”

Thanks to the rigorous statistical methods that have been developed and applied to the assembled data, and to the enormous effort of government agencies and specific investigators in conducting independent reanalyses [e.g., (8)], analyses of observational data have had a large impact on air-quality regulations and on the supporting analyses of their accompanying benefits.

Nonetheless, legitimate concerns remain. Although important progress has been made in adjusting for confounding in observational studies (9–13), there may be unobserved differences across the populations and locations, and sufficient adjustments may not have been made for the measurable differences.

This point is illustrated in the table, which summarizes evidence from (14) that examines the cross-sectional relation between total suspended particulates (TSPs) and mortality rates (deaths per 10,000) among the more than 30 million individuals age 50 and over living in the 501 U.S. counties monitored for TSP in the years 1970–72. The first data column reports on a model that does not include any adjustment for observed confounders. The estimates in the second data column are adjusted for the age distribution, gender, and race of the population and detailed county-level per capita measures of employment; public transfers (e.g., food stamps, Medicare payments); and health expenditures. The rows report on the separate estimation of these two models on mortality rates in each year from 1969 through 1974. The entries report the impact of a 1 μg/m³ increase in TSPs on the mortality rate.

Of the 12 regression estimates, two show a significant positive association between TSPs and mortality, two perversely show a signifi cant negative association, and the remaining eight would be judged statistically insignificant at conventional levels. In this setting, the addition of covariates causes a reduction in the coefficients, but this is not generalizable to other studies, as bias due to confounding can go in either direction. Overall, it is apparent that with these data and the available covariates, the association between TSPs and mortality rates varies widely within a year across models and within a model across years.

This admittedly provocative example illustrates our point: Associational approaches to inferring causal relations can be highly sensitive to the choice of the statistical model and set of available covariates that are used to adjust for confounding.

Quasi-Experiments as an Alternative

QE evaluation techniques provide an opportunity to improve understanding of the relation between human health and particulates air pollution. In a QE evaluation, the researcher compares outcomes between a treatment group and a control group, just as in a classical experiment; but treatment status is determined by politics, an accident, a regulatory action, or some other action beyond the researcher’s control. The key difference with an observational study in this setting is that the QE approach is devoted to identifying treatment-induced variation in particulates that plausibly mitigates confounding or omitted variables bias in the estimated relation between human health and particulates, rather than relying on the variation presented by nature and optimizing agents. Despite the “nonrandom” assignment of treatment status, it is possible to draw causal inferences from the differences in outcomes (by “outcomes,” we refer to both air pollution levels and human health) between the treatment and control groups in a quasi- or natural experiment, provided certain assumptions are met.

This approach has been used extensively in recent years and has permitted more credible inferences about the impacts of a wide range of relations, including the effect of an additional year of schooling on earnings (15), the impact of changes in air pollution on housing prices (16), the effect of Medicare on mortality (17), and the effect of anti-discrimination laws on the earnings of African-Americans (18).

In fact, there is an emerging QE literature on the human health effects of air pollution that relies on designs where an “action” has affected—often drastically—the ambient levels and the chemical composition of air pollution. Some of the most well-known examples are the ban of coal sales in Dublin (19); the differential reduction in TSPs across the United States as a consequence of the 1981–82 recession (20); the air pollution reduction interventions before, during, and after the Beijing Olympic games (21); a steel plant strike (22); features of the U.S. Clean Air Act (6, 23); and the Chinese policy that provided free coal for heating in cities north of the Huai River (24) (see sidebar). [See also (25–28) for detailed reviews.]

Although QE approaches promise more credible estimates, they are not without limitations. It is important that QE designs are able to demonstrate that observable covariates are balanced by the treatment and credibly explain why unobserved ones are likely to be balanced, too. In cases where the covariates are not balanced and/or the unobserved ones are unlikely to be balanced, QE estimates are not likely to be more credible than associational estimates. Further, QE approaches can often be demanding of the data and lack statistical power. As is the case with associational estimates, applying QE estimates to other settings (e.g., places, periods, and demographic groups) requires careful consideration and, in some cases, may be inappropriate. This challenge can be greater with QE approaches where the selection of the study population is dictated by the available treatment (see the box) and therefore is beyond the researcher’s controls.

On the Wrong Side of the River.

Chen et al. (24) illustrates some of the appealing features of QE designs and more specifically of a regression discontinuity (RD) design (34). It exploits a Chinese policy that provided free coal for winter heating in areas north of the Huai River and denied coal-based heating to the south of the river. The idea is to compare locations just north and south of the river. In this setting, the RD design relies on the assumption that any confounders (both observed and unobserved) vary smoothly with latitude, except for the availability of coal-based indoor heating, as one crosses the Huai River. The authors controlled for these potential confounders through adjustment for a flexible polynomial in distance to the river, measured as degrees latitude north of the Huai River.

The authors find that north of the river, the policy led to discrete increases in TSPs and discrete decreases in life expectancy (derived from age-specific mortality rates). The paper’s headline finding is that long-term exposure to an additional 100 μg/m³ of TSPs is associated with a reduction in life expectancy at birth of about 3.0 years (95% confidence interval: 0.4 to 5.6). Further, the paper finds that the effect of TSPs on life expectancy is largely insensitive to whether observable covariates are included in the model, which would be the case in a randomized control trial; this stands in contrast with the relations in the table that were derived from an observational study, rather than one based on a QE design.

A Path to Better Science and Policy

Overall, the literature makes a compelling case that airborne PM is bad for human health—it shortens lives and raises morbidity rates. However, quantitative estimates of the magnitude of the anticipated or actual public health benefits from further reductions are needed and will require careful work. Deeper understanding of this relation could greatly improve regulatory policy in coming years as regulators face two external events.

First, the Obama Administration has started a regulatory look-back initiative that aims to assess how regulations have worked in practice (“retrospective analysis”) and then adjust the regulations on the basis of the results. It would be useful to know whether previous efforts to reduce particulates air pollution actually produced the projected improvements in human health [e.g., (6, 14, 29)]. Second, estimates of the health effects of PM will play a central role in numerous upcoming regulatory decisions. In making those estimates, two sets of critical questions need better answers: (i) What is the shape of the dose-response curve (e.g., would a reduction in PM_2.5 from 12 to 10 μg/m³ produce the same health benefits as a reduction from 14 to 12)? (ii) In light of the wide range of chemical composition across particulates, can we identify chemical components of PM_2.5 and the type of emission sources that regulators should target to protect public health (without imposing unnecessary compliance costs, which would ultimately be incurred by companies, workers, and consumers)? [See also (30, 31).]

A critical question is how to develop better evidence to identify not merely the existence but also the magnitude of public health benefits from cleaner air and especially from further reductions in PM (32). To be sure, regulators must sometimes act under considerable time pressure; the Clean Air Act sharply constrains their ability to consider costs as such; and in some circumstances, use of carefully conducted observational studies remains the only feasible option. However, the path to the best available evidence about the benefits (33) of reducing PM and other air pollutants lies in an increased focus on developing and using QE evidence. Such a shift would honor Earth Day by going back to its foundations, which demanded that we draw on the most reliable science to protect humanity’s well-being.

Table 1.

Estimate of the effect of TSPs on mortality rates of adults over 50

Adults aged over 50 (year)	Adult deaths per 10,000 residents
	No adjustment for confounders	Adjusted for confounders
1969	0.183** (0.062) [0.04]	0.024 (0.030) [0.75]
1970	0.112* (0.068) [0.02]	0.022 (0.024) [0.74]
1971	0.088 (0.091) [0.01]	–0.047 (0.025) [0.74]
1972	0.102 (0.125) [0.01]	–0.130** 0.050 [0.72]
1973	0.208 (0.129) [0.02]	–0.054 (0.061) [0.74]
1974	0.126 (0.115) [0.01]	–0.157*** (0.050) [0.75]

TSPs and mortality rate. The association between TSPs and mortality rates can vary widely within a year across models and within a model across years. Cross-sectional parameter estimates of the effect of a 1 μg/m³ increase in TSPs on mortality rates, heteroskedastic-consistent standard errors (in parentheses), and R-squared statistics (in brackets) associated with a separate regression of the mortality rate of adults over age 50 on TSPs. Adjustments for confounds include controls for age distribution, gender, race, income, employment, income assistance, and medical expenditures.

^*P < 0.1,

^**P < 0.05,

^***P < 0.01. Modified from (14).