Quasi-Experimental Approaches to Evaluating the Impact of Air Pollution on Children’s Health

Abstract

Many studies have shown a correlation between air pollution and poor children’s health. This paper focuses on recent studies that employ quasi-experimental designs in an effort to minimize the effect of confounding factors. These studies are complementary to studies using other designs and confirm that reducing air pollution reduces infant mortality, low birth weight, prematurity, congenital anomalies, asthma hospitalizations, and school absences. These results suggest that lowering the thresholds for acceptable air pollution levels may be prudent, as research has consistently found that some pollutants have negative impacts even at levels below current regulatory thresholds. Policy makers should also consider providing more information to pregnant women and families about when and where the risk of pollution exposure is highest so that they can employ avoidance behavior.

A primary goal of environmental policy is to protect human health, but there is still much debate about the specific health effects and their costs to society. Although the US EPA requires extensive cost-benefit analyses associated with air quality standards being promulgated, surprisingly little is known, and thus included in these analyses, about how air pollution affects some of the most vulnerable members of society – children. The impact of air pollution on children is of increasing interest to researchers and policy makers because prenatal and early life environments affect not only health at birth and during infancy but throughout the lifecycle. For example, on average children of lower birth weight have worse adult outcomes in terms of schooling attainment, test scores, use of disability programs, residence in high income areas, and wages (1, 2, 3, 4, 5). If policies that reduce air pollution improve child health, then the costs of such policies may be recouped through lower subsequent healthcare and societal costs (6).

Many studies of the relationship between the environment and health identify correlations, but these correlations do not necessarily imply causation. Families with greater preferences for cleaner air, either because of underlying health conditions or greater wealth, are more likely to live in less polluted areas. If these families also make other unobservable investments in their children’s health, then this introduces an important source of confounding.

The quasi-experimental techniques we focus on attempt to address this concern, and are complementary to other techniques in the literature (7,8,9). Quasi-experiments focus on the relationship between changes in pollution and changes in outcomes among people in small areas or among siblings in the same family. This design controls for many potential confounders that remain constant over the period covered by the change in pollution. While quasi-experimental techniques typically cannot assess the specific biological mechanisms underlying observed relationships between pollution and health, they can also be thought of as complementary to clinical studies that focus on mechanisms (10, 11).

This paper first describes the main methodological issue that quasi-experimental studies attempt to address. It then describes recent studies in the United States that have employed quasi-experimental techniques, and follows with some estimates of the health cost of pollution from these studies. As discussed below, these studies show that reducing air pollution reduces infant mortality, low birth weight, prematurity, congenital anomalies, asthma hospitalizations, and school absences. This discussion is followed by a description of some of the methodological challenges involved in quasi-experimental studies. We conclude with some policy implications.

The Main Methodological Issue

Without an appropriate control group, it is difficult to control for potential confounding

Many studies observe children’s health and exposure to pollutants at a point in time, and find that children exposed to high levels of pollution also have poorer outcomes. However, pollution is not randomly assigned, and differences in ambient pollution are usually correlated with other determinants of infant health. As seen in Exhibit 1, infants born in higher pollution areas are more likely to have younger, less educated, and unmarried mothers who are less likely to receive prenatal care in the first trimester. These and other unobserved differences may help to explain the poorer outcomes, suggesting a significant threat of confounding (12). Thus, it is probably impossible to say anything about causality using a single cross section of data, though it is possible to identify interesting correlations.

Exhibit 1.

Sample Means for Infants Born in Highest and Lowest Pollution Zip Codes in California

Variable	Lowest 1/3 Pollution Level	Highest 1/3 Pollution Level
Panel A: Pollution Levels
CO 8-hr level	1.176	2.912
PM10 24-hr level	25.647	54.139
O3 8-hr level	34.837	46.705
Panel B: Birth Outcomes
Infant Mortality Rate per 1000	3.583	4.406
Low Birth Weight per 1000	47.094	49.506
Fetal Death per 1000	3.370	3.840
Panel C: Mother’s Characteristics
Mother African American (%)	0.083	0.083
Mother Hispanic (%)	0.317	0.550
Mother Married (%)	0.725	0.629
Mother Foreign (%)	0.394	0.524
Mother HS Dropout (%)	0.254	0.408
Mother HS Graduate (%)	0.359	0.348
Mother Some College (%)	0.148	0.114
Mother College Graduate (%)	0.239	0.130
Mother Age 19 to 25 (%)	0.304	0.366
Mother Age 26 to 30 (%)	0.281	0.281
Mother Age 31 to 35 (%)	0.233	0.187
Public Health Insurance (%)	0.384	0.495
1st Trimester Prenatal Care (%)	0.816	0.742

SOURCE: Janet Currie and Matthew Neidell (17).

NOTES: The table is calculated using the entire sample of births (4,593,001 births in California over the period 1989–2000, excluding 1998). A unit of observation is the birth, and the pollution levels are the pollution levels in the child’s zip code in the week of birth. The units are: parts per million (ppm) for CO; micrograms per cubic meter (ug/m3) for PM10; and parts per billion (ppb) for O3.

An ideal control group would have identical demographics and experiences but different pollution exposures. It would be unethical to design a randomized experiment on children, so researchers employing quasi-experimental designs rely on natural or policy induced variation in pollution levels to find a control group. More concretely, if a random, externally generated (exogenous) event reduces pollution for one group, while pollution levels for a statistically identical group remain the same, the average differences in health outcomes between these two groups following the event are likely to be attributable to differences in pollution. If one can find a source of variation in pollution that is uncorrelated with the multitude of other factors that affect health, then one does not need to explicitly control for all of these other factors. This is a strength of the quasi-experimental method.

Quasi-Experimental Estimates of Effects of Environmental Pollution on Health

Pollution has been shown to increase rates of infant mortality

Due primarily to advances in neonatal technology, infant mortality in the U.S. has declined greatly, from 12.6 infant deaths per 1000 births in 1980 (13) to only 6.8 in 2007 (14). Some of this decline may be due to improvements in air quality.

Chay and Greenstone (15, 16) used the implementation of the Clean Air Act in 1970 and the recession of the 1980’s to study the effect of a decrease in air pollution on infant mortality. Both the Clean Air Act and the recession induced sharper reductions in particulates in some counties than in others, and they use this variation in pollution to identify its effects. They estimate that a one-unit decline in particulates caused by the implementation of the Clean Air Act (recession) led to between five and eight (four and seven) fewer infant deaths per 100,000 live births.

Currie and Neidell (17) analyze the effect of pollution on infant mortality rates in California in a more recent period: 1989 to 2000. They estimate models of the probability that an infant survived, and focus on comparisons of infants who were born into the same zip code, but at times with different pollution levels. Their estimates imply that reductions in carbon monoxide over the 1990s saved approximately 1000 infant lives.

In a study of infants born in New Jersey between 1989 and 2003, Currie, Neidell, and Schmieder (18) extend this analysis in two important ways. First, they follow mothers over time, so that they can compare the outcomes of siblings who were exposed to different pollution levels. This design controls for within family characteristics shared by siblings. Second, they know the exact addresses of mothers (rather than zip codes), so they are able to assign air pollution more accurately than Currie and Neidell. They estimate that in 2003, 449 deaths were averted by reductions in CO from 1989 levels.

Reyes (19) focuses on the removal of lead from gasoline under the Clean Air Act in order to try to identify how prenatal lead exposure affects infant mortality. Using state-level data on the consumption of leaded gasoline and detailed data on nearly all births in the United States between 1975 and 1985, she finds that the complete phase out of leaded gasoline was associated with 3–4 percent reductions in infant mortality.

Low birth weight, prematurity, and congenital anomalies are linked to pollution

In 2007, 8.2 percent of live births were categorized as low birth weight (14), suggesting the need to understand factors that affect this important measure of infant health. Currie, Neidell, and Schmieder (18) find that infants exposed to higher carbon monoxide (CO) levels in the last trimester of pregnancy experienced lower birth weight and shorter gestation length than their siblings even though ambient CO levels were generally much lower than current Environmental Protection Agency (EPA) standards. As shown in Exhibit 2, they also find that the effects of pollution were greater for higher risk mothers, suggesting that the same environmental insult could have worse impacts on the most vulnerable infants.

Exhibit 2.

Estimates of the Effects of CO on Low Birth Weight from Sibling Comparisons

Source: Currie, Neidell and Schmeider (18).

Notes: Bars show the estimated effect of a one part per million change in CO on the probability of low birth weight for the indicated group. “Risk Factors” indicates mothers with pre-existing risk factors for the pregnancy.

Currie and Walker (20) study the introduction of electronic toll collection devices (E-ZPass) in New Jersey and Pennsylvania. E-ZPass reduced pollution near toll plazas by allowing cars to proceed more rapidly through plazas. They compare mothers living near toll plazas to those who also live near busy roadways but further away from toll plazas. They find that among mothers living near toll plazas, the introduction of E-ZPass reduced prematurity by 10.8 percent and LBW by 11.8 percent. Similarly, when they follow mothers who live near toll plazas over time and compare siblings born before and after the adoption of E-ZPass, they find that prematurity declined 1.4 percent and LBW declined by 1.1 percent.

Currie, Greenstone, and Moretti (21) focus on the effects of cleanups of hazardous waste sites under the Superfund program. They examine a sample of women living within 2000 meters of a Superfund site, and compare them to women living 2000 to 5000 meters from a site before and after the site was cleaned up. They find that cleanup reduced the incidence of congenital anomalies in the neighborhood of the site by 20 to 25 percent.

Asthma hospitalizations are associated with air pollution

Asthma is one of the leading chronic illnesses of children and has risen dramatically over the past two decades (22). A 1989 study by Pope (23) assessed the association between hospital admissions and fine particulate pollution (PM10) in Utah Valley during a time period that included the closure and reopening of a local steel mill, the primary source of PM10. The closing of the mill was an exogenous change, and thus the time when the steel mill was closed could be used as a control for when the mill was open. During months when the mill was open and mean PM10 levels were greater than or equal to the EPA threshold of 50 micrograms/m3, hospital admissions for children increased by 89 percent relative to when it was closed. If the mill closure caused economic hardship for families, and hardship worsens child health, then these estimates are likely to understate the effects of pollution at the mill.

Friedman, et al. (24) use the 1996 Olympic Games in Atlanta, GA as an exogenous change in air pollution levels in order to study asthma hospitalizations. In order to reduce traffic downtown during the games, many streets were closed off and traffic was diverted. As a result, peak ozone levels fell by 27.9 percent. The authors show that Medicaid claims for acute asthma care events declined by 48% during the games relative to the baseline period and attribute this to the decline in pollution.

Lleras-Muney (25) uses changes in military children’s location due to military transfers, which occur entirely to “satisfy the needs of the army,” to identify the impact of pollution on children’s hospitalizations for respiratory conditions. She uses individual-level data on military families and their dependents, matched with pollution data at the zip code level, for the period 1989–1995. She finds that when 2–5 year old children are relocated to areas with high ozone, hospitalizations for respiratory conditions increase.

Pollution may cause negative educational outcomes

Respiratory problems, such as asthma, may cause a child to be absent from school, which may have a detrimental effect on education. Using data on the Utah steel mill opening and closing, Ransom and Pope (26) study the relationship between air pollution and absenteeism at elementary schools in Utah Valley. They find that a monthly increase of 100 micrograms of PM10 levels increased absences by 40 percent. Their study is notable in part because they examine many other possible determinants of absences, including weather and the timing of major holidays, and show that their results are robust to controls for these potential confounders.

More recently, Currie, et al. (27) examine absences using administrative data from the 39 largest school districts in Texas. They find that within school districts, when CO levels rise, absences also rise, even when levels are well below current federal air quality standards. For example, they find that reductions in the number of days with high CO levels between 1986 and 2001 in El Paso, an area in Texas with particularly high CO levels, reduced absences by 0.8 percentage points.

Zweig, Ham, and Avol (28) investigate the relationship between changes in outdoor air pollution levels near schools in Los Angeles and changes in test scores on standardized state tests. The authors find that decreases in particulate concentrations near a school raise math and reading test scores.

Parents act to limit their children’s exposure on high pollution days

Public provision of information about health hazards has increased dramatically over time. Examples include the Toxic Release Inventory (http://scorecard.goodguide.com/env-releases/community.tcl, (29) and ozone alert systems (http://www.airnow.gov, (30). Providing information encourages people to engage in avoidance behavior in order to minimize their exposure to pollution. However, if people decrease their exposure on high pollution days, the potential for negative health consequences is understated. Neidell (31) provides one of the first examinations of avoidance behavior by examining responses to “smog alerts”. His estimates imply that if people had not responded to smog alerts (by, for example, staying indoors), double the number of children would have been hospitalized for asthma.

Associated Costs of Environmental Pollution

Assigning a dollar cost to infant mortality is controversial

A number of the papers discussed above attempt to assess the costs of infant mortality due to air pollution. There are a number of approaches to valuing lives and not all of them are obviously applicable to infants. One approach is to infer the value of a statistical life (VSL) by asking how individuals make tradeoffs between risks and things with monetary value. For example, they might accept a job with more risk of death but with a higher wage. But obviously infants do not make these choices.

The court system often values an infant life at “replacement cost,” defined as a combination of money spent raising the infant and a nominal amount for pain and suffering (32).

The EPA generally uses adult VSL estimates for children when conducting cost-benefit analyses. Using a value of $4.8 million (33), estimates from Currie and Neidell (17) imply that the improvement in infant mortality that resulted from the reduction in CO over the 1990s yielded benefits of $4.8 billion in California alone.

The costs associated with LBW, congenital anomalies, and hospitalizations are significant

Currie, Neidell and Schmieder (18) find that a one-unit decrease in the mean level of CO during the last trimester would increase average birth weight by 16.65 grams, which is an increase of about a half a percent. To put this in perspective, smoking 10 cigarettes a day is estimated to reduce birth weight by about 61g in infants. The proportional effects are greater for low birth weight infants, where a one-unit decrease in mean CO would lead to an 8 percent decrease in the incidence of low birth weight, suggesting that infants with the highest risk of low birth weight are also most likely to be affected by air pollution. Further calculations suggest that the decline in CO from 1989 levels increased lifetime earnings for U.S. infants born in 2003 by roughly $720 million.

Challenges in Conducting Quasi-Experimental Studies

Datasets combining health data and air pollution data are incomplete

The ideal data would follow individuals over time and include objectively measured health outcomes, precise measures of the individual’s exposure to pollution, and measures of a wide range of potential confounding variables. Unfortunately, there are no large-scale longitudinal data sets with both health records and pollution exposure repeatedly measured for the same children.

Consequently, researchers must accept tradeoffs. Datasets with objectively reported health outcomes and precisely measured pollution levels often involve small samples and do not lend themselves to a quasi-experimental design; for example, they may lack “control” groups who are not exposed to pollution. Larger datasets often have cruder measures of health and of pollution exposure, as discussed further below.

Infant health is more frequently studied than the health of older children due to data limitations

Data on the health of older children is sparse, whereas data on births can be drawn from the Vital Statistics Natality System (VSNS); that is, from birth certificates and infant death certificates. The VSNS contains data for every birth and infant death, including information on birth weight, length of gestation, congenital anomalies, and cause of death, along with background information about the mother and father. Another reason for the focus on infant health outcomes is that the link between cause and effect is relatively immediate for infants, unlike maladies among older children or adults that may reflect pollution exposure from prior years.

Two sources of data for older children are hospital discharge and school attendance records. Both represent a census of events, thus providing a large number of observations for analysis. Furthermore, data are available over time, outcomes are objectively measured, and geographic identifiers, such as a patient zip code or school address, may be available so that these data can be merged to time-varying data on pollution exposure.

An alternative data source for children’s health is the National Health and Nutrition Examination Survey (NHANES) (34). This recurring cross-sectional survey combines interviews and physical examinations, producing more reliable data than questionnaires alone. However, NHANES samples a new group of counties each year, which poses problems for longitudinal research analyses, which are often a foundation for quasi-experimental designs. Moreover, while the NHANES has some blood measures of pollution exposure, public use files do not give much detail about the child’s residential location, making it difficult to assign pollution exposure. Another problem with linking health and pollution data is that not every area has a monitor. One study using restricted data to link NHANES data to air pollution data was only able to assign county level air pollution values to 43 to 68 percent of respondents, depending on the pollutant (35).

Pollution exposure is often approximated using a child’s residential address

Given the size of the country and geographic distribution of the population, airborne pollutants are not well tracked in the United States, making it difficult to precisely determine the amount of pollution an individual has been exposed to. Ideally, researchers would like to know both contemporaneous and cumulative exposure over a lifetime. Given population mobility, cumulative measures are nearly impossible to obtain, and few studies have attempted to do so.

Large observational studies approximate contemporaneous pollution exposure based on an individual’s general location and the location of ambient pollution monitors, which measure air pollution concentrations hourly, daily, or weekly. Estimating an individual’s pollution exposure based on these monitors is somewhat crude because there can be significant variation in pollution within an area (36), resulting in measurement error in pollution exposure. Recent advancements have improved assignment (37), though it is likely that some error will remain. This error in the measurement of pollution will typically lead to understatement of the effects of pollution on health.

Implications for Policy

More information should be provided about potential pollution exposure

To take one example, the popular smog alert program pioneered in California has been shown to reduce asthma hospitalizations (31), and has now been widely adopted throughout the U.S. By strategically informing people of risks when pollution exceeds a particular threshold, such programs have tremendous potential to improve health. However, too much information can overload people and “dull” behavioral responses (38), so policy makers should use caution when alerting the public. Furthermore, since people change their activities in response to information, the costs of these changes may be nontrivial and need to be recognized.

Optimal pollution policy should be stricter

The studies discussed in this paper typically find that pollutants such as ozone, PM10, and carbon monoxide are harmful to infants at levels below EPA thresholds for concern. Infants and children are some of the most vulnerable members of society and are in need of protection. They cannot make choices for themselves about exposures. Simply providing information on pollutant risks may not be enough. There is evidence that more advantaged people are more likely to take action to avoid risks than those who are less advantaged. For example, Currie (39) shows that college educated white mothers are more likely than others to move into an area following a Superfund cleanup. Therefore, policies that improve information disclosure alone may increase health inequalities, suggesting that stricter standards may also be necessary.

We need better data

In order to learn more about the effects of pollution on children’s health, researchers need data with measures of the health of older children and exposure to pollution. If it proceeds as planned, the National Children’s Study (40), sponsored by the National Institutes of Health (NIH), the Centers for Disease Control and Protection (CDC), and the Environmental Protection Agency (EPA), will help to fill this void. The NCS will follow a nationally representative cohort of approximately 100,000 children from birth to age 21 and will collect many health measures as well as measures of pollution exposures. Ideally, the NCS will also make geographical identifiers such as current street address available to qualified researchers making it possible to more precisely measure proximity to pollution sources. In the meantime, linking pollution measures with administrative data sets from schools, hospitals, and insurers is likely to offer a way forward.

Even low levels of air pollution are linked to poor health outcomes among infants and children. Thresholds for acceptable air pollution levels should be reduced in light of recent evidence, and information on when and where risks of air pollution exposure are highest should be provided to pregnant women, families, and caregivers so that such risks may be avoided.