Table 1.

Characteristics, research questions, and study findings, strengths and limitations of each natural experiment.

Study and Location	Description of Natural Experiment	Chain of Accountability Question	Study Objective(s)	Main Findings	Major Strengths	Major Limitations
Pope et al. 1989, 1991, 1992a, 1992b; Ransom et al. 1992 Utah Valley, Utah	Closure of a large steel mill from August 1986 to September 1987	Did reduced emissions lead to improved air quality and a beneficial human health response?	• To compare PM10 concentrations and the number of health events during strike to before and after strike	Average PM10 concentration reduced from 90 μg/m3 to 51 μg/m3 During strike, reduced: • hospital admissions for pneumonia, pleurisy, bronchitis, and asthma, • childhood school absenteeism, • bronchitis and asthma admissions for pre- school aged children, • respiratory symptoms and reduced peak expiratory flow, • total non-accidental mortality, respiratory mortality, and cardiovascular mortality	• Use of already collected air pollution and hospital admissions and mortality data • A-B-A design to control for confounding by time trends in mortality/morbidity	• Did not have simultaneous temporal comparisons in other Utah counties (control counties) to provide additional control for confounding by time trends
Parker et al, 2008 Utah Valley and other counties in Utah	Closure of a large steel mill from August 1986 to September 1987;	Did reduced emissions lead to a beneficial human health response?	To compare the preterm birth rate during the strike to before and after strike, both in counties in the Utah Valley, and other Utah counties	• Utah Valley mothers who were already pregnant at the time of mill closure were less likely to deliver prematurely than mothers pregnant before or after the closure. • No pattern in mothers from other Utah counties	• Use of already collected birth data • A-B-A design to control for confounding by time trends • Simultaneous temporal comparison in other Utah counties (control counties) provides additional control for time trends	• 13 month period too long to examine specific windows of pregnancy • No examination of air pollution changes in other counties • No complimentary analysis of air pollution and preterm birth to confirm that air pollution changes drove changes in Utah Valley preterm birth rate
Pope et al, 2007 New Mexico, Arizona, Utah, and Nevada	Nationwide copper smelter strike from July 1967 to April 1968	Did reduced emissions lead to a beneficial human health response?	• Compared monthly mortality rates during 8.5 months of smelter strike to rates before and after strike (1960 – 1975)	• Estimated reduction of suspended sulfate of ~2.5 μg/m3 • 1.5% - 4.0% decrease in mortality during strike compared	• Use of already collected mortality data • A-B-A design to control for confounding by time trends • Adjustment for regional and national cardiovascular and respiratory mortality rates to control for confounding by background trends in cardiorespiratory mortality • Adjustment for nationwide influenza/pneumonia rates to control for confounding by national or regional epidemics • Adjusted for monthly mortality counts in neighboring states (control states) to account for confounding by time trends.	• No complementary estimation of change in mortality associated with sulfate or other pollutant concentrations
Clancy et al, 2002 Dublin, Ireland	Ban on marketing and sale of coal in Dublin starting in September, 1990	Did reduced emissions lead to improved air quality and a beneficial human health response?	•Compared black smoke and sulfur dioxide concentrations and cause specific mortality rates in the 6 years before and after the ban	Black smoke reductions: • 50.2 to 14.6 μg/m3 in all seasons • 85.4 to 21.5 μg/m3 in winter Sulfur dioxide reductions: • 33.4 to 22.1 μg/m3 in all seasons • 40.4 to 24.9 μg/m3 in winter Estimated mortality reductions: • total non-trauma: −6% • respiratory: −16% • cardiovascular: −10%	• Use of nationally- maintained mortality data • Use of already collected black smoke and sulfur dioxide measurements • Adjustment for influenza epidemics to control for other causes of respiratory mortality	• Adjustment for national cause- specific mortality rates, does not completely control for background mortality rate changes, resulting in residual confounding by time trends • Population size and age distribution changed substantially during study period. Could only estimate population size, potentially resulting in bias
Dockery et al, 2013 and Goodman et al, 2009 • Cork, Ireland • Arklow, Drogheda, Dundalk, Limerick, & Wexford, Ireland • Dublin, Ireland (reanalysis)	Ban on marketing and sale of coal in • Cork (1995) • 5 cities (1998) • Dublin (1990, reanalysis)	Did reduced emissions lead to improved air quality and a beneficial human health response?	• To compare cause specific mortality rates and hospital admissions in the 5 years before and after the ban in Cork, 5 Cities, and 12 Midland Counties (as a comparison) • To compare cause specific mortality rates in the 10 years before and after the ban in Dublin (re- analysis)	Black smoke reductions (all seasons): • Cork: −49%, 33.7 to 17.2 μg/m3 • 5 Cities: −48% to −61% • No reductions in total and cardiovascular mortality in any city • Significant reductions in respiratory mortality in Dublin (−17%), and non-significant reductions in Cork (−9%) and 5 Cities (−3%) • Dublin ban associated with non-significant small changes (−2% to −3%) in cause-specific mortality rates in 12 Midland counties	• Use of nationally- maintained mortality data for 10 years rather than 6 years, resulting in better control for time trends • Use of already collected black smoke measurements • Adjustment for influenza epidemics to control for other causes of respiratory mortality • Adjustment for mortality rates in coastal counties rather than entire country likely better control of confounding than Clancy et al (2002) • Assessment of 12 Midland Counties as a comparison	• Population size and age distribution changed substantially during study period. Could only estimate population size, potentially resulting in bias
Heinrich et al, 2002; Former East Germany	German reunification	Did improved air quality lead to a beneficial human health response?	• Cross- sectional surveys of children in 3 phases to assess prevalence of respiratory disorders • Estimates of change in respiratory disorders associated with changes in TSP and SO2	• Each 50 μg/m3 increase in TSP associated with increased prevalence of: • bronchitis (202%) • sinusitis (158%) • frequent colds (90%) • febrile infections (79%) • Each 100 μg/m3 increase in SO2 associated with increased prevalence of: • bronchitis (172%) • sinusitis (126%) • frequent colds (81%) • febrile infections (76%)	• Recruited only children living in study areas for 2+years and who had not relocated from more than 2km away • Prospective health data collection should minimize misclassification • Comparison of same children across study periods • Adjustment for multiple measures of socioeconomic status and other environmental contaminants	• Potential for residual confounding by a change to western lifestyle a • Potential for residual confounding by time trends due to no control population experiencing same health rates, but not air pollution changes
Luechinger et al, 2014 Germany	Mandated scrubber installations at all power plants	Did reduced emissions lead to a beneficial human health response?	• Using national mortality and birth data, they estimate effects of this mandated scrubber installations and air pollution reductions on neonatal mortality and the number of infants with low birth weight and length	Each 1 μg/m3 decrease in annual county average SO2 concentration associated with 2.6 less infant deaths per 100,000 births •Decreased SO2 concentrations also associated with increased infant lengths and weights	• Use of already collected air pollution data • Use of already collected birth and infant mortality data for a large population	• No control population to know if similar patterns of lung function improvement observed in areas without an air pollution concentration decrease • Assessed only 1 pollutant and thus could not evaluate associations between these same outcomes and concurrent changes in the concentrations of other pollutants
Gauderman et al, 2015 Long Beach, Mira Loma, Riverside, San Dimas, and Upland, California	Air quality control policies targeted at mobile and stationary sources have reduced air pollutant levels over past few decades in California	Did improved air quality lead to a beneficial human health response?	• Assessment of changes in lung function from ages 11 to 15 associated with decreases in air pollutants during the same time, in children from 3 separate cohorts corresponding to 3 separate calendar periods (1994- 2011), as part of the Children’s Health Study	• Decreased NO2, PM2.5, and PM10 concentrations associated with increases in FEV1 and FVC in children from 11 to 15 years of age	• Large, well characterized cohort with prospective pulmonary function measurements • Use of available air pollution data • Within community analyses, presumably resulting in no confounding by differences between communities	• Potential residual confounding by changes in racial/ethnic composition and other population characteristics within a community over time • No control population possible, to know if similar patterns of lung function improvement observed in areas without an air pollution concentration decrease
Hedley et al, 2002 Hong Kong	Reduction in sulfur content of fuel oil used in power plants and on- road vehicles from 1990- 1995	Did a regulatory action lead to improved air quality and a beneficial human health response?	• To compare changes in monthly deaths and monthly mean air pollutant concentrations in Hong Kong between 1985 and 1995	• Reductions from before to after intervention in SO2 (mean change = −45% over 5 years after intervention) • Seasonal reductions in respiratory, cardiovascular, and total mortality after the intervention	• Use of already collected air pollution and mortality data	• Although likely minimal, residual confounding by infectious disease mortality including influenza is possible • A-B design may result in residual confounding by time trends in mortality and air pollution • No control population could be used to determine if similar changes in seasonal patterns of mortality were observed elsewhere
Kelly et al, 2011 London, England	In 2008, low emission zone established in London (2644 km2) to restrict entry of old and polluting diesel vehicles, but not cars or motorcycles	Did a regulatory action lead to improved air quality?	• To determine whether reductions in congestion, traffic volume, and air pollution concentrations within London zone (2644 km2) were observed • To assess whether a primary care dataset in London could be used to assess whether any reduction in traffic and air pollution had a health benefit	• 26% reduction in the excess delay per kilometer • Reduced traffic volume during the charging hours compared to before the charging scheme was implemented • Modeling efforts predicted 3.8% reductions in NOx and − 2.6% reductions in PM10 emissions after 2 years • Pilot analyses found no association between NOx concentrations and prevalence of respiratory outcomes	• Use of both traffic and air pollution data to assess environmental impacts of policy • Use of primary care dataset to provide comprehensive health data for the population of interest	• Some primary care providers were concerned that patient confidentiality could not be certain and thus did not make some health records available for inclusion in the study, and thus statistical power for future analyses of any health benefits of the low emission zone may be reduced
Currie and Walker, 2011 New Jersey and Pennsylvania, neighborhoods near highway toll plazas	E-ZPass, an electronic toll collection system not requiring cars to stop at toll plazas, was installed at toll plazas in New Jersey and Pennsylvania from 1997 to 2000	Did a policy lead to a beneficial health response?	• To determine whether implementation of E-ZPass resulted in improved birth outcomes among women living near the toll plazas	• Introduction of E-ZPass reduced preterm births (<37 weeks gestation) by 10.8% and low birth weight (<2500g) by 11.8% among mothers living within 2 km of a toll plaza, relative to mothers living 2-10km from a toll plaza	• Use of existing birth records/data from both states • Use of a control population to allow comparison of changes in the frequency of preterm birth and low birth weight before and after E-ZPass, both in mothers living within 2 km of a toll plaza, and mothers living 2-10km from a toll plaza	• No air pollution monitoring done to determine the change in air pollutant concentrations after E-ZPass installation in areas near the toll plazas (<2km) and farther away from the toll plazas (2- 10 km) to confirm E-ZPass program is driving health benefit • No available air pollution monitoring data at toll plazas to assess air pollutant concentration changes there
Beatty and Shimshack, 2011 Puget Sound area of Washington State	Localized emissions reduction program in Washington state to retrofit diesel school buses with aggressive pollution control technologies	Do regulatory actions result in a beneficial health response?	• To determine whether child and adult residents of school districts where diesel bus retrofits were adopted had a greater reduction in monthly counts of hospital admissions for bronchitis, asthma, pneumonia, and pleurisy after the retrofits compared to before the retrofits, relative to residents of the non-adopter school districts	• School bus retrofits induced large reductions in monthly counts of respiratory hospital admissions for both children and adults with chronic conditions • Adopter school districts experience 23% and 37% fewer admissions for bronchitis/asthma and pleurisy/pneumonia in children, relative to non- adopter school districts	• Use of a large existing statewide dataset on hospital admissions likely captured nearly all health outcomes requiring hospitalization in this population • Use of a control population to allow comparison of changes in respiratory outcomes from before to after the retrofits, both in residents of school districts adopting and not-adopting the diesel bus retrofits for their school buses • Several sensitivity analyses conducted to assess whether the control population (non- adopter districts) were similar to adopter districts with regard to demographics and health outcomes before the retrofit program was begun	• No air pollution monitoring done to determine the change in air pollutant concentrations after school bus retrofits in adopter districts relative to non-adopter districts
Friedman et al, 2001 Atlanta, Georgia	1996 Atlanta Olympics – policies to reduce vehicular traffic and congestion	Did a regulatory action lead to improved air quality and a beneficial human health response?	• To determine whether there were reductions in hospital, emergency room, and urgent care center visit data for children (aged 1-5 years) residents of 5 metropolitan Atlanta counties during the games • To determine whether there were reductions in central site PM10, CO, and O3 measurements from the same 5 counties during the games	• Reduced peak daily O3 concentrations (81.3 ppb before games to 58.6 ppb during games), and 22.5% reduction in morning traffic counts during Olympics compared to 4 weeks before and after games • 41.6% reduction in acute care and hospitalizations for asthma during Olympic Games (4.23 to 2.47 daily events)	• Use of existing hospital and emergency room data • Use of existing air pollution data • A-B-A study design to control for confounding by time trends	• Only 73 day study period and only 17 day period of Olympics gave limited statistical power • Potential for residual confounding by seasonal patterns in pollution and hospitalizations • Potential for residual confounding by Atlanta residents changing behaviors and personal activity levels during Games
Peel et al, 2010 Atlanta, Georgia	1996 Atlanta Olympics – policies to reduce vehicular traffic and congestion	Did a regulatory action lead to improved air quality and a beneficial human health response?	• To determine whether there were reductions in hospital and ER admissions data from 1995 to 2004 • To determine whether there were reductions in air pollutant concentrations (PM10, 8-hour maximum O3, 1 hour maximum O3, NO2, NOx, SO2, and CO) measured at stations both within and outside Atlanta • Used traffic data from 18 sites within the 5 county area	• 20%-30% reductions in ambient ozone concentrations in Atlanta, but similar declines in ozone concentration were observed throughout Georgia and the southeastern United States during this time • 2% to 20% reductions in weekday peak morning traffic counts, consistent with, but smaller in magnitude43 than those of Friedman et al (2001). • No reduction in cardiovascular or respiratory ED visits during the Olympics, which was inconsistent with the earlier work by Friedman et al (2001)	Compared to Friedman: • Longer time period with health and air pollution data allowed better control of season and long term time trends) • Traffic data from 18 sites within the five counties better represented regional traffic patterns • More spatially diverse air pollution data Overall: • Large, well characterized ER dataset within a large urban area	• Only 73 day study period and only 17 day period of Olympics gave limited statistical power • Air pollution reduction may have not been large enough to elicit a detectable health response • lack of a ‘control’ area/county/city with a similar population where such a traffic reduction system was not available, which may result in residual confounding by time trends
Zhang et al, 2013; Rich et al, 2012; Huang et al, 2012 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to improved air quality and a beneficial human health response?	• In a panel study of healthy young medical residents, were the levels of cardio- pulmonary biomarkers and (measured twice before, during, and after Olympics) and multiple air pollutants lower during the games compared to before and after the games	• 18% to 60% reductions in all pollutants, but ozone (20% increase) • Large reductions in most biomarkers, but not heart rate variability markers.	• A-B-A design to control for time trends • Prospective measurement of all health and pollution data • Measurement of biomarkers allowing investigation of impacts of air pollution on physiologic mechanisms • Assessed numerous pollutants including ions and PM components	• All pollutants, but ozone, were reduced simultaneously making assessment of health responses associated with individual pollutants difficult • These are healthy young subjects, and thus not those where such biomarker changes would be indicative of actual clinical events (e.g. myocardial infarction, stroke)
Mu et al, 2014 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to a beneficial human health response?	• In a panel study of healthy adult subjects (20-65 years of age) living in Haidan district of Beijing, were there improvements in peak expiratory flow, blood pressure, and respiration rate, measured once before, once during, and once after the Olympics, as well as PM1, PM2.5, PM7, PM10 and TSP measured at 1 centrally located monitor	• All air pollutant concentrations were 54%-60% lower during the games compared to before the games • Peak expiratory flow increased in 80% of study subjects from before to during the Olympic Games. •Percent of subjects with a fast respiration rate (>20/min) decreased during the games compared to before the games, and increased after the games compare to the games • No clear pattern of blood pressure change across periods	• A-B-A design to control for time trends • Prospective measurement of all health and pollution data • Measurement of biomarkers allowing investigation of mechanism	• Potential for residual confounding by a change in personal activities by study subjects during the games, compared to before and after the games.
Lin et al, 2011 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to improved air quality and a beneficial human health response?	• In a panel study of 36 elementary school aged children ( with 5 observation periods for each subject over a 2 year study period, were there lower levels of exhaled nitric oxide (made at multiple times during each observation period) , and black carbon (BC), PM2.5, NOx, SO2, and CO concentrations (measured at a site 650 m from the study school location) during the games compared to before and after the games	• BC and PM2.5 concentrations substantially lower during games, compared to before the games • Increases in BC (4.0 μg/m3) and PM2.5 (149 μg/m3) associated with 16.6% and 18.7% increases in exhaled nitric oxide, respectively	• Prospective measurement of all health and pollution data • Measurement of biomarkers allowing investigation of mechanism	• Did not have A- B-A design to control for time trends • Potential for residual confounding by a change in personal activities by study subjects during the games, compared to before and after the games. • Did not directly estimate change in exhaled nitric oxide levels during the Olympic Games (Visit 5 period) compared to the Pre-Olympic periods (Visits 1- 4)
Su et al, 2015 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in all emissions	Did a regulatory action lead to improved air quality and a beneficial human health response?	• Time series analysis of cardiovascular mortality in Beijing residents •Cardiovascular mortality data obtained from Beijing Center for Disease Control for residents of Beijing • Particle number size distribution measurements made on Peking University campus • NO2, PM10 data collected from 8 Beijing monitors • PM2.5 measured at 1 location ~5km from Peking University site	• 15.9% to 54.1% reductions in all air pollutant concentrations during games compared to before the games, with increases after the games • 8.8% decreased in cardiovascular mortality associated with interquartile range increases in 1 and 5 day average ultrafine particle counts	• Use of already collected mortality and NO2 and PM10 data • A-B-A design to control for time trends • Coupled with other Beijing Olympic studies investigating mechanistic biomarkers, this study’s assessment of cardiovascular mortality provides information on important clinical outcomes, thereby presenting a more complete assessment of health impacts of the air pollution reductions during the Beijing Olympics in adult Beijing residents	• Potential for bias in that Beijing residents may have left Beijing during the games, and thus not be included in a daily count of cardiovascular deaths used in the study. However, authors argue against this since there were no public holidays during the games, and thus little chance to leave the city during the games.
Li et al, 2010 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to a beneficial human health response?	• Time series analysis of daily counts of outpatient asthma visits for adult residents of urban areas of Beijing • PM2.5, O3, SO2, NO2, and CO measurements at 3 stations in urban Beijing areas	• Number of outpatient visits for asthma were lower (7.3/day) during the games compared to the baseline period (June 2008; 12.5/day) • PM2.5 substantially lower (35% to 41%) during the games (46.7 μg/m3) compared to the baseline period (78.8 μg/m3) and pre-Olympic period (72.3 μg/m3) • Increases in PM2.5 (10 μg/m3) and O3 (10 ppb) concentration associated with 2.0% to 4.4% increases in asthma visits	• Use of already collected pollutant and asthma outpatient visit data • Coupled with other Beijing Olympic studies investigating mechanistic biomarkers, this study’s assessment of asthma visits provides an assessment of important clinical outcomes, thereby presenting a more complete assessment of health impacts of the air pollution reductions during the Beijing Olympics in adult Beijing residents	• A-B design may result in residual confounding by time trends and season • Potential for bias in that Beijing residents may have left Beijing during the games, and thus not be included in a daily count of asthma outpatient visits used in the study
He et al, 2016 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to a beneficial human health response?	• To examine association between increases in monthly concentrations of PM10 and monthly mortality rates from January 2006 to December 2010	• Each 10 μg/m3 reduction in monthly PM10 concentration associated with 8.36% to 9.61% reduction in monthly all-cause mortality, and a 8.78% reduction in monthly cardiovascular or respiratory mortality	• Use of large, existing dataset of mortality in Beijing residents • Compared changes in monthly PM10 concentration and mortality rates between cities experiencing large reductions in PM10 (treated cities) and those with little to no reduction (control cities)	• Although plots suggest there was a greater proportional reduction in PM10 concentrations and monthly mortality counts in the cities with large reductions in PM10 compared to cities with small concentration reductions, there was not a multivariable analysis to rule out confounding by other temporal and spatial characteristics
Rich et al, 2015 Beijing, China	2008 Summer Olympics – Driving restrictions, industry and construction shutdowns, drastic reductions in emissions from numerous sources	Did a regulatory action lead to a beneficial human health response?	• Cohort study using birth registry data • Pregnant women living in 4 Beijing districts • Same air pollution as Zhang et al (2013)	• Babies whose 8th month of pregnancy was during Olympics were 23 g heavier than babies with their 8th month of pregnancy during the same calendar dates in 2007 or 2009 • No clear pattern for other months of pregnancy • Increases in concentrations of PM2.5 (19.8 μg/m3), NO2 (13.6 ppb), SO2 (1.8 ppb), and CO (0.3 ppm) during 8th month associated with 17-34g decreases in birth weight	• A-B-A design to control for confounding by time trends • Use of large birth registry to provide ample statistical power • Complimentary analyses of: 1. Changes in birth weight from before to during the games 2. Change in birth weight associated with increased air pollutant concentration	• Potential for residual confounding by women’s personal activities during Olympics • No complimentary measurement of biomarkers of potential mechanisms linking air pollution exposure to fetal growth restriction • No control population experiencing same trends in birth weight and fetal growth, but not the air pollution reductions during the Olympics
Lee et al, 2007 Busan, South Korea	2002 Asian Games, 14 days of traffic volume control	Did a regulatory action lead to a beneficial human health response?	Time series analysis of childhood asthma hospitalization rate in year of Asian Games compared to the 3 years before and 1 year after	• 1%-25% reductions in air pollutants in the Post- games period compared to the Pre- and During- Games period • Reduced risk of asthma hospitalizations (RR = 0.73) in children in the “Post-Games” period compared to the “Pre- and During-Games” period • Increases in concentrations of NO2 (15.09 ppb), SO2 (3.64 ppb), PM10 (28.82 μg/m3) and O3 (12.39 ppb) associated with 24% to 35% increases in the risk of hospital admissions for asthma	• Complimentary analyses of: 1. Changes in a health outcome across 2 periods 2. Change in a health outcome associated with increased air pollutant concentration • A-B-A design comparing year of Asian Games (2002) to years before and after to control for long term time trends	• 14 day period may not be long enough to compare “During-Games” period to “Pre- Games” period as in Beijing Olympic studies • No control population to know if similar patterns of hospitalization and air pollutant concentrations were observed elsewhere