Development and Field Validation of a Community-Engaged Particulate Matter Air Quality Monitoring Network in Imperial, CA

Introduction

Particulate matter (PM), a collection of liquid and solid particles in the air, has been found to be associated with numerous adverse health outcomes across the life course, including adverse birth outcomes, incident childhood asthma, delayed lung function development, cardiovascular disease development, cancer incidence and premature death (Anderson et al., 2011; Brook et al., 2017; Khreis et al., 2017; Atkinson et al., 2016). Asthmatic children are particularly at-risk from high PM levels due to heightened airway responsiveness, such that air pollution can often cause exacerbation of symptoms (Schwartz, 2004). Imperial County, located in southeastern California, has had levels of PM_2.5 (PM under 2.5 μm in diameter) and PM₁₀ (PM under 10 μm in diameter) that have repeatedly exceeded the U.S. Environmental Protection Agency’s (EPA) National Ambient Air Quality Standards (Imperial County Air Pollution Control District, 2013; California Air Resources Board, 2015). The county also has the second highest rate of childhood asthma emergency department visits in the state (California Department of Health Services, 2014).

The regulatory air-monitoring network in Imperial Valley serves to support compliance with ambient air quality standards, provide air pollution data to the public, and support air pollution research studies. The PM network consists of five sites that measure PM_10, of which two also measure PM_2.5 (California Air Resources Board, 2016). One site is designed to assess concentrations near the United States-Mexico border and the community of Calexico, while the other four measure air quality levels throughout the Imperial Valley, largely in a strip of rural farmland running down the middle of Imperial County where most of the population resides in small communities.

The Imperial County Community Air Monitoring Project (NIH R01ES022722) was designed as a community-engaged research study that partnered public health researchers from the California Environmental Health Tracking Program, environmental justice leaders from a local community-based organization, Comite Civico del Valle (CCV), and various academics with experience in air quality and health effects studies to assess environmental quality needs, conduct community-led air quality monitoring, and identify opportunities in which higher spatial resolution environmental data may affect policy and planning efforts in the Valley.

Early in the study, community residents expressed concern that the existing regulatory network did not adequately measure their exposure to air pollution. They desired higher spatial and temporal resolution data to help make decisions on how to best protect themselves and their children during high pollution events. Furthermore, there was interest in additional monitoring where sensitive sub-populations may be exposed to local pollution levels higher than those observed throughout the rest of the region. Eleven communities were chosen as priority areas for air monitoring by a community steering committee (CSC), and local community members volunteered to participate in a process to identify, map, and collect data on potential monitoring sites (English et al., 2017). In order to cost-effectively monitor 40 sites, the study organizers developed a monitoring platform that consisted of low-cost technologies—a commercially available particle counter, additional sensors for temperature and humidity, and a wireless microcontroller. The networked PM monitors provided increased coverage, in terms of number of monitors compared to the regulatory network, by nearly an order of magnitude and allowed for real-time display of air quality levels.

In the past few years there has been a dramatic increase in the number of low-cost next generation air monitoring technologies (Jiao et al., 2016). However, the performance of these monitors in comparison to existing FEM and FRM air monitors is often poorly documented. The South Coast Air Quality Management District and the U.S. EPA have begun to perform laboratory and field experiments for some monitors and have found that performance varies widely (Williams et al., 2014; South Coast Air Quality Management District, 2015). Most low-cost PM monitors operate based on light scattering. So, depending on particle composition, uncorrected measurements can vary regionally and seasonally. Therefore, it is important to calibrate and validate monitors in the field under typical use conditions. A calibration equation for estimating particle mass concentrations using particle count concentrations produced by optical light scattering monitors can be developed by collocating monitors with FEM or FRM instruments and then validated at other sites.

For this study, we collocated a monitor at the California Air Resources Board (CARB) Calexico-Ethel site, located at the Calexico High School on East Belcher Street, which collects reference measurements of PM from both FEM beta-attenuation monitors (BAMs) and FRM filter-based gravimetric samplers. By comparing FEM and FRM data with data from the community air monitors, an equation for estimating mass concentrations from the Dylos’ particle count concentrations was developed. The calibration equation was validated by comparing calibrated monitor results with PM_2.5 measured by collocated reference instruments at six other sites. Both the Dylos and BAM mass measurements were also converted to an Air Quality Index (AQI) and displayed on a website, allowing community partners, air quality stakeholders, and residents access to real-time calibrated data from the Imperial community air monitoring network (Figure S10).

Materials and Methods

The particle counter used in the community monitors was a modified Dylos 1700. The firmware was changed to increase the number of particle size bins from two to four (>0.5 μm, >1.0 μm, >2.5 μm and >10 μm). A custom circuit board was designed to interface the Dylos with an Arduino Yun to add networking capabilities. The circuit board also integrated a HIH 6300 temperature and relative humidity sensor. Each Arduino Yun was then connected to either Wi-Fi, Ethernet, or a T-Mobile cellular modem. This enabled transmission of the Dylos, temperature, and relative humidity data to a database operated by the University of Washington IT staff in collaboration with the Seto laboratory. The Dylos and networking hardware were installed into a NEMA-6 rated enclosure (Figure 1). Not including labor to construct and maintain the monitor, the cost of the parts for the system was around $1,500.

Figure 1.

(a) Dylos particle counter inside NEMA enclosure. (b) Collocation with regulatory monitor.

The Dylos output readings in units of particle number per 0.01 ft³ every 10 seconds to the Arduino Yun, which collected these values and sent the 5-minute average to the database. For this data analysis, the 5-minute averages were downloaded from the database and averaged to 1-hour values. Any hour with less than 75% data completeness was flagged for quality control and removed. All hours with a particle count ≤ 30 were also removed. Values below this threshold indicated a problem with the Dylos monitor; most often that the photodiode was dirty and the monitor needed to be cleaned. All data analyses were performed using R statistical software.

The Calexico-Ethel site (EPA AQS #060250005) was used for collocation and comparison of the Dylos to reference samplers since it had several PM monitors including a PM_2.5 FRM sampler (R&P 2025), a PM₁₀ FRM sampler (Sierra Anderson 1200), and two PM_2.5/PM₁₀ FEM samplers (MetOne BAM-1020). The PM_2.5 FRM sampler used a 16.67 LPM medium volume pump and a 46.2mm teflon filter, which was weighed before and after sampling to determine PM concentration. The PM₁₀ FRM sampler used a 40 LPM high volume pump and a 46.2mm quartz filter, which was weighed before and after sampling to determine PM concentration. PM_2.5 and PM₁₀ FEM samplers used a beta particle source and detector to determine the amount of radiation absorbed by particles caught on a filter tape, which is proportional to particle mass. The Calexico-Ethel PM samplers are located on top of a trailer in a semi-urban area of Calexico, CA away from local sources and nearby obstructions.

The primary PM_2.5 monitor is an FRM sampler operating on a daily schedule. During 2014 and 2015, in addition to the FRM, a pair of collocated PM_2.5 FEM monitors were operated at the site to collect hourly data. At the beginning of 2016, one of the FEM PM_2.5 samplers was converted to a FEM PM₁₀ sampler, which replaced the existing FRM PM₁₀ sampler. At the same time, the second FEM PM_2.5 sampler was converted to a non-FEM PM_2.5 sampler to continue hourly monitoring for local forecasting. PM_2.5 FRM collocation data were available for the entire study period while FEM PM_2.5 data were available for 2015. The PM_2.5 comparison used data collected from 6/18/2015 to 12/21/2015. The PM₁₀ comparison used data collected from 1/15/2016 to 7/12/2016.

A conversion equation was developed to transform the Dylos particle count concentrations into particle mass concentrations using data from the collocation with the FEM PM_2.5 and FEM PM₁₀ BAM at the Calexico-Ethel site. After the data quality control checks, the resulting data were averaged to 1-day values using two different techniques. First, the 5-minute Dylos data were averaged directly to 1-day data for comparison with PM_2.5 and PM₁₀ FRM gravimetric filters. Second, the 1-hour Dylos data from Calexico-Ethel were averaged to 1-day data for comparison with the PM_2.5 and PM₁₀ FEM BAMs using only hours where both the Dylos and BAMs had data. This was done to ensure that missing data in either dataset did not bias the comparison.

The primary and collocated measurements for both PM_2.5 FEM and FRM were compared to validate the precision and accuracy of CARB’s samplers. Hourly data was averaged for the day, only including days that had at least 18 valid hours or 75% completeness. PM_2.5 FEM daily averages with matching primary and collocated days were compared from June 2015, the start of the Dylos collocation, until December 2015, the end of PM_2.5 FEM monitoring.

The first step toward creating a calibration equation was to investigate which Dylos particle size bin or combination of bins was most highly correlated with PM_2.5 and PM₁₀ as measured by the BAMs and filters. A priori it would be expected that the Dylos would best measure PM_2.5 as bin 1 – bin 3 (all particles >0.5μm minus all particles >2.5 μm), and PM₁₀ as bin 1 – bin 4 (all particles >0.5 μm minus all particles >10 μm). However, the correlations between reference PM data and all individual bins and bin combinations were examined using the coefficient of determination (R²), which measures the amount of variance in the dependent variable that is predicted by the independent variable. For PM_2.5, bin1 and bin 1 – 3 were most highly correlated and there was only a slight difference between them. For PM₁₀, bin 3 and bin 3 – bin 4 were most highly correlated; suggesting that Dylos counts for particles larger than 2.5 μm and between 2.5 μm and 10 μm are most representative of PM₁₀. There was only a slight difference between the correlation for bin 3 and bin 3 – bin 4. The highest correlated bins (bin1 for PM_2.5 and bin3 for PM₁₀, respectively) were used in subsequent analyses.

Finally, a conversion equation was developed using the Dylos counts and FEM PM_2.5 and PM₁₀ BAM data. Relative humidity and temperature were tested as possible covariates and it was found that separately both improved the conversion equation. Relative humidity is known to change particle size due to the addition or subtraction of water from particles (Winkler, 1973). Since temperature and relative humidity were moderately correlated, only relative humidity was included in the conversion equation.

PM_2.5 conversion equation:

$$Dylo{s}_{bin1}={\beta }_0+{\beta }_{1}BA{M}_{PM2.5}+{\beta }_{2}RH+e()$$ (1)

PM₁₀ conversion equation:

$$Dylo{s}_{bin3}={\beta }_0+{\beta }_{1}BA{M}_{PM10}+{\beta }_{2}RH+e()$$ (2)

Where β₀ is the intercept, RH is the relative humidity as measured by the RH sensor on our custom circuit board, and e is the residual error. In these models, we were most interested in the error of the Dylos relative to the reference instrument. Thus, the BAM values appear on the right side of the equation, and the error term explains the residual deviation of the Dylos measurement from the BAM measurement after a constant offset (the intercept) and RH have been accounted for. We assumed that the Dylos will have greater error than the BAM since the BAM is a FEM instrument operated by CARB.

This equation was inverted to estimate BAM-measured mass concentrations in units of μg/m³ from the Dylos measurements:

$$Dylo{s}_{bin1}-{\beta }_{2}R{H}_{Dylos}-{\beta }_0={\beta }_{1}BA{M}_{PM2.5}$$ (3)

$$\left(\frac{1}{{\beta }_1}\right)Dylo{s}_{bin1}-\left(\frac{{\beta }_2}{{\beta }_1}\right)R{H}_{Dylos}-\left(\frac{{\beta }_0}{{\beta }_1}\right)=BA{M}_{PM2.5}$$ (4)

To make this simpler to implement we defined the following constants:

$$c_1=\frac{-{\beta }_0}{{\beta }_1}$$ (5a)

$$c_2=\frac{1}{{\beta }_1}$$ (5b)

$$c_3=\frac{-{\beta }_2}{{\beta }_1}$$ (5c)

Furthermore, BAM_PM2.5 was our estimate of Dylos_{PM2.5_mass}, so:

$$Dylo{s}_{PM2.5mass}=c_1+c_2\ast Dylo{s}_{bin1}+c_3\ast R{H}_{Dylos}$$ (6)

And for PM₁₀:

$$Dylo{s}_{PM10mass}=c_1+c_2\ast Dylo{s}_{bin3}+c_3\ast R{H}_{Dylos}$$ (7)

These equations were applied to data collected from all community monitors in our study.

To validate the community monitors’ measurements and the conversion equation for PM_2.5, CARB deployed MetOne EBAMs with size inlets for PM_2.5 at six of the community monitoring sites from 3/2/16 to 7/19/16 (Figure 2). All EBAMs were operated according to CARB standard operating procedures, and only EBAM hourly average mass concentration data that passed their quality assurance were used in our analyses. The monitors at the validation sites were located on the top of school buildings and private businesses. Monitors were sited away from local sources such as HVAC systems and nearby obstructions such as tall trees.

Figure 2.

Collocations with EBAMs were at: (1) Calipatria; (2) Westmorland; (3) Seeley; (4) El Centro Kennedy; (5) El Centro Meadows; (6) Calexico Alvarez. Collocation with FEM BAM and FRM gravimetric samples was at (7) Calexico-Ethel.

Dylos PM_2.5 mass concentrations were compared to the collocated EBAM PM_2.5 mass measurements. One site, Kennedy, was only continuously online during March. As a result, data from the site after March were excluded from the analysis. Dylos counts were converted to PM_2.5 mass concentrations using eq 6. Dylos and EBAM hourly data were averaged into daily measurements. Daily averages were paired when both values met at least 75% completeness, that is, 18 or more valid hours. The Dylos-EBAM collocation sampling was nearly a 5-month period from March through July 2016. The R² value was calculated for the Dylos PM_2.5 mass measurements and the EBAM PM_2.5 mass measurements. Then an assessment of precision and bias between the samplers was conducted using U.S. EPA air monitoring statistics for PM_2.5 daily values (CFR, 2016). Paired daily averages with both values above 3 μg/m³ were used for the assessment as required by the statistical procedure.

Additionally, separate site-specific conversion equations were created for each site, modeling the relationship between the collocated Dylos and EBAM. These models used the same variables and inversion technique as the Calexico-Ethel conversion equation. The site-specific models were used to convert Dylos counts to PM_2.5 separately for each site. The R² for these Dylos PM_2.5 mass measurements and the EBAMs was calculated and compared to the PM_2.5 Dylos-EBAM R² values achieved using eq 6.

Finally, to provide the community with a high spatiotemporal resolution real-time map of air pollution in Imperial County, the CSC and project partners provided input and determined requirements for the display of the air monitoring data. With this guidance, an existing community environmental reporting website operated by CCV was enhanced to display data from the 40 monitor network (www.ivan-imperial.org/air; Figure S10).

Results and Discussion

Conversion

Daily average PM_2.5 FEM and FRM measurements from the Calexico-Ethel site were compared to assess the precision and accuracy of CARB’s samplers and to ensure that data from these samplers could be used as a reference to compare to the Dylos. The average concentration for the primary and collocated samplers were 12.1 and 11.6 μg/m³, respectively, or 4.2% difference. The correlation for PM_2.5 FEM samplers was very good with R² = 0.899, slope = 0.996, and intercept of 0.54 μg/m³ (Figure S1). PM_2.5 FRM were similarly compared for the same time-period. The average concentration for the primary and collocated samplers were 11.1 and 11.2 μg/m³, respectively, or 0.90% difference. The correlation between replicate PM_2.5 FRM samplers was very good with R²= 0.953, slope = 0.971, and intercept of 0.15 μg/m³ (Figure S2). Matching days for the PM_2.5 FRM and FEM primary were compared as a check on method accuracy. The average concentration for the FRM and FEM were 10.7 and 11.9 μg/m³, respectively, or 11% difference. The correlation for the methods was fair with R² = 0.727, slope = 0.664, and intercept of 2.79 μg/m³. The FEM continuous sampler had a slight positive bias over the FRM method during the study period at the Calexico-Ethel collocation site. The FEM and FRM results support the accuracy of both methods and therefore can be used to model the correction for the Dylos.

Correlation of the community air monitor’s Dylos bins with BAM measurements was performed in order to determine which bin(s) most accurately measure PM_2.5 and PM₁₀ (Table S1). It was found that bin 1 and the difference between the 4^th and 1^st bins (bin 1 – bin 4) were most correlated with PM_2.5, with Pearson correlations of 0.78 and 0.78 respectively, and bin 3 and the difference between the 4^th and 3^rd bins (bin 3 – bin 4) were most correlated with PM₁₀, with Pearson correlations of 0.87 and 0.86 respectively. Bin 1 and bin 3 were chosen to represent Dylos PM_2.5 and PM₁₀ counts, respectively.

The Dylos particle counts compared quite well to the FRM filter and FEM BAM for both PM_2.5 and PM₁₀ (Table 1). The Dylos compared better to the BAM on the daily rather than hourly timescale. The addition of RH or temperature in the Dylos-BAM hourly models increased the adjusted R² by 0.03, however including both only raised the adjusted R² by 0.01 over including either temperature or RH. The reason for this may be because observed temperature and RH values were moderately correlated (r = −0.51). Thus, only RH was included in the conversion equation.

Table 1.

Instrument Comparison

Comparisonsa	PM2.5 Adjusted R2 (n)	PM10 Adjusted R2 (n)
Dylos ~ FRM Filter (day avg)	0.79 (300)	0.77 (32)
Dylos ~ FEM BAM (day avg)	0.84 (160)	0.87 (168)
Dylos ~ FEM BAM (hour avg)	0.78 (3907)	0.77 (4008)
Dylos ~ FEM BAM + RH (hour avg)	0.81 (3907)	0.80 (4008)
Dylos ~ FEM BAM + Temp (hour avg)	0.81 (3907)	0.81 (4008)
Dylos ~ FEM BAM + RH +Temp (hour avg)	0.82 (3907)	0.81 (4008)

^ay ~ x is used to signify a linear model with y = β₀ + β₁*x.

The calculated constants from the regression models were as follows: PM_2.5 c₁ = 4.790, c₂ = 7.879×10⁻³, c₃ = −2.294×10⁻¹; PM₁₀ c₁ = 8.045; c₂ = 2.375×10⁻¹; c₃ = −9.661×10⁻¹. These were used to convert the Dylos count data to mass concentration. There were 3,907 hourly data points for the PM_2.5 conversion and 4,008 hourly data points for the PM₁₀ conversion. The conversion produced some negative Dylos PM mass values, which were not omitted from the following analyses.

The resulting R² between the PM_2.5 and PM₁₀ BAM measurements and Dylos measurements at the CARB Calexico-Ethel site after conversion from particle number to particle mass concentration were as follows: PM_2.5 0.79 (hourly), 0.84 (daily); PM₁₀ 0.78 (hourly), 0.81 (daily). The data are plotted in Figure 3. The correlation for PM₁₀ on the daily scale was lower than the pre-conversion correlation due to a few low PM, high RH days where the model did not perform well.

Figure 3.

Comparison of Dylos-derived mass concentrations after count to mass conversion and BAM for (a) daily PM_2.5 and (b) daily PM₁₀ with R². Dashed line indicates the 1:1 relationship.

Previous comparisons of the Dylos to reference instruments have shown R² values ranging from 0.53 to 0.95. South Coast Air Quality Management District (2015) reported a correlation of 0.63 (hourly) and 0.81 (daily) with a PM_2.5 BAM; Northcross et al. (2013) reported correlations of 0.81–0.99 (hourly) with a TSI DustTrak; Williams et al. (2014) and Manikonda et al. (2016) found a correlation of 0.53 (hourly) with a Grimm Model EDM180; and Steinle et al. (2015) found a correlation of 0.70 and 0.90 (hourly) with a TEOM. The R² for Dylos and BAM in this study, 0.79 for hourly PM_2.5 and 0.78 for hourly PM₁₀, is similar to the Dylos-BAM correlations reported in the literature.

A report by the South Coast Air Quality Management District showed good correlation between a Dylos 1700 and a Grimm and the Dylos response seemed to be linear at concentrations relevant to field use (South Coast Air Quality Management District, 2017); i.e. less than ~70,000 particles per 0.01 ft³, which is above the maximum value seen in this study. However, without calibration, the Dylos overestimated the Grimm. This highlights the importance of calibrating the Dylos before trying to interpret its measurements.

The Dylos PM_2.5 mass concentrations compared well to both the PM_2.5 FEM BAM and the PM_2.5 FRM filter. The correlation for Dylos PM_2.5 to the PM_2.5 FRM filter was R² = 0.792 (slope = 1.413 and intercept of −2.27 μg/m³). The average concentrations for the Dylos PM_2.5 and PM_2.5 FRM filter, which were compared from 6/20/2015 to 7/12/2016, were 14.0 and 11.5 μg/m³, respectively, or 20% difference. The correlation for 24-hour Dylos PM_2.5 to the PM_2.5 FEM BAM was R² = 0.843 (slope = 1.108 and intercept of −1.38 μg/m³). The average concentrations for the Dylos PM_2.5 and PM_2.5 FEM BAM, which were compared from 6/20/2015 to 12/21/2015, were identical at 12.6 μg/m³. The correlation for Dylos PM₁₀ to the PM₁₀ FRM filter was R² = 0.808 (slope = 1.572 and intercept of −27.50 μg/m³). The average concentrations for the Dylos PM₁₀ and PM₁₀ FRM filter, which were compared from 6/23/2015 to 1/19/2016, were 52.2 and 50.7 μg/m³, respectively, or 3% difference. The correlation for the 24-hour Dylos PM₁₀ to PM₁₀ FEM BAM was R² = 0.808 (slope = 1.094 and intercept of −5.21 μg/m³). The average concentrations for the Dylos PM₁₀ and PM₁₀ FEM BAM, which were compared from 1/15/2016 to 7/12/2016, were 55.5 and 55.4 μg/m³, respectively, or 0.18% difference.

Validation

Based on the conversion equation developed for the Calexico-Ethel data, the Dylos data from the six EBAM collocation sites were converted from particle number to particle mass concentration. Some error may have been introduced in the following analyses by comparing the Dylos to an EBAM instead of a BAM, which was the instrument used in the development of the conversion equation. However, there were no extra BAMs available to help validate the conversion equation. In future studies, we would suggest either collocating an EBAM with a BAM to understand how they compare or using more BAMs for validation.

Summary statistics calculated for each site for the Dylos and the EBAMs revealed that in general, with one exception, Dylos averages tended to be lower than the EBAM averages (Table 2). Also, nearly all Dylos had lower variability than the collocated EBAMs. CARB’s quality control protocol for this study invalidated hourly EBAM data below −3 μg/m³ to retain data reading near zero that may have variability while removing very low negative values that were suspect.

Table 2.

Summary Statistics for Hourly Average Paired Data from EBAM Collocation Sites

Site	Dylos (μg/m3)a	EBAM (μg/m3)a	n
Seeley	10.4 (−3.83 to 191, 9.30)	9.01 (−3 to 261, 11.6)	3140
Kennedy	4.76 (−2.86 to 50.2, 4.54)	6.47 (−3 to 66, 6.99)	529
Westmorland	8.53 (−0.26 to 104, 7.17)	10.1 (−3 to 119, 10.4)	842
Meadows	7.31 (−3.10 to 301, 13.9)	9.55 (−3 to 514, 20.3)	1951
Calipatria	7.61 (−5.83 to 174, 12.2)	11.6 (−3 to 239, 15.7)	1688
Calexico Alvarez	13.6 (−1.67 to 92.8, 11.2)	14.1 (−3 to 82, 10.9)	1406

^amean (min to max, standard deviation)

PM_2.5 precision of collocated samplers was measured with a Coefficient of Variation (CV) statistic and used a performance goal within ±10%. Although this U.S. EPA criteria is seldom met in national air monitoring networks and the Dylos-EBAM monitoring was not using identical samplers, CV and bias values provide useful indicators of performance (Table 3). Bias was the average percent difference among daily pairs with one sampler being the audit or reference sampler. The EBAM was used as the reference sampler, therefore, a positive bias value indicates that the Dylos was measuring higher than the EBAM, conversely, a negative bias value indicates the Dylos was measuring lower than the EBAM. The average bias of all six collocations was −4.7% with a range of 28.3 to −31.4%. The average precision of all six collocations was 25.2% with a range of 17.5 to 35.2%. As a comparison, the CARB Calexico-Ethel PM_2.5 FRM filter based collocated precision was 22.8 and 8.2%, in 2015 and 2016, respectively. Given that the FRM-FRM comparison was using identical samplers while the Dylos-EBAM had different methodologies, and should be expected to be larger, the precision for the Dylos-EBAM appears to be reasonable for good measurements.

Table 3.

Precision and Bias Statistics for Daily Paired Data from the Collocation Sites

Site	Sampler Pair	Number of Pairs	Collocated Precision CV(%)	Bias (%)
Seeley	EBAM-Dylos	124	24.4	28.3
Kennedy	EBAM-Dylos	13	35.2	−6.8
Westmorland	EBAM-Dylos	35	23.5	−7.0
Meadows	EBAM-Dylos	67	24.7	−14.2
Calipatria	EBAM-Dylos	64	17.5	−31.4
Calexico-Alverez	EBAM-Dylos	57	25.7	3.0
Average			25.2	−4.7
Calexico-Ethel	FRM-FRM 2015	26	22.8	−
Calexico-Ethel	FRM-FRM 2016	28	8.2	−

Time-series plots, scatterplots, and residual plots were created for each site. In general, the Dylos measurements tracked similarly to the EBAM measurements over time and the Dylos detected similar peaks as the EBAM (Figures S3-S8). As can be seen from the scatterplots, the relationship between the Dylos and EBAMs was approximately linear, however the Dylos tended to underestimate PM_2.5 mass at higher concentrations (Figure 4).

Figure 4.

Scatterplots of EBAM and Dylos conversion with R². Dashed line indicates the 1:1 relationship.

To evaluate the relationship between the Dylos conversion over time and the EBAM measurements, the differences were plotted as a function of time, and fit to a linear regression (Figure S9). Most sites had slightly negative slopes for the differences, which seems to point to a gradually increasing under-estimation by the Dylos over the five months of the study. This may be due to dust accumulation on the photodiode, which causes the monitors to read lower. During this study the Dylos were switched out at these sites when they began reading too low (consistently under 30 particles per 0.01 ft³ in bin 1), but no preventative maintenance was performed. Based on our experience with the Dylos, a twice yearly preventative maintenance would be beneficial.

The intercepts were not uniformly negative or positive, pointing to a site-specific variation in the Dylos-EBAM relationship. In particular, the Calipatria site had a −4.1 intercept and 0 slope, indicating that the Dylos consistently read 4.1 μg/m³ lower than the EBAM, and the Seeley site had an intercept of 3.5 and a slope of −0.001, indicating that they Dylos consistently read 3.5 μg/m³ higher than the EBAM. The non-zero intercepts seen at Calipatria and Seeley may be due to an instrument problem or the placement of the EBAM and community monitor at those sites. At Calipatria, the EBAM was located on a secondary roof approximately 10ft higher than the roof on which the community monitor was placed. Whatever the reason, in future investigations we would like to have an extra Dylos and an extra EBAM to move around to different sites during the study period in order to see if shifts like these are due to instrument specific variations. The use of a rover community monitor to collocate for a period of time at sites of interest could also be used to verify performance.

It was hypothesized that the relationship between the Dylos and EBAM might change by site due to site-specific variations in particle composition. A particle’s optical properties, and therefore the Dylos response, is dependent upon its composition. This was tested by comparing the correlation between EBAM and Dylos mass concentration using the Calexico-Ethel conversion versus the site-specific conversion (Table 4). The first column is the R² for Dylos PM_2.5 mass (Calexico-Ethel model) and EBAM PM_2.5 mass. There was heterogeneity between sites with R² values ranging from 0.35 to 0.81. The average R² between the Dylos PM_2.5 mass (Calexico-Ethel model) and EBAM PM_2.5 across all sites was 0.59. The lower R² values for Kennedy (0.37) and Calexico Alvarez (0.35) were due to the smaller range of mass concentrations seen at those sites (Figures 4, S7-S8). The R² values for the four other sites when restricted to concentrations below 93 μg/m³, the maximum concentration seen at Kennedy and Calexico Alvarez, were between 0.31 and 0.62. The second column contains the R² values for Dylos PM_2.5 mass (site-specific model) and EBAM PM_2.5 mass. There was only a small increase in the correlation (0.01 to 0.03) for each site when using a site-specific conversion equation. This suggests that there was only a marginal improvement, if any, to be gained by employing a site-specific model. To our knowledge, the Dylos has never been compared to an EBAM in the literature, but these correlations are within the range of correlations seen between the Dylos and other reference instruments mentioned previously.

Table 4.

Correlation between EBAM and Dylos PM_2.5 Mass Concentration Using the Calexico-Ethel Conversion and a Site-Specific Conversion

Site	n	R2 (Calexico-Ethel Model)	R2 (Site-Specific Model)
Seeley	3140	0.63	0.64
Kennedy	529	0.37	0.38
Westmorland	842	0.62	0.67
Meadows	1951	0.75	0.75
Calipatria	1688	0.81	0.83
Calexico Alvarez	1406	0.35	0.36

Data Completeness

One of the limitations we encountered in this study was loss of data at some of our community monitoring sites. Approximately 40% of the Dylos data were lost across the collocation sites. Data loss occurred due to failure of the Dylos monitor and issues with instability of wifi connections, which often caused the microprocessor to freeze and fail to record data to the on-board SD card. This data loss ranged from <1% to 83% across the six sites (Table S2). The reason for differing data loss across sites was due to both environmental conditions and network type. Location was important to data loss because it determined whether or not the Dylos was exposed to the high dust conditions that can occur in the north of the Imperial Valley (Figure 2). Both the Calexico-Ethel site used for the conversion model and the Calexico Alvarez site, which are located in southern Imperial County, did not have any problems with the Dylos needing to be cleaned. Three of the sites in the central or northern part of the valley needed to have the Dylos cleaned one or more times. As for network connections, the Calexico-Ethel and Kennedy sites used a cellular modem to upload data and Seeley and Meadows used Ethernet; both were very reliable and, except for two short network outages at Calexico-Ethel, experienced basically no data loss. While each monitor had an SD card for internal data storage, network outages would cause the microcontroller to crash and therefore not record any data. This is an issue with the specific type of microcontroller, which is no longer used in our newer monitors.

Generally, ongoing maintenance has been a recurring issue in the study and will remain an issue as we try to sustain community-led monitoring in the valley. Our field crew, a single part-time CCV staff member, has limited resources to dedicate to network maintenance. However, by implementing cellular connections and a more rigorous maintenance schedule our total data loss dropped from 44% to 18% between the study period and 1/1/17–3/1/17. This exceeded our original data completeness goal of 75%.

Conclusions

Our study illustrates the importance of understanding the performance of low-cost next generation air quality monitoring technologies under particular use scenarios. Calibration and validation of low-cost air monitors with regulatory instruments is crucial to ensure quality data. This is particularly important for community air monitoring, in which the data are displayed back to the community members. It should be considered best practice for other communities engaging in community air monitoring to do similar evaluations if possible.

Furthermore, augmenting basic air monitors with relative humidity data and network connectivity can allow for more robust conversion from particle count to particle mass and provide the possibility for real-time data quality control, analysis, and reporting. The results from this study suggest that measurements from the Dylos monitor, with added RH measurements, are correlated with multiple reference instruments and can be used for a networked community monitoring system to augment the existing regulatory network in Imperial and provide higher spatial and temporal data, particularly for susceptible populations.

Although we are very clear that our community air monitoring data are non-regulatory, the interest that we received to collaborate to evaluate new instruments from regulatory agencies like CARB and other academic researchers, suggest strong interest in understanding the performance limitations of new monitoring approaches. Ultimately, we hope that other communities and air quality stakeholders will increasingly embrace community-led efforts to augment regulatory monitoring.

About the Authors

Graeme Carvlin is a graduate student and Edmund Seto is a professor in the Department of Environmental and Occupational Health Sciences at the University of Washington in Seattle, WA.

Humberto Lugo, Luis Olmedo, and Ester Bejarano work for Comite Civico del Valle in Imperial, CA.

Alexa Wilkie, Dan Meltzer, Michelle Wong, Galatea King, and Paul English work at the California Environmental Health Tracking Program in Richmond, CA.

Amanda Northcross is a professor at George Washington University in Washington, DC.

Michael Jerrett is a professor at UCLA in Los Angeles, CA.

Donald Hammond works at the California Air Resources Board in Sacramento, CA.