Size-Resolved Field Performance of Low-Cost Sensors for Particulate Matter Air Pollution

Abstract

Particulate matter (PM) air pollution is a major health hazard. The health effects of PM are closely linked to particle size, which governs its deposition in (and penetration through) the respiratory tract. In recent years, low-cost sensors that report particle concentrations for multiple-sized fractions (PM_1.0, PM_2.5, PM₁₀) have proliferated in everyday use and scientific research. However, knowledge of how well these sensors perform across the full range of reported particle size fractions is limited. Unfortunately, erroneous particle size data can lead to spurious conclusions about exposure, misguided interventions, and ineffectual policy decisions. We assessed the linearity, bias, and precision of three low-cost sensor models, as a function of PM size fraction, in an urban setting. Contrary to manufacturers’ claims, sensors are only accurate for the smallest size fraction (PM₁). The PM_1.0–2.5 and PM_2.5–10 size fractions had large bias, noise, and uncertainty. These results demonstrate that low-cost aerosol sensors (1) cannot discriminate particle size accurately and (2) only report linear and precise measures of aerosol concentration in the accumulation mode size range (i.e., between 0.1 and 1 μm). We recommend that crowdsourced air quality monitoring networks stop reporting coarse (PM_2.5–10) mode and PM₁₀ mass concentrations from these sensors.

Introduction

Airborne particulate matter (PM) is a major public health concern. Exposure to PM contributes to over a million premature deaths worldwide, and populations subject to long-term exposure suffer from significantly higher cardiovascular and respiratory morbidity.¹⁻³ The atmospheric fate and transport of PM is largely determined by particle size (d_p, in μm), as is the penetration and deposition of PM within the human respiratory tract.⁴ How particle size relates to PM health effects remains an active area of research; human exposure to size modes such as ultrafine (d_p < 0.1 μm), fine (d_p < 2.5 μm), and coarse (2.5 < d_p < 10 μm) PM have each been associated with adverse health outcomes.^5,6 Traditional (reference) methods for measuring size-resolved particle concentrations are expensive (e.g., instrument costs from $10,000 to $100,000 each) and resource intensive,⁷ limiting their use to spatially sparse outdoor monitoring networks in high-income countries and research studies with short sampling durations and small samples sizes.

The emergence of low-cost PM sensors (miniaturized, mass-produced devices that cost ∼$15 to $50 each) has facilitated the deployment of cheaper (∼$250/each) monitors in crowdsourced measurement networks (e.g., PurpleAir, Clarity) that are denser than traditional national/regional-scale networks.⁸ These networks are being leveraged to support growing interest in PM exposure and health science globally. Most low-cost PM sensors operate on the principle of aerosol light scattering: a fan draws PM into a small housing, the PM passes through a focused beam of light, and a photodetector measures the intensity of the light scattered by the particles.⁸ Most low-cost PM sensors report mass and number concentrations across a range of sizes (e.g., PM_0.3–0.5 PM_0.5–1.0, PM_1.0–2.5 PM_2.5, PM₁₀). By including concentrations across multiple-sized bins as sensor outputs, manufacturers specify, either explicitly or implicitly, that their sensors can classify particles by size. However, manufacturer documentation often omits the working principle of the sensor (i.e., whether it functions as an optical particle counter or a nephelometer) and the bias/precision of size-resolved outputs.

Particle sizing is challenging even for reference-grade instruments, especially those that rely only on light scattering.⁹ For example, the signal produced by a given particle can be below the limit of detection, saturated, ambiguous (e.g., when two particles coincide in the detector), or inherently biased by aspiration or transmission errors during sampling. Sensor evaluations carried out by manufacturers and third parties generally involve evaluating the accuracy and precision of cumulative mass concentrations (e.g., PM_1.0, PM_2.5, PM₁₀). This approach does not provide sufficient information to assess a sensor’s ability to quantify and classify particles in specific size ranges. Laboratory experiments and physics-based models have shown that most low-cost PM sensors detect particles larger than 1 μm with low efficiency.¹⁰⁻¹² Additionally, field-based data suggest that low-cost PM sensors underestimate ambient concentrations of wind-blown dust (which likely includes many particles >1 μm).^13,14

Here, we compare differential mass concentrations (e.g., PM_1.0–2.5) from three low-cost PM sensors to differential mass concentrations measured by a federal equivalent method (FEM) PM monitor in an outdoor urban environment to better understand the ability of low-cost sensors to detect PM in different size fractions. We focus on the most common PM size ranges (<1.0, 1.0–2.5, and 2.5–10 μm) and compare performance metrics for differential mass to cumulative mass to show how the latter can mask sensor limitations.

Materials and Methods

Low-Cost Sensors

We evaluated three light-scattering sensor models (two units of each model) with prices and form factors that would be appropriate for use in large networks and/or personal exposure monitors (i.e., sensors that cost < $100 and weigh < 50 g): the PMS5003 (Plantower, Beijing, China), the SPS30 (Sensirion, Stäfa, Switzerland), and the IPS-7100 (Piera Systems, Mississauga, Canada). Specifications, descriptions, and data logging setups for all sensors are available in the Supporting Information (see Table S1 and Figure S1). All sensors were new and operated according to manufacturer recommendations (without additional calibration).

A GRIMM EDM 180 (GRIMM Aerosol Technik, Ainring, Germany) was chosen as the reference monitor due to its ability to measure PM in 31 particle size channels (0.25–32 μm). For quality assurance, we compared PM_2.5 and PM₁₀ measurements between the GRIMM EDM 180 and a colocated Beta Attenuation Monitor (5014i, Thermo Scientific, Waltham, MA, USA) (Figure S6). Additionally, we obtained ambient temperature, humidity, barometric pressure, and wind speed data from a colocated weather station (Vantage Pro2, Davis Instruments, Hayward, CA, USA) (see weather data in SI).

Field Deployment

The instruments were deployed on the roof of the Colorado State University Powerhouse Energy Campus in Fort Collins, Colorado, USA (Figure S2a). The building is located in an urban environment, adjacent to a major road and a railway, and experiences seasonal dust and wildfire events, despite having a relatively low background PM concentration. The GRIMM EDM 180 ran continuously on a roof above the second floor of the building. The low-cost sensors were installed on one side of the reference monitor (Figure S2b).

Data Processing

Data were collected during two periods with different weather: a fall/winter data set that spanned from November 23, 2021, to January 9, 2022 (48 days), and a summer data set that spanned from June 13, 2022, to July 30, 2022 (48 days). Data were aggregated to 1-h averages, and PM mass concentrations were calculated for each monitor in the following size ranges: PM_1.0, PM_2.5, PM₁₀, PM_1.0–2.5, and PM_2.5–10 (see variable definitions in SI).

Statistical Analyses

Descriptive statistics, performance metrics, regression models, and graphical tools (e.g., scatter plots) were used to assess sensor precision (coefficient of variation among colocated devices of the same model), linearity (coefficient of determination vs reference), and bias (e.g., RMSE, MAE, NMB vs reference) as a function of particle size range. A complete list and details of the performance metrics and statistical methods used in this study are available in the SI.

Results and discussion

Marked differences were evident in the low-cost sensors’ responses to PM of different size fractions (Figures 1 and 2). The low-cost sensors measured PM_1.0 with relatively low bias (mean absolute error [MAE] ranging from 1.2 to 2.4 μg/m³; Table 1) and strong linear correlation (R² from 0.83 to 0.93). For PM_2.5, the sensors showed slightly worse agreement with the reference monitor. MAE increased by a factor of 2 for all sensors (3.0 to 3.9 μg/m³), and linear correlation decreased (0.65 ≤ R² ≤ 0.76 compared to 0.83 ≤ R² ≤ 0.93, as stated above). A larger disparity was evident for PM₁₀ concentrations reported between the low-cost sensors and reference monitor (R² from 0.07 to 0.23). Consistently low PM₁₀ concentrations reported by the low-cost sensors, relative to the reference monitor, indicated that larger particles (i.e., d_p > 1.0 μm) were not adequately detected by the sensor models we evaluated. Differential mass concentrations provided further evidence of this limitation. The PM_1.0–2.5 and PM_2.5–10 signals were noisy and nearly uncorrelated with the reference measurement throughout the experiment. Some of these patterns (or lack thereof) were clearly identifiable from a simple visual assessment of the time series plots (Figure 1). The readings from all sensors lost accuracy when going from PM₁ to PM_2.5 and more so when going from PM_2.5 to PM₁₀. In those same plots, the PM_2.5–10 signals from the low-cost sensors remained flat and nearly zero throughout the campaigns, despite the reference monitor reporting PM_2.5–10 concentrations of ∼10–50 μg/m³. This unresponsiveness is consistent with a fundamental inability to sense and/or classify particles in that size range (2.5–10 μm), which has been demonstrated previously for some of these same low-cost sensor models in laboratory experiments¹¹ and physical optical models.¹² The regression plots in Figure 2 and performance metrics shown in Table 1 confirm that all the low-cost sensors tested here miss almost all particle mass in the PM_2.5–10 size range under the real-world, outdoor conditions in which these sensors are often used.

Figure 1.

Time series graph of PM concentrations (cumulative and differential). A subset composed of the first 14 days of the fall/winter period is shown to improve readability. A similar plot for the summer period is available in the SI (Figure S7).

Figure 2.

Regression plots of low-cost sensor PM estimates versus the EDM180 reference measurements. Dashed black lines are identity (1:1) lines, and continuous red lines are the lines of best fit (for models that complied with linear regression assumptions). All the hourly averages (n = 1763) were used to compute the regression models, but some points are not shown in the plots due to the axes ranges.

Table 1. Size-Resolved Descriptive Statistics and Performance Metrics over the Full Field Evaluation (Fall/Winter and Summer Periods Combined)^a.

	PM1.0	PM2.5	PM10	PM1.0–2.5	PM2.5–10
Descriptive statistics (EDM180)
Median [μg/m3]	2.9	4.9	12.3	1.4	6.4
Mean [μg/m3]	4.4	6.5	17.8	2.0	11.4
IQR [μg/m3]	4.0	5.4	16.4	1.7	13.1
Range [μg/m3]	0–119.9	0.1–124.7	0.1–351.4	0–51.4	0–295.4
Coefficient of determination (R2)
Piera IPS-7100	0.83	0.65	0.07	0.01	0.07
Sensirion SPS30	0.93	0.72	0.23	0.09	0.22
Plantower PMS5003	0.90	0.76	0.07	0.01	0.00
Slope | Intercept
Piera IPS-7100	0.87 | −1.54	1.70 | −6.51	N/A	N/A	N/A
Sensirion SPS30	0.88 | −0.54	0.78 | −1.39	N/A	N/A	N/A
Plantower PMS5003	1.44 | −1.01	1.68 | −3.01	N/A	N/A	N/A
Mean absolute error [μg/m3]
Piera IPS-7100	2.36	3.87	13.89	2.34	10.42
Sensirion SPS30	1.20	3.00	14.10	1.84	11.26
Plantower PMS5003	1.68	3.45	12.88	2.07	10.88
Normalized mean bias
Piera IPS-7100	–47.8%	–31.0%	–65.7%	5.2%	–85.3%
Sensirion SPS30	–24.4%	–43.7%	–78.9%	–85.3%	–98.8%
Plantower PMS5003	21.0%	21.6%	–47.2%	22.9%	–86.2%
Coefficient of variation
Piera IPS-7100 (2 units)	16.0%	20.4%	18.2%	28.0%	17.3%
Sensirion SPS30 (2 units)	6.0%	6.1%	6.3%	8.0%	50.9%
Plantower PMS5003 (2 units)	22.2%	16.9%	13.4%	16.3%	22.2%

^aParameters marked as N/A correspond to models that violated linear regression assumptions.

Additionally, every sensor demonstrated poor detection of the intermediate differential size fraction (1.0–2.5 μm). As shown in Figure 2, PM_1.0–2.5 errors appear random with no clear trends or consistent bias. Qualitatively, this was seen as large scattering of data points in the regression plots (Figure 2) and the REU plots (Figure S3). Quantitatively, the very low coefficients of determination (R² < 0.1 for all linear regression models) and large errors were coherent with a signal composed mostly of noise. Our results indicate that these sensors can estimate ambient PM mass concentrations in the accumulation mode (0.1 < d_p < 1 μm), but that they cannot reliably detect particle mass in the 1.0–2.5 μm or the 2.5–10 μm size ranges. Our results are consistent with Ouimette et al.,¹² who developed a physical–optical model of the PMS5003 and theorized that this device would struggle to size classify particles due to signal misclassification from multiangle scattering within the instrument’s detection zone. Laboratory evaluations of the SPS30 and the PMS5003 have found that size detection ranges do not adhere to manufacturer specifications and that size distribution data from these sensors are unreliable.^10,11 Our results under real-world conditions confirm these reports. The combined evidence suggests that the size-resolved data reported by these sensors is based on mathematical artifices rather than on real measurements of size distribution. There are several reasons why these optically based, low-cost sensors fail to respond adequately to particles larger than 1 μm. First, the sensing zones in these instruments have truncated viewing angles that fail to capture forward light scattering from particles.¹² Such truncation error has been shown to produce a dramatic loss in signal as particle size increases from 0.5 to 5 μm. Second, these devices are likely to experience inertial losses during aspiration and transmission of particles from ambient air to their respective sensing zones.¹⁵ Finally, the shape and refractive index of aerosol can differ between the coarse mode and the accumulation mode (and/or the factory calibration aerosol), which leads to differential light-scattering response between these size ranges during real-world use.^9,16−19

Ideally, the sensors should provide linear response with relatively low noise (i.e., good precision). Nonzero intercepts and nonunity slopes in the regression calibration models are not a major concern because optical instruments are sensitive to aerosol characteristics such as shape and refractive index, which means that they need to be calibrated to specific sampling conditions in applications where high accuracy is needed.²⁰ For PM_1.0, most points on the regression plots are close to the 1:1 line indicating good agreement with the reference monitor, and all the sensors fit the linear model reasonably well (R² ≥ 0.83; see also Figures S8, S9). Additionally, the relatively small scatter in the PM_1.0 regression plots and the narrow band patterns in the REU plots (Figure S3) denote low noise. The PM_2.5 estimates combine the good performance of PM_1.0 with the random noise of PM_1.0–2.5. This translates to more scattering, larger errors, and poorer fit of the linear models (0.65 ≤ R² ≤ 0.76), but depending on the application, a calibrated sensor could produce usable PM_2.5 estimates, as has been reported previously.²¹⁻²³ For instance, to evaluate PM_2.5 data quality, the uncertainty could be analyzed as the European Commission recommends, which specifies a 50% REU upper bound as the data quality objective (DQO) for low-cost sensors.²⁴ Alternatively, PM_2.5 could be estimated from the more accurate PM_1.0 output by developing ad hoc conversion equations for a combination of sensor model and use case.

On the other hand, the PM₁₀ signals of these sensors are clearly flawed, incorporating the noise of PM_1.0–2.5 and the systematic bias of PM_2.5–10. The regression plots of PM₁₀ show high scattering, large bias and no linear or other type of trend. The REU plots of PM₁₀ show extremely large uncertainty with a combination of bias and noise across all concentrations (Figure S3). Consequently, the PM₁₀ signal of the sensors we tested should be disregarded, as it appears to provide no meaningful output. A field evaluation and calibration by Kosmopoulos et al. reached similar conclusions and recommended the removal of high PM_coarse events from data sets collected with PurpleAir to improve the accuracy of the PM₁ and PM_2.5 signals, but the authors did not find a way to obtain adequate PM₁₀ estimates.²⁵

Some studies where only cumulative concentrations (e.g., PM_1.0, PM_2.5) were analyzed have found good agreement between low-cost sensors and FEM monitors for PM_2.5 and even for PM_10.^26,27 Nevertheless, our analysis of the differential concentrations (i.e., PM_1.0–2.5 and PM_2.5–10) shows that the PM_2.5 performance is driven largely by the performance in the PM_1.0 range. If sensors are tested in environments where PM_1.0 constitutes a substantial proportion of PM_2.5 and PM₁₀, or where the particle size distributions resemble the conditions that the manufacturer used to calibrate the devices, the accuracy of PM_2.5 and PM₁₀ signals can be artificially high. Hence, evaluations do not provide a full picture of sensor performance if a wide range of conditions is not covered and if only cumulative concentrations are analyzed.

Our work does not focus on calibration schemes for improving low-cost sensor data, for which there are many potential strategies.^{14,20,21,28,29} Further, we tested only three different sensor models; however, these models typify the state-of-the-art models for low-cost light-scattering devices and represent the majority of in-use technologies for crowdsourced measurement networks around the world.³⁰⁻³² Although our results are limited to a single geographic location and two seasons, they are consistent with previous theoretical and laboratory investigations.^9,11,12

The limitations of low-cost sensors presented here should be acknowledged by sensor manufacturers (or product integrators), and the networks that leverage these sensors should cease reporting PM₁₀ mass concentrations. Use of inaccurate particle size distribution data for research applications such as source apportionment or outdoor-to-indoor air penetration³³ could lead to wrong conclusions and, ultimately, to misguided public health interventions. For everyday applications where people want to understand local PM₁, PM_2.5, and PM₁₀ sources and levels, it is important to understand how effective (or ineffective) low-cost sensors are at detecting particulate matter depending on the primary sources (e.g., wildfire smoke, vehicle exhaust, cooking aerosol, wind-blown dust) and sizes of the particles. In all cases, clear guidelines for using (or disregarding) sensor outputs based on their trustworthiness (i.e., accuracy, precision, uncertainty) will benefit the user community. Specific applications where low-cost sensors have been shown to produce reliable estimates of aerosol mass concentration include calibration schemes for which the following conditions hold: (1) The aerosol of interest is stable in terms of size and refractive index. (2) The environmental conditions are accurately recorded (especially relative humidity). (3) The aerosol mass median diameter (MMD) falls within the accumulation mode (i.e., 0.1 < MMD < 1 μm). Applications where these conditions have been met may include urban fine particulate matter,^34,35 wildfire smoke,^36,37 and household solid fuel burning.³⁸ When calibration schemes fail to account for changes in particle size, particle refractive index, and ambient relative humidity, the responses from these sensors will be uncertain and subject to bias.^20,39 Whenever possible, calibration schemes should be designed to account for potential variability in particle characteristics and ambient conditions, as noted above. Environmental conditions (i.e., temperature, relative humidity, barometric pressure) during calibration should ideally span ranges typical of the environmental conditions under which the sensors will be deployed in their intended setting(s). Sensor calibration should also occur in the presence of air pollution sources that are similar in composition and magnitude to those that the sensors will experience during deployment. Regardless of the calibration scheme, fundamental safeguards for the use of sensors should include protocols that address data handling and initial processing, outlier detection and removal, sensor detection limit, and data completeness. Further, calibration schemes should disclose the kind of reference air quality monitor used, the duration of the colocation experiment and ambient conditions, the time-averaging intervals used for processing the data, and the statistical model(s) selected with appropriate justification (e.g., evidence of model assumptions being met).^20,29

In summary, we conclude that low-cost PM sensors commonly used in crowdsourced measurement networks are best described as accumulation mode PM (PM_0.1–1.0) sensors with little-to-no sizing ability or measurement reliability outside this size range. None of the devices evaluated here could detect coarse mode PM (PM_2.5–10), and most struggled to measure the 1.0–2.5 μm range when compared to a size-resolved reference monitor. Therefore, PM_2.5 estimates from these low-cost sensors should be interpreted with caution, and PM₁₀ estimates from these sensors should be seen only as a proxy representing the contribution of the accumulation mode to the PM₁₀ fraction.

Glossary

Abbreviations

PM

particulate matter

LCS

low-cost sensor

OPC

optical particle counter

FEM

federal equivalent method

RMSE

root-mean-square error

MAE

mean absolute error

NMB

normalized mean bias

REU

relative expanded uncertainty

DQO

data quality objective

Supporting Information Available

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.estlett.3c00030.

• Descriptions and specifications of the low-cost sensors that were evaluated. Methods used to collect data from the low-cost sensors, including photographs of the dataloggers, housings, and placement on the testing location. Equations and values for all the performance metrics that were calculated, some of which were not included in the main text due to space constraints. Relative expanded uncertainty plots (Figure S3) and the equations that were used to calculate the REU. Weather data (Figures S4 and S5) and a regression plot (Figure S6) of PM_2.5 and PM₁₀ comparing the GRIMM EDM 180 versus the Thermo Scientific 5014i for quality assurance. A time-series graph of PM concentrations over a 14-day period in the summer (Figure S7). Diagnostic plots for the regression models (Figures S8 and S9) (PDF)

The authors declare no competing financial interest.