Evaluation of consumer monitors to measure particulate matter

Abstract

Recently, inexpensive (<$300) consumer aerosol monitors (CAMs) targeted for use in homes have become available. We evaluated the accuracy, bias, and precision of three CAMs (Foobot from Airoxlab, Speck from Carnegie Mellon University, and AirBeam from HabitatMap) for measuring mass concentrations in occupational settings. In a laboratory study, PM_2.5 measured with the CAMs and a medium-cost aerosol photometer (personal DataRAM 1500, Thermo Scientific) were compared to that from reference instruments for three aerosols (salt, welding fume, and Arizona road dust, ARD) at concentrations up to 8500 μg/m³. Three of each type of CAM were included to estimate precision. Compared to reference instruments, mass concentrations measured with the Foobot (r-value = 0.99) and medium-cost photometer (r-value = 0.99) show strong correlation, whereas those from the Speck (r-value range 0.88 – 0.99) and AirBeam (0.7 – 0.96) were less correlated. The Foobot bias was (−12%) for ARD and measurements were similar to the medium-cost instrument. Foobot bias was (< −46%) for salt and welding fume aerosols. Speck bias was at 18% salt for ARD and −86% for welding fume. AirBeam bias was (−36%) for salt and (−83%) for welding fume. All three photometers had a bias (< −82%) for welding fume. Precision was excellent for the Foobot (coefficient of variation range: 5% to 8%) and AirBeam (2% to 9%), but poorer for the Speck (8% to 25%). These findings suggest that the Foobot, with a linear response to different aerosol types and good precision, can provide reasonable estimates of PM_2.5 in the workplace after site-specific calibration to account for particle size and composition.

1. Introduction

In occupational studies, exposure to respirable particulate matter (PM), the fraction of particles that can penetrate to the alveolar regions of the lungs (Antonini, 2003), is associated with respiratory diseases (Antonini, 2003, Taylor, Zimmer, & Roberts, 2004), lung cancer (Sørensen et al., 2007), and cardiovascular diseases (Li et al., 2015). To avoid the development of adverse health effects from inhaling particles, the Occupational Safety and Health Administration (OSHA) requires employers to maintain workplace, 8-h time-weighted average, respirable PM below 5 mg/m³ for particles not otherwise regulated (PNOR) (OSHA, 2006). These measurements are based on gravimetric, filter-based methods (the “gold standard”), or methods deemed equivalent to filter-based methods (NIOSH, 1975). Although accurate and precise, filter-based measurements are expensive, time-consuming, and provide little temporal information. Equivalent methods (e.g.,personal dust monitor; PDM 3700, Thermo Scientific, TSI Inc., Shoreview, MN, USA) often provide high temporal resolution, but are expensive (>$15,000 per monitor), resulting in little spatial information (White, 2009).

Direct-reading instruments are available to measure PM at high temporal resolution and in situ (Cheng, 2008; Yanosky, Williams, & MacIntosh, 2002). Some of these instruments depend on light scattering, such as optical particle counters (OPCs) or photometers. OPCs use the light scattered from individual particles to estimate number concentration for different particle size ranges. These data along with assumptions of particle shape and density can be converted to estimate mass concentrations that compare favorably to reference instruments (T. M. Peters, Ott, & O’Shaughnessy, 2006). Photometers (e.g., personal DataRAM 1500, pDR, Thermo Scientific., Shoreview, MN, USA) rely on the fact that the mass concentration of aerosol scales linearly with the amount of light scattered by an assembly of particles captured at a discrete angle from the incident light (Görner, Bemer, & Fabriés, 1995). The cost of these instruments ($15,000 for OPCs and >$6,000 for photometers) limits their use in the study of occupational PM exposures.

Several original equipment manufacturers (OEMs) now offer low-cost sensors to measure aerosol mass concentrations, including a photometer from Sharp (Sharp GP) and OPCs from Syhitech and Shinyei. These sensors are integrated with other electronics to convert output voltage to a meaningful signal in a variety of commercial products, such as air cleaners, air purifiers, and air quality monitors. Researchers have compared the output of OEM sensors to mass concentrations measured with gravimetric samplers and other direct reading instruments in occupational settings (Sousan, Koehler, Thomas, et al., 2016). Whereas traditional, high-cost OPCs count particle number concentrations in many particle size ranges (multiple bins), the low-cost OPC sensors from Syhitech and Shinyei sprovide an indication of number counts over a single size range. Wang et al. (2015) and Sousan, Koehler, Thomas, et al. (2016) observed high coefficients of determination (R² ≥ 0.95) among the voltage from the Sharp GP and mass concentrations measured with commercial photometers under laboratory conditions. Wang et al. (2015) observed less favorable agreement (R² = 0.89) among output from the Syhitech DSM501A with the SidePak AM510 (TSI, Shoreview, MN, USA) photometer in laboratory conditions. In an urban setting, Johnson, Bergin, Russell, and Hagler (2016) observed poor agreement (R² = 0.3) between output from the Shinyei PPD60PV-T2 and an EPA federal equivalent method sampler. To our knowledge, no one has evaluated the Shinyei PPD60PV-T2 for occupational settings.

Multiple manufacturers package these OEM sensors in consumer aerosol monitors (CAMs), including the Foobot ($200, from Airoxlab, Esch-sur-Alzette, Luxembourg), the Speck ($200, from Carnegie Mellon University, PA, USA), and the AirBeam ($250, from HabitatMap, NY, USA). These inexpensive CAMs use a microprocessor to collect sensor output, convert it to PM_2.5 (particles smaller than 2.5 μm), and store the data internally or transmit it wirelessly to a remote server. Often, these CAMs include additional sensors for measurement of temperature, relative humidity, carbon dioxide and total volatile organic compounds. Manufacturers of these devices use a variety of protocols to convert aerosol sensor output to mass concentration. The calibration protocol can have a dramatic impact on sensor precision, accuracy, and bias. Recently, researchers evaluated the Speck for use in the laboratory, outdoors and in-home. In laboratory tests, Manikonda, Zíková, Hopke, and Ferro (2016) observed a high coefficient of determination for cigarette smoke (R² = 0.92) and Arizona test dust (R² =0.96) among the Speck mass concentration and the calculated mass concentration from an aerodynamic particle sizer (APS; 3321, TSI, United States). In contrast in field work, the same group (Zikova, Hopke, and Ferro (2017)), observed less favorable agreement for indoors (R² = 0.3) and outdoors (R² = 0.1 – 0.2) among the less favorable agreement for the Speck and a GRIMM 1.109 optical particle counter. To our knowledge, no one has rigorously evaluated the performance of the Foobot and AirBeam.

The objective of the current study was to evaluate the performance of the three CAMs (Foobot, AirBeam, and Speck) over a wide range of mass concentrations typical of occupational settings. In laboratory tests, we assessed the linear relationship (slope, intercept, r-value) and bias of mass concentrations (PM_2.5) measured with CAMs and a pDR compared to reference instruments for three aerosols (salt, welding fume, and Arizona road dust, ARD). We also assessed precision by measuring mass concentrations with three of each type of CAM.

2. Methods and Materials

2.1 Consumer Air Quality Monitors (CAMs)

In the first quarter of 2016, we identified three CAMs available for purchase that support PM_2.5 measurement and data download for post processing and analysis (Table 1; Foobot, Speck, and AirBeam). The Foobot relies on natural convection to passively move air through a Sharp GP sensor (cost ~$12) that measures PM_2.5 for particles ranging in size from 0.3 μm to 2.5 μm and concentrations up to 1300 μg/m³ (Airoxlab, 2016). It offers no internal storage, requiring an internet connection to upload measurements to the manufacturer’s server. The manufacturer hosts a website where the uploaded data can be visualized and downloaded. The manufacturer considers the Foobot calibration proprietary (personal communication with the manufacturer). Although not tested in this work, the base model of the Foobot also includes sensors for total volatile organic carbons, carbon dioxide, temperature, and relative humidity.

Table 1.

Manufacturer specifications for the Foobot, Speck, AirBeam, and pDR 1500.

Technical data	Foobot	Speck	AirBeam	pDR
Integrated OEM sensor	Sharp GP2Y1010AU0F	Syhitech DSM501A	Shinyei PPD60PV-T2	NA
Cost ($)	200	200	250	6,000
Concentration range (μg/m3)	0–1300	0–640	0–400	0.001–400,000
Detects number concentration	No	Yes	Yes	No
Detectable size range (μm)	0.3–2.5	0.5–3.0	NA	1–10
Sampling frequency	5 minutes	5 second – 4 minutes	1 second	1 second
Internal fan	No	Yes	Yes	-
Additional air quality measurements	tVOCs, CO2, Temperature and relative humidity	Temperature	Temperature and relative humidity	Temperature and relative humidity
Remote storage	Yes	Yes	Yes	No
Internal storage	No	Yes	No	Yes
Internal rechargeable battery	No	No	Yes	Yes (batteries)
Dimensions in meters [LxW/DxH]	0.17×0.07 [HxD]	0.114×0.035×0.037 [LxHxW]	0.1×0.095×0.04 [LxHxW]	0.181×0.143×0.084 (HxLxW)
Weight (kg)	0.475	0.164	0.198	1200
Wi-Fi connectivity	Yes	Yes	No	No
Visible output on sensor	No	Yes	No	Yes

The Speck uses an internal fan to pull air through a single-bin OPC (Syhitech DSM501A, cost ~$10, Syhitech Co., Ltd). The manufacturer of the Speck states that PM_2.5 is measured for particles ranging from 0.5 μm to 3.0 μm and concentrations up to 640 μg/m³ (Speck, 2016). Each Speck was calibrated by the manufacturer alongside a multi-channel OPC (Met One HHPC-6+, Beckman Coulter, Brea, CA, USA) with polydisperse diatomaceous earth as the reference aerosol (Speck, 2016). A regression model was derived to convert sensor output in millivolts to number concentration based on particle counts from the OPC. Then a proprietary method was used to calculate mass concentrations from number concentrations. The CAM reports number concentration alongside PM_2.5. The Speck includes a temperature sensor and internally stores data that can be download to a personal computer via supplied software.

The AirBeam contains sensors for temperature, relative humidity, and particulate matter. It uses an internal fan to pull air into the particle sensing region of a single-bin OPC (Shinyei PPD60PV-T2, cost ~$250, Shinyei technology Co., LTD) to detect particles ranging from 0.5 μm to 2.5 μm and PM_2.5 concentrations up to 400 μg/m³, according to the manufacturer specifications (HabitatMap, 2016). The AirBeam converts sensor voltage to mass concentrations using a linear regression model, developed in side-by-side tests of the AirBeam and a pDR in an outdoor, urban setting. The manufacturer demonstrated that the AirBeam exhibits a linear relationship with the pDR for concentrations lower than 100 μg/m³, but acknowledged a non-linear relationship for concentrations higher than 100 μg/m³ (HabitatMap, 2016). The AirBeam has no internal storage and is accompanied by an Android application (AirCasting) designed for smartphones or tablets. AirCasting allows the user to save AirBeam measurements, along with GPS coordinates, on the mobile device and/or upload the data to a central server. Users can view and share uploaded data with others on a map. AirBeam code is open source, allowing the user to modify the code to improve accuracy. The AirBeam reports number concentration alongside PM_2.5 after updating the code. Before testing, we updated the firmware of the AirBeam with the most recent version of the code (November 2015).

2.2 Experimental Setup and Protocol

The experimental setup is shown in Figure 1. Following Sousan, Koehler, Thomas, et al. (2016), we generated three polydisperse aerosols (salt, welding fume, and ARD) using three separate systems (Figure 1-B). These aerosols were selected for the following reasons: salt is commonly used to evaluate aerosol instruments and does not absorb light; welding fume is a common hazard in occupational settings and has high light absorption; and ARD is representative of coarse mineral dust (Curtis et al., 2008) found in environmental and occupational settings and is also commonly used to calibrate direct-reading instruments. Salt was generated by nebulizing a 0.9% solution of NaCl (Figure 1-A(I)). Welding fume was generated with a welding system (0.03-inch Flux-Corded MIG Wire, Campbell Hausfeld, USA) operated inside a sandblast cabinet (Item 62454, Central Pneumatic, Byron Center, USA) (Figure 1-A(II)). ARD (Fine Grade, Part No. 1543094, Powder Technology INC., Arden Hills, MN) was aerosolized with a fluidized bed aerosol generator (3300A, TSI Inc., Shoreview, MN, USA) with the concentration adjusted by controlling the feed rate of the dust entering the fluidized bed (Figure 1-A(III)). The aerosol was fed into the mixing/dilution zone (0.64 m × 0.64 m × 0.66 m) of a Plexiglas chamber, diluted with clean air (0.25 m³/min), mixed with a small fan (Figure 1-B), and finally passed through a perforated plate to a sampling zone (0.53 m × 0.64 m × 0.66 m). Three of each type of CAM and a single pDR (personal DataRAM 1500, Thermo Scientific., Shoreview, MN, USA) were positioned inside the center floor of the sampling zone and within a 0.4 m × 0.31 m area and 0.12 m away from the chamber walls.

Figure 1.

Experimental set up used to determine the performance of the CAMs

All CAMs were new and used for the first time in this study. The pDR was included as representative of a commercial, higher cost (~$6,000) photometer, typical of what has been available for approximately thirty years (Table 1). The pDR was operated with a cyclone before particle detection at an airflow rate to achieve a 50% cut-off diameter of 2.5 μm and a filter (glass microfiber, 934-AH, Whatman, USA) after detection for gravimetric measurement of PM_2.5. A scanning mobility particle sizer (SMPS; SMPS-C 5.402, Grimm, Germany) with an impactor (cutoff diameter of 0.804 μm) and an aerodynamic particle sizer (APS; 3321, TSI, United States) were positioned outside the test chamber with direct sampling from the sampling zone. The temperature and relative humidity were relatively constant during the experiments. The mean temperature was 24 °C (SD = 1.1 °C) for salt, 25 °C (SD = 1.4 °C) for welding fume, and 24 °C (SD = 1.3 °C) for ARD. The mean relative humidity was 31% (SD = 1.8%) for salt, 28% (SD = 2.3%) for welding fume, and 27% (SD = 2.1%) for ARD.

All instruments were started at the same time. The Foobot uploaded PM_2.5 mass concentrations every 5-minutes to the manufacturer’s website. The Speck was set to record data every 5-seconds and the AirBeam every 1 second. The pDR was set to record every 1-second. The APS was set to record particle number concentration by size every minute throughout the experiment. The SMPS was set to record every three minutes. A pre-weighed 37-mm glass microfiber filter (934-AH, Whatman, USA) was installed in the pDR for each aerosol type. The 37-mm filter was used to correct mass concentrations for the pDR and the reference instruments (SMPS and APS). Prior to aerosol generation, clean air was passed through the chamber and air inside the chamber was confirmed to be clean with the pDR (<0.1 μg/m³). For each aerosol type, one experiment was performed, and our goal was to generate aerosols at 7 or 8 different mass concentrations and achieve 15-minutes steady state at each mass concentration, so that each CAM would have three readings for each concentration level. Then the three readings for each concentration level were averaged and the standard deviation was calculated and represented as error bars. After each experiment, the filters were weighed and the time-weighted average mass concentration was calculated.

2.3 Data Analysis

For each aerosol, the mass concentration by size for the SMPS and APS was calculated from the number concentration using the volumetric diameter and the aerosol density. The SMPS mobility diameter measurements range from 0.09 μm to 0.90 μm, and the APS aerodynamic diameter measurements range from 0.55 μm to 20 μm. Mobility diameters measured by the SMPS and aerodynamic diameters measured by the APS were converted to The volumetric equivalent diameters. The particle density and shape factor assumed for SMPS and APS mass calculations were the same as those used by Sousan et al. (2016). They are listed in Table S1, Supplementary Information. The SMPS and APS mass concentrations were summed over all particle sizes smaller than 2.47 μm to represent PM_2.5. Then, the pDR and reference (SMPS + APS) mass concentrations were multiplied by a correction factor. The reference (SMPS+APS) mass concentrations were adjusted equally across all bins. The correction factor for the pDR was calculated by dividing the mass concentration measured gravimetrically by the mean of the pDR mass concentrations over the same time frame. From this point on we call the mass corrected pDR data pDR-mc, and the raw uncorrected pDR data pDR-raw. In a similar manner, the correction factor for the SMPS and APS was calculated by dividing the mass concentration measured gravimetrically with the time-weighted average of the SMPS and APS mass concentrations for each experiment. Mass concentrations measured with the Speck, AirBeam, pDR-raw, pDR-mc, SMPS and APS were averaged over five minutes and time paired together to match those from the Foobot. This process yielded three PM_2.5 mass concentrations for each instrument at each concentration.

PM_2.5 from the Foobot, Speck, AirBeam, pDRraw, and pDR-mc were compared to that from the SMPS/APS over concentrations ranges intended for occupational use. Linear regression was performed to obtain slope, intercept, and Pearson coefficient (r). For each time pair, bias was calculated as the ratio of the mass concentration differences to the mass concentration of the reference instruments. The mass concentration differences were between the CAMs and the reference instruments. We calculated the coefficient of variation (CV) as an indicator of sensor precision. The CV was calculated as:

$$CV=\frac{\mathrm{\sigma }}{\mathrm{\mu }}$$ (1)

where σ is the standard deviation and μ is the mean of the ratio of the mass concentration measured with three replicate CAM sensors of the same type. CVs were not calculated for pDR-raw and pDR-mc because only a single pDR was used and the precision of the pDR was not under investigation.

The low-cost CAMs may be a suitable alternative for the more expensive pDR. Therefore, we also compared PM2.5 from the Foobot, Speck, AirBeam with the pDRraw measurements, using the same procedures used to evaluate the CAMs with the SMPS and APS measurements.

3. Results

Scatterplots of PM_2.5 from the Foobot, Speck, AirBeam, and pDR-raw relative to that measured with reference instruments (SMPS + APS) are shown in Figure 2 for salt, Figure 3 for welding fume, and Figure 4 for ARD. These figures are shown on a log-log scale to clearly display low and high mass concentrations. For all aerosol types, PM_2.5 measured with the Foobot and pDR-raw were highly linear with that from reference instruments, whereas PM_2.5 from the Speck and AirBeam were not. PM_2.5 measured with the AirBeam reached a limit of 200 μg/m³ despite concentrations above that concentration indicated by the reference instruments. For salt (Figure 2), PM_2.5 from the Foobot and Speck were underestimated. In contrast, the mass concentration from the AirBeam were overestimated at the lowest concentration (68 μg/m³) but underestimated for higher concentrations (>100 μg/m³). PM_2.5 measurements from the pDR-raw were on or close to the 1:1 line. For welding fume (Figure 3), all three CAMs and the pDR-raw substantially underestimated mass concentrations compared to the reference instruments with the pDR-raw closest to the 1-1 line followed by the Foobot, AirBeam and Speck. For ARD (Figure 4), PM_2.5 measurements from the Foobot and the pDR-raw were similar, overlapping the 1:1 line for concentrations higher than 100 μg/m³. Lower than 100 μg/m³, the Foobot overlapped the 1:1 line but the pDR underestimated mass concentrations. Mass concentrations from the Speck were overestimated for concentrations lower than 400 μg/m³, but underestimated for higher mass concentrations. For concentrations lower than 100 μg/m³, PM_2.5 measurements from the AirBeam were similar to those from the pDR-raw and near the 1:1 line, although PM_2.5 was substantially underestimated for higher concentrations. The ratios of mass concentrations measured gravimetrically and by the SMPS+APS were 1.03 for salt, 2.4 for welding fume, and 1.2 for ARD. These ratios were used to correct the SMPS+APS mass concentrations to better represent gravimetric mass concentration. For comparison, the ratios of gravimetric to pDRraw mass concentrations were 1.07 for salt, 5.6 for welding fume, and 1.5 for ARD.Evaluation of accuracy, bias, and precision of PM_2.5 measured with the Foobot, Speck, AirBeam, pDR-raw and pDR-mc are summarized in Table 2. For linear regression, a slope of 1, intercept of 0, correlation coefficient (r) of 1 indicate perfect agreement with the reference instruments. We followed EPA acceptance criteria based on particles following procedures specified by EPA for federal equivalent methods (EPA, 2006, 2016) and the National Institute for Occupational Safety and Health (NIOSH) for evaluating direct-reading gas instruments (NIOSH, 2012). EPA and NIOSH acceptance criteria include a slope of 1 ± 0.1, a y-intercept of 0 ± 5 μg/m³ (EPA only), and an r ≥ 0.97. Percentage bias is a parameter used by NIOSH to indicate how well a direct-reading instrument agrees with a reference device with ±10% considered acceptable. Lastly, % CV is an indicator of precision among duplicate instruments, where values less than 10% are considered acceptable (EPA, 2016).

Figure 2.

Mass concentrations measured by consumer air quality monitors and the pDR-raw (uncorrected mass) relative to reference mass concentration for salt aerosol. The error bars represent one standard deviation.

Figure 3.

Mass concentrations measured by consumer air quality monitors and the pDR-raw (uncorrected mass) relative to reference mass concentration for welding aerosol. The error bars represent one standard deviation.

Figure 4.

Mass concentrations measured by consumer air quality monitors and the pDR-raw (uncorrected mass) relative to reference mass concentration for ARD aerosol. The error bars represent one standard deviation.

Table 2.

Accuracy, bias, and precision over the entire concentration range (occupational) for the CAMs, pDR-raw (uncorrected mass), and pDR-mc (mass corrected) for A) salt, B) welding fume, and C) ARD

Instrument	Data Pairs	Slope ± Std. Error	Intercept ± Std. Error (μg/m3)	r	% Bias	% CV
A) Salt
Foobot	8	0.5 ± 0.008	−9.8 ± 6.8	0.991,2	−46	5.0
Speck	8	0.2 ± 0.009	5.4 ± 8.1	0.991,2	−68	25
AirBeam	8	0.07 ± 0.03	92 ± 27	0.70	−36	4.0
pDR-raw	8	1.1 ± 0.021,2	−37 ± 18	0.991,2	−6.3	-
pDR-mc	8	1.1 ± 0.021,2	−39 ± 18	0.991,2	1.01,2	-
B) Welding fume
Foobot	7	0.08 ± 0.006	−31 ± 27	0.991,2	−82	7.61
Speck	7	0.02 ± 0.002	−5.5 ± 10	0.971,2	−86	7.91
AirBeam	7	0.02 ± 0.004	44 ± 17	0.88	−83	9.21
pDR-raw	7	0.20 ± 0.01	−54 ± 53	0.991,2	−74	-
pDR-mc	7	1.09 ± 0.011,2	−30 ± 53	0.991,2	−42	-
C) ARD
Foobot	7	0.7 ± 0.02	17 ± 9.1	0.99v	−12	8.01
Speck	7	0.4 ± 0.08	83 ± 40	0.91	18	12
AirBeam	7	0.2 ± 0.02	18 ± 11	0.96	−53	2.01
pDR-raw	7	0.7 ± 0.01	9.1 ± 7.5	0.991,2	−21	-
pDR-mc	7	1.0 ± 0.011,2	12.1 ± 7.5	0.991,2	101,2	-

¹meets EPA criterion

²meets NIOSH criterion

For all aerosols (Table 2), PM_2.5 measured with the Foobot and pDR-raw were highly linear to that from reference devices (r = 0.99). However, slopes varied substantially by aerosol type. The lowest slope was for welding fume (0.08 for Foobot; 0.2 for pDR-raw) and the highest slope for Foobot was (0.7) observed for ARD measurements, whereas that for the pDR-raw was observed for salt (1.1). Bias values for the Foobot and pDR-raw were inversely related to observed slopes because the output of the CAM was highly linear with the reference instruments. The linearity for the Speck and AirBeam were less favorable than the Foobot and pDR-raw. For the Speck, the r-value was (0.99) for salt and (0.97) for welding fume but (0.91) for ARD. The r-values were substantially lower for the AirBeam, ranging from r = 0.70 for salt to r = 0.96 for ARD. Similar to the Foobot and pDR-raw, the slopes for the Speck and AirBeam were highly affected by aerosol type. In all cases, they were substantially lower than those observed for the Foobot and pDR-raw. The fact that the pDR-mc slopes were within 1 ± 0.1 and the bias values were closer to zero than the CAMs and pDR-raw for all aerosols is not surprising. Since the pDR-mc and reference (SMPS + APS) mass concentrations were both adjusted to the gravimetric measurements.

For all aerosols (Table 3), PM_2.5 measured with the Foobot was highly linear with pDR-raw (r = 0.99). However, the slopes varied substantially by aerosol type from 0.46 for salt to 0.96 for ARD. Bias values for the Foobot deviated from the recommended 10% with values of −42% for salt and −48% for welding fume. In contrast, the bias value for ARD was 12%, which is near the recommended 10% value. The linearity for the Speck and AirBeam were less favorable than for the Foobot. For the Speck, the r-value was (0.99) for salt and welding fume but (0.92) for ARD. The r-values were substantially lower for the AirBeam, ranging from 0.66 for salt to 0.97 for ARD. Similar to the Foobot, the slopes for the Speck and AirBeam were highly affected by aerosol type. In all aerosol types, they were substantially lower than those observed for the Foobot.

Table 3.

Accuracy, bias, and precision over the entire concentration range (occupational) for the CAMs compared to pDR-raw (uncorrected mass) for A) salt, B) welding fume, and C) ARD

Instrument	Data Pairs	Slope ± Std. Error	Intercept ± Std. Error (μg/m3)	r	% Bias
A) Salt
Foobot	8	0.46 ± 0.003	7.3 ± 2.7	0.991,2	−42
Speck	8	0.21 ± 0.009	6.1 ± 6.1	0.991,2	−64
AirBeam	8	0.07 ± 0.03	96 ± 28	0.66	−27
B) Welding fume
Foobot	7	0.50 ± 0.02	−5.4 ± 16	0.991,2	−48
Speck	7	0.10± 0.006	−0.3 ± 4.5	0.991,2	−76
AirBeam	7	0.08 ± 0.02	51 ± 18	0.84	−56
C) ARD
Foobot	7	0.96 ± 0.0091,2	8.2 ± 3.1	0.991,2	12
Speck	7	0.58 ± 0.10	76 ± 36	0.92	51
AirBeam	7	0.26 ± 0.03	15 ± 9.7	0.971,2	−42

¹meets EPA criterion

²meets NIOSH criterion

Precision (expressed as CV) of the Foobot and AirBeam were lower than 10%. CV for the Foobot ranged from 5% for salt and 8% for welding fume and ARD. CV for the AirBeam ranged from 2% for ARD to 9.2% for welding fume. In contrast, precision was poorer for the Speck (CV for salt, 25%; for ARD, 12%; and welding fume, 7.9%).

Number and mass particle size distributions for the largest total concentrations measured with the SMPS and APS for each aerosol type are shown in Figure 5. These size distributions are plotted by volumetric particle diameter to unify particle size measured by the SMPS and APS. Small particles (between 0.1 μm and 1 μm) dominated the mass size distributions for salt and welding fume, whereas coarse mode particles (1 μm to 10 μm) dominated the mass size distributions for ARD. The instruments do not have good efficiency for the right tail of the SMPS and left tail of the APS, resulting in a gap in the measured size distributions. The size of the gap depends on the shape factor and the particle density. The smallest gap is for salt which has a shape factor near 1 and the lowest density = 2200 kg/m^3. In contrast, the largest gap is for welding fume because of the high shape factor (3.9) and high density (3400 kg/m³). For this aerosol, the midpoint diameter associated with the largest size channel of the SMPS is converted from a mobility diameter of 1.0 μm to a volumetric diameter of 0.26 μm and that associated with the smallest size channel of the APS is converted from an aerodynamic diameter of 0.54 μm to a volumetric equivalent diameter of 0.59 μm.”

Figure 5.

Fractional number (dN/N_T) and mass concentration (dM/M_T) by size measured for A) Salt; B) Welding fume; and C) ARD

4. Discussion

This work compared the mass concentrations measured with low-cost CAMs and a traditional commercial photometer (pDR) to those measured with reference instruments. PM_2.5 from the Foobot was more closely related to that from reference measurements than other CAMs. For all aerosols, mass concentrations from the Foobot and pDR-raw were highly linear compared to reference instruments (r ≥ 0.99), whereas those from the Speck and AirBeam were less so. The Speck responded differently depending on aerosol type with a non-linear relationship for salt (Figure 2) and a logarithmic curve for ARD (Figure 4). In contrast, for welding fume (Figure 3), the response was fairly linear, except for the deviation at 17 μg/m³. For the Airbeam, a logarithmic curve with a maximum concentration of 200 μg/m³ was observed for all aerosols. Like the Foobot, the pDR responded linearly with mass concentration measured with reference instruments for all aerosol types.

The correction factor differences in SPMS+APS and pDRraw by aerosol type are not supprising. Salt and ARD correction factors were low compared to the welding fume correction factor. Salt particles have a zero absorption index and the pDR is very effective at collecting all the scatter light of the particles that is converted to mass.

The shape of the response curve can be related to the type of OEM sensors integrated into the CAM. The Foobot, like the pDR, employs a photometric-based sensor. Photometers have been shown to scale linearly with mass concentration over a large range (Chakrabarti, Fine, Delfino, & Sioutas, 2004). In contrast, the particle sensors in the Speck and the AirBeam are single-bin OPCs with output that scales directly with particle number concentration. The logarithmic shape of the response curves for all aerosols with the AirBeam and with the Speck for ARD may be explained by undercounting when multiple particles are coincident in the sensing zone. Particle coincidence is a known problem for particle counting instruments due to the counting principles for these instruments (Sachweh, Umhauer, Ebert, Büttner, & Friehmelt, 1998). The reason for the non-linear relationship for the Speck with salt measurements is unclear. OPCs can be good at estimating mass if they have numerous bins, such as the Portable Aerosol Spectrometer 1.108 (T. M. Peters et al., 2006). However, estimates of mass concentration from single-bin OPCs, like the Speck and AirBeam, may be subject to considerable error.

Our results of the Speck were similar to those observed by Manikonda, Zíková, Hopke, and Ferro (2016) with a high correlation (r-value = 0.98). The researchers observed that the Speck measurements for Arizona test dust (concentrations up to 400 μg/m³) overestimated mass concentrations compared to the APS for concentrations lower than 200 μg/m³, but underestimated for higher mass concentrations. However, the slope (2.0–2.3) and intercept (2.9 μg/m³ − 5.3 μg/m³) were different compared to our slope (0.4) and intercept (83 μg/m³). The slope and intercept differences are related to the ARD correction factor we used for adjustment of SMPS+APS mass concentrations. In contrast, our results were different from an in-home and outdoor study conducted by Zikova, Hopke, and Ferro (2017) comparing measurements between the Speck and GRIMM 1.109. The researchers observed an r-value and bias of 0.55 and 40%−400%, respectively, for in-home measurements, and 0.32–0.44 and 150–%1800, respectively, for outdoor measurements. Our r-value and bias were different for all aerosol types because we conducted the experiments in a controlled temperature and relative humidity chamber.

The Foobot and AirBeam evaluation are unavailable in peer-reviewed literature, although our findings are consistent with researchers who have evaluated the OEM sensors used in these CAMs. Our finding that the Foobot (containing a Sharp GP sensor) was highly linear with gravimetrically measured mass concentrations (r = 0.99) for all aerosols is consistent with the work of Sousan, Koehler, Thomas, et al. (2016). They observed a similarly linear relationship among output from the Sharp GP sensor with gravimetrically-corrected reference mass concentrations (r-value = 0.97–0.99). Also, Sousan, Koehler, Thomas, et al. (2016) calibrated the Sharp photometer alongside a pDR and derived separate linear regression models for salt, ARD, welding fume, and diesel fume for concentrations up to 6500 μg/m³. When aerosol-specific calibrations are used, the authors concluded that the linear model tested for the same aerosol type had an excellent performance (slope = 1±0.1, and bias = ±10%, values with EPA and NIOSH standards) when compared to reference instruments. Similarly, the bias for the Foobot with ARD (−12%) was closer to zero than with salt (−46%) or welding fume (−82%), which suggests that the Foobot was initially calibrated with an aerosol similar to ARD. Other researchers have also evaluated the OEM sensors used in the Foobot and AirBeam with reference to other commercial photometers. They observed similar results to us for the Foobot (Wang et al., 2015), and AirBeam (Johnson et al., 2016) results. It is not surprising to mention that the pDR-mc bias values were within (salt and ARD) or close (welding fume) to ±10% compared to the three CAMs and that r-values were 0.99 and slopes were 1 ± 0.1 (linear relationship) for all aerosols, since the pDR-mc measurements were gravimetrically adjusted, as is commonly done when using this device.

The response (slope and bias) of the CAMs compared to the reference instruments varied by aerosol type (Table 2). For salt, all three CAMs underestimated mass concentrations compared to the reference instruments. There was a large fraction of mass that is associated with particles smaller than the particle size detection limit of the aerosol sensors (Figure 5-A). For welding fume, the CAMs and pDR-raw mass concentration measurements were substantially underestimated compared to reference instruments. Both the salt and welding fume contain particles smaller than 0.5 μm by mass fraction as shown in Figure 5. However, the slopes for welding fume were much lower than for salt (e.g., for the Foobot, the slope was 0.5 for salt and 0.08 for welding fume). The difference in slopes may be attributed to the detection efficiency of the instrument or refractive indices of the aerosols. The fraction of particles smaller than 0.5 μm, the size where light scattering instruments experience reduced detection of particles, was slightly greater for salt (99% by number and 48% by mass, Figure 5-A) than for welding fume (98% by number and 38% by mass, Figure 5-B). Therefore, we attribute the difference in slopes between salt and welding fume to refractive index rather than the particle size detection limit. Welding fume has a large absorbing component of the refractive index, whereas salt has none. Thus for welding fume, the particles are undersized because of the absorbing component (Syvitski, 2007). Additionally, this underestimation of particle size can lead to significant errors in estimates of mass concentrations because mass concentrations is proportional to particle diameter cubed.

For ARD, the manufacturer calibration procedures could have contributed to the improvements in response (slope and bias) for all three CAMs compared to salt and welding fume. This could be due to the fact that OEM low-cost sensors are originally designed by the manufacturer to detect changes in dust concentrations, such as the Sharp sensor, designed for air purifiers and air conditioners. Differences in accuracy and bias can be related to the calibration method used by the manufacturer. The Foobot might be calibrated with ARD or a similar dust coarse mode aerosol since the bias was lowest for ARD (−12%), and all points fell on the 1:1 line (Figure 4). The Speck was calibrated using a proprietary method that converted particle number concentrations to mass concentrations while measuring diatomaceous earth aerosols. Both ARD and diatomaceous earth aerosols contain silica particles (Alexander et al., 2013), which could explain why the Speck measurements were closer to the 1-1 line for ARD compared to salt and welding fume. The Speck was the only CAM calibrated based on number concentration, and this could have been a limiting factor for why the Speck bias values were higher than the other two CAMs. Calibrating the Speck with a reference instrument to measure mass concentrations may have improved the results. The AirBeam performance matched the pDR-raw measurements for ARD at mass concentrations lower than 100 μg/m³ because the AirBeam is calibrated by the manufacturer with the pDR.

The Foobot may be a potential substituent to the pDR, especially for dust measurements. Compared to the pDR, the Foobot had the highest r-values for all three aerosols at 0.99. For ARD, the Foobot slope approached unity (1.0) with a low bias (−12) compared to the Speck and the AirBeam. The Foobot slopes were the highest among the three CAMs, and the bias values were the lowest for welding fume (−42%) and ARD. The low AirBeam bias value (−27%) for salt was the result of overestimates at low concentrations and underestimates at high concentrations (Figure 2). The Foobot proprietary calibration method prevents us from understanding its superior performance compared to the Speck and AirBeam.

Precision values indicate manufacturer calibration consistency for the CAMs produced. The Foobot (operating with the Sharp GP) exhibited precision values (CV < 10%) for all aerosols types. Sousan, Koehler, Thomas, et al. (2016) reported that the precision values for Sharp measurements before calibration had an overall mean value of 17% for different aerosol types. However, after calibrating the Sharp photometers with a reference instrument, the overall mean value decreased to 3.1%. Sousan, Koehler, Thomas, et al. (2016) concluded that the Sharp photometers, once calibrated with a reference instrument, the precision can be below 10%. This agrees with the Foobot precision results for our study, where CV values were 10% and lower due to the effective calibration method for the Sharp sensor inside the Foobot. In contrast, the Speck (operating with Syhitech) exhibited poorer precision, with a CV above 10% for for salt (CV = 25%) and ARD (CV = 12%), although precision values for welding fume were (CV = 7.9%). Wang et al. (2015) compared multiple Syhitech OEM sensors and multiple Sharp OEM sensors repeatability by measuring the standard deviation among sensors of the same kind. The authors observed that the Syhitech sensors had lower standard deviation values compared to the Sharp sensors. Wang et al. (2015) observed the opposite of our precision calculations based on the Foobot and Speck measurements. However, Wang et al. (2015) used raw output values (millivolts) from the Sharp and Syhitech sensors and calculated the standard deviation among sensors of the same kind. In contrast, we used the mass concentration output from the Foobot and Speck sensors to calculate the precision among sensors of the same kind. The mass concentration measurements from the Foobot and Speck are based on the manufacturer calibration for these sensors. Similar to our work, Manikonda, Zíková, Hopke, and Ferro (2016) stated that the Speck monitors displayed moderate relative precision between two Speck sensors. The AirBeam (operating with Shinyei), similar to the Foobot, exhibited precision values (CV < 10%) for all aerosols types. To our knowledge, no peer-reviewed publication has evaluated the precision of the AirBeam or the Shinyei sensor.

The limitation of the Foobot is the data acquisition method that might be problematic for occupational use. The Foobots can only be linked to routers with a simple pre-shared key (password only). Thus, IT restrictions could limit the Foobot from connecting to routers that require active directory authentication (user name and password) and render them unusable in environments without wireless internet access. The study was conducted in a controlled environment, thus eliminating the effects of temperature and relative humidity on sensor accuracy and bias. For occupational studies, this could be acceptable in most cases for indoor environments, but for environmental conditions, further evaluation should be done. Future studies should compare the CAMs at typical ambient concentration conditions taking into account temperature and relative humidity effects on mass concentration measurements and with further evaluation at low mass concentrations prevalent in industrialized nations (<35 μg/m³).

5. Conclusions

We evaluated the ability of several CAMs (Foobot, Speck, and AirBeam) to measure particle mass concentrations for several aerosols (salt, welding fume, and ARD) up to 8500 μg/m³. For all aerosols and different concentrations, the Foobot exhibited the best performance with a highly linear response, r-values (>0.97), and lowest bias values. In contrast, the mass concentrations measured with the Speck (for salt and welding fume) and AirBeam (all aerosol types) were less correlated with mass concentration measured with the reference instruments. The AirBeam was unable to respond to mass concentrations higher than 200 μg/m³, which is substantially less than the upper limit specified by the manufacturer. Intra-instrument precision of the Foobot and AirBeam was favorable (CV ≤ 10% for all aerosols), whereas that of the Speck was less so (CV ranged from 7.9% for welding fume to 25% for salt). The finding that the Foobot has good precision and linear response suggests that it may be a potential low-cost alternative to the pDR. In addition, the linear performance of the Foobot suggests that site-specific calibration factors may be used to improve measurement accuracy.