Improving the Accuracy of Smart Devices to Measure Noise Exposure

Abstract

Occupational noise exposure is one of the most frequent hazards present in the workplace; up to 22 million workers have potentially hazardous noise exposures in the US. As a result, noise-induced hearing loss is one of the most common occupational injuries in the United States. Workers in manufacturing, construction, and the military are at the highest risk for hearing loss. Despite the large number of people exposed to high levels of noise at work, many occupations have not been adequately evaluated for noise exposure. The objective of this experiment was to investigate whether or not iOS smartphones and other smart devices (Apple iPhones and iPods) could be used as reliable instruments to measure noise exposures. For this experiment three different types of microphones were tested with a single model of iPod and three generations of iPhones: the internal microphones on the device, a low-end lapel microphone, and a high-end lapel microphone marketed as being compliant with the International Electrotechnical Commission's (IEC) standard for a Class 2-microphone. All possible combinations of microphones and noise measurement applications were tested in a controlled environment using several different levels of pink noise ranging from 60 to 100 dBA. Results were compared to simultaneous measurements made using a Type 1 sound level measurement system. Analysis of variance and Tukey's honest significant difference (HSD) test were used to determine if the results differed by microphone or noise measurement application. Levels measured with external microphones combined with certain noise measurement applications did not differ significantly from levels measured with the Type 1 sound measurement system. Results showed that it may be possible to use iOS smartphones and smart devices, with specific combinations of measurement applications and calibrated external microphones, to collect reliable, occupational noise exposure data under certain conditions and within the limitations of the device. Further research is needed to determine how these devices compare to traditional noise dosimeter under real-world conditions.

Introduction

Smartphones have become ubiquitous in the United States; in 2011 the US Census Bureau estimated that 73.5% of people over the age of 25 used smartphones.⁽¹⁾ In addition to providing a convenient form of communication, these devices have the ability to run computer programs referred to as applications or “apps”. Using the processing power of these devices many companies have applications that can be used to track a user's behaviors, fitness and health.

A large number of applications that may be useful to environmental health professionals and industrial hygienists are available from various sources. Many of these apps provide a convenient way to record safety and health audits, look up regulations or exposure limits, or evaluate centrally-monitored exposure conditions (e.g., heat, weather conditions, or air pollution levels) on a mobile device. Other applications are used as companions to external sensors that communicate wirelessly with the smartphone. One of the most common occupational exposures that smartphone applications are able to measure is noise, as every smartphone is built around a microphone designed to record voices for communication.

Noise is one of the most common occupational exposures. It is estimated that over 22 million people each year are exposed to levels of noise in excess of 85 A-weighted decibels (dBA) as a time weighted average (TWA).² Most professional sound level meters (SLMs) and noise dosimeters are costly to purchase or rent and often require proprietary software to analyze the collected measurements. While it is unlikely that smartphones or smart devices will replace traditional noise measurement devices for compliance purposes, they have the potential to be used as low cost survey tools. Additionally, these devices have immense value in providing “crowd sourced” data for environmental noise levels; in fact, several projects are currently underway that have attempted to map the noise of certain areas.^(3,4) Finally, there is a potential for these applications to be useful in developing countries or low income areas where cheaper versions of smartphones are available, but it is not feasible to use a professional sound level meter or noise dosimeter.⁽⁵⁾

The potential opportunities presented by noise measurement applications are obvious given the prevalence of smartphones, their ease of use, and low cost compared to traditional noise measurement devices. Despite the best efforts of the developers, these applications have not been harmonized to any performance standard. The most comprehensive review of smartphone applications that measure occupational noise was conducted by Kardous and Shaw of the National Institute for Occupational Safety and Health (NIOSH) in 2014,⁽⁶⁾ and found that a small number of applications (4 out of 192 applications tested) offer the functionality and accuracy to be potentially useful for making occupational noise measurements. A subsequent study by another group found that even the best application evaluated was not accurate enough to make reliable noise measurements.⁽⁷⁾

In light of these conflicting results it is clear that further research into the accuracy of noise measurement applications is needed. As Kardous and Shaw indentified, different models of the same smartphone platform (iPhone, Apple Inc, Cupertino, CA) performed differently. This is an issue, especially for Android-based devices, as hundreds of models of smartphones with differing components and operating systems are manufactured each year by multiple manufacturers, and each of these factors could potentially lead to large variations in measurements. In addition, it is not always easy or possible to calibrate the internal microphone of a smartphone, which can lead to systematic error in measured levels. Some applications have a feature to automatically calibrate to a certain microphone, but the effectiveness of this feature has not been independently evaluated. Finally, the size and fragility of the smartphone makes it impractical to be used as a personal noise exposure instrument by mounting it in an individual's hearing zone – a hemisphere around the person's ear with a radius of approximately 18 inches.⁸ If a smartphone's microphone is physically covered by clothing or other materials it is likely that the smartphone would not make an accurate measurement.

To further assess approaches to smartphone-based noise exposure assessment, we compared the accuracy of smartphone noise measurements across different smart devices and applications. We also evaluated the accuracy of measurements made using the devices' internal microphone, as well as using two external microphones, an approach which has been discussed, but not been utilized previously.

Methods

Selection of Devices and Applications

The three applications found by Kardous and Shaw (2014) to perform the most accurate A-weighted noise level measurements were selected for further consideration since they met the NIOSH criteria for functionality and accuracy in this experiment. These applications were NoiSee (EA LAB), SPLnFFT Noise Meter (Fabien Lefebvre), and SoundMeter (Faber Acoustical, LLC) all of which are available on the iTunes Store.⁶ Only applications available on the iOS operating system were considered. This was done because the iOS operating system is more tightly controlled than other mobile operating systems and Apple devices have more uniform hardware than Android devices. The chosen applications ranged in price and features (Table I). All of the applications allowed for a user to select different measurement standards for integrating noise exposure. SPLnFFT and SoundMeter both allowed for user-customized threshold, criterion level, and exchange rate, which allows for greater flexibility in making measurements. Only SPLnFFT and SoundMeter allowed for the export of stored measurements as a comma separated value (.csv) file that can be opened in a spreadsheet program.

Table I.

Summary of the chosen application's features.

Application	Developer	Weightings	Standards	Exchange Rate	Projected Dose	Data Export	Price
NoiSee	EA Lab	A, C, Flat	OSHA/ISO	3, 4, 5	Yes	No	$0.99
SPLnFFT	Fabien Lefebvre	A, B, C, Flat	Custom	3, 4, 5	Yes	Yes A	$3.99
SoundMeter	Faber Acoustical	A, C, Flat	Custom A	3, 4, 5 A	Yes A	Yes A	$20.00

^ARequires additional in-application purchases for an additional $20

Three different Apple device models were evaluated during this experiment, all of which used the latest version of iOS (8.1, except for the iPhone 4 which used iOS 7.1). Three 5^th generation Apple iPods were the primary devices used. iPods are very similar to iPhones except that they lack the ability to communicate with cellular networks. These devices were chosen because they are cheaper to acquire than iPhones, which makes them more practical to deploy. In addition to these devices, the iPhone 4, 4S, and 5S were all evaluated to compare their ability to measure noise levels and provide some insight into the effects of the slight hardware differences between the models. The applications that were evaluated were identical across the different devices.

In addition to evaluating the internal microphones on the devices two additional external microphones were used. One microphone was the iMM-6 Calibrated Measurement Microphone from Dayton Audio (Springboro, OH) and the other was the i436 microphone from MicW (Beijing, China), which complies with the EC's)standard for a Class2 SLM which has a tolerance of +/- 1.4 dB at 1000 Hz.^(9–11) Both microphones have a 3.5 mm audio plug that connects to the headphone jack on smart devices. The microphones were calibrated to 94 dB SPL using the application's calibration setting and a Larson Davis (Provo, UT) Cal 150B SLM calibrator before the start of the experiment.

Experiment 1

The first experiment evaluated the influence of internal vs. external microphones on variability in measured noise levels in the same type of devices running the same applications. This was done by placing three 5^th generation Apple iPods in a reverberant noise chamber at the NIOSH acoustic testing laboratory in Cincinnati, OH. A diffuse sound field could was generated to prevent the location of the device's microphone from influencing the results. Pink noise was generated through three JBL XRX715 two-way loud speakers using the REATPLus software (ViAcoustics, Austin, TX). Sound level measurements were obtained through the Trident Multi-Chanel Acoustic Analyzer Software (ViAcoustics, Austin, TX) using a Larson Davis 2559 ½” inch microphone. The entire system simulates a Type 1 sound level measurement instrument.

Pink noise was generated at 60 dBA and the chamber was allowed 20 seconds to ensure that a stable sound field was established so that the devices would provide a stable reading. Using a USB webcam, measurements from the screens of the 3 devices were recorded and observed remotely, eliminating the need to re-enter the reverberant chamber to record measurements. After the measurements were recorded, the sound level was increased by 5 dBA and allowed to stabilize. This process was repeated in 5 dBA increments up to 100 dBA. This was done 6 times for each combination of microphone and application, so that each of the 3 devices made 54 measurements for each combination of application and microphone, or a total of 162 measurements for each combination of the application and microphone. In total, 1,458 measurements were made in experiment 1.

The results were recorded in Excel (Microsoft, Redmond, WA) and transferred to STATA 14 (College Station, TX) for analysis. The mean difference between the reference microphone and the iPods was calculated for each stimulus noise level for every combination of microphone and application. A difference of 0 dB would indicate perfect agreement between the iPods and the reference system, while a larger difference would indicate worse agreement between the iPods and SLM. In addition, a one-way Analysis of Variance (ANOVA) was used to determine if the three devices produced significantly different measurements. An ANOVA was also used to test if the microphone, application, and noise level had a significant impact on the difference in measurements between the reference system and the iPods. Tukey's HSD test was done post-hoc to determine if differences were observed between the different combinations of microphones and applications.

Experiment 2

In the second experiment we evaluated whether external microphones could be used to reduce the variation of noise measurements between different models of smartphones using the same application. This has practical implications because as new smartphone models are released older models often become obsolete as the manufacturer discontinues updates and support for the older models. A student's t-test was used to compare the measurements of the reference system to the measurements made by the different devices. In addition, an ANOVA was used to compare the mean difference in noise measurements between the different devices using the same application and microphone. A significant difference between the different iOS devices would indicate that replacing a device's internal microphone with an external microphone does not improve the precision of the measurements across different generations of a device. However, if there is not a significant difference, it would suggest that external microphones can be used to help increase the precision of measurements across different generations of devices. Fifty-four measurements were collected for each combination of device, microphone, and application. In total 540 measurements were collected in experiment 2. All other parameters were identical to those used in experiment 1.

Results

Experiment 1

Table II presents a summary of the mean difference calculations between the reference system and the iPods using several different application and iPods combinations. Across all three applications the iPod's internal microphone performed poorly. The NoiSee application could only measure up to 90 dBA using the built-in microphone. Both the iMM-6 and i436 microphones performed well when paired with the SoundMeter application, with only a 1 dB difference in sound level measurements when compared to the reference. Figure 1 provides a graphical summary of the distribution of differences in measurements stratified by application and microphone. The large interquartile range (IQR) for many of the combinations of applications and microphones suggests that only with particular configurations can an smart device be used to make reliable noise measurements.

Table II.

Mean differences between the iPods and sound level meter from experiment 1.

Application	Microphone Type	Reference Noise Level (dBA)
		60 A	65	70	75	80	85	90	95	100
NoiSee	Internal	7.1	7.1	7.1	7.1	7.2	4.5	0.1	>LOQ	>LOQ
	iMM-6	-0.1	-0.1	-0.1	-0.1	0.0	0.0	-0.1	-0.7	-4.3
	i436	1.5	1.3	1.3	1.3	1.3	1.5	0.1	0.2	0.0
SPLnFFT	Internal	2.1	1.6	1.6	1.6	1.6	2.8	1.5	2.8	2.7
	iMM-6	1.1	1.0	1.1	1.1	1.0	2.1	1.6	2.1	2.0
	i436	1.3	1.2	1.2	1.2	1.5	2.0	2.2	2.3	2.3
SoundMeter	Internal	2.9	3.2	3.3	3.3	3.3	3.4	2.2	3.3	3.4
	iMM-6	0.0	-0.1	0.0	0.0	0.0	0.0	0.0	0.0	0.0
	i436	1.0	0.9	1.0	1.0	0.4	0.9	0.9	0.9	1.0

^AN = 18 for each reference noise level, application and, microphone combination, N = 1,458 overall

Figure 1.

Box plot of the difference in measurements between the iPod and SLM stratified by microphone and application. The solid lines indicate the accepted range of accuracy of a type-2 SLM according to ANSI, the dashed lines indicated the accepted range of accuracy of a class-2 SLM according to the IEC.

The ANOVA (results not shown) comparing all the measurements made by the three iPods found that there was no significant difference in the measurements made by the three devices, even when stratified by the application and type of microphone used. This indicates that when the same types of devices use the same applications and microphones the results will likely be precise (i.e., small variability between devices), but not necessarily accurate (i.e., potentially large difference from the true noise level).

The results of the two-way ANOVA model examining the effect of the reference noise level, application, microphone, and the interaction between the application and microphone found that all terms in the model were highly significant (p <0.001). This provides further support for the results in Figure 1 that shows certain combinations of applications and microphones perform better than others. The results also suggest that the accuracy of certain applications or microphones may differ across noise levels. The results are further complicated by the significant interaction term between the application and microphone; this means that microphones will perform differently depending on the application they are paired with.

The results from Tukey's pairwise comparison for the applications and microphones are presented in Table III, which compares the mean difference between the different applications. The SoundMeter application had the lowest mean difference suggesting that it provide the most accurate noise measurements, followed by NoiSee and then SPLnFFT. While both NoiSee and SPLnFFT performed worse than the SoundMeter application, only SPLnFFT had a significantly larger mean difference. All three microphones were found to perform significantly different when compared to one another, with the best performance demonstrated by the iMM-6, then the i436, and then the internal microphone. Both the iMM-6 and i436 microphones, when calibrated, had a mean difference less than 2 dB, which is within the tolerance of a Type 2 sound level meter, suggesting that they may be appropriate to use for making accurate noise measurements. The results suggest that the internal microphone does not consistently provide measurements within the tolerance of a Type 2 sound level meter.

Table III.

Tukey's multiple pairwise comparisons for the mean difference (dB) in measurements between different applications and microphones.

Application 1	Application 2	Mean 1 (dBA)	N 1	Mean 2 (dBA)	N 2	dif	HSD Test Statistic
NoiSee	SPLnFFT	1.49	441	1.70	486	0.22	2.89
NoiSee	SoundMeter	1.49	441	1.35	486	0.13	1.63
SPLnFFT	SoundMeter	1.70	486	1.35	486	0.35	4.52A
Microphone 1	Microphone 2
iMM-6	Internal	0.09	486	3.45	441	3.35	43.11A
iMM-6	i436	0.09	486	1.17	486	1.07	13.77A
Internal	i436	3.45	441	1.17	486	2.28	29.34A

^AIndicates a significant (p<0.05) difference

Experiment 2

The second experiment was designed to determine if an external microphone and application combination would allow different versions of a smartphone to make reliable measurements. Table IV provides the mean difference, standard deviation, and sample size for each configuration tested. Across the different devices and using the internal microphone, the mean difference between the smartphone and reference system ranged from -1.09 to 24.99, with most of the configurations having a mean difference greater than 2 dB, which is outside the accuracy of a Type-2 instrument. When an external microphone was added all devices had a mean difference less than 1 dB. A student's t-test found that devices using the iMM-6 and i436 microphones did not have significantly different measurements than the reference (p= 0.8825 and p= 0.7610, respectively).

Table IV.

Mean difference (dB) between various smartphones, running the SoundMeter application, and a SLM stratified by device and microphone.

Device		Microphone
		iMM-6	Internal	i436
iPhone 4A	Mean		24.99
	SD		0.12
	N		54
iPhone 4S	Mean	-0.11	-1.09	0.50
	SD	0.091	4.08	0.085
	N	54	54	54
iPhone 5S	Mean	0.02	1.76	0.82
	SD	0.08	1.39	0.082
	N	54	54	54
iPod 5G	Mean	-0.55	2.78	-0.01
	SD	0.09	0.16	0.07
	N	54	54	54

^AThe iPhone 4 was not compatible with the external microphones

The results of the one-way ANOVA comparing the mean difference of all the devices running the SoundMeter application found that the difference between the devices to be highly significant (p<0.0001) in all cases. The results of a subsequent Tukey's multiple pairwise comparison between the different devices are presented in table V. Only the 5^th generation iPod and iPhone were found to not have significantly different mean differences.

Table V.

Tukey's multiple pairwise comparisons for the mean difference in measurements between the different devices and SLM. ^B

Device 1A	Device 2	Mean 1	Mean 2	Difference	HSD Test Statistic
iPhone 4	iPhone 4s	17.01	0.21	16.80	133.10C
	iPhone 5s	17.01	1.08	15.94	126.25 C
	iPod 5G	17.01	1.35	15.67	124.09 C
iPhone 4s	iPhone 5s	0.21	1.08	0.87	6.86 C
	iPod 5G	0.21	1.35	1.14	9.01 C
iPhone 5s	iPod 5G	1.08	1.35	0.27	2.15

^AN = 54 for each microphone

^BAll devices were running the SoundMeter application

^CIndicates a significant (p<0.05) difference

Discussion

The results from experiment 1 indicate that it is possible to use different iOS smart devices to make accurate noise measurements under certain conditions. However as Table II shows, the internal microphones on the devices tested are not able to make noise measurements within 2 dB of a reference noise level, which indicates that the internal microphone is not equivalent to a microphone on a Type-2 SLM. This is not surprising, as the internal microphones were designed to only capture a person's voice with sufficient accuracy to communicate information, and not to perform sound level measurements. In addition, when using the NoiSee application with the internal microphone it appears that the application will clip measurements at 90 dBA, effectively limiting the measurement range of this device/application combination. This limits the usefulness of the application as both a SLM and a dosimeter for use in high noise occupational or recreational settings. Based on the results, it appears that smartphone applications measuring noise with the internal microphone should not be used in assessing personal noise exposures.

Our results suggest that an external microphone and source of calibration are needed to make sufficiently accurate noise measurements. This somewhat increases the costs of using smartphones to make noise measurements. However, these microphones are relatively cheap in comparison to the cost of a smart device; the iMM-6 costs approximately $20 while the i436 costs approximately $130. The need for calibration is a larger issue, but calibrators can also be purchased at a relatively small cost. For those without calibration equipment, several applications have pre-defined profiles for certain microphones. However, there has been no evaluation as to the accuracy of using these pre-defined profiles. Additionally, the microphone manufacturer may provide the microphone's sensitivity which can be entered into the application to crudely calibrate the measured levels. Again, there has been no formal investigation in to the accuracy of the measurements using this method, so the results should be interpreted with caution.

Despite the additional technical challenges of using an external microphone the results presented in Table II and Figure 1 indicate that using external microphones is crucial for accuracy. Although the results in Table IV indicate that the iMM-6 microphone performed significantly better than the i436 microphone, both microphones had a mean difference less than 2 dB when compared to the Type-1 SLM. Additionally, the results from experiment 2 show that these microphones may potentially allow different generations of devices to make accurate noise measurements when running the same application. The results of the t-test indicated that the measurements made by devices using either the iMM-6 or i436 external microphones did not differ significantly from the Type-1 SLM. However, as the results from the ANOVA and Tukey's multiple pairwise comparison tests indicates there is still a significant difference between different devices using the same microphone and application. While the different generations of smartphones may give accurate results, the results may vary between different devices.

Another complicating factor in using smartphones to perform noise measurements is the selection of an application. The 3 applications evaluated in this experiment were chosen based on the results from Kardous and Shaw (2014).⁶ Based on the results in Table II & Table IV the SoundMeter application performed better than the other two applications. However, it is important to consider that between 2013 and 2015 Apple has gone from the 8^th to the 9^th iteration of iOS, and additional applications may have been added, removed, or updated in the iTunes application store. For instance, NoiSee has not been updated since 2012. The speed at which applications and software change makes it difficult to say with absolute certainty which application will provide the most accurate measurements. However, the fact that the developer of the SoundMeter application produces other products in addition to the smartphone application makes it likely that the application will continue to be supported in the near future.

Several studies have examined the accuracy of various smartphone applications to measure noise. However, these studies have only evaluated the accuracy of internal microphones. The results from this experiment again demonstrate that generally the internal microphone should not be relied on to make accurate noise measurements.^6,7,12 The only exception has been found by Murphy et al. (2016), who reported that the Sound Level Analyzer Lite (SLA Lite) application for iOS had a mean difference ranging from -0.76 to 0.57 dB.¹³ This is encouraging because using the device's internal microphone reduces technical and logistical barriers to making accurate measurements and more closely emulates how a typical layperson would use their smart device. However, Murphy et al. (2016) also noted that the accuracy of smart devices varied widely, especially for devices running the Android operating system. As demonstrated here, using external microphones greatly reduces the variation of the measurements in different generations of iOS devices. It is possible that using an external microphone can also increase the accuracy and reduce the variability of measurements made by Android devices, but this has not yet been evaluated.

It is also worth noting that Murphy et al. (2016) was examining the accuracy of smart devices for general environmental noise measurements. In this context it is logical to assume that the increased variability from using the device's internal microphone is less important because of the potential to collect hundreds or thousands of measurements. However, in instances where a large number of samples cannot be collected the large measurement variability can drastically impact the exposure estimate. This is especially true in the workplace where samples sizes are typically much smaller, and where overestimation of exposures can lead to the implementation of costly controls, while underestimation of exposures can result in workers not being adequately protected from hazardous noise exposure.

Conclusions

This study expands on previous studies by evaluating applications that were previously identified to be the most accurate in conjunction with inexpensive external microphones. The use of these external microphones dramatically increased the accuracy and precision of the measurements made by the smart devices that were evaluated. The results presented here were from measurements made in a continuous noise environment. Further studies should be conducted looking at the performance of smartphones in calculating noise dose in an environment with intermittent or rapidly changing noise. Despite the technical challenges that were discussed, the results of this study indicate that in certain situations smartphones running the correct application and equipped with an external calibrated microphone can collect noise measurements just as accurately as a Type-2 SLM. It is very unlikely that smartphones will be used for compliance measurements in the near future. However, smartphones have significant value as survey tools, and as SLMs in low resource areas. In addition, these devices can be used to map environmental noise in a community by utilizing a smartphone's GPS function.^14,4,15,3 Finally, as sensor technology improves it may be possible to collect data on multiple physical hazards at once by using the smartphone as the device that stores and exports the data from the sensors.