Usability of a Daily Noise Exposure Monitoring Device for Industrial Workers

Abstract

Objectives:

Usability is an important but often overlooked aspect of personal protective equipment technology. As part of a worksite intervention trial of a new technology for prevention of noise-induced hearing loss that allows workers to monitor their noise exposure inside of hearing protection on a daily basis, we studied the usability of the daily noise exposure monitoring device.

Methods:

We conducted surveys and focus groups for workers enrolled in an intervention trial of daily use of a noise dosimeter with a microphone fitted inside of an individual’s hearing protector (QuietDose). Volunteers completed a baseline and annual survey that included questions about perceived usability of the QuietDose device. Responses to usability questions on the annual survey were abstracted and compared to whether the individual was still using the device. Finally, 16 in-depth focus groups were conducted with subjects to qualitatively explore common themes regarding the usability of the technology.

Results:

Reported problems downloading data or starting and stopping the monitoring device and/or ear discomfort were associated with whether individuals chose to continue monitoring and downloading their noise exposure data. Perceived benefits of the technology included the perception that it could help preserve hearing.

Conclusions:

A novel technology that allows workers to record noise exposures inside of hearing protectors on a daily basis has been developed. Current users of the device report positive perception about how the device is helping them prevent noise-induced hearing loss. However, in its current version, users reported a number of usability barriers that are associated with stopping use of the device. These barriers to use should be addressed as the technology progresses.

INTRODUCTION

Noise-induced hearing loss is one of the most widespread occupational conditions (Karwowski, 1998), and occupational noise exposure is one of the most prevalent hazardous exposures in workplaces. In the early 1980s, it was estimated that >9 million US workers were exposed to ambient noise levels averaging >85 dBA for an 8-h workday (Karwowski, 1998). Hearing conservation programs often rely on the use of hearing protective devices to reduce harmful noise exposures. Yet, evidence suggests that the ‘real world’ effectiveness of hearing protection devices in practice varies widely (Berger et al., 1998) and that as a result, hearing loss continues to occur among individuals participating in hearing conservation programs (Rabinowitz et al., 2003). Comparison of hearing loss rates with recorded ambient noise exposures has suggested that individuals in high ambient noise jobs may lose less hearing than workers exposed to lower noise levels, due to differential use of hearing protection (more effective use in environments with consistently high noise levels) (Rabinowitz et al., 2007).

It is therefore critical that efforts to further improve hearing conservation programs address issues of both usability and effectiveness of hearing protective devices. Recently, a new technology has been developed that allows workers to monitor, on a daily basis, their daily noise exposure inside of their hearing protectors. This technology involves small microphones that fit inside of the hearing protective device (earplug or earmuff) and are connected by wires to a dosimeter that is worn by the worker. The dosimeter measures the cumulative noise exposure over the work shift, and at the end of the work shift, the worker downloads their noise exposure data into a computer and views on a screen their daily noise ‘dose’ received, expressed as a percentage of the OSHA permissible exposure level of 90dB for an 8-h time-weighted average (TWA). Therefore, if the worker experienced a TWA noise exposure of 90 dB over the work shift, the download computer screen would display a 100% dose result (Rabinowitz et al., 2010).

The usability of personal protective equipment (PPE) is an important factor in the effectiveness of such equipment, but this quality is often overlooked. Usability cannot be directly measured (Nielsen and Levy, 2003), but it has been studied by measuring various different usability parameters and metrics. Nielsen (1994) presented a model consisting of five usability parameters: learnability, efficiency, memorability, error avoidance, and subjective satisfaction. Another well-known model, presented in the International Organization for Standardization (ISO) 9241-11: guidance for usability (1998), consists of the concepts of effectiveness, efficiency, and subjective satisfaction. It is believed that perceived tolerability and usability along with comfort will strengthen prevention efforts. There are some studies that have shown some historical tension in the injury prevention and control field about the use and relative effectiveness of ‘active’ behavioral strategies and ‘passive’ environmental/structural strategies, fueled by the success of environmental measures in public health and safety. Gielen and Sleet (2003) suggest that injury reductions will not be achieved unless behavioral components are addressed in concert with environmental and structural changes.

Some studies have explored links between psychosocial epidemiologic predictors and the use of PPE among college students. Williams-Avery and MacKinnon (1996) found that the major Health Belief Model (HBM) constructs (perceived barriers to wearing gear, perceived susceptibility to injury, perceived severity of injury, and perceived benefits of wearing gear) were significant predictors of protective gear use in a group of 217 US college student in-line skaters (Williams-Avery and MacKinnon, 1996). Over 40% of those surveyed owned at least one item of PPE but only 6.5% reported that they consistently wore their protective gear (Williams-Avery and MacKinnon, 1996). The HBM, tested using regression and structural equation modeling, predicted typical gear worn, frequency of gear use, and injuries received while in-line skating (Williams-Avery and MacKinnon, 1996). Barriers, perceived susceptibility to injury, and perceived benefits of equipment use were most highly correlated to gear use and the authors suggest that these constructs may be important targets for injury prevention strategies (Williams-Avery and MacKinnon, 1996).

The novel device described in this study provides workers with the ability to record noise exposures inside of hearing protectors and to log such noise data on a daily basis (Tak et al., 2009). Preliminary evidence suggests that such daily noise monitoring, in a mandatory use program, may lead to a reduction in noise-induced hearing loss over time (Rabinowitz et al., 2010); it is therefore important to assess the usability of this promising technology. We therefore performed quantitative and qualitative analyses of the usability of the noise monitoring device, with a focus on the perceived benefits, perceived susceptibility to hearing loss, and perceived barriers to use of the technology.

METHODS

Selection of subjects

Subjects who were already enrolled in the worksite intervention trial of the QuietDose daily noise exposure monitoring device were eligible for this usability study. These subjects included workers in two aluminum smelters whose jobs ranged from maintenance mechanics to potroom operators. These subjects had previously provided informed consent to participate in the daily noise exposure monitoring intervention. During visits to the study facilities, study subjects were invited by the study personnel to participate in focus groups regarding the usability of the monitoring device, and interested individuals provided additional consent to participate in these groups. In addition, all study subjects were asked by mail to complete an annual written questionnaire that asked specific questions about usability. Informed consent and study protocols were reviewed and approved by the Human Investigation Committee of the Yale University School of Medicine, the Institutional Review Board of the Stanford University School of Medicine, as well as the Alcoa Occupational and Environmental Health Advisory Committee (OEHAC), an independent advisory committee overseeing scientific research within Alcoa.

Assessment of usability

Questionnaires.

Annual surveys were developed and distributed to the study subjects to assess their experiences using the QuietDose device. Surveys contained open and closed questions that were designed to allow respondents to express feelings and expand on ideas about usability and technology improvements for the QuietDose. Additionally, the surveys were designed to ask the respondent to rank order responses based on value judgment. Exclusion and inclusion of survey questions were evaluated for feasibility in terms of job hazard analyses and work-specific tasks for the various workgroups involved in this study. The survey was administered via mail since the study population reported limited access and use of email and other web-based formats.

Table 1 and Table 2 show the usability questions included in the annual survey of volunteers.

Table 1.

Usability questions from the annual questionnaire.

1. Please check whether you have problems with any of the following:

Issue	Never	Sometimes	Usually	Always	Comments
Problems starting or stopping unit for day
Wires catching on things
Problems taking units out when on break
Problems wearing unit in pocket
Problems with batteries running out
Ear discomfort
Problems downloading *(computers, software, etc.)
Problems with helmet clip
Too long of a wait
Wears Tyvek suits
Not enough exposure
2. Please check whether you agree or disagree with any of the following:

Experience issue	Strongly agree	Agree	Disagree	Strongly disagree
The QuietDose unit helps me control noise exposure
The QuietDose unit is helping me preserve my hearing
The QuietDose unit has given me important information about the way I wear hearing protection

Table 2.

Excerpt of the improvements to usability questionnaire.

Improvements to usability	Agree/disagree/no improvement	Description
1. Easier downloads	Agree [ ] Disagree [ ] No Improvement [ ]
2. Device wires (e.g. the wires should hang up, the wires should not pull on my ears when I turn my head, and wires coming out of the unit should pivot on post, etc.)	Agree [ ] Disagree [ ] No Improvement [ ]
3. Smaller unit	Agree [ ] Disagree [ ] No Improvement [ ]
4. Use rechargeable batteries	Agree [ ] Disagree [ ] No Improvement [ ]
5. Convenient download stations	Agree [ ] Disagree [ ] No Improvement [ ]
6. Other	Agree [ ] Disagree [ ] No Improvement [ ]

Furthermore, the survey asked employees to think about future usability improvements that would make the device easier to use and whether they would recommend the QuietDose device to a coworker.

The responses from the survey were numerically coded and entered into SAS 9.01 (SAS Institute, Cary, NC, USA) for further analysis. Participants received a $25 stipend for completion of each survey. This incentive helped stabilize the sample size under investigation, made the study appealing to new participants, and helped minimize loss to follow-up.

Focus groups

To supplement the survey results, a series of focus groups were conducted with study participants at the plant facility. A total of 30 employees volunteered to participate in focus groups. In all, 16 sessions with between two and five participants per session were held to understand differences in perspectives between groups and disclose factors that influence opinions and behaviors. To construct the groups, we randomly selected subjects who remained in the study and who dropped out which created diverse group dynamics. These focus groups explored the entire annual questionnaire in greater detail and allowed the team to examine the impact of peer influence. Qualitative data collection was accomplished through administering open–ended questions focused on the participants’ perceptions of culture and appeal of using the QuietDose device, attitudes and group norms about injury risk and risk taking, and the barriers to wearing PPE. Additionally, the focus groups analyzed employee ideas to improve the comfort and tolerance of the device as well as their responses to the notion of wearing compulsory PPE in the workplace. Notes were transcribed in detail using the constant comparative method (Strauss and Corbin, 1998). Codes that cataloged key concepts were assigned to reoccurring themes. To ascertain whether a code was assigned appropriately, we compared text segments with segments that had been previously assigned the same code to ensure overall congruity (Strauss and Corbin, 1998).

RESULTS

Demographics

Table 3 shows the demographic and other characteristics of the 50 study subjects who completed the questionnaire. The average age of 49.30 years and 17 plus years of previous employment indicate that a majority of subjects in the sample experienced prior exposure to occupational noise at the initiation of intervention. Thirty-two percent of the study subjects had >25 years of experience working at their current employer. The majority of subjects were white and the majority had a history of smoking. Fifty-eight percent of respondents reported consistent use of the quiet dose device.

Table 3.

Characteristics of the study population (N = 50).

Characteristic	N (%)a
Age (years), mean ± SD	49.30 ± 9.93
Sex
Male	44 (88.0)
Female	6 (12.0)
Race
Non-Hispanic White	47 (94.0)
Hispanic or Latino	2 (4.0)
Alaskan Indian or Native American	1 (2.0)
Hearing impairment
Yes	7 (14.0)
No	43 (86.0)
Salary versus hourly
Salaried	3 (6.0)
Hourly	47 (94.0)
Smoking history
Never	17 (47.22)
Former	19 (52.78)
Current	− (0.0)
Plant location
Location 1	10 (21.3)
Location 2	37 (78.7)
Tenure (years), mean ± SD	17.86 ± 11.32
Number of years employed (tenure)
<5	7 (14.0)
5–10	8 (16.0)
10–15	6 (12.0)
15–20	9 (18.0)
20–25	4 (8.0)
>25	16 (32.0)
Stopped downloading before 12/31/10
Yes	21 (42.0)
No	29 (58.0)

^aNumbers may not sum to 50 due to missing data, and percentages may not sum to 100% due to rounding.

Predictors of downloading status

Table 4 shows the association between demographic and questionnaire response variables and whether subjects were currently downloading data using the noise monitoring device. Age and sex were not significant predictors of current use (downloading) status. Race was a significant predictor with a P-value of <0.04; however, there were only three ‘minority’ individuals, all of whom happened to fall into the non-downloading group. Of individuals who had stopped downloading, 55% reported problems stopping or starting the QuietDose unit. Among those who continued to download, only 14% reported problems stopping or starting the QuietDose unit. Reported ear discomfort was a significant predictor of downloading status. Among individuals that had stopped downloading, 50% noted ear discomfort with the device compared to 17% of consistent downloaders who reported ear discomfort. 96.6% of individuals who consistently downloaded throughout the study used the QuietDose unit daily or most of the time (P < 0.001). Individuals who consistently downloaded tended to be more likely than those who had stopped downloading to believe that using the unit helped them control their noise exposure (89 versus 79%) and helped preserve their hearing (96 versus 84%), although these differences were not statistically significant. After analyzing the bivariate associations, the data were analyzed using correlation and logistic regression techniques (Table 3) which only included covariates with bivariate associations displaying P-values < 0.10. Individuals who thought the QuietDose took too long to operate were 11 times more likely to have stopped. Individuals who had problems stopping or starting the QuietDose were seven times more likely to stop downloading before the end of follow-up (i.e. criteria date) than not. Finally, individuals who experienced ear discomfort while wearing the QuietDose were five times more likely to stop downloading before the criteria date than not (Table 5). Although race was a significant characteristic in the study, it was omitted from the multivariate model because it lacked variability within the sample. Correlations between downloading status (x-axis) and the various usability characteristics of interest (y-axis) investigated are presented in Table 6.

Table 4.

Bivariate associations between individual characteristics and current downloading status^a (bivariate model).

Characteristic	Downloading		P †
	Yes (N =29)b	No (N =21)a b
Age (years), mean ± SD	50.31 ± 9.09	47.90 ± 11.06	0.334
Sex, n (%)			0.180
Male	24 (82.8)	20 (95.2)
Female	5 (17.2)	1 (4.7)
Salary, n (%)			0.372
Salaried	1 (3.5)	2 (9.5)
Hourly	28 (96.6)	19 (90.5)
Age, n (%)			0.206
1–50	10 (34.5)	11 (52.4)
<50	19 (65.5)	10 (47.6)
Race, n (%)			0.036
Non-Hispanic White	29 (100.0)	18 (85.7)
Other	0 (0)	3 (14.3)
Hearing impairment, n (%)			0.096
Yes	4 (13.8)	3 (14.3)
No	25 (86.2)	18 (85.7)
Helps control my noise exposure, n (%)			0.304
Yes	26 (89.7)	15 (79.0)
No	3 (10.3)	4 (21.1)
Helps preserve my hearing, n (%)			0.130
Yes	28 (96.6)	16 (84.2)
No	1 (3.5)	3 (15.8)
Problems starting or stopping unit, n (%)			0.003
Yes	4 (14.3)	11 (55.0)
No	24 (85.7)	9 (45.0)
Ear discomfort, n (%)			0.015
Yes	5 (17.2)	10 (50.0)
No	24 (82.8)	10 (50.0)
Too long of a wait, n (%)			0.013
Yes	1 (4.0)	6 (31.6)
No	24 (96.0)	13 (68.4)
Wears Tyvek suits, n (%)			0.077
Yes	4 (15.4)	7 (38.9)
No	22 (84.6)	11 (61.1)
Problems with helmet clip, n (%)			0.350
Yes	2 (18.2)	5 (31.3)
No	18 (81.8)	11 (68.8)
Not enough noise exposure, n (%)			0.166
Yes	3 (13.6)	6 (31.6)
No	19 (86.4)	13 (68.4)
Need to improve downloading stations, n (%)			0.511
Yes	12 (42.9)	9 (52.9)
No	16 (57.1)	8 (47.1)
Need to improve the size of unit (smaller), n (%)			0.095
Yes	20 (71.4)	13 (72.2)
No	8 (28.6)	5 (27.8)
Unit provides useful information about my hearing, n (%)			0.413
Yes	28 (96.6)	19 (100.0)
No	1 (3.5)	0 (0.0)
Used the unit (frequency), n (%)			<0.001
Daily/most of the time	28 (96.6)	9 (47.4)
Sometimes/rarely	1 (3.5)	10 (52.6)

^aNumbers may not sum to totals due to missing data. ^bColumn percentages may not sum to 100% due to rounding.

†P-value for analysis of variance F-test (continuous variable) or χ² test (categorical variable).

Table 5.

Multivariate predictors of downloading status^a.

Effect	Odds ratios	95% Wald confidence limits	P > chi-square
Problems starting or stopping unit	7.3	1.85–29.07	0.005
Ear discomfort	4.8	1.31–17.65	0.018
Too long of a wait	11.1	1.20–102.12	0.034
Need to improve downloading stations	3.9	0.97–16.31	0.054
Used the unit (frequency)	0.03	0.004–0.29	0.002
Hearing impairment	1.0	0.21–5.24	0.096
Wears Tyvek suits	3.5	0.84–14.56	0.085
Need smaller unit	1.0	0.28–3.88	0.095

^aMultivariate logistical model—analysis of maximum likelihood estimates

Table 6.

Descriptive Statistics and Pearson’s Correlation Coefficients among Usability Variables and Downloading Status^a.

	Usability variables	Mean	SD	1	2	3	4	5	6	7	8
1	Problems starting or stopping unit	0.31	0.47	—
2	Ear discomfort	0.30	0.47	0.22	—
3	Too long of a wait	0.15	0.37	0.37*	0.26	—
4	Need to improve downloading stations	0.46	0.50	−0.67	0.28	0.15	—
5	Used the unit (frequency)	0.77	0.42	−0.30*	−0.52	−0.18	−0.08	—
6	Smaller unit	0.72	0.46	0.13	0.21	0.16	0.10	0.01	—
7	Wears Tyvek suit	0.25	0.44	0.16	0.14	0.25	0.29	−0.14	0.16	—
8	Hearing impairment	0.14	0.35	−0.15	0.24	−0.02	0.18	−0.06	0.22	−0.11	—
9	Stopped downloading	0.42	0.49	0.43**	0.34*	0.37*	0.10	0.57**	0.01	0.01	0.01

^aStrength of the linear relationship between two usability variables.

*P < 0.05; **P < 0.01 two-tailed.

Usability characteristics which were not significantly correlated with downloading status included an expression of the need to improve downloading stations, favoring smaller units, wearing Tyvek suits, or the presence of hearing impairment based on the American Medical Association (AMA) definition of hearing impairment (average hearing thresholds in either ear >25 dB for the audiometric frequencies of 0.5, 1, 2, and 3 KHz; AMA, 2008).

Figure 1 depicts the number of subjects enrolled in the study who stopped downloading before the end of the follow-up date of 31 December 2010. Of the number of subjects who stopped downloading, roughly 24% dropped out after the first 2 months of being in the study. Additionally, 29% of the subjects dropped from the study after a 1-year engagement—coinciding with company layoffs due to the economic downturn.

Fig.1.

The number of subjects who stopped downloading before the end of follow-up.

Results of focus groups

Common themes.

The following themes were identified from the focus group interviews:

1. Self-efficacy:

*I can do more to protect myself due to my use of daily monitoring

A common theme among the subjects was that they believed that they used their hearing protection more as a result of what they had learned about their level of noise exposure from the daily monitoring. Some employees were shocked to learn their true noise exposure levels and are now using the appropriate hearing protection. A key finding was that some subjects used the results of the daily monitoring to identify noise sources in the factory and work with management to reduce such exposures.

1. Perceived benefits:

*Daily monitoring helps protect my hearing

Employees reported that the QuietDose has improved their hearing protection. Three groups mentioned that ‘the exposure data that they receive from the QuietDose serves as a reminder to continuously ensure that hearing protection equipment fits properly in order to guarantee full protection for employees’. One employee indicated that ‘they wanted to keep whatever hearing they had left because their group is exposed to more noise than other groups at the facility’. Five of the 16 groups reported that ‘they simply did not realize how loud it really was because they have become so accustomed to their exposures that they do not wear their hearing protection’. Some employees reported learning that their level of noise exposure changed dramatically depending on their positioning at their worksite. Others noted that individuals making noise near them produced higher noise readings.

1. Barriers:

A major barrier to use reported by the focus groups was the interference of the wires on the device with daily activities. Another barrier was the weight of the unit, especially if placed on a helmet. Ear discomfort was another barrier, as well as the subject perception, that if they were using custom plugs, a change in body position could interfere with the seal of the hearing protection device. While some were wearing the device mounted to a hard hat, subjects reported that this could be difficult in a confined space, such as an air compressor room. Others reported that the device could interfere with the use of a respirator. Finally, some employees mentioned that they had stopped downloading because of trouble stopping or starting the unit.

DISCUSSION

This study of usability of a new device for daily monitoring of occupational noise exposures revealed that perceived barriers to the use of the device, as reported on an annual survey, were significant predictors of whether an individual continued to use the device on a voluntary basis. The results of this study indicate that individuals who had problems with a lengthy download time, stopping or starting the QuietDose, and/or ear discomfort were less likely to continue to use the devices. Some usability characteristics were more significant than others (i.e. stopping or starting the unit) in determining whether an employee stopped downloading in the study. At the same time, qualitative results of focus groups indicate that current users perceived that the device was beneficial in helping them protect their hearing.

This study’s findings corroborate those of Gielen and Sleet (2003) and Williams-Avery and MacKinnon (1996) that employ a HBM to explain the use of preventive measures by individuals. This model implies that perceived barriers to wearing protective equipment (in this case the daily monitoring device) and perceived benefits will be significant predictors of protective equipment use. Indeed, in our sample, these barriers and benefits were predictors of whether individuals continued to use the monitoring device.

There were some limitations to this study. Layoffs and retirements due to the economic downturn reduced the rate of follow-up. The relatively small sample size made it more difficult to detect statistically significant differences between current users and those who had stopped daily monitoring.

Nonetheless, this research can be used by the manufacturers of the QuietDose and by companies interested in using it, to overcome barriers to usability. Factors such as discomfort and ease of operation require technical resolutions, but other barriers such as the time taken to download data could be resolved by administrative changes such as allowing more time just before the work shift ends for this activity.

Future workplace intervention trials using devices that require some user interaction will need to consider device usability as an important component as it may determine the efficacy of an intervention. Similarly, assessing usability helps provide critical feedback to engineers and designers of protective equipment that can be used to improve technology and improve device acceptability and effectiveness.

FUNDING

United States Centers for Disease Control and Prevention/National Institute for Occupational Safety and Health (1 R01 OH008641-01A2 CDC/NIOSH), (supplemental grant number 3 R01OH008641-03W1 CDC/NIOSH). This usability assessment was conducted with no commercial interest in the technology tested.