Virtual accident curb risk habituation in workers by restoring sensory responses to real-world warning

Summary

In high-risk work environments, workers become habituated to hazards they frequently encounter, subsequently underestimating risk and engaging in unsafe behaviors. This phenomenon has been termed “risk habituation” and identified as a vital root cause of fatalities and injuries at workplaces. Providing an effective intervention that curbs workers’ risk habituation is critical in preventing occupational injuries and fatalities. However, there exists no empirically supported intervention for curbing risk habituation. To this end, here we investigated how experiencing an accident in a virtual reality (VR) environment affects workers’ risk habituation toward repeatedly exposed workplace hazards. We examined an underlying mechanism of risk habituation at the sensory level and evaluated the effect of the accident intervention through electroencephalography (EEG). The results of pre- and posttreatment analyses indicate experiencing the virtual accident effectively curbs risk habituation at both the behavioral and sensory level. The findings open new vistas for occupational safety training.

Graphical abstract

Highlights

• •Workers exhibited neural evidence of sensory habituation to warning signals
• •Brief virtual reality (VR) training mitigated such habituation
• •VR experience also enhanced workers’ vigilant behaviors to hazards at work
• •VR is a promising intervention tool to curb risk habituation in workers

Behavioral neuroscience; Cognitive neuroscience; Techniques in neuroscience

Introduction

Humans can become habituated to a variety of sensory signals.^1,2,3 In high-risk workplaces, workers often become habituated to risks associated with tasks they perform frequently and consequently engage in unsafe behaviors.^{4,5,6,7,8,9,10} This behavioral tendency has been defined as risk habituation and identified as one of the key contributors to occupational injuries and fatalities.^9,11 For example, in road work zones, workers’ vigilance to approaching struck-by hazards (e.g., construction vehicles) is apt to diminish after frequent exposures.^{12,13,14,15,16} In many instances of struck-by fatal accidents between construction vehicles and pedestrian workers, the construction vehicles were traveling at a speed of less than 10 mph^9,12,17; workers failed to avoid the approaching vehicles because they ignored the warning alarms from the vehicles.¹⁸

In spite of periodic safety training, workers tend to focus on their work tasks while ignoring hazards they routinely encounter.^19,20,21 Curbing workers’ risk habituation is thus critical to prevent injuries and fatalities at workplaces. Theoretically, a human’s habituation can be identified when stimulus-evoked responses decrease in amplitude (e.g., the intensity of sensory responses in the brain) and/or frequency (e.g., the number of behavioral responses elicited by the stimulus) with repeated exposure.^1,3,22,23 Observing the developmental process of workers’ risk habituation in real-world settings is quite challenging,²⁴ and as a result, the treatment of risk habituation within the scientific literature has been largely at the conceptual level. There is very little research that directly measures and quantifies workers’ risk habituation, and there exists no empirically supported intervention for curbing workers’ risk habituation, particularly at the sensory level.

The advancement of virtual reality (VR) technologies offers solutions for observing workers’ risk habituation. VR enables us to expose workers to close-to-real hazardous situations without risking actual injury and observe workers’ behaviors in response to repeated exposure to workplace hazards.^25,26,27,28 VR also allows us to provide time-sensitive interventions when workers’ risk habituation is observed. In our previous study, we developed a method of observing and an intervention that curbs behavioral indicators of risk habituation using VR technologies.²⁹ Our intervention is based on the principle of one-shot learning—rapid and substantial changes in human behaviors associated with a salient episodic memory.^30,31,32 It has been theorized that workers who experienced an occupational injury or accident in the past perceive greater risks associated with workplace hazards and tend to behave more safely.^4,12,33 In an effort to elicit similar changes in participants’ risk perception of frequently encountered hazards, in our previous study, we exposed naive student participants to a virtual accident.²⁹ Furthermore, to empirically examine the effect of experiencing a virtual accident on risk habituation, we created a virtual work environment in which visual attention could be measured. The VR environment exposes participants to repeated struck-by hazards associated with construction vehicles and measures participants’ vigilant orienting behaviors in response to auditory warning alarms from those construction vehicles. During the experiment, participants experienced a virtual accident upon the emergence of habituated behaviors. Although experiencing the VR accident promoted vigilant behaviors in the student participants,²⁹ such vigilant behaviors could simply reflect compensatory goal-directed attention to a revealed threat (i.e., participants actively monitor the vehicle’s movements once they learn it can strike them in the VR task). Whether such vigilant behaviors are the result of changes in how the sensory experience of the alarm is processed is not known, nor is whether such an intervention could curb behavior in experienced construction workers who are subject to significant levels of real-world risk habituation. To this end, the present study aimed to examine the consequence of experiencing the VR accident at the sensory level.

Here, we examined the potential neural indicators of workers’ risk habituation using electroencephalography (EEG). Event-related potentials (ERPs) provide a measure of stimulus-evoked neural responses with high temporal precision.³⁴ ERPs could allow us to determine whether experiencing the VR accident alters the early or late stages of attentional processing. A modulation of early components is thought to be associated with perceptual, more automatic processing,³⁵ while later components are generally associated with more strategic, controlled cognitive processes.³⁶ In two sessions, performed before and after the VR intervention, we recorded ERPs to alarm and control sounds using the equiprobable paradigm (a variant of the oddball paradigm^37,38). The alarm sound was similar to the warning signal used in both our VR environment and real construction sites. We mainly focused on two ERP components, one early (N1) and one late (P3), known to be sensitive to auditory habituation.^39,40,41 Thus, EEG measures aimed to clarify the cognitive mechanisms that underlie the attenuation of risk habituation to warning signals after experiencing an accident in the VR environment. We hypothesized that construction workers would show a blunted ERP response to the alarm sound compared to the control sound and that this difference would either diminish or go away after experiencing the virtual accident. Such an outcome for the N1 component would provide direct evidence for the effect of the VR accident experience on the automatic sensory processing of warning signals. In contrast, the same outcome for the P3 component would suggest that the increase of vigilant behaviors observed after a VR accident²⁹ ensues from controlled cognitive processes, such as goal-directed attention mechanisms. We supposed that an alteration of the automatic sensory processing, reflected by the N1 component, would have more long-lasting effects on behavior, and in consequence, would strongly support the utility of the VR intervention to curb risk habitation in construction workers.

Results

Decrease in vigilant orienting behaviors toward auditory warning alarms

As repeated exposure to workplace hazards might lead to a decrease in workers’ vigilant behaviors as an attentional consequence of risk habituation, we examined (1) the magnitude of reduction in workers’ vigilant behaviors to repeatedly exposed auditory warning alarms from construction vehicles and (2) the effectiveness of experiencing the VR accident caused by participants’ low vigilance (Figure 1A). At a construction site, workers are apt to direct most of their attention to their work tasks and pay less attention to surrounding hazards.^42,43,44 Thus, to create an immersive virtual environment and accelerate participants’ habituation, we designed a virtual road-cleaning task. The task involved removing all debris and cleaning the entire surface of the road with a broom. The participant’s actual/physical sweeping motion was synchronized in the VR environment via motion controllers attached to a real broomstick (Figure 1B). In the experiment, while participants were performing the road-cleaning task, construction vehicles continuously moved back and forth. Auditory warning alarms sounded to warn the proximity of the vehicles (Figure 1C). Measuring the decrease in participants’ visual attention to an approaching hazard provides an empirically rigorous way of observing the development of risk habituation.^45,46 Therefore, we measured participants’ vigilant orienting behaviors using the eye-tracking sensors embedded in the VR headset (Figure 1D). One exposure to the hazard was defined as one reciprocal movement of the vehicle moving behind the participant. In each exposure to the hazard, when a participant looked back and checked the approaching vehicle, the response time—the elapsed time between the presentation of the auditory warning alarm and a participant’s hazard-checking behavior—was documented. The frequency of vigilant behaviors (checking rate) was also recorded. A VR accident was triggered upon repeated ignoring of the approaching vehicle (see Figure 1E and STAR Methods).

Figure 1.

Experimental procedure and intervention model

(A) The experiment timeline. Each experiment session consisted of the VR experiment (preceded by practice) and an EEG session measuring sensory responses to signals. The two experiment sessions were scheduled a week apart.

(B) Schematic of the VR experimental setup with a VR headset, motion controllers, and eye-tracking sensors embedded in the VR headset. The participants’ task was removing all debris and cleaning the entire surface of the road with a broom. The participant’s actual/physical sweeping motion was synchronized in the VR environment via motion controllers attached to a real broomstick.

(C–E) Schematic of the experimental and intervention scenario. While the participants were performing the road-cleaning task, construction vehicles continuously moved back and forth. Auditory warning alarms sounded to warn the proximity of the vehicles. The eye-tracking sensors documented participants’ vigilant orienting behaviors. The participants' ignorance of approaching vehicles triggered a VR accident.

(F) The overview scene of the VR road construction environment.

(G and H) The participants performed the virtual road-cleaning task using a real broom with motion controllers.

(I) The example of participant’s vigilant orienting behavior to check approaching hazards during the VR task.

In the first VR session, 20 out of 31 participants experienced the VR accident triggered by their ignorance of the approaching vehicles (i.e., accident group; AG), and 11 participants did not experience the accident because they exhibited regular hazard checking while performing the task (i.e., no accident group; NAG).

Does the repeated exposure to struck-by hazards result in a delay in workers’ vigilant behaviors in the VR environment? To answer this question, we tested a bivariate linear regression model. The regression model for AG positively predicted participants’ response time (vigilant orienting behavior) in the first VR session (Figure 2A; R² = 0.13, F(1, 54) = 7.16, p = 0.008; B₁ = 0.49, p = 0.008, 95% confidence interval [CI] (0.13, 0.85)). With the increase in the number of exposures to the hazards, the AG’s vigilant behaviors were delayed. Consequently, participants in AG experienced the VR accident. The regression model for NAG was also significant (Figure 2A; R² = 0.02, F(1, 249) = 5.78, p = 0.017; B₁ = 0.13, p = 0.033, 95% CI (0.02, 0.25)). However, the R² value was relatively low because participants in NAG were constantly vigilant toward the hazards. Thus, the association between repeated exposure to the hazards and workers’ vigilant behaviors was attenuated. To investigate further the difference in habituation tendency between AG and NAG, we tested a multiple linear regression analysis model (Figure 2A; R² = 0.08, F(3, 303) = 8.92, p < 0.001). The result indicated that the interaction between response time and the occurrence of the VR accident approached significance (B₃ = 0.36, p = 0.052, 95% CI = −0.01, 0.71). The result of the first VR session indicated the AG’s vigilant orienting behavior was slowed with the increase in the number of exposures to the hazard, culminating in the triggering of the VR accident. On the other hand, the NAG’s vigilant orienting behavior did not significantly slow down over time. Therefore, to supplement the results of the analysis of response time, we analyzed the frequency of participants’ vigilant orienting behaviors (checking rate). There was a significant difference in checking rate between AG and NAG (Figure 2B; Z = 217, p < 0.001). This further suggests that NAG exercised heightened levels of attentiveness toward approaching hazards as compared to AG and, as a consequence, did not experience the VR accident.

Figure 2.

Behavioral risk habituation and the occurrence of the VR accident

(A) The slopes for the effect of the occurrence of the VR accident on response time in the first VR session (see the STAR Methods section for details) for AG and NAG. The shaded envelope indicated the 95% CI for predictions from the linear model.

(B) The average frequency of checking behaviors (checking rate) of NAG and AG. Standard error bars of the mean are included.

Intervention effect of a single VR accident experience on vigilant orienting behaviors

Can experiencing a single VR accident following behavioral indicators of risk habituation curb the tendency to ignore relevant hazards, and is that intervention effect sustained? To answer these questions, we asked the participants to complete the VR task a week after the first participation. The result from the multiple linear regression model for AG, which examines the effect of number of exposures on response time in the second VR session, indicated a significant negative interaction between response time in the second VR session and the VR accident experience in the first VR session (Figure 3A; B₃ = 0.45, p = 0.007, 95% CI = 0.12, 0.77), while this interaction was not significant in NAG (Figure 3C; B₃ = 0.09, p = 0.29, 95% CI = −0.08, 0.26). In addition, AG presented a significant difference (Figure 3B; t₁₉ = −11.09, p < 0.001, 95% CI (−73.98, −50.48), d = −3.08) in the checking rate between the first and second VR sessions, while NAG did not (Figure 3D; t₁₀ = 0.52, p = 0.62, 95% CI (−0.12, 0.18), d = 0.20). After experiencing the VR accident in the first session, AG exhibited consistent vigilant orienting behaviors in the second VR session. The result confirmed that experiencing a single VR accident triggered by the ignoring of a hazard mitigated participants’ habituation and increased participants' vigilant orienting behaviors.

Figure 3.

Intervention effect on increasing vigilant orienting behaviors

(A) The slopes for the effect of the number of exposures on response time in AG for each VR session. The shaded envelope indicated the 95% CI for predictions from the linear model.

(B) Average checking rate of AG for each VR session. On each box, the central mark indicates the median. The whiskers extend to the highest value and the lowest value.

(C) The slopes for the effect of the number of exposures on response time in NAG for each VR session. The shaded envelope indicated the 95% CI for predictions from the linear model.

(D) Average checking rate of NAG for each VR session. On each box, the central mark indicates the median. The whiskers extend to the highest value and the lowest value.

Intervention effect on sensory-perceptual processing of warning alarms

Participants passively listened to the alarm sound and a matched control sound while neural responses to the sounds were recorded using EEG, both before the initial VR session and after the second session. Grand-average ERP waveforms are illustrated as a function of the type of sound for each EEG session in Figure 4. Analyses performed within the 220–420 ms time window revealed a significant main effect of type of sound, F(1, 18) = 16.44, p < 0.001, ${\eta }_p^2$ = 0.477, with higher ERP amplitudes for the control sound than for the alarm sound, no significant main effect of session, F(1, 18) = 0.24, p = 0.627, ${\eta }_p^2$ = 0.013, and no significant interaction between the type of sound and session, F(1, 18) = 2.53, p = 0.129, ${\eta }_p^2$ = 0.123. These results suggest greater early attention to the control sound than to the alarm sound consistently across the two sessions.

Figure 4.

Grand-average ERP waveforms at the Cz-Fz electrode sites for alarm and control sounds

(A) Before and (B) after the VR training. The gray rectangles indicate the time window of the three ERP components analyzed.

Analyses performed within the 540–660 ms time window revealed a significant main effect of type of sound, F(1, 18) = 8.15, p = 0.011, ${\eta }_p^2$ = 0.312, with higher ERP amplitudes for the control sound than for the alarm sound, no significant main effect of session, F(1, 18) = 0.49, p = 0.494, ${\eta }_p^2$ = 0.026, and crucially, a significant interaction between the type of sound and session, F(1, 18) = 5.12, p = 0.036, ${\eta }_p^2$ = 0.222, indicating that ERP amplitude differences between the control sound and the alarm sound significantly decreased after the VR training. Subsequent t-tests showed that ERP amplitudes were significantly greater for the control sound than for the alarm sound before the VR training, t(18) = 3.06, p = 0.007, d = 0.70, whereas ERP amplitudes between the two sounds did not differ significantly after the VR training, t(18) = 0.58, p = 0.284, d = 0.13.

Analyses performed within the 804–980 ms time window also revealed a significant main effect of type of sound, F(1, 18) = 12.64, p = 0.002, ${\eta }_p^2$ = 0.413, with higher ERP amplitudes for the alarm sound than for the control sound, no significant main effect of session, F(1, 18) = 0.15, p = 0.706, ${\eta }_p^2$ = 0.008, and no significant interaction between the type of sound and session, F(1, 18) = 0.70, p = 0.413, ${\eta }_p^2$ = 0.037. Thus, late attention to the alarm sound was higher than to the control sound, and this effect did not differ across the two sessions.

Participants who experienced at least one accident in the VR training (N = 16)

We performed the same analyses as above after removing participants who did not experience the accident in the VR training (N = 3). Results are overall consistent with those observed in the whole group (see details below). The only notable difference was observed within the 220–420 ms time window.

Grand-average ERP waveforms are illustrated as a function of type of sound for each EEG session in Figure 5. Analyses performed within the 220–420 ms time window after stimulus onset revealed a significant main effect of type of sound, F(1, 15) = 10.49, p = 0.006, ${\eta }_p^2$ = 0.412, with higher ERP amplitudes for the control sound than for the alarm sound, no significant main effect of session, F(1, 15) = 0.25, p = 0.628, ${\eta }_p^2$ = 0.016, and an interaction between type of sound and session close to significance, F(1, 15) = 4.22, p = 0.058, ${\eta }_p^2$ = 0.220, suggesting that ERP amplitude differences between the control sound and the alarm sound tended to decrease after the VR training. Subsequent t-tests showed that ERP amplitudes were significantly greater for the control sound than for the alarm sound before the VR training, t(15) = 3.19, p = 0.006, d = 0.80, whereas ERP amplitudes between the two sounds did not differ significantly after the VR training, t(15) = 0.82, p = 0.423, d = 0.21.

Figure 5.

Grand-average ERP waveforms at the Cz and Fz electrode sites for alarm and control sounds

(A) Before and (B) after the VR training for participants who experienced at least one accident in the two VR sessions (N = 16). The gray rectangles indicate the time window of the three ERP components analyzed.

Analyses performed within the 540–660 ms time window after stimulus onset revealed a significant main effect of type of sound, F(1, 15) = 6.04, p = 0.027, ${\eta }_p^2$ = 0.287, with higher ERP amplitudes for the control sound than for the alarm sound, no significant main effect of session, F(1, 15) = 2.18, p = 0.160, ${\eta }_p^2$ = 0.127, and crucially, a significant interaction between type of sound and session, F(1, 15) = 8.84, p = 0.009, ${\eta }_p^2$ = 0.371, indicating that ERP amplitude differences between the control sound and the alarm sound decreased significantly after the VR training. Subsequent t-tests showed that ERP amplitudes were significantly greater for the control sound than for the alarm sound before the VR training, t(15) = 3.01, p = 0.009, d = 0.75, whereas ERP amplitudes between the two sounds did not differ significantly after the VR training, t(15) = 0.26, p = 0.801, d = 0.06.

Analyses performed within the 804–980 ms time window after stimulus onset revealed a significant main effect of type of sound, F(1, 15) = 8.71, p = 0.010, ${\eta }_p^2$ = 0.367, with higher ERP amplitudes for the alarm sound than for the control sound, no significant main effect of session, F(1, 15) = 0.08, p = 0.778, ${\eta }_p^2$ = 0.005, and no significant interaction between type of sound and session, F(1, 15) = 0.95, p = 0.345, ${\eta }_p^2$ = 0.059.

Discussion

We developed a VR safety training environment that exposes experienced construction workers to repeated struck-by hazards and simulates a virtual accident upon the attentional consequence of risk habituation. We warned participants about potential struck-by hazards and requested that they pay attention to approaching vehicles for safety purposes. However, during the experiment, participants became accustomed to repeated exposure to warning alarms associated with struck-by hazards and focused largely on performing the road-cleaning task. Our data show that, overall, participants became increasingly inattentive to struck-by hazards in our VR task, with a substantial proportion of these construction workers experiencing a virtual accident upon repeated ignoring of the hazard. Our findings demonstrate that construction workers’ risk habituation can be observed in the intentionally designed VR environment, thereby providing the potential for manipulating and intervening in risk habituation, a major causal factor of fatal accidents at workplaces.

We further investigated the intervention effect of experiencing the single VR accident on curbing risk habituation. Experiencing the VR accident significantly affected participants’ risk habituation. For participants who experienced the VR accident, a progressive slowing of response time in visual checking was no longer observed in the second session, and the frequency (checking rate) of participants’ vigilant orienting behaviors significantly increased across sessions, with the intervention effects sustaining over at least one week. The findings dovetail with the results of previous conceptual studies claiming that workers who experienced work-related injuries or accidents in real life tend to perceive greater risks associated with workplace hazards.^4,12,33 The results of this study also suggest the potential of VR as a one-shot learning platform that leads to rapid changes in workers’ behaviors with a salient and memorable accident experience.

Our findings also demonstrate that individual workers have different risk habituation tendencies. During the VR task, some participants consistently responded to the warning alarms with vigilant orienting across both VR sessions. However, participants who engaged in the VR accident in the first session frequently ignored the approaching construction vehicles and showed rapidly decaying vigilant orienting behaviors. Our VR task, therefore, differentiates between participants who are more or less prone to risk habituation. The direct measurement of workers’ behavioral responses in the VR environment may help researchers and construction safety managers identify which workers are risk prone or vulnerable to risk habituation, thereby allowing for the provision of tailored safety training. Such tailored safety training may also help workers recognize their risk habituation tendencies toward repeatedly encountered workplace hazards and allow them to understand when they engage in risky behaviors.

The results of EEG data analyses also support the intervention effect of our VR training on restoring workers’ sensory responses to warning signals. EEG data are consistent with behavioral effects observed in the VR sessions, especially for the ERP component measured within the 540–660 ms time window, which presumably reflects the N1 because this component is the first negative wave response recorded after auditory stimulus onset.⁴⁷ The N1 time window is usually earlier (between 100 and 200 ms after stimulus onset^48,49), but the time discrepancy could be due to the specificity of sounds presented in our study (i.e., long- and non-monofrequency), which largely differ from short- and monofrequency sounds commonly used in auditory EEG studies (e.g., 60 ms³⁹; 50 ms⁵⁰; 70 ms⁴¹). The alarm sound presented in the EEG sessions was extracted from the alarm signal used in real construction sites to increase the ecological validity of our experiment.⁵¹ The auditory N1 component is related to early attention^34,52 and sensitive to habituation.^39,50

Early attention to the control sound was greater than to the alarm sound before completing the VR training, as reflected by the ERP amplitude difference within the 540–660 ms time window, suggesting a consequence of habituation to the alarm sound resulting from real-world construction experience preceding the experiment. A potential way to mitigate this issue in construction sites could be to employ a variety of alarm sounds and present them in random sequences (i.e., the use of a specific alarm sound would not be predictable by workers). This method might significantly reduce habituation to each specific warning signal, and as a consequence, workers would be less inclined to engage in unsafe behaviors stemming from ignorance of a hazard.

Importantly, the ERP amplitude difference between the control and the alarm sound within the 540–660 ms time window significantly decreased after the VR training. This outcome is consistent with results observed within the 220–420 ms time window when analyses were performed only with participants who experienced a VR accident. Thus, we found physiological evidence of the VR training reducing habituation to the alarm sound at an early stage of attentional processing. The mechanism for sensory N1 habituation is thought to be related to refractoriness of cell populations involved in basic sensory processing systems rather than high-level cognitive processes exerting top-down control over sensory cortex,^53,54 suggesting that the reduction of the N1 amplitude difference between the two sounds observed after the VR training is not due to explicit, top-down strategies. No modulation of later ERP components was observed after the VR training. Previous studies reported habituation of auditory stimuli for late components (e.g., P3^41,55), but this outcome could be evidenced only in specific conditions (e.g., when attentional resources available to perform the task are reduced³⁹). Altogether, the EEG data suggest that the VR training enhances early attentional and sensory-perceptual processing of auditory stimuli signaling a potential danger in construction sites.

The present study demonstrates habituation built up from protracted real-world experience in the first EEG session that is largely abolished following the VR accident intervention. That is, our data show that a singular experience can undo the consequences of months to years of routine exposure to sensory signals. Such an apparently profound effect on sensory habituation is, to our knowledge, unprecedented and opens new vistas in studies of perception and learning. The consequences of habituation may be much more plastic and sensitive to change than previously thought, particularly in the context of an affectively salient precipitating event. Future research should explore the modulability of sensory habituation more broadly and how it responds to recent salient events.

Limitations of the study

Several limitations of this study should be noted. To avoid manipulation on participants’ behaviors, during the VR experiment, the VR accident was triggered in response to participants’ vigilant orienting behaviors. Therefore, the samples were unevenly distributed, and the number of participants who did not experience the VR accident was relatively small. During the experiment, some participants also never exhibited any vigilant orienting behaviors. The data from these participants were included only in the vigilant behavior frequency analysis. The Occupational Safety and Health Administration (OSHA) recognizes four leading causes of fatal accidents—falls, struck-by, caught-in/between, and electrocution hazards (i.e., fatal-four hazards).⁵⁶ In this study, we focus on observing and curbing construction workers’ risk habituation to struck-by hazards associated with construction vehicles. Future work could extend the proposed intervention to curb workers’ habituation to other fatal-four hazards.

STAR★Methods

Key resources table

REAGENT or RESOURCES	SOURCE	IDENTIFIER
Software and algorithms
VIVE SRanipal Eye Tracking SDK	HTC Corporation	Version 1.1.0.1
Unreal Engine	Epic Games, Inc.	Version 4.22.3
Autodesk 3dS Max	Autodesk Inc.	Version 2019
Autodesk Maya	Autodesk Inc.	Version 2019
PsychoPy	Open Science Tools Ltd.	Version 2021.1.4
OpenBCI GUI	OpenBCI	Version 5.0.4
OpenSync library	OpenSync	Version 3.0
Logic Pro X	Apple Inc.	Version 10.6.3
Python	Python Software Foundation	Version 3.8.2
Python SciPy toolbox	SciPy	Version 1.4.1.33
MNE-Python package	https://doi.org/10.5281/zenodo.7314185	Version 0.24
EEGLAB	https://doi.org/10.1016/j.jneumeth.2003.10.009	Version 2021.1
Artifact Subspace Reconstruction method	This study (https://github.com/moeinrazavi/EEG-ASR-Python)	Version 1.0.0
MATLAB	Mathworks	Version R2021a
Other
HTC Vive Pro Eye	HTC Corporation
OpenBCI EEG System		OpenBCI Board Kit of 32 bits and USB Dongle, a cap containing 20 electrodes

Resource availability

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Changbum R. Ahn (cbahn@snu.ac.kr)

Materials availability

This study did not generate new unique reagents.

Experimental model and subject details

A total of thirty-five construction workers (32 males and 3 females) aged between 20 and 43 years (mean age: 27.26 years, s.d: 6.09 years) were recruited from a construction company in the United States. All the participants were pedestrian workers working at heavy civil construction projects. The experimental protocol was approved by the Institutional Review Board (IRB) at Texas A&M University (IRB 2019-1270D). Informed consent was obtained from all participants. The experiment was conducted in a quiet space at a safety training facility of the construction company. During the experiment, the access to the facility was controlled. To examine the sustained intervention effect, each participant was asked to participate in a second session one week later. Each session, including the VR experiment and the EEG experiment, took approximately 3–3.5 h. Thus, each participant took a total of 6-7 h to complete both sessions. All participants voluntarily participated in this study. Participants were compensated with $200 for participating in the study. Data from four participants—who did not return for their second study session (N = 2) or withdrew from the experiment because of motion sickness (N = 2)—were discarded. Thus, data from thirty-one participants entered the VR training data analyses. Among the thirty-one road construction workers who completed the two VR sessions, the data of twelve participants were excluded from the EEG analyses due to technical problems/malfunction systems (N = 8; i.e., loss of data due to a dysfunction of the Bluetooth system) or artifacts in the EEG recordings (N = 4; see event-related potential analyses section) in at least one of the two EEG sessions.

Methods details

Overview

We designed and performed the experiment with actual road construction workers (1) to examine workers’ behavioral and sensory habituation to repeated auditory warning alarms from construction vehicles and (2) to validate the sustained intervention effect. An overview of the experimental procedure is provided in Figure 1A. In the first session, the participants first completed the EEG session, then the VR training. A week later, the participants completed the second session, with the VR training first and the EEG session afterward.

Apparatus

Virtual reality experiment

The HTC Vive Pro Eye (HTC Corporation, New Taipei City, Taiwan; resolution: 2880 x 1600 pixels; field of view: 98° horizontal and 98° vertical; refresh rate 90 Hz) and Dell Precision T5820 (Dell, Round Rock, TX, USA; CPU: Intel i9-10900 × 3.7 GHz; RAM; DDR4 128 GB; GPU: Nvidia GeForce RTX 3080) were used to display the VR environment. Eye movement was recorded using eye-tracking sensors embedded in the HTC Vive Pro Eye with a peak frequency of 90 Hz.

EEG experiment

A Dell Precision T3620 (Dell, Round Rock, TX, USA) equipped with PsychoPy software v2021.1.4^57,58,59 was used to present the stimuli on a Dell P217H monitor. The participants viewed the monitor from a distance of approximately 70 cm in a dimly lit room. Participants also wore Etymotic ER4XR 45Ω high fidelity, noise-isolating in-ear earphones (Etymotic Research, Elk Grove Village, IL, USA) to listen to all sounds.

The equipment used to perform measurements by EEG was the OpenBCI Board Kit of 32 bits and USB Dongle (www.openbci.com), with a cap containing 20 electrodes pre-organized according to the international 10-20 system.⁶⁰ For our study, we have used 15 channels from the 10–20 electrode placement system: C3, C4, Cz, F3, F4, Fz, Fp1, Fp2, P3, P4, Pz, T3, T4, T5, T6. We embedded OpenSync library in PsychoPy⁵⁸ to synchronize and record EEG signals with the associated alarm/control task markers.⁶¹ All electrode impedances were kept below 15 kΩ. The EEG was sampled and digitized at 125 Hz.

Virtual reality experiment procedure

Virtual reality environment

A road maintenance working environment in which participants would be part of an asphalt milling crew was selected for the experimental scenario and designed. The VR environment was created using Unreal Engine v.4.22.3.⁶² All components included in the VR environment were created using Autodesk 3dS Max v.2019⁶³ and Autodesk Maya v.2019.⁶⁴ To create a VR environment that effectively elicits participants’ risk habituation, the experimental scenario focused on repeated exposure of participants to potential struck-by hazards associated with construction vehicles, with associated auditory warning alarms from the vehicles. The movement of virtual construction vehicles was designed to respond to a participant’s behavior. The reciprocal back-and-forth movement of a vehicle (i.e., street sweeper) that travels behind a participant is controlled by the real-time measurement of its distance from a participant. When the street sweeper reaches the designated minimum distance to a participant—7.5 m, the street sweeper turns off its warning alarm and begins to reverse, thereby repeatedly exposing a participant to the risk of a potential runover accident without interfering with the participant’s virtual road cleaning task. While a participant is performing the sweeping task, dump trucks repeatedly pass by very close to the participant. All the construction vehicles in the VR environment emit operating sounds and auditory warning alarms (i.e., beeping sounds), and these warning alarms are carefully designed to create a realistic auditory experience. The sound attenuation function was adopted. The volume of audio sources is changed in response to the distance and angle between a participant and the audio sources. A participant thus can recognize the proximity and direction of the approaching hazard based on the warning alarms, similar to a real-world road construction site. To design a close-to-real virtual environment, the experimental scenario was reviewed by three experienced construction safety managers at nationwide heavy civil construction companies.

Task in the VR training

While working at a road construction/maintenance site, workers are apt to direct most of their attentional resources to a work task and become less attentive to workplace hazards.^42,43,44 Thus, we designed a virtual road cleaning task not only to accelerate participants’ risk habituation within a short time period but also to promote active participation in the experiment. During the VR experiment, a participant was asked to perform a road cleaning task in a road maintenance work zone. The task was to remove all debris and clean the surface of the working lane with a broom. A participant’s physical sweeping movement with an actual broomstick was captured by VR motion controllers attached to the broomstick and synchronized in the VR environment with a virtual broom.

Vigilant orienting behavior measurement

In this study, we defined vigilant orienting behavior as hazard-checking behavior—participant’s eye and/or head movement to check approaching construction vehicles. While working at a dangerous workplace, workers must pay attention to a hazard in order to correctly respond to it. Therefore, recognizing and evaluating the risk associated with workplace hazards require selective attention (specifically, visual attention).^45,65,66,67 Although orienting behavior does not always lead to workers’ proper risk perception of workplace hazards,^68,69 visual attention is a vital requisite for enhanced risk perception.⁴⁵ Visual attention is intimately related to eye movements,^46,70 and visual checking of surrounding hazards in workplaces is an essential safety behavior. Therefore, measuring the latency (response time) in participants’ visual attention responses to repeatedly encountered workplace hazards provides a concrete and empirically grounded analytical approach to monitoring risk habituation development. In this regard, to measure the latency pattern in participants’ visual attention to the approaching construction vehicle, an eye-tracking system was embedded into the VR environment. During the experiment, eye-tracking sensors integrated with the VR headset document what a participant is looking at with a peak frequency of 90 Hz. The eye movement monitoring system documents the latency of exhibiting vigilant orienting behaviors (response time in each exposure), and the frequency of vigilant orienting behaviors. The collected data from the VR training was preprocessed as follows:

In this study, we defined one exposure to the struck-by hazard as one reciprocal movement of the street sweeper traveling behind a participant. During the VR training, when a participant exhibited a vigilant orienting behavior (looking back and checking the proximity of the approaching vehicle for the first time in each exposure), the latency of response time (i.e., the delay between the beginning of the alarm sounds and participants’ vigilant orienting behaviors) was documented.

The frequency of vigilant orienting behaviors (checking rate, CR) was defined using Equation 1:

$${CR}_i=\frac{{}_{Numberofcheckingcycles}}{Numberofexposures}$$ (Equation 1)

where number of checking cycles = the number of cycles that a participant succeeded in checking the approaching street sweeper by a participant i; and number of exposures = the number of exposures to the struck-by hazard of a participant i.

Intervention (A single memorable experience through a virtual accident experience)

The VR environment includes a system that simulates struck-by accidents with construction vehicles upon a participant’s habituated ignorance of the approaching vehicles. To make the VR accident simulation a memorable event, it was dramatized by emphasizing aversive feedback to a participant. The VR accident simulation includes visual accident scenes, crash sounds, and haptic feedback via the VR motion controllers. In the first VR session, the accident with the street sweeper is triggered by a participant’s habituated ignorance of approaching construction vehicles. To trigger the VR accident upon a participant’s habituation, a behavior-checking system with a moving window was adopted. The moving window counted the number of a participant’s successes in checking the approaching vehicles over the five most recent exposures. When a participant fails to check on the approaching sweeper in three out of these five exposures, the street sweeper starts to move forward toward the participant until it virtually collides with the participant. If a participant recognizes the street sweeper’s erratic movement and succeeds in evading the collision, the street sweeper makes the normal reciprocal movement, and the behavior checking system also restarts to count the participant’s vigilant orienting behaviors. To provide a participant enough time to be aware of surrounding hazards while performing the assigned VR road cleaning task, regardless of participants’ continuous ignorance of the approaching vehicles, the VR accident was not triggered until at least 10 exposures to the struck-by hazard. Although a failure to visually check on three of the most recent five exposures serves as a threshold for the VR accident simulation that is to some degree arbitrary, a participant’s frequent inattention to the repeatedly approaching construction vehicle indicates a proxy for habituation. The VR accident with the sweeper was only triggered in the first VR session.

In the second VR session, a participant was also required to perform the same road cleaning task and was exposed to the same types of struck-by hazards at the first VR session (the risk associated with the street sweeper behind a participant). The latency and frequency of participants’ vigilant orienting behaviors were measured in the same manner as in the first VR session. To observe the sustained intervention effect of experiencing the accident on participants' behaviors, the VR accident with the sweeper was not triggered in the second VR session. However, to make the VR environment more close-to real, another VR accident simulation was designed and embedded in the second VR session. During the VR training, dump trucks repeatedly passed a participant in the next lane of the working lane where a participant was performing the road cleaning task. The accident with the dump truck was triggered at about 20 min after the start of the experiment. Once a participant reached an invisible accident trigger point, one of the dump trucks changed direction and backed up to the lane where the participant was working. The VR accident with the dump truck was also avoidable if a participant recognized the erratic movement of the truck and dodged the collision with the dump truck that was heading toward the participant. Again, the designed and presented struck-by hazards in the VR environment involved auditory warning alarms when they were heading toward a participant. This dump truck interaction was not of interest in the present study and was intended to pilot procedures for a potential future study.

EEG experiment procedure

Auditory stimuli

The two auditory stimuli (alarm and control) were made using Logic Pro X software on a 2017 MacBook Pro (Apple Inc., Cupertino, CA, USA). The original alarm sound—a truck Backing Up Beep sound—was extracted from video files.⁷¹ For creating the control sound, the alarm sound was modified using a sound equalizer. The sound distribution was equated to have the same magnitude over the spectrum of sound (from 39 Hz to 14,200 Hz), which generates white noise ambient. The two sounds lasted 600 ms each and were equated using the normalize function to set the loudness to 23 LUFF.

Procedure

The Oddball paradigm has been widely employed to assess auditory stimulus discrimination.^72,73 This paradigm usually consists of the frequent presentation of one stimulus designated the “standard” interspersed with rare occurrences of one different stimulus designated the “deviant”.⁷⁴ A common alternative is an equiprobable paradigm, in which each auditory stimulus is presented an equal number of times in a random order.^37,47,75 We opted for the latter paradigm in the present study to efficiently control the habituation of sounds. Note that the equiprobable paradigm is often used as a passive task, which requires no behavioral response.³⁸

Twenty images of construction sites were used as backgrounds in the EEG sessions. Half of the images were presented in the first session and the second half in the second session. In each session, each of the 10 images was presented 10 times in a random order with the restriction that the same image could not be presented two times in a row. We refer to sequence the set of events occurring during the presentation of one image. Each sequence included four sounds (of 600 ms) separated by an interstimulus interval of 3500 ms, 4500 ms, or 5500 ms. The first sound of a sequence was presented at 1750 ms or 2750 ms after the beginning of the sequence to make sure that the EEG signal related to the processing of the sound was not (or minimally) affected by the processing of the image. The last sound of a sequence terminated 1750 ms or 2500 ms before the end of the sequence. To keep constant the duration of each sequence, the five intervals that preceded and followed the sounds always lasted 18 s in total. All the possible combinations of intervals that met this requirement were presented, in a random order. Thus, each sequence lasted 20.4 s (Figure 6). In each session, 200 alarm sounds and 200 control sounds were presented. In two consecutive sequences, four-alarm sounds and four control sounds were presented in a random order so that the EEG session never included more than eight similar sounds in a row. Participants had a self-paced break after 50 sequences in each session.[Insert Figure 6 here]

Figure 6.

Example of a sequence of events in the EEG sessions

(A) The example of EEG experiment.

(B) A sequence included four sounds (of 600 ms) separated by an interstimulus interval of 3,500 ms, 4,500 ms, or 5,500 ms. The first sound of a sequence was presented 1,750 ms or 2,750 ms after a background image appeared (which corresponded to the beginning of a sequence) to make sure that the EEG signal related to the processing of the sound was not (or minimally) affected by the processing of the image. The fourth sound terminated 1,750 ms or 2,500 ms before the end of the sequence. To keep constant the duration of each sequence, the five intervals that preceded and followed the sounds always lasted 18 s in total. All the possible combinations of intervals that met this requirement were presented, in a random order. Thus, each sequence lasted 20.4 s.

Quantification and statistical analysis

Behavioral data analyses

Habituated behavior analyses

In this study, we propose that, in the VR environment, workers’ habituated behaviors to repeatedly exposed workplace hazards can be observed. We tested this notion through the following steps: The bivariate linear regression models predicting response time from number of exposures to the hazards were tested using the following equation:

$$\widehat{y_i}=B_0+B_{1}N+r$$ (Equation 2)

where $\widehat{y_i}$ is response time at number of exposures N; $B_0$ is the intercept of the regression line at N = 0; and is the slope of the regression that indicates the change in response time for each increase in number of exposures N. If the test result of the coefficient $B_1$ is significantly positive, the development of participants’ risk habituation can be determined. To avoid data manipulation, if a participant did not check the proximity of the vehicle until the vehicle reached the minimum distance where it starts to back up, that exposure was not included in the response time analysis. Additionally, the Wilcoxon Signed-Ranks test was performed to examine the difference in checking rate between AG and NAG at α = 0.05.

Intervention effect analyses

In this study, we propose the VR accident simulation as a one-shot learning platform that curbs workers’ risk habituation, and our theory is that workers’ habituated behaviors decrease after experiencing a VR accident. We tested this notion using (1) multiple regression analysis estimating response time at number of exposures and VR accident experience in the first VR session, and (2) a paired-samples t-test evaluating the intervention effect on the increase in checking rate for both groups (NAG and AG). Multiple regression analyses were performed to evaluate whether and how a participant’s experience of VR-simulated accidents in the first session affected the latency and frequency of participants’ vigilant orienting behaviors in the second VR session. A participant’s experience of VR-simulated accidents in the first VR session was coded as a categorical variable (dummy-coded as 0 for the no accident group [NAG] and 1 for the accident group [AG]) in the following regression equation:

$$\widehat{y_i}=B_0+B_{1}N+B_{2}A+B_{3}NA+r$$ (Equation 3)

where $\widehat{y_i}$ is the dependent variable (response time) at number of exposures and accident experience A; $B_0$ is the simple intercept of the regression line in the no accident group (A = 0, NAG); $B_1$ is the change in the simple intercept for each increase in number of exposures N; $B_2$ is the difference in simple intercepts, comparing the accident group (A = 1, AG) with NAG; and is the difference in simple slopes, comparing AG with NAG.

In addition, paired-samples t-tests were performed to investigate the intervention effect of experiencing the VR accident in the first VR session on checking rate in the second VR session (pre/post-treatment analysis). Results were presented as the mean checking rate. Using the Cohen’s effect sizes (d) test the intervention effect was evaluated, with the following criteria: 0.2 = small effect, 0.5 = moderate effect, and 0.8 = large effect.⁷⁶ During the first VR session ten participants never exhibited vigilant orienting behaviors. Thus, the data collected from such participants were excluded from the analysis of response time; these data were only used for the analysis of checking rate.

Event-related potential analyses

Python 3.8.2 and EEGLAB v2021.1 were used to do the data preprocessing and analysis.^77,78 First, we used Python SciPy toolbox and applied a forward-backward (non-causal) high-pass filter with Kaiser window, transition band of 0.5–1 Hz, and attenuation of 80 dB to remove the drifts from the signal.^79,80 Second, we used MNE-Python v0.24 package and applied a bandpass filter with 0.5 Hz and 40 Hz cutoff frequencies.⁸¹ Third, we changed the reference of the signals by removing the average of all the 15 channels from the signals.^82,83 Fourth, in order to remove the non-stationary artifacts (e.g., motor artifacts) from the signal, we implanted the Artifact Subspace Reconstruction (ASR) method in Python (our code is available at: https://github.com/moeinrazavi/EEG-ASR-Python).⁸⁴ Fifth, the EEG was segmented relative to the onset of the presentation of each sound stimulus (alarm/control) onset to create stimulus-locked epochs of 1800 ms that included a 300 ms pre-stimulus period. From each epoch, we subtracted the average of signal from −300 ms to −100 ms as the epoch baseline.⁸⁵ Sixth, in order to remove the stationary and non-brain signal artifacts (e.g., eye blink artifacts), we used Independent Component Analysis (ICA) toolbox in EEGLAB with MATLAB R2021a.^86,87 Finally, we visually removed the significantly noisy epochs from the data that were not corrected by the mentioned preprocessing steps.

One of the most prominent auditory ERP components observed in oddball and equiprobable paradigms is N1,^88,89,90 which peaks about 100 ms after stimulus onset and lasts for approximately 100 ms.^48,49 N1 is distributed mostly over the fronto-central region of the scalp (i.e., at the Cz and Fz electrode sites⁸⁹). The auditory N1 peak is linked to early attention.^34,52 Furthermore, an attenuation of the N1 response was reported to repeated auditory stimulus presentations and would reflect neural habituation in sensory cortex.^39,50

Pan et al.⁴¹ reported a P3 habituation (i.e., a decrease of the P3 amplitude) from auditory single-stimulus and oddball paradigms.⁵⁵ Auditory P3 is generally distributed over three midline sites (Cz, Fz, and Pz)^41,55 and peaks within a 220-420-ms time window after stimulus onset.^39,41

Given the specificity of sounds used in the present study (long, non-monofrequency), we referred to the ERP components analyzed by the time-window rather than a specific label (see discussion). Indeed, auditory stimuli used in EEG studies are usually short and monofrequency (e.g., 60 ms³⁹; 50 ms⁵⁰; 70 ms⁴¹). The alarm sound we used in the EEG sessions was extracted from the alarm signal used in real construction sites to increase the ecological validity of our test. This sound lasted 600 ms and included one beep with a rise and a fall. As a consequence, the identification of ERP components observed in our study requires caution.

Two components, which could correspond to N1 and P3, were measured at the time window of 540-660 ms and 804-980 ms post-stimulus onset, respectively. We also performed a post-hoc analysis at the time window of 220–420 ms post-stimulus onset, based on the apparent signal difference observed between the alarm and control sounds during this period. For these three time windows, we computed the mean amplitude separately for alarm and control sounds at the Cz and Fz electrode sites, where the deflection was maximal.

A 2 × 2 repeated-measures analyses of variance (ANOVA) was conducted on mean ERP amplitudes with type of sound (alarm, control) and session (1, 2) as within-subject variables for each time window. Subsequent t tests were performed when appropriate.

Published: January 20, 2023

Data and code availability

• •Data: Experimental data is available from the lead contact on reasonable request.
• •Code: This paper does not report the original code.
• •Additional Information: Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.