Commonly Used Assessment Method to Evaluate Mental Workload for Multiple Driving Distractions: A Systematic Review

Abstract

Background:

We aimed to find the commonly used assessments to evaluate driver’s mental workload and its relationship with driving distraction.

Methods:

Academic articles such as journals, books, reports and conference papers that are related to workload measurements methods used in identifying mental workload among drivers that are dated from Jan 2015 to Apr 2020 were used in this paper. Then, PRISMA checklist and flow diagram are being applied.

Results:

The few commonly used assessments in evaluating mental workload among drivers are Karolinska Sleepiness Scale (KSS), NASA TLX, Electroencephalogram (EEG), Heart Rate (HR), eye tracking and driving performance. Moreover, different types of driving distractions show to affect the driver’s mental workload in one way or another when being evaluated using these assessments.

Conclusion:

The finding of this study can be used to find the gap for future research in vehicle safety by using multimodal monitoring of different types of assessments to increase the validity and robustness in driving assessment.

Introduction

Driving task requires physical demands from the driver to operate the car controls while sitting on the car seat with constrained in the driving posture and car space. This poor condition in longer run might lead to fatigue where it will further lead to road accidents. Most factors that lead to fatigue are account for 40% of all accidents (1). Road accident is among the top 10 global causes of death since year 2000 where it is placed at number 10, however recently in 2016, it moves up to number 8 according to The WHO (2). The increase of road accidents has risen awareness among the road safety and transportation agency to determine factors that can affect driving performance. Thus, the factors that can cause mental workload among the drivers needs to be investigated further to prevent and reduce the number of road accident.

Mental workload occurred among the drivers will cause drowsiness when it is under load or stress when it is overloaded. Underload can lead to reduction in alertness and attention while overload can cause distraction and draw away the drivers’ attention, which can shorten the time for the driver to process information before reacting (3). Furthermore, these distractions will later lead to reduction in alertness, attention and vigilance among the drivers. Then, it will soon weaken the driving performance and vehicle control due to long demand and the use of mental resources (4). Mental workload among the drivers occurs mainly due to distractions, which involves deviation of the driver’s attention from driving because of other activities unrelated to driving. These activities will further reduce the awareness and disrupt the driver’s decision-making and performance that will increase the potential near-accident or accidents that are as equal as accident caused by consuming drugs or alcohol (5).

Driving task could disturb the driver’s condition, where this can be seen through signs of discomfort shown by the driver and the driving performance as well as the drivers’ alertness level (3, 6–9). Furthermore, by evaluating the mental workload, one can suggest the level or amount of cognitive demands placed on the driver (3). Human error related to mental workload that occurred while driving due to poor data handling is among the major causes of common traffic accidents (3). Thus, we aimed to provide the reader with a brief overview of the frequently used different subjective and physiological assessment types to measure mental workload for multiple driver distractions.

Methods

Study parameter

Road accidents and mental workload are highly related where increase in mental workload among the drivers are mainly due to driving distractions. These driving distractions resulted in reduction of driving performance and efficiency, which later might cause accident or even death. Workload measurements that have been broadly used in the literature are subjective workload and physiological workload that includes driving performance (10). Most literature combined these different measurements to investigate drivers’ mental workload as shown in Table 1.

Table 1:

Compilation of pass studies on different types of workload assessment

Authors	Experiment		Type of Distraction			Methods							Findings
	Subject	Simulator/Actual	Road Related	Driver Related	Vehicle Related	NASA TLX	KSS	EEG	HR	Eye Tracker	Driving Performance	Others
Shakouri et al. (11)	30	S	Road work			x		x			x		Workload is higher with higher traffic density.
Belyusar et al. (12)	123	A	Digital billboard							x			Changes in the amount and duration of glances towards billboards.
Ahlstrom et al. (13)	30	S	Rural, Suburban, Traffic				x	x		x			Distractions causes increase of EEG alpha rhythms and longer blink.
Kountouriotis & Merat (14)	15	S	Urban, Rural, Road geometry, Lead car							x	x		Different road geometry effects drivers differently.
Chen et al. (15)	15	S	In city, Monotonous				x	x					Configuration of the functional brain network is related to driver drowsiness.
Faure et al. (16)	24	S	Urban, Rural, Road environment	Secondary task						x			Secondary task increases the blink rate.
Farahmand & Boroujerdian (17)	17	S	Road geometry								x		Complex road geometry reduces fatigue among drivers.
Oviedo-Trespalacios et al. (18)	32	S	Road environment. Road Geometry								x		Road environment had impact towards driving performance.
Siam et al. (19)	30	S	Road geometry	Secondary task								x	Secondary task affected the driving performance.
Tarabay & Abou-Zei (20)	80	S		Secondary task					x		x		Secondary task increases heart rate, skin conductance level and cognitive load.
Perrier et al. (21)	24	A		Sleep deprived			x	x				x	Driving performance, fatigue and sleepiness fluctuations with ToT.
Ahn et al. (22)	11	S		Sleep deprived					x	x		x	Increase in alpha and decrease of beta which indicates fatigue.
Wen et al. (23)	20	S			Music listening		x			x	x		Different type of music affects driver’s mental workload differently.
Guo et al. (24)	20	S	Freeway					x					Reaction time among female, male and elderly are different.
Mohid et al. (25)	12	S	Monotonous				x	x	x				Physiological responses are different at beginning and end of driving session.
Sugiono et al. (26)	3	S	City, Rural, Motorway, High Traffic, Low Traffic					x					City road showed highest level of stress followed by rural and motorway.
Kim & Yang (27)	11	S		Secondary task	In-vehicle technology					x	x		Assessment value increases as drivers undergo mental workload.
Kim& Yang (28)	11	A		Secondary task	In-vehicle technology, Radio					x	x		Visual distraction increases driver’s mental workload.
He et al. (29)	37	S		Secondary task		x		x	x	x	x	x	Workload increase when driver undergoes a task.
Diaz-Piedra et al. (30)	11	S	Duration, monotonous			x		x			x	x	Nasal skin temperature can be used to measure driver’s mental workload.
Getzmann et al. (31)	32	S	Curvy roads		Noise			x			x		Driving distraction affects older and younger mental workload differently.
Foy & Chapman (32)	26	S	City, Suburban roads			x			x	x	x	x	Different road types affected the driver’s mental workload differently.
Sugiono et al. (33)	26	A	Urban, highway, rural roads			x							Highway causes least mental workload followed by rural and city road.
Sugiono et al. (34)	26	A	Rural, city and motorway			x			x				The assessments is suitable to monitor real time mental stress.
Strayer et al. (35)	38	S		Handphone and talking	Audiobook, Radio, e-mail	x		x		x	x		E-mail involved high level of cognitive workload.
Alrefaie et al. (36)	33	S	Overtaking cars	Secondary task	Email				x	x			Quality of takeovers can be evaluated using eye tracker and heart rate.
Prabhakar et al. (37)	12	S		Secondary task				x		x			The assessments used can detect cognitive load increment during secondary task.
Papantoniou et al. (38)	95	S	Rural, urban, low and high traffic	Handphone, Talking							x		Different distraction cause decrement in reaction time among drivers.
Niu et al. (39)	36	S		Hand phone							x		Phone distraction leads to visual, cognitive and motor resource functional limitation.
Nowosielski (40)	38	S	Road complexity		Audiobook						x	x	Environmental and individual affects the driving attention while listening to audiobooks.
Paxion et al. (41)	57	S	Road complexity			x						x	Complexity and lack of experience increased subjective workload.
Nurul et al. (42)	20	A	Low, mid and highly complex environment			x		x			x	x	Complexity had significant effect on mental workload.
Jeong & Liu (43)	24	S	Road complexity	Secondary task						x	x	x	Complexity had significant effect on mental workload.

Study Design

This brief review uses four databases, which topics related to neuroscience, ergonomics and transport (Scopus, Google Scholar, Web of Science, and Science Direct) were searched and this brief review includes publication from year Jan 2015 to Apr 2020. The search approach was established using mixtures of keywords such as mental workload, driving distraction, and driving fatigue, which includes academic articles such as books, reports and conference papers. Then, the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) checklist and flow diagram were used for this brief review where relevant articles are merged and then the duplicate articles were removed as shown in Fig. 1 (10). The impact assessment was defined after reading the full article concerning the following questions:

1. What are the types of driving distractions that can influence the mental workload of a driver?

2. What are the commonly used physiological and subjective measurement methods in identifying a driver’s mental workload?

3. How do these methods indicate the relationship between mental workload and driving distractions?

Fig. 1:

PRISMA flow chart

Results and Discussion

Overall, 927 articles were identified using the keywords above from four databases. Duplicates and irrelevant articles were then excluded. Specifically, articles that are only related to different types of driving distractions plus types of different subjective and physiological assessment used in identifying mental workload among drivers that uses simulator or on real road are included where, finally, 33 articles were related to the paper’s objective and represented as in Table 1. Most articles in the table analyse either one type of driving distraction or combination of different types of driving distractions.

Types of Driving Distractions

Cognitive distractions such as decision making, dreaming, and mind wandering or problem solving can affect driving performance and lower visual information processing among drivers. Moreover, for the past 20 years, driving distraction is said to be one of the major causes of road accident. These driving distractions were being compared to be more or less similar to road accidents that are caused by consuming drugs or alcohol (5). According to Table 1, driving distraction can be divided into three categories that are driver-related distraction, vehicle-related distraction and road-related distraction where its examples are shown in Fig. 2.

Fig. 2:

Types of driving distraction

Driver-related Distraction

Driver-related distraction is considered as secondary task cognitive distraction where the drivers mind is said to be off the road and when driving is no longer the focus of the driver. Examples of these secondary task cognitive distractions are the usage of handphones such as making calls, receiving calls or texting, smoking, eating, or talking to another passenger in the car (5). This may cause unsafe following distance and slower information processing which might lead to rear-end crashes because the driver might fail to react in time. However, cognitive distraction improved lane-keeping where according to analysis on the gaze behavior, drivers tend to dedicate their attention towards the middle of the road when cognitive load increases (3).

Vehicle-related Distraction

Vehicle-related distractions are in-vehicle information systems such as the audio system and the navigation system. The simple act of turning on or adjusting the radio was found to be one of the main causes of distraction-related crashes (44–46). Furthermore, drivers nowadays tend to rely more on vehicle technology such as the navigation system where it is usually placed at the middle-front of the car and drivers will tend to take their eyes away from their main task that is driving and gazing occasionally onto the navigation system. Thus, this in-vehicle technology can be considered as one of the forms of visual distractions. Many studies have been done to investigate these secondary task impacts on driving performance and it is found that, when drivers are distracted, visual data processing and vehicle performance will start to reduce. Moreover, the psychological changes in drivers are more complicated due to the rapid new invention of in-vehicle technologies whose initial purpose is to aid the drivers but somehow turns to be one of the factors that cause distraction towards the drivers (5).

Road-related Distraction

Road-related distractions are the distraction outside the car and are related to road environment. The complex roadway environments can draw away the driver’s driving attention (12). These complex environments include billboards, road signs, buildings, monotonous, urban, city or rural, street parking and traffic will somehow demand a little bit of the driver’s attention visually (12,13,44,46). The act of looking out of the window for example will lead to ignoring the road in front and will affect the driver behaviors such as change in speed or change in lanes (12). Furthermore, at mental fatigue can increase in monotonous driving environment which means that the visual scene remains unchanged for a long time such as driving on highway compared to driving in a city or urban roadways which the visual scenes are livelier with more stimulating environment (13,47). This is because driving task requires consistent vigilance and the lack of visual, motor, or cognitive stimuli that can further change the ability to maintain alertness. Monotonous road environment or geometry has a psychological effect on the drivers in terms of level of vigilance either the level of vigilance increases or decreases (48,49).

Commonly Used Physiological and Subjective Measurements

Results from Table 1 shows that commonly used subjective assessments are NASA-TLX and Karolinska Sleeping Scale (KSS) while commonly used physiological assessments are Electroencephalogram (EEG), Heart Rate (HR) or Electrocardiogram (ECG or EKG), Eye tracker or Electrooculography (EOG) and driving performance where other subjective and physiological assessments that are not that commonly used are being categorized in Fig. 3 as “others” such as using other types of subjective assessments, skin conductance or skin temperature, number of traffic violations, respiration, Electromyography (EMG), facial movement and others. Further detail on these different types of commonly used assessments is explained in the next section.

Fig. 3:

Assessments used in identifying mental workload among drivers

Subjective Measure

Subjective measurements are provided directly by the drivers, which are usually given before the task begins and after the task is completed. They are straightforward but the data cannot be gathered in real time and the results are sometimes biased (11). Many different subjective measurements are being used to assess driver’s mental workload such as NASA TLX and Karolinska Sleepiness Scale.

NASA TLX

NASA Task Load Index (NASA TLX) is a questionnaire containing six sections (mental demand, physical demand, temporal demand, performance, effort, and frustration) to measure the subjective cognitive load. First, the participants had to do a task (similar to the primary experimental task), and after the completion, the participant are asked to make assessment and compare those six sections and select the one they experienced more when performing the sample task. These assessments are used to identify a weighting factor for each section. Then, after the task has been completed, participants are given another form to measure the intensity of each section. This is done by using a 12 cm visual-analog bipolar scale ranging from low to high. The workload is then defined by the result of the weighting factor of each section by its intensity (11).

Karolinska Sleeping Scale (KSS)

Karolinska Sleepiness Scale (KSS) is used to assess subjective sleepiness levels caused by sleep deprivation (50). Currently, KSS is regularly being used in studies particularly in measuring the drivers fatigue due to monotonous long drive such as driving on the highway (22). KSS is used to obtain the self-report sleepiness level of the driver during the driving where there are nine levels of alertness starting from extremely alert to very sleepy.

Physiological Measure

Many researchers chose physiological methods of measuring workload rather than using the subjective measures since they do not involve direct response from the participants compared to the subjective measures based on participant’s feelings, opinions and familiarity during the task. In physiological methods, the response of the body to exterior sources of workload is measured and used as markers of physical and mental workload (11,51).

Electroencephalogram (EEG)

The electroencephalogram (EEG) is the measurement of the electrical activity of the brain on the scalp surface and are logged using electrodes in the form of frequency bands (1). There are five different waves that can be analyzed using EEG, which are alpha, beta, theta, delta and gamma waves. The relation between reductions of human level of vigilance has been discovered to create significant readings in the on-going EEG power spectra readings (52).

Heart Rate (HR)

The heart rate (HR) is the number of heartbeats in a minute for a human. Variations of the heart rate can be associated with the difference in emotional conditions of humans. Thus, heart rate plays an important role in the assessment of the mental condition of a driver throughout their driving performance and it has been studied in many current and past work (1,53).

Eye Tracker

The measurement of the eyelid movement or eye blink can help in detecting the driver’s condition such as whether the driver is in a state of drowsiness or a state of vigilance. Slow eyelid closure or eye blink are greatly associated with high visual vigilance while faster eyelid closure or eye blink are associated with fatigue, increase in workload, or drowsiness among the drivers (3).

Driving Performance

Driving performance is the ability of the drivers to stay alert, perform and maintain safe driving under various road conditions and durations (20). Driving performance includes lateral driving performance measure, steering wheel measure, braking pattern, and line crossing where it is often used to detect the presence of mental workload among the drivers.

The Relationship between Mental Workload and Driving Distractions

NASA TLX

Many studies have used NASA TLX and show different types of distraction has contributed to the increment of mental workload among the drivers such as driving under high traffic density, driving with the presence of multiple crossroads, driving on complex geometry roadways, and driving while undergoing secondary task such as answering calls and texting (4,11, 46).

Karolinska Sleeping Scale (KSS)

The KSS reports are based on the drivers feeling while driving. Countless studies had been done to investigate the impact between driving in low stimulated road environment and a higher-stimulated road environment. The result of mental workload using KSS readings is generally high when driving on monotonous roadways compared to driving in city with multiple road environment distractions (4,13, 15).

Electroencephalogram (EEG)

A reduction in vigilance and decline in performance are related to increased EEG power spectra in theta band and a change in EEG alpha power. Additionally, an increase of EEG power spectra in the beta band are related to increase in vigilance and stimulation or higher mental workload while alpha waves arisen during relaxed situations at decreased attention levels or in drowsy condition, but awake state and theta waves mostly occurred during the sleeping state. A handful of studies exploring the fluctuations of EEG rhythms throughout increased or sustained task demands described that the most obvious event was the increase of the EEG power spectrum in the theta frequency band over the prefrontal cortex, located in a midline scalp position where most mental function takes place such as problem-solving, planning, judgment and behavioral control (1,3–5,15,46–48).

Heart Rate (HR)

The more difficult the task is, the higher will be the HR (1). Additionally, the HR variable associates to certain extent with the existence of mental workload. Specifically, increased HR could be related to an increased mental workload. Furthermore, in the transition between mental fatigue and drowsiness or prolonged simple driving, the HR variable appears to decrease (1,3,4,47). Consequently, different driving distractions can lead to fluctuation of the HR among the driver depending on the workload threshold of the drivers.

Eye Tracker

The eye trackers are commonly being used to study the different road environment complexity effect towards the driver mainly complexity such as monotonous versus the highway road, urban versus rural road, long hours’ drive versus short hours’ drive, night drive versus day drive and high traffic density versus low traffic density. Highly stimulated environment causes slower movement of the eyelid while long drives on a low stimulated environment cause faster eyelid closure which indicates drowsiness (3,12,14,16,46,48,50,54,55).

Driving Performance

Mental workload increases were indicated by increase in the movement of the steering wheel or line crossing and increase for brake applied mostly due to secondary tasks. The driving performance is said to decrease associated with monotonous road conditions, long duration of driving, low traffic, and different time of the day such as after lunch or early morning (11,21,55–60). Furthermore, mental workload among the drivers emerged faster when driving on a low demanding road environment where it is said that it has an impact on driving performance and a high stimulating road environment can prevent fatigue or drowsiness from occurring (11).

Conclusion

Mental workload is considered as one of the main factors that contribute to the increment number of road accident yearly and factors that contribute to driving distraction needs to be investigated more such as the study of different types of driving distraction that includes road-related, driver-related and vehicle-related. There are plenty of studies done on driving distractions but there is lack of studies that focuses on multiple distractions at the same time. The combined usage of the physiological and subjective measurements are important since physiological measurements will remain unchanged even if the subjective measurement results are biased due to the current emotional state of the driver. Presented in this paper are variety of assessments measures that are commonly used in determining driver’s mental workload whereas multiple assessments are usually used to increase the validity and robustness in assessing driver’s mental workload. Furthermore, this paper also includes the relationship between mental workload and driving distractions, which can be seen, on the highs and lows of the assessment reading resulting from different driving distractions that the correspondents had to undergo. The assessment of mental workload could be helpful in the road safety field and future research needs to start to address the idea on how to detect mental workload in real-time for example, creating an in-vehicle system that can detect the drivers’ mental workload or fatigue while driving by alerting these conditions to the driver. For this to happen, an in-depth study on the human-machine interaction that is based on human mental workload needs to be explored widely in hopes it can increase the validity and robustness in driver’s assessment.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.