Visual and cognitive distraction metrics in the age of the smart phone: A basic review

Abstract

Sources of distraction are numerous and varied, and defining and measuring distraction and attention is complicated. The driving task requires constant adjustments and reallocation of attention to cognitive, motor, and visual processes. While it is fairly straightforward to measure distraction in an experimental situation (e.g., simulator, closed course), driver distraction in the real world is highly contextual. While no single metric is capable of capturing the complexities of distraction, several have proved useful in helping researchers gain fuller understanding of it. Few have reached a level of consensus among researchers and user interface designers. ISO and SAE may be considered the ‘gold standard’ for providing mechanisms through which open scientific consensus-based standards can be achieved.

While there are a number of metrics used in predicting distraction, three have been studied closely and are going through the SAE and ISO standards process. They are (1) ‘the occlusion method’; (2) the Lane Change Test (LCT); and (3) the Detection Response Task (DRT). The metrics described here apply generally to the experimental context where driving is tightly controlled. Like any method, there are limitations with each—and they don’t necessarily agree with one another.

Experimental methods and analyses are different than those in naturalistic driving (ND). ND relies more on data mining versus traditional experimental manipulation. ND data are a challenge precisely in that they lack experimental control.

In future, driver metrics will go beyond specific measurement of task load, and will include how drivers self regulate when they choose to be distracted.

INTRODUCTION

The scourge of driver distraction has been around since the invention of the automobile. Sources of distraction are numerous and varied. The bottom line, though, is that whether the source of distraction is an object or incident, whether it is inside or outside of the vehicle, whether it draws the driver’s eye away from the road or his/her thoughts inward—all distractions have the potential to result in crashes or near crashes.

Defining and measuring distraction and attention is complicated. The driving task requires constant adjustments and reallocation of attention to cognitive, motor, and visual processes. While it is fairly straightforward to measure distraction in an experimental setting (e.g., simulator, closed course), driver distraction in the real world is highly contextual. Each conflict, each crash is unique. Each represents the co-incidence of a myriad of factors.

These include (but are not limited to) such things as drivers’ eyes or mind off the road, traffic density, lighting, speed, sleepiness, mood, type of road, weather, and a host of other factors that must interact precisely for a conflict or crash to occur.

Meanwhile, the past 10 years have witnessed an explosion of technological opportunities for distraction. The propagation and increasing complexity of smartphones and in-vehicle information displays is vast and seemingly endless. New apps and forms of social media are constantly expanding on phones and multifunction screens in vehicles. With novelty comes increased attentional demand. There is no going back, however; such devices are, for better or worse, now an inevitable part of the driving experience.

The challenge thus becomes how to help drivers manage information from these sources while they navigate the distractions that have always been a part of driving. One way is to alert drivers when they are about to crash. Among high-end vehicles, almost all auto manufacturers now have collision warning and avoidance systems on the market. These are designed to help mitigate crashes when a driver has a lapse in attention.

Driver distraction can be very broadly divided into two general categories: visual demand and cognitive demand. Each has a number of sub-elements that can be used to further define how drivers are distracted. Researchers and designers have a range of tools to help them identify when a driver’s attention reaches the threshold that signals a measurable decline in performance.

It is first necessary to have a solid, operational definition of distraction. One well-regarded, contemporary definition identifies distraction as “a diversion of attention away from activities critical for safe driving toward a competing activity” (Regan, Lee and Young, 2009). While this definition is fairly straightforward, it includes ambiguities that can be difficult to reconcile. For instance, drivers may experience increased attentional demand when navigating while lost. Yet, this itself may not divert the driver’s attention until another task, like a cell phone operation or a crying baby, tips the demand over into true diversion of attention and causes an error. The key question is whether activities measurably divert attention away from safety-critical driving activities, and what levels of load lead to measureable errors. While most methods and metrics cover diversions of attention, research is now evolving to examine how task load affects attention itself during driving—something like that is a much greater challenge.

There are two general categories for the types of task that can lead to attentional diversion. These are visual-manual distraction and auditory-vocal distraction (which is generally associated with cognitive load). Of these, visual task loading is the easier to measure. Examining the position of the driver’s eyes relative to the road is relatively straightforward and easily accomplished with eye-tracking equipment or even frame-by-frame video analyses. The most basic metric is head up and looking out the forward view vs. down at a device or inside the vehicle. While lighting and movement can sometimes be a challenge in naturalistic driving, such head pose changes are largely salient.

Assessing cognitive load is a bigger challenge, as it is difficult to understand precisely what a driver may be thinking. Identifying what is and is not a cognitive load is also challenging, especially when attempting to determine whether it truly diverts attention.

When task loading exceeds drivers’ available resources, or in some way disrupts their ability to attend to driving, degradation of driving may result. The result of attentional diversion away from safety-critical driving activities may be reflected in a driver’s neglect of lane position, speed, headway, or a general loss of situation awareness. We describe a number of basic mechanisms currently used to capture this degradation, along with some methods and metrics used to measure attention demand.

DISTRACTION METHODS AND METRICS

Researchers have long sought for a simple set of measures able to both detect and predict driver distraction. This is because performing measurements in instrumented vehicles on the road is resource-intensive (cost, labor, and time). While no single metric is capable of capturing the complexities of distraction, several have proved useful in helping researchers gain fuller understanding of it. ISO and SAE are considered the ‘gold standard’ for providing mechanisms through which open scientific consensus-based standards can be achieved. Both put out consensus-based information reports, guidelines and standards arrived at through a long and rigorous process. Only a few metrics have reached this level of standardization. We describe several of them in the following sections.

Visual sampling limits and metrics

Visual sampling refers to looking at something (a screen, the road) to gather information. As mentioned previously, the eyes play a key role in sampling information from the roadway, as well as from in-vehicle information systems and carry-in devices. A couple of studies outline the limits of visual sampling while driving.

Senders, Kristofferson, Levison, et al. (1967) were the first to quantify the amount of visual sampling required to drive a passenger vehicle. In an experiment that used a helmet with a visor to periodically occlude the forward view (Figure 1) they found:

• A driver can easily maintain vehicle control with surprisingly small periodic samples of the roadway.

• Between samples, the driver becomes increasingly uncertain about the state of the vehicle relative to the roadway.

• When the uncertainty exceeds a threshold, the driver samples the forward view.

• Drivers’ are generally well calibrated regarding the buildup of uncertainty relative to the vehicle dynamics and roadway characteristics.

Figure 1.

First experiment to study visual occlusion

In the Senders study, drivers pressed a foot control to operate the visor. As uncertainty built, the driver could open the mask for a quick gulp of the scene. For an entertaining and informative glimpse of this study, see https://www.youtube.com/watch?v=kOguslSPpqo

Wierwille (1993) later complemented this research by describing a sampling model that begins when the driver initiates an in-vehicle task by glancing at a location of interest. Time elapses as the desired information is extracted. Wierwille found that if drivers could “chunk” information (gather what they needed) in one second or less, they would return their glance to the forward roadway. However, if the chunking took longer, drivers continued to glance at the device or object of interest. Uncertainty increased the longer the drivers’ eyes remained off the road, and they quickly felt pressured to return their eyes to the forward scene. If the glance to the in-vehicle location exceeded approximately 1.5 seconds and the information could not be obtained (or chunked), drivers returned their eyes to the forward scene and tried again later. Additional samples were handled the same way, until all required information was obtained (see Figure 2). Subsequent research has indicated that drivers may be able to look away for up to two seconds before a breakdown in lane position or speed occurs. However, the basic principle is that for safe driving, the majority of visual resources (>70%) must be devoted to the forward roadway (Antin, Dingus, Hulse, and Wierwille, 1990).

Figure 2.

Wierwille’s (1993) model of visual sampling for in-vehicle tasks

Current eye-tracking systems make measuring attentional demand using eye-position analyses fairly straightforward. Such systems indicate where a driver is looking, measure glance duration, and track the number of glances to areas of interest inside and outside of the vehicle. Again, the general rule is that drivers cannot look away from the roadway for more than two seconds; frequent glances within a short period of time can also be problematic (Zwahlen, 1988).

Predicting the point at which a driver will be distracted can be more complicated. There are a few methods to better help predict visual distraction. One of these, formally called ‘the occlusion method,’ is accomplished by blocking the participant’s view of the relevant driving scene for short periods of time (e.g., 1.5s) using LCD shutter occlusion goggles (Figure 3). The goggles are then opened for short periods of time (1.5s) to allow the driver to view a secondary task (e.g., an in-vehicle display). The intent is to simulate the process of glancing back and forth between the road and an in-vehicle display. This method is particularly useful when examining display complexity in the context of design (Pettitt, et al., 2010).

Figure 3.

Visual occl. glasses (Pettitt et al., 2010)

The occlusion method has been used in the ongoing effort to establish both the SAE (2003) and ISO (2007) working standards, after it was employed in establishing earlier industry-based standards in Japan¹ and the United States². The primary dependent measure for the occlusion method is Total Shutter Open Time (TSOT). It is calculated as the sum of all times during which the goggle shutters are open from the start of a task until that task is completed. A higher TSOT means a task requires longer periods of visual attention to extract the information. Other measures that are part of the occlusion technique include:

• Total Glance Time (TGT) - time to move glance to/from distractor + duration while on distractor.

• Total Task Time Occluded (TTT_Occl) – measurement taken with the occlusion goggles on, and measured as the duration to complete task of interest, including occluded & unoccluded intervals.

• Total Task Time Unoccluded (TTT_Unoccl) – measurement taken without any occlusion goggles on, and measured as the duration to complete task of interest without visual occlusion procedure & without concurrent task.

• Resumability Ratio (R) - TSOT / TTT_Unoccl, ratio of TSOT (under occlusion) to Total Task Time (unoccluded – no goggles on); in other words, this represents the total time spent looking at the task (when the shutters were open) divided by the “static” task completion time.

• Vision Interval - duration when interface is visible.

• Viewing time - duration when glance is not focused on primary task (driving).

Studies have shown that TSOT is correlated with several metrics: lane keeping, total glance time to a task, some speed-keeping metrics, and total task time (Angell et al., 2006).

Lane Change Test (LCT)

Another useful method for predicting visual attention demand is the Lane Change Test (LCT) (Mattes, 2003; Mattes and Hallen, 2008; Young, Lenne and Williamson, 2011). The LCT is for use in a laboratory setting. It assesses secondary task performance (e.g., using a GPS while driving) and the effect on driving performance relative to psychomotor control.

Participants taking the LCT are instructed to stay in their current lane while driving at a constant speed of 60 km/h (the speed is fixed). At certain points along the drive, signs are introduced and become legible at a certain distance. These signs indicate that the driver should change lanes as quickly and as accurately as possible (Figure 4) (Huemer and Vollrath, 2012; Petzoldt, Bär, Ihle, Krems, 2011). The traffic signs indicate the direction (left or right) and width (one or two lanes) of the lane change. The distance between the signs averages about 150 meters. The symbols appear on the signs at a distance of 40 meters. One trial consists of eighteen lane changes in random order, and takes about 3 minutes. While performing the task, participants are also often given secondary tasks to complete (e.g. tuning a radio, operating an in-vehicle device).

Figure 4.

LCT conceptual and actual view

LCT was designed to evaluate in-vehicle systems, and uses a standard PC gaming steering wheel and pedals. It is usually paired with a normal desktop screen or simulator buck. LCT experimental details are described in the draft of ISO 26022. The reliability of LCT results has been shown to equal that of both high-fidelity driving simulators and on-road systems (Breuer et al., 2003).

The primary performance metric or measure of the LCT is the mean path deviation relative to a reference path through the lane-change course. In other words, the driver’s trajectory through the lane changes is compared to a normative model, the “perfect” pathway along the track, which is identical for each participant (Figure 5; Pitts, et al., 2012). Deviations from this normative path are calculated using the mean deviation (MDEV) for each drive. MDEV values for the dual-task conditions are compared to those from a baseline measure, where the participant is engaged in the primary driving task only. Generally, the more attentionally demanding the task, the greater the MDEV. Although few published studies have been done to compare or validate the MDEV metric to driving performance data, some exploratory work has shown that MDEV correlates with metrics of event detection (Angell, 2010), like the modified Sternberg.

Figure 5.

Lane change task comparison between normative path and actual path during secondary task engagement.

It should be noted that detection of events during the time when secondary tasks are underway requires attentional shifting; it involves both orienting attention and, in some instances, executive attentional functions. In the Collision Avoidance Metrics Program Driver Workload Metrics study, Angell (2006) examined correlates of the LCT to other surrogate measures like the PDT and modified Sternberg test. She found that LCT MDEV was significantly correlated with the Modified Sternberg Test (proportion missed) (r=0.92) and the PDT (in simulator, percent missed) (r=0.92). It was not found to correlate with Visual Occlusion Total Shutter Open Time (TSOT) (r=0.62), however.

Assessing Cognitive Load

Assessing cognitive load is a complicated task—and one that is often difficult to operationally define and measure (Young, 2012). While there are mixed data on whether cognitive distraction is a central issue in attention research (the link between cognitive load and crash risk has not yet been empirically established), such distraction remains a focus of a great deal of research. Several metrics show promise in the measurement of cognitive load and its effects on attentional control and functions. In tightly controlled simulator and on-road studies, the Detection Response Task (DRT) has been recognized by the international community as holding particular promise, and is the subject of an ISO working group (cf. NWI ISO 17488). The DRT involves detecting a series of peripheral stimuli, usually visual or tactile in nature. The method can also be used in analyses of visual demand.

One reason why the DRT method is so attractive is that it is fairly simple. It utilizes a detection-response task where a driver must respond to frequent artificial stimuli presented with some temporal uncertainty. The driver is asked to respond to a peripheral LED or a tactor by pressing a button attached to the index finger against the steering wheel (Figure 6). The primary dependent measures are response time in msec and hit rate. If a driver has increased attentional load, both hit rate and response time will be slower.

Figure 6.

Finger-mounted buttons used to record reaction time (adapted from an early draft for NWI ISO 17488)

The DRT has several different modalities that can be used in on-road, simulation or even non-driving studies:

• Head mounted visual stimulus (HDRT)

• Remote visual stimulus (RDRT)

• Tactile (TDRT)

Each uses the same stimulus frequency and timing. Figures 7 and 8 illustrate the key principles taken from the draft ISO 17488. The parameters are stimulus onset (S_on) and stimulus offset (S_off), basically when the stimulus is turned on and off. Stimulus duration (SD) represents the length of time the stimulus remains on, and maximum stimulus duration (SD_max) represents a pre-set maximum of this measure. According to the pre-ballot draft standard, SD_max should be set to 1 second. The stimulus cycle period (SCP) represents the time from the onset of one stimulus until the onset of the next stimulus. The standard states that the stimulus cycle period should vary randomly from a uniform distribution of values between 3–5 seconds.

Figure 7.

Defintion of parameters relevant for stimulus presentation specification (Pre-Ballot Draft ISO 17488)

Figure 8.

Example of how stimulus duration is determined by responce (R) (Pre-Ballot Draft ISO 17488).

A signal generated by the participant pressing the response button is referred to as a response (R). If the participant responds while the stimulus is on, the response will immediately turn it off (Figure 8).

DRT—Head mounted stimulus

The first DRT method involves the peripheral stimulus being mounted with an apparatus on the participant’s head (HDRT) (Figure 9). As may be seen in the figure, the peripheral LED is attached to the head on a stalk. The advantage of this method is that it makes the stimulus dependable; it will appear in the same area of the participant’s visual field regardless of where the participant is looking. Generally (and in Figure 9), the LED is placed 20 degrees to the left along the horizontal meridian and 10 degrees above the central meridian with the left eye as the reference point.

Figure 9.

HDRT configuration

Peripheral detection—remote visual stimulus

The second DRT method for examining peripheral detection uses a remote visual stimulus in the form of a single, 5-mm diameter red LED (Figure 10). The LED is placed remotely from the driver, but still in the central field of view. Placement is generally on top of the dashboard above the steering wheel. This remote stimulus must be directly viewable by the driver and not reflected off of the windscreen. For on-road testing, the light should be shielded so that it remains salient (Draft ISO 17488).

Figure 10.

Remote visual stimulus—peripheral detection

Tactile detection

The third DRT method uses tactile stimuli and is known as the Tactile DRT (TDRT). A small electrical vibrator called a tactor is placed on the driver’s front left trapezius muscle (Figure 11). As with the visual stimuli, it is generally activated while the participant is being asked to perform some kind of secondary task. The same hit/miss rate and reaction time variables apply to the TDRT as to the other measures. Like the HDRT, the TDRT is useful in that the driver does not have to be looking to the forward roadway to detect the stimuli.

Figure 11.

Tactor location for TDRT (Pre-Ballot Draft ISO 17488)

CONCLUSIONS

We live in a distracting world. The discussion about the effects of electronic distractions on our minds, relationships, parenting, children’s development, and quality of life is vigorous and ongoing in the public sphere. It is only natural that the conversation would also include driving, a significant daily activity in the lives of most people. Adding to this is the most ubiquitous appliance of this century—the smartphone.

Methods such as those described offer some measure of how much demand a task places on a driver’s resources—or how much a driver’s attention is drawn away from the driving task. As we have shown, in some ways, distraction metrics have come a long way from the days of Professor Senders and his visor helmet. In other ways, the fundamental issues remain—how can we, as automotive safety researchers, better understand the complex interaction between driver, vehicle, roadway and world, and how can we help to make that interaction safer.

The metrics described here apply generally to the experimental context where driving is tightly controlled. The latest trend in distraction research is, however, toward “naturalistic” studies, in which new technology allows researchers to collect real-world data with participants actually driving their own vehicles. While the potential for such research is tremendous, there will also be new challenges to assessing distraction.

Naturalistic driving data are a challenge precisely in that they lack experimental control (McGehee and Carsten, 2010; Boyle et al., 2010). Each distraction event, traffic conflict or crash, is essentially a case study. Countless unique factors—the time of day, roadway type, traffic density, number of distractions such as passengers and navigating—come into play, with the potential to elude scientific categorization and analysis. Aggregating data across many contexts will help identify the conditions under which visual and cognitive distraction is likely to occur.

There is still a struggle to determine which research platforms provide a gold standard in driver metrics. Traditionally, scientists have used experimental methods, where research questions are asked and hypotheses constructed, to help understand limits in driver performance. Variables are controlled and manipulated, statistical analyses are performed and results are determined. The evolving science of naturalistic driving is much more about filtering observations and data mining from a fluid and amorphous field where there is no experimental control. Both approaches are necessary to the attempt to understand the complexities of driver performance and behavior (Figure 12).

Figure 12.

Experimental and ND methods

The human mind, however, remains the final frontier. Data analysts are frequently baffled when trying to understand what a driver may be thinking or what might be the cause of a cognitive distraction. A wandering mind, for example, may divert a driver’s attention, but is difficult to classify relative to other cognitive distractions, such as being lost. Meanwhile, both may result in a variation of speed or lane position, and a general loss of situation awareness. While technology is not likely to be able to read a driver’s mind anytime soon, future vehicles will be able to detect distraction by monitoring a combination of driver inputs from steering and throttle variation and eye position. The vehicle will learn to recognize ‘normal’ driving inputs and eye movements so that when a driver is distracted, the system can alert the driver or even take over momentarily to avert a crash or limit its severity.

SAE and ISO provide detailed experimental descriptions of the evaluation methods and metrics described. Several existing and working draft ISO standards address the assessment of secondary task demand in the context of driving. ISO 15007a and 15007b (Measurement of Driver Visual Behavior) provide inputs on how to measure glance behavior, while ISO 16673 (Occlusion Method to Assess Visual Distraction) examines viewing time required to perform a task using in-vehicle information systems. ISO 26002 (Simulated Lane Change Test) provides a technique for evaluating the combined effect of sensory-actuator, perceptual-motor and cognitive demands on a driver’s performance in a combined event-detection-and-vehicle-control-task. These standards are frequently updated and provide useful guidance in overall assessment of driver attention demand.

There remains work to be done to understand the complexities of driver inattention. The naturalistic environment represents fertile ground for progress in this important safety area. Of particular interest is the measurement of self-regulation of distraction. While our exposure to electronic devices in the vehicle steadily increases, the fact is that crash rates are declining. Understanding and measuring this distraction-crash paradox will be central to the future of distraction measurement. There is increasing evidence that drivers may be choosing when to be distracted. Research indicates, for instance, that drivers will wait for stoplights or stop signs to read and type text messages (Funkhauser and Sayer, 2012). They may also slow down, wait for traffic conditions to calm, or simply pull over. How can we subjectively and objectively measure and predict which drivers will self regulate distraction? Are there personality characteristics that could help us do so? New instrumentation that can integrate speed and position information may represent a step toward answering these questions. By precisely measuring when a driver interacts with an electronic device relative to the roadway and traffic context, we may able to determine more precisely when a driver is likely to use such systems.

Finally, it is probable that in the not-too-distant future, intelligent vehicle systems will be able to intervene when they detect an anomalous condition that specifically relates to distraction, thus helping reduce crash severity or prevent some crashes altogether.