Eye-Blink Parameters Detect On-Road Track-Driving Impairment Following Severe Sleep Deprivation

Abstract

Study Objectives:

Drowsiness leads to 20% of fatal road crashes, while inability to assess drowsiness has hampered drowsiness interventions. This study examined the accuracy of eye-blink parameters for detecting drowsiness related driving impairment in real time.

Methods:

Twelve participants undertook two sessions of 2-hour track-driving in an instrumented vehicle following a normal night’s sleep or 32 to 34 hours of extended wake in a randomized crossover design. Eye-blink parameters and lane excursion events were monitored continuously.

Results:

Sleep deprivation increased the rates of out-of-lane driving events and early drive terminations. Episodes of prolonged eyelid closures, blink duration, the ratio of amplitude to velocity of eyelid closure, and John’s Drowsiness Score (JDS, a composite score) were also increased following sleep deprivation. A time-on-task (drive duration) effect was evident for out-of-lane events rate and most eye-blink parameters after sleep deprivation. The JDS demonstrated the strongest association with the odds of out-of-lane events in the same minute, whereas measures of blink duration and prolonged eye closure were stronger indicators of risk for out-of-lane events over longer periods of 5 minutes and 15 minutes, respectively. Eye-blink parameters also achieved moderate accuracies (specificities from 70.12% to 84.15% at a sensitivity of 50%) for detecting out-of-lane events in the same minute, with stronger associations over longer timeframes of 5 minutes to 15 minutes.

Conclusions:

Eyelid closure parameters are useful tools for monitoring and predicting drowsiness-related driving impairment (out-of-lane events) that could be utilized for monitoring drowsiness and assessing the efficacy of drowsiness interventions.

Clinical Trial Registration:

This study is registered with the Australian New Zealand Clinical Trial Registry (ANCTR), http://www.anzctr.org.au/TrialSearch.aspx ACTRN12612000102875.

Citation:

Shekari Soleimanloo S, Wilkinson VE, Cori JM,Westlake J, Stevens B, Downey LA, Shiferaw BA, Rajaratnam SMW, Howard ME. Eye-blink parameters detect on-road track-driving impairment following severe sleep deprivation. J Clin Sleep Med. 2019;15(9):1271–1284.

BRIEF SUMMARY

Current Knowledge/ Study Rationale: Eye-blink parameters have shown promise for the monitoring of driver drowsiness in laboratory and on-road experiments in shift workers. These assessments have not been evaluated to non-shift-work drivers during on-road daytime drives.

Study Impact: This is the first on-road study examining the accuracy of eye-closure parameters for detecting driver drowsiness in the general population after severe sleep deprivation or a normal night’s sleep. Findings revealed that the duration of eyelid closure, percentage of time with eyes closed, and the maximum amplitude to velocity ratios of eyelid movements were impaired after sleep loss, parallel to impairment in driving performance, and could detect out-of-lane events with moderate accuracy. As such, ocular parameters could continuously monitor drowsiness-related driving impairment in the real world.

INTRODUCTION

Drowsiness is a key cause of road crashes¹ that is implicated in approximately 20% of fatal crashes in developed countries.^2–4 Although drowsiness-related crashes pose significant costs globally (eg, estimated at between $12.5 billion to $40 billion annually in the US only ^5,6), the lack of an accurate objective predictor of driver drowsiness has precluded better understanding of drowsiness-related driving impairments and hampered development of effective interventions.

Monitoring of drowsiness ideally requires indicators that are involuntarily affected by drowsiness, can be continuously and noninvasively monitored via existing technologies in ambulatory settings such as driving, and reliably predict impaired driving performance. They should be acceptable to drivers and able to be integrated within driver safety monitoring systems to provide immediate warning to drivers affected by drowsiness.

Drowsiness impairs a range of driving performance indicators, reflected as reduced frequency of steering corrections,⁷ increased variation in vehicle lateral lane position,^8–10 and occurrence of out-of-lane events.^11,12 Although out-of-lane events appear later than lateral lane position deviation ^8–12 and frequency of steering corrections,⁷ they are a critical indicator of sleepiness-related driving impairment. They increase in frequency following sleep deprivation,¹³ occur before 65% of sleep-related road crashes,¹⁴ and are associated with measures of alertness.^15–17 Importantly, drowsiness monitoring technologies are able to predict these events.^18,19

Additional behavioral and physiological indicators of drowsiness (electroencephalography [EEG] activity,^20–25 electrocardiography [ECG] activity,^26–29 and behavioral vigilance^30–33) may precede out-of-lane events. However, these indicators could not be readily measured in real-driving settings. EEG and ECG systems are mostly intrusive^34,35 and have substantial noise in the signals in this setting due to head and body movements.^34,36,37 Likewise, it is not feasible to continuously monitor behavioral vigilance tests (eg, psychomotor vigilance task) during naturalistic driving settings.

Drowsiness impairs oculomotor function in patients who are trying to remain awake.³⁸ The physiological changes in ocular measures, which may directly contribute to drowsiness-related driving impairment, are continuously measurable by existing eye-monitoring technologies.^39,40 Currently, the Optalert system (Sleep Diagnostics Pty Ltd, Melbourne, Australia),⁴¹ which is integrated into driver safety monitoring systems, is widely used in transport,⁴² mining,⁴³ and aviation⁴⁴ sectors for detecting and providing warnings of critical levels of drowsiness.

In laboratory-based experiments, drowsiness has been observed as slower eye and eyelid movements,⁴⁵ especially longer blink durations ^46–49 and episodes of prolonged eyelid closure that can last for more than 10 seconds.^47,50 Drowsiness has reduced the amplitude and velocity of eyelid closure and reopening,^15,46,47 resulting in increased proportion of time with eyes closed.^15,47,51 The slower eyelid movements and longer eyelid closure times are attributed to more relaxed face and eyelid muscles after sleep loss due to reduced inputs to the motor projections to the face and eyelids.³⁹ In addition, these parameters have reliably predicted drowsiness-related errors and impaired vigilance ^49,52,53 that are additionally relevant to safe driving.

Changes in oculomotor function have been specifically described during simulated drives following sleep deprivation as increased proportion of time with eyelids closed and longer duration of eyelid closure that were correlated with both crashes and self-reported sleepiness,^10,15 as well as greater composite algorithms of oculomotor function that have been associated with crashes¹⁰ and out-of-lane events.¹⁵ In addition, pupillary activity ^54,55 and saccadic velocity (ballistic eye movements) have been impaired by extended awake time (sleep deprivation) ⁵⁵ that could further affect undesirable driving outcomes such as crash frequency.^54,55

Although eye-blink parameters have been clearly described in the laboratory setting, ^15,45–53 few studies have evaluated oculomotor changes during naturalistic drives or identified those ocular parameters that best identify drowsiness-related driving impairments. Lee et al¹⁷ evaluated driving performance and ocular parameters in shift workers during daytime track driving following a night shift compared to a well-rested night. Out-of-lane events, near crash events and early drive terminations were increased following night shift, with concurrent increases in slow eye movements, blink duration measures (eg, interevent duration [IED]) and JDS.¹⁷ In a naturalistic study in night-shift nurses commuting from work, ocular parameters were related to self-reported sleep-related events (eg, resting eyes, pulling over for a nap), hazardous driving events, (eg, harsh brake, hitting rumble strips) and inattention (eg, lack of awareness, distraction). In line with increases in sleep-related hazardous and inattentive events, ocular parameters (ie, JDS and maximum blink duration) increased while driving after a night shift compared to before a night shift.⁵⁶ Self-reported sleep related events were five times more likely at a mean maximum blink duration of 8 seconds and inattention was 4.5 times more likely at JDS scores more than 4.5.⁵⁶ In addition to these findings, Liang et al¹⁶ compared the accuracies of eye-blink parameters and driving parameters (when incorporated with individual driver factors) for predicting drowsiness-related driving events (out-of-lane events or microsleep episodes). Adding eye-blink parameters to the driver factors improved the models for detecting out-of-lane events, amplitude to velocity ratio of eyelid closure, and percentage of time with eyes closed, being the best predictors for out-of-lane events.¹⁶ Liang et al found that the accuracy of the models did not differ significantly when ocular parameters were measured within the minute of an out-of-lane event compared to up to 10 minutes prior to the same out-of-lane event.¹⁶

The available body of literature suggests that eye-blink parameters assessing blink duration (eg, blink duration, IED), amplitude to velocity ratio (AVR), percentage of time with eyes closed, and the composite measure (JDS) are consistently associated with impairments in driving performance.^40,57,58 Optalert technology currently has the greatest body of evidence ^40,57,58 demonstrating a good accuracy of these parameters for predicting driving impairment (eg, out-of-lane events and early drive terminations) in laboratory^10,15,40,58 and naturalistic driving settings.^{16,17,40,56,58}

Although the recent naturalistic driving studies^16,17,56 have observed changes in ocular parameters in shift workers, similar to those previously reported in laboratory studies, these changes have not been translated to non-shift-worker drivers during on-road daytime drives. To yield a robust conclusion on the accuracy of the eyelid movement parameters for detecting instantaneous drowsiness-related driving impairments in the general population, further research examining the predictive efficacy of ocular monitoring upon driving performance is necessary.

This study aimed to evaluate the accuracy of a range of eye-blink parameters for detecting drowsiness-related driving impairments and the time course of changes in these parameters relative to driving impairments in non–shift-work drivers. Three hypotheses were considered: (1) eye-blink parameters (blink duration, amplitude to velocity ratio of eyelid movements, percentage of time with eyes closed, and JDS) and driving performance (out of lane driving events and driving terminations) are impaired during daytime driving, following acute sleep deprivation compared to a rested state; (2) these eye-blink parameters are associated with impaired driving performance; and (3) these eye-blink parameters could predict impairment in driving performance. Therefore, the noninvasive and nonintrusive infrared reflectance oculography method was used for continuous recording of a variety of eye-blink parameters⁵⁹ during a 2-hour track drive following a 32- to 34-hour period of extended wake. Changes in gaze in sleep-deprived drivers have already been reported from this study.⁶⁰ The current study is reporting changes in eyelid closure parameters only.

METHODS

Study Design and Participants

Twelve healthy participants undertook two 2-hour sessions of driving in an instrumented vehicle in a randomized crossover design. Participants were recruited via advertisements at the Institute for Breathing and Sleep. Inclusion criteria were: age 21 to 65 years; full driver’s license (car) for a minimum of 3 years; driving experience of 5,000 km or more per year; low to moderate habitual caffeine intake (< 300 mg/d); and alcohol intake < 5 standard units/w. Participants were excluded if they had self-reported medical conditions affecting their driving or posing a hazard during sleep deprivation such as chronic neurological illness (eg, uncontrolled epilepsy). Severe cardiac or respiratory diseases; pregnancy; excessive sleepiness (eg, scores > 16 on Epworth Sleepiness Scale⁶¹); sleep disorders (narcolepsy and obstructive sleep apnea); impaired vision that did not correct with glasses; or regularly used sedating medications such as benzodiazepines, antihistamines, and narcotic analgesics, current night-shift work, and smoking were other exclusion criteria.

Study Protocol

Participants visited the laboratory once prior to the first session to determine their eligibility, provide informed consent, and be fitted with the Optalert glasses. They were instructed to maintain a fixed sleep-wake schedule with 8 hours of time in bed from 11:00 pm to 7:00 am for the 6 nights prior to each driving session. Adherence with this time in bed was monitored by Sense Wear pro 3 Armband (P3AB07202088, Body Media Inc., Pittsburgh, Pennsylvania, United States), which has positively validated against EEG,^62,63 and by sleep diaries. Participants reported their daily caffeine and alcohol intakes through demographics questionnaire upon enrollment.

Each participant undertook two driving sessions under well-rested and sleep-deprived conditions at 1 week to 2 months apart, in a randomized crossover design. In the well-rested condition participants had 8 hours of time in bed (11:00 pm to 7:00 am on the night prior to the track drive (night 7). In the sleep-deprived condition, participants woke up at 7:00 am on the day before the track drive and stayed awake before arriving at the Austin hospital sleep laboratory at 10:00 pm. Research staff continuously monitored participants to ensure they remained awake until the drive on the following day. They were permitted to engage in passive activities such as reading and watching videos. In both well-rested and sleep-deprived conditions no caffeine and other stimulant medication were allowed from 9:00 pm on the night prior to each driving session until the end of the session. Participants were transported by taxi between the study center and test track for both sessions and drove at the same time between 3:00 pm and 5:00 pm, but after 32 to 34 hours of continuous wake in the sleep-deprived condition. After a practice run, participants were instructed to drive in a closed-loop driving track, comprising both straight and curved sections, at posted speed limit of 50 km/h, while keeping the car in the center of the lane. Participants drove a dual-control vehicle while wearing the Optalert glasses. Concurrently with eye-blink parameters, out-of-lane driving events were recorded by Mobileye's Advance Warning System (AWS^™-1000, Mobileye Technologies Ltd., New York, United States) and by an experienced driving instructor manually, who was blind to the sleep conditions. The driver instructor was also responsible for intervening or stopping the vehicle and/or terminating the drive when the driver failed to control the vehicle safely (eg, waving), had a near-crash event or when the driver declared being too drowsy to drive. A researcher was sitting at the back of the car to record self-reported sleepiness scores at the start and end of the drive. This study was approved by Austin Health Human Research Ethics Committee.

Outcome Measures and Instruments

The eye-blink parameters of drowsiness were measured continuously during both drives using an infrared oculography system (Optalert).^15,64 This is a light (30 g) technology, which consists of a frame with an infrared sensor array positioned below the left eye. The sensor array comprises an in-built light emitting diode (LED) and a phototransistor beside the LED. The LED omits brief infrared pulses (IR-A band, 760–1,400 nm) to the left eyelid for less than 100 μs at a frequency of 500 Hz. The phototransistor calculates the infrared light reflected from the eyelid by deducting the ambient infrared light from the total light reflected from eyes and eyelids. The height of each pulse is calculated based on the position of the eye and eyelid relative to the phototransistor.^15,65 The analog output from the phototransistor is transmitted to a microprocessor, located in the arm of the frame, that digitizes the amplitude, duration, and velocity of movements of the eyelid and transfers a serial output (via Bluetooth) to a processing unit in a tablet.^15,65 The tablet is connected to the vehicle ignition system and calculates a series of eye-blink parameters including blink total duration (BTD), IED, long eye closures (LEC), positive amplitude to velocity ratio (AVR+), negative amplitude to velocity ratio (AVR−) and JDS. The composite parameter (JDS) is calculated from amplitude to velocity ratios and other parameters and scales driver drowsiness from 0 to 10, with 4.5, 5, and 10 representing cautionary, high-risk and extreme drowsiness levels, respectively. When the JDS exceeds 4.5 and 5 the system alerts the driver; however, this mechanism was disabled for the project. Table 1 lists these eye-blink parameters and provides their definitions. Optalert complies with Australian spectacle standards and operates in sunlight or darkness⁶⁴ and could be fitted with prescription lenses.

Table 1.

Summary of outcome measures of drowsiness.

Out-of-lane events were recorded by Mobileye system (AWS-1000) in addition to the driving instructor records. Mobileye comprises a forward-facing camera to record events, linked to a warning system that was muted for the study to avoid inducing temporary driver alertness from the sound and light alarm. Out-of-lane events were recorded when one of the car wheels touched the left or right shoulder lines. Instructor-recorded events were used in addition to Mobileye as the system may fail to detect out-of-lane events in the absence of road edge lines, on sharp turns or curves, with vehicle speeds lower than 50 km/h and in poor weather conditions. The rate of out-of-lane events, recorded by the instructor, was considered as the primary driving performance measure.

Subjective sleepiness scores were recorded prior to and after the drives using the Karolinska Sleepiness Scale (KSS), ranging from 1 (extremely alert) to 9 (extremely sleepy). KSS has been shown to be sensitive to sleep deprivation⁶⁶ and time of day.⁶⁷ Outcome measures of this study are provided in Table 1.

Data Analysis

Participants undertook two sessions of 2-hour drive sessions that produced 2,880 observations (in 1-minute bins) for every individual eye-blink and driving performance parameter. Eye-blink parameters and driving performance measures (Table 1) were time locked and grouped in 1-minute, 5-minute, and 15-minute timeframes to assess the association between these variables and out-of-lane events. Out-of-lane events was recorded as an event-based (count) variable. Percent of time with LEC was a continuous variable ranging from 0 to 100 (eg, 3.67, meaning that in 3.67% of a given minute the eyes were closed for more than 10 ms). A count variable was calculated from LEC % with any positive values of the LEC % counted as one LEC event. From the two count variables of out-of-lane events and LEC (count), secondary variables of out-of-lane event rate and LEC rate were calculated to standardize the number of these events over varying driving durations (due to early terminations of some drive sessions). Out-of-lane event rate and rate of LEC were defined as the number of these events per hour and were calculated for every 1-minute, 5-minute, and 15-minute timeframe separately.

The baseline demographic characteristics of the participants and their sleep durations were assessed by descriptive analysis. Mixed linear regression models were fitted on the sleep diary data to examine main effects of sleep condition (1 = well-rested versus 2 = sleep deprived and sleep-monitoring night (0 = first to the sixth monitoring night versus 1 = seventh monitoring night), as well as their interaction effects on time in bed and sleep durations. Mixed linear analyses were also fitted on KSS data to examine the effect of sleep condition on self-reported drowsiness before and after track drives.

Descriptive analyses were conducted on eye-blink parameters and driving performance variables, followed by calculation of rate ratios for count and rate variables, McNemar exact test for drive terminations, and paired t test for continuous variables to examine the effect of sleep condition on all outcome measures. Mixed linear models were fitted on 15-minute datasets to assess the effects of interactions of sleep condition (well rested versus sleep deprived), with time-on-task (8 segments of 15-minute drive) on eye-blink parameters and out-of-lane events. Survival analysis was conducted on the elapsed drive time before drive terminations (the log-rank test) to compare survival probabilities in the two sleep conditions.

The relationships between eye-blink parameters and out-of-lane events were assessed using logistic regression and receiver operating characteristic (ROC) curve analyses. Logistic regression models were fitted on raw continuous eye-blink variables in 1-minute, 5-minute, and 15-minute timeframes and were clustered by participant.^68,69 Odds ratio (OR) from regression analyses were standardized (adjusted for standard deviation of each eye-blink variable). Sensitivity and specificity were assessed for eye-blink parameters and two cutoff points were reported for detecting drowsiness-related out-of-lane events; a high-sensitivity cutoff value, using the highest sensitivity with a specificity of ≥ 50%, and a high-specificity cutoff value by considering the highest specificity with a sensitivity of ≥ 50%. Using mixed linear models effects of sleep conditions on JDS scores in 1 minute, 5 minutes, and 15 minutes before drive terminations were examined. Logistic regression and ROC analyses were also conducted to determine how accurately JDS values could predict drive terminations (STATA 15, StataCorp LP, College Station, Texas).

RESULTS

Demographic Details and Prior Sleep

Twelve participants were recruited (6 males, 6 females, mean age 33.7 ± 7.1 years). All participants scored less than 10, with 40% of them scoring ≤ 3, on the Epworth Sleepiness Scale. All participants conducted the actigraphy. Data from one participant were lost due to technical failure for the week before the two driving sessions. Data from another participant were lost for the week before one of the driving sessions due to skin irritation. Data from the other 10 participants were partially missing from incomplete use of actigraphy for the week before either of the driving sessions. As such, only sleep diary data were used to verify sleep-wake times. There was no difference in the time in bed between the well-rested and the sleep-deprived conditions for the first 6 nights of the protocol (8.1 ± 1.15 hours versus 8.3 ± 0.89 hours respectively, P = .256). As per experimental design, there was a significant difference for the seventh night of the protocol, when drivers spent 8.2 ± .20 hours in bed during the well-rested condition, as opposed to zero hours in the sleep-deprived condition (P < .0001, 95% confidence interval [CI] −9.18 to −7.64).

Sleep duration did not differ significantly between the well-rested and the sleep-deprived conditions for the first to sixth nights of sleep (7.7 ± 1.23 hours versus 7.7 ± 1.03 hours respectively, P = .842). Significant difference in sleep time were observed between the well-rested and the sleep-deprived conditions on the seventh night (7.6 ± 0.42 hours versus zero hours, P < .0001, 95% CI −8.46 to −6.86).

Predrive and Postdrive Self-Reported Drowsiness

A mixed linear model demonstrated that self-reported drowsiness (KSS) in the sleep-deprived condition was greater than in the well-rested condition (P < .0001, 95% CI 1.94 to 4.25). Postdrive KSS was greater than the predrive KSS in both sleep conditions (P = .004, 95% CI 0.54 to 2.85), with no interaction of sleep deprivation by timing of KSS (P = .72). This suggests a significant time-on task effect on the KSS that differs from specific effect of sleep deprivation.

Effect of Sleep Condition on Driving Performance and Eye-Blink Parameters

Driving performance indicators averaged across 1-minute bins were significantly worse during the drive in the sleep deprivation condition relative to the normal sleep condition (Table 2), with out-of-lane events rate increasing by 3.7 times after sleep deprivation (P < .0001). Early drive terminations were observed in 7 of 12 participants in the sleep-deprived condition as opposed to only 1 of 12 in the well-rested condition (OR 1.17, P = .031).

Table 2.

Descriptive analysis of eye-blink parameters and driving variables recorded in 1-minute bins during the well-rested and sleep-deprived conditions.

Similarly, significant differences in all continuous eye-blink parameters (Table 2) were observed after sleep deprivation compared to the well-rested condition (all P < .0001). IED, BTD, and JDS increased by 18.6 ms, 65.3 ms, and one score, respectively, during the sleep deprivation condition. There was a 2.9-fold increase in the rate of LEC, as well as a rise of 0.56% in the percentage of time with eyes closed. The ratio of the maximum amplitude to maximum velocity of eyelid for the reopening phase (AVR−) and closing (AVR+) phases of blinks increased by 0.14 seconds and 0.16 seconds respectively.

Mixed linear analysis showed that JDS scores in 1-minute immediately before drive terminations, were not significantly greater than random JDS scores throughout the course of drive in the same blocks in sleep deprivation condition (P = .514, 95% CI −1.58 to 3.16). Conversely, significant effects of sleep deprivation were evident on JDS scores in the 5 minutes (P = .0001, 95% CI 0.71 to 2.42) and 15 minutes (P = .001, 95% CI 0.39 to 1.49) prior to drive terminations compared with random JDS scores within the same 5-minute and 15-minute timeframes.

Descriptive analyses of driving performance and eye-blink parameters in 1-minute, 5-minute, and 15-minute timeframes are presented in the supplemental material (Figure S1, Figure S2, Table S1 and Table S2).

Time-on-Task Effect on Driving Performance and Eye-Blink Parameters

Linear mixed-effect analyses of eye-blink data, averaged over 15-minute timeframes (driving segments 1 to 8) demonstrated that some eye-blink parameters were affected by time on task (duration of drive) alone, regardless of the sleep condition (deprived or normal sleep). This was evident for JDS scores, with a significant increase from baseline (driving segment 1) after 45 minutes of driving (P = .014), and the AVR−, with a significant increase from baseline (driving segment 1) after 30 minutes of driving (from P < .005 to P < .0037).

Interaction of Time on Task by Sleep Deprivation

Linear mixed-effect analyses of data averaged over 15-minute timeframes revealed the interaction of time-on-task (8 segments of 15 minutes) by sleep condition on early drive terminations (Figure 1A) from the 30th minute of the drive onward. Although 3 of 7 terminations happened before the 60th minute (3rd and 4th segments), 4 of 7 of terminations occurred between the 60th and the 90th minutes into the drives (5th and 6th segments) after sleep deprivation (log rank test, χ²₆ = 39.6, P < .0001). The interaction of driving duration and sleep condition was more prominent for rate of out-of-lane events, increasing significantly after 15 minutes of driving (Figure 1B).

Figure 1. Interaction effect of drive segment by sleep condition on driving performance and eye-blink parameters.

(A) Kaplan Meier survival curve, with dashed line and solid line representing drive terminations following the well-rested and the sleep-deprived conditions respectively. (B-H) Estimated marginal mean values of eye-blink parameters and out-of-lane events after two sleep conditions in 8 consecutive 15-minute driving segments. * Significant interaction between sleep condition and driving segment (P < .05). Error bars are 95% CI. Hollow diamonds (blue) and hollow circles (red) represent well-rested and sleep-deprived conditions, respectively. AVR− = negative amplitude/velocity ratio, BTD = blink total duration, IED = interevent duration, JDS = John’s Drowsiness Score, LEC = long eye closures.

In regard to the eye-blink parameters, there was an early increase in average blink duration, after 30 minutes into the drive in the sleep deprivation condition. There were significant increases in percentage of time with LEC after 45 minutes, as well as the JDS and rate of LEC after 75 minutes (Figure 1B). However, no interaction effect was evident for the AVR− or IED. A similar pattern was evident for the alteration of several eye-blink parameters when sleep deprivation interacted with time on task. There were progressive increases in LEC (rate), LEC (percent), and rate of out-of-lane events, as well as JDS values until the 90th minute into the drive. However, these eye-blink parameters displayed less impairment from the 90th minute into the drive onward (the 6th driving segment). This was the time point that most early-drive terminations from excessive sleepiness occurred.

Relationship Between Eye-blink Parameters and Out-of-Lane Events

The JDS, AVR−, number of LEC, and rate of LEC demonstrating significant relationships when averaged over the 1-, 5-, and 15-minute timeframes (OR 1.02 to 30.09) (Table 3). The BTD and IED only demonstrated significant relationships with out-of-lane events when averaged over longer timeframes (5-minute and 15-minute bins, OR 1.005 to 1.032). Conversely, AVR+ and the percentage of time with LEC were only related to out-of-lane events when assessed over shorter timeframes (1-minute and 5-minute bins, OR 1.2 to 9.1). For all eye-blink parameters, the relationship to driving impairment was greater when variables were averaged over longer timeframes (5 or 15 minutes) rather than 1 minute, except for the number of LEC (Table 3).

Table 3.

The odds ratios and standardized odds ratios of out-of-lane events for eye-blink parameters.

Based on the standardized OR (Table 3), the JDS scores had the strongest association with drowsy driving performance (out-of-lane events) in 1-minute timeframes. A rise of 1.3 score in the JDS (equivalent one standard deviation) was associated with a 2.2-fold greater likelihood of out-of-lane events in the same minute. When averaging over 5-minute timeframes the IED showed the strongest association (standardized OR at 3.2), whereas in 15-minute timeframes the LEC rate had the highest standardized OR of 5.85.

The OR for out-of-lane events for increase in of eye-blink parameters compared to their lowest percentile are presented in Figure 2. The 5% highest BTD, IED, LEC and AVR+ were related to a fivefold to 26-fold increase in odds of out-of-lane events in 1-minute data (Figure 2, left panels), that increased to a tenfold to 32-fold increase for their corresponding averaged values in 5-minute timeframes (Figure 2, right panels). The highest 5% of JDS scores indicated a 31-fold increase in odds of out-of-lane events over a 1-minute timeframe, dropping slightly to a 24-fold increase over a 5-minute timeframe.

Figure 2. Odds ratios for out-of-lane events for 50th, 75th, 95th, and 100th percentiles of eye-blink parameters.

Left panels: 1-minute segments. Right panels: 5-minute timeframes. Error bars are 95% CI. The 50th, 75th, 95th, and 100th percentiles are presenting 26th to 50th centiles, 51st to 75th centiles, 76th to 95th centiles and 96th to 100th centiles respectively. * Significant differences between odds of out-of-lane events between the 50th, 75th, 95th, and 100th percentile relative to the 25th percentile. AVR+ = positive amplitude/velocity ratio, BTD = blink total duration, IED = interevent duration, JDS = John’s Drowsiness Score, LEC = long eye closures.

Cutoff Values for the Eye-Blink Parameters

Receiver Operating Characteristic analysis of eye-blink parameters (Table 3) identified moderate accuracies for detecting out-of-lane events in all timeframes, with greater area under the curve (AUC) over longer timeframes. The JDS had the greatest predictive ability over all timeframes (AUC ranging from 0.76- 0.79), with AVR− and rate of LEC achieving the second predictive ability over a 1-minute timeframe (AUC = 0.68) and for both 5-minute and 15-minute timeframes (AUCs from 0.72 to 0.77) respectively.

Setting the cutoff values at a sensitivity of 50% (or the closest value if 50% could not be achieved) (Table 4, Figure S1 and Figure S2) yielded specificities of 50% to 84% for eye-blink parameters for detecting out-of-lane events in the same 1-minute period (Figure S1A and Figure S2A). These specificities increased to 50% to 90% over 15-minute timeframe (Figure S1C and Figure S2C). Setting the cutoffs to achieve a specificity of at least 50% (or the closest value) provided sensitivities at 42% to 82% for the same 1-minute and 49% to 91% for the same 15-minute timeframes, respectively. At sensitivities of 50% for the same 1-minute timeframe, JDS and rate of LEC had the highest specificity at approximately 84%. The rate of LECs that could not achieve sensitivities more than 42% in a 1-minute timeframe provided the high specificity of 89.2% for a sensitivity of approximately 50% in a 15-minute timeframe.

Table 4.

The cutoff values of selected outcomes for greatest specificity at 50% sensitivity and greatest sensitivity at 50% specificity.

Relationship Between Eye-blink Parameters and Early Drive Terminations

Logistic regression analyses (Table 5) revealed all eye-blink parameters were significantly associated with drive terminations (values of P ranging from .0001 to .001), except for IED (P = .383). For every unit increase in AVR+, AVR−, and JDS the odds for drive terminations increased by 9, 5 and 1.8 times, respectively. Comparison of standardized OR for drive terminations revealed that JDS was most strongly associated with drive terminations, with one standard deviation rise in the JDS and the likelihood of drive terminations increasing by 2.3 times. The AVR− and BTD showed the second and third strongest associations with drive terminations (OR of 2 and 1.5, respectively).

Table 5.

The odds ratios and standardized odds ratios of early drive termination for eye-blink parameters.

The OR for drive terminations with increasing impairment of eye-blink parameters are compared to the lowest quartile of each parameter in Figure 3. The odds of early drive terminations were 21 times greater in minutes with the highest 5% of JDS scores compared to the lowest quartile. The highest 5% of BTD and IED (both OR 20), as well as AVR− (OR 18) also indicated a substantial increased risk for early drive termination.

Figure 3. ROC plots for sensitivity and specificity of all JDS scores for predicting drive terminations.

Left panels: ROC plots for all JDS scores prior to drive termination. Right panels: ROC plots for in JDS scores 75 minutes prior to drive termination. The X and Y axes are presenting sensitivity and 1-specificity respectively. AVR− = negative amplitude/velocity ratio, BTD = blink total duration, IED = interevent duration, JDS = John’s Drowsiness Score, ROC = receiver operating characteristics.

ROC analyses of eye-blink parameters suggest that values of BTD and JDS larger than their respective cutoff values (426 ms and 2.6) could predict drive terminations with moderate accuracies (area under the curve [AUC] 0.71 and 0.73, respectively). BTD and JDS achieved specificities of 78.94% and 83.76% for predicting drive terminations at respective sensitivities of 50% and 54%.

DISCUSSION

This study uniquely determined the accuracy of eye-blink parameters for continuous monitoring and detection of driving impairment during real driving. Consistent with previous findings and our hypotheses, sleep deprivation led to an increased number of out-of-lane events and early drive terminations, with a parallel deterioration in eye-blink parameters that were also impaired by drive duration. Importantly, changes in the eye-blink parameters indicated an increased risk of out-of-lane events and could predict these events with a moderate accuracy. These findings extend previous laboratory work and confirm the capability of individual eye-blink parameters to identify drowsiness during real driving.

The current study observed that 32 to 34 hours of acute sleep deprivation resulted in substantial driving impairment, with a 3.7-fold increase in out-of-lane driving events and a 58% increase in early drive terminations. These changes were more severe than the previously reported out-of-lane rate ratio (2.08) and drive terminations (43%) in the on-road study of Lee et al in shift workers when driving for 2 hours in the morning after a night shift.¹⁷ Premature drive terminations in the current study also commenced earlier (between 30th and 90th minutes) when compared with the study of Lee et al (65th and 110th minutes).¹⁷ Higher severity of sleep deprivation in the current study (32- to 34-hour extended wake versus 11.5- to 17.5-hour extended wake periods) likely explains these greater driving impairments. Conversely, the ratio of out-of-lane events in the current study was less than the previously reported rate ratio (6.0) in the study of Sagaspe et al. during a 2-hour nighttime on-road drive.¹³ The circadian phase and time-on-task effects may have influenced this difference, because in the study of Sagaspe et al the ratios of out-of-lane events were assessed between 3:00 am and 5:00 am and were reported only for the last hour of the 2-hour drive.

Parallel to driving impairment, increments in eye-blink parameters were observed including duration of eyelid closure, percent of time with eyes closed and maximum amplitude to velocity ratios of eyelid movements for both closing and opening phases of blink (AVR), indicating slower eyelid movements during blinks in the sleep deprivation condition during real driving. Similar changes were evident in the JDS score, a single composite average score of drowsiness per minute.¹⁷ The severity of impairments for these parameters in the current study are in line with previously reported impairments during laboratory studies following different types of sleep restriction, such as total sleep deprivation, prior restricted sleep and circadian misalignment, and even greater than impairments in shift workers during a 2- hour track drive.¹⁷ The greater impairments in eyeblink parameters in this study might have stemmed from more severe sleep deprivation causing more relaxed eyelid muscles.³⁹ The progressive increase of out-of-lane events during the course of the drive following sleep deprivation suggests a time-on-task (drive duration) effect on driving impairment following sleep deprivation that was not evident following normal sleep. Similar effects were evident in some of the eye-blink parameters such as duration of eyelid closure, percent of time with eyes closed, and JDS in parallel with early drive terminations. The effect of drive duration on driving impairment has been noted previously in shift workers, and is also evident from road crash data,⁷⁰ with an increased crash risk after 2 hours of driving. However, these track studies suggest the risk is likely to increase earlier in severely sleep-deprived drivers.

Consistent with eye-blink parameters and driving performance outcomes, there was significant increase in self-reported drowsiness post drives when compared with predrive drowsiness in both sleep-deprived and well-rested conditions. This suggests a significant time-on-task effect on perceived drowsiness that is not necessarily derived from sleep deprivation. Interestingly, slight reductions were observed in out-of-lane events and some eye-blink parameters in the last 15 to 30 minutes of the sleep-deprived driving session. From the current data, it is impossible to distinguish underlying factors for these lower impairments. However, missing data from early drive terminations in two-thirds of the drives (in the sleepiest drivers) could have played a role. The anticipatory effect of completing the drive may also have enhanced alertness and performance.⁷¹ There could also have been a circadian rhythm effect on drowsiness and driving performance, given that the last 30 minutes of drives occurred from 4:30 pm, whereas transitioning from the midafternoon circadian dip to higher alertness levels.^72,73

To our knowledge, this is the first study describing the relationship between individual eye-blink parameters and driving impairment during naturalistic track driving following severe sleep deprivation. This study extends findings of a previous track driving study, as well as other laboratory experiments demonstrating eye-gaze parameters, eye-blink duration, the percentage of time with eyes closed, and the amplitude to velocity ratios of eyelid movements are reliable indicators of drowsiness related impairment.^49,60 In the current study the JDS, an algorithm incorporating several eye-blink measures, was the strongest predictor of out-of-lane events in 1-minute timeframes (based on standardized OR and ROC curve AUC). This is consistent with some previous laboratory experiments and a track-driving study in shift workers, but inconsistent with previously described insignificant relationships between JDS and self-reported hazardous events (including hitting rumble strips and having a near miss) in nurses after night shift.⁵⁶ This discrepancy may be explained by purely assessing a specific JDS cutoff (rather than treating it as a continuous variable)⁵⁶ in nurses and that the self-reported nature of these events in nurses is likely to be less reliable than objective measures of impairment used in the current and other studies.

Interestingly, in the current study measures of blink duration (IED and BTD) and the rate of LEC became stronger predictors of out-of-lane events when assessed over 5-minute timeframes, with the rate of LEC being a strong predictor of impairment when assessed over a 15-minute timeframe. Apart from IED, all eye-blink parameters were associated with drive terminations, with JDS having the strongest association with drive terminations, followed by BTD and AVR−. For JDS, BTD and AVR−, the likelihood of early drive terminations increased by 18 to 20 times for their highest 5% of scores compared to their lowest quartile.

In the current study, the eye-blink parameters varied in their ability to achieve high sensitivity and high specificity for detecting out-of-lane events and for most parameters, which improved when assessed over longer timeframes. Blink duration (IED and BTD), AVR−, and the JDS achieved sensitivities of 85% to 91% for detecting driving impairment in the 15-minute timeframe (specificity set at 50%). In the same 15-minute timeframe, JDS scores at the current standard cutoff of 4.5 had a very low false-positive rate (specificity 96% to 100%), but achieved a modest sensitivity. Interestingly, the rate of LEC, at a sensitivity of about 50%, achieved the second highest specificity (89.2%) for detecting driving impairment and so appears to be a later but specific indicator of drowsiness in keeping with previous laboratory findings.⁴⁹ JDS and BTD predicted early drive termination with moderate accuracies (sensitivity about 72%), with high specificities of 83% and 78% respectively, at cutoff values of 4.5 and 426 ms.

Understanding the different properties of eye-blink parameters for detecting drowsiness-related impairment will enable determination of appropriate parameters and cutoffs to utilize those with higher sensitivity when it is critical to avoid missing drowsiness-related impairment, and those with high specificity when it is important to avoid false-positive findings such as legal determination of impairment.

Several limitations need to be considered for the findings of the current study. (1) Given the small sample, these results require replication in a larger group of drivers to ensure generalizability of the findings. (2) This study experimentally induced drowsiness through sleep deprivation. From the current study, it is difficult to distinguish the different effects of time-on-task, circadian rhythm and prolonged wake period on eye-blink parameters. Based on laboratory studies the relationship between these parameters and driving impairment is likely to be similar, irrespective of which factor is causing drowsiness; however, this requires confirmation during real driving.⁵⁹ (3) This study relies on mainly out-of-lane events and to a lesser extent on drive terminations as the key indicators of drowsiness-related driving, which are likely to be indicative of more severe impairment.⁷⁴ Relying solely on out-of-lane events may have resulted in missing episodes of less severe drowsiness related impairments.⁷⁵ We acknowledge that other behaviors and physiological indicators of drowsiness may precede out-of-lane events (EEG activity, ECG activity, and behavioral vigilance), but many of these indicators have limitations in real vehicle studies. It is clearly important that drowsiness monitoring technology is able to predict out- of- lane events. (4) In the current study technical failure of the armband actigraphy meant that estimation of sleep-wake times were based on self-reported sleep diaries rather than objective measurement of sleep. Although this may alter the measurement of prior sleep duration for nights at home, participants were monitored in the laboratory to ensure total sleep deprivation for this condition and this does not affect the relationships identified between the eye-blink parameters and driving performance. (5) In the current study control for stimulants prior to the well-rested condition was based on self-reported refrain from consumption of caffeine and alcohol on the night prior to the drive. This might have affected alertness levels in the well-rested condition if consumed.⁷⁶ All participants were monitored in the laboratory to prevent caffeine/alcohol consumption on the night before the drive. (6) The presence of the driving instructor and the researcher in the vehicle may have altered drivers’ alertness or their normal approach to driving (eg, speed monitoring or levels of concentration on maintaining lane position). Given the extended sleep deprivation on one hand, and in-field limitations of the Mobileye system on the other hand, the presence of a professional driving instructor was necessary to ensure safety of the driver and the researcher and to verify out-of-lane events. (7) This experiment was conducted on a track with other vehicles or hazards not being present, which may have contributed to reduced alertness. The findings of this study require further validation with a larger sample of drivers and under different sleep deprivation and circadian rhythm phases while incorporating lesser degrees of driving impairment.

To conclude, eye-blink parameters are sensitive to acute sleep deprivation and the time-on-task effect of driving duration. Slowed eyelid movements, prolonged blink duration, and frequent episodes of prolonged eye closure are associated with an increased likelihood of out-of-lane events and early drive terminations, with the relationships becoming stronger when measured over longer time periods of up to 15 minutes. As such, ocular-based drowsiness monitoring is a promising method for continuous monitoring of drowsiness-related impairment in the real world, assessing the effect of schedules and practices that result in drowsy driving, as well as for evaluating interventions for drowsy driving.⁴⁰ Hence, alertness monitoring technology needs to be incorporated in a holistic regulatory and industry framework in order to prevent sleep-related crashes and save lives.

DISCLOSURE STATEMENT

This study was performed at the Institute for Breathing and Sleep, Department of Respiratory and Sleep Medicine, Austin Health, Australia. All authors have made substantial contributions to the work presented and have approved the final version of the manuscript. The project was funded through project grants from VicRoads. M.E.H. has received research grants from the Cooperative Research Centre (CRC) for Alertness, Safety and Productivity and the Mining CRC, and loan equipment for research from Optalert and Seeing Machines. The authors report no conflicts of interest.

ABBREVIATIONS

AVR−

negative amplitude/velocity ratio

AVR+

positive amplitude/velocity ratio

BTD

blink total duration

ECG

electrocardiography

EEG

electroencephalography

IED

interevent duration

JDS

John’s Drowsiness Score

KSS

Karolinska Sleepiness Scale

LEC

long eye closures