Feeling sleepy? stop driving—awareness of fall asleep crashes

Abstract

Study Objectives

To examine whether drivers are aware of sleepiness and associated symptoms, and how subjective reports predict driving impairment and physiological drowsiness.

Methods

Sixteen shift workers (19–65 years; 9 women) drove an instrumented vehicle for 2 hours on a closed-loop track after a night of sleep and a night of work. Subjective sleepiness/symptoms were rated every 15 minutes. Severe and moderate driving impairment was defined by emergency brake maneuvers and lane deviations, respectively. Physiological drowsiness was defined by eye closures (Johns drowsiness scores) and EEG-based microsleep events.

Results

All subjective ratings increased post night-shift (p < 0.001). No severe drive events occurred without noticeable symptoms beforehand. All subjective sleepiness ratings, and specific symptoms, predicted a severe (emergency brake) driving event occurring in the next 15 minutes (OR: 1.76–2.4, AUC > 0.81, p < 0.009), except “head dropping down”. Karolinska Sleepiness Scale (KSS), ocular symptoms, difficulty keeping to center of the road, and nodding off to sleep, were associated with a lane deviation in the next 15 minutes (OR: 1.17–1.24, p<0.029), although accuracy was only “fair” (AUC 0.59–0.65). All sleepiness ratings predicted severe ocular-based drowsiness (OR: 1.30–2.81, p < 0.001), with very good-to-excellent accuracy (AUC > 0.8), while moderate ocular-based drowsiness was predicted with fair-to-good accuracy (AUC > 0.62). KSS, likelihood of falling asleep, ocular symptoms, and “nodding off” predicted microsleep events, with fair-to-good accuracy (AUC 0.65–0.73).

Conclusions

Drivers are aware of sleepiness, and many self-reported sleepiness symptoms predicted subsequent driving impairment/physiological drowsiness. Drivers should self-assess a wide range of sleepiness symptoms and stop driving when these occur to reduce the escalating risk of road crashes due to drowsiness.

Graphical Abstract

Statement of Significance.

Drowsy driving remains a significant public health concern, contributing to up to 20% of serious injury/fatal crashes. Public education messaging to reduce the risk of falling asleep at the wheel, recommends that drivers stop and take a break when drowsy. In a prospective, on-road study, we show drivers are aware of sleepiness while driving. As drivers may not routinely ask “how sleepy do I feel,” we recommend that they should also reflect on sleepiness symptoms, particularly those relating to the eyes or driving behavior (e.g. ‘struggling to keep the eyes open and/or the center of the road). Ignoring these earlier warning signs, or waiting for symptoms such as head nodding to develop, represents a significant risk to road safety.

Introduction

Drowsy driving remains a significant public health concern. In the United States, drowsiness is involved in 7% of all motor vehicle crashes, and 13%–21% of those resulting in serious injury or fatality [1]. Due to the difficulty in determining causality in drowsiness-related crashes [2], these figures likely underestimate the true extent of the problem, which is estimated to be 40% of all highway crashes [3]. Despite these risks however, driving while drowsy remains common, such that 1 in 25 drivers report having fallen asleep while driving in the past month [4]; the equivalent of ~1.16 million US drivers falling asleep at the wheel each month [5]. Since drowsiness is the result of insufficient sleep, prolonged wakefulness, and/or driving during the nighttime hours, these crashes are largely preventable. However, many individuals are routinely exposed to drowsiness-inducing factors that are beyond their control. This is particularly evident for night-shift workers, who are at increased risk of drowsy driving, particularly on the commute home following an overnight shift [6–10]. As drowsiness is an inevitable consequence for a large majority of these workers, strategies beyond simply “avoiding drowsiness” are required.

Road safety is a shared responsibility, and drivers should ensure they are safe to drive. From a drowsiness perspective, drivers can use simple scales, such as the Karolinska Sleepiness Scale (KSS), to routinely assess how sleepy they feel prior to driving and throughout the drive. In a study of healthcare workers driving home after an extended duration of work shift, a pre-drive KSS assessment greater than six (“Some signs of sleepiness”) predicted 91% of subsequent drives with an adverse driving event [6]. During the course of a drive however, previous research has suggested that some individuals are poor at assessing sleepiness and/or predicting an impending involuntary sleep episode [11–13]. Contrary to this, our recent systematic review and meta-analysis (_ma) suggested that drivers are aware of sleepiness, such that KSS while driving was correlated with both ocular (r_{ma =} 0.70) and brain (r_{ma =} 0.74) derived measures of drowsiness, and predicted subsequent lane deviations and crash risk [14]. Despite this, the majority of studies captured in the review utilized simulated driving/laboratory environments, highlighting a need for an examination of subjective sleepiness and objective drowsiness and associated driving outcomes in real-world environments. Moreover, as many drivers reporting sleepiness continue to drive [15], it has been suggested that drivers may not recognize sleepiness as serious or specific enough to cease driving [16]. For instance, using a retrospective questionnaire approach, Nordbakke and Sagberg reported that drivers often notice specific symptoms such as “difficulty keeping the eyes open” or “difficulty remaining in the center of the road” prior to falling asleep when driving [16], while a prospective low fidelity simulator study reported that sleepiness symptoms (such as “blurred vision”) were related to both subjective sleepiness (KSS) and driving impairment [17]. While assessing these symptoms may better assist drivers to recognize their level of sleepiness and ability to drive safely, this has not yet been examined prospectively and under real-world conditions.

To specifically address these two gaps in knowledge, we conducted new analyses of our earlier prospective, on-road track study in night-shift workers [18] to determine the extent to which drivers are aware of sleepiness, and how this predicts subsequent physiological drowsiness and unsafe driving outcomes while driving. Specifically, we will address which sleepiness symptoms best predict adverse driving outcomes by examining self-reported sleepiness and provide optimal thresholds of those measures by which drivers should take corrective action.

Methods

Participants

Sixteen night-shift workers (nine women) between the ages of 19 and 65 years (M = 48.7 ± 14.8 years) took part in the study. They worked regular night shifts (at least 5 continuous hours between 22:00–08:00, M = 3.1 shifts/week) across a variety of shift work sectors, held a valid United States or International driver’s license for >2 years (M=27.4 years ±16.5 years), and had normal visual acuity (with or without corrective lenses). Participants provided full informed consent and were remunerated for their time. Ethical approval was obtained from the Brigham and Women’s Institutional Review Board (#2011P000370) and Monash University Human Research Ethics Committee (#25777).

Design and protocol

A within-subject, cross-over design was used as previously described [18]. Participants drove a dual control, instrumented vehicle around a closed-loop track for 2 hours on two occasions: after a night of work (at least 5 hours of work between 10:00 pm–08:00 am) and a night of sleep (at least 5 hours of sleep between 10:00 pm–08:00 am). Due to the ecological nature of the study, no manipulation of sleep/wake schedule was implemented and drive order was not counterbalanced. A post hoc check however, confirmed no effect of drive order on study outcomes (see [18] for details). Sleep/wake timing, medication use, and caffeine/alcohol use was monitored throughout the study using diaries for 1 week prior to each driving session. Participants were transported to and from the test facility via taxi for safety, and accompanied by a researcher to ensure they remained awake and did not consume caffeine in the two hours prior to the drive. Drives were initiated 2 hours post-nightshift which corresponded to between 09:30 am and 02:30 pm. The post-sleep and post-nightshift drive were time-matched to control for any time-of-day effects between drives. Drivers were asked to adhere to typical driving conditions (e.g. within the road markings, within speed limit requirements), and stopped briefly (<2 minutes) every 15 minutes to conduct sleepiness assessments.

Measurements

Subjective ratings of sleepiness and sleepiness symptoms were monitored at 15-minute intervals throughout the drive, alongside continuous assessment of driving impairment and physiological drowsiness (See Table 1).

Table 1.

Study Outcomes

Outcome type	Outcome descriptor	Level of impairment	Description
Subjective	Karolinska Sleepiness Scale	Low to Severe	Sleepiness in the prior 5 minutes ranging from Extremely Alert [1] to Very Sleepy, Fighting Sleep [9].
	Likelihood of Falling Asleep	Low to Severe	Likelihood of falling asleep in the next few minutes, ranging from Very Likely [1] to Very Unlikely [5]
	Sleepiness Symptoms Questionnaire	Low to Severe	Frequency of sleepiness symptom experienced in past 15 minutes, ranging across Not at all [1], Occasionally [3], Frequently [5], and Most of the Time [7].
	SSQ1: struggle to keep eyes open
	SSQ2: Vision Blurred
	SSQ3: Nodding Off to Sleep α
	SSQ4: Difficulty Keeping to Middle of Road
	SSQ5: Difficulty Maintaining Correct Speed
	SSQ6: Mind Wandering α
	SSQ7: Reactions are Slow α
	SSQ8: Head Dropping
Objective	Emergency Braking	Severe	In-Vehicle Safety Instructor Applies the Emergency (second) Brake as a Safety Intervention, representing Near-Crash Events.
	Lane Deviation	Moderate	Number of Times (per 15 minutes) that one or more Vehicle Wheels Crossed the Lane Markings
	Johns Drowsiness Scale ≥4.5	Severe	Composite JDS scores that equaled or exceeded 4.5
	Johns Drowsiness Scale ≥2.7	Moderate	Composite JDS scores that equaled or exceeded 2.7
	EEG Microsleep (Fall Asleep Event)	End-state	>3 s of EEG activity 8hz or less

1. KSS: Participants rated how sleepy they felt in the past 5 minutes on a 9-point scale from “extremely alert” to “extremely sleepy” [19], using the adapted KSS with descriptors on each point [13]. Higher scores indicate higher sleepiness.

2. Likelihood of falling asleep (LFA): Participants rated how likely they might fall asleep in the next 5 minutes on a 5-point scale of “very unlikely” to “very likely” [20], with higher scores indicating higher likelihood (scale reversed after data collection).

3. Sleepiness symptoms questionnaire (SSQ) - Participants rated the frequency of eight sleepiness symptoms on a 7-point scale, ranging from “not at all” to “most of the time” [17]. Higher scores indicated a greater frequency of the sleepiness symptom. The eight sleepiness symptoms (SSQ1–8) can be seen in Table 1 and include ratings related to drowsiness (e.g. struggle to keep eyes open), attention (e.g. mind wandering), and driving performance (e.g. difficulty keeping to the middle of the road).

4. Adverse driving events: Participants drove a 2002 Ford Windstar minivan (Ford Motor Company) around a closed-loop driving track (0.8 KM) for 2 hours. The vehicle was equipped with a dual brake and forward-facing cameras for verification of driving events. A safety observer (W.J.H. or Y.L.), blinded to condition, accompanied the driver in the front passenger seat, and initiated emergency braking procedures if the driver entered a “near-crash” situation. Events, where the vehicle deviated from the lane, were recorded using a forward-facing camera and later verified post-drive by independent assessors blind to condition. These events were respectively categorized as severe (near-crash/emergency braking) and moderate (lane deviation) driving events (See Table 1).

5. Physiological drowsiness events: Eye and eyelid movements were monitored continuously using infrared reflectance oculography (Optalert, Melbourne, Australia). An IR-transducer attached to an open-lens glasses frame, emits, and detects an IR light providing an accurate measure of the opening and closing of the eyelid [21–23] for each blink/eye closure. The Johns drowsiness score (JDS) is a score between 1 and 10 calculated using a proprietary algorithm based on a number of eyelid movements that are sensitive to sleepiness, and is generated every minute throughout the drive. A level of 4.5 is associated with driving “off-road” in a car simulator [22], while a level of 2.6 has been associated with increased risk of an attentional lapse [24] or an out-of-lane driving event [25]. We examined the number of events where the JDS exceeded these established thresholds, and respectively categorized as “severe” and “moderate” drowsiness events (See Table 1).

Electroencephalography (EEG) was continuously monitored during the drive (Vitaport 4, Temec), with electrodes placed down the midline at frontal, central, parietal, and occipital positions (Fz, Cz, Pz, and Oz, respectively). Data were scored for EEG-derived microsleeps defined as activity <8Hz for at least 3 seconds, in any electrode site. These were identified as ‘end state’ fall-asleep events (Table 1).

Data analysis

To describe the effect of the condition on the outcome variables, we used Fishers Exact (emergency braking) or Poisson regression (lane deviation, JDS ≥2.7 and 4.5+ scores, and Microsleep events). Subjective sleepiness ratings were obtained in 15-minute bins. SSQ was analyzed as both a global score and for each item individually. To examine the effect of shift and drive duration on subjective ratings, linear mixed-effects models were used, with condition (post-sleep vs. post-shift) and drive duration (8 × 15-minute bins) included as fixed effects with the interaction term, and participant modeled as a random factor. The covariance structure with the lowest Bayesian Information Criterion was used to interpret the models [26], and a false discovery rate adjustment was used to control for multiple pairwise comparisons (p_adj) [27].

To examine the extent to which subjective ratings predicted an adverse event in the next 15 minutes (main aim), adverse outcomes (emergency braking, lane deviation, JDS ≥2.7 and JDS 4.5+, and EEG-microsleep events) were dichotomized (occurring vs. not occurring in each 15-minute block) and were subject to a binary logistic regression with receiver operating characteristic (ROC) Curve analysis, with the subjective rating as a predictor and dichotomized adverse event as the outcome. Based on a previous study with similar methodology [28] (n = 9/1800 data points), we require 60 data points to predict lane deviations with a medium effect size (OR > 3.47). With 224 observations available for analysis (n = 16 participants × 7 15-minute time bins × 2 conditions), we had >95% power to detect a medium effect.

The Youden’s J index was used to derive the optimal cutoff for each predictor in the ROC analyses, and sensitivity, specificity, and odds ratio for optimal thresholds were reported using binary logistic regression but with a dichotomous predictor (above/below the threshold) and dichotomous outcome (yes/no event). Where the odds ratio was undefined (e.g. a zero in the contingency table), we adjusted the odds ratio using the Haldane-Anscombe correction followed by Fisher's Exact Test of significance [29, 30]. Where there is uncertainty around the true accuracy of the OR (i.e. a high OR with a wide 95% CI), we highlight only ORs that are significant (Fishers Exact) and of moderate effect size (OR > 3.47), and report only the lower limit 95% CI (i.e. providing confidence in observing a medium effect size, but being cautious on interpreting the large effect size). SPSS v27 was used for all statistical analyses.

Missing data

There were eight bins of missing data across the sample due to the drive being terminated early for N = 5 participants during the post-shift drive. As this was due to the instructor deeming the participant too impaired to continue driving, they were not considered missing at random. For N = 1, the first 15-minute subjective assessment of the post-shift drive was not collected, and for N = 2, the seventh 15-minute bin of subjective assessment for the post-sleep drive was lost for all subjective measures. Finally, for N = 1, there were three missing bins of data for the LFA (× 2), and SSQ4 (× 1).

Results

Sixteen shift workers (age 18–65 years) were recruited and completed both the post-sleep and post-night-shift driving conditions. The night shift was, on average (± SD), 8.3 (± 4.1 hours) with at least 5 hours occurring between 22:00 and 08:00. Following the night shift, and relative to the post-sleep condition, participants had less sleep prior to the drive (0.4 [± 1.1 hours] vs. 7.6 [± 2.4 hours]) and had been awake for longer (12.8 [± 4.8 hours] vs. 5.0 [± 1.7 hours]). The time of day was controlled and largely comparable between the two drives (average 1.0 [± 1.2 hours] time difference between the drives). Relative to the post-sleep drive, the post-night-shift drive was associated with more lane deviations (105 vs. 117, p < 0.0001) and severe driving events (0% versus 37.5% of all drives, p < 0.01), as described previously [18]. Relative to driving post-sleep, the night-shift also resulted in an increased number of JDS severe (0.19 vs. 1.14 per 15 minutes, p = 0.003) and moderate impairment scores (1.17 vs. 3.1 per 15 minutes, p = 0.006). There were also more “end state” microsleep events (0.05 vs. 0.21 per 15 minutes [total, 5 vs. 23], p < 0.001). The extent to which drivers were able to identify sleepiness and how subjective ratings of sleepiness predicted these moderate and severe adverse driving outcomes forms the basis of this study.

Subjective sleepiness while driving post-night shift.

Similar changes were evident in self-reported sleepiness and sleepiness symptoms. Compared to driving post-sleep, drivers post-night-shift reported higher KSS (3.4 vs. 5.8, p < 0.001); increased LFA in the next 5 minutes (3.2 vs. 4.4, p < 0.001); increased frequency of sleepiness symptoms (14.0 vs. 25.7, p = 0.001). See Figure 1A–C. These symptoms included struggling to keep the eyes open (p < 0.001), vision becoming blurred (p = 0.007), nodding off to sleep (p = 0.001), difficulty keeping to the middle of the road (p < 0.001), difficulty maintaining the correct speed (p < 0.001), mind wandering to other things (p < 0.001), responses were slowing (p< 0.001), and the head dropping down (p = 0.032). See Supplementary Figure S1. Main effects of drive time were also observed for all subjective variables (p < 0.001, see Figure 1D–F), including individual sleepiness symptoms (p < 0.004, see Supplementary Figure S1), such that the frequency of all sleepiness symptoms increased as a function of drive time. No shift × drive time interactions were observed (p > 0.079), except for difficulty keeping to the middle of the road (p = 0.041) and mind wandering (p = 0.008); each becoming more frequent with drive time for the post-night-shift drive (see Supplementary Figure S1).

Figure 1.

Changes in subjective ratings of sleepiness over the course of the drive following a night with sleep, and a night of shift work. Time is examined in successive 15-minute bins (T1 = 0–15 minutes, T2 = 15–30 minutes…). (A) Karolinska Sleepiness Scale (KSS) scores increased following the night-shift (p < 0.001) and as a function of drive time (p < 0.001); (B) Likelihood of Falling Asleep (LFA) scores changed following the night-shift (p < 0.001) and as a function of drive time; (C) Sleepiness Symptoms Questionnaire (SSQ) global scores increased following the night-shift (p < 0.001) and as a function of drive time. (D–F) Significant effects of time were observed, with most showing an increase with drive time. There was an observed trend for shift × time interaction (p = 0.06).

Subjective sleepiness and prediction of drowsy driving events.

Drives were examined in 15-minute bins, with each bin dichotomized with respect to whether a drowsy driving event occurred or not. As seen in Figure 2, 11 bins were positive for a severe driving event (post-shift = 10 bins), 71 were positive for a moderate (lane deviation) event (post-shift = 40 bins), 74 bins were positive for a severe (JDS 4.5+) drowsiness event (post-shift = 43 bins), 76 for a moderate (JDS 2.7+) drowsiness event (post-shift = 47 bins), and 28 bins were positive for a microsleep event (post-shift = 23 bins). For the post-shift drive, seven participants (44%) had at least one severe (near-crash) event, while 12 participants (75%) exhibited a moderate (lane deviation) driving event. For physiological outcomes, 13 participants (81%) had at least one moderate drowsiness event (JDS 2.7+), 6 (38%) had at least one severe drowsiness event (JDS 4.5+), and 5 (31%) experienced a microsleep. Subsequent analyses sought to examine the extent to which subjective sleepiness predicted the occurrence of an objective sleepiness event in the next 15 minutes (i.e. the subsequent bin).

Figure 2.

Cumulative frequency of drowsiness-related events during the drive following the post-sleep drive and the post-shift drive, for severe driving impairment/emergency braking events (A) moderate driving impairment/lane deviations (B) moderate (JDS ≥ 2.7) and severe (JDS4.5+) ocular-based drowsiness events (C) and end-state fall asleep/microsleep events (D).

Severe driving events (emergency braking).

No severe driving event occurred without noticeable symptoms of sleepiness beforehand. In the 15 minutes prior to all severe driving events, drivers reported a KSS of 7 or greater and a possibility of falling asleep in the next few minutes (50% reported that this was likely or very likely). At least one sleepiness symptom was reported prior to all severe driving events: 90% of severe driving events were preceded by reports of struggling to keep the eyes open and mind wandering, 80% involved reports of the vision becoming blurry and responses slowing, while 70% involved a feeling of nodding off to sleep, difficulty keeping to the middle of the road or maintaining the correct speed. Only 30% of severe driving events involved reports of the head dropping in the prior 15 minutes.

For each point increase in KSS or LFA, there was a respective 2.4 (p = 0.016) and 2.1 (p = 0.009) increased odds of a severe driving event occurring in the next 15 minutes. See Table 2 and Figure 3. All (SSQ) sleepiness symptoms were associated with increasing odds of a severe event occurring in the next 15 minutes, with the exception of head dropping Each one-point increase in the SSQ scale for struggling to keep the eyes open, vision becoming blurred, difficulty keeping to the center of the road, and mind wandering was associated with more than a 2-fold increase in the odds of an impending severe driving event. See Table 2 and Figure 3.

Table 2.

Prediction Accuracy for Subjective Ratings for the Occurrence of Severe/Moderate Drowsy Driving Events in the Next 15 Minutes

Item	Description	OR [95% CI]β	p	AUC [95% CI]	p
Severe driving event: emergency brake
KSS	Subjective sleepiness	2.40 [1.18, 4.91]	0.016	0.90 [0.84, 0.96]	0.001
LFA	Likelihood of falling asleep	2.10 [1.21,3.67]	0.009	0.85 [0.77, 0.92]	0004
SSQ1	Struggle to keep eyes open	2.21 [1.35, 3.62]	0.002	0.91 [0.84, 0.98]	0.001
SSQ2	Vision blurred	2.15 [1.38, 3.36]	0.001	0.91 [0.86, 0.97]	<0.001
SSQ3	Nodding off to sleep	1.76 [1.21, 2.56]	0.003	0.87 [0.80, 0.95]	0.002
SSQ4	Difficulty keeping to middle of road	2.21 [1.39, 3.53]	0.001	0.88 [0.74, 1.00]	0.002
SSQ5	Difficulty maintaining correct speed	1.98 [1.27, 3.10]	0.001	0.81 [0.59, 1.00]	0.009
SSQ6	Mind wandering	2.29 [1.31, 4.01]	0.004	0.86 [0.71, 1.00]	0.002
SSQ7	Reactions are slow	1.79 [1.20, 2.68]	0.004	0.85 [0.75, 0.96]	0.003
SSQ8	Head dropping	—	n.s.	—	n.s.
SSQ global score		1.09 [1.04, 1.16]	0.001	0.88 [0.78, 0.98]	0.002
Moderate driving event: lane deviation
KSS	Subjective sleepiness	1.22 [1.09, 1.36]	<0.001	0.65 [0.57, 0.73]	<0.001
LFA	Likelihood of falling asleep	—	n.s.	—	n.s.
SSQ1	Struggle to keep eyes open	1.17 [1.01, 1.36]	0.039	0.60 [0.51, 0.68]	0.025
SSQ2	Vision blurred	1.24 [1.05, 1.47]	0.013	0.60 [0.51, 0.68]	0.024
SSQ3	Nodding off to sleep	1.21 [1.02, 1.43]	0.029	0.59 [0.51, 0.67]	0.032
SSQ4	Difficulty keeping to middle of road	1.22 [1.02, 1.45]	—	—	n.s.
SSQ5	Difficulty maintaining correct speed	—	—	—	n.s.
SSQ6	Mind wandering	—	—	—	n.s.
SSQ7	Reactions are slow	—	—	—	n.s.
SSQ8	Head dropping	—	—	—	n.s.
SSQ global score		—	n.s.	—	n.s.

Continuous OR—change in odds of a severe/moderate event occurring in the next 15 minutes with each point increase on the predictor scale.

Figure 3.

Predictive capacity of subjective ratings of sleepiness on adverse driving on drowsiness event. Upper Panel—KSS, LFA, and all SSQ items (except head dropping) predict severe driving events (■emergency brake), (A), with reduced predictive capacity for moderate driving events (■lane deviation), (B). Lower Panel - KSS, LFA, and all SSQ items predict severe (■JDS 4.5+), (C) and moderate (■JDS ≥ 2.7), (D) drowsiness events.

Using ROC analyses, ocular-related sleepiness symptoms (struggling to keep eyes open and vision becoming blurred) were the strongest predictors of having a severe driving event in the next 15 minutes (AUC 0.91, p ≤ 0.001, for both). The KSS, LFA, and the sleepiness symptoms of difficulty maintaining lane position, responses being slower, mind wandering to other things, and being aware of having fallen asleep, were also strong predictors of a severe driving event (AUC > 0.85, p < 0.004). In contrast, noticing the head dropping was a poor predictor of a severe driving event (p = 0.21). See Table 2 and Figure 4.

Figure 4.

Receiver operator characteristic (roc) curves for the prediction of severe driving events in the next 15 minutes based on sleepiness symptoms. ROC Outcomes are plotted in increasing severity from perceived changes in driving (driving-related) to perceived fall asleep events (fall asleep-related). Prediction accuracy was high for all behavioral symptoms (AUC > 0.81), except head nodding which was not predictive of severe driving events. Dotted line denotes insignificant.

Moderate driving events (lane deviations).

Increasing KSS was significantly associated with increased odds of a lane deviation in the next 15 minutes (OR 1.22, p < 0.001), although the accuracy of predicting an actual event in the next 15 minutes was only “fair” (AUC 0.65). Increasing LFA was not associated with increased odds of a lane deviation nor did it have any predictive capacity in this respect. For SSQ sleepiness symptoms, difficulty keeping to the center of the road was associated with increased odds in a lane deviation in the next 15 minutes (OR 1.22, p = 0.028), as were ocular sleepiness symptoms (OR 1.17–1.24, p < 0.039) and nodding off to sleep (OR 1.21, p = 0.029). Again, the predictive accuracy of these subjective ratings using ROC analyses was poor (AUC < 0.6), such that the accuracy for predicting an event considered moderately severe (lane deviation) was lower than that observed for severe events (emergency brake) for all subjective ratings. See Table 2, and Figure 5.

Figure 5.

Area under the curve (auc) for the prediction of all events in the next 15 minutes based on sleepiness symptoms. Each panel represents severe (emergency braking or Johns Drowsiness Score (JDS) 4.5+ scores) and moderate (lane deviation, JDS ≥2.7 sores) events, as well as “end-state” fall asleep events. Karolinska Sleepiness Scale (K), Liklihood of Falling Asleep (L) Sleepiness Symptoms Questionnaire Global (G) and individual SSQ items (1–8) are shown. Prediction was more accurate for severe events relative to moderate events, and stronger for physiological/ocular-based drowsiness events (lower panels) relative to the driving events (upper panels).

Subjective sleepiness and prediction of physiological drowsiness while driving

Severe drowsiness events (JDS 4.5+).

78% of severe drowsiness events were preceded by a KSS of at least seven in the prior 15 minutes, while 85% were preceded by a report of possibly falling asleep in the next 5 minutes using the LFA. KSS and LFA were each strong predictors of a severe JDS warning occurring in the next 15 minutes (AUC > 0.80), with each point increase on these scales associated with almost a doubling of odds of a severe drowsiness event occurring in the next 15 minutes (OR 1.76–1.99, p < 0.001). See Table 3 and Figure 3. Many sleepiness symptoms were also strong predictors of severe drowsiness events, with reports of nodding off to sleep being the strongest (AUC = 0.93, p < 0.001), associated with 2.46 times increased odds of a severe drowsiness event occurring in the next 15 minutes with each one-point increase on the SSQ scale. All other sleepiness symptoms were considered very good-to-excellent predictors of severe drowsiness events (AUC 0.85–0.91, p < 0.001). See Table 3 and Figure 5.

Table 3.

Prediction Accuracy for Subjective Ratings for the Occurrence of Severe/Moderate Levels of Physiological Drowsiness Events in the Next 15 Minutes

Item	Description	OR [95% CI]β	p	AUC [95% CI]	p
Severe drowsiness JDS > 4.5
KSS	Subjective sleepiness	1.76 [1.32, 2.33]	<0.001	0.84 [0.75, 0.93]	<0.001
LFA	Likelihood of falling asleep	1.99 [1.42, 2.80]	<0.001	0.80 [0.70, 0.89]	<0.001
SSQ1	Struggle to keep eyes open	2.06 [1.57, 2.71]	<0.001	0.89 [0.84, 0.95]	<0.001
SSQ2	Vision blurred	2.22 [1.66, 2.97]	<0.001	0.90 [0.85, 0.96]	<0.001
SSQ3	Nodding off to sleep	2.46 [1.81, 3.35]	<0.001	0.93 [0.89, 0.98]	<0.001
SSQ4	Difficulty keeping to middle of road	2.31 [1.69, 3.15]	<0.001	0.90 [0.85, 0.95]	<0.001
SSQ5	Difficulty maintaining correct speed	2.67 [1.85, 3.84]	<0.001	0.90 [0.84, 0.96]	<0.001
SSQ6	Mind wandering	2.28 [1.63, 3.20]	<0.001	0.85 [0.74, 0.96]	<0.001
SSQ7	Reactions are slow	2.51 [1.81, 3.48]	<0.001	0.91 [0.85, 0.97]	<0.001
SSQ8	Head dropping	2.81 [1.99, 3.97]	<0.001	0.90 [0.80, 1.00]	<0.001
SSQ global score		1.12 [1.08, 1.17]	<0.001	0.88 [0.77, 0.99]	<0.001
Moderate drowsiness JDS ≥ 2.7
KSS	Subjective sleepiness	1.30 [1.16, 1.46]	<0.001	0.68 [0.61, 0.76]	<0.001
LFA	Likelihood of falling asleep	1.63 [1.30, 2.05]	<0.001	0.66 [0.58, 0.74]	<0.001
SSQ1	Struggle to keep eyes open	1.57 [1.32, 1.87]	<0.001	0.70 [0.62, 0.78]	<0.001
SSQ2	Vision blurred	1.80 [1.45, 2.23]	<0.001	0.72 [0.65, 0.80]	<0.001
SSQ3	Nodding off to sleep	1.81 [1.45, 2.27]	<0.001	0.71 [0.63, 0.78]	<0.001
SSQ4	Difficulty keeping to middle of road	1.79 [1.44, 2.22]	<0.001	0.71 [0.63, 0.79]	<0.001
SSQ5	Difficulty maintaining correct speed	1.64 [1.33, 2.01]	<0.001	0.66 [0.59, 0.75]	<0.001
SSQ6	Mind wandering	1.45 [1.23, 1.71]	<0.001	0.67 [0.60, 0.75]	<0.001
SSQ7	Reactions are slow	1.79 [1.44, 2.23]	<0.001	0.69 [0.61, 0.77]	<0.001
SSQ8	Head dropping	1.78 [1.34, 2.33]	<0.001	0.62 [0.54, 0.70]	<0.001
SSQ global score		1.08 [1.08, 1.12]	<0.001	0.70 [0.62, 0.78]	<0.001
End-state drowsiness (EEG microsleeps)
KSS	Subjective sleepiness	1.24 [1.03, 1.49]	0.026	0.65 [0.50, 0.80]	0.044
LFA	Likelihood of falling asleep	1.63 [1.17, 2.26]	0.004	0.70 [0.56, 0.84]	0.007
SSQ1	Struggle to keep eyes open	1.47 [1.17, 1.84]	<0.001	0.73 [0.60, 0.85]	0.002
SSQ2	Vision blurred	1.29 [1.02, 1.65]	0.041	0.65 [0.51, 0.79]	0.041
SSQ3	Nodding off to sleep	1.30 [1.02, 1.66]	0.038	0.68 [0.54, 0.81]	0.016
SSQ4	Difficulty keeping to middle of road	1.20 [0.91, 1.58]	n.s	0.58 [0.42, 0.73]	n.s
SSQ5	Difficulty maintaining correct speed	1.02 [0.74, 1.41]	n.s	0.52 [0.37, 0.66]	n.s
SSQ6	Mind wandering	1.20 [0.93, 1.55]	n.s	0.63 [0.51, 0.75]	n.s
SSQ7	Reactions are slow	1.14 [0.86, 1.52]	n.s	0.58 [0.43, 0.72]	n.s
SSQ8	Head dropping	0.59 [0.25, 1.36]	n.s	0.42 [0.29, 0.55]	n.s
SSQ global score		1.03 [0.99, 1.07]	n.s	0.64 [0.50, 0.78]	n.s

β, post script note.

Continuous OR—change in odds of a severe/moderate event occurring in the next 15 minutes with each point increase on the predictor scale.

Moderate drowsiness events (JDS ≥2.7).

All subjective measures were associated with increased odds of a moderate drowsiness event occurring in the next 15 minutes. See Table 3 and Figure 3. The accuracy for predicting an individual event was lower for these moderately impaired events (JDS ≥2.7) relative to those considered severe (JDS 4.5+), such that only struggling to keep the eyes open, vision becoming blurred, nodding off to sleep, and difficulty keeping to the middle of the road were considered “good” predictors (AUC > 0.70). See Figure 5.

End-state drowsiness events (EEG-derived microsleep)

As seen in Table 3, LFA and struggling to keep the eyes open were the best predictors of a microsleep occurring in the next 15 minutes, with each one-point increase on the scale associated with approximately 1.5× increased risk of a microsleep occurring. See also Figure 3. The predictive accuracy of these scales were considered “good” (AUC = 0.7–0.73, p < 0.007). Increasing KSS, vision becoming blurred and nodding off to sleep were also associated with increased odds of a microsleep occurring, although the odds ratios were slightly lower (1.24–1.30× increased odds), and with fair accuracy (AUC = 0.65–0.68, p < 0.041). See Figure 5. All other sleepiness symptoms on the SSQ did not significantly predict EEG microsleeps. See Table 3.

Implementation for road safety

Determining a threshold for remedial action.

The Youden’s J statistic indicates the optimal threshold for each subjective rating in predicting each of the five adverse outcomes. See Tables 4 and 5 for prediction of driving and physiological drowsiness outcomes, respectively (and S2 for sensitivity and specificity at all threshold cutoffs). Using Youden’s optimal thresholds, KSS (>7), vision becoming blurred (SSQ>3), and nodding off to sleep (SSQ>2) predicted 100% of severe driving events, (sensitivity = 1.0), with a significant increase in odds of a severe driving event in the next 15 minutes when ratings occurred above this threshold (at least 1.5× using the lower limit of the CI). However, specificity was fair-to-good (0.64–0.77) suggesting a moderately high false positive rate (23%–36%). See Table 4. All other SSQ sleepiness symptoms (except LFA and head dropping) were deemed “very good” predictors of a severe driving event (sensitivity = 0.83) with low false positive rate (<16%). Considering the accurate prediction of events (sensitivity >0.8), low false positive rate (<0.15) and high confidence of a medium effect size (lower limit of the confidence interval > 3.47), the best dichotomous predictors of an imminent severe event were struggling to keep the eyes open, difficulty keeping to the middle of the road, and mind wandering. When reported frequently (e.g. SSQ rating > 5/6), these symptoms were associated with at least a 4-fold increase in severe driving events.

Table 4.

Optimal Thresholds for Predicting Adverse Outcomes and Associated Odds Ratios for Driving Outcomes

Item	Description	Threshold	Sens.	Spec.	OR [95% CI]β	P
		Severe driving event: emergency brake
KSS	Subjective sleepiness	7	1.0	0.75	20.21‡ [2.4, 168.1]β	0.001
LFA	Likelihood of falling asleep	3	0.54	0.75	20.46‡ [2.5, 170.3]β	0.001
SSQ1	Struggle to keep eyes open	6	0.83	0.90	46.00‡ [5.1, 413.5]	<0.001
SSQ2	Vision blurred	3	1.0	0.77	22.18‡ [2.7,184.7]β	<0.001
SSQ3	Nodding off to sleep	2	1.0	0.64	12.23‡ [1.5, 101.3]β	0.006
SSQ4	Difficulty keeping to middle of road	5	0.83	0.90	43.33‡ [4.8, 388.8]	<0.001
SSQ5	Difficulty maintaining correct speed	4	0.83	0.84	25.91‡ [2.9, 229.0]	0.003
SSQ6	Mind wandering	6	0.83	0.89	41.36‡ [4.6, 370.4]	<0.001
SSQ7	Reactions are slow	4	0.83	0.86	29.00‡ [3.3, 257.0]	0.002
SSQ8	Head dropping	3	0.5	0.88	7.16 [1.4, 37.4]	0.02
SSQ global score		34	0.83	0.90	41.59‡ [4.6, 372.4]	<0.001
		Moderate driving event: lane deviation
KSS	Subjective sleepiness	5	0.54	0.69	2.84 [1.6, 5.1]	0.001
LFA	Likelihood of falling asleep	3	0.38	0.78	2.08 [1.1, 3.9]	0.022
SSQ1	Struggle to keep eyes open	2	0.66	0.49	1.99 [1.1, 3.6]	0.025
SSQ2	Vision blurred	2	0.54	0.62	1.95 [1.1, 3.5]	0.024
SSQ3	Nodding off to sleep	2	0.50	0.68	2.11 [1.2, 3.80]	0.013
SSQ4	Difficulty keeping to middle of road	2	0.75	0.44	2.23 [1.2, 4.2]	0.013
SSQ5	Difficulty maintaining correct speed	2	0.63	0.47	n.s	n.s
SSQ6	Mind wandering	3	0.49	0.61	n.s	n.s
SSQ7	Reactions are slow	2	0.60	0.49	n.s	n.s
SSQ8	Head dropping	2	0.35	0.79	2.13 [1.1, 4.0]	0.020
SSQ global score		12	0.75	0.5	3.12 [1.6, 5.9]	<0.001

Dichotomous OR—change in odds of a severe/moderate event occurring in the next 15 minutes when the predictor value is above the identified threshold, compared to when below.

^βOdds ratio calculated with Haldane-Anscombe Correction and significance via Fishers Exact Test.

^‡Odds ratio is inflated due to low n in the false negatives section of the contingency table. Interpretation/reporting of the lower bound 95% confidence interval is recommended.

Table 5.

Optimal Thresholds for Predicting Adverse Outcomes and Associated Odds Ratios for Physiological Drowsiness Outcomes

Item	Description	Threshold	Sens.	Spec.	OR [95% CI]β	p
		Severe drowsiness event: JDS > 4.5
KSS	Subjective sleepiness	7	0.82	0.78	15.80‡ [4.3, 54.5]	<0.001
LFA	Likelihood of falling asleep	3	0.70	0.76	7.89‡ [2.6, 23.6]	<0.001
SSQ1	Struggle to keep eyes open	3	0.94	0.78	40.15‡ [5.2, 310.1]	<0.001
SSQ2	Vision blurred	4	0.76	0.88	25.26‡ [7.6, 84.3]	<0.001
SSQ3	Nodding off to sleep	3	0.88	0.84	37.73‡ [8.2, 173.1]	<0.001
SSQ4	Difficulty keeping to middle of road	3	0.94	0.72	43.58‡ [5.7, 334.2]β	<0.001
SSQ5	Difficulty maintaining correct speed	3	0.94	0.72	40.15‡ [5.2, 310.1]	<0.001
SSQ6	Mind wandering	6	0.81	0.93	33.23‡ [10.2, 108.7]	<0.001
SSQ7	reactions are slow	3	0.88	0.78	26.96‡ [5.9, 122.5]	<0.001
SSQ8	Head dropping	3	0.76	0.93	41.55‡ [12.0, 144.4]	<0.001
SSQ global score		24	0.88	0.86	44.20‡ [9.6, 203.9]	<0.001
		Moderate drowsiness event: JDS > 2.7
KSS	Subjective sleepiness	5	0.59	0.71	3.69 [2.0, 6.7]	<0.001
LFA	Likelihood of falling asleep	3	0.47	0.83	4.49 [2.6, 8.6]	<0.001
SSQ1	Struggle to keep eyes open	3	0.55	0.79	4.65 [2.5, 8.6]	<0.001
SSQ2	Vision blurred	2	0.66	0.70	4.75 [2.6, 8.7]	<0.001
SSQ3	Nodding off to sleep	2	0.61	0.74	4.53 [2.5, 8.3]	<0.001
SSQ4	Difficulty keeping to middle of road	3	0.55	0.77	4.27 [2.3, 7.8]	<0.001
SSQ5	Difficulty maintaining correct speed	2	0.69	0.52	2.44 [1.3, 4.4]	0.003
SSQ6	Mind wandering	3	0.59	0.68	3.10 [1.7, 5.6]	<0.001
SSQ7	Reactions are slow	3	0.50	0.86	6.27 [2.2, 12.2]	<0.001
SSQ8	Head dropping	3	0.28	0.95	8.36 [3.2, 21.9]	<0.001
SSQ global score		23	0.46	0.92	10.94 [5.0, 24.0]	<0.001
		End-state fall asleep event: EEG microsleep
KSS	Subjective sleepiness	6	0.59	0.69	3.19 [1.16, 8.78]	0.029
LFA	Likelihood of falling asleep	3	0.65	0.76	6.17 [2.16, 17.62]	<.001
SSQ1	Struggle to keep eyes open	4	0.53	0.82	5.11 [1.84, 14.18]	0.002
SSQ2	Vision blurred	3	0.53	0.78	3.84 [1.40, 10.53]	0.015
SSQ3	Nodding off to sleep	2	0.71	0.64	4.25 [1.44, 12.57]	0.008
SSQ4	Difficulty keeping to middle of road	3	0.53	0.67	n.s	n.s
SSQ5	Difficulty maintaining correct speed	5	0.18	0.89	n.s	n.s
SSQ6	Mind wandering	4	0.53	0.72	n.s	n.s
SSQ7	Reactions are slow	3	0.42	0.74	n.s	n.s
SSQ8	Head dropping	7	0.00	0.98	n.s	n.s
SSQ global score		16	0.71	0.63	4.15 [1.40, 12.27]	0.009

Dichotomous OR—change in odds of a severe/moderate event occurring in the next 15 minutes when the predictor value is above the identified threshold, compared to when below.

^βOdds ratio calculated with Haldane-Anscombe Correction and significance via Fishers Exact Test.

^‡Odds ratio is inflated due to low n in the false negatives section of the contingency table. Interpretation/reporting of the lower bound 95% confidence interval is recommended

Taking a similar approach, no sleepiness symptom when used above/below a threshold met an acceptable level for predicting a lane deviation event. For instance, while occasionally reporting difficulty keeping to the middle of the road/struggling to keep the eyes open was “fair-to-good” at predicting a moderate driving event occurring in the next 15 minutes, the odds ratio reflected a small effect size (~2-fold) and the false positive rate was poor (25%–34%). Similarly, although feeling neither alert nor sleepy (KSS = 5) was associated with 2.8× greater odds of a lane deviation, sensitivity was low (54%) and the false positive rate high (31%). See Table 4.

All subjective ratings dichotomized according to Youden’s J optimal threshold predicted severe JDS scores in the next 15 minutes. See Table 5. Balancing sensitivity (>0.8), with low false rate (<15%) and the OR, the best predictor of a severe ocular drowsiness event was mind wandering. Here, “frequent” mind wandering (SSQ6 > 6) was associated with at least a 10-fold increase in the odds of an imminent severe drowsiness event (95% CI lower limit: 10.2, Fishers Exact p < 0.001), with 81% of events detected with a 7% false positive rate.

In relation to road safety, sensitivity is paramount, and other notable predictors were identified. See Table 5. Drivers who reported struggling to keep their eyes open more than “occasionally” (SSQ1 >3), experienced at least a 5-fold increase in odds of an imminent severe (JDS 4.5+) drowsiness event (95% CI lower limit: 5.2, Fishers Exact p < 0.001). Here, 94% of severe drowsiness events were detected by the driver, with a 22% false positive rate. “Occasional” difficulty keeping to the center of the road or maintaining the correct speed were also strong predictors of severe ocular events (sensitivity = 0.94, specificity =0.72), associated with at least a 5-fold increase in odds of a severe drowsiness event. While reporting an “occasional” experience of “nodding off” was associated with at least an 8-fold increase in odds of a severe event, sensitivity (= 0.88) was slightly lower.

Although moderate drowsiness events (JDS > 2.7) were significantly predicted by all subjective measures, sensitivity was low (<0.66), and false positive rates varied from “excellent” to “fair” (5% to 32%). See Table 5.

End-state fall-asleep events (micro sleep) were only significantly associated with those symptoms relating to sleepiness/falling asleep (i.e. not driving performance). For instance, reporting a KSS of 6 or more was associated with a 3-fold increase in an end-state drowsiness event (micro sleep). Drivers who report “possibly” falling asleep in the next five minutes (LFA >3) had a 6-fold increase in odds of actually falling asleep in the next 15 minutes (with “fair” sensitivity), while those who report occasionally nodding off to sleep (SSQ 3 >2) had 4× greater odds of a microsleep event (with “good” sensitivity). Finally, the ocular symptoms (struggling to keep the eyes open, vision becoming blurred) were associated with a 3-5-fold increase in odds of a microsleep event when reported “occasionally,” although sensitivity was “poor.” See Table 5.

Discussion

This study provides real on-road evidence that drivers are aware of sleepiness, and that sleepiness symptoms can accurately predict subsequent sleep-related adverse outcomes while driving, particularly those considered severe, such as near-crash events. While we previously described how drivers had a high risk of near-crash driving events when driving after a night shift [18], we now build on those data to show that these drivers were aware of sleepiness while driving, and that their sleepiness ratings accurately predict severe levels of driving impairment and physiological drowsiness. Of note, reflecting on specific symptoms appears important, particularly those relating to the eyes (struggling to keep the eyes open, vision becoming blurred). Although drivers were able to predict earlier (moderate) warning signs of driving impairment using these symptoms, this was less accurate than the prediction of severe (near-crash, JDS 4.5+) events. Predicting the occurrence of microsleeps and lane deviations based on subjective sleepiness alone exhibited the lowest predictive validity.

Severe driving events were recorded whenever a safety observer/qualified instructor initiated an emergency brake maneuver in response to an impending near-crash event, while severe drowsiness events were defined using a JDS score previously associated with the car leaving the lane in simulated driving conditions (JDS4.5+) [22]. All subjective ratings (KSS, LFA, and sleepiness symptoms) predicted a severe driving or drowsiness event, with the exception of the head dropping down. Falling asleep at the wheel represents the end-state in drowsiness-related crashes, with head dropping considered the most extreme of the sleepiness symptoms we evaluated, and likely associated with a stage 1 sleep episode (i.e. >30 seconds of EEG-defined sleep). As head dropping down had the lowest frequency rating of all symptoms (Supplementary Figure S1), our findings may suggest that (1) drivers were less aware of the presence of head dropping/falling asleep, as suggested by Kaplan et al. [12], and/or (2) adverse driving events occurred prior to this end-state and thus fewer reports were observed, i.e. the emergency brake may be applied due to any event compromising safety, including other sleep-related consequences such as diverted attention/distraction [31]. With this in mind, our results indicate that reports of the head dropping down should not be used as an early warning of sleepiness, and driving should cease before this occurs, i.e. when other earlier sleepiness symptoms are experienced.

Although a fall-asleep event is considered end-state in the trajectory of drowsiness, subjective ratings of sleepiness were less accurate at detecting subsequent microsleep events, relative to adverse driving outcomes and ocular-based drowsiness outcomes. Struggling to keep the eyes open and LFA were the strongest predictors of an impending microsleep event. Given that these two symptoms more closely match the fall-asleep state further supports the coupling between subjective and objective sleepiness, i.e. driving/behavioral symptoms such as keeping to the center of the road, were not significant predictors of impending fall-asleep events. Our data are thus consistent with the suggestion that drivers are less able to tell when they are close to falling asleep [12], and we urge public education campaigns to focus on earlier warning signs of sleepiness rather than end-state drowsiness.

Predicting impairment at an earlier point in its trajectory allows time for drivers to find a safe space to stop and take remedial action. Each point increases in subjective sleepiness (KSS), struggling to keep the eyes open, vision blurred, nodding off to sleep, and difficulty keeping to the center of the road were associated with a small 17%–24% increase in the odds of lane deviation occurring in the next 15 minutes, and a slightly larger 30%–71% increase in an ocular event associated with inattention (JDS≥2.7) occurring. ROC analyses confirmed these symptoms to be “good” predictors of ocular-based drowsiness outcomes, yet only “fair” for lane deviation outcomes. While the association between KSS and lane deviations is consistent with a previous naturalistic study (i.e. significant but small effect size) [32], we ask why subjective ratings were less accurate at detecting lane deviations, relative to all other measures (near-crash events, and severe-to-moderate ocular defined drowsiness). First, drivers may be unable to predict a lesser degree of impairment. We suggest this is unlikely however as subjective sleepiness did predict a lesser degree of physiological drowsiness (JDS ≥2.7), which is associated with increased risk of an attentional lapse [24] or an out-of-lane driving event [25]. Second, lane deviations may be a less sensitive indicator of sleep-related impairment due to a large source of variance in what causes a departure from the laneway. Indeed, associations between KSS and lane deviations are (a) stronger in simulator studies— where the roadway is more controlled—relative to “noisy” track/naturalistic settings [14]; (b) lower relative to other outcome measures within the same study design [33, 34] and/or meta-analyses [14]; and (c) are strengthened when lane deviations are cleaned to include only unintentional events [35]. Future studies using lane deviations should therefore use only unintentional lane deviations (and ideally confirmed by driver-facing footage as being sleep-related) or utilize a higher threshold of impairment beyond a single lane deviation (e.g. rate/minute).

Previous research suggests that many drivers continue to drive despite feeling sleepy [15], because those feelings are not recognized as serious or specific enough to cease driving [16]. Despite this, asking a more specific question about the LFA was not more accurate in predicting safety-critical events in our study. Assessing “earlier” specific sleepiness symptoms may be better in this respect. Ocular-based symptoms (struggle to keep eyes open, vision becoming blurred) and nodding off to sleep were consistently significant predictors of all adverse outcomes, including a near-crash event, a lane deviation, and all physiological indices of drowsiness. Our work and others have shown that the eyes provide a strong signal for assessing alertness/drowsiness using objective measurement of eye and eyelid closure (e.g. [23, 24, 36–38]), and our data confirm that this extends to subjective ratings of ocular changes, where “occasionally” struggling to keep the eyes open or vision becoming blurred was associated with a 2-fold increase in the odds of a lane deviation in the next 15 minutes and an almost 5-fold increase in a moderate physiological drowsiness event (JDS≥2.7). Continuing to drive such that these feelings occur “frequently” substantially increased the odds of a near-crash event or end-state drowsiness (micro sleep). Our data thus suggest that “struggling to keep the eyes open” or “vision becoming blurred” are strong warning signs that drivers should stop driving. This confirms experimental driving studies (simulator and on-road) showing that eye symptoms are strongly associated with sleep loss and adverse driving outcomes [17, 39], and epidemiological data whereby difficulty keeping the eyes open was retrospectively the most commonly reported symptom (by 55% of drivers) prior to falling asleep at the wheel [16]. Based on our data, ceasing driving when struggling to keep the eyes open, even “occasionally” (rather than waiting for this to occur more frequently) could result in 83% of near-crashes being avoided, and 94% of severe physiological drowsiness events (with only 10%–20% false positive signals).

Notwithstanding the ideal scenario to avoid driving impairment entirely, noticing early signs of impairment may be useful to avoid extreme safety-critical events. Difficulty keeping to the center of the road and maintaining the correct speed each showed high accuracy in predicting a near-crash event (AUC: 0.88–0.81, respectively). However, while difficulty keeping to the center of the road was associated with a small increase in the odds of a lane deviation event, the capacity to accurately predict one of these events was limited. To date, there is a paucity of knowledge on the extent to which drivers (or non-drivers) are able to accurately reflect on their own performance, and how this might change under conditions of sleep loss [40]. While Philip et al. reported that correlations between self-reported performance and actual driving performance were poor under conditions of sleep loss [32], performance monitoring was general, rather than specific. In contrast, and in line with our findings here, in our previous study of professional drivers undertaking a simulated drive following a night without sleep, difficulty maintaining speed and/or lane position was associated with actually increased variability in lane position and speed, and these symptoms had high accuracy for predicting severe driving impairment [17].

To support future recommendations of when to stop driving, we sought to develop thresholds of subjective ratings which predicted “early” (e.g. a single lane deviation) versus late (e.g. a near-crash event) signs of drowsiness. Reporting any signs of sleepiness (KSS of 6 or more) was associated with 3.5× increased odds of a lane deviation in the next 15 minutes, with 54% of adverse events detected, and a false positive rate of around 30%. Continuing to drive, such that the KSS increased to “sleepy, fighting sleep” (KSS ≥8), drastically increased the odds of a near-crash event, with a false positive rate of around 25% (although we note that the reported odds ratio is likely inflated (20.21) and thus we report the lower CI 95% of 2.43—see below). As drivers may not routinely employ a traditional KSS, education campaigns may wish to employ these two KSS descriptors as an increase in risk trajectory, that is, “some signs of sleepiness” being an early warning, with “sleepy, fighting sleep” suggesting more imminent danger.

We also report on the utility of specific symptoms, and advise drivers that “occasionally” reporting symptoms appear to better reflect earlier/moderate impairment (SSQ > 2/3), while “frequently” reporting them appear to reflect late/severe impairment (SSQ >4). We would therefore urge drivers to seek somewhere to stop and rest when feeling any signs of sleepiness, and occasional signs of struggling to keep the eyes open, blurred vision, difficulty keeping to the center of the road, and nodding off to sleep. Once drivers feel they are “fighting sleep,” likely to fall asleep in the next five minutes, and/or reporting a “frequent” occurrence of these symptoms (in addition to mind wandering), driving should cease as soon (and as safe) as possible. With these recommendations in mind, future work should examine the accuracy of these predictors when presented as a dichotomous assessment in an independent test set comprising a large number of drivers, in ecologically valid (naturalistic) settings.

Our study findings should be interpreted with some considerations in mind. First, some of the analyses undertaken are affected by a small sample size in observable events. Although our overall study sample (n = 16) was adequate (i.e. a recent systematic review of this topic area [awareness of sleepiness] demonstrated this sample size was the most common sample size to address this question, plus we had 95% power to detect a medium effect size), the reduced number of observations particularly for the threshold analyses (Tables 4 and 5) led to an OR which was imprecise (as indicated by the width of 95% confidence interval). We, therefore, took a cautious approach by reporting on those odds ratios where we had confidence of a medium effect size in predictive accuracy. Future work should expand the n and/or number of observations to provide more accurate estimates for easy-to-implement thresholds of self-reported sleepiness. Second, while we demonstrate high accuracy between sleepiness/sleepiness symptoms and adverse driving outcomes, inter-individual differences may be observed. For instance, previous studies, from our group and others, show individual variation in correlations between actual performance and subjective ratings of performance and sleepiness [41, 42]. These individual differences may be systematic, such that predictive accuracy may change as a function of age or sex of the driver. While our study did include male and female drivers, and with a wide age range (19–65 years), we were not powered to specifically examine this and further work is required [14]. Third, we examined the association between subjective sleepiness/sleepiness symptoms and driving following one night without sleep, and this may be different for other sleep conditions, e.g. sleep restriction, chronic sleep loss, extended duration work shifts. Additionally, our drive began 2 hours post night shift (due to transportation to the track/study set up), and thus may not directly apply to night shift workers driving home (although we have previously shown KSS immediately post-shift predicts subsequent self-reported driving impairment [6, 9]). While the association between subjective sleepiness and objective performance becomes more tightly coupled as sleep loss progresses (due to a stronger sleep signal) [41], the association is proposed to uncouple with chronic sleep loss [43]. Future work should therefore examine drivers’ awareness of sleepiness as an accurate predictor of driving impairment under a range of real-world conditions of sleep loss, including different sleep schedules, varied use of countermeasures, such as caffeine (noting that we controlled for caffeine to specifically examine the effect of shift work per se), and for different drive durations (we note that severe events accumulate relatively late in the drive compared to moderate events, which may underpin the strong association with increasing subjective sleepiness).

To summarize, we demonstrate that drivers are aware of sleepiness, and that subjective sleepiness and sleepiness symptoms predict adverse drowsiness events occurring in the next 15 minutes, particularly those events considered severe (near-crash event, JDS4.5+). The overall best subjective predictors included the KSS, self-reports relating to eye symptoms (struggling to keep the eyes open, vision becoming blurred) and those relating to driving performance (particularly difficulty staying in the middle of the road). We note that the subjective ratings that best match the objective outcome appear to be the most ideal (e.g. subjective ocular predicting objective ocular [JDS], subjective driver behavior predicting actual driver behavior, and LFA/nodding off predicting microsleep events). A task force led by the National Highway Traffic Safety Administration (NHSTA) identified a comprehensive set of strategies to end drowsy driving crashes and related serious injuries and deaths. Public awareness, behavior and education were identified as key factors [44], and our data are critical for informing these interventions. We recommend that drivers should routinely assess how sleepy they feel, and any associated sleepiness symptoms. Drivers should find a safe place to stop and take remedial action if they report (1) any signs of sleepiness (nonspecific, KSS greater than 6), and (2) any “occasional” experience of the following (specific) symptoms: struggling to keep the eyes open, blurred vision, nodding off, and/or difficulty keeping to the center of the road/maintaining the correct speed. Waiting for these signs to increase to “sleepy, fighting sleep” or “frequent” sleepiness symptoms poses a heightened risk of serious injury or fatality to the driver and other road users due to drowsiness, and should be avoided.

Funding

This study was supported by a grant from the Institute of Breathing and Sleep Research (to M.E.H.); by Liberty Mutual Insurance; National Institutes of Health Award 5T32HL7901-14 (to M.L.L.); National Space Biomedical Research Institute Award PF03002 (to M.L.L.); Department of Homeland Security Federal Emergency Management Agency Assistance to Firefighter Grant EMW-2010-FP-00521 (to C.A.C.); National Heart, Lung and Blood Institute Cooperative Agreement U01-HL111478 (to C.A.C.); National Institute of Occupational Safety and Health Grant R01-OH0103001 (to C.A.C.); National Institute on Aging Grant R01-AG044416 (to C.A.C.); and an endowed professorship provided to Harvard Medical School by Cephalon, Inc. (to C.A.C.). The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the Federal Emergency Management Agency, Assistance to Firefighters Grant Program, National Institutes of Health, National Space Biomedical Research Institute, Institute of Breathing and Sleep, or Liberty Mutual Insurance Company. The federal sponsors did not have a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.

Disclosure Statement

Financial disclosure: The authors report no conflicts of interests related to the results reported in this paper. In the interest of full financial disclosure, they report: CA has received a research award/prize from Sanofi-Aventis; contract research support from VicRoads, Transport Accident Commission, Rio Tinto Coal Australia, National Transport Commission, Tontine/Pacific Brands, and the Australian Automobile Association; industry funding through ARC Linkage scheme with Seeing Machines and Cogstate Ltd; lecturing fees from Brown Medical School/Rhode Island Hospital, Ausmed, Healthmed and TEVA Pharmaceuticals; and reimbursements for conference travel expenses from Philips Healthcare. In addition, she has served as a consultant to the Rail, Bus and Tram Union, the Transport Accident Commission (TAC), the National Transportation Committee (NTC), VicRoads, and Melius Consulting. CAC reports grants and contracts to BWH from Dayzz Live Well, Delta Airlines, Jazz Pharma, Puget Sound Pilots, Regeneron Pharmaceuticals/Sanofi; is/was paid consultant/speaker for Inselspital Bern, Institute of Digital Media and Child Development, Klarman Family Foundation, M. Davis and Co, National Council for Mental Well-being, National Sleep Foundation, Physician’s Seal, SRS Foundation, State of Washington Board of Pilotage Commissioners, Tencent, Teva Pharma Australia, With Deep and Vanda Pharmaceuticals, in which CAC holds an equity interest; received travel support from Aspen Brain Institute, Bloomage International Investment Group, Inc., Dr. Stanley Ho Medical Development Foundation, German National Academy of Sciences, Ludwig-Maximilians-Universität München, National Highway Transportation Safety Administration, National Safety Council, National Sleep Foundation, Salk Institute for Biological Studies/Fondation Ipsen, Society for Research on Biological Rhythms, Stanford Medical School Alumni Association, Tencent Holdings, Ltd, and Vanda Pharmaceuticals; receives research/education gifts through BWH from Arbor Pharmaceuticals, Avadel Pharmaceuticals, Bryte, Alexandra Drane, Cephalon, DR Capital Ltd, Eisai, Harmony Biosciences, Jazz Pharmaceuticals, Johnson & Johnson, Mary Ann & Stanley Snider via Combined Jewish Philanthropies, NeuroCare, Inc., Optum, Philips Respironics, Regeneron, Regional Home Care, ResMed, Resnick Foundation (The Wonderful Company), San Francisco Bar Pilots, Sanofi SA, Schneider, Simmons, Sleep Cycle AB. Sleep Number, Sysco, Teva Pharmaceuticals, Vanda Pharmaceuticals; receives royalties from the New England Journal of Medicine; McGraw Hill; Houghton Mifflin Harcourt/Penguin; and Philips Respironics, Inc. for the Actiwatch-2 and Actiwatch-Spectrum devices. MH has received research funding from the Cooperative Research Center for Alertness Safety and Productivity, contract research support from Vicroads, Shell International, Rio Tinto and equipment support for research from Optalert and Seeing Machines. He has served as a consultant for Vicroads, the National Transport Commission, Victoria Police and Bus Safety Victoria and received lecturing fees from TEVA Pharmaceuticals, Biogen and Astra-Zeneca. Nonfinancial disclosure: The authors report no conflicts of interests related to the results reported in this paper. In the interest of full nonfinancial disclosure, they report: CA has served as an expert witness and/or consultant in relation to fatigue and drowsy driving, and was a Theme Leader in the Cooperative Research Center for Alertness, Safety and Productivity. CAC is/was an expert witness in legal cases, including those involving Advanced Power Technologies, Aegis Chemical Solutions, Amtrak; Casper Sleep Inc, C&J Energy Services, Catapult Energy Services Group, Covenant Testing Technologies, Crete Carrier Corporation, Dallas Police Association, Enterprise Rent-A-Car, Espinal Trucking/Eagle Transport Group/Steel Warehouse Inc, FedEx, Greyhound, Pomerado Hospital/Palomar Health District, PAR Electrical Contractors, Product & Logistics Services LLC/Schlumberger Technology, Puckett EMS, Puget Sound Pilots, Top Run Energy Services Union Pacific Railroad, UPS, and Vanda Pharmaceuticals; CAC also serves as the incumbent of an endowed professorship given to Harvard by Cephalon. CAC’s interests were reviewed and managed by Brigham and Women’s Hospital and Mass General Brigham in accordance with their conflict of interest policies.

Author Contributions

All authors have made substantial contributions to the work presented and have approved the final version of the manuscript. CA, WH, CAC, and MH designed the study with input from YL and COB. CA, AC, CAC, and MH developed the research questions, and M.L.L., WH, YL, COB, and MH were responsible for the collection of data. CA and AC analyzed and interpreted the data. CA wrote the manuscript with edits and approvals from all authors.