Assessment of Drowsiness Based on Ocular Parameters Detected by Infrared Reflectance Oculography

Abstract

Study Objectives:

Numerous ocular parameters have been proposed as reliable physiological markers of drowsiness. A device that measures many of these parameters and then combines them into a single metric (the Johns Drowsiness Scale [JDS]) is being used commercially to assess drowsiness in professional drivers. Here, we examine how these parameters reflect changes in drowsiness, and how they relate to objective and subjective indices of the drowsy state in a controlled laboratory setting.

Design:

A within subject prospective study.

Participants:

29 healthy adults (18 males; mean age 23.3 ± 4.6 years; range 18-34 years)

Interventions:

N/A.

Measurements and Results:

Over the course of a 30-h extended wake vigil under constant routine (CR) conditions, participants were monitored using infrared reflectance oculography (Optalert) and completed bi-hourly neurobehavioral tests, including the Karolinska Sleepiness Scale (KSS) and Psychomotor Vigilance Task (PVT). Ocular-defined increases in drowsiness were evident with extended time awake and during the biological night for all ocular parameters; JDS being the most sensitive marker of drowsiness induced by sleep regulatory processes (p < 0.0001). In addition, the associations between JDS in the preceding 10-min period and subsequent PVT lapses and KSS were stronger (AUC 0.74/0.80, respectively) than any other ocular metric, such that PVT lapses, mean response time (RT), and KSS increased in a dose-response manner as a function of prior JDS score (p < 0.0001).

Conclusions:

Ocular parameters captured by infrared reflectance oculography detected fluctuations in drowsiness due to time awake and during the biological night. The JDS outcome was the strongest predictor of drowsiness among those tested, and showed a clear association to objective and subjective measures of drowsiness. Our findings indicate this real-time objective drowsiness monitoring system is an effective tool for monitoring changes in alertness and performance along the alert-drowsy continuum in a controlled laboratory setting.

Citation:

Anderson C; Chang AM; Sullivan JP; Ronda JM; Czeisler CA. Assessment of drowsiness based on ocular parameters detected by infrared reflectance oculography. J Clin Sleep Med 2013;9(9):907-920.

Drowsiness is an unstable, rapidly fluctuating state of reduced alertness along a sleep-wake continuum that can result in large variability in performance impairment over relatively short intervals of time. Despite efforts to manage drowsiness, it is often an inevitable consequence of modern society, such that drowsiness is a major contributory factor in ∼20% of motor vehicle crashes^1–3 and poses a significant risk to occupational health and safety, especially for those engaged in shift work.⁴ The potential risk of occupational accident or injury due to drowsiness is largely determined by the fundamental properties that control alertness. These include (1) the number of hours awake⁵; (2) nightly sleep duration where repeated nights of inadequate sleep (sleep restriction) result in gradual decrements in alertness and performance⁶; (3) certain times of day characterized as high or low alertness due to activity of the intrinsic body clock⁷; (4) impairment of alertness soon after waking⁸; and (5) the presence of a sleep disorder.⁹ In order to quantify fluctuations in alertness and drowsiness due to these primary physiological factors, a wide number of neurocognitive and physiological measures have been employed.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Ocular parameters are known to be sensitive to changes in drowsiness, yet current methods are time consuming, invasive, and require extensive offline processing. This study evaluates a non-invasive device based on infrared oculography to detect changes in drowsiness.

Study Impact: Our study provides support for a new approach to monitoring drowsiness in the laboratory environment. The findings presented here have implications for future studies which seek to evaluate changes in drowsiness using a non-invasive method, with minimal off-line processing, and clear thresholds for drowsiness.

Numerous measures shown to depict reliably the developing drowsy state include changes in human electroencephalographic (EEG) output,^10,11 electrooculography (EOG) or other oculographic measurements,^7,12,13 heart rate variability,^14–16 and neurobehavioral measures of attention and vigilance (i.e., psychomotor vigilance task).^6,17,18 While these measurements are sensitive to drowsiness, they can present numerous problems: (1) most measures require substantial off-line processing, especially EEG/EOG; (2) performance measures present a snapshot of performance decrement and are unable to provide a continuous marker of dynamic drowsiness; (3) many require quiet, sterile environments to reduced potential confounds from noise or distraction¹⁹; (4) no well-defined cutoff or threshold for impairment is available for many of these outputs; and (5) while EOG is typically recorded in laboratories, often with the sole purpose of capturing REMs for the purpose of scoring sleep, these do not capture the developing drowsy state but rather a sleep initiation state; and while laboratory-based EOG is capable of distinguishing a blink from a saccade or slow eye movement,⁷ it is not able to depict reliably the drowsy state beyond gross slow eye movements.

Eye blinks and eye movements (“oculometrics”) are inextricably linked to central nervous system function and provide a unique window on general activity within the midbrain reticular formation, which is well known to play a central role in vigilance and attention.²⁰ Various types of eye and eyelid movements have previously been associated with changes in the drowsy state, including: percentage or duration of eye closure,^21–23 blink duration,^{12,22,24–27} blink rate^12,28 or blink amplitude,²⁹ pendular motions,³⁰ slow eye movements,^7,31 lid closing/reopening time,^22,32,33 interval between blinks,³⁴ changes in pupil size,³⁵ saccadic velocities,^36–38 and (more recently) amplitude-velocity ratios of the eye closure.^32,33,39 While the study of ocular parameters to reflect a fluctuating state of alertness is not new,^30,40 recent interest has intensified culminating in development of numerous drowsiness detection technologies,⁴¹ such as PERCLOS²³ and Optalert.^32,33 PERCLOS, a measure of the proportion of time the eyes are closed > 80%, has previously been shown to relate to changes in vigilance, as determined by the psychomotor vigilance task (PVT).^16,42,43 While these systems have been developed to identify and warn of drowsiness in the field, they may provide a useful marker of drowsiness in the laboratory without the associated problems with many current approaches.

One of these commercially available drowsiness detection systems is of particular interest, as it enables several oculomotor parameters to be continually measured via noninvasive, infrared reflectance oculography (Optalert, Melbourne, Australia). A weighted combination of these ocular parameters provides a single, sensitive measure of dynamic alterations in drowsiness levels, the Johns Drowsiness Scale (JDS). The JDS is proposed to be a more sensitive marker of drowsiness and has previously been used to depict drowsy driving,^32,33 vigilance and attention,^32,33,44 changes in alertness following caffeine ingestion,⁴⁵ and changes in EEG-defined drowsiness.⁴⁶ Currently, no independent prospective study exists that examines the extent to which the ocular parameters detected by this system, including the JDS, reflect dynamic changes in alertness and drowsiness over time due to fluctuating levels of drowsiness associated with sleep regulatory processes (i.e., time awake and circadian phase). In addition, the extent to which ocular parameters detected by infrared reflectance oculography relates to established laboratory-based measures of drowsiness is unknown. Examining changes in ocular parameters via infrared reflectance oculography due to changes in sleep drive, and how these outcomes relate to existing laboratory-based objective and subjective measures of drowsiness forms the basis of our study.

METHODOLOGY

Participants

Twenty-nine healthy participants (18 males; mean age 23.3 ± 4.6 years; range 18-34 years) were studied in the Center for Clinical Investigation of Brigham and Women's Hospital. The study was approved by the Partners Human Research Committee at Brigham and Women's Hospital, and subjects provided full written informed consent prior to study. All participants underwent comprehensive physical, psychological, and ophthalmological examinations prior to study and were free from any medical, psychiatric, or sleep disorder. Participants engaged in night and/or shift work in the previous 3 years, and those who traveled across more than 1 time zone in the prior 3 months were excluded from the study. For 3 weeks prior to study, participants were instructed to maintain a consistent, self-selected 16-h wake/8-h sleep schedule as confirmed by time- and date-stamped calls at bedtimes and wake times, in addition to actigraphy (Actiwatch-L, Minimitter, Inc, Bend, OR) 7-14 days prior to admission to the study. During this time, participants were asked to refrain from use of any prescription or non-prescription medication, supplements, recreational drugs, nicotine, caffeine, and alcohol. Compliance with this requirement was verified by urine and blood toxicology during screening and by urine toxicology prior to admission to the study.

Study Protocol

The data described in this manuscript were part of a larger 8-day inpatient study protocol.⁴⁷ Participants were studied for 8 days in an environment free of obvious time cues (no windows, clocks, radio, live TV, newspapers), and continually monitored by staff trained not to reveal the time of day. The protocol consisted of: (i) one baseline day/night (16:8 wake-sleep), followed by (ii) 5 days of timed light exposure designed to phase advance the sleep episode by 8 h while maintaining 8-h sleep episodes; followed by (iii) a 30-h constant routine and final 8-h recovery sleep episode. The 5-day phase advance segment of the protocol may have been 1 of 3 different timing regimes and 1 of 2 lighting conditions (this prior exposure had no effect on the data presented here-see results). Following the 5-day advance protocol, participants underwent a 30-h extended wake episode under constant routine conditions. During that time, participants were asked to remain awake in dim light (3 lux), in a semi-recumbent position (head of the bed at 45°), with the equivalent daily fluid and caloric intake provided in small hourly snacks. During the 30-h constant routine, participants were monitored with infrared reflectance oculography (Optalert) starting 1 h after waking on the CR (sleep inertia testing occurred post wake for 1 h). Infrared reflectance oculography was monitored continuously except during PVT tests when a head-mounted eye tracker was being used. Participants were instructed to remain looking ahead. See supplemental material for schematic example of study protocol.

Infrared Reflectance Oculography

The eye and eye lid movements were monitored using infrared reflectance oculography whereby IR transducers attached to the frame of a pair of special glasses/spectacles detects ocular movement (Optalert Drowsiness Measurement System, Sleep Diagnostics Pty Ltd, Melbourne, Australia; described in full elsewhere³³). The system provides a measure of various components of eyelid movements during spontaneous blinks, measured each minute. These include: the mean duration of the eyelids remaining closed during blinks, the mean interval between the times of maximum closing and reopening velocities of the eyelids during blinks, the number of long eyelid closures per minute, the total % time of long eyelid closures per minute, the mean relative velocity of the eyelids closing during blinks (assessed by the amplitude/velocity ratio [AVR] of each such movement), and the mean and standard deviation of the relative velocity (AVRs) of the eyelids reopening during blinks.

Typically, an eye blink is approximately 300 msec in duration, which includes (i) the closing phase, (ii) full eye closure, and (iii) the reopening phase. These phases are used to calculate the various ocular outcomes. Ocular outcomes that were examined include:

• Johns Drowsiness Score: A score between 1 and 10 (see above)

• Negative amplitude-velocity ratio (AVR): the ratio between the maximum amplitude to the maximum velocity for the reopening phase of all eye blinks per minute epoch. A larger ratio indicates slower reopening of the eye.

• Long eye closure: not including the closing phase and reopening phase of the eye during each blink, this metric examines the interval when the eye is completely shut (full closure). Here, a long eye closure is deemed to be > 10 msec. This measure reflects the percentage of time, per minute, that the eyes are deemed fully closed for > 10 msec.

• Inter-event duration: not including the closing phase and reopening phase of the eye during each blink, the inter-event duration refers to the time between the maximum velocity of the eyelid closing to the maximum velocity of the eyelid reopening for the same blink. This is averaged over one minute.

• Total blink duration: a measure of the total duration of an eye blink, including the duration of the closing phase, the closure, and the reopening phase of the eye blink.

• Time with eyes closed: not including the closing phase and reopening phase of the eye during each blink, this metric examines the interval when the eye is completely shut (full closure). This metric reflects the percentage of time within a 1-min epoch when the eye is fully closed.

Sleepiness and Performance Measurements

Participants rated their subjective sleepiness using the Karolinska Sleepiness Scale (KSS)⁴⁸ every 30 min throughout the extended wake episode (30 h). This is a 9-point scale ranging from “extremely alert: 1” to “extremely sleepy, fighting sleep: 9” with descriptors of increasing sleepiness placed on odd numbers from 1 to 9. Participants completed the KSS by selecting the appropriate number on a computer keyboard.

Neurobehavioral performance was assessed every other hour throughout the 30-h extended wake episode using the visual psychomotor vigilance task (PVT),⁴⁹ a task of sustained attention shown to be a reliable index of objective drowsiness.^17,18,50 Here, participants placed their dominant thumb on the response button, responding as quickly as they could to a millisecond counter which appeared in a small rectangle box in the center of the screen (PVT stimulus). Random inter-stimulus intervals ranged from 1-10 sec, and task duration was approximately 10 min (∼90 stimulus presentations). PVT lapses were defined as response times ≥ 500 msec. Neurobehavioral testing started 2 h after wake time, resulting in 14 tests across the 30-h constant routine.

During the entire protocol, participants reclined in bed at a 45° angle in a hospital bed with the head of the bed elevated. While taking the neurobehavioral assessment, the computer screen was located at eye level and 53 cm away from the face, and participants completed KSS and PVT tasks using the keyboard and response box as laid out in front of them.

Marker of the Circadian Body Clock

Core body temperature (CBT) was measured every minute by a rectal thermistor (Measurement Specialties TPG, Dayton, OH). CBT_min, which was estimated using a nonorthogonal spectral analysis technique as described in detail elsewhere^51,52 was used as the marker of the circadian clock. Participants removed the sensor for bowel movements. Room temperature was maintained at 75° ± 3°F.

Data Analysis

Homeostatic and Circadian Analyses

For analyses with respect to time awake, all ocular parameters were averaged in hourly bins 1 h post-wake. For analyses with respect to circadian time, all ocular parameters were averaged in 1-h bins relative to CBT_min. To establish any effects within participant for time awake or time since CBT_min, we performed PROC MIXED (SAS 9.2) analyses, as this accommodates missing data. In addition, to achieve predictions at the participant level, participant was modeled as a random effect and time awake, or circadian time was modeled as a fixed effect, using the Kenward-Rogers method to calculate degrees of freedom. An auto-regressive co-variance structure was used for repeated measures.⁵³ Post hoc comparisons for any main effect of time awake or time since CBT_min were made using pairwise comparisons with Bonferroni-Holm corrections for multiple comparisons. To reduce type I errors for post hoc testing, for time awake we compared each time bin after 16 h awake versus the average of the first 16 h awake (based on wake beyond 16 h being a near-linear predictor of performance⁶). Here, data for the first 16 h were averaged to form one value. For circadian time, we compared each hourly bin after T_max (estimated as occurring, on average, ∼13 h prior to T_min according to Wyatt et al.⁵⁴) against an averaged “wake” reference point. Here, data were averaged between T_max and T_max-8h to form one value (8 h preceding T_max was the cutoff, as this is associated with the melatonin low point associated with the wake episode⁵⁴).

Comparison of Ocular Outcomes to Laboratory-Based Measures of Drowsiness

Ocular parameters in the 10 min preceding the neurobehavioral test battery were averaged and compared to PVT performance and KSS ratings during the subsequent test session. PVT data were normalized by log transformation of mean reaction time and the slowest 10% of responses and total PVT lapses ≥ 500 msec normalized by expressing the total number of lapses as a function of √(n)+√(n+1). Within participant correlation coefficients were calculated using Pearson correlation analysis to determine the relationship between each ocular parameter and corresponding subjective/objective drowsiness levels for each individual, to examine individual differences in coefficients.

Receiver Operating Characteristics (ROC) Analysis

ROC analysis was used to evaluate the relative strength of each ocular parameter in predicting drowsiness according to a predefined threshold based on laboratory-based measures of drowsiness (KSS, PVT). KSS threshold was defined as ≥ 6 (“greater than some signs of sleepiness”). As no threshold exists for determining an “impaired” versus “non-impaired” PVT test, as per Chua et al.,¹⁶ a binary classification task was used to identify an objective drowsiness threshold. Here, drowsiness was defined as a 50% or 75% increase in the number of PVT lapses, measured relative to the number of lapses in the first 16 h awake during the 30-h extended wake episode. Baseline performance was computed as threshold × range + baseline, where “range” was the difference between the baseline and maximum number of lapses for each individual, “baseline” was the median number of PVT lapses during the first 16 h of wakefulness, and “threshold” was either 0.5 or 0.75.¹⁶ Each PVT session was then categorized as “normal” if the number of lapses was below threshold or “impaired” if the number of lapses exceeded the threshold. For KSS, the threshold was set at 6, depicting self-reported drowsiness level at or above “some signs of drowsiness.”

To assess the relative strength of each ocular parameter in predicting “normal” versus “impaired” performance or drowsiness for every neurobehavioral test for all participants, ROC curves (AUC) were constructed to examine the relative strength of each ocular metric for detecting the drowsy impaired state. Pre-test probability was estimated empirically from the data, which was p = 0.29 for a 50% increase in PVT lapses, p = 0.20 for a 75% increase in PVT lapses, and p = 0.35 for an increase in KSS. Cost-ratios were set at 0.5 for 50% increase in PVT lapses, 0.25 for 75% increase in PVT lapses, and 1.0 for KSS. These values were used to determine optimal cutoff points.

Johns Drowsiness Scores: A New Measure of Drowsiness?

A linear mixed model compared average outcomes on the neurobehavioral test battery as a function of prior JDS (in bins JDS ≤ 1.4, 1.5 ≤ 2.4, 2.5 ≤ 3.4, 3.5 ≤ 4.4, ≥ 4.5). For any individual contributing > 1 PVT/KSS per JDS bin, data were averaged within participant, within JDS bin. Here, participant was used as a random effect and JDS bin as a fixed effect, using the Kenward-Rogers method to calculate degrees of freedom. An auto-regressive co-variance structure was used for repeated measures. Due to missing data for KSS and lapses (see results), participants no longer showed significant inter-individual differences and were modeled only with repeated measures. Post hoc pairwise comparisons with Bonferroni-Holm corrections are reported. Only JDS scores were applied in the dose response analysis because of their well-defined boundaries for changing drowsiness levels.

Linear mixed models were computed using SAS 9.2; ROC curves were constructed using SigmaPlot 11 (Systat Software Inc, San Jose, CA); and all other data were analyzed using IBM SPSS Statistics, Version 19 (SPSS Inc., New York, NY).

RESULTS

We present data from 28 of 29 participants who completed the inpatient segment of the study (n = 28). Data from one participant was excluded due to technological difficulties with the Optalert system. For circadian phase data, we present data on n = 24; n = 4 were not presented due to a loss of temperature and/or an inability to calculate CBT_min. The loss of this data was due to improper positioning of the sensor (n = 2), no use of the sensor due to prior risk of hemorrhoids (n = 1) and quality control of temperature data (n = 1). Data from n = 28 participants is presented for analysis on homeostatic (time awake) changes in drowsiness. In the 10-min interval preceding any neurobehavioral test, only data that met the highest signal quality checks for Optalert signals were included in the analysis. Intervals with poor signal quality were removed on an individual basis, unless they exceeded 50% of the total trials, in which case the participant was removed from all analyses (n = 7 participants). Of a total of 392 sessions (14 sessions*28 participants), 263 passed signal quality checks (67.2%). Of those who were excluded, 28 (7.1%) were excluded because of a variable baseline (improper positioning of the Optalert glasses); 2 (0.5%) were excluded because of a biphasic signal (picking up lower lid movement); 5 (1.3%) were excluded because no eye movements were detected; 6 (1.5%) were excluded because of the participants looking down underneath the barrel; and 88 (22.4%) were excluded because of a combination of these factors. For neurobehavioral data binned according to prior JDS (dose response), not all participants had JDS scores in each binned category: JDS < 1.49 n = 19; JDS 1.5-2.49 n = 23; JDS 2.5-3.49 n = 23; JDS 3.5-4.49 n = 22; JDS > 4.5 n = 13). Three PVT tests (from 2 participants) were removed for final analysis due to incorrect procedures (failure to adhere to study protocol [n = 1] and wrong button presses [n = 2]). Total PVT test sessions included in the analysis were 389/392 (99.2%). KSS data is presented from all 392 trials (100%).

Time Course of Changes in JDS Output: Homeostatic and Circadian Influences

Figure 1 shows hourly binned data for all ocular parameters over the 30-h CR. For analysis purposes, we collapsed hours 1-16 into one time point, in order to examine the effect of time awake (average 2-16^th hour, average 17^th hour, average 18^th hour…average 30^th hour). Linear mixed model revealed a significant main effect of time awake for JDS scores (F_14,330 = 8.71, p < 0.0001; Panel A), AVR (F_14,327 = 9.31, p < 0.0001; Panel B), long eye closures (F_14,316 = 7.82; p < 0.0001; Panel C), inter-event duration (F_14,333 = 2.43, p < 0.003; Panel D), total blink duration (F_14,326 = 5.02, p < 0.0001; Panel E), and time with eyes closed (F_14,321 = 6.59, p < 0.0001; Panel F). Paired t-tests comparing all time points after 16 h awake compared to the average of the first 16 h of wakefulness and are shown in Figure 1 (Panel A-F). Compared to an average of the first 16 h awake, after 30 h awake all ocular parameters reflected increased drowsiness: (JDS: 2.2 ± 0.19 vs. 3.4 ± 0.25; AVR: 1.86 ± 0.06 vs. 2.1 ± 0.07; long eye closure: 0.66 ± 0.10 vs. 2.9 ± 0.50; inter-event duration: 0.13 ± 0.01 vs. 0.18 ± 0.02; total blink duration: 0.46 ± 0.02 vs. 0.87 ± 0.10; and time with eyes closed: 1.99 ± 0.25 vs. 6.34 ± 0.97).

Figure 1. Mean hourly ocular outcomes as a function of hours awake.

Following a main effect of time (p < 0.0001), post hoc tests with Bonferroni-Holm corrections comparing hourly bins to the average of the first 16 h awake are shown (▼ p < 0.05/k-1). Vertical line represents 16 h of wakefulness. Post hoc comparisons were made for each hourly bin after 16 h awake compared to the average value in the first 16 h, as represented by the horizontal line. Means ± SE are shown for (A) JDS, n = 392; (B) amplitude-velocity ratio, n = 392; (C) long eye closures per minute, n = 385; (D) inter-event duration, n = 392; (E) total blink duration per minute, n = 391; and (F) time with eyes closed per minute, n = 391.

To examine any potential confound of averaging the first 16-h interval, linear mixed models were repeated for all time points (2^nd-30^th) without averaging hours 1-16. Here, the significant main effect of time awake for hourly binned JDS scores (F_28,691 = 7.59, p < 0.0001), AVR (F_28,691 = 6.48, p < 0.0001), long eye closures (F_28,688 = 4.45, p < 0.0001), inter-event duration (F_28,696 = 2.87, p < 0.0001. Panel D), total blink duration (F_28,682 = 4.56, p < 0.0001), and time with eyes closed (F_28,688 = 4.45, p < 0.0001) remained. Pairwise comparisons of all time pairs are available in the supplemental material.

As participants entered the CR with varying schedules and prior light exposure history (see Methods), we conducted secondary analyses to examine any effect of protocol (Time Awake*Protocol) on ocular parameters using linear mixed model analyses. There was no effect of protocol (p > 0.4), nor any time awake*protocol interaction (p > 0.9), for any ocular parameter. Protocol was therefore not included in any further analyses.

As individuals entered the CR at unknown circadian phases, we examined JDS relative to circadian time (see supplemental material for the association between core body temperature minimum and time since wake for all participants). As seen in Figure 2, average hourly ocular outcome scores were computed and binned according to circadian time 0 (CBT_min), and compared using a linear mixed model to take into account missing values. For analysis purposes, we averaged CBT_max to CBT_max-8h to reflect one point versus all other hourly bins from CBT_max to 2 h after CBT_min. There was a significant main effect of circadian time for JDS (F_16,306 = 12.37, p < 0.0001; Panel A), AVR (F_16,306 = 8.79, p < 0.0001; Panel B), long eye closures (F_16,306 = 4.35, p < 0.0001; Panel C); inter-event duration (F_16,307 = 3.33, p < 0.0001; Panel D); total blink duration (F_16,304 = 3.59, p < 0.0001; Panel E), and time with eyes closed (F_16,297 = 4.11, p < 0.0001; Panel F). Post hoc tests with Bonferroni-Holm corrections compared all time points after CBT_max to the average reference interval of CBT_max to CBT_max-8h. As seen in Figure 2, compared to an average of CBT_max-CBT_max-8h (“alert”), all ocular parameters indicated increased drowsiness at CBT_min/CBT_min+1 (JDS: 2.26 ± 0.20 vs.3.29 ± 0.25; AVR: 1.88 ± 0.06 vs. 2.15 ± 0.08; long eye closure: 0.71 ± 0.13 vs. 1.32 ± 0.21; inter-event duration: 0.13 ± 0.01 vs. 0.16 ± 0.01; total blink duration: 0.46 ± 0.03 vs. 0.64 ± 0.09; and time with eyes closed: 1.91 ± 0.28 vs. 3.88 ± 0.70).

Figure 2. Mean hourly ocular outcomes as a function of circadian time.

Following a main effect of time (p < 0.0001), post hoc tests show elevated JDS scores from CBT_max (13 h prior to CBT_min) versus the average of the 8 h preceding CBT_max (▼ p < 0.05/k-1). Vertical lines represent an estimate of the start and end of biological night,^51,56,57 and the horizontal line represents the average value used as the reference point for post hoc testing. Means ± SE are shown for (A) JDS, n = 372; (B) amplitude-velocity ratio, n = 367; (C) long eye closures per minute, n = 367; (D) inter-event duration, n = 3,364; (E) total blink duration per minute, n = 3,368; and (F) time with eyes closed per minute, n = 368.

As shown previously for time awake, to examine any potential confounds of averaging CBT_max-CBT_max-8h to form reference value, we repeated the linear mixed model analyses for all time points (CBT_min-23h to CBT_min +2h) without averaging of any circadian time points. The significant main effect of circadian time remained (JDS [F_25,503 = 8.7, p < 0.0001], AVR [F_25,498 = 6.35, p < 0.0001], long eye closures [F_25,4893 = 3.4, p < 0.0001]; inter-event duration [F_25,496 = 3.03, p < 0.0001]; total blink duration [F_25,492 = 3.01, p < 0.0001], and Time with Eye Closed [F_25,483 = 2.89, p < 0.0001]). Post hoc comparisons for all data points can be seen in the supplemental material.

Relationship between Ocular Measures and Objective and Subjective Indices of Drowsiness

To compare sensitivity and specificity of different ocular outcomes for predicting drowsiness according to standard laboratory-based measures of drowsiness, we constructed receiver operator characteristic curves for subjective (KSS) and objective (PVT) thresholds for drowsiness. KSS ≥ 6 (“some signs of sleepiness” or above) was used as a subjective threshold for drowsiness. As no cutoff exists that describes a level of “impaired” performance or well-defined threshold for drowsiness on the PVT, partly due to strong individual differences in “impaired values,” we determined varying levels of impairment as a 50% or 75% increase in lapses compared to baseline (see Methods). Notwithstanding circadian fluctuations, for most individuals, average PVT lapses exceeded the 50% threshold during the subjective night (∼22 h awake), whereas the average 75% threshold occurred close to habitual wake time (∼24 h awake). While all AUCs were significantly greater than 0.5 (p < 0.001), the highest AUCs were reported for JDS, followed by time with eyes closed. ROC curves for JDS versus PVT/ KSS can be seen in Figure 3. While AVR had the largest sensitivity value (0.88), this was only reliable for the highest level of impairment on the PVT. Sensitivity and specificity values were consistently better for JDS than all other ocular parameters, closely followed by long eye closures and time with eyes closed (Table 1).

Figure 3. Receiver Operator Characteristics curves.

Receiver Operator Characteristics curves demonstrating the sensitivity and specificity of JDS in detecting objectively defined drowsiness (50% increase in PVT lapses, left) and subjectively defined drowsiness (KSS ≥ 6, right). Optimal cutoffs for JDS were 2.75 and 3.11 for a 50% increase in PVT lapses from baseline and KSS ≥ 6, respectively.

Table 1.

Sensitivity, specificity, and area under the curve (AUC): ocular parameters versus objective and subjective thresholds for drowsiness

Based on ROC analyses we determined the cutoff value for JDS in detecting drowsiness according to a 50% increase in PVT lapses from baseline as JDS = 2.75. Using as more stringent criteria of a 75% increase in PVT lapses from baseline, the cutoff value for JDS was 2.62 and for subjective levels of drowsiness (KSS ≥ 7), the optimum cutoff value for JDS was 3.11. JDS values between 2.5 and 4.5 were associated with varying levels of sensitivity and specificity (Table 2).

Table 2.

Sensitivity and specificity values associated with JDS thresholds of 2.5-4.5

Within subject correlation coefficients were calculated to examine the relationship between all ocular parameters and drowsiness at measured by the KSS and PVT over time for each individual (see Table 3). The percentage of individuals who exhibited significant statistical relationships is shown.

Table 3.

Average individual correlation coefficients (r) and the percentage of individuals (%) who demonstrate significant relationships between ocular parameters and gold standard measures of objective and subjective drowsiness

JDS versus Current Laboratory-Based Measures of Drowsiness

As seen in Figure 4, the average change in drowsiness levels as reflected by the JDS had similar temporal patterns of change as average PVT and average KSS. Averaged objective and subjective sleepiness levels (average PVT; average KSS) following JDS scores of ≤ 1.49, 1.5 to 2.49, 2.5 to 3.49, 3.5 to 4.49, and ≥ 4.5 were examined. Here JDS scores in the 10 min preceding the neurobehavioral test battery were averaged and compared against mean RT, mean slowest 10%, and mean number of lapses. As seen in Figure 5, JDS scores were indicative of neurobehavioral performance in a dose-response manner. Linear mixed model analyses revealed significant main effect of prior JDS on mean RT (F_4,62.2 = 6.53, p = 0.0002), mean number of lapses (F_4,73.9 = 9.05, p < 0.0001), mean slowest 10% responses (F_4,57.8 = 7.71, p < 0.0001), and KSS (F_4,72.4 = 9.40, p < 0.0001). Post hoc tests are shown in Figure 5.

Figure 4. Change in objectively defined drowsiness.

Change in objectively defined drowsiness over time according to mean (A) JDS, (B) RT, (C) number of lapses, (D) slowest 10% of responses and (E) KSS. All measures show an exponential increase, indicating increased drowsiness, after 16 h awake.

Figure 5. Dose-response relationship.

Dose-response relationship between average JDS score leading up to neurobehavioral battery and the subsequent drowsiness response. Increasing mean JDS scores are associated with increased mean response time, mean number of lapses, increased mean slowest response times and increased mean KSS. JDS < 1.49, n = 19; JDS 1.5-2.49, n = 23; JDS 2.5-3.49, n = 23; JDS 3.5-4.49, n = 22; JDS > 4.5, n = 13.

DISCUSSION

Evidence-based tools to quantify changes in drowsiness are a fundamental part of accurately understanding major physiological determinants of alertness. Using a commercially available technology employing infrared reflectance oculography to detect small, transient changes in ocular parameters, we demonstrated the usefulness of these parameters in identifying the drowsy state, and demonstrated their consistency with established laboratory-based standard measures of both subjective (KSS) and objective (PVT) drowsiness.

As drowsiness fluctuates according to homeostatic and circadian regulatory processes,^54,55 ocular signals that reflect this dynamic state should increase as a function of time awake and during the biological night. Here, we report that the amplitude-velocity ratio of an eye movement, the percentage of long eye closures (> 10 msec), total blink time duration, and percentage of time the eyes were closed increased following > 24 h time awake and during the biological night, consistent with previous studies reporting drowsiness-associated increases in blink duration^{22,24–26,56–58} and/or eye closure.^{7,21–23,56,57} Combining these output parameters via a proprietary algorithm, Optalert technology provides an additional index of drowsiness called the Johns Drowsiness Scale (JDS). Consistent with previous studies,^32,33 compared to other ocular parameters from which the composite JDS is derived, we found the JDS to be the most sensitive marker of drowsiness due to time awake and during the biological night.

Previous studies have identified a drowsiness threshold of JDS 4.5 for identifying impaired performance while driving³²; increased lane variability is clearly associated with this threshold. Using receiver operator characteristic curves to determine the most sensitive/specific threshold for drowsiness detection in a laboratory non-driving setting, we reported cutoff scores for depicting drowsiness-related impairment ranging between 2.6 and 3—substantially lower than driver simulation studies. Our data suggest that when using a JDS threshold of 2.75, 76% of incidences associated with a 50% increase in PVT lapses from baseline would be detected. In comparison, as shown in Table 3, using previously recommended thresholds of JDS 4.5 based on driving impairment, we would only detect 18% of cases where PVT lapses were elevated 50% above baseline levels. Two major implication of this revised cutoff score include: (1) Future laboratory studies which may pursue this system for detecting drowsiness should refrain from using the JDS 4.5 as a marker of drowsiness but instead employ JDS 2.75 as described here; and (2) as we demonstrate JDS scores of 2.75 to be reliably associated with increased PVT lapses and delayed response time, which are essential for safe driving practice, future employment of this device whether scientific or practical should not assume that a JDS < 4.5 is indicative of safe driving or as a lack of drowsiness: previous studies have shown this to be associated with the car leaving the carriageway,³³ which is an extreme level of drowsiness. As shown here, lower JDS levels may be indicative of earlier, lower levels of drowsiness as detected by the PVT. However, further driver simulation-based work is required before the threshold should be lowered for warning of impaired driving as this would impact on the number of false alarms.

Caffier et al.²² captured numerous eye lid closures using infrared reflectance oculography and surmised that blink duration was the most informative parameter for detecting drowsiness, essentially due to the delay of lid reopening time. While we did not find blink duration to be the most informative ocular parameter depicting drowsiness, this was likely because our study captured more sensitive ocular parameters not obtained by Caffier et al. (i.e., AVR, JDS). As seen in Figure 1, we demonstrated a clear increase in JDS and AVR following 16 h of time awake. As these measures principally reflect a delay in lid reopening, this corroborates the interpretation by Caffier et al. that drowsiness preferentially delays lid reopening.²² An additional advantage of JDS and AVR is reduced inter-individual differences in the signal^39,59 commonly found to be problematic in other ocular signals, such as blink rate or blink duration.^22,26,27 AVR remains stable in the alert and drowsy state on the downward phase (lid closing) with a large blink having a corresponding fast velocity. However, when drowsy, velocity is delayed on the upward phase, resulting in a steeper slope such that velocity is slower irrespective of the amplitude.³⁹

Previous studies have described PERCLOS as a sensitive biomarker of drowsiness.^16,42,43 Despite this we were unable to compare this against the ocular metrics generated by the Optalert system. While PERCLOS is based on the percentage of time the eyes are closed > 80%, we examined the percentage of time the eyes were fully closed (Total Eye Closure). For this output parameter, we reported lower AUCs for 50% and 75% increases in lapses than those reported by Chua et al.,¹⁶ likely due to a more stringent criteria (full closure vs. 80% closure). Despite this, percentage of total eye closure remained a sensitive marker of drowsiness-related impairment in our study.

As similarly shown by Cajochen et al.,⁷ the ocular outcomes examined here showed clear circadian changes such that there was a sharp rise in JDS and all other ocular measures during the biological night, estimated using core body temperature based on previous work in our laboratory.^54,60,61 As evident in Figure 2, JDS and AVR were maximal just after circadian time 0 in our study, which is consistent with previous studies that report maximal impairment in cognitive throughput, short term memory, alertness, and psychomotor vigilance just after the core body temperature nadir.⁵⁴

In further comparing JDS to objective and subjective markers of drowsiness, there was a clear association between increasing JDS scores and subsequent psychomotor vigilance, including attention lapses and subjective ratings of drowsiness (KSS), in a dose-response manner. In addition, sensitivity and specificity scores for detecting the drowsy state were higher for JDS than all other ocular outputs (except AVR which had high sensitivity for detecting a 75% increase in lapses, yet the corresponding specificity was poor). Of particular relevance, our study used JDS scores to predict performance over the next 10 min, in contrast to previous studies³³ that have used JDS scores occurring simultaneously with poor performance. Our study is novel in this respect. This tight coupling between ocular indices of drowsiness and subsequent performance impairment demonstrates that drowsiness can be predicted by ocular movements prior to performance impairment, and also in real time. For instance, JDS has previously been associated to simultaneous RT,³² although only two data points were examined (normal sleep versus 27-33 h awake). As our study further examines the change in JDS and ocular parameters due to drowsiness by examining change over 30 h awake, these data build on the current literature on the association between ocular parameters and objectively defined measures of drowsiness.

The sensitivity of eye movements to the drowsy state has a clear physiological basis. Eye movements provide a direct insight into central nervous system function and physiological changes in vigilance via direct feedback from the midbrain reticular formation.²⁰ Eye closures are controlled primarily by the interaction of two muscles: the levator palpebrae, which maintain an open lid when active, and the orbicularis oculi, which contract to close the lid.⁶² With increasing drowsiness, changes in the afferent limbic signals reduce the activity of the levator palpebrae motoneuron, culminating in a closed eye.⁶³ As the JDS reflects a delay in lid reopening, ostensibly driven by changes in AVR as the ratio is altered by slower eye reopening velocities, this metric provides a measurement of vigilance likely reflective of vigilance at the neural level. Together, these findings suggest that infrared reflectance oculography, and in particular the JDS output parameter (Optalert) provides a clear indication of drowsiness in the laboratory setting.

Our study was conducted in a highly controlled, distraction-free laboratory environment, which included participants remaining largely immobile for the duration of the protocol (a requirement of the constant routine for circadian biomarkers⁵¹). As such, this study does not assess the viability of this technology in real-world setting where environmental noise is present and participants are not confined to the laboratory bed. As described here, we made thorough signal quality checks and discarded data due to poor signal quality in a stringent manner similar to that done for electroencephalographic data. Due to methodological constraints, infrared reflectance oculography was measured at all times except during PVT testing, when the glasses were removed (due to a head-based eye tracker being used during that time). As a result, the glasses were removed and replaced 15 times during the 30-h constant routine which may have resulted in the glasses being misplaced at times and may have accounted for some of our poor signal issues due to improper positioning of Optalert glasses. In addition, following extended duration of time awake, our participants would occasionally tilt the head back, forcing the eyes to look down under the barrel. This resulted in poor signal quality which led to post hoc exclusion of data from the analysis.

Irrespective of these caveats, we systematically evaluated the use of infrared oculography as a tool to monitor several ocular parameters to depict dynamic changes in the drowsy state. Here, we report that the strongest predictors of drowsiness included JDS, AVR, and percentage of long eye closures, with JDS being the most promising index of drowsiness. Consistent with previous studies,^32,33 we conclude that the technology employed here and its composite JDS output (JDS), is an exciting technology that holds promise to be an effective research tool within the laboratory; the system requires no calibration, is relatively nonintrusive, and is well tolerated for long periods of time by participants. Furthermore, with more appropriately defined cutoff/thresholds for impairment, this system may be a useful tool in future studies evaluating changes in drowsiness due to sleep loss, circadian misalignment, pharmacologically driven sedation effects and the impact of countermeasures. However, for detection of drowsiness under these circumstances, we suggest lowering the threshold considered “drowsy.” Overall, these data provide evidence that ocular output parameters recorded via infrared oculography, and in particular the composite JDS output derived from these parameters, has a strong potential to be an effective evidence-based tool for quantifying changes in drowsiness in the laboratory, combined or in the absence of existing laboratory-based measures.

DISCLOSURE STATEMENT

This was not an industry supported study. Dr. Anderson reports receiving a research award/prize from Sanofi-Aventis; contract research support from VicRoads; and lecturing fees from Brown Medical School/Rhode Island Hospital and Ausmed. In addition, she has served as consultant to the Rail, Bus and Tram Union to produce a report on best practice management of fatigue in the rail industry, through an agreement between Monash University and the Rail, Bus and Tram Union. Since completion of this study, Monash University, for which Dr. Anderson now works, has received equipment on loan from Optalert TM, for a separate study. Dr. Czeisler has received consulting fees from or served as a paid member of scientific advisory boards for: Actelion, Ltd.; Astra Zeneca; Avera Pharmaceuticals, Inc.; Bombardier Inc.; Boston Celtics; Celadon Trucking Services; Cephalon Inc. (acquired by Teva Pharmaceutical Industries Ltd. October 2011); Columbia River Bar Pilots; Delta Airlines; Eli Lilly and Co.; Federal Motor Carrier Safety Administration (FMCSA), U.S. Department of Transportation; FedEx Kinko's; Fusion Medical Education, LLC; Garda Síochána Inspectorate; Gerson Lehrman Group for Novartis; Global Ground Support; Hypnion, Inc. (acquired by Eli Lilly and Co. in April 2007); Johnson & Johnson; Koninklijke Philips Electronics, N.V.; Minnesota Timberwolves; Morgan Stanley; Norfolk Southern; Portland Trail Blazers; Respironics, Inc. (acquired by Koninklijke Philips Electronics, N.V. in March 2008); Sanofi-Aventis Groupe; Sepracor, Inc.; Sleep Multimedia, Inc.; Sleep Research Society (for which Dr. Czeisler served as president); Somnus Therapeutics, Inc., Takeda Pharmaceuticals, Vanda Pharmaceuticals, Inc., Vital Issues in Medicine, Warburg-Pincus, and Zeo Inc. Dr. Czeisler owns an equity interest in Lifetrac, Inc.; Somnus Therapeutics, Inc.; Vanda Pharmaceuticals, Inc., and Zeo Inc., and received royalties from CNN; the Massachusetts Medical Society/New England Journal of Medicine; McGraw Hill, the New York Times; Penguin Press/Houghton Mifflin Harcourt; and Philips Respironics, Inc. Dr. Czeisler has received lecture fees from the AWHONN (Association of Women's Health, Obstetric and Neonatal Nurses) Accreditation Council of Graduate Medical Education; Alfresa; the Alliance for Epilepsy Research; the American Academy of Allergy, Asthma and Immunology Program Directors; American Academy of Sleep Medicine; American Physiological Society; Association of University Anesthesiologists; Axis Healthcare, LLC; Baylor College of Medicine; Beth-Israel Deaconess Medical Center; Brown Medical School/Rhode Island Hospital; Cephalon, Inc.; Clinical Excellence Commission (Australia); Dalhousie University; Duke University Medical Center; Duke University School of Medicine; Harvard School of Public Health; Harvard University; Hokkaido University Graduate School of Medicine; Institute of Sleep Health Promotion (NPO); Japan Aerospace Exploration Agency (JAXA); LOTTE Health Products; London Deanery; Morehouse School of Medicine; Mount Sinai School of Medicine; National Academy of Sciences; National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK/NIH); National Institutes of Health; National Sleep Foundation; New England College of Occupational and Environmental Medicine (NECOEM); North East Sleep Society; Office of Rare Diseases Research (NIH); Osaka University School of Medicine; Partners HealthCare, Inc.; Rockpointe; Sanofi-Aventis, Inc.; Sleep Research Society; Society for Obstetric Anesthesia and Perinatology (SOAP); Society of Thoracic Surgeons; St. Lukes Roosevelt Hospital; Takeda; Tanabe Seiyaku Co., Ltd.; Tokyo Electric Power Company (TEPCO); U.S. Fire Administration Executive Fire Officer Program [formerly listed as National Emergency Training Center]; University of Chicago; University of Colorado University of Michigan; University of Pennsylvania; University of Pittsburgh; University of Tsukuba; University of Virginia Medical School; University of Washington Medical Center; University of Wisconsin Medical School; World Federation of Sleep Research and Sleep Medicine Societies and WME Entertainment LLC. Dr. Czeisler has also received research prizes with monetary awards from the American Academy of Sleep Medicine; American Clinical and Climatological Association; Association for Patient-Oriented Research; National Institute for Occupational Safety and Health; National Sleep Foundation; and Sleep Research Society; clinical trial research contracts from Cephalon, Inc., Merck & Co., Inc., and Pfizer, Inc.; an investigator-initiated research grant from Cephalon, Inc.; and his research laboratory at the Brigham and Women's Hospital has received unrestricted research and education funds and/or support for research expenses from Cephalon, Inc., Koninklijke Philips Electronics, N.V., ResMed, Committee for Interns and Residents and the Brigham and Women's Hospital. The Harvard Medical School Division of Sleep Medicine (HMS/DSM), which Dr. Czeisler directs, has received unrestricted research and educational gifts and endowment funds from: Boehringer Ingelheim Pharmaceuticals, Inc., Cephalon, Inc., George H. Kidder, Esq., Gerald McGinnis, GlaxoSmithKline, Herbert Lee, Hypnion, Jazz Pharmaceuticals, Jordan's Furniture, Merck & Co., Inc., Peter C. Farrell, Ph.D., Pfizer, ResMed, Respironics, Inc., Sanofi-Aventis, Inc., Sealy, Inc., Sepracor, Inc., Simmons, Sleep HealthCenters LLC, Spring Aire, Takeda Pharmaceuticals, and Tempur-Pedic. The HMS/DSM has received gifts from many outside organizations and individuals including: Aetna US Healthcare, Alertness Solutions, Inc., American Academy of Sleep Medicine, Boehringer Ingelheim Pharmaceuticals, Inc., Brigham & Women's Hospital (Department of Medicine), Brigham & Women's Hospital (Development Office), Bristol-Myers Squibb, Catalyst Group, Cephalon, Inc., Clarus Ventures, Comfortaire Corporation, Committee for Interns and Residents, Eisai, Inc., Eli Lilly and Co., Farrell Family Foundation, Fisher & Paykel Healthcare Corporation, George H. Kidder, Esq., GlaxoSmithKline, Gerald McGinnis, Gosule, Butkus & Jesson, LLP, Herbert Lee, Hypnion, Inc., Innovative Brands Group (Nature's Rest), Jordan's Furniture, King Koil Sleep Products, Land and Sky, Lilly USA, LLC, Merck Research Laboratories, MPM Capital, Neurocare Center for Sleep, Neurocrine Biosciences, Inc., NeuroScience, Novartis Consumer Health, Orphan Medical/Jazz Pharmaceuticals, Park Place Corporation, Pfizer Global Pharmaceuticals, Pfizer Healthcare Division, Pfizer, Inc., Philips-Respironics, Inc., PR21, Praxair US Homecare, Purdue Pharma L.P., Ramiro Sanchez, Jr. M.D., ResMed, Inc., Respironics, Inc., Safeway, Sanofi-Aventis, Inc., Sanofi-Synthelabo, Sealy Mattress Company, Sealy, Inc., Select Comfort Corporation, Sepracor, Inc., Simmons Co., Sleep Ave, LLC, Sleep HealthCenters LLC, SleepCare, LLC, Somaxon Pharmaceuticals, Spring Air Mattress Co., Synchrony Healthcare Communications, Takeda Pharmaceuticals, Tempur-Pedic Medical Division, Total Sleep Holdings, Transcept Pharmaceuticals, Vanda Pharmaceuticals, Inc., Wake Up Narcolepsy, Inc., Watermark Medical, Weight Watchers International, YMCA of the USA, the Zeno Group and Zeo, Inc. (formerly Axon Sleep Research Laboratories, Inc.) The HMS/DSM Sleep and Health Education Program has received Educational Grant funding from Cephalon, Inc., Sanofi-Aventis, Inc. Sepracor, Inc., and Takeda Pharmaceuticals, Dr. Czeisler is the incumbent of an endowed professorship provided to Harvard University by Cephalon, Inc. and holds a number of process patents in the field of sleep/circadian rhythms (e.g., photic resetting of the human circadian pacemaker). Since 1985, Dr. Czeisler has also served as an expert witness on various legal cases related to sleep and/or circadian rhythms. The other authors have indicated no financial conflicts of interest.

SUPPLEMENTAL MATERIAL

The data as described in this manuscript was obtained from the 30 hour constant routine period. Optalert^TM data was obtained continuously and neurobehavioural data was obtained bi-hourly starting 1 hour after wake time during this period.

Figure S2. Example of Study Protocol.

*30 hour constant routine indicted by gray shaded area on days 7 and 8.

Table S1

Time of Core Body Temperature Minimum: Real Clock Time and Time Since Waking