CAN INTERMITTENT VIDEO SAMPLING CAPTURE INDIVIDUAL DIFFERENCES IN NATURALISTIC DRIVING?

Summary

We examined the utility and validity of intermittent video samples from black box devices for capturing individual difference variability in real-world driving performance in an ongoing study of obstructive sleep apnea (OSA) and community controls. Three types of video clips were coded for several dimensions of interest to driving research including safety, exposure, and driver state. The preliminary findings indicated that clip types successfully captured variability along targeted dimensions such as highway vs. city driving, driver state such as distraction and sleepiness, and safety. Sleepiness metrics were meaningfully associated with adherence to PAP (positive airway pressure) therapy. OSA patients who were PAP adherent showed less sleepiness and less non-driving related gaze movements than nonadherent patients. Simple differences in sleepiness did not readily translate to improvements in driver safety, consistent with epidemiologic evidence to date.

INTRODUCTION

“Black box” devices are increasingly employed to study real-world driving performance and safety (Blanchard et al., 2010; Blanchard & Myers, 2010; Crizzle et al., 2011; Huebner et al., 2007; Marshall et al., 2007; Myers et al., 2011) and vary widely in capabilities. Most provide vehicle data such as speed from on-board diagnostic port (OBD-II), some interface with GPS, yet few provide video data. Video samples are necessary to contextualize and classify driver actions but pose analysis and storage challenges. Brief video clips (often triggered by an acceleration exceedance) are adequate to assign fault status to drivers involved in a crash (e.g. DriveCam). Yet crashes are statistically rare events and their value for quantifying individual differences in driver performance for broader research programs on driving safety is unclear. The current study examined preliminary data from intermittent video samples obtained from black box recorders in an ongoing study of real-word driving safety in OSA (Tippin et al., 2012). The broader study aims to examine the effectiveness of PAP adherence in improving excessive daytime sleepiness in patients diagnosed with OSA and its impact on driver safety (Treagar et al., 2009).

The device used in this study was designed to collect three types of video clips: a) trigger b) baseline and c) ignition. Based on previous work in the 100-car naturalistic study (Neale et al., 2005), clips were coded for safety errors, exposure including weather and road culture, and driver state including distraction and sleepiness (see Table 1 for greater details). Trigger clips, 20-seconds in duration, were collected when on-board accelerometers registered at least .35g’s (lateral, longitudinal, or vertical) permitting focused assessment of driver safety, exposure, and state surrounding critical events. The g-setting was based on pilot research which indicated that this setting permits the capture of many safety errors including sign violations, wide/ unsafe turns that are not egregious enough to be near-misses. Baseline clips, 20-seconds in duration, were recorded every 15 minutes of a drive, and permitted an assessment of safety, exposure, and driver state in the absence of critical acceleration events (< .35 g). Drivers take most trips on well-practiced daily routes (e.g., home to work, work to home, home to grocery store). Ignition triggered clips, allowed us capture such trips for repeated assessments of driver safety and state over the same road segments. These clips were also coded in 20-second segments paralleling trigger and baseline clips.

Table 1.

Coded dimensions in 20-second segments from each video clip in three domains of interest

Domain	Dimension	Examples
Safety	Errors	Running stop signs/lights, other traffic sign violations, turn errors, lane keeping, accelerating or braking hard enough to exceed .35 g’s (lateral, vertical, and/or longitudinal)
Driver State	Distraction	Included several categories, e.g. talking, singing, dancing, eating/drinking, tending to personal hygiene (hair, teeth), cell-phone, etc.
	Sleepiness	Slow eye lid closure, fixed gaze, rubbing eyes, yawning, low facial and bodily muscle tone, leaning/holding neck/head
Gaze Movements	Presence/absence of both driving (e.g. mirrors) and non-driving related gaze movements (e.g. reaching for something)
Exposure
Weather	Visibility	Reduced visibility due to fog, smoke or precipitation
	Precipitation	Any precipitation that causes the wipers to be used
	Road Condition	Potentially slippery roads (due to snow, ice or otherwise wet)
	Light-level	Non-day light or diminished light levels
Road Culture	City	multi-lane, single lane city streets, etc.
	Rural	rural highways, gravel roads, etc.
	Highway	interstate, shoulder present/absent highways, entrance/exit ramps, etc.
	Intersection	Sign or light controlled and uncontrolled intersections

The first goal was to compare effectiveness of each clip type in capturing key differences in driver safety, exposure, and state. We expected greater safety errors rates in trigger clips compared to ignition and baseline clips. Regarding road culture (an aspect of driver exposure),we expected trigger clips to capture intersection negotiations and city roads at a greater rate, and baseline clips to capture highway driving at a greater rate compared to other clips. We also expected greater rates of adverse weather/ road conditions in trigger clips. To the extent that sleepiness and distraction are associated with safety errors, we would expect higher levels of these states in trigger clips.

Although epidemiological and meta-analytic studies indicate that drivers with OSA are at increased risk for crashes (Treagar et al., 2009), the link between daytime sleepiness and crash risk is either weak or complex, e.g. nonlinear, moderated by other factors. Nevertheless, for the second and third goals of the study, we examined driver state variables in relation to PAP adherence data and self-reports of excessive sleepiness, and tested mean differences in safety, exposure, and driver state between compliant (at least 4 hours of pap use consistent with clinical recommendations) and noncompliant OSA participants post-pap.

METHODS

Subjects

Two groups of subjects, 37 diagnosed with OSA (26 male, age M = 48year, SD = 6.79) and 20 control participants (10 male, age = M = 45 years, SD = 8.03) participated in the study to date. Control participants were matched with OSA drivers on gender, age within 5 years, education within 2 years, and county of residence for rural vs. urban driving. Patients met ICSD-2 clinical criteria for OSA and had a Respiratory Distress Index > 15, while controls had no sleep complaints and an RDI < 5.

Procedure

Subjects were observed driving their own vehicles using an instrumented vehicle data acquisition system (IV-DAS) for a 2-week period prior to beginning PAP, and a period of 3-months after beginning PAP-use. IV-DAS contains three devices: an internal camera cluster (ICC), a GPS, and a central processing unit obtaining data from OBD-II and accelerometers. The cameras are located underneath the rear view mirror, with one pointing towards the road and the other at the face and upper body of the driver. Electronic data on vehicle state not used in this report are not described.

Measures

PAP-adherence and subjective sleepiness

Nightly pap-use data were downloaded from participant’s micro-sd card during monthly visits in the post-pap phase. Subjective ratings of excessive daytime sleepiness were obtained from Epworth Sleepiness Scale (ESS; John, 1991). For the current report, ratings from pre-pap and one-month into the post-pap phase were used for external validation.

Video clip coding

Driver performance and situational factors were evaluated in 20-second segments in three broad domains: safety, exposure, and driver state. Table 1 provides examples of coded dimensions in each of those domains (Klauer et al., 2006; Neale et al., 2005; Wierville & Elsworth, 1994). Coders were trained on example clips illustrating the range of behaviors in each of the dimensions listed in Table 1 until they showed acceptable levels of inter-rater reliability (e.g. on categorical scales a Kappa of .61 or on continuously distributed composite scales an intra-class correlation of .71). Raw counts of distracted behavior in several categories (e.g. cell-phone, eating, talking etc.) and safety errors in each clip type were divided by number of 20-second segments for each clip type within a day and thus transformed into rates. These rate measures specific to each clip type were averaged across days in the pre-pap and post-pap phases. Gaze movements, driving and non-driving related, could be each credited once in a segment and thus were transformed into relative frequency variables within a day and averaged across days in the pre-PAP and post-PAP phases. Seven indicators of sleepiness were rated using an ordinal scale from 0 (none) to 3 (high) in each segment. The relative frequency of 1, 2 and 3 ratings for each clip type within a single day were pooled into a weighted sum for each of those 7 indicators on a per day basis (e.g. weighted sum of slow eye lid closure, weighted sum of yawning, etc.). The resulting weighted sum scores for the 7 indicators were averaged within a day to create a sleepiness composite score for each clip type at the day-level. This sleepiness composite score was subsequently averaged across days in the pre-PAP and post-PAP phases.

RESULTS

Table 2 presents the descriptive statistics for number of coded segments in each clip type, pap-adherence, and self-reports of sleepiness pre and post-pap. On average OSA participants used pap slightly more than 4 hours per night on 58% of the days. ESS reported sleepiness levels improved from pre to post-pap for OSA but not controls (p < .05 for the interaction effect).

Table 2.

Descriptive data forming the basis of the analyses

	OSA		Control
	Pre-pap M (SD)	Post-pap M (SD)	Pre-pap M (SD)	Post-pap M (SD)
20-sec segments per day for clip type:
Ignition	13.86 (8.78)	14.63 (9.32)	14.21 (8.94)	13.93 (9.85)
Baseline	5.61 (6.15)	4.08 (4.17)	3.87 (3.33)	4.89 (4.05)
Trigger	3.58 (3.81)	3.31 (2.91)	4.58 (4.58)	4.27 (3.98)
ESS	11.92 (4.66)	9.84 (4.89)	7.2 (4.19)	6.89 (4.1)
Percent of days w/pap use >= 4 hrs	-----	.58 (.38)	-----	-----
Average # of hrs of pap use/night	-----	4.29 (2.63)	-----	-----

Question 1) Do the rates of safety errors, exposure (both road culture and weather), and driver state show expected mean differences across clip types?

Table 3 shows the descriptive statistics on coded dimensions in each clip type pre-pap reflecting sensitivity of clip type to variability concerning safety, exposure (weather and road culture), and driver state measures. For brevity, Table 3 only presents pre-pap data for the combined sample, however, the direction of findings were generally the same during the post-pap period. The p-values concern the within-subject main effect of clip-type from the 3×2 two-way mixed design with group as the between-subject factor.

Table 3.

Descriptive statistics on coded dimensions in each clip type pre-pap for OSA and controls

	Ignition M (SD)	Baseline M (SD)	Trigger M (SD)	p-value for clip type main effect
Driver safety and state
Safety errors	.04(.04)	.01 (.03)	.91 (.18)	<.001
Distraction	.58 (.28)	.51 (.46)	.35 (.27)	<.001
Driving related gaze movements	.65 (.13)	.38 (.22)	.71 (.24)	<.001
Non-driving related gaze movement	.50 (.17)	.44 (.24)	.46 (.26)	.098
Sleepiness	4.57 (7.11)	7.82 (10.1)	4.79 (7.15)	.003
Exposure
Road culture
Highway	.06 (.08)	.43 (.30)	.17 (.24)	<.001
Intersection	.44 (.11)	.23 (.23)	.79 (.25)	<.001
Rural	.04 (.09)	.03 (.07)	.03 (.10)	ns
City	.39 (.15)	.31 (.26)	.72 (.26)	<.001
Weather
Visibility	.06 (.08)	.07 (.16)	.05 (.09)	ns
Dark	.26 (.20)	.29 (.27)	.18 (.20)	.004
Precipitation (rain, snow)	.06 (.08)	.06 (.16)	.04 (.08)	ns
Slippery (wet/ icy/ snowy) roadway	.21 (.25)	.17 (.23)	.15 (.21)	ns

Consistent with expectations, Table 3 shows trigger clips contained greater rates of intersection negotiation and city driving while baseline clips contained greater rates of highway driving. Contrary to expectations, none of the weather and road condition related exposure variables differed across clip types, with the exception of light-levels. Low-level of illumination was more common in ignition and baseline clips than in trigger clips. In terms of driver safety and state, the findings were generally meaningful and suggested the presence of self-regulatory influences. For example, safety error rates were elevated in trigger clips. But trigger clips showed low levels of distraction and sleepiness and high rates of driving related gaze movements. Hence, the simplistic view that distracted and sleepy driving will be associated with declines in safety was not upheld. None of the main effects for the between-subject effect was significant.

Question 2) Are pre and post-pap measures of sleepiness from video clips correlated with two external validity measures: a) pap-adherence metrics for those with OSA and b) subjective self-reports of excessive daytime sleepiness pre and post-pap for all drivers?

Figure 1 shows the magnitude and direction of Pearson correlation coefficients of sleepiness measures from three types of clips pre and post-pap with pap-adherence and ESS ratings pre and post-pap. We would expect to see positive correlations between ESS and sleepiness metric but this was true for 2 out of 6 correlations both post-pap. In contrast, we would expect negative correlations between pap-adherence and sleepiness metrics. The findings were generally consistent with negative correlations but not always strong enough to be statistically significant.

Figure 1.

Pearson correlations between sleepiness measures in three clip types with pap-adherence and subjective sleepiness ratings (Note that when |r|>.32, p< .05)

Question 3) Is adherence to pap-therapy, clinically defined as at least 4 hours of use, associated with driver state variables, safety, and exposure post-pap in drivers with OSA?

Table 4 presents descriptive statistics for driver safety, state, and exposure for compliant vs. noncompliant OSA subjects, and associated p-values from independent sample t-tests. While non-driving related gaze movements and sleepiness levels were lower in compliant OSA patients compared to noncompliant patients, those differences did not appear to translate into improvements in safety. None of the mean comparisons on exposure variables indicated better tolerance for road risks as a result of compliance with treatment.

Table 4.

Descriptive data for driver safety, state, and exposure for compliant vs. noncompliant OSA subjects

	Noncompliant M (SD)	Compliant M (SD)	p-value
Safetya	0.94 (.10)	0.99 (.31)	ns
Sleepinessa	8.52 (7.94)	2.18 (2.63)	.007
Distraction	0.40 (.25)	0.44 (.22)	ns
Driving related gaze	0.63 (.18)	0.54 (.09)	.085
Non-driving related gaze	0.57 (.22)	0.40 (.16)	.020
Visibility	0.13 (.27)	.11 (.23)	ns
Darkness	0.21 (.26)	.13 (.19)	ns
Precipitation	0.12 (.29)	.11 (.23)	ns
Slippery roads	0.19 (.29)	.27 (.30)	ns

N = 13 for noncompliant and N= 17 for compliant OSA participants.

^aMeasures reflect level obtained from post-pap trigger clips. Remaining measures reflect average level from all three types of clips post-pap.

DISCUSSION

We investigated whether intermittent samples of video data from black box devices yielded meaningful individual difference variability in driver safety, exposure, and state in drivers with OSA compared to community controls, before and after OSA drivers received PAP-treatment. The findings supported the rationale for the video protocol in systematically capturing variability in all three domains of focus. For example, trigger clips showed elevations in safety errors compared to other clip types. Elevations in driver state variables including distraction and sleepiness were noted in baseline and ignition clips rather than trigger clips. Those findings do not support simplistic views/ hypotheses such as distracted driving and sleepiness compromise safety and should therefore be more common in trigger clips. Nevertheless, PAP-adherent OSA patients showed less sleepiness, less non-driving related gaze movements than noncompliant OSA patients. However, those differences in sleepiness did not appear to translate to significant improvements in safety. This finding is consistent with epidemiologic evidence (Treagar et al., 2009) of increased crash risk in the absence of evidence for correlations between sleepiness and crash risk and concurrent evidence for significant variability in the human ability to tolerate sleep deprivation (Philibert, 2005; Van Dongen et al, 2004). Poorly understood self-regulatory processes may reverse or attenuate such associations. For example, drivers may choose the timing and location of distracted driving (e.g. cell phone conversation, involved conversation with other passengers) to mitigate crash risk. Excessively sleepy drivers may take precautions by reducing speed, increasing headway distance, and other tactics.

Our preliminary findings suggest the influence of self-regulatory processes decouple driver state variables from safety errors when behavior is examined in 20-second windows. Other analyses not presented here, showed large variability in patterns of within-subject associations among drive state metrics (sleepiness, distraction, gaze movements) and safety. For example, sleepiness seemed to accompany non-driving related gaze movements for some drivers, yet the associations were reversed in other cases. Collectively those findings indicate that the association of driver safety with observer measures of excessive sleepiness is complex and highly situational. The variability in the patterning of within-subject associations among driver state variables encourages the construction of multi-dimensional driver state metrics that capture and describe how drivers maintain vigilance at the clip level (e.g., sleepy with no driving related gaze movements, sleepy with some driving related gaze movements, fighting sleepiness with distraction). We will examine these possibilities in forthcoming analyses with new data to further elucidate the complex associations of driver state with safety and exposure.