Sensitive and Reliable Measures of Driver Performance in Simulated Motor-Racing

Abstract

Motor racing is a physically and mentally demanding sport, associated with a high degree of risk for drivers. Hence, driving simulation provides a safe alternative to explore the impact acute physiological perturbations (e.g. heat stress or dehydration) on a driver’s performance. This study aimed to determine sensitive and reliable simulated driving performance parameters that could be employed in future driving performance studies. Thirty-six healthy males (age: 26.5 ± 8.1 y, body mass: 75.6 ± 12.2 kg, mean ± SD) completed a single experimental trial involving four simulated motor-racing drives (2 initial drives and 2 repeat drives) separated by a 1 h period. Drives were conducted under two conditions, with one condition (wearing Fatal Vision Goggles (FVG)) designed to impair driving performance by distorting vision. Sensitivity was assessed by comparing Normal vs FVG outcomes and reliability was determined by comparing initial vs repeat drives for the same condition. Measures of driving performance included lap time (LT), sector-time (ST) for one section of the track, position displacement to a marker on the first track corner (PD), and vehicle Speed at PD. Results indicated that LT and ST were reliable and sensitive performance measures to a visual disturbance. However, PD was neither sensitive nor reliable and Speed at PD was not sensitive as driving performance measures to the study conditions. Overall, this study demonstrates two sensitive and reliable performance measures (LT and ST) that can be used to assess simulated motor-racing performance in future investigations.

INTRODUCTION

Driving a motor vehicle is a complex task integrating sensory, perceptual, cognitive, and motor control components (1, 8). Whether part of a usual daily commute (i.e. on roads with other vehicles) or specific to a sporting context (i.e. motor-racing), driving is associated with a high degree of risk, with errors involving potentially fatal consequences. Given the risks, considerable scientific attention has been directed toward factors (e.g. fatigue, distraction, hydration, alcohol and drug intoxication) that may influence an individual’s ability to safely operate a motor vehicle.

With obvious risks involved in conducting research using road-based vehicles to examine driving performance/safety (3), computerized simulated driving has been developed as a safe alternative. Indeed, driving simulators have been successfully employed to quantify the impact of a variety of physiological and psychological factors on driving performance (4). While research indicates that driving simulators provide relative, rather than absolute validity (i.e. they approximate effects observed in on-road driving, but with directional similarities (12)), as an experimental approach, they offer some advantages to on-road testing. For example, they allow researchers to isolate the impact of specific factors on driving performance, controlling for possible confounding variables (e.g. road and environmental inconsistencies) (14). Nonetheless, performance measures captured during simulated driving must be reliable and valid.

Currently, there is sufficient evidence to support the use of driving simulators as a valid tool to assess vehicle control parameters such as speed and lateral position (7, 12). Indeed, these variables appear to be sensitive to acute perturbations in physiological homeostasis (e.g. alcohol intoxication, manipulations in hydration status) (5, 19). However, the validity and sensitivity of these parameters has been established within commute-style driving (i.e. urban and rural environments with speed limited roadways) simulated scenarios. To date, no study has explored the reliability and sensitivity of driving performance parameters within a simulated motor-racing context. This is important, as the critical parameters for motor-racing performance are likely to be different to those involved in commuting safely.

Motor racing places additional physical and mental demands on drivers, who are required to execute numerous motor and cognitive skills simultaneously, while travelling at speeds often exceeding 150 km · hr⁻¹ (9, 13). The ultimate goal in motor-racing is for the driver to achieve the fastest lap times possible, by pushing the vehicle to its limits (16). This may in part be achieved by drivers adopting a racing trajectory that allows them to take corners or navigate specific sectors of the track in the fastest possible way (2, 11). These variables (i.e. lap/sector times, relative vehicle position tracking) can be easily captured in simulated racing environments by the computer software and, hence can be assessed in terms of their sensitivity and reliability. Doing so will provide greater confidence in the interpretation of future studies assessing factors (e.g. hydration status) that my influence motor-racing performance.

Therefore, the aim of this study was to establish the reliability and sensitivity of selected simulated driving performance indicators employed in a motor racing context; specifically, drivers’ lap time, sector time (i.e. for a dedicated portion of the track) and vehicle positioning and speed relative to the apex of a track corner.

METHODS

Participants

Recruitment involved a combination of convenience and snowball sampling. Inclusion criteria required participants to be healthy males, over 18 years of age. All participants completed a medical screening and signed informed consent before commencing the study. Ethics approval was granted by the University’s Human Ethics Committee (Protocol Number: PBH/57/13/HREC). Thirty-six healthy males were recruited (age: 26.5 ± 8.1 y, body mass: 75.6 ± 12.2 kg, BMI: 24.0 ± 3.2 kg · m⁻²; values are mean ± SD).

Protocol

Participants attended the laboratory once to complete four simulated motor-racing drives (2 initial drives and 2 repeat drives) separated by a 1 h period. Participants completed each pair of drives under two conditions. One drive was completed with participants wearing Fatal Vision Goggles (FVG) (estimated breath alcohol concentration (BrAC), 0.070 – 0.010%) and the other drive without (Normal). FVG were used to create a visual disturbance similar to that observed with alcohol consumption, with the intention to induce deterioration in driving performance (10). Sensitivity was assessed by observing changes in performance between the driving conditions (FVG vs Normal), whilst reliability of performance measures was determined by comparing the same driving conditions (e.g. FVG initial vs FVG repeat). Both initial and repeat drives were completed in a randomized, counterbalanced order.

Participants were instructed to abstain from alcohol for 24 h, and caffeine-containing substances and moderate-strenuous exercise for 12 h prior to the experimental trial. During the 12 h period immediately preceding the trial, participants were asked to follow a set fluid regime to assist with maintaining hydration status. The fluid regime involved consuming: 1000 ml of water between 1900 – 2000 h and an additional 500 ml, 2 h prior to the commencement of the trial. Participants were encouraged to follow their normal dietary behaviors prior to trials, but were asked to report to the lab following a 2 h fast from food and beverages, with the exception of water consumption.

Prior to commencing the trial participants provided a breath sample for confirmation of 0.000% BrAC using a calibrated police grade breathalyzer (Alcolizer LE5, Australia). Verbal compliance to pre-trial standardization procedures of exercise and fasting were confirmed, prior to participants providing a urine sample for the determination of urine specific gravity (USG) (UG-α, Atago Co. Ltd, Japan). Participants registering a USG ≥ 1.020 were deemed hypohydrated and provided with a bolus of water to consume (500 ml) in 15 min, prior to a 30 min rest (this occurred in n = 2 participants). In these individuals a subsequent urine sample was collected for re-testing USG. Once a USG < 1.020 had been established, a finger prick sample of blood was collected (Accu-Chek Performa II, Sydney, Australia) for determination of blood glucose level (BGL) to ensure participants were not in a hypoglycemic state (< 4.0 mmol · L⁻¹).

Participants were then provided an opportunity to become familiar with the driving simulator, driving scenario and FVG equipment. For familiarization on the driving simulator, participants completed a 5-lap drive of the motor racing circuit (~5 min), with the opportunity to complete multiple attempts. Once the participant was comfortable using the driving simulator, they were asked to complete the first of their four, 5-lap experimental drives. Participants’ were encouraged to complete each drive to the best of their ability aiming for consistently fast lap times throughout the drive. The second drive was completed following a short rest break (~2 min). Once two drives had been completed, participants were provided a 1 h rest break and 500 ml of water was given to all participants to standardize fluid intake during the break. At the conclusion of this period, participants provided a urine specimen and finger prick blood sample to retest USG and BGL’s respectively, prior to their final two drives.

A computerized driving simulation task was used to measure driving performance (SCANeR studio simulation engine, v1.2r95, OKTAL, Paris, France). The hardware and software components of the driving simulator have been previously described (6). Participants drove a course of ~2.3 km, which took about ~5 min to complete (5 laps). The course consisted of an initial straight section of ~200 m followed by 8 corners of varied acuity (5 right turns & 3 left turns), and one chicane. A sector time was recorded between corners 5 and 6 (~270 m) and a circular red marker was located on the road surface ~900 mm (. car width) offset from the midpoint of the inner ripple strip boundary. Lane width was uniform for the entire course (~12 m), surrounded by a 1.8 m ripple strip, which was elevated ~5 cm. To assist with course familiarization each corner was individually numbered, road cones were also placed at the midpoint of each corner. Participants were instructed to drive in a competitive manner (i.e. as fast as possible within the confines of the circuit). Lap time (LT), sector time (ST), position displacement to the red marker on the first corner (PD), and vehicle Speed at PD were recorded each lap. For the measure of PD, participants were instructed to drive the center of the vehicle directly over the red marker on each lap. Distance was determined by measuring the center of the rear axle of the vehicle to the dedicated road maker (this would place the right car tires on the edge of the corner ripple strip).

The Stanford Sleepiness Scale (SSS) (15) was administered to participants’ to assess their degree of sleepiness before commencing initial and repeat drives. Degree of sleepiness was ranked from 1 (feeling active, vital, alert, or wide awake) to 7 (no longer fighting sleep, sleep onset soon; having dream like thoughts).

Statistical Analysis

All statistical analyses were completed using SPSS Statistics for Windows, Version 22.0 (SPSS Inc., Chicago, IL, USA). Mean LT, mean ST, mean PD, and mean Speed at PD were calculated using laps 2–5. Lap one data was omitted to account for the proximity of the starting position to corner one (vehicle unable to reach maximum speed). Data from all participants (n = 36) were initially analyzed. To assess the influence of driver ability on the sensitivity and repeatability of performance measures, subsequent analyses were conducted on faster drivers (i.e. approximately half the group that performed with mean LT ≤ 70.00 s). All measures were examined for normality using the Shapiro-Wilk test prior to subsequent analysis. To assess learning effects within drives (i.e. the influence of completing increasing no. of laps within a drive and therefore track familiarity), all outcome variables were initially assessed for differences by driving lap using one-way repeated measures analysis of variance (ANOVA). Coefficient of variation (CV) for all performance variables was calculated between laps for both conditions (i.e. FVG and Normal) using the formula: within drive standard deviations divided by within drive mean of laps 2–5. Paired samples t-tests were used to determine differences between mean LT, mean ST, mean PD, and mean Speed at PD for drives under both conditions (i.e. FVG and Normal). Differences were considered significant at p < 0.05. Test-retest reliability was established if there was no significant difference identified for initial vs repeat drives completed under the same conditions. Sensitivity of performance measures were established if FVG drives were significantly different from Normal drives. All data is reported as mean ± standard deviation.

RESULTS

All participants completed the experimental drives with no reports of simulator sickness. Initial analyses of within drive variability indicated no difference in any of the performance measures as a result of increasing lap completion (p > 0.05), irrespective of analysis being completed on the full participant sample (n = 36) or the subset of faster drivers (n = 20). The CV of performance measures for the full participant group were; LT: FVG = 1.5%, Normal = 1.6%; ST: FVG = 3.1%, Normal = 3.6%; PD: FVG = 52.7%, Normal = 58.6%; and Speed at PD: FVG = 7.6%, Normal = 5.9%. For the subset of participants, CV was calculated as; LT: FVG = 1.2%, Normal = 1.1%; ST: FVG = 2.7%, Normal = 2.2%; PD: FVG = 51.6%, Normal = 63.4%; and Speed at PD: FVG = 7.9%, Normal = 5.9%.

Mean performance outcomes of all 36 participants over the four drives are summarized in Table 1. Significantly faster LT were observed in the initial Normal drives compared to initial FVG drives (p < 0.001), however no difference was noted for LT, ST or PD under the same conditions for repeat drives. The average of all FVG and Normal drives (i.e. initial drives and repeat drives together) indicated that participants drove significantly faster under Normal conditions, for both mean LT (p = 0.021) and mean ST (p = 0.039) measures. No identifiable difference was observed for PD or Speed at PD (p > 0.05). Performance measures for initial and repeated drives (n = 36) under the same conditions revealed a significant difference in LT and ST between drives under FVG conditions (p = 0.007), while PD and Speed at PD remained unaffected. No significant difference was identified between initial and repeat drives for all performance measures during Normal conditions (p > 0.05). No differences were observed for SSS administered pre-initial drives and pre-repeat drives (p > 0.05).

Table 1.

Performance measures of initial, repeat and averaged drives for FVG and Normal conditions (n = 36).

Variable	Initial Drives		Repeat Drives		Averaged Drives
	FVG	Normal	FVG	Normal	FVG	Normal
LT (s)	70.50 (2.07)	69.92 (2.26)a	69.84 (1.81)c	69.73 (2.18)	70.17 (1.82)	69.83 (2.15)b
ST (s)	12.08 (0.59)	11.94 (0.69)	11.86 (0.36)c	11.71 (0.58)	11.97 (0.44)	11.82 (0.55)b
PD (m)	1.89 (1.04)	1.64 (0.72)	2.00 (1.05)	2.02 (1.05)	1.95 (0.89)	1.83 (0.67)
Speed (km · h−1)	115.8 (13.1)	116.4 (10.7)	114.6 (9.8)	117.5 (8.2)	115.2 (11.5)	117.0 (9.5)

Note:

^aSignificant difference Initial LT for FVG vs Normal (p < 0.001).

^bSignificant difference Average LT and ST for FVG vs Normal (p = 0.021 and p = 0.039, respectively). All other comparisons p > 0.05. Reliability:

^cSignificant difference LT for FVG Initial vs Repeat (p = 0.007).

^cSignificant difference ST for FVG Initial vs Repeat (p = 0.005). All other comparisons p > 0.05.

LT, Lap Time; ST, Sector Time; PD, Position Displacement; Speed, Vehicle speed at PD marker. Data are Mean (SD).

Performance of the 20 participants that produced mean LT of ≤ 70.00s (meeting the performance cut-off) over the four drives is summarized in Table 2. LT were significantly faster during Normal initial and repeat drives compared to FVG trials (Initial p < 0.001, repeat p = 0.016). However, no difference was observed between either FVG or Normal conditions across initial and repeat drives for ST, PD and Speed at PD. The average of all FVG and Normal drives (i.e. initial FVG drives and repeat FVG drives together) indicated significantly faster LT (p < 0.001) and ST (p = 0.016) under Normal conditions. No notable difference was observed for PD or Speed at PD across the two conditions (p > 0.05). For the 20 participants meeting the performance cut-off, no significant difference was identified for LT, ST and Speed at PD under each condition (FVG and Normal) (p > 0.05). However, there was a significant difference between conditions for PD (p = 0.017).

Table 2.

Performance measures of initial, repeat and averaged drives for FVG and Normal conditions (n = 20).

Variable	Initial Drives		Repeat Drives		Averaged Drives
	FVG	Normal	FVG	Normal	FVG	Normal
LT (s)	69.34 (1.10)	68.58 (0.90)a	69.00 (1.17)	68.42 (0.81)a	69.17 (0.97)	68.50 (0.79)b
ST (s)	11.90 (0.44)	11.68 (0.46)	11.72 (0.38)	11.42 (0.53)	11.81 (0.36)	11.55 (0.37)b
PD (m)	1.69 (0.79)	1.65 (0.63)	2.17 (1.09)c	2.15 (1.10)	1.93 (0.86)	1.90 (0.67)
Speed (km · h−1)	115.4 (10.1)	117.3 (7.7)	115.7 (9.5)	118.1 (8.4)	115.5 (9.7)	117.7 (7.9)

Note:

^aSignificant difference Initial LT for FVG vs Normal (p < 0.001) and Repeat LT for FVG vs Normal (p = 0.016).

^bSignificant difference Average LT and ST for FVG vs Normal (p < 0.001 and p = 0.016, respectively). All other comparisons p > 0.05. Reliability:

^cSignificant difference PD for FVG Initial vs Repeat (p = 0.017). All other comparisons p > 0.05.

LT, Lap Time; ST, Sector Time; PD, Position Displacement; Speed, Vehicle speed at PD marker. Data are Mean (SD).

DISCUSSION

This preliminary study examined test-retest measures of driving performance from a simulated motor-racing scenario to determine the reliability and sensitivity of performance measures. Results from the study indicated that LT and ST were reliable and sensitive performance measures to a visual disturbance. However, PD was neither sensitive nor reliable and Speed at PD was not sensitive as driving performance parameters to this challenge.

In the present study, performance measures were assessed for sensitivity using FVG to induce impairment via a visual disturbance simulating the effects of alcohol. The driving task variables of mean LT and ST proved to be sensitive to this impairment, with significant differences detected between FVG and Normal conditions. These performance measures also demonstrated relatively low within drive variability, as indicated by low CV values (i.e. < 2% for LT and ~3% for ST). These findings provide confidence in that the measures assessed may be able to detect changes in simulated motor-racing performance following interventions that can result in cognitive impairment (e.g. dehydration). In contrast, PD and Speed at PD results were highly variable (i.e. CV > 50% for PD and ~7% for Speed at PD), thus was not considered as being either a sensitive or reliable measure of performance. One possible explanation for this finding is that participants applied a varied approach to positioning the vehicle into the corner that was best suited to their own driving style in order to achieve the fastest LT. However, analysis of Speed at PD indicated no correlation with improved LT. This measure may be more sensitive and reliable if it was integrated into a corner with greater influence on subsequent corner speed (impacting on sector/overall speed). Alternatively, participants could undertake a familiarization process that tracks their preferred racing line which would subsequently indicate the most frequent approach used to achieve fast LT for a particular individual. Using these data, a marker could be placed on a participant’s preferred racing line with their performance measured against their own driving style pre and post intervention. Unfortunately, this approach was beyond the scope of the methods employed.

Data from this study indicated a noticeable reduction in variance of some performance measures for the sub-sample group (n = 20) meeting the performance cut-off criteria (LT ≤ 70.00 s). For example, the standard deviations of the LT performance measure was approximately half of that observed for the entire sample. These findings are particularly important for subsequent research studies planning to investigate driving performance, highlighting that faster drivers are typically more consistent in repeating performance outcomes. Thus, employing a performance criterion cut-off to only include fast drivers provides the greatest opportunity to detect true effects of an intervention (e.g. dehydration) with the lowest participant sample size possible. Few previous studies have employed performance criteria to select high performance drivers. However, studies by Walker, Ackland, et al. (17) and Walker, Dawson & Ackland (18) were able to detect a difference in performance measures for recruited participants (n = 8) that had a minimum of 2 years competing as professional motorsport drivers. Collectively, these studies demonstrate the importance of establishing performance criteria to assist with variability reduction or recruiting drivers that have proven ability to perform at a high standard.

Overall, this preliminary investigation demonstrates that across a 1 h repeated test interval, LT and ST were sensitive and reliable to detecting changes induced by a visual disturbance and were therefore considered as measures to assess simulated motor-racing performance. To ensure the greatest likelihood of detecting performance changes, a criterion requiring participants to drive consistently fast LT should be employed in future investigations.