Design of an experimental protocol to examine medication non-adherence among young drivers diagnosed with ADHD: A driving simulator study

Abstract

The diagnosis of ADHD among teens and young adults has been associated with a higher likelihood of motor vehicle crashes. Some studies suggest a beneficial effect of ADHD medication but the exact efficacy is still being debated. Further, medication adherence, which is low in this age group, can further reduce effectiveness. Our long-term objective is to reduce unsafe driving among drivers with ADHD by detecting medication non-adherence through driver behavior modeling and monitoring. As a first step, we developed the described lab study protocol to obtain reliable driver behavior data that will then be used to design and train behavior models built through machine learning. This experimental study protocol was developed to systematically compare driving behaviors under two medication conditions (before and after intake of medication) among young adults with ADHD and a control group of non-ADHD. A driving simulator was used to examine driving behaviors and interactions with traffic. The primary outcome was speed management for two comparisons (ADHD vs. non-ADHD and before vs. after medication), and secondary objectives involved understanding differences among the participants utilizing self-reported surveys about ADHD symptoms, drivers' knowledge, and perception about safety. The study protocol was designed to maximize participant safety and efficiency of data collection, as multiple measures were collected over two 2-h study visits. The sampled ADHD drivers were demographically and psychosocially similar but clinically different from the non-ADHD group. Overall, this protocol was effective in participant recruitment and retention, allowed staggered data collection, and can be incorporated in a subsequent clinical trial that examines the efficacy of a machine-learning based driver monitoring intervention.

1. Introduction

Attention-deficit/hyperactivity disorder (ADHD) is a prevalent neurobehavioral disorder in children and adolescents [16,38]. Approximately two-thirds of patients with a childhood diagnosis of ADHD continue to experience clinical symptoms into adulthood [6], intersecting with a period when many young adults start to drive independently. Drivers who are diagnosed with ADHD have shown significant driving impairments [1,2,19], including higher likelihood of motor vehicle crashes, speeding violations and poorer vehicle control, with a relative risk of 1.23 when controlling for exposure [39]. However, not all individuals with ADHD are affected uniformly [17] and it remains unclear the extent to which measures can be developed to distinguish between low and high risk drivers with ADHD on a group level and between low and high risk characteristics on an individual level [15].

Medication intervention, especially stimulant medication, appears to improve the driving deficits exhibited by ADHD drivers [17], although the exact efficacy is still being debated [3,5,8,11,12,24,37]. While the discrepancies may come from methodological limitations and sample size concerns, one general consensus is that the benefits and effects of medication on individuals with ADHD are not uniform [20], confirming the challenges for developing effective measures to distinguish between low and high risk drivers as well as strategic and sustainable treatment plans. In addition, medication adherence is a major problem in adolescence and young adulthood [18,32], as they transition from parent-managed medication to self-managed medication. Therefore, medication non-adherence can further degrade effectiveness of stimulant medication in reducing crash risk.

Despite these safety concerns, very little work has been done to evaluate interventions that can improve ADHD symptom management and driving safety during adolescence and young adulthood, when the crash risk is the highest [33]. To address this gap in knowledge, our long-term objective is to create a machine-learning based monitoring intervention to help manage ADHD symptoms while driving. Such a system is expected to effectively monitor driving behavior in situations where ADHD symptoms are under relative control and when they are not, as determined by levels of medication adherence. As the first step toward fulfilling this long-term goal, we have designed an experimental protocol that involved the use of a driving simulator and other assessment measures and allowed for the comparison of driving behaviors between medication conditions (before and after the consumption of daily stimulant ADHD medication) and groups of participants (with and without ADHD). The current paper reports the design of an experimental protocol for collecting reliable driver behavior data; the recruitment and assessment strategies for the study sample – young adults with and without ADHD; and the comparisons of clinical, psychosocial, and demographic characteristics between the two groups.

2. Research design and methods

2.1. Study design

The primary objective is to quantify the differences in vehicle control behaviors between two groups of participants – individuals with and without ADHD – as well as between two medication conditions (before and after medication administration) among individuals with ADHD. Traditional statistical methods and data mining techniques will be used to compare and contrast patterns of driving behaviors. The secondary objectives include conducting exploratory analysis to examine potential mechanisms and covariates that may explain the behavioral differences between the two groups and the two medication conditions.

2.2. Recruitment strategies

Participants (individuals with self-reported ADHD and zip code matched individuals without ADHD who served as controls) were recruited through a variety of clinical and community settings. These included posting flyers at universities, university counseling centers, bus stops, coffee shops, grocery stores, psychology department subject pool, emailing listservs of undergraduate students and university disability services office, and word-of-mouth referrals from students and colleagues. The study received Institutional Review Board approval from the first author's university.

2.3. Eligibility criteria

ADHD participants: Eligible participants were adults 18–24 years of age who had a confirmed diagnosis of ADHD, had a current prescription for stimulant medication for ADHD, held a restricted or an unrestricted driver's license, had normal or corrected-to-normal vision, and had normal hearing abilities. The exclusion criteria were self-reported pregnancy (females), self-reported neurodevelopmental disorders, intellectual disabilities, psychotic disorders, bipolar disorder, or seizure disorders, as well as participants with a confirmed diagnosis of ADHD but who took non-stimulant medication.

Non-ADHD participants: Eligible participants were adults 18–24 years of age who held a restricted or an unrestricted driver's license, had normal or corrected-to-normal vision, and had normal hearing abilities. The exclusion criteria were the same as those of ADHD participants with the addition of self-reported diagnosis of ADHD.

In addition, the Motion Sickness History Screening Form (MSHSF) [22,23] was used to assess the likelihood of experiencing simulator sickness (a form of motion sickness). The MSHSF includes questions about the frequency of getting carsick, seasick, and airsick. Based on the reported frequencies (a composite score of 7 or higher), we discouraged further participation. For participants who were discouraged but still wished to continue participation, the potential risks and safeguard measures were explained, and they were ensured they would be checked and monitored closely during the study.

2.4. Outcome measures

The primary outcome was participant's average speed while driving in a simulator. This and other associated variables were collected from a high-fidelity, motion-based driving simulator, sampled at 60-Hz. It has an open-cab configuration equipped with a motion-base system capable of a single degree of pitch motion and a 90 ± degree high-quality yaw motion, a 3-channel visual system covering 180-deg forward field-of-view, and a force-feedback steering wheel. The variables derived from the simulator included vehicle control variables (e.g., velocity, throttle, brake, pitch) and vehicle diagnostics variables (e.g., gear, engine RPM). Three cameras capturing the foot movement, over the shoulder view (steering wheel movement), and upper body and face view were also recorded. The traffic scenarios used in the current study as well as the process for computing driving behaviors were previously developed and validated [26,31,34].

To examine secondary objectives, the study included several self-reported surveys:

• a)Demographic information was collected using questions about socio-demographics, driving experience, driving history (accidents), licensure type, learning-to-drive experience (who taught them to drive?), as well as four validated rating scales: Safe Speed Knowledge Test (SSKT) [27], Driving Anger Scale (DAS) [13], Brief Sensation Seeking Scale (BSSS) [30,41], and Driving Behavior Survey (DBS) [9].
• b)ADHD history questionnaire was completed by each participant with ADHD and a friend or family member. These questions collected information about each participant's ADHD medication, symptom onset (age, symptom-related problems), and severity of problems or concerns currently caused by ADHD symptoms (when not taking the medication) in School, Work, Family Relationships, Social Relationships, and Self-Esteem categories.
• c)Conners' Adult ADHD Diagnostic Interview (CAADID) [14] was administered individually. It produced comprehensive demographic and developmental history to support a categorical diagnosis based upon the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV™) criteria for ADHD, during both adulthood and childhood. For screening purposes, both the quantitative and qualitative responses helped delineate the ADHD medical and symptom history by assessing each participant's demographic history, developmental course, ADHD risk factors, and comorbidity screening questions.
• d)Conners' Adult ADHD Rating Scale (CAARS) self-report and observer-report, screening versions [10] were administered individually. The screening versions consisted of 30 items about behaviors or problems sometimes experienced by adults. These rating scales were administered on-line via Multi-Health Systems' management program, and the calculated profile reports included normative T scores on inattentive symptoms, hyperactive-impulsive symptoms, total ADHD symptoms, and ADHD index.
• e)Post-drive survey was used after each driving simulator session. Participants were asked to rate the realism of the simulation and if there were concerns about the traffic scenarios. They were also asked to rate the percentage of time they were speeding, driving inside a lane, and following traffic rules.

2.5. Design of protocol

The study protocol (key elements presented in Table 1) was designed to maximize participant safety and efficiency of data collection. The safety aspect of the protocol included screening for proneness to simulator sickness, confirmation of ADHD medication, safety features in the driving simulator, and continuous monitoring of participants. The efficiency of data collection part of the protocol included web-based self-reports and observer reports, and staggered and balanced study procedures. The order of the medication condition was counter-balanced, and the order of the experimental drives in the driving simulator was randomized.

Table 1.

Key elements of the protocol and their rationale.

Key elements	Details	Justifications
Medication condition	Each ADHD participant was observed under two medication conditions - regular medication and delayed medication	Because of safety concerns, ADHD participants were to be dropped off and picked up by a friend or family member for the study visits
Scheduling of study visit	ADHD participants could be observed on two separate days up to 14 days apart (option 1) or on one day (option 2)	Two options for scheduling the study visits were offered to accommodate the chauffeuring friend or family member's availability
Consumption of ADHD medication	The medication was consumed under supervision	The intake and route of dosage were confirmed
Confirmation of ADHD medication	Participant's medication was first confirmed by a list of US Food and Drug Administration-approved stimulant medications. Post study visit, the medications were confirmed by the study pediatrician	This two-level of confirmation ensured that the medications were stimulants and that any variation of intake frequency and dosage was documented and confirmed
Screening for proneness to simulator sickness and constant monitoring during study visits	A validated survey was used. Participants who scored high on the survey were informed of their likelihood of developing simulator sickness while operating the simulator	Participants were informed of all safety features in the simulator, including a safety gate, wearing seatbelt, emergency stop buttons as well as experimenter's constant monitoring of sickness indicators. Short breaks and cold water were provided.
Use of web-based surveys	CAARS and history questionnaire could be completed online or by phone, by participants (self-report) and their friend or family member (observer report)	Added flexibility and options as to how and when these surveys could be completed
Staggered study procedures	CAADID and self-report surveys were conducted during the 1-h waiting period in the delayed medication condition, after the consumption of medication	This was to minimize the study duration

2.6. Study procedures

When interested individuals contacted the lab via email or phone, the experimenters first confirmed that they met the eligibility criteria. Once enrolled, the experimenters explained the two options for scheduling the study visits to ADHD participants (if option 1 was chosen, the two study visits were up to 14 days apart) and provided details of the lab location. All participants were informed that the purpose of the study is to understand driving behaviors and how the behaviors might be different under various driving and clinical conditions. They were notified that we would collect information about driving behaviors, their knowledge and perceptions about driving, demographic information, and health-related information. They were also informed of the study duration and amount of compensation. Additionally, participants with ADHD were told that they needed to provide the contact information of a friend or family member (observer) that we would then contact and ask to complete health-related surveys about the participant.

A reminder email and phone call were used two days prior to each scheduled study visit. Participants with ADHD were reminded that, because of safety concerns, they needed to be dropped off and picked up by a friend or family member. This could be the same observer who completed the surveys. They were also reminded to not take their ADHD medication on the day of the study visits and that they needed to bring the medication bottle with them.

Fig. 1 lists the steps of study procedures for each medication condition and study visit scheduling option for ADHD participants. A lab room was used for consenting, confirming eligibility criteria, and completing surveys and interviews. The driving simulator located in another lab room was used for the driving portion of the study visits. When participants entered the driving simulator room, they were immediately informed of the safety features. Once they sat in the driver's seat, the experimenters explained the dashboard, center console, and the layout of the simulator. Participants were instructed that they would see and interact with pre-determined traffic events in the virtual driving environment. The driving environment included rural and urban settings with corresponding buildings, trees, curves, intersections, and construction zones. They were asked to wear the seatbelt, obey traffic laws and drive in the simulator as they normally do in a real car on the road. Participants could adjust the seating to their preference; during this time, the experimenters put up curtains, started video recording, and loaded a practice drive that was used to familiarize participants with the dynamics of the simulator.

Fig. 1.

Steps of study procedures of regular and delayed medication conditions for ADHD participants.

Once the practice drive had ended and the participants' questions were answered, they proceeded to the main experimental drives. The order of the four experimental drives was randomized, and prior to starting each drive, participants were reminded to drive normally and legally when interacting with traffic. A short break was offered after the completion of each drive. Upon completing the four experimental drives, participants were asked to complete the post-drive survey and, if applicable, other self-reports. They were then debriefed, thanked, and compensated for their time.

For non-ADHD participants, the study procedures included consenting, completing MSHSF, going to the driving simulator, performing one practice drive and four experimental drives (following pre-determined, randomized order), and completing the post-drive survey and CAARS-self report. There was only one study visit and the duration was 2 h.

2.7. Analytic plan

2.7.1. Sample size justification

An a priori power analysis was performed using G*Power software. Effect size of 0.38 was calculated using average speed during simulator driving from a previous study [19]: 16.7 and 14.9 mph for individuals with and without ADHD, respectively (pooled standard deviation = 2.35). Sample size calculation was based on an alpha level of 0.05, a power level of 0.80, an effect size of 0.38, 2 groups, and 4 repetitions. Based on these parameters, a sample size of 36 participants (18 in each group) was needed to achieve the estimated power level.

2.7.2. Analysis for current paper

In this paper, demographic, clinical, and psychological characteristics for participants with and without ADHD were reported. Only one participant chose the option 2 study procedure; therefore, we did not conduct a sensitivity test on comparing the two options of study procedure. All data were analyzed using SPSS for Windows, version 24. All p-values were two-sided. Between-group differences were evaluated using Student's t, Chi-Square, and Fisher's exact tests as appropriate. For interval variables, Levene's test for equality of variances were conducted. All of the variables, except DBS Exaggerated subscale, had a p-value of > .05, suggesting that the variances of the population from which the two groups were drawn are the same.

2.7.3. Analysis of primary outcomes

The analysis will include average speed as well as other vehicle control variables (such as longitudinal braking profiles and lateral lane deviation) and incidents of crash and near-crash (defined as being within 2 m to another moving or parked vehicle) between the two groups. Results from the crash and near-crash comparisons were reported elsewhere [4]. In addition, speeding profiles during segments of driving (such as straight and curved road segments, gap negotiation with other vehicles) will be compared between the two medication conditions.

Several machine learning and data mining techniques are being used. Specifically, data is being examined at three levels of granularity: (1) whole drive, (2) drive segments (segments of 5–10 s), and (3) full resolution of the collected data (60 Hz). At each granularity, we employ two types of machine learning approaches. First, we employ supervised learning methods such as decision tree learning algorithms [36], support-vector machines (SVMs) [21], and deep neural networks [25] to identify signals in the data that are predictive of ADHD symptoms. At the full resolution time series, we also use time series data mining methods to contrast behavioral patterns during challenging maneuvers, such as lane switching, and the results were reported elsewhere [28]. Second, we use unsupervised learning techniques such as k-means [25] and visualization techniques such as t-SNE [29] to identify regularities in the data and form hypotheses that can later be tested. Moreover, some machine learning approaches can only be applied to the lower levels of granularity (segments and full resolution time series), since considering whole drives as the unit of study results in too small a dataset to detect any interesting trends.

Additionally, we have also been using the collected data to develop models of driving behaviors via learning from observation using custom-designed algorithms, such as those reported in our previous work [35,40]. The resulting driving agents with specific characteristics (un-medicated ADHD, medicated ADHD, and control/baseline non-ADHD) are compared in their performance to the participant data to identify discrepancies.

2.7.4. Analysis of secondary outcomes

Scores on CAARS, SSKT, BSSS, and DBS as well as post-drive surveys will be used as covariates to further explore differences between groups and medication conditions.

3. Results

Recruitment outcome and characteristics of the participants are presented in this section. Results of primary and secondary outcomes will be included in future publications.

3.1. Recruitment flow

Fig. 2, Fig. 3, for ADHD and non-ADHD participants, respectively, show the participants' recruitment process and confirmation of study eligibility criteria. Recruitment began in March 2017 and was completed in November 2017.

Fig. 2.

Enrollment of ADHD participants.

Fig. 3.

Enrollment of non-ADHD participants.

3.2. Participant characteristics

3.2.1. Demographic characteristics

Table 2 reports the demographic information of the participants who met study eligibility criteria. Participants with and without ADHD were similar across all demographic characteristics, except that there were more participants who self-identified being White in the ADHD group.

Table 2.

Demographic characteristics of participants with and without ADHD.

Data are mean ± SD or n		Participants with self-reported ADHD status (n = 21)	Participants with self-reported non-ADHD status (n = 17)	p-value ADHD vs. non-ADHD
Age (years)		21.2 ± 1.5	20.7 ± 2.0	.351
Gender (female)		10	10	.532
Race/Ethnicity	Asian	2	5	.207
	Black	1	3	.307
	White	17	7	.018
	Other	3	0	.238
	Hispanic	3	5	.426
Years of driving experience		4.5 ± 2.0	3.6 ± 1.6	.147
Driving history (# of participants who had …)	License suspended or revoked	2	1	1.000
	Fender benders	15	10	.228
	Stopped by police	12	6	.322
	Traffic tickets	8	5	.631
	At-fault crashes	1	1	1.000
Miles driven per week		126.4 ± 136.7	83.5 ± 105.2	.295
Education	High school	0	3	.099
	Some college	15	8
	Associate degree	1	3
	Bachelor degree	5	3
Employment status	Unemployed	9	4	.227
	Part-time	9	12
	Full-time	3	1
Student status	Not a student	2	2	.321
	Part-time	5	1
	Full-time	14	14

Note: Significant p-value in bold. Chi-square was used for White, Fender benders, Stopped by police, Traffic tickets, Education, Employment, and Student status. Fisher's Exact Test was used for Gender, Race/ethnicity, License suspended, and At-fault crashes. Student's T was used for Age, Years of driving, and Miles driven.

3.2.2. Clinical characteristics

The CAARS scores and ADHD history questionnaire for participants with and without ADHD are listed in Table 3. Participants with ADHD (self-report) had significantly higher T-scores of CAARS inattentive and hyperactivity/impulsive symptoms than participants without ADHD.

Table 3.

Clinical characteristics of participants with and without ADHD.

Data are mean ± SD		Participants with self-reported ADHD status (n = 21)		Participants with self-reported non-ADHD status (n = 17)	p-value ADHD vs. non-ADHD
		Self-report	Observer report
CAARS – Inattentive Symptoms		72.6 ± 12.2	61.9 ± 8.3	53.6 ± 9.4	<.001
CAARS – Hyperactivity/Impulsive Symptoms		66.5 ± 14.3	57.3 ± 10.4	50.8 ± 10.8	<.001
History questionnaire – level of impairment (0 = not at all, 3 = very much)	School	2.7 ± .6	2.3 ± .7	Did not collect
	Work	2.0 ± .9	1.5 ± .8
	Family relationship	1.5 ± .9	1.6 ± .9
	Social relationship	1.9 ± .7	1.7 ± 1.0
	Self-esteem	1.7 ± 1.0	1.6 ± .7

Note: Significant p-value in bold. Student's T was used for comparisons.

3.2.3. Psychosocial characteristics

The scores of Safe Speed Knowledge Test (SSKT), Driving Anger Scale (DAS), Brief Sensation Seeking Scale (BSSS), and Driving Behavior Survey (DBS) are listed in Table 4. Participants with and without ADHD had similar scores across the four questionnaires. Individual SSKT items were also compared: #13 and 14 showed significant differences between the two groups, t (36) = 2.80 and 2.27, p = .01 and .03, respectively, indicating that the amount of speed reduction was smaller (less safe speed management strategy) for participants with ADHD than for participants without ADHD.

Table 4.

Psychosocial characteristics of participants with and without ADHD.

Data are mean ± SD		Participants with self-reported ADHD status (n = 21)	Participants with self-reported non-ADHD status (n = 17)	p-value ADHD vs. non-ADHD
Safe Speed Knowledge Test (SSKT)		−8.5 ± 2.0	−9.4 ± 2.4	.253
		Ranges −18.1 to 0	Ranges −18.2 to −.9
Driving Anger Scale (DAS)		43.5 ± 9.6	39.1 ± 9.4	.168
Brief Sensation Seeking Scale (BSSS)		10.1 ± 4.9	7.5 ± 5.0	.115
Driving Behavior Survey (DBS)	Overall	3.5 ± .5	3.4 ± .6	.471
	Anxiety-based performance deficits subscale	2.6 ± 1.1	2.5 ± .8	.656
	Exaggerated safety/caution behavior subscale	4.8 ± .6	5.0 ± .9	.514
	Hostile/aggressive behavior subscale	3.1 ± 1.0	2.7 ± 1.0	.263

Note: Student's T was used for comparisons.

4. Discussion

ADHD symptom management among teens and young adults remains a challenging public health issue for families and affected individuals for several reasons: 1) not all individuals with ADHD have the same levels and kinds of impairment, 2) the effect of medication is not uniform, 3) late adolescence and young adulthood is developmentally a period of transition from parent-managed to self-managed treatment strategy, and 4) for the last subset of youth who become independent drivers, ADHD symptoms and impairments generally are manifested in suboptimal vehicle control behaviors [17,20,32,39]. We proposed to design and develop an effective, machine-learning based, in-vehicle monitoring intervention via driver behavior modeling and individuated feedback. As the first step toward the designing of such system, an experimental protocol was developed to allow the collection of reliable and objective driver behavior data and assessments from individuals with and without ADHD as well as before and after intake of medication.

As reflected in the results, ADHD and non-ADHD participants were significantly different in clinical characteristics, while being demographically and psychosocially similar. Effective recruitment and enrollment processes (age and zip-code matching), well-defined and standardized eligibility criteria, confirmations of medication, and timely data collection procedures all contributed to achieving this clear distinction between the two groups. We also paid special attention to participant susceptivity to simulator sickness or other factors that might undermine their behaviors during data collection, as reflected by the few participants being terminated or withdrawn.

This clear distinction on clinical characteristics is important for the subsequent modeling work. In order to develop reliable and predictive models, we look for similarities among participants as well as dissimilarities across participants within each group. The former will lead to the discoveries of contrastive patterns of unsafe driving behaviors from each group and representative patterns between groups. The latter will lead to establishing ranges of variability from one driver to another. In other words, the datasets from each sample ideally should be as homogeneous as possible while taking into account individual differences and variability as well as sample-based heterogeneity.

Having collected distinctive datasets from individuals with and without ADHD via the reported study protocol made a significant step toward achieving the long-term goal of designing an in-vehicle monitoring intervention. Extensive computational modeling work is underway to develop individual-level and group-level predictive models of driver behaviors while using self-reported ADHD assessments as covariates to test and refine the parameters and algorithms. Once the models are developed, further simulator testing and on-road studies will be conducted to evaluate the accuracy and efficacy of the in-vehicle monitoring intervention.

Nonetheless, the current study had a few methodological limitations that should be stated. By applying multiple criteria to determine participant eligibility, the evaluable sample size became relatively small; thus, the results should be interpreted with caution. While the eligibility criteria were stringent, they were necessary to target a late adolescent-young adult age group with specifically defined ADHD diagnosis for the study and screen out potential confounding variables such as comorbidity. Relying on self-reported ADHD clinical history, diagnosis, and medication was another limitation. Obtaining a more comprehensive profile of the clinical background (e.g., patient's complete medical records) should be considered for future studies. A third limitation was the use of a driving simulator as an assessment tool. A few participants who met eligibility criteria experienced simulator sickness and had to discontinue the study. Also, even though the simulator used in the study was a high-fidelity, motion-based system, it was still an artificial environment that had the resemblance of a driver's seat and forward roadway scenery. A more naturalistic and ecologically valid testing environment (e.g., on-road naturalistic observations through instrumented vehicles or a test-track environment) should be considered for future studies.

Despite these limitations, the current study summarized an experimental protocol that led to successful recruitment and assessments of young adult drivers diagnosed with and without ADHD. These groups differed clinically but were similar demographically and psychosocially, thus providing strong foundation for subsequent behavioral modeling work. The current study served as the first step toward our long-term objective of designing a machine-learning based, in-vehicle monitoring intervention for automatically detecting unsafe driving behaviors for adolescent and young adult drivers diagnosed with ADHD. We envision that this monitoring intervention will be part of a developmentally appropriate treatment plan as the intended users transition from family to self-management of ADHD during adolescence [7].