Abstract
[Purpose] The Trail Making Test part B (TMT-B) is used in evaluating driving abilities and includes testing for the executive function. A driving simulator version of this test (DTMT-B) was developed to measure drivers’ executive abilities in three-dimensional space. The purpose of the present study was to assess the validity of the DTMT-B for driving assessment. [Subjects] Thirty stroke patients and 65 healthy subjects were recruited. [Methods] Participants performed the TMT-B and DTMT-B. The DTMT-B was run on a driving simulator in which the individual performed a task on virtual roads connecting the lettered and numbered TMT-B points by simulated driving instead of connecting them with lines as in the paper or computerized TMT-B. Intra-class correlation coefficients (ICCs) were used to assess validities. Significant correlations were found between the TMT-B and DTMT-B. [Results] Participants performed the TMT-B and DTMT-B. Intra-class correlation coefficients (ICCs) were used to assess validities. Significant correlations were found between the TMT-B and DTMT-B. [Conclusion] The results suggest that the DTMT-B may be useful as part of driver screening assessment using a driving simulator for stroke patients and that it may also be used to assess the executive functions for healthy people.
Key words: Trail Making Test B, Driving simulator, Driving assessment
INTRODUCTION
Driving is a complex task requiring appropriate reactions to different situations on the road1) and depends on intact sensory and motor functions2). Driving performance is adversely affected by many reasons like visual, neurological, and age-related deficits3). Therefore, formulating a fitness to drive recommendation using psychological and neurological tests is important in keeping potentially unsafe drivers with neurological deficits off the roads4). Various off-road tests to assess an individual’s fitness to drive include Trail Making Test, Stroke Driver Screening Assessment, Useful Field of View test, Cognitive Behavioral Driver’s Inventory, DriveAble, Clock Drawing Test, DriveSafe/DriveAware, computerized sensorimotor and cognitive tests4). Among these tests, the Trail Making Test part B (TMT-B) has been demonstrated, by several researchers, to be a good neuropsychological test for assessing fitness to drive in a laboratory environment5).
The Trail Making Test (TMT) consists of part A and part B in some neuropsychological tools used in clinical assessment. TMT-B is known to be a simple but effective tool for assessment of cognitive abilities like executive function, visuo-conceptual function, visuo-motor tracking and attention6). Betz and Fisher7) suggested that TMT-B be used as a short test for predicting future traffic accidents or driving test failures and occasionally in the evaluation of elderly drivers. Many studies have regarded the TMT-B as an easy and short test with ability to predict fitness to drive in the elderly or in cognitively impaired drivers7,8,9). These studies reported significant association between drivers’ involvement in road traffic accidents and their poor performance on the TMT-B. Other studies have suggested that elderly drivers who perform poorly on the TMT-B had a higher risk of involvement in traffic accidents9,10,11). Although the TMT-B test may be a simple and useful measure of executive function as a predictor of an individual’s fitness to drive, it does not give a clear picture of the function and is not an easy test to administer.
Simulators offer many advantages over conventional computerized neuropsychological tests. This includes their ability to present virtual environments or create realistic situations. They also give a clear picture of the executive function and ensure uniform testing conditions. We, therefore, developed a new test called the Driving TMT-B (DTMT-B) that presented a virtual environment in three-dimensional spaces to test the executive functions of drivers. This study aimed to examine the validity of the DTMT-B as a predictor of an individual’s fitness to drive.
SUBJECTS AND METHODS
Thirty stroke patients (right hemiplegia, 13; left hemiplegia, 17; age, 56.10, ± 9.40, mean ± standard deviation) and 65 healthy control subjects (age, 43.45 ± 15.40) participated in this study. The Stroke Participants’ sitting balance were normal and their brunstrom stage of arm and hand were 2.8 (SD=1.49) and 2.77 (SD=1.52), respectively (Table 1). The participants were inpatients and outpatients recruited at the Korean National Rehabilitation Center. Patients who (1) had visual neglect or apraxia, (2) had limited range of motion, (3) had seizure or epilepsy, (4) could not sit unaided, (5) were pregnant, (6) had psychiatric disease, or (7) had ataxia, were excluded from the study.
Table 1. Demographic characteristics of study participants and task completion times.
| All (N=95) |
Stroke patients (N=30) |
Healthy participants (N=65) |
|
|---|---|---|---|
| Male, n (%) | 53 (55.8) | 20 (66.7) | 33 (50.8) |
| Female, n (%) | 42 (44.2) | 10 (33.3) | 32 (49.2) |
| Age (years) | 47.44 (14.95) | 56.10 (9.40) | 43.45 (15.40) |
| Education (years) | 13.57 (3.73) | 11.37 (4.00) | 14.58 (3.14) |
| KMMSE Score | 28.17 (2.35) | 26.40 (3.21) | 28.98 (1.14) |
| CTMT-B in seconds | 47.77 (3.55) | 85.80 (30.22) | 30.22 (13.83) |
| MAS | 1.60 (1.30) | ||
| Brunstrom Stage (Arm) | 2.80 (1.49) | ||
| Brunstrom Stage (Hand) | 2.77 (1.52) |
All values except gender are represented as Mean (standard deviation). KMMSE: Korean Mini-Mental Status Examination, CTMT-B: Computerized Trail Making Test B (sub-item of CPAD), MAS: Modified Ashworth Scale
All participants were provided a detailed description of the study and informed consent was obtained from all patients prior to participation. Human subject ethical approval was obtained from the Institutional Review Board of Korea National Rehabilitation Center (NRCIRB-2012-01-004) prior to conducting the study.
Participants were screened by using the Korean Mini-Mental Status Examination (KMMSE)12) before enrollment. All participants were briefly interviewed to obtain demographic information and neurological history.
The computerized version of Trail Making Test part B (CTMT-B) emulates the paper and pencil version of the test on a computer screen. The Cognitive Perceptual Assessment for Driving (CPAD) developed by the Korea National Rehabilitation Center has one such CTMT-B embedded as a subtest. The CPAD is a computer-based assessment tool to assess the driving capacity of people with neurological deficits13). The CTMT-B presents 25 items including numbers and letters randomly scattered on an LCD screen. Participants are required to click alternately on numbers and letters in numerical and alphabetical order respectively. A total task completion time over 2 minutes was regarded as failing the CTMT-B. According to Park et al.13), the subjects’ CTMT-B scores were associated with driving performance such as car crash, vehicle positioning, and lane changes.
Traditionally, the TMT-B has been used as a measure of the executive function as well as of processing speed and visual scan14). It imposes strong cognitive demands.
The Korean version of TMT-B15) was used for the paper-and-pencil version of the test. The test required participants to connect a set of numbers and letters positioned randomly on an 8.5 by 11 inch sheet of paper with pencil lines. Participants were asked to alternate between numbers and letters while following numerical or alphabetical order respectively. Primary end point of the test was the time taken for completion of the task16). During the test, errors made by the participants were pointed out by the examiners and participants were redirected to the last correct selection and asked to continue the task from that point. The stopwatch used to measure the performance time continued even when participants made errors. A total task completion time exceeding 5 minutes was regarded as failing the TMT-B15).
DTMT-B is a driver screening assessment performed with a driving simulator incorporating a modified version of the TMT-B. The simulator set up consists of a table, two LED screens, a small steering wheel, an accelerator and a brake. All subjects were explained the task clearly and were allowed to perform in a practice session before the actual DTMT-B test. Participants performed a simulated driving on a virtual road of 2,674 m presented on one LED screen, while a map showing a moving point indicating the current location as well as the final destination were projected on the second LED screen. The virtual roads were in effect the TMT points and this task expected participants to drive along them rather than connect them with lines as in the paper or computerized TMT. Similar to the TMT-B, the DTMT-B required connecting, that is, driving alternately through the locations marked by numbers and letters in numerical and alphabetical order respectively. The driving speed was fixed at 40 km/h and was determined as optimal based on a pilot test conducted prior to the present study. When participants made errors, the computer program indicated the error and relocated the car to the last correct place. Errors made by a participant reflected in their total score as total task completion time. A total task completion time of over 15 min was regarded as failing the DTMT-B (Fig. 1).
Fig. 1.
Driving tools for Driving Trail Making Test-B (DTMT-B)
The dependent measures were the times required to complete the CTMT-B, the TMT-B and the DTMT-B respectively. For convergent validity, our study performed correlation analyses among the DTMT-B, the KMMSE, and the CTMT-B using the Pearson’s product correlation method. In the validation study, intra-class correlation coefficients (ICCs) were used for each task. ICCs values were graded as “poor to moderate” (less than 0.75) good (greater than 0.75) and “excellent” (greater than 0.90)17, 18).
RESULTS
Thirty stroke patients (right hemiplegia, 13; left hemiplegia, 17; age, 56.10 ± 9.40 years) and 65 healthy control subjects (age, 43.45 ± 15.40 years) participated in this study. The stroke participants’ sitting balance were normal and their brunstrom stage of arm and hand were 2.8 (SD=1.49) and 2.77 (SD=1.52) respectively (Table 1). The stroke group’s task completion time was significantly longer than that of healthy participants (Table 2, p<0.001). Significant correlations were found between TMT-B and DTMT-B (Table 3, r = 0.732, p < 0.001). The DTMT-B had moderate correlation with the KMMSE(r=–0.501, p<0.001). The ICC values were 0.66 (95% CI =0.465–0.763, p<0.001) among all participants, 0.574 (95% CI=0.106–0.797, p=0.012) among stroke patients, and 0.499 (95% CI=0.178–0.694, p=0.003) among healthy controls (Table 4).
Table 2. Comparison between stroke patients and healthy participants.
| Measures | Stroke patients Mean (SD) |
Healthy participant Mean (SD) |
p value |
|---|---|---|---|
| TMT-B (seconds) | 233.10 (83.12) | 108.38 (66.78) | <0.001 |
| DTMT-B (seconds) | 723.03 (246.38) | 416.25 (187.97) | <0.001 |
TMT-B: Trail making test part B, DTMT-B: driving simulator running a modified version of the TMT-B
Table 3. Pearson correlation coefficients between tests (N = 95).
| TMT-B | CTMT-B | KMMSE | |
|---|---|---|---|
| DTMT-B | 0.732* | 0.729* | −0.501* |
TMT-B: Trail making test part B, CTMT-B: Computerized Trail Making Test B (sub-item of CPAD), KMMSE: Korean Mini-Mental Status Examination, DTMT-B: driving simulator running a modified version of the TMT-B. * p<0.001
Table 4. Interclass correlation coefficients between TMT-B and DTMT-B across groups.
| ICCs | 95% Confidence Intervals | |||
|---|---|---|---|---|
| Lower | Upper | |||
| All participants | ||||
| TMT-B vs. DTMT-B | 0.644 | 0.465 | 0.763 | * |
| Stroke patients | ||||
| TMT-B vs. DTMT-B | 0.574 | 0.106 | 0.797 | * |
| Healthy controls | ||||
| TMT-B vs. DTMT-B | 0.499 | 0.178 | 0.694 | * |
TMT-B: Trail making test part B, DTMT-B: driving simulator running a modified version of the TMT-B, ICC: Interclass correlation coefficient, CI: confidence interval. *: p<0.05
DISCUSSION
This study found that the newly developed test DTMT-B test, which used a driving simulator, was a reliable instrument for evaluating cognitive abilities like executive function. The DTMT-B was demonstrated to be a good tool for evaluating cognitive abilities in a more realistic environment of driving than done by other variations of the TMT-B test. Many researchers have used driving simulators in the assessment of driving performance in individuals with various diseases and in elderly drivers because of the advantages offered by them9, 19,20,21). Moreover, many studies have found a correlation between driving simulator training and improvement in driving skills22, 23). In one of these studies, Roenker et al.23) reported that elderly drivers improved their driving performance skills. Gamache, Lavalliere, Tremblay, Simoneau and Teasdale reported a case study in which driving skills significantly improved in a patient with total brain injury (TBI) after 1 year of simulator training. This study also found an improvement in the patient’s driving skills and safety in real driving situations24).
The ICCs between DTMT-B and TMT-B was reliable (ICC=0.644). ICC is defined as the correlation between one measurement on a goal (either a single rating or a mean of several ratings) and another measurement obtained on that goal. In our study the ICC of all groups was reliable though it was different between stroke patients and healthy participants respectively (ICC = 0.574, 0.499). Even though DTMT-B was not found to be a highly reliable tool, the ICCs between two groups were valid regarding the discrete discrepancy in their cognitive abilities. The DTMT-B was more difficult to perform and complete than the TMT-B. However, it required the subjects to identify the route path in the 3 dimensional environmental spaces, a task in which the executive function must be employed.
Wang et al.25) developed the Rout Map Recall Test (RMRT) and validated it against the TMT-B. Cronbach’s alpha of the RMRT was 0.752 (p<0.001) and its correlation with the Chinese version of the TMT-B was −0.732 (p<0.001).
Though our study is similar in terms of using the TMT-B for testing participants’ memory, attention, and executive function, we combined a driving simulator with the TMT-B. While Wang et al.25) used a 2-dimensional, planar simulation, we used a simulated 3-dimensional space.
Driving simulator tests are safe as they avoid the risks associated with real driving, such as traffic accidents. They also have higher face validity than pencil-and-paper tests, such as the TMT-B26, 27). The use of a driving simulator also offers the possibility to observe and evaluate clients’ driving abilities in a virtual reality version of real road situations. A variety of studies have used driving simulators for evaluating and training elderly or people with disabilities28,29,30,31).
This study analyzed the DTMT-B which combined a driving simulator with the TMT-B, and the CTMT-B, a computerized version of the TMT-B, for the safe evaluation of drivers’ cognitive functions. This study also analyzed the reliability of these tests by comparing scores obtained in the paper version of the TMT-B.
The TMT is a simple and short neuropsychological test that assesses cognitive abilities such as visuo-conceptual and visuomotor tracking, sustained attention, and task alteration ability6). Other researchers have suggested that the TMT-B assesses processing speed, visual and motor integration, symbol recognition and sequencing, executive function and in addition to other abilities7). Ricker and Axelrod32) developed an oral version of the TMT (OTMT) that had significant correlation with the TMT-B (r = 0.72). In addition, Messinis et al.33) reported that the Color Trail Test had a strong correlation with the Greek TMT. Our research produced results similar to those of previous studies. Correlations between DTMT-B and TMT-B, between DTMT-B and CTMT-B were very strong (r=0.732, 0.729 respectively). This suggests that assessment of executive function for fitness to drive recommendations could be evaluated using the simulator version of the TMT-B.
This study has some limitations. Firstly, the small size of 30 stroke patients and 65 healthy participants is small and the results may not be representative of a larger population. Secondly, age and education level may have considerably influenced the subject’ DTMT-B performance in the present study.
The present study proved that the DTMT-B was effective as part of driver screening assessment for stroke patients and healthy participants for assessment of cognitive performance related to the executive function. This study was successful in developing the DTMT-B as a 3-dimensional virtual environment to test the executive functions of drivers and examined its validity. The DTMT-B strongly and reliably correlated with the TMT-B. The DTMT-B may be useful as part of driver screening assessment to assess the executive functions of stroke patients as well as healthy people.
Acknowledgments
This research was supported by a grant (12-B-01) by Korea National Rehabilitation Research Institute.
REFERENCES
- 1.Yoo I, Kim EJ, Lee JH: Effects of chewing gum on driving performance as evaluated by the STISIM driving simulator. J Phys Ther Sci, 2015, 27: 1823–1825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Shin HK, Lee HC: Characteristics of driving reaction time of elderly drivers in the brake pedal task. J Phys Ther Sci, 2012, 24: 567–570. [Google Scholar]
- 3.Mathias JL, Lucas LK: Cognitive predictors of unsafe driving in older drivers: a meta-analysis. Int Psychogeriatr, 2009, 21: 637–653. [DOI] [PubMed] [Google Scholar]
- 4.Kay LG, Bundy AC, Clemson L, et al. : Contribution of off-road tests to predicting on-road performance: a critical review of tests. Aust Occup Ther J, 2012, 59: 89–97. [DOI] [PubMed] [Google Scholar]
- 5.Devos H, Akinwuntan AE, Nieuwboer A, et al. : Screening for fitness to drive after stroke: a systematic review and meta-analysis. Neurology, 2011, 76: 747–756. [DOI] [PubMed] [Google Scholar]
- 6.Vazzana R, Bandinelli S, Lauretani F, et al. : Trail Making Test predicts physical impairment and mortality in older persons. J Am Geriatr Soc, 2010, 58: 719–723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Betz ME, Fisher J: The Trail-making Test B and driver screening in the emergency department. Traffic Inj Prev, 2009, 10: 415–420. [DOI] [PubMed] [Google Scholar]
- 8.Classen S, Horgas A, Awadzi K, et al. : Clinical predictors of older driver performance on a standardized road test. Traffic Inj Prev, 2008, 9: 456–462. [DOI] [PubMed] [Google Scholar]
- 9.Stutts JC, Stewart JR, Martell C: Cognitive test performance and crash risk in an older driver population. Accid Anal Prev, 1998, 30: 337–346. [DOI] [PubMed] [Google Scholar]
- 10.De Raedt R, Ponjaert-Kristoffersen I: The relationship between cognitive/neuropsychological factors and car driving performance in older adults. J Am Geriatr Soc, 2000, 48: 1664–1668. [DOI] [PubMed] [Google Scholar]
- 11.Lundberg C, Hakamies-Blomqvist L, Almkvist O, et al. : Impairments of some cognitive functions are common in crash-involved older drivers. Accid Anal Prev, 1998, 30: 371–377. [DOI] [PubMed] [Google Scholar]
- 12.Han C, Jo SA, Jo I, et al. : An adaptation of the Korean mini-mental state examination (K-MMSE) in elderly Koreans: demographic influence and population-based norms (the AGE study). Arch Gerontol Geriatr, 2008, 47: 302–310. [DOI] [PubMed] [Google Scholar]
- 13.Park SW, Choi ES, Lim MH, et al. : Association between unsafe driving performance and cognitive-perceptual dysfunction in older drivers. PM R, 2011, 3: 198–203. [DOI] [PubMed] [Google Scholar]
- 14.Sánchez-Cubillo I, Periáñez JA, Adrover-Roig D, et al. : Construct validity of the Trail Making Test: role of task-switching, working memory, inhibition/interference control, and visuomotor abilities. J Int Neuropsychol Soc, 2009, 15: 438–450. [DOI] [PubMed] [Google Scholar]
- 15.Seo EH, Lee DY, Kim KW, et al. : A normative study of the Trail Making Test in Korean elders. Int J Geriatr Psychiatry, 2006, 21: 844–852. [DOI] [PubMed] [Google Scholar]
- 16.Hanks RA, Millis SR, Ricker JH, et al. : The predictive validity of a brief inpatient neuropsychologic battery for persons with traumatic brain injury. Arch Phys Med Rehabil, 2008, 89: 950–957. [DOI] [PubMed] [Google Scholar]
- 17.Sackley CM, Hill HJ, Pound K, et al. : The intra-rater reliability of the balance performance monitor when measuring sitting symmetry and weight-shift activity after stroke in a community setting. Clin Rehabil, 2005, 19: 746–750. [DOI] [PubMed] [Google Scholar]
- 18.Horimoto Y, Osuda Y, Takada C, et al. : Reliability of two protocols for measuring chest wall dimensions in the transverse plane in individuals with severe motor and intellectual disabilities. J Phys Ther Sci, 2011, 23: 221–224. [Google Scholar]
- 19.Devlin A, McGillivray J, Charlton J, et al. : Investigating driving behaviour of older drivers with mild cognitive impairment using a portable driving simulator. Accid Anal Prev, 2012, 49: 300–307. [DOI] [PubMed] [Google Scholar]
- 20.Takasaki H, Treleaven J, Johnston V, et al. : Assessment of driving-related performance in chronic whiplash using an advanced driving simulator. Accid Anal Prev, 2013, 60: 5–14. [DOI] [PubMed] [Google Scholar]
- 21.Horberry T, Inwood C: Defining criteria for the functional assessment of driving. Appl Ergon, 2010, 41: 796–805. [DOI] [PubMed] [Google Scholar]
- 22.Crundall D, Andrews B, van Loon E, et al. : Commentary training improves responsiveness to hazards in a driving simulator. Accid Anal Prev, 2010, 42: 2117–2124. [DOI] [PubMed] [Google Scholar]
- 23.Roenker DL, Cissell GM, Ball KK, et al. : Speed-of-processing and driving simulator training result in improved driving performance. Hum Factors, 2003, 45: 218–233. [DOI] [PubMed] [Google Scholar]
- 24.Gamache PL, Lavallière M, Tremblay M, et al. : In-simulator training of driving abilities in a person with a traumatic brain injury. Brain Inj, 2011, 25: 416–425. [DOI] [PubMed] [Google Scholar]
- 25.Wang TY, Kuo YC, Ma HI, et al. : Validation of the route map recall test for getting lost behavior in Alzheimer’s disease patients. Arch Clin Neuropsychol, 2012, 27: 781–789. [DOI] [PubMed] [Google Scholar]
- 26.Lee HC, Lee AH, Cameron D, et al. : Using a driving simulator to identify older drivers at inflated risk of motor vehicle crashes. J Safety Res, 2003, 34: 453–459. [DOI] [PubMed] [Google Scholar]
- 27.Ivancic K, IV, Hesketh B: Learning from errors in a driving simulation: effects on driving skill and self-confidence. Ergonomics, 2000, 43: 1966–1984. [DOI] [PubMed] [Google Scholar]
- 28.Lengenfelder J, Schultheis MT, Al-Shihabi T, et al. : Divided attention and driving: a pilot study using virtual reality technology. J Head Trauma Rehabil, 2002, 17: 26–37. [DOI] [PubMed] [Google Scholar]
- 29.Hault-Dubrulle A, Robache F, Pacaux MP, et al. : Determination of pre-impact occupant postures and analysis of consequences on injury outcome. Part I: a driving simulator study. Accid Anal Prev, 2011, 43: 66–74. [DOI] [PubMed] [Google Scholar]
- 30.Freund B, Colgrove LA, Burke BL, et al. : Self-rated driving performance among elderly drivers referred for driving evaluation. Accid Anal Prev, 2005, 37: 613–618. [DOI] [PubMed] [Google Scholar]
- 31.Akinwuntan AE, De Weerdt W, Feys H, et al. : Effect of simulator training on driving after stroke: a randomized controlled trial. Neurology, 2005, 65: 843–850. [DOI] [PubMed] [Google Scholar]
- 32.Ricker JH, Axelrod BN: Analysis of an Oral Paradigm for the Trail Making Test. Assessment, 1994, 1: 47–52. [DOI] [PubMed] [Google Scholar]
- 33.Messinis L, Malegiannaki AC, Christodoulou T, et al. : Color Trails Test: normative data and criterion validity for the greek adult population. Arch Clin Neuropsychol, 2011, 26: 322–330. [DOI] [PubMed] [Google Scholar]