Abstract
[Purpose] This study examined the cut-off point of the Trail Making Test in predicting the risk of unsafe driving in stroke patients. [Subjects and Methods] A total of 81 stroke patients with a driver’s license participated in this study. The DriveABLE Cognitive Assessment Tool, Trail Making Test-A, and Trail Making Test-B evaluations were conducted in all participants. All participants were classified into the safety or risk groups based on the DriveABLE Cognitive Assessment Tool evaluation results. The Trail Making Test results underwent a receiver operating characteristic analysis in each group. [Results] The results of the receiver operating characteristic curve analysis showed that the cut-off point for Trail Making Test-A was 32 seconds and the cut-off point for Trail Making Test-B was 79 seconds. The positive predictive values of the Trail Making Test-A and Trail Making Test-B were 98.3% and 98.3%, respectively, and the negative predictive values of the Trail Making Test-A and Trail Making Test-B were 81.0% and 73.9%, respectively. [Conclusion] The Trail Making Test is a useful tool for predicting the risk of unsafe driving in stroke patients. This tool is expected to be used more actively for screening stroke drivers with respect to their cognitive function.
Key words: Cut-off, Stroke driver, Trail making test
INTRODUCTION
Driving cessation causes mobility decline, social isolation, and depression after stroke1). This neurological disease is associated with physical, cognitive, perceptual, and sensory dysfunctions2). Cognitive function is a particularly important element for driving safety in stroke patients3). Therefore, the evaluation system should test cognitive impairment, which is an invisible risk factor for unsafe driving4).
The on-road test is considered the “gold-standard” of testing driving function5). It is, however, difficult to widely use in the overall stroke patient population because of time-related and financial problems3). Alternatively, a simulator test system is used, but it is difficult to fully predict cognitive problems associated with driving6). Thus, driving-related cognitive assessment has been used as a modality for forecasting risk before driving7).
The most commonly used cognitive tests are TMT-A (Trail Making Test-A), TMT-B (Trail Making Test-B), UFOV (Useful Field of View Test), and MMSE-K (Mini Mental State Examination-K)7). The TMT measures attention, memory, sequencing, decision-making, and automatic thinking8, 9). Driving behavior of stroke drivers is influenced through sensory input, core cognitive processing, and higher-order processing. Core cognitive processing includes attention, perception, and memory while higher-order processing includes decision-making, planning, and automatic thinking3). Therefore, the TMT is a very useful evaluation to measure both core cognitive processing and higher-order processing7).
The TMT is known as a useful assessment tool that can predict unsafe driving in stroke and elderly drivers7, 10, 11). Additionally, if studies that can predict criterion validity are performed, it can be clearly used to predict the risk in a clinical trial because it provides a reference point for predicting the risk of unsafe driving. As an advantage, the TMT can determine the level of a patient’s risk of unsafe driving before a detailed driving assessment, such as the DCAT (The DriveABLE Cognitive Assessment Tool) for stroke and elderly drivers, is performed. Hence, studies have been performed that assessed the cut-off point for the TMT’s ability to predict the risk of unsafe driving in older drivers12, 13). In addition, a study that predicts the cut-off point for stroke drivers should be performed. This would make the TMT very useful for predicting the risk of unsafe driving in stroke drivers.
The purpose of this study is to identify the cut-off point to predict the risk of unsafe driving in stroke patients through the TMT evaluation.
SUBJECTS AND METHODS
Eighty-one subjects with stroke participated in the study. The study participants were receiving rehabilitation treatment at the K University rehabilitation center in Korea. Participants with a driver’s license, no visual problems, and no history of seizures or epilepsy within the last six months were included. All the subjects provided written informed consent according to the ethical principles of the Declaration of Helsinki. Table 1 presents the general and driving related characteristics of the participants.
Table 1. General characteristics of the participants (n=81).
| Characteristics | N (%)/mean ± SD | |
|---|---|---|
| Gender | Male | 64 (79.0) |
| Female | 17 (21.0) | |
| Age (years) | 56.22 ± 10.86 | |
| Disease period (months) | 43.41 ± 51.84 | |
| TMT-A (seconds) | 77.07 ± 79.39 | |
| TMT-B (seconds) | 155.26 ± 135.79 | |
| Education | Illiterate | 1 (1.2) |
| Preschool | 10 (12.3) | |
| Middle school | 13 (16.0) | |
| High school | 23 (28.4) | |
| Above high school | 34 (42.1) | |
| Type of stroke | Infarction | 45 (55.6) |
| Hemorrhage | 36 (44.4) | |
| Affected side | Right | 42 (51.9) |
| Left | 39 (48.1) | |
| Past driving experience | <5 years | 14 (17.3) |
| ≥5 years | 4 (4.9) | |
| ≥10 years | 63 (77.8) | |
The study period was from July 2013 to November 2014. The study was carried out in three stages. In the first stage, the DCAT, TMT-A, and TMT-B evaluations were conducted in all participants. In the second stage, the participants were classified into the safety or risk group based on the evaluation results of the DCAT. In the final stage, TMT-A and TMT-B results underwent a receiver operating characteristic (ROC) analysis in each group.
The DCAT is an in-office driving assessment system for predicting the driving risk in the on-road driving assessment14). It assesses the possibility of failing in the on-road driving evaluation and informs the central computer that analyzes the results of memory, attention, judgment, response time, decision making, and judgment of emergency situation tests. The DORE (The DriveABLE On-Road Evaluation) refers to this measured value15). In this study, patients were classified as safe or unsafe drivers. The TMT is divided into the A and B types. It measures the response time, attention, memory, sequencing, decision making, automatic thinking, etc. and is a cognitive test that has a high association with stroke drivers7).
PASW Statistics Version 18 (IBM Corporation, Armonk, NY, USA) and MedCalc Version 16.1 (LIONBRIDGE Inc., LA, USA) were used for statistical analysis. An ROC curve analysis and an area under the curve (AUC) estimate were used to assess the cut-off points for the TMT in stroke drivers. A p value <0.05 was considered statistically significant.
RESULTS
The result of the ROC curve analysis showed that the cut-off point for the TMT-A was 32 seconds. This score was located at the intersection point of sensitivity of 0.937 and specificity of 0.944. The Youden index of the score was 0.881 for the highest score. The cut-off point for the TMT-B was 79 seconds. This score was located at the intersection point of sensitivity of 0.905 and specificity of 0.944. The Youden index of the score was 0.849 for the highest score (Table 2). The AUC for the TMT-A was 0.978 and it was close to 1. The AUC for the TMT-B was 0.956 and it was close to 1 (Table 3).
Table 2. Comparison of the cognitive and driving functions between SDG and USDG.
| TMT-A score | Sensitivity | 95% CI | Specificity | 95% CI | Youden index | +LR | −LR | PPV | NPV |
|---|---|---|---|---|---|---|---|---|---|
| 30 | 0.968 | 0.890–0.996 | 0.833 | 0.586–0.964 | 0.802 | 5.81 | 0.04 | 95.300 | 88.200 |
| 31 | 0.937 | 0.845–0.982 | 0.889 | 0.653–0.986 | 0.825 | 8.43 | 0.07 | 96.700 | 80.000 |
| 32 | 0.937 | 0.845–0.982 | 0.944 | 0.727–0.999 | 0.881 | 16.86 | 0.07 | 98.300 | 81.000 |
| 42 | 0.762 | 0.638–0.860 | 0.944 | 0.727–0.999 | 0.706 | 13.71 | 0.25 | 98.000 | 53.100 |
| TMT-B score | Sensitivity | 95% CI | Specificity | 95% CI | Youden index | +LR | −LR | PPV | NPV |
| 77 | 0.921 | 0.824–0.974 | 0.889 | 0.653–0.986 | 0.810 | 8.29 | 0.09 | 96.700 | 76.200 |
| 78 | 0.905 | 0.804–0.964 | 0.889 | 0.653–0.986 | 0.794 | 8.14 | 0.11 | 96.600 | 72.700 |
| 79 | 0.905 | 0.804–0.964 | 0.944 | 0.727–0.999 | 0.849 | 16.29 | 0.10 | 98.300 | 73.900 |
| 106 | 0.603 | 0.472–0.724 | 0.944 | 0.727–0.999 | 0.548 | 10.86 | 0.42 | 97.400 | 40.500 |
TMT: trail making test, CI: confidence interval, +LR: positive likelihood ratio, −LR: negative likelihood ratio, PPV: positive predictive value, NPV: negative predictive value
Table 3. AUCs for TMT-A and TMT-B.
| AUC | 95% CI | SE | |
|---|---|---|---|
| TMT-A | 0.978* | 0.918–0.998 | 0.0149 |
| TMT-B | 0.956* | 0.886–0.989 | 0.0252 |
*p<0.01, TMT: trail making test, CI: confidence interval, SE: standard error
DISCUSSION
The purpose of this research was to validate the predictive validity of the TMT for predicting the risk of unsafe driving in stroke patients and to determine the cut-off points. The results of the ROC curve analysis showed that the cut-off point was determined to the highest point of the Youden Index and the intersection point of sensitivity and specificity. These points were 32 seconds for the TMT-A and 79 seconds for the TMT-B. In a past study of patients with cognitive impairment, the cut-off points were 39.5 seconds for the TMT-A and 180 seconds for the TMT-B12). In another study of elderly drivers, the cut-off point for the TMT-B was 106.7 seconds12). The cut-off point for the TMT-B is different. This is considered to be because the TMT-B more strictly measures cognitive function than the TMT-A.
For the TMT-A, the PPV (positive predictive value) for accurately determining a safe driver was 98.3%, and the NPV (negative predictive value) for accurately determining an unsafe driver was 81.0%. For the TMT-B, the PPV was 98.3% and the NPV was 73.9%. A study predicted the driving risk of patients with cognitive impairment through the TMT evaluation, and it was observed that the PPV and NPV of the TMT-A were 77% and 62%, respectively, and the PPV and NPV of the TMT-B were 50% and 88%, respectively12). The values obtained in this study are high compared with those of previous studies. Stroke patients mainly use their non-affected side, which may also be assessed with TMT evaluation. Because this study was performed in specific subjects, as opposed to previous studies, the results are likely to be highly predictable.
These findings were confirmed in a prior study that investigated elderly drivers. In a research using the TMT to predict the risk of unsafe driving in older drivers, the sensitivities of the TMT-A and TMT-B were 73% and 77%, respectively, and the specificities of the TMT-A and TMT-B were 68% and 77%, respectively12). In comparison, the sensitivity and specificity of the TMT-A and TMT-B in this study were confirmed to be relatively high, i.e. 90.0% or higher. Unlike the elderly, stroke drivers can compensate for the loss of function in the affected side. Because of these features, it is thought that the predictive value of the assessment with the TMT is higher.
The AUC is the area under the ROC curve and a value closer to 1 is indicative of a correct diagnostic tool16). An AUC ≥0.9 indicates a very accurate tool; an AUC ≥0.7 indicates a moderately accurate tool; an AUC ≥0.5 indicates a marginally accurate tool; and an AUC ≤0.5 indicates a tool without discrimination17). In this study, AUC values for the TMT-A and TMT-B were 0.978 and 0.956, respectively. Hence, the TMT is a very accurate tool to predict the risk of unsafe driving in stroke drivers. However, this study did not determine the cut-off point for the TMT error values. Further studies on this issue are needed.
REFERENCES
- 1.Fisk GD, Owsley C, Pulley LV: Driving after stroke: driving exposure, advice, and evaluations. Arch Phys Med Rehabil, 1997, 78: 1338–1345. [DOI] [PubMed] [Google Scholar]
- 2.Centers for Disease Control and Prevention (CDC): Prevalence of disabilities and associated health conditions among adults—United States, 1999. MMWR Morb Mortal Wkly Rep, 2001, 50: 120–125. [PubMed] [Google Scholar]
- 3.Akinwuntan AE, Wachtel J, Rosen PN: Driving simulation for evaluation and rehabilitation of driving after stroke. J Stroke Cerebrovasc Dis, 2012, 21: 478–486. [DOI] [PubMed] [Google Scholar]
- 4.Nouri FM, Lincoln NB: Predicting driving performance after stroke. BMJ, 1993, 307: 482–483. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Meyers JE, Volbrecht M, Kaster-Bundgaard J: Driving is more than pedal pushing. Appl Neuropsychol, 1999, 6: 154–164. [DOI] [PubMed] [Google Scholar]
- 6.Akinwuntan AE, De Weerdt W, Feys H, et al. : Effect of simulator training on driving after stroke: a randomized controlled trial. Neurology, 2005, 65: 843–850. [DOI] [PubMed] [Google Scholar]
- 7.Marshall SC, Molnar F, Man-Son-Hing M, et al. : Predictors of driving ability following stroke: a systematic review. Top Stroke Rehabil, 2007, 14: 98–114. [DOI] [PubMed] [Google Scholar]
- 8.Edwards JD, Ross LA, Wadley VG, et al. : The useful field of view test: normative data for older adults. Arch Clin Neuropsychol, 2006, 21: 275–286. [DOI] [PubMed] [Google Scholar]
- 9.Reitan RM, Wolfson D: The Halstead-Reitan Neuropsychological Test Battery. Tucson: Neuropsychology Press, 1985. [Google Scholar]
- 10.Mathias JL, Lucas LK: Cognitive predictors of unsafe driving in older drivers: a meta-analysis. Int Psychogeriatr, 2009, 21: 637–653. [DOI] [PubMed] [Google Scholar]
- 11.Seong-Youl C, Jae-Shin L, A-Young S: Cognitive test to forecast unsafe driving in older drivers: meta-analysis. NeuroRehabilitation, 2014, 35: 771–778. [DOI] [PubMed] [Google Scholar]
- 12.Dobbs BM, Shergill SS: How effective is the Trail Making Test (Parts A and B) in identifying cognitively impaired drivers? Age Ageing, 2013, 42: 577–581. [DOI] [PubMed] [Google Scholar]
- 13.Classen S, Wang Y, Crizzle AM, et al. : Predicting older driver on-road performance by means of the useful field of view and trail making test part B. Am J Occup Ther, 2013, 67: 574–582. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dobbs AR: Accuracy of the DriveABLE cognitive assessment to determine cognitive fitness to drive. Can Fam Physician, 2013, 59: e156–e161. [PMC free article] [PubMed] [Google Scholar]
- 15.DriveABLE Assessment Centres Inc: DriveABLE Cognitive Assessment Tool (DCAT) TM Assessment Manual. Canada: Alberta Edmonton, 2010. [Google Scholar]
- 16.Song SW: Using the receiver operating characteristic (ROC) curve to measure sensitivty and specificity. Korean J Fam Med, 2009, 30: 841–842. [Google Scholar]
- 17.Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 1982, 143: 29–36. [DOI] [PubMed] [Google Scholar]