Clinician Assessment of the Driving Competence of Patients with Dementia

Abstract

OBJECTIVES

To determine the validity and reliability of clinician ratings of the driving competence of patients with mild dementia.

DESIGN

Observational study of a cross-section of drivers with mild dementia based on chart review by clinicians with varying types of expertise and experience.

SETTING

Outpatient dementia clinic.

PARTICIPANTS

Fifty dementia subjects from a longitudinal study of driving and dementia.

MEASUREMENTS

Each clinician reviewed information from the clinic charts and the first study visit. The clinician then rated the drivers as safe, marginal, or unsafe. A professional driving instructor compared these ratings with total driving scores on a standardized road test and categorical ratings of driving competence. Clinicians also completed a visual analog scale assessment of variables that led to their determinations of driving competence.

RESULTS

Accuracy of clinician ratings ranged from 62% to 78% for the instructor’s global rating of safe versus marginal or unsafe. In general, there was moderate accuracy and interrater reliability. Accuracy could have been improved in the least-accurate raters by greater attention to dementia duration and severity ratings, as well as less reliance on the history and physical examination. The most accurate predictors were clinicians specially trained in dementia assessment, who were not necessarily the most experienced in their years of clinical experience.

CONCLUSION

Although a clinician may be able to identify many potentially hazardous drivers, accuracy is insufficient to suggest that a clinician’s assessment alone is adequate to determine driving competence in those with mild dementia.

Physicians are often called upon to render judgments regarding the driving competence of cognitively impaired patients. Despite this common practice, there has been a paucity of research regarding the validity and reliability of the physician’s assessment, even though this is an extremely important issue affecting a patient’s independence and sense of autonomy.

Self-report is notoriously invalid because of denial and diminished insight.^1,2 Caregivers exhibit concerns about driving impairment, but their assessments of driving competence are variable and at best modestly predictive of actual driving performance as measured using road tests administered by professional driving instructors.^1–6

Physician prediction about the driving performance of demented individuals has been reported to be of somewhat greater validity. In one study of 19 subjects with Alzheimer’s disease (AD), the physician’s prediction (pass, borderline, or fail) of the patient’s abilities was significantly correlated with total score on a standardized road test.⁷ More recently, predictions of driving performance by dementia patients, family members, and a physician for 50 patients with early AD who underwent a standardized road test were compared. Only the physician’s prediction was significantly related to the driving score, being more sensitive and specific than ratings by family members or the patient drivers themselves.² The physician in that study was an experienced neurologist and dementia specialist. Consequently, it remains to be demonstrated whether this observation can be generalized to other clinicians in actual practice.

Therefore, a follow-up study was conducted to determine the validity and interrater reliability of clinician ratings of the driving competence of patients with mild dementia. For this study, clinicians with a variety of dementia expertise and experience, who made their determinations of driving competence for the same patients using identical sources of information, were chosen.

METHODS

Subjects

Subjects included 50 individuals with probable or possible AD, using National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer’s Disease and Related Disorders Association diagnostic criteria,⁸ who had very mild to mild dementia (Clinical Dementia Rating (CDR)⁹ = 0.5–1.0). All subjects were enrolled in a longitudinal study of drivers with dementia. The methods for this study have been previously described.² All subjects signed a document of informed consent that the Memorial Hospital of Rhode Island Human Subjects in Research Review Committee approved.

Exclusion criteria included reversible causes of dementia and physical, ophthalmological, or neurological disorders other than dementia that might impair driving abilities. Specifically, major physical handicaps such as frozen joints, inadequately healed fractures, monocular blindness, and amputation were exclusionary. In all subjects, corrected visual acuity was better than 20/50 on eye chart testing, and visual fields were normal on confrontation testing. Psychiatric disorders were exclusionary, including mental retardation, schizophrenia, bipolar disorder, or history of alcohol/substance abuse within the previous year. Depression was allowed if it was controlled with medications. Symptomatic antidementia drugs (e.g., cholinesterase inhibitors) and antipsychotic and anxiolytic medications were permitted, but dosages were required to be stable for at least 6 weeks before entry into the study. Informants were individuals who spent time with the participants more than once weekly and who had accompanied the participant while driving at least once monthly during the preceding 12 months. All subjects had valid driver’s licenses. The mean length of time for driving experience was 53.8 years (range 15–70 years). All subjects were Caucasian, with the exception of one who was Asian-American. Other demographics for the subjects were mean age ± standard deviation of 75.7 ± 6.6; education, 13.2 ± 3.5 years; and sex, 31 men and 19 women. Mean duration of dementia was 3.3 ± 1.8 years; mean Mini-Mental State Examination (MMSE)¹⁰ was 23.7 ± 4.0; and CDR was 0.5 for 33 subjects and 1.0 for 17 subjects.

Procedures

Before a road test, participants and informants rated the participant’s driving ability on a trichotomous scale, drives alone with good sense of direction and good driving skills, drives but with some difficulty, or unable to drive safely. A research assistant collected driving history from the informant and participant together, including history of accidents and violations over the previous 3 years, miles driven per month, and miles driven with the informant per month. The accident and violation information was later crosschecked with state driving records.

The primary study physician (BO), blind to participant and informant ratings as well as to on-road test results (which occurred after the office evaluation), assessed the participant’s ability to drive on the same scale as the informant and participant. His rating was based upon information he obtained in the initial diagnostic evaluation (including history and neuropsychological and laboratory tests); administration of the CDR and MMSE; neurological, eye, and general physical examinations; and information obtained from the participant and informant about any history of motor vehicle accidents or traffic violations.

Within 2 weeks of the clinical assessment, a professional, experienced (6 years of licensed, full-time work) driving instructor administered an on-road driving test to participants during daylight hours under good conditions (no precipitation or wet roads). The instructor was blind to the participant’s diagnosis and the driving ability ratings made by the informant and physician. A 10- to 15-minute pretest was completed in the parking lot before the road test to ensure that the test was safe to perform and to familiarize the participant with the car and the instructor. The road test lasted approximately 45 minutes. The driving test was based on a published and reliable driving test, the Washington University Road Test¹¹ adapted for comparable streets in Rhode Island. Although the streets were different, all the same maneuvers and identical scoring procedures were used to produce a comparable test procedure for Rhode Island. Participants received an on-road driving score based on safe completion of each of the required maneuvers, ranging from 0 (best score) to 108 (worst score). The instructor also made a trichotomous global rating of the participant’s driving ability, safe, marginal, or unsafe, which was akin to the trichotomous rating made by the participant, informant, and physician.

Five other clinicians examined the same clinic and baseline study visit records that were available to the first physician but did not directly interview the patients and caregivers. No instructions were given to the clinicians on how they were to derive their ratings of driving competence from the available records.

The clinicians also completed a visual analog rating scale ranging from 0 to 160 mm in length for each of 22 variables that were available to them from the records. The clinicians drew a line across the scale line to indicate the weight that they gave each variable in their assessment of competence, ranging form none to very much. The variables included age; sex; education; occupation; dementia duration; CDR/dementia severity; history of present illness; medical history; driving history of motor vehicle accidents; driving history of traffic violations; medications (e.g., tranquilizers, antipsychotics, or antidementia drugs); physical examination (e.g., musculoskeletal limitations); neurological examination; eye examination (including visual fields and visual acuity); MMSE; neuropsychological tests in general; and specific neuropsychological tests of attention, memory, language, executive function, visuospatial ability, and praxis.

The first clinician (BO) is a dementia specialist with certifications in neurology and geriatric medicine and 25 postgraduate years of clinical experience. The second clinician (CW) is a dementia specialist with certification in neurology and 16 postgraduate years. The third clinician (DA) is a general practitioner with certification in family practice and 7 postgraduate years. The fourth clinician (AC) is a geriatric nurse practitioner with certification in gerontological nursing and 14 postgraduate years. The fifth clinician (AD) is a geriatric neurology fellow with board eligibility in neurology and 5 postgraduate years. The sixth clinician (JB) is a geriatric psychiatry fellow with board eligibility in psychiatry and 5 postgraduate years.

RESULTS

The driving instructor rated 22 subjects as safe, 19 subjects as marginal, and nine subjects as unsafe. The mean total score for the road tests was 13.5 ± 8.7, with a range of 0 to 37 points.

To define the accuracy of the clinician predictions of driving ability, the driving instructor’s and clinicians’ global ratings were dichotomized into safe versus unsafe by combining marginal ratings with unsafe ratings. Clinician accuracy (percentage of correct classifications) ranged from 62% to 78%, with the dementia specialists ranging from 72% to 78% and the general practitioners ranging from 62% to 64%. Sensitivity and specificity between raters was widely variable. The senior geriatric neurologist showed the highest sensitivity and lowest specificity. The geriatric neurology fellow showed the highest specificity and positive predictive value. The senior geriatric neurologist showed the highest negative predictive value. The junior geriatric neurologist showed the best balance between positive and negative predictive values (72%) (Table 1).

Table 1.

Clinician Predictions of Drivers Rated Categorically Safe by the Driving Instructor

Clinician	JB	BO	AD	CW	AC	DA
Sensitivity	59.1	95.2	49.5	59.1	40.9	45.5
Specificity	92.9	59.3	96.4	82.1	82.1	75
Positive predictive value	86.7	64.5	90.9	72.2	64.3	58.8
Negative predictive value	74.3	94.1	69.2	71.9	63.9	63.6
Correct classification	78*	75*	74*	72*	64*	62

^*P < .05, chi-square.

Table 2 compares the ordinal clinician ratings with the continuous scores on the road test. On this basis, the order of accuracy is similar, but the separation between the two most-accurate and the three least-accurate raters is larger. The three most-accurate raters were the senior geriatric neurologist, the geriatric neurology fellow, and the geriatric psychiatry fellow. A significant relationship between global rating of driving competence and the road tests score was not seen for the other three raters.

Table 2.

Comparison of the Ordinal Clinician Ratings to the Continuous Road Test Scores (Kendall’s Tau-Beta)

Clinician	JB	BO	AD	CW	AC	DA
Point estimate	0.49	0.41	0.47	0.09	0.016	0.10
Two-sided confidence interval	0.32 – 0.65	0.23 – 0.59	0.30 – 0.64	−0.16 to 0.33	−0.03 to 0.35	−0.14 to 0.34
P-value*	<.001	<.001	<.001	.46	.17	.41

^*For test of H0: total driving score and clinician’s rating are independent.

Figure 1 shows the average of weights that the three most accurate raters and the three least accurate raters defined in Table 2 gave to each clinical variable. The largest discrepancy was in the weight given to dementia duration, which the most-accurate raters more heavily weighted. The most-accurate raters also placed somewhat more weight on measures of dementia severity (CDR and MMSE) and specific tests of visuospatial ability, praxis, and executive function and relatively less weight on accidents and violations, although each of these variables received a high weight from both groups of raters. The least-accurate raters also placed relatively greater weight on history of illness and aspects of the physical examination.

Figure 1.

Weights placed on variables used by clinicians to make their ratings (visual analog scale). Exam = examination.

DISCUSSION

This study represents the first examination of validity and interrater reliability of clinician assessments of the driving competence of patients with dementia, most of whom probably had early AD. Although each of the clinicians was able to correctly predict driving performance for the majority of subjects, there was wide variability in sensitivity and specificity between the different raters. Moreover, accuracy was greatest for physicians with specialized training in dementia, regardless of their years of clinical experience. Although this evidence may suggest that physicians working in dementia referral clinics may have a meaningful role to play in determining driving privileges or the need for formal road test evaluation, the consistency between physicians in their degree of positive and negative predictive values is insufficient to justify making them the final or sole arbiters of the decision to revoke driving privileges in patients with dementia.

In making their assessments, the most accurate raters emphasized dementia duration; dementia severity measured using CDR and MMSE; specific neuropsychological tests of praxis, visuospatial ability, executive function, and attention; and history of accidents and traffic violations. These are all factors that have been previously demonstrated to be related to driving competence in demented drivers.

For example, one study¹² questioned 130 caregivers of patients with AD and 112 age-matched controls. The crash risk for patients with AD was found to rise above acceptable control rates beyond the third year of the disease. Approximately 50% of drivers had stopped driving completely 3 years after disease onset, similar to the 34-month median figure reported previously by another group of investigators.³ This study¹² suggested that a 3-year guideline may be helpful in defining when to be vigilant for driving incompetence in older people with early cognitive decline. Another study¹³ also found a rise in crashes after 3 years of dementia. In a dementia clinic sample from the United Kingdom, 22% of patients with dementia continued to drive 3 years after onset of illness, and two-thirds of these people were judged to be impaired drivers according to physician assessment.¹⁴

The rate of progression of degenerative dementia such as AD is not uniform across all individuals. Thus, instead of examining disease duration, more-specific measures of disease severity have been examined as predictors of impaired driving in dementia patients. In particular, higher CDR has been associated with poorer driving in multiple studies. A review of the driving and AD literature with regard to CDR status,¹⁵ using a conversion paradigm to stratify prior studies into CDR categories, found a higher crash risk for patients with AD than for age-matched controls and concluded that this crash risk was greater for those with a CDR dementia status of 1.0 than for those with a CDR 0.5 rating.

There is evidence that performance on tests of executive function is related to real-world driving abilities, including performance on mazes,¹⁶ clock drawing,^17,18 and the Trail-Making Test.^19,20 There is also evidence that measures of visual attention and visuoperception are related to driving abilities in individuals with dementia as well as healthy older people.^21–24 In a meta-analysis of the literature regarding neuropsychological tests and driving abilities comparing performance in multiple specific cognitive domains (e.g., attention, language, memory), only performance on neuropsychological tests of visuospatial skills was related to on-road tests of driving abilities, particularly when studies using a control group were excluded.²⁵

There are a number of important limitations to this study. All clinicians were drawn from an academic medical center. Consequently, these results may not fully apply to general community practice. Only one of the clinicians had personal contact with the subjects. Although one of the other raters, who was more accurate, had only chart access, it is possible that the least-accurate raters could have improved their accuracy if able to directly interview the patients and caregivers. Future studies could include videotaped interviews or office assessments for driving competence by more than one clinician.

The accuracy of the general practitioners in this study was insufficient to recommend that all physicians play a major role in determining driving privileges in patients with dementia. Nevertheless, it is possible that general practitioners could become valid raters of driving competence if trained to emphasize and apply standard measures such the MMSE and CDR in their practice and if they were educated on the importance of dementia duration and specific cognitive measures of constructional praxis, executive function, and attention. Given the present state of knowledge about risk factors for hazardous driving in persons with early dementia, physicians should routinely assess their patients for dementia severity and inquire about motor vehicle accidents and moving violations when advising them about driving activity.