Skip to main content
Neurology logoLink to Neurology
. 2014 Oct 28;83(18):1627–1633. doi: 10.1212/WNL.0000000000000943

Predicting quality of life outcomes after subthalamic nucleus deep brain stimulation

Darlene Floden 1,, Scott E Cooper 1, Sandra D Griffith 1, Andre G Machado 1
PMCID: PMC4223084  PMID: 25274851

Abstract

Objectives:

To examine disease, treatment, cognitive, and psychological factors associated with quality of life (QoL) before and after surgery and assess the ability to predict QoL outcomes.

Methods:

We identified a retrospective, cross-sectional sample of 85 patients with Parkinson disease who underwent subthalamic deep brain stimulation (DBS). Patients' QoL was categorized as “improved” and “stable/worsened” using reliable change indices. Univariate correlational analyses identified relationships between Parkinson's Disease Questionnaire–39 ratings and disease (Unified Parkinson's Disease Rating Scale–III [UPDRS-III] motor scores on and off medications, disease duration), treatment (medication burden, unilateral vs bilateral DBS), cognitive (neuropsychological battery), and psychological (depression) variables. Step-wise multiple linear regression and logistic regression models included selected preoperative variables to predict change in QoL ratings and QoL outcome after surgery.

Results:

Fifty-one percent of patients reported clinically significant improvements in QoL while 47% reported stable QoL and 2% worsened. Motor scores (UPDRS-III) were not relevant to QoL changes, potentially because of the rarity of poor motor outcomes, while single-trial learning and depression scores were the most important variables in predicting QoL changes. There was a subtle additional benefit to undergoing bilateral subthalamic nucleus DBS.

Conclusions:

The findings provide greater insight into the nonmotor features that contribute to the success of subthalamic nucleus DBS procedures from the patient's perspective and raise questions about the treatment focus and emphasis on symptom profiles in DBS candidacy evaluations.


Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective treatment for the motor symptoms of Parkinson disease (PD).13 Recent studies have also demonstrated greater improvements in quality of life (QoL) after bilateral STN-DBS compared with best medical therapy.46 The present study sought to evaluate the factors that account for QoL improvement and to determine whether preoperative patient characteristics can predict QoL improvement after STN-DBS.

Available studies point to a strong relationship between QoL and the nonmotor features of PD including sleep disturbance, urinary problems, and mood.68 The goal of the current study was to evaluate the utility of nonmotor variables in predicting QoL outcomes after DBS. Prior work implicates factors such as preoperative cumulative time in the “off” state6 and global cognitive scores9 as significant predictors of QoL changes after STN-DBS. The latter finding, in particular, is of interest because a finer-grained analysis of preoperative cognitive abilities may provide insights into why and to what degree cognitive impairments influence QoL after surgery.

A final question concerns the relative influence of bilateral vs unilateral STN-DBS for changes in QoL. Most studies examine bilateral STN stimulation, although one study of unilateral DBS indicated that unilateral globus pallidus internus stimulation showed greater benefit to QoL compared with STN stimulation.7 Herein, we directly compare QoL changes in patients with unilateral and bilateral STN-DBS.

METHODS

Standard protocol approvals, registrations, and patient consents.

The study was approved by the institutional review board of the Cleveland Clinic.

Patients.

We retrospectively identified patients diagnosed with idiopathic PD who underwent STN-DBS implantation at the Cleveland Clinic between 2006 and 2012. We included all patients who had completed the 39-item Parkinson's Disease Questionnaire (PDQ-39)10 to assess self-reported QoL at both pre- and postoperative assessments. There was an average of 8.2 months (range 3–29) between surgery and postoperative QoL and clinical data collection. To obtain an unbiased characterization of QoL outcomes, we did not exclude patients based on demographic characteristics or cognitive, psychological, or motor symptoms or outcomes, although patients who had undergone prior neurosurgery were excluded. In our clinical decision-making, we sought to avoid operating on patients who, based on our experience and judgment, were not likely to improve, and this may account for the rarity of poor motor outcomes in our patient population.

The study included data for 85 patients. Patients were aged 61.6 ± 8.8 years (range 35–79) with 13.9 ± 3.0 years of education. Sixty-two patients were male. Patients had symptoms of PD an average of 10.5 ± 5.0 years (range 3–28). Thirty-six patients underwent bilateral STN-DBS implantations and 49 patients had unilateral procedures (12 right STN [6 left-handed]; 37 left STN [33 right-handed]). The clinical decision regarding unilateral or bilateral surgical plan was made on a case-by-case basis at a multidisciplinary patient management conference and key factors included symptom asymmetry (e.g., bilateral DBS with symmetrical or largely axial/gait symptoms), patient's goals (e.g., use of dominant hand or bimanual control), potential medication reduction (i.e., likelihood of interactions between medication management and unilateral stimulation), and patient's age and estimated perioperative surgical risk (e.g., unilateral approach favored in older, frailer patients).

QoL and cognitive measurement.

QoL was assessed using the PDQ-3910 during pre- and postoperative neuropsychological evaluations. It includes 39 questions requiring patients to rate the degree to which they experience negative effects of PD in 8 domains: Mobility, Activities of Daily Living, Emotional Well-being, Stigma, Social Support, Cognition, Communication, and Bodily Discomfort. Index scores range from 0 to 100 with higher scores reflecting poorer QoL.

Analysis included neuropsychological variables in several cognitive domains (global cognition, naming, verbal fluency, processing speed, visuospatial function, executive function, single-trial learning, delayed memory) selected based on clinical experience of sensitivity to PD and frequency of reporting in the STN-DBS literature (see table 1). Depression symptoms were assessed using the Beck Depression Inventory–2nd edition (BDI-II)11 in which higher scores represent more severe levels of self-reported depression symptoms.

Table 1.

Baseline and postoperative clinical characteristics

graphic file with name NEUROLOGY2014586479TT1.jpg

Motor function.

A movement disorders neurologist evaluated each patient before and after surgery. Before surgery, each patient completed the UPDRS-III in the medication “on” state (on meds) and in the clinically defined “off” state after abstaining from their regular medications overnight (off meds). After surgery, the UPDRS-III was administered while on medications and on stimulation (on meds/on stimulation) as well as off medications and on stimulation (off meds/on stimulation).

Statistical analysis.

Bonferroni-corrected independent samples t tests compared unilateral and bilateral groups on preoperative characteristics. Repeated-measures general linear models with Bonferroni correction for multiple comparisons examined pre- to postoperative changes in PDQ-39 scores (total and subscales) and other clinical measurements for unilateral vs bilateral STN-DBS groups. Change scores reflected the subtraction of each patient's postoperative scores from their preoperative scores. As such, a positive change score on the PDQ-39 represents improved QoL after STN-DBS. Correlations evaluated the bivariate relationships between QoL change and select demographic or preoperative cognitive/disease variables. Pearson correlations were used for continuous, normally distributed variables while Spearman r was used for categorical variables. Significance tests were not corrected for multiple comparisons because of the exploratory nature of this analysis. Before these analyses, all variables were evaluated for distribution abnormalities, and outlier values (>3 SDs) were truncated at the next highest value. Because of negative skewness, the DRS-2 was transformed using the following equation: srDRS = sqrt(145 − DRS). For all analyses, missing data were not replaced.

Exploratory stepwise regression model (inclusion relaxed to p < 0.1) included change scores showing significant correlations with PDQ-39 change, as well as surgical approach (unilateral vs bilateral), to evaluate how changes in QoL relate to changes in other clinical and disease variables after surgery. A second stepwise regression model included baseline variables with significant correlations with PDQ-39 change (pre- minus post-PDQ-39) to evaluate preoperative predictors of postoperative QoL.

To examine individual differences in QoL outcome, each patient was also classified as reporting improved, worsened, or stable QoL using published PD-specific reliable change index score.9 Specifically, a PDQ-39 change score greater than 10.9 points represents a statistically significant change over and above the difference expected based on test-retest reliability of the PDQ-39 in patients with PD over the same assessment schedule used in this study. Logistic regression and receiver operating characteristic analyses evaluated the utility of linear regression models to correctly predict whether a patient's QoL would significantly improve after surgery. All analyses were performed in SPSS version 17.0 (SPSS, Inc., Chicago, IL).

RESULTS

Table 1 displays pre- and postoperative scores on clinical measures. There were no significant differences between unilateral and bilateral patients on any preoperative variable. There were statistically significant postsurgical declines in levodopa equivalent daily dosage (F1,83 = 27.4, p < 0.001), on-med UPDRS-III scores (on medication vs on medication/on stimulation; F1,77 = 21.7, p < 0.001), and off-med UPDRS-III scores (off medication vs off medication/on stimulation; F1,74 = 218.7, p < 0.001), indicating that STN-DBS therapy successfully reduced medication burden and improved motor function. There was a slightly larger improvement in off-meds vs off-meds/on-stimulation UPDRS-III score in patients with bilateral STN stimulation (F1,74 = 5.0, p = 0.024), but this did not reach significance after correction for multiple comparisons. There were no differences between unilateral and bilateral patients on “on” UPDRS-III scores or medication reductions. There was a significant decline in verbal fluency scores after surgery (F1,79 = 44.3, p < 0.001), but all other cognitive variables were stable over assessments and there were no interactions with bilateral vs unilateral DBS grouping. There was a tendency toward lower depression scores after surgery (F1,83 = 4.6, p = 0.04) that failed to reach significance with correction for multiple comparisons.

Self-reported QoL on the PDQ-39 significantly improved after STN-DBS (F1,83 = 75.5, p < 0.001). As shown in figure 1, postoperative scores were significantly reduced on all PDQ-39 subscales (all F1,83 > 12.6, p < 0.005) except the Cognition (F1,83 = 7.9, p = 0.01) and Communication (F1,83 = 3.8, p = 0.06) subscales. There were no significant interactions with surgical group although there was a tendency toward greater postsurgical change in the Activities of Daily Living subscale in the bilateral group (F1,83 = 7.7, p = 0.01), which failed to reach significance after correction for multiple comparisons. As such, groups were combined for the remainder of the analysis.

Figure 1. PDQ-39 subscale scores.

Figure 1

Self-reported symptoms on the PDQ-39 subscales for patients with unilateral (blue bars) or bilateral (red bars) subthalamic nucleus deep brain stimulation before (pre; darker bars) and after (post; lighter bars) surgery. Lower numbers mean better quality of life. ADL = Activities of Daily Living; Cogn = Cognition; Comm = Communication; Discomf = Bodily Discomfort; Emot = Emotional Well-being; Mobil = Mobility; PDQ-39 = 39-item Parkinson's Disease Questionnaire; Social = Social Support. Error bars represent 2 SEMs.

Correlations between baseline and/or postsurgical change in demographic/disease variables and PDQ-39 change scores in the combined sample are displayed in table 2. Improvements in QoL were associated with reductions in dopaminergic medication burden and depression symptoms, and better single-trial learning after surgery. At the same time, improvements in QoL were associated with several preoperative characteristics including higher preoperative dopaminergic medication burden, better naming ability before surgery, more efficient single-trial learning preoperatively, and higher levels of self-reported depression symptoms before surgery. There were tendencies toward improved QoL in patients with better preoperative response to dopaminergic medications. There was no association of QoL improvements and any demographic variable, duration of PD, or severity of disease as indicated by UPDRS-III score off medications.

Table 2.

Correlations with PDQ-39 change

graphic file with name NEUROLOGY2014586479TT2.jpg

An exploratory stepwise linear regression to examine the association of clinical changes pre- to postsurgically accounted for 31% of the variance in PDQ-39 change scores. Improvements in BDI scores (β = 0.57) and bilateral surgery (β = 0.19) contributed independently to improvements in QoL (F3,72 = 16.9, p < 0.001). Effectiveness of stimulation (off medication vs off medication/on stimulation), change in levodopa equivalent daily dose after surgery, and change in single-trial learning were not significantly associated with changing PDQ-39 scores. Amount of motor improvement did not predict QoL improvement; however, a larger number of subjects who showed little or no motor improvement may be necessary to detect a statistically significant association between motor improvement and QoL improvement, if such a relationship exists.

Predicting QoL outcome.

Preoperative PDQ-39 scores (B = 0.61; better preoperative QoL predicted better postoperative QoL) and single-trial learning (B = −1.99; better preoperative memory performance predicted better postoperative QoL) accounted for nearly half (45%) of the variance in postoperative QoL (F2,78 = 32.6, p < 0.001). Medication burden, medication response, naming ability, and depression ratings did not contribute.

Individual difference analysis.

We found that 51% of our sample (43 of 85 patients) reported improvements in QoL on the PDQ-39 after surgery. Approximately 47% of patients (40 of 85 patients) reported unchanged QoL while 2% (n = 2) reported worsening of QoL. A logistic regression model using preoperative Rey Auditory Verbal Learning Test single-trial memory (odds ratio [OR] = 1.58, 95% confidence interval [CI] = 1.11–2.24), preoperative BDI-II (OR = 0.96, 95% CI = 0.87–1.05), and preoperative PDQ-39 score (OR = 1.05, 95% CI = 1.02–1.09) correctly classified 70% of patients (C statistic = 0.73, p < 0.001, 95% CI = 0.62–0.84; see figure 2). In other words, the odds of postoperative QoL improvement increased by approximately 60% for every additional word remembered on the first list-learning trial, increased by approximately 63% for every 10-point increase in preoperative PDQ-39 score, and decreased by approximately 37% for every 10-point increase in BDI score.

Figure 2. Regression classification accuracy.

Figure 2

Receiver operating characteristic curve demonstrating the classification accuracy (predicted probability of improved vs not improved quality of life) of the logistic regression. The diagonal dashed line represents chance classification accuracy.

DISCUSSION

In our sample of 85 patients with PD, STN-DBS surgery was successful in reducing motor impairment and medication burden. Furthermore, STN-DBS improved overall QoL on the PDQ-39 as well as ratings in subdomains reflecting motor function, mood, and self-consciousness. These areas of QoL change correspond with the typical effects of STN-DBS—reductions in tremor, bradykinesia, rigidity, and dyskinesias, which may improve mood and reduce self-consciousness. Changes were not observed on QoL subscales that index symptoms of PD that do not typically respond to DBS including speech, cognitive function, and hallucinations.12

The nearly 50/50 split between patients reporting improved and stable QoL after STN-DBS is remarkably similar to a recent prospective study of patients treated with STN-DBS or best medical therapy.6 We looked further at the 2 patients who endorsed worsened QoL after surgery. The first patient endorsed no difficulties on the PDQ-39 at baseline. The fact that this patient elected to proceed with DBS would suggest that his preoperative responses were likely invalid. The second patient had a history of well-controlled bipolar disorder and experienced worsening depression postoperatively.

The most important findings of this study are related to variables that can better inform surgical prognostication. The strongest predictor of postoperative QoL was single-trial learning from the Rey Auditory Verbal Learning Test. Single-trial learning reflects the efficiency of complex verbal encoding and retrieval from episodic memory and therefore it is sensitive to dysfunction in distributed frontal and temporal circuits.13 In other words, poorer memory after a single learning trial may indicate a greater “disease burden” outside midbrain and subcortical regions. In that case, our model would suggest that patients with more preserved function in cortical regions before surgery report greater increases in QoL after surgery.

Preoperative QoL was also an important predictor of postoperative QoL ratings. Essentially, patients with poorer QoL before surgery also endorsed relatively poorer QoL after surgery, consistent with the high test-retest reliability of the measure. It is notable that depression contributed unique variance to the model predicting whether patients showed clinically significant change in PDQ scores after surgery, suggesting that mood is a major influence in self-reported QoL. This underscores the importance of psychiatric care accompanying surgical care.

Our model did not show predictive value of variables that predict motor outcome. In particular, preoperative UPDRS was unrelated to QoL outcome in our and others' investigations.6,7 We suspect that this could be attributable to the fact that it is not the degree of motor impairment that occurs in the off state, but rather the amount of time spent in the off state that matters for QoL.6 Likewise, preoperative medication response did not predict QoL rating change. This is perhaps the most surprising of the nonpredictors because better motor outcomes are expected in patients who respond well to medications. We suspect that the lack of predictive value is at least partially attributable to a restricted range of data available—patients with minimal medication response (<30% change in UPDRS-III scores on/off) typically do not undergo surgery because of low expectation for motor benefit. There may be a substantial relationship such that patients with limited medication response/minimal motor improvement may experience no change or worsening QoL, but we are unable to gauge this because of lack of data at that end of the scale.

Previous work suggests that QoL outcomes worsen with lower scores (130–137) on the DRS.9 In contrast, our data indicate that patients with scores even lower than 130 demonstrate stable or improved QoL ratings based on reliable change indices. Notably, patients with low DRS-2 scores often rated their QoL as better than patients with higher DRS scores. It is plausible to speculate that for many patients, borderline cognitive function is less limiting to QoL than inability to move. Gains in independence due to improved mobility may very well compensate for limited cognitive losses. Alternatively, this could reflect lack of insight into one's deficits or the influence of increased support/reduced functional demands that these patients may experience as a consequence of cognitive impairments. Understanding the severity of cognitive loss that will negatively affect DBS outcomes is imperative to guide best practices for patient selection, and this issue clearly requires further attention in future studies.

Co-occurring improvements in depression and undergoing bilateral surgery accounted for substantial variance in PDQ-39 change. The first finding is again consistent with the idea that low mood is associated with actual or perceived reductions in QoL. The benefit of bilateral over unilateral surgery was unexpected. This may reflect the subtle tendency for bilateral patients to report poorer QoL before surgery or perhaps the tendency for greater improvement in postsurgical UPDRS-III scores in bilateral patients when off medications.

There were 3 important negative findings in examining the factors that could accompany changes in QoL after STN-DBS. First, postoperative reduction in medications was not associated with improved QoL. This is a very common surgical goal for patients and achieving that goal would presumably contribute to better QoL. However, a neuroethics study completed at our center indicated that patients often report disparate surgical goals when asked before and after the procedure (Cynthia Kubu, personal communication, 2014). Second, effectiveness of stimulation (off medication vs off medication/on stimulation) had no influence on QoL changes. This can be accounted for by our hypothesis that it is the time in the off state, rather than the severity of motor problems while off, that dictates QoL change. Third, changes in verbal fluency after surgery were not associated with worsening QoL, which indicates that the most frequently reported cognitive decline after DBS implantation does not have a negative impact on outcome from the patient's perspective.

There are a number of potential limitations of the current study. First, this is a retrospective study of patients treated according to clinical decision-making. As such, group differences are likely inherent in comparisons of patients undergoing unilateral vs bilateral DBS. A prospective trial of QoL outcome in patients randomized to unilateral or bilateral STN-DBS would be necessary to draw definitive conclusions about the QoL benefits of surgical approaches. In addition, the current study involves a fairly small sample size for complex model building. We were cautious of power issues in constructing our analyses, which should be considered to be preliminary until cross-validated in other samples. Finally, potentially important motor function variables such as cumulative time off or dyskinesia ratings were not available for the present analysis. The inclusion of these variables could conceivably change the relative weighting of the nonmotor variables in predicting QoL outcome.

Surgical intervention appears to improve or maintain QoL for the vast majority of patients. The focus of prior DBS research has been on predicting motor improvements after surgery, which reflects the success of DBS from the clinician's perspective. The recent interest in QoL outcomes signals a shift to understanding the patient characteristics and disease variables that underlie success from the patient's perspective. Our model indicates that single-trial learning, depression, and self-reported QoL before surgery are important indicators of postoperative QoL outcome.

GLOSSARY

BDI-II

Beck Depression Inventory–2nd edition

CI

confidence interval

DBS

deep brain stimulation

DRS-2

Dementia Rating Scale–2nd edition

OR

odds ratio

PD

Parkinson disease

PDQ-39

39-item Parkinson's Disease Questionnaire

QoL

quality of life

STN

subthalamic nucleus

UPDRS-III

Unified Parkinson's Disease Rating Scale–III

AUTHOR CONTRIBUTIONS

Darlene Floden conceived and designed the study, performed the statistical analysis, interpreted the findings, and wrote and revised the manuscript. Scott E. Cooper contributed to the conceptualization of the study, interpretation of the findings, and revision of the manuscript. Sandra D. Griffith contributed to the data analysis and revision of the manuscript. Andre G. Machado assisted with study conceptualization and design, data interpretation, and manuscript preparation and revision.

STUDY FUNDING

Portions of this research were funded by the Cleveland Clinic Center for Neurological Restoration and the NIH (D.F.: R01 NS058706-01; A.G.M.: R01 HD061363-01).

DISCLOSURE

D. Floden, S. Cooper, and S. Griffith report no disclosures relevant to the manuscript. A. Machado reports the following: potential future distribution from Intellectual Property licensed to Enspire, Cardionomics, and ATI; past distribution (more than 12 months ago) from Intellect Medical/Boston Scientific; and consultant with Spinal Modulation and Functional Neuromodulation. Go to Neurology.org for full disclosures.

REFERENCES

  • 1.Rodriguez-Oroz MC, Obeso JA, Lang AE, et al. Bilateral deep brain stimulation in Parkinson's disease: a multicentre study with 4 years follow-up. Brain 2005;128:2240–2249. [DOI] [PubMed] [Google Scholar]
  • 2.Deuschl G, Schade-Brittinger C, Krack P, et al. A randomized trial of deep-brain stimulation for Parkinson's disease. N Engl J Med 2006;355:896–908. [DOI] [PubMed] [Google Scholar]
  • 3.Weaver FM, Follett K, Stern M, et al. Bilateral deep brain stimulation vs best medical therapy for patients with advanced Parkinson disease: a randomized controlled trial. JAMA 2009;301:63–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Smeding HMM, Speelman JD, Huizenga HM, Schuurman PR, Schmand B. Predictors of cognitive and psychosocial outcome after STN DBS in Parkinson's disease. J Neurol Neurosurg Psychiatry 2011;82:754–760. [DOI] [PubMed] [Google Scholar]
  • 5.Williams A, Gill S, Varma T, et al. Deep brain stimulation plus best medical therapy versus best medical therapy alone for advanced Parkinson's disease (PD SURG trial): a randomised, open-label trial. Lancet Neurol 2010;9:581–591. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Daniels C, Krack P, Volkmann J, et al. Is improvement in the quality of life after subthalamic nucleus stimulation in Parkinson's disease predictable? Mov Disord 2011;26:2516–2521. [DOI] [PubMed] [Google Scholar]
  • 7.Nazzaro JM, Pahwa R, Lyons KE. The impact of bilateral subthalamic stimulation on non-motor symptoms of Parkinson's disease. Parkinsonism Relat Disord 2011;17:606–609. [DOI] [PubMed] [Google Scholar]
  • 8.Zahodne LB, Okun MS, Foote KD, et al. Greater improvement in quality of life following unilateral deep brain stimulation surgery in the globus pallidus as compared to the subthalamic nucleus. J Neurol 2009;256:1321–1329. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Witt K, Daniels C, Krack P, et al. Negative impact of borderline global cognitive scores on quality of life after subthalamic nucleus stimulation in Parkinson's disease. J Neurol Sci 2011;310:261–266. [DOI] [PubMed] [Google Scholar]
  • 10.Peto V, Jenkinson C, Fitzpatrick R, Greenhall R. The development and validation of a short measure of functioning and well being for individuals with Parkinson's disease. Qual Life Res 1995;4:241–248. [DOI] [PubMed] [Google Scholar]
  • 11.Beck AT, Steer RA, Brown GK. Beck Depression Inventory, 2nd ed. San Antonio: The Psychological Corporation; 1996. [Google Scholar]
  • 12.Rodriguez-Oroz MC, Moro E, Krack P. Long-term outcomes of surgical therapies for Parkinson's disease. Mov Disord 2012;27:1718–1728. [DOI] [PubMed] [Google Scholar]
  • 13.Hanslmayr S, Volberg G, Wimber M, Raabe M, Greenlee MW, Bauml KHT. The relationship between brain oscillations and BOLD signal during memory formation: a combined EEG-fMRI study. J Neurosci 2011;31:15674–15680. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Jurica PJ, Leitten CL, Mattis S. Dementia Rating Scale–2: Professional Manual. Lutz, FL: Psychological Assessment Resources; 2001. [Google Scholar]
  • 15.Kaplan E, Goodglass H, Weintraub S. Boston Naming Test. Philadelphia: Lea & Febiger; 1983. [Google Scholar]
  • 16.Benton A, Hamsher KD. Multilingual Aphasia Examination. Iowa City: AJA Associates; 1989. [Google Scholar]
  • 17.Smith A. Symbol Digit Modalities Test Manual (Revised). Los Angeles: Western Psychological Services; 1982. [Google Scholar]
  • 18.Benton AL, Sivan AB, Hamsher KdS, Varney NR, Spreen O. Judgment of Line Orientations. Lutz, FL: Psychological Assessment Resources; 1983. [Google Scholar]
  • 19.Lezak MD. Neuropsychological Assessment, 2nd ed. New York: Oxford University Press; 1983. [Google Scholar]
  • 20.Wechsler D. Wechsler Memory Scale–III Administration and Scoring Manual. San Antonio: The Psychological Corporation; 1997. [Google Scholar]
  • 21.Heaton R. A Manual for the Wisconsin Card Sorting Test. Odessa, FL: Psychological Assessment Resources; 1981. [Google Scholar]

Articles from Neurology are provided here courtesy of American Academy of Neurology

RESOURCES