Abstract
Apathy is a unique, multidimensional syndrome commonly encountered in patients with Parkinson disease (PD). Recently, the Lille Apathy Rating Scale (LARS), a semistructured interview yielding a global score, and composite subscores for different domains of apathy (i.e., cognitive, behavioral, affective, self awareness), was developed and given to a sample of patients with PD in France. This study is the first outside of its original developers to examine the English language version of the LARS in PD. We found the LARS to be a coherent instrument demonstrating both convergent and divergent validity, as compared to the Apathy Scale (AS) and Beck Depression Inventory (BDI-II). Using a receiver operating characteristic (ROC) analysis comparing the LARS to the AS, a validated and widely-used measure, we identified a cut-off score (sensitivity = 64%, specificity = 92%, PPV = 88%, NPV = 75%) that was higher than that proposed by the original authors, who derived their cutoff by comparing LARS global scores to clinical judgments of apathy. Although the present study does not compare the LARS to a diagnostic gold standard or promote its utility for diagnosing apathy, it provides further support for the LARS as a promising instrument to examine apathy in PD.
Keywords: Parkinson’s disease, apathy, rating scales
INTRODUCTION
Apathy is a common neuropsychiatric feature of Parkinson disease (PD) that is distinct from depression.1–3 Although apathy is often a symptom of depression (defined by the Diagnostic and Statistical Manual of Mental Disorders [4th ed.; DSM-IV]), it may also manifest as an independent syndrome.4 Apathy has been defined as a primary lack of motivation that may affect cognitive, behavioral, and affective domains. Cognitively, apathy appears as lack of interest and concern. Behaviorally, apathy manifests in reduced productivity and reliance on others to structure daily activities. This domain may also be conceptualized as a deficit in “auto-activation.”5 Affectively, apathy appears as blunted affect and reduced responsiveness to positive or negative stimuli. One key distinction between apathy and depression is that the symptomatic dysphoria seen in depression is absent in an apathy syndrome. Indeed, a lack of strong emotional responses, regardless of valence, appears to be symptomatic of apathy.6
Recently, the Movement Disorders Society (MDS) commissioned a task force to assess the clinimetric properties of apathy and anhedonia scales in PD.7 This group concluded that evidence supports only the Apathy Scale (AS) and item 4 from the Unified Parkinson Disease Rating Scale as “recommended” for the assessment of apathy in PD. However, as the latter is only a single item, it was recommended only for “crude screening purposes.” The AS8 is a 14-item self-report instrument that provides an acceptable and relatively quick screen for apathy. However, it yields only a global score and thus provides little information about profiles of apathy, or the relative contributions of apathy subcomponents to overall severity.
Recently, the Lille Apathy Rating Scale (LARS) was developed and given to 159 Parkinson’s patients in Lille, France.9 The LARS is a semistructured interview based on Marin’s original conceptualization. It provides an overall apathy score and four composite subscores that presumably reflect four distinct dimensions of apathy: intellectual curiosity, action initiation, emotion, and self awareness. Cut-off scores were derived from ratings by independent clinicians who evaluated apathy presence and severity. Psychometric examination of the LARS indicates strong convergent validity with the clinician version of Marin’s original 18-item AES (r = 0.87) and high reliability with regard to internal consistency (Cronbach’s α = 0.80), four-month test–retest reliability (r = 0.95), and inter-rater reliability (ICC = 0.98). To date, the LARS has neither been validated in patients with PD in the U.S. nor has it been compared to the commonly-used AS. Although the MDS task force described the LARS as “well-designed and promising,” they concluded it is only “suggested” for the purpose of assessing apathy in PD due a lack of use beyond its original developers.7
The purpose of this study was to examine various psychometric characteristics of the LARS in a sample of patients with PD in the U.S. by comparing it to the AS, which is routinely administered at our movement disorders center as part of regular clinical care and clinical trials research. We also sought to explore the relationship between apathy and other patient and disease characteristics in the same sample.
PATIENTS AND METHODS
Participants
Participants included 71 patients with probable idiopathic PD being followed by the University of Florida Movement Disorders Center. Demographic and disease characteristics are shown in Table 1. Diagnoses were made according to the United Kingdom Parkinson’s disease brain bank criteria and confirmed by neurologists specializing in movement disorders.10 Patients were recruited via three methods: consecutively between September 2006 and February 2008 (N = 43) during a research visit included in a pre-DBS candidacy evaluation, as part of a clinical trial for the treatment of apathy (N = 15), and consecutively (N = 13) during the Summer of 2007 during a routine follow-up visit to the UF Movement Disorders Clinic to provide a broader representation of patients and symptom severity.
TABLE 1.
Mean | SD | Range | |
---|---|---|---|
Demographic (N = 71) | |||
Age | 64.0 | 10.0 | [31, 87] |
Education | 15.1 | 2.8 | [10, 20] |
Male/female | 48/23 | ||
Disease | |||
Hoehn & Yahr | 2.4 | .58 | [2, 4] |
UPDRS “on” | 28.7 | 11.4 | [10, 53] |
UPDRS “off” | 39.6 | 15.4 | [12, 81] |
Months with symptoms | 121.7 | 56.4 | [27, 277] |
LED | 846.7 | 413.4 | [100, 1951] |
Disease data were gathered for a subset of patients (Hoehn & Yahr N = 53, UPDRS “on” N = 55, months with symptoms N = 60, LED N = 61).
UPDRS, Unified Parkinson Disease Rating Scale; LED, levodopa equivalent dose.
Mood and Apathy Assessments
Apathy Scale8
Apathy was assessed with the 14-item self-report Apathy Scale (AS) that employs a four-point Likert-type scale. The AS was modified from the 18-item AES, and validity for the latter was established via two methods: the multitrait–multimethod matrix procedure, and experimentally by comparing apathy scores to performance in a novelty toy/waiting room experiment.11 The AS was validated in a sample of 50 consecutive patients with PD and is commonly administered in a variety of clinical and research settings. Scores range from 0 to 42, with higher scores indicating greater apathy. A score of 14 or greater on the AS is indicative of clinically meaningful apathy in PD, as determined by comparison with ratings by a neurologist blinded to their AS score and examination of frequency distributions.
Lille Apathy Rating Scale9
Apathy was also assessed with the Lille Apathy Rating Scale (LARS). The LARS is a 33-item semistructured interview tapping nine domains (i.e., everyday productivity, interests, taking initiative, novelty seeking, voluntary actions, emotional responses, concern, social life, and self-awareness). Each domain contributes equally to the global score. Items are scored yes/no except for the first three questions, which are scored on a five-point Likert-type scale. Global LARS scores range from −36 to +36, with higher scores indicating greater apathy. The LARS also yields four composite subscores that are derived from domain scores and range from −4 to +4.
The developers of the LARS concluded that a global LARS cut-off of −16 best differentiated apathetic and non-apathetic individuals with sensitivity and specificity of 0.89 and 0.92, respectively. Of note, the authors also empirically derived a cut-off score of 21 for the clinician version of the AES in their sample by comparing scores to the same clinician ratings. The authors further proposed a system for classifying the severity of apathy based on LARS global scores, with scores between −36 and −21 suggesting the absence of apathy, between −21 and −16 suggesting mild apathy, between −16 and −9 indicating moderate apathy, and above −9 indicating severe apathy.
Beck Depression Inventory12
Depressive symptom severity was assessed using the Beck Depression Inventory, second edition (BDI-II), a self-report measure commonly administered to a variety of patient populations, including PD. Scores range from 0 to 63, and the recommended cut-off in patients with PD is 14 of 15.13
Procedures
Patients completed the LARS, AS, and BDI-II during the same visit or within 1 day. Global cognitive status was assessed in a subset of patients (N = 35) using the Mattis Dementia Rating Scale, second edition (DRS-2)14 as part of a larger battery of neuropsychological tests. The mean, raw global score was 137.1 (SD = 6.3; range 113–144). Five patients evidenced possible dementia, as defined by a DRS-2 cut-off score of 130.
RESULTS
Recruitment Modality
One-way analyses of variance (ANOVAs) followed by Bonferroni-adjusted follow-up comparisons identified no significant differences on any of the demographic or cognitive variables between the three recruitment modalities. However, as compared to patients recruited from the apathy clinical trial, DBS candidates had lower UPDRS-III scores when assessed “on” medications (X = 24.5 vs. X = 37.3, P < 0.01), lower AS scores (X = 10.6 vs. X = 18.0, P < 0.01), and lower BDI-II scores (X = 10.0 vs. X = 16.5, P = 0.02).
Apathy and Depression Measures
Mean scores on each of the three psychological measures (AS, LARS, and BDI-II) are shown in Table 2. Patients scored slightly below the cut-off for clinically significant apathy, as indexed by the AS (M = 12.8). Thirty-three patients (46%) obtained a score of at least 14 on this measure, indicating the presence of apathy. Of the 66 patients who completed the BDI-II, 20 (30%) obtained a score of at least 15, indicating the presence of depression symptoms. Using the original classification system, 49 patients in our sample fell within the “no apathy” range and 22 fell within the “apathy” range. Of these 22, 11 evidenced “mild” apathy, 7 evidenced “moderate” apathy, and 4 evidenced “severe” apathy. A one-way ANOVA followed by Bonferroni-adjusted comparisons indicated that patients exhibited similar scores on three of the four LARS factorial subscores (see Table 2). However, the mean score on the Self Awareness subscale was significantly lower than on the intellectual curiosity (X = −2.42 vs. X = −3.18, P = 0.002) and emotion (X = −2.27 vs. X = −3.18, P < 0.001) subscales, indicating less severe pathology in the Self Awareness domain.
TABLE 2.
Mean | SD | Range | |
---|---|---|---|
BDI-II | 11.7 | 7.9 | [0, 31] |
AS | 12.8 | 6.6 | [1, 27] |
LARS global | −22.9 | 8.4 | [−34, 15] |
Intellectual curiosity | −2.42 | 1.27 | [−3.75, 2.5] |
Action initiation | −2.77 | 1.28 | [−4, 1.5] |
Emotion | −2.27 | 1.09 | [−4, 1] |
Self awareness | −3.18 | 1.29 | [−4, 2] |
BDI-II, Beck Depression Inventory, 2nd edition; AS, Apathy Scale; LARS, Lille Apathy Rating Scale.
Reliability
Reliability of the LARS was assessed in a subset of 63 patients via two methods for determining internal consistency. Half-test reliability was found to be 0.56, and full-scale split-half reliability was found to be 0.72 after correction with the Spearman Brown prophecy formula. Cronbach’s standardized α coefficients were calculated between items (α = 0.82) and between the nine subscales (α = 0.75). All nine subscales correlated with the LARS global score (range: r = 0.42 to r = 79). Item reliability analyses were conducted on only 32 items, as one item from the Self Awareness domain (Item 33: If, during a discussion, you realize that you’re in the wrong, are you able to admit it—at least to yourself?) was found to have zero variance; all patients in the present sample responded “yes” to this question.
Validity
Convergent and divergent validity were assessed by calculating intraclass correlation coefficients with total AS and total BDI-II scores, respectively. In the ICC analyses, two-way mixed models with 95% confidence intervals were employed. Total AS and LARS scores showed adequate agreement (ICC = 0.75). Further comparisons between AS and each of the four LARS composite subscores revealed significant correlations with intellectual curiosity (r = 0.61; P < 0.001), action initiation (r = 0.42; P < 0.001), and emotion (r = 0.33; P < 0.01). The AS correlated at trend level with Self Awareness (r = 0.229; P = 0.05). With regard to divergent validity, total BDI-II and LARS scores evidenced less agreement (ICC = 0.62). To test the hypothesis that the LARS would correlate more highly with the AS than the BDI-II, William’s t-test15 was conducted, as recommended by Steiger.16 Although the LARS was found to correlate significantly with both AS (r = 0.62; P < 0.001) and BDI-II (r = 0.45; P < 0.001), the former correlation was greater (t (63) = 1.81; P < 0.05).
Operating characteristics for the LARS were calculated against the AS. Apathy was defined as ≥14 on the AS, and a contingency cross tabulation matrix was generated for the original author-recommended LARS cut-off score of −16. Sensitivity, specificity, positive predictive value, and negative predictive value were found to be 0.30, 0.97, 0.91, and 0.62, respectively.
Next, a Receiver Operating Characteristic (ROC) plot,17 shown in Figure 1, was generated. Area under the curve (AUC) was found to be 0.78, which was significant different from the null value of 0.5 (P < 0.001; 95% CI [0.67, 0.90]). A randomly selected individual scoring ≥14 on the AS has a higher LARS score than a randomly selected individual scoring <14 on the AS 78% of the time. Inspection of the ROC curve identified an ideal cut-off score of −22. Contingency cross tabulation tables were generated for this newly-identified cut-off score and for scores immediately above and below. Operating characteristics of each are presented in Table 3.
TABLE 3.
Sensitivity | Specificity | PPV | NPV | |
---|---|---|---|---|
Cut-off-23 | 0.64 | 0.82 | 0.75 | 0.72 |
Cut-off-22 | 0.64 | 0.92 | 0.88 | 0.75 |
Cut-off-21 | 0.58 | 0.92 | 0.86 | 0.71 |
PPV, positive predictive value; NPV, negative predictive value.
Relationship Between LARS Subscores and Other Variables
Table 4 presents a correlation matrix describing the relationships between AS, LARS global and four composite scores, and select patient characteristics. Significant correlations were identified between the LARS global score, the factorial subscores intellectual curiosity and action initiation, and the patient characteristics of depression and motor severity (UPDRS-III “on”). Neither the LARS global score nor any of the four subscores correlated with levodopa equivalent dose, patient self-report of PD symptom duration, or motor severity “off” medications in our sample.
TABLE 4.
AS | LARS total | IC | AI | E | SA | BDI-II | UPDRS “on” | UPDRS “off” | HY | LED | Months with symptoms | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
AS | ||||||||||||
LARS total | 0.615* | |||||||||||
IC | 0.605* | 0.904* | ||||||||||
AI | 0.420* | 0.725* | 0.506* | |||||||||
E | 0.333** | 0.587* | 0.356* | 0.268** | ||||||||
SA | 0.229 | 0.478* | 0.322* | 0.251** | 0.181 | |||||||
BDI-II | 0.582* | 0.450* | 0.448* | 0.344* | 0.207 | 0.730 | ||||||
UPDRS “on” | 0.385** | 0.469* | 0.396* | 0.378* | 0.237 | 0.321** | 0.382* | |||||
UPDRS “off” | −0.061 | 0.043 | −0.038 | 0.079 | 0.073 | 0.101 | 0.029 | 0.248 | ||||
HY | 0.138 | 0.166 | 0.121 | 0.117 | 0.174 | 0.078 | 0.301** | 0.651* | 0.354* | |||
LED | −0.058 | 0.024 | −0.070 | 0.178 | 0.087 | −0.058 | −0.004 | 0.037 | 0.139 | 0.255 | ||
Months with symptoms | −0.099 | −0.157 | −0.178 | 0.031 | −0.171 | −0.067 | 0.338** | 0.062 | 0.338** | 0.328** | 0.473* |
P < 0.001.
P < 0.05.
AS, Apathy Scale; LARS, Lille Apathy Rating Scale; IC, intellectual curiosity; AI, action initiation; E, emotion; SA, self awareness; BDI-II, Beck Depression Inventory, 2nd edition; UPDRS-III, Unified Parkinson’s Disease Rating Scale, Motor; HY, Hoehn & Yahr Stage Scale; LED, levodopa equivalent dosage.
DISCUSSION
This study is the first to examine the performance of the English language version of the Lille Apathy Rating Scale (LARS) in assessing apathy in PD by comparing it to the AS within a sample of patients with PD in the United States. Although the AS is a recommended screening measure for apathy that can be administered to large numbers of patients in a relatively short period of time, it does not provide information about the different clinical manifestations (i.e., cognitive, behavioral, affective) of apathy. The LARS represents a potentially useful instrument in characterizing the PD apathy syndrome. We found the LARS to be a moderately reliable and valid measure of apathy in our sample of patients with PD. The instrument exhibited good between-item and between-subscale correlations as well as split-half reliability, suggesting that it measures a coherent construct. Global scores were highly correlated to AS scores with an ICC greater than 0.70, indicating strong convergent validity. It correlated more highly with the AS than with the BDI-II, correlating with the latter with an ICC less than 0.70, which suggests that the LARS measures apathy per sé rather than global psychological symptomatology or depression.
We compared the LARS to the commonly-used AS using the authors’ proposed cut-off score as well as that identified using the ROC plot in our sample. Although the authors’ cut-off score of −16 demonstrated excellent specificity and positive predictive value compared to the AS cut-off of 13 of 14, its sensitivity was very poor. Many patients who scored 14 or higher on the AS did not score −16 or higher on the LARS (false negatives). This discrepancy is likely attributable to the designers’ deriving a cut-off score by comparing patient scores to judgments made by two clinicians based on interviews with the patient and family member. Although the authors suggest that a cut-off score of −16 indicates “clinically significant apathy,” their extended classification system defines a lower range of scores (−21 to −16) as indicative of “mild apathy.”
The LARS cut-off value determined to possess the best operating characteristics when compared to the AS in our sample was −22. Using this value, the instrument demonstrated similar specificity and positive predictive value and higher sensitivity and negative predictive value. This cut-off score may be better for comparing apathy prevalence to the many studies that have used the AS.
We identified the presence of apathy in 46% of our sample using the AS, which is consistent with previous reports of patients with PD.1,8,18,19 However, this number does not necessarily reflect the true prevalence of apathy in patients with PD because our sample was not randomly selected. Specifically, 61% of our sample comprised candidates for DBS surgery who differed (only from the 15 clinical trial patients) in motor symptom, apathy, and depression severity. Consistent with the findings of Sockeel et al., we found that apathy, as measured by the AS, was most strongly related to the LARS composite subscores of intellectual curiosity and action initiation, which the authors suggest might reflect cognitive and behavioral dimensions of apathy, respectively.9,20
Comparisons between the four LARS composite sub-scores and other patient variables (i.e., depression severity, motor functioning, disease duration, levodopa equivalent dose) revealed that apathy was related to the severity of depression and the severity of motor symptoms “on” medications. It should be noted that some of these variables were only available for a subset of our sample, as they were obtained from an IRB-approved MDC database (see Table 1 note). Interestingly, apathy was not related to levodopa equivalent dosage (LED), motor severity “off” medications, or patient report of PD symptom duration in our sample. Apathy has been found to be greater in patients tested off levodopa medications; however, these authors did not report on a relationship between LED and apathy.21 It has been suggested that apathy may emerge after deep brain stimulation surgery for PD as a result of the reductions in levodopa medications following surgery or DBS itself.22–26 However, research on post-DBS apathy is conflicting due, in part, to methodological limitations.27 Future research is needed to characterize how these important variables interact over the disease course.
An important limitation of this study lies in the use of the AS to classify patients as apathetic, rather than formal diagnostic criteria. Unfortunately, such criteria do not yet exist, although some have been recommended by Starkstein and colleagues.7,28 An important goal in future studies on assessing apathy in PD should aim to validate such criteria, taking into consideration different models of apathy. As a result, our analyses cannot determine the diagnostic utility of the LARS. A related limitation lies in the self-report format of the AS. Patients suffering from dementia and severe apathy may lack the awareness required for valid reporting.29 Although a subset of the present sample believed to be representative did not evidence dementia on average, as determined with a dementia rating scale (DRS-2), 5 of these 35 patients scored below the suggested cut-off for possible dementia (i.e., below 130).
In conclusion, this study represents the first to examine a new, multidimensional, semistructured interview for the assessment of apathy in PD in the U.S. It is also the first to compare the LARS to the commonly-used and recommended AS. Results suggest that it is a coherent instrument with good convergent and divergent validity as well as moderate utility in identifying apathy amongst patients with PD. Although apathy is commonly regarded as a multidimensional construct comprising several domains (i.e., behavioral, cognitive, and affective), little research has aimed to empirically demonstrate the separability of these manifestations in PD, neurobiologically or behaviorally. The first step toward establishing correlates of these domains requires a reliable and valid method of their assessment. In addition, given that the relative prominence of different apathy domains across individuals may have important treatment and prognostic implications, the clinical value of an instrument that can adequately quantify and differentiate these symptom classes is clear. Future research should attempt to explore the utility of the LARS in shedding light on the multiple dimensions of the PD apathy syndrome and their relationships with the disease process and patient quality of life.
Acknowledgments
We would like to acknowledge the support National Parkinson Foundation Center of Excellence, the McKnight Brain Institute, UF and Shands, and the College of Medicine.
Footnotes
Potential conflict of interest: None reported.
Author Roles: LBZ, DB, MSO, and SY carried out conceptualization and design of the study. LBZ executed data analysis and prepared the text of this manuscript. DB and LK-D assisted with writing and/or editing. MSO, HHF, LBZ, SY, and AN carried out patient assessments and data collection. All authors have viewed and approved the content of this manuscript.
References
- 1.Kirsch-Darrow L, Fernandez HH, Marsiske M, Okun MS, Bowers D. Dissociating apathy and depression in Parkinson disease. Neurology. 2006;67:33–38. doi: 10.1212/01.wnl.0000230572.07791.22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Levy ML, Cummings JL, Fairbanks LA, et al. Apathy is not depression. J Neuropsychiatry Clin Neurosci. 1998;10:314–319. doi: 10.1176/jnp.10.3.314. [DOI] [PubMed] [Google Scholar]
- 3.Isella V, Melzi P, Grimaldi M, et al. Clinical, neuropsychological, and morphometric correlates of apathy in Parkinson’s disease. Mov Disord. 2002;17:366–371. doi: 10.1002/mds.10041. [DOI] [PubMed] [Google Scholar]
- 4.Marin RS. Apathy: a neuropsychiatric syndrome. J Neuropsychiatry Clin Neurosci. 1991;3:243–254. doi: 10.1176/jnp.3.3.243. [DOI] [PubMed] [Google Scholar]
- 5.Levy R, Dubois B. Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits. Cereb Cortex. 2006;16:916–928. doi: 10.1093/cercor/bhj043. [DOI] [PubMed] [Google Scholar]
- 6.Mimura M. Depression and apathy in Parkinson disease. Brain Nerve. 2007;59:935–942. [PubMed] [Google Scholar]
- 7.Leentjens AFG, Dujardin K, Marsh L, et al. Apathy and anhedonia rating scales in Parkinson’s disease: critique and recommendations. Mov Disord. 2008;23:2004–2014. doi: 10.1002/mds.22229. [DOI] [PubMed] [Google Scholar]
- 8.Starkstein SE, Mayberg HS, Preziosi TJ, et al. Reliability, validity, and clinical correlates of apathy in Parkinson’s disease. J Neuropsychiatry Clin Neurosci. 1992;4:134–139. doi: 10.1176/jnp.4.2.134. [DOI] [PubMed] [Google Scholar]
- 9.Sockeel P, Dujardin K, Devos D, Denève C, Destée A, Defebvre L. The Lille apathy rating scale (LARS), a new instrument for detecting and quantifying apathy: validation in Parkinson’s disease. J Neurol Neurosurg Psychiatry. 2006;77:579–584. doi: 10.1136/jnnp.2005.075929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hughes AJ, Daniel SE, Kilford L, Lees AJ. Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: a clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry. 1992;55:181–184. doi: 10.1136/jnnp.55.3.181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Marin RS, Biedrzycki RC, Firinciogullari S. Reliability and validity of the Apathy Evaluation Scale. Psychiatry Res. 1991;38:143–162. doi: 10.1016/0165-1781(91)90040-v. [DOI] [PubMed] [Google Scholar]
- 12.Beck AT, Steer R, Brown G. The Beck depression inventory-II. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
- 13.Visser M, Leentjens AF, Marinus J, et al. Reliability and validity of the Beck depression inventory in patients with Parkinson’s disease. Mov Disord. 2006;21:668–672. doi: 10.1002/mds.20792. [DOI] [PubMed] [Google Scholar]
- 14.Mattis S. Dementia rating scale-2. Odessa, FL: Psychological Assessment Resources; 2001. [Google Scholar]
- 15.Williams EJ. The comparison of regression variables. J R Stat Soc Ser B. 1959;21:396–399. [Google Scholar]
- 16.Steiger JH. Tests for comparing elements of a correlation matrix. Psychol Bull. 1980;87:245–251. [Google Scholar]
- 17.Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39:561–577. [PubMed] [Google Scholar]
- 18.Pluck GC, Brown RG. Apathy in Parkinson’s disease. J Neurol Neurosurg Psychiatry. 2002;73:636–642. doi: 10.1136/jnnp.73.6.636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Aarsland D, Larsen JP, Lim NG, et al. Range of neuropsychiatry disturbances in patients with Parkinson’s disease. J Neurol Neurosurg Psychiatry. 1996;67:492–496. doi: 10.1136/jnnp.67.4.492. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Dujardin K, Sockeel P, Devos D, et al. Characteristics of apathy in Parkinson’s disease. Mov Disord. 2007;22:778–784. doi: 10.1002/mds.21316. [DOI] [PubMed] [Google Scholar]
- 21.Czernecki V, Pillon B, Houeto JL, Pochon JB, Levy R, Dubois B. Motivation, reward, and Parkinson’s disease: influence of dopatherapy. Neuropsychologia. 2002;40:2257–2267. doi: 10.1016/s0028-3932(02)00108-2. [DOI] [PubMed] [Google Scholar]
- 22.Funkiewiez A, Ardouin C, Cools R, et al. Effects of levodopa and subthalamic nucleus stimulation on cognitive and affective functioning in Parkinson’s disease. Mov Disord. 2006;21:1656–1662. doi: 10.1002/mds.21029. [DOI] [PubMed] [Google Scholar]
- 23.Tasker RR, Kiss ZHT. The role of the thalamus in functional neurosurgery. Neurosurg Clin N Am. 1995;6:73–104. [PubMed] [Google Scholar]
- 24.Fields JA, Tröster AI. Cognitive outcomes after deep brain stimulation for Parkinson’s disease: a review of initial studies and recommendations for future research. Brain Cognit. 2000;42:268–293. doi: 10.1006/brcg.1999.1104. [DOI] [PubMed] [Google Scholar]
- 25.Drapier D, Péron J, Leray E, et al. Emotion recognition impairment and apathy after subthalamic nucleus stimulation in Parkinson’s disease have separate neural substrates. Neuropsychologia. 2008;46:2796–2801. doi: 10.1016/j.neuropsychologia.2008.05.006. [DOI] [PubMed] [Google Scholar]
- 26.Czernecki V, Shüpbach M, Yaici S, et al. Apathy following subthalamic stimulation in Parkinson disease: a dopamine responsive symptom. Mov Disord. 2008;23:964–969. doi: 10.1002/mds.21949. [DOI] [PubMed] [Google Scholar]
- 27.Kirsch-Darrow L, Mikos A, Bowers D. Does deep brain stimulation induce apathy in Parkinson’s disease? Front Biosci. 2008;13:5316–5322. doi: 10.2741/3083. [DOI] [PubMed] [Google Scholar]
- 28.Starkstein SE. Apathy and withdrawal. Int Psychogeriatrics. 2000;12:135–137. [Google Scholar]
- 29.Starkstein SE, Petracca G, Chemerinski E, Kremer J. Syndromic validity of apathy in Alzheimer’s disease. Am J Psychiatry. 2001;158:872–877. doi: 10.1176/appi.ajp.158.6.872. [DOI] [PubMed] [Google Scholar]