Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 1.
Published in final edited form as: J Clin Exp Neuropsychol. 2016 Jun 7;38(9):1015–1025. doi: 10.1080/13803395.2016.1184232

Computerized assessment of goal-directed behavior in Parkinson’s disease

Whitney Fitts 1, Lauren Massimo 1, Nicholas Lim 1, Murray Grossman 1, Nabila Dahodwala 1,2,*
PMCID: PMC4979569  NIHMSID: NIHMS807891  PMID: 27270271

Abstract

Introduction

Apathy is a syndrome characterized by a reduction in goal-directed behavior. Neurodegenerative diseases frequently exhibit apathy. However, we lack an objective measure of apathy. The Philadelphia Apathy Computerized Task (PACT) measures impairments in goal-directed behavior that contribute to apathy, including initiation, planning and motivation. We sought to examine these mechanisms in Parkinson’s disease (PD) patients.

Methods

PD patients and healthy controls with a caregiver were recruited for the study. Participants were administered the PACT, a novel computerized assessment of goal-directed behavior based on reaction times, and the Starkstein Apathy Scale (AS). Care partners completed the Neuropsychiatric Inventory (NPI). T-tests and Wilcoxon rank sum tests were used to compare baseline demographic characteristics of PD and control participants. Linear regressions were used to compare PD patients to controls on each of the three PACT subtasks (initiation, planning and motivation) while controlling for motor slowing. We then compared performance on each PACT subtask between PD subjects defined as apathetic using the NPI and Starkstein Apathy Scale and controls.

Results

We included 30 PD and 15 control participants in the analysis. When controlling for motor slowing, both all PD and PD apathetic subjects were significantly slower than controls on the planning task and on the initiation task. There were no significant differences between PD patients and controls on the motivation tasks.

Conclusions

PD patients showed specific initiation and planning deficits compared to control participants. After using traditional scales to define apathy, PD apathetic patients still exhibited impaired initiation and planning behaviors. These results suggest that the PACT measures aspects of impaired goal-directed behavior that may contribute to apathy in PD.

Keywords: apathy, parkinsonism, diagnostic testing, validity

Introduction

Apathy is a common non-motor symptom in neurodegenerative diseases, including Parkinson’s disease (PD). When present, it leads to higher morbidity and mortality among patients and greater stress and burden among caregivers (Leroi et al., 2012; Pedersen, Alves, Aarsland, & Larsen, 2009; Vilalta-Franch, Calvó-Perxas, Garre-Olmo, Turró-Garriga, & López-Pousa, 2013). Unfortunately, there is little consistency or consensus on how to best define, measure or treat apathy. Apathy was originally defined by Marin (1991) as a lack of motivation that could not be attributed to other environmental or physical factors, such as dementia, depression or emotional stress. Others have broadened this definition and allowed depression and other conditions to be present with apathy (Starkstein et al., 1992). This is an important distinction as apathy has a high co-morbidity with depression and dementia in PD and other neurodegenerative diseases (Dujardin et al., 2007; Starkstein et al., 1992; Starkstein, Ingram, Garau, & Mizrahi, 2005).

Additionally, apathy is a complex syndrome, and researchers have suggested several subdomains within apathy (Levy & Dubois, 2006; Marin, 1991). Most commonly these are divided into cognitive, emotional, and behavioral factors that contribute to apathetic behavior (Marin, 1991). Defining criteria for apathy typically include deficits in at least one of these subdomains and consequently define a heterogeneous overall syndrome (Robert et al., 2009; Starkstein, Petracca, Chemerinski, & Kremer, 2001). More recently, Levy and Dubois (2006) have refined the definition of apathy as “the quantitative reduction of self-generated voluntary or purposeful behaviors” or goal-directed behavior (GDB). They hypothesize three subtypes of apathy: auto-activation, emotional-affective, and cognitive. These three categories roughly align with Marin’s triadic structure; however, there are some subtle differences. As an example, Marin’s “behavioral apathy” implies a lack of all voluntary behaviors, while Levy and Dubois’ auto-activation categorization specifically focuses on the initiation of voluntary behaviors. Levy and DuBois further argue that patients may have selective deficits in any one component of GDB and this contributes to subtypes of apathy.

Unfortunately, methods to diagnose apathy face equal challenges as the definition of apathy. Currently the most common diagnostic methods are validated surveys (completed by either the patient or caregiver), or a physician evaluation. However, a Movement Disorders Society (MDS) task force identified several limitations among existing apathy scales in PD, These included: 1) lack of established diagnostic criteria for apathy against which to validate these scales; 2) confounding influence of depression symptomatology and cognitive impairment in self-report; and 3) lack of studies to compare caregiver and patient report of apathy (Leentjens et al., 2008). This final point is especially important to consider. Caregiver reported surveys may be influenced by feelings of burden (McKinlay et al., 2008) and may mistake physical limitations caused by the disease as apathy. Similarly, patient reported surveys may lack accuracy if cognitive impairment or depression are present (Leritz, Loftis, Crucian, Friedman, & Bowers, 2004). Finally, while physician evaluation may seem the most objective, this too relies primarily on a history obtained by the patient and/or caregiver. Moreover, as there is no current gold standard for apathy, studies have validated current measures against each other or clinical impression. Current MDS-recommended scales to measure apathy in PD include the single UPDRS item testing motivation (Starkstein & Merello, 2007) and the Apathy Scale (AS) (Starkstein et al., 1992). However, newer scales such as the Lille Apathy Rating Scale, that incorporate a multi-dimensional approach to apathy have been developed (P. Sockeel et al., 2006).

There is increasing evidence of the importance of understanding the multidimensional nature of apathy (Kay, Kirsch-Darrow, Zahodne, Okun, & Bowers, 2012; Levy & Dubois, 2006; P Sockeel et al., 2006). These different behavioral clusters described by Levy and Dubois have significant correlations. For example, patients with lesions to the basal ganglia frequently have difficulty initiating activity (Laplane & Dubois, 2001). However, once these patients initiate activity they perform at very high levels (Laplane & Dubois, 2001). This would exemplify an isolated “auto-activation” deficit. Similarly, patients with acquired brain injury who exhibit executive dysfunction also have symptoms of cognitive apathy (Njomboro & Deb, 2014). In PD, apathy is strongly associated with impaired executive function (Poletti, De Rosa, & Bonuccelli, 2012). Furthermore different studies have found that apathy in PD is associated with specific domains of executive function, including set-shifting (Meyer et al., 2014) and planning (Weintraub et al., 2005) which are needed to enact an activity. Similarly, the emotional-affective syndrome of apathy is classically associated with disruptions in the ventro-medial prefrontal cortex, amygdala and ventral striatum (Marin, 1996). These data suggest that understanding the specific GDB deficits in a given patient population may help us understand the underlying cause, and ultimately treatment, of their apathy.

The Philadelphia Apathy Computerized Task (PACT) was designed to objectively measure the deficits in GDB defined by Levy and Dubois—auto-activation, emotional-affective, and cognitive, for which we use the terms initiation, motivation, and planning, respectively (L Massimo, Evans, & Grossman, 2014). There are two potential advantages to the PACT over recommended apathy measures. First, rather than measuring overall apathy, the PACT attempts to measure the three core processes of GDB--initiation, motivation and planning. This produces detailed data about the underlying mechanisms of apathetic behavior. Second, the PACT does not rely solely on patient or caregiver insight and has the potential to be an objective and standardized measure of impaired goal directed behavior and, in turn, apathy.

The PACT has been used previously to measure impaired GDB in apathetic patients with behavioral variant frontotemporal degeneration (L. Massimo et al., 2015). In this study, impaired performance on each PACT component was associated with corresponding degradation in hypothesized prefrontal neural networks underlying goal directed behavior (initiation – anterior cingulate; planning – dorsolateral prefrontal cortex; motivation – orbitofrontal cortex). This is in contrast to prior studies that have examined correlations between apathy measured by existing scales to the same network of brain regions. One structural MRI study found that worse scores on multiple scales were associated with atrophy in a wide range of fronto-parietal brain regions; however, there was no specific correlation between apathy subtypes and distinct pre-frontal brain regions (Reijnders et al., 2010). Furthermore, Isella et al (2002) did not find any correlations between reported apathy symptoms and frontal brain regions. Thus, in an effort to address empirical and methodological shortcomings in existing scales, we sought to evaluate the PACT as a measure of GDB in a population of PD patients compared to healthy controls.

Methods and Materials

Ethics

The University of Pennsylvania Institutional Review Board approved this protocol. We obtained written informed consent from all participants prior to enrollment.

Sample Recruitment and Selection

PD patients were recruited from the Parkinson’s Disease and Movement Disorders Center at the University of Pennsylvania. Inclusion criteria were a clinical diagnosis of Parkinson’s disease and an informant who could respond to surveys about the patient’s behavior. We excluded individuals with moderate to severe cognitive impairment (Montreal Cognitive Assessment less than 18 or a physician diagnosis of dementia) (Z. Nasreddine, 2015). Because our aim was to measure aspects of GDB, which may not be captured by our current measures of apathy, we did not use the presence of apathy ratings on questionnaires as an inclusion criterion. Control patients were healthy older adults without a self-reported history of psychiatric or neurological ailments.

Measures

We extracted basic clinical and demographic information from the patients’ medical records. PD patients were evaluated for stage of disease (Hoehn and Yahr, HY), concomitant medications, history of Deep Brain Stimulation (DBS) surgery and completed the 15-item version of the Geriatric Depression Scale (GDS)(Yesavage & Sheikh, 1986) to evaluate for depressive symptoms. Dopaminergic medications were converted into a Levodopa Equivalency Dose (LED) using published calculations that were derived from a review of 56 previous reports of how to convert non-levodopa PD medications into a uniform unit in order to compare drug effects across different PD regimens (Tomlinson et al., 2010). All participants completed a cognitive screen (either MoCA or MMSE) to reduce the possibility that severe cognitive impairment had been mistaken for apathy by the patient’s care partner. PD patients completed the Montreal Cognitive Assessment (MoCA)(Z. S. Nasreddine et al., 2005) and control participants completed the Mini-Mental Status Examination (MMSE)(Folstein, Folstein, & McHugh, 1975). To allow for comparisons across groups, we converted MoCA to MMSE scores using a validated conversion factor (van Steenoven et al., 2014). Patients self-completed the Starkstein Apathy Scale (AS) (Starkstein et al., 1992), and informants completed the apathy subscale of the Neuropsychiatric Inventory (NPI)(Cummings et al., 1994). The AS is a 14-item scale that assesses symptoms of apathy in PD. Response options range from 0 (not at all) to 3 (a lot) with a total possible score of 42. A cut-off of 14 is used to identify apathetic individuals. The NPI is a scripted questionnaire in which the informant rates the frequency and severity of a number of apathetic behaviors, and has been previously tested as a measure of apathy in PD (Leentjens et al., 2008). A composite frequency×severity (FxS) score is calculated yielding a total score range of 0–12. Caregivers also rate their own levels of distress. A cut-off of 4 is used to identify apathetic individuals (Dubois et al., 2007).

PACT

Massimo and colleagues developed the PACT to quantify components of GDB that are impaired in patients with apathy (Figure 1). They developed it based on a review of experimental paradigms in the literature and clinical observations of apathy (Jenkins, Jahanshahi, Jueptner, Passingham, & Brooks, 2000; Ruh, Cooper, & Mareschal, 2010). Briefly, we obtained a computerized reaction time to assess initiation, planning, and motivation components of GDB. Participants had a short practice period of several trials for each of the measures described below, and all participants appeared to understand the tasks.

Figure 1.

Figure 1

Open in a new tab

Summary of the basic PACT task

To assess the initiation component, participants began a trial by depressing the “start” key, then a central visual stimulus (triangle) appeared on the computer screen (latency ranging pseudorandomly 500–1,200msec); finally, another fixed central target key must be depressed in response to this stimulus for 30 trials. In our multivariate analysis, we controlled for motor speed using the amount of time it took for the participant to hit the target key, after they had already lifted their finger from the start key. Prior studies using the PACT in frontotemporal dementia did not control for motor slowing due to the lack of bradykinesia as a cardinal feature of the condition (L. Massimo et al., 2015). To obtain an initiation score, we measured the latency for the subject to lift the finger off of the start key in response to the stimulus on the screen.

To assess the motivation component, the participant performed the task described above. In addition, we gave participants an additional amount of money in the form of monetary units at the beginning of the task, and money was taken away as a “penalty” if they did not respond by hitting the target key more rapidly to a stimulus relative to their previous performance, obtained during the task described above. The goal here was to create positive and negative scenarios for goal selection independent of learning effects for the task. Participants received visual feedback (a bank of points appeared on the screen) about their response speed after each trial on the computer screen, compared to their prior reaction time. Participants also performed a “reward” condition where they receive points for responding by hitting the target key more rapidly (reward and penalty conditions were administered in a randomly ordered manner across participants). The motor control used for the reward and penalty conditions was the total latency for the basic PACT task described above (the time it takes for a participant to press the pink key after the stimulus appears when motivators are not present).

Assessing the planning component requires a resource-demanding task that depends on the integration of strategies to meet the challenges of the condition (Sorel & Pennequin, 2008; Toglia & Berg, 2013). Here, participants must correctly press one of two pseudorandomly lateralized keys, contingent on the combination of two features of a central visual-pattern stimulus: if the stimulus is green or has thick stripes, the key on the left is correct; if the stimulus is blue or contains dotted stripes, the key on the right is correct. To assure that planning could be assessed specifically, we minimized the influence of working memory confounds by making the patterns visually available to participants during performance. A planning score was generated by averaging the total latencies on correct trials of the planning task described above. To account for motor slowing, in multivariate analysis we controlled for average motor speed on a non-complex lateralized task. In this task, the participants hit a key on the right if the visual appeared on the right side of the screen and a key on the left side of the keyboard, if the visual stimulus appeared on the left side of the screen.

Data Analysis

We used summary statistics to examine basic clinical and demographic characteristics of PD patients and controls. We used t-tests and Wilcoxon rank sum tests to compare the performance between PD patients and healthy older adults on the three PACT conditions—initiation, planning and motivation. We then created four linear regressions to examine the difference in reaction times for 1) initiation; 2) reward; 3) punishment; and 4) planning while controlling for motor slowing (see descriptions above). Finally, we compared the agreement between the NPI and AS to identify apathetic subjects using Cohen’s kappa. Then we performed subgroup analyses to compare the performance on the three PACT conditions between apathetic PD subjects and healthy controls. We defined apathetic subjects using 3 different case definitions: 1) Apathetic subjects by NPI or AS; 2) Apathetic by AS only; and 3) Apathetic symptoms by NPI).

Results

We recruited a total of 32 PD patients and 15 control participants for the study. Two PD patients were excluded from the analysis for a low MoCA score, leaving 30 PD participants included in the analysis.

Basic Demographics

PD patients were an average age of 66.7 ± 6.6 years old, with a median education of 16 (IQR 15–18) years. Control participants were an average of 68.5 ± 10.8 years old with a median education of 16 (IQR 13.5–18). There were no significant differences in age, education or gender between PD and control participants. However, PD patients had slightly lower MMSE scores than controls (median 29 vs. 30, respectively; z=2.26, p=0.02)(Table 1). In addition, PD patients had mild to moderate disease (median HY 2 IQR(2–2.375)), were on a median of 616 mg LED (IQR 381–1012) and 13.3% (n=4) had DBS surgery. Subjects with DBS had significantly higher median LED (1211 mg (IQR 766–2063)) compared to those without DBS (median LED 540 mg, IQR 350–950); z=−2.01, p=0.04. The median score on the GDS was 3.5 (IQR 2–6) which is below the cutoff for depression. Using the NPI, 20% (n=6) of PD subjects were apathetic and using the AS, 37% (n=11) of PD subjects were apathetic. The agreement between the two scales to identify apathetic subjects was low (Cohen’s K=0.36). There were no significant differences in age, gender, education or cognitive scores between individuals with PD who were depressed and those who were not.

Table 1.

Comparison of sample characteristics between PD and healthy control participants

Characteristic PD (n=30) Healthy Control
(n=15)		 Test statistic p-value
Demographics
Mean Age (SD) 66.7 (6.5) 68.5 (10.8) t(43)=0.67 0.50
% Male (N) 70.0 (21) 40.0 (6) X2(1, N=45)=3.75 0.11
Median Education, years (IQR) 16 (15–18) 16 (13.5–18) z=−0.86 0.40
Neuropsychiatric features
Median MMSE (IQR) 29 (28–30) 30 (29–30) z=2.26 0.02
Median NPI apathy score (IQR) 1 (0–3) --
Median AS score (IQR) 11.5 (8–17) --
Median GDS15 (IQR) 3.5 (2–6) --
PACT subscale performance (in
msec)
Mean initiation (SD) 419.71 (57.17) 366.19 (59.64) t(43)=−2.88 0.006
Mean planning (SD) 1402.16 (355.12) 1021.74 (137.28) t(43)=−3.98 <0.001
Mean reward (SD) 679.25 (110.43) 562.38 (107.86) t(43)=−3.37 0.001
Mean punishment (SD) 677.54 (105.23) 527.38 (115.57) t(43)=−4.34 <0.001
Open in a new tab

MMSE = Mini-Mental Status Examination; NPI = Neuropsychiatric Inventory; AS = Starkstein Apathy Scale; GDS = Geriatric Depression Scale; PACT = Philadelphia Apathy Computerized Test

PACT Results

PD patients were slower on all PACT tasks than controls. After controlling for motor speed, PD patients were significantly slower on the planning and initiation tasks, but neither the reward nor punishment (Table 2). This was true in subgroup analysis when comparing apathetic PD subjects using the three different case definitions for apathy (apathetic by NPI or AS (n=19), AS only (n=11) and apathetic symptoms on NPI (n=17)) to healthy controls. There were no significant correlations between PACT subtasks and NPI and AS when examined as continuous measures, except for a weak, but significant correlation between the reward and punishments subtasks and the NPI (r=0.32; p=0.03 and r=0.35; p=0.02; respectively). After also controlling for MMSE scores and excluding subjects with significant depressive symptoms (GDS > 5), PD subjects still had slower RT on the initiation task compared to controls (β=69.51 [95% CI 16.74, 122.28], F(3,31); p=0.011), while on the planning task differences between groups were mostly explained by differences in cognitive function (for every one point increase in MMSE β= −77.15 [95% CI−152.88, −1.42], F(3,31), p=0.05). In addition, there were no significant correlations between GDS scores and latencies on each PACT task, while there were moderately strong correlations between GDS scores and both NPI scores and AS scores (r = 0.58; p<0.01 and r=0.65; p<0.01; respectively).

Table 2.

Group comparisons of Philadelphia Apathy Computerized Task (PACT) subscale performance controlling for motor slowing

PACT
Subscale		 Subgroup [ref. healthy
controls]		 B coefficient [95% CI] Model
statistics		 p-value R2
Initiation All PD cases 67.07 [22.71,111.43] F(2,42) 0.004 0.19
Apathetic by NPI or AS 63.10 [14.42,111.77] F(2,31) 0.01 0.20
Apathetic by AS 63.72[3.31, 124.14] F(2,23) 0.04 0.20
NPI apathetic symptoms 69.73 [18.30, 121.16] F(2,29) 0.01 0.24
Reward All PD cases −4.35 [−76.44, 67.73] F(2,42) 0.90 0.52
Apathetic by NPI or AS 3.96 [−72.56, 80.48] F(2,31) 0.92 0.54
Apathetic by AS 9.98 [−83.56, 103.51] F(2,23) 0.83 0.52
NPI apathetic symptoms 12.56 [−73.47, 98.60] F(2,29) 0.77 0.54
Punishment All PD cases 42.45 [−34.27, 119.17] F(2,42) 0.27 0.53
Apathetic by NPI or AS 54.41 [−30.74, 139.56] F(2,31) 0.20 0.55
Apathetic by AS 73.33 [−30.48, 176.93] F(2,23) 0.16 0.54
NPI apathetic symptoms 46.65 [−49.62, 142.92] F(2,29) 0.33 0.55
Planning All PD cases 214.78 [28.73, 400.82] F(2,42) 0.03 0.47
Apathetic by NPI or AS 220.07 [28.39, 411.74] F(2,31) 0.03 0.50
Apathetic by AS 322.40 [101.46,543.34] F(2,23) 0.006 0.59
NPI apathetic symptoms 222.19 [17.98, 426.40] F(2,29) 0.03 0.51
Open in a new tab

NPI = Neuropsychiatric Inventory; AS = Starkstein Apathy Scale

In an exploratory analysis, we examined the motivation results on the PACT in more detail. As stated previously, there were no differences in the reward and punishment tasks between control and PD patients once we controlled for motor spend. However, while PD patients performed the reward and punishment task at approximately the same speed, control participants performed much faster on the punishment task than on the reward. Control participants performed an average of 35 ±66.4 ms faster on the punishment task than the reward task, while PD patient performed an average of 0.65 ± 46.5 ms slower on the punishment task than reward. However this difference was not significant (p=0.07)

Discussion

This study examined impairments in GDB in a population where apathy is highly prevalent. Using a novel computerized reaction time test we characterized specific GDB deficits in PD patients, namely impaired initiation and planning. To our knowledge, this is one of the first studies that objectively attempts to characterize GDB deficits among PD patients. Tools such as the PACT will help disentangle the different components of apathy, and ultimately, help develop targeted interventions to improve apathetic behavior.

We found that PD patients were most impaired on the planning task of the PACT. The selection and set-shifting required in the PACT task represents a component of the more complex organizational ability required for planning. Our results are not surprising given that a dysexecutive profile is a common finding in PD (Emre, 2003). Apathy related to ‘cognitive inertia’ can result from impairments in executive functions such as planning, working memory, and task-switching (Levy & Dubois, 2006). In Parkinson’s disease (PD), performance on standardized tests of planning was associated with apathy (Weintraub et al., 2005). Furthermore, greater executive dysfunction is associated with a greater degree of apathy (Pluck & Brown, 2002; Starkstein et al., 1992). These planning deficits can then lead to commonly observed symptoms such as difficulty organizing daily activities, managing money and maintaining attention.

Similarly, PD patients may have difficulty with initiation. In previous study, apathy in PD was specifically associated with initiation deficits (Meyer et al., 2014). While this study used a neuropsychiatric battery to define problems with initiation, there is evidence that PD patients have difficulty initiating behavior in other domains. As an example, freezing of gait, which occurs when the patient is unable to initiate walking, is a common symptom in Parkinson’s disease (Giladi et al., 1992; Giladi et al., 2001). Importantly, once the patient has initiated movement they can walk normally, which implies that this is an initiation deficit rather than an overall motor deficit. The observed trend for impaired initiation as measured by the PACT may be an analogous behavioral impairment in PD patients. One limitation to our initiation data is our use of external cues in the PACT task. There is some evidence that external cues improve initiation in apathetic individuals. A case study examining an individual who developed apathy after an aneurysm found that initiation of activities and adherence to a daily schedule improved significantly with regular reminders given by a paging device (Evans, Emslie, & Wilson, 1998). Another study found that external cues helped an individual with apathy after a traumatic brain injury initiate speech (Sohlberg, Sprunk, & Metzelaar, 1988). In PD, external cueing systems also improve initiation deficits, such as freezing of gait (Burleigh-Jacobs, Horak, Nutt, & Obeso, 1997; Delval et al., 2014; Dibble et al., 2004). In our task, participants did have an external cue to initiate movement—the appearance of the triangle. This may have diminished any inherent initiation deficits that were present. Nevertheless, we still found a significant initiation deficit in our PD patients.

Our results for the motivation analysis, however, were unexpected. After controlling for motor speed, PD patients did not perform differently than healthy controls for both the reward and penalty conditions. This is contrary to other research, which has shown that PD patients have a blunted response to monetary rewards compared to non-apathetic PD patients and healthy controls (Schmidt et al., 2008; van der Vegt et al., 2013). As an example, Schmidt et al. asked PD patients, individuals with basal ganglia lesions and healthy controls to modify grip strength based on direct instruction or monetary incentives (Schmidt et al., 2008). When researchers asked participants to modify grip strength based upon direct instruction, PD patients performed like their healthy counterparts. However when grip strength was self-modified by monetary incentives, PD patients showed a lower response than healthy controls. Other studies also demonstrate that PD patients have a reduced response to monetary reward (van der Vegt et al., 2013).

While we did not find an overall deficit with motivation, particularly in reward scenarios, we did find evidence for a dysfunctional response to monetary incentives. Healthy controls had faster overall reaction times for slow performance in the punishments scenario than for speedy performance in the reward scenario. This pattern is in line with the existing psychological literature on loss aversion. The concept of loss aversion states that people act more strongly to avoid losses than to obtain gains (Tversky & Kahneman, 1983). It is a well-documented phenomenon in studies of human behavior. Its effects have been seen in diverse areas ranging from student test scores (Fryer Jr, Levitt, List, & Sadoff, 2012) to the housing market (Genesove & Mayer, 2001). In our study, PD patients showed reaction times of approximately equal magnitude for both the punishment and reward conditions of the PACT. This difference from healthy control behavior could indicate some degree of impaired loss aversion and incentive processing. There is some research in PD that is in line with this hypothesis. Using the Starkstein apathy scale to define apathy, Martinez-Horta et al. (2014) found differences in the event related potentials (ERP) of apathetic PD patients, non-apathetic PD patients and healthy controls. Specifically, apathetic PD patients showed a reduced ERP in response to losses compared to non-apathetic PD patients and healthy controls—implying a diminished physiological response to losses. However, other research found that PD patients had an increased reaction time in response to a painful stimulus, but not a monetary reward (Shiner et al., 2012). It is important to note that different neural pathways likely process painful stimuli and monetary punishment. Consequently, further study will help fully characterize PD patients’ differential response to rewards and punishments. In addition, there are numerous facets to motivation such as learning effects and sensitivity to negative feedback that can be explored in future studies.

In contrast to our findings in PD, a prior studying measuring GDB in behavioral variant frontotemporal dementia using the PACT found impairments across all three measures of GDB (L. Massimo et al., 2015). Apathy in frontotemporal dementia is the most common neuropsychiatric feature and found in 84% of autopsy-confirmed cases (Rascovsky et al., 2011). One might suspect that the wide-spread frontal degeneration in frontotemporal dementia increases the risk of multi-faceted impairments in goal-directed behavior. However, PD, a primary striato-nigral disorder, can have different behavioral phenotypes depending on predominant ventral vs. dorsal striatal involvement (Tremblay, Worbe, Thobois, Sgambato-Faure, & Feger, 2015). Similarly, schizophrenia is a disorder where a core feature is negative symptoms often characterized by the lack of ability to begin and sustain planned activities. Prior studies have used computer based tasks to measure motivation based on financial incentives (Fervaha et al., 2015; Hartmann et al., 2015; Simon et al., 2010; Wolf et al., 2014) using similar methodology that assesses performance in different monetary reward scenarios. One study found that schizophrenic subjects with apathy had less activation of the ventral striatum during anticipation of a reward in a modified monetary incentive delay task (Simon et al., 2010). However, these tasks do not capture all components of GDB.

Unfortunately, the validated apathy measures we utilized also do not measure subtypes of GDB. We only observed a weak correlation between latencies on the motivation tasks of the PACT and the NPI. An attempt to determine domains (affective, behavioral or cognitive) within the NPI using expert consensus found that there were only 4 items in which at least 6 of the 9 experts agreed upon the categorization (Chow et al., 2009). Upon review of the specific items, many appear to capture motivational aspects of apathy (e.g. “Does the patient seem less interested in the activities and plans of others?” “Has the patient lost interest in family and friends?” “Is the patient less enthusiastic about his/her usual interests?”). Similarly, an analysis of the psychometric properties of the AS found a 2-factor structure, which they termed cognitive-behavioral and general apathy (Pedersen et al., 2012). However, the items within the cognitive-behavioral domain contain items that mostly assess motivation (e.g. “Do you have motivation?” “Does anything interest you?”), while the general apathy domain is non-specific. The Lille Apathy Rating Scale (LARS) is a validated, multi-domain apathy scale that measures intellectual curiosity, self-awareness, emotion and action initiation (P Sockeel et al., 2006). Unfortunately, we did not include this scale in our battery.

As our study has some limitations, we must be cautious in interpreting these findings. First, we did not have an ideal measure against which to validate the PACT. We sought to look at impairments in goal directed behavior that contribute to apathy. Unfortunately, current apathy measures capture global apathy, but not the individual components of goal directed behaviors. Before we can confirm the PACT as a measure of GDB in PD, we would need to perform further validation studies. One possibility would be to use a functional measure, in which we could actually measure patients’ behaviors. As an example, the Direct Assessment of Functional Status would allow us to assess the degree to which the behaviors measured by the PACT manifested in actual daily tasks. Furthermore, traditional instruments to ascertain apathy commonly use proxy report. Unfortunately, this approach is subject to caregiver confounds such as burden and strain, that may affect the evaluation. One goal of the present study thus was to quantify components of GDB in an objective manner that is minimally confounded by proxy report.

An additional challenge that we faced was controlling for baseline motor speed. Because PD affects motor speed, we could not look at isolated reaction times on PACT components. This creates two main challenges. First, the best method of controlling for motor speed is not established. We used reaction times on other PACT components; however, this may have been an overcorrection. Second, controlling for motor speed makes it difficult to establish cutoff reaction times that would define categories of normal or abnormal GDB. In order to create a strict cutoff, we would need to determine population norms based upon patients’ baseline reaction times. This makes the task more complex, and harder to use in routine clinical practice.

Despite these limitations, this task provides a new framework for thinking about apathy measurement. With additional validation studies, including evaluation of the PACT’s ability to predict long-term outcomes, the PACT could provide a less biased, and more thorough method to analyze impairments in goal directed behavior and apathy in PD. Ultimately, this would lead to the creation of more effective treatments that target specific GDB deficits.

Acknowledgments

This work was supported by the National Parkinson Foundation and Parkinson Council. Dr. Dahodwala received funding from the NIA K23 AG034236.

References

  1. Burleigh-Jacobs Anne, Horak Fay B, Nutt John G, Obeso Jose A. Step initiation in Parkinson's disease: influence of levodopa and external sensory triggers. Movement Disorders. 1997;12(2):206–215. doi: 10.1002/mds.870120211. [DOI] [PubMed] [Google Scholar]
  2. Chow TW, Binns MA, Cummings JL, Lam I, Black SE, Miller BL, van Reekum R. Apathy symptom profile and behavioral associations in frontotemporal dementia vs dementia of Alzheimer type. Archives of Neurology. 2009;66(7):888–893. doi: 10.1001/archneurol.2009.92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cummings JL, Mega M, Gray K, Rosenberg-Thompson S, Carusi DA, Gornbein J. The Neuropsychiatric Inventory: comprehensive assessment of psychopathology in dementia. Neurology. 1994;44(12):2308–2314. doi: 10.1212/wnl.44.12.2308. [DOI] [PubMed] [Google Scholar]
  4. Delval A, Moreau C, Bleuse S, Tard C, Ryckewaert G, Devos D, Defebvre L. Auditory cueing of gait initiation in Parkinson's disease patients with freezing of gait. Clinical Neurophysiology. 2014;125(8):1675–1681. doi: 10.1016/j.clinph.2013.12.101. [DOI] [PubMed] [Google Scholar]
  5. Dibble Leland E, Nicholson Diane E, Shultz Barry, MacWilliams Bruce A, Marcus Robin L, Moncur Carolee. Sensory cueing effects on maximal speed gait initiation in persons with Parkinson's disease and healthy elders. Gait & Posture. 2004;19(3):215–225. doi: 10.1016/S0966-6362(03)00065-1. [DOI] [PubMed] [Google Scholar]
  6. Dubois B, Burn D, Goetz C, Aarsland D, Brown RG, Broe GA, Emre M. Diagnostic procedures for Parkinson's disease dementia: recommendations from the movement disorder society task force. Movement Disorders. 2007;22(16):2314–2324. doi: 10.1002/mds.21844. [DOI] [PubMed] [Google Scholar]
  7. Dujardin K, Sockeel P, Devos D, Delliaux M, Krystkowiak P, Destee A, Defebvre L. Characteristics of apathy in Parkinson's disease. Movement Disorders. 2007;22(6):778–784. doi: 10.1002/mds.21316. [DOI] [PubMed] [Google Scholar]
  8. Emre M. Dementia associated with Parkinson's disease. Lancet Neurologyl. 2003;2(4):229–237. doi: 10.1016/s1474-4422(03)00351-x. [DOI] [PubMed] [Google Scholar]
  9. Evans JJ, Emslie H, Wilson BA. External cueing systems in the rehabilitation of executive impairments of action. Journal of the International Neuropsychology Society. 1998;4(4):399–408. [PubMed] [Google Scholar]
  10. Fervaha Gagan, Duncan Mark, Foussias George, Agid Ofer, Faulkner Guy E, Remington Gary. Effort-based decision making as an objective paradigm for the assessment of motivational deficits in schizophrenia. Schizophrenia Research. 2015;168(1):483–490. doi: 10.1016/j.schres.2015.07.023. [DOI] [PubMed] [Google Scholar]
  11. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12(3):189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  12. Fryer Roland G, Jr, Levitt Steven D, List John, Sadoff Sally. Enhancing the efficacy of teacher incentives through loss aversion: a field experiment: National Bureau of Economic Research. 2012 [Google Scholar]
  13. Genesove David, Mayer Christopher. Loss aversion and seller behavior: Evidence from the housing market: National bureau of economic research. 2001 [Google Scholar]
  14. Giladi N, McMahon D, Przedborski S, Flaster E, Guillory S, Kostic V, Fahn S. Motor blocks in Parkinson's disease. Neurology. 1992;42(2):333–339. doi: 10.1212/wnl.42.2.333. [DOI] [PubMed] [Google Scholar]
  15. Giladi N, Treves TA, Simon ES, Shabtai H, Orlov Y, Kandinov B, Korczyn AD. Freezing of gait in patients with advanced Parkinson's disease. Journal of Neural Transmission. 2001;108(1):53–61. doi: 10.1007/s007020170096. [DOI] [PubMed] [Google Scholar]
  16. Hartmann Matthias N, Hager Oliver M, Reimann Anna V, Chumbley Justin R, Kirschner Matthias, Seifritz Erich, Kaiser Stefan. Apathy but not diminished expression in schizophrenia is associated with discounting of monetary rewards by physical effort. Schizophrenia Bulletin. 2015;41(2):503–512. doi: 10.1093/schbul/sbu102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Isella Valeria, Melzi Paola, Grimaldi Marco, Iurlaro Simona, Piolti Roberto, Ferrarese Carlo, Appollonio Ildebrando. Clinical, neuropsychological, and morphometric correlates of apathy in Parkinson's disease. Movement Disorders. 2002;17(2):366–371. doi: 10.1002/mds.10041. [DOI] [PubMed] [Google Scholar]
  18. Jenkins IH, Jahanshahi M, Jueptner M, Passingham RE, Brooks DJ. Self-initiated versus externally triggered movements. II. The effect of movement predictability on regional cerebral blood flow. Brain. 2000;123(Pt 6):1216–1228. doi: 10.1093/brain/123.6.1216. [DOI] [PubMed] [Google Scholar]
  19. Kay DB, Kirsch-Darrow L, Zahodne LB, Okun MS, Bowers D. Dimensions of apathy in Parkinson's disease: exploratory factor analysis of the apathy scale. Journal of Parkinsons Disease. 2012;2(2):161–166. doi: 10.3233/JPD-2012-12085. [DOI] [PubMed] [Google Scholar]
  20. Laplane D, Dubois B. Auto-Activation deficit: A basal ganglia related syndrome. Movement Disorders. 2001;16(5):810–814. doi: 10.1002/mds.1185. [DOI] [PubMed] [Google Scholar]
  21. Leentjens Albert FG, Dujardin Kathy, Marsh Laura, Martinez-Martin Pablo, Richard Irene H, Starkstein Sergio E, Rascol Oliver. Apathy and anhedonia rating scales in Parkinson's disease: critique and recommendations. Movement Disorders. 2008;23(14):2004–2014. doi: 10.1002/mds.22229. [DOI] [PubMed] [Google Scholar]
  22. Leritz E, Loftis C, Crucian G, Friedman W, Bowers D. Self-awareness of deficits in Parkinson disease. The Clinical Neuropsychologist. 2004;18(3):352–361. doi: 10.1080/1385404049052412. [DOI] [PubMed] [Google Scholar]
  23. Leroi I, Harbishettar V, Andrews M, McDonald K, Byrne EJ, Burns A. Carer burden in apathy and impulse control disorders in Parkinson's disease. International Journal of Geriatric Psychiatry. 2012;27(2):160–166. doi: 10.1002/gps.2704. [DOI] [PubMed] [Google Scholar]
  24. Levy R, Dubois B. Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits. Cerebral Cortex. 2006;16(7):916–928. doi: 10.1093/cercor/bhj043. [DOI] [PubMed] [Google Scholar]
  25. Marin RS. Apathy: a neuropsychiatric syndrome. Journal of Neuropsychiatry & Clinical Neurosciences. 1991;3(3):243–254. doi: 10.1176/jnp.3.3.243. [DOI] [PubMed] [Google Scholar]
  26. Marin RS. Apathy: Concept, Syndrome, Neural Mechanisms, and Treatment. Seminars in Clinical Neuropsychiatry. 1996;1(4):304–314. doi: 10.1053/SCNP00100304. [DOI] [PubMed] [Google Scholar]
  27. Martinez-Horta S, Riba J, de Bobadilla RF, Pagonabarraga J, Pascual-Sedano B, Antonijoan RM, Kulisevsky J. Apathy in Parkinson's disease: neurophysiological evidence of impaired incentive processing. Journal of Neuroscience. 2014;34(17):5918–5926. doi: 10.1523/JNEUROSCI.0251-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Massimo L, Evans LK, Grossman M. Differentiating Subtypes of Apathy to Improve Person-Centered Care in Frontotemporal Degeneration. Journal of Gerontological Nursing. 2014:1–8. doi: 10.3928/00989134-20140827-01. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Massimo L, Powers JP, Evans LK, McMillan CT, Rascovsky K, Eslinger P, Grossman M. Apathy in Frontotemporal Degeneration: Neuroanatomical Evidence of Impaired Goal-directed Behavior. Frontiers in Human Neuroscience. 2015;9:611. doi: 10.3389/fnhum.2015.00611. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McKinlay A, Grace RC, Dalrymple-Alford JC, Anderson TJ, Fink J, Roger D. Neuropsychiatric problems in Parkinson's disease: comparisons between self and caregiver report. Aging & Mental Health. 2008;12(5):647–653. doi: 10.1080/13607860802343225. [DOI] [PubMed] [Google Scholar]
  31. Meyer A, Zimmermann R, Gschwandtner U, Hatz F, Bousleiman H, Schwarz N, Fuhr P. Apathy in Parkinson's disease is related to executive function, gender and age but not to depression. Frontiers in Aging Neuroscience. 2014;6:350. doi: 10.3389/fnagi.2014.00350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Nasreddine Z. MoCA Montreal Cognitive Assessment. 2015 Retrieved Dec 20, 2015, from http://www.mocatest.org/faq/ [Google Scholar]
  33. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Chertkow H. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatric Society. 2005;53(4):695–699. doi: 10.1111/j.1532-5415.2005.53221.x. doi: JGS53221 [pii] 10.1111/j.1532-5415.2005.53221.x [doi] [DOI] [PubMed] [Google Scholar]
  34. Njomboro Progress, Deb Shoumitro. Distinct neuropsychological correlates of cognitive, behavioral, and affective apathy sub-domains in acquired brain injury. Frontiers in Neurology. 2014;5 doi: 10.3389/fneur.2014.00073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pedersen KF, Alves G, Aarsland D, Larsen JP. Occurrence and risk factors for apathy in Parkinson disease: a 4-year prospective longitudinal study. Journal of Neurology, Neurosurgery & Psychiatry. 2009;80(11):1279–1282. doi: 10.1136/jnnp.2008.170043. [DOI] [PubMed] [Google Scholar]
  36. Pedersen KF, Alves G, Larsen JP, Tysnes OB, Moller SG, Bronnick K. Psychometric properties of the Starkstein Apathy Scale in patients with early untreated Parkinson disease. American Journal of Geriatric Psychiatry. 2012;20(2):142–148. doi: 10.1097/JGP.0b013e31823038f2. [DOI] [PubMed] [Google Scholar]
  37. Pluck GC, Brown RG. Apathy in Parkinson's disease. Journal of Neurology, Neurosurgery & Psychiatry. 2002;73(6):636–642. doi: 10.1136/jnnp.73.6.636. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Poletti M, De Rosa A, Bonuccelli U. Affective symptoms and cognitive functions in Parkinson's disease. Journal of the Neurological Sciences. 2012;317(1–2):97–102. doi: 10.1016/j.jns.2012.02.022. [DOI] [PubMed] [Google Scholar]
  39. Rascovsky K, Hodges JR, Knopman D, Mendez MF, Kramer JH, Neuhaus J, Miller BL. Sensitivity of revised diagnostic criteria for the behavioural variant of frontotemporal dementia. Brain. 2011;134(Pt 9):2456–2477. doi: 10.1093/brain/awr179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Reijnders JS, Scholtissen B, Weber WE, Aalten P, Verhey FR, Leentjens AF. Neuroanatomical correlates of apathy in Parkinson's disease: A magnetic resonance imaging study using voxel-based morphometry. Movement Disorders. 2010;25(14):2318–2325. doi: 10.1002/mds.23268. [DOI] [PubMed] [Google Scholar]
  41. Robert P, Onyike CU, Leentjens AF, Dujardin K, Aalten P, Starkstein S, Byrne J. Proposed diagnostic criteria for apathy in Alzheimer's disease and other neuropsychiatric disorders. European Psychiatry. 2009;24(2):98–104. doi: 10.1016/j.eurpsy.2008.09.001. [DOI] [PubMed] [Google Scholar]
  42. Ruh N, Cooper RP, Mareschal D. Action selection in complex routinized sequential behaviors. Journal of Experimental Psychology. Human Perception and Performance. 2010;36(4):955–975. doi: 10.1037/a0017608. [DOI] [PubMed] [Google Scholar]
  43. Schmidt L, d'Arc BF, Lafargue G, Galanaud D, Czernecki V, Grabli D, Pessiglione M. Disconnecting force from money: effects of basal ganglia damage on incentive motivation. Brain. 2008;131(Pt 5):1303–1310. doi: 10.1093/brain/awn045. [DOI] [PubMed] [Google Scholar]
  44. Shiner T, Seymour B, Symmonds M, Dayan P, Bhatia KP, Dolan RJ. The Effect of Motivation on Movement: A Study of Bradykinesia in Parkinson’s Disease. PLoS One. 2012;7(10) doi: 10.1371/journal.pone.0047138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Simon Joe J, Biller Armin, Walther Stephan, Roesch-Ely Daniela, Stippich Christoph, Weisbrod Matthias, Kaiser Stefan. Neural correlates of reward processing in schizophrenia—relationship to apathy and depression. Schizophrenia Research. 2010;118(1):154–161. doi: 10.1016/j.schres.2009.11.007. [DOI] [PubMed] [Google Scholar]
  46. Sockeel P, Dujardin K, Devos D, Deneve C, Destée A, Defebvre L. The Lille apathy rating scale (LARS), a new instrument for detecting and quantifying apathy: validation in Parkinson’s disease. Journal of Neurology, Neurosurgery & Psychiatry. 2006;77(5):579–584. doi: 10.1136/jnnp.2005.075929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Sohlberg McKay M, Sprunk Holly, Metzelaar Kristin. Efficacy of an external cuing system in an individual with severe frontal lobe damage. Cognitive Rehabilitation. 1988 [Google Scholar]
  48. Sorel O, Pennequin V. Aging of the planning process: the role of executive functioning. Brain and Cognition. 2008;66(2):196–201. doi: 10.1016/j.bandc.2007.07.006. [DOI] [PubMed] [Google Scholar]
  49. Starkstein SE, Mayberg HS, Preziosi TJ, Andrezejewski P, Leiguarda R, Robinson RG. Reliability, validity, and clinical correlates of apathy in Parkinson's disease. The Journal of Neuropsychiatry and Clinical Neurosciences. 1992;4(2):134–139. doi: 10.1176/jnp.4.2.134. [DOI] [PubMed] [Google Scholar]
  50. Starkstein SE, Merello M. The Unified Parkinson's Disease Rating Scale: validation study of the mentation, behavior, and mood section. Movement Disorders. 2007;22(15):2156–2161. doi: 10.1002/mds.21521. [DOI] [PubMed] [Google Scholar]
  51. Starkstein SE, Petracca G, Chemerinski E, Kremer J. Syndromic validity of apathy in Alzheimer's disease. American Journal of Psychiatry. 2001;158(6):872–877. doi: 10.1176/appi.ajp.158.6.872. [DOI] [PubMed] [Google Scholar]
  52. Starkstein, Sergio E, Ingram L, Garau ML, Mizrahi R. On the overlap between apathy and depression in dementia. Journal of Neurology, Neurosurgery & Psychiatry. 2005;76(8):1070–1074. doi: 10.1136/jnnp.2004.052795. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Toglia J, Berg C. Performance-based measure of executive function: comparison of community and at-risk youth. American Journal of Occupational Therapy. 2013;67(5):515–523. doi: 10.5014/ajot.2013.008482. [DOI] [PubMed] [Google Scholar]
  54. Tomlinson CL, Stowe R, Patel S, Rick C, Gray R, Clarke CE. Systematic review of levodopa dose equivalency reporting in Parkinson's disease. Movement Disorders. 2010;25(15):2649–2653. doi: 10.1002/mds.23429. [DOI] [PubMed] [Google Scholar]
  55. Tremblay L, Worbe Y, Thobois S, Sgambato-Faure V, Feger J. Selective dysfunction of basal ganglia subterritories: From movement to behavioral disorders. Movement Disorders. 2015;30(9):1155–1170. doi: 10.1002/mds.26199. [DOI] [PubMed] [Google Scholar]
  56. Tversky Amos, Kahneman Daniel. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review. 1983;90(4):293. [Google Scholar]
  57. van der Vegt JP, Hulme OJ, Zittel S, Madsen KH, Weiss MM, Buhmann C, Siebner HR. Attenuated neural response to gamble outcomes in drug-naive patients with Parkinson's disease. Brain. 2013;136(Pt 4):1192–1203. doi: 10.1093/brain/awt027. [DOI] [PubMed] [Google Scholar]
  58. van Steenoven I, Aarsland D, Hurtig H, Chen-Plotkin A, Duda JE, Rick J, Weintraub D. Conversion between mini-mental state examination, montreal cognitive assessment, and dementia rating scale-2 scores in Parkinson's disease. Movement Disorders. 2014;29(14):1809–1815. doi: 10.1002/mds.26062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Vilalta-Franch Joan, Calvó-Perxas Laia, Garre-Olmo Josep, Turró-Garriga Oriol, López-Pousa Secundino. Apathy syndrome in Alzheimer's disease epidemiology: prevalence, incidence, persistence, and risk and mortality factors. Journal of Alzheimer's Disease. 2013;33(2):535–543. doi: 10.3233/JAD-2012-120913. [DOI] [PubMed] [Google Scholar]
  60. Weintraub D, Moberg PJ, Culbertson WC, Duda JE, Katz IR, Stern MB. Dimensions of executive function in Parkinson's disease. Dementia and Geriatric Cognitive Disorders. 2005;20(2–3):140–144. doi: 10.1159/000087043. [DOI] [PubMed] [Google Scholar]
  61. Wolf Daniel H, Satterthwaite Theodore D, Kantrowitz Jacob J, Katchmar Natalie, Vandekar Lillie, Elliott Mark A, Ruparel Kosha. Amotivation in schizophrenia: integrated assessment with behavioral, clinical, and imaging measures. Schizophrenia Bulletin. 2014;40(6):1328–1337. doi: 10.1093/schbul/sbu026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Yesavage Jerome A, Sheikh Javaid I. In: Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version. Brink TL, editor. Vol. 5. New York, NY: Haworth Press; 1986. [Google Scholar]

RESOURCES