Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 1.
Published in final edited form as: Cortex. 2017 Mar 20;90:115–124. doi: 10.1016/j.cortex.2017.02.021

Cognitive states influence dopamine-driven aberrant learning in Parkinson’s disease

James F Cavanagh a, Andrea A Mueller a, Darin R Brown a, Jacqueline R Janowich a, Jacqueline H Story-Remer a, Ashley Wegele b, Sarah Pirio Richardson b
PMCID: PMC5470538  NIHMSID: NIHMS862813  PMID: 28384481

Abstract

Individual differences in dopaminergic tone underlie tendencies to learn from reward versus punishment. These effects are well documented in Parkinson’s patients, who vacillate between low and high tonic dopaminergic states as a function of medication. Yet very few studies have investigated the influence of higher-level cognitive states known to affect downstream dopaminergic learning in Parkinson’s patients. A dopamine-dependent cognitive influence over learning would provide a candidate mechanism for declining cognitive integrity and motivation in Parkinson’s patients. In this report we tested the influence of two high-level cognitive states (cost of conflict and value of volition) that have recently been shown to cause predictable learning biases in healthy young adults as a function of dopamine receptor subtype and dopaminergic challenge. It was hypothesized that Parkinson’s patients OFF medication would have an enhanced cost of conflict and a decreased value of volition, and that these effects would be remediated or reversed ON medication. Participants included N=28 Parkinson’s disease patients who were each tested ON and OFF dopaminergic medication and 28 age- and sex-matched controls. The expected cost of conflict effect was observed in Parkinson’s patients OFF versus ON medication, but only in those that were more recently diagnosed (<5 years). We found an unexpected effect in the value of volition task: medication compromised the ability to learn from difficult a-volitional (instructed) choices. This novel finding was also enhanced in recently diagnosed patients. The difference in learning biases ON vs. OFF medication between these two tasks was strongly correlated, bolstering the idea that they tapped into a common underlying imbalance in dopaminergic tone that is particularly variable in earlier stage Parkinsonism. The finding that these decision biases are specific to earlier but not later stage disease may offer a chance for future studies to quantify phenotypic expressions of idiosyncratic disease progression.

Keywords: Parkinson’s Disease, Reinforcement Learning, Dopamine, Conflict, Volition

1 Introduction

Dopaminergic neurodegeneration in Parkinson’s disease not only manifests in difficulties with motor execution, but also in related problems with volitional action selection, high-level cognition, motivation and mood. Each of these related symptoms can cause understated but significant distress. In this report we describe how some of these related symptoms could be exacerbated by subtle but pervasive learning biases caused by low dopaminergic tone. This aberrant learning may have implications in the day-to-day quality of life of patients, and as it may differ depending on disease duration it may open novel avenues to quantify disease progression.

1.1 Aberrant learning in Parkinson’s Disease

Dopaminergic tone influences the sensitivity of the competing cortico-striatal action selection pathways. Phasic dopamine bursts in the cortico–striatal D1 receptor-mediated direct pathway underlie the ability to learn from and seek reward, whereas dopamine dips in the D2 receptor-mediated indirect pathway underlie the ability to learn from and avoid punishment (Kravitz et al., 2010; Kravitz, Tye, & Kreitzer, 2012; Porter-stransky, Seiler, Day, & Aragona, 2013; Tai, Lee, Benavidez, Bonci, & Wilbrecht, 2012). Reduced tonic dopamine in Parkinsonism causes a lower dynamic range of phasic signaling and a loss of synaptic plasticity in the direct pathway (Frank, 2005; Frank, Seeberger, & O’Reilly R, 2004), as well as opposite effects of more effective phasic signaling and enhanced long term potentiation in the indirect pathway (Beeler et al., 2012; Wiecki & Frank, 2010). The outcome of these systemic alterations include impaired reward-related learning and motivation for action selection (diminished D1 effects) (Voon et al., 2010) but also paradoxically boosted learning of active inhibition (enhanced D2 effects) (Kravitz et al., 2012). This phenotype can be reversed with levodopa (L-dopa) treatment, causing hypersensitivity to rewards and inability to learn from punishments (Cools, 2006; Frank, Samanta, Moustafa, & Sherman, 2007; Frank et al., 2004). Given that L-dopa remains the prevailing treatment for PD, there are clear clinical benefits for understanding how striatal D1/D2 balance affects day-to-day life in Parkinson’s patients, in particular understanding the emergence of gambling and impulse control disorders in some patients (Molina et al., 2000) due to L-dopa “overdosing” of the ventral striatum (Cools, 2006; Ryterska, Jahanshahi, & Osman, 2013).

While these overt biases to rewarded and punished outcomes are well defined, a tonic striatal D1/D2 imbalance can lead to other subtle behavioral and decision-making biases (Foerde et al., 2016; Sharp et al. 2015; Chong et al., 2015), suggesting valenced learning biases simply scratch the surface of aberrant learning in Parkinson’s disease (Wiecki & Frank, 2010). In this report we examine two extrinsic factors that have been shown to influence cortico-striatal plasticity during learning: conflict costs and agency preferences. Similar to an effort cost, conflict induces a cognitive demand during learning. Agentic decisions similarly require higher-order decision making, yet this has a different influence on learning. Conflict costs and agency preferences reflect ecologically relevant situations that have recently been shown to contribute to learning biases as a function of striatal D1/D2 balance in healthy young adults (Cavanagh, Masters, Bath, & Frank, 2014; Cockburn, Collins, & Frank, 2014). We postulate that an understanding of the influence of dopaminergic tone on learning under the influence of these higher-level cognitive phenomena could lead to a better mechanistic understanding of cognitive, motivational, and mood symptoms in Parkinson’s patients.

1.2 Cost of Conflict

It was recently shown that cognitive conflict can act as an implicit cost during learning, similar to an effort cost (Cavanagh et al., 2014). Both effort and conflict costs are mediated by dopaminergic tone and are amplified by a striatal D2>D1 imbalance (Cavanagh et al., 2014; Denk et al., 2005; Drew et al., 2007; Salamone, Correa, Farrar, Nunes, & Pardo, 2009; Simpson et al., 2011; Treadway et al., 2012). Cognitive conflict occurs when competing response options compete for control of behavior, and it acts as a trigger of the need for cognitive control (Botvinick, Braver, Barch, Carter, & Cohen, 2001; Cavanagh & Frank, 2014; Shenhav, Botvinick, & Cohen, 2013). Since control is effortful, it acts as a cost when integrating action values in cortico-striatal circuits, and individuals with genetic striatal D2>D1 sensitivities integrate these conflict-related action sequences with greater associated cost (Cavanagh et al., 2014). To note, this effect can also be induced in healthy young adults by administration of cabergoline, a selective D2 agonist which in low doses preferentially acts on striatal D2-autoreceptors to boost indirect pathway function (Cavanagh et al., 2014). This collective set of findings leads to the clear a priori hypothesis that Parkinson’s patients off medication should have an enhanced cost of conflict effect due to striatal D2>D1 imbalance caused by low dopaminergic tone. We think this phenomenon might be ecologically and clinically relevant, since an enhanced cost of conflict may act to increase the intrinsic cost of goal-directed actions (Kool, McGuire, Wang, & Botvinick, 2013; Shenhav et al., 2013). This would suggest a negative influence on the ability to instantiate higher cognition, leading to poor executive function and low motivation (apathy) in Parkinson’s patients.

1.3 Value of Volition

A different study recently explained why volitional choice can be associated with greater value than a-volitional or instructed choice (Cockburn et al., 2014). People prefer options based on previous volitional, agentic choices (Bown, Read, & Summers, 2003; Sharot, Velasquez, & Dolan, 2010), particularly when they may be rewarding (Leotti & Delgado, 2014). Imaging studies have indicated an important role of basal ganglia in this value of volition bias (Leotti & Delgado, 2014; Murty, DuBrow, & Davachi, 2015; Sharot, Shiner, Brown, Fan, & Dolan, 2009), and individuals with a genetic striatal D1>D2 sensitivity show an enhanced value of volition for rewarded options (Cockburn et al., 2014). This latter study advanced the novel hypothesis that this value of volition is the result of a credit-assignment process in the basal ganglia, where an obligatory boost of dopamine to a volitional action increases plasticity in the striatal D1 pathway responsible for rewarded actions. Thus, the “credit” for the reward is assigned to the responsible action selection routine following volitional choice. Since instructed (a-volitional) choices are not as reliant on D1-mediated pathways, they receive smaller obligatory boosts. In the current experiment, Parkinson’s patients off medication were expected to have a diminished value of volition, suggesting that they do not gain the de facto enhancement of the D1 pathway while learning new action-outcome pairings. Such a diminished value of volition would also be a candidate mechanism for apathy in Parkinson’s patients.

1.4 The Current Study

This study aimed to collectively test these hypotheses of an enhanced cost of conflict and a decreased value of volition in Parkinson’s patients. These effects were proposed to be particularly potent off medication, and remediated or reversed when on medication. While we did observe the expected cost of conflict effect in recently diagnosed Parkinson’s patients, we found a different value of volition effect in these patients: an inability to learn from more difficult reinforcement contingencies when they were based on instructed, but not volitional choice. The difference in learning biases on versus off medication between these two tasks was correlated, suggesting that they tapped into a common underlying striatal D1-D2 imbalance that is particularly potent in early stage Parkinsonism.

2 Materials & Methods

2.1 Experimental Design

Participants included N=28 Parkinson’s disease patients recruited from the Albuquerque, New Mexico community and an equal number of sex and age matched controls. Parkinson’s patients visited the lab twice, seven days apart: once on medication and once after a 15-hour overnight withdrawal from their individual prescriptions of dopaminergic medication used to treat PD. Hereafter these conditions are referred to as ON or OFF. All patient and control sessions were run at 9 AM. Patients were counterbalanced across sessions based on a predefined randomized template; there were an equal number of Parkinson’s patients ON versus OFF medication in their first session. Parkinson’s patients completed neuropsychological and questionnaire assessments in their ON state. United Parkinson’s Disease Rating Scale (UPDRS) motor scores were videotaped in each patient session and were scored by a neurologist. All participants had Mini Mental State Exam (MMSE) scores above 26. Parkinson’s and control participants did not differ on any measurements of education or premorbid intelligence (see Table 1). The University of New Mexico Office of the Institutional Review Board approved the study and all participants provided informed consent. Participants were paid $20/hr for participation. Tasks were programmed in Matlab using Psychtoolbox. Participants used left and right index finger trigger buttons on a Logitech F310 gamepad for two alternative forced choice selections. All tasks used different visual stimuli between the two sessions.

Table 1.

Patient and control participant demographics (mean +/− SD). All controls were age and sex matched to a patient. Only BDI differed between groups. BDI = Beck Depression Inventory, MMSE = Mini Mental State Exam, NAART = North American Adult Reading Test, UPDRS = United Parkinson’s Disease Rating Scale (motor), LED = L-Dopa equivalence dose in mg.

PD CTL Statistic
Sex 17 M, 11 F 17 M, 11 F
Age 69.75 (8.59) 69.21 (9.23) t(54)=−.23, p=.82
Years of Education 17.25 (3.24) 16.63 (3.13) t(54)=−.72, p=.47
Parent’s Years Ed 12.49 (3.82) 12.37 (3.41) t(54)=−.23, p=.82
MMSE 28.64 (1.06) 28.82 (1.02) t(54)=.64, p=.52
NAART 45.04 (10.20) 47.00 (7.36) t(54)=.83, p=.41
BDI 7.64 (5.23) 4.93 (4.69) t(54)=−2.05, p=.046
UPDRS ON 22.14 (10.15)
UPDRS OFF 23.79 (8.71)
LED 703 (440)
Years since Diagnosis 5.54 (4.18)
Open in a new tab

2.2 Cost of Conflict Task

This task was highly similar to the seminal study (Cavanagh et al., 2014), with minor changes to make it faster and easier (more discriminating probabilities, fewer blocks) for our patient population. In each training phase, a modified Simon task was utilized to elicit response conflict during the presentation of four unique stimuli (Fig 1a). Each stimulus was presented to the left or right side of the screen. Participants were instructed to press the left gamepad button when the stimulus was yellow and the right button when it was blue. These presentations were thus either spatially congruent (screen side = response hand) or incongruent (screen side ≠ response hand) as in a standard Simon task. Stimuli consisted of four randomly assigned unique shapes (termed ‘A’, ‘B’, ‘C’ and ‘D’). Following an accurate response, participants could gain points (rewarded trial; green +1) or not (punishing trial; red 0) according to a probabilistic schedule described below. Although these points were not relevant for learning the Simon rule contingencies (which again were instructed), participants were informed that some stimuli would be more often rewarding than others and that they should learn which ones were better so they could identify them after the training block (Fig 1b).

Figure 1.

Figure 1

Open in a new tab

The two tasks used in this experiment. A) Each block of the cost of conflict task began with a training phase, where four different stimuli were associated with different reinforcement probabilities. The cost of conflict was manipulated on the B and C stimuli, where B was only rewarded following easy congruent responses and C was rewarded following harder incongruent responses. B) The colors of the stimuli indicated the response requirement of a standard Simon task: when the color rule was congruent with the response side the choice was easy (yellow = left button, stimulus on the left) yet when these were incongruent the choice elicited response conflict (blue = right button, stimulus on left). C) In a subsequent testing phase, participants chose the “most rewarding” stimulus from a series of two-alternative forced choice scenarios. The B vs C contrast provides a test of the cost of conflict on value learning. D) The value of volition task used four different pairs of stimuli with easy (90%/10%) or harder (70%/30%) reinforcement rates and a free choice vs. no choice dimension. E) Participants were informed whether they would ‘choose’ (make a free choice to sample the reinforcement probabilities), or ‘match’, (select the stimulus with the box around it). No-choice trials were yoked to free-choice trials, creating equal reward values during training. F) In a subsequent testing phase, participants chose the “most rewarding” stimulus from a series of two-alternative forced choice scenarios. A value of volition effect is expressed by reliable selection of a free-choice>no- choice option in the context of equal rewards (i.e. Afc>Anc).

The four stimuli had different reinforcement rates: the ‘A’ stimulus was 100% rewarding and the ‘D’ stimulus was 0% rewarding, each consisting of an equal number of rewards on congruent versus incongruent trials (and also on yellow vs. blue colors and left vs. right sides). In contrast, whereas stimuli B and C were equivalently reinforced at 50% rates each, the ‘B’ stimulus was reinforced on 100% of congruent trials and 0% of incongruent trials; whereas the ‘C’ stimulus was reinforced on 0% of the congruent trials and 100% of the incongruent trials. Thus if conflict reduces the value of rewards, it should reduce the learned positive value of C relative to B; conversely, if it amplifies the aversive value of negative outcomes it should cause B to have a more negative value than C. Note that B and C stimuli were also equally reinforced for each yellow and blue occurrence and in left and right locations; the sole difference between them was in the consistent experience of conflict prior to reward (‘C’) or punishment (‘B’).

There were 20 occurrences of each stimulus per training block. The inter-trial interval consisted of a fixation cross for 1000ms. If participants did not respond by the response time (RT) deadline (750ms) or if they made an error, they received informative feedback (‘No Response’ or ‘ERROR!’, respectively) and the same trial was added to the end of the block. The feedback yielded a deterministic reinforcement schedule for each stimulus within each block: participants always experienced the exact reinforcement schedule regardless of errors or delays. We anticipated that the 750ms response deadline may be too fast for some patients, so following practice we allowed it to double (1500ms) if needed. There were four Parkinson’s and one control participant who used the doubled RT deadline; Parkinson’s patients all had the same RT deadline for their ON and OFF sessions.

Following each training phase, participants entered a forced-choice testing phase where they were instructed to select “the most rewarding” stimulus from each unique pair of stimuli (each pair occurred 4 times = 48 trials total, 3000ms response deadline, Fig 1c). This testing phase provided the critical assessment of biased reinforcement learning. First, it was expected that participants should be able to reliably select A>D. In contrast, the B and C stimuli were equally reinforced at 50% rates. Thus, the perfectly crossed design eliminates any chance of performance bias due to task demands unrelated to conflict, but it also obviates an assessment of an aggregate conflict effect on choice preferences – that is, the conflict cost effect reflects a relative reward vs. punishment sensitivity balance. Conflict could diminish the value of reward (on ‘C’ specifically, leading to a B>C bias) or boost the impact of punishment (on ‘B’ specifically, leading to a C>B bias). Previous research has shown that in young adults, low-dose cabergoline administration led to a C>B bias (Cavanagh et al., 2014), thus we expected the same effect here in patients OFF medication. Participants took four train-test blocks and the results were averaged together. The cost of conflict task took an average of 38 mins (SD=7.3) to complete.

2.3 Value of Volition Task

This task was highly similar to the seminal study (Cockburn et al., 2014), with minor changes to make it easier and faster (fewer stimulus pairs, more discriminating probabilities, fewer blocks, easier threshold to move on from training). Participants completed an equal number of free-choice (fc) trials and no-choice (nc) trials. No-choice trials were yoked to free choice selections to ensure identical reinforcement and behavioral selection. This forced the stimuli to be experientially equivalent even in the context of idiosyncratic behavior, with the sole exception of the nature of choice.

During the training phase the participants were presented with four stimulus pairs, where each stimulus was a colored shape associated with a different probabilistic chance of receiving reinforcing (green 1) or punishing (red 0) feedback. These stimulus pairs (and their probabilities of reward) were termed Afc /Bfc (90% /10%) or Cfc /Dfc (70% /30%) for the free choice trials and Anc /Bnc or Cnc /Dnc for the identically reinforced no-choice trials (Fig 1d). All training trials began with an inter-trial-interval of 1000ms. First, an instruction screen informed the participants if they would “Choose” or “Match” (500ms duration) then a crosshair appeared for 800–1000ms (jittered). Then, the stimulus pairs were presented for a maximum of 4000ms, and disappeared immediately after the choice was made. If the participant failed to make a choice within the 4000ms, “No Response Detected” was presented. The no-choice “Match” trials had a square outline around the stimulus that participants were required to select; selection of the other option failed to register (Fig 1e). Following a valid button press, the unselected stimulus disappeared from the screen while feedback was presented centrally for 1250ms. Each training block had 20 presentations of each stimulus pair. Participants completed between three and five training blocks depending on accuracy (they could move to the test phase if both free choice pairs (A / B and C / D) were accurately selected >60% of the time).

During the testing phase all possible stimulus pairs were presented, with eight pairings of each yoked free-choice / no-choice set (i.e. Afc vs Anc) and four repetitions of all other pairings (i.e. Anc vs. Bfc), leading to 128 test trials total (Fig 1f). Timing and RT deadlines were the same as the training phase. No feedback was provided in the testing phase. The value of volition task took an average of 24 mins (SD=3.57) to complete.

2.4 Statistical Analyses

The sample size was chosen to match our recent cabergoline challenge in N=27 healthy young adults (Cavanagh et al., 2014), here we rounded up to N=28 to match counterbalancing of medication (ON vs OFF) during the two experimental sessions. The conflict of cost task consisted of four different training blocks, analyses of learning effects were limited to blocks where the A>D test phase choice was correct at least 75% of the time. This caused a discrepancy in the degrees of freedom for a single case of ON versus OFF contrasts. Since all experimental hypotheses were on the difference between ON and OFF sessions in Parkinson’s patients, control participant data were primarily presented for qualitative comparison that general task performance was not radically altered by Parkinson’s disease (see Table S1 for statistical analyses consisting of null effects with one set of significant findings which is detailed later on). It was expected that not all Parkinson’s patients would show similar effects of medication withdrawal, thus motor UPDRS, L-dopa equivalence dose (LED), and years since diagnosis were a priori candidate moderators of medication effects on performance.

ANOVAs and planned comparison decompositions were used for data analyses. Medication status (ON vs. OFF) was the only main or interaction effect of theoretical interest. For each task, training phase behavior was first assessed to examine any difference in performance, which could drive learning effects. Then, testing phase indicators of general learning were assessed. Finally, each a priori novel effect was tested.

3 Results

Patients did not differ from controls in any demographic or neuropsychological measure except they had higher scores on the Beck Depression Inventory, which was expected (Table 1). Motor UPDRS scores did not significantly change between ON and OFF sessions (t(27)=-1.27, p=.22), suggesting limited utility as a moderator variable. Years diagnosed and LED were strongly correlated (r(28)= .53, p=.0035). Years diagnosed was thus used as a moderator of performance, with groups split into <=median and > median. These groups did not differ on any other demographic or neuropsychological measure, except LED and percent of males (Table 2).

Table 2.

Parkinson’s patients split by years diagnosed into earlier stage (<= median) and later stage (>median) groups (mean +/− SD). Group differences were limited to LED and a higher percent of males in the later stage group. Other percentile differences were on the order of one or two patients.

Earlier Stage Later Stage t statistic
N 16 12 χ2 (1)= 1.5, p=.22χ2 (1)= 5.43, p=.02
Visit 1 ON (%) 0.44 0.58
% Male 0.44 0.83
Years since Dx 2.88 (1.26) 9.08 (4.08) t(26)=−5.77, p=4.5 e-6
LED 573.94 (390.34) 876.50 (460.29) t(26)=−1.88, p=.07
Age 70.13 (10.03) 69.25 (6.58) t(26)=0.26, p=.80
Yrs Ed 16.88 (3.24) 17.75 (3.30) t(26)=−0.70, p=.49
Yrs Ed Parents 11.62 (4.39) 13.64 (2.57) t(26)=−1.48, p=.15
UPDRS ON 22.44 (9.19) 21.75 (11.73) t(26)=0.17, p=.86
UPDRS OFF 23.63 (8.44) 24.00 (9.43) t(26)=−0.11, p=.91
UPDRS DIFF −1.19 (6.76) −2.25 (7.24) t(26)=0.40, p=.69
BDI 6.44 (4.49) 9.25 (5.89) t(26)=−1.44, p=.16
MMSE 28.44 (.96) 28.92 (1.16) t(26)=−1.19, p=.24
NAART 46.25 (7.86) 43.42 (12.88) t(26)=0.72, p=.48
Carbi/Levodopa 87.5% 100%
Pramipexole 12.5% 25%
Ropinerol 12.5% 25%
Azilect 6.25% 16.67%
Selegiline 12.5% 16.67%
Rotigotine 6.25% 0%
Amantidine 0% 8.3%
Open in a new tab

3.1 Cost of Conflict Results

There were no differences in training phase (Simon task) performance between ON and OFF states (Fig 2a). RTs were longer OFF meds, but there was no main effect of medication (F(1,26)=1.71, p=.20) nor an interaction with congruency indicating a difference in the conflict effect (F(1,26)=.02, p=.90). Test phase A>D selection was at ceiling (partially due to filtering of blocks with poor performance), demonstrating that the task was well learned. There was no difference in the conflict cost effect due to medication (t<1), see Figure 2b.

Figure 2.

Figure 2

Open in a new tab

Cost of conflict task performance. Age and sex matched controls are presented for qualitative comparison, but all analyses focused on ON versus OFF medication effects. A) Error rate did not differ due to medication status. RTs showed a linear trend for slower RTs OFF meds compared to ON, but this was not statistically significant. RT conflict (incongruent RT – congruent RT) was similar ON and OFF medication. B) In a post-training test phase, learning biases were assessed via two alternative forced choices. There was no difference in the ability to discern the best>worst option (A>D). Contrary to predictions, there was no effect of the medication manipulation in the cost of conflict effect (B>C). C) When split by recency of diagnosis, the expected cost of conflict effect due to medication manipulation occurred only in the recently diagnosed group. *p<.05 **p<.01

The recency of diagnosis was included as a group-wise moderator variable. The interaction between medication status and years diagnosed on the conflict cost effect was significant (F(1,25)=7.5, p=.01), with simple effects for the early diagnosis group (t(14)=2.46, p=.028) but not the later diagnosis group (t(11)=−1.48, p=.17). In fact, one-sample t-tests revealed a difference from chance for the earlier diagnosis group while OFF medication (t(14)=−2.14, p=.05), but not while ON (t(15)=1.12, p=.28), see Figure 2c. There was no moderating effect of years since diagnosis on training phase performance (Fig S1). Thus, the a priori hypothesis that Parkinson’s patients OFF medication would be more sensitive to experiencing conflict as a cost was confirmed, but only for earlier stage Parkinson’s patients.

3.2 Value of Volition Results

There was no difference in aggregate training accuracy due to medication, see Figure S2. Since no-choice pairs were yoked to free-choice pairs during training, effective learning of all stimulus pair discriminations must be assessed via test phase performance. We next tested the specific effects of medication status (ON vs. OFF) on volitional learning (fc vs. nc) with an additional split for trial difficulty (A/B vs. C/D) on the training phase pairs. This ANOVA revealed a main effect of all variables and moderation by medication status of all variables (medication F(1,27)=5.35, p=.029, volition F(1,27)=36.42, p=.000002, difficulty F(1,27)=10.15, p=.004, medication * volition F(1,27)=5.90, p=.022, medication * difficulty F(1,27)=5.92, p=.022, three-way interaction F(1,27)=5.49, p=.027). In a series of planned comparisons, only the Cnc vs. Dnc condition had a significant effect of medication (t(27)=−3.58, p=.001, all other pairs ts<1, ps>.33), see Figure 3a. To be clear: this effect was not expected. The finding of an inability to effectively learn to discriminate the difficult, a-volitional trials during training ON but not OFF medication precludes the ability to test for biased volitional learning between difficulty conditions.

Fig 3.

Fig 3

Open in a new tab

Value of volition test phase performance on stimulus pairs presented during training. A) An ANOVA revealed main effects of medication, volition, and reward probability, with major medication interactions driven by the Cnc vs. Dnc condition (see main text), where patients were more effective at discerning the reinforcement probabilities in the OFF state. B) When split by years diagnosed, there were significant interactions in the Cfc vs. Dfc and Cnc vs. Dnc conditions, with the latter difference driven by larger effects in the earlier stage diagnosis group. *p<.05 **p<.01

The a priori hypothesis that patients OFF medication would have a reduced value of volition (i.e. Afc = Anc) was not able to be tested due to an unexpected inability for patients ON medication to learn the Cnc vs. Dnc discrimination. Without effective discrimination of all training stimulus pairs we cannot test for differences in the relative strengths of learning between novel pairs. However we describe five reasons to consider this finding of altered difficult a-volitional learning to be a real effect of the medication manipulation and not a statistical aberration.

First, all other aspects of learning on this task were not affected by the medication manipulation. In addition to normative training phase performance (Fig S2), the expected value of volition (fc>nc) was observed in all groups, as in the seminal study (Cockburn et al., 2014), see Figure S3. A previously described interaction between medication status and valenced learning (Frank, Samanta, et al., 2007; Frank et al., 2004) tested the free choice trials in a valence (A/C vs. D/B) * medication (ON vs. OFF) ANOVA. The interaction was non-significant but trended in the expected direction (F(1,27)=2.87, p=.10 see Figure S3), demonstrating that medication led to slightly better reward learning (better A/C discrimination) and abstention led to slightly better punishment learning (better D/B discrimination).

Second, this finding was enhanced in the earlier stage diagnosis group, who were expected to experience a greater effect of medication withdrawal. Following the aforementioned findings of the cost of conflict effect, we examined if this Cnc vs. Dnc effect was moderated by recency of diagnosis. The medication * recency interaction was significant (F(1,26)=5.21, p=.031), with a significant simple effect only in the earlier stage group t(15)=4.29, p=.001, but not the later stage group t(11)=−.83, p=.422, see Figure 3b. Neither of the A vs. B contrasts were moderated by recency of diagnosis (Fs<1), yet Cfc vs. Dfc had a significant interaction (F(1,26)=5.99, p=.02) although no follow-up simple effects were significant (ts <1.8, ps>.09). While this latter interaction was unexpected, it demonstrates that the earlier stage diagnosis group had slightly better discrimination in the Cfc vs. Dfc volitional condition even in the context of a profound impairment in Cnc vs. Dnc discrimination, demonstrating that the conjunction of high difficulty and a-volitional choice was required for altered performance ON vs. OFF medication.

Third, this effect was robust. To test the generality of this finding of altered learning for difficult, a-volitional stimuli in the earlier diagnosed group, all other test phase pairs with Cnc and Dnc stimuli were contrasted. Supplemental Figure S4 shows how Cnc was significantly undervalued and Dnc was significantly overvalued in nearly all contrasts, demonstrating the robustness of this finding of altered discrimination learning in the earlier diagnosis group ON vs. OFF medication.

Fourth, only the OFF group was strongly differentiated from the control group. The only differences between Parkinson’s patients and control participants in any aspect of any task occurred in these contrasts (Table S2). Patients OFF medication were better at identifying the optimal choice in all conditions (main effect of group: F(1,54)=5.80, p=.02) while patients ON medication were better only in volitional conditions (group*volition interaction: F(1,54)=4.41, p=.04). Liberal use of follow-up t-tests indicate that the OFF group differed from controls only in the Cnc > Dnc contrast (t(54)=2.36. p=.02); the ON group was not different from controls (t(54)=−0.89. p=.38). Finally, one-sample t-tests indicate that only the OFF group had significant Cnc > Dnc discrimination above chance (t(27)=7.58, p<.01) whereas the ON group (t(27)=0.49, p=.63) and CTL group (t(27)=1.63, p=.11) did not.

Finally, to test the hypothesis that a common underlying process of D1/D2 imbalance contributed to this aberrant learning effect, the conflict cost effect and the value of volition Cnc vs. Dnc contrast were plotted against each other as a function of change due to medication (ON minus OFF). Figure 4 shows that these medication-induced learning alterations were strongly correlated with each other: r(27)=−.63, p=.4e−3. In summary, abstention from medication caused an increased ability to learn from difficult a-volitional choices; this effect was equivocated in the ON state and the size of this effect was correlated with other indices of D1/D2 imbalance.

Fig 4.

Fig 4

Open in a new tab

Medication induced changes (ON minus OFF) in the cost of conflict effect (B vs. C) and the value of volition effect (Cnc vs. Dnc) were correlated with each other, suggesting that these two effects reflect a common D2>D1 imbalance. **p<.01

4 Discussion

In this report we described how dopaminergic dysfunction in Parkinson’s disease can affect the influence of high-level cognitive states on reinforcement learning and action selection. In more recently diagnosed patients, the implicit cost of conflict was enhanced off medication. Yet in these same patients, medication compromised a heightened ability to learn from a-volitional instructed choices. Below, we interpret these findings in the context of pervasive decision biases that appear to be differentially expressed in Parkinson’s patients due to dopaminergic dysfunction.

4.1 Effort and Agency

Individuals tend to balance the demands of effort and agency in predictable ways. People implicitly seek cognitive efficiencies to reduce effort (Kool, McGuire, Rosen, & Botvinick, 2010; Zipf, 1949), yet they also generally prefer to make agentic choices when possible (Bown et al., 2003). While complex choices may be avoided (Iyengar & Lepper, 2000), people prefer options they have volitionally chosen over other equally-valued assigned items (Egan, Santos, & Bloom, 2007; Sharot et al., 2010) or novel items (Beggan, 1992; Morewedge & Giblin, 2015). Striatal dopamine has been specifically implicated in the cost of effort (Salamone et al., 2009; Westbrook & Braver, 2016), and more recently in the benefits of agency (Sharot et al., 2009; Cockburn et al., 2014; Leotti and Delgado, 2014). Costs are associated with a diminishment of positive prediction error signaling in ventral striatum (Croxson, Walton, O’Reilly, Behrens, & Rushworth, 2009; Hamid et al., 2015; Kurniawan, Guitart-Masip, Dayan, & Dolan, 2013; McGuire & Botvinick, 2010; Walton, Bannerman, Alterescu, & Rushworth, 2003) whereas volitional choices are associated with increased striatal activity (Sharot et al., 2009; Leotti and Delgado, 2014).

4.2 Enhanced Cost of Conflict in Parkinson’s

Parkinson’s patients display altered decision making (Narayanan, Rodnitzky, & Uc, 2013) particularly in the context of seeking wins and avoiding losses (Brand et al., 2004; Foerde et al., 2016; Frank et al., 2004; Mimura, Oeda, & Kawamura, 2006; Ryterska et al., 2013), but the influence of Parkinson’s disease on effortful and agentic decisions remains understudied. Chong et al. (2015) revealed how Parkinson’s patients are less willing to trade effort for reward than control participants, and that dopaminergic medication reverses this bias. Both effort and conflict costs are amplified by low dopaminergic tone and ameliorated by high dopaminergic tone (Cavanagh et al., 2014; Denk et al., 2005; Drew et al., 2007; Salamone et al., 2009; Simpson et al., 2011; Treadway et al., 2012), which in turn affect D2/D1 receptor sensitivities (Frank et al., 2004). Here we found that earlier stage unmedicated Parkinson’s patients had an increased cost of conflict and that this effect was ameliorated by medication. It is likely that long-term systematic adaptation to fluctuating dopamine levels in later-stage Parkinson’s patients causes a rebalancing of cortico-striatal plasticity, causing a relatively insensitivity to subtle higher-level influences like effort costs and volition values. This interpretation is most strongly evidenced by the increase in LED with disease duration, suggesting that increasing insensitivity to medication may blunt these extrinsic effects on learning.

4.3 Altered Value of Volition in Parkinson’s

We were not able to fully test the predicted value of volition effect, since medicated patients displayed an unexpected alteration in the tendency to learn from the difficult a-volitional instructed condition. The conjunction of difficulty and instruction here requires some unpacking. First, the importance of high difficulty in this effect is likely due to the fact that easy discriminations can be learned by the formation of an explicit rule in fronto-hippocampal systems, whereas more difficult discriminations rely on long-term cortico-striatal integration (Foerde & Shohamy, 2011; Frank, Moustafa, Haughey, Curran, & Hutchison, 2007). These two learning systems rely on dissociable aspects of dopamine function (Frank, Moustafa, et al., 2007), and Parkinson’s patients are specifically impaired in cortico-striatal but not fronto-hippocampal learning (Foerde, Braun, & Shohamy, 2012; Foerde & Shohamy, 2011). The novel aspect of the finding reported here is the a-volitional instructed dimension: why do patients have a specific enhancement in a-volitional learning off medication that is abolished with medication? The credit-assignment mechanism underlying the value of volition effect motivated by Cockburn et al. (2014) appears intact (Fig S3a), so it is likely that the a-volitional aspect relies more on D2 indirect pathway activities than previously appreciated. The fact that earlier stage patients have lower accuracy on medication yet higher accuracy off medication would be in line with enhanced sensitivity to a D2-mediated effect.

4.4. Future Directions

The finding that the influences of high-level effort and volitional effects are specific to earlier but not later stage disease offers a chance to quantify phenotypic expressions of idiosyncratic disease progression. Yet years diagnosed is only one manifest predictor of the latent systematic alteration suspected to contribute to aberrant learning in Parkinson’s disease. While imprecise, years diagnosed is easily interpretable (suggesting generalizability), it was correlated with LED (suggesting convergent validity), and it was the most powerful and consistent predictor of experimental effects (Supplemental Table 2). A longitudinal study of these findings in patients would be beneficial for directly testing the hypothesis that these findings are sensitive to disease duration. Such a study would also benefit from validation of test-retest reliability and variation of reinforcement outcomes (i.e. medication effects when seeking rewards vs. avoiding punitive outcomes).

4.5 Conclusion

The hypothesis advanced in the introduction suggested that the enhanced cost of conflict and decreased value of volition in un-medicated Parkinson’s disease would act as a “one-two punch” in perverse synergy to diminish fluidity in action selection: difficult actions would be actively hindered and beneficial actions would fail to be rewarded. The current findings suggest more of a “damned if you do, damned if you don’t” effect: with avoidance of difficult actions off medication but abolished learning of difficult a-volitional actions on medication. These subtle long-term learning alterations may contribute to quality of life issues for Parkinson’s patients, above and beyond the more overt consequences of dopaminergic neurodegeneration. An increased cost of conflict may bias patients to rely more on habits and routines and be less likely to exert complex goal-adjustment decisions. This phenomenon may mechanistically contribute to apathy in Parkinson’s patients. Variation in the ability to learn from instructed actions aligns with other basic learning alterations described in Parkinson’s patients, but implicates a new dimension of medication-variable distress. A longitudinal study of these phenomena would offer a test of the hypothesis that these findings reflect individual differences in striatal D1/D2 balance that varies due to disease progression.

Supplementary Material

supplement
NIHMS862813-supplement.docx (414.3KB, docx)

Acknowledgments

The authors thank the New Mexico Parkinson’s Coalition and Pat Thalhammer for help recruiting participants and Jeff Cockburn for use of his task.

Funding JFC is supported by NIGMS 1P20GM109089-01A1, NIMH 1UH2MH109168-01, and NIAAA R21AA0023947-01A1.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Beeler JA, Frank MJ, McDaid J, Alexander E, Turkson S, Sol Bernandez M, … Zhuang X. A Role for Dopamine-Mediated Learning in the Pathophysiology and Treatment of Parkinson’s Disease. Cell Reports. 2012:1–15. doi: 10.1016/j.celrep.2012.11.014. http://doi.org/10.1016/j.celrep.2012.11.014. [DOI] [PMC free article] [PubMed]
  2. Beggan JK. On the social nature of nonsocial perception: The mere ownership effect. Journal of Personality and Social Psychology. 1992;62(2):229–237. http://doi.org/10.1037/0022-3514.62.2.229. [Google Scholar]
  3. Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. Conflict monitoring and cognitive control. Psychol Rev. 2001;108(3):624–652. doi: 10.1037/0033-295x.108.3.624. Journal Article. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11488380. [DOI] [PubMed] [Google Scholar]
  4. Bown NJ, Read D, Summers B. The Lure of Choice. Journal of Behavioral Decision Makgin. 2003;16:297–209. [Google Scholar]
  5. Brand M, Labudda K, Kalbe E, Hilker R, Emmans D, Fuchs G, … Markowitsch HJ. Decision-making impairments in patients with Parkinson’s disease. Behavioural Neurology. 2004;15(3–4):77–85. doi: 10.1155/2004/578354. http://doi.org/10.1155/2004/578354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cavanagh JF, Frank MJ. Frontal theta as a mechanism for cognitive control. Trends in Cognitive Sciences. 2014:1–8. doi: 10.1016/j.tics.2014.04.012. http://doi.org/10.1016/j.tics.2014.04.012. [DOI] [PMC free article] [PubMed]
  7. Cavanagh JF, Masters SE, Bath K, Frank MJ. Conflict acts as an implicit cost in reinforcement learning. Nature Communications. 2014;5:5394. doi: 10.1038/ncomms6394. http://doi.org/10.1038/ncomms6394. [DOI] [PubMed] [Google Scholar]
  8. Chong TTJ, Bonnelle V, Manohar S, Veromann KR, Muhammed K, Tofaris GK, … Husain M. Dopamine enhances willingness to exert effort for reward in Parkinson’s disease. Cortex. 2015;69:40–46. doi: 10.1016/j.cortex.2015.04.003. http://doi.org/10.1016/j.cortex.2015.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cockburn J, Collins AGE, Frank MJ. A Reinforcement Learning Mechanism Responsible for the Valuation of Free Choice. Neuron. 2014;83(3):551–7. doi: 10.1016/j.neuron.2014.06.035. http://doi.org/10.1016/j.neuron.2014.06.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cools R. Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson’s disease. Neuroscience and Biobehavioral Reviews. 2006;30(1):1–23. doi: 10.1016/j.neubiorev.2005.03.024. http://doi.org/10.1016/j.neubiorev.2005.03.024. [DOI] [PubMed] [Google Scholar]
  11. Croxson PL, Walton ME, O’Reilly JX, Behrens TEJ, Rushworth MFS. Effort-based cost-benefit valuation and the human brain. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2009;29(14):4531–41. doi: 10.1523/JNEUROSCI.4515-08.2009. http://doi.org/10.1523/JNEUROSCI.4515-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Denk F, Walton ME, Jennings Ka, Sharp T, Rushworth MFS, Bannerman DM. Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology. 2005;179(3):587–96. doi: 10.1007/s00213-004-2059-4. http://doi.org/10.1007/s00213-004-2059-4. [DOI] [PubMed] [Google Scholar]
  13. Drew MR, Simpson EH, Kellendonk C, Herzberg WG, Lipatova O, Fairhurst S, … Balsam PD. Transient overexpression of striatal D2 receptors impairs operant motivation and interval timing. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2007;27(29):7731–9. doi: 10.1523/JNEUROSCI.1736-07.2007. http://doi.org/10.1523/JNEUROSCI.1736-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Egan LC, Santos LR, Bloom P. The Origins of Cognitive Dissonance. Psychological Science. 2007;18(11):978–983. doi: 10.1111/j.1467-9280.2007.02012.x. http://doi.org/10.1111/j.1467-9280.2007.02012.x. [DOI] [PubMed] [Google Scholar]
  15. Foerde K, Braun EK, Shohamy D. A Trade-Off between feedback-based learning and episodic memory for feedback events: Evidence from Parkinson’s disease. Neurodegenerative Diseases. 2012;11(2):93–101. doi: 10.1159/000342000. http://doi.org/10.1159/000342000. [DOI] [PubMed] [Google Scholar]
  16. Foerde K, Figner B, Doll BB, Woyke IC, Braun EK, Weber EU, Shohamy D. Dopamine Modulation of Intertemporal Decision-making: Evidence from Parkinson Disease. Journal of Cognitive Neuroscience. 2016;28(5):657–667. doi: 10.1162/jocn_a_00929. http://doi.org/10.1162/jocn_a_00929. [DOI] [PubMed] [Google Scholar]
  17. Foerde K, Shohamy D. Feedback timing modulates brain systems for learning in humans. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2011;31(37):13157–13167. doi: 10.1523/JNEUROSCI.2701-11.2011. http://doi.org/10.1523/JNEUROSCI.2701-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Frank MJ. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci. 2005;17(1):51–72. doi: 10.1162/0898929052880093. Journal Article. http://doi.org/10.1162/0898929052880093. [DOI] [PubMed] [Google Scholar]
  19. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A. 2007;104(41):16311–16316. doi: 10.1073/pnas.0706111104. Journal Article. http://doi.org/0706111104 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318(5854):1309–1312. doi: 10.1126/science.1146157. Journal Article. http://doi.org/1146157 [pii] [DOI] [PubMed] [Google Scholar]
  21. Frank MJ, Seeberger LC, O’Reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 2004;306(5703):1940–1943. doi: 10.1126/science.1102941. Journal Article. http://doi.org/1102941 [pii] [DOI] [PubMed] [Google Scholar]
  22. Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, … Berke JD. Mesolimbic dopamine signals the value of work. Nature Neuroscience. 2015;19(1):117–126. doi: 10.1038/nn.4173. http://doi.org/10.1038/nn.4173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Iyengar SS, Lepper MR. When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology. 2000;79(6):995–1006. doi: 10.1037//0022-3514.79.6.995. http://doi.org/10.1037/0022-3514.79.6.995. [DOI] [PubMed] [Google Scholar]
  24. Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision making and the avoidance of cognitive demand. Journal of Experimental Psychology General. 2010;139:665–682. doi: 10.1037/a0020198. http://doi.org/10.1037/a0020198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kool W, McGuire JT, Wang GJ, Botvinick MM. Neural and behavioral evidence for an intrinsic cost of self-control. PloS One. 2013;8(8):e72626. doi: 10.1371/journal.pone.0072626. http://doi.org/10.1371/journal.pone.0072626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kravitz AV, Freeze BS, Parker PR, Kay K, Thwin MT, Deisseroth K, Kreitzer AC. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature. 2010;466(7306):622–626. doi: 10.1038/nature09159. Journal Article. http://doi.org/nature09159 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nature Neuroscience. 2012;15(6):816–8. doi: 10.1038/nn.3100. http://doi.org/10.1038/nn.3100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kurniawan IT, Guitart-Masip M, Dayan P, Dolan RJ. Effort and valuation in the brain: the effects of anticipation and execution. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2013;33(14):6160–9. doi: 10.1523/JNEUROSCI.4777-12.2013. http://doi.org/10.1523/JNEUROSCI.4777-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Leotti LA, Delgado MR. The value of exercising control over monetary gains and losses. Psychological Science. 2014;25(2):596–604. doi: 10.1177/0956797613514589. http://doi.org/10.1177/0956797613514589. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McGuire JT, Botvinick MM. Prefrontal cortex, cognitive control, and the registration of decision costs. Proceedings of the National Academy of Sciences of the United States of America. 2010;107(17):7922–6. doi: 10.1073/pnas.0910662107. http://doi.org/10.1073/pnas.0910662107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mimura M, Oeda R, Kawamura M. Impaired decision-making in Parkinson’s disease. Parkinsonism & Related Disorders. 2006;12(3):169–175. doi: 10.1016/j.parkreldis.2005.12.003. http://doi.org/10.1016/j.parkreldis.2005.12.003. [DOI] [PubMed] [Google Scholar]
  32. Molina JA, Sainz-Artiga MJ, Fraile A, Jimenez-Jimenez FJ, Villanueva C, Orti-Pareja M, Bermejo-P F. Pathologic gambling in Parkinson’s disease: A behavioral manifestation of pharmacologic treatment? Movement Disorders. 2000;15(5):869–872. doi: 10.1002/1531-8257(200009)15:5<869::aid-mds1016>3.0.co;2-i. http://doi.org/10.1002/1531-8257(200009)15:5<869::AID-MDS1016>3.0.CO;2-I. [DOI] [PubMed] [Google Scholar]
  33. Morewedge CK, Giblin CE. Explanations of the endowment effect: an integrative review. Trends in Cognitive Sciences. 2015;19(6):339–348. doi: 10.1016/j.tics.2015.04.004. http://doi.org/10.1016/j.tics.2015.04.004. [DOI] [PubMed] [Google Scholar]
  34. Murty VP, DuBrow S, Davachi L. The Simple Act of Choosing Influences Declarative Memory. Journal of Neuroscience. 2015;35(16):6255–6264. doi: 10.1523/JNEUROSCI.4181-14.2015. http://doi.org/10.1523/JNEUROSCI.4181-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Narayanan NS, Rodnitzky RL, Uc EY. Prefrontal dopamine signaling and cognitive symptoms of Parkinson’s disease. Reviews in the Neurosciences. 2013;24(3):267–78. doi: 10.1515/revneuro-2013-0004. http://doi.org/10.1515/revneuro-2013-0004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Porter-stransky KA, Seiler JL, Day JJ, Aragona BJ. Development of behavioral preferences for the optimal choice following unexpected reward omission is mediated by a reduction of D2-like receptor tone in the nucleus accumbens. 2013;38(April):2572–2588. doi: 10.1111/ejn.12253. http://doi.org/10.1111/ejn.12253. [DOI] [PubMed] [Google Scholar]
  37. Ryterska A, Jahanshahi M, Osman M. What are people with Parkinson’s disease really impaired on when it comes to making decisions? A meta-analysis of the evidence. Neuroscience and Biobehavioral Reviews. 2013;37(10):2836–2846. doi: 10.1016/j.neubiorev.2013.10.005. http://doi.org/10.1016/j.neubiorev.2013.10.005. [DOI] [PubMed] [Google Scholar]
  38. Salamone JD, Correa M, Farrar AM, Nunes EJ, Pardo M. Dopamine, behavioral economics, and effort. Frontiers in Behavioral Neuroscience. 2009;3:13. doi: 10.3389/neuro.08.013.2009. http://doi.org/10.3389/neuro.08.013.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sharot T, De Martino B, Dolan RJ. How Choice Reveals and Shapes Expected Hedonic Outcome. Journal of Neuroscience. 2009;29(12):3760–3765. doi: 10.1523/JNEUROSCI.4972-08.2009. http://doi.org/10.1523/JNEUROSCI.4972-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sharot T, Shiner T, Brown AC, Fan J, Dolan RJ. Dopamine enhances expectation of pleasure in humans. Curr Biol. 2009;19(24):2077–2080. doi: 10.1016/j.cub.2009.10.025. Journal Article. http://doi.org/S0960-9822(09)01844-2 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sharot T, Velasquez CM, Dolan RJ. Do Decisions Shape Preference?: Evidence From Blind Choice. Psychological Science. 2010;21(9):1231–1235. doi: 10.1177/0956797610379235. http://doi.org/10.1177/0956797610379235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Shenhav A, Botvinick MM, Cohen JD. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron. 2013;79:217–40. doi: 10.1016/j.neuron.2013.07.007. http://doi.org/10.1016/j.neuron.2013.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Simpson EH, Kellendonk C, Ward RD, Richards V, Lipatova O, Fairhurst S, … Balsam PD. Pharmacologic rescue of motivational deficit in an animal model of the negative symptoms of schizophrenia. Biological Psychiatry. 2011;69(10):928–35. doi: 10.1016/j.biopsych.2011.01.012. http://doi.org/10.1016/j.biopsych.2011.01.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Tai LH, Lee aM, Benavidez N, Bonci A, Wilbrecht L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nature Neuroscience. 2012;15(9):1281–9. doi: 10.1038/nn.3188. http://doi.org/10.1038/nn.3188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Treadway MT, Buckholtz JW, Cowan RL, Woodward ND, Li R, Ansari MS, … Zald DH. Dopaminergic mechanisms of individual differences in human effort-based decision-making. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2012;32(18):6170–6. doi: 10.1523/JNEUROSCI.6459-11.2012. http://doi.org/10.1523/JNEUROSCI.6459-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan RJ, Hallett M. Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron. 2010;65(1):135–142. doi: 10.1016/j.neuron.2009.12.027. Journal Article. http://doi.org/S0896-6273(09)01045-9 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Walton ME, Bannerman DM, Alterescu K, Rushworth MFS. Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. The Journal of Neuroscience3: The Official Journal of the Society for Neuroscience. 2003;23(16):6475–9. doi: 10.1523/JNEUROSCI.23-16-06475.2003. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12878688. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Westbrook A, Braver TS. Dopamine Does Double Duty in Motivating Cognitive Effort. Neuron. 2016;89(4):695–710. doi: 10.1016/j.neuron.2015.12.029. http://doi.org/10.1016/j.neuron.2015.12.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wiecki TV, Frank MJ. Neurocomputational models of motor and cognitive deficits in Parkinson’s disease. Prog Brain Res. 2010;183:275–297. doi: 10.1016/S0079-6123(10)83014-6. Journal Article. http://doi.org/S0079-6123(10)83014-6 [pii] [DOI] [PubMed] [Google Scholar]
  50. Zipf GK. Human behavior and the principle of least effort. Cambridge, MA: Addison_Wesley; 1949. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS862813-supplement.docx (414.3KB, docx)

RESOURCES