Reward processing abnormalities in Parkinson disease

Abstract

Background

The primary motor cortex is important for motor learning and response selection, functions that require information on the expected and actual outcomes of behavior. Therefore, it should receive signals related to reward and pathways from reward centers to motor cortex exist in primates. Previously, we showed that gamma aminobutyric acid-A(GABA_A)-mediated inhibition in motor cortex, measured by paired transcranial magnetic stimulation (TMS), changes with expectation and uncertainty of money rewards generated by a slot machine simulation.

Methods

We examined the role of dopamine in this phenomenon by testing 13 mildly affected Parkinson disease patients, off and on dopaminergic medications, and 13 healthy, age-matched controls.

Results

Consistent with a dopaminergic mechanism, reward expectation or predictability modulated the response to paired TMS in controls, but not in unmedicated patients. A single dose of pramipexole restored this effect of reward, mainly by increasing the paired TMS response amplitude during low expectation. Levodopa produced no such effect. Both pramipexole and levodopa increased risk-taking behavior on the Iowa Gambling Task. However, pramipexole increased risk-taking behavior more in patients showing lower paired TMS response amplitude during low expectation.

Conclusions

These results provide evidence that modulation of motor cortex inhibition by reward is mediated by dopamine signaling and that physiological states in the motor cortex are associated with levels of risk-taking behavior in patients on pramipexole. The cortical response to reward expectation may represent an endophenotype for risk-taking behavior in patients on agonist treatment.

Introduction

Parkinson disease (PD) can cause behavioral problems linked to impaired processing of reward and risk,^{1, 2} including pathological gambling, hypersexuality, and medication misuse.³ These occur spontaneously, but more commonly with dopamine (DA) agonists,^4–7 and are a significant cause of morbidity. Impaired reward processing also affects learning: unmedicated PD patients learn from punishment,² not reward;⁸ whereas medicated patients learn from reward,⁹ not punishment.²

Short-interval, paired-pulse transcranial magnetic stimulation (TMS) uses the effect of a subthreshold, conditioning pulse on the motor-evoked potential (MEP) produced by a second, suprathreshold, pulse to measure the state of primary motor area (M1) circuits. The effect produced is either gamma aminobutyric acid-A(GABA_A)-mediated intracortical inhibition (ICI) at interstimulus intervals (ISIs) of 1–4 ms, or glutamatergic intracortical facilitation (ICF) at ISIs of 6–20 ms.^12–14 There is evidence that inhibition is also present simultaneously with ICF at the longer intervals. ICI is increased and ICF decreased by DA agonists,¹⁵ and benzodiazepines;^{16, 17} DA antagonists produce opposite effects.¹⁵ This is consistent with the known actions of DA in the motor cortex: DA neurons excite GABAergic interneurons,¹⁸ which inhibit cortical pyramidal cells.^{19, 20} DA neurons also inhibit pyramidal neurons directly.^{19, 21}

Earlier,²² we examined the influence of reward on ICI in healthy men, using paired-pulse TMS and a slot machine simulation, which manipulated reward expectation. Subjects showed more ICI and less ICF with high expectation of a money reward, relative to low expectation. We attributed this parallel change of ICI and ICF to mesocortical DA signaling. Here, we tested the hypothesis that the expectation-related change ICF is DA-mediated by measuring it in PD patients on and off medications and age-matched controls, using the same paradigm. We expected the patients to show less reward-related change in the unmedicated, compared to the medicated, state and compared to the controls. We administered the Iowa Gambling Task (IGT)^{26, 27} to the patients, on and off both medications, to see if the physiological changes were associated with changes in risk-taking.

Methods

Subjects

We tested 13 PD patients (mean age 69.1 ± 9.1 years; 6 women; Table 1) and 13 healthy volunteers (mean age 68.6 ± 8.4 years; 8 women). We did not consider the sex mismatch between the groups significant, since the women were post-menopausal. Patients were examined by a neurologist to rule out significant neuropsychiatric and neurological comorbidities. Four patients had Hoehn & Yahr Stage I disease (mean age 62.1 ± 7.8 years; 2 women) and nine had Stage II (mean age 74.4 ± 5.7 years; 4 women). Medications are listed in Table 1. None had taken DA agonists within a month before study. All participants provided written informed consent and the study was approved by the Neuroscience Institutional Review Board of the National Institutes of Health.

Table 1.

Parkinson patient demographic and clinical characteristics.

Patient	Sex	Age	Years since diagnosis	Hoehn & Yahr stage	Anti-Parkinson and other relevant medications (levodopa equivalent daily dose)*
1	M	74	1	1	Levodopa/carbidopa (200 mg)
2	M	70	2	2	Levodopa/carbidopa, entacapone, escitalopram (500 mg)
3	M	74	14	2	Levodopa/carbidopa, amantadine, selegiline (325 mg)
4	M	59	1	1	Rasagiline (200 mg)
5	M	66	1	2	Levodopa/carbidopa, rasagiline (500 mg)
6	F	52	3	1	Amantadine, paroxetine, estrogen supplement (200 mg)
7	F	67	1	2	Levodopa/carbidopa, entacapone (350 mg)
8	F	77	6	2	Levodopa/carbidopa (300 mg)
9	F	82	5	2	Levodopa/carbidopa (300 mg)
10	M	73	2	2	Levodopa/carbidopa (600 mg)
11	M	68	8	2	Levodopa/carbidopa, amantadine (800 mg)
12	F	80	1	2	Levodopa/carbidopa (300 mg)
13	F	50	1	1	Mucura pruriens

^*Levodopa equivalent daily dose was calculated using the following conversion factors: Immediate release levodopa: 1, controlled release levodopa: 0.75, entacapone: 0.33, amantadine: 1, oral selegiline: 10, for rasagiline: 100.

Procedure

We tested each patient twice on each of two days, separated by at least 24 hours. Patients took no antiparkinsonian drugs for at least 12 hours before each session. On each of two days, patients were first tested with TMS in the unmedicated state. Next, they were give neither 25/100 mg of levodopa/carbidopa or 0.25 mg of the agonist, pramipexole and retested after 1 hour. On the second day, the same schedule was followed, with the other medication. The order of medications was randomized and counterbalanced across patients. Patients were also tested with the IGT, once in each of the three treatment states. Controls participated in a single session.

Slot Machine Simulation (task)

As in our previous study,²² we used a computer simulation of a three-barrel slot machine,(Presentation^® software; www.neurobs.com). Subjects initiated each trial in response to a ready-cue by pressing the space bar with the non-recorded hand (See EMG, below). All three barrels then began spinning, stopping sequentially at 2-s intervals. Single or paired TMS was delivered 1 s after the second barrel stopped, at which point a 3-way winning match was either possible (2-way match, high reward expectation, high uncertainty) or impossible (no match, low expectation, low uncertainty). The experiment comprised 30 no-match and 120 2-way match trials. Half of the 2-way match trials resulted in a 3-way match. The game delivered money on trials resulting in a 3-way match. There were 30 winning trials paying$ 0.25, 26 paying $1, and 4 paying $5. We used two versions of the simulation, with equivalent numbers of rewarded and non-rewarded trials, but with different sequences, to avoid learning effects. Winnings were represented online and paid later. Paired and single TMS pulses (see below) were delivered in pseudo-randomized order and matched for frequency across trial conditions.

Iowa Gambling Task (IGT)

We used three standard versions of the IGT (ABCD, KLMN and QRST),^{26, 27} each consisting of 100 trials. Subjects selected cards indicating wins or losses of game money from four decks. In each version, two decks provided large average wins initially, but later caused large losses (risky decks), while the two others provided smaller wins throughout the game, which were not offset by losses (non-risky decks). The effect of learning on the IGT is a decreased tendency to select from the risky decks.^{26, 27} Patients performed one version in the unmedicated, and another in each medication, condition. The order of versions was randomized and counterbalanced across subjects.

Electromyography

Surface electrodes were applied to the finger flexors of the dominant hand, except in two patients with tremor on the dominant side, in whom the non-dominant hand was tested. Subjects were instructed to keep the hand relaxed and electromyographic activity was monitored visually. The signal was amplified (5 kHz) and filtered (band-pass 90 Hz to 1 kHz; Coulbourn Instruments), digitized at 2 kHz (Micro 1401^®, Cambridge Electronics Design) and stored for offline analysis.

TMS

At the beginning of each session, resting motor threshold (RMT) in the finger flexors was determined by approach from below in a stepwise manner. RMT was defined as the first stimulus intensity that produced a motor evoked potential (MEP) > 50 μV on 5 out of 10 consecutive trials.¹⁵

After RMT determination, we measured the baseline response to single and paired stimulation with 15 single (test) and 15 paired pulses, randomly intermixed. In our previous study²² significant effects of reward expectation were observed at both the 2 (ICI) and 10 ms (ICF) ISIs, consistent with the effect of DA agonists and antagonists.^{15, 28} However, the effect size was greater at 10 ms and here, in order to minimize the number of trials, we used only the 10 ms ISI. The inter-trial interval varied randomly by 0–20% around a mean of 6 s. During baseline testing, subjects were instructed to sit at rest with their eyes open.

TMS was delivered from two Magstim 200^® stimulators via a Bistim^® module, through a round coil placed in the optimal position for producing MEPs in the finger flexors of the tested hand. For paired TMS, the intensity of the conditioning pulse was set at 65%, and the test pulse at 120%, of RMT for each subject.^{14, 22, 29}

Data analysis and statistics

TMS

Trials where the baseline EMG was contaminated with ≥ 2 SD of the mean amplitude for that subject were excluded from the analysis. Peak-to-peak MEP amplitudes were measured automatically (Signal^® software, Cambridge Electronic Design).

For each subject, the effect of the conditioning pulse in the paired condition (conditioned MEP ratio) was calculated as the mean amplitude of the MEP produced by the paired-pulse divided by the mean MEP amplitude from test pulses for the same behavioral condition.

We analyzed the TMS data with SPSS 15 software, based on a linear model for fixed effects, a method similar to the general linear model, but making fewer assumptions for variance. We performed the following analyses with conditioned MEP ratio as the dependent variable: the effect of Group (patient vs. control) in unmedicated patients and controls tested at baseline (model 1); the effects and interactions of Group (patient vs. control) and Reward Expectation (high vs. low) in unmedicated patients and controls during the task (model 2); the effect of Medication State (unmedicated vs. levodopa vs. pramipexole) at pre-task baseline (model 3);and the effects and interactions of Medication State (unmedicated vs. levodopa vs. pramipexole) and Reward Expectation (high vs. low) in patients during the task (model 4). Finally, we examined the effects and interactions of Medication State and Reward Expectation on the MEP amplitude to single TMS (model 5). All available models of covariance structure were applied and the best fitting model was determined, based Akaike’s information criterion and Schwarz’s Bayesian criterion. This proved to be the first-order autoregressive moving average, as is typical for repeated measures analyses. Denominator degrees of freedom were calculated. Significance level was set at α = .05 and Bonferoni adjustment for multiple comparisons applied in post-hoc comparisons.

IGT

The IGT result files for two patients were corrupted, so 11 were included in the analyses. To evaluate risk-based learning, we divided the 100 trials from each session into 5 blocks of 20 trials and recorded the number of risky selections in each. To test whether Medication State affected learning, we performed a mixed linear model analysis for the effect of Medication State (unmedicated vs. levodopa vs. pramipexole) on the number of risky selections in block 5.

To evaluate whether risk-taking behavior was associated with the TMS measures, we performed a stepwise regression for each level of Medication State (probability of F for entry = 0.05; probability of F for removal = 0.10). The number of risky selections in block 5 was the dependent, and the low and high reward expectation conditioned MEP ratios the independent, variable.

Results

TMS Experiment

Unmedicated patients vs. controls

Group did not affect conditioned MEP ratios at pre-task baseline (model 1: F [1, 262.5] = 2.7; p = 0.1) nor during the task (model 2: F [1, 137.8] = 0.2; p = 0.6). The main finding of the group comparisons was that the controls, but not the patients, showed a change in ICF with reward expectation/uncertainty (model 2, Group × Reward Expectation interaction: F [1, 440.7] = 7.2; p = 0.008; Fig. 1). Controls had more ICF on low than on high expectation trials (model 2, F [1, 464.1] = 5.9; p = 0.003), but unmedicated patients showed no difference (model 2, F [1, 343.5] = 0.03; p = 1.0). Controls also had more ICF than unmedicated patients during low expectation (model 2: F [1, 329.1] = 7.7; p = 0.006), but the two groups had similar ICF with high expectation (model 2: F [1, 237.5] = 0.2; p = 0.7).

Figure 1.

Expectation-related MEP ratiochanges in unmedicated PD patients and healthy controls. Conditioned MEP ratio changed with reward expectation in the age-matched controls, but not in the PD patients. Conditioned MEP ratio = the mean MEP amplitude from the paired pulse/mean MEP amplitude from the test pulse alone; higher ratios signify less inhibition. Bars show standard error.

Effect of medication

Medication State did not affect conditioned MEP ratio during the pre-task baseline test (model 3, F [2, 1364.0]= 0.3; p = 0.8, Fig. 2), nor did it produce a main effect on ICF during the task (model 4, F [2, 1155.9] = 2.5; p = 0.079, Fig. 2). The main finding here was that pramipexole restored the change in ICF associated with reward expectation/uncertainty (model 4, Medication × Reward Expectation interaction: F [2, 1297.0] = 4.9; p = 0.007, Fig. 2). This interaction was driven principally by the increase in ICF during low expectation (model 4, low expectation × Medication interaction: F [2, 1226.5] = 3.13, p = 0.044), although there was also a tendency for ICF to decrease with high expectation (model 4, high expectation × Medication interaction: F [2, 1142.5] = 2.99, p = 0.051). Examining the effect of Reward Expectation separately for each medication state, ICF was greater with low than with high expectation on pramipexole (model 4, pramipexole × Reward Expectation interaction: F [1, 1299.2] = 4.2; p = 0.04), but not on levodopa (model 4, levodopa × Reward Expectation interaction: F [1, 1289.6] = 0.72; p = 0.8) nor in the unmedicated state (model 4, unmedicated state × Reward Expectation interaction: F [1, 249.6] = 0.4; p = 0.9; Fig. 2).

Figure 2.

Expectation-related MEP ratiochanges in PD patients by medication condition. Pramipexole, but not levodopa, restored the conditioned MEP ratio change with altered reward expectation. Conditioned MEP ratio = the mean MEP amplitude from the paired pulse/mean MEP amplitude from the test pulse alone; higher ratios signify less inhibition. Bars show standard error.

There were no main effects or an interaction of Reward Expectation and Medication State on unconditioned test MEP amplitude (model 5, Reward Expectation: F [1, 205.2] = 0.1; p = 0.8; Medication State: F [1, 94.4] = 1.2; p = 0.3).

IGT

Patients made more risky selections in block 5 on both levodopa (F (1, 97.5) = 6.9, p = 0.01) and pramipexole (F [1, 91.0] = 4.2, p = 0.04) compared with the unmedicated condition (Fig. 3). The number of risky selections in the two medication states did not differ significantly.

Figure 3.

Risk-taking on the Iowa Gambling Task. Patients on levodopa and pramipexole chose risky decks more often and showed less risk-based learning on medication than off. Bars show standard error.

On pramipexole, the number of risky selections made by individual patients on block 5 was inversely correlated with their ICF in the low expectation state (Beta = −0.61, p = 0.048; Fig. 4). There were no significant associations of TMS measures and risky selections on block 5 with levodopa or off medication.

Figure 4.

Risky selections on the Iowa Gambling Task (y-axis) were negatively associated with the amount of change in the MEP with low reward expectation in patients on pramipexole. Conditioned MEP ratio = the mean MEP amplitude from the paired pulse/mean MEP amplitude from the test pulse alone; higher ratios signify less inhibition. Bars show standard error.

Discussion

This study was designed to examine the how M1 circuits and output neurons respond to changing reward expectation or uncertainty in the presence and absence of DA. Reward-related modulation of excitability was present in older, healthy subjects, but absent in unmedicated PD patients. A single dose of pramipexole, but not levodopa, restored it in patients. The absence of modulation in unmedicated patients supports our hypothesis, i.e., that the M1 reward response is transmitted via a dopaminergic pathway. We postulated previously²² that reward-related information reaches M1 directly, via the mesocortical projection from the VTA^{30, 31} where neuronal loss likely causes the cortical DA depletion seen in PD.³² Nevertheless, we cannot rule out the possibility that these signals reach M1 via the basal ganglia. Regardless of their mechanism, however, the findings support our claim that M1 processes reward-related signals. The absence of M1 reward-related signaling in PD also provides new insight into how impaired reward and motor learning³³ may be linked.

Previously, we attributed the increased ICI/decreased ICF during high reward expectation to tonic release of DA in M1. The current results suggest that decreased reward expectation (or the concomitant change in uncertainty) is also coded by cortical DA activity, which is impaired in PD. Our results show that cortical reward signaling can be partially restored by medication in unexpected ways: We expected the DA precursor, levodopa, to restore the M1 response to reward expectation and the direct D2/D3 agonist, pramipexole, to decrease ICF across expectation states by occupying postsynaptic DA receptors. However, this did not occur. In the patients, pramipexole caused increased ICF during low expectation, where as levodopa had no significant effect. The explanation may lie in pramipexole’s dose-dependent action on different receptor classes.^{34, 35} In low doses, pramipexole can impair phasic DA release^36–38 and VTA firing³⁹ via presynaptic inhibition. We believe that the low reward expectation condition produced a state equivalent to the decreased tonic DA neuron firing described by Tobler et al.,⁴⁰ when monkeys received a smaller than expected reward. In our experiment, such a firing decrease may have been reinforced by the presynaptic action of pramipexole, augmenting the ICF increase with low expectation and, thereby, restoring the physiological differential between expectation states.

We included the IGT to see whether the physiological response to reward expectation was associated with differences in behavior. Both pramipexole and levodopa increased risky selections on average, presumably by increasing occupation of postsynaptic DA receptors. Pramipexole, however, restored the mean ICF increase with low reward expectation and individuals showing more ICF increase with low expectation made fewer risky choices on block 5 of the IGT (Fig. 4). As stated above, we attribute the physiological effect of pramipexole to its presynaptic action. Among individuals, however, the balance of presynaptic and postsynaptic effects might vary. Patients with less pre-synaptic inhibition would have more synaptic DA available^36–38 and experience less withdrawal⁴⁰ and negative emotional response to IGT losses. The same individuals would also presumably show less ICF increase with low expectation. Those with more presynaptic effect would show the opposite effects. We believe this explains the association in Fig. 4.

The strength of the association between the TMS and IGT measures was notable, with ICF accounting for 37% of the variance in risk-taking behavior. This result underscores the influence of subcortical neuromodulatory systems even on seemingly complex behavior.⁴¹ Other associations between M1 physiology and trait-level behavior have been described in the general population⁴² and neurobehavioral disorders.^43–45 A noninvasive probe for activity in the human reward system may provide additional insight into its function.