Dopamine is associated with prioritization of reward-associated memories in Parkinson’s disease

Studies of memory in Parkinson’s disease have focused on overall memory capacity. Sharp et al. show that dopamine replacement selectively enhances reward-associated memories in patients, suggesting that prioritization of content for long-term storage is impaired in Parkinson’s disease, rather than overall memory capacity.

Abstract

Patients with Parkinson’s disease have reduced reward sensitivity related to dopaminergic neuron loss, which is associated with impairments in reinforcement learning. Increasingly, however, dopamine-dependent reward signals are recognized to play an important role beyond reinforcement learning. In particular, it has been shown that reward signals mediated by dopamine help guide the prioritization of events for long-term memory consolidation. Meanwhile, studies of memory in patients with Parkinson’s disease have focused on overall memory capacity rather than what is versus what isn’t remembered, leaving open questions about the effect of dopamine replacement on the prioritization of memories by reward and the time-dependence of this effect. The current study sought to fill this gap by testing the effect of reward and dopamine on memory in patients with Parkinson’s disease. We tested the effect of dopamine modulation and reward on two forms of long-term memory: episodic memory for neutral objects and memory for stimulus-value associations. We measured both forms of memory in a single task, adapting a standard task of reinforcement learning with incidental episodic encoding events of trial-unique objects. Objects were presented on each trial at the time of feedback, which was either rewarding or not. Memory for the trial-unique images and for the stimulus-value associations, and the influence of reward on both, was tested immediately after learning and 2 days later. We measured performance in Parkinson’s disease patients tested either ON or OFF their dopaminergic medications and in healthy older control subjects. We found that dopamine was associated with a selective enhancement of memory for reward-associated images, but that it did not influence overall memory capacity. Contrary to predictions, this effect did not differ between the immediate and delayed memory tests. We also found that while dopamine had an effect on reward-modulated episodic memory, there was no effect of dopamine on memory for stimulus-value associations. Our results suggest that impaired prioritization of cognitive resource allocation may contribute to the early cognitive deficits of Parkinson’s disease.

Introduction

We are constantly presented with demands on our cognitive resources, whether on attention, memory or sensory processing, that exceed our cognitive processing capacity. One proposed solution for this challenge is that the brain uses environmental cues, like reward, to prioritize certain events or experiences over others (Adcock et al., 2006; Abe et al., 2011; Anderson et al., 2011; Hickey et al., 2015; Gruber et al., 2016; Klink et al., 2017; Braun et al., 2018). Growing evidence suggests that dopamine plays a central role in this reward-guided prioritization process (Shohamy and Adcock, 2010; Redondo and Morris, 2011; McNamara et al., 2014; Kim et al., 2015).

The question of how the brain uses dopamine-mediated reward signals to prioritize cognitive resource allocation is especially relevant to patients with Parkinson’s disease, who have cognitive deficits from the earliest stages of the disease (Foltynie et al., 2004; Aarsland et al., 2010; Dirnberger and Jahanshahi, 2013; Robbins and Cools, 2014). Several aspects of these cognitive deficits are specifically attributed to dopamine neuron loss (Frank et al., 2004; Foerde and Shohamy, 2011; Kehagia et al., 2013; Sharp et al., 2016). Yet, these deficits remain incompletely understood and represent a major unmet need in Parkinson’s disease (Goldman et al., 2018). In particular, it is well established that patients with Parkinson’s disease have altered reward responses related to dopamine neuron loss and dopamine replacement therapy (Frank et al., 2004; Voon et al., 2010; Muhammed et al., 2016; McCoy et al., 2019). Studies of reinforcement learning in patients with Parkinson’s disease also reveal a specific dopamine-dependent deficit in the ability to use reward information to help guide learning and subsequent decisions (Knowlton et al., 1996; Frank et al., 2004; Shohamy et al., 2004; de Wit et al., 2011; Sharp et al., 2016; Grogan et al., 2017; McCoy et al., 2019). However, it is less well known how dopamine loss in Parkinson’s disease affects the prioritization of information for long-term memory.

Dopamine’s role in guiding the prioritization of episodic, or hippocampal-dependent, memory is well established. It has been shown both in humans (Wittmann et al., 2005; Adcock et al., 2006; Callan and Schweighofer, 2008; Shohamy and Adcock, 2010; Wolosin et al., 2012; Gruber et al., 2016; Murty et al., 2017; Patil et al., 2017) and in rodents (Singer and Frank, 2009; McNamara et al., 2014; Ambrose et al., 2016; Takeuchi et al., 2016) that memories encoded around the same time as motivationally significant events, such as reward or novelty, are better remembered over the long term, an effect that is mediated by dopamine (Bethus et al., 2010; Redondo and Morris, 2011; McNamara et al., 2014; Valdés et al., 2015; Takeuchi et al., 2016). Reward can exert a modulatory effect on memory in ways that are both strategic and prospective (in tasks that explicitly guide encoding of reward-relevant information in advance) as well as in ways that are incidental and retroactive (Adcock et al., 2006; Gruber et al., 2016; Braun et al., 2018; Rouhani et al., 2018; Jang et al., 2019). The effect of dopamine on hippocampal memories is thought to be time-dependent, generally emerging after a period of off-line consolidation (Bethus et al., 2010; Redondo and Morris, 2011; Takeuchi et al., 2016; Braun et al., 2018).

These findings raise important questions about the role of dopamine and reward in modulating long-term memory in patients with Parkinson’s disease. Specifically, it is not known whether patients with Parkinson’s disease are impaired at incidental prioritization of memories for long-term storage based on reward information, whether such impairments are specifically linked to the time-dependence of the consolidation process, and whether other forms of long-term memory, namely memory for stimulus-value associations learned from reinforcement, are also modulated by the presence of reward at the time of learning.

To address these open questions, we tested patients either ON or OFF dopaminergic medications on a novel adaptation of a learning and memory task. We were particularly interested in the automatic influence that reward exerts on memory as opposed to the influence on strategy that it can also exert. We therefore adapted a standard task of reinforcement learning (Frank et al., 2004) to integrate environmental reward cues into the encoding process by adding unique incidental episodic encoding events on each trial: images of objects were presented at the time of feedback, which was either rewarding or not. Thus, we used the feedback event itself to create a motivational context during which encoding occurred. There was no explicit advantage to remembering the reward-associated objects and each of them only appeared once. Memory for the trial-unique images was tested immediately after learning and 2 days later.

We hypothesized that dopamine, by signalling reward, would serve to orient the prioritization of memory storage such that images encoded in the presence of reward as compared to non-rewarding feedback would be preferentially remembered and that this effect of reward on prioritization would only emerge after the passage of a consolidation time period. To determine if the effects of reward and dopamine on memory are specific to episodic memory or if they also extend to long-term memory of stimulus-value associations, we also measured the change in memory across the 2 days for the stimulus-value associations that were acquired in the reinforcement learning task.

We found that dopamine replacement in patients with Parkinson’s disease was associated with enhanced episodic memory for reward-associated images, while overall memory capacity was not influenced by medications and was similar to that of healthy control subjects. Contrary to predictions, we found that the dopamine-dependent effect of reward was present across both testing time-points and did not differ with consolidation. Finally, we found that this effect was selective to episodic memory; neither dopamine state nor the presence of reward influenced memory for stimulus-value associations. These findings suggest that patients with Parkinson’s disease display an overall memory capacity that is similar to healthy controls, but have dopamine-dependent impairments in the prioritization of the content for memory storage based on reward cues. Given the important role that reward signals play across cognitive domains, these results raise the possibility that impaired prioritization of cognitive resource allocation could provide a unifying mechanism to explain early cognitive deficits in patients with Parkinson’s disease.

Materials and methods

Participants

Fifty-one patients with Parkinson’s disease were recruited either from the Center for Parkinson’s Disease and other Movement Disorders at the Columbia University Medical Center or from the Michael J Fox Foundation Trial Finder website. Of these, 24 were evaluated after an overnight withdrawal of all dopaminergic medications and 25 were evaluated after taking their usual dose of levodopa (procedure described in greater detail below). Twenty-five healthy control participants were recruited from the local community. Patients were in the mild-to-moderate stage of the disease, had been receiving a stable dose of levodopa for at least 2 months and endorsed levodopa-responsiveness. An additional 10 patients in the ON group and 10 in the OFF group were also being treated with a dopamine agonist. The main analyses reported below were performed on the subgroup of participants who met a basic learning criterion (see task details below); 20 of the 25 healthy controls, 16 of the 25 patients ON and 14 of 25 patients OFF medications met this criterion. Table 1 provides characteristics for this subgroup, and Supplementary Table 1 provides characteristics for the full sample. Our original sample was planned to provide 80% power to detect an effect of medications with a medium-large effect size of 0.72 (Cohen’s d, based on alpha = 0.05, and a one-sided t-test); our main analyses relying on the subgroup of participants that met the learning criterion had 80% power to detect a larger effect size of 0.93. All participants provided written informed consent and were paid $20/h for their participation. The study was approved by the Institutional Review Board of Columbia University.

Table 1.

Demographic and clinical characteristics of participants who met the learning criterion

	Healthy controls (n = 20)	Parkinson's disease ON (n = 16)	Parkinson's disease OFF (n = 14)	P-value *
Age	62.5 (6.4)	62.1 (8.2)	62.4 (6.5)	0.981
Sex, male	7/20	12/16	6/14	0.498
Education, years	17.9 (2.4)	19.3 (3.1)	19.1 (2.2)	0.153
MoCA	28.4 (1.3)	28.9 (1.8)	29.1 (1.3)	0.147
F-A-S fluency	48.9 (13.1)	49.9 (9.9)	50.2 (10.0)	0.730
Trails B	68.1 (24.6)	77.4 (29.9)	64.9 (19.0)	0.829
Digit Span totala	13.2 (2.1)	13.5 (1.7)	12.6 (1.7)	0.453
Geriatric Depression Scale	1.4 (1.8)	2.4 (2.4)	2.2 (3.1)	0.264
Starkstein Apathy Scale	22.4 (4.4)	22.8 (5.8)	21.3 (7.2)	0.631
UPDRS	n/a	20.6 (8.2)	26.3 (9.9)	0.109
Disease duration	n/a	7.3 (3.0)	8.1 (4.5)	0.537
LEED, mgb	n/a	626 (331)	625 (157)	0.989

Values are provided as mean (standard deviation) and include only the participants who met the learning criterion. P-values are based on one-way ANOVA or t-test (where applicable).

LEED = Levodopa equivalent dosing; MoCA = Montreal Cognitive Assessment; n/a = not applicable; UPDRS = Unified Parkinson’s Disease Rating Scale-Part III, tested ON in ON group, and OFF in OFF group and averaged across both testing days (except for four patients who only underwent UPDRS once, and one patient who is missing UPDRS data).

^*One-way ANOVA.

^aDigit Span total = sum of forward and backward span.

^bLEED includes levodopa, dopamine agonists, amantadine, monoamine oxidase inhibitors and catechol-O-methyl transferase inhibitors.

Participants completed a battery of neuropsychological tests focusing on executive function [Montreal Cognitive Assessment (MoCA), Trails A and B, Digit Span, and Phonemic word fluency] and psychiatric domains (Geriatric Depression Scale and Starkstein Apathy Scale). Participants had no history of other major neurological or psychiatric disease.

Procedure

Behavioural testing of OFF patients was conducted after an overnight withdrawal of both levodopa and dopamine agonists (>16 h, which is at least 10 half-lives for the carbidopa-levodopa and two half-lives for the dopamine agonists). Behavioural testing of the patients ON started 1–1.5 h after their usual dose of levodopa. Patients in the ON group who were also on regular doses of dopamine agonists (5 of 16 in the final subgroup) did not receive them because we wanted to isolate the effects of levodopa, which most closely mimics normal dopamine activity (Pothos et al., 1996).

Task

We adapted a probabilistic value learning task (Frank et al., 2004) to include trial unique encoding events in order to get measures of episodic memory and feedback learning from the same task (Fig. 1). We chose this task because it is thought to be sensitive to the influence of dopaminergic medication on learning from positive and negative reinforcement. The task occurred over two sessions, 2 days apart, and comprised a reinforcement learning/object encoding phase and a memory test phase (Fig. 2). Patients tested ON dopaminergic medications were ON for both days and patients tested OFF were OFF for both days so that each participant performed both learning/encoding sessions in the same drug state.

Figure 1.

Experimental design: testing the effect of feedback on episodic memory and stimulus-value memory. Trial sequence of learning task. (A) Participants had to choose one of two abstract stimuli presented in pairs. Each stimulus had a different probability of leading to rewarding feedback ranging from 0.14 to 0.86. To incorporate trial-unique object images presented at the time of feedback meaningfully, we used a task narrative. Participants were told they were going shopping, that the abstract stimuli represented store logos, and that their goal was to accumulate as many objects as possible from a given category (e.g. food). Over time, they would learn which stores were most likely to yield the desired items. At the time of feedback, we presented a trial-unique object from the given category if they were correct, or an object from a different category (e.g. sporting good) if they were incorrect. The object was briefly presented alone, and then positive or negative verbal feedback was additionally presented. No object was presented twice. (B) Both training pairs and novel pairs were presented during the reinforcement learning memory test. Novel pairs were constructed by pairing the best stimulus (0.86) and the worst stimulus (0.14) with all others. (C) During the object recognition memory test, memory for all the trial-unique objects presented during the learning task was tested allowing us to compare memory performance for images encoded in the presence of positive versus negative feedback. (D) During the reinforcement learning memory test participants were presented with the same six stimuli either arranged in the original training pairs or in novel pairs. Performance on ‘Choose the best’ trials reflected memory for learning from positive feedback whereas performance on ‘Avoid the worst’ trials reflected memory for learning from negative feedback.

Figure 2.

Overall study design: testing the effects of dopamine and reward on memory. To compare memory after a short (immediate) and long (2 day) delay, participants performed the reinforcement learning and object encoding task twice, 2 days apart, using non-overlapping ‘store logo’ and object image sets. Memory testing for both learning sessions, and for both the stimulus value memory and the recognition memory, was conducted only in Session 2. This order ensured the surprise element of the test and provided measures of stimulus value and recognition memory after both a short delay and a long delay. During the memory test phase, memory for the reinforcement learning was performed first, with presentation of the short delay store logos (set B) followed by the long delay store logos (set A). This was followed by the object recognition memory test where images belonging to set B (short delay) and set A (long delay) were mixed randomly. Patients in the OFF group were tested OFF for both sessions, and patients in the ON group were tested ON for both sessions.

Feedback learning/object encoding

Three different pairs of abstract stimuli were presented and participants learned to choose one of the two stimuli in the pair based on the feedback provided after each trial (Fig. 1A). Each pair was presented 36 times for a total of 108 trials. Trial order was randomized within blocks of 36 trials, and the side of the screen on which the stimuli of a pair were presented was also randomized. Feedback was probabilistic: the stimuli of the first pair were associated with a probability of receiving rewarding feedback of 0.86 and 0.14, respectively, the stimuli of the second pair were associated with a probability of rewarding feedback of 0.75 and 0.25, and the stimuli of the third pair, the least reliable, were associated a 0.64 and 0.36 probability of reward, respectively. The probabilistic feedback associated with each stimulus pair was counterbalanced across subjects. Rewarding and non-rewarding feedback consisted of the words ‘You won!!’ or ‘Wrong!!’ displayed on the screen. Participants also learned during the practice that each correct response led to a 10-point win. The full point tally was only shown at the very end. At the time of feedback participants also saw an image of an object, which was different on every trial (108 images per learning session). The idea was to associate the exposure of each image with either a rewarding or a non-rewarding event, and later test how this association impacted the likelihood of remembering images. To enhance this association, we developed a task narrative. Participants were told they were going shopping for objects belonging to one of four categories (food, sports, household, kids), that the abstract stimuli were store logos, and that they had to learn which stores were most likely to have that type of object in stock in order to accumulate as many of the right objects as possible. If shopping for food, for instance, after a correct choice a trial-unique image of a food item would appear on screen followed 1.5 s later by the word ‘You won!!’ or, after an incorrect choice, a non-food item followed by the word ‘Wrong!!’ would appear. Critically, participants were unaware they would later be tested on their memory for the stimulus values or for the images. All participants saw images from each category once and category-reward assignments were counterbalanced across participants.

Previous studies using a similar learning task required participants to continue the acquisition task until they reached a learning criterion (Frank et al., 2004). We instead required participants to complete a fixed number of trials because we wanted to expose participants to an equivalent number of feedback images for the subsequent memory test. Because our main analyses focused on the effect of feedback type on memory for the object images and for the reinforcement learning, it was important to exclude participants who did not reach a minimum learning performance, as the reward feedback would not carry the same meaning. We selected only the participants who achieved a minimum of 60% correct choices when faced with the most reliable stimulus pair (i.e. the 0.86/0.14 pair) in the last 30 trials of both acquisition sessions. Twenty of the 25 healthy controls, 16 of the 25 patients ON and 14 of 25 patients OFF reached this criterion. All analyses were performed on this ‘learner’ subgroup of participants (results from the full sample are also shown in the Supplementary material). Furthermore, there was no difference in the proportion of participants reaching the learning criterion between groups (χ² = 2.84, df = 2, P = 0.2414).

Object recognition memory test

Separate surprise tests were administered to test memory for the trial-unique objects and for memory for stimulus-value association from reinforcement learning. To test recognition memory of the objects, participants were shown either one of the objects that had been presented (target), or an entirely new object from the same categories (foil) (Fig. 1C). They had to identify whether the object was ‘Old’ or ‘New’ and were then asked to rate their confidence on a scale of 1 to 5. We are not presenting analysis of the confidence ratings here. Participants were shown an equal number of target and foil objects in random order. Objects were randomly assigned to be target or foils for each participant.

Reinforcement learning memory test

In the reinforcement learning test, participants were shown pairs of stimuli, either in the same combination as during the learning (three ‘training’ pairs, 10 trials each) or in new combinations (eight ‘novel’ pairs, 10 trials each) (Fig. 1B and D). These novel pairs were specifically chosen to allow us to measure the influence of rewarding versus non-rewarding feedback on memory for what was learned. Four of the novel pairs were constructed by pairing the best stimulus (the one with 0.86 probability of being correct) with all others. As in prior studies, choices on these trials were taken to reflect learning to ‘choose the best’, i.e. learning from reward (Frank et al., 2004). The other four novel pairs were constructed by pairing the worst stimulus (the one with 0.14 probability of being correct) with all others; choices on these trials were taken to reflect learning to ‘avoid the worst’, i.e. learning from negative feedback. As before, participants were asked to pick the stimulus they had learned was most likely to be correct but, to avoid the influence of new learning, they received no feedback.

Overall structure

We sought to obtain a measure of memory at two time points—short delay (15–20 min) and long delay (2 days)—and to keep the memory testing a surprise. In order to do so, participants engaged in two separate learning/encoding sessions (with non-overlapping abstract store logos and object image sets), on Days 1 and 3. Memory for both the stimulus-value associations and the object images was tested only on Day 3, thereby providing measures of memory after a short delay (for same-day learning/encoding) and after a long delay (for learning/encoding that took place two days before) (Fig. 2). In total, they learned six pairs of abstract logos (three pairs per session) and encoded 216 object images (108 per session). During the memory test phase, memory for the stimulus-value associations was tested first, and, to minimize interference, the store logos learned that same day were tested first, followed by the store logos learned on Day 1. Recognition memory for the object images was then tested, though here the images encoded on Day 1 and Day 3 were mixed randomly to avoid clustering objects from the same category. Participants were shown a total of 432 images, and within each category (i.e. food, sports, household, kids) the number of old and new images was equal.

Analysis

To compare performance across groups, statistics were computed using mixed-effects logistic and linear regressions with random intercepts and slopes for each within-subject variable grouped by subject, except for the recognition memory where random slopes were not included in the model because there was only one d′ value per condition (R lme4 package) (Bates et al., 2015). In the case of the learning phase of the reinforcement learning task, to compare initial learning from reinforcement between groups, we performed a logistic regression with probability of choosing the optimal stimulus on each trial as the dependent variable, and group, mean-centred block (12 trials per block), and day of testing as independent variables.

In the case of the object recognition memory test, we calculated d′ [z(hit rate) – z(false alarm rate)] to obtain recognition memory scores for each participant as our dependent measure. Because we used image categories (food, household, sports and kids items), both targets and foils could be categorized according to: (i) whether the image category was encoded on the first or second session; and (ii) whether the image was encoded in the presence of positive versus negative feedback. As a result, we could measure the effect of reward and delay on both hit and false alarm rates. In mixed effects linear regressions on d′, group, feedback valence and delay were taken as independent variables. The group × feedback valence × delay interaction was taken as the difference between groups in the effect of feedback valence on the object recognition memory maintenance. We also carried out exploratory analyses on the effect of reward and group on reaction times. We performed a mixed-effects linear regression on log transformed response times, including only correct recognition trials (Hits), with group, feedback valence and delay taken as independent variables.

In the case of the reinforcement learning memory test, we focused only on the trials with novel pairs (80 trials per test session) in order to test memory for the stimulus-value associations, and the influence of rewarding feedback on those memories (Frank et al., 2004). The dependent variable was taken as the probability of making an optimal choice. Group, trial type (choose best versus avoid worst) and delay (short versus long) were taken as independent variables. The group × trial type interaction was taken as the difference in the effect of feedback valence on learning between groups, and the group × trial type × delay interaction was taken as the difference between groups in the effect of feedback valence on maintenance of stimulus value memory.

For all analyses, the three-level categorical group variable was coded using two effect-coded group variables, each with three levels: g1 (1 = ON, 0 = OFF, −1 = Control) and g2 (1 = OFF, 0 = ON, −1 = Control) such that the regression coefficient for g1 represented the difference between ON and the grand mean, and g2 between OFF and the grand mean. In order to test all three possible contrasts between the three groups (ON versus OFF, ON versus healthy controls, OFF versus healthy controls) we used the esticon function in R to compute weighted sums of the relevant coefficients as follows: PD-ON versus PD-OFF  =  βg1 – βg2; PD-ON versus control = 2*βg1 + βg2; PD-OFF versus control  =  βg1 + 2*βg2. We applied the same approach to test the contrasts between the group × variable interactions.

Data availability

We did not obtain consent from participants to share individual data from this study. Summary statistics are available from the corresponding author upon request.

Results

Reinforcement learning

During the initial learning phase, there were no differences in overall accuracy between groups (ON versus OFF P = 0.47; Control versus ON P = 0.28; Control versus OFF P = 0.78) nor in learning rate, which was taken as the interaction between group and block (ON versus OFF P = 0.26; Control versus ON P = 0.92; Control versus OFF P = 0.19) (Supplementary Fig. 1A). The pattern was similar in the full sample: no group differences in overall accuracy (ON versus OFF P = 0.28; Control versus ON P = 0.80; Control versus OFF P = 0.41) nor in learning rate (ON versus OFF P = 0.85; Control versus ON P = 0.20; Control versus OFF P = 0.14) (Supplementary Fig. 1B). The absence of a difference between groups during the initial learning was important as it allowed us to attribute any effects of group on memory to cognitive processes other than the level of initial learning. As expected, there was a main effect of day of testing (β  =  0.31, P < 0.001) indicating that participants performed better on the learning task on the second session, which we accounted for in later analyses (Supplementary Fig. 2).

Influence of reward and dopamine on object recognition memory

We hypothesized that dopamine would support prioritization of memory for images encoded in the presence of positive feedback, and that this dopamine-dependent effect of reward would itself depend on the passage of time, i.e. consolidation. We found the effect of reward on memory to depend on group (ON versus OFF difference estimate = 0.16, P = 0.03; ON versus Control difference estimate = 0.07, P = 0.33; OFF versus Control difference estimate = −0.09, P = 0.19). Specifically, while patients ON had marginally better memory for images encoded in the presence of positive as compared to negative feedback (β  =  0.07, P = 0.093), patients OFF (β = −0.08, P = 0.18) and healthy controls (β  =  0.006, P = 0.89) did not (Fig. 3A). Importantly, there were no group differences in overall memory performance (ON versus OFF difference estimate = −0.10, P = 0.55; ON versus Control difference estimate = −0.10, P = 0.48; OFF versus Control difference estimate = −0.007, P = 0.96). We repeated these analyses in the full sample and, although less reliable, numerically, the general pattern was still observed (ON versus OFF difference estimate for effect of reward on memory = 0.08 P = 0.16; Supplementary Fig. 3). Finally, we also repeated this analysis controlling for motor symptom severity as assessed by the UPDRS scores (averaged across the two sessions) and found that the ON versus OFF difference in the effect of reward on memory remained significant (estimate = 0.077, P = 0.048).

Figure 3.

Effect of reward on object recognition memory is dopamine dependent but not time dependent. (A) Recognition memory for objects encoded on trials where either rewarding or negative feedback was delivered, shown collapsed across the short-delay and long-delay memory tests, for participants who reached the learning criterion on the reinforcement learning task (see Supplementary material for similar pattern in full sample). Lines represent data for individual participants. Memory was better for reward-associated images in patients ON dopaminergic medication compared to patients OFF dopaminergic medications (ON versus OFF difference estimate = 0.16, P = 0.03). There were no differences between groups in overall recognition memory. (B) Recognition memory shown separately for the short and the long-delay memory tests to illustrate the decay across the delay, and grouped according to whether the images were associated with rewarding versus negative feedback, shown for participants who reached the learning criterion. Thick lines represent group averages and thin lines represent individual participants. There was no effect of reward on decay of memory, nor any differences between groups for the effect of reward on memory decay. Error bars represent ±1 standard error of the within group differences. Asterix indicates significant difference (*P < 0.05). HC = healthy controls.

To examine whether the dopamine-dependent effect of reward on memory discriminability was driven by improved recognition of reward-associated images (higher hit rate) versus improved rejection of novel images belonging to the rewarded category (lower false alarm rate), we performed the same analyses on the hit and false alarm rates separately. We found that dopamine was associated with a trend towards increased hit rates for rewarded stimuli (medication × reward interaction estimate = 0.04; P = 0.090) but there was no effect of dopamine and reward on false alarm rates (P = 0.68). We also examined the overall effect of reward on hit and false alarm rates, collapsing across patient group, revealing both higher hit rates (estimate = 0.03, P < 0.001) and false alarm rates (estimate = 0.02, P = 0.002) for the rewarded as compared to not rewarded categories. These findings suggest that dopamine modulates the prioritization of information for entry into memory. Though it also appears that reward led to a criterion shift, causing participants to be more liberal in their recognition as ‘old’ the images belonging to the reward-associated category, this did not differ between groups suggesting that this criterion shift is not dopamine-dependent.

We also explored whether reaction times differed across groups and reward conditions. Participants were overall faster to respond to images of objects previously associated with reward (estimate = −0.01, P < 0.001). Critically, though, there was no difference between patients ON and OFF on the effect of reward (ON versus OFF P = 0.58; ON versus Control P = 0.006; OFF versus Control P = 0.04), nor any overall group differences (ON versus OFF P = 0.08; ON versus Control P = 0.52; OFF versus Control P = 0.22).

Next, we tested whether the dopamine-dependent effect of reward on memory emerged after consolidation, examining group differences in memory maintenance across the delay as a function of feedback valence. We found no group differences in the effect of reward on maintenance (ON versus OFF P = 0.90; ON versus Control P = 0.79; OFF versus Control P = 0.90) (Fig. 3B). Also of interest was the absence of any group differences in overall maintenance of memory (collapsing across reward condition; ON versus OFF P = 0.35; ON versus Control P = 0.59; OFF versus Control P = 0.64).

Taken together these findings indicate that the effect of reward on memory is influenced by dopamine, that it specifically enables prioritization of entry of information into memory, but contrary to predictions, that its effect does not depend on the passage of time to emerge.

Influence of reward and dopamine on reinforcement learning memory

We hypothesized that stimulus-value associations learned from positive, rewarding, feedback would be better remembered than those learned from negative feedback in patients ON compared to OFF medications. We found no main effect of reward, or any difference between groups for the effect of reward on memory for stimulus-value associations (ON versus OFF P = 0.43; ON versus Control P = 0.47; OFF versus Control P = 0.90) (Fig. 4A). As expected, there was a main effect of testing delay such that overall memory for the stimulus-value associations was lower after the delay (β = −0.65, P < 0.00001) (Fig. 4B).

Figure 4.

Memory for stimulus-value associations did not differ based on reward, Parkinson’s disease or dopamine medications. (A) Memory for stimulus-value associations, taken as the performance on the novel pairs, stratified according to whether novel pairs represented learning from rewarding (‘Choose best’) versus non-rewarding feedback (‘Avoid worst’), presented here collapsed across both time-points for participants who reached the learning criterion. Lines represent data for individual participants. There was no significant effect of reward at the time of learning on later memory for stimulus-value associations in any of the groups. (B) Decay in memory for stimulus-value associations across the 2-day delay, presented for stimuli learned from rewarding versus those learned from non-rewarding feedback, for participants who reached the learning criterion. Thick lines represent group averages and thin lines represent individual participants. There was a trend for better maintenance of reward-associated memory in the PD-ON compared to PD-OFF (difference estimate = 0.60 P = 0.072). Error bars represent ±1 standard error of the within group (A), and within group and within condition (B) differences. HC = healthy controls.

We had also hypothesized, as for episodic memory, that effects of dopamine-dependent reward would increase over time. We found some interesting trends: PD-ON compared to PD-OFF showed a trend towards better maintenance of memory across the delay for stimulus-value associations learned from positive rather than negative feedback (difference estimate = 0.60 P = 0.072) and similarly when comparing PD-ON to healthy controls (difference estimate = 0.61 P = 0.047) (Fig. 4B). There were no differences between PD-OFF and healthy controls (P = 0.99).

Given that we identified an improvement in learning from session 1 to session 2, the above analysis was repeated controlling for participants’ average accuracy at each session. The pattern of results did not differ (ON versus OFF P = 0.071; ON versus Control P = 0.047; OFF versus Control P = 0.99). Also of interest was the absence of group differences in overall maintenance of memory for stimulus-value associations (i.e. collapsing across reward condition; ON versus OFF P = 0.37; ON versus Control P = 0.43; OFF versus Control P = 0.85). Reward-dependent differences in memory across groups were not found in the full sample, nor were group differences in overall memory maintenance (Supplementary Fig. 5B). This is not surprising as this sample includes participants who were not able to learn values initially.

Discussion

Altered reward signalling in patients with Parkinson’s disease caused by dopamine deficiency is known to lead to deficits in learning from reward. However, little is known about whether this dopamine-dependent reward processing deficit also affects how patients with Parkinson’s disease prioritize long-term memory for learned information. We found that dopamine replacement was associated with a selective enhancement in the memory for reward-associated images. Importantly, overall episodic memory capacity was not affected by dopaminergic medications or by disease, suggesting that dopamine is more important for adaptive prioritization of ‘what’ is stored in memory rather than ‘how much’ is stored. Contrary to predictions, dopamine status did not influence memory for stimulus-value associations learned during the same task, from the same feedback events. These findings advance our understanding of dopamine’s role across different memory systems and provide insight into the mechanisms underlying early cognitive deficits in patients with Parkinson’s disease. The results also highlight the importance of applying testing that deconstructs cognitive function based on our neurobiological understanding of the underlying brain processes.

Dopamine prioritizes storage of reward-associated information in episodic memory

Our results provide important direct evidence regarding the role of dopamine in mediating reward effects on episodic memory in humans. A dopaminergic mechanism has ample support in rodent studies showing that dopamine acting in the hippocampus improves memory formation (Packard and White, 1991; O’Carroll et al., 2006; Reinholz et al., 2009; Rossato et al., 2009; Bethus et al., 2010; McNamara et al., 2014) and, specifically, mediates novelty and reward-driven memory enhancements (Moncada and Viola, 2007; Wang et al., 2010; Salvetti et al., 2014; Takeuchi et al., 2016). While several functional MRI studies have shown that activation in dopamine-rich regions during encoding (Wittmann et al., 2005; Adcock et al., 2006; Wolosin et al., 2012) and during the immediate post-encoding period (Tompary et al., 2015; Gruber et al., 2016; Murty et al., 2017) is associated with better later memory, only a few studies have directly manipulated dopamine. Pharmacological studies in healthy adults have shown that increasing dopamine transmission enhances memory formation (Zeeuws and Soetens, 2007; Zeeuws et al., 2010; Chowdhury et al., 2012), but that dopamine replacement in Parkinson’s disease may instead interfere with retention of word lists learned over many repetitions (MacDonald et al., 2013; Grogan et al., 2015). However, no studies have specifically investigated how dopamine modulates the effect of reward on memory in humans. Our results therefore provide the first direct evidence in humans that dopamine, present at the time of encoding, is necessary for the process of reward-guided memory prioritization.

Our experimental design aimed to address how the brain sorts through continuous experience to prioritize storage of only a portion of those experiences in memory, without knowing what might be useful in the future. Participants were not aware that their memory would be tested, were not instructed to prioritize reward-related images, and images acquired reward-related information simply by being presented at the time of positive or negative feedback during the learning task. Thus, our task does not demand the use of complex strategies for memory and instead focusses on benefits that are more automatically conveyed by reward associations. This approach complements existing studies that have examined how reward prioritizes memory using motivated memory paradigms in which participants are instructed to strategically prioritize memory for certain stimuli while forming memories to increase their earnings (Adcock et al., 2006; Callan and Schweighofer, 2008; Wolosin et al., 2012; Feld et al., 2014; Murty et al., 2017).

The precise mechanism by which dopamine mediates the effects of reward on episodic memory in our task is not known. The pattern of findings suggests a few potential mechanisms. First, the reward experienced at the time of encoding was induced by a reinforcement learning task that is known to depend on dopaminergic striatal reward prediction errors. It is therefore possible that the dopaminergic projections to the hippocampus also carry information about reward prediction errors (Jang et al., 2019). Alternatively, it is also possible that dopaminergic projections to the striatum and those to the hippocampus convey parallel but distinct signals. Indeed, recent evidence suggests that hippocampal dopamine originates from the locus coeruleus, a region that also suffers neuronal loss in Parkinson’s disease, and encodes novelty (Kempadoo et al., 2016; Takeuchi et al., 2016; Duszkiewicz et al., 2019). Our task was not designed to distinguish novelty from reward, though both images shown at the time of negative and positive feedback were novel. Additionally, it is arguable whether and to what extent the feedback events are truly novel for participants given the repetitive nature of the task and the fact that we selected participants who showed evidence of learning. Another possibility is that reward enhanced attention at the time of positive feedback, resulting in more attentive viewing of the images, and thereby enhancing their later memory. Indeed, there is evidence that reward guides attention allocation in an automatic way, even when the reward is not relevant to the task at hand (Anderson et al., 2011; Hickey et al., 2015), and that it increases the representation of reward-associated stimuli in cortical sensory areas (Schiffer et al., 2014). Similarly, there is evidence that reward guides the formation of attentional sets, leading to attentional biases, which are exaggerated in patients with Parkinson’s disease and cause perseveration towards previously rewarded features (Owen et al., 1993; Fallon et al., 2016), though how dopaminergic medications modulate this effect is less clear (Owen et al., 1993; Moustafa et al., 2008). Future studies are needed to dissect these possibilities. In particular, the combination of behaviour with functional MRI may help determine the localization of the effects of reward and possible interacting effects of reward and dopamine on both the striatum and the hippocampus. Finally, physiological measures of attention such as eye tracking, and neuroimaging measures of stimulus processing may also help delineate the relationship between reward, attention and subsequent memory (Theeuwes and Belopolsky, 2012; Schiffer et al., 2014).

It is also important to note that we did not find an effect of reward on memory in healthy older adults. One possibility is that this is due to age-related loss of midbrain dopamine resulting in a reduced sensitivity to reward-related benefits to memory. Indeed, age-related dopamine decline is well characterized in healthy ageing and is known to impact behaviour (Fearnley and Lees, 1991; Chowdhury et al., 2012, 2013). An alternative possibility is that the observed effects of dopamine on reward-associated memory in the patients are due to an ‘overdose’ from the medication (Fallon et al., 2015). We think this latter possibility is less likely because both rodent studies showing a dopamine-related enhancement in hippocampal memory (O’Carroll et al., 2006; Rossato et al., 2009; Bethus et al., 2010), and human studies showing reward-related memory prioritization, have relied on endogenous dopamine levels (Adcock et al., 2006; Wolosin et al., 2012; Wittmann et al., 2013; Gruber et al., 2016; Patil et al., 2017; Braun et al., 2018).

Influence of dopamine on both short-term and long-term episodic memory

In our study, the interaction between dopaminergic medication and reward did not depend on the passage of time: better memory for reward-associated images in patients ON dopaminergic medications was observed at both the short (∼15–20 min) and long delays (2 days). By contrast, most rodent work shows that the effects of dopamine are not observed until the late-phase of long-term potentiation is expressed (O’Carroll et al., 2006; Bethus et al., 2010; McNamara et al., 2014; Takeuchi et al., 2016). In humans, evidence for delay-dependence is more mixed with evidence that reward or novelty exposure can lead to improved memory even when memory is tested immediately (Callan and Schweighofer, 2008; Fenker et al., 2008; Wolosin et al., 2012; Jang et al., 2019) whereas others have shown that the effect emerges only after the passage of time (typically 24 h) (Wittmann et al., 2005; Murayama and Kitagami, 2014; Patil et al., 2017; Braun et al., 2018), and others still did not include a shorter delay test for comparison (Krebs et al., 2009; Wittmann et al., 2013; Murty et al., 2017). One explanation for these reported discrepancies is that separate mechanisms support the immediate and delayed effects of dopamine and reward. While dopamine may indeed influence human long-term memory in a time-dependent manner, its simultaneous facilitation of selective attention and/or working memory (Anderson et al., 2011; Chatham et al., 2014; Hickey et al., 2015) could support memory formation more generally, through effects outside the hippocampus. Animal studies that manipulate dopamine transmission only in the hippocampus might therefore not detect the more immediate effects of dopamine on memory.

Another possible explanation for the dopamine-dependent reward effect seen after both short and long delays could be that it acts on retrieval instead of (or in addition to) memory formation/consolidation (Scimeca and Badre, 2012). However, evidence of retrieval deficits in patients with Parkinson’s disease is very mixed (Davidson, 2006; Cohn et al., 2010; Edelstyn et al., 2011, 2015; Foerde et al., 2013). Two studies have separately manipulated dopamine state at encoding and retrieval—one found no evidence of retrieval-specific deficits (Grogan et al., 2015), whereas the other found improved retrieval ON dopaminergic medications (MacDonald et al., 2013). In our results, overall recognition memory performance was equivalent across groups, suggesting that effects on the retrieval process as a whole cannot explain the group differences. Furthermore, at the time of retrieval, the reward information was no longer relevant to the task (no reward was delivered during the recognition memory test), and therefore was not likely to selectively modulate reward-associated stimuli at retrieval. Overall, we propose that the most plausible account for our results is that dopamine reward signals influence both encoding and post-encoding processes leading to a prioritization of reward-associated content for entry into short-term memory and for consolidation into long-term memory. However, as with any negative result, the lack of a difference for the effect of dopamine on the short versus the long-delay memory performance must be interpreted with caution. Indeed, because our final sample only provided power to detect large effect sizes, future studies are needed to address the possibility that this finding represents a false negative. It is possible that a more precise test of memory (such as one using a continuous measure of recognition) could have increased our ability to detect changes across the delay but such a measure, because of increased cognitive demands, might have introduced unintended differences between our groups (Sun et al., 2017).

No influence of feedback valence and dopamine on memory for stimulus-value associations

Little is known about the factors that modulate long-term memory for stimulus-value associations, and whether factors such as reward surrounding the initial learning—already known to influence episodic memory—similarly influence memory for reinforcement learning. Recent studies in non-human primates have shown that a subset of midbrain dopamine neurons, as well as the caudate region to which they project, maintain stable representations of learned stimulus values over time (Kim and Hikosaka, 2013, 2015; Kim et al., 2015; Ghazizadeh et al., 2018). Though these findings would suggest a direct role of dopamine reward signalling in the striatum for long-term memory of value, we did not find an effect of feedback valence nor an interaction between valence and dopaminergic medications on stimulus value memory in the patients with Parkinson’s disease. One explanation for this discrepancy is that the above studies did not lesion the basal ganglia nor directly manipulate dopamine levels, thus whether dopamine is necessary for long-term memory of value is not clear. It is also important to note that whether feedback valence influences the selective stability of memory for stimulus-value associations has not, to our knowledge, been previously investigated in humans. Indeed, we did not find such an effect in the older adults. It is therefore possible that reward at the time of learning does not influence long-term memory for stimulus-value associations even in healthy individuals. Because ageing has also been associated with a reduction in striatal dopamine, a similar experiment will need to be performed in young adults to address this question.

We also did not find an effect of dopamine on overall memory for stimulus-value associations (i.e. collapsing across both the stimulus-value associations learned from positive and negative feedback). Only two previous studies have investigated the effect of dopamine on memory for stimulus-value associations, one after a 20-min delay and the other after 24 h (Coulthard et al., 2012; Grogan et al., 2017). Though neither of these studies addressed the specific question of the role of rewarding versus negative feedback on memory stability over time, they did find that dopaminergic medications were associated with improved memory for overall learning, which we did not find. One explanation for this discrepancy is that our memory test relied on the novel pairs, which is thought to be a better assessment of stored learned values (Frank et al., 2004). In contrast, memory tested by presenting the stimuli in the same context as was used during the initial learning (e.g. using the training pairs) could potentially reflect non-striatal memory mechanisms since good performance can be achieved even without successfully learning the stimulus values. Previous studies that have shown an isolated effect of dopamine on the performance during the test (rather than the initial learning) could also be interpreted as evidence for an effect of dopamine on memory, but because these studies only tested memory immediately after learning, they do not address the question of longer-term memory or consolidation (Shiner et al., 2012; Smittenaar et al., 2012). Finally, as discussed above, these null effects must be interpreted with caution because our final sample only provided power to detect large effect sizes. Given this, and given the difficulty experienced by participants in the initial learning phase known to be an issue with this sort of task, it will be important for future research to replicate these results using a simpler task that measures memory for reinforcement learning, and the influence of reward, in isolation from other memory processes [Collins and Frank, 2012; National Advisory Mental Health Council Workgroup on Tasks and Measures for Research Domain Criteria (RDoC), 2016].

Conclusions

We found that patients with Parkinson’s disease are impaired at prioritizing what to remember, rather than how much they remember, that this deficit is dopamine-dependent, and, contrary to predictions, that this deficit is not dependent on the passage of time. The neurobiologically-driven design of the task and procedures, which focused on dopamine’s known role in signalling reward, allowed us to provide a more fine-grained view of memory deficits than can be detected using standard neuropsychological testing, and to provide insight into the underlying mechanism. More broadly, these results also point to a plausible mechanism for the early cognitive deficits in Parkinson’s disease. Our finding of a reduction in reward-driven prioritization of information for both short-term and long-term memory storage could be one manifestation of a system-wide reduction in the ability to use environmental signals to guide the prioritization of information for processing. Indeed, recent work also points to the role of the striatum and of reward-dependent prioritization mechanisms for both attention and working memory (Anderson et al., 2011, 2017; Hickey et al., 2015; Klink et al., 2017), two important areas of early deficit in Parkinson’s disease (Kudlicka et al., 2011; Robbins and Cools, 2014). Future work will be required to determine whether other reward-related prioritization failures exist in patients with Parkinson’s disease as this would present a new and potentially impactful cognitive process that could readily be targeted through behavioural interventions. Furthermore, given evidence that the low dopamine state of patients with Parkinson’s disease is associated with increased sensitivity to punishments, which we did not directly test with our task, it will be interesting to explore whether punishments also influence prioritization of memory storage and other processes in a dopamine-dependent way.