Abstract
The neurotransmitter dopamine is crucial for decision-making under uncertainty but its computational role is still a subject of intense debate. To test potential roles, we had patients with Parkinson’s disease (PD), who have less internally-generated dopamine, participate in a visual decision-making task in which uncertainty in both prior and current sensory information was varied and where behavior is often predicted by Bayesian statistics. We found that many aspects of uncertainty processing were conserved in PD: they could learn the prior uncertainty and utilize both priors and current sensory information. As predicted by prominent theories, we found that dopaminergic medication influenced the weight given to sensory information. However, as PD patients learn, this bias disappeared. In addition, throughout the experiment the patients exhibited lower sensitivity to current sensory uncertainty. Our results provide empirical evidence for the idea that dopamine levels, which are affected by PD and the drugs used for its treatment, influence the reliance on new information.
Every day we face decisions that have associated uncertainty. They may range from very important, life-changing situations (should I marry this person? Which career should I choose?), to minor decisions that we make, sometimes without even noticing we are doing them (where is my wallet? What is that object in the sky?). When making a decision, we combine information from the past, called prior (e.g. how trustworthy has this person shown to be? What types of objects generally appear in the sky?), with current sensory information, or likelihood (what am I feeling now? What is the shape of that object?). Correctly combining these pieces of information is the key to effective decision-making.
To understand how people (and their brains) are making these decisions, we can use a normative approach, i.e. ask what people should be doing if they were to optimally compute information, and then compare it with what they are actually doing. Bayesian theory gives us such a framework. Specifically, it tells us that the statistically optimal way to make decisions based on uncertain information is to use both prior and current sensory information and, moreover, combine them according to their respective relative uncertainties, relying more on current information when prior information is more uncertain, and vice-versa1,2 . One may then expect that, if someone has a more uncertain prior (e.g. because they have trouble storing and keeping prior information), or a more uncertain likelihood (e.g. due to perception problems), this would result in an over-reliance or under-reliance on current information. In this way, the Bayesian framework can give us insights into what specific information (or lack thereof) a person has, and allow characterizing apparent abnormal behavior under a normative light3.
As uncertainty processing and appropriate weighing of prior and current information is essential for decision-making, understanding how the brain performs these calculations seems of extreme importance. Previous research in the lab has indicated that the putamen was particularly important in this process, with activity from the putamen correlating with both increased prior uncertainty and also with individual’s tendencies to sense and attend to current vs. prior information4. However, from the fMRI data alone it is not possible to know in which of these processes putamen activity could have a causal role. Better understanding the role of putamen may shed light on how the brain performs the computations necessary for decision-making under uncertainty.
Existing literature shows that activity from the putamen either directly or indirectly affects decision-making under uncertainty. For example, putamen activity has been associated with ambiguity5, and also learning6,7. Interestingly, when learning a motor sequence putamen activity decreases after training, but remains high if the sequence is random6,7. We could interpret these studies as putamen activity signaling for prior uncertainty, which decreases with learning but keeps being high if the sequence is random or the probabilities of events are unknown. Another interpretation is that it promotes focusing on current incoming information, which would equally predict the observed data. As activity from the putamen is severely compromised in people that suffer from Parkinson’s disease8 we can study Parkinson’s disease (PD) patients to ask fundamental questions about the role of the putamen in decision-making.
PD leads to a depletion of dopamine8, one of the main transmitters of the putamen, and this neurotransmitter probably mediates at least some of putamen’s influence on decision-making under uncertainty. Indeed, dopamine has been associated with uncertainty9–11. However, its precise role is still unclear and subject to intense debate10,12,13. Besides its most known role in reward prediction error14,15, it has been proposed that dopamine is involved in attention and saliency of a stimulus12,16,17. It has also been proposed that dopamine regulates the weight given to bottom-up current sensory information (likelihood) vs. top-down prior beliefs10,11,18,19, and that it codes for the uncertainty in the current stimulus10,11,18. Furthermore, dopamine has been implicated in learning17,20,21, and hence could be crucial for learning the uncertainty of a given event. To distinguish between these ideas we can compare PD patients with healthy participants on a task that allows disambiguating different aspects of decision-making under uncertainty4.
Previous studies have shown that PD patients have impairments in decision-making. They have trouble at reacting to unexpected events18, task switching22 and using negative feedback23. It seems reasonable to assume that some of these deficits may result from the wrong processing of uncertainty information, e.g. diminished capacity to attend to external cues. Studying how PD patients learn and deal with uncertainty in prior and current sensory information may help in understanding the specific challenges PD patients face when making decisions.
Here, we had PD patients and age-and-gender-matched controls perform a decision-making task in which both prior and current uncertainty were independently varied. Patients performed the task once “on” dopaminergic medication (more dopamine in their system) and once “off” medication (less dopamine). We hypothesized that, if the putamen directly stores prior uncertainty information, PD patients would have trouble reacting to the different prior uncertainties. Alternatively, if dopaminergic activity mediated by the putamen signals the weight given to current vs. prior information, then we would expect PD patients off-medication to rely less on current sensory information. Finally, if dopamine is directly involved in signaling the uncertainty in current sensory information10,11,18, we may expect PD patients to react similarly to sensory stimulus with different uncertainties. By studying how PD patients and controls reacted to the changes in uncertainty, we could independently analyze these hypotheses.
Participants performed a visual decision-making task, where they guessed the position of a hidden target (“coin”) on a screen (see Fig. 1a). They received noisy (uncertain) visual information about the position of the target in the form of a cloud of dots centered on the true target position. To estimate accurately the target position, participants could use both prior information (the prior), obtained from the distribution of previous target positions, and current sensory information (the likelihood), obtained from the displayed cloud of dots (see Methods for details). The conditions comprised a two-by-two factorial design (Fig. 1b), with two levels of prior uncertainty (wide, more uncertain prior: P; and narrow, less uncertain prior: p) and two levels of likelihood uncertainty (wide, more uncertain likelihood: L; and narrow likelihood: l), for a total of 4 conditions (pl, pL, Pl and PL). Participants performed the task twice, with PD patients performing it once on-medication (shortly after taking their dopaminergic medication) and once off-medication (dopaminergic medication withdrawn overnight), counterbalanced across participants. By varying prior and current sensory uncertainty and comparing PD patients at different dopaminergic medication levels and with healthy controls, we could study the specific effects of the disease and of dopaminergic medication in decision-making under uncertainty.
To analyze participants’ reaction to changes in uncertainty in both prior and current sensory information (likelihood) we can make, for each condition, a linear regression of the participant’s estimated coin position at each trial as a function of the centroid of the cloud of dots, i.e. as a function of the mean of the current sensory information (see Fig. 2a and Methods). The slope of this regression represents then the relative weight on current sensory information, which we will call the sensory weight. If people rely exclusively on current information, then this slope/sensory weight should be one, and if they completely ignore current sensory information (e.g. rely only on prior knowledge), then this weight should be zero (Fig. 2a). Note that, if we assume PD patients are using only prior or current sensory information, then the weight on prior information is just 1 - sensory weight2. Intuitively, and if participants behave according to the statistical optimum predicted by Bayesian statistics, the sensory weight should be higher when they think the current information is more reliable and when their prior information is more uncertain2. Hence, if and how the value of this weight changes according to the uncertainty in current and prior information can serve as a proxy to tell us if participants are detecting and reacting to changes in uncertainty.
The sensory weights within the PD population are significantly affected by both prior uncertainty and likelihood uncertainty (F1,101=17.51,p-val<0.0001 for main effect of prior, P>p, [0.055-0.153] 95% confidence interval for mean difference (CI); and F1,101=7.64,p-val=0.007 for main effect of likelihood uncertainty, l>L,[0.019-0.118] 95%CI; repeated-measures ANOVA with prior, likelihood, session and medication type as fixed factors; Fig. 2b). PD patients showed increased sensory weights when the current information (likelihood) was more reliable and also when the prior was more uncertain (compare, in Fig. 2b, the average weights in less uncertain (l) vs. more uncertain (L) likelihood conditions; and when the prior was more uncertain (P) vs. more reliable (p)). Furthermore, the weights were highest in the Pl condition, when the prior was more uncertain and the likelihood more reliable, and lowest in the pL condition, where the opposite occurred. This indicates that they noticed differences in both prior and current sensory uncertainty and, moreover, they reacted to them in a way qualitatively predicted by Bayesian statistics2,24 (see Supplementary Information (SI) and Supplementary Fig. 1 for quantitative predictions). Thus, in our experiment, PD patients were able to detect changes in both prior and current sensory uncertainty and modify their behavior accordingly.
To see the effects of dopaminergic medication, we compared PD patients’ weights when they were “on” their medication (i.e. shortly after having taken their medication; PD-on) with their weights when they were “off” their medication (i.e. after being at least 12 hours without taking medication; PD-off). We made this comparison within-participant so to directly analyze the effect of medication independent of participant-specific effects, and we counterbalanced the order by which participants started the experiment (on vs. off) in order to distinguish the effects of medication from the session effects. We found that PD patients, on average, had lower sensory weights when they were off their medication compared to when they were on-medication (F1,101=4.16,p-val=0.044 for main effect of medication, on>off,[0.0014-0.0998] 95%CI; same ANOVA as above; compare the slopes in Fig. 2a; see also Fig. 2b-d). Note that this effect of medication is independent of any session effects. For six of the PD patients the increase in weights with dopaminergic medication was significant even at the individual level. Administration of dopaminergic medication resulted in PD patients, on average, placing an increased weight on current information.
The results also revealed session effects. As stated above, participants performed the task twice, with patients performing it once on-medication and once off-medication (order randomized). For the PD population, session number had a significant main effect on the average weight given to current information (F1,101=15.59,p-val=0.0001; 2>1,[0.049,0.147] 95%CI; Fig. 2). Analyzing each session independently, we observe that in the first session PD patients off-medication had lower average sensory weights compared to patients on-medication (t13=2.2,p-val=0.046,[0.004,0.44] 95%CI,unpaired t-test; Fig. 2c,d), but this difference disappeared at the second session (t13=-0.78,p-val=0.45,[-0.456,0.214] 95%CI, unpaired t-test). Note that this is a between-participants effect. To analyze better potential session effects, we calculated the starting and ending weights for each session (obtained using the first and last 15 trials of each condition, respectively). For the initial weights (first session) there was a striking difference between PD patients that started off-medication vs. on-medication, with PD patients off-medication starting with significantly lower sensory weights (U8,7=50,p-val=0.009, Mann-Whitney U-Test; see SI and Supplementary Fig. 2 for more in-depth analyses of changes in weights across time). At the end of the first session this difference shortened, with patients off-medication still with lower weights but not significantly so (U8,7=42,p-val=0.12, Mann-Whitney U-Test). In the second session, PD patients that were then off-medication had starting weights even somewhat higher than patients on-medication, although not significantly so (U7,8=18,p-val=0.28, Mann-Whitney U-Test). Note that the PD patients off-medication in the second session were the ones on-medication in the first session, suggesting a time/experience effect. This session effect may also help explain the smaller influence of dopaminergic medication in some participants. Thus, although at the start of the experiment PD patients off-medication placed significantly lower weight on current information relative to patients on-medication, experience with the task made this difference disappear.
If dopamine signaling from the putamen mediates the relative weight given to sensory information (as suggested by the observed increased weight on senses while on-medication), then we may see a relation between the effect of dopaminergic medication and how long the person has been diagnosed with PD. Research has shown that while the early stages of PD have specific localized loss of dopaminergic neurons in the putamen, as the disease progresses the loss becomes more generalized, starts affecting other areas in the brain and even also other neurotransmitter/neuromodulator pathways8. Thus, one may expect dopaminergic medication to be particularly effective at the early stages of PD8,25. To test this, we analyzed the difference in weights when the patients were on vs. off medication and correlated it with the time since their PD diagnosis. Time since diagnosis (in months) was, indeed, strongly negatively correlated with how much patients changed their average weights between the on and off medication states (r=-0.77,p-val=0.002, spearman correlation). This seems a specific effect of time since diagnosis and not age, because age was not significantly correlated with change in weights (r=-0.31,p-val=0.27, spearman correlation). These results suggest that, as the disease advances, dopaminergic medication may become ineffective in producing a change in behavior.
To study the specific effects of the disease, we can compare the behavior of PD patients in this task to the behavior of age-and-gender-matched controls (see SI for behavioral analyses within controls). If we use the average weights of both sessions and medication states, no significant difference is found between patients and controls (F1,120=0.21,p-val=0.65 for main effect of population type,[-0.198,0.126] 95%CI; repeated-measures ANOVA with population, prior and likelihood type as fixed factors; Fig. 2). However, we saw that both session number and medication state affected these weights, thus we decided to look at the starting weights for the first session and analyze separately PD patients on-medication and off-medication. The starting weights of PD patients off-medication were significantly lower than the ones of controls (U7,15=18,p-val=0.017, Mann-Whitney U-Test; similar to Fig. 2c,d), while there was no significant difference between the weights of PD patients on-medication and controls (U8,15=59,p-val=0.97, Mann-Whitney U-Test). Together, our results show that PD patients off-medication started the task relying less on current information (when compared with patients on-medication or controls), and that this changed with experience with the task.
Analyzing the interactions between population type (patients or controls) and prior and likelihood effects can give us an idea if PD patients and controls react similarly to changes in prior and likelihood uncertainty. There was no significant interaction between population type and reactions to prior (F1,120=1.02,p-val=0.32 for prior and population interaction term, same ANOVA as above; see SI for additional analyses on reactions to prior uncertainty). In contrast, we found a significant interaction effect between population type and reactions to likelihood (F1,120=8.44,p-val=0.007). These results suggest that PD patients and controls react similarly to changes in prior uncertainty, but differently to changes in current sensory uncertainty (likelihood uncertainty).
To understand better how PD patients differ from controls in their reaction to changes in likelihood uncertainty we analyzed directly their “sensitivity to likelihood uncertainty”, i.e. the difference in weights between conditions of different likelihood uncertainty. Medication state had no significant effect on the average sensitivity to likelihood (t14=0.776,p-val=0.45; [-0.052,0.111] 95%CI, paired t-test; Fig. 3), hence we combined the data from PD patients on and off-medication. PD patients reacted significantly less to changes in likelihood uncertainty compared to controls (F1,29=8.44,p-val=0.007; [0.023,0.134] 95%CI, repeated-measures ANOVA with population and session as fixed factors), while session number had no significant effect (F1,29=1.35,p-val=0.25; [-0.023,0.086] 95%CI, see SI for control analyses ruling out motoric/visual confounds). When the prior was narrow (more precise), the difference between PD patients and controls was salient: the change in likelihood uncertainty did not significantly change PD patients’ behavior (t14=1.33,p-val=0.20, [-0.025,0.106] 95% CI for mean change, one-sample t-test; Fig. 2), while controls showed a solid effect (t14=5.01,p-val=0.0002, [0.077,0.191] 95%CI,one-sample t-test). When the prior was wide (more uncertain), both groups showed an effect of likelihood uncertainty (t14=4.45,p-val=0.0005, [0.05,0.142] 95%CI and t14=7.17,p-val<10-5, [0.112,0.208] 95%CI for patients and controls, respectively, one-sample t-test), having higher sensory weights when sensory information was more precise. Together, our results show that PD patients were less responsive to changes in the uncertainty of the current stimulus compared to controls, and this was particularly evident when the prior was more certain.
So far we have looked at participants’ average behavior, but analyzing how participants react to trial-by-trial variations in sensory uncertainty gives us a more fine-grained understanding of their behavior. If PD patients can react to fast changes in sensory uncertainty and behave according to the predictions of a statistically optimal observer, then we would expect a negative correlation between the specific sensory uncertainty associated with a given trial and the participant’s weight on senses on that trial. This is what we observed. Both PD patients and controls showed consistently negative correlations (albeit of small amplitude) between the standard deviation of a trial’s cloud of dots and the weight they placed on senses on that trial (correlations consistently lower than 0, t14=-6.05,p-val=0.00003, [-0.091, -0.044] 95%CI for correlations in PD patients; t14=-6.61,p-val=0.00001, [-0.117,-0.06] 95%CI for Controls; one-sample t-tests over Pearson’s correlation coefficients; see SI; Fig. 4). However, PD patients off-medication showed weaker correlations compared to controls (t28=2.18,p-val=0.038, unpaired t-test over mean Pearson’s correlations values across groups; [ 0.002, 0.076] 95%CI for mean difference between correlations), while there was no significant difference between PD patients on-medication and controls (t28=0.10,p-val=0.92, [ -0.043, 0.048] 95%CI, unpaired t-test; Fig. 4). Furthermore, if dividing per prior block, we find that PD patients showed stronger negative correlations in the block with higher prior uncertainty than in the block with lower prior uncertainty (t14=-2.6,p-val=0.021, [-0.093, -0.009] 95%CI, paired t-test). These results indicate that PD patients detected trial-by-trial changes in current sensory uncertainty and responded appropriately, but this trial-by-trial reaction was weaker when the prior was more certain or when patients were off-medication.
See SI for additional analyses.
Our goal was to understand how dopamine and Parkinson’s disease affect decision-making under uncertainty. Namely, we wanted to know how PD patients, who have deficits in dopaminergic transmission (specially from the putamen), reacted to changes in prior and current sensory uncertainty, what was the relative weight they placed in each of these two pieces of information, and what was the effect of dopaminergic medication on these behaviors.
We found that PD patients were able to react to changes in both prior and current sensory uncertainty, and they did so in a way predicted by Bayesian statistics. We also found that dopaminergic medication affected the weight given to current vs. prior information, with PD patients off-medication relying less on current information compared to patients on-medication (within and between participant effects) or age-matched controls. Finally, we discovered that although PD patients could still learn the different priors to the same level as controls, they were impaired at reacting to changes in sensory uncertainty.
If dopaminergic transmission from the putamen directly signaled prior uncertainty, then we would have expected PD patients to be impaired at learning/adapting to the prior uncertainty. However, PD patients off-medication were still able to learn the prior uncertainty as well as controls. Dopamine and the putamen have been previously implicated in learning of some tasks17,20,21. Notably, however, they are not required for learning of all types of tasks. For example, in a study with PD patients, it was found that dopaminergic drug state (on vs. off) affected performance but not learning26. Dopaminergic medication increased learning in some tasks but decreased learning in others21,27. It has been proposed that dopaminergic medication may sometimes increase tonic dopamine levels in unaffected areas (“overloading” them), preventing phasic dips from being effective and so decrease performance in tasks that require those areas27. Thus, although PD patients have dopamine deficits that affect transmission from the putamen, patients in our task could still adapt to the different prior uncertainties, suggesting that the prior uncertainty information participants used did not come from there.
The finding that PD patients were still able to learn and use the prior uncertainty provides mixed support to the results of a recent paper studying PD patients in a binary discrimination task28. In their study, PD patients were not able to use prior information, but still seemed somewhat sensitive to it. Although they did not study the specific effects of prior uncertainty or of dopamine medication (all PD patients were on-medication), theirs and ours study reveal a potentially interesting dissociation between learning and using the uncertainty of the prior (as in our task) and using the prior in a binary discrimination task (where the prior’s mean and variance cannot be dissociated). This dissociation is interesting as it suggests that PD may affect the learning and use of the mean or the variance of the prior differently. Previous studies have found that the prior’s mean and variance can be encoded separately4,29 and that they generalize differently30. Alternatively, it may be that the use of the prior is affected in a visual discrimination task (as in 28) but not in a estimation task (as ours). Follow up studies to understand if PD patients react to the mean and variance of the prior in a different way, and how their performance may be differently affected in a discrimination or an estimation task, would be very interesting.
PD patients off-medication started with a lower average weight on current information compared to controls. We found both a within-participant and a between-participant effect of dopaminergic medication on the weight given to sensory information, with administration of dopaminergic medication resulting on an increase in the sensory weight. This effect was strongest in early-stage PD patients, who have a more specific localized loss of dopaminergic neurons in the putamen. These results are in accordance with theories suggesting that dopamine affects the relative weight given to current bottom-up sensory evidence (likelihood) vs. top-down priors10,18,19, and indicate that dopaminergic medication (and potentially dopamine itself) can increase the relative weight given to current information.
PD patients reacted to changes in the uncertainty of current sensory information, but less so than controls. Furthermore, this lower sensitivity was particularly pronounced in the context of a more certain prior. These results agree with the results obtained in Galea and colleagues18, which showed that PD patients were slower to change behavior in response to an improbable stimulus when this stimulus was delivered in the context of an overall predictable sequence (i.e. a more certain prior), but not when the sequence was unpredictable (uncertain prior)18. Altogether, these results are compatible with a role of dopamine in directing attention to current stimuli12,16,17. Increased attention would both give current sensory information more weight in general, and also enable the animal to better detect its uncertainty, and hence respond appropriately to it. Furthermore, this could explain the asymmetry observed: a more uncertain prior may be in itself a drive for increased attention to current stimuli, and hence somewhat compensate for the patients’ general lack of attention to it. However, to really know what dopamine is coding for one should directly record dopaminergic activity9,15,31. Future studies with direct neural recordings, for example in PD patients undergoing deep brain stimulation surgery31, could measure the specific response of dopaminergic neurons to changes in current sensory uncertainty to verify if the observed lower sensitivity could be indeed due to dopamine promoting attention to current information and/or directly encoding its uncertainty10,18,19.
We cannot be sure that the cause for the behavioral differences we observed in PD patients were specifically related to the putamen, as transmission from other structures is also affected in PD. Nevertheless, the putamen seems to be the brain structure most severely compromised in Parkinson’s disease8, and so it is likely that the observed effects are at least partly due to a deficient dopaminergic transmission from the putamen. Furthermore, as referred, we saw significant effects of dopaminergic medication on the sensory weights of early-stage PD patients, whose deficits are more localized to the putamen8. It would be interesting to perform a similar study with other patient populations that have deficits in transmission from the putamen, and see if similar behaviors are observed.
We found that PD patient’s behavior could be well captured by a Bayesian framework. However, quantitatively the weights obtained, both for patients and controls, were not exactly the “statistically optimal” weights assuming the imposed variances. This could be because PD patients (and controls) were “irrational”, i.e. they had all the information available (including the imposed uncertainties) but still did not behave in a statistically optimal way. However, it is also possible (and, in our view, likely) that the participants’ subjective prior and likelihood variances were not identical to the imposed ones, and this could explain the observed behavior. The participants’ subjective variances are unobservable (at least behaviorally), and hence can only be inferred. The goal of Bayesian statistics, in this context, is not to say that people are “rational” or “irrational”, but instead to give a framework by which to understand the subjective information people are using in perception and decision-making. Adopting a Bayesian framework allowed us to find that PD can affect the sensitivity to likelihood uncertainty and the relative weight on senses, and promises to explain decision-making deficits in PD under a normative light 22,23 ,
While some patients in our experiment were taking as dopaminergic medication only levodopa, others were also taking dopamine agonists or Monoamine oxidase B inhibitors, which have different effects on phasic and tonic dopamine and different decay times25. It could well be that if our sample was more homogeneous we would have seen even stronger medication effects. For example, increased tonic dopamine levels could have prevented us from seeing a stronger phasic response on uncertainty processing38. Here we looked at the general effect of medication, but future studies could analyze the specific effect of different types of medication on these behaviors.
In our task, although we made sure that all participants had enough cognitive ability to understand our task, we did not explicitly assessed IQ, years of education or income. Hence, we cannot be sure if the differences we found between groups (e.g. controls and PD patients) were related with the disease or with general differences in socioeconomic stats (SES). Nevertheless, the healthy controls recruited were mainly spouses or other close family members, who we assume would naturally match for most SES measures. Most of our results should not suffer from this problem, as they are within-participant comparisons. Future studies could look if the results we found within our PD group generalize to other tasks and groups.
The neural signaling of uncertainty is among the central topics in computational neuroscience1,32. However, preciously little is known about the way the brain actually solves those problems. There exists a growing literature using fMRI studies asking how neural activities change with uncertainty4,5,33, but these studies cannot directly address causal roles of brain areas or neurotransmitters. Here we have used a different approach and started with a patient population that should, according to multiple theories, have deficits in the signaling of uncertainty. Our findings of retained prior uncertainty learning, a lower reliance on current information in PD patients’ off-medication and a lower sensitivity to changes in current sensory uncertainty, promise to inform the development of new theories about the representation of uncertainty.
Methods
Participants
In total, 15 PD patients (9 women; 62.8±9.6 years old) and 15 age-matched controls (8 women; 63.4±11.6 years) participated in the experiment. Written informed consent was obtained for all participants. All protocols were approved by the Northwestern University IRB. Additional details, including Inclusion and Exclusion Criteria, can be found in the Supplementary Methods.
General procedure
PD patients completed two test sessions, once about 1 hour after they had taken their regular dopaminergic medication (on-medication) and once after overnight withdrawal from dopaminergic medication (off-medication). Session order was counterbalanced across patients (randomly), so in total eight patients started on-medication, and seven started off-medication. Control participants also completed two test sessions, and the results reported for controls were the average of the results obtained in the two sessions.
Coin-catching task
Participants performed a decision-making task in which they had to guess the position of a hidden coin on a screen4 (Fig. 1a). They were told the cover story of a coin being tossed into a pond and informed that their task was to guess where the coin had fallen. They could not see the coin, but they could see 5 blue dots that were the “splashes” produced by the coin falling in. They were told that the person who threw the coin aimed, albeit imperfectly, at the center of the screen (mean of prior). They were also told that, between blocks, the thrower changed, and the new one might be better or worse at throwing (what indirectly informed them that the variance of the prior changed). To estimate the coin position, participants could use (although they were never explicitly told so) both the coin position’s likelihood, obtained from the “splashes”, and its prior (the distribution of previous coin locations). There was no temporal deadline, so participants had all the time they needed to submit their response. See Supplementary Methods for additional details
Data Analysis
Data analyses details can be found in the Supplementary Methods.
Supplementary Material
Acknowledgements
We would like to thank David Klein, Laura Pickering, Dr. Santiago Toledo, Dr. Citlali López-Ortiz and especially Dr. Tania Simuni for help in recruiting Parkinson’s disease patients. We also would like to thank Peter Dayan, Hugo Fernandes and Michele Basso for useful comments on the manuscript.
I.V. was supported by the Portuguese Science Foundation, the Gulbenkian Foundation and the Champalimaud Foundation (PhD fellowship SFRH/BD/33272/2007), and, more recently, by a Principal Research Fellowship from the Wellcome Trust to Professor Read Montague. This work was also supported by NIH grant 2R01NS063399 (to K.P.K.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Footnotes
Data availability
Data are available from the corresponding author on reasonable request.
Code availability
Code to produce the results shown here is available from the corresponding author on reasonable request.
Author Contributions
Both authors designed the experiment. IV ran the experiments and analyzed the data (with the supervision of KK). Both authors wrote the manuscript.
Competing Interestest Statement
The authors declare no competing interests.
References
- 1.Vilares I, Kording K. Bayesian models: the structure of the world, uncertainty, behavior, and the brain. Ann Ny Acad Sci. 2011;1224:22–39. doi: 10.1111/j.1749-6632.2011.05965.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kording KP, Wolpert DM. Bayesian integration in sensorimotor learning. Nature. 2004;427:244–247. doi: 10.1038/nature02169. [DOI] [PubMed] [Google Scholar]
- 3.Corlett PR, Frith CD, Fletcher PC. From drugs to deprivation: a Bayesian framework for understanding models of psychosis. Psychopharmacology. 2009;206:515–530. doi: 10.1007/s00213-009-1561-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Vilares I, Howard JD, Fernandes HL, Gottfried JA, Kording KP. Differential Representations of Prior and Likelihood Uncertainty in the Human Brain. Current Biology. 2012;22:1641–1648. doi: 10.1016/J.Cub.2012.07.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of uncertainty in human decision-making. Science. 2005;310:1680–1683. doi: 10.1126/science.1115327. [DOI] [PubMed] [Google Scholar]
- 6.Poldrack RA, et al. The neural correlates of motor skill automaticity. J Neurosci. 2005;25:5356–5364. doi: 10.1523/Jneurosci.3880-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lehericy S, et al. Distinct basal ganglia territories are engaged in early and advanced motor sequence learning. P Natl Acad Sci USA. 2005;102:12566–12571. doi: 10.1073/pnas.0502762102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kish SJ, Shannak K, Hornykiewicz O. Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease. New England Journal of Medicine. 1988;318:876–880. doi: 10.1056/NEJM198804073181402. [DOI] [PubMed] [Google Scholar]
- 9.Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
- 10.Friston KJ, et al. Dopamine, Affordance and Active Inference. Plos Comput Biol. 2012;8 doi: 10.1371/journal.pcbi.1002327. doi:ARTN e1002327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Friston K. The free-energy principle: a rough guide to the brain? Trends Cogn Sci. 2009;13:293–301. doi: 10.1016/j.tics.2009.04.005. S1364-6613(09)00117-X [pii] [DOI] [PubMed] [Google Scholar]
- 12.Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev. 1998;28:309–369. doi: 10.1016/S0165-0173(98)00019-8. [DOI] [PubMed] [Google Scholar]
- 13.Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483–494. doi: 10.1038/Nrn1406. [DOI] [PubMed] [Google Scholar]
- 14.Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci. 1996;16:1936–1947. doi: 10.1523/JNEUROSCI.16-05-01936.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
- 16.Kapur S. Psychosis as a state of aberrant salience: A framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiat. 2003;160:13–23. doi: 10.1176/appi.ajp.160.1.13. [DOI] [PubMed] [Google Scholar]
- 17.McClure SM, Daw ND, Montague PR. A computational substrate for incentive salience. Trends Neurosci. 2003;26:423–428. doi: 10.1016/S0166-2236(03)00177-2. [DOI] [PubMed] [Google Scholar]
- 18.Galea JM, Bestmann S, Beigi M, Jahanshahi M, Rothwell JC. Action Reprogramming in Parkinson's Disease: Response to Prediction Error Is Modulated by Levels of Dopamine. J Neurosci. 2012;32:542–550. doi: 10.1523/Jneurosci.3621-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Beeler JA, Daw N, Frazier CRM, Zhuang XX. Tonic dopamine modulates exploitation of reward learning. Front Behav Neurosci. 1997;4 doi: 10.3389/Fnbeh.2010.00170. doi:Artn 170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Dayan P, Balleine BW. Reward, motivation, and reinforcement learning. Neuron. 2002;36:285–298. doi: 10.1016/S0896-6273(02)00963-7. [DOI] [PubMed] [Google Scholar]
- 21.Frank MJ, Seeberger LC, O'Reilly RC. By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science. 2004;306:1940–1943. doi: 10.1126/science.1102941. [DOI] [PubMed] [Google Scholar]
- 22.Cools R, Barker RA, Sahakian BJ, Robbins TW. L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease. Neuropsychologia. 2003;41:1431–1441. doi: 10.1016/S0028-3932(03)00117-9. [DOI] [PubMed] [Google Scholar]
- 23.Brand M, et al. Decision-making impairments in patients with Parkinson's disease. Behav Neurol. 2004;15:77–85. doi: 10.1155/2004/578354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Tassinari H, Hudson TE, Landy MS. Combining priors and noisy visual cues in a rapid pointing task. J Neurosci. 2006;26:10154–10163. doi: 10.1523/Jneurosci.2779-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Olanow CW, Obeso JA, Stocchi F. Continuous dopamine-receptor treatment of Parkinson's disease: scientific rationale and clinical implications. Lancet Neurol. 2006;5:677–687. doi: 10.1016/S1474-4422(06)70521-X. [DOI] [PubMed] [Google Scholar]
- 26.Shiner T, et al. Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease. Brain. 2012;135:1871–1883. doi: 10.1093/Brain/Aws083. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Frank MJ. Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cognitive Neurosci. 2005;17:51–72. doi: 10.1162/0898929052880093. [DOI] [PubMed] [Google Scholar]
- 28.Perugini A, Ditterich J, Basso MA. Patients with Parkinson's Disease Show Impaired Use of Priors in Conditions of Sensory Uncertainty. Curr Biol. 2016;26:1902–1910. doi: 10.1016/j.cub.2016.05.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Berniker M, Voss M, Kording K. Learning Priors for Bayesian Computations in the Nervous System. Plos One. 2010;5 doi: 10.1371/journal.pone.0012686. doi:ARTN e12686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Fernandes HL, Stevenson IH, Vilares I, Kording KP. The Generalization of Prior Uncertainty during Reaching. J Neurosci. 2014;34:11470–11484. doi: 10.1523/Jneurosci.3882-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kishida KT, et al. Sub-Second Dopamine Detection in Human Striatum. Plos One. 2011;6 doi: 10.1371/journal.pone.0023291. doi:ARTN e23291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Ma WJ, Beck JM, Latham PE, Pouget A. Bayesian inference with probabilistic population codes. Nat Neurosci. 2006;9:1432–1438. doi: 10.1038/Nn1790. [DOI] [PubMed] [Google Scholar]
- 33.O'Reilly JX, Jbabdi S, Rushworth MF, Behrens TE. Brain Systems for Probabilistic and Dynamic Prediction: Computational Specificity and Integration. PLoS Biol. 2013;11:e1001662. doi: 10.1371/journal.pbio.1001662. PBIOLOGY-D-13-01872 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.