Skip to main content
NCBI home page
Brain logoLink to Brain
. 2012 Oct 31;135(12):3721–3734. doi: 10.1093/brain/aws273

Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making

Elizabeth J Coulthard 1,✉, Rafal Bogacz 2, Shazia Javed 3, Lucy K Mooney 4, Gillian Murphy 5, Sophie Keeley 5, Alan L Whone 1
PMCID: PMC3525052  PMID: 23114368

Abstract

Even simple behaviour requires us to make decisions based on combining multiple pieces of learned and new information. Making such decisions requires both learning the optimal response to each given stimulus as well as combining probabilistic information from multiple stimuli before selecting a response. Computational theories of decision making predict that learning individual stimulus–response associations and rapid combination of information from multiple stimuli are dependent on different components of basal ganglia circuitry. In particular, learning and retention of memory, required for optimal response choice, are significantly reliant on dopamine, whereas integrating information probabilistically is critically dependent upon functioning of the glutamatergic subthalamic nucleus (computing the ‘normalization term’ in Bayes’ theorem). Here, we test these theories by investigating 22 patients with Parkinson’s disease either treated with deep brain stimulation to the subthalamic nucleus and dopaminergic therapy or managed with dopaminergic therapy alone. We use computerized tasks that probe three cognitive functions—information acquisition (learning), memory over a delay and information integration when multiple pieces of sequentially presented information have to be combined. Patients performed the tasks ON or OFF deep brain stimulation and/or ON or OFF dopaminergic therapy. Consistent with the computational theories, we show that stopping dopaminergic therapy impairs memory for probabilistic information over a delay, whereas deep brain stimulation to the region of the subthalamic nucleus disrupts decision making when multiple pieces of acquired information must be combined. Furthermore, we found that when participants needed to update their decision on the basis of the last piece of information presented in the decision-making task, patients with deep brain stimulation of the subthalamic nucleus region did not slow down appropriately to revise their plan, a pattern of behaviour that mirrors the impulsivity described clinically in some patients with subthalamic nucleus deep brain stimulation. Thus, we demonstrate distinct mechanisms for two important facets of human decision making: first, a role for dopamine in memory consolidation, and second, the critical importance of the subthalamic nucleus in successful decision making when multiple pieces of information must be combined.

Keywords: decision making, Parkinson’s disease, mathematical modelling, reasoning, memory

Introduction

Optimal decision making usually requires several factors to be taken into account and appropriately combined or integrated. For example, when an animal is about to pounce upon its prey, many items of information including size, speed and distance of the prey, in addition to the hunter’s own prowess, must be accurately combined to increase the chance of success. Recent computational models have proposed that separate neural populations within the basal ganglia are involved in learning probabilistic associations and combining probabilistic information. In this article, we assess these predictions by testing patients with Parkinson’s disease ON and OFF deep brain stimulation (DBS) to the region of the subthalamic nucleus and ON and OFF dopaminergic medication.

Dopamine is a modulatory neurotransmitter, widely involved in learning from feedback, flexible motor planning and decision making (Egelman et al., 1998; Welberg, 2009). Midbrain dopaminergic neurons facilitate adaptation of behaviour according to upcoming reward or task demands (Hollerman and Schultz, 1998; Cools et al., 2001, 2003; Fiorillo et al., 2003). Along with such trial-by-trial learning effects, neuromodulation by dopamine can take place over a longer time scale. Repeated dopamine release from both ventral tegmental-hippocampal and ventral tegmental-striatal projections can enhance or diminish neural connections over time, a process vital for establishing new longer term memories (Calabresi et al., 1997; Bethus et al., 2010). Hence, one might expect that the level of dopamine released, from the terminal plexus of midbrain pathways, will predict ability to retain newly learned information.

Depletion of dopamine from both nigro-striatal and ventral tegmental-mesolimbic pathways is found in Parkinson’s disease (German et al., 1989), a condition typically treated with medications that increase dopamine receptor stimulation. There is evidence that acquisition (learning) of probabilistic stimulus–response associations from feedback is abnormal in unmedicated patients with Parkinson’s disease (Knowlton et al., 1996; Hälbig et al., 2004), and they are particularly impaired in learning from positive feedback (Frank et al., 2004). Here we investigate the effect of dopaminergic medications on retention of information and predict that memory for learned information should be affected by medications.

One important characteristic of a decision-making network is inhibition of the prepotent response to each individual stimulus, thus avoiding rapid impulsive behaviour that does not weigh up all options. The subthalamic nucleus, a region within the basal ganglia, has such an inhibitory capability. Subthalamic nucleus output results in suppression of motor programmes and consequent response delay, which is particularly pertinent when there is conflict between possible action choices (Frank et al., 2007; Isoda and Hikosaka, 2008; Hershey et al., 2010; Cavanagh et al., 2011; Rodriguez-Oroz et al., 2011).

Computational modelling of decision-making generates predictions about the precise role of the subthalamic nucleus when potential action choices compete. It has been proposed that the cortico-basal-ganglia circuit computes probabilities of different possible motor responses being correct according to Bayes’ theorem (Bogacz and Larsen, 2011). This model assumes that the cortex, together with striatum, computes which response is most likely to be correct (Fig. 1A). The model additionally assumes that feedback from the subthalamic nucleus (via output nuclei and thalamus) subsequently normalizes cortical activity, such that cortical neurons selective for a particular response have activity directly proportional to the logarithm of the probability of responses being correct (Fig. 1B). In particular, the model assumes that the subthalamic nucleus, together with external segment of globus pallidus, compute the normalization term in Bayes’ theorem. This is a simple computation involving a non-linear summation of cortical inputs (provided to the subthalamic nucleus through the hyperdirect pathway), and it has been demonstrated that neurons in the subthalamic nucleus and globus pallidus have connectivity and response properties ideally suited to perform this computation (Bogacz and Gurney, 2007). The details of the model are described in the online Supplementary material.

Figure 1.

Figure 1

Open in a new tab

Computational model of probabilistic accumulation of information (Bogacz and Larsen, 2011), adapted to the reasoning task described. Each row of circles denotes neural populations within the brain area indicated by a label. ‘Integrators’ describe the cortical integrator neurons accumulating evidence for motor actions; ‘Sensory’ denotes visual cortical neurons encoding presented stimulus; ‘STN’ denotes subthalamic nucleus; ‘GP’ denotes external segment of globus pallidus; and ‘Output’ denotes output nuclei of the basal ganglia, i.e. internal segment of globus pallidus and substantia nigra pars reticulata. Black and light grey circles denote neural populations selective for pressing left and right buttons, respectively. Arrows denote excitatory connections, and lines ending with circles denote inhibitory connections. Labels denote the quantities represented by neurons while probabilities of responses are updated on the basis of presented stimulus. The details of this computation are described in the Supplementary material. Briefly, the model assumes that initially (A) the logarithms of prior probabilities of responses are represented by cortical integrators. They are combined in striatum with the log-likelihoods of presented stimuli given the responses. The logarithms of combined probabilities are projected via output nuclei and thalamus back to cortical integrators (top labels in B). On this basis, the subthalamic nucleus–globus pallidus circuit computes the logarithm of the normalization term in the Bayes theorem, which is subtracted in the output nuclei, giving the logarithms of posterior probabilities of responses.

Because the model predicts that the subthalamic nucleus is involved in updating probabilities on the basis of combined pieces of information, we hypothesized that disruption of information processing in the subthalamic nucleus owing to DBS should impair patients’ ability to combine probabilistic information. DBS reversibly modifies activity in the region of the subthalamic nucleus in patients with Parkinson’s disease and improves motor symptoms including tremor, bradykinesia, rigidity and off-state freezing (Deuschl et al., 2006; Weaver et al., 2009; Adrian et al., 2010; Williams et al., 2010). In Parkinson’s disease, subthalamic nucleus activity is abnormal, particularly in dorsolateral regions (Rodriguez-Oroz et al., 2001), and the aim of DBS is to electrically modify aberrant dorsolateral subthalamic nucleus firing, helping to normalize motor function (Plaha et al., 2006). Cognitive processes occur in more ventral subthalamic nucleus, relatively less affected in Parkinson’s disease (Hershey et al., 2010). However, given the several millimetre electrical field surrounding the tip of the electrodes, ventral subthalamic nucleus can also be affected by dorsolaterally sited DBS, putatively leading to interference with cognitive function (Hershey et al., 2010). In terms of cognitive processing, therefore, we can view subthalamic nucleus DBS as a reversible means by which to alter subthalamic nucleus firing, and we have used this in vivo human model to inform us with regard to the role of the subthalamic nucleus in probabilistic choice.

We investigate the effect of both dopamine and subthalamic nucleus activity modulation on behaviour by testing patients with Parkinson’s disease treated with dopamine replacement therapy and/or DBS to the region of the subthalamic nucleus. We use cognitive tasks probing probabilistic learning, memory retention and information integration in patients with Parkinson’s disease. This task is similar to the weather prediction task, variations of which have been used in other human and animal studies (Knowlton et al., 1996; Yang and Shadlen, 2007), but it is modified so that we can differentiate deficits in learning and memory from information integration impairment (Fig. 2). In order to tease apart the effect of dopaminergic medication from that of subthalamic nucleus stimulation, we tested patients ON and OFF both dopaminergic therapy and subthalamic nucleus DBS.

Figure 2.

Figure 2

Open in a new tab

(Ai) Learning stage: participants receive feedback after pressing a button in response to a visual stimulus. (Aii) Sample probabilities of positive feedback for different stimuli. Stimuli in the left and right upper rows yield higher probability of positive feedback after left and right button press, respectively. Probabilities of positive feedback after optimal responses are shown on the right of the panel e.g. for the mouse shown in A, participant had 70% probability of positive feedback after pressing the right button, and 30% chance of receiving positive feedback after pressing the left button. (B) Integration stage: participants selected the best response after seeing three stimuli consecutively (separated by a 50 ms pause). (C) Memory stage: participants were asked to choose left or right after visual stimulus according to what they had previously learned. No feedback was provided in integration or memory stages.

Materials and methods

Participants

Twenty-two patients with Parkinson’s disease were tested (11 treated with bilateral DBS and dopaminergic medications and 11 treated with dopaminergic medications but without DBS, Table 1). We attempted testing in a further two patients treated with DBS who managed to partially complete only one session and therefore we excluded these data. Fifteen healthy age-matched control participants were also recruited (Table 1). Ethics approval was granted by the North Bristol NHS Trust Research Ethics Committee and all participants gave written consent in accordance with the Declaration of Helsinki.

Table 1.

Patient and healthy participant demographics

Parkinson’s DBS group Parkinson’s medication group Healthy participants
Number 11 11 15
Mean age (standard error) 56 (2.3) 58.9 (2.0) 57 (3.4)
Duration of disease (years) 9.5 (0.8) 7.27 (1.65)
Time since stimulator insertion (months) 26.5 ∼
Levodopa equivalence (mg) 944 (133) 902 (148)
Mini-Mental State Examination ∼ 27.0 (0.9) 29.1 (0.3)
DRS2 AEMSS 10.3 ∼ ∼
LARS apathy score −21 (2.7) −22.4 (3) −28.6 (1.3)
BIS impulsivity score 71 58 57
UPDRS motor ON/ON 11.7 (2) ON MEDS 18.8 (5.2)
ON DBS/OFF MEDS 18.6 (1.8) OFF MEDS 28.0 (7.6)
OFF DBS/ON MEDS 17.25 (3.3)
OFF/OFF 43.6 (3.0)
Open in a new tab

BIS = Barratt Impulsiveness Scale (performed ON medication or ON medication and on DBS); DRS2 AEMSS = age-adjusted score for the dementia-rating scale: only scores >6 were included in the study; LARS = Lilly Apathy Rating Scale. Standard errors are in brackets.

Experimental procedure

For DBS patients, our computerized cognitive tasks were administered during routine postoperative inpatient assessments. This meant that DBS could be switched off for a longer duration than has been standard in other studies investigating cognitive effects of DBS. DBS was switched OFF an average of 10 h before testing ‘OFF DBS’ and was switched back ON for a minimum of 2 h before testing ‘ON DBS’. DBS patients were also tested ON and OFF medication. Medication was stopped on average 15 h before testing in the OFF medication state (practically defined OFF) and taken for a minimum of 18 h before testing in the ON medication state. For patients only receiving dopaminergic medication, i.e. those without DBS, medications were stopped an average of 18 h before testing OFF medications or had been taken for a minimum of 18 h before ON medications testing. Thus, DBS patients were tested in four conditions: (i) ON medication/ON DBS; (ii) ON medication/OFF DBS; (iii) OFF medication/ON DBS; and (iv) OFF medication/OFF DBS. Patients were tested in the order OFF/OFF, ON DBS/OFF medication, OFF DBS/ON medication, according to clinical protocol. The ‘ON/ON’ session was performed either first or last to demonstrate that there were no significant practice effects (Supplementary material). Medication-only patients were tested ON and OFF medications, and the order of testing was counterbalanced across patients. Four of 44 testing sessions in the DBS group did not happen due to patient fatigue or the DBS not being turned off or medications failing to be omitted as planned (sessions missed: three OFF DBS/ON medication, one ON DBS/OFF medication, plus one memory phase only missing in the ON/ON condition). Patient demographics, medications and clinical information are listed in Table 1.

Behavioural assessments

At each testing session, a Unified Parkinson’s Disease Rating Scale (UPDRS) motor scale (Part III) subscore was obtained (Fahn and Elton, 1987) (Table 1). DBS patients also undertook the Mattis Dementia Rating Scale (Matteau et al., 2011), whereas medication only patients and normal participants did the Mini-Mental State Examination (Folstein et al., 1975). Further assessments included a trait impulsivity measure (Barratt Impulsiveness Scale, Patton et al., 1995) and the Lille Apathy Rating Scale (Sockeel et al., 2006) (Table 1).

Deep brain stimulation surgical procedures

Patients were all at least 6 months postoperative for deep brain electrode placement. DBS surgery was carried out under general anaesthesia using T2-weighted MRI-directed visualization of the target. Our target in all cases was the posterior subthalamic area including the caudal zona incerta nucleus and the posterior aspect of the subthalamic nucleus. The surgical technique has been described previously (Patel et al., 2007) and involves the use of guide tubes and plastic stylettes to verify the position of the target, which is confirmed with an intraoperative MRI. This is followed by replacement of the stylettes with DBS electrodes (Model 3389, Medtronic Inc.) and allows for image-guided targeting accuracy of within 1 mm.

Cognitive testing

Testing took place in a quiet room in Frenchay Hospital, North Bristol Trust. Stimuli were presented in the vertical midline on a laptop computer. Left and right hand responses were recorded using a Cedrus RB-834 response pad. On each trial of the task, the participants were asked to choose between two response options (left or right hand button) that were associated probabilistically with presented visual information.

In the first testing session, participants were given a practice task to familiarize themselves with the button response box and learning of probabilistic information. In the practice task (40 trials), participants were shown one of two octopuses: one with an 80% probability of left being the correct answer; the other with an 80% probability of right being the correct answer. Otherwise, the experimental procedure for the practice task was as for the learning task (described below).

The main experiment comprised three stages: (i) Learning task; (ii) Integration task; and (iii) Memory task to allow differentiation of each participant’s capacity for learning and remembering as well as integrating information to reach a decision when ON or OFF stimulation and/or ON or OFF medication.

Cognitive testing

Learning task

On each trial of the learning task, participants were shown a single visual stimulus (cartoon animal) for 1 s. There were four different stimuli in each task presented 30 times each across three blocks (10 times per block) making a total of 120 trials (three blocks of 40 trials). The interstimulus interval was 2 s plus reaction time, and the order of stimuli was pseudorandomized such that the number of different stimuli per block was controlled. After the stimulus, participants were shown a GO signal to prompt them to choose either a left or right hand button response (Fig. 2Ai). On the basis of probabilistic feedback after the response, participants could learn the optimal button to press for a given stimulus (Fig. 2Aii). Stimuli and feedback parameters were modified after pilot investigations such that the eventual paradigm used was optimized to discourage declarative learning of stimulus–response associations. The aim was that participants had a vague habitual sense or instinct for the correct response, and verbal strategies for learning information were minimized. Thus, we sought to investigate the type of ‘automatic’ decision making in which the basal ganglia are implicated (Redgrave et al., 2010).

Four groups of four animals (mice, beetles, mosquitoes and fish) were used, and both the animals used in each condition and the meaning of each individual animal were counterbalanced across participants (mice shown in Fig. 2, other stimulus sets shown in Supplementary Fig. 1). Previous reports have suggested that subthalamic nucleus DBS may interfere with processing of stimuli evoking negative emotions such as fear or anger (Lawrence et al., 2002; Dujardin et al., 2004). Consequently, our stimuli were designed to avoid conveying intensely negative emotions and, furthermore, the counterbalancing of stimulus sets and order was chosen to avoid any confound from facial emotional processing deficits.

Integration task

Using the learned response associations mentioned above, participants were then asked to decide which button to press after seeing three stimuli in succession (Fig. 2B). A GO signal was presented after the third stimulus, and participants were informed that they must wait for the GO signal before responding. This was to ensure they viewed all three animals before responding. If an anticipatory response was made, the early response was not recorded and the GO signal remained on the screen until the participant responded again i.e. only responses made after seeing all three animals were analysed. Two blocks of 64 stimuli were presented. No feedback was given.

It was anticipated that participants would combine data in the integration task according to Bayes’ theorem (with equal prior probabilities):

graphic file with name aws273m1.jpg (1)

where P(Si|L) are the probabilities of leftward movement associated with the stimuli in the learning task. The trial was considered as correct if the participant chose left response when P(L|S1,S2,S3) > 0.5 or if the participant chose right response when P(L|S1,S2,S3) < 0.5.

Memory task

After the integration stage, the stimuli were shown again individually (as in the learning task), and participants were asked to respond left or right according to the previously performed learning task. In this memory task, however, in contrast to the learning task, no feedback was provided. Each stimulus was shown a total of five times giving a total of 20 trials in a randomized order. The memory task was commenced ∼20 min after the onset of the learning task.

Data analysis

Accuracy

Accuracy rates (%) were calculated for each of the learning, integration and memory tasks for each condition (ON or OFF DBS and/or ON or OFF dopaminergic medication) in each participant. Overall effects of DBS or medication were investigated according to our a priori hypotheses. In addition, we compared the average performance of patients (collapsed across different DBS and medication conditions) with that of healthy participants by performing three one-way ANOVAs for learning, integration and memory, with average accuracy as dependent variable and subject group as factor.

We hypothesized that dopamine replacement medication would affect learning and memory or retention of information. To investigate this, we looked at the effects of medication on accuracy in the learning and memory parts of the task. Note that learning data presented are based on performance in the final (third) block of the task to reflect the patients’ eventual accuracy in the task rather than the trial and error of the first blocks (see Supplementary material for comparison of performance in last block with that overall). We wanted to separate memory effects at 20 min from immediate learning effects and so data used for our memory ANOVA were (memory–learning) scores in each condition for each individual patient. We performed a repeated measure mixed-design ANOVA in SPSS (PASW 18.) for learning (last block) and memory (memory–learning) accuracy scores separately, with medications state (ON versus OFF) as the within participant factor and group (DBS or medication only) as the between participants factor.

To present the percentage effect of medication on performance, we calculated an overall effect of medication on memory for each of the 22 patients [average percentage difference between (memory–learning) scores in ON compared with OFF medication states, collapsed across DBS state for the DBS patients]. We then looked to see whether the overall performance in the memory tasks in the ‘dopaminergic medication only’ patients differed from that of normal healthy control subjects. We used independent samples t-tests comparing ON medication versus normal participants’ memory accuracy and OFF medication versus normal participants’ memory accuracy separately. Potential effects of DBS on learning and memory were investigated with repeated measures ANOVA on DBS patient data for learning and memory tasks separately with medication and DBS state as within participant factors.

We hypothesized that DBS would negatively impact on accuracy in the integration task. Mean integration accuracy results for each DBS patient, in each condition (ON and OFF both DBS and medication), were analysed using a mixed-design repeated measures ANOVA where the within participant variable was DBS (ON versus OFF) and the between participant factor was medication state (ON or OFF). Further, paired sample t-tests were performed on the integration accuracy scores ON DBS/ON medication versus OFF DBS/ON medication and ON DBS/OFF medication versus OFF DBS/OFF medication. To present the percentage effect of DBS on performance, we calculated an overall effect of DBS on integration for each of the 11 patients (average percentage difference integration accuracy scores in ON compared with OFF DBS states, collapsed across medication state).

We also investigated whether DBS impaired performance on both easy trials (with substantial evidence for one of the alternatives) and difficult trials (with weaker or conflicting evidence). We split trials in the integration task on the basis of overall probability of left response given three stimuli (Equation 1) and refer to trials in which this probability was between 25% and 75% as difficult, and to all other trials as easy. Data are presented graphically (Supplementary Fig. 2A).

Reaction times

We investigated the effects of DBS on reaction time. Specifically, we were interested to see whether the normal delay evoked by the need to dramatically update a decision was affected by subthalamic nucleus DBS. For each participant and condition, we computed the difference between median reaction time on trials when the last stimulus should have prompted a different response from the previous two (e.g. left, left, right) and on trials when all of the stimuli predicted the same direction (e.g. left, left, left). We then compared the delay when DBS was ON compared with OFF using a repeated measures ANOVA with DBS ON versus OFF and medications ON versus OFF as within participant factors. We performed a secondary analysis to ensure that the delay was due to updating of the decision process, rather than being due to processing of contrasting visual material (Supplementary material).

Results

Dopamine replacement therapy ameliorates loss of procedural memory

How did dopamine replacement impact on the ability to acquire and maintain memory? There was a trend towards average performance in both the DBS and medication-only patients being worse than that of healthy age-matched participants in the last block of the learning task [F(2,624) = 2.5, P = 0.1] and no clear difference between patients and healthy participants in the memory task [F(2,780) = 2.9, P = 0.71] (Fig. 3A and B).

Figure 3.

Figure 3

Open in a new tab

Mean accuracy in the last block (40 trials) of learning stage (A) and in memory stage (B). The bar graphs on the left show the results from patients tested ON and OFF medication (MEDS) and DBS, whereas the bar graphs on the right show results from patients’ tests ON and OFF medication. The colours of the bars indicate testing condition. Error bars give the standard error of the mean (SEM) difference in accuracy between DBS ON and OFF for ON and OFF medication conditions separately for the DBS group and average difference between ON and OFF medication accuracy for the medication-only group. *Statistically significant comparison between performance ON and OFF medication in the memory phase of the task. #Although 11 patients performed in the ON medication/ON DBS condition, one memory phase was omitted and hence there are only 10 patients in this condition (Graph B, column 1).

In patients, no consistent effects of dopamine replacement were found in the learning stage [F(1,83.2) = 0.4, P = 0.4; Fig. 3A]. However, unmedicated patients with Parkinson’s failed to retain learnt information as well as medicated patients in the memory stage [significant effects of medication on memory in mixed-design ANOVA incorporating all patients: F(1,2560) = 11.7, P < 0.005; Fig. 3B]. The average benefit of medications on memory accuracy when assessed at 20 min was 15% (standard error 4.3% across all 22 patients).

Accuracy of medication-only patients with Parkinson’s disease (n = 11) on treatment in the memory task was not significantly different from that of normal participants [68% (SEM 5.8%) control subjects versus 61% (SEM 5.3%) medicated patients with Parkinson’s disease, P = 0.46]. However, the accuracy of patients with Parkinson’s disease when unmedicated was significantly less than that of normal participants [68% (SEM 5.8%) control subjects versus 45.5% (SEM 4.3%) unmedicated patients, independent samples t-test, t = 2.1, P < 0.05]. Therefore, dopamine replacement appears to correct defective memory retention in Parkinson’s disease when assessed at 20 min.

Deep brain stimulation to subthalamic nucleus selectively impairs probabilistic choice

Average performance across conditions in the integration task was worse in patients compared with healthy participants [F(2,1124), P < 0.05]. Post hoc analysis with Bonferroni correction demonstrates that both medication-only and DBS patients differ from healthy participants (P < 0.05 for both comparisons).

We predicted that DBS to the subthalamic nucleus would impair choice on the basis of probabilistic information from multiple stimuli. Overall accuracy of integration was impaired in patients ON DBS compared with patients OFF DBS. On average, accuracy in the integration task was 11.5% (SEM 4.3%) higher when patients were tested OFF DBS compared with ON DBS. There was a significant main effect of DBS on integration [F(1,573) = 4.9, P < 0.05], but no interaction between the effects of DBS and medication (interaction dbs × meds) [F(1,176.8) = 0.669, P = 0.426]. There was no effect of DBS on learning or memory [F(1,0.598), P = 0.451 and F(1,291), P = 0.506, respectively] or interaction between medication and DBS on learning and memory. DBS effects were similar regardless of whether patients were ON or OFF medication while performing the integration task and remained significant when patients’ ON medication scores alone were considered (t = 2.6, P < 0.05) (Fig. 4A). Supplementary Fig. 2A shows that DBS affected performance similarly for both easy and difficult trials.

Figure 4.

Figure 4

Open in a new tab

Effects of DBS and medication on performance in the integration task. The colours of the bars indicate the testing condition. (A) Mean accuracy. Error bars show the SEM. (B) The average median reaction time in the trials on which the last stimulus predicts the opposite response to the first and second (incompatible), and the trials on which all three stimuli predict the same response (compatible). For each combination of DBS and medication state, the error bars on both columns show the standard error of the differences between reaction times on trials where the last stimulus is incompatible compared with compatible. *Significant statistical comparison: patients are more accurate in the integration task when DBS is switched OFF compared with DBS ON.

Next we asked whether the patients ON DBS fail to slow down following conflicting stimulus. Normal participants slow down when the last animal shown in the integration task predicts a change of plan, that is, when the third stimulus strongly conflicts with the first two [average median reaction time when the last stimulus predicts different direction than previous two = 558 ms (SEM 63 ms); average median reaction time when all stimuli predict the same direction = 474 ms (SEM 56 ms)]. It is proposed that such behavioural delay results from subthalamic nucleus activity, which slows down responses on high conflict trials (Frank et al., 2007). Patients tested OFF DBS performed as normal participants, with an appropriate delay when the decision had to be updated (Fig. 4B). In contrast, when subthalamic nucleus DBS was switched ON, delay was significantly reduced (repeated measures ANOVA comparing delay when DBS was ON compared with OFF: main effect of DBS [F(1,0.2) = 31.1, P < 0.05], no main effect of medication [F(1,0.16) = 1.14, P = 0.327]), in keeping with the hypothesis that the need to slow down in the face of conflicting information requires normal functioning of the ventral subthalamic nucleus.

Simulations of a computational model

We simulated the model of the cortico-basal-ganglia circuit (Fig. 1) adapted to our task (details of the simulation methods are described in the Supplementary material). Because in our task participants were required to respond after a cue, we followed a common approach to simulating choices in tasks with cued response (e.g. Usher and McClelland, 2001; Bogacz et al., 2006), and to determine a choice in a simulated trial, we simply assumed that the response corresponding to the most active integrator at the time of the cue is chosen.

The effect of medication could be simulated in two ways. The first method corresponds to the hypothesis that medication influences acquisition of new memories, and would involve lowering the rate of learning of cortico-striatal synapses in simulated OFF medication conditions. However, this would result in difference in the accuracies in the learning phases between medication conditions in the simulated data, which is not present in the patient data. The second method corresponds to the hypothesis that the medication influences retention of memories, and involved introduction of decay over time of cortico-striatal synaptic weights in simulated OFF medication conditions. The performance of the simulated model is shown in Fig. 5 and is qualitatively consistent with our patient data, as the accuracies in the memory phase (Fig. 5B), but not learning (Fig. 5A) phase, significantly differ between medication conditions.

Figure 5.

Figure 5

Open in a new tab

Performance of computational model. The colours of the bars indicate the simulation condition. Error bars show the SEM. The mean accuracy of the model in learning (A), memory (B) and integration (C) phases. *Statistical significance analogous to behavioural results: patients ON DBS are impaired in the integration task (C) and dopaminergic medication improves memory (B).

The effect of DBS was simulated in the model by setting the activity level in the subthalamic nucleus to 0. As we describe in detail below, this manipulation impairs accuracy in the integration task in the model, because the neural populations normally have activities limited to certain ranges, and lack of subthalamic nucleus modulations permits the activities to reach limits of these ranges, which owing to physical limitations distorts encoded information. Although subthalamic nucleus neurons are excitatory (glutamatergic), the overall effect of their activity on cortex is inhibitory because the polysynaptic projection from the subthalamic nucleus to cortex involves inhibitory neurons (Fig. 1). Thus, a decrease of subthalamic nucleus activity in the simulated DBS ON condition results in an increase after each stimulus of activity of both cortical integrator neurons representing left response and the neurons representing right response.

In the integration phase, in which multiple stimuli are presented, the activity of integrators may exceed the value representing probability of 1, i.e. exceed their normal range of activity (as a probability is by definition≤1). So high cortical activity may result in output nuclei getting negative activity in simulations, but because firing rate cannot be negative, we set it to 0 whenever high inhibition from the integrators (via striatum) is received. This bounding of activity of output nuclei was done both in simulation of DBS ON and DBS OFF state (although it had little effect in simulated DBS OFF condition). We note that in the real brain, when output nuclei get activity close to 0, usually action is initiated, but because the task requires refraining from the response until the GO signal, there may be additional processes preventing a premature response (and possibly increasing activity in output nuclei) that we do not include in our model. This bounding mostly limits the activity of the integrator corresponding to the correct response (as it receives more input), and hence reduces the difference between the two integrators and thus the probability of making the correct choice. Consequently, the model makes more errors in the integration phase in simulated DBS ON conditions (Fig. 5B). Supplementary Fig. 2B shows that simulated effect of DBS impaired performance on both easy and difficult trials, which were defined in Supplementary Fig. 2A illustrating a similar effect in patients. This suggests that the boundary described above may be reached in simulations of both easy and difficult trials. Nevertheless, in the simulated DBS on condition, the accuracy is higher for easier trials, because the integrator corresponding to the incorrect response gets less input in the easy than in the difficult condition, and hence is more likely to have activity further below that of the integrator corresponding to the correct response.

Discussion

We sought to investigate the role of dopamine and the subthalamic nucleus within the network of brain areas involved in probabilistic decision making. We found that dopamine replacement therapy influenced memory for learned response associations. Dopamine replacement boosted performance in our memory task, invoking a role for dopamine in procedural memory retention. In contrast, switching ON subthalamic nucleus DBS resulted in a specific probabilistic choice deficit and no significant effect on memory. Thus we demonstrate the pivotal role of the subthalamic nucleus in permitting the successful integration of information required for probabilistic decisions.

To perform the integration stage of the task, participants had to accumulate and weigh information from three sequentially, but rapidly, presented stimuli. Optimal performance required updating of the decision process as each stimulus was presented. Reaction time analysis showed that when the subthalamic nucleus was stimulated, participants did not slow down sufficiently to allow the appropriate updating of information mandated for an optimal decision to be reached. Thus, we infer that the subthalamic nucleus is a critical component of the pathway that serves the function of accumulating and integrating probabilistic information.

Effect of dopaminergic medications

How does dopamine therapy improve retention of longer term memories? First, it is important to note that converging evidence points to multiple parallel memory subsystems within the human brain (Poldrack and Packard, 2003). The basal ganglia are centrally involved in encoding procedural memory or stimulus-driven action, rather than more explicit or declarative information (Knowlton et al., 1996). Here, we are making a further distinction between very short-term memory formation over a few seconds or very few minutes and medium or longer term memory retention over several minutes or hours (Smith et al., 2006). Our data support a role for dopamine in medium- to long-term retention of procedural or stimulus-driven memory. One could consider whether it is truly memory retention and not recall that is affected by dopamine, particularly as one previous study has suggested that dopamine is required for recall of information in a novel context (Shiner et al., 2012). However, no such novelty applies in the memory phase of our task as stimuli and task requirements are as for the learning phase.

Dopamine modulates neuronal firing patterns after repeated stimulation, resulting in long-term potentiation and depression (Calabresi et al., 1997; Bethus et al., 2010), processes thought to be important for longer term memory retention (Cetonze et al., 2001; Lisman and Grace, 2005; Shohamy and Adcock, 2010). Dopaminergic networks within basal ganglia and hippocampal–ventrotegmental loops have been implicated in laying down new longer term memories and habit formation (Redgrave et al., 2010). Given that dopamine depletion occurs in both these regions in Parkinson’s disease (German et al., 1989), one might expect that difficulty laying down longer term procedural memories contributes to the failure of automatic behaviour commonly found in patients with Parkinson’s disease (Yin and Knowlton, 2006). Replacement of dopamine within depleted midbrain networks may well be responsible for improvement in longer term memory in our task.

Another possibility for why the withdrawal of medications affected accuracy in memory task but not in the learning task may be formulated on the basis of results from Frank et al. (2004). They showed that withdrawing dopaminergic medication disrupts specifically the ability to learn from positive feedback, and that unmedicated patients have an enhanced ability to learn from negative feedback. It may be possible that the lack of feedback in the integration phase, together with lowered level of dopamine, may drive synaptic plasticity which is normally occurring after negative feedback resulting in extinction of associations acquired in the learning phase. This hypothesis predicts that the effect of medications should disappear if the integration phase is replaced by a delay period.

The extinction hypothesis described above could explain discrepancy between the effects of medications on accuracy we observed and the lack of overall effect observed by Frank et al. (2004) because their task did not include as prolonged a phase without feedback as in our task. However, there is another explanation for the above discrepancy in overall effect of medications. The analysis of learning from positive and negative feedback in our task (Supplementary material) suggests that in all conditions participants predominantly learned only from positive, rather than negative feedback. It is possible that the lack of learning from negative feedback in our task is caused by a particular choice of wording ‘Bad Luck’ (Fig. 2A), which may suggest that the negative outcome is not resulting from the participant’s action. Because our participants predominantly learnt from positive feedback, the impaired performance in the memory phase OFF medications is consistent with the result of Frank et al. (2004), the critical difference being that the memory effects of medication in our task were delayed rather than occurring at the initial learning stage.

Effect of deep brain stimulation

Topographically organized signals travel through the subthalamic nucleus, which is extensively interconnected via a wide arborization with other areas of the basal ganglia, including the globus pallidus, substantia nigra and striatum as well as cerebral cortex (Parent and Hazrati, 1995). A recent model of the basal ganglia proposes that the subthalamic nucleus is critically involved in the accumulation of probabilistic information (Bogacz and Gurney, 2007). Our behavioural data are in accordance with the predictions of the model; here we demonstrate that altering subthalamic nucleus activity through DBS interferes with upcoming response choice when several factors need to be taken into account.

Most of us take time when we change our minds, or update a decision according to new events. We have shown that the subthalamic nucleus is essential to allow us to slow down and revise response choice. Impaired reasoning following alteration of the subthalamic nucleus firing pattern by DBS stems from a failure to accrue information over time and is associated with an inappropriate speeding of the response when new information should conflict with an already prepared motor programme. Thus, we have provided evidence that could explain how millisecond alterations in reaction times, shown in patients with subthalamic nucleus DBS (Frank et al., 2007), may underlie abnormal reasoning over a longer time frame (Frank et al., 2007; Wylie et al., 2010; Rodriguez-Oroz et al., 2011). Such breakdown in the ability to slow and integrate evidence before making a decision also helps to explain why some patients with subthalamic nucleus DBS clinically manifest abnormal behaviour including impulsivity (Voon et al., 2008; Hälbig et al., 2009).

One may consider whether anything other than information integration deficit could account for the observed failure in our integration task. There was no consistent effect of subthalamic nucleus DBS on learning or memory of stimulus–response associations. Therefore, neither learning nor memory deficits can explain the effects on integration. Impaired visuomotor processing speed has been suggested in patients with subthalamic nucleus DBS (Follett et al., 2010). We therefore performed a secondary analysis to see whether the normal delay associated with a dramatic change of decision at the final stimulus was actually a result of slowed visual, rather than motor, processing (Supplementary material). When the last two stimuli both suggested the same response, but were visually different however, there was no greater delay. Hence, the delay truly appears to represent slowing in the face of a more complex decision, rather than slowing provoked by more complex visual processing. Furthermore, if reduced visuomotor processing speed influenced our results, one would expect reaction times to be slower when participants performed more poorly, that is when DBS was switched ON. There were no significant effects of DBS on median overall reaction time (Supplementary material), but a trend towards faster median overall reaction times when DBS was ON, making visuomotor processing speed very unlikely to be a factor in the poorer probabilistic choice observed.

Can overall motor performance explain our results? Clearly both subthalamic nucleus DBS and dopaminergic medications improve motor symptoms in patients with Parkinson’s disease (Deuschl et al., 2006; Weaver et al., 2009). Indeed, as expected, UPDRS scores showed improvement when ON both medications and DBS, with intermediate scores when one or the other was ON alone and with highest scores (most impaired) when OFF both medication and DBS. In the integration task, however, poor performance actually occurred when subthalamic nucleus DBS was switched ON and the patients had relatively improved UPDRS scores. In addition, our observed differential effects of medication and DBS on our integration and memory tasks, despite the fact that both manipulations alleviate motor dysfunction, are also against our behavioural results being explained on the basis of variations in motor function. In addition, neither could the specific probabilistic choice deficit be explained by generic failure of executive function such as perseveration, as a perseverative tendency would affect performance in both the learning and memory phases of our task as well as the integration phase.

Our results may seem inconsistent with those of Halbig et al. (2004) who found better performance with DBS ON than OFF in the original weather prediction task in which learning and integration are not separated into different stages of the experiment. Because most trials in their experiment involved both learning from feedback and combining of probabilistic information, it is not clear which of these cognitive tasks was affected by DBS.

Relationship to the conflict model

A computational model proposed by Frank (2006) has been very successful in explaining the effects of DBS on behaviour in choice tasks in which participants were free to respond at any time after stimulus onset. The model suggests that the subthalamic nucleus computes conflict between evidence representing alternative choices, and its influence on the output nuclei delays choice on more conflicting trials until the conflict resolves. Thus, the model naturally explains how the DBS reduces difference between reaction times on high and low conflict trials (Fig. 4B) (Frank et al., 2007). More recently, it has been shown that increased activity of the subthalamic nucleus has a similar effect to an increase in decision threshold in a very influential abstract model of decision making—the diffusion model (Cavanagh et al., 2011; Ratcliff and Frank, 2012). The threshold parameter describes how much evidence needs to be gathered before choice, and lowering the threshold reduces both accuracy and reaction time. Within this framework, the conflict model would also explain the effect of DBS on accuracy in the integration task (Fig. 4A) by postulating that the reduced subthalamic nucleus activity corresponds to reduced threshold and hence lower accuracy. However, we do not see how the conflict model could account for the effect of DBS on accuracy being only significant in the integration task but not other tasks in our study. By contrast, the model described in this paper was not extended to capture the effect of DBS on reaction time in our task (it would be an interesting direction for future work), but it does explain the selective effect of DBS on accuracy in the integration task. In particular, it suggests that in the integration task, in which more stimuli were presented than in other tasks, the boundaries on the range of allowed activity are more likely to be reached due to longer lack of subthalamic nucleus normalization. Reaching these boundaries reduces the difference between activities of neurons selective for the two responses and thus reduces the probability of the correct response.

For most currently used choice tasks, the conflict model (Frank, 2006) and the Bayesian model (Bogacz and Gurney, 2007; Bogacz and Larsen, 2011), that we adapted in this paper for our task, make similar predictions on the activity of the subthalamic nucleus, because the normalization provided by the subthalamic nucleus according to the Bayesian model is higher on high conflict trials (due to the mathematical definition of the normalization - see definition of SG(t) in Figure 3b in Bogacz & Larsen, 2011). Consequently, Zaghloul et al. (2012), who observed higher activity of the subthalamic nucleus on neurons on high conflict trials, wrote that their results were predicted by both models. We feel that to investigate differences between these models, one will need to design new tasks for which the models make very different predictions.

Conclusions

We have demonstrated that response choice requiring information integration is impaired as a result of subthalamic nucleus DBS and associated with a failure to slow down in order to incorporate new information before taking a decision. Therefore, our data are consistent with the model of probabilistic decision making where the subthalamic nucleus is critical for the Bayesian normalization of accumulated information. Furthermore, we have uncovered an effect of dopamine depletion on memory retention > 20 min, which is somewhat ameliorated by dopamine replacement. In addition to illuminating the neurobiology and the role of the subthalamic nucleus in decision-making networks, these data may have clinical implications including the potential for use of dopamine as a therapeutic agent in conditions where memory retention is impaired.

Funding

This work was funded by the Severnside Alliance for Translational Research (SARTRE) as part of the MRC devolved DPFS portfolio. AW is also supported by the Burden Trust.

Supplementary material

Supplementary material is available at Brain online.

Supplementary Data
supp_135_12_3721__index.html (983B, html)

Acknowledgements

The authors thank Ewa Bogacz for drawing the stimuli, Masud Husain for comments on the pilot behavioural paradigm, Kevin Lloyd for discussion and reading a previous version of the manuscript and Peter Sellars and Amy Laverock for help with patient testing.

Glossary

Abbreviations

DBS

deep brain stimulation

UPDRS

Unified Parkinson’s Disease Rating Scale

References

  1. Bethus I, Tse D, Morris RG. Dopamine and memory: modulation of the persistence of memory for novel hippocampal NMDA receptor-dependent paired associates. J Neurosci. 2010;30:1610–18. doi: 10.1523/JNEUROSCI.2721-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bogacz R, Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 2007;19:442–77. doi: 10.1162/neco.2007.19.2.442. [DOI] [PubMed] [Google Scholar]
  3. Bogacz R, Larsen T. Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Comput. 2011;23:817–51. doi: 10.1162/NECO_a_00103. [DOI] [PubMed] [Google Scholar]
  4. Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psycholo Rev. 2006;113:700–65. doi: 10.1037/0033-295X.113.4.700. [DOI] [PubMed] [Google Scholar]
  5. Calabresi P, Saiardi A, Pisani A, Baik JH, Centonze D, Mercuri NB, et al. Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J. Neurosci. 1997;17:4536–44. doi: 10.1523/JNEUROSCI.17-12-04536.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cavanagh JF, Wiecki TV, Cohen MX, Figueroa CM, Samanta J, Sherman SJ, et al. Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nat Neurosci. 2011;14:1462–7. doi: 10.1038/nn.2925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cetonze D, Picconi B, Gubellini P, Bernardi G, Calabresi P. Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur J Neurosci. 2001;13:1071–7. doi: 10.1046/j.0953-816x.2001.01485.x. [DOI] [PubMed] [Google Scholar]
  8. Cools R, Barker RA, Sahakian BJ, Robbins TW. Enhanced or impaired cognitive function in parkinson's disease as a function of dopaminergic medication and task demands. Cereb Cortex. 2001;11:1136–43. doi: 10.1093/cercor/11.12.1136. [DOI] [PubMed] [Google Scholar]
  9. Cools R, Barker RA, Sahakian BJ, Robbins TW. l-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson’s disease. Neuropsychologia. 2003;41:1431–41. doi: 10.1016/s0028-3932(03)00117-9. [DOI] [PubMed] [Google Scholar]
  10. Deuschl G, Schade-Brittinger C, Krack P, Volkmann J, Schäfer H, Bötzel K, et al. A randomized trial of deep-brain stimulation for Parkinson’s disease. N Engl J Med. 2006;355:896–908. doi: 10.1056/NEJMoa060281. [DOI] [PubMed] [Google Scholar]
  11. Dujardin K, Blairy S, Defebvre L, Krystkowiak P, Hess U, Blond S, et al. Subthalamic nucleus stimulation induces deficits in decoding emotional facial expressions in Parkinson’s disease. J Neurol Neurosurg Psychiatry. 2004;75:202–8. [PMC free article] [PubMed] [Google Scholar]
  12. Egelman DM, Person C, Montague PR. A computational role for dopamine delivery in human decision-making. J Cogn Neurosci. 1998;10:623–30. doi: 10.1162/089892998563022. [DOI] [PubMed] [Google Scholar]
  13. Fahn S, Elton RL. Recent developments in Parkinson's disease. 1987. The Unified Parkinson's Disease Rating Scale Fahn S, Marsden CD, Calne D, Goldstein M, editors. Recent developments in Parkinson's disease. Florham Park, NJ: MacMillan Healthcare Information; 1987. p. 153-63. [Google Scholar]
  14. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  15. Follett KA, Weaver FM, Stern M, Hur K, Harris CL, Luo P, et al. Pallidal versus subthalamic deep-brain stimulation for Parkinson's disease. N Engl J Med. 2010;362:2077–91. doi: 10.1056/NEJMoa0907083. [DOI] [PubMed] [Google Scholar]
  16. Folstein MF, Folstein SE, McHugh PR. Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  17. Frank MJ. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw. 2006;19:1120–36. doi: 10.1016/j.neunet.2006.03.006. [DOI] [PubMed] [Google Scholar]
  18. Frank MJ, Seeberger LC, O'Reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 2004;306:1940–43. doi: 10.1126/science.1102941. [DOI] [PubMed] [Google Scholar]
  19. Frank MJ, Samanta J, Moustafa AA, Sherman SJ. Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science. 2007;318:1309–12. doi: 10.1126/science.1146157. [DOI] [PubMed] [Google Scholar]
  20. German DC, Manaye K, Smith WK, Woodward DJ, Saper CB. Midbrain dopaminergic cell loss in Parkinson's disease: computer visualisation. Ann Neurol. 1989;26:507–14. doi: 10.1002/ana.410260403. [DOI] [PubMed] [Google Scholar]
  21. Hälbig TD, Tse W, Frisina PG, Baker BR, Hollander E, Shapiro H, et al. Subthalamic deep brain stimulation and impulse control in Parkinson’s disease. Eur J Neurol. 2009;16:493–97. doi: 10.1111/j.1468-1331.2008.02509.x. [DOI] [PubMed] [Google Scholar]
  22. Hälbig TD, Gruber D, Kopp UA, Scherer P, Schneider GH, Trottenberg T, et al. Subthalamic stimulation differentially modulates declarative and nondeclarative memory. NeuroReport. 2004;15:539–43. doi: 10.1097/00001756-200403010-00031. [DOI] [PubMed] [Google Scholar]
  23. Hershey T, Campbell MC, Videen TO, Lugar HM, Weaver PM, Hartlein J, et al. Mapping Go–No-Go performance within the subthalamic nucleus region. Brain. 2010;133:3625–34. doi: 10.1093/brain/awq256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1998;1:304–09. doi: 10.1038/1124. [DOI] [PubMed] [Google Scholar]
  25. Isoda M, Hikosaka O. Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci. 2008;28:7209–18. doi: 10.1523/JNEUROSCI.0487-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Knowlton BJ, Mangels JA, Squire LR. A neostriatal habit learning system in humans. Science. 1996;273:1399–402. doi: 10.1126/science.273.5280.1399. [DOI] [PubMed] [Google Scholar]
  27. Lawrence AD, Calder AJ, McGowan SW, Grasby PM. Selective disruption of the recognition of facial expressions of anger. Neuroreport. 2002;13:881–84. doi: 10.1097/00001756-200205070-00029. [DOI] [PubMed] [Google Scholar]
  28. Lisman JE, Grace AA. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron. 2005;46:703–13. doi: 10.1016/j.neuron.2005.05.002. [DOI] [PubMed] [Google Scholar]
  29. Mattau E, Dupré N, Langlois M, Jean L, Thivierge S, Provencher P, et al. Mattis dementia rating scale 2: screening for MCI and dementia. Am J Alzheimers Dis Other Demen. 2011;26:389–98. doi: 10.1177/1533317511412046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Parent A, Hazrati LN. Functional anatomy of the basal ganglia. II. Tha place of subthalamic nucleus and external pallidum in basal ganglai circuitry. Brain Res Brain Res Rev. 1995;20:128–54. doi: 10.1016/0165-0173(94)00008-d. [DOI] [PubMed] [Google Scholar]
  31. Patel NK, Plaha P, Gill SS. Magnetic resonance imaging-directed method for functional neurosurgery using implantable guide tubes. Neurosurgery. 2007;61:358–66. doi: 10.1227/01.neu.0000303994.89773.01. [DOI] [PubMed] [Google Scholar]
  32. Patton JH, Stanford MS, Barratt ES. Factor structure of the barratt impulsiveness scale. J Clin Psychol. 1995;51:768–74. doi: 10.1002/1097-4679(199511)51:6<768::aid-jclp2270510607>3.0.co;2-1. [DOI] [PubMed] [Google Scholar]
  33. Plaha P, Ben-Shlomo Y, Patel NK, Gill SS. Stimulation of the caudal zona incerta is superior to stimulation of the subthalamic nucleus in improving contralateral parkinsonism. Brain. 2006;129:1732–47. doi: 10.1093/brain/awl127. [DOI] [PubMed] [Google Scholar]
  34. Poldrack RA, Packard MG. Competition among multiple memory systems: converging evidence from animal and human brain studies. Neuropsychologia. 2003;41:245–51. doi: 10.1016/s0028-3932(02)00157-4. [DOI] [PubMed] [Google Scholar]
  35. Ratcliff R, Frank MJ. Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Comput. 2012;24:1186–29. doi: 10.1162/NECO_a_00270. [DOI] [PubMed] [Google Scholar]
  36. Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, Bergman H, et al. Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease. Nat Rev Neurosci. 2010;11:760–72. doi: 10.1038/nrn2915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rodriguez-Oroz MC, Rodriguez M, Guridi J, Mewes K, Chockkman V, Vitek J, et al. The subthalamic nucleus in Parkinson's disease: somatotopic organization and physiological characteristics. Brain. 2001;124:1777–90. doi: 10.1093/brain/124.9.1777. [DOI] [PubMed] [Google Scholar]
  38. Rodriguez-Oroz MC, López-Azcárate J, Garcia-Garcia D, Alegre M, Toledo J, Valencia M, et al. Involvement of the subthalamic nucleus in impulse control disorders associated with Parkinson’s disease. Brain. 2011;134:36–49. doi: 10.1093/brain/awq301. [DOI] [PubMed] [Google Scholar]
  39. Shiner T, Seymour B, Wunderlich K, Hill C, Bhatia KP, Dayan P, et al. Dopamine and performance in a reinforcement learning task: evidence from Parkinson’s disease. Brain. 2012;135:1871–83. doi: 10.1093/brain/aws083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Shohamy D, Adcock R. Dopamine and adaptive memory. Trends Cogn Sci. 2010;14:464–72. doi: 10.1016/j.tics.2010.08.002. [DOI] [PubMed] [Google Scholar]
  41. Smith MA, Ghazizadeh A, Shadmehr R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 2006;4:e179. doi: 10.1371/journal.pbio.0040179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Sockeel P, Dujardin K, Devos D, Denève C, Destée A, Defebvre L. The Lille apathy rating scale (LARS), a new instrument for detecting and quantifying apathy: validation in Parkinson’s disease. J Neurol Neurosurg Psychiatry. 2006;77:579–84. doi: 10.1136/jnnp.2005.075929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Usher M, McClelland JL. The time course of perceptual choice: the leaky, competing accumulator model. Psycholog Rev. 2001;108:550–92. doi: 10.1037/0033-295x.108.3.550. [DOI] [PubMed] [Google Scholar]
  44. Voon V, Krack P, Lang AE, Lozano AM, Dujardin K, Schüpbach M, et al. A multicentre study on suicide outcomes following subthalamic stimulation for Parkinson's disease. Brain. 2008;131:2720–8. doi: 10.1093/brain/awn214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Weaver FM, Follett K, Stern M, Hur K, Harris C, Marks WJ, Jr, et al. Bilateral deep brain stimulation vs best medical therapy for patients with advanced Parkinson disease. JAMA. 2009;301:63–73. doi: 10.1001/jama.2008.929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Welberg L. Reward: dopamine's expanding universe. Nat Rev Neurosci. 2009;10:624–5. [Google Scholar]
  47. Williams A, Gill S, Varma T, Jenkinson C, Quinn N, Mitchell R, et al. Deep brain stimulation plus best medical therapy versus best medical therapy alone for advanced Parkinson's disease (PD SURG trial): a randomised, open-label trial. Lancet Neurol. 2010;9:581–91. doi: 10.1016/S1474-4422(10)70093-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Wylie SA, Ridderinkhof KR, Elias WJ, Frysinger RC, Bashore TR, Downs KE, et al. Subthalamic nucleus stimulation influences expression and suppression of impulsive behaviour in Parkinson’s disease. Brain. 2010;133:3611–24. doi: 10.1093/brain/awq239. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Yang T, Shadlen MN. Probabilistic reasoning by neurons. Nature. 2007;447:1075–80. doi: 10.1038/nature05852. [DOI] [PubMed] [Google Scholar]
  50. Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7:464–76. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]
  51. Zaghoul KA, Weidemann CT, Lega BC, Jaggi JL, Baltuch GH, Kahana MJ. Neuronal activity in the human subthalamic nucleus encodes decision conflict during action selection. J Neurosci. 2012;32:2453–60. doi: 10.1523/JNEUROSCI.5815-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
supp_135_12_3721__index.html (983B, html)
supp_aws273_covering_letter_Coulthard_18th_July_2012.doc (79KB, doc)
supp_aws273_Supplementary_material_Coulthard_et_al_June_13th.doc (4.1MB, doc)

Articles from Brain are provided here courtesy of Oxford University Press

RESOURCES