Dissociable effects of dopamine on learning and performance within sensorimotor striatum

Abstract

Striatal dopamine is an important modulator of current behavior, as seen in the rapid and dramatic effects of dopamine replacement therapy in Parkinson Disease (PD). Yet there is also extensive evidence that dopamine acts as a learning signal, modulating synaptic plasticity within striatum to affect future behavior. Disentangling these “performance” and “learning” functions is important for designing effective, long-term PD treatments. We conducted a series of unilateral drug manipulations and dopamine terminal lesions in the dorsolateral striatum of rats highly-trained to perform brief instructed head/neck movements (two-alternative forced choice task). Reaction times and accuracy were measured longitudinally to determine if task behavior changed immediately, progressed over time, and/or persisted after drug withdrawal. Enhanced dopamine signaling with amphetamine caused an immediate, nonprogressive, and bilateral decrease in reaction times (RT). The altered RT distributions were consistent with reduced distance to threshold in the linear approach to threshold with ergodic rate (LATER) model of decision-making. Conversely, the dopamine antagonist flupenthixol caused experience-dependent, persistent changes in RT and accuracy indicative of a “learning” effect. These RT distributions were consistent with a slowed rate of approach to decision threshold. Our results show that dopaminergic signaling makes dissociable contributions to current and future behavior even within a single striatal subregion, and provide important clues for both models of normal decision-making and the design of novel drug therapies in PD.

INTRODUCTION

Difficulty initiating movements is a core symptom of Parkinson Disease (PD) that is markedly improved by dopamine replacement therapy [1]. However, the precise mechanisms by which striatal dopamine loss interferes with, and dopamine replacement improves, movement initiation remain unclear.

The short duration response (SDR) to levodopa is an acute improvement in motor function correlated with plasma levodopa levels [2,3]. In the standard “rate” model of the basal ganglia (BG) [4,5], the SDR arises from dopaminergic effects on striatal medium spiny neuron (MSN) firing rates, leading to reduced activity of BG output neurons that tonically suppress behavior. Alternatively, levodopa may help restore normal patterns of BG activity [6–9], instead of the excessive bursting [10], synchrony [11], and oscillations [12] precipitated by dopamine loss. Whether dopamine influences firing rates or patterns, the speed with which levodopa and other dopaminergic drugs affect behavior is consistent with a critical role for the BG in online motor performance [13,14]. Indeed, behavior can be rapidly altered by optogenetic [15], electrical [16], or pharmacologic [17–19] manipulations of BG circuits.

The long duration response (LDR) is persistent motor improvement after levodopa elimination [20]. It may arise through dopamine-modulated synaptic plasticity that normally supports reinforcement learning [21–25]. With decreased striatal dopamine signaling, normal learning of motor skills is impaired [26], aberrant learning can occur [27], and established performance of various operant tasks may be initially preserved but decline with practice (“experience-dependent” effects; [28,29]). BG output is important for the acquisition of motor sequences [30], but may not be required to perform previously learned sequences [31,32]. Some therefore argue that striatal circuitry with dopamine-dependent plasticity acts primarily as a trainer for other subregions [33] or different brain regions altogether [34,35].

Intracerebral infusions are a powerful means of dissecting the contributions of specific neurotransmitter systems in specific brain regions to behavior. For example, bilateral inactivation of striatal subregions with muscimol can force rats to transition between goal-directed and habitual behavior [36,37], and bilateral intrastriatal infusion of dopamine antagonists reproduces the catalepsy induced by systemically administered neuroleptics [38–41]. Bilateral intrastriatal infusions of amphetamine post-training have been found to improve retention for certain types of tasks [42]. Unilateral putaminal dopamine receptor blockade in nonhuman primates induces contralateral parkinsonism [43]. Finally, intra-caudate infusion of D1 or D2 antagonists modifies the apparent effects of reward expectation on reaction time (RT) [44]. Most of these studies have not investigated striatal contributions to action selection, and none have attempted to distinguish the contributions of striatal dopaminergic signaling to skill acquisition/maintenance vs acute performance.

To help disentangle “performance” and “learning” functions, we transiently suppressed the output of, or altered dopaminergic signaling in, rat dorsolateral striatum (DLS) unilaterally. Rodent DLS is a sensorimotor subregion homologous to human lateral/posterior putamen, the earliest region affected by dopamine loss in PD [45]. The rats engaged in a lateralized reaction time task (a.k.a. two-alternative forced choice, or conditional discrimination [46,47]) in which tone cues prompt brief head/neck movements to the left or right. Behavior was assessed primarily by the fraction of correct responses and RT, including examination of full RT distributions to make inferences about specific striatal contributions to decision-making processes. Drug manipulations were considered to affect learning if behavioral changes were experience-dependent, and/or persisted after drug withdrawal. These selective, reversible interventions then informed the interpretation of more PD-like permanent destruction of dopaminergic terminals using 6-hydroxydopamine (6-OHDA) infusions into DLS.

METHODS

Animals

All animal procedures were approved by the University of Michigan Committee on the Use and Care of Animals. Adult male Long-Evans rats were housed on a 12 hour reversed light-dark cycle, and tested during the dark phase. Rats were food restricted to 15 g of standard laboratory chow per day, but allowed to free-feed one day per week and in the peri-operative period. Their weights were monitored to ensure that they maintained at least 85% of anticipated body weight for their age.

Behavioral task

The operant chamber (Med Associates, St. Albans, VT) and task performance are illustrated in Figure 1 (see also the “Immediate-Go” task of Leventhal et al [47]). One of the three central nose-ports was lit. The rat had an unlimited amount of time to poke and maintain its nose in the lit port, initiating a trial. After a variable delay (uniform distribution from 750–1250 ms), a 1 or 4 kHz pure tone (~60 dB) played for 250 ms, indicating that the rat should move one port to the left or right, respectively. Simultaneous with tone onset, the center port light was extinguished and both adjacent ports were lit. From the start of the tone, the rat was allowed a maximum of 1 s (the “limited hold”, Figure 1c) to poke an adjacent port. Correct responses (the infrared beam for the correct side-port was broken first) were rewarded with a 45 mg fruit punch flavored sucrose pellet delivered at the back of the chamber. On 20% of trials, a linear summation of 4 kHz and 1 kHz tones indicated that either adjacent port would be rewarded with 50% probability (“catch” trials). Each test session lasted one hour. Intertrial intervals were pulled from a uniform distribution between 15–25 s.

Figure 1.

Task performance and infusion sites. a) Schematic diagram of the operant box. Red dashed lines indicate photobeams. b) Illustration of task events c) Timeline of task performance. Thick black bars indicate a nose-port is occupied. The gray bar indicates the cue tone. The gray shaded area indicates the limited hold period (1 s). d) Histologically verified infusion sites projected onto an AP = +0.48 mm coronal atlas section (A-P range −0.72 to + 1.2 mm with respect to bregma) [107]. Small black dots – MUSC, red dots – AMPH, light blue dots – LOW FLU, dark blue dots – HIGH FLU, green dots - RAC, yellow dots - SCH, orange dots – 6OHDA.

Trials were classified as “procedure errors” or “completed trials.” Procedure errors were “wrong starts” (poking an unlit port to initiate a trial), “false starts” (withdrawing from the center port before the cue tone played), and “failures to respond” (failure to poke a side port within 1 s of the tone). “Completed trials” included “incorrect trials” (poking the wrong lit side-port) and “correct trials.” For “catch” trials, no distinction was made between “correct” and “incorrect” completed trials, regardless of whether they were rewarded. After procedural errors, but not completed trials, the house light was lit during the intertrial interval.

Training sequence

Training before surgery lasted about 2 months. Rats were handled daily for one week prior to beginning operant testing, then habituated to the chamber for one to two test sessions during which all ports were lit and poking any port was rewarded. They were then trained to maintain their noses in a single lit port for progressively longer periods to obtain a reward, signaled by a 250 ms burst of white noise. Finally, pure tone instruction cues replaced the white noise at the end of the hold period, and the rats were allowed progressively less time to poke an adjacent port. When they consistently achieved 80% choice accuracy (defined below) with a 1 s limited hold (Figure 1c), they were eligible for cannula implantation. During training, a bias correction was implemented so that if rats moved in the same direction for three consecutive trials, the next instruction cue was to the opposite side. This bias correction was removed for test sessions.

Cannula implants and drug infusions

Guide cannulas (Plastics1, Roanoke, VA) were implanted under isoflurane anesthesia into the left or right striatum at coordinates A-P +0.5 mm, M-L +/−3.5 mm with respect to bregma, and D-V 3.5 mm from the brain surface (Figure 1d). Infusion cannulas extended 1 mm beyond the end of the guide cannula, so that the target infusion site was at D-V 4.5 mm. “Dummy” cannulas were changed daily to maintain patency of the guides. Rats were given one week to recover from surgery before testing began. Beginning 20 minutes prior to behavioral testing, 0.5 μL of either artificial cerebrospinal fluid (aCSF; Harvard Apparatus, Holliston, MA - “vehicle” infusions) or drug dissolved in aCSF was infused via syringe pump (Harvard Apparatus) at a rate of 0.1 μL/min. The infusion cannula was left in place for at least one minute to allow diffusion of drug, then was slowly withdrawn and replaced with a dummy cannula prior to testing. Each rat received a baseline, control infusion of aCSF (“vehicle 1”) followed by one or more days of active drug, labeled as “drug 1”, “drug 2”, etc. Most groups also received a “vehicle 2” infusion after the last drug infusion.

All drugs were obtained from Sigma-Aldrich (St. Louis, MO). Initial doses were based on prior studies: the GABA agonist muscimol [36]; low-dose: 0.05 μg, n = 5; high dose: 0.5 μg, n = 8), the indirect dopamine agonist amphetamine [42]; 5.0 μg, n = 10), the D1/D2 receptor antagonist flupenthixol [48]; low dose: 6 μg, n = 11; high dose: 30 μg, n = 13), the D1 receptor antagonist SCH23390 [49]; 2 μg, n = 16), and the D2 receptor antagonist raclopride [50]; 25 μg, n = 16; a pilot study using 5 μg showed no clear behavioral effects). Some of the low-dose muscimol observations have been reported as supplemental data to an electrophysiology study [46] (none of the animals in the present study were implanted with electrodes). A subset of the muscimol-treated rats subsequently became the subjects for the amphetamine experiment.

Dopamine terminal lesions

These were created using 6-OHDA stabilized with ascorbic acid (15 μg in 3.0 μL aCSF, infused at 0.5 μL/min). Desipramine 25 mg/kg was administered IP 30 minutes prior to 6-OHDA infusions to protect noradrenergic neurons. 6-OHDA lesioned rats were retested on the operant task beginning two weeks after the lesioning procedure, allowing adequate time for the destruction of dopaminergic terminals [51]. Operant performance was tested 30 minutes after IP injection of saline for the first seven days, and 30 minutes after IP levodopa/benserazide (6/15 mg/kg) for the second seven days.

Histology

Rats were perfused with 4% paraformaldehyde. Their brains were removed, stored in 4% paraformaldehyde for 24 hours, then cryoprotected with 30% sucrose in phosphate buffered saline. 45 μm sections were taken in the coronal plane, and cannula locations were verified by Nissl staining. Lesions of dopaminergic terminals were verified by immunohistochemistry against TH. Free-floating sections in phosphate-buffered saline (PBS) were pretreated with 0.3% Triton X-100, then labeled with rabbit anti-TH primary antibody (ab112, Abcam, Cambridge, MA; or P40101-0, Pel-Freez Biologicals, Rogers, AR) at 1:1000 dilution, and visualized using a standard avidin-biotin complex peroxidase reaction (Vectastain Elite, Vector Laboratories, Burlingame, CA). Of 19 6-OHDA-lesioned subjects, 7 had lesions confined to DLS as assessed by an investigator blinded to behavioral outcomes; only data from this subset were used.

Data analysis

Directional movement preferences were quantified in two ways. Choice “accuracy” was calculated (separately for movements directed ipsi- and contralaterally to the infusion site) as the number of correct trials divided by the number of completed trials. “Bias” was defined as the number of contralateral responses on catch trials divided by the number of completed catch trials. A bias of less than 0.5 indicated a disposition towards ipsilateral responses.

Reaction time (RT, Figure 1c) was defined as the interval from the start of the tone to the Nose Out event. It was analyzed separately for ipsi- and contralaterally instructed trials, without regard to which direction the rat actually moved, for which a Nose Out event occurred before the limited hold expired. This included trials for which no Side In event occurred (and the movement direction could therefore not be determined), which was necessary to ensure that median RTs were not artificially shortened by excluding long-RT trials. Movement time (MT, Figure 1c) was defined as the interval from Nose Out to Side In, and was assessed for all completed trials. Medians of single session RT and MT distributions were averaged across subjects to make the plots in figures 2; 5a, b; and 6b.

Figure 2.

Choice accuracy, median reaction time, and median movement time for muscimol, amphetamine, and flupenthixol. a) low-dose muscimol, b) amphetamine, c) low-dose flupenthixol, and d) high-dose flupenthixol. Error bars indicate the standard error in the mean. Black squares – contralaterally-directed movements; gray circles – ipsilaterally-directed movements. Points connected by lines indicate the same treatment was administered on consecutive days. Bold borders indicate at least one significant difference (p < 0.05) between treatment conditions (see Results). V = “vehicle” (aCSF) infusion. Numbers on the x-axis indicate the session number for the infusion of active drug.

Figure 5.

Results summary for RAC and SCH23390 infusions. Choice accuracy, RT, and MT for a) SCH23390, and b) RAC infusions. Black squares – contralaterally-directed movements; gray circles – ipsilaterally-directed movements. Points connected by lines indicate the same treatment was administered on consecutive days. V = “vehicle” (aCSF) infusion. Error bars represent the standard error in the mean. c) Cumulative density function of correlation coefficients between contralateral RT and trial number within sessions after aCSF (black) or SCH23390 (gray) infusion. d) Same as c for aCSF (black) and RAC (gray) infusions. Bold borders indicate at least one statistically significant finding (see Results).

Figure 6.

Dopamine terminal lesions in DLS cause experience-dependent changes in task performance that spontaneously recover. a) Representative TH-stained sections from a single rat illustrating a lesion confined to DLS. b) choice accuracy, RT, and MT at baseline and during repeated testing beginning 14 days after lesioning. Day 0 is the baseline session; lesions were performed on day 1. Before the first seven sessions (days 15–21), rats received IP saline injections; before the next seven sessions (days 22–28), they received IP levodopa. c) vincentized RT CDFs for saline (blue) and levodopa (red) sessions. d) CDFs of correlation coefficients between contralateral RT and trial number within individual sessions. Bold borders indicate at least one statistically significant finding (see Results).

Analysis of full RT distributions can provide additional insight into decision-making mechanisms [52]. As the shape of RT distributions is not necessarily preserved when aggregating RT across subjects, a Vincentization procedure was used [53] (Figure 4b). The RT cumulative density function (CDF) for each session was linearly interpolated to obtain an estimate of RT at 0.05 quantile intervals. RT at each quantile was then averaged across subjects to produce the Vincentized distribution (that is, averaging “horizontally” across the individual RT CDFs in Figure 4b; see also Figures 3b, d; Figure 6c). These distributions were fed into the LATER model [54] to make inferences about potential underlying processes affected by specific interventions.

Figure 4.

Reciprobit analysis of Vincentized RT distributions. a) Simulated data illustrating the LATER model. The left column illustrates the effects of changing the distance to threshold (swivels); the right column illustrates the effects of changing the mean rate of rise (shifts). i - movement initiation occurs when a signal (black lines) crosses a threshold (top), resulting in skewed RT distributions (bottom). Gray lines indicate +/− 2 standard deviations from the mean rate of signal increase. μ indicates the baseline mean rate of approach; μ’ indicates the altered mean rate of approach. ii - Cumulative density functions (CDFs) of the RT distributions in (i), plotted as a function of 1/RT. iii - reciprobit plots of the CDFs in (ii) – the y-axis is scaled by z-scores instead of probabilities. 1/RT axes are reversed so that RT increases to the right. The figure was generated by randomly selecting 100,000 approach rates from normal distributions with different means. b) i – RT distributions from all rats for a single amphetamine session, ii – corresponding RT CDFs (red) and the resulting Vincentized distribution (black), iii – reciprobit plot of the Vincentized RT CDF. Shaded bars at each quantile indicate the distribution of bootstrapped CDFs (darker colors indicate more likely values). The red solid line is the best fit to the “main” distribution; the dashed red line is the best fit to the “express” distribution (see Methods). Inset – histogram of bootstrapped results at a single quantile; the vertical red line indicates where the linear fit to the main distribution falls within the bootstrapped results. c) sample reciprobit plots for contralateral responses in vehicle (black circles) and drug sessions (colored pluses). i – amphetamine, showing best fits under the constraint of a constant RT = ∞ intercept; ii – HIGH FLU, showing best fits under the constraint of a constant slope. 1/RT axes are reversed so that RT increases to the right. d) summary of log-likelihood differences for AMPH, HIGH FLU (FLU), SCH23390 (SCH), and RAC sessions. Ipsilateral RTs are to the left and contralateral to the right. Note the small log-likelihoods clustered around zero for ipsilateral HIGH FLU results, as expected for an intervention that did not affect median RT (indicated by light shading). The dashed gray line indicates a log-likelihood difference of 3 (the level at which one alternative is 20 times more likely than the other). Large black circles indicate the sessions illustrated in c.

Figure 3.

Inter- and intrasession effects of amphetamine and HIGH FLU on contralateral RT. a) RT distributions for a single rat across vehicle (aCSF) and sequential amphetamine infusions. Distributions were normalized to maintain the same area under the curve for each session and smoothed with a 3-point moving average. b) Vincentized contralateral RT cumulative density functions (CDFs) for amphetamine sessions. Note that amphetamine sessions are tightly overlaid with each other. c) same as (a) for sequential HIGH FLU infusions. d) same as b for HIGH FLU infusions. Color codes for the (a, b) and (c, d) pairs are the same. e) scatter plot of contralateral RT as a function of trial number for a single amphetamine session (r = 0.05, p = 0.76) f) cumulative density functions (CDFs) of correlation coefficients between CONTRA RT and trial number (as illustrated in e) for vehicle (black) and amphetamine (red) infusions. g) same as e, but for a single HIGH FLU session (r = 0.50, p = 3 × 10⁻⁴). h) same as f, but for HIGH FLU (blue). Thick axis borders indicate a statistically significant effect in that panel (see Results).

In the LATER model, a signal approaches decision threshold at a constant rate on each trial. This rate is normally distributed across trials, explaining the typical “Gaussian with an exponential tail” RT distribution (Figure 4a) [55]. Changes in the mean approach rate or the distance to threshold alter RT distributions in specific ways, which can be evaluated quantitatively with reciprobit plots. The RT CDF is plotted on a probit axis against 1/RT (Figure 4). Such plots result in a straight line whose slope is proportional to the distance to threshold (but independent of mean approach rate), while the intercept with the RT = ∞ axis is proportional to the mean approach rate (but independent of the distance to threshold). Note that changes in the starting position or the absolute threshold (or both) alter the distance to threshold, and are equivalent in this model. In practice, RT distributions often comprise “express” and “main” components, which manifest as a bend in the reciprobit plot [54] (Figure 4b). A “swivel” of the main distribution about the RT = ∞ intercept indicates modulation of the initial distance to threshold, while a parallel “shift” of the main distribution indicates a change in the mean rate of approach to threshold.

To apply the LATER model to the Vincentized RT distributions, straight lines were fit to the express and main distributions by minimizing a weighted sum of squares. This involved optimizing the slopes and intercepts of the lines, as well as identifying the quantile at which the “bend” between express and main distributions occurred. The variance of RT at each quantile in the reciprobit plots of the Vincentized RT CDFs was estimated by resampling the RT distributions for each subject 5000 times (Fig. 4biii, inset), and the residuals were weighted by the estimated variance at each quantile. This procedure was necessary because of the greater spread in RT at lower quantiles (Fig. 4biii). The express distribution was modeled as a LATER process with mean accumulation rate of zero (that is, an RT = ∞ intercept of zero) [54]. The “bend” quantile was identified as the point at which the weighted sum of squares was minimized for both the express and main distributions.

To determine whether a “swivel” or “shift” best fit the data, log-likelihood functions for linear fits to the main distributions were estimated:

$$L=-ln\left({\prod }_{i=1}^{N}P({RT}_i={RT}_{i,\mathit{pred}})\right),$$

where N is the number of main distribution quantiles (for both baseline and drug infusion sessions), RT_{i, pred} is the predicted (fit) RT at each quantile, and P(RT_i = RT_{i, pred}) is the probability that the “true” Vincentized RT at quantile i is equal to the predicted RT. At each quantile, P(RT_i = RT_{i, pred}) was calculated by assuming that RTs were normally distributed with a mean of the observed Vincentized RT and variance obtained by the bootstrapping procedure (see Fig. 4biii).

These likelihood functions were minimized for pairs of Vincentized CDFs under the constraint of either an invariant RT = ∞ intercept (a “swivel”) or slope (a “shift”). The difference of the minimum log likelihoods provides a measure of how much better a “swivel” or “shift” accounts for the observed data [55]. For example, a log-likelihood difference of 3 indicates that one possibility is e³ = 20 times more likely than the other. All minimization procedures were carried out using the fminsearch function in Matlab, and repeated 100 times using random starting points to ensure that convergence was not to a local minimum.

Statistics

To test the null hypothesis that a behavioral measure was invariant across test sessions, a resampling procedure was used (a repeated-measures “permutation ANOVA”) [56]. This bootstrapping technique does not assume that the data are normally distributed, which is particularly important when comparing choice accuracy across sessions. It also readily handles missing values, which occurred in rare instances when individual rats were substantially impaired in task performance. The standard ANOVA F statistic was calculated for the observed data (F_obs). The outcome measures (e.g., median RT) for individual sessions were then shuffled 10,000 times under the constraint that data could only be shuffled within the same subject. For each permutation, the F-statistic (F_perm) was again calculated, and the p-value was taken as the fraction of F_perm’s that exceeded F_obs. To test whether an intervention influenced the variable of interest, vehicle sessions were included in the analysis. To test whether an effect changed with repeated testing, only “drug” sessions were included. All reported p-values are for the permutation ANOVA unless otherwise noted. All ranges reported in the figures and text are mean ± sem.

RESULTS

Suppression of dorsolateral striatum output impairs contralateral choice

We first investigated the effects of unilateral DLS suppression on task performance. Rats were tested over three consecutive days, with each test session preceded by an infusion of either aCSF (“vehicle”, days 1 and 3) or muscimol (day 2, 0.05 or 0.5 μg). Rats infused with high-dose (0.5 μg) muscimol completed very few trials (mean 12.1 ± 4.3 trials/session), preventing meaningful interpretation of accuracy, bias, and RT. After low-dose (0.05 μg) muscimol infusions, the decrease in trial number was moderate (124.4 ± 10.8, 91.2 ± 15.1, and 137.8 ± 8.8 trials attempted for vehicle 1, muscimol, and vehicle 2 infusions respectively, p < 10⁻⁴, F_obs = 10.9), and we observed a selective impairment in contralateral responding. This was seen as substantially decreased accuracy when rats were cued to move contralaterally to the infusion site (CONTRA trials; Figure 2ai, p = 0.004, F_obs = 3.59) but not when cued to move ipsilaterally (IPSI trials; p = 0.101, F_obs = 1.45). Furthermore, muscimol infusion produced a bias towards ipsilateral responses on catch trials (bias = 0.320 ± 0.085, 0.243 ± 0.104, and 0.402 ± 0.099 for vehicle 1, muscimol, and vehicle 2 sessions; p = 0.032, F_obs = 2.10). Critically, neither effect persisted in the second vehicle session (comparisons between vehicle 1 and vehicle 2 sessions: p = 0.310 for CONTRA accuracy, p = 0.524 for bias, rank sum tests). Muscimol did not affect median RT (Figure 2aii) for either direction. CONTRA MT increased slightly during the muscimol session (p = 0.001, F_obs = 3.80, Figure 2aiii) with resolution during the second vehicle session (p = 0.246 for vehicle 1 to vehicle 2 comparison, rank sum test). The ability of muscimol to selectively and reversibly interfere with contralateral choices indicates that DLS actively participates in online action selection even in a highly trained task, rather than having only a transient role in task acquisition.

Enhanced dopaminergic signaling in DLS acutely speeds RT without affecting choice

Next, we examined the effects of repeated amphetamine infusions into DLS. Following a baseline control (aCSF, day 1), amphetamine was infused prior to each of five consecutive test sessions (days 2–6). Amphetamine had no effect on the number of trials attempted (p = 0.151, F_obs = 0.707) or response accuracy (Figure 2bi; p = 0.536, F_obs = 0.568 for IPSI trials; p = 0.910, F_obs = 0.053 for CONTRA trials), and rats did not develop a response bias on catch trials (p = 0.445, F_obs = 0.500). However, RT substantially decreased after amphetamine infusions (Figure 2bii) for both directions (p = 4.0 × 10⁻⁴, F_obs = 0.929 IPSI; p < 10⁻⁴, F_obs = 1.18 CONTRA). Critically, RT was stable across sessions within the amphetamine block (p = 0.426, F_obs = 0.141 IPSI; p = 0.704, F_obs = 0.051 CONTRA), arguing against an experience-dependent effect on task performance. The shapes of the full RT distributions were unchanged across successive amphetamine sessions, for both CONTRA (Figure 3a,b) and IPSI (not shown) trials. Amphetamine produced a modest speeding of MT (Figure 2biii; p = 0.032, F_obs = 0.654 CONTRA; p = 0.144, F_obs = 0.492 IPSI) that was also stable across successive sessions.

“Learning”-type effects may also be observed as progressive changes within individual test sessions. We computed the correlation coefficient between RT and trial number within each session (Figure 3e), testing whether RT was negatively correlated with the number of trials already performed. The correlation coefficients for both aCSF and amphetamine infusions were nearly identically distributed around zero (Figure 3f, p = 0.925 CONTRA, p = 0.852 IPSI, Kolmogorov-Smirnov test). We conclude that amphetamine did not influence RT in an experience-dependent manner on either an inter- or intra-session timescale. In summary, increasing DLS concentrations of dopamine (and other monoamines) with amphetamine did not result in progressive behavioral changes resembling learning, but instead immediately sped RT in a manner consistent with modulation of “performance.”

DLS dopamine receptor blockade causes progressive impairments in contralateral choice and reaction time

We next tested whether dopamine receptor blockade has an immediate or progressive effect on behavior. We gave control aCSF infusions on days 1 and 7 (“vehicle 1” and “vehicle 2”, respectively), and on each day in between administered the mixed D1/D2 receptor antagonist flupenthixol (6 μg, “LOW FLU” or 30 μg, “HIGH FLU”). Neither dose significantly affected the number of trials attempted (LOW FLU p = 0.206, F_obs = 0.432; HIGH FLU p = 0.066, F_obs = 0.758). Both doses impaired CONTRA responding as assessed by accuracy (Figure 2ci and di; LOW FLU p = 0.021, F_obs = 0.974; HIGH FLU p < 10⁻⁴, F_obs = 2.56). For HIGH FLU (only) there was a significant effect of flupenthixol session number on CONTRA accuracy (LOW FLU p = 0.393, F_obs = 0.282; HIGH FLU p = 0.006, F_obs = 1.59). Furthermore, the progressive decrease in CONTRA accuracy produced by HIGH FLU persisted into the final vehicle session (p = 0.011 for vehicle 1 to vehicle 2 comparison, rank sum test; for LOW FLU p = 0.168). Rats were also biased towards ipsilateral responses on “catch” trials under flupenthixol treatment (vehicle 1 and active drug bias for LOW FLU: 0.580 ± 0.089 and 0.393 ± 0.029 respectively, p = 0.015, F_obs = 1.25; for HIGH FLU: 0.462 ± 0.099 and 0.333 ± 0.048, p = 0.005, F_obs = 0.950), and this bias persisted into the final vehicle session (LOW FLU: vehicle 2 bias = 0.343 ± 0.053, p = 0.028; HIGH FLU vehicle 2 bias = 0.170 ± 0.082, p = 0.026; rank sum test comparing vehicle 1 to vehicle 2 sessions). However, catch trial bias did not change progressively across active drug sessions (LOW FLU p = 0.951, F_obs = 0.051; HIGH FLU p = 0.344, F_obs = 0.319). RT was not significantly affected by LOW FLU (Figure 2cii, p = 0.678, F_obs = 0.130 IPSI; p = 0.331, F_obs = 0.618 CONTRA), but HIGH FLU selectively slowed CONTRA RT (p = 0.009, F_obs = 1.29; CONTRA, p = 0.445, F_obs = 0.299 IPSI) in an experience-dependent manner (p = 0.021, F_obs = 1.21). These effects are also evident in the CONTRA RT distributions (Figure 3c, d), which gradually shift to the right across sessions. Of note, even after several days of flupenthixol treatment, there is little change in the fastest RTs. Instead, the principal effect of nonselective dopamine receptor blockade was to push the peak and tail of the RT distributions to the right.

The experience-dependent effects of flupenthixol can also be seen within individual sessions. The distribution of correlation coefficients between CONTRA RT and trial number was significantly skewed to the right for HIGH FLU compared to vehicle (Figure 3g, h, p = 9.0 × 10⁻³, Kolmogorov-Smirnov test). A similar trend was observed for LOW FLU sessions, which did not reach statistical significance (not shown, p = 0.136, Kolmogorov-Smirnov test). Thus, flupenthixol, particularly at higher doses, has an experience-dependent effect on task performance consistent with a “learning” role for dopamine in DLS.

Amphetamine modulates decision thresholds, but flupenthixol changes the rate of approach to threshold

Mathematical models of decision-making can provide quantitative explanations for the shape of RT distributions [57,58], and may yield useful insights into the underlying processes affected by dopaminergic manipulations. In the LATER model [54], variability in RT arises from (Gaussian) variability in how rapidly a decision variable approaches a decision threshold on each trial. This simple model has the advantage of providing distinct predictions for how RT distributions change when either the distance to threshold or the mean approach rate is altered. Specifically, threshold changes result in a “swivel” of the RT data in reciprobit plots, while changes in the approach rate result in a parallel “shift” (see Methods and Figure 4a, c). For example, in a related rat choice task, increasing reward expectation results in an RT “swivel” consistent with reducing the initial distance to threshold [59].

We therefore examined whether a “swivel” or a “shift” best accounted for the effects of DLS dopaminergic manipulations on RT. Consistently, “swivels” were better fits after intrastriatal amphetamine, while “shifts” were better fits after HIGH FLU (Figure 4c, d). These results suggest that – in some sense - enhanced dopamine signaling lowers the distance to threshold, while nonselective dopamine receptor blockade slows the rate at which the threshold is approached.

Selective D1 receptor blockade gradually slows RT

Amphetamine infusions shortened RT without affecting choice accuracy, while FLU influenced both RT and accuracy. It therefore appears that dopaminergic influences on RT and choice accuracy can be dissociated, at least under some conditions. One possibility is that these dissociable effects arise through actions at distinct dopamine receptors. We infused selective D1 and D2 receptor antagonists into DLS prior to task performance. Compared to baseline aCSF infusions, the D1 antagonist SCH23390 mildly decreased the number of trials attempted (135.6 ± 3.9 trials vs 107.7 ± 7.2 trials, p = 4.00 × 10⁻⁴, F_obs = 3.07). It slightly but significantly increased choice accuracy for IPSI trials (p < 10⁻⁴, F_obs = 3.06; Figure 5ai), and there was a nonsignificant trend towards decreased accuracy on CONTRA trials (p = 0.074, F_obs = 1.53). There was no effect on catch bias (vehicle 1: 0.498 ± 0.074, SCH: 0.438 ± 0.038, vehicle 2: 0.488 ± 0.084, p = 0.457, F_obs = 0.229). However, RT was prolonged for both IPSI and CONTRA trials (Figure 5aii, p = 3 × 10⁻⁴, F_obs = 0.929 IPSI; p < 10⁻⁴, F_obs = 1.75 CONTRA), with a significant effect of SCH23390 session number for CONTRA trials (p = 0.025, F_obs = 0.517 within the SCH23390 block). There was also a small but significant effect on MT (Figure 5aiii; p < 10⁻⁴, F_obs = 2.26 IPSI; p = 0.044, F_obs = 1.13 CONTRA).

Reciprobit analyses for SCH23390 infusions were ambiguous, with small log-likelihood differences that favored “shifts” or “swivels” approximately equally (Figure 4d). The distribution of intrasession correlation coefficients between RT and trial number was significantly skewed to the right compared to vehicle infusions (Figure 5c, p = 9.03 × 10⁻⁵, Kolmogorov-Smirnov test), suggesting that plastic changes occur on a trial-by-trial basis under D1 receptor blockade. In summary, D1 receptor blockade in DLS caused a progressive prolongation of RT without substantially affecting choice accuracy.

Selective D2 receptor blockade acutely affects choice accuracy and reaction time

Compared to aCSF, the D2 antagonist raclopride decreased the number of trials attempted (147.6 ± 3.4 trials vs 98.2 ± 7.4 trials, p < 10⁻⁴, F_obs = 7.71). In contrast to SCH23390, raclopride strongly decreased CONTRA accuracy (p = 0.030, F_obs = 1.56, Figure 5bi) without progression across sessions (p = 0.954, F_obs = 0.100 for comparisons within the raclopride block). The effect on “catch” trial bias was not significant (vehicle 1: 0.548 ± 0.062, RAC: 0.409 ± 0.042, vehicle 2: 0.397 ± 0.065, p = 0.088, F_obs = 0.765). Median RT was prolonged under raclopride (Figure 5bii; p < 10⁻⁴, F_obs = 1.97 IPSI; p < 10⁻⁴, F_obs = 3.16 CONTRA), and this effect was obvious from the first session. If anything, RT shortened with repeated practice, although this trend was nonsignificant when comparing sessions (p = 0.246, F_obs = 0.477 IPSI; p = 0.177, F_obs = 0.789 CONTRA within the raclopride block). Within sessions, CONTRA, but not IPSI, RT tended to be negatively correlated with the number of trials already performed (p = 0.046 and 0.376 respectively, Kolmogorov-Smirnov test; Figure 5d). MT was also slightly prolonged (Figure 5biii; p < 10⁻⁴, F_obs = 2.53 IPSI; 0.002, F_obs = 1.99 CONTRA), without progression across sessions (p = 0.860, F_obs = 0.091 IPSI; 0.885, F_obs = 0.316 CONTRA). Reciprobit analyses suggested that raclopride treatment influenced RT primarily by modulating distance-to-threshold (Figure 4d).

Thus, there are several contrasts between D1 and D2 receptor blockade. D1 blockade had minimal effects on choice accuracy, on which D2 receptor blockade had an immediate, experience-independent effect. Both interventions prolonged RT. However, D1 receptor blockade did so in an experience-dependent manner while the effect of D2 receptor blockade was stable or decreasing over repeated sessions.

Loss of dopaminergic terminals in DLS impairs choice accuracy, RT, and MT

Local drug infusions are useful to evaluate the contributions of distinct aspects of DLS physiology to behavioral control, but do not directly model the permanent loss of striatal dopamine terminals seen in PD. We therefore used the results of our initial experiments to inform the interpretation of the effects of chronic dopamine loss selectively in DLS. Using focal DLS lesions also prevented levodopa induced dyskinesias, which occur at even low levodopa doses with medial forebrain bundle (mfb) lesions.

Beginning two weeks after 6-OHDA lesions, rats were retested daily in the behavioral task (days 15–21 post-lesion, “saline” block). Figure 6a shows representative TH-stained sections from a lesioned rat; all rats included in the analysis had lesions of similar size in the same region. None of the rats had obvious motor impairments in their home cages. We observed a decrease in CONTRA accuracy that first worsened, then partly recovered with repeated testing (Figure 6bi, p = 0.041, F_obs = 0.539 for comparisons between baseline and saline sessions; p = 0.064, F_obs = 0.745 for comparisons within saline sessions alone). Catch trial bias, however, was not affected (baseline: 0.702 ± 0.230, SALINE: 0.350 ± 0.083, p = 0.695, F_obs = 0.266). CONTRA RT also increased and subsequently recovered with repeated testing, though this effect did not reach statistical significance in a simple comparison of baseline and saline sessions (p = 0.065, F_obs = 0.426; Figure 6bii). Over the first several sessions, the shape of the RT distributions evolved in a manner similar to flupenthixol treatment, but almost returned to baseline by the last saline session (Figure 6c). The intrasession analysis of correlations between CONTRA RT and trial number showed no difference between baseline and post-lesion sessions (Figure 6d, p = 0.443, Kolmogorov-Smirnov test). 6-OHDA lesions also affected CONTRA MT (Figure 6biii; p = 0.025, F_obs = 0.255 CONTRA; p = 0.072, F_obs = 0.389 IPSI), and this change was stable across sessions (p = 0.341, F_obs = 0.081). Thus, lesioning dopaminergic terminals in DLS with 6-OHDA produced qualitiatively similar effects to blocking dopamine receptors with flupenthixol, except that the lesioned animals gradually recovered.

In the same rats we next investigated the effects of systemic levodopa administered 30 minutes prior to each test session (days 22–28 post-lesion, “levodopa” block). As assessed by accuracy and RT, task performance was at baseline levels (Figure 6b, p = 0.792, F_obs = 0.539 for CONTRA accuracy between baseline and levodopa sessions; p = 0.934, F_obs = 0.132 for CONTRA RT between baseline and levodopa sessions). Accuracy and RT were stable with repeated levodopa treatments (p = 0.721, F_obs = 0.583 for CONTRA accuracy, p = 0.938, F_obs = 0.115 for CONTRA RT). By contrast, the MT prolongations did not resolve under levodopa treatment (Figure 6biii; comparing MT in baseline to levodopa sessions, p = 0.044, F_obs = 0.551 IPSI; p = 0.007, F_obs = 0.451 CONTRA). There was no within-session effect of trial number on CONTRA RT (Figure 6d, p = 0.272 for comparison between baseline and levodopa correlation coefficients, Kolmogorov-Smirnov test). In summary, deficits in choice accuracy and RT were resolving spontaneously prior to levodopa administration, but fully normalized with levodopa and remained stable across levodopa sessions.

DISCUSSION

This study was designed to help resolve whether dopamine acts upon striatal circuitry to modulate immediate behavioral performance, learning, or both. Our results clearly demonstrate that dopaminergic signaling within a single striatal subregion (DLS) modulates both current and future performance of a choice reaction time task. Further, learning and performance effects can be dissociated by examining distinct aspects of behavior. Our working hypothesis is that these effects correspond to the distinct actions of dopamine on striatal cell excitability (performance) and synaptic plasticity (learning).

Basal ganglia contributions to action control

The impaired performance immediately after muscimol infusions shows that DLS is critical for online contralateral performance of this instructed choice task. This provides further evidence that the BG are key components of brain networks for action selection [60]. How can we reconcile this with observations that many aspects of well-trained behavior are not disrupted when BG output is abolished [30–32]?

One potential explanation involves a highly conserved BG function of orienting towards and approaching potential rewards [61,62]. Unilateral BG manipulations typically cause a response bias in tasks with lateralized response options [29,63,64], and our muscimol, flupenthixol, and 6-OHDA results support a preferential role for the BG in contralateral movement initiation. SNr inactivation leads to contralaterally directed, apparently involuntary saccades [17,65,66] and neck twisting [67]. The BG outflow to superior colliculus (SC) is critical for the appropriate control of contralateral orienting behavior, whether expressed as rapid left vs. right head/neck movements [68] or eye/head/neck movements in primates [69]. By contrast, for other forms of behavior the role of the BG may be more restricted to learning and determining action “vigor” (e.g. [32]).

When considering how the BG contribute to decision-making, the fact that BG manipulations predominantly affect contralateral responses is important: it implies that competing bids for action reside in opposite hemispheres. If so, it is hard to see how one hemisphere alone could resolve this competition (e.g. [70–72]). There are a number of BG pathways by which information can be passed across hemispheres, including bilateral nigrotectal projections [73,74], subthalamic projections to intralaminar thalamus [75–77], and corticostriatal projections [78,79]. Bilateral pathways deserve further investigation in the context of decision-making, and may be involved in our unexpected observation that unilateral amphetamine infusion reduced RT for both movement directions.

Prior investigations using similar tasks have found conflicting results regarding the effects of unilateral 6-OHDA lesions on MT. Carli et al [63] found no effect of single intrastriatal 6-OHDA lesions on MT, though more extensive lesions have caused MT slowing [29,80]. As in our study, Carli et al [63] required responses into adjacent ports, while the studies that showed MT impairments required movement to more distant ports. Nonhuman primates that have received intraputaminal dopamine antagonist infusions [43] and humans with PD have impairments in both RT and MT, as assessed by a range of tasks under different treatment conditions [81–84]. It is therefore unclear whether the apparently inconsistent MT results across studies relate to the specifics of the task structure or the extent of dopamine loss. In either case, the available evidence indicates that striatal dopamine loss, if severe enough, may interfere with both the preparation and implementation of motor plans.

Separate striatal mechanisms affect the “what” and “when” of decision-making

In common conceptualizations of decision-making, choice and RT are closely linked: action selection occurs when a decision process reaches a threshold representing one of the potential options [85]. In practice, choice and RT can vary independently, at least under some conditions. Our muscimol infusions in DLS preferentially affected choice accuracy rather than RT, confirming prior observations in a related task using excitotoxic lesions in lateral striatum [64]. Conversely, that study found that dorsal-medial striatal lesions affected RT more than choice accuracy (see also [44]). We have shown that RT and choice accuracy can be dissociated within the same DLS subregion; understanding how specific microcircuit mechanisms contribute to each aspect is an important direction for future studies.

It is well-known that RTs are longer than expected from conduction delays alone, and variable from trial-to-trial. Many DLS neurons show an accelerating increase in firing shortly before initiation of contralateral responses [86], but it is not clear why this happens earlier on some trials than others. Our finding that dopaminergic manipulations of DLS influence RT indicates that DLS is not simply a passive conduit of movement initiation signals, but an active participant in deciding when to move [87,88]. Previous applications of the LATER model have found that BG manipulations may influence either the rate of approach [16,89] or distance [90] to a theoretical threshold. It is striking that our interventions with primarily “performance” effects on RT (amphetamine and raclopride) affected distance to threshold, similar to altering expected reward [59]. In contrast, flupenthixol had primarily “learning” effects on RT, and influenced rate of approach. It has been suggested that saccade RTs are determined by neural accumulators in cortex (eg, frontal eye fields) and a threshold detector in the SC [87,91,92]. Within this framework, BG output may directly modulate the SC threshold on a trial-by-trial basis, but influence cortical accumulators gradually as synaptic strengths (within the striatum or cortex itself) are altered. The LATER model alone is too limited to provide a full understanding of how the BG influence RT or action selection, and more sophisticated models (e.g. [93–95]) should be developed and employed in future work. Nonetheless, our results help map distinct facets of dopamine modulation in DLS to dissociable decision-making processes.

Dopamine affects multiple components of striatal microcircuitry on multiple timescales. Over seconds to minutes, elevated “tonic” dopamine tends to enhance the excitability of direct pathway (“GO”) MSNs via D1 receptors, while reducing excitability of indirect pathway (“NO-GO”) MSNs via D2 receptors [24,96,97]. Briefer “phasic” changes in dopamine modulate synaptic plasticity to create persistent changes in behavior [98,99]. Dopamine surges increase occupancy of the lower-affinity D1 receptors [100], leading to increased cAMP and (if other conditions are fulfilled) expression of genes important for long-term synaptic change [21]. Dips in dopamine lower occupancy of the higher-affinity D2 receptors, which also produces increased cAMP and closely-related changes in gene expression [21]. A natural hypothesis is that “learning”-like behavioral changes arise through synaptic plasticity (within DLS or downstream regions), while “performance”-type changes arise through altered neuronal excitability.

If normal increases in dopamine signaling can affect both learning and performance, why did we only observe performance-type changes with amphetamine? One limitation of the present study is that drug infusions do not mimic task-related dopamine fluctuations. Artificially-prolonged dopamine signaling can cause homeostatic responses [101] that constrain plasticity mechanisms. Another factor may be extensive training in the behavioral task. Although this training has not produced a floor effect on RT (the rats are capable of shorter median RTs, as shown by the acute effects of amphetamine) it may have saturated the contribution of striatal plasticity to rapid movement initiation. This would leave room for lowered dopamine signaling to alter established synaptic weights and thereby progressively slow RTs, as we observed with FLU and the D1 antagonist SCH23390. A similar effect has been termed “extinction mimicry” [28], and attributed to decreased dopamine reward signaling. Whatever the functional meaning of the dopamine signal, it would be informative to perform drug infusions at earlier stages of training, which typically yield RT distributions with longer right-side “tails” (data not shown). Under such conditions we might predict that amphetamine produces progressive RT speeding.

In contrast to the relatively selective effects of D1 receptor blockade on RT, D2 antagonism led to immediate, nonprogressive changes in both accuracy and RT. It is not yet clear why selective D2 receptor blockade did not cause experience-dependent changes (“NOGO” learning), as observed for rotarod performance [27]. The presence of D2 receptors at multiple pre- and post-synaptic locations in striatum [24], together with the limited set of doses used in these experiments, make interpretation of the raclopride results more challenging. Overall, our use of selective antagonists provided only limited insight into distinct microcircuit mechanisms controlling learning versus performance, and choice versus RT. More studies, incorporating additional techniques, are clearly needed.

Implications for PD

The 6-OHDA lesions produced a combination of learning and performance effects. The first several days of post-lesion testing produced progressive changes in behavior (see also [102]) similar to the effects of dopamine blockade with flupenthixol. However, task performance began to recover in the fifth session, and this recovery is unlikely to be due to restored dopamine transmission given the lack of TH immunoreactivity postmortem. More likely, other cortico-BG loops compensated, as has been observed after elimination of striatopallidal projections from DLS [103]. It is unclear why recovery occurred in lesioned, but not flupenthixol-treated, animals, but compensatory strategies may have emerged in the lesioned rats because DLS dopamine function was continuously depressed.

The LDR represents sustained improvement in motor function despite cessation of dopamine replacement therapy, and is a critical part of the therapeutic response in early PD [20]. It has been argued that the LDR may result from an intracerebral reservoir of levodopa, though that view has been questioned recently [20,23]. Our HIGH FLU and 6-OHDA experiments demonstrate that motor function may be preserved in the setting of reduced DLS dopamine signaling, at least for a while, and provide an alternative BG “learning” explanation for the LDR [23]. It may be beneficial to start dopamine replacement early in the disease course to prevent aberrant learning by the dopamine-depleted striatum. Indeed, a large clinical trial suggested that levodopa may preserve motor function over the long-term [104]. “Learning” effects may be detrimental, however, in both medicated and unmedicated states. Experience-dependent declines in function with decreased DLS dopamine signaling suggest that practicing motor skills (e.g., physical therapy) in the “off” state would impair future performance. Conversely, dysregulated striatal plasticity in persistently hyperdopaminergic states may contribute to levodopa-induced dyskinesias [105]. “Learning” effects also likely extend to nonmotor features, including personality changes and impulse-control disorders [106]. Much work remains to understand how BG “learning” and “performance” functions interact to cause PD symptoms, and how they may be exploited to optimize its treatment.

ABBREVIATIONS

aCSF

artificial cerebrospinal fluid

AMPH

amphetamine

BG

basal ganglia

DLS

dorsolateral striatum

FLU

flupenthixol

LATER

linear approach to threshold with ergodic rate

LDR

long duration response

mfb

medial forebrain bundle

PD

Parkinson Disease

RAC

raclopride

RT

reaction time

SC

superior colliculus

SDR

short duration response

SNr

substantia nigra, pars reticulata