The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Takahiro Doi; Yunshu Fan; Joshua I Gold; Long Ding

doi:10.7554/eLife.56694

. 2020 Jun 22;9:e56694. doi: 10.7554/eLife.56694

The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Takahiro Doi ^1,², Yunshu Fan ^1,³, Joshua I Gold ^1,³, Long Ding ^1,^3,^✉

Editors: Daeyeol Lee⁴, Michael J Frank⁵

PMCID: PMC7308093 PMID: 32568068

Abstract

Our decisions often balance what we observe and what we desire. A prime candidate for implementing this complex balancing act is the basal ganglia pathway, but its roles have not yet been examined experimentally in detail. Here, we show that a major input station of the basal ganglia, the caudate nucleus, plays a causal role in integrating uncertain visual evidence and reward context to guide adaptive decision-making. In monkeys making saccadic decisions based on motion cues and asymmetric reward-choice associations, single caudate neurons encoded both sources of information. Electrical microstimulation at caudate sites during motion viewing affected the monkeys’ decisions. These microstimulation effects included coordinated changes in multiple computational components of the decision process that mimicked the monkeys’ similarly coordinated voluntary strategies for balancing visual and reward information. These results imply that the caudate nucleus plays causal roles in coordinating decision processes that balance external evidence and internal preferences.

Research organism: Rhesus macaque

Introduction

Effective decision-making often requires deliberation over uncertain evidence for and against different alternatives, as well as over the expected outcome associated with those alternatives. The final choice depends on how these two types of information are combined in the deliberation process. For example, when we encounter news that “chocolate is healthy,” our reaction (“should I eat more chocolate?”) can be influenced by both the perceived quality of the evidence (peer-reviewed research article or tabloid) and our desired version of events (“I hope it is true because I love chocolate!”). Previous studies have provided many insights into the kinds of computations that humans and animals use to balance uncertain sensory evidence and reward expectation for adaptive decisions, but it remains unclear where and how these computations are implemented in the brain (Diederich and Busemeyer, 2006; Fan et al., 2018; Feng et al., 2009; Gao et al., 2011; Leite, 2012; Liston and Stone, 2008; Maddox and Bohil, 1998; Mulder et al., 2012; Summerfield and Koechlin, 2010; Teichert and Ferrera, 2010; Voss et al., 2004; Waiblinger et al., 2019; Whiteley and Sahani, 2008).

A prime candidate for mediating these computations is the basal ganglia pathway, which has been a focus of many modeling studies (Bogacz and Gurney, 2007; Ding and Gold, 2013; Hikosaka et al., 2014; Kable and Glimcher, 2009; Lo and Wang, 2006; Rao, 2010; Ratcliff and Frank, 2012; Redgrave et al., 1999; Summerfield and Tsetsos, 2012; Wei et al., 2015). This pathway is known to make separate contributions to perceptual decisions that select based on the interpretation of uncertain sensory evidence and value-based decisions that select among outcome options (Amemori et al., 2018; Cai et al., 2011; Cavanagh et al., 2011; Ding and Gold, 2010; Ding and Gold, 2012b; Hikosaka et al., 2014; Kim and Hikosaka, 2013; Kimchi and Laubach, 2009; Lau and Glimcher, 2008; Nakamura and Hikosaka, 2006b; Samejima and Doya, 2007; Santacruz et al., 2017; Seo et al., 2012; Tachibana and Hikosaka, 2012; Tai et al., 2012; Wang et al., 2018; Yanike and Ferrera, 2014; Yartsev et al., 2018). However, its role in combining those different sources of information remains speculative. For example, bilateral lesions of the striatum in rats affected action vigor but not the decision process that combined reward and olfactory input (Wang et al., 2013), arguing against a casual role of the basal ganglia pathway.

To begin to understand the basal ganglia’s causal roles in combining evidence and outcome expectation in guiding adaptive decision-making, we targeted the caudate nucleus, an input station in the oculomotor basal ganglia pathway, in monkeys trained to report their perceived motion direction of a random-dot kinematogram in an asymmetric-reward task (Figure 1A; Fan et al., 2018). Across trials, we manipulated visual evidence by presenting motion stimuli with varying strengths in two directions. Across blocks of trials, we manipulated reward context by assigning a large reward for one direction and small reward for the other and alternating reward assignments between two consecutive blocks. As we demonstrated previously, monkeys solved this task by adaptively balancing the uncertain visual input and reward information (Fan et al., 2018).

Figure 1. — (A) Task design and timeline. Monkeys reported the perceived motion direction with saccades to one of the two choice targets. The motion stimulus was turned off upon detection of saccade. Correct trials were rewarded based on the reward context. Error trials were not rewarded. The color bars in the timeline indicate epoch definitions for the regression analysis of neural firing rates in Equation 1. (B) Average choice (top) and RT (bottom) behavior of two monkeys (n = 17,493 trials from 38 sessions for monkey C, 29,599 trials from 79 sessions for monkey F). Filled and open circles: data from the two reward contexts. Lines in top row: logistic fits. (C) Histograms of reward bias (half of the horizontal shift between the two reward contexts estimated from logistic fits) for the two monkeys. Positive values indicate biases toward the large-reward choice.

Here, we report four lines of evidence supporting a causal involvement of the caudate nucleus in mediating the monkeys’ adaptive strategy: (1) caudate activity exhibited combined representations of task-relevant visual and reward information, both at the population level and at the single-neuron level; (2) electrical microstimulation in the caudate nucleus during motion viewing affected how the visual and reward information was used to form the decision, often in a reward context-dependent manner; (3) the reward context-dependent microstimulation effects shared certain features with the monkeys’ voluntary, adaptive adjustments in response to the asymmetric reward contexts; and (4) the magnitude of microstimulation effects on these features depended on caudate activity patterns at the stimulation sites. These results imply that the caudate nucleus plays key roles in coordinating the decision process that balances external evidence and internal preferences to guide adaptive behavior.

Results

We trained two monkeys to report their decision by making a saccadic eye movement at a self-determined time. As we reported previously, the monkeys’ performance depended on both the strength and direction of the visual-motion evidence and the reward asymmetry on each trial, with consistent biases toward choices associated with large reward (Figure 1B and C). More details about these two monkeys’ performance on this task can be found in Fan et al., 2018.

Caudate neurons encode both visual and reward information

We first tested if and how individual caudate neurons encoded task-relevant visual and reward information. We predicted that, if the caudate nucleus contributes to integrating these different sources of information, then the activity of individual neurons would be modulated by properties of both the visual motion evidence and the reward context. Alternatively, if the caudate nucleus processes reward and visual information separately, then the activity of individual caudate neurons would be modulated by either type of information alone but not together.

We found that many caudate neurons are sensitive to both visual and reward information used to perform the task. Specifically, we recorded single-unit activity from 142 caudate neurons in the two monkeys (n = 49 for monkey C, 93 for monkey F). These caudate neurons showed diverse patterns of task-dependent modulations that in many cases included a combination of visual and reward modulations. For example, the activity of the neuron depicted in Figure 2A showed three types of modulation: (1) more activity during the blocks when the contralateral choice was paired with small reward and the ipsilateral choice was paired with large reward (green > purple); (2) more activity for trials with stronger versus weaker motion evidence (dark shade > light shade; i.e., higher versus lower coherence levels, respectively), particularly for trials with contralateral choices; and (3) more activity for trials with contralateral versus ipsilateral choices, both during motion viewing and around saccade onset (Contra > Ipsi). This neuron’s activity thus reflected a combination of reward context, motion strength, and eventual choice. The example neuron depicted in Figure 2B showed: (1) more activity on trials with higher coherence levels (dark shade > light shade); (2) a contralateral choice preference, both during motion viewing and around saccade onset (Contra > Ipsi); and (3) more activity when the choice was associated with large reward (red > blue). This neuron’s activity thus reflected choice, the strength of motion stimulus leading to the choice, and the reward size expected for the choice. Additional examples are shown in Figure 2—figure supplement 1.

Figure 2. — (**A, B**) Activity of two example neurons. Shades: coherence levels. Colors: reward context (A) and reward size (B). Firing rates were computed using a 200 ms running window (50 ms steps). Only correct trials were included. (**C-F**) Fractions of neurons showing significant coefficients for task-related regressors in the seven task epochs defined in Figure 1A (see Equation 1 for the formulation of the regression). Horizontal dashed lines: chance levels, adjusted based on the number of tests used and a 5% chance level for a single test. Filled circles: fractions that were significantly above chance levels (Chi-square test, p<0.05/63 (seven epochs and nine comparisons)). Coh: activity with non-zero coefficients for unsigned coherence values. Coh × Reward: activity with non-zero coefficients for the coherence × reward size interaction. Coh + Reward: activity with non-zero coefficients for coherence on trials with either choice and non-zero coefficients for either reward context or reward size.

Figure 2—source data 1. Regression results for Figure 2C–F, using Equation 1.

elife-56694-fig2-data1.csv^{(237.6KB, csv)}

Figure 2—figure supplement 1. — (**A, B**) Activity of two example neurons. Shades: coherence levels. Colors: reward context (A) and reward size (B). Firing rates were computed using a 200 ms running window (50 ms steps). Only correct trials were included. (**C-F**) Fractions of neurons showing significant coefficients for task-related regressors in the seven task epochs defined in Figure 1A (see Equation 1 for the formulation of the regression). Horizontal dashed lines: chance levels, adjusted based on the number of tests used and a 5% chance level for a single test. Filled circles: fractions that were significantly above chance levels (Chi-square test, p<0.05/63 (seven epochs and nine comparisons)). Coh: activity with non-zero coefficients for unsigned coherence values. Coh × Reward: activity with non-zero coefficients for the coherence × reward size interaction. Coh + Reward: activity with non-zero coefficients for coherence on trials with either choice and non-zero coefficients for either reward context or reward size.

Figure 2—source data 1. Regression results for Figure 2C–F, using Equation 1.

elife-56694-fig2-data1.csv^{(237.6KB, csv)}

Across the sampled population, individual caudate neurons were sensitive to choice, reward context, expected reward size, and motion strength. We measured the selectivity of single-unit activity using multiple linear regression for seven task epochs (Figure 2C; Equation 1; epochs are defined in Figure 1A). Selectivity for reward context persisted throughout the trial (green), whereas selectivity for choice, reward size, and motion strength emerged during motion viewing (Epochs 4–7). The selectivity for reward size and motion strength was more prevalent in Epoch 5 (variable duration covering the whole motion viewing period) than in Epoch 4 (fixed duration covering only the early motion viewing period), consistent with a developing latent decision variable that accumulates evidence over time, increasing sensitivity of the regression analysis with longer analysis windows, and/or additional sensitivity to RT closer to the time of saccade. Many neurons also showed joint modulations by both motion strength and either the reward context or expected reward size (“Coh + Reward”, Figure 2D). A smaller proportion of neurons showed a different form of joint modulation by visual and reward-related information: their activity was sensitive to the interaction between motion strength and reward size (“Coh ×Reward”, Figure 2E). Overall, 101 out of 142 neurons showed at least one of these forms of joint modulation in at least one epoch. This fraction was significantly above chance level, even considering the multiple tests done in seven epochs (Figure 2F; Chi-squared test, p=0.0035). These neurons also showed heterogenous modulation patterns, as illustrated for the 44 neurons with joint modulation during motion viewing (Figure 2—figure supplement 2).

Thus, information about both the visual motion evidence and reward expectation were represented in the caudate nucleus at both the population and single-neuron levels. The combined evidence-reward representations at the single-neuron level in some caudate neurons suggest that the caudate nucleus may contribute directly to the process of integrating both sources of information into the decision.

Caudate microstimulation evoked reward context-dependent effects on behavior

We next tested if and how the caudate nucleus contributes causally to decisions that balance visual evidence and reward information. We identified caudate sites with task-modulated activity and delivered electrical microstimulation during motion viewing at these sites in randomly interleaved trials. We were particularly interested in microstimulation effects that depended on visual and reward information interactively, which would imply that the caudate nucleus contributes to the integration of those sources of information into the decision.

Consistent with the heterogeneity of response properties identified above and our previous findings (Ding and Gold, 2012b), we found examples of microstimulation effects that were either independent of or dependent on reward context. Figure 3A shows an example session in which the microstimulation effects depended on visual evidence but not reward context. At this site, microstimulation induced contralateral biases (a leftward shift in the psychometric function), reduced sensitivity to the visual motion evidence (a reduction in the slope of the psychometric function), decreased mean RT for the contralateral choice, and increased mean RT for the ipsilateral choice, with similar magnitudes of changes for the two reward contexts. In contrast, Figure 3B shows an example session in which the microstimulation effects depended on both visual evidence and reward context. At this site, microstimulation induced ipsilateral choice biases, slope reduction, and decreases in RT for ipsilateral saccades. The effects were more prominent when the ipsilateral choice was paired with large reward (open circles). Thus, microstimulation changed the relationship between choice/RT and visual evidence in a reward context-dependent fashion. These changes were consistent with a perturbation of a process that incorporates both visual evidence and reward asymmetry information into the decision.

Figure 3. — (**A, B**) Two example sessions from monkey C showing different patterns of microstimulation effects. Black: trials without microstimulation; red: trials with microstimulation. Open circles and dashed lines: data and fits for blocks in which the ipsilateral choice was paired with large reward and the contralateral choice was paired with small reward. Filled circles and solid lines: data and fits for blocks in which the ipsilateral choice was paired with small reward and the contralateral choice was paired with large reward. Abscissa: signed coherence (positive: motion was toward the contralateral target; negative: motion was toward the ipsilateral target). Top panels: probability of making the contralateral choice. Lines are logistic fits to the choice data. Middle and bottom panels: mean reaction time (RT) for the two reward contexts. Circles and crosses represent correct and error choices, respectively. Lines: linear fits to single-trial RT data. (**C, D**) Summary of effects induced by microstimulation for both reward contexts (C, estim terms) and by interactions between microstimulation and reward context (D, rew × estim terms). The top rows illustrate the effects on psychometric and chronometric curves for positive values of the corresponding terms. Bottom rows show the histograms of changes in choice bias (logistic shift in %coh), perceptual sensitivity (logistic slope in 1/%coh), and intercept and slope of linear regression of RT as a function of unsigned motion coherence. Thick vertical lines indicate the median values. Green lines, Wilcoxon signed-rank test, p<0.05.

Figure 3—figure supplement 1. — (**A, B**) Two example sessions from monkey C showing different patterns of microstimulation effects. Black: trials without microstimulation; red: trials with microstimulation. Open circles and dashed lines: data and fits for blocks in which the ipsilateral choice was paired with large reward and the contralateral choice was paired with small reward. Filled circles and solid lines: data and fits for blocks in which the ipsilateral choice was paired with small reward and the contralateral choice was paired with large reward. Abscissa: signed coherence (positive: motion was toward the contralateral target; negative: motion was toward the ipsilateral target). Top panels: probability of making the contralateral choice. Lines are logistic fits to the choice data. Middle and bottom panels: mean reaction time (RT) for the two reward contexts. Circles and crosses represent correct and error choices, respectively. Lines: linear fits to single-trial RT data. (**C, D**) Summary of effects induced by microstimulation for both reward contexts (C, estim terms) and by interactions between microstimulation and reward context (D, rew × estim terms). The top rows illustrate the effects on psychometric and chronometric curves for positive values of the corresponding terms. Bottom rows show the histograms of changes in choice bias (logistic shift in %coh), perceptual sensitivity (logistic slope in 1/%coh), and intercept and slope of linear regression of RT as a function of unsigned motion coherence. Thick vertical lines indicate the median values. Green lines, Wilcoxon signed-rank test, p<0.05.

Across the 55 sites tested (n = 24 sessions for monkey C, 31 for monkey F), we found many sites with microstimulation effects that reflected different combinations of effects that were independent of reward context and others that showed interactive effects on visual and reward processing. To quantify these diverse microstimulation effects, we fitted logistic (Equation 2) and linear (Equation 3) functions to the choice and RT data, respectively. We parsed the microstimulation effects into two types. First, the “estim” type measured the average microstimulation effect, independent of reward context (Figure 3C). These effects tended to include positive changes in choice bias (biasing toward contralateral choices), negative changes in the slope of the psychometric function (reductions in sensitivity to the visual evidence), negative changes in the intercept of RT, and positive changes in the slope of the RT-coherence function (faster RTs that were less sensitive to motion strength). Second, the “rew × estim” type measured the extent to which the microstimulation effect differed between the two reward contexts (i.e., the interaction between reward context and microstimulation effects), which is the focus of the present study and is equivalent to measuring the microstimulation effects on the monkeys’ behavioral adjustments to the asymmetric reward contexts (Figure 3D). These effects tended to include positive values for choice bias, negative values for the slope of the psychometric function, negative values for the ipsilateral RT, and positive values for the slope of the RT-coherence function. Values for individual sessions are plotted in Figure 3—figure supplement 1B, and summary statistics are in Supplementary file 1a. The median “rew × estim”-type effects tended to have the same signs as the corresponding median “estim”-type effects (compare the vertical lines in Figure 3C and D), indicating that the microstimulation effects tended to be larger when the large reward was paired with choices contralateral to the site of microstimulation. Note that for the directional choice bias, this tendency may manifest itself as microstimulation inducing either larger contralateral biases in “Contra-LR” blocks, or larger ipsilateral bias in “Ipsi-LR” blocks (Figure 3B). These dependencies on reward context were similarly apparent when considering the two reward contexts separately (Figure 3—figure supplement 1A).

Overall, microstimulation induced at least one statistically reliable effect on choice or RT at 48 of 55 caudate sites (colored dots in Figure 3—figure supplement 1B; significantly above the chance level that was adjusted for multiple comparisons, Chi-squared test, p<1e-5). Of these, 27 sites showed at least one “rew × estim”-type effect on choice and/or RT. Most of these sites included effects on psychometric bias, psychometric slope, and/or the slope of RT as a function of motion strength, all of which (unlike the intercept of the RT function) measured the dependence of the monkeys’ behavior on motion evidence.

Because we delivered microstimulation throughout motion viewing for an RT task, the difference in average RT between reward contexts may contribute to “rew × estim”-type microstimulation effects. We found that the average RT tended to be longer for the “Ipsi-LR” and “Contra-LR” blocks for the monkeys C and F, respectively (Figure 3—figure supplement 2A). However, the difference in average RT between reward contexts alone cannot explain the “rew × estim”-type microstimulation effects. For each effect in Figure 3D, we performed a linear regression using the difference in RT as the independent variable (Figure 3—figure supplement 2B). The percentage of variance explained was lower than 30% for monkey C and lower than 10% for monkey F. Even for the case with the largest explained variance, the relationship was opposite to what would be expected, that is, sessions with larger RT differences tended to show smaller “rew × estim”-type microstimulation effects on choice bias for monkey C. Thus, the dependence of the microstimulation effects on reward context more likely reflected a causal involvement of the caudate nucleus in balancing visual evidence and reward asymmetry information.

Microstimulation caused coordinated adjustments to reward-dependent decision biases that mimicked the monkeys’ voluntary strategy

We next examined more closely how these diverse microstimulation effects on choice and RT were related to the specific strategies that the monkeys were using to balance visual evidence and reward asymmetry information. We previously showed that, on the same task without microstimulation, the monkeys’ choice and RT behaviors can be well described by a drift-diffusion model (DDM), in which noisy visual evidence is accumulated over time until reaching a pre-defined, time-varying, bound (Figure 4A; Fan et al., 2018; Ratcliff and Rouder, 1998; Zylberberg et al., 2016). Within this framework, the balance between visual evidence and reward asymmetry is captured by two reward context-dependent parameters that induce asymmetries in the drift rates and the relative bound heights for the two choices, respectively. A change in the drift rate was implemented as additional momentary evidence (me) that is added to the motion evidence at each accumulating step. A change in the relative bound heights was implemented as a shift in the starting point (z) of the DDM. A positive difference in Δdrift or Δbound between reward contexts, that is, positive Δdrift (rew) or Δbound (rew), leads to more and faster choices to the large-reward option. Either Δdrift (rew) or Δbound (rew) can have similar effects on the psychometric function but are distinguishable via different effects on the chronometric function (see Figure 3—figure supplement 1 in Fan et al., 2018). Here, we used the DDM to compare the monkeys’ voluntary decision strategies with the effects of microstimulation on those strategies.

Figure 4. — (A) Drift-diffusion model. Motion evidence (E) is modeled as samples from a Gaussian distribution (mean = signed coherence, variance = 1). The decision variable is computed as the time integral of E and compared at each time point to two (possibly time-varying) decision bounds. Crossing of either bound results in the corresponding choice. RT is modeled as the sum of the time to bound crossing and a non-decision time. Bias can be induced with offsets in evidence (me, biasing the drift rate) or relative bound heights (z, biasing bounds). (B) For the simulations in (**C-F**), reward context-dependent modulation of drift and bound on trials without microstimulation (i.e. Δdrift (rew) and Δbound (rew)) were negatively correlated across sessions. Each colored circle represents a session. (**C and D**) Microstimulation induces independent changes in baseline drift0 and bound0 values (C) or independent changes in Δdrift (rew) and Δbound (rew) (D). Note the absence of negative correlation in the rew × estim effects. (E) Microstimulation additively affected reward context-dependent adjustments in drift and bound in a coordinated manner. The dashed line represents a significant correlation. (F) Microstimulation additively and multiplicatively affected reward context-dependent adjustments in drift and bound in a coordinated manner. Note that the rew × estim effects were negatively correlated between drift and bound (dashed line). The rew × estim effects were negatively correlated with the reward effects without microstimulation in F (compare the orders of color progression to that in B), but not (**C-E**). See Materials and methods for simulation parameters.

As we reported previously, the reward function (average reward rate) on our task depended on perceptual sensitivity, reward context, and the relative sizes of large and small rewards in a given session. The monkeys tended to use positive Δdrift (rew) values that changed in magnitude according to session-by-session variations in the reward function (see Figure 6 in Fan et al., 2018), implying fast adaptation within a session. In addition to these positive, but variable, Δdrift (rew) values, the monkeys tended to use negative Δbound (rew) values that also changed systematically with changes in the reward function, such that the values of Δdrift (rew) and Δbound (rew) were negatively correlated with each other across sessions (Figure 5B). These coordinated adjustments produced overall biases toward the large-reward choice consistently across sessions (see also Figure 1C). We also showed previously that this negative correlation followed the predicted relationship based on a heuristic decision strategy that used features of the session-specific reward function to achieve near-maximal, “good-enough” reward rate (Fan et al., 2018).

Figure 5. — (A) Scatter plots of changes in drift and bound induced by electrical microstimulation (abscissa and top histograms) and by interactions between electrical microstimulation and reward condition (ordinate and right histograms). Solid lines in histograms: mean values across sessions, t-test, p>0.05. Labels (a, b) correspond to the example sessions in Figure 3A and B, respectively. (B) On trials without microstimulation, the differences in Δdrift and Δbound between the two reward contexts were negatively correlated. Line and shaded area: linear regression and 95% confidence interval, t-test, p<0.0001. “a” and “b” indicate the data points for the examples in Figure 3A and B. Data are color-coded by the values of Δbound (rew) no estim. (C) Scatter plots of reward effects on trials without microstimulation (abscissa) and interaction effects (ordinate) for Δdrift (left) and Δbound (right). Lines represent results of linear regression (shaded area: 95% confidence interval). (D) The interaction effects (Δdrift (rew x estim) and Δbound (rew x estim)), equivalent to the difference between “Δdrift/bound (rew) with estim” and “Δdrift/bound (rew) no estim”, were negatively correlated. Same format as B. t-test, p<0.0001. Red dashed line re-plots the linear regression results from Figure 5B, using the appropriate range of Δdrift (rew x estim) as x-values. Data are color-coded by the values of Δbound (rew) no estim. Note that the roughly reversed orders of color progressions in B and D is most consistent with simulated effects in Figure 4F.

Figure 5—source data 1. Fitting results for choice and RT data using the DDM for sessions with significant microstimulation effects.
The same data were used for plots in Figure 5—figure supplement 2, Figure 5—figure supplement 4, Figure 6A and E, Figure 6—figure supplement 3, and for the summary statistics in Supplementary file 1b.

elife-56694-fig5-data1.csv^{(15.3KB, csv)}

Based on these findings, we predicted that if the caudate nucleus contributes causally to the implementation of the monkeys’ decision strategies, then the effects of caudate microstimulation on Δdrift (rew) and Δbound (rew) would show a similar negative correlation across sessions (caudate sites). Alternatively, if the caudate nucleus contributes causally only to the implementation of specific Δdrift (rew) and/or Δbound (rew) values, we predicted that microstimulation would likely induce uncorrelated changes across sessions (caudate sites). To guide our examination of the experimental data, we considered several likely scenarios of microstimulation effects (Figure 4C–F; see Materials and methods for simulation parameters). For all scenarios, we assumed that, in the absence of microstimulation: 1) the monkeys used negatively correlated Δdrift (rew) and Δbound (rew) values across sessions (Figure 4B); and 2) Δdrift (rew) and Δbound (rew) were combined with variable baseline values (drift0 and bound0) to generate the observed choices and RTs for each reward context (Figure 4C–F, left). In the first scenario, we assumed that the caudate nucleus is not involved in the coordination and contributes only to the implementation of reward context-independent me and z adjustments, that is, microstimulation is expected to affect the baseline drift0 and bound0 independently (Figure 4C). In the second scenario, we again assumed that the caudate nucleus is not involved in the coordination, but in this case contributes to the implementation of reward context-dependent me and z adjustments; i.e., microstimulation is expected to affect Δdrift (rew) and Δbound (rew) independently (Figure 4D). In the third and fourth scenarios, the caudate nucleus contributes to the coordination process. Microstimulation added correlated changes in Δdrift (rew) and Δbound (rew) (Figure 4E) and further scaled the correlated Δdrift (rew) and Δbound (rew) values (Figure 4F). Notably, only the coordinated-effect scenarios (Figure 4E and F) predict a similar relationship for interaction effects between reward and microstimulation. Moreover, only the coordinated-scaling scenario (Figure 4F) predicts that the “rew × estim” effects should vary systematically with “rew” effects without microstimulation across sessions (compare the color progression in Figure 4B and C–F).

We fitted the monkeys’ single-trial data with different DDM variants (see fits to example sessions in Figure 5—figure supplement 1 and summary data in Figure 5—figure supplement 2). For the remaining analyses, we focused on 39 sessions in which microstimulation affected at least one DDM component. In these sessions, the full DDM model, in which all parameters were allowed to change with reward context and microstimulation, considerably outperformed a reduced model (“No Estim”), in which all parameters were allowed to change with reward context but not microstimulation (Figure 5—figure supplement 2A). In other words, if the AIC value for the full model was less than that for the “No Estim” model by a difference of at least 7 (to the left of the red arrow in Figure 5—figure supplement 2A), we assumed that a change in at least one DDM parameter was required to account for microstimulation effects in that session. The reduced model without microstimulation effects on parameters describing the collapsing bound (“NoCollapse”) outperformed the full model in most sessions, indicating that caudate microstimulation did not affect the time course of the bound height (Figure 5—figure supplement 2B). In contrast, the full model tended to outperform other reduced models, indicating that caudate microstimulation evoked changes in parameters controlling the speed-accuracy tradeoff (a and k), biases in choice and RT (me and z), and non-decision times (t_contra and t_ipsi; Figure 5—figure supplement 2C). Consistent with the results from logistic fits, the microstimulation effects on these parameters were often reward context-dependent (Figure 5—figure supplement 2D; many data points deviated from the unity-slope lines). Summary statistics of the microstimulation effects, partitioned into “estim” and “rew × estim” types, are shown in Supplementary file 1b.

We found that the microstimulation effects on Δdrift (rew) and Δbound (rew) were most consistent with the coordinated-scaling scenario in Figure 4F, based on several observations. First, DDM variants with microstimulation-induced changes in both me and z outperformed variants with no such changes in me or z (Figure 5—figure supplement 2B and C; the “NoME” and “NoZ” models provided the best fits for only three sessions), suggesting that caudate microstimulation affected both drift rates and bounds. Second, despite the variations in the microstimulation effects on me and z across sessions (Figure 5A), the combined effects produced similar changes in bias to those measured using logistic fits (Figure 5—figure supplement 3), reminiscent of our previous observation that the monkeys used both Δdrift (rew) and Δbound (rew) to produce consistent reward biases. Third, the interaction (“rew × estim”) effects on Δdrift and Δbound maintained a highly consistent relationship across sessions. As expected for a coordinated bias strategy, the monkeys’ voluntary, reward-dependent adjustments were negatively correlated on trials without microstimulation (Figure 5B). Strikingly, and consistent with only the coordinated-effects scenarios in Figure 4E and F, the interaction effects between reward and microstimulation followed almost the same negative correlation (Figure 5D, compare red dashed line and black solid lines). The same patterns were observed in both monkeys (Figure 5—figure supplement 4). Fourth, there were strong negative correlations between the reward effects on trials without microstimulation and interaction effects for both Δdrift and Δbound (Figure 5C; also compare the orders of color progressions in B and D). This result is consistent with the scenario in Figure 4F and thus suggests the presence of a multiplicative component in the microstimulation effects on coordination. Together, these results show that caudate microstimulation recapitulated the coupling between reward context-dependent adjustments to drift rates and relative bound heights that was evident in the monkeys’ voluntary adjustments.

We considered four alternative explanations and found that the correlated “rew × estim” effects on drift and bound were not artifact, trivial, or a necessary consequence of reward-dependent biases. First, our methods of DDM fitting and parsing reward- and microstimulation-related effects might have introduced artificial correlations. This explanation would predict a similar correlation in the overall, reward-independent microstimulation effects on drift and bound. We did not observe such a correlation between Δdrift (estim) and Δbound (estim), despite similar magnitudes of changes to their reward-dependent counterparts (Figure 6A). To further control for this scenario, we shuffled all the fitted DDM parameters across sessions and simulated a new data set with matched trial numbers for each session (“Shuffle 1”). We fitted the full DDM to these simulated data and did not observe a negative correlation as in the original data (Figure 6B–D). Fits with the “NoCollapse” variant did not show negative correlation, either (Figure 6—figure supplement 1).

Figure 6. — (A) The average microstimulation effects on ∆drift and ∆bound that were independent of reward context were not correlated. Same format as Figure 5D. Linear regression, t-test, p=0.60. (**B-D**) The patterns of correlation in Figure 5B and D were not observed in data simulated with shuffled DDM parameters (Shuffle 1). ∆drift and ∆bound values were obtained by fitting the simulated data with the “Full” DDM model. Linear regression, t-test, p=0.78, 0.81, 0.26 for the three panels, respectively. (E) The observed relationship between Δdrift (rew × estim) and Δbound (rew × estim) was different from those observed in shuffled data. Shuffle 2 (red): shuffle ∆drift and ∆bound values for trials with and without microstimulation across sessions independently. Shuffle 3 (gray, solid): shuffle ∆drift and ∆bound values for microstimulation trials only and across sessions independently. Shuffle 4 (gray, dashed): shuffle ∆drift and ∆bound values for microstimulation trials only across sessions while maintaining the coupling between drift and bound. Histograms were obtained from 200 shuffles for each method. Vertical lines indicate the mean slope values.

Figure 6—figure supplement 1. — (A) The average microstimulation effects on ∆drift and ∆bound that were independent of reward context were not correlated. Same format as Figure 5D. Linear regression, t-test, p=0.60. (**B-D**) The patterns of correlation in Figure 5B and D were not observed in data simulated with shuffled DDM parameters (Shuffle 1). ∆drift and ∆bound values were obtained by fitting the simulated data with the “Full” DDM model. Linear regression, t-test, p=0.78, 0.81, 0.26 for the three panels, respectively. (E) The observed relationship between Δdrift (rew × estim) and Δbound (rew × estim) was different from those observed in shuffled data. Shuffle 2 (red): shuffle ∆drift and ∆bound values for trials with and without microstimulation across sessions independently. Shuffle 3 (gray, solid): shuffle ∆drift and ∆bound values for microstimulation trials only and across sessions independently. Shuffle 4 (gray, dashed): shuffle ∆drift and ∆bound values for microstimulation trials only across sessions while maintaining the coupling between drift and bound. Histograms were obtained from 200 shuffles for each method. Vertical lines indicate the mean slope values.

Second, the coordinated effects of microstimulation on Δdrift and Δbound might have been a trivial consequence of reward-dependent biases, independent of the reward context-dependent coordination of the two quantities. This possibility is countered by three findings: 1) the presence of correlated reward-dependent adjustments alone do not necessarily produce correlated “rew × estim” effects (e.g. Figure 4C and D); 2) removing coordinated, reward-induced effects by shuffling the best-fitting values of those two parameters independently across sessions (as in Figure 4D) resulted in substantially less correlated “rew × estim” effects than in the real data (Figure 6E, “Shuffle 2” and “Shuffle 3”, red and gray solid lines, compare to the data in black). That is, even with the mean and variance values matched exactly to the experimental data, the model in Figure 4D cannot capture the correlated pattern we observed in the experimental data; and 3) partial shuffling that disrupted only possible relationships with session-specific properties (e.g. microstimulation sites or voluntary performance), by shuffling the paired Δdrift and Δbound across sessions, also significantly weakened the correlation between “rew × estim” effects (Figure 6E, “Shuffle 4”).

Third, it is possible that the shared negative correlations in Figure 5B and D resulted from persistent effects of microstimulation throughout each session, that is, a spill-over of microstimulation effect. This possibility is inconsistent with three observations: 1) the negative correlations were present between Δdrift (rew) and Δbound (rew) in sessions without microstimulation (data not shown; Fan et al., 2018); 2) As described in Methods, microstimulation sessions began with trials for unit recording only, before any microstimulation was delivered. During these recording-only trials, negative correlations between Δdrift (rew) and Δbound (rew) were also observed, similar to the correlations during later, no-microstimulation trials interleaved with microstimulation (Figure 6—figure supplement 2), indicating that microstimulation was not necessary for the observed negative correlations; and 3) the reward effects on trials without microstimulation were negatively correlated with the interaction effects (Figure 5C), instead of a positive dependency as predicted by a “spill-over” mechanism with linear or exponential decay.

Fourth, given the inter-session variability in microstimulation effects, it is possible that the negatively correlated Δdrift (rew × estim) and Δbound (rew × estim) reflected only a minor consequence of caudate microstimulation. We examined this possibility by comparing: 1) the first principal component (PC) for the eight fitted parameters ([me or z] × [contra-LR or ipsi-LR] × [estim on or off]), with 2) the first PC for the two parameters Δdrift (rew × estim) and Δbound (rew × estim). We projected the eight-parameter and two-parameter data to these two PCs, respectively. The projections showed a strong correlation (Figure 6—figure supplement 3; rho = 0.92, p=1.3e-16), indicating that the coordinated Δdrift (rew × estim) and Δbound (rew × estim) values reflected the dominant feature of the experimental data; that is, the dominant effect of caudate microstimulation on reward-related biases.

Taken together, these results suggest that the shared correlation patterns in reward context-dependent voluntary and evoked adjustments were not trivial, but instead reflected the caudate nucleus’ causal involvements in mediating the monkeys’ specific strategy to balance visual evidence and reward asymmetry information.

Coordinated microstimulation effects depended on neural selectivity patterns and baseline adjustments

The microstimulation effects on Δdrift (rew) and Δbound (rew) tended to be negatively correlated with each other but also showed substantial variability across sessions (Figure 5D). We examined three potential sources of these effects: 1) the monkey’s baseline (microstimulation-independent) adjustments in Δdrift (rew) and Δbound (rew) (Figure 5B), 2) the task-related activity patterns of neurons recorded near the stimulation site, and 3) the anatomical location of the stimulation site. As detailed below, we found that microstimulation effects on Δdrift (rew) and Δbound (rew) depended systematically on both the baseline adjustments and neural selectivity at the site of microstimulation, but not the anatomical location of the site.

We first performed multiple linear regressions, using the baseline adjustments and neural selectivity at the site of microstimulation as the independent variables and the microstimulation effects as the dependent variables (Equation 4). We quantified the baseline adjustments in terms of the two principal components (PCs) of the relationship between Δdrift (rew) and Δbound (rew), which to a first approximation divides the effects into the average coordinated trajectory (the projection onto PC1) and the deviation from the average trajectory (the projection onto PC2). We quantified neural selectivity during motion viewing (Epoch 5in Figure 1A, when the decisions were formed) for each site using a multiple linear regression that also allowed us to identify neurons with combined modulations by visual evidence and reward context (n = 16 sites total, 9 and 7 for monkeys C and F, respectively; Equation 1). We quantified the microstimulation effects as the projections of Δdrift (rew × estim) and Δbound (rew × estim) onto the PCs defined from the baseline adjustments, which measured the degree to which the “rew × estim” interaction effects conformed to the voluntary adjustments in response to the asymmetric reward contexts (Figure 7A and B, right panels).

Figure 7. — (**A and B**) Principal components (PCs) were estimated for reward context modulation of Δdrift and Δbound without microstimulation (left panels; mean subtracted) for the two monkeys separately. The values of Δdrift (rew × estim) and Δbound (rew × estim) were projected onto these two PCs (right panels). (C) Visualization of the dependence of rew × estim effects on neural selectivity and reward effects without microstimulation for sites with combined modulations by motion coherence and reward context (top) and for other sites (bottom). Proj1(2): projection onto PC1(2). Each row shows the results from one multiple linear regression, with the quantity on the ordinate as the dependent variable and the quantities on the abscissa as the independent variables. Significance was assessed with bootstrap methods using 1000 shuffles of the independent variables. The number in each box indicates the number of shuffles with the same or stronger effect as the experimental data (e.g., six means an estimated p value of 0.006 and 0 means an estimated p value of < 0.001). Colors indicate the signs of regression coefficients (significance criterion: p<0.0125 = 0.05/4-regressions). (D) Percent explained variance for the regressions in (C), respectively. A black bar indicates that the experimental value is outside the 95 percentile of the bootstrapped distribution.

Figure 7—source data 1. Neural selectivity for different task factors at the microstimulation sites.
Column “indRewcontCoh” was used to divide the two subsets of stimulation sites in C and D. PCA was performed on DDM fitting results in Figure 5—source data 1.

elife-56694-fig7-data1.csv^{(3.9KB, csv)}

Figure 7—figure supplement 1. — (**A and B**) Principal components (PCs) were estimated for reward context modulation of Δdrift and Δbound without microstimulation (left panels; mean subtracted) for the two monkeys separately. The values of Δdrift (rew × estim) and Δbound (rew × estim) were projected onto these two PCs (right panels). (C) Visualization of the dependence of rew × estim effects on neural selectivity and reward effects without microstimulation for sites with combined modulations by motion coherence and reward context (top) and for other sites (bottom). Proj1(2): projection onto PC1(2). Each row shows the results from one multiple linear regression, with the quantity on the ordinate as the dependent variable and the quantities on the abscissa as the independent variables. Significance was assessed with bootstrap methods using 1000 shuffles of the independent variables. The number in each box indicates the number of shuffles with the same or stronger effect as the experimental data (e.g., six means an estimated p value of 0.006 and 0 means an estimated p value of < 0.001). Colors indicate the signs of regression coefficients (significance criterion: p<0.0125 = 0.05/4-regressions). (D) Percent explained variance for the regressions in (C), respectively. A black bar indicates that the experimental value is outside the 95 percentile of the bootstrapped distribution.

Figure 7—source data 1. Neural selectivity for different task factors at the microstimulation sites.
Column “indRewcontCoh” was used to divide the two subsets of stimulation sites in C and D. PCA was performed on DDM fitting results in Figure 5—source data 1.

elife-56694-fig7-data1.csv^{(3.9KB, csv)}

We found that the microstimulation effects along the average coordinated trajectory (i.e. the PC1 projection of rew × estim effects) depended: 1) negatively on the baseline effects along that trajectory (i.e., the PC1 projection of baseline Δdrift (rew) and Δbound (rew)), 2) negatively on neural selectivity for choice at the site of microstimulation, 3) positively on neural selectivity for reward context at the site of microstimulation, and 4) positively on neural selectivity for motion coherence for trials with contralateral choices at the site of microstimulation (Figure 7C, colored boxes). The explained variance of this regression was 89% and significantly higher than that of shuffled data (Figure 7D, top panel). The microstimulation effects perpendicular to the average coordinated trajectory also depended positively on neural selectivity for motion coherence for trials with contralateral choices, but the explained variance was not significantly different from that of shuffled data. In contrast, for other sites without combined modulations by motion coherence and reward context (n = 23 sites total, 9 and 14 for monkeys C and F, respectively), the microstimulation effects along the average coordinated trajectory also depended negatively on the baseline adjustments along that trajectory, but the microstimulation effects did not otherwise depend on the baseline adjustments or neural tuning from that session (Figure 7C, bottom panel). For these sites, the explained variance was substantially lower than for the sites with combined modulations and did not differ from the shuffled data (Figure 7D, bottom panel). These results imply that the microstimulation effects on the monkeys’ reward bias strategy reflected both their baseline reward-driven biases and specific contributions of caudate neurons at the site of microstimulation, particularly those encoding both reward context and motion coherence, instead of non-specific microstimulation effects from altering overall caudate activity.

In contrast, the microstimulation effects did not show consistent relationships with the anatomical locations of microstimulation sites. We reconstructed the locations of single-unit recording and microstimulation sites on MRI images based on the track coordinates and electrode depth (Figure 7—figure supplement 1). In monkey C, the recording sites spanned from 5 mm anterior to the anterior commissure to 3 mm posterior. In monkey F, the recording sites spanned from the anterior commissure to 7 mm anterior. Neurons with combined modulation by motion and reward-related quantities during motion viewing were observed at all anterior–posterior (A–P) levels (Figure 7—figure supplement 1A, red circles; corresponding to neurons identified in Figure 2—figure supplement 2). Microstimulation sites were sampled in a smaller region for both monkeys (Figure 7—figure supplement 1A, cyan crosses). For our sampled sites, we observed certain tendencies, but these tendencies were not consistent across monkeys. For example, in monkey F, but not monkey C, sites with rew × estim effects on the monkeys’ choice and RT, as measured separately using logistic and linear fits, appeared clustered at more ventral locations (Figure 7—figure supplement 1B and C). In monkey C, but not monkey F, Δdrift (rew × estim) appeared to be greater at more ventral locations (Figure 7—figure supplement 1D). Although we sampled a larger region in monkey C than in monkey F, monkey F showed a larger range of Δdrift (rew × estim) and Δbound (rew × estim) (Figure 7—figure supplement 1D and E). These results suggest that anatomical organization patterns are likely variable and confined to small areas in the caudate nucleus. Consistent with the lack of a large-scale organization pattern, adding location data as independent variables in the regressions for Figure 7 did not significantly improve the explained variance. These results suggest that the rew × estim effects of caudate microstimulation were more closely linked with neural selectivity at the sites than the physical locations of the sites.

Discussion

For monkeys making saccade decisions based on noisy visual motion evidence and asymmetric reward contexts, we observed that the activity of many single neurons in the caudate nucleus was sensitive to both motion strength and reward context or expected reward size. We further showed that microstimulation at these caudate sites evoked changes in the monkeys’ choices and RTs that depended on the reward context and involved adjustments to the drift rates and relative bound heights of an accumulate-to-bound (DDM) decision process. These reward context-dependent microstimulation effects mimicked the coordinated adjustments that the monkeys made, in the absence of microstimulation, to the drift rates and relative bound heights in response to changes in reward context. These evoked, coordinated adjustments also depended on neural selectivity at the microstimulation sites. Taken together, these findings support theoretical proposals that the basal ganglia pathway implements general-purpose computations for action selection, including selection based on multiple types of inputs (Berns and Sejnowski, 1995; Bogacz and Gurney, 2007; Mink, 1996; Rao, 2010; Ratcliff and Frank, 2012; Redgrave et al., 1999). Below we consider how these results inform our understanding of the specific role of the caudate nucleus in forming decisions that integrate sensory and reward information.

The role of the caudate nucleus in coordinating the reward context-dependent modulation of choice bias-related computations (drift rates and bounds in the DDM) appears to be distinct from its previously demonstrated roles in mediating sensory evidence-based decisions. For example, we previously showed that the caudate nucleus contributes causally to adjusting the decision bound (z in the DDM) for decisions based on visual information alone (Ding and Gold, 2012b). In the present study, such an effect would be more related to the Δbound (estim) measurement, instead of the “rew × estim” effects. We did not observe a consistent Δbound (estim) effect in the present study, possibly because of differences in task context (the previous study used an equal reward decision task), the targeted neural subpopulation (the previous study targeted sites with choice-selective activity, whereas the present study targeted sites with additional selectivity for reward-related factors), and/or anatomical locations of the stimulation sites (Figure 7—figure supplement 1A, compare blue and cyan crosses; in the previous study, the sites were more posterior and dorsal in monkey C and more anterior and dorsal in monkey F).

The differences in anatomical locations likely also contributed to the apparent sign difference in reward-independent effect on choice bias: microstimulation at sites in our previously published study induced predominantly ipsilateral biases, whereas microstimulation at other caudate sites induced predominantly contralateral biases for the same equal-reward task (Doi et al., Society for Neurosicence 2012, 198.10). In the present study, the reward-independent microstimulation effects on bias were highly variable and showed an average contralateral bias. In comparison, studies of striatal perturbation in rodents showed consistent inactivation-induced ipsilateral biases and activation-induced contralateral biases (Wang et al., 2018; Yartsev et al., 2018). Interestingly, the effects of caudate microstimulation in monkeys seem more variable than the effects of striatal perturbations in rodents (Amemori et al., 2018; Wang et al., 2018; Yartsev et al., 2018), possibly reflecting the smaller relative volume of tissue that were affected and consequently greater dependence on local activity patterns in the monkey caudate. This difference may also contribute to the lack of effects of striatal lesions on rats’ decisions on a task with manipulations of olfactory evidence and reward context (Wang et al., 2013), along with other differences between species (e.g. the extent of functional equivalence between rat dorsomedial striatum and monkey caudate nucleus) and task designs (e.g. interleaved trials with and without microstimulation minimized any compensatory effects).

The coordinated microstimulation effects reported here also differ from earlier demonstrations of caudate’s involvement in reward/value-dependent behaviors. We found some caudate neurons that exhibited coherence and reward size modulations that were consistent with a flexible value signal that has been observed in the monkey striatum (Kawagoe et al., 1998; Kim and Hikosaka, 2013; Samejima et al., 2005). However, in our task, the coherence-driven decision variable can also be used as a surrogate for confidence, which should be examined in more detail to disentangle its representation from a representation of value in the caudate. One perturbation study showed that pairing caudate microstimulation consistently with a specific stimulus can alter the subjects’ choice by increasing the estimated value of that stimulus (Santacruz et al., 2017). In our study, such an effect on choice bias would have been canceled when microstimulation was delivered randomly but counterbalanced for the two motion directions. Another study showed that pairing pre-saccade caudate microstimulation with a saccade direction can alter RTs on an instructed saccade task with highly visible stimuli (Nakamura and Hikosaka, 2006a). Similar effects likely contributed to our observed changes in non-decision times but do not account for our effects on choice. Finally, caudate microstimulation can prolong saccade RTs on an instructed saccade task by reducing the rate of rise in the LATER model (Watanabe and Munoz, 2010), which assumes a linear rise to a fixed bound (Carpenter and Williams, 1995; Reddi et al., 2003). Similar effects likely contributed to our observed changes in parameter k in the DDM but not the effects on reward-dependent biases that we found. Thus, the coordinated “rew × estim” effects on drift rates and relative bound heights that we observed here reflect a computational role for the caudate nucleus in reward-biased decisions that is distinct from the roles that have been identified previously.

The caudate nucleus’ likely role in coordinating a cognitive process, within the realm of a voluntary decision strategy, is reminiscent of the demonstration that microstimulation in the premotor areas can evoke coordinated body movements, within the realm of natural defensive maneuvers (Graziano et al., 2002). Together, these results suggest a general principle of brain organization, such that motor or cognitive primitives are aggregated into behaviorally relevant combinations. Such a hierarchical organization might facilitate learning and flexible adaptation of both motor and cognitive behaviors (Brooks, 1986). Within the hierarchy, the coordinated “rew × estim” effects on drift rates and relative bound heights, which measured changes in the reward-dependent biasing components of the decision process, might reflect the contributions of a subset of caudate neurons to the coordination of cognitive primitives. In contrast, the uncorrelated “estim” effects on drift rates and relative bound heights, which measured changes in reward-independent biases in the decision process, may reflect another subset of caudate neurons’ contributions to the implementation of those primitives. These results thus suggest that the caudate nucleus may serve multiple decision-related functions, consistent with the heterogeneous neural modulation patterns across the population (Nakamura and Ding, 2017).

Our interpretation of the caudate nucleus’ roles was based on an algorithmic description of the decision process using the DDM framework. Given current uncertainty in the field about the neural implementation of DDM-like decision processes (Lo and Wang, 2006; Rao, 2010; Schall, 2019; Wei et al., 2015), more work is needed to understand how the caudate nucleus interacts with the rest of the cortico-basal ganglia circuitry to implement these algorithms for sensory evidence- and reward-dependent decisions. Here, we propose some possibilities for implementing the negatively correlated Δdrift (rew) and Δbound (rew) that we observed, as follows.

First, Δdrift (rew) and Δbound (rew) could be implemented separately in the brain but then converge in the caudate to be coordinated. Δdrift (rew) might reflect reward modulation of the sensory representation in the middle temporal (MT) visual area, where microstimulation can induce choice biases that depend on the average reward expectation for the two choices (Cicmil et al., 2015). Such modulation might also be relayed from frontal or parietal regions to the caudate nucleus, although existing data, obtained using a similar task with a fixed motion-viewing duration, argue against the direct involvement by lateral intraparietal area (LIP) neurons in implementing reward-dependent drift rate biases (Rorie et al., 2010). In contrast, Δbound (rew) might reflect reward context-dependent activity in the caudate nucleus and elsewhere (including in the LIP), which can emerge before stimulus onset (Coe et al., 2002; Ding and Gold, 2010; Ding and Hikosaka, 2006; Ikeda and Hikosaka, 2003; Kobayashi et al., 2007; Lauwereyns et al., 2002a; Lauwereyns et al., 2002b; Roesch and Olson, 2003; Rorie et al., 2010; Sato and Hikosaka, 2002).

Second, Δdrift (rew) and Δbound (rew) could be implemented as a reward context-dependent, time-varying dynamic bias in the decision variable. The idea of a dynamic bias has been proposed to account for the effects of prior information on perceptual decisions (Hanks et al., 2011). In our case, the caudate nucleus may control the magnitude and time course of such a bias according to reward context. A similar involvement of the caudate nucleus in the incorporation of prior information may contribute to deficits observed in Parkinsonian patients (Perugini et al., 2016).

Third, coordinated Δdrift (rew) and Δbound (rew) might be equivalent to reward context- and time-dependent asymmetric modulations of the two choice bounds. Although caudate activity does not appear to reflect the final bound height at the time of decision (Ding and Gold, 2012b), the caudate nucleus may contribute to modulation of the bounds in the earlier phase of decision formation to bias decisions toward the large-reward options. Future experiments with shorter-duration perturbation delivered at different time points during motion viewing may provide further insights into the caudate nucleus’ roles (Yartsev et al., 2018).

Fourth, Δdrift (rew) and Δbound (rew) might be implemented as reward modulation of network connectivity in a recurrent cortico-basal ganglia network (Lo and Wang, 2006; Wei et al., 2015). It would be interesting to test if and how perturbations of other components of this network (e.g. the frontal and parietal cortical areas, superior colliculus) affect performance on our task.

Collectively, our results suggest that caudate neurons contribute causally to decisions that combine asymmetric rewards and visual information. These contributions are diverse and complex, possibly operating on different levels in hierarchical cognitive processes. Such diversity of functions might partially explain the complicated nature of decision-making impairments with striatal dysfunction.

Materials and methods

Key resources table.

Reagent type (species) or resource	Designation	Source or reference	Identifiers
Software and Algorithms	Python 3.5	Python Software Foundation	https://www.python.org/
Software and Algorithms	MATLAB	Mathworks	https://www.mathworks.com
Software and Algorithms	Psychophysics Toolbox	Kleiner et al., 2007	http://psychtoolbox.org/
Software and Algorithms	Pandas v0.19.2	Python Data Analysis Library	https://pandas.pydata.org/
Software and Algorithms	Scikit-learn v0.18.1	Pedregosa et al., 2011	https://scikit-learn.org/stable/
Software and Algorithms	Scipy v0.18.1	SciPy.org	https://docs.scipy.org/doc/scipy/reference/stats.html

Open in a new tab

Experimental model and subject details

We used two adult male rhesus macaques (Macaca mulatta) for this study. They were first trained extensively on an equal-reward reaction-time random-dot motion discrimination task (Ding and Gold, 2010; Ding and Gold, 2012a; Ding and Gold, 2012b) and then trained with the asymmetric-reward contexts (Fan et al., 2018). All training and experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the University of Pennsylvania Institutional Animal Care and Use Committee (protocol #804726).

Method details

Behavioral task

Task details are reported elsewhere (Fan et al., 2018). Briefly, a trial began with a central fixation point presentation (Figure 1A). Upon acquiring and maintaining fixation, two choice targets were presented to inform the monkeys the two possible motion directions. After a random delay picked from a truncated exponential distribution (mean = 0.7 s, range: 0.4–2.5 s), the fixation point was dimmed and a random-dot kinematogram was shown at the center of the screen (“motion onset”). For each trial, the kinematogram had a constant speed of 6°/s, aperture size of 5°, and randomly interleaved motion direction and strength (five levels of coherence: 3.2, 6.4, 12.8, 25.6, 51.2%). The monkey reported the perceived motion direction by making a self-timed saccade to the corresponding choice target. A minimum 50 ms latency was imposed, although the monkeys rarely made fast-guess responses during this study. Once the monkey’s gaze exited the fixation window (4° square window), the kinematogram was extinguished. Once the monkey’s gaze reached the choice target window (4° square window), a 400 ms minimum fixation time was imposed to register the monkey’s choice. Correct choices were rewarded with juice. Error choices were not rewarded and penalized with a timeout before the next trial (3 s for monkey F, 0.5–2 s for monkey C).

Two asymmetric reward contexts were alternated in a block design. In Contra-LR blocks, the choice contralateral to the recording/stimulation site was paired with large reward (LR). In Ipsi-LR blocks, the choice ipsilateral to the recording/stimulation site was paired with large reward. The other choice was paired with small reward. At the start of each block, the choice targets were presented with different colors to signal the current reward context to the monkeys, followed by two additional high-coherence trials to allow the monkeys to experience the current reward context. These trials were excluded from analysis. For recording sessions, each block consisted of ~49 trials for monkey C (IQR: 43–55) and 40 trials for monkey F (IQR: 36–55). For microstimulation sessions, each block consisted of ~100 trials (IQR: 98–118) for monkey C and ~61 trials (IQR: 60–61) for monkey F. The large:small reward ratio was ~2:1 (2.02 ± 0.19) for monkey C for both recording and microstimulation sessions. For monkey F, the large:small reward ratio was 3:2 (1.48 ± 0.13) for recording sessions and 5:3 (1.65 ± 0.03) for microstimulation sessions. For monkey C, the average small reward was 0.1 and 0.07 mL and the average large reward was 0.2 and 0.12 mL during recording and microstimulation sessions, respectively. For monkey F, the average small reward was 0.2 and 0.12 mL and the average large reward was 0.29 and 0.25 mL during recording and microstimulation sessions, respectively.

Data acquisition

Eye position was monitored using a video-based system (ASL) sampled at 240 Hz. Single-unit recordings focused on putative projection neurons (Ding and Gold, 2010). We searched for task-relevant neurons while the monkeys performed the equal-reward motion discrimination task with horizontal dots motions and determined the presence of task-related modulation of neural activity by visual and audio inspection of ~10–20 trials. For analyses of neural response properties in recording sessions, only well-isolated single units were included. For analyses of microstimulation effects, sites with either single- or multi-unit task-related modulations were used. Neural signals were amplified, filtered and stored using a MAP acquisition system (Plexon, Inc), along with time-stamped event codes, analog eye position signals and trial parameter values. Single-unit activity was identified by offline spike sorting (Offline Sorter, Plexon, Inc). Multi-unit activity was measured using waveforms that passed an offline amplitude threshold. For the microstimulation experiments, we first identified a caudate site with task-related activity and then interleaved trials with and without microstimulation pseudo-randomly at a 1:1 ratio. Electrical microstimulation was delivered during motion stimulus presentation (negative-leading bipolar current pulses, 300 Hz, 50 µA, 250 µs pulse duration; from motion onset until a saccade was detected). Caudate microstimulation with these parameters did not evoke saccades (Ding and Gold, 2012b; Nakamura and Hikosaka, 2006a; Watanabe and Munoz, 2010).

Quantification and statistical analysis

Neural data analysis

For each single/multi-unit dataset, we computed the average firing rates in seven task epochs (Figure 1A): three epochs before motion stimulus onset (Epoch 1: 400 ms window beginning at target onset, Epoch 2: a variable window from target onset to motion onset, and Epoch 3: a 400 ms window ending at motion onset), two epochs during motion viewing (Epoch 4: a fixed window from 100 ms after motion onset to 100 ms before median RT, and Epoch 5: a variable window from 100 ms after motion onset to 100 ms before saccade onset), a peri-saccade 300 ms window beginning at 100 ms before saccade onset (Epoch 6), and a post-saccade 400 ms window beginning at saccade onset (Epoch 7; before feedback and reward delivery). For each unit, a multiple linear regression was performed on the firing rates in correct trials, for each task epoch separately.

\begin{array}{ll} F R & = β_{0} + β_{C h o i c e} \times I_{C h o i c e} + β_{R e w C o n t} \times I_{R e w C o n t} + β_{R e w S i z e} \times I_{R e w S i z e} \\ + β_{C o h - C o n t r a} \times I_{C o h - C o n t r a} + β_{C o h - I p s i} \times I_{C o h - I p s i} \\ + β_{R e w C o h - C o n t r a} \times I_{C o h - C o n t r a} \times I_{R e w S i z e} + β_{R e w C o h - I p s i} \times I_{C o h - I p s i} \times I_{R e w S i z e}, \end{array}

(1)

where

I_{C h o i c e} = {\begin{cases} 1 f o r c o n t r a l a t e r a l c h o i c e \\ - 1 f o r i p s i l a t e r a l c h o i c e \end{cases},

I_{R e w C o n t} = {\begin{cases} 1 f o r c o n t r a l a t e r a l - l a r g e r e w a r d b l o c k s \\ - 1 f o r i p s i l a t e r a l - l a r g e r e w a r d b l o c k s \end{cases},

I_{R e w S i z e} = {\begin{cases} 1 i f a l a r g e r e w a r d i s e x p e c t e d f o r t h e c h o i c e \\ - 1 i f a s m a l l r e w a r d i s e x p e c t e d f o r t h e c h o i c e \end{cases},

I_{C o h - C o n t r a} = {\begin{cases} a b s o l u t e c o h e r e n c e f o r c o n t r a l a t e r a l c h o i c e (c e n t e r e d a t m e a n v a l u e) \\ 0 f o r i p s i l a t e r a l c h o i c e \end{cases},

and

I_{C o h - I p s i} = {\begin{cases} 0 f o r c o n t r a l a t e r a l c h o i c e \\ a b s o l u t e c o h e r e n c e f o r i p s i l a t e r a l c h o i c e (c e n t e r e d a t m e a n v a l u e) \end{cases} .

Significance of non-zero coefficients was assessed using t-test (criterion: p=0.05).

Behavioral analysis

Measuring microstimulation effects on choice using a logistic function

For each microstimulation session, a logistic function was fitted to the choice data for all trials:

P_{c o n t r a c h o i c e} = \frac{1}{1 + e^{- S l o p e \times (C o h + B i a s)}},

(2)

where $C o h$ is the signed motion coherence,

S l o p e = s l o p e 0 + s l o p e_{r e w} \times R e w C o n t + s l o p e_{e s t i m} \times E s t i m + s l o p e_{r e w \times e s t i m} \times R e w C o n t \times E s t i m,

B i a s = b i a s 0 + b i a s_{r e w} \times R e w C o n t + b i a s_{e s t i m} \times E s t i m + b i a s_{r e w \times e s t i m} \times R e w C o n t \times E s t i m,

R e w C o n t = {\begin{cases} 1 f o r c o n t r a l a t e r a l - l a r g e r e w a r d b l o c k s \\ - 1 f o r i p s i l a t e r a l - l a r g e r e w a r d b l o c k s \end{cases},

and

E s t i m = {\begin{cases} 1 f o r m i c r o s t i m u l a t i o n t r i a l s \\ 0 f o r c o n t r o l t r i a l s \end{cases} .

Significance of non-zero coefficients was tested using bootstrap methods by shuffling $E s t i m$ values across trials for each session 200 times (criterion: the experimental value is outside the 95% confidence intervals for shuffled data).

Measuring microstimulation effects on RT using a linear function

RT was measured as the time from motion onset to saccade onset, the latter identified offline with respect to velocity (>40°/s) and acceleration (>8000°/s²). A linear function was fitted to RT data for correct trials:

\begin{array}{ll} R T & = r t 0 + I n t e r c e p t_{r e w} \times R e w C o n t + S l o p e_{r e w} \times R e w C o n t \times C o h_{a b s} \\ + I n t e r c e p t_{e s t i m} \times E s t i m + S l o p e_{e s t i m} \times E s t i m \times C o h_{a b s} \\ + I n t e r c e p t_{r e w \times e s t i m} \times R e w C o n t \times E s t i m + S l o p e_{r e w \times e s t i m} \times R e w C o n t \times E s t i m \times C o h_{a b s}, \end{array}

(3)

where ${C o h}_{a b s}$ is the absolute value of motion coherence. Significance of non-zero coefficients was tested using t-test (criterion: p = 0.05).

Measuring microstimulation effects on both choice and RT using the drift-diffusion model (DDM)

We also fitted the choice and RT data for all trials simultaneously to different variants of the drift-diffusion model (DDM; Figure 4A). The basic DDM assumed that the decision variable (DV) is the time integral of evidence ( $E$ ), which was modeled as a Gaussian distributed random variable,

E \sim N (k * c o h e r e n c e, 1) a n d D V = \int E d t

The scale parameter k controlled the drift rate. At each time point, the DV was compared with two collapsing choice bounds (Zylberberg et al., 2016). The time course of the choice bounds was specified as $a / (1 + e^{β_a l p h a (t - β_d)})$ , where $β_a l p h a$ and $β_d$ controlled the rate and onset of decay, respectively. Note that the effects of collapsing bounds can be equivalently implemented by adding a choice-independent urgency signal (Drugowitsch et al., 2012; Churchland et al., 2008). If DV crossed the upper bound first, a contralateral choice was made; if DV crossed the lower bound first, an ipsilateral choice was made. RT was modeled as the sum of the time till first bound crossing and saccade-specific non-decision times that accounted for evidence-independent sensory/motor delays (t_contra and t_ipsi). Two types of biases were used to account for reward asymmetry-induced biases, a bias in drift rate (me) and a bias in the relative bound heights (z) (Fan et al., 2018).

DDM model fitting was performed, separately for each session, using the maximum a posteriori estimate method (python v3.5.1, pymc 2.3.6) and prior distributions suitable for human and monkey subjects (Wiecki et al., 2013). We performed at least five runs for each variant and used the run with the highest likelihood for further analyses.

We used eight variants of the DDM model: in the “Full” model, all parameters were allowed to vary by reward context and microstimulation status; in the “NoEstim” model, all parameters were allowed to vary by reward context, but not microstimulation status (Figure 5—figure supplement 2A); in the “NoCollapse” model, $β_a l p h a$ and $β_d$ were fixed across microstimulation status; in the “NoA”, “NoK”, “NoME”, and “NoZ” models, a, k, me, and z were fixed across microstimulation status, respectively; and in the “NoT0” model, saccade-specific non-decision times (t_contra and t_ipsi) were fixed across microstimulation status (Figure 5—figure supplement 2B and C). We used the Akaike information criterion (AIC) for model comparisons, with lower values indicating more parsimonious model variants.

For a given DDM parameter ( $P a r a$ ), we parsed the different effects as follows (LR: large reward):

Δ P a r a (b a s e) = (P a r a_{c o n t r a - L R, n o e s t i m} + P a r a_{i p s i - L R, n o e s t i m}) / 2

Δ P a r a (r e w) = (P a r a_{c o n t r a - L R, n o e s t i m} - P a r a_{i p s i - L R, n o e s t i m}) / 2

Δ P a r a (e s t i m) = (P a r a_{c o n t r a - L R, e s t i m} + P a r a_{i p s i - L R, e s t i m} - P a r a_{c o n t r a - L R, n o e s t i m} - P a r a_{i p s i - L R, n o e s t i m}) / 2

Δ P a r a (r e w \times e s t i m) = (P a r a_{c o n t r a - L R, e s t i m} - P a r a_{i p s i - L R, e s t i m} - P a r a_{c o n t r a - L R, n o e s t i m} + P a r a_{i p s i - L R, n o e s t i m}) / 2

Generating predictions for hypothesized reward context-microstimulation interaction effects on drift rate and bound height

We simulated four types of microstimulation effects (Figure 4). For all simulations, we assumed that,

Δ b o u n d (r e w) = γ \times Δ d r i f t (r e w) + δ,

μ_{d r i f t} = m e a n (Δ d r i f t (r e w \times e s t i m)), σ_{d r i f t} = s t d (Δ d r i f t (r e w \times e s t i m)),

μ_{b o u n d} = m e a n (Δ b o u n d (r e w \times e s t i m)), σ_{b o u n d} = s t d (Δ b o u n d (r e w \times e s t i m)),

where $γ$ and $δ$ are the slope and intercept measured from linear regression of Δdrift (rew) and Δbound (rew) in the experimental data (Figure 5B). Δbound (rew) for different sessions (colors in Figure 4B) were set as equally spaced values within the range for the experimental data.

For the simulations in Figure 4C,

Δ d r i f t (r e w \times e s t i m) \sim N o r m a l (0, σ_{d r i f t}),

Δ b o u n d (r e w \times e s t i m) \sim N o r m a l (0, σ_{b o u n d}),

For the simulation in Figure 4D,

Δ d r i f t (r e w \times e s t i m) \sim N o r m a l (μ_{d r i f t}, σ_{d r i f t}),

Δ b o u n d (r e w \times e s t i m) \sim N o r m a l (μ_{b o u n d}, σ_{b o u n d}),

For the simulation in Figure 4E,

Δ d r i f t (r e w \times e s t i m) \sim N o r m a l (μ_{d r i f t}, σ_{d r i f t}),

Δ b o u n d (r e w \times e s t i m) \sim γ \times Δ d r i f t (r e w \times e s t i m) + N o r m a l (0, σ_{b o u n d} - γ \times σ_{d r i f t}),

For the simulation in Figure 4F,

Δ d r i f t (r e w \times e s t i m) \sim k \times Δ d r i f t (r e w) + m + N o r m a l (0, σ_{d r i f t} / 2),

Δ b o u n d (r e w \times e s t i m) \sim γ \times Δ d r i f t (r e w \times e s t i m) + N o r m a l (0, σ_{b o u n d} / 2),

where $k$ and $m$ are slope and intercept, respectively, measured from linear regression of Δ $∆ d r i f t (r e w \times e s t i m)$ and Δ $∆ d r i f t (r e w)$ in the experimental data. The noise terms are reduced to account for variations in Δ $∆ d r i f t (r e w)$ .

Δdrift (rew × estim) and Δbound (rew × estim)

We performed three shuffling-based control analyses (Figure 6). For data shown in Figure 6B–D, we pooled the fitted DDM values across reward-microstimulation conditions and sessions for each parameter, resampled from these values with replacement for each session and reward-microstimulation condition, simulated sessions with matched numbers of trials for each condition using the resampled parameter values, and re-fitted these simulated data with the Full and NoCollapse DDM models (“Shuffle 1”). For data shown in Figure 6E, we performed 200 iterations of shuffling for three types of schemes. For “Shuffle 2”, we shuffled the fitted me and z values for trials with and without microstimulation across sessions. For “Shuffle 3”, we kept the me and z values for trials without microstimulation and shuffled me and z values independently for trials with microstimulation. For “Shuffle 4”, we kept the me and z values for trials without microstimulation and shuffled the session identity for paired me and z values for trials with microstimulation. The fitted values from the simulated/shuffled data were parsed in the same way as those from the experimental data. We considered the experimental result to be significantly different from shuffled results if the former was outside the 95% confidence intervals of the latter.

Relating microstimulation effects to neural selectivity and the monkeys’ voluntary strategy

For the microstimulation experiments, we recorded single- or multi-unit activity before microstimulation and performed the multiple linear regression in Equation 1 for each unit separately. The regression coefficients were normalized by $β_{0}$ in the regression, which measured the average firing rates. These $β$ values indexed the neural selectivity for different task-related regressors. If more than one unit was recorded at a site, for each regressor, we used the $β$ value associated with the lowest p value (i.e. most reliable modulation).

We derived two principal components (PCs) from Δdrift (rew, no estim) and Δbound (rew, no estim) values (Figure 7A and B, left). The projections of Δdrift (rew, no estim) and Δbound (rew, no estim) values onto the two PCs indexed the effects of reward context on the monkeys’ voluntary strategy in each session, relative to their average tendencies across sessions.

We projected Δdrift (rew × estim) and Δbound (rew × estim) values onto the same two PCs (Figure 7A and B, right). These projections indexed the microstimulation effects in each session, relative to the monkeys’ average tendencies in their reward context-dependent voluntary strategy across sessions. We then performed the following multiple linear regression:

\begin{array}{ll} P r o j e c t i o n o f (r e w \times e s t i m) v a l u e s & = α_{0} + β_{C h o i c e} \times α_{C h o i c e} + β_{R e w C o n t} \times α_{R e w C o n t} + β_{R e w S i z e} \times α_{R e w S i z e} \\ + β_{C o h - C o n t r a} \times α_{C o h - C o n t r a} + β_{C o h - I p s i} \times α_{C o h - I p s i} \\ + β_{C o h R e w - C o n t r a} \times α_{C o h R e w - C o n t r a} + β_{C o h R e w - I p s i} \times α_{C o h R e w - I p s i} \\ + P r o j e c t i o n o f (r e w, n o e s t i m) v a l u e s \times α_{p r o j e c t i o n} \end{array}

(4)

The $α$ values (except for $α_{0}$ ) measured the dependence of microstimulation effects on neural selectivity and the monkeys’ daily variations in their voluntary strategy. Significance of $α$ values was assessed by comparing the experimental value with their corresponding null distributions. The null distributions were estimated using 1000 iterations of regression based on shuffled independent variables. The number of iterations with values exceeding the experimental values was used to estimate p values for regression coefficients (Figure 7C) and explained variance (Figure 7D). A criterion of p<0.0125 was used to correct for multiple regressions (0.05/4 regressions).

Acknowledgements

We thank Jean Zweigle for animal care and Drs. Kae Nakamura and Ethan Bromberg-Martin for helpful comments. This work was supported by NIH National Eye Institute (R01-EY022411; LD and JIG), University of Pennsylvania (University Research Foundation Pilot Award; LD), and Hearst Foundations Graduate student fellowship (YF).

Funding Statement

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Contributor Information

Long Ding, Email: lding@pennmedicine.upenn.edu.

Daeyeol Lee, Johns Hopkins University, United States.

Michael J Frank, Brown University, United States.

Funding Information

This paper was supported by the following grants:

National Eye Institute R01-EY022411 to Joshua I Gold, Long Ding.
Hearst Foundations Graduate Student Fellowship to Yunshu Fan.
University of Pennsylvania University Research Foundation Pilot Award to Long Ding.

Additional information

Competing interests

Senior editor, eLife.

No competing interests declared.

Author contributions

Data curation, Formal analysis, Investigation, Visualization, Methodology, Writing - review and editing.

Funding acquisition, Visualization, Writing - review and editing.

Conceptualization, Resources, Data curation, Formal analysis, Supervision, Funding acquisition, Validation, Investigation, Visualization, Methodology, Writing - original draft, Project administration, Writing - review and editing.

Ethics

Animal experimentation: All training and experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the University of Pennsylvania Institutional Animal Care and Use Committee (protocol #804726).

Additional files

Supplementary file 1. a, Median and p values for microstimulation-induced effects for all 55 sites, as measured by logistic fits to the choice data and linear fits to the RT data.P values were from Wilcoxon signed rank test.

Bold: p<0.05. b, Median and p values for microstimulation-induced effects in 39 effective sites, as measured by best DDM fits to the choice and RT data. P values were from Wilcoxon signed rank test. Bold: p<0.05.

elife-56694-supp1.docx^{(16KB, docx)}

Transparent reporting form

elife-56694-transrepform.docx^{(246.3KB, docx)}

Data availability

Source data are uploaded for main data figures/tables (i.e. Figure 2, Figure 3 and figure supplement 1, Figure 5 and its figure supplements 2 and 4, Figure 6A and E, Figure 6 -figure supplement 3, Figure 7, and Supplementary file 1).

References

Amemori K, Amemori S, Gibson DJ, Graybiel AM. Striatal microstimulation induces persistent and repetitive negative Decision-Making predicted by striatal Beta-Band oscillation. Neuron. 2018;99:826–841. doi: 10.1016/j.neuron.2018.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
Berns GS, Sejnowski TJ. How the basal ganglia make decisions. In: Damasio AR, Damasio H, editors. Neurobiology of Decision-Making. Berlin, Springer-Verlag; 1995. pp. 101–113. [DOI] [Google Scholar]
Bogacz R, Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Computation. 2007;19:442–477. doi: 10.1162/neco.2007.19.2.442. [DOI] [PubMed] [Google Scholar]
Brooks R. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation. 1986;2:14–23. doi: 10.1109/JRA.1986.1087032. [DOI] [Google Scholar]
Cai X, Kim S, Lee D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron. 2011;69:170–182. doi: 10.1016/j.neuron.2010.11.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
Carpenter RH, Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature. 1995;377:59–62. doi: 10.1038/377059a0. [DOI] [PubMed] [Google Scholar]
Cavanagh JF, Wiecki TV, Cohen MX, Figueroa CM, Samanta J, Sherman SJ, Frank MJ. Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nature Neuroscience. 2011;14:1462–1467. doi: 10.1038/nn.2925. [DOI] [PMC free article] [PubMed] [Google Scholar]
Churchland AK, Kiani R, Shadlen MN. Decision-making with multiple alternatives. Nature Neuroscience. 2008;11:693–702. doi: 10.1038/nn.2123. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cicmil N, Cumming BG, Parker AJ, Krug K. Reward modulates the effect of visual cortical microstimulation on perceptual decisions. eLife. 2015;4:e07832. doi: 10.7554/eLife.07832. [DOI] [PMC free article] [PubMed] [Google Scholar]
Coe B, Tomihara K, Matsuzawa M, Hikosaka O. Visual and anticipatory Bias in three cortical eye fields of the monkey during an adaptive decision-making task. The Journal of Neuroscience. 2002;22:5081–5090. doi: 10.1523/JNEUROSCI.22-12-05081.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
Diederich A, Busemeyer JR. Modeling the effects of payoff on response Bias in a perceptual discrimination task: bound-change, drift-rate-change, or two-stage-processing hypothesis. Perception & Psychophysics. 2006;68:194–207. doi: 10.3758/BF03193669. [DOI] [PubMed] [Google Scholar]
Ding L, Gold JI. Caudate encodes multiple computations for perceptual decisions. Journal of Neuroscience. 2010;30:15747–15759. doi: 10.1523/JNEUROSCI.2894-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ding L, Gold JI. Neural correlates of perceptual decision making before, during, and after decision commitment in monkey frontal eye field. Cerebral Cortex. 2012a;22:1052–1067. doi: 10.1093/cercor/bhr178. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ding L, Gold JI. Separate, causal roles of the caudate in Saccadic choice and execution in a perceptual decision task. Neuron. 2012b;75:865–874. doi: 10.1016/j.neuron.2012.07.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ding L, Gold JI. The basal ganglia's contributions to perceptual decision making. Neuron. 2013;79:640–649. doi: 10.1016/j.neuron.2013.07.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ding L, Hikosaka O. Comparison of Reward Modulation in the Frontal Eye Field and Caudate of the Macaque. Journal of Neuroscience. 2006;26:6695–6703. doi: 10.1523/JNEUROSCI.0836-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
Drugowitsch J, Moreno-Bote R, Churchland AK, Shadlen MN, Pouget A. The cost of accumulating evidence in perceptual decision making. Journal of Neuroscience. 2012;32:3612–3628. doi: 10.1523/JNEUROSCI.4010-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fan Y, Gold JI, Ding L. Ongoing, rational calibration of reward-driven perceptual biases. eLife. 2018;7:e36018. doi: 10.7554/eLife.36018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Feng S, Holmes P, Rorie A, Newsome WT. Can monkeys choose optimally when faced with noisy stimuli and unequal rewards? PLOS Computational Biology. 2009;5:e1000284. doi: 10.1371/journal.pcbi.1000284. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gao J, Tortell R, McClelland JL. Dynamic integration of reward and stimulus information in perceptual decision-making. PLOS ONE. 2011;6:e16749. doi: 10.1371/journal.pone.0016749. [DOI] [PMC free article] [PubMed] [Google Scholar]
Graziano MS, Taylor CS, Moore T. Complex movements evoked by microstimulation of precentral cortex. Neuron. 2002;34:841–851. doi: 10.1016/S0896-6273(02)00698-0. [DOI] [PubMed] [Google Scholar]
Hanks TD, Mazurek ME, Kiani R, Hopp E, Shadlen MN. Elapsed decision time affects the weighting of prior probability in a perceptual decision task. Journal of Neuroscience. 2011;31:6339–6352. doi: 10.1523/JNEUROSCI.5613-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hikosaka O, Kim HF, Yasuda M, Yamamoto S. Basal ganglia circuits for reward value-guided behavior. Annual Review of Neuroscience. 2014;37:289–306. doi: 10.1146/annurev-neuro-071013-013924. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ikeda T, Hikosaka O. Reward-dependent gain and Bias of visual responses in primate superior colliculus. Neuron. 2003;39:693–700. doi: 10.1016/S0896-6273(03)00464-1. [DOI] [PubMed] [Google Scholar]
Kable JW, Glimcher PW. The neurobiology of decision: consensus and controversy. Neuron. 2009;63:733–745. doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neuroscience. 1998;1:411–416. doi: 10.1038/1625. [DOI] [PubMed] [Google Scholar]
Kim HF, Hikosaka O. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron. 2013;79:1001–1010. doi: 10.1016/j.neuron.2013.06.044. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kimchi EY, Laubach M. Dynamic encoding of action selection by the medial striatum. Journal of Neuroscience. 2009;29:3148–3159. doi: 10.1523/JNEUROSCI.5206-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kleiner M, Brainard D, Pelli D, Ingling A, Murray R, Broussard C. What's new in psychtoolbox-3. Perception. 2007;36:1–16. doi: 10.1068/v070821. [DOI] [Google Scholar]
Kobayashi S, Kawagoe R, Takikawa Y, Koizumi M, Sakagami M, Hikosaka O. Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward. Experimental Brain Research. 2007;176:341–355. doi: 10.1007/s00221-006-0622-4. [DOI] [PubMed] [Google Scholar]
Lau B, Glimcher PW. Value representations in the primate striatum during matching behavior. Neuron. 2008;58:451–463. doi: 10.1016/j.neuron.2008.02.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron. 2002a;33:463–473. doi: 10.1016/S0896-6273(02)00571-8. [DOI] [PubMed] [Google Scholar]
Lauwereyns J, Watanabe K, Coe B, Hikosaka O. A neural correlate of response Bias in monkey caudate nucleus. Nature. 2002b;418:413–417. doi: 10.1038/nature00892. [DOI] [PubMed] [Google Scholar]
Leite FP. A comparison of two diffusion process models in accounting for payoff and stimulus frequency manipulations. Attention, Perception, & Psychophysics. 2012;74:1366–1382. doi: 10.3758/s13414-012-0321-0. [DOI] [PubMed] [Google Scholar]
Liston DB, Stone LS. Effects of prior information and reward on oculomotor and perceptual choices. Journal of Neuroscience. 2008;28:13866–13875. doi: 10.1523/JNEUROSCI.3120-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lo CC, Wang XJ. Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nature Neuroscience. 2006;9:956–963. doi: 10.1038/nn1722. [DOI] [PubMed] [Google Scholar]
Maddox WT, Bohil CJ. Base-rate and payoff effects in multidimensional perceptual categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1998;24:1459–1482. doi: 10.1037/0278-7393.24.6.1459. [DOI] [PubMed] [Google Scholar]
Mink JW. The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology. 1996;50:381–425. doi: 10.1016/S0301-0082(96)00042-1. [DOI] [PubMed] [Google Scholar]
Mulder MJ, Wagenmakers EJ, Ratcliff R, Boekel W, Forstmann BU. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. Journal of Neuroscience. 2012;32:2335–2343. doi: 10.1523/JNEUROSCI.4156-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nakamura K, Ding L. Parsing heterogeneous striatal activity. Frontiers in Neuroanatomy. 2017;11:43. doi: 10.3389/fnana.2017.00043. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nakamura K, Hikosaka O. Facilitation of saccadic eye movements by postsaccadic electrical stimulation in the primate caudate. Journal of Neuroscience. 2006a;26:12885–12895. doi: 10.1523/JNEUROSCI.3688-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nakamura K, Hikosaka O. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. Journal of Neuroscience. 2006b;26:5360–5369. doi: 10.1523/JNEUROSCI.4853-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P. Scikit-Learn: Machine Learning in Python. Journal of Machine Learning Research. 2011;12:2825–2830. [Google Scholar]
Perugini A, Ditterich J, Basso MA. Patients with Parkinson's Disease Show Impaired Use of Priors in Conditions of Sensory Uncertainty. Current Biology. 2016;26:1902–1910. doi: 10.1016/j.cub.2016.05.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rao RP. Decision making under uncertainty: a neural model based on partially observable markov decision processes. Frontiers in Computational Neuroscience. 2010;4:146. doi: 10.3389/fncom.2010.00146. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ratcliff R, Frank MJ. Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Computation. 2012;24:1186–1229. doi: 10.1162/NECO_a_00270. [DOI] [PubMed] [Google Scholar]
Ratcliff R, Rouder JN. Modeling response times for Two-Choice decisions. Psychological Science. 1998;9:347–356. doi: 10.1111/1467-9280.00067. [DOI] [Google Scholar]
Reddi BA, Asrress KN, Carpenter RH. Accuracy, information, and response time in a saccadic decision task. Journal of Neurophysiology. 2003;90:3538–3546. doi: 10.1152/jn.00689.2002. [DOI] [PubMed] [Google Scholar]
Redgrave P, Prescott TJ, Gurney K. The basal ganglia: a vertebrate solution to the selection problem? Neuroscience. 1999;89:1009–1023. doi: 10.1016/S0306-4522(98)00319-4. [DOI] [PubMed] [Google Scholar]
Roesch MR, Olson CR. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. Journal of Neurophysiology. 2003;90:1766–1789. doi: 10.1152/jn.00019.2003. [DOI] [PubMed] [Google Scholar]
Rorie AE, Gao J, McClelland JL, Newsome WT. Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey. PLOS ONE. 2010;5:e9308. doi: 10.1371/journal.pone.0009308. [DOI] [PMC free article] [PubMed] [Google Scholar]
Samejima K, Ueda Y, Doya K, Kimura M. Representation of Action-Specific Reward Values in the Striatum. Science. 2005;310:1337–1340. doi: 10.1126/science.1115270. [DOI] [PubMed] [Google Scholar]
Samejima K, Doya K. Multiple representations of belief states and action values in corticobasal ganglia loops. Annals of the New York Academy of Sciences. 2007;1104:213–228. doi: 10.1196/annals.1390.024. [DOI] [PubMed] [Google Scholar]
Santacruz SR, Rich EL, Wallis JD, Carmena JM. Caudate microstimulation increases value of specific choices. Current Biology. 2017;27:3375–3383. doi: 10.1016/j.cub.2017.09.051. [DOI] [PMC free article] [PubMed] [Google Scholar]
Sato M, Hikosaka O. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. The Journal of Neuroscience. 2002;22:2363–2373. doi: 10.1523/JNEUROSCI.22-06-02363.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
Schall JD. Accumulators, neurons, and response time. Trends in Neurosciences. 2019;42:848–860. doi: 10.1016/j.tins.2019.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
Seo M, Lee E, Averbeck BB. Action selection and action value in frontal-striatal circuits. Neuron. 2012;74:947–960. doi: 10.1016/j.neuron.2012.03.037. [DOI] [PMC free article] [PubMed] [Google Scholar]
Summerfield C, Koechlin E. Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices. Frontiers in Human Neuroscience. 2010;4:208. doi: 10.3389/fnhum.2010.00208. [DOI] [PMC free article] [PubMed] [Google Scholar]
Summerfield C, Tsetsos K. Building bridges between perceptual and economic Decision-Making: neural and computational mechanisms. Frontiers in Neuroscience. 2012;6:70. doi: 10.3389/fnins.2012.00070. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tachibana Y, Hikosaka O. The primate ventral pallidum encodes expected reward value and regulates motor action. Neuron. 2012;76:826–837. doi: 10.1016/j.neuron.2012.09.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tai LH, Lee AM, Benavidez N, Bonci A, Wilbrecht L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nature Neuroscience. 2012;15:1281–1289. doi: 10.1038/nn.3188. [DOI] [PMC free article] [PubMed] [Google Scholar]
Teichert T, Ferrera VP. Suboptimal integration of reward magnitude and prior reward likelihood in categorical decisions by monkeys. Frontiers in Neuroscience. 2010;4:186. doi: 10.3389/fnins.2010.00186. [DOI] [PMC free article] [PubMed] [Google Scholar]
Voss A, Rothermund K, Voss J. Interpreting the parameters of the diffusion model: an empirical validation. Memory & Cognition. 2004;32:1206–1220. doi: 10.3758/BF03196893. [DOI] [PubMed] [Google Scholar]
Waiblinger C, Wu CM, Bolus MF, Borden PY, Stanley GB. Stimulus context and reward contingency induce behavioral adaptation in a rodent tactile detection task. The Journal of Neuroscience. 2019;39:1088–1099. doi: 10.1523/JNEUROSCI.2032-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wang AY, Miura K, Uchida N. The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nature Neuroscience. 2013;16:639–647. doi: 10.1038/nn.3377. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wang L, Rangarajan KV, Gerfen CR, Krauzlis RJ. Activation of striatal neurons causes a perceptual decision Bias during visual change detection in mice. Neuron. 2018;97:1369–1381. doi: 10.1016/j.neuron.2018.01.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
Watanabe M, Munoz DP. Saccade suppression by electrical microstimulation in monkey caudate nucleus. Journal of Neuroscience. 2010;30:2700–2709. doi: 10.1523/JNEUROSCI.5011-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wei W, Rubin JE, Wang XJ. Role of the indirect pathway of the basal ganglia in perceptual decision making. Journal of Neuroscience. 2015;35:4052–4064. doi: 10.1523/JNEUROSCI.3611-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
Whiteley L, Sahani M. Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes. Journal of Vision. 2008;8:2–15. doi: 10.1167/8.3.2. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wiecki TV, Sofer I, Frank MJ. HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Frontiers in Neuroinformatics. 2013;7:14. doi: 10.3389/fninf.2013.00014. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yanike M, Ferrera VP. Representation of outcome risk and action in the anterior caudate nucleus. Journal of Neuroscience. 2014;34:3279–3290. doi: 10.1523/JNEUROSCI.3818-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
Yartsev MM, Hanks TD, Yoon AM, Brody CD. Causal contribution and dynamical encoding in the striatum during evidence accumulation. eLife. 2018;7:e34929. doi: 10.7554/eLife.34929. [DOI] [PMC free article] [PubMed] [Google Scholar]
Zylberberg A, Fetsch CR, Shadlen MN. The influence of evidence volatility on choice, reaction time and confidence in a perceptual decision. eLife. 2016;5:e17688. doi: 10.7554/eLife.17688. [DOI] [PMC free article] [PubMed] [Google Scholar]

eLife. doi: 10.7554/eLife.56694.sa1

Decision letter

Editor: Daeyeol Lee¹

Reviewed by: Daeyeol Lee², Chandramouli Chandrasekaran³

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

The authors showed that heterogeneous activity of neurons in the caudate nucleus reflected all major aspects during perceptual decision with asymmetric rewards. They also found that the effect of electrical stimulation in the caudate nucleus was quantitatively related not only to the behavioral strategy of the animal but also to the functional properties of the neurons recorded in the stimulation site. These results establish the role of the caudate nucleus in mediating the reward bias during perceptual decision making.

Decision letter after peer review:

Thank you for submitting your article "The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Daeyeol Lee as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Michael Frank as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Chandramouli Chandrasekaran (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

As the editors have judged that your manuscript is of interest, but as described below that some additional analyses might be required before it is published, we would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). First, because many researchers have temporarily lost access to the labs, we will give authors as much time as they need to submit revised manuscripts. We are also offering, if you choose, to post the manuscript to bioRxiv (if it is not already there) along with this decision letter and a formal designation that the manuscript is 'in revision at eLife'. Please let us know if you would like to pursue this option. (If your work is more suitable for medRxiv, you will need to post the preprint yourself, as the mechanisms for us to do so are still in development.)

Summary:

This study examined the role of the caudate nucleus in combining uncertain sensory information with reward bias during decision making by analyzing the effects of electrical stimulation and neural activity within the frame of a drift-diffusion model. In their previous study, the authors have already characterized the biasing effect of unequal reward on perceptual decision making using two parameters corresponding to the change in the drift rate (me) and bound (z). There are at least two important findings in this study. First, they showed that heterogeneous activity of neurons in the caudate nucleus reflected all major aspects of the task, including coherence (i.e., signal-to-noise ratio), choice, and reward bias, and their interactions. Second, they found that the effect of electrical stimulation was quantitatively related not only to the behavioral strategy of the animal but also to the functional properties of the neurons recorded in the stimulation site. While these results seem to firmly establish the role of the caudate nucleus in mediating the reward bias during perceptual decision making, there is a room for improvement in the analyses and presentation. The authors should improve their exposition in the modeling section and to simplify their results where possible to make it easier on the reader. A large number of parameters introduced in the paper makes it difficult to keep track of everything.

Essential revisions:

1) The section that describes the relationship between the effect of microstimulation and the animal's strategy (subsection “Microstimulation caused coordinated adjustments to reward-dependent decision biases that mimicked the monkeys’ voluntary strategy”) was very hard to follow and is not entirely convincing. This section relies heavily on the findings from the previous study by the authors that the negative correlation between the changes in the drift and bound reflected the animal's heuristic strategy for reward maximization. The results in that study also suggest that such adjustment occurred slowly across many sessions. Was it the case then that the effect of stimulation changed similarly during the same period? If so, the observed relationship between the animal's strategy and stimulation effect might be spurious and caused by a slow cross-session drift (any slow changes in the animal's strategy not captured by the model).

2) Microstimulation result in Figure 3 is complex to parse. There are 12 distributions with 6 panels of scatter plots. Might it make sense to perhaps obtain a figure that demonstrates the effect size for rew x estim. Appropriately normalized regressions might allow such a plot. That way, one could understand the magnitude of the effect.

3) The modeling in Figures 4 and 5 are nice. The coordinated additive and scaling hypothesis assume that the monkeys change bound height and drift rate across sessions. Microstimulation has a coordinated effect on Δbound and Δdrift. The data in Figure 5 is largely consistent with the Figure 4E and F. However, the results are not entirely convincing, in part because they entirely depend on the contrasts in the model parameters for reward context dependent and independent components. However, if the underlying parameters (e.g., Para_contra-LR,_estim, etc.) are correlated, the results might need a different interpretation. It might be possible to address this, using PCA. For example, if the authors run PCA on the parameters used to quantify the effect of reward without stimulation (e.g., Para_{contro-LR,no estim}) and PAC on parameters used to quantify ΔPara(estim) and ΔPara(rew x estim), do they find significant correlation for more than one PC?

There are also some discrepancies between the effect of microstimulation presented in Figure 4E, F. First microstimulation leads to a shift diagonally upwards from the line in Figure 5D which is inconsistent with Figure 4C and F. Whereas the coordinated additive and coordinated scaling push it downward. This needs to be better explained to ameliorate concerns for the reader. For example, placing the scatter plots from Figure 4C-F middle and right columns and placing them in Figure 5 might be helpful. The colormap should also be the same between Figures 4 and 5 so that one can compare both of these reliably. They could sacrifice some of the logistic vs. DDM plots to the supplements. There are several additional concerns.

1) Model predicts more lower Δdrift and lower Δbound for the coordinated scaling case, but the data don't seem to show that. Is this important?

2) What happens when you change the size of the noise for Figure 4D, does it show a less steeper change in Figure 4D right panel?

3) Quantitative evidence brought to bear to favor the coordinated scaling over the independent model in Figure 4D right panel would make it more compelling. Currently, one can look at Figure 4F and Figure 5F and see they are similar.

4) The fact that correlation coefficient (slope) between changes in drift and bound (Figure 6E) tends to be negative for Shuffles 3 and 4 is a concern, because this raises the possibility that the observed relationship between the two measures might be also an artifact. Can this be addressed using a simpler approach of shuffling the parameters fit for different stimulation and reward context (e.g., Para_{contra-LR,estim}), rather than simulating individual trials and re-estimating the model parameters?

5) There are some problems in how neural activity was analyzed using a regression model. For example, the activity during a variable window might be confounded with reaction time (e.g., Epoch 5; subsection “Caudate neurons encode both visual and reward information”, Figure 2—figure supplement 2), which could lead to mis-identification of signals related to coherence. Also, the statement that a majority of the neurons showed at least one of many effects is not particularly meaningful, since the proportion of such neurons is expected to increase with the number of tests when they were not corrected for multiple comparisons. Similarly, the effects of microstimulation on multiple types of behavioral data (subsection “Caudate microstimulation evoked reward context-dependent effects on behavior”) should be reported more carefully.

6) Some of the results reported in this study might require more careful interpretation. For example, the fact that microstimulation effected varied with the reward context by itself might not provide strong evidence that the caudate nucleus is causally involved in balancing visual evidence and reward bias, because such interaction might occur when the magnitude of microstimulation varies with the reaction time (which is in turn influenced by reward context).

eLife. 2020 Jun 22;9:e56694. doi: 10.7554/eLife.56694.sa2

Author response

Essential revisions:

1) The section that describes the relationship between the effect of microstimulation and the animal's strategy (subsection “Microstimulation caused coordinated adjustments to reward-dependent decision biases that mimicked the monkeys’ voluntary strategy”) was very hard to follow and is not entirely convincing. This section relies heavily on the findings from the previous study by the authors that the negative correlation between the changes in the drift and bound reflected the animal's heuristic strategy for reward maximization. The results in that study also suggest that such adjustment occurred slowly across many sessions. Was it the case then that the effect of stimulation changed similarly during the same period? If so, the observed relationship between the animal's strategy and stimulation effect might be spurious and caused by a slow cross-session drift (any slow changes in the animal's strategy not captured by the model).

The reviewers are correct that our previous study showed that the negative correlation between the reward context-dependent adjustments in drift rates and bounds reflected the animal’s heuristic strategy for achieving good-enough reward outcomes. However, as we showed in that paper, the monkeys’ adjustments reflected the reward functions for the two types of blocks in a given session, arguing for fast adaptation that occurred within a block of trials, instead of slow adaptation across many sessions. As shown in the Author response image 1, the monkeys’ adjustments on no-estim trials varied from session to session, but with no clear trend across sessions. We thus do not believe that the alternative explanation raised by the reviewers could explain our estim data. We have clarified this point in the main text, including the following: "The monkeys tended to use positive ∆drift (rew) values that changed in magnitude according to session-by-session variations in the reward function (see Figure 6 in (Fan et al., 2018)), implying fast adaptation within a session."

Author response image 1. — Data were from trials without microstimulation in microstimulation sessions.

2) Microstimulation result in Figure 3 is complex to parse. There are 12 distributions with 6 panels of scatter plots. Might it make sense to perhaps obtain a figure that demonstrates the effect size for rew x estim. Appropriately normalized regressions might allow such a plot. That way, one could understand the magnitude of the effect.

We apologize for the dense figure. We now present the summary histograms and cartoon illustrations of the effects of these parameters in the main figure and present scatterplots of individual sessions in Figure 3—figure supplement 1. We hope these revisions make the admittedly complex results more accessible to the readers.

3) The modeling in Figures 4 and 5 are nice. The coordinated additive and scaling hypothesis assume that the monkeys change bound height and drift rate across sessions. Microstimulation has a coordinated effect on Δbound and Δdrift. The data in Figure 5 is largely consistent with the Figure 4E and F. However, the results are not entirely convincing, in part because they entirely depend on the contrasts in the model parameters for reward context dependent and independent components. However, if the underlying parameters (e.g., Para_{contra-LR,estim}, etc.) are correlated, the results might need a different interpretation. It might be possible to address this, using PCA. For example, if the authors run PCA on the parameters used to quantify the effect of reward without stimulation (e.g., Para_{contro-LR,no estim}) and PAC on parameters used to quantify ΔPara(estim) and ΔPara(rew x estim), do they find significant correlation for more than one PC?

The reviewers raised a valid concern about whether the coordination we observed in Δdrift (rew × estim) and Δbound (rew × estim) represented the primary feature in the experimental data, or secondary to other correlation patterns. We thank the reviewers for suggesting the PCA-based analyses to address this concern. We performed PCA on the eight raw fitted parameters ([me or z] × [contra-LR or ipsi-LR] × [with estim or no estim]). We then examined the relationship between the first PC from this analysis

(“PC1 (8 para)”) and the coordination of Δdrift (rew × estim) and Δbound (rew × estim), the latter being quantified as the first PC (“PC1 (coord)”) in the two-parameter space. We projected the fitted parameters from the experimental data onto the PC1 (8para) and PC1 (coord), respectively, and found a strong correlation between the projection values (rho = 0.92, p = 1.3e-16). These results strongly suggest that the coordinated effects were the primary feature of the experimental data; i.e., the dominant effects of caudate microstimulation. We now include these results in Figure 6—figure supplement 3 and text:

“Fourth, given the inter-session variability in microstimulation effects, it is possible that the negatively correlated ∆drift (rew × estim) and ∆bound (rew × estim) reflected only a minor consequence of caudate microstimulation. […] The projections showed a strong correlation (Figure 6—figure supplement 3; rho=0.92, p=1.3e-16), indicating that the coordinated ∆drift (rew × estim) and ∆bound (rew × estim) values reflected the dominant feature of the experimental data; i.e., the dominant effect of caudate microstimulation on reward-related biases.”

There are also some discrepancies between the effect of microstimulation presented in Figure 4E, F. First microstimulation leads to a shift diagonally upwards from the line in Figure 5D which is inconsistent with Figure 4C and F. Whereas the coordinated additive and coordinated scaling push it downward. This needs to be better explained to ameliorate concerns for the reader. For example, placing the scatter plots from Figure 4C-F middle and right columns and placing them in Figure 5 might be helpful. The colormap should also be the same between Figures 4 and 5 so that one can compare both of these reliably. They could sacrifice some of the logistic vs. DDM plots to the supplements. There are several additional concerns.

1) Model predicts more lower Δdrift and lower Δbound for the coordinated scaling case, but the data don't seem to show that. Is this important?

2) What happens when you change the size of the noise for Figure 4D, does it show a less steeper change in Figure 4D right panel?

3) Quantitative evidence brought to bear to favor the coordinated scaling over the independent model in Figure 4D right panel would make it more compelling. Currently, one can look at Figure 4F and Figure 5F and see they are similar.

We again apologize for the confusion. Figure 4 aimed to illustrate the idea that coordinated changes would predict correlated rew x estim terms, whereas independent changes would not. The parameters used for these illustrations were arbitrarily picked just for illustration purposes and not fitted to the experimental data. This is why the simulated data points in Figure 4F were still somewhat mismatched from the experimental data in Figure 5.

We understand the reviewers’ concern and have generated new illustrations with parameters that were based on the experimental data, with comparable magnitudes of both stimulation effects and noise. As expected, the newly simulated data in Figure 4F much more closely match the experimental data (e.g., the fitted line in Figure 4F is within the 95% confidence intervals of the fitted line in Figure 5D), without the concerning features that were noted in the concerns (1) and (2).

For the concern (3), we believe that we have already provided this quantitative evidence: the “Shuffle 2” test in Figure 6E, which shows that, even with the mean and variance values matched exactly to the experimental data, the model in Figure 4D cannot capture the correlated pattern we observed in the experimental data. We now indicate clearly in the text that this shuffle rules out the model in Figure 4D. In contrast, the correlation is built-in for the model in Figure 4F.

They could sacrifice some of the logistic vs. DDM plots to the supplements.

We followed the reviewers’ suggestion and moved these plots to the supplements. To further simply the figures, we have removed the panels related to “rew effects with estim” in Figures 4 and 5, which are not essential, nor informative, for understanding the stimulation effects.

4) The fact that correlation coefficient (slope) between changes in drift and bound (Figure 6E) tends to be negative for Shuffles 3 and 4 is a concern, because this raises the possibility that the observed relationship between the two measures might be also an artifact. Can this be addressed using a simpler approach of shuffling the parameters fit for different stimulation and reward context (e.g., Para_{contra-LR,estim}), rather than simulating individual trials and re-estimating the model parameters?

We apologize for the confusion. Shuffles 3 and 4 represent partial shuffles that maintained different features of the experimental data and thus were expected to maintain some relationship between the two parameters. Even so, the resulting correlations were significantly weaker than in the experimental data.

The full shuffling of individual parameters, as the reviewers suggested, resulted in a distribution that is centered at zero and similar to the “Shuffle 2” data in Figure 6E (when we fully shuffled Δdrift (rew) and Δbound (rew) for both stimulation conditions). To highlight the importance of shuffle 2 for our interpretation, we changed the color codes for the three types of shuffles in Figure 6E. We also revised the text to make these points more clearly:

“Second, the coordinated effects of microstimulation on ∆drift and ∆bound might have been a trivial consequence of reward-dependent biases, independent of the reward context-dependent coordination of the two quantities. […] That is, even with the mean and variance values matched exactly to the experimental data, the model in Figure 4D cannot capture the correlated pattern we observed in the experimental data; and 3) partial shuffling that disrupted only possible relationships with session-specific properties (e.g., microstimulation sites or voluntary performance), by shuffling the paired ∆drift and ∆bound across sessions, also significantly weakened the correlation between “rew × estim” effects (Figure 6E, “Shuffle 4”).”

5) There are some problems in how neural activity was analyzed using a regression model. For example, the activity during a variable window might be confounded with reaction time (e.g., Epoch 5; subsection “Caudate neurons encode both visual and reward information”, Figure 2—figure supplement 2), which could lead to mis-identification of signals related to coherence.

We agree with the reviewers about this concern. This was why, in addition to the variable window (epoch 5), we also analyzed activity in a fixed-duration window (epoch 4 in Figure 2C-F). The results from the two epochs are qualitatively similar, with epoch 5 showing a larger fraction of neurons. These results suggest that the coherence modulation was not simply a result of a dependence on reaction time. We now comment on this observation: “The selectivity for reward size and motion strength was more prevalent in Epoch #5 (variable duration covering the whole motion viewing period) than in Epoch #4 (fixed duration covering only the early motion viewing period), consistent with a developing latent decision variable that accumulates evidence over time, increasing sensitivity of the regression analysis with longer analysis windows, and/or additional sensitivity to RT closer to the time of saccade.”

We show the heatmap of significant coefficients for Epoch 5 in Figure 2—figure supplement 2 to illustrate the diversity of neural modulation. To make this point explicit, we revised the text: “These neurons also showed heterogenous modulation patterns, as illustrated for the 44 neurons with joint modulation during motion viewing, with heterogeneous modulation patterns (Figure 2—figure supplement 2).”

We agree with the reviewers that more comprehensive description of the neural data, including a direct examination of RT selectivity, is valuable. To that end, we are preparing a separate manuscript with more in-depth analysis of the caudate data, which will include time-dependent regression results, using normalized RT as a regressor, for activity aligned to motion and saccade onset, respectively. However, we agree with the reviewers that the present manuscript needs to be simplified, not expanded, and thus prefer to leave these additional analyses to the other manuscript.

Also, the statement that a majority of the neurons showed at least one of many effects is not particularly meaningful, since the proportion of such neurons is expected to increase with the number of tests when they were not corrected for multiple comparisons.

We assume that this statement referred to “Overall, a majority of neurons (101/142) showed at least one of these forms of joint modulation in at least one epoch (Figure 2F)”. We now support this statement with a statistical test based on the comparison of the observed fraction and a chance level that considers the multiple comparisons. For example, the chance levels are 0.0975 for observing at least one modulation by coh x reward out of two terms for contra and ipsi, respectively, and 0.0095 for observing modulations by both coh and reward (“Coh + Reward”). The chancel level for the fraction in Figure 2F is thus 1 – (1-0.0975) * (1-0.0095) = 0.1061, for a single epoch. The overall chance level for observing at least one epoch with joint modulation, out of 7 epochs, is thus 0.5438. The number count we reported (101 out of 142) was significantly above this chance level (chi-square test, p = 0.0035). We revised the text as follows: “Overall, 101 out of 142 neurons showed at least one of these forms of joint modulation in at least one epoch. This fraction was significantly above chance level, even considering the multiple tests done in 7 epochs (Figure 2F; Chi-squared test, p=0.0035).” We also added to the figure legend that the chance level was “adjusted based on the number of tests used and a 5% chance level for a single test”.

Similarly, the effects of microstimulation on multiple types of behavioral data (subsection “Caudate microstimulation evoked reward context-dependent effects on behavior”) should be reported more carefully.

Following similar logic, we revised the text: “Overall, microstimulation induced at least one statistically reliable effect on choice or RT at 48 of 55 caudate sites (colored dots in Figure 3—figure supplement 1B; significantly above the chance level that was adjusted for multiple comparisons, Chi-squared test, p<1e-5).”

6) Some of the results reported in this study might require more careful interpretation. For example, the fact that microstimulation effected varied with the reward context by itself might not provide strong evidence that the caudate nucleus is causally involved in balancing visual evidence and reward bias, because such interaction might occur when the magnitude of microstimulation varies with the reaction time (which is in turn influenced by reward context).

We agree with the reviewers that, because we used a RT task with contextual modulation of RT and perturbed the caudate nucleus during the whole motion-viewing period, it is possible that the contextual difference in perturbation effects might be due to contextual differences in RT. We examined this alternative explanation and did not find much support in our data. We added Figure 3—figure supplement 2 and stated in the text:

“Because we delivered microstimulation throughout motion viewing for an RT task, the difference in average RT between reward contexts may contribute to “rew × estim”-type microstimulation effects. […] Thus, the dependence of the microstimulation effects on reward context more likely reflected a causal involvement of the caudate nucleus in balancing visual evidence and reward asymmetry information.”

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Figure 2—source data 1. Regression results for Figure 2C–F, using Equation 1.

elife-56694-fig2-data1.csv^{(237.6KB, csv)}

Figure 3—source data 1. Fitting results for choice (logistic) and RT (linear) data.

The same data were used to plot Figure 3—figure supplement 1 and generate summary statistics in Supplementary file 1a.

elife-56694-fig1.csv^{(29.6KB, csv)}

Figure 5—source data 1. Fitting results for choice and RT data using the DDM for sessions with significant microstimulation effects.

The same data were used for plots in Figure 5—figure supplement 2, Figure 5—figure supplement 4, Figure 6A and E, Figure 6—figure supplement 3, and for the summary statistics in Supplementary file 1b.

elife-56694-fig5-data1.csv^{(15.3KB, csv)}

Figure 7—source data 1. Neural selectivity for different task factors at the microstimulation sites.

Column “indRewcontCoh” was used to divide the two subsets of stimulation sites in C and D. PCA was performed on DDM fitting results in Figure 5—source data 1.

elife-56694-fig7-data1.csv^{(3.9KB, csv)}

elife-56694-supp1.docx^{(16KB, docx)}

Transparent reporting form

elife-56694-transrepform.docx^{(246.3KB, docx)}

Data Availability Statement

[bib1] Amemori K, Amemori S, Gibson DJ, Graybiel AM. Striatal microstimulation induces persistent and repetitive negative Decision-Making predicted by striatal Beta-Band oscillation. Neuron. 2018;99:826–841. doi: 10.1016/j.neuron.2018.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib2] Berns GS, Sejnowski TJ. How the basal ganglia make decisions. In: Damasio AR, Damasio H, editors. Neurobiology of Decision-Making. Berlin, Springer-Verlag; 1995. pp. 101–113. [DOI] [Google Scholar]

[bib3] Bogacz R, Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Computation. 2007;19:442–477. doi: 10.1162/neco.2007.19.2.442. [DOI] [PubMed] [Google Scholar]

[bib4] Brooks R. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation. 1986;2:14–23. doi: 10.1109/JRA.1986.1087032. [DOI] [Google Scholar]

[bib5] Cai X, Kim S, Lee D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron. 2011;69:170–182. doi: 10.1016/j.neuron.2010.11.041. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib6] Carpenter RH, Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature. 1995;377:59–62. doi: 10.1038/377059a0. [DOI] [PubMed] [Google Scholar]

[bib7] Cavanagh JF, Wiecki TV, Cohen MX, Figueroa CM, Samanta J, Sherman SJ, Frank MJ. Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nature Neuroscience. 2011;14:1462–1467. doi: 10.1038/nn.2925. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib8] Churchland AK, Kiani R, Shadlen MN. Decision-making with multiple alternatives. Nature Neuroscience. 2008;11:693–702. doi: 10.1038/nn.2123. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib9] Cicmil N, Cumming BG, Parker AJ, Krug K. Reward modulates the effect of visual cortical microstimulation on perceptual decisions. eLife. 2015;4:e07832. doi: 10.7554/eLife.07832. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib10] Coe B, Tomihara K, Matsuzawa M, Hikosaka O. Visual and anticipatory Bias in three cortical eye fields of the monkey during an adaptive decision-making task. The Journal of Neuroscience. 2002;22:5081–5090. doi: 10.1523/JNEUROSCI.22-12-05081.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib11] Diederich A, Busemeyer JR. Modeling the effects of payoff on response Bias in a perceptual discrimination task: bound-change, drift-rate-change, or two-stage-processing hypothesis. Perception & Psychophysics. 2006;68:194–207. doi: 10.3758/BF03193669. [DOI] [PubMed] [Google Scholar]

[bib12] Ding L, Gold JI. Caudate encodes multiple computations for perceptual decisions. Journal of Neuroscience. 2010;30:15747–15759. doi: 10.1523/JNEUROSCI.2894-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib13] Ding L, Gold JI. Neural correlates of perceptual decision making before, during, and after decision commitment in monkey frontal eye field. Cerebral Cortex. 2012a;22:1052–1067. doi: 10.1093/cercor/bhr178. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib14] Ding L, Gold JI. Separate, causal roles of the caudate in Saccadic choice and execution in a perceptual decision task. Neuron. 2012b;75:865–874. doi: 10.1016/j.neuron.2012.07.021. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib15] Ding L, Gold JI. The basal ganglia's contributions to perceptual decision making. Neuron. 2013;79:640–649. doi: 10.1016/j.neuron.2013.07.042. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib16] Ding L, Hikosaka O. Comparison of Reward Modulation in the Frontal Eye Field and Caudate of the Macaque. Journal of Neuroscience. 2006;26:6695–6703. doi: 10.1523/JNEUROSCI.0836-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib17] Drugowitsch J, Moreno-Bote R, Churchland AK, Shadlen MN, Pouget A. The cost of accumulating evidence in perceptual decision making. Journal of Neuroscience. 2012;32:3612–3628. doi: 10.1523/JNEUROSCI.4010-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib18] Fan Y, Gold JI, Ding L. Ongoing, rational calibration of reward-driven perceptual biases. eLife. 2018;7:e36018. doi: 10.7554/eLife.36018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib19] Feng S, Holmes P, Rorie A, Newsome WT. Can monkeys choose optimally when faced with noisy stimuli and unequal rewards? PLOS Computational Biology. 2009;5:e1000284. doi: 10.1371/journal.pcbi.1000284. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib20] Gao J, Tortell R, McClelland JL. Dynamic integration of reward and stimulus information in perceptual decision-making. PLOS ONE. 2011;6:e16749. doi: 10.1371/journal.pone.0016749. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib21] Graziano MS, Taylor CS, Moore T. Complex movements evoked by microstimulation of precentral cortex. Neuron. 2002;34:841–851. doi: 10.1016/S0896-6273(02)00698-0. [DOI] [PubMed] [Google Scholar]

[bib22] Hanks TD, Mazurek ME, Kiani R, Hopp E, Shadlen MN. Elapsed decision time affects the weighting of prior probability in a perceptual decision task. Journal of Neuroscience. 2011;31:6339–6352. doi: 10.1523/JNEUROSCI.5613-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib23] Hikosaka O, Kim HF, Yasuda M, Yamamoto S. Basal ganglia circuits for reward value-guided behavior. Annual Review of Neuroscience. 2014;37:289–306. doi: 10.1146/annurev-neuro-071013-013924. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib24] Ikeda T, Hikosaka O. Reward-dependent gain and Bias of visual responses in primate superior colliculus. Neuron. 2003;39:693–700. doi: 10.1016/S0896-6273(03)00464-1. [DOI] [PubMed] [Google Scholar]

[bib25] Kable JW, Glimcher PW. The neurobiology of decision: consensus and controversy. Neuron. 2009;63:733–745. doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib26] Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neuroscience. 1998;1:411–416. doi: 10.1038/1625. [DOI] [PubMed] [Google Scholar]

[bib27] Kim HF, Hikosaka O. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron. 2013;79:1001–1010. doi: 10.1016/j.neuron.2013.06.044. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib28] Kimchi EY, Laubach M. Dynamic encoding of action selection by the medial striatum. Journal of Neuroscience. 2009;29:3148–3159. doi: 10.1523/JNEUROSCI.5206-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib29] Kleiner M, Brainard D, Pelli D, Ingling A, Murray R, Broussard C. What's new in psychtoolbox-3. Perception. 2007;36:1–16. doi: 10.1068/v070821. [DOI] [Google Scholar]

[bib30] Kobayashi S, Kawagoe R, Takikawa Y, Koizumi M, Sakagami M, Hikosaka O. Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward. Experimental Brain Research. 2007;176:341–355. doi: 10.1007/s00221-006-0622-4. [DOI] [PubMed] [Google Scholar]

[bib31] Lau B, Glimcher PW. Value representations in the primate striatum during matching behavior. Neuron. 2008;58:451–463. doi: 10.1016/j.neuron.2008.02.021. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib32] Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron. 2002a;33:463–473. doi: 10.1016/S0896-6273(02)00571-8. [DOI] [PubMed] [Google Scholar]

[bib33] Lauwereyns J, Watanabe K, Coe B, Hikosaka O. A neural correlate of response Bias in monkey caudate nucleus. Nature. 2002b;418:413–417. doi: 10.1038/nature00892. [DOI] [PubMed] [Google Scholar]

[bib34] Leite FP. A comparison of two diffusion process models in accounting for payoff and stimulus frequency manipulations. Attention, Perception, & Psychophysics. 2012;74:1366–1382. doi: 10.3758/s13414-012-0321-0. [DOI] [PubMed] [Google Scholar]

[bib35] Liston DB, Stone LS. Effects of prior information and reward on oculomotor and perceptual choices. Journal of Neuroscience. 2008;28:13866–13875. doi: 10.1523/JNEUROSCI.3120-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib36] Lo CC, Wang XJ. Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nature Neuroscience. 2006;9:956–963. doi: 10.1038/nn1722. [DOI] [PubMed] [Google Scholar]

[bib37] Maddox WT, Bohil CJ. Base-rate and payoff effects in multidimensional perceptual categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1998;24:1459–1482. doi: 10.1037/0278-7393.24.6.1459. [DOI] [PubMed] [Google Scholar]

[bib38] Mink JW. The basal ganglia: focused selection and inhibition of competing motor programs. Progress in Neurobiology. 1996;50:381–425. doi: 10.1016/S0301-0082(96)00042-1. [DOI] [PubMed] [Google Scholar]

[bib39] Mulder MJ, Wagenmakers EJ, Ratcliff R, Boekel W, Forstmann BU. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. Journal of Neuroscience. 2012;32:2335–2343. doi: 10.1523/JNEUROSCI.4156-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib40] Nakamura K, Ding L. Parsing heterogeneous striatal activity. Frontiers in Neuroanatomy. 2017;11:43. doi: 10.3389/fnana.2017.00043. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib41] Nakamura K, Hikosaka O. Facilitation of saccadic eye movements by postsaccadic electrical stimulation in the primate caudate. Journal of Neuroscience. 2006a;26:12885–12895. doi: 10.1523/JNEUROSCI.3688-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib42] Nakamura K, Hikosaka O. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. Journal of Neuroscience. 2006b;26:5360–5369. doi: 10.1523/JNEUROSCI.4853-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib43] Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P. Scikit-Learn: Machine Learning in Python. Journal of Machine Learning Research. 2011;12:2825–2830. [Google Scholar]

[bib44] Perugini A, Ditterich J, Basso MA. Patients with Parkinson's Disease Show Impaired Use of Priors in Conditions of Sensory Uncertainty. Current Biology. 2016;26:1902–1910. doi: 10.1016/j.cub.2016.05.039. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib45] Rao RP. Decision making under uncertainty: a neural model based on partially observable markov decision processes. Frontiers in Computational Neuroscience. 2010;4:146. doi: 10.3389/fncom.2010.00146. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib46] Ratcliff R, Frank MJ. Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Computation. 2012;24:1186–1229. doi: 10.1162/NECO_a_00270. [DOI] [PubMed] [Google Scholar]

[bib47] Ratcliff R, Rouder JN. Modeling response times for Two-Choice decisions. Psychological Science. 1998;9:347–356. doi: 10.1111/1467-9280.00067. [DOI] [Google Scholar]

[bib48] Reddi BA, Asrress KN, Carpenter RH. Accuracy, information, and response time in a saccadic decision task. Journal of Neurophysiology. 2003;90:3538–3546. doi: 10.1152/jn.00689.2002. [DOI] [PubMed] [Google Scholar]

[bib49] Redgrave P, Prescott TJ, Gurney K. The basal ganglia: a vertebrate solution to the selection problem? Neuroscience. 1999;89:1009–1023. doi: 10.1016/S0306-4522(98)00319-4. [DOI] [PubMed] [Google Scholar]

[bib50] Roesch MR, Olson CR. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. Journal of Neurophysiology. 2003;90:1766–1789. doi: 10.1152/jn.00019.2003. [DOI] [PubMed] [Google Scholar]

[bib51] Rorie AE, Gao J, McClelland JL, Newsome WT. Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey. PLOS ONE. 2010;5:e9308. doi: 10.1371/journal.pone.0009308. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib52] Samejima K, Ueda Y, Doya K, Kimura M. Representation of Action-Specific Reward Values in the Striatum. Science. 2005;310:1337–1340. doi: 10.1126/science.1115270. [DOI] [PubMed] [Google Scholar]

[bib53] Samejima K, Doya K. Multiple representations of belief states and action values in corticobasal ganglia loops. Annals of the New York Academy of Sciences. 2007;1104:213–228. doi: 10.1196/annals.1390.024. [DOI] [PubMed] [Google Scholar]

[bib54] Santacruz SR, Rich EL, Wallis JD, Carmena JM. Caudate microstimulation increases value of specific choices. Current Biology. 2017;27:3375–3383. doi: 10.1016/j.cub.2017.09.051. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib55] Sato M, Hikosaka O. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. The Journal of Neuroscience. 2002;22:2363–2373. doi: 10.1523/JNEUROSCI.22-06-02363.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib56] Schall JD. Accumulators, neurons, and response time. Trends in Neurosciences. 2019;42:848–860. doi: 10.1016/j.tins.2019.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib57] Seo M, Lee E, Averbeck BB. Action selection and action value in frontal-striatal circuits. Neuron. 2012;74:947–960. doi: 10.1016/j.neuron.2012.03.037. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib58] Summerfield C, Koechlin E. Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices. Frontiers in Human Neuroscience. 2010;4:208. doi: 10.3389/fnhum.2010.00208. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib59] Summerfield C, Tsetsos K. Building bridges between perceptual and economic Decision-Making: neural and computational mechanisms. Frontiers in Neuroscience. 2012;6:70. doi: 10.3389/fnins.2012.00070. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib60] Tachibana Y, Hikosaka O. The primate ventral pallidum encodes expected reward value and regulates motor action. Neuron. 2012;76:826–837. doi: 10.1016/j.neuron.2012.09.030. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib61] Tai LH, Lee AM, Benavidez N, Bonci A, Wilbrecht L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nature Neuroscience. 2012;15:1281–1289. doi: 10.1038/nn.3188. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib62] Teichert T, Ferrera VP. Suboptimal integration of reward magnitude and prior reward likelihood in categorical decisions by monkeys. Frontiers in Neuroscience. 2010;4:186. doi: 10.3389/fnins.2010.00186. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib63] Voss A, Rothermund K, Voss J. Interpreting the parameters of the diffusion model: an empirical validation. Memory & Cognition. 2004;32:1206–1220. doi: 10.3758/BF03196893. [DOI] [PubMed] [Google Scholar]

[bib64] Waiblinger C, Wu CM, Bolus MF, Borden PY, Stanley GB. Stimulus context and reward contingency induce behavioral adaptation in a rodent tactile detection task. The Journal of Neuroscience. 2019;39:1088–1099. doi: 10.1523/JNEUROSCI.2032-18.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib65] Wang AY, Miura K, Uchida N. The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nature Neuroscience. 2013;16:639–647. doi: 10.1038/nn.3377. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib66] Wang L, Rangarajan KV, Gerfen CR, Krauzlis RJ. Activation of striatal neurons causes a perceptual decision Bias during visual change detection in mice. Neuron. 2018;97:1369–1381. doi: 10.1016/j.neuron.2018.01.049. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib67] Watanabe M, Munoz DP. Saccade suppression by electrical microstimulation in monkey caudate nucleus. Journal of Neuroscience. 2010;30:2700–2709. doi: 10.1523/JNEUROSCI.5011-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib68] Wei W, Rubin JE, Wang XJ. Role of the indirect pathway of the basal ganglia in perceptual decision making. Journal of Neuroscience. 2015;35:4052–4064. doi: 10.1523/JNEUROSCI.3611-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib69] Whiteley L, Sahani M. Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes. Journal of Vision. 2008;8:2–15. doi: 10.1167/8.3.2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib70] Wiecki TV, Sofer I, Frank MJ. HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Frontiers in Neuroinformatics. 2013;7:14. doi: 10.3389/fninf.2013.00014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib71] Yanike M, Ferrera VP. Representation of outcome risk and action in the anterior caudate nucleus. Journal of Neuroscience. 2014;34:3279–3290. doi: 10.1523/JNEUROSCI.3818-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib72] Yartsev MM, Hanks TD, Yoon AM, Brody CD. Causal contribution and dynamical encoding in the striatum during evidence accumulation. eLife. 2018;7:e34929. doi: 10.7554/eLife.34929. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib73] Zylberberg A, Fetsch CR, Shadlen MN. The influence of evidence volatility on choice, reaction time and confidence in a perceptual decision. eLife. 2016;5:e17688. doi: 10.7554/eLife.17688. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Takahiro Doi

Yunshu Fan

Joshua I Gold

Long Ding

Roles

Abstract

Introduction

Figure 1. Monkeys biased toward choices associated with large reward.

Results

Caudate neurons encode both visual and reward information

Figure 2. Caudate activity reflected motion strength, reward context, choice, and the expected reward size associated with the choice.

Figure 2—figure supplement 1. Example neurons with different kinds of task-relevant modulations.

Figure 2—figure supplement 2. Modulation patterns of “combination neurons” during motion viewing.

Caudate microstimulation evoked reward context-dependent effects on behavior

Figure 3. Caudate microstimulation affected the monkeys’ decision behavior.

Figure 3—figure supplement 1. Effects of caudate microstimulation for all sessions.

Figure 3—figure supplement 2. The average RT difference between the two reward contexts alone could not account for the reward context-dependent microstimulation effects.

Microstimulation caused coordinated adjustments to reward-dependent decision biases that mimicked the monkeys’ voluntary strategy

Figure 4. Illustration of correlation patterns induced by different hypothesized actions of microstimulation.

Figure 5. Microstimulation induced correlated changes in the reward modulation of drift and bound.

Figure 5—figure supplement 1. DDM fits to example sessions in Figure 3.

Figure 5—figure supplement 2. DDM fitting results.

Figure 5—figure supplement 3. Biases in drift and bounds together accounted for biases measured in logistic fits.

Figure 5—figure supplement 4. Both monkeys showed similar correlation patterns between ∆drift and ∆bound.

Figure 6. Control analysis results.

Figure 6—figure supplement 1. ∆drift and ∆bound values obtained by fitting the simulated data to the “NoCollapse” DDM model did not show the same correlation patterns as the original data.

Figure 6—figure supplement 2. Both monkeys showed significant negative correlation between ∆drift (rew) and ∆bound (rew) on trials before microstimulation began.

Figure 6—figure supplement 3. The correlated ∆drift (rew × estim) and ∆bound (rew × estim) effects represented the dominant features of the data.

Coordinated microstimulation effects depended on neural selectivity patterns and baseline adjustments

Figure 7. Microstimulation effects on coordination depended on neural selectivity and baseline adjustments of drift and bound.

Figure 7—figure supplement 1. MRI reconstruction of recording and microstimulation sites.

Discussion

Materials and methods

Key resources table.

Experimental model and subject details

Method details

Behavioral task

Data acquisition

Quantification and statistical analysis

Neural data analysis

Behavioral analysis

Measuring microstimulation effects on choice using a logistic function

Measuring microstimulation effects on RT using a linear function

Measuring microstimulation effects on both choice and RT using the drift-diffusion model (DDM)

Generating predictions for hypothesized reward context-microstimulation interaction effects on drift rate and bound height

Δdrift (rew × estim) and Δbound (rew × estim)

Relating microstimulation effects to neural selectivity and the monkeys’ voluntary strategy

Acknowledgements

Funding Statement

Contributor Information

Funding Information

Additional information

Competing interests

Author contributions

Ethics

Additional files

Data availability

References

Decision letter

Roles

Author response

Author response image 1. Monkeys did not show trends in their drift/bound biases across sessions.

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases