The basal ganglia’s contributions to perceptual decision-making

Abstract

Perceptual decision-making is a computationally demanding process that requires the brain to interpret incoming sensory information in the context of goals, expectations, preferences, and other factors. These integrative processes engage much of cortex but also require contributions from subcortical structures to affect behavior. Here we summarize recent evidence supporting specific computational roles of the basal ganglia in perceptual decision-making. These roles likely share common mechanisms with the basal ganglia’s other, more well-established functions in motor control, learning, and other aspects of cognition and thus can provide insights into the general roles of this important subcortical network in higher brain function.

Introduction

For over three centuries, scholars have debated the basal ganglia’s contributions to perception. Extensive projections to the basal ganglia from nearly all parts of the cerebral cortex led early neuroscientists to suggest that these sub-cortical structures are “the seat of the ‘sensorium commune’” (Thomas Willis, 1667, as cited in Wilson, 1914) and that “the royal road of the sensations of the body to the soul is through the corpora striata [the primary input to the basal ganglia] and all determinations of the will also descend by that road” (Emanuel Swedenborg, 1740, as cited in Wilson, 1914). However, this focus on sensation was overshadowed by discoveries that disturbances in the basal ganglia cause muscle contraction and movement disorders, leading to extensive studies of their roles in the selection, initiation, and execution of voluntary movements (Denny-Brown, 1962; Ferrier, 1873; Wilson, 1914). This work, in turn, has provided a framework for recent re-examinations of the basal ganglia’s contributions to non-motor functions (reviewed in Brown et al., 1997). For example, it has been suggested that the parallel anatomical loops within the basal ganglia pathway provide a “centralized selection mechanism” that resolves conflicts at multiple levels and in different domains, thus allowing the basal ganglia to “play a comparable role in cognition to that of action selection in motor control” (Redgrave et al., 1999). In this article we tie these ideas about the basal ganglia’s role in motor control and cognition back to perception, summarizing recent work that implicates this subcortical circuit in specific computations used to form perceptual decisions.

Perceptual decisions are categorical judgments about the presence or identity of sensory stimuli. Examples include determining whether or not a car is approaching, identifying a face in a crowd, or detecting a faint cry for help. These kinds of decisions are not reflexive responses to sensory input but rather deliberative processes that combine the available sensory evidence with information related to the known alternatives, past experiences, goals, and other factors to reach a categorical judgment that can guide behavior. These deliberative processes contribute to all measurable aspects of perception, from sensitivity to low-level features of sensory input to the ability to parse complex scenes (Gold and Ding, 2013; Parker and Newsome, 1998). The underlying neural mechanisms include contributions from many parts of the cerebral cortex, particularly integrative areas in the parietal and prefrontal cortices (Gold and Shadlen, 2007; Heekeren et al., 2008). However, the complex computations needed to implement these processes and use them to guide behavior are not solely a cortical phenomenon.

Here we synthesize from recent electrophysiological, imaging, and computational studies a summary of our current understanding of the basal ganglia’s contributions to the computations that are responsible for perceptual decisions. The article is organized as follows. First we provide some background on mechanisms of perceptual decision-making in the brain, focusing on experimental paradigms that have allowed researchers to identify neural signatures of key computations of the decision process. We then briefly describe the basal ganglia circuitry that we will subsequently relate to perceptual decision-making. We then review recent evidence that supports possible roles for this circuitry in three types of decision-related computations. We close with a discussion of open questions related to the role of the basal ganglia in perceptual decision-making.

Computational and neural substrates of perceptual decision-making

Psychophysical techniques developed over the past 150 years have provided the tools needed to examine quantitatively how the brain converts noisy sensory input into a categorical choice. An important advance in these techniques was the incorporation of principles of Signal Detection Theory, which established the usefulness of analyzing perceptual decisions in terms of computationally separable processes (Green and Swets, 1966; Macmillan and Creelman, 2004). These processes include formation of the decision variable, which is a scalar quantity representing all available evidence (including signal and noise) used to form the decision, and application of the decision rule, which converts the decision variable into a categorical choice. Later extensions of this Theory, representing a form of statistical decision theory known as Sequential Analysis, further characterized formation of the decision variable as a temporally dynamic process that takes advantage of incoming streams of sensory data to balance the speed and accuracy of the decision process (Bogacz et al., 2006; Gold and Shadlen, 2007; Link and Heath, 1975; Ratcliff and Smith, 2004). Critically, these computational frameworks have provided not only a description of decision outcomes under certain conditions, but also insights into the underlying neural mechanisms.

These principles have been applied extensively to a task that involves a decision about the global motion direction of a field of moving, randomly positioned dots on a computer screen (the “dots task”; Britten et al., 1992; Morgan and Ward, 1980). For this task, experimenters can precisely control the difficulty of the decision by changing the percentage of coherently moving dots (coherence). On high-coherence trials, the majority of dots move in the same direction, making it easy to decide the correct global motion direction. On low-coherence trials, only a small percentage of dots move in the same direction, while the other dots move randomly, making the direction decision more difficult. Both human and monkey subjects can be trained to perform with high accuracy even for low-coherence stimuli. Performance also depends critically on viewing duration, with increasing accuracy for longer viewing durations, particularly for low-coherence stimuli. Accordingly, when a subject is instructed to respond as soon as the decision is formed, as in response-time (RT) versions of the task, responding quickly yields lower accuracy, whereas taking longer to respond corresponds to higher-accuracy decisions (Palmer et al., 2005; Roitman and Shadlen, 2002).

Successful models of this decision process typically assume that the sensory evidence, which fluctuates noisily from moment-to-moment relative to a constant average value on a given trial, is integrated over time (Figure 1; Mazurek et al., 2003). This form of Sequential Analysis increases the signal-to-noise ratio of the decision variable as a function of viewing time. For the RT task, many models further assume a decision rule in the form of a pair of stopping bounds, or thresholds: when the accumulating evidence reaches one of these predefined values (often corresponding to a positive value for one choice, a negative value of equal magnitude for the alternative), the process stops. The identity of the reached bound determines the choice; the time of bound-crossing determines the RT. Adjusting the bound governs the speed-accuracy trade-off: a higher bound provides higher accuracy but longer RTs, whereas a lower bound provides lower accuracy and shorter RTs. This process can be modeled using the mathematical description of the position of a subatomic particle undergoing Brownian motion, which corresponds to the noisy, accumulating decision variable. This Drift Diffusion Model (DDM) can effectively describe psychometric (accuracy versus motion coherence) and chronometric (RT versus motion coherence) performance data (Palmer et al., 2005; Ratcliff and McKoon, 2008; Ratcliff and Rouder, 1998).

Figure 1.

Drift-diffusion model (DDM). Noisy sensory evidence (top; independent, identically distributed picks from the Gaussian distribution at the right) is accumulated in time to form a decision variable (bottom). Note the slightly positive mean value of the distribution of evidence, which governs the upward rate of rise of the decision variable. When the decision variable hits one of the choice bounds, this choice is made.

The computations described by the DDM have been identified in several brain regions (see review by Gold and Shadlen, 2007). The sensory evidence for this task is represented, at least in part, in the middle temporal (MT) and medial superior temporal (MST) areas of extrastriate visual cortex (Britten et al., 1996; Britten et al., 1992,1993; Celebrini and Newsome, 1994, 1995). Neurons in these brain regions respond selectively to visual stimuli moving in particular directions and thus provide a moment-by-moment representation of the dots stimulus. Electrical microstimulation of MT sites affects both choice and RT and the combined effects are consistent with MT neurons providing momentary evidence to an accumulator (Ditterich et al., 2003; Hanks et al., 2006; Salzman et al., 1990). The temporal accumulation of momentary evidence is reflected in the activity of certain neurons outside the primary visual areas, including in the lateral intraparietal area (LIP) of parietal cortex (Shadlen and Newsome, 1996). Unlike MT neurons, these LIP neurons have activity that builds up (or down) during the decision process, with coherence and time-dependence consistent with a decision variable in the DDM. For an RT version of the task, this activity appears to reach a pre-defined value just prior to the decision, as prescribed by the DDM (Roitman and Shadlen, 2002). LIP is not alone, however, in representing a DDM-like decision process for this task. Similar activity and/or causal relationship with visual perceptual decisions have been found in several other brain regions that are strongly interconnected with LIP, including the frontal eye field (FEF) and other parts of prefrontal cortex and the superior colliculus (Ding and Gold, 2012a; Ferrera et al., 2009; Horwitz and Newsome, 1999; Kim and Shadlen, 1999; Krauzlis, 2004; Lovejoy and Krauzlis, 2010; Ratcliff et al., 2003; Ratcliff et al., 2007).

The involvement of multiple brain regions in the oculomotor network reflects the behavioral context in which these perceptual decisions were studied (but may also be more general; see Bennur and Gold, 2011; Freedman and Assad, 2006; Rishel et al., 2013). Specifically, the monkeys were trained to indicate their direction decisions with saccadic eye movements to visual targets located along the axis of coherent motion. Under these conditions, the brain appears to treat the perceptual decision as a form of saccadic selection, representing a form of “embodied cognition” in which higher brain functions like perceptual decision-making are implemented directly in the service of behavioral planning and control (Gibson, 1966). According to this view, other oculomotor brain regions may also participate in saccade-linked perceptual decisions.

The basal ganglia are well positioned functionally and anatomically to contribute to saccade-linked decisions (Figure 2). The caudate nucleus is the primary oculomotor component of the striatum, with signals related to the preparation and execution of saccadic eye movements (Hikosaka et al., 2000). It receives inputs from both FEF and LIP. Its output is split along direct and indirect pathways, which are thought to have facilitatory and inhibitory effects, respectively, on behavior (Albin et al., 1989; Alexander and Crutcher, 1990; DeLong, 1990; Kravitz et al., 2010; Smith et al., 1998). These pathways converge in the substantia nigra, pars reticulata (SNr), which sends the output of the oculomotor basal ganglia to the superior colliculus and, via the thalamus, back up to cortex. Thus, the basal ganglia carry oculomotor-related signals and are intricately interconnected with other brain areas that are implicated strongly in perceptual decisions that instruct saccadic eye movements.

Figure 2.

A simplified basal ganglia circuit. The dashed black line represents feedback pathways. Abbreviations: FEF: frontal eye field; GPi and GPe: the internal and external segments of the globus pallidus; LIP: lateral intraparietal area of the parietal cortex; SC: superior colliculus; SNc: substantia nigra pars compacta; SNr: substantia nigra pars reticulata; STN: subthalamic nucleus; VTA: ventral tegmental area.

This oculomotor basal ganglia circuit has long been thought to play primarily a permissive role in the generation of saccadic eye movements. Tonic inhibition from the SNr to the superior colliculus is briefly released around the time that a saccade plan is activated, allowing for enough excitatory drive to activate the brainstem saccade generators and thus initiate the movement (Hikosaka and Wurtz, 1983d). In addition, this and other basal ganglia circuits are thought to be important loci for reinforcement-based motor control, which on relatively long timescales helps to optimize policies for action selection to maximize opportunities to obtain benefits and avoid costs when interacting with the world (Barto, 1995; Hikosaka et al., 2006; Houk et al., 1995; Redgrave et al., 1999; Shadmehr and Krakauer, 2008; Turner and Desmurget, 2010).

Below we review current evidence that these motor-related functions of the basal ganglia can also play specific roles in the interpretation of sensory input. These roles are likely implemented in the service of helping to select impending or delayed movements and thus can be thought of in an “embodied” framework. Within this framework, the basal ganglia appear to provide specific computations to help form perceptual decision variables, implement decision rules, and evaluate and modify the decision process via learning.

Formation of the decision variable

Given the connectivity of the oculomotor circuit and the presence of DDM-like decision signals in certain neurons in LIP, FEF, and the superior colliculus, an obvious question is whether or not these signals are sent through the basal ganglia pathway, and if so, what, if any, functional role they play in the decision process. To answer these questions, we recently targeted the oculomotor caudate with neuronal recordings, electrical microstimulation, and computational modeling. In short, we found that caudate neurons can represent and causally contribute to the accumulating decision variable used to make the final saccadic choice.

Figure 3 compares the decision variable predicted by the DDM and neural activity we measured in the caudate and FEF of monkeys performing an RT version of the dots task. Figure 3A and B shows simulated trials that terminated with a choice associated with the upper bound. After stimulus onset, the decision variable rises in a manner that depends on stimulus strength and then terminates upon reaching the upper bound. For the alternative choice, the decision variable follows downward trajectories until reaching the lower bound (not shown). Caudate activity shows a similar dependence on motion strength and viewing time, at least relatively early in the decision process (Ding and Gold, 2010; Figure 3C). After motion onset, there is a brief delay as the relevant visual signals propagate from the retina to the basal ganglia. Subsequently, there is a motion strength-dependent rise in responses on trials that ultimately result in a saccadic eye movement to the target located in each neuron’s spatial response field. This rise in activity is similar, albeit slightly weaker, on error trials (see Ding and Gold, 2010). This pattern of activity is consistent with a DDM-like decision variable that represents not the sensory evidence itself but rather the interpretation of that evidence to arrive at the final choice, similar to LIP, FEF, and the superior colliculus (example FEF activity is shown in Figure 3E).

Figure 3.

Decision formation. A,B, Simulated data illustrating the average decision variable trajectories in a DDM for different input strengths, aligned to the beginning (A) or end (B) of the accumulation process. Colors indicate coherence levels. C,D, Average activity of a subset of caudate neurons recorded in monkeys performing the dots task, aligned to stimulus (C) and saccade (D) onset, respectively. IN: choices toward the recorded neuron’s response field (RF). OUT: choices away from the RF. Colors indicate data from trials with different motion coherences. Note the choice- and coherence-dependent modulation of caudate activity during motion viewing and the lack of a final “bound-like” pattern of activity level around saccade time. Modified from Ding and Gold (2010). E,F, Average activity of a subset of FEF neurons recorded in monkeys performing the dots task, aligned to stimulus (E) and saccade (F) onset, respectively. Modified from Ding and Gold (2012a).

In contrast, caudate activity that occurs later in the decision process does not match predictions of the DDM. In the DDM, the decision process ends when the accumulating decision variable reaches a fixed bound. Accordingly, when the decision variable is aligned in time to the end of the decision process, all of the curves should converge at a common level, regardless of their rate of rise (Figure 3B). Certain FEF and LIP neurons show this behavior (Figure 3F; Ding and Gold, 2012a; Roitman and Shadlen, 2002). However, caudate activity does not (Ding and Gold, 2010; Figure 3D). Instead of converging to a peak level of activity that immediately precedes saccades, average caudate responses converge on a value that is lower than the peak activity achieved during motion viewing. Together these results imply that the caudate’s contributions to the formation of the decision variable might be limited to early in the decision process.

These contributions can causally affect the outcome of the ongoing decision process. To establish this causal role, we used electrical microstimulation in the caudate to bias both the choices and RTs of monkeys performing the dots task (Ding and Gold, 2012b). In relation to the DDM, these effects had two distinguishable components. One component reflected a bias in non-perceptual processes, such that non-decision times (i.e., the components of the monkey’s RT that was not accounted for by the DDM-like decision process, probably including basic sensory and motor processing) increased for ipsilateral choices and decreased for contralateral choices. This result is consistent with the basal ganglia’s known role in facilitating saccadic eye movements to contralateral targets. The second component included a decrease/increase in the total amount of accumulated evidence required for ipsilateral/contralateral choices. This component can be interpreted as a caudate-mediated offset in the value of the decision variable in the DDM and was similar to results from LIP microstimulation, albeit opposite in sign (LIP microstimulation tended to cause a bias towards contralateral choices; Hanks et al., 2006).

Neural activity reminiscent of an offset in the initial value of the decision variable was also observed in a small subpopulation of caudate neurons (Ding and Gold, 2010). This type of activity emerges early, well before motion onset. As illustrated in Figure 4A, a positive starting value reduces the total amount of evidence required for the choice with positive decision bound, thus making it more likely for the decision variable to cross that bound and creating a choice bias. This biasing effect is more profound when stimulus strength is low. In other words, on more difficult trials, in which low-coherence motion stimuli do not provide much evidence for either choice, the relative magnitude of the starting value is more predictive of the monkey’s subsequent saccadic choice. On easier trials, the effect of the starting value is drowned out by strong evidence and the starting value is not predictive of choice. Figure 4B shows an example of a caudate neuron that encoded this kind of process.

Figure 4.

Initial bias signals. A, Simulated data illustrating the effects of a non-zero starting value on a DDM-based decision process. Gray lines show trajectories of a decision variable in 15 simulated trials with low- (left) and high- (right) coherence inputs and using either a zero (top) or slightly positive (bottom) starting value. Note the stronger effects of the initial bias on the final choices for the low-versus high-coherence simulations. B, Coherence-dependent predictive index of pre-stimulus activity in a caudate neuron. The predictive index quantifies the correlation between neuronal activity and subsequent behavioral choices. A higher value indicates that the activity is more predictive of the final choice. Colors indicate motion coherence. The right panel shows the mean predictive index from the shaded time window in the left panel, plotted as a function of motion coherence. Modified from Ding and Gold (2010).

The starting value-related signal appears similar to a reward bias-related signal that has been identified in the caudate nucleus. In one notable study, monkeys were trained to make a saccadic eye movement to a target flashed at one of two possible locations (Lauwereyns et al., 2002). Critically, one of the locations was paired with juice reward and the other was not rewarded. Behaviorally, the monkeys tended to have shorter RTs when instructed to make a saccadic eye movement to foveate the rewarded target. These reward-driven biases in RT were correlated with the magnitude of neuronal activation of oculomotor caudate neurons before target presentation. One parsimonious explanation for these results is that the basal ganglia modulates the initial value and development of a decision variable based on reward expectation and other factors, ultimately biasing not just movement execution but also movement selection.

These results are supported by several recent fMRI studies. When prior probability or reward association is unequal for the two motion directions, human subjects’ behavior is biased toward the choice associated with higher prior probability or larger reward (Feng et al., 2009; Forstmann et al., 2010; Mulder et al., 2012; Nagano-Saito et al., 2012; Voss et al., 2004). This bias reflects a non-zero starting value in a DDM-like decision process and is encoded in parts of the striatum (Forstmann et al., 2010; Nagano-Saito et al., 2012). Collectively, these experimental results suggest that the basal ganglia can incorporate expectations about sensory stimuli and reward outcomes to bias the value of a developing decision variable.

An even more expansive role for the basal ganglia in the formation of decision variables has been proposed by a recent theoretical study. Bogacz and Gurney (2007) suggested that the basal ganglia network may implement a multi-hypothesis sequential probability ratio test (MSPRT) for perceptual decision making. The MSPRT estimates the conditional probabilities of the multiple hypotheses being true given sensory stimuli and commits to decision i if the logarithm of the corresponding conditional probability (L_i, which can take different forms including log-likelihood, log-likelihood ratios, or log-odds (Lepora and Gurney, 2012)) reaches a predefined threshold. L_i is proportional to a time integral of sensory evidence for one choice and normalized across all alternative choices. According to this model, the direct pathway, in which the striatum projects directly to the pallidal output neurons in GPi, relays the un-normalized values of these probabilities. The indirect pathway, in which cortical inputs are further processed in the interconnected STN-GPe circuits, gathers information related to all alternatives and provides the (possibly modifiable) normalization quantity through the STN-GPi projection. The output of the basal ganglia thus reflects L_i, upon which a threshold may be applied to generate a decision. This intriguing idea awaits experimental testing.

Implementation of the decision rule

As noted above, a key feature of Signal Detection Theory is that the decision variable and the decision rule are distinct components of the decision process, with identifiably different consequences on behavior (Green and Swets, 1966; Macmillan and Creelman, 2004). Given the dominant view of basal ganglia function in terms of action selection, it is natural to consider its role in implementing the final rule, or selection process, of a winner-take-all decision between multiple alternatives (Berns and Sejnowski, 1995; Mink, 1996; Redgrave et al., 1999; Wickens, 1993). A possible scheme that is consistent with the basal ganglia’s known roles in action selection is as follows. Different cortex-striatum ensembles form separate processing units that link inputs to actions. A specific input pattern leads to activation of the corresponding pallidal neurons, which subsequently disinhibit downstream thalamus/colliculus areas and enables the corresponding action. Activation of the same cortex-striatum ensemble also disinhibits subthalamic neurons via the GPe, which provides delayed and diffuse activation of pallidal projection neurons, such that all other actions are suppressed. In principle, if the specific input pattern represents the prediction of a preferred outcome, this scheme can support value-based decisions. Conversely, if the specific input pattern represents certain properties of sensory stimuli, this scheme can support perceptual decisions.

If such a scheme is implemented in the basal ganglia, one might expect to observe correlates of a DDM-like bound crossing at the end of the decision process, representing a commitment to one of the two possible outcomes. As noted above, in monkeys performing an RT version of the dots task, this kind of activity is observed in LIP and FEF but not in the caudate (Figure 3). One interpretation of this difference between caudate and LIP/FEF activity at the time of decision commitment is that the basal ganglia are only involved in the early part of the decision process. Alternatively, bound crossing may occur downstream from the caudate in the basal ganglia pathway and then get sent back up to cortex. These ideas have not yet been tested directly.

Despite the questions about if and how the basal ganglia might implement the decision rule, several lines of evidence suggest that they can at least help to adaptively modulate its implementation. For example, changing task demands can cause human subjects to adjust their speed-accuracy trade-offs on an RT version of the dots task. These adjustments correspond to reliable changes in activation of the anterior striatum measured using fMRI (Forstmann et al., 2010; Forstmann et al., 2008). Using a linear ballistic accumulation model – one variant of the accumulation-to-bound models – to relate fMRI signals to behavior, the between-condition difference of striatal activation was shown to correlate with the difference in the estimated bound height, but not drift rate, of the model fit to behavior.

This result is consistent with predictions from a biophysically motivated model of the cortex-basal ganglia-superior colliculus network (Lo and Wang, 2006). The model contains a recurrent cortical network that generates buildup activity similar to that observed in LIP and FEF, a recurrent superior colliculus network that produces burst of activity for saccade generation and resets the cortical network through feedback connections, and a basal ganglia network that connects the cortical and collicular networks through only the direct pathway. With biophysical parameters consistent with experimental observations, the model detects threshold crossing of the ramping cortical activity through the cortex-colliculus pathway. However, the fine-tuning of the effective threshold value can be done more efficiently with changes in the corticostriatal projection than with changes in the cortico-collicular projection. Thus, within the context of this model, the basal ganglia may play modulatory roles in the adjustment of the decision bound, even if the decision rule is not implemented there.

A modulatory role of the basal ganglia is also supported by a study of the subthalamic nucleus in an impulse-control behavioral paradigm (Cavanagh et al., 2011). Although not a perceptual task, the impulse-control behavior in the study can also be described by a DDM, with the decision bound influencing the propensity for impulsive choices. The decision bound is correlated with theta-power signals in the medial prefrontal cortex and lower-frequency local field potential in the subthalamic nucleus. Both the decision bound and the prefrontal signals can be altered with deep brain stimulation in the subthalamic nucleus. Combined with the striatal results detailed above, it seems that multiple nuclei in the basal ganglia can influence how a decision bound is set, suggesting an important role for the basal ganglia in modifying the decision rule.

Evaluation and learning

Since the discovery of reward prediction error signals in the dopaminergic neurons, reinforcement learning – especially the so-called temporal-difference learning processes – has been linked with basal ganglia functions (Barto, 1995; Houk et al., 1995; Schultz, 1998). These learning processes, such as Q-learning or actor-critic methods, make decisions using a modifiable policy that operates on estimates of the values of alternative actions and/or estimates of current states (Sutton and Barto, 1998). Action/state values are estimated based on current sensory input, past reinforcement history, and knowledge of the decision policy. With repeated trial-and-error, these methods improve both decision policy and estimates of action/state values, allowing the decision maker to adapt to current task demand for better performance; i.e., more overall reinforcement.

In temporal-difference learning, action value is updated based on the error between its actual and predicted values, called the reward prediction error. Dopaminergic neurons, which provide strong modulatory input to the striatum and elsewhere, are a classic example of a neural representation of the reward prediction error (Schultz, 1998,2002). When a reward is unexpected, these neurons respond with phasic activation to reward delivery. When the reward can be fully predicted by a sensory cue, these neurons respond with phasic activation to the cue, but no longer to the reward itself. When expected reward does not arrive, these neurons respond with suppression of activity at the expected time of reward delivery. When the reward can be partially predicted by the cue, the magnitude of these neurons’ phasic activation is correlated with the difference between received and predicted reward. These patterns of dopaminergic neuron activity resemble prediction error signals used in temporal-difference learning. Furthermore, the basal ganglia circuits, especially interactions between striatal and midbrain dopaminergic neurons, provide the primary candidate substrate for acquisition of such neural signals (reviewed in Joel et al., 2002).

In the context of perceptual decision-making, stimulus uncertainty can also give rise to prediction errors that might drive learning. For example, for the dots task, higher coherence and/or longer viewing times give rise to decision variables that are more likely to produce the correct answer. For many tasks, the correct answer leads to a reward (e.g., juice for monkeys, money for people), whereas an error is not rewarded. Thus, in principle, a reward prediction error can be computed by comparing the confidence associated with the final value of the decision variable with whether or not a reward was actually received at the end of a trial. In fact, such a signal is sufficient to drive learning on the dots task and can account for both changes in behavior and changes in decision-related neuronal activity measured in area LIP during training (Law and Gold, 2009).

Signals related to reward prediction errors in the context of the dots task have recently been reported for dopaminergic neurons in the substantia nigra pars compacta (Figure 5A). Nomoto and colleagues (2010) used a version of dots that included manipulations of both motion strength and the magnitude of reward given for correct responses. When large rewards were expected, dopaminergic neurons gave a phasic response just after motion stimulus onset that was not sensitive to motion strength. In contrast, a second phasic response around the time of saccade onset was modulated positively by motion strength. After reward feedback onset, this modulation by motion strength was reversed, such that larger activation was associated with lower motion strength. When an error was made, there was a brief suppression in activity after feedback. This pattern of activity is consistent with a reward prediction error based on the perceptual decision variable at the time of decision.

Figure 5.

Reward prediction error (RPE) signals. A, Population activity of DA neurons in a monkey performing an asymmetric-reward task when large reward was expected. Data are plotted with respect to onset of the visual stimulus (STIM), the saccadic response (SAC), and either reward delivery (REW) or error feedback (FDBK). Colors indicate coherence levels. Modified from Nomoto et al. (2010). B, Reward prediction error (RPE) signals derived from a DDM simulation. At time t during motion viewing, $\mathrm{RPE}=\frac{accumulatedevidence}{\sqrt{t}}$. After motion viewing, RPE decays exponentially with a time constant of 400 ms. After feedback onset, RPE is updated with the difference between feedback value (simulated as a positive square pulse for rewarded trials and 0 for error trials) and the RPE value at 100 ms after feedback onset. C, Activity of a caudate neuron during the dots task for one choice. Data for the other choice showed similar patterns. Modified from Ding and Gold (2010).

In principle, such a reward prediction error can be computed continuously as the decision variable is being formed, in anticipation of the impending choice and subsequent reward. The prediction can be computed from the signal-to-noise ratio of the decision variable, with higher signal-to-noise ratio corresponding to higher confidence in obtaining a reward. In the DDM, the sensory evidence is assumed to be independent samples from a Gaussian distribution. Thus, the signal is equal to the drift rate multiplied by elapsed time, and the standard deviation (noise) of the accumulating decision variable is proportional to the square root of elapsed time. Figure 5B shows a simulated reward prediction error computed this way. After motion stimulus onset, the reward prediction error ramps up in a manner that depends on the strength of the motion signal but is the same for both choices. Around the time of the saccadic response, the reward prediction error peaks at different levels for different motion strengths and then decays until the time of expected reward delivery. After reward onset, the motion-strength modulation reverses signs, such that larger activation is associated with lower motion strength. When an error is made, the reward prediction error is suppressed after feedback. We found signals loosely conforming to these patterns in the caudate nucleus of monkeys trained on the RT dots task (Figure 5C; Ding and Gold, 2010). Although caudate neurons showing the full aspects of these response patterns were rare, subsets of these response patterns were frequently observed in the population. Thus, these populations may represent ongoing estimates of predicted action values in the context of perceptual decisions.

The predicted action value may, in principle, play multiple computational roles in decision formation. One recent study implemented a partially observable Markov decision process (POMDP) model to identify these roles (Rao, 2010). This model includes: 1) a cortical component (e.g., LIP and FEF for the dots task) that encodes a belief about the identity of noisy sensory inputs; 2) highly convergent corticostriatal projections that reduce the dimensionality of the cortical belief representation; 3) dopamine neurons that learn to evaluate the striatal representation through temporal-difference learning; and 4) a striatum-pallidal-STN network that learns to pick appropriate actions based on the evaluation. At each time step, the model either commits to a decision about motion direction, which results in a large reward for correct decisions or no reward for errors, or opts to observe the motion stimulus longer, which takes a small effort (negative reward) for waiting. The model initially makes random choices. Over multiple trials, the model learns to optimize performance based on tradeoff among the three reward outcomes, producing realistic choice and RT behaviors. Thus, the basal ganglia may convert cortical representations of sensory evidence into evaluative quantities, upon which decisions can be both generated and adjusted.

Open questions

As summarized above, there is a growing body of experimental and theoretical support for the idea that the basal ganglia play key, well-defined computational roles in the formation and adaptive modification of perceptual decisions. These roles may complement and/or share common mechanisms with the basal ganglia’s contributions to motor control and value-based decision-making. However, this work is still in its infancy, especially compared to studies of perceptual processing in sensory, prefrontal, and parietal cortices. Below we touch on some of the key, open questions about the exact roles played by the basal ganglia in perceptual decision-making.

First, what is the nature of the signals that the basal ganglia receive as input in the context of perceptual decisions? Each component of the oculomotor network shown in Figure 2 contains a diversity of response properties related to sensory, memory, decision, motor, and reward processing in the context of visual-oculomotor decision tasks (For a limited sample, see Basso and Wurtz, 1997,2002; Bruce and Goldberg, 1985; Ding and Gold, 2010,2012a; Ding and Hikosaka, 2006; Freedman and Assad, 2009; Glimcher and Sparks, 1992; Gottlieb et al., 1998; Hanes et al., 1998; Hikosaka et al., 1989a, b, c; Hikosaka and Wurtz, 1983a, b; Hikosaka and Wurtz, 1983c; Horwitz and Newsome, 1999; Leon and Shadlen, 2003; McPeek and Keller, 2002; Meister et al., 2013; Schall et al., 1995; Thompson et al., 1996). For example, DDM-like bound crossings are represented in LIP and FEF but not caudate, implying that this signal is not provided as an input to the basal ganglia (Ding and Gold, 2010,2012a; Roitman and Shadlen, 2002). Is it represented in the output nuclei (i.e., SNr) and then sent back to the oculomotor circuits? Likewise, do the bias-related signals found in caudate originate there, or are they passed from LIP and FEF? Those cortical areas represent similar bias-related signals in the context of reward-driven saccadic instructions (but not necessarily perceptual decisions), but it is not known whether these signals are present in the subset of neurons that project to the caudate (Coe et al., 2002; Ding and Gold, 2012a; Ding and Hikosaka, 2006; Ikeda and Hikosaka, 2003; Meister et al., 2013; Platt and Glimcher, 1999; Roitman and Shadlen, 2002; Rorie et al., 2010; Sato and Hikosaka, 2002). Characterizing these kinds of input properties in detail will help to identify the basal ganglia’s unique contributions to the decision process.

Second, what is the computational role of each basal ganglia nucleus? Answering this question will require more complete descriptions of perceptual decision-related signals encoded in the basal ganglia. Computational models can provide useful starting points for these studies. For example, in Bogacz and Gurney’s model (2007), the average STN activity is predicted to be proportional to the logarithm of the normalization term in Bayes’ theorem, which in the model is used to form the decision variable in terms of the accumulated evidence. In Rao’s model (2010), the STN is partly responsible for choosing the best action based on belief representation in the striatum, although it was not explicitly reported what STN firing rate would look like. A comparison among the model predictions and actual STN activity patterns during the dots task will help to elucidate the STN’s roles in the decision process. Likewise, more extensive recordings from the output nuclei of the basal ganglia, including the SNr for the oculomotor circuit, are needed to understand how the inputs are transformed and subsequently affect processing elsewhere.

Third, how do the basal ganglia’s roles in perceptual decision-making relate to their known functional and anatomical properties? For example, do the direct and indirect pathways play similar, complementary roles in perceptual decision-making as they do in motor control? Are perceptual decisions processed in their own functional loops, in loops related to the motor context of the decision, or in more general functional loops? The relationship between perceptual and reward-based processing merits particular attention. One intriguing possibility is that the same circuit contributes to both types of decisions, converting sensory evidence and value expectation into a common currency that can be used as a decision variable. One way to answer this question is to train monkeys on a perceptual task (e.g., the dots task) and a value-based decision task (e.g., the asymmetric reward saccade task) and directly test if and how the same neurons are influenced by manipulations of sensory properties and reward expectation. Alternatively, one can train monkeys to perform a single task with manipulations of both sensory properties and reward associations (Nomoto et al., 2010; Rorie et al., 2010) and examine whether single neurons respond to variations in both sensory evidence and reward expectation, and if so, how such variations are combined in the basal ganglia.

Lastly, why is basal ganglia dysfunction more frequently associated with motor than with perceptual deficits? This widely recognized clinical observation has been a pillar in motor-centric views of the basal ganglia. Is it merely due to an observational bias, such that motor deficits are more often expected and tested for and therefore reported? Or does it reflect more fundamental differences in the time course and distribution of impairment in the motor and perceptual components in the basal ganglia? In general, a more systematic characterization of perceptual deficits in the patient population, perhaps using tasks equivalent to those used in monkey electrophysiology experiments, would not only improve our understanding of the functional organization of the basal ganglia, but may also allow us to exploit behavioral differences in the motor and perceptual domains to improve disease diagnostic and monitoring.