Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions

Abstract

Midbrain dopamine (DA) neurons respond to sensory stimuli associated with future rewards. When reward is delivered probabilistically, DA neurons reflect this uncertainty by increasing their firing rates in a period between the sensory cue and reward delivery time. Probability of reward, however, has been externally conveyed by visual cues, and it is not known whether DA neurons would signal uncertainty arising internally. Here we show that DA neurons code the uncertainty associated with a perceptual judgment about the presence or absence of a vibrotactile stimulus. We observed that uncertainty modulates the activity elicited by a go cue instructing monkey subjects to communicate their decisions. That is, the same go cue generates different DA responses depending on the uncertainty level of a judgment made a few seconds before the go instruction. Easily detected suprathreshold stimuli elicit small DA responses, indicating that future reward will not be a surprising event. In contrast, the absence of a sensory stimulus generates large DA responses associated with uncertainty: was the stimulus truly absent, or did a low-amplitude vibration go undetected? In addition, the responses of DA neurons to the stimulus itself increase with vibration amplitude, but only when monkeys correctly detect its presence. This finding suggests that DA activity is not related to actual intensity but rather to perceived intensity. Therefore, in addition to their well-known role in reward prediction, DA neurons code subjective sensory experience and uncertainty arising internally from perceptual decisions.

Midbrain dopamine (DA) neurons modulate the excitability of cortical and subcortical circuits, thus allowing prompt behavioral responses to relevant sensory stimuli (1–9). DA activity also signals when rewards differ from expectation. Larger-than-expected rewards generate strong responses, whereas smaller rewards result in marked inhibition (10–13). In this manner, DA cells code a surprise index that assigns saliency to external events (14). In addition to transient responses to external events, DA neurons also modulate their activity more gradually, within a timescale of seconds, spanning the interval between a sensory cue and the time of reward delivery (4). Interestingly, it was found that the slope of this build-up correlated with uncertainty, being maximal when monkeys obtain reward randomly on half the trials. Reward probability, in the context of these previous experiments, was chosen by the experimenter and communicated to the monkeys by means of different visual cues. However, uncertainty about an outcome is not always determined by external sources; it often arises from our own decisions (15–19). Estimating the degree of uncertainty associated with decisions allows the individual to predict payoffs and plan accordingly (20–23). This led us to study the activity of DA signals in relation to perceptual decisions about the presence or absence of a sensory stimulus. We found that DA activity elicited by a go cue instructing the monkeys to communicate their decisions was higher for stimulus-absent choices compared with stimulus-present choices. We propose that this increase in DA reflects uncertainty arising from the inability to distinguish absence from undetected presence of a sensory stimulus. DA neurons were also activated by vibrotactile stimuli, but only when monkeys detected its presence. This coupling between DA activity and perceptual reports suggests that midbrain DA neurons have access to the processes by which sensory stimuli generate perception.

Results

DA Activity Is Modulated by Uncertainty of Decisions.

We trained two monkeys on a vibrotactile detection task in which they had to indicate the presence or absence of a vibratory stimulus applied to a fingertip (Fig. 1A) (24, 25). They obtained reward for correctly identifying stimulus presence (hit trials) or absence (correct rejection trials) and received no reward when they failed to detect the stimulus (miss trials) or incorrectly reported stimulus presence when stimulus amplitude was zero (false alarm trials). Monkeys were instructed to communicate their decisions by means of a go cue that appeared at the end of a delay period following stimulus presentation (Fig. 1A).

Fig. 1.

Events in the detection task and spiking activity of a DA neuron. (A) Vertical displacement of the stimulator's probe as a function of time (not drawn to scale). After fingertip indentation (start cue), monkeys placed their nonstimulated hand on a fixed key at waist level (hold key), and a variable prestimulus delay ensued (1.5–3.5 s, uniform distribution). On half the trials, a 20-Hz sinusoidal vibration was presented for 0.5 s (stimulus). Stimulus amplitude varied pseudorandomly across trials. At the end of a 3-s poststimulus delay, fingertip indentation was removed (go cue), instructing the monkeys to release the immoveable key and press one of two push-buttons to communicate their decision about stimulus presence or absence (movement). A 0.2-mL drop of liquid was delivered by opening a valve for 0.4 s upon button press on correct trials (hit and correct rejection). (B) Each dot marks the occurrence of an action potential, and each row depicts the activity on one trial. Gray and red markers indicate events in rewarded an unrewarded trials, respectively. Trials are sorted by outcome and amplitude (μm) to facilitate comparisons. Action potentials are aligned to start cue, stimulus onset, and reward delivery. This neuron exemplifies the typical dopaminergic responses to the start cue, stimulus, go cue, and reward events. A significant reduction in spiking activity can be observed after unrewarded behavioral responses (miss and false alarm).

We observed a large DA response to the go instruction when monkeys communicated stimulus-absent decisions (correct rejection and miss trials; Figs. 1B and 2). In comparison, stimulus-present choices (hit and false alarm trials) resulted in significantly smaller responses to the go cue (Figs. 1B and 2). To test whether this DA modulation could be related to decision uncertainty, we calculated the probability of obtaining reward associated with each choice. We found that, in our detection task, stimulus-present choices were rewarded more often [P(reward) = 0.76 ± 0.008 SE] than stimulus-absent choices [P(reward) = 0.64 ± 0.006 SE]. Given that on a two-choice task uncertainty is maximal when reward probability is 0.5, this result indicated that stimulus-present decisions have significantly lower uncertainty than stimulus-absent choices (P < 0.01, two-proportion z test) (26) and suggested that DA go responses might be reflecting uncertainty associated with the monkeys’ perceptual decisions.

Fig. 2.

Population average activity of dopaminergic neurons. Mean activity (black line, ± SEM colored bands) is plotted as a function of time for the four trial types of the detection task. (Left) Activity is aligned to time of fingertip indentation (start), which generates similar responses across trials. Activity is aligned to stimulus onset (Center), where a stimulus response can be observed on hit trials (mean activity across all stimulus amplitudes). Neurons fail to respond to the stimulus on miss trials (all stimulus amplitudes). Responses to the go cue are lower on stimulus-present decisions (hit and false alarm trials) compared with stimulus-absent choices (miss and correct rejection trials). (Right) Activity is aligned to reward when a liquid drop was delivered through a mouthpiece after the button press. Dotted line indicates 5.5 spikes per second, for reference.

We considered the possibility that the reduced activity to the go cue observed during stimulus-present decisions could be explained solely by the presence of the stimulus and not by the decision itself. To test this possibility we compared the activity elicited by the go cue in correct rejection and false alarm trials. Stimulus amplitude is zero in both trial types, but the monkeys’ decision differs between them. We found that the stimulus-present decisions of false alarm trials generated a significantly smaller response than the stimulus-absent choices of correct rejection trials (Fig. 3A; P < 0.05, two-sample t test on standardized values), supporting the idea that it is the monkey's subjective percept, and not only the presence of a sensory stimulus, that modulates the responses to the go cue.

Fig. 3.

Modulation of dopaminergic responses according to decision outcome and stimulus amplitude. (A) Comparison of go cue responses across trial types. The stimulus-present decisions resulting in hit and false alarm (FA) trials significantly reduced the dopaminergic response to the go cue in comparison with the stimulus-absent choices of miss and correct rejection (CR) trials. *Significant difference between false alarm and correct rejection trials (P < 0.05). Activity was standardized (z score) for each neuron with respect to correct rejection trials. (B) The probability of a stimulus-present choice increases as a function of stimulus amplitude. Dotted line indicates a logistic fit to the data. (C) Responses to the go instruction significantly decrease as a function of stimulus amplitude. Activity was measured in a 200-ms window centered 170 ms after the go cue and was standardized with respect to a precue window. (D) Vibration stimulus generates a dopaminergic response that is proportional to stimulus amplitude only on hit trials. Miss trials generate no significant response. Activity was measured in a window centered on stimulus presentation and was normalized with respect to a prestimulus window.

In a detection task, uncertainty of stimulus-absent decisions stems from the presence of low-amplitude stimuli. Near-threshold and subthreshold stimuli often go undetected, resulting in incorrect stimulus-absent decisions (miss trials). It is important to note that from the point of view of the subject performing the detection task, there is no way to distinguish the true absence of stimulus from the presence of an undetected low-amplitude vibration. In contrast, the successful detection of a suprathreshold vibration significantly reduces the uncertainty of obtaining a reward on stimulus-present choices. This reduction of uncertainty as a function of stimulus amplitude is well captured by the psychometric detection curve, which shows that the probability of correctly reporting stimulus-present increases as a function of stimulus amplitude (Fig. 3B). This led us to hypothesize that if DA responses to the go cue signal uncertainty, they should decrease as a function of the amplitude of the stimulus on which decisions are based. Fig. 3C shows a linear regression on the activity elicited by the go cue as a function of stimulus amplitude. The significantly negative slope (R² = 0.78, P < 0.001) provides further support to the idea that DA responses to the go cue reflect the uncertainty associated with decisions on the detection task.

After the go cue, monkeys used their nonstimulated hand to press one of two push buttons to communicate their decision (Fig. 1A, movement). Reward was delivered immediately after pressing the button on correct trials (hits and correct rejections), but no sensory cue other than reward itself was used as feedback. We observed that the lack of reward on incorrect trials reduced the activity of DA cells (Fig. 2, reward), whereas the reward on correct trials (hits and correct rejections) increased their firing rates (P < 0.01, two-sided t test, correct vs. incorrect trials). Interestingly, reward-related activity was stronger on correct rejection trials compared with hit trials (Fig. 2, reward; P < 0.01). This observation is consistent with the proposed role of DA as a prediction signal indicating the difference between expected and actual rewards (3, 13, 27), because the higher uncertainty associated with stimulus-absent decisions explains the larger DA response to reward. It agrees with the interpretation that the same reward magnitude could have different hedonic values depending on the context in which it is obtained: uncertain rewards tend to be more valuable than the same reward delivered without uncertainty (4).

DA Activity Reflects Subjective Perception.

In addition to the responses evoked by the go cue and primary reward, DA cells were modulated by the vibrotactile stimulus itself. Notably, we found that the stimulus elicited significant DA responses only on hit trials (Fig. 2, stimulus; P < 0.01, paired t test). No response was observed on miss trials on which monkeys failed to detect the mechanical vibration (P = 0.55). This result contrasts sharply with sensory responses previously observed in the primary somatosensory cortex, for which neuronal activity on hit and miss trials are not significantly different (24, 25). It suggests that the DA activity elicited by the vibrotactile stimulus is not simply a sensory response but rather reflects whether the monkey perceived the stimulus or not. Moreover, we found that on hit trials DA activity during stimulus presentation positively correlated with vibration amplitude (Fig. 3D; R² = 0.87, P < 0.001). These findings imply that the stimulus-locked DA responses are not uniquely dictated by the physical parameters of the stimulus (28) but instead are related to the monkey's subjective percept of intensity or saliency. Further work is needed to elucidate the relationship between this type of activity and decision confidence (17, 18).

In our detection task, trials started with the probe of a mechanical stimulator indenting the glabrous skin of a fingertip (Fig. 1A, start cue). DA neurons responded to this indentation similarly on every trial (Fig. 2, start cue). Indentation signaled the beginning of a new trial after a variable intertrial interval, so these results are consistent with previous findings suggesting that DA neurons respond to behaviorally relevant stimuli that occur at unpredictable times (3). After fingertip indentation, monkeys placed their nonstimulated hand on an immovable key (Fig. 1A, hold key), and a variable prestimulus delay ensued (1.5–3.5 s, uniform distribution). Interestingly, we observed that DA activity remained significantly elevated with respect to a control period before the start cue (Fig. 2). This tonic activity remained above baseline for the remainder of the trial (3% increase, P < 0.01), and we speculate that it might be a sign of DA modulation of attentional circuits recruited during the detection task (7, 8).

DA Activity Under No Uncertainty.

To gain further insight into the relationship between DA neurons and perceptual decisions, we wondered how DA responses to the stimulus and to the go cue would change in a condition with no uncertainty about reward. For this, we performed a control experiment in which the correct response button was illuminated at the beginning of each trial, before the start cue, but everything else proceeded as in the standard task. Because it indicated the correct response, the light instruction disambiguated the trial outcome and dissipated any uncertainty associated with obtaining a reward (Fig. 4, light instruction). Two findings came out of this control experiment. First, responses to the go cue became similar for stimulus-absent and stimulus-present trials (P = 0.34, two-sample t test), supporting the idea that it is decision uncertainty, not just a difference in motor planning, that modulates the DA responses to the go cue. Second, the light instruction completely abolished the responses to the vibratory stimulus (P = 0.72, paired t test; Fig. 4, stimulus). It is important to emphasize that the same set of stimuli was used in normal detection and control tasks. However, in the control task the presence or absence of the vibratory stimulus becomes irrelevant with respect to obtaining a reward because the light instruction directly identifies the correct choice. The lack of stimulus activation then confirms that DA neurons respond only to behaviorally relevant stimuli. In agreement with this idea, in the control task large DA responses to the light instruction were elicited at the beginning of each trial, and large responses following the go cue were still seen. Importantly, the DA activity was not modulated by the monkey's choice when, in the control task, the reward was obtained without uncertainty.

Fig. 4.

Dopaminergic activity in the absence of uncertainty. The control task was identical to the normal detection task, but now the correct push-button was illuminated at the beginning of each trial (light instruction). Trials were sorted according to stimulus presence or absence. Error rate was 0.6% (22 of 3,724 trials) across conditions, indicating that the light instruction was followed successfully by the monkeys. Note the lack of response to the vibratory stimulus and the large activations to the light instruction and the go cue.

Discussion

We observed that DA activity triggered by a go cue is modulated according to a perceptual decision. When monkeys communicated stimulus-present decisions DA activity was reduced compared with DA activity observed on stimulus-absent decisions. This modulation was present even when comparing different choices made in the absence of a stimulus (false alarm vs. correct rejection trials; Fig. 2A). In the next paragraphs we discuss possible alternative explanations and we evaluate our findings in the context of previous DA research.

It is known that training transfers DA activity originally elicited by reward to a conditioned stimulus (CS) predicting future reward. It can be argued that, in our detection task, the vibrotactile stimulus could act as a CS reducing DA activity elicited by the go cue. However, we think transfer cannot fully account for our results. Responses to the go cue and to reward itself are observed even after presentation of the largest stimulus amplitude. Thus, even after extensive training, there is not complete transfer of DA responses to the vibratory stimulus (Figs. 2 and 3C). A simple transfer explanation predicts that DA activation elicited by successful stimulus detection would abolish DA activation by the go cue and the reward itself. Instead, we observed go cue responses that are graded according to stimulus amplitude (Fig. 3C). Also incompatible with the transfer explanation, we observed a reduction in DA go cue responses on false alarm trials compared with correct rejection trials. The fact that stimulus amplitude is zero on both trial types favors the view that it is the monkey's decision that modulates responses to the go cue.

It has been shown that DA responses to behaviorally relevant events can be modulated by the amount of time elapsed before the event onset (29). In our detection task, there was a variable delay before the stimulus period, causing the go cue to appear randomly 5–7 s after the hold key event (Fig. 1A). However, on trials on which the vibrotactile stimulus is larger than zero there is only a 3-s delay between the last sensory event and the go cue (Fig. 1A). To evaluate a possible effect of elapsed time on DA go cue responses, we divided correct rejection trials into short and long categories (5–6 s and 6–7 s, respectively) and compared the go cue activity on those groups. We found no significant effect of elapsed time on the DA responses elicited by the go cue (P = 0.12).

Schultz and colleagues (4) found that a third of the population of DA neurons code reward uncertainty by gradually increasing their firing rate from the onset of a visual stimulus to the time of reward delivery. In our detection task, the delay after the stimulus period allows (Fig. 1A), in principle, that subjects experience uncertainty before the onset of the go cue. However, we did not observe sustained or ramping activity during the delay period preceding the go cue. This discrepancy might be attributable to differences in task design. In contrast to previous experiments in which the delay is a passive waiting period, the monkeys in our detection task must choose and prepare an action during the delay. We speculate that a go cue allows the uncertainty signals to be incorporated precisely at the time when a behavioral action is required. A previous study of DA and decision making also incorporated a delay period before a go cue (5). Consistent with our results, they did not report sustained DA activity during this delay.

DA activity in the context of sensory evaluation was studied by Sakagami and colleagues (30). They found postdecision DA activity that could be related to the expected reward value of the perceptual decision. They also showed an inverse relationship between stimulus strength and the DA responses evoked by a feedback cue that indicated a correct or incorrect choice. However, the go cue in our detection task does not reveal the trial outcome. Therefore, a reward prediction error signal cannot explain the modulation of the DA activity we observed according to the monkey's decision.

In the detection task, uncertainty is not explicitly indicated by an external cue but instead arises internally by a yet-unknown mechanism likely related to sensory evaluation and decision making. Our results show that uncertainty indeed modulates the DA responses to a sensory cue initiating a behavioral action. The weak responses of DA neurons to the go cue observed under low uncertainty could indicate that the upcoming reward will not be a surprising event. In contrast, the high levels of DA activity generated under high uncertainty may alert target brain areas that future events could not be predicted from past information, thus increasing the levels of attention or arousal to the upcoming events. We speculate that the burst of DA generated by the go cue serves as an alert signal that prepares cortical and subcortical structures to analyze the outcome of behavioral responses (14). Unlike stimulus-present decisions that arise from detection of a physical signal, stimulus-absent decisions cannot be confirmed by an external event and thus always carry a nonzero level of uncertainty. Our results suggest that DA responses correlate with the ability of the primate's decision-making system to correctly infer that, when looking for a specific sensory feature, the absence of evidence is not evidence of absence.

Materials and Methods

Trials in the detection task were classified according to monkey's choice and stimulus amplitude. Stimulus-present choices resulted in rewarded hit trials if vibration amplitude was larger than zero and in unrewarded false alarm trials if amplitude was zero. Conversely, stimulus-absent choices generated rewarded correct rejection trials if stimulus amplitude equaled zero, and unrewarded miss trials if vibration amplitude was larger than zero. The probability of reward on stimulus-present choices was calculated by dividing the number of hit trials by the number of hit and false alarm trials. Conversely, reward probability on stimulus-absent decisions was calculated as the proportion of correct rejection trials to correct rejection plus miss trials. Significance of stimulus response was calculated with a paired t test that compared the firing rate on a 500-ms window before stimulus onset, with the firing rate on a window centered on stimulus presentation. Significance of increase in baseline activity was calculated by comparing the firing rates on a 500-ms window before fingertip indentation, with a 500-ms window before stimulus onset. The somatic stimulus was a 20-Hz vibration, superimposed on a 0.5-mm indentation (start cue, Fig. 1), and it was delivered through the 2-mm round-tip plastic probe of a mechanical stimulator (BME Systems). Firing rates as a function of time (Figs. 2 and 4) were calculated within a 300-ms window displaced in 50-ms steps. Recordings were obtained with quartz-coated platinum–tungsten microelectrodes (2 to 3 MΩ; Thomas Recording) inserted through a recording chamber located over the central sulcus, parallel to the midline. Midbrain DA neurons were identified on the basis of their characteristic regular and low tonic firing rates (1–10 spikes per second) and by their long extracellular spike potential (2.4 ms ± 0.4 SD) (31, 32). A total of 82 cells were isolated (monkey A, n = 45; monkey B, n = 37), of which 69 were recorded with at least three repetitions per stimulus amplitude (monkey A, n = 39; monkey B, n = 30) of the normal detection task. Each cell was recorded for 45–90 min while monkeys performed the normal and control tasks. Monkeys (Macaca mulatta) were handled in accordance with the US National Institutes of Health guide for the care and use of laboratory animals and with Society for Neuroscience guidelines.