Representation of confidence associated with a decision by neurons in the parietal cortex

Abstract

The degree of confidence in a decision provides a graded and probabilistic assessment of expected outcome. Although neural mechanisms of perceptual decisions have been studied extensively in primates, little is known about the mechanisms underlying choice certainty. Here, we show that the same neurons that represent formation of a decision encode certainty about the decision. Rhesus monkeys made decisions about the direction of moving random dots, spanning a range of difficulties. They were rewarded for correct decisions. On some trials, after viewing the stimulus, the monkeys could opt out of the direction decision for a small but certain reward. Monkeys exercised this option in a manner that revealed their degree of certainty. Neurons in parietal cortex represented formation of the direction decision and the degree of certainty underlying the decision to opt out.

Choice certainty — the degree to which a decision maker believes a choice is likely to be correct — affects a variety of cognitive functions: how we plan subsequent actions, how we react and learn from mistakes and how we justify our choices to others. Choice certainty is pivotal for planning actions in a complex environment in which subsequent decisions depend on pending outcomes of previous decisions (1–3). For example, a decision to undergo a risky operation depends, among other factors, on the degree of certainty that the diagnosis is correct. Psychologists have long proposed that choice certainty serves as a link between the physical world and belief: it provides a graded scale which allows us to translate our convictions into suitable actions (4, 5).

Despite the importance of choice certainty, its neural mechanisms are poorly understood. It is well established that choice certainty is closely correlated with both decision accuracy and reaction time (6–11). This close relationship suggests that the same mechanism that underlies the decision making process might underlie certainty judgments (1, 10, 12, 13). It has been suggested that neurons in orbitofrontal cortex and cingulate cortex, which are known to represent reward expectation or conflict, represent reward expectations associated with decision uncertainty (14–16). However, these neurons do not give rise to a representation of decision uncertainty but presumably receive this information from neurons that compute this quantity in the decision making process.

The neural mechanism of decision-making has been investigated using simple perceptual tasks in which a monkey makes a categorical choice between two or more discrete options based on a sensory stimulus (17). When the monkey is required to report the perceived direction of motion by a saccadic eye movement, neurons in lateral intraparietal cortex (LIP) represent the accumulation of evidence, termed a decision variable, that supports the target in their response fields (17–19). Furthermore, these neurons signal the termination of the decision process when their firing rates reach a critical level or bound (19–21). Theoretical and experimental studies raise the possibility that the neural computations approximate a form of probabilistic reasoning about the alternatives (22–24). We hypothesize that the graded, time-dependent firing rates of LIP neurons also represent choice certainty. Our hypothesis, therefore, unifies the representation of the three components of decisions—choice, RT and certainty—in a single neural population.

Two monkeys made perceptual decisions about the net direction of motion in a dynamic random dot display (Fig. 1A)(25). Task difficulty was controlled by varying both the percentage of coherently moving dots and the viewing duration. After a delay period, the fixation point was extinguished, which instructed the monkey to indicate its direction choice by making an eye movement to one of the direction-choice targets. On a random half of trials, the monkey was given the option to abort the direction discrimination and to choose instead a small but certain reward associated with a third saccade target. Importantly, this “sure-target” was shown during the delay period, at least 500 ms after the random dot motion was extinguished. During motion viewing, the monkey did not know whether the sure-bet option would arise. The task design, a form of post-decision wagering (26–28), ensured that the monkey made a decision about motion direction on each trial. We hoped that the monkey would choose the sure target when less certain of the high-stakes direction choice, allowing us to study neural responses associated with choice-certainty.

Figure 1.

Post-decision wagering behavior in monkeys is indicative of choice certainty. (A) The sequence of events in the task. After acquiring a central fixation point (small red point), two “direction targets” (large red spots) appeared on the screen, one inside the neural response field (RF; gray shading), the other on the opposite side of the screen. The motion stimulus appeared after a short delay, remained visible for 100–900 ms, and was followed by another delay (1200–1800 ms). On half of the trials (lower branch) the delay persisted until the fixation point was turned off, which served as a Go signal that instructed the monkey to indicate the perceived direction of motion by a saccadic eye movement to one of the “direction targets.” A correct response led to a liquid reward; a wrong response led to no reward and a brief timeout. On the other half of the trials (upper branch) a third target was presented 500–750 ms after extinction of the motion. Choosing this “sure target” (T_s; blue spot) led to a smaller reward (~80% of correct reward). On these trials the monkey could choose T_s or a direction choice. The two trial types were randomly interleaved. (B) The frequency of choosing T_s was greater when the motion strength (%coherence) was weak or the duration brief. The points are data grouped in duration quantiles (deciles). Error bars (SE) are smaller than the symbols. (C) Decision accuracy when T_s option was waived. The graph compares performance on trials in which T_s was not shown (open symbols, dashed curves) with trials in which T_s was offered but waived (filled symbols, solid curves).

We first describe behavioral observations, which demonstrate that the post-decision wager reflects choice certainty. We then demonstrate a neural correlate of this certainty in the LIP firing rate. Together, these observations support a mechanism in which the same decision variable, represented by LIP neurons, underlies both the choice and the degree of certainty in that choice.

The monkeys opted for the sure target when the chance of making a correct decision about motion direction was small. They exercised this option more frequently for the weaker motion strengths and for the shorter stimulus durations (p<10⁻⁸, Eq. 1, see (25); Fig. 1B), that is when the probability of making an error was higher (p<10⁻⁸, Eq. 2; Fig. 1C). More interestingly, when the monkeys waived this option, the choice accuracy was better than on the trials when they were not offered the post-decision wager (p<10⁻³, Eq. 3; Fig. 1C). This improvement was apparent at almost all motion strengths and stimulus durations. It implies that the monkeys did not choose the sure target on the basis of stimulus difficulty but instead based on a sense of uncertainty on each trial. This same pattern was observed on a subset of trials in which identical random dot patterns were repeated (p<0.025; Fig. S1), suggesting that the source of information about difficulty is not governed solely by properties of the stimulus but also by internal variability that renders the evidence more or less reliable to the decision-maker.

We recorded extracellularly from 70 LIP neurons while the monkeys performed this task. These neurons exhibited spatially selective persistent activity that predicted whether an eye movement was planned into the response field (RF) of the neuron on a memory guided saccade task(29–31). For the main motion task, we placed one of the direction targets (T_in) in the RF of the recorded neuron, the other direction target (T_opp) on the opposite side of the screen, and the sure target (T_s) orthogonal to the axis that connected the two direction targets.

Figure 2A shows responses of an example neuron for trials without the sure target. The neural activity following motion onset underwent a brief dip and then diverged to indicate the monkey’s decision for T_in or T_opp. The activity persisted through the delay period until the eye movement (18). For simplicity, the graph combines all motion strengths and stimulus durations (see Methods), but as shown previously (18–21), the buildup of firing rate reflected the stimulus strength (p=0.01, Eq. 10; Fig. S2), compatible with the representation of accumulated evidence in favor of T_in. We observed a similar divergence and persistence of activity for T_in and T_opp choices on the trials in which T_s was presented but was waived by the monkey (Fig. 2B, solid traces).

Figure 2.

LIP activity predicts direction choices and the post-decision wager. (A) Responses from one neuron on trials in which T_s was not presented. Average firing rates for T_in (black) and T_opp (gray) choices are shown for all correct choices (and the 0% coherent motion strength), during motion viewing and the delay period. Averages are aligned to motion onset (left part of graph) and saccade initiation (right). (B) Responses from the same neuron on trials in which T_s was presented. The dashed lines show neural activity on trials in which T_s was chosen (black and gray, motion toward T_in and T_opp, respectively). The middle portion of the graph shows activity in the delay period, aligned to onset of T_s. (C–D) Population average responses of 70 LIP neurons from two monkeys. Same conventions as in A and B. Firing rates from each neuron were normalized to the mean level prior to onset of the motion stimulus. (E) The activity before T_s presentation was smaller for T_s choices than for T_in choices. Each data point represents the mean activity of an LIP neuron in the 200 ms before T_s presentation (hatched rectangle in D). Error bars represent SEM. Shading in the histogram shows significant cases (p<0.05). The arrow shows the mean difference of normalized activity across the population (mean±SEM, −0.20±0.03). (F) The activity before T_s presentation was larger for T_s choices than for T_opp choices. Same conventions as E (mean difference=0.18±0.02).

In contrast, when the monkey chose T_s, the activity following the motion changed more gradually and achieved intermediate values compared to the T_in and T_opp choices. This pattern persisted into the delay period until T_s appeared (p<10⁻⁸, t-test). Note that before T_s appeared, the monkey did not know whether the sure bet would be offered. Following onset of T_s, there was a dip in activity, followed by a return to the level of activity preceding onset of T_s. When the monkey chose T_s the response gradually converged to the T_opp level. Importantly, the profile of activity suggests that even before the onset of T_s, the neuron was informative about whether the monkey would choose or waive this option should it be offered.

We observed a similar pattern of activity across the population of 70 LIP neurons (32). Intermediate firing rates during motion viewing and the early delay were associated with choosing the sure target later in the trial, as shown by the population average firing rates (Fig. 2D). To quantify this effect in single neurons, we compared activity in the 200 ms period before T_s onset (Fig. 2D, hatched box) for trials in which the monkey selected or waived the sure bet option. For motion toward T_in, the neural activity across the population was significantly smaller for T_s choices than for T_in choices (p=0.007, ANOVA; Fig. 2E). For motion toward T_opp, the activity was significantly larger for T_s choices than for T_opp choices (p=0.001; Fig. 2F).

These observations demonstrate that the monkey is more likely to opt for T_s when the LIP activity achieves an intermediate level of firing rate. However, a possible concern is that the intermediate level of activity represented by the mean firing rates from many trials is an unfair representation of the activity on single trials. According to this argument, the intermediate means might represent a mixture of the high and low firing rates that would have corresponded to T_in and T_opp choices, had the monkey indicated a direction choice on these trials. This ‘mixture of states’ alternative makes a clear prediction, which is not supported by the data. If the intermediate means were mixtures of the responses associated with T_in and T_opp choices, then the variance should reflect the dispersion of values associated with these extremes. This idea is rejected: The variance associated with T_s choices was significantly smaller than the variance associated with the mixtures of T_in and T_opp choices (p=4.7×10⁻⁵; F-test). We conclude that these intermediate levels of activity are not artifactual but represent a low state of certainty.

This conclusion is supported further by comparing the activity from neurons on single trials with the monkey’s decision to choose or waive the T_s option (Fig. 3A). For each trial, from each neuron, we calculated the deviation of firing rate, in the epoch just preceding T_s onset, from an intermediate level. The magnitude of this deviation was inversely related to the probability that the monkey chose the sure bet option (p=2.3×10⁻⁵, Eq. 11; Fig. 3A). The influence of a single neuron on the probability of a post-decision wager is expected to be small because it is but one member of a large population of neurons that govern the behavior, presumably (33–35). Nonetheless, the significance of the effect is a strong indication that LIP responses represent the choice certainty.

Figure 3.

T_s choices were correlated with trial-to-trial variation of neural activity. Responses from single trials were represented as the absolute deviation, in units of standard deviation, from the mean value using all the trials from a neuron (z-score). (A) The frequency of choosing T_s as a function of deviation from mean in the activity before T_s presentation. Curves are fits of Eq. 11 (25) to individual trials. The points illustrated on the graph were formed by grouping trials into 5 bins. (B) The frequency of choosing T_s as a function of deviations from the mean buildup rate of activity after motion onset. Same conventions as in A.

This single-trial analysis addresses another possible concern. Since stimulus difficulty (i.e., motion coherence and duration) affects both LIP responses and confidence judgments, it seems possible that the correlation between LIP activity and the post-decision wager is merely accidental, that is, totally explained by the stimulus difficulty. Alternatively, if T_s choices are based on LIP activity, they should be influenced by both the stimulus and the noisy fluctuations of LIP firing rates. To address this, we performed a variant of the single-trial analysis described in the previous paragraph. We calculated the trial-to-trial fluctuation of LIP responses relative to the mean response dictated by each motion strength and direction. These residual fluctuations before the sure target onset had significant leverage on the probability of choosing the sure target (p=4.0×10⁻⁵, Eq. 12). This finding also held for the subset of trials in which we used identical random-dot motion stimuli (p=0.015). Therefore, the linkage of neural responses with sure target choices is not explained merely by their shared covariation with the stimulus. We conclude from these analyses that the variable discharge of LIP activity was related to the monkey’s choice certainty, whether these variations were caused by experimental manipulations (i.e., motion strength and duration) or random effects (i.e., neural noise).

The single-trial analyses have focused thus far on neural activity in the delay period, immediately preceding the onset of T_s. Is the LIP activity during decision formation also related to choice certainty? The evolution of neural activity accompanying motion viewing suggests an affirmative answer. The rate of change of LIP activity after motion onset, termed the buildup rate (36), was related to the probability of choosing T_s later in the trial. For stronger stimuli, the buildup was steeper (p<10⁻⁸, Eq. 10), consistent with the accumulation of stronger evidence, shorter decision times, and ultimately more accurate decisions (13, 17). According to our hypothesis, the buildup rates should tend toward intermediate values when the monkey chose T_s. To test this, we performed a logistic regression analysis using buildup rates estimated from single trials. Deviation of the buildup rate from intermediate values was associated with a lower probability of choosing the sure target (p=0.017, Eq. 11; Fig. 3B). Moreover, this link was not simply due to covariation of buildup rates and choice certainty with motion strength (p=0.0018, Eq. 12).

It is also interesting to note that although the fluctuations in buildup rate and delay period activity were weakly correlated (r=0.10, p<10⁻⁸), each exerted independent leverage on the likelihood that the monkey would opt for the T_s wager (p<0.03; Eq. 13). In other words, both the evolution of decision-related activity and the sustained activity in the delay period carry information about choice certainty. While both quantities reflect the state of evidence, variation in the buildup rate also affects the amount of time it takes to reach a decision (19–21, 37, 38), consistent with the long held view that decision time contributes to choice certainty (8, 9, 12).

Indeed, a Bayesian framework that incorporates both evidence and decision time explains several aspects of the data. As previously shown, the left-right choices on this task are governed by the accumulation of evidence favoring one or the other option (17). This accumulation, which we call a decision variable, v(t), is represented by the firing rates of LIP neurons. It begins at a neutral value, and undergoes a random walk with drift (also termed drift diffusion) as evidence accumulates for and against the two direction alternatives. The decision terminates naturally when there is no more evidence (e.g., when the stimulus duration is short) or when v reaches a critical level or bound. In both cases, the choice is determined by the sign of v. As previously shown, this simple model explains the monkey’s accuracy as a function of stimulus strength and viewing time. It explains both the diminishing returns associated with prolonged viewing in our experiment (Fig. S3) (20) and the tradeoff between speed and accuracy in reaction time experiments (13, 39, 40). It also explains the saturating firing rate curves in Figure 2.

A simple extension of this bounded evidence accumulation model also explains the post-decision wagering. The key insight is that both v and t convey information about certainty. Figure 4A shows the distribution of v(t), combined across all stimulus strengths when the rewarded direction is, for example, rightward. Application of the decision rule described in the previous paragraph to v(t) would lead to different proportions of correct and incorrect choices, depending on the motion strength. This transformation is shown in Figure 4B, which replaces the probability distribution of v(t) with the log odds of a correct decision. This is the log-posterior odds based solely on v(t). For example, if a rightward stimulus is shown, the log odds of a correct choice is simply the log posterior odds that the stimulus is to the right

$$\underset{log\text{posterior}\text{odds}}{\underbrace{\mathit{Log}\frac{p(S_1\mid v(t))}{p(S_2\mid v(t))}}}=\underset{log\text{likelihood}\text{ratio}}{\underbrace{\mathit{Log}\frac{{\displaystyle \sum _i}p(v(t)\mid S_1,C_i)p(C_i\mid S_1)}{{\displaystyle \sum _i}p(v(t)\mid S_2,C_i)p(C_i\mid S_2)}}}+\underset{log\text{prior}\text{odds}}{\underbrace{\mathit{Log}\frac{p(S_1)}{p(S_2)}}}$$

where C is motion coherence, and S₁ and S₂ represent the rightward and leftward motion direction, respectively. The last term vanishes, because the prior probability that motion is left or right is equal. The summation terms implement marginalization over motion strength. The left side of the equation formalizes belief in the proposition S=S₁.

Figure 4.

A simple bounded evidence accumulation model predicted both the behavioral results and the modulation of LIP responses. (A–C) The model. On each trial, the accumulation, v(t), diffuses to one of the decision bounds (gray lines). The process terminates when v(t) reaches a bound or the stream of motion evidence ceases. (A) Representation of v(t) as a propagating probability density, for all motion strengths, when the rewarded direction is rightward. Positive values for v(t) represent accumulated evidence in favor of rightward. At time zero, the distribution is a delta function at v=0. As time elapses, the range of v(t) expands to fill the space between the two bounds, and there is a drift toward positive values, as shown by the probability density of v at t=800ms (inset to the right of color map). The distribution associated with leftward motion (not shown) is the mirror symmetric graph reflected about v=0. (B) The log odds of a correct response based on the value of v(t) at decision time. Correct responses are associated with larger v, but the relationship between v and probability correct changes with decision time. (C) T_s is chosen when the probability of a correct response is less than a criterion level. (D–E) Model fits and predictions. The three model parameters (Table S1) were fit to the observed frequency of correct responses on trials in which T_s was not shown and the observed frequency of T_s choices on trials in which T_s was shown. These parameters predict the probability of a correct response on trials in which T_s was waived (solid curves in E). (F) Comparison of model predictions and neural data. The average trajectory of v(t) in the model was calculated for different coherence levels using the fit parameters. The calculation is based on the stimulus durations used in the experiment, and assumes that v is fixed from termination of accumulation process. The calculated trajectories (top) resemble the LIP responses (bottom). Neural responses were detrended by subtracting the mean response at each moment and shifted by the neural latency (200 ms).

From the depiction in Figure 4B, it is easy to imagine that opting out of the direction decision might happen when the expected chance of success based on v(t) at decision time is less than a criterion level (Fig. 4C). This simple model explains the observed behavior and successfully predicts the amount of improvement in probability correct for trials in which the monkey waives T_s. The model has only three free parameters (Table S1), which were set by fitting the proportion of T_s choices and the probability correct for trials without T_s (dashed curves; Fig. 4D, R²=0.97; Fig. 4E, R²=0.98). This establishes a prediction (not a fit) for the probability correct on the trials in which T_s was shown but waived (Fig. 4E, solid curves, R²=0.97). The agreement between this simple model and the data affirms the plausibility of the “Bayesian sequential sampling” framework (41).

Moreover, the evolution of v(t), predicted by the model resembles qualitatively the responses of LIP neurons (Fig. 4F and S4). For stronger motion, the decision variable associated with T_s choices follows less intermediate trajectories (note separation of dashed curves), and the decision variable associated with direction choices rises (or falls) faster toward its plateau level. Both of these features are evident in the neural responses. The agreement is only approximate, presumably because neurons besides the ones we recorded contribute to the estimation of certainty (1, 22). These neurons might represent evidence for other directions of motion, but they are unlikely to represent the T_s choice option directly, as shown next.

To gain a better understanding of the representation of choice certainty across the population of LIP neurons, we recorded from 19 cells using the task configuration shown in Figure 5. T_s was in the RF whereas the direction-choice targets were not. Although T_s was not displayed until late in the delay period and only on half of trials, its position was fixed through the course of the experiment. Nevertheless, these neurons did not show a significant modulation of activity during the motion stimulus or in the ensuing delay period (42). Moreover, the weak activity that was present was uninformative about the choice to forego or choose the T_s option (Fig. 5B; p>0.1 for both directions of motion). Unlike the neurons with a direction choice target in the RF, the neurons that encode the location of T_s do not appear to represent choice certainty.

Figure 5.

Activity of LIP neurons when the location of T_s was in the RF. (A) Task design. For 19 neurons from two monkeys we placed T_s in the RF. The high-stakes direction targets were outside the RF. Task sequence was otherwise unchanged. (B) Responses on trials in which T_s was not offered. (C) Responses on trials in which T_s was presented. Firing rates were normalized to the visual activity in the 300 ms epoch following onset of T_s.

This observation argues against an alternative explanation of our finding based on allocation of attention to the T_s location. More generally, it provides additional evidence that the monkey made a decision about the motion direction in the period preceding onset of T_s, even on trials when he opted out of the direction task. There is no indication that the monkey approached the task as a choice between three alternatives, T_in, T_opp and T_s. However, after the appearance of T_s, these neurons with sure target in their RF became predictive of the post-decision wager. Although it is not obvious from the traces, the visual response in the first 200 ms was slightly larger when the monkey would choose T_s (p<0.01, ANOVA), suggesting that T_s was more salient when there was greater choice uncertainty (43). The effect was weak (median difference = 7.7%), but as time elapsed during the remainder of the delay period the firing rates gave a clear indication of whether the monkey would choose the T_s.

Discussion

A connection between signal reliability, choice accuracy and confidence has been proposed previously (1, 13, 14, 44, 45), but until now this connection has not been observed directly in the same neurons. Neurons in a variety of brain structures represent the size, preference and probability of obtaining a reward (15, 46–54), but it is not known how these representations arise. The present results show that the same neurons that participate in decision formation (20, 55) carry the relevant signals for assigning the probability of obtaining a reward. It therefore seems likely that the computation of choice certainty is passed, from LIP, to brain structures that anticipate reward, and it is likely that feedback from these structures affects LIP in the epoch after the appearance of T_s to mediate the decision to choose or forgo the T_s option.

The mechanism underlying the representation of certainty in LIP is linked to the same evidence accumulation that underlies choice and decision time (17, 20). This accumulation is encoded in the firing rates of LIP neurons with response fields aligned to the choice targets representing the direction alternatives (18, 55–57). This is the decision variable, v(t), that governs the choice of direction, having either attained a critical level — a decision termination bound —or by comparison to a criterion if the evidence stream ceases. This mechanism can be viewed as a merging of decision models based on sequential analysis (58–60) and signal detection theory (61). The magnitude of this decision variable, combined with knowledge of elapsed time, maps directly to the probability of obtaining a reward. An associative learning process based on LIP responses can therefore underlie the monkey’s choice of T_s. The ability to explain the rich pattern of behavioral results and the qualitative agreement between model and physiology favors the simple conceptual model. It is probably also consistent with other models that exploit a broader population of LIP neurons to encode posterior probability (22, 41).

This simple mechanism brings certainty, which is commonly conceived as a subjective aspect of decision making, under the same rubric as choice and RT (1, 62) and removes the need to resort to meta-cognitive explanations for certainty monitoring (45). Our findings support a low-level explanation of post-decision wagering in our task, but they do not preclude the possibility that an animal that experiences subjective awareness of degree of certainty might base such impressions on neural signals like the ones exposed here.

LIP neurons are hypothesized to encode the attentional salience or expected value of a visual saccade target (52, 53, 63), but these concepts cannot explain the pattern of LIP activity in our experiment. For example, a diversion of attention away from T_in to the potential location of T_s should have led to a reduction in firing rate for both T_in and T_opp directions during motion viewing and in the delay period before T_s appeared. Attention (or motor planning) might explain the activity just preceding saccades, but it does not explain the intermediate firing rates in the key epochs of interest. A second alternative, expected reward, seems more plausible at least to the extent that it mimics the belief that a choice will be correct. However the expected value of T_in, in the objective sense (from economics), changes as a function of motion strength (psychometric function; Fig. 1C), whereas the firing rate is minimally affected by motion strength when the monkey waives T_s (Fig. 4F and S4). Even subjective expected value, which is synonymous with certainty, fails to fully capture the deeper insight our experiment reveals about mechanism: the evolution of decision-related activity that gives rise to a choice also underlies certainty and a wager based upon it.

A famous controversy in the history of probability theory concerned whether it is meaningful to embrace the truth of a hypothesis as a graded quantity expressed as a probability, or whether instead hypotheses are simply true or false. The latter approach led frequentists to reject the Bayesian concept of degree of belief, relegating probability to the analysis of error rates in truth-assertions (64, 65). Our finding suggests that when the brain embraces a truth, it does so in a graded way so that even a binary choice leaves in its wake a quantity that represents degree of belief. From this perspective, our neural recordings support the idea of a “Bayesian brain” (66) and a neural mechanism of decision making that does not flip into a fixed point or attractor state but instead approximates the formation of a probability distribution (41, 67). Accordingly, the intermediate levels of activity associated with less certain choices might be a sign of a more homogeneous level of activity across the population of neurons. Fundamentally, our results advance understanding of the neural mechanisms that underlie decision-making by coupling for the first time the mechanisms leading to decision formation and the establishment of a degree of confidence.