Confidence in value-based choice

Summary

Decisions are never perfect with confidence in one’s choices fluctuating over time. How subjective confidence and valuation of choice options interact at the level of brain and behavior is unknown. Using a dynamic model of the decision process we show that confidence reflects the evolution of a decision variable over time, explaining the observed relation between confidence, value, accuracy and reaction time. As predicted by our dynamic model, we show that an fMRI signal in human ventromedial prefrontal cortex (vmPFC) reflects both value comparison and confidence in the value comparison process. Crucially, individuals varied in how they related confidence to accuracy, allowing us to show that this introspective ability is predicted by a measure of functional connectivity between vmPFC and rostrolateral prefrontal cortex (rlPFC). Our findings provide a mechanistic link between noise in value comparison and metacognitive awareness of choice, enabling us both to want and to express knowledge of what we want.

Introduction

The subjective confidence we have in our decision-making, and that of others, has far-reaching consequences. For example, the recommendations of a financial advisor who expresses high confidence in a particular investment option will carry more weight than one who is ambivalent. Conversely, expressing doubt, or caution in a particular course of action can lead one to question or revisit a previous decision. Prior work has established that the ventromedial prefrontal cortex (vmPFC) plays a central role in computing the value of potential choice options^1-5, with activity in this region reflecting the dynamic evolution of a value comparison⁶. However, this work has focused exclusively on the choice process, without considering the subject’s level of confidence in their decision. Consequently, it is unknown how a process of value comparison, instantiated in vmPFC, relates to subjective confidence.

Previous studies have reported neural correlates of decision confidence in brain regions associated with a value representation. For example, firing rates in rat orbitofrontal cortex⁷ and fMRI signal in human vmPFC⁸ show graded changes as perceptual decisions become more difficult. However, as these studies delineated confidence in terms of factors governing choice, they are unable to tease apart the relationship between trial-to-trial subjective confidence and decision value. In contrast, the field of perceptual decision-making has highlighted the fact that confidence can be measured independently of the choice process itself^9,10, where it is conceptualized as reflecting a “second-order” metacognitive evaluation. Critically, dissociating confidence from other features of the decision process requires acquisition of separate measures of choice and confidence¹¹.

Here we implement such an approach to dissociate value and confidence during decision-making, and to identify their respective neural substrates. We collected trial-by-trial estimates of decision confidence while healthy volunteers chose between pairs of snack items. We additionally measured the subjective value of each snack item via a standard incentive compatible bidding procedure (BDM; see Methods). This allowed us to dissociate confidence from value, and in so doing provide evidence that confidence reflects an online assessment of choice accuracy.

To explore systematic relationships between confidence, accuracy, choice and reaction time, we modeled our data using a variant of a race model^7,12 (part of a larger class of dynamic models of decision-making¹³). This model predicts that subjective confidence reflects the stochastic accumulation of evidence during the value comparison process. Consistent with this prediction we show that the same anatomical region in ventromedial prefrontal cortex (vmPFC) not only reflects a difference in value between available options, but also the confidence associated with a value comparison process. Finally, we show that individual differences in participants’ ability to relate confidence to decision performance is linked to increased functional connectivity between vmPFC and rostrolateral prefrontal cortex (RLPFC), a region previously shown to play a role in metacognitive appraisal¹⁴.

Results

We scanned twenty hungry participants while they made choices between food items that they could consume later (Figure 1a). After making each choice participants reported the degree of confidence in their decision (choice confidence). Note that confidence, or certainty, in the present study is conceptually distinct from risk, in that each choice determines a known outcome. Confidence here reflects the degree of subjective certainty in having made the best choice, which equates to choosing the higher valued item. To establish value for individual items we asked participants at the end of the scanning session to place a bid for each food item using a standard incentive compatible procedure, the Becker–DeGroot–Marschak (BDM) mechanism¹⁵. BDM is widely used in behavioural economics and neuroeconomics to elicit non-strategic reservation prices also known as willingness-to-pay (WTP). In this phase subjects were required to state their maximum willingness-to-pay for each food item (see Methods for details). A number of studies have shown that this mechanism reliably elicits goal values that are used by the decision maker to guide choice^16-18. Participants additionally provided a rating of their confidence in each bid (bid confidence). Participants’ bids were then used to calculate a signed difference in value (DV) between each pair of items (V_right – V_left), which was then entered into a logistic regression to predict the probability that the subject chose the rightmost item on each trial (Figure 1 b – dotted line). In line with previous studies we show DV is a reliable predictor of participants’ choices^2,19 with the slope of the logistic regression being a measure of choice accuracy, or noise in the choice process²⁰.

Figure 1. Task and behavioral results.

(a) fMRI task (red box): Subjects were presented with a choice between 2 confectionary items and were then required to choose (2.5 sec) one item to consume at the end of the experiment. After each choice, subjects indicated their level of confidence in having made a correct decision (choice confidence). Post-scanning task: Subjects were presented with each item individually and had to submit a bid to buy each item. After each bid, they were asked to rate their level of confidence in having provided a correct bid price (bid confidence). (b) Probability of choosing the item on the right as a function of the difference in value (i.e. bid price) between the 2 items (logistic fit) for an exemplar subject (see Figure S2 for all individual subjects). Dotted line = all choices; black line = low confidence choices; grey line = high confidence choices. The red arrow indicates the increase in choice accuracy (change in slope) for high versus low confidence trials used in the between subject analyses (Figure 4b and 5b) (c) The slope of the logistic fit is systematically higher (sharper) in high compared to low confidence trials (p<0.0001). (d) Average choice reaction time data as a function of confidence and ∣DV∣. (e) Heatmap showing mean z-scored confidence (colorbar) across subjects, as a function of subject-specific ∣DV∣ and RT quantiles. Error bars represent the standard error of the mean (s.e.m.).

Choice, confidence and reaction time

Importantly, we observed that unsigned ∣DV∣ only accounted for an average of 17.7% of the variance in participants’ confidence ratings (r =0.42 ±0.19 SD). This partial independence between confidence and ∣DV∣ allowed us to ask whether confidence reflects changes in choice accuracy (the selection of items with higher subjective value). By splitting our logistic regression fit into high and low confidence trials, we showed that higher confidence was consistently associated with increased choice accuracy (Figure 1 b, c and Supplementary Figure 1). This effect of confidence on choice was also reflected in reaction time (RT), with main effects of both ∣DV∣ and confidence (both P < 0.001), but no interaction (Figure 1d). The three-way relationship between ∣DV∣, confidence and RT is plotted in Figure 1e. We recognize that aside from ∣DV∣ and RT other factors (internal and external) are likely to affect subjective confidence. Here we report a limited set of these factors (see Supplementary Table 1) for which we could exercise good experimental control.

Using logistic regression, we next compared models of the interaction between confidence and value comparison. Note that choice confidence, unlike DV, is in itself not a predictor of choice (right or left item), but instead refers to accuracy of the decision. We thus expected choice confidence to modulate the link between DV and choice. Model 1 predicted choice using DV alone; Model 2 included choice confidence (i.e. confidence at the decision time) as a modulator of DV (DV*confidence); Models 3-5 examined whether bid confidence (i.e. confidence at the bid time) could explain additional variance in the link between DV and choice (see Methods). In accordance with our predictions, Model 2 provided a significantly better account (i.e. lower Bayesian information criterion – BIC) of participants’ choices than the other four models (Figure 2a) as shown by the difference in BIC relative to Model 2: Model 1, 214.6; Model 3, 196.2; Model 4, 251.7; Model 5, 111.9. Furthermore Model 2 was a better fit than the canonical Model 1 in 19 out of 20 participants as assessed via a likelihood ratio test (α = 0.05). This analysis confirms that a critical modulator of choice accuracy is 2^nd level confidence arising in the context of the comparison process (Model 2) as opposed to 1^st level confidence in the item values (Models 3-5).

Figure 2. Computational model.

(a) Comparison of regression models. Plotted are Bayesian Information Criterion (BIC) scores (Model 1: 5424; Model 2: 4995; Model 3: 5388; Model 4: 5498; Model 5 5291). Smaller numbers indicate a better model fit. See text for details of each model. (b) Dynamic (race) model of value comparison. Evidence in favor of each option accumulates over time, with a choice in favor of one or other option being made when threshold is reached. In this model decision confidence is derived from the absolute difference between the two accumulators at the time of the decision (Δe). (c-e) Model predictions. (c) When Δe is large (i.e. high confidence) choice accuracy is predicted to increase, reflected by a sharper curve in the logistic regression. (d) Reaction times are predicted to decrease when either ∣DV∣ or Δe increase. (e) Matrix representing how model confidence changes across ∣DV∣ and RT quantiles. Note the close similarity between the model predictions and behavior (Figure 1c-e).

Stability of confidence over time

We next examined whether the relationship between confidence and choice is stable over time. Splitting the logistic regression analysis into separate sessions revealed a robust main effect of confidence (F_(1,19) = 39.75; P < 0.0001) but a non-significant main effect of session (F_(3,57) = 0.3; P = 0.7) and a lack of interaction between session and confidence (F_(3,57) = 0.13; P = 0.9; Supplementary Figure 2). To examine whether local fluctuations in attention affected confidence, we constructed a serial autocorrelation regression model that predicted the current confidence rating from the confidence ratings given on the immediately preceding 5 trials, in addition to ∣DV∣. None of the autocorrelation coefficients reached group-level significance (all t < 1.2, P > 0.27). Together these results indicate that confidence is a stable predictor of choice accuracy, and does not reflect local changes in attention.

As each item pairing was presented twice (once in each spatial configuration) it was also possible to examine the relationship between confidence ratings given for identical choice pairs. As confidence is partly determined by absolute difference in value (∣DV∣, which does not vary across choice pairs) we expected some stability purely driven by DV. Thus to address this question we computed the partial correlation between 1^st and 2^nd confidence ratings, controlling for DV. There was no significant difference between mean confidence ratings for the first and second presentations of the same item pairs (t(19) = −0.64, P = 0.53). For 19 out of 20 subjects, there was a significant partial correlation (P < 0.05) between confidence ratings for repeated item pairs after controlling for the influence of ∣DV∣, indicating stability in confidence for judgments of particular item pairs that cannot be accounted for by ∣DV∣ alone.

We additionally examined whether choices were stable over time. On average, 14.7 % of choices (± 5.7 % SD) were reversed on the second presentation. Choices that would be subsequently reversed were associated with significantly lower initial confidence than those that would subsequently be repeated (reversal confidence (a.u.) = 210.6 ± 72.4 SD; repetition confidence = 340.2 ± 53.5 SD; t₍₁₉₎ = 12.1, P < 10⁻¹⁰). In a logistic regression model predicting subsequent reversal from both ∣DV∣ and initial confidence, initial confidence was a significant negative predictor of choice reversal (mean standardised regression coefficient −0.0083 ± 0.0034 SD; one-sample t-test t₍₁₉₎ = −10.9, P < 10⁻⁹). These data support a hypothesis that low confidence may be associated with subsequent changes of mind.

Race model

Our best-fitting regression model suggested that confidence reflects accuracy in a value comparison. This led us to explore in more detail the precise mechanism by which confidence and value interact during the decision process. We adapted a race model^7,21 wherein evidence in favor of each option (the snacks presented on the left and right side of the screen) is accumulated over time and the decision is made on the basis of the first option to reach a threshold (Figure 2b). In this model confidence is defined as the absolute difference between the two accumulators at decision time (Δe). Such a model predicts that when Δe is large then choice accuracy is increased, reflected by a sharper slope in the logistic regression (Figure 2c). Thus, the race model neatly accounts for an increase in choice accuracy we observe behaviourally in the high confidence condition (Figure 1b; Supplementary Table 2). Furthermore this model predicts a decrease in RT when either ∣DV∣ or Δe are increased (Figure 2d), as seen in the behavioural data (Figure 1d). The intuition is that, even within a particular level of initial DV, inter-trial noise in the value comparison process results in some trials having greater final DV’s (higher confidence) than others. Such decisions will tend to be made more quickly, be more accurate, and be associated with higher confidence (Figure 2e). Indeed, this predicted inter-relationship between RT, ∣DV∣ and confidence closely matches what is observed in the behavioural data (Figure 1e). Finally, since the model predicts that confidence reflects the stochastic evolution of a value comparison process, it will only be weakly related to initial DV. This feature of the model provides a parsimonious explanation for why DV and confidence are dissociable in our behavioural data.

Confidence and value in vmPFC

We next hypothesized that if choice confidence is an emergent property of a value comparison process, the same brain regions involved in value-based decision-making should also represent subjective confidence in a value estimate. In other words, if a brain region involved in value comparison is implementing a process akin to a race model⁶, then activity here should be modulated by both initial ∣DV∣ and noise (confidence) on that trial. To test this hypothesis we constructed a general linear model (GLM) of our fMRI data in which each trial was modulated by two parametric regressors: ∣DV∣ and confidence orthogonalised with respect to ∣DV∣. We show that activity in vmPFC is indeed modulated by both value and confidence (Figure 3a, b and Supplementary Table 3; [12, 47, −11], p<0.05 family wise error (FWE) corrected at cluster level). This pattern is consistent with the established role of this region in encoding goal-values^1,2 and with our novel hypothesis that this region also represents the confidence associated with a value comparison.

Figure 3. vmPFC.

(a) Brain activity in precuneus and vmPFC [MNI space coordinates (x, y, z) 12, 56, 4] correlating with increases in difference in value between the two items presented (p<0.05 family wise error (FWE) corrected at cluster level). (b) Brain activity in precuneus and vmPFC [12, 47, −11] correlating with increases in subjective confidence (p<0.05 FWE corrected at the cluster level). (c) Signal in vmPFC [6 mm sphere centered at 12, 56, 4] showing significant main effects of difference in value and level of confidence in the absence of an interaction. Note that the bar plot (extracted from GLM 2 – see Methods) is shown only to clarify the signal pattern in vmPFC (i.e. lack of interaction between confidence and DV) and to confirm statistical inference (from GLM 1) regarding the main effects of DV and confidence. Error bars represent the s.e.m.

We next investigated whether ∣DV∣ and confidence interacted in vmPFC by splitting the model into high and low confidence trials, both parametrically modulated by ∣DV∣ (Figure 2c). This analysis showed main effects of ∣DV∣ and confidence in vmPFC, but importantly no interaction (2×2 ANOVA with factors value, confidence: main effect of value: F_(1,19) = 5.1, p<0.05; main effect of confidence: F_(1,19) = 7.6, p<0.05; interaction: F_(1,19) = 0.7, p>0.5) (Figure 3c). The absence of an interaction at the neural level is consistent with a theoretical independence between value and noise in the choice process, such that one can have high confidence in a low value choice, and vice-versa. Furthermore, the pattern across conditions closely resembles that seen for RTs (Figure 1d) providing convergent evidence that vmPFC activity is tightly linked to behavior. Note that we additionally tested and confirmed that the response to confidence is not driven by a categorical response to errors⁸ (see Supplementary Figure 3).

Confidence in rRLPFC

A key question is how confidence-related information represented in vmPFC becomes available for self-report. One computationally plausible hypothesis is an hierarchical model where confidence in a comparison process is “read out” by an anatomically distinct second-order network^22-24. Right rostrolateral prefrontal cortex (rRLPFC) is a likely candidate as this region is widely implicated in metacognitive assessments of perceptual decisions^9,14,25.Consequently, we tested whether this region plays a more general role in metacognitive appraisal by enabling explicit report of confidence in a value comparison.

We first established that rRLPFC tracks changes in reported confidence, but does not code for difference in value (Figure 4a, b, Supplementary Figure 4 and Supplementary Table 3, [39, 41, 16], p<0.005 SVC), as expected for a region providing a read-out of decision confidence. We next harnessed individual differences in metacognition to provide a more stringent test for the role of rRLPFC. We defined an individual’s metacognitive accuracy as the change in choice accuracy (slope of the logistic fit) between low and high confidence trials (Figure 1b). We reasoned that if rRLPFC plays a role in the metacognitive appraisal of confidence, activity in this region and/or its coupling with vmPFC should predict this change in slope across individuals. To test our first prediction, we entered change in slope as a between-subjects covariate in the whole-brain analysis of confidence-related activity, finding that this parameter significantly modulated the response to confidence in rRLPFC (p<0.05; SVC for multiple comparisons). In other words, participants manifest a neurometric-psycometric match between their behavioral and neural responses to change in confidence level (Figure 4c).

Figure 4. RLPFC.

(a) Brain activity in rRLPFC correlating with decreases in subjective confidence (p<0.005 small volume FWE corrected). (b) Signal in rRLPFC [6 mm sphere MNI space coordinates (x, y, z) 39, 41, 16] showing a main effect of confidence but not difference in value. Note that the bar plot (extracted from GLM2 – see Methods) is shown only to clarify the signal pattern in rRLPFC (i.e. absence of main effect of DV). (c) Between-subject regression analysis entering the change in choice accuracy (slope of the logistic fit) between low and high confidence trials (see red arrow in Figure 1b) as a covariate for confidence-related activity in rRLPFC [peak (x, y, z) 27, 44, 16; p<0.05 small volume FWE corrected]. Note that the scatter-plot is not used for statistical inference (which was carried out in the SPM framework), and is shown solely for illustrative purposes. Error bars represent the s.e.m.

Metacognitive access: functional interaction of vmPFC and rRLPFC

To test our second prediction, that these two regions are part of the same functional network (in the context of our task), we performed a psychophysiological interaction (PPI) analysis using rRLPFC as a seed (Figure 5 a – in blue). This analysis revealed a robust modulation of connectivity between rRLPFC and vmPFC [peak (x, y, z) 9, 50, −11; z = 3.05; p<0.05 small volume FWE corrected] by confidence level (Figure 5 a, b). Furthermore, the strength of connectivity between these two regions also predicted metacognitive accuracy across subjects (vmPFC 15, 56, −5; p<0.05; SVC for multiple comparisons) (Figure 5b). Thus, both the level of activity in rRLPFC itself and its coupling strength with vmPFC influences the degree to which confidence is effectively “read out” for metacognitive report.

Figure 5. Connectivity analysis.

(a) PPI analysis: vmPFC (circled in black) shows increases in connectivity with a region of rlPFC [6 mm sphere (x, y, z) 39, 41, 16] (in blue) previously identified as being modulated by confidence [vmPFC peak (x, y, z) 9, 50, −11; z = 3.05; p<0.05 small volume FWE corrected]. (b) Between-subject regression analysis entering the increase in choice accuracy (see red arrow in Figure 1b) between high confidence and low confidence conditions as a covariate for the modulation of connectivity [vmPFC peak (x, y, z) 15, 56, −5; z = 3.91; p<0.05 small volume FWE corrected]. Note that the scatter-plot is not used for statistical inference (which was carried out in the SPM framework), and is shown solely for illustrative purposes. Error bars represent the s.e.m.

How might this read-out process relate to our computational model of confidence? Intuitively, if reported confidence is a noisy facsimile of the confidence inherent in a decision process, the relationship between confidence and behavior will weaken, and metacognitive accuracy will decrease²⁶. We were able to modify the race model, introduced previously, to account for the inter-subject variability in metacognitive reports observed experimentally. We introduced an additional parameter (σ_conf) governing the noise in the read-out of Δe (i.e. decision confidence) computed during the value comparison. Variation in this parameter captured variability in the change in slope between high and low confidence conditions, despite overall choice accuracy remaining equal (Figure 6b). Together with our imaging results, this analysis suggests that rRLPFC may indeed mediate variability in reported confidence (see Figure 6 and Discussion).

Figure 6. Schematic of network relating confidence to subjective report.

Summary of the relationship between our computational model and neuroimaging analyses. (a) Confidence in the decision (Δe) emerges from the value comparison process instantiated in vmPFC. (b) In order to reach metacognitive awareness (and be reported by the participant) this information is transferred to rRLPFC. An additional parameter (σ_conf) governs the noise in the read-out of Δe (i.e. decision confidence). If σ_conf is zero the information about confidence (Δe) is uncorrupted, resulting in a pronounced shift in the choice accuracy between high confidence and low confidence trials (red arrow). As the level of metacognitive noise increases (more positive values of σ_conf ) the shift between the two curves (low and high confidence) diminishes. Differences in σ_conf account for the inter-subject variability in metacognitive reportability we observed behaviorally.

Discussion

Here we show that decision confidence emerges from a value-comparison process in vmPFC, and that this region is in turn accessed by rRLPFC to enable a subjective assessment of confidence. Our neural findings are consistent with previous evidence showing that choice difficulty is coded by vmPFC in humans and analogous OFC neurons in rodents^7,8. There is also an established body of work showing that this brain area represents the expected value of an outcome^1-6. However, as previous studies defined confidence in terms of factors governing choice, they were unable to tease apart the relationship between value and confidence. Our results go beyond these studies by dissociating subjective confidence from a difference in value (DV). In so doing we demonstrate that neural activity in the same anatomical region represents both variables, suggesting that confidence and DV are separate behavioural manifestations of the same underlying decision variable.

Choice confidence can be seen to emerge from the dynamics of noisy accumulators in the race model^7,21, leading to dual effects of difference in value and reaction time on confidence²⁷. The race model has previously been proposed to account for decision confidence in perceptual decision-making. In keeping with recent research efforts that have incorporated dynamic models into the field of economic decision-making²⁸, we find that this model captures several features of the relationship between choice, reaction time and confidence in a value-based choice paradigm. The separation between confidence and BDM values in the present study provides a novel perspective on how an underlying decision variable can be fractionated into distinct behavioural components. Given that both difference in value and confidence had independent effects on vmPFC activity, this result provides convergent support for the idea that vmPFC acts as a dynamic accumulator of choice values⁶. Our findings also accord with a theoretical Bayesian scheme in which uncertainty, or precision, is an inherent property of the neural code^29-31.

A central problem for computational models of metacognition is how confidence information is “read out” for appraisal and communication to others. Insabato et al.¹⁸ proposed that such a computation can be achieved by a two-layer neural network architecture, in which the second-order network receives information about the performance of the first-order network, and uses this information to generate reports of confidence (see also ²⁴). Our fMRI data can be interpreted in this framework and suggests that rRLPFC is a plausible locus for this second-order network. First, rRLPFC represented confidence, but not DV, as predicted for a brain region that has access to information about confidence but is not directly involved in value comparison. Second, both confidence-related activity in rRLPFC and coupling between rRLPFC and vmPFC predicted the relationship between confidence and accuracy across individuals. This result can be explained if the coupling between vmPFC and rRLPFC reflects the fidelity with which reported confidence tracks the evolution of a putative accumulator process in vmPFC (Figure 2b; Figure 6). Notably, confidence-related activity in rRLPFC is also seen in perceptual decision-making¹⁴, together with a modulation of connectivity with visual cortex. This pattern of findings suggests that rRLPFC might play a domain-general role in metacognitive evaluation of decision-making, supporting the notion of segregated neural process governing metacognitive access^22,24,26.

An alternative interpretation of our data is that information about choice confidence is coded elsewhere, perhaps in parallel to the construction of choice values, and is then communicated to vmPFC (possibly via rRLPFC) where it is incorporated into the choice process. This mechanism would be analogous to a modulation of the vmPFC value signal during self-control by dlPFC³¹. Resolving this possibility is beyond the design of the current study and will require other techniques with high temporal resolution, such as MEG, that can track the evolution of confidence and valuation in the brain.

Our data show that humans have metacognitive access to noise in a value comparison, and that increased choice accuracy is associated with high subjective confidence. In other words, while choices often appear noisy from the point of view of the experimenter^20,33, subjective confidence ratings reveal systematic changes in this noise, reflected by changes in choice accuracy. Metacognitive access to confidence in a value comparison is likely to be useful for revisiting a choice that did not turn out as expected. Alternatively, but not mutually exclusively, metacognitive access may facilitate communication of confidence to others³⁴, as when a financial advisor directs a client towards one stock option over another.

By integrating computational modeling with neural analysis, we provide evidence that subjective confidence is integral to the brain’s representation of value in the vmPFC. Our work outlines a novel neural schema for how confidence-related information is computed and transferred to a distinct brain region (rRLPFC), supporting metacognitive report. Far from being a blind process of selection corrupted by noise, it would appear that value-based choices are accompanied by fluctuations in subjective confidence. A metacognitive access to value computation enables us not only to want, but also to know what we want.

Online Methods

Participants

28 participants (mean age 24.24) took part in the study. 4 participants were excluded because of excessive head motion. 3 participants were excluded for erratic choice patterns that prevented reliable estimation of a logistic fit (an inverse temperature parameter 5 or more times larger than the average of the group). Participants were only included if they used a sufficient range of confidence ratings (standard deviation > 0.8) to allow estimation of metacognitive ability. This criterion led to the exclusion of one further subject. 20 participants were included in the final analysis.

Scanning task

Participants were required to fast for 4 hours before the study. During scanning they were required to make a series of binary choices between 19 common confectionary items (2000 ms) to consume later (see Supplementary Table 4 for a list of items). Participants were asked to choose between each combination of items (n=170) twice, counterbalanced across left-right spatial configurations (total number of choices = 340) and divided into 4 sessions. After each choice, participants were asked to indicate their confidence in their decision (i.e. “How confident are you that the choice you made was the right one for you?”) on a continuous sliding scale between 1 (low confidence) and 6 (high confidence). Participants had 3500ms to move the pointer to the position that accurately reflected their confidence in the previous decision.

Post-scanning BDM task

Participants were presented each item on a computer screen and asked to submit a bid (from £0 to £3 using a sliding scale) to buy the item (unlimited time). After each bid participants were asked to indicate their confidence in the bid they had just submitted (i.e. “How confident are you that the bid you made was the right one for you?”; bid confidence) on a continuous sliding scale between 1 (low confidence) and 6 (high confidence). At the end of the experiment one choice from the scanning phase was played out and the subject had the opportunity to buy the chosen item by means of an auction administered according to the Becker Degroot Marschak (BDM) procedure¹⁵. More specifically, the experimenter randomly extracted a price from a uniform distribution [£0 to £3] – the “market price” of that item. If the participant’s bidding price (WTP) was above the market price no transaction occurred. If the subject’s bidding price was below the market price the participant bought the snack item at the market price. At the end of the experiment participants had to remain in the lab for an additional hour. During this hour, the only food they were allowed to consume was the item purchased in the auction, if any. This procedure encouraged subjects to choose preferred snacks during the scanning phase^16,18. Participants were compensated £40 for participation in the study. The price of any item purchased by a subject was deducted from this £40 participation fee.

Behavioural analysis and model

To examine the effect of value and confidence on choice we compared five candidate logistic regression models. All had the form:

$$P(c=R\mid X)=\Lambda \left(X\beta \right)$$

where Λ(x)is the logistic cumulative distribution function:

$$\begin{array}{c}\hfill \Lambda (x;u,s)=\frac{1}{1+e^{-(x-u)∕s}}\hfill \\ \hfill P(c=R\mid X)=\frac{1}{1+e^{-a+b_1{x}_1+b_2{x}_2+\dots b_n{x}_n}}\hfill \\ \hfill \log \frac{P(c=R)}{1-P(c=R)}=a+b_1{x}_1+b_2{x}_2+..b_n{x}_n\hfill \end{array}$$

The simplest candidate model predicts the probability of choosing the rightmost option from the signed difference in value (DV), defined as V_R − V_L:

$$X=\mathit{DV}$$ (Model 1)

The slope of this function is assumed to result from randomness in choice²⁰. If, on the other hand, subjects have metacognitive access to the noise in their decision process, we might expect choice confidence to modulate the impact of DV on choice:

$$X=[\mathit{DV}\mathit{DV}\ast {\mathit{conf}}_{\mathit{choice}}]$$ (Model 2)

A second set of models examined whether confidence in the item price (bid confidence) modulates the link between DV and choice. On each trial there were two bid confidences (one for each item). Model 3 modulated DV by the mean bid confidence to enable direct comparison with Model 2; Model 4 split the DV predictor by item confidence (low, high and mixed low/high, based on a subject-specific median split); Model 5 extended Model 4 by including additional regressors for the modulation of choice confidence (i.e. Model 2 split by different bid confidences):

$$X=[\mathit{DV}\mathit{DV}\ast {\overline{\mathit{conf}}}_{\mathit{bdm}}]$$ (Model 3)

$$X=\left[{\mathit{DV}}_{\mathit{low}}{\mathit{DV}}_{\mathit{mixed}}{\mathit{DV}}_{\mathit{high}}\right]$$ (Model 4)

$$\begin{array}{cc}\hfill X=[{\mathit{DV}}_{\mathit{low}}\mathit{DV}& {}_{\mathit{low}}\ast {\mathit{conf}}_{\mathit{choice}}{\mathit{DV}}_{\mathit{mixed}}{\mathit{DV}}_{\mathit{mixed}}\ast {\mathit{conf}}_{\mathit{choice}}{\mathit{DV}}_{\mathit{high}}{\mathit{DV}}_{\mathit{high}}\hfill \\ \hfill & \ast {\mathit{conf}}_{\mathit{choice}}]\hfill \end{array}$$ (Model 5)

Models were compared via BIC scores using a fixed-effects analysis, where a difference in BIC of 5 indicates strong evidence for one model over another³⁵. Two subjects were excluded from this analysis (i.e. Model 4 and Model 5) due to a low variability in item confidence precluding a median split. In addition, we assessed the improvement in model fit obtained for Model 2 over nested Model 1 for each subject individually using a likelihood ratio test (χ², 1 d.f.).

Dynamic model of value comparison (race model)

To predict how value, confidence and reaction time interact during decision-making, we harnessed a dynamic model of the value comparison process^7,21. In the race model, separate decision variables accumulate evidence for distinct options, with the final decision determined by which accumulator reaches threshold first. On each time step during accumulation, a new evidence sample is drawn from a normally distributed random variable $s_t=\mathcal{N}({\mu }_{\mathit{stim}},{\sigma }_{\mathit{stim}}).u_{\mathit{stim}}$. u_stim is positive if the correct choice (higher value item) the right item; negative if the correct choice is the left item. Due to s_t being drawn from a normal distribution, the actual value of s_t at each time step may be positive or negative. The accumulators evolve according to the following equations:

$$\begin{array}{cc}\hfill R_{t+1}& =\{\begin{array}{c}\hfill R_t+s_t,s_t>0\hfill \\ \hfill R_t,s_t\le 0\hfill \end{array}\phantom{\}}\hfill \\ \hfill L_{t+1}& =\{\begin{array}{c}\hfill L_t,s_t\ge 0\hfill \\ \hfill L_t-s_t,s_t<0\hfill \end{array}\phantom{\}}\hfill \end{array}$$

The race terminates when either R_t or L_t reach a predetermined threshold, θ, with the decision being determined by which accumulator reaches threshold first. Therefore at decision time, t(θ), either R_t or L_t = θ. The finishing point of the losing accumulator depends on the values of u_stim and σ_stim.

An estimate of decision confidence, Δe, can be recovered from the race model as the distance between the two accumulators R_t and L_t at the time the race is terminated (Figure 2b; ^7,12).

We simulated the model using the same parameters of ⁷. We simulated 1000 trials at each level of u_stim, and recorded mean choice, confidence and reaction time. We display the simulation output in an identical manner to the behavioural data (Figure 2c-e).

Even for identical levels of decision performance, it is known that the relationship between subjective confidence and decision-making varies between tasks and individuals^9,36. We sought to account for this variability by introducing an additional parameter relating model confidence to subjective confidence, σ_conf. On each trial, reported confidence was drawn from a Gaussian distribution centred on $\Delta \mathrm{e}:\text{conf}=\mathcal{N}(\Delta e,{\sigma }_{\mathit{conf}})$. This feature of our model is consistent with the notion that reported confidence is derived from a higher-order stage of decision-making corrupted by noise ²⁶. We note that other functional forms for the link between model confidence and reported confidence are possible, but we do not investigate these here. We repeated the simulation three times with three levels of σ_conf, for a fixed σ_stim. Examination of psychometric function plots (Figure 6b) shows that σ_conf can account for the variability in change in slope observed across individuals.

Image Acquisition and Analysis

Scanning acquisition was performed using a Siemens 3.0 Telsa Allegra MRI Scanner (Erlangen, Germany). Gradient echo T2* weighted EPI (echoplanner) functional images with BOLD-sensitive contrast were acquired (imaging parameters: 48 transverse slices; TR, 2.88 seconds; TE, 30ms; 3 × 3 in-plane resolution; 2mm slice thickness; 1mm gap between adjacent slices; z-shim −0.4 mT/m; positive phase encoding direction; slice tilt −30°), optimized to detect changes in orbitofrontal cortex. 228 volumes per session were collected for each subject (total number of volumes over 4 sessions = 912) followed by a whole brain high-resolution T1-weighted anatomical structural scan and local field maps. Image analysis was performed using SPM8 (www.fil.ion.ucl.ac.uk/spm). The first 5 volumes from each session were discarded to allow for T1 equilibration. Raw functional, structural and field map files were reconstructed using TBR. Field maps were reconstructed into a single phase file. This field map file was then used to realign and unwarp EPI functional images. Structural images were reregistered to mean EPI images and segmented into grey and white matter. These segmentation parameters were then used to normalise and bias correct the functional images. Normalized images were smoothed using a Gaussian kernel of 8mm full-width at half-maximum.

General Linear Model (GLM) 1 (parametric)

Onset regressors beginning at the presentation of the 2 items were modulated by 2 parametric regressors: (1) unsigned difference in value (∣DV∣) defined as the absolute difference in value between the item presented on the right (V₁) and the item presented on the left (V₂) with values V₁ and V₂ ascertained from subjects’ bids in the post-scanning phase (∣DV∣=∣V₁-V₂∣); (2) post-choice confidence ratings which ranged from 0 to 500 on an arbitrary scale. In this model, confidence is orthogonalised with respect to ∣DV∣ by the SPM8 software. Second, for each subject we constructed a separate general linear model using a factorial design. GLM 2 (factorial): Events were split into regressors based on confidence level (i.e. low and high confidence) using a median split for each individual subject. Each of these regressors was modulated by a ∣DV∣ parametric regressor (defined above).

Statistical inference

Second level group contrasts from GLM 1 were calculated as one-sample t-tests against zero for each first-level linear contrast. Activations were reported as significant if they survived family wise error correction (FWE) for multiple comparisons across the whole brain at the cluster-level. For rRLPFC, we employed small volume correction using a 8-mm sphere centred on the coordinates [36, 44, 28] taken from ¹⁴. For GLM 2, rfxplot was used to extract percent signal change at each region of interest (³⁷: http://rfxplot.sourceforge.net/) defined by 6mm spheres around the vmPFC/rRLPFC peak voxels from GLM 1. These values were entered into 2×2 ANOVAs (factors value, confidence) to further clarify the pattern of activity seen in GLM 1. In analysis of individual differences we employed small-volume correction (SVC) using 8mm spheres centred on the peak activations in vmPFC and rRLPFC taken from GLM 1.

PPI analysis

To assess changes in connectivity between rRLPFC and vmPFC as a function of confidence we carried out a psychophysiological interaction (PPI) analysis. PPI is a measure of context-dependent connectivity, explaining the regional activity of other brain regions (here, vmPFC) in terms of the interaction between responses in a seed region (here, rRLPFC) and a cognitive or sensory process. We used the second GLM (factorial) to run our PPI analysis (for details see paragraph above). We carried out PPI analysis using the Generalised PPI toolbox for SPM (gPPI; http://www.nitrc.org/projects/gppi). gPPI creates a new GLM in which the deconvolved activity of the seed region is assigned to separate regressors dependent on the status of the original psychological variable (high or low confidence), and reconvolved with the haemodynamic response function. Average timecourses were extracted from all voxels within a 6 mm sphere surrounding the right rlPFC peak coordinate [39, 41, 16]. The main effects of high and low confidence, the seed region timecourse and motion parameters were included as regressors of no interest. The PPI contrast compares high_conf*rRLPFC (+1) with low_conf*rRLPFC (−1). This analysis showed a significant activation in vmPFC [9, 50, −11] that reflects the increased connectivity between vmPFC and rRLPFC during high compared to low confidence trials.