Skip to main content
NCBI home page
Human Brain Mapping logoLink to Human Brain Mapping
. 2010 Sep 30;32(10):1637–1648. doi: 10.1002/hbm.21136

The neural correlates of intertemporal decision‐making: Contributions of subjective value, stimulus type, and trait impulsivity

Chandra Sekhar Sripada 1,✉, Richard Gonzalez 2, K Luan Phan 1,3,4, Israel Liberzon 1,3,4
PMCID: PMC6870159  PMID: 20886577

Abstract

Making choices between payoffs available at different points in time reliably engages a decision‐making brain circuit that includes medial prefrontal cortex (mPFC), posterior cingulate cortex (PCC), and ventral striatum (VS). Previous neuroimaging studies produced differing accounts of the functions of these regions, including that these regions: (1) are sensitive to the value of rewards discounted by a function of delay ('subjective value'); (2) are differentially sensitive to the availability of an immediate reward; and (3) are implicated in impulsive decision‐making. In this event‐related fMRI study of 20 volunteers, these hypotheses were investigated simultaneously using a delay discounting task in which magnitude of rewards and stimulus type, i.e., the presence or absence of an immediate option, were independently varied, and in which participants' trait impulsivity was assessed with the Barratt Impulsiveness Scale. Results showed that mPFC, PCC, and VS are sensitive to the subjective value of rewards, whereas mPFC and PCC, but not VS, are sensitive to the presence of an immediate reward in the choice option. Moderation by individual differences in trait impulsivity was specific to the mPFC. Conjunction analysis showed significant overlap in mPFC and PCC for the main effects of subjective value and stimulus type, indicating these regions may serve multiple distinct roles during intertemporal decision‐making. These findings significantly advance our understanding of the specificity and overlap of functions subserved by different regions involved in intertemporal decision‐making, and help to reconcile conflicting accounts in the literature. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.

Keywords: fMRI, decision‐making, intertemporal choice, temporal discounting, reward, impulsivity, individual‐differences

INTRODUCTION

People routinely make decisions that involve trading off outcomes that occur at different points in time. Recent neuroimaging studies have sought to clarify the neural basis of intertemporal choice using the delay discounting task [Myerson and Green,1995]. In this task, participants are presented with a series of binary choices between smaller magnitude monetary rewards delivered with a smaller delay and larger magnitude monetary rewards delivered with a larger delay. These neuroimaging studies have converged on a common set of regions that are activated during the task, especially medial prefrontal cortex (mPFC), posterior cingulate cortex (PCC), and ventral regions of striatum (VS) [Ballard and Knutson,2009; Kable and Glimcher,2007; Luhmann et al.,2008; McClure et al.,2004,2007; Peters and Buchel,2009; Weber and Huettel,2008], though the role these regions play in intertemporal decision‐making is still disputed.

Economic and behavioral theories posit that humans value future rewards based not on their objective magnitude, but rather on their subjective value, which is derived from the objective magnitude discounted by some function of the delay imposed on receipt of the reward [Frederick et al.,2002; Samuelson,1937]. Recent neuroimaging studies suggest that the subjective value of available rewards contributes to activation in mPFC, PCC, and VS during intertemporal choice. Kable and Glimcher [2007] presented participants with a choice between an immediate reward fixed in magnitude and a delayed reward whose magnitude and delay were parametrically varied. They found precise fits between activation in mPFC, PCC, and VS and a regressor that models the subjective value of the later reward. However, in this design, the subjective value of the later reward was perfectly collinear with the difference of the subjective values of the two rewards, as well as the sum of both values. However, if a design is used in which the magnitude and delay of the earlier reward is also allowed to vary, which perhaps is more representative of the kinds of intertemporal choices people confront in real‐world settings, these functions of the subjective values of the two available rewards are no longer collinear and can be distinguished. This raises the intriguing question of which subjective value function is represented in the brain.

A second factor that may account for patterns of activation in mPFC, PCC, and VS during intertemporal choice is stimulus type, in particular the presence or absence of an option to receive an immediate reward. According to the ‘two systems model’ from McClure et al. [2007,2004], separate neural systems value rewards available at different points in time. One system located primarily in ventral and dorsal lateral prefrontal cortex and superior parietal regions values rewards available at all points in time irrespective of delay. A second valuation system, which includes mPFC, PCC, and VS, is specialized for representing the value of immediate rewards. Consistent with this hypothesis, they demonstrated [McClure et al.,2004,2007] that mPFC, PCC, and VS are significantly more active during choices in which an immediate option is available compared with choices where both options are delayed. However, controversy remains about whether enhanced activation in mPFC, PCC, and VS during trials with an immediate option arises only because immediate options are more subjectively valuable [Kable and Glimcher,2007; Kable and Glimcher,2010], or rather because these regions genuinely exhibit specificity for the availability of immediate rewards [McClure et al.,2004,2007].

A third factor that may influence brain activation during intertemporal decision‐making is subjects' level of trait impulsivity. Previous studies have demonstrated that one's ability to delay gratification is a relatively stable psychological trait, and this ability is predictive of a host of future positive life outcomes, including academic, psychological and interpersonal outcomes [Duckworth and Seligman,2005; Mischel et al.,1988; Tangney et al.,2004]. Behavioral studies have demonstrated that individuals with higher levels of trait impulsivity [Richards et al.,1999], or patient groups hypothesized to exhibit greater trait impulsivity [Kirby et al.,1999; Solanto et al.,2001; Vuchinich and Simpson,1998], tend to discount future outcomes more steeply. Previous fMRI studies of delay discounting tasks have investigated neural correlates of impulsivity during delay discounting using subjects' own choices during the fMRI task as the index of impulsiveness [Ballard and Knutson,2009; Luhmann et al.,2008]. However, the neural correlates of trait impulsivity during delay discounting, as indexed by a scale validated to measure this construct, have not yet been directly explored.

In this study, we investigated the contributions of subjective value, stimulus type (i.e., the presence of an option for an immediate reward), and trait impulsivity to patterns of brain activation during delay discounting. We utilized a delay discounting task in which the magnitude of rewards and the presence or absence of an immediate option are independently varied. In addition, we assessed all participants with the Barratt Impulsiveness Scale [Patton et al.,1995] and conducted an exploratory analysis of patterns of brain activation during delay discounting that are associated with stable individual differences in impulsivity.

METHODS

Participants

Twenty right‐handed males (all adults, mean age 28.7 ± 11.4 years) participated. All were healthy, not taking medications, and had no contraindications to participating in an fMRI experiment. Informed consent was obtained using a form approved by the Institutional Review Board of the University of Michigan Medical School.

Task

The protocol for our delay discounting task was adapted from McClure et al. [2004]. Participants were told that they were involved in a study that assesses how people decide between monetary rewards that are delivered with different amounts of delay. Task instructions were reviewed with each participant until clear understanding was voiced. Participants were also told that at the end of the scanning session, two trials would be picked at random and participants would be paid the amounts, with the delays, they had chosen in these trials in the form of Amazon.com gift certificates delivered by email. Participants were explicitly instructed that given this payment scheme, they should treat each trial as if it were one that would actually be paid, and that each trial is equally likely to be selected.

All participants received the same semirandom sequence of trials where this sequence was generated by specifying values for the four parameters listed below. Values for these parameters were drawn from the following sets.

  • Time of earlier reward = {today, 2 weeks, 4 weeks}

  • Time of later reward = time of earlier reward + {2 weeks, 4 weeks}

  • Magnitude of earlier reward = value randomly drawn from {Guassian distribution with mean $20 and s.d. $10}

  • Magnitude of later reward = magnitude of earlier reward + {1%, 10%, 25%, 50%} of magnitude of earlier reward

Each trial proceeded as follows (see Fig. 1). The two options were presented on the left and right sides of the screen with the earlier option always presented on the left. Two yellow triangles were situated below each dollar amount indicating that participants could make their choice by button box (mounted to their right hand). Participants were then allowed as much time as needed to make their selection. Once the response was made, the corresponding yellow triangle turned red for 2 s indicating that the response had been recorded. The screen then displayed a fixation cross for 12 s, followed by the next trial. Participants performed 80 trials over four runs.

Figure 1.

Figure 1

Open in a new tab

Schematic diagram of task. During fMRI scanning, 20 healthy volunteers made choices (n = 80) between smaller earlier rewards and larger later rewards. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

In the “Immediacy Present” condition (32 trials), one option was an immediate reward and the other a delayed reward. In the “Immediacy Absent” condition (48 trials), both options were delayed. Of note, subjects also completed an attention task during this scanner session that is not related to the topic of this report.

Neuroimaging

MRI scanning occurred on a General Electric (Waukesha, WI) 3T Signa scanner [LX (8.3) release, neurooptimized gradients]. Scanning began with structural acquisition of a standard T1 image (T1‐overlay) for anatomic normalization and alignment. We used a T2*‐weighted, reverse spiral acquisition sequence [GRE; repetition time, 2,000 ms; echo time, 30 ms; flip angle, 90°; field of view (FOV), 20 cm; 40 slice; thickness/skip, 3.0/0 mm matrix size equivalent to 64 × 64], which has been shown [Glover and Law,2001] to minimize signal drop‐out in regions such as ventral striatum and orbitofrontal cortex that are vulnerable to susceptibility artifact. After discarding four initial volumes to permit thermal equilibration of the MRI signal, 200 volumes were acquired per run. After acquisition of functional volumes, a high‐resolution T1 scan was obtained for anatomic normalization [three‐dimensional spoiled gradient‐recalled acquisition in a steady state (SPGR); 24 FOV; thickness/skip, 1.0/0 mm]. Stimuli were presented and responses recorded using a computer running E‐prime (Psychology Software Tools, Pittsburgh, PA), interfaced to project stimuli onto MR‐compatible liquid crystal display goggles (Resonance Technology, Northridge, CA). Choices were made using an MRI‐compatible button box.

Neuroimaging Analysis

Data from all 20 participants met criteria for high quality (uniformity and homogeneity of T2* images in regions such as ventral striatum and orbitofrontal cortex prone to susceptibility artifact) and scan stability (motion correction <2 mm displacement), and were subsequently included in the data processing. Preprocessing steps were implemented using Statistical Parametric Mapping 2 software (SPM2; Wellcome Department of Cognitive Neurology, London; http://www.fil.ion.ucl.ac.uk/spm). Preprocessing followed conventional procedures: (1) slice time correction; (2) spatial realignment; (3) normalization to the structural (T1‐weighted) Montreal Neurologic Institute (MNI) template through the use of a nonlinear warping algorithm followed by resampling by sync interpolation resulting in a voxel size of 3 × 3 × 3 mm3; (4) spatial smoothing through the use of a Gaussian 5 mm full‐with‐half‐maximum kernel; (5) high‐pass temporal filtering with a cut‐off of 128 s to remove low‐frequency drifts in signal. After preprocessing, statistical analyses were performed at the individual and group level using the general linear model (GLM) and Gaussian random field theory as implemented in SPM2.

For the individual level analyses, four regressors were entered into the GLM to model subjective value of rewards, stimulus type (i.e., Immediacy Present or Immediacy Absent), the interaction between subjective value and stimulus type, and reaction time. To model the effect of subjective value, we first constructed a number of different candidate subjective value regressors. To construct these regressors, we computed the subjective value of each reward present in each trial. We estimated discounting rates for each subject using that subject's own expressed choices during the 80 trials of this experiment using a generalized linear mixed model, with a logit link function and subject as a random effect factor. We assumed an exponential discounting function: SV = δtA, where δ is the discount rate, t is time, and A is the dollar amount of the reward. Each subject's discounting rate was then used to determine a subjective value for each reward in each trial. On the basis of previous literature and our own a priori hypotheses, we tested candidate subjective value regressors that encoded (1) the larger subjective value of the two available rewards; (2) the sum of the subjective values; and (3) the difference of the later minus the earlier subjective values. Each candidate subjective value regressor was tested separately as part of a four regressor model that also included stimulus type, stimulus type × subjective value interaction, and reaction time. In a separate analysis, we assumed a hyperbolic form for the discounting function [SV = A*(1/h+1)t]. Since the assumption of a hyperbolic discounting function produced subjective value estimates that were highly similar to the exponential discounting function (correlation between the exponential and hyperbolic difference of subjective value regressors was 0.992), we retained the subjective value regressors generated by the exponential function to maximize compatibility of our findings with a previous study using a similar paradigm [McClure et al.,2004] that also assumed an exponential form for the discounting function.

The interaction term in our GLM was derived by multiplying the mean‐centered stimulus type and subjective value regressors. To control for the possible influence of statistically significant differences in response time across different trial types, two strategies were used. First, based on prior studies that identify a four second window of decision‐related activation post‐stimulus presentation [Kable and Glimcher,2007], we used a four second trial duration that began when the binary choice option was first presented to the participants, and also included a regressor modeling reaction time in the first‐level GLM. The second strategy involved using a variable decision‐making interval defined for each trial that began when the binary choice option was first presented to the participant and terminated when the participant indicated his or her choice by push‐button. Both methods yielded nearly identical results and did not impact findings of the current study. Thus we report the results from the first method to maintain continuity with existing studies.

For all group analyses, subjects were treated as a random effect. Images were thresholded by using a voxel‐wise threshold of P < 0.005, combined with a cluster size threshold of 32 contiguous voxels (864 mm3). This combined threshold was estimated with a Monte Carlo simulation using AlphaSim (Douglas Ward, http://afni.nimh.nih.gov/pub/dist/doc/program_help/AlphaSim.html) to give an overall threshold of P < 0.05, corrected for multiple comparisons across the entire brain. Whole brain, voxel‐wise multiple regression analysis of group‐level maps, as implemented in SPM2, was used to identify brain regions whose activations correlated with scores on the Barratt Impulsivity Scale, and are displayed at the AlphaSim‐derived threshold mentioned previously. To identify brain regions that are sensitive both to the effects of subjective value as well as stimulus type, we conducted a conjunction analysis [Nichols et al.,2005]. Group‐level whole brain maps of regions sensitive to subjective value and stimulus type were thresholded at P < 0.005 and binarized. These maps were then summed to create a conjunction map reflecting regions of common activation in both maps, and all regions larger than five voxels (135 mm3) are displayed.

RESULTS

Behavioral Results

Behavioral results showed that participants chose the earlier of the two rewards (across all trial types) on 46.4% of trials, choosing the immediate reward in 48.3% of Immediacy Present trials and the smaller delay reward in 45.1% of Immediacy Absent trials—a difference that was not statistically significant. Subjects' choices were well characterized by an exponential discounting function. We use the fixed effect exponential discounting model (i.e., every subject has the same discounting parameter, which was estimated at 0.956) as the baseline model. The baseline model yielded an accuracy of 79% correct predictions across all subjects and items. The full model that allows heterogeneity in the subjects' discount parameter using a random effect model yielded an accuracy of 87% correct predictions. The pseudo R 2 for the improvement of the heterogeneous model error variance over the baseline model is 34%, which means that allowing for heterogeneity of discount parameters reduced the error variance by 34% (likelihood ratio test comparing the full and baseline models (X 2(1) = 577.9, P < 0.0001). The average rate of discounting was 0.9493, which means payoffs lost roughly 5% of their value per week of delay. Rates of discounting ranged from 0.8094 for our most impulsive subject to 0.9865 for our least impulsive subject. Barratt scores (mean: 57.00 ± 5.60) were correlated with choice of the earlier reward in both Immediacy Present trials (r = 0.43, P = 0.059) as well as Immediacy Absent trials (r = 0.56, P = 0.011). Barratt scores also correlated with subjects' exponential discounting rates (r = 0.54, P = 0.014). Mean response time for all trials was 3.07 s ± 0.60 s. Reaction time was significantly longer for Immediacy Absent trials (3.26 s ± 0.70 s) compared with Immediacy Present trials (2.77 s ± 0.51; paired T(19) = 6.84, P < 0.001), and we discuss the significance of this finding in the discussion section.

Neuroimaging Results

Main effect of subjective value

In whole brain voxel‐wise analysis, we tested associations between brain activation and the three candidate subjective value regressors (each regressor was tested in a separate whole brain analysis). Regressors encoding the larger subjective value of the two available rewards (mean, 18.75; range, 1.77–47.94) and the sum of the subjective values of two available rewards (mean, 34.89; range, 1.77–46.94) were not found to significantly correlate either positively or negatively with any brain regions. However, the regressor encoding the difference in the subjective values of the two available rewards (the “DiffSV” regressor; mean 0.65, range −18.09 to +11.22) positively correlated with activation in a number of regions including mPFC, PCC (and large regions of adjacent precuneus), VS, as well as dorsal striatum, amygdala, and medial temporal regions (Fig. 2, Table I). No brain regions were found to be negatively correlated with the DiffSV regressor.

Figure 2.

Figure 2

Open in a new tab

Main effect of subjective value. Activation in medial prefrontal cortex, posterior cingulate cortex, and ventral and dorsal striatum is significantly and selectively correlated with a regressor that encodes the difference in the subjective value of the later minus the earlier of two available rewards. All activations are displayed at whole brain voxel‐wise P corrected < 0.05. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Table I.

Activation results from whole brain voxel‐wise analysis

Contrast map and brain region Cluster size (mm3) MNI coordinates (x, y, z) Analysis z
Difference of subjective value of later minus earlier reward (‘Diffsv’)
 Precuneus/Posterior Cingulate 12,852 15, −48, 42 4.59
 Middle frontal gyrus 3,807 −24, 63, 24 4.44
 Lingual gyrus 1,539 0, −66, −9 4.34
 Hippocampus/Amygdala 4,293 12, −6, −18 4.21
 Cuneus 1,782 −6, −90, 30 4.16
 Medial orbital frontal gyrus 3,240 9, 30, −3 4.02
 Middle frontal gyrus 4,563 −24, 42, 39 3.93
 Hippocampus 5,238 −15, −39, −12 3.89
 Postcentral gyrus 1,728 42, −27, 57 3.86
 Putamen/Ventral striatum 1,053 −24, 9, 6 3.73
 Putamen/Ventral striatum 891 27, 15, 3 3.68
 Inferior parietal lobule 2,916 −60, −48, 42 3.63
 Angular gyrus 1,377 −42, −63, 42 3.59
 Cerebellum 1,323 33, −78, −21 3.56
 Caudate 918 −21, −12, 21 3.51
 Medial orbital frontal gyrus 1,728 −3, 54, −3 3.45
 Cerebellum 1,107 −18, −45, −24 3.45
 Middle temporal gyrus 3,186 60, −24, −3 3.34
 Angular gyrus 1,188 54, −60, 36 3.31
 Insula 999 −39, −6, 6 3.25
Choose Later > Choose Earlier
 Superior occipital gyrus 3,915 −12, −90, 18 4.68
 Thalamus 1,242 21, −12, 24 4.24
 Putamen 1,080 −18, 15, 6 4.2
 Parahippocampal gyrus/Hippocampus 972 −21, −27, −18 3.93
 Middle frontal gyrus 945 −42, 48, 12 3.92
 Middle cingulate gyrus 2,349 −9, −24, 45 3.87
 Lingual gyrus 1,728 −15, −48, 0 3.79
 Medial superior frontal gyrus 3,753 −12, 57, −3 3.79
 Cerebellum 1,728 −21, −45, −21 3.74
 Rolandic operculum 1,323 57, −18, 12 3.71
 Parahippocampal gyrus 864 18, −27, −18 3.68
 Superior temporal lobe 1,080 60, −24, 0 3.58
 Cuneus 1,026 24, −75, 21 3.58
 Angular gyrus 945 −48, −69, 51 3.52
 Cuneus 891 −3, −72, 30 3.35
Immediacy Present > Immediacy Absent
 Medial superior frontal gyrus 5,265 −9, 66, 12 5.14
 Angular gyrus 3,618 −57, −69, 42 4.6
 Medial superior frontal gyrus 1,458 −6, 57, 33 4.21
 Superior frontal gyrus 1,593 −9, 36, 51 4.16
 Middle cingulate gyrus 1,377 −9, −45, 36 3.92
 Precuneus/Posterior cingulate 2,214 −6, −54, 12 3.9
Immediacy Absent > Immediacy Present
 Middle occipital gyrus/Superior parietal lobule 9,720 −27, −75, 39 5.74
 Precental gyrus 2,511 60, 15, 24 4.49
 Calcarine cortex 4,401 18, −105, −3 4.37
 Superior parietal lobule 8,559 33, −75, 33 4.12
 Supplementary motor area 3,888 6, 9, 54 4.12
 Middle frontal gyrus (Dorsal lateral prefrontal cortex) 945 39, 45, 36 3.92
 Middle frontal gyrus 1,755 30, −3, 69 3.67
Immediacy Present > Immediacy Absent, Inverse Correlation With Barratt Scores
 Middle occipital gyrus/Cuneus 1,890 30, −75, 21 4.01
 Parahippocampal gyrus 1,350 27, −39, −9 4.24
 Superior medial frontal gyrus (Anterior mPFC) 1,242 6, 54, 15 3.34
Conjunction analyses: (Immediacy present > Immediacy absent) AND (DiffSV)
 Medial superior frontal gyrus 891 −6, 54, −12
 Precuneus 351 −6, −66, 15
 Posterior cingulate 351 −9, −39, 30
(Immediacy Present > Immediacy Absent) and (Regions positively correlated with RT in Immediacy Present trials)
 Supplementary motor area 2,457 3, 15, 51
 Superior parietal lobule 756 24, −63, 51
(Immediacy present > Immediacy absent) and (Regions positively correlated with RT in immediacy absent trials)
 Supplementary motor area 2,754 0, 12, 51
 Superior parietal lobule 1,323 30, −60, 51
 Superior parietal lobule 648 −27, −69, 48
Open in a new tab

Of note, as would be expected, DiffSV is highly positively correlated with subjects' choices for delayed rewards because when the difference in subjective value between the later versus earlier reward is large, subjects are more likely to choose the later reward. We performed an additional analysis in which a regressor encoding subjects' choices was entered into the GLM in place of the DiffSV regressor. In whole brain search for regions more active during choices for later rewards compared to earlier rewards, we observed activation in mPFC, PCC, and VS, as well as thalamus and medial temporal regions (Table I). For the purposes of the remaining analyses, we retained the DiffSV regressor in our General Linear Model, and we discuss the fact that it is not possible in the current study to dissociate contributions of DiffSV from contributions of subjects' choices in the discussion section.

Main effect of stimulus type

We next conducted a whole brain voxel‐wise search for brain regions sensitive to the main effect of stimulus type. We observed greater activation in Immediacy Present trials versus Immediacy Absent trials in mPFC, PCC/mid‐cingulate cortex, precuneus/retrosplenial cortex, and left inferior parietal cortex (Fig. 3A, Table I). In addition, activations were observed in regions of middle temporal gyrus and left superior frontal gyrus (Table I). We observed greater activation in Immediacy Absent trials compared to Immediacy Present trials in a diverse network including bilateral superior parietal cortex, right dorsolateral prefrontal cortex, and supplementary motor cortex extending to the border of dorsal anterior cingulate cortex (Fig. 3B, Table I). Overall, these results are consistent with two previous studies by McClure and colleagues [McClure et al.,2004,2007], who found similar regions activated in contrasts that are comparable to those employed in this study.

Figure 3.

Figure 3

Open in a new tab

Main effect of stimulus type. A: Enhanced activation in medial prefrontal cortex, posterior/mid‐cingulate cortex, precuneus/retrosplenial cortex in trials in which one option is immediate compared to trials in which both options are delayed. B: In the opposite contrast, a distinct network of brain regions including right dorsal lateral prefrontal cortex, supplementary motor cortex, and right superior parietal cortex is observed. All activations are displayed at whole brain voxel‐wise P corrected < 0.05. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Conjunction analyses

The group‐level whole brain maps derived from the previous analyses provided evidence that a number of regions including mPFC and PCC are sensitive to subjective value as well as stimulus type (i.e., Immediacy Present > Immediacy Absent condition). We performed a conjunction analysis of these group‐level whole brain maps to identify brain regions that are sensitive to both of these effects. A conjunction analysis of the map of regions correlated with the DiffSV regressor and the map of regions more active in Immediacy Present trials (>Immediacy Absent trials) revealed common areas of activation in mPFC, PCC, and precuneus (Fig. 4, Table I). The analogous conjunction analysis for Immediacy Absent trials (>Immediacy Present trials) revealed no areas of overlap.

Figure 4.

Figure 4

Open in a new tab

Conjunction analysis of effects of subjective value and stimulus type. Whole brain group‐level maps of regions responsive to the main effect of subjective value (i.e., activation is correlated with the difference in subjective value of the two rewards) and stimulus type (i.e., greater activation for trials with an immediate reward available) were generated from a general linear model that modeled the effects of subjective value, stimulus type, and their interaction. A conjunction analysis of these whole brain maps revealed that medial prefrontal cortex, posterior cingulate cortex, and precuneus, regions reliably activated in prior fMRI studies of delay discounting (see Introduction), are independently sensitive to both subjective value and stimulus type. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Given the significantly longer mean reaction times in the Immediacy Absent versus Immediacy Present condition, we performed additional conjunction analyses to identify whether reaction time might contribute to activation differences in the Immediacy Present (>Immediacy Absent) and Immediacy Absent (>Immediacy Present) conditions. We generated maps of regions whose activation is correlated with reaction time for Immediacy Present trials and Immediacy Absent trials separately. Conjunction analysis revealed no overlap between regions observed in the contrast of Immediacy Present (>Immediacy Absent) trials and regions correlated either positively or negatively with reaction time. However, we did observe overlap between regions more active in Immediacy Absent (>Immediacy Present) trials and regions positively correlated with reaction time in supplementary motor area and bilateral superior parietal cortex (Table I).

Stimulus type × DiffSV interaction

The Stimulus type × DiffSV interaction term identifies brain regions whose activation is significantly more correlated with the DiffSV regressor in one stimulus condition versus the other. Whole brain voxel‐wise search revealed no brain regions that exhibited a significant positive or negative interaction.

Individual‐differences in trait impulsivity

In whole brain, voxel‐wise multiple regression analysis, we correlated participants' Barratt scores (higher scores = greater impulsivity) with activation maps derived from the main effects identified above. In the map of brain regions sensitive to the availability of immediate rewards (Immediacy Present > Immediacy Absent), we observed a significant cluster in anterior mPFC for which activation was inversely correlated with Barratt scores (MNI coordinates: [6, 54, 15]; 1242 mm3) (Fig. 5; Table I), indicating that more impulsive individuals had less activation in this region in trials with an immediate option present compared to trials in which an immediate option is absent. Of note, no brain regions were positively correlated with Barratt scores.

Figure 5.

Figure 5

Open in a new tab

Anterior medial prefrontal cortex and Barratt scores. A: In the whole brain map of regions sensitive to the main effect of stimulus type (i.e., greater activation for trials in which one option is immediate versus trials in which both options are delayed), voxel‐wise multiple regression analysis revealed activation in anterior medial prefrontal cortex is inversely correlated with Barratt scores. All activations are displayed at whole brain voxel‐wise P corrected < 0.05. B: Scatter plot showing distribution of Barratt scores and BOLD responses in this anterior medial prefrontal cortex region. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

DISCUSSION

In this study, we examined contributions of subjective value, stimulus type, and trait impulsivity to brain activity during a delay discounting task in which participants made choices between smaller, earlier rewards and larger, later rewards. Our primary result is that (1) mPFC, PCC, and VS, three regions reliably activated in prior studies of delay discounting, are sensitive to the subjective value of rewards (in particular, the difference in subjective value of later minus earlier reward); and (2) mPFC and PCC are sensitive to the presence of an immediate reward in the choice option. Individual‐differences in trait impulsivity (as expressed in Barratt scores) moderated the reactivity of mPFC during the contrast of trials in which one option is immediate against trials in which both options are delayed. These findings substantially nuance our understanding of the function of brain regions involved in intertemporal choice and suggest intriguing hypotheses about the brain basis of trait impulsivity.

Activation in mPFC, PCC, and VS were found to robustly correlate with the difference in subjective value of the later minus earlier reward (“DiffSV”), but not to the sum or the larger subjective value of the two available rewards. A prior fMRI study of delay discounting by Kable and Glimcher [2007] found evidence for representations of subjective value of rewards in these same regions, but, due to differences in design, they could not distinguish between the hypothesis that these regions represent the subjective value of the later of the two rewards, their sum, or their difference. Our results suggest it is the difference in subjective value of the later minus earlier reward that is represented, rather than these alternatives. One limitation of this study is that since DiffSV will naturally be highly collinear with choice for the later reward, we could not rule out the competing hypothesis that mPFC, PCC, and VS activation is driven by making a choice for the later reward itself. Of note, other studies that have used designs better suited for disentangling subjective value from choice [Chib et al.,2009; Hare et al.,2009; Montague et al.,2006; O'Doherty et al.,2006], have found activation in mPFC and VS is associated with subjective value of rewards, providing some evidence in favor of the view that these regions are indeed involved in representation of subjective value rather than the choice itself.

We found trials in which an immediate option was available, compared to trials in which both options were delayed, activated multiple regions including mPFC, PCC, precuneus, and medial temporal regions, and several hypotheses are available to explain this enhanced activation. One hypothesis explains this enhanced activation in terms of the greater subjective value of immediate rewards, i.e., ‘the special preference for immediacy’, which behavioral studies have reliably detected [Frederick et al.,2002; Kirby and Herrnstein,1995]. However, arguing against this hypothesis, we did not find behavioral evidence for a special preference for immediacy in this study. Subjects tended to choose the immediate reward in Immediacy Present trials and the earlier reward in Immediacy Absent trials with roughly equal frequency. Furthermore, subjective values (in particular DiffSV values) calculated by assuming an exponential versus hyperbolic form for the discounting function (where the latter, but not the former, can capture the special preference for immediacy) were nearly identical (r = 0.992), showing again that subjects tended not to overvalue immediate options. The absence of an immediacy effect in this study may arise from a combination of factors including the relatively low dollar amounts and narrow range of delays used, as well as the use of Amazon.com gift certificates for payment whose utilization is associated with inherent delays. An alternative hypothesis that might explain the enhanced activation of multiple brain regions during Immediacy Present (>Immediacy Absent) trials involves cognitive load. Previous studies have found a network of midline and lateral parietal regions, including mPFC and PCC, are reliably more active during rest periods between blocks of cognitively demanding tasks, and also more active in less cognitively demanding tasks contrasted with more demanding tasks [Mazoyer et al.,2001; McKiernan et al.,2003; Raichle et al.,2001]. Immediacy Present trials may be less cognitively demanding than Immediacy Absent trials as evidenced by the fact that reaction time, often used as an indirect measure of cognitive load, is significantly shorter in Immediacy Present trials. However, we also found that regions more active in Immediacy Present (>Immediacy Absent) trials did not overlap with regions whose activation is inversely correlated with reaction time, providing some evidence against the cognitive load hypothesis. Other explanations for enhanced activation in mPFC and PCC during Immediacy Present trials are also available, and we discuss these in the context of McClure and colleagues' two systems model below.

Whole brain conjunction analysis revealed there was significant overlap between the regions that were sensitive to the availability of immediate rewards and the regions found to represent DiffSV (see Fig. 4) in mPFC, PCC, and precuneus. The fact that some of the same regions in mPFC and PCC are independently sensitive to both subjective value and stimulus type sheds new light on an important controversy in the recent literature. McClure and colleagues have proposed a ‘two systems’ model [McClure et al.,2004,2007; Sanfey et al.,2006] in which mPFC, PCC, and VS constitute a valuation system specialized for representing the subjective value of immediate rewards. In contrast, Kable and Glimcher [2007] have proposed a ‘one system’ model in which mPFC, PCC, and VS represent the subjective value of rewards at all points in time. Consistent with Kable and Glimcher, our data support the view that mPFC, PCC, and VS represent the subjective value of rewards at all points in time (in particular they represent the difference in subjective value of the later minus earlier reward). But in agreement with McClure and colleagues, we found two of these regions, in particular mPFC and PCC, are also sensitive to the presence of immediate rewards.

It is useful to discuss our results in light of McClure and colleagues' two system model in greater detail. McClure et al. hypothesize that mPFC, PCC, and VS are part of a valuation system specialized for responding to immediate rewards and that biases choice towards the immediate option. In contrast, we found regions that are more active when an immediate reward is available did not predict choices for more immediate rewards. Rather, consistent with the role of mPFC and PCC in representing DiffSV, we found these regions are more active during choices for more delayed rewards, which is in agreement with other studies that also report higher activation in these regions predicts choices for later rewards [Kable and Glimcher,2007; Peters and Buchel,2009; Weber and Huettel,2008]. The two systems model also holds that a second valuation system, which includes dorsal lateral prefrontal cortex and superior parietal cortex, represents rewards at all points in time and biases choices for more delayed rewards. McClure and colleagues propose that these regions implement high‐level executive functions including deliberative reasoning [Krawczyk,2002; Smith and Jonides,1999], and numerical computation [Dehaene et al.,1998; Fias et al.,2003], which are needed to represent rewards available at differing points in time, as they are more abstract and intangible than immediate rewards [McClure et al.,2004; Sanfey et al.,2006]. However, we found dorsal lateral prefrontal cortex and superior parietal cortex are not equally active across all trial types (as McClure et al. propose) but rather are significantly more active during trials in which both rewards are delayed compared to trials in which one reward is immediate. A third region, supplementary motor cortex, which has been implicated in prior studies of decision‐making [Ridderinkhof et al.,2004], especially with uncertain choices involving probability and risk [Volz et al.,2003,2004], was also more active in Immediacy Absent (>Immediacy Present) trials. Moreover activation in superior parietal cortex and supplementary motor cortex was found to significantly overlap with regions whose activation positively correlated with reaction time, providing additional evidence that these regions are involved in high cognitive load processing. Our finding of enhanced activation in multiple executive processing regions in Immediacy Absent trials (>Immediacy Present trials) is not necessarily inconsistent with McClure et al.'s two systems model, and may be perhaps explained if representing two abstract, intangible delayed rewards places greater demands on executive and numerical processing than representing just one delayed reward. However, we did not find that any of the regions that were more active during Immediacy Absent (>Immediacy Present) trials predicted choices for more delayed rewards (Table I). Thus while our results support McClure et al.'s view that regions that support executive processing and numerical cognition are engaged during intertemporal choice, we did not find evidence that these regions constitute a valuation system that biases choices for more delayed rewards.

In whole brain multiple regression analysis, we found that activation in mPFC was inversely correlated with Barratt scores, a measure of trait impulsivity, in the contrast of Immediacy Present trials and Immediacy Absent trials. Another recent study of delay discounting [Luhmann et al.,2008] found a similar inverse correlation between a measure of participants' preference for immediacy (derived from participants' choice‐behavior during the decision‐making task) and activity in a similar anterior mPFC region. These findings of anterior mPFC hypoactivation in more impulsive individuals are intriguing in light of a number of recent proposals that anterior mPFC serves as an executive/regulatory region that plays a high‐level role in coordinating and integrating competing cognitive operations [Buckner and Carroll,2007; Burgess et al.,2007a,b; Gallagher and Frith,2003; Gilbert et al.,2006; Ramnani and Owen,2004]. Consistent with this view, other decision‐making studies have found activation in anterior mPFC during conflicted choice. For example, Moll et al. [2006] studied a social donation game that pits personal selfish interest against social benefits, and found anterior mPFC activation specifically in trials in which social benefit was chosen over selfish interest (see also de Quervain et al. [2004] for a similar finding). More recently, anterior mPFC activation has been detected in a novel decision‐making study in which opportunities for immediate payoffs compete with chances to secure larger later payoffs [Diekhof and Gruber,2010]. Of note, previous studies have linked impulsivity with alterations in VS reward‐related responsivity. For example, VS activity during a monetary rewards task has been found to correlate with temporal discounting rates [Hariri et al.,2006] as well as Barratt Impulsivity Scale scores [Forbes et al.,2009]. A key difference between these studies and the present one is that we measured brain activity during the decision phase of a delay discounting task in which feedback was absent, while these previous studies measured brain activity collapsing across the decision and feedback phases of a guessing game task in which subjects received stochastic positive and negative feedback. Thus this study highlights a link between anterior mPFC and trait impulsivity specifically during reward‐related decision‐making, consistent with a number of prior studies that activate anterior mPFC during decision‐making, and especially during conflicted choice [de Quervain et al.,2004; Diekhof and Gruber,2010; Moll et al.,2006]. Future studies should directly investigate whether trait impulsivity is linked to alterations in distinct brain regions (anterior mPFC versus VS) during distinct phases (decision‐phase versus outcome‐phase) of reward‐related tasks.

In sum, we examined contributions of subjective value, stimulus type, and trait impulsivity to brain activity with fMRI and a delay discounting task. We found that mPFC, PCC, and VS, three regions reliably activated in prior studies of delay discounting, are sensitive to subjective value, mPFC and PCC are sensitive to stimulus type (i.e., the presence of an immediate option), and that trait impulsivity moderated activity in mPFC. These findings significantly advance our understanding of the specificity and overlap of functions subserved by different regions involved in intertemporal decision‐making, and raise new hypotheses about the brain basis of impulsivity.

REFERENCES

  1. Ballard K, Knutson B ( 2009): Dissociable neural representations of future reward magnitude and delay during temporal discounting. Neuroimage 45: 143–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Buckner RL, Carroll DC ( 2007): Self‐projection and the brain. Trends Cogn Sci 11: 49–57. [DOI] [PubMed] [Google Scholar]
  3. Burgess PW, Dumontheil I, Gilbert SJ ( 2007a): The gateway hypothesis of rostral prefrontal cortex (area 10) function. Trends Cogn Sci 11: 290–298. [DOI] [PubMed] [Google Scholar]
  4. Burgess PW, Gilbert SJ, Dumontheil I ( 2007b): Function and localization within rostral prefrontal cortex (area 10). Philos Trans R Soc Lond B Biol Sci 362: 887–899. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chib VS, Rangel A, Shimojo S, O'doherty JP ( 2009): Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. J Neurosci 29: 12315–12320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. de Quervain DJ, Fischbacher U, Treyer V, Schellhammer M, Schnyder U, Buck A, Fehr E ( 2004): The neural basis of altruistic punishment. Science 305: 1254–1258. [DOI] [PubMed] [Google Scholar]
  7. Dehaene S, Dehaene‐Lambertz G, Cohen L ( 1998): Abstract representations of numbers in the animal and human brain. Trends Neurosci 21: 355–361. [DOI] [PubMed] [Google Scholar]
  8. Diekhof EK, Gruber O ( 2010): When desire collides with reason: Functional interactions between anteroventral prefrontal cortex and nucleus accumbens underlie the human ability to resist impulsive desires. J Neurosci 30: 1488–1493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Duckworth AL, Seligman ME ( 2005): Self‐discipline outdoes IQ in predicting academic performance of adolescents. Psychol Sci 16: 939–944. [DOI] [PubMed] [Google Scholar]
  10. Fias W, Lammertyn J, Reynvoet B, Dupont P, Orban GA ( 2003): Parietal representation of symbolic and nonsymbolic magnitude. J Cogn Neurosci 15: 47–56. [DOI] [PubMed] [Google Scholar]
  11. Forbes EE, Brown SM, Kimak M, Ferrell RE, Manuck SB, Hariri AR ( 2009): Genetic variation in components of dopamine neurotransmission impacts ventral striatal reactivity associated with impulsivity. Mol Psychiatry 14: 60–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Frederick S, Loewenstein G, O'Donoghue T ( 2002): Time discounting and time preference: A critical review. J Econ Lit 15: 351–401. [Google Scholar]
  13. Gallagher HL, Frith CD ( 2003): Functional imaging of ‘theory of mind’. Trends Cogn Sci 7: 77–83. [DOI] [PubMed] [Google Scholar]
  14. Gilbert SJ, Spengler S, Simons JS, Steele JD, Lawrie SM, Frith CD, Burgess PW ( 2006): Functional specialization within rostral prefrontal cortex (area 10): A meta‐analysis. J Cogn Neurosci 18: 932–948. [DOI] [PubMed] [Google Scholar]
  15. Glover GH, Law CS ( 2001): Spiral‐in/out BOLD fMRI for increased SNR and reduced susceptibility artifacts. Magn Reson Med 46: 515–522. [DOI] [PubMed] [Google Scholar]
  16. Hare TA, Camerer CF, Rangel A ( 2009): Self‐control in decision‐making involves modulation of the vmPFC valuation system. Science 324: 646–648. [DOI] [PubMed] [Google Scholar]
  17. Hariri AR, Brown SM, Williamson DE, Flory JD, de Wit H, Manuck SB ( 2006): Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J Neurosci 26: 13213–13217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kable JW, Glimcher PW ( 2007): The neural correlates of subjective value during intertemporal choice. Nat Neurosci 10: 1625–1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kable JW, Glimcher PW ( 2010): An “as soon as possible” effect in human inter‐temporal decision making: behavioral evidence and neural mechanisms. J Neurophysiol 103(5):2513–2531. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kirby KN, Herrnstein RJ ( 1995): Preference reversals due to myopic discounting of delayed reward. Psychological Sci 6: 83–89. [Google Scholar]
  21. Kirby KN, Petry NM, Bickel WK ( 1999): Heroin addicts have higher discount rates for delayed rewards than non‐drug‐using controls. J Exp Psychol Gen 128: 78–87. [DOI] [PubMed] [Google Scholar]
  22. Krawczyk DC ( 2002): Contributions of the prefrontal cortex to the neural basis of human decision making. Neurosci Biobehav Rev 26: 631–664. [DOI] [PubMed] [Google Scholar]
  23. Luhmann CC, Chun MM, Yi DJ, Lee D, Wang XJ ( 2008): Neural dissociation of delay and uncertainty in intertemporal choice. J Neurosci 28: 14459–14466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Mazoyer B, Zago L, Mellet E, Bricogne S, Etard O, Houde O, Crivello F, Joliot M, Petit L, Tzourio‐Mazoyer N ( 2001): Cortical networks for working memory and executive functions sustain the conscious resting state in man. Brain Res Bull 54: 287–298. [DOI] [PubMed] [Google Scholar]
  25. McClure SM, Ericson KM, Laibson DI, Loewenstein G, Cohen JD ( 2007): Time discounting for primary rewards. J Neurosci 27: 5796–5804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. McClure SM, Laibson DI, Loewenstein G, Cohen JD ( 2004): Separate neural systems value immediate and delayed monetary rewards. Science 306: 503–507. [DOI] [PubMed] [Google Scholar]
  27. McKiernan KA, Kaufman JN, Kucera‐Thompson J, Binder JR ( 2003): A parametric manipulation of factors affecting task‐induced deactivation in functional neuroimaging. J Cogn Neurosci 15: 394–408. [DOI] [PubMed] [Google Scholar]
  28. Mischel W, Shoda Y, Peake PK ( 1988): The nature of adolescent competencies predicted by preschool delay of gratification. J Pers Soc Psychol 54: 687–696. [DOI] [PubMed] [Google Scholar]
  29. Moll J, Krueger F, Zahn R, Pardini M, de Oliveira‐Souza R, Grafman J ( 2006): Human fronto‐mesolimbic networks guide decisions about charitable donation. Proc Natl Acad Sci USA 103: 15623–15628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Montague P, King‐Casas B, Cohen J ( 2006): Imaging valuation models in human choice. Annu Rev Neurosci 29:417–448. [DOI] [PubMed] [Google Scholar]
  31. Myerson J, Green L ( 1995): Discounting of delayed rewards: Models of individual choice. J Exp Anal Behav 64: 263–276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Nichols T, Brett M, Andersson J, Wager T, Poline JB ( 2005): Valid conjunction inference with the minimum statistic. Neuroimage 25: 653–660. [DOI] [PubMed] [Google Scholar]
  33. O'Doherty J, Buchanan T, Seymour B, Dolan R ( 2006): Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron 49: 157–166. [DOI] [PubMed] [Google Scholar]
  34. Patton JH, Stanford MS, Barratt ES ( 1995): Factor structure of the Barratt impulsiveness scale. J Clin Psychol 51: 768–774. [DOI] [PubMed] [Google Scholar]
  35. Peters J, Buchel C ( 2009): Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. J Neurosci 29: 15727–15734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL ( 2001): A default mode of brain function. Proc Natl Acad Sci USA 98: 676–682. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Ramnani N, Owen AM ( 2004): Anterior prefrontal cortex: Insights into function from anatomy and neuroimaging. Nat Rev Neurosci 5: 184–194. [DOI] [PubMed] [Google Scholar]
  38. Richards JB, Zhang L, Mitchell SH, de Wit H ( 1999): Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. J Exp Anal Behav 71: 121–143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S ( 2004): The role of the medial frontal cortex in cognitive control. Science 306: 443–447. [DOI] [PubMed] [Google Scholar]
  40. Samuelson PA ( 1937): A note on the measurement of utility. Rev Econ Studies 4: 155–161. [Google Scholar]
  41. Sanfey AG, Loewenstein G, McClure SM, Cohen JD ( 2006): Neuroeconomics: Cross‐currents in research on decision‐making. Trends Cogn Sci 10: 108–116. [DOI] [PubMed] [Google Scholar]
  42. Smith EE, Jonides J ( 1999): Storage and executive processes in the frontal lobes. Science 283: 1657–1661. [DOI] [PubMed] [Google Scholar]
  43. Solanto MV, Abikoff H, Sonuga‐Barke E, Schachar R, Logan GD, Wigal T, Hechtman L, Hinshaw S, Turkel E ( 2001): The ecological validity of delay aversion and response inhibition as measures of impulsivity in AD/HD: A supplement to the NIMH multimodal treatment study of AD/HD. J Abnorm Child Psychol 29: 215–228. [DOI] [PubMed] [Google Scholar]
  44. Tangney JP, Baumeister RF, Boone AL ( 2004): High self‐control predicts good adjustment, less pathology, better grades, and interpersonal success. J Pers 72: 271–324. [DOI] [PubMed] [Google Scholar]
  45. Volz KG, Schubotz RI, von Cramon DY ( 2003): Predicting events of varying probability: Uncertainty investigated by fMRI. Neuroimage 19( Part 1): 271–280. [DOI] [PubMed] [Google Scholar]
  46. Volz KG, Schubotz RI, von Cramon DY ( 2004): Why am I unsure? Internal and external attributions of uncertainty dissociated by fMRI. Neuroimage 21: 848–857. [DOI] [PubMed] [Google Scholar]
  47. Vuchinich RE, Simpson CA ( 1998): Hyperbolic temporal discounting in social drinkers and problem drinkers. Exp Clin Psychopharmacol 6: 292–305. [DOI] [PubMed] [Google Scholar]
  48. Weber BJ, Huettel SA ( 2008): The neural substrates of probabilistic and intertemporal decision making. Brain Res 1234: 104–115. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Human Brain Mapping are provided here courtesy of Wiley

RESOURCES